Overview of course policies; meet and greet; presentation of schedule of topics with examples.

Syllabus

Video recording of first class

Representative survey of Audio Processing:

CMRA

Overview of physical properties of sound waves: frequency, amplitude, phase, intensity, loudness.

Measuring loudness: logarithmic scales and decibels.

Basics of digital signals and sampling; quantization error, aliasing, and choice of bit depth and sample rate; dynamic range; the Nyquist Theorem.

Introductory reading on waves and digital signals: PDF

Read sections 4.1 - 4.5 thoroughly.

Sorry for the bad job with the copying!

Tutorial (with animations) of sound waves: http://www.acs.psu.edu/drussell/demos.html

Look at these animations: 

• What is a Wave;
• Wave Motion in Space and Time;
• Superposition of Two Waves;
• Phase changes upon reflection from hard, soft, and mixed boundaries;
• Reflections from Impedance and the Standing Wave Ratio.

Lecture One: PDF

Fun video by Vi Hart on basics of sound: HTML

Cool video of visualizing sound waves: HTML

Useful summary of loudness and amplitude issues (and stereo panning): HTML

Slides from talk surveying amplitude/intensity of sound waves (a bit too much detail for us, but here if you are interested): PDF

Sinusoids created from sum of pure sine waves; time domain vs frequency domain representations.

Introduction to musical acoustics: How do guitar strings vibrate?

Basics of musical signals: Fundamental frequency, harmonic series, partials, timbre.

Musical acoustics concluded: open vs closed pipes; vibrating soundboards; the vocal tract.

Excellent reading on spectra as they relate to musical instruments: HTML

Then take a look at the following two Wiki articles, which are excellent:

Harmonic Series

Timbre

Acoustics of Strings: HTML (read first part, stop when you see "Harmonics in Music")

Acoustics of Open and Closed Pipes: HTML (read whole thing!)

Acoustics of Vocal Tract: HTML (optional)

Lecture Two: PDF

Beautiful Wikibook on some (not all) important topics in Sound Synthesis: HTML

Interactive applet showing how two sine waves add together--with sound! HTML

Limited but beautiful animation of Fourier Synthesis using real phasors: HTML

Phasor addition animation: HTML

Nice description of how standing waves in pipes actually work, with animation: HTML

Similar animation for traveling waves: HTML

Interesting talks at this workshop on musical timbre: HTML

Additive synthesis:

Additive Synthesis in Python

Introduction to HW 01

Very complete technical chapter on audio synthesis (from Musimathics): PDF

Wikibook page on Additive Synthesis: HTML

CRMA on additive synthesis: HTML

Short description of ASDR envelopes: HTML

Wikipedia article on Tremolo: HTML

Remind yourself of the definition of exponential decay by reading the Wiki article (HTML) but stop when you get to "solution of the differential equation."

Intro To Audio Programming: ZIP

HW 01: ZIP

HW 01 Solution: IPYNB

A good summary of why and how Python makes a good language for audio programming: PDF

Download site for Conda (recommended for installing packages for Jupyter Notebooks): HTML

Conda cheatsheet: PDF

Modulation synthesis: amplitude, ring, and frequency modulation.

Wikibook page on Amplitude and Ring modulation: HTML

CMRA page on Frequency Modulation (with applet and sound samples): HTML

John Chowning's original paper on FM:: HTML

Notebook with FM Code: IPYNB

Animation with sound for amplitude modulation: HTML

Conclusions on Ring and Frequency Synthesis: spectra and phase issues.

Physical Modelling Synthesis; Karplus-Strong String Synthesis

CRMA on Physical Modeling and Karplus-Strong: HTML

Notebook with KS Code: IPYNB

Very complete summary/taxonomy of music synthesis techniques: HTML

WaveNet for speech synthesis: HTML

Here is the first paper on Karplus-Strong: PDF

Here is a more extensive examination of the Karplus-Strong Algorithm and its extensions: PDF

Nice blog post with code: HTML

Conclusions on Physical Modelling: Spatialization and Reverberation; Filters and Convolution

Excellent summary of Sound Spacialization and Reverberation: PDF

Wikipedia on Moving Average Filters: HTML

Wikipedia article on convolution: HTML (just try to understand the basic definition, but do look at the animations down the page).

Here is a short tutorial on software for convolution reverb (with audio examples): HTML

Lecture 6: PDF

Notebook with Convolution and Reverb Code: IPYNB

Here is a brief tutorial on convolution with examples in 1D and 2D: HTML; skip the stuff on impulse response and linear, time-invariant systems and just concentrate on the examples as a way of understanding the algorithm

Here is a nice interactive animation of convolution in the continuous case: HTML

Informative article about collecting impulse responses for convolution reverb: HTML

Collection of impulse responses from around the USA: HTML

Nice tutorial on Panning Laws: HTML

Introduction to Digital Signal Processing; sliding-window-based analysis of musical signals.

What happens when we multiply (or square) sine waves? Correlation and auto-correlation.

Algorithms for determining fundamental frequency: Zero-crossing rate, correlation, and auto-correlation

Nice explanation of the basic formula for multipling sine waves: HTML

Correlation

Auto-Correlation, especially the section on Signal Processing, and especially Formula 7 here, with the understanding that we are dealing with real signals, so y = y bar -- so you can ignore the bar in the formula. Essentially, convolution, correlation and auto-correlation are variations of the same idea, and all are just another use of the dot product.

Lecture 7: PDF

Lecture 7 Notebook: IPYNB

Determination of fundamental frequency using auto-correlation with "peak picking,"  using interpolation to improve the accuracy; upsampling; other methods (zero-crossing, histograms); general problems  (which peak is F0?). 

Here is a short survey of F0 determination algorithms: HTML

More extensive survey: PDF

Lecture 8 Notebook: IPYNB

The Discrete Sin  Transform; Using complex numbers to bundle together frequency, amplitude, and phase; The Discrete Fourier Transform and the Fast Fourier Transform

Very general overview of applications of the FT in digital music: HTML

Reasonably simple explanation of the basic concepts, with fun animations: HTML

Here is a short, intuitive explanation of the Discrete Fourier Transform in the complex-number case: HTML

Fun animation video on YT: HTML

Interesting summary of FFT techniques, including sliding window analysis, applied to bird songs: PDF

Issues with the FFT: Resolution in time and frequency, "leakage" and "spray," Hann Windows.

Intro To Audio Programming: ZIP

HW 02 (Due 11/24): ZIP

FFT concluded: Interpreting and using spectra; Varieties of spectrograms; The Inverse Fourier Transform,

Here is a nice summary of the FFT: HTML

Lecture 11 Notebook: IPYNB

Here is a fun video introducing the topic of wavetable synthesis on a hardware synthesizer: HTML

The Wikipedia article is also worth reading: HTML

Lecture 12 Notebook: IPYNB

Changing time (and therefore pitch): 1D interpolation and resampling;

Unlocking pitch and time: The phase vocoder

The wiki articles are useful: HTML1, HTML2

Here is a fairly readable introduction to the basic principles: PDF

Two short, readable summaries: HTML1, HTML2

Here is a short tutorial on phase vocoders used for pitch shifting: HTML

The Wiki article is good, and has the basic references to the literature: HTML

Lecture 13 Notebook: IPYNB

Tutorial (with references) on T/P modification: HTML

A very complete survey of techniques for Pitch and Time Scale Modification: PDF

One of the most important improvements to Phase Vocoder for Pitch/Time Scale Mod is here: PDF

Short overview of Onset Detection in Librosa: HTML

Book chapter on onset detection and rhythm analysis: PDF

Tutorial paper on onset detection: PDF

Survey and comparison of Onset Detection methods: PDF

Lecture 14 Notebook: IPYNB

Book chapter on onset detection and rhythm analysis: PDF

Beat detection; Discussion of Final Projects

Lecture 16 Notebook: IPYNB

The Wikipedia article on Self-Similarity Matrices is short but useful: HTML

Here is a relatively readable early paper on structure detection, with some nice diagrams: PDF

Book chapter on Chord Recognition: PDF

Lecture 16 Notebook: IPYNB

Book chapter on Content-based Retrieval: PDF