Spring, 2017






Class 
Date 
Topic and/or Event 
Readings 
Lecture Slides & Code Examples 
Homeworks, Tests & Misc. 
0 
R 1/19  Overview of course policies; meet and greet; presentation of schedule of topics with examples. 
Web Pages with Comprehensive Coverage of Audio Processing: CMRA: HTML Beautiful Wikibook on some (not all) important topics in Sound Synthesis: HTML 


1 
T 1/24  Overview of physical properties of sound waves: frequency, amplitude, phase, intensity, loudness. Measuring loudness: logarithmic scales and decibels. Basics of digital signals and sampling; quantization error, aliasing, and choice of bit depth and sample rate; dynamic range; the Nyquist Theorem. 
Introductory reading on waves and digital signals: PDF Read sections 4.1  4.5 thoroughly. Sorry for the bad job with the copying! Tutorial (with animations) of sound waves: http://www.acs.psu.edu/drussell/demos.html Look at the first three animations: What is a Wave, Wave Motion in Space and Time, and Supersposition of Two Waves. 
Lecture One: PPT 
Fun video by Vi Hart on basics of sound: HTML Cool video of visualizing sound waves: HTML Slides from talk surveying amplitude/intensity of sound waves (a bit too much detail for us, but here if you are interested): PDF Another very complete lecture on human perception of loudness: PDF

2 
R 1/26  Real Phasors as a way of understanding sine waves and sampling artifacts. Aliases due to negative frequencies, negative amplitudes, and phase. Aliases due to frequencies beyond the Nyquist Limit.

CMRA on Phasors: HTML  Lecture Two: PPT  HW 01: HTML HW 01 Solution: HTML A good summary of why and how Python makes a good language for audio programming: PDF 
3  T 1/31  Sinusoids created from sum of pure sine waves; time domain vs frequency domain representations. Introduction to musical acoustics: How do guitar strings vibrate? Basics of musical signals: Fundamental frequency, harmonic series, partials, timbre. Musical acoustics concluded: The vocal tract; open vs closed pipes.

Reading about Fourier/Additive Synthesis: HTML Excellent reading on spectra as they relate to musical instruments: HTML Then take a look at the following two Wiki articles, which are excellent: Acoustics of Strings: HTML (read first part, stop when you see "Harmonics in Music") Acoustics of Open and Closed Pipes: HTML (read whole thing!) Acoustics of Vocal Tract: HTML (optional)

Lecture Three: PPT  Interactive applet showing how two sine waves add togetherwith sound! HTML Limited but beautiful animation of Fourier Synthesis using real phasors: HTML Phasor addition animation: HTML Nice description of how standing waves in pipes actually work, with animation: HTML Similar animation for traveling waves: HTML Interesting talks at this workshop on musical timbre: HTML

4  R 2/2  Conclusion on spectra and spectrograms for musical signals. "Peakpicking" spectral synthesis of musical sounds
Wavetable and samplebased synthesis; linear interpolation; (locked) pitch and time shifting by interpolation.

Wikibook page on Additive Synthesis: HTML CRMA on additive synthesis: HTML The Wiki article on Linear Interpolation (HTML) is very good, take a look if you are unfamiliar with the concept. Wikibook page on Wavetable synthesis: HTML

Lecture Four: PPT  Very complete technical chapter on audio synthesis (from Musimathics): PDF Very complete paper on wavetable synthesis: PDF 
5  T 2/7  Modulation synthesis: amplitude, ring, and frequency modulation. 
Short description of ASDR envelopes: HTML Remind yourself of the definition of exponential decay by reading the Wiki article (HTML) but stop when you get to "solution of the differential equation." Wikipedia article on Tremolo: HTML Wikibook page on Amplitude and Ring modulation: HTML CMRA page on Frequency Modulation (with applet and sound samples): HTML

Lecture Five: PPT  Animation with sound for amplitude modulation: HTML
HW02: HTML HW02 Solution: HTML 
R 2/9  Snow day!  
6  T 2/14  Conclusions on Ring and Frequency Synthesis: spectra and phase issues. Smoothing a signal


Lecture Six: PPTX  Technical article on audio smoothing: HTML 
7  R 2/16  Smoothing and digital Filters

CRMA on filters: HTML

Lecture Seven: PPTX 
HW 03: HTML (due 2/26) HW 03 Solution (probs 1  6): PDF HW 03 Python Solution: PY HW 03 Solutions (p712): ZIP CRMA on waveshaping: HTML Wikibook chapter on subtractive synthesis: HTML CRMA on filters: HTML CRMA on formant synthesis: HTML 
T 2/21  NO CLASS (Monday Schedule) 


8  R 2/23  Physical modeling synthesis; A glimpse at speech synthesis; the KarplusStrong algorithm. 
CRMA on Physical Modeling and KarplusStrong: HTML

Lecture Eight: PPTX Lecture 08 Video:MOV On YouTube: HTML 
Here is the first paper on KarplusStrong: PDF Here is a more extensive examination of the KarplusStrong Algorithm and its extensions: PDF 
9  T 2/28  Conclusions on KarplusStrong; Conclusions on music synthesis; Demo of software synthesizer. 
Very complete summary/taxonomy of music synthesis techniques: HTML WaveNet for speech synthesis: HTML 
Lecture Nine: PPT 

R 3/2  Midterm One  Midterm Study Guide: HTML Sample Exam from 2014: PDF Sample Exam from 2016: PDF Midterm 1 Solution: PDF 

SPRING BREAK  Book Report: HTML (due 3/19) HW 04: HTML (due 3/19) HW 04 Solution: PDF HW 04 Solution Code: hw04.py 

T 3/14  Snow Day!  
10  R 3/16

Introduction to Digital Signal Processing; slidingwindowbased analysis of musical signals. What happens when we multiply (or square) sine waves? Algorithms for determining fundamental frequency: Zerocrossing rate; correlation and autocorrelation. Determination of fundamental frequency using autocorrelation with "peak picking"; difference functions; pitch tracking. 
Nice explanation of the basic formula for multiply sine waves: HTML AutoCorrelation, especially the section on Signal Processing, and especially the formula here, with the understanding that we are dealing with real signals, so y = y bar. 
Lecture Ten: PPT Lecture Ten Addendum (Video Lecture): Part 1, Part 2, Part 3 (about HW 05) 
Here is a short survey of F0 determination algorithms: HTML 
11  T 3/21  Introduction to Fourier Analysis: The Discrete Sine Transform.  Lecture Eleven: PDF  HW 05: HTML (due 3/26) HW 05 solutions: HTML (Problem A 1), PDF (rest of problems), PY (all but problem A 1) 

12  R 3/23 
Accounting for phase: motivation for complex exponential representation of signals. Representing signals using complex numbers and complex exponentials; complex phasors. 
Reading on representing signals using complex numbers: PDF The wiki article on phasors has too much detail for us, but has some beautiful animations: HTML Here is another tutorial with a useful animation for manipulating phasors: HTML 
Used the board!  Cmath library in Python for Complex Number: HTML https://betterexplained.com/articles/understandingwhycomplexmultiplicationworks/ https://betterexplained.com/articles/avisualintuitiveguidetoimaginarynumbers/ https://betterexplained.com/articles/intuitiveunderstandingofeulersformula/

13  T 3/28  The Discrete Fourier Transform in the complex domain.

Very general overview of applications of the FT in digital music: HTML Reasonably simple explanation of the basic concepts, with fun animations: HTML Here is a short, intuitive explanation of the Discrete Fourier Transform in the complexnumber case: HTML Fun animation video on YT: HTML Interesting summary of FFT techniques, including sliding window analysis, applied to bird songs: PDF 
Used the board and then walked through some Python code: Code: lecture13.py Output: PDF 
HW 06: HTML (due 4/2) https://betterexplained.com/articles/aninteractiveguidetothefouriertransform/ HW 06 Solution: HTML 
14  R 3/30 
Issues with the DFT: Resolution in time and frequency, "leakage" and "spray," Hann Windows. 
Reading on using the FT, windowing, leakage, etc.: PDF  Lecture 14: PPTX  
15  T 4/4  The Fast Fourier Transform The Inverse Fourier Transform: Fast Synthesis, Filters, Convolution, and Reverb 
Here is a nice summary of the FFT: HTML Wikipedia article on convolution: HTML (just try to understand the basic definition, but do look at the animations down the page). Here is a short tutorial on software for convolution reverb (with audio examples): HTML 
Lecture 15: PPTX  Here is a brief tutorial on convolution with examples in 1D and 2D: HTML; skip the stuff on impulse response and linear, timeinvariant systems and just concentrate on the examples as a way of understanding the algorithm Here is a nice interactive animation of convolution in the continuous case: HTML Informative article about collecting impulse responses for convolution reverb: HTML Collection of impulse responses from around the USA: HTML 
16  R 4/6  Spectrograms Changing time (and therefore pitch): 1D interpolation and resampling 
The wiki articles are useful: HTML1, HTML2 Here is a fairly readable introduction to the basic principles: PDF 
Lecture 16: HTML  HW 07: HTML (due TUESDAY 4/11) Part A Solutions: HTML Part B Code: hw07.py Part B Printout: PDF 
17  T 4/11  Phase vocoder: Unlocking pitch and time  Here is a short tutoring on phase vocoders used for pitch shifting: HTML The Wiki article is good, and has the basic references to the literature: HTML 
Tutorial (with references) on T/P modification: HTML A very complete survey of techniques for Pitch and Time Scale Modification: PDF One of the most important improvements to Phase Vocoder for Pitch/Time Scale Mod is here: PDF


R 4/13  Midterm Two 

Study Guide for Midterm: Everything since last midterm is fair game! The exam will consist entirely of variations of homework problems from HW 04  HW 07. You will have a choice of doing 4 out of 5, with one mandatory problem. The mandatory problem will be related to a Part B problem (you may have to write or rewrite some code, or interprete some results produced by an algorithm we implemented) and the others will be very similar to Part A problems. You may bring a single 3x5 card to the exam. 

18  T 4/18  Introduction to Music Information Retrieval: Onset, Rhythm, and Meter detection; cepstrum analysis. 
Powerpoint Overview of Onset, Rhythm, and Meter analysis: PDF Book chapter on beat detection: PDF Intro reading on onset detection: PDF Paper on Rhythm Analysis: PDF 
Lecture 18: PDF  Nice summary of distance metrics: HTML 
19  R 4/20  Pitch Tracking and melody analysis; Classification of musical signals: identifying instruments, identifying chords.  Lecture 19: PDF  Chapter 7 in this course is a compact summary of the issues in pitch tracking: HTML Slides from survey talk on music classification:PDF Project Proposal Due 4/23 

20  T 4/25  Time Warping, signalsignal alignment, scoresignal alignment. 
Lecture 20: PDF  
21  R 4/27  Structure Analysis.  Lecture 21: PDF  
22  T 5/2  Last class cancelled: work on your projects!  Here is a free Ebook from O'Reilly about "Music Science": HTML  
F 5/12  Final Project Due by 5pm.  Final Project Due by 5pm. 