CS 591 S1 - Introduction to Computational Audio

Spring, 2017


Instructor and TF Coordinates

Useful Links

Instructor:Wayne Snyder
      Email: waysnyder@gmail.com
      Office: MCS 290
      Office Hours:
W 3 - 7pm, T 7:00 - 10:00 in Rich Hall Cinema Room
      Cell Phone: 617-966-(2^10 + 41)(email vastly preferred)

Instructor: Daniel Valentine
      Email: dval@bu.edu
      Office Hours: 3:30 - 6:30 pm Friday


Schedule of Lectures and Events

Class
Date
Topic and/or Event
Readings
Lecture Slides & Code Examples
Homeworks, Tests & Misc.
0
R 1/19

Overview of course policies; meet and greet; presentation of schedule of topics with examples.

Syllabus

Web Pages with Comprehensive Coverage of Audio Processing:

CMRA: HTML

Beautiful Wikibook on some (not all) important topics in Sound Synthesis: HTML

 

 

 

1
T 1/24

Overview of physical properties of sound waves: frequency, amplitude, phase, intensity, loudness.

Measuring loudness: logarithmic scales and decibels.

Basics of digital signals and sampling; quantization error, aliasing, and choice of bit depth and sample rate; dynamic range; the Nyquist Theorem.

Introductory reading on waves and digital signals: PDF

Read sections 4.1 - 4.5 thoroughly.

Sorry for the bad job with the copying!

Tutorial (with animations) of sound waves: http://www.acs.psu.edu/drussell/demos.html

Look at the first three animations: What is a Wave, Wave Motion in Space and Time, and Supersposition of Two Waves.

Lecture One: PPT

 

Cool video of visualizing sound waves: HTML

Slides from talk surveying amplitude/intensity of sound waves (a bit too much detail for us, but here if you are interested): PDF

 

2
R 1/26

Real Phasors as a way of understanding sine waves and sampling artifacts.

Aliases due to negative frequencies, negative amplitudes, and phase.

Aliases due to frequencies beyond the Nyquist Limit.

 

CMRA on Phasors: HTML Lecture Two: PPT

HW 01: HTML

HW 01 Solution: HTML

 

3 T 1/31

Sinusoids created from sum of pure sine waves; time domain vs frequency domain representations.

Introduction to musical acoustics: How do guitar strings vibrate?

Basics of musical signals: Fundamental frequency, harmonic series, partials, timbre.

Musical acoustics concluded: The vocal tract; open vs closed pipes.

 

 

Reading about Fourier/Additive Synthesis: HTML

Excellent reading on spectra as they relate to musical instruments: HTML

Then take a look at the following two Wiki articles, which are excellent:

Harmonic Series

Timbre

Acoustics of Strings: HTML (read first part, stop when you see "Harmonics in Music")

Acoustics of Open and Closed Pipes: HTML (read whole thing!)

Acoustics of Vocal Tract: HTML (optional)

 

Lecture Three: PPT

Interactive applet showing how two sine waves add together--with sound! HTML

Limited but beautiful animation of Fourier Synthesis using real phasors: HTML

Phasor addition animation: HTML

Nice description of how standing waves in pipes actually work, with animation: HTML

Similar animation for traveling waves: HTML

Interesting talks at this workshop on musical timbre: HTML

 

4 R 2/2

Conclusion on spectra and spectrograms for musical signals.

"Peak-picking" spectral synthesis of musical sounds

 

Wave-table and sample-based synthesis; linear interpolation;

(locked) pitch and time shifting by interpolation.

 

 

Wikibook page on Additive Synthesis: HTML

CRMA on additive synthesis: HTML

The Wiki article on Linear Interpolation (HTML) is very good, take a look if you are unfamiliar with the concept.

Wikibook page on Wavetable synthesis: HTML

 

Lecture Four: PPT

Very complete technical chapter on audio synthesis (from Musimathics): PDF

Very complete paper on wavetable synthesis: PDF

5 T 2/7

Modulation synthesis: amplitude, ring, and frequency modulation.

Short description of ASDR envelopes: HTML

Remind yourself of the definition of exponential decay by reading the Wiki article (HTML) but stop when you get to "solution of the differential equation."

Wikipedia article on Tremolo: HTML

Wikibook page on Amplitude and Ring modulation: HTML

CMRA page on Frequency Modulation (with applet and sound samples): HTML

 

Lecture Five: PPT

Animation with sound for amplitude modulation: HTML

 

HW02: HTML

HW02 Solution: HTML

  R 2/9 Snow day!      
6 T 2/14

Conclusions on Ring and Frequency Synthesis: spectra and phase issues.

Smoothing a signal

 

 

 

 

Lecture Six: PPTX Technical article on audio smoothing: HTML
7 R 2/16

Smoothing and digital Filters

 

 

 

CRMA on filters: HTML

 

Lecture Seven: PPTX

 

HW 03: HTML (due 2/26)

HW 03 Solution (probs 1 - 6): PDF

HW 03 Python Solution: PY

HW 03 Solutions (p7-12): ZIP

CRMA on waveshaping: HTML

Wikibook chapter on subtractive synthesis: HTML

CRMA on filters: HTML

CRMA on formant synthesis: HTML

  T 2/21

NO CLASS (Monday Schedule)

   

 

 

 

8 R 2/23

Physical modeling synthesis; A glimpse at speech synthesis; the Karplus-Strong algorithm.

CRMA on Physical Modeling and Karplus-Strong: HTML

 

Lecture Eight: PPTX

Lecture 08 Video:MOV

On YouTube: HTML

Here is the first paper on Karplus-Strong: PDF

Here is a more extensive examination of the Karplus-Strong Algorithm and its extensions: PDF

9 T 2/28

Conclusions on Karplus-Strong;

Conclusions on music synthesis;

Demo of software synthesizer.

Very complete summary/taxonomy of music synthesis techniques: HTML

WaveNet for speech synthesis: HTML

Lecture Nine: PPT

 

  R 3/2 Midterm One    

Midterm Study Guide: HTML

Sample Exam from 2014: PDF

Sample Exam from 2016: PDF

    SPRING BREAK    

Book Report: HTML (due 3/19)

HW 04: HTML (due 3/19)

10

T 3/14

Happy π Day!

Introduction to Digital Signal Processing; sliding-window-based analysis of musical signals.

What happens when we multiply (or square) sine waves?

Algorithms for determining fundamental frequency: Zero-crossing rate; correlation and auto-correlation.

 

Nice explanation of the basic formula for multiply sine waves: HTML

Correlation

Auto-Correlation, especially the section on Signal Processing, and especially the formula here, with the understanding that we are dealing with real signals, so y = y bar.

Lecture Ten: PPT

Lecture Ten Addendum (Video Lecture): Part 1, Part 2, Part 3 (about HW 05)

 
11

R 3/16

Determination of fundamental frequency using auto-correlation with "peak picking"; difference functions; pitch tracking.

 

 

 

  Here is a short survey of F0 determination algorithms: HTML
12 T 3/21

Introduction to Fourier Analysis: The Discrete Sine Transform.

Accounting for phase: motivation for complex exponential representation of signals.

Representing signals using complex numbers and complex exponentials; complex phasors.

Reading on representing signals using complex numbers: PDF

The wiki article on phasors has too much detail for us, but has some beautiful animations: HTML

Here is another tutorial with a useful animation for manipulating phasors: HTML

Lecture Eleven: PDF HW 05: HTML (due 3/26)
13 R 3/23

The Discrete Fourier Transform in the complex domain.

The Fast Fourier Transform

Here is a short, intuitive explanation of the Discrete Fourier Transform in the complex-number case: HTML

Interesting summary of FFT techniques, including sliding window analysis, applied to bird songs: PDF

   
14 T 3/28 Issues with the FFT: Resolution, zero padding, "leakage" and "spray," Hann Windows. Reading on using the FT, windowing, leakage, etc.: PDF   HW 06: HTML (due 4/2)
15 R 3/30

The Inverse Fourier Transform: Fast Synthesis, Convolution, Filters and Reverb

 

   
16 T 4/4

Spectrograms

Phase Vocoder: Unlocked time and pitch shifting.

The wiki articles are useful: HTML1, HTML2

Here is a fairly readable introduction to the basic principles: PDF

Two short, readable summaries: HTML1, HTML2

   
17 R 4/6 Midterm Two

 

 

   
18 T 4/11 Introduction to Music Information Retrieval: Onset Detection;

Here are some papers:

Overview: PDF

Tutorial on onset detection: PDF

 

HW 07: HTML (due 4/16)

Semester Project Proposal due 4/16

19 R 4/13 Beat, rhythm, and meter detection; cepstrum analysis. Paper on Rhythm Analysis: PDF    
24 T 4/18 Instrument recognition by statistical pattern matching; chord recognition.      
25 R 4/20 Time Warping, signal-signal alignment, score-signal alignment.     Semester Project Progress Report due 4/23
26 T 4/25 Structure Analysis.      
27 R 4/27 Audio database retrieval: Query by Humming and Fingerprinting      
28 T 5/2 Music and data science Here is a free E-book from O'Reilly about "Music Science": HTML    
  F 5/12 Final Project Due by 5pm.     Final Project Due by 5pm.