CS 591 S1 - Introduction to Computational Audio

Spring, 2017


Instructor and TF Coordinates

Useful Links

Instructor:Wayne Snyder
      Email: waysnyder@gmail.com
      Office: MCS 290
      Office Hours:
W 3 - 7pm, T 7:00 - 10:00 in Rich Hall Cinema Room
      Cell Phone: 617-966-(2^10 + 41)(email vastly preferred)

Instructor: Daniel Valentine
      Email: dval@bu.edu
      Office Hours: 3:30 - 6:30 pm Friday


Schedule of Lectures and Events

Class
Date
Topic and/or Event
Readings
Lecture Slides & Code Examples
Homeworks, Tests & Misc.
0
R 1/19

Overview of course policies; meet and greet; presentation of schedule of topics with examples.

Syllabus

Web Pages with Comprehensive Coverage of Audio Processing:

CMRA: HTML

Beautiful Wikibook on some (not all) important topics in Sound Synthesis: HTML

 

 

 

1
T 1/24

Overview of physical properties of sound waves: frequency, amplitude, phase, intensity, loudness.

Measuring loudness: logarithmic scales and decibels.

Basics of digital signals and sampling; quantization error, aliasing, and choice of bit depth and sample rate; dynamic range; the Nyquist Theorem.

Introductory reading on waves and digital signals: PDF

Read sections 4.1 - 4.5 thoroughly.

Sorry for the bad job with the copying!

Tutorial (with animations) of sound waves: http://www.acs.psu.edu/drussell/demos.html

Look at the first three animations: What is a Wave, Wave Motion in Space and Time, and Supersposition of Two Waves.

Lecture One: PPT

Fun video by Vi Hart on basics of sound: HTML

Cool video of visualizing sound waves: HTML

Slides from talk surveying amplitude/intensity of sound waves (a bit too much detail for us, but here if you are interested): PDF

Another very complete lecture on human perception of loudness: PDF

 

2
R 1/26

Real Phasors as a way of understanding sine waves and sampling artifacts.

Aliases due to negative frequencies, negative amplitudes, and phase.

Aliases due to frequencies beyond the Nyquist Limit.

 

CMRA on Phasors: HTML Lecture Two: PPT

HW 01: HTML

HW 01 Solution: HTML

A good summary of why and how Python makes a good language for audio programming: PDF

3 T 1/31

Sinusoids created from sum of pure sine waves; time domain vs frequency domain representations.

Introduction to musical acoustics: How do guitar strings vibrate?

Basics of musical signals: Fundamental frequency, harmonic series, partials, timbre.

Musical acoustics concluded: The vocal tract; open vs closed pipes.

 

 

Reading about Fourier/Additive Synthesis: HTML

Excellent reading on spectra as they relate to musical instruments: HTML

Then take a look at the following two Wiki articles, which are excellent:

Harmonic Series

Timbre

Acoustics of Strings: HTML (read first part, stop when you see "Harmonics in Music")

Acoustics of Open and Closed Pipes: HTML (read whole thing!)

Acoustics of Vocal Tract: HTML (optional)

 

Lecture Three: PPT

Interactive applet showing how two sine waves add together--with sound! HTML

Limited but beautiful animation of Fourier Synthesis using real phasors: HTML

Phasor addition animation: HTML

Nice description of how standing waves in pipes actually work, with animation: HTML

Similar animation for traveling waves: HTML

Interesting talks at this workshop on musical timbre: HTML

 

4 R 2/2

Conclusion on spectra and spectrograms for musical signals.

"Peak-picking" spectral synthesis of musical sounds

 

Wave-table and sample-based synthesis; linear interpolation;

(locked) pitch and time shifting by interpolation.

 

 

Wikibook page on Additive Synthesis: HTML

CRMA on additive synthesis: HTML

The Wiki article on Linear Interpolation (HTML) is very good, take a look if you are unfamiliar with the concept.

Wikibook page on Wavetable synthesis: HTML

 

Lecture Four: PPT

Very complete technical chapter on audio synthesis (from Musimathics): PDF

Very complete paper on wavetable synthesis: PDF

5 T 2/7

Modulation synthesis: amplitude, ring, and frequency modulation.

Short description of ASDR envelopes: HTML

Remind yourself of the definition of exponential decay by reading the Wiki article (HTML) but stop when you get to "solution of the differential equation."

Wikipedia article on Tremolo: HTML

Wikibook page on Amplitude and Ring modulation: HTML

CMRA page on Frequency Modulation (with applet and sound samples): HTML

 

Lecture Five: PPT

Animation with sound for amplitude modulation: HTML

 

HW02: HTML

HW02 Solution: HTML

  R 2/9 Snow day!      
6 T 2/14

Conclusions on Ring and Frequency Synthesis: spectra and phase issues.

Smoothing a signal

 

 

 

 

Lecture Six: PPTX Technical article on audio smoothing: HTML
7 R 2/16

Smoothing and digital Filters

 

 

 

CRMA on filters: HTML

 

Lecture Seven: PPTX

 

HW 03: HTML (due 2/26)

HW 03 Solution (probs 1 - 6): PDF

HW 03 Python Solution: PY

HW 03 Solutions (p7-12): ZIP

CRMA on waveshaping: HTML

Wikibook chapter on subtractive synthesis: HTML

CRMA on filters: HTML

CRMA on formant synthesis: HTML

  T 2/21

NO CLASS (Monday Schedule)

   

 

 

 

8 R 2/23

Physical modeling synthesis; A glimpse at speech synthesis; the Karplus-Strong algorithm.

CRMA on Physical Modeling and Karplus-Strong: HTML

 

Lecture Eight: PPTX

Lecture 08 Video:MOV

On YouTube: HTML

Here is the first paper on Karplus-Strong: PDF

Here is a more extensive examination of the Karplus-Strong Algorithm and its extensions: PDF

9 T 2/28

Conclusions on Karplus-Strong;

Conclusions on music synthesis;

Demo of software synthesizer.

Very complete summary/taxonomy of music synthesis techniques: HTML

WaveNet for speech synthesis: HTML

Lecture Nine: PPT

 

  R 3/2 Midterm One    

Midterm Study Guide: HTML

Sample Exam from 2014: PDF

Sample Exam from 2016: PDF

Midterm 1 Solution: PDF

    SPRING BREAK    

Book Report: HTML (due 3/19)

HW 04: HTML (due 3/19)

HW 04 Solution: PDF

HW 04 Solution Code: hw04.py

  T 3/14 Snow Day!      
10

R 3/16

 

Introduction to Digital Signal Processing; sliding-window-based analysis of musical signals.

What happens when we multiply (or square) sine waves?

Algorithms for determining fundamental frequency: Zero-crossing rate; correlation and auto-correlation.

Determination of fundamental frequency using auto-correlation with "peak picking"; difference functions; pitch tracking.

Nice explanation of the basic formula for multiply sine waves: HTML

Correlation

Auto-Correlation, especially the section on Signal Processing, and especially the formula here, with the understanding that we are dealing with real signals, so y = y bar.

Lecture Ten: PPT

Lecture Ten Addendum (Video Lecture): Part 1, Part 2, Part 3 (about HW 05)

Here is a short survey of F0 determination algorithms: HTML
11 T 3/21 Introduction to Fourier Analysis: The Discrete Sine Transform.   Lecture Eleven: PDF

HW 05: HTML (due 3/26)

HW 05 solutions: HTML (Problem A 1), PDF (rest of problems), PY (all but problem A 1)

12 R 3/23

 

Accounting for phase: motivation for complex exponential representation of signals.

Representing signals using complex numbers and complex exponentials; complex phasors.

Reading on representing signals using complex numbers: PDF

The wiki article on phasors has too much detail for us, but has some beautiful animations: HTML

Here is another tutorial with a useful animation for manipulating phasors: HTML

Used the board!

Cmath library in Python for Complex Number: HTML

https://betterexplained.com/articles/understanding-why-complex-multiplication-works/

https://betterexplained.com/articles/a-visual-intuitive-guide-to-imaginary-numbers/

https://betterexplained.com/articles/intuitive-understanding-of-eulers-formula/

 

13 T 3/28

The Discrete Fourier Transform in the complex domain.

 

Very general overview of applications of the FT in digital music: HTML

Reasonably simple explanation of the basic concepts, with fun animations: HTML

Here is a short, intuitive explanation of the Discrete Fourier Transform in the complex-number case: HTML

Fun animation video on YT: HTML

Interesting summary of FFT techniques, including sliding window analysis, applied to bird songs: PDF

Used the board and then walked through some Python code:

Code: lecture13.py

Output: PDF

YT Video: Part 1, Part2

HW 06: HTML (due 4/2)

https://betterexplained.com/articles/an-interactive-guide-to-the-fourier-transform/

HW 06 Solution: HTML

Part B Solutions: PY, PDF

14 R 3/30

 

Issues with the DFT: Resolution in time and frequency, "leakage" and "spray," Hann Windows.

Reading on using the FT, windowing, leakage, etc.: PDF Lecture 14: PPTX  
15 T 4/4

The Fast Fourier Transform

The Inverse Fourier Transform: Fast Synthesis, Filters, Convolution, and Reverb

Here is a nice summary of the FFT: HTML

Wikipedia article on convolution: HTML (just try to understand the basic definition, but do look at the animations down the page).

Here is a short tutorial on software for convolution reverb (with audio examples): HTML

Lecture 15: PPTX

Here is a brief tutorial on convolution with examples in 1D and 2D: HTML; skip the stuff on impulse response and linear, time-invariant systems and just concentrate on the examples as a way of understanding the algorithm

Here is a nice interactive animation of convolution in the continuous case: HTML

Informative article about collecting impulse responses for convolution reverb: HTML

Collection of impulse responses from around the USA: HTML

16 R 4/6

Spectrograms

Changing time (and therefore pitch): 1D interpolation and resampling

The wiki articles are useful: HTML1, HTML2

Here is a fairly readable introduction to the basic principles: PDF

Two short, readable summaries: HTML1, HTML2

Lecture 16: HTML

HW 07: HTML (due TUESDAY 4/11)

Part A Solutions: HTML

Part B Code: hw07.py

Part B Printout: PDF

17 T 4/11 Phase vocoder: Unlocking pitch and time

Here is a short tutoring on phase vocoders used for pitch shifting: HTML

The Wiki article is good, and has the basic references to the literature: HTML

 

Tutorial (with references) on T/P modification: HTML

A very complete survey of techniques for Pitch and Time Scale Modification: PDF

One of the most important improvements to Phase Vocoder for Pitch/Time Scale Mod is here: PDF

 

  R 4/13 Midterm Two

 

 

Study Guide for Midterm: Everything since last midterm is fair game! The exam will consist entirely of variations of homework problems from HW 04 - HW 07. You will have a choice of doing 4 out of 5, with one mandatory problem. The mandatory problem will be related to a Part B problem (you may have to write or rewrite some code, or interprete some results produced by an algorithm we implemented) and the others will be very similar to Part A problems.

You may bring a single 3x5 card to the exam.

18 T 4/18 Introduction to Music Information Retrieval: Onset, Rhythm, and Meter detection; cepstrum analysis.

 

Powerpoint Overview of Onset, Rhythm, and Meter analysis: PDF

Book chapter on beat detection: PDF

Intro reading on onset detection: PDF

Paper on Rhythm Analysis: PDF
Lecture 18: PDF Nice summary of distance metrics: HTML
19 R 4/20 Pitch Tracking and melody analysis; Classification of musical signals: identifying instruments, identifying chords.   Lecture 19: PDF

Chapter 7 in this course is a compact summary of the issues in pitch tracking: HTML

Slides from survey talk on music classification:PDF

Project Proposal Due 4/23

20 T 4/25

Time Warping, signal-signal alignment, score-signal alignment.

  Lecture 20: PDF  
21 R 4/27 Structure Analysis.   Lecture 21: PDF  
22 T 5/2 Last class cancelled: work on your projects! Here is a free E-book from O'Reilly about "Music Science": HTML    
  F 5/12 Final Project Due by 5pm.     Final Project Due by 5pm.