CAS CS 585 Image and Video Computing - Spring 2021

Course Info - Course Objectives - Course Materials - Grading Policy - Collaboration and Academic Integrity
Help - Course Schedule - Labs - Assignments - Computer Vision Companies


Lectures: Tuesday, Thursday 9:30 am - 10:45 pm in CAS 502.
Labs: Friday 2:30-3:20 and 3:35-4:25 in KCB 107 and 4:40-5:30 in EMA 304
Instructor: Prof. Margrit Betke
Teaching Assistants: Hao Yu and Ryan Yu
Class web page:
http://www.cs.bu.edu/faculty/betke/cs585
Class mailing list:
cascs585a1-l@bu.edu
Piazza page: https://piazza.com/class/kkcsajmqq235sh
Contact Information:

Staff Best Way to Contact Zoom Office Hours Zoom Link
Margrit Betke betke@bu.edu Mon 16:00-18:00, Tue/Thu 10:45-11:15 https://bostonu.zoom.us/my/margritbetke
Hao and Ryan Piazza. But note that there is a 24-hour timer on your Piazza questions to encourage other students to answer questions before TFs will able to see your question.
Only send an email if your question is urgent and requires an answer earlier than 24 hours. In that case, send an email to both TFs: Hao haoyu@bu.edu and Ryan ryu1@bu.edu
Tue 16:00-18:00 EST,
Wed 7:00-9:00 EST
TF zoom link

Seeing Me in My Office:
Due to the pandemic, I will mostly work from home. Please try to reach me after class or on Monday afternoon during my office hours. You can also make an appointment by email. I'm happy to talk with you about the course, computer vision research, your plans for the future, or anything else. Check out my personal web page to get to know me a little.

Teaching Fellow Responsibilities:
The TFs are responsible for teaching the Laboratory section, helping you out during their office hours, and grading of the homework (or managing graders).


Course Objectives

Our goal is to build computer systems that analyze images automatically and determine what the computer "sees" or "recognizes." The course gives you a fundamental introduction to computer vision methods. Applications include human-computer interfaces, face detection, medical image processing, infrared image analysis of animals, and vision systems for intelligent vehicles.

Prerequisites: 1 year programming experience (e.g., Python, C++ or Java at CAS CS 112 level), linear algebra or geometric algorithms, and calculus.


Course Materials

The syllabus below will be updated throughout the semester with lecture notes, videos, and reading materials. Links to homework assignments will come alive when the assignment is ready for you to work on.

Textbook: I recommend some chapters in Robot Vision by BKP Horn, MIT Press. The book is not required. I will propose alternative reading material.

Computing Environment: You will use the Computer Science Department's main server csa2.bu.edu to submit programming solutions. If you do not have a CS account yet, you may activate your account online at http://www.bu.edu/cs/account. You can also access csa2 remotely using scp and ssh (linux) or putty and WinSCP (Windows). You also have the opportunity to use GPUs provided by the BU IST Department's Scientific Computing Cluster.


Grading Policy

Homework: The homework includes bi-weekly programming, reading assignments, and problem sets. The due dates are listed below. Programs and reports must be submitted electronically. Solutions to problem sets must be submitted in class. They do not need to be typed but should look professional, in particular, write legibly and leave a margin for grading comments.

Guidelines for submission are provided with each assignment. Late solutions will be levied a late penalty of 20% per day (up to three days). After three days, no credit will be given.

You must demo your bi-weekly programming projects and your final class project to the TFs or the graders within one of the offered time slots in order to receive full credit for your project submission. A signup sheet with available demo times will be circulated during class before the due date.

Your electronic project submission should include detailed instructions for compiling and setting up your program. The grader will recompile your program and test it on your input videos.

Project: Please read the project guidelines. You can propose your own project topic or use one of my project suggestions. I will discuss your project's scope, design, and presentation with you in my office hours and provide guidance throughout the semester. You must work in a group of 2-4 students. You will be asked to select a project topic by the middle of the semester and present the final project in class at the end of the semester. Here will be the project schedule.

Computer Vision Talks: Students are strongly encouraged to attend the Image and Video Computing talks, which are part of the new AI Research Seminar series on Mondays, 11-12 am, online.

Class Participation: Come to class, either in person or remotely, and participate regularly. Reading the textbook and listening in class will only give you a "passive understanding" of the material. I encourage discussions in class to help you acquire an "active understanding" of the material so that you can evaluate existing computer vision techniques critically and develop your own creative solutions. Participation counts toward your overall grade. If you have trouble connecting via zoom in real time, let us know and make sure to be active on Piazza so that you do not receive a failing grade in participation.

Talk Writups: You must attend at least three talks on subjects related to CS 585 and write a summary on each talk (at most 2 pages per summary). Your summary should give a problem definition, summarize the algorithms and results, discuss the work critically, and also briefly explain how the work relates to material discussed in class. You must submit a printout of your summary at the beginning of class on the dates listed in the syllabus. Check your text for typographical and grammatical errors (e.g., use grammarly.com). You will lose points if you simply copy the speaker's abstract, your review is late, contains typographical and grammatical errors, does not contain a discussion, and does not contain a statement how the talk relates to class material.

Exams: There will be two exams on the material discussed in the class and practiced with homeworks. The exams will be quite easy for students who come to class, participate in our discussions, and keep up with homework assignments and programming projects. The date of the midterm exam is Thursday, February 25 the date of the final exam is in the final exam period, as determined by the Registrar's Office. The final exam will focus on material discussed in the second part of the course, but may test earlier material. You are allowed to use one double-sided page of notes in each exam.

Grading Policy: Your final grade will be determined roughly as follows:


Collaboration and Academic Integrity

You are encouraged to collaborate on the solution of the homework. If you do, you must acknowledge your collaborators. Each student must submit his or her own electronic version of the solutions. You can request an exception to this rule for your final project. If you use algorithms or code that are not your own original work and that were not provided in class or discussed in the textbook, you must give a detailed acknowledgment of your source .

You are not allowed to collaborate on the solution of the exams. Sources must be acknowledged.

Cheating and plagiarism are not worthy of Boston University students. I expect you to abide by the rule stated above and the standards of academic honesty and computer ethics policy described in http://www.bu.edu/computing/ethics/ and http://www.bu.edu/academics/policies/academic-conduct-code


Help

Image and Video Computing is an elective course that will introduce you to an exciting topic in computer science. It should be fun and not too much of a struggle for you. Make sure that you have had the prerequisites. Depending on your level of programming experience and/or mathematics background, the course may be challenging for you. If you do not understand the material, ask for help immediately. Ask questions in class. If one student is confused about something, then maybe others are also confused and grateful that someone asked. Come and see me or the TF for help or send us email. Our task is to help you learn a very interesting topic!

Course Schedule

Dates Topics --- Links will become active with lecture Readings --- Links will become active Assignments
1/26/21 Course Introduction: Why study IVC? Industry successes and current needs. Lecture 1 links, Wiki Intro or Horn Ch. 1.  
1/28/21 Image formation, image and video formats, color. Programming with Images: Pitfalls. Image Projections, Flood Fill Algorithm, Sequential Multi-object Labeling Algorithm. Sequential Labeling, Lectures 2&3. 1/29/21: A1 out (after lab)
2/2 - 2/4/21 Template matching background differencing. Skin-based face detection. Similarity Functions (SSD, NCC), Motion: Template-based Tracking. Image Pyramids. Binary Image Analysis: Moments, centroid. Wiki on template matching, normalized correlation. Moments and distances. Image Moments, Binary Image Analysis. Skeleton, Horn Ch. 3, A1 due.
2/9/21 Binary Image Analysis: Object orientation, circularity measures. Image Moments, Binary Image Analysis. Horn Ch. 3. .
2/11/21 Circularity and distance measures. Hu Moments. Tumor Detection in Computed Tomography Images. Object Skeletons. Confusion matrix analysis. Moments and distances. Skeleton. Hu Moments, Fawcett (confusion matrix analysis), Horn Ch. 4. A2 out.
2/18/21 No class on Tuesday, February 16 (Monday schedule).
Morphology. Lung image analysis with morphology operators. Neighborhoods. Border following algorithm. Hausdorff Distance. Segmentation: Thresholding techniques. Censusing Bats. Image Smoothing, Edge Detection.
Border following algorithm. Segmentation (not all covered). Morphology, Erosion, Hausdorff distance. Segmentation methods, Segmentation examples, Segmentation in biomedical images. Wiki on: Edge Detection, Censusing Millions of Bats. Thresholding, Optional: Wang et al. 2005. A2 due, A3 out
2/23 Edge detection examples. Lecture notes on edge detection & active contours Canny Edge Detection. Active Contours. Sobel, Prewitt, Roberts, Mexican Hat, Difference of Gaussians, Canny Edge Detector. Williams and Shah, 1992: paper, figures, and lecture. Hausdorff distance. A3 due.
2/25 Midterm Exam    
3/2 - 3/4/21
Last day to drop class (without a 'W' grade), 3/1/2021. Active Contours. Tracking Methods and Applications: Tracking with Alpha-beta Filter, Kalman Filter, Tracking Groups of Animals, Multiple Object Tracking and Data Association.
Bat tracking presentation, Alpha beta filter, Kalman filter. Multiple-Object Tracking A4 out, Project proposal P1 out
3/17 2D Optical Flow and the Lucas-Kanade and Horn-and-Schunk Algorithms by Margrit Betke. Lecture video by Margrit Betke.
Optical Flow, Horn 81
.
3/9/21 Computer Vision Interfaces for People with Disabilities: Camera Mouse. Facial Expressions, Black/Yacoob 95, Yacoob1 avi, Yacoob2 avi Camera Mouse  
3/12, 3/16,3/30 TBA TBA  
4/1/21 Lung Image Analysis of COVID19 and Cancer Patients. Absolute Orientation in 2D. Lecture video by Margrit Betke.
Absolute Orientation, Horn 89
.
4/6 Lung Image Analysis of COVID-19 and Cancer Patients: CT Scan Alignment Part 2: Quaternions. Horn's 3D Absolute Orientation Algorithm. Iterative Closest Point Algorithm. Lung Surface Alignment. Lecture video by Margrit Betke.
Besl and McKay, 1992. Ko and Betke, 2001. Betke, Hong et al., 2003
Project proposal feedback. Project assignment P2 out.
4/8/21, 4/13/21, 4/15/21, 4/20/21, 4/22/21 Pinhole Model, Binocular Stereo, Lens Equation
Style Transfer with Neural Models
Lecture video by Margrit Betke.
Lecture video by Xide Xia.
Thin lens equation for image formation.
Project update P2 due. Talk 1 review due. A5 out (no programming)
4/27/21 Binocular Stereo, Multiview Stereo, Epipolar Geometry, Active Stereo with Structured Light, Photometric Stereo
Lecture video by Margrit Betke.
Wiki on Lambert's law, Lambertian reflectance, Lecture slides on photometric stereo
 
4/29/21 Student Projects: Guidelines, Topics, Schedule   Projects (P3) due, A5 due. Talk 2 review due.
Wed., May 6, 12-2 Takehome Final Exam: May 5, 00:01 am, to May 6, 11:59 pm    

Labs

The links will become active as the semester progresses.

Assignments

The assignments will have some programming components and some paper-and-pencil exercises. The links will became active when the assignment is announced.

There will be two assignments that relate to your projects (P1 and P2). For project ideas, check here.

(Potential) Topics:

Grades

Please find your grades here: CS585 Grades.

Computer Vision Links

Check out http://www.cs.bu.edu/faculty/betke/links.html if you need additional ideas for your class project, if you are looking for a job in computer vision (list of companies), or if you are interested in computer vision research. You will find a list of links to computer vision conferences, journals, research groups, and companies.

Calculus Background

I do not expect you to have a background multivariate calculus. I will introduce the tools we will need. You may find the first few chapters of these notes by Cain and Herod useful, in particular, partial derivatives, Taylor polynomial, Multivariate Taylor polynomial.

Margrit Betke, Professor
Computer Science Department
Boston University
111 Cummington Mall (campus map)
Boston, MA 02215
Email: betke @ bu.edu
URL: http://www.cs.bu.edu/faculty/betke
Phone: 617-353-8919
Fax: 617-353-6457

Last updated: January 25, 2021