Course Description

Can computers understand the visual world as we do? This course treats vision as a process of inference from noisy and uncertain data and emphasizes probabilistic, statistical, data-driven approaches. Topics include image processing; segmentation, grouping, and boundary detection; recognition and detection; motion estimation and structure from motion. This class will also lead you to the discussion of applications regarding state-of-the-art techniques in recognition, detection, and video analysis.
The course will consist of five programming projects, one final project, and a few self-tutorial sessions (12 minutes for each team of 5 students). Please find information about projects and self-tutorial sessions in the syllabus.

Prerequisites

This course requires programming experience (mainly Python) as well as linear algebra, basic calculus, and basic probability. Previous knowledge of visual computing will be helpful.

Textbook

Readings will be assigned in "Computer Vision: Algorithms and Applications" by Richard Szeliski. The book is available for free online or available for purchase.

Resource

Awesome computer vision GitHub link
Awesome deep learning GitHub link

Grading

Your final grade will be made up from

  • 60% 5 programming projects
  • 30% final projects (includes proposal, midtern report, project pitch, project presentation, and project report). 5 students maximum (Project Ideas)
  • 10% self-tutorial + class participation
You will lose 10% each day for late projects. However, you have up to three "late days" for the whole course. That is to say, the first 24 hours after the due date and time counts as 1 day, up to 48 hours is two and 72 for the third late day. "Late days" can be used across projects or consecutively on a single one. This will not be reflected in the initial grade reports for your assignment, but they will be factored in and distributed at the end of the semester so that you get the most points possible.

Contact Info and Office Hours

You can contact the professor with any of the following:

Office Hours
  • Min Sun / Delta (台達館) 962 · Appointment via email
  • 趙浚宏 / EECS (資電館) 711 · Fri. 15:30 to 16:30
  • 鄭欽安 / EECS (資電館) 722 · Fri. 15:30 to 16:30

Tentative Syllabus

Lecture Class Dates Topic Slides Reading Extra Info (e.g., Homework/Exam)
1 F,
Sept. 14
Python tutorial
Git and Github
pdf1 homework 0 out
2 F,
Sept. 21, Video Lecture
Introduction to computer vision and cameras & optics
Light and color
pdf1
pdf2
pdf3
Szeliski 1, 2.1 (especially 2.1.5)
Szeliski 2.2, 2.3, and 3.2
3 F,
Sept. 28, Video Lecture
Image filtering pdf1 Szeliski 3.4
Szeliski 3.5.2 and 8.1.1
homework 0 due
homework 1 (hybrid image) out
4 F,
Oct. 5
Thinking in frequency
Image pyramids and applications
pdf1
pdf2
Szeliski 4.2
5 F,
Oct. 12
Edge detection
Interest points, corners, and local image features
pdf1
pdf2
pdf3
Szeliski 4.3
6 F,
Oct. 19, Video Lecture
Feature matching and hough transform
Model fitting and RANSAC
self-tutorial
pdf1 pdf2 Szeliski 9
Szeliski 7
homework 1 due
7 F,
Oct. 26, Video Lecture
self-tutorial Szeliski 4.1.4 and 8.4 project proposal due
homework 2 (image stitching) out
8 F,
Nov. 2
Panorama Stitching
Stereo and Structure from Motion
self-tutorial
pdf1 pdf2
9 F,
Nov. 9, Video Lecture
SfM & cere-solver
Feature Tracking and Optical Flow
self-tutorial
Szeliski 14
Szeliski 14.3.2
homework 2 due
10 F,
Nov. 16
Machine learning intro and clustering
Machine learning: classification
Szeliski 14.1 homework 3 (scene recognition) out
11 F,
Nov. 23
Recognition overview, bag of features
Large-scale instance recognition
homework 4 (face detection) out
12 F,
Nov. 30
Detection with sliding windows: Viola Jones and Dalal Triggs
Mixture of Gaussians and advanced feature encoding
homework 3 due
midterm project report due
13 F,
Dec. 7, Video Lecture
Modern Object Detection: DPM
Modern Object Detection: Selective Search
homework 4 due
homework 5 (deep classification) out
14 F,
Dec. 14
Deep Learning: introduction
Deep Learning: CNN
15 F,
Dec. 21
Deep Learning: recent work
Deep Learning: object detector
homework 5 due
16 F,
Dec. 28
Project presentation: 9 teams
17 F,
Jan. 4
Project presentation: 9 teams
18 F,
Jan. 11
Project presentation: 10 teams
F,
Jan. 18
final project report due

Project Proposal Format

- Max 4 Pages
- 3 Sections

  • title and authors
  • sec 1. intro: problem you want to solve and why
  • sec 2. technical part: how do you propose to solve it?
  • sec 3. milestones (dates and sub-goals)
  • references
- CVPR Format PDF, please!

Project Progress (mid-term) Report Format

- Max 4 Pages
- 4 Sections

  • title and authors
  • sec 1. intro: problem you want to solve and why
  • sec 2. technical part: how do you propose to solve it?
  • sec 3. milestones achieved so far
  • sec 4. remaining milestones (dates and sub-goals)
  • references
- CVPR Format PDF, please!

Project Final Report Format

- Max 10 Pages
- Title and Authors
- Abstract: short summary of the project with main results
- 5 sections

  • Sec 1. Introduction: introduce the problem you want to solve, expain why it is important to solve it; and indicate the method you used to solve it. add a concept figure showing the overall idea behind the method you are presenting
  • Sec 2.1. Review of previous work (i.e. previous methods that have explored a similar problem)
  • Sec 2.2. Say why your method is better than previous work; and/or summarize the key main contributions of your work
  • Sec 3.1: Technical part: Summary of the technical solution
  • Sec 3.2: Technical part: Details of the technical solution; you may want to decompose this section into several subsections; add figures to help your explanation
  • Sec 4: Experiments: present here experimental results of the method you have implemented with plots, graphs, images and visualizations
  • Sec 5: Conclusions: what's the take home message
  • References
- CVPR Format PDF, please!

Project Evaluation

  • Project Report Evaluation

    Your project report will be evaluated based on the quality of the writing, the clarity of your technical explanation and, overall, how well you get your message across. If you follow the structure above, you'll have good chances to do a good job. :)

  • Project Source Code

    There is no need to attach a print out of the source codes to the manuscript. Final source codes of your working program need to be shared with TA and the instructor on GitHub (open or private repos are fine); Source codes should be submitted before the project submission deadline date.

  • Project Presentation in Class

    The presentation must be at most 15 minutes long. Please refer to the detailed presentation guidelines below.

  • Presentation Format

    Your slides should consist of a title slide, followed by slides that discuss the following aspects of your project

    • Problem Motivation/Description
    • Technical Approach
    • Some Results
    Please do not plan on using more than 5 - 6 slides.

  • Project Presentation Evaluation

    Your team will be evaluated based on the clarity of the presentation, quality of the slides, how well you get your message across, and how well you handle the questions at the end. Note that the presentation can still contain ongoing/preliminary results; final results may be included in the final report. - We will use a peer-review system.