EE 6485 Computer Vision 計算機視覺

Fall 2016, Wed. 3:30pm to 6:20pm, Location DELTA台達209
Instructor: Min Sun

TAs: 胡厚寧eborboihuc@gmail.com
陳增鴻 paul516800@gmail.com
陳金博 stu94116@yahoo.com.tw

Course Description

Can computers understand the visual world as we could? This course treats vision as a process of inference from noisy and uncertain data and emphasizes probabilistic, statistical, data-driven approaches. Topics include image processing; segmentation, grouping, and boundary detection; recognition and detection; motion estimation and structure from motion. This class will also lead you to the discussion of applications applying state-of-the-art techbiques in recognition, detection, and video analysis.

The course will consist of four programming projects, one final project, and a few self-tutorial sessions (12 minutes for each team of 5 students). Please find information about projects and self-tutorial sessions in the syllabus.

Prerequisites

This course requires programming experience (mainly Matlab) as well as linear algebra, basic calculus, and basic probability. Previous knowledge of visual computing will be helpful.

Textbook

Readings will be assigned in "Computer Vision: Algorithms and Applications" by Richard Szeliski. The book is available for free online or available for purchase.

Resource

Awesome computer vision github link
Awesome deep learning github link

Grading

Your final grade will be made up from

• 60% 5 programming projects
• 30% final projects (includes proposal, midtern report, project pitch, project presentation, and project report). 3 students maximum (Project Ideas)
• 10% self-tutorial + class participation

You will lose 10% each day for late projects. However, you have three "late days" for the whole course. That is to say, the first 24 hours after the due date and time counts as 1 day, up to 48 hours is two and 72 for the third late day. This will not be reflected in the initial grade reports for your assignment, but they will be factored in and distributed at the end of the semester so that you get the most points possible.

Important Links:

• Github course repositories: slides, homeworks, etc.

Contact Info and Office Hours:

You can contact the professor with any of the following:

• Min Sun: sunmin@ee.nthu.edu.tw

Office Hours

• Min Sun, 台達管962, 電話 035731058, 時間：Tue. 2:30pm-3:30pm
• 胡厚寧和陳增鴻, 資電711, 時間：Tue. 3:30pm-4:30pm

Tentative Syllabus

lecture	Class Dates	Topic	Slides	Reading	Extra Info (e.g., Homework/Exam)
1	W, Sept. 14	Introduction to computer vision and cameras & optics	pdf,pdf2	Szeliski 1, 2.1 (especially 2.1.5)
		git and github	pdf		homework 0 out
	W, Sept. 21	No Class (Plz attend GTC)			Make up class in the following weekend.
		Image Formation and Filtering
2	W, Sept. 24	Light and color & Image filtering & MATLAB Tutorial	pdf,pdf2,pdf3	Szeliski 2.2, 2.3, and 3.2	homework 0 due
	W, Sept. 28	No Class (Teachers' day)
3	W, Oct. 5	Thinking in frequency	pdf	Szeliski 3.4	homework 1 (hybrid image) out
		Image pyramids and applications & self-tutorial	pdf	Szeliski 3.5.2 and 8.1.1
		Feature Detection and Matching
4	W, Oct. 12	Continue & Edge detection	pdf	Szeliski 4.2
		Interest points, corners, and local image features	pdf pdf2	Szeliski 4.2
5	W, Oct. 19	Feature matching and hough transform	pdf	Szeliski 4.3	homework 1 due
		Model fitting and RANSAC & self-tutorial	pdf2	Szeliski 4.3	homework 2 (image stitching) out
6	W, Oct. 26	*Panorama Stitching	pdf	Szeliski 9
		Multiple Views and Motiong
		Stereo and Structure from Motion	pdf pdf2	Szeliski 7
7	W, Nov. 2	SfM & cere-solver & self-tutorial	see above	Szeliski 4.1.4 and 8.4
		Feature Tracking and Optical Flow	pdf	Szeliski 4.1.4 and 8.4
		Machine Learning Crash Course
8	W, Nov. 9	Machine learning intro and clustering	pdf	Szeliski 5.3	homework 2 due & project proposal due
		Machine learning: classification	pdf pdf2		homework 3 (scene recognition) out
		Recognition
	W, Nov. 16	Sports Day (運動會). No Class
9	W, Nov. 23	Recognition overview, bag of features	pdf	Szeliski 14
		large-scale instance recognition & self-tutorial	pdf	Szeliski 14.3.2
10	W, Nov. 30	Detection with sliding windows: Viola Jones and Dalal Triggs	pdf pdf2	Szeliski 14.1	homework 3 due
		Mixture of Gaussians and advanced feature encoding	pdf		homework 4 (face detection) out
11	W, Dec. 7	Modern Object Detection: DPM &self-tutorial	pdf
		Modern Object Detection: Selective Search	pdf
12	W, Dec. 14	Deep Learning: introduction	pdf pdf		homework 4 due
		Deep Learning: CNN	pdf		homework 5 (deep classification) out
13	W, Dec. 21	Deep Learning: recent work	pdf1 pdf2		midterm project report due
		Deep Learning: object detector	pdf
14	W, Dec. 28	Invited Talk: Video Understanding Using Weak Supervision from Natural Language			homework 5 due
		Project Pitch 2 hours
15	W, Jan. 4	Project presentation: 15 teams	pdf
16	W, Jan. 11	Project presentation: 10 teams
	Th, Jan. 12				final project report due

Project Proposal Format:

- max 4 pages;
- 3 sections:

• title and authors
• sec 1. intro: problem you want to solve and why
• sec 2. technical part: how do you propose to solve it?
• sec 3. milestones (dates and sub-goals)
• references

- final format: pdf, please!

Project Progress (mid-term) Report Format:

- max 4 pages;
- 3 sections:

• title and authors
• sec 1. intro: problem you want to solve and why
• sec 2. technical part: how do you propose to solve it?
• sec 3. milestones achieved so far
• sec 4. remaining milestones (dates and sub-goals)
• references

- CVPR final format: pdf, please!

Project Pitch:

- Max 5 minutes (See videos in Kickstarter for inspiration); - Content:

• What do you want to do (problem statement)?
• Why do you want to do this (motivation)?
• What will the "ideal" results or applications look like? The results can be manually created just to show the idea. Don't worry if you can actually build a system to generate these ideal results or application.

- We will use a peer-review system.

Project Final Report Format:

- Max 10 pages;
- Title and authors
- Abstract: short summary of the project with main results
- 6 sections:

• Sec 1. Introduction: introduce the problem you want to solve, expain why it is important to solve it; and indicate the method you used to solve it. add a concept figure showing the overall idea behind the method you are presenting.
• Sec 2.1. Review of previous work (i.e. previous methods that have explored a similar problem)
• Sec 2.2. Say why your method is better than previous work; and/or summarize the key main contributions of your work;
• Sec 3.1: Technical part: Summary of the technical solution
• Sec 3.2: Technical part: Details of the technical solution; you may want to decompose this section into several subsections; add figures to help your explanation.
• Sec 4: Experiments: present here experimental results of the method you have implemented with plots, graphs, images and visualizations.
• Sec 5: Conclusions: what's the take home message?
• Sec 6: References

- CVPR final format: pdf, please!
You can look at one of the recent instructor publications (such as this) as an example.

Project Report Evaluation:

- Your project report will be evaluated based on the quality of the writing, the clarity of your technical explanation and, overall, how well you get your message across. If you follow the structure above, you'll have good chances to do a good job. :)

Project Source Code:

There is no need to attach a print out of the source codes to the manuscript. Final source codes of your working program need to be shared with TA and the instructor on github (open or private repos are fine); this file is due on the project submission deadline date.

Project Presentation in Class:

- The presentation must be at most 10 minutes long. Please see below for detailed presentation guidelines,

Presentation format:

Your slides should consist of a title slide, followed by slides that discuss the following aspects of your project:

• Problem Motivation/Description
• Technical Approach
• Some Results

Please do not plan on using more than 5-6 slides.

Evaluation:

- Your team will be evaluated based on the clarity of the presentation, quality of the slides, how well you get your message across, and how well you handle the questions at the end. Note that the presentation can still contain ongoing/preliminary results; final results may be included in the final report. - We will use a peer-review system.

Acknowledgements:

The materials from this class rely significantly on slides prepared by other instructors, especially James Hayes, Fei-Fei Li, and Silvio Savarese. Each slide set and assignment contains acknowledgements.

Comments, questions to Min Sun.