Schedule:
Lectures CSC-B41: Tue, Thu, 2-3:20 (Another more convenient time may be negotiated with students in first week of classes.
Labs in Computer Studies Building, CSC 2-35
Course newsgroup: ualberta.courses.cmput.613
Syllabus:
Central to Computer Vision, Computer Graphics and Image Processing
are the mathematical models governing image formation and methods
for processing and recovering information based on these.
Computer vision is about making interpretations of what's seen from
(possibly many) 2D images. Here we study 3D computer vision, which
focusses on how to make use of the spatial and temporal coherence
imposed by camera geometry to reconstruct a 3D geometric model from
e.g. a moving video camera, stereo camera rig or multiple views from
a still camera. As a contrast image processing, pattern recognition and
other image analysis often focus on 2D processing, while here we focus
on the 3D aspects. In Computer Graphics, one renders 2D images from
a 3D model, and the basic mathematics is the same, but the process is
a forward process (and hence easier).
The main feature of this course is a solid treatment of geometry to
reach and understand the modern Non-Euclidean (projective) formulation
of camera imaging. This theory found its form and dominated the
computer vision conferences in the past decade.
However, we warm up with some easier topics
in mainly 2D processing for tracking before tacking the more challenging
geometry. Finally, we cover recent developments in using variational
methods and PDE's to represent and recover surfaces, which is currently
a very hot topic in the imaging research literature. Applications
of the mathematical techniques are interspersed at appropriate
course moments.
- Preliminaries: Basic image processing
- Dynamic vision: We study how the temporal constraints in e.g.
video can be used to track object and form a coherent interpretation
of the motions.
- Geometry: Multiple view constraints in Euclidean, affine and
projective geometry imposed by different levels of camera calibration and
types of camera models. This allow us to mathematically understand the relation between the 3D world and it's projection in 2D images. Additionally we learn
how to use these to reconstruct a 3D scene model from several 2D images.
- Reflectance and photometry: The physics of interacton
between light and material allows the deduction of surface normals. In photometric stereo this is used to integrate shape. In general model refinement it is
incorporated with geometry into a photoconsistancy function and therefrom
derived PDE's to solve for a globally optimal 3D model of geometry and
reflectance from images.
- Other applications: computer vision, and recent cross-disciplinary
use in e.g. image-based modeling and rendering as well as vision based
robot motion control.
A preview of selected lecture material and slides can be found in my recent
IEEE VR course
Participation:
Course participation consisits of:
- Following readings on schedule, posing questions, and actively
participating in lectures.
- Four small warmup labs to get familiar with the lab environment.
Topics: Intro to matlab and image processing for optic flow. Capturing video and tracking motion. 3D Reconstruction from 2D images, photometric stereo.
- A project involving literature research and implementation of
a computer vision method consistent with the course (possibly applied to robotics or graphics, e.g as in image-based rendering).
For the project you will:
- Hand in a written proposal.
- Prepare one presentation on your topic for the class.
- Hand in a written final report at end of classes. (about 6-8 pages
in the format and style of a research report.
Examination:
You will present your findings and progress during the course, and
write a final report. Grading: 32% Labs (8% each), 30% Project, 8% In class presentations and class participation, 30% In-class exams. See calendar for the dates.
Course Literature
We will primarily use:
Multiple View Geometry in Computer Vision
Richard Hartley and Andrew Zisserman
Cambridge University Press, 2004.
Available in the university booksstore or from web book stores: Chapters , Amazon .
This is one of the best, yet most easily understandable texts on
the geometry of image formation. It is widely used at US and Europeean
top universities, and a standard reference on most researchers bookshelf.
Some of this material is also covered in:
Three other useful and recent computer vision books:
Ma, Y., Soatto, S., Kosecka J., Sastry, S.S. An Invitation to 3-D Vision
From Images to Geometric Models Interdisciplinary Applied Mathematics , Vol. 26, Springer-Verlag 2004.
Ma et.al's book is strong on the geometry chapters. It will most closely match the topics we cover in the course, but using a somewhat different mathematical
formalism.
David Forsyth and Jean Ponce Computer Vision -- A modern approach, Prentice-Hall 2003
Forsyth and Ponce has a more encyclopedic coverage of the computer vison field.