CS 510/610: Computational Photography
 

Instructor: Feng Liu

  
Office: FAB 120-09      
Office Hours: TR 15:30-16:30  
Email: fliu@cs.pdx.edu  
Announcements
  • 04/25: Mid-term Project Presentation on 05/09
  • 04/02: Matlab for Image Processing Tutorial available
  • 03/28: Reading and project policies announced.
  • Although this is a graduate-level course, we encourage senior undergraduate students to take it. Please send an email to the instructor to discuss the enrollment.
Computational photography is a field where computer graphics, computer vision, optics, and photography come together to create high-quality pictures. This course will discuss computational techniques to overcome the limitations of traditional cameras and enhance their capabilities. This course will cover topics ranging from concepts of digital camera and photography to computer vision and graphics techniques for creating high-quality pictures, including high dynamic range imaging, panorama stitching, image segmentation & matting, video stabilization, etc.
General Course Information
Textbook  
We do not require any textbooks for this class. But the following two books will be very helpful.

Computer Vision: Algorithms and Applications, by R. Szeliski

This book is available online. Please download its latest version.

 Learning OpenCV: Computer Vision with the OpenCV Library,
by Gary Bradski and Adrian Kaehler.

This book provides not only a good reference to the OpenCV library, but also gives very good descriptions of a vast amount of vision algorithms.
Lectures
Acknowledgement: Many of the lecture notes were modified from materials for similar classes taught at other universities by Professor Yung-Yu Chuang, Fr¨¦do Durand, Chuck Dyer, Marc Levoy, Steve Seitz, Li Zhang, and Dr. Stephen Chenney and Richard Szeliski. Without their generous help, this class could not have been developed.
Date Topic Notes *Readings
W10: 06/06 Final Project Presentation II Enabling Warping on Stereoscopic Images
Yuzhen Niu, Wu-chi Feng, and Feng Liu
SIGGRAPH Asia 2012

Production Rules for Stereo Acquisition
Frederik Zilly, Josef Kluger, Peter Kauff
Proceedings of IEEE, 2011

Looking Beyond Stereoscopic 3D's Revival
Kirk L. Kroeker,  Communications of the ACM, 2010
W10: 06/04 Final Project Presentation I  
W9: 05/30 Stereoscopic 3D II Notes Compositing Digital Images
Tom Porter and Tom Duff. SIGGRAPH 1984
Blue screen matting
Smith, Alvy Ray and Blinn, James F. SIGGRAPH 1996
W9: 05/28  Stereoscopic 3D I Notes Lazy Snapping.
Yin Li, Jian Sun, Chi-Keung Tang and Heung-Yeung Shum. SIGGRAPH 2004
W8: 05/23 Video Stabilization II Notes Full-Frame Video Stabilization with Motion Inpainting
Matsushita, Yasuyuki and Ofek, Eyal and Ge, Weina and Tang, Xiaoou and Shum, Heung-Yeung. IEEE PAMI 2006
Content-Preserving Warps for 3D Video Stabilization
Feng Liu, Michael Gleicher, Hailin Jin, and Aseem Agarwala
SIGGRAPH 2009
Subspace Video Stabilization
Feng Liu, Michael Gleicher, Jue Wang, Hailin Jin, and Aseem Agarwala, ACM Transactions on Graphics, 2011
W8: 05/21 Video Stabilization I Notes
W7: 05/16 Matting Notes Compositing Digital Images
Tom Porter and Tom Duff. SIGGRAPH 1984
Blue screen matting
Smith, Alvy Ray and Blinn, James F. SIGGRAPH 1996
W7: 05/14  Graph-cut based Image Editing Notes Lazy Snapping.
Yin Li, Jian Sun, Chi-Keung Tang and Heung-Yeung Shum. SIGGRAPH 2004
W6: 05/09 Mid-term Project Presentation    
W6: 05/07 Image Segmentation Notes Normalized Cuts and Image Segmentation.
Shi and Malik. IEEE PAMI 2000.
(This week we only have one paper to read.)
W5: 05/02  Panorama III Notes Multiperspective Imaging
Steven M. Seitz and Jiwon Kim. IEEE CG&A 2003
W5: 04/30  Panorama II Notes

Szeliski book, Chapter 9
Recognising Panoramas,
M. Brown and D. G. Lowe , ICCV 2003
W4: 04/25 Panorama I Notes Image Alignment and Stitching: A Tutorial. Chapter 2 and 4
R. Szeliski. Foundations and Trends in Computer Graphics and Vision, 2009
Creating full view panoramic image mosaics and texture-mapped models.  R. Szeliski and H.-Y. Shum. SIGGRAPH 97.
W4: 04/23 Relighting II  Notes Recovering High Dynamic Range Radiance Maps from Photographs
Paul E. Debevec and Jitendra Malik, SIGGRAPH 1997.
High Dynamic Range Digital Photography,
G. Brown, RPS Journal, Nov. 2006
W3: 04/18  Relighting I Notes Interactive local adjustment of tonal values.
Lischinski, Dani and Farbman, Zeev and Uyttendaele, Matt and Szeliski, Richard. ACM SIGGRAPH 06
W3: 04/16  Light and Color Notes

Color2Gray: Salience-Preserving Color Removal.
Amy A. Gooch, Sven C. Olsen, Jack Tumblin, and Bruce Gooch. SIGGRAPH 05
Robust color-to-gray via nonlinear global mapping.
Yongjin Kim and Cheolhun Jang and Julien Demouth and Seungyong Lee. SIGGRAPH ASIA 2009
W2: 04/11 De-noise Notes Video Enhancement using Per-pixel Virtual Exposures
Eric P. Bennett and Leonard McMillan. SIGGRAPH 2005
A Non-local Algorithm for Image Denoising.
Buades, A., Coll, B., Morel, J.-M. IEEE CVPR 2005
Accelerating Spatially Varying Gaussian Filters
Baek, J., Jacobs, D. E. SIGGRAPH Asia 2010  
W2: 04/09 Filter Notes #Szeliski book: Chapter 3.2.
Hybrid Images
A. Oliva, A. Torralba, P.G. Schyns, , SIGGRAPH 2006
Bilateral Filtering for Gray and Color Images
C. Tomasi and R.Manduchi. IEEE ICCV 1998
W1: 04/04

Camera

 Notes Computational Cameras: Redefining the Image.
Nayar, S.K.. IEEE Computer, 39(8), 2006
W1: 04/02

Class overview

 Notes

Photography's Bright Future
K. Kroeker, Communications of ACM, 52(2), 2009
Computational Photography,
B. Hayes, American Scientist 96, 2007

* You might need to be on the PSU network or have an ACM/IEEE account to download some of papers.
#  You cannot select readings from a book for your reading assignment.
Reading Assignments:
Each week, you will read a few papers posted in the course web page. Besides, you need to pick one of the papers you read, write and send a short summary to fliu@cs.pdx.edu every Thursday afternoon before 500 pm.
In your email, please follow the following format for the email subject.
CS 510 Reading: You full name + Week number + Paper Title.

Your summary should be concise, no more than 500 words. You need to address the following five questions if appropriate.

1. What problem is addressed?
2. How it is solved?
3. The advantages of the presented method?
4. The limitations of the presented method?
5. Future work
 
Project:

You have the following three project options.

Option 1. Literature study
If you choose this option, you cannot work in group. You will read a rich set of papers on one of the topics in computational photography, and write a survey paper. This is an O.K. option for you to fulfill the course requirement. Personally, this is not very exciting.

Option 2. System development
If you choose this option, you can work in group. A good group size will be 1~3. Your project size should be proportional to the group size. You will be graded as a group. So be cautious when you form a group. You can pick an algorithm you are interested in and implement an easy-to-use system. This is a popular and interesting option.

Option 3. Research
If you choose this option, you can work in group. A good group size will be 1~3. Your project size should be proportional to the group size. You will be graded as a group. You can define a new problem and solve it. Or, you can develop a new solution to an existing problem. This is an exciting, but less popular option.

For all these options, a good way to look for topics is to look into recent SIGGRAPH/SIGGRAPH Asia/ICCV/CVPR/ECCV conference proceedings. We will also go over a range of topics in the first few lectures in our class.

Candidate Project Topics

 
Project Timeline:

04/16: Submit a project proposal to fliu@cs.pdx.edu. In your email subject, put CS510 Project Proposal + Your name. If you work in group, each of the group member needs to submit a copy, which can be identical. Your summary need to be concise, less than 300 words.
Programming Resouces
1. Matlab Tutorial for Image Processing and sample files, provided by Prof. C. Dyer and his TA Tuo Wang at the University of Wisconsin, Madison.