Spring 2011: MAS 132/532 Future of 3D Imaging: Capture, Displays, and Interaction

Class Times: Friday 2pm to 4pm
Location: E14-493
Instructor: Ramesh Raskar, Douglas Lanman

Office Hours: By appointment (Room E14-474)

3D movies, 3D games, and 3D medical imaging are at the forefront of modern entertainment and healthcare applications and share many common principles in terms of image capture, display, and interaction modes. In this seminar course we will survey modern imaging techniques. This year’s special focus on 3D imaging reflects that of current market forces; 3D displays, gesture-based interaction, augmented reality applications and stereoscopic cameras are becoming commonplace. We will explore how these technologies will mature in the coming years and how the shift to depth-sensitive imaging will affect both the technical and social aspects of imaging for entertainment, science, and healthcare. In addition, students will have the option to build their own imaging platforms, particularly those for 3D applications, by hacking provided off-the-shelf hardware.

Driven by the recent resurgence of 3D cinema, depth-sensitive cameras and displays are becoming commonplace in the consumer market. Introduced last October, Microsoft Kinect has already fostered homebrew gesture-based interaction for applications well beyond the intended Xbox 360 platform. Similarly, a wide range of consumer electronics manufactures have begun selling stereoscopic displays and have exhibited inexpensive stereoscopic cameras. Most existing 3D displays require cumbersome eyewear, such as the NVIDIA 3D Vision shutter glasses kit; yet, inexpensive, glasses-free 3D displays are imminent with the release of the Nintendo 3DS in February. Thus, the sustained consumer fascination with 3D has brought these technologies out of the laboratory and into our living rooms. Similarly, computational imaging approaches are dramatically extending the capabilities of existing 2D imaging systems in medical and scientific imaging. This course will explore how computational imaging will similarly revolutionize
3D imaging.

Lectures will be given by invited speakers as well as student led, presenting summaries of research and development areas (e.g., automultiscopic displays, active illumination for 3D scanning, gesture-based interaction, tomography and microscopy techniques). Periodically, instructors will provide state-of-the-art reports on similar sub-topics, as well as do-it-yourself tutorials on building homebrew 3D cameras and displays.

Students are expected to: (1) participate in weekly class discussions and (2a) write a survey article OR (2b) participate in a class project. Student teams will be given access to recent commercial hardware for the course project option.

Hardware to be discussed and available for (optional) projects:

Microsoft Kinect
Nintendo 3DS
NVIDIA 3D Vision
Dual-stacked LCDs (http://www.hr3d.info, http://www.bidiscreen.com)
3D medical imaging displays ( http://cameraculture.media.mit.edu/NETRA)
projector-camera systems (http://mesh.brown.edu/byo3d/source.html)
Range cameras (http://www.mesa-imaging.ch/prodview4k.php)
Mobile phones (Andoid and iOS devices)
Retinal imaging devices
Thermal IR cameras
Light field cameras (5x5 camera array)

Topics include:

We will survey the landscape of 3D imaging, displays, and interaction techniques and learn how to conduct imaging research.

Modern imaging devices: modern optics and sensor design
What will a camera look like in 10-20 years?
How to come up with new research ideas in the imaging domain?
Emerging imaging devices: optics, sensors, and computational imaging
co-design strategies
Computer Vision algorithms for 3D analysis
How to read an imaging research paper?
How to build your own 3D camera, display, and interaction platform?
Computational imaging in sciences and medicine
Emerging trends in 3D medical devices
How to give a research presentation?
What will be the popular forms of consumer entertainment in 10-20 years?
What will be in Photoshop 2030?
Mobile photography, using GPS and inertial sensors
Cameras in developing countries
Preparing for final project: How to write a research paper in imaging?
Security and Trust in imaging
Solutions for the visually impaired
Future products and business models

Stellar course website:
Spring 2011

Facebook page:
Future of Imaging

Course websites for previous years:

Spring 2010
Spring 2009
Spring 2008