August 2012
Spotlight Summary by Roarke Horstmeyer
Spectral imaging by synchronizing capture and illumination
As digital post-processing continues to enable new camera designs and functionalities, the role of the camera flash has remained largely unaltered over the past several decades. In their recent article, Tominaga and Horiuchi propose an image capture method set on changing this paradigm. With a form of modified flash that can quickly illuminate a scene with many different colors, their camera setup seeks to extend conventional RGB imaging into the multispectral realm. Once a set of multispectral images is captured, effects like re-lighting a scene with any specified color of illumination, or classifying materials based on their reflected spectra, become possible with cleverly designed post-processing methods.
Previous multispectral image setups have typically relied on filtering the many colors that are reflected off of a scene. For example, one of the easiest ways to create a multispectral image set is to sequentially capture images of the same scene, each with a different narrowband spectral filter placed over the camera lens. More advanced methods can decrease the number of required exposures by combining many different types of filter over the digital sensor, or by inserting dispersive elements at an intermediate image plane. Unfortunately, all of these filter-based approaches suffer from two unavoidable consequences: object inter-reflections confuse attempts at assigning each object a particular spectrum, and optical efficiency is decreased during the spectral filtering process.
While others have also considered altering the way a scene is illuminated to solve these issues, Tominaga and Horiuchi’s approach offers two crucial advantages that will help push multispectral illumination into widespread use: speed and flexibility. Their high-speed multispectral projector (based on a digital micromirror device [DMD]) is capable of shifting the projected spectrum several thousand times per second, and their high-speed camera can capture up to 780 frames per second. Control over each pixel in the DMD likewise allows their projector to either emit one narrowband color at a time, or an arbitrarily shaped spectrum of colors all at once. The authors use this fine control over the projected spectra to solve for and display basis functions for a given target illuminant, leading to an even larger speed-up in acquisition time.
Besides using multispectral illumination for the article’s two proposed applications, spectral rendering and reflectance recovery, this form of high-speed project-and-capture system has potential use in several other disciplines. For example, quickly acquiring and processing the multispectral reflectance of a biological sample may help biologists identify hard-to-see features of interest. Or, florescent imaging disciplines, like the growing area of optogenetics, may take advantage of the proposed optical setup to selectively excite certain florescent proteins without exciting others. Finally, systems interested in identifying materials not just through the light they reflect, but also through the light they scatter and absorb may benefit from this fast and flexible approach.
Given previous trends within computational photography, it is clear that simple optical modifications can go a long way in assisting with the large amount of digital manipulation applied to our imagery and video. The synchronized camera-illumination setup proposed by this article offers a solid starting point for others to begin to explore novel uses for having a detailed spectrum of reflected light at each pixel in their images.
You must log in to add comments.
Previous multispectral image setups have typically relied on filtering the many colors that are reflected off of a scene. For example, one of the easiest ways to create a multispectral image set is to sequentially capture images of the same scene, each with a different narrowband spectral filter placed over the camera lens. More advanced methods can decrease the number of required exposures by combining many different types of filter over the digital sensor, or by inserting dispersive elements at an intermediate image plane. Unfortunately, all of these filter-based approaches suffer from two unavoidable consequences: object inter-reflections confuse attempts at assigning each object a particular spectrum, and optical efficiency is decreased during the spectral filtering process.
While others have also considered altering the way a scene is illuminated to solve these issues, Tominaga and Horiuchi’s approach offers two crucial advantages that will help push multispectral illumination into widespread use: speed and flexibility. Their high-speed multispectral projector (based on a digital micromirror device [DMD]) is capable of shifting the projected spectrum several thousand times per second, and their high-speed camera can capture up to 780 frames per second. Control over each pixel in the DMD likewise allows their projector to either emit one narrowband color at a time, or an arbitrarily shaped spectrum of colors all at once. The authors use this fine control over the projected spectra to solve for and display basis functions for a given target illuminant, leading to an even larger speed-up in acquisition time.
Besides using multispectral illumination for the article’s two proposed applications, spectral rendering and reflectance recovery, this form of high-speed project-and-capture system has potential use in several other disciplines. For example, quickly acquiring and processing the multispectral reflectance of a biological sample may help biologists identify hard-to-see features of interest. Or, florescent imaging disciplines, like the growing area of optogenetics, may take advantage of the proposed optical setup to selectively excite certain florescent proteins without exciting others. Finally, systems interested in identifying materials not just through the light they reflect, but also through the light they scatter and absorb may benefit from this fast and flexible approach.
Given previous trends within computational photography, it is clear that simple optical modifications can go a long way in assisting with the large amount of digital manipulation applied to our imagery and video. The synchronized camera-illumination setup proposed by this article offers a solid starting point for others to begin to explore novel uses for having a detailed spectrum of reflected light at each pixel in their images.
Add Comment
You must log in to add comments.
Article Information
Spectral imaging by synchronizing capture and illumination
Shoji Tominaga and Takahiko Horiuchi
J. Opt. Soc. Am. A 29(9) 1764-1775 (2012) View: Abstract | HTML | PDF