A Revolution in Images How Insect-Eyed Cameras Could Change Our Lives
New cameras with hundreds of tiny lenses have recently been overcoming obstacles that have frustrated shutterbugs since the dawn of photography. German researchers are now working to find new applications that could make these mini-cameras an almost omnipresent part of our lives.
First shoot -- and then focus! The news of this advance in photographic technology made waves last year when Lytro, a California-based start-up, introduced a digital camera that allows users to decide after the fact, on a computer, which part of a picture should be in focus. "Photographers will have one less excuse for having missed that perfect shot," the New York Times wrote.
The camera was the work of a team of developers under 32-year-old Ren Ng (his Chinese last name is pronounced "Ang"), whose doctoral dissertation at Stanford University was on digital light-field photography. Previously, light-field cameras consisted of dozens of individual cameras connected to a supercomputer. Ng managed to cram the entire set-up into a handy little square tube remotely resembling half of an old-fashioned pair of opera glasses. The camera comes with special software that allows users to focus the pictures at a later point on their computers.
Although it began as a sensation, Lytro's camera has ended up as little more than a gimmick. It's certainly amusing to shift a picture's focus back and forth, say, from a flower in the foreground to a building in the background. But technology journals, such as the German magazine c't, have criticized the camera's "hardly impressive quality," low-quality monitor and "catastrophic" white balance, as well as its lack of a flash, remote shutter release and standard data format -- all at a hefty price of $400 (320).
An Inconspicuous Revolution
Though it seems highly unlikely that such a device will revolutionize photography, the technology behind the Lytro camera could profoundly impact it. Its full potential could be revealed once liberated from the trendy trappings of a consumer-market camera and allowed to work in a smaller, less conspicuous setting.
The search for this silent revolution leads to the eastern German city of Jena, home to the Carl Zeiss company, where physicist Ernst Abbe first calculated microscope optics with mathematical precision around 1870. A boxy, purpose-built center on the city outskirts houses the Fraunhofer Institute for Applied Optics and Precision Engineering (IOF), one of the many research institutes belonging to the Fraunhofer Society.
Here, tucked away in dimly lit optics laboratories, scientists are working hard to create computer chips that can see. Small and as flat as a fingernail, these chips also have tiny metal legs that make them resemble millipedes. Invisible lenses make their surfaces appear blank.
Frank Wippermann, 37, holds one of these chips at arm's length, and his face -- with its high forehead and slim-framed glasses -- suddenly appears on a laptop connected to the chip. He then holds the chip just a hair's length from his nose, making his every pore visible on the laptop. No external lenses, shutters or shutter releases are necessary, just the light-field chip itself, an electronic component hardly bigger than a piece of confetti. Electronics and optics have merged.
Invisible but Everywhere
The fact that these sight-capable chips are practically invisible makes it entirely possible that they will be everywhere in a few years, watching us from MP3 players, toothbrushes and shirt buttons.
"The light-field principle is simple," says Wippermann, who leads a team of 10 developers at the IOF. "We make them like an insect's compound eye. Instead of one large lens, we use over 200 tiny ones arranged on the chip's surface." Each individual lens is no bigger than a grain of sand and captures an image just 40 by 40 pixels large, or only a small section of the object being photographed. Once the picture has been taken, these partial images can be assembled into a composite image.
For now, the photographic chips in Jena are just prototypes. They produce low-light images with the poor resolution of an old television. But the shortcomings of the tiny light-field cameras are offset by other enormous advantages:
- The chips are only half as thick as conventional cell-phone cameras;
- Since they have no movable parts, they are fast, sturdy and energy-efficient;
- Mass production could bring the price of a chip down to just a few cents; and
- The resulting images can be electronically focused after the fact or processed into 3-D images.
An Old Dream Fulfilled
For the first time, light-field cameras are breaking away from a basic principle that has changed little since photography was first invented in the 1830s. Conventional cameras project an image through a single lens onto a two-dimensional surface. Since all the light passes through the same lens, information regarding three-dimensional depth is lost.
Light-field cameras consist of not one lens, but hundreds. Each of these tiny lenses represents a slightly different perspective, which allows a computer to generate 3-D images out of those differences at a later point or to adjust the image's focus as desired.
This new technology fulfills a very old dream, since discontent with the rigidity of a camera's one-eyed gaze is nearly as old as photography itself. In 1908, Nobel Prize-winning French physicist Gabriel Lippmann came up with "photographies intégrales," or integral photography, a technique designed to create comprehensive, integrated photographs with the aid of light fields. This was known as plenoptics, meaning "seeing fully." Proponents of such "fully-seeing" cameras lacked just one thing: the computing capacity to make it work.
Now that modern computer technology provides more than enough of this capacity, the use of multiple lenses is booming. Artists such as David Hockney attempt to escape a fixed central perspective by using entire arrays of cameras, while drone researchers equip their aircraft with insect-like eyes that allow them to see in all directions.
Light-field cameras are also experiencing some success in industrial applications. Raytrix, for example, a company based in the northern German city of Kiel was selling light-field cameras long before Lytro arrived on the scene. Manufacturers use Raytrix's 3-D devices for applications such as testing the quality of components coming off assembly lines.
The IOF researchers in Jena report that they have also received plenty of interest, especially from the electronics industry. Apple, Sony and Intel have all made inquiries, they say. Shortly before his death last year, Apple CEO Steve Jobs also met with Ng, the founder of Lytro.
This interest stems from the fact that light-field technology also makes another form of miniaturization possible. "The thickness of a cell phone is largely determined by the depth of the camera," explains Andreas Bräuer, 60, who heads the IOF's working group on microlenses. "Our millimeter-thin microarrays can make cell phones even slimmer."
Then there's a second considerable advantage: These 3-D cameras are ideal for detecting hand gestures since they can recognize the difference between objects near and far as well as easily distinguish a person's hand from the background.
Bräuer also collaborates with the automotive industry, where miniature cameras could be used to track a driver's gaze. Here, Bräuer considers it an advantage that these little insect-eye devices don't look like conventional cameras: "With a traditional lens, many drivers would feel like they were being watched," he says.
From Capturing to Projecting
Bräuer and his colleagues owe this opportunity to reinvent how cameras see to another researcher who made a similar breakthrough, though with sound rather than sight. The light-field camera project in Jena is funded by the Fraunhofer-Zukunftsstiftung, the Fraunhofer Society's "Future Foundation," which in turn is funded by licensing revenue from the MP3 music compression format. Karlheinz Brandenburg helped create this format while working as a Fraunhofer researcher in the city of Erlangen 20 years ago. In 2011 alone, MP3 licensing brought in 125 million ($155 million).
Meanwhile, the researchers in Jena are turning their attention in the opposite direction, following the principle that technology capable of capturing light could be easily tweaked to emit it, as well. A projector just six millimeters (0.2 inches) thick is projecting a clip from "The Simpsons" onto the wall of the darkened laboratory. The principle is the same as for the insect-eye cameras. But in this case, rather than light sensors, hundreds of light-emitting diodes are behind the microlenses. The device is the brainchild of Marcel Sieler, a 27-year-old doctoral student who has already applied for seven patents.
The light-field projector's unique feature is its ability to project sharply focused images even on slanted or uneven surfaces, since the focus for each section of the image can be separately adjusted. A miniature projector can also be built to be so small and flat that it would fit inside a cell phone.
Translated from the German by Ella Ornstein
© SPIEGEL ONLINE 2012
All Rights Reserved
Reproduction only allowed with the permission of SPIEGELnet GmbH