Our precious peepers are valued primarily for one crucial function: letting us see. But eyes can also be a source of information to those observing them, mostly by giving intuitive clues to emotions or intentions. And now two computer scientists from Columbia University have come up with a way to make use of a hitherto unexploited property of eyes: their ability to mirror the world around them.

Specifically, postdoc researcher Ko Nishino and Prof. Shree Nayar, codirector of the Columbia Vision and Graphics Center, have devised a system to capture and analyze the evanescent pictures displayed on our own little ocular movie screens. Their "corneal imaging system" seems at first rather prosaic: basically it involves using a high-resolution digital camera to snap a close-up of a face. The real action takes place when the image is downloaded to the computer: sophisticated software isolates the circular area around the iris called the limbus, where a film of tear fluid over the cornea reflects the world like a clear midsummer lake. From there it's possible to lift a wide-angle view of the person's surroundings when the shot was snapped.

Nishino and Nayar, who began the work in March 2003, quickly realized that the "spherical panorama" reflected in the eye is broader than what actually falls to the retina (and is subsequently "seen"). That means that if you digitally capture the scene in your eye at a given moment, it's possible to play it back and view what you missed if you focused your vision in a different direction. As for what you're actually looking at, the Columbia researchers, making use of the eye's anatomical characteristics and some carefully constructed algorithms, can calculate your "gaze direction" and identify what's actually hitting your retina. This potentially renders inoperable the "I didn't notice" excuse to a spouse when a foxy or hunky specimen passes. After working on this project, jokes Nishino, "I check myself when there's something I should not be looking at."

The potential uses go far beyond marriage management. The most obvious is security. Nayar, making the disclaimer that he wants no part of Big Brotherism, outlines a scenario where, using his system, you could get clues about the location of a terrorist by looking at a picture taken at his hideout. But one can also imagine surveillance cameras at security checkpoints that monitor whether people are excessively staring at the precautions. Less controversially, psychologists studying human reactions can know with certainty what people are looking at in a given moment. Computer-interface designers may use the ideas to allow software to provide you information based on what you're looking at on the screen.

Later this summer, the Columbia duo will present a paper to siggraph, the computer-graphics Woodstock, to show how the system could help digital moviemaking. When you examine the reflections in an eye, it becomes much easier to insert an object into a scene, or even replace a person with a digital stand-in, while precisely replicating the original lighting conditions.

Nayar contemplates exhuming the reflections from old photographs and viewing the surroundings from the point of view of the subject. "Can you imagine discovering what Martin Luther King Jr. was seeing when his photograph was taken?" he asks.

Eyes used to be windows to the soul. Now they're windows to the world.

Editor's Pick