July 29, 2008 at 7:25 pmBinocular Vision and Stereopsis
I want to write about what I do in Tech. University of Tampere .
Stereopsis (stereo effect) - a feeling of the length of space and bumps that arise when observing real objects, viewing stereo pairs, stereo photographs, stereo images and holograms. Often referred to as "depth perception."
As you know (in any case, most of us have heard something about it), the image visible with the left eye is slightly different from the image obtained with the right eye. Thanks to what our brain is able to restore the "depth" of the observed scene. However, exactly how he does it, and how far from many people know how it is possible at all.
In 1838, the English scholar Charles Wheatstonediscovered (more precisely explained) the nature of 3-dimensional vision.
If we imagine the human optical system from two eyes with more or less parallel optical axes (parallax), it turns out that the difference in images (disparity) is just determined by depth. More specifically, disparity (or disparity ) is inversely proportional to depth (distance), i.e. for example, an infinitely distant point will be projected equally onto both retinas (disparity = 0), and a closely-lying point will be projected to completely different places of the retinas (large disparity). This is approximately depicted in a picture pulled off from Russian Wikipedia. (For more, see Disparity )
Despite the long history of stereo technology, in the last few years inaboutacademia has seen a surge in popularity for stereo-vision. With the invention and improvement of computers, it became possible to measure the depth of scenes using machines equipped with digital photo / video cameras. It can be argued that over the past decade, the number of scientific publications devoted to stereo-vision has increased exponentially. Very noticeable progress is in the creation of algorithms for restoring depth from a given stereo pair.
Naturally, where there are two cameras (stereo), it is not difficult for a computer to add twenty-two more to improve the quality of depth restoration (as many 3-dimensional scanners do), however, from a scientific point of view, stereo vision is the most fundamental (and hence an interesting) task since any multi-camera approaches can be considered as a set of several stereo cameras, and process them independently.
Moreover, it is believed that the human brain copes well with the task of restoring depth from two pictures (actually a very controversial statement), then computer programs (stereo-matching, dense depth estimation) should restore depth with a very good result.
There is a special site in the scientific community where different groups of scientists compare the achieved results (in assessing the depth) with each other: vision.middlebury.edu/stereo/eval
So progress in this area is very well tracked by the results table that has accumulated over several years. Here are a couple of results:
These are the results of the Scanline optimization algorithm (1D optimization using horizontal smoothness terms), 2002.
These are the results of the Double Belief Propagation algorithm, 2007 publication. As they say, feel the difference.
(Source: vision.middlebury.edu/stereo/eval )
(Note 1: Real ground truth and test picture frames from which depth estimates are obtained can be viewed on the same page)
(Note 2: Since in this framework it is estimated disparity, not distance, the light areas in the picture correspond to nearby objects, the dark areas correspond to distant ones)
As you can see, over the past 5 years, the quality of restoring the scene depth based on a stereo pair has almost come close to as much as possible. Nevertheless, there are still many places in this area where researchers need to make an effort. Which is likely to be done in the coming years. A further development of this area could be Scene Understanding, when the computer not only “sees”, but also “understands” what kind of objects are in front of it.
If readers of this blog will like the new topic, then I am ready to continue to raise topics on Digital Image Processing. In particular, I would like to highlight the topic of the theory (and maybe practice) of Image Denoising, Image Deblurring / focusing, etc.
What to read further:
en.wikipedia.org/wiki/Stereopsis
en.wikipedia.org/wiki/Binocular_disparity
ru. wikipedia.org/wiki/%D0%94%D0%B8%D1%81%D0%BF%D0%B0%D1%80%D0%B0%D1%82%D0%BD%D0%BE%D1%81 82% D1% 8C% D1%
vision.middlebury.edu/stereo
www.stereoscopy.com
scholar.google.fi/scholar?q=Stereo+matching&hl=en&lr=&btnG=Search
Wheatstone Charles, Contributions to the Physiology of Vision. Part the first. On some remarkable, and hitherto unobserved, Phenomena of Binocular Vision.
Stereopsis (stereo effect) - a feeling of the length of space and bumps that arise when observing real objects, viewing stereo pairs, stereo photographs, stereo images and holograms. Often referred to as "depth perception."
As you know (in any case, most of us have heard something about it), the image visible with the left eye is slightly different from the image obtained with the right eye. Thanks to what our brain is able to restore the "depth" of the observed scene. However, exactly how he does it, and how far from many people know how it is possible at all.
In 1838, the English scholar Charles Wheatstonediscovered (more precisely explained) the nature of 3-dimensional vision.
If we imagine the human optical system from two eyes with more or less parallel optical axes (parallax), it turns out that the difference in images (disparity) is just determined by depth. More specifically, disparity (or disparity ) is inversely proportional to depth (distance), i.e. for example, an infinitely distant point will be projected equally onto both retinas (disparity = 0), and a closely-lying point will be projected to completely different places of the retinas (large disparity). This is approximately depicted in a picture pulled off from Russian Wikipedia. (For more, see Disparity )
Despite the long history of stereo technology, in the last few years in
Naturally, where there are two cameras (stereo), it is not difficult for a computer to add twenty-two more to improve the quality of depth restoration (as many 3-dimensional scanners do), however, from a scientific point of view, stereo vision is the most fundamental (and hence an interesting) task since any multi-camera approaches can be considered as a set of several stereo cameras, and process them independently.
Moreover, it is believed that the human brain copes well with the task of restoring depth from two pictures (actually a very controversial statement), then computer programs (stereo-matching, dense depth estimation) should restore depth with a very good result.
There is a special site in the scientific community where different groups of scientists compare the achieved results (in assessing the depth) with each other: vision.middlebury.edu/stereo/eval
So progress in this area is very well tracked by the results table that has accumulated over several years. Here are a couple of results:
These are the results of the Scanline optimization algorithm (1D optimization using horizontal smoothness terms), 2002.
These are the results of the Double Belief Propagation algorithm, 2007 publication. As they say, feel the difference.
(Source: vision.middlebury.edu/stereo/eval )
(Note 1: Real ground truth and test picture frames from which depth estimates are obtained can be viewed on the same page)
(Note 2: Since in this framework it is estimated disparity, not distance, the light areas in the picture correspond to nearby objects, the dark areas correspond to distant ones)
As you can see, over the past 5 years, the quality of restoring the scene depth based on a stereo pair has almost come close to as much as possible. Nevertheless, there are still many places in this area where researchers need to make an effort. Which is likely to be done in the coming years. A further development of this area could be Scene Understanding, when the computer not only “sees”, but also “understands” what kind of objects are in front of it.
If readers of this blog will like the new topic, then I am ready to continue to raise topics on Digital Image Processing. In particular, I would like to highlight the topic of the theory (and maybe practice) of Image Denoising, Image Deblurring / focusing, etc.
What to read further:
en.wikipedia.org/wiki/Stereopsis
en.wikipedia.org/wiki/Binocular_disparity
ru. wikipedia.org/wiki/%D0%94%D0%B8%D1%81%D0%BF%D0%B0%D1%80%D0%B0%D1%82%D0%BD%D0%BE%D1%81 82% D1% 8C% D1%
vision.middlebury.edu/stereo
www.stereoscopy.com
scholar.google.fi/scholar?q=Stereo+matching&hl=en&lr=&btnG=Search
Wheatstone Charles, Contributions to the Physiology of Vision. Part the first. On some remarkable, and hitherto unobserved, Phenomena of Binocular Vision.