September 23, 2014 at 13:12Reconstruction of a 3D model of a moving face
Researchers from the University of Washington prepared an interesting presentation for the European Conference on Computer Vision (ECCV-2014). This is a technology for frame-by-frame reconstruction of a 3D model on video .
Using a video from YouTube, the program automatically builds 3D models of a high degree of detail for each frame.
This is a very impressive result, given the complexity of the task, because the facial expressions of a human face are very complex. To recognize emotions, it is important to see the exact position of the eyes, the bend of the eyebrows, wrinkles. The slightest error in such a 3D model is very striking.
The vast majority of other programs for 3D tracking and face digitization use the blend shape method, when the shape of an object changes, "flowing" from one state to another. The method of smooth deformations lacks just the loss of small details that are so important for the perception of the face. The authors of the new algorithm abandoned this approach.
On the other hand, although in the frame-by-frame reconstruction, each model exists “separately” from each other, but when reproduced at 30/60 frames per second, the result should be more realistic than in the case of smooth modification.
And further. Unlike other similar technologies, it does not require human participation in the test video. Instead, a large archive of his photographs is used in different lighting conditions and postures, using the optical optical flow method (3D optical flow). The author of the scientific work says that in our time a large archive of photos has been compiled for each person.
The program has successfully proved itself in different light conditions and on different facial expressions.
The work was presented at the European Conference on Computer Vision, which took place on September 6-12 in Zurich.
Scientific Article
Accompanying Material
The technology can be used in cinema, 3D printing, computer games and virtual reality, perhaps to restore the exact appearance and facial expressions of a deceased person from his photographs and videos.
Well, the most direct application of the technology seems to be the creation of more realistic 3D movies and the animation of facial expressions of characters in computer games.
Using a video from YouTube, the program automatically builds 3D models of a high degree of detail for each frame.
This is a very impressive result, given the complexity of the task, because the facial expressions of a human face are very complex. To recognize emotions, it is important to see the exact position of the eyes, the bend of the eyebrows, wrinkles. The slightest error in such a 3D model is very striking.
The vast majority of other programs for 3D tracking and face digitization use the blend shape method, when the shape of an object changes, "flowing" from one state to another. The method of smooth deformations lacks just the loss of small details that are so important for the perception of the face. The authors of the new algorithm abandoned this approach.
On the other hand, although in the frame-by-frame reconstruction, each model exists “separately” from each other, but when reproduced at 30/60 frames per second, the result should be more realistic than in the case of smooth modification.
And further. Unlike other similar technologies, it does not require human participation in the test video. Instead, a large archive of his photographs is used in different lighting conditions and postures, using the optical optical flow method (3D optical flow). The author of the scientific work says that in our time a large archive of photos has been compiled for each person.
The program has successfully proved itself in different light conditions and on different facial expressions.
The work was presented at the European Conference on Computer Vision, which took place on September 6-12 in Zurich.
Scientific Article
Accompanying Material
The technology can be used in cinema, 3D printing, computer games and virtual reality, perhaps to restore the exact appearance and facial expressions of a deceased person from his photographs and videos.
Well, the most direct application of the technology seems to be the creation of more realistic 3D movies and the animation of facial expressions of characters in computer games.
