Video stabilization algorithm using 3D scene modeling

Researchers at Microsoft Research have developed an algorithm for automatically stabilizing first-person video captured using a head-mounted or other video camera while traveling along a route. For example, when cycling or climbing.



Examples of stabilized videos can be seen here , in the same place the original (unprocessed) videos, for comparison. The source code and explanations on the operation of the algorithm are published. Soon, developers will release a Windows application so that everyone can try the technology on their own videos.

The developers argue that their result cannot be achieved using alternative stabilization methods. The method is based on the reconstruction of a 3D scene, after which the algorithm optimizes the “movement” of the camera along the route to eliminate vibration, and combines the frames pixel by pixel into a smooth video sequence.

For example, the illustration on the right shows the real route (in black) and the optimized route that the program for rendering the video generated (in red).

The result is the so-called hyperlapse video (named by analogy with time-lapse, slow-motion filming).