A new Microsoft Research project called Hyperlapse was revealed last month which is aimed to make all the GoPro videos captured with a helmet camera during activities such as rock climbing or bicycling more watchable than ever. You can’t watch the video of someone climbing a mountain from a first person perspective, when you watch it in 10x speed, it will become too shaky. At high speed-up rates, simple frame sub-sampling coupled with existing video stabilization methods does not work, because the erratic camera shake present in first-person videos is amplified by the speed-up. Microsoft’s Hyperlapse project tries to fix this issue with their own techniques.
Last month, Facebook announced Hyperlapse from Instagram, a new app to capture high-quality time lapse videos even while in motion. Hyperlapse from Instagram features built-in stabilization technology that lets you create moving, handheld time lapses that result in a cinematic look, quality and feel—a feat that has previously only been possible with expensive equipment. You can choose a playback speed that you like between 1x-12x and save the video.
Since both the projects have the same name and the output videos looked similar, people got confused about the differences between these projects. Microsoft Research team explained it in their blog as follows,
There are some fundamental differences between their and our technology, though, which we want to explain here.
Instagram’s Hyperlapse is similar to existing video stabilization algorithms in that it warps each video frame in order to remove slight camera shake. Unlike Adobe After Effects or the Youtube video stabilizer it does not rely on image analysis but rather the camera’s built-in gyroscope to determine the necessary amount of rotation for each frame. To avoid visible out-of-frame regions it zooms into the video to leave some buffer area for cropping.
This works well for sequences with only a little bit of motion, such as walking carefully around an object or filming out of a plane window. However, in less controlled situations, for example with a wearable camera, it breaks down. To see why, consider this hiking video:
Every frame in the left video was generated by warping just a single input frame. As you see, there are lots of out-of-frame pixels visible. So, existing methods would have to either stabilize less to follow the camera motion, or crop to a tiny common area in the center.
Our method is fundamentally different from previous approaches. It reconstructs a full 3D camera path and world model. This enables smoothing the camera path in space-time and generating an output video with a constant-speed camera, skipping over ‘slow’ parts of the input video, such as waiting times in front of red lights. Just as importantly, our method can fill the missing regions in the video above by stitching together pixels from multiple input frames. Thanks to these two innovations we can handle much ‘wilder’ input videos, such as climbing or riding.
Read more about Microsoft’s Hyperlapse here and watch the technical demo below.
