Our of our readers Chinito Pinoy has shared a pretty remarkable video with us that we would like to share. It is upscaling of sub HD resolution high-speed video at 240p, 480p and 960p from the Sony RX10 IV at 480 and 960fps and the A7S Mark III at 240fps from Sub HD to 4k UHD.
We got over 20 messages with essentially the same video sample in our inbox this week. They all touted the new interpolation from the DAIN experimental App or (Depth Aware Video Interpolation App) which now analyses footage with a Neural network AI algorithm that crunches motion vectors and even what seemed impossible before “Object Occlusion” to generate higher frame rates from lower fps sources. The technology is pretty fascinating and should be further improved by more training and samples over the coming years.
For stop motion animators, this is a complete game-changer as now you could animate with as little as 8fps and then interpolate to 30fps or 60fps with very little in the way of tearing and artifacting as long as the footage is well lit and objects clearly defined. To make matters more interesting, it also analyses footage with shallow depth of field yielding impressive results.
Multiply Your Video Frame Rate with Interpolation or the “I” Word for Slow Motion Enthusiasts:
We visited the Interpolation topic in the past on our Fake Slow Motion article and concluded that then, the quality of interpolation while good was far from usable and you really could not compromise real high fps footage from interpolated versions except in very simple cases.
Now with DAIN technology, we have no choice but to re-visit the cases and analyze what it is capable of. We looked at a few dozen examples and it is clear the technology has progressed forward so much that now stop motion animation, 2D Cell-based cartoon animation, and even 3D animated sequences rendered at 30p can easily be turned in higher fps increments yielding impressive and in some cases miraculous results.
We would like you to first watch the video below to understand what a depth map is and how the software in DAIN can create frames from nothing that look just like real ones. A depth map will generate an approximated view of the world in a Lidar-Like vision representation to figure out to the best of the AI estimation where objects are in a scene according to their location close or far to the camera.
Depth-Aware Video Frame Interpolation by Wenbo Bao:
Even at 48fps from 12fps source, it is clear the technology in DAIN can yield impressive results even with heavy organic detail in the background including foliage. The software does an admirable job of estimating the relative position of objects in the video scenes.
AI使用フレーム補間アプリ DAIN APP byTALBOの実験室 Ch.: