Tag Archives: AI

Rife-App Creates Higher frame Rates 25x Faster!

A few months back we took a look at Dain app, and how it was able to use AI and machine learning to create in between frames from almost any source footage and create something that looked and felt like real footage taken with higher fps cameras.  The algorithm was so revolutionary that it took the world by storm, making older software that used re-timing from Adobe and others look antiquated and underpowered.  The Dain-App was great and it was a pay what you want App but had an Achilles heel. The software required a powerful Nvidia GPU with as much VRAM as you could muster to be able to convert footage and re-time it.  

The new Rife-App which is the direct successor of Dain App by the creator GRisk is up to 25x faster than the original, improves the algorithm, and by many examples betters it by creating more seamless transitions. The flow of frames is frankly jaw-droppingly beautiful, especially on low frame rate animation.   We estimate that Animation studios in 2D will eat this app up immediately, and even 3D animation studios could reduce their render times by calculating fewer frames and using Rife-App to increase them to 24p, 30p or 60p from a lower source like 20fps or 12fps.   → Continue Reading Full Post ←

Multiply Your Video Frame Rate with Dain-App !

Multiply Your Video Frame Rate

We got over 20 messages with essentially the same video sample in our inbox this week. They all touted the new interpolation from the DAIN experimental App or (Depth Aware Video Interpolation App)  which now analyses footage with a Neural network AI algorithm that crunches motion vectors and even what seemed impossible before “Object Occlusion” to generate higher frame rates from lower fps sources.  The technology is pretty fascinating and should be further improved by more training and samples over the coming years.

For stop motion animators, this is a complete game-changer as now you could animate with as little as 8fps and then interpolate to 30fps or 60fps with very little in the way of tearing and artifacting as long as the footage is well lit and objects clearly defined.  To make matters more interesting, it also analyses footage with shallow depth of field yielding impressive results.

Multiply Your Video Frame Rate with Interpolation or the “I” Word for Slow Motion Enthusiasts:

We visited the Interpolation topic in the past on our Fake Slow Motion article and concluded that then, the quality of interpolation while good was far from usable and you really could not compromise real high fps footage from interpolated versions except in very simple cases.

Now with DAIN technology, we have no choice but to re-visit the cases and analyze what it is capable of.  We looked at a few dozen examples and it is clear the technology has progressed forward so much that now stop motion animation, 2D Cell-based cartoon animation, and even 3D animated sequences rendered at 30p can easily be turned in higher fps increments yielding impressive and in some cases miraculous results.

We would like you to first watch the video below to understand what a depth map is and how the software in DAIN can create frames from nothing that look just like real ones.  A depth map will generate an approximated view of the world in a Lidar-Like vision representation to figure out to the best of the AI estimation where objects are in a scene according to their location close or far to the camera.

Depth-Aware Video Frame Interpolation by Wenbo Bao:

Even at 48fps from  12fps source, it is clear the technology in DAIN can yield impressive results even with heavy organic detail in the background including foliage. The software does an admirable job of estimating the relative position of objects in the video scenes.

AI使用フレーム補間アプリ DAIN APP byTALBOの実験室 Ch.: → Continue Reading Full Post ←

Canon EOS R5 Makes it Expensive to record 120fps 4k Video!

Canon EOS R5

Canon has now released the final specs and estimated release dates for the EOS R5 and R6 cameras. There is a lot of good advancement in features especially in video mode as what many believe is an afront to Sony and Panasonic who dominate video recording on ILCs.  The R5 can record 24fps and 30fps 8k video or 8192 x 4320 pixels in RAW quality which is a staggering 1TB of storage or 2600 Mbits/s for only 51 minutes of recording time. That means that every hour you will eat up through a full TB of your RAID backup storage. We see this mode being transcoded to an intermediate format like Blackmagic RAW or BRAW or Apple ProRes as soon as possible.

There is more bad news, on the 120fps 4k front there is no other option of recording but All-I which saves every frame independently in 10-bit 4:2:2 which is good for excellent quality but at the cost of 1,880 Mbits/s  which is 224MB/sec or 13,447MB / minute of recording time. Better get a ton of memory cards and hard drives ready!

EOS R5 Video Specs:

Video Rec Modes Raw 12-Bit
DCI 8K (8192 x 4320) at 23.976p/24.00p/25p/29.97p [2600 Mb/s]
H.265 4:2:2 10-Bit
DCI 8K (8192 x 4320) at 23.976p/24.00p/25p/29.97p [680 to 1300 Mb/s]
UHD 8K (7680 x 4320) at 23.976p/25p/29.97p [680 to 1300 Mb/s]
DCI 4K (4096 x 2160) at 23.976p/24.00p/25p/29.97p/59.94p/100p/119.88p [170 to 1880 Mb/s]
UHD 4K (3840 x 2160) at 23.976p/25p/29.97p/50p/59.94p/100p/119.88p [170 to 1880 Mb/s]
Full HD (1920 x 1080) at 23.976p/25p/29.97p/50p/59.94p [28 to 230 Mb/s]
H.264 4:2:0 8-Bit
DCI 8K (8192 x 4320) at 23.976p/24.00p/25p/29.97p [470 to 1300 Mb/s]
UHD 8K (7680 x 4320) at 23.976p/25p/29.97p [470 to 1300 Mb/s]
DCI 4K (4096 x 2160) at 23.976p/24.00p/25p/29.97p/50p/59.94p/100p/119.88p [120 to 1880 Mb/s]
UHD 4K (3840 x 2160) at 23.976p/25p/29.97p/50p/59.94p/100p/119.88p [120 to 1880 Mb/s]
Full HD (1920 x 1080) at 23.976p/25p/29.97p/50p/59.94p [12 to 180 Mb/s]

You read that right 120fps 4k video on the EOS R5 is only possible at the 1,880Mbits/sec data rate even in H.264 mode. Also interesting to know is the lack of 120fps Full HD 1080p or higher fps modes which are common on cameras from Panasonic or Sony.

 

Overheating what?

Yes, the R5 gets overheated after recording video on the following modes and times:

  • 8k RAW 30p/24p – 20 Minutes rec time
  • 8k NON-RAW 30p/24p – 20 Minutes rec time
  • 4k 120fps – 15 Minutes rec time
  • 4k 60p – 35Min Rec Time No Crop
  • 4k 60p – Crop Mode 5.1k Oversampling – 25 Min Rec Time
  • 4k High-Quality 30p- 8.2k Oversampling- 30 Min Rec Time
  • 4k 30p – No Overheating Limit

We do not find any of these limits to be unbearable if you shoot clips of a few minutes but for event shooters, this is not the camera for you for many reasons, chief among the rec time limit, and secondly, the massive recording space needed.  For slow motion, the limit of 4k 120fps at 15 minutes is probably more than you will ever need so that is not a problem. See the video below:

Canon EOS R5 the TRUTH about OVERHEATING!: Armando Ferreira

It costs what :-0?

Yes, the EOS R5 costs a staggering $3,899.00 US Body only and expected to ship at the end of July on the 31st.   The camera it stems from the venerable 5D Mark IV DSLR went on sale in September 2016 with a retail price of $3,499 (Now about $2,000USD), so the new R5 is now $400 USD more expensive at launch plus the new lenses are really expensive plus also needed to get the most out of this camera’s 45MP sensor and be able to get the 8 stop Sensor IBIS Stabilization with lens+body.  A good R5 kit will probably cost you around $10k for a three-lens setup with body and memory cards.  Having the latest and greatest from Canon or any other big brand will cost you, and this is a prime example of it.

4k 120fps Mode samples?

Gladly we were able to find 4k 120fps video samples from the R5 and they look very good along with full AF which is an excellent feature to have in slow motion recording see below:

Canon EOS R5 Field Test – with 4k 120fps samples:  Jarrad Seng → Continue Reading Full Post ←

Canon EOS R5 gets serious about high frame rates!

Canon EOS R5 Slow Motion

We were very skeptical earlier in the year when the R5 specs were just a rumor. Canon has not really been delivering powerful video features out of their Cinema line and we expected this new camera to be a crippled continuation of the trend.  We were not only wrong but now know that Canon is banking the future of the EOS line on cameras like the R5 with features that really put it in a class all by itself.

The original video darling was the EOS 5D Mark II which made it possible to use the EOS line of lenses with a video mode that delivered good enough quality for the web and some careful productions.  The camera had issues like severe rolling shutter and a tendency to moire and alias so bad that many shots were really ruined.  However, the excellent Canon color quality and lens choices made it so popular that it really forced companies to change the video options forever.  Now Canon intends to do as big a splash with the introduction of 8k video internally on the new camera.

Canon EOS R5 Released Specs:

  • 8K RAW internal video recording up to 29.97 fps
  • 8K internal video recording up to 29.97 fps in 4:2:2 10-bit Canon Log (H.265)/4:2:2 10-bit HDR PQ (H.265).
  • 4K internal video recording up to 119.88 fps in 4:2:2 10-bit Canon Log (H.265)/4:2:2 10-bit HDR PQ (H.265). 4K external recording is also available up to 59.94 fps.
  • No crop 8K and 4K video capture using the full-width of the sensor.*
  • Dual Pixel CMOS AF available in all 8K and 4K recording modes.
  • Canon Log available in 8K and 4K internal recording modes.
  • A Canon first, the EOS R5 will feature 5-axis In-Body Image Stabilization, which works in conjunction with Optical IS equipped with many of the RF and EF lenses.
  • Dual-card slots: 1x CFexpress and 1x SD UHS-II.

Yes, Canon grows up the video mode…

You read that right we have 8k 30p and 24p using the full-frame sensor width with no big crop “excellent for punch-in interviews” and also up to 120fps at 4k 10 bit 4:2:2 color which beats Panasonic’s 60fps limit on that mode. The R5 is a beast of a camera that is set to deliver on every front.

We love that it has the dual pixel AF on all modes with the latest intelligent AF modes including eye AF for people and animals. This is probably the most advanced and useful video AF that produces excellent results without much stutter.  The nightmare of focus pullers as their job just got harder.

What about 1080p HD frame rates?

Sadly Canon has not released any information regarding the HD modes on the camera but we have an idea of what that may be.  Since 4k will have 120fps in NTSC and 100fps in PAL, we expect at least to have 240fps in Full HD 1080p with probably no 720p modes due to redundancy.

We may see a sensor crop on full HD to allow such frame rates at 120fps and 240fps but we do not expect a crop on 24p, 25p, 30p and 50/60p which should be full frame.  Crops on higher frame rates are common as many sensors cannot scan that fast for the entire sensor area. We may even see some crop at 4k 120p which is not specified in the released information.   It will be interesting what Canon shows when the full specs are out including the price which we may see reach a range of $3,299 to $3,999 USD as an educated guess.

Interesting Developments:

We cannot be left unimpressed by the EOS R5, Canon has finally awakened from their video slumber in the midrange lines and is delivering a hard punch to the competition.

Sony and Panasonic will not stay put and allow Canon to take the praise without some hard releases.  We expect Panasonic to have a new S1 camera that may shoot 8k as a GH camera in a GH6 or similar that should be able to shoot 8k also. As for frame rates in HD resolution, we may start to see higher than 240fps on competitor cameras. Panasonic, after all, has had 240p on the GH5s and 180p on the S1 series with good results.

Sony may also release a new sensor with a higher memory buffer allowing super slow motion like their RX line of cameras.  Could we see the first Alpha camera with 480p or 1000p in reduced time recording modes? That will be a nice sight to see.

We commend Canon for pushing the industry forward with a Camera that may be just as important as the 5D Mark II was over 10 years ago, now as an 8k capable powerhouse with incredible quality and features like sensor/Lens Hybrid Full-frame IBIS stabilization and full-time Dual Pixel AF with machine learning.

We just need to get that Virus beat so we can go out and test these awesome cameras. More information on the R5 as it becomes available -HSC

amzn_assoc_placement = "adunit0";
amzn_assoc_search_bar = "true";
amzn_assoc_tracking_id = "hispeedcams-20";
amzn_assoc_ad_mode = "manual";
amzn_assoc_ad_type = "smart";
amzn_assoc_marketplace = "amazon";
amzn_assoc_region = "US";
amzn_assoc_title = "EOS Cameras At Amazon";
amzn_assoc_linkid = "736cd5d58ac3c9bb24beabc7eaf65dfd";
amzn_assoc_asins = "B01KURGS9E,B07H484HLT,B07NNS2YVV,B07J1Y3HQV";

Z Cam E2 Starts Shipping To Customers!

The Z Cam E2 has finally started shipping and is now available for direct order at their site. The camera caught our attention for its low price of just $1999 and the high frame rate features of 240fps 1080p and 120fps 4k.  It seems to be using the same 4/3 sensor as the great Panasonic GH5s but with extended features.

The camera company ZCam has also released some samples in a variety of scenarios so you can see what the camera is capable of. At first inspection, it is clear this camera is a clear step up from the Z Cam E1 which had some image quality issues. The E2 seems to go out of the box ready to produce class-leading imagery and offer some very nice high frame rate action. There are also some rumored comments that suggest the team is testing 300fps at 720p as an added bonus for future software.

Z Cam E2 Main Features:

  • 4K @120fps, 10-bit color
  • 4/3″ Dual Native ISO sensor
  • 13 stops dynamic range
  • 16 Stops in Extended DR Mode
  • Gigabit Ethernet for data, control & live streaming*
  • iOS app for control setting & live preview
  • Multi-cam synchronization**
  • HDMI Clean Signal Out!
  • Monitoring Required*

Z Ca, E2 Sample footage at 120fps:

Max Variable Frame Rate 120 fps for 4096 x 2160
120 fps for 3840 x 2160
59.94 fps for 3696 x 2772
240 fps for 1920 x 1080300fps 720p (Rumored Testing)

16 Stop Dynamic Range Test:

More footage from the Z Cam E2 Here at their official company Vimeo Page!

Z-Cam E2 Testing at Afton by EatsTooMuchJam:

It is clear the Z Cam E2 camera is capable of some very high-quality video output and could be easily used for cinema productions as a crash camera with remote access built in.  However, the camera’s lack of screen and irregular cuboid shape may put off some buyers and it may not be for everyone.  Add the fact that it shoots 4k 120fps and 240fps at 1080p makes it a desirable higher frame rate solution.  For under 2k this is one camera you may be able to afford as a production assist with also the quality needed for drone imagery without being too bulky. HSC

More info on the Z Cam E2 at their official website here!

and Facebook here:

https://www.facebook.com/zcamera/

NVIDIA Slow Motion Interpolation With AI Deep Learning Tech!

NVIDIA Slow Motion Interpolation

NVIDIA has been hard at work on the problem posed by high frame rate interpolation of video data shot on lower fps.  We have had this tech since the late 1990s with the advent of Twixtor and refined over the decades in systems like Twixtor Pro and Adobe’s Optical Flow in After Effects. You are still not getting real temporal detail data since the frames are created by extrapolating velocity and direction vectors plus pixel values between frames to get the result.

We explored this technique in our post on interpolation here and why it is no substitute from a real slow motion camera solution.  NVIDIA’s new method uses machine learning along with 11,000 videos to arrive at a more convincing result. Considering the relatively small sample size we can imagine a future where hundreds of thousands or millions of footage samples are used to generate near flawless interpolation. This technique takes some serious computation and data sets so as of now it is not really ready for the mass market but that could change with the cloud very soon.

NVIDIA Slow Motion Is New But Still flawed:

As you can see in the sample video below the artifacts produced by interpolation are very evident and more so when a fluid or fabric motion is introduced. The human eye can hide some of these in real time playback due to the persistence of vision effect and brain image processing but it is still quite apparent if you look at it witha  critical eye.

Transforming Standard Video Into Slow Motion with AI by NVIDIA:

There is no question this might be the best looking interpolation method we have seen to date but it is still not generating new information that has any scientific value. In other words, you can’t create something from nothing.  Nothing being the estimated values between two distant frames in time.  It sure is a marvel of computation and could really help in getting many more frames where detail is vast and artifacts suppressed but there is no real image captured from a live event.  If you record an explosion or fluid with this technique you will get what the computer estimates should be there and not what actually happened. Any rogue debris or physically distinct motion phenomena will not be there.  This technique is completely useless for education and scientific research.

That said the technique can make your slow-mo videos shot on your phone just a little more interesting even when shot at 30 or 60fps. As with any interpolation technique you can get better results the more frames you give the system.  If you shoot at 1000fps with a shutter of 1/4000 for example you will get the ability to interpolate down to 3k or 4k fps without much artifacting happening.  Then again if you shoot at 4000fps like what an edgertronic SC2+ can do you could interpolate down to 16,000fps without much in the way of artifacting.

We can certainly see a future in which you can upload your lower frame rate footage to the cloud and choose which frame rate you want it at within a reasonable range.   The cloud AI with the Machine Learning algorithms will get better with more and more videos being added to the collection. It is possible to do it with millions of samples instead of only 11,000 videos like the NVIDIA researchers were using in the lab.    The interpolation should get better and better as the computer learns from the added content.

It will also be possible to create footage from scratch by using video parts much like what Google did with Machine image learning to create new art.  What an interesting future it will be.

We are all for better interpolation but do not believe the hype when you are told you may never need a slow-motion camera again. In fact, temporal detail and nature recording cannot be interpolated to generate real information. So you better continue to use your slow motion camera and expect to get one more capable as technology improves and the price continues to lower. -HSC

Nvidia Slow Motion Interpolation Press Release on the Technology Below:

Link to the article here: https://news.developer.nvidia.com/transforming-standard-video-into-slow-motion-with-ai/?ncid=–43539

Transforming Standard Video Into Slow Motion with AI

June 18, 2018

Researchers from NVIDIA developed a deep learning-based system that can produce high-quality slow-motion videos from a 30-frame-per-second video, outperforming various state-of-the-art methods that aim to do the same.  The researchers will present their work at the annual Computer Vision and Pattern Recognition (CVPR) conference in Salt Lake City, Utah this week.

“There are many memorable moments in your life that you might want to record with a camera in slow-motion because they are hard to see clearly with your eyes: the first time a baby walks, a difficult skateboard trick, a dog catching a ball,” the researchers wrote in the research paper.  “While it is possible to take 240-frame-per-second videos with a cell phone, recording everything at high frame rates is impractical, as it requires large memories and is power-intensive for mobile devices,” the team explained.

With this new research, users can slow down their recordings after taking them.

Using NVIDIA Tesla V100 GPUs and cuDNN-accelerated PyTorch deep learning framework the team trained their system on over 11,000 videos of everyday and sports activities shot at 240 frames-per-second. Once trained, the convolutional neural network predicted the extra frames.

The team used a separate dataset to validate the accuracy of their system.

The result can make videos shot at a lower frame rate look more fluid and less blurry.

“Our method can generate multiple intermediate frames that are spatially and temporally coherent,” the researchers said. “Our multi-frame approach consistently outperforms state-of-the-art single frame methods.”

To help demonstrate the research, the team took a series of clips from The Slow Mo Guys, a popular slow-motion based science and technology entertainment YouTube series created by Gavin Free, starring himself and his friend Daniel Gruchy, and made their videos even slower.

The method can take everyday videos of life’s most precious moments and slow them down to look like your favorite cinematic slow-motion scenes, adding suspense, emphasis, and anticipation.

The researchers, which include Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, and Jan Kautz, will present on Thursday, June 21 from 2:50 – 4:30 PM at CVPR.