Orbiting 360 Motion Video in Action

Most of the 360 motion video you see today is filmed with the camera locked down in a static position. This makes a lot of sense. For one, it makes masking and stitching easier. And since a lot of 360 video is intended to be viewed in a VR headset, filmmakers are wary of doing any intense camera movement that might make the unsuspecting viewer becoming incredibly motion sick. As a result, experiments with moving 360 cameras have been limited to slow and steady dolly pans or elevator lifts.

But we think the format can handle more! With the latest technology, we think we can introduce more dynamic camera movement while also controlling for motion-sickness, masking, and stitch quality.

Inspiration

It all started with this clip. Using a pretty simple setup, the filmmaker was able to create a really stunning visual experience. And since the point of focus remained fixed on a central target even as the camera orbited around, it seems like this could translate to 360 motion video without being a completely jarring experience for the viewer. So…we put it to the test!

Test Shoot

Before we started strapping thousands of dollars of video equipment to ropes and swinging them around, we decided to work on a proof of concept in a more controlled situation. We used a Samsung Gear 360, an arm, and a swiveling chair at the office — no skis or lift tickets for this round! We filmed at 60fps — the maximum for the camera — so we could slow it down in post. Here’s what we started out with:

Right away, you can see some issues. The stabilization is completely off and the movement is insanely fast. We need to fix both of these issues to make this watchable. So let’s get to work.

Timewarp

First things first, we need to slow this down. The camera was rotating pretty slowly around me (if it was being swung from a rope it would probably be going twice as fast), and it’s still really intense. So let’s get this thing slowed down. First, we’ll reinterpret the footage to 24fps. Since we shot at 60, this will get us a good bit slower without applying any effects. That said, it’s still way too fast.

Our next move is to apply the Timewarp effect in After Effects. This gives us the ability to slow the footage down even more while using frame blending and smoothing to keep the movement from getting too jittery. We tested out a few settings and found that running the footage at 25% speed brought it down to a watchable speed.

Stabilization

While the speed is decent now, we still have the issue of stabilization. You can see that the horizon is bouncing around a lot in the shot. And while After Effects CC 2018 now includes Mettle’s 360 stabilization tool, that doesn’t actually give us what we need. We’re not just looking to stabilize the camera, we want to make it so that a certain point stays in the exact center of the frame the entire time. So, we’re going to have to get a little creative.

We started out by motion-tracking the subject, using the corner of the eye as our tracking point. That gave us a motion path that showed how much drift there was over the course of the video.

Now we need to do some scripting. What we want to do is take the motion from the Tracker and convert it to degrees of rotation on the 360 skybox. So, we apply VR Rotate Sphere onto the footage and added scripts to the “Tilt (X Axis)” and “Pan (Y Axis)” settings. I won’t bore you too much with the math, but basically it converts the pixels from the tracker position to degrees by dividing it by the comp width and height divided by 360. I also added an “X Offset” and “Y Offset” slider so I can move the center point around to match with the footage.

To make it simpler, I saved those scripts out as an After Effects animation preset, which you can download here if you want.

And just like that, our footage was stabilized! The corner of the eye remained in the exact center of the frame the entire shot. With a little color correction, the final rendered footage looked like this:

When viewed in a VR headset, the experience was…not bad! Obviously, the footage was low-res because of the camera we used, but as a proof of concept it worked great!

Next Steps

Our timing for this test couldn’t have been any better! Insta360, the maker of our beloved primary 360 camera, has just announced a software update for its portable camera that has built-in motion tracking and stabilization. The sizzle reel they released looks very promising, and their pocket sized camera and accessories seem perfectly suited for this kind of task!

We’re eager to get our hands on this and try it out for our next 360 motion video. We’ve already gotten some sneak peeks at what it can do in action!

Learning how to use bullet-time on the #insta360 pic.twitter.com/P5fnTmzaqM

— Lance Priebe (@lancepriebe) March 19, 2018

Interested in more 360 Motion Video topics?

Discover what’s rad, fad, and bad about the latest 360 video trends for 2018 or check out surprising facts about VR 360 video.