Q&A: Evolving 360° Storytelling
Last Updated by
Editor's Note: The opinions expressed in this blog post are solely those of the respondents.
Over the past few years, 360° video, Virtual Reality (VR) and Augmented Reality (AR) have played a larger role in the way we consume our media. These unique experiences have surfaced across film festivals, are sometimes used in social media campaigns and the promotion of movie trailers, and have more recently taken off through game apps. Increasingly, people are demanding to be involved in the action, and with film especially, are looking for new ways to be participants of the narrative placed before them.
With the rise of these experiences as a platform for storytelling, PBS spoke with Senior Emerging Media Producer at NET Nebraska, Mary Anne Andrei, about the use of 360° video experiences in the film industry. Andrei takes us behind her work as the producer for “Under the Milky Way,” a 360° video experience and official selection in this year’s Online Film Festival, and discusses her vision for 360° in the future of film.
PBS: Can you tell me more about your background as a filmmaker and what sparked your interest in 360° film?
Mary Anne Andrei: I’m a storyteller, first and foremost. I’ve been creating content for NET Nebraska for the last three years, from traditional flat video and multimedia educational content to podcasts and 360° video. I welcomed the opportunity to explore 360° video as a storytelling medium about two years ago. It’s been fun and challenging to work in the 360° sphere. It’s forced me to reimagine and expand my own idea of narrative beyond the fixed frame.
Even though 360° viewers choose how they move around in the sphere and what direction they are looking, as the producer, I’ve still carefully selected the scene for them to explore and nested a series of scenes to help drive the narrative. While I believe it’s possible to tell a story in 360°, I’ve discovered that not every story can be a truly immersive experience.
"It’s been fun and challenging to work in the 360° sphere. It’s forced me to reimagine and expand my own idea of narrative beyond the fixed frame."
PBS: There must be a lot that comes into play during production when you’re working with the space around you, whether that’s the setting itself or the placement of the subjects featured in your film. Do you have any on set lessons or tips for your fellow filmmakers that you’ve gained from your own experiences learning to create 360° film?
Mary Anne: I was a photojournalist prior to my tenure at NET Nebraska. As someone who is practiced in framing shots, it was challenging to think in 360° while in the field, especially given that I was intent on determining if 360° was a viable story-telling medium. Our Emerging Media team began with the concept that we would shoot “spots, not shots.” The idea was that the 360° experience would immerse the viewer in a particular place. But that didn’t really work for video storytelling where the story emerges from a series of wide to close shots from various angles. It takes a long time to set up and capture a 360° shot, so we have far fewer shots with which to work. There’s no “run and gun” in 360° production.
We also learned over time that the best shots put the viewer in the middle of the action. In order to gather a range of shots, we use Insta360 One and now Insta360 One X cameras mounted on a selfie stick. We give the selfie stick to our subjects with instructions on the best way to hold the stick and aim the camera and what shots we’re hoping to capture, and then we get out of the shot. We have to find creative spots to hide from all of our shots, which is sometimes difficult and amusing when we have a larger crew. We hide behind rocks, and trees, and trucks, and lie down in tall prairie grass, and we’ve learned that it’s important to look where you’re sitting. During our “Under the Milky Way” shoot in the Nebraska Sandhills, I managed to sit on a cactus hidden in the tangled prairie grass. And we’ve come across a rattlesnake or two.
PBS: What do you see for the future of 360° and its place in the film industry?
Mary Anne: I would like to see 360° video enter the mainstream of digital storytelling because it’s such a unique and rewarding experience that allows viewers to engage more fully with a story. As we’ve pushed the boundaries and explored the limits of 360° storytelling, the camera technology and editing software have continued to advance. As camera equipment and editing tools improve, producers will be able to give the viewer an ever richer experience. 360° shots can also be flattened for use in traditional video. I think the film industry will soon catch on to this technique and its use in flat video will grow.
360° cameras like the InstaOne X may come to replace GoPro shots, because they give us greater creative control over field of view, and we can even change the camera angle in post-production. I think the audience for 360° content will not be restricted to headsets, but will continue to grow on YouTube and Facebook, and hopefully more web-based platforms will expand their support for 360° videos. At the same time, it’s not hard to imagine a proliferation of virtual reality theaters like the National Geographic Museum’s VR theater in Washington, D.C.
PBS: When viewing a 360° film, there’s a lot for the audience to take in as they immerse themselves in the environment you’ve created for them. How does this way of filming impact the pacing and direction of your storytelling?
"The best 360° stories use action to drive the narrative in each scene, so when the viewer looks around they have a lot to take in. We’ve found that it’s important to keep the narrative simple while featuring a subject matter that’s visually engaging."
Mary Anne: It’s important to acknowledge the limitations of 360° video — as with any storytelling medium, not every story is a 360° story. For instance, at this stage in the development of the medium, 360° video pairs best with less complicated narratives. The best 360° stories use action to drive the narrative in each scene, so when the viewer looks around they have a lot to take in. We’ve found that it’s important to keep the narrative simple while featuring a subject matter that’s visually engaging.
I don’t think it’s possible to capture a lot going on in every shot, but it’s a good goal. In order to capture a variety of shots beyond the fixed shot, we use the Insta360 One and One X cameras on a selfie stick for tracking shots and drone shots. We also use time-lapse imagery. This footage can help with pacing and perspective. And it’s nice to have a guide as you move through the sphere. Given that immersive media demands the full attention of the viewer — there’s so much to see that it can be difficult for some viewers to listen to the narration as they look around the sphere — we’ve found it’s important to reinforce what the narrator is saying. That’s why we use text graphics to reflect or add to the narration.
PBS: What type of equipment is crucial to shooting a 360° film, including any gear you may have used during your night shoots?
Mary Anne: We use Insta360 cameras: the Insta360 Pro, Insta360 One, and Insta360 One X (and we’ve just acquired the Insta360 Titan). The Titan will give us the best resolution at 11k, but it’s larger and less mobile. We also use a DJI Mavic Pro 2 drone with mounted Insta360 One and One X cameras that mount on the top and bottom of the drone for our 360° drone shots.
For “Under the Milky Way,” we were able to capture long night time-lapse shots using an Insta360 Pro camera, which we connected through a converter to a marine battery in order to run the camera for 12 hours in a remote location. From a nearby tent, we monitored the camera throughout the night. To capture the whole spherical soundfield in each scene and give the viewer a more complete immersive experience, we use a Sennheiser AMBEO VR microphone. As with any production, you have to gather the best possible quality audio. The layering and mixing of the audio and the addition of ambisonic can greatly enhance the audience’s experience as they move through the spherical landscape.
PBS: How does the film editing process differ when shooting in 360°?
Mary Anne: With 360° editing, every single shot is a special effects shot. We have to keep in mind stitch lines that can occur as the software assembles the spherical image from each individual lens. And as we set up our shots and interviews, we consider how we will cover our tripod and mic at the nadir of the sphere.
The interview process is also a bit complicated, as we need to make sure to capture a clean environmental shot (think the visual equivalent of “room tone”), so that we can mask the crew out of the image. Because masking is so easy, we also have ethical considerations. We don’t mask anything that is naturally in the shot and we take care to remove just the crew from interview shots. With “Under the Milky Way,” we judiciously used masking on top of the baseline effects to enhance the viewer experience. In most cases, such effects should be obvious to the viewer.
Finally, to edit and output these videos takes a considerable amount of computing power. In the process of creating about 30 360° videos, we burned through one souped-up Dell desktop (which fortunately was still under warranty). We’re taking bets on the life span of the next computer.