Assembling A Story
I would like to present three ways to think about assembling a story in 360 video. I believe effective storytelling will take place utilizing all three of these perspectives, connecting shots and scenes, connecting audience gaze, and connecting audience mode and behavior.
Connecting Shots and Scenes
The first is the connecting shots and scenes. What arrangement of environments in what way best tells the story? This is the most 'traditional' approach (and probably the best way to start). Storyboarding the 360 video out and ensuring that there is narrative cohesion and purpose to each shot and scene. In 360, it is helpful to think of what environments will we be moving to, in what order, and why.
Connecting Audience Gaze
The second is where things actually get interesting and unique to 360 video. Building on Jessica Brillhart's work defining "The Hero's Journey" as she calls it, we make intelligent predictions of where the audience is looking and align the shots up in such a way as to engage the audience through shots. We can have multiple 'paths' through the story.
I build on this by saying we can give up completely, or partially, our knowledge or attempted knowledge of the audience's gaze, and regain it as (before) needed; switching between directed and undirected scenes. Getting the audience's gaze back is a matter of clever staging, blocking, movement, and so on.
Connecting Audience Mode and Behavior
This is where the power of immersive storytelling really comes to the forefront. Because the audience is in an active role, there is probably and active and definable behavior that we can identify as determining how the audience is looking around.
Are they searching for something? Are they following or tracking an object? Are they trying to understand the environment? Are they bored? Are they trying to subvert the filmmaker? Are they nauseous? Are they just looking straight ahead, too bored or incapable of moving a lot? Are they predicting an object's motion and giving it appropriate headroom? Are they only looking at the one character they care about? Are they trying to find little details?
The point is, the audience behaves because of reasons, and these reasons are - as any game developer will tell you - manipulatable.
I consistently stress connecting these motivations to the story and narrative, as the primary factor for manipulating the motivation. If the audience is engaged with the narrative, then we can tell our story.
Regardless, there are cinematic, theatrical, and architectural (rather, level/theme park design) tools at our disposal as well. Should the audience be searching all around the scene? Let's give them detailed, noisy environment that includes objects that begin in their field of view and extend out of it, for example.
When we tell our story, we want to be aware of our thread of connected motivations. Using points of interest to manipulate the audience's gaze is not the only tool, we have to manipulate their relationship (their 'mode' or controlling 'behavior'). Get the audience engaged with a search for an object. Make sure the audience cares about the correct character, so on.
Let's consider an example, a scene where the main character is hiding from the boogeyman under their bed. They run into the room first, then the boogeyman enters, looks for them, then leaves to search elsewhere. A tense horror scene.
We have a few different behaviors we could try for. We want the audience to feel the tension in the risk of being found, so let's aim for that. We can build up this tension by giving the audience more knowledge than the boogeyman. We can show the audience where the character is hiding, then when the boogeyman searches, we fear it might be obvious.
So first we film the character running and hiding under the bed. As they crawl under, their shoe falls off and is visible on the floor. The shoe will be our environmental indicator, the object of focus that will give away the hiding spot. Next, we cut to the doorway, where audio shows the boogeyman walking up to the camera, and looking past it, into the room. The audience turn's around and sees the shoe lying there, visible. Will the boogeyman make the connection? The audience probably is behaving like the boogeyman, wondering if the boogeyman character - seeing what they currently see - will make the shoe-bed connection, which the audience knows. But there is a different way for them to behave. The main character is hiding, won't move. The boogeyman, however, is where the next piece of action is. They will notice and find the character, or they will leave. So the audience is either focused on that shoe or watching the boogeyman for any movement/action.
We don't know which direction the audience is looking, both are valid. Let's cut to an under-the-bed shot, to the side of the character (and we cue some hushed panicked breathing for sound design). Thanks to this camera placement, we have the main character, the boogeyman's feet, and the shoe in question all in the same 90-degree field of view, we certainly have the audience gaze back. Linger on this shot, show the boogeyman taking a step forward towards the shoe, the main character holding their breath. We know the audience at this point, is like the main character, watching the feet of the boogeyman nervously.
The boogeyman stops and turns away, and we get to experience the same sigh of relief as the main character.
If we tried to block that scene out without thinking about why the audience is looking certain directions, we would probably end up focusing on the boogeyman (where the narrative tension/action lies) for the entire scene, as that's the most logical place to control the audience's gaze.
Instead, we consider how we can manipulate the audience's behavior. In this case by building a knowledge gap between audience/character, and using it to show the boogeyman's perspective and the character's fear.
For this work, I have not talked about playtesting much, because, in the effort of crafting a top-down framework for understanding 360 video, a bottom-up trial and error approach is tangential. Of course playtesting is extremely important and should be used by editors - showing their footage to naive audiences and seeing if it 'works'. Editors do this anyway.
For us, we need to go one step forward, and ask ourselves why the audience is looking around the way they are, and then make decisions on how to tackle, amplify, adjust, or deal with that behavior. It might be as simple as asking the audience (smart) questions, asking the audience to narrate their thinking while watching, or other playtest methods that this work does not have the scope to approach.