On Immersion and Storytelling

Technologists and technology advocates spend a lot of time talking up VR and 360 video. A common cause for celebration: immersion. 360 video is one of a few tools that will provide immersive experiences like never before, with realism like we've never seen before.

Put The Story First

Do we want immersion? If the goal is immersion and realism, how do we achieve this while telling effective stories? What types of experiences does this approach leave us with? Travel montage videos? Documentaries? News coverage? Writing in early 2017, most 360 videos are along these domains. Half of every 360 video I watch feels like the type of videos playing on loop at a Best Buy, to show off high definition televisions. Creators brainstormed ideas for stories that would benefit from realism and then created those stories. The end-goal is immersion.

Putting immersion before storytelling is, I feel, the biggest problem facing 360 video content today. The feeling of immersion is a tool, one that must be used to service the stories - not using storytelling to service immersion.

Now, my goal is storytelling in this medium. A full suite of tools, perspectives, and strategies - including immersion - will be required.

Going Further

I could end this section here, the fact that storytelling must come first is perhaps all that needs to be said. There are some other interesting notes to make on immersion and the conversation around it.

The Technology

Sadly, fully immersive technology isn't here yet. We are still, best case, staring at a screen a few inches from our heads and pivoting a camera around a view that doesn't quite match our head movement. We have sound - perhaps recorded binaurally, or spatialized.

Frankly, how well does the tech ever need to work before we cross some mythical threshold into full immersion? We have fully immersive experiences every day, they are our every day. The ability to 'enter another world' is given to us not through advanced mediums, but by our own minds and imagination.

The story, the content, the experience. Without these, it doesn't matter how good the immersive technology is.

Mixed Messages

We are sending mixed messages to the viewers. On one hand, we are trying to make them feel as "present" as we can, while at the same time we are trying to tell a story within a medium. These are not necessarily conflicting, but when they aren't, the storytelling potential is severely limited by the lack of creative freedom to distort time, space, and perspective.

Further conflicting, the viewer's brains are receiving more and more signals that they are "present" but ... they aren't. They are in their own environment with a screen inches from their face. The physiological side of things doesn't match up. Ultimately, the experiences are underwhelming.

How can we be sure to send the appropriate messages through the appropriate channels?

Is 360 Video A Failure?

Is 360 video a failure? Is it just a technological milestone on a path towards even more immersive storytelling? It will be if we treat it the medium this way.

So do we need to wait (again) until VR technology improves? No! Of course not. We must, as creators, be honest with the tools at our disposal and not oversell and underdeliver. We also must be honest with ourselves about these tools. Instead of scrying for a dramatic "fully immersive" end goal that doesn't even serve stories well, we should take a cold hard look at what the technology can do, and what we can do with this medium.

360 Video is Unique

This is a new and exciting space in between video games and filmmaking. There is interactivity without affecting plot and without the viewer feeling like they are being "tested". This space is without the performative bend of video games that makes winners feel great and losers feel bad. We have, at best, the ability to give the audiences greater intimacy, empathy, and connectedness to a story. The audience going through decision-making processes can give them moments of curiosity, wonder, and discovery that are impossible in filmmaking alone. Yet, we get to keep story, pacing, and dramaturgical control as filmmakers to deliver concise and deep stories with compounding filmic elements all serving a story!

If done well, we can capitalize on both of these attributes; if done poorly the audience is in for an uncomfortable experience that doesn't seem sure what it want's to be. The audience is too left unsure how they are supposed to react, and feeling like perhaps they have done something wrong. At worst, it can be even more disengaging than film OR video games.

Stories Are In The Mind of The Reader

Stories exist in the mind of the reader, as communicated by the storyteller.

The story, no matter how immersive, ultimately takes place inside the viewers head. They ask themselves questions about characters and motivations and wonder about possibilities. About conflict. About plot. Uncertainty and curiosity engage the viewer and empathy and morality allow them to relate to and support the plot emotionally.

We must communicate to the audience these cues. The angry character needs to be recognizable as angry, and the audience should understand why. If we want the most "realistic" stories possible, we must communicate effectively. Communicating the internal attitudes of characters, however, can't be done without some creative liberties. Stories that exaggerate, where the paths are exceptionally treacherous, the enemies swathed in shadows, and the heroes glistening in armor help the audience understand the unspoken elements of characters.

Stories have always been woven with their telling. Even reading a book to children, a parent may provide flair - doing voices, leaning forwards or backward, and deliver dramatic hand gestures. Storytellers get excited during the exciting times and distraught during the sad ones. No good story gets read stoically. Achieving "realism" above all else is a lot like a monotone and robotic delivery of words on a page.

A story where the audience understands character motivations is far more immersive than a story that aims to appear realistic, as the audience gets to enter the world of the characters in the audience's head. Their thoughts are not focused on details like environments or neat sound design. They aren't dissecting the plot or thinking about what they will eat for dinner. The audience is focused on the characters - be they fictional or not - and the motivations, the plot, the conflict. In other words: the story.

Conclusion

I hope I have repeated myself enough in this section. Story matters.

When planning 360 videos, or thinking about what is our there today, stop thinking or talking about immersion, and start thinking and talking about storytelling. Do both immersion and storytelling, and you might have something especially amazing and innovative. Without storytelling, we are left with shiny technology demo's that may impress some but leave most unfulfilled, and take no step towards future possibilities.