Storytelling in VR Masterclass with Mike Jones Recap 1


On Saturday, some of the Hive members were lucky enough to attend the “Concepts of Narrative VR with Mike Jones” masterclass hosted by the New Zealand Film Commission in Dunedin’s creative co-working space Petridish.

Narrative-VR-Masterclass-Dunedin

Part of the NZ Film Commission’s GPS 2026 series, the event was introduced by their Head of Talent Development (and BAFTA-nominated Director) Dale Corlett.

The aim?

To get the strong local film industry thinking about future developments.

After a quick introduction on Mike Jones – a race through an impressive list including writing, interactive projects for Warner Brothers, through to co-creator of the latest Virtual Reality (VR) detective thriller “VR Noir,” Mike jumped straight into a rollercoaster 2-hour ride through storytelling in VR.

What made this session different to any other was its focus, not so much on the technology of VR that we’re used to hearing, but instead on the creativity required to make such projects a success.

Mike asked the very important question:

How do we tell a story in these VR environments?

And then he proceeded to answer it…

Mike explained how we traditionally defined our writing skills in a platform-specific way. You were a writer of a books or a script writer. That is becoming increasingly difficult as the mediums diversify and now we describe writing more generally – as the ability to craft stories across platforms. In a very short space of time we have added new media to our platform options, but none of them have changed what fundamentally makes a good story.

Stories-and-Media-VR

As Mike explained, we do need to understand how the human brain makes sense of a story in its most basic form in order to understand how it will work in VR – that if you give any human two objects they will try to connect them. It’s when we “narrativise” like this that we create the story.

Therefore, trying to head into VR without thinking about the viewer’s ability to narrativise could lead to us tripping over the tech.

And there are serious challenges here.

Current VR writers and producers are essentially trying to build content for a medium that isn’t yet in the mass public domain. That has never happened before. And its either good, or very weird!

But Mike advised that now is the time to test approaches and “leave skin on the road.” Art is always a negotiation of creativity and commerce. We need to be ready. Investment in VR is already happening, and will increase as usage does.

And VR usage is increasing at a never-before-seen rate:

In fact VR unit sales of headsets is expected to reach 24 million by 2018. That’s from a base of zero a mere 24 months ago. No new technology in history, not even CDs, grew that quickly. So how many does it take to be considered a mass media? 500 million apparently. And as the research Mike quoted* estimates, that’s just 2 years away.

However, the growth of new mediums like VR doesn’t mean that we’re swapping one audience for another. It simply means that the audience are spreading themselves across more media. So VR isn’t taking over, it’s simply becoming another slice of the pie. This also means a potentially much broader audience for VR than those who simply migrated from gaming.

In order to tackle the challenges that VR content creation brings, Mike tells us to find the familiar in the foreign – what would be universal to any media? In essence, the VR issues are the same as those facing any small production screen company; repeatability, scalability and sustainability. Because every shot in VR is an FX shot, and that means a very expensive cost per minute.

Mike admits that VR is currently an experience like no other, with YouTube littered with hilarious videos of people’s first shocking experience. But he reminded us that we, as society, have been here before.

When the Lumiere Brothers premiered their arrival of a train short film in 1896 history tells us that people fled in panic at the sight of a moving train coming towards them on a big screen.

Shock-Cinema-VR-Headset

So when the “shock factor” of VR dies away, then we will get to the really interesting stuff. And writers will be expected to grab our attention, keep it, and fully immerse us in this new medium, if VR is to thrive…

In the same way we learned a new language when writing for cinema – the language of the edit – we must now learn a new grammar to approach VR creation.

The important questions to ask yourself before starting out, according to Mike, include:

What genres work better in VR?

How do we visually tell a story when the audience can look wherever they want?

Who is the audience?

Where do the audience want to go?

This last question is fundamental, says Mike, because with many other mediums, when choosing a story we will ask each other “What do you want to read/watch?” whereas in VR the first question is often “Where is it set?”

Mike helpfully took us through in detail the four universal story elements that should be discussed in the VR writers’ room:

Genre – the “What do you feel like watching?” question is a great place to start because audiences make emotional choices. So ultimately, how do you want the audience to feel? It is, according to Mike, inconsistency of tone in this case that will often be the downfall of any story. He recommends a thesaurus for this exercise. Words like “scared” are too broad when you consider that “shock” is a very different emotion to “dread”, etc.

Story World Design – this is platform agnostic and not VR-specific, however constructing a world does take on a literal meaning in VR. So Mike suggests a “world before plot” approach. He explained how every good TV show does this, particularly sitcoms that use worlds made up of forces that allow hundreds of stories and plot lines to play out. So ask yourself “What are the rules of this world?” Bear in mind that we don’t critic environments in other mediums the way we do in VR.

Plot / Narration / Story – Mike warned us that these terms are often used interchangeably but are, in fact, different things, especially in VR. For example plot is a series of events with cause and effect. That is not the same as narration, which is the telling of the plot (which could be told linear or completely mixed up). In VR the audience is part of the narration, just like in gaming, so that requires writing multiple modes of narration! Then we have story which is the culmination of the two – plot + narration. And story is created in the mind of the audience.

Escalation – Mike explained how lack of escalation is one of the biggest failings in current VR projects. Every story ever told is about stuff getting worse, otherwise we don’t care. Successful storytelling needs the audience to care. That means two of the most useful words for writers critiquing their own story development are “BUT” and “SO.” Narrative moves as the question the audience is asking changes. This element will transform a VR project from being a simple abstract experience to actually VR storytelling.

Storytelling-in-VR

Mike then took us through the key VR concepts:

Immersion and Presence in VR

Immersion requires the VR audience to be cognitively engaged or involved. We need the narrative immersion (so that we care) but we must also achieve spacial immersion – when the simulated world is perceptually convincing enough that we forget we’re wearing a headset. That means trying to get an audience lost in both the story and the visual experience.

Presence is the feeling of actually being there. This is unique to VR because no other medium can fool the brain the way VR can. We often get lost in stories, like books or films, but not lost in the actual medium. When we can combine the technological presence with the psychological presence, that’s where we create “AGENCY” – the feeling that a story simply wouldn’t happen without us being there.

So when planning a VR story it’s important to think about what stories naturally embody an active and meaningful role for the viewer?

Then add interactivity, and you take the VR story experience to the next level.

But interactivity is a spectrum.

So we have to decide how much control or influence we want give an audience. They already have the agency in VR to look around the environment, which is in itself compelling. That’s the minimum level of control we can give them in VR. As Mike explains, this may partly be a question that is answered by budget, but it should also be a key story question.

The way to answer this question on your own VR project, according to Mike, is to describe your audience using an active verb. Are they the author, explorer, or builder? Perhaps a solver, finder or guide? Replace passive words for active words. For example a viewer is passive, whereas a voyeur watches but tries not to be seen.

Once we know how much influence we want the audience to have, we break agency and interactivity down into three main segments – motivation (compelling the viewer to action), action and then reward (the pay off for acting). Miss out on any of these and the interactivity is merely button pushing. There needs to be cause and effect. It’s also important to give the audience clarity of the actions required, because once they are confused we lose the immersion. And the most powerful reward system? Giving the audience access to the next part of the story!

Creating-agency-interactivity-VR

In the second part of the workshop Mike talked us through:

Spatial Antecedents in VR

We looked at video games which followed radio, which followed theatre, which followed ancient architecture.

VR is now utilising many of the tactics from these primal sources to create and use space, primarily around three spatial dynamics:

  1. Motivation
  2. Proximity
  3. Direction

Mike explained that the spacial motivation used in VR is like that referenced by Norman Klein in his brief history of special effects – “gentle motivation posing as free will,” i.e. we are being led around an environment subtly in the same way shopping centre designers lead us to maximum exposure to increase spend.

With spacial proximity Mike referenced Edward T. Hall’s study of behaviour and non-verbal communication (dubbed “Proxemics”). This is the idea that distance varies with the level of human relationships. This creates powerful meaning as close-ups can instantly connect us to a character that comes into our intimate area of space. This, according to Mike, is simply choreography. In VR we can no longer frame shots to create intimacy or scale, so proximity is the tool we use.

Then we looked at spatial direction. Mike referenced one of the most common editing techniques in cinema, the J edit (whereby we hear the sound from the next scene before the cut). Psychologically this is a useful device in film because it mimics our real-world experience. While our eyes see in 200 degrees, our ears are always driving our eyes. In fact all our other sense are holistic, but the ears are the one we can utilise in VR storytelling.

This is architecture at a micro-level. Mike recommends a sound designer in the writers’ room to advise on creating these dramatic devices that will compel the viewer to move around and lead them through an environment that they technically have the ability to move freely in. This can be achieved with light too, as well as movement and colour contrast.

These are the skills people like theatre lighting designers have been using for decades.

All of this led Mike to rethink the writing process, as these three elements give us a new way to write for VR. He talked about putting it into practice on his latest project “Awake,” the culmination of three creators – a writer, a cinematographer and a director/designer.

Finally, when thinking about the key question as to whether VR can express an idea purely through space and presence (without character or dialogue) Mike claimed that there is a precedent.

He took us back to the arcade days of his childhood and the story behind Missile Command in 1983.

Inherent in this basic 2D game is a moral dilemma. Do you attempt to save all the cities and end up saving none? Do you save only the cities with the best chance of survival success? Or do you save the missiles first, the only line of defence against the enemy attack?

The original creator wanted to convey that nuclear war couldn’t really be won by anyone. Hence why each attempt finishes with the words “The End” and not the traditional “Game Over.”

Missile-Command-VR-Storytelling

A profound message communicated through a purely 2D spacial arrangement in a world with two forces…

It has been done. And it is our job to achieve it in the future of VR storytelling.

 

Some of the example VR stories Mike cited during his workshop:

Interactivity and role for the viewer – VR Noir by Start VR

A story that can only be told in VR – Awake

Achieving escalation – Sonar 360

An active role for the viewer – Collisions by acclaimed filmmaker and artist Lynette Wallworth

Creating agency from mystery – Remember by The Pulse, Sydney

Spacial motivation – Allumette the retelling of The Little Match Girl by Penrose Studios

Spacial proximity building relationship – Invasion! by Baobab Studios

Spatial direction through movement, light and contrast – Help! 360 Google Spotlight Story directed by Justin Lin.

 

You can follow Mike Jones on Twitter @MikeJonesTV 

 

*Superdata Report, April 2016

  • Natalia DM

    Thanks for the recap! It was a great presentation in Christchurch as well.