From Interactive to Participatory

The emergence of immersive documents, wherein unreality is perceived as reality by the ‘reader’, is a consequence of three converging technologies:

  • networked/mobile computing becoming pervasive
  • multimedia becoming multisensory
  • interactive becoming participatory

Alongside this synthesis, we can identify five tangential areas of interest, in which developments contribute to the facilitation of immersive documents. These are:

  • enabling technologies such as virtual reality, made popular by devices such as Oculus Rift
  • developments in graphic art and design ( see Diagon Alley brought to life for the immersive Harry Potter theme park)
  • new understanding of creative writing techniques underpinned by research into transmedia, literary/narrative theory, scriptwriting and game design
  • the desire by people (players/audience/readers) to participate, evidenced by activities such as cosplay, interactive gaming, web 2.0, participatory theatre, films, e-books and exhibitions.
  • a small but growing interest from the library and information science community on the implications of ‘immersive’ documents for our profession (collecting, indexing, retrieving, preserving, making available to readers or users)

The usual definition of the term ‘reader’ is expanded here to encompass the person or persons experiencing, or participating in, the unreal, immersive document. This may be by engaging with a transmedia story, by joining the audience of an immersive play, or by interfacing with virtual reality technology to enter a virtual world. The activity of ‘reading’ thus becomes participatory, so that the reader perceives the documented world as a reality, and posseses the ability to make choices in the story, and influence the eventual outcome.

The image above shows the cast from immersive theatre play “Venice Preserv’d” in action; drawing the audience into a timeless world filled with contemporary meaning. There are an increasing number of participatory theatre experiences on offer to those willing to suspend reality and join the cast (if only at the superficial level of donning a robe and singing along) but the number of attendees at such events demonstrates the lure of participation.

In talking about participation we are obliged to mention the whole spectrum of video games – hardly a new phenomenon, but one which is becoming increasingly sophisticated and to some extent, perhaps merging with immersive, transmedia e-books, so that the boundary between what is a game, and what is participating in a story becomes blurred. The educational opportunity for immersive games is already evident, in prototype products including: In Ulysses: Proteus and Dolus: Finding the Journal of Odysseus.

On the boundary between what is a game and what is a participatory ‘story’, writer Mike Jones offers the clarification that

“A Game does not need, nor have to have, a Story.”

He uses the phrase ‘interactive narrative‘ to further distinguish between immersive games and immersive stories:

“Interactive Narrative.. a term which can encompass a broad range of experiences where the audience is asked to play a role, to participate or to engage directly with character and plot through action. An experience that involves game-play but does so in the context and service of telling a story.”

Old Dramatic Principles in New, Interactive Narratives. Mike Jones 11/08/2014. http://www.mikejones.tv/journal/2014/8/11/old-dramatic-principles-in-new-interactive-narratives.html [accessed 16/10/2014]

I think this is helpful in attempting to understand what sort of things could be immersive documents, and the differences between them.

Immersive documents do not yet exist. Today’s emergent versions are still reliant on the suspension of disbelief – but technological advances fuelled by the popular desire to participate are moving us towards documents that allow us to perceive an unreal story as reality.

Below is a list of resources supporting the move from interaction (where the computer generated world is separate from the user) to participation (where the experience is more believable). They emphasize the strong desire in many people to escape reality and engage with a scripted world. This listing is in its early stages, and it will develop over time.

Further proof: our lovely #citylis student @MeganWaples, participating in “Venice Preserv’d”.

Venice Preserv'd + Meghan

Immersive Plays/Theatre

The Kindness of Strangers

Thomas Otway’s Venice Preserv’d

Punchdrunk’s The Drowned Man – London

Punchdrunk’s Sleep No More – New York

Immersive Films

What it’s like to shoot a feature film for Oculus Rift

Immersive Books/Texts

The Craftsman

Immersive Games

Blood and Laurels

In Ulysses: Proteus

Dolus: Finding the Journal of Odysseus

Immersive Exhibitions

David Bowie is

Tomorrow

Diagon Alley Theme Park at the Wizarding World of Harry Potter

Cosplay

At Hyper Japan

Japanese ‘Cosplay’ craze becoming popular in London

Immersive/Transmedia Writing

Will virtual reality reshape documentary journalism?

The Writing Platform

Marie-Laure Ryan on Narrative Theory

Mike Jones: Old Dramatic Principles in New Interactive Narratives

Enabling Technologies

Julian McCrea from Portal Entertainment talks about how audience’s facial data can be used in immersive entertainment.

Stretchable electronics could lead to robotic skin, computerised clothes.

On telling immersive stories

Neil Gaiman

I joined a packed house at the Barbican last Friday, to listen to the words and voice of Neil Gaiman [@neilhimself]. The audience sat in rapture for several hours, listening as he read his stories out loud, and I was reminded that storytelling is not only a powerful art, it is a furiously popular one. Gaiman’s darkish genre is not immediately appealing to me, but his performance, billed as ‘A revolutionary new concept of multi-media storytelling’, certainly was.

The multi-media component of the evening comprised a blending of the author’s mellifluous narrative, with projected drawings by the artist Eddie Campbell, and the ethereal acoustics of string quartet FourPlay. This seems rather low key when we are bombarded with announcements of increasingly realistic virtual reality applications all day, but it was effective enough to draw me into the world inhabited by the characters within the story ‘The truth is a cave in the black mountains’.

Good stories have always been immersive. Whilst the grading of a story as good or not is somewhat subjective, the aim of the writer is surely to draw the reader as close as possible to perceiving the tale as reality; to suspend disbelief, if only for a while.

I often refer to JK Rowling’s Harry Potter series to illustrate what is meant by ‘immersive’, as many readers easily agree the world depicted in the mind solely by the text persuades the reader that Harry and his chums really exist. Artwork pushes this further, giving the reader something concrete to imagine. Many stories, have achieved this, possibly since the earliest narratives were written down. It is arguable that even earlier, oral documents, stories told by telling or singing, are an effective mechanism by which to deliver the feeling of immersion.

The level of immersion offered by oral documents, or those consisting of text and drawings, is limited, however. The reader takes the role of a passive observer in the fictional world. The cinematic experience, film and video, provides a richer environment from which to fuel our feeling of immersion – but still we are unable to participate in any way.

If we add contemporary interactive technology to multimedia’s sound and vision, we are granted permission to enter the unreal world and perform actions which influence the outcome. With video games for example, we are able to contribute in some way to the world loaded into computer memory.

I have written previously, that the combination of pervasive networked computing, multi-sensory, rather than merely multimedia communication, plus participatory interaction, will eventually allow us to experience unreality as reality – to experience a story, a game, a film or any other scripted device as reality. Our disbelief will be suspended to the extent that we cannot distinguish between reality and the virtual world. Documents will offer us truly immersive experiences.

Immersive documents then, are the containers for a story, experience, fantasy, game, memory or idea, which allow the reader to perceive unreality as reality. As technology progresses, we creep closer and closer to the worlds portrayed in science fiction. The world of the holosuite for example.

It is not however, merely the enabling technologies which carry us along on the quest for ever more believable stories. It is something also of human nature. The desire to suspend reality, the willingness to enter fully into the unreal world is popular. We, as readers or users of immersive documents,  wish to participate in, or interact with the story, often in a way which allows us to  influence the sequence of events or the final outcome. In unreality, we may be offered a level of control unimaginable in real life.

The legacy of immersive documents undoubtedly stems from the pleasure of reading a good book. Before the spread of digital technologies some authors attempted to allow the reader a modicum of interactivity – to choose an ending to the story, either by selecting from pre-written options, or by voting by post. As technology advanced, more realistic interaction has been supported by interactive video or online gaming and by web 2.0 technologies leading to dynamic web pages and applications such as Second Life. In the cultural sector we witness the popularity of immersive exhibitions such as the recent David Bowie is shown at the V & A, immersive theatre, 3D cinema and the astonishing outpouring of content created by cultmedia fans, including simulated worlds and real-life cosplay.

The possibilities here are endless, and immersive stories can move beyond fiction and entertainment to include teaching and learning in realistic, yet safe environments. There is a dark side too, though. The unreal world may be somewhere we prefer to stay. Whilst Neil Gaiman came to end of his excellent dark reading and we all went home, the immersive documents just around the corner may be harder to switch off.

 

Sensing (#immersive) Spaces at the Royal Academy

sensingspaces immersive

The attractive colours scattered over this construction by architect Diébédo Francis Kéré reminded me of those tiny, coloured sugar balls used to decorate sweets and cakes. A visual feast then, the space was conceived to tempt visitors inside to meet each other as they navigated the narrowing between the two main areas. The multicoloured plastic straws (an industrial echo of real straw used to finish clay buildings in Gando) were provided to entice us to participate in the creation of the space – to leave our own contribution to the installation. There were some fine pieces of straw modelling, and even though I felt a little self-conscious, I wound my purple and pink bendy plastic into a decorative bag to allow a very small part of me to join in the dynamic.

Participation and immersion within architecture and design. Sensing Spaces at the Royal Academy until 6th April 2014.