Augmented Reality – A Showcase What The Marketers Need To Know – October 28th 5:40pm – 6:30pm

“Is AR just like a lift-the-flap book?”
What value can AR offer for people and enterprises?
The session outlines various definitions of AR such as location based platform or image recognition platform and how the technology provides valuable insight for marketing. Exploring various case studies, we will discuss the current situations and issues for AR as well as its future.

Gary will be showing several of his Augmented Reality branding overview videos such as the recent Recognition – Business Opportunities but also one of three Location Based Augmented Reality Story developments in this space  called ‘Time Treasure” – a rudimentary, story rich LBARG (location based AR game?!) that MUVEDesign are currently story designing & coding for Android tablets. A hort 2 minute taster video embedded below…

Without giving the plot away, the structure of this game is quite straightforward. There are ten layers of time from 2050 back to 5000BC that you slowly penetrate following stories, clues and trails all based at POIs (points of interest, precise locations) around your city. The traditional MMOG talking-head quest and story givers are a unique part of this as well as a range of capture & loot quests that require you in some cases to do a little ‘real world’ grinding… ok not too much For me the challenge as always is about creating strong ‘call to actions’ and constructing a narrative backbone to make it worth your while walking and in some cases running around town! I will do a post when this reaches a full working pilot.

We will keep you updated in the next months of progress on this and the branded client games also in progress.

We have come a long way since cardboard cutout VRML back in the 90s. It is interesting to watch a revolution take place at the moment in Mixed & Augmented Reality and one particular subset, augmented reality. Gary produced a compilation in 2008 of various AR and MR area in this short video (at the bottom of this post you will find a transcript)

Another exciting area of AR is emergent game play. This example from a Japanese company called Geisha Tokyo Entertainment is your very own augmented reality Geisha Girl. It is interesting watching a story develop so imagine the potential now for story tellers or performers who can give their ‘participant’ audience the tools to bring the storyworld out into their personal space. I think in advance of the oft mentioned ’surround-o-film’ (yes holograms of Bond jumping around you in your front room, god help us!) this is a simple and elegant solution – given we all have webcams built into laptops, games consoles, mobiles etc etc: With this particular package you get two cubes and two sticks but imagine a cube for a range of characters or sets for your latest film or kids TV show, perhaps available as a premium.

Finally the combination of Augmented Reality and Social Media is probably the most obvious and compelling near term offering. Being able to layer the rich social web over the top of the real world adds real value. This video speaks for itself.

Below is the slightly futuristic transcript from my video compilation above, for those who can’t hear the narration. (BTW a better audio version available as a 50MB download here). Attribution for most clips appear in this post.

Is this reality? Is this fantasy? In 2008 the great transition truly began. Whereas a decade earlier we used to ‘log in’ to cyberspace, now we began to semi-permanently inhabit virtual space and the digital world started to leak out into everyday life. It began to infiltrate our shopping malls and became integrated with our everyday lives. Humanity started the relentless journey towards the natural future where virtuality became a greater reality,  and where digital fantasy and organic reality were inextricably intertwined.

Through the early 2000s pervasive wearable computing started to break down the walls that used to separate our virtual existence from our physical one. This same technology also allowed us to embed ourselves seamlessly into virtual worlds which is where we could truly experiment with future forms of entertainment, art, education and business.

In these new places we became the real time, living celebrity. Many of us became avatar stars, pixel gods – exalted as those linear film and TV icons. This became the only place for many. By 2007 hardcore gamers were already spending fifty five hours per week ‘inside’ these shared worlds and learned to forgot about the real world. This became the place where their most meaningful hours were spent. Everyone began to blend the two domains, combining them in new ways, driven by an invisible urge. Thus started the global meshing, of today’s mixed reality.

But in 2009 questions were asked. As in the many films that hinted at our transhumanist future, would the physical embodied self ever become redundant? Would we truly evolve as self contained, digital entities? May we as well be slumped in a chair with a cable plugged into our heads? Regardless, as the real world became inhospitable we all started to crave for the immersion and began moving our physical selves into these spaces. Even more than that we wanted our peers our un-virtual audiences to be able to share our experiences there. It became a contagious as the most poignant moments and memories were the ones in virtual space.

But we needed to look beyond these screens and into our own souls. Find the truth written along the ‘thin white line’ that separates analog from digital, person from avatar. This was the beginning. It always was. As far as the meshing or blending of reality and virtuality, in 2009 they were the equivalent of grainy black and white movies seventy years earlier. Our experiences were simultaneously part real, part virtual but it didn’t matter which was which anymore. Both were valid and they depended on each other. Those early parallel existences of twenty years ago started to move humanity forward, released our imaginations and we coexisted with our precious and ultimately fragile, physical selves. The great crossing had begun.

Tran-social-media-play [tran-soc-shuh-mee-dee-uh-pley] noun, verb

1 noun – a new form, a means of collaborative communication through play in constructed shared ‘media-rich’ environments

2 verb – taking part in game-like activity across and within online and offline social networks and media portals.

Can we truly create meaningful immersive media ‘experiences’ for others? Ones that last, are memorable, have impact & emotion and keeps the ‘experience players’ coming back for more? I am currently building services, working on papers and delivering courses on Experience Design and this post is a quick summation of some of the background thinking and good case studies.

Life is filled with so many exciting twists and turns. Hop off the straight and narrow whenever you can and take the winding paths. Experience the exhilaration of the view from the edge. Because the moments spent there, that take your breath away, are what make you feel truly alive. © 2000 Stacey Charter

Filmmakers and marketeers are clearly moving into #TranSocialMedia Play in a big way with a triple whammy at the moment of Star Trek, Terminator & Lost (fan) ARG’s and a raft of social media campaigns across other film and TV properties (some much better than others!). I have written about worlds immersion and cross-media design many times in the past but this post looks at the addition of social play into the mix – a permanent fixture.

Transocialmedia play design has historically been the domain of theme park builders, after dinner murder mystery writers, letterboxers or installation artistes – always based in the physical realm with the best examples reliant on group social interaction. But immersive play like this has also been with us since the first MMOGs social games inside fantasy environments and are another side of a triangle which also includes the new kid on the block (adolescent teenager) 2D social media. The main premise of the below diagram is simple – all encompassing transocialmedia design should be centered in the triangle and incorporate these three dominant forces. (there is a lot more to the diagram especially around hybrid services, but it will have to be another post)

TANGENT

A minor diversion but totally related to hybrid services, I was sent a tweet from Ms. NPRIL last night.

BettinaTizzy“Could Disney-Pixar’s 3D film up lift virtual worlds?” this blogger asks. http://tr.im/lEkm – What do you guys think of this? @GaryPHayes?

@BettinaTizzy Good 3D opens the door, inside? the anim film delivered as an auto story route in a home 3D engine where you can also explore!

The actual article was about exposing large numbers of non-gamers to 3D and whether this would turn them into ‘virtual world’ participants. But more than this I am always keen on drawing attention to media forms in collision and future looking mash-ups – as cinema becomes 3D and games engines become filmic (quality wise) why not deliver the film inside the game engine – not as cut cinematic 2D but the whole thing in 3D. You press play and the story arcs unfold around your avatar (perhaps your invisible perhaps your an extra – who knows!), voices play out from the characters and in some scenarios you can interact where the story makes sense. You as the film player can move around inside the environment the drama is taking place, certain action scenes you can jump inside characters and so on. That way from a screen entertainment point of view at least, you can also play/backchat with your social peers inside and around the story – now that would be an experience?. But this is really another whole post so I will continue…

BACK TO THE STORY

The ‘online distributed story’ courses to Directors and Foundation students at AFTRS often start with thinking about what is the nature of true meaningful collaborative play and what serves the story. What does one need to ‘create’ to turn passive story viewers into active story players? When we talk about play there are two key ‘spaces’ – within the screen and with-out the screen. Students already immersed in online 2D social space immediately don’t see the value of the social mediums – asking a fish to explain water. They soon see the potential though unlike many ‘non-social online’ creators who often try to merge crude old-school practise into fragile social media and like a bull in a china shop find it hard to engage. I use a variety of tools and exercises but the simple workshop template below helps story tellers plan and quickly understand two sides of the triangle – 2D social media and 3D online worlds.

But as we know screen media only tells part of the story which is why promotion agencies and enlightened storytellers alike are (to borrow from Dale’s Cone of Experience again) moving into memorable forms of contrived and direct purposeful experiences in the real world and realise that real stickiness will come from the activities at the bottom part of Dale’s Cone.

This is all leading to a few recent examples in the transocialmedia space – mostly campaign casestudies hence the marketing bias and hat tip to casemovies for some. The first is from 42 entertainment, demonstrating how to create purpose, community and elements of role play on massive scale as part of the lead up to the Dark Knight film last year.

Another major launch last year was for Halo 3 and as they were really promoting 3D virtual space it made sense that the unique elements of this transocialmedia offering (albeit low on the social side) was the creation of a large physical model to bring virtual into physical space and then promote the ‘memorial’ throughout urban environments. Halo 3 incidentally became the biggest entertainment launch of all time.

I often use these four simple levels of cross-media design that I wrote on wikipedia some years ago as a framework to differentiate using platforms as just distribution (CM1) at one end to a seamless contiguous experience at the other (CM4). The fan created Star Trek Alternate Reality Game linear summary of a typical user journey, certainly adheres to the description CM4 ‘experience’ and goes where other ARGs have not gone before!

Cross-media 1.0 – Push. Minor variations of the same content placed on different platforms. E.g.: A slight re-edit of the audio for podcast or a transcript diced and sliced for a making of type website and more of now, watch the TV show on the TV, web or the DVB-H or 3G mobile phone or an iPod video download.

Cross-media 2.0 – Extras. Content that is already being produced but not destined to be delivered and repackaged onto other platforms so it appeared to be ‘original’ or at least specially created. More about the wonders of a good content management system than the true potential of cross-media.

Cross-media 3.0 – Bridges. Where most innovative 360 thinking is currently rooted – where the content placed on the other platform is critical to staying in touch with the narrative or service.The narratives tease you towards investigating or moving (via a bridge) to another media form/platform.

Cross-media 4.0 – Experience. To some extent this is producer ‘hands-off’- in that they have created an environment, much like a game, that the participant/s ‘lives’ inside of, following their own path, personalizing the experience. A cross-media 4.0 property is at its height a creation, a collaboration with the audience across many devices, which evolves and grows a life of its own.

As an interlude, an example that only uses two sides of the triangle that I had to include which demonstrates the power of a contrived, controlled physical event that then massively propagates across 2D social media – yes it’s Antwerp Train Station does Sound of Music.

An example of traditional ARG design that fails to use two sides of the transocialmedia triangle and will get lost in the noise is the rather stalled Skynet series of videos that were being used to launch the new Terminator movie. It does have an interesting Twitter element at http://twitter.com/skynetresearch that is now spilling out into Facebook and beyond – worth keeping an eye on. “Skynet Research is dedicated to eliminating human error.”

I had to add this example for two reasons. It is very clever marketing first and foremost morphing the meaning of a basic food product into one of the scariest browser ‘environments’ on the web – also it combined the two sides of the triangle 2D social and 3D world by creating the browser world – Hotel 626 as real horror experience. Many users were actually too terrified to explore this space – not bad for Dorritos!

Transocialmedia play design is still in its infancy as all sides of the triangle above evolve. Social networking is constantly moving towards being more real time as twitter growth demonstrates. 3D worlds are becoming much more integrated with real and 2D social space as well as becoming a far more ‘representative renderl’ of the real world which itself is being invaded with highly sophisticated mobile and pervasive computing devices. The real challenge at the moment is avoiding gimmick and surface superficiality but developing deep well threaded stories that afford a deeper more investigative engagement. I have noted the design of recent ‘media plays’ to be rather linear, find clue A that leads to clue B and so on vs much more non-linear quest. Many also fall into being over authored, complete and don’t allow contribution or collaboration (although that will happen regardless somewhere else if you haven’t allowed for it). One of the biggest challenges is to mature the form.

There are many who see transocialmedia play as purely marketing, there to sit on the coat tails of existing well known brands (such as AI, Halo, MacDonalds/Olympics) vs being of themselves for themselves. Admittedly there are a few of these scattered around but the player base is usually much lower. To finish a small, mini ARG transocialmedia play I designed last year called the Old Forest – which had a good mix of 3D mashed with physical world and a team social component. The story was fictional, looking at witches, spells and fires – you can find more detail about this and other mARGS on the Laboratory for Advanced Media Production wiki.

There was a rather sad epitaph to this mARG as exactly a year later the whole town of Marysville was tragically burnt with great loss of life in the worst Bushfires in Australia’s history. Truth is far stranger than fiction.

Appendix

For reference here is a abridge copy/paste from a post on an immersion post from 2006 – looking at the factors that enable immersive experiences.

SCALE – of the experience. The size of the screen and the amount of story world to explore has enormous impacts on immersion – as well as the detail of individual objects within the ‘world’. One of the reasons cinema will exist for a long time is that the large dark room filled with people is a captivating environment. Now, imagine a cinema where the image is a locked off-shot, of a shared world and all the audience are controlling and representing different characters engaged in a common goal or story. Ummm.

SENSES – goes without sayiing that the amount of senses that are engaged by an experience gives it most potential to immerse. Now as I have said before we dont need to consider full immersive reality rather make sure as well as intellectual and emotional engagement you consider sounds and the grammar of visual. Probably forget about touch, taste or smell for the moment – leave that to the porn industry to work out.

SERENDIPITY – how the world or show you are watching has elements of surprise. As mentioned earlier the more scripted and formulaic the less immersive. People only watch a film for the fifteenth time, I believe, because they strangely hope that there may be something different OR they are peeling back the layers, looking at minute detail and looking way beyond the basic narrative.

STORY – does the narrative engage. This is obvious, if there is nothing for you to be drawn along by (even your own story in some cases) then you will switch off.

PERSONALIZATION – Hence the title of this blog. How much can you minutely affect the world and yourself in it? How much will the world reflect you for being there? Most importantly, how much of your real world personality can you bring with you into the experience.

RESONANCE/CHOICE – How much control or agency do you have over the experience? Are your actions permanent and seen by all? Can you really do and say what you want – freedom of choice. True resonance is like a virtuous circle, you do something and there is a response that forever changes the environment. Like real life. The pushed media of TV, radio, cinema has zero resonance, it all happens in your head. Which is why stories ‘have’ to be based on life’s shared drama. In truly interactive models your actions have impact and will reaction will take place.

TEMPORALITY – How real time does the experience feel? Scheduled TV never feels real time – the only successful shows in the future will be live events, music, sports, live news etc: Everything else has a dubious future in the scheduled world. MMORPGs feel real time when you are in them because of all of the above. Ones that have scheduled events or require you to invade or fight at a certain time are more about story than true immersion.

ESCAPISM – or ‘play’. This goes back to my earlier point about the reason for play and associated spirituality. Is it as much about escaping reality or constructing ideality?

Does the representational nature of these ‘experience’ worlds mean so much more subconciously than endless souless advertisements on TV, or another episode of a soap, or fomulaic hollywood film? Does selecting an identity that is impossible to achieve in real life become a most powerful addictive escape? I suspect all of the above. In terms of building ‘play’ – it should be as fun making it as doing it. I have mentioned before that sometimes authors of experience get so lost in the creation process they forget someone has to watch, play or take part in it! Then it is much weaker an experience.

OK the folks at University of Teeside will probably not be calling to get me to write their taglines yet what they are doing with Meta-Mole is pretty adventurous and most importantly critical at the moment to glue the discontinuous metaverse. The Meta-Mole in simple terms from their press release is:

The Meta-Mole, created by the Centre for Design in the Digital Economy (D-LAB) based within the University’s Institute of Digital Innovation, will ultimately be a dedicated searchable online resource for the 350 plus virtual worlds currently existing on the Internet.
Philip McClenaghan, Deputy Director of D-LAB explains: ‘We were analysing virtual world platforms and realised that there doesn’t appear to be a comprehensive service offering to list and compare key data for major 2D and 3D environments. This surprised us considering the current popularity of virtual worlds. We intend to fill the gap with the Meta-Mole.’

The metaverse as we know is the entity that forms when all virtual worlds have useable interoperability, you are able to move inventory between them, have a common profile (to a degree) and be able to communicate across worlds – even teleport between them (Habbo to vSide anyone!). At the moment it is all about walled-gardens, locked in economies/commerciality/currency and propriety tech – basically the metaverse doesn’t yet exist, it is disconnected. So to build a dedicated VW search engine that will trawl and spider as many as “350″ worlds is an ambitious vision – down this long and winding road. The benefits particularly to business and education is obvious (finding consumers or interest groups) but also making many social virtual worlds, more social – finding a long lost friend in IMVU whose into football, then setting up a match in Football Superstars, followed by a bit of post match role playing in Second Life – all ‘fueled’ by one search engine – means social activity will increase. Sadly the Meta-Mole is only digging around a few lower level worlds presently (see below) and I would encourage everyone to support such an initiative -  as long as it isn’t sold to a commercial entity who start advertising all over the search pages!  I would also suggest to the team they look at the open API’s of the big 2D social networks also – migration from 2D to 3D will be far quicker if the 2D (FB, MySpace, Twitter)  folk could actually sample the wonders of 3D social space

“Meta-Mole.com is a comprehensive database of Virtual World environments designed for public, corporate and government use.

Created by d|lab at the University of Teesside in 2008, Meta-Mole.com provides users with a total solution that includes:

• comprehensive search facility
• generalised and detailed technical information for a wide array of platforms
• images and streaming video media
• dynamic matrices for comparative analysis of environment functionality

The Meta-Mole.com application has been built from the ground up as a unique service offering for Virtual World users, developers and researchers to increase the profile of the market, ensure environment/consumer compatibility and enable new development of Virtual World markets through corporate education, relationship building and continued innovation.”

The advanced search, is not very functional or all encompassing yet, but suggests a lot about where they are heading with this. It will need to be pretty generic and high level to be able to map & interrogate elements such as scripting support or development tools. One thing it may bring about is a wisdom of the crowds ‘standard’ – just think for example if (on the development side) 8 out of 10 worlds allow importing from Blender or uploading of AIFF stereo sound then the other 20% might get busy on their backend! Lets hope so. The other key area is being able to link in existing out of world databases, especially from a eCommerce perspective the likes of SLExchange (a big out of world catalogue for Second Life consumers) and where searching for a world that lets you drive an Aston Martin is made easier for Mr Mole as databases such as SLEX are already half way there.

The Meta-Mole will initially be released as a Beta version focussing on 3D virtual world platforms. Forterra, Blink 3D and Twinity are among those who have already uploaded their details. Future developments will cater for all platforms, including 3D, 2D and MMO environments. Platform developers are able to participate in the Meta-Mole by contacting developers@Meta-Mole.com or visiting www.meta-mole.com.

For more information contact:
Philip McClenaghan
Deputy Director (D-LAB)
Tel: 0044 1642 738103
Email: p.a.mcclenaghan@tees.ac.uk

Rather than just referring to the many standard press articles about the growing number of companies or formats that are setting up in Second Life (the media’s favourite MUVE at the moment) I have been ‘living it’ so to speak. My inworld blog JustVirtual provides an inworld perspespective, my avatars POV, of close encounters with the outside world’s ‘brand immigrants’. They are indeed seen that way by many residents, not disimilar to any alien brand entering a real ‘developing’ world. The recent companies include NBC, Philips Design, IBM, Reuters, Endemol and others that suggest that the next generation of entrants are moving in, with a far more robust and serious vision than the first wave who were just pushing product.

I can’t break the 4th wall in my other ‘fictional’ blog, I know strange, so I thought I would comment on two recent events in Second Life in 3rd person. The first is the preliminary days of the Virtual Big Brother and the second is Second Life’s most attended inworld event, the switching on of the Christmas lights in Rockefeller Plaza sponsored by NBC.

Big Brother Begins

With these new entrants we are seeing a move to second gear, a shift in the maturity of the platform. Not technically (it still falls over an awful lot) or from a business model perspective (subs are still in the low 100k’s vs nearly 2 million registrants) but a web 3.0 paradigm shift – using the environment for quite practical and engaging services. I personally use Second Life (and other MUVE’s) for educational and creative purposes so I am probably already ahead of the curve. But onto to show. Big Brother selected its final 15 contestants (pic below) and I covered much of the lead up in this post. Most of the ‘housemates’ are women, I suspect because the producers are mostly male (I wonder how many of the females avatars women have a real world male driving them? We shall see.) I was actually on the last final 60 shortlist, but being able to commit to 8 hours a day online for the whole of December was impossible for me, so I didn’t pursue.

A couple of friends are in there and already it is really interesting to see how the virtual Big Brother is becoming even more compelling than the real one! As well as the traditional backstabbing, personality differences or supportive housemate interactions (remember this is real people talking/communicating with other real people) all involved are are allowed to communicate to and lobby the many hundreds of passing/visiting ‘audience’ members. This ‘conversation’ is very engaging for those involved, the audience is resontating with the programme, or rather with the ‘experience’ – because this is no longer pre-packaged or controlled reality ‘video’.

For the audience inworld this is belonging to, and being part of the ‘game’. Lillani Lowell (my current vote and friend leading up to this event) gives an view down this particular rabbit hole for those who are not in Second Life (or likely to be) in her insightful updated blog from inside the house (the equivalent of the diary room I suppose – but here we get to see all entries). The odd but also thrilling part of the experience is being able to go and talk to the ‘stars’ of the show, whenever you want. They are there 8 hours a day ready and willing to ‘interact’ with you, albeit just to get you on their side perhaps, secure your vote, still it beats, hands down, an SMS into the void that we get with most so called ‘interactive’ shows. It will be very interesting to see how Endemol adapt to and learn from this process, this is a really exciting experiment as games/tv and online social neworks collide. Will they copy elements from this evolving format out into the rather stale ‘real world’ version? Perhaps not, as this is after all a self contained hybrid form which is best left to grow and learn to stand on its own feet.

Above a group of loyal fans of Lillani (a talented scripter and builder) discuss the challenges ahead – in this Big Brother the housemates make things, complex 3D builds for charity. It is easy to appreciate the benefits of a this very diverse audience being used for these kinds of social experiments, game/tv formats. This is web 2.0 meets ‘TV form’ meets ‘games’, all evolving before our very eyes. It is also about game psychology, potentially interesting passive video generation and definitely about immersion for those involved. This particular version of the ‘show’ is thankfully nothing like the ‘real’ big brother, but it needs only the brand really to garner interest. The exciting thing is, it can and will go a lot further and I suspect there will be many other TV ‘formats’ and channels licking their lips. This is a really cheap way to pilot and user test web/tv/game shows after all. The ironic thing is though that this will likely gradually become the format for the masses over the next few years rather than the return to the traditional linear video route.

A Virtual Christmas

The next event is one that starts to show the way forward as regards ‘merged media entertainment’ as I call it. The NBC virtual christmas was synchronised with the real event in the real New York, there was a virtual camera inworld streaming the virtual event onto the web and there was real TV news reports covering the inworld going’s on plus – a live band playing into the environment. My inworld post ‘Virtual Christmas with NBC’ gives an insight into the experience with far more images, for the 1000+ avatars across 18 or so duplicated sims – second lifes largest inworld event.

So TV, web 1, 2 and 3, virtual world, live concert and outdoor all mixed and mashed together. I found myself checking nearly all my blog categories for this one! Another important part of this event was the sympathetic branding in the environment. It wasn’t in your face, it felt part of the experience particularly as the incredible build of the Rockefeller plaza and tower was copied down to the finest detail. Aimee Weber and Bedazzle Design are now experienced developers who themselves are old residents and have empathy with residents. This is critical to the future of these spaces. Better films are made by filmmakers who love watching film, games by games players, music by musicians who love to listen to a variety of sounds and virtual worlds by those who inhabit them. No brainer really.

Above we can see the branding which didn’t spoil the experience for those waiting for the switch on of the christmas lights. The over representational build, the vane attention to duplicating every detail of the real world is not to my ‘particular’ taste but I can appreciate the ‘craft’ and stupendous effort. I think the real reason we are witnessing these 2nd generation ‘build and brand’ combinations is to draw more audience. The more cloning of familiar real world spaces the more tempted to enter, will be those normally reticent in entering MUVEs, virtual worlds and MMORPGs. I also think the more they see (in posts like this even) familiar brands the more comfortable they will be in taking the leap. The familiar chant of ‘get a life’ generally come from those who dont understand these 3D social networks, so now that sustained business models (millionaire Angshe), bona fide identifiable brands and big TV formats are taking root I am sure they may at least want to understand. Its going to be an interesting Christmas 2006.

Posted by Gary Hayes ©2006