1. At the TV Show Australia next week he is presenting about Social Television now and in the near future and how Inspiring the stories of tomorrow with social mediawill make TV truly and finally interactive
   

   Santa Clara Convention Center

2. In Santa Clara, CA he is opening the business track by presenting New Augmented Reality Business Models at the worlds biggest Augmented Reality Conference known as the Event
3. And in June Gary is presenting at GameTech on Pervasive Entertainment and the exciting merging of Games, Film/TV, Geo-Caching and Social Media

Gary is also presenting at various adhoc conferences such as Creative Sydney here where he is MC’ing & presenting on various Transmedia & Multi-Platform Content sessions.

More details on each of these below and if you want to speak to us about how MUVEDesign can help you realise your projects in these areas look at our service area and contact us here.

‘Multi-Platform’ TV Show

Australia 2011 27-29 Apr Sydney

Day 2  2.10pm Inspiring the stories of tomorrow with social media

The Augmented Reality Event 2011

May 17-18 Santa Clara, CA, USA

The Augmented Reality Event 2011 will feature industry luminaries: Bruce Sterling, Vernor Vinge, Will Wright, and Blaise Aguera y Arcas. ARE 2011 will include more than 100(!) speakers in 30 sessions organized into 3 tracks: business, technology & programming, and production & design.

The 2 day event will feature more than 33 hours of talks running the gamut of AR essentials in 3 tracks: Business, Technology and Production:

1. Business – For executives of established and start-up AR companies, as well as mobile hardware companies – in search of business models and promising verticals for AR;  a venue to form partnerships, learn about latest innovations, and most importantly speak with clients.
2. Technology – For Developers, programmers and technologists seeking the latest and greatest engines and tools for AR; learn from case studies and post mortems delivered by experienced developers from the leading companies in the space.
3. Production – For Producers, designers, project managers (in gaming companies, agencies, marketers, brands, and artists) hungry for proven techniques to leverage augmented reality to advance your brand, attract and keep your customers, and build successful campaigns and products that will delight users.

Day One – Tuesday 17-May-2011

• 8:15-9:00 am ARE 2011 Press Conference  Moderated by Ryan Wagner Great America XK – First Floor. Opportunity for AR companies to announce new products with major tech media
• 9:00-9:45 am Keynote: Bruce Sterling Wired Main Theater
• 10:00-11:00 am AR Market: Today and Tomorrow Business Track  (Great America J – First Floor)
  • Gary Hayes (MUVEDesign)- New AR Business Models
  • Laurel Papworth (Community Crew) – Building Mobile AR Social Communities for Business

GameTech

21-22 June 2011 Sydney

Video games and interactive entertainment has come of age, and it’s serious business. Games have emerged as the pinnacle of consumer entertainment. The interactive entertainment industry is now faced with unprecedented popularity, unparalleled growth and significant opportunity.

“The global video gaming market is expected to grow at a CAG Rate of 8.9% to reach $76.1bn in 2013”*

The benefits of this are not just restricted to the video game developers and publishers- all areas of the value chain and the wider industry arereaping the rewards of this growth opportunity. As users are shifting towards new platforms developers and publishers are changing their business models to wrestle for market share.

• Gametech is a 2-day conference blitz of learning, inspiration, solutions, and networking. It is the most comprehensive event on games and interactive entertainment in the region, and will provide an unparalleled event experience.
• Gametech is for those who wish to benefit from the explosive growth in video gaming. Whether you are part of the industry value chain or whether you are considering interactive entertainment as a tool for your business.
• Gametech features visionary presentations, insightful case studies, lively debate and expert information on timely cutting-edge business topics of interactive entertainment and gaining from video gaming technology.

Gary’s is presenting on key areas of Merged Media and how Gamification is acritical component of all multi-platform content forms

1355 – Pervasive Entertainment – Games, Film, Music, Print & TV merging with audience networks

• Assessing the concept of Pervasive Entertainment and how it is affecting the games industry
• How are brands extending to transmedia?
• Reviewing the business models behind geo-social / augmented reality games
• Learning form relevant case studies
• What models of media production, distribution, and consumption are implied by these future
• visions of entertainment?

Gary Hayes, Director MUVEDesign & Founder StoryLabs

ROUNDTABLE 15.10 - Unleash the Power of Interactive Entertainment – Diversifying Your Product Offering with Video Games

• How has the appeal of video games widened to new audiences?
• Underlying the process of developing games to compliment existing products
• Is there a limit as to the nature of business capable of benefiting from games
• Finding the right monetization model to support a standalone games product
• Key ingredients for successful games for social media platforms
• How can games and interactive entertainment incorporate features such as user-generated content, sharing, rewards and referral programs?
• Key steps to monetize new social gaming environments?

Sam Doust Creative Director, Strategic Development
 ABC

David Peattie Managing Director
 Hasbro

Hugh Baldwin Director of Television and Content Acquisitions 
Nickelodeon, MTV

Gary Hayes Director MUVEDesign & Founder 
StoryLabs

Augmented Reality – A Showcase What The Marketers Need To Know – October 28th 5:40pm – 6:30pm

“Is AR just like a lift-the-flap book?”
What value can AR offer for people and enterprises?
The session outlines various definitions of AR such as location based platform or image recognition platform and how the technology provides valuable insight for marketing. Exploring various case studies, we will discuss the current situations and issues for AR as well as its future.

Gary will be showing several of his Augmented Reality branding overview videos such as the recent Recognition – Business Opportunities but also one of three Location Based Augmented Reality Story developments in this space  called ‘Time Treasure” – a rudimentary, story rich LBARG (location based AR game?!) that MUVEDesign are currently story designing & coding for Android tablets. A hort 2 minute taster video embedded below…

Without giving the plot away, the structure of this game is quite straightforward. There are ten layers of time from 2050 back to 5000BC that you slowly penetrate following stories, clues and trails all based at POIs (points of interest, precise locations) around your city. The traditional MMOG talking-head quest and story givers are a unique part of this as well as a range of capture & loot quests that require you in some cases to do a little ‘real world’ grinding… ok not too much For me the challenge as always is about creating strong ‘call to actions’ and constructing a narrative backbone to make it worth your while walking and in some cases running around town! I will do a post when this reaches a full working pilot.

We will keep you updated in the next months of progress on this and the branded client games also in progress.

Get it via the iTunes App store now or go to the Apple App preview page

1.0 RELEASE VERSION 1.0 – Description and screen shot

Social Media Counts is an amazing and hypnotic real time display of eighty four user, content and business metrics across social media, games, mobile and traditional or heritage media. The data is based on actual reported numbers which are listed in the embedded info panel and this rolling ‘count-up counter’ is a projection forward in time based on these real numbers. With over 40 million impressions already of the embeddable flash version across the web this counter gives real insight into the tsunami of content, proliferation of devices and the money being made from a range of entertainment and services.

Features

include the ability to switch in real time between the social. mobile, games and heritage lists as well as step forward in increments of day, week, month and a fixed view of a year ahead. You can also reset the counter at any point using the ‘begin’ button. An info screen contains a list of most of the data sources used in the counter and a web link takes you to a web page with these listed and links to the sources pages also.

Please Note: The application is intentionally not highly interactive as it is meant to be used as a display in portrait mode only, making it readable at presentations and demonstrations to others in small meeting environments as it allows easy visibility of the amazingly large growing numbers.

The app will be updated regularly – approx once a month with updated statistics where available and suggestions are always welcomed for new, remarkable figures (make sure you have a reliable per day, month or year figure though!)

For reference here are the current list of real time counter metrics being displayed:

Social Media Tab

new blog posts, members added on Facebook, US dollars spent on virtual goods globally, tweets sent on Twitter, videos watched on YouTube, iPhone apps downloaded, US dollars spent on Facebook gifts, hours uploaded onto YouTube, new Twitter accounts, text messages inside Second Life, US dollars made from global messaging & data, iPads sold, new members on LinkedIn, pieces of music bought on iTunes, searches made on Google, emails sent globally, US dollars made in ad revenue on Facebook, sms’s sent worldwide, photos uploaded to Facebook, images uploaded to Flickr, new internet users globally

Mobile Media Tab

new mobile phones shipped, US dollars generated from mobile games, sms’ sent worldwide, new phones GPS enabled, US dollars made by global mobile data, WiFi and 3G iPads sold globally, iPhones sold, GBs sent across all mobile devices, hours of mobile video watched in USA, US dollars made from all mobile services globally, new users of mobile social networks, US dollars spent on mobile advertising worldwide, new mobile subscribers globally, new portable pcs / laptops shipped, mms messages in the US, iPhones unlocked, US dollars made from mobile music, people using location mobile services logged onto the web using mobile, made from mobile video, new mobile 3G subscribers globally

Games
join a ‘social’ game from Zynga, US dollars made from MMOG players China, quests completed by WoW players, US dollars revenue from games sold in US, user2user tranactions Second Life, transactions in Eve Online, game units sold worldwide, US dollars spent on virtual goods globally, new global MMORPG subscribers, US dollars generated from Virtual World ads, logged into Fantasy Westward Journey, virtual goods created in Second Life, US dollars invested in Virtual World companies, new US kids (3-11) subscribing to VWs, messages between Second Life users, players transactions in Eve Online, US dollars net revenue made World of Warcraft, portable game units sold, user hours by Chinese online gamers, messages posted on Gaia forums, user hours by Second Life users

Heritage Media
US dollars made at the US box office, US dollars lost to US economy due to movie piracy, read a newspaper in the US, Dollars TV revenue generated globally, single music tracks downloaded, attend a US orchestral concert, US dollars made from global print ads, US dollars total spend making Australian films, US dollars total spend on making UK films, US dollars wages for jobs in US movie industry, US dollarshardcover books sold worldwide, kindle books sold just by Amazon, people in US tuning into radio, US dollars made from music concerts worldwide, US dollars made from DVD & Blu-ray in US, US dollars lost by pirated music & movies Spain, US dollars made from all music worldwide, hours of TV watched by all UK viewers, movie tickets sold US EU China Japan,US dollars spent on new TV sets in the US, printed press page views

APPLE SDK

We are registered iOS 4.1 developers with several apps coming out via the App store soon. We can also place paid apps through onto the store for clients

ANDROID SDK

We are part of the Android development community and can develop applications for android tablets and mobile devices and distribute into the Android market for clients

GEO LOCATION DEVELOPMENT

MUVEDesign are registered developers on both the Layar and Junaio location based browsers that work on Android and iPhad platforms. We can also develop Junaio Glue, marker or pattern based recognition

MARKER & IMAGE RECOGNITION AUGMENTED REALITY

We can develop simple AR marker webcam services using the industry standard Flash AR toolkit or utilise some of the latest technology such as Qualcomm’s Augmented Reality SDK which we are beta registered for

GAMES AND VIRTUAL WORLDS AND SOCIAL MEDIA

MUVEDesign can naturally build a range of social media sites such as WordPress/BuddyPress networks, Drupal or Joomla installs and is well known already as a key Second Life developer but also can create standalone games using software such as Unity – for iPhone, Web, Standalone Mac & PC and even Wii games.

AUGMENTED WORLDS Pt.1 - Business Opportunities: RECOGNITION.

An ‘iPad 4.0 – like prototype’ stroll around an augmented Sydney. Video, music devised & created by GARY HAYES of MUVEDesign.com – there are some key assumptions with this video detailed here -

ASSUMPTIONS WITH THIS CONCEPT

OK this tablet device with camera is running on a very fast connection, instant video and image so we are assuming city wide wifi or 4G type service connection. Also some elements pop up immediately so we can assume the user in the video (me) has setup some preferences or it has learned what type of service I like so it gives instant fulfilment vs dropping into endless menus (more on the interface next). Also the device is extremely aware of its location to a few inches, so we can assume that some of the ‘recognition’ elements are speeded up, being tied into precisely where the user is stood/orientated ‘combined’ with recognition – this will speed up the overall process vs using only Kooaba like Smart Visuals Recognition which puts a lot of pressure on server-side processing.

Notice on the bottom of the interface I have put the words “CONNECTED TO: Sydney CBD People and Places cached data” which is basically pointing out that the speed of recognition to data display is super accelerated when all information is pre-cached, and the way intelligent caching works means the more popular the ‘target’ the faster the response. Privacy – there are a few sequences here and in the longer video that show people being ‘recognised’. Obviously a real concern at the moment, so the concept video assumes the users being targeted have given their consent to outdoor recognition…I know this is a post in itself (see one of mine recently) but lets assume like TagWhat and FourSquare like services, users are allowing their ‘trusted’ friends and companies (for benefits) assess to being recognised – at least for the purposes of this concept!

But the purpose of this pretty detailed post is to simply list and try to categorise the many types of business Augmented Reality apps appearing in the market and to try to identify opportunities.

Augmented Reality Business by Gary Hayes

According to wikipedia, the first manifestations of AR appeared in the late 60s, became real in the 70s and by the 90s were already being used by major companies. Now portable computing is finally powerful enough to deliver AR to anyone who has a smart phone or latest generation PC or console. For those unfamiliar here is my simple definition of Augmented Reality.

Information, 3D models or live action blended with or overlaid onto the physical world around us, in real time. A camera & attached screen is used to view the combination of real world and metadata or rich media. Devices or systems commonly used for AR include:

• Mobile devices with inbuilt cameras such as iPhone, DS Lite, PSP or Android
• A head mounted display HMDs (eg: glasses or futuristic contact lenses) attached to a wearable networked computer
• A PC or Mac with webcam
• A games console with camera accessory
• A large TV screen with advanced Set Top box and Web cam
• Others in development

There will be hundreds of business, marketing and educational applications alongside the many 1st generation entertainment examples currently emerging. As usual the video game and porn industries are pioneering the research and development of the technology but we are about to see a plethora of more mundane utility & life enhancing applications about to swamp the market. Combine this with hundreds of space cadet ‘toys’ and viral fluff and we start to see the beginnings of a large industry. Of course the usual caveats apply but unlike say virtual worlds which suffered from too much early hype, these apps are much more accessible to anyone with a decent smart phone, up to date games console or computer with webcam – there is a big difference.

But before going onto my list (which is of course non-exhaustive) here are my basic types of Augmented Reality from a slightly technical perspective but which we can apply to commercial and marketing applications. My presentations and paper have much more detail with specific case studies of each.

GARY’S FIVE TECHNICAL TYPES OF AUGMENTED REALITY

1. Surface – The most understandable form of ‘reality that is augmented’ would be screens, floors, walls etc that respond to the touch of people in them providing them with virtual real time information or collaboration
2. Pattern – The AR system performs simple pattern recognition on a shape, marker (usually on a framed card in the real world scene) or face and replaces it with a static or moving element e.g: a 3D model, info, audio, video stream or loop etc: You view the ‘items’ in the scene with you
3. Outline – This is where your hand, eye or body outline is picked up and seamlessly ‘merged’ with the virtual elements. Simple example where you can pick up a 3D object that doesn’t exist because the system is tracking your hand outline.
4. Location – Based on detailed GPS or triangulation location & position/view of the camera/device the AR system can overlay information precisely over buildings or people as you move through real space.
5. Hologram – Using ‘smoke & spinning mirrors’ literally in some cases, virtual or real items are ‘projected’ into the physical space you are in and can be interactive with based on cameras tracking real world impulses e.g: hand gestures or audio signals

Before the more detailed list I embed my summary flickr chart whose purpose is to try to categorise types of business orientated augmented reality apps so to identify opportunities. The graph places 16 on axes of commercial value (likely revenue or marketing potential) vs adoption (scale of popularity vs a niche, client user base). It is a starting document to aid classification of this emerging commercial sector that I hope you find useful. The AR types, color key linked to each model is suggestive only. (Click on image for enlargement over at my flickr pages)

Below are longer descriptions of my top 16 Business Applications/Models for Augmented Reality and I suspect I will develop twenty more before the day is out! I have tried to break up some of the big AR areas such as ‘locative AR’ and separated them into several opportunities. This is an exercise looking at the marketing or commercial ‘intention’ vs a traditional pure business model, broad approach. It is a starting document to aid classification of an emerging commercial sector that I hope you find useful.

1. IN SITU: Aiding sale by seeing projects & products placed in the environment before completion. The benefit of a customer or client seeing a finished project, before it is complete. For example 1) real estate agents can scan and show an empty house full of stylish furniture or 2) an architect who can show the billion dollar client the skyscraper as a model perfectly aligned with the other buildings on the empty site and 3) Customers who want to see what the clothes look like on ‘them’ (as in the Zugara video above and Cisco below). The list of applications goes on.
2. UTILITY: Selling life enhancing AR applications perceived as useful. Development and commercial sale of applications such as underground train orientation, bus stops & times, traffic alerts, airport gates & plane arrivals etc: all overlaid in real space. Sometimes called AR browsers as they cross reference what or who you are looking at with anything or everything off the web or like this UPS example allows you to see if your ‘stuff’ will fit the ‘virtual’ postal boxes.
3. TRAINING: Hands-on with complex equipment and work scenarios. Using ‘outline’ recognition this allows us to be virtually ‘hands-on’ with complex equipment in difficult-to-practise work scenarios. Bomb disposal, surgery, flight simulation. . Indeed according to wikipedia the actual phrase Augmented Reality was coined by Tom Caudell in 1992 while at Boeing where workers trained to wire aircraft on AR systems. A massive industry for the developer community charging b2b rates.
4. SOCIAL GAMING: Both connotations of the word, pay-per-play mixed reality games in physical space. The potential to run pay per play (e.g: virtual paintball style) games in physical location and also live connected betting on sports or other competitive play – e.g: You point your iPhone at the horse and wirelessly place a bet – mid race! (odds adjusted of course) or using basic surface AR you play with others in a new kind of ‘games’ room!
5. LOCATION LAYERS: Blended guides to new places, tourism, enhanced travelling or themed space. For travellers just arrived at your city, theme park or other experience you can provide them with pay for tools that will help them take the most ‘mutually beneficial’ route after they arrive. Free data from wikipedia, local bloggers or more commercial entities add depth.
6. VIRTUAL DEMO: Display to promote sale, of product in pre-release or remotely via catalogue etc: To promote advance sales before the consumer gismo hits the stores, an AR display or the device/s so potential customers can manipulate it, see it from all sides, even customise the order. We may see future stores displaying the majority of items on the shop floor as AR while the item is shipped to your house before you get home! The reverse of this, an AR catalogue (as in the Ikea video below) that pops up models to help you build or see the product in 3D.
7. EXPERIENTIAL EDUCATION: Pay-per-visit educational services to museums, ancient sites etc: Pay-per-visit (ppv) to visit highly experiential museums, theme parks, zoos, ancient sites or exhibitions but with a higher purpose of providing deeper levels of information & visual sense than a simple plague or hard to follow guide book ever will. The sleepy animals in the zoo come to life, the ancient fossil is animated into an overlaid Google Earth, the Battle is enacted ‘on the original battlefield.
8. ENHANCED CLASSIFIEDS: An AR directory that promotes local 3rd parties product & services overlaid at the location. One of the obvious apps where someone in a city or town looking for a specific item could be ‘guided’ to it. A very affiliate model where the company that owns the Augmented Reality listing mechanism will take a slice of any fulfilled sales. A lot more to this of course.
9. 3D VIRALS: Branded company or personal promotion & ads using ‘cool’ 3D toys. Pattern based 3D model that entertains and is spread virally. The YouTube moment as a million links to cool ‘3D stuff’ that takes place next to you. Already we see some AR apps that allow you to record scenes of you interacting with said ‘3D viral’ and pass those around too, titillation, quirky giveaways– JibJab-type, put ‘you’ in the cartoon but revered, they are with you in 3D space.
10. PERSONALIZED SHOPPING: Walking around stores made relevant, opt in personalization and targeting. The oft mentioned Minority Report example. But in the pulled model, here you can deliver information to potential customers scanning stores, streets or shelves for discounted or personally relevant products.
11. COOPERATION: Service industry for augmented virtual meetings. We are all familiar with video conferencing, a few have dabbled in 3D virtual world get togethers but AR meetings are a game changer. The potential here using ‘discrete’ personal screens is to have the inevitable remote meeting with live feeds of your colleagues, blended into your room – pay-per-ARmeet
12. BLENDED BRANDING: The equivalent of hoardings, virtual poster ads. Once given a reason to be scanning outdoor areas with their AR devices the potential to deliver topical, timely and relevant ads or branding into the scene. Again care must be taken as AR spam (like social network spam) will quickly irritate, but like free to air TV, using various sponsored or freemium biz models will mean a certain amount of branding will be acceptable.
13. AUGMENTED EVENTS: Pay-per-use of enhanced sport or pop concerts. At live events spectators can pay and then scan their view of the ‘match’ for the latest information on sporting achievement or pop star gossip and of course tracking trails or watching replays in situ, merged over say a static real sporting scene.
14. INTERTAINMENT: New form experiential TV and films. Following on from my ‘AR story’ post, we know people will pay a premium for a new kind of ‘film’ experience where you ‘live the experience’. How about one that plays out at ‘your’ place. Semi customised marker or location AR apps will layer Brad or Angie into your lounge, onto the coffee table or your ‘composited’ in real time into the latest Mixed Reality TV show. Combined with 3D viewing technology will make Blu-Ray seem so 18th century.
15. UNDERSTANDING SYSTEMS: Creating AR for internal or exploded views of complex objects. Primarily useful in training or helping sell something where physically taking it apart is not possible a view of a car or other complex object can be enhanced. Labels or even an exploded view in real time can help get the message across.
16. RECOGNITION & TARGETING: Pushing ‘relevance’ to outdoor consumers – facial recognition linked to online data. To be used with care! It will be interesting to see how privacy laws affect this but in a pushed model you could ‘scan’ visitors to your store, identify their faces, do background links to their ‘social networks’ followed by personal targeting while they are shopping. We all know this is going to happen!

Some examples:

CISCO’s future of shopping

BMW Future of Car repairs

Augmented Reality in Marketing and Sales

James Cameron’s Avatar – Augmented Reality Entertainment

IKEA’s Future Augmented Reality Catalogues

To finish a few lines that describe one of my presentations coming up that acts as a nice conclusion…

It has been called the future of social mobile networking, experiential marketing and entertainment, Augmented Reality is about to snowball. With hundreds of apps launching onto iPhones, Games Consoles, Advanced TV and your PC this new technology allows us to view our world in exciting new ways. Once the domain of science fiction or super computers this new technology, blending the real with virtual, is now literally in the hands of everyday consumers and business. Augmented Reality is simply using a camera with screen to layer rich media and information over ‘our’ real world, in real time so we can now scan and layer web info over products and people in the store or street and provide virtual, ‘in situ’ hands-on demonstrations and deliver the ultimate ‘guide’ to our complex physical world. Is AR the new 3D viral experiential marketing or another temporary fad?

Finally, finally for those interested here is my Augmented and Mixed Reality YouTube playlist if you have an hour or so to spare to view the current collection of 60 +

We have come a long way since cardboard cutout VRML back in the 90s. It is interesting to watch a revolution take place at the moment in Mixed & Augmented Reality and one particular subset, augmented reality. Gary produced a compilation in 2008 of various AR and MR area in this short video (at the bottom of this post you will find a transcript)

Another exciting area of AR is emergent game play. This example from a Japanese company called Geisha Tokyo Entertainment is your very own augmented reality Geisha Girl. It is interesting watching a story develop so imagine the potential now for story tellers or performers who can give their ‘participant’ audience the tools to bring the storyworld out into their personal space. I think in advance of the oft mentioned ’surround-o-film’ (yes holograms of Bond jumping around you in your front room, god help us!) this is a simple and elegant solution – given we all have webcams built into laptops, games consoles, mobiles etc etc: With this particular package you get two cubes and two sticks but imagine a cube for a range of characters or sets for your latest film or kids TV show, perhaps available as a premium.

Finally the combination of Augmented Reality and Social Media is probably the most obvious and compelling near term offering. Being able to layer the rich social web over the top of the real world adds real value. This video speaks for itself.

Below is the slightly futuristic transcript from my video compilation above, for those who can’t hear the narration. (BTW a better audio version available as a 50MB download here). Attribution for most clips appear in this post.

Is this reality? Is this fantasy? In 2008 the great transition truly began. Whereas a decade earlier we used to ‘log in’ to cyberspace, now we began to semi-permanently inhabit virtual space and the digital world started to leak out into everyday life. It began to infiltrate our shopping malls and became integrated with our everyday lives. Humanity started the relentless journey towards the natural future where virtuality became a greater reality,  and where digital fantasy and organic reality were inextricably intertwined.

Through the early 2000s pervasive wearable computing started to break down the walls that used to separate our virtual existence from our physical one. This same technology also allowed us to embed ourselves seamlessly into virtual worlds which is where we could truly experiment with future forms of entertainment, art, education and business.

In these new places we became the real time, living celebrity. Many of us became avatar stars, pixel gods – exalted as those linear film and TV icons. This became the only place for many. By 2007 hardcore gamers were already spending fifty five hours per week ‘inside’ these shared worlds and learned to forgot about the real world. This became the place where their most meaningful hours were spent. Everyone began to blend the two domains, combining them in new ways, driven by an invisible urge. Thus started the global meshing, of today’s mixed reality.

But in 2009 questions were asked. As in the many films that hinted at our transhumanist future, would the physical embodied self ever become redundant? Would we truly evolve as self contained, digital entities? May we as well be slumped in a chair with a cable plugged into our heads? Regardless, as the real world became inhospitable we all started to crave for the immersion and began moving our physical selves into these spaces. Even more than that we wanted our peers our un-virtual audiences to be able to share our experiences there. It became a contagious as the most poignant moments and memories were the ones in virtual space.

But we needed to look beyond these screens and into our own souls. Find the truth written along the ‘thin white line’ that separates analog from digital, person from avatar. This was the beginning. It always was. As far as the meshing or blending of reality and virtuality, in 2009 they were the equivalent of grainy black and white movies seventy years earlier. Our experiences were simultaneously part real, part virtual but it didn’t matter which was which anymore. Both were valid and they depended on each other. Those early parallel existences of twenty years ago started to move humanity forward, released our imaginations and we coexisted with our precious and ultimately fragile, physical selves. The great crossing had begun.

Tran-social-media-play [tran-soc-shuh-mee-dee-uh-pley] noun, verb

1 noun – a new form, a means of collaborative communication through play in constructed shared ‘media-rich’ environments

2 verb – taking part in game-like activity across and within online and offline social networks and media portals.

Can we truly create meaningful immersive media ‘experiences’ for others? Ones that last, are memorable, have impact & emotion and keeps the ‘experience players’ coming back for more? I am currently building services, working on papers and delivering courses on Experience Design and this post is a quick summation of some of the background thinking and good case studies.

Life is filled with so many exciting twists and turns. Hop off the straight and narrow whenever you can and take the winding paths. Experience the exhilaration of the view from the edge. Because the moments spent there, that take your breath away, are what make you feel truly alive. © 2000 Stacey Charter

Filmmakers and marketeers are clearly moving into #TranSocialMedia Play in a big way with a triple whammy at the moment of Star Trek, Terminator & Lost (fan) ARG’s and a raft of social media campaigns across other film and TV properties (some much better than others!). I have written about worlds immersion and cross-media design many times in the past but this post looks at the addition of social play into the mix – a permanent fixture.

Transocialmedia play design has historically been the domain of theme park builders, after dinner murder mystery writers, letterboxers or installation artistes – always based in the physical realm with the best examples reliant on group social interaction. But immersive play like this has also been with us since the first MMOGs social games inside fantasy environments and are another side of a triangle which also includes the new kid on the block (adolescent teenager) 2D social media. The main premise of the below diagram is simple – all encompassing transocialmedia design should be centered in the triangle and incorporate these three dominant forces. (there is a lot more to the diagram especially around hybrid services, but it will have to be another post)

TANGENT

A minor diversion but totally related to hybrid services, I was sent a tweet from Ms. NPRIL last night.

BettinaTizzy“Could Disney-Pixar’s 3D film up lift virtual worlds?” this blogger asks. http://tr.im/lEkm – What do you guys think of this? @GaryPHayes?

@BettinaTizzy Good 3D opens the door, inside? the anim film delivered as an auto story route in a home 3D engine where you can also explore!

The actual article was about exposing large numbers of non-gamers to 3D and whether this would turn them into ‘virtual world’ participants. But more than this I am always keen on drawing attention to media forms in collision and future looking mash-ups – as cinema becomes 3D and games engines become filmic (quality wise) why not deliver the film inside the game engine – not as cut cinematic 2D but the whole thing in 3D. You press play and the story arcs unfold around your avatar (perhaps your invisible perhaps your an extra – who knows!), voices play out from the characters and in some scenarios you can interact where the story makes sense. You as the film player can move around inside the environment the drama is taking place, certain action scenes you can jump inside characters and so on. That way from a screen entertainment point of view at least, you can also play/backchat with your social peers inside and around the story – now that would be an experience?. But this is really another whole post so I will continue…

BACK TO THE STORY

The ‘online distributed story’ courses to Directors and Foundation students at AFTRS often start with thinking about what is the nature of true meaningful collaborative play and what serves the story. What does one need to ‘create’ to turn passive story viewers into active story players? When we talk about play there are two key ‘spaces’ – within the screen and with-out the screen. Students already immersed in online 2D social space immediately don’t see the value of the social mediums – asking a fish to explain water. They soon see the potential though unlike many ‘non-social online’ creators who often try to merge crude old-school practise into fragile social media and like a bull in a china shop find it hard to engage. I use a variety of tools and exercises but the simple workshop template below helps story tellers plan and quickly understand two sides of the triangle – 2D social media and 3D online worlds.

But as we know screen media only tells part of the story which is why promotion agencies and enlightened storytellers alike are (to borrow from Dale’s Cone of Experience again) moving into memorable forms of contrived and direct purposeful experiences in the real world and realise that real stickiness will come from the activities at the bottom part of Dale’s Cone.

This is all leading to a few recent examples in the transocialmedia space – mostly campaign casestudies hence the marketing bias and hat tip to casemovies for some. The first is from 42 entertainment, demonstrating how to create purpose, community and elements of role play on massive scale as part of the lead up to the Dark Knight film last year.

Another major launch last year was for Halo 3 and as they were really promoting 3D virtual space it made sense that the unique elements of this transocialmedia offering (albeit low on the social side) was the creation of a large physical model to bring virtual into physical space and then promote the ‘memorial’ throughout urban environments. Halo 3 incidentally became the biggest entertainment launch of all time.

I often use these four simple levels of cross-media design that I wrote on wikipedia some years ago as a framework to differentiate using platforms as just distribution (CM1) at one end to a seamless contiguous experience at the other (CM4). The fan created Star Trek Alternate Reality Game linear summary of a typical user journey, certainly adheres to the description CM4 ‘experience’ and goes where other ARGs have not gone before!

Cross-media 1.0 – Push. Minor variations of the same content placed on different platforms. E.g.: A slight re-edit of the audio for podcast or a transcript diced and sliced for a making of type website and more of now, watch the TV show on the TV, web or the DVB-H or 3G mobile phone or an iPod video download.

Cross-media 2.0 – Extras. Content that is already being produced but not destined to be delivered and repackaged onto other platforms so it appeared to be ‘original’ or at least specially created. More about the wonders of a good content management system than the true potential of cross-media.

Cross-media 3.0 – Bridges. Where most innovative 360 thinking is currently rooted – where the content placed on the other platform is critical to staying in touch with the narrative or service.The narratives tease you towards investigating or moving (via a bridge) to another media form/platform.

Cross-media 4.0 – Experience. To some extent this is producer ‘hands-off’- in that they have created an environment, much like a game, that the participant/s ‘lives’ inside of, following their own path, personalizing the experience. A cross-media 4.0 property is at its height a creation, a collaboration with the audience across many devices, which evolves and grows a life of its own.

As an interlude, an example that only uses two sides of the triangle that I had to include which demonstrates the power of a contrived, controlled physical event that then massively propagates across 2D social media – yes it’s Antwerp Train Station does Sound of Music.

An example of traditional ARG design that fails to use two sides of the transocialmedia triangle and will get lost in the noise is the rather stalled Skynet series of videos that were being used to launch the new Terminator movie. It does have an interesting Twitter element at http://twitter.com/skynetresearch that is now spilling out into Facebook and beyond – worth keeping an eye on. “Skynet Research is dedicated to eliminating human error.”

I had to add this example for two reasons. It is very clever marketing first and foremost morphing the meaning of a basic food product into one of the scariest browser ‘environments’ on the web – also it combined the two sides of the triangle 2D social and 3D world by creating the browser world – Hotel 626 as real horror experience. Many users were actually too terrified to explore this space – not bad for Dorritos!

Transocialmedia play design is still in its infancy as all sides of the triangle above evolve. Social networking is constantly moving towards being more real time as twitter growth demonstrates. 3D worlds are becoming much more integrated with real and 2D social space as well as becoming a far more ‘representative renderl’ of the real world which itself is being invaded with highly sophisticated mobile and pervasive computing devices. The real challenge at the moment is avoiding gimmick and surface superficiality but developing deep well threaded stories that afford a deeper more investigative engagement. I have noted the design of recent ‘media plays’ to be rather linear, find clue A that leads to clue B and so on vs much more non-linear quest. Many also fall into being over authored, complete and don’t allow contribution or collaboration (although that will happen regardless somewhere else if you haven’t allowed for it). One of the biggest challenges is to mature the form.

There are many who see transocialmedia play as purely marketing, there to sit on the coat tails of existing well known brands (such as AI, Halo, MacDonalds/Olympics) vs being of themselves for themselves. Admittedly there are a few of these scattered around but the player base is usually much lower. To finish a small, mini ARG transocialmedia play I designed last year called the Old Forest – which had a good mix of 3D mashed with physical world and a team social component. The story was fictional, looking at witches, spells and fires – you can find more detail about this and other mARGS on the Laboratory for Advanced Media Production wiki.

There was a rather sad epitaph to this mARG as exactly a year later the whole town of Marysville was tragically burnt with great loss of life in the worst Bushfires in Australia’s history. Truth is far stranger than fiction.

Appendix

For reference here is a abridge copy/paste from a post on an immersion post from 2006 – looking at the factors that enable immersive experiences.

SCALE – of the experience. The size of the screen and the amount of story world to explore has enormous impacts on immersion – as well as the detail of individual objects within the ‘world’. One of the reasons cinema will exist for a long time is that the large dark room filled with people is a captivating environment. Now, imagine a cinema where the image is a locked off-shot, of a shared world and all the audience are controlling and representing different characters engaged in a common goal or story. Ummm.

SENSES – goes without sayiing that the amount of senses that are engaged by an experience gives it most potential to immerse. Now as I have said before we dont need to consider full immersive reality rather make sure as well as intellectual and emotional engagement you consider sounds and the grammar of visual. Probably forget about touch, taste or smell for the moment – leave that to the porn industry to work out.

SERENDIPITY – how the world or show you are watching has elements of surprise. As mentioned earlier the more scripted and formulaic the less immersive. People only watch a film for the fifteenth time, I believe, because they strangely hope that there may be something different OR they are peeling back the layers, looking at minute detail and looking way beyond the basic narrative.

STORY – does the narrative engage. This is obvious, if there is nothing for you to be drawn along by (even your own story in some cases) then you will switch off.

PERSONALIZATION – Hence the title of this blog. How much can you minutely affect the world and yourself in it? How much will the world reflect you for being there? Most importantly, how much of your real world personality can you bring with you into the experience.

RESONANCE/CHOICE – How much control or agency do you have over the experience? Are your actions permanent and seen by all? Can you really do and say what you want – freedom of choice. True resonance is like a virtuous circle, you do something and there is a response that forever changes the environment. Like real life. The pushed media of TV, radio, cinema has zero resonance, it all happens in your head. Which is why stories ‘have’ to be based on life’s shared drama. In truly interactive models your actions have impact and will reaction will take place.

TEMPORALITY – How real time does the experience feel? Scheduled TV never feels real time – the only successful shows in the future will be live events, music, sports, live news etc: Everything else has a dubious future in the scheduled world. MMORPGs feel real time when you are in them because of all of the above. Ones that have scheduled events or require you to invade or fight at a certain time are more about story than true immersion.

ESCAPISM – or ‘play’. This goes back to my earlier point about the reason for play and associated spirituality. Is it as much about escaping reality or constructing ideality?

Does the representational nature of these ‘experience’ worlds mean so much more subconciously than endless souless advertisements on TV, or another episode of a soap, or fomulaic hollywood film? Does selecting an identity that is impossible to achieve in real life become a most powerful addictive escape? I suspect all of the above. In terms of building ‘play’ – it should be as fun making it as doing it. I have mentioned before that sometimes authors of experience get so lost in the creation process they forget someone has to watch, play or take part in it! Then it is much weaker an experience.

A cross posting from JustVirtual but there has been lots of excitement about Second Life becoming a more ‘evocative’ engine (visually speaking) for at least 18 months with lots of posts and short demo videos. Recently the Illclan’ers posted an item suggesting we are quite close to having an official Linden Lab release here, Dynamic Lighting and Shadow Engine Coming to Second Life. They like me are also very interested in the ‘controlled’ lighting effects using artificial (isn’t it all?!) light sources vs the ambient ‘sun’. But for starters here is a quick ambient test video…

A quick exploration of some of my old builds which may not be there much longer! Using the Space Navigator and running Windlight in Day cycle mode (the sun and moon take a minute or two to do a full rotation)  to produce lots of moving shadows across the landscapes, people and builds. Rather than just show shadows I was keen to tie some ‘psych trance’ music into fast moving space navigator footage hence the constant movement – all shots took into account the timing with the shadows too.

The whole process was about 1 hour of capture, 1.5 hour edit and 2 hours on music track. Music was composed on Logic Pro mostly using Spectrasonics Omnisphere plug-in software ‘processor-eating’ synth.

I had access to a top end NVidia GTX280 high spec graphics card and quad processor machin so I put all SL graphics settings at max for once! The video was captured at PAL resolution using Fraps and the raw files edited using Adobe Premiere.

To have a go at this yourself make sure you have a top flight graphics card from NVidia or ATI and then download the Shadow Viewer client from Kirsten here or I believe a more recent one (that I used) from Boy Lane here. I am not sure of the widespread use of shadows given the grunt your computer needs to handle this, windlight, voice on top of all the usual networking issues – but for those with computer horsepower it definitely brings the place to life.

Published & created under creative commons – attribution, non-commercial, non-derivative, 23 May 2009 in Sydney, Australia

My Second Life sim builds included: Esperance (AFTRS), ABC Island, Melbourne Laneways, Thursdays Fictions, Deakin, The Pond and others. (I would have loved to show some more commercial & arty builds but non-disclosure and all that!)