AUGMENTED WORLDS Pt.1 - Business Opportunities: RECOGNITION.

An ‘iPad 4.0 – like prototype’ stroll around an augmented Sydney. Video, music devised & created by GARY HAYES of MUVEDesign.com – there are some key assumptions with this video detailed here -

ASSUMPTIONS WITH THIS CONCEPT

OK this tablet device with camera is running on a very fast connection, instant video and image so we are assuming city wide wifi or 4G type service connection. Also some elements pop up immediately so we can assume the user in the video (me) has setup some preferences or it has learned what type of service I like so it gives instant fulfilment vs dropping into endless menus (more on the interface next). Also the device is extremely aware of its location to a few inches, so we can assume that some of the ‘recognition’ elements are speeded up, being tied into precisely where the user is stood/orientated ‘combined’ with recognition – this will speed up the overall process vs using only Kooaba like Smart Visuals Recognition which puts a lot of pressure on server-side processing.

Notice on the bottom of the interface I have put the words “CONNECTED TO: Sydney CBD People and Places cached data” which is basically pointing out that the speed of recognition to data display is super accelerated when all information is pre-cached, and the way intelligent caching works means the more popular the ‘target’ the faster the response. Privacy – there are a few sequences here and in the longer video that show people being ‘recognised’. Obviously a real concern at the moment, so the concept video assumes the users being targeted have given their consent to outdoor recognition…I know this is a post in itself (see one of mine recently) but lets assume like TagWhat and FourSquare like services, users are allowing their ‘trusted’ friends and companies (for benefits) assess to being recognised – at least for the purposes of this concept!

But the purpose of this pretty detailed post is to simply list and try to categorise the many types of business Augmented Reality apps appearing in the market and to try to identify opportunities.

Augmented Reality Business by Gary Hayes

According to wikipedia, the first manifestations of AR appeared in the late 60s, became real in the 70s and by the 90s were already being used by major companies. Now portable computing is finally powerful enough to deliver AR to anyone who has a smart phone or latest generation PC or console. For those unfamiliar here is my simple definition of Augmented Reality.

Information, 3D models or live action blended with or overlaid onto the physical world around us, in real time. A camera & attached screen is used to view the combination of real world and metadata or rich media. Devices or systems commonly used for AR include:

• Mobile devices with inbuilt cameras such as iPhone, DS Lite, PSP or Android
• A head mounted display HMDs (eg: glasses or futuristic contact lenses) attached to a wearable networked computer
• A PC or Mac with webcam
• A games console with camera accessory
• A large TV screen with advanced Set Top box and Web cam
• Others in development

There will be hundreds of business, marketing and educational applications alongside the many 1st generation entertainment examples currently emerging. As usual the video game and porn industries are pioneering the research and development of the technology but we are about to see a plethora of more mundane utility & life enhancing applications about to swamp the market. Combine this with hundreds of space cadet ‘toys’ and viral fluff and we start to see the beginnings of a large industry. Of course the usual caveats apply but unlike say virtual worlds which suffered from too much early hype, these apps are much more accessible to anyone with a decent smart phone, up to date games console or computer with webcam – there is a big difference.

But before going onto my list (which is of course non-exhaustive) here are my basic types of Augmented Reality from a slightly technical perspective but which we can apply to commercial and marketing applications. My presentations and paper have much more detail with specific case studies of each.

GARY’S FIVE TECHNICAL TYPES OF AUGMENTED REALITY

1. Surface – The most understandable form of ‘reality that is augmented’ would be screens, floors, walls etc that respond to the touch of people in them providing them with virtual real time information or collaboration
2. Pattern – The AR system performs simple pattern recognition on a shape, marker (usually on a framed card in the real world scene) or face and replaces it with a static or moving element e.g: a 3D model, info, audio, video stream or loop etc: You view the ‘items’ in the scene with you
3. Outline – This is where your hand, eye or body outline is picked up and seamlessly ‘merged’ with the virtual elements. Simple example where you can pick up a 3D object that doesn’t exist because the system is tracking your hand outline.
4. Location – Based on detailed GPS or triangulation location & position/view of the camera/device the AR system can overlay information precisely over buildings or people as you move through real space.
5. Hologram – Using ‘smoke & spinning mirrors’ literally in some cases, virtual or real items are ‘projected’ into the physical space you are in and can be interactive with based on cameras tracking real world impulses e.g: hand gestures or audio signals

Before the more detailed list I embed my summary flickr chart whose purpose is to try to categorise types of business orientated augmented reality apps so to identify opportunities. The graph places 16 on axes of commercial value (likely revenue or marketing potential) vs adoption (scale of popularity vs a niche, client user base). It is a starting document to aid classification of this emerging commercial sector that I hope you find useful. The AR types, color key linked to each model is suggestive only. (Click on image for enlargement over at my flickr pages)

Below are longer descriptions of my top 16 Business Applications/Models for Augmented Reality and I suspect I will develop twenty more before the day is out! I have tried to break up some of the big AR areas such as ‘locative AR’ and separated them into several opportunities. This is an exercise looking at the marketing or commercial ‘intention’ vs a traditional pure business model, broad approach. It is a starting document to aid classification of an emerging commercial sector that I hope you find useful.

1. IN SITU: Aiding sale by seeing projects & products placed in the environment before completion. The benefit of a customer or client seeing a finished project, before it is complete. For example 1) real estate agents can scan and show an empty house full of stylish furniture or 2) an architect who can show the billion dollar client the skyscraper as a model perfectly aligned with the other buildings on the empty site and 3) Customers who want to see what the clothes look like on ‘them’ (as in the Zugara video above and Cisco below). The list of applications goes on.
2. UTILITY: Selling life enhancing AR applications perceived as useful. Development and commercial sale of applications such as underground train orientation, bus stops & times, traffic alerts, airport gates & plane arrivals etc: all overlaid in real space. Sometimes called AR browsers as they cross reference what or who you are looking at with anything or everything off the web or like this UPS example allows you to see if your ‘stuff’ will fit the ‘virtual’ postal boxes.
3. TRAINING: Hands-on with complex equipment and work scenarios. Using ‘outline’ recognition this allows us to be virtually ‘hands-on’ with complex equipment in difficult-to-practise work scenarios. Bomb disposal, surgery, flight simulation. . Indeed according to wikipedia the actual phrase Augmented Reality was coined by Tom Caudell in 1992 while at Boeing where workers trained to wire aircraft on AR systems. A massive industry for the developer community charging b2b rates.
4. SOCIAL GAMING: Both connotations of the word, pay-per-play mixed reality games in physical space. The potential to run pay per play (e.g: virtual paintball style) games in physical location and also live connected betting on sports or other competitive play – e.g: You point your iPhone at the horse and wirelessly place a bet – mid race! (odds adjusted of course) or using basic surface AR you play with others in a new kind of ‘games’ room!
5. LOCATION LAYERS: Blended guides to new places, tourism, enhanced travelling or themed space. For travellers just arrived at your city, theme park or other experience you can provide them with pay for tools that will help them take the most ‘mutually beneficial’ route after they arrive. Free data from wikipedia, local bloggers or more commercial entities add depth.
6. VIRTUAL DEMO: Display to promote sale, of product in pre-release or remotely via catalogue etc: To promote advance sales before the consumer gismo hits the stores, an AR display or the device/s so potential customers can manipulate it, see it from all sides, even customise the order. We may see future stores displaying the majority of items on the shop floor as AR while the item is shipped to your house before you get home! The reverse of this, an AR catalogue (as in the Ikea video below) that pops up models to help you build or see the product in 3D.
7. EXPERIENTIAL EDUCATION: Pay-per-visit educational services to museums, ancient sites etc: Pay-per-visit (ppv) to visit highly experiential museums, theme parks, zoos, ancient sites or exhibitions but with a higher purpose of providing deeper levels of information & visual sense than a simple plague or hard to follow guide book ever will. The sleepy animals in the zoo come to life, the ancient fossil is animated into an overlaid Google Earth, the Battle is enacted ‘on the original battlefield.
8. ENHANCED CLASSIFIEDS: An AR directory that promotes local 3rd parties product & services overlaid at the location. One of the obvious apps where someone in a city or town looking for a specific item could be ‘guided’ to it. A very affiliate model where the company that owns the Augmented Reality listing mechanism will take a slice of any fulfilled sales. A lot more to this of course.
9. 3D VIRALS: Branded company or personal promotion & ads using ‘cool’ 3D toys. Pattern based 3D model that entertains and is spread virally. The YouTube moment as a million links to cool ‘3D stuff’ that takes place next to you. Already we see some AR apps that allow you to record scenes of you interacting with said ‘3D viral’ and pass those around too, titillation, quirky giveaways– JibJab-type, put ‘you’ in the cartoon but revered, they are with you in 3D space.
10. PERSONALIZED SHOPPING: Walking around stores made relevant, opt in personalization and targeting. The oft mentioned Minority Report example. But in the pulled model, here you can deliver information to potential customers scanning stores, streets or shelves for discounted or personally relevant products.
11. COOPERATION: Service industry for augmented virtual meetings. We are all familiar with video conferencing, a few have dabbled in 3D virtual world get togethers but AR meetings are a game changer. The potential here using ‘discrete’ personal screens is to have the inevitable remote meeting with live feeds of your colleagues, blended into your room – pay-per-ARmeet
12. BLENDED BRANDING: The equivalent of hoardings, virtual poster ads. Once given a reason to be scanning outdoor areas with their AR devices the potential to deliver topical, timely and relevant ads or branding into the scene. Again care must be taken as AR spam (like social network spam) will quickly irritate, but like free to air TV, using various sponsored or freemium biz models will mean a certain amount of branding will be acceptable.
13. AUGMENTED EVENTS: Pay-per-use of enhanced sport or pop concerts. At live events spectators can pay and then scan their view of the ‘match’ for the latest information on sporting achievement or pop star gossip and of course tracking trails or watching replays in situ, merged over say a static real sporting scene.
14. INTERTAINMENT: New form experiential TV and films. Following on from my ‘AR story’ post, we know people will pay a premium for a new kind of ‘film’ experience where you ‘live the experience’. How about one that plays out at ‘your’ place. Semi customised marker or location AR apps will layer Brad or Angie into your lounge, onto the coffee table or your ‘composited’ in real time into the latest Mixed Reality TV show. Combined with 3D viewing technology will make Blu-Ray seem so 18th century.
15. UNDERSTANDING SYSTEMS: Creating AR for internal or exploded views of complex objects. Primarily useful in training or helping sell something where physically taking it apart is not possible a view of a car or other complex object can be enhanced. Labels or even an exploded view in real time can help get the message across.
16. RECOGNITION & TARGETING: Pushing ‘relevance’ to outdoor consumers – facial recognition linked to online data. To be used with care! It will be interesting to see how privacy laws affect this but in a pushed model you could ‘scan’ visitors to your store, identify their faces, do background links to their ‘social networks’ followed by personal targeting while they are shopping. We all know this is going to happen!

Some examples:

CISCO’s future of shopping

BMW Future of Car repairs

Augmented Reality in Marketing and Sales

James Cameron’s Avatar – Augmented Reality Entertainment

IKEA’s Future Augmented Reality Catalogues

To finish a few lines that describe one of my presentations coming up that acts as a nice conclusion…

It has been called the future of social mobile networking, experiential marketing and entertainment, Augmented Reality is about to snowball. With hundreds of apps launching onto iPhones, Games Consoles, Advanced TV and your PC this new technology allows us to view our world in exciting new ways. Once the domain of science fiction or super computers this new technology, blending the real with virtual, is now literally in the hands of everyday consumers and business. Augmented Reality is simply using a camera with screen to layer rich media and information over ‘our’ real world, in real time so we can now scan and layer web info over products and people in the store or street and provide virtual, ‘in situ’ hands-on demonstrations and deliver the ultimate ‘guide’ to our complex physical world. Is AR the new 3D viral experiential marketing or another temporary fad?

Finally, finally for those interested here is my Augmented and Mixed Reality YouTube playlist if you have an hour or so to spare to view the current collection of 60 +

We have come a long way since cardboard cutout VRML back in the 90s. It is interesting to watch a revolution take place at the moment in Mixed & Augmented Reality and one particular subset, augmented reality. Gary produced a compilation in 2008 of various AR and MR area in this short video (at the bottom of this post you will find a transcript)

Another exciting area of AR is emergent game play. This example from a Japanese company called Geisha Tokyo Entertainment is your very own augmented reality Geisha Girl. It is interesting watching a story develop so imagine the potential now for story tellers or performers who can give their ‘participant’ audience the tools to bring the storyworld out into their personal space. I think in advance of the oft mentioned ’surround-o-film’ (yes holograms of Bond jumping around you in your front room, god help us!) this is a simple and elegant solution – given we all have webcams built into laptops, games consoles, mobiles etc etc: With this particular package you get two cubes and two sticks but imagine a cube for a range of characters or sets for your latest film or kids TV show, perhaps available as a premium.

Finally the combination of Augmented Reality and Social Media is probably the most obvious and compelling near term offering. Being able to layer the rich social web over the top of the real world adds real value. This video speaks for itself.

Below is the slightly futuristic transcript from my video compilation above, for those who can’t hear the narration. (BTW a better audio version available as a 50MB download here). Attribution for most clips appear in this post.

Is this reality? Is this fantasy? In 2008 the great transition truly began. Whereas a decade earlier we used to ‘log in’ to cyberspace, now we began to semi-permanently inhabit virtual space and the digital world started to leak out into everyday life. It began to infiltrate our shopping malls and became integrated with our everyday lives. Humanity started the relentless journey towards the natural future where virtuality became a greater reality,  and where digital fantasy and organic reality were inextricably intertwined.

Through the early 2000s pervasive wearable computing started to break down the walls that used to separate our virtual existence from our physical one. This same technology also allowed us to embed ourselves seamlessly into virtual worlds which is where we could truly experiment with future forms of entertainment, art, education and business.

In these new places we became the real time, living celebrity. Many of us became avatar stars, pixel gods – exalted as those linear film and TV icons. This became the only place for many. By 2007 hardcore gamers were already spending fifty five hours per week ‘inside’ these shared worlds and learned to forgot about the real world. This became the place where their most meaningful hours were spent. Everyone began to blend the two domains, combining them in new ways, driven by an invisible urge. Thus started the global meshing, of today’s mixed reality.

But in 2009 questions were asked. As in the many films that hinted at our transhumanist future, would the physical embodied self ever become redundant? Would we truly evolve as self contained, digital entities? May we as well be slumped in a chair with a cable plugged into our heads? Regardless, as the real world became inhospitable we all started to crave for the immersion and began moving our physical selves into these spaces. Even more than that we wanted our peers our un-virtual audiences to be able to share our experiences there. It became a contagious as the most poignant moments and memories were the ones in virtual space.

But we needed to look beyond these screens and into our own souls. Find the truth written along the ‘thin white line’ that separates analog from digital, person from avatar. This was the beginning. It always was. As far as the meshing or blending of reality and virtuality, in 2009 they were the equivalent of grainy black and white movies seventy years earlier. Our experiences were simultaneously part real, part virtual but it didn’t matter which was which anymore. Both were valid and they depended on each other. Those early parallel existences of twenty years ago started to move humanity forward, released our imaginations and we coexisted with our precious and ultimately fragile, physical selves. The great crossing had begun.

But back to social virtual worlds and I have quickly mapped some dates and SVW events (most are recent) onto a slightly modified Gartner ‘hype cycle’ curve. For the uninitiated here is a brief wikipedia definition.

Since 1995, Gartner has used hype cycles to characterize the over-enthusiasm or “hype” and subsequent disappointment that typically happens with the introduction of new technologies. Hype cycles also show how and when technologies move beyond the hype, offer practical benefits and become widely accepted.

Of course in reality we have really been through a sine wave quite a few times since 1987 through 2005 but since the relative maturity of services like Second Life, There and others true and robust applications are emerging, beyond the traditional ‘gamer/entertainment’ use. MUVED created a post a few weeks ago showing widespread investment in a range of niche virtual worlds and this trend is seemingly continuuing apace. Raymond de Villiers, CEO of Wisdom Games is very bullish about the use of these worlds for business communication in a post entitled Growth of Virtual Environment Expected in 2009…

Throughout 2008 the corporate world was exposed to the reality of virtual worlds, which has seen a growing influence on how companies train, market, advertise and communicate. This is a trend that is expected to continue into 2009 as organisations begin to recognise the merits of incorporating gaming into their basket of communication tools.

Raymond makes some mature philosophical points about how the corporate world is now host to a new generation of employee who don’t see these spaces as for geeks and timewasters

For those who have never played a computer game in their lives, virtual worlds look foreign, sinister, scary, and like a waste of valuable corporate resources. However, for those who grew up in a world where they played virtually, their expectation of being able to apply the consequent life lessons will be expressed in their expectations of the work environment. Computer gaming, and the associated virtual worlds, will, as a result, be seen as an increasingly important infrastructural consideration.

Virtual Worlds News puts more of a realistic spin as regards raw investment (tracked over at VirtualWorldsManagement also) into new ventures with a decline (albeit way ahead of the global recession) of venture capital put into worlds – around $101m in q4 2008. In its item $101M Invested in 13 Virtual Worlds-Related Companies in Q4 2008 VWNews believes that there is a move into older generation worlds and those with much more of a niche focus.

With the harsh economy looming larger than ever, it seems like more and more focus is being put on unique niches and markets, established business opportunities, and quick-to-market strategies.

Around the world there are major government and state agency investments looking at the efficiencies and cost savings of using Virtual Worlds. Close to home (Australia) we have the MDA in Singapore committing overall around $70 million to project and services in the virtual world education space, creating around 300 jobs in the process too. I could go on with many examples of global investment in the public sector but we are also seeing expansion of existing services in the private sector a good example being Linden Lab buying two Virtual Goods companies XStreet SL and OnRez. As reported by Venture Beat the micro economy around Second Life is still in growth and Linden Lab will need a slice of that pie (in small rev share of the user to user transaction) to continue its own growth. This is long overdue by the Second Life creators and Venture Beat were keen to state the obvious also

Virtual goods have expanded to become a $1.5 billion market, but Second Life was engineered in the days before the business model was fully proven. Linden Lab residents exchanged more than $360 million worth of virtual goods and services in 2008, making the virtual economy and the ability to make a living one of the big draws of the virtual world. Top merchants in Second Life have made more than $500,000 in real money in 2008.

So there is definitely something afoot at the moment even in serious global recession there is a kind of exodus into virtual alternatives happening. The most important message is that developing on a reasonably stable and mature platform is not a big investment and companies, education departments and entertainment property owners should still see this investment as two key returns in the ‘real time, immersive arena’  – how to create experiential services and how to engage with a community. I leave you with a diagram I created nearly 3 years ago that shows a transition from web 1.0 to web 3.0 and let you decide if and how it maps onto my mapped Gartner hype curve above.

A video insight into the build process of the new Deakin Arts Education Centre Island built and designed by MUVEDesign’s Gary Hayes completed early 2008.

Jenny Grenfell, Arts Lecturer at Deakin said:

“This Deakin University Arts Education community in Second Life aims to stir the imagination. It is a world we create and create within. It is at once a work of art and a place to inspire, make, share and present art. Our presence in SL is both for performance and is in itself a performance.

Our Second Life existence is an extension of our involvement in the real world where we are passionately immersed in the practices of teaching and learning in our art forms of visual art, drama, music, dance, and media.”

Medium rez 105MB MP4 available here. Music composed and performed by Gary Hayes. These images with comments can be seen here.

Video, music & 447 Images © Gary Hayes and Deakin University

Below is the flickr slideshow embedded as an alternative