Dominique Friedl holds up his pink iPad to his laptop screen. "This is actually my daughter's iPad," he says with a laugh. "It got commandeered."
In a few seconds, it's easy to see why Friedl's five-year-old took such a shine to what is actually, technically, the family's iPad. On the laptop screen is a picture of a Band-Aid - one that's commercially available in the United States. It's covered in the happy, smiling faces of The Muppets, but otherwise looks like any normal children's plaster. Friedl matches up a dotted line on his iPad app with the displayed Band-Aid, and suddenly Kermit the Frog is on screen, sitting on a swing, suspended from the top of the iPad screen.
"Hey ho!" says Kermit. "Feeling blue? Well not for long, because we're going to have a sing-a-long! Yay!"
He starts strumming on his banjo and singing, "The Rainbow Connection."
"And you can swing him," says Friedl, tilting the iPad back and forth as, through the illusion of augmented reality, Kermit appears to be suspended from the frame of the iPad, swinging over the conference table and in danger of bumping into the assortment of other devices laid out in front of us.
Enhanced experience
Augmented reality, often abbreviated to AR, is the term used to describe how technology allows us to place extra data over visuals from the real world captured by a camera.
It's being used by a number of brands worldwide to create an enhanced and interactive experience. Another example Friedl demonstrates is an app from American Apparel, which allows you to scan an item of clothing with your phone and bring up additional details about it, like the price and the variety of colours it's available in.
At the moment, Friedl explains, AR apps contain almost all of their data locally. The technology works by accessing a device's camera. The app scans the environment through the camera for recognisable images. Once it finds one, it reaches out to a database of images on a server called a "target management system". The database tells the app what graphics (stored on the device) to bring up overlayed on what the camera is displaying.
Unfortunately, storing the graphics locally means the app is only able to recognise about 80 images. In the future, however, the graphics will be stored online - in the cloud - and pulled down on demand. This means augmented reality apps will soon be able to recognise millions of images.
"Think when you walk into Edgars how many items you could hold your device over? Now, with the cloud and being able to access information out of the cloud, we can really deliver more information around augmented reality," says Friedl. "It's really becoming this image recognition tool that delivers further information to enhance the mobile experience."
It's the enhancement of this mobile experience that is driving retail and brand adoption of AR, according to Friedl. The applications of the technology, however, are much broader than marketing and sales.
African dream
Friedl is the local development director of Qualcomm, a global telecommunications company best known for its semiconductors and chipsets, including the Snapdragon processors (which power the latest Nokia Lumia phones). They own a number of patents, among which are ones for wireless cellular and 3G standards most of us use today. One of their key focuses now, however, is getting involved in developer communities.
"Technology requires a use," says Friedl simply.
It's not enough for us to just "connect the unconnected" and "bank the unbanked", we need to give people a reason to use technology, he explains.
"You see this great adoption of smartphones in the US and in Europe, etcetera, and that's because they have people who will write relevant applications. So people have a reason to use the data, it's relevant to them, it's contextual. And we're missing that piece in Africa."
Friedl hopes that through punting AR, Qualcomm can inspire local developers to make apps that serve a purpose and fill a need. The company provides a number of SDKs (software development kits) that developers can use to add functionality to their apps. For example, AllJoyn, which allows for simple peer-to-peer file sharing; Gimbal, which can make applications context aware; and Vuforia, which provides augmented reality basics so developers don't have to reinvent the wheel when coding. These are all available for free through the Qualcomm Developer Network (DevNet) for iOs, Android, Windows phone and HTML5.
Qualcomm's VP for sub-Sahara, James Munn, has said the company believes it's the local application developers and ecosystem that will truly unlock the smartphone potential in Africa.
"Augmented reality is one of those things that it's not about the technology, it's more about the creativity, and it's about what people choose to create within augmented reality," says Friedl.
"We can now sense the world around us and it's up to developers and their imaginations to create more. The platforms are there. It's really... what is that big idea? That's what it's about at the end of the day."
Big ideas
Friedl shares a few examples of "big ideas" using AR from around the world. He retreats to his office briefly and comes back with a toy gun. He plugs in his phone and instantly, on the phone's display, we see the boardroom around us filling with meteorites, which he shoots at as they crash down on office chairs and plastic plants. The combination of AR and kinetic technology will revolutionise gaming, he says.
Augmented reality is one of those things that it's not about the technology, it's more about the creativity, and it's about what people choose to create within augmented reality. - Dominique Friedl
Another example he gives is of a concept app from the University of Johannesburg, which uses a Lego box as a trigger. The user is then played an AR video of how to construct the Lego helicopter contained in the box, step-by-step. When completed, the helicopter takes off and whizzes around our heads. While developing a separate app for every Lego construction the company made would not be viable, Friedl points out the other possible applications for the same idea - how to build a house, how to fix a car, how to play tennis. Interactive instruction manuals that demonstrate exactly where block B and section D slot together, so there are no more excuses for wonky DIY projects.
"I think we need to learn from the success in marketing, being able to drive up sales through engaging consumers through augmented reality. Can you imagine what you could be able to do in education? You'd really be able to drive up results because learning is now fun and engaging," Friedl says.
James May from BBC's Top Gear recently appeared in an AR app for the London Science Museum, which talks users through key advancements in human technology. Word Lens is an AR app that translates text viewed through the smartphone camera into another language on screen.
If AR is making inroads in education, that's nothing, however, to what it can do for medical.
Qualcomm is offering an Xprize to anyone who manages to develop a tool capable of capturing key health metrics and diagnosing a set of 15 diseases using a mobile device that fits into the palm of the hand. An endorsement by Star Trek: The Next Generation's Brent Spiner emphasises the project's similarity to the fictional "tricorder" from the Star Trek universe, which allowed characters to run medical scans by simply sweeping a handheld device over one's body.
Changing the world
AR has a place on Gartner's Hype Cycle, but Friedl believes it's far more than hype. In fact, he sees AR as becoming a part of everyday life, something we take for granted.
The final example he shows is of a home that is controlled by cellphone - the lights can be dimmed, the TV can be switched on, the temperature can be told, the doors can be closed, the music can be selected. There is no fridge in the picture, but it's probably there too, obnoxiously SMSing you a grocery list.
"I don't believe it is a phase. I really think that yes, we love it now because it's 'AR', augmented reality, and it's geeky, but it's going beyond geeky now. I do believe it will just become the norm."
Friedl's daughter, at five-years-old, already considers it so, he says. She holds "her" iPad over magazines expectantly and is surprised when nothing happens on the screen.
"Dad, it's broken!" she complains.
"It just shows you," says Friedl, "it's so intuitive that she would use a device to sense the world around her. So I think that's the ultimate end game; we have this fantastic thing that's creating a digital sixth sense."
* The Digital Sixth Sense is the theme of Qualcomm's presentation at the Mobile Monday event, in Cape Town, next week that falls on the eve of the Africa Com Conference. Friedl will showcase some of these AR examples there. Attendance is free of charge. Sign up to attend and find out more details here.
(Disclaimer: ITWeb is a media partner for the Mobile Monday event.)
Share