The New Reality: VR, QR, AR and Interactive Print Merge the Digital and the Physical

In author Vernor Vinge’s 2006 Hugo Award-winning novel "Rainbow’s End," a hospital patient from the early 2000s wakes up Rip Van Winkle-like in the 2030s and finds that much of the world is experienced via augmented reality (AR); wearable computers that transmit information to contact-lens-based visual displays that overlay simulated content on the physical environment. This is how people—especially young people—experience the world, through their own personalized realities.

Sound implausible and far-fetched? No, we didn’t think so either, because it seems like that’s the logical progression of AR and its related technologies. 

The idea of combining the “virtual” with the “physical” has been a trope of science-fiction and fantasy for centuries. The first-ever description of what we would today call “augmented reality” was written in 1901 by L. Frank Baum, author of "The Wonderful Wizard of Oz" and its sequels, in a novel called "The Master Key." The term “virtual reality” (“la réalité virtuelle”) was first used in 1938 by French playwright Antonin Artaud, although he was using it to describe the illusory world of the theater. In the modern era, movies such as the seminal "Tron" (1982) and TV shows like "Star Trek: The Next Generation" and its spinoffs had human (and humanoid) characters immersed in virtual environments often via the Holodeck.

In the late 1980s and 1990s, the idea of virtual reality (VR)—in which a user donned a special helmet and gloves to interact with a digital environment—entered the contemporary zeitgeist; and in 1994, even the prime-time situation comedy "Mad About You" had an episode in which the main character (Paul Reiser) was talked into investing in a VR system, and the episode itself had Reiser entering a virtual environment in which he could gawk at supermodels. (All technological advancements inevitably have some kind of creepiness factor.) 

VR, with all its various accouterments, never really took off outside of some education and training applications, but it is poised to make a comeback, and if you have been to some recent trade shows, you may have seen the likes of EFI, Fujifilm, Konica Minolta and others using VR technology to virtually demonstrate equipment that wasn’t physically there—a great solution to the twin problems of the expense of shipping large equipment and limited booth space. 

VR has been outpaced by newer technologies such as AR, and the various methodologies that comprise what we would more properly call “interactive print” have started to proliferate. What are these technologies? How do they differ? And, most importantly, how are they relevant to the printing industry? 

A Place for Print

“For a decade now, people been saying print is dead; and yet here it still is, and to me it’s always going to be there,” said Edwin Rivera. Rivera first got involved in AR in 2009 and was the founder of a start-up company in the early 2010s called Reality BOOST (http://reality-boost.com) that was among the first to commercialize augmented reality services and products for a wide range of businesses on local, national and international levels. “I think that the savvy printer is going to know how to leverage these types of technologies to get the desired effect and desired engagement. AR for the sake of AR is not going to help any printer.”

When Rivera founded Reality BOOST, AR was still a largely unknown quantity.

“The market wasn’t quite that ready,” he said.

Still, some people got it. A landmark project for Reality BOOST came in 2011 when Walmart wanted to develop a tie-in app for Marvel’s "The Avengers" movie with which users navigated the store using the Avengers characters and their superpowers.

Walmart understood and loved it immediately and, even better, said Rivera, “They had a 700% increase in traffic on their YouTube channel when we posted the how-to video. It was a really great lesson for us and really opened my eyes early on to show what the technology was capable of.”

MJ Anderson, chief experience officer at RealityBLU (https://www.realityblu.com), said that the company has designed a licensing model that allows printing companies to extend access to its BLUairspace Product Suite to clients, allowing them to take ownership of the creation process.

“The printer can extend access to the software to the desktop of the designer creating the print file who is best placed to determine what the virtual content should look like and how it behaves. But in many cases, they do not want to own the technology. This gives printers a unique opportunity to be that technology interchange between them and the marketer.” 

Leveraging AR for Loyalty

BLUairspace is a software-as-a-service (SaaS) model that can be integrated with a company’s mobile app if one exists, or RealityBLU can create a customized app. Anderson used an example of a consumer entering a retail store and opening the store’s loyalty app to discover specific information about the store’s offerings.

“The average consumer is becoming aware that AR is a thing and will eventually be looking for an organic AR experience. In the retail store case, there might be a call to action on point-of-sale signage that says ‘use our app to see what’s on sale today,’ creating an experience that has as much information as a web site but that is designed for the small screen.”

Anderson also described an effort underway at a major fast-moving consumer goods (FMCG) company to make its packaging a portkey to any or all content relevant to that product along its life cycle.

“For example,” he said, “the consumer can scan the package in the store to get more information prior to purchasing. Perhaps it delivers a discount coupon that can be used at checkout. Once the purchase has been made and the consumer leaves the store, the messaging connected to that package can change automatically based on proximity. Now it’s not about purchase, but about care and use, specifications, auto-refills, connections to Alexa for reordering, or other things the consumer might want that is associated with the product. That significantly enhances the value of the packaging, both to the consumer and the brand.”

An example of marker-based AR on a beverage can, providing access to email, web or phone contact with the brand.

Print. Make. WeAR.

Anything printable can be made to be interactive—and that includes an increasing number of materials and surfaces. One of Reality BOOST’s hallmarks was AR-enabled shirts for schools. A T-shirt would have an image of a television on it and when a user scanned the shirt, they would see a video or some other content playing on the TV. Reality BOOST created a way of standardizing the development of the TV-based shirts and the accompanying app so that it became something affordable for other schools to implement.

“If they’ve got a school spirit video, we can say, ‘We’ll put it on the TV,’” he said. “‘Your video plays inside the TV and activates on the shirt.’ We learned early on how to amortize things—and price points have come down significantly, too.” 

As with anything, one breakout success can goose the entire market. That breakout came in the summer of 2016.

“Pokémon Go really helped the technology get front and center and reach critical mass,” Rivera said. “I can’t tell you how many phone calls I got when that was released, and there was so much excitement. People who had been on the fence about investing in my startup called me and said, ‘Hey, you were right!’” (Rivera  sold Reality BOOST in 2014.)

He has watched the technology and the interest in it grow, and today, he is a consultant for companies looking to integrate AR. There are many ways to implement AR—from commercial “box” solutions and apps to highly customized, built-from-the-ground up apps—and part of Rivera’s job is to advise the client on what the best option is based on what it is they are specifically looking to accomplish.

He is under NDA with most of his clients, so he can speak in only very general terms; but the projects run the gamut from true AR to what we would more correctly call “print reality” (PR) or “interactive print” (see “The 4Rs” sidebar). One of his clients is a healthcare company that is building a next-generation health and wellness facility.

“It feels more like an Apple Store than an emergency room,” he said. “They’re really proud of this initiative and they’re going to launch it in 2020.”

They are developing an interactive print brochure and map that, when you point your phone at them, you can do a virtual walk-through of the facility.

“You’ve got this digital portal that can be attached to physical products like print,” he said. “With the processing power that we have in in the palms of our hands today, you can add so much detail and you can make certain parts of the models interactive and do things. It’s incredible that, really, imagination is the only thing that holds you back.” 

Virtual Budgets

Well, imagination and budget, because while reality is often augmented, budgets for these kinds of projects rarely are. Fortunately, there are enough options available today that virtually any budget can be accommodated.

“You have your platforms that require a higher level of engineering and programming, and then you have your out-of-the-box solutions that are geared more toward the layperson,” Rivera said. “We’ve used both because when I speak with a potential client, I’ll ask them what their needs are.”

Take, for example, a school that is looking to add videos to a printed graduation program. There are ready-made apps that can create that kind of simple interactive print experience.

“It’s very economical and you’re talking maybe a couple hundred dollars, instead of a few thousand dollars if a manufacturer or brand wants to have a scalable enterprise-level type of experience. So, it really depends on the needs of the client.”

On the print provider side, RealityBLU’s Anderson said that as far as printing businesses are concerned, “they have no problem plopping down a million dollars for a piece of equipment—they clearly see the revenue-generating possibilities. By overlooking the AR opportunity, however, they run the risk of not ‘owning the space’ as it develops; rather, as marketers increasingly implement marker-based AR, the printing aspect simply becomes a utility, and the printing service is commoditized.”

Bringing a business card to life with marker-based AR.

Creating the “trigger”—which is the term for the code or other page element that launches an interactive experience is the easy part. As anyone who gets even part of the way into AR and related interactive experiences can tell you, the challenge—and expense—comes in actually creating whatever it is that the trigger directs the user to. If you’re seeking to overlay an interactive map on a smartphone camera’s display, the map and all of its elements need to be created. If you’re adding 3D-rendered Pokémon-like creatures, they need to be drawn, rendered and animated. And even if you’re just launching a video, it still needs to be shot and edited, assuming it’s something beyond what’s available on YouTube. So that can be the sticking point with interactive print, and for those who are helping develop those interactive experiences. 

“People say the devil’s in the details, and when it comes to augmented reality, that is the big devil—content,” Rivera said. “It goes back to the brand. What are you looking to do? When I sit with clients, I don’t ask them what the budget is. I ask them what they are looking to accomplish. Who is your target audience? What’s the demographic? Gender? Now, where do you want to drive them?”

The brand and the desired experience will then define next steps—and ergo the budget. BMW is launching an app for its new hybrid car that, when the user scans the car, it shows what is going on under the hood. The nature of the brand and the desired experience define the approach.

“When you deal with a manufacturer like a BMW, it can’t look cartoonish,” Rivera said.

He added that a way for brand owners to amortize the AR development costs is to include the experience in other media. BMW may have a scannable car that triggers an experience, but the same experience can also be linked to its printed marketing and promotional materials. 

One App to Rule Them All?

There is a lot of potential in AR and PR, but one major obstacle still remains.

“The biggest challenge right now is the fact that it has to go through a specific app, that it's not just acknowledged or recognized by your camera,” Rivera said. “Also, you’ve got different hardware, but holding up a phone or an iPad for an extended period of time kind of gets old.”

What’s next for display technology? The head-mounted display/goggle-based approach that has been the hallmark of virtual reality?

“Nobody wants to walk around looking like an astronaut with goggles on,” Rivera said. “The industry is very anxiously awaiting the likes of Apple coming out with a pair of glasses so it’s a more natural and immersive experience. You don’t have to necessarily hold up your hands to see that digital layer, you can just wear the glasses. That's going to change things.”

Rumor has it that Apple is working on such a pair of glasses, but so far there has been no official—or even unofficial—news. 

RealityBLU’s Anderson said his company is working on an emerging technology—Web AR—which is the ability to scan for augmented reality content without the use of an app, using the phone’s web browser to initiate the action.

“While the marketplace is screaming for app-less AR, I don’t believe apps are going away anytime soon," he said. "The onus is being placed on the marketer to make their app solutions real consumer channels, and AR is one way we can add value to that channel.”

Made to Measure

OK, the cool things you can do with AR and interactive print are all well and good, but there is one big advantage that interactive print has over non-interactive print: it is measurable, and we don’t mean in terms of trim size. It can provide data. Today’s marketing professionals need to know which media channels are working and which aren’t. Electronic media like email and social are beloved by marketers because electronic media are trackable. You give out a printed brochure at a trade show, but do you really know if anyone did anything as a result of getting that brochure? Not really, no. 

However, QR, AR and PR can all be used to acquire better data by pushing print recipients into online track-and-trace marketing environments, and then into marketing automation systems. Print can now be included in the metrics and analytics that chief marketing officers (CMOs) require to gauge the ROI on a particular campaign. A scanned code or AR trigger on a printed direct mail piece or a sign can send the viewer to a website, and the CMO can see that the mail piece or the sign was the source. 

So, the interactive element not only benefits the consumer (a quick link to get more information, an immersive rich media experience or some other payoff) but also the marketer (accurate analytics). Print can be as measurable as electronic media—since it now feeds directly into electronic media—and can thus stimulate marketers to use more print. We’ve always known that print was effective—but now we have data to prove it.