Even if you’re not certain what’s meant by the buzzwords “Virtual Reality” and “Augmented Reality”, you have surely heard their growing buzz. This year’s Game Developers Conference includes a two-day VR Developers Conference, and GDC’s Expo will feature at least 4 VR headsets and 70 VR games. While the game industry is consistently an early adopter of new interactive technologies, VR is already a multi-media phenomenon: a “virtual reality experience within Amazon Video” is in development, film festivals are featuring VR movies, and there’s a VR broadcast of the Coachella Music Festival. VR & AR are enjoying rapt attention from both industry press and general news media.
The new display technologies are also getting financial attention. Facebook’s startling 2014 acquisition of VR developer Oculus was big news, based on the huge price: $2 billion. Another 120 VR deals in 2015 drew another $632 million from dozens of firms and funds, and inspired the creation of VR/AR-specific funds and incubators.
Will this excitement inevitably drive the proliferation of VR & AR experiences? Or will it bring VR & AR to an early Peak of Inflated Expectations, followed by a descent into a deep Trough of Disillusionment? (To borrow the fantasy-fictional jargon of the autological Gartner Hype Cycle.) Will VR & AR become as ubiquitous as touchscreen displays? Or are the ballooning expectations dangerously over-inflated? To all these questions, the answer is “Yes, but relax about it.”
Despite all the talk about VR & AR, the words themselves are vaguely defined. For many years, VR consistently referred to “an exciting rendering technology, where I cannot afford the peripheral.” In 1980, this included the simplest possible real-time 3D rendering technology, but the rapid proliferation of personal computers in the next few years brought real-time 3D to the desktops of the masses, and real-time 3D on a flat screen was no longer considered “VR”. In the early ‘90s, haptic feedback was VR technology, but Microsoft’s 1997 introductions of affordable force-feedback joysticks and steering wheel were welcomed as game accessories, not “VR devices”.
In other words, until recently VR was an aspirational buzzword; it referred to technologies that were not yet ready for widespread consumer distribution. Another aspirational buzzword is Artificial Intelligence, which essentially refers to decision-making or semantic-modeling technologies that are not yet fully feasible. Out of AI have fledged such important technologies as predictive analytics, voice interfaces, and robotic vacuum cleaners, all of which are no longer thought of as AI technologies. Similarly, Big Data denotes datasets whose analysis is not fully feasible, and Home Automation covers exactly those systems that are unwelcome in my house: a generic 7-day programmable thermostat is not an example of Home Automation, but the Nest Learning Thermostat certainly is. The Nest autonomously downloaded defective software this January, abruptly shutting off heat for many homeowners, amidst record-setting cold.
For some of us, VR continues to denote rich, multimodal, real-time simulations of reality. But today the phrase predominantly refers to any one of a number of head–mounted displays that include stereoscopic video output, while reading sensors that indicate position, movement, and perhaps location. In other worlds, today VR usually means “a bucket over your head, with a video projector inside, and a lot of sensors.”
Augmented Reality, in the usual modern sense, refers to that same VR bucket, except AR’s bucket has transparency. This allows the user to interact with the real world, overlaid with fully-reactive computer imagery. That distinction accounts for the many game designers and computer scientists who are unimpressed with VR, and deeply enthusiastic about the potential for AR. As one computer scientist notes, AR “has the entire world and much of human experience as raw material to be augmented,” into which it can introduce virtual objects or relevant information, by stark contrast to the fully-immersive VR experience. While enjoying a VR (in the newer, headset-wearing sense of VR) experience, it’s unwise to get out of your seat, let alone walk around and interact with your environment; you are fully blind to the world. In an AR experience, you might see your hands, as they create magical items, or mundane craftworks. And as AR systems learn to map and model the environment around them, your virtual creations could be place on your actual living-room mantel, for the viewing pleasure of anyone who shares that AR-enhanced view.
Given the overlap in VR & AR technologies, particularly in terms of sensor-enhanced head-mounted display technologies, these distinct concepts are often lumped together, as in this article. Or one might be used to encompass the other, as in GDC’s Virtual Developers Conference, which targets creators of “immersive VR (and AR) experiences”. Google’s Noah Falstein made a brave and reasoned effort to posit “Transmogrified Reality”, to include AR & VR, while highlighting the power of their effects.
As with VR, the meaning of AR has evolved. It historically has referred to any computerized output that overlays the real world. For example, the “3D Compass” app has long been a simple, if useful, example of AR. The app superimposes a compass display atop the real world (as displayed on a smartphone screen, via the phone’s camera), while showing an oriented map in half the screen. As with VR, that older sense of AR remains at large, and includes augmentations in the form of sound or text, but the usual usage refers to head-mounted video or graphical augmentation.
Finally, AR as a buzzword has shared an aspirational quality with VR; a fighter-pilot’s heads-up display, with essential data projected onto the windscreen, was recognized widely as “AR”. But as the same HUD appears in consumer automobiles, the driver accepts it as simply how she gets to see her speed, or route, without having to take her eyes off the road. If that HUD is referred to as “AR”, the speaker is probably a marketing professional.
Although AR might have the widest range of application, bringing data-display into interactions with an airplane’s wiring harness, or with a surgical patient’s peripheral arteries, VR, too, has applicability beyond games, or video entertainment. Game designer and author Raph Koster wrote an early analysis of Facebook’s $2 billion Oculus acquisition that underscored the importance to social interaction of presence, that quality of rich interactive connection that will ensure eternal demand for physical college campuses in the face of rapidly-improving online courseware, and for physical conferences in the face of online-collaboration technologies. Facebook’s core business remains one of human connection. The potential for VR to enrich that connection logically motivates the Oculus deal, even if the nature of VR-based social-networking interaction remains unclear.
Indeed many details of the future of VR & AR experiences are unclear, even if the potential is compelling. Competing VR displays now span a range from Google Cardboard, which was distributed free to New York Times subscribers, to Microsoft’s $3000 HoloLens Development Edition. Somewhere in that spectrum is a threshold of “good enough” for broad consumer demand, for any given application of the technology. Elsewhere in that spectrum is the corresponding threshold of “cheap enough” for that demand to be satisfied. As these thresholds converge, the promises of VR & AR could be realized.
There’s just one problem: when it comes to entertainment media, technology can be the easy part. It can take years for content creators to find the right application of a new technology, and to design the content that takes advantage of it. During that time, a medium can be “huge, just two or three years from now!” for over ten years.
That first compelling application that paves the way for an entire platform is the original sense of “killer app”: the original spreadsheet program, VisiCalc, released in 1979, drove the success of the Apple II computer and motivated IBM’s release of the PC. Similarly, a single game title can reveal the potential for an entire genre of experiences. (This applies to the extent that I define game genre as “a hit game, and its imitators”.) Each VR-device manufacturer is seeking its own killer app, which will probably be an entertainment experience. AR’s wider applicability might lead to its success emerging from a wider range of genre-defining experiences in various industries or content categories.
While new interactive technologies can make many new experiences possible, not all of them are appropriate. Touchscreens, once a rare and exciting technology, are becoming commonplace, appearing everywhere. Unfortunately, this includes touchscreens serving as the main systems-interface in automobiles, replacing the knobs and dials that had allowed drivers to keep their eyes on the road.
When real-time 3D animation was new, there was similarly ill-conceived over-application of that technology. In the mid 1990’s, retailers were excited about bringing the sales process online, which history has shown to be a wise impulse. But many of them sought to do so with a VR experience, which at that time meant a 3D-animated simulation of a real-world in-store shopping experience. The results were very high-tech, and visually exciting (for its day), but also an efficient means to bring all the inconvenience and frustration of real-world shopping to the otherwise-efficient online store.
Another pathology of new media is often “shovelware”, the careless, hurried redeployment of the previous medium’s content onto the new one. When CDs were new, “multimedia” content became the rage: encyclopedias, textbooks, courseware, and games were all compelled to appear on optical media, with music, animations, video, and whatever else would exploit the new technology. This did not last as a medium in its own right. But the integration of sound and images with a broad range of content did become commonplace, even while “multimedia” became a term of derision. (“I survived the multimedia scare of 1993.”) The technology succeeded, even as it disappeared as a product category.
The multimedia era showed that the success of a technology need not correlate with the success of the innovators in that technology. Multimedia was not kind to its parents. Similarly, even as the personal computer industry grew dramatically, bringing computers into every household, and later onto every desk, the PC manufacturers suffered.
The fact that Internet Service Providers prospered in step with the growth in Internet access reflects their monopoly position, granted by municipalities in the 1980’s when Community Access Television (aka CATV) was seen as an important public good, to enable access to the broadcast (over the air) television signals. The thousands of cable companies that became today’s Comcast (and its very few competitors) were each local monopolists. That was an unusual trick of history; today’s innovators would do better to heed the warnings from multimedia, from personal-computer manufacturing, and from the various console-game platforms that failed to build a roster of compelling proprietary content.
VR & AR offer an inherent value that has led investors, manufacturers, and content developers to a shared confidence in its future. This distinguishes VR & AR from 3D television. 3DTV was driven by television manufacturers, who were desperate to find arguments for consumers to replace their perfectly good large flat-screen TVs. The content industry experimented in the medium, and turned away. At best, a 3DTV production could hope to resemble a 3D movie: an incremental enhancement to an already well-defined experience that remains fundamentally unchanged. And an enhancement delivered through burdensome production costs, with mixed results.
With time, the creative balances are found, and the truly valuable technologies become prevalent, even ubiquitous, exactly while they become unremarkable. A cynic might snark at the way an “aspirational buzzword” such as AI might apply only to those technologies that are not clearly feasible, but the value of a field such as AI is proven by the wide range of its alumni. The success of VR and AR similarly will be proven by the casual acceptance, to the point of disregard, with which consumers will greet the most engrossing entertainment platform, or the most enriching workplace knowledge base.
Dan Scherlis is an executive producer of health games, including the NIH-funded BreatheFree smoking-cessation intervention. Dan was founding Content Director of Comverse Mobile Games. At Turbine, he was CEO, and Producer of the Asheron’s Call MMO.
