Augmented Reality is in the midst of its moment in the sun. While Virtual Reality has had a death grip on the hype spotlight since Facebook acquired Oculus in 2014, AR has been oddly quiet. According to Gartner’s Hype Cycle, AR is in the “Trough of Disillusionment” 5-10 years away from plateau, whereas VR sits on the “Slope of Enlightenment” 2-5 years from its plateau. We believe that this accurately reflects mainstream acceptance of head-mounted AR – but mobile-based AR and other similar platforms (e.g., heads-up displays) are delivering value today across many industries under the umbrella of technologies we identify as “AR”.
AR has crept into our daily lives without the fanfare some would expect, via technologies not traditionally classified as AR: car-based heads-up displays, photo filters, etc. Indeed, much of the AR research that companies such as Apple, Microsoft, and Google, have invested in remains mostly under wraps. Shrouded in secrecy and patent filings, Magic Leap has been steadily building a mountain of hype surrounding a new generation of AR products. Products that promise to blend the virtual with the physical will compete with our own biology’s ability to tell them apart. They claim an ability to deliver a world where data is no longer restricted to a glass rectangle but is instead woven into our environment seamlessly. These visions of the future may seem distant and lofty, but we are already experiencing the early Genesis and will soon integrate these capabilities into our everyday lives.
The Back Story
For years, a single type of AR reigned supreme. Marker-Based AR, known colloquially as “QR Code AR,” has been around since the turn of the century. The idea is simple: a camera points at a fiducial marker (i.e., QR Code), the program finds the target it was looking for, and it displays the 3D model – reorienting the model to match the pose information that the camera’s perspective of the marker creates. It performs these functions as fast as the camera captures the frames, therefore updating the position, rotation, and scale of the virtual object in real time. All of this conveys the illusion that the object is in front of the camera, but this method lacks true spatial awareness of the space surrounding the marker. In order for any virtual object to persist in the physical world, it must understand the physical world’s spatial properties. Newly developed computer vision (CV) techniques alongside performance gains in modern computing hardware have enabled a new class of AR applications that can do just that.
Enter “Markerless AR”
Spearheaded by the development of the Microsoft Hololens in 2016, Markerless AR has quickly become a movement unto itself. Google experimented with a suite of devices and custom software collectively called Project Tango which, like the Hololens, used LiDAR to map environments in real time and allowed users to place virtual objects in space with respect to physical boundaries like actual floors and walls. No longer were markers needed to interact with holograms. However, this hardware proved to be quite expensive to mass-produce, especially so early in the technology’s lifecycle. The Microsoft Hololens sells for a whopping $3,000 USD, and although Tango devices did make it to market, they were never intended for consumer use. With only a handful of developers able to afford the hardware – and near zero consumer adoption – Google shut down Project Tango and the Hololens became little more than a marketing tool for Microsoft (albeit a powerful one!). However, everything changed in mid-2017 when Facebook, Apple, Google, and Snapchat each announced their own Markerless AR solutions for mobile devices.
While Facebook and Snapchat added world tracking features to their existing camera apps, Apple developed an entire AR platform from the ground up for iOS. While ARKit doesn’t allow for some advanced features that the Hololens supports such as cross-session persistence (saving room scans and recognizing them automatically), or head-mounted holograms (still a handheld iPhone), it did effectively eliminate AR’s barrier to entry. For the first time, consumers had instant access to high-quality Markerless AR content. With consumer adoption in hand, developer interest piqued; and the largest market for immersive content to ever exist was instantly created.
2018 and Beyond
Now in 2018, there is more interest in Augmented Reality than ever before. Google answered Apple’s ARKit with ARCore, and in a few months, Magic Leap will release the first consumer-facing untethered Markerless AR Head-Mounted Display. Soon, we will be interacting with the world in ways we could have never imagined, dwarfing the creativity of fantasy and science fiction, prompting the query from future generations: “What is a screen?”