History of Augmented Reality: Glow Pucks to Immersive Reality

To download a PDF of this article, click here.

In 2011, I was designing for a global eCommerce retailer, and I still remember the heated discussions about whether people would actually buy products on a mobile device. Could they be bothered to shop for things on a phone? At the time, mobile purchases accounted for only 4% of our business’s revenue. However, as others pointed out, that number had gone from 2% to 4% in only 6 months. Fast forward a few years, and we all know how this story ends.

The reason you need an augmented reality (AR) strategy now is so you’re not catching up.

In the not so distant future, price point, versatility, and portability of AR devices will cross an important threshold—a threshold where nearly half of humanity will want and be able to afford one. As Alan Smithson, CEO of MetaVRse and Board Advisor to SXSW aptly points out, “By the end of 2019, there will be over 2 billion Augmented Reality enabled smartphones in the world.”

Maybe we have a few years before serious AR adoption kicks in. But those few years will happen quickly. Here’s what CEOs of some of the largest technology companies are saying about AR.

Tim Cook, Apple.
“I regard Augmented Reality as a big idea like the smartphone. It’s for everyone. It’s huge.”

Satya Nadella, Microsoft
“I feel Augmented Reality is perhaps the ultimate computer.”

Mark Zuckerberg, Facebook
“VR will merge with AR and become part of everyday life.”

Yet, the value of AR technology is still largely understated outside of Silicon Valley. For decades, it was only the most adventurous teams who were willing to make things on augmented reality platforms and put them on display. Starting now, that’s no longer the case.

When Microsoft released its HoloLens 2 in February 2019, the headset was about two years ahead of what people expected from a performance perspective. At Toptal, this is important news. Prior to the HoloLens 2 launch, demand for augmented reality skills had already increased 140%+ year/year across the Toptal client-base. Specific to our Fortune 500 clients, we’ve witnessed hundreds of millions of strategic investments, predominantly in developer tools and mobile AR productivity, the prevailing business case being that if you already buy from an audience, then why not sell them a new technology that makes their operations more efficient, as you’ll profit from both sides of the exchange? A perfect mandate for innovation leads.

Surprisingly, our work together involved large but simple breakthroughs on how customers spatially calibrated their phones, down to the menial gesture, and even what it means to be a telecom company in the face of 5G. But that’s a story for their articles.

The purpose of this article is to review the history of augmented reality and peer into the future of AR so that technology leaders can get ahead of the inevitable saturation of AR technologies to come.

In a nutshell, the power of AR is infinite wisdom, and we’ll come back to that point at the end. For now, let’s quickly summarize how we got here, historically speaking.

Fighter Jets, 1960

Augmented reality is the overlay of human symbols and technologies on the physical world. In this sense, it’s nearly impossible to define AR’s true origin. In 370 BC, Socrates considered writing to be an emerging technology, but a street sign to Athens doesn’t feel like AR, probably even to him. Others point to Pepper’s ghost in 1862 as the first incantation of AR. If we take a more obvious approach, the first augmented reality experiences mimic much of today’s, and they were definitely for fighter pilots.

When Tom Furness started his research in 1965 as a US Air Force officer at Wright-Patterson Air Force Base, he worked on designing better cockpits for fighter jets. In his own words, “My job was to solve the problem of getting bandwidth to and from the pilot’s brain given the sheer complexity of the systems they needed to operate in highly stressful and dangerous environments. This problem is what motivated me to explore augmented reality approaches that might increase pilot awareness of the relationship of their aircraft and to the real world. The idea was to organize and portray information in the form of virtual images projected and superimposed over the real world via helmet-worn devices.”

52 years later, in the forward to Helen Papagiannis’s “Augmented Human: How Technology is Shaping the New Reality,” Tom Furness encourages readers to ponder that “in the end, we must answer the question: Does augmenting us make our lives better?”

For pilots who needed to understand how their aircraft was operating under fire, it certainly did.

In the 1960s, AR was life or death.

Touchless Computers, 1970

If connecting planes was the first big step in AR, connecting computers was the second.

In his 2019 Design and Tech Report delivered at SXSW, John Maeda, oracle of the design industry argues that, “AR is more likely than VR, because you can [sic] see reality, etc.”

He then points to Myron Krueger’s foundational work from the mid 1970s and 80s. To John’s point, it is worth a closer look. While earning his Ph.D. from the University of Wisconsin and then while later teaching at the University of Connecticut, Myron explored the art of interaction between humans, computers, and projected images. Rather than go the joystick, keyboard, or glove route, Myron used motion, much like we see on HoloLens 2 today. In the demo from 1985, below, Myron demonstrates how, “I can type slowly”

In the 1970s and 1980s, AR was art.

Hockey Pucks, 1990

The first real AR splash made towards mass adoption was often described as among the most contested inventions in sports history. The Glow Puck made professional hockey more watchable for newcomers and people with normal TVs but outraged most serious fans. The concept is simple. Watching hockey is hard because the puck is small. When the puck traveled slowly, the Glow Puck used a proprietary camera trick to leave a bright blue trail. When the puck traveled quickly, it left a bright red one. The original Glow Puck only lasted two seasons, and yet this technique permanently cemented itself into the instant replay toolkit and elsewhere in broadcast sports.

Siblings of the Glow Puck include the NFL’s larger-than-life first-down-line, the MLB’s pitch-tracker, and similar techniques, many of which trace back to the Glow Puck’s inventor, FoxTrax.

The Glow Puck brought genuine augmented reality to the homes of hundreds of millions.

Over twenty years later, and in a recent partnership with SAP, the NHL is back at it again. Their new embedded-microchip solution debuted at the 2019 NHL All-Star Game and is slated as a staple in every arena for the 2019-2020 season.

In the 1990s, AR was sport.

Science Fiction, 2000

AR’s roots in science fiction stretch as early as 1901. In L. Frank Baum’s illustrated novel of that year, The Master Key, he describes a magical pair of glasses that sound more like a recent patent filing than a concept born at the turn of the 20th century.

“It consists of this pair of spectacles. While you wear them everyone you meet will be marked upon the forehead with a letter indicating his or her character. The good will bear the letter ‘G,’ the evil the letter ‘E.’ The wise will be marked with a ‘W’ and the foolish with an ‘F.’ The kind will show a ‘K’ upon their foreheads and the cruel a letter ‘C.’ Thus you may determine by a single look the true natures of all those you encounter.” - Frank Baum

Fast forward to the year 2000, and it’s hard to find a futurist vision of technology where augmented reality doesn’t play a foundational role. By 2002’s Minority Report, the holograms were everywhere and gesture controls ranked central to the action. Half the movie posters even gave nod to their prominence.

The promise of hands-free, ubiquitous computing has unanswered privacy, safety, and ethical implications. And with a few notable exceptions, the science fiction palette tends to be a palette of grim realities. Google, a first mover, was overwhelmed with public outcry and legislative action when they launched their first generation of Google Glass.

Like Terminator 6, you kill it and it keeps coming back. In 2015, Google Glass was shuttered by public opinion, and yet rose from the ashes to serve a new enterprise target market only two years later. Google’s new Enterprise Edition customers include DHL, General Electric, and Dignity Health. It’s the lightest device of its class.

In the 2000s, AR was invincible.

Smartphones, 2010

It was hard not to notice Pokémon Go, the augmented reality throwback by Niantic that encouraged players to get outside and catch digital monsters on their phones. Players were everywhere. You could see them from your living room window. Within one week of its July 2016 launch, Nintendo’s market cap was up $9 billion and the app had captured more daily active users than Twitter, Instagram, Facebook, and Google Maps. Nearly three years later, still, no particular game has risen to stardom as quickly. Fast-forward a bit and, while public interest has certainly waned, the cash machine is still ringing. As of March 2019, Pokémon Go revenue was estimated to be up 30% year/year which makes for yet another great addition to the $90+ billion Pokémon franchise. In simple terms, AR made the Pokémon franchise the highest-grossing media franchise ever. Even bigger than Mickey and friends.

More recently, the same foundational technologies that powered Pokémon Go are powering new eCommerce experiences like Warby Parker’s virtual try-on for eyewear. The common theme is that spatial calibration is becoming less burdensome for end users, and that subtle, incremental improvement in spatial accuracy is opening up totally new use-cases all over the place.

In the 2010s, AR is fun.

Immersive, 2020

In March 2019, supply-chain analyst Kuo Ming-Chi predicted that mass production of Apple’s first-generation AR headset, powered wirelessly by iPhone, would ramp up by Q4 2019. If history is any indication, when Apple enters the market, the new game is no longer solely R&D. Kuo’s prediction came soon after Microsoft’s expectation-shattering HoloLens 2, a device that offers plug-and-play content creation for enterprise usage, comfort, and a wide field of view.

It wasn’t long before that, that Magic Leap unveiled their highly secretive, highly anticipated, first-generation headset, Magic Leap One—a device that delivered on its promise to blur the line between what’s real and digital with advanced physics and situationally aware lighting and shading of virtual objects. If you’re not laughing after your first slingshot attempt in Angry Birds for Magic Leap, take another shot and hit something. Objects will crash and bounce off your physical space, and it’s pretty darn amusing.

All angles aside, given that Microsoft and Apple have similar market caps, vast resources, and a tradition of out-competing, it will be interesting to see whose immersive hardwear captures more market share over the next decade. For now, my bet’s on Microsoft. Easy.

That’s not to say Lenovo and others couldn’t feasibly take the reins. Lenovo’s Star Wars Jedi Challenges was a clear signal of their unique approach and strength. One thing is for certain, immersive AR solutions will get lighter, more versatile, and less expensive to produce. And then they’ll be everywhere, someday.

As of 2019, serious augmented reality hardware is heavy, hot, cumbersome, and expensive, but the Microsoft HoloLens2 is significantly more comfortable and versatile than the norm. SpaceX and Tesla CEO Elon Musk’s Neuralink promises a brain to computer interface ready for mass adoption, in a few short years. If its first use case is any indication, it’s fair to assume our species could lean heavily on this. Neuralink is helping patients with Alzheimer’s live better lives by essentially replacing functions of the hippocampus, the brain’s epicenter of both emotion and memory. Somewhere on the spectrum between today’s devices, invisible contact lenses, and brain implants, we’ll witness an immersive device so compelling nearly everyone will have one.

In describing the broader potential of his Neuralink technology, “It will enable anyone who wants to have superhuman cognition,” Musk continues, “Anyone who wants.”

Augmented reality will put the vast knowledge of science and humanity in a meaningful context, right where it matters most.

In the 2020s, AR is everywhere.

In the meantime, we have our work cut out for us.