Design thinking in three dimensions.

Virtual reality is no longer in the realm of science fiction. In the last few years, the virtual and augmented reality (VR/AR) market has been heating up with big tech companies like Microsoft, Google, HTC, Facebook, and even Apple jumping in. And the technology is no longer just for gamers but is being used for everything from car sales to corporate training. Forbes predicts VR and AR headset sales will hit $80 million by 2021 and TechCrunch is making a case that AR could become the primary driver of a $108 billion AR/VR market by 2021.

Right now, the VR/AR landscape looks a bit like the wild west. Not many standards or guidelines exist, and many people are still not entirely sure what VR/AR is, and why it matters. For designers, VR/AR offers an opportunity to be in on the ground floor of creating and designing an entirely new industry — developing the design standards, rather than following preset rules outlined by someone else.

Samsung VR headset - AR VR design

Since most designers have spent years honing their craft on flat screens: computers, mobile devices, or even print design, it may seem daunting to jump into 3D design. It may be surprising to learn that many of the skills you already have apply to this new frontier.

As with any design project, it’s essential to know and understand as much as possible about the platform, the target audience, and what you (or the client) are trying to achieve. Be realistic. This is a new frontier, so don’t underestimate how long it will take to grasp the technology —– prepare to make mistakes and learn from them.

Getting Started: Immerse Yourself

The best way to get started is to experience VR/AR for yourself if you haven’t already, and if you’ve tried it before, it’s time for total immersion. Since this space is so new, it’s impossible to start designing without getting as much first-hand experience as possible.

For beginners on a budget, start by downloading VR/AR apps from the Google and Apple App stores onto your smartphone or tablet. Many apps are still free, and there’s a pretty wide variety to choose from. Pick up Google Cardboard which is the lowest cost way to start experiencing VR.

If you’re feeling a little more adventurous and have a little more money to throw around, Google Daydream and Samsung Gear VR offer more integrated and higher quality experiences on the lower end.

Virtual design - AR VR MR design

Augmented Reality & Mixed Reality (AR/MR) combines virtual world elements with the real world using headsets or mobile devices. Virtual Reality (VR) completely immerses users in virtual worlds.

For those with bigger pocketbooks, Oculus, HTC Vive, and Sony offer higher-end platforms that focus more on gaming. These are not for the faint of heart, and costs can be high, from $949 for a complete PlayStation VR bundle to several thousand dollars for a Vive or Oculus system with a high-end PC. These systems also require quite a bit of set-up and tech savvy.

As you explore, make notes on what works and doesn’t work from a UX/UI standpoint. Make sure you take into account technical limitations and whether the hardware is helping or hindering the user experience. Also, make a note of what elements you’re drawn to, do you like the total immersion of VR or do you prefer the mixed reality of AR? This will help you determine where to spend your time learning.

First Person vs. Flat Design

VR/AR is all about three dimensions, and the user interface is no longer confined to a flat rectangular screen or surface. So, it’s essential to understand spatial design and spatial UI elements a little better.

Types of spatial UI elements:

Follow along VR AR design

  • Follow-along – UI elements follow the user as they move through the environment and are not fixed. A real-world example of this would be the dashboard of your car.

World-locked AR VR design

  • World-locked – UI elements have a specifically assigned space within the environment and do not move. In the real world, examples of this are street signs, traffic lights or crosswalk indicators.

Start to think about how design in the real world can apply to the virtual world. Make sure you think about sight lines and font sizes and be prepared to do some rapid prototyping in VR to test your assumptions. Sometimes you can make do with a hotlink on an object for simple interaction; for more complex interactions you’ll need to create dynamic menus with secondary or even tertiary interactions. Gamers are already familiar with many of these concepts and grasp the importance of each type of spatial UI element.

New Inputs, New Challenges

For the first time, users are being asked to interact with a UI without tactile interaction. No more mouse, touchpad, or touchscreen. Although some methods will mimic that tactile interaction, in most cases the user will be interacting with a virtual environment through some unfamiliar inputs, so it’s important to understand how this impacts UI design.

VR design

Types of inputs:

  • Gesture
  • Gaze (eye tracking)
  • Voice
  • Hardware inputs:
    • Platform specific
    • Touch & click controllers
    • Game controllers
    • Haptic

Each platform or device will have their own unique set of input methods and may have all, or only some of the methods explored above; therefore it’s essential to understand the platform you are designing for.

Back to Basics: the Process of Virtual Design

Every great design begins the same way, no matter what the end product. Once you have a good grasp of the project and goals, start collaborating and brainstorming. Use whatever you currently use for early design concepts: whiteboards, pen and paper, or software. Collect feedback and iterate as possible to refine and narrow the focus before moving on.

AR design sketch for a dashboard

As with any design project, sketching or whiteboarding at the brainstorming stage is the best way to flesh out ideas. The image above shows initial concepts for a three dimensional AR/VR dashboard.

After initial concepts are refined, the next step is getting into the mindset of the user by creating user flows. By tracing the path that a user would take through the UI, we’re able to make sure that the path is simple and easy for them to understand.

The final stage is wireframes. Designing the way that the UI will be laid out and accessible to the user.

Start Small: Rapid Prototyping

Don’t be afraid to take sketches or wireframes into rapid VR/AR prototypes at an early stage. It will help you work out any unforeseen issues with your ideas before getting too deep into the development woods.

Some simple tools can be used to test out some of these early concepts quickly:

  • Sketch to VR – a plug-in that enables designers to use Sketch to create Google VR enabled demos. An excellent tool for creating and showing early prototypes to stakeholders.
  • Vizor – a little more advanced and allows designers to create photo- or 3D scene-based interactive demos. Storyscape walks through how to get started.
  • WebVR – an open platform that lets you experience VR in a web browser.

You can also use software such as Sketchup, Blender, and Cinema 4D to mockup and prototype VR/AR interfaces and scenes. Alternatively, if you want to jump right into one of the primary tools for design and development in VR/AR, go directly to Unity and set up a free account. It’s got a bit more of a learning curve, but there are plenty of tutorials out there to get you started.

Working in 3D - prototyping for VR AR design

Once user interface ideas begin to take shape, there are a variety of ways to prototype concepts. Shown above are 3D user interface elements that were modeled and animated in Cinema 4D before exporting to Unity Pro.

VR AR design with Unity Pro

Runtime capture from Unity Pro.

Visualizing the Virtual

As with traditional UI design, excellent visual design is critical. Apply what you know and don’t try to reinvent the wheel or change your aesthetic too dramatically. Use tools you’re already familiar with to avoid unnecessary learning curves (Adobe CC, Sketch, Framer, etc.).

Keep in mind the device the end user will use to experience your UI. Higher resolution screens and headsets (tablet AR, Oculus, Vive, PlayStation VR) allow for more detailed designs, and smaller, more elegant fonts. Lower end or small screen devices (smartphone VR/AR, Cardboard, Daydream, Gear VR) will need to use larger, thicker fonts and less detailed UI elements.

Keeping it Real

Guide users through your interface using hints, and always provide feedback with each interaction so the user gets oriented quickly. Stick with UX/UI best practices and create clean, simple, user-friendly graphics, allow your users to come to grips with virtual experiences by giving them familiar reference points and visual metaphors.

Believe it or not, after all these years moving away from skeuomorphism in design, VR/AR is bringing it back. Using a gesture to turn a dial or pick up an object feels natural to users. Just ask hardcore gamers, they’ve been interacting with virtual worlds for years.

AR design for an immersive children's book

The image above shows a mini-game made for an immersive children's book. Users grab parts and place them over the blueprint to construct robotic power legs.

Jump in Now!

There are no limits to where creative people can go in VR/AR. It’s an open field with loads of new opportunities. It’s rare that designers can get in on the ground floor of something utterly new, and with so many possibilities. Immersive technology desperately needs designers to jump in, get involved, take risks, and make something new.

• • •

Further reading:

Understanding the Basics

What is the difference between virtual reality and augmented reality?

Virtual reality (VR) immerses users in a fully artificial digital environment. Augmented reality (AR) overlays virtual objects on the real-world environment.

About the author

Ted Evans, United States
member since January 3, 2017
Ted has a passion for creative problem solving and has brought ideas to life for companies such as Sony, Apple, Nintendo, Comcast, Intel, Microsoft, Disney, Cox, and Virgin Media TV. He has a proven track record creating award-winning designs for virtually any media or platform, including print, live events, broadcast television, web, mobile, and emerging technologies (AR/VR/MR). [click to continue...]
Hiring? Meet the Top 10 Freelance UI Designers for Hire in October 2018

Comments

Tim Gentle
Congrats Ted, brilliant article. Breaks down creating VR/AR design UI/UX into manageable chunks. Will share with my community. Thanks Tim
Marco Silva
Here in Brazil very few people are adept at virtual reality because of the high costs, but I truly believe that one day, virtual and augmented reality will be so popular and so common that people will use the streets and public transportation
EvansStudios
Thanks Tim. Glad you enjoyed it. The more creative people that get involved with VR/AR, the better it will become.
EvansStudios
Hi Marco. I agree with you. As more companies embrace the technology and create platforms and hardware that have useful and fun applications, the more it will be adopted by consumers. Over time, it will be as commonplace as the smartphone or tablets are today. Thanks for reading!
comments powered by Disqus
Subscribe
Free email updates
Get the latest content first.
No spam. Just great articles & insights.
Free email updates
Get the latest content first.
Thank you for subscribing!
Check your inbox to confirm subscription. You'll start receiving posts after you confirm.
Trending articles
Relevant Technologies
About the author
Ted Evans
Designer
Ted has a passion for creative problem solving and has brought ideas to life for companies such as Sony, Apple, Nintendo, Comcast, Intel, Microsoft, Disney, Cox, and Virgin Media TV. He has a proven track record creating award-winning designs for virtually any media or platform, including print, live events, broadcast television, web, mobile, and emerging technologies (AR/VR/MR).