22 May 2021 Frank Spillers


VR and AR user experience design needs these skills

VR and AR user experience design needs these skills

The challenge of designing great user experiences for VR and AR

Designing a great VR/AR user experience requires essential skills that take advantage of the immersive, visual, and dream-like quality of this experiential media. We need to push beyond boring or unengaging experiences in 3D. It’s not enough for users to be mildly entertained, confused, unsure of what to do, or lost trying to do something. Immersive 3D experiences require judicious use of the limitations and the opportunities afforded by the technology, the environment (physical and virtual), and the user.

There are 3 key skills you need to do good VR/AR UX

  1. Storytelling- using narrative to make interaction interesting.
  2. Improvisation- being spontaneous to work with real-time interactions.
  3. ‘Imagineering’- going beyond limits for magical and fun interactions.

Let’s examine each in turn…

1. Storytelling for VR

What? Storytelling follows the structure found in oral tradition, books, and movies. This is called a “Story Arc” (with a Beginning, a Middle, and an End in its simplest form). Story orders understanding, like a user interface that indicates an action is required. Story is central to how our brains work. And it’s essential to any Holographic or Spatial UI.

Relevancy to VR/AR: Narrative acts as a task flow/ story flow “guidance” system. Narrative can be used to “unfold” your scene, or situate the user in the space, or scenario. Narrative mechanics can help shape the experience for the user in an immersive environment. We believe storytelling is a design essential for VR/AR.

Example: Rick and Morty VR- Virtual Rick-Ality. Notice the clever use of onboarding in this piece.

The story starts in a familiar setting: the living room with a TV and DVD’s that the user inserts into a DVD player. Next, a clone of Morty (you, the user) is introduced in the garage with the mission.

2. Improv for immersive experience

What? Improvisation is used in comedy and performance. Its key benefit is generating ideas or thoughts from a non-judgmental state of utilizing what’s in front of you or incorporating. “Incorporating” means continuing with whatever comes up, and adding to it. “Yes, and..” is the classic catalyst question when prompting an improvisation.

Relevancy to VR/AR: Improv can help you act out a portion of the spatial UI you are prototyping. In fact, ‘walking the space’ as you build the story experience is a common VR UX design tactic.

Example: Wolves in the Walls, a children’s story adaptation. Notice the UX of this piece includes a very “improv” onboarding experience for the user…

The experience begins with the user as co-author literally of the world. The character “draws your hands”,  you are gradually introduced to the world, to her world.

3. Imagineering for VR/AR

What? Imagineering is a Disney World approach to experience design. It is an intentional approach to creating engagement and delight. Imagineering gives you creative license to think “hyper-naturally” for VR/AR.

“Designers should strive to create magical hyper-natural interactions.  As a start, consider adding magical powers to ordinary objects to enable users to fulfill their fundamental desires—including teleportation and omniscience” -Hae Jin Lee, Hololens Interaction Designer.

Relevancy to VR/AR: Imagineering can help you think creatively about narrative from a spatial and emotional standpoint. eg Disney’s brainstorming triangle is a valuable way to evaluate your VR/AR piece

Exit Burrito


Screen Shot 2018 02 16 At 6.00.21 PM

Example: Job Simulator “exit burrito”–the way you close the game is by eating a burrito with the words “Exit” on it. Notice this “UI” fits logically and emotionally with the narrative of working in a kitchen or workplace food preparation area. Read how they imagineered the decision here…

Applying the 3 skills to VR and AR User Experience 

Designing for VR and AR adds a new level of calculated creativity to your experience design. Using Improv, Storytelling, and Imagineering techniques, you will be able to expand your repertoire of ways to help users interact with states, events, UI elements, and the overall narrative.

Design in VR/AR requires you to align your UX/UI with the story.  Think of narrative as a “UI” engagement tool. Storytelling as UI, extends to any VR/AR experience (not just games or entertainment). Remember, as you design, use spatial thinking and walk the space as Disney did… Hint: Your low-fidelity prototyping will look more like an impromptu stage-based, story walk-through.

As you design VR/AR, storytelling, improvisation, and Imagineering techniques can aid you in designing using all the deliberateness that a usability discipline brings.  Immersive media allows us to get outside the 2D box. The more you leverage these powerful skills, the more intentional and captivating your user experience will be for your users and the success of your VR/AR project.

Want to learn how to apply this to our next VR/AR project? Check out Frank Spillers  3D UX Design workshop- how to design engaging and immersive experiences

About the Author

headshot of frank spillers

Frank Spillers

Founder - UX Inner Circle

Frank Spillers, MS, founded the UX Inner Circle to share his knowledge and skills with his students from the Interaction Design Foundation where he has provided select trainings for the past 8 years. He leads UX and Service Design consulting at Experience Dynamics, an award-winning consultancy. He works with the world’s leading brands to deliver cutting-edge strategy for products, services, and experiences. Starting out in the mid-’90s in social VR, Frank has consulted on 600 UX projects including enterprise web applications, nonprofits, government and more. He’s an Inclusive Design evangelist, and expert in Accessibility, Emotion Design, VR/AR, Cross-cultural Design and UX Management. Frank brings 25 years of experience as a Sr. UX Director and Service Design leader. He has lifted conversion rates by 88% and enhanced revenue by 300% for firms like Nike, Intel, Microsoft, the City of New York, Global Disability Rights Now!, Four Seasons, Capital One, the World Bank, Women Enabled International, and many more.

Apply to start your membership at $99 or 50% Discount

for student/non-profit or Global South

Stay up to date with the UX Inner Circle, join our email list: