These are the guidelines I am currently following when developing extended reality (XR) applications. They are based on my personal experiences developing and using AR and VR apps, reading about others’ experiences, and watching others’ experiences.

A collection of videos of what can go terribly wrong in XR is provided at the end.

Briefly:

In VR, don’t move virtually
In VR, don’t move physically
In VR, don’t look or turn around quickly
Avoid first person shooters (FPS) and sports in VR
Discourage rapid arm movements like punching or slashing motions
Design for physical safety
Design for trolls
Design for shared experiences
In Augmented Reality, virtual objects should interact with the real world
Limit Mixed Reality experiences for phones and tablets to 1-2 minutes
Limit Mixed Reality interactions for phones and tablets
For Apple’s Mixed Reality headset, design companion (lite) apps that run on an iPhone or iPad

(1) In VR, don’t move virtually

Virtual movement is where you physically stay in one location, but the virtual version of you moves through an environment. Unfortunately, for a large percentage of users, this mismatch between movement perceived by the eyes and lack of movement perceived by the inner ear and the rest of the body induces nausea. 🤮 Nausea is perhaps the number one complaint I’ve seen for VR.

(2) In VR, don’t move physically

Don’t encourage the user to physically move around a room while in VR. When you are cut off from the real world, moving is dangerous. You can run into furniture, walls, trip over pets, or simply lose your balance and fall over. People often get so immersed in the VR experience that they will kick, run, jump, or dive while completely oblivious to the dangers of obstacles around them. See list of videos at the end of this post. 🤕

(3) In VR, don’t look or turn around quickly

When you are cut off from the real world, turning quickly can be very disorienting, and this can cause people to fall over or lose track of where they might be in the room.

(4) Avoid first person shooters (FPS) and sports in VR

Guidelines (1), (2), and (3) pretty much rule out most of the AAA games that have dominated 3D gaming on PCs and game consoles for years. As Epic Games CEO Tim Sweeney said:

The thing that we do in Fortnite every day as gamers is run through an environment rapidly, and it’s the kind of experience that involves intense motion and doesn’t work as well in VR. (link)

Two apps with the most long-term success on Meta’s Quest headset - Beats Saber and Supernatural - follow rules 1-3. The user largely stays in one place, primarily looks forward, and does not move virtually through an environment.

(5) Discourage rapid arm movements like punching or slashing motions

There is a real danger of the user hitting solid things like walls and furniture (and thus seriously hurting their hands) as well as accidentally punching friends or family.

If you do design an experience with rapid arm movements, every time, encourage the user to first check to see if there is plenty of empty space around them and that people (especially kids) can’t walk in front of them.

For a ton of examples of people’s injured hands or people accidentally punching someone, see the videos and reddit list referenced at the end of this post.

(6) Design for physical safety

Guidelines 1-5 are largely about designing experiences for physical safety. Fortunately, mixed reality headsets can compensate for dangers by changing the amount of the real world the user sees.

For example, a virtual cafe app could let a user enjoy a coffee or a meal while mostly immersed in a virtual environment, perhaps sharing the virtual reality space with a friend in another city. The area on the table where the coffee or food is could be shown in real reality, while everything else is in virtual reality (maybe the user is at a sidewalk cafe in Paris).

But if the user stands up to go get some more coffee, the mixed reality headset could automatically switch to almost complete real reality (that is, the user sees their actual room) perhaps leaving the friend displayed as a virtual reality avatar in their room. The user could safely navigate their kitchen, pour some coffee, and then return to the table. As the user sits down at the table, the headset switches back to the virtual reality location.

(7) Design for trolls

One of the most powerful aspects of virtual reality is the sense of presence. The feeling that someone is right there with you is so much more powerful than any other interaction with a computer.

The down side of this powerful sense of presence is that harassment in virtual reality feels much more powerful too.

(8) Design for shared experiences

Consistently, I’ve heard the thing that people enjoy most about virtual reality is the sense of presence they have with someone who may be thousands of miles away. Do keep (7) in mind, however.

(9) In Augmented Reality, virtual objects should interact with the real world

To create the illusion of reality, virtual objects should in some way interact with the real world. This can include casting a shadow on real world surfaces, dropping from a few inches and then landing on a table or floor, bouncing off surfaces, being occluded by real world objects like people and furniture, emitting sound that seems to originate from where the object is, looking at the user, and when moving through a room, seemingly intelligently moving around furniture and through doors (instead of walking through furniture or walls like a ghost).

(10) Limit Mixed Reality experiences for phones and tablets to 1-2 minutes

Holding a phone or tablet up in the air can get very tiring for the user very quickly. Also, VR and especially AR consume a lot of energy. This drains the battery quickly and causes the device to heat up. iPhones and iPads compensate for this heat by dropping the frame rate which creates a less magical experience.

So for phones and tablets, mixed reality experiences should only be smaller parts of a larger experience.

(11) Limit Mixed Reality interactions for phones and tablets

The user is typically holding the phone or tablet with both hands, so the user cannot comfortably and safely hold the device (i.e., avoid dropping their device, especially tablets) with one hand while interacting with objects on the screen with the other hand.

(12) For Apple’s Mixed Reality headset, design companion (lite) apps that run on an iPhone or iPad

Apple has been developing and delivering libraries (e.g., ARKit and RealityKit), content creation tools (e.g., RoomPlan), and tooling for AR and VR on iPhone and iPad for years. Apple’s mixed reality headset will certainly heavily leverage and then extend the same capabilities that are available on iPhones and iPads.

The user base for Apple’s mixed reality headset will be small for many years (I’ve seen estimates of a million headsets sold in the first year). Meanwhile, there are about a billion iPhones and iPads that support augmented and virtual reality.

For example, an architecture mixed reality app for a headset could be designed for prolong interactive experiences in a house or building. An iPhone/iPad companion app could allow an architect’s clients to experience part of the environment (e.g., a single room) for a 1-2 minute viewings without needing to buy a full headset.

Don’t ignore a possible billion users of your mixed reality experience, but just remember guidelines (10) and (11).