“the thing will get super hot, like the hottest this thing ever gets … You feel it. You feel it. Like the sun is burning your face” -Brad Lynch on the Meta Quest Pro

Augmented Reality's (AR’s) dirty little secret is that it is an energy beast, and consuming all that energy generates a ton of heat.

An AR device has to do four core things:

Identify where it is in an environment
identify the things in the environment and where they are in 3D space
generate virtual 3D content that is integrated with things in the real world
display that content.

Each of these tasks consumes a lot of energy.

I develop AR apps for Apple’s iPhone and iPads, and I’ve been disappointed that after several minutes of use the frames per second (fps) - how often the screen refreshes - drops from 60 fps down to 30 fps and eventually 20 fps.

30 fps, and definitely 20 fps, create bad experiences.

Eventually, I learned Apple throttles the frame rate when the iPhone or iPad gets too hot, so I thought maybe I was doing something wrong.

This year Apple came clean. They said an AR experience should only last for at most 1-2 minutes, in part because of thermal issues.

What?!

OK, it does make sense, especially for a device like an iPhone that is super compact, has no ventilation, and is probably wrapped in a case too. It just cannot cool itself off. The iPhone is a computing powerhouse, but it can only be a powerhouse for short periods of time.

But that design constraint - AR experiences only lasting for at most 60-120 seconds - is going to force me to rethink the design of my apps.

More Than Just iOS Devices

And it is not just Apple’s iOS devices. See the full quote below on Meta Quest Pro baking your face.

And those amazing real-time 3D PC graphics you see in videos online? Dell’s Alienware Aurora R15 Gaming Desktop comes with a 1350 Watt power supply. 1.35 Kilowatts! The heat generated by 1.3 Kilowatts is intense. Don’t expect stand-alone AR headsets to generate 3D content anything like demos you see online generated by computers consuming 1.3 Kilowatts.

AR and generating high-quality 3D content consumes a lot of energy, and consuming a lot of energy means creating a lot of heat.

Don’t expect the dream of lightweight AR glasses you can wear all day to come true anytime soon, if ever.

Apple Reality Pro

What about Apple’s rumored Reality Pro headset? It supposedly has a fan. Hopefully it will be able to dump all the heat without baking your face or singing your hair with the exhaust.

Effectively getting rid of heat for a device sitting on your face may be one of the most underappreciated challenges to creating a successful AR experience.

After the recent dismal reviews of the Meta Quest Pro, the huge financial losses at Meta’s Reality Labs division, the drastic cutbacks in Microsoft’s mixed reality efforts including the disbanding of their Mixed Reality Toolkit team, this field really needs a win.

References on the Heat Issue

Brad Lynch’s review of the Meta Quest Pro

Jump to the 7:16 point in the video

The entire color passthrough overlaying also takes about 35% of overall processing just to do that overlay color feed.
And the thing will get super hot, like the hottest this thing ever gets is if you use this thing with AR mode for maybe 10 minutes or more.
You feel it. You feel it. Like the sun is burning your face in some instances.

WWDC22: Qualities of great AR experiences

Jump to the 11:43 point in the video:

Finally, craft experiences that last no more than a minute or two.
This is for the ergonomic reasons I mentioned earlier, but also because AR is a resource-intensive superpower that has a big impact on battery and thermals.

Create parametric 3D room scans with RoomPlan

Jump to the 15:16 point in the video:

Another thing to keep in mind is battery and thermals of the device.
We have done many optimizations on RoomPlan API to ensure a good scanning experience.
Nevertheless, it's best to avoid repeated scans or single long scans over 5 minutes.
These could not only cause fatigue but also drain out the battery and create thermal issues which might in turn impact the user experience of your app.