Thread by @BartronPolygon

Let's talk about the most important feature of the Apple #VisionPro that is getting the least attention right now: high quality spatial audio. Apple understands audio and TDG's VP, Mike Rockwell, was formerly a VP at Dolby. Look the size of those speaker drivers! (1/)

Audio is a huge part of our perception of space. The soundscape around us helps us localize objects both directly -- when they emit sound, esp. beyond our visual field -- and indirectly, when reflections give us a sense of the dimensions and composition of our 3D space. (2/)

Spatial audio is an important cue that grounds virtual objects in our space. This isn't just important for making a dinosaur in our room feel present. It helps orient abstract objects, like app UI, allowing a unified mental model of the real and virtual. Less mental work. (3/)

Like most HMDs, AVP has extra-aural speakers that sit above the ears and directionally beam audio into them. The spatiality of a sound is caused by the relative delay between when it is picked up by our biological stereo microphones ("ears") as well as their shape. (4/)

Our heads and ears are all unique and our brains have learned to interpret audio that passes through them. This is modeled by a head-related transfer function (HRTF). In this simple top-down version, it is a function of angle and distance to the sound source per ear. (5/)

Reportedly, AVP will allow for personalized HRTFs. I can't think of any other commercial system with this capability. Systems like HoloLens and Meta Quest use a generic HRTF. But wait, there's more! AVP takes things *even* further... (6/)

Sound is affected by the environment: reflected by surfaces and modulated by materials it passes through. With depth sensors and computer vision, AVP can build a simplified model of your space and perform audio ray-tracing from a virtual object to your ears! (7/)

All of this excites me for 3 reasons: 1) unrivalled immersion, 2) lower cognitive load tracking 3D UX, and 3) it opens up some awesome pro/creator use cases that I'll talk about more next week! Where are my SFX and music designers at? The spatial future *needs* you! :) (8/8)

DISCLAIMER ADDENDUM: This and other threads are me speaking as an unaffiliated and independent AR developer, citing public info only. My knowledge on this is from WWDC but I’ve leveraged spatial audio in my personal HL and Quest work. Excited to hear for myself when it’s out :)