Android XR Will Let Apps Access The Passthrough Camera View, Google Confirms.

While headsets like Quest 3 and Apple Vision Pro use cameras to let you see the real world, today only the system software gets raw access to these cameras. Third-party developers can use passthrough as a background, sure, but they don't actually get access to it. They instead get higher-level data derived by the system, such as hand and body skeletal coordinates, a 3D mesh of your environment with bounding boxes for furniture, and limited object tracking capabilities. That means they can't run their own computer vision models, which severely limits the augmentation capabilities of these headsets.

The exception is that on visionOS 2, Apple is now giving enterprise companies raw access to Vision Pro's passthrough cameras for non-public internal apps, but this requires a special licence from Apple and is restricted to "in a business setting only".

Now, Google confirmed to veteran VR developer Antony Vitillo that Android XR apps will get access to the front camera view by requesting the same Camera permission as regular Android apps and using the same APIs.

Here's what Google told Vitillo:

https://hubs.li/Q0369bTF0