Could Meta Quest 3 Lite Support Color Mixed Reality?

Republished By Plato

Followers: 0

Meta's next headset needs to support color mixed reality, and hints suggest it will.

Mixed reality is a headline feature of Quest 3. Meta even markets Quest 3 as "the first mainstream mixed reality headset". Despite this, most of the Quest Store remains VR-only. Some apps offer passthrough as an optional background instead of a virtual environment, but few developers offer full mixed reality content that truly integrates with the geometry of your physical space.

That's because most Quest customers are still on Quest 2, but only Quest 3 can automatically generate a 3D mesh of your environment. More basic room-aware mixed reality is possible on Quest 2, but requires manually marking out your walls and furniture in an arduous and imprecise process most users simply won't want to bother with. And of course, it's in black and white, which just isn't appealing.

Quest 3 Lite Should Expand The Mixed Reality Market

A leaked Meta roadmap last year revealed Meta planned to release a new headset after Quest 3 in 2024 "at the most attractive price point in the VR consumer market", a project called Ventura. Reports from The Wall Street Journal, Bloomberg, and a Chinese analyst who has been reliable in the past suggest this headset will feature the Snapdragon XR2 Gen 2 chipset from Quest 3 but use the old fresnel lenses from Quest 2, to hit a low enough price to replace Quest 2 in Meta's lineup.

The Chinese analyst suggested Quest 3 Lite will only have black and white passthrough like Quest 2, but there's reason to believe he's mistaken.

By including the XR2 Gen 2 chipset, Meta will enable Quest 3 Lite to run the same VR content at the same quality level as the main Quest 3. From a developer perspective, there would be no difference. But if Quest 3 Lite didn't have similar mixed reality capabilities, developers would have to treat it very differently if integrating mixed reality capabilities.

Supporting the idea that Quest 3 Lite must include mixed reality, last week a Meta engineering director seemed to directly hint at all future Quest headsets supporting mixed reality. In a recent Meta blog post, Director of Engineering for XR Tech Paul Furgale said "after Quest 3, I’m convinced that Passthrough and MR will be a standard feature on all future headsets.”

While the dual 4-megapixel color cameras on Quest 3 may have some special properties such as a global shutter, it's unlikely they are any significant expense. The real cost in passthrough is having a chip that can efficiently process those 8 million pixels per frame efficiently, and the XR2 Gen 2 handles that.

Even Without A Depth Projector

What could start to drive the price up beyond Meta's target of a Quest 2 replacement however is also including an IR depth projector. But there is evidence that Quest 3 Lite will not include a depth projector.

Quest 3 was codenamed Eureka, just as:

Quest Pro (the result of Project Cambria) was codenamed Seacliff
Quest 2 was codenamed Hollywood
Oculus Quest (the result of Project Santa Cruz) was codenamed Monterey
Oculus Go was codenamed Pacific

Back in October, Quest firmware dataminer Samulia discovered a new headset codenamed Panther. Samulia previously discovered many of Quest Pro's specs a year before it launched, as well as Quest 3's resolution and 3D room meshing capabilities.

The reference to Panther was in a debug feature for "simulating" it on Quest 3 (Eureka). Applying this setting, according to the debug log text, prevents the depth projector being used.

Feature found in the Quest firmware by dataminer Samulia.

Many people believe that Quest 3's depth projector is an essential part of how its passthrough works, but this isn't true. In reality, it's only used for generating the 3D scene mesh at room setup. The realtime depth map used for reprojecting the passthrough, and exposed by the new Depth API to support dynamic occlusion, is in fact generated by a computer vision software algorithm that compares the views from the two frontal greyscale tracking cameras.

And in theory, the depth projector shouldn't be necessary to even generate the 3D scene mesh at all. Estimating depth without hardware-level depth sensing is an area of rapid improvement in the field of computer vision today, and the fact that Quest 3 is already doing this suggests the same estimate could be used to populate the scene mesh. Though it would likely be less accurate than Quest 3 and require you to get closer to each wall by walking near it, these seem like acceptable tradeoffs for a markedly lower-cost device.

To me, that's what the simulation text above strongly implies. And going to this effort would make little sense for a headset with only black and white passthrough.

Vital For Meta's Mixed Reality Ecosystem

If Meta abandoned mixed reality in its next device after making it the primary focus of Quest 3 it would send a signal to developers that it's simply not a technology worth developing for, killing any real chance of Meta building a mixed reality content ecosystem to set the stage for competition with Apple Vision Pro and Samsung's Google-powered headset.

With color passthrough, Quest 3 Lite becomes a cheaper Quest 3 with a bulkier design and less sharp lenses. A lower quality but not a fundamentally different device. It would move both VR forward and expand the market for mixed reality. While it's of course possible Meta could value ruthless VR cost-cutting over its mixed reality goals, the hint from Paul Furgale and finding of Samulia suggests that won't be the case.