Mixed Reality

Apple Vision Pro

Elisabeth Balistreri
Jun 6, 2023

Top questions regarding Apple's scheduled 2024 release of the mixed reality headset named the Apple Vision Pro.

Apple Vision Pro Mixed Reality Headset

June 5th 2023 - WWDC Announcement Recap

During the WWDC event, Apple made a significant announcement about its highly anticipated AR headset called Apple Vision Pro. This groundbreaking device will carry a price tag of $3,499 and is set to hit the market in early 2024. Following an extensive seven-year development process, Apple has finally unveiled its AR headset, marking a remarkable moment reminiscent of the iconic "one more thing" reveals. While the effectiveness and future success of this product are yet to be determined, one thing is clear: Unlike the long-speculated Apple television and car, the Apple Vision Pro is real and finally here.

During his introduction of the new headset at WWDC 2023, Apple CEO Tim Cook emphasized that Vision Pro liberates users from the limitations of traditional displays. Unlike previous reports suggesting a focus on mixed reality, this system places a stronger emphasis on augmented reality rather than virtual reality. Apple refers to this paradigm shift as "spatial computing," an innovative way to interact with the user interface, similar to how a mouse or trackpad operates. Visually, the headset closely resembles ski goggles and is evidently designed for indoor use, rather than for wearing outdoors or on the go.

The headset features an aluminum frame with a curved glass front. It incorporates a physical button for capturing images and a digital crown for adjustments. The rear portion consists of a flexible strap, while a visor extends from the display to the wearer's head, effectively blocking out external light. Integrated into the sides are "audio pods" that provide sound output. This marks a significant moment for spatial audio, shedding light on Apple's prior emphasis on this technology. The headset runs on the M2 chip, the standard processor for Apple's devices, coupled with a new R1 chip dedicated to video streaming. The micro-OLED displays boast an impressive density of 64 pixels per unit, roughly equivalent to that of a standard iPhone pixel. The three-element lens system aims to enhance imaging quality from all angles. In collaboration with Zeiss, Apple offers custom prescription glass inserts, eliminating the need for users to wear glasses while using the headset. A notable feature called "EyeSight" uses a front-facing display to reveal the user's eyes to others in the room, compensating for the opaque visor and providing an authentic representation of the user on the curved front glass. To achieve this, an initial facial scan is conducted, and the resulting image serves as the user's avatar when interacting with others wearing the headset.

Despite having mixed reality capabilities, the company appears uninterested in engaging in the virtual reality domain. Curiously, the Apple Vision Pro seems primarily tailored for work-related tasks rather than gaming. It is worth noting that Apple enables users to project a version of their Mac desktop onto the headset, reinforcing the company's efforts to avoid introducing a touchscreen Mac.

The opaque display of the headset creates a darkened environment around images. Apple particularly emphasizes the integration of 2D photos captured with iPhones. Additionally, the device features a built-in 3D camera, enabling users to capture "Spatial" photos and videos. A movie theater option takes advantage of the illusion of a large, immersive screen. While gaming is supported, it is not in the form of a traditional VR experience but rather as a projection on a large screen, accommodating standard games. Disney is the first content partner for the headset, indicating promising prospects for future experiences. During the event, Bob Iger, the entertainment mogul, took the stage to announce that Disney+ will be available on the Apple Vision Pro at launch, showcasing his confidence in Apple's latest innovation.

Contrary to earlier rumors suggesting an adapted version of iPadOS, Apple introduced VisionOS, a new operating system specifically designed "from the ground up" for spatial computing. Similar to the initial launch of the iPad, VisionOS incorporates various educational features, including apps related to astronomy, health, and music creation.

Apple states that the Vision Pro system will be compatible with Microsoft Office apps and teleconferencing services such as WebEX and Zoom. VisionOS is built on the same framework as iOS and iPadOS, ensuring seamless compatibility for existing apps. Additionally, Apple will launch a dedicated App Store for the headset and has partnered with Unity to support game development tools, offering an enhanced experience compared to merely porting existing games.

Who will be the target audience of Apple’s mixed reality headset?

Patent drawings and leaked information from Apple points towards an enterprise-facing headset to start that resembles the HoloLens 2 more so than the Meta Quest Pro–which uses video pass through viewing and hand controllers instead of see through viewing and hands free control like the HoloLens 2.

Apple has always been a consumer-facing brand, and ease of use has always been their biggest value to consumers over other brands. As mixed reality hardware developers continue to experience breakthroughs in the technology that make it more user friendly, the development towards consumer headsets will pick up speed.

For example, the HoloLens 2 was the first headset with the capability to operate without being tethered to a computer–a giant leap towards being able to wear this technology wherever we go. Of course the bulky feel of current headsets must be reduced in order for these to target consumer audiences, but the momentum towards this outcome continues to grow.

It makes sense for Apple to begin with an enterprise-facing headset with which they can continue to develop breakthroughs that bring us closer to consumer mixed reality hardware. Until they reveal what they have been working on, we won’t know how far into this race they have made it.

When will Apple announce their mixed reality glasses?

It is hard to say exactly. Every prediction so far has been incorrect. The latest rumor is an announcement at WWDC in June. This lines up with the long-time predicted release date for their first headset sometime in 2023/24. Apple's mixed reality will be exclusively manufactured by Pegatron, a Taiwanese electronics manufacturing company. Time will tell. But, with the release of Meta Quest Pro, It’s doubtful Apple will want to continue to drag their feet. 

What will Apple’s mixed reality headset be called?

A legal firm that has represented Apple in many other trademark filings across the globe has recently filed trademark applications for “Reality Pro”, “Reality Processor”, and “Reality One”.

Which of the rumored glasses is Apple going to release first?

Most likely to be released first is the Apple Vision Pro mixed reality headset that will rival the Microsoft HoloLens 2, Magic Leap 2 and the Meta Quest Pro in the high end mixed reality device market.

What is the name of the mixed reality operating system for Apple’s AR MR glasses?

The name most likely according to trademark filings and leaked code is RealityOS. However, more recent developments propose Apple has pivoted to the name xrOS for their new extended reality operating system.

What are some of the features and specs expected to be released in the Apple mixed reality headset?

Potential features include two 4K micro-OLED displays, 14 cameras, two main processors, eye tracking, hand gesture control, and object tracking. The headset is equipped with two processors: a main system on a chip (M2) and a video processing unit (ISP). The ISP is believed to efficiently stream and correct the distorted image captured by the integrated cameras, enabling high-quality mixed reality experiences. The low latency of this process is essential for smooth mixed reality interactions.

Describe the displays that will be in the new Apple mixed reality headset:

The headset will feature two ultra-high-resolution displays developed with Sony Group for VR, and external cameras for AR "pass-through mode," allowing users to see the real world. Custom lenses for prescription glasses will also be available. The device will have a "Digital Crown" similar to the Apple Watch, allowing users to switch between VR and AR.

What will the construction of Apple's mixed reality headset look like?

The Apple XR headset is reported to be constructed from aluminum, glass, and carbon fiber, making it both lightweight and compact. Its power source is a waist-mounted battery that can be easily swapped out while in use. This unique design was reportedly created by Jony Ive, Apple's former chief designer.

What will the Apple Vision Pro design look like?

They will most likely be similar to the Meta Quest Pro with the front glass, with a resemblance to a pair of ski goggles. Supposedly the outward-facing cameras are concealed better than the Quest Pro and the overall device will come in thinner and lighter than the Quest Pro, which weighs 722 grams / 1.59 pounds. 

Why is Apple releasing a mixed reality headset before an augmented reality headset?

As reported by the Financial Times, Cook chose to proceed with the product development despite facing opposition from the company's design team who advocated for delaying the launch until AR glasses technology was more advanced. In contrast, the operations department favored releasing a mixed reality headset first. Ultimately, Cook aligned with operations chief Jeff Williams.

How much will the Apple mixed reality headset cost?

Early information suggests that the headset will cost around $3,000. This puts it at the higher end range closer to the prices of HoloLens 2 ($3700+) and Magic Leap 2 ($3,299+). The question is does Apple try and push the cost lower towards the Meta Quest Pro price ($1,499)? 

Why do organizations need a mixed reality strategy specific to Apple products?

It is likely that Apple will continue to operate within a “closed ecosystem.” In other words, they will continue to control software compatibility with their hardware in a way that will differ from other hardware platforms. That’s why it’s so important for organizations to have a mixed reality strategy that focuses both on Apple specifically, as well as other platforms.

When Apple comes out with their first headset–which is rumored to target enterprise customers–organizations should already have a strategy in place to immediately begin developing consumer-facing programs for Apple headsets. Why? Because even an enterprise facing headset from Apple will open the door for organizations to begin preparing for mainstream adoption of mixed reality. 

Right now, Microsoft’s HoloLens 2 dominates the mixed reality market for enterprise solutions; however–assuming they are able to produce a quality product–Apple will in all likelihood corner the consumer market for mixed reality headsets. Apple currently corners the market for smartphones in the U.S., and depending on the quality of the product they are able to produce in mixed reality, it’s likely they will maintain or even grow this market share as mixed reality is adopted by consumers.

What is Apple's mixed reality patent USPTO number 11570417?

Apple's patent details a system that enhances video fidelity in the area where a user is looking and reduces the graphical intensity in their peripheral vision. This could help conserve processing power and extend the battery life of the Vision Pro headset.

Who is manufacturing the headset for Apple?

According to a recent report by Nikkei Asia, Apple is deviating from its usual practice of relying on Taiwanese suppliers Foxconn or Pegatron for the development of its first-generation products. Instead, Apple is said to be partnering with Chinese company Luxshare for its upcoming mixed reality headset.

The report states that Luxshare has taken over the development team in Shanghai that was previously owned by Pegatron. Five anonymous sources who are familiar with the matter provided this information.

What is the field of view for Apple's headset?

According to reports, the field of view for the headset is a generous 120 degrees, wider than the Meta Quest Pro's 106 degrees and significantly wider than open augmented reality headsets like the Magic Leap 2 or Hololens, which have a field of view of around 50 degrees.

When will Apple Glasses come out?

Apple was said to be developing true AR glasses after the Vision Pro VR/MR headset, but plans for those may have been delayed in favor of a more cost-effective mixed reality VR headset. This is reportedly following the release of the first expensive Vision Pro model by Apple. We do not have a crystal ball... but, our best estimate is 2024/25 for when we'll finally see true AR glasses announced from Apple.

How many headsets will Apple make?

According to market research firm Counterpoint Research, it is estimated that Apple will ship less than 500,000 headsets in a year.