At its WWDC 2023 conference, Apple revealed the Vision Pro, the long-awaited augmented reality (AR) headset that has many asking, “What does Apple Vision Pro do?” As one of Apple’s most ambitious undertakings, the Vision Pro has been a year in development, loaded with groundbreaking technology, including the introduction of a new operating system, visionOS, which alone could justify its significant price tag.
What does Apple Vision Pro do
The Vision Pro hsa been released on February 2, with preorders currently available on the Apple Store. Despite the nearness of its launch, the device has a surprisingly small number of natively developed apps, lacking support for major services like Netflix, Spotify, and YouTube. Nonetheless, early users of the headset can anticipate access to 3D movies and films, streaming services such as Disney+, and 180-degree videos enhanced by Apple’s Spatial Audio technology.
This brief overview serves as a guide to everything Apple has shared about the Vision Pro and visionOS, aiming to clarify what does Apple Vision Pro do for interested customers and tech enthusiasts.
Design
The design of the Apple Vision Pro immediately brings to mind high-end ski goggles, raising the question among potential users, what does Apple Vision Pro do beyond its striking appearance. The headset features a tinted, continuous front panel that wraps around the wearer’s eyes, serving both as a lens and as a concealment for the cooling system’s fan. A magnetic “light seal” enhances the viewing experience by preventing ambient light intrusion.
Functionality is at the forefront with a button and digital crown located at the back for seamless navigation and fine-tuning, including software-based pupillary distance calibration. The Vision Pro’s adaptability extends to its interchangeable band, allowing users to customize their fit with several options provided.
Display technology is a key component of this device and it’s equipped with two high-resolution displays—one for each eye—boasting a combined 23 million pixels, supplemented by an external OLED panel. Custom 3D lenses ensure the user interface remains visible, while HDR and “wide color” features aim to deliver superior image quality.
We need an Apple Vision Air ASAP: Here is why
For audio, built-in earbuds offer an immersive experience with support for Dolby Atmos and lossless audio, utilizing “audio ray tracing” to tailor sound to the environment. Apple has collaborated with Zeiss to offer magnetic lenses for prescription users, available for custom order, addressing the diverse needs of its users and further enriching the Vision Pro experience.
In terms of interaction, internal IR cameras track the wearer’s eye movements, while external cameras monitor hand movements. Lidar sensors enhance the device’s spatial awareness by detecting objects and people in real-time. This advanced tracking system supports intuitive hand gestures, allowing users to interact with virtual elements naturally, as highlighted by reports of the Vision Pro’s effective hand tracking and spatial computing capabilities.
The Vision Pro’s technical prowess is driven by 23 sensors, including a dozen cameras and six microphones, all integrated with the R1 chip. This chip is engineered to minimize lag, promising image delivery to the displays at unparalleled speeds. Despite a slight latency in the passthrough effect, it’s touted as one of the best implementations seen in a headset.
This detailed exploration not only answers what does Apple Vision Pro do but also showcases Apple’s commitment to merging design with advanced technology, setting a new benchmark in the augmented reality space.
Apps
At its debut, the Vision Pro will support a selection of apps running natively on the device, including offerings from Adobe (notably Lightroom), Microsoft (Office), and other significant developers. This lineup demonstrates Apple’s commitment to providing a diverse and functional ecosystem right from the start.
Apple’s in-house applications are primed for the Vision Pro, alongside popular third-party apps such as Temu, Uber and Uber Eats, Tinder, Notion, CNN, Washington Post, Reddit, and Discord. To encourage the development of additional apps, Apple has initiated a development kit program for members of the Apple Developer Program, addressing the query what does Apple Vision Pro do by ensuring a growing catalog of applications.
Interestingly, the Vision Pro is designed to run most iPhone and iPad apps directly from the App Store, offering a workaround for services like YouTube, Netflix, and Spotify, which haven’t yet released Vision Pro-specific apps. However, it’s important to note that while these emulated apps expand the device’s utility, they don’t fully leverage the advanced features of the Vision Pro, such as motion tracking.
Apple has also tailored several of its native apps to harness the immersive potential of the Vision Pro. Users can expect Vision Pro-optimized versions of Calendar, Mail, Messages, and FaceTime, enhancing the device’s functionality and addressing the query “What does Apple Vision Pro do?” in terms of keeping users connected and organized. The FaceTime app, for instance, uses the Vision Pro’s EyeSight feature to create a virtual persona that mimics the user’s facial expressions, adding a layer of personalization to calls.
Best Vision Pro apps so far
For productivity and entertainment, Apple is introducing enhanced versions of Messages, Safari, Freeform, Notes, and Files, along with media viewers for Photos, Music, and TV apps. The inclusion of Keynote, without the rest of the iWork suite for now, points to Apple’s focus on creating a comprehensive yet selective app experience.
The Vision Pro also offers unique content experiences, such as “Encounter Dinosaurs,” developed with Jon Favreau and the team behind the Apple TV+ show “Prehistoric Planet.” This application immerses users in a prehistoric environment, showcasing the device’s potential for educational and entertainment purposes. Additionally, the Mindfulness app, adapted from Apple’s other devices, provides a serene experience aimed at promoting relaxation and well-being.
With approximately 600 apps specifically updated for the Vision Pro, Apple is launching a new app store alongside the headset. This catalog includes essential enterprise meeting apps like Microsoft Teams, Webex, and Zoom, all supporting virtual avatars to enhance the meeting experience on the Vision Pro.
Zoom’s app, in particular, illustrates the innovative use of the Vision Pro’s AR capabilities by integrating virtual meetings into the user’s physical environment. Microsoft’s contributions, such as Word and Excel, feature simplified interfaces optimized for the Vision Pro, showcasing the device’s potential to revolutionize how users interact with traditional productivity tools through eye control and other intuitive interactions.
The early selection of third-party apps for the Vision Pro is as diverse as it is innovative, answering the broader question of what does Apple Vision Pro do with a variety of applications designed to enhance both productivity and leisure. Among these, Fantastical offers a comprehensive calendar and task management system, while JigSpace provides an augmented reality platform for creating 3D presentations that include animated slides and detailed views inside objects.
PCalc brings a robust calculator experience from the iPad, and Sky Guide allows users to explore the night sky and constellations in augmented reality. Djay transforms the AR space into a DJ’s turntable, and Complete HeartX offers an anatomical model simulator for detailed organ studies. OmniPlan caters to project management needs, Bills to Budget helps users manage their finances, Subjects organizes schoolwork, and TV Remote offers control over various TV sets and set-top boxes.
Games
The gaming experience on the Vision Pro is equally expansive, with Apple announcing that hundreds of Apple Arcade titles will be available from day one, including games like Demeo, a mixed-reality game inspired by Dungeons & Dragons, and Just Hoops, which brings arcade-style basketball into your living room. NowPlaying combines music trivia with Apple Music, while What the Golf? introduces a quirky take on golfing physics. Super Fruit Ninja integrates eye and hand tracking to enhance the classic fruit-cutting gameplay.
For TV and movie enthusiasts, the Vision Pro launch includes new and immersive video experiences. Apple introduces Immersive Video, a format that offers 180-degree 3D 8K recordings with spatial audio. This includes exclusive content like Alicia Keys: Rehearsal Room, providing an intimate look at the artist’s creative process, and Adventure, showcasing athletes in breathtaking locations. Wild Life gives a unique perspective on wildlife, and Prehistoric Planet Immersive extends the exploration to the era of dinosaurs.
Apple also facilitates access to 3D movie versions through the Apple TV app, with several streaming services like Disney+ offering 3D content at launch. This includes blockbuster films such as “Avatar: The Way of Water” and “Avengers: Endgame,” among others. Disney+’s support for the Vision Pro at launch is highlighted by its immersive viewing options, featuring environments from Disney+ Theater to iconic scenes like Marvel’s Avengers Tower.
TV experience
The early selection of third-party apps for the Vision Pro is as diverse as it is innovative, answering the broader question of what does Apple Vision Pro do with a variety of applications designed to enhance both productivity and leisure. Among these, Fantastical offers a comprehensive calendar and task management system, while JigSpace provides an augmented reality platform for creating 3D presentations that include animated slides and detailed views inside objects. PCalc brings a robust calculator experience from the iPad, and Sky Guide allows users to explore the night sky and constellations in augmented reality. Djay transforms the AR space into a DJ’s turntable, and Complete HeartX offers an anatomical model simulator for detailed organ studies. OmniPlan caters to project management needs, Bills to Budget helps users manage their finances, Subjects organizes schoolwork, and TV Remote offers control over various TV sets and set-top boxes.
The gaming experience on the Vision Pro is equally expansive, with Apple announcing that hundreds of Apple Arcade titles will be available from day one, including games like Demeo, a mixed-reality game inspired by Dungeons & Dragons, and Just Hoops, which brings arcade-style basketball into your living room. NowPlaying combines music trivia with Apple Music, while What the Golf? introduces a quirky take on golfing physics. Super Fruit Ninja integrates eye and hand tracking to enhance the classic fruit-cutting gameplay.
For TV and movie enthusiasts, the Vision Pro launch includes new and immersive video experiences. Apple introduces Immersive Video, a format that offers 180-degree 3D 8K recordings with spatial audio. This includes exclusive content like Alicia Keys: Rehearsal Room, providing an intimate look at the artist’s creative process, and Adventure, showcasing athletes in breathtaking locations. Wild Life gives a unique perspective on wildlife, and Prehistoric Planet Immersive extends the exploration to the era of dinosaurs.
Apple also facilitates access to 3D movie versions through the Apple TV app, with several streaming services like Disney+ offering 3D content at launch. This includes blockbuster films such as “Avatar: The Way of Water” and “Avengers: Endgame,” among others. Disney+’s support for the Vision Pro at launch is highlighted by its immersive viewing options, featuring environments from Disney+ Theater to iconic scenes like Marvel’s Avengers Tower.
Exploring the Vision Pro’s EyeSight capability
The EyeSight feature of the Vision Pro aims to bridge the gap between the digital and the real world, allowing wearers to maintain a connection with their surroundings. This technology renders the headset’s front panel see-through when someone approaches, enabling the wearer to see them, while also projecting the wearer’s eyes on the external display. Powered by the R1 chip, EyeSight creates a virtual avatar, or “persona,” that mimics the wearer’s facial expressions with remarkable accuracy, thanks to advanced AI modeling. This feature not only enhances interpersonal interactions but also signals to others when the wearer is engaged with the device, such as taking photos or launching augmented reality environments.
Introducing Optic ID authentication
With the Vision Pro, Apple unveils Optic ID, a pioneering approach to biometric security in augmented reality. This system leverages iris recognition technology, employing invisible LED light to authenticate the user for tasks like unlocking the device, autofilling passwords, and processing payments with Apple Pay. Ensuring maximum security, Optic ID data is encrypted and stored locally on the Vision Pro, inaccessible to any apps and never leaving the device.
Understanding spatial video and audio
The Vision Pro elevates the viewing experience with support for spatial video, a format that can be captured by the latest iPhone models and the Vision Pro itself. Unlike traditional 2D videos, spatial videos on the Vision Pro can be expanded into a fully immersive experience. This is complemented by spatial audio, Apple’s surround sound technology, which, with dynamic head tracking, creates a theater-like auditory experience. When combined, spatial video and audio offer a deeply engaging and novel augmented reality experience.
Battery
The Vision Pro distinguishes itself from other AR headsets by externalizing its battery into a portable pack, connected to the headset via a five-foot wire. This setup provides up to two hours of general use or two and a half hours of video playback, with the potential for shorter durations during intensive app or game use. For extended use, the Vision Pro can be plugged into a power source, allowing for continuous operation, albeit with the limitation of being tethered.
visionOS
At the heart of the Vision Pro’s groundbreaking capabilities is visionOS, heralded by Apple as “the first OS designed from the ground up for spatial computing.” This term encapsulates Apple’s vision for immersive augmented reality (AR) and virtual reality experiences. visionOS marries the foundational elements of MacOS and iOS with an innovative “real-time subsystem,” optimizing the processing of interactive visuals on the Vision Pro. The introduction of visionOS marks a pivotal moment in computing, offering a three-dimensional interface that liberates applications from the confines of traditional displays, allowing for a more dynamic and responsive user experience.
visionOS’s interface adeptly adjusts to natural lighting, utilizing shadows to convey scale and distance, enhancing the realism of digital interactions. Supporting a variety of input methods, including Bluetooth keyboards and trackpads as well as intuitive finger, wrist, and eye gestures, visionOS is a testament to Apple’s commitment to versatility and user-centric design. Moreover, visionOS integrates SharePlay and introduces unique features like Travel Mode and Guest User, expanding the ways in which users can share and enjoy content.
One of the most captivating aspects of visionOS is its Environments feature, which envelops users in dynamic landscapes, adjusting based on the time of day to either enhance focus or minimize distractions. The ability to customize the level of immersion through the Vision Pro’s digital crown showcases the thoughtful integration of hardware and software, pushing the boundaries of what’s possible in AR.
Availability and pricing
Available from February 2 in the U.S., the Vision Pro sets a new standard in AR with a starting price of $3,499 for the 256GB model. Additional accessories, such as Zeiss optical inserts, enhance the customization options for users, ensuring a personalized experience. Apple’s comprehensive package includes everything needed to enjoy the Vision Pro out of the box, emphasizing the company’s dedication to delivering a premium product.
Preparing for your Vision Pro experience
Prospective buyers are encouraged to visit an Apple Store for a personalized demo and fitting session, highlighting Apple’s focus on customer satisfaction and the importance of a tailored fit for the optimal use of the Vision Pro. The purchasing process includes a facial scan to determine the appropriate light seal and headbands, underscoring the bespoke nature of the Vision Pro experience. For those requiring optical inserts, a valid prescription must be provided, ensuring that the Vision Pro caters to a wide range of visual needs.
As excitement builds for the Vision Pro, Apple offers a comprehensive guided tour of its features, including EyeSight, Immersive Video, and the transformative environments of visionOS. This preview not only whets the appetite for what’s to come but also underscores the Vision Pro’s potential to redefine personal technology.
Featured image credit: Apple
