Apple adds the Steam Frame's most anticipated feature, foveated streaming, to the Vision Pro
When I got my hands on the Steam Frame last year, I was filled with excitement. The new Valve VR headset promised a fresh take on home VR with a focus on convenience and ease-of-use—in large part led by a technology called foveated streaming. We're yet to see the Steam Frame due to ongoing launch and pricing issues, but foveated streaming is clearly a good idea. Now even Apple wants in.
Foveated streaming works by limiting the bitrate of streamed data based on where a user is looking. The fovea being the part of your eye with the clearest vision—a device requires eye tracking for this to work. It's useful for devices that wirelessly receive data from another source, as wireless connections are more limited than wired in terms of bandwidth and reliability.
It works on a similar principle to foveated rendering, or more generally, variable rate shading. These technologies lower demands on hardware by rendering different parts of the screen at varying rates of detail. Sony's PS VR2 uses foveated rendering.
A good spot by 9to5mac here, Apple's developer documentation now contains an entry for foveated streaming in the visionOS 26.4+ beta for Apple Vision Pro.
It says:
"The Foveated Streaming framework provides a session-based API for establishing connections from Apple Vision Pro to local and cloud streaming endpoints. The endpoint host streams high quality content only where necessary based on information about the approximate region where the person is looking, ensuring performance."
That sounds about the same. A varying level of detail for streamed content based on a user's vision. In Apple's case, either content from a local PC or streamed from the cloud.
Apple makes mention of one peculiar but interesting use case here. The Vision Pro overlays windows on top of the real or virtual worlds, and using both streaming and rendering together could lower demands for the headset.
Here's Apple's example:
"A flight simulator app can render a cockpit using RealityKit, and stream a processor-intensive landscape from a remote computer to the device."
Essentially, using it to stream background content while freeing up the Vision Pro's processor for any windows that are a user's main focus. Another good way to utilise this clever technology, by the sounds of it, and unfortunately, we may see Apple's uber-expensive headset reap the benefits of it before we do.
The Steam Frame is caught up in the memory crisis. Valve has said it is unable to announce the release date or price tag for any of its upcoming products, including the Steam Machine and Steam Controller. It's also recently added a note to the Steam Deck store page that the plucky handheld may be unavailable in some regions due to shortage in memory and storage supply.