The FoveatedStreaming framework lets visionOS apps connect to external devices (like a PC) to stream OpenXR content with eye-tracked foveated video compression, backed by NVIDIA CloudXR under the hood. Developers build a native receiver app in SwiftUI while the heavy PC-side rendering is streamed over Wi-Fi with minimal latency.
⢠Enables existing PC-based OpenXR apps (games, simulations, design tools) to deliver immersive experiences on Apple Vision Pro without porting the full render engine
⢠Foveated compression driven by Apple Vision Pro's eye tracking intelligently allocates bitrate where the user is looking, yielding high perceived quality over Wi-Fi
⢠ARKit, RealityKit, and SwiftUI integrate natively with the stream ā letting developers blend real-world data, native UI, and on-device rendering with streamed content
Demonstrates how to create a FoveatedStreamingSession, trigger endpoint discovery and pairing, then present the streamed OpenXR content inside a visionOS ImmersiveSpace with a companion SwiftUI control window.
import SwiftUI
import FoveatedStreaming
// MARK: - App Entry Point
@main
struct FoveatedStreamingApp: App {
@State private var session = FoveatedStreamingSession()
var body: some Scene {
// Control window: pause / resume the session
WindowGroup("Controls", id: "controls") {
ContentView(session: session)
}
// Immersive space that renders the streamed content
ImmersiveSpace(id: "stream") {
StreamImmersiveView(session: session)
}
.immersionStyle(selection: .constant(.progressive), in: .progressive)
}
}
// MARK: - Control Window
struct ContentView: View {
let session: FoveatedStreamingSession
@State private var isConnected = false
@Environment(\.openImmersiveSpace) private var openImmersiveSpace
@Environment(\.dismissImmersiveSpace) private var dismissImmersiveSpace
var body: some View {
VStack(spacing: 20) {
Text(isConnected ? "Streaming" : "Not Connected")
.font(.title)
Button(isConnected ? "Disconnect" : "Connect to PC") {
Task {
if isConnected {
session.disconnect()
await dismissImmersiveSpace()
isConnected = false
} else {
// connect() presents native endpoint picker and QR pairing UI
try await session.connect()
await openImmersiveSpace(id: "stream")
isConnected = true
}
}
}
.buttonStyle(.borderedProminent)
if isConnected {
Button("Send Level Select") {
// Message channels: send opaque data to the PC host
let command = ["action": "loadLevel", "level": "canyon_01"]
if let data = try? JSONSerialization.data(withJSONObject: command) {
session.defaultMessageChannel.send(data)
}
}
}
}
.padding()
}
}
// MARK: - Immersive Space
struct StreamImmersiveView: View {
let session: FoveatedStreamingSession
var body: some View {
// Passing the session to ImmersiveSpace renders the streamed OpenXR content.
// RealityView content composites on top with correct depth occlusion.
RealityView { content in
// Add native RealityKit entities alongside the stream if needed
let anchor = AnchorEntity(.head)
content.add(anchor)
}
.foveatedStreamingSession(session)
}
}⢠The framework was introduced in visionOS 26.4, not the base visionOS 26 / iOS 27 release ā check exact availability before shipping ⢠Pairing requires the endpoint to display a QR code containing a CloudXR client token and certificate hash; scanning is gaze-based ⢠PlayStation VR2 Sense Controllers pass through via CloudXR but controller mapping must be handled on the OpenXR side ⢠Message channel payloads are opaque Data blobs ā define your own serialization contract between the visionOS app and the PC host ⢠A depth buffer and alpha channel must be provided by the OpenXR app for proper occlusion and environment blending
Requires Apple Vision Pro; streaming endpoint must run NVIDIA CloudXR SDK on Windows PC or cloud; Wi-Fi network required for cable-free streaming
More iOS 27 APIs land every week.
Get notified when new capabilities are published ā no noise, just signal.