RealityKit & USDKit in iOS 27: Spatial Computing for iOS Developers

iOS 27 introduces new SwiftUI drag and drop APIs including reorderable, reorderContainer, dragContainer, and configuration modifiers that enable reordering within and across collections, multi-item drag, and fine-grained control over how data is transferred during drag and drop operations.

PaperKit is Apple's full-featured canvas framework — previously internal-only — now publicly available in iOS/macOS/visionOS 27. It powers the drawing and markup experience in Notes, Preview, and Freeform, giving developers access to a complete pencil, shapes, images, and text canvas with a rich data model.

USDKit is a new first-party Swift framework introduced in iOS/macOS 27 that brings native USD scene creation, composition, modification, and export capabilities to Apple platform apps, with deep RealityKit and Spatial Preview integration.

Reality Composer Pro 3 introduces a node-based visual scripting system called Script Graph that lets developers and designers build interactive RealityKit experiences — including full games — without writing any code. Features include drag gesture events, physics integration, prototyped subgraphs, custom events, and live preview directly on Vision Pro.

visionOS 27 introduces ClippingComponent and enhanced ManipulationComponent placement patterns in RealityKit, enabling multiple users in a SharePlay session to simultaneously manipulate, disassemble, and cross-section structured 3D assemblies in shared space.

visionOS 27 introduces Gaussian Splatting support in RealityKit, enabling developers to scan real-world objects and render them as photorealistic 3D Gaussian splats directly in their spatial experiences. This bridges the gap between physical reality and virtual content without requiring manual 3D modeling.

The Evaluations framework in Xcode 27 lets developers generate and validate synthetic test data at scale using the SampleGenerator API, then run robust evaluations against agentic workflows that involve tool calling — all from Swift code.

iOS 27 introduces APIs and standards for building live production tools for Apple Immersive Video, enabling developers to capture, transport, record, and play back 3D immersive video using ProRes over SMPTE 2110 with spatial audio and per-frame metadata. The format combines ProRes-encoded stereo video (2110-22), ASAF spatial audio (2110-30), and JSON metadata (2110-41) into a unified production pipeline.

visionOS supports fully photo-realistic, spatially-aware 360° immersive environments with true depth, parallax, spatial audio, and motion — going far beyond static panoramas. Developers can now surface these environments on both native visionOS apps and the spatial web.