visionOS 27 expands ARKit object tracking with high frame rate tracking, metric-space pose queries, a new extended training mode in Create ML, custom spatial accessory support, and brings object tracking to iOS for the first time.
• Added high frame rate tracking via new ReferenceObjectConfiguration API in ARKit on visionOS
• Introduced ARKit Coordinate Space Correction API to return object poses in metric space without display corrections
• Brought object tracking to iOS 27 using the existing ARWorldTrackingConfiguration with detectionObjects and trackingObjects
• Added extended training mode in Create ML's Object Tracking template for increased accuracy when tracking handheld items
• High frame rate tracking and metric-space pose queries (ARKit Coordinate Space Correction API) unlock precise spatial measurement use cases like surgical training and home remodeling
• Object tracking arrives on iOS 27, letting developers reuse the same USDZ reference objects across visionOS and iOS apps with minimal code changes
• The new spatial accessory framework lets anyone build custom tracked hardware (steering wheels, medical probes, etc.) that communicates with Apple Vision Pro via Bluetooth + LEDs + IMU
Demonstrates loading a reference object on iOS 27, running an ARKit world tracking session, and reading the object anchor's metric-space transform when detected.
import ARKit
import RealityKit
import UIKit
class ObjectTrackingViewController: UIViewController, ARSessionDelegate {
var arView: ARView!
var session: ARSession!
override func viewDidLoad() {
super.viewDidLoad()
arView = ARView(frame: view.bounds)
arView.autoresizingMask = [.flexibleWidth, .flexibleHeight]
view.addSubview(arView)
session = arView.session
session.delegate = self
guard let referenceObjectURL = Bundle.main.url(
forResource: "flashlight",
withExtension: "referenceobject"
) else {
print("Reference object not found in bundle")
return
}
ARReferenceObject.referenceObjects(
inGroupNamed: nil,
bundle: Bundle.main
)
let referenceObject = try! ARReferenceObject(archiveURL: referenceObjectURL)
let configuration = ARWorldTrackingConfiguration()
// Use trackingObjects for high frame rate tracking of moving/handheld objects
configuration.trackingObjects = [referenceObject]
// Use detectionObjects for mostly stationary objects
// configuration.detectionObjects = [referenceObject]
session.run(configuration, options: [.resetTracking, .removeExistingAnchors])
}
// MARK: - ARSessionDelegate
func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
for anchor in anchors.compactMap({ $0 as? ARObjectAnchor }) {
print("Object detected: \(anchor.referenceObject.name ?? "unknown")")
attachVirtualContent(to: anchor, in: arView)
}
}
func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
for anchor in anchors.compactMap({ $0 as? ARObjectAnchor }) {
// iOS 27: obtain pose in metric space without display corrections
// Use .none for measurement tasks; .rendered for visual alignment
let metricTransform = anchor.transform(for: .none)
let renderedTransform = anchor.transform(for: .rendered)
let position = SIMD3<Float>(
metricTransform.columns.3.x,
metricTransform.columns.3.y,
metricTransform.columns.3.z
)
print("Metric position (meters): \(position)")
}
}
func session(_ session: ARSession, didRemove anchors: [ARAnchor]) {
for anchor in anchors.compactMap({ $0 as? ARObjectAnchor }) {
print("Object removed: \(anchor.referenceObject.name ?? "unknown")")
// Remove any entities tied to this anchor
}
}
private func attachVirtualContent(to anchor: ARObjectAnchor, in arView: ARView) {
let anchorEntity = AnchorEntity(anchor: anchor)
let mesh = MeshResource.generateBox(size: 0.02)
let material = SimpleMaterial(color: .systemBlue, isMetallic: true)
let modelEntity = ModelEntity(mesh: mesh, materials: [material])
anchorEntity.addChild(modelEntity)
arView.scene.addAnchor(anchorEntity)
}
}Extended training mode in Create ML takes significantly longer than standard mode; metric-space poses (CoordinateSpaceCorrection.none) should only be used for measurement, not for aligning virtual content visually; spatial accessories require a validated USDZ annotated with LED and IMU positions before Create ML training
visionOS features require Apple Vision Pro; iOS object tracking requires a device with LiDAR or compatible ARKit support; spatial accessories require custom hardware with LEDs, IMU, and Bluetooth
More iOS 27 APIs land every week.
Get notified when new capabilities are published — no noise, just signal.