Capabilities

Foundation Models is a new Apple framework introduced in iOS 27 that gives developers on-device access to the same Apple Intelligence language model powering system features, enabling text generation, structured output, and tool-calling entirely on-device without a network connection.

Liquid Glass is Apple's new material and visual design language introduced in iOS 27, bringing translucent, refractive glass-like surfaces to system and custom UI elements. It replaces the frosted vibrancy aesthetic with a more dynamic, depth-aware material that responds to content beneath it.

Visual Intelligence brings iOS 17's Visual Look Up capabilities to a new developer-facing API surface in iOS 27, letting apps pipe live camera frames or static images through on-device scene understanding to extract subjects, text, barcodes, and rich semantic labels without any cloud round-trip.

iOS 27 opens the Foundation Models framework to third-party LLM providers via a new public LanguageModel protocol, allowing anyone to integrate server-based or local models (e.g. Claude, Gemini, CoreAI, MLX) using the same API as Apple's on-device system model.

PaperKit is Apple's full-featured canvas framework — previously internal-only — now publicly available in iOS/macOS/visionOS 27. It powers the drawing and markup experience in Notes, Preview, and Freeform, giving developers access to a complete pencil, shapes, images, and text canvas with a rich data model.

Xcode 27 introduces an agentic localization workflow that lets you ask a coding agent to translate your entire app directly inside Xcode, leveraging String Catalog context — including where and how strings are used — to produce accurate, consistent translations across all languages.

Dynamic profiles let you switch language models, instructions, and tools within a single LanguageModelSession, enabling multi-agent orchestration patterns like baton-pass and phone-a-friend directly in your app. Combined with session properties and lifecycle modifiers, you can build context-aware, multi-stage AI workflows on-device and in the cloud.

SpotlightSearchTool is a new Foundation Models tool that lets a language model directly query your app's Core Spotlight index, enabling conversational, context-aware search over your app's own donated content. Developers can drop it into any LanguageModelSession to get AI-generated answers grounded in app data.

Visual Intelligence now lets apps register as image search providers via App Intents, returning matched entities when users highlight and search images. Introduced in iOS 26 and now expanded to iPadOS and macOS in iOS 27, apps can surface rich, ranked results directly in the Visual Intelligence UI.

Core AI is a new iOS 27 framework that lets developers bring their own on-device AI models (vision transformers, LLMs, etc.) directly into apps, using the familiar FoundationModels session API. Models run entirely on-device with no cloud dependency, no per-token cost, and no data leaving the device.

App Schemas let developers map their app's entities and actions to predefined Siri-understandable structures, grouped into App Schema Domains (e.g. messages, photos, mail), so Siri can find content, answer questions, and execute actions in natural language without any custom NLU code.

iOS 27 introduces security patterns and APIs for protecting agentic features built with Foundation Models and App Intents, including lifecycle event modifiers to inject deterministic security checkpoints into LLM-driven agent execution loops.

iOS 27 opens the Foundation Models framework to third-party LLM providers via a new public LanguageModel protocol, enabling anyone to integrate custom, server-based, or open-source models using the same Swift API as Apple's on-device system model.

App Schemas let developers describe their app's content and actions using pre-defined domain schemas (like the Calendar domain) so Siri can understand, search, and act on app data without custom NLP. Entities conforming to IndexedEntity are donated to Spotlight's semantic index, enabling natural-language queries over app content.

USDKit is a new first-party Swift framework introduced in iOS/macOS 27 that brings native USD scene creation, composition, modification, and export capabilities to Apple platform apps, with deep RealityKit and Spatial Preview integration.

A new Apple framework for measuring the quality and reliability of intelligent features powered by generative AI. It integrates with Swift Testing to let developers define datasets, metrics, and optimization targets that automatically assess probabilistic outputs from language models.

Core AI is Apple's new on-device inference framework for iOS 27, giving developers direct access to the same high-performance ML execution engine that powers Apple Intelligence. It supports CPU, GPU, and Neural Engine across all Apple Silicon devices.

iOS 27 introduces new SwiftUI drag and drop APIs including reorderable, reorderContainer, dragContainer, and configuration modifiers that enable reordering within and across collections, multi-item drag, and fine-grained control over how data is transferred during drag and drop operations.

iOS 27 integrates heart rate and cycling power zone tracking directly into HealthKit, automatically calculating time spent in each intensity zone during workouts and delivering live zone-change updates to your app.

SwiftUI now makes it straightforward to compose rich, animated visual pipelines by combining Metal shaders (colorEffect, distortionEffect, layerEffect) with TimelineView for per-frame time-driven animation and alignment guides for precise floating-view attachment. These building blocks chain together like pipes to produce production-quality effects without needing a separate rendering engine.

Swift 6.3 and 6.4 introduce a range of language and library improvements including anyAppleOS availability syntax, module selectors for disambiguation, task cancellation shielding, improved Swift Testing interoperability with XCTest, and a new Foundation ProgressManager type.

The Evaluations framework in Xcode 27 lets developers generate and validate synthetic test data at scale using the SampleGenerator API, then run robust evaluations against agentic workflows that involve tool calling — all from Swift code.

Retention Messaging lets developers configure custom messages, images, and promotional offers that appear on the App Store cancellation page when a subscriber is about to cancel. It supports both static configuration via App Store Connect and real-time server-driven responses through the Retention Messaging API.

The HTML <model> element is a native browser element that lets web developers embed interactive 3D USDZ models in Safari on iOS, iPadOS, and macOS — no JavaScript library required. It was previously visionOS-only and now ships across all Apple platforms in iOS/iPadOS/macOS 2026.

iOS 26/27 introduces Liquid Glass, a new design language that separates apps into a UI layer (navigation, tab bars, toolbars) and a content layer (scroll views, imagery, color). This architecture lets brand color and content breathe edge-to-edge while native controls float above and dynamically pick up brand colors.

Safari 27 introduces Customizable Select, allowing developers to apply `appearance: base-select` CSS to fully style the native HTML `<select>` element and its picker, including custom HTML content inside `<option>` elements, while retaining built-in accessibility and form semantics.

Reality Composer Pro 3 introduces a node-based visual scripting system called Script Graph that lets developers and designers build interactive RealityKit experiences — including full games — without writing any code. Features include drag gesture events, physics integration, prototyped subgraphs, custom events, and live preview directly on Vision Pro.

iOS 27 introduces the `accessibilityLinkedGroup` SwiftUI modifier, which lets developers connect multiple separate text views so VoiceOver and Speak Screen navigate across them seamlessly as a single continuous reading experience.

iOS 27 introduces APIs and standards for building live production tools for Apple Immersive Video, enabling developers to capture, transport, record, and play back 3D immersive video using ProRes over SMPTE 2110 with spatial audio and per-frame metadata. The format combines ProRes-encoded stereo video (2110-22), ASAF spatial audio (2110-30), and JSON metadata (2110-41) into a unified production pipeline.

CSS Grid Lanes is a new web layout mode available in Safari 26 that enables masonry-style layouts natively in CSS — no JavaScript required. Items flow into multiple columns or rows, each settling into the shortest available lane while preserving natural proportions.

iOS 27 expands App Intents to deeply integrate with Siri's on-device language model, enabling natural-language invocation of app actions with richer parameter resolution and multi-step task chaining. Apps can now expose capabilities that Siri understands contextually without rigid phrase matching.

iOS 27 brings a sweeping set of SwiftUI enhancements including Liquid Glass design adoption, new toolbar visibility and overflow APIs, document architecture improvements with Observable, and resizable iPhone app support — all designed to modernize app UIs with minimal code changes.

iOS 27 adds sectioned queries, codable model attributes, ResultsObserver for non-SwiftUI change observation, and HistoryObserver for reacting to persistent history changes in SwiftData.

iOS 27 brings significant updates to Live Activities and the Dynamic Island, including new expanded presentations, richer interactivity via App Intents directly from the island, and updated layout APIs for more expressive minimal and compact views.

MetricKit has been rebuilt from the ground up in iOS 27 with a contextually rich, Swift-first API that delivers metrics and diagnostics as async streams, plus new capabilities like Metal frame rate metrics, memory exception diagnostics, and per-state metric breakdowns via the StateReporting framework.

iOS 27 expands the App Intents framework with RelevantEntities for contextual content surfacing, EntityCollection for high-performance batch parameter handling, SyncableEntity for cross-device Siri continuity, native Duration/PersonNameComponents parameter types, and @UnionValue for multi-type parameters — plus extended intent execution time limits.

iOS 27 extends App Intents with new APIs for custom Siri dialog responses, interaction donations, OwnershipProvidingEntity for smart confirmations, and IndexedEntityQuery for semantic Spotlight indexing — enabling richer, more personal Siri and Apple Intelligence experiences.

iOS 27 enforces stricter adaptivity requirements for UIKit apps, making iPhone apps fully resizable in iPhone Mirroring and on iPad, while introducing new navigation bar minimization controls, sidebar opt-in for iPhone tab bars, and prominent tab customization.

iOS 27 (and macOS 15) brings the systemExtraLargePortrait widget family — previously visionOS-only — to iPhone, iPad, and Mac, giving widgets a tall canvas to display rich app content. Developers can now support this new family size alongside existing families with minimal code changes.

visionOS supports fully photo-realistic, spatially-aware 360° immersive environments with true depth, parallax, spatial audio, and motion — going far beyond static panoramas. Developers can now surface these environments on both native visionOS apps and the spatial web.

iOS/macOS 2026 adds NSGestureRecognizerRepresentable, NSHostingMenu, and NSHostingSceneRepresentation, making it easier than ever to incrementally adopt SwiftUI inside existing AppKit apps without a full rewrite. Observable tracking in NSView draw methods is also enabled by default.

iOS 27 introduces NSTextViewportRenderingSurface and NSTextViewportRenderingSurfaceKey protocols, and makes UITextView/NSTextView conform to NSTextViewportLayoutControllerDelegate — enabling developers to override viewport layout lifecycle methods to build rich extensions like line-number gutters without leaving the safety of framework text views.

iOS 27 introduces updated search field placement patterns with Liquid Glass styling, including ergonomic bottom-toolbar search that animates over the keyboard, prominent Search Tab options, and adaptive toolbar search that collapses to a button on constrained layouts.

Xcode 27 introduces a fully customizable toolbar and theme system, untitled scratch projects, coding agent integration directly in the editor, and the new Device Hub for evaluating apps across simulators and physical devices side-by-side.

macOS 27 ships a pre-installed `fm` command-line tool and a pip-installable Python SDK that expose the on-device Apple Foundation Model outside of Swift apps, enabling shell scripts, automation pipelines, and Python-based ML evaluation workflows.

macOS 27 introduces NSTextSelectionManager for custom text selection in any view, while reinforcing best practices for gesture recognizers, keyboard navigation, and NSWindowRestoration to make AppKit apps feel native and resilient.

tvOS 27 introduces system-wide Dynamic Type support, allowing users to choose preferred text sizes (Large through Accessibility XXXL) in Settings. Apps built with standard UIKit and SwiftUI text styles adapt automatically, while custom layouts can respond via the dynamicTypeSize environment value.

visionOS 27 introduces Gaussian Splatting support in RealityKit, enabling developers to scan real-world objects and render them as photorealistic 3D Gaussian splats directly in their spatial experiences. This bridges the gap between physical reality and virtual content without requiring manual 3D modeling.

visionOS 27 introduces ClippingComponent and enhanced ManipulationComponent placement patterns in RealityKit, enabling multiple users in a SharePlay session to simultaneously manipulate, disassemble, and cross-section structured 3D assemblies in shared space.