Visual Intelligence brings iOS 17's Visual Look Up capabilities to a new developer-facing API surface in iOS 27, letting apps pipe live camera frames or static images through on-device scene understanding to extract subjects, text, barcodes, and rich semantic labels without any cloud round-trip.
• On-device privacy: all inference runs locally via Apple's Neural Engine, so no user images are uploaded to a server
• Drop-in SwiftUI view modifier surfaces real-time results with minimal boilerplate, replacing a complex AVFoundation + Vision pipeline
• Unified API spans QR/barcode, text recognition, subject lift, and scene classification under one coherent request model
Foundation Models is a new Apple framework introduced in iOS 27 that gives developers on-device access to the same Apple Intelligence language model powering system features, enabling text generation, structured output, and tool-calling entirely on-device without a network connection.
Core AI is a new iOS 27 framework that lets developers bring their own on-device AI models (vision transformers, LLMs, etc.) directly into apps, using the familiar FoundationModels session API. Models run entirely on-device with no cloud dependency, no per-token cost, and no data leaving the device.
App Schemas let developers describe their app's content and actions using pre-defined domain schemas (like the Calendar domain) so Siri can understand, search, and act on app data without custom NLP. Entities conforming to IndexedEntity are donated to Spotlight's semantic index, enabling natural-language queries over app content.
In-depth guide
iOS 27 On-Device AI & Apple Intelligence →