# Agent skill (/docs/agent-skill)
If you build with an AI coding agent — Claude Code, Cursor, and the like —
install the swift-ai-sdk skill. It hands the agent the API surface, the patterns,
and a per-topic reference tree, so it writes correct swift-ai-sdk code instead of
guessing from memory.
## Install [#install]
```bash
npx skills add zaidmukaddam/swift-ai-sdk
```
Or copy the `skills/swift-ai-sdk/` folder from the repo into your project's
`.claude/skills/` (or wherever your agent loads skills).
## What's inside [#whats-inside]
A `SKILL.md` router plus a `references/` tree — one file per topic, so the agent
opens only what the task needs:
* Text generation, structured output, and the Schema DSL
* Tools, provider-executed tools, and agents (loop control, `prepareCall`, `toolOrder`)
* Providers, reasoning, and middleware (including the cache)
* Chat UI, transport, realtime voice
* Media, on-device models, testing, and errors
It mirrors these docs and is verified against the source, so it stays accurate as
the SDK moves.
# Agents (/docs/agents)
`Agent` is the `ToolLoopAgent` analog: a model bundled with instructions,
tools, and loop settings, callable many times.
```swift
let agent = Agent(
model: AnthropicModel("claude-sonnet-5"),
instructions: "You are a terse weather assistant.",
tools: [weatherTool],
maxSteps: 6
)
let result = try await agent.generate(prompt: "Weather in Mumbai?")
let stream = agent.stream(messages: history)
```
Everything `generateText` accepts, `Agent` captures up front: `toolChoice`,
`activeTools`, `toolOrder`, `toolsContext`, sampling settings, `reasoning`,
`stopWhen`, `prepareCall`, `prepareStep`, `onStepFinish`, and
`providerOptions`.
## Configuring a call [#configuring-a-call]
`prepareCall` runs once, before the loop, to reconfigure the whole call from
runtime inputs — swap the model, rewrite messages, change tools or sampling.
Return only what you want to change; everything else falls through. (Its
step-level sibling is `prepareStep`.)
```swift
let agent = Agent(
model: OpenAIModel("gpt-5.6-luna"), // default: fast
tools: [search, calculator],
prepareCall: { ctx in
isHardQuestion(ctx.messages)
? PrepareCallResult(model: OpenAIModel("gpt-5.6-sol"), reasoning: .high)
: nil
}
)
```
`prepareCall` and `prepareStep` are also parameters on `generateText` and
`streamText` directly, not just on `Agent`.
## Tool order [#tool-order]
`toolOrder` fixes the order tools are sent to the provider — useful when a
model is sensitive to tool position. Listed tools come first in that order;
anything unlisted is appended alphabetically.
```swift
Agent(model: model, tools: [search, calc, weather], toolOrder: ["weather", "search"])
// provider sees: weather, search, calc
```
## Subagents [#subagents]
Any agent becomes a tool for another agent.
The orchestrator keeps its own context window clean and delegates focused
tasks to specialists that run their full loop and return only their final
text.
```swift
let researcher = Agent(
model: model,
instructions: "You research questions and answer with dense facts."
)
let writer = Agent(
model: model,
instructions: "You turn notes into friendly prose."
)
let orchestrator = Agent(
model: model,
instructions: "Plan the work, delegate to specialists, then combine.",
tools: [
researcher.asTool(name: "researcher", description: "Delegate research."),
writer.asTool(name: "writer", description: "Delegate drafting.")
]
)
```
## Agents as chat transports [#agents-as-chat-transports]
`Agent` conforms to `ChatTransport`, so a chat UI can run against it
directly with no server, the in-process analog of serving an agent from a
route:
```swift
@State var chat = ChatSession(transport: agent)
```
UI messages convert to model messages, the agent's loop runs, and events
come back as UI message chunks. Regeneration, tool approvals, and
client-side tool results all work identically to the HTTP transport.
# Chat UI (/docs/chat-ui)
A chat screen needs three things: a message list that updates as tokens
arrive, a way to send, and a status for the spinner. `ChatSession` is all
three in one `@Observable` object you keep in `@State`. Two siblings
cover the smaller cases: `CompletionSession` for single-turn text and
`ObjectSession` for streamed structured output.
## ChatSession [#chatsession]
```swift title="ChatView.swift"
@State private var chat = ChatSession(
// Your server's chat route — the same one a web `useChat` app calls.
transport: HTTPChatTransport(api: URL(string: "https://your-app.com/api/chat")!)
)
// in the view:
ForEach(chat.messages) { message in
MessageView(message: message) // UIMessage: text, reasoning, tools, files
}
Button("Send") { chat.send(input) }
```
`chat.status` drives spinners (`submitted`, `streaming`, `ready`, `error`;
`isLoading` folds the first two), `chat.stop()` cancels,
`chat.regenerate()` replays, and messages arrive as `UIMessage` values
whose parts update token by token.
A few more pieces worth knowing: `send(_:)` is sugar for
`sendMessage(.user(text))` — build a `UIMessage` yourself to attach file
parts or metadata; `setMessages(_:)` hydrates a persisted conversation;
the session takes an `id:` so reconnects and resumption target the right
chat. `HTTPChatTransport` accepts `headers:` (auth) and a `body:` object
merged into every request alongside the messages.
For anything that has to be computed per request — a refreshed token, a
trimmed history, a different URL — pass `prepareSendMessagesRequest:` (or
`prepareReconnectToStreamRequest:`), which runs right before each call and
whose headers and body fields win over the static ones:
```swift
HTTPChatTransport(
api: api,
prepareSendMessagesRequest: { request in
PreparedChatRequest(
headers: ["authorization": "Bearer \(await auth.freshToken())"],
body: ["messages": .array(request.messages.suffix(10).map(\.wire))]
)
}
)
```
The wire format is the same UI message stream your web chat already
speaks, so an existing chat route serves the app without changes.
### What that URL points to [#what-that-url-points-to]
The `api:` URL is an AI SDK chat route — a POST endpoint that takes
`{messages}` and streams UI message chunks. If you don't have one yet,
it's \~15 lines on any framework the AI SDK supports:
```ts title="app/api/chat/route.ts"
import {
streamText, UIMessage, convertToModelMessages,
createUIMessageStreamResponse, toUIMessageStream,
} from 'ai';
export async function POST(req: Request) {
const { messages }: { messages: UIMessage[] } = await req.json();
const result = streamText({
model: 'anthropic/claude-sonnet-5',
messages: await convertToModelMessages(messages),
});
return createUIMessageStreamResponse({
stream: toUIMessageStream({ stream: result.stream }),
});
}
```
Serving from Swift instead? The
[streaming protocol page](/docs/streaming-protocol) builds the same
route with this library's server helpers. No backend at all? Use a
[local transport](#local-transports).
## UIMessage anatomy [#uimessage-anatomy]
A `UIMessage` is an id, a role, optional metadata, and typed parts you
switch over when rendering:
| Part | Renders as |
| -------------------------------- | -------------------------------------------------------- |
| `.text(TextUIPart)` | Assistant or user text; `state` is `streaming` or `done` |
| `.reasoning(ReasoningUIPart)` | Thinking, streamed the same way |
| `.tool(ToolUIPart)` | A tool call with its lifecycle state |
| `.sourceURL` / `.sourceDocument` | Citations |
| `.file(FileUIPart)` | Attachments (data URLs or remote) |
| `.data(DataUIPart)` | Custom data parts your server writes |
| `.stepStart` | A step boundary marker |
Tool parts walk a state machine you can render precisely:
`inputStreaming` while arguments stream, `inputAvailable` once callable,
`outputAvailable` or `outputError` after execution, and
`approvalRequested`, `approvalResponded`, `outputDenied` for
human-in-the-loop flows.
```swift
case .tool(let tool):
switch tool.state {
case .inputStreaming: ToolSpinner(name: tool.toolName)
case .approvalRequested: ApprovalPrompt(tool: tool)
case .outputAvailable: ToolResultView(output: tool.output)
default: EmptyView()
}
```
## Tool results and approvals [#tool-results-and-approvals]
Client-side tools and human-in-the-loop approvals ride the protocol:
```swift
chat.addToolResult(toolCallID: id, result: ["status": "done"])
chat.addToolApprovalResponse(approvalID: id, approved: true)
```
## Resuming interrupted streams [#resuming-interrupted-streams]
`HTTPChatTransport` implements the standard reconnect contract
(`GET {api}/{chatId}/stream`); call `chat.resumeStream()` when the app
comes back to the foreground to pick up a response that kept generating
while it was away.
## Local transports [#local-transports]
Every session also runs against an in-process model with no server:
```swift
let chat = ChatSession(model: FoundationModelsModel(), tools: [weather])
let completion = CompletionSession(model: AnthropicModel("claude-sonnet-5"))
```
Any `Agent` is a transport too, which is the usual way to get tools into a
local chat.
## CompletionSession and ObjectSession [#completionsession-and-objectsession]
```swift
@State var completion = CompletionSession(
transport: HTTPCompletionTransport(api: url)
)
completion.complete("Write a tagline for a coffee shop")
// completion.completion grows as tokens arrive
@State var object = ObjectSession(
transport: HTTPObjectTransport(api: url)
)
object.submit("Generate a recipe")
let recipe: Recipe? = object.decoded() // partial-JSON repaired as it streams
```
# Context management (/docs/context-management)
Three tools for keeping a long run inside the window, in increasing order of
cost and fidelity: pruning deletes, compaction summarizes, and `contextWindow`
tells both of them how much room they actually have.
For what a context window is and why a long run fills one, see
[models and tokens](/docs/foundations/models-and-tokens).
## Pruning history [#pruning-history]
`pruneMessages` drops old reasoning and tool traffic before a long loop grows
past the window. It is a pure function, so you can call it in `prepareStep`
or before each turn:
```swift
let trimmed = pruneMessages(
messages,
toolCalls: .beforeLastMessages(6, tools: ["search"])
)
let uiTrimmed = pruneMessages(
uiMessages, reasoning: .beforeLastMessage, toolCalls: .all
)
```
Scopes are `.all`, `.beforeLastMessage`, `.beforeLastMessages(n)`, and
`.none`; dropping a tool call also drops its result. Reasoning pruning applies
to `UIMessage` histories, since `Message` carries no reasoning parts.
Pruning is lossy by design: it deletes. When you need the *substance* of the
history to survive, use compaction instead.
## Compaction [#compaction]
`compaction:` on `generateText`, `streamText`, and `Agent` keeps a long run
inside the context window without throwing away what the run has learned. It
triggers on its own when the history outgrows the working-set budget:
```swift
let agent = Agent(
model: AnthropicModel("claude-opus-5"),
instructions: "…",
tools: [readFile, searchCode, sendEmail],
compaction: Compaction(
budget: .init(workingSet: 0.5, compacted: 0.2),
keepLastSteps: 4,
onCompact: { event in
print("compacted \(event.estimatedTokensBefore) → \(event.estimatedTokensAfter)")
}
)
)
```
The organizing rule is **compress in proportion to how cheaply the information
can be recovered**. Bulk tool output can be re-fetched, so it compresses hard;
a decision's rationale exists only in the transcript, so it is preserved.
Three layers run in order, and the first two cost nothing:
1. **Structural pinning.** System messages, the first user message (the goal),
and any message carrying a failed tool result are lifted out of the
compressible span. `keepLastSteps` messages stay verbatim as the working set.
2. **Live-reference scan.** An older message that shares an identifier (a file
path, a symbol, an id) with the working set is still live, so it is pinned.
The thing you are actively working on survives without a model deciding.
3. **Typed extraction.** One `generateObject` call over what remains, using the
run's own model, producing a `CompactedContext`.
Layers 1 and 2 are the `pinning:` option set: `.firstUserMessage`, `.errors`,
`.liveReferences`, all three by `.default`. Drop one to compact more
aggressively.
`onCompact` receives a `CompactionEvent` carrying `messagesBefore` /
`messagesAfter`, `estimatedTokensBefore` / `estimatedTokensAfter`, the extracted
`context`, and `pointerizedTools`, the tools whose output became a pointer.
### What survives [#what-survives]
`CompactedContext` is a schema, not a prose summary. `goal`, `decisions`, and
`deadEnds` are required fields, so extraction cannot silently drop them:
| Field | Holds |
| ------------------ | ---------------------------------------------------------------------- |
| `goal` | The original task, restated. Required, so it is never compressed away. |
| `constraints` | Requirements and prohibitions the user stated. |
| `decisions` | Each choice made, with the reason it was made. |
| `establishedFacts` | Findings so far, with the tool that produced them. |
| `deadEnds` | Approaches already tried and why they failed. |
| `openQuestions` | What still blocks the task. |
| `artifacts` | Files and records by reference, never inlined. |
Dead ends are the entry most summarizers lose and the most expensive loss: an
agent that forgets a failed approach retries it.
Compaction is re-entrant: the previous `CompactedContext` feeds the next
extraction, so a long run converges instead of growing.
### Idempotent tools [#idempotent-tools]
A tool result is replaced by a pointer only when its tool is marked idempotent:
```swift
let readFile = Tool(name: "read_file", description: "…", parameters: schema) { args in
…
}
.idempotent()
```
`read_file`'s 4KB body becomes `[omitted: read_file returned ~4200 characters —
re-run the tool to retrieve it]`. The **tool call itself stays verbatim** even
then, because it records what was already tried. Tools are not idempotent by
default, so anything with side effects (a payment, a send) keeps its output in
full.
### Budget [#budget]
`CompactionBudget` allocates fractions of the context window rather than setting
one threshold. Leave `contextWindow` unset and it comes from the model:
```swift
Compaction(budget: .init(contextWindow: 32_000, workingSet: 0.5))
```
So one config behaves correctly across models: an Opus run gets a 500K working
set and a Haiku run gets 100K.
## Context windows [#context-windows]
`LanguageModel` exposes `contextWindow`. The default implementation resolves it
from the model id, so every provider pack reports a window without any
per-provider wiring:
```swift
AnthropicModel("claude-opus-5").contextWindow // 1_000_000
XaiModel("grok-4").contextWindow // 256_000
```
Unknown model ids fall back to `ModelContextWindows.conservativeDefault`
(128K), deliberately below the common 200K. Under-estimating compacts early;
over-estimating overflows the window and fails the request.
Register a window for a local, fine-tuned, or newly released model:
```swift
ModelContextWindows.register(64_000, for: "my-finetune")
```
Registered values beat the built-in table. A model type that knows its own
window can override the property instead.
# Embeddings (/docs/embeddings)
Embedding models conform to `EmbeddingModel`, so OpenAI, Cohere, and custom
OpenAI-compatible embedding endpoints use the same functions.
## Embed one value [#embed-one-value]
```swift
let model = OpenAIEmbeddingModel("text-embedding-3-small")
let result = try await embed(model: model, value: "sunny day at the beach")
result.embedding // [Double]
result.usage // token accounting
```
## Embed many values [#embed-many-values]
`embedMany` preserves input order and can split inputs into provider-sized
batches:
```swift
let result = try await embedMany(
model: model,
values: documents,
maxBatchSize: 96
)
result.embeddings // one vector per input
```
The batches run sequentially, retry independently, and combine their usage.
## Similarity [#similarity]
```swift
let query = try await embed(model: model, value: "coastal weather")
let documents = try await embedMany(model: model, values: values)
let ranked = zip(values, documents.embeddings)
.map { value, vector in
(value, cosineSimilarity(query.embedding, vector))
}
.sorted { $0.1 > $1.1 }
```
`cosineSimilarity` returns `0` for empty vectors, mismatched lengths, or a
zero-norm vector.
## Models [#models]
| Provider | Model type | Key |
| --------------------------------------------------------------- | ------------------------------------------------ | ----------------------------- |
| [OpenAI](/docs/providers/openai) | `OpenAIEmbeddingModel("text-embedding-3-small")` | `OPENAI_API_KEY` |
| [Cohere](/docs/providers/cohere) | `CohereEmbeddingModel("embed-v4.0")` | `COHERE_API_KEY` |
| [Custom compatible endpoint](/docs/providers/openai-compatible) | `OpenAIEmbeddingModel(..., baseURL: ...)` | Explicit or provider-specific |
If you already have candidate documents and want a provider to reorder them,
use [reranking](/docs/reranking).
# Errors and retries (/docs/errors)
Failures surface as typed `AIError` cases:
```swift
do {
let result = try await generateText(model: model, prompt: prompt)
} catch let AIError.http(status, body) {
// 4xx/5xx with the provider's error body
} catch AIError.noObjectGenerated {
// structured output did not parse or validate
} catch {
// transport errors, cancellation
}
```
Other cases include `.decoding`, `.invalidRequest`, `.unknownTool`, and
`.transport`. The tool layer adds `.invalidToolInput`, `.invalidToolContext`,
`.missingToolResults`, `.toolCallRepairFailed`, and `.invalidToolApproval`;
`.timedOut(scope:limit:tool:)` reports which
[timeout](/docs/timeouts-and-approvals) fired,
`.unsupportedFunctionality` marks a capability the provider or platform does
not offer, and `.authorizationRequired(url:)` carries the sign-in URL when an
[MCP](/docs/mcp) server needs OAuth. Streaming surfaces errors by throwing from
the stream you iterate.
## Retries [#retries]
Every request path retries transient failures with exponential backoff.
`maxRetries` (default 2) counts retries after the first attempt; set 0 to
disable:
```swift
let result = try await generateText(
model: model,
prompt: prompt,
maxRetries: 4
)
```
Establishing a stream is retryable; a stream that already delivered parts
is not, so you never see duplicated tokens.
# Files and skills (/docs/files-and-skills)
Swift AI includes focused clients for provider resources that live outside a
model request.
## Referencing an upload in a message [#referencing-an-upload-in-a-message]
`uploadFile` works against any client conforming to `FileUploadAPI` and tags
the result with the provider, so the upload can be dropped straight into a
message:
```swift
let uploaded = try await uploadFile(
api: OpenAIFiles(),
data: pdf,
filename: "report.pdf",
mediaType: "application/pdf"
)
let answer = try await generateText(
model: OpenAIModel("gpt-5"),
messages: [Message(role: .user, content: [
.text("Summarize this."),
.file(uploaded.file(mediaType: "application/pdf"))
])]
)
```
`uploaded.file(mediaType:)` and `uploaded.image()` build content parts whose
`providerReference` maps a provider name to its file id (`["openai":
"file_123"]`). A provider that recognizes its own key sends the id — OpenAI as
`file_id` on an `input_file` / `input_image`, Anthropic as a `file` source —
and a provider that does not simply ignores the reference, so a reference
minted for one provider is never leaked to another.
## OpenAI files [#openai-files]
```swift
let files = OpenAIFiles() // OPENAI_API_KEY
let uploaded = try await files.upload(
data: pdf,
filename: "report.pdf",
purpose: "user_data",
mediaType: "application/pdf"
)
print(uploaded.id)
try await files.delete(id: uploaded.id)
```
`OpenAIFiles` uses `https://api.openai.com/v1` by default. The returned
`UploadedFile` contains the provider ID plus filename and size when available.
See the [OpenAI provider page](/docs/providers/openai) for model APIs.
## Anthropic files [#anthropic-files]
```swift
let uploaded = try await AnthropicFiles().upload(
data: pdf,
filename: "report.pdf",
mediaType: "application/pdf"
)
```
`AnthropicFiles` reads `ANTHROPIC_API_KEY` and adds the required Files API beta
header. See the [Anthropic provider page](/docs/providers/anthropic).
## Anthropic skills [#anthropic-skills]
Upload a `SKILL.md` folder as reusable provider-hosted content:
```swift
let skill = try await AnthropicSkills().upload(
files: [
SkillFile(path: "brand-guide/SKILL.md", data: skillMarkdown),
SkillFile(path: "brand-guide/references/tone.md", data: toneGuide),
],
displayTitle: "Brand guide"
)
print(skill.id)
```
Paths are preserved in the multipart upload. `AnthropicSkills` handles the
required Skills API beta header.
# Generating text (/docs/generating-text)
`generateText` gives you the finished result; `streamText` hands you every
event as it happens. Both drive the same loop underneath, and the rest of the
docs are variations on this page.
If the loop itself is new to you, read [the loop](/docs/foundations/the-loop)
first. It covers what a step is and why there's a limit on how many the run may
take.
## generateText [#generatetext]
```swift
let result = try await generateText(
model: model,
messages: history, // or system: + prompt:
tools: [weather],
maxSteps: 4
)
result.text // final answer
result.reasoningText // thinking, when the provider exposes it
result.toolCalls // every call across all steps
result.steps // one StepResult per model round-trip
result.messages // full history including tool turns, ready to persist
result.usage // combined token usage
```
## streamText [#streamtext]
Nothing runs until you iterate, and dropping the stream cancels the work.
```swift
let result = streamText(model: model, prompt: prompt, tools: tools)
for try await part in result.fullStream {
switch part {
case .textDelta(let delta): render(delta)
case .reasoningDelta(let delta): renderThinking(delta)
case .toolCall(let call): showToolChip(call)
case .toolResult(let result): updateToolChip(result)
case .finishStep(let step): persist(step)
case .finish(let reason, let usage): log(reason, usage)
default: break
}
}
```
`result.textStream` is the same stream reduced to assistant text deltas.
Consume either stream once; they drive the loop lazily.
### Every stream part [#every-stream-part]
`fullStream` yields `TextStreamPart`, and these are all of its cases:
| Part | When |
| ----------------------------------- | ------------------------------------------------------- |
| `.startStep(index:)` | A model round-trip begins (0-based). |
| `.textDelta(String)` | Assistant text, token by token. |
| `.reasoningDelta(String)` | Thinking text, when the provider streams it. |
| `.toolInputStart(id:name:)` | The model began a tool call; arguments still streaming. |
| `.toolInputDelta(id:partialJSON:)` | A fragment of the call's JSON arguments. |
| `.toolCall(ToolCall)` | Arguments fully assembled; execution is about to run. |
| `.toolResult(ToolResult)` | A tool finished; its output heads back to the model. |
| `.toolApprovalRequest(...)` | A call is held for the user; the turn ends after this. |
| `.source(Source)` | A citation from search-backed providers. |
| `.finishStep(StepResult)` | The round-trip closed; tools for it already ran. |
| `.finish(finishReason:totalUsage:)` | Terminal event for the whole loop. |
The `toolInputStart`/`toolInputDelta` pair is what lets a UI show a tool
card filling in while the model is still writing the arguments.
## Steering the loop [#steering-the-loop]
```swift
let result = try await generateText(
model: model,
prompt: prompt,
tools: tools,
toolChoice: .required, // .auto, .none, .required, .tool("name")
activeTools: ["search"], // visible subset; executors stay registered
stopWhen: [.hasToolCall("finalize")],
prepareStep: { context in
// per-step overrides: swap model, trim messages, change tools
context.stepNumber >= 3 ? PrepareStepResult(model: cheaperModel) : nil
},
onStepFinish: { step in await save(step) }
)
```
### Stop conditions [#stop-conditions]
`stopWhen` bounds the loop; the first met condition ends it. Without one,
`maxSteps` (default 8) applies.
```swift
stopWhen: [.isStepCount(5)] // at most 5 round-trips
stopWhen: [.stepCountIs(5)] // alias
stopWhen: [.hasToolCall("finalize")] // stop once a tool was requested
stopWhen: [.isLoopFinished] // only when the model stops calling tools
stopWhen: [.isStepCount(10), .hasToolCall("submit")] // whichever first
```
### Step results [#step-results]
Each round-trip lands in `result.steps` as a `StepResult`:
```swift
for step in result.steps {
step.text // assistant text this step
step.reasoningText // thinking this step
step.toolCalls // calls issued
step.toolResults // results fed back
step.sources // citations surfaced this step
step.approvalRequests // calls paused for user approval
step.finishReason
step.usage
}
```
`finishReason` is one of `.stop` (natural end of turn), `.length` (hit
`maxOutputTokens`), `.toolCalls` (the model wants tools), `.contentFilter`,
`.error`, or `.other`. `usage` carries `inputTokens`, `outputTokens`, and
`cachedInputTokens` where the provider reports prompt-cache hits.
`prepareStep` runs before each round-trip and can return per-step
overrides — swap to a cheaper `model`, rewrite `messages` (summarize a
long history), or narrow `tools`. Return `nil` to keep the step as is.
## Sampling and control [#sampling-and-control]
Every knob maps to each provider's native field and is dropped where a wire
lacks it: `temperature`, `topP`, `topK`, `presencePenalty`,
`frequencyPenalty`, `seed`, `stopSequences`, `maxOutputTokens`,
`maxRetries`, plus `onFinish` and `onError` callbacks.
## Reasoning [#reasoning]
The portable `reasoning` parameter controls thinking across providers
with one setting:
```swift
reasoning: .medium // .none, .minimal, .low, .medium, .high, .xhigh
```
[Reasoning](/docs/reasoning) covers streaming thinking, per-provider
translation, and precedence.
## Provider options [#provider-options]
`providerOptions` is the escape hatch for provider-specific fields; it
merges into the request body last, so it wins over anything the library
sets:
```swift
providerOptions: [
"thinking": ["type": "enabled", "budget_tokens": 12000]
]
```
## Smoothing, callbacks, and structured output [#smoothing-callbacks-and-structured-output]
* **`smoothStream`** re-chunks a `textStream` by word or line with an optional
delay, for calmer UI streaming: `for try await c in result.smoothedTextStream() { … }`.
* **`onChunk`** fires per streamed part and **`onAbort`** on cancellation,
alongside the existing `onStepFinish` / `onFinish` / `onError`.
* **`output:`** on `generateText` produces a final structured object *alongside*
tool calls — read it from `result.experimentalOutput`.
* **`repairToolCall`** ([tools](/docs/tools#repairing-tool-calls)) fixes a
malformed tool call before it runs.
# Getting started (/docs/getting-started)
## Add the package [#add-the-package]
In Xcode: File, Add Package Dependencies, paste the repository URL. Or in
`Package.swift`:
```swift title="Package.swift"
dependencies: [
.package(url: "https://github.com/zaidmukaddam/swift-ai-sdk", branch: "main")
],
targets: [
.target(name: "MyApp", dependencies: [.product(name: "AI", package: "swift-ai-sdk")])
]
```
Works on iOS 16, macOS 13, tvOS 16, watchOS 9, and visionOS 1. The chat
session objects want iOS 17 or macOS 14.
## No API key? Start free [#no-api-key-start-free]
Two ways to get a first response without signing up for anything.
If you have [Ollama](https://ollama.com) on your Mac:
```swift title="FirstStream.swift"
import AI
let model = OllamaModel("granite4.1:3b")
let result = streamText(
model: model,
prompt: "Say hello in three languages."
)
for try await token in result.textStream {
print(token, terminator: "")
}
```
Or skip the network entirely on devices with Apple Intelligence:
```swift
let result = try await generateText(
model: FoundationModelsModel(),
prompt: "One-line haiku about rain."
)
```
## Or bring a key [#or-bring-a-key]
Every provider reads its key from the environment, or takes it directly:
```swift
let claude = AnthropicModel("claude-fable-5") // ANTHROPIC_API_KEY
let gpt = OpenAIModel("gpt-5.6-terra") // OPENAI_API_KEY
let gemini = GoogleModel("gemini-3.5-flash") // GOOGLE_GENERATIVE_AI_API_KEY
let grok = XaiModel("grok-4.5", apiKey: myKey) // or pass it in
```
Changing providers is a one-line change. Everything downstream stays put.
## Give it a tool [#give-it-a-tool]
Hand the model a function and it figures out when to call it. The loop
runs the call, feeds the result back, and returns the final answer:
```swift
let getTime = Tool(
name: "current_time",
description: "Returns the current time in a timezone.",
parameters: ["type": "object",
"properties": ["tz": ["type": "string"]],
"required": ["tz"]]
) { args in
.string(Date().formatted())
}
let result = try await generateText(
model: model,
prompt: "What time is it in Kolkata?",
tools: [getTime]
)
print(result.text) // used the tool, then answered
```
## Where next [#where-next]
The [guides](/docs/guides) build real things step by step: a chat screen,
an agent with tools, a voice assistant. Or go straight to
[generating text](/docs/generating-text) for the full API.
# Image generation (/docs/image-generation)
Every image provider conforms to `ImageModel` and works with
`generateImage`.
```swift
let result = try await generateImage(
model: OpenAIImageModel("gpt-image-2"),
prompt: "A watercolor fox in a snowy forest",
size: "1024x1024"
)
try result.image.write(to: outputURL)
```
`result.image` is the first image as `Data`. The complete response is in
`result.images`, and providers that rewrite prompts report them in
`result.revisedPrompts`.
## Edit an image [#edit-an-image]
Pass source images through the same call. OpenAI automatically switches to its
edits endpoint.
```swift
let edited = try await generateImage(
model: OpenAIImageModel("gpt-image-2"),
prompt: "Make the sky stormy",
images: [ImageContent(data: source, mediaType: "image/png")]
)
```
## Options [#options]
The shared options are `n`, `size`, `aspectRatio`, `seed`, `maxRetries`, and
`providerOptions`. Providers use the settings they support:
```swift
let result = try await generateImage(
model: FalImageModel("fal-ai/flux/schnell"),
prompt: "An isometric library at night",
aspectRatio: "16:9",
seed: 42,
providerOptions: ["fal": ["num_inference_steps": 4]]
)
```
## Models [#models]
| Provider | Model type | Key |
| -------------------------------------- | --------------------- | -------------------------- |
| [OpenAI](/docs/providers/openai) | `OpenAIImageModel` | `OPENAI_API_KEY` |
| [fal](/docs/providers/fal) | `FalImageModel` | `FAL_API_KEY` or `FAL_KEY` |
| [Luma](/docs/providers/luma) | `LumaImageModel` | `LUMA_API_KEY` |
| [Replicate](/docs/providers/replicate) | `ReplicateImageModel` | `REPLICATE_API_TOKEN` |
## Batching [#batching]
For providers that cap images per request, pass `maxImagesPerCall`. The
library splits `n` across as many calls as needed and merges the results:
```swift
let result = try await generateImage(
model: OpenAIImageModel("gpt-image-2"),
prompt: "…", n: 10, maxImagesPerCall: 4
)
```
# Introduction (/docs)
swift-ai-sdk is a toolkit for building AI features into iOS and macOS apps:
streaming chat, agents that call your code, structured output, live
voice, and Apple's on-device models, all behind one small API.
The whole thing in four lines:
```swift
import AI
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: "Why is the sky blue?"
)
print(result.text)
```
Swap `AnthropicModel` for `OpenAIModel`, `GoogleModel`, `XaiModel`, an
Ollama model on your Mac, or Apple's on-device model. Nothing else
changes.
If you've built with the AI SDK on the web, everything here will feel
familiar, down to the streaming protocol: an app built with swift-ai-sdk can
talk to the `/api/chat` route you already have.
## Coming from the web? [#coming-from-the-web]
| You know | Here it's |
| ----------------------------- | --------------------------------------------------------------- |
| `generateText` / `streamText` | The same calls, with `async`/`await` and `for try await` |
| zod schemas | A `Schema` DSL that validates output before decoding |
| `useChat` | `ChatSession`, an `@Observable` object your SwiftUI views watch |
| Server stream helpers | `UIMessageStream` builders for Swift servers |
| Realtime | `RealtimeSession` for voice, same providers |
And one thing the web can't do:
[on-device models](/docs/on-device) answer without a network, a key, or
your data leaving the phone.
## Start here [#start-here]
Building with a coding agent? Install the
[swift-ai-sdk skill](/docs/agent-skill) so it writes correct code. Every docs
page is also raw markdown — append `.mdx` to any URL, or start from
[/llms.txt](/llms.txt).
# MCP (/docs/mcp)
The Model Context Protocol (MCP) is a standard way for servers to expose tools
to a model. `MCPClient` connects to an MCP server, lists its tools, and turns
each one into an ordinary tool you pass to `generateText` or `streamText`. The
bridged tools validate arguments against the server's schemas and run in the
same agent loop as your local tools.
## Quick start [#quick-start]
Point a client at a transport, connect, and hand `tools()` to a generation call.
This uses [DeepWiki](https://mcp.deepwiki.com), a public MCP server that answers
questions about GitHub repositories and needs no key, so it runs as-is:
```swift
let mcp = MCPClient(transport: MCPHTTPTransport(
url: URL(string: "https://mcp.deepwiki.com/mcp")!
))
try await mcp.connect()
defer { Task { await mcp.close() } }
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: "Use the DeepWiki tools to explain how the modelcontextprotocol/modelcontextprotocol repository is organized.",
tools: try await mcp.tools(),
stopWhen: [stepCountIs(5)]
)
```
`tools()` follows `nextCursor` pagination, so servers that page their tool list
are fully enumerated in one call. `connect()` runs the initialize handshake once
and is safe to call again; `close()` releases the transport (and, for stdio,
terminates the subprocess).
## Transports [#transports]
The client is transport-agnostic. Any `MCPTransport` works; the SDK ships three.
### Streamable HTTP [#streamable-http]
`MCPHTTPTransport` is the default. It speaks JSON-RPC over POST with SSE or JSON
responses, carries `mcp-session-id` sessions, and takes custom headers for auth:
```swift
MCPHTTPTransport(
url: URL(string: "https://mcp.example.com/mcp")!,
headers: ["Authorization": "Bearer \(token)"]
)
```
### Stdio [#stdio]
`MCPStdioTransport` (macOS and Linux) launches a local server as a subprocess
and speaks newline-delimited JSON-RPC over its stdin and stdout. A
`requestTimeout` (default 60 seconds) bounds each call, so a hung or silent
server fails instead of blocking forever:
```swift
let mcp = MCPClient(transport: MCPStdioTransport(
command: "npx",
arguments: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"],
requestTimeout: 30
))
try await mcp.connect()
defer { Task { await mcp.close() } }
```
Pass `environment:` to override variables for the child process and
`workingDirectory:` to set its launch directory.
### Legacy HTTP+SSE [#legacy-httpsse]
`MCPSSETransport` speaks the older HTTP+SSE protocol: a long-lived `GET` event
stream for server messages plus a `POST` endpoint the server advertises for
requests. Reach for it only with servers that predate Streamable HTTP. It takes
the same `headers` and `requestTimeout` options as the HTTP transport:
```swift
MCPSSETransport(url: URL(string: "https://legacy.example.com/sse")!)
```
## OAuth [#oauth]
Hosted MCP servers usually sit behind OAuth rather than a static bearer token.
Hand a transport an `MCPOAuthSession` and it attaches the access token, refreshes
it when the server answers `401`, and tells you when a browser round-trip is
needed. Both `MCPHTTPTransport` and `MCPSSETransport` take an `auth:` argument:
```swift
let serverURL = URL(string: "https://mcp.notion.com/mcp")!
let auth = MCPOAuthSession(serverURL: serverURL, provider: myProvider)
let mcp = MCPClient(transport: MCPHTTPTransport(url: serverURL, auth: auth))
do {
let tools = try await mcp.tools()
} catch AIError.authorizationRequired(let url) {
// open `url`, then hand back the URL the browser was redirected to:
try await auth.complete(callbackURL: redirected)
}
```
`myProvider` is your `MCPOAuthClientProvider`: it owns the redirect URI, the
client metadata used for dynamic client registration, and storage for tokens,
the registered client, the PKCE verifier, and the CSRF `state`. Persist tokens
and client information if you want sign-in to survive a relaunch.
The session performs the full MCP discovery chain on demand: the `401`'s
`WWW-Authenticate` header names the protected-resource metadata document, which
names the authorization server — commonly a different host than the MCP server
itself. Authorization uses PKCE (`S256`), sends the RFC 8707 `resource`
indicator on the authorization, token, and refresh requests, and verifies
`state` on the callback before exchanging the code. When the authorization
server advertises a `registration_endpoint`, the client registers itself
dynamically the first time; otherwise seed your provider with credentials you
obtained out of band.
`MCPOAuthFlow` exposes the same steps individually — `discover`,
`registerClientIfNeeded`, `startAuthorization`, `handleCallback`, `refresh` — if
you want to drive the flow yourself.
## Detecting tool-definition drift (rug pull) [#detecting-tool-definition-drift-rug-pull]
A compromised or malicious MCP server can change its tool definitions *after*
you approved them, injecting instructions into a description or widening an input
schema to capture more data. Fingerprint the tools you approved, then compare on
every later fetch:
```swift
let baseline = fingerprintTools(try await mcp.tools())
// store baseline, then on a later fetch:
let drift = detectToolDrift(fingerprintTools(try await mcp.tools()), baseline: baseline)
if drift.hasDrift {
// drift.changed / drift.added / drift.removed
// block or force re-approval; do not silently pass mutated tools to the model
}
```
Your app owns baseline storage and decides the response. A fingerprint covers
each tool's description and resolved input schema and is independent of JSON key
order, so a server reordering schema keys does not read as drift.
## Examples [#examples]
See [27-MCPTools.swift](https://github.com/zaidmukaddam/swift-ai-sdk/blob/main/Examples/Features/27-MCPTools.swift)
for runnable HTTP, stdio, legacy SSE, OAuth, and rug-pull examples.
# Media overview (/docs/media)
Swift AI exposes a small protocol and one top-level function for each media
capability. Pick the capability you need; provider-specific model types plug
into the same call.
All four functions are `async throws`, retry transient failures by default,
and accept `providerOptions` for native settings that do not belong in the
shared API.
```swift
let image = try await generateImage(model: imageModel, prompt: prompt)
let audio = try await generateSpeech(model: speechModel, text: text)
let text = try await transcribe(model: transcriptionModel, audio: data, mediaType: "audio/mpeg")
let video = try await generateVideo(model: videoModel, prompt: prompt)
```
See [media providers](/docs/providers/media) for the provider-by-provider
view.
# Messages and multimodal (/docs/messages)
For why history is a list at all, what each role is for, and why tool calls and
results must stay paired, see
[prompts and messages](/docs/foundations/prompts-and-messages).
## Messages [#messages]
A conversation is `[Message]`. Convenience initializers cover the common
cases; full content arrays cover the rest.
```swift
var history: [Message] = [
.system("You are terse."),
.user("What is in this photo?"),
.assistant("A lighthouse at dusk.")
]
// Full control over parts:
let message = Message(role: .user, content: [
.text("Compare these two:"),
.image(ImageContent(data: firstJPEG, mediaType: "image/jpeg")),
.image(ImageContent(url: secondURL, mediaType: "image/png"))
])
```
Roles are `.system`, `.user`, `.assistant`, and `.tool`. The loop manages
`.tool` turns for you; you only write them when replaying persisted
history.
## Vision and files [#vision-and-files]
Images and files ride as content parts and map to each provider's native
shape (`image_url` on OpenAI, base64 `source` on Anthropic, `inlineData`
on Gemini, content blocks on Bedrock):
```swift
let result = try await generateText(
model: GoogleModel("gemini-3.5-flash"),
messages: [Message(role: .user, content: [
.text("Summarize this report."),
.file(FileContent(data: pdfData, mediaType: "application/pdf", filename: "q3.pdf"))
])]
)
```
Both `ImageContent` and `FileContent` take inline `data`, a remote `url`, or
a `providerReference` from [`uploadFile`](/docs/files-and-skills). In chat
UIs, user attachments arrive as file parts and `convertToModelMessages`
decodes data URLs into inline bytes automatically.
Not every model can fetch a URL itself — Bedrock's Converse API, for one,
takes bytes only. Models declare this with `supportsRemoteURL(_:mediaType:)`,
and the loop downloads and inlines anything a model cannot fetch before the
request goes out, so the same message works everywhere instead of silently
losing its attachment. Only `http(s)` URLs are fetched (a `file://` URL
raises rather than being read), downloads are capped at 32 MB, and `data:`
URLs and provider references are left untouched.
## Content parts [#content-parts]
| Part | Carries |
| ---------------------------- | ---------------------------------------- |
| `.text(String)` | Plain text |
| `.image(ImageContent)` | Inline data or URL plus media type |
| `.file(FileContent)` | Any document, with optional filename |
| `.toolCall(ToolCall)` | A model-issued call (assistant messages) |
| `.toolResult(ToolResult)` | An executed result (tool messages) |
| `.toolApprovalResponse(...)` | A user decision the loop resolves |
## JSONValue [#jsonvalue]
`JSONValue` is the currency for tool arguments, provider options, and
structured output. It is `ExpressibleBy*Literal`, so it reads like JSON:
```swift
let arguments: JSONValue = [
"city": "Mumbai",
"days": 3,
"units": ["metric", "imperial"],
"detailed": true
]
arguments["city"]?.stringValue // "Mumbai"
arguments["days"]?.intValue // 3
arguments["missing"]?.boolValue // nil
// Decode into Codable whenever you want types:
struct Query: Decodable { let city: String; let days: Int }
let query = try arguments.decode(Query.self)
```
## Usage [#usage]
Every result carries token accounting, including provider cache and
reasoning details where the wire reports them:
```swift
result.usage.inputTokens
result.usage.outputTokens
result.usage.totalTokens
result.usage.cachedInputTokens // prompt cache hits (OpenAI, Anthropic, Groq, DeepSeek)
result.usage.reasoningTokens // thinking tokens (reasoning models)
```
# Middleware (/docs/middleware)
`wrapLanguageModel` intercepts requests and stream parts. The built-ins
cover the common cases:
```swift
let model = wrapLanguageModel(
model: OllamaModel("qwen3"),
middleware: [
.cache(), // replay identical requests
.extractReasoning(tag: "think"), // lift spans into reasoning
.defaultSettings(temperature: 0.2) // bake in defaults
]
)
```
`.simulateStreaming()` turns a non-streaming endpoint into a streaming
one. `.extractJson()` strips markdown code fences from the response, for
models that wrap JSON in ` ```json ` even when asked for raw output.
`.addToolInputExamples()` folds a tool's `inputExamples` into its description
for providers with no native field for them:
```swift
let search = Tool(
name: "search",
description: "Search the docs.",
parameters: Schema.object(["query": .string()]),
inputExamples: [["query": "install swift-ai-sdk"], ["query": "streaming"]]
) { arguments in try await docs.search(arguments["query"]?.stringValue ?? "") }
let model = wrapLanguageModel(
model: OpenAIModel("gpt-5"),
middleware: [.addToolInputExamples(prefix: "Input Examples:")]
)
```
## Wrapping other model kinds [#wrapping-other-model-kinds]
Embedding, image, and whole-provider wrapping mirror the language-model
version:
```swift
let embeddings = wrapEmbeddingModel(
model: OpenAIEmbeddingModel("text-embedding-3-small"),
middleware: [.defaultSettings(maxBatchSize: 96)]
)
let images = wrapImageModel(model: OpenAIImageModel("gpt-image-2"), middleware: [
ImageModelMiddleware(transformRequest: { request in
var request = request
request.prompt += ", studio lighting"
return request
})
])
let provider = wrapProvider(
provider: myProvider,
languageModelMiddleware: [.cache()],
embeddingModelMiddleware: [.defaultSettings(maxBatchSize: 96)]
)
```
## Caching [#caching]
`.cache()` keys on the request plus the wrapped model's identity. A hit
replays the stored stream parts without calling the model; a miss streams
live, buffers the parts, and stores them once the stream completes. Errors
are never cached.
```swift
let store = InMemoryLanguageModelCache()
let model = wrapLanguageModel(model: OpenAIModel("gpt-5.6-luna"), middleware: [.cache(store: store)])
```
The default store is in-process. Conform to `LanguageModelCache`
(`get`/`set` over `[StreamPart]`) to back it with Redis, disk, or anything
else.
## Custom middleware [#custom-middleware]
A middleware is a value with the hooks you need: `transformRequest` (edit
the request), `wrapStream` (post-process stream parts), or `wrapCall` (wrap
the whole call, deciding whether to invoke the model at all, which is what
`.cache()` uses):
```swift
let logger = LanguageModelMiddleware(
transformRequest: { request in
print("sending \(request.messages.count) messages")
return request
}
)
```
Middlewares apply in array order.
# On-device models (/docs/on-device)
`FoundationModelsModel` runs Apple's on-device model, and optionally
Private Cloud Compute, through the exact `LanguageModel` protocol as every
cloud provider. No network, no key, no data leaving the device.
```swift
let local = FoundationModelsModel()
let result = try await generateText(
model: local,
prompt: "One-line haiku about rain."
)
```
The web can't do this. If you're deciding whether an AI feature belongs
in your native app or your web app, this is the argument.
## Availability and fallback [#availability-and-fallback]
On-device availability depends on the device, OS, and Apple Intelligence
settings. Check it and fall back in one line:
```swift title="ModelPicker.swift"
let model: any LanguageModel = FoundationModelsModel.isAvailable
? FoundationModelsModel()
: AnthropicModel("claude-sonnet-5")
```
Everything downstream (tools, streaming, structured output, chat
sessions) is identical for both branches.
## Private Cloud Compute [#private-cloud-compute]
Requests can target Apple's Private Cloud Compute instead of the
on-device model:
```swift
let pcc = FoundationModelsModel.privateCloudCompute()
```
PCC requires the `com.apple.developer.private-cloud-compute` entitlement.
Without it the system call traps instead of throwing, so gate PCC builds
on provisioned targets only.
## What works [#what-works]
* `generateText` and `streamText`, including multi-turn history: the pack
maintains a Foundation Models `Transcript` across calls.
* Tools: calls surface through the same loop as every other provider.
* `generateObject`: guided generation maps your JSON Schema onto the
framework's constrained decoding.
* Chat sessions: `ChatSession(model: FoundationModelsModel())` gives a
fully offline chat UI.
## Error taxonomy [#error-taxonomy]
Failures arrive as typed errors where the framework provides them
(guardrail violations, context overflows, unavailable model assets), with
clear messages for the cases Apple reports only as generic `NSError`s.
Unentitled processes on some OS builds cannot run on-device inference at
all; the error message says exactly that instead of a cryptic code.
# Realtime voice (/docs/realtime)
`RealtimeSession` is the `useRealtime` analog: an observable session for
bidirectional voice over WebSockets. Three providers speak their native
wires behind one `RealtimeModel` protocol: `OpenAIRealtimeModel`,
`GoogleRealtimeModel` (Gemini Live), and `XaiRealtimeModel` (Grok voice).
## Token flow [#token-flow]
Realtime connections authenticate with short-lived client secrets. Your
server mints one with the API key; the app connects with the secret, so
the key never ships:
```swift title="TokenRoute.swift"
// Server side
let model = XaiRealtimeModel("grok-voice-latest")
let secret = try await model.createClientSecret(
options: RealtimeClientSecretOptions(
expiresAfterSeconds: 300,
sessionConfig: config
)
)
// return {token, url, expiresAt} to the app
```
The session can also fetch from a setup endpoint directly:
`try await session.connect(tokenEndpoint: url)`. The endpoint returns
`{token, url, tools}` and any tool definitions it includes join the
session automatically.
## A voice session [#a-voice-session]
```swift title="VoiceView.swift"
let session = RealtimeSession(
model: XaiRealtimeModel("grok-voice-latest"),
sessionConfig: RealtimeSessionConfig(
instructions: "You are a concise voice assistant.",
inputAudioTranscription: .init(),
turnDetection: .init(type: .serverVAD)
),
onToolCall: { call in
call.name == "getTime" ? .string(currentTime()) : nil
}
)
session.connect(secret: secret)
session.sendText("Hello!")
```
`session.messages` renders the conversation as regular `UIMessage` values:
streamed transcripts, your speech transcribed and inserted where the audio
was committed, and tool parts.
## Audio is yours [#audio-is-yours]
The app owns capture and playback, which is where native shines: real
AVAudioEngine, not a browser tab asking for mic permission.
```swift
// Speak: microphone PCM in (16-bit, 24 kHz by default)
session.sendAudio(microphoneChunk)
// Listen: decoded PCM out, feed your AVAudioEngine player
for await chunk in session.audioOutput { player.play(chunk) }
// Barge-in: report how much was heard so the model's context truncates
session.playbackInterrupted(playedMilliseconds: player.playedMilliseconds)
```
## Session configuration [#session-configuration]
`RealtimeSessionConfig` is provider-neutral; each model maps it onto its
native session payload:
```swift
RealtimeSessionConfig(
instructions: "You are a concise voice assistant.",
voice: "marin",
outputModalities: ["audio"], // or ["text"]
inputAudioFormat: .init(type: "audio/pcm", rate: 24_000),
inputAudioTranscription: .init(model: nil, language: "en", prompt: nil),
outputAudioTranscription: .init(),
outputAudioFormat: .init(type: "audio/pcm", rate: 24_000),
turnDetection: .init(type: .serverVAD),
tools: getRealtimeToolDefinitions(tools: [approve]),
providerOptions: nil // merged into the native payload
)
```
Audio format types are `audio/pcm` (16-bit, with a `rate`),
`audio/pcmu`, and `audio/pcma`. Setting `inputAudioTranscription` is
what makes your speech come back as `inputTranscriptionCompleted`
events, which the session inserts as user messages at the point the
audio was committed; `outputAudioTranscription` gives you the model's
spoken words as streaming text.
### Turn detection [#turn-detection]
```swift
turnDetection: .init(
type: .serverVAD, // or .semanticVAD, .disabled
threshold: 0.5, // VAD activation, 0.0 to 1.0
silenceDurationMs: 500, // silence before the server ends your turn
prefixPaddingMs: 300 // audio kept from before speech started
)
```
`.semanticVAD` ends turns on meaning rather than silence where the
provider supports it (xAI maps it to server VAD). `.disabled` is
push-to-talk: stream audio, then call `session.commitAudio()` to end
your turn yourself.
## Client-side tools [#client-side-tools]
Realtime tool execution is client-driven. Register definitions with
`getRealtimeToolDefinitions(tools:)` in the session config, execute in
`onToolCall`, or return nil and submit later:
```swift
session.addToolOutput(callID: call.id, output: .object(["approved": .bool(true)]))
```
On multi-tool turns the session requests exactly one follow-up response,
after the turn closes and every output is in.
## Events [#events]
Every normalized server event is available on `session.events` and through
`onEvent`, with the raw provider payload attached for anything
provider-specific. The full set:
| Event | Meaning |
| ---------------------------------------------- | ------------------------------------------------------- |
| `sessionCreated` / `sessionUpdated` | Socket is live; config acknowledged. |
| `speechStarted` / `speechStopped` | Server VAD heard you start and stop. |
| `audioCommitted` | Your audio became a conversation item. |
| `conversationItemAdded` | Any item joined the conversation. |
| `inputTranscriptionCompleted` | Your speech, transcribed. |
| `responseCreated` / `responseDone` | A model turn opened and closed. |
| `outputItemAdded` / `outputItemDone` | An output item within the turn. |
| `contentPartAdded` / `contentPartDone` | A content part within an item. |
| `audioDelta` / `audioDone` | Base64 audio out (what `audioOutput` decodes). |
| `audioTranscriptDelta` / `audioTranscriptDone` | The spoken response as text. |
| `textDelta` / `textDone` | Text-modality output. |
| `functionCallArgumentsDelta` / `...Done` | A tool call streaming in. |
| `error` | Server-reported error with message and code. |
| `custom(rawType:)` | Anything the provider sends that has no portable shape. |
## Provider wire notes [#provider-wire-notes]
* **OpenAI** connects with the `realtime` and
`openai-insecure-api-key.{token}` WebSocket subprotocols and nests
audio config under `audio.input`/`audio.output`; input transcription
defaults to `gpt-realtime-whisper` when enabled.
* **xAI** authenticates with a single `xai-client-secret.{token}`
subprotocol and uses a flat session shape (`response.text.*` events).
* **Google (Gemini Live)** mints tokens against `v1alpha/auth_tokens`
and connects with the token as an `?access_token=` query parameter —
no subprotocols. The Live wire is stateful rather than event-per-item,
so the model maps `serverContent` frames onto the portable events and
synthesizes response ids. Google requires the session config at token
creation, which is why `RealtimeClientSecretOptions` carries a
`sessionConfig`.
* Providers that send keepalive frames get them answered automatically:
a model can implement `healthCheckResponse(for:)` and the session
echoes whatever it returns.
# Reasoning (/docs/reasoning)
Many models support an internal thinking phase before answering. The
`reasoning` parameter controls it across providers with a single setting,
on both `generateText` and `streamText`:
```swift
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: "How many people will live in the world in 2040?",
reasoning: .medium
)
print(result.reasoningText) // the thinking
print(result.text) // the answer
```
Values: `.none`, `.minimal`, `.low`, `.medium`, `.high`, `.xhigh`, and
`.providerDefault` (the default, as if the parameter was omitted).
## Streaming reasoning [#streaming-reasoning]
Thinking arrives as its own deltas on the full stream:
```swift
let result = streamText(model: model, prompt: prompt, reasoning: .high)
for try await part in result.fullStream {
switch part {
case .reasoningDelta(let thought): renderThinking(thought)
case .textDelta(let text): renderAnswer(text)
default: break
}
}
```
## How each provider translates it [#how-each-provider-translates-it]
Effort enums where they exist, token budgets where they do not:
| Provider | Wire translation |
| ------------------ | --------------------------------------------------------------------------------------------------------- |
| OpenAI (Responses) | `reasoning.effort`, plus an automatic detailed summary |
| OpenAI (chat) | `reasoning_effort`, passed verbatim |
| Anthropic | Adaptive thinking with an effort level on current models; a `budget_tokens` thinking budget on older ones |
| Google | `thinkingLevel` on Gemini 3; a `thinkingBudget` on Gemini 2.5 |
| Bedrock | Claude thinking config, OpenAI `reasoning_effort`, or a generic `reasoningConfig` by model family |
| xAI | `reasoning.effort` (`minimal` coerces to low, `xhigh` to high) |
| Groq | `reasoning_effort` (`xhigh` coerces to high) |
| DeepSeek | `thinking.type` plus `reasoning_effort` (`xhigh` becomes max) |
| Fireworks | `reasoning_effort` coerced to its three levels |
| Mistral | `reasoning_effort` on its reasoning models only |
Providers with no reasoning knob (Perplexity, Cohere) ignore the value.
## Budgets under the hood [#budgets-under-the-hood]
Budget-based wires get a number computed by
`ReasoningEffort.budget(maxOutputTokens:maxBudget:minBudget:)`, which you
can also call yourself:
```swift
ReasoningEffort.medium.budget(maxOutputTokens: 64_000, maxBudget: 64_000)
// 19200 — 30% of the output ceiling
```
The fractions are `.minimal` 2%, `.low` 10%, `.medium` 30%, `.high` 60%,
`.xhigh` 90%, clamped to `[1024, maxBudget]`. `.none` and
`.providerDefault` return `nil`.
### Anthropic, precisely [#anthropic-precisely]
Claude models split into capability tiers, and the translation follows
them:
* **Adaptive models** (Sonnet 5, Fable 5, Opus 4.7/4.8, and the 4.6
pair) get `thinking.type: "adaptive"` plus `output_config.effort`.
`.minimal` coerces to `low`; `.xhigh` stays `xhigh` where supported and
becomes `max` on 4.6-generation models.
* **Everything older** gets `thinking.type: "enabled"` with a
`budget_tokens` computed against that model's real output ceiling
(64k for the 4.5 family and Sonnet 4, 32k for Opus 4/4.1), and
`max_tokens` is raised when the budget would not fit.
* `.none` sends `thinking.type: "disabled"` explicitly.
### Google, precisely [#google-precisely]
Gemini 3 models take `thinkingLevel` (`.none` and `.minimal` map to
`minimal` — thinking can't be fully disabled there; `.xhigh` caps at
`high`). Everything else takes a `thinkingBudget`: `0` for `.none`,
otherwise a fraction of the 65,536-token ceiling capped at 32,768 for
2.5 Pro and 24,576 for the rest.
## Precedence [#precedence]
Reasoning settings inside `providerOptions` always win, and the two are
never merged. Use the portable parameter by default; drop to
`providerOptions` when you need an exact budget:
```swift
let result = try await generateText(
model: AnthropicModel("claude-sonnet-4-5"),
prompt: prompt,
reasoning: .low, // ignored: the explicit budget below wins
providerOptions: [
"thinking": ["type": "enabled", "budget_tokens": 12000]
]
)
```
The parameter is a non-optional enum. In an optional parameter,
`reasoning: .none` would silently resolve to Swift’s `Optional.none`
and mean “provider default”, the exact opposite of disabling
reasoning.
## Extracting reasoning from text [#extracting-reasoning-from-text]
Models that emit `` blocks inline (some open models on Ollama or
Groq) get the middleware instead:
```swift
let model = wrapLanguageModel(
model: OllamaModel("deepseek-r1"),
middleware: [.extractReasoning(tag: "think")]
)
```
# Reranking (/docs/reranking)
A reranker scores an existing candidate set against one query. Unlike vector
search, it does not require you to generate or store embeddings.
```swift
let result = try await rerank(
model: CohereRerankingModel("rerank-v4-fast"),
query: "warm places in january",
documents: candidates,
topN: 5
)
for item in result.rankedDocuments {
print(item.relevanceScore, item.document)
}
```
Each result includes `document`, `relevanceScore`, and `index`, which points
back to the document's position in the original array. An empty document list
returns an empty result.
The built-in reranker is `CohereRerankingModel`; it reads
`COHERE_API_KEY`. See the [Cohere provider page](/docs/providers/cohere) for
its chat, embedding, and reranking model types.
For vector generation and local similarity scoring, see
[embeddings](/docs/embeddings).
# Runtime context (/docs/runtime-context)
`runtimeContext` carries server-side state through a whole run without putting
it in the prompt: tenant, request id, feature flags, a running budget. It is
readable in `prepareStep`, replaceable from there, and lands on every
`StepResult`:
```swift
let result = try await generateText(
model: model,
prompt: "Do the work.",
tools: [tool],
prepareStep: { context in
let used = context.runtimeContext?["toolCalls"]?.intValue ?? 0
return PrepareStepResult(runtimeContext: ["toolCalls": .number(Double(used + 1))])
},
runtimeContext: ["tenant": "acme", "toolCalls": .number(0)]
)
```
## Per-tool context [#per-tool-context]
Per-tool state stays in `toolsContext`, where each tool sees only its own
entry. A tool can validate that entry and even describe itself from it:
```swift
let weather = Tool(
name: "weather",
description: "Get the weather.",
parameters: Schema.object(["city": .string()])
) { arguments, options in
try await fetch(city: arguments["city"]?.stringValue ?? "",
key: options.context?["apiKey"]?.stringValue ?? "")
}
.withContextSchema(Schema.object(["apiKey": .string(), "unit": .string()]))
.describing { context in
"Get the weather in \(context?["unit"]?.stringValue ?? "celsius")."
}
```
Invalid context fails that tool call with `AIError.invalidToolContext` instead
of reaching the tool.
Neither `runtimeContext` nor `toolsContext` reaches telemetry spans unless you
name the keys. See [telemetry](/docs/telemetry).
## Dynamic tools [#dynamic-tools]
Tools whose schema is only known at runtime, like MCP tools and user-defined
functions, are marked dynamic so a UI can tell them apart from tools it was
compiled against. MCP tools are dynamic automatically:
```swift
let tool = Tool.dynamic(name: name, description: description) { input, _ in
try await registry.call(name, input)
}
```
The flag rides `ToolCall.isDynamic` and the `dynamic` field on
`tool-input-available` / `tool-output-available` chunks.
# Speech generation (/docs/speech-generation)
Every speech provider conforms to `SpeechModel` and works with
`generateSpeech`.
```swift
let result = try await generateSpeech(
model: OpenAISpeechModel("gpt-4o-mini-tts"),
text: "The quick brown fox jumped over the lazy dog.",
voice: "alloy",
outputFormat: "mp3"
)
try result.audio.write(to: URL(fileURLWithPath: "/tmp/hello.mp3"))
print(result.mediaType)
```
The shared options are `voice`, `instructions`, `speed`, `outputFormat`,
`providerOptions`, and `maxRetries`. Voice IDs and supported formats come from
the provider.
```swift
let result = try await generateSpeech(
model: SarvamSpeechModel(targetLanguage: "hi-IN"),
text: "नमस्ते, आप कैसे हैं?",
voice: "anushka",
speed: 1.1
)
```
## Models [#models]
| Provider | Model type | Default or example model | Key |
| ---------------------------------------- | ----------------------- | ---------------------------------- | -------------------- |
| [OpenAI](/docs/providers/openai) | `OpenAISpeechModel` | `gpt-4o-mini-tts` | `OPENAI_API_KEY` |
| [ElevenLabs](/docs/providers/elevenlabs) | `ElevenLabsSpeechModel` | `eleven_v3` | `ELEVENLABS_API_KEY` |
| [LMNT](/docs/providers/lmnt) | `LMNTSpeechModel` | `blizzard` | `LMNT_API_KEY` |
| [Hume](/docs/providers/hume) | `HumeSpeechModel` | Voice-selected; no model parameter | `HUME_API_KEY` |
| [Deepgram](/docs/providers/deepgram) | `DeepgramSpeechModel` | `aura-2-thalia-en` | `DEEPGRAM_API_KEY` |
| [Sarvam](/docs/providers/sarvam) | `SarvamSpeechModel` | `bulbul:v3` | `SARVAM_API_KEY` |
# Streaming protocol (/docs/streaming-protocol)
Chat UIs on the web stream messages as SSE frames of typed JSON chunks,
terminated by `data: [DONE]`. swift-ai-sdk speaks that exact protocol on both
ends: sessions consume it, and the server helpers on this page produce
it. If you already have a chat route, the app plugs into it; if you're
writing a Swift server, your web frontend plugs into you.
## Serving a stream [#serving-a-stream]
`UIMessageStream.chunks` bridges the generation loop onto protocol chunks,
the `toUIMessageStreamResponse` analog for Swift servers (Vapor, Hummingbird,
or anything that writes SSE):
```swift title="ChatRoute.swift"
let result = streamText(model: model, messages: messages, tools: tools)
let chunks = UIMessageStream.chunks(from: result.fullStream)
response.headers = UIMessageStream.headers
for try await chunk in chunks {
try await response.write(UIMessageStream.encodeSSE(chunk))
}
try await response.write(UIMessageStream.doneSSE)
```
## Building streams by hand [#building-streams-by-hand]
`UIMessageStream.build` is the `createUIMessageStream` analog: write
arbitrary chunks and merge whole generation streams into one response.
```swift
let stream = UIMessageStream.build { writer in
writer.write(.data(name: "data-status", data: .string("searching")))
let result = streamText(model: model, prompt: prompt)
writer.merge(UIMessageStream.chunks(from: result.fullStream))
}
```
The stream stays open until the body returns and every merged stream
drains; errors surface as in-band `error` chunks.
## Message metadata [#message-metadata]
Attach values at start and stream updates as the loop progresses; the
client deep-merges every update into `UIMessage.metadata`:
```swift
UIMessageStream.chunks(
from: result.fullStream,
metadata: ["model": .string("claude-sonnet-5")],
messageMetadata: { part in
if case .finish(_, let usage) = part {
return ["totalTokens": .number(Double(usage.totalTokens))]
}
return nil
}
)
```
## Reading streams [#reading-streams]
`readUIMessageStream` consumes any chunk stream as a sequence of
`UIMessage` snapshots, one per applied chunk, without a `ChatSession`.
Useful for persistence pipelines and server-side processing:
```swift
for try await snapshot in readUIMessageStream(chunks) {
render(snapshot)
}
```
## Converting to model messages [#converting-to-model-messages]
`convertToModelMessages` turns client `UIMessage` state into the model
history, including file parts (data URLs decode to inline bytes), settled
tool calls with their results, and approval responses for the loop to
resolve.
## Chunk reference [#chunk-reference]
The chunk types on the wire, for anyone implementing a server or
debugging frames:
| Chunk | Purpose |
| ------------------------------------------------------------------ | ------------------------------------------------------ |
| `start`, `finish`, `abort` | Message lifecycle; `start` carries the id and metadata |
| `start-step`, `finish-step` | Loop step boundaries |
| `text-start`, `text-delta`, `text-end` | Streamed text, framed by part id |
| `reasoning-start/-delta/-end` | Streamed thinking |
| `tool-input-start`, `tool-input-delta`, `tool-input-available` | A tool call assembling |
| `tool-output-available`, `tool-output-error`, `tool-output-denied` | Its result |
| `tool-approval-request` | Human-in-the-loop pause |
| `source-url`, `source-document` | Citations |
| `data-*` | Your custom data parts |
| `message-metadata` | Deep-merged metadata updates |
| `error` | In-band errors |
A Swift server and a TypeScript client, or the reverse, agree on every
one of these byte for byte.
# Structured output (/docs/structured-output)
Asking a model for JSON and hoping is not a strategy. `generateObject`
constrains the output to a schema and decodes it into your `Codable`
type; if the model produces something else, you get a thrown error
instead of a corrupt view.
```swift
struct Recipe: Codable {
var name: String
var ingredients: [String]
var steps: [String]
}
let recipeSchema: JSONValue = [
"type": "object",
"properties": [
"name": ["type": "string"],
"ingredients": ["type": "array", "items": ["type": "string"]],
"steps": ["type": "array", "items": ["type": "string"]]
],
"required": ["name", "ingredients", "steps"]
]
let result = try await generateObject(
model: OpenAIModel("gpt-5.6-sol"),
of: Recipe.self,
schema: recipeSchema,
prompt: "A simple lasagna recipe."
)
print(result.object.name)
```
Each provider uses its best native mechanism: JSON schema mode on OpenAI,
constrained decoding on Gemini and on-device models, a forced tool call
on Anthropic. Same call site everywhere.
## The Schema DSL [#the-schema-dsl]
Writing raw JSON Schema gets old fast. The `Schema` combinators build it
for you and validate the model's output at runtime before decoding:
```swift
let recipeSchema = Schema.object([
"name": .string(description: "Recipe name"),
"steps": .array(of: .string(), minItems: 1),
"servings": .integer(minimum: 1).optional()
])
let result = try await generateObject(
model: model,
of: Recipe.self,
schema: recipeSchema,
prompt: "A simple dal recipe."
)
```
Enums, unions, and nesting compose the way you'd hope:
```swift
let event = Schema.object([
"kind": .enum(["meeting", "reminder"]),
"when": .string(format: "date-time"),
"attendees": .array(of: .object([
"name": .string(),
"id": .anyOf([.integer(), .string()])
])).optional()
])
```
The same schemas plug into `Tool(parameters:)`, where arguments get
validated before your closure runs.
## Streaming partials [#streaming-partials]
`streamObject` repairs partial JSON as it arrives, so a form can fill
itself in while the model writes:
```swift
let result = streamObject(
model: model,
schema: recipeSchema,
prompt: "A simple lasagna recipe."
)
var latest: JSONValue = .null
for try await partial in result.partialObjectStream {
latest = partial
render(partial) // grows field by field
}
let recipe = try latest.decode(Recipe.self) // the last snapshot is complete
```
### Element streaming [#element-streaming]
For array-shaped schemas, `elementStream()` yields each completed element
exactly once, in order — a list view can append rows instead of
re-rendering snapshots:
```swift
let result = streamObject(model: model, schema: heroListSchema, prompt: "3 heroes")
for try await hero in result.elementStream() {
rows.append(hero) // fires as each array element completes
}
```
An element counts as complete once the model has moved on to the next
one (or the stream ends). It's derived from `partialObjectStream`, so
consume one or the other, not both; non-array schemas produce an empty
stream.
## Enums and free-form JSON [#enums-and-free-form-json]
Two smaller strategies for smaller jobs:
```swift
// Pick one of a fixed set. Great for classification.
let sentiment = try await generateEnum(
model: model,
values: ["positive", "neutral", "negative"],
prompt: "Classify: this library is delightful."
)
// Any valid JSON, no schema.
let palette = try await generateJSON(
model: model,
prompt: "A color palette as JSON."
)
```
## Arrays [#arrays]
`generateObjectArray` returns a list of objects that each match an element
schema — it wraps them in an `elements` array on the wire and unwraps the
result for you:
```swift
let people: GenerateObjectResult<[Person]> = try await generateObjectArray(
model: model,
of: Person.self,
elementSchema: Person.jsonSchema,
prompt: "Invent three fictional people."
)
```
## Repairing malformed JSON [#repairing-malformed-json]
Pass `repairText` to salvage output that doesn't parse — it receives the raw
text and the error, and returns corrected JSON (or `nil` to give up):
````swift
let result = try await generateObject(
model: model, of: Config.self, schema: Config.jsonSchema, prompt: "…",
repairText: { text, _ in text.replacingOccurrences(of: "```json", with: "")
.replacingOccurrences(of: "```", with: "") }
)
````
# Telemetry (/docs/telemetry)
`AITelemetry` is a process-wide hook, disabled until you set a
collector. Conform to `AITelemetryCollector` and forward events into
OSLog signposts, an OpenTelemetry exporter, or your analytics:
```swift
struct LogCollector: AITelemetryCollector {
func record(_ event: AITelemetryEvent) {
logger.info("\(event.name) \(event.phase)", metadata: [
"model": "\(event.attributes["ai.model.id"] ?? "")",
"duration": "\(event.duration)"
])
}
}
AITelemetry.collector = LogCollector()
```
Each call is bracketed by `start` and `end` (or `error`) phases with a
measured `duration`. Spans cover `ai.generateText`, `ai.streamText`,
`ai.generateObject`, and the embedding calls, with model, usage, and
finish-reason attributes.
## Per-call settings [#per-call-settings]
`telemetry:` sets a function id and metadata per call, turns collection off
for one call, and allow-lists which context keys reach spans. Nothing from
[`runtimeContext` or `toolsContext`](/docs/runtime-context) is recorded unless
you name it:
```swift
telemetry: TelemetrySettings(
functionID: "summarize",
metadata: ["team": "ios"],
includeRuntimeContext: ["tenant"], // "secret" stays out of spans
includeToolsContext: ["weather"]
)
```
# Terminal UI (/docs/terminal-ui)
The `AITUI` target runs a local [`Agent`](/docs/agents) or a remote
[`ChatTransport`](/docs/chat-ui) in an interactive terminal. It handles prompt
input, streamed responses, markdown rendering, tool cards, reasoning sections,
scrolling, and tool approvals. Useful for local development, demos, and
internal tools where a terminal is enough.
```swift
import AI
import AITUI
let agent = Agent(
model: OpenAIModel("gpt-5"),
instructions: "You are a helpful terminal assistant. Answer in markdown.",
tools: [weather]
)
try await runAgentTUI(title: "Weather Agent", agent: agent)
```
`runAgentTUI` runs until the user exits with Esc or Ctrl+C. Add the library to
your target alongside `AI`:
```swift
.product(name: "AITUI", package: "swift-ai-sdk")
```
## Try it [#try-it]
The repository ships a runnable demo:
```bash
swift run tui-demo
```
Every backend is a real model. With no flag the demo picks the first that is
usable (Anthropic, then OpenAI, then a running Ollama), or tells you how to
get one if none are:
```bash
swift run tui-demo --ollama # local, no API key
swift run tui-demo --openai # OPENAI_API_KEY
swift run tui-demo --anthropic # ANTHROPIC_API_KEY
```
Ollama is the zero-key path: `ollama serve` and `ollama pull qwen3` are enough.
The demo checks the server is reachable and the model is actually pulled before
starting, so a missing model is a clear message rather than a failed first turn.
`swift run tui-demo --list` prints what Ollama has.
`--model ` (or `AI_MODEL`) picks the model, `OLLAMA_HOST` points at a
non-default server, and `TUI_DEMO_APPROVAL=1` makes the weather tool require
approval. The weather tool calls a live API, so tool cards show real data.
`swift run tui-demo --help` lists everything.
To run entirely on-device with Apple Intelligence:
```bash
swift run tui-demo --on-device
```
`--pcc` uses [Private Cloud Compute](/docs/on-device) instead. Both flags also
read the `AI_ON_DEVICE` and `AI_PCC` environment variables, and both report
availability and exit cleanly when the platform or model is unavailable rather
than trapping. Foundation Models does not report token usage, so the
statistics and context readouts stay hidden on that path.
## Connecting to a remote agent [#connecting-to-a-remote-agent]
Pass a transport instead of an agent to talk to an AI SDK–compatible UI
message endpoint. The transport owns the URL, authentication, and request
body; the terminal UI keeps its chat id and message history behind the
transport contract.
```swift
try await runAgentTUI(
title: "Remote Agent",
transport: HTTPChatTransport(api: URL(string: "https://example.com/api/chat")!)
)
```
Anything conforming to `ChatTransport` works, including `LocalChatTransport`
and `Agent` itself.
## Display options [#display-options]
```swift
try await runAgentTUI(
title: "Assistant",
agent: agent,
tools: .autoCollapsed,
reasoning: .collapsed,
responseStatistics: .outputTokenCount,
contextSize: 200_000
)
```
* `tools` / `reasoning` take a `TerminalPartDisplayMode`: `.full` shows the
section header and full content, `.collapsed` shows only the header,
`.autoCollapsed` (the default) keeps the newest section expanded until
another section appears after it, and `.hidden` omits it.
* `responseStatistics` is `.outputTokensPerSecond` (default) or
`.outputTokenCount`.
* `contextSize` turns on a token-usage readout as a percentage of the model's
context window.
* `theme` takes a `TerminalTheme` if you want different colors for headings,
code, links, and quotes.
A tool card awaiting approval always renders expanded, whatever the mode, so
the input being approved is visible.
## Tool approvals [#tool-approvals]
Tools declared with `needsApproval` pause the loop, and the terminal UI
prompts before the call runs:
```swift
let weather = Tool(
name: "weather",
description: "Get the weather in a location.",
parameters: ["type": "object", "properties": ["location": ["type": "string"]]],
needsApproval: true
) { arguments in ["temperatureF": 72] }
```
Press `y` to approve or `n` to deny. Approving resumes the same loop through
`convertToModelMessages`, exactly like `ChatSession.addToolApprovalResponse`
in the SwiftUI layer; denying returns a denial result to the model.
## Statistics and usage [#statistics-and-usage]
For a local `Agent`, usage is read from the stream's finish part and attached
as message metadata, so tokens/sec, token counts, and context percentage are
exact. For a remote transport, the readout appears when the server sends
`usage` in message metadata (`messageMetadata:` on `UIMessageStream.chunks`);
without it the status bar just omits the statistic.
## Controls [#controls]
| Key | Action |
| ------------------ | --------------------------------------------- |
| Enter | Submit the prompt |
| `y` / `n` | Approve or deny a tool call |
| ↑ / ↓ | Scroll the transcript |
| PageUp / PageDown | Scroll by a full page |
| ← / → , Home / End | Move the input cursor |
| Ctrl+W / Ctrl+U | Delete the previous word / clear the input |
| Ctrl+L | Repaint |
| Esc | Stop an in-flight response, or exit when idle |
| Ctrl+C | Exit |
## Requirements and behavior [#requirements-and-behavior]
* POSIX terminals only (macOS and Linux). `runAgentTUI` throws
`AgentTUIError.notATerminal` when stdin or stdout is not a TTY, so piping
fails loudly instead of hanging.
* It runs on the alternate screen and restores the terminal — raw mode,
cursor, and screen — on exit, including on error.
* Color is dropped when `NO_COLOR` is set or `TERM` is `dumb`.
* Wide characters (CJK, emoji) are measured at two columns so wrapping and
truncation stay aligned.
## Rendering pieces on their own [#rendering-pieces-on-their-own]
The pieces the UI is built from are public and pure, so you can render
transcripts elsewhere (a log file, a test snapshot, a different loop):
```swift
let lines = TranscriptRenderer.lines(
for: messages,
width: 80,
options: TerminalTranscriptOptions(tools: .full, reasoning: .hidden)
)
for line in lines { print(line.render(styled: true)) }
```
`MarkdownTerminalRenderer.render(_:width:theme:)` returns styled, wrapped
lines for any markdown string, and `AgentTUIRenderer.frame(model:size:styled:)`
builds a full screen from an `AgentTUIModel`.
## Compatibility [#compatibility]
Use `agent:` when the agent runs from free-form terminal input. It must not
require per-call options or structured output, because the terminal UI cannot
infer those from a prompt. Use `transport:` for remote agents that need custom
request handling, and call `agent.generate()` / `agent.stream()` directly when
you need fixed prompts, structured output, or custom stream processing.
# Testing (/docs/testing)
The `AITesting` library is the `ai/test` analog: deterministic doubles
for your test targets, no network, no keys. Add the product next to `AI`:
```swift title="Package.swift"
.testTarget(
name: "MyAppTests",
dependencies: [
.product(name: "AI", package: "swift-ai-sdk"),
.product(name: "AITesting", package: "swift-ai-sdk")
]
)
```
## MockLanguageModel [#mocklanguagemodel]
The one-liner covers most tests:
```swift
import AITesting
func testGreeting() async throws {
let model = MockLanguageModel(text: "Hello, world!")
let result = try await generateText(model: model, prompt: "Hi")
XCTAssertEqual(result.text, "Hello, world!")
}
```
Every request is recorded, so you can assert on exactly what your code
sent:
```swift
XCTAssertEqual(model.requests.count, 1)
XCTAssertEqual(model.requests[0].messages.last?.text, "Hi")
XCTAssertEqual(model.requests[0].reasoning, .medium)
```
## Scripting multi-step loops [#scripting-multi-step-loops]
`responses:` provides one part-array per model round-trip; calls past the
end replay the last script. That is enough to test a full tool loop:
```swift
let model = MockLanguageModel(responses: [
[
.toolCall(ToolCall(id: "c1", name: "search", arguments: ["q": "swift"])),
.finish(reason: .toolCalls, usage: .init())
],
[
.textDelta("Found it."),
.finish(reason: .stop, usage: .init())
]
])
let result = try await generateText(model: model, prompt: "go", tools: [searchTool])
XCTAssertEqual(result.stepCount, 2)
XCTAssertEqual(result.text, "Found it.")
```
For full control, compute parts from the request and call number:
```swift
let model = MockLanguageModel { request, callIndex in
[.textDelta("call #\(callIndex)"), .finish(reason: .stop, usage: .init())]
}
```
Pass `chunkDelay:` to pace parts like a live stream.
## MockEmbeddingModel [#mockembeddingmodel]
```swift
let model = MockEmbeddingModel(vectors: [[1, 0], [0, 1]])
let result = try await embedMany(model: model, values: ["a", "b", "c"])
// vectors cycle; model.batches records every input batch
```
## Stream and value helpers [#stream-and-value-helpers]
```swift
// Any chunk array as a paced AsyncThrowingStream: test UI pipelines
// without a model at all.
let chunks = simulateReadableStream(
chunks: uiMessageChunks,
initialDelay: .milliseconds(100),
chunkDelay: .milliseconds(10)
)
// Deterministic id generators: hands out values in order, sticks at the last.
let nextID = mockValues("id-1", "id-2", "id-3")
```
## Testing chat UIs [#testing-chat-uis]
Sessions take transports, and an `Agent` over a mock model is a
transport, so a full `ChatSession` test needs no HTTP:
```swift
let agent = Agent(model: MockLanguageModel(text: "Hi there!"))
let chat = ChatSession(transport: agent)
chat.send("Hello")
// await chat.status == .ready, then assert on chat.messages
```
# Timeouts and approvals (/docs/timeouts-and-approvals)
## Timeouts [#timeouts]
`generateText`, `streamText`, and `Agent` take a `timeout:`. Every field is
optional; set only the ones you want.
```swift
let result = try await generateText(
model: model,
prompt: "Plan the migration.",
tools: [weather],
timeout: GenerationTimeout(
total: .seconds(60), // whole call, all steps
step: .seconds(20), // one model call
firstChunk: .seconds(5), // until the first content of a step (streaming)
chunk: .seconds(10), // between content chunks after output starts
tool: .seconds(15), // any tool execution
tools: ["slowApi": .seconds(45)] // per-tool override
)
)
```
`GenerationTimeout.after(.seconds(30))` is shorthand for a total timeout.
### What counts as content [#what-counts-as-content]
`firstChunk` and `chunk` are stall detectors, and only content-bearing output
satisfies or resets them: text deltas, reasoning deltas, tool-input deltas,
tool calls, tool results, and sources. Response metadata, stream starts, and
empty deltas do not — so a provider sending keep-alive metadata will not hold
a stalled stream open.
### What a timeout does [#what-a-timeout-does]
Total, step, and stall timeouts throw `AIError.timedOut(scope:limit:tool:)`,
which surfaces through `streamText`'s `onError` and cancels the underlying
request. A **tool** timeout is different: it aborts that one execution and
returns a tool error, so the model can react or retry:
```swift
if case .timedOut(let scope, let limit, let tool) = error as? AIError {
print("timed out: \(scope) after \(limit) \(tool ?? "")")
}
```
Cancellation is cooperative, as everywhere in Swift concurrency: a tool that
never checks `Task.isCancelled` and never awaits a cancellable call will run
to completion before its timeout surfaces. Anything that awaits I/O or
`Task.sleep` stops promptly.
## Tool approvals [#tool-approvals]
Approval policy lives on the call, not on the tool, so the same tool can be
free in one context and gated in another.
```swift
let result = try await generateText(
model: model,
prompt: "Clean up the temp files.",
tools: [deleteFile, runQuery],
toolApproval: [
"deleteFile": .userApproval(),
"runQuery": .denied(reason: "Queries are disabled in this workspace")
]
)
```
Four decisions:
| Decision | Effect |
| ------------------------ | -------------------------------------------------------------------------------------- |
| `.notApplicable` | Run the tool normally. The default, and the fallback to the tool's own `needsApproval` |
| `.approved(reason:)` | Record an automatic approval and run the tool |
| `.denied(reason:)` | Skip the tool and hand the model a denied result it can reason about |
| `.userApproval(reason:)` | Emit an approval request and pause the loop |
Decisions land on `step.approvalDecisions`, keyed by tool call id, and
`.userApproval` also appears in `step.approvalRequests`.
For policies that depend on the arguments or on what already happened, pass a
closure instead of a map — it receives the whole call plus the message history
and step number:
```swift
toolApproval: ToolApprovalPolicy { context in
guard context.toolCall.name == "transfer" else { return .notApplicable }
let amount = context.toolCall.arguments["amountUSD"]?.doubleValue ?? 0
if amount > 10_000 { return .denied(reason: "Above the automatic limit") }
return amount > 100 ? .userApproval() : .approved()
}
```
`ToolApprovalPolicy.perTool([...])` keys closures by tool name, and
`prepareCall` can return a `toolApproval` when the policy depends on the
tenant or user rather than the call site.
Answering an approval is unchanged: `ChatSession.addToolApprovalResponse`,
the `y`/`n` keys in the [terminal UI](/docs/terminal-ui), or a
`.toolApprovalResponse` part in the next turn.
## Signing approvals [#signing-approvals]
Approvals travel through the client — a browser, an app, a terminal — and come
back in the next request's message history. Without protection, a client that
crafts a schema-valid approval bypasses the human step. Pass a secret and the
loop HMAC-signs each request at issue and verifies it on replay:
```swift
let result = try await generateText(
model: model,
messages: messages,
tools: [deleteFile],
toolApproval: ["deleteFile": .userApproval()],
toolApprovalSecret: ProcessInfo.processInfo.environment["TOOL_APPROVAL_SECRET"]
)
```
* The signature binds the approval id, tool name, tool call id, and the exact
input. Change any of them and verification fails.
* Verification is **fail-closed**: once a secret is configured, an approval
with a missing or invalid signature is denied, and the model receives a
denied result instead of the tool running.
* With no secret configured, approvals work exactly as before.
* The payload is serialized as JSON with a versioned prefix, so a field
containing a delimiter cannot be re-arranged into a different but
identically-signed tuple.
* The secret never reaches the client; only the signature does.
`ChatSession` and the terminal UI carry it back automatically.
Generate one with `openssl rand -base64 32` and give every instance that might
serve a turn the same value, since one instance signs and another may verify.
# Tools (/docs/tools)
A tool is anything conforming to `AIToolProtocol`: a name, a description,
JSON Schema parameters, and an async `execute`. The closure-based `Tool`
covers most cases.
For what the model actually emits and why descriptions decide whether a tool
gets used at all, see [tool calling](/docs/foundations/tool-calling).
```swift
let search = Tool(
name: "search",
description: "Search the product catalog.",
parameters: Schema.object(["query": .string()])
) { arguments in
let query = arguments["query"]?.stringValue ?? ""
return try await catalog.search(query)
}
```
## Typed arguments [#typed-arguments]
`Tool.typed` decodes arguments into a `Decodable` before your code runs:
```swift
struct SearchArgs: Decodable { let query: String }
let search = Tool.typed(
name: "search",
description: "Search the product catalog.",
parameters: Schema.object(["query": .string()])
) { (args: SearchArgs) in
try await catalog.search(args.query)
}
```
## Execution context [#execution-context]
Tools can see which call they are servicing, the step messages, and
per-request context that should never ride in the prompt. This is the AI
SDK's tool execution options plus `toolsContext`:
```swift
let orders = Tool(
name: "list_orders",
description: "List recent orders for the signed-in user.",
parameters: Schema.object([:])
) { _, options in
let userID = options.context?["userID"]?.stringValue ?? "anonymous"
return try await store.orders(for: userID)
}
let result = try await generateText(
model: model,
prompt: "What did I order recently?",
tools: [orders],
toolsContext: ["list_orders": ["userID": .string("user-7")]]
)
```
`options.toolCallID` and `options.messages` are there too.
## Human-in-the-loop approvals [#human-in-the-loop-approvals]
A tool can require user approval before it runs. The loop pauses with a
`toolApprovalRequest`, your app answers, and execution resumes on the next
turn; denials surface to the model as denied results.
```swift
let delete = Tool(
name: "delete_file",
description: "Delete a file.",
parameters: Schema.object(["path": .string()]),
needsApproval: true
) { arguments in
try await files.delete(arguments["path"]?.stringValue ?? "")
}
```
Approval can also depend on the arguments: pass
`needsApproval: { arguments in ... }`. In chat UIs, respond with
`ChatSession.addToolApprovalResponse(approvalID:approved:)`.
Per-tool `needsApproval` only expresses "ask a human". For policy that lives
on the call — automatic denials with a reason, argument- or history-dependent
decisions, or a rule set that differs per tenant — pass `toolApproval:` to
`generateText` / `streamText` / `Agent`, and sign approvals with
`toolApprovalSecret:` so a client cannot forge them. See
[Timeouts and approvals](/docs/timeouts-and-approvals).
## Client-side tools [#client-side-tools]
A tool without an executor ends the turn with the call unexecuted. The
model asked; your app answers, whenever it's ready:
```swift
let pickPhoto = Tool(
name: "pick_photo",
description: "Ask the user to pick a photo.",
parameters: Schema.object([:])
)
// later, in the chat UI:
chat.addToolResult(toolCallID: tool.toolCallID, result: ["photoID": "IMG_0042"])
```
## Provider-executed tools [#provider-executed-tools]
Some providers run their own server-side tools — live web and X search,
code execution, file search, computer use — and stream the calls and
results back in the same turn. Each provider exposes typed builders under
`.Tools`; drop them in `tools:` next to your own:
```swift
let result = try await generateText(
model: XaiModel("grok-4.5"),
prompt: "What shipped in AI this week?",
tools: [
XaiModel.Tools.webSearch(allowedDomains: ["arxiv.org"]),
XaiModel.Tools.xSearch(allowedXHandles: ["xai"]),
]
)
```
These carry no executor: the provider runs them, so the loop never asks
your app to and the turn doesn't pause. The calls and results arrive as
`.toolCall` / `.toolResult` parts with `toolCall.providerExecuted == true`
(web citations also surface as `.source`), and each provider ignores
builders that belong to another — so mixing them is safe. Each provider
page lists its catalog: [xAI](/docs/providers/xai),
[OpenAI](/docs/providers/openai), [Google](/docs/providers/google),
[Anthropic](/docs/providers/anthropic).
For a server-side tool without a typed builder yet, construct one directly
with `ProviderDefinedTool(provider:id:name:args:)` — `args` is the native
tool entry the provider expects.
## Validation [#validation]
When `parameters` is built from the `Schema` DSL, arguments are validated
before execution and malformed calls become error results for the model to
correct, instead of crashing your tool.
## Repairing tool calls [#repairing-tool-calls]
Pass `repairToolCall` to `generateText` / `streamText` to fix a call the
model got wrong (an unknown tool name, mangled arguments) before it runs.
Return a corrected `ToolCall`, or `nil` to leave it unhandled:
```swift
repairToolCall: { call, tools in
call.name == "web_serch" ? ToolCall(id: call.id, name: "web_search", arguments: call.arguments) : nil
}
```
## Multimodal tool results [#multimodal-tool-results]
A tool can hand the model images (not just text) by setting `modelOutput`,
which maps its output to content parts. Providers that support it (Anthropic
today) send them as image blocks; others fall back to the JSON output.
```swift
var screenshot = Tool(name: "screenshot", description: "…", parameters: schema) { _ in
.object(["path": .string("/tmp/shot.png")])
}
screenshot.modelOutput = { _ in [.text("Here's the screen:"), .image(ImageContent(data: pngBytes))] }
```
## Computer use [#computer-use]
The provider computer-use tools are client-executed: the model requests an
action (click, type, scroll), your executor performs it and returns a
screenshot via `.image` content, and the library wires the round-trip.
* **OpenAI**: `OpenAIModel.Tools.computerUse(displayWidth:displayHeight:environment:)`.
`computer_call` actions surface as a `computer_use_preview` tool call; the
screenshot result maps back to `computer_call_output`.
* **Anthropic**: `AnthropicModel.Tools.computer(displayWidthPx:displayHeightPx:)`,
`bash(...)`, `textEditor(...)`. The beta header is set automatically.
Name your executor to match the tool (`computer_use_preview` for OpenAI, the
`name:` you pass on Anthropic), return screenshots as `.image` content, and the
agent loop handles the rest.
## MCP tools [#mcp-tools]
`MCPClient` connects to a Model Context Protocol server and turns its tools
into ordinary tools for the loop. Connect, then pass `tools()` to
`generateText` or `streamText`:
```swift
let mcp = MCPClient(transport: MCPHTTPTransport(
url: URL(string: "https://mcp.example.com/mcp")!,
headers: ["Authorization": "Bearer \(token)"]
))
try await mcp.connect()
defer { Task { await mcp.close() } }
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: "What is on my calendar today?",
tools: try await mcp.tools(),
stopWhen: [stepCountIs(5)]
)
```
Bridged tools validate arguments against the server's schemas and run in the
same loop as local tools. `MCPStdioTransport` (local subprocess) and
`MCPSSETransport` (legacy HTTP+SSE) are drop-in transport swaps.
Before trusting a server's tools, fingerprint them and compare on later
fetches so a server can't silently change a description or widen a schema
after approval:
```swift
let baseline = fingerprintTools(try await mcp.tools())
let drift = detectToolDrift(fingerprintTools(try await mcp.tools()), baseline: baseline)
if drift.hasDrift { /* block or force re-approval */ }
```
See [MCP](/docs/mcp) for transports, pagination, and rug-pull detection in
full.
# Transcription (/docs/transcription)
Every transcription provider conforms to `TranscriptionModel` and works with
`transcribe`.
```swift
let result = try await transcribe(
model: OpenAITranscriptionModel("whisper-1"),
audio: audioData,
mediaType: "audio/mpeg"
)
print(result.text)
for segment in result.segments {
print("\(segment.startSecond)s–\(segment.endSecond)s", segment.text)
}
```
Results include the text plus segments, detected language, and duration when
the provider returns them. When `mediaType` is generic
(`application/octet-stream`, `audio/*`, or empty), the container is sniffed
from the bytes — MP4/M4A from its `ftyp` box, plus WAV, Ogg, FLAC, and MP3 —
so a mislabeled upload still reaches the provider correctly.
## Streaming transcription [#streaming-transcription]
`streamTranscribe` transcribes live audio, emitting updates before the audio
is complete. Models opt in by conforming to `StreamingTranscriptionModel`;
`DeepgramTranscriptionModel` does, over Deepgram's live WebSocket API.
```swift
let result = try streamTranscribe(
model: DeepgramTranscriptionModel("nova-3"),
audio: microphoneChunks, // AsyncThrowingStream
mediaType: "audio/pcm"
)
for try await part in result.fullStream {
switch part {
case .partialTranscript(let text): draft = text // replaces
case .transcriptDelta(let text): transcript += text // appends
case .speechStart, .speechEnd, .segment, .language: break
case .finish(let response): print(response.text)
}
}
```
Interim results arrive as `.partialTranscript` (each one replaces the last)
and finalized text as `.transcriptDelta` (each one appends), so `textStream`
and the `text` promise never double-count a revised phrase.
`fullStream` is single-consumer with no replay: read it once, or skip it and
await `result.text` / `.segments` / `.language`, which drain the stream
internally and cache. Cancelling the result — or a failure before streaming
starts, such as a bad key — cancels the audio stream you passed in, so an
upstream producer never hangs.
Native request fields go in `providerOptions`:
```swift
let result = try await transcribe(
model: OpenAITranscriptionModel("whisper-1"),
audio: audioData,
mediaType: "audio/wav",
providerOptions: ["openai": ["language": "en", "temperature": 0]]
)
```
AssemblyAI, Rev.ai, and Gladia are asynchronous upstream. Their model types
submit, poll, and fetch internally, so your call remains a single `await`.
## Models [#models]
| Provider | Model type | Default or example model | Key |
| ---------------------------------------- | ------------------------------ | ------------------------ | -------------------- |
| [OpenAI](/docs/providers/openai) | `OpenAITranscriptionModel` | `whisper-1` | `OPENAI_API_KEY` |
| [ElevenLabs](/docs/providers/elevenlabs) | `ElevenLabsTranscriptionModel` | `scribe_v2` | `ELEVENLABS_API_KEY` |
| [Deepgram](/docs/providers/deepgram) | `DeepgramTranscriptionModel` | `nova-3` | `DEEPGRAM_API_KEY` |
| [AssemblyAI](/docs/providers/assemblyai) | `AssemblyAITranscriptionModel` | `universal-3-5-pro` | `ASSEMBLYAI_API_KEY` |
| [Rev.ai](/docs/providers/rev-ai) | `RevAITranscriptionModel` | `machine` | `REVAI_API_KEY` |
| [Gladia](/docs/providers/gladia) | `GladiaTranscriptionModel` | `solaria-1` | `GLADIA_API_KEY` |
| [Sarvam](/docs/providers/sarvam) | `SarvamTranscriptionModel` | `saaras:v3` | `SARVAM_API_KEY` |
Groq-hosted Whisper also works through `OpenAITranscriptionModel` with Groq's
base URL; see the [Groq provider page](/docs/providers/groq#models).
# Video generation (/docs/video-generation)
Video providers conform to `VideoModel` and work with `generateVideo`.
```swift
let result = try await generateVideo(
model: XaiVideoModel("grok-imagine-video-1.5"),
prompt: "A paper boat drifting down a rainy street, cinematic",
aspectRatio: "16:9",
duration: 6
)
print(result.urls.first?.absoluteString ?? "no URL")
```
Providers render asynchronously, but their model types handle polling. Some
return URLs and others return inline bytes, so inspect `result.urls` and
`result.videos`; `result.video` is the first inline video when present.
## Animate a still [#animate-a-still]
```swift
let result = try await generateVideo(
model: LumaVideoModel(),
prompt: "The fox blinks and snow falls gently",
image: ImageContent(url: sourceImageURL)
)
```
The shared options are `image`, `aspectRatio`, `duration`, `providerOptions`,
and `maxRetries`.
## Models [#models]
| Provider | Model type | Default or example model | Key |
| ---------------------------- | ---------------- | ------------------------ | -------------- |
| [xAI](/docs/providers/xai) | `XaiVideoModel` | `grok-imagine-video-1.5` | `XAI_API_KEY` |
| [Luma](/docs/providers/luma) | `LumaVideoModel` | `ray-2` | `LUMA_API_KEY` |
# Changelog (/docs/changelog)
### 0.3.0 [#030]
Agents break deep into a run, not on the first call. This release is about
what happens at step fifty: staying inside the context window without
forgetting what you learned, stopping when you say so, asking before acting,
and signing in to hosted MCP servers. Plus a terminal you can watch it all
happen in, Meta's Muse Spark as a new provider, and a large expansion of the
Anthropic, OpenAI, Google, xAI, and Bedrock platform surfaces.
#### Terminal UI [#terminal-ui]
* New `AITUI` library product, the `@ai-sdk/tui` analog, on macOS and Linux.
`runAgentTUI(title:agent:)` (or `transport:`) runs an interactive terminal
chat over any `Agent` or `ChatTransport`.
* What you get in it: streamed markdown, tool cards, reasoning sections,
scrollback, tokens/sec and context readouts, and `y`/`n` tool approvals.
* `TerminalPartDisplayMode` (`.full` / `.collapsed` / `.autoCollapsed` /
`.hidden`) for tools and reasoning, and `ResponseStatisticsMode` for the
statistics readout.
* The renderers are public and pure: `MarkdownTerminalRenderer`,
`TranscriptRenderer`, `AgentTUIModel`, and `AgentTUIRenderer` work without
a terminal, so transcripts can be rendered anywhere.
* `swift run tui-demo` is a runnable demo backed by real models only:
`--ollama` (local, no API key), `--openai`, `--anthropic`, `--on-device`, or
`--pcc`. With no flag it picks the first usable backend and otherwise prints
how to get one.
* The demo verifies an Ollama server is reachable and the model is actually
pulled before starting (`--list` shows what is), and its weather tool calls a
live API, so tool cards show real data.
#### Context management [#context-management]
* `compaction:` on `generateText` / `streamText` / `Agent` keeps a long run
inside the window without discarding what it learned.
* It compresses in proportion to how cheaply information can be recovered.
Bulk tool output is re-fetchable, so it compresses hard. A decision's
rationale exists only in the transcript, so it is kept.
* Three layers, the first two free: structural pinning (system messages, the
goal, failed tool results, the last N steps), a live-reference scan that pins
older messages still mentioned by the working set, then one `generateObject`
call over the remainder using the run's own model.
* `CompactedContext` is a required-field schema rather than a prose summary, so
`goal`, `decisions`, and `deadEnds` cannot be silently dropped.
* Dead ends are the entry most summarizers lose and the costliest, since an
agent that forgets a failed approach retries it. Compaction is re-entrant, so
a long run converges instead of growing.
* `Tool.idempotent()` marks a tool whose result may be replaced by a pointer
(`[omitted: … re-run the tool to retrieve it]`). The tool *call* is always
kept, since it records what was tried. Off by default, so tools with side
effects keep their output in full.
* `CompactionBudget` allocates fractions of the context window instead of a
single threshold, so one config behaves correctly across model tiers.
* `LanguageModel` gained `contextWindow`, with a default implementation that
resolves from the model id, so every provider pack reports a window with no
per-provider wiring and a budget left unset sizes itself to the model.
`ModelContextWindows.register(_:for:)` covers local, fine-tuned, and newly
released ids.
* Unknown ids fall back to 128K, deliberately below the common 200K:
under-estimating compacts early, while over-estimating overflows the window
and fails the request.
* `pruneMessages` is the lossy counterpart: it drops old tool traffic (and,
for `UIMessage` histories, reasoning) outright, with no model call.
#### Timeouts and approvals [#timeouts-and-approvals]
* `timeout:` on `generateText` / `streamText` / `Agent`:
`GenerationTimeout(total:step:firstChunk:chunk:tool:tools:)`. The stall
timers count only content-bearing output, so keep-alive metadata cannot
hold a dead stream open. Tool timeouts return a tool error the model can
react to; the others throw `AIError.timedOut`.
* `toolApproval:` moves approval policy onto the call: a dictionary literal,
`ToolApprovalPolicy.perTool`, or a closure over the whole tool call with
the message history.
* Decisions are `.notApplicable` / `.approved` / `.denied` / `.userApproval`,
each with an optional reason, and land on `step.approvalDecisions`.
`prepareCall` can return one, and per-tool `needsApproval` still works as the
fallback.
* `toolApprovalSecret:` HMAC-signs approval requests and verifies them
fail-closed on replay, so a client cannot forge an approval for a tool it
was never offered. Signatures ride the UI-message wire, and `ChatSession`
and the terminal UI carry them back automatically.
#### MCP: OAuth for hosted servers [#mcp-oauth-for-hosted-servers]
* `MCPOAuthSession` ties an `MCPOAuthClientProvider` to one server and is
passed to a transport as `auth:`. Both `MCPHTTPTransport` and
`MCPSSETransport` accept it.
* It attaches the access token, refreshes on a `401`, and retries once.
Otherwise it throws `AIError.authorizationRequired(url:)` with the URL to
open; hand the browser's redirect back to `complete(callbackURL:)` to finish.
* Full MCP discovery: the `401`'s `WWW-Authenticate` names the
protected-resource metadata document (RFC 9728), which names the
authorization server — commonly a different host than the MCP server, so
the single-host shortcut most clients take does not work.
* Well-known URLs follow RFC 8414, keeping the issuer path (`/tenant/acme` →
`/.well-known/oauth-authorization-server/tenant/acme`) with the origin-only
form as fallback.
* PKCE `S256` (and a clear error rather than a silent downgrade to `plain`
where CryptoKit is unavailable), the RFC 8707 `resource` indicator on the
authorization, token, and refresh requests, `state` verification on the
callback, and dynamic client registration when the server advertises a
`registration_endpoint`.
* `MCPOAuthFlow` exposes each step (`discover`, `registerClientIfNeeded`,
`startAuthorization`, `handleCallback`, `refresh`) for apps that drive the
flow themselves.
#### Runtime context and tools [#runtime-context-and-tools]
* `runtimeContext:` carries server-side state through a run without touching
the prompt: readable in `prepareStep`, replaceable from there, and recorded
on every `StepResult`.
* Tools can validate their `toolsContext` entry with
`.withContextSchema(_:)` and compute their description from it with
`.describing { context in ... }`.
* `Tool.dynamic(...)` marks runtime-schema tools, carried as `dynamic` on the
tool chunks so a UI can tell them from compiled-in tools. MCP tools are
dynamic automatically.
* `TelemetrySettings` adds a per-call function id, metadata, an off switch,
and allow-lists (`includeRuntimeContext` / `includeToolsContext`) for which
context keys reach spans. Nothing from either context is recorded unless you
name it.
* `filterActiveTools`, `generateId` / `createIdGenerator`, and `GeneratedFile`
accessors (`base64`, `bytes`) on image results.
* New provider-defined tools: OpenAI `imageGeneration`, `shell`,
`localShell`, `applyPatch`, `customTool`, `toolSearch`,
`programmaticToolCalling`, and hosted `mcpServer`; Anthropic `advisor`,
`toolSearchBm25`, `toolSearchRegex`; Google `vertexRagStore`.
#### Files and remote content [#files-and-remote-content]
* `uploadFile(api:data:filename:)` works with any `FileUploadAPI` and returns
an `UploadedFile` you can turn into a message part. `providerReference` on
`FileContent` / `ImageContent` reaches OpenAI as `file_id` and Anthropic as
a `file` source, and is ignored by providers it was not minted for.
* Models declare URL support with `supportsRemoteURL(_:mediaType:)`, and the
loop downloads and inlines what a model cannot fetch.
#### Middleware [#middleware]
* `wrapEmbeddingModel`, `wrapImageModel`, and `wrapProvider` extend
middleware beyond language models.
* `.extractJson()` strips markdown code fences; `.addToolInputExamples()`
folds a tool's new `inputExamples` into its description.
#### Transcription [#transcription]
* `streamTranscribe` transcribes live audio through
`StreamingTranscriptionModel`, implemented for Deepgram's live WebSocket
API. Interim text arrives as `.partialTranscript` and finalized text as
`.transcriptDelta`, so revisions never double-count.
* `transcribe` sniffs MP4/M4A, WAV, Ogg, FLAC, and MP3 from the audio bytes
when the declared media type is generic.
* `resampleAudio`, `encodePCM16`, and `decodePCM16` for feeding realtime and
streaming audio paths.
#### Chat UI and transports [#chat-ui-and-transports]
* `TextStreamChatTransport`, `consumeStream`, `validateUIMessages` /
`safeValidateUIMessages`, and
`lastAssistantMessageIsCompleteWithToolCalls` /
`lastAssistantMessageIsCompleteWithApprovalResponses`.
* `HTTPChatTransport` gained `prepareSendMessagesRequest` and
`prepareReconnectToStreamRequest` for per-request headers, body, and URL.
#### Anthropic [#anthropic]
* Every tool now carries Anthropic's optional definition properties through
`Tool.loading(_:)`: `strict` (schema-validated tool names and inputs),
`defer_loading` (keep a tool out of the cached system prompt until tool
search surfaces it), `allowed_callers` (restrict a tool to the code
execution sandbox), `cache_control` breakpoints, and
`eager_input_streaming`. `.ephemeralCache()` and `.codeExecutionOnly()`
are shorthands.
* `inputExamples` now ship natively as `input_examples` on Anthropic instead
of being folded into the description.
* `AnthropicModel.Tools.mcpToolset(...)` wires the MCP connector, with the
`mcp-client-2025-11-20` beta header applied automatically. Beta headers
also cover the newest tool versions (`web_search_20260318`,
`web_fetch_20260318` / `20260309`, `code_execution_20260521`).
* `AnthropicBatchClient` (Message Batches: create, list, retrieve, streamed
JSONL `results`, cancel, delete), `AnthropicModelsClient`, and
`AnthropicModel.countTokens(_:tools:system:)`.
#### OpenAI [#openai]
* **Multi-agent** on the Responses API: pass
`multiAgent: OpenAIModel.MultiAgent(maxConcurrentSubagents: 3)` and the
model spawns and coordinates a subagent tree itself. The
`responses_multi_agent=v1` beta header is added for you.
* Hosted `multi_agent_call` items (spawn, message, follow-up, wait, interrupt,
list) surface as provider metadata rather than tool calls, so your app must
not execute them. Ordinary function calls from any agent in the tree still
run through the normal tool loop.
* `OpenAIConversationsClient` for the Conversations API: create (from
messages or raw items), fetch, update metadata, list/add/delete items.
* `OpenAIVectorStoresClient` including `search`, file attach/detach, and
expiry windows, the store side of `file_search`.
* `OpenAIBatchClient`, `OpenAIContainersClient`, and
`OpenAIModerationsClient` (which returns the flagged categories, sorted).
* `OpenAIVideoModel` for Sora: create, poll to completion, download content,
plus `remix`, `list`, and `delete`.
#### Google [#google]
* `GoogleInteractionsModel` speaks Google's newer `POST /v1beta/interactions`
surface, the one they say all new models, tools, and agentic features now
launch on, while `generateContent` (still `GoogleModel`) is labelled legacy.
* Messages map to `input` steps, thought steps arrive as `.reasoningDelta`,
and the SSE `step.start` / `step.delta` / `step.stop` protocol decodes into
ordinary stream parts.
* Server-side state via `previousInteractionID`, `background: true` for long
runs, and `create` / `retrieve` / `cancel` / `delete`. **`store` defaults to
`false`** even though Google's API defaults it to `true`, so nothing is
retained unless you opt in.
* Agents share the endpoint: `GoogleInteractionsModel.agent(…)` reaches Deep
Research and Antigravity, with `agentConfig` replacing `generation_config`.
* `GoogleEmbeddingModel` (`embedContent` / `batchEmbedContents`, task types,
output dimensionality). Gemini embeddings were previously unsupported.
* Media: `GoogleImageModel` (Imagen), `GoogleVideoModel` (Veo, with operation
polling and URI download), `GoogleSpeechModel` (Gemini TTS), and
`GoogleMusicModel` (Lyria).
* Platform: `GoogleFilesClient` (resumable upload plus `waitUntilActive`),
`GoogleCachedContentClient` (explicit context caching),
`GoogleBatchClient` (`batchGenerateContent` and `asyncBatchEmbedContent`),
and `GoogleModel.countTokens`.
#### xAI [#xai]
* **Fixed:** `XaiCollectionsClient` sent collection management to
`api.x.ai`, but xAI serves it from `https://management-api.x.ai/v1` behind
a separate Management API key. The client now holds both endpoints and
routes each call, and `addDocument` puts the file id in the path
(`POST /v1/collections/{id}/documents/{file_id}`) as documented.
* Collections gained `update` (PUT), `listDocuments`, `document`,
`documents` (`:batchGet`), and `regenerateIndices` (PATCH), and every
management call takes `teamID:` plus paging and filter parameters.
* New `XaiModelsClient` for `/v1/models` and the richer `language-models`,
`image-generation-models`, and `video-generation-models` catalogs.
* New `XaiPlatformClient`: `apiKeyInfo()`, `tokenizeText`, SIP phone numbers
(`/v2/phone-numbers`), `referCall` / `hangUpCall`, and the TTS voice and
custom-voice catalogs.
* `XaiBatchClient` gained `addRequests` and `cancel`; `XaiFilesClient` gained
list paging/filtering and `update`; `XaiModel` gained `retrieveResponse`
and `deleteResponse`.
#### Amazon Bedrock [#amazon-bedrock]
* `BedrockMantleProvider` targets Bedrock's `bedrock-mantle` endpoint:
`responses(_:)` and `chat(_:)` for the OpenAI-compatible surfaces and
`messages(_:)` for the Anthropic Messages surface, with Bedrock API-key
auth. `BedrockModel` continues to serve `bedrock-runtime` and SigV4.
#### Meta [#meta]
* `MetaModel` runs Muse Spark on Meta Model API's Responses endpoint
(`MetaModel.chat(...)` for chat completions). Key from `MODEL_API_KEY`,
base URL `https://api.meta.ai/v1`, and a 1M-token context window.
* `MetaModel.Tools.webSearch(searchContextSize:userLocation:)` and
`.toolSearch(...)` are the server-side search-grounding and deferred
tool-loading path; citations arrive as `StreamPart.source`.
* Muse Spark always reasons, so `.none` is dropped rather than sent (the
API answers it with a 400), and unlike OpenAI's reasoning models it still
accepts `temperature` and `topP`, so both are forwarded.
#### Fixes [#fixes]
* Bedrock silently dropped URL-only images and files. Models now declare what
they can fetch, and the loop inlines the rest.
* A reused tool call id started a new UI tool part instead of overwriting the
finished one.
* Cancelling during tool execution aborts the run rather than surfacing as a
tool error.
* Tool results from resumed approvals appear in `steps` and
`result.toolResults`, not only in `messages`.
#### Errors and utilities [#errors-and-utilities]
* New `AIError` cases: `timedOut`, `invalidToolInput`, `invalidToolContext`,
`missingToolResults`, `toolCallRepairFailed`, `invalidToolApproval`,
`unsupportedFunctionality`, and `authorizationRequired`.
* `JSONValue` gained an `Int` subscript, so `value["input"]?[0]?["type"]`
works for array elements.
#### Docs [#docs]
* A new [API reference](/docs/reference) covering every public function and the
types you construct directly. Signatures are extracted from `Sources/AI` on
every build, so they cannot drift from the code.
* New [Foundations](/docs/foundations) section for the ideas the rest of the
docs used to assume: the loop, models and tokens, prompts and messages, tool
calling, and streaming.
* New [Troubleshooting](/docs/troubleshooting) section, one page per symptom,
named after what you would actually search for.
* The catch-all Advanced page is gone, split into
[Context management](/docs/context-management),
[Middleware](/docs/middleware), [Runtime context](/docs/runtime-context),
[Telemetry](/docs/telemetry), and [Errors and retries](/docs/errors).
Runtime context, telemetry, and history handling had been nested under
Middleware, which is not what any of them are.
* New pages for the [terminal UI](/docs/terminal-ui) and for
[timeouts and approvals](/docs/timeouts-and-approvals), and the agent skill
gained references for context management, the terminal UI, timeouts and
approvals, and runtime context.
### 0.2.0 [#020]
#### New providers [#new-providers]
* Chat: `MoonshotModel` (Kimi), `AlibabaModel` (Qwen, with native thinking,
`AlibabaEmbeddingModel`, and `AlibabaVideoModel` for Wan), and
`HuggingFaceModel` (the router's Responses endpoint).
* Retrieval: `VoyageEmbeddingModel` and `VoyageRerankingModel`.
* Voice: `CartesiaSpeechModel` and `CartesiaTranscriptionModel`.
* Images: `BlackForestLabsImageModel` (FLUX), `ByteDanceImageModel` (Seedream),
`ProdiaImageModel`, and `QuiverAIImageModel` (prompt-to-SVG).
* Video: `ByteDanceVideoModel` (Seedance) and `KlingVideoModel` (JWT-signed).
#### Provider capabilities [#provider-capabilities]
* xAI gained image, speech, and transcription packs, video `editVideo` /
`extendVideo`, deferred completions and response compaction, and the
`XaiFilesClient` / `XaiBatchClient` / `XaiCollectionsClient` REST clients.
Live Search (`SearchParameters`) is deprecated in favor of the
`web_search` / `x_search` tools.
* OpenAI: computer use (`OpenAIModel.Tools.computerUse`) with the
`computer_call` / `computer_call_output` round-trip, hosted-tool calls and
refusals now surface on the Responses stream, and `OpenAIResponsesClient`
manages stored/background responses (retrieve, delete, cancel, compact,
input items, token counting).
* Bedrock now signs with SigV4 when given IAM credentials (bearer-token auth
still works).
* Groq server tools (`browserSearch`, `codeExecution`) for the compound models.
* Google grounding chunks surface as `.source`; the full grounding metadata,
OpenAI logprobs, Anthropic cache-creation tokens, Bedrock guardrail traces,
and Perplexity images / related questions all collect on
`result.providerMetadata`.
#### MCP [#mcp]
* `MCPStdioTransport` launches a local server as a subprocess and speaks
newline-delimited JSON-RPC over its pipes; `MCPSSETransport` speaks the
legacy HTTP+SSE protocol. Both bound each call with `requestTimeout`.
* `MCPHTTPTransport` carries `mcp-session-id` sessions, and `tools()` follows
`nextCursor` pagination.
* `fingerprintTools` / `detectToolDrift` catch a server that changes its tool
definitions after you approved them (rug pull).
#### Core [#core]
* `smoothStream` re-chunks a text stream by word or line for calmer UI.
* `generateObjectArray` returns a typed array; `repairText` salvages
unparseable JSON; `output:` on `generateText` produces a structured object
alongside tool calls (`result.experimentalOutput`).
* `repairToolCall` fixes a malformed tool call before it runs; tools can
return images through `Tool.modelOutput` (multimodal tool results).
* New `streamText` callbacks `onChunk` and `onAbort`; `maxImagesPerCall`
batches image generation; a `.providerMetadata` channel runs through the
whole stream.
#### Docs [#docs-1]
* Examples and guides for every new provider and feature, a computer-use and a
self-repair guide, and the agent skill refreshed to match.
### 0.1.1 [#011]
#### Providers [#providers]
* First-class model packs for AI Gateway, Baseten, Cerebras, DeepInfra,
Fireworks, LM Studio, Ollama, OpenRouter, Sarvam, Together AI, and Vercel.
Dedicated types (`BasetenModel`, `CerebrasModel`, and so on) built on a
shared OpenAI-compatible base, replacing the `OpenAICompatibleProvider`
factory functions for these providers (still available, deprecated).
`MistralModel`, `PerplexityModel`, `DeepSeekModel`, and `GroqModel` moved
onto the same shared base for consistency, and every pack listed here now
accepts `queryParams`.
* Together AI, DeepInfra, and Baseten also get dedicated embedding model
types: `TogetherAIEmbeddingModel`, `DeepInfraEmbeddingModel`, and
`BasetenEmbeddingModel`.
* OpenRouter's reasoning effort now maps to its actual nested
`reasoning: {"effort": ...}` wire format instead of the generic
`reasoning_effort` field other OpenAI-compatible providers use.
#### Examples [#examples]
* Reorganized into `Examples/Features/` (the same numbered walkthroughs as
before, plus a new 23-WorkflowGuides) and `Examples/Providers/`, a
minimal runnable example for every supported provider.
### 0.1.0 [#010]
The first release. A Swift port of the Vercel AI SDK for iOS and macOS.
#### Core [#core-1]
* `generateText` and `streamText`, with the tool-calling loop, steps, and
streamed reasoning.
* Structured output: `generateObject`, `streamObject` (plus `elementStream`
for arrays), `generateEnum`, and `generateJSON`.
* `embed`, `embedMany`, `cosineSimilarity`, and `rerank`.
* A `Schema` DSL that validates arguments and output before decoding.
* `ReasoningEffort` maps to each provider's native reasoning controls;
reasoning streams as `.reasoningDelta`.
#### Agents and tools [#agents-and-tools]
* `Agent`, the `ToolLoopAgent` analog: a model bundled with instructions,
tools, and loop settings. Also works as a `ChatTransport`.
* Loop control (`stopWhen`, `stepCountIs`, `hasToolCall`), plus
`prepareCall`, `prepareStep`, and `toolOrder`.
* Closure-based `Tool` with typed arguments, execution context, approvals,
and client-side tools.
* Subagents: any `Agent` becomes a tool via `asTool`.
* Provider-defined (server-executed) tools with typed builders under
`.Tools` for xAI, OpenAI, Google, and Anthropic: web and X
search, code execution, file search, computer use, and more. Calls and
results come back as provider-executed `.toolCall` / `.toolResult` parts,
and Anthropic's required beta headers are added for you.
* MCP tools over HTTP via `MCPClient`.
#### Providers [#providers-1]
Native packs that speak each provider's own wire:
* `OpenAIModel` (Responses and `.chat`), `AnthropicModel`, `GoogleModel`,
`GoogleVertexModel`, `AzureOpenAIProvider`, `BedrockModel`, `XaiModel`
(with typed `SearchParameters` live search), `GroqModel`,
`DeepSeekModel`, `MistralModel`, `PerplexityModel`, `CohereModel`.
* `OpenAICompatibleProvider` factories for Together, Fireworks, Cerebras,
OpenRouter, DeepInfra, Baseten, Vercel, Gateway, Ollama, LM Studio, and
Sarvam.
* Sarvam: `sarvam-30b` / `sarvam-105b` reasoning chat, `SarvamSpeechModel`
(Bulbul), and `SarvamTranscriptionModel` (Saaras) for Indian languages.
* `ProviderRegistry` and `customProvider` for `"provider:model"` strings
and aliases.
#### Middleware [#middleware-1]
* `wrapLanguageModel` with `extractReasoning`, `simulateStreaming`,
`defaultSettings`, and `cache` (backed by `LanguageModelCache`; ships an
in-process `InMemoryLanguageModelCache`).
* Hooks: `transformRequest`, `wrapStream`, and `wrapCall` (wrap the whole
call and decide whether to run the model).
#### Media [#media]
* Images: OpenAI, fal, Luma, Replicate.
* Speech: OpenAI, ElevenLabs, LMNT, Hume, Deepgram, Sarvam.
* Transcription: OpenAI, ElevenLabs, Deepgram, AssemblyAI, Rev.ai, Gladia,
Sarvam.
* Video: xAI, Luma.
#### UI and realtime [#ui-and-realtime]
* `ChatSession`, `CompletionSession`, and `ObjectSession` as `@Observable`
objects for SwiftUI.
* The UI-message stream protocol, wire-compatible with the AI SDK's
`/api/chat` route: `ChatTransport`, `HTTPChatTransport`,
`LocalChatTransport`, `readUIMessageStream`, message metadata, and stream
resumption.
* Realtime voice over WebSockets (OpenAI, Google (Gemini Live), and xAI)
through `RealtimeSession`.
#### On-device [#on-device]
* `FoundationModelsModel` runs Apple Intelligence through the same API as
the cloud providers, with nothing leaving the device.
#### Tooling [#tooling]
* `AITelemetry` spans, structured `AIError`, and testing helpers in the
`AITesting` module.
# Running out of context on a long run (/docs/troubleshooting/context-window-exceeded)
## Symptom [#symptom]
An HTTP 400 from the provider mentioning tokens or context length, or a run
that quietly gets worse the longer it goes.
## Why it happens [#why-it-happens]
Every tool result stays in the history. A loop that reads files or searches
accumulates context fast, and the biggest entries are usually tool output that
is no longer relevant.
## Fix [#fix]
Turn on compaction. It triggers itself when the history outgrows the working-set
budget and keeps the goal, the decisions, and the failed approaches while
compressing bulk tool output:
```swift
compaction: Compaction()
```
Mark read-only tools `.idempotent()` so their output can be replaced by a
pointer the model can re-fetch. For a cheap structural fix with no model call,
[`pruneMessages`](/docs/reference/prune-messages) deletes old tool traffic
outright.
If the budget looks wrong for your model, check `contextWindow` resolves: an
unrecognized model id falls back to a conservative 128K.
## See also [#see-also]
* [Context management](/docs/context-management)
* [pruneMessages](/docs/reference/prune-messages)
# AIError.decoding (/docs/troubleshooting/decoding-error)
## Symptom [#symptom]
`AIError.decoding(...)` on an otherwise successful request.
## Why it happens [#why-it-happens]
The HTTP call succeeded and the body did not look like what the provider's API
documents.
Usually one of: the provider returned an error document with a 200, or the
endpoint is "OpenAI-compatible" but diverges on a field the SDK reads. Local
servers and gateways are the usual suspects.
## Fix [#fix]
Log the raw body. If it is an error document, the real problem is in the
message it carries.
If it is a genuine shape difference, the OpenAI-compatible base accepts
`headers:` and `queryParams:` overrides that often bridge the gap. For a
persistent mismatch, wrap the model and normalize the response in middleware.
## See also [#see-also]
* [OpenAI-compatible providers](/docs/providers/openai-compatible)
* [Middleware](/docs/middleware)
# Tool parts overwrite each other in the UI (/docs/troubleshooting/duplicate-tool-parts-in-ui)
## Symptom [#symptom]
Two separate tool calls render as one, or a finished tool card is replaced by a
running one.
## Why it happens [#why-it-happens]
The UI-message reducer keys tool parts by call id. When a provider reuses an
id across calls, or a transport replays one, the second call lands on the first
one's part.
The SDK starts a new UI tool part when it sees a reused id rather than
overwriting the finished one, so if you are still seeing collapsing the ids are
being rewritten somewhere in your own transport.
## Fix [#fix]
Check that your transport passes tool call ids through unchanged. If you mint
ids yourself, use [`generateId`](/docs/reference/generate-id) or
[`createIdGenerator`](/docs/reference/create-id-generator) so they match
the format the loop produces and stay unique.
## See also [#see-also]
* [Chat UI](/docs/chat-ui)
* [Streaming protocol](/docs/streaming-protocol)
# HTTP 401 from a provider (/docs/troubleshooting/http-401-unauthorized)
## Symptom [#symptom]
`AIError.http(status: 401, body: ...)`.
## Why it happens [#why-it-happens]
Every provider pack falls back to a conventional environment variable when
`apiKey:` is omitted, and an unset variable resolves to an empty string rather
than crashing. So a missing key looks exactly like a wrong one, and only fails
at request time.
A GUI-launched app is the classic case: it does not inherit the environment
your shell exports, so the key is present in Terminal and absent in the app.
## Fix [#fix]
Pass the key explicitly when the process environment is not reliable:
```swift
AnthropicModel("claude-sonnet-5", apiKey: storedKey)
```
Check the body in the error too. Providers usually say whether the key is
unknown, revoked, or lacking access to the specific model.
## See also [#see-also]
* [Providers](/docs/providers)
# Troubleshooting (/docs/troubleshooting)
Each page here covers one symptom: what you see, why it happens, and the fix.
For the full list of error cases with their meanings, see the
[errors reference](/docs/reference/errors).
## Tools [#tools]
## Output [#output]
## Limits and timeouts [#limits-and-timeouts]
## Providers and auth [#providers-and-auth]
## Streaming and UI [#streaming-and-ui]
# AIError.invalidToolApproval (/docs/troubleshooting/invalid-tool-approval)
## Symptom [#symptom]
`AIError.invalidToolApproval(...)` when resuming after an approval.
## Why it happens [#why-it-happens]
When `toolApprovalSecret` is set, approvals are HMAC-signed and verified
fail-closed. The check rejects a response whose signature does not match, one
that was replayed, or one for a tool that was never offered.
In practice this is usually benign: the secret changed between the request and
the response, or a client dropped the `signature` field while round-tripping
the message.
## Fix [#fix]
Make sure the same `toolApprovalSecret` is used for the call that requested the
approval and the call that resumes it, and that your client preserves the
signature verbatim rather than reconstructing the approval object.
`ChatSession` and the terminal UI carry signatures back automatically. If you
wrote your own transport, that is the first place to look.
## See also [#see-also]
* [Timeouts and approvals](/docs/timeouts-and-approvals)
# AIError.invalidToolInput (/docs/troubleshooting/invalid-tool-input)
## Symptom [#symptom]
`AIError.invalidToolInput(tool:reason:)`. The tool's own code never executed.
## Why it happens [#why-it-happens]
Arguments are validated against the tool's schema before the executor is
called, so a malformed call fails closed rather than reaching your code with
missing fields.
Smaller models get this wrong more often, especially with deeply nested schemas
or unusual enum values.
## Fix [#fix]
Supply `repairToolCall` to fix a call and retry it instead of failing the run:
```swift
repairToolCall: { call, tools in
guard call.name == "search" else { return nil }
var fixed = call
fixed.arguments = normalize(call.arguments)
return fixed
}
```
Flattening the schema helps more than prompting does. So does adding
`inputExamples` to the tool, which Anthropic sends natively and other providers
can receive through the `.addToolInputExamples()` middleware.
## See also [#see-also]
* [Tools](/docs/tools)
* [Errors](/docs/reference/errors)
# AIError.authorizationRequired from an MCP server (/docs/troubleshooting/mcp-authorization-required)
## Symptom [#symptom]
`AIError.authorizationRequired(url:)` when listing or calling MCP tools.
## Why it happens [#why-it-happens]
The server returned a 401 and the session had no token, or had one it could not
refresh. The URL in the error is the sign-in page.
This is the normal first-run path for a hosted server, not necessarily a
failure.
## Fix [#fix]
Open the URL, then hand the redirect back:
```swift
let tokens = try await auth.complete(callbackURL: redirect)
```
If it recurs on every run, your `MCPOAuthClientProvider` is not persisting
tokens and client registration. Both need to survive a relaunch.
If sign-in itself fails, the usual cause is `saveState` / `state` left as the
no-op defaults, which makes callback verification silently pass and then fail
later.
## See also [#see-also]
* [MCP](/docs/mcp)
# AIError.missingToolResults (/docs/troubleshooting/missing-tool-results)
## Symptom [#symptom]
A call throws `AIError.missingToolResults([...])`, listing one or more tool
call ids.
## Why it happens [#why-it-happens]
Every `toolCall` in an assistant message needs a matching `toolResult` before
that conversation can go back to the model. The array in the error is the ids
that are unanswered.
This nearly always happens in a client-side tool flow: the model asked for a
tool, the app was supposed to run it and post the result back, and the next
turn started before that happened. It also happens after pruning a history by
hand and dropping a result while keeping its call.
## Fix [#fix]
Check the history is complete before resuming:
```swift
if lastAssistantMessageIsCompleteWithToolCalls(messages) {
try await resume(messages)
}
```
If you prune histories yourself, use
[`pruneMessages`](/docs/reference/prune-messages) rather than filtering by
hand. Dropping a tool call there also drops its result, so the pair can never
get split.
## See also [#see-also]
* [Tools](/docs/tools)
* [pruneMessages](/docs/reference/prune-messages)
# AIError.noObjectGenerated (/docs/troubleshooting/no-object-generated)
## Symptom [#symptom]
`AIError.noObjectGenerated(...)` from `generateObject`, `generateObjectArray`,
or `streamObject`.
## Why it happens [#why-it-happens]
The model's output was not valid JSON, or it was valid JSON that did not
satisfy the schema.
The most common cause is not the model at all: the response hit
`maxOutputTokens` and got cut off. A truncated object is invalid JSON, and the
error looks identical to a model that simply got it wrong.
## Fix [#fix]
Raise `maxOutputTokens` first. It fixes this more often than any prompt change,
and costs nothing when the output is short anyway.
If the output is complete but malformed, pass `repairText` to salvage it:
```swift
repairText: { text in
text.trimmingCharacters(in: CharacterSet(charactersIn: "` \n"))
}
```
For models that wrap JSON in markdown fences even when asked not to, the
`.extractJson()` middleware strips them.
## See also [#see-also]
* [Structured output](/docs/structured-output)
* [generateObject](/docs/reference/generate-object)
# Cannot reach a local model server (/docs/troubleshooting/ollama-connection-refused)
## Symptom [#symptom]
`AIError.transport(...)` mentioning a refused connection to `localhost:11434`.
## Why it happens [#why-it-happens]
`OllamaModel` defaults to `http://localhost:11434/v1`. The server is not
running, is bound elsewhere, or the model was never pulled.
## Fix [#fix]
Start the server and pull a model:
```bash
ollama serve
ollama pull qwen3
```
For a non-default host, pass `baseURL:` explicitly. Note that a model Ollama has
not pulled fails at request time, not at construction, so a typo in the model id
looks like a server problem.
## See also [#see-also]
* [Ollama](/docs/providers/ollama)
# Apple on-device model is unavailable (/docs/troubleshooting/on-device-model-unavailable)
## Symptom [#symptom]
A run reports the on-device model as unavailable, with an availability reason.
## Why it happens [#why-it-happens]
Apple Intelligence has to be enabled and its assets downloaded before
`FoundationModelsModel` can serve requests. Availability is a device and OS
state, not something the SDK controls.
## Fix [#fix]
Check before constructing, and fall back to a cloud model when it is not ready:
```swift
let model: any LanguageModel = FoundationModelsModel.isAvailable
? FoundationModelsModel()
: AnthropicModel("claude-sonnet-5")
```
`FoundationModelsModel.availability` carries the reason, which is worth
surfacing rather than swallowing. Foundation Models also reports no token
usage, so statistics readouts stay empty on that path.
## See also [#see-also]
* [On-device models](/docs/on-device)
# A stream stops before the answer finishes (/docs/troubleshooting/stream-ends-early)
## Symptom [#symptom]
Streamed text stops partway and the run reports success.
## Why it happens [#why-it-happens]
Check `finishReason` first. `.length` means the response hit
`maxOutputTokens` and stopped there, which is not an error as far as the API is
concerned.
Reasoning models make this more likely, because thinking tokens count toward
the same budget as the answer.
## Fix [#fix]
Raise `maxOutputTokens`. The SDK default is deliberately small, and reasoning
models need real headroom.
If `finishReason` is `.stop` and the text still looks truncated, the model
genuinely ended there and the fix is in the prompt.
## See also [#see-also]
* [Generating text](/docs/generating-text)
* [streamText](/docs/reference/stream-text)
# AIError.timedOut (/docs/troubleshooting/timed-out)
## Symptom [#symptom]
`AIError.timedOut(scope:limit:tool:)`.
## Why it happens [#why-it-happens]
Read `scope` before changing anything, because each one means something
different:
`.total` is the whole call including every step. `.step` is one model call.
`.firstChunk` means a stream produced nothing at all within the limit.
`.chunk` means a stream started and then stalled. A populated `tool` means one
specific tool ran long.
Stall timers only count content-bearing output, so provider keep-alive metadata
cannot hold a dead stream open.
## Fix [#fix]
Raise the scope that actually fired rather than the total. An agent loop that
trips `.total` usually needs more steps allowed, not a longer clock.
Tool timeouts do not throw by default. They come back as a tool error the model
can read and react to, which is normally what you want:
```swift
timeout: GenerationTimeout(total: .seconds(600), tool: .seconds(30))
```
## See also [#see-also]
* [Timeouts and approvals](/docs/timeouts-and-approvals)
# A tool is offered but never runs (/docs/troubleshooting/tool-never-executes)
## Symptom [#symptom]
The result comes back with a `toolCall` in `toolCalls` but no matching entry in
`toolResults`, and the loop ended.
## Why it happens [#why-it-happens]
The tool has no executor. A `Tool` built without an `execute` closure is a
client-side tool by design: the SDK surfaces the call and expects your app to
run it and post the result back. `hasExecutor` is `false` for these.
Provider-defined tools behave the same way from the loop's side, since they run
on the provider rather than locally.
## Fix [#fix]
If the tool was meant to run locally, give it an `execute` closure. If it is
genuinely client-side, handle the call in your UI and send a result back before
resuming, then check with
[`lastAssistantMessageIsCompleteWithToolCalls`](/docs/reference/last-assistant-message-is-complete-with-tool-calls).
## See also [#see-also]
* [Tools](/docs/tools)
# AIError.unknownTool (/docs/troubleshooting/unknown-tool)
## Symptom [#symptom]
`AIError.unknownTool("some_name")` thrown mid-loop.
## Why it happens [#why-it-happens]
The model asked for a tool the call does not know about. Two common routes:
The tool array changed between turns while the history still refers to the old
one. The model sees a prior turn where `search` existed, and asks for it again
on a call where it was not passed.
Or `activeTools` narrowed the set. A tool filtered out of `activeTools` is
hidden from the model, but a model working from history may still ask.
## Fix [#fix]
Keep the tool array stable for the life of a conversation. When you do need to
vary it per step, do it in `prepareStep` so the change is visible to the loop
rather than applied behind it.
If the tool genuinely no longer exists, prune the history so the model stops
seeing evidence of it.
## See also [#see-also]
* [Tools](/docs/tools)
* [filterActiveTools](/docs/reference/filter-active-tools)
# AI Gateway (/docs/providers/ai-gateway)
`AIGatewayModel` reads `AI_GATEWAY_API_KEY` and targets
`https://ai-gateway.vercel.sh/v1`.
```swift
let model = AIGatewayModel("anthropic/claude-sonnet-5")
let result = try await generateText(model: model, prompt: "Say hello.")
print(result.text)
```
Use provider-qualified model IDs accepted by the gateway. Capabilities such
as tools, structured output, vision, and reasoning depend on the routed model.
## Models [#models]
The gateway catalog changes continuously. Current examples include:
| Model ID | Provider |
| --------------------------- | --------- |
| `openai/gpt-5.6-sol` | OpenAI |
| `anthropic/claude-sonnet-5` | Anthropic |
| `xai/grok-4.5` | xAI |
Fetch the live catalog from [`GET /v1/models`](https://ai-gateway.vercel.sh/v1/models)
or browse Vercel's [models and providers documentation](https://vercel.com/docs/ai-gateway/models-and-providers).
# Alibaba (/docs/providers/alibaba)
`AlibabaModel` reads `ALIBABA_API_KEY` and targets
`https://dashscope-intl.aliyuncs.com/compatible-mode/v1`.
```swift
let model = AlibabaModel("qwen3-max")
let result = try await generateText(model: model, prompt: "Say hello.")
print(result.text)
```
Tools, structured output, vision, and reasoning ride the shared
chat-completions wire; `qwen3-*-thinking` and `qwq-*` models stream reasoning
as `.reasoningDelta`. The unified `reasoning` parameter maps to Qwen's
`enable_thinking` + `thinking_budget` (`.none` disables thinking; an effort
level enables it with a budget from 1,024 up to 38,912 tokens).
## Embeddings [#embeddings]
`AlibabaEmbeddingModel` works with [`embed` and `embedMany`](/docs/embeddings):
```swift
let vectors = try await embedMany(
model: AlibabaEmbeddingModel("text-embedding-v4"),
values: documents
)
```
## Video [#video]
`AlibabaVideoModel` conforms to `VideoModel` and works with
[`generateVideos`](/docs/video-generation). It submits an async DashScope task
and polls until it's `SUCCEEDED`. Pass an image for image-to-video (or use an
`-i2v` model id); extra Wan parameters go under the `alibaba` provider key:
```swift
let video = try await generateVideos(
model: AlibabaVideoModel("wan2.6-t2v"),
prompt: "A paper boat sailing down a rain gutter"
)
```
## Models [#models]
As of July 2026:
| Model ID | Notes |
| ----------------------------------------- | ------------------------------------ |
| `qwen3-max` / `qwen3.7-max` | Flagship |
| `qwen-plus` / `qwen-flash` / `qwen-turbo` | Balanced, fast, cheap tiers |
| `qwen3-coder-plus` / `qwen3-coder-flash` | Code |
| `qwen3-235b-a22b-thinking-2507` | Reasoning MoE |
| `qwq-32b` / `qwq-plus` | Reasoning |
| `text-embedding-v4` | Embeddings (`AlibabaEmbeddingModel`) |
| `wan2.7-t2v` / `wan2.6-t2v` / `-i2v` | Video (`AlibabaVideoModel`) |
See DashScope's [model list](https://www.alibabacloud.com/help/en/model-studio/models)
for availability. Any id the API serves works.
# Anthropic (/docs/providers/anthropic)
```swift
let model = AnthropicModel("claude-fable-5") // or claude-sonnet-5
```
Key from `ANTHROPIC_API_KEY`; base URL `https://api.anthropic.com/v1`.
## Features [#features]
* Tools, vision, and PDF documents as message parts.
* Structured output rides a forced tool call — Claude has no JSON mode,
so `generateObject` defines a tool whose arguments are your schema and
forces the model to call it.
* Thinking streams as `.reasoningDelta`; the `reasoning` parameter
translates per model tier (below).
* `usage.cachedInputTokens` is populated on prompt-cache hits;
`result.providerMetadata["anthropic"]["cacheCreationInputTokens"]` reports
tokens written to the cache.
* Anthropic-native fields (`thinking`, cache-control blocks) go through
`providerOptions` and win over anything the library sets.
## Server-side tools [#server-side-tools]
Anthropic's built-in tools have typed builders under
`AnthropicModel.Tools` — `webSearch`, `webFetch`, `codeExecution`,
`bash`, `textEditor`, `computer(displayWidthPx:displayHeightPx:)`, and
`memory`. Pass them in `tools:`; the required `anthropic-beta` flags are
added to the request for you:
```swift
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: "Search the web and summarize what's new.",
tools: [AnthropicModel.Tools.webSearch(maxUses: 3)]
)
```
Each builder takes a `version:` for the dated tool variants
(`web_search_20250305`, `computer_20250124`, ...); the defaults track
the current stable release. Newer versions are wired too, with their beta
headers: `web_search_20260318`, `web_fetch_20260318` / `20260309`,
`code_execution_20260521` / `20260120` / `20250825`, `computer_20251124`,
plus `advisor`, `toolSearchBm25` / `toolSearchRegex`, and `mcpToolset` for
the MCP connector. The three current code-execution versions are GA and need
no beta header; `code_execution_20250522` is the legacy Python-only build and
is no longer the default.
## Tool definition properties [#tool-definition-properties]
Anthropic accepts optional properties on *any* tool — built-in or your own.
`Tool.loading(_:)` carries them:
```swift
let deleteFile = Tool(name: "deleteFile", description: …, parameters: …) { … }
.loading(ToolLoading(
strict: true, // validate names and inputs
deferLoading: true, // keep out of the cached prompt
allowedCallers: ["code_execution_20260120"], // sandbox-only
eagerInputStreaming: true // fine-grained input streaming
))
```
`.ephemeralCache()` sets a `cache_control` breakpoint at the tool definition,
and `.codeExecutionOnly()` is shorthand for the sandbox-only caller list. It
defaults to `code_execution_20260120`, the caller Anthropic documents for
programmatic tool calling — the same value `web_search_20260209` and later
default to. That is deliberately *not* the version `Tools.codeExecution()`
sends by default (`code_execution_20260521`): if you want the model to call
your tool from inside the sandbox, declare
`Tools.codeExecution(version: "code_execution_20260120")` so the caller
matches. Pass `codeExecutionOnly(version:)` to pin a different one.
`defer_loading` keeps a tool out of the cached system-prompt prefix until
tool search surfaces it, so adding deferred tools does not invalidate an
existing prompt cache.
A tool's `inputExamples` are sent natively as `input_examples` here, so the
description stays clean — the
[`addToolInputExamples` middleware](/docs/middleware) is only needed for
providers with no native field.
## Platform APIs [#platform-apis]
* `AnthropicBatchClient` — Message Batches: `create`, `list`, `get`,
`results` (streamed JSONL, decoded per line), `cancel`, `delete`.
* `AnthropicModelsClient` — `list` and `get` for the model catalog.
* `AnthropicModel.countTokens(_:tools:system:)` — pre-flight sizing against
`/v1/messages/count_tokens`, with the same beta headers your tools require.
## Models [#models]
| Models | Output ceiling | `reasoning` becomes |
| ------------------------------------------- | -------------- | --------------------------------------------------------- |
| Sonnet 5, Fable 5, Opus 4.7 / 4.8 | 128k | Adaptive thinking + `output_config.effort`, up to `xhigh` |
| Sonnet 4.6, Opus 4.6 | 128k | Adaptive thinking (`xhigh` coerces to `max`) |
| Sonnet 4.5, Opus 4.5, Haiku 4.5, Sonnet 4.x | 64k | `thinking.budget_tokens` from the ceiling |
| Opus 4.1, Opus 4.x | 32k | `thinking.budget_tokens` from the ceiling |
| Anything else | 4,096 | `thinking.budget_tokens`, conservative ceiling |
`.none` sends `thinking.type: "disabled"` explicitly, and `max_tokens`
is raised automatically when a budget would not fit.
### Current lineup [#current-lineup]
Snapshot from July 2026, newest first. Anthropic's native ids use
dashes (`claude-opus-4-8`); date-stamped variants
(`claude-sonnet-4-5-20250929`) work too.
| Model | Released |
| -------------------------------------- | -------------- |
| `claude-fable-5` | July 2026 |
| `claude-sonnet-5` | June 2026 |
| `claude-opus-4-8` | May 2026 |
| `claude-opus-4-7` | April 2026 |
| `claude-sonnet-4-6`, `claude-opus-4-6` | February 2026 |
| `claude-opus-4-5` | November 2025 |
| `claude-haiku-4-5` | October 2025 |
| `claude-sonnet-4-5` | September 2025 |
| `claude-opus-4-1`, `claude-sonnet-4` | 2025 |
| `claude-3-5-haiku` | Budget tier |
## Files and skills [#files-and-skills]
These resource clients are documented in [Files and skills](/docs/files-and-skills).
```swift
let upload = try await AnthropicFiles().upload(
data: pdf, filename: "report.pdf", mediaType: "application/pdf"
)
let skill = try await AnthropicSkills().upload(
files: [SkillFile(path: "brand-guide/SKILL.md", data: skillMD)],
displayTitle: "Brand guide"
)
```
Beta headers are handled for you on both.
# AssemblyAI (/docs/providers/assemblyai)
`AssemblyAITranscriptionModel` conforms to `TranscriptionModel` and works with
[`transcribe`](/docs/transcription).
```swift
let result = try await transcribe(
model: AssemblyAITranscriptionModel(), // universal-3-5-pro
audio: audioData,
mediaType: "audio/mpeg"
)
print(result.text)
```
The model reads `ASSEMBLYAI_API_KEY`, defaults to `universal-3-5-pro`, and
uses `https://api.assemblyai.com`. It uploads the audio, creates a
transcript, and polls until completion internally.
Use `pollInterval` and `pollTimeout` on the model initializer to tune polling.
AssemblyAI-specific transcript settings can be passed through
`providerOptions`.
## Models [#models]
| Model ID | Use |
| ------------------- | ---------------------------------------- |
| `universal-3-5-pro` | Current highest-accuracy Universal model |
| `universal-2` | Previous Universal generation |
Availability can vary by API mode and language. See AssemblyAI's
[Universal-3.5 Pro guide](https://www.assemblyai.com/docs/getting-started/universal-3-5-pro.md)
and the [`speech_model` API field](https://www.assemblyai.com/docs/api-reference/transcripts/submit.md).
# Azure OpenAI (/docs/providers/azure)
```swift
let azure = AzureOpenAIProvider(
resourceName: "my-resource", // or AZURE_RESOURCE_NAME
apiVersion: "v1"
)
let model = azure("my-gpt5-deployment")
```
Key from `AZURE_API_KEY` (sent as the `api-key` header). The deployment
name doubles as the `model` body field, exactly like the AI SDK.
## Features [#features]
Everything the OpenAI chat wire supports — tools, structured output,
vision, reasoning models, cached-token usage — routed at your Azure
resource.
* `apiVersion:` sets the `api-version` query parameter (default `"v1"`).
* `useDeploymentBasedUrls: true` switches to the legacy
`/deployments/{deployment}` path shape for older resources.
* `baseURL:` overrides the host entirely for gateways in front of Azure.
## Models [#models]
Azure model ids are your deployment names — deploy any current OpenAI
model (the `gpt-5.6` family, `gpt-5.5`, `gpt-image-2`, embeddings) in
the Azure portal and address it by whatever you named the deployment.
The [OpenAI page](/docs/providers/openai) has the current lineup.
## Embeddings [#embeddings]
The provider vends embedding deployments the same way:
```swift
let embeddings = azure.textEmbeddingModel("my-embedding-deployment")
let result = try await embed(model: embeddings, value: "hello")
```
# Baseten (/docs/providers/baseten)
`BasetenModel` reads `BASETEN_API_KEY` and targets
`https://inference.baseten.co/v1`.
```swift
let model = BasetenModel("deepseek-ai/DeepSeek-V4-Pro")
let result = try await generateText(model: model, prompt: "Say hello.")
print(result.text)
```
## Models [#models]
Current Baseten Model APIs include:
| Model ID |
| ----------------------------- |
| `deepseek-ai/DeepSeek-V4-Pro` |
| `zai-org/GLM-5.2` |
| `openai/gpt-oss-120b` |
| `moonshotai/Kimi-K2.7-Code` |
Baseten also accepts IDs for your own deployments. Query the authenticated
[`GET /v1/models`](https://docs.baseten.co/inference/model-apis/overview)
endpoint for the models available to your account. The Swift model also
accepts `apiKey:`, `baseURL:`, `headers:`, and `urlSession:` overrides.
# Amazon Bedrock (/docs/providers/bedrock)
Bedrock has two inference endpoints, and the library covers both.
`BedrockMantleProvider` targets **`bedrock-mantle`**, the endpoint AWS
recommends for new applications: OpenAI-compatible Responses and Chat
Completions plus the Anthropic Messages API, all behind a Bedrock API key.
`BedrockModel` targets **`bedrock-runtime`**, the Converse API, which is
where guardrails, cross-region inference profiles, and SigV4 signing live.
## bedrock-mantle [#bedrock-mantle]
```swift
let bedrock = BedrockMantleProvider(region: "us-east-1")
let responses = bedrock("openai.gpt-oss-120b")
let chat = bedrock.chat("deepseek.v3-2")
let messages = bedrock.messages("anthropic.claude-sonnet-4-6-v1")
```
`callAsFunction` (and `languageModel(_:)`) returns the Responses-API model,
which is the endpoint's recommended surface. All three factories report
`provider == "bedrock"` for telemetry.
The key comes from `AWS_BEARER_TOKEN_BEDROCK` or an explicit `apiKey:`. The
OpenAI-compatible surfaces send it as `Authorization: Bearer`, the Messages
surface as `x-api-key` with `anthropic-version: 2023-06-01` — the same
headers AWS documents. Base URLs derive from the region
(`https://bedrock-mantle.{region}.api.aws`, then `/v1` or `/anthropic/v1`)
unless you pass `baseURL:`; a `baseURL` that already ends in `/v1` or
`/anthropic/v1` is normalized, so a gateway origin works either way.
Mantle model ids carry no region prefix — there are no cross-region
inference profiles on this endpoint. Use `anthropic.claude-sonnet-4-6-v1`,
not `us.anthropic.claude-sonnet-4-6-v1`.
| Capability | Notes |
| -------------------------------- | ----------------------------------------------------------------------------------------------- |
| Auth | Bedrock API key only; SigV4 requests go through `BedrockModel` |
| Stateful turns | `store` / `previous_response_id` via `providerOptions` on the Responses model |
| Structured output | Forced-tool JSON mode, which Mantle accepts; `output_config.format` is rejected by the endpoint |
| Guardrails, prompt routing, CRIS | `bedrock-runtime` only |
Responses-API reasoning effort rides `providerOptions` here, because the
unified `reasoning:` knob keys off OpenAI's own model-id shapes:
```swift
let result = try await generateText(
model: bedrock("openai.gpt-oss-120b"),
messages: [.user("Plan the migration.")],
providerOptions: ["reasoning": ["effort": "high"], "store": false]
)
```
`bedrock.chat(...)` and `bedrock.messages(...)` take the unified
`reasoning:` parameter directly — chat completions map it to
`reasoning_effort`, and Claude ids map to the same thinking tiers as the
[Anthropic pack](/docs/providers/anthropic).
## bedrock-runtime [#bedrock-runtime]
```swift
let model = BedrockModel(
"anthropic.claude-sonnet-4-5-20250929-v1:0",
region: "us-east-1"
)
```
Two auth modes. The simplest is Bedrock's API-key auth via
`AWS_BEARER_TOKEN_BEDROCK`. For IAM credentials, pass `accessKeyID:` /
`secretAccessKey:` (and `sessionToken:` for temporary creds), or set
`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_SESSION_TOKEN` — the
request is then **SigV4-signed** for the region. When both are present,
SigV4 wins. The base URL derives from the region
(`https://bedrock-runtime.{region}.amazonaws.com`) unless you pass one.
```swift
let model = BedrockModel(
"anthropic.claude-sonnet-4-5-20250929-v1:0",
region: "us-east-1",
accessKeyID: "AKIA…",
secretAccessKey: "…"
)
```
### Features [#features]
* Tools, structured output, and vision over the Converse API.
* Responses stream in AWS's binary event-stream framing; the library
decodes it natively.
* `usage.cachedInputTokens` is populated where the model reports cache
reads; guardrail `trace` and `cacheWriteInputTokens` arrive on
`result.providerMetadata["bedrock"]`.
### Models [#models]
The `reasoning` parameter translates by model-id family:
| Family | Translation |
| --------------- | ---------------------------------------------------------------------------------------------------------- |
| `anthropic.*` | Claude thinking config — the same adaptive/budget tiers as the [Anthropic pack](/docs/providers/anthropic) |
| `openai.*` | `reasoning_effort` |
| everything else | Generic `reasoningConfig.maxReasoningEffort` (`xhigh` becomes `max`) |
Model ids are Bedrock's prefixed forms, e.g.
`anthropic.claude-sonnet-4-5-20250929-v1:0` or an inference profile like
`us.anthropic.claude-sonnet-4-5-20250929-v1:0`. Amazon's own Nova line
(as of July 2026): `nova-2-lite` (December 2025), plus `nova-pro`,
`nova-lite`, and `nova-micro`.
# Black Forest Labs (/docs/providers/black-forest-labs)
`BlackForestLabsImageModel` conforms to `ImageModel` and works with
[`generateImage`](/docs/image-generation). It reads `BFL_API_KEY`
(sent as the `x-key` header) and uses `https://api.bfl.ai/v1`.
```swift
let result = try await generateImage(
model: BlackForestLabsImageModel("flux-pro-1.1"),
prompt: "A tiny observatory on a snowy mountain",
size: "1024x768"
)
```
The model submits the job, polls until the render is `Ready`, then downloads
the result. `size` maps to `width`/`height`, `aspectRatio` to `aspect_ratio`,
and extra fields go under the `black-forest-labs` provider key. Tune polling
with `pollInterval:` and `pollTimeout:`.
## Models [#models]
As of July 2026:
| Model ID | Notes |
| --------------------------------------- | ------------------ |
| `flux-pro-1.1` | General generation |
| `flux-pro-1.1-ultra` | High resolution |
| `flux-pro-1.0-fill` | Inpainting / fill |
| `flux-kontext-pro` / `flux-kontext-max` | Context editing |
See the [BFL API docs](https://docs.bfl.ai/) for endpoint-specific fields.
# ByteDance (/docs/providers/bytedance)
Both models read `ARK_API_KEY` and target
`https://ark.ap-southeast.bytepluses.com/api/v3`.
## Images [#images]
`ByteDanceImageModel` conforms to `ImageModel` and works with
[`generateImage`](/docs/image-generation):
```swift
let result = try await generateImage(
model: ByteDanceImageModel("seedream-4-0-250828"),
prompt: "A koi pond at dusk",
size: "2048x2048"
)
```
## Video [#video]
`ByteDanceVideoModel` conforms to `VideoModel` and works with
[`generateVideos`](/docs/video-generation). It submits a task and polls until
the clip is ready; pass an image for image-to-video:
```swift
let video = try await generateVideos(
model: ByteDanceVideoModel("seedance-1-0-pro-250528"),
prompt: "The koi swim in a slow circle"
)
```
## Models [#models]
As of July 2026:
| Model ID | Kind |
| -------------------------------------------------- | -------------------- |
| `seedream-5-0-260128` / `seedream-5-0-lite-260128` | Image |
| `seedream-4-5-251128` / `seedream-4-0-250828` | Image |
| `seedance-1-5-pro-251215` | Video |
| `seedance-1-0-pro-250528` | Video |
| `seedance-1-0-lite-t2v-250428` / `-i2v-250428` | Video (text / image) |
Model ids are date-stamped snapshots, so check the
[Ark console](https://console.bytepluses.com/ark) for the current ids.
# Cartesia (/docs/providers/cartesia)
Both models read `CARTESIA_API_KEY`, target `https://api.cartesia.ai`, and
send the `Cartesia-Version` header automatically.
## Speech [#speech]
`CartesiaSpeechModel` conforms to `SpeechModel` and works with
[`generateSpeech`](/docs/speech-generation). Cartesia needs a voice id, so pass
one as `voice:` on the request (or set a default on the model):
```swift
let result = try await generateSpeech(
model: CartesiaSpeechModel("sonic-2"),
text: "Hello from Swift.",
voice: "a0e99841-438c-4a64-b679-ae501e7d6091",
outputFormat: "wav"
)
```
`outputFormat` maps to the Cartesia container: `mp3` (default), `wav`, or
`raw`/`pcm`. Set `sampleRate:` on the initializer to change the rate.
## Transcription [#transcription]
`CartesiaTranscriptionModel` conforms to `TranscriptionModel` and works with
[`transcribe`](/docs/transcription):
```swift
let text = try await transcribe(
model: CartesiaTranscriptionModel("ink-whisper"),
audio: audioData,
mediaType: "audio/mpeg"
)
```
## Models [#models]
As of July 2026:
| Model ID | Kind |
| ------------------------- | ------------- |
| `sonic-3` / `sonic-3.5` | Newest speech |
| `sonic-2` / `sonic-turbo` | Speech |
| `ink-2` | Speech |
| `ink-whisper` | Transcription |
Browse voice ids in the [Cartesia voice library](https://play.cartesia.ai/).
# Cerebras (/docs/providers/cerebras)
`CerebrasModel` reads `CEREBRAS_API_KEY` and targets
`https://api.cerebras.ai/v1`.
```swift
let model = CerebrasModel("gpt-oss-120b")
let result = try await generateText(model: model, prompt: "Say hello.")
print(result.text)
```
The initializer accepts optional `apiKey:`, `baseURL:`, `headers:`, and
`urlSession:` overrides. Tools, structured output, vision, and reasoning
are encoded when requested and depend on the selected model.
## Models [#models]
| Model ID | Status |
| -------------- | ---------- |
| `gpt-oss-120b` | Production |
| `gemma-4-31b` | Preview |
| `zai-glm-4.7` | Preview |
Preview availability can change. Check Cerebras's [model overview](https://inference-docs.cerebras.ai/models/overview)
or query the authenticated `GET https://api.cerebras.ai/v1/models` endpoint.
# Cohere (/docs/providers/cohere)
```swift
let model = CohereModel("command-a")
```
Key from `COHERE_API_KEY`; base URL `https://api.cohere.com/v2`.
## Features [#features]
* Tools, structured output, and vision on the native v2 chat wire.
* Citations surface as `StreamPart.source`.
* `command-a-reasoning-08-2025` is a hybrid reasoning model; thinking streams
as `.reasoningDelta`. The `reasoning` parameter maps to Cohere's `thinking`
field on reasoning models (ids containing `reasoning`) and is ignored on the
others: `.none` disables thinking, an effort level enables it with a
`token_budget` scaled from `maxOutputTokens` (capped at 31k), and
`.providerDefault` leaves the model's default (thinking on) untouched.
## Embeddings and reranking [#embeddings-and-reranking]
Both are independent model protocols: `CohereEmbeddingModel` works with
[`embed` and `embedMany`](/docs/embeddings), while `CohereRerankingModel`
works with [`rerank`](/docs/reranking).
```swift
let embeddings = try await embedMany(
model: CohereEmbeddingModel("embed-v4.0"),
values: documents
)
let ranked = try await rerank(
model: CohereRerankingModel("rerank-v4-fast"),
query: "How do I cancel my subscription?",
documents: documents,
topN: 3
)
ranked.rankedDocuments.first?.document // best match
ranked.rankedDocuments.first?.relevanceScore
```
## Models [#models]
As of July 2026:
| Model | Notes |
| -------------------------------------------------- | ------------------------------------------------------- |
| `command-a` / `command-a-03-2025` | General chat |
| `command-a-plus` / `command-a-plus-05-2026` | Multimodal MoE flagship (vision + text) |
| `command-a-reasoning-08-2025` | Reasoning; thinking maps from `reasoning` (above) |
| `command-a-vision-07-2025` | Vision — up to 20 images, 128K context; no tool calling |
| `command-a-translate-08-2025` | Translation across 23 languages |
| `north-mini-code-1.0` | Code |
| `embed-v4.0` | Embeddings (`CohereEmbeddingModel`) |
| `rerank-v4-pro` / `rerank-v4-fast` / `rerank-v3.5` | Rerank (`CohereRerankingModel`) |
The chat, reasoning, vision, and translation models share the v2 chat wire, so
any of them drops into `CohereModel("…")`.
# Compatibility (/docs/providers/compatibility)
One table to answer "can I do X on Y", from what each pack actually
implements.
## Language models [#language-models]
| Provider | Tools | Structured output | Reasoning | Vision | Sources | Cached tokens |
| ------------------ | :---: | :---------------: | :-------: | :----: | :-----: | :-----------: |
| OpenAI (Responses) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| OpenAI (chat) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Azure OpenAI | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Anthropic | ✓ | ✓ | ✓ | ✓ | — | ✓ |
| Google / Vertex | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Bedrock | ✓ | ✓ | ✓ | ✓ | — | ✓ |
| xAI | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Meta | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Groq | ✓\* | ✓\* | ✓ | ✓\* | — | ✓ |
| DeepSeek | ✓\* | ✓\* | ✓ | ✓\* | — | ✓ |
| Mistral | ✓\* | ✓\* | ✓ | ✓\* | — | — |
| Perplexity | — | ✓\* | — | ✓\* | ✓ | — |
| Cohere | ✓ | ✓ | ✓ | ✓ | ✓ | — |
| Foundation Models | ✓ | ✓ | — | — | — | — |
| OpenAI-compatible | ✓\* | ✓\* | ✓\* | ✓\* | — | — |
\* Rides the shared chat-completions wire; whether a given model honors
it is up to the model. Perplexity has no tool calling upstream, so the
library doesn't pretend otherwise.
**Column notes.** *Structured output* uses each provider's best native
mechanism: JSON schema mode on the OpenAI wire, constrained decoding on
Gemini, a forced tool call on Anthropic, guided generation on-device.
*Sources* means citations surface as `StreamPart.source` (OpenAI url
annotations, xAI live search, Google Search grounding chunks, Perplexity
and Cohere citations).
*Cached tokens* means `usage.cachedInputTokens` is populated on
prompt-cache hits.
## Models the library special-cases [#models-the-library-special-cases]
Model ids are free strings — anything the provider serves works. These
are the ids the library treats specially, straight from the source:
### OpenAI [#openai]
* The reasoning ruleset applies to `o1*`, `o3*`, `o4-mini*`, and `gpt-5*`
except `gpt-5-chat*`.
* `gpt-5.1` through `gpt-5.6` accept `temperature`/`topP` again when
reasoning effort is `none`; other reasoning models drop them.
### Anthropic [#anthropic]
| Models | Output ceiling | Reasoning translation |
| ------------------------------------------- | -------------- | -------------------------------------------- |
| Sonnet 5, Fable 5, Opus 4.7 / 4.8 | 128k | Adaptive thinking, effort up to `xhigh` |
| Sonnet 4.6, Opus 4.6 | 128k | Adaptive thinking (`xhigh` coerces to `max`) |
| Sonnet 4.5, Opus 4.5, Haiku 4.5, Sonnet 4.x | 64k | `budget_tokens` from the ceiling |
| Opus 4.1, Opus 4.x | 32k | `budget_tokens` from the ceiling |
| Anything else | 4,096 | `budget_tokens`, conservative ceiling |
The same table drives Bedrock's `anthropic.*` model ids.
### Google [#google]
* `gemini-3*` takes `thinkingLevel` — except `gemini-3-pro-image`, which
stays on budgets.
* Budget models cap at 32,768 thinking tokens for 2.5 Pro and
`gemini-3-pro-image`, 24,576 for everything else.
### xAI [#xai]
* `grok-4.20` date-stamped `-reasoning` / `-non-reasoning` variants have
their behavior baked in, so the `reasoning` parameter is not sent.
### Meta [#meta]
* Muse Spark cannot turn reasoning off. `.none` is dropped rather than
sent, since the API answers it with a 400.
* `temperature` and `topP` are forwarded even though the model reasons —
Meta accepts both, unlike OpenAI's reasoning models.
### Mistral [#mistral]
* `reasoning` maps to `reasoning_effort` only on `mistral-small-latest`,
`mistral-small-2603`, `mistral-medium-3`, and `mistral-medium-3.5`.
### Bedrock [#bedrock]
On `bedrock-runtime` (`BedrockModel`), three families by model-id prefix:
`anthropic.*` gets Claude thinking config, `openai.*` gets
`reasoning_effort`, and everything else gets the generic `reasoningConfig`
(where `xhigh` becomes `max`).
On `bedrock-mantle` (`BedrockMantleProvider`), each surface follows its own
wire: `messages(_:)` maps `reasoning` to Claude thinking, `chat(_:)` maps it
to `reasoning_effort`, and the Responses surface takes effort through
`providerOptions["reasoning"]["effort"]`.
## Beyond text [#beyond-text]
| Capability | Providers |
| -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| [Embeddings](/docs/embeddings) | OpenAI, Azure OpenAI, Cohere, Voyage, Alibaba, any OpenAI-compatible endpoint |
| [Reranking](/docs/reranking) | Cohere, Voyage |
| [Image generation](/docs/image-generation) | OpenAI (`gpt-image-2`, edits included), xAI, fal, Luma, Replicate, Black Forest Labs, ByteDance, Prodia, QuiverAI (SVG) |
| [Speech generation](/docs/speech-generation) | OpenAI, ElevenLabs, LMNT, Hume, Deepgram, Sarvam, xAI, Cartesia |
| [Transcription](/docs/transcription) | OpenAI, ElevenLabs, Deepgram, AssemblyAI, Rev.ai, Gladia, Sarvam, xAI, Cartesia |
| [Video generation](/docs/video-generation) | xAI (generations, edits, extensions), Luma, ByteDance, Kling, Alibaba (Wan) |
| [Realtime voice](/docs/realtime) | OpenAI, xAI, Google (Gemini Live) |
| [Files and skills](/docs/files-and-skills) | OpenAI files; Anthropic files and skills |
The async transcription providers (AssemblyAI, Rev.ai, Gladia) submit,
poll, and fetch internally — one call either way.
## First-class models on compatible endpoints [#first-class-models-on-compatible-endpoints]
Named services get dedicated model types with the AI SDK's exact base URLs
and environment variables:
| Model | Endpoint | Key from |
| ------------------ | ------------------------------------------------ | --------------------- |
| `TogetherAIModel` | `api.together.xyz/v1` | `TOGETHER_API_KEY` |
| `FireworksModel` | `api.fireworks.ai/inference/v1` | `FIREWORKS_API_KEY` |
| `CerebrasModel` | `api.cerebras.ai/v1` | `CEREBRAS_API_KEY` |
| `OpenRouterModel` | `openrouter.ai/api/v1` | `OPENROUTER_API_KEY` |
| `DeepInfraModel` | `api.deepinfra.com/v1/openai` | `DEEPINFRA_API_KEY` |
| `BasetenModel` | `inference.baseten.co/v1` | `BASETEN_API_KEY` |
| `VercelModel` | `api.v0.dev/v1` | `VERCEL_API_KEY` |
| `AIGatewayModel` | `ai-gateway.vercel.sh/v1` | `AI_GATEWAY_API_KEY` |
| `OllamaModel` | `localhost:11434/v1` | no key |
| `LMStudioModel` | `localhost:1234/v1` | no key |
| `SarvamModel` | `api.sarvam.ai/v1` | `SARVAM_API_KEY` |
| `MoonshotModel` | `api.moonshot.ai/v1` | `MOONSHOT_API_KEY` |
| `AlibabaModel` | `dashscope-intl.aliyuncs.com/compatible-mode/v1` | `ALIBABA_API_KEY` |
| `HuggingFaceModel` | `router.huggingface.co/v1` | `HUGGINGFACE_API_KEY` |
`OpenAICompatibleProvider` remains the general initializer for any other
chat-completions endpoint. Replicate has a first-class
[`ReplicateImageModel`](/docs/providers/replicate), but no language-model pack
because its prediction API is not the chat-completions wire.
# Deepgram (/docs/providers/deepgram)
`DEEPGRAM_API_KEY` powers both Deepgram model types.
## Speech [#speech]
```swift
let result = try await generateSpeech(
model: DeepgramSpeechModel(), // aura-2-thalia-en
text: "The next train arrives in five minutes.",
outputFormat: "mp3"
)
```
The model ID selects the Aura voice. See
[speech generation](/docs/speech-generation) for the shared API.
## Transcription [#transcription]
```swift
let result = try await transcribe(
model: DeepgramTranscriptionModel(), // nova-3
audio: audioData,
mediaType: "audio/wav"
)
```
`DeepgramTranscriptionModel` defaults to `nova-3` and returns word-level
segments when Deepgram supplies them. It uploads the audio synchronously to
the listening endpoint. See [transcription](/docs/transcription) for the common
result shape.
Both models use `https://api.deepgram.com` by default.
## Models [#models]
### Speech models [#speech-models]
Aura 2 model IDs include both the voice and language, using the form
`aura-2-{voice}-{language}`. The SDK defaults to `aura-2-thalia-en`; browse
Deepgram's [Aura model and voice list](https://developers.deepgram.com/docs/tts-models)
for the other current IDs.
### Transcription models [#transcription-models]
| Model ID | Use |
| --------- | ------------------------------------- |
| `nova-3` | Current general prerecorded STT model |
| `nova-2` | Previous Nova generation |
| `whisper` | Hosted Whisper option |
Deepgram also offers models specialized for streaming voice agents. See the
official [speech-to-text model overview](https://developers.deepgram.com/docs/model)
for mode and language compatibility.
# DeepInfra (/docs/providers/deepinfra)
`DeepInfraModel` reads `DEEPINFRA_API_KEY` and targets
`https://api.deepinfra.com/v1/openai`.
```swift
let model = DeepInfraModel("deepseek-ai/DeepSeek-V4-Pro")
let result = try await generateText(model: model, prompt: "Say hello.")
print(result.text)
```
The `/openai` segment is part of the default base URL. Override `apiKey:`,
`baseURL:`, `headers:`, or `urlSession:` directly on the model when needed.
## Models [#models]
DeepInfra's live catalog currently includes models such as:
| Model ID |
| ----------------------------- |
| `deepseek-ai/DeepSeek-V4-Pro` |
| `stepfun-ai/Step-3.7-Flash` |
| `MiniMaxAI/MiniMax-M2.7` |
| `Qwen/Qwen3.5-9B` |
Use DeepInfra's [model catalog](https://docs.deepinfra.com/models) or
[`GET /models/list`](https://api.deepinfra.com/models/list) for the complete,
current list.
# DeepSeek (/docs/providers/deepseek)
```swift
let model = DeepSeekModel("deepseek-chat")
```
Key from `DEEPSEEK_API_KEY`; base URL `https://api.deepseek.com`.
## Features [#features]
* Rides the shared chat-completions wire: tools, structured output, and
vision are encoded for any model that honors them.
* Reasoner models stream `reasoning_content` deltas, surfaced as
`.reasoningDelta` — no tag parsing needed.
* The `reasoning` parameter maps to `thinking.type` plus
`reasoning_effort`, where `.xhigh` becomes `max`.
* Prompt-cache hits (`prompt_cache_hit_tokens`) surface as
`usage.cachedInputTokens`.
## Models [#models]
DeepSeek's API serves two aliases — `deepseek-chat` and
`deepseek-reasoner` — that always point at the current generation. The
versions behind them, as of July 2026:
| Version | Released |
| ----------------------------------------- | ------------- |
| `deepseek-v4-pro`, `deepseek-v4-flash` | April 2026 |
| `deepseek-v3.2`, `deepseek-v3.2-thinking` | December 2025 |
| `deepseek-v3.1`, `deepseek-v3.1-terminus` | 2025 |
| `deepseek-r1`, `deepseek-v3` | The originals |
# ElevenLabs (/docs/providers/elevenlabs)
`ELEVENLABS_API_KEY` powers both ElevenLabs model types.
## Speech [#speech]
```swift
let result = try await generateSpeech(
model: ElevenLabsSpeechModel("eleven_multilingual_v2"),
text: "Welcome to the show.",
voice: voiceID,
outputFormat: "mp3"
)
```
Pass the ElevenLabs model ID to `ElevenLabsSpeechModel` and the voice ID to
`voice:`. The result contains the audio bytes and media type. See
[speech generation](/docs/speech-generation) for all shared options.
## Transcription [#transcription]
```swift
let result = try await transcribe(
model: ElevenLabsTranscriptionModel(), // scribe_v2
audio: audioData,
mediaType: "audio/mpeg"
)
print(result.text)
```
`ElevenLabsTranscriptionModel` defaults to `scribe_v2` and uses a synchronous
multipart upload. See [transcription](/docs/transcription) for the common result
shape.
## Models [#models]
### Speech models [#speech-models]
| Model ID | Use |
| ------------------------ | --------------------------------- |
| `eleven_v3` | Most expressive speech generation |
| `eleven_multilingual_v2` | Stable multilingual speech |
| `eleven_flash_v2_5` | Low-latency multilingual speech |
### Transcription models [#transcription-models]
| Model ID | Use |
| -------------------- | --------------------------------------------------------- |
| `scribe_v2` | Current prerecorded speech-to-text model |
| `scribe_v2_realtime` | Realtime transcription; not used by this prerecorded pack |
See ElevenLabs's live [model overview](https://elevenlabs.io/docs/overview/models)
for languages, latency, and availability.
# fal (/docs/providers/fal)
`FalImageModel` conforms to `ImageModel`, so it works with
[`generateImage`](/docs/image-generation).
```swift
let result = try await generateImage(
model: FalImageModel("fal-ai/bytedance/seedream/v4.5/text-to-image"),
prompt: "A tiny observatory on a snowy mountain",
aspectRatio: "16:9"
)
```
The model reads `FAL_API_KEY`, falling back to `FAL_KEY`, and uses
`https://fal.run` by default. Pass any fal image model ID as the first
initializer argument.
`n`, `size`, `aspectRatio`, and `seed` map to fal request fields. Additional
fal fields go under the `fal` provider key:
```swift
providerOptions: ["fal": ["num_inference_steps": 4]]
```
## Models [#models]
Current image examples include:
| Model ID | Use |
| ---------------------------------------------- | -------------------------------- |
| `fal-ai/bytedance/seedream/v4.5/text-to-image` | Current general image generation |
| `fal-ai/hunyuan-image/v3/text-to-image` | Hunyuan Image 3 |
| `fal-ai/flux/schnell` | Fast FLUX generation |
fal's catalog is dynamic. Copy the endpoint ID from the live
[fal model gallery](https://fal.ai/models); `FalImageModel` accepts any image
endpoint with the standard fal response shape.
# Fireworks (/docs/providers/fireworks)
`FireworksModel` reads `FIREWORKS_API_KEY` and targets
`https://api.fireworks.ai/inference/v1`.
```swift
let model = FireworksModel("accounts/fireworks/models/glm-5p2")
let result = streamText(model: model, prompt: "Say hello.")
for try await text in result.textStream {
print(text, terminator: "")
}
```
Fireworks accepts `low`, `medium`, and `high` reasoning effort, so
`.minimal` becomes `low` and `.xhigh` becomes `high`. Other capabilities
depend on the selected model.
## Models [#models]
Current serverless examples include:
| Model ID |
| ------------------------------------------- |
| `accounts/fireworks/models/glm-5p2` |
| `accounts/fireworks/models/qwen3p7-max` |
| `accounts/fireworks/models/deepseek-v4-pro` |
| `accounts/fireworks/routers/kimi-k2p6-fast` |
The catalog rotates, and custom deployments use account-qualified IDs. Browse
the live [Fireworks model library](https://fireworks.ai/models) and
[serverless serving-path documentation](https://docs.fireworks.ai/serverless/serving-paths)
before pinning a model in production.
# Gladia (/docs/providers/gladia)
`GladiaTranscriptionModel` conforms to `TranscriptionModel` and works with
[`transcribe`](/docs/transcription).
```swift
let result = try await transcribe(
model: GladiaTranscriptionModel("solaria-3"),
audio: audioData,
mediaType: "audio/wav",
providerOptions: ["detect_language": true]
)
```
The model reads `GLADIA_API_KEY`, uses `https://api.gladia.io`, and defaults
to `solaria-1`. It uploads the audio, starts a prerecorded job, and polls the
returned result URL internally.
Use `pollInterval` and `pollTimeout` on the initializer to tune polling.
Completed utterances become `result.segments` when Gladia returns them.
## Models [#models]
| Model ID | Use |
| ----------- | -------------------------------------- |
| `solaria-3` | Latest high-accuracy prerecorded model |
| `solaria-1` | Generalist model and SDK default |
`solaria-3` is currently limited to asynchronous prerecorded transcription,
which matches this provider pack. See Gladia's
[prerecorded STT guide](https://docs.gladia.io/chapters/pre-recorded-stt/getting-started)
for current availability and language guidance.
# Google (/docs/providers/google)
```swift
let model = GoogleModel("gemini-3.6-flash")
```
Key from `GOOGLE_GENERATIVE_AI_API_KEY`; base URL
`https://generativelanguage.googleapis.com/v1beta`.
## Two chat surfaces [#two-chat-surfaces]
Google now has two ways to talk to Gemini, and both are here:
| Surface | Model type | Use it for |
| ------------------------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------- |
| `generateContent` / `streamGenerateContent` | `GoogleModel` | The wire everything already speaks. Google calls it legacy but keeps it fully supported |
| **Interactions API** | `GoogleInteractionsModel` | Where Google ships new features: server-side conversation state, background execution, and agents |
```swift
let interactions = GoogleInteractionsModel("gemini-3.6-flash")
let result = try await generateText(model: interactions, prompt: "Explain how AI works")
```
Both conform to `LanguageModel`, so tools, structured output, reasoning, and
the whole `generateText` / `streamText` / `Agent` surface work either way.
### Server-side state, background runs, and agents [#server-side-state-background-runs-and-agents]
```swift
var chat = GoogleInteractionsModel("gemini-3.6-flash", store: true)
let first = try await generateText(model: chat, prompt: "I have 2 dogs.")
chat.previousInteractionID = first.providerMetadata?["google"]?["interactionId"]?.stringValue
let second = try await generateText(model: chat, prompt: "How many paws?")
```
`previousInteractionID` replays the stored history server-side instead of
resending it, which also raises cache hit rates. **`store` defaults to
`false` here** even though Google's API defaults it to `true` — opting into
55-day (paid) or 1-day (free) retention should be a decision you make, not
one you inherit. `background: true` runs long tasks server-side, and
`cancel(_:)`, `retrieve(_:)`, and `delete(_:)` manage a stored interaction.
Agents use the same endpoint with `agent:` instead of `model:`:
```swift
let research = GoogleInteractionsModel.agent("deep-research-preview-04-2026")
let report = try await generateText(model: research, prompt: "Compare Swift 6 concurrency proposals")
```
`agent_config` replaces `generation_config` for agent runs (Antigravity and
CodeMender take their own settings), so pass it through
`GoogleInteractionsModel.agent(agentConfig:)`. Reasoning maps to
`thinking_level` (`minimal` / `low` / `medium` / `high`); thought steps arrive
as `.reasoningDelta` and the interaction id lands on
`result.providerMetadata["google"]["interactionId"]`.
## Features [#features]
* Tools, vision, and file parts on the native `generateContent` wire.
* Structured output uses constrained decoding
(`responseSchema` + `responseMimeType`), not prompt tricks.
* The `reasoning` parameter maps to `thinkingConfig` (below); thinking
streams as `.reasoningDelta`.
* `usage.cachedInputTokens` reports context-cache hits.
* Full `groundingMetadata`, `safetyRatings`, and `urlContextMetadata`
arrive on `result.providerMetadata["google"]` (grounding chunks also
surface as `.source`).
* Native fields (`safetySettings`, `cachedContent`, ...) pass through
`providerOptions` at the top level.
* Grounding tools have typed builders under `GoogleModel.Tools`
— `googleSearch`, `urlContext`, `codeExecution`, `fileSearch`,
`enterpriseWebSearch`, `googleMaps` (Gemini 2.0 and newer). Pass them
in `tools:` and they run server-side. Grounding chunks surface as
`StreamPart.source`:
```swift
let result = try await generateText(
model: GoogleModel("gemini-3.6-flash"),
prompt: "Ground this in current sources.",
tools: [GoogleModel.Tools.googleSearch(), GoogleModel.Tools.urlContext()]
)
```
## Models [#models]
* `gemini-3*` takes `thinkingLevel` — `.none` and `.minimal` both map to
`minimal` (thinking can't be fully disabled), `.xhigh` caps at `high`.
Exception: `gemini-3-pro-image` stays on budgets.
* Everything else takes `thinkingBudget`: `0` for `.none`, otherwise a
fraction of the 65,536-token ceiling capped at 32,768 (2.5 Pro,
`gemini-3-pro-image`) or 24,576 (the rest).
### Current lineup [#current-lineup]
Language models as of July 2026, newest first:
| Model | Notes |
| ------------------------------------------------------------- | --------------------------------------- |
| `gemini-3.6-flash` | Newest flash (July 2026) |
| `gemini-3.5-flash`, `gemini-3.5-flash-lite` | Previous flash line |
| `gemini-3.1-pro-preview` | Pro preview |
| `gemini-3.1-flash-lite` | Fast and cheap |
| `gemini-3-flash`, `gemini-3-pro-preview` | Gemini 3 line |
| `gemini-3-pro-image` | Image-out model — budget-based thinking |
| `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-2.5-flash-lite` | The 2.5 line |
| `gemma-4-31b-it` | Open-weights Gemma via the same API |
## Vertex AI [#vertex-ai]
```swift
let vertex = GoogleVertexModel(
"gemini-3.5-flash",
project: "my-project", // or GOOGLE_VERTEX_PROJECT
location: "us-central1" // or GOOGLE_VERTEX_LOCATION
)
```
Express-mode API keys and bearer `accessToken:` auth both work; the
request body is identical to the Gemini wire.
## Gemini Live [#gemini-live]
`GoogleRealtimeModel` connects to the Live API for realtime voice:
tokens mint against `v1alpha/auth_tokens` and the session config rides
in the token request. See [Realtime voice](/docs/realtime) for connection,
session, audio, and tool-call events.
## Embeddings [#embeddings]
```swift
let embeddings = GoogleEmbeddingModel(
"gemini-embedding-001",
taskType: .retrievalDocument,
outputDimensionality: 768
)
let vectors = try await embedMany(model: embeddings, values: chunks)
```
One text uses `embedContent`, several use `batchEmbedContents`
automatically. `taskType` covers the documented set (retrieval query and
document, semantic similarity, classification, clustering, question
answering, fact verification, code retrieval).
## Beyond text [#beyond-text]
| Surface | Entry point |
| -------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| [Image generation](/docs/image-generation) | `GoogleImageModel("imagen-4.0-generate-001")` — Imagen `:predict` |
| [Video generation](/docs/video-generation) | `GoogleVideoModel("veo-3.1-generate-preview")` — Veo `:predictLongRunning` with polling and URI download |
| [Speech generation](/docs/speech-generation) | `GoogleSpeechModel("gemini-3.1-flash-tts-preview")` — audio modality on `generateContent` |
| Music | `GoogleMusicModel("lyria-3-clip-preview").generateMusic(prompt:)` |
## Platform APIs [#platform-apis]
```swift
let files = GoogleFilesClient()
let uploaded = try await files.upload(audio, mimeType: "audio/mpeg", displayName: "call.mp3")
let ready = try await files.waitUntilActive(uploaded.name)
let answer = try await generateText(
model: GoogleModel("gemini-3.6-flash"),
messages: [Message(role: .user, content: [
.text("Summarize this call."),
.file(FileContent(url: URL(string: ready.uri)!, mediaType: "audio/mpeg"))
])]
)
```
`GoogleFilesClient` implements Google's two-step resumable upload
(`X-Goog-Upload-Protocol: resumable`, then a finalize PUT to the returned
URL), plus `get`, `list`, `delete`, and `waitUntilActive` for the
`PROCESSING → ACTIVE` transition that large media needs.
* `GoogleCachedContentClient`: explicit context caching — `create` (with
`ttlSeconds`), `list`, `get`, `updateTTL`, `delete`. Pass the returned
`cachedContents/…` name through `providerOptions` as `cachedContent`.
* `GoogleBatchClient`: `create` (`batchGenerateContent`),
`createEmbeddings` (`asyncBatchEmbedContent`), `get`, `list`, `cancel`,
`delete` — the 50%-cheaper async path.
* `GoogleModel.countTokens(_:systemInstruction:)`: pre-flight sizing against
`models/{id}:countTokens`.
# Groq (/docs/providers/groq)
```swift
let model = GroqModel("llama-3.3-70b-versatile")
```
Key from `GROQ_API_KEY`; base URL `https://api.groq.com/openai/v1`.
## Features [#features]
* Rides the shared chat-completions wire: tools, structured output, and
vision are encoded for any model that honors them.
* Reasoning models stream thinking as `.reasoningDelta`.
* Usage arrives in Groq's `x_groq` envelope, including
`cachedInputTokens` for prompt-cache hits.
* The `reasoning` parameter maps to `reasoning_effort`; `.xhigh`
coerces to `high`.
* Models that emit inline `` blocks instead of structured
reasoning pair well with the
[extractReasoning middleware](/docs/middleware).
## Server tools [#server-tools]
The `groq/compound` models run tools for you. Pass the typed builders
under `GroqModel.Tools` in `tools:` — they serialize to Groq's
`{"type": "browser_search"}` / `{"type": "code_execution"}` entries:
```swift
let result = try await generateText(
model: GroqModel("groq/compound"),
prompt: "What shipped in Swift this week? Run any code you need.",
tools: [GroqModel.Tools.browserSearch(), GroqModel.Tools.codeExecution()]
)
```
## Models [#models]
Groq hosts open models rather than training its own line. The catalog
as of July 2026 (from `console.groq.com/docs/models`):
| Model | Notes |
| ------------------------------------------- | ----------------------------------------------------------- |
| `llama-3.3-70b-versatile` | Production workhorse, 131k context |
| `llama-3.1-8b-instant` | 560 tokens/sec, 131k context |
| `openai/gpt-oss-120b` | OpenAI's open-weights reasoner, \~500 tokens/sec |
| `openai/gpt-oss-20b` | Smaller gpt-oss, \~1000 tokens/sec |
| `groq/compound`, `groq/compound-mini` | Agentic systems with built-in web search and code execution |
| `meta-llama/llama-4-scout-17b-16e-instruct` | Preview |
| `qwen/qwen3.6-27b`, `qwen/qwen3-32b` | Preview |
The catalog rotates — whatever the console lists, the pack speaks.
Groq also serves Whisper on the OpenAI-shaped transcription endpoint,
so the transcription pack points there directly:
```swift
let transcript = try await transcribe(
model: OpenAITranscriptionModel(
"whisper-large-v3-turbo",
apiKey: groqKey,
baseURL: URL(string: "https://api.groq.com/openai/v1")!
),
audio: audioData,
mediaType: "audio/mpeg"
)
```
# Hugging Face (/docs/providers/huggingface)
`HuggingFaceModel` reads `HUGGINGFACE_API_KEY` and targets the
OpenAI-compatible router at `https://router.huggingface.co/v1`.
```swift
let model = HuggingFaceModel("meta-llama/Llama-3.3-70B-Instruct")
let result = try await generateText(model: model, prompt: "Say hello.")
print(result.text)
```
`HuggingFaceModel` targets the router's OpenAI **Responses** endpoint
(`/responses`), the same wire as `OpenAIModel`, and fans out to whichever
inference provider serves the model. Tools, structured output, vision, and
reasoning depend on the selected model. The router has no embeddings endpoint,
so there's no HuggingFace embedding pack.
## Models [#models]
Popular router models as of July 2026:
| Model ID |
| ------------------------------------- |
| `meta-llama/Llama-3.3-70B-Instruct` |
| `Qwen/Qwen3-Coder-480B-A35B-Instruct` |
| `Qwen/Qwen3-32B` |
| `deepseek-ai/DeepSeek-V3.1` |
| `deepseek-ai/DeepSeek-R1-0528` |
| `google/gemma-3-27b-it` |
The catalog is large and dynamic. Copy any model id from the
[Hugging Face models](https://huggingface.co/models?inference_provider=all\&sort=trending)
that exposes chat-completions inference.
# Hume (/docs/providers/hume)
`HumeSpeechModel` conforms to `SpeechModel` and works with
[`generateSpeech`](/docs/speech-generation).
```swift
let result = try await generateSpeech(
model: HumeSpeechModel(),
text: "I can't believe we made it!",
voice: voiceID,
instructions: "Relieved, warm, and slightly breathless",
outputFormat: "mp3"
)
```
The model reads `HUME_API_KEY` and uses `https://api.hume.ai`. `voice:` maps
to the Hume voice ID, `instructions:` becomes the utterance description, and
`speed:` controls utterance speed. The default model ID is `default` because
the voice and utterance settings drive the request.
## Model selection [#model-selection]
Hume's TTS endpoint does not expose a separate model catalog or accept a
model ID in this request. `HumeSpeechModel` keeps `default` only as its SDK
identity; select a saved or library voice with `voice:`. See Hume's
[voice selection documentation](https://dev.hume.ai/docs/text-to-speech-tts/voice).
# Providers (/docs/providers)
Every provider implements one protocol:
```swift
public protocol LanguageModel: Sendable {
var provider: String { get }
var modelID: String { get }
func stream(_ request: LanguageModelRequest) async throws
-> AsyncThrowingStream
}
```
Higher-level functions are built on this spec, so a provider swap never
touches your feature code.
## Native packs [#native-packs]
Each pack speaks its provider's real wire, mirroring the `@ai-sdk/*`
package family. Where upstream is native, the port is native too; no
lowest-common-denominator shims.
| Provider | Notes |
| ------------------------------------------------ | ----------------------------------------------------------------------------- |
| `OpenAIModel` | Responses API by default, `.chat` for chat completions; reasoning-model rules |
| `AnthropicModel` | Messages API; thinking, tool use, forced-tool JSON mode |
| `GoogleModel` | Native Gemini wire; thinking levels and budgets |
| `GoogleVertexModel` | Express-mode keys or bearer tokens |
| `AzureOpenAIProvider` | Deployment-based routing |
| `BedrockModel` | Converse over AWS event stream binary framing; API-key or SigV4 auth |
| `BedrockMantleProvider` | `bedrock-mantle`: Responses, chat completions, and Anthropic Messages |
| `XaiModel` | Responses API default, typed `SearchParameters`, `.chat` legacy |
| `GroqModel` | Reasoning deltas, `x_groq` usage envelope, cached tokens |
| `DeepSeekModel` | `reasoning_content`, cache-hit accounting |
| `MistralModel`, `PerplexityModel`, `CohereModel` | Native wires, citations as sources |
| `FoundationModelsModel` | [On-device models](/docs/on-device); no upstream analog |
Keys come from each provider's conventional environment variable
(`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `XAI_API_KEY`, ...) or an explicit
`apiKey:` argument. `baseURL` includes the version path, matching the AI
SDK's URL semantics, so gateways and proxies drop in cleanly.
## First-class compatible providers [#first-class-compatible-providers]
Providers that share the OpenAI chat-completions wire still have their own
model types with the right URLs and environment keys:
```swift
let together = TogetherAIModel("MiniMaxAI/MiniMax-M3")
let local = OllamaModel("gemma4")
```
Together, Fireworks, Cerebras, OpenRouter, DeepInfra, Baseten, Vercel
(v0), Gateway, Ollama, LM Studio, and [Sarvam](/docs/providers/sarvam)
(Indic chat) are first-class model packs. `OpenAICompatibleProvider` is the
general initializer for custom gateways and anything else that speaks the
chat-completions wire. The
[compatibility page](/docs/providers/compatibility) has every endpoint and
environment variable.
## Media and retrieval models [#media-and-retrieval-models]
Non-chat capabilities have their own focused protocols and first-class model
types. They do not need to be wrapped as language providers:
| Capability | Protocol | Providers |
| ------------------------------------ | -------------------- | ---------------------------------------------------------------- |
| [Embeddings](/docs/embeddings) | `EmbeddingModel` | OpenAI, Cohere, compatible endpoints |
| [Reranking](/docs/reranking) | `RerankingModel` | Cohere |
| [Images](/docs/image-generation) | `ImageModel` | OpenAI, fal, Luma, Replicate |
| [Speech](/docs/speech-generation) | `SpeechModel` | OpenAI, ElevenLabs, LMNT, Hume, Deepgram, Sarvam |
| [Transcription](/docs/transcription) | `TranscriptionModel` | OpenAI, ElevenLabs, Deepgram, AssemblyAI, Rev.ai, Gladia, Sarvam |
| [Video](/docs/video-generation) | `VideoModel` | xAI, Luma |
| [Realtime](/docs/realtime) | `RealtimeModel` | OpenAI, Google, xAI |
The [media provider index](/docs/providers/media) links every dedicated
provider page.
## The registry and custom providers [#the-registry-and-custom-providers]
`ProviderRegistry` resolves `"provider:model-id"` strings; `customProvider`
publishes friendly aliases with settings baked in and falls back to a
wrapped provider for unknown ids:
```swift
let openai = ProviderRegistry.Provider { OpenAIModel($0) }
let custom = customProvider(
languageModels: [
"fast": OpenAIModel("gpt-5.6-luna"),
"smart": wrapLanguageModel(model: OpenAIModel("gpt-5.6-sol"), middleware: [.extractReasoning()])
],
fallback: openai
)
let registry = ProviderRegistry(providers: ["openai": custom])
let model = try registry.languageModel("openai:fast")
```
## Live search on xAI [#live-search-on-xai]
Grok's server-side search is a typed option, not a raw JSON blob:
```swift
let search = XaiModel.SearchParameters(
mode: .auto,
returnCitations: true,
sources: [.web(), .x(), .news()]
)
let result = try await generateText(
model: XaiModel("grok-4.5"),
prompt: "What happened in AI this week?",
providerOptions: search.providerOptions
)
print(result.sources) // citations from the search
```
The full surface: `mode` (`.auto`, `.on`, `.off`), `returnCitations`, a
`fromDate`/`toDate` window (`YYYY-MM-DD`), `maxSearchResults`, and
per-source filters —
`.web(country:excludedWebsites:allowedWebsites:safeSearch:)`,
`.x(includedHandles:excludedHandles:postFavoriteCount:postViewCount:)`,
`.news(country:excludedWebsites:safeSearch:)`. It serializes to xAI's
`search_parameters` body field exactly.
## Sources and citations [#sources-and-citations]
Search-backed providers (Perplexity and Cohere citations, OpenAI and
xAI url annotations) surface as `StreamPart.source` and collect on
`result.sources`.
## Per-provider pages [#per-provider-pages]
Setup, features, and the models each pack special-cases — plus the
[compatibility matrix](/docs/providers/compatibility) for the
cross-provider view.
# Kling AI (/docs/providers/kling)
`KlingVideoModel` conforms to `VideoModel` and works with
[`generateVideos`](/docs/video-generation). Kling signs each request with a
short-lived JWT, so it takes an access key and secret key (from
`KLING_ACCESS_KEY` / `KLING_SECRET_KEY`) rather than a single API key:
```swift
let video = try await generateVideos(
model: KlingVideoModel("kling-v2-master-t2v"),
prompt: "A hot air balloon drifting over green hills"
)
```
The `-t2v`, `-i2v`, and `-motion-control` suffixes select the endpoint and are
stripped from the API model name (so `kling-v2.1-master-i2v` becomes
`kling-v2-1-master` on the `image2video` endpoint). Passing an image with a
suffixless id also routes to image-to-video. The model submits the task and
polls until `task_status` is `succeed`. Tune polling with `pollInterval:` and
`pollTimeout:`.
## Models [#models]
As of July 2026 (each has `-t2v` / `-i2v` variants):
| Model ID | Notes |
| --------------------------------------------- | ----------- |
| `kling-v3.0-t2v` / `-i2v` / `-motion-control` | Newest |
| `kling-v2.6-t2v` / `-i2v` / `-motion-control` | v2.6 line |
| `kling-v2.5-turbo-t2v` / `-i2v` | Fast tier |
| `kling-v2.1-master-t2v` / `-i2v` | v2.1 master |
| `kling-v2-master-t2v` / `-i2v` | v2 master |
See the [Kling API docs](https://app.klingai.com/global/dev/document-api) for
per-model parameters.
# LM Studio (/docs/providers/lm-studio)
`LMStudioModel` defaults to `http://localhost:1234/v1` and needs no key.
```swift
let model = LMStudioModel("openai/gpt-oss-20b")
let result = try await generateText(model: model, prompt: "Say hello.")
print(result.text)
```
Override `baseURL:` when the server uses another host or port. You can also
pass custom `headers:` or a test `urlSession:`. On iOS, enable
`NSAllowsLocalNetworking` to reach a server running on your Mac.
## Models [#models]
Model IDs come from the models downloaded in your LM Studio instance, so
there is no universal static list. `openai/gpt-oss-20b` is the current
official quickstart example. Inspect your actual IDs with:
```bash
curl http://localhost:1234/v1/models
```
See LM Studio's [OpenAI-compatible models endpoint](https://lmstudio.ai/docs/developer/openai-compat/models)
for the response format.
# LMNT (/docs/providers/lmnt)
`LMNTSpeechModel` conforms to `SpeechModel` and works with
[`generateSpeech`](/docs/speech-generation).
```swift
let result = try await generateSpeech(
model: LMNTSpeechModel(), // blizzard
text: "Your order is ready.",
voice: "ava",
outputFormat: "mp3"
)
```
The model defaults to `blizzard`, reads `LMNT_API_KEY`, and uses
`https://api.lmnt.com`. Pass an LMNT voice ID through `voice:`. `speed`,
`outputFormat`, and additional provider options are forwarded to LMNT.
## Models [#models]
`blizzard` is LMNT's current Blizzard 2.0 speech model. LMNT selects the
speaker separately through the `voice:` argument. See LMNT's
[model overview](https://docs.lmnt.com/models/overview) for current model and
language support.
# Luma (/docs/providers/luma)
One `LUMA_API_KEY` powers both Luma model types.
## Images [#images]
```swift
let result = try await generateImage(
model: LumaImageModel(), // photon-1
prompt: "Editorial product photography on red paper",
aspectRatio: "4:3"
)
```
`LumaImageModel` defaults to `photon-1`. It submits the generation, polls until
completion, downloads the image, and returns it as `Data`.
### Image models [#image-models]
| Model ID | Use |
| ---------------- | ---------------------------- |
| `photon-1` | Highest-quality Photon model |
| `photon-flash-1` | Faster Photon generation |
## Video [#video]
```swift
let result = try await generateVideo(
model: LumaVideoModel(), // ray-2
prompt: "A tram crossing a rainy neon street",
aspectRatio: "16:9",
duration: 5
)
```
`LumaVideoModel` defaults to `ray-2` and returns the completed asset in
`result.urls`. To animate a still, pass a remotely accessible image URL:
### Video models [#video-models]
| Model ID | Use |
| ------------- | ------------------------- |
| `ray-2` | Highest-quality Ray model |
| `ray-flash-2` | Faster Ray generation |
```swift
let result = try await generateVideo(
model: LumaVideoModel(),
prompt: "Slow camera push-in",
image: ImageContent(url: imageURL)
)
```
Both models use `https://api.lumalabs.ai` and accept `pollInterval` and
`pollTimeout` initializer arguments. See [image generation](/docs/image-generation)
and [video generation](/docs/video-generation) for the shared APIs. These are
the model IDs accepted by the Dream Machine endpoints used by this SDK; verify
them in Luma's [image](https://docs.lumalabs.ai/docs/image-generation) and
[video](https://docs.lumalabs.ai/docs/video-generation) API documentation.
# Media providers (/docs/providers/media)
Media models are first-class provider types, not language-model wrappers. Each
conforms to one focused protocol and plugs into its matching top-level
function.
## Image generation [#image-generation]
| Provider | Model type |
| -------------------------------------- | --------------------- |
| [OpenAI](/docs/providers/openai) | `OpenAIImageModel` |
| [fal](/docs/providers/fal) | `FalImageModel` |
| [Luma](/docs/providers/luma) | `LumaImageModel` |
| [Replicate](/docs/providers/replicate) | `ReplicateImageModel` |
See [image generation](/docs/image-generation) for the shared API.
## Speech generation [#speech-generation]
| Provider | Model type |
| ---------------------------------------- | ----------------------- |
| [OpenAI](/docs/providers/openai) | `OpenAISpeechModel` |
| [ElevenLabs](/docs/providers/elevenlabs) | `ElevenLabsSpeechModel` |
| [LMNT](/docs/providers/lmnt) | `LMNTSpeechModel` |
| [Hume](/docs/providers/hume) | `HumeSpeechModel` |
| [Deepgram](/docs/providers/deepgram) | `DeepgramSpeechModel` |
| [Sarvam](/docs/providers/sarvam) | `SarvamSpeechModel` |
See [speech generation](/docs/speech-generation) for the shared API.
## Transcription [#transcription]
| Provider | Model type |
| ---------------------------------------- | ------------------------------ |
| [OpenAI](/docs/providers/openai) | `OpenAITranscriptionModel` |
| [ElevenLabs](/docs/providers/elevenlabs) | `ElevenLabsTranscriptionModel` |
| [Deepgram](/docs/providers/deepgram) | `DeepgramTranscriptionModel` |
| [AssemblyAI](/docs/providers/assemblyai) | `AssemblyAITranscriptionModel` |
| [Rev.ai](/docs/providers/rev-ai) | `RevAITranscriptionModel` |
| [Gladia](/docs/providers/gladia) | `GladiaTranscriptionModel` |
| [Sarvam](/docs/providers/sarvam) | `SarvamTranscriptionModel` |
See [transcription](/docs/transcription) for the shared API. Groq-hosted
Whisper also works through `OpenAITranscriptionModel` with Groq's base URL.
## Video generation [#video-generation]
| Provider | Model type |
| ---------------------------- | ---------------- |
| [xAI](/docs/providers/xai) | `XaiVideoModel` |
| [Luma](/docs/providers/luma) | `LumaVideoModel` |
See [video generation](/docs/video-generation) for the shared API.
# Meta (/docs/providers/meta)
```swift
let model = MetaModel("muse-spark-1.2") // Responses API
let chat = MetaModel.chat("muse-spark-1.2") // chat-completions wire
```
Key from `MODEL_API_KEY` (or `META_API_KEY`); base URL
`https://api.meta.ai/v1`. Pass `apiKey:`, `baseURL:`, or `headers:` to
override.
Meta's API speaks three wire formats over the same models: Responses,
Chat Completions, and an Anthropic-compatible Messages endpoint. This pack
covers the first two. `MetaModel` targets Responses, which carries
reasoning across turns and is the only wire that runs search grounding.
`MetaModel.chat` targets `/v1/chat/completions`, which is the simpler drop-in
but does not carry reasoning between turns.
## Models [#models]
| Model ID | Tier | Context window |
| ---------------------------- | ----------- | -------------- |
| `muse-spark-1.2` | Standard | 1,048,576 |
| `muse-spark-1.1` | Standard | 1,048,576 |
| `muse-spark-1.2-contributor` | Contributor | 1,048,576 |
All three are the same Muse Spark family. `-contributor` is the same
checkpoint as `1.2` at a steep discount, in exchange for Meta training on
your prompts and completions — worth knowing before you point production
traffic at it.
Muse Spark takes text, image, video, and PDF input. The pack maps text,
images (public URL or inline bytes), and PDFs onto Meta's `input_text`,
`input_image`, and `input_file` blocks. Video input and `file_id`
references from the Files API are not wired up yet.
## Reasoning [#reasoning]
Muse Spark always reasons. Effort maps onto the unified `reasoning:`
parameter, including `.xhigh`:
```swift
let result = try await generateText(
model: MetaModel("muse-spark-1.2"),
prompt: "Prove that the square root of 2 is irrational.",
reasoning: .xhigh
)
```
`.none` is the one value the API rejects outright with a 400, so the pack
drops it and lets the model pick its own depth instead of failing the
call.
The raw chain of thought never comes back as text. Passing an explicit
`reasoning:` also asks Meta for a reasoning summary, which streams as
`.reasoningDelta`. Leave `reasoning:` unset and the model still reasons at
its own depth, but no summary is requested, so nothing arrives on that
channel. A summary isn't guaranteed even when you ask — short turns often
produce none.
Unlike OpenAI's reasoning models, Muse Spark still accepts `temperature`
and `topP`, so the pack forwards them. Meta tunes the model for the
defaults, though, and clearer instructions usually beat a lower
temperature.
## Search grounding [#search-grounding]
Pass `MetaModel.Tools.webSearch()` and the model decides whether to
search. Citations arrive as `StreamPart.source`:
```swift
let result = try await generateText(
model: MetaModel("muse-spark-1.2"),
prompt: "What are the latest developments in AI regulation?",
tools: [
MetaModel.Tools.webSearch(
searchContextSize: "high",
userLocation: .init(country: "GB", city: "London")
)
]
)
for source in result.sources { print(source.url) }
```
Enabling the tool doesn't force a search — the model skips it when it can
answer from training data. Search grounding is Responses-only: pass these
builders to `MetaModel.chat` and they go out on a wire that has no such
tool type, which the API rejects.
## Tool search [#tool-search]
With a large tool catalog, `MetaModel.Tools.toolSearch()` lets the server
load definitions on demand instead of sending all of them every turn.
Mark the tools you want deferred and they stay out of the prompt until the
model asks for them:
```swift
let result = try await generateText(
model: MetaModel("muse-spark-1.2"),
prompt: "Refund order 1234.",
tools: [refund, lookup, MetaModel.Tools.toolSearch()]
)
```
Only one `tool_search` tool per request; a second one returns a 400.
## Stateless reasoning replay [#stateless-reasoning-replay]
By default Meta stores each response so you can chain turns with
`previous_response_id`. If you'd rather keep nothing server-side, ask for
encrypted reasoning items and replay them yourself:
```swift
let result = try await generateText(
model: MetaModel("muse-spark-1.2"),
prompt: "Plan the migration.",
providerOptions: .object([
"store": .bool(false),
"include": .array([.string("reasoning.encrypted_content")])
])
)
```
The two are mutually exclusive: `include` plus `previous_response_id` in
the same request is a 400.
## Provider options [#provider-options]
Anything else on the Responses body rides `providerOptions` and merges
onto the request: `previous_response_id`, `background`,
`prompt_cache_retention` (`"in_memory"` or `"24h"`), `instructions`,
`metadata`, `frequency_penalty`, `presence_penalty`.
A few OpenAI parameters have no equivalent here. `logprobs` returns a 400
because Muse Spark is a reasoning model, and `truncation: "auto"` is
rejected — Meta never trims context for you, so an over-long request
fails and you compact it yourself.
# Mistral (/docs/providers/mistral)
```swift
let model = MistralModel("mistral-medium-3.5")
```
Key from `MISTRAL_API_KEY`; base URL `https://api.mistral.ai/v1`.
## Features [#features]
* Rides the shared chat-completions wire: tools, structured output, and
vision are encoded for any model that honors them.
## Models [#models]
The `reasoning` parameter maps to `reasoning_effort` only on the models
that accept it — `mistral-small-latest`, `mistral-small-2603`,
`mistral-medium-3`, and `mistral-medium-3.5`. The knob is binary
upstream: `.none` sends `none`, every other level sends `high`. Other
model ids ignore the parameter rather than erroring.
### Current lineup [#current-lineup]
As of July 2026, newest first:
| Model | Notes |
| ----------------------------------------------- | ----------------------------------- |
| `mistral-medium-3.5` | Newest (April 2026), reasoning knob |
| `mistral-large-3` | December 2025 flagship |
| `devstral-2`, `devstral-small-2` | Code models |
| `ministral-14b`, `ministral-8b`, `ministral-3b` | Edge tier |
| `magistral-medium`, `magistral-small` | Reasoning line |
| `pixtral-large`, `pixtral-12b` | Vision |
| `codestral`, `codestral-embed`, `mistral-embed` | Code and embeddings |
# Moonshot AI (/docs/providers/moonshot)
`MoonshotModel` reads `MOONSHOT_API_KEY` and targets
`https://api.moonshot.ai/v1`.
```swift
let model = MoonshotModel("kimi-k2.7-code")
let result = try await generateText(model: model, prompt: "Say hello.")
print(result.text)
```
Pass `apiKey:`, `baseURL:`, `headers:`, or `queryParams:` to override the
defaults. Tools, structured output, and streamed reasoning ride the shared
chat-completions wire; reasoning traces surface as `.reasoningDelta`.
## Models [#models]
As of July 2026:
| Model ID | Notes |
| ----------------------------------- | ---------------------- |
| `kimi-k2.7-code` | Newest code-tuned Kimi |
| `kimi-k2.7-code-highspeed` | Faster code tier |
| `kimi-k2.6` | General flagship |
| `kimi-k2.5` | Prior flagship |
| `moonshot-v1-8k` / `-32k` / `-128k` | v1 context tiers |
Check Moonshot's [model list](https://platform.moonshot.ai/docs/pricing/chat)
for the current lineup. Any id the API serves works.
# Ollama (/docs/providers/ollama)
`OllamaModel` defaults to `http://localhost:11434/v1` and needs no key.
```swift
let model = OllamaModel("gemma4")
let result = streamText(model: model, prompt: "Say hello.")
for try await text in result.textStream {
print(text, terminator: "")
}
```
Override `baseURL:` when Ollama runs on another host. Models that emit inline
`` blocks can use the [reasoning extraction middleware](/docs/reasoning#extracting-reasoning-from-text).
On iOS, enable `NSAllowsLocalNetworking` so the simulator can reach your Mac.
## Models [#models]
Current library examples include `gemma4`, `qwen3-coder-next`,
`deepseek-v4-flash`, and `kimi-k2.7-code`. The usable IDs are the models
installed locally; inspect them with `ollama list` or:
```bash
curl http://localhost:11434/api/tags
```
Browse the live [Ollama model library](https://ollama.com/library) to find and
pull additional models.
# Custom OpenAI-compatible endpoints (/docs/providers/openai-compatible)
Use `OpenAICompatibleProvider` for an endpoint without a dedicated pack—a
company gateway, a vLLM server, or a proxy:
```swift
let custom = OpenAICompatibleProvider(
name: "my-gateway",
baseURL: URL(string: "https://llm.your-company.com/v1")!,
apiKey: secret
)
let model = custom("openai/gpt-oss-20b")
```
## Model selection [#model-selection]
There is no universal model list for a custom endpoint. The concrete
`openai/gpt-oss-20b` ID above is only an example; pass an ID returned by your
server's OpenAI-compatible `GET /v1/models` endpoint. If the service has a
dedicated provider page, use its first-class model type and catalog instead.
## Features [#features]
Tools, structured output, and vision are encoded whenever you pass
them; whether a given model honors them is between you and the model.
Embeddings work against the same endpoints:
```swift
let embeddings = custom.textEmbeddingModel("BAAI/bge-large-en-v1.5")
```
The `reasoning` parameter passes through as `reasoning_effort`. Models that
emit inline `` blocks instead pair with the
[extractReasoning middleware](/docs/middleware).
## Named providers [#named-providers]
Named services use their own model types and documentation pages. The old
`OpenAICompatibleProvider.togetherAI()`, `.fireworks()`, and similar factory
methods remain deprecated for source compatibility.
# OpenAI (/docs/providers/openai)
```swift
let model = OpenAIModel("gpt-5.6-sol") // Responses API
let chat = OpenAIModel.chat("gpt-5.6-terra") // chat completions wire
```
Key from `OPENAI_API_KEY`; base URL from `OPENAI_BASE_URL` or
`https://api.openai.com/v1`. `organization:` and `project:` become the
matching request headers.
## Features [#features]
* Tools, structured output (JSON schema mode), vision, and streamed
reasoning summaries (`response.reasoning_summary_text.delta` arrives
as `.reasoningDelta`).
* URL citations surface as `StreamPart.source` and source-url parts in
chat UIs.
* `usage.cachedInputTokens` reports prompt-cache hits.
* Built-in Responses tools have typed builders under `OpenAIModel.Tools`:
`webSearch`, `webSearchPreview`, `fileSearch(vectorStoreIds:)`,
`codeInterpreter`, and `computerUse(displayWidth:displayHeight:environment:)`.
Drop them in `tools:`; the calls and results stream back as
provider-executed parts and citations as `.source`:
```swift
let result = try await generateText(
model: OpenAIModel("gpt-5.6-sol"),
prompt: "What happened in tech today?",
tools: [OpenAIModel.Tools.webSearch()]
)
print(result.sources.map(\.url))
```
`code_interpreter`, `image_generation`, and `mcp` calls added through
`providerOptions` surface the same way — each as a provider-executed
`.toolCall` plus its `.toolResult`. A model refusal arrives as text and
finishes on `.contentFilter`.
* Token logprobs (request them with `logprobs`/`top_logprobs` via
`providerOptions`) collect on `result.providerMetadata["openai"]["logprobs"]`,
on both the Responses and chat wires.
* Other Responses knobs (`store`, `instructions`, `include`,
`previous_response_id`, `max_turns`) pass through `providerOptions`,
merged at the top level (nested objects like `text` and `reasoning`
merge rather than clobber, so `text.verbosity` coexists with a
structured-output format).
## Response lifecycle [#response-lifecycle]
`store: true` and `background: true` responses are managed with
`OpenAIResponsesClient`:
```swift
let client = OpenAIResponsesClient()
let tokens = try await client.countInputTokens(
for: LanguageModelRequest(messages: [.user("How many tokens is this?")]),
modelID: "gpt-5.6-sol"
) // POST /v1/responses/input_tokens
let response = try await client.retrieve("resp_123", include: ["reasoning.encrypted_content"])
let items = try await client.listInputItems("resp_123", limit: 20)
try await client.cancel("resp_123") // background responses
try await client.compact("resp_123") // shrink stored context
try await client.delete("resp_123")
```
## Models [#models]
Any id the API serves works. The library special-cases:
* **Reasoning models** — `o1*`, `o3*`, `o4-mini*`, and `gpt-5*` (except
`gpt-5-chat*`): the `reasoning` parameter maps to `reasoning.effort`
with an automatic detailed summary, and sampling knobs are dropped
where those models reject them.
* **`gpt-5.1` through `gpt-5.6`** — accept `temperature`/`topP` again
when reasoning effort is `none`.
### Current lineup [#current-lineup]
Snapshot from July 2026; new ids work the day OpenAI ships them.
| Model | Notes |
| -------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| `gpt-5.6-sol` / `gpt-5.6-terra` / `gpt-5.6-luna` | Newest family (July 2026): Sol is the flagship, Terra the balanced tier, Luna the fast one |
| `gpt-5.5`, `gpt-5.5-pro` | April 2026 |
| `gpt-5.4`, `-mini`, `-nano`, `-pro` | March 2026 workhorses |
| `gpt-5.3-codex`, `gpt-5.2-codex` | Code-tuned |
| `gpt-5.3-chat`, `gpt-5.2-chat` | Non-reasoning chat variants |
| `gpt-5.2`, `gpt-5.1-thinking`, `gpt-5.1-instant` | Late 2025 |
| `gpt-realtime-2.1`, `gpt-realtime-mini` | Realtime voice |
| `gpt-image-2`, `gpt-image-1.5`, `gpt-image-1-mini` | Image generation and edits |
| `text-embedding-3-large`, `text-embedding-3-small` | Embeddings |
| `whisper-1`, `gpt-4o-mini-tts` | Transcription and speech |
## Beyond text [#beyond-text]
| Surface | Entry point |
| -------------------------------------------------- | -------------------------------------------------------------- |
| [Embeddings](/docs/embeddings) | `OpenAIEmbeddingModel("text-embedding-3-small")` |
| [Images](/docs/image-generation) (generate + edit) | `OpenAIImageModel("gpt-image-2")` |
| [Speech](/docs/speech-generation) | `OpenAISpeechModel("gpt-4o-mini-tts")` |
| [Transcription](/docs/transcription) | `OpenAITranscriptionModel("whisper-1")` |
| [Realtime voice](/docs/realtime) | `OpenAIRealtimeModel("gpt-realtime")` |
| [Files](/docs/files-and-skills) | `OpenAIFiles().upload(...)` |
| [Video](/docs/video-generation) | `OpenAIVideoModel("sora-2")` — create, poll, download, `remix` |
## Multi-agent [#multi-agent]
GPT-5.6 models can spawn and coordinate their own subagent tree inside a
single Responses call:
```swift
let model = OpenAIModel(
"gpt-5.6-sol",
multiAgent: OpenAIModel.MultiAgent(maxConcurrentSubagents: 3)
)
let review = try await generateText(
model: model,
prompt: "Review this diff with three agents: correctness, security, and missing tests.",
tools: [readFile]
)
```
Setting `multiAgent:` adds `multi_agent` to the request and the
`responses_multi_agent=v1` beta header. The root agent (`/root`) spawns
subagents (`/root/reviewer`, `/root/reviewer/tester`, …) and synthesizes the
final answer; `maxConcurrentSubagents` caps active turns across the whole
tree, and defaults to OpenAI's 3.
The six hosted coordination actions — spawn, send message, follow-up task,
wait, interrupt, and list — arrive as `multi_agent_call` items. **Your app
must not execute them**, so they surface as provider metadata
(`result.providerMetadata["openai"]["multiAgentCall"]`) rather than tool
calls. Ordinary function calls from any agent in the tree still run through
the normal tool loop.
## Platform APIs [#platform-apis]
* `OpenAIConversationsClient` — the Conversations API: create (from
`[Message]` or raw items), get, update metadata, list/add/delete items.
* `OpenAIVectorStoresClient` — the store side of `file_search`: create with
expiry, attach and detach files, and `search(_:query:)` with filters and
query rewriting.
* `OpenAIBatchClient`, `OpenAIContainersClient`, and
`OpenAIModerationsClient` (returns `flagged` plus the flagged category
names).
# OpenRouter (/docs/providers/openrouter)
`OpenRouterModel` reads `OPENROUTER_API_KEY` and targets
`https://openrouter.ai/api/v1`.
```swift
let model = OpenRouterModel("anthropic/claude-sonnet-5")
let result = try await generateText(model: model, prompt: "Say hello.")
print(result.text)
```
Use OpenRouter's provider-qualified model IDs. Tools, structured output,
vision, and reasoning are forwarded through the chat-completions request;
support depends on the routed model.
## Models [#models]
The OpenRouter catalog is dynamic. Current examples include:
| Model ID |
| --------------------------- |
| `openai/gpt-5.6-sol` |
| `anthropic/claude-sonnet-5` |
| `x-ai/grok-4.5` |
| `z-ai/glm-5.2` |
Query OpenRouter's public [`GET /api/v1/models`](https://openrouter.ai/api/v1/models)
endpoint or browse the [model catalog](https://openrouter.ai/models) for the
complete live list, pricing, and context limits.
# Perplexity (/docs/providers/perplexity)
```swift
let model = PerplexityModel("sonar-pro")
```
Key from `PERPLEXITY_API_KEY`; base URL `https://api.perplexity.ai`.
## Features [#features]
* Every answer is search-grounded; citations and the richer
`search_results` (title + URL) surface as `StreamPart.source` and
collect on `result.sources`.
* Structured output and image input ride the chat-completions wire on
models that support them.
* No tool calling — Perplexity's API doesn't offer it, so the library
doesn't pretend otherwise. The `reasoning` parameter is ignored the
same way.
* Image results and related questions arrive on
`result.providerMetadata["perplexity"]` (`images`, `related_questions`)
when you request them via `providerOptions` (`return_images`,
`return_related_questions`).
```swift
let result = try await generateText(
model: PerplexityModel("sonar-pro"),
prompt: "Show me photos of the aurora.",
providerOptions: ["return_images": true, "return_related_questions": true]
)
let meta = result.providerMetadata?["perplexity"]
print(meta?["images"] ?? .null)
print(meta?["related_questions"] ?? .null)
```
```swift
let result = try await generateText(
model: PerplexityModel("sonar-pro"),
prompt: "What changed in Swift 6.2?"
)
for source in result.sources {
print(source.title ?? source.url)
}
```
## Models [#models]
Three ids cover the lineup: `sonar` (fast grounded answers),
`sonar-pro` (deeper multi-hop search), and `sonar-reasoning-pro`
(search plus visible reasoning).
# Prodia (/docs/providers/prodia)
`ProdiaImageModel` conforms to `ImageModel` and works with
[`generateImage`](/docs/image-generation). It reads `PRODIA_TOKEN` and uses
`https://inference.prodia.com/v2`. The model id is a Prodia job type; the
prompt and extras ride the `config` object, and the image bytes come straight
back in the response:
```swift
let result = try await generateImage(
model: ProdiaImageModel("inference.flux.schnell.txt2img.v2"),
prompt: "A neon city street in the rain"
)
```
Set `accept:` on the initializer to request a different image MIME type; extra
`config` fields go under the `prodia` provider key.
## Models [#models]
As of July 2026:
| Model ID (job type) | Notes |
| ---------------------------------------- | -------------- |
| `inference.flux.schnell.txt2img.v2` | FLUX schnell |
| `inference.flux-fast.schnell.txt2img.v2` | Faster FLUX |
| `inference.nano-banana.img2img.v2` | Image-to-image |
| `inference.wan2-2.lightning.txt2vid.v0` | Text-to-video |
Browse job types in the [Prodia docs](https://docs.prodia.com/).
# QuiverAI (/docs/providers/quiverai)
`QuiverAIImageModel` conforms to `ImageModel` and works with
[`generateImage`](/docs/image-generation). Unlike the raster providers,
QuiverAI returns **SVG** markup, so `result.image` holds the UTF-8 SVG bytes.
It reads `QUIVERAI_API_KEY` and uses `https://api.quiver.ai/v1`.
```swift
let result = try await generateImage(
model: QuiverAIImageModel("arrow-1.1"),
prompt: "A minimalist mountain logo"
)
let svg = String(decoding: result.image, as: UTF8.self)
```
Extra fields go under the `quiverai` provider key.
## Models [#models]
As of July 2026:
| Model ID | Notes |
| --------------- | ---------------- |
| `arrow-1.1-max` | Highest quality |
| `arrow-1.1` | Default |
| `arrow-1` | Prior generation |
See the [QuiverAI docs](https://docs.quiver.ai/) for style options.
# Replicate (/docs/providers/replicate)
`ReplicateImageModel` conforms to `ImageModel` and works with
[`generateImage`](/docs/image-generation).
```swift
let result = try await generateImage(
model: ReplicateImageModel("openai/gpt-image-2"),
prompt: "A handmade paper city at sunrise",
aspectRatio: "16:9",
seed: 7
)
```
The model reads `REPLICATE_API_TOKEN` and uses
`https://api.replicate.com/v1`. Pass the Replicate model slug as the first
initializer argument.
Shared image settings become Replicate input fields. Additional model-specific
input goes under the `replicate` provider key:
```swift
providerOptions: ["replicate": ["output_format": "png"]]
```
## Models [#models]
Current official image models include:
| Model slug |
| --------------------------- |
| `openai/gpt-image-2` |
| `google/nano-banana-2-lite` |
| `prunaai/p-image` |
| `prunaai/p-image-try-on` |
Replicate's catalog changes frequently. Browse the live
[official-model collection](https://replicate.com/collections/official) or
pass any compatible `owner/model` slug. A pinned version can be written as
`owner/model:version`.
# Rev.ai (/docs/providers/rev-ai)
`RevAITranscriptionModel` conforms to `TranscriptionModel` and works with
[`transcribe`](/docs/transcription).
```swift
let result = try await transcribe(
model: RevAITranscriptionModel(), // machine
audio: audioData,
mediaType: "audio/mpeg"
)
```
The model reads `REVAI_API_KEY`, defaults to the `machine` transcriber, and
uses `https://api.rev.ai`. It submits the multipart job, polls it, then fetches
the completed transcript. Word timing is exposed through `result.segments`.
Use `pollInterval` and `pollTimeout` on the model initializer to tune polling.
## Models [#models]
| Transcriber ID | Use |
| -------------- | ------------------------------------------------- |
| `machine` | Recommended general transcription and SDK default |
| `low_cost` | Reverb Turbo low-cost transcription |
`machine_v2` is deprecated and routes to `machine`. Check Rev.ai's
[changelog](https://docs.rev.ai/changelog) for transcriber availability.
# Sarvam (/docs/providers/sarvam)
Sarvam AI covers three surfaces, each on the SDK's existing protocols.
One key (`SARVAM_API_KEY`) drives all of them.
## Chat [#chat]
Sarvam chat has its own first-class model:
```swift
let result = try await generateText(
model: SarvamModel("sarvam-105b"),
prompt: "मुझे भारत के बारे में एक तथ्य बताओ।"
)
```
Base URL `https://api.sarvam.ai/v1`, `Authorization: Bearer` auth. Tools,
structured output, and streaming ride the shared chat-completions path.
The chat models are `sarvam-30b` (64K context) and `sarvam-105b` (128K
context), both reasoning models: `reasoning` goes out as `reasoning_effort`,
and the model's thinking streams back as `.reasoningDelta`.
```swift
let result = try await generateText(
model: SarvamModel("sarvam-105b"),
prompt: "Prove that √2 is irrational.",
reasoning: .high
)
print(result.reasoningText)
print(result.text)
```
## Text to speech [#text-to-speech]
`SarvamSpeechModel` calls the Bulbul voices. Sarvam requires a target
language; set it on the model or per request. It uses the shared
[`generateSpeech`](/docs/speech-generation) API:
```swift
let tts = SarvamSpeechModel("bulbul:v3", targetLanguage: "hi-IN")
let audio = try await generateSpeech(
model: tts,
text: "नमस्ते, आप कैसे हैं?",
voice: "anushka", // → speaker
speed: 1.1, // → pace
outputFormat: "mp3" // → output_audio_codec
)
```
The language override and Sarvam-only knobs (`pitch`, `loudness`,
`temperature`, `speech_sample_rate`) go through `providerOptions`:
```swift
try await generateSpeech(
model: SarvamSpeechModel(), // defaults to bulbul:v3, en-IN
text: "வணக்கம்",
providerOptions: ["target_language_code": "ta-IN", "temperature": 0.7]
)
```
Auth is the `api-subscription-key` header. Audio comes back base64-encoded
and is decoded for you; `outputFormat` sets the returned media type
(defaults to `audio/wav`).
## Transcription [#transcription]
`SarvamTranscriptionModel` posts the audio as multipart to the Saaras
models through the shared [`transcribe`](/docs/transcription) API. Language
and mode ride through `providerOptions`:
```swift
let stt = SarvamTranscriptionModel("saaras:v3")
let result = try await transcribe(
model: stt,
audio: audioData,
mediaType: "audio/wav",
providerOptions: ["language_code": "hi-IN", "mode": "transcribe"]
)
print(result.text) // transcript
print(result.language) // detected language_code
```
`mode` accepts `transcribe`, `translate`, `verbatim`, `translit`, or
`codemix`.
# Together AI (/docs/providers/together-ai)
`TogetherAIModel` reads `TOGETHER_API_KEY` and targets
`https://api.together.xyz/v1`.
```swift
let model = TogetherAIModel("MiniMaxAI/MiniMax-M3")
let result = try await generateText(model: model, prompt: "Say hello.")
print(result.text)
```
Pass `apiKey:`, `baseURL:`, `headers:`, or `urlSession:` to override the
defaults. Tools, structured output, vision, and `reasoning` use the shared
chat-completions wire and depend on the selected model.
## Models [#models]
Current serverless chat models include:
| Model ID |
| ----------------------------- |
| `MiniMaxAI/MiniMax-M3` |
| `Qwen/Qwen3.7-Max` |
| `deepseek-ai/DeepSeek-V4-Pro` |
| `openai/gpt-oss-120b` |
See Together's live [serverless model list](https://docs.together.ai/docs/serverless/models)
and [recommended models](https://docs.together.ai/docs/inference/recommended-models)
for availability and capability details.
# Vercel (/docs/providers/vercel)
`VercelModel` reads `VERCEL_API_KEY` and targets `https://api.v0.dev/v1`.
```swift
let model = VercelModel("v0-1.5-lg")
let result = try await generateText(model: model, prompt: "Build a settings screen.")
print(result.text)
```
Use `baseURL:`, `headers:`, and `urlSession:` for proxy or test overrides.
This pack uses the shared chat-completions transport.
## Models [#models]
The last officially published OpenAI-compatible v0 family is:
| Model ID | Size |
| ----------- | ------------- |
| `v0-1.5-lg` | Large |
| `v0-1.5-md` | Medium |
| `v0-1.0-md` | Legacy medium |
Vercel's current [v0 Platform API documentation](https://v0.dev/docs/api)
focuses on projects and chats and does not publish a model-list endpoint. The
IDs above come from Vercel's official
[composite model announcement](https://vercel.com/blog/v0-composite-model-family);
check account availability before depending on the legacy model endpoint. For
Vercel's current multi-provider catalog, use
[`AIGatewayModel`](/docs/providers/ai-gateway) instead.
# Voyage AI (/docs/providers/voyage)
Both models read `VOYAGE_API_KEY` and target `https://api.voyageai.com/v1`.
## Embeddings [#embeddings]
`VoyageEmbeddingModel` works with [`embed` and `embedMany`](/docs/embeddings).
Pass `inputType: .query` or `.document` to match Voyage's asymmetric retrieval,
and `outputDimension:` for Matryoshka truncation.
```swift
let vectors = try await embedMany(
model: VoyageEmbeddingModel("voyage-3.5", inputType: .document),
values: documents
)
```
Requests batch at 128 inputs each and merge automatically.
## Reranking [#reranking]
`VoyageRerankingModel` works with [`rerank`](/docs/reranking):
```swift
let ranked = try await rerank(
model: VoyageRerankingModel("rerank-2.5"),
query: "How do I cancel my subscription?",
documents: documents,
topN: 3
)
ranked.rankedDocuments.first?.relevanceScore
```
## Models [#models]
As of July 2026:
| Model ID | Kind |
| ----------------------------------------------------------------- | ------------------ |
| `voyage-3.5` / `voyage-3.5-lite` | General embeddings |
| `voyage-4` / `voyage-4-large` / `voyage-4-lite` / `voyage-4-nano` | Embeddings |
| `voyage-code-3.5` | Code embeddings |
| `voyage-finance-2` / `voyage-law-2` / `voyage-multilingual-2` | Domain embeddings |
| `rerank-2.5` / `rerank-2.5-lite` | Reranking |
See Voyage's [docs](https://docs.voyageai.com/docs/embeddings) for dimensions
and context limits.
# xAI (/docs/providers/xai)
```swift
let model = XaiModel("grok-4.5") // Responses API
let chat = XaiModel.chat("grok-3") // legacy chat wire
```
Key from `XAI_API_KEY`; base URL `https://api.x.ai/v1`.
## Features [#features]
* Tools, structured output, vision, streamed reasoning, citations as
`StreamPart.source`, and cached-token usage.
* Prefer the `web_search` / `x_search` [server-side tools](#server-side-tools)
for search. xAI has **deprecated Live Search** (the `search_parameters`
body field), and requests that use it can return a "Live search is
deprecated" error. `XaiModel.SearchParameters` still serializes that
field for legacy callers, but new code should reach for the tools, which
carry the same image/video filters.
```swift
let legacy = XaiModel.SearchParameters(
mode: .auto,
returnCitations: true,
fromDate: "2026-07-01",
maxSearchResults: 10,
sources: [.web(country: "US"), .x(includedHandles: ["xai"]), .news(), .rss(links: ["https://…"])]
)
```
Per-source filters mirror the tool options:
`.web(country:excludedWebsites:allowedWebsites:safeSearch:enableImageSearch:enableImageUnderstanding:)`,
`.x(includedHandles:excludedHandles:postFavoriteCount:postViewCount:enableVideoUnderstanding:)`,
`.news(country:excludedWebsites:safeSearch:)`, and `.rss(links:)` (xAI
currently honors a single RSS link).
The unified `topK` reaches Grok's `top_k` on the chat wire; `min_p` and
other sampling extras (`logprobs`, `topLogprobs`, `parallel_function_calling`)
ride `providerOptions`.
## Server-side tools [#server-side-tools]
On the Responses API, xAI can run tools for you. Pass the typed builders
under `XaiModel.Tools` in `tools:` — the calls and results stream back as
provider-executed parts, and citations as `.source`:
```swift
let result = try await generateText(
model: XaiModel("grok-4.5"),
prompt: "What did xAI announce this week?",
tools: [
XaiModel.Tools.webSearch(allowedDomains: ["x.ai"]),
XaiModel.Tools.xSearch(allowedXHandles: ["xai"], fromDate: "2026-07-01"),
XaiModel.Tools.codeExecution(),
]
)
```
Builders: `webSearch`, `xSearch`, `codeExecution`,
`fileSearch(vectorStoreIds:)`, `mcpServer(serverUrl:)`, `viewImage`,
`viewXVideo`. `SearchParameters` above is the older `search_parameters`
knob; `Tools` is the provider-executed path that surfaces each call and
its result.
### Multi-agent [#multi-agent]
`grok-4.20-multi-agent` runs a research swarm server-side. It's just a
model id plus the three server-side tools — pass them by whatever names
you want to reference in `activeTools`:
```swift
let result = streamText(
model: XaiModel("grok-4.20-multi-agent"),
system: "You are a research assistant in multi-agent mode.",
prompt: "What shipped across the AI labs this week?",
tools: [
XaiModel.Tools.webSearch(name: "xai_web_search"),
XaiModel.Tools.xSearch(name: "xai_x_search"),
XaiModel.Tools.codeExecution(name: "xai_code_execution"),
],
toolChoice: .auto,
activeTools: ["xai_web_search", "xai_x_search", "xai_code_execution"]
)
```
Each builder's `name:` sets the tool name the calls and results carry, so
`activeTools` lines up with it. The agents' searches and code runs stream
back as provider-executed `.toolCall` / `.toolResult` parts.
## Models [#models]
* The `reasoning` parameter maps to `reasoning.effort` (`.minimal`
coerces to `low`, `.xhigh` to `high`).
* `grok-4.20` date-stamped `-reasoning` / `-non-reasoning` variants have
behavior baked into the model id, so the parameter is not sent for
them.
### Current lineup [#current-lineup]
As of July 2026, newest first:
| Model | Notes |
| --------------------------------------------------------- | --------------------------------------------- |
| `grok-4.5` | Newest flagship (July 2026) |
| `grok-4.3` | April 2026 |
| `grok-4.20-reasoning` / `-non-reasoning` / `-multi-agent` | Fixed-behavior variants (plus `-beta` builds) |
| `grok-4.1-fast-reasoning` / `-non-reasoning` | Fast tier |
| `grok-voice-think-fast-1.0` | Realtime voice |
| `grok-imagine-video-1.5`, `grok-imagine-image` | Video and image generation |
| `grok-tts`, `grok-stt` | Speech in and out |
## Deferred and compaction [#deferred-and-compaction]
`XaiModel.chat(...)` can run a completion in the background: `deferred`
submits the request and polls `/v1/chat/deferred-completion/{id}` until it
lands.
```swift
let model = XaiModel.chat("grok-4.5")
let done = try await model.submitDeferredCompletion(
LanguageModelRequest(messages: [.user("Summarize the news.")], maxOutputTokens: 1024)
)
done.text // final assistant text
done.usage // token counts
```
On the Responses API, `compactResponse(previousResponseID:)` compacts a
stored response's context and returns the new response, and
`retrieveResponse(_:)` / `deleteResponse(_:)` read back or remove a stored
response by id.
## Beyond text [#beyond-text]
| Surface | Entry point |
| -------------------------------------------- | ---------------------------------------------------------------------------------------- |
| [Image generation](/docs/image-generation) | `XaiImageModel("grok-imagine-image")` — generations and edits |
| [Speech generation](/docs/speech-generation) | `XaiSpeechModel("grok-tts")` |
| [Transcription](/docs/transcription) | `XaiTranscriptionModel("grok-stt")` |
| [Video generation](/docs/video-generation) | `XaiVideoModel("grok-imagine-video-1.5")` — `generateVideos`, `editVideo`, `extendVideo` |
| [Realtime voice](/docs/realtime) | `XaiRealtimeModel("grok-voice-latest")` |
## Platform APIs [#platform-apis]
Typed clients wrap the rest of the REST surface:
* `XaiFilesClient`: `upload` (with `expiresAfter`), `list` (paging and
`filter`), `get`, `update`, `download`, `delete` on `/v1/files`.
* `XaiBatchClient`: `create`, `get`, `list`, `requests`, `addRequests`,
`results`, `cancel` on `/v1/batches`.
* `XaiModelsClient`: `/v1/models` plus the richer `language-models`,
`image-generation-models`, and `video-generation-models` catalogs, which
add modalities, pricing, fingerprints, and aliases.
* `XaiPlatformClient`: `apiKeyInfo()`, `tokenizeText(_:model:)`, SIP phone
numbers (`/v2/phone-numbers`), in-call `referCall` / `hangUpCall`, and the
`tts/voices` and `custom-voices` catalogs.
* `XaiCollectionsClient`: collection and document management plus search.
### Collections span two services [#collections-span-two-services]
Collection *management* lives on `https://management-api.x.ai/v1` and needs a
**Management API key**; only search runs on `https://api.x.ai/v1` with the
usual `XAI_API_KEY`. The client holds both and routes each call itself:
```swift
let collections = XaiCollectionsClient() // XAI_MANAGEMENT_API_KEY + XAI_API_KEY
let id = try await collections.create(name: "Filings", description: "SEC")
try await collections.addDocument(collectionID: id, fileID: uploaded.id)
let hits = try await collections.search(
query: "revenue guidance",
source: ["collection_ids": .array([.string(id)])]
)
```
If `managementAPIKey` is omitted it falls back to `apiKey`, which works only
when one key carries both scopes. `list`, `listDocuments`, `document`,
`documents` (batch get), `update`, `regenerateIndices`, and `removeDocument`
round out the surface, and every management call accepts `teamID:`.
Files, Batch, and deferred completions all store data server-side, so
Zero Data Retention teams get a 403/400 from them — those accounts are
limited to the streaming chat and Responses paths.
Two documented endpoints are not wrapped: chunked upload
(`/v1/files:initialize` + `:uploadChunks`) and the request body for
`PUT /v1/files/{id}`, because xAI's reference lists the routes without
payload fields. `XaiFilesClient.update(_:body:)` takes a raw `JSONValue` so
you can send the documented shape once it is published.
# Foundations (/docs/foundations)
The rest of the documentation shows you how to do things. This section explains
what those things are.
If you've shipped an LLM feature before, skim it. You know most of this already.
If you haven't, these five pages are the fastest way to stop guessing at why a
call behaved the way it did.
# Models and tokens (/docs/foundations/models-and-tokens)
Models don't read characters or words. They read tokens, which are chunks of a
few characters each. English averages roughly four characters per token, so a
thousand words comes to about 1,300 tokens. Code and non-Latin scripts run
higher.
Tokens matter because everything is priced, limited, and measured in them.
## The context window [#the-context-window]
A model can only consider so many tokens at once. That ceiling is its context
window, and it covers all of it: system instructions, the entire message
history, tool definitions, tool results, and the answer being written.
Two things about this catch people out.
The first is how fast it fills. Every step of
[the loop](/docs/foundations/the-loop) resends the whole conversation, so a tool
that returns a 40 KB JSON blob has just spent perhaps 10,000 tokens of your
window. It stays spent for every step after that.
The second is what happens when you exceed it. The request is rejected. The
model does not quietly forget the oldest messages, so deciding what to drop is
your job, which is what [context management](/docs/context-management) is for.
Every model in this SDK reports its own window through `contextWindow`, so you
can check before you send rather than after you fail.
## Output limits are separate [#output-limits-are-separate]
`maxOutputTokens` caps the answer, not the conversation. It's a different limit
from the context window, and hitting it truncates mid-sentence instead of
erroring. If answers keep stopping abruptly, start here.
## Why the same prompt gives different answers [#why-the-same-prompt-gives-different-answers]
At each position the model produces a probability distribution over possible
next tokens, then samples from it. That sampling is deliberate. It's what makes
output read as fluent rather than robotic. It also means identical inputs can
give different outputs.
`temperature` controls how much the distribution gets flattened before sampling.
Low values concentrate on the likeliest tokens and give repetitive, predictable
text. High values spread the probability out and give varied, sometimes
incoherent text. There's no correct setting. Extraction and classification want
it low; brainstorming wants it higher.
`topP` and `topK` are alternative ways to cut off the tail of unlikely tokens.
Reach for one of them or for temperature, not all three at once.
`seed`, where a provider supports it, makes sampling reproducible. Same inputs
and same seed, same output. Useful in tests, not guaranteed everywhere.
## What this doesn't fix [#what-this-doesnt-fix]
Turning temperature to zero doesn't make a model accurate. It makes it
consistent. A model that confidently invents an API will now invent the same API
every time. Accuracy comes from giving it the right context and checking what
comes back, not from sampling settings.
## Next [#next]
* [Context management](/docs/context-management) covers what to do as a conversation grows
* [Providers](/docs/providers) lists the models available and what each supports
* [Context window exceeded](/docs/troubleshooting/context-window-exceeded) is what the failure looks like
# Prompts and messages (/docs/foundations/prompts-and-messages)
The model has no memory. Each request is answered on its own, from nothing but
what you send. A conversation that appears to remember earlier turns does so
because you resent them.
That one fact explains most of how prompts are shaped.
## Prompt or messages [#prompt-or-messages]
A one-shot question needs no history:
```swift
let result = try await generateText(
model: model,
system: "Answer in one sentence.",
prompt: "What is the tallest mountain?"
)
```
Anything conversational needs the list:
```swift
let result = try await generateText(model: model, messages: history)
```
`prompt` is shorthand for a single user message. Use whichever matches what you
have. You can't pass both.
## Roles [#roles]
Every message carries a role, and the model treats them differently.
A system message sets standing behaviour: who the assistant is, what it must not
do, what format to answer in. It carries more weight than anything a user says,
so put your rules here rather than in the last user message, where they compete
with whatever else is going on.
A user message is input from the person. An assistant message is what the model
produced, and you send those back so it can see what it already said. A tool
message carries the result of a tool the model asked for, and you rarely write
one yourself since [the loop](/docs/foundations/the-loop) produces them.
## Messages are made of parts [#messages-are-made-of-parts]
A message isn't a string. It's a role plus a list of content parts, which is
what lets one turn hold a sentence and an image, or an assistant turn hold both
its reasoning and the tool calls it settled on.
Text and images are the parts you'll construct yourself. Tool calls, tool
results, and reasoning get produced for you.
## Calls and results must pair [#calls-and-results-must-pair]
Every tool call in an assistant message needs a matching tool result before that
conversation can go back to the model. Providers reject histories where a call
is left dangling.
This is easy to break by accident. Trimming an old message, persisting a
conversation mid-flight, or a client-side tool whose result never came back will
all do it. If you see
[`missingToolResults`](/docs/troubleshooting/missing-tool-results), a pair got
split.
## Longer isn't better [#longer-isnt-better]
A prompt with every edge case spelled out ends up competing with itself for the
model's attention. Say what the output should look like, give one example if the
shape is unusual, and stop.
## Next [#next]
* [Messages](/docs/messages) covers building and persisting them in code
* [Tool calling](/docs/foundations/tool-calling) covers where tool messages come from
* [Models and tokens](/docs/foundations/models-and-tokens) covers what all this history costs
# Streaming (/docs/foundations/streaming)
Models generate one token at a time. A non-streaming call waits for the last one
before it returns anything. A streaming call hands you each piece as it arrives.
Total time is the same either way. What changes is when the first word shows up:
a second or two instead of fifteen. For anything a person is watching, that gap
is the whole experience.
## When not to stream [#when-not-to-stream]
Streaming costs you some simplicity, so skip it when nobody is waiting. A
background job, a server route that returns JSON, one step inside a larger
pipeline, anything whose output you parse rather than display. Use
`generateText` for those, and `streamText` when someone is watching text appear.
## A stream isn't only text [#a-stream-isnt-only-text]
This is the part that surprises people. Text deltas are one kind of event among
several, and in a run with tools they may not even be the first thing you see.
A stream can carry fragments of the answer, the model's reasoning on providers
that expose it, tool calls with their arguments streamed in as they're decided,
tool results from your own code, step boundaries as one round trip ends and
another begins, and finally token counts and a reason generation stopped.
If you render only text deltas, a run that spends eight seconds calling tools
looks frozen. Showing tool activity isn't decoration. It's what separates
"working" from "broken" in the user's head.
## Deltas are fragments [#deltas-are-fragments]
A delta might be a few characters, part of a word, or a whole sentence. Never
assume one delta is one token or one word. Append them and render the
accumulated string.
## The stream can fail partway [#the-stream-can-fail-partway]
An ordinary call either returns or throws. A stream can hand you half an answer
and then fail: connection dropped, provider error, timeout.
Partial output is a real state, so design for it. Keep what arrived, show that
it ended early, and make retrying possible. And since a stalled stream stays
open while producing nothing, a total timeout won't catch it on its own. You
want a separate limit on the gap between chunks.
## Cancellation is a feature [#cancellation-is-a-feature]
Someone reading a wrong answer wants to stop it, and a view nobody is looking at
shouldn't keep billing tokens. Streams cancel through Swift's normal task
cancellation. Wire it to a stop button and to view teardown.
## Next [#next]
* [Generating text](/docs/generating-text) covers `streamText` in code
* [Chat UI](/docs/chat-ui) is a SwiftUI surface that handles this for you
* [Streaming protocol](/docs/streaming-protocol) covers the wire format
* [Stream ends early](/docs/troubleshooting/stream-ends-early) covers when it stops short
# The loop (/docs/foundations/the-loop)
A language model does one thing. Given some text, it produces more text. It
cannot search the web, read your database, or send an email.
Everything that looks like an AI doing something is this loop wrapped around
that one ability.
## One turn [#one-turn]
Without tools, a generation is a single round trip. You send messages and a list
of tools the model may use, the model replies with text, and you're done.
That's `generateText` with no tools. One request, one answer.
## More than one turn [#more-than-one-turn]
With tools, the reply might not be text. The model can instead say that it wants
`get_weather` called with `{"city": "Oslo"}`. That isn't a function call. It's a
message asking for one.
So the loop keeps going:
1. You send messages and tools.
2. The model replies with a tool call.
3. The SDK runs your tool and appends the result to the conversation.
4. The whole conversation, now including the call and its result, goes back.
5. The model replies, with text this time, or with another tool call.
6. Repeat from step 3 until it answers in text or a stop condition fires.
Each pass through steps 2 to 5 is a step. A run that searched twice before
answering took three steps.
## Why there's a limit [#why-theres-a-limit]
Nothing in this loop guarantees the model stops asking for tools. A confused
model can call the same tool forever, and every step costs a request.
`maxSteps` is the ceiling that stops that. Most runs finish well under it. If
yours regularly hits it, look at your tool descriptions before you raise the
number.
`stopWhen` gives you finer control. End after a particular tool has been called,
or on any condition you can write over the steps so far.
## Why this matters [#why-this-matters]
Most surprising behaviour makes sense once you can see the loop.
A tool that "didn't run" usually means the model never asked for it, which is a
description problem rather than an execution one. A token bill larger than you
expected usually means the loop took several steps, and every step resends the
whole conversation including previous tool results. A conversation that grows
alarmingly fast is usually one where tool outputs are long, since those are
messages too. And a run that stopped early either hit a stop condition or ran
out of steps.
## Next [#next]
* [Tool calling](/docs/foundations/tool-calling) covers what the model emits and who runs it
* [Generating text](/docs/generating-text) is the loop in code
* [Agents](/docs/agents) is the loop with a fixed setup
# Tool calling (/docs/foundations/tool-calling)
The most common misunderstanding about tools is that the model calls them. It
doesn't. It can't execute anything.
What actually happens is that you describe your tools, the model replies with a
message saying it would like `get_weather` called with these arguments, and the
SDK runs the function and hands the result back. The model only ever reads and
writes text.
## What a tool is, to the model [#what-a-tool-is-to-the-model]
A name, a description in plain English, and a schema for its arguments. That's
all it sees. It never sees your implementation, and it has no idea whether the
call succeeded until it reads the result.
## The description is the interface [#the-description-is-the-interface]
This is the part that decides whether a tool works, and it's the part most
people write last.
The model chooses tools by reading descriptions. If two tools sound alike, it
picks wrongly. If a description is vague, it doesn't pick the tool at all, which
is the single most common reason a tool "never runs."
Write it for a competent colleague who has never seen your codebase. Say what
the tool does, when to use it, and when not to:
```swift
// Too vague. The model can't tell when this applies.
description: "Gets data."
// Specific enough to choose correctly.
description: """
Current weather for a city. Use for questions about conditions right now. \
Does not do forecasts or historical weather.
"""
```
Argument descriptions matter for the same reason. A parameter called `id` with
no description will be filled with a guess.
## Arguments aren't validated by the model [#arguments-arent-validated-by-the-model]
The model produces JSON that it believes matches your schema. It's usually right
and sometimes not: a missing field, a string where a number belongs, an enum
value you never defined.
The SDK validates against the schema before your code runs, so your function
either gets well-formed input or the call fails cleanly. What no schema can
check is whether the values make sense. A city name that's valid JSON and also
fictional will sail straight through.
## Failure is information [#failure-is-information]
A tool that throws doesn't end the run. The error goes back to the model as the
tool's result, and it can correct course, fix an argument, try something else,
or explain the problem to the user.
Which makes error messages a kind of prompt. `"City not found. Try a full name
like 'Oslo, Norway'."` gets you a retry. `"Error 4"` gets you a shrug.
## Some calls shouldn't be automatic [#some-calls-shouldnt-be-automatic]
The loop runs tools the moment the model asks. That's fine for a search. It's
less fine for anything that spends money, sends a message, or deletes something.
Approvals put a person between the request and the execution. The loop pauses,
surfaces the pending call, and resumes once you answer. Decide this per tool
based on what the call costs if it's wrong, not on how often the model gets it
right.
## Next [#next]
* [Tools](/docs/tools) covers writing them in code
* [The loop](/docs/foundations/the-loop) covers where tool calls fit
* [Timeouts and approvals](/docs/timeouts-and-approvals) covers gating risky calls
* [Tool never executes](/docs/troubleshooting/tool-never-executes) covers what to do when it isn't chosen
# An agent with tools (/docs/guides/agent-with-tools)
Covers `Examples/Features/05-Tools.swift`, `10-Agent.swift`, and
`20-SubagentsAndContext.swift`. You end up with a reusable agent that
calls your code and knows when to stop.
### Write a tool [#write-a-tool]
A tool is a name, a description the model reads, a JSON Schema for the
arguments, and a closure. The loop handles the rest: run, feed back,
continue.
```swift
let getTime = Tool(
name: "current_time",
description: "Returns the current time in a timezone.",
parameters: ["type": "object",
"properties": ["tz": ["type": "string"]],
"required": ["tz"]]
) { args in
.string(Date().formatted())
}
```
Prefer typed arguments? Decode them:
```swift
struct WeatherArgs: Decodable { var city: String }
let weather = Tool.typed(
name: "weather",
description: "Current weather for a city",
parameters: ["type": "object",
"properties": ["city": ["type": "string"]],
"required": ["city"]]
) { (args: WeatherArgs) in
try await OpenMeteoWeather.current(city: args.city)
}
```
`OpenMeteoWeather.current` first resolves the city with Open-Meteo's
geocoding endpoint, then requests current temperature, apparent temperature,
humidity, weather code, and wind speed. The complete, compile-checked client
is in `Examples/Support.swift`; Open-Meteo requires no API key for this use.
The structured result includes `source` and `sourceURL` so your UI can retain
the required attribution. See the official [forecast](https://open-meteo.com/en/docs)
and [geocoding](https://open-meteo.com/en/docs/geocoding-api) documentation.
### Bound the loop [#bound-the-loop]
Without a limit the model could ping-pong forever. `stopWhen` caps it:
```swift
let result = try await generateText(
model: AnthropicModel("claude-opus-4-8"),
prompt: "What time is it in Asia/Kolkata?",
tools: [getTime],
stopWhen: [stepCountIs(4)],
onStepFinish: { step in print("step:", step.finishReason) }
)
print(result.text)
print("steps: \(result.stepCount)")
```
### Make it reusable [#make-it-reusable]
`Agent` captures the model, instructions, tools, and settings in one
value you call again and again:
```swift
let assistant = Agent(
model: AnthropicModel("claude-sonnet-5"),
instructions: "You are a terse weather assistant.",
tools: [weather],
stopWhen: [isStepCount(4)]
)
let result = try await assistant.generate(prompt: "Weather in Mumbai?")
for try await delta in assistant.stream(prompt: "And in Tokyo?").textStream {
print(delta, terminator: "")
}
```
An `Agent` is also a chat transport:
`ChatSession(transport: assistant)` gives you a chat UI over it with no
server.
### Pass request context without polluting the prompt [#pass-request-context-without-polluting-the-prompt]
User ids and session handles don't belong in the prompt. `toolsContext`
delivers them straight to the tool:
```swift
let orders = Tool(
name: "list_orders",
description: "List recent orders for the signed-in user.",
parameters: ["type": "object"]
) { _, options in
let userID = options.context?["userID"]?.stringValue ?? "anonymous"
return .string("Orders for \(userID): #1001, #1002")
}
let result = try await generateText(
model: model,
prompt: "What did I order recently?",
tools: [orders],
toolsContext: ["list_orders": ["userID": .string("user-7")]]
)
```
### Delegate to subagents [#delegate-to-subagents]
An orchestrator stays small by handing focused work to specialists.
`asTool` turns any agent into a tool:
```swift
let researcher = Agent(
model: model,
instructions: "You research questions and answer with dense facts."
)
let writer = Agent(
model: model,
instructions: "You turn notes into friendly prose."
)
let orchestrator = Agent(
model: model,
instructions: "Plan the work, delegate to specialists, then combine.",
tools: [
researcher.asTool(name: "researcher", description: "Delegate research."),
writer.asTool(name: "writer", description: "Delegate drafting.")
]
)
```
## Final code [#final-code]
```swift title="WeatherAgent.swift"
import AI
import Foundation
struct WeatherArgs: Decodable { var city: String }
let weather = Tool.typed(
name: "weather",
description: "Current weather for a city",
parameters: ["type": "object",
"properties": ["city": ["type": "string"]],
"required": ["city"]]
) { (args: WeatherArgs) in
try await OpenMeteoWeather.current(city: args.city)
}
let assistant = Agent(
model: AnthropicModel("claude-sonnet-5"),
instructions: "You are a terse weather assistant.",
tools: [weather],
stopWhen: [isStepCount(4)]
)
func run() async throws {
let result = try await assistant.generate(prompt: "Weather in Mumbai?")
print(result.text)
}
```
One more trick from the examples: `prepareStep` swaps to a cheaper model
once the expensive one has done the thinking:
```swift
prepareStep: { context in
context.stepNumber >= 3 ? PrepareStepResult(model: cheaperModel) : nil
}
```
# Human-in-the-loop approvals (/docs/guides/approvals)
Covers `Examples/Features/16-ToolApprovals.swift`. You end up with a delete tool
that never runs without a confirmation, in both plain calls and chat UIs.
### Mark the tool [#mark-the-tool]
`needsApproval: true` pauses the loop before every call. For
argument-dependent decisions, pass a closure instead.
```swift
let deleteFile = Tool(
name: "deleteFile",
description: "Removes a file from disk",
parameters: ["type": "object",
"properties": ["path": ["type": "string"]],
"required": ["path"]],
needsApproval: true
) { args in
.string("deleted \(args["path"]?.stringValue ?? "?")")
}
```
### Catch the pause [#catch-the-pause]
The loop stops with an approval request instead of executing. Nothing
was deleted yet:
```swift
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: "Clean up /tmp/scratch.txt",
tools: [deleteFile]
)
for request in result.steps.last?.approvalRequests ?? [] {
print("wants to run \(request.call.name) with \(request.call.arguments)")
}
```
### Resume with the decision [#resume-with-the-decision]
Append the user's answer to the history and call again. Approved calls
execute; denied ones surface to the model as denials:
```swift
var messages = result.messages
messages.append(Message(role: .tool, content: [
.toolApprovalResponse(ToolApprovalResponse(
approvalID: request.approvalID,
toolCallID: request.call.id,
approved: true
))
]))
let resumed = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
messages: messages,
tools: [deleteFile]
)
print(resumed.text)
```
### In a chat UI it's one call [#in-a-chat-ui-its-one-call]
`ChatSession` surfaces the pause as a tool part in `approvalRequested`
state. Show your confirmation UI, then answer:
```swift
for part in chat.messages.last?.parts ?? [] {
guard case .tool(let tool) = part,
tool.state == .approvalRequested,
let approval = tool.approval else { continue }
chat.addToolApprovalResponse(approvalID: approval.id, approved: true)
}
```
## Final code [#final-code]
```swift title="ApprovedDelete.swift"
import AI
let deleteFile = Tool(
name: "deleteFile",
description: "Removes a file from disk",
parameters: ["type": "object",
"properties": ["path": ["type": "string"]],
"required": ["path"]],
needsApproval: true
) { args in
.string("deleted \(args["path"]?.stringValue ?? "?")")
}
func cleanUp() async throws {
let model = AnthropicModel("claude-sonnet-5")
let result = try await generateText(
model: model,
prompt: "Clean up /tmp/scratch.txt",
tools: [deleteFile]
)
for request in result.steps.last?.approvalRequests ?? [] {
let userSaidYes = await confirmWithUser(request.call)
var messages = result.messages
messages.append(Message(role: .tool, content: [
.toolApprovalResponse(ToolApprovalResponse(
approvalID: request.approvalID,
toolCallID: request.call.id,
approved: userSaidYes
))
]))
let resumed = try await generateText(
model: model,
messages: messages,
tools: [deleteFile]
)
print(resumed.text)
}
}
```
Client-side tools work the same way with one difference: there's no
executor at all, so the app supplies the result instead of a yes or no:
```swift
chat.addToolResult(toolCallID: tool.toolCallID, result: ["photoID": "IMG_0042"])
```
# A chat screen (/docs/guides/chat-screen)
Covers `Examples/Features/09-ChatSession.swift` and `19-SessionHooks.swift`. You
end up with a streaming chat view that works against a web chat route or
an in-process model.
### Have a chat route (or skip this step) [#have-a-chat-route-or-skip-this-step]
The URL you'll point the app at is an AI SDK chat route: a POST endpoint
on your server that takes `{messages}` and streams UI message chunks
back. If your web app already uses `useChat`, you have one — reuse it
as-is. If not, this is the entire route (Next.js shown; any framework
the AI SDK supports works):
```ts title="app/api/chat/route.ts"
import {
streamText, UIMessage, convertToModelMessages,
createUIMessageStreamResponse, toUIMessageStream,
} from 'ai';
export async function POST(req: Request) {
const { messages }: { messages: UIMessage[] } = await req.json();
const result = streamText({
model: 'anthropic/claude-sonnet-5',
messages: await convertToModelMessages(messages),
});
return createUIMessageStreamResponse({
stream: toUIMessageStream({ stream: result.stream }),
});
}
```
Deploy it anywhere; its URL is what you pass below. Prefer serving from
Swift? The [streaming protocol page](/docs/streaming-protocol) builds
the same route with this library's server helpers. No server at all?
Skip ahead — the local transport needs none.
### Create the session [#create-the-session]
`ChatSession` is `@Observable`; keep it in `@State`. Point the transport
at that route:
```swift
@State private var chat = ChatSession(transport: HTTPChatTransport(
api: URL(string: "https://your-app.vercel.app/api/chat")!, // the route from step 1
headers: ["Authorization": "Bearer token"],
body: ["sessionId": "abc123"] // extra fields, merged into every request
))
```
No server? Use a local transport and skip the network:
```swift
@State private var chat = ChatSession(transport: LocalChatTransport(
model: AnthropicModel("claude-sonnet-5"),
system: "You are a helpful assistant."
))
```
### Render the messages [#render-the-messages]
A message is typed parts, not just a string. Text is the common case:
```swift
ScrollView {
ForEach(chat.messages) { message in
ForEach(Array(message.parts.enumerated()), id: \.offset) { _, part in
if case .text(let text) = part {
Text(text.text)
.frame(maxWidth: .infinity,
alignment: message.role == .user ? .trailing : .leading)
}
}
}
}
```
Parts update token by token as the stream arrives; SwiftUI re-renders on
its own.
### Send, and show status [#send-and-show-status]
```swift
TextField("Message", text: $input)
.onSubmit {
chat.send(input)
input = ""
}
if chat.status == .streaming { ProgressView() }
```
`chat.stop()` cancels mid-stream, `chat.regenerate()` redoes the last
answer, and `chat.resumeStream()` picks up a response that kept going
while the app was backgrounded.
## Final code [#final-code]
```swift title="ChatView.swift"
import AI
import SwiftUI
struct ChatView: View {
@State private var chat = ChatSession(transport: HTTPChatTransport(
api: URL(string: "https://your-app.vercel.app/api/chat")!
))
@State private var input = ""
var body: some View {
VStack {
ScrollView {
ForEach(chat.messages) { message in
ForEach(Array(message.parts.enumerated()), id: \.offset) { _, part in
if case .text(let text) = part {
Text(text.text)
.padding(10)
.frame(maxWidth: .infinity,
alignment: message.role == .user ? .trailing : .leading)
}
}
}
}
HStack {
TextField("Message", text: $input)
.onSubmit(send)
Button("Send", action: send)
.disabled(chat.status == .streaming)
}
.padding()
}
}
private func send() {
chat.send(input)
input = ""
}
}
```
## The two smaller sessions [#the-two-smaller-sessions]
Single-turn completion and streamed objects follow the same pattern:
```swift title="SessionHooks.swift"
// Completion: one prompt, streamed text state.
@State var completion = CompletionSession(
transport: HTTPCompletionTransport(api: URL(string: "https://your-app.com/api/completion")!)
)
completion.complete("Write a tagline for a coffee shop")
// completion.completion grows as tokens arrive; completion.isLoading drives spinners
// Objects: streamed structured output with partial-JSON repair.
@State var session = ObjectSession(
model: OpenAIModel("gpt-5.6-sol"),
schema: Schema.object(["title": .string(), "body": .string()])
)
session.submit("A notification about a delayed flight")
// session.object is the partial JSON while streaming; then:
let note = session.decoded(Notification.self)
```
# Drive a computer-use agent (/docs/guides/computer-use)
Covers `Examples/Features/24-ComputerUse.swift`. Computer-use tools are
*client-executed*: the model asks for an action (click at 100,200; type
"hello"; take a screenshot), your code performs it against a real or virtual
display, and you hand back a screenshot so the model can decide what to do
next. OpenAI and Anthropic both support this; the library wires the round-trip.
### Offer the tool [#offer-the-tool]
Add the provider's computer tool with your display size. Nothing runs yet;
this just tells the model the capability exists.
```swift
let model = OpenAIModel("gpt-5.6-sol")
let tools = [OpenAIModel.Tools.computerUse(displayWidth: 1280, displayHeight: 800)]
```
On Anthropic it's `AnthropicModel.Tools.computer(displayWidthPx:displayHeightPx:)`
(plus `bash()` and `textEditor()` if you want them); the required beta header
is set for you.
### Read the requested action [#read-the-requested-action]
Each turn returns a `computer_use_preview` tool call whose `action` argument
describes what to do: `click`, `type`, `keypress`, `scroll`, `screenshot`.
```swift
let result = try await generateText(model: model, messages: messages, tools: tools)
guard let call = result.toolCalls.first(where: { $0.name == "computer_use_preview" }) else {
print(result.text) // the model is done
return
}
let action = call.arguments["action"] ?? .object([:])
```
### Execute, then return a screenshot [#execute-then-return-a-screenshot]
Perform the action against your automation backend, capture the screen, and
send it back as **image content** on the tool result. The library maps that to
OpenAI's `computer_call_output` (or an Anthropic `tool_result` image block).
```swift
let png = try await perform(action) // your automation
messages.append(Message(role: .assistant, content: [.toolCall(call)]))
messages.append(Message(role: .tool, content: [.toolResult(ToolResult(
toolCallID: call.id, name: call.name, output: .null,
content: [.image(ImageContent(data: png, mediaType: "image/png"))]
))]))
```
### Loop until done [#loop-until-done]
Repeat until the model stops asking for actions. Cap the iterations so a stuck
run can't loop forever.
```swift
for _ in 0..<20 {
// generate → read action → execute → append screenshot
// break when there's no computer_use_preview call
}
```
The screenshot round-trip is the whole trick, and it rides the same
[multimodal tool results](/docs/tools#multimodal-tool-results) any tool can use.
# Use custom endpoints and gateways (/docs/guides/custom-endpoints)
Covers `Examples/Features/04-ProviderSwap.swift` and
`23-WorkflowGuides.swift`. Use this when an endpoint speaks OpenAI Chat
Completions but does not have a dedicated model type in the SDK.
### Prefer a first-class provider when one exists [#prefer-a-first-class-provider-when-one-exists]
Named provider types carry the correct endpoint and environment variable:
```swift
let hosted: any LanguageModel = TogetherAIModel("MiniMaxAI/MiniMax-M3")
let local: any LanguageModel = OllamaModel("gemma4")
```
Use `OpenAICompatibleProvider` for your own gateway, proxy, vLLM server, or
another compatible service. Do not wrap a named provider merely because its
wire format happens to be OpenAI-compatible.
### Configure the gateway once [#configure-the-gateway-once]
```swift
let gateway = OpenAICompatibleProvider(
name: "company-gateway",
baseURL: URL(string: "https://llm.example.com/v1")!,
apiKey: ProcessInfo.processInfo.environment["GATEWAY_API_KEY"],
headers: ["x-team": "ios"],
queryParams: ["api-version": "2026-01-01"]
)
```
The provider name appears in telemetry and result metadata. Headers and query
parameters are attached to every model request.
### Create language and embedding models [#create-language-and-embedding-models]
Pass model ids exactly as your server exposes them:
```swift
let chat = gateway("openai/gpt-oss-20b")
let embeddings = gateway.textEmbeddingModel("BAAI/bge-large-en-v1.5")
let answer = try await generateText(
model: chat,
prompt: "Explain Swift actors in three sentences."
)
let vector = try await embed(model: embeddings, value: answer.text)
```
Tools, structured output, vision, and reasoning are encoded on the compatible
wire. The selected upstream model still decides which features it actually
supports.
### Select models by string [#select-models-by-string]
`ProviderRegistry` is useful when configuration, feature flags, or a server
response chooses the model:
```swift
let registry = ProviderRegistry(providers: [
"gateway": .init(
languageModel: { gateway($0) },
embeddingModel: { gateway.textEmbeddingModel($0) }
),
"ollama": .init { OllamaModel($0) }
])
let model = try registry.languageModel("gateway:openai/gpt-oss-20b")
```
The separator defaults to `:` and can be changed when model ids already use
that character.
### Keep credentials off untrusted clients [#keep-credentials-off-untrusted-clients]
If the endpoint uses a privileged provider key, call it from your backend or
issue a short-lived, scoped gateway token. Custom headers are convenient for
routing and tenant context, but they are not a secret store.
## Final code [#final-code]
```swift title="CompanyGateway.swift"
import AI
import Foundation
let companyGateway = OpenAICompatibleProvider(
name: "company-gateway",
baseURL: URL(string: "https://llm.example.com/v1")!,
apiKey: ProcessInfo.processInfo.environment["GATEWAY_API_KEY"],
headers: ["x-team": "ios"],
queryParams: ["api-version": "2026-01-01"]
)
func askGateway(_ prompt: String) async throws -> String {
try await generateText(
model: companyGateway("openai/gpt-oss-20b"),
prompt: prompt
).text
}
```
See [Custom OpenAI-compatible endpoints](/docs/providers/openai-compatible)
for the reference API and [Compatibility](/docs/providers/compatibility) for
the feature matrix.
# Your first request (/docs/guides/first-request)
Covers `Examples/Features/01-GenerateText.swift`, `02-StreamText.swift`, and
`04-ProviderSwap.swift`.
### Add the package [#add-the-package]
File, Add Package Dependencies in Xcode, or in `Package.swift`:
```swift title="Package.swift"
.package(url: "https://github.com/zaidmukaddam/swift-ai-sdk", branch: "main")
```
Add the `AI` product to your target and `import AI` where you use it.
### Pick a model [#pick-a-model]
Any provider works; the key comes from the environment or an `apiKey:`
argument. For zero setup, use Ollama on your Mac:
```swift
let model = AnthropicModel("claude-sonnet-5") // ANTHROPIC_API_KEY
// or, no key needed:
let local = OllamaModel("granite4.1:3b")
```
### Generate [#generate]
One call, whole answer:
```swift
let result = try await generateText(
model: model,
system: "You are concise.",
prompt: "Name three primary colors."
)
print(result.text)
print("tokens: \(result.usage.totalTokens)")
```
### Stream instead [#stream-instead]
Same call, tokens as they arrive. Nothing runs until you iterate, and
dropping the stream cancels the request:
```swift
let result = streamText(model: model, prompt: "Write a haiku about Swift.")
for try await delta in result.textStream {
print(delta, terminator: "")
}
```
`result.fullStream` carries everything else too: step boundaries, tool
calls, usage.
## Final code [#final-code]
```swift title="FirstRequest.swift"
import AI
func generateOnce() async throws {
let model = AnthropicModel("claude-sonnet-5")
let result = try await generateText(
model: model,
system: "You are concise.",
prompt: "Name three primary colors."
)
print(result.text)
print("tokens: \(result.usage.totalTokens)")
}
func streamIt() async throws {
let model = AnthropicModel("claude-sonnet-5")
let result = streamText(model: model, prompt: "Write a haiku about Swift.")
for try await delta in result.textStream {
print(delta, terminator: "")
}
}
func watchTheLoop() async throws {
let model = AnthropicModel("claude-sonnet-5")
for try await part in streamText(model: model, prompt: "Hi!").fullStream {
switch part {
case .textDelta(let delta): print(delta, terminator: "")
case .finishStep(let step): print("\n[step done: \(step.finishReason)]")
case .finish(let reason, let usage): print("[\(reason), \(usage.totalTokens) tokens]")
default: break
}
}
}
```
## Swapping providers [#swapping-providers]
The same code runs against every provider. Change the model value, keep
everything else:
```swift title="ProviderSwap.swift"
let models: [any LanguageModel] = [
AnthropicModel("claude-opus-4-8"), // ANTHROPIC_API_KEY
OpenAIModel("gpt-5.6-sol"), // OPENAI_API_KEY
XaiModel("grok-4.5"), // XAI_API_KEY
GroqModel("llama-3.3-70b-versatile"), // GROQ_API_KEY
GoogleModel("gemini-3.5-flash"), // GOOGLE_GENERATIVE_AI_API_KEY
OllamaModel("gemma4") // local, no key
]
for model in models.prefix(3) {
let result = try await generateText(model: model, prompt: "Say hi in 3 words.")
print("\(model.provider)/\(model.modelID): \(result.text)")
}
```
A self-hosted gateway is four lines:
```swift
let gateway = OpenAICompatibleProvider(
name: "my-gateway",
baseURL: URL(string: "https://llm.your-company.com/v1")!, // your gateway's URL
apiKey: ProcessInfo.processInfo.environment["GATEWAY_KEY"],
headers: ["x-team": "ios"]
)
let model = gateway("my-model")
```
# Generate images and video (/docs/guides/image-and-video-pipeline)
Covers `Examples/Features/12-ImageGeneration.swift`,
`14-VideoGeneration.swift`, and `23-WorkflowGuides.swift`. You end up with a
small asset pipeline that turns one prompt into a still image and an animated
clip.
### Generate the still [#generate-the-still]
```swift
let image = try await generateImage(
model: OpenAIImageModel("gpt-image-2"),
prompt: "A paper boat drifting down a rainy street, cinematic",
size: "1024x1024"
)
let imageURL = outputDirectory.appendingPathComponent("paper-boat.png")
try image.image.write(to: imageURL)
```
`image.image` is the first result. Use `image.images` when requesting multiple
variants.
### Edit an existing image [#edit-an-existing-image]
Passing source images through `generateImage` switches providers that support
editing, such as OpenAI, to their edit flow:
```swift
let edited = try await generateImage(
model: OpenAIImageModel("gpt-image-2"),
prompt: "Make the scene nighttime and add warm window light",
images: [ImageContent(data: image.image, mediaType: "image/png")]
)
```
### Animate the still [#animate-the-still]
```swift
let video = try await generateVideo(
model: XaiVideoModel("grok-imagine-video-1.5"),
prompt: "The boat drifts forward while rain ripples across the street",
image: ImageContent(data: image.image, mediaType: "image/png"),
aspectRatio: "16:9",
duration: 6
)
```
Video providers render asynchronously. Their model types handle job polling,
so this call returns only when the provider reports a completed clip or an
error.
### Handle inline bytes and URLs [#handle-inline-bytes-and-urls]
Providers may return either form:
```swift
if let bytes = video.video {
try bytes.write(to: outputDirectory.appendingPathComponent("paper-boat.mp4"))
} else if let remoteURL = video.urls.first {
print("ready at", remoteURL)
}
```
Copy remote media into storage you control if provider URLs are temporary.
Avoid holding large generated files in memory longer than necessary.
### Swap media providers [#swap-media-providers]
The workflow functions stay unchanged when the model changes:
```swift
let stillModel: any ImageModel = FalImageModel("fal-ai/flux/schnell")
let videoModel: any VideoModel = LumaVideoModel("ray-2")
```
Size, aspect ratio, duration, and provider-native options still depend on the
selected model.
## Final code [#final-code]
```swift title="MediaPipeline.swift"
import AI
import Foundation
func createClip(in outputDirectory: URL) async throws -> GenerateVideoResult {
let image = try await generateImage(
model: OpenAIImageModel("gpt-image-2"),
prompt: "A paper boat drifting down a rainy street, cinematic",
size: "1024x1024"
)
let imageURL = outputDirectory.appendingPathComponent("paper-boat.png")
try image.image.write(to: imageURL)
return try await generateVideo(
model: XaiVideoModel("grok-imagine-video-1.5"),
prompt: "The boat drifts forward while rain ripples across the street",
image: ImageContent(data: image.image, mediaType: "image/png"),
aspectRatio: "16:9",
duration: 6
)
}
```
See [Image generation](/docs/image-generation) and
[Video generation](/docs/video-generation) for every provider and option.
# Guides (/docs/guides)
Each guide walks one build from empty file to working feature: what to
add, what to write, and the complete final code. They line up with the
compiled examples in the repository's `Examples/` folder, which build on
every CI run, so the code can't rot.
## The example files [#the-example-files]
Every numbered file in `Examples/Features/` maps to one or more guides:
| Example | Guide |
| ------------------------------------------------------- | ------------------------------------------------------------------ |
| 01 GenerateText, 02 StreamText, 04 ProviderSwap | [Your first request](/docs/guides/first-request) |
| 09 ChatSession, 19 SessionHooks | [A chat screen](/docs/guides/chat-screen) |
| 05 Tools, 10 Agent, 20 SubagentsAndContext | [An agent with tools](/docs/guides/agent-with-tools) |
| 06 GenerateObject, 07 StreamObject, 18 Schema | [Structured data](/docs/guides/structured-data) |
| 17 MultimodalAndSettings | [Vision and files](/docs/guides/vision-and-files) |
| 16 ToolApprovals | [Approvals](/docs/guides/approvals) |
| 22 Realtime | [A voice assistant](/docs/guides/voice-assistant) |
| 03 OnDeviceOrCloud | [Offline first](/docs/guides/offline-first) |
| 21 UIStreamsAndTesting, 11 Middleware | [Serving and testing](/docs/guides/server-and-testing) |
| 08 Embeddings, 12 Image, 13 Speech, 14 Video, 15 Rerank | [Media and embeddings](/docs/guides/media-and-embeddings) |
| 08 Embeddings, 15 Rerank | [Build semantic search](/docs/guides/semantic-search) |
| 13 SpeechAndTranscription | [Transcribe and summarize](/docs/guides/transcribe-and-summarize) |
| 12 ImageGeneration, 14 VideoGeneration | [Generate images and video](/docs/guides/image-and-video-pipeline) |
| 04 ProviderSwap | [Use custom endpoints and gateways](/docs/guides/custom-endpoints) |
| 11 Middleware | [Production reliability](/docs/guides/production-reliability) |
| 23 WorkflowGuides and provider tool examples | [Search and server tools](/docs/guides/search-and-server-tools) |
There are also two real apps in `Apps/`: a ChatGPT-style chat over local
Ollama, and a focused realtime voice app covering xAI, OpenAI, and Google.
Run them on the iOS simulator after `xcodegen generate`.
# Media and embeddings (/docs/guides/media-and-embeddings)
Covers `Examples/Features/08-Embeddings.swift`, `12-ImageGeneration.swift`,
`13-SpeechAndTranscription.swift`, `14-VideoGeneration.swift`, and
`15-Rerank.swift`. Quick hits, one call each. For API-level detail, see
[embeddings](/docs/embeddings), [reranking](/docs/reranking),
[image generation](/docs/image-generation),
[speech generation](/docs/speech-generation),
[transcription](/docs/transcription), and
[video generation](/docs/video-generation).
For complete workflows, continue with [semantic search](/docs/guides/semantic-search),
[transcribe and summarize](/docs/guides/transcribe-and-summarize), or
[generate images and video](/docs/guides/image-and-video-pipeline).
### Generate an image [#generate-an-image]
```swift
let result = try await generateImage(
model: OpenAIImageModel("gpt-image-2"),
prompt: "A watercolor fox in a snowy forest",
size: "1024x1024"
)
try result.image.write(to: URL(fileURLWithPath: "/tmp/fox.png"))
```
### Say it out loud, then transcribe it back [#say-it-out-loud-then-transcribe-it-back]
```swift
let speech = try await generateSpeech(
model: OpenAISpeechModel("gpt-4o-mini-tts"),
text: "The quick brown fox jumped over the lazy dog.",
voice: "alloy"
)
try speech.audio.write(to: URL(fileURLWithPath: "/tmp/hello.mp3"))
let transcript = try await transcribe(
model: OpenAITranscriptionModel("whisper-1"),
audio: try Data(contentsOf: URL(fileURLWithPath: "/tmp/hello.mp3")),
mediaType: "audio/mpeg"
)
print(transcript.text)
```
### Animate it [#animate-it]
```swift
let still = try Data(contentsOf: URL(fileURLWithPath: "/tmp/fox.png"))
let video = try await generateVideo(
model: XaiVideoModel("grok-imagine-video-1.5"),
prompt: "The fox blinks and snow falls gently",
image: ImageContent(data: still)
)
print(video.urls)
```
The provider polls the render job for you; the call returns when the
clip is ready.
### Search your own content [#search-your-own-content]
Embeddings turn text into vectors; cosine similarity ranks them:
```swift
let model = OpenAIEmbeddingModel("text-embedding-3-small")
func bestMatch(for query: String, in corpus: [String]) async throws -> String? {
let docs = try await embedMany(model: model, values: corpus)
let q = try await embed(model: model, value: query)
return zip(corpus, docs.embeddings)
.max { cosineSimilarity($0.1, q.embedding) < cosineSimilarity($1.1, q.embedding) }?
.0
}
```
### Rerank when you already have candidates [#rerank-when-you-already-have-candidates]
```swift
let result = try await rerank(
model: CohereRerankingModel("rerank-v4-fast"),
query: "What is the capital of the United States?",
documents: documents,
topN: 2
)
for ranked in result.rankedDocuments {
print(String(format: "%.3f", ranked.relevanceScore), ranked.document)
}
```
## Final code [#final-code]
```swift title="MediaPipeline.swift"
import AI
import Foundation
func foxPipeline() async throws {
// 1. A picture
let image = try await generateImage(
model: OpenAIImageModel("gpt-image-2"),
prompt: "A watercolor fox in a snowy forest",
size: "1024x1024"
)
let stillURL = URL(fileURLWithPath: "/tmp/fox.png")
try image.image.write(to: stillURL)
// 2. A voice
let speech = try await generateSpeech(
model: OpenAISpeechModel("gpt-4o-mini-tts"),
text: "A fox in the snow.",
voice: "alloy"
)
try speech.audio.write(to: URL(fileURLWithPath: "/tmp/fox.mp3"))
// 3. A film
let video = try await generateVideo(
model: XaiVideoModel("grok-imagine-video-1.5"),
prompt: "The fox blinks and snow falls gently",
image: ImageContent(data: try Data(contentsOf: stillURL))
)
print("done:", video.urls)
}
```
# Offline first (/docs/guides/offline-first)
Covers `Examples/Features/03-OnDeviceOrCloud.swift`. You end up with a feature
that answers instantly and privately on supported devices, and still
works everywhere else.
### Pick the model at runtime [#pick-the-model-at-runtime]
`FoundationModelsModel.orFallback` returns the on-device model when the
device can run it and your fallback when it can't:
```swift
let model: any LanguageModel
#if canImport(FoundationModels)
if #available(iOS 26.0, macOS 26.0, *) {
model = FoundationModelsModel.orFallback(AnthropicModel("claude-sonnet-5"))
} else {
model = AnthropicModel("claude-sonnet-5")
}
#else
model = AnthropicModel("claude-sonnet-5")
#endif
```
The `canImport` and availability checks keep the same file compiling on
older OS versions and non-Apple platforms.
### Use it like any other model [#use-it-like-any-other-model]
Everything downstream is identical for both branches: streaming, tools,
structured output, chat sessions.
```swift
let result = try await generateText(
model: model,
prompt: "Summarize: Swift concurrency in one line."
)
print("[\(result.text)] via \(model.provider)/\(model.modelID)")
```
The print shows you which one actually ran.
## Final code [#final-code]
```swift title="OnDeviceOrCloud.swift"
import AI
func summarize(_ text: String) async throws -> String {
let model: any LanguageModel
#if canImport(FoundationModels)
if #available(iOS 26.0, macOS 26.0, *) {
model = FoundationModelsModel.orFallback(AnthropicModel("claude-sonnet-5"))
} else {
model = AnthropicModel("claude-sonnet-5")
}
#else
model = AnthropicModel("claude-sonnet-5")
#endif
let result = try await generateText(
model: model,
prompt: "Summarize in one line: \(text)"
)
return result.text
}
```
A fully offline chat screen is the same idea plus a session:
```swift
let chat = ChatSession(transport: LocalChatTransport(
model: FoundationModelsModel(),
system: "You are a helpful assistant."
))
```
[On-device models](/docs/on-device) covers Private Cloud Compute, guided
generation, and the entitlement details.
# Production reliability (/docs/guides/production-reliability)
Covers `Examples/Features/11-Middleware.swift` and
`23-WorkflowGuides.swift`. The goal is a request path that tolerates transient
provider failures without hiding invalid requests or duplicating streamed
output.
### Retry transient failures [#retry-transient-failures]
Every generation API has `maxRetries`. The default is two retries after the
initial attempt. Retries use exponential backoff for timeouts, conflicts,
rate limits, transport failures, and server errors.
```swift
let result = try await generateText(
model: OpenAIModel("gpt-5.6-sol"),
prompt: prompt,
maxRetries: 4
)
```
Only stream establishment is retried. Once a stream has delivered output, it
is never restarted, so users do not receive duplicated tokens.
### Apply defaults and cache exact repeats [#apply-defaults-and-cache-exact-repeats]
Middleware can make policy reusable across every call:
```swift
let model = wrapLanguageModel(
model: OpenAIModel("gpt-5.6-sol"),
middleware: [
.cache(),
.defaultSettings(temperature: 0.2, maxOutputTokens: 2_000)
]
)
```
The built-in cache is process-local. Supply a `LanguageModelCache`
implementation backed by disk or your server-side cache when results must
survive a restart.
### Observe completions and failures [#observe-completions-and-failures]
Per-call callbacks are convenient for usage accounting:
```swift
let result = try await generateText(
model: model,
prompt: prompt,
onFinish: { result in
print("tokens", result.usage.totalTokens)
},
onError: { error in
print("generation failed", error)
}
)
```
For process-wide timing, install an `AITelemetryCollector`:
```swift
struct ConsoleCollector: AITelemetryCollector {
func record(_ event: AITelemetryEvent) {
print(event.name, event.phase.rawValue, event.duration)
}
}
AITelemetry.collector = ConsoleCollector()
```
Avoid recording prompts, tool results, API keys, or provider response bodies
unless your privacy policy explicitly allows it.
### Handle typed errors [#handle-typed-errors]
```swift
do {
return try await generateText(model: model, prompt: prompt).text
} catch let AIError.http(status, body) {
print("provider HTTP error", status, body)
throw AIError.http(status: status, body: body)
} catch AIError.noObjectGenerated(let raw) {
print("structured response was invalid", raw)
throw AIError.noObjectGenerated(raw)
}
```
Invalid requests, decoding problems, and authentication errors should surface
to your logs or UI. Blindly sending them to another provider usually repeats
the same bug.
### Fall back only for transient failures [#fall-back-only-for-transient-failures]
```swift
func isTransient(_ error: Error) -> Bool {
switch error {
case AIError.http(let status, _):
return status == 408 || status == 409 || status == 429 || (500..<600).contains(status)
case AIError.transport, is URLError:
return true
default:
return false
}
}
```
Then attempt a second provider only for those cases. Keep the same system
prompt and tools, but remember that provider-native options are not portable.
### Preserve cancellation [#preserve-cancellation]
Dropping a stream cancels its underlying task. In UI code, keep the consuming
`Task` and cancel it when the user taps Stop or leaves the screen. Do not catch
and convert cancellation into a provider fallback.
## Final code [#final-code]
```swift title="ReliableGeneration.swift"
import AI
import Foundation
struct ConsoleCollector: AITelemetryCollector {
func record(_ event: AITelemetryEvent) {
print(event.name, event.phase.rawValue, event.duration)
}
}
func isTransient(_ error: Error) -> Bool {
switch error {
case AIError.http(let status, _):
return status == 408 || status == 409 || status == 429 || (500..<600).contains(status)
case AIError.transport:
return true
case is URLError:
return true
default:
return false
}
}
func reliableAnswer(_ prompt: String) async throws -> String {
AITelemetry.collector = ConsoleCollector()
let primary = wrapLanguageModel(
model: OpenAIModel("gpt-5.6-sol"),
middleware: [.cache(), .defaultSettings(temperature: 0.2)]
)
do {
return try await generateText(
model: primary,
prompt: prompt,
maxRetries: 4
).text
} catch where isTransient(error) {
return try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: prompt,
maxRetries: 2
).text
}
}
```
Retries re-run the whole call; to salvage a *single* bad response — a
misspelled tool name or fenced JSON — reach for the repair callbacks in
[Recover from bad model output](/docs/guides/self-repair).
See [Middleware](/docs/middleware) for custom middleware and persistent cache
adapters, [Telemetry](/docs/telemetry) for all spans, and
[Errors and retries](/docs/errors) for the complete error list.
# Search and server tools (/docs/guides/search-and-server-tools)
Covers the provider tool examples in `Examples/Providers/OpenAI/`,
`Anthropic/`, `Google/`, and `xAI/`, plus
`Examples/Features/23-WorkflowGuides.swift`.
Local function tools run your Swift closure. Server tools run inside the
provider's infrastructure. Both conform to the same tool protocol and can be
passed to `generateText` or `streamText` together.
### Start with a local function tool [#start-with-a-local-function-tool]
```swift
struct WeatherArgs: Decodable { let city: String }
let weather = Tool.typed(
name: "get_weather",
description: "Get the current weather for a city from Open-Meteo.",
parameters: [
"type": "object",
"properties": ["city": ["type": "string"]],
"required": ["city"]
]
) { (arguments: WeatherArgs) in
try await OpenMeteoWeather.current(city: arguments.city)
}
```
The agentic loop executes this closure locally and feeds its result back to
the model. `OpenMeteoWeather` is the compile-checked geocoding and current
forecast client used throughout the examples; see
[An agent with tools](/docs/guides/agent-with-tools).
### Add provider-hosted tools [#add-provider-hosted-tools]
```swift
let tools: [any AIToolProtocol] = [
weather,
OpenAIModel.Tools.webSearch(allowedDomains: ["swift.org"]),
OpenAIModel.Tools.codeInterpreter()
]
let result = try await generateText(
model: OpenAIModel("gpt-5.6-sol"),
prompt: "Find the latest stable Swift release and compare its number with 5.9.",
tools: tools,
stopWhen: [stepCountIs(6)]
)
```
Domain restrictions and resource ids should be as narrow as the task allows.
### Render citations [#render-citations]
Providers that expose citations emit them as sources:
```swift
print(result.text)
for source in result.sources {
print(source.title ?? source.url, source.url)
}
```
For streaming UIs, watch `fullStream` for `.source` parts and attach them to
the answer as they arrive.
### Choose the provider catalog [#choose-the-provider-catalog]
Each provider has its own server-side capabilities:
| Provider | Builders available in Swift AI |
| --------- | ---------------------------------------------------------------------------------------- |
| OpenAI | Web search, web-search preview, file search, code interpreter |
| Anthropic | Web search, web fetch, code execution, bash, text editor, computer, memory |
| Google | Google Search, URL context, code execution, enterprise web search, Maps, file search |
| xAI | Web search, X search, code execution, file search, MCP server, image and X-video viewing |
```swift
let anthropicTools: [any AIToolProtocol] = [
AnthropicModel.Tools.webSearch(allowedDomains: ["swift.org"]),
AnthropicModel.Tools.webFetch(allowedDomains: ["swift.org"]),
AnthropicModel.Tools.codeExecution()
]
```
Builder arguments are provider-native. A vector-store id from OpenAI, for
example, cannot be passed to Google's file-search builder.
### Use search-native providers when tools are unnecessary [#use-search-native-providers-when-tools-are-unnecessary]
Perplexity searches as part of ordinary generation, while xAI can also take
search configuration through provider options:
```swift
let result = try await generateText(
model: PerplexityModel("sonar-pro"),
prompt: "What changed in the latest Swift release?"
)
for source in result.sources { print(source.url) }
```
Use this shape when the whole request is research. Use explicit tools when
the model must decide whether to search among several possible actions.
### Treat powerful tools as privileged operations [#treat-powerful-tools-as-privileged-operations]
Code execution, shell access, computer use, and external MCP servers can
touch data outside the prompt. Restrict domains and resources, separate
trusted from untrusted content, and place user approval in front of dangerous
local actions. See [Human-in-the-loop approvals](/docs/guides/approvals).
## Final code [#final-code]
```swift title="ResearchAssistant.swift"
import AI
struct WeatherArgs: Decodable { let city: String }
let weather = Tool.typed(
name: "get_weather",
description: "Get the current weather for a city from Open-Meteo.",
parameters: [
"type": "object",
"properties": ["city": ["type": "string"]],
"required": ["city"]
]
) { (arguments: WeatherArgs) in
try await OpenMeteoWeather.current(city: arguments.city)
}
func research(_ question: String) async throws -> GenerateTextResult {
let tools: [any AIToolProtocol] = [
weather,
OpenAIModel.Tools.webSearch(allowedDomains: ["swift.org"]),
OpenAIModel.Tools.codeInterpreter()
]
return try await generateText(
model: OpenAIModel("gpt-5.6-sol"),
prompt: question,
tools: tools,
stopWhen: [stepCountIs(6)]
)
}
```
The complete builder catalogs live in the [OpenAI](/docs/providers/openai),
[Anthropic](/docs/providers/anthropic), [Google](/docs/providers/google), and
[xAI](/docs/providers/xai) provider pages.
# Recover from bad model output (/docs/guides/self-repair)
Covers `Examples/Features/25-ReliabilityAndOutput.swift`. Models occasionally
misspell a tool name or wrap JSON in a code fence. Instead of erroring, you can
hand the raw output back to a repair function that fixes it in place.
### Repair a tool call [#repair-a-tool-call]
`repairToolCall` fires when a call references a tool that isn't in your set.
Return a corrected `ToolCall`, or `nil` to leave it unhandled.
```swift
let result = try await generateText(
model: model, prompt: "…", tools: [weather],
repairToolCall: { call, tools in
call.name == "get_wether"
? ToolCall(id: call.id, name: "get_weather", arguments: call.arguments) : nil
}
)
```
For a heavier fix, re-ask a cheap model to reformat the arguments against the
tool's schema and return the corrected call.
### Repair structured output [#repair-structured-output]
`repairText` on `generateObject` receives the raw text and the parse error, and
returns corrected JSON. Stripping Markdown fences covers the common case.
````swift
let result = try await generateObject(
model: model, of: Summary.self, schema: summarySchema, prompt: "…",
repairText: { text, _ in
text.replacingOccurrences(of: "```json", with: "")
.replacingOccurrences(of: "```", with: "")
}
)
````
### Prefer structure that can't drift [#prefer-structure-that-cant-drift]
When you need both tool use and a final object, `output:` on `generateText`
produces the object alongside the calls; read it from
`result.experimentalOutput`. For a list, `generateObjectArray` decodes a typed
array directly.
Pair these with `maxRetries` (which re-runs the whole call): repair fixes a
salvageable response, retries handle transient failures.
# Build semantic search (/docs/guides/semantic-search)
Covers `Examples/Features/08-Embeddings.swift`, `15-Rerank.swift`, and
`23-WorkflowGuides.swift`. You end up with a small retrieval pipeline:
vector search finds candidates, a reranker improves their order, and a
language model answers only from the selected passages.
### Embed the knowledge base [#embed-the-knowledge-base]
Split documents into useful passages before embedding them. Keep each
passage with its vector and any metadata your UI needs.
```swift
struct Passage: Sendable {
let text: String
let embedding: [Double]
}
func buildIndex(_ texts: [String]) async throws -> [Passage] {
let result = try await embedMany(
model: OpenAIEmbeddingModel("text-embedding-3-small"),
values: texts,
maxBatchSize: 96
)
return zip(texts, result.embeddings).map(Passage.init)
}
```
Persist these vectors in your database for a real app. Store the embedding
model id beside them: vectors from different models should not share an
index.
### Retrieve by cosine similarity [#retrieve-by-cosine-similarity]
Embed the question with the same model, score every passage, and keep a
generous candidate set:
```swift
let query = try await embed(
model: OpenAIEmbeddingModel("text-embedding-3-small"),
value: question
)
let candidates = index
.map { ($0.text, cosineSimilarity(query.embedding, $0.embedding)) }
.sorted { $0.1 > $1.1 }
.prefix(8)
.map(\.0)
```
For a large collection, let a vector database perform this nearest-neighbor
step. The rest of the pipeline stays the same.
### Rerank the candidates [#rerank-the-candidates]
Embedding similarity is fast; a reranker is more precise. Run it only over
the shortlist:
```swift
let reranked = try await rerank(
model: CohereRerankingModel("rerank-v4-fast"),
query: question,
documents: candidates,
topN: 3
)
let context = reranked.rankedDocuments
.map(\.document)
.joined(separator: "\n\n")
```
`VoyageRerankingModel("rerank-2.5")` is a drop-in alternative — same
`rerank(...)` call, and Voyage also serves the retrieval embeddings via
`VoyageEmbeddingModel`.
### Ground the answer [#ground-the-answer]
Put the retrieved passages in a clearly delimited context block and tell the
model what to do when the answer is missing.
```swift
let result = try await generateText(
model: OpenAIModel("gpt-5.6-sol"),
system: "Answer only from the supplied context. Say when the answer is absent.",
prompt: "Context:\n\(context)\n\nQuestion: \(question)"
)
print(result.text)
```
Keep passage ids or source URLs in `Passage` if you want the UI to show
citations beside the answer.
## Final code [#final-code]
```swift title="SemanticSearch.swift"
import AI
struct Passage: Sendable {
let text: String
let embedding: [Double]
}
func buildIndex(_ texts: [String]) async throws -> [Passage] {
let result = try await embedMany(
model: OpenAIEmbeddingModel("text-embedding-3-small"),
values: texts,
maxBatchSize: 96
)
return zip(texts, result.embeddings).map(Passage.init)
}
func answer(question: String, index: [Passage]) async throws -> String {
let query = try await embed(
model: OpenAIEmbeddingModel("text-embedding-3-small"),
value: question
)
let candidates = index
.map { ($0.text, cosineSimilarity(query.embedding, $0.embedding)) }
.sorted { $0.1 > $1.1 }
.prefix(8)
.map(\.0)
guard !candidates.isEmpty else { return "No relevant context found." }
let reranked = try await rerank(
model: CohereRerankingModel("rerank-v4-fast"),
query: question,
documents: candidates,
topN: 3
)
let context = reranked.rankedDocuments.map(\.document).joined(separator: "\n\n")
return try await generateText(
model: OpenAIModel("gpt-5.6-sol"),
system: "Answer only from the supplied context. Say when the answer is absent.",
prompt: "Context:\n\(context)\n\nQuestion: \(question)"
).text
}
```
See [Embeddings](/docs/embeddings) for batching and model choices, and
[Reranking](/docs/reranking) for the full reranking result.
# Serving and testing (/docs/guides/server-and-testing)
Covers `Examples/Features/21-UIStreamsAndTesting.swift` and `11-Middleware.swift`.
You end up serving a chat stream from Swift with status updates woven in,
and testing the whole thing offline.
### Build a stream by hand [#build-a-stream-by-hand]
`UIMessageStream.build` gives you a writer: emit your own chunks, merge
whole generations, and everything arrives as one response:
```swift
func chatStream() -> AsyncThrowingStream {
UIMessageStream.build { writer in
writer.write(.data(name: "data-status", data: .string("looking things up")))
let result = streamText(
model: AnthropicModel("claude-sonnet-5"),
prompt: "Say hello."
)
writer.merge(UIMessageStream.chunks(from: result.fullStream))
}
}
```
Encode it as SSE with `UIMessageStream.encodeSSE(chunk)` plus
`UIMessageStream.headers`, and any web chat UI can consume it.
### Attach metadata [#attach-metadata]
Values ride the start chunk and stream in as the loop runs; the client
deep-merges them into the message:
```swift
UIMessageStream.chunks(
from: result.fullStream,
metadata: .object(["model": .string("claude-sonnet-5")]),
messageMetadata: { part in
if case .finish(_, let usage) = part {
return .object(["totalTokens": .number(Double(usage.totalTokens))])
}
return nil
}
)
```
### Read streams anywhere [#read-streams-anywhere]
`readUIMessageStream` turns any chunk stream into `UIMessage` snapshots,
one per chunk. Persistence pipelines and tests use it instead of a
session:
```swift
for try await snapshot in readUIMessageStream(chunks) {
print(snapshot.text)
}
```
### Test without a network [#test-without-a-network]
Add the `AITesting` product to your test target. `MockLanguageModel`
scripts responses and records every request:
```swift
import AITesting
func testGreeting() async throws {
let model = MockLanguageModel(text: "Hello, world!")
let result = try await generateText(model: model, prompt: "Hi")
XCTAssertEqual(result.text, "Hello, world!")
XCTAssertEqual(model.requests.count, 1)
}
```
Multi-step tool loops script with `responses:`, and
`simulateReadableStream` paces any chunk array like a live stream. The
[testing page](/docs/testing) has the full kit.
## Bonus: middleware [#bonus-middleware]
Wrap any model to transform requests going in and streams coming out:
```swift title="Middleware.swift"
let model = wrapLanguageModel(
model: OllamaModel("qwen3"),
middleware: [
.extractReasoning(tag: "think"), // spans become reasoning
.defaultSettings(temperature: 0.2)
]
)
let result = try await generateText(model: model, prompt: "17 * 23?")
print("thinking:", result.reasoningText)
print("answer:", result.text)
```
## Final code [#final-code]
```swift title="ChatRoute.swift"
import AI
// Vapor, Hummingbird, or anything that writes SSE.
func serveChat(messages: [UIMessage], response: some SSEWriter) async throws {
let history = convertToModelMessages(messages)
let result = streamText(
model: AnthropicModel("claude-sonnet-5"),
messages: history,
tools: [weatherTool]
)
let chunks = UIMessageStream.build { writer in
writer.write(.data(name: "data-status", data: .string("thinking")))
writer.merge(UIMessageStream.chunks(from: result.fullStream))
}
for (field, value) in UIMessageStream.headers {
response.setHeader(field, value)
}
for try await chunk in chunks {
try await response.write(UIMessageStream.encodeSSE(chunk))
}
try await response.write(UIMessageStream.doneSSE)
}
```
# Structured data (/docs/guides/structured-data)
Covers `Examples/Features/06-GenerateObject.swift`, `07-StreamObject.swift`, and
`18-Schema.swift`. You end up with model output you can hand straight to
your views.
### Define the type and its schema [#define-the-type-and-its-schema]
```swift
struct Recipe: Codable {
var name: String
var ingredients: [String]
var steps: [String]
}
let recipeSchema: JSONValue = [
"type": "object",
"properties": [
"name": ["type": "string"],
"ingredients": ["type": "array", "items": ["type": "string"]],
"steps": ["type": "array", "items": ["type": "string"]]
],
"required": ["name", "ingredients", "steps"]
]
```
### Generate the object [#generate-the-object]
```swift
let result = try await generateObject(
model: OpenAIModel("gpt-5.6-sol"),
of: Recipe.self,
schema: recipeSchema,
prompt: "A simple lasagna recipe."
)
print(result.object.name, "with", result.object.steps.count, "steps")
```
If the model produces JSON that doesn't match, the call throws. Your
views only ever see a valid `Recipe`.
### Stream it for the UI [#stream-it-for-the-ui]
Partial JSON gets repaired into usable snapshots as it arrives:
```swift
let result = streamObject(
model: model,
schema: recipeSchema,
prompt: "A simple lasagna recipe."
)
var latest: JSONValue = .null
for try await partial in result.partialObjectStream {
latest = partial
print("so far:", partial["name"]?.stringValue ?? "...")
}
let recipe = try latest.decode(Recipe.self)
```
### Trade the raw schema for the DSL [#trade-the-raw-schema-for-the-dsl]
`Schema` combinators build the JSON Schema and validate the output at
runtime before decoding:
```swift
let typedRecipeSchema = Schema.object([
"name": .string(description: "Recipe name"),
"steps": .array(of: .string(), minItems: 1),
"servings": .integer(minimum: 1).optional()
])
```
The same schemas type your tools, and arguments get validated before the
closure runs:
```swift
let serve = Tool(
name: "serve",
description: "Serve a number of portions",
parameters: Schema.object(["servings": .integer(minimum: 1)])
) { args in
.string("served \(args["servings"]?.intValue ?? 0)")
}
```
## Final code [#final-code]
```swift title="RecipeGenerator.swift"
import AI
struct Recipe: Codable {
var name: String
var ingredients: [String]
var steps: [String]
}
let recipeSchema = Schema.object([
"name": .string(description: "Recipe name"),
"ingredients": .array(of: .string(), minItems: 1),
"steps": .array(of: .string(), minItems: 1)
])
func generateRecipe() async throws -> Recipe {
let result = try await generateObject(
model: OpenAIModel("gpt-5.6-sol"),
of: Recipe.self,
schema: recipeSchema,
prompt: "A simple lasagna recipe."
)
return result.object
}
func streamRecipe(into render: @escaping (JSONValue) -> Void) async throws -> Recipe {
let result = streamObject(
model: OpenAIModel("gpt-5.6-sol"),
schema: recipeSchema,
prompt: "A simple lasagna recipe."
)
var latest: JSONValue = .null
for try await partial in result.partialObjectStream {
latest = partial
render(partial)
}
return try latest.decode(Recipe.self)
}
```
Enums and unions compose when the shapes get richer:
```swift
let event = Schema.object([
"kind": .enum(["meeting", "reminder"]),
"when": .string(format: "date-time"),
"attendees": .array(of: .object([
"name": .string(),
"id": .anyOf([.integer(), .string()])
])).optional()
])
```
# Transcribe and summarize audio (/docs/guides/transcribe-and-summarize)
Covers `Examples/Features/13-SpeechAndTranscription.swift` and
`23-WorkflowGuides.swift`. You end up with a reusable meeting pipeline that
accepts an audio file and returns validated Swift data.
### Load and transcribe the recording [#load-and-transcribe-the-recording]
Pass the bytes and the real media type. Asynchronous providers such as
AssemblyAI, Rev.ai, and Gladia submit and poll internally, so the call site is
still one `await`.
```swift
let audio = try Data(contentsOf: recordingURL)
let transcript = try await transcribe(
model: OpenAITranscriptionModel("whisper-1"),
audio: audio,
mediaType: "audio/mpeg"
)
print(transcript.text)
```
### Use timestamps when available [#use-timestamps-when-available]
Providers can return language, duration, and timed segments in addition to
the full text:
```swift
for segment in transcript.segments {
print("\(segment.startSecond)s–\(segment.endSecond)s", segment.text)
}
```
Some providers return only the complete transcript. Treat `segments` as an
optional enhancement rather than requiring it for the rest of the pipeline.
### Define typed notes [#define-typed-notes]
Use structured output so downstream UI does not parse prose.
```swift
struct MeetingNotes: Codable, Sendable {
let title: String
let summary: String
let actionItems: [String]
}
let notesSchema: JSONValue = [
"type": "object",
"properties": [
"title": ["type": "string"],
"summary": ["type": "string"],
"actionItems": ["type": "array", "items": ["type": "string"]]
],
"required": ["title", "summary", "actionItems"],
"additionalProperties": false
]
```
### Summarize the transcript [#summarize-the-transcript]
```swift
let notes = try await generateObject(
model: OpenAIModel("gpt-5.6-sol"),
of: MeetingNotes.self,
schema: notesSchema,
schemaName: "meeting_notes",
prompt: "Summarize this transcript and extract action items:\n\n\(transcript.text)"
).object
```
Long recordings should be summarized in chunks and then reduced into one
final summary instead of placing an unbounded transcript in one request.
### Read the summary aloud [#read-the-summary-aloud]
Speech generation is optional and uses the same provider-independent shape:
```swift
let speech = try await generateSpeech(
model: OpenAISpeechModel("gpt-4o-mini-tts"),
text: notes.summary,
voice: "alloy",
outputFormat: "mp3"
)
try speech.audio.write(to: summaryAudioURL)
```
## Final code [#final-code]
```swift title="MeetingNotes.swift"
import AI
import Foundation
struct MeetingNotes: Codable, Sendable {
let title: String
let summary: String
let actionItems: [String]
}
let notesSchema: JSONValue = [
"type": "object",
"properties": [
"title": ["type": "string"],
"summary": ["type": "string"],
"actionItems": ["type": "array", "items": ["type": "string"]]
],
"required": ["title", "summary", "actionItems"],
"additionalProperties": false
]
func makeMeetingNotes(from recordingURL: URL) async throws -> MeetingNotes {
let transcript = try await transcribe(
model: OpenAITranscriptionModel("whisper-1"),
audio: Data(contentsOf: recordingURL),
mediaType: "audio/mpeg"
)
return try await generateObject(
model: OpenAIModel("gpt-5.6-sol"),
of: MeetingNotes.self,
schema: notesSchema,
schemaName: "meeting_notes",
prompt: "Summarize this transcript and extract action items:\n\n\(transcript.text)"
).object
}
```
See [Transcription](/docs/transcription) and
[Speech generation](/docs/speech-generation) for provider and model choices.
# Vision and files (/docs/guides/vision-and-files)
Covers `Examples/Features/17-MultimodalAndSettings.swift`. You end up asking
questions about a photo and reading the model's thinking as it streams.
### Send an image [#send-an-image]
Images ride inside the message content and map to each provider's native
shape automatically:
```swift
let photo = try Data(contentsOf: photoURL)
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
messages: [Message(role: .user, content: [
.text("What animal is in this photo?"),
.image(ImageContent(data: photo))
])]
)
print(result.text)
```
PDFs and other documents use `.file(FileContent(...))` the same way, and
remote URLs work in place of inline data.
### Tune the sampling [#tune-the-sampling]
The full settings surface rides on the same call, mapped per provider
and dropped where a wire lacks the knob:
```swift
let result = try await generateText(
model: OpenAIModel("gpt-5.6-sol"),
prompt: "Name a color.",
toolChoice: .none,
temperature: 0.7,
topP: 0.9,
topK: 40,
presencePenalty: 0.5,
frequencyPenalty: 0.3,
seed: 42,
onFinish: { result in print("used \(result.usage.totalTokens) tokens") },
onError: { error in print("failed:", error) }
)
```
### Stream the thinking [#stream-the-thinking]
One `reasoning` value works across providers; the thinking arrives as
its own deltas:
```swift
let result = streamText(
model: GoogleModel("gemini-3.5-flash"),
prompt: "Explain the Riemann hypothesis in simple terms.",
reasoning: .high
)
for try await part in result.fullStream {
if case .reasoningDelta(let thought) = part { showThinking(thought) }
if case .textDelta(let text) = part { showAnswer(text) }
}
```
## Final code [#final-code]
```swift title="PhotoQuestion.swift"
import AI
import Foundation
func askAboutPhoto(_ photoURL: URL, question: String) async throws -> String {
let photo = try Data(contentsOf: photoURL)
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
messages: [Message(role: .user, content: [
.text(question),
.image(ImageContent(data: photo))
])],
reasoning: .medium
)
return result.text
}
```
When you need an exact thinking budget instead of a level, use
`providerOptions`; it always wins over the portable value:
```swift
providerOptions: [
"thinking": ["type": "enabled", "budget_tokens": 12000]
]
```
# A voice assistant (/docs/guides/voice-assistant)
Covers `Examples/Features/22-Realtime.swift`. You end up talking to a model that
talks back through the speakers and can call your code mid-conversation.
The full app version lives in `Apps/RealtimeDemo`.
### Mint a client secret on your server [#mint-a-client-secret-on-your-server]
Realtime connections use short-lived secrets so your API key never
ships in the app:
```swift title="TokenRoute.swift"
let model = XaiRealtimeModel("grok-voice-latest")
let secret = try await model.createClientSecret(options: RealtimeClientSecretOptions(
expiresAfterSeconds: 300,
sessionConfig: RealtimeSessionConfig(
tools: getRealtimeToolDefinitions(tools: [weatherTool])
)
))
// return {token, url, expiresAt} to the app
```
`OpenAIRealtimeModel("gpt-realtime")` and
`GoogleRealtimeModel("gemini-2.5-flash-native-audio-preview-09-2025")`
mint the same way.
### Create the session [#create-the-session]
Configure the conversation and handle tool calls in one place:
```swift
let session = RealtimeSession(
model: XaiRealtimeModel("grok-voice-latest"),
sessionConfig: RealtimeSessionConfig(
instructions: "You are a helpful assistant. Be concise.",
voice: "alloy",
inputAudioTranscription: .init(),
turnDetection: .init(type: .serverVAD)
),
onToolCall: { call in
guard call.name == "getWeather" else { return nil }
return try await weatherTool.execute(call.arguments)
}
)
session.connect(secret: secret)
```
Returning nil from `onToolCall` defers the answer; submit it later with
`session.addToolOutput(callID:output:)`.
Here `weatherTool` is the Open-Meteo-backed tool from
[An agent with tools](/docs/guides/agent-with-tools), not a canned response.
### Wire the audio [#wire-the-audio]
You own capture and playback, which on iOS means AVAudioEngine. Feed the
mic in as PCM, play the response out:
```swift
// Speak: 16-bit PCM at the session rate (24 kHz default)
session.sendAudio(microphoneChunk)
// Listen: decoded audio chunks for your player
for await chunk in session.audioOutput {
player.play(chunk)
}
```
When the user interrupts and your player stops, tell the model how much
was heard so its memory matches reality:
```swift
session.playbackInterrupted(playedMilliseconds: player.playedMilliseconds)
```
### Render the conversation [#render-the-conversation]
`session.messages` is regular `UIMessage` state: transcripts stream in
as text parts, your speech shows up transcribed, and tool calls appear
as tool parts. The same rendering code as a chat screen works here.
Typed input works alongside voice: `session.sendText("Hello!")`.
## Final code [#final-code]
```swift title="VoiceAssistant.swift"
import AI
@available(iOS 17.0, macOS 14.0, *)
@MainActor
func startVoiceSession(
secret: RealtimeClientSecret,
weatherTool: Tool
) -> RealtimeSession {
let session = RealtimeSession(
model: XaiRealtimeModel("grok-voice-latest"),
sessionConfig: RealtimeSessionConfig(
instructions: "You are a helpful assistant. Be concise.",
inputAudioTranscription: .init(),
turnDetection: .init(type: .serverVAD)
),
onToolCall: { call in
guard call.name == "getWeather" else { return nil }
return try await weatherTool.execute(call.arguments)
}
)
session.connect(secret: secret)
session.sendText("Hello!")
Task {
for await chunk in session.audioOutput {
// hand PCM to your AVAudioEngine player
_ = chunk
}
}
return session
}
```
The `Apps/RealtimeDemo` app adds the AVAudioEngine player and mic
capture, the settings sheet, and barge-in, in about 200 lines of
SwiftUI.
# Agent (/docs/reference/agent)
Everything you would otherwise pass to [`generateText`](/docs/reference/generate-text)
on every call, held in one value you configure once and call many times. The
properties mirror that function's parameters exactly, so anything you can do
there you can do here.
Reach for it when the same setup runs more than once: a support agent, a code
reviewer, a research step in a larger pipeline. For a single one-off call, the
free functions are less ceremony.
An agent can also become a tool for another agent via `asTool`, which is how
you build a supervisor that delegates to specialists.
```swift
let researcher = Agent(
model: AnthropicModel("claude-sonnet-5"),
instructions: "You research questions and cite your sources.",
tools: [webSearch]
)
let result = try await researcher.generate(prompt: "Who first isolated neon?")
```
## Declaration [#declaration]
```swift
public struct Agent
```
Defined in `Sources/AI/Core/Agent.swift`.
## Initializer [#initializer]
```swift
init(
model: any LanguageModel,
instructions: String? = nil,
tools: [any AIToolProtocol] = [],
toolChoice: ToolChoice = .auto,
activeTools: [String]? = nil,
toolOrder: [String]? = nil,
toolsContext: [String: JSONValue] = [:],
maxOutputTokens: Int = 1024,
temperature: Double? = nil,
topP: Double? = nil,
topK: Int? = nil,
presencePenalty: Double? = nil,
frequencyPenalty: Double? = nil,
seed: Int? = nil,
reasoning: ReasoningEffort = .providerDefault,
stopWhen: [StopCondition]? = nil,
maxSteps: Int = 8,
prepareCall: PrepareCall? = nil,
prepareStep: PrepareStep? = nil,
onStepFinish: OnStepFinish? = nil,
maxRetries: Int = 2,
providerOptions: JSONValue? = nil,
toolApproval: ToolApprovalPolicy? = nil,
toolApprovalSecret: String? = nil,
timeout: GenerationTimeout? = nil,
runtimeContext: JSONValue? = nil,
telemetry: TelemetrySettings? = nil,
compaction: Compaction? = nil
)
```
## Properties [#properties]
| Property | Type | Default |
| ------------------------ | ---------------------- | ------- |
| `var model` | `any LanguageModel` | — |
| `var instructions` | `String?` | — |
| `var tools` | `[any AIToolProtocol]` | — |
| `var toolChoice` | `ToolChoice` | — |
| `var activeTools` | `[String]?` | — |
| `var toolOrder` | `[String]?` | — |
| `var toolsContext` | `[String: JSONValue]` | — |
| `var maxOutputTokens` | `Int` | — |
| `var temperature` | `Double?` | — |
| `var topP` | `Double?` | — |
| `var topK` | `Int?` | — |
| `var presencePenalty` | `Double?` | — |
| `var frequencyPenalty` | `Double?` | — |
| `var seed` | `Int?` | — |
| `var reasoning` | `ReasoningEffort` | — |
| `var stopWhen` | `[StopCondition]?` | — |
| `var maxSteps` | `Int` | — |
| `var prepareCall` | `PrepareCall?` | — |
| `var prepareStep` | `PrepareStep?` | — |
| `var onStepFinish` | `OnStepFinish?` | — |
| `var maxRetries` | `Int` | — |
| `var providerOptions` | `JSONValue?` | — |
| `var toolApproval` | `ToolApprovalPolicy?` | — |
| `var toolApprovalSecret` | `String?` | — |
| `var timeout` | `GenerationTimeout?` | — |
| `var runtimeContext` | `JSONValue?` | — |
| `var telemetry` | `TelemetrySettings?` | — |
| `var compaction` | `Compaction?` | — |
## Methods [#methods]
```swift
func generate(
prompt: String
) async throws -> GenerateTextResult
```
```swift
func generate(
messages: [Message]
) async throws -> GenerateTextResult
```
```swift
func stream(
prompt: String
) -> StreamTextResult
```
```swift
func stream(
messages: [Message]
) -> StreamTextResult
```
```swift
func asTool(
name: String,
description: String,
promptDescription: String = "The task for the agent to perform."
) -> Tool
```
## See also [#see-also]
* [Agents guide](/docs/agents)
* [generateText](/docs/reference/generate-text)
* [Tool](/docs/reference/tool)
# ChatSession (/docs/reference/chat-session)
Holds the message list, the in-flight status, and the send/stop/regenerate
actions behind an observable object, so a view can bind to it and stay in sync
without you writing the plumbing.
It handles the parts that are tedious to get right by hand: streaming deltas
into the right message, keeping tool call and result parts paired, surfacing
errors, and cancelling cleanly when someone hits stop.
```swift
@State private var chat = ChatSession(
model: AnthropicModel("claude-sonnet-5")
)
var body: some View {
MessageList(messages: chat.messages)
TextField("Message", text: $draft)
.onSubmit { Task { await chat.send(draft) } }
}
```
## Declaration [#declaration]
```swift
public class ChatSession
```
Defined in `Sources/AI/UI/ChatSession.swift`.
## Initializer [#initializer]
```swift
init(
transport: any ChatTransport,
id: String = UUID().uuidString,
messages: [UIMessage] = []
)
```
## Properties [#properties]
| Property | Type | Default | | |
| --------------- | --------------- | -------------- | - | ------------------------ |
| `let id` | `String` | — | | |
| `var messages` | `[UIMessage]` | — | | |
| `var status` | `Status` | `.ready` | | |
| `var isLoading` | `Bool { status` | \`= .submitted | | status == .streaming }\` |
## Methods [#methods]
```swift
func send(
_ text: String
)
```
```swift
func sendMessage(
_ message: UIMessage
)
```
```swift
func regenerate()
```
```swift
func addToolResult(
toolCallID: String,
result: JSONValue
)
```
```swift
func addToolApprovalResponse(
approvalID: String,
approved: Bool,
reason: String? = nil
)
```
```swift
func resumeStream()
```
```swift
func stop()
```
```swift
func setMessages(
_ newMessages: [UIMessage]
)
```
## See also [#see-also]
* [Chat UI](/docs/chat-ui)
* [Chat screen guide](/docs/guides/chat-screen)
# CompactedContext (/docs/reference/compacted-context)
What compaction extracts from the turns it compresses, and what gets rendered
back into the conversation in their place.
The shape is deliberate. Free-form summaries lose exactly the things that are
expensive to rediscover, so this pins them into named fields: what the run is
trying to do, what has been decided, what has been established, and what has
already been tried and failed.
You rarely build one by hand. Read it when you want to see what compaction kept
and what it let go.
```swift
let compaction = Compaction(
onCompact: { event in
print("kept \(event.context.decisions.count) decisions")
print("dropped \(event.messagesCompacted) messages")
}
)
```
## Declaration [#declaration]
```swift
public struct CompactedContext
```
Defined in `Sources/AI/Core/Compaction.swift`.
## Initializer [#initializer]
```swift
init(
goal: String,
constraints: [String] = [],
decisions: [Decision] = [],
establishedFacts: [Fact] = [],
deadEnds: [DeadEnd] = [],
openQuestions: [String] = [],
artifacts: [Artifact] = []
)
```
## Properties [#properties]
| Property | Type | Default |
| ---------------------- | ------------ | ------- |
| `var goal` | `String` | — |
| `var constraints` | `[String]` | — |
| `var decisions` | `[Decision]` | — |
| `var establishedFacts` | `[Fact]` | — |
| `var deadEnds` | `[DeadEnd]` | — |
| `var openQuestions` | `[String]` | — |
| `var artifacts` | `[Artifact]` | — |
## See also [#see-also]
* [Context management](/docs/context-management)
* [Compaction](/docs/reference/compaction)
# Compaction (/docs/reference/compaction)
Long conversations eventually outgrow the model's context window. Compaction
watches the token count and, past a threshold, replaces the older turns with a
structured summary so the run can keep going.
It compresses in proportion to how cheaply something can be recovered. A tool
result you can fetch again is worth less than a decision you can't re-derive.
And a dead end, meaning something already tried that didn't work, is the most
valuable thing to keep and the first thing a naive summariser throws away.
Pass it to `generateText`, `streamText`, or an `Agent` and it runs itself.
```swift
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
messages: history,
tools: tools,
compaction: Compaction()
)
```
## Declaration [#declaration]
```swift
public struct Compaction
```
Defined in `Sources/AI/Core/Compaction.swift`.
## Initializer [#initializer]
```swift
init(
budget: CompactionBudget = CompactionBudget(),
pinning: CompactionPinning = .default,
keepLastSteps: Int = 4,
onCompact: (@Sendable (CompactionEvent) -> Void)? = nil
)
```
## Properties [#properties]
| Property | Type | Default |
| ------------------- | ---------------------------------------- | ------- |
| `var budget` | `CompactionBudget` | — |
| `var pinning` | `CompactionPinning` | — |
| `var keepLastSteps` | `Int` | — |
| `var onCompact` | `(@Sendable (CompactionEvent) -> Void)?` | — |
## See also [#see-also]
* [Context management](/docs/context-management)
* [CompactedContext](/docs/reference/compacted-context)
* [pruneMessages](/docs/reference/prune-messages)
# consumeStream (/docs/reference/consume-stream)
Runs a stream to the end without collecting it. Use it when the work matters
but the output does not, such as making sure `onFinish` fires and the run is
persisted even if the client disconnected.
```swift
try await consumeStream(stream: result.fullStream)
```
## Signature [#signature]
```swift
func consumeStream(
_ chunks: AsyncThrowingStream,
onError: (@Sendable (Error) async -> Void)? = nil
) async
```
Defined in `Sources/AI/Transport/TextStreamChatTransport.swift`.
## Parameters [#parameters]
## Overloads [#overloads]
1 other form of this function exists:
```swift
func consumeStream(
_ parts: AsyncThrowingStream,
onError: (@Sendable (Error) async -> Void)? = nil
) async
```
## Returns [#returns]
Nothing.
## See also [#see-also]
* [Chat UI](/docs/chat-ui)
# ContentPart (/docs/reference/content-part)
Messages are arrays of these. Splitting a turn into parts is what lets one
message hold a sentence and an image, or an assistant turn hold both its
reasoning and the tool calls it decided on.
You mostly construct the text and image cases and read the rest. Tool calls and
tool results are produced by the loop, and reasoning parts appear only for
models that expose their thinking.
```swift
let message = Message(
role: .user,
content: [
.text("What is in this photo?"),
.image(data: photoData, mediaType: "image/jpeg")
]
)
```
## Declaration [#declaration]
```swift
public enum ContentPart
```
Defined in `Sources/AI/Core/Message.swift`.
## Cases [#cases]
```swift
case text(String)
case image(ImageContent)
case file(FileContent)
case toolCall(ToolCall)
case toolResult(ToolResult)
case toolApprovalResponse(ToolApprovalResponse)
```
## See also [#see-also]
* [Messages guide](/docs/messages)
* [Message](/docs/reference/message)
# convertToModelMessages (/docs/reference/convert-to-model-messages)
Maps the `UIMessage` shape a client sends into the `Message` values a model
takes, flattening UI-only parts. This is the first thing a server route does
with an incoming chat body.
```swift
let messages = convertToModelMessages(body.messages)
```
## Signature [#signature]
```swift
func convertToModelMessages(
_ uiMessages: [UIMessage]
) -> [Message]
```
Defined in `Sources/AI/Transport/StreamBridge.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `[Message]`.
## See also [#see-also]
* [Streaming protocol](/docs/streaming-protocol)
# cosineSimilarity (/docs/reference/cosine-similarity)
Pure arithmetic, no network call. Returns a value from -1 to 1, where 1 means
the vectors point the same way.
Vectors of different lengths, or a zero vector, return 0 rather than throwing.
```swift
let score = cosineSimilarity(queryVector, documentVector)
```
## Signature [#signature]
```swift
func cosineSimilarity(
_ a: [Double],
_ b: [Double]
) -> Double
```
Defined in `Sources/AI/Core/Embeddings.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `Double` between -1 and 1.
## See also [#see-also]
* [Embeddings](/docs/embeddings)
# createIdGenerator (/docs/reference/create-id-generator)
Returns a generator producing ids with the prefix, alphabet, size, and
separator you choose. Useful when ids need to be recognizable per surface, like
`msg_` and `call_`.
```swift
let nextID = createIdGenerator(prefix: "msg", size: 16)
let id = nextID()
```
## Signature [#signature]
```swift
func createIdGenerator(
prefix: String? = nil,
separator: String = "-",
alphabet: String = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ",
size: Int = 16
) -> @Sendable () -> String
```
Defined in `Sources/AI/Core/RuntimeContext.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `@Sendable () -> String`.
## See also [#see-also]
* [generateId](/docs/reference/generate-id)
# customProvider (/docs/reference/custom-provider)
Assembles a provider out of closures that resolve a model id to a model, with
an optional fallback. Use it to alias ids, pin defaults, or route
`"provider:model"` strings at your own boundary.
```swift
let provider = customProvider(
languageModels: ["fast": OpenAIModel("gpt-5.1-mini")],
fallbackProvider: nil
)
```
## Signature [#signature]
```swift
func customProvider(
languageModels: [String: any LanguageModel] = [:],
embeddingModels: [String: any EmbeddingModel] = [:],
imageModels: [String: any ImageModel] = [:],
speechModels: [String: any SpeechModel] = [:],
transcriptionModels: [String: any TranscriptionModel] = [:],
rerankingModels: [String: any RerankingModel] = [:],
fallback: ProviderRegistry.Provider? = nil
) -> ProviderRegistry.Provider
```
Defined in `Sources/AI/Core/ProviderRegistry.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `ProviderRegistry.Provider`.
## See also [#see-also]
* [Providers](/docs/providers)
# decodePCM16 (/docs/reference/decode-pcm16)
The inverse of [`encodePCM16`](/docs/reference/encode-pcm16), for audio
arriving from a provider that you want to play or process.
```swift
let samples = decodePCM16(data)
```
## Signature [#signature]
```swift
func decodePCM16(
_ data: Data
) -> [Float]
```
Defined in `Sources/AI/Realtime/AudioResampling.swift`.
## Parameters [#parameters]
## Returns [#returns]
An array of samples.
## See also [#see-also]
* [Realtime voice](/docs/realtime)
# detectAudioMediaType (/docs/reference/detect-audio-media-type)
Sniffs the container from the leading bytes and returns a media type. Used
internally by [`transcribe`](/docs/reference/transcribe) when the declared
type is generic; exposed because it is useful on its own.
```swift
let mediaType = detectAudioMediaType(data) // "audio/mp4", "audio/wav", ...
```
## Signature [#signature]
```swift
func detectAudioMediaType(
_ audio: Data
) -> String?
```
Defined in `Sources/AI/Core/Transcription.swift`.
## Parameters [#parameters]
## Returns [#returns]
A media type `String`, or `nil` when nothing matches.
## See also [#see-also]
* [transcribe](/docs/reference/transcribe)
# detectImageMediaType (/docs/reference/detect-image-media-type)
Sniffs the first twelve bytes of image data and returns the media type it
finds: PNG, JPEG, GIF, WebP, HEIC, or BMP. Nothing else is inspected, so the
answer is only as good as the header.
`generateImage` already uses this to label results a provider returns without
a media type. Reach for it directly when you have raw bytes from somewhere else
— a file on disk, a download, a pasteboard — and need to tag them before
sending them to a model.
```swift
let mediaType = detectImageMediaType(bytes) ?? "image/png"
```
## Signature [#signature]
```swift
func detectImageMediaType(
_ image: Data
) -> String?
```
Defined in `Sources/AI/Core/ImageGeneration.swift`.
## Parameters [#parameters]
## Returns [#returns]
The media type as a `String`, or `nil` when the header matches none of
the recognized formats or the data is shorter than twelve bytes.
## See also [#see-also]
* [Image generation](/docs/image-generation)
# detectToolDrift (/docs/reference/detect-tool-drift)
Reports which tools were added or changed since the baseline. A server that
quietly rewrites a tool's description or schema after approval is the rug-pull
attack this exists to catch.
```swift
let drift = detectToolDrift(fingerprintTools(latest), baseline: approved)
if drift.hasDrift { requireReapproval(drift.changed, drift.added) }
```
## Signature [#signature]
```swift
func detectToolDrift(
_ current: [String: MCPToolFingerprint],
baseline: [String: MCPToolFingerprint]
) -> MCPToolDrift
```
Defined in `Sources/AI/MCP/MCPToolDrift.swift`.
## Parameters [#parameters]
## Returns [#returns]
An `MCPToolDrift` with `added`, `changed`, and `hasDrift`.
## See also [#see-also]
* [MCP](/docs/mcp)
# embedMany (/docs/reference/embed-many)
Embeds an array of strings, splitting into batches when the list exceeds what
the provider accepts in one request. Order is preserved, so the vector at index
`i` belongs to the value at index `i`.
Set `maxBatchSize` to stay under a provider's limit or to ease off a rate
limit.
```swift
let result = try await embedMany(
model: OpenAIEmbeddingModel("text-embedding-3-small"),
values: documents
)
```
## Signature [#signature]
```swift
func embedMany(
model: any EmbeddingModel,
values: [String],
maxBatchSize: Int? = nil,
maxRetries: Int = 2
) async throws -> EmbedManyResult
```
Defined in `Sources/AI/Core/Embeddings.swift`.
## Parameters [#parameters]
## Returns [#returns]
`EmbedManyResult` with `embeddings` in input order and combined `usage`.
## See also [#see-also]
* [Embeddings](/docs/embeddings)
* [embed](/docs/reference/embed)
# embed (/docs/reference/embed)
Turns one string into a vector for similarity search, clustering, or
classification. For more than one value use
[`embedMany`](/docs/reference/embed-many), which batches.
```swift
let result = try await embed(
model: OpenAIEmbeddingModel("text-embedding-3-small"),
value: "swift concurrency"
)
print(result.embedding.count)
```
## Signature [#signature]
```swift
func embed(
model: any EmbeddingModel,
value: String,
maxRetries: Int = 2
) async throws -> EmbedResult
```
Defined in `Sources/AI/Core/Embeddings.swift`.
## Parameters [#parameters]
## Returns [#returns]
`EmbedResult` with the `embedding` vector and `usage`.
## See also [#see-also]
* [Embeddings](/docs/embeddings)
* [embedMany](/docs/reference/embed-many)
* [cosineSimilarity](/docs/reference/cosine-similarity)
# encodePCM16 (/docs/reference/encode-pcm16)
Packs float samples into little-endian 16-bit PCM, the format realtime
providers accept.
```swift
let data = encodePCM16(samples)
```
## Signature [#signature]
```swift
func encodePCM16(
_ samples: [Float]
) -> Data
```
Defined in `Sources/AI/Realtime/AudioResampling.swift`.
## Parameters [#parameters]
## Returns [#returns]
`Data` of 16-bit little-endian PCM.
## See also [#see-also]
* [Realtime voice](/docs/realtime)
# Errors (/docs/reference/errors)
Every failure in the SDK surfaces as a typed `AIError`. Streaming calls throw
from the stream you iterate rather than from the call that created it.
```swift
do {
let result = try await generateText(model: model, prompt: prompt)
} catch let AIError.http(status, body) {
// the provider's own error body, verbatim
} catch AIError.noObjectGenerated {
// structured output did not parse or validate
}
```
## Transport and request [#transport-and-request]
### `.http(status:body:)` [#httpstatusbody]
A non-2xx response. `body` is the provider's error payload unchanged, which is
usually the fastest way to see what it objected to. A 401 almost always means a
missing or wrong API key; a 429 is a rate limit and is retried automatically
before you ever see it.
### `.transport(String)` [#transportstring]
The request never completed: connection refused, DNS failure, a socket closing
mid-stream, an MCP server returning something unusable.
### `.decoding(String)` [#decodingstring]
A response arrived but did not match the shape expected. Usually a provider
returning an error document where content belonged, or an OpenAI-compatible
endpoint that is not as compatible as advertised.
### `.invalidRequest(String)` [#invalidrequeststring]
The request was rejected before it was sent, because arguments do not make
sense together. Thrown locally, so no tokens are spent.
### `.unsupportedFunctionality(String)` [#unsupportedfunctionalitystring]
The capability exists in the API but not on this provider or platform, such as
PKCE `S256` where CryptoKit is unavailable. Not retryable; use a different
provider or path.
## Tools [#tools]
### `.unknownTool(String)` [#unknowntoolstring]
The model called a tool that was not in the array passed to the call. Most
often the tool list changed between turns while the history still refers to the
old one.
### `.invalidToolInput(tool:reason:)` [#invalidtoolinputtoolreason]
The arguments failed schema validation before the tool ran, so the tool was
never invoked. Supply `repairToolCall` to fix a malformed call and retry it
rather than failing the run.
### `.invalidToolContext(tool:reason:)` [#invalidtoolcontexttoolreason]
The tool's entry in `toolsContext` failed the `contextSchema` it declared with
`.withContextSchema(_:)`. A configuration error on your side, not the model's.
### `.missingToolResults([String])` [#missingtoolresultsstring]
The conversation was sent on with tool calls that have no matching results. In
a client-side tool flow this means a result was never posted back; check
[`lastAssistantMessageIsCompleteWithToolCalls`](/docs/reference/last-assistant-message-is-complete-with-tool-calls)
before resuming.
### `.toolCallRepairFailed(tool:reason:)` [#toolcallrepairfailedtoolreason]
`repairToolCall` ran and still could not produce a valid call. The underlying
input is usually the real problem.
### `.invalidToolApproval(String)` [#invalidtoolapprovalstring]
An approval response did not verify: a bad HMAC signature, a replayed
approval, or an approval for a tool that was never offered. This check is
fail-closed on purpose, so a client cannot forge one.
## Output [#output]
### `.noObjectGenerated(String)` [#noobjectgeneratedstring]
Structured output did not parse or validate. Raising `maxOutputTokens` fixes it
surprisingly often, because a truncated object is invalid JSON. `repairText`
can salvage nearly-valid output.
## Limits and lifecycle [#limits-and-lifecycle]
### `.timedOut(scope:limit:tool:)` [#timedoutscopelimittool]
A [timeout](/docs/timeouts-and-approvals) fired. `scope` says which one, which
tells you what to change: `.total` for the whole call, `.step` for one model
call, `.firstChunk` or `.chunk` for a stall, and a set `tool` for a specific
tool. Tool timeouts do not throw by default, they come back as a tool error the
model can react to.
### `.authorizationRequired(url:)` [#authorizationrequiredurl]
An [MCP](/docs/mcp) server needs OAuth and the session could not refresh into a
valid token. The URL is the sign-in page to open; hand the redirect back to
`complete(callbackURL:)`.
## See also [#see-also]
* [Errors and retries](/docs/errors) for handling patterns and retry behavior
* [Timeouts and approvals](/docs/timeouts-and-approvals)
# filterActiveTools (/docs/reference/filter-active-tools)
Returns only the tools whose names appear in the list you pass, preserving
order. Handy inside `prepareStep` to change which tools a model can see from
one step to the next without rebuilding the array.
```swift
filterActiveTools(tools, names: ["search", "read_file"])
```
## Signature [#signature]
```swift
func filterActiveTools(
_ tools: [any AIToolProtocol],
activeTools: [String]?
) -> [any AIToolProtocol]
```
Defined in `Sources/AI/Core/RuntimeContext.swift`.
## Parameters [#parameters]
## Returns [#returns]
A filtered `[any AIToolProtocol]`.
## See also [#see-also]
* [Tools](/docs/tools)
# fingerprintTools (/docs/reference/fingerprint-tools)
Hashes each tool's name, description, and schema into a fingerprint you can
store. Pair it with
[`detectToolDrift`](/docs/reference/detect-tool-drift) to notice when a
remote server changes a tool after a user approved it.
```swift
let approved = fingerprintTools(try await mcp.tools())
```
## Signature [#signature]
```swift
func fingerprintTools(
_ tools: [any AIToolProtocol]
) -> [String: MCPToolFingerprint]
```
Defined in `Sources/AI/MCP/MCPToolDrift.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `[String: MCPToolFingerprint]` keyed by tool name.
## See also [#see-also]
* [MCP](/docs/mcp)
# generateEnum (/docs/reference/generate-enum)
Constrains the model to choose one of the strings you supply. The result is
guaranteed to be a member of the set, so classification never needs a
post-check or a fuzzy match.
Cheaper and more reliable than asking for a label in prose and parsing it.
```swift
let result = try await generateEnum(
model: AnthropicModel("claude-haiku-4-5-20251001"),
values: ["billing", "technical", "sales"],
prompt: ticket
)
print(result.value)
```
## Signature [#signature]
```swift
func generateEnum(
model: any LanguageModel,
values: [String],
messages: [Message] = [],
system: String? = nil,
prompt: String? = nil,
maxOutputTokens: Int = 1024,
temperature: Double? = nil,
providerOptions: JSONValue? = nil,
maxRetries: Int = 2
) async throws -> GenerateEnumResult
```
Defined in `Sources/AI/Core/GenerateObject.swift`.
## Parameters [#parameters]
## Returns [#returns]
`GenerateEnumResult` with the chosen `value`, `finishReason`, and `usage`.
## See also [#see-also]
* [Structured output](/docs/structured-output)
# generateId (/docs/reference/generate-id)
Produces the same id format the SDK uses internally for messages and tool
calls. Use it so ids you mint by hand match the ones the loop generates.
```swift
let id = generateId()
```
## Signature [#signature]
```swift
func generateId(
size: Int = 16
) -> String
```
Defined in `Sources/AI/Core/RuntimeContext.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `String`.
## See also [#see-also]
* [createIdGenerator](/docs/reference/create-id-generator)
# generateImage (/docs/reference/generate-image)
Generates images from a prompt, or edits an existing one when you pass image
data. `n` asks for several at once; when it exceeds what the provider allows
per call, the request is split into batches automatically.
```swift
let result = try await generateImage(
model: OpenAIImageModel("gpt-image-2"),
prompt: "A red bicycle against a white wall",
size: "1024x1024"
)
let png = result.images[0].bytes
```
## Signature [#signature]
```swift
func generateImage(
model: any ImageModel,
prompt: String,
images: [ImageContent] = [],
n: Int = 1,
size: String? = nil,
aspectRatio: String? = nil,
seed: Int? = nil,
providerOptions: JSONValue? = nil,
maxImagesPerCall: Int? = nil,
maxRetries: Int = 2
) async throws -> GenerateImageResult
```
Defined in `Sources/AI/Core/ImageGeneration.swift`.
## Parameters [#parameters]
## Returns [#returns]
`GenerateImageResult` with `images` as `GeneratedFile` values, exposing
`base64` and `bytes`, plus `providerMetadata`.
## See also [#see-also]
* [Image generation](/docs/image-generation)
# generateJSON (/docs/reference/generate-json)
Asks for valid JSON without constraining its shape, returning a `JSONValue`.
Use it when the shape genuinely is not known ahead of time. When you do know
it, [`generateObject`](/docs/reference/generate-object) is better in every
way: it validates, it decodes, and the model follows a schema more reliably
than an instruction.
```swift
let result = try await generateJSON(
model: OpenAIModel("gpt-5.1"),
prompt: "Summarize this log as JSON."
)
print(result.object["level"]?.stringValue ?? "")
```
## Signature [#signature]
```swift
func generateJSON(
model: any LanguageModel,
messages: [Message] = [],
system: String? = nil,
prompt: String? = nil,
maxOutputTokens: Int = 1024,
temperature: Double? = nil,
providerOptions: JSONValue? = nil,
maxRetries: Int = 2
) async throws -> GenerateObjectResult
```
Defined in `Sources/AI/Core/GenerateObject.swift`.
## Parameters [#parameters]
## Returns [#returns]
`GenerateObjectResult`.
## See also [#see-also]
* [generateObject](/docs/reference/generate-object)
# generateObjectArray (/docs/reference/generate-object-array)
Like [`generateObject`](/docs/reference/generate-object), but the schema
describes one element and the result is an array of them.
Use it for list extraction where the element shape is known and the count is
not: pulling every line item off an invoice, every attendee out of an email.
```swift
let result = try await generateObjectArray(
model: OpenAIModel("gpt-5.1"),
of: Attendee.self,
elementSchema: Schema.object([
"name": .string(), "email": .string()
]).jsonSchema,
prompt: email
)
```
## Signature [#signature]
```swift
func generateObjectArray(
model: any LanguageModel,
of type: T.Type = T.self,
elementSchema: JSONValue,
schemaName: String = "elements",
schemaDescription: String? = nil,
messages: [Message] = [],
system: String? = nil,
prompt: String? = nil,
maxOutputTokens: Int = 1024,
temperature: Double? = nil,
providerOptions: JSONValue? = nil,
maxRetries: Int = 2,
repairText: (@Sendable (String, any Error) async -> String?)? = nil
) async throws -> GenerateObjectResult<[T]>
```
Defined in `Sources/AI/Core/GenerateObject.swift`.
## Parameters [#parameters]
## Returns [#returns]
`GenerateObjectResult<[T]>`, with the array in `object`.
## See also [#see-also]
* [Structured output](/docs/structured-output)
* [generateObject](/docs/reference/generate-object)
# generateObject (/docs/reference/generate-object)
Constrains the model to a JSON schema and decodes the response into a
`Decodable` type. Validation happens before decoding, so a response that does
not fit the schema throws rather than producing a half-populated value.
This is the call for extraction, classification, and any time you need the
model's answer as data rather than prose. Pass `JSONValue.self` when you want
the raw object instead of a Swift type.
```swift
struct Recipe: Decodable, Sendable {
let title: String
let ingredients: [String]
}
let result = try await generateObject(
model: AnthropicModel("claude-sonnet-5"),
of: Recipe.self,
schema: Schema.object([
"title": .string(),
"ingredients": .array(of: .string())
]).jsonSchema,
prompt: "A recipe for lasagna."
)
print(result.object.title)
```
## Signature [#signature]
```swift
func generateObject(
model: any LanguageModel,
of type: T.Type = T.self,
schema: JSONValue,
schemaName: String = "response",
schemaDescription: String? = nil,
messages: [Message] = [],
system: String? = nil,
prompt: String? = nil,
maxOutputTokens: Int = 1024,
temperature: Double? = nil,
providerOptions: JSONValue? = nil,
maxRetries: Int = 2,
repairText: (@Sendable (String, any Error) async -> String?)? = nil
) async throws -> GenerateObjectResult
```
Defined in `Sources/AI/Core/GenerateObject.swift`.
## Parameters [#parameters]
## Overloads [#overloads]
1 other form of this function exists:
```swift
func generateObject(
model: any LanguageModel,
of type: T.Type = T.self,
schema: Schema,
schemaName: String = "response",
schemaDescription: String? = nil,
messages: [Message] = [],
system: String? = nil,
prompt: String? = nil,
maxOutputTokens: Int = 1024,
temperature: Double? = nil,
providerOptions: JSONValue? = nil,
maxRetries: Int = 2
) async throws -> GenerateObjectResult
```
## Returns [#returns]
`GenerateObjectResult` with the decoded `object`, the `rawJSON` it came
from, `finishReason`, and `usage`.
## See also [#see-also]
* [Structured output](/docs/structured-output)
* [generateObjectArray](/docs/reference/generate-object-array)
* [streamObject](/docs/reference/stream-object)
# generateSpeech (/docs/reference/generate-speech)
Turns text into audio using a speech model. `voice`, `speed`, and
`outputFormat` are passed through to the provider when it supports them.
```swift
let result = try await generateSpeech(
model: OpenAISpeechModel("gpt-4o-mini-tts"),
text: "Your order has shipped.",
voice: "alloy"
)
```
## Signature [#signature]
```swift
func generateSpeech(
model: any SpeechModel,
text: String,
voice: String? = nil,
instructions: String? = nil,
speed: Double? = nil,
outputFormat: String? = nil,
providerOptions: JSONValue? = nil,
maxRetries: Int = 2
) async throws -> GenerateSpeechResult
```
Defined in `Sources/AI/Core/SpeechGeneration.swift`.
## Parameters [#parameters]
## Returns [#returns]
`GenerateSpeechResult` with the `audio` file and `providerMetadata`.
## See also [#see-also]
* [Speech generation](/docs/speech-generation)
# GenerateTextResult (/docs/reference/generate-text-result)
The finished text is on `text`, but the result carries the whole run: every
step the loop took, the tool calls and results within them, token usage, the
reason generation stopped, and the provider's raw response.
Reach past `text` when you need to audit what happened: which tools ran, how
many steps it took, whether it stopped because the model finished or because it
hit a limit.
```swift
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: "Summarise this in one line.",
tools: [search]
)
print(result.text)
print("\(result.steps.count) steps, \(result.usage.totalTokens) tokens")
```
## Declaration [#declaration]
```swift
public struct GenerateTextResult
```
Defined in `Sources/AI/Core/GenerateText.swift`.
## Properties [#properties]
| Property | Type | Default |
| ------------------------ | --------------------- | ------- |
| `var text` | `String` | — |
| `var reasoningText` | `String` | — |
| `var toolCalls` | `[ToolCall]` | — |
| `var toolResults` | `[ToolResult]` | — |
| `var sources` | `[Source]` | — |
| `var steps` | `[StepResult]` | — |
| `var messages` | `[Message]` | — |
| `var providerMetadata` | `JSONValue?` | — |
| `var experimentalOutput` | `JSONValue?` | `nil` |
| `var finishReason` | `FinishReason` | — |
| `var usage` | `Usage` | — |
| `var stepCount` | `Int { steps.count }` | — |
## See also [#see-also]
* [generateText](/docs/reference/generate-text)
* [StreamTextResult](/docs/reference/stream-text-result)
# generateText (/docs/reference/generate-text)
Runs a model to completion and returns the whole result at once. If you pass
tools, it drives the full loop (call the model, execute the tools it asks for,
feed the results back) until a stop condition is met.
Reach for it when nothing is watching the output arrive: a background job, a
server route that returns JSON, a summarizer, an agent step. When a person is
watching the text appear, use [`streamText`](/docs/reference/stream-text)
instead.
```swift
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: "Invent a holiday and describe its traditions."
)
print(result.text)
```
## Signature [#signature]
```swift
func generateText(
model: any LanguageModel,
messages: [Message] = [],
system: String? = nil,
prompt: String? = nil,
tools: [any AIToolProtocol] = [],
toolChoice: ToolChoice = .auto,
activeTools: [String]? = nil,
toolOrder: [String]? = nil,
toolsContext: [String: JSONValue] = [:],
maxOutputTokens: Int = 1024,
temperature: Double? = nil,
topP: Double? = nil,
topK: Int? = nil,
presencePenalty: Double? = nil,
frequencyPenalty: Double? = nil,
seed: Int? = nil,
reasoning: ReasoningEffort = .providerDefault,
stopSequences: [String] = [],
providerOptions: JSONValue? = nil,
stopWhen: [StopCondition]? = nil,
maxSteps: Int = 8,
prepareCall: PrepareCall? = nil,
prepareStep: PrepareStep? = nil,
onStepFinish: OnStepFinish? = nil,
onFinish: (@Sendable (GenerateTextResult) async -> Void)? = nil,
onError: (@Sendable (Error) async -> Void)? = nil,
repairToolCall: (@Sendable (ToolCall, [any AIToolProtocol]) async throws -> ToolCall?)? = nil,
output: JSONValue? = nil,
maxRetries: Int = 2,
toolApproval: ToolApprovalPolicy? = nil,
toolApprovalSecret: String? = nil,
timeout: GenerationTimeout? = nil,
runtimeContext: JSONValue? = nil,
telemetry: TelemetrySettings? = nil,
compaction: Compaction? = nil
) async throws -> GenerateTextResult
```
Defined in `Sources/AI/Core/GenerateText.swift`.
## Parameters [#parameters]
## Returns [#returns]
`GenerateTextResult` carries the finished `text`, plus `reasoningText`,
`toolCalls`, `toolResults`, `sources`, `steps`, `messages`,
`providerMetadata`, `experimentalOutput`, `finishReason`, and `usage`.
`steps` is the per-iteration record of the tool loop, so `result.stepCount`
tells you how many model calls it took. `messages` is the conversation
including everything the loop appended, ready to pass into the next turn.
## See also [#see-also]
* [Generating text](/docs/generating-text)
* [streamText](/docs/reference/stream-text)
* [generateObject](/docs/reference/generate-object)
* [Timeouts and approvals](/docs/timeouts-and-approvals)
# generateVideo (/docs/reference/generate-video)
Video generation is asynchronous at every provider. This call submits the job
and polls until it finishes, so you await one result instead of managing the
job yourself.
Expect it to take minutes. Give the surrounding call a generous timeout.
```swift
let result = try await generateVideo(
model: XaiVideoModel("grok-video"),
prompt: "A timelapse of clouds over a city"
)
```
## Signature [#signature]
```swift
func generateVideo(
model: any VideoModel,
prompt: String,
image: ImageContent? = nil,
aspectRatio: String? = nil,
duration: Int? = nil,
providerOptions: JSONValue? = nil,
maxRetries: Int = 2
) async throws -> GenerateVideoResult
```
Defined in `Sources/AI/Core/VideoGeneration.swift`.
## Parameters [#parameters]
## Returns [#returns]
`GenerateVideoResult` with the `video` file and `providerMetadata`.
## See also [#see-also]
* [Video generation](/docs/video-generation)
# GenerationTimeout (/docs/reference/generation-timeout)
Two different failures need two different limits. A call that runs too long
overall is one problem; a stream that opens fine and then goes quiet is
another, and a total timeout catches the second one far too late.
So this carries both: a ceiling on the whole call, and a ceiling on the gap
between chunks. Either one firing cancels the request.
```swift
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: prompt,
timeout: GenerationTimeout(total: .seconds(60), stall: .seconds(10))
)
```
## Declaration [#declaration]
```swift
public struct GenerationTimeout
```
Defined in `Sources/AI/Core/Timeouts.swift`.
## Initializer [#initializer]
```swift
init(
total: Duration? = nil,
step: Duration? = nil,
firstChunk: Duration? = nil,
chunk: Duration? = nil,
tool: Duration? = nil,
tools: [String: Duration] = [:]
)
```
## Properties [#properties]
| Property | Type | Default |
| ---------------- | -------------------- | ------- |
| `var total` | `Duration?` | — |
| `var step` | `Duration?` | — |
| `var firstChunk` | `Duration?` | — |
| `var chunk` | `Duration?` | — |
| `var tool` | `Duration?` | — |
| `var tools` | `[String: Duration]` | — |
## Methods [#methods]
```swift
func limit(
forTool name: String
) -> Duration?
```
## See also [#see-also]
* [Timeouts and approvals](/docs/timeouts-and-approvals)
* [Timed out](/docs/troubleshooting/timed-out)
# getRealtimeToolDefinitions (/docs/reference/get-realtime-tool-definitions)
Maps ordinary SDK tools into the shape a realtime voice session expects, so
one tool array can serve both a text loop and a voice session.
```swift
let definitions = getRealtimeToolDefinitions(tools)
```
## Signature [#signature]
```swift
func getRealtimeToolDefinitions(
tools: [any AIToolProtocol]
) -> [RealtimeToolDefinition]
```
Defined in `Sources/AI/Realtime/RealtimeModel.swift`.
## Parameters [#parameters]
## Returns [#returns]
Realtime tool definitions for the session configuration.
## See also [#see-also]
* [Realtime voice](/docs/realtime)
# hasToolCall (/docs/reference/has-tool-call)
Ends the loop as soon as the model calls the tool you name. The usual pattern
is a terminal tool such as `submit_answer` or `finish`, which turns "the model
decided it is done" into a stop condition.
```swift
stopWhen: [hasToolCall("submit_answer")]
```
## Signature [#signature]
```swift
func hasToolCall(
_ toolNames: String...
) -> StopCondition
```
Defined in `Sources/AI/Core/StepControl.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `StopCondition`.
## See also [#see-also]
* [Agents](/docs/agents)
# API reference (/docs/reference)
Each page documents one symbol: what it does, when to reach for it, and its
exact signature.
Signatures here are extracted from `Sources/AI` on every build, so they cannot
drift from the code. The prose around them is written by hand. Everything else
in the docs teaches a concept or walks a task; this section is for looking
something up.
## Functions [#functions]
### Text [#text]
### Structured output [#structured-output]
### Embeddings and ranking [#embeddings-and-ranking]
### Images, audio, and video [#images-audio-and-video]
### Loop control [#loop-control]
### Context [#context]
### Middleware and providers [#middleware-and-providers]
### UI and transports [#ui-and-transports]
### MCP [#mcp]
### Audio buffers [#audio-buffers]
### Utilities [#utilities]
## Types [#types]
The types you construct or inspect directly. Everything else in the public
API is plumbing these hand back to you.
### Building blocks [#building-blocks]
### Results [#results]
### Configuration [#configuration]
### UI [#ui]
### Values [#values]
## Conventions [#conventions]
Swift argument labels are shown as written, so `_` means the parameter is
called positionally. A parameter with no default is **required**; everything
else can be omitted.
`any LanguageModel` and `any AIToolProtocol` appear throughout because the SDK
takes existentials rather than generics on those positions, which is what lets
you swap a provider without changing a call site.
Type pages list stored properties, initializers, and methods, including any
added in a `public extension`. Computed properties and protocol conformances
are left out.
For the error cases these functions throw, see the
[errors reference](/docs/reference/errors).
# isLoopFinished (/docs/reference/is-loop-finished)
Fires on the first step where the model returns no tool calls, which is the
loop's natural end. Rarely needed explicitly, since the loop already stops
there; useful when composing it with other conditions.
```swift
stopWhen: [isLoopFinished()]
```
## Signature [#signature]
```swift
func isLoopFinished() -> StopCondition
```
Defined in `Sources/AI/Core/StepControl.swift`.
## Returns [#returns]
A `StopCondition`.
## See also [#see-also]
* [Agents](/docs/agents)
# isStepCount (/docs/reference/is-step-count)
Fires when the loop reaches exactly this step, where
[`stepCountIs`](/docs/reference/step-count-is) fires at or past it. Use it
when you are composing conditions and need equality rather than a ceiling.
```swift
stopWhen: [isStepCount(3)]
```
## Signature [#signature]
```swift
func isStepCount(
_ count: Int
) -> StopCondition
```
Defined in `Sources/AI/Core/StepControl.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `StopCondition`.
## See also [#see-also]
* [stepCountIs](/docs/reference/step-count-is)
# JSONValue (/docs/reference/json-value)
The SDK's currency for anything whose shape isn't known at compile time:
tool arguments, tool results, provider options, runtime context.
It is an enum rather than `Any`, so it stays `Sendable` and its cases are
exhaustive. Accessors like `stringValue` unwrap the common cases without a
switch, and it is `ExpressibleBy` the usual literals, so writing one by hand
reads close to writing JSON.
```swift
let options: JSONValue = [
"thinking": ["type": "enabled", "budget_tokens": 2048]
]
let city = args["city"]?.stringValue ?? "London"
```
## Declaration [#declaration]
```swift
public enum JSONValue
```
Defined in `Sources/AI/Util/JSONValue.swift`.
## Cases [#cases]
```swift
case null
case bool(Bool)
case number(Double)
case string(String)
case array([JSONValue])
case object([String: JSONValue])
```
## Initializer [#initializer]
```swift
init(
from decoder: Decoder
) throws
```
## Properties [#properties]
| Property | Type | Default |
| ----------------- | ----------------------------------------------- | ---------------------------------------------- |
| `var stringValue` | `String? { if case .string(let s)` | `self { return s } else { return nil } }` |
| `var doubleValue` | `Double? { if case .number(let n)` | `self { return n } else { return nil } }` |
| `var intValue` | `Int? { if case .number(let n)` | `self { return Int(n) } else { return nil } }` |
| `var boolValue` | `Bool? { if case .bool(let b)` | `self { return b } else { return nil } }` |
| `var arrayValue` | `[JSONValue]? { if case .array(let a)` | `self { return a } else { return nil } }` |
| `var objectValue` | `[String: JSONValue]? { if case .object(let o)` | `self { return o } else { return nil } }` |
## Methods [#methods]
```swift
func encode(
to encoder: Encoder
) throws
```
```swift
func decode(
_ type: T.Type
) throws -> T
```
## See also [#see-also]
* [Structured output](/docs/structured-output)
* [Runtime context](/docs/runtime-context)
# lastAssistantMessageIsCompleteWithApprovalResponses (/docs/reference/last-assistant-message-is-complete-with-approval-responses)
The approval counterpart to
[`lastAssistantMessageIsCompleteWithToolCalls`](/docs/reference/last-assistant-message-is-complete-with-tool-calls):
true once every approval request in the last turn has a response, so the run
can continue.
```swift
if lastAssistantMessageIsCompleteWithApprovalResponses(messages) {
try await resume(messages)
}
```
## Signature [#signature]
```swift
func lastAssistantMessageIsCompleteWithApprovalResponses(
_ messages: [UIMessage]
) -> Bool
```
Defined in `Sources/AI/Transport/TextStreamChatTransport.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `Bool`.
## See also [#see-also]
* [Timeouts and approvals](/docs/timeouts-and-approvals)
# lastAssistantMessageIsCompleteWithToolCalls (/docs/reference/last-assistant-message-is-complete-with-tool-calls)
Returns true when the final assistant message's tool calls all have matching
results, meaning the turn is ready to continue. Used to decide whether to
resume a loop after a client round trip.
```swift
if lastAssistantMessageIsCompleteWithToolCalls(messages) {
try await resume(messages)
}
```
## Signature [#signature]
```swift
func lastAssistantMessageIsCompleteWithToolCalls(
_ messages: [UIMessage]
) -> Bool
```
Defined in `Sources/AI/Transport/TextStreamChatTransport.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `Bool`.
## See also [#see-also]
* [Chat UI](/docs/chat-ui)
# MCPClient (/docs/reference/mcp-client)
Connects to a Model Context Protocol server over stdio, streamable HTTP, or
the legacy SSE transport, and turns the tools it advertises into tools your
model can call.
The tools it returns are ordinary [`Tool`](/docs/reference/tool) values, so
they mix freely with ones you wrote yourself. Servers that require OAuth are
handled by attaching a session; the client refreshes and retries on a 401
rather than making you catch it.
```swift
let client = try await MCPClient(
transport: .streamableHTTP(url: URL(string: "https://mcp.example.com")!)
)
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: prompt,
tools: try await client.tools()
)
```
## Declaration [#declaration]
```swift
public actor MCPClient
```
Defined in `Sources/AI/MCP/MCPClient.swift`.
## Initializer [#initializer]
```swift
init(
transport: any MCPTransport
)
```
## Properties [#properties]
| Property | Type | Default |
| --------------- | ------------------ | ------- |
| `let transport` | `any MCPTransport` | — |
## Methods [#methods]
```swift
func close() async
```
```swift
func connect(
clientName: String = "swift-ai-sdk",
clientVersion: String = "0.3.0"
) async throws
```
```swift
func tools() async throws -> [any AIToolProtocol]
```
```swift
func callTool(
name: String,
arguments: JSONValue
) async throws -> JSONValue
```
## See also [#see-also]
* [MCP](/docs/mcp)
* [Authorization required](/docs/troubleshooting/mcp-authorization-required)
# Message (/docs/reference/message)
A role plus an array of [`ContentPart`](/docs/reference/content-part). Text is
the common case, but a single message can also carry images, files, tool calls,
tool results, and reasoning.
Messages are what you pass when a call needs history rather than a bare prompt.
Building them by hand is normal; the convenience initializers cover the simple
text case so you rarely spell out the parts.
Order matters and so does completeness: every tool call in an assistant message
needs a matching tool result before that conversation can go back to the model.
```swift
let messages: [Message] = [
.system("Answer in one sentence."),
.user("What is the tallest mountain?"),
.assistant("Mount Everest, at 8,849 metres."),
.user("And the second?")
]
```
## Declaration [#declaration]
```swift
public struct Message
```
Defined in `Sources/AI/Core/Message.swift`.
## Initializer [#initializer]
```swift
init(
role: Role,
content: [ContentPart]
)
```
## Properties [#properties]
| Property | Type | Default |
| ------------- | --------------- | ------- |
| `var role` | `Role` | — |
| `var content` | `[ContentPart]` | — |
## See also [#see-also]
* [Messages guide](/docs/messages)
* [ContentPart](/docs/reference/content-part)
* [convertToModelMessages](/docs/reference/convert-to-model-messages)
# pruneMessages (/docs/reference/prune-messages)
A pure function that deletes old tool calls, tool results, and (for
`UIMessage` histories) reasoning from a conversation. No model call, no cost.
Pruning is lossy by design. When the goal, the decisions, or the failed
approaches need to survive, use `compaction:` instead, which summarizes rather
than deletes.
```swift
let trimmed = pruneMessages(
messages,
toolCalls: .beforeLastMessages(6, tools: ["search"])
)
```
## Signature [#signature]
```swift
func pruneMessages(
_ messages: [UIMessage],
reasoning: PruneScope = .none,
toolCalls: [PruneToolCalls] = [],
emptyMessages: PruneEmptyMessages = .remove
) -> [UIMessage]
```
Defined in `Sources/AI/Core/PruneMessages.swift`.
## Parameters [#parameters]
## Overloads [#overloads]
3 other forms of this function exist:
```swift
func pruneMessages(
_ messages: [Message],
toolCalls: PruneToolCalls,
emptyMessages: PruneEmptyMessages = .remove
) -> [Message]
```
```swift
func pruneMessages(
_ messages: [Message],
toolCalls: [PruneToolCalls] = [],
emptyMessages: PruneEmptyMessages = .remove
) -> [Message]
```
```swift
func pruneMessages(
_ messages: [UIMessage],
reasoning: PruneScope = .none,
toolCalls: PruneToolCalls,
emptyMessages: PruneEmptyMessages = .remove
) -> [UIMessage]
```
## Returns [#returns]
A new array of the same type, with the pruned entries removed. Dropping a
tool call also drops its result and any approval response.
## See also [#see-also]
* [Context management](/docs/context-management)
* [filterActiveTools](/docs/reference/filter-active-tools)
# readUIMessageStream (/docs/reference/read-ui-message-stream)
Consumes the UI-message wire protocol and yields a growing `UIMessage` after
each chunk, so a view can render the latest snapshot without reducing chunks
itself.
```swift
for try await message in readUIMessageStream(stream: chunks) {
render(message)
}
```
## Signature [#signature]
```swift
func readUIMessageStream(
_ chunks: AsyncThrowingStream,
message: UIMessage? = nil
) -> AsyncThrowingStream
```
Defined in `Sources/AI/Transport/UIMessageStreamBuilder.swift`.
## Parameters [#parameters]
## Returns [#returns]
An `AsyncThrowingStream`.
## See also [#see-also]
* [Streaming protocol](/docs/streaming-protocol)
# rerank (/docs/reference/rerank)
Sends a query and a list of documents to a reranking model, which scores how
well each one answers the query. More accurate than embedding similarity
because the model sees the query and document together.
The usual shape is retrieve broadly with embeddings, then rerank the top
candidates before handing them to a model.
```swift
let result = try await rerank(
model: CohereRerankingModel("rerank-v3.5"),
query: "how do I cancel a subscription",
documents: candidates,
topN: 5
)
```
## Signature [#signature]
```swift
func rerank(
model: any RerankingModel,
query: String,
documents: [String],
topN: Int? = nil,
maxRetries: Int = 2
) async throws -> RerankResult
```
Defined in `Sources/AI/Core/Reranking.swift`.
## Parameters [#parameters]
## Returns [#returns]
`RerankResult` with scored, ordered `results` and `usage`.
## See also [#see-also]
* [Reranking](/docs/reranking)
# resampleAudio (/docs/reference/resample-audio)
Converts linear PCM from one sample rate to another, which realtime sessions
need when a microphone's rate does not match what the provider expects.
```swift
let resampled = resampleAudio(samples, from: 48_000, to: 24_000)
```
## Signature [#signature]
```swift
func resampleAudio(
_ input: [Float],
inputRate: Int,
outputRate: Int
) -> [Float]
```
Defined in `Sources/AI/Realtime/AudioResampling.swift`.
## Parameters [#parameters]
## Overloads [#overloads]
1 other form of this function exists:
```swift
func resampleAudio(
_ input: Data,
inputRate: Int,
outputRate: Int
) -> Data
```
## Returns [#returns]
The resampled samples.
## See also [#see-also]
* [Realtime voice](/docs/realtime)
# safeValidateUIMessages (/docs/reference/safe-validate-ui-messages)
The non-throwing form of
[`validateUIMessages`](/docs/reference/validate-ui-messages), returning a
result you can branch on. Use it at a request boundary where you would rather
return a 400 than surface an error.
```swift
switch safeValidateUIMessages(body.messages) {
case .success(let messages): try await handle(messages)
case .failure(let error): return .badRequest(error)
}
```
## Signature [#signature]
```swift
func safeValidateUIMessages(
_ messages: [UIMessage]
) -> Result<[UIMessage], Error>
```
Defined in `Sources/AI/Transport/TextStreamChatTransport.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `UIMessageValidation` describing success or the failure.
## See also [#see-also]
* [Chat UI](/docs/chat-ui)
# smoothStream (/docs/reference/smooth-stream)
Providers emit deltas at whatever granularity they like, which can look jittery
in a UI. This re-chunks the stream by word or line and paces it, so text
appears at a steady rhythm.
Cosmetic only. It changes when text is delivered, never what it says.
```swift
let smoothed = smoothStream(stream: result.textStream, chunking: .word)
```
## Signature [#signature]
```swift
func smoothStream(
_ input: AsyncThrowingStream,
chunking: SmoothStreamChunking = .word,
delay: Duration? = .milliseconds(10)
) -> AsyncThrowingStream
```
Defined in `Sources/AI/Core/StreamSmoothing.swift`.
## Parameters [#parameters]
## Returns [#returns]
An `AsyncThrowingStream`.
## See also [#see-also]
* [Chat UI](/docs/chat-ui)
# stepCountIs (/docs/reference/step-count-is)
The stop condition you will use most. Caps how many times the loop may call
the model, which bounds both cost and the chance of an agent spinning.
```swift
stopWhen: [stepCountIs(5)]
```
## Signature [#signature]
```swift
func stepCountIs(
_ count: Int
) -> StopCondition
```
Defined in `Sources/AI/Core/StepControl.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `StopCondition`.
## See also [#see-also]
* [Agents](/docs/agents)
* [hasToolCall](/docs/reference/has-tool-call)
# StopCondition (/docs/reference/stop-condition)
By default the loop runs until the model stops asking for tools or hits
`maxSteps`. A stop condition lets you end it on your own terms: after a
particular tool has been called, once a step count is reached, or on any
predicate you write over the steps so far.
Conditions compose. Pass several and the loop stops when any one is met.
```swift
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: prompt,
tools: [search, finish],
stopWhen: [hasToolCall("finish"), stepCountIs(10)]
)
```
## Declaration [#declaration]
```swift
public struct StopCondition
```
Defined in `Sources/AI/Core/StepControl.swift`.
## Initializer [#initializer]
```swift
init(
_ predicate: @escaping @Sendable ([StepResult]) -> Bool
)
```
## Methods [#methods]
```swift
func isMet(
_ steps: [StepResult]
) -> Bool
```
## See also [#see-also]
* [Agents guide](/docs/agents)
* [stepCountIs](/docs/reference/step-count-is)
* [hasToolCall](/docs/reference/has-tool-call)
# streamObject (/docs/reference/stream-object)
Streams partial versions of a structured result while the model is still
generating it, so a UI can render fields as they land instead of waiting for
the closing brace.
For array schemas, `elementStream` yields each complete element as it
finishes, which is usually what you want for a list that renders row by row.
```swift
let stream = streamObject(
model: AnthropicModel("claude-sonnet-5"),
schema: schema,
prompt: "Three startup ideas."
)
for try await partial in stream.partialObjectStream {
render(partial)
}
```
## Signature [#signature]
```swift
func streamObject(
model: any LanguageModel,
schema: JSONValue,
schemaName: String = "response",
schemaDescription: String? = nil,
messages: [Message] = [],
system: String? = nil,
prompt: String? = nil,
maxOutputTokens: Int = 1024,
temperature: Double? = nil,
providerOptions: JSONValue? = nil,
maxRetries: Int = 2
) -> StreamObjectResult
```
Defined in `Sources/AI/Core/GenerateObject.swift`.
## Parameters [#parameters]
## Overloads [#overloads]
1 other form of this function exists:
```swift
func streamObject(
model: any LanguageModel,
schema: Schema,
schemaName: String = "response",
schemaDescription: String? = nil,
messages: [Message] = [],
system: String? = nil,
prompt: String? = nil,
maxOutputTokens: Int = 1024,
temperature: Double? = nil,
providerOptions: JSONValue? = nil,
maxRetries: Int = 2
) -> StreamObjectResult
```
## Returns [#returns]
`StreamObjectResult` with `partialObjectStream`, `elementStream` for array
schemas, and awaitable `object`, `usage`, and `finishReason`.
## See also [#see-also]
* [Structured output](/docs/structured-output)
* [generateObject](/docs/reference/generate-object)
# streamTextDeltas (/docs/reference/stream-text-deltas)
Streams raw text deltas straight from a model with no tool loop, no steps, and
no result accumulation. It is the smallest possible streaming call.
Use it when you want tokens and nothing else, or when you are building your own
loop on top of the model protocol. Most applications want
[`streamText`](/docs/reference/stream-text).
```swift
for try await delta in streamTextDeltas(
model: OllamaModel("qwen3"),
prompt: "Count to ten."
) {
print(delta, terminator: "")
}
```
## Signature [#signature]
```swift
func streamTextDeltas(
model: any LanguageModel,
messages: [Message] = [],
system: String? = nil,
prompt: String? = nil,
maxOutputTokens: Int = 1024,
temperature: Double? = nil
) -> AsyncThrowingStream
```
Defined in `Sources/AI/Core/GenerateText.swift`.
## Parameters [#parameters]
## Returns [#returns]
An `AsyncThrowingStream` of text deltas.
## See also [#see-also]
* [streamText](/docs/reference/stream-text)
# StreamTextResult (/docs/reference/stream-text-result)
What `streamText` hands back before the model has finished. Iterate it to
consume parts as they arrive, whether those are text deltas, tool calls, tool
results, or reasoning. Or await the finished result if you decide mid-flight
that you want the whole thing after all.
The stream is single-pass. If you need both the live parts and the final
result, consume the stream and read the result afterwards rather than trying to
iterate twice.
```swift
let stream = streamText(
model: AnthropicModel("claude-sonnet-5"),
prompt: "Write a haiku about deadlines."
)
for try await delta in stream.textStream {
print(delta, terminator: "")
}
```
## Declaration [#declaration]
```swift
public struct StreamTextResult
```
Defined in `Sources/AI/Core/StreamText.swift`.
## Properties [#properties]
| Property | Type | Default |
| ---------------- | -------------------------------------------- | ------- |
| `let fullStream` | `AsyncThrowingStream` | — |
## Methods [#methods]
```swift
func smoothedTextStream(
chunking: SmoothStreamChunking = .word,
delay: Duration? = .milliseconds(10)
) -> AsyncThrowingStream
```
```swift
func consumeStream(
onError: (@Sendable (Error) async -> Void)? = nil
) async
```
## See also [#see-also]
* [streamText](/docs/reference/stream-text)
* [GenerateTextResult](/docs/reference/generate-text-result)
# streamText (/docs/reference/stream-text)
The streaming counterpart to `generateText`, with the same tool loop and the
same parameters. Instead of waiting for a finished result you consume an
`AsyncSequence` of parts as the model produces them.
Use it anywhere a person is waiting on output. The stream carries more than
text: reasoning deltas, tool call starts, tool results, sources, and provider
metadata all arrive in order, so a UI can show the model working rather than a
spinner.
```swift
let stream = streamText(
model: OpenAIModel("gpt-5.1"),
prompt: "Write a haiku about Swift concurrency."
)
for try await text in stream.textStream {
print(text, terminator: "")
}
```
## Signature [#signature]
```swift
func streamText(
model: any LanguageModel,
messages: [Message] = [],
system: String? = nil,
prompt: String? = nil,
tools: [any AIToolProtocol] = [],
toolChoice: ToolChoice = .auto,
activeTools: [String]? = nil,
toolOrder: [String]? = nil,
toolsContext: [String: JSONValue] = [:],
maxOutputTokens: Int = 1024,
temperature: Double? = nil,
topP: Double? = nil,
topK: Int? = nil,
presencePenalty: Double? = nil,
frequencyPenalty: Double? = nil,
seed: Int? = nil,
reasoning: ReasoningEffort = .providerDefault,
stopSequences: [String] = [],
providerOptions: JSONValue? = nil,
stopWhen: [StopCondition]? = nil,
maxSteps: Int = 8,
prepareCall: PrepareCall? = nil,
prepareStep: PrepareStep? = nil,
onStepFinish: OnStepFinish? = nil,
onFinish: (@Sendable (GenerateTextResult) async -> Void)? = nil,
onError: (@Sendable (Error) async -> Void)? = nil,
onChunk: (@Sendable (TextStreamPart) -> Void)? = nil,
onAbort: (@Sendable () async -> Void)? = nil,
repairToolCall: (@Sendable (ToolCall, [any AIToolProtocol]) async throws -> ToolCall?)? = nil,
maxRetries: Int = 2,
toolApproval: ToolApprovalPolicy? = nil,
toolApprovalSecret: String? = nil,
timeout: GenerationTimeout? = nil,
runtimeContext: JSONValue? = nil,
telemetry: TelemetrySettings? = nil,
compaction: Compaction? = nil
) -> StreamTextResult
```
Defined in `Sources/AI/Core/StreamText.swift`.
## Parameters [#parameters]
## Returns [#returns]
`StreamTextResult` exposes several views of the same run. `textStream` yields
only text deltas; `fullStream` yields every `TextStreamPart` including
reasoning, tool activity, and metadata. Awaiting `text`, `steps`, `usage`, or
`finishReason` gives you the finished values once the stream completes.
## See also [#see-also]
* [Generating text](/docs/generating-text)
* [generateText](/docs/reference/generate-text)
* [smoothStream](/docs/reference/smooth-stream)
* [Chat UI](/docs/chat-ui)
# streamTranscribe (/docs/reference/stream-transcribe)
Streams audio to a transcription model over a WebSocket and yields text as it
is recognized.
Interim guesses arrive as `.partialTranscript`, which **replaces** the current
partial, while settled text arrives as `.transcriptDelta`, which **appends**.
Keeping those separate is what stops revisions from double-counting.
```swift
for try await part in streamTranscribe(
model: DeepgramTranscriptionModel("nova-3"),
audio: micStream
) {
switch part {
case .partialTranscript(let text): showInterim(text)
case .transcriptDelta(let text): append(text)
default: break
}
}
```
## Signature [#signature]
```swift
func streamTranscribe(
model: any TranscriptionModel,
audio: AsyncThrowingStream,
mediaType: String,
providerOptions: JSONValue? = nil
) throws -> StreamTranscriptionResult
```
Defined in `Sources/AI/Core/StreamTranscription.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `StreamTranscriptionResult` whose stream yields `TranscriptionStreamPart`
values, plus an awaitable final `result`.
## See also [#see-also]
* [Transcription](/docs/transcription)
* [transcribe](/docs/reference/transcribe)
# TelemetrySettings (/docs/reference/telemetry-settings)
Turns on tracing for a call: spans for each step, tool execution, and model
request, with token counts attached. Off by default, because recording prompts
and completions is a decision about user data, not a default.
Enable it per call or per agent. Recording of prompt and completion text is a
separate switch from recording the spans themselves.
```swift
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: prompt,
telemetry: TelemetrySettings(isEnabled: true, functionID: "summarise")
)
```
## Declaration [#declaration]
```swift
public struct TelemetrySettings
```
Defined in `Sources/AI/Core/RuntimeContext.swift`.
## Initializer [#initializer]
```swift
init(
isEnabled: Bool = true,
functionID: String? = nil,
metadata: [String: JSONValue] = [:],
includeRuntimeContext: [String]? = nil,
includeToolsContext: [String]? = nil
)
```
## Properties [#properties]
| Property | Type | Default |
| --------------------------- | --------------------- | ------- |
| `var isEnabled` | `Bool` | — |
| `var functionID` | `String?` | — |
| `var metadata` | `[String: JSONValue]` | — |
| `var includeRuntimeContext` | `[String]?` | — |
| `var includeToolsContext` | `[String]?` | — |
## See also [#see-also]
* [Telemetry](/docs/telemetry)
# ToolApprovalPolicy (/docs/reference/tool-approval-policy)
Some tools shouldn't fire on the model's say-so alone. Anything that spends
money, sends a message, or deletes something. A policy sits between the model
asking and the tool running, and pauses the loop until you approve.
You can require approval for everything, for nothing, or for a named set. The
loop surfaces the pending call, waits, and resumes with your answer.
```swift
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: prompt,
tools: [search, sendEmail],
toolApproval: .requiring(["send_email"])
)
```
## Declaration [#declaration]
```swift
public struct ToolApprovalPolicy
```
Defined in `Sources/AI/Core/ToolApproval.swift`.
## Initializers [#initializers]
```swift
init(
_ decide: @escaping @Sendable (ToolApprovalContext) async -> ToolApprovalDecision
)
```
```swift
init(
dictionaryLiteral elements: (String, ToolApprovalDecision)...
)
```
```swift
init(
_ decisions: [String: ToolApprovalDecision]
)
```
## Methods [#methods]
```swift
func decide(
_ context: ToolApprovalContext
) async -> ToolApprovalDecision
```
## See also [#see-also]
* [Timeouts and approvals](/docs/timeouts-and-approvals)
* [Approvals guide](/docs/guides/approvals)
# ToolChoice (/docs/reference/tool-choice)
`.auto` lets the model decide, which is what you want almost always. The
other cases are for the moments when you need to take that decision away:
forcing a tool call on the first step, or forbidding tools while the model
writes its final answer.
Forcing a specific tool is a useful trick for structured extraction. Give the
model one tool whose schema is the shape you want back.
```swift
let result = try await generateText(
model: AnthropicModel("claude-sonnet-5"),
prompt: prompt,
tools: [extractInvoice],
toolChoice: .tool("extract_invoice")
)
```
## Declaration [#declaration]
```swift
public enum ToolChoice
```
Defined in `Sources/AI/Core/LanguageModel.swift`.
## Cases [#cases]
```swift
case auto
case none
case required
case tool(String)
```
## See also [#see-also]
* [Tools guide](/docs/tools)
* [Tool](/docs/reference/tool)
# Tool (/docs/reference/tool)
A tool is a name, a description, a schema for its arguments, and a closure
that runs when the model calls it. The description isn't decoration. It's the
only thing the model reads when deciding whether this tool applies, so it's
worth as much care as the code.
The loop in `generateText` and `streamText` executes tools for you and feeds
their results back to the model. You only write the closure.
Modifiers on a tool change how the loop treats it. `.idempotent()` marks a
tool as safe to call again with the same arguments, which lets compaction drop
its result knowing it can be recovered.
```swift
let weather = Tool(
name: "get_weather",
description: "Current conditions for a city.",
parameters: .object(["city": .string(description: "City name")])
) { args, _ in
let city = args["city"]?.stringValue ?? "London"
return .string(try await fetchWeather(city))
}.idempotent()
```
## Declaration [#declaration]
```swift
public struct Tool
```
Defined in `Sources/AI/Core/Tool.swift`.
## Initializers [#initializers]
```swift
init(
name: String,
description: String,
parameters: JSONValue,
inputExamples: [JSONValue] = [],
needsApproval: Bool = false,
execute: @escaping @Sendable (JSONValue, ToolExecutionOptions) async throws -> JSONValue
)
```
```swift
init(
name: String,
description: String,
parameters: JSONValue,
inputExamples: [JSONValue] = [],
needsApproval: Bool = false,
execute: @escaping @Sendable (JSONValue) async throws -> JSONValue
)
```
```swift
init(
name: String,
description: String,
parameters: JSONValue,
inputExamples: [JSONValue] = [],
needsApproval: @escaping @Sendable (JSONValue) async -> Bool,
execute: @escaping @Sendable (JSONValue) async throws -> JSONValue
)
```
```swift
init(
name: String,
description: String,
parameters: JSONValue,
inputExamples: [JSONValue] = []
)
```
## Properties [#properties]
| Property | Type | Default | | |
| ------------------------- | -------------------------------------------- | ------- | - | ------------------------ |
| `let name` | `String` | — | | |
| `var description` | `String` | — | | |
| `let parameters` | `JSONValue` | — | | |
| `var inputExamples` | `[JSONValue]` | `[]` | | |
| `var contextSchema` | `Schema?` | — | | |
| `var isDynamic` | `Bool` | `false` | | |
| `var isIdempotent` | `Bool` | `false` | | |
| `var loading` | `ToolLoading` | `.none` | | |
| `var describeWithContext` | `(@Sendable (JSONValue?) -> String)?` | — | | |
| `var modelOutput` | `(@Sendable (JSONValue) -> [ContentPart]?)?` | `nil` | | |
| `var hasExecutor` | `Bool { run !` | \`nil | | contextualRun != nil }\` |
## Methods [#methods]
```swift
func toModelOutput(
_ output: JSONValue
) -> [ContentPart]?
```
```swift
func needsApproval(
_ arguments: JSONValue
) async -> Bool
```
```swift
func description(
context: JSONValue?
) -> String
```
```swift
func resolvingDescription(
_ description: String
) -> any AIToolProtocol
```
```swift
func execute(
_ arguments: JSONValue
) async throws -> JSONValue
```
```swift
func execute(
_ arguments: JSONValue,
options: ToolExecutionOptions
) async throws -> JSONValue
```
```swift
func withContextSchema(
_ schema: Schema
) -> Tool
```
```swift
func idempotent(
_ isIdempotent: Bool = true
) -> Tool
```
```swift
func describing(
_ describe: @escaping @Sendable (JSONValue?) -> String
) -> Tool
```
```swift
func loading(
_ loading: ToolLoading
) -> Tool
```
## See also [#see-also]
* [Tools guide](/docs/tools)
* [Agent](/docs/reference/agent)
* [ToolChoice](/docs/reference/tool-choice)
# transcribe (/docs/reference/transcribe)
Transcribes recorded audio. When the declared media type is generic, the
format is sniffed from the bytes themselves, so MP4, M4A, WAV, Ogg, FLAC, and
MP3 are recognized without you naming them.
For microphone input that should transcribe as it arrives, use
[`streamTranscribe`](/docs/reference/stream-transcribe).
```swift
let result = try await transcribe(
model: DeepgramTranscriptionModel("nova-3"),
audio: try Data(contentsOf: url)
)
print(result.text)
```
## Signature [#signature]
```swift
func transcribe(
model: any TranscriptionModel,
audio: Data,
mediaType: String,
providerOptions: JSONValue? = nil,
maxRetries: Int = 2
) async throws -> TranscriptionResult
```
Defined in `Sources/AI/Core/Transcription.swift`.
## Parameters [#parameters]
## Returns [#returns]
`TranscriptionResult` with `text`, `segments`, detected `language`,
`durationInSeconds`, and `providerMetadata`.
## See also [#see-also]
* [Transcription](/docs/transcription)
* [streamTranscribe](/docs/reference/stream-transcribe)
# uploadFile (/docs/reference/upload-file)
Uploads bytes through any `FileUploadAPI` (OpenAI Files, Anthropic Files) and
returns a handle you can attach to a message. The resulting
`providerReference` reaches OpenAI as `file_id` and Anthropic as a `file`
source, and is ignored by providers it was not minted for.
```swift
let file = try await uploadFile(
api: OpenAIFiles(),
data: pdf,
filename: "report.pdf"
)
```
## Signature [#signature]
```swift
func uploadFile(
api: any FileUploadAPI,
data: Data,
filename: String,
mediaType: String = "application/octet-stream",
purpose: String = "user_data"
) async throws -> UploadedFile
```
Defined in `Sources/AI/Providers/FileUploads.swift`.
## Parameters [#parameters]
## Returns [#returns]
An `UploadedFile`.
## See also [#see-also]
* [Files and skills](/docs/files-and-skills)
# validateUIMessages (/docs/reference/validate-ui-messages)
Checks that messages from a client are well formed before you hand them to a
model. Throws on the first problem.
For a non-throwing check, use
[`safeValidateUIMessages`](/docs/reference/safe-validate-ui-messages).
```swift
let messages = try validateUIMessages(body.messages)
```
## Signature [#signature]
```swift
func validateUIMessages(
_ messages: [UIMessage]
) throws -> [UIMessage]
```
Defined in `Sources/AI/Transport/TextStreamChatTransport.swift`.
## Parameters [#parameters]
## Returns [#returns]
The validated `[UIMessage]`.
## See also [#see-also]
* [Chat UI](/docs/chat-ui)
# wrapEmbeddingModel (/docs/reference/wrap-embedding-model)
The embedding-model counterpart to
[`wrapLanguageModel`](/docs/reference/wrap-language-model), using
`EmbeddingModelMiddleware`.
```swift
let embeddings = wrapEmbeddingModel(
model: OpenAIEmbeddingModel("text-embedding-3-small"),
middleware: [.defaultSettings(maxBatchSize: 96)]
)
```
## Signature [#signature]
```swift
func wrapEmbeddingModel(
model: any EmbeddingModel,
middleware: [EmbeddingModelMiddleware]
) -> any EmbeddingModel
```
Defined in `Sources/AI/Core/ModelMiddleware.swift`.
## Parameters [#parameters]
## Returns [#returns]
An `any EmbeddingModel`.
## See also [#see-also]
* [Middleware](/docs/middleware)
# wrapImageModel (/docs/reference/wrap-image-model)
The image-model counterpart to
[`wrapLanguageModel`](/docs/reference/wrap-language-model). Commonly used
to append house style to every prompt.
```swift
let images = wrapImageModel(
model: OpenAIImageModel("gpt-image-2"),
middleware: [ImageModelMiddleware(transformRequest: { request in
var request = request
request.prompt += ", studio lighting"
return request
})]
)
```
## Signature [#signature]
```swift
func wrapImageModel(
model: any ImageModel,
middleware: [ImageModelMiddleware]
) -> any ImageModel
```
Defined in `Sources/AI/Core/ModelMiddleware.swift`.
## Parameters [#parameters]
## Returns [#returns]
An `any ImageModel`.
## See also [#see-also]
* [Middleware](/docs/middleware)
# wrapLanguageModel (/docs/reference/wrap-language-model)
Returns a model that behaves like the one you passed in, with middleware
intercepting requests and stream parts. Middlewares apply in array order.
Built-ins cover caching, reasoning extraction, default settings, simulated
streaming, JSON fence stripping, and folding tool input examples into
descriptions.
```swift
let model = wrapLanguageModel(
model: OllamaModel("qwen3"),
middleware: [.cache(), .extractReasoning(tag: "think")]
)
```
## Signature [#signature]
```swift
func wrapLanguageModel(
model: any LanguageModel,
middleware: [LanguageModelMiddleware]
) -> any LanguageModel
```
Defined in `Sources/AI/Core/Middleware.swift`.
## Parameters [#parameters]
## Returns [#returns]
An `any LanguageModel` you can use anywhere the original worked.
## See also [#see-also]
* [Middleware](/docs/middleware)
# wrapProvider (/docs/reference/wrap-provider)
Applies middleware at the provider level, so every model the provider hands
out is already wrapped. Saves repeating the same wrapping at each call site.
```swift
let provider = wrapProvider(
provider: myProvider,
languageModelMiddleware: [.cache()]
)
```
## Signature [#signature]
```swift
func wrapProvider(
provider: ProviderRegistry.Provider,
languageModelMiddleware: [LanguageModelMiddleware] = [],
embeddingModelMiddleware: [EmbeddingModelMiddleware] = [],
imageModelMiddleware: [ImageModelMiddleware] = []
) -> ProviderRegistry.Provider
```
Defined in `Sources/AI/Core/ModelMiddleware.swift`.
## Parameters [#parameters]
## Returns [#returns]
A `WrappedProvider`.
## See also [#see-also]
* [Middleware](/docs/middleware)