Messages and multimodal

For why history is a list at all, what each role is for, and why tool calls and results must stay paired, see prompts and messages.

Messages

A conversation is [Message]. Convenience initializers cover the common cases; full content arrays cover the rest.

var history: [Message] = [
  .system("You are terse."),
  .user("What is in this photo?"),
  .assistant("A lighthouse at dusk.")
]

// Full control over parts:
let message = Message(role: .user, content: [
  .text("Compare these two:"),
  .image(ImageContent(data: firstJPEG, mediaType: "image/jpeg")),
  .image(ImageContent(url: secondURL, mediaType: "image/png"))
])

Roles are .system, .user, .assistant, and .tool. The loop manages .tool turns for you; you only write them when replaying persisted history.

Vision and files

Images and files ride as content parts and map to each provider's native shape (image_url on OpenAI, base64 source on Anthropic, inlineData on Gemini, content blocks on Bedrock):

let result = try await generateText(
  model: GoogleModel("gemini-3.5-flash"),
  messages: [Message(role: .user, content: [
    .text("Summarize this report."),
    .file(FileContent(data: pdfData, mediaType: "application/pdf", filename: "q3.pdf"))
  ])]
)

Both ImageContent and FileContent take inline data, a remote url, or a providerReference from uploadFile. In chat UIs, user attachments arrive as file parts and convertToModelMessages decodes data URLs into inline bytes automatically.

Not every model can fetch a URL itself — Bedrock's Converse API, for one, takes bytes only. Models declare this with supportsRemoteURL(_:mediaType:), and the loop downloads and inlines anything a model cannot fetch before the request goes out, so the same message works everywhere instead of silently losing its attachment. Only http(s) URLs are fetched (a file:// URL raises rather than being read), downloads are capped at 32 MB, and data: URLs and provider references are left untouched.

Content parts

JSONValue

JSONValue is the currency for tool arguments, provider options, and structured output. It is ExpressibleBy*Literal, so it reads like JSON:

let arguments: JSONValue = [
  "city": "Mumbai",
  "days": 3,
  "units": ["metric", "imperial"],
  "detailed": true
]

arguments["city"]?.stringValue     // "Mumbai"
arguments["days"]?.intValue        // 3
arguments["missing"]?.boolValue    // nil

// Decode into Codable whenever you want types:
struct Query: Decodable { let city: String; let days: Int }
let query = try arguments.decode(Query.self)

Usage

Every result carries token accounting, including provider cache and reasoning details where the wire reports them:

result.usage.inputTokens
result.usage.outputTokens
result.usage.totalTokens
result.usage.cachedInputTokens    // prompt cache hits (OpenAI, Anthropic, Groq, DeepSeek)
result.usage.reasoningTokens      // thinking tokens (reasoning models)