AnyLanguageModel

A Swift package that provides a drop-in replacement for Apple's Foundation Models framework with support for custom language model providers. All you need to do is change your import statement:

- import FoundationModels
+ import AnyLanguageModel

import AnyLanguageModel

struct WeatherTool: Tool {
    let name = "getWeather"
    let description = "Retrieve the latest weather information for a city"

    @Generable
    struct Arguments {
        @Guide(description: "The city to fetch the weather for")
        var city: String
    }

    func call(arguments: Arguments) async throws -> String {
        "The weather in \(arguments.city) is sunny and 72°F / 23°C"
    }
}

let model = SystemLanguageModel.default
let session = LanguageModelSession(model: model, tools: [WeatherTool()])

let response = try await session.respond {
    Prompt("How's the weather in Cupertino?")
}
print(response.content)

Features

Supported Providers

• Apple Foundation Models
• Core ML models
• MLX models
• llama.cpp (GGUF models)
• Ollama HTTP API
• Anthropic Messages API
• Google Gemini API
• OpenAI Chat Completions API
• OpenAI Responses API

Requirements

• Swift 6.1+
• iOS 17.0+ / macOS 14.0+ / visionOS 1.0+

Installation

Add this package to your Package.swift:

dependencies: [
    .package(url: "https://github.com/mattt/AnyLanguageModel.git", from: "0.2.0")
]

Conditional Dependencies

AnyLanguageModel uses Swift 6.1 traits to conditionally include heavy dependencies, allowing you to opt-in only to the language model backends you need. This results in smaller binary sizes and faster build times.

Available traits:

• CoreML: Enables Core ML model support (depends on huggingface/swift-transformers)
• MLX: Enables MLX model support (depends on ml-explore/mlx-swift-examples)
• Llama: Enables llama.cpp support (requires mattt/llama.swift)

By default, no traits are enabled. To enable specific traits, specify them in your package's dependencies:

// In your Package.swift
dependencies: [
    .package(
        url: "https://github.com/mattt/AnyLanguageModel.git",
        branch: "main",
        traits: ["CoreML", "MLX"] // Enable CoreML and MLX support
    )
]

Usage

Apple Foundation Models

Uses Apple's system language model (requires macOS 26 / iOS 26 / visionOS 26 or later).

let model = SystemLanguageModel.default
let session = LanguageModelSession(model: model)

let response = try await session.respond {
    Prompt("Explain quantum computing in one sentence")
}

Core ML

Run Core ML models (requires CoreML trait):

let model = CoreMLLanguageModel(url: URL(fileURLWithPath: "path/to/model.mlmodelc"))

let session = LanguageModelSession(model: model)
let response = try await session.respond {
    Prompt("Summarize this text")
}

Enable the trait in Package.swift:

.package(
    url: "https://github.com/mattt/AnyLanguageModel.git",
    branch: "main",
    traits: ["CoreML"]
)

MLX

Run MLX models on Apple Silicon (requires MLX trait):

let model = MLXLanguageModel(modelId: "mlx-community/Qwen3-0.6B-4bit")

let session = LanguageModelSession(model: model)
let response = try await session.respond {
    Prompt("What is the capital of France?")
}

Enable the trait in Package.swift:

.package(
    url: "https://github.com/mattt/AnyLanguageModel.git",
    branch: "main",
    traits: ["MLX"]
)

llama.cpp (GGUF)

Run GGUF quantized models via llama.cpp (requires Llama trait):

let model = LlamaLanguageModel(modelPath: "/path/to/model.gguf")

let session = LanguageModelSession(model: model)
let response = try await session.respond {
    Prompt("Translate 'hello world' to Spanish")
}

Enable the trait in Package.swift:

.package(
    url: "https://github.com/mattt/AnyLanguageModel.git",
    branch: "main",
    traits: ["Llama"]
)

OpenAI

Supports both Chat Completions and Responses APIs:

let model = OpenAILanguageModel(
    apiKey: ProcessInfo.processInfo.environment["OPENAI_API_KEY"]!,
    model: "gpt-4o-mini"
)

let session = LanguageModelSession(model: model)
let response = try await session.respond {
    Prompt("Write a haiku about Swift")
}

For OpenAI-compatible endpoints that use older Chat Completions API:

let model = OpenAILanguageModel(
    baseURL: URL(string: "https://api.example.com")!,
    apiKey: apiKey,
    model: "gpt-4o-mini",
    apiVariant: .chatCompletions
)

Anthropic

Uses the Messages API with Claude models:

let model = AnthropicLanguageModel(
    apiKey: ProcessInfo.processInfo.environment["ANTHROPIC_API_KEY"]!,
    model: "claude-sonnet-4-5-20250929"
)

let session = LanguageModelSession(model: model, tools: [WeatherTool()])
let response = try await session.respond {
    Prompt("What's the weather like in San Francisco?")
}

Google Gemini

Uses the Gemini API with Gemini models:

let model = GeminiLanguageModel(
    apiKey: ProcessInfo.processInfo.environment["GEMINI_API_KEY"]!,
    model: "gemini-2.5-flash"
)

let session = LanguageModelSession(model: model, tools: [WeatherTool()])
let response = try await session.respond {
    Prompt("What's the weather like in Tokyo?")
}

Gemini models use an internal "thinking process" that improves reasoning and multi-step planning. You can configure how much Gemini should "think" using the thinking parameter:

// Enable thinking
var model = GeminiLanguageModel(
    apiKey: apiKey,
    model: "gemini-2.5-flash",
    thinking: true /* or `.dynamic` */,
)

// Set an explicit number of tokens for its thinking budget
model.thinking = .budget(1024)

// Revert to default configuration without thinking
model.thinking = false /* or `.disabled` */

Gemini supports server-side tools that execute transparently on Google's infrastructure:

let model = GeminiLanguageModel(
    apiKey: apiKey,
    model: "gemini-2.5-flash",
    serverTools: [
        .googleMaps(latitude: 35.6580, longitude: 139.7016) // Optional location
    ]
)

Available server tools:

• .googleSearch Grounds responses with real-time web information
• .googleMaps Provides location-aware responses
• .codeExecution Generates and runs Python code to solve problems
• .urlContext Fetches and analyzes content from URLs mentioned in prompts

Tip

Gemini server tools are not available as client tools (Tool) for other models.

Ollama

Run models locally via Ollama's HTTP API:

// Default: connects to http://localhost:11434
let model = OllamaLanguageModel(model: "qwen3")

// Custom endpoint
let model = OllamaLanguageModel(
    endpoint: URL(string: "http://remote-server:11434")!,
    model: "llama3.2"
)

let session = LanguageModelSession(model: model)
let response = try await session.respond {
    Prompt("Tell me a joke")
}

First, pull the model: ollama pull qwen3:0.6b

Testing

Run the test suite to verify everything works correctly:

swift test

Tests for different language model backends have varying requirements:

• CoreML tests: swift test --enable-trait CoreML + ENABLE_COREML_TESTS=1 + HF_TOKEN (downloads model from HuggingFace)
• MLX tests: swift test --enable-trait MLX + ENABLE_MLX_TESTS=1 + HF_TOKEN (uses pre-defined model)
• Llama tests: swift test --enable-trait Llama + LLAMA_MODEL_PATH (points to local GGUF file)
• Anthropic tests: ANTHROPIC_API_KEY (no traits needed)
• OpenAI tests: OPENAI_API_KEY (no traits needed)
• Ollama tests: No setup needed (skips in CI)

Example setup for all backends:

# Environment variables
export ENABLE_COREML_TESTS=1
export ENABLE_MLX_TESTS=1
export HF_TOKEN=your_huggingface_token
export LLAMA_MODEL_PATH=/path/to/model.gguf
export ANTHROPIC_API_KEY=your_anthropic_key
export OPENAI_API_KEY=your_openai_key

# Run all tests with traits enabled
swift test --enable-trait CoreML --enable-trait MLX --enable-trait Llama