Introducing AnyLanguageModel: One API for Local and Remote LLMs on Apple Platforms

Introducing AnyLanguageModel: A Unified API for Local and Remote LLMs on Apple Platforms

As AI-powered applications become increasingly popular, developers are facing a significant challenge: integrating language models into their software. For Apple developers, this process is often unnecessarily painful due to the various APIs, requirements, and integration patterns associated with different models. In this article, we'll introduce AnyLanguageModel, a Swift package that aims to simplify the process of working with large language models (LLMs) on Apple platforms.

The Problem

Developers building AI-powered apps typically adopt a hybrid approach, combining local models using Core ML or MLX for privacy and offline capability with cloud providers like OpenAI or Anthropic for frontier capabilities. Apple's Foundation Models also serve as a system-level fallback. However, each of these options comes with different APIs, requirements, and integration patterns, making it a daunting task for developers to manage.

When we spoke with developers about building AI-powered apps, friction with model integration was a common theme. One developer succinctly put it: "I thought I'd quickly use the demo for a test and maybe a quick and dirty build, but instead wasted so much time. Drove me nuts." The cost of experimentation is high, which discourages developers from discovering that local, open-source models might actually work great for their use case.

The Solution

AnyLanguageModel aims to reduce the friction of working with LLMs on Apple platforms and make it easier to adopt open-source models that run locally. The core idea is simple: swap your import statement, keep the same API.

// Import FoundationModels
import FoundationModels

// Import AnyLanguageModel
import AnyLanguageModel

Here's an example of how to use AnyLanguageModel:

// Create a local model using MLX
let model = MLXLanguageModel(modelId: "mlx-community/Qwen3-4B-4bit")
let session = LanguageModelSession(model: model)

// Create a cloud model using OpenAI
let model = OpenAILanguageModel(
    apiKey: ProcessInfo.processInfo.environment["OPENAI_API_KEY"]!,
    model: "text-davinci-003"
)
let session = LanguageModelSession(model: model)

AnyLanguageModel supports a range of providers, including:

• Apple Foundation Models: Native integration with Apple's system model (macOS 26+ / iOS 26+)
• Core ML: Run converted models with Neural Engine acceleration
• MLX: Run quantized models efficiently on Apple Silicon
• llama.cpp: Load GGUF models via the llama.cpp backend
• Ollama: Connect to locally-served models via Ollama's HTTP API
• OpenAI, Anthropic, Google Gemini: Cloud providers for comparison and fallback
• Hugging Face Inference Providers: Hundreds of cloud models powered by world-class inference providers.

Why Foundation Models as the Base API

When designing AnyLanguageModel, we faced a choice: create a new abstraction that tries to capture everything, or build on an existing API. We chose the latter, using Apple's Foundation Models framework as the template.

This might seem counterintuitive. Why tie ourselves to Apple's choices? A few reasons:

• Foundation Models is genuinely well-designed. It leverages Swift features like macros for an ergonomic developer experience, and its abstractions around sessions, tools, and generation map well to how LLMs actually work.
• It's intentionally limited. Foundation Models represents something like a lowest common denominator for language model capabilities. Rather than seeing this as a weakness, we treat it as a stable foundation. Every Swift developer targeting Apple platforms will encounter this API, so building on it directly means less conceptual overhead.
• It keeps us grounded. Each additional layer of abstraction takes you further from the problem you're actually solving. Abstractions are powerful, but stack too many and they become a problem in themselves.

Package Traits: Include Only What You Need

One challenge with multi-backend libraries is dependency bloat. If you only want to run MLX models, you shouldn't have to pull in llama.cpp and all its dependencies. AnyLanguageModel uses Swift 6.1 package traits to solve this. You opt in to only the backends you need:

// Import AnyLanguageModel with MLX trait
dependencies: [
    .package(
        url: "https://github.com/mattt/AnyLanguageModel.git",
        from: "0.4.0",
        traits: ["MLX"]  // Pull in MLX dependencies only
    )
]

Available traits include CoreML, MLX, and Llama (for llama.cpp / llama.swift). By default, no heavy dependencies are included. You get the base API plus cloud providers, which only require standard URLSession networking.

Image Support (and API Design Trade-offs)

Vision-language models are incredibly capable and widely used. They can describe images, extract text from screenshots, analyze charts, and answer questions about visual content. Unfortunately, Apple's Foundation Models framework doesn't currently support sending images with prompts. Building on an existing API means accepting its constraints. Apple will likely add image support in a future release (iOS 27, perhaps?), but vision-language models are too useful to wait for. So we've extended beyond what Foundation Models offers today.

Here's an example sending an image to Claude:

// Create a cloud model using Anthropic
let model = AnthropicLanguageModel(
    apiKey: ProcessInfo.processInfo.environment["ANTHROPIC_API_KEY"]!,
    model: "claude-sonnet-4-5-20250929"
)

// Create a session with image support
let session = LanguageModelSession(model: model)
let response = try await session.respond(
    to: "What's in this image?",
    image: .init(url: URL(fileURLWithPath: "/path/to/image.png"))
)

We're taking a calculated risk here; we might design something that conflicts with Apple's eventual implementation. But that's what deprecation warnings are for. Sometimes you have to write the API for the framework that doesn't exist yet.

Try It Out: chat-ui-swift

To see AnyLanguageModel in action, check out chat-ui-swift, a SwiftUI chat application that demonstrates the library's capabilities. The app includes:

• Apple Intelligence integration via Foundation Models (macOS 26+)
• Hugging Face OAuth authentication for accessing gated models
• Streaming responses
• Chat persistence

It's meant as a starting point: Fork it, extend it, swap in different models. See how the pieces fit together and adapt it to your needs.

What's Next

AnyLanguageModel is currently pre-1.0. The core API is stable, but we're actively working on bringing the full feature set of Foundation Models to all adapters, namely:

• Tool calling across all providers
• MCP integration for tools and elicitations
• Guided generation for structured outputs
• Performance optimizations for local inference

This library is the first step toward something larger. A unified inference API provides the scaffolding needed to build seamless agentic workflows on Apple platforms — applications where models can use tools, access system resources, and accomplish complex tasks. More on that soon.

AnyLanguageModel is available on GitHub, and chat-ui-swift is available on GitHub as well. We're excited to see what you build.

Source: https://huggingface.co/blog/anylanguagemodel