AIChatKitMLX adds on-device Apple MLX inference to any app already using AIChatKit. Models download from Hugging Face Hub on first use and are cached locally after that. Runs on Metal GPU and the Apple Neural Engine — no network calls during inference, for both text-only LLMs and vision-language models.
MLXProvider() picks a model based on the device's available RAM: a ~26B MoE vision-language model (~4B active params) on Macs with 16GB+ RAM, or a lean ~4B text-only model everywhere else. Call MLXProvider.recommendedModelId() to check which one will load.
Models are pulled from Hugging Face Hub on first use and cached — every call after that runs entirely on-device with no network dependency. The cache is shared with Python's huggingface_hub, so models already downloaded via Python tools are reused automatically.
loadModel(progressHandler:) pre-warms the model and reports fractionCompleted so you can render a real download progress bar instead of a spinner.
When the larger model is selected, the underlying ModelContainer supports image input via UserInput — no separate integration path for text vs. vision.
Pass any mlx-community Hub ID, or a pre-downloaded local directory. The correct factory (VLM or LLM) is chosen automatically from the model's config.json.
MLXProvider implements the same AIChatCore protocol as the cloud providers — swap it into an existing ChatSession without touching the rest of your chat UI.
.package(url: "https://github.com/NerdSnipe-Inc/AIChatKit", from: "0.1.0"),
.package(url: "https://github.com/NerdSnipe-Inc/AIChatKitMLX", from: "0.1.0").product(name: "AIChatCore", package: "AIChatKit"),
.product(name: "AIChatUI", package: "AIChatKit"),
.product(name: "AIChatMLX", package: "AIChatKitMLX")Requires AIChatKit and Apple Silicon (M-series Mac or A-series iPhone/iPad). Do not add this target to builds that must run on Intel Macs or the Simulator.
After the initial Hugging Face download, MLXProvider makes no network calls. Chat history, prompts, and generated text stay on-device — useful for apps that need to work fully offline or avoid sending conversations to a third-party API.
No. MLX requires Apple Silicon. Don't add the AIChatMLX target to builds that need to run on Intel Macs or the iOS Simulator.
Roughly 2–3 GB for the smaller text-only model, and 8–10 GB for the larger vision-language model on Macs with 16GB+ RAM.
Yes — pass any mlx-community Hugging Face model ID to MLXProvider(modelId:), or a local directory path to MLXProvider(modelPath:).
Add AIChatKitMLX alongside AIChatKit and swap in MLXProvider() — same ChatSession API, no network dependency after the first model download.