dalao.ai

Rapid-MLX

An open-source LLM inference server for Apple Silicon built on MLX, with OpenAI- and Anthropic-compatible APIs and a focus on reliable tool calling for coding agents, Apache-2.0-licensedOpen source

About Rapid-MLX

Rapid-MLX is an open-source LLM inference server for Apple Silicon, built on MLX, with OpenAI- and Anthropic-compatible APIs and a focus on reliable tool calling for coding agents. It runs as a CLI or a macOS desktop app, with models running on the Mac itself, and both the engine and the desktop app are free. It is licensed under the Apache License 2.0, with a project header at the top of the LICENSE file.

EditorialUpdated 8 min read

What problem it solves

The GitHub description calls Rapid-MLX an open-source OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX and focused on reliable tool calling for coding agents.

Problem and approach. The homepage says one Mac can answer Cursor, Claude Code, agents and in-house apps at the same time, with callers sharing the machine instead of queueing; the site describes Rapid-MLX as a server first, using continuous batching to work on calls that arrive together in the same pass.

History. The NOTICE says Rapid-MLX began as vLLM-MLX, created by Wayner Barrios, and the repository's initial commit, dated 2025-12-06, is that project; it was renamed Rapid-MLX on 2026-03-13, and since then the codebase has been extensively modified, extended and in large part rewritten, with the Python package now using the rapid_mlx import name and keeping vllm_mlx as a deprecated compatibility shim. The NOTICE also states that Rapid-MLX is not affiliated with, and is not a distribution of, the vLLM project. The GitHub repository was created on 2026-02-25 08:41 (UTC+8). Recent releases: v0.16.7 on 2026-10-22 07:53 (UTC+8), v0.16.8 on 2026-10-23 04:42 (UTC+8) and v0.16.9 on 2026-10-25 21:36 (UTC+8), each with a matching rapid-mac desktop app release.

Team: its predecessor vLLM-MLX was created by Wayner Barrios; contact entity named in the terms of service: MachineFi Lab.

Up to 4× faster than Apple's MLX (mlx-lm), 1.5× on a typical task — same model, same weights.

Rapid-MLX README tagline

Core capabilities

Compatible APIs. The README lists OpenAI's /v1/chat/completions and /v1/responses and Anthropic Messages at /v1/messages, served by default at localhost:8000/v1; clients that accept an OpenAI- or Anthropic-compatible endpoint can usually connect without an adapter.

Tool calling. The README cites 27 tool-call parser modules, including an auto-detect fallback, and the site says they recover automatically when quantized models degrade (as reported by the vendor).

Reasoning models. The site says chain-of-thought is separated for Qwen3.x, GLM-5.x, gpt-oss, Gemma 4 and DeepSeek, keeping thinking out of the answer stream.

Caching and concurrency. The README lists continuous batching, an in-memory radix prefix cache with state snapshots for hybrid models that is saved to disk on shutdown and restored at startup, and a quantized KV cache.

Multimodal. The site says vision, image and video runtimes ship in the base install; audio has 44 aliases covering TTS, speech-to-text and zero-shot voice cloning; video generation covers Wan 2.2, CogVideoX-Fun and LTX-2.3/2.5 behind an asynchronous /v1/videos API. The README says image and video generation need Python 3.11+. The homepage stat bar gives 264 MLX-native models, while the FAQ on the same page says it supports 268 models across 15 families; both cover text, vision, audio and video (as reported by the vendor).

Hugging Face import. The site says that before any download it inspects weight format, architecture, chat template and memory fit, in its words A model it can prove won't run is refused, with the reason.; unquantized checkpoints convert with one explicit command.

Desktop app. The site says the Desktop app offers local chat and local dictation with Whisper, Qwen3-ASR, SenseVoice or Parakeet, and the app is signed and notarised by Apple.

Community benchmarks. The community benchmark notes say results from rapid-mlx benchmark run stay on the Mac and are uploaded to the site leaderboard only after a confirmed share.

Who uses it

As of 2026-10-26 the GitHub API shows 4,029 stars and 438 forks.

As of 2026-10-26, PyPI Stats shows 5,511 downloads of rapid-mlx in the last 30 days; downloads include repeat and automated fetches and are not a count of installs or users.

Integrations. The README says five Tier-1 agents, Claude Code, Codex CLI, Hermes, Aider and DeepSeek Harness, must pass an end-to-end test on real weights before every release, or the version cannot publish; agents such as OpenCode, Qwen Code and OpenHands are tested in the compatibility matrix each release (as reported by the vendor).

Community projects. The homepage's From the community panel lists rapidmlx-menubar, a menu-bar tool, and mlx-Chronos, an independent benchmark suite that measures Rapid-MLX among other engines.

How it compares

Against mlx-lm: mlx-lm is published by Apple's ml-explore team under the MIT licence and is also one of Rapid-MLX's runtime dependencies. The mlx-lm benchmark page says that on a Mac mini M4 Pro 48 GB with Qwen3.5-9B 4-bit, greedy decoding and both servers at their defaults, Rapid-MLX 0.15.6 reached up to 4.27× over mlx-lm 0.32.0 on a single task, a median decode speedup of 1.50× across 18 streaming tasks and a median of 1.37× end to end over 20 tasks (as reported by the vendor, measured 2026-10-04). The page attributes the gain to speculative decoding on by default, through MTP and prompt lookup; mlx-lm needs a separate draft model to use speculative decoding.

Against oMLX: oMLX is Apache-2.0, and its GitHub description reads LLM inference server with continuous batching & SSD caching for Apple Silicon, managed from the macOS menu bar. The homepage How it compares table says that on a Mac Studio M5 Max 36 GB with Qwen3.5-9B 4-bit, single-stream decode was 115.5 tok/s for Rapid-MLX 0.16.1, 77.9 tok/s for oMLX 0.7.0 and 81.0 tok/s for LM Studio, with a single run per engine and Rapid-MLX using speculative decoding on a text-only lane by default (as reported by the vendor, measured 2026-10-12).

Against Ollama: Ollama is MIT. The README feature table says Ollama has desktop apps for macOS and Windows and offers an MLX engine alongside its GGML/GGUF engine; Rapid-MLX supports only Apple Silicon and runs on MLX.

Who it's for / Not for

Good fit:

  • Developers running local models on Apple Silicon Macs and serving them to coding agents such as Claude Code and Codex CLI.
  • Small teams planning to leave one always-on Mac as a LAN inference endpoint for several apps at once.
  • Local apps that need both OpenAI and Anthropic API formats.

Not a fit:

  • Intel Mac, Windows or Linux users: the site FAQ requires an Apple Silicon Mac, M1 or newer, on macOS 14 or later, and the README says Windows and Linux desktop builds are not available yet.
  • Selection processes that rely on independent third-party benchmarks: the site's benchmark figures are measured by the vendor, some as single runs.
  • Direct Cursor connections to the Mac: the site FAQ says Cursor calls models from its own servers, so it needs an HTTPS tunnel to reach the Mac.

Getting started

Installation. The README gives two routes: brew install rapid-mlx, a prebuilt bottle from homebrew-core, or the guided installer curl -fsSL https://rapidmlx.com/install.sh | bash. The site FAQ says the script probes for Python 3.10+, creates a virtual environment at ~/.rapid-mlx and symlinks rapid-mlx into ~/.local/bin, with no sudo required. The README also lists uv and pip, with pip needing Python 3.10+, and links install security notes.

Serving. rapid-mlx serve <model> downloads the model on first run and serves at http://localhost:8000/v1; rapid-mlx chat opens a chat; rapid-mlx recipe lists suggested models for the Mac's memory.

Agents. The README says that after installing a coding agent, a command such as rapid-mlx launch codex connects it; rapid-mlx launch codex --remove removes Rapid's private profile without changing existing Codex settings.

Multi-machine use. The site says rapid-mlx serve --host 0.0.0.0 runs it headless, with --trusted-hosts set to the IP or host name clients use and --api-key for authentication.

Desktop. The site offers a macOS app download that requires macOS 14 or later on Apple Silicon; the Desktop page says 16 GB of RAM is comfortable and 8 GB works for smaller models.

Pricing & license

Price: free. The homepage says the engine and the Desktop app are both free and open source under Apache 2.0, with no account and no cloud inference. The terms of use say the Desktop app is distributed free of charge for personal or commercial use, but a modified build may not be redistributed as though it were an official release.

Licence: the LICENSE file opens with a project header reading Rapid-MLX and Copyright 2025-2026 the Rapid-MLX contributors, states that it is licensed under the Apache License, Version 2.0, and notes that portions originate in other Apache-2.0 and MIT licensed projects, with attribution in the NOTICE file; the full Apache License 2.0 text follows the header. The README, the site, pyproject.toml and PyPI all mark it Apache-2.0, matching the LICENSE. The GitHub API licence field shows NOASSERTION.

NOTICE. The repository root has a NOTICE file. The NOTICE lists third-party code in the repository with its licences, among them MIT, Apache-2.0, CC0 1.0 and BSD 3-Clause; the runtime dependencies MLX, mlx-lm, mlx-vlm and mlx-audio are MIT. The NOTICE says optional Python extras pull in additional dependencies under their own terms, including copyleft licences, and those extras are not part of the Desktop app.

Model weights. The NOTICE says no model weights are distributed with the software; weights are downloaded at runtime from their publishers under their own licences, and the terms of use say some models forbid commercial use.

Risks & limits

Telemetry on by default. The privacy page says anonymous usage telemetry is on by default after a notice and can be turned off in Desktop Settings → Privacy or with rapid-mlx telemetry off, and setting RAPID_MLX_TELEMETRY=0 or DO_NOT_TRACK=1 also disables it; the Desktop app runs one update check per launch that sends only the app version. The privacy page says prompts, conversations, file contents and API keys are never collected.

Online tools. The site says models run on the Mac, while online tools send their requests to their own services.

Share Compute. This is an experimental feature that is off until enabled in settings. The guide says it lets the Mac serve live QuickSilver requests in exchange for QuickSilver API credit, which is not a cash payout; it needs a QuickSilver provider key, and the sharing model is unavailable to local chat while sharing runs.

Streaming issues. P1 streaming issue #5259, opened 2026-10-26 03:27 (UTC+8) and titled reasoning-only finalization copies private reasoning into visible answer lanes, was closed at 2026-10-26 09:28 (UTC+8); #5260, opened at the same time, on Responses and Messages streams emitting native tool envelopes alongside parsed calls, was closed at 2026-10-26 07:34 (UTC+8).

Version gap. As of 2026-10-26 the stable rapid-mlx version in homebrew-core is 0.15.7, while the latest GitHub release is v0.16.9.

Code origin. The NOTICE says neither the original authors nor any upstream project endorses this distribution.

Benchmarks. The site's benchmark table notes a single run per engine, with Rapid-MLX 0.16.1 using speculative decoding on a text-only lane by default (as reported by the vendor).

Our verdict

What sets Rapid-MLX apart: it runs on Apple Silicon Macs, speaks both OpenAI and Anthropic APIs and puts tool-call parsing, continuous batching and prefix caching into one server; it is licensed under Apache 2.0, and the Desktop app is free.

Its limits: it runs only on Apple Silicon; its speed figures are measured by the vendor; telemetry is on by default and can be switched off.

Sources

Official

  1. Rapid-MLX — Up to 4× Faster Than Apple's MLX | Local LLMs on Apple Siliconrapidmlx.com · Accessed Oct 26, 2026
  2. Rapid-MLX vs mlx-lm, up to 4× faster than Apple's MLX, measured task by taskrapidmlx.com · Accessed Oct 26, 2026
  3. Rapid-MLX vs oMLX vs LM Studio: 1.50× 27B decode on M5 Maxrapidmlx.com · Accessed Oct 26, 2026
  4. Rapid-MLX 0.16.9 — Smoother Desktop chats and reliable model loadingrapidmlx.com · Accessed Oct 26, 2026
  5. Rapid-MLX Desktop (macOS app page)rapidmlx.com · Accessed Oct 26, 2026
  6. Privacy | Rapid-MLXrapidmlx.com · Accessed Oct 26, 2026
  7. Terms of Use | Rapid-MLXrapidmlx.com · Accessed Oct 26, 2026
  8. Rapid-MLX READMEGitHub · Accessed Oct 26, 2026

Updates & reading