dalao.ai

Exo

Cluster your Macs to run models too big for oneOpen source

About Exo

exo, maintained by Exo Labs, is local inference-cluster software: it links Macs over Thunderbolt or a LAN, auto-discovers devices, splits models across them, and serves OpenAI-, Claude- and Ollama-compatible APIs. Today's exo is the 1.x rewrite of late 2025, Apache-2.0 since 2025-12-18; the old GPL-3.0 version is archived. Only Apple silicon Macs are really supported: Linux is CPU-only and phones are out. The latest release, v1.0.71, shipped on 2026-04-23; 48,036 stars as of 2026-10-25. The API has no authentication and listens on every interface, by design per the maintainers.

EditorialUpdated 15 min read

The Exo interface

What it is

exo is open-source software that turns several devices into an inference cluster. Each machine runs an exo process; the processes discover each other automatically and then, based on each machine's memory, compute and links, split a large model across several machines. To the user the whole cluster looks like one machine: opening localhost:52415 in a browser shows a dashboard, and programs can call it in OpenAI Chat Completions, OpenAI Responses, Claude Messages or Ollama format.

Today's exo is the 1.x version. Its only inference backend is Apple's MLX, and machines talk to each other over MLX distributed; on Macs running macOS 26.2 with Thunderbolt 5 ports it can also use direct RDMA. The main code is Python (3.13 required), with some Rust bindings; the dashboard is a web frontend and the macOS app is written in Swift. The repo description is a single line: "Run frontier AI locally".

The maintainer is Exo Labs. The FAQ on the company website says Exo Labs is the company behind exo and local.ai; the repo's licence file names the copyright holder as Exo Technologies Ltd. VentureBeat reported on 2024-11-14 that the company was founded in March 2024 by co-founders Alex Cheema and Mohamed Baioumy.

As of 2026-10-25 the GitHub repo had 48,036 stars, 3,580 forks and 98 contributors with GitHub accounts, with 210 open issues and 213 open PRs. The last commit on main, bc412e2, was at 2026-10-14 04:11 (UTC+8).

this is intended behaviour, exo's api is unauthorized. run exo behind a reverse proxy and provide your own authentication if you wish to expose it downstream.

An exo maintainer, replying on issue #1833

Core capabilities

Automatic discovery and networking. Machines running exo on the same network find each other automatically, with no hand-written node list. Here the release and main differ. In the latest release, v1.0.71 (2026-04-23), the networking layer is libp2p, and its README (lines 230 and 303) says that to keep your cluster separate from someone else's on the same network you set EXO_LIBP2P_NAMESPACE, which the macOS app exposes under advanced settings. Commit 09f9ea3 on main at 2026-06-03 23:31 (UTC+8) switched the networking layer to zenoh and renamed the variable EXO_ZENOH_NAMESPACE, but from then on the variable was only written to the logs and had no effect. Commit b1afa48 at 2026-10-12 14:54 (UTC+8) says in its message that after the move to zenoh the macOS app's namespace setting was silently ignored, and all nodes of the same version on the same network merged into one cluster. In other words, from 2026-06-03 to 2026-10-12, clusters run from main's source couldn't be isolated with this variable. That commit fixed the problem and kept the old variable name as a deprecated fallback. Neither zenoh nor the fix is in any release yet.

Topology-aware model splitting. The README says exo decides how to split a model from the live device topology, taking into account each machine's resources and each link's latency and bandwidth. Besides placing different layers on different machines, 1.x also supports tensor parallelism, splitting the same layer across machines; the README gives speedups of up to 1.8× on 2 machines and up to 3.2× on 4.

RDMA over Thunderbolt 5. macOS 26.2 added RDMA over Thunderbolt 5, and Jeff Geerling's article calls it the headline feature of exo 1.0. The README says latency between machines drops by 99%; Jeff Geerling's test article of 2025-12-18 puts it at from about 300 microseconds to under 50. That article tested 4 M3 Ultra Mac Studios: Qwen3 235B ran at about 32 tokens per second across the whole cluster, and Kimi K2 Thinking at about 30 tokens per second.

Separate prefill and decode. An Exo Labs blog post of 2025-10-15 demonstrated a combination: an NVIDIA DGX Spark does the compute-heavy prefill, then streams the KV cache layer by layer to a Mac Studio with higher memory bandwidth for decoding. In the post's test with Llama-3.1 8B and an 8,192-token prompt, the combination took 2.32 seconds in total against 6.42 seconds on an M3 Ultra alone, a 2.8× speedup. The demo relies on CUDA on Linux, while the public release supports only CPU on Linux.

Compatible APIs. The four API formats let existing clients connect with almost no changes; custom models can be pulled from Hugging Face. For offline use, set EXO_OFFLINE to use only models already on disk.

Who it's for / Not for

Good fit:

  • Individuals or small teams who already have two or more recent Apple silicon Macs and want to run large models that don't fit in one machine's memory.
  • People with Thunderbolt 5 machines who are willing to fiddle with Recovery mode and cables for multi-machine speedups.
  • Developers who need to serve a local model to tools such as Claude Code and Cursor on an internal network, provided they can manage the network boundary themselves.
  • Engineers studying distributed inference who want a working codebase to read.

Poor fit:

  • Users with only NVIDIA GPUs or Windows PCs: the public release uses only CPU on Linux, and Windows has no planned near-term support.
  • People hoping to pool old phones and tablets into a cluster: that was the selling point of the 2024 version, and 1.x doesn't support it.
  • Teams that need multi-user authentication and audit logs: the API has no logins or tokens, and the maintainers suggest adding your own reverse proxy.
  • Production setups that need a steady release cadence: the latest release is v1.0.71 from 2026-04-23, with no new release as of 2026-10-25.

How it compares

Versus llama.cpp's RPC mode. llama.cpp can also split a model across several machines, by handing different layers to remote ggml-rpc-server instances, and it supports mixing CUDA, Metal and CPU. Its own docs say the feature is still a proof of concept, "fragile and insecure", and should not be run on an open network. Jeff Geerling's article of 2025-12-18 says llama.cpp's RPC spreads layers across nodes in a way that "scales but is inefficient", getting slower as nodes are added in most tests; though when testing DeepSeek V3.1, llama.cpp also gained slightly on two nodes, which surprised the author himself. In the same article exo got faster as nodes were added in the Qwen3 235B test, and of the two Geerling compared, llama.cpp and exo, only exo supported RDMA at the time. The author tested pre-release software, and the article has a section headed "Stability Issues"; in his words, "They performed great... when they performed." llama.cpp is free under the MIT licence and had 132,278 stars as of 2026-10-25.

Versus distributed-llama. This project is also pitched as a way to "Connect home devices into a powerful cluster", using tensor parallelism synchronised over Ethernet; it supports Linux, macOS and Windows, is optimised for ARM and x86 CPUs, and has experimental Vulkan support. It is free under the MIT licence and had 3,114 stars as of 2026-10-25. Its hardware coverage is wider than exo's and its model list narrower, and models must first be converted to its own format.

Versus GPUStack. GPUStack targets GPU server clusters, scheduling inference engines such as vLLM and SGLang, and leans toward data-centre operations. It is free under Apache-2.0 and had 5,886 stars as of 2026-10-25.

Basis for prices: llama.cpp, distributed-llama, GPUStack and exo are all free to use under the open-source licences stated in their repos, and the cost is mainly hardware. exo's difference is that it uses the Mac's unified memory and Thunderbolt 5 RDMA, at the price of working almost only on Macs.

Getting started

There is an official macOS app. Download the v1.0.71 dmg (about 413 MB) from GitHub Releases, or install the exo cask with Homebrew. The steps below come from the v1.0.71 release README. The app runs in the background; on first launch a dialog explains that it needs to install a system service, and the user can click "Not Now" to skip it. Clicking install and entering an administrator password installs a system-level LaunchDaemon called io.exo.networksetup. It runs once when loaded and then every 1,786 seconds, and each run creates and switches to a network location named exo, turns off the Thunderbolt Bridge, and uses PlistBuddy to remove bridge0 from the system network configuration to prevent network loops. So a Thunderbolt Bridge the user turns on manually is turned off again at the next run. To uninstall, run sudo ./app/EXO/uninstall-exo.sh in the repo directory; the script stops the daemon and switches back to the "Automatic" network location.

System requirements are written differently in different places: the README and the Exo Labs website FAQ both say the app needs macOS Tahoe 26.2 or later; the Homebrew page says macOS 15 or later, arm64 only; and the app's auto-update feed says a minimum of 15.1. RDMA needs macOS 26.2.

Running from source. Per the v1.0.71 README, macOS needs uv, Node, Rust and Xcode's Metal toolchain: clone the repo, build the dashboard, then run uv run exo; if Nix is set up, run nix run .#exo in the cloned repo directory. On Linux you likewise build from source and run uv run exo, using only CPU. The README on main as of 2026-10-14 changed this to first install the CPU backend with uv sync --extra mlx-cpu, and tells NVIDIA users to use mlx-cuda13 or mlx-cuda12 instead; those option names come from commit 4466cd5 of 2026-05-13, and the corresponding options in v1.0.71's pyproject.toml are called cuda12 and cuda13. When running from source, the cluster-isolation variable is EXO_LIBP2P_NAMESPACE in the release and EXO_ZENOH_NAMESPACE on main.

Per the README, enabling RDMA means shutting down, holding the power button for 10 seconds to reach startup options, running rdma_ctl enable in the Recovery-mode Terminal, then restarting. The caveats the README lists: every machine taking part in RDMA must be directly connected to every other; cables must support Thunderbolt 5; on the Mac Studio, the Thunderbolt 5 port next to the Ethernet port can't be used; and every machine must run exactly the same macOS version, down to the beta build number.

Connecting tools. The dashboard and API are both on port 52415; OpenAI or Ollama clients connect by pointing their address at that port.

Pricing & license

The software is free. exo is released under Apache-2.0, with no charge for personal or commercial use, and there are no licence keys or activation checks in the code. The Exo Labs website has an "EXO for Business" entry; enterprise pricing is by contacting sales, through the inquiry form or by writing to sales@exolabs.net. On company funding, VentureBeat reported on 2024-11-14 that it had raised money from private investors.

The real cost is hardware. Jeff Geerling's article of 2025-12-18 gives prices for the setup used: M3 Ultra Mac Studios with 512 GB unified memory and 32 CPU cores at $11,699 each, and 256 GB models at $8,099 each, two of each, which the article calls "just shy of $40,000" in total; Apple lent the machines. The basis for the prices is the per-unit prices listed in that article, excluding cables and rack.

Licence history. The main repo's first licence file was added by Alex Cheema in commit bde1e53 ("add license") at 2024-07-17 05:06 (UTC+8), the full GPL-3.0 text. Commit 0fcee70 ("prep repo for v1") at 2025-12-17 23:31 (UTC+8) deleted the old code and that licence; the old code moved to the archived ex-exo repo, which is still GPL-3.0. The new 1.x code comes from a separate commit history whose earliest commit is from 2025-06-15, with no common ancestor with the old code; it was merged into main at 2025-12-18 21:46 (UTC+8). Jake Hillion's commit 57d3813 ("re-add LICENSE") at 2025-12-18 22:35 (UTC+8) added the full Apache-2.0 text; commit 9e2bdee at 2025-12-19 04:24 (UTC+8) filled in the copyright line as "Copyright 2025 Exo Technologies Ltd". Going by commit times, main had no licence file between 2025-12-17 23:31 (UTC+8) and 2025-12-18 22:35 (UTC+8). GitHub detects the repo licence as Apache-2.0.

At the dependency level: 1.x depends on third-party packages such as transformers, torch and MLX, each with its own licence, and pyproject.toml doesn't summarise them.

Risks & limits

No API authentication. The code binds the API to 0.0.0.0, that is, every network interface; there are no logins, tokens or key checks; and the CORS settings allow any origin with credentials. CVE-2026-50759 targets exactly this: an attacker first lists all model instances with GET /state, then deletes them one by one with DELETE /instance/{id}, interrupting inference on the cluster. The precondition is that the attacker can reach port 52415 on one of the machines, for example from the same LAN or because the port is forwarded to the internet; no account or user interaction is needed. The record was assigned by MITRE, dated 2026-07-21, and its description names 1.0.69; the CVSS 3.1 score of 7.5 (high) in the record comes from CISA-ADP, and neither MITRE nor NVD gave a score. The reporter filed issue #1833 at 2026-04-02 16:26 (UTC+8); at 22:20 (UTC+8) the same day a maintainer replied that this is intentional, exo's API is simply unauthenticated, and anyone exposing it should put it behind a reverse proxy and add their own authentication, then closed it as "not planned". The code on main was still like this on 2026-10-25, so it applies to both 1.0.71 and main. Endpoints for adding custom models, starting downloads and deleting downloads are likewise unauthenticated.

Vision cache key lacks dimensions. CVE-2026-14738 points out that the vision models' feature cache hashes only an image's raw pixel bytes with SHA-256, without width and height, so two images with identical pixel bytes but different dimensions (for example 6×4 and 4×6) hit the same cache entry and get features computed from the other image, producing wrong answers. The precondition is that the attacker can send requests to the same instance running a vision model and craft an image with exactly the same pixel bytes as the target but different dimensions, which is hard. The record was assigned and scored by VulDB, with affected versions given as "up to 1.0.71", published at 2026-07-05 18:15 (UTC+8): CVSS 3.1 is 3.7 (low); CVSS 4.0 is given as 6.3 in the record, while the NVD page shows 2.9 for the same source's 4.0 vector. Fix PR #2152 was opened at 2026-06-05 11:18 (UTC+8) and was still unmerged as of 2026-10-25; main's _image_cache_key hasn't changed.

Remote code. The README says "Custom models requiring trust_remote_code in their configuration must be explicitly enabled (default is false) for security." In the code, custom models pulled from Hugging Face are indeed off by default; but the model-card field defaults to on, and none of the built-in model cards under the repo's resources/inference_model_cards/ set the field, so all are treated as on; those cards correspond to model repos chosen by the project itself. The constant TRUST_REMOTE_CODE is also set to on, with a comment beside it reading "TODO: We should really make this opt-in, but Kimi requires trust_remote_code=True". When a vision model loads its image processor, the fallback path after the first two methods fail (vision.py lines 352 to 353) passes on unconditionally, regardless of the model card's setting. The v1.0.71 code is the same (model_cards.py lines 150 and 228, constants.py line 16, vision.py line 353). In practice, Python code in a model repo may run on the local machine when the model is loaded.

Platform support is narrower than advertised. PLATFORMS.md's tier-1 support lists only Apple silicon machines: Mac Studio M3 Ultra, Mac mini M4 Pro, and MacBook Pro M5 and M4 Max; Linux CUDA and CPU support are both listed as "planned", and Windows as "longer term". The README on main says on one hand that Linux currently uses only CPU, and on the other tells NVIDIA users in its install commands to use mlx-cuda13 or mlx-cuda12, and main's pyproject.toml does have those two options; the v1.0.71 README says only that Linux uses CPU.

Maintenance cadence. There was no new release after v1.0.71 as of 2026-10-25. Between 2026-06-22 21:38 (UTC+8) and 2026-10-12 14:54 (UTC+8), main had only one documentation commit, on 2026-08-26. The October 2026 commits all fall between 10-12 and 10-14, all by Alex Cheema. These changes are all still only on main.

Outbound data visible in the code. The code has no usage-analytics or automatic crash-reporting component. The macOS app menu has "Share Bug Report…", which uploads logs, event logs and cluster state to reports.exolabs.net only when the user clicks it; auto-update uses Sparkle to read an update feed on assets.exolabs.net.

How it got here

The 2024 version. The first commit in the main repo was made by Alex Cheema at 2024-06-24 06:28 (UTC+8). Back then the README's tagline was "Run your own AI cluster at home with everyday devices": in commit 5f18fae at 2025-10-31 02:59 (UTC+8), the old README still listed support for iPhone, iPad, Android, Mac, NVIDIA GPUs and Raspberry Pi, with tinygrad and MLX as inference backends, under GPL-3.0. The old version's alpha releases now survive only as Git tags; as of 2026-10-25 the GitHub Releases page lists only 16 1.x versions. After that the old code line all but stopped; in the same period the new code was advancing on a separate history.

The rewrite. The new code was developed on a separate history from 2025-06-15 and merged into main on 2025-12-18. On the day of the merge, the old code moved to the ex-exo archive, the licence changed from GPL-3.0 to Apache-2.0, and the macOS app and build process were opened up as well. Jeff Geerling published the same day, calling exo 1.0 "launched today", with RDMA over Thunderbolt 5 as the headline feature and test machines lent by Apple; the same article also says the author still had "some underlying trust issues with Exo, since the developers went AWOL for a while". Releases began with v1.0.60-alpha0 at 2025-12-22 22:17 (UTC+8), and v1.0.71 was released at 2026-04-23 23:04 (UTC+8).

The second half of 2026. Main changed its networking layer on 2026-06-03 and then went quiet for a long stretch; the namespace breakage caused by the switch was fixed only on 2026-10-12. A batch of changes landed between 10-12 and 10-14, including usability fixes for the dashboard and macOS app, fixes to networking, node discovery and leader election, and small CLI changes, none of them released yet. On the company side, the website focuses on DGX Spark plus Mac Studio combinations and enterprise services, and a new blog post on 2026-09-15 discusses training across slow networks.

Our verdict

For "running models too big for one machine on a few Macs", among the open-source options llama.cpp RPC, distributed-llama and GPUStack, exo is the one that combines automatic networking, topology-aware splitting, tensor parallelism and Thunderbolt 5 RDMA; of the two Jeff Geerling compared on 2025-12-18, llama.cpp and exo, only exo supports RDMA. In the same tests exo got faster with more nodes, while llama.cpp's RPC approach got slower as nodes were added in most tests. But those were one author's tests on pre-release software, and the article also records stability issues. The software is free under Apache-2.0, and the money goes mainly on hardware.

Its limits: first, the platform is almost exclusively Apple silicon Macs; the phone, Android and NVIDIA support advertised for the old version doesn't exist in 1.x, and Linux runs on CPU only. Second, the API has no authentication, listens on every interface and allows any CORS origin, and the maintainers say they don't plan to change it; the vision-cache issue has a ready fix, but the PR was still unmerged as of 2026-10-25. Third, the latest release is v1.0.71 from 2026-04-23, with no new release as of 2026-10-25; main has since changed its networking layer and fixed a batch of problems, none of which is in a release, and the October commits are concentrated in three days and come from one person.

The maintainers' position is that anyone exposing the API should put it behind a reverse proxy and add their own authentication; llama.cpp and distributed-llama also support hardware other than Macs.

Why it stands out

Exo connects several Apple Silicon Macs into one AI cluster and runs models none of them could hold alone. Machines running exo find each other on the network, and exo decides how to split a model across them from a live view of each machine's memory and the links between them: by layers, or with tensor parallelism, which the README puts at up to 1.8x faster on two devices and 3.2x on four. Over Thunderbolt 5 it uses RDMA, which the project says cuts latency between machines by 99%. MLX does the inference, and the cluster serves OpenAI-, Claude- and Ollama-compatible APIs plus a built-in dashboard.

Alex Cheema started it in 2024 to pool the memory of machines people already own. The 1.x rewrite released in December 2025 is written in Python on MLX, with Rust components, a Swift macOS app and a Svelte dashboard, and is maintained by Exo Labs.

It runs models too large for one Mac across several Apple Silicon Macs, splitting them by each machine's memory, and serves OpenAI-, Claude- and Ollama-compatible APIs.

Sources

Official

  1. GitHub - exo-explore/exo: Run frontier AI locally.GitHub · Accessed Oct 25, 2026
  2. exo/README.md at bc412e2230f395e652c23426a5eb88dfa515769b · exo-explore/exoGitHub · Accessed Oct 25, 2026
  3. exo/PLATFORMS.md at bc412e2230f395e652c23426a5eb88dfa515769b · exo-explore/exoGitHub · Accessed Oct 25, 2026
  4. exo/pyproject.toml at bc412e2230f395e652c23426a5eb88dfa515769b · exo-explore/exoGitHub · Accessed Oct 25, 2026
  5. exo/LICENSE at bc412e2230f395e652c23426a5eb88dfa515769b · exo-explore/exoGitHub · Accessed Oct 25, 2026
  6. add license · exo-explore/exo@bde1e53GitHub · Accessed Oct 25, 2026
  7. prep repo for v1 · exo-explore/exo@0fcee70GitHub · Accessed Oct 25, 2026
  8. re-add LICENSE · exo-explore/exo@57d3813GitHub · Accessed Oct 25, 2026

Updates & reading