ExoProduct update
Exo · v1.0.69
EXO v1.0.69 turns on continuous batching by default, automatically batching inference requests on single-node, multi-node and RDMA instances, and pipeline parallel prefill now splits the prompt into chunks and overlaps computation and communication, which the release notes say is up to 1.98x faster on 2 nodes. Qwen3.5 and M5 Pro/Max chips are supported, with an upgraded macmon letting M5 Pro/Max report memory and GPU usage so they can participate in clusters, and a new POST /v1/cancel/{command_id} endpoint cancels ongoing text generations.
EXO will automatically batch together inference requests, on single node and multi-node instances including RDMA instances.
Saved . A shortened excerpt from GitHub Releases; the original publisher retains authorship.