Sushi 1.4.0 runs a 1.75-bit quant of DeepSeek V4.1 Flash 1017 on a 128GB Mac

Tue, Oct 27 18:56 UTC+8Updated Sat, Oct 31 20:55 UTC+8

Sushi v1.4.0 adds the DeepSeek-V4.1-Flash-1017-Sushi-STQ-1.75bpw pack (332 GB with its Engram tables), which streams its routed experts from the SSD within --ssd-budget-gb so it runs on a 128 GB Mac, with up to 1M tokens of context. On an M5 Max it decodes 76.6 tok/s at 4k on code and 49.3 tok/s on prose, and prefills 1,024 tok/s (sushi bench, --ssd-budget-gb 108).

Why it matters

Sushi v1.4.0 ships a 1.75-bit quant of DeepSeek V4.1 Flash 1017 that runs locally on a Mac with 128GB of memory.

Sources2

GitHub · beamivalice/sushiGitHubTue, Oct 27 18:56 UTC+8sushi v1.4.0View original · Tue, Oct 27 18:56 UTC+8
@ivanfioravantiXTue, Oct 27 21:04 UTC+8Sushi 🍣 DeepSeek 4.1 Flash 1017 - STQ 1.75 bits is here! Enjoy it on your 128GB machines! x.com/Beamsters1/sta…View original · Tue, Oct 27 21:04 UTC+8

Written by dalao.ai: the title, summary and why it matters are our own words. The original reports are the record; their publishers keep authorship.