⬢github C++ · 104 ★ +27 since we first saw it · pushed 7 d ago · MIT
davide-troiani/moeshift
Memory efficient and high performance Mixture-of-Experts inference for Apple Silicon
MoEShift is a C++ runtime for memory-efficient, high-performance Mixture-of-Experts LLM inference on Apple Silicon Macs. It runs large models like a 125B Qwen model from about 10 GB of RAM by streaming experts from SSD, serves an OpenAI-compatible API, and benchmarks faster and leaner than Strata on M4 Pro hardware.
Why now: A fresh alpha release with head-to-head benchmarks versus Strata, plus an Hacker News discussion showing M4 Pro vs. M5 Pro results at 374 tok/s prefill.
Who it is for: Mac users with Apple Silicon who want to run large local LLMs without a cloud subscription or huge RAM.
apple-siliconllama-cppllm-inferencelocal-llmmemory-optimizationmixture-of-expertsmoe
Stars over our 177 snapshots: 77 to 104, since 1 d ago.
Where people talked about it
API: https://socialmediatrends-api.osmike.com/v1/repos/davide-troiani/moeshift