⬢github Python · 693 ★ +602 since we first saw it · pushed 1 h ago · Apache-2.0
synchopate/llama-modchip
🐋 Run 284B DeepSeek-V4-Flash (<59GiB RAM, 44.4 tok/s pp, 9.1-14.7 tok/s) & 30B MoEs (25-111 tok/s) on a pure 64GB CPU at 256K ctx (0 GPUs). SwiGLU-H2, Stockfish-PVS L3 & 13ms Claude Code restore.
llama-modchip is a Python tool for running large Mixture-of-Experts language models on CPUs alone, no GPU required. It fits a 284B DeepSeek-V4-Flash model into under 59 GiB of RAM (via a SwiGLU-H2 compression technique) at 256K context, runs 30B MoE models at 25–111 tok/s, uses Stockfish-based speculative drafting for speedups, and offers a 13ms Claude Code session restore.
Why now: It was featured on Hacker News for the striking claim of running a 284B-parameter model on a plain 64GB-RAM machine with zero GPUs, plus a fresh v2.4.0 release and a benchmark leaderboard.
Who it is for: Developers and hobbyists with CPU-only machines (or modest GPUs) who want to run frontier-scale and 30B-class MoE models locally.
ai-agentsapple-siliconavx512claude-codecpu-inferencedeepseekdeepseek-v4ggufhpcllama-cpp
Stars over our 108 snapshots: 91 to 693, since 1 d ago.
Where people talked about it
API: https://socialmediatrends-api.osmike.com/v1/repos/synchopate/llama-modchip