⬢github C · 47K ★ +833 since we first saw it · pushed 2 h ago · Apache-2.0
JustVugg/colibri
A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
colibri is a tiny inference engine, written in pure C with no dependencies, that runs very large mixture-of-experts language models on ordinary hardware. It keeps only the active part of each MoE model in RAM and streams expert weights from disk on demand, so huge models run on a CPU without a GPU. It supports many model families, GGUF files, and optional CUDA/Vulkan backends.
Why now: People are excited that a minimal single-file C engine can run hundreds-of-billions-parameter MoE models on machines with as little as 8 GB of RAM, no GPU required — a striking approach to consumer-hardware inference.
Who it is for: Developers and hobbyists who want to run frontier open-weight MoE models locally on the CPUs and disks they already own.
Stars over our 99 snapshots: 46.1K to 47K, since 1 d ago.
Where people talked about it
- ⬢github JustVugg/colibri 10 min ago
API: https://socialmediatrends-api.osmike.com/v1/repos/JustVugg/colibri