⬢github first seen 1 d ago, last 7 min ago, peak #4
Colibri engine streams MoE model experts from NVMe
Original: JustVugg/colibri
A developer known as JustVugg has released Colibri, an open-source inference engine written in C for large mixture-of-experts language models. It keeps frequently used experts on the GPU via CUDA or Vulkan, stores warm ones in RAM, and streams cold experts from NVMe storage, allowing frontier-scale models to run on existing consumer hardware. The project is drawing attention among local AI enthusiasts for lowering hardware requirements.
Why now: It promises to run large frontier MoE models on ordinary hardware, which excites the local AI community.
Rank over time, top of the chart is #1. 2 snapshots from 22 min ago to 7 min ago.
Evidence
- JustVugg/colibri · JustVugg · 16
A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
API: https://socialmediatrends-api.osmike.com/v1/trends/4610467