⬢github first seen 1 d ago, last 11 min ago, peak #4
Colibri inference engine runs MoE models on modest hardware
Original: JustVugg/colibri
A new open-source project called Colibri by developer JustVugg offers a hierarchical inference engine for large mixture-of-experts models. Hot experts run on the GPU via CUDA or Vulkan, warm ones sit in RAM, and cold ones stream from NVMe storage. Written entirely in C, it targets people running frontier-size models on existing consumer hardware, and the lightweight approach is drawing attention in the local-model community.
Why now: Interest in running large MoE models cheaply on consumer hardware without datacenter GPUs is surging.
Rank over time, top of the chart is #1. 155 snapshots from 1 d ago to 11 min ago.
Evidence
- JustVugg/colibri · JustVugg · 16
A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
API: https://socialmediatrends-api.osmike.com/v1/trends/4610467