⬢github first seen 1 d ago, last 4 min ago, peak #4
New Colibri engine runs huge MoE models on modest hardware
Original: JustVugg/colibri
A developer has released Colibri, an open-source inference engine written in C aimed at frontier-scale mixture-of-experts models. It tiers model experts by use: hot ones stay on the GPU via CUDA or Vulkan, warm ones sit in RAM, and cold ones stream from NVMe storage, letting people run large models on existing consumer hardware. Early attention is coming from developers interested in local AI inference.
Why now: Growing interest in running large MoE models locally without expensive data-center hardware
Rank over time, top of the chart is #1. 110 snapshots from 1 d ago to 4 min ago.
Evidence
- JustVugg/colibri · JustVugg · 16
A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
API: https://socialmediatrends-api.osmike.com/v1/trends/4610467