⬢github first seen 1 d ago, last just now, peak #4

Colibri inference engine streams MoE models from NVMe

Original: JustVugg/colibri

A new open-source project called Colibri offers a hierarchical inference engine for large mixture-of-experts language models. Hot experts run on the GPU via CUDA or Vulkan, warm ones stay in RAM, and cold ones are streamed from NVMe storage. Written in C, it aims to run frontier-scale models on ordinary hardware people already own.

Why now: Developers are interested in tools that let large MoE models run locally without expensive hardware.

ColibriJustVuggMoE modelsCUDAVulkan

Open on github →

Rank over time, top of the chart is #1. 25 snapshots from 5 h ago to just now.

Evidence

• JustVugg/colibri · JustVugg · 16

  A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.

API: https://socialmediatrends-api.osmike.com/v1/trends/4610467