⬢github Jupyter Notebook · 36.9K ★ +801 since we first saw it · pushed 3 h ago · Apache-2.0
lyogavin/airllm
AirLLM 70B inference with single 4GB GPU
AirLLM is a Python library that dramatically shrinks inference memory requirements so very large open-source LLMs (70B up to 2.8T parameters) can run on small GPUs — e.g. a 70B model on a single 4GB card — without quantization, distillation, or pruning, by streaming model layers. It also adds training support for huge models on limited VRAM via streamed frozen weights plus adapters.
Why now: Recent updates announced support for brand-new flagship models like Kimi K3 (2.8T) and Qwen3.8-Flash-Next (125B) with striking low-VRAM numbers, plus a v3.0 with FP8 support and training on small GPUs.
Who it is for: Developers and researchers who want to run or fine-tune very large open-source LLMs locally on consumer GPUs with limited VRAM.
chinese-llmchinese-nlpfinetunegenerative-aiinstruct-gptinstruction-setllamallmloraopen-models
Stars over our 89 snapshots: 36.1K to 36.9K, since 22 h ago.
Where people talked about it
- ⬢github lyogavin/airllm 1 min ago
API: https://socialmediatrends-api.osmike.com/v1/repos/lyogavin/airllm