⬢github Python · 156 ★ +117 since we first saw it · pushed 39 min ago · MIT
ShinMegamiBoson/lookatmyquant
A quantization compiler that delivers 12× lower KL than Q4_K_M at the same file size
LookAtMyQuant is a Python quantization compiler that compresses LLMs by measuring how much each piece of a model hurts output quality when compressed, then assigning per-row codes (trellis codes, Fisher-information bit allocation, distillation) to hit a size budget with minimal damage. It has produced tiny GGUF-style models, like a 4B model at 2.3 bits per weight, beating comparable-size Unsloth quants on math and other benchmarks.
Why now: Recently released quants, including Inkling-2-Mini at 5.4 GB, claim much lower KL divergence and better benchmark scores than same-size Unsloth GGUFs, drawing attention on Hacker News.
Who it is for: People running LLMs locally on memory-constrained hardware, especially Apple Silicon, who want smaller files with minimal quality loss.
Stars over our 9 snapshots: 39 to 156, since 2 h ago.
Where people talked about it
- Yhn Inkling-2-Mini in 5.4 GB Memory 14 min ago
API: https://socialmediatrends-api.osmike.com/v1/repos/ShinMegamiBoson/lookatmyquant