MikeTrendsTrends right now

⬢github C++ · 320 ★ +272 since we first saw it · pushed 4 d ago · Apache-2.0

huawei-csl/MACaQ

SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.

MACaQ is Huawei's open-source, model-agnostic quantization toolkit that produces high-quality GGUF and safetensors models for local inference across text, image, video, and audio. It targets llama.cpp, vLLM, Diffusers, and PyTorch backends, supports 2–4 bit formats, and claims fast, memory-efficient calibration with quality better than existing methods like MR-GPTQ and DuQuant++.

Why now: Huawei just released its previously internal quantizer alongside an arXiv paper, drawing Hacker News attention because its speed and quality claims beat current state-of-the-art GGUF quantization tools.

Who it is for: Developers and researchers running or serving large generative models locally on consumer or Ascend hardware.

quantizationggufllmlocal-inferencellama-cppdiffusers

ascendaudioaudio-generationdiffusersggufgguf-quantizationimage-generationllama-cppllmlocal-ai

Open on GitHub →

Stars over our 460 snapshots: 48 to 320, since 4 d ago.

Where people talked about it

API: https://socialmediatrends-api.osmike.com/v1/repos/huawei-csl/MACaQ