Hacker News AIT2·Tools
78 pts
Huawei has open-sourced MACaQ, an internal model-agnostic calibrated quantizer that spans text, image, video and audio models across llama.cpp, vLLM, Diffusers and PyTorch backends. It claims to quantize a 122B-parameter model in 64 minutes using only 32.5 GB of device memory, and to match MR-GPTQ and DuQuant++ perplexity on Llama-3-8B with 11x less quantization time.