Huawei Open-Sources MACaQ Quantizer for Fast Local Inference

1 reports archived1 independent sourcesupdated 11/12/2026, 04:23:57

Synthesis & Latest Updates

1 Sources Cross-Validated

Huawei has open-sourced MACaQ, an internal model-agnostic calibrated quantizer that spans text, image, video and audio models across llama.cpp, vLLM, Diffusers and PyTorch backends. It claims to quantize a 122B-parameter model in 64 minutes using only 32.5 GB of device memory, and to match MR-GPTQ and DuQuant++ perplexity on Llama-3-8B with 11x less quantization time.

LATEST/Huawei has open-sourced MACaQ, an internal model-agnostic calibrated quantizer that spans text, image, video and audio models across llama.cpp, vLLM, Diffusers and PyTorch backends. It claims to quantize a 122B-parameter model in 64 minutes using only 32.5 GB of device memory, and to match MR-GPTQ and DuQuant++ perplexity on Llama-3-8B with 11x less quantization time.

Hourly heat curve

2 fully observed hours

Fewer than 3 fully observed hours; trend not plotted yet.

Coverage timeline

Total 1 reports · Latest first

Hacker News AIT2Nov 12, 04:23·78 pts
Huawei Open-Sources MACaQ Quantizer for Fast Local Inference
- Model-agnostic calibrated quantization covering text, image, video and audio models
- Quantizes a 122B-parameter model in 64 minutes with just 32.5 GB of device memory
- Matches MR-GPTQ and DuQuant++ perplexity on Llama-3-8B with 11x less quantization time
- Ships backends for llama.cpp, vLLM, Diffusers and PyTorch
View Related Reports & Analysis Original Source