LittleBit-2 Achieves Sub-1-Bit LLM Compression via Latent Geometry Alignment in Low-Rank Factorization

1 reports archived1 independent sourcesupdated 10/8/2026, 21:29:46

Synthesis & Latest Updates

1 Sources Cross-Validated

LittleBit-2 introduces a novel method for compressing large language models to the sub-1-bit regime by factorizing weight matrices into low-rank latent factors and binarizing them. By employing Internal Latent Rotation with Joint Iterative Quantization (Joint-ITQ) to address latent geometry misalignment during initialization, it achieves extreme compression down to 0.1 bits per weight while preserving the original inference architecture without additional overhead.

LATEST/LittleBit-2 introduces a novel method for compressing large language models to the sub-1-bit regime by factorizing weight matrices into low-rank latent factors and binarizing them. By employing Internal Latent Rotation with Joint Iterative Quantization (Joint-ITQ) to address latent geometry misalignment during initialization, it achieves extreme compression down to 0.1 bits per weight while preserving the original inference architecture without additional overhead.

TIMELINECoverage timeline

Total 1 reports · Latest first

Hacker News AIT2·Oct 8, 21:2968 pts
LittleBit-2 Achieves Sub-1-Bit LLM Compression via Latent Geometry Alignment in Low-Rank Factorization
Original: Sub-1-Bit LLM Compression via Latent Factorization
- Achieves sub-1-bit compression (0.1 to 1.0 bits/weight) using low-rank latent factorization combined with binarization.
- LittleBit-2 employs Joint-ITQ to correct geometric misalignment in SVD-derived factors, significantly improving quantization accuracy.
- The method requires no architectural changes at inference time and incurs zero additional computational overhead, facilitating easy integration.
View Evidence Original Source