Termsslug: low-rank-approximation

Low-Rank Approximation

低秩近似

Using low-dimensional matrices to approximate high-dimensional attention matrices to save memory.

Curated Intelligence

1 items · Page 1/1
Hugging Face BlogT1·Models
42 pts
Hugging Face Surveys Long-Range Transformer Optimizations

Hugging Face published a technical survey on Long-range Transformers, systematically reviewing methods to reduce computational complexity. The article highlights architectural improvements such as sparse attention, low-rank approximations, and linearization to address the quadratic bottleneck of standard Transformers in long-sequence processing.