Termsslug: flash-attention

Flash Attention

An algorithm that accelerates attention computation and reduces memory usage by optimizing memory access patterns.

Curated Intelligence

2 items · Page 1/1
Hugging Face BlogT1·Models
85 pts
ModernBERT Released: A Modern Replacement for BERT

Hugging Face and partners released ModernBERT, designed to replace the decade-old BERT architecture. By integrating modern techniques like RoPE, SwiGLU, and extended context windows (up to 8k), it significantly outperforms baselines in retrieval, classification, and entity recognition while maintaining inference efficiency.