ALL INTELLIGENCE
Hugging Face BlogT1·Models

Falcon Mamba: First Strong Attention-Free 7B Model

Original title: Welcome Falcon Mamba: The first strong attention-free 7B model
68AI Score

TII released Falcon Mamba, the first strong-performing attention-free large language model at the 7B scale. Built on the Mamba architecture, it replaces traditional Transformer attention layers with state space models to significantly reduce inference costs and enhance long-context processing capabilities.

Full Report

Welcome Falcon Mamba: The first strong attention-free 7B model