Termsslug: state-space-model-ssm

State Space Model (SSM)

状态空间模型

Sequence modeling architecture with fixed-size state and linear complexity, fast at inference but prone to long-context degradat

Curated Intelligence

9 items · Page 1/1
Hacker News AIT2·Models
71 pts
New Two-Stage Recipe Boosts Transformer-to-Mamba Distillation

Researchers propose a two-stage recipe for distilling Transformers into Mamba-like architectures: first distill into a linearized Attention model, then transfer to Mamba with a principled initialization, avoiding the performance loss of naive distillation. The work also provides compute-optimal distillation recipes for both existing-teacher and train-from-scratch scenarios, reducing the risks of large-scale distillation.

Apple Machine Learning ResearchT1·Models
62 pts
Weighted State Souping Boosts Long-Context Retrieval Efficiency

A NeurIPS 2026 PALM workshop paper introduces weighted state souping, which learns query-dependent weights to recombine cached state-space model encodings. The method substantially improves long-context performance on single- and multi-hop QA, needle-in-a-haystack retrieval, variable tracking, and tool use, while a sparse variant achieves near-constant encoding cost with respect to context length.

Apple Machine Learning ResearchT1·Models
71 pts
Gating Mechanism Found to Bias State Space Models Toward Memorization

Through theory and experiments, this work shows that the gating mechanism common in modern recurrent networks explains why State Space Models lag behind Transformers on in-context learning and precise retrieval: gating drives SSMs to first learn an in-weights memorization solution, delaying or even preventing convergence to a correct in-context learning solution, even absent architectural or memory-capacity limits. Conversely, gating often benefits generalization to long sequence lengths, revealing its dual role in training dynamics.

AWS Machine Learning BlogT1·Tools
38 pts
Cartesia Achieves Sub-100ms Voice AI Using SSMs on AWS

AWS Partner Cartesia leverages State Space Models (SSMs) and AWS infrastructure to deliver sub-100ms text-to-speech and speech-to-text at global scale. This solution addresses latency bottlenecks in real-time conversational AI, enabling developers to build voice applications with human-like responsiveness.

Apple Machine Learning ResearchT1·Models
58 pts
New Two-Stage Recipe Boosts Transformer-to-Mamba Distillation

The paper proposes a two-stage recipe for distilling Transformers into Mamba-like architectures: first distilling knowledge into a linearized Attention model, then transferring to Mamba with a principled initialization, overcoming the failure of naive distillation to preserve teacher performance. It also provides compute-optimal distillation recipes for both the existing-teacher and jointly-trained-teacher scenarios, reducing risks of large-scale distillation.

Google ResearchT1·Models
78 pts
Google Proposes Titans Architecture: Neural Memory Meets Attention for Long Contexts

Google introduces Titans and MIRAS, proposing a novel sequence model that integrates online optimization, associative memory, and architectural design. This approach addresses the computational bottlenecks of Transformers in ultra-long contexts by incorporating neural long-term memory modules beyond standard Euclidean paradigms, offering superior information capture compared to traditional RNNs/SSMs. The research provides a new theoretical framework and architectural path for building AI systems with true 'long-term memory'.

Hugging Face BlogT1·Models
78 pts
TII Releases Open-Source Falcon 3 Family with Mamba Variant

The Technology Innovation Institute (TII) has released the open-source Falcon 3 family of LLMs, ranging from 1B to 10B parameters. The series emphasizes enhanced capabilities in science, math, and code, notably introducing a 7B base model built on the Mamba architecture to improve training efficiency and inference performance.

Hugging Face BlogT1·Models
68 pts
Falcon Mamba: First Strong Attention-Free 7B Model

TII released Falcon Mamba, the first strong-performing attention-free large language model at the 7B scale. Built on the Mamba architecture, it replaces traditional Transformer attention layers with state space models to significantly reduce inference costs and enhance long-context processing capabilities.