i14's Laplace 3B Linear Attention Model Runs on AWS Trainium2 with Native PyTorch

1 reports archived1 independent sourcesupdated 10/21/2026, 05:14:10

Synthesis & Latest Updates

1 Sources Cross-Validated

i14's Laplace 3B utilizes a linear attention architecture with a fixed recurrent state of 29.5 MB per sequence, independent of context length. By leveraging TorchNeuron Native, the model achieves native PyTorch support on AWS Trainium2 while preserving its Gated Linear Attention (GLA) design and Muon optimizer. Benchmarks show training speeds of ~120k tokens/s and inference throughput of 11.5k output tokens/s across 1,280 concurrent sequences on a single trn2.48xlarge instance.

LATEST/i14's Laplace 3B utilizes a linear attention architecture with a fixed recurrent state of 29.5 MB per sequence, independent of context length. By leveraging TorchNeuron Native, the model achieves native PyTorch support on AWS Trainium2 while preserving its Gated Linear Attention (GLA) design and Muon optimizer. Benchmarks show training speeds of ~120k tokens/s and inference throughput of 11.5k output tokens/s across 1,280 concurrent sequences on a single trn2.48xlarge instance.

HEAT TRENDHourly heat curve

3 fully observed hours

Now

18.8

Peak

19.706:00

24h change

–

The line compares only sources observed throughout; gap hours are interpolated to keep the trend continuous.

TIMELINECoverage timeline

Total 1 reports · Latest first

AWS Machine Learning BlogT1·Oct 21, 05:1468 pts
i14's Laplace 3B Linear Attention Model Runs on AWS Trainium2 with Native PyTorch
- Laplace 3B maintains a fixed recurrent state size of 29.5 MB, eliminating the KV Cache growth issue inherent in standard Transformers
- Achieves native PyTorch training and inference on AWS Trainium2 via TorchNeuron Native without modifying low-level operator logic
- Delivers 120k tokens/s training speed and 11.5k output tokens/s high-concurrency inference throughput on a single trn2.48xlarge instance
View Related Reports & Analysis Original Source