树莓派 Pico 2 W 运行微型世界模型与强化学习智能体
开发者在树莓派 Pico 2 W 微控制器上部署了基于循环 Transformer 的微型世界模型，实现了游戏帧预测与强化学习智能体的实时交互。该方案通过权重复用显著降低内存占用，验证了边缘设备运行生成式 AI 模型的可行性。
报道正文
本文目录3 个章节
Using 2 small neural networks to generate and play the game at the same time on a Raspberry Pi Pico 2 W. The first one creates the game: given the current frame and the action, it predicts the next frame and the reward. The second one plays that game: it uses the generated frame and picks an action.
 |  |  |  |
Every frame above was sampled by the world model network, and the actions were taken by the actor network.
Why
I got interested in looped transformers, where one transformer block is applied several times with the same weights instead of stacking different layers. Reusing weights is cheap on memory, so I wanted to see if a looped world model could be small enough to run on a microcontroller. I picked the Pico 2 W because it is cheap and has an FPU, and small games because I could write the real versions directly into numpy and check every generated frame against them. Later I added a streaming RL agent so the board could also learn to play inside its own dream.
Contents: the two networks · the games · the world model · the agents · quantization · on the chip · flashing · browser · reproduce · code layout
The world model and the actor
| network 1: the world model | network 2: the actor | |
|---|---|---|
| job | frame + action → next frame + reward | frame → action |
| architecture | looped transformer, q int8 | MLP with two hidden layers, float32 |
| size | 26k–131k parameters | 9k–16k parameters |
| cost per frame | 2–8M multiply-adds | 9k–16k multiply-adds |
| trained | on recorded real games (how) | with stream-x, in the real game or inside the dream (how) |
The size depends on the game, some games require more parameters to learn them to be playable, we tried to get to keep the number of the parameters small enough.
On the chip neither network ever uses the real game: the actor only sees frames the world model, and the world model only gets actions from the actor. Both run on core 1 of the RP2350, while core 0 handles USB and the display.
The pretrained actor can be swapped for a fresh stream-x agent that learns from scratch on the board. And any button press hands control to you, so you can play the dream yourself.
The games
Each game is a few hundred lines of numpy in src/tinyworldformer/games/. It renders to a small grid of token ids, and one frame always holds the whole game state, so hidden state has to be drawn into the frame: the ball's heading is part of its token, and Doom-lite has a minimap.
| game | frame | rules | random event | world model | exact frames, 2 loops |
|---|---|---|---|---|---|
| Breakout | 20×16 | paddle, ball, 56 bricks, 5 lives | the serve after a miss | 26k params, 4.3M MACs | 99.99% |
| Doom-lite | 32×32 | raycast first-person view of an 8×8 maze; an enemy walks shortest paths to you; you shoot | the enemy's respawn | 55k params, 4.2M MACs | 83.6% (see below) |
| Flappy Bird | 24×24 | gravity, flaps, scrolling pipes | the next gap height | 46k params, 2.0M MACs | 100% |
| Pong | 20×32 | you against a CPU paddle that reacts late, first to 5 | the serve after a point | 46k params, 2.3M MACs | 99.45% |
| Pac-Man lite | 48×48 | 23×21 maze, 249 pellets, two ghosts on shortest paths, 3 lives | the ghosts' respawn | 131k params, 8.2M MACs | 86.4% |
"Exact frames" is the share of held-out transitions where every cell of the predicted frame matches the real game. Frames right after a random event are left out, since those can only be sampled, not predicted. Doom-lite's 3D view has 768 cells and a single shaded wall pixel off counts the whole frame as wrong, so its number looks worse than it plays; the game state is almost always right (see checking the dream).
You can play the real games in a terminal: uv run python -m tinyworldformer.games.play --game pacman (or breakout, doomlite, flappy, pong).
（注：更多技术实现细节与完整 API 文档请查阅原项目 README）