手写 85 页笔记与 6 小时视频从零构建 LLM 推理引擎

原标题:Hours of YouTube and 85 handtyped pages of notes on LLM Inference – Start here
58AI 研判分

开发者开源了 samir-vllm 教学项目,用 C++ 与 CUDA 从零实现一个小型推理引擎,完整覆盖从加载 SafeTensor 权重到 Transformer 各层 GPU 执行的流程。配套材料包括 85 页手写笔记 PDF 与 6 小时以上视频,强调无 AI 代写,适合想深入理解推理底层机制的工程师系统学习。

报道正文

本文目录2 个章节

samir-vllm

Teaching LLM inference from first principles by building a small engine in C++ and CUDA.

The series follows what happens between a prompt and a generated token, from loading model weights to running each part of the transformer on the GPU.

My inference notes

Read Inference from Scratch (PDF) for 80+ pages of my handwritten notes and diagrams. I developed all of the writing in this PDF myself, with NO AI writing.

Build series | 6+ Hours of footage

<table width="100%"> <tr> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_eac3fec291addfee77109b9e08ae5c09.webp" width="300" alt="Part 0: AI inference explained"><br><strong>Part 0 · AI inference explained</strong>](https://youtu.be/ef0mOukUE6U)</td> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_5d9e93b28e2e00a8d6adb5a4c059880e.webp" width="300" alt="Part 1: Loading SafeTensor weights"><br><strong>Part 1 · Loading model weights</strong>](https://youtu.be/39_gWVbYgB4)</td> </tr> <tr> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_192b53adc76f264c0eb86218c54dc9de.webp" width="300" alt="Part 2: Embedding gather kernel"><br><strong>Part 2 · Embedding gather</strong>](https://youtu.be/0V1lVIzGyqs)</td> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_3616a5e6c69820e72ed9d7e6c3dcac5a.webp" width="300" alt="Part 3: RMSNorm kernel"><br><strong>Part 3 · RMSNorm</strong>](https://youtu.be/my8VrOUVWo0)</td> </tr> <tr> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_06afc5444a5ede239f98a579e08d56d8.webp" width="300" alt="Part 4: Rotary position embeddings"><br><strong>Part 4 · RoPE</strong>](https://youtu.be/4gL0hd9UEAY)</td> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_8d3a9ecd172326fb10e8db2c94c644c6.webp" width="300" alt="Part 5: Q, K, and V projections with cuBLAS"><br><strong>Part 5 · Q/K/V with cuBLAS</strong>](https://youtu.be/8_HJH7pbw78)</td> </tr> <tr> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_49d9be3e70d388738d5eee0a27685a98.webp" width="300" alt="Part 6: Multi-head self-attention"><br><strong>Part 6 · Self-attention</strong>](https://youtu.be/tTULacSPXkc)</td> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_b6559aa7a4240d313e34cf848b702cf4.webp" width="300" alt="Part 7: Feed-forward network and SwiGLU"><br><strong>Part 7 · Feed-forward network</strong>](https://youtu.be/HjllfB1IPrE)</td> </tr> <tr> <td align="center" width="50%">[<img src="https://img.youtube.com/vi/yFwPpH1Hb14/maxresdefault.jpg" width="300" alt="Part 8: Decode and KV cache"><br><strong>Part 8 · Decode and KV cache</strong>](https://youtu.be/yFwPpH1Hb14)</td> <td align="center" width="50%">[<img src="https://img.youtube.com/vi/ei3M0Ex0qHA/maxresdefault.jpg" width="300" alt="Part 9: Static and continuous batching"><br><strong>Part 9 · Static and continuous batching</strong>](https://youtu.be/ei3M0Ex0qHA)</td> </tr> <tr> <td align="center" width="50%">[<img src="https://img.youtube.com/vi/As13goWXIx8/maxresdefault.jpg" width="300" alt="Part 10: Paged KV cache"><br><strong>Part 10 · Paged KV cache</strong>](https://youtu.be/As13goWXIx8)</td> <td align="center" width="50%">[<img src="https://img.youtube.com/vi/fle2du_R8DI/maxresdefault.jpg" width="300" alt="Finale: Paged attention"><br><strong>Finale · Paged attention</strong>](https://youtu.be/fle2du_R8DI)</td> </tr> </table>

Companion notes for every video are in the study-notes index, with separate Markdown and PDF folders. You can also read the complete PDF collection.

(注:更多技术实现细节与完整 API 文档请查阅原项目 README)