Handwritten Notes and Six Hours of Video Teach LLM Inference From Scratch
A developer has open-sourced samir-vllm, a teaching project that builds a small LLM inference engine from scratch in C++ and CUDA, covering everything from loading SafeTensor weights to executing each transformer stage on GPU. It ships with 85 pages of handwritten notes and over six hours of video, explicitly written without AI assistance, making it a solid structured resource for engineers who want to understand inference internals.
Related Entities
Full Report
Contents2 sections
samir-vllm
Teaching LLM inference from first principles by building a small engine in C++ and CUDA.
The series follows what happens between a prompt and a generated token, from loading model weights to running each part of the transformer on the GPU.
My inference notes
Read Inference from Scratch (PDF) for 80+ pages of my handwritten notes and diagrams. I developed all of the writing in this PDF myself, with NO AI writing.
Build series | 6+ Hours of footage
<table width="100%"> <tr> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_eac3fec291addfee77109b9e08ae5c09.webp" width="300" alt="Part 0: AI inference explained"><br><strong>Part 0 · AI inference explained</strong>](https://youtu.be/ef0mOukUE6U)</td> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_5d9e93b28e2e00a8d6adb5a4c059880e.webp" width="300" alt="Part 1: Loading SafeTensor weights"><br><strong>Part 1 · Loading model weights</strong>](https://youtu.be/39_gWVbYgB4)</td> </tr> <tr> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_192b53adc76f264c0eb86218c54dc9de.webp" width="300" alt="Part 2: Embedding gather kernel"><br><strong>Part 2 · Embedding gather</strong>](https://youtu.be/0V1lVIzGyqs)</td> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_3616a5e6c69820e72ed9d7e6c3dcac5a.webp" width="300" alt="Part 3: RMSNorm kernel"><br><strong>Part 3 · RMSNorm</strong>](https://youtu.be/my8VrOUVWo0)</td> </tr> <tr> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_06afc5444a5ede239f98a579e08d56d8.webp" width="300" alt="Part 4: Rotary position embeddings"><br><strong>Part 4 · RoPE</strong>](https://youtu.be/4gL0hd9UEAY)</td> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_8d3a9ecd172326fb10e8db2c94c644c6.webp" width="300" alt="Part 5: Q, K, and V projections with cuBLAS"><br><strong>Part 5 · Q/K/V with cuBLAS</strong>](https://youtu.be/8_HJH7pbw78)</td> </tr> <tr> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_49d9be3e70d388738d5eee0a27685a98.webp" width="300" alt="Part 6: Multi-head self-attention"><br><strong>Part 6 · Self-attention</strong>](https://youtu.be/tTULacSPXkc)</td> <td align="center" width="50%">[<img src="/api/v1/file/hot_img_b6559aa7a4240d313e34cf848b702cf4.webp" width="300" alt="Part 7: Feed-forward network and SwiGLU"><br><strong>Part 7 · Feed-forward network</strong>](https://youtu.be/HjllfB1IPrE)</td> </tr> <tr> <td align="center" width="50%">[<img src="https://img.youtube.com/vi/yFwPpH1Hb14/maxresdefault.jpg" width="300" alt="Part 8: Decode and KV cache"><br><strong>Part 8 · Decode and KV cache</strong>](https://youtu.be/yFwPpH1Hb14)</td> <td align="center" width="50%">[<img src="https://img.youtube.com/vi/ei3M0Ex0qHA/maxresdefault.jpg" width="300" alt="Part 9: Static and continuous batching"><br><strong>Part 9 · Static and continuous batching</strong>](https://youtu.be/ei3M0Ex0qHA)</td> </tr> <tr> <td align="center" width="50%">[<img src="https://img.youtube.com/vi/As13goWXIx8/maxresdefault.jpg" width="300" alt="Part 10: Paged KV cache"><br><strong>Part 10 · Paged KV cache</strong>](https://youtu.be/As13goWXIx8)</td> <td align="center" width="50%">[<img src="https://img.youtube.com/vi/fle2du_R8DI/maxresdefault.jpg" width="300" alt="Finale: Paged attention"><br><strong>Finale · Paged attention</strong>](https://youtu.be/fle2du_R8DI)</td> </tr> </table>Companion notes for every video are in the study-notes index, with separate Markdown and PDF folders. You can also read the complete PDF collection.
(注:更多技术实现细节与完整 API 文档请查阅原项目 README)