ALL INTELLIGENCE
Hacker News AIT2·Tools

Open-Source Guide Building LLMs from Scratch in Node.js and TypeScript

Original title: Show HN: Building an LLM from Scratch in TypeScript
58AI Score

This project offers a comprehensive guide to building Large Language Models from scratch using Node.js and TypeScript, covering data loaders, embeddings, attention mechanisms, and Transformer blocks. All code is packaged as runnable modules on GitHub, aiming to demystify model internals by reimplementing core concepts in a non-Python environment.

Full Report

Building an LLM in NodeJS book cover

Building an LLM in NodeJS

The from-scratch guide to building large language models in NodeJS and TypeScript.Start with Chapter 2

Learn how large language models actually work by building one from scratch in Node.js and TypeScript.

No black boxes

We’ll build every layer, from attention mechanisms to the training loop.

From scratch

Dataset, dataloader, attention, transformer block, GPT, and the training loop built up chapter by chapter.

Runnable

Every chapter has a runnable entry point in the @node-llm/chapters package, all on GitHub.

Interactive

Soon

Live WebGPU demos from the @node-llm/playground package are embedded throughout.


作者发帖说明(HN):

I wanted to understand transformers at a level where nothing is magic, so I'm building a ChatGPT-style model from first principles in TypeScript. Why not Python? I've found that reimplementing concepts in a different language forces me to grasp the nuance of the implementation. Chapters cover dataloaders, embeddings, and the attention mechanism so far. Feedback welcome if anything's unclear.