18 lessons — available now
01
LLMs from scratch
How large language models actually work, built up one piece at a time. It starts with “what is a token” and ends with mixture-of-experts routing, speculative decoding and RLHF — no maths background assumed.
- —Tokens, embeddings, attention and the transformer block
- —Pretraining, scaling laws, SFT, RLHF and DPO
- —Interactive diagrams in every lesson, plus checked exercises
- —Progress saved locally — no account, no tracking
The map
- 1What a language model actually does
- 2Tokens: how text becomes numbers
- 3Embeddings: meaning as direction
- 4Neurons, matrices and nonlinearity
- 5Attention: queries, keys and values
- 6Multi-head attention
- 7Position: how the model knows word order
- 8The transformer block
- 9The full forward pass
- 10Sampling: probabilities into text
- 11Context windows and the KV cache
- 12Pretraining: where weights come from
- 13Scaling laws: why bigger worked
- 14Post-training: SFT, RLHF and DPO
- 15Prompting and chain-of-thought
- 16RAG, tools and agents
- 17Making models cheap
- 18Evaluation, hallucination, open problems