How to build an LLM, starting from nothing

What you will learn

• What a derivative really tells you, felt numerically rather than recalled from school
• How to build an autograd engine: values that remember how they were computed
• The chain rule as a local rule each operation applies to its own inputs
• What a forward pass, a backward pass and a gradient descent step are
• Why you must zero gradients between steps — and what happens when you forget

Practical exercises

• Estimate a derivative by handPick a function such as f(x) = 3x² − 4x + 5. Work out the slope at x = 3 on paper. Then compute it numerically in Python by nudging x by a tiny h and measuring the change in f. Try h = 0.1, 0.001, 0.0000001.Done when: Your numerical estimate matches the hand-derived answer, and you can explain what goes wrong when h gets too small.
• Build the Value classWrite a Value that wraps a number, records which operation produced it and from which inputs, and supports +, * and tanh. Give each operation a backward that pushes gradient to its inputs. Build a small expression, call backward on the output, and check every gradient against the same expression written in PyTorch.Done when: Every gradient your engine produces matches PyTorch to several decimal places.
• Train a tiny network to fit four examplesAssemble Neuron, Layer and MLP on top of Value. Train the MLP from the lecture on its four-example dataset until the loss is near zero. Then delete the line that zeroes the gradients and run it again.Done when: The loss reaches near zero, and you can explain from the output why removing zero_grad breaks training rather than merely slowing it.
• Add a new operationStretchExtend the engine with exp, log and division, then implement one operation the lecture does not cover — ReLU is a good choice — deriving its backward rule yourself.Done when: Your new operation’s gradient agrees with PyTorch, and you derived the rule before checking.

Code and reading

What you will learn

• How to frame text generation as predicting the next character
• PyTorch tensors: indexing, broadcasting, and the mistakes broadcasting invites
• Negative log likelihood, and why it is the right thing to minimise
• Sampling from a probability distribution, and how temperature changes the output
• Why smoothing matters — an unseen pair costs you infinite loss without it

Practical exercises

• Count, then generateBuild the 27×27 matrix of "which character follows which" over the names dataset. Normalise the rows into probabilities and sample twenty new names.Done when: Your samples look like clumsy names rather than random letters, and you can point to the row of the matrix that produced a given character.
• Measure the model instead of eyeballing itCompute the average negative log likelihood of the training set under your counting model. Then add 1 to every count before normalising and compute it again.Done when: You can state your loss as a single number and explain why a zero count would otherwise make the loss infinite.
• Train the neural version and compareBuild the one-layer network: one-hot encode the input character, multiply by a weight matrix, softmax the output, minimise the same negative log likelihood with gradient descent.Done when: The trained network’s loss lands close to the counting model’s, and you can explain why that is the expected result rather than a coincidence.
• Move to a trigram modelStretchUse the previous two characters instead of one. Split the data into training, dev and test sets, then tune the smoothing strength on dev only.Done when: Your trigram dev loss beats your bigram dev loss, and you never looked at the test set while tuning.

Code and reading

What you will learn

• What an embedding is, and why characters end up near each other in that space
• Minibatch training, and the trade it makes between speed and noise
• How to find a usable learning rate instead of guessing
• Train/dev/test discipline, and what the test set is actually for
• Reading underfitting and overfitting off a loss curve

Practical exercises

• Build and train the MLPEmbed three characters of context, concatenate them, pass through a tanh hidden layer, then to output logits and cross entropy. Train it with minibatches on the names dataset.Done when: Your dev loss beats the trigram model from the previous module.
• Find the learning rate rather than guessing itSweep the learning rate across several orders of magnitude, recording the loss for each, and plot loss against learning rate. Pick your rate off the plot, then decay it late in training.Done when: You have a plot with a clear valley, and you chose your learning rate from it rather than from a tutorial.
• Look at what the model learnedTrain with a 2-dimensional embedding so it can be plotted, then scatter all 27 characters with labels.Done when: You can point at a cluster and explain why those characters sit together — vowels are the usual give-away.
• Push the architectureStretchScale up the context, embedding size and hidden layer to get the dev loss as low as you can. Keep a short log of each change and its effect.Done when: Your log shows at least one change that you expected to help and which did not, and you can say why.

Code and reading

What you will learn

• What a saturated tanh looks like in a histogram, and why it kills learning
• Why weight initialisation scale matters, and where Kaiming initialisation comes from
• What batch normalisation does, and the coupling between examples it introduces
• The diagnostic plots worth reaching for when a network will not train
• Why the loss at step zero tells you whether your output layer is initialised sanely

Practical exercises

• Fix the confidently wrong startWork out what the loss should be at initialisation if the model predicted every character with equal probability. Compare it to what your model actually starts at, then scale down the output layer’s weights so the two agree.Done when: Your starting loss matches the hand-calculated value, and the hockey-stick drop at the beginning of the curve is gone.
• See the saturationPlot a histogram of the hidden layer’s tanh outputs, and an image of which activations are beyond ±0.99. Then rescale the hidden weights and plot both again.Done when: You have a before-and-after pair of plots, and can explain why a fully white saturation image would mean a dead layer.
• Add batch normalisationInsert a BatchNorm layer before the activation. Train and compare against your previous run. Then use the model to predict a single name on its own, which forces you to deal with the running statistics used at inference.Done when: Single-example inference gives the same answer as inference in a batch, and you can explain why that needed running statistics.
• Go deeper on purposeStretchBuild a five- or six-layer version, first without careful initialisation and without BatchNorm, then with both. Record the loss curves side by side.Done when: You have personally seen a deep network fail to train and then train fine after the fix.

Code and reading

What you will learn

• How gradients flow through matrix multiplication, and why the transposes land where they do
• The gradient of cross entropy, derived and then simplified
• Backpropagating batch normalisation, where examples are coupled
• Gradient accumulation into an embedding table via repeated indices
• How to verify a hand-derived gradient against autograd and localise a mismatch

Practical exercises

• Backprop the loss and the output layerStarting from the official Colab, derive the gradient of cross entropy with respect to the logits, then back through the final linear layer to its weights, bias and inputs. Check each against PyTorch’s own gradients.Done when: Every comparison for these tensors reports an exact match.
• Backprop tanh, BatchNorm and the embeddingContinue backwards through the activation, the batch-norm layer and the embedding lookup. The batch-norm step is the hard one; derive it in the long form first.Done when: All comparisons pass, including batch norm, without copying the video’s answer.
• Collapse it into the short formStretchRewrite the cross-entropy and batch-norm gradients in their compact single-expression forms, and confirm they still match.Done when: You can write both short forms from memory and explain what cancelled to produce them.

Code and reading

What you will learn

• Why a hierarchical structure beats one flat layer as context grows
• How to write your own Linear, BatchNorm1d and Tanh layers with a torch.nn-style interface
• Sequential containers, and organising a model you can keep changing
• Reading and debugging multi-dimensional tensor shapes without guessing
• Moving code out of a notebook and into a repository as it settles

Practical exercises

• Rebuild your layers as classesWrap your model’s pieces into Linear, BatchNorm1d, Tanh, Embedding, Flatten and Sequential classes with the same call and parameters interface torch.nn uses. Retrain and confirm you match your previous loss.Done when: The model trains to the same loss as before and reads as a short list of layers.
• Make it hierarchicalChange the flattening so characters are fused in pairs across several levels rather than all at once, and grow the context window to eight characters.Done when: Your dev loss improves on the flat model at the same parameter count.
• Debug a shape on purposeInsert a deliberate shape mistake, read the error, and locate it by printing shapes layer by layer rather than by reverting.Done when: You found it from the shapes alone, and can explain what broadcasting was doing.

Code and reading

What you will learn

• Self-attention derived from a running average, then generalised — three different ways of computing the same thing
• Queries, keys and values, and what the scaling factor is for
• Causal masking, and why a language model must not see the future
• Multi-head attention, and why several smaller heads beat one large one
• Residual connections and layer normalisation, and why deep stacks need them
• The difference between the decoder you built and the original encoder-decoder Transformer

Practical exercises

• Start from the bigram baselineLoad the Shakespeare text, build a character tokenizer, and train the simplest bigram model in this new codebase. Generate a sample.Done when: You have a training loop, a loss number and generated text — bad text, but a working baseline to improve.
• Derive attention three waysCompute the average of all previous positions with an explicit loop; then with a masked lower-triangular matrix multiplication; then with a masked softmax. Confirm all three produce the same tensor.Done when: All three agree numerically, and you can explain why the softmax version is the one that generalises into attention.
• Build a single attention headAdd query, key and value projections, scale the scores, mask the future, softmax, and aggregate the values. Train and compare the loss to your baseline.Done when: Loss improves over the bigram baseline, and you can say what each of the three projections is for.
• Assemble the full GPTAdd multi-head attention, the feed-forward network, residual connections, layer normalisation and dropout. Stack several blocks and train to convergence.Done when: Your samples read as Shakespeare-shaped English with speaker names and line breaks, and your validation loss is clearly below the single-head version.
• Train it on your own textStretchReplace Shakespeare with a corpus of your own — your chat exports, a favourite author, song lyrics, your own writing. Keep the architecture and retune only what you must.Done when: The model produces text that is recognisably in the style of your corpus, and you can say which hyperparameter mattered most.

Code and reading

What you will learn

• UTF-8 bytes, and why working from bytes avoids an unbounded vocabulary
• Byte pair encoding: training the merges, then encode and decode
• Why GPT-2 and GPT-4 split text with a regular expression before merging
• Special tokens, and how they mark conversation boundaries
• Which known LLM weaknesses trace directly back to this stage

Practical exercises

• Write a basic BPE tokenizerImplement train(text, vocab_size), encode(text) and decode(ids) over raw UTF-8 bytes, learning merges from a paragraph of text.Done when: decode(encode(s)) == s for every string you try, including emoji and accented characters.
• Reproduce the odditiesWith your own tokenizer and with tiktoken, tokenize: a number with many digits, the same word with and without a leading space, a line of Python indentation, and a sentence in a non-English language.Done when: You can predict, before running it, which of these will use surprisingly many tokens.
• Add regex splitting and special tokensSplit text with the GPT-4 pattern before merging so merges never cross word boundaries, then add special tokens that encode to fixed ids and are never merged.Done when: Your token counts are in the same ballpark as tiktoken on the same text, and your special tokens survive a roundtrip intact.
• Train on a language you knowStretchTrain your tokenizer on a corpus in a language other than English and compare tokens-per-word against a GPT-4 tokenizer on the same text.Done when: You can quantify the difference and explain what it implies about cost and quality for speakers of that language.

Code and reading

What you will learn

• Implementing GPT-2 faithfully enough to load the published weights
• Where the speed comes from: bfloat16, flash attention, torch.compile, and sizes that fit the hardware
• The training recipe: AdamW, weight decay, gradient clipping, warmup and cosine decay
• Gradient accumulation, and reaching a large batch size on small hardware
• Distributed data parallel across several GPUs
• Evaluating on HellaSwag rather than trusting the loss curve alone

Practical exercises

• Load the real weights into your own codeWrite the GPT-2 module so that the published 124M checkpoint loads into it without shape errors, then generate samples from it.Done when: Your implementation produces coherent GPT-2 output from the official weights, proving your architecture matches.
• Make it fast, and measureRecord a baseline of tokens per second. Then add bfloat16 autocast, flash attention and torch.compile, and round the vocabulary size up to a friendlier number — measuring after each change.Done when: You have a table of throughput per change and can name which single change bought the most.
• Run a real training runTrain on a FineWeb-Edu sample with the full recipe — warmup, cosine decay, gradient clipping, gradient accumulation to a large effective batch. Scale the token budget to the hardware you can afford.Done when: The loss curve is smooth and your validation loss beats the untrained baseline by a wide margin.
• Evaluate honestlyScore your checkpoint on HellaSwag and compare against the published GPT-2 124M number.Done when: You have a benchmark number for your own model and can say whether it is in the right region.
• Train across multiple GPUsStretchConvert the script to distributed data parallel and run it on more than one GPU, confirming the loss matches the single-GPU run for the same number of tokens.Done when: Throughput scales roughly with the GPU count and the loss curves agree.

Code and reading

What you will learn

• Supervised finetuning: conversations as training data, and who writes them
• The chat template — how turns become one token sequence with special tokens
• Why refusals, uncertainty and persona have to be taught explicitly
• Reinforcement learning on verifiable tasks, and reasoning emerging from it
• RLHF: a reward model standing in for human judgement, and how it gets gamed
• Which failure modes post-training fixes, and which it cannot

Practical exercises

• Write a small SFT dataset yourselfHand-write twenty conversations in the style you would want from an assistant. Include at least three where the right answer is a refusal and three where it is an admission of not knowing.Done when: You have twenty examples and can articulate the labelling policy you were applying — which is the document a real labelling team works from.
• Turn conversations into tokensApply a real chat template to your examples and tokenize the result. Print the token ids and find the special tokens marking each turn boundary.Done when: You can point to the exact tokens that tell the model where the user stops and the assistant starts.
• Design a reward, then break itPick a task and write a reward function for it. Then spend ten minutes deliberately finding outputs that score well while being useless.Done when: You found at least one way to game your own reward, and can explain why that is the central difficulty of RLHF.
• Actually finetune a small base modelStretchTake a small base model and finetune it on your twenty examples, then compare its answers to the base model on the same prompts.Done when: The finetuned model replies in the conversational format while the base model continues the document.

Code and reading