PyTorch for LLMs: Build GPT from Scratch

About this course

Note: This course contains the use of artificial intelligence.

PyTorch for LLMs: Build GPT from Scratch is a hands-on course that takes you from an empty file to a working language model you've written and trained yourself. It's built for developers, students, and anyone who uses AI tools and wants to understand what's actually happening inside them — no machine-learning background required.

Over the course you go from the single idea a language model rests on — predicting the next token — to a small but complete GPT, the same architecture behind today's large models, running and training on your own machine. Every part is written by hand in PyTorch, so by the end nothing about it stays a black box.

The course opens with foundations. You set up Python and PyTorch and cover the groundwork the rest of the course leans on: the vectors and matrix multiplication behind every layer, the small amount of probability and logarithms that show up in training, and a plain-language look at how a neural network actually learns. Nothing is assumed; each idea is introduced right before it's needed.

From there you build the model's front end — a character-level tokenizer that turns text into numbers, the data handling that feeds it, and a simple baseline you can train to see a loss curve move. Then comes the heart of the course: attention. You implement self-attention directly, with queries, keys, and values and the causal masking that keeps the model honest about what it can see, then extend it to multiple heads and wrap it, along with feed-forward layers, normalization, and residual connections, into transformer blocks. You stack those blocks, add positional information, and arrive at a full GPT.With the model built, you train it — watching the loss fall as it teaches itself to write from raw text — then generate from it and see how sampling choices shape what it produces. In the final part, you reshape your data into question-and-answer form and retrain, turning the model you built

Prices, discounts and availability are set by Udemy. We may earn a commission when you purchase through links on this site.