Stochastic Blog
Build a small transformer from scratch
In the last three posts we built up the pieces of a language model: sequence modeling, tokenization, and the decoder-only setup that predicts the next token. Now we put the pieces together into an actual transformer. The through-line for this post is a single question we keep asking