In Posts 25 and 26 we built up the machinery of sequence models, one token at a time, and watched a recurrent network carry a hidden state forward through a sentence. That works fine when the input and the output are the same length. Translation is not so polite. A
In the last few posts we turned raw text into clean tokens, then into bags of words and TF-IDF vectors. Every one of those representations threw away the order of the words. This post puts the structure back. We load the Universal Dependencies English EWT treebank, train a Conditional
In posts 23 and 24 we taught the machine to read: we loaded raw text, cleaned it with regular expressions, and ended up with a tidy table of documents. What we never did was decide what a document is once the cleaning is done. A string of characters is not
In Post 23 we closed the gap between a model that memorizes and a model that generalizes, using weight decay, dropout, and early stopping to hold the bias-variance balance. That gave us a network that fits CIFAR-10 without falling apart on the test set. But a well-regularized
In the background posts we built the pieces: tensors and automatic differentiation in Posts 2 to 5, and in Post 18 the MNIST dataset, its loader, and a first baseline model. This post puts those pieces under load. We build a multilayer perceptron on raw pixels, check a hand-derived
Post 20 gave us the probability toolkit, and Post 21 pushed uncertainty into deep models. This post keeps the same question, uncertainty, and moves it onto sequences and functions. We will read a temporal model as a graph, fit a Gaussian hidden Markov model to hourly power use, run four
In Post 3 we fit a least-squares line to a dataset and read a single number off it. In Post 4 we learned to distrust that number until we saw it on held-out rows. This post builds the machinery that replaces the single number with a distribution. We
In Post 13 we learned to ask causal questions, and in Post 14 we saw how randomization answers them cleanly. This post takes the same question into the messy world where we cannot randomize: did a job training program actually raise people's earnings? We will work through the
In Post 14 we built the baselines that every forecast has to beat, and in Post 13 we learned to read autocorrelation and lag features. Now we point both at a real meter. This post takes four years of minute-by-minute household electricity readings and turns them into a
In Post 2 we built softmax regression from scratch, in Post 13 we let autograd compute our gradients, and in Post 14 we added regularization to keep the weights honest. Every one of those posts quietly assumed the same thing: that once we had a gradient, we knew what to
Post 15 taught us to find structure in unlabeled tabular data by clustering rows that look alike. This post goes after a different kind of unlabeled structure, the kind that lives between rows rather than inside them: what gets bought together. There is no target column, no train and test
In post 15 we built the digits loading and inspection pipeline, and we ended with a matrix of 60,000 images, each one a 28 by 28 grid of pixels. Flatten that grid and every image becomes a point in 784 dimensions. This post takes those points and asks a