Test-Time Training.
A student-led seminar on networks that keep learning at inference — adapting weights, or a small piece of them, to the specific example the model is looking at right now. This page will hold the slide-by-slide study companion once the deck is available.
§0Status
The seminar hasn't been presented yet, so no slides are available. Once the deck lands, this page will get the same slide-by-slide walkthrough treatment as the other seminars — every slide embedded verbatim with a first-principles explanation underneath.
§1What the seminar covers
Test-Time Training (TTT) breaks the usual assumption that a model's weights are frozen at inference. Instead, for each test example (or short window of examples), the model runs a tiny self-supervised optimisation loop on the input itself — for example, a masked-reconstruction objective — nudging its parameters to fit that specific instance before producing the prediction.
The recent TTT-layers line (Sun et al., 2024) reframes this idea as a sequence model: the RNN's hidden state is a small neural network whose weights are updated by gradient descent at every timestep, so the "state" has essentially unlimited capacity. This connects it directly to State-Space Models and linear RNNs, which are also efficient sequence models with a rich internal state.
The bounded-state problem that TTT tries to solve is discussed in Lecture 6 §5 under linear attention's limitations. LoRA and adapters — parameter-efficient train-time analogues — are covered in the 2024 mid-sem PYQ.