Efficient Deployment.
A student-led seminar on serving large language models cheaply — the engineering that decides real-world cost per token, latency, and throughput once training is done. This page will hold the slide-by-slide study companion once the deck is available.
§0Status
The seminar hasn't been presented yet, so no slides are available. Once the deck lands, this page will get the same slide-by-slide walkthrough treatment as the other seminars — every slide embedded verbatim with a first-principles explanation underneath.
§1What the seminar covers
Training LLMs and serving them are two very different engineering problems. Serving is where a model actually gets used, and its unit economics come from a small stack of well-known tricks:
- Quantisation — running the model at INT8, INT4, or lower without noticeable quality loss.
- KV-cache optimisation — paged, quantised, or shared caches that keep long-context inference within memory budgets.
- Continuous batching — packing decode steps from many concurrent requests into a single GPU kernel launch instead of running them one at a time.
- Speculative decoding — using a small draft model to propose several tokens at once, then verifying them cheaply with the big model.
- Distillation and pruning — shrinking the model itself for the deployment target.
The seminar likely walks through several of these, with the goal of turning "we trained a nice model" into "we can serve it at a price users will pay".