Context

Where each idea lives in real engines

Once you have built a technique yourself, the fastest way to go deeper is to read someone else's version. This maps each chapter onto the engines worth reading, from datacenter fleets to a 2,944-line C program on a Raspberry Pi.

Feature matrix

A blank means the engine does not need the feature rather than that it is missing one: batching is meaningless at batch 1, and paging is meaningless when one user owns the machine. Entries marked ? are not documented publicly.

Technique	vLLM	SGLang	TRT-LLM	llama.cpp	picoLM	quant.cpp	baseRT
S02BPE / SentencePiece	✓	✓	✓	✓	✓	✓	✓
S03GQA	✓	✓	✓	✓	✓	✓	✓
S05KV cache	✓	✓	✓	✓	✓	✓	✓
S06PagedAttention	✓	radix	✓	partial	—	—	✓
S07Prefix caching	✓	✓	✓	reuse	save/load	save/load	✓
S08Weight quantization	FP8/AWQ/GPTQ	FP8/AWQ	FP8/INT4	GGUF K	Q2_K–Q8_0	IQ2–Q8	.base Q2–Q8
S08KV quantization	FP8	FP8	INT8/FP8	Q8/Q4	fp16	4-bit	yes
S09Continuous batching	✓	✓	in-flight	parallel seqs	—	—	✓
S10Preemption	recompute	recompute	swap	—	—	—	?
S11Chunked prefill	✓	✓	chunked ctx	—	—	—	?
S12FlashAttention	✓	✓	✓	✓	online softmax	✓	✓
S13CUDA graphs / AOT	✓	✓	full AOT	—	n/a (CPU)	n/a	✓
S14Speculative decoding	ngram/EAGLE	EAGLE	✓	lookahead	—	—	?
S15Grammar constraints	XGrammar	built-in	✓	GBNF	JSON only	—	?
S16MoE	✓	✓	✓	✓	—	✓	✓
S17Tensor parallelism	✓	✓	✓	RPC	—	—	—
S18Disaggregated P/D	✓	✓	✓	—	—	—	—
S19OpenAI-compatible API	✓	✓	Triton	✓	—	bindings	✓

What to read, and why

vLLMdatacenterPython + CUDA
The reference implementation of most of this course. Introduced PagedAttention; the v1 engine core is where scheduler, block manager and model runner meet.
Read it for: the scheduler and the KV cache manager. They are the clearest production versions of S06, S07, S10 and S11.
SGLangdatacenterPython + CUDA
RadixAttention makes the prefix cache and the allocator one structure. Strongest story on structured output and jump-forward decoding.
Read it for: S07 and S15, where its design diverges most from vLLM.
TensorRT-LLMdatacenterC++ / Python
Compiles the whole model to a fixed graph ahead of time. Fastest on NVIDIA hardware; least flexible about shapes.
Read it for: what S13 looks like taken to its conclusion.
llama.cpplocalC++
The GGUF ecosystem and K-quants. Runs anywhere, and defined the quantization formats everyone else now reads.
Read it for: S02 and S08, the tokenizer and the quantization kernels.
picoLMembeddedC11
2,944 lines with no dependencies, running TinyLlama on a Raspberry Pi in 45 MB of resident RAM — that is the KV cache at a ~2k context; the mmap'd weights stay file-backed and are not counted. Fused dequant+dot, online softmax, grammar-constrained JSON.
Read it for: the whole of S01–S05 in a form you can hold in your head.
quant.cpplocalC
Single-header engine specialising in KV cache compression: Lloyd-Max codebooks, random Hadamard transforms, a full-precision window over recent tokens. (Its README says "lossless"; the codebook path is lossy but near-lossless in practice.)
Read it for: S08 applied to the cache rather than the weights. Most engines treat that as an afterthought.
baseRTlocalRust + Metal
One CLI that pulls a model, chats, and serves an OpenAI-compatible API. Paged KV, continuous batching and prefix caching on Apple Silicon and GB10.
Read it for: S19, and what the serving surface looks like when it targets one machine rather than a fleet.

Choosing one, honestly

Serving many users on GPUs: vLLM or SGLang. Both are excellent; SGLang is ahead on structured output and prefix-heavy workloads, vLLM has the broader hardware and feature surface.
Maximum throughput on NVIDIA, willing to compile: TensorRT-LLM.
One user, one machine: llama.cpp or baseRT. Batching and paging buy you nothing at this scale, and the win comes from quantization instead.
Very long context on a laptop: quant.cpp, whose whole design centres on compressing the KV cache rather than the weights.
Learning, or embedding in something small: picoLM. It is the only one on this list you can read end to end in a sitting.