From paper mechanisms to serving systems
Three focused engineering lessons with derivations, failure cases and reproducible experiment specifications.
KV memory, paging and serving deadlines →
Separate allocation efficiency from model compute and queueing delay.
Speculative decoding: correctness before speed →
Derive acceptance and residual sampling with a two-token example.
MoE: active compute is not total memory →
Audit routing, capacity, communication and expert imbalance.