From paper mechanisms to serving systems
Three focused engineering lessons with derivations, failure cases and reproducible experiment specifications.
This technical article displays the English edition.
KV memory, paging and serving deadlines →
Separate allocation efficiency from model compute and queueing delay.
Speculative decoding: correctness before speed →
Derive acceptance and residual sampling with a two-token example.
MoE: active compute is not total memory →
Audit routing, capacity, communication and expert imbalance.