https://xuewyang.github.io/tags/kv-cache/ Recent content in KV Cache on Xuewen Yang Hugo en Mon, 09 Nov 2026 00:00:00 +0000 https://xuewyang.github.io/posts/kv/ Mon, 09 Nov 2026 00:00:00 +0000 https://xuewyang.github.io/posts/kv/ Inference as prefill plus decode, why generation needs a KV cache and training doesn’t, a cache in PyTorch, how big it gets and how MLA shrinks it, where decode step times come from, continuous batching and PagedAttention with a toy vLLM you can run on a CPU, how SGLang’s RadixAttention differs, and how speculative decoding verifies a draft in one pass.