LLM on Xuewen Yang

LLM on Xuewen Yang https://xuewyang.github.io/tags/llm/ Recent content in LLM on Xuewen Yang Hugo en Sun, 01 Nov 2026 00:00:00 +0000 Parallelism for LLM training, with numbers you can check by hand https://xuewyang.github.io/posts/parallelism/ Sun, 01 Nov 2026 00:00:00 +0000 https://xuewyang.github.io/posts/parallelism/ Data, ZeRO, tensor, sequence, context, pipeline and expert parallelism, each worked through with a tiny example: what gets split, what moves between GPUs, and when to use which.