跳到正文
Sebastian Raschka· @rasbt · X·· 2026-09-06AI 评分34
AI 导读

Sebastian Raschka 发布《Reasoning from Scratch》系列第二期视频,讲解 LLM 的文本生成流程与 KV 缓存,为后续加入推理技术打基础。视频涵盖 tokenization、加载预训练 Qwen3 模型、logits 与贪心解码、流式生成、KV 缓存加速实测,以及 torch.compile 编译与 CPU/GPU 性能对比。

正文 · 原文

Reasoning from scratch round 2: In this video, I cover the text generation process in LLMs and KV caching (to prepare the base model before adding reasoning techniques in the upcoming ones).

00:00 Introduction and reasoning model demo
01:55 How to work through the book
05:00 Chapter 2 overview
08:25 Checking PyTorch and hardware support
10:26 Apple silicon and MPS caveats
15:00 Cloud GPU options
16:08 Tokens and tokenization
18:20 Qwen3 and the Reasoning From Scratch package
23:05 Encoding and decoding text
26:24 Downloading weights and selecting a device
31:01 Loading the pretrained Qwen3 model
34:32 How LLMs generate text
36:47 Input tensors and batch dimensions
41:48 Running the model in inference mode
44:11 Logits and next-token predictions
49:21 Greedy decoding with argmax
52:28 Building a streaming text generator
01:01:28 Generating text and handling end-of-sequence tokens
01:06:00 Benchmarking text generation
01:14:34 How KV caching works
01:17:22 Adding KV caching and measuring the speedup
01:24:31 Model compilation with torch.compile
01:30:33 Combining compilation with KV caching
01:32:53 Comparing CPU and GPU performance
01:35:32 Recap and next steps

来源:Sebastian Raschka · x.com