Post-Training Is Where Models Are Made Now | Maxime Labonne
1 hr 3 min
In this episode, we talked with Maxime Labonne, Head of Post-Training at Liquid AI and author of the LLM Course on GitHub, about what happens to a model after pre-training and why that stage now matters so much. Maxime walks us through the short history of post-training, from supervised fine-tuning and preference optimization to reinforcement learning with verifiable rewards and now multi-domain on-policy distillation (MOPD), where separate teachers for code, math and other domains get merged back into one student. He explains why you still can’t skip SFT, why the order of the stages matters, and why RL works even though a long rollout gives you about one bit of signal. His intuition is that SFT and distillation raise pass@k, and RL then sharpens it into pass@1. We also get into training small models for agentic tasks directly inside the harness, which open-source RL stacks are good enough for serious work, looping transformers, and how much of the work big models do today could move to small ones running locally. We also talk about why models write worse than they used and safety issues. We talk about regulation, open source and Chinese models, whether a small startup can beat the big labs in a niche, and why he thinks RL environment companies are overrated. We finish with France vs San Francisco, Mistral and the regulation moat, and whether Europe can produce frontier models. Topics Why post-training matters more every yearThe modern pipeline: SFT, RL and distillationTraining agents and small modelsWhy models write worse than they used toSafety, incentives and regulationOpen source and Chinese modelsStartups vs the big labsEurope, Mistral and sovereign AIChapters 0:00 Intro 0:57 Why post-training matters 3:44 A short history of post-training 7:20 The modern pipeline: SFT, RL and distillation 19:41 Agents, architectures and small models 27:15 Why models write worse now 31:44 Safety, incentives and regulation 38:26 Open source and Chinese models 41:39 Recursive sel...