Why a 1B Model Beat a 70B LLM at Following the Rules
1 hr 0 min
👉 Discover how Cambridge Spark helps organisations build the data and AI capabilities needed to turn strategy into measurable impact: [cambridgespark.com](https://www.cambridgespark.com/?utm_campaign=6955829-Podcast%202024%20-%20Onwards&utm_source=Podcast&utm_medium=RSS) An LLM can give well-formed output 99% of the time and still break the domain's rules in most of it. You want to put an LLM into a real decision process, such as invoice approvals, pricing or clinical recommendations. The answers read well. But are they actually allowed, and do they solve the problem? Professor Alessandra Russo is Professor in Applied Computational Logic at Imperial College London and Director of the Thomson Reuters Imperial Frontier AI Research Lab. She co-directs the UKRI Centre for Doctoral Training in Safe and Trusted AI with King's College London. Her neuro-symbolic AI research combines logic with machine learning to build AI systems with functional guarantees. Jeremy Bradley and Alessandra go inside SEM-CTRL, her team's method for semantically controlled decoding in open weight models. You'll hear how a symbolic checker sits in the decoder and masks invalid tokens before they are ever chosen. Monte Carlo tree search then steers generation towards correct answers. No fine-tuning required. This episode is for data scientists, ML engineers and technical leaders responsible for enterprise AI deployment in regulated or high-stakes settings. It assumes some comfort with how LLMs generate tokens. If you want a light overview of AI trends, this one goes further. ## Chapter Markers 00:00 Why AI should be able to say "I don't know" 00:19 Introducing Professor Alessandra Russo 01:32 What organisations worry about beyond hallucination 04:30 Open weight models versus frontier APIs 08:37 The risks that come with open weights 11:16 Inside the SE-CTRL paper 12:39 Validity versus correctness in the blocks world 16:46 A pricing guardrail that needs memory 19:48 Context-sensitive...
