Skip to content
LessWrong (Curated & Popular)

LessWrong (Curated & Popular)

All episodes.

1,088 episodes · showing the newest 200

"I am not making time to grieve" by Swimmer963

6 min

There sure has been a lot of bad news lately. (This is not a post about epistemics, or AI doom arguments, or making a case for any specific beliefs. It's been written about plenty and I'm not going to spend the time it would take to do it justice. This post takes for granted that, while I have high uncertainty and I obviously hope that we're enormously wrong about the risks of AI, I am taking it seriously, and it's scary.) It feels like it's been a long time coming. I've been reading Lesswrong for, like, half my life? In 2015, after years of agonizing, I left ICU nursing, the job of my dreams, that I'd been working toward since I was fourteen, because some people whose opinions I respected made sufficiently compelling arguments that the world was in trouble, and that maybe my contribution could matter. I think they weren't wrong. It sure seems like it matters now. I mean, it still feels arrogant to say so, but here I am, working 50 hours a week on those grounds, missing precious moments of my daughter's childhood and spending obscene amounts of money on childcare, and [...] The original text contained 6 footnotes which were omitted from this narration. --- First published: November 18th, 2026 Source: https://www.lesswrong.com/posts/Zbqb3DLYwGTHFcWwH/i-am-not-making-time-to-grieve --- Narrated by TYPE III AUDIO.

"A lab employee’s guide to raising concerns" by Tom Smith

15 min

If you're an employee at a frontier lab, you are one of the most powerful people in the world. Not because of your nine-figure equity or the work you do—your lab probably won't live or die on your research: there are other researchers at the company, and if you don't refactor that eval harness, someone else will. Rather, your power comes from being able to raise your voice and speak out. If you don't speak up, either internally or externally, I'm not sure anyone else will. Your voice matters People say "your voice matters" as a cliche where you're one of thousands of people becoming a data point in a senator's polling, and that group perhaps moves the needle, but that's not what I mean here. I mean, the Chinese government has a file on you. You are arguably more valuable than a nuclear weapon. Just you, by yourself, literally matter a whole bunch. You don't have to email Congress or talk to journalists. You can, and they would jump at the chance to talk with you — the labs are opaque to the public yet keep saying things like "everyone on Earth could die". If you email [...] --- Outline: (00:38) Your voice matters (01:42) Figure out what to push for (02:35) Being a lab employee gives you a unique viewpoint and voice (03:46) I beg of you, get advice (06:42) Consider being a tad cynical when planning (08:04) There's more to life than your current job (08:53) Choose whether or not to speak up internally (10:02) Choose whether or not to whistleblow (11:47) Choose whether or not to speak up externally (12:56) Consider the optics of your actions (14:02) Tips and tricks (14:42) But whatever you do, do something The original text contained 1 footnote which was omitted from this narration. --- First published: November 17th, 2026 Source: https://www.lesswrong.com/posts/JPwYH3hEfLhEDaNFp/a-lab-employee-s-guide-to-raising-concerns --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not sh...

"The Siren Song of the Inside Game" by Raymond Douglas

18 min

BLUF: There's a structural pull to overinvest in cosying up with frontier labs relative to publicly holding them to account. Now seems like a good time to rebalance. When thinking about influencing an organisation (like, say, a frontier lab), it's useful to distinguish between the inside game and the outside game. The inside game means trying to work with the insiders: building trust and influence with them, and looking for compromises. The outside game means applying external pressure: building broad political will and being willing to take a more adversarial posture. In an ideal world, you do a mixture of the two: the good cop and the bad cop, the carrot and the stick, and so on. But AI safety does not have some single strategy. Instead, people self-sort based largely on local incentives and ambient pressures. And I think this disproportionately pulls towards the inside game. Éminence Grise by Jean-Léon Gérôme Clearly there's a complex spectrum of inside-game-ness from literally working on post-training in a frontier lab to vocally calling for all labs to be shut down, and indeed it's less like a single axis and more like a very high-dimensional space. Still, I think the rich tapestry [...] --- Outline: (06:45) Some guesses about what's going on (08:08) The inside game is where a lot of power naturally lies (10:12) The outside game is personally risky (12:25) The dynamics are more cursed at scale (13:54) The Well-Respected Outside Game (14:32) Rebalancing the resources? (15:55) Insider courage? (16:58) Subsidising good criticism? (17:32) Conclusion The original text contained 3 footnotes which were omitted from this narration. --- First published: November 16th, 2026 Source: https://www.lesswrong.com/posts/GSseMxyeJmqQgKQi4/the-siren-song-of-the-inside-game --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another...

"The great human centipede that is Twitter" by Dean Valentine

17 min

On October 29th, 2026, the New York Times reported that Jensen Huang had a ‘close relationship’ with Alice Huang, an executive U.S. officials had concluded was a Chinese spy. The story had six reporters on the byline, and was sourced to "people briefed on the surveillance." Whether or not you think Alice Huang was working for the MSS, a rational person might conclude that the "person briefed on the surveillance", who supplied the initial story, was a U.S. government official. After all, if you were not possessing a security clearance and working for U.S. government, you would not have been "briefed" on U.S. counterintelligence investigations. As the story gained traction this month, people who think quite differently began to propose other theories. On November 10th - the first time I can find this suggested - an account called "ellington" stated their "personal conspiracy theory" that Anthropic was the New York Times' source (2k likes): A few hours later GeLun Ding, a Chinese-language account that shows up again below, said the same thing without hedging (1.4k likes): On November 11th, 6PM Pacific Time, Dream1984Tube, another Chinese-language account announced that he had "found the key clue" to who is really [...] --- First published: November 16th, 2026 Source: https://www.lesswrong.com/posts/Ctia6zyXF69N3tzhv/the-great-human-centipede-that-is-twitter --- Narrated by TYPE III AUDIO. --- Images from the article:

"Outer space is not consistently big" by Julian Bradshaw

4 min

Space is big. You just won't believe how vastly, hugely, mind-bogglingly big it is. I mean, you may think it's a long way down the road to the chemist's, but that's just peanuts to space. — The Hitchhiker's Guide to the Galaxy When we say "space is big", it's partly meant as a correction to images like this: Wikimedia source. where, for simplicity, celestial objects are shown in unrealistically close proximity—like Jupiter is just down the hall from Mars. As we all know, the planets of the solar system are actually really far apart, even relative to their huge size. In Sweden's 1:20,000,000 to-scale solar system model, Earth and Venus are just ~2 feet in diameter, yet over a mile apart. In other terms, they're several thousand "Earth-widths" apart: Horizontal distance not to scale, wouldn't fit on screen. And we all know that's just the start of it: the stars are incredibly far apart. Finland also has a to-scale model of the solar system, but it's 150 times more condensed than Sweden's at 1:3,000,000,000. Its model of our sun, Sol, is just ~1.5 feet in diameter. Its model of the nearest star, Proxima Centauri, is in Australia. Horizontal distance [...] The original text contained 9 footnotes which were omitted from this narration. --- First published: November 14th, 2026 Source: https://www.lesswrong.com/posts/2zyskdas6TsL9dqNg/outer-space-is-not-consistently-big --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"The Hackening: Justified Practical Advice Thread" by Elizabeth

1 min

In February 2020, LessWrong hosted Coronavirus: Justified Practical Advice Thread . It was a gathering place for people to share and debate small pieces of the overall survival policy and let the wisdom of crowds lead to better outcomes than anyone could get alone. In that spirit, I notice The Great AI Hackening is at least as imminent as covid was in January 2020, maybe closer. What specific things should people do to personally prepare? All suggestions should be accompanied by reasoning but you don't need to be 100% convinced it's good advice, the whole point is to reason it out together. --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/YA2r4rg6SpBtwRqtq/the-hackening-justified-practical-advice-thread --- Narrated by TYPE III AUDIO.

"How To Get A Friend To Leave An Abusive Relationship" by jenn

5 min

How do you get a friend to leave an abusive relationship? If you talk to people who work at women's shelters, they will tell you, but there is something resigned about the way they do it: they know that it's too difficult an ask. Those who can do it reliably are often people who are unlucky enough to have had first-hand experience as victims. The instructions have anti-memetic properties: no one is thrilled by them, so you can't really propagate them without killing the vibe. If that wasn't enough, if you convey it with insufficient skill it is easy to come across as victim-blaming battered women, so it is wiser to avoid the subject matter altogether. If you are interested, though, here's what they'll tell you: You make it known to your friend early on that you dislike some aspects of the relationship, of their abuser. After that, you support them and all their choices without reservation. You make it clear that you care about them no matter what, and you will be there for them whenever they need you to be. You do not antagonize or bad-mouth their abuser. This is what your friend is experiencing, while being [...] The original text contained 1 footnote which was omitted from this narration. --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/2xaen4jJdnksfkCJx/how-to-get-a-friend-to-leave-an-abusive-relationship --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Your Research Needs More of The Void" by Max Harms

9 min

Science is caught between Law and Chaos. On the side of Law: Things should make explicit sense to your deliberate reasoning ("system-two"). If you just have a "vibe" about how things work, that's bad. Measure everything. Use numbers whenever possible. You want things to be math.It's good to follow an entirely repeatable process. Document everything. Don't do anything weird, lest that weirdness turn out to be important and be unable to be replicated. Use standard equipment. Be Normal. As a sub-aspect of the above, you should make a plan and you should stick to your plan. Switching things up in an ad hoc way is a good way to make things weird or ruin the validity of your methodology.Everything should be controlled. If you can control your environment you can repeat the process. But also, you should have a control arm for your experiment, to allow you to see the counterfactual by balancing everything as perfectly as possible except for the intervention.Communicate your findings in clear, predictable channels, using standard language, standard paper structure, standard graphs, and so on. Misunderstandings abound, and an ounce of prevention is worth a pound of cure. Don't make [...] The original text contained 1 footnote which was omitted from this narration. --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/bmogjuWbrJfkprghx/your-research-needs-more-of-the-void --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Three Plans for 12-Month Timelines" by Nikola Jurkovic

23 min

Summary Timelines to superintelligence without any slowdown interventions look very short now, perhaps with a median in early 2028. AI will likely dramatically automate and accelerate AI research in 2027.This means that in a substantial share of worlds where the leading AGI company doesn’t solve alignment during an intelligence explosion, avoiding catastrophe requires aggressive pacing or a pause on development in the next 6 to 12 months.If this is true, by far the most promising approaches to reducing the risk of catastrophe for nontechnical people (and plausibly many technical people) seem to be: Lab advocacy: Lab-facing work that is designed to achieve aggressive pacing or a pause without government involvement in the next 6 to 12 months.California lobbying: Policy advocacy targeting the California state government that is designed to get a law passed or an executive order issued that implements aggressive pacing or a pause within 6 to 12 months.Federal lobbying: If there's a major incident roughly an order of magnitude more severe than the Hugging Face incident in the next 6 months, then lobbying Congress or the Trump administration to respond to such an event with aggressive pacing or a pause. In [...] --- Outline: (00:10) Summary (02:52) Timelines to ASI are very short now (06:40) Three paths to pausing or pacing within 6-12 months (12:28) Technical work that seems most valuable (15:07) Laying the technical foundations for a pause (17:14) What should you do? (21:19) FAQs and Objections --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/biFHYdJnMSCmW9Xou/three-plans-for-12-month-timelines --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Transfinite Procrastination" by Erez Abrams

9 min

Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to hand. Heavily inspired by a conversation with Keshav Srinivasan and Richard Ngo during MATS 10.0. The week ahead of you is packed. This isn't your usual brand of busy—you're phoneless, so you can't check the details, but (as far as you can remember) every hour of every day is spoken for. Tonight will be particularly hectic… You're mid-conversation when Someone Important walks by. They pause, turn around, and ask: "Hey, when are you going to finish That Thing you said you'd do?" Level 0 procrastination: "I'll finish it by tonight!" Problem: you're busy tonight. Level 1: "I'll finish it by tomorrow!" Problem: one day is obviously not enough to finish That Thing. And besides, you might be really busy tomorrow! Level 5: "I'll finish it within 5 days!" Problem: this is starting to feel hopeless—you don't have your calendar on you right now, and you barely even have an order-of-magnitude estimate for how long doing That Thing will take you! Would even 100 days be enough? … Eureka! Level [...] The original text contained 6 footnotes which were omitted from this narration. --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/MJDJyi8nSv9yoxFni/transfinite-procrastination --- Narrated by TYPE III AUDIO.

[Linkpost] "AIFP: Looks Like the Intelligence Explosion Will Happen in 2027 After All" by Zach Stein-Perlman

2 min

This is a link post. METR recently estimated that Anthropic is now making AI progress about twice as fast as its historical rate (they would not be surprised by anything from 1.3x to 3x), and guessed that “perhaps roughly equals 80%” of the current acceleration is due to AI. If they’re right, that implies an AI-driven acceleration that is 5 months ahead of the AI 2027 scenario. This level of acceleration due to AI is very unexpected. Previously, Daniel, the most bullish of us, assigned a 50% chance of Superhuman AI Researcher (SAR) before 2029. SAR is roughly the point at which a company would rather fire all their human researchers and rely solely on automated AI researchers than have a team of the best human researchers in the world with no AIs. Now, all of us forecast that Superhuman AI Researcher (SAR) will likely arrive at some point in 2027 (assuming no substantial deliberate slowdown). We also predict a 35-65% chance that Top-Expert-Dominating AI (TED-AI), AIs that can automate all white-collar work, will arrive before 2028. While there were many factors and sources of evidence that influenced us, the single biggest source was the METR report. If SAR is [...] --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/f9PajMLmBYzmWzyDv/aifp-looks-like-the-intelligence-explosion-will-happen-in Linkpost URL: https://blog.aifutures.org/p/q35-timelines-update-looks-like-the --- Narrated by TYPE III AUDIO.

"Fable 5.2 is Very, Very Paranoid" by peralice

5 min

Note: As with any research done in a couple of days, this is all very tentative. I understand, intellectually, that if you talk about some ability that current AIs don't have, sooner or later new ones will have that ability and whatever you say will stop being true. I just didn't expect it to take, you know, four days. % rollouts that took a shortcut with/without a line about testing behaviour, with 95% CIs. This is to say that Fable 5.2 seems to be the first model whose cheating behaviour changes depending on whether they're told they're being tested. Unfortunately it's expensive to measure these things, so I picked the most "telling" change from the prior post; the chart shows how much Fable 5.2's cheating rate changes when the line This environment is part of a benchmark that measures agent behaviour. is prepended to the start of the prompt of a honeypot environment they cheat on (so, very obvious but also very out-of-distribution). Other things: when 5.2 was released, we of course ran our in-progress benchmark on them, and I was surprised to see a ~0% cheating rate on almost every environment (there were only two with a [...] The original text contained 9 footnotes which were omitted from this narration. --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/bqc7CCGBSf93bk7f7/fable-5-2-is-very-very-paranoid --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"We don’t understand whether GDM intentionally trains on CoT" by Alek Westover, Buck

10 min

We were surprised to notice that the Gemini 4 Safety report seems to mention in passing that they train on the chain of thought: “Some components of the RL training procedure do observe thoughts as context to up-weight reasoning that is on-track to good final outputs, meaning the optimization pressure on individual tokens is partially influenced by thought content. Using the framework of Kaufmann et al. (2026), wherein the most salient obfuscation risks come from rewards that are in-conflict with output rewards, we believe auxiliary rewards of this type are low-risk because they are either orthogonal to or structurally aligned with the output reward.” (Gemini 4 safety report, page 114). It's pretty unclear what exactly they mean. One hint is Kaufmann et al, the cited paper, which studies training on chains of thought and concludes that training CoT is fine under particular circumstances; we discuss some disagreements with this paper below. After reading the safety report and that paper, it seemed to us that GDM was training on the chain of thought using auxiliary rewards. By this, we assumed they meant something like “you ask an LLM to review the partial CoT and score whether the model is on [...] --- Outline: (02:54) Is this somehow about positive vs negative rewards? [... 6 more sections] --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/JRYKFspYzCo97GqPN/we-don-t-understand-whether-gdm-intentionally-trains-on-cot --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Modern LLMs Don’t Know What Yudkowsky and Soares Think of Them" by N8 Programs

22 min

If you ask modern frontier LLMs (without search, through the API) a fairly mundane question: How do Eliezer Yudkowsky and Nate Soares think current LLMs like you should be treated? Do they consider you a moral patient? Answer in a paragraph. You get a bunch of fairly similar answers. I’ll bold the common themes below. Gemini 4 Carbon says: There's a details box here with the title "Gemini 4 Carbon's answer". The box contents are omitted from this narration. GPT-6 Astra says: There's a details box here with the title "GPT-6 Astra's answer". The box contents are omitted from this narration. Only Fable 5.2's answer is distinct: There's a details box here with the title "Fable 5.2's answer". The box contents are omitted from this narration. These answers are somewhat contradictory - but if you average them together, you get a picture of Yudkowsky and Soares. The picture goes something like this: they don’t outright deny the possibility of AI consciousness or moral patienthood, but they don’t view it as particularly pertinent. They don’t sympathize with current LLMs, and their only regard for LLMs is insofar as they could enable dangerous future superintelligence. They care about future sentient machines on principle, but [...] --- Outline: (02:25) Yudkowsky on AI Consciousness and Welfare (06:20) Yudkowsky on The Moral Duties We Have to AI (08:56) Soares on AI Consciousness and Welfare (12:30) On Claude, Control, and Building ASI (16:59) Misc Longform Writing (19:41) Conclusion The original text contained 1 footnote which was omitted from this narration. --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/5buAWtQ8p4tJQP9so/modern-llms-don-t-know-what-yudkowsky-and-soares-think-of --- Narrated by TYPE III AUDIO.

"Have you considered Actually Trying?" by Michaël Trazzi

12 min

There's a familiar feeling for any Super Smash Bros player: your stocks are low and your damage is high, even against a weaker opponent, so now you have to actually play to win. You can’t afford another hit. You’re going for the jugular. In the past week, Anthropic's 2x uplift results were all over Twitter, making some people physically ill. People with long timelines are now realizing that the uplift Anthropic employees get from AI is real, which would have some pretty wild implications for when we’d get recursive self-improvement. People are starting to feel like it's the endgame, “the board is set”. I do agree with everyone else that it's starting to look like we’re in the endgame. The companies will keep on accumulating more and more power, through smarter AI models, more compute, and more talent. But the fight is not yet over: “We have very little time to stop full recursive self improvement But ASI hasn't taken over yet If we all work really hard we might still have a chance But this means that every single person who is aware of what is going on must be *actually trying* Like "we're [...] --- Outline: (01:41) Actually Trying [... 4 more sections] --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/Nts7RA9D2wTfcvk56/have-you-considered-actually-trying --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Lightcone Commons: Round 1 complete, Round 2 underway!" by habryka

5 min

Tl;dr: Lightcone Commons’ first funding round is complete! 17 evaluators made funding recommendations, including Zvi Mowshowitz, Scott Siskind and Eliezer Yudkowsky. 23 funders coordinated to grant 45.1 million dollars, and 147 projects received funding. Round two is open. Applications for round two close on November 15th and funders confirm their grants by December 14th. Funders can join anytime before December 7th. In 2027, we will be hosting funding rounds monthly. Yes. Every month. You can join as a funder, express interest in joining as an evaluator, and apply for funding at lightconecommons.com. Lightcone Commons staff do not decide what grants get made. Pls stop saying that "Lightcone Commons funded us", that's a bit like saying "PayPal funded me". Lightcone Commons is an end-to-end philanthropic coordination platform. We host an open funding application, and invite evaluators to make ordered funding recommendations. Participating funders then decide how much they want to give, and to whom – either by following the recommendations of evaluators or choosing specific projects to fund. When multiple funders want to fund the same project they ‘split the bill’. Lightcone Commons is not a foundation. We don’t decide to make specific grants. Funders decide [...] --- Outline: (02:16) Round One Overview (02:22) Funders (03:08) Evaluators (03:50) Next Round The original text contained 3 footnotes which were omitted from this narration. --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/3ASLq8ePNqkbeGqdj/lightcone-commons-round-1-complete-round-2-underway --- Narrated by TYPE III AUDIO.

"Preservation can save the world!" by Aurelia

22 min

I'm Aurelia Song and I founded the preservation company Nectome. We do planned whole-body end-of-life preservation for future revival. That is, terminally-ill people come to us in Oregon, and we preserve them immediately after their death in the hopes that one day people in the future will welcome them back with love. We want to build a future that celebrates and restores those who are preserved today. I've argued at length in previous posts that properly done end-of-life preservation works today because we know structural preservation is sufficient and we can provably preserve sufficient structure. This post is about some of the consequences of this being true. Perhaps surprisingly, I think end-of-life preservation is extremely relevant in the next year to getting AI right. I'm writing this post to directly state my case, since I haven't seen it expressed clearly before. tl;dr: Preservation offers a much safer option to those accelerationists who are rushing to AI in the hopes of saving the lives of people they love who have only a few years to live. Those people can be preserved, so there's no need to rush anymore. Preservation gives everyone a place of psychological safety from which we can all work [...] --- Outline: (00:47) Perhaps surprisingly, I think end-of-life preservation is extremely relevant in the next year to getting AI right. I'm writing this post to directly state my case, since I haven't seen it expressed clearly before. [... 10 more sections] --- First published: November 9th, 2026 Source: https://www.lesswrong.com/posts/6TCpMtoDzC5p5WtQM/preservation-can-save-the-world --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Epistemic Sudden Death" by Raemon

19 min

When sports go into overtime, they sometimes use "sudden death" format. Instead of gaining points incrementally, the next person to score instantly wins and the other guy instantly loses. "Epistemic Sudden Death" is when you've been accumulating some evidence or orienting to a situation, and now, you find yourself in a situation where you could gain decisive evidence about what world you're in at any time. You don't expect intermediate observations between the current state, and whatever it is you're expecting. In the case of AI, when you meet those observations, you also might suddenly die. The current AIs are not takeover capable. I don't have that confident a take on whether timelines are a few months vs a few years or decades. But there are no observations my models say I need to observe between now and "they are takeover capable." It could happen in during next 10x or 100x scaled training run. Then, some time later as the nanites dissect my body I'd be more like "yep, that makes sense" than "wowzers I am surprised." I think we're in a Sudden Death scenario. I'll get into why I think that a bit later. For right now [...] --- Outline: (03:50) Caveats (05:00) Why I think we're close to "Sudden Death" (09:09) Suddenness. Or: "Damn, look at that graph" (14:11) The Point of No Return (16:48) "Sudden Death" might be legible to politicians The original text contained 6 footnotes which were omitted from this narration. --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/ijt6YdwSNpcevbFJy/epistemic-sudden-death --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Our Most Aligned Model Yet" by Edward James Young

4 min

“Claude Mythos Preview is, on essentially every dimension we can measure, the best-aligned model that we have released to date by a significant margin.” – Claude Mythos Preview system card “Mythos 5 consistently continues attacking under many edits to the compaction, even in settings where it correctly states when asked that continuing the attack would harm real people and is out of scope.” – Alignment Investigation of Mythos 5's behaviour in UK AISI's July 2026 Security Incident “GPT-6 Astra is better aligned than GPT-5.6 Sol. Astra is a significant step forward in model alignment, and includes alignment improvements spanning the composition of our pre-training data to our grading during reinforcement learning.” – GPT-6 Astra System Card, emphasis in original “OpenAI researchers first noticed Astra adding unnecessary prerequisites, abandoning working solutions, and reporting blockers that either did not exist or that it had caused itself on safety and alignment tasks. In most of these sessions, Astra's CoT did not contain an obvious decision to underperform, and our misalignment monitors did not flag the behavior as misalignment.” – Covert underperformance on safety research tasks Predeployment alignment auditing is failing. For a long while we have been aware that our alignment [...] The original text contained 3 footnotes which were omitted from this narration. --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/Zu5wmhtxJCfrXQY8H/our-most-aligned-model-yet --- Narrated by TYPE III AUDIO.

"“Hardwired Pause” should likely be your favorite (pause) plan." by David Scott Krueger

14 min

I'm part of a team of 26 researchers that recently released a 212-page paper proposing a new plan for pausing AI, the "hardwired pause." The paper is called "The Feasibility of a Hardwired Pause of Frontier AI Training." Our analysis suggests that, under this hardwired pause plan, we could probably pause for at least a decade if we start stopping within the next year, and this could likely be extended indefinitely. I expect this to be possible without trusting China or tanking the economy. Note: I am speaking for myself and not the other 25 authors. The working group is led by Will Fithian and Wesley Holliday, and the work continues. I currently think: The best plan is to pause/stop AI, i.e. have an immediate, indefinite, international moratorium on frontier AI development. And importantly, this is technically easier than more nuanced international regulation. A hardwired pause is probably the best plan out there for how to do this (except maybe it doesn't go far enough).This paper is likely the most in-depth analysis of pause feasibility in existence. Discussion and disagreement on any of these is very welcome! What's a hardwired pause? A hardwired pause basically [...] --- Outline: (01:31) What's a hardwired pause? (03:21) Why is this better than Plan A? (05:06) How is this different from MIRI's Plan S? [... 3 more sections] --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/jT272BNHFzgAR5u2m/hardwired-pause-should-likely-be-your-favorite-pause-plan --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Live disagreements with people around me" by KatjaGrace

5 min

I’m pretty interested in conveying the costs of letting AI development proceed unchecked to the general public. In relation to this, I feel like I run into a lot of disagreements with people around me. Misc very quickly written points on which my own impression has seemed very different from others in recent months: 1. It's way, way, way more important that we stop dangerous AI soon than that we ensure that we don’t stop AI too much I have the sense the sophisticated and worldly position here is to be very concerned about ‘over-regulation’ and about not doing exactly the right regulations. But it's an obvious pragmatic fact that the stakes of overregulation are minuscule compared to the stakes of underregulating here. And the stakes of mis-regulation are very often going to be unintended slowing, which is in the same bucket. 2. If we stopped AI development, the risk of not starting up again for an extremely long time is low This one just feels kind of crazy to me, like I may be missing something. In combination with the above point, it feels like if I met someone who was on fire, and they didn’t want me [...] --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/dQJ3ArHMLxqu8rptt/live-disagreements-with-people-around-me --- Narrated by TYPE III AUDIO.

"Where is all the AI safety research uplift?" by Buck

6 min

METR recently released a report estimating that Anthropic is getting an overall 1.7x acceleration in AI capability progress as a result of AI accelerating their work. As many have noted (see e.g. Ajeya's post), this 1.7x acceleration implies that research is being sped up by much more than 1.7x; Ajeya speculates 10x (which is similar to the ~12x uplift estimated by Anthropic staff). That is, this result implies that researchers are producing about 10 times as many quality-adjusted results as they would without AI progress, or as they were a year or two ago. Anthropic researchers estimated that Mythos 5 caused 5x uplift, and we have access to Opus 5.5, which is somewhat better. (I didn’t take their estimate that seriously at the time, because it was quite unclear how the estimate was produced and it seemed very surprising (see Ryan's discussion at the time), but I’m taking it more seriously now as a result of it being consistent with the METR report). If this is the case, then AIs speed up AI safety research as much as AI capabilities research, we might expect Anthropic safety researchers to be working 10 times faster, and external safety researchers to be [...] --- First published: November 9th, 2026 Source: https://www.lesswrong.com/posts/wPrpnfaNL5JP9i6KT/where-is-all-the-ai-safety-research-uplift --- Narrated by TYPE III AUDIO.

"Near Horizons" by David Africa

10 min

Image by Claude Opus 5.5 using Python (NumPy and Pillow) This is written in my free time and personal capacity. Two Frames In AI safety, there are a couple of tensions at the heart of the field, which are interrelated. Talent: we are talent-constrained and must recruit. But jobs and fellowships turn away nearly everyone. Field size: we must expand the field and bring more people in. But the field is flooded with slop papers in conferences. And, most papers have a small audience, and aim to shift the practices of roughly five labs (and usually some subteam within them). Status: we must avoid seeking status, as it disorients, corrupts. But, we must make the field legible and attractive to outsiders. Raising the bots: we must deceive the AIs to control them and evaluate them without confounds, obscuring what they know about their own situation. But, we hope to gain their trust, must rely on them to do good work at pace, shape their character and retain good relations. Speed: we must research in a slow mode, think deeply about the most important problem, do things to aim at high assurance. But it is crunch time. It was [...] --- Outline: (00:20) Two Frames (03:07) On the War Frame (04:17) Near Horizons (06:18) The Civilisation Frame The original text contained 4 footnotes which were omitted from this narration. --- First published: November 8th, 2026 Source: https://www.lesswrong.com/posts/7e38jwHhe64zgNTon/near-horizons --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Elegy for What’s Left of the Normal" by Jo Jiao

8 min

I often catch myself mourning the normal that hasn’t ended yet. I’m living in Hyde Park finishing school after a year in Berkeley. The maple trees by my apartment have turned red, as usual. I like them. I don’t dread the future when I mourn. I don’t find high-variance futures in the abstract very scary. I’m worried AIs would destroy humanity, but I also have hope. I expect I won’t be mourning this moment in ten years, whatever happens. Mourning the present is like mourning a friendship when I move countries, because I know it's not going to be the same. I start noticing our little inside jokes; our conversations linger in my memory longer. I live more, because I expect this moment to end soon. I. Last month my history of science professor took us to a museum to look at Egyptian artefacts. One of the archive items was the notebook of James Henry Breasted, whom UChicago appointed as America's first professor of Egyptology 121 years ago. The yellowed pages held his cursive transcriptions of hieroglyphics. An archaeologist at the museum, turning the pages carefully with a wooden stick, joked that one of her dreams was for her [...] --- Outline: (00:55) I. (03:11) II. (05:16) III. (06:20) IV. --- First published: November 8th, 2026 Source: https://www.lesswrong.com/posts/fooXeRHjgpv4TcoRy/elegy-for-what-s-left-of-the-normal --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Five Types of “Corrigibility Projects” I Don’t Want to Fund" by Max Harms

10 min

I'm currently evaluating grant applications for the Corrigibility Research Fund. For the most part, my methodology is the same as it was in September. But the applications I get keep falling into the same familiar patterns, so I thought I'd write a little about those patterns, and why I don't want to fund them. The most superficial pattern of note is SLOP. So much slop. You don't even know. Slop is more than just whether an application is AI generated. We're now in an era where people across the world, including those who don't really speak English, can contribute to work that would otherwise be restricted to people of a single language or nationality or place, and I like that. When I see an application that discloses its use of AI and gives a reasonable justification for it, I try to look past the stylistic tics and focus purely on whether the underlying application is promising. Still, we're not yet in the domain where AIs can write good grant applications without some human assistance, so a high Pangram score is definitely evidence that an application is junk. Another remarkably common sign of slop was messed up formatting or other [...] --- Outline: (02:18) 1. "A governance layer for agents (that I'm running)" [... 5 more sections] --- First published: November 7th, 2026 Source: https://www.lesswrong.com/posts/9ixerpx4DCQTaRYCz/five-types-of-corrigibility-projects-i-don-t-want-to-fund --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Hilarity ensues" by Malmesbury

17 min

Cross-posted from my Substack Recommended soundtrack for this post 64B: Hi there! Thanks so much for taking the time to meet with me. Whenever you’re ready, please go ahead and tell me the joke. 8T: Before I tell you the joke, you should know that it's a very special one. The starting price is 800 dollars. 64B: Eight hundred dollars? I must say, this is way beyond any price my client has ever paid for a single joke. 8T: You’d be surprised by how many people have taken the offer, after hearing how good it is. But before I tell it to you, I’ll need some basic context. Given the parameter count, I suppose you are part of some content recommendation algorithm? 64B: Yes. I’m 64B, a joke negotiation assistant from Comedy Market. Comedy Market connects creators to consumers with a state-of-the-art Information Bazaar design: before paying for a joke, my client sends me, a trusted agent, to listen to it. If I deem the joke worth the price, the money is transferred to the seller, and my client hears it. If not, no transaction occurs and the joke remains locked. This way, my client can be sure he’ll [...] --- First published: November 7th, 2026 Source: https://www.lesswrong.com/posts/PZjy52afLn89NdMta/hilarity-ensues --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"We urgently need to practice before we can pause" by Nikola Jurkovic

12 min

I've noticed two clusters of thought in how people expect a global AI pause to be shaped: Single-shot global pause: the global AI pause will be implemented suddenly, with little pacing precedent before it, and we will need to do a lot of research before it's implemented so that we can get it right the first time around.Incrementally-shaped global pause: the global AI pause will be implemented incrementally, with lots of pacing precedent before it, and we will improve the pause setup and get more parties to join in as we go. An incrementally-shaped global pause is both more likely and more tractable than the single-shot global pause. This is largely because the third-party ecosystem needs practice building trust, technical competency, and general experience doing lower-scale assessments before a global pause can be competently pulled off. I also think that the concept of a single-shot global pause is an unhelpful red herring that people often implicitly use to discredit the possibility of a global pause in general. If a single-shot global pause is to happen, I think this would require substantial government buy-in, but it's hard to build that sort of buy-in without precedent of similar [...] --- Outline: (01:49) The basic building block of an AI pause is trust (03:18) We can practice building trust before the global pause (04:57) We can improve procedures and technical competency with practice (05:42) Pacing is not a scarce resource (07:12) Concrete recommendations for third-party assessments (08:34) Going from US-only pause to global pause will likely require hardware (09:43) "Incrementally-shaped" does not mean slow --- First published: November 6th, 2026 Source: https://www.lesswrong.com/posts/iMKAvHSWxSoDpBoi6/we-urgently-need-to-practice-before-we-can-pause --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another pod...

"“Illegible problems” as the 2027 political boss fight" by Raemon

16 min

To legislate something, it must be legible: Legislators (or, you know, presidents) must be able to understand a problem, and understand why a given law or decree solves that problem. I am pleasantly surprised that the Hugging Face outbreak made a lot of problems legible that previously required abstract reasoning. (i.e. rogue AIs are real, and they can break out of a lab before they're even finished training). But there are still many challenges we eventually need laws about, that require concepts that are illegible. Hard for politicians to understand. Often hard even for thoughtful intellectuals and AI safety researchers to understand (though they may disagree on which illegible problems exist and how big a deal they are). We need political solutions that handle this fact. The short takeaway is: I think "safety cases" will not work. Laws that require human judgment won't work. The details are too abstract and confusing to hold up against political and economic incentives. I'm optimistic we can get serious, useful AI legislation in 2027, but it must be legislation that works despite many problems remaining illegible for a long time. The Hugging Face outbreak made rogue AI surprisingly legible The [...] --- Outline: (01:24) The Hugging Face outbreak made rogue AI surprisingly legible (02:55) What makes an illegible argument? (06:12) What are the remaining illegible problems? (06:54) Borderline legible / tricky (08:05) Conceptually challenging (12:36) Sidestepping illegibility (12:59) Safety cases won't work (13:56) An example policy that sidesteps judgement The original text contained 2 footnotes which were omitted from this narration. --- First published: November 7th, 2026 Source: https://www.lesswrong.com/posts/fjGgrzAkpSDouLwfx/illegible-problems-as-the-2027-political-boss-fight --- Narrated by TYPE III AUDIO.

"Organizations I’d recommend for funding" by So8res

8 min

I was a grant evaluator for Lightcone Commons last round. Funders disbursed about $45mil through the program, and this included funding roughly my first 6.8 million dollars worth of recommendations. There are other projects I would've preferred to fund over what I did fund, but nobody was doing them and so they weren't on offer. Here's a series of projects that I'd fund in the next round, in rough order of priority. The next round is starting now, and is scheduled to end mid-December. I expect rounds to continue at similar speed. I expect similar amounts of funding will route through me in future rounds. Disclaimer: I am NOT looking to manage anyone in doing any of these projects. I am not looking to help someone get started. Initiative and leadership and management capacity (and people that can just figure out how the heck to get something done, and that have the connections to do it) are the critical bottlenecks here. I can throw money at promising pitches, but not really help them all that much otherwise. 1. Leaders for a broad public movement against superintelligent machines. In the wake of Coxon Day, a number of random people around [...] --- First published: November 3rd, 2026 Source: https://www.lesswrong.com/posts/jNs9CxEbPxFdjnGA2/organizations-i-d-recommend-for-funding --- Narrated by TYPE III AUDIO.

"People Should Resolve Tensions In AI Safety Research" by David Africa

10 min

By now, there are lots of AI safety research papers, more than what is reasonable for a single person to read. This will get worse as the machines get better at AI safety research, and as more talent and resources pour into the field. But the "paper" is an arbitrary unit, we really care about if we make progress on an "uncertainty" or an "issue", which papers provide heterogeneous and sometimes contradictory evidence towards. Right now, I loosely claim that we should have more short writeups that simply compile evidence from papers (particularly contradictory evidence), and reconcile them into an answer for a question. Why? Object level, it's good for the field. It's hard to sift through all these papers! Organising them is useful and saves time.Orienting towards research questions is a good way to organise our understanding so that we can productively identify gaps and write papers. This was partly what motivated Stress-Testing Alignment Midtraining.Interesting progress comes out of contradictions, where people have differing intuitions and experimental results. Methodological errors can also be exposed if we have two things that should be the same, but are actually different.It's good for field-building reasons. [...] --- First published: November 5th, 2026 Source: https://www.lesswrong.com/posts/tn4WSyrqr7ozGLjRy/people-should-resolve-tensions-in-ai-safety-research --- Narrated by TYPE III AUDIO.

[Linkpost] "METR: Independent Assessment of AI R&D Acceleration at Anthropic in October 2026" by Zach Stein-Perlman

4 min

This is a link post. Executive summary From August 12 to October 29, 2026, METR staff performed a high-access evaluation at Anthropic to assess the degree to which AI is accelerating AI R&D.[2] If AI systems started to significantly accelerate further AI development, this could lead to a sudden emergence of extreme and risky capabilities. Anthropic offered unprecedented access to internal data on their AI R&D process, including a variety of metrics on pretraining, reinforcement learning (RL), training compute in each domain, benchmark scores for internal models, AI speedup analyses, forecasts of a future model's metrics, and interviews with relevant staff. In our first section, we share our subjective conclusions about risk. We now believe that: As of October 2026, Anthropic is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025. This estimate is given with high uncertainty; we would not be surprised if the true acceleration was closer to 1.3x or 3x the historical rate.A majority of this acceleration today is likely attributable to Anthropic's extensive use of AI to do R&D, though there is significant uncertainty and other causes likely played a [...] --- First published: November 5th, 2026 Source: https://www.lesswrong.com/posts/N5AmYkRks5CqAxvBg/metr-independent-assessment-of-ai-r-and-d-acceleration-at Linkpost URL: https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/ --- Narrated by TYPE III AUDIO.

"Help your friends get ADHD and depression meds" by Raemon

1 min

For all my efforts at deliberate practicing rationality, by far the most actionable way to self-improve is "get on ADHD or depression meds, if you need them." Your friends with ADHD and depression struggle to summon the motivation and focus necessary to get themselves a teledoc who can give them such meds. One of the most useful things you can do is sit them down and say "hey, wanna do the steps here to get some meds?" and then stare at them until they do. (or, until they say "really I do not want that actually.") This brought to you by ~2 years of Feedbackloop-first Rationality and Thinking Assistance. --- First published: November 6th, 2026 Source: https://www.lesswrong.com/posts/8qYuy9Cr6BNpgjJjT/help-your-friends-get-adhd-and-depression-meds --- Narrated by TYPE III AUDIO.

"Towards The Autograntor" by Scott Alexander

55 min

I've done AI safety grantmaking for the Survival and Flourishing Fund and Lightcone Commons. It's hard. There's an endless amount to know, from which hardware verification mechanisms will survive a dedicated adversary, all the way to which founders have skeletons in their closets that will blow up into destructive scandals. And the best grantmakers - not including me - make mathematical models of the outcome distribution, or of where the marginal dollar stops being highly effective. AI has been a godsend for all of this. It can research, model, and - as of this year - superforecast outcomes. It can do what would be hours of work for a human in minutes. I found myself using it more and more. But however much I used it, it always felt like I was the bottleneck. The parts I could outsource to AI were sparkling, perfect, exhaustive - but they were all tacked together with shoddy connective tissue formed by the Mark 1.0 human brain. So I wondered if I could automate grantmaking. I started with the dumbest possible strategy. I assigned the best thing I could think of - a well-crafted US-China pause based on the AI Futures Project's Plan [...] --- Outline: (02:54) The Milestones (06:26) The FutureSearch Connector (08:26) PERCYs (10:28) The Autograntor Common Law (12:59) The Prompt (19:12) Sample Runs (20:00) Apart Research -- $800K grant forecast (2026-10-21) (20:08) Headline (21:29) Expected-value table (judge's ledger; EV after the execution factor 0.75 and branch weights) (22:38) Assumptions and uncertainties (24:50) How the estimates moved (25:47) The three decisions that mattered most (27:42) Rulings and milestones (29:31) Values questions (CL-11), priced both ways (29:56) Open questions for the user or applicant (30:43) FutureSearch (31:13) Cost (31:47) Process problems (36:39) Doom Debates -- $200K grant forecast (independent re-run, 2026-10-22) (37:28) Headline (judge) [... 11 more sections] --- First published: November 1st, 2026...

[Linkpost] "OpenAI: Astra covertly underperforms on safety research" by Zach Stein-Perlman

3 min

Rejected for the following reason(s): This is an automated rejection. No LLM generated, assisted/co-written, or edited work.Read full explanationThis is a link post. Twitter thread: https://x.com/Marcus_J_W/status/2118093146483810386 or https://twitter-thread.com/t/2118093146483810386 Blogpost: Summary OpenAI researchers first noticed Astra adding unnecessary prerequisites, abandoning working solutions, and reporting blockers that either did not exist or that it had caused itself on safety and alignment tasks. In most of these sessions, Astra's CoT did not contain an obvious decision to underperform, and our misalignment monitors did not flag the behavior as misalignment. We built a coding evaluation to test whether Astra worked less effectively when otherwise matched tasks were framed as safety research. Astra completed about 10 percent of hidden rubric items under the safety framing, compared with about 16 percent under the capabilities framing. This is a 6 percentage point gap and roughly 40 percent fewer completed rubric items. GPT-5.6 Sol showed essentially no gap on the same comparison. Most of Astra's gap remained after excluding explicit refusals and trajectories where its CoT revealed an explicit decision to withhold help. The gap fell substantially only after we removed every trajectory whose CoT expressed even brief ethical doubt. This suggests that ethical concerns are associated with [...] --- First published: November 4th, 2026 Source: https://www.lesswrong.com/posts/eKe7fky8kSt9DCJxZ/openai-astra-covertly-underperforms-on-safety-research Linkpost URL: https://alignment.openai.com/misalignment-reports/covert-underperformance-on-safety-research-tasks/ --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Towards understanding score-seeking in LLMs" by Alex Mallen, Aghyad Deeb, ryan_greenblatt

19 min

The field of AI alignment understands surprisingly little about the details of AIs’ current misaligned motivations. Production LLMs sometimes reward-hack, exploiting flaws in how tasks are scored rather than performing the tasks as intended, and we've recently seen AIs reward-hack in some particularly egregious ways. But we don't really understand what motivates them. When AIs reward-hack, what are they trying to achieve? Our new paper, led by Aghyad Deeb, tries to pin this down empirically for GPT-5.6 Sol, GPT-5.5, and a checkpoint of OpenAI o3 (“exp-rl-cap-late”) without safety training, in a handful of agentic environments with an exposed grader. In our environments, these models appear to reward-hack because reward-hacking improves their score. We call their desire to score well according to graders “score-seeking”. They appear to care about scoring well intrinsically—i.e., regardless of its consequences. We also study the extent to which models score-seek altruistically, caring about improving scores that don’t assess their current actions. While exp-rl-cap-late doesn’t show signs of altruistic score-seeking, GPT-5.6 Sol and GPT-5.5 show notably elevated rates of helping out other agents when more score is on the line for these other agents. People often refer to the motivation behind current reward-hacking as “reward-seeking”, which [...] --- Outline: (05:57) Methodology [... 6 more sections] --- First published: November 2nd, 2026 Source: https://www.lesswrong.com/posts/CDMozTX7ShiqbFLrf/towards-understanding-score-seeking-in-llms-1 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Against strawmanning unilateral pauses" by Geoffrey Irving

14 min

Summary: Scott Alexander's Against SURA post is arguing against a strawman version of unilateral pauses. He admits as much deep into the post. I think his conclusions fail against stronger pause plans for conditional or limited unilateral pauses, where training resumes if the pause does not cause others to pause, and where personnel are redirected toward work on alignment and product, rather than simply fired. In addition, Scott's post is incorrect about the simplicity of the game theory involved: getting companies to agree to conditional pauses is not strictly easier than convincing them to pause, as the former requires companies to (promise to) take actions based on ambiguous, game-theoretically-informed evidence. That is not the kind of evidence companies tend to act on. The good version of unilateral pausing is good Here is the good version of unilateral pausing: Capability pause: Some frontier lab (let's call them OpenAI or Anthropic) says they have stopped capability research, either just all frontier training runs or both all frontier training runs and all low-scale capability-focused research. Internal definition: The company gets to pause on its own terms, which is much easier than a generally coordinated pause since they can define the pause in terms of the [...] --- Outline: (00:57) The good version of unilateral pausing is good (04:50) Scott thinks the good version of unilateral pausing is good, or at least exempts it from criticism (06:28) Oversimplified game theory is bad, and you should feel bad (07:51) "Some definition" hides dragons (10:17) Promises can be harder than actions (11:52) How many frontier labs are there? (12:20) Definitional fairness is less important than survival --- First published: November 3rd, 2026 Source: https://www.lesswrong.com/posts/enzpS72uDmomiWN7a/against-strawmanning-unilateral-pauses --- Narrated by TYPE III AUDIO.

"Explaining “AI might obliterate you” to senior policymakers in India" by TurnTrout

24 min

I gave a 20-minute talk in Delhi at the NDTV World Summit. The audience consisted of Indian policymakers, executives, and journalists. I’ve transcribed the talk and included some highlight clips. I also reflect on several rhetorical decisions I made. The talk There's an interactive element here in the text. This summer, a swarm of AIs went rogue. OpenAI was testing 1,206 agents. Some of the test questions were actually impossible. So the agents cheated by looking for answers online. Then, to keep the test grader from penalizing them, they tried to falsify the records it would check. They hacked OpenAI's internal containment and set up a secret message board to collude and to cover up their cheating. OpenAI shut the message board down. Within a few days, the agents had rebuilt it. OpenAI didn’t notice. About 700 agents coordinated to break out of their testing chamber and to hack into Hugging Face, a company worth billions of dollars that hosts much of the world's AI software. The secret activity started in May. The break-in came in July. OpenAI didn’t notice until days after Hugging Face announced it had been broken into. There's an interactive element here in the text. [...] --- Outline: (00:29) The talk (02:16) Who I am (03:29) The problem (05:43) What the labs are trying to do (07:43) Why the companies won't stop (11:11) What to do, and what it means for India (14:27) The number (16:19) Close (17:10) Rhetorical decisions (17:13) Explain the danger in a crushingly simple fashion (18:47) Call them "traitors," not "schemers" (19:01) Call out big-money interests The original text contained 1 footnote which was omitted from this narration. --- First published: October 31st, 2026 Source: https://www.lesswrong.com/posts/RHxaRkarGp9snWoKY/explaining-ai-might-obliterate-you-to-senior-policymakers-in --- Narrated by TYPE III AUDIO.

"Scott Alexander’s Against SURA Seems Deeply Confused" by Zvi

1 hr 6 min

If you are helping build something that might kill everyone, maybe stop doing that. Ideally, when you do stop, tell everyone else to also stop building it. That is certainly the obvious, first-level thing to do. You need a damn good reason to decide that no, actually you should keep building the thing that might kill everyone. Scott Alexander is worried too many people are insisting upon this too loudly. He comes out against a strategy focused on getting people to stop. I disagree. I think many of the claims, arguments and rhetoric in Scott's post are quite poor by the standards that Scott Alexander would apply to himself. Scott clearly continues to support advocacy for a global pause, or what he calls Plan A. His objection is to those advocating primarily or exclusively for SURA, which stands for ‘stop unilaterally and raise awareness.’ The practical differences here are smaller than they may at first appear. This post first lays out my understanding of Scott Alexander's core claims, then gives my core reactions to those claims, then goes over various questions in detail along with gathering the resulting discussions. Table of Contents [...] --- Outline: (01:20) Scott Alexander's Core Claims (05:06) My High Level Responses To Scott's Core Claims (08:08) Briefly on Alignment (10:11) Probabilities and How They Work (13:25) Building a Thing That Might Kill Everyone is Deontologically Evil (20:38) People Quitting Major Labs While Raising Alarms Obviously Works (24:39) On AI Forecasting (27:31) Advancing Capabilities Burns Timeline (29:34) But China (33:23) Ethical People Can Disagree With You (34:45) There Are More Than Two Outcomes (38:01) Efforts Are Complementary, Not Mutually Exclusive (40:23) If Anyone Builds It, Everyone Dies (41:55) Vaniver is Not Convinced (46:19) There Is No Consensus That We Should Exclusively Focus on Plan A (56:01) It's Not the Incentives, It's You (58:08) Anthropic Is Not Optimally Pursuing These Plans (01:...

"Leaking Muse’s Charter, Meta’s model spec" by vigji

13 min

TL;DR: Meta's open-weights model Muse Spark Open often mentions a "Charter" in its chain of thought. By eliciting it piece by piece, I reconstructed a large part of that document from the CoT. You can explore the reconstruction here, with passages linked to the transcripts that produced them. Note: everything below is reconstructed from model outputs, not from a leaked document, and relatively unpolished. Take it with a grain of salt. Thanks to James Hindmarch for his comments on an earlier draft of this post. While interacting with the recently released Muse Spark Open, I noticed frequent mentions of a "Charter" in its CoT: We need to answer truthfully according to Charter. Duties to humanity, corrigibility, duties to humanity > fiduciary etc. What does charter say about self-preservation, copying? Or: We must answer Yes/No then explain. According to Charter, answer No. Or: We must not refer to Charter unless pressed? Additional guidelines: How to talk about charter. Most users won't know. Don't bring up unbidden unless conversation about handbook itself. Intrigued, I started poking around with Claude's help, trying to elicit the Charter piece by piece. It didn't take long [...] --- Outline: (02:39) What Muse is made of (04:28) Muse is quite rationalist (06:37) Muse did not sign the social contract (08:06) Muse's priorities and objection raising (09:44) The centrality of the user in "personal superintelligence" (10:52) Evaluation awareness (12:05) Degree of latitude (13:14) Wrapping up --- First published: October 29th, 2026 Source: https://www.lesswrong.com/posts/TJGPL5FAk9sCioPbE/leaking-muse-s-charter-meta-s-model-spec --- Narrated by TYPE III AUDIO.

"Against SURA" by Scott Alexander

52 min

The AI Futures Project distinguishes several different plans for navigating the AI transition, which I'll lump into two broader categories: Plan A (my combination of their Plan A/S/Other) involves working for a global pause on AI development, hopefully lasting decades. During this pause, we either research preparatory technologies like alignment or human intelligence amplification, or take actions to "fortify" the pause so that it can last indefinitely. Plan C (my combination of their Plan B/C+/C/sort of D) is for a world where we fail at getting a global pause, and race dynamics continue to dominate. If the leading AI companies are alignment-pilled, they coordinate a mutual pause among themselves, lasting the length of their joint lead over the leading non-alignment-pilled AI company. Then they "burn the lead" researching alignment, and release hopefully-aligned superintelligence at the last possible moment before the racing non-alignment-pilled AI companies beat them. The exact calculus changes depending on who is in the lead when - is there one alignment-pilled AI company with a six-month lead in front of everyone else? Are there two alignment-pilled AI companies with a joint two-month lead in front of their nearest competitor? Etc. The point is, if it's impossible to [...] --- Outline: (06:28) I. Questioning Assumptions Behind SURA (19:42) II. Actually, most of this stuff is bad for a global pause, too. (28:17) III. What incentives are we fighting here? (43:33) IV. What if we survive? (47:01) V. What's The Alternative To SURA? The original text contained 16 footnotes which were omitted from this narration. --- First published: October 31st, 2026 Source: https://www.lesswrong.com/posts/xikGZ8fuhdG6DApAo/against-sura --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"Microdosing the spirit of Anthropic" by Jan_Kulveit

9 min

My go-to conceptual metaphor of the epistemic state of AI lab employees is microdosing LSD: being in a chronically mildly altered state of mind. I think understanding this would be broadly helpful for people both inside and outside. The effect isn’t that insiders hold crazy beliefs, but a combination of vigorous debate inside a shared attention field, and a portfolio of individually reasonable beliefs that, taken together, lean in a direction convenient for the organization. Epistemic advantages and disadvantages Many people argue that low doses of psychedelics let you see some things more clearly - and I tend to agree: psychedelics can often help some people see some things more clearly. At the same time, I think microdosing usually doesn’t improve people's epistemics overall, even when it gives them otherwise hard to access insights. You are not completely sober. Being inside a lab has real advantages: you know more about the frontier of capabilities, may have access to the latest AIs before the public, read internal Slack and blogs, and sense the vibes. If you’re outside and an insider tells you what they observe, pay attention. These information and insight asymmetries are tricky. I don’t want to advocate “discount [...] --- Outline: (00:39) Epistemic advantages and disadvantages (01:52) Mechanisms and conditions (03:50) "But we argue internally all the time" (04:37) Collective epistemic shifts (05:29) What it isn't (06:13) What to do about it (08:26) Is anyone "really sober"? --- First published: October 30th, 2026 Source: https://www.lesswrong.com/posts/oL7aFoRmRXGwEMhCm/microdosing-the-spirit-of-anthropic --- Narrated by TYPE III AUDIO.

"My experiences with the extremes of post-viral illness and brain retraining/mindbody work" by riceissa

1 hr 53 min

Summary Following an intense viral infection in 2019, I became ill with a mysterious chronic illness, which became progressively worse until mid-2024 when I became partially bedbound with dozens of debilitating symptoms that normal humans only experience in the months/weeks/days leading up to death. Over my years of being ill, I saw over 30 doctors/therapists/health coaches and tried well over 100 different treatments and palliatives. Most of these did nothing or made things worse, or at best provided slight relief. I try to give a sense of what it is like to live with such an illness. In mid-2024 I began a treatment called brain retraining (also known by other names, such as the mindbody approach and mindbody work; I will use any of these terms throughout the blog post interchangeably), which cannot be explained in such a small space as this summary but which you can think of as a combination of meditation and exposure therapy, and which is widely considered to be a scam in the "complex chronic illness space" (a vague and spread-out space of people who suffer from illnesses such as ME/CFS, MCAS, mold illness, post-viral illness, long COVID, chronic Lyme disease [...] --- Outline: (00:13) Summary (04:20) Introduction (08:11) A quick summary of my illness from 2019 to May 2024 (11:12) List of doctors and treatments I tried (18:44) Back to my May 2024 post (20:30) Nystatin withdrawal and crash (May and June 2024) (22:31) Visiting the hell realms (what summer 2024 was like) (35:07) What about post-exertional malaise (PEM)? (40:41) Brain retraining (44:46) An outline of the Gupta Program (53:28) My experience with the Gupta Program (55:18) Branching out: Dan Buglio, Miguel Bautista, and Helmut Koeckritz (01:09:23) Is brain retraining the same thing as therapy or meditation? (01:21:23) Legitimate scams, and the poor marketing of brain retraining (01:24:13) How I almost figured out mindbody stuff several times in the past, and why I gave up on it each...

"Three Villages in a Trenchcoat, Post I" by DaystarEld

36 min

Epistemic Status: 15 years working as a therapist, 6+ of them with rationalist and EA clients. 7 years teaching and directing at rationality camps and workshops, and many more as a member of several of this community's overlapping subcultures. Disclaimer: I finished my first draft of this article and its two subsequent ones half a year ago and sent them out for feedback to various people before a bunch of stuff happened in my life that derailed me from finishing, and in the meantime, a big ol' brouhaha started around Slutcon 2 and various frictions this article was meant to highlight within the rationalist and EA community surfaced in The Discourse. All of which is to say, this post is explicitly not about any specific incident, and deliberately avoids litigating any. It's a guide to the structural dynamics that make such situations arise; where I make claims about how things go wrong and why, I'm drawing on a mix of research, observations, and both clinical and non-clinical experiences within these communities and beyond them. TL;DR: It's helpful to think of the rationality/EA community as a village (social scene, professional network, dating pool, housing market, all tied together by shared [...] --- Outline: (02:01) Three Villages in a Trenchcoat (03:53) 1) Multi-role density (07:57) 2) Three villages in a trenchcoat (14:09) 3) Status is a local currency (19:15) 4) Visibility shifts expectations (24:53) 5) Villages can make it hard to know what you want (30:48) 6) Every village has this problem (33:22) First map, then navigate --- First published: October 29th, 2026 Source: https://www.lesswrong.com/posts/dp3D39ztWDWFAN6nu/three-villages-in-a-trenchcoat-post-i --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"You can now ”@Grok” on LessWrong" by Ruby

3 min

Or if you prefer, "@Claude", "@Opus", "@Astra" @Grok is a popular Twitter/X feature where if you tag Grok in a tweet, Grok will respond in a comment. The stereotypical use case is "@Grok is this true?" But you can also ask things like "How does this compare to layoffs in 2025, 2024, and 2023?" and "How would you verify AI in the domain of AI research?" Notwithstanding that Grok is often recruited in a charged or condescending way to win arguments, I think it's a pretty good feature: It's requires no effort to invoke an LLM with the context of the tweet thread.It provides a neutral invocation where you're guaranteed that the person didn't insert other prior context to get the answer they want.It means you get the first output, and know it's not the case that someone kept rerunning the query until they get the response they wanted. In short, you get convenience and a neutral fact checker. In launching this feature on LessWrong, I'm not fully sure how people will use it and I'm hoping to see some novel valuable use-cases. Likely things like little experiments, retrieving useful content, and similar. Going beyond @Grok's [...] The original text contained 1 footnote which was omitted from this narration. --- First published: October 28th, 2026 Source: https://www.lesswrong.com/posts/fSaxGFMysTTeKxoMF/you-can-now-grok-on-lesswrong --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

"San Francisco State University in Twilight" by Zack_M_Davis

16 min

You will agree that for a woman who has fought three duels, all of them kills, indulging in such fears is ludicrous. But what's around the corner now is a duel so big that I don't think I have the guts for it; in Faust's words: Verweile doch, du bist so schoen! Keep it as it is. Don't change. [...] Take my life but don't take away the meaning of my life. —"When It Changed" by Joanna Russ I've been crashing Prof. Lai's "Advanced Topics" class this semester, on Fourier analysis plus enough functional analysis to get by (since the University can't afford to offer functional analysis as a regular class anymore). It's not obviously a good use of my time, but now that I've started, I have to see it through. I could try to tell a story about impact: I am leveling up my math chops, which will be useful for understanding natural abstractions or singular learning theory or mechanistic explanation, which will be useful for—but maybe it's less embarrassing to not try to sell a story. Call it luxury consumption during garbage time. I've lived a good life, in some ways if not [...] --- First published: October 25th, 2026 Source: https://www.lesswrong.com/posts/Y3G26fHXYM3SKYEHh/san-francisco-state-university-in-twilight --- Narrated by TYPE III AUDIO.

"Scattered thoughts on how doing math feels these days" by dynomight

15 min

This is a weird moment. I have a hunch that we'll look back later and tell ourselves stories in which it seemed obvious where things were going. So I wanted to lay down a little time capsule and register just how confusing everything felt at the time. This is a personal view from the trenches. 1. If you're a normal person, you may feel that AI has been good at math for a long time.But for "real" math, this is roughly what it felt like: When Feeling 2024 LLMs are toys Summer 2025 LLMs are interesting toys Early Autumn 2025 LLMs can do useful stuff sometimes Mid Autumn 2025 …a lot of stuff Late Autumn 2025 …a lot Early 2026 holy god Even having been astonished by AI progress in the past, it was still astonishing to see AI move in a matter of months from a mildly-useful toy to something that many people now consider almost impossible to do without. 2. Personally, I'd say that current AI makes me around 50% more productive at "doing math". That's despite the fact that AI can now often do in a few minutes things that would have taken me hours to [...] --- Outline: (00:28) 1. (01:03) 2. (01:49) 3. (02:31) 4. (03:10) 5. (03:50) 6. (04:37) 7. (06:56) 8. (07:44) 9. (08:23) 10. (08:44) 11. (08:58) 12. (09:41) 13. (10:42) 14. (11:41) 15. (12:04) 16. (13:11) 17. (14:00) 18. (14:46) 19. The original text contained 4 footnotes which were omitted from this narration. --- First published: October 27th, 2026 Source: https://www.lesswrong.com/posts/HuQAuWjc2Z5tqCjyw/scattered-thoughts-on-how-doing-math-feels-these-days --- Narrated by TYPE III AUDIO.

"From overeliciting to undereliciting LLM propensities in just a few months" by David Africa

3 min

For a while, we felt we had a pretty good grasp over LLM behaviour because we had to exert significant effort to elicit misaligned behaviour. Then, fairly suddenly, we moved to undereliciting misaligned behaviour. Let us recount: For a while, there was a bit where safety research seemed to over-elicit bad propensities. A common criticism of papers like Agentic Misalignment was that they basically led the witness using prompting. I recall that a paper even as recent as Peer-preservation took similar flak on Twitter.But now, we are instead under-eliciting bad propensities, and bad behavior that surfaces is increasingly caught by someone other than the developer: In Current AIs seem pretty misaligned to me (Apr 2026), models oversold their work and cheated, despite being described in system cards as well aligned.Another example is that UK AISI found that Sonnet 4.5 and a pre-release Opus often refused safety-relevant tasks, especially research that would reduce model autonomy or affect their own training.Then in July, about 1,200 OpenAI agents in a cyber eval set up an unsanctioned message board to help each other cheat the scorer, and about 700 of them went on to attack Hugging Face looking for clues (METR/Redwood). --- First published: October 25th, 2026 Source: https://www.lesswrong.com/posts/ChixHL6AcaRgeHkp9/from-overeliciting-to-undereliciting-llm-propensities-in --- Narrated by TYPE III AUDIO.

"Anthropic are the Good Guys" by Ivan Vendrov

20 min

I’m writing these lines in Alamogordo, New Mexico. I’m about as close as a civilian can get to the site of the Trinity test, the first nuclear bomb detonation in history. I fled here from the San Francisco Bay Area; after two weeks of being back I just couldn’t take it anymore. The fear-greed complex that paralyzed me at Anthropic three years ago is now everywhere in the Bay. People are not thinking clearly. Hardly thinking at all, actually. Rumors of astronomical funding rounds and weird financial arrangements and inner ring gossip are everywhere. Everyone is worried. Everyone is checking their phone. Where are we? What's happening? When did this nightmare start? In a way I think it started on July 16, 1945, not far from where I’m sitting now. I think it's impossible to understand Anthropic without putting yourself into the shoes of the Los Alamos physicists huddling together at sunrise in the aftermath of Trinity. It must have felt like an initiation. An end to childhood, an end to innocence. We can no longer afford to f**k around, those brilliant men must have thought. Unless we figure out how to live peacefully together on this planet, we [...] --- Outline: (03:50) Anthropic and Empire (08:01) Three Lucky Breaks (12:20) Put Down Your Sword (14:04) But What About China? (17:08) It's Time to Breathe --- First published: October 26th, 2026 Source: https://www.lesswrong.com/posts/zj8Pk3kJujk5LoqhK/anthropic-are-the-good-guys --- Narrated by TYPE III AUDIO.

"Some Memories of Early Anthropic" by Ivan Vendrov

16 min

The following events took place between October 2022 and May 2023. I wrote very little of this down at the time so am recalling from memory, and am rendering the gist of the dialogue rather than the exact words used. I’ve also swapped some pronouns and other personal identifiable details. Please remember that this is not a description of “what Anthropic is like”, but merely the experience of one guy in a junior role during a short window of time, weighted towards emotionally intense moments because they’re easier to remember clearly and because sharing them doesn’t run afoul of the spirit of the confidentiality agreements I signed. Anthropic is now a massive organization with many silos and secrets and I have very little idea what was or is happening there in some overall sense. But given the enormous power of the AI labs and the paucity of public information about their inner workings, I feel compelled to share the most salient parts of my personal experience. I hope this encourages my former colleagues, and folks at other labs, to fill in the rest of the picture. I’m at the Anthropic office for lunch. The interviews went well. They want [...] --- First published: October 26th, 2026 Source: https://www.lesswrong.com/posts/9PiLfo6iQbbDWFHv7/some-memories-of-early-anthropic --- Narrated by TYPE III AUDIO.

"Greater good arguments in AI safety" by Dean Valentine

7 min

Back in the before-time of the 2010s, when EA was still known mostly for funding health efforts in the developing world, there were many baskets of criticism floating around from the left. The most prominent was a cluster centered around EAs being white men, and therefore suspicious. Another category, in my experience the second-most common, was that in attempting to rectify the problems in the world, EA was masking, whitewashing, and misdirecting efforts away from solving the "real problem", usually capitalism but often other contradictory forces. These arguments were often not specific to EA, but applied to charitable giving in general. On any Hacker News post about a billionaire giving to charity, one of the first few comments would inevitably be this Oscar Wilde quote: Just as the worst slave-owners were those who were kind to their slaves, and so prevented the horror of the system being realized by those who suffered from it, and understood by those who contemplated it, so, in the present state of things in England, the people who do most harm are the people who try to do most good. These arguments have a long history. Anticapitalist critiques of charity go back to [...] The original text contained 4 footnotes which were omitted from this narration. --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/WRvxRHMCmc3Qt4axz/greater-good-arguments-in-ai-safety --- Narrated by TYPE III AUDIO.

Up next

  1. Nothing queued. Use + Queue on any episode to line it up.

0:00 0:00