Richard Sutton – Father of RL thinks LLMs are a dead end - Dwarkesh Podcast

Richard Sutton is the father of reinforcement learning, winner of the 2024 Turing Award, and author of <a target="_blank" href="http://www.incompleteideas.net/IncIdeas/BitterLesson.html">The Bitter Lesson.</a> And he thinks LLMs are a dead end.After interviewing him, my steel man of Richard’s position is this: LLMs aren’t capable of learning on-the-job, so no matter how much we scale, we’ll need some new architecture to enable continual learning.And once we have it, we won’t need a special training phase — the agent will just learn on-the-fly, like all humans, and indeed, like all animals.This new paradigm will render our current approach with LLMs obsolete.In our interview, I did my best to represent the view that LLMs might function as the foundation on which experiential learning can happen… Some sparks flew.A big thanks to the <a target="_blank" href="https://www.amii.ca/">Alberta Machine Intelligence Institute</a> for inviting me up t...