Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
David Silver and Richard Sutton, two renowned AI scientists, argue in a new paper that artificial intelligence is about to enter a new phase, the “Era of Experience.” This is where AI systems rely increasingly less on human-provided data and improve themselves by gathering data from and interacting with the world.
While the paper is conceptual and forward-looking, it has direct implications for enterprises that aim to build with and for future AI agents and systems.
Both Silver and Sutton are seasoned scientists with a track record of making accurate predictions about the future of AI. The validity predictions can be directly seen in today’s most advanced AI systems. In 2019, Sutton, a pioneer in reinforcement learning, wrote the famous essay “The Bitter Lesson,” in which he argues that the greatest long-term progress in AI consistently arises from leveraging large-scale computation with general-purpose search and learning methods, rather than relying primarily on incorporating complex, human-derived domain knowledge.
David Silver, a senior scientist at DeepMind, was a key contributor to AlphaGo, AlphaZero and AlphaStar, all important achievements in deep reinforcement learning. He was also the co-author of a paper in 2021 that claimed that reinforcement learning and a well-designed reward signal would be enough to create very advanced AI systems.
The most advanced large language models (LLMs) leverage those two concepts. The wave of new LLMs that have conquered the AI scene since GPT-3 have primarily relied on scaling compute and data to internalize vast amounts of knowledge. The most recent wave of reasoning models, such as DeepSeek-R1, has demonstrated that reinforcement learning and a simple reward signal are sufficient for learning complex reasoning skills.
What is the era of experience?
The “Era of Experience” builds on the same concepts that Sutton and Silver have been discussing in recent years, and adapts them to recent advances in AI. The authors argue that the “pace of progress driven solely by supervised learning from human data is demonstrably slowing, signalling the need for a new approach.”
And that approach requires a new source of data, which must be generated in a way that continually improves as the agent becomes stronger. “This can be achieved by allowing agents to learn continually from their own experience, i. …