Richard Sutton departs from Carmack to establish his own artificial intelligence laboratory.

      One of the most acclaimed figures in AI is embarking on a solo venture. Richard Sutton, who shared the 2024 Turing Award for his role in establishing modern reinforcement learning, announced on Monday that he is departing from John Carmack’s startup Keen Technologies to launch a new company, Oak Lab. He made the announcement simply on X, expressing admiration for Carmack and Keen before mentioning that he and his colleague Khurram Javed had “broken away to start our own startup” in pursuit of “a slightly different approach to understanding intelligence.”

      Sutton's assessment of the current state of the field is straightforward. He stated that existing deep-learning techniques are “weak and inefficient” and require not just adjustments but a complete overhaul with fundamentally new ideas.

      Oak Lab's central thesis revolves around the origins of intelligence. Sutton has always believed that intelligence arises and is sustained through real-time experience rather than being derived from a neatly curated human dataset.

      This distinction is more significant than it may seem. Contemporary models learn from data that has been gathered, cleaned, and filtered by humans, while genuine experience tends to be messier; some aspects are predictable, but much of it amounts to noise.

      In the lab’s initial research publication, Sutton and Javed quantified the issue. They found that the standard optimizer, SGD, fails to differentiate between relevant information and noise, a limitation shared by similar optimizers like Adam. SGD distributes blame for every mistake across all its parameters, allowing it to assimilate the noise without discerning it.

      To address this, they updated an earlier concept of Sutton’s, introducing an algorithm named IDBD and a new neural variant they call NetworkIDBD, which learns to allocate credit selectively. It focuses on rewarding signals that accurately predict outcomes. In their experiments, it identifies genuine patterns where SGD struggles amidst irrelevant data.

      The essence of their work is efficiency. Their methods learn incrementally from a continuous stream of experience without storing or replaying data. Oak Lab claims this requires significantly less computational power and energy compared to current methods.

      This leads to the lab’s ambitious goal: to develop a trillion-parameter agent capable of learning and planning in real-time while consuming just 20 watts. This power consumption is comparable to that of the human brain. While current leading models are trained once in data centers that consume megawatts, Sutton aims for a system that continuously learns on minimal energy.

      Sutton has a history of challenging conventional views. His 2019 essay, “The Bitter Lesson,” is frequently cited within the AI community. His textbook, co-authored with Andrew Barto, has shaped a generation of researchers. However, he questions whether scaling up pretrained language models is the pathway to true intelligence.

      This perspective aligns him with other notable figures. Yann LeCun has similarly advocated for world models over larger chatbots, having stepped away from Meta with a $1 billion bet on this approach. David Silver from AlphaGo is pursuing an alternative strategy as well. They all share the belief that machines should learn through experiences similar to those of a child, rather than from a static snapshot of the internet.

      The timing resonates with the current sentiment in AI. The race is no longer solely focused on producing the largest models; cost and efficiency are now of equal importance, prompting researchers to explore how models reason rather than simply scaling them up.

      Whether Oak Lab will succeed remains uncertain, as it is striving for a goal that has eluded the entire field thus far. Nonetheless, Sutton is investing his distinguished career in this endeavor. He envisions the future of AI as less about a larger brain in a bigger structure and more about a smaller one that is constantly learning.