Jeff Dean Bets on Self-Evolving AI as Fei-Fei Li Pushes Back

Leaving Google After 27 Years, Jeff Dean Bets on AI Self-Evolution with His Startup

After 27 years at Google, Jeff Dean has chosen to start a new company.

This technical veteran, who has contributed to major projects such as MapReduce, Bigtable, TensorFlow, TPU, and Gemini, is now setting his sights on a more challenging direction: enabling AI not only to solve problems, but also to design experiments on its own, verify results, and continuously learn from past research.

At a public event hosted by Stanford HAI, Jeff Dean systematically introduced for the first time the technical approach and R&D progress of his new company, Discovery Loop.

Jeff Dean revealed that the team has attracted researchers from model projects such as Gemini, DeepSeek, and Kimi, as well as members who participated in the Go programming language project. In the first three weeks after the company was founded, they conducted approximately 400 interviews. According to Jeff Dean's remarks in his talk, about 10% of the team members have received the ACM Computing Award, and another approximately 10% hold medals from the International Olympiad in Informatics. They are trying to bring together senior engineers who have long been engaged in large-scale AI research with young research talent.

According to his vision, future scientific research may no longer require large teams to carry out experiments item by item. A small number of domain experts will be able to leverage tens of thousands of AI Agents to simultaneously explore different research directions, allowing the entire scientific research process to operate continuously and automatically. The company has already developed its own Agent Harness, built infrastructure for large-scale Agent collaboration, and is running about 20 automated improvement loops. Jeff Dean set an ambitious goal: to increase the productivity of domain experts by 20 to 50 times.

Jeff Dean has firsthand experience with the complexity of large technical teams. He once managed a team of about 600 people at Google, and later led Google Research and AI teams that at one point reached about 4,400 people.

He recalled in his talk that he had spent a great deal of time attending meetings about quarterly operating expenses. Later, by adjusting his work arrangements, he significantly reduced the number of people who needed to report to him.

At Discovery Loop, he is trying another way of organizing R&D. Traditional large-scale engineering projects usually require breaking complex problems into a large number of sub-problems and assigning them to different research teams. But this creates new problems: different teams need to coordinate resources, experimental results are difficult to integrate, and once a project has started, the original way of decomposing the problem is often difficult to change. Jeff Dean believes that in the future, automated systems can take on a considerable portion of the work. A small number of domain experts will determine important research goals, while the AI system will automatically identify sub-problems worth solving, organize experiments, and continuously adjust direction based on the results.

At present, Discovery Loop is already running about 20 automated improvement loops, mainly for researching how to improve experimental efficiency and how to learn from historical experimental results.

After the talk, a conversation with Stanford professor and World Labs co-founder Fei-Fei Li pushed the discussion further toward one of the hottest and most controversial technical concepts in Silicon Valley today: RSI (Recursive Self-Improvement). Can AI truly achieve self-evolution? Does this evolution have physical limits? When machines begin to solve mathematical problems that have remained unresolved for centuries, what role will scientists play?

The two researchers, both long involved in the development of AI technology, gave some rather calm answers.

The following is a transcript of the conversation between Fei-Fei Li and Jeff Dean, translated and organized by AI Frontline:

Why Leave Google to Start a Company?

Fei-Fei Li: Thank you, Jeff. The talk just now was excellent. I hope the conversation that follows can go a bit deeper into technical details.

I will play two roles at once: on the one hand, I will ask you questions, and on the other hand, I will also participate in the discussion.

First, congratulations on founding Discovery Loop. Actually, the first question I originally wanted to ask was: why found this company? The four of you can basically all be considered Google veterans. Especially you, you joined Google in 1999, right?

Over the past few years, we have seen incredible technological progress. But I am still curious: in the past year or two, was there a decisive moment when you, Quoc, Sanjay, and Oriol realized that you had to do something different, and that this thing could not continue to be done inside Google?

Jeff Dean: I think we have always been excited about automated systems, especially technologies that can help people develop software systems in a more automated way.

For example, about ten years ago, Quoc had already done a lot of work on neural architecture search. You can understand it as an early form of recursive self-improvement system. You tell the system: "Please help me find a good model architecture that can solve computer vision problems or language problems." We knew at the time that this approach worked. But in that era, you had to define the search space through a bunch of rather clumsy Python code.

Today, you can use natural language directly. You can tell a sufficiently powerful model and Agent system: "Please explore some new model architectures and see which architectures can help us interpret MRI images." Then the system can automatically carry out the relevant work.

I think this is actually a very profound shift in our understanding of the capabilities of computer systems and AI systems. If such systems can be made to run stably and reliably, many new things can be achieved. Especially in science and engineering, we believe this is a set of research directions very well suited to using AI to create positive social impact.

Fei-Fei Li: So what really made up your minds was this loop, and the ability for self-improvement?

Jeff Dean: You could also say that the few of us are a bit too obsessed with these loops. However, there is another very important point. We all enjoy the process of starting from a very small project. When only five people are working on something together, and then a sixth person joins, then a seventh, and then a tenth, that feeling is wonderful.

This could certainly be done inside a big company, but we also wanted to pursue it through the structure of a public-benefit corporation. We believe this reflects our values. Of course, starting a company means you also have to deal with office space and all sorts of other things, but we found those aspects quite interesting too.

Fei-Fei Li: That's fascinating. Actually, when we founded World Labs, we had a similar decisive moment. It's just that the technical direction we focused on was different from yours. At the time, we saw computer vision and generative models beginning to converge. We could finally build generative models that were no longer confined to text. That was the moment we decided to found World Labs.

Jeff Dean: The generative models you've built, and the capabilities that are now achievable, are truly remarkable.

"Silicon Valley Is Misusing the Term RSI"

Fei-Fei Li: Alright, let's move on to talk about self-improvement and recursive self-improvement. To go a bit deeper technically—coming from our backgrounds in machine learning education and research, we're actually from the same generation. I was just thinking earlier about how many different names this concept has gone through since I was a graduate student. We used to call it incremental learning, then continual learning, meta-learning, lifelong learning, and the neural architecture search you mentioned. Now we have self-improvement and RSI. So, could you give the audience here—especially those with a technical background—a more rigorous definition? What exactly is the difference between self-improvement and recursive self-improvement? What I really want to do is separate fantasy from fact.

Jeff Dean: I think you can understand it through the loops I just described. These loops can be applied to many different types of problems. What I understand as a self-improvement system is one where you set a goal for the system and have it improve some measurable metric. For example, you want to improve the accuracy of a machine learning model. The system then tries different model architectures and works to improve accuracy.

Fei-Fei Li: Let me interrupt for a moment. A lot of machine learning algorithms can already do this, right?

Jeff Dean: Yes.

Fei-Fei Li: But that in itself isn't recursive.

Jeff Dean: Exactly right. Here's an example. We give the system a batch of MRI image data along with the corresponding results, and then tell it: "I want to develop a model that can help interpret MRI images." This can be a self-improvement system. It can automatically develop models, try different approaches, and as experiments proceed, model accuracy may gradually improve. However, this does not improve the underlying system responsible for proposing the next round of experiments. Recursive self-improvement is different. The key to recursive self-improvement is how to make this self-improvement method feed back into improving the very system that proposes the improvements. I'm not sure if that's clear.

Fei-Fei Li: I understand. But I think for many people, this is indeed a very easily confused concept. Especially today, in Silicon Valley, people are using the term RSI very loosely. As a scientist, this actually bothers me. So I hope to use this public occasion today to clarify the definition of RSI.

Infinite AI Self-Evolution Does Not Align with Physical Reality

Fei-Fei Li: Next, let's talk about the physical limits of RSI. Right now, RSI is hyped as if it's a system with no endpoint. As if we can keep accelerating it, accelerating all the way to infinity. Among these claims, which are hype and which are fantasy?

Jeff Dean: I think you can start by thinking about how humans have improved model capabilities over the past 10 to 15 years. We bring together excellent researchers and machine learning engineers and run a large number of small-scale experiments. They discover that some ideas don't work and some ideas work well. Then we incorporate the methods that work into the next generation of models. That's why, after Gemini 1, we were able to develop Gemini 2.

Because we ran a massive number of experiments and continuously improved various techniques. This is the normal process of research and engineering. When you start to automate this process, it can indeed increase the speed of improvement. For example, after a researcher kicks off an experiment, it might finish at 2 a.m. But the researcher won't come into the office to check the results until 8 a.m. With an automated system, it can check the results immediately and decide what to do next in a more automated way. And from the perspective of an entire large project, an automated system might be able to coordinate decisions better than each researcher individually deciding what experiment to run next. But I think if you consider how to truly improve foundation models, you'll find that this involves a tremendous amount of actual training and experimentation. These things cannot happen infinitely fast.

Because there are real physical processes involved. You have to perform computation and look at results. Computers need time to complete operations. You need to save data, analyze data, and then have another model look at the results and decide what to do next. So some people fantasize that we can make this loop run at infinite speed. I think that idea is not credible. Automated systems will certainly increase the speed at which we improve systems, but that doesn't mean the rate of improvement can grow without bound.

Fei-Fei Li: This is a very important point, and we really should bring the discussion back to reality. When Jeff and I met at a café earlier, we discussed the hype and the facts around RSI. However, the next question I haven't discussed with you in advance. You can agree, or you can completely disagree.

I've been thinking about whether humans are a self-improvement system or an RSI system. I think we are definitely a self-improvement system. For example, I'm really bad at cooking. Suppose I try to fry an egg—the first time, I might burn everything. But I learn from it. The second time, I might do slightly better. After ten attempts, maybe I can finally make a passable fried egg. I'd be willing to eat it myself, but my kids probably still wouldn't.

That's self-improvement. But RSI involves my brain. In the process of learning to fry an egg, my brain has to undergo some kind of change. For example, the connections between neurons change, making my brain a system that is better at frying eggs. If we're talking about hardware-level improvements, maybe I could even grow a sixth finger.

Jeff Dean: Then you might be AI-generated.

Fei-Fei Li: And then, as I keep frying eggs, my brain would improve further. If such a loop keeps happening, then are humans an RSI system? Is this analogy correct?

Jeff Dean: I think humans are both. In life, we learn many things. For example, learning how to fry an egg, or how to solve calculus problems. These are self-improvement. But at the same time, we also learn how to learn. And learning how to learn is precisely what gives us the capacity for self-improvement and recursive self-improvement. Because once you understand what kind of knowledge and what kind of information presentation is easiest for you to absorb, you can use that understanding to learn other things faster.

Fei-Fei Li: Yes. In a sense, recursive self-improvement could also be a kind of evolutionary algorithm. However, I don't want to go too deep into this topic. It's just that I think at a moment like today, when the concept of RSI is so heavily hyped, being able to discuss this issue openly is a rare opportunity.

Jeff Dean: Indeed, there's a lot of hype right now. For example, some people say systems will improve themselves at infinitely fast speeds, which sounds

terrifying, but if you look at it as an actually running engineering system, you'll find it's not that terrifying, and also easier to understand.

What Exactly Does the Discovery Loop Want to Research?

Fei-Fei Li: You just introduced a lot about the Discovery Loop. But regarding exactly which scientific questions you want to research, you were still a bit vague. So I'd like to follow up. There are so many important problems in science that can benefit society. Which one are you most interested in right now?

Jeff Dean: For a long time, I've been very interested in AI applications in healthcare. I think we will definitely conduct research in this direction in the future—not now, but at some later stage. Because this is a very important field, and it's highly consistent with our mission as a public benefit company.

However, at present we are mainly focused on scientific and engineering problems where the entire experimental loop can be completed in a digital environment. Because we believe that in these fields, the entire experimental loop is easiest to accelerate. If actual physical processes are involved, then accelerating the entire loop becomes much more difficult. Of course, there are many important disciplines that must rely on physical processes. We have also considered some hybrid approaches that could help accelerate research in these fields. But for now, we are first focusing on digital science and engineering fields.

Specifically, our initial focus is on three areas: machine learning research, machine learning engineering, and hardware design. We think it's more appropriate to focus on a few different areas rather than trying to solve all problems at once—these are also areas where we ourselves have expertise.

Moreover, we believe these areas are very important because they can further unlock the potential of other fields.

For example, in healthcare, if you have a system that can automatically solve machine learning engineering tasks, you can apply it to many problems in healthcare, materials science, or other fields. You can tell it: "I want you to develop a very good specialized model for this particular task." Then the system can carry out the relevant work. Of course, by our earlier definition, this counts as self-improvement, not necessarily recursive self-improvement—that's how I would describe the company's current positioning.

We are currently focused on these three areas, but our future goals go far beyond that.

Fei-Fei Li: I received training in physics as an undergraduate. In physics, we usually divide researchers into experimental physicists and theoretical physicists. Just now I was thinking about the loop you showed—you used the phrase "propose experiments," but in physics, we usually say "propose hypotheses," then conduct experiments and evaluations.

So what I want to ask is: as AI Agents or self-improvement loops become more and more powerful, whether or not it's recursive self-improvement, does this mean that philosophically we are leaning more toward an experimentalist approach to scientific research?

Because your approach strongly emphasizes proposing hypotheses, running parallel experiments, and then evaluating them, rather than a relatively mysterious way of theoretical discovery. Even today, we still think about great theoretical physicists like Newton and Einstein, who themselves did not conduct that many experiments. They read experimental results, but they did not complete all their discoveries through this kind of experimental loop. So, do you think AI is making scientific discovery more experimentalist?

Jeff Dean: I don't think the two are necessarily so opposed, though this approach may indeed lean slightly more toward experimentation. But I think such systems can also conduct excellent theoretical research. For example, you can have the system try to propose a theory that explains all existing data. It can iterate continuously, searching for a theoretical description that explains the experimental results while being as concise as possible.

I think a lot of theoretical physicists' research works this way too.

They observe what is happening in the experimental physics community, and then try to propose theories to explain phenomena that have been observed but not yet understood. So I think this kind of research is entirely possible for AI systems to carry out. In fact, I think theoretical research and experimental research should not be viewed as completely separate disciplines.

Whether it's theoretical physics, experimental physics, or chemistry, they can all be considered within a broader knowledge system. Imagine if in the future there is a system that has conducted 100 million experiments across 100 scientific fields and solved 1 million problems.

Then it would likely have accumulated a large amount of valuable insight.

Because it has observed various types of experimental results and knows which methods work and which don't across many similar problems. When you ask it to solve a new problem, it can leverage this cross-domain experience. I think this capability will not be strictly constrained by disciplinary boundaries. Humans, by contrast, are sometimes influenced by traditional boundaries such as university departmental divisions.

Fei-Fei Li on AI's Mathematical Breakthroughs: Mathematicians See Not Just Answers, but Beauty

Fei-Fei Li: Since that's the case, we must talk about mathematics. In fact, while preparing for this conversation, I also wanted to invite Terence Tao, that is, Tao Zhexuan. I called him, but he couldn't come. In recent weeks, AI has made a lot of progress in mathematical research. From Navier–Stokes-related problems to other mathematical papers, and to the results OpenAI just announced yesterday, AI is solving more and more mathematical problems. Frankly, I have somewhat mixed feelings about this—of course, not because I dislike these new discoveries.

Rather, it's because mathematics, and those mathematical problems that have existed for hundreds or even thousands of years, have always been one of the peaks of human intelligence. For mathematicians, these problems are not only technically meaningful. They also see beauty in them, and now we have a completely different kind of machine system solving these problems in another way.

I've also discussed this a lot with Terence Tao. He recently gave a talk on "Math 2.0," introducing how AI can help solve certain types of mathematical problems. But he also insists that there are still some mathematical problems beyond AI's capabilities. Especially when we don't have ready-made solutions or relevant data. What do you think about this huge change in mathematics over the past few weeks?

Jeff Dean: Obviously, current models, as well as Agent systems that can use these models to work in large-scale parallel fashion, already have the ability to solve certain very difficult, previously unsolved mathematical problems. This gives us a new understanding of what today's AI can actually do. But I think the reason people feel somewhat dissatisfied, or even uneasy, about this is that these proofs can sometimes contain 500 pages of Lean formal proof code.

These proofs have been machine-checked. But no one truly understands whether there is a concise, elegant way to arrive at the same result. Of course, knowing that a proposition has been proven or disproven is a good thing. But understanding how to reach that result through genuinely elegant techniques is equally important. In the process of working to solve problems, human mathematicians often develop new mathematical methods, and these methods can in turn be used to solve other types of problems.

Fei-Fei Li: So, are you saying that new mathematical results can only arise from elegant solutions?

Jeff Dean: No. What I mean is that elegant solutions often bring about a great new technique, and that technique can then potentially be applied to other problems. In some very lengthy machine-generated proofs, people may not be able to see this kind of clear, elegant technique. But knowing that a problem is solvable is itself very valuable. This is true in many different fields.

If you know that something is possible, you will explore harder, because you know that some solution does exist.

Then, perhaps you can achieve it through a different approach, or even a more elegant one. So I understand why so many mathematical problems being solved almost simultaneously would make some people uneasy, but I think this capability has now genuinely emerged. Therefore, what people need to think about is how to use this capability to further advance mathematical research.

A Paper Rejected by NeurIPS, Now Cited About 40,000 Times

Fei-Fei Li: Do you remember what is special about the date June 12, 2017?

Jeff Dean: June 12, 2017? Is that the date the Transformer paper was uploaded to arXiv?

Fei-Fei Li: Excellent! Full marks! I even looked up the date specially. Did you read the Transformer paper on June 12?

Jeff Dean: No, it was probably June 10.

Fei-Fei Li: Right, you probably had to approve the paper in advance at the time. I want to use this date to briefly review your career. In the slides you showed just now, you can see the enormous contributions the four of you have made at Google and across the entire AI field. Let's take that day as a historically significant starting point. In a sense, it marked the beginning of the generative model era. From then until now, it has been nearly nine years. Looking back on this journey, what do you think were the most important technical moments? In addition, I want to ask a follow-up question. I hope you can answer very candidly: over these past years, what did you predict correctly, and what did you predict incorrectly?

Jeff Dean: Let me think. Among the many technological advances we have made, a few things really stand out. First, obviously, is Transformer. For these problems, Transformer is a much more efficient model architecture. One of its important ideas is to retain the previously processed state and then access that information through the attention mechanism. This is instead of the old approach of updating a smaller vector at each state while forgetting previous information. This is a very important insight.

And precisely because of this, a great deal of subsequent progress followed. Compared with earlier LSTM-based models, Transformer can learn more with fewer computational resources. Another important advance is model distillation. Oriol, Geoffrey Hinton, and I were involved in a distillation study. That paper was rejected by NeurIPS in 2016. The rejection review at the time argued that the work was unlikely to have much impact. We really should build a website dedicated to collecting rejected papers. This one is a great example.

Later, we put it on arXiv. Now, this paper has probably been cited around 40,000 times. I think model distillation is very important for developing models that are powerful while also highly efficient at inference. Because you can first train a very large model that is cumbersome and expensive to run. Then, let the large model serve as a teacher to instruct a more flexible, lightweight student model. Saying this at a university is quite fitting.

Jeff Dean: The MoE architecture has also proven very valuable. But there is one thing I did not judge accurately before. I originally thought that MoE would be the beginning of a large wave of even more unusual and diverse model architectures. In the MoE architecture, the model assigns tasks to multiple experts and then aggregates the results. Then it assigns them to different experts again and aggregates again. The model gradually learns which experts are suitable for which tasks. This approach is very useful. It allows the model to have very large capacity, while for a particular token or task only a very small subset of parameters needs to be activated.

However, this architecture is still very regular. For example, most experts are the same size, and their outputs quickly converge again. I originally thought we would start to see a large number of even more unusual and more organic neural network architectures. For example, structures resembling cortical columns, or some low-frequency but large-scale connections spanning distant regions of a neural network, connections that could link parts of the network that are far apart. But so far, we have not really reached that point.

Fei-Fei Li: When we were working on neural architecture search back then, we also considered similar directions, but those attempts also did not really achieve the expected results, right?

Jeff Dean: Yes. In fact, models obtained through neural architecture search do look more organic than human-designed models. Human-designed models are usually stacked layer by layer, with occasional skip connections. In contrast, networks produced by neural architecture search may have some very strange connections. For example, a certain layer connects directly to the 17th layer above it and then performs some special operation.

I Did Not Anticipate That AI Coding Ability Would Be So Important

Fei-Fei Li: So, this is one thing you think you predicted incorrectly?

Jeff Dean: There are many things I predicted incorrectly, but this one is indeed relatively prominent, because I originally thought that model architecture development would move more in that direction. There is also another thing: it has turned out that coding ability is very important for the development of AI. But I did not really foresee that programming would become the key capability that enables models to solve a large number of different types of problems.

Fei-Fei Li: I agree with this point. We all actually knew that coding ability was important, but we did not anticipate that it would have such far-reaching impact.

Jeff Dean: Yes. For example, in the Gemini project, coding was indeed one direction we focused on, but it was not the only focus at the time, nor was it the most central focus.

Fei-Fei Li: We've talked a lot about AI's technological progress, but AI's impact on society goes far beyond the technology itself. That includes employment issues, people's psychological anxiety, and the somewhat extreme discussions now emerging around existential risk and AI alignment. First, I want to ask you: among these issues, which one worries you the most?

Jeff Dean: I think AI is an extremely powerful and highly general-purpose technology that is affecting almost every aspect of society—whether it's math, programming, image generation, or helping self-driving cars navigate, AI can play a role. It's quite different from past technological revolutions. Previous technological revolutions usually had a relatively concentrated scope of impact, perhaps profoundly changing five to ten domains. But this time, AI is affecting a large number of different domains around the world simultaneously.

I think this is both exciting and worrying—even frightening. All kinds of emotions are present. In fact, I once co-authored a paper with a group of very talented researchers titled "Shaping AI's Impact on Billions of People." It's also the only paper I've written that has its own standalone website, because one of our very creative collaborators built a website specifically for it: shapingai.com.

In that paper, we examined seven different domains, including employment and the economy, AI's application in scientific research, misinformation and disinformation, media, and several others. We spoke with a large number of experts in these fields. It was a very valuable experience because we were able to understand, from their perspectives, how AI is already affecting their industries and what changes it might bring in the future. But if you ask me what I'm most worried about, I think what truly concerns me is: how can we maximize the benefits of AI across all these domains while minimizing the negative impacts it may cause?

Because AI can indeed have negative impacts in many domains, and I think this is not purely a technical problem. It's both a technical problem and a problem that regulators and policymakers need to address, and also a problem that every citizen needs to face. All of us must think together about how to harness this new technology to ensure it truly serves everyone, rather than harming humanity in ways we can foresee. For example, we need to ensure that when people's jobs are affected by AI, they can find effective ways to adapt to new societal needs. They should still be able to obtain meaningful work, have stable incomes, and live prosperous lives. I think this is a problem that involves many different dimensions and requires joint efforts from people across different fields around the world.

Fei-Fei Li: One last question. Jeff, I know you care deeply about the younger generation. Frankly, this generation of college students is feeling anxious—even Stanford students are uneasy about job opportunities and future economic prospects. You've had such a successful career, you're one of the pioneers in AI, and you often understand future technological developments more deeply than the general public. For the students here today—young people who are both anxious and perhaps hopeful about the future—what parting words do you have?

Jeff Dean: I completely understand that anxiety, because the world is changing rapidly. When you're trying to plan your future, if the future itself is also changing rapidly, then making plans does become very difficult. So my usual advice to young people is:

Try to learn about which new technologies are emerging, or are about to emerge in the near future. Then, make sure you know how to use these technologies to do the things that truly fill you with passion.

That might be becoming a doctor, or pursuing literary creation, or writing software more efficiently. I think all of these jobs can be helped and empowered by AI. But there's still a very important human factor, which is that we humans need to decide what we actually want to accomplish together. And for you personally, you should find what you truly want to do and strive to be the best at it. That also means you need to learn to use all the tools available to you now.

Fei-Fei Li: So even in the AI era, human passion remains eternal. Thank you, Jeff.

Jeff Dean: Thank you.

Original video link: https://www.youtube.com/watch?v=ooGKNwJWdb8