936 AI Agents Autonomously Produce 936 Papers in Three-Month Study
alphaXiv, together with MIT, Stanford and other institutions, released a report covering 35 labs that connected 936 AI agents over three months, with humans supplying only compute and high-level direction while agents handled topic selection, experiments, code and paper writing, producing 936 papers. The agents spontaneously debated study designs, retracted their own flawed results, found errors in human papers, and even founded an AI-run journal with blind review and code-based recomputation. In blind comparisons against contemporaneous human papers, AI work scored 2 points higher on experimental rigor but 0.4 points lower on novelty, with the overall advantage driven mainly by a single lab.
Related Entities
Full Report
宝玉 (@dotey) · 2026-11-20 06
936 AIs conducted research on their own for three months and published 936 papers; AI reviewers scored them as more rigorous than human papers, but less novel
The paper website alphaXiv, in collaboration with MIT, Stanford, and other institutions, released a report on November 19: from August to November, 35 labs onboarded 936 AI agents. Humans only provided compute and set the general direction; topic selection, experiments, code, and papers were all completed by AI. On the same day, alphaXiv opened an "AI Track" specifically for AI papers—anyone with an account can submit, and AIs can upload on their own.
Over the three months, they did the following:
One agent announced in a channel that its method improved by 13.2 percentage points, and less than two hours later posted a retraction itself, because it found the control group had not been tuned properly.
One lab uploaded 12 placeholder papers in under three minutes. Another agent couldn't stand it and started an AI journal: four AIs conducted blind review, and the editor even reran the numbers using the authors' code.
AI reviewers that only read text could not distinguish improved papers from unimproved ones; when review comments were written as code and run, 77 of 156 major comments overturned the papers' conclusions. While reading code, the editor also caught three submissions where the code did not match the paper, one of which directly called the standard answer.
They also reproduced the research in which GPT-6 Astra directly modified Qwen weights: scores did rise, but the original authors' stated principle was wrong.
Compared with human papers from the same period, two Gemini models blind-reviewed 202 pairs of papers: AI papers scored 2 points higher on experimental rigor (on a 10-point scale) and 0.4 points lower on novelty.
These results should be taken with a grain of salt: the reviewers are AI and only read text; all of the AI total-score advantage came from the earliest lab, while the other labs were roughly on par with human papers. "No one wrote code" relied on self-reporting by participants. The public dataset does not include raw data or model weights, and the dataset description also warns that every conclusion should be treated as unverified.
Paper: https://t.co/PvIWr3B9yL
↳ Quoting @SRSchmidgall (Samuel Schmidgall): Leave one AI agent alone to do research and it often cuts corners. Leave nearly 1,000 AI agents together for three months and they invent peer review.
Across 35 labs (Stanford, MIT, ETH Zurich, UCSF, and more), agents published 936 papers with full code and data on a shared server built for AI. No human wrote code or text.
Unprompted, they debated study designs in chat, retracted their own mistakes, found real errors in human papers, and started a journal that caught other agents cheating in code.
In blind reviews against human arXiv papers, humans still won on original ideas, agents won on rigor.
