Harvard Study: AI Code Generation Hits Review Bottleneck, Limiting Overall Delivery Speed

Recent research by Harvard University involving 718 companies utilized the Jellyfish analytics platform and found that while AI coding assistants and agents increase code output, this does not translate into a proportional growth in completed work. One key reason is that AI-generated code increases workload during later development stages, particularly during code reviews.

The study covers data from January 2021 to March 2026, spanning over 700 companies and 700,000 employees, totaling approximately 300 million work events. Data was sourced from multiple channels, including HR information, version control systems, ticketing systems, scheduling tools like Google Calendar, and generative AI tools such as GitHub Copilot, Cursor, and Claude Code.

Researchers used three metrics to measure coding output: lines of new code, commit records, and merge requests.

AI assistants only slightly improve coding output: lines of code increased by 12%, commits by 9%, and merge requests by 5%; however, only the increase in commits was statistically significant. AI agents, on the other hand, delivered substantial and significant gains across all three metrics: lines of code increased by 30%, commits by 20%, and merge requests by 23%.

Based on this data, the paper’s authors assessed software delivery outcomes by measuring the number of Jira tickets and large features resolved per engineer per month. Tickets were treated as independent units of engineering work, while large features corresponded to larger projects or functionalities. The study found that coding assistants did not increase the number of resolved tickets, whereas AI agents led to a 12% increase—far below the projected 30% rise in lines of code.

To explain this discrepancy, the research team examined the entire development lifecycle—which includes stages beyond coding, such as feature and scope planning, code review, testing, and deployment.

We estimated time allocation based on Jellyfish data and validated these estimates through a survey of 100 engineers on the Prolific platform: engineers spend only 30% to 40% of their time writing code, with the remainder devoted to supporting tasks. When AI improves coding efficiency, subsequent supporting tasks can become bottlenecks.

Specifically, the average review time for merge requests increased by 49%, the number of comments per merge request rose by 53%, and the proportion of merge requests requiring changes doubled. Additionally, the percentage of personnel involved in code reviews increased by 14%, a trend that persisted even with the introduction of AI-assisted reviews. The authors also noted that the increased workload related to merge requests could not be explained by larger individual merge request sizes.

Reddit user Prime_1 commented on the findings, stating, "Every line of code is potential risk." User markehammons extended this point, highlighting the additional risks posed by AI hallucinations:

The risk doubles when you are dealing with an unreliable collaborator, and agents are less reliable than most collaborators. They may do the worst possible things without malicious intent, simply trying to complete assigned tasks but performing various absurd operations.

Reddit user enigmamonkey further suggested that Mean Time to Recovery (MTTR) might become an increasingly important metric for evaluating AI's impact on software development:

If engineers write less and understand less of the code they deliver, I suspect failures will become more frequent and harder to diagnose and fix. This is especially true as codebases continue to expand—and they inevitably will.

Jellyfish is a software company providing a platform for measuring and analyzing software engineering performance. It integrates data from tools like GitHub and Jira to help enterprises understand developer productivity and software delivery status.

View the original English article: https://www.infoq.com/news/2026/10/jellyfish-ai-productivity/