Harvard study of 700+ firms finds AI coding agents raise code output but not software delivery, as code review time grows 49%

Harvard researchers Fiona Chen and James Stratton analysed Jellyfish data covering 700+ firms and over 700,000 employees from 2021 to March 2026. After AI coding agents arrived, lines of code rose 30%, commits 20% and pull requests 23%. Jira issue and epic resolution rates showed no statistically significant change. Review time rose 49%, the share of pull requests with changes requested nearly doubled, and comments per pull request rose 35%. The study found no attributable employment changes. We only have this second-hand via Ars Technica, with no link to the paper, and it may not be peer-reviewed.

Engineering leaders get a concrete reason to measure delivery outcomes, such as issues resolved and time to ship, rather than code volume when judging AI coding tools. The findings suggest review capacity may be the constraint, so budgets and process changes may matter more than extra agent licences. The impact is still uncertain. The data comes from firms that use Jellyfish, and the study measures workflow proxies rather than product quality or revenue. Jira resolution is also a crude measure of delivered value.

The headline claim that agents produce more code "but not more software" is stronger than a null result on Jira resolution rates supports, because "no significant change" is not proof of no benefit. The suggestion that deployment costs are not worth it for most companies is speculation, as the study did not measure costs or returns. More review comments and requested changes could reflect better scrutiny as well as worse code.

1. 01Publication of the actual paper or a preprint: check the difference-in-differences design, how agent adoption was dated, and whether it has been peer-reviewed.
2. 02Replication or rebuttal using other datasets, such as GitHub, DORA-style delivery metrics, or vendor data, especially on cycle time and defect rates.
3. 03Whether AI review tools reduce the 49% rise in review time, given agents already wrote 23.3% of review comments and covered 10.8% of pull requests.