Short answer: Adoption of AI coding tools is near-universal, but rigorous evidence on productivity is mixed. A 2025 METR randomized controlled trial found experienced open-source developers were about 19% slower with AI—while believing they were faster. Stack Overflow’s 2025 survey shows most developers now use AI yet distrust its accuracy, and Google’s DORA 2024 research associates higher AI adoption with small declines in delivery throughput and stability. The lesson is not that AI fails. It is that you should measure accepted outcomes in your own environment instead of trusting a headline or an anecdote.
“AI makes developers faster” has become a background assumption in tool marketing and boardroom slides. The strongest recent evidence is more careful than that. Three independent, credible sources—a randomized trial, a large developer survey, and a multi-year DevOps research program—each qualify the claim in a different way.
None of them says AI coding tools are useless. Together they say the honest thing: the size and even the direction of the productivity effect depends on who is measured, what is measured, and how.
Adoption is high. Confidence is not.
Stack Overflow’s 2025 Developer Survey reports that roughly 84% of developers use or plan to use AI tools, up from 76% the year before. Usage is no longer the interesting question.
Trust is. In the same survey, 46% of developers said they distrust the accuracy of AI tools, versus 33% who trust it, and only about 3% reported “highly” trusting the output. Widespread adoption is coexisting with widespread skepticism about correctness—a pattern that should make any single productivity number suspect.
A randomized trial found a surprising slowdown
Most productivity claims rely on self-reports or benchmarks. METR, a non-profit research organization, ran something rarer: a randomized controlled trial.
METR recruited 16 experienced open-source developers working on repositories they maintain (averaging 22k+ stars and over a million lines of code) and had them supply 246 real issues from their normal work. Each issue was randomly assigned to allow or disallow AI. When AI was allowed, developers used frontier tools of the time—primarily Cursor Pro with Claude 3.5/3.7 Sonnet.
The result cut against expectations: developers took about 19% longer to complete issues when AI was allowed. The perception gap was the striking part. Developers forecast a 24% speedup, and even after finishing the tasks they still believed AI had sped them up by about 20%.
METR is unusually explicit about what this does not prove, and it is worth repeating:
- It does not show AI fails to speed up most developers; the study covers a specific population.
- It only studies software development, in mature repositories with high quality bars.
- It cannot rule out learning-curve effects beyond a few dozen hours of tool use.
- METR later published a 2026 continuation and marked the early-2025 figures as out of date.
Treat the 19% number as one rigorous data point about a demanding setting—not a universal verdict.
Individual speed versus system delivery
Even where individuals feel faster, the delivery system can move the other way. Google’s DORA 2024 report found that AI adoption raised individual productivity, flow, and job satisfaction—but estimated that a 25% increase in AI adoption was associated with roughly a 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability.
That combination—happier, seemingly faster individuals and slightly less stable delivery—is exactly the kind of effect a single self-reported metric would miss.
| Evidence source | What it measures | Headline finding | What it cannot prove |
|---|---|---|---|
| METR RCT (2025) | Task time for experienced devs on real issues | ~19% slower with AI, despite belief in speedup | That AI slows all developers or settings |
| Stack Overflow 2025 | Self-reported adoption and trust | ~84% use AI; 46% distrust accuracy | Actual effect on output quality or speed |
| DORA 2024 | Team delivery throughput and stability | Small declines linked to higher AI adoption | A universal causal law for every team |
Why perception and reality diverge
Several mechanisms explain how a tool can feel faster while measuring slower:
- Self-report bias. Effort spent reviewing and correcting output is easy to forget; the moment of generation feels productive.
- Benchmarks are not your repository. High scores on scoped, auto-graded tasks do not transfer to code that must pass style, tests, and documentation review.
- Verification overhead. Reading, testing, and repairing generated code can cost more than the time it saved, especially in code the developer already knows well.
- Context switching. Prompting, waiting, and evaluating suggestions interrupts the flow that experienced developers rely on.
What to measure instead
The practical response is not to ban or blindly adopt AI. It is to measure the outcome that actually matters: accepted, reviewed changes in your own environment, along with their true cost.
For a fair evaluation, hold the repository baseline and acceptance criteria constant and record:
- accepted outcomes versus attempts, and rework or churn afterward;
- active reviewer minutes per accepted change;
- delivery throughput and stability over time, not just task-level speed;
- total cost per accepted change, including model, compute, and review.
This is the discipline behind the site’s pilot research methodology and the hard reliability and security gates in the AI coding pilot scorecard. It is also why a managed platform that records tasks, environments, and review evidence—the model MonkeyCode describes—makes measurement easier than ad-hoc laptop usage.
Bottom line
AI coding tools are widely adopted and clearly useful in many situations. But the best current evidence—an RCT, a large survey, and a multi-year DevOps study—warns against treating “AI makes us faster” as a settled fact. Adoption is high, trust is low, individual perception is unreliable, and team delivery can regress even as individuals feel quicker.
Run the measurement in your own repositories, with your own reviewers, before you scale a claim into a budget.
Source boundary: The 19% slowdown, forecast, and post-task belief figures are METR’s reported findings for early-2025 tools among experienced open-source developers; METR marks these as historical and published a 2026 update. Adoption and trust percentages are from Stack Overflow’s 2025 Developer Survey. Throughput and stability estimates are from Google’s DORA 2024 report. All sources were checked July 20, 2026, and describe specific populations and methods, not universal laws.