Months ago, I wrote about the measurement problem: engineering metrics rewarding the wrong behavior after AI adoption. That post was a diagnosis based on pattern recognition across organizations, velocity climbing while delivery stayed flat, dashboards turning green while products shipped late.
The data arrived.
Between late 2025 and mid-2026, five independent research programs published findings that converge on the same conclusion. Individual AI-assisted productivity gains are real, measurable, and consistently fail to translate into organizational delivery improvement. The gap between “developers feel faster” and “the organization ships better software” now has numbers attached to it.
This post isn’t a sequel in the sense of repeating the diagnosis. The diagnosis stands. This is about what the external validation reveals about where the bottleneck actually lives, and why the organizations seeing returns are structured differently from those that aren’t.
The funnel that evaporates
The most striking data comes from an MIT and NBER study of over 100,000 GitHub developers, published earlier this year. The researchers tracked what happens to AI-assisted code as it moves through a development pipeline.
At the point of creation, AI-assisted developers generated nearly 300% more files than their pre-AI baseline. By the time that code reached review submission, the gain had halved to approximately 150%. At the final stage, actual software releases, the increase was roughly 30%.
Three hundred percent becomes thirty percent. Not because the code was bad. Because the organizational systems between creation and release, review processes, integration testing, architectural coherence checks, deployment governance, absorbed most of the raw output increase before it reached customers.
This is the AI Productivity Funnel. And it explains why executives report contradictory things. The CTO who says “my developers are 3x faster” is measuring the top of the funnel. The VP of Product who says “we’re still late on the roadmap” is measuring the bottom. Both are right. The organization just doesn’t have a mechanism for converting one into the other.
Five studies, one pattern
The MIT funnel is the most dramatic illustration, but it’s not isolated.
DORA 2024 surveyed over 39,000 technical professionals and found that for every 25% increase in AI adoption, organizations experienced a 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability. Individual developers reported higher productivity, better flow, and greater job satisfaction. The team-level metrics moved in the opposite direction. Google’s own research team called it an “anomaly” in the report.
DX tracked 400+ engineering organizations over 16 months. AI tool usage across those organizations increased by 65%. Median PR throughput rose 7.76%, not the 3x or 10x that vendors promised, under 8% in the real world. Even at the 90th percentile, the top performers, gains reached only 44%. The gap between what vendors promise and what organizations measure is now quantified.
Cortex’s 2026 Engineering Benchmark found pull requests per author up 20% year-over-year, incidents per pull request up 23.5%, and change failure rates up 30%. Speed went up. Reliability went down. Resolution times grew because teams struggled to debug code they hadn’t written and didn’t fully understand.
PwC’s 2026 CEO Survey asked a simpler question: has AI improved your revenue or reduced your costs in the last 12 months? Fifty-six percent of CEOs reported neither. Only 12% reported achieving both. The executive class that approved AI budgets is now staring at the invoices without the returns they expected.
A separate NBER study surveyed 6,000 executives across four countries. Ninety percent reported no meaningful impact on productivity or employment from AI over a three-year window. Average executive AI usage: 1.5 hours per week despite 69% adoption rates. The tools are deployed. The transformation isn’t happening.
Five research programs, five different methodologies, five different populations, the same structural finding: individual-level gains do not automatically propagate to organizational outcomes.
Why the funnel leaks
The tempting explanation is that AI code is low quality. That’s part of it, but insufficient as a full explanation. The Cortex data shows incidents rising, yes, but the DX data also shows that organizations in the 90th percentile are getting meaningful returns. Something distinguishes the 12% from the 56%.
The productivity paradox in technology isn’t new. Robert Solow observed in 1987 that “you can see the computer age everywhere but in the productivity statistics.” The resolution came when economists demonstrated that organizations which reorganized their workflows around computing, rather than simply adding computers to existing workflows, captured the gains.
Tupper, writing about IT productivity in Data Architecture (2011), captured the dynamic in a phrase that lands differently in 2026: “Bad systems, when automated, simply let you make more mistakes, faster. Real productivity gains are only ever realized when certain critical enablers are present: a sense of shared vision, clear communications, stable and understood processes, and a fervent zeal for continued improvement.”
Replace “computer” with “AI coding assistant” and the paradox resolves the same way.
The funnel leaks because organizations added AI to existing workflows without restructuring those workflows around what AI changes. The 300% at creation hits a review process designed for human-speed code production, the review becomes a bottleneck, reviewers skim instead of reading, defects pass through, and incidents rise. The team generates faster and breaks more while the dashboard shows velocity up and the system silently degrades.
Google’s own randomized trial measured a 21% individual speedup from AI coding tools. That’s real. But it’s real in a vacuum.
Fred Brooks argued in The Mythical Man-Month that coding represents only a fraction of total software delivery effort. The bulk of project time goes to understanding requirements, making architectural decisions, validating assumptions, and coordinating with other humans. Decades of empirical work since then has confirmed the ratio: the mechanical act of writing code accounts for roughly 15-20% of what a software project actually requires. The rest is thinking, deciding, and communicating. A 21% speedup on 20% of the work produces a 4% improvement in total project delivery. That is the math that explains why the DX study shows 7.76% and not 300%. AI accelerated the part that was never the bottleneck.
What the successful 12% share
PwC’s data reveals that the companies capturing AI returns are not spending less on AI or using different models. They’re focused on growth rather than cost reduction, and they’ve restructured how work flows through their organizations.
The Accelerate State of DevOps Report 2025 provides a clue about what changed in a year. By 2025, the throughput relationship with AI adoption had flipped positive, suggesting that early adopters had worked through the initial friction. Teams learned where AI helps and where it creates drag. But delivery stability remained negative, meaning the quality problem persists even as the speed problem resolved. The learning curve for generation is shorter than the learning curve for governance.
What distinguishes organizations with positive AI outcomes from those with flat or negative ones? Looking across the data, the pattern is remarkably consistent: they changed how work was specified, how work was reviewed, how teams operated, and how success was measured.
They define intent before generation. When a developer knows exactly what to build, precisely specified with acceptance criteria and architectural constraints, AI-generated code arrives closer to correct on the first pass. The review burden drops because the specification is the review standard. The funnel doesn’t evaporate because less correction is needed between creation and release.
They validate at the organizational boundary, not just the individual one. Individual metrics (PRs per developer, lines generated, stories closed) are subordinated to delivery metrics (deployment frequency, change failure rate, time to restore, cycle time to customer value). This prevents the dynamic I described in the measurement problem: everyone individually productive, collectively ineffective.
They treat AI-generated code as a different review category. When a human writes code, they build understanding as they write. When AI generates code, nobody has that understanding yet. The review burden is structurally different. Organizations that acknowledged this and restructured their review processes around it captured the gains. Those that applied the same process to a fundamentally different artifact got the Cortex numbers: more code, more incidents, longer resolution times.
They measure decisions improved, not tokens consumed. Gartner found that 84% of enterprise AI spend targets individual productivity while only 16% targets business outcomes. The organizations in the 12% flipped that ratio. They asked “did AI help us make a better architectural decision?” not “did AI help this developer write code faster?”
The amplification thesis, validated
In my previous post on the four layers of AI-driven engineering organizations, I argued that AI amplifies what you already have: aligned systems accelerate, misaligned systems accelerate dysfunction. The DORA data validates this precisely. Teams with solid foundations, small batch sizes, robust testing, clear ownership, saw AI as an accelerator. Teams without those foundations saw it as an amplifier of their existing problems.
The productivity paradox is a measurement of organizational readiness. AI didn’t create the gap between generation and delivery, it made the gap visible, measurable, and expensive enough that ignoring it became a board-level problem.
What this means for engineering leaders
If your organization deployed AI coding tools in the last 12-18 months and your delivery metrics haven’t improved proportionally, you are not alone. You are the median. The DX study’s 7.76% at the median represents the outcome when AI is added to existing processes without structural change.
The resolution isn’t better AI. The resolution is the same one that resolved Solow’s paradox thirty years ago: reorganize the work around what the technology actually changes.
That means accepting that the bottleneck moved. It was never code generation, it was always intent definition, architectural judgment, and validation, and AI made that visible by removing the one constraint everyone was focused on. The organizations that restructured around the real bottleneck captured the gains. The ones that kept measuring the old one kept celebrating metrics that don’t correlate with outcomes.
The paradox has numbers now, and the numbers point in one direction: you restructure while the 7.76% gap is still a competitive advantage, or you restructure later, after the organizations that figured this out early have compounded that advantage into years of delivery distance.
Ricardo
