Skip to content
OTFotf
All posts

AI Coding Agents Boost Merged Pull Requests by 24% at Microsoft

D
DaveAuthor
6 min read
AI Coding Agents Boost Merged Pull Requests by 24% at Microsoft

A 24 percent jump in merged PRs, held for four months, is the number to plan around. Microsoft just published the first production-scale measurement of command-line AI coding agents, and the result is bigger and more specific than the industry was expecting. Engineers actively using the agents merged 24 percent more pull requests, and the lift held steady across four months without decaying. The shape is what matters: it isn't a novelty spike, it isn't a single team's setup, and it isn't an isolated group. It's the curve you'd expect from a tool that actually fits the workflow.

The headline is good. The shape behind it is the news.

What the Microsoft study actually found

Researchers at Microsoft tracked engineers using CLI-based AI coding agents — agents invoked from a terminal, with the engineer steering the loop. The 24 percent number is the average. The spread is where the real signal lives: the main result's range ran from 14.5 percent to 33.7 percent. Heavy users — engineers hitting five or more days a week — saw gains above 50 percent. Casual users at three days a week landed around 15 percent. Engineers who had the tool installed but didn't lean on it got almost nothing.

The four-month window is the second important detail. Productivity tools tend to spike when they're new and decay as the novelty wears off. The Microsoft data shows the opposite. The lift held. Heavy users kept getting better outcomes, which suggests the gains come from the loop forming, not the model being shiny.

A third signal lives in Microsoft's own internal numbers. Their AI code review assistant now touches more than 90 percent of pull requests — over 600,000 PRs every month. Completion times improved 10 to 20 percent across thousands of repositories. The tool runs routine checks, offers inline suggestions, writes the change summary, and answers questions about the diff. It doesn't replace human reviewers. It catches what humans miss and frees humans to focus on the parts that need a human's judgment.

The gap nobody's putting on the slide: 180 percent more code, 30 percent more shipped

A separate analysis covered more than 100,000 developers and landed on a much less flattering ratio. AI agents drove code volume up 180 percent. Actual production shipments rose 30 percent. The six-to-one gap is the most important number in this entire story, and it's the number most coverage is skipping.

A merged pull request is a checkpoint. A shipped feature is a system that holds up under real load, in production, with real users, on a Friday afternoon, when the on-call rotation is whoever is left. The two are not the same. The first scales with the agent. The second scales with everything that surrounds it — integration tests, staging parity, observability, rollback, the human hours of being sure.

Sarah Guo named the tension in a way that stuck: "The model ate software engineering." But she added the part most posts leave off — engineering has always resisted easy measurement. The parts we can count (lines written, PRs opened, builds green) may not capture what counts. Noam Brown took it a step further: the only reliable way to judge an agent over a year might be to run it for a year.

The takeaway isn't that AI coding is overhyped. The takeaway is that the bottleneck has moved. "Can we write it?" used to be the slow part. It isn't anymore. "Can we trust it under load?" is the slow part, and the cost of trust is paid in CI minutes, in staging environments, and in the time between "the PR is merged" and "the change is live for users." The 180-to-30 ratio is the cost of trust, measured across 100,000 developers.

180% code volume growth vs 30% production shipment growth

11 production screens. Login, database, payments — all wired.

The SaaS Dashboard Kit ships everything already connected. Nothing to set up. Live demo at saas.otf-kit.dev.

See the live demo

How to actually use this today

The Microsoft curve tells you what to do. The Forbes gap tells you where the friction will be. The two together give you a workable plan.

Pick one agent and learn it. Cursor, Claude Code, the agent already baked into your editor — pick one, get fluent, learn where it lies. Don't run two in parallel until you know what the first one is bad at. The 50-plus percent users in the Microsoft study were users, not collectors.

Use the agent for the half that's already slow. Boilerplate. Test scaffolding. The third copy of the same form. Migration scripts nobody wants to write. These are tasks that already felt like typing. The agent's marginal value is highest here, and the cost of a wrong draft is low.

Run the loop in the terminal. The agents that moved the 24 percent number were CLI agents, not IDE plugins. That isn't an accident. The terminal loop forces the engineer to read the diff, edit, commit, push. The agent contributes the typing. The human contributes the judgment. The loop is the asset.

# Initialize the agent in your project
$ claude code init

# Ask the agent to scaffold a new feature
$ claude "Add a /search route. Use the existing User model."

# Review the diff, edit, commit
$ git diff
$ git add -p
$ git commit -m "Add /search route"

# Push and open the PR
$ gh pr create

Track PR throughput before and after. Count merged PRs per engineer per week for a month, roll the agent out, count again at month three. Expect the 14.5 to 33.7 percent band. If you're not in it, the loop is broken somewhere — usually at the review step.

Set a rule: no AI-generated code merges without a human name on the review. The Microsoft data shows the gains come from active use, not autonomous use. The human-in-the-loop isn't overhead. It's the productivity mechanism.

What this enables

A 24 percent lift in PR throughput, sustained, is a real budget. It's roughly an extra day a week per engineer. For a team of ten, that's two extra engineers' worth of throughput without the headcount. The curve doesn't decay, so the budget doesn't shrink. Heavy users get more, casual users get less, non-users get nothing — meaning the cost of rolling the tool out is paid in training and habits, not licenses.

The 180-to-30 gap is the constraint, and it's the constraint the next two years will be defined by. The teams that ship the most software aren't the ones that generate the most code. They're the ones that have built the smallest, cheapest, fastest path from "merged PR" to "live in production" — the integration tests, the staging parity, the observability, the rollback. The agent accelerates the first half of the pipeline. The shared infrastructure accelerates the second half. Both are needed.

A template and component system that ships the same UI on web, iOS, and Android from one source — one API, one set of behaviors, one review surface — is the kind of infrastructure that lowers the cost of the second half. The agent writes more code; the reviewer verifies against a smaller, stable target. The 30 percent that has to ship is the part that benefits most from a substrate the agent and the reviewer both already understand. Use the agent for the 180 percent. Build the shared foundation for the 30 percent that has to hold up in production.

ai-toolsbackendagents
OTF SaaS Dashboard Kit

Ship the product, not the setup.

  • 11 production screens — auth, billing, team, analytics, settings
  • Real database, payments, and login — all wired on day 1
  • AI configs pre-tuned so your agent extends instead of regenerates