Datadog's AI-Powered Test-Driven Migration Success

The 45-minute operation that nobody wanted to touch

Datadog's Stream Router was bleeding milliseconds. As the routing table grew, the most demanding operations ballooned to 45 minutes — executed across thousands of sequential round trips to a key-value store that was never designed to behave like a relational database. That's the kind of perf cliff where most teams start scheduling a rewrite, freeze new features, and quietly look for other jobs.

What Datadog actually did is more interesting than a rewrite. They used Claude and Cursor to accelerate a test-driven refactor of the existing codebase, kept production running on the old model while the new model landed in parallel, and shipped without a feature freeze. Engineer Arnold Wakim walked through the playbook in a recent InfoQ piece, and the three pieces — schema redesign, AI-assisted refactoring, parallel deployment — compose into something every team with a legacy service should study.

This is what AI in the loop looks like when it's used well. Not code generation. Tightly scoped transformation, with a deterministic oracle (the tests) telling the model whether each change held the line.

The KV model was a relational model wearing the wrong clothes

The original Stream Router was eventually consistent on a key-value store — FoundationDB, in their case. Nothing wrong with that choice at the time. But the routing data was shaped relationally: routes had targets, targets had attributes, attributes had values. None of that was expressed in the schema. Instead, the application code was the database.

So every operation looked like:

1. Pull tens of thousands of entries into a pod process.
2. Walk the in-process tree to reconstruct the relationship.
3. Hope nothing changed under you between read and write.

That's not a database. That's a cache masquerading as one, with concurrency bugs you don't even know you have. When the routing table grew, the in-process reconstruction got slow in the way only in-process reconstruction can: linearly, then quadratically, then 45-minute-ly.

The first fix wasn't an AI fix. It was a schema fix. Datadog redesigned the model so that relationships between domain entities — routes, targets, attributes — were expressed as foreign keys in the new relational backend (PostgreSQL). Logic that used to live in the application moved into the schema where the database could enforce it.

That schema work is the enable. Everything after it is just typing.

AI didn't generate the migration — it accelerated it

Once the new schema existed, the team still had to rewrite every method that touched the old model. That's a lot of mechanical, high-risk work: the kind where one missed call site ships a quiet corruption at 3am.

Datadog's pattern for handing this to AI is worth copying. For each method, the team provided the old implementation, the new schema, and a failing test. The models would generate a first pass, and the tests told them whether it was correct. Three inputs, one oracle. No "ask the AI to refactor this whole service." No "generate the new module." Each AI call is bounded by:

• The old implementation — a concrete reference.
• The new schema — a concrete target.
• A failing test — a concrete pass/fail.

The output is a first-pass candidate. The test is the source of truth. If it passes, you keep it. If it doesn't, you feed the failure back and try again.

This is the right shape for AI-assisted refactoring. It's also the shape that breaks the moment any of the three inputs is missing. "Refactor this to use the new API" without a test is a vibes-based refactor and you'll discover its bugs in production.

The three things that made the migration safe

The migration was only possible because three preconditions were already in place. Without them, the AI work would have been a coin flip.

1. Strong code modularity. The new Stream Router implemented the same API as the old one — it just did it on PostgreSQL instead of FoundationDB. Callers didn't need to change. If your migration requires rewriting every consumer, you're not migrating, you're rewriting the company. Build the seam first.

2. A comprehensive test suite. The tests weren't optional — they were the AI's job description. Every generated change had a deterministic verdict before any human reviewed it. This is the single highest-use investment any team can make before bringing AI into a refactor. The test suite is the tooling.

3. Parallel infrastructure. Two independent instances of Stream Router ran side by side, handling the same requests while clients were routed between them via feature flags. A dedicated validator service runs in every cluster, periodically comparing the two implementations' responses.

That third piece is the one most teams skip. Parallel run with a side-by-side comparator is how you find the cases your tests didn't cover. It's also how you roll back instantly if the new version misbehaves — flip the flag, traffic returns to the old version, and the on-call gets their evening back.

How to run this pattern today

This isn't research. You can stand up the same workflow this week.

Tools. Claude and Cursor in agent mode are the pair Wakim called out. The IDE is incidental; the prompt shape is the thing. Anything that accepts structured input and returns diffable code works.

The refactor prompt. For each method, your AI call should look like:

# Refactor target
Implement `RouteResolver.resolveTargets(routeId)` against the new schema.

# Old implementation (reference only — do not preserve its assumptions)
[old code]

# New schema
[schema excerpt]

# Failing test (this MUST pass before you submit)
[test]

# Output
Return only the new method body. Do not modify the test.

The test is the contract. If the model can't make it pass in two or three attempts, the prompt is wrong or the schema is wrong — not the model. Fix the inputs, retry.

The harness. A test that runs in under a second per method is the actual product here. If your suite takes 20 minutes per run, you can't iterate. That's a refactoring-tools problem, not an AI problem.

The parallel deploy. Two versions, one flag, one validator service. The validator doesn't have to be sophisticated — a sampling comparator that logs divergent responses is enough for the first iteration. The point is to ship the new version into real traffic with a safety net, not to bet the migration on staging.

What this actually changes

When you have this loop working, the cost of a backend migration drops dramatically. Not because AI is smart, but because the loop is bounded and verifiable. The team at Datadog didn't need to stop the world to change the storage engine under a critical path.

The principle generalizes. Any time your system has a clear seam between the implementation and the contract — the API, the schema, the UI component boundary — you can swap the implementation without rewriting the world. The AI is just a faster way to traverse that seam.

This is also where the durable layer matters. When a backend rewrite changes the shape of the data a screen consumes, the screen still has to render correctly on web, iOS, and Android. If your components are tightly coupled to the data shape they were first built against, the backend migration becomes a frontend migration becomes a six-month project. If the component layer treats the data shape as a prop, the migration stays contained.

The part that doesn't change when the model does

Claude and Cursor will not be the same tools next year. The model behind them will be cheaper, faster, or different in ways that matter. What won't change is the shape of a good refactor: bounded inputs, a deterministic oracle, a parallel run for safety. Build those three things and you can ride whatever the model vendors ship next.

Build them once. Reuse them on every migration, every framework upgrade, every "we should rewrite this service" conversation. That's the part that compounds — across model churn, across team churn, across whatever the next 45-minute operation turns out to be.