AI coding agent acceptance checklist: prove the change before merge

An AI coding agent acceptance checklist turns “the code looks right” into evidence you can review. Before merging an agent-generated change, verify the user-visible behavior, the data and authorization boundaries, the failure states, and the checks that ran. If the agent cannot show how the change was tested, the change is not finished.

The checklist should be specific to the task, not a generic request to run tests. Start with the intended behavior, name what must not change, and require proof for both the happy path and the likely failure path.

Define the result before asking for code

Write acceptance criteria as observable outcomes. “Improve the settings page” leaves too much room for interpretation. “A signed-in member can update their display name, sees a validation error for an empty value, and cannot edit another workspace” gives the agent a boundary.

A useful task brief has five parts:

Goal:
Allow a workspace member to update their display name.

Must remain unchanged:
Workspace membership rules and the existing response shape.

Acceptance:
- Valid input persists and appears after refresh.
- Empty input returns a field error.
- A member cannot update another workspace.
- Double submission does not create duplicate audit events.

Checks:
Run the focused service tests, route tests, type check, and build check.

Keep acceptance criteria close to the issue, pull request, or task file. The agent should be able to quote the criteria in its completion report. If the requirement cannot be checked by a person or a test, rewrite it until it can.

Check the boundary before the implementation

An agent can make a local function pass while weakening the boundary around it. Before reviewing the diff, identify where the request enters, where identity is established, where resources are selected, and where side effects occur.

For a server operation, the expected path might be:

request → authentication → input validation → resource authorization
        → business operation → persistence → response

Do not treat this as a diagram to paste into production documentation. Treat it as a review order. A change that moves persistence before authorization deserves attention even if its output looks correct.

Ask the agent to name the boundary files it inspected and the boundary it deliberately left unchanged. This is more useful than a long summary of every line it touched.

For a production AI feature, apply the same rule to model calls and tools. The model may propose an action, but the application validates arguments, checks ownership, applies policy, and controls side effects. The OpenAI guidance on guardrails and human review separates automatic checks from approval decisions; your acceptance checklist should do the same.

Test the happy path and the denial path

Every acceptance item should have at least one positive and one negative case where a boundary is involved. If a member can update a record, test a member who may update it and a member who may not.

describe("update display name", () => {
  it("updates the member's own workspace", async () => {
    const result = await updateDisplayName({
      actorId: "member-a",
      workspaceId: "workspace-a",
      displayName: "Luna",
    })

    expect(result.displayName).toBe("Luna")
  })

  it("rejects a workspace outside the actor's membership", async () => {
    await expect(
      updateDisplayName({
        actorId: "member-a",
        workspaceId: "workspace-b",
        displayName: "Luna",
      }),
    ).rejects.toMatchObject({ code: "FORBIDDEN" })
  })
})

The exact test framework is less important than the assertion. A screenshot can show that a page renders. It cannot prove that a user from another workspace is blocked. Put authorization and data tests next to the service that owns the rule.

Include malformed input, expired sessions, missing records, duplicate requests, provider failures, and retry behavior when they are relevant. Agents tend to optimize for the path demonstrated in the prompt. The negative cases tell you whether they understood the contract or merely reproduced the example.

Make UI acceptance criteria behavioral

Visual review matters, but “looks good” is not an acceptance criterion. Name the states a user can encounter:

• Loading preserves context and prevents duplicate submission.
• Empty explains what the user can do next.
• Invalid identifies the field and keeps recoverable input.
• Error explains whether retry is safe.
• Success confirms completion without hiding useful context.
• Disabled communicates why the action is unavailable.

For a shared web and mobile experience, define what must remain consistent and what may vary. The label, permission rule, validation behavior, and state meaning may be shared. Navigation composition and input feedback may differ by platform.

The React Native accessibility documentation documents properties such as labels, hints, roles, and live regions, while noting that Android and iOS implementations differ. Add the platform-specific check to the task instead of asking an agent to make the interface “accessible” in the abstract.

A useful acceptance line is: “When the save action completes, assistive technology receives a meaningful status update, the button cannot be submitted twice, and the user’s position remains stable.” That can be tested and reviewed.

Inspect the diff for scope drift

An agent may solve the requested problem and edit unrelated files along the way. Review the changed-file list before reading every line. Ask four questions:

1. Does each changed file belong to the stated task?
2. Did a public interface, migration, or configuration change?
3. Did generated files or lockfiles change for a clear reason?
4. Did the agent edit a protected area without calling it out?

Then inspect the diff for hidden scope changes: a new dependency, a relaxed authorization check, a renamed response field, a broad formatter pass, or a fallback that masks an upstream failure.

Use a clean working tree before starting when possible. If unrelated local work exists, tell the agent which files are out of scope. A clean diff is not proof of correctness, but a noisy diff makes correctness harder to establish.

Require an evidence report

The completion response should be structured around evidence, not confidence. Require:

Changed:
- Added the display-name service validation.
- Added the route authorization test.

Verified:
- Focused service tests: passed.
- Route tests: passed.
- Type check: passed.

Not run:
- Full end-to-end suite; local browser service was unavailable.

Risks:
- The migration was not needed.
- The provider timeout path still needs an integration fixture.

“Not run” is a useful result. It tells the reviewer what remains open. Do not let an agent replace an unavailable check with a claim that the code should work. Record the missing evidence and decide whether it blocks merge.

For AI workflows, include request IDs, tool names, policy decisions, retry counts, and safe error categories in the verification plan. The LLM observability guide explains why generated text alone is not enough to understand a production result.

Test changes against real state transitions

A single test run can miss transitions that happen across requests. If the change affects billing, background jobs, permissions, or model workflows, test the state before and after each meaningful event.

Examples:

• A job is queued, claimed, retried, and completed.
• A user loses membership between planning and execution.
• A model output fails validation, then succeeds on a retry.
• A subscription changes state and access follows the verified event.
• A tool approval is rejected, and no side effect occurs.

Use fixtures that are safe to replay. Give duplicate events stable identifiers and assert that a retry does not create a second record or send a second notification. If the change crosses a provider boundary, separate provider response fixtures from product authorization tests.

A good acceptance suite tests decisions rather than implementation trivia. It should survive a refactor that keeps the behavior intact and fail when a boundary is weakened.

Keep project instructions aligned with the checklist

The acceptance checklist is most useful when the repository tells the agent how to run it. Claude’s project memory documentation describes CLAUDE.md as a place for project architecture, workflows, coding standards, and commands. Put durable verification rules there, and keep task-specific criteria in the issue or request.

Document:

• The focused test command for each application surface.
• The type check and build command.
• The locations of authorization and data-access rules.
• The files that are generated or protected.
• The deployment smoke checks.
• The evidence expected in a completion report.

Keep the instructions concise. A rule that applies only to one directory belongs near that directory. A procedure that changes often should live in its own document and be linked from the project instructions.

This pairs with an agent-readable repository structure: the repository makes the path discoverable, and the acceptance checklist proves the change stayed on that path.

Use a merge gate that matches the risk

Not every change needs the same review. A copy change may need a focused UI check. A permission or billing change needs denial tests, replay tests, and a careful diff. A migration needs a backup and rollback plan.

Set the gate by consequence:

OTF’s paid full-stack kits include AI-tool configuration files and tested prompts for extending code the buyer owns. That context can make the checklist easier for an agent to find, but it does not replace the checks themselves. The OTF templates page has the current options.

An AI coding agent acceptance checklist is a small contract between the task, the implementation, and the reviewer. Define observable behavior, test both permission and failure paths, inspect scope, require an evidence report, and match the merge gate to the consequence of the change. The agent can write the patch. The checklist tells you whether the patch is ready to ship.

Sources

• OpenAI Agents guardrails and human review
• React Native accessibility
• Claude Code project memory
• OTF templates