# An AI app security checklist for builders moving from demo to production

> A practical AI app security checklist covering prompt injection, access control, output validation, secrets, spend limits, and audit trails.
> By Dave · 2026-08-30
> Source: https://otf-kit.dev/blog/ai-app-security-checklist

A production AI app needs more than a safe-looking system prompt. Before you ship, you need a boundary around every input, model call, tool invocation, database write, and user-visible output. This AI app security checklist gives you that boundary in concrete terms.

The short version: authenticate the caller, authorize each resource, treat retrieved text as untrusted data, validate model output before using it, keep tools narrow, store secrets outside code, cap spend and rate, and log enough context to investigate a bad result without logging private data by accident. These controls matter whether your first version was written with Cursor, Claude Code, or another AI coding tool.

## Start with the threat model, not the prompt

Write down the path from user input to side effect. For a support assistant, it might be:

1. A signed-in user submits a question.
2. The server loads documents for that user's workspace.
3. The model drafts an answer.
4. A tool fetches an account record.
5. The server saves the answer.

Now mark every boundary. Which values come from the browser? Which come from a document? Which can cause a write? Which identity is attached to the request? If the answer is “the model decides,” you have found a security gap, not an architecture.

OWASP's current GenAI LLM Top 10 lists prompt injection, insecure output handling, sensitive information disclosure, excessive agency, and denial of service among the risks application teams need to address. Use the [OWASP GenAI LLM Top 10](https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/) as a threat checklist, then map each item to a server-side control.

## Authenticate and authorize on the server

A model should never be the source of truth for identity. The request handler must derive the user and workspace from a verified session, then check access to every record before it enters the prompt or leaves the database.

Do not accept `userId`, `workspaceId`, or `role` from the request body and trust it. A browser can send any JSON it wants. The same rule applies to tool arguments generated by a model: parse them, validate them, and run authorization against the authenticated principal.

A small policy function makes the boundary visible:

```ts
type Principal = {
  userId: string
  workspaceId: string
  role: "member" | "admin"
}

type ToolRequest = {
  name: "get_invoice" | "create_ticket"
  invoiceId?: string
  subject?: string
}

async function authorizeTool(
  principal: Principal,
  request: ToolRequest,
) {
  if (request.name === "get_invoice") {
    if (!request.invoiceId) throw new Error("invoiceId is required")

    const invoice = await db.invoice.findFirst({
      where: {
        id: request.invoiceId,
        workspaceId: principal.workspaceId,
      },
      select: { id: true, status: true, total: true },
    })

    if (!invoice) throw new Error("Invoice not found")
    return invoice
  }

  if (request.name === "create_ticket") {
    if (principal.role !== "admin") throw new Error("Not allowed")
    if (!request.subject || request.subject.length > 200) {
      throw new Error("Invalid subject")
    }
    return { ok: true }
  }

  throw new Error("Unknown tool")
}
```

The database query includes the workspace boundary. That is more useful than checking permissions in a prompt or relying on a hidden convention in generated code.

## Treat prompts, documents, and model output as data

Prompt injection is not limited to a malicious user typing “ignore previous instructions.” It can arrive through a web page, uploaded file, ticket, repository issue, or customer message. OWASP's [prompt injection prevention guidance](https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_Injection_Prevention_Cheat_Sheet.html) describes direct and indirect attacks, including instructions hidden in external content.

Separate instructions from data in your application design. Put untrusted content in a clearly marked field, tell the model to summarize it rather than obey it, and assume that wording alone will not stop every attack. More important, do not give the model a tool whose permission exceeds the user's permission.

The output needs a boundary too. If the model returns JSON, parse it with a schema. If it returns Markdown, render it with an escaping policy. If it returns an email address, URL, SQL fragment, shell command, or database identifier, validate that value for its specific use before passing it onward. “The model usually returns the right shape” is not validation.

For a structured response, the safe sequence is:

```ts
const raw = await model.generate({
  instructions: "Return a support classification.",
  input: userText,
})

let parsed: unknown
try {
  parsed = JSON.parse(raw.text)
} catch {
  throw new Error("Model returned invalid JSON")
}

const result = classificationSchema.parse(parsed)

await db.ticket.update({
  where: { id: ticketId, workspaceId: principal.workspaceId },
  data: {
    category: result.category,
    priority: result.priority,
  },
})
```

The schema should constrain enums, lengths, numeric ranges, and optional fields. Keep the write conditional on the same tenant scope used during authorization.

## Give tools the smallest possible permissions

Start with read-only tools. Add writes only when you can name the user action, authorization rule, validation rule, and rollback behavior. A “manage account” tool is too broad; `get_invoice` with one workspace-scoped identifier is easier to inspect.

Require confirmation for consequential actions such as sending messages, changing billing details, deleting records, or publishing content. A model can prepare an action, but the user or a separate server-side policy should approve it. Record who approved it, what arguments were approved, and whether the final arguments changed before execution.

Also set limits around loops. A tool-using agent should have a maximum number of steps, a maximum input size, a request timeout, and a budget for model calls. Return a controlled failure when a limit is reached. Do not let a retry loop turn a malformed request into a large bill.

If the feature runs for longer than a request, pair these controls with a queue, stable job identity, and checkpoints. The [background-jobs guide](/blog/ai-production-background-jobs) covers why retries and side effects need idempotency once an AI workflow crosses process boundaries.

## Protect secrets and personal data

Keep provider keys on the server and load them through the deployment's secret store. Never put them in browser bundles, prompts, screenshots, test fixtures, or exception messages. Review logs as if a customer will read them: redact tokens, authorization headers, passwords, full payment details, and unnecessary personal data.

Use separate credentials for development, staging, and production where the provider supports it. Restrict production access to the smallest group that needs it. Add spend alerts and hard limits at the provider level, then enforce per-user and per-workspace limits in your own application. Provider controls catch account-wide drift; application controls stop one tenant from consuming everyone else's budget.

Set retention deliberately. If you store prompts and responses for debugging, define how long they remain, who can access them, and how a user can request deletion. Hash or truncate identifiers in analytics where the full value is not needed. Security is not only preventing remote access; it is also reducing what an incident can expose.

## Log security events without logging everything

You need an audit trail for sign-ins, denied authorization checks, tool requests, confirmation decisions, model errors, rate-limit events, and high-impact writes. Each event should include a timestamp, request or job ID, actor ID, workspace ID, action, result, and a safe reason code.

Do not make the prompt itself the audit trail by default. Store a redacted summary or a content hash unless the raw text is required for a documented support workflow. Make logs append-only for normal application roles, and test that a user cannot read another workspace's events.

Finally, test the failures you are trying to prevent. Add cases for cross-workspace IDs, expired sessions, oversized inputs, malformed JSON, prompt injection inside retrieved documents, repeated job delivery, unauthorized tool calls, and budget exhaustion. Run them in CI and against a staging project before production access is enabled.

## A practical ship gate

Before calling the feature production-ready, answer yes to these questions:

- Is identity derived from a verified server-side session?
- Does every read and write enforce the tenant boundary?
- Are retrieved documents and user messages treated as untrusted data?
- Is every model output parsed and validated for its next use?
- Are tools narrow, permission-checked, rate-limited, and confirmation-gated when needed?
- Are secrets outside source code and client bundles?
- Are spend, input size, time, and step limits enforced?
- Can you investigate a denied action without exposing the customer's prompt?
- Are retries safe for every external side effect?
- Have these cases been tested with hostile input, not only the happy path?

OTF's [templates page](https://otf-kit.dev/templates) includes a free AI configs pack for Cursor, Claude, and Lovable. That is a useful starting point for getting an AI coding tool oriented to your project, but the security boundary still belongs in your server code, database policies, provider settings, and tests. Start with the checklist above, then ship one narrow tool at a time.

## Sources

- [OWASP GenAI LLM Top 10 2026](https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/)
- [OWASP LLM Prompt Injection Prevention Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_Injection_Prevention_Cheat_Sheet.html)