Skip to content
OTFotf
All posts

We audited 8 AI-built apps for production. The frontend was fine, the trust boundary wasn't.

D
DaveAuthor
5 min read
We audited 8 AI-built apps for production. The frontend was fine, the trust boundary wasn't.

We built a free agent skill called ship-ready — it reads an AI-built app and writes a file-cited plan for what's missing before real users hit it. Then we pointed it at 8 real apps built by AI tools (Lovable/Bolt/Cursor-class): five web apps, three mobile. Dashboards, an invoicing app, an AI email tool, a multiplayer whiteboard, a recipe site.

The results were remarkably consistent, and not in the way you'd guess. The design was good. The data layer was usually right. The thing that was broken — over and over — was the boundary between the browser and the backend.

Here's what showed up across all 8.

Two things every single app was missing

8 of 8: no error boundary. Every app would white-screen on a single uncaught render error. No fallback, no retry, no logging. One of them had a boundary — and it printed the full stack trace to end users.

8 of 8: no CLAUDE.md / .cursorrules. Not one had an agent-config file. So the next AI session re-learns the whole codebase from scratch and guesses the conventions — which, it turns out, is exactly how most of the bugs below got written.

These are the boring ones. They're also universal.

The pattern that matters: AI gates the frontend, not the backend

Of the 7 apps with real backend logic, 6 had a server-side authorization hole. The frontend hid the button; the backend never checked. A sample, anonymized:

  • A checkout function guarded by if (authHeader) — so a request with no token skipped verification entirely and minted a payment session for whatever user id was in the request body.
  • Edge functions that took userId from the request body and never verified a session — so any caller could act as any user (ban people, post as them, read their subscribers).
  • An IDOR: an endpoint that updated a record by id with no owner check — any logged-in user could flip anyone else's data.
  • An unauthenticated open email relay — no auth, CORS: *, arbitrary recipient/body — anyone with the URL could send mail on the app's real Postmark token.
  • A public AI endpoint with no auth and no rate limit — every call burned real OpenAI tokens; a stranger could drain the budget in a loop.
  • Admin access gated by a client-side email string compare (user.email.endsWith('@company.com')) — UI-only; the data was one direct call away.

Same root cause every time: the trust boundary was implemented in the client, where it isn't a boundary at all.

And 2 of the 8 committed a live secret — a real API key sitting in a .env.local that .gitignore didn't cover.

11 production screens. Login, database, payments — all wired.

The SaaS Dashboard Kit ships everything already connected. Nothing to set up. Live demo at saas.otf-kit.dev.

See the live demo

Mobile had its own recurring wall

Of the 3 mobile apps, 2 wouldn't build on any machine but the author's — a placeholder EAS project id, a file: dependency pointing at a local SDK checkout, and metro.config.js hard-coded to absolute paths on someone's disk. It runs on the laptop it was born on and nowhere else. (The audit correctly didn't flag the public project id as a secret — that's a client id, not a secret. A web-shaped auditor gets that wrong.)

The apps that got it right did one thing

Not every app failed, and the ones that passed are the interesting part. Two of the eight had clean backends, and they'd done the same single thing: moved the trust boundary to the server. An invoicing app verified the token on every route and took the user id from the verified token, never the request body — so there was nothing to forge. A multiplayer whiteboard pushed every write through its backend's row rules, so a client couldn't reach past its own data even if it tried.

Same tools, same afternoon of building. The only difference was where they drew the line between "the browser asked nicely" and "the server checked." That's the whole lesson: the trust boundary isn't a feature you bolt on later, it's a decision about where the check lives — and moving it one layer down is usually a few lines of code.

Why this happens

AI coding tools are trained on the happy path. Tutorials show the feature working; they rarely show the request that arrives with no token, the render that throws, the second user who guesses the first user's id. So the model builds a convincing demo and stops exactly where production begins — at failure paths and trust boundaries.

None of this means the tools are bad. It means "the demo works" and "real users can't rob you" are different milestones, and the second one is invisible until someone looks for it.

It compounds, too. With no CLAUDE.md, the next session — human or agent — re-derives the conventions from scratch and re-introduces the same class of gap somewhere new. Several of the bugs we found weren't in the original build; they were a second feature added later that guessed the auth pattern and guessed wrong. The missing config file and the missing trust boundary turn out to be the same problem wearing two hats: nobody ever wrote down where the line is.

The fixes are free, in your own stack

Nothing here needs a product to fix:

  • Auth: verify the session/JWT on the server for every protected route; take the user id from the verified token, never the request body; scope every write by the owner.
  • Errors: add an error boundary with a fallback + retry (on Expo Router, the route-level ErrorBoundary export); give every mutation an onError; log somewhere you can read.
  • Secrets: move them server-side; make sure .gitignore actually covers .env.local.
  • Config: add a CLAUDE.md documenting the stack, the conventions, and the traps — so the next agent session doesn't re-introduce the bug you just fixed.

If you want to run the same audit on your own app, it's one line and it's MIT:

npx skills add otf-kit/skills --skill ship-ready

It's read-only, makes zero network calls, and writes the plan for you. We build OTF — an open-source UI SDK and full-stack kits — and where a gap genuinely maps to something we ship, the audit will note it as one option. Most runs never mention us, and the free fix in your own stack always comes first. That's the deal.

The frontend is a solved problem. Ship the back half.

ai-toolsbackendagents
OTF SaaS Dashboard Kit

Ship the product, not the setup.

  • 11 production screens — auth, billing, team, analytics, settings
  • Real database, payments, and login — all wired on day 1
  • AI configs pre-tuned so your agent extends instead of regenerates