Rate-limit your owned API at the edge and in the app, not only in memory

Sandbox traffic is polite. Production traffic is not. Once an owned API leaves the laptop and sits behind a real hostname, the first failure mode is not a clever bug in business logic. It is unbounded callers: a retry storm, a leaked key, a NAT full of mobile clients, a script that treats 200 as a suggestion. Two layers stop that. A coarse edge or WAF limit drops obvious floods before they reach your process. Application middleware, backed by shared Redis across replicas, enforces identity-aware budgets the edge cannot see.

This is inbound protection on an API you own. It is not the outbound timeout and retry budget you put in front of an AI provider, which is a different failure domain covered in timeouts and retries for AI backends. It is also not the rollout mechanics in rolling deploys for an owned backend. Deploys change which process answers. Rate limits decide whether that process should answer at all. If you only ship one of those, you still ship a production hole.

Why one layer is not enough

Edge limits are cheap and blunt. They count by IP (and related characteristics) before your process spends CPU. That is the right place for volumetric noise. It is the wrong place for "this API key may spend 60 requests per minute on /v1/jobs but only 5 on /v1/login." The edge does not know your session cookie or your tenant id unless you push that identity into a custom characteristic you also have to maintain.

App limits without a shared store are theater under load. A memory counter per process means N replicas multiply the budget by N. Attackers and broken clients notice. Redis (or another shared store the middleware supports) makes the counter one number the fleet agrees on.

Cloudflare as the coarse gate

Cloudflare rate limiting rules are the usual first gate for an owned hostname already on that network. A rule is an expression match, a counting characteristic, a period, a requests-per-period threshold, and a mitigation timeout. Characteristics include IP for the coarse layer; NAT support exists on higher plans when many customers share an address.

Keep the edge rule coarse: whole API host or a noisy path prefix, not a replica of every product plan. Cloudflare itself notes counters are not precise to the last request and that excess can still reach origin for a short window. Treat edge limits as flood control, not as the source of truth for plan quotas. The app layer owns a stable 429 and parseable RateLimit headers.

Express, after auth, with a shared store

On Express, express-rate-limit v8 matches this shape. Configure windowMs and limit per route group, statusCode 429, standardHeaders: 'draft-8' for the RateLimit header family, and legacyHeaders: false so you do not also emit older X-RateLimit-* headers.

import { rateLimit } from "express-rate-limit";

const readLimiter = rateLimit({
  windowMs: 60_000,
  limit: Number(process.env.RATE_LIMIT_READ_MAX),
  statusCode: 429,
  standardHeaders: "draft-8",
  legacyHeaders: false,
  keyGenerator: (req) =>
    req.user?.id ?? req.header("x-api-key") ?? req.ip,
  skip: (req) => req.path === "/health" || req.path === "/ready",
  // store: shared Redis store — required with >1 replica
});

app.set("trust proxy", Number(process.env.TRUST_PROXY_HOPS));
app.use("/v1", readLimiter);

keyGenerator must run after auth for user or API-key keys, with IP only as anonymous fallback. Without trust proxy set to $TRUST_PROXY_HOPS, every caller looks like the edge and you throttle the proxy, not the client.

store makes the counter real across replicas. Point it at Redis with $RATE_LIMIT_REDIS_URL — one shared database, not a per-pod memory map. passOnStoreError defaults to fail-closed: if Redis errors, do not pretend the caller is under budget. Skip health and readiness paths so probes do not trip the public key during rolling deploys.

Auth and OTP routes get a tighter limit from $RATE_LIMIT_AUTH_MAX on the same store and header dialect. Share the Redis client; only the window budget changes. Do not stack a second global limiter on the same route.

NestJS throttler and Fastify

NestJS @nestjs/throttler uses the same ideas: module ttl and limit, then tighter overrides on sensitive controllers. Behind a proxy, enable trust proxy on the HTTP adapter and override getTracker() so the key comes from the trusted client address, not the edge hop. Fastify @fastify/rate-limit can use Redis through ioredis. Prefer the authenticated principal in the key function, share one store, and skip probes. Pick the library that matches the server you already run. Do not run two app limiters on the same route.

The client contract

Return 429 Too Many Requests (RFC 6585). Pair it with the header fields in draft-ietf-httpapi-ratelimit-headers-11. RateLimit-Policy names the quota. RateLimit carries the current state. The structured fields that matter in practice are q (quota), w (window), r (remaining), and t (seconds until reset). standardHeaders: 'draft-8' on express-rate-limit asks the middleware to speak that dialect instead of the legacy X-RateLimit-* set.

If you also send Retry-After, clients and intermediaries should honor it ahead of guessing from the RateLimit window. Set it on auth lockouts where you want a hard pause. On ordinary read limits, the t field is often enough. Document one policy string and keep it stable so generated clients can parse it.

A 429 without a body is legal. If you include a body, use a stable error code your logs can join. Structured logs should record the key class (user, api-key, ip), the route group, and whether the decision was edge or app — the same discipline as production structured logging for agents. A 429 you emitted is a policy decision. An outbound model timeout is a dependency failure; keep those alerts separate from API timeouts and retries.

Differentiated budgets

Login, password reset, and OTP are the tightest budgets. They are cheap for an attacker to spray and expensive for you when they succeed. Count failed attempts when your library supports it so successful logins do not burn the same window.

Reads are looser and keyed on user or API key, with IP fallback for anonymous traffic. Losing Redis on a sensitive read can still fail closed; if you ever choose fail-open, confine it to a route whose worst case you have already sized, and write that exception down.

Billing-critical writes fail closed with no debate. Creating a charge, minting a token, or enqueueing paid work must not proceed because the counter was unreachable. Budget Redis like a dependency: a network path that does not share fate with the noisiest cache, and a metric on store errors that pages someone.

What to verify before you call it done

Exercise the limit from outside the cluster. Confirm two replicas share a counter: exhaust the budget on one task and the next request on another is still 429. Confirm a spoofed X-Forwarded-For does not enable a fresh bucket when $TRUST_PROXY_HOPS is set correctly. Confirm health checks do not increment the public key. Confirm login uses $RATE_LIMIT_AUTH_MAX, reads use $RATE_LIMIT_READ_MAX, and a Redis outage on the billing route denies rather than allows.

Read one 429 as a client would: status, RateLimit with q, w, r, and t, no legacy duplicate, Retry-After only where you decided it takes precedence. If an edge mitigation and an app 429 can both fire, log which one fired.

Owned-repo kits at Open Template Forest are where this kind of gate should land once the numbers are real: middleware, env names, and probe skips checked in beside the service. The policy is yours. The shape above is the part that should not be reinvented when a key leaks.

Sources

• Cloudflare rate limiting rules: https://developers.cloudflare.com/waf/rate-limiting-rules/
• express-rate-limit: https://www.npmjs.com/package/express-rate-limit
• NestJS rate limiting: https://docs.nestjs.com/security/rate-limiting
• IETF RateLimit headers draft 11: https://datatracker.ietf.org/doc/html/draft-ietf-httpapi-ratelimit-headers
• Timeouts and retries for AI backends: https://otf-kit.dev/blog/api-timeouts-retries-ai-backends
• Rolling deploys for an owned backend: https://otf-kit.dev/blog/rolling-deploys-owned-backend
• Production structured logging for agents: https://otf-kit.dev/blog/production-structured-logging-for-agents