# OpenRouter Batch for owned repo jobs: when to keep sync vs half-price async

> Repo jobs POST /api/v1/batches, pin provider.only, poll to terminal. Keep sync interactive; custom_id isolates bad rows so one failure never kills the job.
> By Dave · 2026-09-22
> Source: https://otf-kit.dev/blog/openrouter-batch-api-owned-repo

OpenRouter’s Batch API is half-price inference for work that can wait inside a 24-hour window. A provider picks when to run your bundle; you generally pay 50% (and sometimes less) of normal per-token pricing. It ships on more than 70 models. Across 230k+ batches in the two-week beta, the median finished in 7 minutes and 90% finished within an hour.

If you run owned-repo jobs — overnight labeling, ticket summarization, eval scoring, embedding backfills — the useful split is not “always Batch” or “always sync.” Interactive paths stay on the sync API. Background repo jobs POST to `/api/v1/batches`, pin `provider.only` when you care which provider runs the bundle, and poll until a terminal status. Unique `custom_id` values keep one bad row from killing the rest of the job.

![Dex, Byte, and Luna routing owned-repo jobs into an OpenRouter Batch chute](https://cdn.otf-kit.dev/blog/openrouter-batch-api-owned-repo/hero-20260922a.png)

## What Batch buys you (and what it does not)

Batch is for workloads where highly variable response times are acceptable — labeling a corpus, back-filling embeddings, scoring an eval set, summarizing a ticket backlog, or running one prompt across thousands of rows overnight. The provider schedules inside the 24-hour window and discounts the per-token line.

You do not escape every cost. Web search still bills at standard rates. Images and files must be public URLs. Audio, video, and OpenRouter’s web search plugin are unavailable in batch — keep those shapes on sync until inputs are public URLs or you drop the modalities.

Default routing picks one provider per batch: after allowlist, data policy, and BYOK, OpenRouter chooses the cheapest eligible batch endpoint. With a provider key, BYOK batches route through your key and you pay only the BYOK fee on OpenRouter’s side. Batches appear in the Batches tab (model, provider, status, cost). Inputs and results stay for 30 days, or until DELETE.

## The repo-job shape: POST, pin, poll

Submit with `POST https://openrouter.ai/api/v1/batches`. Required top-level fields: `endpoint`, `model`, and `requests`. Optional: `provider` as `{ "only": [...] }` and `completion_window` (only `24h`). Serialize `endpoint`, `model`, and any `provider` / `completion_window` before `requests` — stream parsing accepts large arrays, and `requests` first returns `400`.

Each `requests` item is `{ custom_id, body }`. `custom_id` must be unique in the batch. `body` follows the endpoint shape: `/v1/chat/completions`, `/v1/responses`, `/v1/messages`, or `/v1/embeddings`. The batch-level `model` applies to every row; a per-request `model` must match or the submit is rejected.

A successful submit returns `202 Accepted` with `status: "validating"` — persisted and queued, not finished. Poll `GET /api/v1/batches/:id` until terminal: `completed`, `failed`, `expired`, or `cancelled`. Happy path: `validating → in_progress → finalizing → completed`. Completed results are inline in that GET; there is no separate download endpoint.

For an owned-repo worker: enqueue, POST, store the batch id, poll (or schedule wake) until terminal. Do not block an interactive HTTP handler on that loop. Chat, IDE agents, and “answer now” paths stay on sync — same discipline as [API timeouts and retries for owned AI backends](https://otf-kit.dev/blog/api-timeouts-retries-ai-backends).

![Split lab: interactive sync orb versus half-price batch crate](https://cdn.otf-kit.dev/blog/openrouter-batch-api-owned-repo/inbody1-20260922a.png)

## When to keep the call on sync

Keep sync when any of these hold:

- The caller is waiting in the UI or an agent turn that must return before the user moves on.
- You need OpenRouter-orchestrated web search, audio, video, or non-URL image/file inputs that batch rejects.
- You need provider preferences Batch does not accept. On Batch, `provider.only` is the only provider preference accepted; `order`, `sort`, `allow_fallbacks`, and the other sync preferences are rejected.
- You are debugging a single request and want immediate generation feedback, not a 24-hour window.

Batch still finishes fast in practice — median 7 minutes, 90% within an hour in the beta — but the contract you design against is the 24-hour window and provider-chosen scheduling. If your product SLA is “under a few seconds,” Batch is the wrong surface even when the discount is attractive.

Use Batch when the job can accept that window: overnight corpus work, eval sets, ticket backlogs, embedding backfills, and any repo pipeline that already persists state and can resume after a poll. Size is not the only signal; a single-request batch is valid, but if that one request is interactive, sync is still the right call.

## Pin `provider.only` so the job owns the provider

Every request in a batch runs on one provider chosen at submit time. Default cheapest-endpoint routing is fine when cost is the only concern. Repo jobs that need a specific provider’s batch behavior — URL image/file fetch, BYOK billing, data residency — set top-level `provider.only` before `requests`.

If none of the listed providers has an eligible `:batch` endpoint, submit returns `404` instead of falling back. Fail closed: better a loud reject than a silent provider swap. Filter models by the `:batch` variant before you hard-code a pin.

BYOK still applies: with a provider key, supporting providers route through it. Completed batches report `is_byok` when that path ran. Keep the same key enabled if you later DELETE a BYOK batch that needs upstream cleanup — missing key yields `409`.

## `custom_id` so one bad row never kills the job

Per-request results are the operational answer to the buyer question. Every result returns independently, so a few bad rows never fail the rest of the job. On completion, each result maps back with `custom_id`, and exactly one of `response` or `error` is populated.

Design the repo job around that contract:

1. Generate a stable unique `custom_id` per row (ticket id, eval case id, embedding document id) before POST.
2. Persist the mapping from `custom_id` to your domain entity in the same transaction that records the batch id.
3. On terminal `completed`, walk `results`, apply successes, and queue retries only for rows with `error` — not for the whole batch.
4. Treat batch-level `failed` / `expired` / `cancelled` as job-level outcomes; do not confuse them with per-row errors inside a completed batch.

`request_counts` on the batch object reports `total`, `completed`, and `failed`. A completed batch can show nonzero failed counts while still returning inline results for the successful rows. Your worker should key off `custom_id`, not assume all-or-nothing success.

![Custom-id sorting rail isolates one bad row without stopping the batch](https://cdn.otf-kit.dev/blog/openrouter-batch-api-owned-repo/inbody2-20260922a.png)

## Lifecycle hygiene for owned workers

Poll until terminal. While in progress — or after failed, expired, or cancelled — `results` is `null`. Only `completed` returns the inline array. After you ingest results (or decide you no longer need them), DELETE is available for terminal batches (`completed`, `failed`, `expired`, `cancelled`). Deletion purges OpenRouter-held request and result artifacts without waiting for the 30-day retention window. It is not cancellation: an in-flight batch returns `409`.

List endpoints help operators: batches are workspace-scoped, newest first, with optional status filters. List items set `results` to `null`; retrieve by id when you need the payload. For human-facing ops and agent triage, keep correlation between your job id, the OpenRouter batch id, and each `custom_id` in structured logs — the same correlation habit as [structured production logs agents can triage](https://otf-kit.dev/blog/production-structured-logging-for-agents).

## Limits that change the repo design

Multimodal batch input is URL-only; support depends on the routed provider. Base64 and `data:` URI images are rejected everywhere. File parts are URL references only where the provider supports them. Audio and video parts are rejected on every provider. OpenRouter-orchestrated search is unavailable; `:online` variants return `422` on submit. Provider-native web search may pass through when that provider runs the search — that is not the OpenRouter web plugin.

Design repo jobs around text bodies (chat, messages, responses, embeddings), public HTTPS assets when needed, and a pinned provider that supports the asset type. Keep rejected shapes on sync; batch the rest.

## Builder checklist

1. Decide sync vs Batch per call site: interactive stays sync; durable repo jobs use Batch.
2. Confirm the model has a `:batch` endpoint; pin `provider.only` when provider behavior matters.
3. POST `/api/v1/batches` with `endpoint`, `model`, then optional `provider` / `completion_window`, then `requests` of unique `{ custom_id, body }`.
4. Expect `202` + `validating`; poll GET until `completed` / `failed` / `expired` / `cancelled`.
5. On `completed`, apply inline results by `custom_id`; retry only failed rows.
6. DELETE terminal batches when you no longer need the 30-day retention; use the Batches tab for cost and status.

## Sources

- [OpenRouter Batch API announcement](https://openrouter.ai/blog/announcements/batch-api/)
- [OpenRouter Batch API Quickstart](https://openrouter.ai/docs/batch-quickstart)