AI provider portability: build one model boundary without hiding provider differences

AI provider portability works when your application owns one stable model boundary while adapters preserve the differences that matter. Do not flatten every provider into a fake universal API. Normalize the inputs and outputs your product actually needs, keep provider-specific options behind an explicit escape hatch, and record which provider produced each result.

That design lets a production app change vendors, add a fallback, or run an evaluation without rewriting every route and screen. It also avoids the opposite failure: scattering provider-specific request shapes through handlers, background jobs, tests, and UI code until a price or availability change becomes a migration project.

What should an AI provider boundary own?

Your boundary should own product behavior, not vendor vocabulary. A useful first version accepts a task, an input, a policy, and an execution context. It returns text or structured output, usage metadata when available, and a typed error when the request fails.

export type AiTask = {
  name: string;
  input: string;
  system?: string;
  schema?: Record<string, unknown>;
  maxOutputTokens?: number;
  providerHint?: string;
};

export type AiResult = {
  text: string;
  provider: string;
  model: string;
  usage?: {
    inputTokens?: number;
    outputTokens?: number;
  };
  raw?: unknown;
};

export interface AiProvider {
  complete(task: AiTask): Promise<AiResult>;
}

The raw field is deliberate. If you throw away the original response, debugging and provider-specific features become harder. Keep it out of normal product logic, redact it before persistence, and expose it only to code that explicitly opts in.

The adapter should translate from this contract into the provider’s request format. OpenAI’s Responses API reference documents response inputs, developer or system instructions, structured input items, background responses, and response output objects. Anthropic’s Messages API reference documents alternating user and assistant turns, a top-level system parameter, content blocks, and a required maximum token limit. Those are similar concepts with different shapes. The adapter is where the translation belongs.

How do you keep the common path small?

Start with the narrowest capability your product needs. If the first feature is “classify a support message into one of four labels,” do not design a universal abstraction for every tool call, image input, citation, and streaming mode on day one.

const task: AiTask = {
  name: 'support-label',
  input: messageText,
  system: 'Return one label: billing, bug, account, or other.',
  maxOutputTokens: 20,
};

const result = await ai.complete(task);
const label = parseSupportLabel(result.text);

The route knows the task name and its acceptance rules. It does not know whether the selected provider expects max_tokens, max_output_tokens, a content array, or a different message role. That knowledge stays in the adapter.

Keep capability declarations next to the adapter:

export type ProviderCapabilities = {
  structuredOutput: boolean;
  streaming: boolean;
  toolCalls: boolean;
  imageInput: boolean;
};

export type RegisteredProvider = {
  id: string;
  model: string;
  provider: AiProvider;
  capabilities: ProviderCapabilities;
};

A capability check should fail before a request is sent. If a task needs structured output and the selected provider cannot guarantee it, return a typed configuration error or choose an explicitly approved alternative. Do not silently downgrade to free-form text and hope a parser survives.

How should adapters handle provider differences?

Keep one adapter per provider and one translator per capability. Avoid a giant conditional that checks provider names throughout the application.

class OpenAiAdapter implements AiProvider {
  constructor(private readonly client: OpenAiClient) {}

  async complete(task: AiTask): Promise<AiResult> {
    const response = await this.client.responses.create({
      model: 'configured-model',
      instructions: task.system,
      input: task.input,
      max_output_tokens: task.maxOutputTokens,
    });

    return {
      text: response.output_text,
      provider: 'openai',
      model: response.model,
      usage: {
        inputTokens: response.usage?.input_tokens,
        outputTokens: response.usage?.output_tokens,
      },
      raw: response,
    };
  }
}

The client and response fields in this example represent an adapter boundary; use the installed provider client’s current types rather than copying names into a different implementation. The important property is that the rest of the app receives AiResult, not a vendor response object.

For a second provider, write a separate adapter that maps its conversation and content format to the same result. Keep provider-specific features explicit:

type ProviderOptions = {
  common: {
    maxOutputTokens?: number;
  };
  openai?: { background?: boolean };
  anthropic?: { cacheControl?: 'ephemeral' };
};

If a route passes anthropic.cacheControl, it has opted into a provider-specific contract. That is better than pretending the option has identical semantics everywhere. Portability is not sameness; it is controlled change.

How do you design fallback without duplicate side effects?

Fallback belongs around model calls that are safe to repeat, not around an entire business operation. If a request also charges a card, sends an email, or writes a record, separate those effects from model selection.

async function completeWithFallback(
  task: AiTask,
  providers: RegisteredProvider[],
): Promise<AiResult> {
  let lastError: unknown;

  for (const candidate of providers) {
    if (task.schema && !candidate.capabilities.structuredOutput) continue;

    try {
      return await candidate.provider.complete(task);
    } catch (error) {
      lastError = error;
      if (!isRetryableModelError(error)) throw error;
    }
  }

  throw new Error(`No approved provider completed ${task.name}`, {
    cause: lastError,
  });
}

The retry classifier needs a small, reviewed set of conditions. A timeout or temporary upstream failure may be retryable. An invalid request, policy refusal, malformed schema, or authentication error generally needs a different path. Do not retry every exception, and do not send the same prompt to three providers without recording that choice.

Use an idempotency key for the surrounding job. Background jobs for AI features explains why a worker needs explicit state and safe retries; the same rule applies when a model fallback is inside that worker. Store the task ID, attempt number, provider, model, status, and validation result. If the worker restarts after the first provider returned a response, it should know whether to reuse, validate, or deliberately retry it.

How do you preserve structured output across providers?

Treat structured output as a validation contract, not as a formatting preference. The provider may offer a schema feature, but your application still needs to validate the returned value before using it.

import { z } from 'zod';

const SupportLabel = z.object({
  label: z.enum(['billing', 'bug', 'account', 'other']),
  confidence: z.number().min(0).max(1),
});

function parseSupportLabel(text: string) {
  const value = JSON.parse(text);
  return SupportLabel.parse(value);
}

If your project does not use this validation library, use the validator already established in the repository. The invariant is what matters: parse, validate, and reject before a model result changes application state.

Keep fixtures for every provider adapter. One fixture should be a valid result, one should be truncated, one should contain unexpected fields, and one should represent a provider error. Then run the same task set through each adapter and compare the normalized result, not only the raw text.

For quality measurement, link each result to an evaluation case. The LLM evaluation loop covers the practical cycle: define cases, run a baseline, change one variable, inspect failures, and keep the evidence. Portability is useful only if switching providers does not erase your ability to compare behavior.

What should you observe and budget?

Record provider, model, latency, input and output usage when returned, retry count, validation status, and a redacted task identifier. LLM observability for production apps describes why the trace should connect the request to retrieval, model calls, tool authorization, validation, and the final outcome.

type AiTrace = {
  taskId: string;
  taskName: string;
  provider: string;
  model: string;
  durationMs: number;
  attempts: number;
  validation: 'passed' | 'failed' | 'skipped';
  inputTokens?: number;
  outputTokens?: number;
};

Do not log prompts or outputs by default when they may contain private customer data. Sample deliberately, redact before storage, and set retention by task type. A provider switch is a data-flow change, so review where prompts travel and where responses are retained before changing the routing table.

Your budget should include failed attempts. If fallback turns one request into two paid calls, the cost model must show that. Compare successful task rate, validated-result rate, latency, retry rate, and cost per accepted result. A cheaper token price is not a cost win if validation failures create manual work.

When is portability worth the extra code?

Portability is worth paying for when you have a real reason to change or compare providers: regional availability, workload-specific quality, cost variation, data handling requirements, or a fallback policy. It is not worth building a ten-provider abstraction before one production task exists.

Keep the first adapter small, put it behind the task boundary, and add a second provider only when a measured requirement justifies it. OTF’s full-stack app templates are one starting point for builders who want owned application code plus AI-tool configuration before they add this boundary; the provider adapters and policy decisions remain yours.

AI provider portability is a code-ownership decision as much as a vendor decision. Normalize the product contract, preserve differences behind adapters, validate every result, and log enough metadata to compare accepted outcomes. That gives you room to change providers without pretending their APIs or behavior are interchangeable.

Sources

• Responses API reference — OpenAI
• Messages API — Anthropic
• AI provider portability implementation reference — JSON Schema

Originally published at otf-kit.dev — full-stack app templates for web and mobile. See the templates →