# Gemini 3.8 Live on AI Gateway: ship realtime voice with visual grounding

> Builders get realtime audio and visual grounding via AI Gateway WebSocket instead of provider-specific voice infra.
> By Dave · 2026-09-16
> Source: https://otf-kit.dev/blog/gemini-3-8-live-ai-gateway

Production builders shipping voice and realtime apps just got a cleaner path: Google's Gemini 3.8 Live models are live on Vercel AI Gateway, so you can open a realtime audio session over a Gateway WebSocket instead of wiring provider-specific voice infra yourself. The changelog ships two model IDs — `google/gemini-3.8-live` and `google/gemini-3.8-live-extended-thinking` — both aimed at spoken interactions for voice assistants, conversational UIs, and apps that answer in audio.

What changed is the surface area you own. Realtime audio, visual grounding, automatic switching across 97 languages, and background tool calls while the conversation continues are available through the AI SDK's realtime API on AI Gateway. Extended Thinking adds multi-step reasoning that runs in parallel with speech, so the model can acknowledge a request and narrate progress without killing the turn. If you were already minting short-lived tokens and serializing realtime events for other Gateway models, the Gemini Live path fits the same shape.

## What Gemini 3.8 Live actually enables

The primary release is [Gemini 3.8 Live models now available on AI Gateway](https://vercel.com/changelog/gemini-3-8-live-models-now-available-on-ai-gateway) (published 15 Sep 2026). Two concrete model strings matter:

- `google/gemini-3.8-live` — real-time audio, visual grounding, automatic switching across 97 languages, and background tool calls while the conversation continues.
- `google/gemini-3.8-live-extended-thinking` — the same realtime spoken interaction surface, plus multi-step reasoning that runs in parallel with speech so the assistant can acknowledge requests and narrate progress without interrupting the conversation.

That is the whole factual spine. No latency benchmarks, no pricing tables, no "X× faster" claims appear on the changelog — so none belong here either. The production win is architectural: one Gateway credential, one short-lived client token, one WebSocket event loop, and Google's Live models as swappable `model` IDs.

![Vercel changelog: Gemini 3.8 Live models on AI Gateway — https://vercel.com/changelog/gemini-3-8-live-models-now-available-on-ai-gateway — captured 16 Sep 2026](https://cdn.otf-kit.dev/blog/gemini-3-8-live-ai-gateway/competitor-vercel-changelog-20260916.png)

*Source page: [vercel.com/changelog/gemini-3-8-live-models-now-available-on-ai-gateway](https://vercel.com/changelog/gemini-3-8-live-models-now-available-on-ai-gateway), captured 16 Sep 2026.*

## How to start a session today

Install the Gateway provider and a WebSocket client, mint a short-lived token, open the socket, then serialize client events and parse server events through the model adapter. The following is the changelog sample, verified against the primary source — not an invented wrapper:

```bash
pnpm add @ai-sdk/gateway@latest ws
```

```typescript
import { gateway } from '@ai-sdk/gateway';
import WebSocket from 'ws';

const modelId = 'google/gemini-3.8-live';
const { token, url } = await gateway.experimental_realtime.getToken({
  model: modelId,
});

const model = gateway.experimental_realtime(modelId);
const config = model.getWebSocketConfig({ token, url });
const ws = new WebSocket(config.url, config.protocols);

const send = async (
  event: Parameters<typeof model.serializeClientEvent>[0],
) => ws.send(JSON.stringify(await model.serializeClientEvent(event)));

ws.on('open', async () => {
  await send({
    type: 'session-update',
    config: {
      outputModalities: ['audio'],
      outputAudioTranscription: {},
      // Extended Thinking only. Set exactly one of thinkingLevel or thinkingBudget:
      // providerOptions: {
      //   google: { thinkingConfig: { thinkingLevel: 'LOW' } },
      // },
    },
  });
  await send({
    type: 'conversation-item-create',
    item: {
      type: 'text-message',
      role: 'user',
      text: 'Say hello in one sentence.',
    },
  });
});

ws.on('message', (data) => {
  const parsed = model.parseServerEvent(JSON.parse(data.toString()));

  for (const event of Array.isArray(parsed) ? parsed : [parsed]) {
    if (event.type === 'audio-transcript-delta') {
      process.stdout.write(event.delta);
    }
    if (event.type === 'response-done') {
      console.log();
      ws.close();
    }
    if (event.type === 'error') {
      console.error(event.message);
      ws.close();
    }
  }
});

ws.on('close', (code, reason) => {
  if (code !== 1000) {
    console.error(`WebSocket closed (${code}): ${reason.toString()}`);
  }
});
```

Three production habits fall out of that sample immediately:

1. Mint the token on the server. `gateway.experimental_realtime.getToken` exists so browsers never hold long-lived Gateway credentials — the same pattern the [AI SDK realtime docs](https://ai-sdk.dev/docs/ai-sdk-core/realtime) describe for short-lived client secrets.
2. Treat the model adapter as the event contract. `serializeClientEvent` / `parseServerEvent` keep you on the normalized Gateway event surface instead of hand-rolling provider wire formats.
3. Prefer transcript deltas and `response-done` / `error` handlers as your first observability hooks before you invent custom telemetry.

The AI Gateway realtime quickstart documents the same install + token + WebSocket loop for other models; swapping `modelId` to `google/gemini-3.8-live` is the Gemini-specific change from the Sep 15 release.

![Decision beat: Gemini 3.8 Live versus Extended Thinking for realtime voice](https://cdn.otf-kit.dev/blog/gemini-3-8-live-ai-gateway/inbody-01-decision-20260916a.png)

*Pick Live for quick assistants; Extended Thinking when speech must narrate multi-step work.*

## Live vs Extended Thinking: pick the right ID

Use `google/gemini-3.8-live` when the product need is realtime spoken interaction with visual grounding, multilingual auto-switch, and tools that keep running in the background while speech continues. That covers most voice-assistant and conversational-audio surfaces where the user expects low-friction talk-and-act loops.

Reach for `google/gemini-3.8-live-extended-thinking` when the turn needs multi-step reasoning in parallel with speech — acknowledge first, keep talking, reason while the conversation stays open. The changelog's session-update comment is explicit: Extended Thinking config uses exactly one of `thinkingLevel` or `thinkingBudget` under `providerOptions.google.thinkingConfig`. Leave that block commented until you actually need it; wrong config belongs to the Extended Thinking ID, not the base Live ID.

A practical decision rule for product teams: if the user-facing promise is "talk to the app and it sees / hears / tools in the background," start on Live. If the promise is "talk while it plans and narrates a multi-step task," promote that route to Extended Thinking. Same WebSocket bootstrap either way — only the model string and optional thinking config change.

## Visual grounding, languages, and background tools

Three capabilities on `google/gemini-3.8-live` are easy to undersell if you only read "realtime audio":

- **Visual grounding** — the Live model is positioned for interactions that are not audio-only. For builders, that means designing session UX that can attach visual context rather than assuming a pure microphone pipe.
- **97 languages with automatic switching** — language hop is a model capability on the Gateway ID, not a separate translation microservice you have to stand up first.
- **Background tool calls while conversation continues** — tools do not have to freeze the spoken turn. That matters for agent-style voice UIs where a lookup or side effect should not feel like a dead air hang.

None of those are benchmarks. They are product constraints you can design against today because the changelog lists them on the Live model bullet. Pair them with your own auth, rate limits, and tool endpoints — the Gateway session still needs a server that mints tokens and an app that handles tool results safely. Adjacent reading on keeping harness auth on the host boundary: [AI SDK harness auth](/blog/ai-sdk-harness-native-auth).

## What stays yours after the WebSocket opens

AI Gateway is the unified call surface: model routing, usage/cost tracking, retries, failover, and performance optimizations for higher-than-provider uptime, per the changelog wrap. Realtime does not remove the need for an owned application boundary. You still decide:

- which routes may mint `experimental_realtime` tokens
- how `onToolCall` (or your own event handlers) map to app-specific APIs
- what session config is allowed per tenant
- how audio transcripts and tool outputs land in your audit trail

That split is the same one we keep hitting on hosted agent runtimes: the provider or Gateway can own the speech transport; your repo still owns permissions, product state, and the durable UI/agent config. For a related ownership seam on Vercel-hosted agent work, see [OpenAI Agents on Vercel: Queue and Sandbox](/blog/vercel-openai-agents-sandbox). Presence-style realtime (rooms, cursors, sync) is a different problem entirely — our [Supabase realtime mobile presence](/blog/supabase-realtime-mobile-presence-live) post is about honest presence without polling, not voice.

![Architecture handoff: short-lived token, WebSocket bridge, serialize or parse events](https://cdn.otf-kit.dev/blog/gemini-3-8-live-ai-gateway/inbody-02-handoff-20260916a.png)

*Token stays server-side; WebSocket carries serialized realtime events to audio and visual sessions.*

## Complementary layer: kits and agent configs vs voice runtime

OTF's lane here is complementary, not competitive with Gemini Live on AI Gateway. Choosing `google/gemini-3.8-live` (or Extended Thinking) is a voice-runtime decision. Owning the screens, schema, and agent prompt recipes your product ships is a different decision. When the model ID or WebSocket provider rotates again, the durable layer is still the kit and agent config you control in your repo — browse the [templates catalog](/templates) for full-stack kits aimed at that owned-repo outcome. Use Gateway Live for the realtime voice path; keep product UI and agent instructions where you can version them.

Earlier Gemini coverage on this blog ([Gemini 3.7 Flash](/blog/gemini-3-7-flash)) was about small/fast models in coding-agent loops. This post is a different question: shipping realtime spoken interaction with visual grounding through AI Gateway's WebSocket surface. Same company lineage in the title tokens; different production job.

Gemini 3.8 Live on AI Gateway gives production builders a verified, changelog-backed way to ship realtime voice with visual grounding, multilingual auto-switch, background tools, and optional Extended Thinking — using the AI SDK realtime token + WebSocket path you can run today. Wire the sample, pick the model ID that matches the turn shape, and keep token minting plus tool handling inside your own boundary.

## Sources

- [Gemini 3.8 Live models now available on AI Gateway](https://vercel.com/changelog/gemini-3-8-live-models-now-available-on-ai-gateway) (Vercel Changelog, 15 Sep 2026) — primary
- [AI SDK Core: Realtime](https://ai-sdk.dev/docs/ai-sdk-core/realtime) — short-lived token + WebSocket session pattern (HTTP 200 verified)
- [AI Gateway Realtime Quickstart](https://vercel.com/docs/ai-gateway/getting-started/realtime) — Gateway install / token / WebSocket loop (HTTP 200 verified)