Jahanzaib
Back to Blog
Voice AIAI NewsVoice AIAI Agents

Google's Voice Model Won the Benchmark. The One It Tells You to Use Placed Fifth.

A breakdown of what Google shipped in Gemini 3.8 Live, why the headline benchmark belongs to the sibling model, and what actually breaks in your voice agent the day you change the model string.

Jahanzaib Ahmed
·15 min read
Google's Voice Model Won the Benchmark. The One It Tells You to Use Placed Fifth.

Google shipped two voice models on September 15 and led the announcement with a number one ranking. The ranking is real. It just belongs to the model most teams will not run.

Google DeepMind announcement page for Gemini 3.8 Live and 3.8 Live Extended Thinking, dated September 15 2026
The announcement names two models in one headline. The benchmark claims that follow apply to only one of them.

What did Google actually ship on September 15?

Google released two audio to audio models on the Gemini API, and the split between them matters more than the shared version number suggests, because they carry different capability contracts, different function calling rules, and very different task completion scores despite sitting at identical prices. One is called Gemini 3.8 Live. The other is Gemini 3.8 Live Extended Thinking.

Google's own docs describe Gemini 3.8 Live as "the default option for most low-latency voice agent experiences and real-time dialogue without reasoning-induced delays." Extended Thinking is the one recommended "when higher background reasoning is required." Both take text, images, audio and video in. Both emit text and audio. Both have a 131,072 token input limit and a 65,536 token output limit. Both replace gemini-3.1-flash-live-preview, which the docs now label a legacy preview model.

The announcement was written by Tom Ouyang and Malini Jaganathan on behalf of the Gemini Audio team. It lists integrations with Agora, Fishjam, LangChain, LiveKit, Pipecat, Vercel and Vision Agents, plus named customers including Salesforce, Genspark and Lumeris. That partner list is the useful part of the post. If you already run a managed voice platform, your vendor probably has this model behind a config flag already.

Which Gemini 3.8 Live model won the number one spot?

Extended Thinking won it, and the gap between the two siblings is far wider than the gap between Google and the competition it beat, which is the single most useful fact in this launch and the one the announcement never states in words. Google reports 82.6 on the Artificial Analysis Speech to Speech Quality Index. That score belongs to Extended Thinking.

I ran the numbers off Artificial Analysis directly rather than reading Google's chart images, because a chart is a rendering and the leaderboard publishes the values. Here is the top of it, and where the default model actually lands.

ModelSpeech to Speech Quality Indexτ-Voice task completionBig Bench Audio
Gemini 3.8 Live Extended Thinking (High)82.668.6%97.7%
GPT-Live-1 (Astra, medium)81.567.9%90.1%
Grok Voice Think Fast 2.0 High81.356.5%97.2%
GPT-Live-1 (Sol, low)80.159.3%89.0%
Gemini 3.8 Live76.030.1%91.7%
Gemini 3.1 Flash Live Preview (High)71.537.7%96.6%

Extended Thinking beats OpenAI's GPT-Live-1 by 1.1 points on the composite index and by 0.8 points on τ-Voice. Those are thin margins. Meanwhile the base Gemini 3.8 Live sits 6.6 points below its own sibling on the composite and 38.5 points below it on τ-Voice, the benchmark that measures whether a voice agent finishes the task it was given. Extended Thinking completes 2.3 times as many agentic voice tasks as the model Google calls the default.

Artificial Analysis Speech to Speech leaderboard showing Gemini 3.8 Live Extended Thinking at 82.6 and Gemini 3.8 Live at 76.0 in fifth place
The composite index is a weighted average of speech reasoning, agentic performance, arena preference and task success rate. The default model is the fifth bar, not the first.

Google's post is accurate on every number it prints. It attributes 82.6, 68.6% and 97.7% to Extended Thinking, and separately notes that Gemini 3.8 Live took second place in the Speech Agent Arena, which is a preference contest rather than a task completion measure. Nothing is false. The framing simply invites you to carry the flagship number over to the cheaper model, and the data says you can't.

Is Gemini 3.8 Live better than the model it replaces?

On conversation quality yes, on getting work done no, and that mixed result is worth sitting with before you change a model string in production, because the docs recommend the upgrade without qualifying it. Gemini 3.8 Live scores 76.0 on the composite index against 71.5 for Gemini 3.1 Flash Live Preview at high thinking. A clear 4.6 point gain.

Then the other two columns go the wrong way. On τ-Voice, the new default scores 30.1% against the legacy preview model's 37.7%. That's 7.7 points worse at completing agentic tasks. On Big Bench Audio it scores 91.7% against 96.6%, another 4.9 points worse at reasoning. The model that replaces a legacy preview is measurably weaker than that preview on two of three published benchmarks.

There is a coherent reading of this. Gemini 3.8 Live drops thinking depth as a knob to buy latency, so it talks better and reasons less, while Extended Thinking takes the reasoning and pays for it in background compute. Google says as much when it describes the default as delivering dialogue "without reasoning-induced delays." I just would rather read that tradeoff as two numbers than as a phrase in a product page, and Google did not publish the two numbers.

What breaks when you migrate to Gemini 3.8 Live?

Five behaviours change by default and two configuration fields now throw errors, which means a migration that looks like a one line model string swap can ship a voice agent that behaves differently on every call without failing a single test you already have. The docs list all of it. Almost nobody will read that far.

What changesOld behaviourNew behaviour on gemini-3.8-live
thinking_levelControlled thinking depth, defaulted to minimalNot supported. Omit it or the setup is wrong
Function callingSequential only. The model waited for your tool responseAsync NON_BLOCKING is the default. The model keeps talking while your tool runs
Proactive audioOpt inPermanently on. Setting proactive_audio: false returns an error
Affective dialogueOpt in via enable_affective_dialogRemoved from the API. Delete the config
Turn coverageNarrower defaultDefaults to all video activity. Frames are sent unless you stop them
Response modalityText or audioAudio. Turn on output transcription if you need text

The function calling change is the one I would guard hardest. Under the old model the conversation stalled while your tool ran, which is bad product design and excellent accident insurance, because a slow database query simply produced silence. Async by default means the model fills that silence with speech it generated before your tool returned. If your tool was going to say "that account is locked," the agent may have already told the caller something friendlier. Blocking mode still exists on the base model through behavior: BLOCKING, and I would keep it on any tool whose result changes what the agent is allowed to say.

That escape hatch does not exist on Extended Thinking. There, function calling is async only, blocking mode returns a hard error, and function scheduling is unsupported. The more capable model is also the one that removes your ability to serialise a tool call.

Gemini API documentation page for gemini-3.8-live listing caching, code execution, file search and structured outputs as not supported
The capability table is where the real constraints live. Caching, structured outputs, code execution, file search and URL context all read "Not supported."

That capability table deserves its own paragraph. Gemini 3.8 Live supports function calling, search grounding, audio generation and interleaved thinking. It doesn't support context caching, structured outputs, code execution, file search, URL context, Maps grounding or the Batch API. So you have a 131,072 token input window and no way to cache the long system prompt and knowledge block you will inevitably put in it, and no schema guarantee on the way out. Every structured result has to come back through a function call. Plan the architecture around that rather than discovering it in week three.

Why does turnComplete no longer mean the turn is complete?

Because Extended Thinking keeps reasoning after it stops speaking, and Google changed the meaning of an existing protocol field rather than adding a new one, which makes this the single most likely way a working client breaks silently during an upgrade. The docs are direct about it. turnComplete: true no longer indicates an idle session.

Every Live API client I have seen treats turnComplete as the end of the exchange. It's where you re enable the microphone, close the span, write the transcript row, hand off to the next node in your graph. Under asynchronous reasoning the server may still be running background reasoning or waiting on async tool calls when that flag arrives, and more audio frames or tool calls can follow it.

The replacement is a field called interaction_status with two values. IN_PROGRESS means the server is still working and more output may follow. IDLE means it has genuinely finished and is waiting for the user. If you migrate to Extended Thinking without reading that field, your agent will look correct in testing and will start cutting itself off in production the moment a tool call runs slow. This is a client state machine change dressed as a model upgrade.

What does Gemini 3.8 Live cost to run?

Audio input runs $0.005 per minute and audio output runs $0.018 per minute on the paid tier, which prices a realistic support call in single digit cents, and the pricing table charges Gemini 3.8 Live, Extended Thinking and the legacy 3.1 Flash Live Preview at exactly the same rates. Per token, the reasoning model is not more expensive. It just emits more tokens.

Here is what that means per call, assuming the caller speaks for the whole call and the agent talks for half of it.

Call lengthAudio onlyPer 1,000 callsWith video streaming on
3 minutes$0.0420$42.00$0.0480
6 minutes$0.0840$84.00$0.0960
10 minutes$0.1400$140.00$0.1600

Two things are hiding in that last column. Video input bills at $0.002 per minute, and the new default turn coverage sends video frames to the model unless you stop them. For an app that streams video at all, that is a flat 14.3% on top of the audio bill for frames you may never have asked for. Google's own migration note says to send frames only when needed to manage context and cost, which is a polite way of saying the default is the expensive one.

Thinking tokens bill at the output rate, $12.00 per million for audio and $4.50 per million for text. So Extended Thinking's premium shows up as volume on your invoice rather than as a higher rate on the price sheet, and you won't see it until the bill arrives. If you have read my breakdown of what voice agents actually cost per deployment, this is the same pattern that Google's Flash pricing schedule already trained me to check twice.

Gemini Developer API pricing page showing the paid tier advertising access to context caching
The paid tier advertises access to context caching. The Live models are exactly the ones that cannot use it.

Where do Google's own docs contradict each other?

In three places I could verify inside twenty minutes, and each one is the kind of conflict that costs an afternoon of debugging because the page you happened to read first was written for a different model in the same family. The Live API documentation set has not caught up with the 3.8 release.

First, affective dialogue. The 3.8 Live model page says it is removed from the API and that you must delete enable_affective_dialog from your config. The Live API capabilities guide still documents the feature, still shows you the Python and JavaScript to enable it, and notes only that it is unsupported on Gemini 3.1 Flash Live. Read the capabilities guide and you'd reasonably conclude the feature works on 3.8.

Second, proactive audio. Same shape. The model page says it is permanently enabled and that setting it to false returns an error. The capabilities guide presents it as an opt in you configure in the setup message.

Third, language counts. The Live API overview says the model converses in 70 supported languages. The capabilities guide says the Live API supports the following 97 languages and then lists them. Both pages went up under the same product.

There is a fourth one that is not a contradiction so much as a status worth noticing. The Gemini 3.8 Live model page lists its version as stable. The Live API capabilities guide opens with a banner reading "Preview: The Live API is in preview." The model is stable. The API you reach it through is not.

Live API capabilities guide showing a preview banner above the model comparison table for Gemini 3.8 Live and Extended Thinking
A stable model served over an API that still carries a preview banner. Read the model comparison table below it before you pick a version.

What are the hard limits nobody mentions in the launch post?

Session duration is the one that will reshape your architecture, because the Live API caps an audio only session at 15 minutes and an audio plus video session at 2 minutes, and neither number appears anywhere in the announcement. Both caps can be extended with session management. Neither can be ignored.

Two minutes is shorter than most video support interactions. A 30 minute call needs two handoffs on audio and fifteen on audio plus video, and every handoff is a place where conversation state, tool state and transcript continuity can drop. That's not a reason to avoid the model. It's a reason to build session resumption before you build features.

The session context window is 128k tokens for native audio output models and 32k for the rest, which is separate from the 131,072 token model input limit and easier to hit than you would guess on a long call with video frames streaming by default. Client authentication is server to server unless you use ephemeral tokens, so a browser client that talks directly to the Live API needs that token exchange built before launch, not after. I keep seeing teams ship browser based voice interfaces with a long lived key in the client bundle, and it's a bad week when someone notices.

What would I actually ship on this?

I would run Extended Thinking for anything that has to complete a task and the base model for anything that only has to hold a conversation, and I would treat the 30.1% task completion figure as disqualifying for booking, billing or account changes rather than as a number to optimise later. The split is unusually clean for a model launch.

If the agent qualifies a lead, books an appointment, or touches a record, task completion is the product and Extended Thinking is the only one of the two that is competitive. Accept the async only function calling, read interaction_status, and budget for the extra thinking tokens. If the agent answers questions, routes calls, or does the voice equivalent of a FAQ, the base model is cheaper to reason about and its conversational scores are good.

What I would not do is take the 82.6 headline, point it at gemini-3.8-live because that is the name Google's docs call the default, and find out in production that the model finishes fewer than one task in three. That mistake is available to anyone who reads the announcement and not the leaderboard. I was wrong the same way once, on a platform whose engineering post shipped zero latency numbers, and the cost was a rebuild rather than a config change. That one bit me for a fortnight.

The broader pattern holds across this whole category. Vendor benchmark claims are usually true and usually about the configuration you were not going to run. Check which variant earned the number, at which thinking level, on which benchmark. Google published theirs clearly enough that the check took me an afternoon, which is more than most.

If you're working out whether a voice agent belongs anywhere in your operation before you start arguing about model strings, the AI readiness assessment walks the same ground I would cover on a first call, and the agent build pages lay out what production actually requires. The home services voice agents I've shipped failed and succeeded on task completion, never on how natural the voice sounded.

Frequently asked questions

Is Gemini 3.8 Live the number one speech to speech model?

No. Gemini 3.8 Live Extended Thinking holds the number one spot on the Artificial Analysis Speech to Speech Quality Index at 82.6. The base Gemini 3.8 Live scores 76.0, which places it fifth behind two OpenAI GPT-Live-1 configurations and Grok Voice Think Fast 2.0.

Should I migrate from gemini-3.1-flash-live-preview?

Migrate, but pick the target deliberately. Gemini 3.8 Live improves on the composite index by 4.6 points while scoring 7.7 points lower on agentic task completion and 4.9 points lower on Big Bench Audio than the preview model. If your agent completes tasks, Extended Thinking is the upgrade. If it holds conversations, the base model is.

What does Gemini 3.8 Live cost?

On the paid tier, audio input is $3.00 per million tokens or $0.005 per minute, and audio output is $12.00 per million tokens or $0.018 per minute. Text input is $0.75 and text output $4.50 per million. Gemini 3.8 Live, Extended Thinking and 3.1 Flash Live Preview are all charged at the same rates.

Does Gemini 3.8 Live support structured outputs?

No. Structured outputs, context caching, code execution, file search, URL context, Maps grounding and the Batch API are all listed as unsupported on both 3.8 Live models. Structured data has to come back through function calling, and a long system prompt cannot be cached.

Why does my Live API client stop early after upgrading?

Most likely you are still treating turnComplete: true as the end of the turn. On Extended Thinking that flag no longer means the session is idle, because background reasoning and async tool calls can continue afterwards. Read the interaction_status field and wait for IDLE.

How long can a Gemini Live API session run?

Audio only sessions are capped at 15 minutes and audio plus video sessions at 2 minutes. Both can be extended using the session management techniques in the Live API capabilities guide, but the caps apply by default and are not mentioned in the launch announcement.

Sources: Gemini 3.8 Live Extended Thinking scored 82.6 on the Speech to Speech Quality Index, 68.6% on τ-Voice and 97.7% on Big Bench Audio, per Google DeepMind (September 15, 2026). Per model scores including the base model's 76.0 and 30.1% were read from Artificial Analysis Speech to Speech Leaderboard (September 16, 2026). Capability tables, the 131,072 token limit and migration notes come from Gemini API model docs and Extended Thinking docs. The 15 minute and 2 minute session caps and the model comparison table come from the Live API capabilities guide. Rates of $0.005 and $0.018 per minute are from Gemini Developer API pricing.
Feed to Claude or ChatGPT