How to Choose an LLM Router for AI Agents

An LLM router decides what handles each request. In an agent that means the model, the tool, the subagent (including Claude Code subagents) and what to do after a failure. Rules, a trained classifier, a decision model like Jev (TypeSafe AI's model that returns typed answers instead of text) or a frontier LLM can all do the job. I've priced each, timed two in recorded runs, and noted where each breaks. This guide covers routing inside your agent, not gateways such as OpenRouter or LiteLLM.

Before you start

• Steps 1 to 4 are paper exercises: you list decisions, pick routers and set thresholds in a spreadsheet, and each step ends with a check.
• Step 5 uses Claude Code and Python 3.10 or newer on macOS or Linux. Run claude --version first: the cache line in /usage needs v2.1.251 or later, and /skill-doctor needs v2.1.252.
• A decision model like Jev needs an API key from TypeSafe, which may not be open to everyone yet. Step 5 also has a rules version and a Haiku version.

Step 1: List the decisions your agent makes

Many LLM router guides focus on one decision: send a cheap question to a cheap model and a hard one to a strong model. An agent makes many more, at every step of the loop.

• Tool choice. Which of 20 tools fits this step.
• Subagent choice. Which subagent or skill takes the request.
• Model tier. A file lookup doesn't need the model that redesigns a service.
• Failure handling. After a failed call, whether to retry, fix the argument or stop and ask a human.

Open your agent's code or a session log and list every place it picks from a short list, with the options and what a wrong pick costs. A wrong tool choice might waste a call, while a wrong action on a destructive tool might lose data. The check: you end with a short table of decisions, options and wrong pick costs, which Steps 2 and 3 use.

None of these is a writing problem. The answer is a label from a short list, not a paragraph. I borrow the framing in my Jev vs Laya video, which takes Daniel Kahneman's split between slow deliberate thinking (System 2) and fast automatic judgment (System 1). That's my framing of Kahneman, not his claim about software. Asking a frontier model to reason its way to a label is overkill. What Is Jev AI covers the decision model used as the example below.

Step 2: Pick an LLM router type for each decision

Cost per 1,000 calls is arithmetic on list prices and a token count from my run. Speed figures are my recorded runs from October 2026, unless marked as not measured.

First the numbers. Latency and cost per 1,000 routing calls, with Jev's row highlighted because it is the one that hands back a calibrated confidence number.

Now the choice itself. When does each one fit, and where does each one break?

The 55 ms local figure and the 340 ms figure with the network aren't like for like, so don't read them as a ranking of the two models. You can ask a frontier LLM to state a confidence, but that number is text it generated, not a calibrated probability, and I haven't tested how well it tracks accuracy. So I don't gate on it.

The cost column uses the 315 input tokens per call that my 16 message run averaged (5,044 tokens over 16 calls), plus 12 output tokens for the LLM rows, which is my assumption. The LLM range runs from Haiku 5.5 at the low end to Opus 5.5 at the high end, before your router prompt. Jev charges $0.042 per million input tokens with free output, so a call costs about $0.0000132. The pricing page lists Haiku 5.5 at $0.10 input and $0.50 output per million, and Opus 5.5 at $4 and $20. On these assumptions Jev and a Haiku router are roughly two to three times apart, and both round to nothing. Claude's tokenizer may count more tokens than Jev's for the same text (the pricing page says newer Claude models produce about 30% more tokens than earlier ones, and I don't know how Jev's tokenizer compares), so treat the LLM rows as rough. The real difference is whether you get a probability back.

Published evidence for trained routers is real but narrow. The LMSYS team's RouteLLM reports cost cuts of over 85% on MT Bench, 45% on MMLU and 35% on GSM8K against GPT-4 alone, while keeping 95% of GPT-4's performance. Those are chat benchmarks and a model choice, not agent decisions. TypeSafe's claims in my video, 20 to 200 times faster, 40 to 400 times cheaper and 70 to 500 ms end to end, are vendor claims, as is its 97% score on a 49 task classification benchmark. I measured the 340 ms median once. I'm not promising a speedup, and whether any router helps depends on your traffic. For the business side of model routing, see my post on Stripe's OpenRouter deal.

A starting rule: use rules where phrasing is predictable, a decision model or classifier where it varies, and a frontier model for what they escalate. The check: each row of your Step 1 table now names a router.

Step 3: Set confidence lanes and test them

A decision model only helps if you gate on its confidence. TypeSafe's confidence docs suggest a starting pattern of three ranges: act automatically on high confidence, proceed with caution on medium, and don't act on low, routing to a human, a clarifying question or another system. They add that thresholds scale with risk: a wrong guess about a destructive action costs more than one about a read only action. My lanes below gate a yes or no probability instead, where the probability itself carries the uncertainty: act above 0.90, discard below 0.10, and send the band between to the slow lane, where a frontier model or a person takes a proper look.

To set yours, pick a high and a low threshold, run 15 to 20 real examples through the router and count how many land in each lane. The check: the slow lane should hold the cases you'd hesitate over. My worked example is the 16 message inbox test from my video, recorded in October 2026. Each message got one question: is this a genuine request from a real customer that a person should answer?

Jev sent 7 messages to the inbox, 6 to trash and 3 to the slow lane, at a 340 ms median including the network and 5,044 input tokens in total, about $0.00021 for the batch. The slow lane held a message that just said "Hello", an Instagram followers seller, and a real report that single sign on was down for a whole team, scored at about 0.79. Jev didn't trash the outage report, so a person or a bigger model reads it next. A real Stripe webhook report scored exactly 0.90 and only just made the inbox, which is why you tune thresholds on your own traffic.

I ran the same 16 messages through Laya locally, at about 55 ms per decision, and with that question it was confidently wrong. It scored the $1,000 gift card scam 0.98 genuine, and the single sign on outage 0.01, which sends a real customer to trash. Then I asked a different question, "is this spam?". With that wording the same model flagged most of the junk, though a prince scam still got only 0.19. The first question failed; the second mostly worked. A confidence number only helps if you've checked it on your own question wording, which is why I'd shadow test any router before letting it act.

Step 4: Route once at the start to protect the cache

Route at the start of a session, because the prompt cache punishes switching. The Claude Code prompt caching docs say each model has its own cache, and switching models means the next request reads the entire conversation with no cache hits, even if the content is identical.

The docs list several ways to switch: /model, opusplan toggling between plan and execution, automatic model fallback, and a skill whose frontmatter names a different model. Turning on fast mode deep into a session also causes a miss, and a PreModelSwitch hook exists if you want to intercept switches.

On Opus 5.5 and Sonnet 5.5 a cache hit costs 0.05 times the base input price and a 5 minute cache write costs 1.25 times, so a miss costs 25 times a hit. On Haiku 5.5 it's 12.5 times and on Fable 5.1, 50 times.

Here is what one switch costs on a 150,000 token conversation, computed from the multipliers above.

That's list price arithmetic, not a measured bill, and one hour writes cost more. The check: no step in your design switches the main model mid conversation.

Step 5: Route Claude Code subagents and skills

I checked the Claude Code facts below against the docs on October 11, 2026. A subagent file takes a model field (sonnet, opus, haiku, fable, a full model ID or inherit), and Claude Code resolves a subagent's model in order: the per invocation parameter, the file's frontmatter, the CLAUDE_CODE_SUBAGENT_MODEL environment variable, then the main conversation's model. A subagent runs in its own context window, so the parent's cache is unaffected. Its first request doesn't read the parent's cache, so every new subagent run starts cold.

The simplest router needs no code: a project subagent named Explore overrides the built in one and keeps its own model, so .claude/agents/Explore.md with model: haiku sends file search to Haiku. Skills are the other menu. The skills docs say that with many skills Claude Code drops descriptions to fit a budget of 1% of the context window, so trim that menu first: /doctor and /skill-doctor show its cost.

Jev vs Laya: The Free AI Fell for a $1,000 Scam

If you still want a hook, here's a short design sketch. It routes to three subagents: Explore (the Haiku override above), plus fixer and architect, which you create in .claude/agents/.

# .claude/agents/fixer.md
---
name: fixer
description: Makes one focused change: a bug fix, a small feature, a test update.
tools: Read, Glob, Grep, Edit, Write, Bash
model: sonnet
---
Make the smallest change that solves the request, then run the relevant tests.

# .claude/agents/architect.md
---
name: architect
description: Designs or restructures across modules or services before code is written.
tools: Read, Glob, Grep
model: opus
---
Read the code, propose a plan with trade offs and the files involved. Do not edit.

A UserPromptSubmit hook can return additionalContext, which Claude reads as a system reminder. It's advice: the subagents docs say naming a subagent lets Claude decide, while an @ mention guarantees the run. I ran this script locally outside Claude Code, feeding it sample JSON, and it printed the expected hint for "where is the bug in retry" (Explore), "find the bug and fix it" (fixer) and a redesign request (architect), and nothing for "prefix the file", "debug this" or chat. I haven't run it inside a live Claude Code session.

#!/usr/bin/env python3
# .claude/hooks/route.py  (design sketch; rules router, no API key)
import json, pathlib, re, sys, time

# First match wins. Word boundaries stop "prefix" matching "fix" and "debug" matching "bug".
RULES = [("architect", r"\b(redesign|architecture|restructure)\b"),
         ("Explore", r"\b(where is|which file|usages of)\b"),
         ("fixer", r"\b(fix|bug|failing test|add a)\b"),
         ("Explore", r"\bfind\b")]
LOG = pathlib.Path.home() / ".claude" / "router-log.jsonl"
SHADOW = False  # True: log only, send no hint

def decide(prompt):
    text = prompt.lower()
    for door, pattern in RULES:
        if re.search(pattern, text):
            return door
    return None

def log(record):
    with LOG.open("a") as f:
        f.write(json.dumps(record) + "\n")

def main():
    event = json.load(sys.stdin)
    kind = event.get("hook_event_name")
    if kind == "UserPromptSubmit":
        try:
            door = decide(event.get("prompt", ""))
        except Exception:
            door = None  # fail open: no hint, Claude carries on
        log({"t": time.time(), "session": event.get("session_id"), "router": door})
        if door and not SHADOW:
            hint = f"Router hint: this looks like a job for the {door} subagent."
            print(json.dumps({"hookSpecificOutput": {"hookEventName": kind, "additionalContext": hint}}))
    elif kind == "SubagentStart":
        log({"t": time.time(), "session": event.get("session_id"), "subagent": event.get("agent_type")})

main()

To use a model instead of rules, replace decide. I haven't run either version below, because both need a key. The Jev version follows the request shape in the TypeSafe docs, and every prompt would leave your machine for a third party's API. TypeSafe also says Jev isn't a drop in replacement for the model behind Claude Code, so it sits beside it as this routing step.

# Jev: needs TYPESAFE_API_KEY (not run by me)
import http.client, json, os
QUESTION = {"type": "choice", "instructions": "Which subagent should handle this coding request?",
            "criteria": {"Explore": "Locate files, symbols or usages. Nothing changes.",
                         "fixer": "One focused change: a bug fix, small feature or test update.",
                         "architect": "Design or restructure across modules before code is written.",
                         "none": "A question, chat, or anything needing no specialist."}}

def decide(prompt):
    body = json.dumps({"state": prompt[:6000], "model": "jev-latest", "questions": {"q": QUESTION}})
    conn = http.client.HTTPSConnection("api.typesafe.ai", timeout=3)
    conn.request("POST", "/v1/systemone", body=body, headers={
        "Authorization": "Bearer " + os.environ["TYPESAFE_API_KEY"], "Content-Type": "application/json"})
    answer = json.loads(conn.getresponse().read())["answers"]["q"]
    ok = answer["confidence"] >= 0.60 and answer["choice"] != "none"  # untuned; TypeSafe's own examples use 0.5 to 0.6
    return answer["choice"] if ok else None

# Haiku 5.5 through the Messages API: needs ANTHROPIC_API_KEY (not run by me)
import http.client, json, os

def decide(prompt):
    body = json.dumps({"model": "claude-haiku-5-5", "max_tokens": 10,
        "system": "Reply with one word, the best worker for this coding request: Explore, fixer, architect or none.",
        "messages": [{"role": "user", "content": prompt[:6000]}]})
    conn = http.client.HTTPSConnection("api.anthropic.com", timeout=3)
    conn.request("POST", "/v1/messages", body=body, headers={
        "x-api-key": os.environ["ANTHROPIC_API_KEY"], "anthropic-version": "2023-06-01",
        "content-type": "application/json"})
    text = json.loads(conn.getresponse().read())["content"][0]["text"].strip()
    return text if text in ("Explore", "fixer", "architect") else None

Save the script as .claude/hooks/route.py; the hook runs it with python3, so it needs no chmod. Test it with sample JSON, then register it under both events in .claude/settings.json (or ~/.claude/settings.json for every project; see the hooks docs) with a short timeout, and check it with /hooks. Create the folder first with mkdir -p .claude/hooks.

echo '{"hook_event_name":"UserPromptSubmit","session_id":"test","prompt":"where is the bug in retry"}' | python3 .claude/hooks/route.py

{
  "hooks": {
    "UserPromptSubmit": [
      { "hooks": [ { "type": "command", "command": "python3",
          "args": ["${CLAUDE_PROJECT_DIR}/.claude/hooks/route.py"], "timeout": 5 } ] }
    ],
    "SubagentStart": [
      { "hooks": [ { "type": "command", "command": "python3",
          "args": ["${CLAUDE_PROJECT_DIR}/.claude/hooks/route.py"], "timeout": 5 } ] }
    ]
  }
}

Step 6: Measure whether the router helps you

Run the router in shadow mode first. In the sketch, set SHADOW = True and it only logs what the router would pick, without sending Claude a hint, which would otherwise inflate agreement. It also logs which subagent Claude Code actually started. After a week, compare the two per session.

• Agreement rate. How often the router's choice matches what the big model chose. If it's low, either the router is weak or your option descriptions are vague. Read the disagreements; sometimes the router is right.
• Slow lane share. With a confidence gate, what fraction of decisions fall in the middle band. A router that escalates nearly everything saves nothing, and one that escalates almost nothing may be overconfident.
• Cost and cache. Compare /usage before and after, including the Prompt cache (main) miss count, since a router that causes model switches can cost more than it saves.
• Time. Measure it on your own sessions before you believe any claim, mine included.

A later step is training: log the big model's routing choices for a few weeks, then train a small local model such as Laya on those labels. That's the two brain AI agent idea from my video, where the fast brain learns from the slow one. It's my proposed approach, untested. For where routing sits in a larger design, see AI agent architecture patterns, and model your volumes in the AI agent cost calculator.

When not to route

With a handful of subagents and modest volume, skip the router. The Explore override to Haiku is one file and needs no key, and Claude's own matching handles a short menu. A hook also adds a call to every prompt. Add routing when the menu is large or you want a measured confidence on each decision, and confirm it with the Step 6 numbers.

Troubleshooting

These are likely causes from the docs and my recorded runs, not live diagnoses.

• The router is confident and wrong. Reword the question and rerun your logged prompts. In my Laya run, the "genuine request" wording failed on a scam and an outage, while the "is this spam" wording flagged most of the junk.
• Nearly everything lands in the slow lane. Your options overlap, or the question is too abstract. Make each option an exact condition.
• Claude ignores the hint. That's expected, since a hint isn't an instruction. @ mention the subagent when you know the one you want.
• A subagent doesn't appear. The subagents docs have a section on files Claude Code skips, such as a missing name or description.

The same 16 message test, run on the free local model, is in Jev vs Laya on 16 test messages.

Frequently asked questions

What is an LLM router?

An LLM router is a layer between your application and one or more models that decides which model handles each request. In an agent the same idea extends to tools, subagents and failure handling. It can be rules, a small classifier, a decision model or another LLM, and the trade is routing cost and latency against saved work downstream.

What is the difference between LLM routing and an AI agent router?

LLM routing usually means choosing a model per request. Agent routing also chooses among tools, subagents or skills, and decides what happens after a failed call. Agent routing adds state and side effects, so a wrong choice can do more than give a weaker answer.

Can I route Claude Code subagents to different models?

Yes. Each subagent file has a model field that accepts sonnet, opus, haiku, fable, a full model ID or inherit. Putting model: haiku in a project subagent named Explore overrides the built in search subagent. A hook can only hint at a subagent, while an @ mention guarantees the run.

Does routing between models break the prompt cache?

Switching the main model does. Claude Code's docs say each model has its own cache, and the request after a switch reads the whole conversation with no cache hits. A subagent on another model leaves the parent's cache alone, but its own first request starts cold with a five minute cache lifetime.

Can Jev replace the model behind Claude Code?

No. TypeSafe's docs say Jev isn't a drop in replacement for the model behind Claude Code or similar tools, because it returns typed answers and can't write text or code. Access was gated behind an early access waitlist at launch, so confirm current availability with TypeSafe first.

If you'd rather have routing designed and measured for your own agents, that's what I do: see how I build AI agents.