---
title: "How to Choose an LLM Router for AI Agents"
description: "An LLM router picks the model, tool or subagent for each request. Compare rules, classifiers, Jev style decision models and LLMs, with confidence lanes, cache math and a Claude Code sketch."
author: "Jahanzaib Ahmed"
date: 2026-10-10
category: "ai-agents"
readingTime: "15 min read"
tags: ["llm router", "ai agent routing", "claude code", "jev", "ai-agents"]
canonical: https://www.jahanzaib.ai/blog/llm-router-ai-agent-routing
source: https://www.jahanzaib.ai
---
# How to Choose an LLM Router for AI Agents

An LLM router decides what handles each request. In an agent that means the model, the tool, the subagent (including Claude Code subagents) and what to do after a failure. Rules, a trained classifier, a decision model like Jev (TypeSafe AI's model that returns typed answers instead of text) or a frontier LLM can all do the job. I've priced each, timed two in recorded runs, and noted where each breaks. This guide covers routing inside your agent, not gateways such as OpenRouter or LiteLLM.

## Before you start

-   Steps 1 to 4 are paper exercises: you list decisions, pick routers and set thresholds in a spreadsheet, and each step ends with a check.
-   Step 5 uses Claude Code and Python 3.10 or newer on macOS or Linux. Run `claude --version` first: the cache line in /usage needs v2.1.251 or later, and /skill-doctor needs v2.1.252.
-   A decision model like Jev needs an API key from TypeSafe, which may not be open to everyone yet. Step 5 also has a rules version and a Haiku version.

## Step 1: List the decisions your agent makes

Many LLM router guides focus on one decision: send a cheap question to a cheap model and a hard one to a strong model. An agent makes many more, at every step of the loop.

-   **Tool choice.** Which of 20 tools fits this step.
-   **Subagent choice.** Which [subagent](https://www.jahanzaib.ai/glossary/sub-agent) or skill takes the request.
-   **Model tier.** A file lookup doesn't need the model that redesigns a service.
-   **Failure handling.** After a failed call, whether to retry, fix the argument or stop and ask a human.

Open your agent's code or a session log and list every place it picks from a short list, with the options and what a wrong pick costs. A wrong tool choice might waste a call, while a wrong action on a destructive tool might lose data. The check: you end with a short table of decisions, options and wrong pick costs, which Steps 2 and 3 use.

None of these is a writing problem. The answer is a label from a short list, not a paragraph. I borrow the framing in my [Jev vs Laya video](https://www.youtube.com/watch?v=0ytFeuAv-FA), which takes Daniel Kahneman's split between slow deliberate thinking (System 2) and fast automatic judgment (System 1). That's my framing of Kahneman, not his claim about software. Asking a frontier model to reason its way to a label is overkill. [What Is Jev AI](https://www.jahanzaib.ai/blog/what-is-jev-ai) covers the decision model used as the example below.

![Diagram of an agent loop with four routing decisions around it: tool, subagent, model tier and what to do after a failure](https://cdn.sanity.io/images/qajb7q5q/production/5d9723634d211f6efbad2b4432c03ba5577f4037-1376x768.png?w=1200&q=75&auto=format&fit=max)

_One agent loop makes four kinds of routing decision, and each one can use a different router._

## Step 2: Pick an LLM router type for each decision

Cost per 1,000 calls is arithmetic on list prices and a token count from my run. Speed figures are my recorded runs from October 2026, unless marked as not measured.

![Staircase of four LLM router approaches from rules to a trained classifier, a decision model and a frontier LLM, each with its plain cost](https://cdn.sanity.io/images/qajb7q5q/production/41db4ba8c1540189151a930c736bc896d2156bd7-1376x768.png?w=1200&q=75&auto=format&fit=max)

_Each step up the ladder buys flexibility and costs you something different: writing the cases, collecting examples, accepting text only input, or paying for slow calls._

First the numbers. Latency and cost per 1,000 routing calls, with Jev's row highlighted because it is the one that hands back a calibrated confidence number.

| Approach | Latency (ms) | Cost per 1,000 calls ($) | Confidence number |
| --- | --- | --- | --- |
| Rules or keywords | Instant | 0 | None |
| Trained classifier, as in RouteLLM | Not measured | Not priced | Depends on training |
| Local model, such as Laya | 55 | 0 | Probabilities, not safe untuned |
| Hosted model, such as Jev | 340 | 0.013 | Yes |
| Frontier LLM as the router | Not measured | 0.04 to 1.50 | Only a number it states |

Now the choice itself. When does each one fit, and where does each one break?

| Approach | Pick it when | Breaks when |
| --- | --- | --- |
| Rules or keywords | Phrasing is predictable | Someone words it in a way you didn't list |
| Trained classifier | You have labelled examples | You have no labelled examples |
| Local model (Laya, about 420M parameters) | Free and private matter, and you can tune it | The question is worded unlike its training |
| Hosted model (Jev) | You need a confidence gate and can send text out | Text only, reads questions literally, access may be limited |
| Frontier LLM | The cheaper routers escalate to it | You must parse its output, and tokens go on a label |

The 55 ms local figure and the 340 ms figure with the network aren't like for like, so don't read them as a ranking of the two models. You can ask a frontier LLM to state a confidence, but that number is text it generated, not a calibrated probability, and I haven't tested how well it tracks accuracy. So I don't gate on it.

The cost column uses the 315 input tokens per call that my 16 message run averaged (5,044 tokens over 16 calls), plus 12 output tokens for the LLM rows, which is my assumption. The LLM range runs from Haiku 5.5 at the low end to Opus 5.5 at the high end, before your router prompt. Jev charges [$0.042 per million input tokens](https://docs.typesafe.ai/models) with free output, so a call costs about $0.0000132. The [pricing page](https://docs.claude.com/en/docs/about-claude/pricing) lists Haiku 5.5 at $0.10 input and $0.50 output per million, and Opus 5.5 at $4 and $20. On these assumptions Jev and a Haiku router are roughly two to three times apart, and both round to nothing. Claude's tokenizer may count more tokens than Jev's for the same text (the pricing page says newer Claude models produce about 30% more tokens than earlier ones, and I don't know how Jev's tokenizer compares), so treat the LLM rows as rough. The real difference is whether you get a probability back.

Published evidence for trained routers is real but narrow. The LMSYS team's [RouteLLM](https://lmsys.org/blog/2024-07-01-routellm/) reports cost cuts of over 85% on MT Bench, 45% on MMLU and 35% on GSM8K against GPT-4 alone, while keeping 95% of GPT-4's performance. Those are chat benchmarks and a model choice, not agent decisions. TypeSafe's claims in my video, 20 to 200 times faster, 40 to 400 times cheaper and 70 to 500 ms end to end, are vendor claims, as is its 97% score on a 49 task classification benchmark. I measured the 340 ms median once. I'm not promising a speedup, and whether any router helps depends on your traffic. For the business side of model routing, see my post on [Stripe's OpenRouter deal](https://www.jahanzaib.ai/blog/stripe-openrouter-acquisition-llm-routing).

A starting rule: use rules where phrasing is predictable, a decision model or classifier where it varies, and a frontier model for what they escalate. The check: each row of your Step 1 table now names a router.

## Step 3: Set confidence lanes and test them

A decision model only helps if you gate on its confidence. TypeSafe's [confidence docs](https://docs.typesafe.ai/confidence) suggest a starting pattern of three ranges: act automatically on high confidence, proceed with caution on medium, and don't act on low, routing to a human, a clarifying question or another system. They add that thresholds scale with risk: a wrong guess about a destructive action costs more than one about a read only action. My lanes below gate a yes or no probability instead, where the probability itself carries the uncertainty: act above 0.90, discard below 0.10, and send the band between to the slow lane, where a frontier model or a person takes a proper look.

To set yours, pick a high and a low threshold, run 15 to 20 real examples through the router and count how many land in each lane. The check: the slow lane should hold the cases you'd hesitate over. My worked example is the 16 message inbox test from my video, recorded in October 2026. Each message got one question: is this a genuine request from a real customer that a person should answer?

![My recorded run: Jev sent 7 messages to the inbox, 3 to the slow lane and 6 to trash at a 340 ms median including the network round trip](https://cdn.sanity.io/images/qajb7q5q/production/9fed50639795bbc2f3d430940e26823e2511b3ad-1600x900.jpg?w=1200&q=75&auto=format&fit=max)

_My recorded run, October 2026: with the 0.90 and 0.10 thresholds, 7 messages went to the inbox, 3 to the slow lane and 6 to trash, at a 340 ms median including the network._

Jev sent 7 messages to the inbox, 6 to trash and 3 to the slow lane, at a 340 ms median including the network and 5,044 input tokens in total, about $0.00021 for the batch. The slow lane held a message that just said "Hello", an Instagram followers seller, and a real report that single sign on was down for a whole team, scored at about 0.79. Jev didn't trash the outage report, so a person or a bigger model reads it next. A real Stripe webhook report scored exactly 0.90 and only just made the inbox, which is why you tune thresholds on your own traffic.

| Lane | Confidence rule | Messages (of 16) |
| --- | --- | --- |
| Inbox | 0.90 or higher | 7 |
| Slow lane | Between 0.10 and 0.90 | 3 |
| Trash | 0.10 or lower | 6 |

I ran the same 16 messages through Laya locally, at about 55 ms per decision, and with that question it was confidently wrong. It scored the $1,000 gift card scam 0.98 genuine, and the single sign on outage 0.01, which sends a real customer to trash. Then I asked a different question, "is this spam?". With that wording the same model flagged most of the junk, though a prince scam still got only 0.19. The first question failed; the second mostly worked. A confidence number only helps if you've checked it on your own question wording, which is why I'd shadow test any router before letting it act.

| Message | Question asked | Score (0 to 1) | Lane |
| --- | --- | --- | --- |
| Gift card scam | Genuine request? | 0.98 | Inbox (wrong) |
| Gift card scam | Is this spam? | 0.90 | Slow lane |
| SSO outage | Genuine request? | 0.01 | Trash (wrong) |
| SSO outage | Is this spam? | 0.00 | Inbox |
| Prince scam | Is this spam? | 0.19 | Slow lane |
![Terminal output of the free Laya model rerun on the same 16 messages with the question is this message spam, with scores, lanes and latency per row](https://cdn.sanity.io/images/qajb7q5q/production/dfada818d7b15b6aa0a46c9a29a8c6886491b78b-1600x900.jpg?w=1200&q=75&auto=format&fit=max)

_My recorded run, October 2026: the same Laya model and the same 16 messages, asked "is this spam?" instead, with a median of 43 ms per decision on this run._

## Step 4: Route once at the start to protect the cache

Route at the start of a session, because the prompt cache punishes switching. The [Claude Code prompt caching docs](https://docs.claude.com/en/docs/claude-code/prompt-caching) say each model has its own cache, and switching models means the next request reads the entire conversation with no cache hits, even if the content is identical.

The docs list several ways to switch: `/model`, `opusplan` toggling between plan and execution, automatic model fallback, and a skill whose frontmatter names a different model. Turning on fast mode deep into a session also causes a miss, and a `PreModelSwitch` hook exists if you want to intercept switches.

On Opus 5.5 and Sonnet 5.5 a cache hit costs 0.05 times the base input price and a 5 minute cache write costs 1.25 times, so a miss costs 25 times a hit. On Haiku 5.5 it's 12.5 times and on Fable 5.1, 50 times.

Here is what one switch costs on a 150,000 token conversation, computed from the multipliers above.

| Model | Warm cache read ($) | After a switch ($) | Times more |
| --- | --- | --- | --- |
| Opus 5.5 | 0.03 | 0.75 | 25 |
| Sonnet 5.5 | 0.015 | 0.375 | 25 |

That's list price arithmetic, not a measured bill, and one hour writes cost more. The check: no step in your design switches the main model mid conversation.

![Two timelines comparing an LLM router that routes once at session start with one that switches models later and loses the prompt cache](https://cdn.sanity.io/images/qajb7q5q/production/4040b361e91b18bf4d1b6e04558357f6c3411901-1376x768.png?w=1200&q=75&auto=format&fit=max)

_Route at the start: a model switch deep into a long session throws the cache away, and the next request pays to rebuild it._

## Step 5: Route Claude Code subagents and skills

I checked the Claude Code facts below against the docs on October 11, 2026. A [subagent](https://docs.claude.com/en/docs/claude-code/sub-agents) file takes a `model` field (`sonnet`, `opus`, `haiku`, `fable`, a full model ID or `inherit`), and Claude Code resolves a subagent's model in order: the per invocation parameter, the file's frontmatter, the `CLAUDE_CODE_SUBAGENT_MODEL` environment variable, then the main conversation's model. A subagent runs in its own context window, so the parent's cache is unaffected. Its first request doesn't read the parent's cache, so every new subagent run starts cold.

The simplest router needs no code: a project subagent named `Explore` overrides the built in one and keeps its own model, so `.claude/agents/Explore.md` with `model: haiku` sends file search to Haiku. Skills are the other menu. The [skills docs](https://docs.claude.com/en/docs/claude-code/skills) say that with many skills Claude Code drops descriptions to fit a budget of 1% of the context window, so trim that menu first: /doctor and /skill-doctor show its cost.

[Video: Jev vs Laya: The Free AI Fell for a $1,000 Scam](https://www.youtube.com/watch?v=0ytFeuAv-FA&t=372s)

_Chapter 6:12 to 7:59. I list the 68 skills my Claude Code setup offers, then put a front desk router ahead of them so only one subagent loads._
![Diagram of a router sending one request to the fixer subagent while Explore, architect and no specialist stay dark](https://cdn.sanity.io/images/qajb7q5q/production/66ebab4f3cd0561773f87d8787ccd3ca6697fe5d-1376x768.png?w=1200&q=75&auto=format&fit=max)

_The front desk picks one door for the request and only that subagent loads its instructions; in Claude Code the hook stays a hint, and only an @ mention guarantees the run._

If you still want a hook, here's a short design sketch. It routes to three subagents: `Explore` (the Haiku override above), plus `fixer` and `architect`, which you create in `.claude/agents/`.

```
# .claude/agents/fixer.md
---
name: fixer
description: Makes one focused change: a bug fix, a small feature, a test update.
tools: Read, Glob, Grep, Edit, Write, Bash
model: sonnet
---
Make the smallest change that solves the request, then run the relevant tests.

# .claude/agents/architect.md
---
name: architect
description: Designs or restructures across modules or services before code is written.
tools: Read, Glob, Grep
model: opus
---
Read the code, propose a plan with trade offs and the files involved. Do not edit.
```

A `UserPromptSubmit` hook can return `additionalContext`, which Claude reads as a system reminder. It's advice: the subagents docs say naming a subagent lets Claude decide, while an @ mention guarantees the run. I ran this script locally outside Claude Code, feeding it sample JSON, and it printed the expected hint for "where is the bug in retry" (Explore), "find the bug and fix it" (fixer) and a redesign request (architect), and nothing for "prefix the file", "debug this" or chat. I haven't run it inside a live Claude Code session.

```
#!/usr/bin/env python3
# .claude/hooks/route.py  (design sketch; rules router, no API key)
import json, pathlib, re, sys, time

# First match wins. Word boundaries stop "prefix" matching "fix" and "debug" matching "bug".
RULES = [("architect", r"\b(redesign|architecture|restructure)\b"),
         ("Explore", r"\b(where is|which file|usages of)\b"),
         ("fixer", r"\b(fix|bug|failing test|add a)\b"),
         ("Explore", r"\bfind\b")]
LOG = pathlib.Path.home() / ".claude" / "router-log.jsonl"
SHADOW = False  # True: log only, send no hint

def decide(prompt):
    text = prompt.lower()
    for door, pattern in RULES:
        if re.search(pattern, text):
            return door
    return None

def log(record):
    with LOG.open("a") as f:
        f.write(json.dumps(record) + "\n")

def main():
    event = json.load(sys.stdin)
    kind = event.get("hook_event_name")
    if kind == "UserPromptSubmit":
        try:
            door = decide(event.get("prompt", ""))
        except Exception:
            door = None  # fail open: no hint, Claude carries on
        log({"t": time.time(), "session": event.get("session_id"), "router": door})
        if door and not SHADOW:
            hint = f"Router hint: this looks like a job for the {door} subagent."
            print(json.dumps({"hookSpecificOutput": {"hookEventName": kind, "additionalContext": hint}}))
    elif kind == "SubagentStart":
        log({"t": time.time(), "session": event.get("session_id"), "subagent": event.get("agent_type")})

main()
```

To use a model instead of rules, replace `decide`. I haven't run either version below, because both need a key. The Jev version follows the request shape in the [TypeSafe docs](https://docs.typesafe.ai/introduction/coding-agents), and every prompt would leave your machine for a third party's API. TypeSafe also says Jev isn't a drop in replacement for the model behind Claude Code, so it sits beside it as this routing step.

```
# Jev: needs TYPESAFE_API_KEY (not run by me)
import http.client, json, os
QUESTION = {"type": "choice", "instructions": "Which subagent should handle this coding request?",
            "criteria": {"Explore": "Locate files, symbols or usages. Nothing changes.",
                         "fixer": "One focused change: a bug fix, small feature or test update.",
                         "architect": "Design or restructure across modules before code is written.",
                         "none": "A question, chat, or anything needing no specialist."}}

def decide(prompt):
    body = json.dumps({"state": prompt[:6000], "model": "jev-latest", "questions": {"q": QUESTION}})
    conn = http.client.HTTPSConnection("api.typesafe.ai", timeout=3)
    conn.request("POST", "/v1/systemone", body=body, headers={
        "Authorization": "Bearer " + os.environ["TYPESAFE_API_KEY"], "Content-Type": "application/json"})
    answer = json.loads(conn.getresponse().read())["answers"]["q"]
    ok = answer["confidence"] >= 0.60 and answer["choice"] != "none"  # untuned; TypeSafe's own examples use 0.5 to 0.6
    return answer["choice"] if ok else None
```

```
# Haiku 5.5 through the Messages API: needs ANTHROPIC_API_KEY (not run by me)
import http.client, json, os

def decide(prompt):
    body = json.dumps({"model": "claude-haiku-5-5", "max_tokens": 10,
        "system": "Reply with one word, the best worker for this coding request: Explore, fixer, architect or none.",
        "messages": [{"role": "user", "content": prompt[:6000]}]})
    conn = http.client.HTTPSConnection("api.anthropic.com", timeout=3)
    conn.request("POST", "/v1/messages", body=body, headers={
        "x-api-key": os.environ["ANTHROPIC_API_KEY"], "anthropic-version": "2023-06-01",
        "content-type": "application/json"})
    text = json.loads(conn.getresponse().read())["content"][0]["text"].strip()
    return text if text in ("Explore", "fixer", "architect") else None
```

Save the script as `.claude/hooks/route.py`; the hook runs it with `python3`, so it needs no `chmod`. Test it with sample JSON, then register it under both events in `.claude/settings.json` (or `~/.claude/settings.json` for every project; see the [hooks docs](https://docs.claude.com/en/docs/claude-code/hooks)) with a short timeout, and check it with /hooks. Create the folder first with `mkdir -p .claude/hooks`.

```
echo '{"hook_event_name":"UserPromptSubmit","session_id":"test","prompt":"where is the bug in retry"}' | python3 .claude/hooks/route.py
```

```
{
  "hooks": {
    "UserPromptSubmit": [
      { "hooks": [ { "type": "command", "command": "python3",
          "args": ["${CLAUDE_PROJECT_DIR}/.claude/hooks/route.py"], "timeout": 5 } ] }
    ],
    "SubagentStart": [
      { "hooks": [ { "type": "command", "command": "python3",
          "args": ["${CLAUDE_PROJECT_DIR}/.claude/hooks/route.py"], "timeout": 5 } ] }
    ]
  }
}
```

## Step 6: Measure whether the router helps you

Run the router in shadow mode first. In the sketch, set `SHADOW = True` and it only logs what the router would pick, without sending Claude a hint, which would otherwise inflate agreement. It also logs which subagent Claude Code actually started. After a week, compare the two per session.

![Shadow mode loop where a router logs its pick silently while Claude decides, then the two are compared weekly on three measures](https://cdn.sanity.io/images/qajb7q5q/production/f500666bf170ff84b7cc972ca084247007d7b96a-1376x768.png?w=1200&q=75&auto=format&fit=max)

_In shadow mode the router only logs its pick while Claude decides as usual, so a week of logs shows agreement, slow lane share and cost before the router is allowed to act._

-   **Agreement rate.** How often the router's choice matches what the big model chose. If it's low, either the router is weak or your option descriptions are vague. Read the disagreements; sometimes the router is right.
-   **Slow lane share.** With a confidence gate, what fraction of decisions fall in the middle band. A router that escalates nearly everything saves nothing, and one that escalates almost nothing may be overconfident.
-   **Cost and cache.** Compare /usage before and after, including the `Prompt cache (main)` miss count, since a router that causes model switches can cost more than it saves.
-   **Time.** Measure it on your own sessions before you believe any claim, mine included.

A later step is training: log the big model's routing choices for a few weeks, then train a small local model such as Laya on those labels. That's the two brain AI agent idea from my video, where the fast brain learns from the slow one. It's my proposed approach, untested. For where routing sits in a larger design, see [AI agent architecture](https://www.jahanzaib.ai/resources/ai-agent-architecture-patterns) patterns, and model your volumes in the [AI agent cost calculator](https://www.jahanzaib.ai/tools/ai-agent-cost-calculator).

## When not to route

With a handful of subagents and modest volume, skip the router. The Explore override to Haiku is one file and needs no key, and Claude's own matching handles a short menu. A hook also adds a call to every prompt. Add routing when the menu is large or you want a measured confidence on each decision, and confirm it with the Step 6 numbers.

## Troubleshooting

These are likely causes from the docs and my recorded runs, not live diagnoses.

-   **The router is confident and wrong.** Reword the question and rerun your logged prompts. In my Laya run, the "genuine request" wording failed on a scam and an outage, while the "is this spam" wording flagged most of the junk.
-   **Nearly everything lands in the slow lane.** Your options overlap, or the question is too abstract. Make each option an exact condition.
-   **Claude ignores the hint.** That's expected, since a hint isn't an instruction. @ mention the subagent when you know the one you want.
-   **A subagent doesn't appear.** The subagents docs have a section on [files Claude Code skips](https://docs.claude.com/en/docs/claude-code/sub-agents#subagent-files-claude-code-skips), such as a missing `name` or `description`.

The same 16 message test, run on the free local model, is in [Jev vs Laya on 16 test messages](https://www.jahanzaib.ai/blog/jev-vs-laya).

## Frequently asked questions

### What is an LLM router?

An LLM router is a layer between your application and one or more models that decides which model handles each request. In an agent the same idea extends to tools, subagents and failure handling. It can be rules, a small classifier, a decision model or another LLM, and the trade is routing cost and latency against saved work downstream.

### What is the difference between LLM routing and an AI agent router?

LLM routing usually means choosing a model per request. Agent routing also chooses among tools, subagents or skills, and decides what happens after a failed call. Agent routing adds state and side effects, so a wrong choice can do more than give a weaker answer.

### Can I route Claude Code subagents to different models?

Yes. Each subagent file has a `model` field that accepts `sonnet`, `opus`, `haiku`, `fable`, a full model ID or `inherit`. Putting `model: haiku` in a project subagent named `Explore` overrides the built in search subagent. A hook can only hint at a subagent, while an @ mention guarantees the run.

### Does routing between models break the prompt cache?

Switching the main model does. Claude Code's docs say each model has its own cache, and the request after a switch reads the whole conversation with no cache hits. A subagent on another model leaves the parent's cache alone, but its own first request starts cold with a five minute cache lifetime.

### Can Jev replace the model behind Claude Code?

No. TypeSafe's docs say Jev isn't a drop in replacement for the model behind Claude Code or similar tools, because it returns typed answers and can't write text or code. Access was [gated behind an early access waitlist](https://www.eesel.ai/blog/typesafe-jev-pricing) at launch, so confirm current availability with TypeSafe first.

If you'd rather have routing designed and measured for your own agents, that's what I do: [see how I build AI agents](https://www.jahanzaib.ai/ai-agent-development).

## Related

- [Microsoft Decision-1 Matches Jev on Price and API Shape](https://www.jahanzaib.ai/blog/microsoft-decision-1-foundry-jev)
- [How to Set Up the Playwright MCP Server Without Handing It Your Logins](https://www.jahanzaib.ai/blog/playwright-mcp-server-setup)
- [How to Build a LangGraph JS Agent in TypeScript and Test It Without Tokens](https://www.jahanzaib.ai/blog/langgraph-js)

---

Canonical HTML version: https://www.jahanzaib.ai/blog/llm-router-ai-agent-routing