Specialist routing: one agent front door, many experts behind it
Register several agents, each with its own system prompt, model, and tools, and let Jev send every request to the one best suited to handle it. When no specialist clearly fits, the general JevAgent handles the task, and the run records which agent answered and why.
1. Write focused agents, let Jev pick the right one
A single system prompt that tries to cover billing, debugging, and release writing ends up diluting each job. Specialist routing lets you keep those jobs apart. You build one ordinary agent per kind of work, describe what each one handles, and register them on a JevAgent. Your application still calls one agent.
At the start of every run, Jev (the System 2 decision model) receives the user's request and your specialist descriptions, then answers one fixed Choice question: which specialist does this request fit? If Jev's pick is confident enough, that specialist handles the entire task in an isolated fork, with its own prompt, model, tools, and permissions. If nothing fits, or the match is weak, JevAgent's own general loop handles the task.
Try it below. Pick a request and watch Jev score it against each specialist. The white tick on each bar is the match threshold.
Route a request
request
“I was charged twice for invoice INV-2291. Can you refund one of them?”
Jev · System 2 decision
Which specialist does `request` fit?
billing91%Invoices, refunds, payment failures, and plan changes.
openai · gpt-4.1-mini · lookup_invoice
debugger3%API errors, stack traces, and failed requests.
anthropic · claude-sonnet-5
release_notes1%Release notes and changelog entries from merged PRs.
openai · gpt-4.1
no_suitable_agent5%Reserved option: the request fits no specialist's stated scope.
billing handles this task
P = 0.91 ≥ threshold 0.60 → forked and run for the whole task
Sample probabilities are illustrative, not recorded Jev output. The two-part payment question splits Jev's confidence, so its top pick falls below 0.6 and the general agent takes it.
1: Build each specialist as an ordinary agent
A specialist is any configured `Agent` (`BaseAgent`), the same class you would use on its own. Give each one a narrow system prompt and only the tools its job needs. Specialists do not have to share a provider or a model: put a cheap model on routine work and a stronger one on hard diagnosis.
These agents act as templates. Jev never runs the object you pass in directly; each routed run works on a fresh fork, so one task's history, tools, or MCP connections never leak into the next.
Three focused agents
from vidbyte import Agent, tool
@tool
def lookup_invoice(invoice_id: str) -> dict[str, str]:
"""Return the status, amount, and due date for one invoice."""
return billing_api.invoice(invoice_id)
billing = Agent(
name="billing",
system_prompt=(
"You resolve invoices, refunds, and plan changes. "
"Always look up the invoice before quoting an amount."
),
provider="openai",
model_name="gpt-4.1-mini",
tools=[lookup_invoice],
)
debugger = Agent(
name="debugger",
system_prompt="You diagnose API errors from stack traces and request logs, then propose the smallest fix.",
provider="anthropic",
model_name="claude-sonnet-5",
)
release_writer = Agent(
name="release-writer",
system_prompt="You turn merged pull requests into clear, customer-facing release notes.",
provider="openai",
model_name="gpt-4.1",
)2: Describe each specialist's scope
Wrap each agent in a `JevSpecialist` with a stable `id` and a `description`. Jev never reads the specialist's system prompt. The request, the IDs, and the descriptions are all it uses to route, so the description is the routing contract.
Jev's routing rules are strict on purpose. A request fits a specialist only when every piece of the work it asks for is inside that specialist's stated scope. Work the description does not mention is outside its scope, even if the ID suggests otherwise. A request that mixes two specialists' work goes to the general agent instead of being half-handled.
| Field | Rule | Why |
|---|---|---|
| `id` | Non-blank, trimmed, unique in the catalog, at most 4,096 characters, and never `no_suitable_agent`. | It becomes a Choice option name; `no_suitable_agent` is the reserved no-match option. |
| `description` | Non-blank, trimmed, at most 32,000 characters. | It is sent to Jev on every routed run as the option's meaning. |
| `agent` | A configured agent (`Agent` / `BaseAgent`). A `JevAgent` is rejected. | Specialists are forked and run directly; nested Jev routing is not allowed. |
| Catalog | Up to 254 specialists; all IDs and descriptions together fit in 1,000,000 characters. | TypeSafe allows 255 Choice options, and one is reserved for no-match. |
Every rule is checked when you construct `JevSpecialist` or `JevAgentSettings` and raises `ConfigurationError`. Construction never needs a TypeSafe key.
Describe the work, not the persona
from vidbyte import JevSpecialist
specialists = [
JevSpecialist(
id="billing",
description="Invoices, refunds, payment failures, and plan upgrades or downgrades for an existing account.",
agent=billing,
),
JevSpecialist(
id="debugger",
description="Diagnosing errors, stack traces, failed API requests, and unexpected responses from the public API.",
agent=debugger,
),
JevSpecialist(
id="release_notes",
description="Writing or editing release notes and changelog entries from merged pull requests.",
agent=release_writer,
),
]
# Weak descriptions Jev cannot route on reliably:
# description="Billing agent." -> no scope to compare against
# description="You are a friendly expert. Always..." -> persona instructions, not scope3: Register the catalog on JevAgentSettings
Pass the specialists to `JevAgentSettings.agents`. The JevAgent's own `system_prompt`, provider, and model make up the general agent: it answers when no specialist fits. Write that prompt as a capable generalist, not as a router.
Routing needs a TypeSafe decision-model key, read from `TYPESAFE_API_KEY` or passed as `decision=DecisionModelConfig(api_key=...)`. With `agents=()`, the default, JevAgent behaves exactly as before and makes no routing call.
One front door
from vidbyte import JevAgent, JevAgentSettings
agent = JevAgent(
JevAgentSettings(
name="support-desk",
system_prompt=(
"You are a general product support agent. Answer questions about the product "
"and hand off clearly when you cannot resolve something."
),
provider="openai",
model_name="gpt-4.1-mini",
agents=specialists,
specialist_match_threshold=0.6, # the default
)
)4: Run a task: what happens before any tool is called
Call `agent.arun(prompt)` as usual. Routing happens once, before any model or tool does task work. If you enabled preflight presets, the preflight gate runs first; a request the gate stops never reaches a specialist. Jev then receives one state field, `request`, which holds the current prompt only (not the conversation history), and answers the Choice question.
When a specialist qualifies, JevAgent forks it and calls its `generate_reply` with the original prompt plus the run's context and conversation history. It returns the specialist's output as its own reply and always closes the fork's MCP connections afterward. Every other path leads to the unchanged general loop. Pick a scenario to trace its path.
Trace one run
“Refund one of the two charges on INV-2291” → billing at P = 0.91 clears 0.60. The billing fork runs with its lookup_invoice tool.
- 1↳ gate stops → no agent runs; routing is None
Preflight gate
Runs only if preflight presets are enabled. It can stop the run to ask the user a clarifying question.
- 2↳ empty catalog → general loop, no Jev call
Catalog check
agents=() disables routing entirely.
- 3↳ routing_input_too_large → general
Routing input size
prompt + all IDs + descriptions + fixed question ≤ 1,000,000 chars
- 4↳ decision_unavailable → general (missing key, network error, bad answer)
Jev decision
One TypeSafe Choice call; its usage is recorded on the parent JevAgent.
- 5↳ no_suitable_agent → general
Jev's choice
One specialist ID, or the reserved no_suitable_agent.
- 6↳ below_probability_threshold → general
Threshold
P(choice) ≥ specialist_match_threshold
Specialist runs the whole task
fork() → generate_reply(prompt, context, history) → close MCP servers
General JevAgent loop
JevAgent's own prompt, model, tools, and tool selector
Every exit on the right is fail-open: the task still runs, and agent.response.routing records the reason. Only the preflight gate can stop a run.
5: Tune specialist_match_threshold
Jev returns a probability for every option. A chosen specialist runs only when its probability is at least `specialist_match_threshold`, a finite number from 0 through 1 (default `0.6`). Lower values route more borderline requests to specialists. Higher values keep uncertain work on the general agent.
A no-match answer always goes to the general agent, whatever the threshold. Drag the slider to see how the same five decisions change.
Same decisions, different threshold
Balanced: confident matches route; split or unfamiliar requests stay with the general agent.
Refund one of the two charges on INV-2291.
POST /v1/quizzes returns 422. Here's the trace.
Summarize this week's merged PRs for customers.
My card failed and the dashboard showed an error.
Plan a three-day trip to Lisbon.
Probabilities are illustrative. The white tick on each bar marks the current threshold.
6: Read which agent answered, and why
The reply comes back like any other agent reply. After each run, `agent.response.routing` holds a `JevSpecialistRouting` record describing that run. It is replaced when the next run starts, and it is `None` when no catalog is configured or the preflight gate stopped the run. It never contains the raw prompt or secrets.
Jev's decision usage is recorded once on the parent JevAgent. The specialist's own model usage is reported separately on `routing.usage`, so you can attribute cost per specialist.
| routing.fallback | When | Also set |
|---|---|---|
| `None` (specialist ran) | Jev chose a specialist at or above the threshold. | `specialist`, `choice`, `probability`, `usage` |
| `no_suitable_agent` | Jev judged that the request fits no specialist's scope. | — |
| `below_probability_threshold` | Jev picked a specialist, but with too little confidence. | `choice`, `probability` (the pick that was not taken) |
| `decision_unavailable` | Missing key, TypeSafe or network failure, or an unusable answer. | `error_type` when an exception was raised |
| `routing_input_too_large` | Prompt plus catalog exceeds 1,000,000 characters. | — |
`routing.routed` is True exactly when `specialist` is set. The record always names exactly one handler: a specialist or a fallback reason.
Inspect the routing record
reply = await agent.arun("I was charged twice for invoice INV-2291. Can you refund one of them?")
print(reply.content)
routing = agent.response.routing
if routing is None:
print("no routing this run")
elif routing.routed:
print(f"{routing.specialist} answered (P={routing.probability:.2f}, threshold={routing.threshold})")
if routing.usage:
print("specialist tokens:", routing.usage.total_tokens)
else:
print("general agent answered:", routing.fallback.value)
if routing.choice:
print(f"Jev leaned toward {routing.choice} at P={routing.probability:.2f}")
if routing.error_type:
print("decision error:", routing.error_type)8. Isolation and failure behavior
Routing is designed to fail open before the work starts and never to run a task twice.
One decision per run, made before any task tool executes. There are no mid-run handoffs, and specialists never delegate to each other.
The selected template is forked for each run. Its history, tools, permissions, middleware, and loop settings apply only to that run, and the catalog agents are never mutated.
The fork's MCP connections are always closed afterward, even when the specialist raises.
If Jev is unavailable, routing falls back to the general agent, so your app keeps answering without a TypeSafe key.
If a specialist fails after it starts, the error propagates. The task is not replayed on the general agent, because the specialist may already have caused side effects.
A specialist keeps its own tools. The general agent's tool selector applies only when the task falls back to the general agent.
9. Example: a support desk with three specialists
A complete script. One public agent fronts billing, API debugging, and account security. Each specialist gets only the tools its job needs. The refund tool is declared `WRITE`, and only the billing specialist's permission policy allows it, so an ambiguous request can never reach it: ambiguous requests fall back to the general agent, which has no tools.
support_desk.py
import asyncio
from vidbyte import Agent, JevAgent, JevAgentSettings, JevSpecialist, ToolPermission, tool
from vidbyte.tools.security import PermissionPolicy
# billing_api, logs_api, and auth_api stand in for your own service clients.
@tool
def lookup_invoice(invoice_id: str) -> dict[str, str]:
"""Return the status, amount, and due date for one invoice."""
return billing_api.invoice(invoice_id)
@tool(permission=ToolPermission.WRITE)
def issue_refund(invoice_id: str, amount_cents: int) -> str:
"""Refund part or all of one paid invoice."""
return billing_api.refund(invoice_id, amount_cents)
@tool
def search_request_logs(request_id: str) -> list[dict[str, str]]:
"""Return gateway log lines for one API request ID."""
return logs_api.search(request_id)
@tool
def list_active_sessions(account_id: str) -> list[dict[str, str]]:
"""List signed-in devices and API keys for one account."""
return auth_api.sessions(account_id)
billing = Agent(
name="billing",
system_prompt="You resolve invoices and refunds. Look up the invoice before quoting or refunding any amount.",
provider="openai",
model_name="gpt-4.1-mini",
tools=[lookup_invoice, issue_refund],
permission_policy=PermissionPolicy.allow_all(), # billing may call its WRITE refund tool
)
debugger = Agent(
name="debugger",
system_prompt="You diagnose failed API calls from request IDs and stack traces, then propose the smallest fix.",
provider="anthropic",
model_name="claude-sonnet-5",
tools=[search_request_logs],
)
security = Agent(
name="security",
system_prompt="You help users secure their account: review sessions and explain how to rotate keys. Never reveal secrets.",
provider="openai",
model_name="gpt-4.1",
tools=[list_active_sessions],
)
desk = JevAgent(
JevAgentSettings(
name="support-desk",
system_prompt="You are a general support agent. Answer product questions and say clearly when a human is needed.",
provider="openai",
model_name="gpt-4.1-mini",
agents=[
JevSpecialist(
id="billing",
description="Invoices, charges, refunds, and payment failures on an existing account.",
agent=billing,
),
JevSpecialist(
id="api_debugging",
description="Diagnosing failed or unexpected public API requests from request IDs, status codes, or stack traces.",
agent=debugger,
),
JevSpecialist(
id="account_security",
description="Reviewing signed-in sessions and API keys, suspected account compromise, and key rotation.",
agent=security,
),
],
)
)
TICKETS = [
"I was charged twice for INV-2291. Please refund the duplicate.",
"Request req_8f2c returned 500 from /v1/roadmaps. What broke?",
"I don't recognize a device signed in to my account. What should I do?",
"What's the difference between the Pro and Team plans?",
]
async def main():
for ticket in TICKETS:
reply = await desk.arun(ticket)
routing = desk.response.routing
handler = routing.specialist if routing.routed else f"general ({routing.fallback.value})"
print(f"[{handler}] {ticket}")
print(" ", reply.content[:120], "...")
asyncio.run(main())What a run might print
[billing] I was charged twice for INV-2291. Please refund the duplicate.
I found two charges of $49.00 on INV-2291 and refunded the second one ...
[api_debugging] Request req_8f2c returned 500 from /v1/roadmaps. What broke?
The gateway log shows an upstream timeout after 30s while generating module 4 ...
[account_security] I don't recognize a device signed in to my account. What should I do?
There are three active sessions. The one from an unfamiliar Linux device ...
[general (no_suitable_agent)] What's the difference between the Pro and Team plans?
Pro is for one person; Team adds shared workspaces and seat management ...10. Example: an engineering copilot with a stricter threshold
Here the specialists run on different providers, and one of them can query a production replica. The team raises the threshold to 0.75 so the SQL specialist, and its database tool, runs only on unambiguous data questions. Anything borderline goes to the general agent, which has no tools.
eng_copilot.py
import asyncio
from vidbyte import Agent, JevAgent, JevAgentSettings, JevSpecialist, tool
from vidbyte.lib.config import DecisionModelConfig
# replica and load_secret stand in for your own database client and secret store.
@tool
def run_readonly_sql(query: str) -> list[dict[str, object]]:
"""Run one SELECT statement against the analytics replica and return at most 200 rows."""
return replica.select(query, limit=200)
reviewer = Agent(
name="code-reviewer",
system_prompt=(
"You review diffs for correctness, security, and readability. "
"Rank findings by severity and quote the exact lines."
),
provider="anthropic",
model_name="claude-sonnet-5",
)
analyst = Agent(
name="sql-analyst",
system_prompt="You answer product-metrics questions by writing and running read-only SQL, then explain the result.",
provider="openai",
model_name="gpt-4.1",
tools=[run_readonly_sql],
max_iterations=6,
)
scribe = Agent(
name="incident-scribe",
system_prompt="You turn incident timelines and chat logs into blameless postmortems with action items.",
provider="openai",
model_name="gpt-4.1-mini",
)
copilot = JevAgent(
JevAgentSettings(
name="eng-copilot",
system_prompt="You are a senior engineer who answers general engineering questions concisely.",
provider="openai",
model_name="gpt-4.1-mini",
decision=DecisionModelConfig(api_key=load_secret("typesafe")),
specialist_match_threshold=0.75,
agents=(
JevSpecialist(
id="code_review",
description="Reviewing a pasted diff or code snippet for bugs, security issues, and readability.",
agent=reviewer,
),
JevSpecialist(
id="metrics_sql",
description="Answering questions about product usage metrics that require querying the analytics database.",
agent=analyst,
),
JevSpecialist(
id="postmortem",
description="Writing a postmortem or incident summary from a pasted timeline or chat log.",
agent=scribe,
),
),
)
)
async def ask(prompt: str) -> None:
reply = await copilot.arun(prompt)
routing = copilot.response.routing
if routing.routed:
cost = routing.usage.cost_usd if routing.usage else None
print(f"→ {routing.specialist} (P={routing.probability:.2f}, cost={cost})")
else:
print(f"→ general: {routing.fallback.value} (leaned {routing.choice}, P={routing.probability})")
print(reply.content, "\n")
async def main():
await ask("How many workspaces created a roadmap in the last 7 days?")
await ask("Review this diff:\n- if user.is_admin:\n+ if user.is_admin or DEBUG:")
await ask("Should we pick Postgres or DynamoDB for an append-only event log?")
asyncio.run(main())11. Example: a study coach that keeps multi-turn context
A learning app can route each message on its own. Routing looks only at the current prompt, while the chosen specialist still receives the conversation history. A follow-up like "now quiz me on it" can therefore move from the explainer to the quiz maker without losing the topic.
study_coach.py
import asyncio
from vidbyte import Agent, JevAgent, JevAgentSettings, JevSpecialist
socratic_tutor = Agent(
name="socratic-tutor",
system_prompt=(
"You coach learners through problems with guiding questions. "
"Never state the final answer; ask one question at a time."
),
provider="anthropic",
model_name="claude-sonnet-5",
)
explainer = Agent(
name="concept-explainer",
system_prompt="You explain a concept from first principles in under 200 words, then give one concrete example.",
provider="openai",
model_name="gpt-4.1-mini",
)
quiz_maker = Agent(
name="quiz-maker",
system_prompt="You write 5 retrieval-practice questions on the topic being studied, with answers hidden at the end.",
provider="openai",
model_name="gpt-4.1-mini",
)
coach = JevAgent(
JevAgentSettings(
name="study-coach",
system_prompt="You are a friendly study coach. Help plan study sessions and answer general learning questions.",
provider="openai",
model_name="gpt-4.1-mini",
agents=[
JevSpecialist(
id="homework_help",
description="Helping a learner work through a specific homework or practice problem they are stuck on.",
agent=socratic_tutor,
),
JevSpecialist(
id="explain_concept",
description="Explaining what a concept, term, or theorem means.",
agent=explainer,
),
JevSpecialist(
id="make_quiz",
description="Creating practice questions or a quiz on a topic.",
agent=quiz_maker,
),
],
)
)
CONVERSATION = [
"What does the chain rule actually say?",
"I'm stuck on d/dx of sin(x^2). Can you help me get there myself?",
"Now quiz me on it.",
"How should I split 6 hours of study across calculus and physics this week?",
]
async def main():
for message in CONVERSATION:
reply = await coach.arun(message)
routing = coach.response.routing
who = routing.specialist if routing.routed else "general coach"
print(f"you: {message}")
print(f"{who}: {reply.content[:100]} ...\n")
asyncio.run(main())Where each message went
"What does the chain rule actually say?" → explain_concept
"I'm stuck on d/dx of sin(x^2)..." → homework_help (asks, never tells)
"Now quiz me on it." → make_quiz (sees the history: chain rule)
"How should I split 6 hours of study..." → general coach (no_suitable_agent)