Most agent failures are not writing failures. The model writes well. It starts on a request that was missing the one detail that mattered, reaches for the wrong tool, says it is done when one part of the job is still missing, or grinds through the same failing step for twenty iterations. Each of those is a judgment call, and an ordinary agent loop leaves every one of them to the same model that is busy doing the work.
JevAgent gives those calls to someone else. Your model still reads the request, uses the tools, and writes every word. Jev, TypeSafe's decision model, makes short, structured judgments at fixed points in the run and never writes the reply. This post walks through each of those points, what Jev asks there, and four problems the split solves.
Anatomy of a JevAgent run
Each Jev check is opt-inRequest arrives
Your model · System 1
Nothing yet. No main-model tokens are spent until Jev lets the run start.
Jev · System 2
Is the request clear? Which specialist should take it? Which tools could help?
14 yes/no · 1 choice · 1 yes/no per tool
- Unclear: the run returns 1 to 6 questions
- Specialist chosen: its own loop runs
Before work
Your model · System 1
Writes the run state: goal, what not to do, the hard part, and named deliverables.
Jev · System 2
Is the scope one example, or every member of a group?
1 choice
Working
Your model · System 1
Calls tools and reasons. A writer keeps a verified brief of the run every few iterations.
Jev · System 2
Do the run's facts and brief say one agent is no longer enough?
12 evidence questions per option
- Clone selected: 2 to 4 copies try in parallel
Finish attempt
Your model · System 1
Claims it is done. A handoff cites numbered events as evidence.
Jev · System 2
Is every deliverable and requirement shown in the evidence?
1 batched request · per-item veto
- A check fails: back to the same loop, 3 times by default
Reply
Your model · System 1
The answer stands and is returned to your app.
Jev · System 2
Nothing more to decide. Every verdict and its usage is on the response.
agent.response
In this article
- Your model does the work. Jev makes the calls.How JevAgent splits generation from judgment.
- Preflight questions: ask before spending a tokenThe clarity check that runs before your model starts.
- Agent alignment: the right agent, the right toolsSpecialist routing and tool selection, plus skills in review.
- Continuation gates: done means every part is doneHow Jev judges each finish attempt and sends work back.
- Dynamic compaction: a memory of the run that cannot driftThe verified run brief, plus context folding in review.
- Mid-run compute: Jev decides when one agent is not enoughRun facts, evidence questions, and clones.
- Four problems, solved with JevAgentVague requests, early finishes, stalled runs, and routing.
- What you can see, and what it costsThe response record and the usage ledger.
- Start with one checkWhere to go next.
Your model does the work. Jev makes the calls.
JevAgent borrows the System 1 and System 2 framing from psychology, then applies it literally. System 1 is your model. You pick the provider, the model, the system prompt, the tools, and the loop limits, exactly as you would for any Vidbyte SDK agent. Every reply, tool call, and piece of planning comes from it.
System 2 is Jev. Jev is a decision model: it answers yes/no and choice questions with probabilities, quickly and cheaply, and it cannot write prose. JevAgent sends it a small number of fixed, well-specified questions at fixed points in the run, then code decides what to do with the answers. Thresholds, vetoes, and caps live in the SDK, not in a prompt the model might talk itself out of.
Because the questions are fixed, the behavior is inspectable. Every check you turn on leaves its scores, its answers, and its usage on agent.response, so you can see why a run stopped to ask, why it went back to work, or why it brought in help.
Your model — Writes every reply, plans, and uses the tools. Any provider the SDK supports, at its normal price.
Jev — Answers fixed questions with probabilities. Input costs $0.042 per million tokens and output is free.
Code — Turns Jev's probabilities into actions through thresholds, per-item vetoes, and continuation caps.
A Vidbyte API key — Needed only when a Jev check is enabled. Building the agent, or running it with no checks, never calls Jev.
Create a JevAgent
1from vidbyte import (
2 JevAgent, JevAgentSettings, JevRuntimeSettings, JevPreflightPreset,
3)
4
5agent = JevAgent(
6 JevAgentSettings(
7 name="support-agent",
8 system_prompt="Help the user resolve account questions.",
9 provider="openai",
10 model_name="gpt-4.1-mini",
11 ),
12 JevRuntimeSettings(preflight=(JevPreflightPreset.CLARITY,)),
13)
14
15reply = agent.run("Summarize the renewal options for the Acme account.")
16print(reply.content)Preflight questions: ask before spending a token
A human contractor who gets a vague brief asks a question. An agent usually guesses, and it guesses at full price: a long loop, tool calls, and a confident answer to the wrong question. The clarity check moves that question to the front of the run, where it costs the least.
When it is on, Jev reads the request against 14 fixed questions before your model sees anything. Does the request name its target? Does it supply the material the work needs? Does every “this” point at something? The check fails if the average confidence drops below 0.75, or if any single answer falls below 0.2, so one glaring gap is never averaged away by thirteen clean ones.
If the check fails, JevAgent does not start the main loop. Your model, running with no tools and a fixed prompt, turns the weakest gaps into one to six questions, each with two to four recommended answers. The run returns those questions as its reply and records them on the response. Jev reads only the new message, so your app re-runs the agent with the original request and the user's answers together.
Jev answers the 14 clarity questions
All of them go out in one batched request, alongside routing and tool questions if those are on.
Code scores the answers
The check passes when the mean is at least 0.75 and no single answer is below 0.2.
An unclear request stops and asks
Your model writes 1 to 6 questions with recommended answers. The main loop never starts.
Re-run with the answers
Send the original request plus the answers. The gate checks the new message, and work begins.
Handle a clarification
1request = "Write the migration report."
2reply = agent.run(request)
3
4if agent.response.needs_clarification:
5 # The run stopped before your model wrote a single token of the report.
6 answers = []
7 for item in agent.response.clarification.questions:
8 choice = ask_user(item.question, item.recommendations) # your UI
9 answers.append(f"{item.question} {choice}")
10
11 # Jev reads only the new message, so send the request and the answers together.
12 reply = agent.run(request + "\n\nAnswers:\n" + "\n".join(answers))
13
14print(reply.content)The 14 things the clarity check looks at
Each question is answered yes or no with a probability. Each flagged example is a request that fails that one question.
- Action
- What Jev asksDoes the request state an action?
- Flagged example“The login page times out.”
- Object
- What Jev asksDoes each action have something to act on?
- Flagged example“Fix this.”
- Deliverable
- What Jev asksDoes it show the kind of result expected?
- Flagged example“Look into the checkout errors.”
- Target
- What Jev asksDoes it name the target of the work?
- Flagged example“Fix the bug from yesterday.”
- References
- What Jev asksIs every “this” or “it” resolved?
- Flagged example“Explain this error.”
- Scope parts
- What Jev asksDoes it say which parts to cover?
- Flagged example“Refactor the backend.”
- Scope size
- What Jev asksDoes it say how large the result should be?
- Flagged example“Write a summary of the report.”
- Completion
- What Jev asksDoes it state a finish line?
- Flagged example“Make the search page faster.”
- Information
- What Jev asksDoes it supply the material the work needs?
- Flagged example“Review the login code.”
- Constraints
- What Jev asksDoes it state limits for an open choice?
- Flagged example“Design a schema for the app.”
- Priorities
- What Jev asksDoes it order aims that compete?
- Flagged example“Make it faster and use less memory.”
- Consistency
- What Jev asksCan every instruction be followed together?
- Flagged example“Keep the API unchanged and rename an endpoint.”
- Time context
- What Jev asksDoes it fix the time frame the answer depends on?
- Flagged example“What changed in the last release we used?”
- Single reading
- What Jev asksDoes every key word have one meaning?
- Flagged example“Clean up the data.”
| Check | What Jev asks | Flagged example |
|---|---|---|
| Action | Does the request state an action? | “The login page times out.” |
| Object | Does each action have something to act on? | “Fix this.” |
| Deliverable | Does it show the kind of result expected? | “Look into the checkout errors.” |
| Target | Does it name the target of the work? | “Fix the bug from yesterday.” |
| References | Is every “this” or “it” resolved? | “Explain this error.” |
| Scope parts | Does it say which parts to cover? | “Refactor the backend.” |
| Scope size | Does it say how large the result should be? | “Write a summary of the report.” |
| Completion | Does it state a finish line? | “Make the search page faster.” |
| Information | Does it supply the material the work needs? | “Review the login code.” |
| Constraints | Does it state limits for an open choice? | “Design a schema for the app.” |
| Priorities | Does it order aims that compete? | “Make it faster and use less memory.” |
| Consistency | Can every instruction be followed together? | “Keep the API unchanged and rename an endpoint.” |
| Time context | Does it fix the time frame the answer depends on? | “What changed in the last release we used?” |
| Single reading | Does every key word have one meaning? | “Clean up the data.” |
What happens if Jev is unavailable?
The check fails open: an outage or a missing answer lets the run start as usual, and the same is true if the question writer fails. The exception is access. A missing, malformed, or rejected Vidbyte key raises an error instead of silently skipping the check.
Why yes/no questions instead of one “is this clear?” prompt
A single vague question gets a vague answer. Fourteen narrow questions each have one job, and the weakest ones tell the question writer exactly which gap to ask about, so the user sees three precise questions instead of a generic “can you tell me more?”
Agent alignment: the right agent, the right tools
An agent can only do good work with the right instructions and the right tools in front of it. Most teams handle this by growing one system prompt and one tool list until both are too long for the model to use well. JevAgent aligns the agent to the request before the loop starts instead.
Two alignment checks are on SDK main today. Specialist routing hands the whole task to the agent whose job covers it. Tool selection narrows a large tool catalog to the tools that could help with this request. Both ride in the same preflight request as the clarity check, so turning all three on still costs one Jev call before work begins.
A third layer, aligning the system prompt, tools, and skills to each request at run time, is in open pull requests. It is described in its own tab with its status.
Give JevAgent a team of specialists, each a title, one sentence of scope, and an ordinary SDK agent. Jev answers one choice question that ranks every scope, plus “none”, by the kind of work the request needs. The top answer wins. A chosen specialist runs its own loop with its own model and tools, and “none” keeps the task with the main agent.
Route billing to a specialist
1from vidbyte import BaseAgent, JevAgent, JevAgentSettings, JevSpecialist
2
3billing = BaseAgent(
4 name="billing", system_prompt="Resolve invoices, refunds, and plan changes.",
5 provider="openai", model_name="gpt-4.1-mini",
6)
7scope = "Invoices, refunds, double charges, and plan changes."
8agent = JevAgent(
9 JevAgentSettings(
10 name="front-door",
11 system_prompt="Answer general product questions.",
12 provider="openai",
13 model_name="gpt-4.1-mini",
14 agents=(JevSpecialist("billing", scope, billing),),
15 ),
16)
17
18reply = agent.run("I was charged twice for October.")
19print(agent.response.specialist) # the specialist that ran, or NoneWith the tool selector on, Jev asks one question per configured tool: could this tool materially help at any point in this request? Tools at or above the threshold (default 0.20) stay, in their original order, and the rest never reach the model. If Jev is unavailable, the full catalog is kept.
Narrow a large tool catalog
1from vidbyte import JevAgent, JevPreflightPreset, JevRuntimeSettings
2
3runtime = JevRuntimeSettings(
4 preflight=(JevPreflightPreset.TOOL_SELECTOR,),
5 tool_selector_threshold=0.2, # keep a tool when P(useful) >= 0.2
6)
7# analyst_settings: your JevAgentSettings, with e.g. 30 SQL, CRM, and docs tools
8agent = JevAgent(analyst_settings, runtime)
9
10reply = agent.run("Which enterprise customers opened the most tickets last week?")
11print(reply.metadata["jev_tool_selector"]) # candidate and kept countsSkill preload (#653) — One batched Jev call scores every skill you offer against the request, and only the relevant skills' text is added to this run's system prompt.
Skill sources (#655) — Load skills from outside sources, including native Claude skills, rather than only inline text.
Prompt alignment (#654) — A decision model finds gaps between the request and your system prompt, and bounded edits for those gaps apply to this run only.
Tool alignment (#654) — A tool scout searches your configured catalogs, validates candidates, and attaches approved MCP tools for this run. The original prompt and tools are restored on every exit.
Continuation gates: done means every part is done
The most expensive agent failure is the one you do not notice: a polished answer that skipped one of the three things you asked for. Agents finish early because the model decides when it is finished, and a model that has produced something reasonable is ready to stop.
Continuation takes that decision away from the model. Before work starts, your model writes a run state: the goal, what not to do, the hardest requirement, and the named deliverables. Each time the agent tries to finish, a handoff compiles the evidence for each item from a numbered log of what actually happened. A deliverable counts when the log shows it was made, not when the final answer promises it.
Jev then judges every enabled check in one request. Most checks use their threshold as a per-item veto as well as a bar for the average, so one clear “no” sends the agent back even when everything else looks finished. The agent returns to the same loop, with its history and tools intact, carrying a message that lists the original request, the run state, the evidence, the failed checks, and what to focus on. The cap is three continuations by default. After that, the current answer stands and every verdict stays on the response.
Write down what done means
Your model writes the run state once, before the loop: the goal, what not to do, the hard part, and one section per check you enabled.
Collect evidence at each finish attempt
A handoff cites numbered events (E2, E14, ...) for every deliverable. Code assigns those ids, so evidence has to point at something that happened in the run.
Judge everything in one request
Jev answers every enabled check's questions together. Each check is scored on its own, with per-item vetoes.
Continue, or let the answer stand
A failure appends a focused continuation message to the same loop, up to three times. A pass, or the cap, ends the run.
Check every deliverable
1from vidbyte import JevAgent, JevContinualSettings, JevDoneCheck, JevRuntimeSettings
2
3runtime = JevRuntimeSettings(
4 continual=JevContinualSettings(
5 checks=(JevDoneCheck.MULTI_PART, JevDoneCheck.SELF_REVIEW),
6 max_continuations=3,
7 ),
8)
9agent = JevAgent(engineer_settings, runtime) # your settings, with repo tools
10
11reply = agent.run(
12 "Fix the webhook signature bug, add a regression test, "
13 "and update the on-call runbook."
14)
15for check, result in agent.response.done.items():
16 print(check.value, result.passed, result.incomplete)
17print("sent back", agent.response.continuations, "times")- Who continues
- Same context (default)The main agent, in its own loop
- FreshA clean agent with the same model, tools, and permissions
- What it sees
- Same context (default)Its full history plus the continuation message
- FreshThe request, run state, evidence, and an assessment of what failed
- Best for
- Same context (default)Most runs, where history helps
- FreshRuns where a long history keeps the agent stuck
- Setting
- Same context (default)
gate="same_context" - Fresh
gate="fresh"
| Same context (default) | Fresh | |
|---|---|---|
| Who continues | The main agent, in its own loop | A clean agent with the same model, tools, and permissions |
| What it sees | Its full history plus the continuation message | The request, run state, evidence, and an assessment of what failed |
| Best for | Most runs, where history helps | Runs where a long history keeps the agent stuck |
| Setting | gate="same_context" | gate="fresh" |
Did it make everything it was asked for? · 5 checks
- MULTI_PART
- What it catchesA named deliverable that is missing or only partly made.
- OUTPUT_COUNT
- What it catchesThe wrong number of items. Code counts the distinct items before Jev looks.
- OUTPUT_EXTENT
- What it catchesA length or size limit that was missed. Code measures the final answer.
- EXPERT_DEPTH
- What it catchesThe hard part of a deliverable handled in its quick, shallow form.
- CAN_SIMPLIFY
- What it catchesAn implementation with a clear simplification that keeps its behavior.
| Check | What it catches |
|---|---|
| MULTI_PART | A named deliverable that is missing or only partly made. |
| OUTPUT_COUNT | The wrong number of items. Code counts the distinct items before Jev looks. |
| OUTPUT_EXTENT | A length or size limit that was missed. Code measures the final answer. |
| EXPERT_DEPTH | The hard part of a deliverable handled in its quick, shallow form. |
| CAN_SIMPLIFY | An implementation with a clear simplification that keeps its behavior. |
Did it cover the whole scope? · 6 checks
- SCOPE_COVERAGE
- What it catchesA change applied to one member of a group the user asked to change everywhere.
- INPUT_SET_COVERAGE
- What it catchesA file, page, or record the user named that was never really read.
- INPUT_EXHAUSTION
- What it catchesA paginated or growing collection that was not traversed to the end.
- DISCOVERED_ITEM_COVERAGE
- What it catchesItems a tool surfaced mid-run that were never processed.
- NEGATIVE_COVERAGE
- What it catchesAn “all clear” reported for a target that was never inspected.
- MOTIVATING_CASE
- What it catchesThe unusual case the user cared about, swapped for the easy path.
| Check | What it catches |
|---|---|
| SCOPE_COVERAGE | A change applied to one member of a group the user asked to change everywhere. |
| INPUT_SET_COVERAGE | A file, page, or record the user named that was never really read. |
| INPUT_EXHAUSTION | A paginated or growing collection that was not traversed to the end. |
| DISCOVERED_ITEM_COVERAGE | Items a tool surfaced mid-run that were never processed. |
| NEGATIVE_COVERAGE | An “all clear” reported for a target that was never inspected. |
| MOTIVATING_CASE | The unusual case the user cared about, swapped for the easy path. |
Is the work real and finished? · 7 checks
- TARGET_OUTCOME
- What it catchesA proxy milestone, such as “it compiles”, reported as the real result.
- COMPLETION_EVIDENCE
- What it catchesA “done” status the run's observations do not support.
- CLAIMS
- What it catchesAn assertion in the final answer that no evidence backs.
- FAITHFUL_SCOPE
- What it catchesThe hardest requirement narrowed, mocked, skipped, or hard-coded.
- PROBLEMS_RESOLVED
- What it catchesA problem hit mid-run that was never repaired and re-checked.
- GUARANTEED_NEXT_ACTIONS
- What it catchesA follow-on action the request made necessary but left undone.
- ASSUMPTIONS_RECONCILED
- What it catchesDownstream work never updated after an assumption changed.
| Check | What it catches |
|---|---|
| TARGET_OUTCOME | A proxy milestone, such as “it compiles”, reported as the real result. |
| COMPLETION_EVIDENCE | A “done” status the run's observations do not support. |
| CLAIMS | An assertion in the final answer that no evidence backs. |
| FAITHFUL_SCOPE | The hardest requirement narrowed, mocked, skipped, or hard-coded. |
| PROBLEMS_RESOLVED | A problem hit mid-run that was never repaired and re-checked. |
| GUARANTEED_NEXT_ACTIONS | A follow-on action the request made necessary but left undone. |
| ASSUMPTIONS_RECONCILED | Downstream work never updated after an assumption changed. |
Did it follow the process and the plan? · 6 checks
- REQUIRED_SEQUENCE
- What it catchesStages done out of the order the user required. Code checks the order.
- REQUIRED_ACTIONS
- What it catchesA required procedure with no direct evidence. Cited excerpts must exist.
- PHASE_PROGRESS
- What it catchesA run that never entered a stage it needed and hit no blocker.
- REPORT_ACTION_ALIGNMENT
- What it catchesA plan the agent announced, then quietly did not carry out.
- CUMULATIVE_OBLIGATIONS
- What it catchesA requirement from an earlier user turn lost after later turns.
- SELF_REVIEW
- What it catchesAn in-scope objection from a strict reviewer that still stands.
| Check | What it catches |
|---|---|
| REQUIRED_SEQUENCE | Stages done out of the order the user required. Code checks the order. |
| REQUIRED_ACTIONS | A required procedure with no direct evidence. Cited excerpts must exist. |
| PHASE_PROGRESS | A run that never entered a stage it needed and hit no blocker. |
| REPORT_ACTION_ALIGNMENT | A plan the agent announced, then quietly did not carry out. |
| CUMULATIVE_OBLIGATIONS | A requirement from an earlier user turn lost after later turns. |
| SELF_REVIEW | An in-scope objection from a strict reviewer that still stands. |
Dynamic compaction: a memory of the run that cannot drift
Long runs bury their own important facts. The decision made at iteration 4 is forty tool results deep by iteration 40. The usual fix is to ask a model to summarize the history, and summaries drift: they paraphrase, merge, and sometimes invent.
JevAgent keeps a run brief that cannot drift that way. Starting at iteration 3, and then every few iterations (ten by default), a separate writer with no tools reads only the events since its last pass and proposes new notes. Each note must be a verbatim passage of up to 500 characters with the id of the event it came from. Code checks every note against the full event text and rejects anything that does not match or that repeats an earlier note. The writer gets one retry with the error. The original request stays the brief's goal, so the goal itself never gets rewritten, and the newest 50 notes are kept.
The writer can run on a cheaper model from the same provider. The brief is what the mid-run compute checkpoint reads, and it is what clones start from, so a fresh agent inherits the facts of the run without inheriting its whole transcript.
When — First at iteration 3, then every
every_iterationsiterations (default 10).What — Verbatim notes of up to 500 characters, each citing an event id. Code verifies every one.
Who writes — A tool-free writer on your model, or a cheaper model you name.
Where it goes — The compute checkpoint and clone prompts, and
agent.response.run_brieffor you to inspect.
In review: fold finished work out of the context (SDK #455)
An open pull request adds unit-of-work compaction. At each step, Jev answers one choice question: does the latest step continue the current unit of work, or start a new one? When a new unit starts with probability of at least 0.8, the previous unit closes. Once closed units would reclaim about 8,000 tokens, code replaces all of them in one rewrite, so one prompt-cache miss pays for every fold. Each record has fixed sections: Goal, Done, Found, Changed, and Open.
The newest closed unit, the system prompt, and the unit in progress always stay raw. Whole iterations are removed together, so a tool call is never separated from its result. If the history changes in a way the ledger did not expect, compaction turns itself off rather than guess.
Mid-run compute: Jev decides when one agent is not enough
Some runs need more than one agent's attempt. Most do not. Spending extra compute on every run wastes money, and never spending it leaves stuck runs stuck. JevAgent decides during the run, from evidence, instead of at configuration time.
On every iteration that continues the loop, JevAgent reads exact facts from the loop itself: the iteration, the number of tool calls, the current streak of failed tool calls, and the tokens used. No model is involved. Each time the run brief updates, Jev gets one request holding twelve evidence questions for each compute option you enabled. The questions ask about what has happened, such as whether the same failure persists or whether the plan keeps changing, rather than asking Jev to forecast whether help would work.
The option with the highest average confidence wins, but only at 0.8 or above. Today the option that acts is CLONE. It runs two to four copies of your agent in parallel, each starting from the request and the verified brief with an empty history, then hands their replies back to the main agent as one message. The other options are recognized and recorded, so you can see when Jev would have reached for them.
- CLONE
- What Jev recognizesOne bounded, checkable unit is left, and independent tries could succeed where this one keeps failing
- What happens todayActs. Starts 2 to 4 copies, default 2, once per run
- FRESH_AGENT
- What Jev recognizesThe run is degrading, with rising errors, repeated failures, or drift, and the brief supports a clean restart
- What happens todayRecorded on the response
- FORK_AGENT
- What Jev recognizesSeveral distinct approaches from the same verified state, with no clear winner yet
- What happens todayRecorded on the response
- SUBAGENT
- What Jev recognizesA separate subtask with its own inputs and deliverable that can be handed off
- What happens todayRecorded on the response
- SWARM
- What Jev recognizesMany large, independent units of work
- What happens todayIn review (SDK #584): a planned team of up to 10 helpers
| What Jev recognizes | What happens today | |
|---|---|---|
| CLONE | One bounded, checkable unit is left, and independent tries could succeed where this one keeps failing | Acts. Starts 2 to 4 copies, default 2, once per run |
| FRESH_AGENT | The run is degrading, with rising errors, repeated failures, or drift, and the brief supports a clean restart | Recorded on the response |
| FORK_AGENT | Several distinct approaches from the same verified state, with no clear winner yet | Recorded on the response |
| SUBAGENT | A separate subtask with its own inputs and deliverable that can be handed off | Recorded on the response |
| SWARM | Many large, independent units of work | In review (SDK #584): a planned team of up to 10 helpers |
How SWARM would work (SDK #584, in review)
When Jev recognizes many substantial, independent units, the main agent receives a one-time launch_swarm tool and writes shared conventions plus one assignment per unit. Before any helper starts, Jev checks each assignment: is it self-contained, disjoint from the others, in scope, and checkable? Up to ten helpers then run concurrently, and their results come back for the main agent to verify and merge.
What mid-run compute does not do
It does not change your model, its reasoning effort, or its thinking budget in the middle of a run. The only budget change on main is the faithful-scope continuation, which adds 2 iterations, 16,000 tokens, and 4 tool calls to limits you set.
Four problems, solved with JevAgent
Each example below pairs a real failure pattern with the configuration that addresses it. The code uses settings on SDK main. The traces show what a run looks like, but their numbers are illustrative and not recorded Jev output.
Someone types “Write the migration report.” into an internal agent. A plain agent picks a database, a time window, and an audience on its own, then spends a full loop writing a confident report about the wrong migration. The fix is not a smarter model. The agent needs to notice that the request is missing information before it starts.
Turn on the clarity check and handle the questions
1request = "Write the migration report."
2reply = agent.run(request)
3
4if agent.response.needs_clarification:
5 # The run stopped before your model wrote a single token of the report.
6 answers = []
7 for item in agent.response.clarification.questions:
8 choice = ask_user(item.question, item.recommendations) # your UI
9 answers.append(f"{item.question} {choice}")
10
11 # Jev reads only the new message, so send the request and the answers together.
12 reply = agent.run(request + "\n\nAnswers:\n" + "\n".join(answers))
13
14print(reply.content)Jev reads the message
One batched request answers the 14 clarity questions. Target, information, time frame, and size all come back low.
The run stops before your model starts
The mean is under 0.75, so your model, with no tools, writes three short questions, each with recommended answers. The main loop never begins.
Your app asks the user
Each recommendation can be a one-click button, so answering takes seconds instead of a back-and-forth chat.
Re-run with the answers
The new message passes the check, and the main agent starts on a brief it cannot misread.
Illustrative trace · not real Jev output
preflight · clarity mean P(yes) = 0.41 → stop and ask
weakest target 0.06 · information 0.09 · time_context 0.12 · scope_size 0.18
1. Which migration should the report cover?
- The Postgres 15 to 16 upgrade on the orders cluster
- The billing service move to the new event bus
- Every migration shipped this quarter
2. Who will read the report?
- The on-call engineers: steps, checks, and rollback
- Leadership: outcome, risk, and timeline
3. Where is the migration's record?
- The #db-migrations Slack channel
- MIGRATIONS.md in the infra repository
main model tokens spent: 0A coding agent gets a three-part task: fix a webhook signature bug, add a regression test, and update the on-call runbook. It fixes the bug, adds the test, writes “I’ll update the runbook next,” and stops. That answer sounds finished. Continuation checks whether it is finished.
Check every deliverable and run a strict self-review
1from vidbyte import JevAgent, JevContinualSettings, JevDoneCheck, JevRuntimeSettings
2
3runtime = JevRuntimeSettings(
4 continual=JevContinualSettings(
5 checks=(JevDoneCheck.MULTI_PART, JevDoneCheck.SELF_REVIEW),
6 max_continuations=3,
7 ),
8)
9agent = JevAgent(engineer_settings, runtime) # your settings, with repo tools
10
11reply = agent.run(
12 "Fix the webhook signature bug, add a regression test, "
13 "and update the on-call runbook."
14)
15for check, result in agent.response.done.items():
16 print(check.value, result.passed, result.incomplete)
17print("sent back", agent.response.continuations, "times")Write down what done means
Before any work, your model writes a run state with the goal, what not to do, and three named deliverables.
Judge the first finish attempt
A tool-free reviewer raises objections. A handoff then cites numbered events as evidence, and Jev judges every deliverable and objection in one request.
Send it back to the same loop
The runbook scores 0.08, below the 0.8 bar. The agent gets one message with the original request, run state, evidence, the failed checks, and what to focus on.
Judge again
The second attempt edits the runbook and covers the reviewer's empty-body case. Every check passes, so the answer stands.
Illustrative trace · not real Jev output
run state
signature_fix The webhook verifies signatures against the raw request body.
regression_test A test that fails on the old parsing and passes on the fix.
runbook_update The on-call runbook explains the new failure mode.
finish attempt 1 evidence E2-E31
signature_fix P(yes)=0.95 E14 edit webhooks/verify.py · E19 pytest: 1 passed
regression_test P(yes)=0.91 E18 add tests/test_verify.py
runbook_update P(yes)=0.08 no tool call touched docs/runbook.md
self_review 1 objection an empty request body is not covered
→ continuation 1, same loop: failed checks + focus
finish attempt 2 evidence E32-E41
runbook_update P(yes)=0.93 E36 edit docs/runbook.md
self_review resolved E34 test_empty_body passed
→ every check passed: the answer standsA long port of a config parser reaches its last unit, nested tables, at iteration 27. The approach is settled and the test gives a clear pass or fail, but six tool calls in a row have failed, and each new try sits on top of the last one in the agent's context. A plain agent keeps grinding until it runs out of budget. JevAgent recognizes a bounded, checkable unit where independent tries could succeed and brings in more compute for that unit only.
Run brief, compute checkpoint, and a faithful-scope finish check
1from vidbyte import (
2 JevAgent, JevComputeSettings, JevContinualSettings, JevDoneCheck,
3 JevRunBriefSettings, JevRuntimeSettings,
4)
5
6runtime = JevRuntimeSettings(
7 continual=JevContinualSettings(checks=(JevDoneCheck.FAITHFUL_SCOPE,)),
8 compute=JevComputeSettings(
9 brief=JevRunBriefSettings(model_name="gpt-4.1-mini", every_iterations=8),
10 dynamic_compute=("fresh_agent", "clone"),
11 clones=3,
12 ),
13)
14agent = JevAgent(porter_settings, runtime) # your settings, with repo tools
15
16reply = agent.run("Port the config parser to the new grammar; make every test pass.")
17for decision in agent.response.compute_decisions:
18 print(decision.iteration, decision.option)Read exact facts every iteration
The iteration count, tool calls, failed-call streak, and tokens are read straight from the loop, with no model call.
Keep a verified brief
At iteration 3, and every 8 iterations after that, a cheaper writer model adds notes. Code rejects any note that is not a verbatim passage from the run's own log.
Let Jev read the evidence
After each brief update, Jev answers 12 evidence questions for each option. At iteration 27, CLONE scores 0.86: one bounded unit is left, the approach is settled, and a passing test is easy to check.
Start fresh from a clean brief
Three clones work on the problem in parallel. Each starts from the request and the verified brief, not the 27-iteration transcript, and their replies join the main loop as one message.
Finish without cutting corners
Before the run can finish, the faithful-scope check confirms that “make every test pass” was met without skipping or mocking a test.
Illustrative trace · not real Jev output
iteration 3 facts tool_calls=6 error_streak=0 brief +5 notes (E2, E4, E5, E7, E9)
iteration 11 facts tool_calls=24 error_streak=0 brief +4 notes
compute fresh_agent 0.31 · clone 0.22 → no change
iteration 19 facts tool_calls=41 error_streak=3 brief +3 notes
compute fresh_agent 0.52 · clone 0.61 → no change
iteration 27 facts tool_calls=63 error_streak=6 brief +2 notes (E88, E91)
compute fresh_agent 0.71 · clone 0.86 → CLONE
3 clones start from the request + verified brief → replies join the loop
iteration 31 E104 test_nested_tables passed
finish faithful_scope P(yes)=0.89 → the answer standsA support product has one chat box and a lot of possible jobs: billing disputes, data questions, and general help. One giant system prompt with thirty tools attached is slow and easily confused. JevAgent puts one front door in front of a small team of agents and picks the tools for each request.
One front door: a specialist plus tool selection
1from vidbyte import (
2 BaseAgent, JevAgent, JevAgentSettings, JevPreflightPreset,
3 JevRuntimeSettings, JevSpecialist,
4)
5
6billing = BaseAgent(name="billing", system_prompt="Resolve billing issues.",
7 provider="openai", model_name="gpt-4.1-mini")
8desk = JevSpecialist("billing", "Invoices, refunds, and plan changes.", billing)
9agent = JevAgent(
10 JevAgentSettings(
11 name="front-door", system_prompt="Answer product and data questions.",
12 provider="openai", model_name="gpt-4.1",
13 tools=tuple(support_tools), agents=(desk,), # e.g. 30 support tools
14 ),
15 JevRuntimeSettings(preflight=(JevPreflightPreset.TOOL_SELECTOR,)),
16)
17
18reply = agent.run("Which enterprise customers opened the most tickets last week?")
19print(agent.response.specialist, reply.metadata.get("jev_tool_selector"))Choose by the kind of work
One choice question ranks each specialist's scope against the request, plus a “none” option. Jev chooses by the kind of work the outcome needs, not by keywords.
Hand off, or keep the task
A chosen specialist runs its own loop, with its own tools and model. “None”, or a missing answer, keeps the task with the main agent.
Narrow the tools
When the main agent keeps the task, the tool selector asks one yes/no question per tool, and only the useful ones reach the model.
Illustrative trace · not real Jev output
"I was charged twice for October."
specialist billing 0.81 · none 0.19 → billing runs its own loop
"Which enterprise customers opened the most tickets last week?"
specialist none 0.88 · billing 0.12 → main agent keeps the task
tool selector 30 candidates → 4 kept (P >= 0.20)
sql_query 0.97 · ticket_search 0.93 · crm_lookup 0.88 · chart 0.34What you can see, and what it costs
JevAgent records every decision it makes. After a run, agent.response holds the preflight results, any clarification, the specialist that ran, the run state, the latest evidence and review, every done verdict, the number of continuations, the run facts, the brief and its updates, every compute decision, and any clone replies.
Usage is held to the same standard. Every model call in the run, from the question writer to the clones, goes to one usage ledger. If a call cannot be priced or its usage was not reported, the run raises instead of handing you an incomplete bill. That means your model must be in the SDK's price list. In exchange, the total on the response is the whole run, not just the main loop.
Decisions —
response.results,clarification,specialist,done,continuations,compute_decisions,clone.Evidence —
response.run_state,handoff,review,run_facts,run_brief.Usage —
response.usagesplits the run's total into generative and decision parts.Jev pricing — $0.042 per million input tokens, with free output. Your model is billed at its normal rate.
Start with one check
You do not need every capability on day one. Turn on the one that matches the failure you see most: the clarity check if your agent starts on bad briefs, multi-part continuation if it finishes early, or the compute checkpoint if long runs stall. Each one is independent and costs nothing until it runs.