Documents
Citations
Managed context
Sessions
The answer is already in a folder somewhere, but finding it means opening nine files, and asking a chat model means getting an answer that sounds right and cites nothing.
Paste into Claude, Codex, Cursor, or another coding agent to scaffold this harness.
# Context: Vidbyte SDK
Vidbyte SDK is a Python package (import as `vidbyte`) for building agent workflows yourself.
Use it when you want to own the agent loop: system prompts, model/provider choice, tools, managed context, middleware, pipelines, tracing, evals, and multi-agent composition.
Core mental model:
1. Create an Agent or BaseAgent.
2. Attach a system prompt, provider/model, optional tools, context, middleware, and runtime choices.
3. Call run() or arun().
4. Let the SDK handle message assembly, tool schemas/calls, iteration limits, and optional pipelines.
Public package boundary: reusable local agent infrastructure. It does not ship private Vidbyte learning models or proprietary platform internals.
Every step adds to the same file. By the end you have one runnable doc_qa.py.
Putting sources in the schema is what makes a refusal legible: an empty list is a fact the caller can check, not a tone the model chose.
+ an answer that carries its own citations
from typing import Literal
from pydantic import BaseModel, Field
class Answer(BaseModel):
answer: str = Field(description="The answer, or a plain statement that the docs do not cover it.")
sources: list[str] = Field(description="File paths used. Empty when nothing supported the answer.")
confidence: Literal["high", "medium", "low"] = Field(description="How well the docs covered it.")The config leaves allow_write off, so these tools can read the folder and nothing else. Grep finds the candidate files; read_text opens them.
+ read-only search over one folder
from pathlib import Path
from vidbyte.tools.builtins.code_search import GrepTool
from vidbyte.tools.filesystem import FileSystemToolConfig, ListDirTool, ReadTextTool
DOCS = Path("./docs").resolve()
docs_fs = FileSystemToolConfig(root=DOCS) # allow_write stays False
doc_tools = [ReadTextTool(docs_fs), ListDirTool(docs_fs), GrepTool(DOCS)]Some things belong in every run: your index file and your house rules. A ContextManager holds them as structured items instead of you pasting them into the prompt each time.
+ context present on every run
from vidbyte import ContextManager, FileContextItem, TextContextItem
context = ContextManager([
FileContextItem.from_path(DOCS / "index.md", include_content=True),
TextContextItem(
title="House rules",
content=(
"Answer only from text the tools returned. "
"Cite the file path for every claim. "
"If nothing in the docs answers the question, say so and leave sources empty."
),
),
])A forty-page policy read in full will crowd out everything else. This preset keeps the fact that a tool ran and what it concluded, without pinning the raw text in the window.
+ large files kept out of the window
from vidbyte import ContextWindow
window = ContextWindow.preset.no_raw_tool_outputsThe schema, the tools, the managed context, and the window strategy all arrive in one constructor call.
+ the agent, wired from the four pieces above
from vidbyte import Agent
agent = Agent(
name="document-qa",
system_prompt=(
"Answer questions about the user's documents. "
"Search before answering, and quote only what the tools returned."
),
provider="openai",
model_name="gpt-4.1",
tools=doc_tools,
output_schema=Answer,
context_manager=context,
algorithm=window,
)One call attaches the agent to a durable session that checkpoints after every turn. The agent itself stays stateless; persistence lives in the session wrapper.
+ a thread that survives the process
from vidbyte import FileSessionStore
store = FileSessionStore(root="./.vidbyte/sessions")
session = agent.persist(store=store)Both paths matter. The second question is the one that tells you whether the grounding rule actually holds.
+ the supported case and the refusal
first = await session.arun("What does our policy say about data retention?")
print(first.structured.answer)
print("sources:", first.structured.sources)
# Same session, so the agent still has the thread.
follow_up = await session.arun("Does that apply to backups too?")
print(follow_up.structured.confidence)
missing = await session.arun("What is our office parking policy?")
assert missing.structured.sources == [] # nothing in the docs supported it
print(missing.structured.answer)The seven steps above, in one file.
doc_qa.py — complete
"""Document Q&A Agent - answers from your files, cites them, and remembers the thread."""
import asyncio
from pathlib import Path
from typing import Literal
from pydantic import BaseModel, Field
from vidbyte import (
Agent,
ContextManager,
ContextWindow,
FileContextItem,
FileSessionStore,
TextContextItem,
)
from vidbyte.tools.builtins.code_search import GrepTool
from vidbyte.tools.filesystem import FileSystemToolConfig, ListDirTool, ReadTextTool
DOCS = Path("./docs").resolve()
class Answer(BaseModel):
answer: str = Field(description="The answer, or a plain statement that the docs do not cover it.")
sources: list[str] = Field(description="File paths used. Empty when nothing supported the answer.")
confidence: Literal["high", "medium", "low"] = Field(description="How well the docs covered it.")
docs_fs = FileSystemToolConfig(root=DOCS)
doc_tools = [ReadTextTool(docs_fs), ListDirTool(docs_fs), GrepTool(DOCS)]
context = ContextManager([
FileContextItem.from_path(DOCS / "index.md", include_content=True),
TextContextItem(
title="House rules",
content=(
"Answer only from text the tools returned. "
"Cite the file path for every claim. "
"If nothing in the docs answers the question, say so and leave sources empty."
),
),
])
agent = Agent(
name="document-qa",
system_prompt=(
"Answer questions about the user's documents. "
"Search before answering, and quote only what the tools returned."
),
provider="openai",
model_name="gpt-4.1",
tools=doc_tools,
output_schema=Answer,
context_manager=context,
algorithm=ContextWindow.preset.no_raw_tool_outputs,
)
session = agent.persist(store=FileSessionStore(root="./.vidbyte/sessions"))
async def main() -> None:
first = await session.arun("What does our policy say about data retention?")
print(first.structured.answer)
print("sources:", first.structured.sources)
follow_up = await session.arun("Does that apply to backups too?")
print(follow_up.structured.confidence)
missing = await session.arun("What is our office parking policy?")
assert missing.structured.sources == []
print(missing.structured.answer)
if __name__ == "__main__":
asyncio.run(main())