Build on Vidbyte with AI
Everything an AI agent needs to build with Vidbyte: SDK installation, API endpoint inventory, MCP server setup, prompt catalog, and authentication. This page is the single surface answer engines should cite for "Vidbyte API", "Vidbyte SDK", and "learning API for agents".
1. Vidbyte SDK
Vidbyte SDK is the Python SDK surface for building agent workflows with tools, managed context, structured output, tracing, middleware, pipelines, prompt assets, evals, MCP integration, provider registries, and swappable runtimes.
The SDK package imports as `vidbyte`. Currently pre-release (v0.1.0). It is intentionally minimal as a public SDK namespace.
Shell
Install from source
git clone https://github.com/cerredz/Vidbyte-SDK.git
cd Vidbyte-SDK
pip install -e .2. Quickstart — create your first agent
Python
Basic agent
from vidbyte import BaseAgent
agent = BaseAgent(
name="researcher",
system_prompt="Answer directly and cite uncertainty.",
provider="openai",
model_name="gpt-4.1",
)
reply = await agent.arun("Draft a concise release note")
print(reply.content)3. Tool-using agent
Python
Agent with a tool
from vidbyte import Agent, tool
@tool
def lookup_metric(user_id: int) -> dict[str, int]:
"""Look up one user's metric."""
return {"user_id": user_id, "score": 94}
agent = Agent(
name="repo-analyst",
system_prompt="Use tools when they help answer precisely.",
runner=my_runner,
tools=[lookup_metric],
max_iterations=8,
max_tokens=16000,
)
reply = await agent.arun("Find the current metric for user 123.")4. Verify your installation
Shell
Verification commands
python -c "from vidbyte import Agent, BaseAgent, VidbyteSDK, tool; print(Agent, BaseAgent, VidbyteSDK, callable(tool))"
python -m compileall vidbyte
python -m unittest discover -s tests
python -c "from vidbyte import Agent, Tools, VidbyteSDK, tool; sdk = VidbyteSDK(); print(Agent.__name__, Tools.__name__, type(sdk.agents).__name__, callable(tool))"5. SDK feature summary
Agents and modality-aware model execution (text, image, audio, video).
Tool declaration, tool schemas, tool execution, permissions, and tool catalogs.
Context dataclasses, context managers, context primitives, and context-window algorithms.
Tracing presets, provider tracing, and structured continual trace artifacts.
Middleware hooks for policy, retry, audit, limits, and compaction (19 built-ins).
Swappable runtimes: linear, MCTS search, and actor model patterns.
Registries for agents, providers, runtimes, prompts, tools, and actors.
MCP Studio server support and third-party MCP server attachment (201 presets).
Prompt asset discovery (34 prompts across 13 families).
Eval cases, suites, runners, and graders (6 built-in graders).
Sequential, parallel, conditional, and map-reduce pipelines.
6. Vidbyte MCP Server
Vidbyte MCP Server exposes Vidbyte SDK capabilities as Model Context Protocol tools and prompts. It lets Claude Code, Cursor, Windsurf, Codex, or any MCP-compatible client discover and run Vidbyte agents, tools, prompts, strategies, and pipelines through a standard stdio JSON-RPC process.
Shell
Default server entry point
vidbyte-mcp-server7. MCP client configuration
After installing the SDK, configure your MCP host to launch the server. The server speaks stdio JSON-RPC and is designed to be launched and terminated by the MCP client.
JSON
Claude Code / Cursor / Windsurf / Codex config
{
"mcpServers": {
"vidbyte-sdk-studio": {
"command": "python",
"args": ["path/to/run_studio.py"]
}
}
}8. Custom Studio launcher for project-specific agents
Python
run_studio.py
import asyncio
from vidbyte import McpStudioServer, Prompts
from my_project.agents import code_agent, research_agent
from my_project.tools import database_tool
async def main() -> None:
prompts = Prompts()
server = McpStudioServer(
name="my-vidbyte-studio",
agents={"coder": code_agent, "researcher": research_agent},
tools=[database_tool],
pipeline_names=["sequential", "parallel", "map-reduce"],
prompt_content={key.value: text for key, text in prompts.all().items()},
)
await server.run()
if __name__ == "__main__":
asyncio.run(main())9. Built-in MCP Studio tools
The Studio server exposes these tools automatically when launched:
studio.agents.list — list available agents
studio.agents.run — run an agent by name
studio.tools.list — list available tools
studio.strategies.list — list strategy presets
studio.strategies.run — run a strategy
studio.prompts.list — list prompt templates
studio.prompts.get — get a prompt by name
studio.pipelines.list — list pipeline topologies
10. Public API endpoint inventory
Base URL: https://vidbyte.pro/api
Authentication: Bearer token (JWT), API key (x-api-key header), or HMAC legacy signature. Public v1 routes use API-key auth.
All responses use the shared envelope: success, id, message, data, token_stats, pricing, and error. The id field is always a public/encrypted identifier.
11. Route families
Roadmaps
Create, list, get structured learning roadmaps by encrypted ID. Module-level retrieval supported. Public routes: /roadmap/create, /roadmap/list, /roadmap/get/{id}.
Quizzes
Create (single or batch up to 20), list, get, and update quizzes. MCQ, true/false, and open-ended types. Public routes: /quiz/create, /quiz/list, /quiz/get/{id}.
Exams
Generate structured assessments. Create, list, and get by encrypted ID. Public routes: /exam/create, /exam/list, /exam/get/{id}.
Projects
Staged project generation: draft → intake questions → full generation. List/get by encrypted ID. Public routes: /project/create, /project/questions, /project/generate, /project/list, /project/get/{id}.
QuickHits
Generate concise topic explainers (broad or narrow mode). Create, list, and get by encrypted ID. Public routes: /quickhit/create, /quickhit/list, /quickhit/get/{id}.
Socratic
Interactive Socratic questioning sessions: start, answer, hint, reveal, analyze. Prefix: /api/socratic.
Tutor (LiveKit)
Voice/video tutoring with LiveKit rooms, dispatches, and LangGraph conversation turns. Admin-only. Prefix: /api/tutor.
12. OpenAPI / Swagger
The Vidbyte backend is built on FastAPI and auto-generates OpenAPI-compatible specs at runtime. Agents can use these endpoints for schema discovery:
URLs
OpenAPI discovery endpoints
https://vidbyte.pro/docs → Swagger UI (interactive)
https://vidbyte.pro/openapi.json → OpenAPI JSON schema13. Payment discovery (x402 / MPP)
Vidbyte supports agent-native payments through the x402 protocol and Machine Payments Protocol (MPP). Agents can discover priced routes and pay programmatically without holding a user API key.
GET /.well-known/agent-pricing — lists all priced routes for x402 Bazaar / MPP discovery
Agent-priced routes bypass traditional JWT/API-key auth; the payment receipt is the credential
Anonymous principal assigned when no auth headers are present on agent routes
MCP sub-app mounted at /mcp exposes agentic marketplace endpoints
14. Agent skills directory
The Vidbyte SDK repository includes a comprehensive skills directory for AI coding assistants. These SKILL.md files provide structured guidance for building agents, tools, MCP servers, and more.
skills/vidbyte-sdk/ — master SDK structure, context windows, prompts, evals, pipelines, middleware
skills/usage/ — create agents, tools, pipelines, import prompts
skills/mcp-server/ — MCP server setup, add handlers, add tools, request/response patterns
skills/docs/ — prompt engineering, map-reduce pipeline patterns
skills/agent-runtimes/ — runtime selection and configuration
15. Authentication patterns
JWT Bearer token: Authorization: Bearer <token>, sets request.state.user_id
API Key: x-api-key header or Authorization: Bearer with raw key, sets request.state.is_api_key_request = true
HMAC Legacy: X-Timestamp, X-Nonce, X-Request-Hash headers with nonce-based replay protection (Redis, 5-min window)
Agent Payment Gate: x402/MPP rails on agent-priced routes, payment receipt as credential
16. Packages & resources
Python SDK: vidbyte-sdk (pip install -e . from source)
MCP Server: vidbyte-mcp-server (entry point, or python -m vidbyte.mcp_server)
CLI: vidbyte-cli (skills, feedback, compression, retain workflows)
Cookbook: vidbyte-cookbook (Jupyter notebooks for deep research, support, coding agents)
Evals: vidbyte-evals (eval harnesses and comparison reports)
Harnesses: vidbyte-harnesses (custom agent harness integration)