Make your vibe-coded app production ready
vibe-coded-to-productioncerredz/vidbyte-cli
- Coming soon
- Security
- Testing
- Debugging
- Vibe coding
- Lovable
- Supabase
- Production readiness
Coming soon: this skill needs vidbyte-cli 0.3.0, which is not on PyPI yet. The install command appears here once it ships.
Summary
Find and fix the security holes in an app built with Lovable, Bolt, v0, Replit, or Cursor, and keep the fixes in place with tests and CI.
- Audits the code for the holes AI builders repeat: Supabase tables without row-level security, secret keys in client code, browser-only auth, and open API routes.
- Drives the live app in a cloud browser the way a new user would, and reports each bug with the steps to reproduce it.
- Merges the audit and the hunt into one report ordered by harm, then fixes only what you approve, with a test for each hole.
- Adds a CI workflow that fails when a later AI edit undoes a fix.
- Ends with the steps only you can take, starting with rotating any leaked key.
- Vidbyte charges nothing for it. It runs on your own Codex and Browserbase accounts.
SKILL.md
Make a vibe-coded app production ready
The problem
A founder or a small team built a web app with Lovable, Bolt, v0, Replit, or Cursor. The builder wrote most of the code from prompts, the app is deployed, and real people now sign up, store their data in it, and sometimes pay. Nobody on the team has read the code the way an attacker would, and often nobody on the team can. They ask "is my app secure?" or "make my Lovable app production ready" because they cannot judge the answer themselves.
AI-built apps repeat the same holes, because a builder writes code that works on the first try, not code that refuses the wrong caller:
- Supabase tables created without row-level security, or with a policy that allows
true, so the public anon key that every visitor downloads can read or change every row. - Secret keys, such as a Stripe secret key, an OpenAI key, or the Supabase service-role key, in a
NEXT_PUBLIC_*orVITE_*variable. These are compiled into the JavaScript bundle every visitor downloads, so the key is public even when the repository is private. - Sign-in and admin checks done only in the browser. The button is hidden, but the data behind it is served to anyone who calls the API directly.
- API routes and edge functions that never check who is calling, or that trust a user id sent in the request, so one user can open another user's records.
- No rate limit on routes that call paid APIs, send email, or handle sign-in, so one script can run up the owner's bill or flood users' inboxes.
- CORS open to every site, request input that reaches raw SQL or HTML unchecked, and Stripe webhooks that never verify the signature, so anyone can mark an order paid.
None of these shows up in normal use. The app works, the demo works, and the first sign is a leaked table, a drained API budget, or a customer who can see a stranger's invoice. These apps also rarely have tests or CI, so a fix made once is quietly undone the next time the builder regenerates the file, and nobody notices until the hole is back. Reading the code alone also misses part of the problem: some holes show only when the deployed app is used, such as a page that hides a button but still returns the data to anyone who asks for it.
This skill is a first pass for a deployed web app that holds users' data. It is not a formal penetration test or a compliance audit such as SOC 2, HIPAA, or PCI DSS.
How vidbyte-cli solves it
This skill drives vidbyte-cli, the Vidbyte command-line tool, and its Bug Finder agent. First the
coding agent runs a free local audit of the repository for the holes listed above. Then, with the
user's yes, vidbyte-cli agents bug-finder run clones the app's GitHub repository into a temporary
directory and starts one fresh Codex agent at a time. Each agent reads the code and drives the
deployed app in a Browserbase cloud browser the way a new user would. It then reports one bug with
its evidence: the steps to reproduce it, the files and lines, the likely root cause, and a
regression test in plain words. The run stops when an agent finds nothing new, or after
--max-bugs bugs. Every bug goes into a ledger that vidbyte-cli agents bug-finder mark updates,
so a later run never reports a known bug as new, and reports a fixed bug that broke again as
regressed.
The coding agent then merges the audit and the hunt into one report ordered by harm. It fixes what the user approves, with a test for each hole, and adds a CI workflow that fails when a later AI edit undoes a fix. Vidbyte charges nothing for the Bug Finder. It runs on the user's own Codex and Browserbase accounts. At the end the user has the report, the approved fixes with their tests, the CI workflow, and a short list of the steps only they can take, starting with rotating any leaked key.
Goal and done-when
Find the security and reliability holes in a live AI-built app from two sides, a free local read of the repository for the failures these apps repeat and one Bug Finder run that drives the deployed app in a cloud browser while it reads the code, and turn both into one report with data exposure and auth bypass first. Then fix what the user approves, so that each fix has a test and a CI workflow runs those tests on every push.
Done when: the user has seen the report; any leaked key was the first thing they were told, with
the instruction to rotate it; every fix they approved is in the working tree with a passing test;
.github/workflows/security-guards.yml runs those tests; every fixed Bug Finder bug is marked
fixed; and the user has the list of steps only they can take.
Preflight
Run these in order. If any fails, read references/setup.md, fix only what failed, and run the
preflight again.
vidbyte-cli --version # must be 0.3.0 or newer vidbyte-cli --json whoami # exit 4 means no Vidbyte key is stored vidbyte-cli --json runtime doctor # read the "codex" row in data.hosts
whoamiexit 4 does not block this skill. The Bug Finder never contacts the Vidbyte backend and needs no Vidbyte key or balance. Tell the user that other Vidbyte agents need a key (references/setup.mdstep 2), and continue.- If the
codexrow shows"available": false, the run uses the Codex bundled with the SDK. Either way Codex must be logged in, withcodex loginor withOPENAI_API_KEYset. A login that relies only onCODEX_API_KEYfails, because the run blanks that variable.
Then check what this skill needs:
-
The code is on GitHub. Run
git remote -v. The Bug Finder clones from GitHub only. TakeOWNER/NAMEfromhttps://github.com/OWNER/NAME.gitorgit@github.com:OWNER/NAME.gitand drop the trailing.git, which the CLI refuses. With no GitHub remote, ask the user to connect the project to GitHub first; Lovable, Bolt, and Replit can each export or sync a project to a GitHub repository. The run hunts the repository's default branch, not this checkout, so tell the user whengit status -sbshows unpushed commits or a different branch. -
Node.js is installed. Run
npx --version. A web hunt starts its browser tool throughnpx; without it the run stops withnpx was not found, so the browser tool cannot start. -
The app URL is public. Ask the user for the deployed address, and prefer a staging or preview deployment over production (see Cost and consent). The CLI refuses an address that is not
httporhttps, has no host, carries a login (user:pass@), contains a space, or is longer than 2,000 characters. It refuses the hostslocalhost,127.0.0.1,0.0.0.0,::1, and any host ending in.localhostwithInvalid bug finder run option: app url must be reachable from the internet, not a local address.It does not refuse private network addresses such as10.0.0.5or192.168.1.20, but the browser runs in Browserbase's cloud and cannot reach them either, so refuse them yourself. -
A GitHub account is linked and a Browserbase key is stored. No command checks only these. The run checks both offline before it clones anything, exits 4 with
No GitHub account is linked for this profile.orNo Browserbase API key is stored for this profile., and spends nothing. When the user has never set them up, followreferences/setup.mdstep 3 now: the user runsvidbyte-cli github link(it needsVIDBYTE_GITHUB_CLIENT_ID) andvidbyte-cli browserbase loginin their own terminal. If either cannot be set up, skip Run and Read the result, deliver the local audit alone, and say that the live app was not exercised. -
This shell holds no production secrets. The Bug Finder agent runs the repository's code, and that code can read the environment the CLI starts from. The CLI blanks Vidbyte keys,
CODEX_API_KEY, and non-OpenAI provider keys, and nothing else. List variable names only, never values:env | cut -d= -f1 | grep -iE 'secret|service_role|private|token|password|database_url|^aws_|^stripe_|^supabase_'If a production secret is listed, ask the user to start the run from a shell that does not hold it.
Gather inputs
Do all of this locally before asking the user anything. It is free. Never print a secret's value:
every command below prints names, file paths, line numbers, and masked prefixes only. The commands
use git grep, which searches exactly the files committed to the repository (what GitHub, the Bug
Finder, and the deployed bundle see), and Node.js for masking. Run them from the repository root.
1. Detect the stack
Read package.json (dependencies and scripts), the lockfile name, .env.example or
.env.sample, supabase/config.toml, supabase/migrations/, supabase/functions/,
firestore.rules, and the framework config (next.config.*, vite.config.*, vercel.json,
netlify.toml). Write down the framework, the backend (Supabase, Firebase, or the app's own API),
the auth provider, payments, the paid APIs the app calls (OpenAI, Anthropic, Resend, Twilio), the
test runner, and whether .github/workflows/ exists. A Lovable project is usually Vite, React,
and Supabase, with the client in src/integrations/supabase/client.ts.
Some keys are public by design and are not leaks: the Supabase anon or publishable key
(sb_publishable_..., or a JWT whose role claim is anon) and the Firebase web apiKey. They
are safe only while row-level security or Firestore rules protect every table, so the database
checks below decide whether they are a problem.
2. Run the audit
A. Secrets.
# Env files committed to git (example and template files are fine) git ls-files | grep -E '(^|/)\.env' | grep -vE '\.(example|sample|template)$' # Client-exposed variables whose names say they hold a secret (prints names only) git grep -n -o -i -I -E '(NEXT_PUBLIC|VITE|REACT_APP|EXPO_PUBLIC|PUBLIC)_[A-Z0-9_]*(SECRET|SERVICE_ROLE|PRIVATE|PASSWORD|OPENAI|ANTHROPIC|STRIPE_SECRET|RESEND|SENDGRID|TWILIO)[A-Z0-9_]*' # Secret-shaped values: prints file, line, the masked prefix, and the length git grep -n -I -o -E '(sk_live_|rk_live_|sk_test_|sk-proj-|sk-ant-|sb_secret_|ghp_|github_pat_|AKIA)[A-Za-z0-9_-]{16,}|sk-[A-Za-z0-9]{40,}' | node -e 'let s="";process.stdin.on("data",c=>s+=c).on("end",()=>{for(const l of s.split("\n").filter(Boolean)){const i=l.lastIndexOf(":"),m=l.slice(i+1);console.log(l.slice(0,i),m.match(/^(sk_live_|rk_live_|sk_test_|sk-proj-|sk-ant-|sb_secret_|ghp_|github_pat_|AKIA|sk-)/)[0]+"[masked]",m.length+" chars")}})' git grep -l -I -E 'BEGIN [A-Z ]*PRIVATE KEY' # JWT-shaped keys (Supabase legacy keys): prints only the role claim, never the token git grep -n -I -o -E 'eyJ[A-Za-z0-9_-]+\.eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+' | node -e 'let s="";process.stdin.on("data",c=>s+=c).on("end",()=>{for(const l of s.split("\n").filter(Boolean)){const i=l.lastIndexOf(":");let r="unreadable";try{r=JSON.parse(Buffer.from(l.slice(i+1).split(".")[1],"base64url").toString()).role??"none"}catch{}console.log(l.slice(0,i),"role="+r)}})' # Secrets deleted from the code but still in git history (commit ids only, never the diff) for p in sk_live_ sk-proj- sk-ant- sb_secret_ 'PRIVATE KEY-----'; do git log --all --format='%h %ad %s' --date=short -S"$p"; done
A finding: a committed env file that holds a secret, a secret-named client variable, a
secret-shaped value, a private key, a JWT with role=service_role, or a history hit. Each one is
Critical. A value in a NEXT_PUBLIC_, VITE_, REACT_APP_, EXPO_PUBLIC_, or PUBLIC_ variable
is compiled into the JavaScript every visitor downloads, so it is public even when the repository
is private.
B. Database access rules.
git ls-files supabase/migrations git grep -n -i -E 'create table' -- supabase/migrations git grep -n -i -E '(enable|disable) row level security' -- supabase/migrations git grep -n -i -E '(using|with check)[[:space:]]*\([[:space:]]*true[[:space:]]*\)' -- supabase/migrations git grep -n -i 'security definer' -- supabase/migrations git grep -n -E 'verify_jwt[[:space:]]*=[[:space:]]*false' -- supabase/config.toml git grep -n -E 'if true|allow read, write;' -- '*.rules'
A policy spans several lines, so open the file at each true hit and read the whole
create policy statement to see which command it covers. A finding: a table with no
enable row level security in any migration, or a later disable (Critical); a policy that uses
true for insert, update, delete, or all (Critical); a select policy that uses true on
a table that holds users' data (Critical; on public content such as published posts it can be
intended, so ask the user); a security definer function that
never checks auth.uid() (High); a function with verify_jwt = false that does not verify the
caller itself (High); Firestore or Storage rules that allow everyone (Critical). If the app uses
Supabase but has no supabase/migrations/ folder, the audit cannot see the database. Add a manual
step to the report: open the Supabase dashboard, check the Security Advisor, and confirm in the
Table Editor that row-level security is on for every table.
C. Server code that never checks who is calling.
# Route files with no server-side session or signature check (read each one) git grep -L -E 'getUser\(|getClaims\(|auth\(\)|getServerSession|currentUser\(|verifyIdToken|getToken\(|constructEvent' -- '*app/*route.*' '*pages/api/*' 'api/*' '*server/*' '*supabase/functions/*index.ts' # Server code that trusts getSession(), which reads the session without verifying it git grep -n -E 'auth\.getSession\(' -- '*app/*' '*pages/api/*' 'api/*' '*server/*' '*supabase/functions/*' '*middleware.*' # The service-role key referenced outside server-only folders git grep -n -i 'service_role' -- '*.ts' '*.tsx' '*.js' '*.jsx' '*.vue' '*.svelte' ':!*supabase/functions/*' ':!*pages/api/*' ':!*server/*' ':!api/*' # Admin or owner checks that may exist only in the browser git grep -n -i -E "is_?admin|role[[:space:]]*===?[[:space:]]*['\"]admin|localStorage\.getItem\(['\"](role|admin|isAdmin)" -- '*.ts' '*.tsx' '*.js' '*.jsx' # Stripe webhook handlers that never verify the signature git grep -L 'constructEvent' -- '*webhook*'
Read every file these list before you call it a finding. A finding: a route that reads or changes
users' data with no session check (Critical); a route that checks the session but trusts a user id
or record id from the request without checking that the record belongs to the caller (Critical);
getSession() trusted on the server instead of getUser() (High); the service-role key in a file
that ships to the browser, such as anything under src/ in a Vite app or a file that starts with
'use client' (Critical, and the key must be rotated); an admin or owner check that only hides
part of the page while the data behind it is not protected on the server or by RLS (High); a
Stripe webhook without constructEvent (Critical, because anyone can mark an order paid). A
health check, a route that serves public content, or a webhook that verifies its signature is not
a finding.
D. Abuse limits.
git grep -n -i -E 'ratelimit|rate-limit|rate_limit|limiter' -- ':!*.lock' ':!package-lock.json' git grep -l -i -E 'openai|anthropic|resend|sendgrid|twilio|stripe' -- '*app/*route.*' '*pages/api/*' 'api/*' '*server/*' '*supabase/functions/*'
A finding: a route that calls a paid API, sends email or SMS, or handles sign-in, sign-up, or password reset, with no rate limit on it (High: anyone can run up the owner's bill or flood users' inboxes).
E. CORS.
git grep -n -i -E "Access-Control-Allow-(Origin|Credentials)|origin:[[:space:]]*(true|['\"]\*['\"])|credentials:[[:space:]]*true"
A finding: an allowed origin of *, or origin: true (which echoes any origin back), on a route
that uses cookies or credentials: true (High). * on an edge function that requires an
Authorization header is common; list it as Low.
F. Input handling.
git grep -n -E 'request\.json\(\)|req\.json\(\)|req\.body|formData\(\)' -- '*app/*route.*' '*pages/api/*' 'api/*' '*server/*' '*supabase/functions/*' git grep -n -E '"(zod|valibot|yup|joi)"' -- package.json git grep -n -E 'dangerouslySetInnerHTML|\$queryRawUnsafe|\.unsafe\(|innerHTML[[:space:]]*='
A finding: request input that reaches raw SQL (High); one user's content rendered as HTML for other users (High); a write route that stores the request body with no schema check (Medium).
G. Tests and CI.
git ls-files .github/workflows git grep -n -E '"test"[[:space:]]*:' -- package.json git ls-files | grep -cE '\.(test|spec)\.[cm]?[jt]sx?$'
A finding: no test script, no test files, or no workflow (Low on its own, but it is why fixes get undone; Deliver adds the regression guards).
3. Report a leaked key at once
When check A, B, or C finds a live secret, tell the user before anything else, even before the cost question: which key (by provider and masked prefix), where it is (file and line, git history, or the shipped bundle), and that they must rotate it now in that provider's dashboard: create a new key, store it only in the server-side environment of their hosting, and revoke the old one. Deleting the key from the code does not undo the leak.
4. Ask the user only for
- The deployed URL to hunt, and whether it is staging, a preview, or production.
- Confirmation that they own the app or have written permission to test it.
Steps
The whole skill, in order. Each step names the section that holds its details.
- Check the setup. Run
vidbyte-cli --version,whoami, andruntime doctor, then check for a GitHub remote, Node.js, a public app URL, a linked GitHub account, a stored Browserbase key, and a shell that holds no production secrets. (Preflight) - Audit the repository, free and local. Detect the stack, then run checks A to G: secrets, database rules, server code that never checks the caller, abuse limits, CORS, input handling, and tests and CI. (Gather inputs, steps 1 and 2)
- Report a leaked key at once, before anything else, with the instruction to rotate it. (Gather inputs, step 3)
- Ask for the app URL and permission to test it. Stop if the user does not own the app and has no written permission. (Gather inputs, step 4)
- Get the first yes. Say what the hunt uses and what it does to the live app, and wait. (Cost and consent)
- Hunt the live app and its code with
vidbyte-cli agents bug-finder run. Check progress withruns list, and recover lost output withruns last. If the GitHub link or the Browserbase key cannot be set up, skip this step and deliver the audit alone. (Run) - Merge the audit and the bugs into one report, Critical first. (Read the result)
- Show the report and get the second yes for the items to fix. (Deliver, steps 1 and 2)
- Fix on a new branch, test first. Add the guard test and
.github/workflows/security-guards.yml, then run the project's full test suite and build. (Deliver, steps 3 and 4) - Mark each fixed bug with
vidbyte-cli agents bug-finder mark --state fixed. (Deliver, step 5) - Hand back what was fixed, what only the user can do, and what is still open. (Deliver, step 6)
CLI commands
Every vidbyte-cli command this skill runs or asks the user to run, in the order they come up.
Root options such as --json and --no-input always come before the command path.
| Command | Used in | What it does |
|---|---|---|
vidbyte-cli --version | Preflight | Prints the installed CLI version. This skill needs 0.3.0 or newer. Free and offline. |
vidbyte-cli --json whoami | Preflight | Shows the Vidbyte account behind the stored key. Exit 4 means no key is stored, which does not block this skill, because the Bug Finder needs no Vidbyte key. Free. |
vidbyte-cli --json runtime doctor | Preflight | Reports the working directory and which local agent hosts are available. Read its codex row. Free and offline. |
vidbyte-cli github link | Preflight, by the user | Links a GitHub account to the active profile through GitHub's device flow and stores the token in the system keyring only, so the Bug Finder can clone the repository, private ones included. It needs VIDBYTE_GITHUB_CLIENT_ID. The user runs it in their own terminal. Free. |
vidbyte-cli browserbase login | Preflight, by the user | Checks a Browserbase API key with Browserbase and stores it in the system keyring only. A web hunt opens one Browserbase session per agent turn with it. The user runs it in their own terminal, so the key never enters the chat. Free. |
vidbyte-cli --json --no-input agents bug-finder run | Run | Clones the GitHub repository into a temporary directory and hunts it one fresh Codex agent at a time. Each agent drives the deployed app in a Browserbase cloud browser and reports one bug or nothing new. Writes one Markdown and one JSON file and a ledger entry per bug, and prints one bug_finder.result document. No Vidbyte charge; it uses the user's Codex and Browserbase accounts, so it needs the first yes. |
vidbyte-cli --json runs list --agent agents.bug-finder | Run | Lists the agent runs still going on this machine. The hunt is finished when it is no longer listed. Free and offline. |
vidbyte-cli --json runs last --agent agents.bug-finder | Run | Shows the newest Bug Finder run on this machine. Its data.run.output holds the result document when the output was lost. Free and offline. |
vidbyte-cli --json --no-input agents bug-finder mark | Deliver | Sets one bug's state in the repository's ledger without starting an agent or using the network. fixed lets a later run report the bug as regressed; rejected, intended, and wont-fix silence it for every later run. Free and offline. |
This skill never runs vidbyte-cli billing top-up. The Bug Finder does not use the Vidbyte
balance.
Cost and consent
This skill asks for two separate yeses.
The hunt. It runs vidbyte-cli agents bug-finder run, which costs: No Vidbyte charge + Codex
usage on the user's own Codex account + Browserbase session time on the user's own Browserbase
account. The run buys no Vidbyte admission and sends nothing to the Vidbyte backend. With
--max-bugs 10 it starts at most 10 Codex agent turns, one per bug, each cut off after one hour,
and each turn opens one Browserbase session. It does not edit this workspace: it clones the
default branch into a temporary directory that it deletes, and it writes its ledger and one
Markdown and one JSON file per bug to the Vidbyte data directory. It does use the live app the way
a new user would: it signs up, fills forms with empty, oversized, and unusual input, and submits
twice. On production that creates real records and can send real emails.
Ask: "Do you own <URL>, or have written permission to test it? I'd like to run the Vidbyte Bug Finder against it and against the OWNER/NAME code on GitHub. Vidbyte charges nothing for it. It uses up to 10 Codex turns on the account Codex is logged in with and up to 10 Browserbase sessions on your Browserbase account, each up to an hour. It will sign up and submit forms on the site, so a staging or preview URL is safer than production. Shall I start it?" Wait for a yes.
The fixes. After the report, fixing costs nothing from Vidbyte, and the coding agent edits files in this workspace: new migration files, server routes, tests, and a CI workflow. List the items and the files that will change, ask which items to fix, and wait for a yes.
The Bug Finder takes no --idempotency-key, because it buys no admission, so there is no key to
make. Running it again starts a new set of Codex turns, so ask before every new run.
Run
vidbyte-cli --json --no-input agents bug-finder run \ --github-repo OWNER/NAME \ --project-type web \ --app-url "https://staging.example.com" \ --max-bugs 10
--github-repo OWNER/NAME: fromgit remote -v, without.git. The lowercase name also keys the bug ledger, so later runs never report the same bug as new twice.--project-type web: the agent drives the deployed app in a cloud browser while it reads the code behind each page. That is what catches pages that check sign-in only in the browser and endpoints that answer anyone, which a code read alone misses.--app-url: the staging or preview address the user approved.--max-bugs 10: twice the default of 5, because AI-built apps usually carry several holes at once. The run still stops early when an agent finds nothing new, and 10 is the spend bound the user agreed to: at most 10 turns and 10 browser sessions.
A run can take hours, because each turn may run for up to an hour. If the shell tool has a time
limit, start the command in the background. Check it with the free, offline
vidbyte-cli --json runs list --agent agents.bug-finder no more than once a minute; the run is
finished when the listing no longer shows it. If the output was lost, read it back with
vidbyte-cli --json runs last --agent agents.bug-finder (kind runs.run), whose
data.run.output holds the same result document.
Read the result
The result is one JSON document, {"schema_version": 1, "kind": "bug_finder.result", "data": {...}}.
| Field | Use it for |
|---|---|
data.bugs[] | One entry per bug this run reported: id (bug-0007), title, severity, state, outcome, artifact_markdown, artifact_json. |
data.bugs[].outcome | new and regressed are findings. known or suppressed means the run ended on a bug already in the ledger; do not list it as new. |
data.bugs[].artifact_markdown | Open it. It holds the steps to reproduce, expected and actual behaviour, the files and line ranges, the ranked root causes, the confidence, and a plain-words regression test. Use the last one for the guard test in Deliver. |
data.stop_reason | no_bug_found or known_bug_reported: the hunt ran out of new bugs. max_bugs_reached: more may remain; offer another run after the fixes are deployed. |
data.coverage | What the last agent exercised and what it could not reach. Copy the gaps into the report's "Not checked" section. |
data.commit_sha | The commit that was hunted. If it differs from the local HEAD, say so. |
data.repository | The lowercase OWNER/NAME to pass to agents bug-finder mark. |
Merge into one report
Put each audit finding and each Bug Finder bug into one table, sorted by severity and then by how
many users it can hurt. Map the Bug Finder's severities blocker and critical to Critical,
high to High, medium to Medium, and low and trivial to Low. When a bug and an audit finding
describe the same hole in the same file, keep one row, name both sources, and keep the bug id,
because mark needs it. Present a bug whose confidence is below 50 as unconfirmed.
## Hardening report: <app> (<date>) ### Do this first (only you can) 1. Rotate the Stripe secret key `sk_live_[masked]` found in `src/lib/stripe.ts:4`; it ships to every visitor. ### Critical | # | Hole | Where | Source | Proposed fix | |---|---|---|---|---| | C1 | Table `orders` has no row-level security | `supabase/migrations/20250301120000_init.sql:12` | audit | Enable RLS with an owner-only policy | | C2 | bug-0002: any signed-in user can open another user's invoice | `src/pages/Invoice.tsx:40` | Bug Finder | Check ownership in RLS and in the route | ### High ### Medium ### Low ### Not checked
Critical holes are the ones that expose data or let someone act as another user: leaked secrets,
missing or open RLS, open routes, and unverified webhooks. High holes are the ones that let someone
abuse the app or get around a check: browser-only authorization, getSession() on the server, no
rate limit on a costly route, and credentialed CORS. List them in that order.
Deliver
-
Show the report. A leaked key comes first, as in Gather inputs step 3. Show the report in the chat. Do not commit it: in a public repository it is a map of open holes.
-
Ask which items to fix (the second yes in Cost and consent). Suggest every Critical and High item that can be fixed in code. Before you fix a table's access rules, ask the user who should read and write it when the code does not make that clear: only the owner, every signed-in user, or everyone.
-
Fix on a new branch, such as
git switch -c harden/<YYYY-MM-DD>. For each item, write the test first and watch it fail, then fix, then watch it pass. Fixes by class:-
Secret in the client: move the call that needs it into a server route or edge function that reads a server-only variable, remove the
NEXT_PUBLIC_orVITE_variable from the code and from.env.example, and add rotation to the user's to-do list. -
Missing or open RLS: add a new migration,
supabase/migrations/<YYYYMMDDHHMMSS>_harden_rls.sql. Never edit a migration that has already been applied. For an owner-only table:alter table public.orders enable row level security; create policy "orders_select_own" on public.orders for select to authenticated using (auth.uid() = user_id); create policy "orders_insert_own" on public.orders for insert to authenticated with check (auth.uid() = user_id); -
Open route: verify the caller on the server (
supabase.auth.getUser(), notgetSession()), return 401 without a session, and return 404 when the record belongs to someone else. -
Browser-only admin check: enforce it on the server or in an RLS policy, and keep the UI check only as a convenience.
-
No rate limit: limit by user and by IP on the costly routes, with the limiter the app or its host already has. If one needs a new service account, add that to the user's to-do list instead.
-
CORS: allow only the app's own origins.
-
Unvalidated input: validate with the schema library the app already uses, or
zodwhen it has none. -
Stripe webhook: verify with
stripe.webhooks.constructEventand the webhook signing secret.
-
-
Add the regression guards, so a later AI edit cannot undo a fix without a red check:
-
tests/security/guards.test.mjs, run withnode --test tests/security/guards.test.mjs. It needs no dependencies, network, database, or secrets. It readsgit ls-filesand asserts that every table created insupabase/migrations/has row-level security enabled and never disabled later; that no policy forinsert,update,delete, orallusestrue; that no tracked file names a client-exposed variable holding a secret; and that no tracked file holds a secret-shaped value. It reports only file and line, never the value, and it skips its own file, which contains the patterns. -
One behavioural test per fixed route in the project's own test runner: a request without a session gets 401, and a request for another user's record is refused. Base each test on the "Regression test" section of the bug's Markdown artifact when the hole came from the Bug Finder.
-
.github/workflows/security-guards.yml. Replacenpm ciwith the command for the lockfile the project has, such aspnpm install --frozen-lockfile:name: security-guards on: [push, pull_request] jobs: guards: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 20 - run: node --test tests/security/guards.test.mjs - run: npm ci - run: npm run test --if-present
A builder that syncs with GitHub can commit straight to the default branch, so this workflow may not be able to block its edits. It flags them instead: a red check after an AI edit means a fix was undone. Run the project's full test suite and its build before you hand back.
-
-
Mark each fixed Bug Finder bug once its fix and test pass and the user agrees:
vidbyte-cli --json --no-input agents bug-finder mark --github-repo OWNER/NAME --bug bug-0002 --state fixedThe result is
kind: "bug_finder.bug.marked", anddata.bug.stateisfixed. A later run that finds the same bug again reports it asregressed. Useconfirmedfor a real bug the user wants to fix later. Userejected,intended, orwont-fixonly when the user says so, because those silence the bug for every later run. -
Hand back:
- What was fixed, with the files changed and the tests added.
- What only the user can do, in this order: rotate every leaked key; move secrets into the
hosting dashboard's server-side environment variables and redeploy; apply new migrations to
the live database (
supabase db push, or the Supabase SQL editor); turn on row-level security in the dashboard when the project has no migrations; review, merge, and deploy the branch. - What is still open, with its severity, and what was not checked.
- Next: after the fixes are deployed, run this skill again. The Bug Finder reads the same
ledger, so a fixed bug that fails again comes back as
regressed.
Troubleshooting
| Exit | Do this |
|---|---|
| 2 | Fix the flag the message names. Do not retry unchanged. Invalid bug finder run option: app url must be reachable from the internet, not a local address. means a local address: ask for the deployed URL. The GitHub repository is missing or not readable by the linked account. means a wrong OWNER/NAME, or the linked account cannot read the repository. |
| 4 | Stop. No GitHub account is linked for this profile. or The linked GitHub account needs to be linked again.: the user runs vidbyte-cli github link (references/setup.md step 3). No Browserbase API key is stored for this profile. or Browserbase rejected the API key.: the user runs vidbyte-cli browserbase login. Any other exit 4: references/setup.md step 2. |
| 5 | Stop. Follow "Out of credits" in references/troubleshooting.md. Never top up without the user's yes. The Bug Finder never charges the Vidbyte balance, so exit 5 here comes from another command. |
| 1 | A bug finder agent did not complete its hunt.: bugs recorded before it stay in the ledger, and the same command continues without reporting them twice, but it is a new set of Codex turns, so ask first. Browserbase refused to open a browser session.: the Browserbase plan limits session length, concurrent sessions, or minutes; the user checks the Browserbase dashboard. Anything else: references/troubleshooting.md. |
| Other | Read references/troubleshooting.md. |
Guardrails
- Never ask the user to paste an API key into the chat, and never print one.
- Never pass
vidbyte-cli billing top-up --confirmunless the user explicitly approved that top-up. - Run at most one file-writing Vidbyte agent in a workspace at a time.
- Audit and hunt only an app the user owns or has written permission to test, and hunt only the URL they approved. A yes for one URL does not cover another.
- When RLS is off or a policy or route is open, never read, count, or dump rows to prove it, and never call the app's database or API with its keys to test it. Report the misconfiguration and the file that defines it.
- Mask every secret everywhere: the chat, the report, commit messages, test fixtures, and pull
request text. Show at most the prefix (
sk_live_), the file and line, and the length. - When a key leaked, the first thing the user hears is which key and that they must rotate it now.
- Never disable auth, RLS, a policy, or a guard test to make a test or a build pass. Never add a
policy that uses
true, except aselectpolicy on data the user said is public; never skip or delete a failing guard; and never put a service-role key in a test or in CI. - Start the Bug Finder only from a shell that holds no production secrets.
- Do not push, merge, deploy, apply migrations to the live database, change dashboard settings, or rewrite git history yourself. Those are the user's steps, and Deliver lists them.