Seven Defects We Check First in Any Vibe-Coded App (proposed; old title in the notes)
What they find isn't random. AI coding tools make the same kinds of mistakes because they get asked the same kinds of questions: make this screen work, fix this error. Each answer works on its own. Nobody asked for the part that holds the answers together, so it isn't there.
When we took over Oria, an oral history platform built fast with AI coding agents, the list read like this: broken authorization, organizations able to see each other's data, hardcoded secrets, no tests and no deploy process. The full case is in We inherited an AI-generated codebase. This is the wider checklist: seven defects we look for first, and how to check each one before a stranger does.
1. Authorization that lives in the browser
The paywall is a React component: the upgrade button is greyed out for free users, but the API behind it never checks the plan. Anyone with the browser console gets Pro for free.
Something close to this ended Enrichlead in March 2025. The founder had said publicly that the product was built with Cursor, with zero hand-written code. Within days, people were bypassing the subscription and maxing out his API keys, and the project was shut down.
2. Database rules left open
AI tools scaffold the database with permissive rules so development goes smoothly, and nobody tightens them before launch. With Supabase or Firebase the browser talks to the database directly, so an open rule means anyone holding the public key can read the table. In early 2026 Wiz found Moltbook's production database open to anyone for reading and writing, because row-level security was missing.
At Oria the same class of mistake meant organizations could see each other's data. For a platform holding family stories, it was the worst finding.
3. Secrets in the repository
A .env file committed to git, or a live Stripe secret key sitting in source. The fastest way to make a generated integration work is to paste the key where the code needs it, and Oria had hardcoded secrets too.
Moving the key into an environment variable afterwards doesn't fix it, because the old value is still in the git history. Treat every key that was ever committed as leaked. We rotate keys before we change anything else.
4. No migrations
The schema is whatever the agent decided last Tuesday. Columns named data_json and temp_status_v2, and a staging database that quietly differs from production. You can't safely add a feature to a database you can't reproduce. The fix is boring: capture today's schema as a baseline migration, then route every change through migration files.
Check your repo against the seven defects
Describe your stack and who built what. ChatGPT will rank the seven defects by how likely they are in your codebase and give you a quick check for each.
“|”
5. Two auth systems at once
NextAuth on some routes, a hand-rolled JWT cookie on others. The agent added each one when a prompt needed it, and nothing ever removed the old one. Users end up with two sessions, and logging out of one leaves the other alive. Pick one system, and delete the other once everything has moved to it.
6. AI features with unguarded tool access
The product's chat assistant can call internal tools, like looking up an account or sending an email. Nothing limits what it does when a user, or a document the user uploaded, tells it to ignore its instructions. If your AI feature can reach the database, assume someone will try to talk it into reading tables it shouldn't. Give it the permissions of the user it's acting for and never more, and check every tool call on the server.
7. No tests and no monitoring
The first sign of a bug is a screenshot from a confused customer. Oria had no tests either. You don't need hundreds of generated tests: start with login and the one flow the product exists for.
What we do about it
Throwing the product away is almost never the answer. The screens and the product decisions behind them usually survive. What gets replaced is the foundation underneath. At Oria that meant rebuilding authorization, tenant isolation, the upload and transcription pipeline and the deploy path, while the product on top kept serving its partner institutions.
Vibe coding is a fine way to find out whether anyone wants the product. It's a poor way to keep one running once they do.
Recognize some of these in your repo?
The Vibe Code Rescue audit is $500: a written audit in 48 hours covering these seven places, then a flat-price plan for what to keep, fix or replace. Send your email and we'll set up a 30-minute call within 24 hours.
The seven checks
- Call a paid endpoint with a free account. Does it refuse?
- Query your main tables using only the public key, while logged out. Do you get rows back?
- Search the git history for keys, for example
git log -p | grep -iE "sk_live|service_role|api_key". Is anything there? - Can you rebuild the production schema from migration files alone?
- How many ways can a user sign in, and does logging out end all of them?
- What can your AI feature do if a user tells it to ignore its instructions?
- Is there a test for login, and does anyone get alerted when production throws errors?
Related reading
- We inherited an AI-generated codebase. Here is what we found: the full Oria rescue, finding by finding
- Burning tokens is not shipping: why more AI output isn't the same as more product
- How we use Claude Code in production: how we build with AI agents without handing them the architecture
Built with a specific tool? Cursor, Lovable, Bolt, Claude Code. The service: Vibe Code Rescue.
Enjoyed this article? Share it with others
Related Posts
The Transfer Pattern: Five Ways an Agency Project Goes Quiet (proposed; old title in the notes)
The deadline slipped twice and the Slack went quiet. Stalled agency projects tend to stall the same way, and the signs show up months early. Here are the five patterns, a five-minute check, and what to do when three of them match.
You're Allowed to Fire Your Dev Team. Here Is the Email.
Founders keep paying agencies that stopped shipping because firing them feels harsh. They were paid and they didn't deliver. Here is the email to send, the four things to have ready first, and what to do the morning after.
Our 14-Day Transfer-and-Ship Playbook
What happens in the two weeks between letting the old team go and a working production deploy. Day by day: what the audit checks, what you own by day 3, what ships first, and the three steps founders most often skip.