Production audit, sample report
Eight findings. One that cannot wait.
This app takes real payments and holds real customer data, and its payment endpoint can be told a payment succeeded when it did not. That gets fixed first. The rest of the report works down from there, and every finding can be checked with the verification test at the end of it.
This is an illustrative sample. It is a composite example built from patterns seen across AI built apps. It is not a real customer and not a real app. The app described is a subscription app built with Lovable and Supabase, with Stripe billing and hosting on Vercel. The audit it shows was read only in the way that matters: checks ran on the public app and a test account the audit created, no settings, code, or real customer data were changed, and the test account and its data were deleted when the report shipped.
Summary
The app works, and customers are paying for it, which is exactly why the first finding matters. The Stripe webhook endpoint accepts events without verifying their signatures, so the payment confirmation path can be spoofed with a single request. Behind that sit two data problems: row level security is off on the table that holds orders, and a service role key ships in the browser bundle. Either one hands a stranger the customer list. The remaining findings are the quieter kind: signups leaking away through preview links and spam folders, a backup that has never been restored, and no monitoring, so the owner would learn about most of this from a customer.
The findings
Sorted by severity. Each one names the check it came from, what we observed, how to reproduce it, what the fix is, what fixing it yourself looks like, and the test that proves the fix held.
The Stripe webhook endpoint accepts events without checking their signature.
What we observed. The endpoint that marks orders as paid responds to a plain POST request with a payment shaped payload and no Stripe signature header. When we sent one naming an unpaid test order on our own test account, created for the audit, the order flipped to paid and the test account gained subscriber access. No card was charged and nothing in Stripe recorded an event.
Why it matters. Anyone who finds this endpoint, and it sits at a guessable address, can give themselves your paid product for free, at scale, whenever they like. This is the finding that turns into a very bad weekend, and it is why it is at the top.
How to reproduce.
- On your own test account, place an order without paying, so it sits in an unpaid state.
- Send a POST request to the webhook address with a payment succeeded payload naming that order, and no signature header.
- Refresh the order. In this app it shows as paid, and the account has full access.
The fix. Verify the Stripe signature on every event before acting on it, using the endpoint's signing secret held as a server side environment variable, and reject anything that fails. Then confirm the endpoint also ignores events for orders it cannot match.
Fix it yourself. This report gives guidance and the verification test rather than a copy and paste prompt for this finding, because a wrong change here can break live billing or look fixed while staying open. The shape of the fix: read the raw request body, call Stripe's signature verification with the event, the signature header, and the signing secret, and only then run the existing order update code. Stripe's own webhook documentation walks through it for each framework.
Verify it. Repeat step 2 after the fix. The endpoint should answer with a 400 error and the order should stay unpaid. Then complete a real test mode payment and confirm that the genuine event still upgrades the order.
Row level security is disabled on the table that holds customer orders.
What we observed. The app ships its Supabase anon key in the browser bundle, which is expected and safe on its own. With row level security off on the orders table, that key alone was enough to list rows without logging in, including names and email addresses belonging to other customers. Nothing was modified and no credentials were guessed.
Why it matters. Anyone who opens your site can pull the key out of the page in under a minute, and with it, your entire customer list. This is the leak that turns into a breach email to every user you have.
How to reproduce.
- Log out and open the browser console on the home page.
- Run the read query included in the full report, which uses only the public anon key.
- If it returns rows from the orders table, the finding is live. In this app it returned the full table.
The fix. Enable row level security on the table and add policies scoped to the signed in user's ID, so an account can read and write only its own orders. Any part of the app that legitimately needs cross account access should do it from the server, never from the browser.
Fix it yourself. Guidance and the verification test, rather than a prompt, for the same reason as Finding 01: a policy can look right and still leak, or lock every customer out of their own orders. Write the policies against the table's owner column, test with two accounts before you call it done, and keep a note of the policies you added so the next change starts from what is actually there.
Verify it. Run the same console query logged out. It should return nothing. Then log in as two different test accounts and confirm each one sees exactly its own orders and cannot open the other's by changing the ID.
A Supabase service role key is present in the code the browser downloads.
What we observed. Searching the JavaScript bundle the site serves turned up a key that decodes as a service role key for this app's Supabase project. The service role bypasses row level security entirely, so this key on its own is full access to the database, held by anyone who downloads the page.
Why it matters. This key is worse than Finding 02, because fixing the table policies does not close it. The key has to be treated as leaked from the moment it shipped: rotated first, then removed from the client code, then checked for in the repo history.
How to reproduce.
- Open the app's home page and view the loaded JavaScript files.
- Search the bundle for a long token beginning with the characters Supabase keys start with.
- Decode the middle section of the token. In this app it names the service role.
The fix. Rotate the service role key in the Supabase dashboard first, so the leaked one stops working. Then move whatever used it into server side code that reads the new key from an environment variable, and confirm the app only ever ships the anon key to the browser.
Fix it yourself. Guidance and verification here too. Rotation order matters: rotate before you clean up, or the old key keeps working while you tidy. After rotating, search the bundle and the repo history for the old key, and expect to update the one or two places that used it server side.
Verify it. Try the old key against the project address. It should be refused. Then search the freshly built bundle for any service role token and confirm only the anon key remains.
Confirmation and password reset links point at a preview domain.
What we observed. We signed up with a fresh email address. The confirmation email arrived, and its link pointed at the builder's preview domain rather than the live one. On that preview, the app is an older build, and the account we confirmed there did not exist on the live site. Password reset emails carry the same wrong address.
Why it matters. A share of your new users are landing on the wrong version of your app, or giving up at a page that does not know them. It looks like weak demand from the inside. It is actually a settings page that never got updated when the real domain went on.
How to reproduce.
- Sign up with an email address that has never been used on the app.
- Open the confirmation email and look at where the button link points before clicking.
- Repeat with the password reset flow. In this app both point at the preview domain.
The fix. In the auth settings, set the site URL to the live domain and make sure the redirect allowlist contains the live domain and its auth return paths. Remove the preview and localhost entries once the live ones are confirmed working.
Fix it yourself. The settings change itself happens by hand in the dashboard, above. Here is a safe prompt you can paste into your builder afterwards. It only searches and reports, and it is written so it cannot change anything:
Search the codebase for anywhere an email confirmation, magic link, or password reset URL is constructed. Show me every place a domain or redirect address is hardcoded, especially any preview domain or localhost. Do not change any files yet, just list what you find with the file names.
Verify it. Sign up with another fresh email. The confirmation link should point at the live domain, the account should work there immediately, and a password reset should complete on the live site from start to finish.
Backups exist, but a restore has never been tested.
What we observed. The database platform takes nightly backups, and the owner believed that covered recovery. Nobody has ever restored one. There is no note of when backups run, how far back they reach, or how long a restore would take with the app's current data in it.
Why it matters. An untested backup is a hope, not a backup. The day it is needed is the worst day to discover that it is empty, partial, or slower than the business can stand. AI built apps carry an extra version of this risk, because a builder session can rewrite or drop a table in a way that syncs everywhere at once.
How to reproduce.
- Ask one question: when was the last restore, and where is it written down?
- Check the backup list in the database dashboard and note the oldest restore point.
- In this app, both answers came back as intentions rather than records.
The fix. Restore the latest backup into a separate throwaway copy, check the main tables against their expected row counts, log in against the copy, and write down how long it took. Then put the test on a monthly rhythm so the answer never goes stale.
Fix it yourself. Guidance and the verification test, rather than a prompt: restores touch your only copy of customer data when they are done wrong, and they should always target a throwaway copy, never the live database. On the Ground Crew plan this test runs every month and the result lands on the health card.
Verify it. The restore completes into the throwaway copy, the row counts match the backup time, and a test login works there. If any of the three fails, the finding is still open, whatever the backup list says.
Auth email sends from a default shared domain, with no SPF, DKIM, or DMARC behind it.
What we observed. Confirmation and reset emails arrive from the platform's default shared sending address, not from this app's domain. The domain has no SPF, DKIM, or DMARC records at all. Of our three test signups across two mail providers, one confirmation landed in spam.
Why it matters. Every point of signup friction costs users, and spam placement is invisible from the founder's chair. It also makes your most important email, the one that lets a new customer in, look like it came from somebody else.
How to reproduce.
- Sign up with a fresh address at a major mail provider and see where the confirmation lands.
- Look at which address sent it.
- Check the domain's DNS for SPF, DKIM, and DMARC records. In this app there are none.
The fix. Send auth mail from your own domain through a proper sending service or your platform's custom domain option, and publish SPF, DKIM, and DMARC records for it. Start DMARC in monitoring mode so you can see who else sends as your domain before you enforce anything.
Fix it yourself. The DNS records are the safe configuration kind of fix. Your sending provider will generate the exact values. This prompt is a safe one to paste into your builder or an assistant while you work:
I am setting up email authentication for my domain. Explain in plain steps where SPF, DKIM, and DMARC records are added in my DNS host, what each record does, and how I check they are working after I add them. Use placeholder values I will replace with the ones my email provider gives me. Do not invent record values.
Verify it. Send a fresh confirmation to an address at a major provider. It should arrive in the inbox, from your domain, and a DNS lookup should show all three records published.
No error monitoring, so production crashes are invisible.
What we observed. When we deliberately triggered an error in the core flow as a test user, the app showed a blank panel and nothing anywhere recorded it. There is no error tracking in the app, so the only crash reports this business gets are the ones customers bother to send.
Why it matters. Most users who hit a crash say nothing. They leave, and the founder reads the silence as disinterest rather than a bug with a name and an address.
How to reproduce.
- As a test user, interrupt the core flow in a way that produces an error.
- Note what the screen shows.
- Ask where that error was recorded. In this app, nowhere.
The fix. Add error tracking, a free tier of a service like Sentry is plenty at this size, wired so a crash in production sends an alert with enough detail to find it again.
Fix it yourself. A safe prompt for the wiring, once you have created the account and have your project key:
Add error tracking to this app using the project key in my environment variables. Capture unhandled errors and failed page loads, send them with the app version, and show the user a simple message with a retry button instead of a blank panel when something fails. Do not change any business logic.
Verify it. Trigger the same error again. This time it should appear in the tracking dashboard within a minute, and the user should see a message rather than a blank panel.
The certificate expires in 24 days and nothing will warn anyone.
What we observed. The site's certificate is valid today and expires in 24 days. It was uploaded manually rather than issued by the host, so it will not renew itself, and there is no expiry alert anywhere. The hosting dashboard shows the date if someone goes looking, which nobody does.
Why it matters. An expired certificate turns your site into a full screen browser warning that tells visitors the connection is not private. Founders usually find out from a screenshot a customer sends, after some hours of lost traffic.
How to reproduce.
- Click the padlock in the browser address bar and read the certificate dates.
- Check whether renewal is automatic in the hosting settings.
- Ask who gets told if it gets close. In this app, nobody.
The fix. Replace the uploaded certificate with one the host issues and renews automatically, or renew this one manually well before the date, and add an expiry alert at 30, 14, and 7 days routed to an inbox a person reads.
Fix it yourself. This is a hosting settings change, most of it in the dashboard. A safe prompt for the alerting piece:
Set up a certificate expiry check for my domain that runs weekly and emails me when fewer than 30 days remain before expiry. Show me where the alert goes and how I change the address later. Do not change my DNS or hosting settings.
Verify it. The renewal setting shows as on, and a test alert reaches the right inbox. After the next renewal, the padlock dates should have rolled forward without anyone touching anything.
What is fine
Plenty passed, and it deserves naming so it does not get changed by accident. Prices in checkout match the prices on the marketing pages, in amount and currency. Test and live Stripe keys are not mixed up. HTTPS is enforced everywhere and HTTP redirects to it. The core flow completes start to finish as a free user, on a phone sized screen as well as a desktop. Privacy policy and terms exist and are reachable from the footer. Two of the six checklist groups, transport basics beyond the certificate and the signup walk itself once you are inside the app, were in decent shape. The problems here are concentrated in configuration and access rules, not in the product, and that is the common pattern: the app is fine, the layer underneath it is unfinished.
Recommended order
- Finding 01, the webhook. Money first. Until the signature check is in, the paid tier is open to anyone who asks it nicely.
- Finding 03, the leaked key. Rotate it before fixing Finding 02, because the service role key walks straight past whatever policies get written.
- Finding 02, row level security. With the key rotated, the policies become the real boundary, and the two account test proves them.
- Findings 04 and 06, the signup leaks. Together they decide how many of the people you attract ever become users.
- Findings 05, 07, and 08, the recovery layer. The restore test, the error tracking, and the certificate alert turn future surprises into scheduled work.
The standing option, and the guarantee
The Ground Crew plan at $99 per month re-runs this checklist monthly and sends the health card you can preview in the sample monthly report, tests a backup restore every month, watches uptime, errors, and certificate expiry in between, and includes small fixes under rules you set. It is quoted once here and never chased. Fix sprint work quoted from a report like this one carries a 30 day guarantee: if a fixed finding comes back within 30 days, the refix is free.