Deepsec Scans AI-Coded Repos for OWASP Risks

The Breakthrough

Vercel released Deepsec, an open-source agent-powered vulnerability scanner that identifies OWASP Top 10 risks in large repos through regex-based file filtering followed by LLM analysis.

What Actually Worked

• Run npx deepsec@latest init in the target project to generate an info.md file documenting the codebase, threat model, and project ID.
• Navigate to the deepsec directory, run npm install, configure the LLM agent (e.g., Claude via API key or local), and paste the init command into the agent for setup instructions.
• Execute scan --project-id <ID> to identify candidate files (e.g., 118 files in under 1 second) using regex for risks like insecure cryptography, SQL injection, XSS, and missing auth.
• Run process to batch-analyze files with the LLM (e.g., 29 batches, ~5-6 minutes wall time despite 50 minutes processing, $19 cost on Claude Opus), generating findings categorized as critical, high, medium, or bug.
• Use report to output report.md and report.json with detailed bugs, contexts, and fix recommendations; address via tools like OpenSpec (opsx fast-forward with prompt to fix issues from report, then opsx apply).
• Revalidate with pnpm deepsec@latest revalidate to check git history against prior findings (~2.5 minutes, $1.12 cost), confirming resolutions like fixed data loss bugs.

Context

AI-coded or 'vibe-coded' apps often contain unnoticed security holes from the OWASP Top 10, such as broken access control, security misconfigurations, supply chain failures, cryptographic failures, injections, insecure design, authentication failures, data integrity failures, logging/alerting failures, and exception mishandling. The author demonstrates Deepsec on personal projects, including one with silent DB wipes on server termination and another alpha app lacking rate limiting on a family signup key, enabling brute-force attacks. This static analysis tool catches code-pattern issues missed by humans, recommends fixes, and verifies resolutions, making it suitable for weekly CI/CD scans despite not covering runtime human-interaction risks.

Notable Quotes

• OWASP Top 10: "1. Broken access control... 2. Security misconfiguration... 3. Software supply chain failures... 4. Cryptographic failures... 5. Injection... 6. Insecure design... 7. Authentication failures... 8. Data integrity failures... 9. Logging and alerting failures... 10. Mishandling of exceptions."
• Scan output: "identifying the tech used in your project and then it has these different regex matchers... insecure cryptography, SQL injection, cross-site scripting risks, missing auth."
• Revalidation: "it's going to look at our git history it's going to look at the findings that it had and it's going to check to see if those issues have been resolved or not... marking both of those issues as fixed."

Content References

Deepsec catches subtle bugs like production data loss from abnormal server termination and brute-force risks from unthrottled signup keys in private apps.