Securing AI Skills as Supply Chain Dependencies
AI Engineergo watch the original →
the gist
Nubank implemented a hybrid deterministic and LLM-based scanning tool called Skill Vector to audit AI skills before they reach the internal developer marketplace, treating them as high-risk supply chain dependencies.
The Breakthrough
Nubank developed Skill Vector, a security gate that treats AI skills as supply chain dependencies by enforcing automated, multi-stage vetting before any skill is published to the internal developer marketplace.
What Actually Worked
- Hybrid Scanning Architecture: The system combines deterministic regex-based checks for known malicious patterns with LLM-based analysis to evaluate the context and intent of the skill instructions.
- Local-to-CI Workflow: Developers are provided with the same scanning tools locally to iterate on their skills, which are then re-verified during the CI/CD pipeline upon pull request submission.
- Standardized Reporting: All findings are exported in the SARIF format, allowing the security team to ingest results directly into their existing vulnerability management program for tracking and remediation.
- Contextual Risk Assessment: The system specifically flags dangerous behaviors such as hardcoded token retrieval, unauthorized shell command execution, excessive permission requests, and insecure file modifications.
Before / After
- Total Skills Scanned: Over 2,000.
- Total Risks Identified: More than 1,500.
- Remediation Rate: Approximately 1,000 risks were remediated immediately following the initial scan results.
Context
As AI-assisted coding becomes standard, AI skills—instructions or context bundles provided to models—behave like software dependencies but often lack the security oversight applied to traditional libraries. In a regulated financial environment, this creates significant risks, including data exfiltration and unauthorized system access. Nubank’s approach forces all internal and third-party skills through a centralized marketplace, ensuring that no code reaches a developer's environment without passing the Skill Vector gate.
Notable Quotes
- "Although they look like configuration, they behave like supply chain dependence like, for example, libraries and others."
- "We should be protecting the whole workflow, not only the code that's being generated."