Scaling VS Code: From Monthly to Weekly Releases with AI

The Shift to Weekly Velocity

VS Code moved from a monthly to a weekly release cycle to keep pace with the rapid evolution of AI models and to reduce the risk associated with large, infrequent batch deployments. The team found that monthly planning cycles were no longer sufficient for the AI-native development environment, where competitive pressure and model updates necessitated faster iteration. However, this increased velocity led to a 3x increase in issues and pull requests, requiring a fundamental shift in their engineering systems to prevent quality degradation.

AI-Native Engineering Inner Loop

To maintain quality at speed, the team developed bespoke agentic tooling. A key innovation is the use of component-level testing where UI components are extracted from the main product, allowing agents to validate changes in isolation. Engineers use agents to perform "conversational PRs" where the agent iterates on UI changes—such as color swaps or layout adjustments—and verifies them against visual fixtures. This reduces the need for full-product builds and manual testing, enabling developers to validate changes via GitHub mobile or local agent loops.

Automated Quality and Triage

With over 100 commits landing daily, the team relies on "skills"—encapsulated expert knowledge—to automate performance benchmarking and bug detection. By encoding performance expectations into these skills, the team democratized access to expert-level analysis, allowing any developer to run performance checks on complex renderers. Furthermore, the team implemented automated triage systems that use semantic matching to handle duplicate issues and telemetry-driven error analysis to generate potential fixes, significantly reducing the burden on human maintainers.

Prototyping as Documentation

Traditional specification documents were replaced by "living prototypes." Engineers and PMs collaborate by building functional prototypes within the VS Code codebase. These prototypes serve as the "spec," allowing stakeholders to interact with the feature and identify edge cases early. This approach reduces the time spent on abstract documentation and ensures that the final product implementation is aligned with user needs from the start.

Key Takeaways

• Decouple UI components: Extract components from the main product to allow for rapid, isolated agent testing and visual validation.
• Encapsulate expert knowledge: Turn senior engineer insights into "skills" that agents can execute, allowing the whole team to perform complex tasks like performance tuning.
• Replace specs with prototypes: Use functional prototypes as the primary communication tool for new features to uncover edge cases early.
• Automate the triage backlog: Use semantic matching and telemetry to automatically group and address incoming issues, preventing the backlog from becoming a bottleneck.
• Implement staged rollouts: Move to weekly releases but use staged rollouts to mitigate the risk of small, frequent updates.