Implementing Shopify's AI Agentic Coding Workflow

The Orchestrated Agentic Loop

The breakthrough here is the implementation of a rigid, multi-stage gatekeeper workflow for AI coding agents, inspired by Shopify's internal 'Helix' tool. Instead of allowing a single agent to build a feature, the process uses an 'Orchestrator' skill to manage sub-agents, ensuring that no code is finalized unless it passes four distinct validation gates. This prevents the common issue of agents hallucinating or shipping incomplete work by forcing them to adhere to a strict checkpoint-based progression.

The Four-Gate Quality System

• Checkpoint Planning: A 'Checkpoint Planner' agent breaks large features into small, sequential tasks stored in a JSON file. A custom web viewer allows the human to review these tasks before any implementation begins.
• Gate 1: Behavior: Before writing feature code, the agent generates unit tests. The orchestrator runs these tests; the gate only passes if all tests pass, ensuring functional correctness without requiring a browser.
• Gate 2: UI Review: The system compares the implementation against an HTML prototype and a design.md file. Two separate Claude sub-agents independently review the visuals and behavior, returning a combined verdict on whether the UI matches the design specification.
• Gate 3: Adversarial Review: An adversarial agent assumes the code is flawed and searches for issues, while a 'fixer' agent addresses the feedback. This loop continues until the adversarial agent approves the code.
• Gate 4: Human-in-the-Loop: The final stage requires human verification. Feedback provided here is converted into new checkpoints and saved to a 'learnings' file, which is ingested by agents in future cycles to prevent recurring errors.

Implementation Mechanics

To prevent the agent from quitting prematurely, the workflow uses a custom hook that throws an exit code 2 whenever the agent attempts to stop before all gates are cleared. This forces the agent to re-evaluate the task status and continue working until the criteria are met. By using sub-agents for each task, the system maintains fresh context windows, preventing the degradation of performance that occurs when a single agent attempts to manage a large, complex codebase in one session.