Scaling AI Agents in Global Shipping Operations

The Breakthrough

The core discovery is that agent reliability at scale is not a product of model size or prompt engineering, but of building a rigorous, iterative refining loop that converts tribal knowledge into bounded, executable code through over 100,000 individual corrections.

What Actually Worked

• SOP Translation: Converted legacy Standard Operating Procedures (SOPs) from human-readable screenshots into machine-executable formats requiring explicit preconditions, decision identifiers, backend calls, validation logic, and recovery paths.
• Evidence-Based Triage: Implemented a feedback loop where failure traces serve as shared evidence between human experts and engineers, allowing for the clustering of errors into heat maps that prioritize engineering effort.
• Production Guardrails: Replaced vague instructions like "please be careful" with hard constraints, including classifier evals, right-gate checks, and mandatory human-in-the-loop approvals for critical paths to ensure safe hand-offs.
• Composite Tooling: Aggregated successful execution sequences into reusable, composite tools that distill bloated legacy responses into precise function calls, allowing for consistent rollouts across hundreds of countries.

Before / After

• System Scale: The system currently runs over 200 concurrent instances in production.
• Improvement Effort: The team performed over 100,000 corrections over a nine-month period to reach current reliability levels.
• Development Ratio: The corpus of procedures outweighs the runtime code by a ratio of approximately 20 to 1.
• Correction Cost: A single block of improvement on the team's heat map typically requires one to two months of effort from the entire team.

Context

Maersk faced the challenge of automating the "long tail" of shipping exceptions, where processes were previously documented only as screenshots. Because shipping steps vary significantly by country and depend on complex legacy backends, the team found that standard agent loops failed without a dedicated infrastructure to manage the "tribal dungeons" of operational knowledge. The resulting architecture prioritizes a harness that makes unsafe actions impossible, moving away from "vibe coding" toward a systematic, spec-driven approach to agent reliability.

Notable Quotes

• "A correction only counts once it becomes an executable change, which is the line between an opinion and a production fix."
• "Discovery needs agent freedom, production needs a cage, and a harness exists to make the dumb mistakes impossible rather than to give the model more room."
• "The agent loop is not the system. The refining loop around it is."