Simulating Full-Stack Infrastructure for Autonomous Agents

The Infrastructure Fidelity Gap

Most current AI agent benchmarks, such as SWE-bench or Terminal Bench, restrict agents to operating within a single codebase or container. This approach fails to replicate the reality of production engineering, where tasks involve managing distributed systems, handling network failures, and navigating organizational context like tickets and postmortems. Agents trained in isolated sandboxes lack the experience required to manage the blast radius of live traffic or reason through the cascading failures common in distributed databases.

Moving Beyond Single-Node Sandboxes

To improve agent autonomy, Emulated proposes moving from single-node containers to high-fidelity, multi-node environments that simulate entire companies. This involves provisioning real cloud resources, including VPCs, subnets, and security groups, while enforcing production-grade constraints like authentication, throttling, and billing. By simulating the full lifecycle of a service—from resource provisioning and deployment to health monitoring and DNS management—agents can learn to handle the messy, long-horizon realities of infrastructure work that cannot be captured by simple code diffs.

Challenges in High-Fidelity Simulation

Scaling these environments introduces significant technical hurdles, particularly regarding simulation speed and cost. Spinning up a full stack, such as an AWS Lambda-like environment, can take hours, which is incompatible with standard post-training pipelines. Furthermore, the "sim-to-real" gap persists even with real resources, as agents must still contend with the unpredictability of live customer traffic and the specific operational nuances of distributed systems like Kubernetes or etcd consensus clusters.