Building Long-Running AI Agents: Harnesses and Adversarial Evaluation
AI Engineergo watch the original →
the gist
Anthropic engineers explain that long-running agents require moving beyond simple self-evaluation to adversarial 'generator-evaluator' architectures, structured task decomposition, and persistent state management to avoid coherence drift.
The Evolution of Agentic Harnesses
Anthropic’s approach to long-running agents has shifted from simple model-based execution to sophisticated, multi-component harnesses. Early attempts at long-running tasks suffered from 'context rot' and 'context anxiety,' where models would lose coherence or rush to finish as they approached token limits. The team moved from single-session loops to architectures that treat the harness as a co-evolving partner to the model. As models like Opus 4.6 improved in planning and tool-use, the harness evolved to handle more complex orchestration, such as managing sub-agents and persistent state.
Adversarial Evaluation vs. Self-Correction
A core insight is that asking an LLM to critique its own work is often ineffective due to sycophancy. Instead, the team advocates for a 'GAN-style' adversarial architecture: a generator agent builds the product, while a separate, specialized evaluator agent uses tools like Playwright to test functionality and visual quality. This separation allows the evaluator to be tuned for harsh, objective criticism—a task LLMs excel at—while the generator focuses on execution. If the generator fails to meet the rubric, the harness can discard the attempt and restart, preventing the 'patching' cycle that often leads to low-quality output.
Structured Task Decomposition
To maintain coherence over hours of operation, agents must break vague prompts into granular, testable 'sprint contracts.' By using persistent artifacts (like JSON-based feature lists rather than markdown, which models are prone to overwriting), the harness tracks progress across fresh context windows. This modular approach ensures that if a specific feature fails, the agent can iterate on that isolated component without losing the state of the entire application.
Designing for Taste and Functionality
When building front-end applications, the team applies a rubric-based grading system covering originality, craft, and functionality. By providing few-shot examples of high-quality design, the evaluator agent learns to reject 'AI slop' (e.g., generic purple gradients). This allows the agent to iterate through multiple versions until the output aligns with the desired aesthetic and functional standards, a process that would be impossible in a single-pass generation loop.