Harness Design Impacts LLM Performance and Cost
Prompt Engineeringgo watch the original →
the gist
The performance of an LLM on complex tasks is often determined more by the surrounding harness—which manages memory, context, and reasoning state—than by the model's raw intelligence.
The Impact of Harness Architecture on Model Performance
The performance gap between a standard, minimal harness and an optimized, provider-specific harness is significant. When evaluating models on the ARC-AGI benchmark, the same model (GPT-6 Astra) scored 62.7% under a standard harness but reached 99.9% when using a harness that preserves reasoning state and utilizes history compaction. Standard harnesses often rely on rolling truncation, which discards the model's internal scratchpad and past actions, forcing the model to re-solve problems from scratch. In contrast, optimized harnesses maintain the reasoning state across turns and summarize history, allowing the model to retain discovered rules and implications.
Cost and Efficiency Trade-offs
Optimized harnesses are not only more accurate but also more cost-effective. Data from the ARC-AGI evaluation shows that an optimized harness runs 3.6 times faster and uses 49% fewer tokens than a standard interface. Furthermore, the relationship between reasoning effort and cost is non-linear. Under a standard harness, increasing reasoning effort can reduce the total number of API calls required, thereby lowering costs. However, under an optimized harness, the model's ability to retain memory often renders extensive re-reasoning unnecessary, leading to lower token usage across the board.
Harness Design for Real-World Applications
In practical developer scenarios, the choice of harness dictates the cost structure through prompt caching and context management. Minimal harnesses may appear efficient for short tasks but often fail or become prohibitively expensive on long-horizon tasks due to the lack of context management. Conversely, overly aggressive reasoning settings can cause smaller models to exhaust their output token budgets on internal thought processes without producing functional code. Developers should test models across multiple harness configurations to determine if performance bottlenecks stem from the model's limitations or the constraints imposed by the wrapper.