Testing Claude Opus 5.5 Effort Levels: A 3D World Build
Nate Herk | AI Automationgo watch the original →
the gist
Nate Herk tests Claude Opus 5.5's performance across six effort levels by tasking it to build a 3D virtual conference, finding that higher effort levels do not guarantee better results and often lead to diminishing returns in quality and cost.
The Experiment: Scaling AI Effort
Nate Herk tested Claude Opus 5.5 by providing a consistent, complex prompt: building a 3D, explorable virtual conference environment based on 105GB of existing event assets. The goal was to evaluate how different "effort levels"—Low, Medium, High, Extra, Max, and Ultracode—impacted the final output, runtime, cost, and overall polish.
Performance vs. Cost Analysis
The results revealed a non-linear relationship between effort and quality. While the "Low" effort level produced a glitchy, unbranded prototype, the "Medium" and "High" levels provided the most functional and aesthetically pleasing results. Surprisingly, the "Max" and "Ultracode" levels, despite significantly higher token usage and longer runtimes, introduced more bugs, navigation issues, and failed to improve upon the "High" effort output. The "High" effort level stood out as the most balanced, offering a polished experience with working features like closed captions, functional UI elements, and stable navigation.
The Role of Verification and Interaction
A critical observation was the model's reliance on self-verification. The "Extra" effort level performed the most checks (34), yet still struggled with navigation bugs. Notably, almost every effort level failed to ask clarifying questions, with the "High" effort level being the only one to engage the user for input. This highlights a tendency for the model to make assumptions rather than seeking guidance, which can lead to wasted compute time on incorrect architectural decisions.
Key Takeaways
- Diminishing Returns: Higher effort levels do not equate to higher quality; the "High" setting often hits a sweet spot for complex creative tasks.
- Cost Efficiency: Max and Ultracode levels significantly increase API costs and latency without proportional gains in functionality or design.
- Proactive Communication: Models rarely ask for clarification. If the output is critical, users should explicitly prompt the model to ask questions before proceeding with complex builds.
- Verification Overhead: More automated checks do not guarantee fewer bugs; sometimes, the model's attempt to "fix" its own code creates new issues.
- Start Mid-Range: When using Claude's effort settings, start at "Medium" or "High" and scale only if the initial output lacks specific necessary features.