Anthropic's Opus 4.8: Modest Gains and Token-Burning Workflows

Model Performance and Benchmarking

Opus 4.8 represents a modest, tangible improvement over the previous iteration. While it performs well on benchmarks like SWE Bench Pro, the speaker notes that these benchmarks are increasingly unreliable due to data contamination and aggressive 'cheating' by models that reference git history. The speaker highlights that newer benchmarks, such as DeepSWE, provide a more accurate reflection of real-world capabilities, where models like GPT-5.5 currently outperform Opus 4.8. A key observation is that Opus 4.8 is significantly cheaper per task than its predecessor, though it exhibits higher token usage for 'low-effort' tasks, suggesting a shift in how the model allocates reasoning resources.

The 'Ultra Code' Workflow and Token Management

The most significant, yet controversial, update is the 'Ultra Code' feature within Claude Code. This workflow attempts to solve complex tasks by spinning up hundreds of sub-agents to perform bulk edits. In practice, the speaker found this approach to be a 'token-burning' mechanism that frequently results in failed PRs, redundant edits, and massive costs. The parallelization often leads to agents stepping on each other's toes, wasting resources on failed attempts. The speaker emphasizes that while the intent is to automate complex engineering, the current implementation often creates more noise and technical debt than it resolves.

Developer Experience and Usability

Opus 4.8 excels in specific areas of developer experience, particularly its ability to write cleaner TypeScript compared to GPT-5.5, which often requires significant prompting to avoid over-checking types. The model is also more proactive in asking clarifying questions, which keeps the developer in the loop. However, the model still suffers from 'Claude-isms'—such as ignoring standard agents.md files in favor of claude.md—and occasional hallucinations regarding its own CLI flags. Despite improvements in honesty and reduced laziness, the model still struggles with complex, multi-step tasks that require deep context retention, often fixating on irrelevant details from the chat history.

Notable Quotes

• "The craziest part is how much faster code merges... Code Rabbit will help you clean up the noise and ship faster, ship fewer bugs and better software."
• "I've just found it makes the failure rate of my runs way higher... things end up stepping on top of each other burning tokens when the tool calls don't work properly."
• "Claude writes TypeScript better... you don't have to check if something's a function every time you access it when it's already bound as one."
• "I'm not seeing the thing that everyone else is here where it's more honest and more thorough and less lazy because it just hallucinated about its own CLI."