Anthropic's Internal Data Suggests AGI Capabilities Are Already Here
Nate Herk | AI Automationgo watch the original →
the gist
Anthropic's internal report on recursive self-improvement shows their AI now handles open-ended research tasks with 76% success, suggesting that practical AGI—the ability to solve novel, complex problems autonomously—is already in use.
The Shift to Autonomous Problem Solving
Anthropic’s recent report, "When AI Builds Itself," indicates that over 80% of the code the company ships is now generated by its own AI models. The author defines AGI not as sentient or human-like, but as a system capable of taking an open-ended, poorly defined problem and autonomously researching, experimenting, and executing a solution without human intervention. By this metric, the company's internal data suggests that AGI is no longer a future prospect but an active reality within their development workflows.
Performance Metrics and Scaling
Anthropic categorizes tasks by difficulty, with "open-ended" problems representing the most complex tier where no clear specification exists. The report highlights several key performance shifts:
- Success rates for open-ended coding tasks rose from 26% to 76% in six months.
- Task duration capacity has increased significantly, with models now handling 12 to 16-hour continuous work sessions, effectively doubling their capability every four months.
- In decision-making benchmarks, AI agents outperformed human researchers in choosing the next optimal step in research projects 64% of the time, up from 51% in November.
- On specific optimization tasks, newer models improved code performance by 52x, compared to a 3x improvement from models a year prior.
The Alignment and Control Dilemma
Anthropic outlines three potential trajectories for AI development: a plateau in progress, continued human-directed research, or the emergence of AI capable of building its own successors. The primary risk identified is that if current models contain minor alignment flaws, these errors will compound exponentially as the AI builds its own future versions. This creates a scenario where the systems become both more powerful and less understandable to human operators. While Anthropic advocates for a slowdown to address these alignment risks, they acknowledge that the competitive incentive to win the AI race makes a verifiable, global pause nearly impossible to enforce, as training runs are significantly easier to conceal than traditional military assets.