Refactoring Legacy Codebases with AI Agents

The Case for Proactive Refactoring

Denys Linkov’s team at Wisedocs faced significant technical debt across ten disparate repositories, which hindered their ability to process massive medical claim PDFs. Despite the rapid advancement of AI coding tools, the team opted to perform a six-month manual refactor to consolidate these into a monorepo rather than waiting for AI to reach a point where it could handle the entire migration autonomously. The decision proved successful, resulting in a permanent increase in commit velocity and enabling the team to ship features in under a week that previously required months of effort.

AI Agent Reliability and Benchmarking

Linkov audited the effectiveness of AI agents by comparing their performance on a specific refactoring task over time. Early attempts using o3 required three hours of manual intervention and still resulted in ten major errors. Newer models like Claude 3.5 Sonnet and Opus 4.8 significantly reduced this effort, with Opus capable of completing the task in a single pass. However, attempts to have frontier models (e.g., GPT-5.5) perform the entire refactor zero-shot resulted in "AI psychosis," where the model generated 2,000 lines of scaffolding code while failing to implement the core logic, explicitly admitting in its output that it had skipped deployment and bootstrap commands.

Strategic Guardrails for Agentic Workflows

To avoid wasting compute and developer attention on failed agent runs, Linkov emphasizes the importance of evaluating model performance at the 80% to 99% success rate threshold rather than the 50% mark commonly cited in benchmarks. Effective agentic development requires:

• Constructing well-defined specs and plans before handing tasks to agents.
• Implementing sub-agents for specific criteria evaluation to prevent hallucinated features.
• Using modern harnesses that support plan calls, shell command execution, and automated verification.
• Maintaining human-in-the-loop PR reviews to build team context and ensure code quality during large-scale migrations.