Prioritizing AI for Code Comprehension over Generation
AI Engineergo watch the original →
the gist
Analysis of 116 AI sessions at Sentry reveals that 67% of developer prompts are for comprehension, while only 2% are for code generation, highlighting the need for structured exploration over blind automation.
The Breakthrough
Priscila Andre de Oliveira identified that in a complex, 15-year-old codebase, the primary value of AI is not code generation but rather rapid comprehension, leading to the creation of a structured "Catch Me Up" prompting framework that forces the developer to align their mental model before triggering agentic planning or implementation.
What Actually Worked
- Categorized Exploration Modes: The "Catch Me Up" skill uses six specific modes to query the codebase: architecture, conventions, feature traces, syntax, testing, and history.
- Structured Prompting: Instead of vague requests, the author uses a local Markdown-based prompt file that forces the LLM to output findings in structured tables, ensuring the developer can verify the agent's research before proceeding to implementation.
- Verification Loop: The author enforces a strict workflow where the agent's research must be reviewed and understood by the human developer to prevent "slop" or misaligned code, effectively treating the AI as a teammate that requires steering.
- Usage Analysis: By analyzing 116 sessions, the author discovered that 67% of her interactions were comprehension-based, which allowed her to formalize her most frequent questions into a reusable, local skill set.
Context
Working at Sentry, a platform with 15 years of legacy code and 100 PRs merged daily, the author found that traditional methods of code navigation—such as manual git-blame or waiting for asynchronous Slack responses—were bottlenecks. By shifting the focus from "vibe coding" (generating code without understanding) to a research-first approach, she maintains high-quality contributions while leveraging AI to navigate complex architectural changes and legacy debt.
Notable Quotes
- "The biggest unlock from AI in a large code base isn't generation, it's comprehension."
- "You need to understand the research your agent did, because maybe it's going the wrong direction or maybe you need to explore something else."
- "Don't ship slop code into the code base that pays your salary."