Designing AI Agents for Knowledge Work
AI Engineergo watch the original →
the gist
Coding agents are a poor template for general knowledge work because they rely on durable, grep-able cues. Effective knowledge agents require hierarchical task decomposition and multi-modal search tools to bridge the 'oracle gap' between raw data and actionable insight.
The Failure of Coding-Centric Agent Design
Coding agents succeed because code is a unique, highly structured domain with durable identifiers, file paths, and explicit method definitions. Developers provide narrow, ticket-based instructions that minimize ambiguity. In contrast, general knowledge work—such as legal or medical research—is defined by implicit meaning, where the same string (e.g., "30 days") can represent a deadline, a retention rule, or a medication schedule depending on context. Generalizing agent design from coding to knowledge work fails because it ignores the necessity of intent reconstruction and contextual reasoning.
Optimizing the Tool and Organization Loop
Human knowledge work has evolved through a dual loop of tool development and organizational hierarchy. Better tools (from library catalogs to search engines) enable new roles, which in turn generate more complex knowledge, necessitating further tool refinement. This is not a neutral process; tooling determines the scalability and economic viability of a task.
Benchmark data from the BrowseComp+ and MQA datasets demonstrate that:
- Poorly optimized BM25 baselines yield ~60% accuracy, rendering the tool useless for professional workflows.
- Optimizing search primitives (e.g., switching from OCR-based text to multi-modal PDF parsing) significantly raises the accuracy ceiling.
- Introducing a hierarchical agent architecture—where a primary agent decomposes a complex query into sub-tasks for specialized searcher agents—reduces the "oracle gap" (the distance between perfect information and system output) by approximately 40%.
Designing for Knowledge Agents
To move beyond the limitations of current systems, developers should stop treating tools as neutral add-ons and instead co-design them with the agent's orchestration layer. Agents must be trained to recognize that primitives like grep, BM25, and semantic search are distinct tools for different contexts. Because context window size is a finite and expensive resource, agents must act as orchestrators that break down open-ended problems into specific, researchable memos rather than attempting to ingest entire document corpora into a single prompt.