Optimizing Agentic Workflows and Context Management

The U-Curve Context Problem

Large language models exhibit a U-curve performance pattern when processing large context windows. They prioritize initial and final inputs while effectively purging or ignoring the middle content. Simply increasing the context window does not solve this, as models struggle to determine which information is relevant. To address this, developers should move away from dumping raw data into prompts and instead implement strategic context optimization.

Context Optimization Techniques

• Iterative Retrieval: Acts as a library card system, indexing data so agents only pull relevant code snippets when needed. This is cost-effective and requires low developer input.
• Hierarchical Summarization: Generates summaries for files and folders to help agents filter information. This requires high upfront LLM processing costs whenever code changes.
• Knowledge Graphs: Maps logical dependencies between files and repositories. This is highly effective for complex architectures but requires significant initial effort to build and maintain.
• Self-Correction: Uses a critic node to evaluate if an agent's output aligns with the initial goal. If the output fails the check, the system triggers a retry, adding latency but reducing the need for complex upfront indexing.

The Orchestration Paradox and Hybrid Architecture

Capable models often waste tokens in infinite loops, researching methods to solve a problem rather than executing the solution. Qodo implements an 80/20 hybrid approach to resolve this. The system assigns 80% of the task to high-reasoning models for discovery and planning, while the remaining 20% is handled by lighter, deterministic models that enforce hard gates and validation.

Qodo’s code review architecture utilizes a context collector to bifurcate data, sending specific segments to specialized agents (e.g., security, code quality, Jira integration). A judge node then synthesizes these disparate results, weighing them against historical PR data and developer feedback. Every accepted or rejected suggestion dynamically adjusts the weights for future reviews, creating a self-calibrating system that learns organizational preferences over time.