Recursive Language Models: Moving Beyond Context Window Limits

AI Engineer2026-09-09go watch the original →

the gist

Recursive Language Models (RLMs) treat context as a symbolic object in a REPL, allowing models to iterate, compute, and recursively delegate subtasks to themselves, which significantly outperforms traditional RAG and agentic tool-calling patterns on long-context tasks.

The Breakthrough

Recursive Language Models (RLMs) shift the paradigm from stuffing context windows with tokens to treating context as a mutable object within a Python REPL, enabling the model to perform symbolic computation, iterate on data frames, and recursively delegate subtasks to itself.

What Actually Worked

Symbolic REPL Integration: Instead of passing JSON strings back and forth as in standard agentic tool-calling, the model interacts directly with the environment variables, allowing it to slice, compute, and iterate over large datasets without bloating the context window.
Recursive Decomposition: The model delegates complex subtasks to sub-models (often itself) with specific parameters, ensuring only relevant results are returned to the main context, which prevents context rot.
Self-Directed Stopping: The model manages its own execution loop by deciding when it has sufficient information to run a submit function, rather than relying on a hard-coded number of steps or external orchestration.
Schema Enforcement: Using frameworks like DSPy, developers can enforce strict input and output schemas between the main model and sub-models, which improves reliability and allows for the use of smaller, cheaper models for specific sub-tasks.

Before / After

Long Chain of Thought Benchmark: Accuracy improved from 2.6% using standard methods to 45.4% using the RLM approach, with the most significant gains observed in logic-heavy tasks like chemistry, chess, and code analysis.

Context

Traditional RAG and agentic workflows often fail on large or dense inputs because they rely on stuffing the context window or shuttling strings between loosely coupled components. RLMs solve this by keeping logic, execution, and results in a single, tightly coupled environment. This approach is particularly effective for tasks that require deep iteration over large corpora, such as consolidating hundreds of pages of invoices or analyzing security vulnerabilities across hundreds of thousands of lines of code, where traditional chunking and embedding strategies typically degrade in quality.

Content References

tool: DSPy, Omar Khattab et al., cited
tool: Predict RLM, mentioned
tool: Axe, mentioned
tool: Halo, mentioned
tool: Fast RLM, mentioned