Scaling Heterogeneous Intelligence via Task-Specific Routing

The Breakthrough

By decomposing complex workflows into subtasks and routing them to specialized hardware and smaller models rather than relying on a single frontier model, Callosum achieved state-of-the-art performance on benchmarks while drastically reducing latency and cost.

What Actually Worked

• Heterogeneous Recursion: Instead of loading full context windows, the system treats context as an environment, using a coding agent to interact with files via Python REPL to extract relevant subcontexts for smaller, recursive agents.
• Task-Specific Model Routing: The system offloads simple subtasks like visual zooming and parsing to smaller models (e.g., Qwen 3 VL8B) while reserving frontier models (e.g., Kimi K2.5) for high-complexity reasoning.
• Hardware-Aware Scheduling: Workloads are mapped to optimal silicon based on computational demand, utilizing Cerebras or Groq for specific recursive tasks to bypass the inefficiencies of monolithic GPU clusters.
• Automated Orchestration Layer: Rather than making bespoke manual decisions for every subtask, an automation layer detects task complexity and dynamically predicts the best-suited model and hardware combination.

Before / After

• Long Context Reasoning (Ulong Benchmark): Using Cerebras instead of a frontier model resulted in 7x lower costs and 5x faster latency while maintaining accuracy.
• Visual Web Navigation (Video Web Arena): A mixture of Qwen 3 VL8B and Kimi K2.5 outperformed GPT-4o and Gemini 1.5 Pro by 18% and 25% respectively, while being 3.7x cheaper and 3x faster.
• Subtask Efficiency: Offloading visual parsing and zooming to smaller models yielded 11x speed improvements and 43x cost reductions compared to using GPT-4o for the same operations.

Context

Adrian Bertagnoli argues that the era of scaling monolithic models on identical GPU clusters is reaching a point of diminishing returns. Real-world problems are inherently multi-step and open-ended, requiring a diverse set of architectures and silicon to solve efficiently. Callosum is building an automation layer to unify these heterogeneous components, treating model architecture, chip type, and workflow as variables to be optimized in tandem.

Content References

• Paper: "Recursive Language Models" (MIT), cited as the foundational concept for the heterogeneous recursion workflow.