Knowledge Systems for AI Agents: Intrinsic, Extrinsic, and Learned
AI Engineergo watch the original →
the gist
Microsoft's Pablo Castro categorizes AI knowledge into intrinsic model memory, extrinsic organizational grounding, and learned agent optimization loops, demonstrating how to automate agent configuration via hill-climbing evaluations.
The Three Pillars of AI Knowledge
Knowledge in AI systems is categorized into three distinct layers: intrinsic, extrinsic, and learned. Intrinsic knowledge refers to the parametric memory acquired during model training, which serves as the foundation for reasoning. Extrinsic knowledge involves grounding agents in private, ambient organizational data such as documents, emails, and data warehouses. Learned knowledge represents the iterative improvement of agent performance by observing task execution and automatically refining configurations through feedback loops.
Implementing Extrinsic Grounding and Agentic Retrieval
To move beyond simple vector search, developers should implement hybrid retrieval systems that combine vector, lexical, and semantic methods. The Foundry IQ platform allows users to abstract this complexity by providing a tiered stack: developers can either use high-level defaults for chunking and indexing or drop down to granular controls for vector quantization and indexing algorithms. For complex information needs, agentic retrieval is recommended, where the system reflects on retrieved content to verify if the information satisfies the user request before returning a response.
Automating Agent Optimization
Learned knowledge is materialized through an agentic optimization loop that treats agent configuration as a variable to be tuned. By externalizing instructions, tool definitions, and skill sets, developers can use an automated hill-climbing process to improve agent performance. The workflow involves:
- Generating a task-adherence evaluation dataset from existing agent traces.
- Running an optimization process that iterates through configuration candidates to maximize performance against the evaluation rubric.
- Applying the optimized configuration, which often results in machine-generated instructions that outperform hand-written prompts.
This approach turns the agent's operational history into a self-improving asset, allowing organizations to compound their unique capabilities over time.