Building Verifiable AI for Financial Services
AI Engineergo watch the original →
the gist
Kepler replaces probabilistic LLM outputs with a deterministic substrate, ensuring financial figures are traceable to source filings and verified through code rather than model reasoning.
The Breakthrough
Kepler achieves verifiable financial AI by decoupling the LLM from numerical computation, forcing the model to act as a planner that triggers deterministic code for data extraction and calculation rather than generating numbers itself.
What Actually Worked
- Atomic Provenance: The system forces the model to generate a reference to a source document rather than outputting a raw number, ensuring every figure is linked to a specific, verifiable origin.
- Scope Determinism: The architecture strictly separates nondeterministic reasoning tasks from deterministic mathematical operations, offloading calculations to standard CPU-based code execution instead of using the LLM for arithmetic.
- Derivation Chains: The platform maintains a replayable audit trail of every transformation applied to raw data, allowing users to trace how a specific ratio or metric was derived from underlying source filings.
- Deterministic Reconciliation: When calculating complex ratios like EBITDA adjustments or gross margins, the system applies firm-specific logic to structured data, ensuring the output adheres to the organization's unique accounting standards.
Context
Financial services firms struggle to adopt AI because LLMs are probabilistic machines that hallucinate figures, making them unsuitable for high-stakes work like DCF modeling or investment memos. Current RAG-based approaches only provide citations, which act as an after-the-fact audit rather than a guarantee of accuracy. Kepler treats the LLM as an overworked VP that plans the workflow, while the heavy lifting of data extraction and math is handled by a deterministic substrate that mimics traditional database integrity.
Notable Quotes
- "Eval are not verifiable you cannot take a non-deterministic LLM and eval your way to something deterministic."
- "The model is really good at reasoning and planning... what the model cannot do is math and why would it? Why would I run 1 plus 1 through a multi-billion parameter model instead of one CPU cycle?"
- "A citation is effectively an after-the-act audit. A verification is a deterministic repeatable numerically verifiable mechanism."