Preventing AI Email Hallucinations with Draft-Check-Stop

Dylan Davis2026-06-06go watch the original →

the gist

To prevent AI from making unauthorized commitments in emails, implement a three-step workflow that forces the model to adopt your specific voice, provide source citations for every claim, and gate high-risk messages in a draft state.

The Three-P Risk Framework

AI-generated emails often sound professional while inadvertently committing users to problematic promises, prices, or policies. To mitigate these risks, users must treat AI email generation as a high-stakes process that requires verification before transmission.

Establishing Voice and Context

To ensure the AI writes in your specific style, use desktop agent tools like Claude Co-work or Codeex to analyze 200 to 250 previously sent emails. Categorize these into five or six distinct buckets to avoid overwhelming the model. Within separate threads for each category, prompt the AI to create a writing fingerprint, then encapsulate that style into a reusable skill.

The Draft-Check-Stop Workflow

Implement a mandatory three-step verification process for all AI-drafted communications:

Drafting: Always keep AI-generated emails in a draft state. Use specific system instructions for each category to ensure the AI calls the correct voice-based skill.
Checking: Require the AI to provide a separate receipt or citation channel (e.g., Slack or a side-by-side document) that links every promise, price, or policy mentioned in the email back to your source material (rate cards, signed MSAs, or approved project plans).
Stopping: For high-risk categories, instruct the AI to explicitly insert [needs approval] brackets in the draft body for any claim it cannot verify against provided context. This forces a manual review where you can approve, edit, or delete the flagged content before sending.

Self-Improving Context

Store source material (policies, pricing, and promises) as context files within the AI agent's project or skill folders. As you review drafts and approve new, ad-hoc promises, instruct the AI to append these to the context file, allowing the system to improve its accuracy over time.