Stop Prompting and Start Building Project Rooms
Nate B Jonesgo watch the original →
the gist
Prevent AI hallucinations in high-stakes work by shifting from 'prompt engineering' to building structured 'project rooms' where agents inventory, compare, and validate source files before drafting.
The Shift from Prompting to Structuring
High-stakes AI hallucinations, such as those seen in legal filings, are rarely fixed by better prompting. Because language models lack a separate truth-check mechanism, instructing them not to hallucinate is ineffective. Instead, the failure is structural: the model is forced to synthesize messy, contradictory, or stale source material without a clear hierarchy of truth. The breakthrough is to treat the AI as a partner that helps build a 'project room'—a bounded, local file-based workspace—before any drafting begins.
The Project Room Workflow
Instead of asking an agent to 'do the thing' immediately, the first interaction should be to organize the data environment. This creates a canvas that makes the agent's judgment visible and inspectable. The workflow relies on three specific artifacts generated by the agent:
- Source Inventory Table: A master list of every file in the project, recording the path, type, date, apparent authority, and usage instructions. This allows the user to verify the agent's understanding of the source material before synthesis.
- Conflict Log: A report surfacing contradictions between sources, such as conflicting dates, stakeholder names, or unsupported numbers. This forces the agent to highlight disagreements rather than smoothing them over.
- Missing Context List: A list of gaps, such as missing decision records or absent data files, which prevents the model from inventing information to fill holes in the context window.
Managing Data Integrity
Duplicates are a reasoning problem, not just housekeeping. If an agent sees multiple versions of a plan, it may average the assumptions, leading to inaccurate outputs. The agent should be tasked with generating a duplicates report and isolating suspected duplicates into a separate folder for human review. Once the project room is established, the final writing prompt becomes significantly shorter and more effective, as it can reference the validated inventory and authoritative sources directly. This approach transforms the AI from a 'gopher' into a senior colleague capable of maintaining structural integrity across complex, multi-document projects.