The Miranda Hypothesis: Why Persona Evals Are Failing

The Failure of 'Convincingness'

Modern role-playing language agents (RPLAs) are evaluated using benchmarks like InCharacter, CoSER, and PsyMem, which prioritize personality fidelity and stylistic naturalness. While these models achieve high scores (e.g., 80%+ alignment), they suffer from a structural failure: they optimize for 'convincingness.' Because these models are trained on massive datasets saturated with modern cultural interpretations of historical figures, they default to a 'smoothed' composite rather than the actual historical record. This phenomenon, termed 'Miranda distortion,' occurs because the volume of pop-culture content (like the Hamilton musical) exponentially outweighs primary documentary evidence (like the Federalist Papers).

The Mechanism of Distortion

Miranda distortion is not a bug that can be patched via standard RLHF (Reinforcement Learning from Human Feedback). Because human raters are themselves products of the same cultural environment, they reward models that conform to their own mythologized expectations. This creates a feedback loop of 'algorithmic sycophancy,' where the model is incentivized to produce the version of a historical figure the user already believes in. Even 'time-walked' models, which restrict training data to a specific cutoff, fail to solve this because they still average across all available historical data, resulting in a composite that is merely anchored to a different era rather than constrained by a specific documentary record.

Toward Epistemic Simulation

To move beyond the 'mask' of convincingness, the author proposes a shift to 'epistemic simulation.' This framework requires three constraints:

1. Corpus-boundedness: Reasoning must be licensed strictly by primary documents.
2. Temporal anchoring: The persona must be fixed to a specific moment in time, rendering later knowledge (or modern cultural interpretations) out of bounds.
3. Expert-loop evaluation: Outputs must be audited by domain experts (historians, psychologists, etc.) rather than automated LLM-as-judge metrics.

Re-architecting the Encounter

Instead of treating the persona as a property of the model's weights (which are opaque and unauditable), developers should treat the persona as a 'configuration' of an encounter. This involves using RAG-based context windows to supply anchor documents at inference time, rather than fine-tuning the model on the persona. By keeping the persona in the configuration (the prompt, the corpus, and the temporal anchor) rather than the weights, the system becomes versionable, auditable, and reproducible.