Hands-On Agent Evals Pipeline with Phoenix

Overcoming the Vibes Problem in Agent Testing

Agents fail unpredictably due to cascading errors, adversarial inputs, and dumb user queries that expose vocabulary gaps. Manual testing—running a few queries and eyeballing outputs—misses regressions, doesn't integrate into CI/CD, and can't verify if prompt tweaks fix one issue without breaking others. Evals solve this by providing automated, reproducible tests on traces (nested JSON logs of LLM calls, tool invocations, inputs/outputs, metadata like tokens/timing). For agents, evals must handle multi-step paths: right tool selection, correct parameters, proper output interpretation, without being overly prescriptive (e.g., don't hardcode exact tool sequence, as smarter models optimize paths). Distinguish capability evals (hard tasks to benchmark improvement) from regression evals (ensure existing functionality persists post-changes). Real-world teams like Decrypt, Bolt.new, and Anthropic's Claude Code shifted from vibes-based shipping to formal evals for production reliability.

Traces form the foundation: spans capture each step (agent turn spans nest tool/LLM spans). Without instrumentation, no data for evals. Use Phoenix (Arize's open-source observability) for capture, storage, UI visualization, and experiment running. Install integrations like openinference-instrumentation-crewai (assuming 'claw a' is CrewAI) to auto-log without framework modifications. Phoenix Cloud avoids local setup; send traces via API key. Pre-built agent (financial analysis) uses Claude; traces reveal raw execution for diagnosis before evals.

Failure Analysis Drives Eval Design

Before writing evals, inspect traces manually: categorize failures by root cause (e.g., wrong tool, hallucination, misparsed output). This prevents misaligned metrics—e.g., a correctness eval scores 0/13 on forward-looking financial data (model lacks current year knowledge), while faithfulness scores 13/13. Patterns emerge across runs: systematic prompt flaws vs. one-offs. Use LLM to auto-categorize eval explanations (LLMs all the way down), avoiding manual review of thousands. Key principle: right eval > perfect tuning. Humans build golden datasets (known-good outputs) for validation, but fatigue causes ~50% error rate, so limit to novel failures.

Code Evals for Deterministic Checks

Start with code evals: fast (ms), cheap, reproducible Python/TS functions. Validate JSON output, token limits (<500), required fields, forbidden phrases, format. Brittle for complex semantics, but ideal for basics. Example: parse agent JSON response, check keys like 'recommendation', 'confidence'. Run in CI for regressions.

LLM-as-Judge for Semantic Flexibility

For nuance, deploy stronger LLM (e.g., Claude) as judge against rubrics (prompt-defined rules). Types: correctness (accurate answer?), faithfulness (sticks to sources?). Flexible for tone, completeness (e.g., budget recs missing costs). Outputs: score (0-1), label (pass/fail), explanation (actionable why). Downsides: costly, nondeterministic—mitigate with meta-evals (judge the judge's agreement). Built-in Arize evals simplify; custom from scratch: prompt template + few-shot examples. Example rubric: "Judge if response faithful to context: hallucinated? Extrapolated beyond sources?"

Meta-evaluation: run multiple judges (vary models/prompts), compute agreement (e.g., pairwise). Low agreement flags bad rubric. Experiments: datasets of input-output pairs; A/B test prompts/models, plot scores over runs to prove improvements (not vibes).

Iteration Frameworks: Data Flywheel and Impact Hierarchy

Full loop: instrument → trace → eval → annotate (categorize) → analyze → iterate (prompt/app changes). Data flywheel: evals generate labeled data for fine-tuning. Impact hierarchy: prioritize high-failure spans first. Pairwise evals (judge A vs. B preference); reliability scoring (judge consistency). Post-workshop: scale to multi-agent routing (right sub-agent? Hand-offs?).

Notable Quotes:

• "The sharpest lesson: choosing the right eval matters more than tuning it. A correctness eval scored 0 out of 13 on the same agent that a faithfulness eval scored 13 out of 13." (On eval selection via financial agent demo.)
• "Without evals you can't change your system prompt to fix a tone issue because the tone might get better but suddenly the bot might be hallucinating product features." (Explaining regression risks.)
• "Agents can get things right in a way that you weren't expecting... your evals will break if they are too prescriptive." (Warning against rigid tool-sequence checks.)
• "LLMs all the way down: use a third LLM to read the explanations and turn them into categories." (Scaling failure analysis.)
• "Humans get it wrong 50% of the time... fatigue makes them miss things." (On human eval limits.)

Key Takeaways

• Instrument early with Phoenix integrations for traces; inspect before evals to identify failure modes.
• Use code evals for deterministic checks (format, length); LLM judges for semantics (faithfulness, correctness).
• Categorize failures from trace explanations to pick evals; meta-eval judges for reliability.
• Build golden datasets from humans for novel cases; run experiments to validate prompt/model changes.
• Avoid prescriptive evals; capability → regression progression; prioritize via impact hierarchy.
• Full pipeline: trace → eval → analyze → iterate; proves shipping without vibes.
• Complements: code + LLM + human; run in CI for regressions, model swaps.
• Agents amplify issues: test cascades, not isolates; flexible rubrics handle paths.