Chess Coach: Stockfish + Detectors + LLM Translator

The Breakthrough

Take Take Take built a production chess coach app that achieves sub-3-second latency by using Stockfish for position evaluation, custom detectors for tactics (forks, pins, skewers) and positional themes (doubled pawns), and an LLM solely as a translator of structured JSON signals into explanatory English.

What Actually Worked

• Stockfish runs through the entire game to identify best moves, which serve as the ground truth solution in each position.
• Custom detectors extract context including threats, plans, tactics, and structural weaknesses; they fire signals like 'F5 threatens to trap the queen' while noting defenses such as capturing the central pawn.
• Maya, a neural chess engine from University of Toronto, predicts human move probabilities by rating (e.g., 1500 Elo) to assess move difficulty beyond Stockfish optimality.
• LLM (Gemini 1.5 Flash) receives all extracted JSON—including Stockfish evals, detector signals, and Maya probs—and translates it into nuanced commentary, e.g., explaining a knight capture on E5 as brilliant because it leads to checkmate while addressing specific threats.
• User-flagged bad commentary posts to Slack and injects into a running Claude Code session via Channels (research preview MCP feature); Claude runs a 'commentary triage skill' to investigate the position, modify prompts or detectors, regenerate commentary, verify it, post clarifying questions to Slack, and submit a PR for human review.

Before / After

Gemini 1.5 Flash passes 75% of 16 eval scenarios (tactical patterns, blunders, hallucination limits); Claude with thinking reaches 60% but with higher latency; GPT-4o mini has lower accuracy and latency than Flash. End-to-end latency averages 3 seconds (1s time-to-first-token) versus unpredictable reasoning model times exceeding that.

Context

LLMs hallucinate in chess due to lack of calculation; traditional engines like Stockfish excel at play but not explanation. Take Take Take grounded LLM output in Stockfish + detectors to teach users 'why' behind moves, enabling consumer-grade speed. The Claude feedback loop accelerates iteration: during demo, Claude triaged a flagged position, found no issue, and closed the ticket.

Notable Quotes

• "Keeping the model as a translator rather than a reasoner is what makes it work at sub-3-second latency."
• "LLM's only job is translating those structured signals into English."
• "We have closed the loop from user feedback to the PR request essentially with humans in the loop."

Content References

Stockfish (tool, mentioned); Maya chess engine (tool, University of Toronto, mentioned); Claude Code Channels (tool, Anthropic research preview, used).