Thinking Machines' 200ms Micro-Turns Enable Real-Time AI

Core Architecture

Thinking Machines' interaction model processes text, audio, and video in continuous 200ms micro-turns. The system uses encoder-free early fusion with minimal preprocessing: text generates embeddings, video and images use 40x40 patches passed to a uMLP layer, and audio computes m-spectrogram features into a bag of embeddings. All inputs feed into a transformer tokenized by 200ms windows.

A 276 billion parameter mixture-of-experts model (12 billion active parameters), called TML Interaction Small, handles real-time user interactions. It offloads reasoning-intensive or knowledge-intensive tasks asynchronously to a more capable background model with tool access; responses feed back into the interaction model.

Key Capabilities

The micro-turn tokenization supports seamless dialogue management by tracking user intent and conversation state across inputs without full-turn waits. For example, the model listens to a story and counts animal mentions (deer, sheep) incrementally as the user speaks, responding only on triggers.

It enables multimodal verbal and visual interjections, such as posture correction from video: "Sit up straight and you'll be golden" or "Try pulling your shoulders back." Time awareness emerges natively since the model tracks passed tokens to estimate elapsed time, unlike GPT-4o Advanced Voice Mode which requires external tools.

Demos show real-time reframing of speech into professional language ("I cannot stand your lateness" becomes "We'd love to explore opportunities to enhance your timeliness") and finger-counting from live video ("Five fingers up, Two fingers up").

Inference and Infrastructure

Inference uses streaming sessions to optimize frequent small prefills and decodes every 200ms, as existing LLM libraries incur high per-turn overhead. This custom infrastructure supports strict latency for real-time performance.

The architecture builds on open-source work like Mushi from Qwen and contrasts traditional full-duplex systems (voice activity detection + STT + LLM + TTS + orchestrator) by unifying components into time-tokenized processing.

Benchmarks and Comparisons

Self-reported benchmarks plot TML Interaction Small high on intelligence vs. interaction quality and intelligence vs. responsiveness axes. It outperforms GPT-4o realtime and Gemini in live finger-counting demos, where those models miss hand visibility due to sampled intervals.

The team includes ex-DeepMind, OpenAI, and Anthropic members. No model or API release yet; external validation pending.