Real-Time Interactive Video as a New Medium

The Shift to Real-Time Programmability

Real-time interactive video represents a fundamental change in medium rather than a mere performance optimization. Much like how GPS transformed maps from static references into continuous, actionable data, or how digital viewfinders enabled the high-quality content production seen on platforms like Instagram and TikTok, real-time generation allows users to steer content as it is rendered. Current batch-based generative models act as "slot machines" that return static files, whereas real-time world models allow for continuous interaction, infinite duration, and immediate feedback loops.

Engineering Requirements for Interactive Models

Building for real-time video requires a departure from standard batch-inference playbooks. Developers must manage stateful sessions to maintain context, such as ensuring a character remembers their environment when turning away. Because the experience relies on sub-100ms latency, compute must be distributed globally to place GPUs as close to the user as possible. The infrastructure must prioritize streaming pixel data over returning completed files, necessitating specialized handling of memory and context windows to maintain consistency over time.

Emerging Use Cases

Interactive video models are currently being applied across three primary domains:

• Control-Driven Media: Filmmakers and creators use real-time feedback to adjust visual elements, such as inserting brand logos or altering scenes on the fly, effectively ending the need for pre-rendered assets.
• Simulation and Training: Robotics labs utilize these models to generate infinite training data in simulated environments, while medical and culinary applications allow users to test variables in interactive, safe-to-fail scenarios.
• Programmable Worlds: Developers are building interactive live streams where audiences vote on narrative outcomes, as well as pre-visualization tools for Hollywood that allow for real-time video-to-video editing and visual effects application.

Evaluation Challenges

Despite progress in model architecture, objective evaluation remains an unsolved problem for the research community. Current assessment relies heavily on human judgment and visual inspection, as there are no standardized metrics for fidelity or temporal consistency in real-time generative video.