Scaling Robotics via Generalist 'Omni-bodied' AI
AI Engineergo watch the original →
the gist
Robotics has stagnated for 70 years because it was treated as a hardware-specific engineering problem; Skild AI argues for a 'general brain' approach that uses a data flywheel of simulation, human video, and teleoperation to enable any robot to perform any task.
The 70-Year Stagnation
Robotics has been "almost here" for decades, yet modern demos are visually indistinguishable from those in the 1950s. Pathak argues this is due to the "hardware-first" fallacy. While AI progressed rapidly in language and vision, robotics remained trapped in specialized, hand-coded state machines. This is a manifestation of Moravec’s Paradox: tasks humans find difficult (like complex math) are easy for computers, while tasks humans find easy (like navigating stairs or manipulating objects) are incredibly difficult for robots because they require real-time, high-dimensional sensory integration.
The Data Flywheel Strategy
Scaling robotics requires a "general brain"—an omni-bodied model that abstracts away hardware differences. Because there is no "internet of robot data," Pathak proposes a three-tiered data strategy:
- Pre-training: Use high-scale, diverse, but lower-quality data sources like simulation and human video to build foundational spatial and physical reasoning.
- Post-training: Use high-quality, low-volume teleoperation data to fine-tune the model for specific physical tasks.
- Deployment Flywheel: Once deployed, robots collect real-world data that feeds back into the pre-training loop, creating a self-improving system.
End-to-End Intelligence
Skild AI rejects traditional robotics pipelines (mapping, planning, state machines) in favor of true end-to-end learning: raw camera input to motor torque. This approach allows for surprising robustness. For example, robots can learn to manipulate objects (like inserting AirPods or cooking omelets) using cheap $4,000 hardware without force sensors, relying entirely on vision-based intelligence. Pathak notes that "stairs are harder than backflips" because stairs require environmental perception and adjustment, whereas backflips are closed-loop, body-centric maneuvers that computers have mastered via simulation.
The Role of Simulation and Imagination
To overcome the data bottleneck, the model uses "imagined" training data—a form of latent space dreaming where the robot simulates potential scenarios to multiply its learning efficiency. This allows the system to achieve high performance with less than 10 hours of actual robot interaction time, proving that the bottleneck is not hardware capability, but the intelligence required to drive it.