Running Doom via ONNX Model Tensors

The Breakthrough

Anthony Shaw demonstrated that an ONNX model can function as a Turing-complete CPU emulator by representing RAM as a tensor and mapping RISC-V machine code instructions to ONNX graph nodes, successfully running the Doom demo program.

What Actually Worked

• Compiled the Doom source code into RISC-V machine code to create a portable instruction set for the emulator.
• Built a CPU emulator graph in Netron where the RAM state is stored as a tensor and processed through logical instruction nodes.
• Used Excel formulas to disassemble the machine code and verify the logic, effectively treating the spreadsheet as a rudimentary CPU interpreter.
• Implemented performance optimizations by compacting the ONNX graph nodes and reducing the total compute overhead, which increased the frame processing speed from 2 kHz to 4 kHz.

Context

The experiment explored the limits of ONNX model execution by moving beyond standard inference tasks. By treating the game state as a tensor, the author demonstrated that AI models are not just static weight containers but executable graphs capable of complex logic. The project highlights the necessity of benchmarking when coaching AI agents, as agents often propose inefficient optimizations like parallelization for single-threaded tasks without verifying actual performance gains.

Content References

• tool: Netron, https://netron.app/, mentioned
• tool: Microsoft Copilot, mentioned
• tool: Excel, mentioned
• tool: Hugging Face, mentioned