Codex App Visualizes Piano Notes for Theory Learning

The Breakthrough

Dan Shipper prompted OpenAI's Codex to create a web app that displays real-time piano notes, chords, and theory when he connects his MIDI piano to his computer.

What Actually Worked

• Prompt Codex: "Hey I'm going to plug my piano into my computer can you make a little app to show me what I'm playing?" The app shows pressed keys and identifies notes/chords.
• Extend with recording prompt: "Hey I want to be able to record stuff and I want you to tell me about what it is."
• Record a flourish from Lizzy McAlpine's "Older", then prompt: "Go take a look at it. Explain to me what I'm doing... what's the music theory behind it... chord progression... influences or flavors in there."
• Codex analyzes: "you're doing a flat add 9 to a flat sus 2 basically a a four to a one chord"; Shipper recognizes it as a G# chord spread across octaves (arpeggiated up the keyboard).
• Prompt Codex for external analysis: "go open it up watch it and then explain to me how it works and then help me apply it" on Open Studio's YouTube exercise "Root Shell Pretty"; Codex browses the video and explains.

Context

Shipper skips traditional piano drills like endless scales, preferring to noodle on songs he enjoys. The app lets him record casual playing from favorites like "Older" by Lizzy McAlpine (tutorial by Taylor McCall), query Codex for breakdowns, and apply insights to improvise or adapt techniques. This bridges fun noodling to theory without a human teacher, generalizing patterns across keys/songs.

Notable Quotes

• "okay this is fucking sick you can use codeex to teach you how to play piano"
• "I just find myself noodling around more... I have this whole universe of possibilities"
• "it doesn't replace having an actual teacher but... if you're like just a curious person this is the best time in the world to like learn anything just literally fire up Codeex"