Granola's tracing UI and web previews speed AI iteration

The Breakthrough

Granola engineers created custom internal tracing tools and refactored their Electron app's frontend into a web shell to enable rapid iteration on AI chat features that fail in production due to web search costs and user-specific needs.

What Actually Worked

• Engineers built custom tracing that logs tool calls, search trails, reasoning traces, and costs to a database with a wrapper around a framework (referred to as 'zk'), then displays them in a frontend UI accessible to all employees including product, data, and CX teams.
• They abstracted IPC APIs (system APIs) and React APIs (routers, sessions, query layer) to fallback to web standards, allowing the renderer process to run as a standalone web app deployed online.
• CI generates preview links for every PR, enabling parallel testing of feature variants without local Electron installs.
• Cursor automatically tests PR changes, self-verifies AI work, and uploads screenshots to PRs.

Context

Granola's meeting notes app uses real-time transcription from system and microphone audio, then generates summaries that incorporate user notes. Their AI chat feature, which answers questions across meetings, initially failed in production: it mishandled to-dos, web search was slow and costly at 10p per chat with context blowups, provider updates degraded results overnight, and one prompt could not serve sales users wanting deal summaries, engineers wanting blockers and Linear tickets, or HR with other needs. These issues stemmed from treating LLMs as black boxes without visibility or fast testing in a desktop app environment. The custom tools created feedback loops where anyone traces failures end-to-end, iterates prompts or variants, and tests in realistic conditions, making AI features feel like 'magic' rather than unreliable one-shots.

Notable Quotes

• 'each chat could be costing you like 10 p'
• 'one prompt can't generally serve everyone'
• 'the answer isn't to one shot better... make that feedback loop where it kind of feels like playing a tennis game with LLM'

Content References