Semantic Work Primitives Over Computer Access

The Breakthrough

Nate Jones asserts that the core primitive for AI agents is the semantic work unit, such as a refund or rescheduled meeting, which conveys meaning beyond raw access via computer use like Codex's browser control.

Three Control Layers

Agents operate across access, meaning, and authority layers. Access comes from computer use, which lets agents open browsers, click buttons, and fill forms. Meaning arises from semantic work primitives that clarify what an action entails, such as a calendar shift notifying others, breaking commitments, or conflicting priorities. Authority governs permissions, reversibility, financial impact, and approvals.

Jones urges using the richest semantic interface available: connectors first, then protocols, typed objects with permissions, and browser or desktop control only as fallback. Plugins and MCP servers provide this richer access over screenshots or visual navigation.

Why Coding Agents Led

Coding agents succeeded first because codebases offer dense semantics through modules, dependencies, tests, type systems, linters, package managers, and Git history. Agents inspect repositories, edit files, run tests, observe errors, and revise actions with built-in feedback loops. Non-coding work lacks this legibility; calendars hide politics, sales processes omit history.

Platform Strategies

Perplexity evolves from search to browser (Comet), desktop, and personal computer to assemble cross-app work graphs, turning research into permissioned actions. Hyperscalers like those behind Claude and Codex leverage code understanding to bridge to other work. Salesforce pursues headless architectures for semantics; SAP blocks agents to retain control.

Context

Companies build demos of agents clicking buttons, but this distracts from platform power in defining work meaning. Legacy software assumes human interpreters for dashboards, Excel, and procurement. Without semantics, agents guess on high-stakes tasks like contracts or refunds, requiring supervision. Solving semantic legibility creates moats, especially for startups targeting agent-native software where humans and AI coexist.

Notable Quotes

• "The real fight is over who defines what the button means."
• "Computer use is the universal adapter for the messy middle."
• "Agents should use the richest semantic interface available."
• "The real primitive... is a semantically meaningful unit of work: a refund, a reschedule, a payment authorization."