The Dark Arts of Skill Engineering: Beyond Prompting

AI Engineergo watch the original →

To build reliable AI skills, move beyond prose-based prompting into 'harness engineering'—using deterministic scripts, sub-agent orchestration, and persistent local storage to overcome model gravity and slop.

The Failure of Prose-Only Prompting

Most AI skills rely on 'system prompt and a prayer,' which inevitably leads to 'slop'—the tendency for models to converge on a median, overused aesthetic (e.g., 'Claude beige' or purple gradients). Banning specific outputs (like fonts or colors) is ineffective because it simply pushes the model to the next nearest cluster in latent space. The model's gravity is too strong for prose instructions to overcome; true control requires treating the skill as an extension of the development harness rather than just a set of instructions.

Harness Engineering: The Nine Techniques

Skill engineering requires shifting from passive prompting to active harness manipulation. Key strategies include:

  • Adversarial Sub-Agents: Avoid self-grading bias by spawning two blind sub-agents: one acting as a director (evaluating aesthetics) and one running a deterministic linter (checking constraints). The main thread synthesizes these conflicting perspectives.
  • Forcing Divergence: To escape the 'median' output, use anti-attractors. This includes 'shaving' the top three predicted tokens (forcing the model to choose less probable options) or injecting random seeds via scripts (e.g., using celebrity archetypes or external data files to seed a design palette).
  • Mixture of Experts Routing: Instead of one massive, blurry prompt, split skills into specialized registers. Use internal routing to load different rule sets based on the user's intent (e.g., distinguishing between 'brand/landing page' design and 'functional product' UI).
  • Persistent Memory: Since models lack long-term memory, use the local filesystem to store state. By saving critiques or user preferences into a hidden folder within the repo, subsequent sessions can read these files to understand the history and evolution of a project.
  • Deterministic Hooks: Use scripts that 'talk back' to the model. If a gate can be skipped, the model will skip it; therefore, implement hooks that block execution or force a re-evaluation if specific quality thresholds are not met.

Every model is overfitted differently. When distributing a skill, assume the user might be on a constrained harness (like Codebase or a specific IDE plugin). If a capability (like sub-agent spawning) is unavailable, explicitly instruct the model to inform the user of the 'degraded experience' rather than silently failing or taking the path of least resistance. This forces the model to maintain integrity even when its environment is limited.

The Discipline of Evaluation

Evaluation is not just about checking if the output is 'good'—it is about building a feedback loop. By combining deterministic linters with LLM-based critique, you create a system that can actually learn from its own mistakes. The goal is to build a 'design harness' that acts as an extension of the developer's workflow, capable of maintaining a design system's integrity across multiple iterations.

  • #tutorial
  • #ai
  • #dev-tooling

summary by google/gemini-3.1-flash-lite. probably wrong about something. check the source.