Converting Tacit Knowledge into Self-Improving AI Skills

Extracting Knowledge via AI Interviewing

To move a process from a human expert's head into an AI, the author suggests a two-stage extraction process. For judgment-heavy tasks, initiate an AI interview where the model asks a series of 15 to 30 questions to force the expert to verbalize their decision-making criteria. For routine tasks, skip the interview and move directly to proving the workflow.

Validating and Building Skills

Validation is the mandatory step that most users skip. Before creating a formal skill, run the process manually within a chat interface (e.g., Claude or ChatGPT) using real-world inputs. This step exposes missing steps and edge-case traps that are not captured in a high-level SOP. Once the workflow is proven, use an AI agent to encapsulate the logic into a reusable skill. When prompting the creation of this skill, explicitly instruct the AI to remain agnostic of the specific input used during the validation phase to ensure the skill generalizes to future tasks.

Implementing Self-Improvement Loops

To prevent the skill from becoming stale, embed a feedback loop directly into the AI's configuration. Instruct the AI to maintain a corrections.mmd file that tracks every correction provided during operation. Use the following prompt pattern to manage this:

Anytime I correct you, take those corrections and add them to a file named corrections.mmd. Keep the entries short and dense.

On a recurring basis, prompt the AI to review this file and suggest targeted updates to the base skill logic:

Based on the corrections.mmd file, are there any small or targeted updates we can make to the skill to improve it? If so, provide them back to me.

If the suggested updates are satisfactory, instruct the AI to apply those changes to the skill definition, creating an iterative improvement cycle that adapts to evolving business requirements.