From 0287de4a1c536228c5678dc36e49647816312026 Mon Sep 17 00:00:00 2001 From: ayobamiseun Date: Wed, 2 Sep 2026 10:32:21 +0100 Subject: [PATCH] docs: add authoring rule against model-specific workarounds (#534) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A step that exists because one model gets a specific call wrong constrains agents that would have solved the task directly and spends turns stronger models need for the actual work — cross-model transfer measurements (WikiSkill, arXiv:2608.27454) show such skills can leave a stronger model worse off than no skill at all. Add 'Write the Procedure, Not the Workaround' to skill-anatomy: the rule (no step justified only by naming a model, version, or one agent's private tool name), the two smells (over-specified mechanics, fragmented diagnostics), a model-neutral writing principle, and a matching pre-flight line in CONTRIBUTING. --- CONTRIBUTING.md | 1 + docs/skill-anatomy.md | 14 ++++++++++++++ 2 files changed, 15 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f08c85ee..42d183da 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,6 +15,7 @@ This pack already covers most of the development lifecycle, and many proposals o 3. **Check rejected proposals.** Search the [skill-change rejection ledger](evals/skill-impact.md) for earlier proposals that overlap with your idea and review their eval evidence before repeating the work. 4. **Read the anatomy.** Confirm your idea fits the format in [docs/skill-anatomy.md](docs/skill-anatomy.md), an actionable workflow with verification, not vague advice. 5. **Justify the gap in your PR description.** State explicitly why this isn't covered by an existing skill, open PR, or previously rejected proposal. If it overlaps, propose extending the existing skill instead of adding a new one. +6. **Strip model-specific workarounds.** If a step can't be justified without naming a model, a model version, or one agent's private tool name, it doesn't belong in a skill — describe the capability instead (see [Write the Procedure, Not the Workaround](docs/skill-anatomy.md#write-the-procedure-not-the-workaround)). If your idea is a refinement of an existing skill, prefer a focused edit to that skill over a new directory. diff --git a/docs/skill-anatomy.md b/docs/skill-anatomy.md index 5dcd5b37..f468c6c8 100644 --- a/docs/skill-anatomy.md +++ b/docs/skill-anatomy.md @@ -147,6 +147,20 @@ When a skill ships runnable helpers under `scripts/`, each script follows these 4. **Anti-rationalization.** Every skip-worthy step needs a counter-argument in the rationalizations table. 5. **Progressive disclosure.** Main SKILL.md is the entry point. Supporting files are loaded only when needed. 6. **Token-conscious.** Every section must justify its inclusion. If removing it wouldn't change agent behavior, remove it. +7. **Model-neutral.** Write the procedure, not the workaround. See [Write the Procedure, Not the Workaround](#write-the-procedure-not-the-workaround). + +## Write the Procedure, Not the Workaround + +Skills in this pack run on many agents and model generations. A step that exists because one model gets a specific call wrong is a liability on every other model: it constrains agents that would have solved the task directly, and it spends turns that stronger models need for the actual work. Cross-model transfer measurements ([WikiSkill, arXiv:2608.27454](https://arxiv.org/abs/2608.27454)) show skills authored against one model's execution failures can leave a stronger model performing *worse than with no skill at all*. + +**The rule:** if a step cannot be justified without naming a model, a model version, or one agent's private tool name, it belongs in an issue, not in a skill. Describe the capability ("run the focused test command", "write the file"), not the mechanism one runtime happens to expose (`run_command`, `write_to_file`). + +Two smells to check for before proposing skill content: + +- **Over-specified mechanics.** Prescribing an exact command form or serialization detail where stating the goal alone would do. If the justification is "model X gets this wrong otherwise", it's a workaround, not a procedure. +- **Fragmented diagnostics.** A multi-step inspection sequence where one step would confirm the same thing. Every extra step spends turns, and turn budget is finite. + +This is about skill *content*; the portability of `references/` *paths* is a separate concern (see [Shared References](#shared-references)). ## Naming Conventions