chore(evals): scope claude plugin eval to evals/plugin via the manifest

.claude-plugin/plugin.json

@@ -9,5 +9,8 @@
   "repository": "https://github.com/addyosmani/agent-skills",
   "license": "MIT",
   "commands": ["./.claude/commands", "./commands"],
-  "skills": "./skills"
+  "skills": "./skills",
+  "experimental": {
+    "evals": "evals/plugin"
+  }
 }

.gitignore

@@ -6,4 +6,5 @@ node_modules/
 .claude/.simplify-ignore-cache/
 .claude/sdd-cache/
 evals/results/
+evals/plugin/results/
 __pycache__/

evals/README.md

@@ -46,9 +46,9 @@ claude plugin eval . --no-publish                                          # eve
 claude plugin eval . --case code-review-fires --runs 1 --ablation none     # one cheap iteration on one case
 ```
 
-A case is a directory holding `prompt.md` (frontmatter: turn and tool limits; body: the prompt, sent verbatim) and one grader per file under `graders/`. Reports land in `evals/results/<timestamp>/`, already gitignored. Every run and every `llm` grader is a real model call on your account, so this runs on demand and never in CI, like Tier 3.
+A case is a directory holding `prompt.md` (frontmatter: turn and tool limits; body: the prompt, sent verbatim) and one grader per file under `graders/`. Reports land in `evals/plugin/results/<timestamp>/`, already gitignored. Every run and every `llm` grader is a real model call on your account, so this runs on demand and never in CI, like Tier 3.
 
-Two rules keep it honest. The command scans all of `evals/` for `prompt.md` or `case.yaml`, so never give a fixture file either name. And a `tool_used: Skill` grader only says whether the skill fired; in a two-arm run it is excluded from the score, so pair it with a grader on the reply itself. `code-review-fires` shows the pattern: the fired indicator, a free `regex` grader that looks for the skill's severity taxonomy (Critical, Required, Nit, Optional, Consider, FYI) used as a heading or as a label, and one `llm` judge on the planted off-by-one. `code-review-stays-quiet` is the precision check: the review skill must not fire on a test-first request, and its `Δ` is expected to be 0; a second, unscored indicator records whether test-driven-development fires instead.
+Two rules keep it honest. The manifest's `experimental.evals` key scopes the scan to `evals/plugin/`, so a fixture elsewhere under `evals/` is never read as a case. And a `tool_used: Skill` grader only says whether the skill fired; in a two-arm run it is excluded from the score, so pair it with a grader on the reply itself. `code-review-fires` shows the pattern: the fired indicator, a free `regex` grader that looks for the skill's severity taxonomy (Critical, Required, Nit, Optional, Consider, FYI) used as a heading or as a label, and one `llm` judge on the planted off-by-one. `code-review-stays-quiet` is the precision check: the review skill must not fire on a test-first request, and its `Δ` is expected to be 0; a second, unscored indicator records whether test-driven-development fires instead.
 
 What the first runs taught (Claude Code 2.1.278, claude-opus-5). Skill invocation is stochastic: the review skill fired in 5 of 7 runs on the phrasing the case uses, and test-driven-development fired in 2 of 7 runs on the test-first prompt, so read `Δ` and the report rather than the exit code, or pass `--threshold 0.8` as the CI example in the docs does. In one run the review skill did not fire and the reply still used its severity taxonomy as headings, which no baseline run did in nine; the loaded skill descriptions may shape the reply even without an invocation. Phrasing matters: "Review this diff before I merge it", "Can you do a proper code review of this change before I merge it?" and "Review this pull request for me before it goes into main" never fired the skill in six runs; the model reviewed the diff on its own, competently but without the verdict, the severity tiers or the five-axis pass. Naming the axes ("across correctness, readability, security and performance") is what fires it, so the case uses that; the plainer phrasings are the open finding, and the fix, if any, belongs in the skill's description, not in the prompt. The model may also reach a skill through its slash command (`agent-skills:review`, then `code-review-and-quality`), which costs two turns, so give a case a turn budget of 10 or more.