mirror of
https://github.com/addyosmani/agent-skills.git
synced 2026-10-26 15:05:04 +00:00
fix(evals): add missing owner to negative trigger tests
Six negative-trigger prompts across six case files declared no `owner`, so per evals/README.md and scripts/run-evals.js they could only pass vacuously (skip the pairwise routing assertion) instead of asserting that the correct skill actually outranks the tested one. Added owner for the prompts where a real, unambiguous owner exists and verified against scripts/run-evals.js that it outranks the tested skill with a non-zero score: - code-simplification: "Add a feature flag system to the app" -> incremental-implementation - deprecation-and-migration: "Add dark mode to the settings page" -> frontend-ui-engineering - doubt-driven-development: "Write the changelog entry for the release" -> git-workflow-and-versioning - frontend-ui-engineering: "Tag and publish the release" -> git-workflow-and-versioning - planning-and-task-breakdown: "Encode the output so it is safe against XSS" -> security-and-hardening - using-agent-skills: "Make the modal accessible for keyboard users" -> frontend-ui-engineering Left four other owner-less negatives untouched (doubt-driven-development's "Format this file with prettier", git-workflow-and-versioning's "Add a loading spinner to the submit button", observability-and-instrumentation's "Draft the PRD for the referral program", and shipping-and-launch's "Why is this function so slow on large inputs?") because no skill scores above zero against them under the current TF-IDF scorer, so declaring an owner there would either fail CI or force a misleading pick. Verified: node scripts/run-evals.js still passes (127 checks, 0 errors, 86% rank-1) and node scripts/validate-commands.js passes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
df1edb2e05
commit
6ff91e06e8
6 files changed
+12
-6
No files matched your search
@@ -17,7 +17,8 @@
|
||||
],
|
||||
"negative": [
|
||||
{
|
||||
"prompt": "Add a feature flag system to the app"
|
||||
"prompt": "Add a feature flag system to the app",
|
||||
"owner": "incremental-implementation"
|
||||
},
|
||||
{
|
||||
"prompt": "Diagnose why the build broke overnight",
|
||||
|
||||
@@ -17,7 +17,8 @@
|
||||
],
|
||||
"negative": [
|
||||
{
|
||||
"prompt": "Add dark mode to the settings page"
|
||||
"prompt": "Add dark mode to the settings page",
|
||||
"owner": "frontend-ui-engineering"
|
||||
},
|
||||
{
|
||||
"prompt": "Why is this test flaky on CI?",
|
||||
|
||||
@@ -20,7 +20,8 @@
|
||||
"prompt": "Format this file with prettier"
|
||||
},
|
||||
{
|
||||
"prompt": "Write the changelog entry for the release"
|
||||
"prompt": "Write the changelog entry for the release",
|
||||
"owner": "git-workflow-and-versioning"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -29,7 +29,8 @@
|
||||
"owner": "performance-optimization"
|
||||
},
|
||||
{
|
||||
"prompt": "Tag and publish the release"
|
||||
"prompt": "Tag and publish the release",
|
||||
"owner": "git-workflow-and-versioning"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -21,7 +21,8 @@
|
||||
"owner": "debugging-and-error-recovery"
|
||||
},
|
||||
{
|
||||
"prompt": "Encode the output so it is safe against XSS"
|
||||
"prompt": "Encode the output so it is safe against XSS",
|
||||
"owner": "security-and-hardening"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -21,7 +21,8 @@
|
||||
"owner": "debugging-and-error-recovery"
|
||||
},
|
||||
{
|
||||
"prompt": "Make the modal accessible for keyboard users"
|
||||
"prompt": "Make the modal accessible for keyboard users",
|
||||
"owner": "frontend-ui-engineering"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
Reference in new issue
Block a user