Affaan Mustafa
c70874fae9
Adds a curated, Pi-native, skills+prompts-only profile at pi/core/ for downstream packagers that mirror GitHub Releases. - manifests/pi-core.json: explicit include lists, per-item exclusion reasons, curation rules, safety allowlists; every root skill and command must be classified. - scripts/build-pi-core.js regenerates pi/core deterministically (package.json from VERSION, LICENSE, README.md, CURATION.md, skills/, commands/); --check fails CI on drift. council is renamed ecc-council inside pi/core only. - Safety checks: no callable endpoints outside the allowlist, no npx/curl|sh/pip install, no secrets, no absolute home paths, no symlinks, valid frontmatter, no duplicate names. - CI: build + drift check and an offline Pi CLI load test of pi/core. - Release: VERSION, package.json and pi/core/package.json at 2.2.2, CHANGELOG, tag-triggered release verification, and a two-week cadence in CONTRIBUTING.md. pi/core: 123 of 293 skills and 24 of 94 commands; 35,006 characters of skill description text.
2.4 KiB
Hook Integration for Session-Stop Self-Evaluation
Add this hook to hooks/hooks.json to remind the agent to self-evaluate at the end of every session (the hook echoes a reminder; it does not run the evaluator automatically):
{
"hooks": {
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "echo '[Self-Eval] Session complete. Consider running agent-self-evaluation to rate your output.'"
}
],
"description": "Remind agent to self-evaluate at session end"
}
]
}
}
Stop events do not require a matcher field (it is optional for Stop, Notification, UserPromptSubmit, and SubagentStop per scripts/ci/validate-hooks.js). If omitted, the hook object only needs hooks and metadata such as description.
Integration with the Python Evaluator
The scripts/evaluate.py script can be used as a standalone tool:
# Pipe agent output directly
echo "Your agent response here" | python3 skills/agent-self-evaluation/scripts/evaluate.py
# From files
python3 skills/agent-self-evaluation/scripts/evaluate.py --task task.txt --output response.txt
To integrate it into hooks, capture the last agent output to a file first, then run the evaluator. For lightweight reminders after shell-based verification, use a simple supported matcher string:
{
"hooks": {
"PostToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "echo '[Self-Eval] If this command completed verification for a non-trivial task, consider running agent-self-evaluation.'"
}
],
"description": "Remind agent to self-evaluate after shell verification"
}
]
}
}
This avoids documenting unsupported command-expression matcher syntax. If your harness supports command-level matcher expressions, prefer a word-boundary regex such as \b(pytest|npm test|go test)\b rather than a broad test substring.
These hooks are opt-in. Add them to your local hooks/hooks.json if you want automated evaluation prompts.
Manual Usage (Recommended)
The most reliable approach is manual invocation — the agent runs self-evaluation as part of its workflow when the agent-self-evaluation skill is active, without requiring hook configuration. The skill's "When to Activate" section already covers trigger conditions (multi-file changes, debugging sessions, design documents).