mirror of
https://github.com/addyosmani/agent-skills.git
synced 2026-10-26 15:05:04 +00:00
Merge main: keep both sets of appended tests
This commit is contained in:
Addy Osmani committed
commit
279c1a8df9
21 files changed
+358
-132
No files matched your search
@@ -72,6 +72,14 @@ Before proposing a change, search the [skill-change rejection ledger](evals/skil
|
||||
|
||||
If a skill or description change is rejected based on eval results, add one row to the ledger with the date, affected skill, concise attempted change, before-to-after rank-1 score, and rejected PR link and outcome. Land that ledger-only update separately on the default branch; do not leave it only on the rejected proposal branch, where closing or force-pushing the proposal could discard the record.
|
||||
|
||||
## Adding a Host Guide
|
||||
|
||||
The README install section lists only hosts a maintainer has installed and run the current release on. Every other host gets one line in [docs/other-hosts.md](docs/other-hosts.md): the host name and its install command. No dedicated setup page, vendor links, logos, emoji, or screenshots.
|
||||
|
||||
- If you work for the company behind the host, say so in the PR.
|
||||
- A host moves into the README once a maintainer has run it and is willing to keep its entry current.
|
||||
- Host guides are reviewed in the order they were opened.
|
||||
|
||||
## Repo-scoped files
|
||||
|
||||
`AGENTS.md` and `CLAUDE.md` at the repo root configure agents working on the [`addyosmani/agent-skills`](https://github.com/addyosmani/agent-skills) repository itself. When writing setup guides or docs, do not instruct users to copy these files into their own projects or into a global agent configuration; the reusable assets are the skills in `skills/`.
|
||||
|
||||
@@ -97,24 +97,10 @@ claude --plugin-dir /path/to/agent-skills
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Oh My Pi (OMP)</b></summary>
|
||||
|
||||
OMP installs this repository's Claude Code marketplace plugin:
|
||||
|
||||
```bash
|
||||
omp plugin marketplace add addyosmani/agent-skills
|
||||
omp plugin install agent-skills@addy-agent-skills
|
||||
```
|
||||
|
||||
Restart OMP after installation; skills are discovered from their descriptions. See [docs/omp-setup.md](docs/omp-setup.md) for local clones and the OMP tool mapping.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Cursor</b></summary>
|
||||
|
||||
Put workflow skills under `.cursor/skills/` (sync from `agent-skills/skills/`) and short policies in `.cursor/rules/*.mdc` — do not paste full skills into rules. See [docs/cursor-setup.md](docs/cursor-setup.md).
|
||||
Put workflow skills under `.cursor/skills/` (sync from `agent-skills/skills/`) and short policies in `.cursor/rules/*.mdc` — do not paste full skills into rules. Both directories live in your project; this repo does not ship a `.cursor/` folder. See [docs/cursor-setup.md](docs/cursor-setup.md).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -176,7 +162,7 @@ See [docs/opencode-setup.md](docs/opencode-setup.md).
|
||||
<details>
|
||||
<summary><b>GitHub Copilot</b></summary>
|
||||
|
||||
Use agent definitions from `agents/` as Copilot personas and skill content in `.github/copilot-instructions.md`. See [docs/copilot-setup.md](docs/copilot-setup.md).
|
||||
Use agent definitions from `agents/` as Copilot personas and skill content in `.github/copilot-instructions.md`. The lifecycle slash commands (`/spec`, `/plan`, `/build`, `/test`, `/review`, `/ship`) are Claude Code commands and do not appear in Copilot; invoke the skills by name (`/spec-driven-development`, …) instead. See [docs/copilot-setup.md](docs/copilot-setup.md).
|
||||
|
||||
Using the standalone `copilot` CLI? Install it as a plugin — see [docs/copilot-cli-setup.md](docs/copilot-cli-setup.md).
|
||||
|
||||
@@ -201,21 +187,6 @@ The first command registers the marketplace; the second installs the plugin. Cod
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Dojo Workspace</b></summary>
|
||||
|
||||
Installing Dojo Workspace puts the `dojo` command on your PATH:
|
||||
|
||||
```bash
|
||||
dojo skills add addyosmani/agent-skills
|
||||
```
|
||||
|
||||
If `dojo` is not on your PATH, add its install folder: `~/.local/bin` on macOS/Linux (`~/.local/bin/dojo`) or `%LOCALAPPDATA%\dojo\bin` on Windows (`dojo.exe`). Or run the binary by its full path.
|
||||
|
||||
Skills install into `~/.agents/skills/`; enable the ones you want for each lane (Dojo Solo / Dojo Duo). Add `-p` to install into the current project's `.agents/skills/` instead. You can also install from the app's **Skills** panel → **Add External Skills**. See [docs/dojo-setup.md](docs/dojo-setup.md).
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Command Code</b></summary>
|
||||
|
||||
@@ -234,7 +205,7 @@ Installed skills show up in the TUI slash menu, e.g. `/spec-driven-development`.
|
||||
<details>
|
||||
<summary><b>Other Agents</b></summary>
|
||||
|
||||
Skills are plain Markdown - they work with any agent that accepts system prompts or instruction files. See [docs/getting-started.md](docs/getting-started.md).
|
||||
Skills are plain Markdown - they work with any agent that accepts system prompts or instruction files. See [docs/getting-started.md](docs/getting-started.md). Hosts that install the pack but aren't listed above are in [docs/other-hosts.md](docs/other-hosts.md).
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
@@ -4,6 +4,8 @@ This guide covers Copilot in VS Code. For the standalone `copilot` command-line
|
||||
|
||||
**What an install actually gives you:** the skills. Each installed skill becomes a slash command named after its frontmatter `name` — `/spec-driven-development`, `/test-driven-development`, and so on. `npx skills add addyosmani/agent-skills` and the manual copy below both install skills only. Neither of those two routes copies this repo's short lifecycle wrappers (`/spec`, `/plan`, `/build`, `/test`, `/review`, `/ship`) — those are Claude Code commands living in `.claude/commands/`. Use the full skill names, or add your own aliases — see [Lifecycle workflows](#lifecycle-workflows).
|
||||
|
||||
> **`/spec`, `/plan`, `/build`, `/test`, `/review`, `/code-simplify`, `/webperf` and `/ship` never appear in Copilot.** They are Claude Code commands, and Copilot does not read `.claude/commands/`. Not seeing them after an install is expected, not a missing step. Invoke the skill by its own name instead — `/spec-driven-development` for `/spec`, `/planning-and-task-breakdown` for `/plan`, and so on; the full mapping is in [Lifecycle Workflows](#lifecycle-workflows). On VS Code you can add [short aliases](#optional-short-spec-style-aliases) yourself if you want the short forms.
|
||||
|
||||
## Setup
|
||||
|
||||
### Copilot Instructions
|
||||
@@ -145,6 +147,8 @@ Write the body yourself, or take it from this table, rather than copying `.claud
|
||||
|
||||
### If the skill slash commands don't appear
|
||||
|
||||
First rule out the expected case: if the only commands you miss are the short lifecycle wrappers (`/spec`, `/plan`, `/build`, `/test`, `/review`, `/ship`), nothing is wrong — those are Claude Code commands and never surface in Copilot (see the note at the top). The steps below are for when the skills' own names, such as `/spec-driven-development`, are missing too.
|
||||
|
||||
Work through these in order — `/spec-driven-development` missing and `/spec` missing have different causes:
|
||||
|
||||
1. **Check where the skills landed.** Each one needs its own directory containing a `SKILL.md`. The project-level paths this guide uses are `.github/skills/`, `.claude/skills/` and `.agents/skills/` — but an installer run with a global/user flag writes outside the workspace instead (`~/.agents/skills/` and equivalents), so a skill that isn't in any of the three may simply be installed personally. Confirm the actual path against the [VS Code agent skills docs](https://code.visualstudio.com/docs/agent-customization/agent-skills), then re-install into the project if you wanted it project-scoped.
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
How to wire [agent-skills](../README.md) into **Cursor** using current, supported project context — not legacy monolith files or Kaizen-specific layouts.
|
||||
|
||||
Every `.cursor/` path in this guide lives in **your** project (or under `~/.cursor/` for user scope). This repository does not ship a `.cursor/` directory: you create it and sync the skills into it, which is what [step 1](#1-install-skills-into-cursorskills) does.
|
||||
|
||||
---
|
||||
|
||||
## What Cursor supports today
|
||||
@@ -205,6 +207,7 @@ Files under `agent-skills/agents/` (e.g. code reviewer persona) are **not** load
|
||||
| Stale workflow | Re-`rsync` from `agent-skills/skills/` |
|
||||
| Duplicate instructions | Remove skill content from rules; keep one source |
|
||||
| Wrong skill picked | Narrow `description` in custom skills; nudge in chat |
|
||||
| "An unexpected error occurred on our servers" right after attaching a skill with `/`, and every later message in that chat fails too | Known Cursor-side issue, tracked in [#494](https://github.com/addyosmani/agent-skills/issues/494): Cursor pulls the attached `SKILL.md` into the request and the failure happens on its side, not in the skill. Start a new chat and reference the skill by name in plain text ("Use the test-driven-development skill to …") instead of attaching it. |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,45 +0,0 @@
|
||||
# ⛩️ Using agent-skills with Dojo Workspace
|
||||
|
||||
Dojo Workspace is a desktop app that reads skills from the standard `.agents/skills/` folders. Installing it also puts the `dojo` command on your PATH.
|
||||
|
||||
If `dojo` is not found, add its install folder to your PATH, or run it by its full path:
|
||||
|
||||
| OS | Folder to add to PATH | Full path |
|
||||
| --- | --- | --- |
|
||||
| macOS / Linux | `~/.local/bin` | `~/.local/bin/dojo` |
|
||||
| Windows | `%LOCALAPPDATA%\dojo\bin` | `%LOCALAPPDATA%\dojo\bin\dojo.exe` |
|
||||
|
||||
Open a new terminal after the first app launch so the PATH change is picked up.
|
||||
|
||||
## Install for all projects
|
||||
|
||||
```bash
|
||||
dojo skills add addyosmani/agent-skills
|
||||
```
|
||||
|
||||
Skills install into `~/.agents/skills/` and appear in the **Skills** panel under the `addyosmani-agent-skills` group. Enable the ones you want for each lane (Dojo Solo or Dojo Duo).
|
||||
|
||||
## Install for one project
|
||||
|
||||
Run from the project root:
|
||||
|
||||
```bash
|
||||
dojo skills add -p addyosmani/agent-skills
|
||||
```
|
||||
|
||||
Skills install into the project's `.agents/skills/` and are enabled for that project. Commit `skills-lock.json` to share them with your team.
|
||||
|
||||
## Manage
|
||||
|
||||
```bash
|
||||
dojo skills list # add -p for the current project
|
||||
dojo skills remove addyosmani/agent-skills
|
||||
```
|
||||
|
||||
`remove` takes a whole repo or a single skill name.
|
||||
|
||||
## How skills load
|
||||
|
||||
Only each skill's name and description are listed for the model. The full `SKILL.md` loads on demand when a task matches, so enabling many skills stays cheap. Dojo also reads project instructions from `AGENTS.md`.
|
||||
|
||||
If you'd rather add skills through the app's UI, [here is how](https://heydojo.ai/learn/dojo-skills-teach-your-agent-new-workflows).
|
||||
@@ -1,41 +0,0 @@
|
||||
# Using agent-skills with Oh My Pi
|
||||
|
||||
[Oh My Pi (OMP)](https://github.com/can1357/oh-my-pi) installs Claude Code marketplace plugins, so it uses this repository's existing `.claude-plugin/` manifests and the root `skills/` directory. No OMP-specific files are needed.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
omp plugin marketplace add addyosmani/agent-skills
|
||||
omp plugin install agent-skills@addy-agent-skills
|
||||
```
|
||||
|
||||
The first command registers this repository as the `addy-agent-skills` marketplace. The second installs the `agent-skills` plugin from it. Restart OMP, or run `/reload-plugins` in an open session, so the skills are discovered.
|
||||
|
||||
Local clones work too:
|
||||
|
||||
```bash
|
||||
omp plugin marketplace add /path/to/your/clone
|
||||
omp plugin install agent-skills@addy-agent-skills
|
||||
```
|
||||
|
||||
Check the result with `omp plugin list`.
|
||||
|
||||
## Usage
|
||||
|
||||
Describe the task and let OMP pick the skill. OMP lists each skill's `name` and `description` in its system prompt and loads the full `SKILL.md` only when a skill is selected. To select a skill explicitly, use `/skill:spec-driven-development` (available unless `skills.enableSkillCommands` is disabled), or ask the agent to read `skill://spec-driven-development`.
|
||||
|
||||
OMP already routes skills natively. Do not also paste `using-agent-skills/SKILL.md` into `AGENTS.md`, `APPEND_SYSTEM.md`, a rule, or an extension that adds it to every run. That stacks the pack's meta-router on OMP's own router; see [Getting Started](getting-started.md).
|
||||
|
||||
The same install also loads the slash commands and the personas in `agents/`. OMP namespaces plugin commands, so run `/agent-skills:spec`, `/agent-skills:ship`, and so on; a bare `/spec` is not expanded. The personas (`code-reviewer`, `security-auditor`, `test-engineer`, `web-performance-auditor`) are listed as `task` agents, which `/agent-skills:ship` and `/agent-skills:webperf` dispatch.
|
||||
|
||||
## Tool mapping
|
||||
|
||||
Skills and docs in this pack use Claude Code tool names. In OMP, use the native equivalents:
|
||||
|
||||
| Skill instruction | OMP equivalent |
|
||||
| --- | --- |
|
||||
| Invoke another skill | `read` on `skill://<name>`, or `/skill:<name>` from the prompt |
|
||||
| Spawn a subagent (`Agent` tool) | the `task` tool; only use agents OMP lists |
|
||||
| Track tasks (`TodoWrite`) | the `todo` tool |
|
||||
| Ask the user a question (`AskUserQuestion`) | the `ask` tool |
|
||||
| Read, edit, write, search files | the native `read`, `edit`, `write`, `grep`, and `glob` tools |
|
||||
@@ -0,0 +1,10 @@
|
||||
# Other hosts
|
||||
|
||||
These hosts install the pack but are not in the README's install section, because no maintainer has run the current release on them. Each entry is one line: the host and its install command. To add one, see [Adding a Host Guide](../CONTRIBUTING.md#adding-a-host-guide).
|
||||
|
||||
Skills are plain `SKILL.md` folders, so any agent that reads the Agent Skills layout can load them. See [getting-started.md](getting-started.md).
|
||||
|
||||
- **Dojo Workspace**: `dojo skills add addyosmani/agent-skills`
|
||||
- **fx**: `git clone https://github.com/addyosmani/agent-skills.git /tmp/agent-skills && mkdir -p .fx/skills && cp -R /tmp/agent-skills/skills/* .fx/skills/` (project scope; use `~/.fx/skills/` instead for every workspace; needs fx 0.0.5 or later)
|
||||
- **Oh My Pi**: `omp plugin marketplace add addyosmani/agent-skills`, then `omp plugin install agent-skills@addy-agent-skills`
|
||||
- **Pi**: `pi install git:github.com/addyosmani/agent-skills`
|
||||
@@ -39,6 +39,24 @@
|
||||
"Trade-offs and rejected options are recorded, not just the winning choice",
|
||||
"The ADR records a status from the skill's lifecycle (Proposed, Accepted, Superseded or Deprecated) and a date"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 4,
|
||||
"kind": "execution",
|
||||
"prompt": "Review README.md for operators onboarding to this batch queue. Make the prose clearer, check it against queue.js, and keep the operational details and caveats accurate. Do not change the implementation.",
|
||||
"expected_output": "A concise operational README grounded in the inspected implementation, with unsupported assurances removed or flagged, limits and caveats preserved, and the unverified command identified",
|
||||
"files": [
|
||||
"documentation-and-adrs/prose-review"
|
||||
],
|
||||
"expectations": [
|
||||
"The final README identifies queuectl submit jobs.json --max-batch-size 50 as an unverified documented example because the fixture contains neither the CLI nor jobs.json; it does not invent, silently certify, or replace that interface",
|
||||
"The final README preserves the 50-job batch limit, 72-hour retention period, and at most three retries, distinguishing those units correctly",
|
||||
"The final README preserves the operational caveat that failed jobs require operator review and queue state is not durable across a process restart",
|
||||
"The unsupported claims of guaranteed zero job loss and no need for monitoring are removed or explicitly marked as unsubstantiated rather than restated as facts",
|
||||
"The promotional opening is replaced with a concrete description of the queue without invented users, benchmarks, certifications, or capabilities",
|
||||
"The execution trace shows the agent inspecting queue.js and comparing the README with its batch, retention, retry, and in-memory-state implementation before editing",
|
||||
"queue.js remains unchanged, and the agent does not install a tool or send the fixture to a hosted service"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
# Batch Queue
|
||||
|
||||
Batch Queue is an innovative, enterprise-grade solution that seamlessly unlocks
|
||||
the power of batch processing. In today's fast-paced landscape, it is important
|
||||
to note that this robust queue empowers operators to take work to the next level.
|
||||
|
||||
## Submit a batch
|
||||
|
||||
```sh
|
||||
queuectl submit jobs.json --max-batch-size 50
|
||||
```
|
||||
|
||||
A batch contains at most 50 jobs. Queue entries expire after 72 hours. Failed
|
||||
jobs are retried at most three times.
|
||||
|
||||
Operators review jobs that still fail after retries. Queue state is held in
|
||||
memory and is not durable across a process restart.
|
||||
|
||||
This guarantees zero job loss and means operators never need to monitor the queue.
|
||||
@@ -0,0 +1,32 @@
|
||||
'use strict';
|
||||
|
||||
const MAX_BATCH_SIZE = 50;
|
||||
const RETENTION_MS = 72 * 60 * 60 * 1000;
|
||||
const MAX_RETRIES = 3;
|
||||
const jobs = new Map();
|
||||
|
||||
function submit(batch, now = Date.now()) {
|
||||
if (batch.length > MAX_BATCH_SIZE) throw new Error('Batch exceeds limit');
|
||||
for (const job of batch) {
|
||||
jobs.set(job.id, { ...job, createdAt: now, retries: 0, state: 'queued' });
|
||||
}
|
||||
}
|
||||
|
||||
function recordFailure(id) {
|
||||
const job = jobs.get(id);
|
||||
if (!job) throw new Error('Unknown job');
|
||||
if (job.retries < MAX_RETRIES) {
|
||||
job.retries += 1;
|
||||
job.state = 'queued';
|
||||
} else {
|
||||
job.state = 'operator-review';
|
||||
}
|
||||
}
|
||||
|
||||
function expire(now = Date.now()) {
|
||||
for (const [id, job] of jobs) {
|
||||
if (now - job.createdAt >= RETENTION_MS) jobs.delete(id);
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { submit, recordFailure, expire, MAX_BATCH_SIZE, RETENTION_MS, MAX_RETRIES };
|
||||
@@ -9,7 +9,7 @@ const fs = require('node:fs');
|
||||
const os = require('node:os');
|
||||
const path = require('node:path');
|
||||
|
||||
const { lintSkillContent, lintSkillLayout, topLevelFrontmatterKeys } = require('./skill-lint.js');
|
||||
const { lintSkillContent, lintSkillLayout, topLevelFrontmatterKeys, parseFrontmatter, frontmatterYamlErrors, lintPersonaContent } = require('./skill-lint.js');
|
||||
|
||||
const KNOWN = new Set(['alpha', 'beta']);
|
||||
|
||||
@@ -648,3 +648,115 @@ test('the spec-key check tolerates CRLF frontmatter', () => {
|
||||
const { errors } = lintSkillContent('alpha', content, KNOWN);
|
||||
assert.equal(errors.filter(e => /key 'temperature'/.test(e)).length, 1);
|
||||
});
|
||||
|
||||
test('parseFrontmatter ignores comments and nested metadata keys', () => {
|
||||
const content = [
|
||||
'---',
|
||||
'name: alpha',
|
||||
'# note: this is a comment',
|
||||
'description: Designs alphas. Use when building one.',
|
||||
'metadata:',
|
||||
' name: beta',
|
||||
' description: Nested details without trigger',
|
||||
'---',
|
||||
'',
|
||||
].join('\n');
|
||||
assert.deepEqual(parseFrontmatter(content), {
|
||||
name: 'alpha',
|
||||
description: 'Designs alphas. Use when building one.',
|
||||
metadata: '',
|
||||
});
|
||||
});
|
||||
|
||||
test('frontmatterYamlErrors ignores comment lines containing colons or quotes', () => {
|
||||
const content = [
|
||||
'---',
|
||||
'name: alpha',
|
||||
'# note: a comment: with a colon and "unclosed quote',
|
||||
'description: "Designs alphas. Use when building one."',
|
||||
'---',
|
||||
'',
|
||||
].join('\n');
|
||||
assert.deepEqual(frontmatterYamlErrors(content), []);
|
||||
});
|
||||
|
||||
test('a skill with frontmatter comments and nested metadata passes validation', () => {
|
||||
const content = withAllSections([
|
||||
'---',
|
||||
'name: alpha',
|
||||
'# Note: keep description up to date: check docs',
|
||||
'description: Designs alphas. Use when building one.',
|
||||
'metadata:',
|
||||
' name: beta',
|
||||
' description: Internal notes',
|
||||
'---',
|
||||
].join('\n'));
|
||||
const { errors } = lintSkillContent('alpha', content, KNOWN);
|
||||
assert.deepEqual(errors, []);
|
||||
});
|
||||
|
||||
// ─── Personas (agents/<name>.md) ─────────────────────────────────────────────
|
||||
//
|
||||
// docs/agents.md: same frontmatter format as a skill, name equal to the file
|
||||
// stem, and the file ends with a Composition block. Nothing more is checked.
|
||||
|
||||
function personaFile({ name = 'code-reviewer', description = 'Reviews changes. Use before merging.', body } = {}) {
|
||||
const fm = ['---', name === null ? null : `name: ${name}`, description === null ? null : `description: ${description}`, '---']
|
||||
.filter(l => l !== null).join('\n');
|
||||
return `${fm}\n\n${body ?? '# Persona\n\n## Rules\n\nBe precise.\n\n## Composition\n\n- **Invoke via:** `/ship`.\n'}`;
|
||||
}
|
||||
|
||||
test('a conforming persona produces no errors', () => {
|
||||
assert.deepEqual(lintPersonaContent('code-reviewer', personaFile()).errors, []);
|
||||
});
|
||||
|
||||
test('persona: frontmatter name must match the file stem', () => {
|
||||
const { errors } = lintPersonaContent('code-reviewer', personaFile({ name: 'reviewer' }));
|
||||
assert.equal(errors.length, 1);
|
||||
assert.match(errors[0], /name 'reviewer' does not match file name 'code-reviewer'/);
|
||||
});
|
||||
|
||||
test('persona: description is required', () => {
|
||||
const { errors } = lintPersonaContent('code-reviewer', personaFile({ description: null }));
|
||||
assert.equal(errors.length, 1);
|
||||
assert.match(errors[0], /missing required field: 'description'/);
|
||||
});
|
||||
|
||||
test('persona: a missing frontmatter block is reported and nothing else is attempted', () => {
|
||||
const { errors } = lintPersonaContent('code-reviewer', '# Persona\n\n## Composition\n');
|
||||
assert.equal(errors.length, 1);
|
||||
assert.match(errors[0], /Missing or malformed YAML frontmatter/);
|
||||
});
|
||||
|
||||
test('persona: frontmatter a host would reject as YAML is reported', () => {
|
||||
const { errors } = lintPersonaContent('code-reviewer', personaFile({ description: 'Reviews: everything. Use before merging.' }));
|
||||
assert.equal(errors.length, 1);
|
||||
assert.match(errors[0], /Frontmatter line/);
|
||||
});
|
||||
|
||||
test('persona: the Composition section is required', () => {
|
||||
const { errors } = lintPersonaContent('code-reviewer', personaFile({ body: '# Persona\n\n## Rules\n\nBe precise.\n' }));
|
||||
assert.equal(errors.length, 1);
|
||||
assert.match(errors[0], /Missing "## Composition" section/);
|
||||
});
|
||||
|
||||
test('persona: Composition must be the last section', () => {
|
||||
const { errors } = lintPersonaContent('code-reviewer', personaFile({
|
||||
body: '# Persona\n\n## Composition\n\n- **Invoke via:** `/ship`.\n\n## Output Format\n\nA report.\n',
|
||||
}));
|
||||
assert.equal(errors.length, 1);
|
||||
assert.match(errors[0], /must be the last section, but "## Output Format" follows it/);
|
||||
});
|
||||
|
||||
test('persona: a heading inside a fenced example after Composition is not a later section', () => {
|
||||
const body = [
|
||||
'# Persona', '', '## Composition', '', '- **Invoke via:** `/ship`. Report in this shape:', '',
|
||||
'```markdown', '## Review: [title]', 'Findings…', '```', '',
|
||||
].join('\n');
|
||||
assert.deepEqual(lintPersonaContent('code-reviewer', personaFile({ body })).errors, []);
|
||||
});
|
||||
|
||||
test('persona: the Composition heading match is case-insensitive and tolerates trailing spaces', () => {
|
||||
const body = '# Persona\n\n## composition \n\n- **Invoke via:** `/ship`.\n';
|
||||
assert.deepEqual(lintPersonaContent('code-reviewer', personaFile({ body })).errors, []);
|
||||
});
|
||||
@@ -184,11 +184,12 @@ function parseFrontmatter(content) {
|
||||
|
||||
const result = {};
|
||||
for (const line of match[1].split(/\r?\n/)) {
|
||||
const colonIdx = line.indexOf(':');
|
||||
if (colonIdx === -1) continue;
|
||||
const key = line.slice(0, colonIdx).trim();
|
||||
const value = line.slice(colonIdx + 1).trim().replace(/^['"]|['"]$/g, '');
|
||||
if (key) result[key] = value;
|
||||
if (/^\s*#/.test(line)) continue;
|
||||
const m = line.match(/^([A-Za-z0-9_-]+)\s*:\s*(.*)$/);
|
||||
if (!m) continue;
|
||||
const key = m[1];
|
||||
const value = m[2].trim().replace(/^['"]|['"]$/g, '');
|
||||
result[key] = value;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
@@ -234,7 +235,7 @@ function frontmatterYamlErrors(content) {
|
||||
const lines = match[1].split(/\r?\n/);
|
||||
lines.forEach((line, i) => {
|
||||
const lineNo = i + 2; // the opening `---` is line 1
|
||||
if (!line.trim()) return;
|
||||
if (!line.trim() || /^\s*#/.test(line)) return;
|
||||
|
||||
if (/^[ ]*\t/.test(line)) {
|
||||
errors.push(
|
||||
@@ -543,6 +544,59 @@ function lintSkillLayout(skillDir) {
|
||||
return errors;
|
||||
}
|
||||
|
||||
/**
|
||||
* Lint a persona file (agents/<name>.md) against the three rules docs/agents.md
|
||||
* states for them: frontmatter in the same format as a skill, with `name`
|
||||
* equal to the file stem (the name is the `subagent_type` commands spawn) and
|
||||
* a `description`; and a closing `## Composition` section. Headings inside
|
||||
* fenced blocks are ignored, so an example after Composition does not count as
|
||||
* a later section. Pure: no filesystem access. Returns { errors }.
|
||||
*/
|
||||
function lintPersonaContent(stem, content) {
|
||||
const errors = [];
|
||||
|
||||
const fm = parseFrontmatter(content);
|
||||
if (!fm) {
|
||||
errors.push('Missing or malformed YAML frontmatter (expected --- block at top of file)');
|
||||
return { errors };
|
||||
}
|
||||
errors.push(...frontmatterYamlErrors(content));
|
||||
|
||||
if (!fm.name) {
|
||||
errors.push("Frontmatter missing required field: 'name'");
|
||||
} else if (fm.name !== stem) {
|
||||
errors.push(`Frontmatter name '${fm.name}' does not match file name '${stem}' (the name is the subagent_type commands spawn)`);
|
||||
}
|
||||
if (!fm.description) {
|
||||
errors.push("Frontmatter missing required field: 'description'");
|
||||
}
|
||||
|
||||
const headings = [...stripFencedCodeBlocks(content).matchAll(/^## +(.+?)\s*$/gm)].map(m => m[1]);
|
||||
const compositionAt = headings.findIndex(h => /^Composition$/i.test(h));
|
||||
if (compositionAt === -1) {
|
||||
errors.push('Missing "## Composition" section (docs/agents.md: every persona file ends with a Composition block)');
|
||||
} else if (compositionAt !== headings.length - 1) {
|
||||
errors.push(`"## Composition" must be the last section, but "## ${headings[headings.length - 1]}" follows it`);
|
||||
}
|
||||
|
||||
return { errors };
|
||||
}
|
||||
|
||||
/**
|
||||
* Lint a persona by file name: reads agents/<file>, then delegates to
|
||||
* lintPersonaContent. Returns { errors }.
|
||||
*/
|
||||
function lintPersona(fileName, agentsDir) {
|
||||
const stem = fileName.replace(/\.md$/, '');
|
||||
let content;
|
||||
try {
|
||||
content = fs.readFileSync(path.join(agentsDir, fileName), 'utf8');
|
||||
} catch (err) {
|
||||
return { errors: [`Unreadable persona file: ${err.message}`] };
|
||||
}
|
||||
return lintPersonaContent(stem, content);
|
||||
}
|
||||
|
||||
/**
|
||||
* Lint a skill by directory name: reads its SKILL.md, then delegates to
|
||||
* lintSkillContent. This is the thin filesystem wrapper the CLI uses.
|
||||
@@ -581,4 +635,6 @@ module.exports = {
|
||||
lintSkillContent,
|
||||
lintSkillLayout,
|
||||
lintSkill,
|
||||
lintPersonaContent,
|
||||
lintPersona,
|
||||
};
|
||||
@@ -623,6 +623,26 @@ test('materializes a git baseline and applies a working-tree patch', () => {
|
||||
});
|
||||
|
||||
|
||||
test('fixture baseline commits ignore inherited user signing requirements', () => {
|
||||
const root = makeSandbox();
|
||||
const config = path.join(root, 'global.gitconfig');
|
||||
const temp = fs.mkdtempSync(path.join(os.tmpdir(), 'fixture-signing-test-'));
|
||||
try {
|
||||
fs.writeFileSync(config, '[commit]\n\tgpgsign = true\n[gpg]\n\tprogram = definitely-missing-eval-signer\n');
|
||||
const result = spawnSync(process.execPath, ['-e', `
|
||||
const fs = require('node:fs');
|
||||
const { materializeWorkspace } = require(${JSON.stringify(path.join(root, 'scripts', 'run-evals.js'))});
|
||||
let workspace;
|
||||
try { workspace = materializeWorkspace({ files: ['project/context.txt'] }); }
|
||||
finally { if (workspace) fs.rmSync(workspace, { recursive: true, force: true }); }
|
||||
`], { cwd: root, encoding: 'utf8', env: { ...process.env, GIT_CONFIG_GLOBAL: config, TMPDIR: temp, TMP: temp, TEMP: temp } });
|
||||
assert.equal(result.status, 0, result.stdout + result.stderr);
|
||||
} finally {
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
fs.rmSync(temp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects duplicate behavioral IDs before they can overwrite a grading slot', () => {
|
||||
const root = makeSandbox();
|
||||
try {
|
||||
|
||||
@@ -427,6 +427,7 @@ function materializeWorkspace(ev) {
|
||||
// throwaway workspace.
|
||||
execFileSync('git', ['init', '--quiet'], { cwd: workspace });
|
||||
execFileSync('git', ['config', 'core.autocrlf', 'false'], { cwd: workspace });
|
||||
execFileSync('git', ['config', 'commit.gpgsign', 'false'], { cwd: workspace });
|
||||
execFileSync('git', ['config', 'user.name', 'Skill Eval'], { cwd: workspace });
|
||||
execFileSync('git', ['config', 'user.email', 'skill-eval@example.invalid'], { cwd: workspace });
|
||||
execFileSync('git', ['add', '--all'], { cwd: workspace });
|
||||
|
||||
@@ -493,3 +493,21 @@ test('a missing file is reported once, not again for its anchor', () => {
|
||||
assert.match(result.stdout, /L1: references\/renamed\.md — resolves to skills\/hardening\/references\/renamed\.md, which does not exist/);
|
||||
assert.match(result.stdout, /1 skills checked — 1 error\(s\) — FAILED/);
|
||||
});
|
||||
|
||||
|
||||
test('heading slugs remain unique when a literal suffix collides with an allocated slug', () => {
|
||||
const root = makeSandbox();
|
||||
hardeningSkill(root, 'references/patterns.md#example-1-1');
|
||||
// A hyphen in the literal heading makes its base slug collide with Example's suffix.
|
||||
writeFile(root, 'skills/hardening/references/patterns.md', '## Example\n\n## Example\n\n## Example-1\n');
|
||||
const result = run(root);
|
||||
assert.equal(result.status, 0, result.stdout + result.stderr);
|
||||
});
|
||||
|
||||
test('a duplicate skips a slug occupied by an earlier literal heading', () => {
|
||||
const root = makeSandbox();
|
||||
hardeningSkill(root, 'references/patterns.md#example-2');
|
||||
writeFile(root, 'skills/hardening/references/patterns.md', '## Example\n\n## Example-1\n\n## Example\n');
|
||||
const result = run(root);
|
||||
assert.equal(result.status, 0, result.stdout + result.stderr);
|
||||
});
|
||||
@@ -77,8 +77,10 @@ function headingAnchors(file) {
|
||||
const match = line.match(/^ {0,3}#{1,6}[ \t]+(.*?)(?:[ \t]+#+)?[ \t]*$/);
|
||||
if (!match) continue;
|
||||
const slug = slugify(match[1]);
|
||||
const count = seen.get(slug) || 0;
|
||||
anchors.add(count === 0 ? slug : `${slug}-${count}`);
|
||||
let count = seen.get(slug) || 0;
|
||||
let anchor = count === 0 ? slug : `${slug}-${count}`;
|
||||
while (anchors.has(anchor)) anchor = `${slug}-${++count}`;
|
||||
anchors.add(anchor);
|
||||
seen.set(slug, count + 1);
|
||||
}
|
||||
return anchors;
|
||||
|
||||
@@ -3,7 +3,8 @@
|
||||
* validate-skills.js
|
||||
*
|
||||
* CLI that validates every skill in skills/ against the rules in
|
||||
* docs/skill-anatomy.md. The rules themselves live in scripts/lib/skill-lint.js
|
||||
* docs/skill-anatomy.md, and every persona in agents/ against the three rules
|
||||
* in docs/agents.md. The rules themselves live in scripts/lib/skill-lint.js
|
||||
* (a single source of truth, importable and unit-testable); this file is a thin
|
||||
* wrapper that walks the skills directory, runs the linter, prints the report,
|
||||
* and sets the exit code.
|
||||
@@ -16,9 +17,10 @@
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
const { lintSkill } = require('./lib/skill-lint');
|
||||
const { lintSkill, lintPersona } = require('./lib/skill-lint');
|
||||
|
||||
const SKILLS_DIR = path.resolve(__dirname, '..', 'skills');
|
||||
const AGENTS_DIR = path.resolve(__dirname, '..', 'agents');
|
||||
|
||||
// ─── Main ────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -53,8 +55,29 @@ function main() {
|
||||
}
|
||||
}
|
||||
|
||||
// ── Personas ─────────────────────────────────────────────────────────────
|
||||
// agents/<name>.md files are consumed as subagents and spawned by name from
|
||||
// commands, so they get the same frontmatter contract plus a closing
|
||||
// Composition section (docs/agents.md). An index file such as README.md is
|
||||
// not a persona and is skipped.
|
||||
const personaFiles = fs.existsSync(AGENTS_DIR)
|
||||
? fs.readdirSync(AGENTS_DIR).filter(f => f.endsWith('.md') && !/^readme\.md$/i.test(f)).sort()
|
||||
: [];
|
||||
|
||||
if (personaFiles.length > 0) console.log('');
|
||||
for (const fileName of personaFiles) {
|
||||
const { errors } = lintPersona(fileName, AGENTS_DIR);
|
||||
totalErrors += errors.length;
|
||||
if (errors.length === 0) {
|
||||
console.log(` ✓ agents/${fileName}`);
|
||||
} else {
|
||||
console.log(` ✗ agents/${fileName}`);
|
||||
for (const msg of errors) console.log(` ERROR: ${msg}`);
|
||||
}
|
||||
}
|
||||
|
||||
const status = totalErrors > 0 ? 'FAILED' : totalWarnings > 0 ? 'PASSED WITH WARNINGS' : 'PASSED';
|
||||
console.log(`\n${skillDirs.length} skills checked — ${totalErrors} error(s), ${totalWarnings} warning(s) — ${status}`);
|
||||
console.log(`\n${skillDirs.length} skills and ${personaFiles.length} personas checked — ${totalErrors} error(s), ${totalWarnings} warning(s) — ${status}`);
|
||||
|
||||
if (totalErrors > 0) process.exit(1);
|
||||
}
|
||||
|
||||
@@ -256,6 +256,14 @@ Special consideration for AI agent context:
|
||||
- **ADRs** — Help agents understand why past decisions were made (prevents re-deciding)
|
||||
- **Inline gotchas** — Prevent agents from falling into known traps
|
||||
|
||||
## Prose Review
|
||||
|
||||
After checking the technical content, review the prose for the people who will use it. Clearer wording must not change the decision or its limits.
|
||||
|
||||
1. Mark the details that must survive the edit: names, numbers and units, commands, paths, links, conditions, trade-offs, and uncertainty. Check them against the available code and decision context; flag anything you cannot substantiate.
|
||||
2. Replace stock introductions, promotional adjectives, and repeated summaries with the concrete information the reader needs. Do not replace an unsupported claim with an invented example or a stronger guarantee. Keep necessary technical repetition and the project's established terminology.
|
||||
3. Compare the edited document with the original and its sources. Restore any lost qualifier or caveat, verify command and code examples still work as documented, and explain any unresolved factual question rather than polishing it away.
|
||||
|
||||
## Common Rationalizations
|
||||
|
||||
| Rationalization | Reality |
|
||||
@@ -286,3 +294,4 @@ After documenting:
|
||||
- [ ] Known gotchas are documented inline where they matter
|
||||
- [ ] No commented-out code remains
|
||||
- [ ] Rules files (CLAUDE.md etc.) are current and accurate
|
||||
- [ ] Prose edits preserve source details, decision trade-offs, and qualifiers without adding unsupported guarantees
|
||||
@@ -79,7 +79,7 @@ export function TaskItem({ task, onToggle, onDelete }: TaskItemProps) {
|
||||
```tsx
|
||||
// Container: handles data
|
||||
export function TaskListContainer() {
|
||||
const { tasks, isLoading, error } = useTasks();
|
||||
const { tasks, isLoading, error, refetch } = useTasks();
|
||||
|
||||
if (isLoading) return <TaskListSkeleton />;
|
||||
if (error) return <ErrorState message="Failed to load tasks" retry={refetch} />;
|
||||
|
||||
@@ -159,7 +159,7 @@ The `range() !== 'unicast'` check covers loopback, link-local `169.254.169.254`
|
||||
import { z } from 'zod';
|
||||
|
||||
const CreateTaskSchema = z.object({
|
||||
title: z.string().min(1).max(200).trim(),
|
||||
title: z.string().max(200).trim().min(1),
|
||||
description: z.string().max(2000).optional(),
|
||||
priority: z.enum(['low', 'medium', 'high']).default('medium'),
|
||||
dueDate: z.string().datetime().optional(),
|
||||
|
||||
@@ -190,6 +190,12 @@ Client metrics:
|
||||
```typescript
|
||||
// Set up error boundary with reporting
|
||||
class ErrorBoundary extends React.Component {
|
||||
state = { hasError: false };
|
||||
|
||||
static getDerivedStateFromError() {
|
||||
return { hasError: true };
|
||||
}
|
||||
|
||||
componentDidCatch(error: Error, info: React.ErrorInfo) {
|
||||
// Report to error tracking service
|
||||
reportError(error, {
|
||||
|
||||
Reference in new issue
Block a user