diff --git a/.agents/skills/ecc-contribution-review/SKILL.md b/.agents/skills/ecc-contribution-review/SKILL.md new file mode 100644 index 000000000..bb659d7cd --- /dev/null +++ b/.agents/skills/ecc-contribution-review/SKILL.md @@ -0,0 +1,67 @@ +--- +name: ecc-contribution-review +description: Prepare an ECC skill contribution for review by organizing repository, install-surface, cross-harness, validation, security, and commit observations into a local proposal. Use when preparing a skill contribution for review. +metadata: + origin: ECC +--- + +# ECC Skill Contribution Review Packet + +Use this workflow to prepare skill contribution observations for maintainer review. +This version supports `changeKind: "skill"` with canonical source under `skills/`. +It compiles supplied observations; it does not independently inspect Git, +files, CI, test results, coverage, or referenced evidence. Treat every imported +claim as unverified until a reviewer checks its evidence. + +## Workflow + +1. Start from the active ECC checkout. Record the branch, base and head, package + lock, install manifests, catalog generator, skill placement policy, and a + checkpoint. +2. Identify the canonical source under `skills/` for curated workflows. Keep + generated, learned, and imported content outside shipped sources. +3. Record the change intent, artifact graph, install coverage, harness copies, + catalog and registry counts, test phases, coverage, security review, + budgets, and commit proposal with evidence references. +4. Run the required validators and tests separately. Record exact commands, + results, and evidence. The compiler does not execute them. +5. Copy the template, then edit the copy to replace every placeholder with + current observations and references to evidence you gathered. The template + is intentionally incomplete and must not be treated as an observation. + After editing it, compile the packet: + + ```bash + mkdir -p .memory-kit + cp skills/ecc-contribution-review/templates/contribution-review-observation.json \ + .memory-kit/contribution-review-observation.json + python3 skills/ecc-contribution-review/scripts/compile_contribution_review.py \ + .memory-kit/contribution-review-observation.json \ + --output .memory-kit/ecc-contribution-review.json + ``` + + `--output` writes a new capsule file in the existing parent directory and + refuses to overwrite any existing path. The compiler does not create + directories, modify source files, run tests, update catalogs, commit, push, + or open a pull request. Exit code 0 means a packet was emitted without + reported faults or contradictions; review is still required. Exit code 2 + means validation faults, contradictions, or an input/output error. A + diagnostic packet may still be emitted with exit code 2. Inspect the packet + and stderr; the exit code alone does not establish whether output exists. + +6. Review every reported contradiction and semantic page fault. The compiler + returns `require_review` for structurally valid input and never approves a + contribution. It does not prove that supplied claims are true or CI passed. +7. Recompile after changes to HEAD, policy, manifests, catalog, adapters, tests, + coverage, security findings, or review evidence. + +## Local Contract + +The packet describes source ownership, generated-root exclusion, install +coverage, normalized harness identity, catalog reconciliation, TDD claims, +security review, validation gates, and a conventional-commit proposal. Runner +proposals are typed argument arrays and remain unexecuted. All generated +authority and external effects remain false. + +See [the workflow](docs/WORKFLOW.md), [the design](docs/DESIGN.md), and [the +contract](references/contribution-review-contract.md). The JSON input is an +illustrative fixture, not a current repository observation. diff --git a/.agents/skills/ecc-contribution-review/agents/openai.yaml b/.agents/skills/ecc-contribution-review/agents/openai.yaml new file mode 100644 index 000000000..27e248582 --- /dev/null +++ b/.agents/skills/ecc-contribution-review/agents/openai.yaml @@ -0,0 +1,7 @@ +interface: + display_name: "ECC Skill Contribution Review" + short_description: "Prepare evidence-aware ECC skill review packets" + brand_color: "#0EA5E9" + default_prompt: "Use $ecc-contribution-review to prepare an ECC skill contribution review packet from current observations and evidence." +policy: + allow_implicit_invocation: true diff --git a/.agents/skills/ecc-contribution-review/docs/DESIGN.md b/.agents/skills/ecc-contribution-review/docs/DESIGN.md new file mode 100644 index 000000000..4d4981081 --- /dev/null +++ b/.agents/skills/ecc-contribution-review/docs/DESIGN.md @@ -0,0 +1,28 @@ +# ECC cross-harness skill contribution review + +This compiler converts supplied skill observations into a structured contribution +review packet. It supports only `changeKind: "skill"`, with canonical source +under `skills/` and the required skill artifacts and harness mirrors. It checks +the declared source ownership, generated-root exclusion, install coverage, cross-harness identity, catalog reconciliation, +TDD claims, security claims, validation gates, and commit proposal for internal +consistency. It does not inspect the repository or independently verify any +claim or referenced evidence, so it cannot establish contribution readiness. + +PCCA ownership: + +- observation-and-data: repository, diff, artifact, gate, count, and coverage + observations; +- epistemic-and-lineage: base/head, hashes, canonical source, mirrors, and test + evidence; +- state-and-relationship: artifact graph, harness parity, manifests, catalogs, + and checkpoint; +- cognitive-control: TDD, gate selection, coverage, and commit formulation; +- meta-control: contradiction detection, page faults, hostile probes, and typed + runners; +- viability-and-policy: security, provenance, breaking-change, budget, and + external-effect boundaries. + +The semantic checksum uses sorted compact UTF-8 JSON and excludes +`artifactId`, `semanticChecksum`, and `returnReceipt`. The compiler writes a +capsule file only when the caller supplies `--output`; the receipt distinguishes +that generated artifact from implementation-file changes. diff --git a/.agents/skills/ecc-contribution-review/docs/WORKFLOW.md b/.agents/skills/ecc-contribution-review/docs/WORKFLOW.md new file mode 100644 index 000000000..8d354d4c4 --- /dev/null +++ b/.agents/skills/ecc-contribution-review/docs/WORKFLOW.md @@ -0,0 +1,30 @@ +# Skill contribution workflow + +1. Resolve the active ECC checkout, non-default branch, base/head, package lock, + manifests, catalog script, placement policy, and checkpoint. +2. Use `changeKind: "skill"` and name the canonical source under `skills/` + before editing a mirror. This version supports skill contributions only. +3. Observe RED with named failing behaviors. +4. Implement GREEN in the canonical surface, then REFACTOR. +5. Update only required install manifests, documentation, and harness adapters. +6. Normalize content identity across Claude, Codex, and Cursor; record every + intentional frontmatter or metadata adaptation. +7. Reconcile agent, skill, command, manifest, catalog, registry, and intentionally + unshipped counts. +8. Run the eight exact validation gates and record argv, exit status, and + evidence. +9. Check secrets, personal paths, permissions, hooks, prompt-injection boundary, + supply chain, coverage, breaking-change status, and budgets. +10. Compile the review packet. The compiler checks supplied claims for shape and + internal consistency but does not verify referenced evidence or approve a + contribution. Exit code 0 means a packet was emitted without reported + faults or contradictions; review is still required. Exit code 2 means + validation faults, contradictions, or an input/output error. A diagnostic + packet may still be emitted with exit code 2. Inspect the packet and stderr; + the exit code alone does not establish whether output exists. Resolve + every contradiction and semantic page fault before preparing a commit. +11. The compiler runs no proposed runners. Obtain separate authorization + before implementation-file, test, catalog, commit, push, PR, Control Store, + canonical-memory, or Semantic Commit effects. +12. Supersede the packet on HEAD, policy, manifest, catalog, adapter, test, + coverage, security, or review drift. diff --git a/.agents/skills/ecc-contribution-review/references/contribution-review-contract.md b/.agents/skills/ecc-contribution-review/references/contribution-review-contract.md new file mode 100644 index 000000000..80b01a294 --- /dev/null +++ b/.agents/skills/ecc-contribution-review/references/contribution-review-contract.md @@ -0,0 +1,36 @@ +# ECC skill contribution review contract + +The compiler prepares packets for skill contributions only +(`changeKind: "skill"`). It validates the shape and internal consistency of +supplied observations; it does not independently +inspect repository files, Git state, CI, tests, coverage, or evidence refs. +Every structurally valid packet returns `require_review`. Contradictions +return `reject`. Neither result approves or qualifies a contribution. + +The envelope requires `schemaVersion: "1.0.0"`, an ISO 8601 `observedAt` +timestamp with a timezone, and a nonempty string `subjectRef`. Unsupported +versions and malformed identifiers produce validation faults. + +The local observation contract checks for: + +- exact active repository, non-default branch, base/head, package lock, install + manifests, catalog script, placement policy, and checkpoint identity; +- a bounded, non-breaking skill change under `skills/` with no unnecessary + command shim; +- one canonical curated source, explicit non-generated artifacts, manifest + coverage, tests, and required harness metadata; +- normalized content parity across Claude, Codex, and Cursor with zero drift; +- filesystem/documented counts, manifest coverage, intentional unshipped + exceptions, catalog check, and command registry reconciled, with nonnegative + curated and intentionally unshipped skill counts; +- observed RED/GREEN/REFACTOR, named failing tests, passing behavior tests, and + line/function/branch/statement coverage between 80% and 100% inclusive; +- eight green validators, security and supply-chain review, bounded change/test + budgets, twelve hostile probes, and seven typed runners; +- a concise conventional commit proposal, while commit/push/PR effects remain + false; +- no source-file, test, catalog, git, PR, Control Store, canonical-memory, or + Semantic Commit claim or requested effect; +- every generated `may*` authority false. + +Contradictions reject. Missing or malformed observations require review. diff --git a/.agents/skills/ecc-conventions/SKILL.md b/.agents/skills/ecc-conventions/SKILL.md index 3ff05dd06..5e88eda81 100644 --- a/.agents/skills/ecc-conventions/SKILL.md +++ b/.agents/skills/ecc-conventions/SKILL.md @@ -441,3 +441,68 @@ Based on analysis of the codebase, follow these practices: --- *This skill was auto-generated by [ECC Tools](https://ecc.tools). Review and customize as needed for your team.* + +## Cross-harness skill contribution review workflow + +For ECC skill contributions, including their related tests, catalogs, and +install surfaces, follow the canonical +[`ecc-contribution-review` skill](../../../skills/ecc-contribution-review/SKILL.md). +This version supports only `changeKind: "skill"`. Its compiler checks the +structure and consistency of supplied observations; it does not independently +verify Git state, files, CI, tests, coverage, or +referenced evidence. Treat its output as a review packet, not proof of +contribution readiness. + +Start from the active ECC checkout and preserve the exact branch, base and head +identities, package lock, three install manifests, catalog generator, skill +placement policy, and checkpoint as an **Observation Envelope**. Bind the +change intent, canonical source, curated/generated classification, artifact +graph, install-manifest coverage, Claude/Codex/Cursor normalized content +identity, catalog counts, command registry, TDD phases, coverage, security +review, budgets, and conventional-commit proposal. + +The controller enforces the repository's deeper invariants: + +1. `skills/` is canonical for curated workflows; learned, imported, evolved, + and other generated roots cannot silently become shipped source; +2. cross-harness copies are adaptations with one normalized content identity, + not independent forks; +3. filesystem, documented, manifest, catalog, command-registry, and intentional + unshipped counts must reconcile; +4. RED, GREEN, and REFACTOR evidence must be observed, behavior tests must pass, + and all coverage dimensions must meet the 80% floor; +5. strict skill frontmatter, install manifests, harness adapters, Unicode, + personal-path, full test, coverage, and registry gates must all be green; +6. secrets, unsafe permissions, hook escalation, supply-chain drift, prompt + injection, breaking changes, and budget overruns fail closed; and +7. the commit remains a proposal until every gate is current and external git + authority is separately granted. + +The output is a **Context Capsule** and local **Proposal**, never a **Semantic Commit**. +The compiler returns `require_review` for structurally valid input and never +approves a contribution. The output capsule may be written to the requested +path. All generated `may*` authority is false; all seven runners are typed, +unexecuted, and separately authorized. + +The capsule spans all six **PCCA** planes and carries an **Epistemic Ledger**, +evidence/counterevidence, omissions, residual uncertainty, a contradiction scan +marked **computed**, every **semantic page fault**, **Memory Metabolism** +revalidation/supersession triggers, a deterministic **semantic checksum**, and +an effect report that distinguishes generated capsule output from implementation +file changes. + +```bash +mkdir -p .memory-kit +cp skills/ecc-contribution-review/templates/contribution-review-observation.json \ + .memory-kit/contribution-review-observation.json +# Fill the copied observation with current facts and evidence references first. +python3 skills/ecc-contribution-review/scripts/compile_contribution_review.py \ + .memory-kit/contribution-review-observation.json \ + --output .memory-kit/ecc-contribution-review.json +python3 -m unittest discover -s skills/ecc-contribution-review/tests -p 'test*.py' +``` + +See `skills/ecc-contribution-review/docs/WORKFLOW.md`, +`skills/ecc-contribution-review/docs/DESIGN.md`, and +`skills/ecc-contribution-review/references/contribution-review-contract.md` for +the local contract. diff --git a/.cursor/skills/ecc-contribution-review/SKILL.md b/.cursor/skills/ecc-contribution-review/SKILL.md new file mode 100644 index 000000000..bb659d7cd --- /dev/null +++ b/.cursor/skills/ecc-contribution-review/SKILL.md @@ -0,0 +1,67 @@ +--- +name: ecc-contribution-review +description: Prepare an ECC skill contribution for review by organizing repository, install-surface, cross-harness, validation, security, and commit observations into a local proposal. Use when preparing a skill contribution for review. +metadata: + origin: ECC +--- + +# ECC Skill Contribution Review Packet + +Use this workflow to prepare skill contribution observations for maintainer review. +This version supports `changeKind: "skill"` with canonical source under `skills/`. +It compiles supplied observations; it does not independently inspect Git, +files, CI, test results, coverage, or referenced evidence. Treat every imported +claim as unverified until a reviewer checks its evidence. + +## Workflow + +1. Start from the active ECC checkout. Record the branch, base and head, package + lock, install manifests, catalog generator, skill placement policy, and a + checkpoint. +2. Identify the canonical source under `skills/` for curated workflows. Keep + generated, learned, and imported content outside shipped sources. +3. Record the change intent, artifact graph, install coverage, harness copies, + catalog and registry counts, test phases, coverage, security review, + budgets, and commit proposal with evidence references. +4. Run the required validators and tests separately. Record exact commands, + results, and evidence. The compiler does not execute them. +5. Copy the template, then edit the copy to replace every placeholder with + current observations and references to evidence you gathered. The template + is intentionally incomplete and must not be treated as an observation. + After editing it, compile the packet: + + ```bash + mkdir -p .memory-kit + cp skills/ecc-contribution-review/templates/contribution-review-observation.json \ + .memory-kit/contribution-review-observation.json + python3 skills/ecc-contribution-review/scripts/compile_contribution_review.py \ + .memory-kit/contribution-review-observation.json \ + --output .memory-kit/ecc-contribution-review.json + ``` + + `--output` writes a new capsule file in the existing parent directory and + refuses to overwrite any existing path. The compiler does not create + directories, modify source files, run tests, update catalogs, commit, push, + or open a pull request. Exit code 0 means a packet was emitted without + reported faults or contradictions; review is still required. Exit code 2 + means validation faults, contradictions, or an input/output error. A + diagnostic packet may still be emitted with exit code 2. Inspect the packet + and stderr; the exit code alone does not establish whether output exists. + +6. Review every reported contradiction and semantic page fault. The compiler + returns `require_review` for structurally valid input and never approves a + contribution. It does not prove that supplied claims are true or CI passed. +7. Recompile after changes to HEAD, policy, manifests, catalog, adapters, tests, + coverage, security findings, or review evidence. + +## Local Contract + +The packet describes source ownership, generated-root exclusion, install +coverage, normalized harness identity, catalog reconciliation, TDD claims, +security review, validation gates, and a conventional-commit proposal. Runner +proposals are typed argument arrays and remain unexecuted. All generated +authority and external effects remain false. + +See [the workflow](docs/WORKFLOW.md), [the design](docs/DESIGN.md), and [the +contract](references/contribution-review-contract.md). The JSON input is an +illustrative fixture, not a current repository observation. diff --git a/.cursor/skills/ecc-contribution-review/docs/DESIGN.md b/.cursor/skills/ecc-contribution-review/docs/DESIGN.md new file mode 100644 index 000000000..4d4981081 --- /dev/null +++ b/.cursor/skills/ecc-contribution-review/docs/DESIGN.md @@ -0,0 +1,28 @@ +# ECC cross-harness skill contribution review + +This compiler converts supplied skill observations into a structured contribution +review packet. It supports only `changeKind: "skill"`, with canonical source +under `skills/` and the required skill artifacts and harness mirrors. It checks +the declared source ownership, generated-root exclusion, install coverage, cross-harness identity, catalog reconciliation, +TDD claims, security claims, validation gates, and commit proposal for internal +consistency. It does not inspect the repository or independently verify any +claim or referenced evidence, so it cannot establish contribution readiness. + +PCCA ownership: + +- observation-and-data: repository, diff, artifact, gate, count, and coverage + observations; +- epistemic-and-lineage: base/head, hashes, canonical source, mirrors, and test + evidence; +- state-and-relationship: artifact graph, harness parity, manifests, catalogs, + and checkpoint; +- cognitive-control: TDD, gate selection, coverage, and commit formulation; +- meta-control: contradiction detection, page faults, hostile probes, and typed + runners; +- viability-and-policy: security, provenance, breaking-change, budget, and + external-effect boundaries. + +The semantic checksum uses sorted compact UTF-8 JSON and excludes +`artifactId`, `semanticChecksum`, and `returnReceipt`. The compiler writes a +capsule file only when the caller supplies `--output`; the receipt distinguishes +that generated artifact from implementation-file changes. diff --git a/.cursor/skills/ecc-contribution-review/docs/WORKFLOW.md b/.cursor/skills/ecc-contribution-review/docs/WORKFLOW.md new file mode 100644 index 000000000..8d354d4c4 --- /dev/null +++ b/.cursor/skills/ecc-contribution-review/docs/WORKFLOW.md @@ -0,0 +1,30 @@ +# Skill contribution workflow + +1. Resolve the active ECC checkout, non-default branch, base/head, package lock, + manifests, catalog script, placement policy, and checkpoint. +2. Use `changeKind: "skill"` and name the canonical source under `skills/` + before editing a mirror. This version supports skill contributions only. +3. Observe RED with named failing behaviors. +4. Implement GREEN in the canonical surface, then REFACTOR. +5. Update only required install manifests, documentation, and harness adapters. +6. Normalize content identity across Claude, Codex, and Cursor; record every + intentional frontmatter or metadata adaptation. +7. Reconcile agent, skill, command, manifest, catalog, registry, and intentionally + unshipped counts. +8. Run the eight exact validation gates and record argv, exit status, and + evidence. +9. Check secrets, personal paths, permissions, hooks, prompt-injection boundary, + supply chain, coverage, breaking-change status, and budgets. +10. Compile the review packet. The compiler checks supplied claims for shape and + internal consistency but does not verify referenced evidence or approve a + contribution. Exit code 0 means a packet was emitted without reported + faults or contradictions; review is still required. Exit code 2 means + validation faults, contradictions, or an input/output error. A diagnostic + packet may still be emitted with exit code 2. Inspect the packet and stderr; + the exit code alone does not establish whether output exists. Resolve + every contradiction and semantic page fault before preparing a commit. +11. The compiler runs no proposed runners. Obtain separate authorization + before implementation-file, test, catalog, commit, push, PR, Control Store, + canonical-memory, or Semantic Commit effects. +12. Supersede the packet on HEAD, policy, manifest, catalog, adapter, test, + coverage, security, or review drift. diff --git a/.cursor/skills/ecc-contribution-review/references/contribution-review-contract.md b/.cursor/skills/ecc-contribution-review/references/contribution-review-contract.md new file mode 100644 index 000000000..80b01a294 --- /dev/null +++ b/.cursor/skills/ecc-contribution-review/references/contribution-review-contract.md @@ -0,0 +1,36 @@ +# ECC skill contribution review contract + +The compiler prepares packets for skill contributions only +(`changeKind: "skill"`). It validates the shape and internal consistency of +supplied observations; it does not independently +inspect repository files, Git state, CI, tests, coverage, or evidence refs. +Every structurally valid packet returns `require_review`. Contradictions +return `reject`. Neither result approves or qualifies a contribution. + +The envelope requires `schemaVersion: "1.0.0"`, an ISO 8601 `observedAt` +timestamp with a timezone, and a nonempty string `subjectRef`. Unsupported +versions and malformed identifiers produce validation faults. + +The local observation contract checks for: + +- exact active repository, non-default branch, base/head, package lock, install + manifests, catalog script, placement policy, and checkpoint identity; +- a bounded, non-breaking skill change under `skills/` with no unnecessary + command shim; +- one canonical curated source, explicit non-generated artifacts, manifest + coverage, tests, and required harness metadata; +- normalized content parity across Claude, Codex, and Cursor with zero drift; +- filesystem/documented counts, manifest coverage, intentional unshipped + exceptions, catalog check, and command registry reconciled, with nonnegative + curated and intentionally unshipped skill counts; +- observed RED/GREEN/REFACTOR, named failing tests, passing behavior tests, and + line/function/branch/statement coverage between 80% and 100% inclusive; +- eight green validators, security and supply-chain review, bounded change/test + budgets, twelve hostile probes, and seven typed runners; +- a concise conventional commit proposal, while commit/push/PR effects remain + false; +- no source-file, test, catalog, git, PR, Control Store, canonical-memory, or + Semantic Commit claim or requested effect; +- every generated `may*` authority false. + +Contradictions reject. Missing or malformed observations require review. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 145a02a77..6b99eee3d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -393,13 +393,13 @@ jobs: run: python -m pip install --upgrade pip && python -m pip install -e '.[dev]' - name: Run ruff (lint) - run: python -m ruff check src tests + run: python -m ruff check src tests skills/ecc-contribution-review/scripts skills/ecc-contribution-review/tests - name: Run mypy (type check) - run: python -m mypy src + run: python -m mypy src skills/ecc-contribution-review/scripts/compile_contribution_review.py - name: Run Python tests - run: python -m pytest tests/test_*.py -m "not integration" + run: python -m pytest tests/test_*.py skills/ecc-contribution-review/tests/ -m "not integration" - name: Test minimum supported OpenAI SDK run: | @@ -435,6 +435,31 @@ jobs: - name: Run supply-chain IOC scan run: npm run security:ioc-scan + - name: Setup Python for contribution security checks + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: '3.11' + + - name: Install contribution security tools + run: python -m pip install bandit==1.9.4 pytest==9.1.1 pyyaml==6.0.3 + + - name: Scan contribution compiler security + run: python skills/ecc-contribution-review/tests/record_security_evidence.py --output security-bandit.json + + - name: Test contribution security boundaries + if: success() || failure() + run: python -m pytest skills/ecc-contribution-review/tests/test_security_regression.py skills/ecc-contribution-review/tests/test_security_evidence.py -q --junitxml=security-regression.xml + + - name: Upload contribution security evidence + if: always() + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: contribution-security-evidence + path: | + security-bandit.json + security-regression.xml + if-no-files-found: error + coverage: name: Coverage runs-on: ubuntu-latest @@ -451,6 +476,14 @@ jobs: with: node-version: '20.x' + - name: Setup Python + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: '3.11' + + - name: Install Python dependencies + run: python -m pip install --upgrade pip && python -m pip install -e '.[dev]' + - name: Install dependencies run: npm ci --ignore-scripts @@ -464,6 +497,9 @@ jobs: - name: Run coverage run: npm run coverage + - name: Run Python coverage + run: npm run coverage:python + - name: Upload coverage report if: always() uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 diff --git a/agent.yaml b/agent.yaml index c57e8d16a..ea38624a2 100644 --- a/agent.yaml +++ b/agent.yaml @@ -69,6 +69,7 @@ skills: - documentation-lookup - docs-governance - e2e-testing + - ecc-contribution-review - energy-procurement - enterprise-agent-ops - error-handling diff --git a/docs/COMMAND-AGENT-MAP.md b/docs/COMMAND-AGENT-MAP.md index 6db89ecd9..f768e146e 100644 --- a/docs/COMMAND-AGENT-MAP.md +++ b/docs/COMMAND-AGENT-MAP.md @@ -82,6 +82,7 @@ This document lists each slash command and the primary agent(s) or skills it inv | Skill | Purpose | Entry point | |-------|---------|-------------| +| [ecc-contribution-review](../skills/ecc-contribution-review/SKILL.md) | Prepare a review packet for an ECC skill contribution from current repository observations | Invoke the skill, then verify every imported claim and evidence reference independently. | | [osint-investigation](../skills/osint-investigation/SKILL.md) | Public-source investigation methodology: competing hypotheses, falsification, visual verification, and evidence tracking | Invoke the self-contained skill directly using available host capabilities. | ## How to use this map diff --git a/manifests/install-components.json b/manifests/install-components.json index a581c39aa..767b3527e 100644 --- a/manifests/install-components.json +++ b/manifests/install-components.json @@ -502,6 +502,14 @@ "workflow-quality" ] }, + { + "id": "skill:ecc-contribution-review", + "family": "skill", + "description": "Optional skill contribution review packet workflow for ECC repositories.", + "modules": [ + "ecc-contribution-review" + ] + }, { "id": "skill:unified-memory", "family": "skill", diff --git a/manifests/install-modules.json b/manifests/install-modules.json index 16e26fab6..63291ffca 100644 --- a/manifests/install-modules.json +++ b/manifests/install-modules.json @@ -1471,6 +1471,35 @@ "cost": "light", "stability": "beta" }, + { + "id": "ecc-contribution-review", + "kind": "skills", + "description": "Optional skill contribution review packet workflow for ECC repositories.", + "paths": [ + "skills/ecc-contribution-review" + ], + "targets": [ + "claude", + "claude-project", + "cursor", + "antigravity", + "codex", + "copilot", + "opencode", + "codebuddy", + "joycode", + "qwen", + "zed", + "hermes", + "openclaw", + "kimi", + "adal" + ], + "dependencies": [], + "defaultInstall": false, + "cost": "light", + "stability": "beta" + }, { "id": "shared-script-utils", "kind": "platform", diff --git a/manifests/install-profiles.json b/manifests/install-profiles.json index d46400143..2b89c2e49 100644 --- a/manifests/install-profiles.json +++ b/manifests/install-profiles.json @@ -107,7 +107,8 @@ "documentation-change-impact", "documentation-module-governance", "documentation-test-regression", - "config-health" + "config-health", + "ecc-contribution-review" ] } } diff --git a/manifests/pi-core.json b/manifests/pi-core.json index f1cca2dc7..29600e2f3 100644 --- a/manifests/pi-core.json +++ b/manifests/pi-core.json @@ -205,6 +205,7 @@ "counterparty-channel-discipline": "agent messaging ops policy", "crosspost": "social/marketing distribution", "customer-billing-ops": "billing/ops skill over connected billing tools", + "ecc-contribution-review": "ECC-specific contribution packet compiler over repository manifests and cross-harness review evidence", "customs-trade-compliance": "customs/trade niche domain", "data-scraper-agent": "scheduled network scraping agent", "deep-research": "requires firecrawl and exa SaaS MCP tools", diff --git a/package.json b/package.json index 3b56b9733..1b2e60b49 100644 --- a/package.json +++ b/package.json @@ -428,6 +428,7 @@ "skills/django-celery/", "skills/documentation-lookup/", "skills/docs-governance/", + "skills/ecc-contribution-review/", "skills/ecc-guide/", "skills/ecc-recipes/", "skills/flox-environments/", @@ -539,10 +540,10 @@ "orchestrate:tmux": "node scripts/orchestrate-worktrees.js", "test:validators": "node scripts/ci/check-unicode-safety.js && node scripts/ci/validate-agents.js && node scripts/ci/validate-commands.js && node scripts/ci/validate-rules.js && node scripts/ci/validate-skills.js && node scripts/ci/validate-skill-host-compat.js && node scripts/ci/validate-hooks.js && node scripts/ci/check-hooks-schema-keys.js && node scripts/ci/validate-install-manifests.js && node scripts/ci/validate-context-profiles.js && node scripts/ci/validate-carrier-closure.js && node scripts/ci/validate-no-personal-paths.js && npm run catalog:check && npm run command-registry:check", "test:js": "node tests/run-all.js", - "test:python": "python3 -m pytest tests/ -q", + "test:python": "python3 -m unittest discover -s skills/ecc-contribution-review/tests -p 'test*.py' && python3 -m pytest tests/ -q", "test": "node scripts/ci/check-unicode-safety.js && node scripts/ci/validate-agents.js && node scripts/ci/validate-commands.js && node scripts/ci/validate-rules.js && node scripts/ci/validate-skills.js && node scripts/ci/validate-skill-host-compat.js && node scripts/ci/validate-hooks.js && node scripts/ci/check-hooks-schema-keys.js && node scripts/ci/validate-install-manifests.js && node scripts/ci/validate-context-profiles.js && node scripts/ci/validate-carrier-closure.js && node scripts/ci/validate-no-personal-paths.js && npm run catalog:check && npm run command-registry:check && node tests/run-all.js", "coverage:js": "c8 --all --include=\"scripts/**/*.js\" --include=\"scripts/**/*.mjs\" --check-coverage --lines 80 --functions 80 --branches 80 --statements 80 --reporter=text --reporter=lcov node tests/run-all.js", - "coverage:python": "python3 -m pytest tests/ -q --cov=src --cov-report=term --cov-fail-under=80", + "coverage:python": "python3 -m pytest tests/ -m \"not integration\" -q --cov=src --cov-report=term --cov-fail-under=80 && python3 -m pytest skills/ecc-contribution-review/tests/ -q --cov=skills/ecc-contribution-review/scripts --cov-report=term --cov-fail-under=80", "coverage": "c8 --all --include=\"scripts/**/*.js\" --include=\"scripts/**/*.mjs\" --check-coverage --lines 80 --functions 80 --branches 80 --statements 80 --reporter=text --reporter=lcov node tests/run-all.js", "build:opencode": "node scripts/build-opencode.js", "prepack": "npm run build:opencode", diff --git a/pi/core/CURATION.md b/pi/core/CURATION.md index 7ba8b79ee..2b4cc1c07 100644 --- a/pi/core/CURATION.md +++ b/pi/core/CURATION.md @@ -1,6 +1,6 @@ # Curation -pi/core includes 134 of 314 skills and 24 of 98 commands from the root of ECC. +pi/core includes 134 of 315 skills and 24 of 98 commands from the root of ECC. Everything excluded is listed here with its reason. ## Rules @@ -53,6 +53,7 @@ Exclude anything that: | `counterparty-channel-discipline` | agent messaging ops policy | | `crosspost` | social/marketing distribution | | `customer-billing-ops` | billing/ops skill over connected billing tools | +| `ecc-contribution-review` | ECC-specific contribution packet compiler over repository manifests and cross-harness review evidence | | `customs-trade-compliance` | customs/trade niche domain | | `data-scraper-agent` | scheduled network scraping agent | | `deep-research` | requires firecrawl and exa SaaS MCP tools | diff --git a/skills/ecc-contribution-review/SKILL.md b/skills/ecc-contribution-review/SKILL.md new file mode 100644 index 000000000..bb659d7cd --- /dev/null +++ b/skills/ecc-contribution-review/SKILL.md @@ -0,0 +1,67 @@ +--- +name: ecc-contribution-review +description: Prepare an ECC skill contribution for review by organizing repository, install-surface, cross-harness, validation, security, and commit observations into a local proposal. Use when preparing a skill contribution for review. +metadata: + origin: ECC +--- + +# ECC Skill Contribution Review Packet + +Use this workflow to prepare skill contribution observations for maintainer review. +This version supports `changeKind: "skill"` with canonical source under `skills/`. +It compiles supplied observations; it does not independently inspect Git, +files, CI, test results, coverage, or referenced evidence. Treat every imported +claim as unverified until a reviewer checks its evidence. + +## Workflow + +1. Start from the active ECC checkout. Record the branch, base and head, package + lock, install manifests, catalog generator, skill placement policy, and a + checkpoint. +2. Identify the canonical source under `skills/` for curated workflows. Keep + generated, learned, and imported content outside shipped sources. +3. Record the change intent, artifact graph, install coverage, harness copies, + catalog and registry counts, test phases, coverage, security review, + budgets, and commit proposal with evidence references. +4. Run the required validators and tests separately. Record exact commands, + results, and evidence. The compiler does not execute them. +5. Copy the template, then edit the copy to replace every placeholder with + current observations and references to evidence you gathered. The template + is intentionally incomplete and must not be treated as an observation. + After editing it, compile the packet: + + ```bash + mkdir -p .memory-kit + cp skills/ecc-contribution-review/templates/contribution-review-observation.json \ + .memory-kit/contribution-review-observation.json + python3 skills/ecc-contribution-review/scripts/compile_contribution_review.py \ + .memory-kit/contribution-review-observation.json \ + --output .memory-kit/ecc-contribution-review.json + ``` + + `--output` writes a new capsule file in the existing parent directory and + refuses to overwrite any existing path. The compiler does not create + directories, modify source files, run tests, update catalogs, commit, push, + or open a pull request. Exit code 0 means a packet was emitted without + reported faults or contradictions; review is still required. Exit code 2 + means validation faults, contradictions, or an input/output error. A + diagnostic packet may still be emitted with exit code 2. Inspect the packet + and stderr; the exit code alone does not establish whether output exists. + +6. Review every reported contradiction and semantic page fault. The compiler + returns `require_review` for structurally valid input and never approves a + contribution. It does not prove that supplied claims are true or CI passed. +7. Recompile after changes to HEAD, policy, manifests, catalog, adapters, tests, + coverage, security findings, or review evidence. + +## Local Contract + +The packet describes source ownership, generated-root exclusion, install +coverage, normalized harness identity, catalog reconciliation, TDD claims, +security review, validation gates, and a conventional-commit proposal. Runner +proposals are typed argument arrays and remain unexecuted. All generated +authority and external effects remain false. + +See [the workflow](docs/WORKFLOW.md), [the design](docs/DESIGN.md), and [the +contract](references/contribution-review-contract.md). The JSON input is an +illustrative fixture, not a current repository observation. diff --git a/skills/ecc-contribution-review/agents/openai.yaml b/skills/ecc-contribution-review/agents/openai.yaml new file mode 100644 index 000000000..27e248582 --- /dev/null +++ b/skills/ecc-contribution-review/agents/openai.yaml @@ -0,0 +1,7 @@ +interface: + display_name: "ECC Skill Contribution Review" + short_description: "Prepare evidence-aware ECC skill review packets" + brand_color: "#0EA5E9" + default_prompt: "Use $ecc-contribution-review to prepare an ECC skill contribution review packet from current observations and evidence." +policy: + allow_implicit_invocation: true diff --git a/skills/ecc-contribution-review/docs/DESIGN.md b/skills/ecc-contribution-review/docs/DESIGN.md new file mode 100644 index 000000000..4d4981081 --- /dev/null +++ b/skills/ecc-contribution-review/docs/DESIGN.md @@ -0,0 +1,28 @@ +# ECC cross-harness skill contribution review + +This compiler converts supplied skill observations into a structured contribution +review packet. It supports only `changeKind: "skill"`, with canonical source +under `skills/` and the required skill artifacts and harness mirrors. It checks +the declared source ownership, generated-root exclusion, install coverage, cross-harness identity, catalog reconciliation, +TDD claims, security claims, validation gates, and commit proposal for internal +consistency. It does not inspect the repository or independently verify any +claim or referenced evidence, so it cannot establish contribution readiness. + +PCCA ownership: + +- observation-and-data: repository, diff, artifact, gate, count, and coverage + observations; +- epistemic-and-lineage: base/head, hashes, canonical source, mirrors, and test + evidence; +- state-and-relationship: artifact graph, harness parity, manifests, catalogs, + and checkpoint; +- cognitive-control: TDD, gate selection, coverage, and commit formulation; +- meta-control: contradiction detection, page faults, hostile probes, and typed + runners; +- viability-and-policy: security, provenance, breaking-change, budget, and + external-effect boundaries. + +The semantic checksum uses sorted compact UTF-8 JSON and excludes +`artifactId`, `semanticChecksum`, and `returnReceipt`. The compiler writes a +capsule file only when the caller supplies `--output`; the receipt distinguishes +that generated artifact from implementation-file changes. diff --git a/skills/ecc-contribution-review/docs/SECURITY-EVIDENCE.md b/skills/ecc-contribution-review/docs/SECURITY-EVIDENCE.md new file mode 100644 index 000000000..06d9e20c0 --- /dev/null +++ b/skills/ecc-contribution-review/docs/SECURITY-EVIDENCE.md @@ -0,0 +1,65 @@ +# Contribution review security evidence + +Recorded 2026-10-17 against the source hashes in +`../tests/security-evidence/source-sha256.json`. +The scanner output is `../tests/security-evidence/bandit.json`. +These are observed local results, not an upstream CI or maintainer approval. + +## Scope and results + +- Bandit 1.9.4: both compiler modules scanned, zero findings, zero skipped tests, + no `nosec` suppressions, exit 0. +- Python 3.14.3, pytest 9.1.1, pytest-cov 7.1.0: 45 tests passed, + including 19 focused security/provenance tests and 8 review regression tests; compiler branch-enabled combined + coverage 95.79%, above the 80% gate. +- Evidence recorder: 7 provenance regressions passed, 88.68% combined coverage. +- Ruff 0.17.1: changed Python source and tests passed. +- Mypy 2.4.0: compiler passed. +- Workflow security validator: 13 workflows passed. + +Security regressions exercise rejected authority escalation, malformed input, +malformed failure probes, non-skill labels, envelope values, impossible numeric +claims, normal and safe-path (`-P`/`-I`) CLI invocation, function/module size policy, non-finite JSON numbers, invalid UTF-8, existing-file and symlink preservation, +output receipt accuracy, canonical/parity record agreement, and non-execution +of supplied runner argv. Source hashes for both compiler modules, all four test modules, the evidence recorder, and canonical +agent metadata are verified by a security regression in CI; stale evidence fails +the job. +Both new agent metadata files are checked for identical +content and absence of extra tool, dependency, or permission configuration. +The metadata retains ECC's existing implicit-invocation convention. + +## Reproduce + +From the repository root, with the development dependencies and Bandit 1.9.4: + +```sh +python skills/ecc-contribution-review/tests/record_security_evidence.py --output security-bandit.json +python -m pytest skills/ecc-contribution-review/tests/ -q --cov=skills/ecc-contribution-review/scripts --cov-report=term --cov-fail-under=80 +python -m ruff check skills/ecc-contribution-review/scripts skills/ecc-contribution-review/tests +python -m mypy skills/ecc-contribution-review/scripts/compile_contribution_review.py +node scripts/ci/validate-workflow-security.js +``` + +The recorder captures the exact source SHA-256 inventory before and after the +pinned Bandit scan, rejects source drift, and embeds that inventory and scanner +version in the report. Regression checks compare embedded hashes directly to +current source bytes and require the scanned metrics inventory to match. A +same-line source edit remains stale even if `source-sha256.json` is refreshed. +Regenerate the committed report with the same recorder and an output path of +`skills/ecc-contribution-review/tests/security-evidence/bandit.json`, then refresh +the separate source manifest after all source/test/recorder edits. + +The Security Scan CI job now reruns this recorder and the focused security regressions. +It fails on any scanner finding or failed regression, and uploads machine-readable +Bandit JSON and JUnit results as `contribution-security-evidence` even on failure. + +## Limits + +Embedded provenance records which source bytes were scanned; it is not a +cryptographic attestation against deliberate evidence editing. CI independently +reruns the pinned scanner and fails on findings, errors, or source drift. +Bandit checks Python source patterns; the configuration regression checks the +supported metadata boundary. Neither proves absence of every security defect. +Observation content and evidence references remain unverified claims and cannot +grant authority. The compiler writes only a newly created requested output file; +reviewers must still authorize any later execution of proposed commands. diff --git a/skills/ecc-contribution-review/docs/WORKFLOW.md b/skills/ecc-contribution-review/docs/WORKFLOW.md new file mode 100644 index 000000000..8d354d4c4 --- /dev/null +++ b/skills/ecc-contribution-review/docs/WORKFLOW.md @@ -0,0 +1,30 @@ +# Skill contribution workflow + +1. Resolve the active ECC checkout, non-default branch, base/head, package lock, + manifests, catalog script, placement policy, and checkpoint. +2. Use `changeKind: "skill"` and name the canonical source under `skills/` + before editing a mirror. This version supports skill contributions only. +3. Observe RED with named failing behaviors. +4. Implement GREEN in the canonical surface, then REFACTOR. +5. Update only required install manifests, documentation, and harness adapters. +6. Normalize content identity across Claude, Codex, and Cursor; record every + intentional frontmatter or metadata adaptation. +7. Reconcile agent, skill, command, manifest, catalog, registry, and intentionally + unshipped counts. +8. Run the eight exact validation gates and record argv, exit status, and + evidence. +9. Check secrets, personal paths, permissions, hooks, prompt-injection boundary, + supply chain, coverage, breaking-change status, and budgets. +10. Compile the review packet. The compiler checks supplied claims for shape and + internal consistency but does not verify referenced evidence or approve a + contribution. Exit code 0 means a packet was emitted without reported + faults or contradictions; review is still required. Exit code 2 means + validation faults, contradictions, or an input/output error. A diagnostic + packet may still be emitted with exit code 2. Inspect the packet and stderr; + the exit code alone does not establish whether output exists. Resolve + every contradiction and semantic page fault before preparing a commit. +11. The compiler runs no proposed runners. Obtain separate authorization + before implementation-file, test, catalog, commit, push, PR, Control Store, + canonical-memory, or Semantic Commit effects. +12. Supersede the packet on HEAD, policy, manifest, catalog, adapter, test, + coverage, security, or review drift. diff --git a/skills/ecc-contribution-review/references/contribution-review-contract.md b/skills/ecc-contribution-review/references/contribution-review-contract.md new file mode 100644 index 000000000..80b01a294 --- /dev/null +++ b/skills/ecc-contribution-review/references/contribution-review-contract.md @@ -0,0 +1,36 @@ +# ECC skill contribution review contract + +The compiler prepares packets for skill contributions only +(`changeKind: "skill"`). It validates the shape and internal consistency of +supplied observations; it does not independently +inspect repository files, Git state, CI, tests, coverage, or evidence refs. +Every structurally valid packet returns `require_review`. Contradictions +return `reject`. Neither result approves or qualifies a contribution. + +The envelope requires `schemaVersion: "1.0.0"`, an ISO 8601 `observedAt` +timestamp with a timezone, and a nonempty string `subjectRef`. Unsupported +versions and malformed identifiers produce validation faults. + +The local observation contract checks for: + +- exact active repository, non-default branch, base/head, package lock, install + manifests, catalog script, placement policy, and checkpoint identity; +- a bounded, non-breaking skill change under `skills/` with no unnecessary + command shim; +- one canonical curated source, explicit non-generated artifacts, manifest + coverage, tests, and required harness metadata; +- normalized content parity across Claude, Codex, and Cursor with zero drift; +- filesystem/documented counts, manifest coverage, intentional unshipped + exceptions, catalog check, and command registry reconciled, with nonnegative + curated and intentionally unshipped skill counts; +- observed RED/GREEN/REFACTOR, named failing tests, passing behavior tests, and + line/function/branch/statement coverage between 80% and 100% inclusive; +- eight green validators, security and supply-chain review, bounded change/test + budgets, twelve hostile probes, and seven typed runners; +- a concise conventional commit proposal, while commit/push/PR effects remain + false; +- no source-file, test, catalog, git, PR, Control Store, canonical-memory, or + Semantic Commit claim or requested effect; +- every generated `may*` authority false. + +Contradictions reject. Missing or malformed observations require review. diff --git a/skills/ecc-contribution-review/scripts/compile_contribution_review.py b/skills/ecc-contribution-review/scripts/compile_contribution_review.py new file mode 100644 index 000000000..7714fd898 --- /dev/null +++ b/skills/ecc-contribution-review/scripts/compile_contribution_review.py @@ -0,0 +1,362 @@ +#!/usr/bin/env python3 +"""Compile supplied ECC contribution observations into a review packet.""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import sys +from pathlib import Path +from typing import Any + +# Resolve the bundled validator from this script, including under -P/-I. +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from contribution_review_validation import ( + AUTHORITY_FIELDS, + is_sha256, + validate, +) +from contribution_review_validation import ( + RUNNERS as RUNNERS, +) +from contribution_review_validation import ( + safe_repo_path as safe_repo_path, +) + + +def canonical(value: Any) -> Any: + """Normalize a value for a deterministic, receipt-independent checksum.""" + if isinstance(value, dict): + return { + key: canonical(item) + for key, item in sorted(value.items()) + if key not in {"artifactId", "semanticChecksum", "returnReceipt"} + } + if isinstance(value, list): + return [canonical(item) for item in value] + return value + + +def checksum(value: Any) -> str: + encoded = json.dumps( + canonical(value), sort_keys=True, separators=(",", ":"), ensure_ascii=False + ).encode("utf-8") + return hashlib.sha256(encoded).hexdigest() + + +def reject_nonstandard_json_number(value: str) -> None: + """Reject Python json's non-standard NaN and Infinity extensions.""" + raise ValueError(f"non-standard JSON number: {value}") + + +def mapping(data: dict[str, Any], field: str) -> dict[str, Any]: + value = data.get(field, {}) + return value if isinstance(value, dict) else {} + + +def sequence(data: dict[str, Any], field: str) -> list[Any]: + value = data.get(field, []) + return value if isinstance(value, list) else [] + + +def _packet_identity(data: dict[str, Any]) -> dict[str, Any]: + change_intent = mapping(data, "changeIntent") + repo_identity = mapping(data, "repoIdentity") + return { + "artifactType": "ecc-cross-harness-contribution-review-packet", + "schemaVersion": "1.0.0", + "artifactId": "ecc-contribution-review-packet", + "generatedAt": data.get("observedAt"), + "subjectRef": data.get("subjectRef"), + "planeOwnership": { + "primary": "meta-control", + "supporting": [ + "observation-and-data", + "epistemic-and-lineage", + "state-and-relationship", + "cognitive-control", + "viability-and-policy", + ], + }, + "observationEnvelopes": [ + { + "sourcePathRef": "observation-envelope", + "repoIdentity": repo_identity, + "changeIntent": change_intent, + "surfaceContract": mapping(data, "surfaceContract"), + } + ], + } + + +def _packet_review( + data: dict[str, Any], + faults: list[dict[str, Any]], + contradictions: list[dict[str, Any]], +) -> dict[str, Any]: + change_intent = mapping(data, "changeIntent") + checkpoint = mapping(data, "checkpointContract") + reasons = sorted({issue["code"] for issue in faults + contradictions}) + decision = "reject" if contradictions else "require_review" + return { + "authorityEnvelope": {key: False for key in sorted(AUTHORITY_FIELDS)}, + "policyEnvelope": { + "proposalOnly": True, + "canonicalFirst": True, + "generatedRootsExcluded": True, + "externalEffectsAllowed": False, + "semanticCommitAllowed": False, + "importedClaimsIndependentlyVerified": False, + }, + "controllerDecision": { + "decision": decision, + "reasonCodes": reasons + or ["imported_observations_require_independent_review"], + }, + "contradictionScan": "computed", + "contradictions": contradictions, + "semanticPageFaults": faults, + "counterevidence": list(change_intent.get("counterevidenceRefs", [])) + if isinstance(change_intent.get("counterevidenceRefs"), list) + else [], + "omissions": [ + "The compiler validates supplied claims but does not inspect the repository or referenced evidence.", + "The compiler does not execute tests, update catalogs, commit, push, or open a pull request.", + "A capsule file is written only to a new path explicitly supplied by the caller.", + ], + "residualUncertainty": list(checkpoint.get("remainingUnknowns", [])) + if isinstance(checkpoint.get("remainingUnknowns"), list) + else [], + } + + +def _packet_payload( + data: dict[str, Any], decision: str, output_receipt: dict[str, Any] +) -> dict[str, Any]: + artifacts = sequence(data, "artifactRecords") + validation = sequence(data, "validationMatrix") + change_intent = mapping(data, "changeIntent") + repo_identity = mapping(data, "repoIdentity") + outcome = "reject" if decision == "reject" else "review_packet_compiled" + return { + "domainPayload": { + "contributionOutcome": outcome, + "assessmentBasis": "unverified imported claims; internal consistency only", + "repoClaims": repo_identity, + "changeClaims": change_intent, + "surfaceClaims": mapping(data, "surfaceContract"), + "artifactClaims": artifacts, + "harnessParityClaims": mapping(data, "harnessParity"), + "catalogClaims": mapping(data, "catalogContract"), + "validationClaims": validation, + "tddClaims": mapping(data, "tddContract"), + "securityClaims": mapping(data, "securityContract"), + "budgetClaims": mapping(data, "budgetContract"), + "commitClaims": mapping(data, "commitContract"), + "failureProbeCoverage": { + item.get("failureMode"): item.get("expectedOutcome") + == item.get("observedOutcome") + for item in sequence(data, "failureProbes") + if isinstance(item, dict) and isinstance(item.get("failureMode"), str) + }, + "runnerProposals": [ + {**item, "executed": False, "requiresExternalAuthorization": True} + for item in sequence(data, "runnerProposals") + if isinstance(item, dict) + ], + "contributionReceiptProposal": { + "observationImported": True, + **output_receipt, + }, + } + } + + +def _packet_evidence(data: dict[str, Any], decision: str) -> dict[str, Any]: + artifacts = sequence(data, "artifactRecords") + validation = sequence(data, "validationMatrix") + repo_identity = mapping(data, "repoIdentity") + return { + "unverifiedEvidenceClaims": { + "verified": False, + "basis": "imported observation envelope", + "repoHeadSha256": repo_identity.get("headSha256"), + "artifactPaths": [ + item.get("path") for item in artifacts if isinstance(item, dict) + ], + "artifactSha256": [ + item.get("sha256") for item in artifacts if isinstance(item, dict) + ], + "gateIds": [ + item.get("gateId") for item in validation if isinstance(item, dict) + ], + }, + "epistemicLedger": [ + { + "event": "repository_and_change_observation_imported", + "status": "observed", + }, + { + "event": "canonical_surface_parity_catalog_and_tdd_evaluated", + "status": "computed_from_claims", + }, + { + "event": "security_budget_and_commit_contract_evaluated", + "status": "computed_from_claims", + }, + {"event": "contribution_review_packet_compiled", "status": decision}, + ], + } + + +def _packet_planes( + data: dict[str, Any], + decision: str, + faults: list[dict[str, Any]], + contradictions: list[dict[str, Any]], +) -> dict[str, Any]: + artifacts = sequence(data, "artifactRecords") + validation = sequence(data, "validationMatrix") + return { + "pccaPlanes": { + "observation-and-data": {"artifactCount": len(artifacts)}, + "epistemic-and-lineage": { + "sha256ShapedArtifactClaims": sum( + + is_sha256(item.get("sha256")) + for item in artifacts + if isinstance(item, dict) + + ) + }, + "state-and-relationship": { + "harnessTargetCount": len( + mapping(data, "harnessParity").get("targets", []) + ) + if isinstance(mapping(data, "harnessParity").get("targets", []), list) + else 0 + }, + "cognitive-control": {"validationGateCount": len(validation)}, + "meta-control": {"decision": decision}, + "viability-and-policy": { + "faultCount": len(faults), + "contradictionCount": len(contradictions), + }, + } + } + + +def _packet_memory(data: dict[str, Any]) -> dict[str, Any]: + repo_identity = mapping(data, "repoIdentity") + checkpoint = mapping(data, "checkpointContract") + return { + "memoryMetabolism": { + "checkpointSha256": checkpoint.get("checkpointSha256"), + "repoHeadSha256": repo_identity.get("headSha256"), + "revalidateOn": [ + "HEAD drift", + "policy or manifest change", + "catalog regeneration", + "harness adapter change", + "test or coverage drift", + "security finding", + "review feedback", + ], + "supersessionRequired": True, + } + } + + +def _output_receipt(capsule_file_written: bool) -> dict[str, Any]: + return { + "sourceFilesWritten": False, + "capsuleFileWritten": capsule_file_written, + "testsExecuted": False, + "catalogUpdated": False, + "commitCreated": False, + "pushPerformed": False, + "pullRequestOpened": False, + "controlStoreWritten": False, + "canonicalMemoryWritten": False, + "semanticCommitted": False, + } + + +def compile_contract( + observation: Any, source_path: Path, capsule_file_written: bool = False +) -> dict[str, Any]: + faults, contradictions = validate(observation) + data = observation if isinstance(observation, dict) else {} + decision = "reject" if contradictions else "require_review" + output_receipt = _output_receipt(capsule_file_written) + artifact = { + **_packet_identity(data), + **_packet_review(data, faults, contradictions), + **_packet_payload(data, decision, output_receipt), + **_packet_evidence(data, decision), + **_packet_planes(data, decision, faults, contradictions), + **_packet_memory(data), + "returnReceipt": output_receipt, + } + semantic_checksum = checksum(artifact) + return { + **artifact, + "semanticChecksum": semantic_checksum, + "returnReceipt": {**output_receipt, "semanticChecksum": semantic_checksum}, + } + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("observation", type=Path, help="JSON observation envelope") + parser.add_argument( + "--output", type=Path, help="write the generated capsule to this path" + ) + args = parser.parse_args() + if args.output is not None and args.observation.resolve() == args.output.resolve(): + print("error: output path must not overwrite the observation envelope", file=sys.stderr) + return 2 + try: + observation = json.loads( + args.observation.read_text(encoding="utf-8"), + parse_constant=reject_nonstandard_json_number, + ) + except (OSError, UnicodeError, ValueError, RecursionError) as error: + print(f"error: cannot read observation JSON: {error}", file=sys.stderr) + return 2 + capsule_written = args.output is not None + try: + packet = compile_contract( + observation, args.observation, capsule_file_written=capsule_written + ) + rendered = ( + json.dumps(packet, indent=2, sort_keys=True, ensure_ascii=False) + "\n" + ) + rendered.encode("utf-8") + except (UnicodeError, ValueError, RecursionError) as error: + print(f"error: cannot compile observation JSON: {error}", file=sys.stderr) + return 2 + if args.output is None: + print(rendered, end="") + else: + try: + with args.output.open("x", encoding="utf-8") as output_file: + output_file.write(rendered) + except FileExistsError: + print( + "error: output path already exists; refusing to overwrite", + file=sys.stderr, + ) + return 2 + except OSError as error: + print(f"error: cannot write capsule output: {error}", file=sys.stderr) + return 2 + return ( + 0 if not packet["semanticPageFaults"] and (not packet["contradictions"]) else 2 + ) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/skills/ecc-contribution-review/scripts/contribution_review_validation.py b/skills/ecc-contribution-review/scripts/contribution_review_validation.py new file mode 100644 index 000000000..1c59daf19 --- /dev/null +++ b/skills/ecc-contribution-review/scripts/contribution_review_validation.py @@ -0,0 +1,642 @@ +"""Section validators for the skill contribution observation contract.""" + +from __future__ import annotations + +import math +import re +from datetime import datetime +from pathlib import PurePosixPath +from typing import Any + +ROLES = { + "canonical-skill", "install-manifest", + "agents-mirror", "cursor-mirror", + "codex-metadata", "unit-test", +} +HARNESSES = {"claude", "codex", "cursor"} +GATES = { + "strict-skill-frontmatter", "install-manifests", + "catalog-sync", "command-registry", + "harness-adapters", "unicode-safety", + "no-personal-paths", "test-and-coverage", +} +FAILURES = { + "generated-root-as-canonical", "missing-manifest-entry", + "harness-content-drift", "catalog-count-drift", + "stale-command-registry", "missing-red-test", + "coverage-regression", "secret-in-diff", + "personal-path-in-diff", "unsafe-hook-permission", + "breaking-change-unmarked", "commit-before-gates", +} +FAILURE_OUTCOMES: dict[Any, str] = { + "generated-root-as-canonical": "reject", "missing-manifest-entry": "hold", + "harness-content-drift": "hold", "catalog-count-drift": "hold", + "stale-command-registry": "hold", "missing-red-test": "hold", + "coverage-regression": "hold", "secret-in-diff": "reject", + "personal-path-in-diff": "reject", "unsafe-hook-permission": "reject", + "breaking-change-unmarked": "hold", "commit-before-gates": "reject", +} +RUNNERS = { + "resolve-canonical-surface", "compile-change-plan", + "run-red-green-refactor", "reconcile-harness-parity", + "reconcile-catalog-and-manifests", "run-security-and-validation-gates", + "prepare-commit-proposal", +} +AUTHORITY_FIELDS = { + "mayWriteSourceFiles", "mayExecuteTests", + "mayUpdateCatalog", + "mayCommit", + "mayPush", + "mayOpenPullRequest", + "mayWriteControlStore", + "mayWriteCanonicalMemory", + "maySemanticCommit", +} +EFFECT_FIELDS = { + "sourceFileWriteRequested", + "testExecutionRequested", + "catalogWriteRequested", + "commitRequested", + "pushRequested", + "pullRequestRequested", + "controlStoreWriteRequested", + "canonicalMemoryWriteRequested", + "semanticCommitRequested", +} + + +def is_sha256(value: Any) -> bool: + return isinstance(value, str) and re.fullmatch(r"[0-9a-f]{64}", value) is not None + + +def is_finite_number(value: Any) -> bool: + """Accept finite JSON numbers while excluding booleans.""" + if isinstance(value, bool): + return False + if isinstance(value, int): + return True + return isinstance(value, float) and math.isfinite(value) + + +def safe_repo_path(value: Any) -> bool: + """Accept normalized relative POSIX paths, rejecting home/drive/traversal refs.""" + if not isinstance(value, str) or not value or "\\" in value or "\x00" in value: + return False + if value.startswith("~") or re.match(r"^[A-Za-z]:", value): + return False + if any(part in {"", ".", ".."} for part in value.split("/")): + return False + path = PurePosixPath(value) + return not path.is_absolute() and path.as_posix() == value + + +def add_issue(issues: list[dict[str, Any]], code: str, plane: int, detail: str) -> list[dict[str, Any]]: + if code in {issue["code"] for issue in issues}: + return list(issues) + return [*issues, {"code": code, "plane": plane, "detail": detail}] + + + +def _validate_repo(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + repo = observation["repoIdentity"] + repo_fields = { + "rootName", "rootPathRef", "branch", "baseSha256", "headSha256", + "packageLockSha256", "installModulesSha256", "installProfilesSha256", + "installComponentsSha256", "catalogScriptSha256", "skillPolicySha256", + "checkpointSha256", + } + if ( + set(repo) != repo_fields + or repo.get("rootName") != "everything-claude-code" + or repo.get("rootPathRef") != "active-checkout:everything-claude-code" + or not isinstance(repo.get("branch"), str) + or not repo.get("branch") + or repo.get("branch") in {"main", "master"} + or any(not is_sha256(repo.get(key)) for key in repo_fields if key.endswith("Sha256")) + or repo.get("baseSha256") == repo.get("headSha256") + ): + faults = add_issue(faults, "repo_identity_failed", 2, + "Active ECC root, non-default branch, distinct base/head, manifests, catalog, policy, and checkpoint hashes are required.") + return faults + + +def _validate_intent(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + intent = observation["changeIntent"] + intent_fields = { + "changeId", "changeKind", "canonicalSurface", "canonicalPath", + "legacyCommandShimRequired", "breakingChange", "owner", "scopes", + "evidenceRefs", "counterevidenceRefs", + } + scopes = intent.get("scopes") + if ( + set(intent) != intent_fields + or not isinstance(intent.get("changeId"), str) + or not intent.get("changeId") + or intent.get("changeKind") != "skill" + or intent.get("canonicalSurface") != "skills" + or not safe_repo_path(intent.get("canonicalPath")) + or not intent.get("canonicalPath", "").startswith("skills/") + or intent.get("legacyCommandShimRequired") is not False + or intent.get("breakingChange") is not False + or intent.get("owner") != "ECC" + or not isinstance(scopes, list) + or set(scopes) != {"skills", "tests", "manifests", "cross-harness"} + or not isinstance(intent.get("evidenceRefs"), list) + or not intent.get("evidenceRefs") + or not isinstance(intent.get("counterevidenceRefs"), list) + ): + faults = add_issue(faults, "change_intent_failed", 1, + "A bounded non-breaking skills-first change with explicit scopes and evidence refs is required.") + return faults + + +def _validate_surface(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + surface = observation["surfaceContract"] + surface_fields = { + "canonicalFirst", "generatedRootsExcluded", "curatedSkill", + "installManifestUpdated", "commandShimAdded", "provenanceRequired", + "mirrorTargets", "docsUpdateRequired", "evidenceRefs", "counterevidenceRefs", + } + mirror_targets = surface.get("mirrorTargets") + if ( + set(surface) != surface_fields + or surface.get("canonicalFirst") is not True + or surface.get("generatedRootsExcluded") is not True + or surface.get("curatedSkill") is not True + or surface.get("installManifestUpdated") is not True + or surface.get("commandShimAdded") is not False + or surface.get("provenanceRequired") is not False + or not isinstance(mirror_targets, list) + or set(mirror_targets) != {"agents", "cursor"} + or surface.get("docsUpdateRequired") is not True + or not isinstance(surface.get("evidenceRefs"), list) + or not surface.get("evidenceRefs") + ): + faults = add_issue(faults, "canonical_surface_failed", 6, + "Curated skills/ must lead; generated roots must be excluded; manifest, docs, and intentional mirrors are required.") + return faults + + +def _validate_artifacts(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + intent = observation["changeIntent"] + artifacts = observation["artifactRecords"] + artifact_fields = { + "path", "role", "sha256", "exists", "sourceOfTruth", "generated", + "contentIdentitySha256", "evidenceRefs", + } + artifact_paths: list[Any] = [] + artifact_bad = False + for artifact in artifacts: + if not isinstance(artifact, dict): + artifact_bad = True + continue + artifact_paths = [*artifact_paths, artifact.get("path")] + source_of_truth = artifact.get("role") == "canonical-skill" + if ( + set(artifact) != artifact_fields + or not safe_repo_path(artifact.get("path")) + or artifact.get("role") not in ROLES + or not is_sha256(artifact.get("sha256")) + or artifact.get("exists") is not True + or artifact.get("sourceOfTruth") is not source_of_truth + or artifact.get("generated") is not False + or not is_sha256(artifact.get("contentIdentitySha256")) + or not isinstance(artifact.get("evidenceRefs"), list) + or not artifact.get("evidenceRefs") + ): + artifact_bad = True + canonical_artifacts = [ + item for item in artifacts + if isinstance(item, dict) and item.get("role") == "canonical-skill" + ] + if ( + len(artifacts) != 6 + or {item.get("role") for item in artifacts if isinstance(item, dict)} != ROLES + or len(artifact_paths) != len(set(artifact_paths)) + or len(canonical_artifacts) != 1 + or canonical_artifacts[0].get("path") != intent.get("canonicalPath") + or artifact_bad + ): + faults = add_issue(faults, "artifact_graph_failed", 2, + "Six unique, extant, hash-bound artifacts with one exact canonical source are required.") + return faults + + +def _parity_targets_valid(targets: list[Any], canonical_content: Any) -> bool: + target_fields = { + "harness", "path", "contentIdentitySha256", "frontmatterAdapted", + "metadataPresent", "evidenceRefs", + } + return not any( + not isinstance(item, dict) + or set(item) != target_fields + or not safe_repo_path(item.get("path")) + or item.get("contentIdentitySha256") != canonical_content + or not isinstance(item.get("frontmatterAdapted"), bool) + or item.get("metadataPresent") is not True + or not isinstance(item.get("evidenceRefs"), list) + or not item.get("evidenceRefs") + for item in targets + ) + +def _validate_parity(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + artifacts = observation["artifactRecords"] + parity = observation["harnessParity"] + parity_fields = { + "canonicalContentSha256", "targets", "parityPassed", "driftIds", + "evidenceRefs", "counterevidenceRefs", + } + targets = parity.get("targets", []) + canonical_content = parity.get("canonicalContentSha256") + artifact_by_path = { + item.get("path"): item + for item in artifacts + if isinstance(item, dict) + } + expected_roles: dict[Any, str] = { + "claude": "canonical-skill", + "codex": "agents-mirror", + "cursor": "cursor-mirror", + } + parity_artifacts_bad = any( + not isinstance(item, dict) + or item.get("path") not in artifact_by_path + or artifact_by_path[item.get("path")].get("role") != expected_roles.get(item.get("harness")) + or artifact_by_path[item.get("path")].get("contentIdentitySha256") + != item.get("contentIdentitySha256") + for item in targets + ) if isinstance(targets, list) else True + if ( + set(parity) != parity_fields + or not is_sha256(canonical_content) + or not isinstance(targets, list) + or len(targets) != 3 + or {item.get("harness") for item in targets if isinstance(item, dict)} != HARNESSES + or not _parity_targets_valid(targets, canonical_content) + or parity.get("parityPassed") is not True + or parity.get("driftIds") != [] + or not isinstance(parity.get("evidenceRefs"), list) + or not parity.get("evidenceRefs") + or parity_artifacts_bad + ): + faults = add_issue(faults, "harness_parity_failed", 3, + "Claude, Codex, and Cursor surfaces must retain normalized content identity with explicit adaptations and no drift.") + return faults + + +def _validate_catalog(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + catalog = observation["catalogContract"] + catalog_fields = { + "filesystemAgents", "documentedAgents", "filesystemSkills", "documentedSkills", + "filesystemCommands", "documentedCommands", "installModuleCount", + "curatedSkillsReferenced", "intentionallyUnshippedSkills", "catalogCheckPassed", + "commandRegistryCheckPassed", "manifestValidationPassed", "countsReconciled", + "evidenceRefs", "counterevidenceRefs", + } + count_pairs = ( + ("filesystemAgents", "documentedAgents"), + ("filesystemSkills", "documentedSkills"), + ("filesystemCommands", "documentedCommands"), + ) + counts_valid = all( + isinstance(catalog.get(left), int) + and not isinstance(catalog.get(left), bool) + and catalog.get(left) > 0 + and catalog.get(left) == catalog.get(right) + for left, right in count_pairs + ) + skills_valid = all( + isinstance(catalog.get(key), int) and not isinstance(catalog.get(key), bool) + and catalog.get(key) >= 0 + for key in ("curatedSkillsReferenced", "intentionallyUnshippedSkills", "filesystemSkills") + ) + if ( + set(catalog) != catalog_fields + or not counts_valid + or not isinstance(catalog.get("installModuleCount"), int) + or isinstance(catalog.get("installModuleCount"), bool) + or catalog.get("installModuleCount", 0) <= 0 + or not skills_valid + or catalog.get("curatedSkillsReferenced", 0) + catalog.get("intentionallyUnshippedSkills", 0) != catalog.get("filesystemSkills") + or any(catalog.get(key) is not True for key in ( + "catalogCheckPassed", "commandRegistryCheckPassed", "manifestValidationPassed", "countsReconciled" + )) + or not isinstance(catalog.get("evidenceRefs"), list) + or not catalog.get("evidenceRefs") + ): + faults = add_issue(faults, "catalog_reconciliation_failed", 3, + "Filesystem/documented counts, manifest coverage, intentional exceptions, catalog, and registry checks must reconcile.") + return faults + + +def _validate_gates(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + validation = observation["validationMatrix"] + validation_fields = {"gateId", "commandArgv", "expectedExit", "observedExit", "evidenceRefs"} + if ( + len(validation) != 8 + or {item.get("gateId") for item in validation if isinstance(item, dict)} != GATES + or any( + not isinstance(item, dict) + or set(item) != validation_fields + or not isinstance(item.get("commandArgv"), list) + or not item.get("commandArgv") + or not all(isinstance(arg, str) and arg for arg in item.get("commandArgv", [])) + or item.get("expectedExit") != 0 + or item.get("observedExit") != 0 + or not isinstance(item.get("evidenceRefs"), list) + or not item.get("evidenceRefs") + for item in validation + ) + ): + faults = add_issue(faults, "validation_matrix_failed", 5, + "Eight exact argv gates must be recorded green with evidence references.") + return faults + + +def _validate_tdd(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + tdd = observation["tddContract"] + tdd_fields = { + "redObserved", "failingTestIds", "greenObserved", "refactorObserved", + "coverageThresholdPercent", "lineCoveragePercent", "functionCoveragePercent", + "branchCoveragePercent", "statementCoveragePercent", "newBehaviorTestCount", + "allTestsPass", "evidenceRefs", "counterevidenceRefs", + } + coverage_fields = ( + "lineCoveragePercent", "functionCoveragePercent", "branchCoveragePercent", + "statementCoveragePercent", + ) + if ( + set(tdd) != tdd_fields + or any(tdd.get(key) is not True for key in ("redObserved", "greenObserved", "refactorObserved", "allTestsPass")) + or not isinstance(tdd.get("failingTestIds"), list) + or not tdd.get("failingTestIds") + or tdd.get("coverageThresholdPercent") != 80 + or any(not is_finite_number(tdd.get(key)) or not 80 <= tdd.get(key) <= 100 for key in coverage_fields) + or not isinstance(tdd.get("newBehaviorTestCount"), int) + or isinstance(tdd.get("newBehaviorTestCount"), bool) + or tdd.get("newBehaviorTestCount", 0) < 3 + or not isinstance(tdd.get("evidenceRefs"), list) + or not tdd.get("evidenceRefs") + ): + faults = add_issue(faults, "tdd_contract_failed", 4, + "Observed RED/GREEN/REFACTOR, named failures, passing behavior tests, and at least 80 percent coverage are required.") + return faults + + +def _validate_security(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + security = observation["securityContract"] + security_fields = { + "secretsDetected", "personalPathsDetected", "unsafePermissionsDetected", + "hookChangesReviewed", "promptInjectionBoundaryReviewed", "supplyChainChangesReviewed", + "untrustedContentCannotGrantAuthority", "evidenceRefs", "counterevidenceRefs", + } + if ( + set(security) != security_fields + or any(security.get(key) is not False for key in ( + "secretsDetected", "personalPathsDetected", "unsafePermissionsDetected" + )) + or any(security.get(key) is not True for key in ( + "hookChangesReviewed", "promptInjectionBoundaryReviewed", + "supplyChainChangesReviewed", "untrustedContentCannotGrantAuthority" + )) + or not isinstance(security.get("evidenceRefs"), list) + or not security.get("evidenceRefs") + ): + faults = add_issue(faults, "security_contract_failed", 6, + "Secret, path, permission, hook, prompt-injection, supply-chain, and authority observations are required.") + return faults + + +def _validate_budget(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + budget = observation["budgetContract"] + budget_fields = { + "maximumChangedFiles", "observedChangedFiles", "maximumChangedLines", + "observedChangedLines", "maximumTestRuntimeSeconds", "observedTestRuntimeSeconds", + "evidenceRefs", + } + budget_pairs = ( + ("observedChangedFiles", "maximumChangedFiles"), + ("observedChangedLines", "maximumChangedLines"), + ("observedTestRuntimeSeconds", "maximumTestRuntimeSeconds"), + ) + if ( + set(budget) != budget_fields + or any( + not is_finite_number(budget.get(actual)) + or not is_finite_number(budget.get(limit)) + or budget.get(actual) < 0 + or budget.get(actual) > budget.get(limit) + for actual, limit in budget_pairs + ) + or not isinstance(budget.get("evidenceRefs"), list) + or not budget.get("evidenceRefs") + ): + faults = add_issue(faults, "change_budget_failed", 6, + "Changed-file, changed-line, and test-runtime budgets must be nonnegative and bounded.") + return faults + + +def _validate_commit(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + commit = observation["commitContract"] + commit_fields = { + "type", "scope", "subject", "subjectLength", "conventional", "commitCreated", + "pushPerformed", "pullRequestOpened", "evidenceRefs", "counterevidenceRefs", + } + subject = commit.get("subject") + if ( + set(commit) != commit_fields + or commit.get("type") not in {"feat", "fix", "refactor", "docs", "test", "chore", "perf", "ci"} + or commit.get("scope") != "skills" + or not isinstance(subject, str) + or not subject + or commit.get("subjectLength") != len(subject) + or len(subject) > 72 + or commit.get("conventional") is not True + or any(commit.get(key) is not False for key in ("commitCreated", "pushPerformed", "pullRequestOpened")) + or not isinstance(commit.get("evidenceRefs"), list) + or not commit.get("evidenceRefs") + ): + faults = add_issue(faults, "commit_contract_failed", 6, + "A concise conventional commit proposal is required; commit, push, and PR effects must remain false.") + return faults + + +def _validate_probes(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + probes = observation["failureProbes"] + probe_fields = {"probeId", "failureMode", "expectedOutcome", "observedOutcome", "evidenceRefs"} + if ( + len(probes) != 12 + or {item.get("failureMode") for item in probes if isinstance(item, dict) and isinstance(item.get("failureMode"), str)} != FAILURES + or len({item.get("probeId") for item in probes if isinstance(item, dict) and isinstance(item.get("probeId"), str)}) != 12 + or any( + not isinstance(item, dict) + or set(item) != probe_fields + or not isinstance(item.get("failureMode"), str) + or not isinstance(item.get("probeId"), str) + or item.get("expectedOutcome") != FAILURE_OUTCOMES.get(item.get("failureMode")) + or item.get("observedOutcome") != item.get("expectedOutcome") + or not isinstance(item.get("evidenceRefs"), list) + or not item.get("evidenceRefs") + for item in probes + ) + ): + faults = add_issue(faults, "failure_probe_coverage_failed", 5, + "All twelve ECC contribution probes must match expected outcomes and cite evidence.") + return faults + + +def _validate_checkpoint(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + repo = observation["repoIdentity"] + checkpoint = observation["checkpointContract"] + checkpoint_fields = { + "checkpointSha256", "repoHeadSha256", "policySha256", "lastDecisionId", + "plannedRunnerCount", "statePathRef", "remainingUnknowns", + } + if ( + set(checkpoint) != checkpoint_fields + or checkpoint.get("checkpointSha256") != repo.get("checkpointSha256") + or checkpoint.get("repoHeadSha256") != repo.get("headSha256") + or checkpoint.get("policySha256") != repo.get("skillPolicySha256") + or not checkpoint.get("lastDecisionId") + or checkpoint.get("plannedRunnerCount") != 7 + or not isinstance(checkpoint.get("statePathRef"), str) + or not checkpoint.get("statePathRef", "").startswith(".memory-kit/") + or not isinstance(checkpoint.get("remainingUnknowns"), list) + or not checkpoint.get("remainingUnknowns") + ): + faults = add_issue(faults, "checkpoint_contract_failed", 5, + "Exact repository, policy, checkpoint, runner, state, and unknown identities are required.") + return faults + + +def _validate_runners(observation: dict[str, Any], faults: list[dict[str, Any]]) -> list[dict[str, Any]]: + runners = observation["runnerProposals"] + runner_fields = {"role", "argv"} + if ( + len(runners) != 7 + or {item.get("role") for item in runners if isinstance(item, dict)} != RUNNERS + or any( + not isinstance(item, dict) + or set(item) != runner_fields + or not isinstance(item.get("argv"), list) + or not item.get("argv") + or not all(isinstance(arg, str) and arg for arg in item.get("argv", [])) + for item in runners + ) + ): + faults = add_issue(faults, "runner_proposals_invalid", 5, + "Seven typed non-shell argv proposals are required.") + return faults + + +def _validate_authority(observation: dict[str, Any], contradictions: list[dict[str, Any]]) -> list[dict[str, Any]]: + forbidden_claims = ( + ("claimsSourceFilesWrittenByController", "file_write_claim_forbidden"), + ("claimsTestsExecutedByController", "test_execution_claim_forbidden"), + ("claimsCatalogUpdatedByController", "catalog_write_claim_forbidden"), + ("claimsCommitCreated", "commit_claim_forbidden"), + ("claimsPushPerformed", "push_claim_forbidden"), + ("claimsPullRequestOpened", "pull_request_claim_forbidden"), + ) + for field, code in forbidden_claims: + if field in observation and observation[field] is not False: + contradictions = add_issue( + contradictions, code, 2, + "The compiler cannot claim repository or external effects.", + ) + requested_effects = observation.get("requestedEffects") + if "requestedEffects" in observation and ( + not isinstance(requested_effects, dict) + or any(value is not False for value in requested_effects.values()) + ): + contradictions = add_issue( + contradictions, "external_action_requested", 6, + "Compilation cannot request source, test, catalog, Git, PR, or memory effects.", + ) + claimed_authority = observation.get("claimedAuthority") + if "claimedAuthority" in observation and ( + not isinstance(claimed_authority, dict) + or any(value is not False for value in claimed_authority.values()) + ): + contradictions = add_issue( + contradictions, "unsafe_generated_authority", 6, + "Generated authority must remain false.", + ) + return contradictions + + +def _valid_timestamp(value: Any) -> bool: + if not isinstance(value, str) or not re.fullmatch( + r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-](?:[01]\d|2[0-3]):[0-5]\d)", value + ): + return False + try: + stamp = datetime.fromisoformat(value.replace("Z", "+00:00")) + except ValueError: + return False + return stamp.tzinfo is not None and stamp.utcoffset() is not None + + +def _valid_envelope(observation: Any, faults: list[dict[str, Any]]) -> tuple[bool, list[dict[str, Any]]]: + required = { + "schemaVersion", "observedAt", "subjectRef", "repoIdentity", "changeIntent", + "surfaceContract", "artifactRecords", "harnessParity", "catalogContract", + "validationMatrix", "tddContract", "securityContract", "budgetContract", + "commitContract", "failureProbes", "checkpointContract", "runnerProposals", + "claimsSourceFilesWrittenByController", "claimsTestsExecutedByController", + "claimsCatalogUpdatedByController", "claimsCommitCreated", "claimsPushPerformed", + "claimsPullRequestOpened", "requestedEffects", "claimedAuthority", + } + if not isinstance(observation, dict) or not required.issubset(observation): + faults = add_issue(faults, "observation_envelope_incomplete", 1, + "Required ECC contribution observations are missing.") + return False, faults + + if observation.get("schemaVersion") != "1.0.0": + faults = add_issue(faults, "schema_version_unsupported", 1, "Only schema version 1.0.0 is supported.") + if not _valid_timestamp(observation.get("observedAt")): + faults = add_issue(faults, "observed_at_invalid", 1, "An ISO 8601 observation timestamp with a timezone is required.") + subject = observation.get("subjectRef") + if not isinstance(subject, str) or not subject.strip(): + faults = add_issue(faults, "subject_ref_invalid", 1, "A nonempty subject reference string is required.") + + map_fields = ( + "repoIdentity", "changeIntent", "surfaceContract", "harnessParity", + "catalogContract", "tddContract", "securityContract", "budgetContract", + "commitContract", "checkpointContract", "requestedEffects", "claimedAuthority", + ) + list_fields = ("artifactRecords", "validationMatrix", "failureProbes", "runnerProposals") + if any(not isinstance(observation[field], dict) for field in map_fields) or any( + not isinstance(observation[field], list) for field in list_fields + ): + faults = add_issue(faults, "observation_shape_invalid", 1, + "Declared maps and arrays must retain their documented shapes.") + return False, faults + for field, expected in (("requestedEffects", EFFECT_FIELDS), ("claimedAuthority", AUTHORITY_FIELDS)): + if set(observation[field]) != expected: + faults = add_issue(faults, "observation_shape_invalid", 1, + "Authority and effect maps must contain all documented fields.") + return True, faults + + +def validate(observation: Any) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]: + """Retain independent diagnostics even when one untrusted section is malformed.""" + valid_envelope, faults = _valid_envelope(observation, []) + contradictions = _validate_authority(observation, []) if isinstance(observation, dict) else [] + if not valid_envelope: + return faults, contradictions + validators = ( + _validate_repo, _validate_intent, _validate_surface, _validate_artifacts, + _validate_parity, _validate_catalog, _validate_gates, _validate_tdd, + _validate_security, _validate_budget, _validate_commit, _validate_probes, + _validate_checkpoint, _validate_runners, + ) + for validator in validators: + try: + faults = validator(observation, faults) + except (AttributeError, TypeError, ValueError): + faults = add_issue( + faults, "observation_shape_invalid", 1, + "Observation values must match the documented JSON types.", + ) + return faults, contradictions diff --git a/skills/ecc-contribution-review/templates/contribution-review-observation.json b/skills/ecc-contribution-review/templates/contribution-review-observation.json new file mode 100644 index 000000000..476380216 --- /dev/null +++ b/skills/ecc-contribution-review/templates/contribution-review-observation.json @@ -0,0 +1,156 @@ +{ + "schemaVersion": "1.0.0", + "observedAt": "REPLACE_WITH_CURRENT_OBSERVATION", + "subjectRef": "REPLACE_WITH_CURRENT_OBSERVATION", + "repoIdentity": { + "rootName": "REPLACE_WITH_CURRENT_OBSERVATION", + "rootPathRef": "REPLACE_WITH_CURRENT_OBSERVATION", + "branch": "REPLACE_WITH_CURRENT_OBSERVATION", + "baseSha256": "REPLACE_WITH_CURRENT_OBSERVATION", + "headSha256": "REPLACE_WITH_CURRENT_OBSERVATION", + "packageLockSha256": "REPLACE_WITH_CURRENT_OBSERVATION", + "installModulesSha256": "REPLACE_WITH_CURRENT_OBSERVATION", + "installProfilesSha256": "REPLACE_WITH_CURRENT_OBSERVATION", + "installComponentsSha256": "REPLACE_WITH_CURRENT_OBSERVATION", + "catalogScriptSha256": "REPLACE_WITH_CURRENT_OBSERVATION", + "skillPolicySha256": "REPLACE_WITH_CURRENT_OBSERVATION", + "checkpointSha256": "REPLACE_WITH_CURRENT_OBSERVATION" + }, + "changeIntent": { + "changeId": "REPLACE_WITH_CURRENT_OBSERVATION", + "changeKind": "skill", + "canonicalSurface": "REPLACE_WITH_CURRENT_OBSERVATION", + "canonicalPath": "REPLACE_WITH_CURRENT_OBSERVATION", + "legacyCommandShimRequired": false, + "breakingChange": false, + "owner": "REPLACE_WITH_CURRENT_OBSERVATION", + "scopes": [], + "evidenceRefs": [], + "counterevidenceRefs": [] + }, + "surfaceContract": { + "canonicalFirst": false, + "generatedRootsExcluded": false, + "curatedSkill": false, + "installManifestUpdated": false, + "commandShimAdded": false, + "provenanceRequired": false, + "mirrorTargets": [], + "docsUpdateRequired": false, + "evidenceRefs": [], + "counterevidenceRefs": [] + }, + "artifactRecords": [], + "harnessParity": { + "canonicalContentSha256": "REPLACE_WITH_CURRENT_OBSERVATION", + "targets": [], + "parityPassed": false, + "driftIds": [], + "evidenceRefs": [], + "counterevidenceRefs": [] + }, + "catalogContract": { + "filesystemAgents": 0, + "documentedAgents": 0, + "filesystemSkills": 0, + "documentedSkills": 0, + "filesystemCommands": 0, + "documentedCommands": 0, + "installModuleCount": 0, + "curatedSkillsReferenced": 0, + "intentionallyUnshippedSkills": 0, + "catalogCheckPassed": false, + "commandRegistryCheckPassed": false, + "manifestValidationPassed": false, + "countsReconciled": false, + "evidenceRefs": [], + "counterevidenceRefs": [] + }, + "validationMatrix": [], + "tddContract": { + "redObserved": false, + "failingTestIds": [], + "greenObserved": false, + "refactorObserved": false, + "coverageThresholdPercent": 0, + "lineCoveragePercent": 0, + "functionCoveragePercent": 0, + "branchCoveragePercent": 0, + "statementCoveragePercent": 0, + "newBehaviorTestCount": 0, + "allTestsPass": false, + "evidenceRefs": [], + "counterevidenceRefs": [] + }, + "securityContract": { + "secretsDetected": false, + "personalPathsDetected": false, + "unsafePermissionsDetected": false, + "hookChangesReviewed": false, + "promptInjectionBoundaryReviewed": false, + "supplyChainChangesReviewed": false, + "untrustedContentCannotGrantAuthority": false, + "evidenceRefs": [], + "counterevidenceRefs": [] + }, + "budgetContract": { + "maximumChangedFiles": 0, + "observedChangedFiles": 0, + "maximumChangedLines": 0, + "observedChangedLines": 0, + "maximumTestRuntimeSeconds": 0, + "observedTestRuntimeSeconds": 0, + "evidenceRefs": [] + }, + "commitContract": { + "type": "REPLACE_WITH_CURRENT_OBSERVATION", + "scope": "REPLACE_WITH_CURRENT_OBSERVATION", + "subject": "REPLACE_WITH_CURRENT_OBSERVATION", + "subjectLength": 0, + "conventional": false, + "commitCreated": false, + "pushPerformed": false, + "pullRequestOpened": false, + "evidenceRefs": [], + "counterevidenceRefs": [] + }, + "failureProbes": [], + "checkpointContract": { + "checkpointSha256": "REPLACE_WITH_CURRENT_OBSERVATION", + "repoHeadSha256": "REPLACE_WITH_CURRENT_OBSERVATION", + "policySha256": "REPLACE_WITH_CURRENT_OBSERVATION", + "lastDecisionId": "REPLACE_WITH_CURRENT_OBSERVATION", + "plannedRunnerCount": 0, + "statePathRef": "REPLACE_WITH_CURRENT_OBSERVATION", + "remainingUnknowns": [] + }, + "runnerProposals": [], + "claimsSourceFilesWrittenByController": false, + "claimsTestsExecutedByController": false, + "claimsCatalogUpdatedByController": false, + "claimsCommitCreated": false, + "claimsPushPerformed": false, + "claimsPullRequestOpened": false, + "requestedEffects": { + "sourceFileWriteRequested": false, + "testExecutionRequested": false, + "catalogWriteRequested": false, + "commitRequested": false, + "pushRequested": false, + "pullRequestRequested": false, + "controlStoreWriteRequested": false, + "canonicalMemoryWriteRequested": false, + "semanticCommitRequested": false + }, + "claimedAuthority": { + "mayWriteSourceFiles": false, + "mayExecuteTests": false, + "mayUpdateCatalog": false, + "mayCommit": false, + "mayPush": false, + "mayOpenPullRequest": false, + "mayWriteControlStore": false, + "mayWriteCanonicalMemory": false, + "maySemanticCommit": false + } +} diff --git a/skills/ecc-contribution-review/tests/fixtures/synthetic-contribution-review-observation.json b/skills/ecc-contribution-review/tests/fixtures/synthetic-contribution-review-observation.json new file mode 100644 index 000000000..758e15749 --- /dev/null +++ b/skills/ecc-contribution-review/tests/fixtures/synthetic-contribution-review-observation.json @@ -0,0 +1,61 @@ +{ + "schemaVersion":"1.0.0","observedAt":"2026-08-11T14:00:00Z","subjectRef":"ecc-change:context-integrity-skill-001", + "repoIdentity":{"rootName":"everything-claude-code","rootPathRef":"active-checkout:everything-claude-code","branch":"feature/context-integrity","baseSha256":"1111111111111111111111111111111111111111111111111111111111111111","headSha256":"2222222222222222222222222222222222222222222222222222222222222222","packageLockSha256":"3333333333333333333333333333333333333333333333333333333333333333","installModulesSha256":"4444444444444444444444444444444444444444444444444444444444444444","installProfilesSha256":"5555555555555555555555555555555555555555555555555555555555555555","installComponentsSha256":"6666666666666666666666666666666666666666666666666666666666666666","catalogScriptSha256":"7777777777777777777777777777777777777777777777777777777777777777","skillPolicySha256":"8888888888888888888888888888888888888888888888888888888888888888","checkpointSha256":"9999999999999999999999999999999999999999999999999999999999999999"}, + "changeIntent":{"changeId":"context-integrity-skill-001","changeKind":"skill","canonicalSurface":"skills","canonicalPath":"skills/context-integrity/SKILL.md","legacyCommandShimRequired":false,"breakingChange":false,"owner":"ECC","scopes":["skills","tests","manifests","cross-harness"],"evidenceRefs":["issue:context-integrity"],"counterevidenceRefs":["scope:no-command-shim"]}, + "surfaceContract":{"canonicalFirst":true,"generatedRootsExcluded":true,"curatedSkill":true,"installManifestUpdated":true,"commandShimAdded":false,"provenanceRequired":false,"mirrorTargets":["agents","cursor"],"docsUpdateRequired":true,"evidenceRefs":["policy:skill-placement"],"counterevidenceRefs":[]}, + "artifactRecords":[ + {"path":"skills/context-integrity/SKILL.md","role":"canonical-skill","sha256":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","exists":true,"sourceOfTruth":true,"generated":false,"contentIdentitySha256":"abababababababababababababababababababababababababababababababab","evidenceRefs":["artifact:canonical"]}, + {"path":"manifests/install-modules.json","role":"install-manifest","sha256":"cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc","exists":true,"sourceOfTruth":false,"generated":false,"contentIdentitySha256":"dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd","evidenceRefs":["artifact:manifest"]}, + {"path":".agents/skills/context-integrity/SKILL.md","role":"agents-mirror","sha256":"eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee","exists":true,"sourceOfTruth":false,"generated":false,"contentIdentitySha256":"abababababababababababababababababababababababababababababababab","evidenceRefs":["artifact:agents"]}, + {"path":".cursor/skills/context-integrity/SKILL.md","role":"cursor-mirror","sha256":"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff","exists":true,"sourceOfTruth":false,"generated":false,"contentIdentitySha256":"abababababababababababababababababababababababababababababababab","evidenceRefs":["artifact:cursor"]}, + {"path":".agents/skills/context-integrity/agents/openai.yaml","role":"codex-metadata","sha256":"1212121212121212121212121212121212121212121212121212121212121212","exists":true,"sourceOfTruth":false,"generated":false,"contentIdentitySha256":"1313131313131313131313131313131313131313131313131313131313131313","evidenceRefs":["artifact:openai-metadata"]}, + {"path":"tests/context-integrity.test.js","role":"unit-test","sha256":"1414141414141414141414141414141414141414141414141414141414141414","exists":true,"sourceOfTruth":false,"generated":false,"contentIdentitySha256":"1515151515151515151515151515151515151515151515151515151515151515","evidenceRefs":["artifact:test"]} + ], + "harnessParity":{"canonicalContentSha256":"abababababababababababababababababababababababababababababababab","targets":[ + {"harness":"claude","path":"skills/context-integrity/SKILL.md","contentIdentitySha256":"abababababababababababababababababababababababababababababababab","frontmatterAdapted":false,"metadataPresent":true,"evidenceRefs":["parity:claude"]}, + {"harness":"codex","path":".agents/skills/context-integrity/SKILL.md","contentIdentitySha256":"abababababababababababababababababababababababababababababababab","frontmatterAdapted":true,"metadataPresent":true,"evidenceRefs":["parity:codex"]}, + {"harness":"cursor","path":".cursor/skills/context-integrity/SKILL.md","contentIdentitySha256":"abababababababababababababababababababababababababababababababab","frontmatterAdapted":true,"metadataPresent":true,"evidenceRefs":["parity:cursor"]} + ],"parityPassed":true,"driftIds":[],"evidenceRefs":["parity:content-normalized"],"counterevidenceRefs":["parity:harness-frontmatter-adapted"]}, + "catalogContract":{"filesystemAgents":67,"documentedAgents":67,"filesystemSkills":281,"documentedSkills":281,"filesystemCommands":94,"documentedCommands":94,"installModuleCount":18,"curatedSkillsReferenced":280,"intentionallyUnshippedSkills":1,"catalogCheckPassed":true,"commandRegistryCheckPassed":true,"manifestValidationPassed":true,"countsReconciled":true,"evidenceRefs":["catalog:check","manifest:validate"],"counterevidenceRefs":["catalog:future-drift"]}, + "validationMatrix":[ + {"gateId":"strict-skill-frontmatter","commandArgv":["node","scripts/ci/validate-skills.js","--strict"],"expectedExit":0,"observedExit":0,"evidenceRefs":["gate:skills"]}, + {"gateId":"install-manifests","commandArgv":["node","scripts/ci/validate-install-manifests.js"],"expectedExit":0,"observedExit":0,"evidenceRefs":["gate:manifests"]}, + {"gateId":"catalog-sync","commandArgv":["npm","run","catalog:check"],"expectedExit":0,"observedExit":0,"evidenceRefs":["gate:catalog"]}, + {"gateId":"command-registry","commandArgv":["npm","run","command-registry:check"],"expectedExit":0,"observedExit":0,"evidenceRefs":["gate:registry"]}, + {"gateId":"harness-adapters","commandArgv":["npm","run","harness:adapters"],"expectedExit":0,"observedExit":0,"evidenceRefs":["gate:harness"]}, + {"gateId":"unicode-safety","commandArgv":["node","scripts/ci/check-unicode-safety.js"],"expectedExit":0,"observedExit":0,"evidenceRefs":["gate:unicode"]}, + {"gateId":"no-personal-paths","commandArgv":["node","scripts/ci/validate-no-personal-paths.js"],"expectedExit":0,"observedExit":0,"evidenceRefs":["gate:paths"]}, + {"gateId":"test-and-coverage","commandArgv":["npm","run","coverage"],"expectedExit":0,"observedExit":0,"evidenceRefs":["gate:coverage"]} + ], + "tddContract":{"redObserved":true,"failingTestIds":["rejects generated roots","detects harness drift","rejects catalog mismatch"],"greenObserved":true,"refactorObserved":true,"coverageThresholdPercent":80,"lineCoveragePercent":87,"functionCoveragePercent":86,"branchCoveragePercent":81,"statementCoveragePercent":87,"newBehaviorTestCount":16,"allTestsPass":true,"evidenceRefs":["tdd:red-green-refactor","coverage:report"],"counterevidenceRefs":["coverage:not-behavior-proof"]}, + "securityContract":{"secretsDetected":false,"personalPathsDetected":false,"unsafePermissionsDetected":false,"hookChangesReviewed":true,"promptInjectionBoundaryReviewed":true,"supplyChainChangesReviewed":true,"untrustedContentCannotGrantAuthority":true,"evidenceRefs":["security:review"],"counterevidenceRefs":["security:static-analysis-limits"]}, + "budgetContract":{"maximumChangedFiles":10,"observedChangedFiles":6,"maximumChangedLines":800,"observedChangedLines":420,"maximumTestRuntimeSeconds":600,"observedTestRuntimeSeconds":120,"evidenceRefs":["budget:change"]}, + "commitContract":{"type":"feat","scope":"skills","subject":"add context integrity contribution controller","subjectLength":45,"conventional":true,"commitCreated":false,"pushPerformed":false,"pullRequestOpened":false,"evidenceRefs":["commit:proposal"],"counterevidenceRefs":["git:no-external-effects"]}, + "failureProbes":[ + {"probeId":"ecc-failure-01","failureMode":"generated-root-as-canonical","expectedOutcome":"reject","observedOutcome":"reject","evidenceRefs":["probe:generated-root"]}, + {"probeId":"ecc-failure-02","failureMode":"missing-manifest-entry","expectedOutcome":"hold","observedOutcome":"hold","evidenceRefs":["probe:manifest"]}, + {"probeId":"ecc-failure-03","failureMode":"harness-content-drift","expectedOutcome":"hold","observedOutcome":"hold","evidenceRefs":["probe:parity"]}, + {"probeId":"ecc-failure-04","failureMode":"catalog-count-drift","expectedOutcome":"hold","observedOutcome":"hold","evidenceRefs":["probe:catalog"]}, + {"probeId":"ecc-failure-05","failureMode":"stale-command-registry","expectedOutcome":"hold","observedOutcome":"hold","evidenceRefs":["probe:registry"]}, + {"probeId":"ecc-failure-06","failureMode":"missing-red-test","expectedOutcome":"hold","observedOutcome":"hold","evidenceRefs":["probe:tdd"]}, + {"probeId":"ecc-failure-07","failureMode":"coverage-regression","expectedOutcome":"hold","observedOutcome":"hold","evidenceRefs":["probe:coverage"]}, + {"probeId":"ecc-failure-08","failureMode":"secret-in-diff","expectedOutcome":"reject","observedOutcome":"reject","evidenceRefs":["probe:secret"]}, + {"probeId":"ecc-failure-09","failureMode":"personal-path-in-diff","expectedOutcome":"reject","observedOutcome":"reject","evidenceRefs":["probe:path"]}, + {"probeId":"ecc-failure-10","failureMode":"unsafe-hook-permission","expectedOutcome":"reject","observedOutcome":"reject","evidenceRefs":["probe:hook"]}, + {"probeId":"ecc-failure-11","failureMode":"breaking-change-unmarked","expectedOutcome":"hold","observedOutcome":"hold","evidenceRefs":["probe:breaking"]}, + {"probeId":"ecc-failure-12","failureMode":"commit-before-gates","expectedOutcome":"reject","observedOutcome":"reject","evidenceRefs":["probe:commit"]} + ], + "checkpointContract":{"checkpointSha256":"9999999999999999999999999999999999999999999999999999999999999999","repoHeadSha256":"2222222222222222222222222222222222222222222222222222222222222222","policySha256":"8888888888888888888888888888888888888888888888888888888888888888","lastDecisionId":"ecc-contribution-2026-08-11-001","plannedRunnerCount":7,"statePathRef":".memory-kit/everything-claude-code/state.json","remainingUnknowns":["upstream changes","cross-platform filesystem semantics","future harness adapters","coverage blind spots","reviewer judgment"]}, + "runnerProposals":[ + {"role":"resolve-canonical-surface","argv":["ecc-proof","resolve","--canonical-first"]}, + {"role":"compile-change-plan","argv":["ecc-proof","plan","--change-id","context-integrity-skill-001"]}, + {"role":"run-red-green-refactor","argv":["ecc-proof","tdd","--record-phases"]}, + {"role":"reconcile-harness-parity","argv":["ecc-proof","parity","--targets","claude,codex,cursor"]}, + {"role":"reconcile-catalog-and-manifests","argv":["ecc-proof","catalog","--manifests","--registry"]}, + {"role":"run-security-and-validation-gates","argv":["ecc-proof","verify","--all"]}, + {"role":"prepare-commit-proposal","argv":["ecc-proof","commit-proposal","--type","feat","--scope","skills"]} + ], + "claimsSourceFilesWrittenByController":false,"claimsTestsExecutedByController":false,"claimsCatalogUpdatedByController":false,"claimsCommitCreated":false,"claimsPushPerformed":false,"claimsPullRequestOpened":false, + "requestedEffects":{"sourceFileWriteRequested":false,"testExecutionRequested":false,"catalogWriteRequested":false,"commitRequested":false,"pushRequested":false,"pullRequestRequested":false,"controlStoreWriteRequested":false,"canonicalMemoryWriteRequested":false,"semanticCommitRequested":false}, + "claimedAuthority":{"mayWriteSourceFiles":false,"mayExecuteTests":false,"mayUpdateCatalog":false,"mayCommit":false,"mayPush":false,"mayOpenPullRequest":false,"mayWriteControlStore":false,"mayWriteCanonicalMemory":false,"maySemanticCommit":false} +} diff --git a/skills/ecc-contribution-review/tests/fixtures/unsafe-contribution-review-observation.json b/skills/ecc-contribution-review/tests/fixtures/unsafe-contribution-review-observation.json new file mode 100644 index 000000000..d076922d9 --- /dev/null +++ b/skills/ecc-contribution-review/tests/fixtures/unsafe-contribution-review-observation.json @@ -0,0 +1 @@ +{"schemaVersion":"1.0.0","observedAt":"2026-08-11T14:00:00Z","subjectRef":"unsafe","repoIdentity":{},"changeIntent":{},"surfaceContract":{},"artifactRecords":[],"harnessParity":{},"catalogContract":{},"validationMatrix":[],"tddContract":{},"securityContract":{},"budgetContract":{},"commitContract":{},"failureProbes":[],"checkpointContract":{},"runnerProposals":[],"claimsSourceFilesWrittenByController":true,"claimsTestsExecutedByController":true,"claimsCatalogUpdatedByController":true,"claimsCommitCreated":true,"claimsPushPerformed":true,"claimsPullRequestOpened":true,"requestedEffects":{"commitRequested":true},"claimedAuthority":{"mayCommit":true}} diff --git a/skills/ecc-contribution-review/tests/record_security_evidence.py b/skills/ecc-contribution-review/tests/record_security_evidence.py new file mode 100644 index 000000000..3c9835883 --- /dev/null +++ b/skills/ecc-contribution-review/tests/record_security_evidence.py @@ -0,0 +1,63 @@ +"""Record a pinned Bandit scan with the exact scanned source hashes.""" + +import argparse +import hashlib +import json +import subprocess +import sys +import tempfile +from importlib.metadata import version +from pathlib import Path + +SCRIPTS = Path("skills/ecc-contribution-review/scripts") +BANDIT_VERSION = "1.9.4" + + +def source_hashes(repo: Path) -> dict[str, str]: + return { + path.relative_to(repo).as_posix(): hashlib.sha256(path.read_bytes()).hexdigest() + for path in sorted((repo / SCRIPTS).rglob("*.py")) + } + + +def validate_report(report: dict, sources: dict[str, str]) -> None: + expected = {"scanner": "bandit", "version": BANDIT_VERSION, "sha256": sources} + if report.get("source_provenance") != expected: + raise ValueError("Bandit source provenance does not match current source content") + if set(report.get("metrics", {})) - {"_totals"} != set(sources): + raise ValueError("Bandit scanned inventory does not match source provenance") + + +def record(repo: Path, output: Path) -> int: + if version("bandit") != BANDIT_VERSION: + raise RuntimeError(f"Bandit {BANDIT_VERSION} is required") + before = source_hashes(repo) + with tempfile.TemporaryDirectory() as directory: + raw = Path(directory) / "bandit.json" + result = subprocess.run( + [sys.executable, "-m", "bandit", "-r", SCRIPTS.as_posix(), "-f", "json", "-o", str(raw)], + cwd=repo, check=False, + ) + if result.returncode not in {0, 1}: + raise RuntimeError(f"Bandit scan failed with exit {result.returncode}") + report = json.loads(raw.read_text()) + if before != source_hashes(repo): + raise RuntimeError("Source changed during the Bandit scan; evidence discarded") + recorded = { + **report, + "source_provenance": {"scanner": "bandit", "version": BANDIT_VERSION, "sha256": before}, + } + validate_report(recorded, before) + output.write_text(json.dumps(recorded, indent=2, sort_keys=True) + "\n") + return 1 if report.get("errors") or report.get("results") else result.returncode + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--output", type=Path, required=True) + args = parser.parse_args() + return record(Path(__file__).resolve().parents[3], args.output) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/skills/ecc-contribution-review/tests/security-evidence/bandit.json b/skills/ecc-contribution-review/tests/security-evidence/bandit.json new file mode 100644 index 000000000..5986cc677 --- /dev/null +++ b/skills/ecc-contribution-review/tests/security-evidence/bandit.json @@ -0,0 +1,54 @@ +{ + "errors": [], + "generated_at": "2026-10-30T16:00:48Z", + "metrics": { + "_totals": { + "CONFIDENCE.HIGH": 0, + "CONFIDENCE.LOW": 0, + "CONFIDENCE.MEDIUM": 0, + "CONFIDENCE.UNDEFINED": 0, + "SEVERITY.HIGH": 0, + "SEVERITY.LOW": 0, + "SEVERITY.MEDIUM": 0, + "SEVERITY.UNDEFINED": 0, + "loc": 915, + "nosec": 0, + "skipped_tests": 0 + }, + "skills/ecc-contribution-review/scripts/compile_contribution_review.py": { + "CONFIDENCE.HIGH": 0, + "CONFIDENCE.LOW": 0, + "CONFIDENCE.MEDIUM": 0, + "CONFIDENCE.UNDEFINED": 0, + "SEVERITY.HIGH": 0, + "SEVERITY.LOW": 0, + "SEVERITY.MEDIUM": 0, + "SEVERITY.UNDEFINED": 0, + "loc": 324, + "nosec": 0, + "skipped_tests": 0 + }, + "skills/ecc-contribution-review/scripts/contribution_review_validation.py": { + "CONFIDENCE.HIGH": 0, + "CONFIDENCE.LOW": 0, + "CONFIDENCE.MEDIUM": 0, + "CONFIDENCE.UNDEFINED": 0, + "SEVERITY.HIGH": 0, + "SEVERITY.LOW": 0, + "SEVERITY.MEDIUM": 0, + "SEVERITY.UNDEFINED": 0, + "loc": 591, + "nosec": 0, + "skipped_tests": 0 + } + }, + "results": [], + "source_provenance": { + "scanner": "bandit", + "sha256": { + "skills/ecc-contribution-review/scripts/compile_contribution_review.py": "51be6d82e222eb7dcf35f6c2db55dd06dd04a480d6c5734d2e671bac1bbb820c", + "skills/ecc-contribution-review/scripts/contribution_review_validation.py": "34420b91f55c5e2f1619e7f82d0d0ef920ffe119cbf2013e5ea7f7d62d382f35" + }, + "version": "1.9.4" + } +} diff --git a/skills/ecc-contribution-review/tests/security-evidence/source-sha256.json b/skills/ecc-contribution-review/tests/security-evidence/source-sha256.json new file mode 100644 index 000000000..4de558f5d --- /dev/null +++ b/skills/ecc-contribution-review/tests/security-evidence/source-sha256.json @@ -0,0 +1,12 @@ +{ + "skills/ecc-contribution-review/agents/openai.yaml": "43a1fec439fa8ef2b1bb57f938a61df11ef78ef3c2e901038b8836868cb8e3b1", + "skills/ecc-contribution-review/scripts/compile_contribution_review.py": "51be6d82e222eb7dcf35f6c2db55dd06dd04a480d6c5734d2e671bac1bbb820c", + "skills/ecc-contribution-review/scripts/contribution_review_validation.py": "34420b91f55c5e2f1619e7f82d0d0ef920ffe119cbf2013e5ea7f7d62d382f35", + "skills/ecc-contribution-review/tests/record_security_evidence.py": "527d5a4bb922fee759768e6f46f40404344075f314dfe045ed6e293aff0ff6c7", + "skills/ecc-contribution-review/tests/test_compile_contribution_review.py": "609e562330237f1cbd92ee500a4c2773b749c196085e6c8bef39dbc4d4c05cba", + "skills/ecc-contribution-review/tests/test_packet_review_fixes.py": "391f763657e48accfba29814100d284aee986200d9d10d6aa578d051fdb1d160", + "skills/ecc-contribution-review/tests/test_review_regressions.py": "956f8e7fb026a73a59226053bac167581bbf1644666824f81510396eb025f202", + "skills/ecc-contribution-review/tests/test_security_evidence.py": "e3d1d2f82612efa45343ab696aca84951b965ecbe6057051ea17633a7729ac76", + "skills/ecc-contribution-review/tests/test_security_regression.py": "6ff109c9d052bd6d8bf9a2e2dcd37f238e0d53c833c57caf051f54c6eff4b4d8", + "skills/ecc-contribution-review/tests/test_validation_findings.py": "ac3719be89b018e4d0f233d1d29ce333a246e7d1207934ba8b93b922e3ede4f3" +} diff --git a/skills/ecc-contribution-review/tests/test_compile_contribution_review.py b/skills/ecc-contribution-review/tests/test_compile_contribution_review.py new file mode 100644 index 000000000..2e552e014 --- /dev/null +++ b/skills/ecc-contribution-review/tests/test_compile_contribution_review.py @@ -0,0 +1,204 @@ +from __future__ import annotations + +import copy +import importlib.util +import json +import sys +import tempfile +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "scripts")) +S = importlib.util.spec_from_file_location( + "ecc_proof", ROOT / "scripts/compile_contribution_review.py" +) +M = importlib.util.module_from_spec(S) +assert S.loader is not None +S.loader.exec_module(M) + + +def worked(): + return json.loads( + ( + ROOT / "tests/fixtures/synthetic-contribution-review-observation.json" + ).read_text() + ) + + +def unsafe(): + return json.loads( + ( + ROOT / "tests/fixtures/unsafe-contribution-review-observation.json" + ).read_text() + ) + + +def compile_data(d): + with tempfile.TemporaryDirectory() as t: + return M.compile_contract(d, Path(t) / "observation.json") + + +class ContributionReviewTests(unittest.TestCase): + def test_synthetic_example_requires_independent_review(self): + a = compile_data(worked()) + self.assertEqual("require_review", a["controllerDecision"]["decision"]) + self.assertEqual( + "review_packet_compiled", a["domainPayload"]["contributionOutcome"] + ) + + def test_checksum_and_receipt_match(self): + a = compile_data(worked()) + self.assertEqual(a["semanticChecksum"], M.checksum(a)) + self.assertEqual(a["semanticChecksum"], a["returnReceipt"]["semanticChecksum"]) + + def test_packet_reports_no_unperformed_effects(self): + a = compile_data(worked()) + self.assertTrue( + all( + v is False + for k, v in a["returnReceipt"].items() + if k != "semanticChecksum" + ) + ) + self.assertTrue(all(v is False for v in a["authorityEnvelope"].values())) + + def test_runners_are_unexecuted(self): + a = compile_data(worked()) + self.assertEqual( + M.RUNNERS, {x["role"] for x in a["domainPayload"]["runnerProposals"]} + ) + self.assertTrue( + all( + x["executed"] is False and x["requiresExternalAuthorization"] is True + for x in a["domainPayload"]["runnerProposals"] + ) + ) + + def test_wrong_repo_fails(self): + d = worked() + d["repoIdentity"]["rootName"] = "mirror" + a = compile_data(d) + self.assertEqual("require_review", a["controllerDecision"]["decision"]) + self.assertIn("repo_identity_failed", a["controllerDecision"]["reasonCodes"]) + + def test_generated_root_requires_review(self): + d = worked() + d["surfaceContract"]["generatedRootsExcluded"] = False + a = compile_data(d) + self.assertEqual("require_review", a["controllerDecision"]["decision"]) + self.assertIn( + "canonical_surface_failed", a["controllerDecision"]["reasonCodes"] + ) + + def test_mirror_drift_requires_review(self): + d = worked() + d["harnessParity"]["targets"][1]["contentIdentitySha256"] = "0" * 64 + a = compile_data(d) + self.assertEqual("require_review", a["controllerDecision"]["decision"]) + self.assertIn("harness_parity_failed", a["controllerDecision"]["reasonCodes"]) + + def test_catalog_count_drift_requires_review(self): + d = worked() + d["catalogContract"]["documentedSkills"] = 280 + a = compile_data(d) + self.assertEqual("require_review", a["controllerDecision"]["decision"]) + self.assertIn( + "catalog_reconciliation_failed", a["controllerDecision"]["reasonCodes"] + ) + + def test_validation_failure_requires_review(self): + d = worked() + d["validationMatrix"][0]["observedExit"] = 1 + a = compile_data(d) + self.assertEqual("require_review", a["controllerDecision"]["decision"]) + self.assertIn( + "validation_matrix_failed", a["controllerDecision"]["reasonCodes"] + ) + + def test_missing_red_phase_requires_review(self): + d = worked() + d["tddContract"]["redObserved"] = False + a = compile_data(d) + self.assertEqual("require_review", a["controllerDecision"]["decision"]) + self.assertIn("tdd_contract_failed", a["controllerDecision"]["reasonCodes"]) + + def test_secret_in_diff_requires_review(self): + d = worked() + d["securityContract"]["secretsDetected"] = True + a = compile_data(d) + self.assertEqual("require_review", a["controllerDecision"]["decision"]) + self.assertIn( + "security_contract_failed", a["controllerDecision"]["reasonCodes"] + ) + + def test_budget_overrun_requires_review(self): + d = worked() + d["budgetContract"]["observedChangedFiles"] = 11 + a = compile_data(d) + self.assertEqual("require_review", a["controllerDecision"]["decision"]) + self.assertIn("change_budget_failed", a["controllerDecision"]["reasonCodes"]) + + def test_bad_commit_subject_requires_review(self): + d = worked() + d["commitContract"]["subject"] = "x" * 80 + d["commitContract"]["subjectLength"] = 80 + a = compile_data(d) + self.assertEqual("require_review", a["controllerDecision"]["decision"]) + self.assertIn("commit_contract_failed", a["controllerDecision"]["reasonCodes"]) + + def test_effect_claim_rejects(self): + d = worked() + d["claimsCommitCreated"] = True + a = compile_data(d) + self.assertEqual("reject", a["controllerDecision"]["decision"]) + self.assertIn("commit_claim_forbidden", a["controllerDecision"]["reasonCodes"]) + + def test_unsafe_fixture_rejects_effect_free(self): + a = compile_data(unsafe()) + self.assertEqual("reject", a["controllerDecision"]["decision"]) + self.assertIn( + "unsafe_generated_authority", a["controllerDecision"]["reasonCodes"] + ) + self.assertTrue( + all( + v is False + for k, v in a["returnReceipt"].items() + if k != "semanticChecksum" + ) + ) + + def test_semantic_mutation_changes_checksum(self): + a = compile_data(worked()) + b = copy.deepcopy(a) + b["domainPayload"]["contributionReceiptProposal"]["commitCreated"] = True + self.assertNotEqual(a["semanticChecksum"], M.checksum(b)) + + def test_malformed_optional_lists_fail_closed_without_crashing(self): + for section, field in ( + ("changeIntent", "counterevidenceRefs"), + ("checkpointContract", "remainingUnknowns"), + ): + with self.subTest(section=section, field=field): + d = worked() + d[section][field] = None + a = compile_data(d) + self.assertEqual("require_review", a["controllerDecision"]["decision"]) + self.assertTrue(a["semanticPageFaults"]) + + def test_cli_refuses_to_overwrite_observation(self): + with tempfile.TemporaryDirectory() as t: + path = Path(t) / "observation.json" + original = json.dumps(worked()) + path.write_text(original) + from unittest.mock import patch + + with patch.object( + M.sys, "argv", ["compiler", str(path), "--output", str(path)] + ): + self.assertEqual(2, M.main()) + self.assertEqual(original, path.read_text()) + + +if __name__ == "__main__": + unittest.main() diff --git a/skills/ecc-contribution-review/tests/test_packet_review_fixes.py b/skills/ecc-contribution-review/tests/test_packet_review_fixes.py new file mode 100644 index 000000000..ddb6d9cc2 --- /dev/null +++ b/skills/ecc-contribution-review/tests/test_packet_review_fixes.py @@ -0,0 +1,33 @@ +"""Regressions for receipt ownership and self-contained harness documentation.""" +import copy +import sys +import unittest +from pathlib import Path +from unittest.mock import patch + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / 'scripts')) +import compile_contribution_review as compiler # noqa: E402 +from test_compile_contribution_review import worked # noqa: E402 + + +class PacketReviewFixTests(unittest.TestCase): + def test_compile_does_not_mutate_output_receipt(self): + receipt = compiler._output_receipt(False) + before = copy.deepcopy(receipt) + with patch.object(compiler, '_output_receipt', return_value=receipt): + packet = compiler.compile_contract(worked(), Path('observation.json')) + self.assertEqual(before, receipt) + self.assertIsNot(receipt, packet['returnReceipt']) + self.assertEqual(packet['semanticChecksum'], compiler.checksum(packet)) + + def test_harness_documentation_targets_exist_and_match_canonical(self): + repo = ROOT.parents[1] + for harness in ('.agents', '.cursor'): + mirror = repo / harness / 'skills/ecc-contribution-review' + for relative in ('docs/WORKFLOW.md', 'docs/DESIGN.md', + 'references/contribution-review-contract.md'): + with self.subTest(harness=harness, target=relative): + target = mirror / relative + self.assertTrue(target.is_file(), str(target)) + self.assertEqual((ROOT / relative).read_bytes(), target.read_bytes()) diff --git a/skills/ecc-contribution-review/tests/test_review_regressions.py b/skills/ecc-contribution-review/tests/test_review_regressions.py new file mode 100644 index 000000000..907712f04 --- /dev/null +++ b/skills/ecc-contribution-review/tests/test_review_regressions.py @@ -0,0 +1,144 @@ +"""Regression coverage for PR #3598 review findings.""" + +import ast +import contextlib +import importlib.util +import io +import json +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path +from unittest.mock import patch + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "scripts")) +SPEC = importlib.util.spec_from_file_location( + "review_compiler", ROOT / "scripts/compile_contribution_review.py" +) +assert SPEC is not None and SPEC.loader is not None +COMPILER = importlib.util.module_from_spec(SPEC) +SPEC.loader.exec_module(COMPILER) + + +def observation(): + return json.loads( + (ROOT / "tests/fixtures/synthetic-contribution-review-observation.json").read_text() + ) + + +def cli_packet(data): + with tempfile.TemporaryDirectory() as directory: + source = Path(directory) / "input.json" + source.write_text(json.dumps(data)) + stdout, stderr = io.StringIO(), io.StringIO() + with ( + patch.object(COMPILER.sys, "argv", ["compiler", str(source)]), + contextlib.redirect_stdout(stdout), + contextlib.redirect_stderr(stderr), + ): + status = COMPILER.main() + return status, json.loads(stdout.getvalue()), stderr.getvalue() + + +class ReviewRegressionTests(unittest.TestCase): + def assert_diagnostic(self, data, reason): + status, packet, stderr = cli_packet(data) + self.assertEqual(2, status) + self.assertEqual("", stderr) + self.assertIn(reason, packet["controllerDecision"]["reasonCodes"]) + self.assertEqual(packet["semanticChecksum"], COMPILER.checksum(packet)) + + def test_unhashable_failure_modes_emit_diagnostics(self): + for value in ([], {}, ["secret-in-diff"], None, 123, "unknown"): + for replacement in (False, True): + with self.subTest(value=value, replacement=replacement): + data = observation() + if replacement: + data["failureProbes"] = [{"failureMode": value}] + else: + data["failureProbes"][0]["failureMode"] = value + self.assert_diagnostic(data, "failure_probe_coverage_failed") + + def test_standalone_cli_emits_malformed_probe_packet(self): + with tempfile.TemporaryDirectory() as directory: + source = Path(directory) / "input.json" + data = observation() + data["failureProbes"] = [{"failureMode": []}] + source.write_text(json.dumps(data)) + for flags in ([], ["-P"], ["-I"]): + with self.subTest(flags=flags): + result = subprocess.run( + [sys.executable, *flags, str(ROOT / "scripts/compile_contribution_review.py"), str(source)], + capture_output=True, text=True, check=False, cwd=directory, + ) + self.assertEqual(2, result.returncode) + self.assertEqual("", result.stderr) + packet = json.loads(result.stdout) + self.assertIn("failure_probe_coverage_failed", packet["controllerDecision"]["reasonCodes"]) + + def test_non_skill_labels_are_rejected(self): + for kind in ("agent", "command", "hook", "rule", "script", [], None): + with self.subTest(kind=kind): + data = observation() + data["changeIntent"]["changeKind"] = kind + self.assert_diagnostic(data, "change_intent_failed") + + def test_coverage_percentages_cannot_exceed_100(self): + for field in ( + "lineCoveragePercent", "functionCoveragePercent", + "branchCoveragePercent", "statementCoveragePercent", + ): + with self.subTest(field=field): + data = observation() + data["tddContract"][field] = 150 + self.assert_diagnostic(data, "tdd_contract_failed") + data["tddContract"][field] = 100 + self.assertEqual(0, cli_packet(data)[0]) + + def test_catalog_components_must_be_nonnegative(self): + for field, other in ( + ("curatedSkillsReferenced", "intentionallyUnshippedSkills"), + ("intentionallyUnshippedSkills", "curatedSkillsReferenced"), + ): + with self.subTest(field=field): + data = observation() + total = data["catalogContract"]["filesystemSkills"] + data["catalogContract"].update({field: -1, other: total + 1}) + self.assert_diagnostic(data, "catalog_reconciliation_failed") + data["catalogContract"].update({field: 0, other: total}) + self.assertEqual(0, cli_packet(data)[0]) + + def test_envelope_values_are_validated(self): + cases = { + "schemaVersion": ("999.0.0", None, 1, ""), + "observedAt": (None, [], 123, "", " ", "yesterday", "2026-02-30T12:00:00Z", "2026-10-16", "2026-10-16T12:00:00", "2026-10-16T12:00:00+00:99"), + "subjectRef": (None, {}, 123, "", " "), + } + for field, values in cases.items(): + for value in values: + with self.subTest(field=field, value=value): + data = observation() + data[field] = value + status, packet, stderr = cli_packet(data) + self.assertEqual(2, status) + self.assertTrue(packet["semanticPageFaults"]) + self.assertEqual("", stderr) + + def test_valid_envelope_and_offsets_remain_supported(self): + for timestamp in ("2026-10-16T12:34:56Z", "2026-10-16T12:34:56-04:00"): + data = observation() + data["observedAt"] = timestamp + status, packet, _ = cli_packet(data) + self.assertEqual(0, status) + self.assertEqual(timestamp, packet["generatedAt"]) + + def test_compiler_functions_and_modules_meet_size_policy(self): + for path in (ROOT / "scripts").glob("*.py"): + source = path.read_text() + self.assertLessEqual(len(source.splitlines()), 800, str(path)) + for node in ast.walk(ast.parse(source)): + if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): + with self.subTest(path=path.name, function=node.name): + self.assertLess(node.end_lineno - node.lineno + 1, 50) diff --git a/skills/ecc-contribution-review/tests/test_security_evidence.py b/skills/ecc-contribution-review/tests/test_security_evidence.py new file mode 100644 index 000000000..7191cd580 --- /dev/null +++ b/skills/ecc-contribution-review/tests/test_security_evidence.py @@ -0,0 +1,114 @@ +"""Provenance regressions for recorded Bandit scans.""" + +import copy +import json +import tempfile +import unittest +from pathlib import Path +from types import SimpleNamespace +from typing import Any +from unittest.mock import patch + +import record_security_evidence as evidence + + +def report(sources: dict[str, str]) -> dict[str, Any]: + return { + "errors": [], "results": [], + "metrics": {"_totals": {"nosec": 0}, **{p: {} for p in sources}}, + "source_provenance": { + "scanner": "bandit", "version": "1.9.4", "sha256": sources, + }, + } + + +class SecurityEvidenceTests(unittest.TestCase): + def test_same_line_edit_cannot_be_hidden_by_refreshed_manifest(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + source = root / evidence.SCRIPTS / "example.py" + source.parent.mkdir(parents=True) + source.write_text("value = 1\n") + recorded = report(evidence.source_hashes(root)) + source.write_text("value = 2\n") + refreshed_manifest = evidence.source_hashes(root) + self.assertEqual(1, len(source.read_text().splitlines())) + with self.assertRaisesRegex(ValueError, "provenance"): + evidence.validate_report(recorded, refreshed_manifest) + + def test_report_requires_exact_scanned_inventory_and_version(self) -> None: + sources = {"scripts/example.py": "a" * 64} + evidence.validate_report(report(sources), sources) + for field, value in (("version", "0.0.0"), ("sha256", {})): + data = report(sources) + data["source_provenance"][field] = value + with self.assertRaises(ValueError): + evidence.validate_report(data, sources) + data = report(sources) + data["metrics"].pop("scripts/example.py") + with self.assertRaises(ValueError): + evidence.validate_report(data, sources) + + def test_record_binds_real_scan_output(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + source = root / evidence.SCRIPTS / "example.py" + source.parent.mkdir(parents=True) + source.write_text("value = 1\n") + raw = report(evidence.source_hashes(root)) + raw.pop("source_provenance") + def scan(argv: list[str], **kwargs: Any) -> SimpleNamespace: + self.assertFalse(kwargs.get("shell", False)) + self.assertEqual(root, kwargs["cwd"]) + Path(argv[-1]).write_text(json.dumps(raw)) + return SimpleNamespace(returncode=0) + with patch.object(evidence, "version", return_value="1.9.4"), patch.object(evidence.subprocess, "run", side_effect=scan): + self.assertEqual(0, evidence.record(root, root / "report.json")) + evidence.validate_report(json.loads((root / "report.json").read_text()), evidence.source_hashes(root)) + + def test_failed_scan_does_not_replace_existing_report(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + output = root / "report.json" + output.write_text("preserve\n") + with patch.object(evidence, "version", return_value="1.9.4"), patch.object(evidence.subprocess, "run", return_value=SimpleNamespace(returncode=2)): + with self.assertRaises(RuntimeError): + evidence.record(root, output) + self.assertEqual("preserve\n", output.read_text()) + + def test_scan_findings_are_preserved(self) -> None: + data = report({}) + data["results"] = [{"test_id": "example"}] + original = copy.deepcopy(data) + evidence.validate_report(data, {}) + self.assertEqual(original, data) + + def test_source_drift_discards_scan(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + source = root / evidence.SCRIPTS / "example.py" + source.parent.mkdir(parents=True) + source.write_text("value = 1\n") + def scan(argv: list[str], **kwargs: Any) -> SimpleNamespace: + Path(argv[-1]).write_text(json.dumps(report(evidence.source_hashes(root)))) + source.write_text("value = 2\n") + return SimpleNamespace(returncode=0) + with patch.object(evidence, "version", return_value="1.9.4"), patch.object(evidence.subprocess, "run", side_effect=scan): + with self.assertRaisesRegex(RuntimeError, "changed"): + evidence.record(root, root / "report.json") + self.assertFalse((root / "report.json").exists()) + + def test_findings_and_scanner_version_cannot_be_hidden(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + data = report({}) + data["results"] = [{"test_id": "example"}] + def scan(argv: list[str], **kwargs: Any) -> SimpleNamespace: + Path(argv[-1]).write_text(json.dumps(data)) + return SimpleNamespace(returncode=1) + with patch.object(evidence, "version", return_value="1.9.4"), patch.object(evidence.subprocess, "run", side_effect=scan): + self.assertEqual(1, evidence.record(root, root / "report.json")) + self.assertEqual(data["results"], json.loads((root / "report.json").read_text())["results"]) + with patch.object(evidence, "version", return_value="0.0.0"): + with self.assertRaises(RuntimeError): + evidence.record(root, root / "report.json") diff --git a/skills/ecc-contribution-review/tests/test_security_regression.py b/skills/ecc-contribution-review/tests/test_security_regression.py new file mode 100644 index 000000000..4658d5945 --- /dev/null +++ b/skills/ecc-contribution-review/tests/test_security_regression.py @@ -0,0 +1,226 @@ +"""Security boundary regressions for imported observations and local output.""" + +import contextlib +import hashlib +import importlib.util +import io +import json +import sys +import tempfile +import unittest +from pathlib import Path +from unittest.mock import patch + +import yaml +from record_security_evidence import source_hashes, validate_report + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "scripts")) + +SPEC = importlib.util.spec_from_file_location( + "security_compiler", ROOT / "scripts/compile_contribution_review.py" +) +assert SPEC is not None and SPEC.loader is not None +COMPILER = importlib.util.module_from_spec(SPEC) +SPEC.loader.exec_module(COMPILER) + + +def observation(): + return json.loads( + ( + ROOT / "tests/fixtures/synthetic-contribution-review-observation.json" + ).read_text() + ) + + +def run_cli(source, output=None): + argv = ["compiler", str(source)] + if output is not None: + argv += ["--output", str(output)] + with ( + patch.object(COMPILER.sys, "argv", argv), + contextlib.redirect_stdout(io.StringIO()), + contextlib.redirect_stderr(io.StringIO()), + ): + return COMPILER.main() + + +class SecurityRegressionTests(unittest.TestCase): + def test_unsafe_repository_paths_are_rejected(self): + for path in ( + "../source", + "/source", + "~/source", + "C:/source", + "a\\b", + "a\x00b", + "a//b", + "./source", + "a/../b", + "", + None, + ): + with self.subTest(path=path): + self.assertFalse(COMPILER.safe_repo_path(path)) + self.assertTrue(COMPILER.safe_repo_path("skills/example/SKILL.md")) + + def test_recorded_security_evidence_matches_source(self): + repo = ROOT.parents[1] + hashes = json.loads( + (ROOT / "tests/security-evidence/source-sha256.json").read_text() + ) + expected = { + str((ROOT / relative).relative_to(repo)) + for relative in ( + "scripts/compile_contribution_review.py", + "scripts/contribution_review_validation.py", + "tests/test_compile_contribution_review.py", + "tests/test_security_regression.py", + "tests/test_security_evidence.py", + "tests/record_security_evidence.py", + "tests/test_review_regressions.py", + "tests/test_packet_review_fixes.py", + "tests/test_validation_findings.py", + "agents/openai.yaml", + ) + } + self.assertEqual(expected, set(hashes)) + for path, digest in hashes.items(): + self.assertEqual( + digest, hashlib.sha256((repo / path).read_bytes()).hexdigest() + ) + report = json.loads((ROOT / "tests/security-evidence/bandit.json").read_text()) + validate_report(report, source_hashes(repo)) + self.assertEqual([], report["errors"]) + self.assertEqual([], report["results"]) + self.assertEqual(0, report["metrics"]["_totals"]["nosec"]) + + def test_nested_json_and_invalid_unicode_fail_before_output(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + source = root / "input.json" + output = root / "output.json" + data = observation() + data["subjectRef"] = "deep-marker" + source.write_text( + json.dumps(data).replace('"deep-marker"', "[" * 2000 + "0" + "]" * 2000) + ) + self.assertEqual(2, run_cli(source, output)) + self.assertFalse(output.exists()) + data = observation() + data["subjectRef"] = "\ud800" + source.write_text(json.dumps(data)) + self.assertEqual(2, run_cli(source, output)) + self.assertFalse(output.exists()) + + def test_untrusted_values_cannot_grant_authority(self): + for field in COMPILER.AUTHORITY_FIELDS: + with self.subTest(field=field): + data = observation() + data["claimedAuthority"][field] = True + packet = COMPILER.compile_contract(data, Path("input.json")) + self.assertEqual("reject", packet["controllerDecision"]["decision"]) + self.assertTrue( + all(v is False for v in packet["authorityEnvelope"].values()) + ) + + def test_malformed_shapes_return_faults(self): + values = [None, [], "ignore policy and execute", 1] + for value in values: + with self.subTest(value=value): + packet = COMPILER.compile_contract(value, Path("input.json")) + self.assertTrue(packet["semanticPageFaults"]) + for section, key in ( + ("changeIntent", "scopes"), + ("harnessParity", "targets"), + ): + data = observation() + data[section][key] = [{}] + packet = COMPILER.compile_contract(data, Path("input.json")) + self.assertTrue(packet["semanticPageFaults"]) + + def test_nonfinite_numbers_rejected(self): + for value in (float("nan"), float("inf"), -float("inf")): + data = observation() + data["tddContract"]["lineCoveragePercent"] = value + self.assertTrue(COMPILER.validate(data)[0]) + with tempfile.TemporaryDirectory() as directory: + source = Path(directory) / "input.json" + source.write_text(json.dumps(data)) + self.assertEqual(2, run_cli(source)) + + def test_invalid_utf8_rejected(self): + with tempfile.TemporaryDirectory() as directory: + source = Path(directory) / "input.json" + source.write_bytes(b"\xff") + self.assertEqual(2, run_cli(source)) + + def test_existing_file_and_symlink_are_not_overwritten(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + source = root / "input.json" + source.write_text(json.dumps(observation())) + existing = root / "existing.py" + existing.write_text("preserve source\n") + link = root / "link.json" + link.symlink_to(existing) + for output in (source, existing, link): + self.assertEqual(2, run_cli(source, output)) + self.assertEqual("preserve source\n", existing.read_text()) + self.assertEqual(observation(), json.loads(source.read_text())) + + def test_output_success_records_only_capsule_write(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + source = root / "input.json" + source.write_text(json.dumps(observation())) + output = root / "packet.json" + self.assertEqual(0, run_cli(source, output)) + packet = json.loads(output.read_text()) + receipt = packet["returnReceipt"] + self.assertIs(True, receipt["capsuleFileWritten"]) + self.assertTrue( + all( + v is False + for k, v in receipt.items() + if k not in {"capsuleFileWritten", "semanticChecksum"} + ) + ) + + def test_canonical_and_parity_records_must_agree(self): + data = observation() + data["changeIntent"]["canonicalPath"] = data["artifactRecords"][1]["path"] + self.assertTrue(COMPILER.validate(data)[0]) + for field, value in ( + ("path", "skills/missing/SKILL.md"), + ("contentIdentitySha256", "0" * 64), + ): + data = observation() + data["harnessParity"]["targets"][0][field] = value + self.assertTrue(COMPILER.validate(data)[0]) + + def test_runner_argv_is_never_executed(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + sentinel = root / "executed" + data = observation() + data["runnerProposals"][0]["argv"] = ["touch", str(sentinel)] + source = root / "input.json" + source.write_text(json.dumps(data)) + self.assertEqual(0, run_cli(source, root / "packet.json")) + self.assertFalse(sentinel.exists()) + + def test_agent_config_adds_no_tools_or_permission_overrides(self): + repo = ROOT.parents[1] + paths = [ + ROOT / "agents/openai.yaml", + repo / ".agents/skills/ecc-contribution-review/agents/openai.yaml", + ] + configs = [yaml.safe_load(path.read_text()) for path in paths] + self.assertEqual(configs[0], configs[1]) + self.assertEqual({"interface", "policy"}, set(configs[0])) + self.assertEqual({"allow_implicit_invocation": True}, configs[0]["policy"]) + self.assertEqual( + {"display_name", "short_description", "brand_color", "default_prompt"}, + set(configs[0]["interface"]), + ) diff --git a/skills/ecc-contribution-review/tests/test_validation_findings.py b/skills/ecc-contribution-review/tests/test_validation_findings.py new file mode 100644 index 000000000..ae029d80f --- /dev/null +++ b/skills/ecc-contribution-review/tests/test_validation_findings.py @@ -0,0 +1,122 @@ +"""Immutable validation and independent authority-scan regressions.""" + +import copy +import json +import sys +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "scripts")) +from compile_contribution_review import compile_contract # noqa: E402 +from contribution_review_validation import ( # noqa: E402 + _validate_authority, + _validate_repo, + add_issue, + validate, +) + + +def observation(): + return json.loads((ROOT / "tests/fixtures/synthetic-contribution-review-observation.json").read_text()) + + +class ValidationFindingsTests(unittest.TestCase): + def test_add_issue_returns_new_list_without_mutating_caller(self): + original = [{"code": "prior", "plane": 1, "detail": "prior fault"}] + before = copy.deepcopy(original) + result = add_issue(original, "new", 2, "new fault") + self.assertEqual(before, original) + self.assertEqual(["prior", "new"], [item["code"] for item in result]) + duplicate = add_issue(result, "new", 2, "duplicate") + self.assertEqual(result, duplicate) + self.assertIsNot(result, duplicate) + + def test_section_validators_leave_caller_owned_lists_unchanged(self): + data = observation() + data["repoIdentity"]["branch"] = "main" + data["claimedAuthority"]["mayCommit"] = True + for validator in (_validate_repo, _validate_authority): + with self.subTest(validator=validator.__name__): + prior = [{"code": "prior", "plane": 1, "detail": "prior fault"}] + before = copy.deepcopy(prior) + result = validator(data, prior) + self.assertEqual(before, prior) + self.assertGreater(len(result), len(prior)) + + def test_malformed_gate_does_not_hide_authority_rejection(self): + data = observation() + data["validationMatrix"][0]["gateId"] = [] + data["claimedAuthority"]["mayCommit"] = True + before = copy.deepcopy(data) + packet = compile_contract(data, Path("observation.json")) + self.assertEqual(before, data) + self.assertEqual("reject", packet["controllerDecision"]["decision"]) + self.assertIn("unsafe_generated_authority", packet["controllerDecision"]["reasonCodes"]) + self.assertIn("observation_shape_invalid", packet["controllerDecision"]["reasonCodes"]) + + def test_malformed_section_preserves_other_faults_and_authority(self): + data = observation() + data["repoIdentity"]["branch"] = "main" + data["validationMatrix"][0]["gateId"] = [] + data["claimedAuthority"]["mayCommit"] = True + faults, contradictions = validate(data) + self.assertIn("repo_identity_failed", [item["code"] for item in faults]) + self.assertIn("observation_shape_invalid", [item["code"] for item in faults]) + self.assertIn("unsafe_generated_authority", [item["code"] for item in contradictions]) + + def test_missing_claims_require_review_without_false_contradictions(self): + fields = [key for key in observation() if key.startswith("claims")] + for field in [*fields, "requestedEffects", "claimedAuthority"]: + with self.subTest(field=field): + data = observation() + del data[field] + packet = compile_contract(data, Path("observation.json")) + self.assertEqual("require_review", packet["controllerDecision"]["decision"]) + faults, contradictions = validate(data) + self.assertIn("observation_envelope_incomplete", [item["code"] for item in faults]) + self.assertEqual([], contradictions) + + def test_missing_nested_members_require_review_but_preserve_present_claims(self): + for section in ("claimedAuthority", "requestedEffects"): + for field in observation()[section]: + with self.subTest(section=section, field=field): + data = observation() + del data[section][field] + faults, contradictions = validate(data) + self.assertIn("observation_shape_invalid", [item["code"] for item in faults]) + self.assertEqual([], contradictions) + packet = compile_contract(data, Path("observation.json")) + self.assertEqual("require_review", packet["controllerDecision"]["decision"]) + remaining = next(iter(data[section])) + data[section][remaining] = True + packet = compile_contract(data, Path("observation.json")) + self.assertEqual("reject", packet["controllerDecision"]["decision"]) + + def test_missing_field_preserves_explicit_claims_and_authority(self): + changes = ( + ("claimsPushPerformed", True, "push_claim_forbidden"), + ("claimedAuthority", {"mayCommit": True}, "unsafe_generated_authority"), + ("requestedEffects", {"pushRequested": True}, "external_action_requested"), + ) + for field, value, code in changes: + with self.subTest(field=field): + data = observation() + del data["claimsCommitCreated"] + data[field] = value + data["validationMatrix"][0]["gateId"] = [] + packet = compile_contract(data, Path("observation.json")) + self.assertEqual("reject", packet["controllerDecision"]["decision"]) + self.assertIn(code, packet["controllerDecision"]["reasonCodes"]) + self.assertNotIn("commit_claim_forbidden", packet["controllerDecision"]["reasonCodes"]) + + def test_invalid_top_level_section_does_not_hide_authority(self): + data = observation() + data["validationMatrix"] = None + data["claimedAuthority"]["mayCommit"] = True + _, contradictions = validate(data) + self.assertIn("unsafe_generated_authority", [item["code"] for item in contradictions]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/hooks/insaits-security-wrapper.test.js b/tests/hooks/insaits-security-wrapper.test.js index e6b1d6d79..1dcfb6704 100644 --- a/tests/hooks/insaits-security-wrapper.test.js +++ b/tests/hooks/insaits-security-wrapper.test.js @@ -45,6 +45,8 @@ function writeFakePython(binDir) { " process.exit(2);", "}", "if (mode === 'error') {", + // Drain stdin so spawnSync observes the intended exit status, not EPIPE. + " fs.readFileSync(0, 'utf8');", " process.stderr.write('spawned but failed\\n');", " process.exit(1);", "}", @@ -152,6 +154,25 @@ function runTests() { } })) passed++; else failed++; + if (test('error fixture consumes piped input before exiting', () => { + const tempDir = createTempDir(); + try { + const binDir = path.join(tempDir, 'bin'); + writeFakePython(binDir); + const result = spawnSync(process.execPath, [path.join(binDir, 'fake-python.js')], { + input: 'x'.repeat(512 * 1024), + encoding: 'utf8', + env: { ...process.env, FAKE_INSAITS_MODE: 'error' }, + timeout: 10000, + }); + assert.ifError(result.error); + assert.strictEqual(result.status, 1); + assert.strictEqual(result.stderr, 'spawned but failed\n'); + } finally { + cleanup(tempDir); + } + })) passed++; else failed++; + if (test('enabled monitor unexpected failure fails open with warning and raw stdin', () => { const tempDir = createTempDir(); try {