_parse_classification returned {} for garbage or non-dict output, so a failed classification looked like zero matched steps and graded as non-compliant. Raise ClassifierParseError (a RuntimeError, like the non-zero-rc path) instead.
Preserve complete source ancestry for PRs #3512, #3513, #3515, #3516, #3517 and #3518. Repair bounded recovery, alias compatibility and evaluator packaging gaps while keeping registry reuse inside the canonical loader. Verified actual hook, setup, dashboard, evaluator, sandbox and profile workflows plus full npm, Python, lint and supply-chain checks. ECC-043 preparation only; integration remains with the delivery owner.
* fix(skill-comply): stop a failed step supplying evidence downstream
`_check_temporal_order` fell back to the raw classifier output whenever
the referenced step was absent from `resolved`. A step only enters
`resolved` once it passes, so "failed" and "not graded yet" were the
same thing to that lookup, and a dependant could pass on an event
belonging to a prerequisite that had failed its own ordering check.
With three steps C, A (before C) and B (after A) and events C@T0,
A@T1, B@T2, A fails and B passed on A's classified event: 2/3 instead
of 1/3. It compounds down a chain, so one failed prerequisite could
leave a five-step workflow reading 4/5.
The grader now tracks which steps have been graded at all. A referenced
step that was graded and is missing from `resolved` failed, and its
events are refused with a reason that says so. A step not graded yet is
a forward reference to a step declared later, and the fallback stays as
it was: that is what makes an out-of-order declaration work, and the
existing regression for it goes red if the fallback is removed instead.
`before_step` deliberately keeps the old fallback. The two fail in
opposite directions: an `after_step` fallback can only turn a failure
into a pass, a `before_step` one can only turn a pass into a failure,
so dropping it would relax a constraint because some other step failed.
* fix(skill-comply): revoke a pass that rested on a later-failing prerequisite
Review of #3109 found the mirror image of the case that PR fixes. `graded`
only catches a prerequisite that had already failed when its dependant was
graded. A step declared *before* its `after_step` is graded against the
classifier's raw events for a step that has not run yet — the fallback that
makes an out-of-order declaration work — and nothing revisited it once that
step went on to fail its own checks.
Add a pass after grading that demotes any detected step whose `after_step`
ended up failing, repeated to a fixed point: one demotion can invalidate
whatever depended on it, in either declaration order. Demotion only removes
passes, so it terminates. `compliance_rate` is computed from the demoted
results.
Also from review: build `graded` and the chain test's step list as new
objects rather than mutating (AGENTS.md immutability rule), and annotate the
injected mocks in the tests this PR owns.
_parse_stream_json() persisted raw tool_input/tool_response content into
ObservationEvents that grade() scores and generate_report() writes to
results/<skill>.md -- a report meant to be shared and reviewed.
--add-dir restricts the agent's additional accessible directory to the
sandbox (SANDBOX_BASE = /tmp/skill-comply-sandbox), but that doesn't stop
the agent's own tool calls (a Bash command using ~ expansion, a scenario
setup_commands entry referencing a dotfile) from emitting the operator's
home directory into tool_input/tool_response -- which then lands
verbatim, truncated but not sanitized, in the written report.
Adds _redact_home_path(), pure stdlib (Path.home()), applied to both
input_str and output_str before they're stored on the ObservationEvent.
Scoped deliberately to the home directory only -- grade() needs real
tool-call semantics for LLM-based compliance classification, so
truncating/stripping content the way a pure logging hook could isn't an
option here; only the operator-identifying path component needs to go.
New TestParseStreamJsonRedactsHomePath class in
skills/skill-comply/tests/test_runner.py (3 tests) -- full file now
10/10 passing, up from 7/7. Confirmed tests/test_invariant_runner.py (the
sandbox-execution security tests from #2149) still passes clean, 4/4.
Fixes#2730
- add Vite and Redis pattern skills from closed stale PRs
- add frontend-slides support assets
- port skill-comply runner fixes and LLM prompt/provider regressions
- harden agent frontmatter validation and sync catalog counts
haelyra