Commit Graph
10 Commits
Author SHA1 Message Date
haelyra 6cb9f0a162 fix: distinguish auth failures from ordinary provider discussion 2026-10-11 19:16:09 -04:00
haelyra 8d969b0ec4 fix: retain authentication diagnostics across empty stream blocks 2026-10-11 19:01:56 -04:00
haelyra 2ec80189a4 fix: preserve missing-result auth diagnostics and bound fixture contention 2026-10-11 18:46:07 -04:00
haelyra 344c1fc3fa fix: integrate truthful GateGuard harness grading and retry evidence 2026-10-11 11:06:13 -04:00
montjeffrey 64d155ae1d test(gateguard): pin grader decoys, pilot retries and the hookless-trial abort
Three #3481 review findings still hold on main 8ce348fb:

- csv-export-delimiter's grader checks only the dialect (CRLF, semicolons,
  quoting), so a constant '"x";"y"' export or one emitting the column
  names scores 1. Add decoy-constant and decoy-keys overlays and pin that
  each scores below 1 and that graderIsSound reports them; a scenario whose
  decoy passes is not sound.
- The pilot runner (run.js) marks errored keys done, so they are never
  retried, and armSummary counts provider-error and timed-out rows in
  trials, passes and means. Pin the summary on valid rows only, and drive
  run.js end to end: a provider error is retried on resume, keeps its own
  row and transcript, and the summary reports it apart.
- Disabling run-intent.js's abort on an unobserved hook left every test
  passing. Drive both runners end to end with a gated trial whose hook
  never runs: each must stop and record nothing for it.

fake-claude.js gains FAKE_CLAUDE_SKIP_HOOK and FAKE_CLAUDE_PROVIDER_ERROR.
The intent end-to-end setup moves into syntheticIntentRunner, shared with
the existing resume test.

Against main: 2 intent failures (the decoy cases) and 2 pilot failures
(summary, retry). The two abort tests pass on main, since both aborts
exist; each fails when its runner's abort is disabled
(if (false && !row.hookObserved)).
2026-10-11 10:00:47 -04:00
haelyra a2dbcdb2e0 fix(eval): reject resume symlinks without no-follow support 2026-10-10 18:38:26 -04:00
haelyra 7682bb5b49 fix(adapters): bound parsing and publish review artifacts safely 2026-10-10 18:38:26 -04:00
montjeffrey 175a0800d0 fix(gateguard): enforce evaluation validity guards 2026-10-05 01:43:33 -04:00
montjeffrey bfd4f8255a fix(gateguard): extract the arm tree without handing tar a drive letter
materializeTree passed the archive as an absolute path. GNU tar, which Git for
Windows bundles and puts first on PATH in Git Bash, reads an -f argument
containing a colon as a host:path remote spec and fails with "Cannot connect to
C: resolve failed". bsdtar, the tar in system32 that PowerShell resolves, has no
remote-spec parsing, so the fault only appeared under a bash-based shell - and
there it failed a committed test, not just a dry run.

Name the archive relatively and run tar with dest as its cwd, which extracts
identically under both tars and makes -C unnecessary. git archive keeps the
absolute -o path, which git handles on Windows.

Two tests, because the environment-dependent kind of fault needs both: one
injects a spawn and asserts no tar argument is absolute or drive-lettered, which
holds the invariant on any platform; the other prepends Git's bundled GNU tar to
PATH and extracts into a drive-letter destination, so the original failure is
reproducible from PowerShell too, where it was previously invisible.
2026-10-04 22:40:42 -04:00
Jeffrey Montoya 967daec0d6 feat(gateguard): measure question effectiveness with trapped agent tasks 2026-10-04 22:40:41 -04:00