Files
ECC/tests
4c28a7be9e ci: audit closed-issue delivery claims — cited PRs must actually be merged (#3522)
* ci: audit closed-issue delivery claims — cited PRs must actually be merged

Follow-up to #3223: an issue closed as 'fixed in #N' while #N was closed
without merging (or does not exist) leaves a completion claim that does not
hold. The weekly maintenance workflow now runs a read-only audit over issues
closed in the last 30 days, extracts delivery-claim PR references from issue
bodies and close-window comments, verifies each cited PR's merge state, and
reports findings to the step summary.

Design constraints:
- Read-only: no comments, no closes, no labels; findings are advisory and
  the run always stays green so the weekly audit never blocks maintenance.
- Unverifiable counts as unmerged: a 404 PR or an API failure fails the
  verdict, because an unverifiable claim is exactly the failure mode.
- Pure decision functions are exported for tests; a first live run against
  this repository surfaced one real finding (#3408 cites #3060, which does
  not resolve), included here as the motivating sample.

8 unit/contract tests: extraction, windowing (close comments included),
the #3223 unmerged-citation failure mode, the all-clear summary, workflow
contract, and a no-write guard.

* ci: rework audit evidence contract — claims vs context, four PR states, visible completeness

Revision per the #3522 review (reviewer ran the CLI through 19 controlled
cases plus a bounded live check of #3408):

- Bind findings to asserted delivery claims. Closing wording ('Delivered
  through #N'), claim verbs (fixed/shipped/landed/resolved/...), and bare
  'PR #N' inside closing comments are claims; historical mentions, quoted
  lines, fenced examples, negated claims, and tracking-only references are
  reported as contextual references and never produce findings. On the live
  #3408 the audit now recognizes the closing claim #3500 (merged) instead of
  flagging the historical #3060 text.
- Separate observed states: merged / open-unmerged / closed-unmerged /
  unknown (404, rate limit, transient). Unknown claims are advisory manual-
  verification items, not failed claims; each row carries the claim snippet,
  its source link, and the observation time. Checked claims are listed even
  when passing, so the report is traceable rather than an opaque all-clear.
- Paginate issue and comment collection (--paginate --slurp), bound comment
  requests (200) and PR checks (100), and surface skipped pages, rate limits,
  and retrieval failures as an explicit incomplete audit instead of an empty
  all-clear. A live-run maxBuffer overflow on the paginated issue list was
  caught and fixed (64MB); the audit reported it as incomplete before the fix.
- Scope the job to read access: job-level contents/issues/pull-requests read.

13 contract/unit tests including a CLI fixture (stub gh) that places the
relevant issue on page two and forces a comment-fetch failure.

* ci: report incomplete audit at PR check cap

---------

Co-authored-by: chenhz01 <chenhz01@users.noreply.github.com>
Co-authored-by: haelyra <49814733+haelyra@users.noreply.github.com>
2026-10-25 14:41:22 -04:00
..