Commit Graph
10 Commits
Author SHA1 Message Date
2SinceJulyand2SinceJuly 6c08320654 feat(process): add a non-executing dry run to capture_process_scenario (#8469)
* feat(process): add a non-executing dry run to capture_process_scenario

Preparation prompts asked an agent to prepare a scenario without running
it but could only suggest the tool that launches the target. Add
dry_run: true so the CLI and MCP resolve the executable, working
directory, observation paths and admission checks, then return planned
Evidence under the rea.process-scenario-plan predicate. Planned Evidence
is not capture Evidence and comparison tools refuse it as a capture.

The preparation prompt now suggests the dry-run call, and the scenario
identity commitments exclude the operation-only field.

Fixes #8398

* fix(process): redact planned sensitive input and share admission

Address review findings on the dry-run contract:

- Redact sensitive input events in planned Evidence and preserve only
  their byte length.
- Recheck cancellation after the last awaited admission step.
- Run the sensitive-matcher and filesystem-root admission checks that a
  capture performs before launch.
- Assert the prompt advertises the dry_run field without freezing
  explanatory wording.

* test(process): hold the final admission step for cancellation

Add a preparation host seam so the regression can cancel while the last
filesystem admission call is pending, then assert the returned
AnalysisCancelledError retains the caller's reason.

* refactor(process): remove the unused plan alias

Keep the plan schema as the single exported contract and satisfy the
unused-export check.

* fix(process): report admission-time aborts as planning cancellation

`prepareProcessScenarioPlan` returned the admission error verbatim, so an
abort that landed while `admitFilesystemObservationRoots` awaited its
`lstat` surfaced as a cancelled `ProcessCaptureError` with
`operation: process_capture` even though no process ran. Convert that
result through the same `cancelled()` projection the neighbouring checks
use, and pin it with a test.

* fix(contracts): keep the process result invariants on the dry-run union

The output schema for `capture_process_scenario` became a union of the
capture and plan shapes, and the union no longer carried the description
that advertised the cross-field invariants to MCP clients — the boundary
test reads `normalized_result.description`. Attach a description to the
union that names both variants and keeps the capture invariants verbatim.

* test(process): exercise the post-admission cancellation check

The previous regression aborted immediately after starting preparation and
released the held admission afterwards. Preparation awaits runtime path
resolution first, so that abort was always observed by the pre-admission
check; deleting the post-admission "if (aborted()) return cancelled();"
still left the test green.

Signal from the admitRoots seam when admission is actually entered, abort
only after that point, then let admission resolve ok. Preparation now has
to reach the check between a successful admission and the ok result, and
the assertion pins both the AnalysisCancelledError tag and the caller's
reason as its cause.

Verified red-green: removing only the post-admission check fails this test
(expected true to be false at prepared.ok) while the other two pass, and
restoring it passes all three.

* test(process): gate the admission seam on acknowledgement

Race the admission acknowledgement against planning's own settlement and a
bounded deadline, and release and join planning in `finally`, so a planning
result that arrives before admission fails the test instead of stalling it
until the suite timeout.

* fix(contracts): hold plans to the scenario refinements and redact inputs

`processScenarioPlanSchema` described the resolved dry-run response but
dropped every cross-field check the input contract applies, so a plan
could advertise out-of-order events, an event past `timeout_ms`, or a
terminal beyond the renderer cell budget while claiming it had been
validated. Extract those checks into `scenarioRefinements` and refine the
plan's scenario with them.

The plan's input event also carried `sensitive` and an optional
`input_bytes`, which accepted a sensitive event that still shipped its
data. Model the public and sensitive forms as separate exact variants:
public input forbids `input_bytes`, sensitive input requires an empty
`data` plus its byte length.

---------

Co-authored-by: 2SinceJuly <202121119+2SinceJuly@users.noreply.github.com>
2026-11-06 20:49:20 +08:00
morluto 60149282fc test: strengthen integration oracles and verification (#8295)
* test: strengthen integration oracles and verification

* test: bound lifecycle gates and strengthen workflow checks

* test: verify complete paged and extracted results

* test: preserve bytecode and cancellation evidence

* test: strengthen retained evidence and fixture cleanup

* test: require cross-projection bridge identities

* test: retain fixture owners through cleanup failures

* test: verify graph examples and project selectors

* test: tighten provider and lifecycle regressions

* test: consolidate hex coverage and bound cutter cleanup

* test: bound subprocess output waits and strengthen graph checks

* test: check mcp recovery and preserve cleanup owners

* test: verify structured mcp argument failures

* test: preserve artifact cleanup and gdb limit evidence

* test: preserve evidence identities and bound provider gates

* test: retain browser and comparison cleanup owners

* test: bound startup and strengthen fixture oracles

* test: bound lifecycle gates and verify recovered evidence

* test: verify public outcomes and lifecycle ownership

* test: assert precise evidence and release lifecycle gates

* test: verify selected facts and bound cleanup waits

* test: check retained evidence and decoded records

* test: verify final reports and retain cleanup ownership

* test: bound reviewed startup gates and retain cleanup

* test: verify PE errors and preserve MCP cleanup

* test: preserve MCP fixture cleanup ownership

* test: release provider gates before late startup
2026-11-01 00:15:28 +08:00
morluto 2886cc1efe test(process): compare progress with retained PTY chunks 2026-10-24 06:43:44 +08:00
Cassie_FoxandClaude Sonnet 5.5 21b836a327 test(process): print the progress frames when the sparse-child bound fails (#6427)
The sparse-child capture test asserted the frame bound with a bare boolean,
so a failure only printed "expected false to be true". The assertion now
carries the running progress details (elapsed time, frames, samples,
interactions) as its failure message. This only adds diagnostics to a
failure; it does not change what passes and does not fix the flake.

Refs #5889.

🤖 Generated with Claude Code (Sonnet 5.5) 🦊

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-23 17:52:21 +08:00
morluto dc936f433c test(process): distinguish native preparation from capture allocation
A missing temporary path stops Darwin native-inspector preparation before
capture allocation can account for resources. Keep its progress coverage
unknown; Linux reaches the capture root allocation and reports not-required
resources. Assert the public typed failure preserves the selected temp path.

The combined Electron/progress focused rerun passed 33 tests on macOS arm64
with Node 24.18.0. The seven progress cases passed; native Linux was not run.
Static checks, docs checks and test discovery also passed on the final tree.
2026-10-19 20:04:00 +08:00
gensokyo-dev b999e72476 fix(ci): repair capture ownership and portable ABI validation (#5224)
* fix(tests): restore reliable snapshot and container CI checks

Keep exact cancellation causes and partial snapshot facts in regression oracles. Match fault injection to real directory handles and isolated MCP module ownership; await terminal cleanup progress within the existing test deadline. Reuse the valid ASAR Pickle fixture and preserve truncated-container rejection.

* fix(tests): isolate runtime faults and own CLI subprocesses

Keep the runtime provider mock in the isolated project after reused CLI graphs exposed lost fault injection. Use the existing owned CLI fixture for Inspector parity and update command checks, retain caller environment choices, and require the removed alias to fail with COMMAND_NOT_FOUND. Keep current test deadlines and production behavior.

* fix(process): retain executable handles through read cancellation

Use the existing borrowed descriptor reader so stream cancellation cannot invalidate a handle owned by capture. Preserve the exact cancellation and permit immediate scope reuse after cleanup. Reproduce the native-stream failure with actual file IO and retain completed/failed-close controls.

Synchronize runtime cancellation and backwards-clock injection with observed running progress. The corrected clock oracle now rejects a Date.now-based deadline under controlled rollback; keep test deadlines and narrow host cleanup checks unchanged.

* test(process): observe prelaunch cancellation at descriptor reads

* fix(process): retain read failures alongside observed cancellation

* fix(flutter): advertise portable Unicode ABI selectors
2026-10-19 19:32:22 +08:00
morluto fc7e41ff3b fix(process): attribute finite trace omissions and cleanup states 2026-10-15 03:43:29 +08:00
N0zoM1z0 3cd80cbb88 test(process): prepare SDK validation before cancellation timing 2026-10-10 18:54:57 +08:00
N0zoM1z0 6cd5ea0fb2 fix(process): coalesce delayed capture progress
Retain one pending intermediate observation and drop stale live status before terminal cleanup. Preserve monotonic counts and final delivery while an observer is slow or rejects an update.

Verify the send rate at its controlled-clock owner and lifecycle notification density at the CLI/MCP boundary, where receive timestamps include buffering and scheduling.
2026-10-10 18:54:57 +08:00
Cursor Agent 706bba6e13 feat(process): report live capture progress through CLI and MCP
Long captures emitted only a final result, so a quiet child looked idle.
Pass the shared rate-limited progress reporter through the capture lifecycle
and report the observed phase, elapsed time, and collected-unit counts.
CLI writes those lines to stderr; MCP forwards them only when the client
supplied a progress token. Final Evidence on stdout keeps its shape.
2026-10-10 18:54:57 +08:00