diff --git a/.agents/skills/rea-tool-design/SKILL.md b/.agents/skills/rea-tool-design/SKILL.md index ca069df009..3dd31ffdb1 100644 --- a/.agents/skills/rea-tool-design/SKILL.md +++ b/.agents/skills/rea-tool-design/SKILL.md @@ -13,7 +13,9 @@ documentation, and live catalogs may describe different versions. For provider-backed work, exercise the analyst's task through REA and compare relevant native provider and independent artifact observations where available. Check applicable partial results, recovery and session boundaries; report what -was not exercised. Use [workflow verification](../../../docs/testing.md#provider-backed-workflow-verification) +was not exercised. For failure guidance, follow [design recovery around the +failure](../../../docs/tool-design.md#design-recovery-around-the-failure). +Use [workflow verification](../../../docs/testing.md#provider-backed-workflow-verification) to select evidence for the proposed contract instead of seeking provider API parity. diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md index 616b53c5f1..7f718848af 100644 --- a/.agents/skills/reverse-engineer-anything/SKILL.md +++ b/.agents/skills/reverse-engineer-anything/SKILL.md @@ -378,8 +378,9 @@ unless the group overrides them. The grouped name union is the complete tool inventory; use `tools/list` for canonical tool schemas. The default summary keeps each provider candidate's availability and target-support status, code and reason; `omitted_fields` lists what it leaves out. Request `{"detail": "complete"}` -only when you need provider diagnostics, candidate capability sets, or tool -surface, effect and annotation flags; in those closed flag vocabularies, listed +only when you need an unavailable candidate's remediation, provider +diagnostics, candidate capability sets, or tool surface, effect and annotation +flags; in those closed flag vocabularies, listed flags are true and omitted known flags are false. Provider candidate facts describe possible providers, while top-level session capabilities describe the current route and `analysis_provider_binding` names the selected provider. Do @@ -432,7 +433,10 @@ output also streams without a single MCP frame. When the retained parent ID is unknown, use `get_evidence_bundle` with `detail: "summary"` and exact filters for operation, target SHA-256, analysis-profile digest or canonical procedure address. Select its returned -Evidence ID for the next view instead of requesting the complete bundle. +Evidence ID for the next view instead of requesting the complete bundle. When +that record has no supported view (for example an annotation), read it with +`detail: "records"` and an `evidence_id` filter; `missing_evidence_links` +names linked sources this session does not retain. `retention: "complete-record"` means the whole record is stored; native value-flow truncation and unknown metadata remain separate. For an exported portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`, diff --git a/.agents/skills/reverse-engineer-anything/references/evidence-workflows.md b/.agents/skills/reverse-engineer-anything/references/evidence-workflows.md index 32a5962d40..0a83d37e79 100644 --- a/.agents/skills/reverse-engineer-anything/references/evidence-workflows.md +++ b/.agents/skills/reverse-engineer-anything/references/evidence-workflows.md @@ -73,6 +73,14 @@ Parent Evidence ID plus source index preserves same-address alternatives. Follow `coverage.next_offset` only as far as the question needs; exhausting retained rows does not establish complete program discovery. +`batch_decompile` returns every body unless you select `detail: "summary"` when +the connected tool advertises it. Use its parent Evidence ID with +`view: {kind: "batch-decompile", collection: "items"}` for body-free rows and +exact `where.indices`, `addresses`, `procedure_addresses` or `statuses`. Add +`include_pseudocode: true` for several chosen bodies, or use +`collection: "pseudocode"` with one row's `index` to page a large body. Batch +limitations aggregate every item, so do not attribute them to one successful row. + Use `record_unknown` to track unresolved questions in the user's investigation and name the authority or environment still required. Supply supporting and contradicting evidence IDs. It updates the session registry directly. diff --git a/.agents/skills/reverse-engineer-anything/references/javascript-applications.md b/.agents/skills/reverse-engineer-anything/references/javascript-applications.md index 9caf17076e..34594c8bd3 100644 --- a/.agents/skills/reverse-engineer-anything/references/javascript-applications.md +++ b/.agents/skills/reverse-engineer-anything/references/javascript-applications.md @@ -169,7 +169,10 @@ leaving it, and `both` when the question needs both. Trace traversal includes containment relationships, so even a narrow seed can expand many nodes; a bidirectional trace can reach unrelated sibling modules. A module item or page is preferable when its observations already answer the question. Do not treat -all reached nodes as participants in the feature. +all reached nodes as participants in the feature. For a broad trace, pass +`view: {"kind": "feature-trace", "collection": "paths"}` (or `summary`) to +retain the complete trace and read terminal paths first; page `nodes` or `edges` +with exact `where.node_ids` through `inspect_analysis_view` as needed. Reuse retained parent Evidence when the server advertises it, or supply complete Evidence inline. Include Hopper or Ghidra Evidence only when its artifact digest diff --git a/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md b/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md index d039ee1b90..6dabb4ad78 100644 --- a/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md +++ b/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md @@ -356,8 +356,9 @@ the complete tool inventory grouped by current status. Expand each group's overrides them; availability, reasons, remediation, and modes remain inline. Provider candidates keep availability and target-support status, code, and reason. `omitted_fields` names what the summary leaves out; `{"detail": -"complete"}` adds provider diagnostics, candidate capability sets, and tool -surface, `effects`, and `annotations`. Those arrays name true flags from fixed +"complete"}` adds each unavailable candidate's `remediation`, provider +diagnostics, candidate capability sets, and tool surface, `effects`, and +`annotations`. Those arrays name true flags from fixed vocabularies, so an omitted known flag is false. `tools/list` remains the canonical source for full tool schemas when target or provider state changes. diff --git a/.github/codeql/production.yml b/.github/codeql/production.yml new file mode 100644 index 0000000000..f96a1c7609 --- /dev/null +++ b/.github/codeql/production.yml @@ -0,0 +1,21 @@ +name: REA production code + +# Match the existing default-setup scan's local-input threat model. +threat-models: [local] + +# Retain root traversal, including future production roots. Hidden directories +# need their own root; use a directory so Python can share this configuration. +paths: + - . + - docs/.vitepress + +# Keep the default query suite and all production roots, including bridge/, +# install.sh, runtime scripts, release controllers, and GitHub Actions. +# Exclude test inputs at extraction time so they cannot taint production callers. +paths-ignore: + - tests + - scripts/fixtures + - "**/*.test.*" + - "**/*.fixture.*" + - "**/test_*.py" + - "**/*_test.py" diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml new file mode 100644 index 0000000000..e2e5bb47bc --- /dev/null +++ b/.github/workflows/codeql.yml @@ -0,0 +1,61 @@ +name: CodeQL production + +on: + push: + branches: [main] + pull_request: + branches: [main] + schedule: + - cron: "37 5 * * 1" + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: codeql-production-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true + +jobs: + analyze: + # A repository administrator must disable default setup before enabling + # this workflow; default setup rejects advanced CodeQL result uploads. + if: vars.REA_CODEQL_ADVANCED == 'true' + name: Analyze production (${{ matrix.language }}) + runs-on: ubuntu-latest + timeout-minutes: 45 + permissions: + contents: read + security-events: write + strategy: + fail-fast: false + max-parallel: 1 + matrix: + language: [javascript-typescript, python, actions] + steps: + - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 + with: + persist-credentials: false + - name: Initialize production analysis + uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4 + with: + languages: ${{ matrix.language }} + build-mode: none + config-file: .github/codeql/production.yml + threads: 2 + ram: 4096 + - name: Analyze production code + uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4 + with: + category: /language:${{ matrix.language }} + output: ${{ runner.temp }}/codeql-production-results + threads: 2 + ram: 4096 + - name: Retain complete analysis evidence + if: always() + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 + with: + name: codeql-production-${{ matrix.language }} + path: ${{ runner.temp }}/codeql-production-results + if-no-files-found: error + retention-days: 7 diff --git a/AGENTS.md b/AGENTS.md index b701e8d65d..27408d5bef 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -28,6 +28,10 @@ Assess brittleness in touched code during implementation, debugging, review, and When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why. +After confirming a bug, search sibling adapters, consumers and similar workflows +for the same failure. Verify suspected matches, record working comparisons, and +group findings by root cause instead of filing duplicate reports. + When addressing a review thread, inspect the producing boundary and affected consumers for brittleness. Complete directly related refactoring in the same change. Before resolving the thread, explain the root cause, structural fix, validation, and any remaining cleanup. Parse unknown boundary values and model expected failures with the tagged error algebra and `Result`. Preserve partial facts and meaningful failure reasons. Keep identity and lookup values separate from display formatting; preserve source values and explicit unknowns when normalization loses information. Absolute paths, file URLs, and HTTP paths have different semantics. @@ -75,6 +79,9 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions. +When changing installers or setup recovery, follow +[Checking installers and publisher downloads](docs/testing.md#checking-installers-and-publisher-downloads). + Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/testing.md#developer-commands). `docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts. @@ -83,6 +90,12 @@ Coordinate expensive checks across agents. If a test unexpectedly consumes heavy For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage. +For user- or agent-facing tool problems, prioritize fixing the owning behavior, +contract, and actionable result or recovery at the point of use. Document the +result in product docs when explanation is needed. Add skill guidance only for +workflow judgment or context that the tool cannot reasonably convey; do not use +a skill as the primary repair for a discoverability or contract defect. + For test changes or provider claims, use [docs/testing.md](docs/testing.md). Select applicable [boundary regressions](docs/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior. Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope. diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 40409b89eb..e58c4d94f2 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -60,7 +60,7 @@ pin inspection sources; they are not included in the npm package. | ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. | | JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. | | jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. | -| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.986/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.986/NOTICE) | Bring your own `kuna`; REA verified 1.986 and never downloads, installs or bundles it. REA invokes it as a separate process. | +| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.988/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.988/NOTICE) | Bring your own `kuna`; REA verified 1.988 and never downloads, installs or bundles it. REA invokes it as a separate process. | | Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. | | Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. | | Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. | diff --git a/bridge/ghidra/ReaGhidraBridge.java b/bridge/ghidra/ReaGhidraBridge.java index 99bf59c93d..eb26f63d84 100644 --- a/bridge/ghidra/ReaGhidraBridge.java +++ b/bridge/ghidra/ReaGhidraBridge.java @@ -693,6 +693,7 @@ public final class ReaGhidraBridge extends HeadlessScript { } row.addProperty("status", relocation.getStatus().toString()); row.addProperty("type", relocation.getType()); + row.add("symbol", GSON.toJsonTree(relocation.getSymbolName())); row.add("values", GSON.toJsonTree(relocation.getValues() == null ? new long[0] : relocation.getValues())); byte[] original = relocation.getBytes(); row.add("original_bytes_hex", original == null ? JsonNull.INSTANCE : GSON.toJsonTree(bytesHex(original))); @@ -2274,9 +2275,6 @@ public final class ReaGhidraBridge extends HeadlessScript { throw new RequestFailure("invalid_request", "mode must be literal or regex"); } boolean caseSensitive = requireBoolean(params, "case_sensitive"); - List inventory = procedureSearch - ? procedures().stream().map(FunctionEntry::item).toList() - : strings(); ValueMatcher matcher; if (mode.equals("literal") && !caseSensitive) { if (!params.has("literal_ascii_candidate")) @@ -2290,7 +2288,26 @@ public final class ReaGhidraBridge extends HeadlessScript { : regexMatcher(expression, caseSensitive); } JsonArray items = new JsonArray(); - for (InventoryItem item : inventory) { + if (procedureSearch) { + // Search the names exact lookup accepts, keeping one row per + // procedure under its primary name. + for (FunctionEntry entry : procedures()) { + monitor.checkCancelled(); + ProcedureNameMatch match = matchProcedureNames(entry, matcher); + if (!match.matched()) { + continue; + } + JsonObject item = resultItem(entry.item, null); + if (!match.aliases().isEmpty()) { + JsonArray aliases = new JsonArray(); + for (String alias : match.aliases()) aliases.add(alias); + item.add("matched_aliases", aliases); + } + items.add(item); + } + return items; + } + for (InventoryItem item : strings()) { monitor.checkCancelled(); if (!matcher.matches(item.value)) { continue; @@ -2300,6 +2317,36 @@ public final class ReaGhidraBridge extends HeadlessScript { return items; } + /** + * The one definition of the names that select a procedure, shared by exact + * lookup and search: its primary name and every other symbol at its exact + * entry, each in leaf or namespace-qualified spelling. A label inside the + * body never names the procedure. Matching secondary symbols are reported + * by their qualified names, sorted. + */ + private ProcedureNameMatch matchProcedureNames(FunctionEntry entry, ValueMatcher matcher) throws Exception { + Function function = entry.function; + boolean primary = spellingMatches(function.getName(), entry.item.value, matcher); + Set aliases = null; + for (Symbol symbol : currentProgram.getSymbolTable().getSymbols(function.getEntryPoint())) { + monitor.checkCancelled(); + String qualified = symbol.getName(true); + if (!spellingMatches(symbol.getName(), qualified, matcher)) continue; + if (qualified.equals(entry.item.value)) { + primary = true; + continue; + } + if (aliases == null) aliases = new TreeSet<>(); + aliases.add(qualified); + } + return new ProcedureNameMatch(primary, aliases == null ? List.of() : List.copyOf(aliases)); + } + + /** A leaf is the suffix of its qualified spelling, so a literal found in it is found in both. */ + private static boolean spellingMatches(String leaf, String qualified, ValueMatcher matcher) { + return matcher.matches(qualified) || (!leaf.equals(qualified) && matcher.matches(leaf)); + } + private List names() throws Exception { if (nameInventory != null) { return nameInventory; @@ -2402,26 +2449,13 @@ public final class ReaGhidraBridge extends HeadlessScript { // exact symbol name, even when every character is a hexadecimal digit. boolean explicitAddress = isExplicitReaAddress(value); if (!explicitAddress) { + // Imported labels (for example Mach-O _main) can share a function + // entry without becoming its primary symbol; search accepts the same. + ValueMatcher selector = value::equals; List matches = new ArrayList<>(); for (FunctionEntry entry : procedures()) { monitor.checkCancelled(); - Function function = entry.function; - Symbol symbol = function.getSymbol(); - String qualified = symbol == null ? function.getName() : symbol.getName(true); - boolean matchesName = function.getName().equals(value) || qualified.equals(value); - if (!matchesName) { - // Imported labels (for example Mach-O _main) can share a - // function entry without becoming its primary symbol. Only - // exact entries qualify; an interior label is not a selector. - for (Symbol entrySymbol : currentProgram.getSymbolTable().getSymbols(function.getEntryPoint())) { - monitor.checkCancelled(); - if (entrySymbol.getName().equals(value) || entrySymbol.getName(true).equals(value)) { - matchesName = true; - break; - } - } - } - if (matchesName) matches.add(function); + if (matchProcedureNames(entry, selector).matched()) matches.add(entry.function); } if (matches.size() > 1) throw new RequestFailure("ambiguous", "Ghidra procedure name is ambiguous: " + value + @@ -5138,6 +5172,11 @@ public final class ReaGhidraBridge extends HeadlessScript { private record AddressNameConflict(Namespace namespace, String leaf, List
requesters, List
holders) {} private record AddressNameFailure(String reason, AddressNameConflict conflict) {} private record FunctionEntry(Function function, InventoryItem item) {} + private record ProcedureNameMatch(boolean primary, List aliases) { + boolean matched() { + return primary || !aliases.isEmpty(); + } + } private record FunctionAnalysisObservation(JsonObject dossier, List limitations, JsonObject relationshipCoverage, boolean decompilerFailed) {} private record CallBoundaryObservation(List limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {} private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {} diff --git a/bridge/process/ProcessRunTokenReader.swift b/bridge/process/ProcessRunTokenReader.swift index d5152ab50c..599fbdb1b4 100644 --- a/bridge/process/ProcessRunTokenReader.swift +++ b/bridge/process/ProcessRunTokenReader.swift @@ -15,6 +15,8 @@ struct ProcessIdentityObservation: Encodable { let effective_uid: UInt32? let lifecycle: String? let relation: ProcessRelationObservation? + /** Display-only kernel accounting name (p_comm, at most MAXCOMLEN bytes). */ + let name: String? let reason: String? } @@ -31,6 +33,7 @@ struct ProcessMetadata { let parentPid: Int32 let processGroupId: Int32 let pointerSize: Int + let name: String } struct ProcessSnapshot { @@ -92,7 +95,8 @@ func readProcessSnapshot( lifecycle: process.kp_proc.p_stat == SZOMB ? "zombie" : "non-zombie", parentPid: process.kp_eproc.e_ppid, processGroupId: process.kp_eproc.e_pgid, - pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4 + pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4, + name: processName(process) ) ) }, @@ -103,6 +107,13 @@ func readProcessSnapshot( return ProcessSnapshot(metadata: [:], failure: "process_table_failed_\(ENOMEM)") } +/** Decode the NUL-terminated accounting name; invalid UTF-8 is replaced for display. */ +private func processName(_ process: kinfo_proc) -> String { + withUnsafeBytes(of: process.kp_proc.p_comm) { bytes in + String(decoding: bytes.prefix { $0 != 0 }, as: UTF8.self) + } +} + func readProcessIdentities(pids: [Int32]) -> [ProcessIdentityObservation] { readProcessIdentities(pids: pids, snapshot: readProcessSnapshot()) } @@ -130,6 +141,7 @@ func readProcessIdentities( effective_uid: nil, lifecycle: nil, relation: nil, + name: nil, reason: "not-in-snapshot" ) } @@ -144,6 +156,7 @@ func readProcessIdentities( parent_pid: metadata.parentPid, process_group_id: metadata.processGroupId ), + name: metadata.name, reason: nil ) } @@ -157,6 +170,7 @@ private func unavailableIdentity(pid: Int32, reason: String) -> ProcessIdentityO effective_uid: nil, lifecycle: nil, relation: nil, + name: nil, reason: reason ) } diff --git a/docs/adb-device-analysis.md b/docs/adb-device-analysis.md index 634a004b0a..1dfcb826da 100644 --- a/docs/adb-device-analysis.md +++ b/docs/adb-device-analysis.md @@ -257,12 +257,12 @@ beyond already-connected devices (`adb connect`) are not exposed yet. ## Real verification -`npm run verify:adb` exercises every operation against a live device. It +`npm run verify:adb` exercises observation and transfer operations against a live device. It requires `adb` (or `REA_ADB_PATH`) and any attached emulator or phone; `--serial` selects one when several are attached, `--pull PACKAGE` pulls a -real package and verifies the returned digests against the files on disk, -and `--install-apk PATH` drives a full install → resolve → uninstall -lifecycle with a lane-owned probe. +real package and verifies the returned digests against the files on disk. +The optional `--install-apk` lifecycle is unavailable until the selected APK +identity and device cleanup ownership can be established safely. Record: verified against adb 34.0.5 on Linux against an Android 14 (API 34) x86_64 emulator — all operations above, a 224-package inventory, diff --git a/docs/binary-diagnostics.md b/docs/binary-diagnostics.md index b58b8d04dc..56427673af 100644 --- a/docs/binary-diagnostics.md +++ b/docs/binary-diagnostics.md @@ -424,8 +424,10 @@ rea inspect-native-exports ./libexample.so --detail summary --evidence_output ./ without an export directory is a complete empty inventory. - ELF rows list defined `.dynsym` symbols with global, weak or GNU-unique binding and default or protected visibility. Each row keeps `value` - (`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type` - and `binding`. ELF string tables do not declare a text encoding. Names that + (`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`, + `binding` and `visibility` (`"default"` or `"protected"`; `"unknown"` in older + historical Evidence where visibility was not recorded). ELF string tables do + not declare a text encoding. Names that contain non-ASCII bytes are decoded as UTF-8 when those bytes form valid UTF-8; `name_utf8_decodable` records whether the non-ASCII name bytes are valid UTF-8 and `name_bytes_base64` retains those exact bytes (`null` for diff --git a/docs/cli.md b/docs/cli.md index 205da957c2..a0a7c6dae3 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -561,6 +561,14 @@ be verified. See [MCP contracts](mcp-contracts.md) for session lifecycle and [Ghidra first-query deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery) for import, client timeouts and recovery. +A one-shot command releases its provider session and REA runtime before it +prints a failure, so it omits the same-session reuse and +`update_analysis_settings` guidance an MCP connection receives. Its +`details.limit` reports `scope: "rea-invocation"` and +`applies_at: "next_invocation"`; set the named `environment` (or the per-call +`parameter`) for a new invocation and rerun. For example, a Ghidra decompile +deadline names `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS`. + If one-shot analysis fails and closing its session also fails, the original error details remain inline and `details.cleanup_error` describes the later close failure. Each diagnostic retains its own reported resources; the later @@ -615,6 +623,30 @@ cached result. `close_binary` accepts `snapshot_path` and optional `overwrite: true` to save before releasing provider resources. A failed save leaves the session open so the caller can resolve the output failure. +When a validated snapshot does not match the selected binding, the error's +`details.binding_mismatch` reports confirmed target identities, provider +identities, or profile digests that differ. `snapshot` is the saved value and +`selected` is the requested value. Multiple differences can be reported. A +`profile_comparison: "not_compared"` means the profile was not available at +that boundary; it does not establish a match. A profileless snapshot can report +`workflow_scope: "incompatible"` even when its target and profile match. +Choose a matching target, provider, and profile, or analyze without the snapshot +and save a new one for the intended binding. + +For a profile mismatch, the saved snapshot contains +`binding.analysis_profile.parameters`. If `binary_session` reports an active +profile whose digest matches `binding_mismatch.profile_digest.selected`, its +`analysis_provider_binding.analysis_profile.parameters` can be compared with +those saved parameters. A rejected switch leaves the previous session active, +so its profile may be different from the attempted selection; check the digest +before using it for this comparison. A configuration difference does not by +itself establish why analysis results differ. + +If no target is open and you intend to inspect the selected binding, open that +target without the snapshot, then check its profile digest in `binary_session` +before comparing parameters. This may start the selected provider; the saved +snapshot remains a separate file. + If a selected snapshot's file descriptor cannot close, `open_binary` reports `cleanup_incomplete`. `binary_session` reports `cleanup_pending` even when no target opened. Retry `close_binary` to release the retained descriptor before @@ -1263,6 +1295,18 @@ new file. A publication that has committed can win a later cancellation; the after commit, the `cleanup_incomplete` error reports the same file as `details.published_path` and `details.bytes`. +When a failure retains a completed observation and no bundle committed, +`details.source_closure` states what that observation cites. `publication` is +`not_published`. `records` holds authentic source Evidence this invocation +captured, within 64 KiB. `omitted` lists up to 64 other identities and +`omitted_count` the total: `not_captured` means the invocation holds no such +record, so nothing is shown for it or its ancestors; `not_inlined` means it was +captured but is not in this response. `complete` is true only when every cited +record is in `records` or embedded in the observation. If examining the closure +itself fails, `collection_failure` names the error type and the closure state is +unknown; the original failure is still returned. Nothing is retained after exit, +and running the command again is a new observation. + When the reader of piped output closes early, as `rea ... | head` does, the CLI stops writing, prints no error and exits `141` (`128 + 13`, the status of a SIGPIPE-terminated writer). The operation itself may have completed; only its diff --git a/docs/design/limits-and-recovery.md b/docs/design/limits-and-recovery.md index 94d9642010..9eccda38fb 100644 --- a/docs/design/limits-and-recovery.md +++ b/docs/design/limits-and-recovery.md @@ -222,6 +222,25 @@ Error copying, attribution, redaction, and delivery must preserve these facts. Human-readable remediation explains the same failure; clients should not need to parse that sentence to identify an editable setting. +`scope` and `applies_at` describe a continuing caller, such as an MCP +connection. A one-shot invocation, such as a CLI command, has released its +provider session and exited its runtime before its caller reads the failure, +so its adapter projects with an explicit `"invocation"` caller lifetime. +`details.limit` then keeps every other field and reports +`scope: "rea-invocation"`, `applies_at: "next_invocation"` and the launch +`environment` seed when one exists. `agent_editable: true` still holds, because +the caller chooses that environment value or per-call `parameter` for the new +invocation; `rea_restart_required: false` means no running REA process needs a +restart. A producer keeps guidance that holds only while its caller retains the +session, such as "the Ghidra session remains usable" or a same-session runtime +update, in `continuingCallerGuidance`; the shared presentation adds it only for +a continuing caller. The producer's own message and remediation must stay true +after the session is released. A composed error records each embedded child +failure through `childFailureDiagnostics`, which snapshots it at composition +for both lifetimes when they differ, so cleanup history keeps its state and +the final projection selects the caller's form. Failures retained inside +Evidence keep the continuing form, so CLI and MCP Evidence stay identical. + Cache policy is part of enforcement. Successful reusable analysis can survive a budget update. Transient Ghidra decompile failures remain uncached so a raised deadline can recover on the same Program. Admission checks still apply to reused @@ -286,7 +305,9 @@ whether live repair is possible. ## What agents do Use `details.limit` for a direct runtime update when it names an editable -setting. Follow `remediation.action` for per-call, fixed and external controls. +setting with `scope: "rea-runtime"`. For `scope: "rea-invocation"`, set its +`environment` or `parameter` for a new invocation instead. +Follow `remediation.action` for per-call, fixed and external controls. Inspect settings only when bounds or application details are missing. Restart only when it says the value is read at startup. A sentence that a control does not raise the limit is not a request to change it. @@ -316,8 +337,12 @@ request to change it. `runtime-configuration:` check. Re-run `rea setup` for that client when the setting is on the registration allowlist, then reconnect so the new process sees it. -- `invalid_request`: correct the reported fields from the current schema or - CLI help. Do not repeat the unchanged call. +- `invalid_request`: follow the specific recovery and reported issue. Inspect + the schema or CLI help when the input violates the contract or the response + asks for `inspect_input_contract`. A valid selector can instead require a + different choice, such as selecting a reported address; an inventory miss + does not establish that the target lacks the item. Do not repeat the + unchanged call when the response says it cannot succeed unchanged. A restart is required to adopt a changed launch environment or registration. The curated runtime settings above require no REA restart. It is not the first step for a transient diff --git a/docs/electron-observation.md b/docs/electron-observation.md index 08c1984e8a..0c6d78adf6 100644 --- a/docs/electron-observation.md +++ b/docs/electron-observation.md @@ -147,6 +147,14 @@ window-targeted renderer reload/crash actions, and synthetic `open-url` or WebContents identities, process metrics, and IPC channel and value-shape metadata. Payload values are not retained. +For the exact Playwright launch sequence, the active hook removes only REA's +bootstrap arguments from the application-visible `process.argv`. The vector keeps +the selected entry point and every caller argument, in order, including repeated +hook paths, `.asar` filenames, and debug-like options. The private launch handoff +is consumed before application code runs. If the observed sequence cannot be +matched exactly, argv stays unchanged and capture coverage reports `hook_conflict`; +REA does not guess an entry point or remove arguments by filename patterns. + `window_index` selects from Playwright's current Page list at the start of each action. REA resolves that Page to its corresponding BrowserWindow once; the action's `target` uses the BrowserWindow's capture-scoped `window_id`, matching the window inventory. Reordering either enumeration after selection does not retarget the action. Handle cleanup failures retain the completed action records and remain owned for provider cleanup retry. Run the real fixture verifier with an operator-provided Electron runtime: @@ -193,6 +201,9 @@ neither result means the requested host operation occurred. The existing `openExternal`, `openPath`, and `trashItem` blocks reject their promises. The `trashItem` block prevents that call from reaching the OS trash operation; it does not intercept other ways the application can remove files. +Process rejection and uncaught-exception events are observed at dispatch without +adding exception handlers. The runtime and application retain their warning, +handler, and fatal-exit policies, including for rejected shell-block promises. CLI SIGINT/SIGTERM requests cancellation and awaits owned-process cleanup. Window IDs, URLs, and cached titles come from one main-process Electron snapshot, so a crashed renderer does not require a successful renderer-side title query. diff --git a/docs/ghidra-provider.md b/docs/ghidra-provider.md index e4d2c6bff7..ad65004b3c 100644 --- a/docs/ghidra-provider.md +++ b/docs/ghidra-provider.md @@ -172,6 +172,11 @@ have no file backing. ELF mappings are observed without independent format-specific load-image verification, as the result's status and limitations state. +Relocation rows retain Ghidra's `symbol` value, including an empty string or +null when returned by Ghidra. It is a provider observation, not proof of the +runtime-resolved address. Older Evidence may not have recorded this field; +a relocation page requesting it reports that uncertainty. + For large relocation inventories, select MCP `inspect_native_load_image` with `detail: "summary"`. It returns verification state, observed identity and counts, while retaining complete Evidence. Use `normalized_result.parent_evidence_id` diff --git a/docs/ida-provider.md b/docs/ida-provider.md index 685d638b9a..03362fec16 100644 --- a/docs/ida-provider.md +++ b/docs/ida-provider.md @@ -43,7 +43,8 @@ rea decompile /absolute/path/to/program main --json If the registration cannot be read or validated, provider diagnostics retain its selected `registration_path` and the configuration owner's setting/constraint -facts in `configuration_details.settings`, plus `configuration_remediation`. +facts in `configuration_details.settings`. The availability's `remediation`, and +each operation it makes unavailable, carry the configuration owner's repair. `configured` means a path was selected, not that it is valid or an IDA connection was established; failed registrations remain unavailable. Correct `REA_IDA_MCP_CONFIG` in the next CLI invocation's environment or the MCP server's diff --git a/docs/installation.md b/docs/installation.md index 7763d7c906..041df39f82 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -666,7 +666,8 @@ Hopper is separate commercial software with its own license. Its free demo has vendor-defined limits, and a paid license is optional. REA reuses any detected installation and preserves Hopper during uninstall. -The automated Hopper installer supports macOS 12+ and x86_64 hosts running +The automated Hopper installer supports macOS 12+, subject to the minimum +macOS version of the current Hopper release, and x86_64 hosts running Ubuntu 24.04+, Fedora 41+, Nobara 44+, Arch Linux, CachyOS, or Omarchy. The official Linux packages are x86_64-only, so setup does not offer Hopper on other Linux architectures; configure [Ghidra](#ghidra) there instead. REA itself runs on any @@ -675,12 +676,33 @@ an existing Hopper launcher is selected through `HOPPER_LAUNCHER_PATH`. Ghidra and IDA have their own provider-specific host requirements; Windows Ghidra uses the [experimental P0 boundary](windows-ghidra-p0.md). -If the Linux package download receives an HTTP failure, setup includes the -observed HTTP status in its `download_failed` diagnostic. The status does not -establish the underlying cause or guarantee that retrying will succeed. +Bazzite uses RPM packages, but REA does not install Hopper automatically on it. +Setup skips the Hopper offer because Bazzite is outside the supported host list, +regardless of whether Xvfb is installed. Bazzite documents +[containers for installing Linux packages](https://docs.bazzite.gg/Installing_and_Managing_Software/Containers/). +If using a container, check its distribution and architecture against the list +above. Install REA and Hopper in that container, give it access to the files you +want to analyze, and configure your agent to launch REA there. Verify installation +and analysis before relying on that workflow. + +`rea doctor` checks health; it does not offer to install Hopper. Run `rea setup` +for installation. Setup cannot prompt when its output is redirected or during a +container image build. To approve Hopper installation without prompts, run +`rea setup --yes --install-hopper`. + +If a Hopper request fails on macOS or Linux, the setup result's `download` +field names the request (`release_feed`, `package` or `public_key`), its `url`, +and the observed outcome `kind`: `http_status` with `httpStatus`, `timeout`, +`network`, `response_too_large`, or `url_refused`. The remediation names the +same URL and outcome. HTTP 404 or 410 means the file is missing, so an +immediate retry is unlikely to help; 5xx, 408 and 429 indicate a temporary +service failure. The status does not establish the underlying cause or guarantee +that retrying will succeed. Doctor and Linux launches report the same request +and status when publisher verification cannot download the current package. Hopper metadata and package downloads require HTTPS throughout the redirect -chain; a redirect to HTTP is refused before that destination is requested. +chain; a redirect to HTTP or to a URL carrying credentials is refused before +that destination is requested, and those credentials are not reported. On macOS, approved setup downloads the official DMG, checks its published size and digest, validates the application bundle, and atomically installs it to @@ -688,6 +710,15 @@ and digest, validates the application bundle, and atomically installs it to operator can choose its demo mode or activate an existing license. Homebrew and administrator access are not used. +Each Hopper release declares its own minimum macOS version, so the setup plan +cannot confirm support before approval. Setup compares this Mac's version with +the release metadata's `deployment_target` before downloading the DMG, and with +the signed application's `LSMinimumSystemVersion` before copying it. If this Mac +is older, setup fails with `incompatible_host_version`, names both versions, and +installs nothing. Update macOS, or install a Hopper release that supports this +macOS version yourself and set `HOPPER_LAUNCHER_PATH`. An existing Hopper +installation is not checked against the current release's minimum. + The Hopper launcher action used by REA creates a document from an executable; its supported command-line interface does not attach to an already-open document. While another live REA session owns the same target and loader @@ -728,6 +759,12 @@ Rotating signing subkeys remain bound to that primary identity. The feed row itself is not PGP-signed; doctor labels its display version as `reported_release_version`, separately from the authenticated launcher identity. +Even a correctly signed package can contain a launcher whose hash disagrees with +the feed. REA stops before installing that package. When reporting this error, +include the release version, package format and both hashes from the diagnostic. +Each format needs its own check: a matching Ubuntu DEB does not establish that +the Fedora RPM matches. + The approval plan includes the caller-selected cache location. Approved setup retains the signed archive, public key and release metadata under `${XDG_CACHE_HOME:-$HOME/.cache}/rea/hopper-releases/`. @@ -761,6 +798,16 @@ host mount and the rest of `/tmp` remain unchanged; this fallback never invokes `sudo`. `rea doctor --provider hopper --json` reports the selected strategy and both host and effective mount facts. +You do not need to start a separate `Xvfb :99` server. REA starts and manages its +own display. After installation, run `rea doctor --provider hopper --json` and +check `hopper-demo-runtime` for display and Python problems. Setup also runs these +health checks: installing the package is only one step toward a working provider. + +If an image build ends with `exit status 1`, read the setup output immediately +before that summary to find the failing step. On minimal Ubuntu images, run +`apt-get update` before setup if an earlier layer deleted the package indexes. +REA's package installation commands do not refresh those indexes. + Hopper also embeds Python; the Python executable and standard library alone are insufficient on Ubuntu. Approved setup checks the system interpreter at `/usr/bin/python3` and, when necessary, installs its matching @@ -1284,7 +1331,8 @@ fails with `provider_timeout` and that message; other Ghidra tools stay available. If the caller also cancels while the timeout settles, the session still closes and the message says so. Session reuse concerns an active MCP or library session; a standalone CLI command still closes its own session when it -finishes. In MCP, `ghidra.decompile_timeout_seconds` can override the launch +finishes, so its timeout names `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS` for a new +invocation instead. In MCP, `ghidra.decompile_timeout_seconds` can override the launch value for newly accepted operations on the existing Program; explicit `null` removes the deadline. A transient timeout remains uncached so the same function can be retried after an update. See the diff --git a/docs/javascript-application-workflows.md b/docs/javascript-application-workflows.md index c3a975f842..83802e1c2b 100644 --- a/docs/javascript-application-workflows.md +++ b/docs/javascript-application-workflows.md @@ -173,7 +173,14 @@ Results follow retained node order and default to offset 0 and limit 64. Supply to examine relationships from that node. Other typed seeds do not search labels. The result contains the matching basis, complete reachable subgraph, terminal -paths, authority summaries, and native handoffs. A handoff binds +paths, authority summaries, and native handoffs. A bidirectional trace of a +large application can exceed one MCP response. Over MCP, pass +`view: {"kind":"summary"}` or a `feature-trace` page of `paths`, `nodes`, +`edges`, `seed_matches` or `native_handoffs` to retain the complete trace in +the session and return only selected rows with the trace's identity and +coverage. Page the retained trace with `inspect_analysis_view`; an oversized +complete reply offers its first `paths` page as a ready-to-call continuation. +See [MCP contracts](mcp-contracts.md) for filters. A handoff binds the exact native artifact digest and requested exports from the application graph. Trace direction selects the reachable addons; requested exports for each selected addon come from the complete supplied graph, so an diff --git a/docs/javascript-artifact-reconstruction.md b/docs/javascript-artifact-reconstruction.md index 88d97864a9..7ec2febebb 100644 --- a/docs/javascript-artifact-reconstruction.md +++ b/docs/javascript-artifact-reconstruction.md @@ -1002,7 +1002,12 @@ reason in the renderer observations. HTML script source ranges follow the HTML parser across LF, CRLF, and bare CR line endings, preserving UTF-16 columns. Inline JavaScript in local HTML, meaning classic or module `` : "" }`); - } - }); + } + }, + ); server.listen(0, "127.0.0.1"); await once(server, "listening"); return { diff --git a/scripts/lib/browser-scenario-storage-e2e.mjs b/scripts/lib/browser-scenario-storage-e2e.mjs index 568c0e891f..ef352f16ac 100644 --- a/scripts/lib/browser-scenario-storage-e2e.mjs +++ b/scripts/lib/browser-scenario-storage-e2e.mjs @@ -1,5 +1,4 @@ import assert from "node:assert/strict"; -import { createServer } from "node:http"; import { once } from "node:events"; import { setTimeout as delay } from "node:timers/promises"; @@ -11,6 +10,7 @@ import { scenarioProfiles, } from "./browser-scenario-verifier.mjs"; import { withVerifierCleanup } from "./verifier-cleanup.mjs"; +import { createBrowserFixtureHttpServer } from "./browser-fixture-http.mjs"; const scenarioFor = (browser, origin) => browserScenarioSchema.parse({ @@ -149,28 +149,29 @@ const storagePage = (kind, port) => { `; }; -const startStorageSite = async () => { +export const startStorageSite = async () => { let port = 0; - const server = createServer((request, response) => { - const path = new URL(request.url ?? "/", "http://127.0.0.1").pathname; - const kind = - path === "/storage-main" - ? "main" - : path === "/storage-frame" - ? "frame" - : path === "/storage-debugger" - ? "debugger" - : path === "/storage-top" - ? "top" - : undefined; - if (kind === undefined) { - response.writeHead(404).end(); - return; - } - response.setHeader("content-type", "text/html; charset=utf-8"); - response.end(storagePage(kind, port)); - }); - server.listen(0, "::"); + const server = createBrowserFixtureHttpServer( + (_request, response, { pathname: path }) => { + const kind = + path === "/storage-main" + ? "main" + : path === "/storage-frame" + ? "frame" + : path === "/storage-debugger" + ? "debugger" + : path === "/storage-top" + ? "top" + : undefined; + if (kind === undefined) { + response.writeHead(404).end(); + return; + } + response.setHeader("content-type", "text/html; charset=utf-8"); + response.end(storagePage(kind, port)); + }, + ); + server.listen(0, "127.0.0.1"); await once(server, "listening"); port = server.address().port; return { diff --git a/scripts/lib/real-ghidra-entry-aliases.mjs b/scripts/lib/real-ghidra-entry-aliases.mjs index 9fd3cefdfc..7b87ec9bb3 100644 --- a/scripts/lib/real-ghidra-entry-aliases.mjs +++ b/scripts/lib/real-ghidra-entry-aliases.mjs @@ -87,11 +87,50 @@ export async function verifyGhidraEntryAliases({ (await call("procedure_info", { procedure: alias.value })).entrypoint, alias.address, ); + // Search matches the names exact lookup accepts, as one procedure row + // under its primary name. + const escaped = alias.value.replace(/[.*+?^${}()|[\]\\]/gu, "\\$&"); + for (const query of [ + { pattern: alias.value }, + { pattern: `^${escaped}$`, mode: "regex", case_sensitive: true }, + ]) { + assert.deepEqual( + (await call("search_procedures", query)) + .filter(({ address }) => address === alias.address) + .map(({ value, matched_aliases }) => ({ value, matched_aliases })), + [{ value: canonical.value, matched_aliases: [alias.value] }], + `Procedure search missed the entry alias for ${JSON.stringify(query)}`, + ); + } + assert.ok( + (await call("list_procedures")).every( + (row) => row.matched_aliases === undefined, + ), + "Listings must not report search matches", + ); + const traced = await call("trace_feature", { query: alias.value }); + assert.ok( + traced.matches.some( + ({ type, address, matched_aliases }) => + type === "procedure" && + address === alias.address && + matched_aliases?.includes(alias.value) === true, + ), + "trace_feature missed the procedure its entry alias names", + ); const interior = names.find( ({ value }) => value === `${prefix}rea_interior`, ); assert.ok(interior); assert.notEqual(interior.address, alias.address); + assert.ok( + (await call("search_procedures", { pattern: interior.value })).every( + ({ address, matched_aliases = [] }) => + address !== interior.address && + !matched_aliases.includes(interior.value), + ), + "An interior label must not match procedure search", + ); assert.equal( interior.symbol.type, "label", diff --git a/scripts/lib/real-ghidra-namespace-annotations.mjs b/scripts/lib/real-ghidra-namespace-annotations.mjs index 35ecfaa3c8..2e4736a9cd 100644 --- a/scripts/lib/real-ghidra-namespace-annotations.mjs +++ b/scripts/lib/real-ghidra-namespace-annotations.mjs @@ -68,6 +68,21 @@ export async function verifyGhidraNamespaceAnnotations({ assert.equal(ambiguous.code, "invalid_request"); for (const { address } of entries) assert.ok(JSON.stringify(ambiguous.details.issues).includes(address)); + // Search accepts the leaf spelling exact lookup accepts, keeping each + // procedure as one row under its qualified primary name. + const leafMatches = await call("search_procedures", { + pattern: "^same$", + mode: "regex", + case_sensitive: true, + }); + for (const { namespace, address } of entries) + assert.deepEqual( + leafMatches + .filter((row) => row.address === address) + .map(({ value }) => value), + [`${namespace}::same`], + `Leaf-name search missed ${namespace}::same`, + ); for (const { namespace, address } of entries) { const leaf = await call("annotate_native_function", { procedure: address, diff --git a/scripts/lib/real-ghidra-relocation-symbols.mjs b/scripts/lib/real-ghidra-relocation-symbols.mjs new file mode 100644 index 0000000000..2ce4522bcf --- /dev/null +++ b/scripts/lib/real-ghidra-relocation-symbols.mjs @@ -0,0 +1,92 @@ +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { readFile } from "node:fs/promises"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { parseEvidence } from "../../dist/domain/evidence.js"; +import { PrivateRuntimeRoot } from "../../dist/process/PrivateRuntimeRoot.js"; +import { withVerifierCleanup } from "./verifier-cleanup.mjs"; +import { createVerifierCommands } from "./verify-commands.mjs"; + +const source = fileURLToPath( + new URL( + "../../tests/conformance/ghidra/relocation-symbol.c", + import.meta.url, + ), +); + +/** Verify named and unnamed ELF relocations through the shipped CLI and Ghidra. */ +export async function verifyGhidraRelocationSymbols({ clang, lld, env }) { + const commands = await createVerifierCommands(); + let root; + let commandsClosed = false; + return withVerifierCleanup(async () => { + root = await PrivateRuntimeRoot.create({ prefix: "rea-relocation-proof-" }); + const object = join(root.path, "relocation-symbol.o"); + const target = join(root.path, "relocation-symbol.so"); + const commandEnv = { + ...env, + TMPDIR: root.path, + TMP: root.path, + TEMP: root.path, + }; + const run = (command, args) => + commands.exec(command, args, { + env: commandEnv, + timeout: 120000, + maxBuffer: 16 * 1024 * 1024, + }); + await run(clang, [ + "--target=x86_64-linux-gnu", + "-fPIC", + "-O0", + "-c", + source, + "-o", + object, + ]); + await run(lld, ["-shared", object, "-o", target]); + const hash = async () => + createHash("sha256") + .update(await readFile(target)) + .digest("hex"); + const before = await hash(); + const { stdout } = await run(process.execPath, [ + "scripts/rea.mjs", + "inspect-native-load-image", + target, + "--provider", + "ghidra", + "--json", + ]); + const evidence = parseEvidence(JSON.parse(stdout)); + assert.equal(evidence.subject?.digest.sha256, before); + assert.equal(evidence.provider.id, "ghidra"); + const report = evidence.normalized_result; + assert.equal(report.status, "unsupported"); + const relocations = report.observations.relocations; + assert.ok( + relocations.some(({ symbol }) => symbol === "rea_external"), + "The Ghidra relocation must retain its imported symbol", + ); + assert.ok( + relocations.some(({ symbol }) => symbol === ""), + "The provider's empty symbol must remain distinct from an unrecorded field", + ); + assert.equal(await hash(), before); + return { + source, + target_sha256: before, + named_symbol: "rea_external", + empty_symbol: true, + }; + }, [ + async () => { + await commands.close(); + commandsClosed = true; + }, + async () => { + if (commandsClosed && root !== undefined) await root.close(); + }, + ]); +} diff --git a/scripts/lib/test-discovery.d.mts b/scripts/lib/test-discovery.d.mts index 259a21fbb8..54b6d1f3ee 100644 --- a/scripts/lib/test-discovery.d.mts +++ b/scripts/lib/test-discovery.d.mts @@ -10,5 +10,10 @@ export function compareTestDiscovery( readonly file: string; readonly projects: readonly string[]; }[]; + readonly wrongProjects: readonly { + readonly file: string; + readonly expected: string; + readonly actual: string; + }[]; readonly discovered: number; }; diff --git a/scripts/lib/test-discovery.mjs b/scripts/lib/test-discovery.mjs index dbcafb220a..99af5bc6b7 100644 --- a/scripts/lib/test-discovery.mjs +++ b/scripts/lib/test-discovery.mjs @@ -1,4 +1,5 @@ import { relative, resolve } from "node:path"; +import { projectForTest } from "./test-projects.mjs"; /** Compare actual Vitest discovery with repository-owned test paths. */ export const compareTestDiscovery = (expectedPaths, discovery, root) => { @@ -36,6 +37,14 @@ export const compareTestDiscovery = (expectedPaths, discovery, root) => { const projects = projectsByFile.get(file); return projects.length > 1 ? [{ file, projects: sorted(projects) }] : []; }), + wrongProjects: sorted(expected).flatMap((file) => { + const projects = projectsByFile.get(file); + if (!projects || projects.length !== 1) return []; + const expectedProject = projectForTest(file).name; + return projects[0] === expectedProject + ? [] + : [{ file, expected: expectedProject, actual: projects[0] }]; + }), discovered: projectsByFile.size, }; }; diff --git a/scripts/test-website.py b/scripts/test-website.py index 54b4679585..227dd4b122 100644 --- a/scripts/test-website.py +++ b/scripts/test-website.py @@ -539,11 +539,13 @@ class WebsiteMaintenanceChecks(unittest.TestCase): self.assertIn(b"color: black", response.read()) with self.assertRaises(HTTPError) as error: urlopen(base + prefix + "missing") - self.assertEqual(error.exception.code, 404) + with contextlib.closing(error.exception) as response: + self.assertEqual(response.code, 404) if prefix != "/": with self.assertRaises(HTTPError) as error: urlopen(base + "/guide/") - self.assertEqual(error.exception.code, 404) + with contextlib.closing(error.exception) as response: + self.assertEqual(response.code, 404) finally: server.shutdown() worker.join() diff --git a/scripts/verify-adb.mjs b/scripts/verify-adb.mjs index 58183f04a1..fd4ed01285 100644 --- a/scripts/verify-adb.mjs +++ b/scripts/verify-adb.mjs @@ -15,8 +15,10 @@ if (options.help) { process.stdout.write( "Usage: npm run verify:adb [-- --serial SERIAL --pull PACKAGE]\n" + "Requires a caller-selected adb binary (REA_ADB_PATH or PATH) and a\n" + - "connected device or emulator. Verifies all five operations against\n" + - "the live device; --pull acquires a real package and re-digests the\n" + + "connected device or emulator. Verifies observation and transfer operations\n" + + "against the live device. --install-apk is unavailable until APK identity\n" + + "and device cleanup ownership can be established safely.\n" + + "--pull acquires a real package and re-digests the\n" + "files on disk against the returned SHA-256 values.\n", ); process.exit(0); @@ -356,79 +358,6 @@ try { `versionCode=${String(details.version_code)}`, ); } - if (options.installApk !== null && options.installApk !== undefined) { - const probePackage = "com.rea.adb.probe"; - const installedProbe = await run("install_adb_package", { - serial, - apk_path: options.installApk, - replace: false, - }); - if (installedProbe !== null) { - check( - "install reports the APK digest", - typeof installedProbe.sha256 === "string" && - installedProbe.sha256.length === 64, - `${installedProbe.bytes} bytes`, - ); - const resolved = await run("resolve_adb_packages", { - serial, - query: "rea.adb.probe", - }); - if (resolved !== null) - check( - "resolve finds the installed probe uniquely", - resolved.matches.length === 1 && - resolved.matches[0]?.package_name === probePackage, - `${resolved.matches.length} match(es)`, - ); - const started = await run("start_adb_app", { - serial, - package: probePackage, - }); - if (started !== null) - check( - "app start resolves and launches the launcher activity", - typeof started.component === "string" && - started.component.startsWith(probePackage), - String(started.component), - ); - const observedProcesses = await run("list_adb_processes", { serial }); - if (observedProcesses !== null) - check( - "started app appears in the process listing", - observedProcesses.processes.some( - (process) => process.name === probePackage, - ), - ); - const stopped = await run("stop_adb_app", { - serial, - package: probePackage, - }); - if (stopped !== null) - check( - "force-stop completes the dynamic-analysis run", - stopped.package_name === probePackage, - ); - const removed = await run("uninstall_adb_package", { - serial, - package: probePackage, - }); - if (removed !== null) - check( - "uninstall completes the lifecycle", - removed.package_name === probePackage, - ); - const afterRemoval = await run("resolve_adb_packages", { - serial, - query: "rea.adb.probe", - }); - if (afterRemoval !== null) - check( - "resolve reports zero matches after removal", - afterRemoval.matches.length === 0, - ); - } - } } finally { await rm(transferRoot, { recursive: true, force: true }); } @@ -459,7 +388,7 @@ if (failures > 0) { } function parseArguments(argv) { - const parsed = { help: false, serial: null, pull: null, installApk: null }; + const parsed = { help: false, serial: null, pull: null }; for (let index = 0; index < argv.length; index += 1) { const argument = argv[index]; if (argument === "--help" || argument === "-h") parsed.help = true; @@ -470,8 +399,10 @@ function parseArguments(argv) { parsed.pull = argv[index + 1] ?? null; index += 1; } else if (argument === "--install-apk") { - parsed.installApk = argv[index + 1] ?? null; - index += 1; + process.stderr.write( + "--install-apk is unavailable: APK identity and device cleanup ownership cannot be established safely\n", + ); + process.exit(1); } else { process.stderr.write(`unknown argument: ${argument}\n`); process.exit(2); diff --git a/scripts/verify-bun-standalone.mjs b/scripts/verify-bun-standalone.mjs index 19d8e0024c..82b6f7edda 100644 --- a/scripts/verify-bun-standalone.mjs +++ b/scripts/verify-bun-standalone.mjs @@ -8,7 +8,7 @@ import { withVerifierCleanup } from "./lib/verifier-cleanup.mjs"; import { createVerifierCommands } from "./lib/verify-commands.mjs"; const root = await PrivateRuntimeRoot.create({ prefix: "rea-bun-verifier-" }); -const commands = await createVerifierCommands(); +let commands; const run = (command, args, options = {}) => commands.exec(command, args, { timeout: 120_000, @@ -19,6 +19,7 @@ const run = (command, args, options = {}) => // only after every owned command family has settled. let commandsClosed = false; await withVerifierCleanup(async () => { + commands = await createVerifierCommands(); const source = join(root.path, "hello.ts"); await writeFile(source, 'console.log("REA_BUN_VERIFICATION");\n'); const docker = process.argv.includes("--docker"); @@ -104,7 +105,7 @@ await withVerifierCleanup(async () => { ); }, [ async () => { - await commands.close(); + if (commands !== undefined) await commands.close(); commandsClosed = true; }, async () => { diff --git a/scripts/verify-claude-client.mjs b/scripts/verify-claude-client.mjs index 4ed2a7356d..94a4366bb5 100644 --- a/scripts/verify-claude-client.mjs +++ b/scripts/verify-claude-client.mjs @@ -2,7 +2,7 @@ import assert from "node:assert/strict"; import { createHash } from "node:crypto"; import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { join } from "node:path"; +import { isAbsolute, join } from "node:path"; import { fileURLToPath } from "node:url"; import Ajv2020 from "ajv/dist/2020.js"; import { createCatalogIdentity } from "../dist/catalogIdentity.js"; @@ -15,6 +15,7 @@ import { TOOL_CONTRACTS, toolContract, } from "../dist/contracts/toolContracts.js"; +import { isPathWithinRoot } from "../dist/domain/localPath.js"; import { createVerifierCommands } from "./lib/verify-commands.mjs"; import { withVerifierCleanup } from "./lib/verifier-cleanup.mjs"; import { saveVerifierCommandOutput } from "./lib/verifier-command-output.mjs"; @@ -306,7 +307,9 @@ await withVerifierCleanup(async () => { assert( parsed.plannedActions.every( (a) => - a.target === configPath || a.target.startsWith(skillDirectory), + a.target === configPath || + (isAbsolute(a.target) && + isPathWithinRoot(skillDirectory, a.target)), ), ); } diff --git a/scripts/verify-commandcode-client.mjs b/scripts/verify-commandcode-client.mjs index e88fa3ee9f..70d6f5f565 100644 --- a/scripts/verify-commandcode-client.mjs +++ b/scripts/verify-commandcode-client.mjs @@ -6,6 +6,7 @@ import { isAbsolute, join, relative } from "node:path"; import { fileURLToPath } from "node:url"; import Ajv2020 from "ajv/dist/2020.js"; import { TOOL_CONTRACTS } from "../dist/contracts/toolContracts.js"; +import { isPathWithinRoot } from "../dist/domain/localPath.js"; import { createVerifierCommands } from "./lib/verify-commands.mjs"; import { withVerifierCleanup } from "./lib/verifier-cleanup.mjs"; import { saveVerifierCommandOutput } from "./lib/verifier-command-output.mjs"; @@ -54,6 +55,7 @@ const expectedNames = TOOL_CONTRACTS.map( ).sort(); let clientVersion = null; let expectedSkill; +let skillCallId; let inventoryChecked = false; let catalogSize = null; let skillLoaded = false; @@ -180,10 +182,24 @@ await withVerifierCleanup(async () => { finalSeen = true; return { role: "assistant", content: "REA_CLIENT_COMPATIBILITY_OK" }; } - if (stage === 0) - return call("activate_skill", { name: "reverse-engineer-anything" }); + if (stage === 0) { + const request = call("activate_skill", { + name: "reverse-engineer-anything", + }); + skillCallId = request.tool_calls[0].id; + return request; + } assert( - strings.some((text) => text.includes(expectedSkill)), + skillCallId !== undefined && + expectedSkill !== undefined && + body.messages.some( + (message) => + message.role === "tool" && + message.tool_call_id === skillCallId && + textValues(message.content).some((text) => + text.includes(expectedSkill), + ), + ), "The model must receive the complete activated skill body", ); skillLoaded = true; @@ -322,8 +338,8 @@ await withVerifierCleanup(async () => { result.plannedActions.every( (action) => action.target === configPath || - action.target === skillDirectory || - action.target.startsWith(skillDirectory + "/"), + (isAbsolute(action.target) && + isPathWithinRoot(skillDirectory, action.target)), ), "Refuse setup writes outside the disposable account's configuration and skill", ); @@ -342,9 +358,16 @@ await withVerifierCleanup(async () => { assert.deepEqual(repeated.appliedActions, []); assert.equal(await readFile(configPath, "utf8"), configured); assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original); - expectedSkill = (await readFile(join(skillDirectory, "SKILL.md"), "utf8")) - .replace(/^---\r?\n[\s\S]*?\r?\n---\r?\n/, "") - .trim(); + const installedSkill = await readFile( + join(skillDirectory, "SKILL.md"), + "utf8", + ); + const frontmatter = /^---\r?\n[\s\S]*?\r?\n---(?:\r?\n|$)/u.exec( + installedSkill, + ); + assert(frontmatter, "Installed skill must have authored frontmatter"); + expectedSkill = installedSkill.slice(frontmatter[0].length).trim(); + assert(expectedSkill.length > 0, "Installed skill body must not be empty"); const response = await saveVerifierCommandOutput( () => commands.exec( diff --git a/scripts/verify-copilot-client.mjs b/scripts/verify-copilot-client.mjs index acf9f9eba3..3f61f89d2a 100644 --- a/scripts/verify-copilot-client.mjs +++ b/scripts/verify-copilot-client.mjs @@ -12,6 +12,7 @@ import { import { advertisedInputJsonSchema } from "../dist/contracts/toolSchemaMetadata.js"; import { presentAdvertisedInputSchema } from "../dist/contracts/compactInputSchema.js"; import { compactPresentationForMcpInputSchemaProfile } from "../dist/config/mcpInputSchemaProfile.js"; +import { isPathWithinRoot } from "../dist/domain/localPath.js"; import { createVerifierCommands } from "./lib/verify-commands.mjs"; import { withVerifierCleanup } from "./lib/verifier-cleanup.mjs"; import { saveVerifierCommandOutput } from "./lib/verifier-command-output.mjs"; @@ -300,8 +301,8 @@ await withVerifierCleanup(async () => { parsed.plannedActions.every( (a) => a.target === configPath || - a.target === skillDirectory || - a.target.startsWith(skillDirectory + "/"), + (isAbsolute(a.target) && + isPathWithinRoot(skillDirectory, a.target)), ), ); } diff --git a/scripts/verify-deepseek-client.mjs b/scripts/verify-deepseek-client.mjs index 9b800422f9..1d960a0ec0 100644 --- a/scripts/verify-deepseek-client.mjs +++ b/scripts/verify-deepseek-client.mjs @@ -56,6 +56,7 @@ let catalog; let patternCount = 0; let skillRequested = false; let skillLoaded = false; +let expectedSkillBody; let evidenceSeen = false; let finalSeen = false; let artifactDigest; @@ -169,11 +170,34 @@ await withVerifierCleanup(async () => { const toolMessages = (body.messages ?? []).filter( (message) => message.role === "tool", ); - const serialized = JSON.stringify(toolMessages); - skillLoaded ||= - serialized.includes( - '', - ) && serialized.includes(dirname(skillPath)); + skillLoaded ||= toolMessages.some((message) => { + if ( + typeof message.content !== "string" || + expectedSkillBody === undefined + ) + return false; + const open = + '\n\n'; + const start = message.content.indexOf(open); + if (start < 0) return false; + const resourcesStart = start + open.length; + const instructions = `\n\n\n\n${expectedSkillBody}\n\n`; + const resourcesEnd = message.content.indexOf( + instructions, + resourcesStart, + ); + if (resourcesEnd < 0) return false; + const expectedPath = dirname(skillPath) + .replaceAll("&", "&") + .replaceAll("<", "<") + .replaceAll(">", ">"); + const resources = message.content.slice(resourcesStart, resourcesEnd); + return ( + !resources.includes("") && + !resources.includes("") && + resources.includes(expectedPath) + ); + }); for (const message of toolMessages) { const found = findClientResult( message.content, @@ -281,11 +305,15 @@ await withVerifierCleanup(async () => { const configured = await readFile(configPath, "utf8"); assert.deepEqual(withoutRea(configured), withoutRea(original)); assert(configured.includes("!!js process.cwd()")); - assert( - (await readFile(skillPath, "utf8")).includes( - "name: reverse-engineer-anything", - ), + const installedSkill = await readFile(skillPath, "utf8"); + assert(installedSkill.includes("name: reverse-engineer-anything")); + const frontmatter = /^---\r?\n[\s\S]*?^---(?:\r?\n|$)/mu.exec( + installedSkill, ); + assert(frontmatter, "Installed skill must have its authored frontmatter"); + expectedSkillBody = installedSkill.slice(frontmatter[0].length).trim(); + assert(expectedSkillBody.length > 0); + lifecycle.push("setup"); const repeated = await runRea( ["setup", "--client", "dsh", "--yes"], diff --git a/scripts/verify-flutter.mjs b/scripts/verify-flutter.mjs index b20d76d72b..0425711fda 100644 --- a/scripts/verify-flutter.mjs +++ b/scripts/verify-flutter.mjs @@ -104,6 +104,10 @@ await withVerifierCleanup( } // The same checks apply to the default selection and each remaining ABI. const preferred = await run("inspect_dart_aot", { path: apk }); + assert.ok( + abis.some((abi) => abi.abi === preferred.abi), + "Default AOT selection must be one of the identified ABIs", + ); for (const abi of abis) { const aot = abi.abi === preferred.abi diff --git a/scripts/verify-frida.mjs b/scripts/verify-frida.mjs index e42bd4a805..396ddf4f88 100644 --- a/scripts/verify-frida.mjs +++ b/scripts/verify-frida.mjs @@ -2,7 +2,7 @@ import { basename } from "node:path"; import { FridaInstrumentationService } from "../dist/application/frida/FridaInstrumentationService.js"; import { FridaInstrumentationManager } from "../dist/frida/FridaInstrumentationManager.js"; -import { isValidFridaRemoteAddress } from "../dist/contracts/fridaRemoteAddress.js"; +import { isValidFridaRemoteAddress } from "../dist/domain/fridaRemoteAddress.js"; import { withVerifierCleanup } from "./lib/verifier-cleanup.mjs"; const options = parseArguments(process.argv.slice(2)); @@ -80,6 +80,10 @@ try { !Array.isArray(observed.messages) ) throw new Error("Frida returned no message observation"); + if (!Number.isSafeInteger(observed.pid) || observed.pid <= 0) + throw new Error("Frida returned no valid target PID"); + if (remote !== undefined && observed.pid !== options.pid) + throw new Error("Frida observed a different target PID"); const messageObserved = observed.messages.some( (message) => typeof message === "object" && diff --git a/scripts/verify-gemini-client.mjs b/scripts/verify-gemini-client.mjs index 9837ad53dd..eab017d6b4 100644 --- a/scripts/verify-gemini-client.mjs +++ b/scripts/verify-gemini-client.mjs @@ -2,13 +2,14 @@ import assert from "node:assert/strict"; import { createHash } from "node:crypto"; import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { join, resolve, sep } from "node:path"; +import { isAbsolute, join, relative, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import Ajv2020 from "ajv/dist/2020.js"; import { TOOL_CONTRACTS, toolContract, } from "../dist/contracts/toolContracts.js"; +import { isPathWithinRoot } from "../dist/domain/localPath.js"; import { createVerifierCommands } from "./lib/verify-commands.mjs"; import { saveVerifierCommandOutput } from "./lib/verifier-command-output.mjs"; import { withVerifierCleanup } from "./lib/verifier-cleanup.mjs"; @@ -65,6 +66,7 @@ let fixture; let passed = false; let evidenceSeen = false; let skillLoaded = false; +let expectedSkillBody; let skillRequested = false; let finalSeen = false; let catalogSize = null; @@ -150,13 +152,21 @@ await withVerifierCleanup(async () => { if (args.includes("--dry-run")) { assert(parsed.plannedActions.some((a) => a.target === configPath)); assert( - parsed.plannedActions.some((a) => - a.target.startsWith(join(profile, ".agents", "skills") + sep), - ), + parsed.plannedActions.some((a) => { + const skills = join(profile, ".agents", "skills"); + return ( + isAbsolute(a.target) && + relative(skills, a.target) !== "" && + isPathWithinRoot(skills, a.target) + ); + }), ); assert( - parsed.plannedActions.every((a) => - a.target.startsWith(profile + sep), + parsed.plannedActions.every( + (a) => + isAbsolute(a.target) && + relative(profile, a.target) !== "" && + isPathWithinRoot(profile, a.target), ), ); } @@ -189,11 +199,15 @@ await withVerifierCleanup(async () => { assert.deepEqual(JSON.parse(repeated.stdout).appliedActions, []); assert.equal(await readFile(configPath, "utf8"), configured); assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original); - assert( - (await readFile(skillPath, "utf8")).includes( - "name: reverse-engineer-anything", - ), + const installedSkill = await readFile(skillPath, "utf8"); + assert(installedSkill.includes("name: reverse-engineer-anything")); + const frontmatter = /^---\r?\n([\s\S]*?)\r?\n---(?:\r?\n([\s\S]*))?/u.exec( + installedSkill, ); + assert(frontmatter, "Installed skill must have authored frontmatter"); + expectedSkillBody = (frontmatter[2] ?? "").trim(); + assert(expectedSkillBody.length > 0); + const target = join(lab, "fixture α测试"); await mkdir(target); await writeFile(join(target, "app.js"), CLIENT_JAVASCRIPT_FIXTURE_SOURCE); @@ -268,16 +282,32 @@ await withVerifierCleanup(async () => { .flatMap((c) => c.parts ?? []) .filter((p) => p.functionResponse) .map((p) => p.functionResponse); - skillLoaded ||= responses.some( - (r) => - r.name === "activate_skill" && - r.response?.output?.includes( - ``, - ) && - r.response.output.includes( + skillLoaded ||= responses.some((response) => { + const output = response.response?.output; + if ( + response.name !== "activate_skill" || + typeof output !== "string" || + expectedSkillBody === undefined + ) + return false; + const start = `\n \n ${expectedSkillBody}\n \n\n \n `; + const at = output.indexOf(start); + if (at < 0) return false; + const resourcesStart = at + start.length; + const resourcesEnd = output.indexOf( + "\n \n", + resourcesStart, + ); + if (resourcesEnd < 0) return false; + const resources = output.slice(resourcesStart, resourcesEnd); + return ( + !resources.includes("") && + !resources.includes("") && + resources.includes( join(profile, ".agents", "skills", "reverse-engineer-anything"), - ), - ); + ) + ); + }); const observed = findClientResult( responses, isClientJavaScriptEvidence, diff --git a/scripts/verify-grok-client.mjs b/scripts/verify-grok-client.mjs index 753c8ae998..0ea587e45f 100644 --- a/scripts/verify-grok-client.mjs +++ b/scripts/verify-grok-client.mjs @@ -6,6 +6,7 @@ import { isAbsolute, join, relative } from "node:path"; import { fileURLToPath } from "node:url"; import Ajv2020 from "ajv/dist/2020.js"; import { TOOL_CONTRACTS } from "../dist/contracts/toolContracts.js"; +import { isPathWithinRoot } from "../dist/domain/localPath.js"; import { createVerifierCommands } from "./lib/verify-commands.mjs"; import { withVerifierCleanup } from "./lib/verifier-cleanup.mjs"; import { saveVerifierCommandOutput } from "./lib/verifier-command-output.mjs"; @@ -402,8 +403,8 @@ enabled = false parsed.plannedActions.every( (action) => action.target === configPath || - action.target === skillDirectory || - action.target.startsWith(skillDirectory + "/"), + (isAbsolute(action.target) && + isPathWithinRoot(skillDirectory, action.target)), ), "Refuse writes outside the owned profile/skill", ); diff --git a/scripts/verify-http3-network-captures.mjs b/scripts/verify-http3-network-captures.mjs index 55997a95dc..dbf9dedba0 100644 --- a/scripts/verify-http3-network-captures.mjs +++ b/scripts/verify-http3-network-captures.mjs @@ -373,6 +373,7 @@ try { } const report = { run_id: run.run_id, + input_mode: inspectExisting ? "existing-captures" : "producer-attempt", status: primaryError || cleanupFailures.length ? "failed" : "passed", finished_at: new Date().toISOString(), ...(primaryError @@ -386,8 +387,13 @@ const report = { verifier, producer_audit: producerAudit, scope: - "Real Linux reverse HTTP/3, mitmproxy 12.2.3, aioquic 1.2.0 and guest cURL", + "Linux reverse HTTP/3 capture verification for mitmproxy 12.2.3, aioquic 1.2.0 and guest cURL", limitations: [ + ...(inspectExisting + ? [ + "Existing capture producer provenance was not authenticated by this attempt.", + ] + : []), "No local/WireGuard, browser, QUIC v2, migration, 0-RTT, mTLS or replay claim.", "No HAR, interruption, certificate-failure or body-completeness coverage in this profile.", ], diff --git a/scripts/verify-il2cpp-metadata.mjs b/scripts/verify-il2cpp-metadata.mjs index 5bc7d1d6f8..362208cd09 100644 --- a/scripts/verify-il2cpp-metadata.mjs +++ b/scripts/verify-il2cpp-metadata.mjs @@ -17,7 +17,7 @@ import { hashVerifierFile } from "./lib/verifier-file-resources.mjs"; const args = process.argv.slice(2); if (args.includes("--help") || args.includes("-h")) { console.log( - "Usage: npm run verify:il2cpp:metadata -- [--metadata PATH]\nPATH is a caller-owned, unencrypted global-metadata.dat (version 24, 27, 29, 31 or 39). Without --metadata, only the advertised contract is verified.", + "Usage: npm run verify:il2cpp:metadata -- [--metadata PATH]\nPATH is a caller-owned, unencrypted global-metadata.dat (version 24, 27, 29, 31 or 39). Without --metadata, only the MCP tool name is checked.", ); process.exit(0); } @@ -178,4 +178,8 @@ await withVerifierCleanup(async () => { ); } }, [() => client.close(), () => transport.close(), () => resources.close()]); -console.log("All asserted IL2CPP metadata CLI/MCP checks passed"); +console.log( + path === null + ? "IL2CPP metadata MCP tool presence check passed" + : "All asserted IL2CPP metadata CLI/MCP checks passed", +); diff --git a/scripts/verify-jeb.mjs b/scripts/verify-jeb.mjs index 812917d73c..6abd439a5d 100644 --- a/scripts/verify-jeb.mjs +++ b/scripts/verify-jeb.mjs @@ -68,7 +68,9 @@ try { }); if (!decompiled.ok) throw new Error(decompiled.error.message); const text = decompiled.value.normalized_result.text; - if (!text.includes(target.marker)) + if (text.trim().length === 0) + throw new Error("Decompiled pseudo-code was empty"); + if (target.marker !== undefined && !text.includes(target.marker)) throw new Error( "Decompiled pseudo-code did not contain the fixture marker", ); @@ -83,7 +85,7 @@ try { open_project_observed: openRefusal === null, ...(openRefusal === null ? {} : { open_project_refusal: openRefusal }), unit_count: units.value.normalized_result.units.length, - decompiled_marker_observed: true, + decompiled_marker_observed: target.marker !== undefined, }; }, [ async () => { @@ -110,7 +112,7 @@ async function resolveTarget(selected, directory) { return { path: "/engine-side/target", itemAddress: selected.itemAddress, - marker: selected.marker ?? "", + marker: selected.marker, }; await writeFile( join(directory, "Hello.java"), @@ -153,6 +155,8 @@ function parseArguments(args) { values.set(flag, value); index += 1; } + if (values.get("--marker") === "") + throw new Error("--marker must not be empty"); return { help: false, ...(values.has("--item-address") diff --git a/scripts/verify-omp-client.mjs b/scripts/verify-omp-client.mjs index bb2599e828..0cd3d0227b 100644 --- a/scripts/verify-omp-client.mjs +++ b/scripts/verify-omp-client.mjs @@ -2,9 +2,10 @@ import assert from "node:assert/strict"; import { createHash } from "node:crypto"; import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises"; import { homedir, tmpdir } from "node:os"; -import { join, relative } from "node:path"; +import { isAbsolute, join, relative } from "node:path"; import { fileURLToPath } from "node:url"; import { TOOL_CONTRACTS } from "../dist/contracts/toolContracts.js"; +import { isPathWithinRoot } from "../dist/domain/localPath.js"; import { createVerifierCommands } from "./lib/verify-commands.mjs"; import { withVerifierCleanup } from "./lib/verifier-cleanup.mjs"; import { saveVerifierCommandOutput } from "./lib/verifier-command-output.mjs"; @@ -314,8 +315,8 @@ await withVerifierCleanup(async () => { parsed.plannedActions.every( (action) => action.target === configPath || - action.target === skillDirectory || - action.target.startsWith(skillDirectory + "/"), + (isAbsolute(action.target) && + isPathWithinRoot(skillDirectory, action.target)), ), "Refuse setup writes outside the owned config/skill", ); diff --git a/scripts/verify-opencode-client.mjs b/scripts/verify-opencode-client.mjs index acf39a2494..8877b4664d 100644 --- a/scripts/verify-opencode-client.mjs +++ b/scripts/verify-opencode-client.mjs @@ -2,10 +2,11 @@ import assert from "node:assert/strict"; import { createHash } from "node:crypto"; import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { isAbsolute, join, relative, sep } from "node:path"; +import { isAbsolute, join } from "node:path"; import { fileURLToPath } from "node:url"; import Ajv2020 from "ajv/dist/2020.js"; import { TOOL_CONTRACTS } from "../dist/contracts/toolContracts.js"; +import { isPathWithinRoot } from "../dist/domain/localPath.js"; import { createVerifierCommands } from "./lib/verify-commands.mjs"; import { saveVerifierCommandOutput } from "./lib/verifier-command-output.mjs"; import { withVerifierCleanup } from "./lib/verifier-cleanup.mjs"; @@ -49,19 +50,15 @@ const skillDirectory = join( "skills", "reverse-engineer-anything", ); -const isManagedSkillTarget = (target) => { - const value = relative(skillDirectory, target); - return ( - value === "" || - (value !== ".." && !value.startsWith(`..${sep}`) && !isAbsolute(value)) - ); -}; assert.equal( - isManagedSkillTarget(`${skillDirectory}-sibling/AGENTS.md`), + isPathWithinRoot(skillDirectory, `${skillDirectory}-sibling/AGENTS.md`), false, ); assert.equal( - isManagedSkillTarget(join(skillDirectory, "..", "outside", "AGENTS.md")), + isPathWithinRoot( + skillDirectory, + join(skillDirectory, "..", "outside", "AGENTS.md"), + ), false, ); let clientVersion = null; @@ -271,7 +268,10 @@ await withVerifierCleanup(async () => { ); assert( parsed.plannedActions.every( - (a) => a.target === configPath || isManagedSkillTarget(a.target), + (a) => + a.target === configPath || + (isAbsolute(a.target) && + isPathWithinRoot(skillDirectory, a.target)), ), ); } diff --git a/scripts/verify-package-update.d.mts b/scripts/verify-package-update.d.mts new file mode 100644 index 0000000000..72133a6dfb --- /dev/null +++ b/scripts/verify-package-update.d.mts @@ -0,0 +1,14 @@ +import type { PackageVerifierCommands } from "./lib/verify-package-commands.mjs"; + +export function verifyPackageUpdate(options: { + readonly prefix: string; + readonly packageRoot: string; + readonly tarball: string; + readonly workspace: string; + readonly environment: NodeJS.ProcessEnv; + readonly commands: Pick; +}): Promise<{ + readonly installed_version: string; + readonly exact_release: true; + readonly maintenance_read_only: true; +}>; diff --git a/scripts/verify-package-update.mjs b/scripts/verify-package-update.mjs index 09295a0b6f..575d55666f 100644 --- a/scripts/verify-package-update.mjs +++ b/scripts/verify-package-update.mjs @@ -35,13 +35,20 @@ export async function verifyPackageUpdate({ response.destroy(); return; } - response.writeHead(500); + response.writeHead(500, { + "Content-Type": "text/plain; charset=utf-8", + "X-Content-Type-Options": "nosniff", + }); response.end(String(error)); }); }); let registry; async function serveRegistry(request, response) { - const pathname = new URL(request.url, registry).pathname; + // A leading // belongs to an origin-form HTTP path, not a URL authority. + const requested = new URL( + request.url.startsWith("/") ? `${registry}${request.url}` : request.url, + ); + const { pathname } = requested; if (pathname === "/rea-agents") { response.setHeader("Content-Type", "application/json"); response.end( @@ -65,7 +72,10 @@ export async function verifyPackageUpdate({ response.setHeader("Content-Type", "application/json"); response.end("{}"); } else { - const upstream = await fetch(`https://registry.npmjs.org${request.url}`); + const upstreamUrl = new URL("https://registry.npmjs.org"); + upstreamUrl.pathname = pathname; + upstreamUrl.search = requested.search; + const upstream = await fetch(upstreamUrl); const body = Buffer.from(await upstream.arrayBuffer()); response.writeHead(upstream.status, { "Content-Type": diff --git a/scripts/verify-package.mjs b/scripts/verify-package.mjs index 847a092667..0e7bbf41a1 100644 --- a/scripts/verify-package.mjs +++ b/scripts/verify-package.mjs @@ -266,7 +266,6 @@ await withVerifierCleanup(async () => { extractedPluginMcp: true, targetFree: true, targetLifecycle: true, - boundedRegexBridge: true, update, mcpStartup, mcpModuleLoading, diff --git a/scripts/verify-qwen-client.mjs b/scripts/verify-qwen-client.mjs index a47e0d6373..6011c6d711 100644 --- a/scripts/verify-qwen-client.mjs +++ b/scripts/verify-qwen-client.mjs @@ -9,9 +9,10 @@ import { symlink, } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { join, resolve, sep } from "node:path"; +import { isAbsolute, join, relative, resolve, sep } from "node:path"; import { fileURLToPath } from "node:url"; import { createHash } from "node:crypto"; +import { isPathWithinRoot } from "../dist/domain/localPath.js"; import { createVerifierCommands } from "./lib/verify-commands.mjs"; import { saveVerifierCommandOutput } from "./lib/verifier-command-output.mjs"; import { withVerifierCleanup } from "./lib/verifier-cleanup.mjs"; @@ -58,9 +59,10 @@ let clientVersion = null; let fixture; let passed = false; let evidenceSeen = false; -let skillSeen = false; let finalSeen = false; let skillRequested = false; +let skillCallId; +let expectedSkillBody; let searchCallId; let analysisCallId; let artifactReadByAgent = false; @@ -133,8 +135,11 @@ await withVerifierCleanup(async () => { assert( result.plannedActions.every( (a) => - a.target.startsWith(profile + sep) || - a.target.startsWith(sharedSkills + sep), + isAbsolute(a.target) && + ((relative(profile, a.target) !== "" && + isPathWithinRoot(profile, a.target)) || + (relative(sharedSkills, a.target) !== "" && + isPathWithinRoot(sharedSkills, a.target))), ), ); await writeFile( @@ -162,6 +167,20 @@ await withVerifierCleanup(async () => { command: "unrelated-server", disabled: true, }); + const skillFile = join( + sharedSkills, + "reverse-engineer-anything", + "SKILL.md", + ); + const normalizedSkill = (await readFile(skillFile, "utf8")) + .replace(/^\uFEFF/u, "") + .replace(/\r\n|\r/gu, "\n"); + const frontmatter = /^---\n([\s\S]*?)\n---(?:\n|$)([\s\S]*)$/u.exec( + normalizedSkill, + ); + assert(frontmatter, "Installed skill must have authored frontmatter"); + expectedSkillBody = frontmatter[2].trim(); + assert(expectedSkillBody.length > 0); clientPath = baseEnvironment.PATH; if (reducedPath) { assert.equal( @@ -200,9 +219,6 @@ await withVerifierCleanup(async () => { directory: lab, prefix: `${client}-${mode}`, onRequest: async ({ body, tools, requestIndex }) => { - skillSeen ||= JSON.stringify(body.messages ?? []).includes( - "reverse-engineer-anything", - ); const messages = (body.messages ?? []).map((message) => ({ role: message.role, callId: message.tool_call_id, @@ -213,14 +229,14 @@ await withVerifierCleanup(async () => { .map((part) => part.text ?? "") .join("\n"), })); - const messageText = messages.map((message) => message.text).join("\n"); const toolResponseText = (callId) => messages.find( (message) => message.role === "tool" && message.callId === callId, )?.text ?? ""; - selectedSkillLoaded ||= messageText.includes( - join(sharedSkills, "reverse-engineer-anything"), - ); + const expectedSkill = `Base directory for this skill: ${join(sharedSkills, "reverse-engineer-anything")}\nImportant: ALWAYS resolve absolute paths from this base directory when working with skills.\n\n${expectedSkillBody}\n`; + selectedSkillLoaded ||= + skillCallId !== undefined && + toolResponseText(skillCallId).includes(expectedSkill); const toolText = JSON.stringify( (body.messages ?? []).filter((m) => m.role === "tool"), ); @@ -298,6 +314,7 @@ await withVerifierCleanup(async () => { } else if (skill && !skillRequested) { skillRequested = true; message = call(skill.name, { skill: "reverse-engineer-anything" }); + skillCallId = message.tool_calls[0].id; } else if (persisted && shell) { artifactPath = persisted[1]; assert( diff --git a/scripts/verify-real-cutter.mjs b/scripts/verify-real-cutter.mjs index 705bd902d2..aa7ee19bab 100644 --- a/scripts/verify-real-cutter.mjs +++ b/scripts/verify-real-cutter.mjs @@ -32,8 +32,6 @@ const runCli = async (...arguments_) => { return JSON.parse(stdout); }; -const projectRoot = await mkdtemp(join(tmpdir(), "rea-cutter-smoke-")); -const projectPath = join(projectRoot, "smoke.rzdb"); const rizinQuote = (value) => `"${value.replaceAll("\\", "\\\\").replaceAll('"', '\\"')}"`; const unavailableAnalysis = { @@ -44,16 +42,21 @@ const unavailableAnalysis = { }, }; const expectedPlatform = { + linux: "linux", Linux: "linux", macOS: "darwin", Windows: "win32", }[process.env.EXPECTED_PLATFORM]; -if (expectedPlatform !== undefined) +if (process.env.EXPECTED_PLATFORM !== undefined) { + assert.ok(expectedPlatform, "Unrecognized Cutter runner platform."); assert.equal( process.platform, expectedPlatform, "Cutter runner did not match its declared platform.", ); +} +const projectRoot = await mkdtemp(join(tmpdir(), "rea-cutter-smoke-")); +const projectPath = join(projectRoot, "smoke.rzdb"); try { const discovery = await runCli("list-cutter-sessions"); assert.equal( diff --git a/scripts/verify-real-electron.mjs b/scripts/verify-real-electron.mjs index 45306de604..192b9d3805 100644 --- a/scripts/verify-real-electron.mjs +++ b/scripts/verify-real-electron.mjs @@ -79,7 +79,11 @@ const output = { windows: capture.windows, verified: capture.coverage.status === "partial_attach" && - capture.actions.every(({ status }) => status === "completed") && + capture.actions.length === input.actions.length && + capture.actions.every( + ({ step_id, status }, index) => + step_id === input.actions[index].step_id && status === "completed", + ) && capture.actions.some( ({ kind, window_index }) => kind === "renderer-reload" && window_index === 1, diff --git a/scripts/verify-real-ghidra-assertions.mjs b/scripts/verify-real-ghidra-assertions.mjs index f79f287df9..c5fed71b0a 100644 --- a/scripts/verify-real-ghidra-assertions.mjs +++ b/scripts/verify-real-ghidra-assertions.mjs @@ -1,6 +1,6 @@ import { hasTypedSwitchEvidence } from "./verify-real-ghidra-switch-assertions.mjs"; -import { access } from "node:fs/promises"; +import { lstat } from "node:fs/promises"; import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js"; import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraReleasePolicy.js"; @@ -437,10 +437,11 @@ export function summary(observed) { async function exists(path) { try { - await access(path); + await lstat(path); return true; - } catch { - return false; + } catch (cause) { + if (cause?.code === "ENOENT") return false; + throw cause; } } diff --git a/scripts/verify-real-ghidra-com.mjs b/scripts/verify-real-ghidra-com.mjs index 2513d493ac..8eb48b1669 100644 --- a/scripts/verify-real-ghidra-com.mjs +++ b/scripts/verify-real-ghidra-com.mjs @@ -3,6 +3,7 @@ import assert from "node:assert/strict"; import { requireMcpLifecycleResult, requireMcpOutput, + requireMcpToolError, } from "./lib/mcp-verifier-results.mjs"; import { execFile } from "node:child_process"; import { createHash } from "node:crypto"; @@ -183,7 +184,8 @@ try { item.includes("did not attempt a decode"), ), ); - const advertised = (await client.listTools()).tools.find( + const tools = (await client.listTools()).tools; + const advertised = tools.find( (tool) => tool.name === "inspect_native_instruction", ); assert.ok(advertised?.outputSchema); @@ -198,6 +200,34 @@ try { nativeInstructionSchema.safeParse({ ...unlisted, status: "undecodable" }) .success, ); + const contextDefinition = tools.find( + (tool) => tool.name === "inspect_address_context", + ); + assert.ok(contextDefinition?.outputSchema); + const documents = await call("list_documents"); + assert.equal(documents.length, 1); + const document = documents[0]; + for (const input of [ + { address: fixture.entry, document: `${document}-not-a-program` }, + { address: "not-an-address", document }, + ]) { + const direct = requireMcpToolError( + await client.callTool({ name: "address_name", arguments: input }), + ); + const response = await client.callTool( + { name: "inspect_address_context", arguments: input }, + { timeout: 240000, toolDefinition: contextDefinition }, + ); + const context = requireMcpLifecycleResult( + response, + "inspect_address_context", + ); + assert.equal(context.name.state, "unavailable"); + assert.equal(context.name.error.code, "invalid_request"); + assert.deepEqual(context.name.error.details.issues, direct.details.issues); + assert.equal(context.name.reason, direct.message); + assert.equal(context.name.remediation, direct.remediation.action); + } assert.equal( (await call("resolve_native_call_targets", { address: unlistedAddress })) .status, @@ -357,6 +387,7 @@ try { clear_comments: true, cli_mcp_parity: true, }, + context_input_errors_preserved: true, original_unchanged: true, address_truncation_rejected: true, limitations: image.limitations, diff --git a/scripts/verify-real-ghidra-dos.mjs b/scripts/verify-real-ghidra-dos.mjs index 9c6590a611..7bad32a135 100644 --- a/scripts/verify-real-ghidra-dos.mjs +++ b/scripts/verify-real-ghidra-dos.mjs @@ -125,6 +125,13 @@ try { assert.equal(image.module_bytes, fixture.module_bytes); assert.equal(image.entry.linear_address, expected.entry); assert.ok(image.checks.every((check) => check.matched)); + assert.ok(image.observations.relocations.length > 0); + assert.ok( + image.observations.relocations.every( + ({ symbol }) => symbol === null || typeof symbol === "string", + ), + "Each real Ghidra relocation must record its symbol observation", + ); const cliImage = await cliEvidence("inspect-native-load-image"); // Snapshot filenames are session-specific. Every measured byte/range/check is stable. const stableImage = (value) => ({ diff --git a/scripts/verify-real-ghidra-function.mjs b/scripts/verify-real-ghidra-function.mjs index dbf4e77b7f..1e4f2c2cf9 100644 --- a/scripts/verify-real-ghidra-function.mjs +++ b/scripts/verify-real-ghidra-function.mjs @@ -415,7 +415,7 @@ async function verifyRequestCancellationAndTimeout(client, entry) { } async function verifyConcurrentRequests(client, entry, leaf) { - const concurrent = await Promise.all([ + await Promise.all([ functionCall(client, "procedure_info", { document: null, procedure: entry.address, @@ -425,9 +425,7 @@ async function verifyConcurrentRequests(client, entry, leaf) { address: leaf.address, }), ]); - if (concurrent.length !== 2) - throw new Error("Ghidra serial request queue lost a concurrent result"); - return "serialized-two-results"; + return "two-concurrent-requests-completed"; } /** Prove dense jump-table completeness on the optimized jump-table fixture. */ diff --git a/scripts/verify-real-ghidra-nativeaot.mjs b/scripts/verify-real-ghidra-nativeaot.mjs index fb5064613f..6b0bb5851a 100644 --- a/scripts/verify-real-ghidra-nativeaot.mjs +++ b/scripts/verify-real-ghidra-nativeaot.mjs @@ -375,8 +375,13 @@ try { JSON.stringify(report, null, 2) + "\n", ); } + const outcome = report.typed_failure + ? "MCP refusal" + : mode === "default-native" + ? "native load image without metadata recovery" + : "MCP metadata"; console.log( - `PASS real Ghidra NativeAOT ${oracle.runtime}/${mode}: MCP metadata, source identity and owned cleanup${report?.compute ? ", independent type/slot oracle and pseudocode" : ""}${report?.cli_metadata ? ", equivalent CLI" : ""}`, + `PASS real Ghidra NativeAOT ${oracle.runtime}/${mode}: ${outcome}, source unchanged and runtime directory empty${report?.compute ? ", independent type/slot oracle and pseudocode" : ""}${report?.cli_metadata ? ", equivalent CLI" : ""}`, ); } catch (error) { console.error(stderr); diff --git a/scripts/verify-real-ghidra.mjs b/scripts/verify-real-ghidra.mjs index 2d4aa94e9b..a32a005b08 100644 --- a/scripts/verify-real-ghidra.mjs +++ b/scripts/verify-real-ghidra.mjs @@ -32,6 +32,7 @@ import { } from "./verify-real-ghidra-inventory.mjs"; import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs"; import { hasTypedSwitchEvidence } from "./verify-real-ghidra-switch-assertions.mjs"; +import { verifyGhidraRelocationSymbols } from "./lib/real-ghidra-relocation-symbols.mjs"; import { verifyDenseDefaultReturn, @@ -281,6 +282,14 @@ try { } await writeFile(malformedPath, Buffer.from("not-a-binary\n", "utf8")); + const relocationSymbols = crossFormat + ? await verifyGhidraRelocationSymbols({ + clang, + lld, + env: process.env, + }) + : null; + const debug = await verifyTarget(debugPath, "debug", expectedNativeTarget); assertDebugFixture(debug); const layout = await verifyTarget( @@ -331,6 +340,7 @@ try { ? (crossResults[0]?.native_api_cli ?? null) : jumpTable.native_api_cli, native_value_e2e: debug.native_values, + relocation_symbols: relocationSymbols, custom_target: custom === null ? null : summary(custom), cleanup: "complete", }; diff --git a/scripts/verify-real-kuna.mjs b/scripts/verify-real-kuna.mjs index b301013c6d..13659746b8 100644 --- a/scripts/verify-real-kuna.mjs +++ b/scripts/verify-real-kuna.mjs @@ -35,6 +35,7 @@ import { verifyKunaStrings, } from "./lib/kuna-string-verifier.mjs"; import { nativeKunaIncomingSites } from "./lib/kuna-native-reference-verifier.mjs"; +import { MULTI_MODE_REMEDIATION } from "../dist/application/CapabilityInventory.js"; import { assertVerifierCleanupComplete, completeVerifierRun, @@ -298,6 +299,36 @@ const verifyKuna = async ( const answers = []; let coldFailure; try { + const status = await cold.call("binary_session", { detail: "complete" }); + if (status.value.format === "elf") { + const availability = status.value.tool_availability; + const dispatch = availability.find( + ({ name }) => name === "inspect_native_dispatch_metadata", + ); + const trace = availability.find( + ({ name }) => name === "trace_native_ui_action", + ); + assert.equal(dispatch?.available, false); + assert.equal( + trace?.modes?.find(({ name }) => name === "native_seed_call_trace") + ?.available, + false, + ); + if (process.platform === "darwin") { + const authored = trace?.modes?.find( + ({ name }) => name === "authored_ui_dispatch", + ); + const nativeSeed = trace?.modes?.find( + ({ name }) => name === "native_seed_call_trace", + ); + assert.equal(trace?.remediation, MULTI_MODE_REMEDIATION); + assert.notEqual(authored?.remediation, nativeSeed?.remediation); + assert.ok( + authored?.missing_operations.includes("decode_interface_builder"), + ); + assert.ok(nativeSeed?.missing_operations.includes("list_names")); + } + } answers.push({ strings: await verifyKunaStrings({ call: cold.call, diff --git a/scripts/verify-test-discovery.mjs b/scripts/verify-test-discovery.mjs index 46a6aced87..5db6905852 100644 --- a/scripts/verify-test-discovery.mjs +++ b/scripts/verify-test-discovery.mjs @@ -55,7 +55,8 @@ try { if ( report.missing.length || report.unexpected.length || - report.duplicates.length + report.duplicates.length || + report.wrongProjects.length ) { console.error(JSON.stringify(report, null, 2)); process.exitCode = 1; diff --git a/scripts/verify/javascript/application-workflows.mjs b/scripts/verify/javascript/application-workflows.mjs index 725e48a9ab..95b861b039 100644 --- a/scripts/verify/javascript/application-workflows.mjs +++ b/scripts/verify/javascript/application-workflows.mjs @@ -41,10 +41,18 @@ const [leftPath, rightPath] = await Promise.all([ ]); if (leftPath === rightPath) throw new Error("--left and --right must resolve to distinct paths"); -const [left, right] = await Promise.all([ +const analyses = await Promise.allSettled([ analyze(options.left), analyze(options.right), ]); +const failures = analyses.filter((result) => result.status === "rejected"); +if (failures.length === 1) throw failures[0].reason; +if (failures.length > 1) + throw new AggregateError( + failures.map((result) => result.reason), + "Application analyses failed", + ); +const [left, right] = analyses.map((result) => result.value); const comparison = compareApplicationVersionsEvidence({ left, right }); if (!comparison.ok) throw comparison.error; const comparisonResult = comparison.value.result; diff --git a/scripts/verify/javascript/runtime-observation.mjs b/scripts/verify/javascript/runtime-observation.mjs index 8587e5341e..bd1781ca52 100644 --- a/scripts/verify/javascript/runtime-observation.mjs +++ b/scripts/verify/javascript/runtime-observation.mjs @@ -1,8 +1,17 @@ #!/usr/bin/env node -import { readFile, realpath } from "node:fs/promises"; +import { realpath } from "node:fs/promises"; import { parseArgs } from "node:util"; +import { readStableArtifact } from "../../../dist/artifacts/readStableArtifact.js"; +import { ArtifactReaderFailure } from "../../../dist/artifacts/ArtifactReader.js"; +import { ArtifactResourceScope } from "../../../dist/artifacts/ArtifactResourceScope.js"; +import { + ArtifactFileCleanupFailure, + artifactFileCompletion, + artifactFilePrimaryFacts, +} from "../../../dist/artifacts/ArtifactFileOperation.js"; + import { reconcileJavaScriptRuntimeEvidence } from "../../../dist/application/javascript/JavaScriptRuntimeReconciliationService.js"; import { analyzeJavaScriptApplicationWithOwnedScope } from "../../lib/javascript-application-analysis.mjs"; import { @@ -52,9 +61,11 @@ for (const encoded of options.mapping) { const mapping = parseMapping(encoded, staticLayers.length); staticLayers[mapping.layer].runtime_mappings.push(mapping.value); } -const runtimeObservations = ( - await Promise.all(options["runtime-evidence"].map(readEvidenceFile)) -).flat(); +const runtimeObservations = []; +for (const path of options["runtime-evidence"]) { + const observations = await readEvidenceFile(path); + for (const observation of observations) runtimeObservations.push(observation); +} const reconciled = reconcileJavaScriptRuntimeEvidence({ static_layers: staticLayers, runtime_observations: runtimeObservations, @@ -95,10 +106,47 @@ process.stdout.write( ); async function readEvidenceFile(path) { - const bytes = await readFile(await realpath(path)); - if (bytes.byteLength > 64 * 1024 * 1024) - throw new Error(`Runtime Evidence exceeds 64 MiB: ${path}`); - const value = JSON.parse(bytes.toString("utf8")); + const scope = new ArtifactResourceScope(); + let outcome; + try { + outcome = { + ok: true, + value: await readStableArtifact( + await realpath(path), + 64 * 1024 * 1024, + undefined, + undefined, + scope, + ), + }; + } catch (cause) { + outcome = { ok: false, cause }; + } + try { + await scope.close(); + } catch (cleanupCause) { + if (!outcome.ok) + throw new AggregateError( + [outcome.cause, cleanupCause], + "Verifier cleanup failed", + { cause: outcome.cause }, + ); + throw cleanupCause; + } + if (!outcome.ok && outcome.cause instanceof ArtifactFileCleanupFailure) { + const completion = artifactFileCompletion(outcome.cause); + outcome = + completion === undefined + ? { ok: false, cause: artifactFilePrimaryFacts(outcome.cause).cause } + : { ok: true, value: completion.value }; + } + if (!outcome.ok) { + const cause = outcome.cause; + if (cause instanceof ArtifactReaderFailure && cause.reason === "limit") + throw new Error(`Runtime Evidence exceeds 64 MiB: ${path}`, { cause }); + throw cause; + } + const value = JSON.parse(outcome.value.bytes.toString("utf8")); return Array.isArray(value) ? value : [value]; } diff --git a/src/adb/AdbConfiguration.ts b/src/adb/AdbConfiguration.ts index d0056dbc7b..a25bbe0010 100644 --- a/src/adb/AdbConfiguration.ts +++ b/src/adb/AdbConfiguration.ts @@ -23,6 +23,12 @@ import type { JsonValue } from "../domain/jsonValue.js"; export const ADB_BINARY_ENV = "REA_ADB_PATH"; const VERSION_TIMEOUT_MS = 10_000; + +const ADB_VERSION_REMEDIATION = `Set ${ADB_BINARY_ENV} to an Android SDK Platform-Tools adb binary whose \`adb version\` reports its version, or put one on PATH.`; +/** Repair when the selected adb binary does not exist. */ +export const ADB_MISSING_REMEDIATION = `Set ${ADB_BINARY_ENV} to an absolute path to an existing adb binary.`; +/** Repair when execution of the selected adb binary is denied. */ +export const ADB_NOT_EXECUTABLE_REMEDIATION = `Check file type and execute permission for ${ADB_BINARY_ENV} or select an executable adb binary.`; const VERSION_MAX_OUTPUT_BYTES = 64 * 1024; /** How the adb binary was selected for this provider instance. */ @@ -61,6 +67,7 @@ export const resolveAdbBinary = async ( throw new AdbConfigurationFailure( "not_absolute", `${ADB_BINARY_ENV} must be an absolute path to the adb binary`, + `Set ${ADB_BINARY_ENV} to an absolute adb executable path, or unset it to use adb from PATH.`, { configured_path: configured }, ); return { @@ -93,10 +100,10 @@ const resolveWindowsAdb = async ( throw new AdbConfigurationFailure( failure.kind === "missing" ? "binary_missing" : "binary_unresolved", "The adb binary could not be selected from the Windows PATH/PATHEXT", + `Set ${ADB_BINARY_ENV} to an absolute adb executable, or add it to PATH/PATHEXT.`, { selected_binary: "adb", path_source: "path", - remediation: `Set ${ADB_BINARY_ENV} to an absolute adb executable, or add it to PATH/PATHEXT`, ...(failure.kind === "invalid_extension" ? { invalid_extension: failure.extension } : {}), @@ -123,10 +130,12 @@ export type AdbConfigurationFailureCode = | "binary_not_executable" | "version_unresolved"; +/** Why the selected adb binary is unusable, with the owner's repair instruction. */ export class AdbConfigurationFailure extends Error { constructor( readonly code: AdbConfigurationFailureCode, message: string, + readonly remediation: string, readonly diagnostics: Readonly> = {}, options?: ErrorOptions, ) { @@ -181,6 +190,7 @@ export const inspectAdbClient = async ( throw new AdbConfigurationFailure( "version_unresolved", "The selected adb binary exited successfully but did not report a recognizable protocol or build version", + ADB_VERSION_REMEDIATION, { selected_binary: selection.binary, path_source: selection.pathSource, @@ -237,10 +247,10 @@ export const inspectAdbClient = async ( throw new AdbConfigurationFailure( "binary_missing", `The selected adb binary was not found: ${selection.binary}`, + ADB_MISSING_REMEDIATION, { selected_binary: selection.binary, path_source: selection.pathSource, - remediation: `Set ${ADB_BINARY_ENV} to an absolute path to an existing adb binary`, }, { cause }, ); @@ -248,17 +258,18 @@ export const inspectAdbClient = async ( throw new AdbConfigurationFailure( "binary_not_executable", `Execution of the selected adb binary was denied (${failure.code}): ${selection.binary}`, + ADB_NOT_EXECUTABLE_REMEDIATION, { selected_binary: selection.binary, path_source: selection.pathSource, filesystem_error: failure.code, - remediation: `Check file type and execute permission for ${ADB_BINARY_ENV} or select an executable adb binary`, }, { cause }, ); throw new AdbConfigurationFailure( "version_unresolved", "The selected adb binary did not report a usable version", + ADB_VERSION_REMEDIATION, { selected_binary: selection.binary, ...(failure === undefined @@ -287,10 +298,10 @@ const rejectionCodeOf = ( export const adbConfigurationAvailability = ( cause: AdbConfigurationFailure, -): ProviderAvailability => - ({ - status: "unavailable", - code: rejectionCodeOf(cause.code), - reason: cause.message, - diagnostics: { ...cause.diagnostics }, - }) as ProviderAvailability; +): ProviderAvailability => ({ + status: "unavailable", + code: rejectionCodeOf(cause.code), + reason: cause.message, + remediation: cause.remediation, + diagnostics: { ...cause.diagnostics }, +}); diff --git a/src/adb/AdbProvider.test.ts b/src/adb/AdbProvider.test.ts index d81ade095a..08303731fb 100644 --- a/src/adb/AdbProvider.test.ts +++ b/src/adb/AdbProvider.test.ts @@ -591,7 +591,8 @@ it("reports a missing selected binary as a capability gap with recovery", async const availability = await provider.inspectAvailability(); expect(availability).toMatchObject({ status: "unavailable" }); if (availability.status !== "unavailable") return; - expect(availability.diagnostics.remediation).toContain("REA_ADB_PATH"); + expect(availability.remediation).toContain("REA_ADB_PATH"); + expect(availability.diagnostics).not.toHaveProperty("remediation"); const result = await provider.execute({ operation: "list_adb_devices", input: {}, diff --git a/src/android/JadxConfiguration.ts b/src/android/JadxConfiguration.ts index cd614af68e..9e2487eae9 100644 --- a/src/android/JadxConfiguration.ts +++ b/src/android/JadxConfiguration.ts @@ -72,10 +72,10 @@ type ConfigurationFailureCode = | "open_options_invalid"; /** - * Failure kinds tagged at the JADX configuration boundary. Consumers (the - * doctor adapter) project the boundary's remediation and classification for - * a kind; they must not reinterpret the phase/setting evidence to derive - * them. + * Failure kinds tagged at the JADX configuration boundary. Availability + * carries the boundary's remediation for a kind and the doctor adapter + * projects its classification; neither reinterprets the phase/setting + * evidence to derive them. */ export type JadxFailureKind = | "windows_native_artifact" @@ -101,72 +101,52 @@ export const JADX_FAILURE_KINDS: readonly JadxFailureKind[] = [ "unsupported_host", ]; -export interface JadxFailureOutcome { - readonly remediation: string; - readonly classification: DoctorProviderCheck["classification"]; -} +/** The boundary-owned repair instruction for one tagged failure kind. */ +const jadxFailureRemediation = (kind: JadxFailureKind): string => { + switch (kind) { + case "windows_native_artifact": + case "windows_native_runtime": + return JADX_WINDOWS_NATIVE_REMEDIATION; + case "java_runtime": + case "java_home": + return JADX_JAVA_RUNTIME_REMEDIATION; + case "jvm_options": + return JADX_JVM_OPTIONS_REMEDIATION; + case "operation_timeout": + return JADX_OPERATION_TIMEOUT_REMEDIATION; + case "jar": + return JADX_JAR_CONFIGURATION_REMEDIATION; + case "java_probe": + return JADX_JAVA_PROBE_REMEDIATION; + case "unsupported_host": + return JADX_UNSUPPORTED_HOST_REMEDIATION; + } +}; /** - * Project the boundary-owned remediation and classification for a failure - * kind. A null kind (missing or corrupt tag) keeps JAR guidance, matching - * the previous fallthrough for unclassified failures. + * Project the doctor classification for a failure kind. A null kind (an + * untagged readiness failure) is classified as configuration drift. */ -export const projectJadxFailureOutcome = ( +export const jadxFailureClassification = ( kind: JadxFailureKind | null, configured: boolean, -): JadxFailureOutcome => { +): DoctorProviderCheck["classification"] => { const drift = configured ? "config_drift" : "missing_analysis_engine"; switch (kind) { case "windows_native_artifact": - return { - remediation: JADX_WINDOWS_NATIVE_REMEDIATION, - classification: "missing_dependency", - }; - case "windows_native_runtime": - return { - remediation: JADX_WINDOWS_NATIVE_REMEDIATION, - classification: "runtime_failure", - }; case "java_runtime": - return { - remediation: JADX_JAVA_RUNTIME_REMEDIATION, - classification: "missing_dependency", - }; - case "java_home": - return { - remediation: JADX_JAVA_RUNTIME_REMEDIATION, - classification: drift, - }; - case "jvm_options": - return { - remediation: JADX_JVM_OPTIONS_REMEDIATION, - classification: drift, - }; - case "operation_timeout": - return { - remediation: JADX_OPERATION_TIMEOUT_REMEDIATION, - classification: drift, - }; - case "jar": - return { - remediation: JADX_JAR_CONFIGURATION_REMEDIATION, - classification: drift, - }; - case "java_probe": - return { - remediation: JADX_JAVA_PROBE_REMEDIATION, - classification: drift, - }; + return "missing_dependency"; + case "windows_native_runtime": + return "runtime_failure"; case "unsupported_host": - return { - remediation: JADX_UNSUPPORTED_HOST_REMEDIATION, - classification: "unsupported_host", - }; + return "unsupported_host"; + case "java_home": + case "jvm_options": + case "operation_timeout": + case "jar": + case "java_probe": case null: - return { - remediation: JADX_JAR_CONFIGURATION_REMEDIATION, - classification: drift, - }; + return drift; } }; @@ -227,6 +207,7 @@ export const inspectJadxAvailability = async ( status: "unavailable", code: cause.code, reason: cause.message, + remediation: jadxFailureRemediation(cause.kind), diagnostics: { ...cause.diagnostics, failure_kind: cause.kind }, }; const prerequisite = ownedProcessPrerequisiteAvailability(cause); diff --git a/src/android/JadxDoctor.ts b/src/android/JadxDoctor.ts index bb2102080b..9667f49920 100644 --- a/src/android/JadxDoctor.ts +++ b/src/android/JadxDoctor.ts @@ -8,9 +8,10 @@ import type { JsonValue } from "../domain/jsonValue.js"; import { inspectJadxAvailability, JADX_FAILURE_KINDS, - projectJadxFailureOutcome, + jadxFailureClassification, type JadxFailureKind, } from "./JadxConfiguration.js"; +import { JADX_JAR_CONFIGURATION_REMEDIATION } from "./JadxRelease.js"; /** Project the validated JADX configuration into the doctor contract. */ export const inspectJadxProvider = async ( @@ -93,14 +94,14 @@ const configurationCheck = ( : {}), }; } - // The failure kind is tagged at the configuration boundary; project its - // remediation and classification here instead of reinterpreting the - // phase/setting evidence. Every failed check keeps a remediation. + // Availability carries the configuration boundary's remediation and tags + // the failure kind; project its classification here instead of + // reinterpreting the phase/setting evidence. Every failed check keeps a + // remediation. const kind = availability.status === "unavailable" ? asFailureKind(availability.diagnostics["failure_kind"]) : null; - const outcome = projectJadxFailureOutcome(kind, configured); return { name: "configuration", ok: false, @@ -109,8 +110,11 @@ const configurationCheck = ( availability.status === "available" ? "Validated JADX configuration reported no JAR path." : availability.reason, - remediation: outcome.remediation, - classification: outcome.classification, + remediation: + availability.status === "available" + ? JADX_JAR_CONFIGURATION_REMEDIATION + : availability.remediation, + classification: jadxFailureClassification(kind, configured), ...(kind === "java_probe" && availability.status === "unavailable" ? { details: probeDetails(availability.diagnostics) } : kind !== null && diff --git a/src/apktool/ApktoolConfiguration.test.ts b/src/apktool/ApktoolConfiguration.test.ts index b3e4a4d142..e8d1479477 100644 --- a/src/apktool/ApktoolConfiguration.test.ts +++ b/src/apktool/ApktoolConfiguration.test.ts @@ -271,9 +271,9 @@ describe("apktoolInvocation", () => { expect(apktoolConfigurationAvailability(failure)).toMatchObject({ status: "unavailable", code: "not_configured", + remediation: expect.stringContaining(APKTOOL_COMMAND_ENV), diagnostics: { selected_command: "C:\\tools & more\\apktool.cmd", - remediation: expect.stringContaining(APKTOOL_COMMAND_ENV), }, }); }); diff --git a/src/apktool/ApktoolConfiguration.ts b/src/apktool/ApktoolConfiguration.ts index 39180047a2..8c77f61d5a 100644 --- a/src/apktool/ApktoolConfiguration.ts +++ b/src/apktool/ApktoolConfiguration.ts @@ -46,6 +46,12 @@ const apktoolVersionProbeTimeoutGuidance = (): { remediationAction: `This ${String(VERSION_TIMEOUT_MS)} ms apktool version probe has no runtime override. decode_timeout_ms does not raise it. Allow execution, cleanup and delivery headroom in the outer MCP client deadline. Confirm outstanding work and cleanup before retrying.`, }); +const APKTOOL_VERSION_REMEDIATION = `Set ${APKTOOL_COMMAND_ENV} to an Apktool 2.7.0 or newer launcher whose --version prints its version, or put one on PATH.`; +/** Repair when the selected apktool launcher does not exist. */ +export const APKTOOL_MISSING_REMEDIATION = `Set ${APKTOOL_COMMAND_ENV} to an absolute apktool launcher (the upstream wrapper or a java -jar script).`; +/** Repair when execution of the selected apktool launcher is denied. */ +export const APKTOOL_NOT_EXECUTABLE_REMEDIATION = `Check file type and execute permission for ${APKTOOL_COMMAND_ENV} or select an executable apktool launcher.`; + /** How the apktool launcher was selected for this provider instance. */ export interface ApktoolCommandSelection { readonly command: string; @@ -77,6 +83,7 @@ export const resolveApktoolCommand = async ( throw new ApktoolConfigurationFailure( "not_absolute", `${APKTOOL_COMMAND_ENV} must be an absolute path to the apktool launcher`, + `Set ${APKTOOL_COMMAND_ENV} to an absolute apktool launcher path, or unset it to use apktool from PATH.`, { configured_command: configured }, ); return { command: configured, commandSource: "environment" }; @@ -106,6 +113,7 @@ export const resolveApktoolCommand = async ( throw new ApktoolConfigurationFailure( "command_unresolved", "Windows PATHEXT must contain filename extensions, not paths", + `Correct PATHEXT so it lists filename extensions only, or set ${APKTOOL_COMMAND_ENV} to an absolute apktool launcher.`, { selected_command: "apktool", command_source: "path", @@ -116,6 +124,7 @@ export const resolveApktoolCommand = async ( throw new ApktoolConfigurationFailure( "command_unresolved", `The PATH-selected apktool launcher could not be inspected: ${failure.path}`, + `Make the PATH-selected apktool launcher readable, or set ${APKTOOL_COMMAND_ENV} to an absolute apktool launcher.`, { selected_command: "apktool", command_source: "path", @@ -131,10 +140,10 @@ export const resolveApktoolCommand = async ( throw new ApktoolConfigurationFailure( "command_missing", "The apktool launcher was not found on the selected Windows PATH/PATHEXT", + `Set ${APKTOOL_COMMAND_ENV} to an absolute apktool launcher, or add its directory and extension to PATH and PATHEXT.`, { selected_command: "apktool", command_source: "path", - remediation: `Set ${APKTOOL_COMMAND_ENV} to an absolute apktool launcher, or add its directory and extension to PATH and PATHEXT`, }, ); }; @@ -148,10 +157,12 @@ export type ApktoolConfigurationFailureCode = | "version_unresolved" | "unsupported_version"; +/** Why the selected apktool launcher is unusable, with the owner's repair instruction. */ export class ApktoolConfigurationFailure extends Error { constructor( readonly code: ApktoolConfigurationFailureCode, message: string, + readonly remediation: string, readonly diagnostics: Readonly> = {}, options?: ErrorOptions, ) { @@ -287,10 +298,10 @@ export const inspectApktoolClient = async ( throw new ApktoolConfigurationFailure( "launcher_unsupported", `The selected apktool launcher cannot be run through cmd.exe: ${cause.message}`, + `Set ${APKTOOL_COMMAND_ENV} to a launcher path without cmd.exe metacharacters (" % ^ ! & | < >), or to a native apktool executable.`, { selected_command: selection.command, command_source: selection.commandSource, - remediation: `Set ${APKTOOL_COMMAND_ENV} to a launcher path without cmd.exe metacharacters (" % ^ ! & | < >), or to a native apktool executable`, }, { cause }, ); @@ -349,6 +360,7 @@ export const inspectApktoolClient = async ( : new ApktoolConfigurationFailure( recordedFailure.code, recordedFailure.message, + recordedFailure.remediation, { ...recordedFailure.diagnostics, ...(cancellation === undefined @@ -376,6 +388,7 @@ export const inspectApktoolClient = async ( throw new ApktoolConfigurationFailure( "version_unresolved", "The owned version command completed without an output observation", + APKTOOL_VERSION_REMEDIATION, { selected_command: selection.command }, ); if (!interpreted.ok) throw interpreted.error; @@ -393,6 +406,7 @@ const interpretApktoolVersion = ( throw new ApktoolConfigurationFailure( "version_unresolved", "The selected apktool launcher exited successfully but did not report a recognizable version", + APKTOOL_VERSION_REMEDIATION, { selected_command: selection.command, command_source: selection.commandSource, @@ -406,12 +420,12 @@ const interpretApktoolVersion = ( throw new ApktoolConfigurationFailure( "unsupported_version", `The selected Apktool version ${identity.version} is older than the supported minimum 2.7.0`, + `Use Apktool 2.7.0 or newer; see docs/apktool-resource-analysis.md.`, { selected_command: selection.command, command_source: selection.commandSource, reported_version: identity.version, minimum_version: "2.7.0", - remediation: `Use Apktool 2.7.0 or newer; see docs/apktool-resource-analysis.md.`, }, ); return identity; @@ -434,13 +448,13 @@ const rejectionCodeOf = ( export const apktoolConfigurationAvailability = ( cause: ApktoolConfigurationFailure, -): ProviderAvailability => - ({ - status: "unavailable", - code: rejectionCodeOf(cause.code), - reason: cause.message, - diagnostics: { ...cause.diagnostics }, - }) as ProviderAvailability; +): ProviderAvailability => ({ + status: "unavailable", + code: rejectionCodeOf(cause.code), + reason: cause.message, + remediation: cause.remediation, + diagnostics: { ...cause.diagnostics }, +}); /** Interpret the command owner's outcome, without inferring a new failure from a late signal. */ const versionProbeFailure = ( @@ -501,14 +515,13 @@ const versionProbeFailure = ( cause.code === "EACCES" ? `Execution of the selected apktool launcher was denied: ${selection.command}` : `The selected apktool launcher was not found: ${selection.command}`, + cause.code === "EACCES" + ? APKTOOL_NOT_EXECUTABLE_REMEDIATION + : APKTOOL_MISSING_REMEDIATION, { filesystem_error: cause.code, selected_command: selection.command, command_source: selection.commandSource, - remediation: - cause.code === "EACCES" - ? `Check file type and execute permission for ${APKTOOL_COMMAND_ENV} or select an executable apktool launcher` - : `Set ${APKTOOL_COMMAND_ENV} to an absolute apktool launcher (the upstream wrapper or a java -jar script)`, }, { cause }, ); @@ -535,6 +548,7 @@ const versionProbeFailure = ( return new ApktoolConfigurationFailure( "version_unresolved", "The selected apktool launcher did not report a usable version", + APKTOOL_VERSION_REMEDIATION, { selected_command: selection.command, ...(observed === undefined diff --git a/src/application/AnalysisContextQueries.test.ts b/src/application/AnalysisContextQueries.test.ts index e9453d0e48..8322fadf43 100644 --- a/src/application/AnalysisContextQueries.test.ts +++ b/src/application/AnalysisContextQueries.test.ts @@ -1,4 +1,5 @@ import { describe, expect, it } from "vitest"; +import { createDeferred } from "../../tests/fixtures/binarySession.js"; import { getNavigationContext, @@ -9,10 +10,19 @@ import { type AnalysisOperationPort, } from "./AnalysisProvider.js"; import { + AnalysisCapabilityUnavailableError, AnalysisCancelledError, AnalysisOutputError, AnalysisProtocolError, } from "../domain/analysisErrorCore.js"; +import { AnalysisCleanupLease } from "../artifacts/AnalysisCleanupLease.js"; +import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; +import { + primaryAnalysisError, + type AnalysisError, +} from "../domain/analysisErrorBase.js"; +import type { AnalysisExecution } from "./AnalysisProvider.js"; +import type { Result } from "../domain/result.js"; import type { JsonValue } from "../domain/jsonValue.js"; import { err, ok } from "../domain/result.js"; @@ -437,3 +447,262 @@ it("does not start address facets after document selection completes under cance expect(result.error.cause).toBe(reason); expect(calls).toEqual(["current_document"]); }); + +it("retains both facet cleanup owners until their independent retries succeed", async () => { + const first = new AnalysisCapabilityUnavailableError( + provider.id, + "comment", + "failed", + { + cleanup: { reason: "first release failed", resources: ["first-owner"] }, + }, + ); + const second = new AnalysisCapabilityUnavailableError( + provider.id, + "inline_comment", + "failed", + { + cleanup: { reason: "second release failed", resources: ["second-owner"] }, + }, + ); + let firstRetryAllowed = false; + const firstLease = new AnalysisCleanupLease(async () => { + if (!firstRetryAllowed) throw new Error("still owned"); + }, ["first-owner"]); + const secondLease = new AnalysisCleanupLease(async () => {}, [ + "second-owner", + ]); + firstLease.bindCleanupError(first); + secondLease.bindCleanupError(second); + const result = await inspectAddressContext( + { + execute: async (operation) => + operation === "comment" + ? err(first) + : operation === "inline_comment" + ? err(second) + : ok(createAnalysisExecution(null, provider)), + }, + { address: "0x401000", document: "fixture" }, + ); + if (result.ok) throw new Error("expected retained cleanup failures"); + expect(projectAnalysisError(result.error).code).toBe("cleanup_incomplete"); + expect(result.error.cleanupResources).toEqual( + expect.arrayContaining(["first-owner", "second-owner"]), + ); + await expect(firstLease.release()).rejects.toThrow("still owned"); + expect(result.error.cleanupResources).toEqual( + expect.arrayContaining(["first-owner", "second-owner"]), + ); + await secondLease.release(); + expect(result.error.cleanupIncomplete).toBe(true); + expect(result.error.cleanupResources).toEqual(["first-owner"]); + firstRetryAllowed = true; + await firstLease.release(); + expect(result.error.cleanupIncomplete).toBe(false); + expect(result.error.cleanupResources).toEqual([]); + expect(projectAnalysisError(result.error).code).toBe( + "capability_unavailable", + ); +}); + +it("waits for late facet cleanup after cancellation and retains its owner", async () => { + const controller = new AbortController(); + const started = createDeferred(); + const late = createDeferred>(); + const failure = new AnalysisCapabilityUnavailableError( + provider.id, + "comment", + "failed", + { + cleanup: { reason: "late cleanup failed", resources: ["late-owner"] }, + }, + ); + const lease = new AnalysisCleanupLease(async () => {}, ["late-owner"]); + lease.bindCleanupError(failure); + const analysis: AnalysisOperationPort = { + execute: (operation) => { + if (operation === "comment") { + started.resolve(); + return late.promise; + } + return Promise.resolve(ok(createAnalysisExecution(null, provider))); + }, + }; + let settled = false; + const pending = inspectAddressContext( + analysis, + { address: "0x401000", document: "fixture" }, + controller.signal, + ).then((result) => { + settled = true; + return result; + }); + await started.promise; + controller.abort(); + await Promise.resolve(); + expect(settled).toBe(false); + late.resolve(err(failure)); + const result = await pending; + if (result.ok) throw new Error("expected cancellation with cleanup failure"); + expect(projectAnalysisError(result.error).code).toBe("cleanup_incomplete"); + expect(result.error.cleanupResources).toEqual(["late-owner"]); + await lease.release(); + expect(projectAnalysisError(result.error).code).toBe("cancelled"); +}); + +it("reports cancellation after a late successful facet that ignored its signal", async () => { + const controller = new AbortController(); + const started = createDeferred(); + const late = createDeferred>(); + const pending = inspectAddressContext( + { + execute: (operation) => { + if (operation === "comment") { + started.resolve(); + return late.promise; + } + return Promise.resolve(ok(createAnalysisExecution(null, provider))); + }, + }, + { address: "0x401000", document: "fixture" }, + controller.signal, + ); + await started.promise; + controller.abort(); + late.resolve(ok(createAnalysisExecution("finished", provider))); + const result = await pending; + if (result.ok) throw new Error("expected cancellation"); + expect(projectAnalysisError(result.error).code).toBe("cancelled"); +}); + +it("does not start facets for an already-cancelled explicit document", async () => { + const controller = new AbortController(); + controller.abort(); + const calls: string[] = []; + const result = await inspectAddressContext( + { + execute: async (operation) => { + calls.push(operation); + return ok(createAnalysisExecution(null, provider)); + }, + }, + { address: "0x401000", document: "fixture" }, + controller.signal, + ); + if (result.ok) throw new Error("expected cancellation"); + expect(projectAnalysisError(result.error).code).toBe("cancelled"); + expect(calls).toEqual([]); +}); + +it("preserves a provider's cancellation for the same signal", async () => { + const controller = new AbortController(); + const started = createDeferred(); + const reason = { request: "controlled cancellation" }; + let providerCancellation: AnalysisCancelledError | undefined; + const pending = inspectAddressContext( + { + execute: (operation) => { + if (operation !== "address_name") + return Promise.resolve(ok(createAnalysisExecution(null, provider))); + started.resolve(); + return new Promise((resolve) => { + controller.signal.addEventListener( + "abort", + () => { + providerCancellation = new AnalysisCancelledError(operation, { + cause: controller.signal.reason, + diagnostics: { owner: "controlled provider" }, + }); + resolve(err(providerCancellation)); + }, + { once: true }, + ); + }); + }, + }, + { address: "0x401000", document: "fixture" }, + controller.signal, + ); + await started.promise; + controller.abort(reason); + const result = await pending; + if (result.ok) throw new Error("expected cancellation"); + expect(result.error).toBe(providerCancellation); + expect(projectAnalysisError(result.error).details).toMatchObject({ + diagnostics: { owner: "controlled provider" }, + }); +}); + +it.each([false, true])( + "keeps an earlier cleanup failure primary when cancellation arrives later (cancellation cleanup=%s)", + async (cancelCleanup) => { + const controller = new AbortController(); + const firstDone = createDeferred(); + const failure = new AnalysisCapabilityUnavailableError( + provider.id, + "comment", + "failed", + { + cleanup: { reason: "cleanup failed", resources: ["first-owner"] }, + }, + ); + const lease = new AnalysisCleanupLease(async () => {}, ["first-owner"]); + lease.bindCleanupError(failure); + const cancelled = new AnalysisCancelledError( + "address_name", + cancelCleanup + ? { + cleanup: { + reason: "cancel cleanup failed", + resources: ["cancel-owner"], + }, + } + : undefined, + ); + const cancellationLease = new AnalysisCleanupLease(async () => {}, [ + "cancel-owner", + ]); + if (cancelCleanup) cancellationLease.bindCleanupError(cancelled); + const analysis: AnalysisOperationPort = { + execute: async (operation) => { + if (operation === "address_name") + return new Promise((resolve) => { + controller.signal.addEventListener( + "abort", + () => resolve(err(cancelled)), + { once: true }, + ); + }); + if (operation === "comment") { + firstDone.resolve(); + return err(failure); + } + return ok(createAnalysisExecution(null, provider)); + }, + }; + const pending = inspectAddressContext( + analysis, + { address: "0x401000", document: "fixture" }, + controller.signal, + ); + await firstDone.promise; + await Promise.resolve(); + await Promise.resolve(); + controller.abort(); + const result = await pending; + if (result.ok) throw new Error("expected retained cleanup failure"); + expect(primaryAnalysisError(result.error)).toBe(failure); + expect(failure.secondaryFailures).toContain(cancelled); + expect(projectAnalysisError(result.error).code).toBe("cleanup_incomplete"); + await lease.release(); + if (cancelCleanup) { + expect(result.error.cleanupIncomplete).toBe(true); + expect(result.error.cleanupResources).toEqual(["cancel-owner"]); + await cancellationLease.release(); + } + expect(projectAnalysisError(result.error).code).toBe( + "capability_unavailable", + ); + }, +); diff --git a/src/application/AnalysisContextQueries.ts b/src/application/AnalysisContextQueries.ts index 769bd11f74..03176264cf 100644 --- a/src/application/AnalysisContextQueries.ts +++ b/src/application/AnalysisContextQueries.ts @@ -4,7 +4,17 @@ import type { AnalysisOperationPort, } from "./AnalysisProvider.js"; import type { AnalysisError } from "../domain/analysisErrorBase.js"; -import { AnalysisOutputError } from "../domain/analysisErrorCore.js"; +import { + AnalysisCancelledError, + AnalysisInputError, + AnalysisOutputError, +} from "../domain/analysisErrorCore.js"; +import { + projectAnalysisError, + type AnalysisErrorProjection, +} from "../domain/analysisErrorProjection.js"; +import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js"; +import { analysisErrorWithCancellation } from "../domain/analysisErrorCancellation.js"; import type { JsonValue } from "../domain/jsonValue.js"; import { analysisBookmarkSchema } from "../domain/native/nativeInventory.js"; import { parseContainingProcedureResolution } from "../domain/native/nativeObservation.js"; @@ -18,6 +28,7 @@ type Facet = readonly state: "unavailable"; readonly reason: string; readonly remediation: string; + readonly error?: AnalysisErrorProjection; }; const parameters = ( @@ -98,18 +109,39 @@ export const inspectAddressContext = async ( "inline_comment", "list_bookmarks", ] as const satisfies readonly AnalysisOperation[]; - const results = await Promise.all( - operations.map((operation) => - analysis.execute( - operation, - operation === "list_bookmarks" ? parameters(document.value) : args, - executionOptions(signal), - ), - ), - ); - for (const result of results) { - if (!result.ok && isRequestLevelFailure(result.error)) return result; + const completed: Result[] = []; + if (signal?.aborted === true) + return err( + new AnalysisCancelledError("inspect_address_context", { + cause: signal.reason, + }), + ); + let signalCancellation: AnalysisCancelledError | undefined; + const cancelled = () => { + signalCancellation = new AnalysisCancelledError("inspect_address_context", { + cause: signal?.reason, + }); + completed.push(err(signalCancellation)); + }; + signal?.addEventListener("abort", cancelled, { once: true }); + let results: Result[]; + try { + results = await Promise.all( + operations.map(async (operation) => { + const result = await analysis.execute( + operation, + operation === "list_bookmarks" ? parameters(document.value) : args, + executionOptions(signal), + ); + completed.push(result); + return result; + }), + ); + } finally { + signal?.removeEventListener("abort", cancelled); } + const fatal = contextFailure(completed, signalCancellation); + if (fatal !== undefined) return err(fatal); const name = facet(results[0], operations[0]); const procedureResult = results[1]; const resolution = procedureResult?.ok @@ -149,17 +181,82 @@ const facet = ( const unavailableFacet = ( error: AnalysisError | undefined, operation: AnalysisOperation, -): Facet => ({ - state: "unavailable", - reason: error?.message ?? `${operation} returned no result`, - remediation: `Call ${operation} directly to inspect provider availability and diagnostics.`, -}); +): Facet => { + if (error instanceof AnalysisInputError) { + const projected = projectAnalysisError(error); + if (projected.code === "invalid_request") + return { + state: "unavailable", + reason: projected.message, + remediation: projected.remediation.action, + error: projected, + }; + } + return { + state: "unavailable", + reason: error?.message ?? `${operation} returned no result`, + remediation: `Call ${operation} directly to inspect provider availability and diagnostics.`, + }; +}; const isRequestLevelFailure = (error: AnalysisError): boolean => error._tag === "AnalysisCancelledError" || error._tag === "AnalysisTimeoutError" || error._tag === "NoBinaryOpenError"; +/** Wait for all facets and retain every unresolved cleanup owner. */ +const contextFailure = ( + results: readonly Result[], + signalCancellation?: AnalysisCancelledError, +): AnalysisError | undefined => { + const failures = [ + ...new Set(results.flatMap((result) => (result.ok ? [] : [result.error]))), + ]; + // A provider's cancellation for this exact signal carries its own diagnostic + // and cleanup facts. Attribute that returned error to the abort event rather + // than replacing it with our generic observation of the same signal. + const marker = + signalCancellation === undefined + ? -1 + : failures.indexOf(signalCancellation); + if (marker >= 0) { + const reported = failures.findIndex( + (failure, index) => + index > marker && + failure instanceof AnalysisCancelledError && + failure.cause === signalCancellation?.cause, + ); + if (reported >= 0) { + failures[marker] = failures[reported]!; + failures.splice(reported, 1); + } + } + const first = failures.find( + (failure) => isRequestLevelFailure(failure) || failure.cleanupIncomplete, + ); + if (first === undefined) return undefined; + let combined = first; + for (const failure of failures) { + if (failure === first) continue; + if (failure instanceof AnalysisCancelledError) + combined = analysisErrorWithCancellation(combined, failure); + if (failure.cleanupIncomplete) { + const previous = combined; + combined = analysisErrorWithCleanupFailure( + previous, + failure, + "inspect_address_context", + ).bindCleanupDependencies([previous, failure]); + } else if ( + !(failure instanceof AnalysisCancelledError) && + isRequestLevelFailure(failure) + ) { + combined.retainSecondaryFailure(failure); + } + } + return combined; +}; + const matchingBookmarks = (value: JsonValue, address: string): Facet => { if (!Array.isArray(value)) return { diff --git a/src/application/AnalysisEvidenceSources.ts b/src/application/AnalysisEvidenceSources.ts index ff43a6a8b0..b7795ba283 100644 --- a/src/application/AnalysisEvidenceSources.ts +++ b/src/application/AnalysisEvidenceSources.ts @@ -1,6 +1,7 @@ import { readArtifactInspectionSources } from "../domain/artifactInspection.js"; import type { AnalysisError } from "../domain/analysisErrorBase.js"; -import type { Evidence } from "../domain/evidence.js"; +import { tryParseEvidence, type Evidence } from "../domain/evidence.js"; +import { jsonValueByteLength } from "../domain/jsonValueByteLength.js"; import { err, ok, type Result } from "../domain/result.js"; /** Resolve the embedded publication closure without reinterpreting it in delivery adapters. */ @@ -15,3 +16,128 @@ export const analysisEvidenceSources = ( return err(embedded.error.retainPartialObservation(complete)); return ok([...embedded.value, complete]); }; + +/** Matches the diagnostics allowance retained for other finalized one-shot work. */ +const SOURCE_CLOSURE_INLINE_BYTES = 64 * 1024; +/** Identities beyond this stay counted; direct links remain on the observation. */ +const SOURCE_CLOSURE_LISTED_OMISSIONS = 64; +const SOURCE_CLOSURE_RECOVERY = + "No bundle was written and this invocation retains nothing after exit. Keep any records shown here, or run the operation again after following remediation; that is a new observation and may differ."; + +/** What a failed one-shot delivery holds for one record its observation cites. */ +export type SourceClosureOmission = { + readonly evidence_id: string; + /** + * `not_captured`: this invocation holds no such record, so none can be shown. + * `not_inlined`: captured, but not part of this bounded failure response. + */ + readonly reason: "not_captured" | "not_inlined"; +}; + +/** Bounded failure handoff for a completed observation whose bundle never committed. */ +export type UndeliveredSourceClosure = { + readonly publication: "not_published"; + /** Every cited record is in `records` or embedded in the observation itself. */ + readonly complete: boolean; + readonly records: Evidence[]; + readonly omitted_count: number; + readonly omitted: SourceClosureOmission[]; + readonly recovery: string; +}; + +/** The closure state is unknown because examining it failed. */ +export type UnavailableSourceClosure = { + readonly publication: "not_published"; + readonly complete: false; + readonly collection_failure: string; + readonly recovery: string; +}; + +const collectSourceClosure = ( + observation: Evidence, + captured: readonly Evidence[], + options: { readonly inlineRecords: boolean }, +): UndeliveredSourceClosure | undefined => { + const inspection = analysisEvidenceSources(observation); + const embedded = new Map( + (inspection.ok ? inspection.value : []).map((record) => [ + record.evidence_id, + record, + ]), + ); + const held = new Map(captured.map((record) => [record.evidence_id, record])); + // Set iteration reaches entries added while it runs, which makes this a + // deduplicating breadth-first queue: directly cited records spend the + // allowance first, and each link costs one lookup however often it repeats. + const closure = new Set([observation.evidence_id]); + for (const evidenceId of observation.evidence_links) closure.add(evidenceId); + for (const evidenceId of held.keys()) closure.add(evidenceId); + const records: Evidence[] = []; + const omitted: SourceClosureOmission[] = []; + let omittedCount = 0; + let inlineBytes = options.inlineRecords ? SOURCE_CLOSURE_INLINE_BYTES : 0; + const omit = ( + evidenceId: string, + reason: SourceClosureOmission["reason"], + ) => { + omittedCount += 1; + if (omitted.length < SOURCE_CLOSURE_LISTED_OMISSIONS) + omitted.push({ evidence_id: evidenceId, reason }); + }; + for (const evidenceId of closure) { + if (evidenceId === observation.evidence_id) continue; + const inObservation = embedded.get(evidenceId); + const record = inObservation ?? held.get(evidenceId); + if (record === undefined) { + omit(evidenceId, "not_captured"); + continue; + } + for (const link of record.evidence_links) closure.add(link); + if (inObservation !== undefined) continue; + const bytes = jsonValueByteLength(record, inlineBytes); + if (bytes > inlineBytes) { + omit(evidenceId, "not_inlined"); + continue; + } + records.push(record); + inlineBytes -= bytes; + } + if (closure.size === 1) return undefined; + return { + publication: "not_published", + complete: omittedCount === 0, + records, + omitted_count: omittedCount, + omitted, + recovery: SOURCE_CLOSURE_RECOVERY, + }; +}; + +/** + * Describe the source closure of a completed observation after delivery failed + * before commit. Only already captured records are reported, so an unresolved + * link stays an explicit omission and its own ancestors remain unknown. + * + * This annotates a failure that must reach the caller, so it never throws. If + * examining the closure fails, the state is reported as unknown, naming only + * the bounded error type rather than a message that could quote analyzed data. + */ +export const undeliveredSourceClosure = ( + partialObservation: unknown, + captured: readonly Evidence[], + options: { readonly inlineRecords: boolean }, +): UndeliveredSourceClosure | UnavailableSourceClosure | undefined => { + try { + const observation = tryParseEvidence(partialObservation); + return observation === undefined + ? undefined + : collectSourceClosure(observation, captured, options); + } catch (cause: unknown) { + return { + publication: "not_published", + complete: false, + collection_failure: cause instanceof Error ? cause.name : "unknown", + recovery: SOURCE_CLOSURE_RECOVERY, + }; + } +}; diff --git a/src/application/AnalysisProvider.ts b/src/application/AnalysisProvider.ts index ed926de452..74d41d2d81 100644 --- a/src/application/AnalysisProvider.ts +++ b/src/application/AnalysisProvider.ts @@ -2,6 +2,7 @@ import type { AnalysisWorkScope } from "./AnalysisWorkScope.js"; import type { AnalysisObservation } from "../domain/analysisObservation.js"; import type { ProviderInspectionScope } from "./ProviderInspectionScope.js"; import type { BinaryTarget } from "../domain/binaryTargetTypes.js"; +import type { TargetApplicability } from "../domain/targetApplicability.js"; import type { OfficialToolName } from "../contracts/officialToolContracts.js"; import type { EnhancedToolName } from "../contracts/enhancedInputs.js"; import type { NativeToolName } from "../contracts/native/nativeToolContracts.js"; @@ -257,9 +258,16 @@ type CapabilityAvailability = readonly available: false; readonly reason: string; readonly availabilityCode?: ProviderRejectionCode; + /** + * Repair instruction from the readiness owner that made this operation + * unavailable. Static capability facts omit it. + */ + readonly remediation?: string; }; export type CapabilityDescriptor = CapabilityAvailability & { + /** Provider-owned static target requirements, evaluated for an open target. */ + readonly targetApplicability?: TargetApplicability; /** * Snapshot opts immutable, profile-bound reads into replay despite incidental * private filesystem writes. Live state must always be re-observed. Omitting @@ -301,7 +309,10 @@ export type ProviderAvailability = | { readonly status: "unavailable"; readonly code: ProviderRejectionCode; + /** What the owner observed; not a repair instruction. */ readonly reason: string; + /** The owner's repair instruction for this exact failure. */ + readonly remediation: string; readonly diagnostics: Readonly>; }; diff --git a/src/application/AppBundleExecutable.ts b/src/application/AppBundleExecutable.ts index 58761256a1..e8209cc166 100644 --- a/src/application/AppBundleExecutable.ts +++ b/src/application/AppBundleExecutable.ts @@ -461,10 +461,11 @@ const resolveLayoutExecutable = async ( let name: string; try { - name = hasBinaryPlistSignature(plist) - ? await (fileSystem.decodeBinaryPlist?.(plistPath) ?? - parseBinaryPlistExecutable(plist)) - : parseXmlPlistExecutable(decodeXmlPlistText(plist)); + name = + hasBinaryPlistSignature(plist) && + fileSystem.decodeBinaryPlist !== undefined + ? await fileSystem.decodeBinaryPlist(plistPath) + : executableName(decodeInfoPlistValue(plist)); } catch (cause: unknown) { return err( new BinaryTargetError( @@ -546,12 +547,18 @@ const resolveLayoutExecutable = async ( } }; -/** Decode the top-level executable name with an XML parser, not a pattern. */ -const parseXmlPlistExecutable = (plist: string): string => { - // An unrelated `__proto__` entry must not make the bundle unreadable. - const { value } = parseXmlPropertyList(plist); - return executableName(value); -}; +/** + * Decode one Info.plist's top-level value in-process, binary or XML, so no + * host plist tool is required. Values are read with format-aware parsers, not + * patterns; an unrelated `__proto__` entry must not make the bundle unreadable. + */ +export const decodeInfoPlistValue = (plist: Buffer): unknown => + hasBinaryPlistSignature(plist) + ? decodeBinaryPlistValue( + plist, + preflightPropertyListDecode(plist, BINARY_PLIST_DECODE_BYTES), + ) + : parseXmlPropertyList(decodeXmlPlistText(plist)).value; /** The Info.plist decoded but declares no string CFBundleExecutable. */ class UndeclaredBundleExecutable extends Error {} @@ -565,12 +572,6 @@ const executableName = (value: unknown): string => { const executableEntrySchema = z.looseObject({ CFBundleExecutable: z.string() }); -/** Decode a binary plist in-process, so no host plist tool is required. */ -const parseBinaryPlistExecutable = (plist: Buffer): string => { - const plan = preflightPropertyListDecode(plist, BINARY_PLIST_DECODE_BYTES); - return executableName(decodeBinaryPlistValue(plist, plan)); -}; - /** Names are one local path component; backslash is literal on POSIX hosts. */ const isSafeExecutableName = (name: string): boolean => name.length > 0 && diff --git a/src/application/BinaryTargetResolver.ts b/src/application/BinaryTargetResolver.ts index a8a4472327..3ce2e473f3 100644 --- a/src/application/BinaryTargetResolver.ts +++ b/src/application/BinaryTargetResolver.ts @@ -23,7 +23,7 @@ import { import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js"; import { jsonValueSchema } from "../domain/jsonValue.js"; -import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; +import { childFailureDiagnostics } from "../domain/analysisErrorProjection.js"; import { constants, type Stats } from "node:fs"; import { createHash } from "node:crypto"; import { access, realpath, stat, type FileHandle } from "node:fs/promises"; @@ -607,14 +607,12 @@ const targetFileCleanupError = ( : String(failure.cleanupCause), resources: [failure.path], }, - diagnostics: { - ...(primary === undefined + ...childFailureDiagnostics( + primary === undefined ? {} : { primary_error: { error: primary } }, + completedTarget === undefined ? {} - : { primary_error: projectAnalysisError(primary) }), - ...(completedTarget === undefined - ? {} - : { completed_target: completedTarget }), - }, + : { completed_target: completedTarget }, + ), }, ); bindRegularFileCleanup(failure, error); diff --git a/src/application/BrowserScenarioCaptureService.test.ts b/src/application/BrowserScenarioCaptureService.test.ts index 34a3c588cb..c370e77ffa 100644 --- a/src/application/BrowserScenarioCaptureService.test.ts +++ b/src/application/BrowserScenarioCaptureService.test.ts @@ -114,6 +114,38 @@ it.each([ }, ); +it("retains completed scenario Evidence when cancellation arrives during capture", async () => { + const caller = new AbortController(); + const reason = { requested: "stop after capture" }; + const provider: BrowserScenarioCapturePort = { + identity: () => scenarioIdentity, + captureScenario: async (received, options) => { + expect(received).toBe(scenario); + expect(options?.signal).toBe(caller.signal); + caller.abort(reason); + return ok(capture); + }, + }; + const result = await captureBrowserScenario(provider, scenario, { + signal: caller.signal, + }); + if (result.ok) throw new Error("Expected late cancellation"); + expect(result.error).toBeInstanceOf(AnalysisCancelledError); + expect(result.error.cause).toBe(reason); + expect(parseEvidence(result.error.partialObservation)).toMatchObject({ + operation: "capture_browser_scenario", + provider: scenarioIdentity, + confidence: "observed", + authority: "controlled-replay", + parameters: { + browser: scenario.browser, + start_url: scenario.start_url, + actions: scenario.actions, + }, + normalized_result: capture, + }); +}); + it("keeps startup cancellation and its incomplete capture", async () => { const reason = { requested: "stop during initial navigation" }; const completed = scriptScenarioFixture(); diff --git a/src/application/CallPathTracing.ts b/src/application/CallPathTracing.ts index a114b5efb8..58a897b105 100644 --- a/src/application/CallPathTracing.ts +++ b/src/application/CallPathTracing.ts @@ -1,17 +1,22 @@ import type { CallRelationshipDirection } from "../contracts/enhancedInputs.js"; -import { CALL_RELATIONSHIP_TRAVERSALS } from "./CallRelationshipTraversal.js"; +import { + cleanupStoppedTraversal, + readCallRelationships, + type CallRelationshipObserver, +} from "./CallRelationshipTraversal.js"; import type { - AnalysisExecution, AnalysisOperation, ExecutionOptions, } from "./AnalysisProvider.js"; import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import { AnalysisCancelledError } from "../domain/analysisErrorCore.js"; import type { AnalysisError } from "../domain/analysisErrorBase.js"; -import { parseRelatedAddresses } from "../domain/native/nativeObservation.js"; import type { JsonValue } from "../domain/jsonValue.js"; import { err, ok, type Result } from "../domain/result.js"; -import type { EnhancedResult } from "./EnhancedToolTypes.js"; +import { + failedWithPartialResult, + type EnhancedResult, +} from "./EnhancedToolTypes.js"; import { resolveProcedureAddress } from "./ProcedureAddressResolution.js"; import type { RelationshipCoverage } from "../domain/relationshipCoverage.js"; @@ -22,13 +27,6 @@ type AnalysisCall = ( selectorOrigin?: ExecutionOptions["selectorOrigin"], ) => Promise>; -type RelationshipCall = ( - name: AnalysisOperation, - arguments_: Readonly>, - signal?: AbortSignal, - selectorOrigin?: ExecutionOptions["selectorOrigin"], -) => Promise>; - export interface CallPathTraceInput { readonly start: string; readonly goal?: string | undefined; @@ -80,7 +78,7 @@ interface DiscoveredEdge { /** Trace direct caller/callee relationships until the graph is exhausted or the goal is reached. */ export const traceCallPath = async ( call: AnalysisCall, - callRelationship: RelationshipCall, + callRelationship: CallRelationshipObserver, input: CallPathTraceInput, signal?: AbortSignal, ): EnhancedResult => { @@ -123,7 +121,16 @@ export const traceCallPath = async ( current, ...(signal === undefined ? {} : { signal }), }); - if (!expanded.ok) return expanded; + if (!expanded.ok) { + if (!expanded.error.cleanupIncomplete) return expanded; + state.residual.add( + cleanupStoppedTraversal(address, state.queue.length - state.queueIndex), + ); + return failedWithPartialResult( + expanded.error, + projectTrace(resolvedInput, state), + ); + } if (expanded.value) break; } if (isAborted(signal)) @@ -148,44 +155,40 @@ const createTraceState = (input: CallPathTraceInput): TraceState => ({ }); const expandAddress = async ( - call: RelationshipCall, + call: CallRelationshipObserver, expansion: AddressExpansion, ): Promise> => { const { input, state, address, current, signal } = expansion; - const { relation, operation: tool } = - CALL_RELATIONSHIP_TRAVERSALS[input.direction]; - const result = await call(tool, { procedure: address }, signal, "provider"); - if (!result.ok) { - if ( - result.error._tag === "AnalysisCancelledError" || - signal?.aborted === true - ) - return result; - } + const read = await readCallRelationships( + call, + input.direction, + address, + signal, + ); + // A provider failure delivered with cancellation remains the selected outcome. + if ( + read.status === "stopped" || + (read.status === "unavailable" && isAborted(signal)) + ) + return err(read.error); if (isAborted(signal)) return err( new AnalysisCancelledError("trace_call_path", { cause: signal?.reason }), ); - if (!result.ok) { + if (read.status !== "observed") { state.failures.push({ address, - error: projectAnalysisError(result.error), + error: projectAnalysisError(read.error), }); - state.residual.add(`Call relationships were unavailable for ${address}.`); + state.residual.add( + `Call relationships were ${read.status} for ${address}.`, + ); return ok(false); } - const related = parseRelatedAddresses(result.value.result, relation, tool); - if (!related.ok) { - state.failures.push({ - address, - error: projectAnalysisError(related.error), - }); - state.residual.add(`Call relationships were unreadable for ${address}.`); - return ok(false); - } - for (const limitation of result.value.limitations) + const { execution, related } = read; + for (const limitation of execution.limitations) state.limitations.add(limitation); - const coverage = result.value.relationshipCoverage ?? { + const coverage = execution.relationshipCoverage ?? { status: "unknown", reason: "Provider did not report relationship coverage.", }; @@ -204,7 +207,7 @@ const expandAddress = async ( state.residual.add( `Call relationship coverage is unknown for ${address}: ${coverage.reason}`, ); - for (const relatedAddress of sortedUniqueAddresses(related.value)) { + for (const relatedAddress of sortedUniqueAddresses(related)) { recordEdge(input, state, { address, relatedAddress, diff --git a/src/application/CallRelationshipTraversal.test.ts b/src/application/CallRelationshipTraversal.test.ts new file mode 100644 index 0000000000..c52d6a06d2 --- /dev/null +++ b/src/application/CallRelationshipTraversal.test.ts @@ -0,0 +1,125 @@ +import { describe, expect, it } from "vitest"; + +import { AnalysisCleanupLease } from "../artifacts/AnalysisCleanupLease.js"; +import { AnalysisError } from "../domain/analysisErrorBase.js"; +import { + AnalysisCancelledError, + AnalysisCapabilityUnavailableError, +} from "../domain/analysisErrorCore.js"; +import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; +import { err, ok } from "../domain/result.js"; +import { + createAnalysisExecution, + type AnalysisOperationPort, +} from "./AnalysisProvider.js"; +import { EnhancedTools } from "./EnhancedTools.js"; + +const provider = { id: "fixture", name: "Fixture", version: "1" }; +const tools = ["get_call_graph", "trace_call_path"] as const; +const input = (tool: (typeof tools)[number]) => + tool === "get_call_graph" + ? { address: "0x1", direction: "forward" } + : { start: "0x1", goal: "0x4", direction: "forward" }; + +const cleanupFailure = () => + new AnalysisCapabilityUnavailableError( + provider.id, + "procedure_callees", + "child failed", + { cleanup: { reason: "worker cleanup failed", resources: ["worker"] } }, + ); + +/** 0x1 calls 0x2 and 0x3; 0x2's read is supplied by the test. */ +const port = ( + reads: string[], + second: (signal?: AbortSignal) => AnalysisError, +): AnalysisOperationPort => ({ + execute: async (operation, parameters, options) => { + const procedure = String(parameters.procedure); + if (operation === "procedure_address") + return ok(createAnalysisExecution(procedure, provider)); + reads.push(procedure); + if (procedure === "0x2") return err(second(options?.signal)); + return ok( + createAnalysisExecution( + procedure === "0x1" ? ["0x2", "0x3"] : ["0x4"], + provider, + { relationshipCoverage: { status: "complete" } }, + ), + ); + }, +}); + +describe.each(tools)("%s cleanup ownership", (tool) => { + it("returns the provider's cleanup failure so its owner resolves the outcome", async () => { + const failure = cleanupFailure(); + let released = false; + const lease = new AnalysisCleanupLease(async () => { + if (!released) throw new Error("still owned"); + }, ["worker"]); + lease.bindCleanupError(failure); + failure.bindCleanupRetry({ operation: "close_binary" }); + const reads: string[] = []; + + const result = await new EnhancedTools(port(reads, () => failure)).execute( + tool, + input(tool), + ); + + if (result.ok) throw new Error("expected incomplete cleanup"); + expect(result.error).toBe(failure); + expect("partialResult" in result).toBe(true); + expect(reads).toEqual(["0x1", "0x2"]); + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "cleanup_incomplete", + recovery: { kind: "retry_cleanup", operation: "close_binary" }, + }); + await expect(lease.release()).rejects.toThrow("still owned"); + expect(projectAnalysisError(result.error).code).toBe("cleanup_incomplete"); + released = true; + await lease.release(); + expect(projectAnalysisError(result.error).code).toBe( + "capability_unavailable", + ); + }); + + it("keeps a cleanup failure primary when cancellation arrives with it", async () => { + const controller = new AbortController(); + const failure = cleanupFailure(); + const reads: string[] = []; + + const result = await new EnhancedTools( + port(reads, () => { + controller.abort({ requested: "stop" }); + return failure; + }), + ).execute(tool, input(tool), controller.signal); + + if (result.ok) throw new Error("expected incomplete cleanup"); + expect(result.error).toBe(failure); + expect(projectAnalysisError(result.error).code).toBe("cleanup_incomplete"); + expect(reads).toEqual(["0x1", "0x2"]); + }); + + it("stops at a provider cancellation whose own cleanup is unconfirmed", async () => { + const controller = new AbortController(); + const reads: string[] = []; + let cancellation: AnalysisCancelledError | undefined; + + const result = await new EnhancedTools( + port(reads, (signal) => { + controller.abort(); + cancellation = new AnalysisCancelledError("procedure_callees", { + cause: signal?.reason, + cleanup: { reason: "cancel cleanup failed", resources: ["worker"] }, + }); + return cancellation; + }), + ).execute(tool, input(tool), controller.signal); + + if (result.ok) throw new Error("expected cancellation"); + expect(result.error).toBe(cancellation); + expect(projectAnalysisError(result.error).code).toBe("cleanup_incomplete"); + expect(reads).toEqual(["0x1", "0x2"]); + }); +}); diff --git a/src/application/CallRelationshipTraversal.ts b/src/application/CallRelationshipTraversal.ts index 7b6cfea6c0..6c53ad2da0 100644 --- a/src/application/CallRelationshipTraversal.ts +++ b/src/application/CallRelationshipTraversal.ts @@ -1,5 +1,14 @@ import type { CallRelationshipDirection } from "../contracts/enhancedInputs.js"; -import type { AnalysisOperation } from "./AnalysisProvider.js"; +import type { + AnalysisExecution, + AnalysisOperation, + ExecutionOptions, +} from "./AnalysisProvider.js"; +import type { AnalysisError } from "../domain/analysisErrorBase.js"; +import type { AnalysisOutputError } from "../domain/analysisErrorCore.js"; +import type { JsonValue } from "../domain/jsonValue.js"; +import { parseRelatedAddresses } from "../domain/native/nativeObservation.js"; +import type { Result } from "../domain/result.js"; /** The provider read and response relation for each supported traversal direction. */ export const CALL_RELATIONSHIP_TRAVERSALS = { @@ -14,3 +23,65 @@ export const CALL_RELATIONSHIP_TRAVERSALS = { } > >; + +/** One observed provider operation, as composed workflows dispatch it. */ +export type CallRelationshipObserver = ( + name: AnalysisOperation, + arguments_: Readonly>, + signal?: AbortSignal, + selectorOrigin?: ExecutionOptions["selectorOrigin"], +) => Promise>; + +/** + * One procedure's relationship read. An unavailable or unreadable read leaves + * the rest of the traversal usable; a stopped read must end it. + */ +export type CallRelationshipRead = + | { + readonly status: "observed"; + readonly execution: AnalysisExecution; + readonly related: readonly string[]; + } + | { readonly status: "unavailable"; readonly error: AnalysisError } + | { readonly status: "unreadable"; readonly error: AnalysisOutputError } + | { readonly status: "stopped"; readonly error: AnalysisError }; + +/** + * Whether a failed child read must end a traversal rather than become one of + * its unknowns. A cancelled read ends it; so does a read whose provider cannot + * confirm release of resources it owns, because that read has not finished and + * dispatching more reads would build on that state. + */ +export const stopsTraversal = (error: AnalysisError): boolean => + error._tag === "AnalysisCancelledError" || error.cleanupIncomplete; + +/** Read and parse one procedure's direct relationships for a traversal. */ +export const readCallRelationships = async ( + observe: CallRelationshipObserver, + direction: CallRelationshipDirection, + procedure: string, + signal?: AbortSignal, +): Promise => { + const { operation, relation } = CALL_RELATIONSHIP_TRAVERSALS[direction]; + const execution = await observe(operation, { procedure }, signal, "provider"); + if (!execution.ok) + return { + status: stopsTraversal(execution.error) ? "stopped" : "unavailable", + error: execution.error, + }; + const related = parseRelatedAddresses( + execution.value.result, + relation, + operation, + ); + return related.ok + ? { status: "observed", execution: execution.value, related: related.value } + : { status: "unreadable", error: related.error }; +}; + +/** The admitted traversal's caveat when unconfirmed cleanup stopped it. */ +export const cleanupStoppedTraversal = ( + procedure: string, + unexpanded: number, +): string => + `Traversal stopped at ${procedure} because the provider did not confirm cleanup of resources it owns; ${unexpanded === 1 ? "1 discovered procedure was" : `${unexpanded} discovered procedures were`} not expanded.`; diff --git a/src/application/CapabilityInventory.test.ts b/src/application/CapabilityInventory.test.ts index 6c965c26c9..089685cd33 100644 --- a/src/application/CapabilityInventory.test.ts +++ b/src/application/CapabilityInventory.test.ts @@ -8,6 +8,7 @@ import { compactToolAvailability, } from "./CapabilityInventory.js"; import { TOOL_CONTRACTS } from "../contracts/toolContracts.js"; +import { cleanupRetryRemediation } from "../domain/analysisErrorRecovery.js"; import { expandToolAvailability, toolAvailabilityInventory, @@ -52,6 +53,29 @@ const enabledPolicy: Parameters[1] = { v8InspectorObservationEnabled: true, }; +it("adds the retaining close owner's prerequisite without guessing from a pending flag", () => { + const retry = { operation: "close_binary" } as const; + const input = { open: false, capabilities: [], cleanup_pending: true }; + const unavailable = buildCapabilityInventory( + { ...input, cleanup_retry: retry }, + enabledPolicy, + ).find(({ name }) => name === "procedure_assembly"); + expect(unavailable).toMatchObject({ + available: false, + reason: "target_required", + remediation: expect.stringContaining(cleanupRetryRemediation(retry)), + }); + const unknownOwner = buildCapabilityInventory(input, enabledPolicy).find( + ({ name }) => name === "procedure_assembly", + ); + const original = buildCapabilityInventory( + { open: false, capabilities: [] }, + enabledPolicy, + ).find(({ name }) => name === "procedure_assembly"); + expect(unknownOwner).toEqual(original); + expect(unknownOwner?.reason).toBe("target_required"); +}); + const status = ( options: { readonly open?: boolean; @@ -184,6 +208,7 @@ describe("ADB and Apktool host availability", () => { status: "unavailable" as const, code: "not_configured" as const, reason: "REA_ADB_PATH is invalid", + remediation: "Set REA_ADB_PATH to an absolute adb executable path.", diagnostics: {}, }; const readyInventory = buildCapabilityInventory(status(), { @@ -205,6 +230,7 @@ describe("ADB and Apktool host availability", () => { status: "unavailable", code: "unsupported_version", reason: "Apktool 2.6.3 is older than 2.7.0", + remediation: "Repair the fixture provider.", diagnostics: {}, }, }); @@ -227,6 +253,7 @@ describe("capability failure reasons", () => { status: "unavailable", code: "runtime_unavailable", reason: "The JADX native self-test failed.", + remediation: "Repair the fixture provider.", diagnostics: {}, }, }); diff --git a/src/application/CapabilityInventory.ts b/src/application/CapabilityInventory.ts index 6fef4ab20c..b296483882 100644 --- a/src/application/CapabilityInventory.ts +++ b/src/application/CapabilityInventory.ts @@ -42,6 +42,11 @@ import type { JebAnalysisAvailability } from "./jeb/JebAnalysisPort.js"; import { FRIDA_TOOL_CONTRACTS } from "../contracts/fridaToolContracts.js"; import { TOOL_CONTRACTS } from "../contracts/toolContracts.js"; import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; +import { + analysisCleanupRetrySchema, + cleanupRetryRemediation, + type AnalysisCleanupRetry, +} from "../domain/analysisErrorRecovery.js"; import { isBundleResourceOperation } from "./binary/BundleResourceSession.js"; const FRIDA_OPERATION_NAMES: ReadonlySet = new Set( @@ -50,6 +55,7 @@ const FRIDA_OPERATION_NAMES: ReadonlySet = new Set( const statusSchema = z.object({ open: z.boolean(), + cleanup_retry: analysisCleanupRetrySchema.optional(), session_kind: z.literal("bundle_resources").optional(), kind: z.enum(["executable", "database", "archive", "artifact"]).optional(), format: z.string().optional(), @@ -61,6 +67,7 @@ const statusSchema = z.object({ type ToolAvailabilityReason = "available" | ToolUnavailabilityReason; type ProviderDescriptor = ProviderCapability; +type UnavailableDescriptor = Extract; /** Optional integrations whose verified hosts exclude some platforms. */ export const HOST_RESTRICTED_FEATURES = [ @@ -126,10 +133,15 @@ type AvailabilityModeResult = | (Extract & { readonly reason: ToolUnavailabilityReason; }); + +/** Recovery when unavailable modes require different actions. */ +export const MULTI_MODE_REMEDIATION = + "No mode is currently advertised as available. Inspect each mode's missing operations and remediation."; type AvailabilityContext = { readonly name: string; readonly kind: ToolKind; readonly targetOpen: boolean; + readonly cleanupRetry: AnalysisCleanupRetry | undefined; readonly resourceSession: boolean; readonly requiresActiveTarget: boolean; readonly targetKind: @@ -258,6 +270,7 @@ export const buildCapabilityInventory = ( name: contract.name, kind: contract.kind, targetOpen: status.open, + cleanupRetry: status.cleanup_retry, resourceSession: status.session_kind === "bundle_resources", requiresActiveTarget: "requiresActiveTarget" in contract && @@ -572,7 +585,7 @@ const workflowAvailabilityFor = ({ if (profile.reason !== "available") return profile; const limiter = policy.evmLimiterAvailability; if (limiter?.status === "unavailable") - return { reason: "provider_missing", remediation: limiter.reason }; + return { reason: "provider_missing", remediation: limiter.remediation }; return { reason: "available", remediation: null }; } if (name === "inspect_recorded_crash") @@ -606,7 +619,7 @@ const workflowAvailabilityFor = ({ ? "provider_unavailable" : "provider_missing", remediation: - policy.androidAnalysisAvailability?.reason ?? + policy.androidAnalysisAvailability?.remediation ?? "Set REA_JADX_MCP_JAR to a caller-supplied jadx-headless-mcp 0.7.1 JAR and provide a full JDK on Linux, macOS, or Windows x64 with its matching bundled native controls.", }; if (kind === "adb-provider") @@ -698,7 +711,7 @@ const ownerProbeAvailability = ( : observation.code === "not_configured" ? "provider_missing" : "provider_unavailable", - remediation: observation.reason, + remediation: observation.remediation, }; }; @@ -762,7 +775,10 @@ const hostAvailability = ({ const descriptor = descriptors.get(name); return descriptor?.available === false && descriptor.availability_code === "unsupported_host" - ? { reason: "unsupported_host", remediation: descriptor.reason } + ? { + reason: "unsupported_host", + remediation: descriptorRemediation(descriptor), + } : null; }; @@ -772,22 +788,30 @@ const targetAvailability = ({ targetKind, targetFormat, targetOpen, + cleanupRetry, requiresActiveTarget, descriptors, }: AvailabilityContext): Availability | null => { if (!targetOpen && requiresActiveTarget) return { reason: "target_required", - remediation: "Call open_binary with a supported local target.", + remediation: + cleanupRetry === undefined + ? "Call open_binary with a supported local target." + : `${cleanupRetryRemediation(cleanupRetry)} After cleanup succeeds, call open_binary with a supported local target.`, }; const descriptor = descriptors.get(name); if ( targetOpen && + kind !== "enhanced" && descriptor?.available === false && (descriptor.availability_code === "target_kind_unsupported" || descriptor.availability_code === "target_format_unsupported") ) - return { reason: "target_unsupported", remediation: descriptor.reason }; + return { + reason: "target_unsupported", + remediation: descriptorRemediation(descriptor), + }; if ( targetKind !== undefined && targetKind !== "executable" && @@ -836,11 +860,19 @@ const providerAvailability = ({ ? { reason: "available", remediation: null } : { reason: "provider_unavailable", - remediation: - descriptor.reason ?? "Choose another target or configured provider.", + remediation: descriptorRemediation(descriptor), }; }; +/** + * Prefer the readiness owner's repair instruction. A static capability fact + * declares no separate repair, so its reason remains the only guidance. + */ +const descriptorRemediation = (descriptor: UnavailableDescriptor): string => + descriptor.remediation ?? + descriptor.reason ?? + "Choose another target or configured provider."; + const composedAvailability = ( name: string, descriptors: ReadonlyMap, @@ -887,7 +919,8 @@ const composedAvailabilityFor = ( (descriptor) => descriptor?.available === true, ), unavailable: observations.find( - (descriptor) => descriptor?.available === false, + (descriptor): descriptor is UnavailableDescriptor => + descriptor?.available === false, ), }; }) @@ -918,8 +951,7 @@ const composedAvailabilityFor = ( unavailable.availability_code === "unsupported_host" ? "unsupported_host" : "provider_unavailable", - remediation: - unavailable.reason ?? "Choose another target or configured provider.", + remediation: descriptorRemediation(unavailable), }, }; return { @@ -980,6 +1012,10 @@ const availabilityForModes = ( )[0]; const specificFailureRemediation = failure?.reason === "provider_missing" ? undefined : failure?.remediation; + const distinctModeRemediations = + defaultMode === undefined && + failures.length > 1 && + new Set(failures.map((mode) => mode.remediation)).size > 1; const facts = { ...(defaultMode === undefined ? {} @@ -997,9 +1033,10 @@ const availabilityForModes = ( return { ...facts, reason: failure?.reason ?? "provider_missing", - remediation: - specificFailureRemediation ?? - `Configure a provider for one of: ${missingOperations.join(", ")}.`, + remediation: distinctModeRemediations + ? MULTI_MODE_REMEDIATION + : (specificFailureRemediation ?? + `Configure a provider for one of: ${missingOperations.join(", ")}.`), }; }; diff --git a/src/application/DirectAnalysis.ts b/src/application/DirectAnalysis.ts index 9ef8010a6b..70935557c5 100644 --- a/src/application/DirectAnalysis.ts +++ b/src/application/DirectAnalysis.ts @@ -35,15 +35,14 @@ import { AnalysisSnapshotMismatchError, type PublishedEvidenceFile, } from "../domain/evidenceErrors.js"; -import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import type { AnalysisError } from "../domain/analysisErrorBase.js"; import { access, stat } from "node:fs/promises"; import { absoluteCallerPath } from "../filesystem/callerPath.js"; import { requiresNativeSourceAdmissionBeforeReplay } from "./binary/AnalysisSnapshotCache.js"; import { classifyBinaryTargetFilesystemFailure } from "./BinaryTargetResolver.js"; import { + snapshotBindingMismatch, snapshotEvidenceForQuery, - snapshotMatchesTarget, } from "../domain/analysisSnapshot.js"; import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js"; import { profilelessNativeDispatchProfile } from "./binary/ProfilelessNativeWorkflowSnapshot.js"; @@ -60,6 +59,7 @@ import { composedLimitationBudget, executeComposedWorkflow, createWorkflowEvidence, + retainPartialWorkflowEvidence, workflowSnapshotRecord, recordWorkflowUnknowns, } from "./WorkflowEvidence.js"; @@ -74,6 +74,7 @@ import type { BinaryTargetError } from "../domain/configurationErrors.js"; import { directAnalysisCleanupFailure, projectDirectAnalysisFailure, + projectInvocationFailure, managedAnalysisCleanupFailure, withSessionCleanup, } from "./DirectAnalysisCleanup.js"; @@ -119,7 +120,10 @@ const ARTIFACT_ANALYSIS_OPERATION_SET: ReadonlySet = new Set( /** * Open one binary, execute one tool, and always release provider resources. * Unlike MCP mode, every CLI invocation is intentionally isolated and does not - * retain a target or provider client for a subsequent command. + * retain a target, provider session or runtime setting for a subsequent + * command. Failures are projected for that invocation lifetime, so they never + * offer same-session reuse or runtime updates; a new invocation's launch + * environment or arguments carry any changed limit. */ export const runDirectAnalysis = async ( dependencies: DirectAnalysisDependencies, @@ -624,6 +628,12 @@ const executeAnalysisTool = async (input: { new EnhancedTools(observed).execute(tool, input.arguments, signal), composedLimitationBudget(tool), ); + retainPartialWorkflowEvidence(result, { + target: input.openedTarget, + operation: tool, + parameters: input.evidenceParameters, + upstreamProfile: evidenceProfile, + }); result = retainAnalysisResultSources( tool, (source) => session.recordEvidence(source), @@ -742,10 +752,16 @@ const prepareSnapshot = async (options: { ...targetSelection(options), }); if (!target.ok) return target; - if (!snapshotMatchesTarget(loaded.value.target, target.value)) + const mismatch = snapshotBindingMismatch( + loaded.value, + target.value, + undefined, + ); + if (mismatch?.target !== undefined) return err( new AnalysisSnapshotMismatchError( "Analysis snapshot target does not match the requested binary", + mismatch, ), ); return ok({ snapshot: loaded.value, target: target.value }); @@ -849,6 +865,6 @@ const cliError = ( ): JsonValue => ({ error: "Analysis failed", ...(callerInputPath === undefined - ? projectAnalysisError(error) + ? projectInvocationFailure(error) : projectDirectAnalysisFailure(error, callerInputPath)), }); diff --git a/src/application/DirectAnalysisCleanup.ts b/src/application/DirectAnalysisCleanup.ts index f8d901e5fd..003b09e86e 100644 --- a/src/application/DirectAnalysisCleanup.ts +++ b/src/application/DirectAnalysisCleanup.ts @@ -1,5 +1,12 @@ import type { AnalysisError } from "../domain/analysisErrorBase.js"; -import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; +import { + childFailureDiagnostics, + projectAnalysisError, +} from "../domain/analysisErrorProjection.js"; +import { + analysisErrorUserMessage, + composedCallerGuidance, +} from "../domain/analysisErrorPresentation.js"; import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js"; import { ProviderAdapterError } from "../domain/providerAdapterError.js"; import { err, type Result } from "../domain/result.js"; @@ -11,12 +18,21 @@ import { withEvidenceCallerInputPath, } from "../domain/evidence.js"; +/** + * Project a one-shot failure for its invocation, which has released its + * provider session and runtime settings by the time the caller reads it. + */ +export const projectInvocationFailure = ( + error: AnalysisError, +): ReturnType => + projectAnalysisError(error, "invocation"); + /** Add the actual outer request path to authenticated partial Evidence. */ export const projectDirectAnalysisFailure = ( error: AnalysisError, callerInputPath: string, ): ReturnType => { - const projection = projectAnalysisError(error); + const projection = projectInvocationFailure(error); const observation = tryParseEvidence(projection.details?.partial_observation); return observation === undefined ? projection @@ -63,7 +79,7 @@ export const directAnalysisCleanupFailure = ( ): JsonValue => { const project = (failure: AnalysisError) => callerInputPath === undefined - ? projectAnalysisError(failure) + ? projectInvocationFailure(failure) : projectDirectAnalysisFailure(failure, callerInputPath); const cleanup = project(error); if (!result.ok) { @@ -114,18 +130,20 @@ export const managedAnalysisCleanupFailure = ( : failure, ); } - const cleanup = projectAnalysisError(error); const failure = new ProviderAdapterError( error instanceof ProviderAdapterError ? error.providerId : "rea", "close_binary", { cause: error, - remediationAction: cleanup.remediation.action, - cleanup: { reason: cleanup.message, resources: error.cleanupResources }, - diagnostics: { - cleanup_error: cleanup, - partial_observation: jsonValueSchema.parse(result.value), + ...composedCallerGuidance({ remediation: error }), + cleanup: { + reason: analysisErrorUserMessage(error, "invocation"), + resources: error.cleanupResources, }, + ...childFailureDiagnostics( + { cleanup_error: { error } }, + { partial_observation: jsonValueSchema.parse(result.value) }, + ), }, ); return err( diff --git a/src/application/DirectAnalysisStatus.ts b/src/application/DirectAnalysisStatus.ts index 31d624644f..8214f5b203 100644 --- a/src/application/DirectAnalysisStatus.ts +++ b/src/application/DirectAnalysisStatus.ts @@ -1,4 +1,3 @@ -import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import { jsonObjectSchema, type JsonValue } from "../domain/jsonValue.js"; import { createServerIdentity } from "../serverIdentity.js"; import { silentLogger } from "../logger.js"; @@ -12,6 +11,7 @@ import { catalogIdentityForToolFamilies } from "../catalogIdentity.js"; import { compactPresentationForMcpInputSchemaProfile } from "../config/mcpInputSchemaProfile.js"; import { directAnalysisCleanupFailure, + projectInvocationFailure, withSessionCleanup, } from "./DirectAnalysisCleanup.js"; import { ok } from "../domain/result.js"; @@ -25,7 +25,7 @@ export const runSessionStatus = async ( logger: Logger = silentLogger, ): Promise => { const config = dependencies.readConfiguration(); - if (!config.ok) return { error: projectAnalysisError(config.error) }; + if (!config.ok) return { error: projectInvocationFailure(config.error) }; const session = await dependencies.createBinarySession(config.value, logger); return withSessionCleanup( session, diff --git a/src/application/EnhancedToolTypes.ts b/src/application/EnhancedToolTypes.ts index 006313b250..686a7bab83 100644 --- a/src/application/EnhancedToolTypes.ts +++ b/src/application/EnhancedToolTypes.ts @@ -10,7 +10,23 @@ import { type EnhancedToolName, } from "../contracts/enhancedInputs.js"; -export type EnhancedResult = Promise>; +/** A failure that stopped a workflow after it admitted part of its result. */ +export interface PartialEnhancedFailure { + readonly ok: false; + readonly error: AnalysisError; + /** The workflow's own result shape, holding only work admitted before the stop. */ + readonly partialResult: JsonValue; +} +export type EnhancedOutcome = + | Result + | PartialEnhancedFailure; +export type EnhancedResult = Promise; + +/** Keep the stopping failure primary while retaining the admitted result. */ +export const failedWithPartialResult = ( + error: AnalysisError, + partialResult: JsonValue, +): PartialEnhancedFailure => ({ ok: false, error, partialResult }); export type TraceReference = { readonly target_address: string; readonly source_address: string; diff --git a/src/application/EnhancedTools.ts b/src/application/EnhancedTools.ts index 4abca5d938..a01fb6546b 100644 --- a/src/application/EnhancedTools.ts +++ b/src/application/EnhancedTools.ts @@ -1,8 +1,15 @@ import { permitsCapabilityFallback } from "./CapabilityFallback.js"; import { withOverviewInventoryCoverage } from "../domain/native/inventoryCoverage.js"; import type { NativeCallGraph } from "../domain/native/nativeCallGraph.js"; +import type { + BatchDecompileItem, + BatchProcedureIdentity, +} from "../domain/native/batchDecompile.js"; import { nativeProcedureInputSchema } from "../domain/native/nativeProcedureInput.js"; -import { CALL_RELATIONSHIP_TRAVERSALS } from "./CallRelationshipTraversal.js"; +import { + cleanupStoppedTraversal, + readCallRelationships, +} from "./CallRelationshipTraversal.js"; import { z } from "zod"; import type { @@ -28,7 +35,6 @@ import { parseNativeInventoryCount, addressedValueSchema, } from "../domain/native/nativeInventory.js"; -import { parseRelatedAddresses } from "../domain/native/nativeObservation.js"; import { parseSegments, projectBinaryOverview, @@ -59,6 +65,7 @@ import { } from "../domain/apple/machoTargetSelection.js"; import { + failedWithPartialResult, invalidEnhancedInput, type EnhancedResult, type ValidatedEnhancedCall, @@ -73,18 +80,6 @@ import { import { traceLiteralFeature } from "./EnhancedLiteralTracing.js"; import { projectNativeApiInspection } from "./native/NativeApiInspection.js"; -type BatchProcedureIdentity = - | { - readonly status: "resolved"; - readonly address: string; - readonly name: string | null; - readonly name_error?: ReturnType; - } - | { - readonly status: "unknown"; - readonly error: ReturnType; - }; - type BatchResolution = { readonly address: string; readonly result: Result; @@ -96,16 +91,6 @@ type BatchReadyItem = { readonly providerProcedure: string; }; -type BatchDecompileItem = { - readonly address: string; - readonly procedure: BatchProcedureIdentity; - readonly status: "ok" | "error"; - readonly pseudocode?: string; - readonly error?: ReturnType; - /** Caveats of this item's own observation when it supplied no pseudocode. */ - readonly limitations?: string[]; -}; - /** * Guidance for a batch item whose provider returned null pseudocode, by * whether that observation reported caveats for the item to carry. @@ -615,8 +600,6 @@ export class EnhancedTools { }, signal?: AbortSignal, ): EnhancedResult { - const { relation, operation: tool } = - CALL_RELATIONSHIP_TRAVERSALS[input.direction]; const limitations = new Set(); const resolved = await this.#observe( "procedure_address", @@ -635,40 +618,41 @@ export class EnhancedTools { ]; let queueIndex = 0; const graph: Record = {}; + const sortedLimitations = () => + [...limitations].sort(compareUnicodeCodePoints); while (queueIndex < queue.length) { const current = queue[queueIndex++]; if (current === undefined) continue; - const level = String(current.depth); - graph[level] ??= []; - - const result = await this.#observe( - tool, - { procedure: current.address }, + const read = await readCallRelationships( + this.#observe.bind(this), + input.direction, + current.address, signal, - "provider", ); - if (!result.ok) { - if (isAnalysisCancellation(result.error)) return result; - graph[level].push({ + if (read.status === "stopped") { + if (!read.error.cleanupIncomplete) return err(read.error); + limitations.add( + cleanupStoppedTraversal(current.address, queue.length - queueIndex), + ); + return failedWithPartialResult(read.error, { + graph, + limitations: sortedLimitations(), + }); + } + const nodes = (graph[String(current.depth)] ??= []); + if (read.status !== "observed") { + nodes.push({ address: current.address, status: "error", - error: projectAnalysisError(result.error), + error: projectAnalysisError(read.error), } satisfies NativeCallGraph["graph"][string][number]); continue; } - const related = parseRelatedAddresses(result.value.result, relation); - if (!related.ok) { - graph[level].push({ - address: current.address, - status: "error", - error: projectAnalysisError(related.error), - } satisfies NativeCallGraph["graph"][string][number]); - continue; - } - for (const limitation of result.value.limitations) + const { execution, related } = read; + for (const limitation of execution.limitations) limitations.add(limitation); - const reportedCoverage = result.value.relationshipCoverage; + const reportedCoverage = execution.relationshipCoverage; const coverage = reportedCoverage ?? { status: "unknown" as const, reason: "Provider did not report relationship coverage.", @@ -676,7 +660,7 @@ export class EnhancedTools { const relationshipLimitations = reportedCoverage === undefined ? [ - ...result.value.limitations, + ...execution.limitations, "Provider did not report relationship coverage.", ] : [ @@ -692,26 +676,23 @@ export class EnhancedTools { const nodeLimitations = [...new Set(relationshipLimitations)].sort( compareUnicodeCodePoints, ); - graph[level].push({ + nodes.push({ address: current.address, status: "ok", - calls: [...related.value], + calls: [...related], relationship_coverage: coverage, ...(nodeLimitations.length === 0 ? {} : { limitations: nodeLimitations }), } satisfies NativeCallGraph["graph"][string][number]); - for (const address of related.value) { + for (const address of related) { if (!discovered.has(address)) { discovered.add(address); queue.push({ address, depth: current.depth + 1 }); } } } - return ok({ - graph, - limitations: [...limitations].sort(compareUnicodeCodePoints), - }); + return ok({ graph, limitations: sortedLimitations() }); } async #analyzeSwiftTypes( diff --git a/src/application/EvidenceBundleCommands.ts b/src/application/EvidenceBundleCommands.ts index b103089cbf..d88e555dd5 100644 --- a/src/application/EvidenceBundleCommands.ts +++ b/src/application/EvidenceBundleCommands.ts @@ -212,6 +212,11 @@ export const publishEvidenceDelivery = async (input: { return ok({ delivered: input.delivered, evidenceBundle: written.value }); }; +/** A delivery failure can follow commit; its receipt then owns the source closure. */ +export const evidenceDeliveryCommitted = (error: AnalysisError): boolean => + error.diagnostics?.evidence_bundle !== undefined || + (error instanceof EvidenceFileError && error.published !== undefined); + /** Publish a completed bundle and retry its exact staged-output cleanup owner. */ const persistEvidenceBundleCommand = async ( bundle: EvidenceBundle, @@ -292,8 +297,10 @@ const projectWrite = ( export const inspectEvidenceBundleCommand = async ( input: { readonly path: string; - readonly detail: "complete" | "summary"; + readonly detail: "complete" | "summary" | "records"; readonly filters?: EvidenceRecordFilters | undefined; + readonly offset?: number | undefined; + readonly limit?: number | undefined; }, signal?: AbortSignal, ) => { diff --git a/src/application/HopperDownload.ts b/src/application/HopperDownload.ts index efe25c32b9..8d3af8d25b 100644 --- a/src/application/HopperDownload.ts +++ b/src/application/HopperDownload.ts @@ -1,14 +1,51 @@ -/** Download result; HTTP failures may carry their observed status for diagnostics. */ -export interface HopperDownload { - readonly ok: boolean; - readonly bytes: Uint8Array; - readonly httpStatus?: number; -} +import { sanitizeBrowserUrl } from "../domain/browserObservation.js"; +import { err, ok, type Result } from "../domain/result.js"; + +/** Why a URL was refused before any vendor response could be admitted. */ +export type HopperUrlRefusal = + | "invalid_url" + | "non_https" + | "credentials" + | "redirect_without_location" + | "redirect_limit"; + +/** + * Transport facts for one Hopper request that yielded no admitted bytes. + * `url` is the caller-selected request with any URL userinfo removed, never a + * redirect target. URLs carrying credentials are refused before any request, + * so no transport detail can contain them. + */ +export type HopperDownloadFailure = { readonly url: string } & ( + | { + readonly kind: "http_status"; + /** Absent when the response did not carry a valid HTTP failure status. */ + readonly httpStatus?: number; + } + | { readonly kind: "timeout"; readonly timeoutMs: number } + | { readonly kind: "network"; readonly detail: string } + | { readonly kind: "response_too_large"; readonly maxBytes: number } + | { readonly kind: "url_refused"; readonly refusal: HopperUrlRefusal } +); + +/** Admitted response bytes, or the observed reason none were admitted. */ +export type HopperDownload = Result; + +/** The vendor request a Hopper workflow made; identifies the failed stage. */ +export type HopperVendorRequest = "release_feed" | "package" | "public_key"; + +/** A failed download attributed to the workflow stage that requested it. */ +export type HopperRequestFailure = HopperDownloadFailure & { + readonly request: HopperVendorRequest; +}; /** Maximum admitted vendor response bytes, shared by Hopper setup adapters. */ export const HOPPER_DOWNLOAD_BYTES = 100_000_000; -/** Bound observed response bytes before retaining them, including chunked bodies. */ +/** + * Bound observed response bytes before retaining them, including chunked bodies. + * Expected transport outcomes are returned; caller cancellation still rejects + * with the caller's abort reason so cancellation stays distinct from failure. + */ export const downloadHopperPackage = async ( url: string, options: { @@ -20,7 +57,6 @@ export const downloadHopperPackage = async ( const timeoutMs = options.timeoutMs ?? 120_000; if (!Number.isSafeInteger(timeoutMs) || timeoutMs < 1 || timeoutMs > 120_000) throw new RangeError("Invalid Hopper response time budget"); - const deadline = AbortSignal.timeout(timeoutMs); const maxBytes = options.maxBytes ?? HOPPER_DOWNLOAD_BYTES; if ( !Number.isSafeInteger(maxBytes) || @@ -28,37 +64,69 @@ export const downloadHopperPackage = async ( maxBytes > HOPPER_DOWNLOAD_BYTES ) throw new RangeError("Invalid Hopper response byte budget"); - const response = await fetchHopperResponse( - url, - options.signal === undefined - ? deadline - : AbortSignal.any([options.signal, deadline]), - ); - const refused = { ok: false, bytes: new Uint8Array() }; - if (response === undefined) return refused; + const deadline = AbortSignal.timeout(timeoutMs); + // Remove transport credentials once, where every failure is created. + const reported = sanitizeBrowserUrl(url).url; + try { + return await readHopperResponse( + url, + reported, + options.signal === undefined + ? deadline + : AbortSignal.any([options.signal, deadline]), + maxBytes, + ); + } catch (cause: unknown) { + if (options.signal?.aborted === true) throw cause; + if (deadline.aborted) + return err({ url: reported, kind: "timeout", timeoutMs }); + // Fetch reports network errors (DNS, TLS, reset, truncated body) as TypeError. + if (cause instanceof TypeError) + return err({ + url: reported, + kind: "network", + detail: networkDetail(cause), + }); + throw cause; + } +}; + +const readHopperResponse = async ( + requested: string, + url: string, + signal: AbortSignal, + maxBytes: number, +): Promise => { + const fetched = await fetchHopperResponse(requested, signal); + if (!fetched.ok) + return err({ url, kind: "url_refused", refusal: fetched.error }); + const response = fetched.value; if (!response.ok) { - await response.body?.cancel(); - return Number.isInteger(response.status) && - response.status >= 300 && - response.status <= 599 - ? { ...refused, httpStatus: response.status } - : refused; + await releaseBody(response.body); + return err( + Number.isInteger(response.status) && + response.status >= 300 && + response.status <= 599 + ? { url, kind: "http_status", httpStatus: response.status } + : { url, kind: "http_status" }, + ); } + const tooLarge = { url, kind: "response_too_large", maxBytes } as const; if (Number(response.headers.get("content-length")) > maxBytes) { - await response.body?.cancel(); - return refused; + await releaseBody(response.body); + return err(tooLarge); } - if (response.body === null) return { ok: true, bytes: new Uint8Array() }; + if (response.body === null) return ok(new Uint8Array()); const reader = response.body.getReader(); const chunks: Uint8Array[] = []; let bytes = 0; try { while (true) { const chunk = await reader.read(); - if (chunk.done) return { ok: true, bytes: Buffer.concat(chunks, bytes) }; + if (chunk.done) return ok(Buffer.concat(chunks, bytes)); if (chunk.value.byteLength > maxBytes - bytes) { - await reader.cancel(); - return refused; + await releaseBody(reader); + return err(tooLarge); } chunks.push(chunk.value); bytes += chunk.value.byteLength; @@ -68,6 +136,29 @@ export const downloadHopperPackage = async ( } }; +/** + * Release a body REA will not admit. A stream that already failed (for example + * a truncated error body) has nothing left to release, and its error must not + * replace the response outcome that was already observed. + */ +const releaseBody = async ( + body: { cancel(): Promise } | null, +): Promise => { + try { + await body?.cancel(); + } catch (cause: unknown) { + // best-effort cleanup: an errored stream is already released. + void cause; + } +}; + +const networkDetail = (cause: TypeError): string => { + const underlying = cause.cause instanceof Error ? cause.cause.message : ""; + return underlying === "" || underlying === cause.message + ? cause.message + : `${cause.message}: ${underlying}`; +}; + // HTTP-redirect fetch admits at most 20 redirects; retain that existing Fetch // bound while checking each destination before the next request. const FETCH_REDIRECT_LIMIT = 20; @@ -76,31 +167,34 @@ const REDIRECT_STATUSES = new Set([301, 302, 303, 307, 308]); const fetchHopperResponse = async ( url: string, signal: AbortSignal, -): Promise => { +): Promise> => { let selected: URL; try { selected = new URL(url); } catch { - return undefined; + return err("invalid_url"); } let requested = url; for (let redirects = 0; ; redirects++) { - if (selected.protocol !== "https:") return undefined; + if (selected.protocol !== "https:") return err("non_https"); + // Fetch cannot request userinfo and would echo it in its TypeError message. + if (selected.username !== "" || selected.password !== "") + return err("credentials"); const response = await fetch(requested, { redirect: "manual", headers: { accept: "application/json", "user-agent": "rea-installer" }, signal, }); - if (!REDIRECT_STATUSES.has(response.status)) return response; + if (!REDIRECT_STATUSES.has(response.status)) return ok(response); const location = response.headers.get("location"); - await response.body?.cancel(); - if (location === null || redirects === FETCH_REDIRECT_LIMIT) - return undefined; + await releaseBody(response.body); + if (location === null) return err("redirect_without_location"); + if (redirects === FETCH_REDIRECT_LIMIT) return err("redirect_limit"); let next: URL; try { next = new URL(location, selected); } catch { - return undefined; + return err("invalid_url"); } // Fetch inherits the previous fragment when Location has none. if (!location.includes("#")) next.hash = selected.hash; @@ -108,3 +202,64 @@ const fetchHopperResponse = async ( requested = next.href; } }; + +const requestSubjects: Readonly> = { + release_feed: "Hopper release feed", + package: "Hopper package", + public_key: "Hopper public key", +}; + +const refusals: Readonly> = { + invalid_url: "a request or redirect URL was not a valid absolute URL", + non_https: "a request or redirect URL did not use HTTPS", + credentials: "a request or redirect URL contained credentials", + redirect_without_location: "a redirect response omitted its Location", + redirect_limit: `the response redirected more than ${FETCH_REDIRECT_LIMIT} times`, +}; + +/** One sentence naming the failed request, its URL and the observed outcome. */ +export const describeHopperRequestFailure = ( + failure: HopperRequestFailure, +): string => { + const subject = `The official ${requestSubjects[failure.request]} request to ${failure.url}`; + switch (failure.kind) { + case "http_status": + return failure.httpStatus === undefined + ? `${subject} received an unsuccessful HTTP response without a valid status.` + : `${subject} returned HTTP ${failure.httpStatus}.`; + case "timeout": + return `${subject} did not complete within ${failure.timeoutMs / 1000} seconds.`; + case "network": + return `${subject} failed with a network error (${failure.detail}).`; + case "response_too_large": + return `${subject} exceeded REA's ${failure.maxBytes}-byte response limit.`; + case "url_refused": + return `${subject} was refused because ${refusals[failure.refusal]}.`; + } +}; + +/** Next step for a failed request, owned with the transport facts that select it. */ +export const hopperRequestRecovery = ( + failure: HopperDownloadFailure, +): string => { + switch (failure.kind) { + case "http_status": { + const status = failure.httpStatus; + if (status === 404 || status === 410) + return "The vendor reports this file as missing, so an immediate retry is unlikely to help; check for a newer REA release, then retry."; + if ( + status === 408 || + status === 429 || + (status !== undefined && status >= 500) + ) + return "The vendor service reported a temporary failure; wait, then retry."; + return "Check proxy, firewall or network policy for this request, then retry."; + } + case "timeout": + case "network": + return "Check network, DNS, proxy and TLS access to this URL, then retry."; + case "response_too_large": + case "url_refused": + return "The vendor response did not meet REA's download limits; check for a newer REA release, then retry."; + } +}; diff --git a/src/application/HopperReleaseFeed.ts b/src/application/HopperReleaseFeed.ts index dae500bb04..e6da3d70d5 100644 --- a/src/application/HopperReleaseFeed.ts +++ b/src/application/HopperReleaseFeed.ts @@ -7,7 +7,11 @@ import { type SafeJsonParseResult, } from "../domain/safeJson.js"; import { err, ok, type Result } from "../domain/result.js"; -import { HOPPER_DOWNLOAD_BYTES } from "./HopperDownload.js"; +import { + describeHopperRequestFailure, + HOPPER_DOWNLOAD_BYTES, + type HopperRequestFailure, +} from "./HopperDownload.js"; import { urlDeclaresQueryOrFragment } from "../domain/urlComponents.js"; /** Vendor endpoints, shared by installation and existing-build inspection. */ @@ -86,6 +90,18 @@ export const parseMacHopperReleases = ( return ok(retained); }; +/** + * The minimum macOS version the selected release declares (`deployment_target`). + * Undefined when the row declares none as text; the signed application's own + * requirement is still checked before it is copied. + */ +export const macHopperReleaseMinimumSystemVersion = ( + release: MacHopperRelease, +): string | undefined => + typeof release.deployment_target === "string" + ? release.deployment_target + : undefined; + /** Stable publisher identity; signing subkeys are admitted through this primary. */ export const HOPPER_LINUX_PRIMARY_FINGERPRINT = "7C2D73BAFAF5B9A710A69DA1A26DF815099B1BCD"; @@ -100,6 +116,8 @@ const suffixes = { deb: ".deb", rpm: ".rpm", arch: ".pkg.tar.xz" } as const; export class HopperReleaseError extends Error { readonly _tag = "HopperReleaseError"; + /** The vendor request whose transport failure produced this error. */ + readonly download: HopperRequestFailure | undefined; constructor( readonly reason: | "release_metadata" @@ -112,10 +130,23 @@ export class HopperReleaseError extends Error { | "cleanup_failed" | "cache_missing", message: string, - options?: ErrorOptions, + options?: ErrorOptions & { readonly download?: HopperRequestFailure }, ) { super(message, options); this.name = this._tag; + this.download = options?.download; + } + + /** Describe a failed vendor request once, from the transport's observed facts. */ + static fromDownload( + reason: HopperReleaseError["reason"], + download: HopperRequestFailure, + ): HopperReleaseError { + return new HopperReleaseError( + reason, + describeHopperRequestFailure(download), + { download }, + ); } } diff --git a/src/application/JsonFiles.ts b/src/application/JsonFiles.ts index 1218671e41..93c104f902 100644 --- a/src/application/JsonFiles.ts +++ b/src/application/JsonFiles.ts @@ -20,7 +20,7 @@ import { AnalysisResourceConstraintError, } from "../domain/analysisErrorCore.js"; import { AnalysisError } from "../domain/analysisErrorBase.js"; -import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; +import { childFailureDiagnostics } from "../domain/analysisErrorProjection.js"; import { err, ok, type Result } from "../domain/result.js"; import { JsonInputPhysicalReadFailure, @@ -170,7 +170,7 @@ const readJsonFileInput = async ( cleanupPrimary: primary, ...(primary === undefined ? {} - : { diagnostics: { primary_error: projectAnalysisError(primary) } }), + : childFailureDiagnostics({ primary_error: { error: primary } })), }); bindRegularFileCleanup(cause, error); return err(error); @@ -272,7 +272,7 @@ const jsonFileCleanupError = ( ), cleanupPrimary: primary, cleanup: cleanup.cleanup, - diagnostics: { primary_error: projectAnalysisError(primary) }, + ...childFailureDiagnostics({ primary_error: { error: primary } }), }).bindCleanupDependencies([primary, cleanup]); }; diff --git a/src/application/LinuxHopper.test.ts b/src/application/LinuxHopper.test.ts index fec5529dbb..803a2ad372 100644 --- a/src/application/LinuxHopper.test.ts +++ b/src/application/LinuxHopper.test.ts @@ -76,7 +76,11 @@ class RecordingLinuxHost implements LinuxHopperInstallHost { Promise.resolve(ok(releasesFor(new Uint8Array([1, 2, 3]))[family])); download = (url: string): Promise => { this.downloads.push(url); - return Promise.resolve({ ok: this.archiveOk, bytes: this.archive }); + return Promise.resolve( + this.archiveOk + ? ok(this.archive) + : err({ url, kind: "http_status", httpStatus: 503 }), + ); }; createTemporaryDirectory = (): Promise => Promise.resolve("/tmp/rea test"); @@ -394,7 +398,19 @@ describe("Linux Hopper installation failure classification", () => { ...(scenario === "package manager failure" ? { packageFailure: host.packageFailure } : {}), + ...(scenario === "package request failure" + ? { + download: { + request: "package", + url: host.downloads[0], + kind: "http_status", + httpStatus: 503, + }, + } + : {}), }); + if (scenario === "package request failure") + expect(host.cleaned).toEqual([]); if (host.installed.length > 0) expect(host.cleaned).toEqual(["/tmp/rea test"]); }); diff --git a/src/application/LinuxHopper.ts b/src/application/LinuxHopper.ts index 420f104088..9e88b2dd5f 100644 --- a/src/application/LinuxHopper.ts +++ b/src/application/LinuxHopper.ts @@ -11,6 +11,7 @@ import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js"; import { downloadHopperPackage, type HopperDownload, + type HopperRequestFailure, } from "./HopperDownload.js"; import { inspectLinuxHopperPythonRuntime, @@ -244,6 +245,8 @@ export type LinuxHopperInstallResult = readonly pythonRuntimeFailure?: LinuxHopperPythonError; /** The operation that threw, retained separately from its outcome class. */ readonly operationFailure?: LinuxHopperOperationFailure; + /** The vendor request (feed, package or key) whose transport failure stopped setup. */ + readonly download?: HopperRequestFailure; /** Original caller cancellation, without flattening its value into text. */ readonly cancellationReason?: unknown; } & { readonly cleanupFailure?: LinuxHopperTemporaryCleanupFailure }); @@ -388,14 +391,12 @@ export const installLinuxHopper = async ( options.signal, ); if (!selected.ok) - return { - status: "failed", - verificationFailure: selected.error, - reason: - selected.error.reason === "cancelled" - ? "cancelled" - : "release_metadata", - }; + return publisherFailure( + selected.error, + selected.error.reason === "cancelled" + ? "cancelled" + : "release_metadata", + ); const release = selected.value; stage = "download"; const archiveDownload = await host.download(release.filename, options); @@ -403,14 +404,7 @@ export const installLinuxHopper = async ( return { status: "failed", reason: "download", - ...(archiveDownload.httpStatus === undefined - ? {} - : { - operationFailure: { - operation: "download", - detail: `Package download returned HTTP ${archiveDownload.httpStatus}.`, - }, - }), + download: { request: "package", ...archiveDownload.error }, }; stage = "archive_write"; temporary = await host.createTemporaryDirectory(); @@ -421,7 +415,7 @@ export const installLinuxHopper = async ( `hopper.${distribution.packageFamily}`, ); operationPath = archive; - await host.writeArchive(archive, archiveDownload.bytes); + await host.writeArchive(archive, archiveDownload.value); stage = "publisher_verification_failed"; const verified = await host.verifyArchive( distribution.packageFamily, @@ -430,28 +424,24 @@ export const installLinuxHopper = async ( options.signal, ); if (!verified.ok) - return { - status: "failed", - verificationFailure: verified.error, - reason: - verified.error.reason === "cache_missing" - ? "verification_unavailable" - : verified.error.reason, - }; + return publisherFailure( + verified.error, + verified.error.reason === "cache_missing" + ? "verification_unavailable" + : verified.error.reason, + ); const retained = await host.retainVerifiedArchive( verified.value, archive, options.signal, ); if (!retained.ok) - return { - status: "failed", - verificationFailure: retained.error, - reason: - retained.error.reason === "cache_missing" - ? "verification_unavailable" - : retained.error.reason, - }; + return publisherFailure( + retained.error, + retained.error.reason === "cache_missing" + ? "verification_unavailable" + : retained.error.reason, + ); if (isAborted(options.signal)) return cancelledInstall(options.signal?.reason); stage = "authorization_or_package_manager"; @@ -741,6 +731,17 @@ const installSystemPackage = async ( const isAborted = (signal: AbortSignal | undefined): boolean => signal?.aborted === true; +/** Retain the publisher's error and any vendor request facts it observed. */ +const publisherFailure = ( + error: HopperReleaseError, + reason: Exclude, +): LinuxHopperInstallResult => ({ + status: "failed", + reason, + verificationFailure: error, + ...(error.download === undefined ? {} : { download: error.download }), +}); + const cancelledInstall = (cause: unknown): LinuxHopperInstallResult => ({ status: "failed", reason: "cancelled", diff --git a/src/application/LinuxHopperPublisher.ts b/src/application/LinuxHopperPublisher.ts index da31e86bc1..20e63cfe5a 100644 --- a/src/application/LinuxHopperPublisher.ts +++ b/src/application/LinuxHopperPublisher.ts @@ -125,11 +125,11 @@ export class LinuxHopperPublisher { timeoutMs: RELEASE_METADATA_TIMEOUT_MS, }); if (!response.ok) - throw new HopperReleaseError( - "release_metadata", - "The official Hopper release feed is unavailable.", - ); - const decoded = parseHopperReleaseJson(response.bytes); + throw HopperReleaseError.fromDownload("release_metadata", { + request: "release_feed", + ...response.error, + }); + const decoded = parseHopperReleaseJson(response.value); if (!decoded.ok) throw decoded.cause; this.#feed = parseLinuxHopperReleases(decoded.value); } @@ -189,11 +189,11 @@ export class LinuxHopperPublisher { timeoutMs: RELEASE_METADATA_TIMEOUT_MS, }); if (!response.ok) - throw new HopperReleaseError( - "verification_unavailable", - "The official Hopper public key is unavailable.", - ); - publicKey = decodeUtf8(response.bytes); + throw HopperReleaseError.fromDownload("verification_unavailable", { + request: "public_key", + ...response.error, + }); + publicKey = decodeUtf8(response.value); } const packageSha256 = createHash("sha256").update(bytes).digest("hex"); // Every read still validates file identity, byte bounds and selected metadata. @@ -533,12 +533,12 @@ export class LinuxHopperPublisher { signal === undefined ? {} : { signal }, ); if (!response.ok) - throw new HopperReleaseError( - "download", - "The official Hopper package is unavailable for publisher verification.", - ); + throw HopperReleaseError.fromDownload("download", { + request: "package", + ...response.error, + }); const archive = join(root, "archive"); - await writeFile(archive, response.bytes, { flag: "wx", mode: 0o600 }); + await writeFile(archive, response.value, { flag: "wx", mode: 0o600 }); const verified = await this.#verifyArchive( family, selected.value, diff --git a/src/application/MacHopper.test.ts b/src/application/MacHopper.test.ts index 11e4aab67f..f6ad8d2ef3 100644 --- a/src/application/MacHopper.test.ts +++ b/src/application/MacHopper.test.ts @@ -2,8 +2,15 @@ import { createHash } from "node:crypto"; import { describe, expect, it } from "vitest"; +import { err, ok } from "../domain/result.js"; +import type { + HopperDownload, + HopperDownloadFailure, +} from "./HopperDownload.js"; import { installMacHopper, + macHopperHostVersionDetail, + macHopperIncomparableMinimumDetail, type MacHopperBundleVerification, type MacHopperBundleInstallResult, type MacHopperInstallHost, @@ -11,7 +18,8 @@ import { class FakeMacHopperHost implements MacHopperInstallHost { readonly archive = new TextEncoder().encode("hopper-dmg"); - archiveOk = true; + feedFailure: HopperDownloadFailure | undefined; + packageFailure: HopperDownloadFailure | undefined; feedBytes: Uint8Array | undefined; packageDownloads = 0; mounts = 0; @@ -28,14 +36,15 @@ class FakeMacHopperHost implements MacHopperInstallHost { opens = 0; replaces = false; - download = (url: string) => { - if (!url.includes("files-api")) this.packageDownloads += 1; + download = (url: string): Promise => { + const feed = url.includes("files-api"); + if (!feed) this.packageDownloads += 1; + const failure = feed ? this.feedFailure : this.packageFailure; + if (failure !== undefined) return Promise.resolve(err(failure)); return Promise.resolve( - url.includes("files-api") - ? { - ok: true, - bytes: - this.feedBytes ?? + ok( + feed + ? (this.feedBytes ?? new TextEncoder().encode( JSON.stringify({ "OS X": { @@ -47,9 +56,9 @@ class FakeMacHopperHost implements MacHopperInstallHost { : createHash("sha1").update(this.archive).digest("hex"), }, }), - ), - } - : { ok: this.archiveOk, bytes: this.archive }, + )) + : this.archive, + ), ); }; createTemporaryDirectory = () => Promise.resolve("/tmp/rea-hopper-test"); @@ -70,6 +79,17 @@ class FakeMacHopperHost implements MacHopperInstallHost { this.verificationFailure === undefined ? Promise.resolve(this.verification) : Promise.reject(this.verificationFailure); + hostVersion: string | undefined = "14.5"; + appMinimum: string | undefined = "12.4"; + appMinimumFailure: Error | undefined; + appMinimumReads = 0; + macosVersion = () => Promise.resolve(this.hostVersion); + minimumSystemVersion = () => { + this.appMinimumReads += 1; + return this.appMinimumFailure === undefined + ? Promise.resolve(this.appMinimum) + : Promise.reject(this.appMinimumFailure); + }; destinationExists = () => Promise.resolve(this.existing); installBundle = ( _source: string, @@ -99,6 +119,12 @@ class FakeMacHopperHost implements MacHopperInstallHost { destination = () => "/home/user/Applications/Hopper Disassembler.app"; } +const feedText = async (host: FakeMacHopperHost): Promise => { + const feed = await host.download("files-api"); + if (!feed.ok) throw new Error("Expected the fixture feed"); + return new TextDecoder().decode(feed.value); +}; + describe("macOS Hopper installation", () => { it("installs, verifies, opens, and cleans the official package", async () => { const host = new FakeMacHopperHost(); @@ -224,6 +250,182 @@ describe("macOS Hopper installation", () => { }); }); +const PACKAGE_URL = + "https://www.hopperapp.com:443/downloader/public/Hopper-6.6.1-demo.dmg"; + +/** The refusal detail carries the observed facts, independent of its wording. */ +const expectNamed = ( + result: Awaited>, + facts: readonly string[], +): void => { + const diagnostic = result.status === "failed" ? result.diagnostic : undefined; + for (const fact of facts) expect(diagnostic).toContain(fact); +}; + +/** The fixture's macOS row with vendor fields such as `deployment_target`. */ +const macFeed = ( + host: FakeMacHopperHost, + fields: Readonly>, +): Uint8Array => + new TextEncoder().encode( + JSON.stringify({ + "OS X": { + filename: PACKAGE_URL, + file_length: String(host.archive.byteLength), + file_hash: createHash("sha1").update(host.archive).digest("hex"), + version: "6.6.1", + ...fields, + }, + }), + ); + +describe("macOS Hopper release host requirements", () => { + it.each(["12", "12.0", "12.3", "12.3.9"])( + "refuses macOS %s below the release deployment target before downloading the package", + async (hostVersion) => { + const host = new FakeMacHopperHost(); + host.hostVersion = hostVersion; + host.feedBytes = macFeed(host, { deployment_target: "12.4" }); + + const result = await installMacHopper({}, host); + expect(result).toEqual({ + status: "failed", + reason: "incompatible_host_version", + diagnostic: macHopperHostVersionDetail({ + source: "release_feed", + packageUrl: PACKAGE_URL, + minimumVersion: "12.4", + hostVersion, + }), + }); + expectNamed(result, [PACKAGE_URL, "12.4", hostVersion]); + expect(host.packageDownloads).toBe(0); + expect(host.mounts).toBe(0); + expect(host.installs).toBe(0); + }, + ); + + it.each(["12.4", "12.4.0", "12.4.1", "12.10", "13", "26.1"])( + "installs on macOS %s at or above both declared minimums", + async (hostVersion) => { + const host = new FakeMacHopperHost(); + host.hostVersion = hostVersion; + host.feedBytes = macFeed(host, { deployment_target: "12.4" }); + + expect(await installMacHopper({}, host)).toMatchObject({ + status: "installed", + }); + expect(host.installs).toBe(1); + }, + ); + + it.each([ + { label: "an undeclared", fields: {} }, + { label: "an unrecognized", fields: { deployment_target: "twelve" } }, + { label: "a non-text", fields: { deployment_target: 12.4 } }, + { label: "a lower", fields: { deployment_target: "11.0" } }, + ])( + "lets the signed app refuse the host after $label feed deployment target", + async ({ fields }) => { + const host = new FakeMacHopperHost(); + host.hostVersion = "12.0"; + host.feedBytes = macFeed(host, fields); + + const result = await installMacHopper({}, host); + expect(result).toEqual({ + status: "failed", + reason: "incompatible_host_version", + diagnostic: macHopperHostVersionDetail({ + source: "signed_app", + minimumVersion: "12.4", + hostVersion: "12.0", + }), + }); + expectNamed(result, ["LSMinimumSystemVersion", "12.4", "12.0"]); + expect(host.packageDownloads).toBe(1); + expect(host.installs).toBe(0); + expect(host.opens).toBe(0); + expect(host.unmounts).toBe(1); + expect(host.cleanups).toBe(1); + }, + ); + + it.each(["invalid_signature", "unverified_publisher"] as const)( + "reads the app's requirement only after its signature checks pass: %s", + async (verification) => { + const host = new FakeMacHopperHost(); + host.verification = verification; + + expect(await installMacHopper({}, host)).toMatchObject({ + status: "failed", + }); + expect(host.appMinimumReads).toBe(0); + expect(host.installs).toBe(0); + }, + ); + + it("installs when neither the feed nor the signed app declares a minimum", async () => { + const host = new FakeMacHopperHost(); + host.hostVersion = "12.0"; + host.appMinimum = undefined; + + expect(await installMacHopper({}, host)).toMatchObject({ + status: "installed", + }); + }); + + it("reports an incomparable signed-app minimum as a bundle failure without copying", async () => { + const host = new FakeMacHopperHost(); + host.appMinimum = "12.x"; + + const result = await installMacHopper({}, host); + expect(result).toEqual({ + status: "failed", + reason: "bundle", + diagnostic: macHopperIncomparableMinimumDetail("12.x", "14.5"), + }); + expectNamed(result, ['"12.x"', "14.5"]); + expect(host.installs).toBe(0); + expect(host.unmounts).toBe(1); + expect(host.cleanups).toBe(1); + }); + + it("preserves an unreadable signed-app requirement as a bundle diagnostic", async () => { + const host = new FakeMacHopperHost(); + host.appMinimumFailure = new Error("Info.plist: permission denied"); + + expect(await installMacHopper({}, host)).toMatchObject({ + status: "failed", + reason: "bundle", + diagnostic: "Info.plist: permission denied", + }); + expect(host.installs).toBe(0); + expect(host.unmounts).toBe(1); + expect(host.cleanups).toBe(1); + }); + + it.each([undefined, "", "14beta", "14.0\n"])( + "refuses an incomparable host version %j before any vendor request", + async (hostVersion) => { + const host = new FakeMacHopperHost(); + host.hostVersion = hostVersion; + const download = host.download; + let requests = 0; + host.download = (url) => { + requests += 1; + return download(url); + }; + + expect(await installMacHopper({}, host)).toEqual({ + status: "failed", + reason: "unsupported_host", + }); + expect(requests).toBe(0); + expect(host.mounts).toBe(0); + }, + ); +}); + it("reports an unresolved destination probe without claiming an existing installation", async () => { const host = new FakeMacHopperHost(); host.destinationExists = async () => { @@ -273,8 +475,7 @@ it("retains the exact DMG temporary root with its retry capability", async () => it("refuses malformed UTF-8 release metadata before downloading or mounting the package", async () => { const host = new FakeMacHopperHost(); - const feed = await host.download("files-api"); - const text = new TextDecoder().decode(feed.bytes); + const text = await feedText(host); // Replacement decoding would produce valid JSON and hide the malformed transport byte. host.feedBytes = new Uint8Array([ ...new TextEncoder().encode('{"vendor_note":"'), @@ -294,10 +495,10 @@ it.each([false, true])( "accepts a genuine replacement character in release metadata (BOM=%s)", async (bom) => { const host = new FakeMacHopperHost(); - const feed = await host.download("files-api"); - const text = new TextDecoder() - .decode(feed.bytes) - .replace('{"OS X":{', '{"OS X":{"vendor_note":"雪�",'); + const text = (await feedText(host)).replace( + '{"OS X":{', + '{"OS X":{"vendor_note":"雪�",', + ); host.feedBytes = new TextEncoder().encode((bom ? "\uFEFF" : "") + text); expect(await installMacHopper({}, host)).toMatchObject({ status: "installed", @@ -310,8 +511,7 @@ it.each([false, true])( it("refuses release URLs that normalize outside the public channel before package download", async () => { const host = new FakeMacHopperHost(); - const feed = await host.download("files-api"); - const row = JSON.parse(new TextDecoder().decode(feed.bytes)); + const row = JSON.parse(await feedText(host)); row["OS X"].filename = "https://www.hopperapp.com:443/downloader/public/%2e%2e/private/Hopper-demo.dmg"; host.feedBytes = new TextEncoder().encode(JSON.stringify(row)); @@ -323,3 +523,28 @@ it("refuses release URLs that normalize outside the public channel before packag expect(host.packageDownloads).toBe(0); expect(host.mounts).toBe(0); }); + +it.each(["feed", "package"] as const)( + "attributes a failed %s request to its stage before any mount", + async (stage) => { + const host = new FakeMacHopperHost(); + const failure = { + url: "https://fixture.invalid/request", + kind: "timeout", + timeoutMs: 120_000, + } as const; + if (stage === "feed") host.feedFailure = failure; + else host.packageFailure = failure; + expect(await installMacHopper({}, host)).toEqual({ + status: "failed", + reason: stage === "feed" ? "release_metadata" : "download", + download: { + request: stage === "feed" ? "release_feed" : "package", + ...failure, + }, + }); + expect(host.packageDownloads).toBe(stage === "feed" ? 0 : 1); + expect(host.mounts).toBe(0); + expect(host.cleanups).toBe(0); + }, +); diff --git a/src/application/MacHopper.ts b/src/application/MacHopper.ts index 4b3edeb347..834311bc20 100644 --- a/src/application/MacHopper.ts +++ b/src/application/MacHopper.ts @@ -4,6 +4,7 @@ import { mkdir, mkdtemp, readdir, + readFile, rename, rm, writeFile, @@ -12,21 +13,32 @@ import { tmpdir } from "node:os"; import { join, posix } from "node:path"; import { promisify } from "node:util"; +import { z } from "zod"; + import { appendCallerPath } from "../filesystem/callerPath.js"; -import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js"; import type { AnalysisCleanupObservation } from "../domain/analysisErrorBase.js"; import { HOPPER_PUBLISHER } from "../domain/hopperPublisher.js"; import { macDeveloperIdRequirement } from "../process/MacCodeSignature.js"; import { macHopperUserApplicationPath } from "./MacHopperPaths.js"; -import { downloadHopperPackage } from "./HopperDownload.js"; +import { + downloadHopperPackage, + type HopperDownload, + type HopperRequestFailure, +} from "./HopperDownload.js"; import { HOPPER_RELEASES_URL, HOPPER_DOWNLOAD_PREFIX, parseHopperReleaseJson, parseMacHopperReleases, hopperArchiveIntegrityMatches, + macHopperReleaseMinimumSystemVersion, } from "./HopperReleaseFeed.js"; import type { HopperInstallFailureReason } from "./SetupInstallFailure.js"; +import { decodeInfoPlistValue } from "./AppBundleExecutable.js"; +import { + macosMajorVersion, + macosVersionMeetsMinimum, +} from "../domain/macosVersion.js"; const execFileAsync = promisify(execFile); const HOPPER_PUBLISHER_REQUIREMENT = macDeveloperIdRequirement( @@ -42,6 +54,8 @@ export type MacHopperBundleVerification = type MacHopperInstallFailureReason = Extract< HopperInstallFailureReason, + | "unsupported_host" + | "incompatible_host_version" | "release_metadata" | "download" | "integrity" @@ -73,8 +87,10 @@ export interface MacHopperBundleInstallOperations { removeStagingRoot(path: string): Promise; } -/** Stable network and command effects disclosed before macOS Hopper setup. */ +/** Stable effects and checks disclosed before macOS Hopper setup. */ export const macHopperInstallDisclosure = { + detail: + "Download the official Hopper package, verify it, and install it. Before the download, and again from the signed app before copying, setup compares the release's minimum macOS version with this Mac's and stops if this Mac is older. Hopper may show its demo or license prompt when first opened.", networkOrigins: [HOPPER_RELEASES_URL, HOPPER_DOWNLOAD_PREFIX] as const, commands: [ "hdiutil attach -readonly -nobrowse -mountpoint ", @@ -97,14 +113,14 @@ export type MacHopperInstallResult = readonly status: "failed"; readonly reason: MacHopperInstallFailureReason; readonly diagnostic?: string; + /** The vendor request (feed or package) whose transport failure stopped setup. */ + readonly download?: HopperRequestFailure; readonly cleanupFailure?: string; }; /** External effects used by the bounded macOS Hopper installer. */ export interface MacHopperInstallHost { - download( - url: string, - ): Promise<{ readonly ok: boolean; readonly bytes: Uint8Array }>; + download(url: string): Promise; createTemporaryDirectory(): Promise; createMountDirectory(path: string): Promise; writePackage(path: string, bytes: Uint8Array): Promise; @@ -112,6 +128,10 @@ export interface MacHopperInstallHost { unmount(mountPath: string): Promise; appBundle(mountPath: string): Promise; validBundle(appPath: string): Promise; + /** This Mac's version, read through the probe setup planning used. */ + macosVersion(): Promise; + /** The mounted app's declared LSMinimumSystemVersion; undefined when it declares none. */ + minimumSystemVersion(appPath: string): Promise; destinationExists(path: string): Promise; installBundle( source: string, @@ -128,14 +148,20 @@ export interface MacHopperInstallHost { destination(): string; } -/** Download, verify, and install Hopper into the current user's Applications directory. */ +/** + * Download, verify, and install Hopper into the current user's Applications + * directory. The selected release's own macOS minimum gates installation: the + * feed's `deployment_target` before the package is downloaded, and the signed + * app's `LSMinimumSystemVersion` before it is copied. + */ export const installMacHopper = async ( - options: { readonly replaceExisting?: boolean } = {}, - host: MacHopperInstallHost = systemMacHopperInstallHost( - homeDirectoryFromEnvironment(process.env, process.platform), - process.env, - ), + options: { readonly replaceExisting?: boolean }, + host: MacHopperInstallHost, ): Promise => { + const hostVersion = await host.macosVersion(); + // Without a comparable host version no declared minimum can be checked. + if (hostVersion === undefined || macosMajorVersion(hostVersion) === undefined) + return { status: "failed", reason: "unsupported_host" }; let temporary: string | undefined; let mounted = false; let bundleCleanupFailure: string | undefined; @@ -144,17 +170,43 @@ export const installMacHopper = async ( try { result = await (async (): Promise => { const metadata = await host.download(HOPPER_RELEASES_URL); - if (!metadata.ok) return { status: "failed", reason: "release_metadata" }; - const decoded = parseHopperReleaseJson(metadata.bytes); + if (!metadata.ok) + return { + status: "failed", + reason: "release_metadata", + download: { request: "release_feed", ...metadata.error }, + }; + const decoded = parseHopperReleaseJson(metadata.value); if (!decoded.ok) throw decoded.cause; const parsed = parseMacHopperReleases(decoded.value); if (!parsed.ok) return { status: "failed", reason: "release_metadata" }; const release = parsed.value; + const releaseMinimum = macHopperReleaseMinimumSystemVersion(release); + // An unrecognized feed value defers to the signed app checked below. + if ( + releaseMinimum !== undefined && + macosVersionMeetsMinimum(hostVersion, releaseMinimum) === false + ) + return { + status: "failed", + reason: "incompatible_host_version", + diagnostic: macHopperHostVersionDetail({ + source: "release_feed", + packageUrl: release.filename, + minimumVersion: releaseMinimum, + hostVersion, + }), + }; installStage = "download"; const archive = await host.download(release.filename); - if (!archive.ok) return { status: "failed", reason: "download" }; + if (!archive.ok) + return { + status: "failed", + reason: "download", + download: { request: "package", ...archive.error }, + }; installStage = "integrity"; - if (!hopperArchiveIntegrityMatches(archive.bytes, release)) + if (!hopperArchiveIntegrityMatches(archive.value, release)) return { status: "failed", reason: "integrity" }; installStage = "destination_inspection"; @@ -167,7 +219,7 @@ export const installMacHopper = async ( const packagePath = posix.join(temporary, "hopper.dmg"); const mountPath = posix.join(temporary, "mounted"); await host.createMountDirectory(mountPath); - await host.writePackage(packagePath, archive.bytes); + await host.writePackage(packagePath, archive.value); if (!(await host.mount(packagePath, mountPath))) return { status: "failed", reason: "mount" }; mounted = true; @@ -179,6 +231,30 @@ export const installMacHopper = async ( return { status: "failed", reason: "bundle" }; if (verification === "unverified_publisher") return { status: "failed", reason: "publisher_verification_failed" }; + // An app that declares no minimum is accepted, as macOS does. + const appMinimum = await host.minimumSystemVersion(source); + if (appMinimum !== undefined) { + const compatible = macosVersionMeetsMinimum(hostVersion, appMinimum); + if (compatible === false) + return { + status: "failed", + reason: "incompatible_host_version", + diagnostic: macHopperHostVersionDetail({ + source: "signed_app", + minimumVersion: appMinimum, + hostVersion, + }), + }; + if (compatible === undefined) + return { + status: "failed", + reason: "bundle", + diagnostic: macHopperIncomparableMinimumDetail( + appMinimum, + hostVersion, + ), + }; + } installStage = "copy"; const installation = await host.installBundle( source, @@ -360,6 +436,7 @@ export const installMacHopperBundle = async ( export const systemMacHopperInstallHost = ( homeDirectory: string, environment: Readonly, + macosVersion: () => Promise, ): MacHopperInstallHost & { close(): Promise; } => { @@ -420,6 +497,18 @@ export const systemMacHopperInstallHost = ( return "unverified_publisher"; return "verified"; }, + macosVersion, + async minimumSystemVersion(appPath) { + const plist = posix.join(appPath, "Contents", "Info.plist"); + const declared = minimumSystemVersionSchema.safeParse( + decodeInfoPlistValue(await readFile(plist)), + ); + if (!declared.success) + throw new TypeError( + `${plist} does not declare LSMinimumSystemVersion as text`, + ); + return declared.data.LSMinimumSystemVersion; + }, async destinationExists(path) { try { await access(path); @@ -480,6 +569,28 @@ export const systemMacHopperInstallHost = ( }; }; +/** Installer detail naming the declared minimum this Mac's version does not meet. */ +export const macHopperHostVersionDetail = ( + refusal: ( + | { readonly source: "release_feed"; readonly packageUrl: string } + | { readonly source: "signed_app" } + ) & { readonly minimumVersion: string; readonly hostVersion: string }, +): string => + refusal.source === "release_feed" + ? `The current Hopper release (${refusal.packageUrl}) declares deployment_target ${refusal.minimumVersion}; this Mac runs macOS ${refusal.hostVersion}. The package was not downloaded.` + : `The signed Hopper application declares LSMinimumSystemVersion ${refusal.minimumVersion}; this Mac runs macOS ${refusal.hostVersion}. Nothing was copied.`; + +/** Installer detail for a signed-app minimum REA cannot compare with this Mac's version. */ +export const macHopperIncomparableMinimumDetail = ( + declared: string, + hostVersion: string, +): string => + `The signed Hopper application declares LSMinimumSystemVersion ${JSON.stringify(declared)}, which REA cannot compare with macOS ${hostVersion}. Nothing was copied.`; + +const minimumSystemVersionSchema = z.looseObject({ + LSMinimumSystemVersion: z.string().optional(), +}); + const commandSucceeds = async ( command: string, args: readonly string[], diff --git a/src/application/Setup.configuration.test.ts b/src/application/Setup.configuration.test.ts index bea24fec81..360157b0e8 100644 --- a/src/application/Setup.configuration.test.ts +++ b/src/application/Setup.configuration.test.ts @@ -243,6 +243,53 @@ describe("optional Hopper installation failures", () => { expect(result.remediation).toContain("injected skill write failure"); expect(host.skillInstalls).toBe(1); }); + + it("returns the failed Hopper request facts beside its code, including after a later failure", async () => { + const download = { + request: "package", + url: "https://www.hopperapp.com:443/downloader/public/Hopper-demo.dmg", + kind: "http_status", + httpStatus: 503, + } as const; + const ready = new FakeSetupHost(); + ready.hopperInstallSucceeds = false; + ready.hopperInstallFailure = { + status: "failed", + code: "download_failed", + remediation: "Package returned HTTP 503.", + download, + }; + expect(await runSetup(options(true, true), ready)).toMatchObject({ + status: "needs_human", + code: "download_failed", + remediation: "Package returned HTTP 503.", + download, + }); + + const skillFailure = new FakeSetupHost(); + skillFailure.hopperInstallSucceeds = false; + skillFailure.hopperInstallFailure = ready.hopperInstallFailure; + skillFailure.skill = "failed"; + skillFailure.clients = [{ name: "cursor", configPath: "/cursor.json" }]; + expect( + await runSetup( + { + ...options(true, true), + clientIds: ["cursor"], + skillClientIds: ["cursor"], + }, + skillFailure, + ), + ).toMatchObject({ code: "skill_install_failed", download }); + }); + + it("omits request facts when the Hopper failure observed none", async () => { + const host = new FakeSetupHost(); + host.hopperInstallSucceeds = false; + expect(await runSetup(options(true, true), host)).not.toHaveProperty( + "download", + ); + }); }); describe("configured Linux Hopper selection", () => { diff --git a/src/application/Setup.fixture.ts b/src/application/Setup.fixture.ts index ca334f9495..aac93940c3 100644 --- a/src/application/Setup.fixture.ts +++ b/src/application/Setup.fixture.ts @@ -7,7 +7,10 @@ import type { SetupProviderEnvironment, } from "./SetupTypes.js"; import type { SetupClient } from "./SupportedClients.js"; -import type { SetupHopperInstallResult } from "./SetupInstallFailure.js"; +import type { + SetupHopperInstallFailure, + SetupHopperInstallResult, +} from "./SetupInstallFailure.js"; import { err, ok, type Result } from "../domain/result.js"; import type { DoctorCheck, @@ -38,6 +41,11 @@ export class FakeSetupHost implements SetupHost { ghidra: string | undefined; javaHome: string | undefined; hopperInstallSucceeds = true; + hopperInstallFailure: SetupHopperInstallFailure = { + status: "failed", + code: "download_failed", + remediation: "Download failed.", + }; skill: "installed" | "unchanged" | "failed" = "installed"; clients: readonly SetupClient[] = []; availableClients: readonly SetupClient[] | undefined; @@ -99,11 +107,7 @@ export class FakeSetupHost implements SetupHost { launcherPath: this.hopper, }); } - return Promise.resolve({ - status: "failed", - code: "download_failed", - remediation: "Download failed.", - }); + return Promise.resolve(this.hopperInstallFailure); }; detectedClients = (): Promise => Promise.resolve(this.clients); diff --git a/src/application/Setup.recovery.test.ts b/src/application/Setup.recovery.test.ts index edf0be2768..0071617240 100644 --- a/src/application/Setup.recovery.test.ts +++ b/src/application/Setup.recovery.test.ts @@ -6,6 +6,8 @@ import { ConfigurationError } from "../domain/configurationErrors.js"; import { SUPPORTED_NODE_VERSION_PROSE } from "../domain/runtimeVersion.js"; import { FakeSetupHost, options } from "./Setup.fixture.js"; import { runSetup } from "./Setup.js"; +import { discoverSetupState } from "./SetupPlan.js"; +import { macHopperInstallDisclosure } from "./MacHopper.js"; import { parseLinuxDistribution, type LinuxDistribution, @@ -690,6 +692,8 @@ describe("setup Hopper install eligibility", () => { { version: "14.", installs: false }, { version: "14.0\n", installs: false }, { version: "99999999999999999", installs: false }, + // REA's own floor admits these; the selected release's minimum is + // compared by the installer, which this recording host replaces. { version: "12", installs: true }, { version: "12.0", installs: true }, ])( @@ -709,6 +713,35 @@ describe("setup Hopper install eligibility", () => { }); }, ); + + it.each(["12.0", "12.3", "12.4", "14.5"])( + "leaves Hopper support on macOS %s unknown until the release minimum is read", + async (version) => { + const host = new FakeSetupHost("darwin"); + host.version = version; + + const discovery = await discoverSetupState({ + host, + providerConfigured: false, + forceHopperInstall: false, + proposeHopper: true, + initialDoctor: await host.doctor(), + }); + expect(discovery).toMatchObject({ + hopperInstall: { supported: "unknown" }, + installHopper: true, + }); + const plan = await runSetup( + { ...options(false, true), dryRun: true }, + host, + ); + // The installer owns the disclosed check; its refusal is tested there. + expect( + plan.plannedActions.find(({ kind }) => kind === "install_hopper") + ?.detail, + ).toBe(macHopperInstallDisclosure.detail); + }, + ); }); it("leaves ownerless caller-host failures unconfirmed after an unobserved direct retry", async () => { diff --git a/src/application/Setup.ts b/src/application/Setup.ts index 4bdc1cf3f6..754b902013 100644 --- a/src/application/Setup.ts +++ b/src/application/Setup.ts @@ -372,6 +372,7 @@ const runSetupWorkflow = async ( remediation: combineRemediation(hopperFailure.remediation, result.remediation) ?? hopperFailure.remediation, + ...hopperDownload(hopperFailure), }; if (clientFailure !== undefined) return withHopperFailure(await fail(clientFailure)); @@ -394,7 +395,8 @@ const runSetupWorkflow = async ( appliedActions.includes("installed_hopper"), doctor.healthy, // Offer installation only where the explicit gate accepts it. - hopperPath === undefined && planDiscovery.hopperInstall.supported, + hopperPath === undefined && + planDiscovery.hopperInstall.supported !== false, ), hopperWarning, manualRemediation, @@ -407,10 +409,17 @@ const runSetupWorkflow = async ( doctor, clientStates, ...(remediation === undefined ? {} : { remediation }), - ...(hopperFailure === undefined ? {} : { code: hopperFailure.code }), + ...(hopperFailure === undefined + ? {} + : { code: hopperFailure.code, ...hopperDownload(hopperFailure) }), }; }; +const hopperDownload = ( + failure: SetupHopperInstallFailure, +): Pick => + failure.download === undefined ? {} : { download: failure.download }; + const installHopperAction = async (input: { readonly host: SetupHost; readonly options: SetupOptions; diff --git a/src/application/SetupHost.ts b/src/application/SetupHost.ts index 0b465f78b0..d59260c3b3 100644 --- a/src/application/SetupHost.ts +++ b/src/application/SetupHost.ts @@ -130,7 +130,9 @@ export const systemSetupHost = ( const { homeDirectory } = doctorHost; const macHopperHost = platform === "darwin" - ? systemMacHopperInstallHost(homeDirectory, environment) + ? systemMacHopperInstallHost(homeDirectory, environment, () => + doctorHost.macosVersion(), + ) : undefined; const resources = new ArtifactResourceScope(); const linuxDistribution = () => doctorHost.linuxDistribution(); @@ -227,17 +229,20 @@ export const systemSetupHost = ( } if (result.status === "installed") return { status: "installed", launcherPath: result.launcherPath }; - const failure = setupInstallFailure( - result.reason, - result.packageFailure, - "linux", - ); + const failure = setupInstallFailure(result.reason, { + packageFailure: result.packageFailure, + platform: "linux", + download: result.download, + }); + // A failed vendor request is already described by its owned facts. const detail = - result.verificationFailure?.message ?? - result.pythonRuntimeFailure?.message ?? - (result.operationFailure === undefined + result.download !== undefined ? undefined - : `${result.operationFailure.operation}${result.operationFailure.path === undefined ? "" : ` at ${result.operationFailure.path}`}: ${result.operationFailure.systemCode === undefined ? "" : `${result.operationFailure.systemCode}: `}${result.operationFailure.detail}`); + : (result.verificationFailure?.message ?? + result.pythonRuntimeFailure?.message ?? + (result.operationFailure === undefined + ? undefined + : `${result.operationFailure.operation}${result.operationFailure.path === undefined ? "" : ` at ${result.operationFailure.path}`}: ${result.operationFailure.systemCode === undefined ? "" : `${result.operationFailure.systemCode}: `}${result.operationFailure.detail}`)); return detail !== undefined ? { ...failure, @@ -264,7 +269,9 @@ export const systemSetupHost = ( ? {} : { cleanupFailure: result.cleanupFailure }), }; - const failure = setupInstallFailure(result.reason); + const failure = setupInstallFailure(result.reason, { + download: result.download, + }); const diagnostic = setupInstallerDiagnostic(result.diagnostic); return { ...failure, diff --git a/src/application/SetupInstallFailure.test.ts b/src/application/SetupInstallFailure.test.ts index 2b606745b9..afdd3494f9 100644 --- a/src/application/SetupInstallFailure.test.ts +++ b/src/application/SetupInstallFailure.test.ts @@ -20,16 +20,30 @@ describe("macOS Hopper setup failure projection", () => { "The mounted package did not contain the expected Hopper application bundle. Retry setup; if it repeats, update REA before installing.", }); }); + + it("keeps a host below the release minimum distinct from an unsupported host", () => { + const incompatible = setupInstallFailure("incompatible_host_version"); + expect(incompatible).toMatchObject({ + status: "failed", + code: "incompatible_host_version", + }); + expect(incompatible.remediation).toContain("HOPPER_LAUNCHER_PATH"); + const unsupported = setupInstallFailure("unsupported_host"); + expect(unsupported.code).toBe("unsupported_host"); + expect(incompatible.remediation).not.toBe(unsupported.remediation); + }); }); describe("Linux package-manager setup failure projection", () => { it("quotes the failing command and a sanitized bounded diagnostic", () => { const result = setupInstallFailure("authorization_or_package_manager", { - command: 'pkexec pacman -U "/tmp/hopper package.pkg.tar.xz"', - exitCode: 1, - stdout: "", - stderr: "\u001b[31merror:\u001b[0m package conflict\nrefused", - outputTruncated: false, + packageFailure: { + command: 'pkexec pacman -U "/tmp/hopper package.pkg.tar.xz"', + exitCode: 1, + stdout: "", + stderr: "\u001b[31merror:\u001b[0m package conflict\nrefused", + outputTruncated: false, + }, }); expect(result).toMatchObject({ status: "failed", @@ -55,11 +69,13 @@ describe("Linux package-manager setup failure projection", () => { it("keeps truncated package output within its excerpt budget", () => { const result = setupInstallFailure("authorization_or_package_manager", { - command: "pacman -U hopper.pkg.tar.xz", - exitCode: 1, - stdout: "", - stderr: `${"warning ".repeat(100)}final package conflict`, - outputTruncated: true, + packageFailure: { + command: "pacman -U hopper.pkg.tar.xz", + exitCode: 1, + stdout: "", + stderr: `${"warning ".repeat(100)}final package conflict`, + outputTruncated: true, + }, }); if (result.status !== "failed") throw new Error("expected failure"); const excerpt = result.remediation diff --git a/src/application/SetupInstallFailure.ts b/src/application/SetupInstallFailure.ts index 6610e9f661..c815f0223a 100644 --- a/src/application/SetupInstallFailure.ts +++ b/src/application/SetupInstallFailure.ts @@ -1,9 +1,15 @@ import { HOPPER_PUBLISHER } from "../domain/hopperPublisher.js"; +import { + describeHopperRequestFailure, + hopperRequestRecovery, + type HopperRequestFailure, +} from "./HopperDownload.js"; import { HOPPER_LINUX_PRIMARY_FINGERPRINT } from "./HopperReleaseFeed.js"; import type { LinuxPackageInstallFailure } from "./LinuxHopper.js"; export type SetupFailureCode = | "unsupported_host" + | "incompatible_host_version" | "download_failed" | "integrity_mismatch" | "authorization_or_package_manager_failed" @@ -33,6 +39,8 @@ export type SetupHopperInstallResult = readonly status: "failed"; readonly code: SetupFailureCode; readonly remediation: string; + /** The vendor request whose observed transport failure produced `code`. */ + readonly download?: HopperRequestFailure; }; export type SetupHopperInstallFailure = Extract< @@ -42,6 +50,7 @@ export type SetupHopperInstallFailure = Extract< export type HopperInstallFailureReason = | "unsupported_host" + | "incompatible_host_version" | "release_metadata" | "download" | "integrity" @@ -104,11 +113,37 @@ const packageManagerRemediation = ( return `The system package-manager command failed (${failure.command}; ${status}); ${observed} Resolve the reported problem, grant the authorization the command needs, or install the planned Hopper package and Linux runtime dependencies manually, then rerun rea setup.`; }; -/** Map provider installer failures to stable setup codes and safe recovery. */ +/** + * Map provider installer failures to stable setup codes and safe recovery. A + * failed vendor request replaces the stage's generic advice with its observed + * URL and outcome, and stays structured for programmatic callers. + */ export const setupInstallFailure = ( reason: HopperInstallFailureReason, - packageFailure?: LinuxPackageInstallFailure, - platform: NodeJS.Platform = "darwin", + context: { + readonly packageFailure?: LinuxPackageInstallFailure | undefined; + readonly platform?: NodeJS.Platform; + readonly download?: HopperRequestFailure | undefined; + } = {}, +): SetupHopperInstallFailure => { + const failure = stageFailure( + reason, + context.packageFailure, + context.platform ?? "darwin", + ); + return context.download === undefined + ? failure + : { + ...failure, + remediation: `${describeHopperRequestFailure(context.download)} ${hopperRequestRecovery(context.download)}`, + download: context.download, + }; +}; + +const stageFailure = ( + reason: HopperInstallFailureReason, + packageFailure: LinuxPackageInstallFailure | undefined, + platform: NodeJS.Platform, ): SetupHopperInstallFailure => { switch (reason) { case "unsupported_host": @@ -117,6 +152,13 @@ export const setupInstallFailure = ( code: "unsupported_host", remediation: "Use a supported host, then rerun rea setup.", }; + case "incompatible_host_version": + return { + status: "failed", + code: "incompatible_host_version", + remediation: + "The current Hopper release requires a newer macOS version than this Mac runs, so REA did not install it. Update macOS to at least the version the installer detail names, or install a Hopper release that supports this macOS version yourself and set HOPPER_LAUNCHER_PATH to its launcher, then rerun rea setup.", + }; case "integrity": return { status: "failed", diff --git a/src/application/SetupPlan.ts b/src/application/SetupPlan.ts index d92d545cc0..cf6da52fc8 100644 --- a/src/application/SetupPlan.ts +++ b/src/application/SetupPlan.ts @@ -26,9 +26,15 @@ import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; import { CURRENT_ENTRYPOINT_PATH } from "../process/CurrentEntrypointPath.js"; import { macosMajorVersion } from "../domain/macosVersion.js"; -/** Whether automated Hopper installation can run on this host, and why not. */ +/** + * Whether automated Hopper installation can run on this host, and why not. + * `unknown`: the host meets REA's own prerequisites, but the selected release + * declares its minimum macOS version only in metadata the installer reads + * after approval, so planning cannot claim the release supports this host. + */ export type HopperInstallEligibility = | { readonly supported: true } + | { readonly supported: "unknown" } | { readonly supported: false; readonly remediation: string }; /** Read-only setup facts shared by client selection and preflight. */ @@ -74,7 +80,7 @@ export const discoverSetupState = async (input: { ); const installHopper = input.forceHopperInstall || - (hopperInstall.supported && + (hopperInstall.supported !== false && input.proposeHopper && (!input.providerConfigured || linuxHopperRepairNeeded)); const [clients, detectedClients] = await Promise.all([ @@ -327,7 +333,7 @@ const setupPlan = (input: { detail: input.platform === "linux" ? `Download the current official Hopper demo, verify its PGP publisher and launcher bytes, and retain archive, public-key.asc and release.json under ${input.hopperCacheDirectory ?? `${input.homeDirectory}/.cache/rea/hopper-releases`}/ for offline verification. Preserve replaced damaged evidence beside it as .backup-. Install Hopper plus its Xvfb demo-session dependencies${input.linuxPackageFamily === "deb" ? ", including the shared library matching the distribution Python interpreter" : ""}. REA uses a private display and selects Hopper's offered demo mode for each analysis session.` - : "Download the official Hopper package, verify it, and install it. Hopper may show its demo or license prompt when first opened.", + : macHopperInstallDisclosure.detail, external: true, operation: "install" as const, ...(input.platform === "linux" @@ -393,7 +399,7 @@ const linuxDisclosure = (family: LinuxPackageFamily) => { export const hopperInstallRefusal = ( discovery: Pick, ): string | undefined => - discovery.installHopper && !discovery.hopperInstall.supported + discovery.installHopper && discovery.hopperInstall.supported === false ? discovery.hopperInstall.remediation : undefined; @@ -401,7 +407,8 @@ export const hopperInstallRefusal = ( * Decide automated Hopper installation once from the host facts read during * discovery, so proposals, explicit requests, and recovery advice agree. An * unknown or unparseable macOS version and an unidentified Linux distribution - * are not eligible. + * are not eligible. macOS 12 is REA's own floor; the selected Hopper release's + * minimum is unknown until the installer reads it. */ const hopperInstallEligibility = ( platform: NodeJS.Platform, @@ -411,7 +418,7 @@ const hopperInstallEligibility = ( if (platform === "darwin") { const major = macosMajorVersion(macosVersion); return major !== undefined && major >= 12 - ? { supported: true } + ? { supported: "unknown" } : { supported: false, remediation: "Upgrade to macOS 12 or newer." }; } if (platform === "linux") diff --git a/src/application/SetupTypes.ts b/src/application/SetupTypes.ts index 695c66b83e..0f9a51467d 100644 --- a/src/application/SetupTypes.ts +++ b/src/application/SetupTypes.ts @@ -1,6 +1,7 @@ import type { ClientConfigurationWriteProblem } from "./ClientConfigurationPublication.js"; import type { DoctorReport, DoctorScope, runDoctor } from "./Doctor.js"; import type { LinuxDistribution } from "./LinuxHopper.js"; +import type { HopperRequestFailure } from "./HopperDownload.js"; import type { HopperReleaseError } from "./HopperReleaseFeed.js"; import type { SkillDestination, @@ -159,6 +160,8 @@ export interface SetupResult { readonly clientStates: readonly SetupClientState[]; readonly remediation?: string; readonly code?: SetupFailureCode; + /** The Hopper vendor request (stage, URL and observed outcome) whose failure stopped installation. */ + readonly download?: HopperRequestFailure; /** Identified resources still retained after finalization; absent after confirmed release. */ readonly cleanup?: AnalysisCleanupObservation | undefined; } diff --git a/src/application/WebRuntimeService.test.ts b/src/application/WebRuntimeService.test.ts index 63fe30f8c2..600b64aa7d 100644 --- a/src/application/WebRuntimeService.test.ts +++ b/src/application/WebRuntimeService.test.ts @@ -139,6 +139,13 @@ it("retains completed execution Evidence on late abort and keeps invalid output expect(completed.error.cause).toBe(cancellationReason); expect(parseEvidence(completed.error.partialObservation)).toMatchObject({ operation: "observe_web_execution", + provider: { id: "web-runtime", name: "Runtime fixture", version: "1" }, + parameters: input, + confidence: "observed", + authority: "external-service", + limitations: expect.arrayContaining( + webExecutionFixture(input.target_id, input.observation_ms).limitations, + ), normalized_result: webExecutionFixture( input.target_id, input.observation_ms, diff --git a/src/application/WebSourceLocationService.test.ts b/src/application/WebSourceLocationService.test.ts index 7faa175c3a..86f436a855 100644 --- a/src/application/WebSourceLocationService.test.ts +++ b/src/application/WebSourceLocationService.test.ts @@ -220,7 +220,8 @@ it("honors cancellation both before acquisition and across decoder completion", if (!response.ok) expect(response.error.cause).toBe(firstReason); const controller = new AbortController(); const lateReason = { caller: "after source-map provider completion" }; - const late = await service(undefined, { + const fixture = webSourceLocationFixture(); + const late = await service(fixture, { trace: (input) => { controller.abort(lateReason); return decoder.trace(input); @@ -232,6 +233,28 @@ it("honors cancellation both before acquisition and across decoder completion", expect(parseEvidence(late.error.partialObservation)).toMatchObject({ operation: "trace_web_source_location", confidence: "derived", + authority: "historical-reference", + subject: { + local_path: fixture.sourceFile.path, + digest: { sha256: fixture.sourceFile.sha256 }, + }, + parameters: args, + raw_result: { source_map_text: fixture.sourceMap.text }, + normalized_result: { + source_map: { + ...fixture.sourceMap.file, + url: args.source_map.url, + association: "caller-selected", + }, + lookup: { requested: args.generated_position }, + source_authenticity: "unknown", + execution: "unknown", + }, + locations: [ + fixture.sourceFile.path, + fixture.manifestFile.path, + fixture.sourceMap.file.path, + ].map((path) => ({ kind: "artifact-path", path })), }); const malformedDecoder: WebSourceMapPort = { diff --git a/src/application/WorkflowEvidence.ts b/src/application/WorkflowEvidence.ts index 1b59ed7fdd..2623fa1f4a 100644 --- a/src/application/WorkflowEvidence.ts +++ b/src/application/WorkflowEvidence.ts @@ -7,8 +7,8 @@ import type { AnalysisOperation, AnalysisOperationPort, } from "./AnalysisProvider.js"; -import type { AnalysisError } from "../domain/analysisErrorBase.js"; -import { ok, type Result } from "../domain/result.js"; +import { ok } from "../domain/result.js"; +import type { EnhancedResult } from "./EnhancedToolTypes.js"; import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js"; import type { WorkflowSnapshotRecordInput } from "./binary/BinarySessionRecords.js"; import { @@ -70,21 +70,26 @@ export const composedLimitationBudget = ( ): ComposedLimitationBudget | undefined => operation === "decompile_all" ? DECOMPILE_ALL_LIMITATIONS : undefined; +/** What one composed call answered, or admitted before a failure stopped it. */ +export interface ComposedWorkflowObservation { + readonly result: JsonValue; + readonly limitations: readonly string[]; + readonly cachePolicy?: "live"; + readonly evidenceLinks?: readonly string[]; +} + +/** A composed outcome; only a stopped failure carries its admitted observation. */ +export type ComposedWorkflowResult = + AnalysisResultWithSources & { + readonly partial?: ComposedWorkflowObservation; + }; + /** Retain the limits of exactly the observations used by one composed call. */ export const executeComposedWorkflow = async ( analysis: AnalysisOperationPort, - execute: ( - observed: AnalysisOperationPort, - ) => Promise>, + execute: (observed: AnalysisOperationPort) => EnhancedResult, budget?: ComposedLimitationBudget, -): Promise< - AnalysisResultWithSources<{ - readonly result: JsonValue; - readonly limitations: readonly string[]; - readonly cachePolicy?: "live"; - readonly evidenceLinks?: readonly string[]; - }> -> => { +): Promise => { const limitations = new Map(); const evidenceLinks = new Set(); const sourceLedger = new EvidenceLedger(); @@ -149,24 +154,32 @@ export const executeComposedWorkflow = async ( .then(() => execute(observed)) .finally(() => workScope.clear()); const sources = sourceLedger.recordsForAnalysis(); - if (!result.ok) return withEvidenceSources(result, sources); + if (!result.ok && !("partialResult" in result)) + return withEvidenceSources(result, sources); for (const source of sources) evidenceLinks.add(source.evidence_id); + const observation = (value: JsonValue): ComposedWorkflowObservation => ({ + result: value, + limitations: [ + ...renderSourceLimitations([...limitations.values()]), + ...(omittedObservations === 0 + ? [] + : [ + `${omittedObservations} per-item provider limitation observation${omittedObservations === 1 ? " was" : "s were"} omitted from this answer; ${budget?.retainedIn}.`, + ]), + ], + ...(cachePolicy === undefined ? {} : { cachePolicy }), + ...(evidenceLinks.size === 0 + ? {} + : { evidenceLinks: [...evidenceLinks].sort(compareUnicodeCodePoints) }), + }); return withEvidenceSources( - ok({ - result: result.value, - limitations: [ - ...renderSourceLimitations([...limitations.values()]), - ...(omittedObservations === 0 - ? [] - : [ - `${omittedObservations} per-item provider limitation observation${omittedObservations === 1 ? " was" : "s were"} omitted from this answer; ${budget?.retainedIn}.`, - ]), - ], - ...(cachePolicy === undefined ? {} : { cachePolicy }), - ...(evidenceLinks.size === 0 - ? {} - : { evidenceLinks: [...evidenceLinks].sort(compareUnicodeCodePoints) }), - }), + result.ok + ? ok(observation(result.value)) + : { + ok: false as const, + error: result.error, + partial: observation(result.partialResult), + }, sources, ); }; @@ -206,6 +219,27 @@ export const createWorkflowEvidence = (input: { : {}), }); +/** Retain a stopped workflow's admitted observation as partial workflow Evidence on its failure. */ +export const retainPartialWorkflowEvidence = ( + outcome: ComposedWorkflowResult, + context: Pick< + Parameters[0], + "target" | "operation" | "parameters" | "upstreamProfile" + >, +): void => { + if (outcome.ok || outcome.partial === undefined) return; + const evidence = createWorkflowEvidence({ + ...context, + result: outcome.partial.result, + upstreamLimitations: outcome.partial.limitations, + ...(outcome.partial.evidenceLinks === undefined + ? {} + : { upstreamEvidenceLinks: outcome.partial.evidenceLinks }), + }); + // A partial observation the failing read already reported stays its own. + outcome.error.retainPartialObservation(evidence); +}; + const batchProcedureLocations = (result: JsonValue): EvidenceLocation[] => { if ( result === null || diff --git a/src/application/analysisView/AnalysisViewService.ts b/src/application/analysisView/AnalysisViewService.ts index 93e4f9f0b1..ac26cbd160 100644 --- a/src/application/analysisView/AnalysisViewService.ts +++ b/src/application/analysisView/AnalysisViewService.ts @@ -225,6 +225,7 @@ const projectResolvedAnalysisView = ( const projected = projection.project({ evidenceId: parent.evidence_id, operation: parent.operation, + predicateType: parent.predicate_type, provider: parent.provider, analysisProfile: parent.analysis_profile, authority: parent.authority, diff --git a/src/application/analysisView/BrowserScenarioViews.test.ts b/src/application/analysisView/BrowserScenarioViews.test.ts index 79bfd43fb1..bb48a39940 100644 --- a/src/application/analysisView/BrowserScenarioViews.test.ts +++ b/src/application/analysisView/BrowserScenarioViews.test.ts @@ -436,6 +436,7 @@ describe("browser scenario network observation", () => { const project = (capture: unknown) => { const result = projectBrowserScenarioView( { + predicateType: "rea.browser-scenario-capture", evidenceId: `ev:${"a".repeat(64)}`, operation: "capture_browser_scenario", normalizedResult: capture, diff --git a/src/application/analysisView/NativeLoadImageViews.test.ts b/src/application/analysisView/NativeLoadImageViews.test.ts index fe99d3f719..c1616d57c2 100644 --- a/src/application/analysisView/NativeLoadImageViews.test.ts +++ b/src/application/analysisView/NativeLoadImageViews.test.ts @@ -67,6 +67,38 @@ const view = async (report: NativeLoadImage, request: unknown) => { return analysisViewResultSchema.parse(result.value.normalized_result); }; +it("distinguishes recorded, absent, and historical unrecorded relocation symbols", async () => { + const report = source(); + report.observations.relocations[0]!.symbol = "external_symbol"; + report.observations.relocations[1]!.symbol = null; + const request = { kind: "native-load-image", collection: "relocations" }; + const current = await view(report, request); + if (current.kind !== "native-load-image") throw new Error("Expected page"); + if (current.items.collection !== "relocations") + throw new Error("Expected relocations"); + expect(current.items.rows).toMatchObject([ + { record: { symbol: "external_symbol" } }, + { record: { symbol: null } }, + ]); + expect(current.unknowns.join(" ")).not.toContain("symbol was not recorded"); + + report.observations.relocations[1]!.symbol = ""; + const emptyName = await view(report, request); + if (emptyName.kind !== "native-load-image") throw new Error("Expected page"); + if (emptyName.items.collection !== "relocations") + throw new Error("Expected relocations"); + expect(emptyName.items.rows?.[1]?.record).toMatchObject({ symbol: "" }); + expect(emptyName.unknowns.join(" ")).not.toContain("symbol was not recorded"); + + delete report.observations.relocations[0]!.symbol; + const historical = await view(report, request); + if (historical.kind !== "native-load-image") throw new Error("Expected page"); + expect(historical.items.rows?.[0]?.record).not.toHaveProperty("symbol"); + expect(historical.unknowns.join(" ")).toContain("symbol was not recorded"); + const omitted = await view(report, { ...request, fields: ["address"] }); + expect(omitted.unknowns.join(" ")).not.toContain("symbol was not recorded"); +}); + it("summarizes measured ELF facts without upgrading unsupported verification", async () => { const report = source(); const parent = evidence(report); @@ -302,6 +334,7 @@ it("rejects collection-inapplicable fields and malformed sibling facts even for ).toBe(false); const report = source(); const parent = { + predicateType: "rea.analysis", evidenceId: "ev_" + "b".repeat(64), operation: "inspect_native_load_image", normalizedResult: report, diff --git a/src/application/binary/AnalysisProviderEvaluation.ts b/src/application/binary/AnalysisProviderEvaluation.ts index 359401a76b..0d69c0d597 100644 --- a/src/application/binary/AnalysisProviderEvaluation.ts +++ b/src/application/binary/AnalysisProviderEvaluation.ts @@ -17,6 +17,7 @@ import type { } from "../AnalysisProvider.js"; import type { AnalysisProviderCandidateStatus } from "./AnalysisProviderRegistry.js"; import { ProviderAdapterError } from "../../domain/providerAdapterError.js"; +import { analysisErrorRemediationAction } from "../../domain/analysisErrorPresentation.js"; /** Profile-resolution truth for one discovered provider candidate. */ export interface AnalysisProviderCandidateEvaluation { @@ -76,9 +77,15 @@ export const evaluateAnalysisProviderCandidate = async ( }, }, } - : rejectedProfile(candidate, status, error.message, { - error_tag: error._tag, - }); + : rejectedProfile( + candidate, + status, + { + reason: error.message, + remediation: analysisErrorRemediationAction(error), + }, + { error_tag: error._tag }, + ); return ok({ ...rejection, profileError: error }); } if (signalIsAborted(signal)) @@ -88,11 +95,10 @@ export const evaluateAnalysisProviderCandidate = async ( try { if (resolved.value.profile === null) return ok( - rejectedProfile( - candidate, - status, - "Provider version or analysis profile could not be resolved", - ), + rejectedProfile(candidate, status, { + reason: "Provider version or analysis profile could not be resolved", + remediation: PROFILE_REMEDIATION, + }), ); const profile = analysisProfileSchema.parse(resolved.value.profile); const identity = candidate.identity(); @@ -103,11 +109,10 @@ export const evaluateAnalysisProviderCandidate = async ( profile.provider.version !== identity.version) ) return ok( - rejectedProfile( - candidate, - status, - "Resolved analysis profile has mismatched provider identity", - ), + rejectedProfile(candidate, status, { + reason: "Resolved analysis profile has mismatched provider identity", + remediation: PROFILE_REMEDIATION, + }), ); return ok({ candidate, @@ -120,11 +125,10 @@ export const evaluateAnalysisProviderCandidate = async ( } catch (cause: unknown) { void cause; return ok( - rejectedProfile( - candidate, - status, - "Provider returned an invalid analysis profile", - ), + rejectedProfile(candidate, status, { + reason: "Provider returned an invalid analysis profile", + remediation: PROFILE_REMEDIATION, + }), ); } }; @@ -132,10 +136,13 @@ export const evaluateAnalysisProviderCandidate = async ( const signalIsAborted = (signal: AbortSignal | undefined): boolean => signal?.aborted === true; +const PROFILE_REMEDIATION = + "Run rea doctor to check this provider's installation and version, then reopen the target, or select another analysis provider."; + const rejectedProfile = ( candidate: AnalysisProviderCandidate, status: AnalysisProviderCandidateStatus, - reason: string, + rejection: { readonly reason: string; readonly remediation: string }, diagnostics: Readonly> = {}, ): AnalysisProviderCandidateEvaluation => ({ candidate, @@ -144,7 +151,7 @@ const rejectedProfile = ( availability: { status: "unavailable", code: "version_unresolved", - reason, + ...rejection, diagnostics: jsonObjectSchema.parse({ ...status.availability.diagnostics, ...diagnostics, diff --git a/src/application/binary/AnalysisProviderRegistry.test.ts b/src/application/binary/AnalysisProviderRegistry.test.ts index a0497b9b11..1eb030451c 100644 --- a/src/application/binary/AnalysisProviderRegistry.test.ts +++ b/src/application/binary/AnalysisProviderRegistry.test.ts @@ -93,6 +93,7 @@ it("uses one owner readiness snapshot for candidate availability and capabilitie status: "unavailable", code: "runtime_unavailable", reason: "native inspection failed", + remediation: "Repair the fixture provider.", diagnostics: { failure_stage: "self-test" }, }, capabilities: [capability(base.identity())], @@ -462,6 +463,7 @@ describe("analysis provider registry: rejection origins", () => { status: "unavailable", code: "open_options_invalid", reason: "Configured registration is invalid", + remediation: "Repair the fixture provider.", diagnostics: { registration_path: "/configured/alpha.json" }, }), }, @@ -727,6 +729,7 @@ const candidate = ( status: "unavailable", code: options.unconfigured ? "not_configured" : "runtime_missing", reason: `${id} runtime is unavailable`, + remediation: "Repair the fixture provider.", diagnostics: { executable_path: `/opt/${id}/bin/analyze` }, }, inspectTargetSupport: (target): ProviderTargetSupport => diff --git a/src/application/binary/AnalysisProviderRegistry.ts b/src/application/binary/AnalysisProviderRegistry.ts index ba7df2051f..ef6cdb8012 100644 --- a/src/application/binary/AnalysisProviderRegistry.ts +++ b/src/application/binary/AnalysisProviderRegistry.ts @@ -914,6 +914,7 @@ const selectedRejection = ( origin, code: rejection.code, reason: rejection.reason, + ...("remediation" in rejection ? { remediation: rejection.remediation } : {}), // ProviderSelectionError owns the single defensive diagnostic copy. diagnostics: rejection.diagnostics, }); diff --git a/src/application/binary/AnalysisSnapshotCache.ts b/src/application/binary/AnalysisSnapshotCache.ts index 8263cd55f5..1289c2cb43 100644 --- a/src/application/binary/AnalysisSnapshotCache.ts +++ b/src/application/binary/AnalysisSnapshotCache.ts @@ -20,8 +20,8 @@ import { createAnalysisSnapshotEntry, createAnalysisSnapshotWorkflowEntry, snapshotBinding, + snapshotBindingMismatch, snapshotMatchesProfile, - snapshotMatchesTarget, snapshotTarget, parseAnalysisSnapshot, immutableAnalysisSnapshot, @@ -30,6 +30,7 @@ import { type AnalysisSnapshotWorkflowEntry, type AnalysisSnapshotBinding, type AnalysisSnapshotTarget, + type SnapshotBindingMismatch, } from "../../domain/analysisSnapshot.js"; import type { AnalysisExecution, @@ -158,13 +159,22 @@ export class AnalysisSnapshotCache { target: BinaryTarget, profile: AnalysisProfileCommitment | undefined, ): boolean { - return ( - (this.#target === undefined && this.#binding === undefined) || - (this.#target !== undefined && - this.#binding !== undefined && - profile !== undefined && - snapshotMatchesTarget(this.#target, target) && - snapshotMatchesProfile(this.#binding, profile)) + return this.mismatch(target, profile) === undefined; + } + + /** Explain staged mismatch without treating missing binding facts as absent. */ + mismatch( + target: BinaryTarget, + profile: AnalysisProfileCommitment | undefined, + ): SnapshotBindingMismatch | undefined { + if (this.#target === undefined && this.#binding === undefined) + return undefined; + if (this.#target === undefined || this.#binding === undefined) + return { staged_binding: "incomplete" }; + return snapshotBindingMismatch( + { target: this.#target, binding: this.#binding }, + target, + profile, ); } @@ -388,16 +398,20 @@ export class AnalysisSnapshotCache { }), ); } - if ( - active !== undefined && - (!snapshotMatchesTarget(validated.target, active.target) || - !snapshotMatchesProfile(validated.binding, active.profile)) - ) - return err( - new AnalysisSnapshotMismatchError( - "Analysis snapshot profile_mismatch: target, provider, or analysis profile does not match the active binary", - ), + if (active !== undefined) { + const mismatch = snapshotBindingMismatch( + validated, + active.target, + active.profile, ); + if (mismatch !== undefined) + return err( + new AnalysisSnapshotMismatchError( + "Analysis snapshot target, provider, or analysis profile does not match the active binary", + mismatch, + ), + ); + } const importedEvidence = mergeEvidence(validated.evidence_bundle); return importedEvidence.ok ? ok(this.stage(validated)) : importedEvidence; } diff --git a/src/application/binary/BinarySession.ts b/src/application/binary/BinarySession.ts index 1a1ecdbc73..84369c5253 100644 --- a/src/application/binary/BinarySession.ts +++ b/src/application/binary/BinarySession.ts @@ -49,6 +49,7 @@ import { InvestigationRecords } from "../investigation/InvestigationRecords.js"; import { recordEvidenceSources } from "../investigation/recordEvidenceSources.js"; import { BinarySessionRecords } from "./BinarySessionRecords.js"; import { binarySessionStatus } from "./BinarySessionStatus.js"; +import { SessionRuntimeAvailability } from "./SessionRuntimeAvailability.js"; import { isSourceEvidenceRetainable } from "./AnalysisSnapshotCache.js"; import type { BinarySessionDetail } from "../../contracts/sessionStatusContract.js"; import { @@ -179,6 +180,7 @@ export class BinarySession { #active: SessionBinding | undefined; #externalMutationCleanupFailure: AnalysisError | undefined; + #lastCloseCleanupFailure: AnalysisError | undefined; #pendingCleanup: | { readonly run: SessionRun; @@ -197,10 +199,7 @@ export class BinarySession readonly #bundleResourceFactory: BundleResourceClientFactory | undefined; #activeBundleResources: ActiveBundleResourceSession | undefined; #pendingBundleResourceCleanup: ActiveBundleResourceSession[] = []; - readonly #runtimeUnavailability = new Map< - string, - { readonly reason: string } - >(); + readonly #runtimeAvailability = new SessionRuntimeAvailability(); readonly #availabilityListeners = new Set<() => void | Promise>(); constructor( @@ -459,8 +458,8 @@ export class BinarySession current: this.#active, target, options, - stagedSnapshotMatches: (target, profile) => - this.matchesSnapshot(target, profile), + stagedSnapshotMismatch: (target, profile) => + this.snapshotMismatch(target, profile), }).then((resolved) => resolved.ok && isAborted(options.signal) ? err( @@ -488,8 +487,8 @@ export class BinarySession current: this.#active, path, options: admitted, - stagedSnapshotMatches: (target, profile) => - this.matchesSnapshot(target, profile), + stagedSnapshotMismatch: (target, profile) => + this.snapshotMismatch(target, profile), }), options, ); @@ -507,8 +506,8 @@ export class BinarySession ...resolved, current: this.#active, options, - stagedSnapshotMatches: (target, profile) => - this.matchesSnapshot(target, profile), + stagedSnapshotMismatch: (target, profile) => + this.snapshotMismatch(target, profile), }), ), options, @@ -1012,16 +1011,31 @@ export class BinarySession closed: Result, retry: AnalysisCleanupRetry | undefined, ): Result { + this.#lastCloseCleanupFailure = undefined; if ( !closed.ok && retry !== undefined && closed.error.cleanupIncomplete && this.#retainsCloseCleanup() - ) + ) { closed.error.bindCleanupRetry(retry); + this.#lastCloseCleanupFailure = closed.error; + } return closed; } + /** Only a retired target's known close owner can offer this status retry. */ + #statusCleanupRetry(): AnalysisCleanupRetry | undefined { + if ( + this.#active !== undefined || + this.#activeBundleResources !== undefined || + this.#externalMutationCleanupFailure?.cleanupIncomplete === true || + !this.#retainsCloseCleanup() + ) + return undefined; + return this.#lastCloseCleanupFailure?.cleanupRetry; + } + /** Owners the next close retries; external mutation owners retry elsewhere. */ #retainsCloseCleanup(): boolean { return ( @@ -1240,6 +1254,7 @@ export class BinarySession this.#pendingBundleResourceCleanup[0] ?? this.#pendingCleanup?.run; const client = run?.client; + const cleanupRetry = this.#statusCleanupRetry(); return binarySessionStatus({ ...options, investigation: this.investigationStatistics(), @@ -1252,11 +1267,12 @@ export class BinarySession }), route: this.#currentRoute(), router: this.#providerRouter, - runtimeUnavailability: this.#runtimeUnavailability, + runtimeUnavailability: this.#runtimeAvailability.entries, runId: run?.runId, runtimeLineageSnapshots: client?.runtimeLineageSnapshots?.() ?? [], requestActivitySnapshots: client?.requestActivitySnapshots?.() ?? [], providerOperationHealth: client?.operationHealthSnapshot?.() ?? null, + ...(cleanupRetry === undefined ? {} : { cleanupRetry }), cleanupPending: this.#externalMutationCleanupFailure?.cleanupIncomplete === true || this.#retainsCloseCleanup(), @@ -1414,6 +1430,7 @@ export class BinarySession // A failed reply cannot prove that a dispatched metadata write rolled back. // Stop replay before dispatch, including reads overlapping the write. if (capability.effects.mutatesArtifact) this.invalidateSnapshot(); + const dispatch = this.#runtimeAvailability.dispatch(); const call = active.client.execute(name, arguments_, options); const result = unrecordedExecution(await call); const profiled = bindExecutionTarget( @@ -1442,7 +1459,15 @@ export class BinarySession this.#active === active ) this.#active.nativeSourceAdmitted = true; - this.#observeRuntimeAvailability(name, execution); + if ( + this.#runtimeAvailability.observe( + dispatch, + capability, + active.route.capabilities.values(), + execution, + ) + ) + this.#emitAvailabilityChanged(); // Source facts belong to the observation, even when this query cannot be // replayed (for example a Ghidra procedure read without a document selector). if (execution.ok && execution.value.sourceEvidence !== undefined) { @@ -1486,56 +1511,8 @@ export class BinarySession return execution; } - #observeRuntimeAvailability( - operation: AnalysisOperation, - result: Result, - ): void { - if (result.ok) { - if (this.#markRuntimeAvailable(operation)) - this.#emitAvailabilityChanged(); - return; - } - if (result.error._tag === "AnalysisCapabilityUnavailableError") { - if (this.#markRuntimeUnavailable(operation, result.error.message)) - this.#emitAvailabilityChanged(); - return; - } - if ( - [ - "ProviderAdapterError", - "HopperProcessError", - "HopperStartError", - ].includes(result.error._tag) - ) { - const capabilities = this.#active?.route.capabilities; - const providerId = capabilities?.get(operation)?.provider.id; - let changed = false; - for (const descriptor of capabilities?.values() ?? []) - if (providerId !== undefined && descriptor.provider.id === providerId) - changed = - this.#markRuntimeUnavailable( - descriptor.operation, - `${result.error._tag}: ${result.error.message}`, - ) || changed; - if (changed) this.#emitAvailabilityChanged(); - } - } - - #markRuntimeAvailable(operation: string): boolean { - return this.#runtimeUnavailability.delete(operation); - } - - #markRuntimeUnavailable(operation: string, reason: string): boolean { - const current = this.#runtimeUnavailability.get(operation); - if (current?.reason === reason) return false; - this.#runtimeUnavailability.set(operation, { reason }); - return true; - } - #clearRuntimeAvailability(): void { - if (this.#runtimeUnavailability.size === 0) return; - this.#runtimeUnavailability.clear(); - this.#emitAvailabilityChanged(); + if (this.#runtimeAvailability.clear()) this.#emitAvailabilityChanged(); } #emitAvailabilityChanged(): void { diff --git a/src/application/binary/BinarySessionInvestigation.test.ts b/src/application/binary/BinarySessionInvestigation.test.ts index abad3a7ea6..81039e084c 100644 --- a/src/application/binary/BinarySessionInvestigation.test.ts +++ b/src/application/binary/BinarySessionInvestigation.test.ts @@ -99,8 +99,10 @@ it("publishes engine retirement with its pending cleanup owner already visible", pending.push(closing); await awaitAcknowledgement(entered.promise, closing, "engine close entry"); expect(notifications.length).toBeGreaterThan(0); - for (const status of await Promise.all(notifications)) + for (const status of await Promise.all(notifications)) { expect(status).toMatchObject({ open: false, cleanup_pending: true }); + expect(status).not.toHaveProperty("cleanup_retry"); + } release.resolve(); expect((await closing).ok).toBe(true); expect(await session.status()).toMatchObject({ diff --git a/src/application/binary/BinarySessionOpen.ts b/src/application/binary/BinarySessionOpen.ts index f1ccc7ff57..81b4655c4f 100644 --- a/src/application/binary/BinarySessionOpen.ts +++ b/src/application/binary/BinarySessionOpen.ts @@ -8,9 +8,10 @@ import { artifactTargetSelectedPath } from "../../domain/binaryTargetTypes.js"; import type { AnalysisProviderSelector } from "../../domain/providerSelection.js"; import type { AnalysisProfileCommitment } from "../../domain/analysisProfile.js"; import { - snapshotMatchesBinding, + snapshotBindingMismatch, snapshotMatchesTarget, snapshotTarget, + type SnapshotBindingMismatch, } from "../../domain/analysisSnapshot.js"; import type { AnalysisSnapshot } from "../../domain/analysisSnapshot.js"; import { parseBinaryTarget } from "../BinaryTargetResolver.js"; @@ -23,7 +24,7 @@ import type { AnalysisError } from "../../domain/analysisErrorBase.js"; import { err, type Result } from "../../domain/result.js"; import type { SessionProviderRoute } from "./SessionProviderRouter.js"; import { - isProfilelessNativeDispatchSnapshot, + profilelessSnapshotMismatch, profilelessNativeDispatchProfile, } from "./ProfilelessNativeWorkflowSnapshot.js"; import { SessionProviderRouter } from "./SessionProviderRouter.js"; @@ -68,10 +69,10 @@ interface ResolveSessionOpenInput { readonly current: CurrentOpenBinding | undefined; readonly path: string; readonly options: BinarySessionOpenOptions; - readonly stagedSnapshotMatches: ( + readonly stagedSnapshotMismatch: ( target: BinaryTarget, profile: AnalysisProfileCommitment | null, - ) => boolean; + ) => SnapshotBindingMismatch | undefined; } interface ResolveSessionTargetInput { @@ -79,10 +80,10 @@ interface ResolveSessionTargetInput { readonly current: CurrentOpenBinding | undefined; readonly target: BinaryTarget; readonly options: BinarySessionOpenOptions; - readonly stagedSnapshotMatches: ( + readonly stagedSnapshotMismatch: ( target: BinaryTarget, profile: AnalysisProfileCommitment | null, - ) => boolean; + ) => SnapshotBindingMismatch | undefined; } /** Parse a target, resolve its provider route, and validate snapshot binding. */ @@ -134,7 +135,7 @@ export const validateResolvedSessionTarget = ( readonly route: SessionProviderRoute; }, ): Result => { - const { current, target, route, options, stagedSnapshotMatches } = input; + const { current, target, route, options, stagedSnapshotMismatch } = input; const sameTarget = sameSessionTarget(current?.target, target); const snapshotError = validateSnapshot({ snapshot: options.snapshot, @@ -142,7 +143,7 @@ export const validateResolvedSessionTarget = ( profile: route.profile, profilelessWorkflowProfile: profilelessNativeDispatchProfile(route), current, - stagedSnapshotMatches, + stagedSnapshotMismatch, }); if (snapshotError !== undefined) return err(snapshotError); return { ok: true, value: { target, route, sameTarget } }; @@ -166,10 +167,10 @@ interface ValidateSnapshotInput { readonly profile: AnalysisProfileCommitment | null; readonly profilelessWorkflowProfile: AnalysisProfileCommitment | undefined; readonly current: CurrentOpenBinding | undefined; - readonly stagedSnapshotMatches: ( + readonly stagedSnapshotMismatch: ( target: BinaryTarget, profile: AnalysisProfileCommitment | null, - ) => boolean; + ) => SnapshotBindingMismatch | undefined; } const validateSnapshot = ({ @@ -178,31 +179,34 @@ const validateSnapshot = ({ profile, profilelessWorkflowProfile, current, - stagedSnapshotMatches, + stagedSnapshotMismatch, }: ValidateSnapshotInput): EvidenceIntegrityError | undefined => { - if ( - snapshot !== undefined && - (profile === null - ? profilelessWorkflowProfile === undefined || - !isProfilelessNativeDispatchSnapshot( - snapshot, + if (snapshot !== undefined) { + const mismatch = + profile === null + ? profilelessSnapshotMismatch( + snapshot, + target, + profilelessWorkflowProfile, + ) + : snapshotBindingMismatch(snapshot, target, profile); + if (mismatch !== undefined) + return new AnalysisSnapshotMismatchError( + "Analysis snapshot target, provider, or analysis profile does not match the requested binary", + mismatch, + ); + } + const stagedMismatch = + current === undefined + ? stagedSnapshotMismatch( target, - profilelessWorkflowProfile, + profile ?? profilelessWorkflowProfile ?? null, ) - : !snapshotMatchesBinding(snapshot, target, profile)) - ) + : undefined; + if (stagedMismatch !== undefined) return new AnalysisSnapshotMismatchError( - "Analysis snapshot profile_mismatch: target, provider, or analysis profile does not match the requested binary", - ); - if ( - current === undefined && - !stagedSnapshotMatches( - target, - profile ?? profilelessWorkflowProfile ?? null, - ) - ) - return new AnalysisSnapshotMismatchError( - "Analysis snapshot profile_mismatch: staged target, provider, or analysis profile does not match", + "Analysis snapshot does not match the staged target, provider, or analysis profile", + stagedMismatch, ); return undefined; }; diff --git a/src/application/binary/BinarySessionRecords.ts b/src/application/binary/BinarySessionRecords.ts index 9bb334c1ac..3333bc37d6 100644 --- a/src/application/binary/BinarySessionRecords.ts +++ b/src/application/binary/BinarySessionRecords.ts @@ -47,7 +47,7 @@ import { ProviderCleanupError } from "../../domain/providerCleanupError.js"; import { workflowAnalysisProfile } from "../WorkflowAnalysisProfile.js"; import { hasOnlyProfilelessNativeDispatchEntries, - isProfilelessNativeDispatchSnapshot, + profilelessSnapshotMismatch, } from "./ProfilelessNativeWorkflowSnapshot.js"; const SNAPSHOT_MUTATION_RECOVERY = @@ -267,16 +267,20 @@ export abstract class BinarySessionRecords { ), ); const profile = active?.profile ?? active?.profilelessWorkflowProfile; - if ( - active?.profile === null && - (profile === undefined || - !isProfilelessNativeDispatchSnapshot(snapshot, active.target, profile)) - ) - return err( - new AnalysisSnapshotMismatchError( - "Analysis snapshot profile_mismatch: the snapshot is not bound to the active native dispatch reader", - ), + if (active?.profile === null) { + const mismatch = profilelessSnapshotMismatch( + snapshot, + active.target, + profile, ); + if (mismatch !== undefined) + return err( + new AnalysisSnapshotMismatchError( + "Analysis snapshot is not bound to the active native dispatch reader", + mismatch, + ), + ); + } const imported = this.#snapshot.import( snapshot, active === undefined @@ -295,11 +299,11 @@ export abstract class BinarySessionRecords { return imported; } - protected matchesSnapshot( + protected snapshotMismatch( target: BinaryTarget, profile: AnalysisProfileCommitment | null, - ): boolean { - return this.#snapshot.matches(target, profile ?? undefined); + ) { + return this.#snapshot.mismatch(target, profile ?? undefined); } protected selectSnapshot( diff --git a/src/application/binary/BinarySessionStatus.test.ts b/src/application/binary/BinarySessionStatus.test.ts index c1d97d8eda..e4e81f0dfe 100644 --- a/src/application/binary/BinarySessionStatus.test.ts +++ b/src/application/binary/BinarySessionStatus.test.ts @@ -28,6 +28,10 @@ import { nativeMacOSCapabilities, } from "../../native/NativeMacOSProviderMetadata.js"; import { NATIVE_MACOS_PROVIDER_IDENTITY } from "../../native/NativeMacosProviderIdentity.js"; +import { + buildCapabilityInventory, + MULTI_MODE_REMEDIATION, +} from "../CapabilityInventory.js"; const projectedStatusSchema = z.object({ capabilities: sessionProviderFacts.shape.capabilities, @@ -517,6 +521,146 @@ it.each(["elf", "pe"] as const)( }, ); +const dispatchTarget = (format: "elf" | "mach-o"): BinaryTarget => ({ + kind: "executable", + format, + path: "/fixture", + sha256: "a".repeat(64), + architecture: "x86_64", + availableArchitectures: ["x86_64"], +}); + +it.each([ + { format: "elf", symbols: false, nativeAvailable: false, route: false }, + { format: "elf", symbols: true, nativeAvailable: false, route: true }, + { format: "mach-o", symbols: false, nativeAvailable: true, route: true }, +] as const)( + "projects $format dispatch applicability and symbol fallback ($symbols)", + async ({ format, symbols, nativeAvailable, route }) => { + const target = dispatchTarget(format); + const native = nativeMacOSCapabilities("darwin").find( + (item) => item.operation === "inspect_native_dispatch_metadata", + ); + if (native === undefined) throw new Error("Missing native dispatch reader"); + const capabilities = [ + native, + descriptor(providerA, "decode_interface_builder", []), + descriptor(providerA, "procedure_address", []), + descriptor(providerA, "procedure_callees", []), + ...(symbols ? [descriptor(providerA, "list_names", [])] : []), + ]; + const status = await projectStatus(capabilities, [], { target }); + const parsed = projectedStatusSchema.parse(status); + const projected = expandCanonicalCapabilities(parsed.capabilities).find( + (item) => item.operation === "inspect_native_dispatch_metadata", + ); + expect(projected).toMatchObject({ + available: nativeAvailable, + availability_code: nativeAvailable ? null : "target_format_unsupported", + }); + const inventory = buildCapabilityInventory(status, { + processCaptureEnabled: false, + }); + const dispatch = inventory.find( + (item) => item.name === "inspect_native_dispatch_metadata", + ); + expect(dispatch?.available).toBe(route); + const trace = inventory.find( + (item) => item.name === "trace_native_ui_action", + ); + expect( + trace?.modes?.find((mode) => mode.name === "native_seed_call_trace"), + ).toMatchObject({ + available: route, + missing_operations: route + ? [] + : ["inspect_native_dispatch_metadata", "list_names"], + }); + if (!route) { + expect(trace?.remediation).toBe(MULTI_MODE_REMEDIATION); + const authored = trace?.modes?.find( + (mode) => mode.name === "authored_ui_dispatch", + ); + const nativeSeed = trace?.modes?.find( + (mode) => mode.name === "native_seed_call_trace", + ); + expect(authored?.remediation).not.toBe(nativeSeed?.remediation); + expect(authored?.missing_operations).toContain( + "decode_interface_builder", + ); + } + }, +); + +it("does not impose the Apple reader's target rule on another provider", async () => { + const target = dispatchTarget("elf"); + const status = await projectStatus( + [descriptor(providerA, "inspect_native_dispatch_metadata", [])], + [], + { target }, + ); + const projected = expandCanonicalCapabilities( + projectedStatusSchema.parse(status).capabilities, + ); + expect(projected).toMatchObject([ + { operation: "inspect_native_dispatch_metadata", available: true }, + ]); + const inventory = buildCapabilityInventory(status, { + processCaptureEnabled: false, + }); + expect( + inventory.find((item) => item.name === "inspect_native_dispatch_metadata"), + ).toMatchObject({ available: true }); +}); + +it("keeps distinct recovery for an unbound non-bundle session", async () => { + const native = nativeMacOSCapabilities("darwin").find( + (item) => item.operation === "inspect_native_dispatch_metadata", + ); + if (native === undefined) throw new Error("Missing native dispatch reader"); + const status = await projectStatus( + [native, descriptor(providerA, "decode_interface_builder", [])], + [], + { target: dispatchTarget("mach-o") }, + ); + const inventory = buildCapabilityInventory(status, { + processCaptureEnabled: false, + }); + const trace = inventory.find( + (item) => item.name === "trace_native_ui_action", + ); + expect(trace).toMatchObject({ + available: false, + reason: "provider_unavailable", + remediation: MULTI_MODE_REMEDIATION, + }); + const authored = trace?.modes?.find( + (mode) => mode.name === "authored_ui_dispatch", + ); + const nativeSeed = trace?.modes?.find( + (mode) => mode.name === "native_seed_call_trace", + ); + expect(authored?.missing_operations).toContain("decode_interface_builder"); + expect(nativeSeed?.missing_operations).toContain("procedure_address"); + expect(authored?.remediation).not.toBe(nativeSeed?.remediation); +}); + +it("keeps host unavailability ahead of target applicability", async () => { + const native = nativeMacOSCapabilities("linux").find( + (item) => item.operation === "inspect_native_dispatch_metadata", + ); + if (native === undefined) throw new Error("Missing native dispatch reader"); + const target = dispatchTarget("elf"); + const projected = await project([native], [], new Map(), target); + expect(expandCanonicalCapabilities(projected.capabilities)).toMatchObject([ + { + operation: "inspect_native_dispatch_metadata", + available: false, + availability_code: "unsupported_host", + }, + ]); +}); + describe("binary session summary projection", () => { it("keeps each candidate's status, code and reason in summary without diagnostics or capability sets", async () => { const unavailable: AnalysisProviderCandidateStatus = { @@ -524,7 +668,8 @@ describe("binary session summary projection", () => { availability: { status: "unavailable", code: "not_configured", - reason: "Set PROVIDER_B_HOME; see docs/provider-b.md.", + reason: "PROVIDER_B_HOME is not set.", + remediation: "Set PROVIDER_B_HOME; see docs/provider-b.md.", diagnostics: { checks: [{ name: "configuration", ok: false }] }, }, targetSupport: { @@ -582,7 +727,7 @@ describe("binary session summary projection", () => { availability: { status: "unavailable", code: "not_configured", - reason: "Set PROVIDER_B_HOME; see docs/provider-b.md.", + reason: "PROVIDER_B_HOME is not set.", }, target_support: { status: "unsupported", @@ -593,6 +738,13 @@ describe("binary session summary projection", () => { }, ]); expect(JSON.stringify(raw)).not.toContain("diagnostics"); + // Complete detail keeps the owner's repair; summary omits it. + expect( + complete.analysis_provider_candidates[1]?.availability, + ).toMatchObject({ + remediation: "Set PROVIDER_B_HOME; see docs/provider-b.md.", + }); + expect(JSON.stringify(raw)).not.toContain("PROVIDER_B_HOME;"); expect( analysisProviderCandidate .array() @@ -631,3 +783,68 @@ it("narrows summary capabilities without inventing omitted candidate facts", asy reason: "No target is open.", }); }); + +it("groups readiness repairs exactly and drops one replaced by a runtime observation", async () => { + const unavailable = ( + operation: CapabilityDescriptor["operation"], + remediation: string, + ): CapabilityDescriptor => { + const declared = descriptor(providerA, operation, [], { + available: false, + reason: "Configured registration is invalid.", + }); + if (declared.available) throw new Error("Expected an unavailable fixture"); + return { ...declared, remediation }; + }; + const status = await project( + [ + unavailable("procedure_info", "Correct REA_FIXTURE_CONFIG."), + unavailable("read_bytes", "Correct REA_FIXTURE_CONFIG."), + unavailable("list_names", "Unset REA_FIXTURE_CACHE."), + unavailable("xrefs", "Correct REA_FIXTURE_CONFIG."), + ], + [], + new Map([["xrefs", { reason: "Runtime operation failure" }]]), + ); + const groups = status.capabilities.flatMap(({ groups }) => groups); + expect(groups).toEqual([ + expect.objectContaining({ + operations: ["list_names"], + remediation: "Unset REA_FIXTURE_CACHE.", + }), + expect.objectContaining({ + operations: ["procedure_info", "read_bytes"], + reason: "Configured registration is invalid.", + remediation: "Correct REA_FIXTURE_CONFIG.", + }), + expect.not.objectContaining({ remediation: expect.anything() }), + ]); + expect(groups[2]).toMatchObject({ + operations: ["xrefs"], + reason: "Runtime operation failure", + }); + const inventory = new Map( + buildCapabilityInventory( + { open: true, kind: "executable", capabilities: status.capabilities }, + { processCaptureEnabled: true }, + ).map((entry) => [entry.name, entry]), + ); + expect(inventory.get("procedure_info")?.remediation).toBe( + "Correct REA_FIXTURE_CONFIG.", + ); + // A static or runtime-only refusal has no separate repair to prefer. + expect(inventory.get("xrefs")?.remediation).toBe("Runtime operation failure"); + // A scoped registration narrows operations without dropping their repair. + const scoped = projectedStatusSchema.parse( + restrictAdvertisedCapabilities( + status, + (operation) => operation === "procedure_info", + ), + ); + expect(scoped.capabilities.flatMap(({ groups }) => groups)).toEqual([ + expect.objectContaining({ + operations: ["procedure_info"], + remediation: "Correct REA_FIXTURE_CONFIG.", + }), + ]); +}); diff --git a/src/application/binary/BinarySessionStatus.ts b/src/application/binary/BinarySessionStatus.ts index 563b0637e1..f17413f91e 100644 --- a/src/application/binary/BinarySessionStatus.ts +++ b/src/application/binary/BinarySessionStatus.ts @@ -1,6 +1,7 @@ import { z } from "zod"; import type { InvestigationStatistics } from "../../domain/investigationLifecycle.js"; import { artifactTargetRefusal } from "../../domain/apple/artifactTargetApplicability.js"; +import { targetApplicabilityRefusal } from "../../domain/targetApplicability.js"; import type { BinaryTarget } from "../../domain/binaryTargetTypes.js"; import type { ResolvedAppBundleResourceRoot } from "../AppBundleExecutable.js"; import { binaryTargetSummary } from "./BinaryTargetSummary.js"; @@ -25,6 +26,7 @@ import type { } from "./SessionProviderRouter.js"; import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js"; import type { BinarySessionDetail } from "../../contracts/sessionStatusContract.js"; +import type { AnalysisCleanupRetry } from "../../domain/analysisErrorRecovery.js"; interface BinarySessionStatusInput { readonly investigation: InvestigationStatistics; @@ -42,6 +44,7 @@ interface BinarySessionStatusInput { readonly requestActivitySnapshots: readonly ProviderRequestActivitySnapshot[]; readonly providerOperationHealth?: ProviderOperationHealth | null; readonly cleanupPending?: boolean; + readonly cleanupRetry?: AnalysisCleanupRetry; readonly detail?: BinarySessionDetail; readonly signal?: AbortSignal; } @@ -60,6 +63,7 @@ export const binarySessionStatus = async ({ requestActivitySnapshots, providerOperationHealth, cleanupPending = false, + cleanupRetry, detail = "complete", signal, }: BinarySessionStatusInput): Promise => { @@ -76,7 +80,12 @@ export const binarySessionStatus = async ({ const targetRefusal = target === undefined ? undefined - : artifactTargetRefusal(descriptor.operation, target); + : descriptor.targetApplicability === undefined + ? artifactTargetRefusal(descriptor.operation, target) + : targetApplicabilityRefusal( + descriptor.targetApplicability, + target, + ); if (targetRefusal !== undefined && descriptor.available) return { ...descriptor, @@ -87,11 +96,7 @@ export const binarySessionStatus = async ({ const unavailable = runtimeUnavailability.get(descriptor.operation); return unavailable === undefined ? descriptor - : { - ...descriptor, - available: false as const, - reason: unavailable.reason, - }; + : runtimeUnavailable(descriptor, unavailable.reason); }); const capabilities = providerCapabilitySets(routeCapabilities); const analysisProviderBinding = @@ -135,6 +140,9 @@ export const binarySessionStatus = async ({ const common = { investigation, cleanup_pending: cleanupPending, + ...(cleanupRetry === undefined + ? {} + : { cleanup_retry: { ...cleanupRetry } }), capabilities, analysis_provider_binding: analysisProviderBinding, analysis_provider_candidates: analysisProviderCandidates, @@ -172,6 +180,19 @@ export const binarySessionStatus = async ({ }; }; +/** A runtime observation replaces the declared state and its repair instruction. */ +const runtimeUnavailable = ( + descriptor: CapabilityDescriptor, + reason: string, +): CapabilityDescriptor => { + if (descriptor.available) { + const { availabilityCode: _available, ...declared } = descriptor; + return { ...declared, available: false, reason }; + } + const { remediation: _superseded, ...declared } = descriptor; + return { ...declared, reason }; +}; + /** Capability surfaces a registration may narrow without hiding provider facts. */ const advertisedCapabilitySurfaces = z.object({ capabilities: z.array(providerCapabilitySet), @@ -332,7 +353,11 @@ type CapabilityProjection = Omit< "operation" | "available" | "reason" > & { readonly operation: string } & ( | { readonly available: true; readonly reason: null } - | { readonly available: false; readonly reason: string } + | { + readonly available: false; + readonly reason: string; + readonly remediation?: string; + } ); type ProviderCapabilityGroup = ProviderCapabilitySet["groups"][number]; @@ -382,6 +407,9 @@ export const providerCapabilitySets = ( available: false as const, reason: descriptor.reason, availability_code: availabilityCode, + ...(descriptor.remediation === undefined + ? {} + : { remediation: descriptor.remediation }), }; const key = JSON.stringify([ effects, @@ -389,6 +417,7 @@ export const providerCapabilitySets = ( status.available, status.reason, status.availability_code, + status.available ? null : (status.remediation ?? null), ]); const group = groups.get(key); if (group === undefined) diff --git a/src/application/binary/ProfilelessNativeWorkflowSnapshot.ts b/src/application/binary/ProfilelessNativeWorkflowSnapshot.ts index 7958007aa2..3640029822 100644 --- a/src/application/binary/ProfilelessNativeWorkflowSnapshot.ts +++ b/src/application/binary/ProfilelessNativeWorkflowSnapshot.ts @@ -1,8 +1,9 @@ import { analysisProfilesEqual } from "../../domain/analysisProfile.js"; import type { AnalysisProfileCommitment } from "../../domain/analysisProfile.js"; import { - snapshotMatchesBinding, + snapshotBindingMismatch, type AnalysisSnapshot, + type SnapshotBindingMismatch, } from "../../domain/analysisSnapshot.js"; import type { BinaryTarget } from "../../domain/binaryTargetTypes.js"; import { workflowAnalysisProfile } from "../WorkflowAnalysisProfile.js"; @@ -28,15 +29,22 @@ export const profilelessNativeDispatchProfile = ( return structuredClone(profile); }; -/** Require only exact native dispatch workflow observations for this target/profile. */ -export const isProfilelessNativeDispatchSnapshot = ( +/** Preserve the difference between a binding mismatch and incompatible contents. */ +export const profilelessSnapshotMismatch = ( snapshot: AnalysisSnapshot, target: BinaryTarget, - profile: AnalysisProfileCommitment, -): boolean => - snapshotMatchesBinding(snapshot, target, profile) && - snapshot.entries.length === 0 && - hasOnlyProfilelessNativeDispatchEntries(snapshot.workflow_entries, profile); + profile: AnalysisProfileCommitment | undefined, +): SnapshotBindingMismatch | undefined => { + const mismatch = snapshotBindingMismatch(snapshot, target, profile); + if (mismatch !== undefined) return mismatch; + if ( + profile === undefined || + snapshot.entries.length !== 0 || + !hasOnlyProfilelessNativeDispatchEntries(snapshot.workflow_entries, profile) + ) + return { workflow_scope: "incompatible" }; + return undefined; +}; /** Reject mixed operations and entries derived from a different source profile. */ export const hasOnlyProfilelessNativeDispatchEntries = ( diff --git a/src/application/binary/SessionRuntimeAvailability.ts b/src/application/binary/SessionRuntimeAvailability.ts new file mode 100644 index 0000000000..7519865a4b --- /dev/null +++ b/src/application/binary/SessionRuntimeAvailability.ts @@ -0,0 +1,169 @@ +import type { AnalysisError } from "../../domain/analysisErrorBase.js"; +import type { Result } from "../../domain/result.js"; +import type { CapabilityDescriptor } from "../AnalysisProvider.js"; + +/** Failures of the provider adapter or process rather than of one request. */ +const PROVIDER_WIDE_FAILURES: ReadonlySet = new Set([ + "ProviderAdapterError", + "HopperProcessError", + "HopperStartError", +]); + +interface Failure { + readonly dispatch: number; + readonly reason: string; +} + +interface ProviderFailure extends Failure { + readonly operation: string; +} + +/** Newest failure and newest success, both by dispatch, for one owner. */ +interface Outcomes { + failure: F | undefined; + succeeded: number; +} + +type RoutedOperation = Pick< + CapabilityDescriptor, + "operation" | "provider" | "available" +>; + +/** Reason recorded for an operation's own refusal or provider-wide failure. */ +export const runtimeFailureReason = ( + failure: AnalysisError, +): string | undefined => { + if (failure._tag === "AnalysisCapabilityUnavailableError") + return failure.message; + return PROVIDER_WIDE_FAILURES.has(failure._tag) + ? `${failure._tag}: ${failure.message}` + : undefined; +}; + +/** Explain unavailability inferred for an operation no request failed for. */ +export const inferredUnavailabilityReason = ( + providerId: string, + failure: Pick, +): string => + `Inferred from the ${failure.operation} failure (${failure.reason}). Retry after repairing the provider; a later successful ${providerId} request clears this.`; + +/** + * Own the runtime unavailability overlaid on the active route's declared + * capabilities. + * + * An operation is unavailable while its newest refusal or provider-wide + * failure was dispatched after its newest success. A declared-available + * operation is also unavailable, as inferred, while its provider's newest + * provider-wide failure was dispatched after any success from that provider. + * One success therefore returns inferred siblings to their declared + * availability without proving each is healthy, and keeps failures observed + * for other operations. + * + * Only the newest dispatch of each outcome matters, so the projection does not + * depend on the order in which overlapping replies complete. Dispatch is the + * latest order this session owns; it does not establish how a provider + * interleaves overlapping requests. + */ +export class SessionRuntimeAvailability { + readonly #operations = new Map>(); + readonly #providers = new Map>(); + #projected: ReadonlyMap = new Map(); + #dispatched = 0; + + /** Current runtime unavailability keyed by operation. */ + get entries(): ReadonlyMap { + return this.#projected; + } + + /** Order one request immediately before it reaches the provider client. */ + dispatch(): number { + this.#dispatched += 1; + return this.#dispatched; + } + + /** + * Record one dispatched request's outcome against the route it ran on and + * report whether visible availability changed. + */ + observe( + dispatch: number, + capability: RoutedOperation, + route: Iterable, + result: Result, + ): boolean { + const operation = outcomes(this.#operations, capability.operation); + const provider = outcomes(this.#providers, capability.provider.id); + if (result.ok) { + operation.succeeded = Math.max(operation.succeeded, dispatch); + provider.succeeded = Math.max(provider.succeeded, dispatch); + // A success can only remove unavailability. + if (this.#projected.size === 0) return false; + } else { + const reason = runtimeFailureReason(result.error); + if (reason === undefined) return false; + operation.failure = newer(operation.failure, { dispatch, reason }); + if (PROVIDER_WIDE_FAILURES.has(result.error._tag)) + provider.failure = newer(provider.failure, { + dispatch, + reason, + operation: capability.operation, + }); + } + return this.#project(route); + } + + /** Forget runtime outcomes when the active provider client is replaced. */ + clear(): boolean { + this.#operations.clear(); + this.#providers.clear(); + if (this.#projected.size === 0) return false; + this.#projected = new Map(); + return true; + } + + #project(route: Iterable): boolean { + const projected = new Map(); + for (const descriptor of route) { + const reason = this.#unavailableReason(descriptor); + if (reason !== undefined) projected.set(descriptor.operation, { reason }); + } + const changed = + projected.size !== this.#projected.size || + [...projected].some( + ([operation, { reason }]) => + this.#projected.get(operation)?.reason !== reason, + ); + this.#projected = projected; + return changed; + } + + #unavailableReason(descriptor: RoutedOperation): string | undefined { + const operation = this.#operations.get(descriptor.operation); + if ( + operation?.failure !== undefined && + operation.failure.dispatch > operation.succeeded + ) + return operation.failure.reason; + // A declared refusal already explains its operation and is never dispatched. + if (!descriptor.available) return undefined; + const provider = this.#providers.get(descriptor.provider.id); + return provider?.failure !== undefined && + provider.failure.dispatch > provider.succeeded + ? inferredUnavailabilityReason(descriptor.provider.id, provider.failure) + : undefined; + } +} + +const outcomes = ( + owners: Map>, + key: string, +): Outcomes => { + const existing = owners.get(key); + if (existing !== undefined) return existing; + const created: Outcomes = { failure: undefined, succeeded: 0 }; + owners.set(key, created); + return created; +}; + +const newer = (current: F | undefined, next: F): F => + current !== undefined && current.dispatch > next.dispatch ? current : next; diff --git a/src/application/decompileDirectory/decompileAll.test.ts b/src/application/decompileDirectory/decompileAll.test.ts index 17470a61d3..f7d6de44e2 100644 --- a/src/application/decompileDirectory/decompileAll.test.ts +++ b/src/application/decompileDirectory/decompileAll.test.ts @@ -138,6 +138,7 @@ const harness = (options?: { Record >; readonly info?: "ok" | "unavailable" | "error"; + readonly listProfile?: typeof profile; readonly subject?: boolean; readonly mismatch?: string; readonly unprofiled?: string; @@ -170,7 +171,7 @@ const harness = (options?: { })), provider, { - analysisProfile: profile, + analysisProfile: options?.listProfile ?? profile, ...(options?.subject === false ? {} : { subject }), }, ), @@ -614,6 +615,37 @@ describe("decompile_all resume", () => { await readFile(join(directory(), "functions/0x1000.c"), "utf8"), ).toContain(sentinel); }); + + it("names the resume identity that differs instead of directing callers to the input schema", async () => { + expect((await run(harness().tools, directory())).ok).toBe(true); + + const resumed = await run( + harness({ listProfile: otherProfile }).tools, + directory(), + { resume: true }, + ); + + expect(resumed.ok).toBe(false); + if (resumed.ok) return; + expect(resumed.error).toBeInstanceOf(AnalysisInputError); + if (!(resumed.error instanceof AnalysisInputError)) return; + expect(resumed.error.userMessage).toBe( + "The decompile journal does not match the current analysis profile digest.", + ); + expect(resumed.error.conformsToInputContract).toBe(true); + expect(resumed.error.remediationAction).toContain("new output directory"); + expect(resumed.error.issues).toEqual([ + expect.objectContaining({ + path: ["output_directory"], + expected: expect.objectContaining({ + analysis_profile_digest: profile.digest, + }), + observed: expect.objectContaining({ + analysis_profile_digest: otherProfile.digest, + }), + }), + ]); + }); }); describe("decompile_all filters and layout", () => { diff --git a/src/application/decompileDirectory/exportTree.ts b/src/application/decompileDirectory/exportTree.ts index 8a79e19082..e28229bd36 100644 --- a/src/application/decompileDirectory/exportTree.ts +++ b/src/application/decompileDirectory/exportTree.ts @@ -29,9 +29,9 @@ import { type DecompileJournalHeader, } from "../../contracts/decompileDirectory.js"; import { + AnalysisInputError, AnalysisOutputError, AnalysisResourceConstraintError, - type AnalysisInputError, } from "../../domain/analysisErrorCore.js"; import { AnalysisError } from "../../domain/analysisErrorBase.js"; import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js"; @@ -663,20 +663,18 @@ const readJournal = async ( if (!parsed.ok) return parsed; const unknown = unknownIdentity(parsed.value.header, identity); if (unknown !== undefined) return err(unknown); - return sameIdentity(parsed.value.header, identity) - ? ok({ - records: parsed.value.records, - written: parsed.value.written, - paths: parsed.value.paths, - committedBytes: parsed.value.committedBytes, - journalBytes: parsed.value.journalBytes, - }) - : err( - decompileInputError( - "output_directory", - "The decompile journal does not match this target, provider, analysis profile, or layout.", - ), - ); + const mismatches = identityMismatches(parsed.value.header, identity); + if (mismatches.length > 0) + return err( + journalIdentityMismatch(parsed.value.header, identity, mismatches), + ); + return ok({ + records: parsed.value.records, + written: parsed.value.written, + paths: parsed.value.paths, + committedBytes: parsed.value.committedBytes, + journalBytes: parsed.value.journalBytes, + }); }; const parseJournal = ( @@ -747,16 +745,62 @@ const unknownIdentity = ( return undefined; }; -const sameIdentity = ( +/** Identity components in the order presented to the caller. */ +const identityMismatches = ( header: DecompileJournalHeader, identity: ExportIdentity, -): boolean => - header.target_sha256 === identity.targetSha256 && - header.analysis_profile_digest === identity.analysisProfileDigest && - header.layout === identity.layout && - header.provider.id === identity.provider.id && - header.provider.name === identity.provider.name && - header.provider.version === identity.provider.version; +): readonly string[] => [ + ...(header.target_sha256 === identity.targetSha256 ? [] : ["target SHA-256"]), + ...(header.analysis_profile_digest === identity.analysisProfileDigest + ? [] + : ["analysis profile digest"]), + ...(header.layout === identity.layout ? [] : ["layout"]), + ...(header.provider.id === identity.provider.id ? [] : ["provider id"]), + ...(header.provider.name === identity.provider.name ? [] : ["provider name"]), + ...(header.provider.version === identity.provider.version + ? [] + : ["provider version"]), +]; + +/** + * Report the concrete resume identity that differs. The request is valid, so + * generic input-schema recovery would send the caller to the wrong boundary. + */ +const journalIdentityMismatch = ( + header: DecompileJournalHeader, + identity: ExportIdentity, + mismatches: readonly string[], +): AnalysisInputError => { + const message = `The decompile journal does not match the current ${mismatches.join(", ")}.`; + return new AnalysisInputError( + "decompile_all", + { + userMessage: message, + conformsToInputContract: true, + remediationAction: + "Resume with the target, provider, analysis profile, and layout that created this directory, or choose a new output directory. The journal and existing files were left unchanged.", + }, + [ + { + path: ["output_directory"], + reason: "invalid_value", + message, + expected: { + target_sha256: header.target_sha256, + analysis_profile_digest: header.analysis_profile_digest, + layout: header.layout, + provider: header.provider, + }, + observed: { + target_sha256: identity.targetSha256, + analysis_profile_digest: identity.analysisProfileDigest, + layout: identity.layout, + provider: identity.provider, + }, + }, + ], + ); +}; const incompatibleJournal = (): AnalysisInputError => decompileInputError( diff --git a/src/application/electron/ElectronFuseService.test.ts b/src/application/electron/ElectronFuseService.test.ts index 1a501fee62..16c693f509 100644 --- a/src/application/electron/ElectronFuseService.test.ts +++ b/src/application/electron/ElectronFuseService.test.ts @@ -30,11 +30,12 @@ const report: ElectronFuseReport = { it("retains completed fuse Evidence when the provider completes after abort", async () => { const controller = new AbortController(); + const reason = { requested: "stop after fuse inspection" }; const result = await new ElectronFuseService({ identity, close: async () => {}, inspect: async () => { - controller.abort(); + controller.abort(reason); return ok(report); }, }).inspect( @@ -44,8 +45,21 @@ it("retains completed fuse Evidence when the provider completes after abort", as expect(result.ok).toBe(false); if (!result.ok) { expect(result.error._tag).toBe("AnalysisCancelledError"); + expect(result.error.cause).toBe(reason); expect(parseEvidence(result.error.partialObservation)).toMatchObject({ operation: "inspect_electron_fuses", + provider: identity, + subject: { + local_path: report.artifact.resolved_path, + digest: { sha256: report.artifact.sha256 }, + format: report.artifact.format, + }, + parameters: { path: report.artifact.selected_path }, + confidence: "observed", + limitations: expect.arrayContaining(report.limitations), + locations: [ + { kind: "artifact-path", path: report.artifact.selected_path }, + ], normalized_result: report, }); } diff --git a/src/application/investigation/InspectEvidenceBundle.ts b/src/application/investigation/InspectEvidenceBundle.ts index 755440f727..8831becc2f 100644 --- a/src/application/investigation/InspectEvidenceBundle.ts +++ b/src/application/investigation/InspectEvidenceBundle.ts @@ -2,19 +2,26 @@ import type { EvidenceBundle } from "../../domain/evidenceBundle.js"; import { AnalysisInputError } from "../../domain/analysisErrorCore.js"; import { err, ok, type Result } from "../../domain/result.js"; import { + selectEvidenceRecords, summarizeEvidenceBundle, type EvidenceBundleSummary, type EvidenceRecordFilters, + type EvidenceRecordSelection, } from "../../domain/evidenceRecordSummary.js"; -/** Select complete bundle delivery or discovery metadata without analysis payloads. */ +/** Select complete bundle delivery, discovery metadata, or a page of complete selected records. */ export const inspectEvidenceBundle = ( bundle: EvidenceBundle, options: { - readonly detail: "complete" | "summary"; + readonly detail: "complete" | "summary" | "records"; readonly filters?: EvidenceRecordFilters | undefined; + readonly offset?: number | undefined; + readonly limit?: number | undefined; }, -): Result => { +): Result< + EvidenceBundle | EvidenceBundleSummary | EvidenceRecordSelection, + AnalysisInputError +> => { if (options.detail === "complete" && options.filters !== undefined) return err( new AnalysisInputError("get_evidence_bundle", undefined, [ @@ -22,11 +29,36 @@ export const inspectEvidenceBundle = ( path: ["filters"], reason: "invalid_value", message: - "Record filters require detail: summary; complete returns the canonical bundle.", - expected: "detail: summary", + "Record filters require detail: summary or records; complete returns the canonical bundle.", + expected: "detail: summary or records", }, ]), ); + if (options.detail !== "records") { + const paging = (["offset", "limit"] as const).filter( + (field) => options[field] !== undefined, + ); + if (paging.length > 0) + return err( + new AnalysisInputError( + "get_evidence_bundle", + undefined, + paging.map((field) => ({ + path: [field], + reason: "invalid_value" as const, + message: `${field} pages complete records and requires detail: records.`, + expected: "detail: records", + })), + ), + ); + } + if (options.detail === "records") + return ok( + selectEvidenceRecords(bundle, options.filters, { + offset: options.offset, + limit: options.limit, + }), + ); return ok( options.detail === "summary" ? summarizeEvidenceBundle(bundle, options.filters) diff --git a/src/application/javascript/ElectronObservationService.test.ts b/src/application/javascript/ElectronObservationService.test.ts index 01e9e8c0e2..5393957b0c 100644 --- a/src/application/javascript/ElectronObservationService.test.ts +++ b/src/application/javascript/ElectronObservationService.test.ts @@ -150,6 +150,7 @@ it("returns a typed output error for malformed Electron target output after abor it("retains completed target facts when cancellation arrives before Evidence publication", async () => { const controller = new AbortController(); + const reason = { requested: "stop after listing targets" }; const targets = electronTargetListSchema.parse({ browser: inspection.browser, targets: [], @@ -159,7 +160,7 @@ it("retains completed target facts when cancellation arrives before Evidence pub const provider: ElectronObservationPort = { identity: () => identity, listTargets: async () => { - controller.abort(); + controller.abort(reason); return ok(targets); }, inspectPage: async () => @@ -173,8 +174,22 @@ it("retains completed target facts when cancellation arrives before Evidence pub expect(result.ok).toBe(false); if (!result.ok) { expect(result.error).toBeInstanceOf(AnalysisCancelledError); + expect(result.error.cause).toBe(reason); expect(parseEvidence(result.error.partialObservation)).toMatchObject({ operation: "list_electron_targets", + provider: identity, + parameters: { cdp_endpoint: input.cdp_endpoint }, + confidence: "observed", + authority: "external-service", + environment: { + id: `${inspection.browser.product}@${inspection.browser.revision}`, + platform: "unknown", + architecture: "unknown", + isolation: "none", + }, + limitations: expect.arrayContaining([ + "Browser host platform and architecture were not observed.", + ]), normalized_result: targets, }); } diff --git a/src/application/javascript/JavaScriptAnalysisControl.test.ts b/src/application/javascript/JavaScriptAnalysisControl.test.ts index a16892ef9a..0dc60c78a0 100644 --- a/src/application/javascript/JavaScriptAnalysisControl.test.ts +++ b/src/application/javascript/JavaScriptAnalysisControl.test.ts @@ -1,5 +1,6 @@ import { setImmediate } from "node:timers/promises"; import { describe, expect, it } from "vitest"; +import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js"; import { completeJavaScriptAnalysisSteps } from "./JavaScriptAnalysisControl.js"; describe("JavaScript cooperative computation lifecycle", () => { @@ -15,9 +16,19 @@ describe("JavaScript cooperative computation lifecycle", () => { } } const cancellation = setImmediate().then(() => controller.abort(reason)); - await expect( - completeJavaScriptAnalysisSteps(work(), controller.signal), - ).rejects.toMatchObject({ reason: "cancelled", cause: reason }); + const failure = await completeJavaScriptAnalysisSteps( + work(), + controller.signal, + ).then( + () => { + throw new Error("Expected cancellation"); + }, + (cause: unknown) => cause, + ); + expect(failure).toBeInstanceOf(ArtifactReaderFailure); + if (!(failure instanceof ArtifactReaderFailure)) throw failure; + expect(failure.reason).toBe("cancelled"); + expect(failure.cause).toBe(reason); await cancellation; expect(released).toBe(true); }); @@ -32,9 +43,19 @@ describe("JavaScript cooperative computation lifecycle", () => { yield; return 42; } - await expect( - completeJavaScriptAnalysisSteps(work(), controller.signal), - ).rejects.toMatchObject({ reason: "cancelled", cause: reason }); + const failure = await completeJavaScriptAnalysisSteps( + work(), + controller.signal, + ).then( + () => { + throw new Error("Expected cancellation"); + }, + (cause: unknown) => cause, + ); + expect(failure).toBeInstanceOf(ArtifactReaderFailure); + if (!(failure instanceof ArtifactReaderFailure)) throw failure; + expect(failure.reason).toBe("cancelled"); + expect(failure.cause).toBe(reason); expect(started).toBe(false); expect(await completeJavaScriptAnalysisSteps(work())).toBe(42); }); diff --git a/src/application/javascript/JavaScriptRecoveryService.test.ts b/src/application/javascript/JavaScriptRecoveryService.test.ts index fbd6ae9a60..1977b6031b 100644 --- a/src/application/javascript/JavaScriptRecoveryService.test.ts +++ b/src/application/javascript/JavaScriptRecoveryService.test.ts @@ -1,6 +1,7 @@ import { describe, expect, it } from "vitest"; import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js"; import { err, ok } from "../../domain/result.js"; +import { parseEvidence } from "../../domain/evidence.js"; import type { JavaScriptRecoveryResult } from "../../domain/javascript/javascriptRecovery.js"; import { createAnalysisExecution } from "../AnalysisProvider.js"; import { JavaScriptRecoveryService } from "./JavaScriptRecoveryService.js"; @@ -179,7 +180,17 @@ it("returns cancellation with the validated Evidence when the caller aborts afte if (result.ok) throw new Error("expected cancellation"); expect(result.error._tag).toBe("AnalysisCancelledError"); expect(result.error.cause).toBe(reason); - expect(result.error.partialObservation).toMatchObject({ + expect(parseEvidence(result.error.partialObservation)).toMatchObject({ operation: "recover_javascript_sources", + provider: providerIdentity, + parameters: input, + subject: { + local_path: artifact.path, + digest: { sha256: artifact.sha256 }, + format: "javascript", + }, + confidence: "derived", + authority: "shipped-artifact", + normalized_result: recoveryResult(), }); }); diff --git a/src/application/managed/ManagedReconstructionService.test.ts b/src/application/managed/ManagedReconstructionService.test.ts index cf489058d4..81a3786238 100644 --- a/src/application/managed/ManagedReconstructionService.test.ts +++ b/src/application/managed/ManagedReconstructionService.test.ts @@ -41,36 +41,6 @@ it("managed reconstruction reports UTF-8 byte totals while preserving UTF-16 tex }); }); -it("managed reconstruction rejects text hash drift", () => { - const rejected = resolveManagedReconstructionImport({ - ...exampleInput(), - methods: [ - { - ...exampleMethod(), - reconstruction: { - ...exampleMethod().reconstruction, - text_sha256: "0".repeat(64), - }, - }, - ], - }); - expect(rejected).toMatchObject({ - ok: false, - error: { - _tag: "AnalysisInputError", - operation: "import_managed_reconstruction", - issues: [ - { - path: ["methods", 0, "reconstruction", "text_sha256"], - reason: "invalid_value", - message: expect.stringContaining("Reconstruction text hash mismatch"), - }, - ], - }, - }); - if (!rejected.ok) expect(rejected.error).toBeInstanceOf(AnalysisInputError); -}); - it("managed reconstruction rejects stale method locks", () => { const staleInput = { ...exampleInput(), @@ -234,13 +204,15 @@ describe("managed reconstruction input diagnostics", () => { }); expect(result.ok).toBe(false); if (result.ok) return; + expect(result.error).toBeInstanceOf(AnalysisInputError); expect(result.error).toMatchObject({ _tag: "AnalysisInputError", + operation: "import_managed_reconstruction", issues: [ { path: ["methods", 0, "reconstruction", "text_sha256"], reason: "invalid_value", - message: expect.stringMatching(/text hash mismatch/u), + message: expect.stringContaining("Reconstruction text hash mismatch"), }, ], }); diff --git a/src/application/native/NativeFunctionMatchingService.ts b/src/application/native/NativeFunctionMatchingService.ts index d7544ba2b8..02e75920cf 100644 --- a/src/application/native/NativeFunctionMatchingService.ts +++ b/src/application/native/NativeFunctionMatchingService.ts @@ -20,6 +20,7 @@ import { resolveNativeMatchingSources } from "./NativeMatchingSources.js"; import type { ExecutionOptions } from "../AnalysisProvider.js"; import type { NativeMatchingAllowance } from "../../config/nativeMatchingAllowance.js"; import { FUNCTION_COMPARISON_PROVIDER } from "../../domain/comparisonEvidenceIdentity.js"; +import { NATIVE_FUNCTION_MATCHING_PREDICATE } from "../../domain/native/functionMatching.js"; const OPERATION = "match_native_functions"; @@ -144,7 +145,7 @@ const matchWithAdmission = async ( await checkpoint(); const evidence = createEvidence(undefined, FUNCTION_COMPARISON_PROVIDER, { operation: OPERATION, - predicateType: "rea.native-function-matching", + predicateType: NATIVE_FUNCTION_MATCHING_PREDICATE, confidence: "derived", authority: "analyst-inference", parameters: { diff --git a/src/application/native/NativeFunctionMatchingViews.test.ts b/src/application/native/NativeFunctionMatchingViews.test.ts index c577b9521e..b80dc719ad 100644 --- a/src/application/native/NativeFunctionMatchingViews.test.ts +++ b/src/application/native/NativeFunctionMatchingViews.test.ts @@ -119,6 +119,7 @@ it("rebuilds native row selection when a caller's sealed flag covers only a shal ); Object.freeze(report); const parent = { + predicateType: matched.value.evidence.predicate_type, evidenceId: matched.value.evidence.evidence_id, operation: matched.value.evidence.operation, normalizedResult: report, diff --git a/src/application/native/NativeMatchingSources.ts b/src/application/native/NativeMatchingSources.ts index 3ad34d8bbd..e8950fd914 100644 --- a/src/application/native/NativeMatchingSources.ts +++ b/src/application/native/NativeMatchingSources.ts @@ -1,7 +1,11 @@ import type { MatchNativeFunctionsInput } from "../../contracts/native/functionMatchingContracts.js"; import { digestCanonicalValue } from "../../domain/canonicalDigest.js"; import { AnalysisInputError } from "../../domain/analysisErrorCore.js"; -import { isImmutableEvidence, type Evidence } from "../../domain/evidence.js"; +import { + ANALYSIS_EVIDENCE_PREDICATE, + isImmutableEvidence, + type Evidence, +} from "../../domain/evidence.js"; import { readEvidence } from "../../domain/evidenceValidation.js"; import { retainedEvidenceReferenceSchema } from "../../domain/evidenceReference.js"; import { @@ -82,7 +86,7 @@ export const resolveNativeMatchingSources = async ( ): Promise => { const evidence = await resolve(raw, path, borrowable); if ( - evidence.predicate_type !== "rea.analysis" || + evidence.predicate_type !== ANALYSIS_EVIDENCE_PREDICATE || !operations.includes(evidence.operation) || evidence.subject === null ) { diff --git a/src/application/native/NativeUiActionTrace.ts b/src/application/native/NativeUiActionTrace.ts index 50e3d8e9f0..edd111ee5b 100644 --- a/src/application/native/NativeUiActionTrace.ts +++ b/src/application/native/NativeUiActionTrace.ts @@ -1,5 +1,6 @@ import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js"; import { permitsCapabilityFallback } from "../CapabilityFallback.js"; +import { stopsTraversal } from "../CallRelationshipTraversal.js"; import { z } from "zod"; import type { AnalysisOperationPort } from "../AnalysisProvider.js"; @@ -459,11 +460,7 @@ const expandNativeCalls = async (input: { ); examined += 1; sourceLimitations.push(...execution.limitations); - if ( - !execution.outcome.ok && - (execution.outcome.error._tag === "AnalysisCancelledError" || - execution.outcome.error.cleanupIncomplete) - ) + if (!execution.outcome.ok && stopsTraversal(execution.outcome.error)) return { error: execution.outcome.error }; const unknowns = execution.unknowns; for (const [index, unknown] of unknowns.entries()) { diff --git a/src/application/native/NativeValueTrace.fixture.ts b/src/application/native/NativeValueTrace.fixture.ts index 725a01fad2..a11b11897f 100644 --- a/src/application/native/NativeValueTrace.fixture.ts +++ b/src/application/native/NativeValueTrace.fixture.ts @@ -5,7 +5,8 @@ import { import { ghidraFunctionDossier } from "../../domain/ghidraValues.fixture.js"; import { functionDossierSchema } from "../../domain/native/functionDossier.js"; import { jsonValueSchema } from "../../domain/jsonValue.js"; -import { ok } from "../../domain/result.js"; +import type { AnalysisError } from "../../domain/analysisErrorBase.js"; +import { err, ok } from "../../domain/result.js"; /** Simulated provider seam for value traces; not a real decompiler observation. */ export const valueTraceSubject = { @@ -215,3 +216,42 @@ export const valueTraceDiamond = (options: GraphOptions = {}) => }, options, ); + +/** + * The diamond's reads with max_depth 1 when nothing fails. 0x2000's call to + * 0x3000 is resolved but not followed, because 0x3000 is already queued. + */ +export const VALUE_TRACE_DIAMOND_READS = [ + "analyze_function:0x1000", + "resolve_native_call_targets:0x1004", + "resolve_native_call_targets:0x1008", + "analyze_function:0x2000", + "resolve_native_call_targets:0x2004", + "analyze_function:0x3000", +] as const; +export type ValueTraceDiamondRead = (typeof VALUE_TRACE_DIAMOND_READS)[number]; + +/** Fail each listed `operation:selector` trace read and record every trace read. */ +export const failingValueTraceReads = ( + port: AnalysisOperationPort, + failures: Partial>, +) => { + const reads: string[] = []; + const failed = new Map(Object.entries(failures)); + const failing: AnalysisOperationPort = { + execute: async (operation, parameters, options) => { + if ( + operation !== "analyze_function" && + operation !== "resolve_native_call_targets" + ) + return ok(createAnalysisExecution(null, valueTraceProvider)); + const read = `${operation}:${String(parameters.procedure ?? parameters.address)}`; + reads.push(read); + const failure = failed.get(read); + return failure === undefined + ? port.execute(operation, parameters, options) + : err(failure); + }, + }; + return { port: failing, reads }; +}; diff --git a/src/application/native/NativeValueTrace.ts b/src/application/native/NativeValueTrace.ts index 39e06465ff..de1e2efb68 100644 --- a/src/application/native/NativeValueTrace.ts +++ b/src/application/native/NativeValueTrace.ts @@ -11,13 +11,22 @@ import { } from "../../domain/analysisErrorCore.js"; import { err, ok } from "../../domain/result.js"; import type { AnalysisOperationPort } from "../AnalysisProvider.js"; -import type { EnhancedResult } from "../EnhancedToolTypes.js"; +import type { AnalysisError } from "../../domain/analysisErrorBase.js"; +import { stopsTraversal } from "../CallRelationshipTraversal.js"; +import { + failedWithPartialResult, + type EnhancedResult, +} from "../EnhancedToolTypes.js"; import { renderSourceLimitations, type SourceLimitation, } from "../SourceLimitations.js"; import { createValueTraceBudget } from "./NativeValueTraceBudget.js"; +/** The located unknown where unconfirmed provider cleanup stopped a trace. */ +export const CLEANUP_STOPPED_VALUE_TRACE = + "Trace stopped because the provider did not confirm cleanup of resources it owns"; + /** Compose bounded high-p-code def-use graphs across exactly referenced call destinations. */ export const traceNativeValues = async ( analysis: AnalysisOperationPort, @@ -52,6 +61,27 @@ export const traceNativeValues = async ( let targetSha256: string | undefined; let decompilations = 0; let callSites = 0; + // A provider that cannot confirm cleanup of resources it owns stops the + // trace; its own failure becomes the outcome, holding the admitted work. + let stopped: AnalysisError | undefined; + /** Record a failed read after the validated seed; true when it ends the trace. */ + const endsTrace = ( + error: AnalysisError, + procedure: string, + address: string | null, + ): boolean => { + if (!stopsTraversal(error)) { + unknowns.push({ procedure, address, reason: error.message }); + return false; + } + if (error.cleanupIncomplete) + budget.narrow("cleanup_incomplete", { + procedure, + address, + reason: CLEANUP_STOPPED_VALUE_TRACE, + }); + return true; + }; while (pending.length > 0) { if (signal?.aborted) return err( @@ -91,14 +121,12 @@ export const traceNativeValues = async ( decompilations++; if (!execution.ok) { // A seed failure has no trace to qualify. Preserve the provider's typed - // refusal; failures after a validated seed remain partial observations. - if (next.depth === 0 || execution.error._tag === "AnalysisCancelledError") - return err(execution.error); - unknowns.push({ - procedure: next.procedure, - address: null, - reason: execution.error.message, - }); + // refusal; later failures are unknowns unless they end the trace. + if (next.depth === 0) return err(execution.error); + if (endsTrace(execution.error, next.procedure, null)) { + stopped = execution.error; + break; + } continue; } const evidence = createAnalysisExecutionEvidence( @@ -252,13 +280,10 @@ export const traceNativeValues = async ( signal === undefined ? {} : { signal }, ); if (!resolved.ok) { - if (resolved.error._tag === "AnalysisCancelledError") - return err(resolved.error); - unknowns.push({ - procedure: address, - address: operation.address, - reason: resolved.error.message, - }); + if (endsTrace(resolved.error, address, operation.address)) { + stopped = resolved.error; + break; + } continue; } const targets = nativeCallTargetsSchema.safeParse(resolved.value.result); @@ -330,8 +355,10 @@ export const traceNativeValues = async ( } } // Remaining work is attributed to the rejecting budget recorded above. - if (budget.exhausted()) break; + if (stopped !== undefined || budget.exhausted()) break; } + // Cancellation leaves no outcome to qualify; unconfirmed cleanup keeps one. + if (stopped !== undefined && !stopped.cleanupIncomplete) return err(stopped); if (targetSha256 === undefined) return err( new AnalysisOutputError( @@ -456,7 +483,12 @@ export const traceNativeValues = async ( ...renderSourceLimitations(sourceLimitations), ], }); - if (Buffer.byteLength(JSON.stringify(result)) > 32 * 1024 * 1024) + const oversized = + Buffer.byteLength(JSON.stringify(result)) > 32 * 1024 * 1024; + // The stopping failure stays primary over output size and a later abort. + if (stopped !== undefined) + return oversized ? err(stopped) : failedWithPartialResult(stopped, result); + if (oversized) return err( new AnalysisOutputError( "trace_native_values", diff --git a/src/application/native/NativeValueTraceStops.test.ts b/src/application/native/NativeValueTraceStops.test.ts new file mode 100644 index 0000000000..fae841232e --- /dev/null +++ b/src/application/native/NativeValueTraceStops.test.ts @@ -0,0 +1,144 @@ +import { describe, expect, it } from "vitest"; +import type { AnalysisOperationPort } from "../AnalysisProvider.js"; +import { + AnalysisCancelledError, + AnalysisCapabilityUnavailableError, +} from "../../domain/analysisErrorCore.js"; +import { nativeValueTraceSchema } from "../../domain/native/nativeValueTrace.js"; +import { + CLEANUP_STOPPED_VALUE_TRACE, + traceNativeValues, +} from "./NativeValueTrace.js"; +import { + failingValueTraceReads, + valueTraceDiamond, + valueTraceProvider as provider, + VALUE_TRACE_DIAMOND_READS, + type ValueTraceDiamondRead, +} from "./NativeValueTrace.fixture.js"; + +describe("native value trace child stops", () => { + const cleanup = { + cleanup: { reason: "cleanup proof refused", resources: ["owned worker"] }, + }; + const operationOf = (read: ValueTraceDiamondRead) => + read.startsWith("analyze_function") + ? "analyze_function" + : "resolve_native_call_targets"; + const childReads = [ + "resolve_native_call_targets:0x1004", + "analyze_function:0x2000", + ] as const; + const readsThrough = (read: ValueTraceDiamondRead) => + VALUE_TRACE_DIAMOND_READS.slice( + 0, + VALUE_TRACE_DIAMOND_READS.indexOf(read) + 1, + ); + + it.each(childReads)( + "returns a cancelled %s read without a partial trace", + async (read) => { + const cancelled = new AnalysisCancelledError(operationOf(read)); + const { port, reads } = failingValueTraceReads(valueTraceDiamond().port, { + [read]: cancelled, + }); + + const result = await traceNativeValues(port, { + procedure: "0x1000", + max_depth: 1, + }); + + expect(result).toStrictEqual({ ok: false, error: cancelled }); + expect(reads).toEqual(readsThrough(read)); + }, + ); + + it.each( + childReads.flatMap((read) => + (["caller abort", "cancelled read"] as const).map((race) => ({ + read, + race, + })), + ), + )( + "keeps unconfirmed cleanup primary over a $race at $read", + async ({ read, race }) => { + const controller = new AbortController(); + const failure = + race === "cancelled read" + ? new AnalysisCancelledError(operationOf(read), cleanup) + : new AnalysisCapabilityUnavailableError( + provider.id, + operationOf(read), + "controlled child failure", + cleanup, + ); + const { port, reads } = failingValueTraceReads(valueTraceDiamond().port, { + [read]: failure, + }); + // The caller aborts while the failing read settles. + const aborting: AnalysisOperationPort = { + execute: async (operation, parameters, options) => { + const selector = `${operation}:${String(parameters.procedure ?? parameters.address)}`; + if (selector === read) controller.abort("caller stopped the trace"); + return port.execute(operation, parameters, options); + }, + }; + + const result = await traceNativeValues( + aborting, + { procedure: "0x1000", max_depth: 1 }, + controller.signal, + ); + + expect(reads).toEqual(readsThrough(read)); + if (result.ok || !("partialResult" in result)) + throw new Error("Expected a stopped trace with its admitted result"); + expect(result.error).toBe(failure); + expect(nativeValueTraceSchema.parse(result.partialResult)).toMatchObject({ + procedures: [{ address: "0x1000" }], + traversal: { status: "truncated", reasons: ["cleanup_incomplete"] }, + truncated: true, + }); + }, + ); + + it("records the stop as a located unknown in the admitted trace", async () => { + const failure = new AnalysisCapabilityUnavailableError( + provider.id, + "resolve_native_call_targets", + "controlled child failure", + cleanup, + ); + const { port } = failingValueTraceReads(valueTraceDiamond().port, { + "resolve_native_call_targets:0x1008": failure, + }); + + const result = await traceNativeValues(port, { + procedure: "0x1000", + max_depth: 1, + }); + + if (result.ok || !("partialResult" in result)) + throw new Error("Expected a stopped trace with its admitted result"); + const admitted = nativeValueTraceSchema.parse(result.partialResult); + // 0x2000 was queued from 0x1004 but never analyzed; the stop, not a depth + // limit, accounts for it, and the admitted call edge to it remains. + expect(admitted.traversal).toEqual({ + status: "truncated", + reasons: ["cleanup_incomplete"], + }); + expect(admitted.unknowns.filter(({ address }) => address !== null)).toEqual( + [ + { + procedure: "0x1000", + address: "0x1008", + reason: CLEANUP_STOPPED_VALUE_TRACE, + }, + ], + ); + expect(admitted.edges).toContainEqual( + expect.objectContaining({ kind: "call", target: "0x2000" }), + ); + }); +}); diff --git a/src/application/ownedProcessAvailability.ts b/src/application/ownedProcessAvailability.ts index 918d777714..9c60cea3e9 100644 --- a/src/application/ownedProcessAvailability.ts +++ b/src/application/ownedProcessAvailability.ts @@ -73,6 +73,7 @@ export const ownedProcessPrerequisiteAvailability = ( status: "unavailable", code: "runtime_unavailable", reason: `${refusal.reason}. ${REMEDIATION}`, + remediation: REMEDIATION, diagnostics: { stage: "ownership_preparation", prerequisite_provider_id: refusal.providerId, @@ -85,6 +86,7 @@ export const ownedProcessPrerequisiteAvailability = ( status: "unavailable", code: "runtime_unavailable", reason: `${darwin.message}. ${DARWIN_REMEDIATION}`, + remediation: DARWIN_REMEDIATION, diagnostics: { stage: "ownership_preparation", ...(darwin.compilerOutcome === undefined diff --git a/src/artifacts/ArtifactPaths.test.ts b/src/artifacts/ArtifactPaths.test.ts index 26c4814d01..6a203ee234 100644 --- a/src/artifacts/ArtifactPaths.test.ts +++ b/src/artifacts/ArtifactPaths.test.ts @@ -181,7 +181,7 @@ it("retains both source spellings and the native failure in a destination collis expect(collision?.cause).toBe(cause); }); -it.each([["app.js"], ["app.js", "App.js"], ["other.js"]])( +it.each([["app.js", "App.js"], ["other.js"]])( "does not invent a destination alias from listing %j", (...names) => { expect(destinationCaseCollision("res/app.js", names)).toBeUndefined(); diff --git a/src/artifacts/BundleResourceArtifactClient.test.ts b/src/artifacts/BundleResourceArtifactClient.test.ts deleted file mode 100644 index 90fd7ef12a..0000000000 --- a/src/artifacts/BundleResourceArtifactClient.test.ts +++ /dev/null @@ -1,60 +0,0 @@ -import { mkdtemp, mkdir, rm, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { build } from "plist"; -import { afterEach, expect, it } from "vitest"; - -import { ArtifactProvider } from "./ArtifactProvider.js"; -import type { ResolvedAppBundleResourceRoot } from "../application/AppBundleExecutable.js"; - -const temporaryDirectories: string[] = []; - -afterEach(async () => { - await Promise.all( - temporaryDirectories - .splice(0) - .map((path) => rm(path, { recursive: true, force: true })), - ); -}); - -it("returns keyed archive evidence with root provenance and unknown executable identity", async () => { - const bundlePath = await mkdtemp(join(tmpdir(), "rea-resource-reader-")); - temporaryDirectories.push(bundlePath); - await mkdir(join(bundlePath, "Resources")); - await writeFile( - join(bundlePath, "Info.plist"), - '', - ); - const archive = build({ - $archiver: "NSKeyedArchiver", - $version: 100000, - $objects: ["$null", "sample"], - $top: { root: { UID: 1 } }, - }); - await writeFile(join(bundlePath, "Resources", "Archive.plist"), archive); - const root: ResolvedAppBundleResourceRoot = { - callerPath: bundlePath, - canonicalPath: bundlePath, - infoPlistPath: join(bundlePath, "Info.plist"), - layout: "root", - executableSha256: null, - }; - const provider = new ArtifactProvider({}, "linux"); - const client = provider.createBundleResourceClient(root); - const execution = await client.execute("inspect_keyed_archive", { - path: "Resources/Archive.plist", - }); - expect(execution.ok).toBe(true); - if (execution.ok) { - expect(execution.value.result).toMatchObject({ - target_sha256: null, - resource_root: { - caller_path: bundlePath, - canonical_path: bundlePath, - executable_sha256: null, - }, - archive_path: "Resources/Archive.plist", - }); - } - expect((await client.close()).ok).toBe(true); -}); diff --git a/src/artifacts/FitArtifactReader.test.ts b/src/artifacts/FitArtifactReader.test.ts index 2090ecc960..10c44f41e5 100644 --- a/src/artifacts/FitArtifactReader.test.ts +++ b/src/artifacts/FitArtifactReader.test.ts @@ -55,6 +55,41 @@ const firstEntry = async (reader: FitArtifactReader) => { return entry; }; +it.each(["close", "cancel"] as const)( + "refuses to finish a paused FIT inventory after %s", + async (action) => { + const reader = new FitArtifactReader( + await writeFit("paused-inventory.itb", FIT_EXTERNAL), + ); + const controller = new AbortController(); + const reason = { stage: "after final entry" }; + const iterator = reader.entries(controller.signal)[Symbol.asyncIterator](); + try { + expect((await iterator.next()).done).toBe(false); + expect((await iterator.next()).done).toBe(false); + if (action === "close") await reader.close(); + else controller.abort(reason); + const failure: unknown = await iterator.next().then( + () => { + throw new Error("Expected the paused inventory to fail"); + }, + (cause: unknown) => cause, + ); + expect(failure).toMatchObject( + action === "close" + ? { reason: "unavailable" } + : { reason: "cancelled" }, + ); + if (action === "cancel") { + if (!(failure instanceof ArtifactReaderFailure)) throw failure; + expect(failure.cause).toBe(reason); + } + } finally { + await reader.close(); + } + }, +); + it.each([ ["embedded", FIT_EMBEDDED, "embedded"], ["external", FIT_EXTERNAL, "data-offset"], diff --git a/src/artifacts/FitArtifactReader.ts b/src/artifacts/FitArtifactReader.ts index 1dfb0931ae..e3dea26293 100644 --- a/src/artifacts/FitArtifactReader.ts +++ b/src/artifacts/FitArtifactReader.ts @@ -82,6 +82,7 @@ export class FitArtifactReader implements ArtifactReader { this.#assertOpen(signal); yield copyArtifactEntry(entry); } + this.#assertOpen(signal); } async open(entry: ArtifactEntry, signal?: AbortSignal): Promise { diff --git a/src/artifacts/GodotPckArtifactReader.test.ts b/src/artifacts/GodotPckArtifactReader.test.ts index 4340130faf..b7a9a81a15 100644 --- a/src/artifacts/GodotPckArtifactReader.test.ts +++ b/src/artifacts/GodotPckArtifactReader.test.ts @@ -184,21 +184,6 @@ describe("GodotPckArtifactReader", () => { }, ); - it("preserves declared MD5 for the shared inventory integrity verifier", async () => { - const bytes = buildGodotPck(3, [ - { path: "a.gd", data: "real", md5: Buffer.alloc(16, 1) }, - ]); - await withReader(bytes, async (reader) => { - const [entry] = await list(reader); - if (entry === undefined) throw new Error("expected one entry"); - expect(entry.declaredIntegrity).toEqual({ - algorithm: "md5", - digest: "01".repeat(16), - }); - expect(await text(await reader.open(entry))).toBe("real"); - }); - }); - it("reports encrypted members without reading them", async () => { const bytes = buildGodotPck(3, [ { path: "secret.gd", data: "x", flags: 1 }, diff --git a/src/artifacts/UImageArtifactReader.boundaries.test.ts b/src/artifacts/UImageArtifactReader.boundaries.test.ts index d4a711dbc3..74de505d2a 100644 --- a/src/artifacts/UImageArtifactReader.boundaries.test.ts +++ b/src/artifacts/UImageArtifactReader.boundaries.test.ts @@ -10,6 +10,8 @@ import { buildUImage, buildMultiPayload } from "../../tests/fixtures/uimage.js"; import { parseUImageHeaderFields, parseMultiImageTable, + UIMAGE_HEADER_SIZE, + UIMAGE_MULTI_TABLE_WINDOW, } from "../domain/uimageFormat.js"; const fixture = async (bytes: Buffer): Promise => { @@ -110,7 +112,31 @@ it("admits a multi-image count before scanning the remaining payload", async () reason: "limit", boundary: "capacity", }); - expect(reads).toHaveBeenCalledTimes(2); // header and bounded table, no payload scan + expect(reads.mock.calls.length).toBeGreaterThan(0); + for (const call of reads.mock.calls) { + const [, , length, position] = call as unknown as [ + Buffer, + number, + number, + number, + ]; + expect(Number.isSafeInteger(position)).toBe(true); + expect(Number.isSafeInteger(length)).toBe(true); + expect(position).toBeGreaterThanOrEqual(0); + expect(length).toBeGreaterThanOrEqual(0); + if (length > 0) + expect(position + length).toBeLessThanOrEqual( + UIMAGE_HEADER_SIZE + UIMAGE_MULTI_TABLE_WINDOW, + ); + } + const results = await Promise.all( + reads.mock.results.map( + ({ value }) => value as Promise<{ bytesRead: number }>, + ), + ); + expect( + results.reduce((sum, result) => sum + result.bytesRead, 0), + ).toBeLessThanOrEqual(UIMAGE_HEADER_SIZE + UIMAGE_MULTI_TABLE_WINDOW); } finally { await reader.close(); reads.mockRestore(); diff --git a/src/artifacts/pyinstaller/CArchive.fixture.ts b/src/artifacts/pyinstaller/CArchive.fixture.ts index 2732420858..481eb82cf1 100644 --- a/src/artifacts/pyinstaller/CArchive.fixture.ts +++ b/src/artifacts/pyinstaller/CArchive.fixture.ts @@ -5,7 +5,8 @@ import { pythonPycHeader, } from "../../python/PythonPyc.fixture.js"; -import { PYINSTALLER_COOKIE_MAGIC } from "./CArchiveLayout.js"; +// PyInstaller CArchiveWriter's on-disk cookie signature, independent of REA's reader. +const CARCHIVE_COOKIE_MAGIC = Buffer.from("4d45490c0b0a0b0e", "hex"); /** Final CPython 3.9 bytecode magic, little-endian with the CR LF trailer. */ export const PYTHON_39_MAGIC = Buffer.from("610d0d0a", "hex"); @@ -172,7 +173,7 @@ export const cArchiveBytecodeExecutable = () => { /** Cookie whose package range starts before the file. */ export const cookieRangeOutsideFile = (): Buffer => { const cookie = Buffer.alloc(88); - PYINSTALLER_COOKIE_MAGIC.copy(cookie, 0); + CARCHIVE_COOKIE_MAGIC.copy(cookie, 0); cookie.writeUInt32BE(200, 8); cookie.writeUInt32BE(80, 12); cookie.writeUInt32BE(32, 16); @@ -255,7 +256,7 @@ const cookieBytes = (input: { }): Buffer => { const size = input.layout === "2.1" ? 88 : 24; const bytes = Buffer.alloc(size); - PYINSTALLER_COOKIE_MAGIC.copy(bytes, 0); + CARCHIVE_COOKIE_MAGIC.copy(bytes, 0); bytes.writeUInt32BE(input.tocOffset + input.tocLength + size, 8); bytes.writeUInt32BE(input.tocOffset, 12); bytes.writeUInt32BE(input.tocLength, 16); diff --git a/src/artifacts/pyinstaller/PyInstallerArtifactReader.test.ts b/src/artifacts/pyinstaller/PyInstallerArtifactReader.test.ts index a92b2e2a52..221255522f 100644 --- a/src/artifacts/pyinstaller/PyInstallerArtifactReader.test.ts +++ b/src/artifacts/pyinstaller/PyInstallerArtifactReader.test.ts @@ -909,67 +909,6 @@ it("retains the descriptor owner when admission cleanup fails", async () => { } }); -it("inventories and extracts a Windows member name under its canonical path", async () => { - const storedName = String.raw`pkg\data.bin`; - const pyzName = String.raw`res\icon.bin`; - const payload = Buffer.from("nested-bytes"); - const icon = Buffer.from("icon-bytes"); - const path = await writeArchive( - "windows", - cArchiveExecutable({ - carrier: "pe", - layout: "2.1", - pythonVersion: 309, - pythonLibrary: "python39.dll", - members: [ - { name: storedName, typeCode: "x", payload, compress: false }, - { - name: "PYZ.pyz", - typeCode: "z", - payload: pyzArchive({ - magic: PYTHON_39_MAGIC, - modules: [{ name: pyzName, typeCode: 2, body: icon }], - }), - compress: false, - }, - ], - }), - ); - const snapshot = await scanCanonicalArtifactInventory(path); - const member = snapshot.occurrences.find( - (occurrence) => occurrence.logical_path === "pkg/data.bin", - ); - const resource = snapshot.occurrences.find( - (occurrence) => occurrence.logical_path === "PYZ.pyz/res/icon.bin", - ); - expect(member?.format_metadata).toMatchObject({ source_name: storedName }); - expect(resource?.format_metadata).toMatchObject({ source_name: pyzName }); - expect(member?.source_path).toBe(storedName); - expect(resource?.source_path).toBe(pyzName); - const directory = await createTestTempDirectory("rea-pyinstaller-out-"); - const output = join(directory, "extracted"); - await materializeArtifactInventory( - { - inputPath: path, - inputFormat: "pe", - outputRoot: output, - environment: process.env, - integrityPolicy: "fail", - }, - path, - snapshot, - ); - expect( - Buffer.compare(await readFile(join(output, "pkg", "data.bin")), payload), - ).toBe(0); - expect( - Buffer.compare( - await readFile(join(output, "PYZ.pyz", "res", "icon.bin")), - icon, - ), - ).toBe(0); -}); - const posixPathFixture = async () => { const backslashName = String.raw`a\b.txt`; const slashName = "a/b.txt"; diff --git a/src/browser/BrowserScenario.test.ts b/src/browser/BrowserScenario.test.ts index 16a7b00293..9684839ffb 100644 --- a/src/browser/BrowserScenario.test.ts +++ b/src/browser/BrowserScenario.test.ts @@ -227,9 +227,6 @@ describe("browserScenarioSchema", () => { at_end: ["dom", "storage"], events: ["console", "page-errors", "network", "websockets"], }); - expect(parsed.capture.limits).toEqual( - DEFAULT_BROWSER_SCENARIO_CAPTURE_LIMITS, - ); }); it("allows callers to choose visible browsers and service workers", () => { diff --git a/src/browser/PlaywrightElectronActiveActions.test.ts b/src/browser/PlaywrightElectronActiveActions.test.ts index 16618377db..c9c4ec8302 100644 --- a/src/browser/PlaywrightElectronActiveActions.test.ts +++ b/src/browser/PlaywrightElectronActiveActions.test.ts @@ -188,8 +188,6 @@ it("preserves local Playwright action diagnostics", async () => { status: "failed", error: `locator ${selector} failed with token=raw-secret`, }); - expect(result[0]?.error).toContain(selector); - expect(result[0]?.error).toContain("raw-secret"); }); it("retains complete action errors and Electron window text", async () => { diff --git a/src/browser/PlaywrightElectronActiveProvider.test.ts b/src/browser/PlaywrightElectronActiveProvider.test.ts index 4c3e10bec5..e3f7ec6a19 100644 --- a/src/browser/PlaywrightElectronActiveProvider.test.ts +++ b/src/browser/PlaywrightElectronActiveProvider.test.ts @@ -3,6 +3,8 @@ import { cleanupElectronProcesses } from "./ElectronProcessCleanup.js"; import { afterEach, beforeEach, expect, it, vi } from "vitest"; import { writeFile } from "node:fs/promises"; import { dirname, join } from "node:path"; +import { createHash } from "node:crypto"; +import { fileURLToPath } from "node:url"; import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js"; import { @@ -30,6 +32,27 @@ it("passes the selected environment and ownership token to Electron launch", asy vi.stubEnv("LANG", "ambient-language"); vi.stubEnv("XAUTHORITY", "/ambient/.Xauthority"); let launchedEnvironment: unknown; + const callerArguments = [ + "--", + "/caller/plugin.asar", + "-r", + "caller-hook.cjs", + "\\server\\share\\".repeat(150), + ]; + const launchPlan = [ + "-r", + fileURLToPath( + new URL("../../scripts/electron-active-hook.cjs", import.meta.url), + ), + process.execPath, + ...callerArguments, + ]; + const handoff = JSON.stringify({ + length: launchPlan.length, + sha256: createHash("sha256") + .update(JSON.stringify(launchPlan)) + .digest("hex"), + }); const provider = new PlaywrightElectronActiveProvider( { LANG: "selected-language", @@ -38,6 +61,7 @@ it("passes the selected environment and ownership token to Electron launch", asy XAUTHORITY: "/selected/display-authorization", WAYLAND_DISPLAY: "selected-wayland", REA_PRIVATE_VALUE: "not-a-launch-setting", + REA_ELECTRON_CAPTURE_ARGV: "forged ambient launch plan", }, async (options) => { launchedEnvironment = options?.env; @@ -48,6 +72,7 @@ it("passes the selected environment and ownership token to Electron launch", asy ...electronActiveObservationInputSchema.parse({ executable_path: process.execPath, application_path: process.execPath, + args: callerArguments, }), application_root: dirname(process.execPath), }); @@ -60,7 +85,9 @@ it("passes the selected environment and ownership token to Electron launch", asy XAUTHORITY: "/selected/display-authorization", WAYLAND_DISPLAY: "selected-wayland", REA_PROCESS_RUN_ID: expect.any(String), + REA_ELECTRON_CAPTURE_ARGV: handoff, }); + expect(handoff.length).toBeLessThan(128); }); it("passes required Windows runtime variables without inheriting unrelated values", () => { diff --git a/src/browser/PlaywrightElectronActiveProvider.ts b/src/browser/PlaywrightElectronActiveProvider.ts index 8ec944753c..ed9cf23117 100644 --- a/src/browser/PlaywrightElectronActiveProvider.ts +++ b/src/browser/PlaywrightElectronActiveProvider.ts @@ -6,7 +6,7 @@ import { ElectronCaptureCleanup } from "./ElectronCaptureCleanup.js"; import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js"; import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; import { electronFuseFailure } from "../electron/ElectronFuseFailures.js"; -import { randomUUID } from "node:crypto"; +import { createHash, randomUUID } from "node:crypto"; import { fileURLToPath } from "node:url"; import type { _electron as electron } from "playwright-core"; @@ -155,11 +155,25 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati await systemProcessOwnershipHost.captureBaseline?.(options.signal); if (options.signal?.aborted) throw new BrowserObservationError(OPERATION, "cancelled"); + const applicationArgs = [ + "-r", + hookPath, + paths.application, + ...input.args, + ]; const launchOptions: Parameters[0] = { executablePath: paths.executable, cwd: paths.root, - env: safeElectronEnvironment(this.environment, runId), - args: ["-r", hookPath, paths.application, ...input.args], + env: { + ...safeElectronEnvironment(this.environment, runId), + REA_ELECTRON_CAPTURE_ARGV: JSON.stringify({ + length: applicationArgs.length, + sha256: createHash("sha256") + .update(JSON.stringify(applicationArgs)) + .digest("hex"), + }), + }, + args: applicationArgs, timeout: Math.max(1, startupDeadline - performance.now()), }; application = await containLaunchRejections(() => { diff --git a/src/cli/analysisEvidenceDelivery.ts b/src/cli/analysisEvidenceDelivery.ts index 0550210b15..1858bd3fa8 100644 --- a/src/cli/analysisEvidenceDelivery.ts +++ b/src/cli/analysisEvidenceDelivery.ts @@ -1,5 +1,8 @@ import { recordCliEvidencePublication } from "./cliEvidencePublication.js"; -import { analysisEvidenceSources } from "../application/AnalysisEvidenceSources.js"; +import { + analysisEvidenceSources, + undeliveredSourceClosure, +} from "../application/AnalysisEvidenceSources.js"; import type { AnalysisViewRequest } from "../domain/analysisView/analysisView.js"; import { z } from "incur"; import { @@ -14,12 +17,18 @@ import { redactExplicitFailure } from "../domain/explicitSensitiveFailure.js"; import { err, ok, type Result } from "../domain/result.js"; import { AnalysisInputError } from "../domain/analysisErrorCore.js"; import { createEvidenceBundle } from "../domain/evidenceBundle.js"; -import { publishEvidenceDelivery } from "../application/EvidenceBundleCommands.js"; +import { + evidenceDeliveryCommitted, + publishEvidenceDelivery, +} from "../application/EvidenceBundleCommands.js"; import { inspectAnalysisViewValidated } from "../application/analysisView/AnalysisViewService.js"; import { localPathStringSchema } from "../domain/localPath.js"; import { absoluteCallerPath } from "../filesystem/callerPath.js"; import { finishAnalysisCommand } from "./finishAnalysisCommand.js"; -import { projectCliAnalysisResult } from "./projectCliAnalysisResult.js"; +import { + isObject, + projectCliAnalysisResult, +} from "./projectCliAnalysisResult.js"; import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js"; import type { ZodSafeParseResult } from "zod"; import { bindCliEvidenceCallerPath } from "./cliEvidenceCallerPath.js"; @@ -169,6 +178,29 @@ export const deliverCliAnalysisEvidence = async ( }); }; +/** + * A requested bundle that never committed cannot silently drop what a completed + * observation cites. Absent delivery or observation leaves the failure untouched. + */ +const withUndeliveredSourceClosure = ( + failure: JsonValue, + input: CliEvidenceDelivery, +): JsonValue => { + if (input.evidenceOutput === undefined || !isObject(failure)) return failure; + const { details } = failure; + if (!isObject(details)) return failure; + const closure = undeliveredSourceClosure( + details.partial_observation, + input.sourceEvidence ?? [], + // Error output is not the caller's selected Evidence sink. Keep records + // out of it when this invocation carries explicit exclusions. + { inlineRecords: (input.sensitiveValues?.length ?? 0) === 0 }, + ); + return closure === undefined + ? failure + : { ...failure, details: { ...details, source_closure: closure } }; +}; + /** Publish already finalized CLI analysis without opening a second provider/session. */ export const publishCliAnalysisEvidence = async ( result: Result, @@ -179,7 +211,9 @@ export const publishCliAnalysisEvidence = async ( projectAnalysisError( redactExplicitFailure(error, input.sensitiveValues ?? []), ); - if (!result.ok) return result.error; + const undelivered = (error: AnalysisError) => + withUndeliveredSourceClosure(projectFailure(error), input); + if (!result.ok) return withUndeliveredSourceClosure(result.error, input); const complete = parseEvidence(result.value); const inputError = cliEvidenceDeliveryInputError(input); if (inputError !== undefined) { @@ -197,7 +231,7 @@ export const publishCliAnalysisEvidence = async ( }) : ok(complete); if (!delivered.ok) - return projectFailure(delivered.error.retainPartialObservation(complete)); + return undelivered(delivered.error.retainPartialObservation(complete)); // This projection answers the same direct invocation as its parent. Later // inspect_analysis_view calls own their own input and do not inherit it. const selected = @@ -205,7 +239,7 @@ export const publishCliAnalysisEvidence = async ( ? withEvidenceCallerInputPath(delivered.value, complete.caller_input_path) : delivered.value; const inspectionSources = analysisEvidenceSources(complete); - if (!inspectionSources.ok) return projectFailure(inspectionSources.error); + if (!inspectionSources.ok) return undelivered(inspectionSources.error); const published = await publishEvidenceDelivery({ bundle: createEvidenceBundle([ ...new Map( @@ -220,7 +254,10 @@ export const publishCliAnalysisEvidence = async ( operation, signal, }); - if (!published.ok) return projectFailure(published.error); + if (!published.ok) + return evidenceDeliveryCommitted(published.error) + ? projectFailure(published.error) + : undelivered(published.error); const receipt = { ...(input.sensitiveValues?.some( (literal) => diff --git a/src/cli/analysisViewCommands.ts b/src/cli/analysisViewCommands.ts index 7ac18af53f..a6fea44e69 100644 --- a/src/cli/analysisViewCommands.ts +++ b/src/cli/analysisViewCommands.ts @@ -26,12 +26,12 @@ export const registerAnalysisViewCommands = ( ): void => { cli.command(CLI_COMMANDS.inspectEvidenceBundle, { description: - "Inspect complete or selected discovery metadata from a local Evidence bundle", + "Inspect a local Evidence bundle completely, as discovery metadata, or as a page of selected complete records", args: z.object({ inputJson: z .string() .describe( - "JSON or JSON file with absolute path, detail and optional record filters", + "JSON or JSON file with absolute path, detail, optional record filters, and records offset/limit", ), }), run: ({ args }) => diff --git a/src/cli/applicationCommands.ts b/src/cli/applicationCommands.ts index e7a64f5048..6a88982ae4 100644 --- a/src/cli/applicationCommands.ts +++ b/src/cli/applicationCommands.ts @@ -218,7 +218,7 @@ const registerJsonCommand = ({ ? result.value : { error: "Application workflow failed", - ...projectAnalysisError(result.error), + ...projectAnalysisError(result.error, "invocation"), }; }), }); diff --git a/src/cli/finishAnalysisCommand.ts b/src/cli/finishAnalysisCommand.ts index 74ca59058f..ba4b58943b 100644 --- a/src/cli/finishAnalysisCommand.ts +++ b/src/cli/finishAnalysisCommand.ts @@ -34,8 +34,10 @@ export const finishAnalysisCommand = async < ) => Value | undefined; } = {}, ): Promise>> => { + // The command releases its provider resources and runtime before the caller + // reads this result, so guidance needing a continuing session is omitted. const projectFailure = (error: AnalysisError) => - projectAnalysisError(options.redactFailure?.(error) ?? error); + projectAnalysisError(options.redactFailure?.(error) ?? error, "invocation"); let result: Result; let cleanup: AnalysisError | undefined; try { @@ -61,6 +63,7 @@ export const finishAnalysisCommand = async < result.error.capturedOutput !== undefined ? projectAnalysisErrorWithoutCapturedOutput( options.redactFailure?.(result.error) ?? result.error, + "invocation", ) : initialFailure; try { diff --git a/src/cli/projectCliAnalysisResult.ts b/src/cli/projectCliAnalysisResult.ts index 3b541c7aa8..61bbc03208 100644 --- a/src/cli/projectCliAnalysisResult.ts +++ b/src/cli/projectCliAnalysisResult.ts @@ -16,7 +16,7 @@ export const projectCliAnalysisResult = ( return projectInputFields(result.error, inputBinding); }; -const isObject = ( +export const isObject = ( value: JsonValue | undefined, ): value is Record => value !== null && typeof value === "object" && !Array.isArray(value); diff --git a/src/composition/jeb.ts b/src/composition/jeb.ts index cc6cc0438c..2cbfe295d1 100644 --- a/src/composition/jeb.ts +++ b/src/composition/jeb.ts @@ -2,6 +2,7 @@ import type { JebAnalysisPort } from "../application/jeb/JebAnalysisPort.js"; import { parseJebMcpTimeoutMs } from "../config/environment.js"; import { ConfigurationError } from "../domain/configurationErrors.js"; import { AnalysisInputError } from "../domain/analysisErrorCore.js"; +import { analysisErrorRemediationAction } from "../domain/analysisErrorPresentation.js"; import { err } from "../domain/result.js"; import type { JebMcpConnectionFactory } from "../jeb/JebMcpConnection.js"; import { JebProvider } from "../jeb/JebProvider.js"; @@ -53,6 +54,7 @@ const unconfiguredJebProvider = ( status: "unavailable", code: "not_configured", reason: `${setting} is invalid: ${error.settings[0]?.constraint ?? error.message}`, + remediation: analysisErrorRemediationAction(error), diagnostics: { setting }, }), close: () => Promise.resolve(), diff --git a/src/config.test.ts b/src/config.test.ts index 7abcf113eb..5b6264bd97 100644 --- a/src/config.test.ts +++ b/src/config.test.ts @@ -359,6 +359,12 @@ describe("configuration failure diagnostics", () => { it.each(["1", "90000", "2147483647"])( "accepts Swift compile deadline %s without changing Hopper's deadline", (value) => { + expect( + parseEnvironmentValues({ REA_SWIFTC_COMPILE_TIMEOUT_MS: value }), + ).toMatchObject({ + ok: true, + value: { REA_SWIFTC_COMPILE_TIMEOUT_MS: Number(value) }, + }); expect( parseConfig({ REA_SWIFTC_COMPILE_TIMEOUT_MS: value }), ).toMatchObject({ diff --git a/src/contracts/analysisDetail.test.ts b/src/contracts/analysisDetail.test.ts index 18292692b0..6922350ef7 100644 --- a/src/contracts/analysisDetail.test.ts +++ b/src/contracts/analysisDetail.test.ts @@ -18,12 +18,14 @@ import { toolContract } from "./toolContracts.js"; const projectSummary = (parent: { readonly evidence_id: string; + readonly predicate_type: string; readonly operation: string; readonly normalized_result: unknown; readonly limitations: readonly string[]; }) => projectAnalysisView( { + predicateType: parent.predicate_type, evidenceId: parent.evidence_id, operation: parent.operation, normalizedResult: parent.normalized_result, @@ -74,6 +76,7 @@ it("advertises only that operation's summary from analysis detail outputs", () = ); const parent = analysisViewCaptureEvidence(environmentStateCaptureFixture()); const captureParent = { + predicateType: parent.predicate_type, evidenceId: parent.evidence_id, operation: parent.operation, normalizedResult: parent.normalized_result, diff --git a/src/contracts/analysisViewToolContracts.ts b/src/contracts/analysisViewToolContracts.ts index 14ffdf63bb..87fdfec068 100644 --- a/src/contracts/analysisViewToolContracts.ts +++ b/src/contracts/analysisViewToolContracts.ts @@ -17,7 +17,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [ ...toolContractMetadata("inspect_analysis_view"), kind: "application", description: - 'Use javascript-routes collections to query route patterns, source-linked occurrences, source status, unknowns or summary counts from retained JavaScript application or web-bundle Evidence. Use web-bundle collections endpoints or inferences to page endpoint observations and inferences from retained analyze_web_bundle Evidence; summary returns target URL, origin, and counts. Inspect selected views of supported completed Evidence without repeating analysis. Choose a view compatible with the parent operation described below. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Native export tables use kind native-exports, collection exports/issues, exact names or original indices, selected fields and matching-row pagination; parent Evidence plus source_index identifies each row. Source coverage stays separate from page coverage. BEAM views use kind beam, collections atoms/exports/imports/lambdas/chunks, exact names/indices/fields and matching-row pages; import modules intersects function names. source_index identifies original parent-and-collection rows, preserving every arity and raw atom index. beam.counts preserves null missing tables versus decoded empty tables; runtime absence remains unknown. PYC views use kind python-pyc, collection code_objects, names/indices/fields and paging; source_index scopes every same-name alternative to the parent, nested_code keeps code-constant references, and pyc.coverage preserves source coverage. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. Rust metadata uses kind rust, collection packages/edges, exact names/versions or original indices, selected fields, edge from_indices/to_indices and matching-row pagination. Embedded-record coverage is separate from unknown runtime dependencies. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent also supports summary and browser-scenario steps/events/transactions/text pages, preserving step_indices, event_kinds, transaction_ids and selected artifacts. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the first related write and reports entity types with an identity field (proved by a path parameter or inferred from a field name), entities in collection order with per-view values where views differ, documents for non-parameterized reads that are not entities and have one initial request line (item-shaped paths stay recorded), and a render plan per read tool: item and list views reproduce their latest recordings from the state (a list also learns the filters, search, sort and paging its recordings verify, and answers new values of them from the state), and every other read tool is a recorded view with a reason. It also reports a write plan per state-changing tool (create, update, delete or document, built from its first successful call) and write_clock; a plan is served only when replaying the capture shows a later read confirming its effect and none contradicting it. Types and identity are inferences; values are recorded data, so it refuses parents inspected with sensitive values, and a scenario parent must have selected response bodies; its values keep the capture redaction markers, and a write whose URL it redacted ends the state of every type. The environment-coverage view, view {kind: "environment-coverage", template_rules: [...]}, summarizes that state without its entities: reads.state and reads.recorded count read tools answered from the state and replayed (recorded_reasons per view reason), writes.served lists the served write tools, and writes.withheld lists each other write tool with its reason, plan record, contradicting read and a hint about what to record next; it refuses what the environment-state view refuses. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. A completed Evidence JSON file on this host uses {kind: "evidence-file", path: ""}: REA reads and authenticates it, then projects it like retained Evidence, so a large capture is bounded by the file reader rather than by inline-input accounting. The file may also be an Evidence bundle written by CLI --evidence_output or export_evidence_bundle; select evidence_id when it contains more than one record. For layout, JavaScript, inventory, WASM, Mach-O, Interface Builder, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Typed module references use view {kind: "javascript", collection: "modules", projection: "references", where: {path: "main.js", kind: "javascript-module"}, offset: 0, limit: 64}; where matches exact retained strings (null selects unavailable paths). Rows include canonical identity, retained source-range count and observation support states. Source coverage and selection remain explicit; exhausting matching rows does not prove runtime loading or complete analysis. For exact file/native facts and processing gaps, use view {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"]}. Rows page original observations with canonical identities and Evidence; missing properties remain omitted. Paths match recorded artifact-path/source-range locations; optional kinds filter nodes. To discover field names without values, opt into include_available_property_keys: true with property_keys: []. Canonical identity and Evidence are separate from properties; keep each node_id/observation_id pair together when composing results. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions. Occurrences accept exact logical_paths and occurrence_ids; nodes accept artifact_ids. Batch filters intersect (omitted means all available, [] means none); fields selects columns while retaining identity links. source_complete describes retained graph membership; source_coverage describes producer coverage, including unknown encrypted PCK directories. Page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. Mach-O views use kind macho and facet entrypoints/architectures/build_metadata/load_commands/dependencies/imports/exports/segments/provenance; source_exhaustive false means unreturned rows may exist. Interface Builder views use kind interface-builder and collection documents/nodes/edges/coverage; nodes and edges accept an exact document relative_path. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Managed member summaries keep identity, selection with unmatched names, coverage and per-collection counts; managed-members pages use collection types/exported_types/fields/methods/member_refs/call_edges/field_accesses/user_strings. Exact names and declaring_types filters intersect before paging where supported; fields selects columns (distinct from collection: "fields"). Identity/source_index, module MVID, source coverage and unresolved filter counts remain; omit body to avoid IL anchors. Omitted fields preserves full rows. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Protobuf views use kind protobuf, collection bodies/frames/fields, with record_ordinals, body_pointers and fields; field pages also select frame_indices, field_ids, field_numbers or schema_names. Byte identity, offset basis and body status remain present. Text and schema_value are opt-in page columns. APK signing views use kind apk-signature and collection signers/digests/lineage, with ids, signer_ids, schemes, fields and paging. Identity links remain when fields is empty. Declared identities and digest matches do not establish signature verification or authenticity. Model views use kind model-artifact and collection metadata (keys), tensors (names/types) or types (derived totals), with indices, fields and paging. Header SHA-256 scopes declarations; full-file identity remains unknown unless full hashing was selected. Arrays retain typed eight-element previews with explicit omissions. IL2CPP metadata views use kind il2cpp-metadata and collection sections/images/types/methods/fields/properties/events/parameters/string_literals/code_gen_modules/method_addresses/return_types/field_types/parameter_types/generic_method_instances/generic_method_issues. Select indices, fields and pages; sections/images/types/methods/fields/properties/events/parameters/return_types/field_types/parameter_types/generic_method_instances accept exact names (native type names match retained rendered values, preserving same-name rows; null names never match), types also accept image_indices/namespaces/declaring_types, and methods/members accept declaring_types. Parameters and parameter_types accept method_indices; code_gen_modules accept image_indices; method_addresses and generic_method_instances accept addresses/states (generic_method_instances also method_indices), preserving every folded alternative; generic_method_issues accept rows. Row index is scoped to the metadata digest; native pages add binary_source. Uncollected native tables have null counts and remain unavailable. Desktop directory views use kind desktop-application and collection files/omissions/candidates/components/next_probes, with fields and pagination; files and omissions accept exact paths or their stable IDs, and candidates accept families. Source coverage and stable row IDs remain available. Native load-image Evidence uses kind native-load-image with collection source_files/mappings/relocations/entry_points/entry_context/checks/sections, original indices, collection-specific fields and offset/limit paging. Null collections remain unrecorded or inapplicable; pages preserve unsupported verification. Native inventory Evidence from list_names, list/search_procedures or list/search_strings uses view: {kind: "native-inventory", offset: 0, limit: 64, fields: ["value"]}. Rows keep source_index and address even with selected fields. Summary counts describe retained rows; provider discovery coverage remains unknown. Native function Evidence accepts kind summary for identity, availability and facet counts, or kind native for selected facts: view: {kind: "native", facet: "procedure"}. Facets select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', + 'Use javascript-routes collections to query route patterns, source-linked occurrences, source status, unknowns or summary counts from retained JavaScript application or web-bundle Evidence. Use web-bundle collections endpoints or inferences to page endpoint observations and inferences from retained analyze_web_bundle Evidence; summary returns target URL, origin, and counts. Inspect selected views of supported completed Evidence without repeating analysis. Choose a view compatible with the parent operation described below. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Native export tables use kind native-exports, collection exports/issues, exact names or original indices, selected fields and matching-row pagination; parent Evidence plus source_index identifies each row. Source coverage stays separate from page coverage. BEAM views use kind beam, collections atoms/exports/imports/lambdas/chunks, exact names/indices/fields and matching-row pages; import modules intersects function names. source_index identifies original parent-and-collection rows, preserving every arity and raw atom index. beam.counts preserves null missing tables versus decoded empty tables; runtime absence remains unknown. PYC views use kind python-pyc, collection code_objects, names/indices/fields and paging; source_index scopes every same-name alternative to the parent, nested_code keeps code-constant references, and pyc.coverage preserves source coverage. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. Rust metadata uses kind rust, collection packages/edges, exact names/versions or original indices, selected fields, edge from_indices/to_indices and matching-row pagination. Embedded-record coverage is separate from unknown runtime dependencies. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent also supports summary and browser-scenario steps/events/transactions/text pages, preserving step_indices, event_kinds, transaction_ids and selected artifacts. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the first related write and reports entity types with an identity field (proved by a path parameter or inferred from a field name), entities in collection order with per-view values where views differ, documents for non-parameterized reads that are not entities and have one initial request line (item-shaped paths stay recorded), and a render plan per read tool: item and list views reproduce their latest recordings from the state (a list also learns the filters, search, sort and paging its recordings verify, and answers new values of them from the state), and every other read tool is a recorded view with a reason. It also reports a write plan per state-changing tool (create, update, delete or document, built from its first successful call) and write_clock; a plan is served only when replaying the capture shows a later read confirming its effect and none contradicting it. Types and identity are inferences; values are recorded data, so it refuses parents inspected with sensitive values, and a scenario parent must have selected response bodies; its values keep the capture redaction markers, and a write whose URL it redacted ends the state of every type. The environment-coverage view, view {kind: "environment-coverage", template_rules: [...]}, summarizes that state without its entities: reads.state and reads.recorded count read tools answered from the state and replayed (recorded_reasons per view reason), writes.served lists the served write tools, and writes.withheld lists each other write tool with its reason, plan record, contradicting read and a hint about what to record next; it refuses what the environment-state view refuses. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. A completed Evidence JSON file on this host uses {kind: "evidence-file", path: ""}: REA reads and authenticates it, then projects it like retained Evidence, so a large capture is bounded by the file reader rather than by inline-input accounting. The file may also be an Evidence bundle written by CLI --evidence_output or export_evidence_bundle; select evidence_id when it contains more than one record. For layout, JavaScript, inventory, WASM, Mach-O, Interface Builder, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Typed module references use view {kind: "javascript", collection: "modules", projection: "references", where: {path: "main.js", kind: "javascript-module"}, offset: 0, limit: 64}; where matches exact retained strings (null selects unavailable paths). Rows include canonical identity, retained source-range count and observation support states. Source coverage and selection remain explicit; exhausting matching rows does not prove runtime loading or complete analysis. For exact file/native facts and processing gaps, use view {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"]}. Rows page original observations with canonical identities and Evidence; missing properties remain omitted. Paths match recorded artifact-path/source-range locations; optional kinds filter nodes. To discover field names without values, opt into include_available_property_keys: true with property_keys: []. Canonical identity and Evidence are separate from properties; keep each node_id/observation_id pair together when composing results. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions. Occurrences accept exact logical_paths and occurrence_ids; nodes accept artifact_ids. Batch filters intersect (omitted means all available, [] means none); fields selects columns while retaining identity links. source_complete describes retained graph membership; source_coverage describes producer coverage, including unknown encrypted PCK directories. Page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. Mach-O views use kind macho and facet entrypoints/architectures/build_metadata/load_commands/dependencies/imports/exports/segments/provenance; source_exhaustive false means unreturned rows may exist. Interface Builder views use kind interface-builder and collection documents/nodes/edges/coverage; nodes and edges accept an exact document relative_path. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Managed member summaries keep identity, selection with unmatched names, coverage and per-collection counts; managed-members pages use collection types/exported_types/fields/methods/member_refs/call_edges/field_accesses/user_strings. Exact names and declaring_types filters intersect before paging where supported; fields selects columns (distinct from collection: "fields"). Identity/source_index, module MVID, source coverage and unresolved filter counts remain; omit body to avoid IL anchors. Omitted fields preserves full rows. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Protobuf views use kind protobuf, collection bodies/frames/fields, with record_ordinals, body_pointers and fields; field pages also select frame_indices, field_ids, field_numbers or schema_names. Byte identity, offset basis and body status remain present. Text and schema_value are opt-in page columns. APK signing views use kind apk-signature and collection signers/digests/lineage, with ids, signer_ids, schemes, fields and paging. Identity links remain when fields is empty. Declared identities and digest matches do not establish signature verification or authenticity. Model views use kind model-artifact and collection metadata (keys), tensors (names/types) or types (derived totals), with indices, fields and paging. Header SHA-256 scopes declarations; full-file identity remains unknown unless full hashing was selected. Arrays retain typed eight-element previews with explicit omissions. IL2CPP metadata views use kind il2cpp-metadata and collection sections/images/types/methods/fields/properties/events/parameters/string_literals/code_gen_modules/method_addresses/return_types/field_types/parameter_types/generic_method_instances/generic_method_issues. Select indices, fields and pages; sections/images/types/methods/fields/properties/events/parameters/return_types/field_types/parameter_types/generic_method_instances accept exact names (native type names match retained rendered values, preserving same-name rows; null names never match), types also accept image_indices/namespaces/declaring_types, and methods/members accept declaring_types. Parameters and parameter_types accept method_indices; code_gen_modules accept image_indices; method_addresses and generic_method_instances accept addresses/states (generic_method_instances also method_indices), preserving every folded alternative; generic_method_issues accept rows. Row index is scoped to the metadata digest; native pages add binary_source. Uncollected native tables have null counts and remain unavailable. Desktop directory views use kind desktop-application and collection files/omissions/candidates/components/next_probes, with fields and pagination; files and omissions accept exact paths or their stable IDs, and candidates accept families. Source coverage and stable row IDs remain available. Native load-image Evidence uses kind native-load-image with collection source_files/mappings/relocations/entry_points/entry_context/checks/sections, original indices, collection-specific fields and offset/limit paging. Null collections remain unrecorded or inapplicable; pages preserve unsupported verification. Native inventory Evidence from list_names, list/search_procedures or list/search_strings uses view: {kind: "native-inventory", offset: 0, limit: 64, fields: ["value"]}. Rows keep source_index and address even with selected fields. Summary counts describe retained rows; provider discovery coverage remains unknown. Native function Evidence accepts kind summary for identity, availability and facet counts, or kind native for selected facts: view: {kind: "native", facet: "procedure"}. Facets select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Retained batch_decompile Evidence accepts kind summary for counts and sizes, or view: {kind: "batch-decompile", collection: "items", where: {statuses: ["error"]}} for body-free rows with exact indices, addresses, procedure_addresses or statuses (include_pseudocode adds bodies), or collection "pseudocode" with an index to page one body. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', inputSchema: inspectAnalysisViewInputSchema, outputSchema: evidenceResultOf(analysisViewResultSchema), examples: [ diff --git a/src/contracts/applicationToolContracts.ts b/src/contracts/applicationToolContracts.ts index 0170604a00..e8bb3cfb27 100644 --- a/src/contracts/applicationToolContracts.ts +++ b/src/contracts/applicationToolContracts.ts @@ -1,6 +1,8 @@ import { applicationInventoryProjectionInputSchema } from "../domain/mobileApplicationGraphProjection.js"; import { z } from "zod"; import { + featureTraceSummaryViewResultSchema, + featureTraceViewResultSchema, sourceToBundleSummaryViewResultSchema, sourceToBundlePageViewResultSchema, } from "../domain/analysisView/analysisView.js"; @@ -13,7 +15,7 @@ import { compareApplicationVersionsRequestSchema, compareJavaScriptExportShapesRequestSchema, compareSourceToBundleRequestSchema, - traceApplicationFeatureRequestSchema, + traceApplicationFeatureToolRequestSchema, traceJavaScriptSemanticsRequestSchema, } from "./javascript/applicationWorkflowInputContracts.js"; import { reconstructionCoverageEvaluationInputSchema } from "../domain/reconstructionCoverageInput.js"; @@ -51,7 +53,13 @@ import { MACOS_APPLICATION_GRAPH_EXAMPLE, } from "./mobileApplicationGraphExamples.js"; -const traceOutputSchema = evidenceResultOf(applicationFeatureTraceResultSchema); +const traceOutputSchema = evidenceResultOf( + z.union([ + applicationFeatureTraceResultSchema, + featureTraceSummaryViewResultSchema, + featureTraceViewResultSchema, + ]), +); const semanticTraceOutputSchema = evidenceResultOf( javaScriptSemanticTraceResultSchema, ); @@ -131,9 +139,9 @@ export const APPLICATION_TOOL_CONTRACTS = [ name: "trace_application_feature", ...toolContractMetadata("trace_application_feature"), description: - "Trace a typed literal seed through every reachable part of an authenticated JavaScript Application Graph supplied as inline Evidence or exact same-session retained references. Original static, native, passive-runtime, inferred, and unknown authorities remain distinct; native addon handoffs never open a provider or execute the application.", + "Trace a typed literal seed through every reachable part of an authenticated JavaScript Application Graph supplied as inline Evidence or exact same-session retained references. Original static, native, passive-runtime, inferred, and unknown authorities remain distinct; native addon handoffs never open a provider or execute the application. An optional view returns selected facts and retains the complete trace.", kind: "application", - inputSchema: traceApplicationFeatureRequestSchema, + inputSchema: traceApplicationFeatureToolRequestSchema, outputSchema: traceOutputSchema, examples: [ { diff --git a/src/contracts/enhancedToolContracts.ts b/src/contracts/enhancedToolContracts.ts index 2a4087899e..8952f42447 100644 --- a/src/contracts/enhancedToolContracts.ts +++ b/src/contracts/enhancedToolContracts.ts @@ -47,8 +47,10 @@ export const ENHANCED_TOOL_CONTRACTS = [ ), enhanced( "batch_decompile", - "Decompile each explicit procedure symbol or address, with one provider read active per batch. Preserves each input selector in address and reports the observed canonical entry and entry label in procedure; identity or label failures remain explicit without discarding usable pseudocode. Returns ordered per-item success or error results and aggregate counts. An item whose provider answer has no pseudocode keeps that observation's limitations, including any reason the provider reported. The complete reply retains every pseudocode body; use selected small batches to bound retained output. A per-function limit is not a batch deadline; allow for every item or split calls.", - enhancedInputSchemas.batch_decompile, + "Decompile each explicit procedure symbol or address, with one provider read active per batch. Preserves each input selector in address and reports the observed canonical entry and entry label in procedure; identity or label failures remain explicit without discarding usable pseudocode. Returns ordered per-item success or error results and aggregate counts. An item whose provider answer has no pseudocode keeps that observation's limitations, including any reason the provider reported. The complete reply carries every pseudocode body. Select detail summary to retain the batch and receive only counts, then read rows or one body by index through inspect_analysis_view. A per-function limit is not a batch deadline; allow for every item or split calls.", + enhancedInputSchemas.batch_decompile.extend({ + detail: analysisDetailSchema, + }), ), enhanced( "decompile_all", diff --git a/src/contracts/errorSchemas.ts b/src/contracts/errorSchemas.ts index 4395468939..aa8b8f8aac 100644 --- a/src/contracts/errorSchemas.ts +++ b/src/contracts/errorSchemas.ts @@ -34,9 +34,12 @@ const generic = (code: Code) => .object({ code: z.literal(code), ...common, details: genericDetails }) .strict(); +/** The canonical invalid-input projection, also used for facet-local refusals. */ +export const analysisInputErrorProjectionSchema = generic("invalid_request"); + /** Stable discriminated schema shared by every CLI and MCP error surface. */ export const analysisErrorProjectionSchema = z.discriminatedUnion("code", [ - generic("invalid_request"), + analysisInputErrorProjectionSchema, generic("access_denied"), generic("artifact_changed"), generic("unreadable_output"), diff --git a/src/contracts/javascript/applicationWorkflowInputContracts.ts b/src/contracts/javascript/applicationWorkflowInputContracts.ts index 2bf4e8737b..08a4294584 100644 --- a/src/contracts/javascript/applicationWorkflowInputContracts.ts +++ b/src/contracts/javascript/applicationWorkflowInputContracts.ts @@ -3,6 +3,7 @@ import { localPathStringSchema, } from "../../domain/localPath.js"; import { sourceToBundleViewRequestSchema } from "../../domain/analysisView/sourceToBundleView.js"; +import { featureTraceViewRequestSchema } from "../../domain/analysisView/featureTraceView.js"; import { z } from "zod"; import { evidenceInputSchema } from "../../domain/evidenceReference.js"; @@ -25,6 +26,20 @@ export const traceApplicationFeatureRequestSchema = z.strictObject({ application: evidenceInputSchema, }); +/** MCP trace request; a selected view retains the complete trace and returns only those facts. */ +export const traceApplicationFeatureToolRequestSchema = + traceApplicationFeatureRequestSchema.extend({ + view: z + .union([ + z.strictObject({ kind: z.literal("summary") }), + featureTraceViewRequestSchema, + ]) + .optional() + .describe( + "Omitted returns the complete trace. summary or a feature-trace page retains the complete trace in this session first; query it again with inspect_analysis_view and the returned parent_evidence_id.", + ), + }); + /** Semantic trace request accepting inline or same-session application Evidence. */ export const traceJavaScriptSemanticsRequestSchema = traceJavaScriptSemanticsInputSchema.extend({ diff --git a/src/contracts/officialToolContracts.ts b/src/contracts/officialToolContracts.ts index e8b7e23ef3..9d1ab1a899 100644 --- a/src/contracts/officialToolContracts.ts +++ b/src/contracts/officialToolContracts.ts @@ -55,7 +55,7 @@ export const OFFICIAL_TOOL_CONTRACTS = [ ), official( "inspect_native_load_image", - "Inspect the provider's measured source digests, memory mappings, relocations, entry points and entry register context. Independent snapshot verification supports DOS MZ, explicitly selected COM, raw binaries and GameCube DOL; other formats retain their observations with unsupported verification. Overlay and uninitialized coverage remain explicit. Select detail summary for verification state and counts, retaining complete Evidence for inspect_analysis_view kind native-load-image: collection source_files, mappings, relocations, entry_points, entry_context, checks or sections, with indices, fields and offset/limit paging. Relocation bytes and check expected/observed payloads require explicit fields; defaults keep metadata. Null collections are unrecorded or inapplicable. Does not execute or modify the target.", + "Inspect the provider's measured source digests, memory mappings, relocations, entry points and entry register context. Ghidra relocation symbols are provider observations, not runtime binding proof; older Evidence may not record them. Independent snapshot verification supports DOS MZ, explicitly selected COM, raw binaries and GameCube DOL; other formats retain their observations with unsupported verification. Overlay and uninitialized coverage remain explicit. Select detail summary for verification state and counts, retaining complete Evidence for inspect_analysis_view kind native-load-image: collection source_files, mappings, relocations, entry_points, entry_context, checks or sections, with indices, fields and offset/limit paging. Relocation bytes and check expected/observed payloads require explicit fields; defaults keep metadata. Null collections are unrecorded or inapplicable. Does not execute or modify the target.", z.strictObject({ detail: analysisDetailSchema }), { evidenceParameters: "omit-detail" }, ), @@ -218,7 +218,7 @@ export const OFFICIAL_TOOL_CONTRACTS = [ ), official( "search_procedures", - `Search every analyzed procedure name using literal matching by default or regex when requested. Results are deterministic and complete. ${hopperRegexDeadline} Select detail summary for counts and retained native-inventory pages; complete remains the default.`, + `Search every analyzed procedure name, including secondary names exact lookup accepts, using literal matching by default or regex when requested. Each procedure appears once under its primary name; matched_aliases lists matching secondary names. Results are deterministic and complete. ${hopperRegexDeadline} Select detail summary for counts and retained native-inventory pages; complete remains the default.`, z.object({ ...nativeSearchInput, detail: analysisDetailSchema }), { evidenceParameters: "omit-detail" }, ), diff --git a/src/contracts/sessionStatusContract.ts b/src/contracts/sessionStatusContract.ts index e7c9bcbe44..20d188fca6 100644 --- a/src/contracts/sessionStatusContract.ts +++ b/src/contracts/sessionStatusContract.ts @@ -10,6 +10,7 @@ export type BinarySessionDetail = (typeof BINARY_SESSION_DETAILS)[number]; * available from the same session with detail: "complete". */ export const BINARY_SESSION_SUMMARY_OMITTED_FIELDS = [ + "analysis_provider_candidates.availability.remediation", "analysis_provider_candidates.availability.diagnostics", "analysis_provider_candidates.target_support.diagnostics", "analysis_provider_candidates.capabilities", @@ -24,7 +25,7 @@ export const binarySessionInputSchema = z.strictObject({ .enum(BINARY_SESSION_DETAILS) .default("summary") .describe( - "summary (default) keeps open state, the binding, current-route capabilities, each provider candidate's availability and target-support status, code and reason, and every tool's availability, reason and remediation; omitted_fields lists what it leaves out. complete adds provider diagnostics, candidate capabilities and static tool surface, effect and annotation flags.", + "summary (default) keeps open state, the binding, current-route capabilities, each provider candidate's availability and target-support status, code and reason, and every tool's availability, reason and remediation; omitted_fields lists what it leaves out. complete adds each unavailable candidate's remediation, provider diagnostics, candidate capabilities and static tool surface, effect and annotation flags.", ), expected_package_version: z.string().min(1).optional(), expected_catalog_digest: z diff --git a/src/contracts/sessionToolContracts.ts b/src/contracts/sessionToolContracts.ts index dc245ecaed..87af0a7368 100644 --- a/src/contracts/sessionToolContracts.ts +++ b/src/contracts/sessionToolContracts.ts @@ -173,7 +173,7 @@ export const SESSION_TOOL_CONTRACTS = [ ), session( "get_evidence_bundle", - "Return every retained Evidence record and residual unknown as a complete canonical bundle by default. detail=summary discovers retained records without copying or transferring their payloads; optionally filter by exact Evidence ID, operation, target SHA-256, analysis-profile digest or canonical native procedure address. Metadata distinguishes complete record retention from native value-flow truncation or unavailable/unknown coverage. Use a discovered evidence_id with inspect_analysis_view without restarting a provider. Filters apply only to summary delivery.", + "Return every retained Evidence record and residual unknown as a complete canonical bundle by default. detail=summary discovers retained records without copying or transferring their payloads; optionally filter by exact Evidence ID, operation, target SHA-256, analysis-profile digest or canonical native procedure address. Metadata distinguishes complete record retention from native value-flow truncation or unavailable/unknown coverage. detail=records returns a page (offset, limit) of the matching complete records with unchanged IDs and their missing direct evidence_links; it is not an importable bundle. Use a discovered evidence_id with inspect_analysis_view without restarting a provider. Filters apply to summary and records delivery.", getEvidenceBundleInputSchema, ), { @@ -186,7 +186,7 @@ export const SESSION_TOOL_CONTRACTS = [ }, session( "inspect_address_context", - "Inspect one explicit reproducible address for its analyzed name, containing procedure, regular and inline comments, and matching bookmarks. Each unsupported facet returns a typed unavailable outcome; use xrefs, assembly, or pseudocode for those deeper views.", + "Inspect one explicit reproducible address for its analyzed name, containing procedure, regular and inline comments, and matching bookmarks. Each unsupported facet returns a typed unavailable outcome. A facet rejected for invalid input retains its canonical error and issues alongside any successful facets. Use xrefs, assembly, or pseudocode for those deeper views.", addressContextInputSchema, ), ] as const satisfies readonly ToolContract[]; diff --git a/src/contracts/sessionToolSchemas.ts b/src/contracts/sessionToolSchemas.ts index 5f0926517c..a8a0f31ada 100644 --- a/src/contracts/sessionToolSchemas.ts +++ b/src/contracts/sessionToolSchemas.ts @@ -4,13 +4,28 @@ import { absoluteLocalPathStringSchema } from "../domain/localPath.js"; import { bundleComparisonInputSchema } from "../domain/bundleComparison.js"; import { prefixedDigestSchema } from "./../domain/digests.js"; -import { evidenceRecordFiltersSchema } from "../domain/evidenceRecordSummary.js"; +import { + EVIDENCE_RECORD_SELECTION_LIMIT, + evidenceRecordFiltersSchema, +} from "../domain/evidenceRecordSummary.js"; import { residualUnknownSchema } from "../domain/residualUnknown.js"; -/** Complete retained Evidence, or selected discovery metadata without payloads. */ +/** Complete retained Evidence, selected discovery metadata, or a page of selected complete records. */ export const getEvidenceBundleInputSchema = z.strictObject({ - detail: z.enum(["complete", "summary"]).default("complete"), + detail: z.enum(["complete", "summary", "records"]).default("complete"), filters: evidenceRecordFiltersSchema.optional(), + offset: z + .int() + .nonnegative() + .optional() + .describe("records only: first matching record to return; default 0."), + limit: z + .int() + .positive() + .optional() + .describe( + `records only: maximum complete records to return; default ${String(EVIDENCE_RECORD_SELECTION_LIMIT)}.`, + ), }); /** Optional document selection for volatile navigation context. */ diff --git a/src/contracts/toolOutputSchemaGroups.ts b/src/contracts/toolOutputSchemaGroups.ts index 4b8cab98b3..669eb0a0e8 100644 --- a/src/contracts/toolOutputSchemaGroups.ts +++ b/src/contracts/toolOutputSchemaGroups.ts @@ -10,6 +10,7 @@ import { nativeFunctionMatchingSummaryViewResultSchema, nativeFunctionSummaryViewResultSchema, nativeInventorySummaryViewResultSchema, + batchDecompileSummaryViewResultSchema, nativeLoadImageSummaryViewResultSchema, artifactInventorySummaryViewResultSchema, interfaceBuilderSummaryViewResultSchema, @@ -22,6 +23,7 @@ import { byteEntropySchema } from "../domain/native/byteEntropy.js"; import { nativeFunctionMatchingSchema } from "../domain/native/functionMatching.js"; import { nativeByteReadSchema } from "../domain/native/nativeByteRead.js"; import { analysisDetailOutputSchemaOf } from "./analysisDetail.js"; +import { batchDecompileResultSchema } from "../domain/native/batchDecompile.js"; import { functionDossierSchema, functionInstructionWindowSchema, @@ -42,7 +44,10 @@ import { } from "../domain/investigationLifecycle.js"; import { jsonRecordSchema } from "../domain/jsonRecord.js"; import { byteSearchResultSchema } from "../domain/native/bytePattern.js"; -import { evidenceBundleSummarySchema } from "../domain/evidenceRecordSummary.js"; +import { + evidenceBundleSummarySchema, + evidenceRecordSelectionSchema, +} from "../domain/evidenceRecordSummary.js"; import { liveBinaryTargetSummaryVariants } from "../domain/liveBinaryTargetSummary.js"; import { nativeFunctionAnnotationsSchema } from "../domain/native/nativeFunctionAnnotations.js"; import { nativeLoadImageSchema } from "../domain/native/nativeLoadImage.js"; @@ -91,7 +96,10 @@ import { callPathResultSchema } from "../domain/callPathSchemas.js"; import { staticRuntimeCorrelationResultSchema } from "../domain/staticRuntimeCorrelation.js"; import { reconstructionVerificationResultSchema } from "../domain/reconstructionVerificationSchemas.js"; import { decompileDirectoryManifestSchema } from "./decompileDirectory.js"; -import { analysisErrorProjectionSchema } from "./errorSchemas.js"; +import { + analysisErrorProjectionSchema, + analysisInputErrorProjectionSchema, +} from "./errorSchemas.js"; import { relationshipCoverageSchema } from "../domain/relationshipCoverage.js"; import { nativeDispatchMetadataResultSchema } from "../domain/native/objcSwiftMetadata.js"; import { nativeInvestigationTraceSchema } from "../domain/native/nativeInvestigationGraph.js"; @@ -117,6 +125,7 @@ const contextFacetSchema = z.discriminatedUnion("state", [ state: z.literal("unavailable"), reason: z.string(), remediation: z.string(), + error: analysisInputErrorProjectionSchema.optional(), }), ]); const bookmarkFacetSchema = z.discriminatedUnion("state", [ @@ -128,6 +137,7 @@ const bookmarkFacetSchema = z.discriminatedUnion("state", [ state: z.literal("unavailable"), reason: z.string(), remediation: z.string(), + error: analysisInputErrorProjectionSchema.optional(), }), ]); @@ -305,23 +315,6 @@ const callPathTraceOutput = evidenceResultOf( }), ); -/** Identity facts attached to one requested batch selector. */ -const batchProcedureIdentity = z.discriminatedUnion("status", [ - z.object({ - status: z.literal("resolved"), - address: z.string().min(1).describe("Observed canonical procedure entry."), - name: z - .string() - .nullable() - .describe("Provider's entry label, or null when unavailable."), - name_error: analysisErrorProjectionSchema.optional(), - }), - z.object({ - status: z.literal("unknown"), - error: analysisErrorProjectionSchema, - }), -]); - /** Exact structured-content schemas for composed analysis workflows. */ export const enhancedOutputSchemas = { inspect_native_dispatch_metadata: evidenceResultOf( @@ -332,38 +325,9 @@ export const enhancedOutputSchemas = { get_objc_classes: objcClassDiscoveryOutput, get_objc_protocols: symbolDiscoveryOutput("protocols"), decompile_all: evidenceResultOf(decompileDirectoryManifestSchema), - batch_decompile: evidenceResultOf( - z.object({ - items: z.array( - z.discriminatedUnion("status", [ - z.object({ - address: z - .string() - .describe("Original caller-selected symbol or address."), - procedure: batchProcedureIdentity, - status: z.literal("ok"), - pseudocode: z.string().min(1), - }), - z.object({ - address: z - .string() - .describe("Original caller-selected symbol or address."), - procedure: batchProcedureIdentity, - status: z.literal("error"), - error: analysisErrorProjectionSchema, - limitations: z - .array(z.string()) - .optional() - .describe( - "Caveats of this item's own provider observation when it supplied no pseudocode, including any reason the provider reported.", - ), - }), - ]), - ), - total: z.int().min(0), - succeeded: z.int().min(0), - failed: z.int().min(0), - }), + batch_decompile: analysisDetailOutputSchemaOf( + batchDecompileResultSchema, + batchDecompileSummaryViewResultSchema, ), get_call_graph: analysisDetailOutputSchemaOf( nativeObservationSourceSchemas.get_call_graph, @@ -569,7 +533,11 @@ export const sessionOutputSchemas = { ), export_evidence_bundle: lifecycleResultOf(evidenceBundleExportReceiptSchema), get_evidence_bundle: lifecycleResultOf( - z.union([evidenceBundleSchema, evidenceBundleSummarySchema]), + z.union([ + evidenceBundleSchema, + evidenceBundleSummarySchema, + evidenceRecordSelectionSchema, + ]), ), get_navigation_context: lifecycleResultOf( z.object({ diff --git a/src/contracts/toolOutputSchemaPrimitives.ts b/src/contracts/toolOutputSchemaPrimitives.ts index fc41b2b90b..dad89d6607 100644 --- a/src/contracts/toolOutputSchemaPrimitives.ts +++ b/src/contracts/toolOutputSchemaPrimitives.ts @@ -2,6 +2,7 @@ import { procedureInfoSchema } from "../domain/native/procedureInfo.js"; import { z } from "zod"; import { investigationStatisticsSchema } from "../domain/investigationLifecycle.js"; import { utcDateTimeSchema } from "../domain/offsetDateTime.js"; +import { analysisCleanupRetrySchema } from "../domain/analysisErrorRecovery.js"; import { functionDossierSchema } from "../domain/native/functionDossier.js"; import { analysisProfileSchema } from "../domain/analysisProfile.js"; @@ -49,6 +50,9 @@ export const requireOutputSchema = < return schema; }; +/** The readiness owner's repair instruction; never the failure description. */ +const providerRemediation = z.string().min(1); + const providerCapabilityFacts = { operation: z.string(), effects: z.object({ @@ -75,6 +79,7 @@ export const providerCapability = z.discriminatedUnion("available", [ available: z.literal(false), reason: z.string().nullable(), availability_code: z.enum(PROVIDER_REJECTION_CODES).nullable(), + remediation: providerRemediation.exactOptional(), }), ]); export type ProviderCapability = z.output; @@ -108,6 +113,7 @@ const providerCapabilityGroup = z.discriminatedUnion("available", [ available: z.literal(false), reason: z.string().nullable(), availability_code: z.enum(PROVIDER_REJECTION_CODES).nullable(), + remediation: providerRemediation.exactOptional(), }), ]); @@ -133,6 +139,9 @@ export const expandProviderCapabilities = ( available: group.available, reason: group.reason, availability_code: group.availability_code, + ...("remediation" in group && group.remediation !== undefined + ? { remediation: group.remediation } + : {}), effects: Object.fromEntries( PROVIDER_EFFECT_NAMES.map((name) => [ name, @@ -255,6 +264,7 @@ const providerAvailability = z.discriminatedUnion("status", [ diagnostics: jsonObjectSchema, }), providerAvailabilityStates.unavailable.extend({ + remediation: providerRemediation, diagnostics: jsonObjectSchema, }), ]); @@ -304,7 +314,7 @@ export const analysisProviderCandidate = z.object({ capabilities: z.array(providerCapabilitySet), }); -/** Summary candidate: the same status, code, and reason without diagnostics or capabilities. */ +/** Summary candidate: the same status, code, and reason without remediation, diagnostics or capabilities. */ export const analysisProviderCandidateSummary = z.strictObject({ provider: providerIdentity, availability: z.discriminatedUnion("status", [ @@ -535,6 +545,7 @@ export const sessionProviderFacts = z .object({ investigation: investigationStatisticsSchema, cleanup_pending: z.boolean(), + cleanup_retry: analysisCleanupRetrySchema.optional(), capabilities: z.array(providerCapabilitySet), analysis_run: analysisRun, analysis_activity: analysisActivity, diff --git a/src/domain/analysisErrorBase.ts b/src/domain/analysisErrorBase.ts index d17099e6d0..cb0b73ba09 100644 --- a/src/domain/analysisErrorBase.ts +++ b/src/domain/analysisErrorBase.ts @@ -54,6 +54,27 @@ export interface AnalysisCleanupObservation { readonly resources: readonly string[]; } +/** + * How long the adapter delivering a failure keeps REA's runtime and provider + * session for its caller. An MCP connection continues: later calls reuse the + * session and can update runtime settings. A one-shot invocation, such as a + * CLI command, releases its provider session and exits before its caller can + * act; only a new invocation's launch settings or arguments apply. + */ +export type AnalysisCallerLifetime = "continuing" | "invocation"; + +/** + * Producer guidance that holds only for a continuing caller, kept apart from + * the request's outcome. The error's own message and remediation must remain + * true after an invocation has released the provider session. + */ +export interface AnalysisContinuingCallerGuidance { + /** Appended to the outcome, such as that the provider session remains usable. */ + readonly userMessage?: string; + /** Replaces the remediation, such as a same-session runtime setting update. */ + readonly remediationAction?: string; +} + /** Optional provider-neutral context that must survive typed error projection. */ export interface AnalysisErrorOptions extends ErrorOptions { /** Typed primary retained when a later lifecycle failure composes this error. */ @@ -61,6 +82,16 @@ export interface AnalysisErrorOptions extends ErrorOptions { /** Producer-owned, caller-safe facts retained through every typed projection. */ readonly diagnostics?: Readonly> | undefined; readonly limit?: AnalysisSettingLimit | undefined; + readonly continuingCallerGuidance?: + | AnalysisContinuingCallerGuidance + | undefined; + /** + * The same diagnostics recorded for an invocation caller, present only when + * an embedded child failure projects differently for that lifetime. + */ + readonly invocationDiagnostics?: + | Readonly> + | undefined; readonly capturedOutput?: AnalysisCapturedOutput | undefined; readonly cleanup?: AnalysisCleanupObservation | undefined; readonly partialObservation?: AnalysisPartialObservation | undefined; @@ -87,6 +118,12 @@ export abstract class AnalysisError extends Error { | undefined; #cleanupDependents: WeakRef[] = []; readonly diagnostics: Readonly> | undefined; + readonly continuingCallerGuidance: + | Readonly + | undefined; + readonly invocationDiagnostics: + | Readonly> + | undefined; readonly capturedOutput: AnalysisCapturedOutput | undefined; readonly cleanup: AnalysisCleanupObservation | undefined; cleanupIncomplete: boolean; @@ -103,6 +140,14 @@ export abstract class AnalysisError extends Error { options?.diagnostics === undefined ? undefined : structuredClone(options.diagnostics); + this.continuingCallerGuidance = + options?.continuingCallerGuidance === undefined + ? undefined + : Object.freeze({ ...options.continuingCallerGuidance }); + this.invocationDiagnostics = + options?.invocationDiagnostics === undefined + ? undefined + : structuredClone(options.invocationDiagnostics); this.capturedOutput = options?.capturedOutput === undefined ? undefined @@ -186,10 +231,27 @@ export abstract class AnalysisError extends Error { retainDiagnostics(diagnostics: Readonly>): this { Object.assign(this, { diagnostics: structuredClone({ ...this.diagnostics, ...diagnostics }), + ...(this.invocationDiagnostics === undefined + ? {} + : { + invocationDiagnostics: structuredClone({ + ...this.invocationDiagnostics, + ...diagnostics, + }), + }), }); return this; } + /** Diagnostics recorded for a caller with this lifetime. */ + diagnosticsFor( + callerLifetime: AnalysisCallerLifetime, + ): Readonly> | undefined { + return callerLifetime === "invocation" + ? (this.invocationDiagnostics ?? this.diagnostics) + : this.diagnostics; + } + retainSecondaryFailure( failure: AnalysisError, diagnostics?: Readonly>, diff --git a/src/domain/analysisErrorCleanup.ts b/src/domain/analysisErrorCleanup.ts index 0f1f74fb7d..81aa84dcdc 100644 --- a/src/domain/analysisErrorCleanup.ts +++ b/src/domain/analysisErrorCleanup.ts @@ -1,19 +1,18 @@ import { ProviderAdapterError } from "./providerAdapterError.js"; -import { - projectAnalysisError, - projectAnalysisErrorWithoutCapturedOutput, -} from "./analysisErrorProjection.js"; +import { childFailureDiagnostics } from "./analysisErrorProjection.js"; import type { AnalysisError } from "./analysisErrorBase.js"; -import { analysisErrorRemediationAction } from "./analysisErrorPresentation.js"; +import { + analysisErrorUserMessage, + composedCallerGuidance, +} from "./analysisErrorPresentation.js"; /** Preserve the original lifecycle failure alongside incomplete cleanup. */ export const analysisErrorWithCleanupFailure = ( primary: AnalysisError, cleanup: AnalysisError, operation = "close_binary", -): AnalysisError => { - const projected = projectAnalysisError(cleanup); - return new ProviderAdapterError( +): AnalysisError => + new ProviderAdapterError( cleanup instanceof ProviderAdapterError ? cleanup.providerId : "rea", operation, { @@ -23,9 +22,8 @@ export const analysisErrorWithCleanupFailure = ( "Analysis failed and cleanup remains incomplete.", { cause: primary }, ), - userMessage: projectAnalysisError(primary).message, // A retry offer belongs to the owner of the composed resources, if any. - remediationAction: analysisErrorRemediationAction(cleanup), + ...composedCallerGuidance({ message: primary, remediation: cleanup }), ...(primary.capturedOutput === undefined ? {} : { capturedOutput: primary.capturedOutput }), @@ -33,7 +31,9 @@ export const analysisErrorWithCleanupFailure = ( ? {} : { partialObservation: primary.partialObservation }), cleanup: { - reason: cleanup.cleanup?.reason ?? projected.message, + reason: + cleanup.cleanup?.reason ?? + analysisErrorUserMessage(cleanup, "invocation"), resources: [ ...new Set([ ...(primary.cleanupIncomplete ? primary.cleanupResources : []), @@ -41,13 +41,14 @@ export const analysisErrorWithCleanupFailure = ( ]), ], }, - diagnostics: { - primary_error: - primary.capturedOutput === undefined - ? projectAnalysisError(primary) - : projectAnalysisErrorWithoutCapturedOutput(primary), - cleanup_error: projected, - }, + ...childFailureDiagnostics({ + primary_error: { + error: primary, + ...(primary.capturedOutput === undefined + ? {} + : { capturedOutput: "omit" as const }), + }, + cleanup_error: { error: cleanup }, + }), }, ); -}; diff --git a/src/domain/analysisErrorPresentation.ts b/src/domain/analysisErrorPresentation.ts index 026b937323..76a4e0ff40 100644 --- a/src/domain/analysisErrorPresentation.ts +++ b/src/domain/analysisErrorPresentation.ts @@ -25,6 +25,7 @@ import { ResourceSnapshotMismatchError, } from "./evidenceErrors.js"; import { + HOPPER_STARTUP_TIMEOUT_REMEDIATION, HopperRemoteError, HopperProcessError, HopperStartError, @@ -38,6 +39,8 @@ import { fixedAllowanceRemediation } from "./fixedAllowanceRemediation.js"; import { primaryAnalysisError, resolvedAnalysisError, + type AnalysisCallerLifetime, + type AnalysisContinuingCallerGuidance, type AnalysisError, type AnalysisErrorTag, } from "./analysisErrorBase.js"; @@ -48,12 +51,22 @@ export const MCP_TRANSPORT_EVIDENCE_EXPORT = "Export complete retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. The connection remains usable."; /** Next step when one MCP frame cannot carry a complete retained session bundle. */ -export const MCP_TRANSPORT_EVIDENCE_DISCOVERY = `Discover retained records with get_evidence_bundle and detail: summary, then inspect compatible records through inspect_analysis_view using an exact Evidence reference. ${MCP_TRANSPORT_EVIDENCE_EXPORT}`; +export const MCP_TRANSPORT_EVIDENCE_DISCOVERY = `Discover retained records with get_evidence_bundle and detail: summary, then read selected complete records with detail: records and an exact evidence_id, or inspect compatible records through inspect_analysis_view using an exact Evidence reference. ${MCP_TRANSPORT_EVIDENCE_EXPORT}`; +/** Next step when one MCP frame cannot carry a selected page of complete records. */ +export const MCP_TRANSPORT_EVIDENCE_RECORDS = `Request fewer records with get_evidence_bundle detail: records, a smaller limit or an exact evidence_id filter; inspect compatible records in pages through inspect_analysis_view. A single record larger than one response needs export. ${MCP_TRANSPORT_EVIDENCE_EXPORT}`; + +/** Name the next step that is true for how long the caller keeps REA's runtime and session. */ export const analysisErrorRemediationAction = ( error: AnalysisError, + callerLifetime: AnalysisCallerLifetime = "continuing", ): string => { error = resolvedAnalysisError(error); + const continuing = + callerLifetime === "continuing" + ? error.continuingCallerGuidance?.remediationAction + : undefined; + if (continuing !== undefined) return continuing; if ( (error instanceof ProviderAdapterError || error instanceof AnalysisCapabilityUnavailableError) && @@ -86,7 +99,7 @@ export const analysisErrorRemediationAction = ( ? "Call close_binary without snapshot_path to close this resource-only session." : "Open an executable analysis target before importing a binary analysis snapshot."; if (error instanceof AnalysisSnapshotMismatchError) - return "Run analysis without this snapshot, then save a fresh snapshot using the intended artifact, provider, and analysis profile."; + return "Select an artifact, provider, and analysis profile matching the snapshot, or run analysis without it and save a fresh snapshot for the intended binding."; if (error instanceof AnalysisUnsupportedTargetError) return ( error.remediationAction ?? @@ -124,7 +137,7 @@ export const analysisErrorRemediationAction = ( return "Use an artifact within the capacity reported in details.detail. Changing output detail or retrying the unchanged input does not remove this boundary."; if (error instanceof HopperTimeoutError) return error.operation === undefined - ? "Review details.launcher and inspect Hopper for ongoing initial analysis or a pending loader, demo, or license dialog. For a large target, use update_analysis_settings with setting hopper.startup_timeout_ms and action set for the next startup attempt without restarting REA, or increase REA_HOPPER_STARTUP_TIMEOUT_MS for future REA processes. Allow at least that long in the MCP client. Resolve retained cleanup before reopening." + ? HOPPER_STARTUP_TIMEOUT_REMEDIATION : error.providerState === "busy" ? "Check binary_session.analysis_activity, wait for the active Hopper request to finish, then retry." : "Check binary_session for Hopper health, then retry the operation."; @@ -361,8 +374,68 @@ const STATIC_ERROR_CATEGORIES: Readonly< BinaryTargetError: "unavailable", }; -export const analysisErrorUserMessage = (error: AnalysisError): string => { +/** Describe the request's outcome, adding continuing-caller facts only for that caller. */ +export const analysisErrorUserMessage = ( + error: AnalysisError, + callerLifetime: AnalysisCallerLifetime = "continuing", +): string => { + const { outcome, continuing } = analysisErrorUserMessageParts(error); + return callerLifetime === "invocation" || continuing === undefined + ? outcome + : `${outcome} ${continuing}`; +}; + +/** The outcome every caller receives and the addition only a continuing caller does. */ +const analysisErrorUserMessageParts = ( + error: AnalysisError, +): { readonly outcome: string; readonly continuing?: string } => { error = resolvedAnalysisError(error); + const outcome = outcomeUserMessage(error); + const continuing = error.continuingCallerGuidance?.userMessage; + return continuing === undefined ? { outcome } : { outcome, continuing }; +}; + +/** + * Re-attribute one failure's message and another's remediation to a composed + * error, keeping the form each caller lifetime receives. + */ +export const composedCallerGuidance = (sources: { + readonly message?: AnalysisError; + readonly remediation: AnalysisError; +}): { + readonly userMessage?: string; + readonly remediationAction: string; + readonly continuingCallerGuidance?: AnalysisContinuingCallerGuidance; +} => { + const message = + sources.message === undefined + ? undefined + : analysisErrorUserMessageParts(sources.message); + const remediationAction = analysisErrorRemediationAction( + sources.remediation, + "invocation", + ); + const continuingRemediation = analysisErrorRemediationAction( + sources.remediation, + ); + const continuingCallerGuidance = { + ...(message?.continuing === undefined + ? {} + : { userMessage: message.continuing }), + ...(continuingRemediation === remediationAction + ? {} + : { remediationAction: continuingRemediation }), + }; + return { + ...(message === undefined ? {} : { userMessage: message.outcome }), + remediationAction, + ...(Object.keys(continuingCallerGuidance).length === 0 + ? {} + : { continuingCallerGuidance }), + }; +}; + +const outcomeUserMessage = (error: AnalysisError): string => { if (error instanceof AnalysisSnapshotMismatchError) return error.message; if (error instanceof AnalysisUnsupportedTargetError) return error.message; if (error instanceof AnalysisResourceConstraintError) return error.reason; diff --git a/src/domain/analysisErrorProjection.test.ts b/src/domain/analysisErrorProjection.test.ts index a88e632aae..921845a980 100644 --- a/src/domain/analysisErrorProjection.test.ts +++ b/src/domain/analysisErrorProjection.test.ts @@ -16,6 +16,8 @@ import { import { BrowserObservationError } from "./browserObservationError.js"; import { EvidenceFileError, EvidenceIntegrityError } from "./evidenceErrors.js"; import { + HOPPER_STARTUP_TIMEOUT_CONTINUING_REMEDIATION, + HOPPER_STARTUP_TIMEOUT_REMEDIATION, HopperProcessError, HopperRemoteError, HopperStartError, @@ -39,6 +41,12 @@ import { import { ProviderSelectionError } from "./providerSelectionError.js"; import { analysisErrorProjectionSchema } from "../contracts/errorSchemas.js"; import { nativeCallPartialObservationSchema } from "./native/nativeCallPartialObservation.js"; +import { analysisSettingLimit } from "./analysisSettingDefinitions.js"; +import { redactExplicitFailure } from "./explicitSensitiveFailure.js"; +import { redactExplicitText } from "./explicitSensitiveValues.js"; +import { analysisErrorWithCleanupFailure } from "./analysisErrorCleanup.js"; +import type { AnalysisCallerLifetime } from "./analysisErrorBase.js"; +import { ProviderCleanupError } from "./providerCleanupError.js"; describe("agent recovery metadata", () => { it("does not mark deterministic input validation as retryable", () => { @@ -1331,3 +1339,232 @@ it("preserves ordinary configuration recovery and projects a producer-owned sess }); expect(repairing.code).toBe("configuration_invalid"); }); + +const continuingOnly = { + userMessage: "The fixture session remains usable.", + remediationAction: "Update the fixture runtime setting on this session.", +}; + +describe("analysis error projection: caller lifetime", () => { + it.each([ + new AnalysisTimeoutError("procedure_pseudo_code", 5000, { + userMessage: "Only this request stopped.", + remediationAction: "Rerun with a longer launch deadline.", + continuingCallerGuidance: continuingOnly, + }), + new ProviderAdapterError("fixture", "procedure_pseudo_code", { + userMessage: "Only this request stopped.", + remediationAction: "Rerun with a longer launch deadline.", + continuingCallerGuidance: continuingOnly, + }), + new AnalysisResourceConstraintError( + "search_strings", + "cpu", + "Only this request stopped.", + null, + { + remediationAction: "Rerun with a longer launch deadline.", + continuingCallerGuidance: continuingOnly, + }, + ), + ])( + "states continuing-session guidance only to a continuing caller ($_tag)", + (error) => { + expect(projectAnalysisError(error)).toMatchObject({ + message: + "Only this request stopped. The fixture session remains usable.", + remediation: { + action: "Update the fixture runtime setting on this session.", + }, + }); + const invocation = projectAnalysisError(error, "invocation"); + expect(invocation).toMatchObject({ + code: projectAnalysisError(error).code, + message: "Only this request stopped.", + remediation: { action: "Rerun with a longer launch deadline." }, + }); + expect(analysisErrorUserMessage(error, "invocation")).toBe( + "Only this request stopped.", + ); + expect(analysisErrorRemediationAction(error, "invocation")).toBe( + "Rerun with a longer launch deadline.", + ); + }, + ); + + it("re-scopes a limit for a finished invocation and keeps its enforced facts", () => { + const limit = analysisSettingLimit( + "regex.matching_deadline_ms", + 250, + "setting", + ); + const error = new AnalysisResourceConstraintError( + "search_strings", + "cpu", + "Regex matching exceeded its deadline.", + null, + { limit }, + ); + expect(projectAnalysisError(error).details?.limit).toEqual(limit); + expect(projectAnalysisError(error, "invocation").details?.limit).toEqual({ + setting: "regex.matching_deadline_ms", + effective_value: 250, + source: "setting", + unit: "ms", + minimum: limit.minimum, + maximum: limit.maximum, + agent_editable: true, + rea_restart_required: false, + applies_at: "next_invocation", + scope: "rea-invocation", + environment: "REA_REGEX_MATCHING_DEADLINE_MS", + }); + // The error keeps its runtime facts for other projections. + expect(error.limit).toEqual(limit); + }); + + it("names a per-call parameter, not an environment, when a setting has no launch seed", () => { + const error = new AnalysisTimeoutError("inspect_wasm_artifact", 100, { + limit: analysisSettingLimit("wasm.command_timeout_ms", 100, "request"), + }); + expect(projectAnalysisError(error, "invocation").details?.limit).toEqual( + expect.objectContaining({ + setting: "wasm.command_timeout_ms", + source: "request", + parameter: "command_timeout_ms", + applies_at: "next_invocation", + scope: "rea-invocation", + }), + ); + expect( + projectAnalysisError(error, "invocation").details?.limit, + ).not.toHaveProperty("environment"); + }); + + it("delivers each lifetime's Hopper startup recovery", () => { + const error = new HopperTimeoutError( + 30_000, + undefined, + undefined, + "not_started", + "launch", + undefined, + { + limit: analysisSettingLimit( + "hopper.startup_timeout_ms", + 30_000, + undefined, + ), + }, + ); + expect(projectAnalysisError(error).remediation.action).toBe( + HOPPER_STARTUP_TIMEOUT_CONTINUING_REMEDIATION, + ); + const invocation = projectAnalysisError(error, "invocation"); + expect(invocation.remediation.action).toBe( + HOPPER_STARTUP_TIMEOUT_REMEDIATION, + ); + expect(invocation.details?.limit).toMatchObject({ + setting: "hopper.startup_timeout_ms", + effective_value: 30_000, + scope: "rea-invocation", + applies_at: "next_invocation", + environment: "REA_HOPPER_STARTUP_TIMEOUT_MS", + }); + // A busy request is not a startup: it carries no lifetime-specific text. + const busy = new HopperTimeoutError(30_000, "trace_feature", 42, "busy"); + expect(projectAnalysisError(busy, "invocation").remediation).toEqual( + projectAnalysisError(busy).remediation, + ); + }); +}); + +describe("analysis error projection: composed failures for a caller lifetime", () => { + it("projects a composed cleanup failure's children for the caller and keeps their history", () => { + const limit = analysisSettingLimit( + "regex.matching_deadline_ms", + 250, + "setting", + ); + const primary = new AnalysisTimeoutError("search_strings", 250, { + limit, + userMessage: "Only this request stopped.", + remediationAction: "Rerun with a longer launch deadline.", + continuingCallerGuidance: continuingOnly, + }); + const cleanup = new ProviderCleanupError( + "fixture", + ["fixture-process"], + { reason: "exit unconfirmed" }, + { remediationAction: "Close the retained fixture process." }, + ); + const composed = analysisErrorWithCleanupFailure( + primary, + cleanup, + ).bindCleanupDependencies([primary, cleanup]); + const child = (lifetime: AnalysisCallerLifetime) => + projectAnalysisError(composed, lifetime).details; + expect(child("invocation")).toMatchObject({ + limit: { scope: "rea-invocation" }, + diagnostics: { + primary_error: projectAnalysisError(primary, "invocation"), + cleanup_error: { code: "cleanup_incomplete" }, + }, + }); + expect(child("continuing")).toMatchObject({ + limit: { scope: "rea-runtime" }, + diagnostics: { primary_error: projectAnalysisError(primary) }, + }); + expect(projectAnalysisError(composed, "invocation").message).toBe( + "Only this request stopped.", + ); + expect(projectAnalysisError(composed).message).toBe( + `Only this request stopped. ${continuingOnly.userMessage}`, + ); + // After cleanup resolves, history keeps the composition-time snapshot. + cleanup.resolveCleanup(); + for (const lifetime of ["continuing", "invocation"] as const) + expect(child(lifetime)?.cleanup_history).toMatchObject({ + primary_error: projectAnalysisError(primary, lifetime), + cleanup_error: { code: "cleanup_incomplete" }, + }); + }); + + it("records one lifetime form when no child depends on the caller", () => { + const composed = analysisErrorWithCleanupFailure( + new AnalysisInputError("read_bytes", undefined, [ + { path: ["address"], reason: "invalid_format" }, + ]), + new ProviderCleanupError("fixture", ["fixture-process"], {}), + ); + expect(composed.invocationDiagnostics).toBeUndefined(); + expect(projectAnalysisError(composed, "invocation").details).toEqual( + projectAnalysisError(composed).details, + ); + }); + + it("keeps continuing guidance through redaction and redacts its text", () => { + const guidance = { + userMessage: "The session for secret-token remains usable.", + remediationAction: "Retry secret-token on this session.", + }; + const error = new AnalysisTimeoutError("procedure_pseudo_code", 5000, { + userMessage: "Only this request stopped.", + remediationAction: "Rerun with a longer launch deadline.", + continuingCallerGuidance: guidance, + }); + const redact = (text: string) => redactExplicitText(text, ["secret-token"]); + const redacted = redactExplicitFailure(error, ["secret-token"]); + expect(projectAnalysisError(redacted)).toMatchObject({ + message: `Only this request stopped. ${redact(guidance.userMessage)}`, + remediation: { action: redact(guidance.remediationAction) }, + }); + expect(JSON.stringify(projectAnalysisError(redacted))).not.toContain( + "secret-token", + ); + expect(projectAnalysisError(redacted, "invocation")).toMatchObject({ + message: "Only this request stopped.", + remediation: { action: "Rerun with a longer launch deadline." }, + }); + }); +}); diff --git a/src/domain/analysisErrorProjection.ts b/src/domain/analysisErrorProjection.ts index 6d22799f5a..9211830bdf 100644 --- a/src/domain/analysisErrorProjection.ts +++ b/src/domain/analysisErrorProjection.ts @@ -31,6 +31,7 @@ import { } from "./configurationErrors.js"; import { BrowserObservationError } from "./browserObservationError.js"; import { + AnalysisSnapshotMismatchError, EvidenceFileError, EvidenceIntegrityError, EvidenceReferenceError, @@ -49,23 +50,94 @@ import { providerRetryAction } from "./providerOperationHealth.js"; import { primaryAnalysisError, resolvedAnalysisError, + type AnalysisCallerLifetime, type AnalysisError, + type AnalysisErrorOptions, type AnalysisErrorTag, } from "./analysisErrorBase.js"; +import { invocationSettingLimit } from "./analysisSettingDefinitions.js"; -/** Project expected failures into exhaustive, secret-safe caller fields. */ +/** + * Project expected failures into exhaustive, secret-safe caller fields. The + * default describes a continuing caller, such as an MCP connection, and is + * also the adapter-independent form retained inside Evidence. An adapter that + * releases its provider session and runtime after this request, such as a + * one-shot CLI command, passes `"invocation"`. + */ export const projectAnalysisError = ( error: AnalysisError, -): AnalysisErrorProjection => projectError(error, true); + callerLifetime: AnalysisCallerLifetime = "continuing", +): AnalysisErrorProjection => projectError(error, true, callerLifetime); /** Cleanup history references output retained by its enclosing error owner. */ export const projectAnalysisErrorWithoutCapturedOutput = ( error: AnalysisError, -): AnalysisErrorProjection => projectError(error, false); + callerLifetime: AnalysisCallerLifetime = "continuing", +): AnalysisErrorProjection => projectError(error, false, callerLifetime); + +/** A child failure embedded in a composed error's diagnostics. */ +export interface AnalysisChildFailure { + readonly error: AnalysisError; + /** Omit output that the composing error already retains. */ + readonly capturedOutput?: "omit"; +} + +/** + * Record child failures as composition-time snapshots, so cleanup history + * keeps the state it had. The invocation form is recorded only when a child's + * projection depends on the caller lifetime; the final projection selects it. + */ +export const childFailureDiagnostics = ( + children: Readonly>, + diagnostics: Readonly> = {}, +): Pick => { + const entries = Object.entries(children); + const project = ( + { error, capturedOutput }: AnalysisChildFailure, + callerLifetime: AnalysisCallerLifetime, + ) => projectError(error, capturedOutput !== "omit", callerLifetime); + const continuing = { + ...diagnostics, + ...Object.fromEntries( + entries.map(([key, child]) => [key, project(child, "continuing")]), + ), + }; + const varying = entries.filter(([, { error }]) => + projectionDependsOnCallerLifetime(error), + ); + return varying.length === 0 + ? { diagnostics: continuing } + : { + diagnostics: continuing, + invocationDiagnostics: { + ...continuing, + ...Object.fromEntries( + varying.map(([key, child]) => [key, project(child, "invocation")]), + ), + }, + }; +}; + +/** Every lifetime-dependent input the projection reads lies on this chain. */ +const projectionDependsOnCallerLifetime = (error: AnalysisError): boolean => { + for ( + let node: AnalysisError | undefined = error; + node !== undefined; + node = node.cleanupPrimary + ) + if ( + node.limit !== undefined || + node.continuingCallerGuidance !== undefined || + node.invocationDiagnostics !== undefined + ) + return true; + return false; +}; const projectError = ( error: AnalysisError, includeCapturedOutput: boolean, + callerLifetime: AnalysisCallerLifetime, ): AnalysisErrorProjection => { const composed = error; error = resolvedAnalysisError(error); @@ -75,7 +147,9 @@ const projectError = ( diagnosticSource.cleanupPrimary !== undefined ) diagnosticSource = diagnosticSource.cleanupPrimary; - const diagnostics = diagnosticSource.diagnostics; + const diagnostics = diagnosticSource.diagnosticsFor(callerLifetime); + const cleanupHistory = + composed === error ? undefined : composed.diagnosticsFor(callerLifetime); const capturedOutput = composed.capturedOutput ?? error.capturedOutput; const partialObservation = composed.partialObservation ?? error.partialObservation; @@ -96,7 +170,14 @@ const projectError = ( const primaryDetails = errorDetails(primary); const details = { ...primaryDetails, - ...(primary.limit === undefined ? {} : { limit: { ...primary.limit } }), + ...(primary.limit === undefined + ? {} + : { + limit: + callerLifetime === "invocation" + ? { ...invocationSettingLimit(primary.limit) } + : { ...primary.limit }, + }), ...(cleanupReport === undefined ? {} : { cleanup_report: jsonValueSchema.parse(cleanupReport) }), @@ -108,9 +189,9 @@ const projectError = ( ? {} : { provider_diagnostics: primaryDetails.diagnostics }), }), - ...(composed === error || composed.diagnostics === undefined + ...(cleanupHistory === undefined ? {} - : { cleanup_history: composed.diagnostics }), + : { cleanup_history: cleanupHistory }), ...(error.cleanupIncomplete ? { cleanup: "incomplete", @@ -143,11 +224,11 @@ const projectError = ( cleanupRetry === undefined ? errorRecovery(error) : cleanupRetryRecovery(cleanupRetry); - const remediation = analysisErrorRemediationAction(error); + const remediation = analysisErrorRemediationAction(error, callerLifetime); return { code, category: analysisErrorCategory(error), - message: analysisErrorUserMessage(error), + message: analysisErrorUserMessage(error, callerLifetime), retryable: RETRYABLE_CODES.has(code) || (error instanceof AnalysisCapabilityUnavailableError && @@ -397,6 +478,13 @@ const requestErrorDetails = ( expected: error.expected, actual: error.actual, }; + if (error instanceof AnalysisSnapshotMismatchError) + return { + reason: error.message, + ...(error.bindingMismatch === undefined + ? {} + : { binding_mismatch: jsonValueSchema.parse(error.bindingMismatch) }), + }; if (error instanceof EvidenceIntegrityError) return { reason: error.message }; return undefined; }; @@ -479,6 +567,9 @@ const providerErrorDetails = ( origin: rejection.origin, code: rejection.code, reason: rejection.reason, + ...(rejection.remediation === undefined + ? {} + : { remediation: rejection.remediation }), diagnostics: rejection.diagnostics, })), }; diff --git a/src/domain/analysisErrorRecovery.ts b/src/domain/analysisErrorRecovery.ts index 419759a15d..301863363a 100644 --- a/src/domain/analysisErrorRecovery.ts +++ b/src/domain/analysisErrorRecovery.ts @@ -31,6 +31,11 @@ export interface AnalysisCleanupRetry { readonly operation: "close_binary"; } +/** The retained owner's same-connection retry, also exposed in session status. */ +export const analysisCleanupRetrySchema = z + .object({ operation: z.literal("close_binary") }) + .strict(); + export const cleanupRetryRecovery = ( retry: AnalysisCleanupRetry, ): AnalysisErrorRecovery => ({ diff --git a/src/domain/analysisSearch.ts b/src/domain/analysisSearch.ts index 3937feaf3d..a41e370501 100644 --- a/src/domain/analysisSearch.ts +++ b/src/domain/analysisSearch.ts @@ -43,3 +43,24 @@ export const analysisSearchPredicate = ( } return literalSubstringPredicate(query.pattern, query.case_sensitive); }; + +/** + * Match one procedure by its primary name and the secondary names its provider + * accepts for exact selection. A procedure remains one row whose value is its + * primary name; matched_aliases keeps each distinct matching secondary + * spelling in provider order, so an alias never becomes a procedure of its own. + */ +export const matchProcedureNames = ( + row: Row, + aliases: readonly string[], + matches: (value: string) => boolean, +): (Row & { readonly matched_aliases?: string[] }) | undefined => { + const matched = + aliases.length === 0 + ? [] + : [...new Set(aliases)].filter( + (alias) => alias !== row.value && matches(alias), + ); + if (matched.length > 0) return { ...row, matched_aliases: matched }; + return matches(row.value) ? row : undefined; +}; diff --git a/src/domain/analysisSettingDefinitions.ts b/src/domain/analysisSettingDefinitions.ts index 95712f7cbe..1dd1feef6c 100644 --- a/src/domain/analysisSettingDefinitions.ts +++ b/src/domain/analysisSettingDefinitions.ts @@ -1,5 +1,9 @@ import { z } from "zod"; -import type { AnalysisSettingId } from "./analysisSettings.js"; +import type { + AnalysisInvocationSettingLimit, + AnalysisSettingId, + AnalysisSettingLimit, +} from "./analysisSettings.js"; import { DEFAULT_ASAR_MAX_HEADER_BYTES, MAXIMUM_ASAR_MAX_HEADER_BYTES, @@ -225,8 +229,8 @@ export const analysisSettingValueSchema = (id: AnalysisSettingId) => { export const analysisSettingLimit = ( setting: AnalysisSettingId, effective_value: number | null, - source: import("./analysisSettings.js").AnalysisSettingLimit["source"], -): import("./analysisSettings.js").AnalysisSettingLimit => { + source: AnalysisSettingLimit["source"], +): AnalysisSettingLimit => { const definition = analysisSettingDefinition(setting); return { setting, @@ -244,3 +248,18 @@ export const analysisSettingLimit = ( : { parameter: definition.parameter }), }; }; + +/** Re-scope an enforced limit for a caller whose runtime ended with this invocation. */ +export const invocationSettingLimit = ({ + applies_at: _appliesAt, + scope: _scope, + ...facts +}: AnalysisSettingLimit): AnalysisInvocationSettingLimit => { + const environment = analysisSettingDefinition(facts.setting).environment; + return { + ...facts, + applies_at: "next_invocation", + scope: "rea-invocation", + ...(environment === undefined ? {} : { environment }), + }; +}; diff --git a/src/domain/analysisSettings.ts b/src/domain/analysisSettings.ts index 0b5e438c02..03476d1b46 100644 --- a/src/domain/analysisSettings.ts +++ b/src/domain/analysisSettings.ts @@ -36,3 +36,19 @@ export interface AnalysisSettingLimit { readonly required_value?: number; readonly parameter?: string; } + +/** + * The same limit for a caller whose REA runtime ended with one invocation. No + * runtime override can reach it; a new invocation reads the launch + * environment seed or the per-call argument. The caller can still choose that + * value, and no running REA process needs a restart. + */ +export interface AnalysisInvocationSettingLimit extends Omit< + AnalysisSettingLimit, + "applies_at" | "scope" +> { + readonly applies_at: "next_invocation"; + readonly scope: "rea-invocation"; + /** Launch-environment seed that a new invocation reads, when one exists. */ + readonly environment?: string; +} diff --git a/src/domain/analysisSnapshot.ts b/src/domain/analysisSnapshot.ts index 1dcea4e47a..7aba9dd72f 100644 --- a/src/domain/analysisSnapshot.ts +++ b/src/domain/analysisSnapshot.ts @@ -457,14 +457,72 @@ export const snapshotMatchesProfile = ( committedProvidersEqual(binding.provider, profile.provider) && analysisProfilesEqual(binding.analysis_profile, profile); -/** Require an exact target and selected provider/profile cache partition. */ -export const snapshotMatchesBinding = ( +/** Confirmed differences in the same target and profile commitments used for replay. */ +export interface SnapshotBindingMismatch { + readonly target?: { + readonly snapshot: AnalysisSnapshotTarget; + readonly selected: AnalysisSnapshotTarget; + }; + readonly provider?: { + readonly snapshot: AnalysisSnapshotBinding["provider"]; + readonly selected: AnalysisProfileCommitment["provider"]; + }; + readonly profile_digest?: { + readonly snapshot: string; + readonly selected: string; + }; + /** A selected profile was not available at this validation boundary. */ + readonly profile_comparison?: "not_compared"; + /** A matching profileless binding contains workflows outside its permitted scope. */ + readonly workflow_scope?: "incompatible"; + /** Staged internal state has no complete binding to compare. */ + readonly staged_binding?: "incomplete"; +} + +/** Explain only comparisons made against authenticated binding facts. */ +export const snapshotBindingMismatch = ( snapshot: Pick, target: BinaryTarget, - profile: AnalysisProfileCommitment, -): boolean => - snapshotMatchesTarget(snapshot.target, target) && - snapshotMatchesProfile(snapshot.binding, profile); + profile: AnalysisProfileCommitment | undefined, +): SnapshotBindingMismatch | undefined => { + const targetMatches = snapshotMatchesTarget(snapshot.target, target); + const profileMatches = + profile !== undefined && snapshotMatchesProfile(snapshot.binding, profile); + if (targetMatches && profileMatches) return undefined; + return { + ...(targetMatches + ? {} + : { + target: { + snapshot: { ...snapshot.target }, + selected: snapshotTarget(target), + }, + }), + ...(profile === undefined + ? { profile_comparison: "not_compared" as const } + : { + ...(committedProvidersEqual( + snapshot.binding.provider, + profile.provider, + ) + ? {} + : { + provider: { + snapshot: { ...snapshot.binding.provider }, + selected: { ...profile.provider }, + }, + }), + ...(profileMatches + ? {} + : { + profile_digest: { + snapshot: snapshot.binding.analysis_profile.digest, + selected: profile.digest, + }, + }), + }), + }; +}; /** Find exact persisted CLI Evidence without starting an analysis provider. */ export const snapshotEvidenceForQuery = ( @@ -486,7 +544,7 @@ export const snapshotEvidenceForQuery = ( provider, evidenceProfile, } = query; - if (!snapshotMatchesBinding(snapshot, target, bindingProfile)) + if (snapshotBindingMismatch(snapshot, target, bindingProfile) !== undefined) return undefined; const encodedParameters = canonicalJson(parameters); const queryId = analysisQueryId( diff --git a/src/domain/analysisView/analysisView.test.ts b/src/domain/analysisView/analysisView.test.ts index a26e5658fd..a21fa68919 100644 --- a/src/domain/analysisView/analysisView.test.ts +++ b/src/domain/analysisView/analysisView.test.ts @@ -38,6 +38,7 @@ import { const layoutParent = (): AnalysisViewParent => { const evidence = analysisViewLayoutEvidence(); return { + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, normalizedResult: evidence.normalized_result, @@ -48,6 +49,7 @@ const layoutParent = (): AnalysisViewParent => { const javascriptParent = ( analysis = analysisViewJavaScriptAnalysis(), ): AnalysisViewParent => ({ + predicateType: JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.predicate_type, evidenceId: JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.evidence_id, operation: "analyze_javascript_application", normalizedResult: analysis, @@ -57,6 +59,7 @@ const javascriptParent = ( const nativeParent = ( result: unknown = ghidraFunctionDossier(), ): AnalysisViewParent => ({ + predicateType: "rea.analysis", evidenceId: "ev_" + "b".repeat(64), operation: "analyze_function", normalizedResult: result, @@ -363,6 +366,7 @@ it("rejects artifact summary views when inspection substep coverage count disagr const source = ARTIFACT_COMPARISON_EXAMPLE.left; const inspection = createArtifactInspection(source); const parent: AnalysisViewParent = { + predicateType: "rea.analysis", evidenceId: `ev_${"a".repeat(64)}`, operation: "inspect_artifact", normalizedResult: { @@ -1577,6 +1581,7 @@ it("rejects unsupported parent operations and incompatible collections", () => { ).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } }); const unsupported = projectAnalysisView( { + predicateType: "rea.analysis", evidenceId: `ev_${"d".repeat(64)}`, operation: "inspect_recorded_crash", normalizedResult: { path: "/artifacts/crash.core" }, diff --git a/src/domain/analysisView/analysisView.ts b/src/domain/analysisView/analysisView.ts index 7b278e1184..91937ee723 100644 --- a/src/domain/analysisView/analysisView.ts +++ b/src/domain/analysisView/analysisView.ts @@ -1,4 +1,7 @@ -import { NATIVE_OBSERVATION_OPERATIONS } from "../native/nativeObservation.js"; +import { + isNativeObservationOperation, + NATIVE_OBSERVATION_OPERATIONS, +} from "../native/nativeObservation.js"; import { projectNativeObservationView } from "./nativeObservationView.js"; import { projectDevicetreeView } from "./devicetreeView.js"; import { projectApkSignatureView } from "./apkSignatureView.js"; @@ -14,6 +17,8 @@ import { import { projectJavaScriptRoutesView } from "./javascriptRoutesView.js"; import { projectWebBundleView } from "./webBundleView.js"; import { projectSourceToBundleView } from "./sourceToBundleView.js"; +import { projectFeatureTraceView } from "./featureTraceView.js"; +import { projectBatchDecompileView } from "./batchDecompileView.js"; import { projectGpuCodeObjectsView } from "./gpuCodeObjectsView.js"; import { projectModelArtifactView } from "./modelArtifactView.js"; import { projectCommandObservationView } from "./commandObservationView.js"; @@ -22,7 +27,12 @@ import { projectGoBinaryView } from "./goBinaryView.js"; import { projectRustBinaryView } from "./rustBinaryView.js"; import { projectNativeFunctionMatchingView } from "./nativeFunctionMatchingView.js"; import { projectNativeInventoryView } from "./nativeInventoryView.js"; -import { NATIVE_INVENTORY_OPERATIONS } from "../native/nativeInventory.js"; +import { + isNativeInventoryOperation, + NATIVE_INVENTORY_OPERATIONS, +} from "../native/nativeInventory.js"; +import { ANALYSIS_EVIDENCE_PREDICATE } from "../evidence.js"; +import { NATIVE_FUNCTION_MATCHING_PREDICATE } from "../native/functionMatching.js"; import { z } from "zod"; import { projectBrowserScenarioView } from "./browserScenarioView.js"; import { projectNativeLoadImageView } from "./nativeLoadImageView.js"; @@ -139,6 +149,9 @@ import { nativeFunctionSummaryViewResultSchema, sourceToBundleSummaryViewResultSchema, sourceToBundlePageViewResultSchema, + featureTraceSummaryViewResultSchema, + batchDecompileSummaryViewResultSchema, + featureTraceViewResultSchema, analysisViewResultSchema, type AnalysisViewResult, type UnsignedAnalysisView, @@ -203,6 +216,9 @@ export { nativeFunctionSummaryViewResultSchema, sourceToBundleSummaryViewResultSchema, sourceToBundlePageViewResultSchema, + featureTraceSummaryViewResultSchema, + batchDecompileSummaryViewResultSchema, + featureTraceViewResultSchema, analysisViewResultSchema, type AnalysisViewResult, type UnsignedAnalysisView, @@ -319,6 +335,8 @@ const retainedViewProjectors = new Map([ retain("inspect_gpu_code_objects", projectGpuCodeObjectsView), retain("match_native_functions", projectNativeFunctionMatchingView), retain("compare_source_to_bundle", projectSourceToBundleView), + retain("trace_application_feature", projectFeatureTraceView), + retain("batch_decompile", projectBatchDecompileView), retain("inspect_artifact", projectArtifactInventoryView), retain("inventory_artifact", projectArtifactInventoryView), retain("analyze_web_bundle", projectWebBundleView), @@ -414,6 +432,29 @@ export const projectAnalysisView = ( }, ]), ); + const expectedPredicate = + parent.operation === "match_native_functions" + ? NATIVE_FUNCTION_MATCHING_PREDICATE + : parent.operation === "analyze_function" || + parent.operation === "batch_decompile" || + isNativeObservationOperation(parent.operation) || + isNativeInventoryOperation(parent.operation) + ? ANALYSIS_EVIDENCE_PREDICATE + : undefined; + if ( + expectedPredicate !== undefined && + parent.predicateType !== expectedPredicate + ) + return err( + analysisViewInputError([ + { + path: ["source", "predicate_type"], + reason: "invalid_value", + expected: expectedPredicate, + message: `${parent.operation} requires ${expectedPredicate} Evidence; received ${parent.predicateType}.`, + }, + ]), + ); if (view.kind === "javascript-routes") { const projected = projectJavaScriptRoutesView(parent, view); return sealProjected(projected); diff --git a/src/domain/analysisView/analysisViewParent.test.ts b/src/domain/analysisView/analysisViewParent.test.ts index 93f56b57db..b0d037f225 100644 --- a/src/domain/analysisView/analysisViewParent.test.ts +++ b/src/domain/analysisView/analysisViewParent.test.ts @@ -8,6 +8,7 @@ import type { AnalysisViewParent } from "./analysisView.js"; import { createAnalysisViewResultReader } from "./analysisViewParent.js"; const parent = (normalizedResult: unknown): AnalysisViewParent => ({ + predicateType: "rea.analysis", evidenceId: `ev_${"a".repeat(64)}`, operation: "inspect_python_pyc", normalizedResult, diff --git a/src/domain/analysisView/analysisViewParent.ts b/src/domain/analysisView/analysisViewParent.ts index 4f1698ccd1..9de5108195 100644 --- a/src/domain/analysisView/analysisViewParent.ts +++ b/src/domain/analysisView/analysisViewParent.ts @@ -10,6 +10,7 @@ import { ownedJsonValueSchema } from "../jsonValue.js"; export interface AnalysisViewParent { readonly evidenceId: string; readonly operation: string; + readonly predicateType: string; readonly normalizedResult: unknown; readonly provider?: Evidence["provider"]; readonly analysisProfile?: Evidence["analysis_profile"]; diff --git a/src/domain/analysisView/analysisViewRequest.ts b/src/domain/analysisView/analysisViewRequest.ts index 697b0ac9c8..cfa24c93f6 100644 --- a/src/domain/analysisView/analysisViewRequest.ts +++ b/src/domain/analysisView/analysisViewRequest.ts @@ -10,6 +10,8 @@ import { nativeExportsViewRequestSchema } from "./nativeExportsView.js"; import { javaScriptRoutesViewRequestSchema } from "./javascriptRoutesView.js"; import { webBundleViewRequestSchema } from "./webBundleView.js"; import { sourceToBundleViewRequestSchema } from "./sourceToBundleView.js"; +import { featureTraceViewRequestSchema } from "./featureTraceView.js"; +import { batchDecompileViewRequestSchema } from "./batchDecompileView.js"; import { gpuCodeObjectsViewRequestSchema } from "./gpuCodeObjectsView.js"; import { modelArtifactViewRequestSchema } from "../model/modelArtifactQuery.js"; import { rustBinaryViewRequestSchema } from "../rust/rustBinaryQuery.js"; @@ -66,6 +68,8 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [ webBundleViewRequestSchema, nativeFunctionMatchingViewRequestSchema, sourceToBundleViewRequestSchema, + featureTraceViewRequestSchema, + batchDecompileViewRequestSchema, summaryViewSchema, goBinaryViewRequestSchema, rustBinaryViewRequestSchema, diff --git a/src/domain/analysisView/analysisViewResult.ts b/src/domain/analysisView/analysisViewResult.ts index f7ee6a1aa1..98328d9b2c 100644 --- a/src/domain/analysisView/analysisViewResult.ts +++ b/src/domain/analysisView/analysisViewResult.ts @@ -63,6 +63,16 @@ import { sourceToBundleViewItemsSchema, sourceToBundleViewContextSchema, } from "./sourceToBundleView.js"; +import { + featureTraceSummarySchema, + featureTraceViewCollections, + featureTraceViewContextSchema, +} from "./featureTraceView.js"; +import { + batchDecompileSummarySchema, + batchDecompileViewCollections, + batchDecompileViewContextSchema, +} from "./batchDecompileView.js"; import { gpuCodeObjectsViewRequestSchema, gpuCodeObjectsSummarySchema, @@ -946,6 +956,75 @@ export const sourceToBundlePageViewResultSchema = z.strictObject({ items: sourceToBundleViewItemsSchema, }); +/** Compact retained feature-trace summary; pages carry the same trace context. */ +export const featureTraceSummaryViewResultSchema = z.strictObject({ + ...summaryViewResultBase, + artifact: z.null(), + parent_operation: z.literal("trace_application_feature"), + summary: featureTraceSummarySchema, +}); +const featureTracePage = < + Collection extends keyof typeof featureTraceViewCollections, +>( + collection: Collection, +) => + z.strictObject({ + ...viewResultBase, + ...featureTraceViewContextSchema.shape, + kind: z.literal("feature-trace"), + artifact: z.null(), + parent_operation: z.literal("trace_application_feature"), + view: featureTraceViewCollections[collection].request, + items: featureTraceViewCollections[collection].rows, + }); +// As with Mach-O pages, the kind tag keeps dispatch while each collection is +// advertised with exactly its own rows. +export const featureTraceViewResultSchema = z + .object({ kind: z.literal("feature-trace") }) + .passthrough() + .pipe( + z.union([ + featureTracePage("paths"), + featureTracePage("nodes"), + featureTracePage("edges"), + featureTracePage("seed_matches"), + featureTracePage("native_handoffs"), + ]), + ); + +/** Counts and identities of a retained batch without any pseudocode body. */ +export const batchDecompileSummaryViewResultSchema = z.strictObject({ + ...summaryViewResultBase, + artifact: artifactSchema.nullable(), + parent_operation: z.literal("batch_decompile"), + summary: batchDecompileSummarySchema, +}); +const batchDecompilePage = { + ...viewResultBase, + ...batchDecompileViewContextSchema.shape, + kind: z.literal("batch-decompile"), + artifact: artifactSchema.nullable(), + parent_operation: z.literal("batch_decompile"), +}; +// The kind tag keeps dispatch while each collection keeps exactly its own shape. +export const batchDecompileViewResultSchema = z + .object({ kind: z.literal("batch-decompile") }) + .passthrough() + .pipe( + z.union([ + z.strictObject({ + ...batchDecompilePage, + view: batchDecompileViewCollections.items.request, + items: batchDecompileViewCollections.items.rows, + }), + z.strictObject({ + ...batchDecompilePage, + view: batchDecompileViewCollections.pseudocode.request, + item: batchDecompileViewCollections.pseudocode.item, + }), + ]), + ); + /** Bind the existing nested facet selection to its producer-owned rows. */ const machoPage = (facet: Facet) => z.strictObject({ @@ -984,6 +1063,8 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ javaScriptRoutesViewResultSchema, webBundlePageViewResultSchema, sourceToBundlePageViewResultSchema, + featureTraceViewResultSchema, + batchDecompileViewResultSchema, z.strictObject({ ...viewResultBase, ...nativeFunctionMatchingContextSchema.shape, @@ -1073,6 +1154,8 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ nativeFunctionMatchingSummaryViewResultSchema, nativeFunctionSummaryViewResultSchema, sourceToBundleSummaryViewResultSchema, + featureTraceSummaryViewResultSchema, + batchDecompileSummaryViewResultSchema, webBundleSummaryViewResultSchema, ]), z.strictObject({ diff --git a/src/domain/analysisView/androidPackageView.test.ts b/src/domain/analysisView/androidPackageView.test.ts index c97b866966..bbb7cac2cf 100644 --- a/src/domain/analysisView/androidPackageView.test.ts +++ b/src/domain/analysisView/androidPackageView.test.ts @@ -90,6 +90,7 @@ const parent = (value = report()): AnalysisViewParent => { }, ); return { + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, normalizedResult: value, diff --git a/src/domain/analysisView/androidSymbolView.test.ts b/src/domain/analysisView/androidSymbolView.test.ts index 72c38013f0..c0cbe5f530 100644 --- a/src/domain/analysisView/androidSymbolView.test.ts +++ b/src/domain/analysisView/androidSymbolView.test.ts @@ -166,6 +166,7 @@ const fixture = (operation: Operation, report: unknown, rawMetadata = "") => { { immutable: true }, ); const parent: AnalysisViewParent = { + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation, normalizedResult: diff --git a/src/domain/analysisView/artifactInventoryView.test.ts b/src/domain/analysisView/artifactInventoryView.test.ts index 4323daebd5..1eba5cfe22 100644 --- a/src/domain/analysisView/artifactInventoryView.test.ts +++ b/src/domain/analysisView/artifactInventoryView.test.ts @@ -84,6 +84,7 @@ const freshInventory = () => { return inventory; }; const parent = (normalizedResult: unknown): AnalysisViewParent => ({ + predicateType: carrier.predicate_type, evidenceId: carrier.evidence_id, operation: "inventory_artifact", subject: diff --git a/src/domain/analysisView/batchDecompileView.ts b/src/domain/analysisView/batchDecompileView.ts new file mode 100644 index 0000000000..19aae24827 --- /dev/null +++ b/src/domain/analysisView/batchDecompileView.ts @@ -0,0 +1,432 @@ +import { z } from "zod"; +import { + committedProviderSchema, + type CommittedProviderIdentity, +} from "../analysisProfile.js"; +import { digestSchema } from "../digests.js"; +import { providerSchema } from "../evidence.js"; +import { + batchDecompileResultSchema, + batchItemLimitationsSchema, + batchProcedureIdentitySchema, + type BatchDecompileItem, + type BatchDecompileResult, +} from "../native/batchDecompile.js"; +import { analysisErrorProjectionSchema } from "../../contracts/errorSchemas.js"; +import { err, ok } from "../result.js"; +import { + boundedViewMessages, + completeWithinViewCoverage, + pageViewCoverage, +} from "./analysisViewCoverage.js"; +import { analysisViewInputError } from "./analysisViewErrors.js"; +import { + createAnalysisViewResultReader, + type AnalysisViewParent, +} from "./analysisViewParent.js"; +import { + ownedView, + type AdmittedView, + type SummaryAnalysisView, +} from "./analysisViewShape.js"; +import { + analysisTextPage, + analysisTextPageItemSchema, +} from "./analysisViewText.js"; +import { selectViewRows } from "./selectViewRows.js"; + +const OPERATION = "batch_decompile" as const; + +const offset = z.int().nonnegative().default(0); + +const itemsRequestSchema = z.strictObject({ + kind: z.literal("batch-decompile"), + collection: z.literal("items"), + offset, + limit: z.int().positive().default(64), + where: z + .strictObject({ + indices: z + .array(z.int().nonnegative()) + .optional() + .describe( + "Exact zero-based positions in the original batch, which keep duplicate selectors distinct. Omitted selects all; [] selects none.", + ), + addresses: z + .array(z.string()) + .optional() + .describe( + "Exact caller-selected symbols or addresses as submitted. Omitted selects all; [] selects none.", + ), + procedure_addresses: z + .array(z.string()) + .optional() + .describe( + "Exact observed canonical entries. An item with unknown procedure identity never matches. Omitted selects all; [] selects none.", + ), + statuses: z + .array(z.enum(["ok", "error"])) + .optional() + .describe("Item outcomes. Omitted selects all; [] selects none."), + }) + .default({}), + include_pseudocode: z + .boolean() + .default(false) + .describe( + "Opt into each selected successful item's complete pseudocode. Otherwise rows report pseudocode_code_units only.", + ), +}); + +const pseudocodeRequestSchema = z.strictObject({ + kind: z.literal("batch-decompile"), + collection: z.literal("pseudocode"), + index: z.int().nonnegative().describe("Exact item position from a row."), + offset: offset.describe("UTF-16 code-unit offset into this item's text."), + limit: z + .int() + .positive() + .default(16_384) + .describe( + "UTF-16 code units in this page. A page size only: coverage reports the complete body and the next offset.", + ), +}); + +/** Select rows of a completed batch or page one retained pseudocode body. */ +export const batchDecompileViewRequestSchema = z.discriminatedUnion( + "collection", + [itemsRequestSchema, pseudocodeRequestSchema], +); + +type BatchDecompileViewRequest = z.output< + typeof batchDecompileViewRequestSchema +>; + +const itemIdentity = { + index: z.int().nonnegative().describe("Position in the original batch."), + address: z.string().describe("Original caller-selected symbol or address."), + procedure: batchProcedureIdentitySchema, +}; + +const batchDecompileRowSchema = z.discriminatedUnion("status", [ + z.strictObject({ + ...itemIdentity, + status: z.literal("ok"), + pseudocode_code_units: z.int().positive(), + pseudocode: z.string().min(1).exactOptional(), + }), + z.strictObject({ + ...itemIdentity, + status: z.literal("error"), + error: analysisErrorProjectionSchema, + limitations: batchItemLimitationsSchema, + }), +]); + +/** + * Producer counts and identities repeated with every view, so one selected row + * stays traceable to its batch and to the provider that decompiled it. + */ +export const batchDecompileViewContextSchema = z.strictObject({ + batch: z.strictObject({ + total: z.int().nonnegative(), + succeeded: z.int().nonnegative(), + failed: z.int().nonnegative(), + }), + source_provider: providerSchema.nullable(), + source_analysis_profile_digest: digestSchema.nullable(), + decompiling_provider: committedProviderSchema + .nullable() + .describe( + "Provider committed by the batch's upstream analysis profile, or null when the parent records none.", + ), +}); + +/** Summary context adds the facts an analyst needs to choose what to read. */ +export const batchDecompileSummarySchema = + batchDecompileViewContextSchema.extend({ + procedures: z.strictObject({ + distinct_resolved: z.int().nonnegative(), + unknown_identity: z.int().nonnegative(), + label_unavailable: z + .int() + .nonnegative() + .describe( + "Resolved items whose entry label is null, whether the provider had none or its lookup failed.", + ), + }), + pseudocode: z.strictObject({ + code_units: z.int().nonnegative(), + largest: z + .strictObject({ + index: z.int().nonnegative(), + code_units: z.int().positive(), + }) + .nullable(), + }), + limitation_count: z.int().nonnegative(), + }); + +/** Each collection's request bound to its own rows, so a reply cannot mix them. */ +export const batchDecompileViewCollections = { + items: { + request: itemsRequestSchema, + rows: z.array(batchDecompileRowSchema), + }, + pseudocode: { + request: pseudocodeRequestSchema, + item: z.strictObject({ + ...itemIdentity, + ...analysisTextPageItemSchema.shape, + pseudocode_code_units: z.int().positive(), + }), + }, +} as const; + +const checkedBatch = createAnalysisViewResultReader(batchDecompileResultSchema); + +const upstreamProfileSchema = z.object({ + upstream_analysis_profile: z.object({ provider: committedProviderSchema }), +}); + +/** A composed batch's Evidence provider is the workflow; its profile names the decompiler. */ +const decompilingProvider = ( + parent: AnalysisViewParent, +): CommittedProviderIdentity | null => { + const parsed = upstreamProfileSchema.safeParse( + parent.analysisProfile?.parameters, + ); + return parsed.success ? parsed.data.upstream_analysis_profile.provider : null; +}; + +const batchContext = ( + parent: AnalysisViewParent, + { total, succeeded, failed }: BatchDecompileResult, +) => ({ + batch: { total, succeeded, failed }, + source_provider: parent.provider ?? null, + source_analysis_profile_digest: parent.analysisProfile?.digest ?? null, + decompiling_provider: decompilingProvider(parent), +}); + +const summarizeBatch = (items: readonly BatchDecompileItem[]) => { + const resolved = new Set(); + let unknownIdentity = 0; + let labelUnavailable = 0; + let codeUnits = 0; + let largest: { index: number; code_units: number } | null = null; + for (const [index, item] of items.entries()) { + if (item.procedure.status === "unknown") unknownIdentity++; + else { + resolved.add(item.procedure.address); + // Null covers an answer without a label and a failed label lookup alike. + if (item.procedure.name === null) labelUnavailable++; + } + if (item.status !== "ok") continue; + codeUnits += item.pseudocode.length; + if (largest === null || item.pseudocode.length > largest.code_units) + largest = { index, code_units: item.pseudocode.length }; + } + return { + procedures: { + distinct_resolved: resolved.size, + unknown_identity: unknownIdentity, + label_unavailable: labelUnavailable, + }, + pseudocode: { code_units: codeUnits, largest }, + }; +}; + +const batchUnknowns = (items: readonly BatchDecompileItem[]): string[] => { + const unknownIdentity = items.filter( + ({ procedure }) => procedure.status === "unknown", + ).length; + return [ + ...(unknownIdentity === 0 + ? [] + : [ + `${String(unknownIdentity)} items have unknown procedure identity; their address is the caller's selector, not an observed entry.`, + ]), + "Evidence limitations aggregate every item's provider observations and are not attributed to one successful item. A batch covers only its selected procedures, and exhausting a page establishes nothing outside it.", + ]; +}; + +const selected = (values: readonly Value[] | undefined) => + values === undefined ? undefined : new Set(values); + +const itemRow = ( + item: BatchDecompileItem, + index: number, + includePseudocode: boolean, +): z.output => { + const identity = { + index, + address: item.address, + procedure: item.procedure, + }; + if (item.status === "ok") + return { + ...identity, + status: "ok", + pseudocode_code_units: item.pseudocode.length, + ...(includePseudocode ? { pseudocode: item.pseudocode } : {}), + }; + return { + ...identity, + status: "error", + error: item.error, + ...(item.limitations === undefined + ? {} + : { limitations: item.limitations }), + }; +}; + +const pseudocodeRefusal = (message: string) => + err( + analysisViewInputError([ + { path: ["view", "index"], reason: "invalid_value", message }, + ]), + ); + +/** Select retained batch facts without repeating decompilation or changing its record. */ +export const projectBatchDecompileView = ( + parent: AnalysisViewParent, + admitted: AdmittedView, +) => { + const view = ownedView( + admitted, + "summary", + "batch-decompile", + ); + if (view === undefined) + return err( + analysisViewInputError([ + { + path: ["view"], + reason: "invalid_value", + message: + "batch_decompile Evidence supports summary and batch-decompile views of items or pseudocode.", + }, + ]), + ); + const parsed = checkedBatch(parent); + if (!parsed.ok) return parsed; + const batch = parsed.value; + const shared = { + parent_evidence_id: parent.evidenceId, + parent_operation: OPERATION, + parent_digest: parent.evidenceId.slice(3), + artifact: parent.artifact ?? null, + limitations: boundedViewMessages( + parent.limitations, + parent.evidenceId, + "limitations", + ), + unknowns: boundedViewMessages( + [], + parent.evidenceId, + "unknowns", + batchUnknowns(batch.items), + ), + }; + const context = batchContext(parent, batch); + if (view.kind === "summary") + return ok({ + kind: "summary" as const, + view, + ...shared, + summary: { + ...context, + ...summarizeBatch(batch.items), + limitation_count: new Set(parent.limitations).size, + }, + coverage: completeWithinViewCoverage(1, 1), + }); + if (view.collection === "items") { + const indices = selected(view.where.indices); + const addresses = selected(view.where.addresses); + const entries = selected(view.where.procedure_addresses); + const statuses = selected(view.where.statuses); + const { rows, total } = selectViewRows( + batch.items, + view, + (item, index) => + (indices === undefined || indices.has(index)) && + (addresses === undefined || addresses.has(item.address)) && + (statuses === undefined || statuses.has(item.status)) && + (entries === undefined || + (item.procedure.status === "resolved" && + entries.has(item.procedure.address))), + (item, index) => itemRow(item, index, view.include_pseudocode), + ); + return ok({ + kind: "batch-decompile" as const, + view, + ...shared, + ...context, + items: rows, + coverage: pageViewCoverage(view.offset, rows.length, total), + }); + } + const item = batch.items[view.index]; + if (item === undefined) + return pseudocodeRefusal( + `Item ${String(view.index)} is outside this batch of ${String(batch.items.length)}.`, + ); + if (item.status !== "ok") + return pseudocodeRefusal( + `Item ${String(view.index)} has status error and no pseudocode; read its error and limitations through the items collection.`, + ); + const page = analysisTextPage(item.pseudocode, view.offset, view.limit); + if (!page.ok) return page; + return ok({ + kind: "batch-decompile" as const, + view, + ...shared, + ...context, + item: { + index: view.index, + address: item.address, + procedure: item.procedure, + pseudocode_code_units: item.pseudocode.length, + ...page.value.item, + }, + coverage: page.value.coverage, + }); +}; + +const deliveredPageSchema = z.looseObject({ + view: batchDecompileViewRequestSchema, + items: z + .array(z.looseObject({ index: z.int(), status: z.string() })) + .optional(), +}); + +/** + * The next selection when a delivered page did not fit: fewer rows, then one + * body's text pages, which need no export. Other views are refused by kind + * first, because delivery asks about every view it returns. + */ +export const smallerBatchDecompileSelection = ( + delivered: unknown, +): z.input | undefined => { + if ( + typeof delivered !== "object" || + delivered === null || + !("kind" in delivered) || + delivered.kind !== "batch-decompile" + ) + return undefined; + const page = deliveredPageSchema.safeParse(delivered); + if (!page.success) return undefined; + const { view, items = [] } = page.data; + if (view.collection === "pseudocode") + return view.limit > 1 + ? { ...view, limit: Math.floor(view.limit / 2) } + : undefined; + if (items.length > 1) return { ...view, limit: Math.floor(items.length / 2) }; + const [row] = items; + return row?.status === "ok" && view.include_pseudocode + ? { kind: view.kind, collection: "pseudocode", index: row.index } + : undefined; +}; diff --git a/src/domain/analysisView/environmentCoverageView.test.ts b/src/domain/analysisView/environmentCoverageView.test.ts index 49478d7321..99223be011 100644 --- a/src/domain/analysisView/environmentCoverageView.test.ts +++ b/src/domain/analysisView/environmentCoverageView.test.ts @@ -35,6 +35,7 @@ const parentOf = ( environmentStateCaptureFixture(), ), ): AnalysisViewParent => ({ + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, normalizedResult: evidence.normalized_result, diff --git a/src/domain/analysisView/environmentManifestView.test.ts b/src/domain/analysisView/environmentManifestView.test.ts index fdbdbbe93b..f14d9b2f38 100644 --- a/src/domain/analysisView/environmentManifestView.test.ts +++ b/src/domain/analysisView/environmentManifestView.test.ts @@ -39,6 +39,7 @@ const parentOf = ( ) => { const evidence = analysisViewCaptureEvidence(capture); return { + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, normalizedResult: evidence.normalized_result, @@ -854,6 +855,7 @@ it("refuses non-HAR formats, sensitive parents, unsupported parents and malforme expect( projectAnalysisView( { + predicateType: layout.predicate_type, evidenceId: layout.evidence_id, operation: layout.operation, normalizedResult: layout.normalized_result, @@ -871,6 +873,7 @@ it("refuses non-HAR formats, sensitive parents, unsupported parents and malforme expect( projectAnalysisView( { + predicateType: "rea.analysis", evidenceId: `ev_${"a".repeat(64)}`, operation: "inspect_web_network_capture", normalizedResult, diff --git a/src/domain/analysisView/environmentStateView.test.ts b/src/domain/analysisView/environmentStateView.test.ts index 498d831ca3..80d18e6eff 100644 --- a/src/domain/analysisView/environmentStateView.test.ts +++ b/src/domain/analysisView/environmentStateView.test.ts @@ -57,6 +57,7 @@ const parentOf = ( environmentStateCaptureFixture(), ), ): AnalysisViewParent => ({ + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, normalizedResult: evidence.normalized_result, diff --git a/src/domain/analysisView/featureTraceView.ts b/src/domain/analysisView/featureTraceView.ts new file mode 100644 index 0000000000..b9a848401d --- /dev/null +++ b/src/domain/analysisView/featureTraceView.ts @@ -0,0 +1,349 @@ +import { z } from "zod"; +import { prefixedDigestSchema } from "../digests.js"; +import { applicationCoverageSchema } from "../javascript/javascriptApplicationEvidenceSchemas.js"; +import { + applicationEdgeSchema, + applicationNodeSchema, + type ApplicationEdge, + type ApplicationNode, +} from "../javascript/javascriptApplicationGraphSchemas.js"; +import { + applicationFeatureTraceResultSchema, + type ApplicationFeatureTraceResult, +} from "../javascript/javascriptFeatureTraceSchemas.js"; +import { err, ok } from "../result.js"; +import { + boundedViewMessages, + completeWithinViewCoverage, + pageViewCoverage, +} from "./analysisViewCoverage.js"; +import { analysisViewInputError } from "./analysisViewErrors.js"; +import { + createAnalysisViewResultReader, + type AnalysisViewParent, +} from "./analysisViewParent.js"; +import { + ownedView, + type AdmittedView, + type SummaryAnalysisView, +} from "./analysisViewShape.js"; +import { + javascriptFactBasis, + javascriptFactBasisSchema, +} from "./javascriptModuleReferences.js"; +import { selectViewRows } from "./selectViewRows.js"; + +const OPERATION = "trace_application_feature" as const; +const trace = applicationFeatureTraceResultSchema.shape; + +const nodeIdsFilter = z + .array(prefixedDigestSchema("jag_node")) + .optional() + .describe( + "Exact trace node identities. Omitted selects all; [] selects none.", + ); +const page = { + kind: z.literal("feature-trace"), + offset: z.int().nonnegative().default(0), + limit: z.int().positive().default(64), +}; + +const pathsRequestSchema = z.strictObject({ + ...page, + collection: z.literal("paths"), + where: z + .strictObject({ + node_ids: nodeIdsFilter.describe( + "Select terminal paths that visit any of these exact node identities. Omitted selects all; [] selects none.", + ), + end_kinds: z.array(applicationNodeSchema.shape.kind).optional(), + }) + .default({}), +}); +const nodesRequestSchema = z.strictObject({ + ...page, + collection: z.literal("nodes"), + where: z + .strictObject({ + node_ids: nodeIdsFilter, + kinds: z.array(applicationNodeSchema.shape.kind).optional(), + }) + .default({}), + include_observations: z + .boolean() + .default(false) + .describe( + "Opt into each selected node's complete retained observations, including properties and provenance.", + ), +}); +const edgesRequestSchema = z.strictObject({ + ...page, + collection: z.literal("edges"), + where: z + .strictObject({ + node_ids: nodeIdsFilter.describe( + "Select edges whose source or target is one of these exact node identities. Omitted selects all; [] selects none.", + ), + relations: z.array(applicationEdgeSchema.shape.relation).optional(), + }) + .default({}), +}); +const seedMatchesRequestSchema = z.strictObject({ + ...page, + collection: z.literal("seed_matches"), +}); +const handoffsRequestSchema = z.strictObject({ + ...page, + collection: z.literal("native_handoffs"), +}); + +/** Page one collection of a completed feature trace in its retained order. */ +export const featureTraceViewRequestSchema = z.discriminatedUnion( + "collection", + [ + pathsRequestSchema, + nodesRequestSchema, + edgesRequestSchema, + seedMatchesRequestSchema, + handoffsRequestSchema, + ], +); + +type FeatureTraceViewRequest = z.output; + +/** Compact node identity and support; full observations are opt-in. */ +const featureTraceNodeRowSchema = applicationNodeSchema + .pick({ node_id: true, kind: true, identity: true }) + .extend({ + seed: z + .boolean() + .describe("Whether this node matched the trace seed (a trace root)."), + observation_count: z.int().positive(), + basis: z.array(javascriptFactBasisSchema).min(1), + observations: applicationNodeSchema.shape.observations.exactOptional(), + }); + +/** + * Each collection's request bound to its own rows, so a reply cannot pair one + * collection with another's rows. Rows are the producer's exact records or the + * compact node projection. + */ +export const featureTraceViewCollections = { + paths: { request: pathsRequestSchema, rows: z.array(trace.paths.element) }, + nodes: { + request: nodesRequestSchema, + rows: z.array(featureTraceNodeRowSchema), + }, + edges: { request: edgesRequestSchema, rows: z.array(applicationEdgeSchema) }, + seed_matches: { + request: seedMatchesRequestSchema, + rows: z.array(trace.seed_matches.element), + }, + native_handoffs: { + request: handoffsRequestSchema, + rows: z.array(trace.native_handoffs.element), + }, +} as const; + +/** Trace identity and coverage repeated with every view so selected rows stay traceable. */ +export const featureTraceViewContextSchema = z.strictObject({ + trace_id: trace.trace_id, + source_evidence_id: trace.source_evidence_id, + source_graph_id: trace.source_graph_id, + seed: trace.seed, + direction: trace.direction, + summary: trace.summary, + trace_coverage: trace.coverage, + graph_coverage: applicationCoverageSchema + .nullable() + .describe("The traced subgraph's coverage, or null when no seed matched."), + evidence_links: trace.evidence_links, +}); + +/** Summary context adds the producer's already paged no-match discovery. */ +export const featureTraceSummarySchema = featureTraceViewContextSchema.extend({ + semantic_discovery: trace.semantic_discovery, + limitation_count: z.int().nonnegative(), +}); + +const checkedTrace = createAnalysisViewResultReader( + applicationFeatureTraceResultSchema, +); + +const selected = (values: readonly Value[] | undefined) => + values === undefined ? undefined : new Set(values); + +const traceContext = (result: ApplicationFeatureTraceResult) => ({ + trace_id: result.trace_id, + source_evidence_id: result.source_evidence_id, + source_graph_id: result.source_graph_id, + seed: result.seed, + direction: result.direction, + summary: result.summary, + trace_coverage: result.coverage, + graph_coverage: result.graph?.coverage ?? null, + evidence_links: result.evidence_links, +}); + +const traceUnknowns = (result: ApplicationFeatureTraceResult): string[] => [ + ...(result.coverage.status === "partial" + ? [ + `trace_coverage.status is partial because the source application graph is ${result.coverage.source_graph_status}; unmatched seeds and frontiers remain unknown.`, + ] + : []), + "Trace connectivity does not prove runtime execution or reachability. Exhausting a page establishes nothing outside this trace.", +]; + +const nodeRow = ( + node: ApplicationNode, + seeds: ReadonlySet, + includeObservations: boolean, +): z.output => ({ + node_id: node.node_id, + kind: node.kind, + identity: node.identity, + seed: seeds.has(node.node_id), + observation_count: node.observations.length, + basis: javascriptFactBasis(node.observations.map(({ evidence }) => evidence)), + ...(includeObservations ? { observations: node.observations } : {}), +}); + +/** One page per collection, built in its own branch so view and rows stay correlated. */ +const tracePage = ( + result: ApplicationFeatureTraceResult, + view: FeatureTraceViewRequest, + shared: Shared, +) => { + const pageOf = ( + selection: View, + { rows, total }: { readonly rows: Row[]; readonly total: number }, + ) => ({ + kind: "feature-trace" as const, + view: selection, + ...shared, + items: rows, + coverage: pageViewCoverage(selection.offset, rows.length, total), + }); + const all = () => true; + if (view.collection === "seed_matches") + return pageOf( + view, + selectViewRows(result.seed_matches, view, all, (row) => row), + ); + if (view.collection === "native_handoffs") + return pageOf( + view, + selectViewRows(result.native_handoffs, view, all, (row) => row), + ); + if (view.collection === "paths") { + const nodes = selected(view.where.node_ids); + const ends = selected(view.where.end_kinds); + return pageOf( + view, + selectViewRows( + result.paths, + view, + (path) => + (ends === undefined || ends.has(path.end_kind)) && + (nodes === undefined || path.node_ids.some((id) => nodes.has(id))), + (path) => path, + ), + ); + } + const graph = result.graph; + if (view.collection === "edges") { + const nodes = selected(view.where.node_ids); + const relations = selected(view.where.relations); + return pageOf( + view, + selectViewRows( + graph?.edges ?? [], + view, + (edge) => + (relations === undefined || relations.has(edge.relation)) && + (nodes === undefined || + nodes.has(edge.source_node_id) || + nodes.has(edge.target_node_id)), + (edge) => edge, + ), + ); + } + const ids = selected(view.where.node_ids); + const kinds = selected(view.where.kinds); + const seeds = new Set(result.seed_matches.map(({ node_id: id }) => id)); + return pageOf( + view, + selectViewRows( + graph?.nodes ?? [], + view, + (node) => + (ids === undefined || ids.has(node.node_id)) && + (kinds === undefined || kinds.has(node.kind)), + (node) => nodeRow(node, seeds, view.include_observations), + ), + ); +}; + +/** Select retained trace facts without re-running traversal or changing its coverage. */ +export const projectFeatureTraceView = ( + parent: AnalysisViewParent, + admitted: AdmittedView, +) => { + const view = ownedView( + admitted, + "summary", + "feature-trace", + ); + if (view === undefined) + return err( + analysisViewInputError([ + { + path: ["view"], + reason: "invalid_value", + message: + "Application feature traces support summary and feature-trace pages of paths, nodes, edges, seed_matches or native_handoffs.", + }, + ]), + ); + const parsed = checkedTrace(parent); + if (!parsed.ok) return parsed; + const result = parsed.value; + const limitations = [ + ...parent.limitations, + ...result.limitations, + ...(result.graph?.limitations ?? []), + ]; + const shared = { + parent_evidence_id: parent.evidenceId, + parent_operation: OPERATION, + parent_digest: parent.evidenceId.slice(3), + artifact: null, + limitations: boundedViewMessages( + limitations, + parent.evidenceId, + "limitations", + ), + unknowns: boundedViewMessages( + [], + parent.evidenceId, + "unknowns", + traceUnknowns(result), + ), + }; + const context = traceContext(result); + if (view.kind === "summary") + return ok({ + kind: "summary" as const, + view, + ...shared, + summary: { + ...context, + ...(result.semantic_discovery === undefined + ? {} + : { semantic_discovery: result.semantic_discovery }), + limitation_count: new Set(limitations).size, + }, + coverage: completeWithinViewCoverage(1, 1), + }); + return ok(tracePage(result, view, { ...shared, ...context })); +}; diff --git a/src/domain/analysisView/goBinaryView.test.ts b/src/domain/analysisView/goBinaryView.test.ts index 807b10f14d..a82c68f6ca 100644 --- a/src/domain/analysisView/goBinaryView.test.ts +++ b/src/domain/analysisView/goBinaryView.test.ts @@ -38,6 +38,7 @@ const parent = (): AnalysisViewParent => { }, ); return { + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, subject: evidence.subject, diff --git a/src/domain/analysisView/gpuCodeObjectsView.test.ts b/src/domain/analysisView/gpuCodeObjectsView.test.ts index 8bc5b9e8ae..a800cee49c 100644 --- a/src/domain/analysisView/gpuCodeObjectsView.test.ts +++ b/src/domain/analysisView/gpuCodeObjectsView.test.ts @@ -32,6 +32,7 @@ const report = () => }); const parent = (normalizedResult: unknown): AnalysisViewParent => ({ + predicateType: "rea.analysis", evidenceId: `ev_${"b".repeat(64)}`, operation: "inspect_gpu_code_objects", normalizedResult, diff --git a/src/domain/analysisView/il2cppMetadataView.test.ts b/src/domain/analysisView/il2cppMetadataView.test.ts index 9aa9a4b9e4..23e27840fc 100644 --- a/src/domain/analysisView/il2cppMetadataView.test.ts +++ b/src/domain/analysisView/il2cppMetadataView.test.ts @@ -67,6 +67,7 @@ const report: Il2cppMetadata = { }; const parent = (normalizedResult: unknown, sealedResult: boolean) => ({ + predicateType: "rea.analysis", evidenceId: `ev_${"b".repeat(64)}`, operation: "inspect_il2cpp_metadata", normalizedResult, diff --git a/src/domain/analysisView/interfaceBuilderView.test.ts b/src/domain/analysisView/interfaceBuilderView.test.ts index 9ec2dd0bfe..9c9921be59 100644 --- a/src/domain/analysisView/interfaceBuilderView.test.ts +++ b/src/domain/analysisView/interfaceBuilderView.test.ts @@ -91,6 +91,7 @@ const parent = ( normalizedResult: unknown = report(), sha256 = target, ): AnalysisViewParent => ({ + predicateType: "rea.analysis", evidenceId: `ev_${"c".repeat(64)}`, operation: "decode_interface_builder", normalizedResult, diff --git a/src/domain/analysisView/javascriptApplicationView.ts b/src/domain/analysisView/javascriptApplicationView.ts index 46872962ec..6405275e88 100644 --- a/src/domain/analysisView/javascriptApplicationView.ts +++ b/src/domain/analysisView/javascriptApplicationView.ts @@ -10,7 +10,10 @@ import type { ApplicationNode } from "../javascript/javascriptApplicationGraphSc import { err, ok, type Result } from "../result.js"; import { z } from "zod"; import type { JavaScriptModuleReference } from "./javascriptModuleReferences.js"; -import { javascriptModuleReferencesRequestSchema } from "./javascriptModuleReferences.js"; +import { + javascriptFactBasis, + javascriptModuleReferencesRequestSchema, +} from "./javascriptModuleReferences.js"; import { javascriptApplicationSourceCoverage, javascriptStaticSourceCoverageSummary, @@ -200,27 +203,21 @@ const moduleReference = ({ path, }: ModuleEntry): JavaScriptModuleReference => { let ranges = 0; - const support = new Map(); - for (const { evidence } of node.observations) { + for (const { evidence } of node.observations) if ( evidence.location.available && evidence.location.value.kind === "source-range" ) ranges += 1; - const { authority, state, confidence } = evidence; - support.set(`${authority}:${state}:${confidence}`, { - authority, - state, - confidence, - }); - } return { node_id: node.node_id, kind: node.kind, path, identity: node.identity, retained_source_range_count: ranges, - basis: [...support.values()], + basis: javascriptFactBasis( + node.observations.map(({ evidence }) => evidence), + ), }; }; diff --git a/src/domain/analysisView/javascriptModuleReferences.ts b/src/domain/analysisView/javascriptModuleReferences.ts index 475ca0d81e..e97f4b912c 100644 --- a/src/domain/analysisView/javascriptModuleReferences.ts +++ b/src/domain/analysisView/javascriptModuleReferences.ts @@ -27,12 +27,28 @@ export const javascriptModuleReferencesRequestSchema = z.strictObject({ }); /** Unique support states, without promoting mixed observations to one fact. */ -const basisSchema = z.strictObject({ +export const javascriptFactBasisSchema = z.strictObject({ authority: applicationGraphEvidenceSchema.shape.authority, state: applicationGraphEvidenceSchema.shape.state, confidence: applicationGraphEvidenceSchema.shape.confidence, }); +type JavaScriptFactBasis = z.infer; + +/** Distinct support states in first-observed order; mixed support stays mixed. */ +export const javascriptFactBasis = ( + evidence: Iterable, +): JavaScriptFactBasis[] => { + const support = new Map(); + for (const { authority, state, confidence } of evidence) + support.set(`${authority}:${state}:${confidence}`, { + authority, + state, + confidence, + }); + return [...support.values()]; +}; + /** Reusable facts rather than task-specific dependency interpretations. */ export const javascriptModuleReferenceSchema = z.strictObject({ node_id: prefixedDigestSchema("jag_node"), @@ -40,7 +56,7 @@ export const javascriptModuleReferenceSchema = z.strictObject({ path: z.string().nullable(), identity: applicationNodeIdentitySchema, retained_source_range_count: z.int().nonnegative(), - basis: z.array(basisSchema).min(1), + basis: z.array(javascriptFactBasisSchema).min(1), }); export type JavaScriptModuleReference = z.infer< diff --git a/src/domain/analysisView/javascriptRoutesView.test.ts b/src/domain/analysisView/javascriptRoutesView.test.ts index 68e542a7a0..7b09e25b82 100644 --- a/src/domain/analysisView/javascriptRoutesView.test.ts +++ b/src/domain/analysisView/javascriptRoutesView.test.ts @@ -28,6 +28,7 @@ const inventory = ( ]), ); const parent = (): AnalysisViewParent => ({ + predicateType: "rea.javascript-application-analysis", evidenceId: `ev_${"b".repeat(64)}`, operation: "analyze_javascript_application", limitations: [], diff --git a/src/domain/analysisView/machoView.test.ts b/src/domain/analysisView/machoView.test.ts index 40f650a2cf..13369b8988 100644 --- a/src/domain/analysisView/machoView.test.ts +++ b/src/domain/analysisView/machoView.test.ts @@ -194,6 +194,7 @@ const parent = ( local_path: "/bin/tool", }, ): AnalysisViewParent => ({ + predicateType: "rea.analysis", evidenceId: `ev_${"c".repeat(64)}`, operation: "inspect_macho", normalizedResult, diff --git a/src/domain/analysisView/managedMemberView.test.ts b/src/domain/analysisView/managedMemberView.test.ts index e6d8f401f4..122f27640a 100644 --- a/src/domain/analysisView/managedMemberView.test.ts +++ b/src/domain/analysisView/managedMemberView.test.ts @@ -10,6 +10,7 @@ import { import { projectManagedMemberView } from "./managedMemberView.js"; const parent = (normalizedResult: unknown): AnalysisViewParent => ({ + predicateType: "rea.analysis", evidenceId: `ev_${"a".repeat(64)}`, operation: "inspect_managed_members", normalizedResult, diff --git a/src/domain/analysisView/nativeExportsView.ts b/src/domain/analysisView/nativeExportsView.ts index 1d9be08030..2dad3ee5cd 100644 --- a/src/domain/analysisView/nativeExportsView.ts +++ b/src/domain/analysisView/nativeExportsView.ts @@ -67,6 +67,7 @@ export const nativeExportsViewItemsSchema = z.union([ name_utf8_decodable: elfRow.shape.name_utf8_decodable .removeDefault() .optional(), + visibility: elfRow.shape.visibility.removeDefault().optional(), }), ), z.array(issue.extend(sourceIndex)), diff --git a/src/domain/analysisView/nativeInventoryView.test.ts b/src/domain/analysisView/nativeInventoryView.test.ts index ed946415ca..cf184fa8b9 100644 --- a/src/domain/analysisView/nativeInventoryView.test.ts +++ b/src/domain/analysisView/nativeInventoryView.test.ts @@ -19,6 +19,7 @@ const rows = () => [ { address: "0x20", value: "last" }, ]; const parent = (normalizedResult: unknown): AnalysisViewParent => ({ + predicateType: "rea.analysis", evidenceId: `ev_${"a".repeat(64)}`, operation: "search_procedures", normalizedResult, diff --git a/src/domain/analysisView/nativeLoadImageView.ts b/src/domain/analysisView/nativeLoadImageView.ts index c4975e00dc..eb5b56af88 100644 --- a/src/domain/analysisView/nativeLoadImageView.ts +++ b/src/domain/analysisView/nativeLoadImageView.ts @@ -43,6 +43,7 @@ const defaultFields: Partial> = { offset: true, status: true, type: true, + symbol: true, values: true, }) .keyof().options, @@ -65,7 +66,7 @@ const request = (name: Name) => .optional() .describe( name === "relocations" - ? "Defaults to address, segment, offset, status, type and values. Select original_bytes_hex or memory_bytes_hex explicitly." + ? "Defaults to address, segment, offset, status, type, symbol and values. Select original_bytes_hex or memory_bytes_hex explicitly." : name === "checks" ? "Defaults to name, matched, file_offset and address. Select expected or observed explicitly; these can contain complete comparison inventories." : "Omit to return every recorded field in each selected row.", @@ -290,8 +291,19 @@ export const projectNativeLoadImageView = ( : undefined : observed[view.collection]; const page = selectRows(selected ?? [], view); + const symbolUnrecorded = + view.collection === "relocations" && + (view.fields === undefined || + view.fields.some((field: string) => field === "symbol")) && + page.rows.some(({ record }) => !("symbol" in record)); return ok({ ...shared, + unknowns: symbolUnrecorded + ? [ + ...shared.unknowns, + "The relocation symbol was not recorded for at least one selected row.", + ] + : shared.unknowns, kind: "native-load-image" as const, verification_status: report.status, view, diff --git a/src/domain/analysisView/pcapCaptureView.test.ts b/src/domain/analysisView/pcapCaptureView.test.ts index 0f5cf34dda..1e16f2b02d 100644 --- a/src/domain/analysisView/pcapCaptureView.test.ts +++ b/src/domain/analysisView/pcapCaptureView.test.ts @@ -65,6 +65,7 @@ const capture = ( }, ); return { + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, normalizedResult: report, diff --git a/src/domain/analysisView/protobufCaptureView.test.ts b/src/domain/analysisView/protobufCaptureView.test.ts index d0619f31b7..18d737fd1c 100644 --- a/src/domain/analysisView/protobufCaptureView.test.ts +++ b/src/domain/analysisView/protobufCaptureView.test.ts @@ -113,6 +113,7 @@ const capture = async () => { return { report, parent: { + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, normalizedResult: report, diff --git a/src/domain/analysisView/rustBinaryView.test.ts b/src/domain/analysisView/rustBinaryView.test.ts index 065b86a6fd..43d7906e70 100644 --- a/src/domain/analysisView/rustBinaryView.test.ts +++ b/src/domain/analysisView/rustBinaryView.test.ts @@ -54,6 +54,7 @@ const parent = (value = structuredClone(report)): AnalysisViewParent => { }, ); return { + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, subject: evidence.subject, diff --git a/src/domain/analysisView/scenarioCaptureRecords.test.ts b/src/domain/analysisView/scenarioCaptureRecords.test.ts index 6f9516cb08..25d9dfdbbf 100644 --- a/src/domain/analysisView/scenarioCaptureRecords.test.ts +++ b/src/domain/analysisView/scenarioCaptureRecords.test.ts @@ -282,6 +282,7 @@ it("reserves retained request-body expansion before schema decoding or projectio try { const projected = admitScenarioCaptureParent( { + predicateType: "rea.browser-scenario-capture", evidenceId: `ev_${"a".repeat(64)}`, operation: "capture_browser_scenario", normalizedResult: capture, @@ -330,6 +331,7 @@ it("reserves request text even when another event's body was not captured", () = // The capture's containers fit this budget; only its request text does not. const projected = admitScenarioCaptureParent( { + predicateType: "rea.browser-scenario-capture", evidenceId: `ev_${"a".repeat(64)}`, operation: "capture_browser_scenario", normalizedResult: { ...capture, events: { ...capture.events, items } }, diff --git a/src/domain/analysisView/webBundleView.test.ts b/src/domain/analysisView/webBundleView.test.ts index d146529887..c277b39683 100644 --- a/src/domain/analysisView/webBundleView.test.ts +++ b/src/domain/analysisView/webBundleView.test.ts @@ -109,6 +109,7 @@ const sampleAnalysis = (): WebBundleAnalysis => ({ const makeParent = ( overrides?: Partial, ): AnalysisViewParent => ({ + predicateType: "rea.web-bundle-analysis", evidenceId: `ev_${"c".repeat(64)}`, operation: "analyze_web_bundle", limitations: ["Parent run was bounded"], @@ -476,6 +477,7 @@ it.each(["indices", "values", "script_keys", "mechanisms"])( it("rejects non-web-bundle parent operation with invalid_value", () => { const invalidParent: AnalysisViewParent = { + predicateType: "rea.javascript-application-analysis", evidenceId: `ev_${"d".repeat(64)}`, operation: "analyze_javascript_application", limitations: [], @@ -627,6 +629,7 @@ it("correctly counts zero findings in summary for an empty web bundle", () => { it("rejects malformed parent analysis report", () => { const malformedParent: AnalysisViewParent = { + predicateType: "rea.web-bundle-analysis", evidenceId: `ev_${"e".repeat(64)}`, operation: "analyze_web_bundle", limitations: [], diff --git a/src/domain/analysisView/webNetworkApiSurfaceView.test.ts b/src/domain/analysisView/webNetworkApiSurfaceView.test.ts index d2e9662bd7..617811f81b 100644 --- a/src/domain/analysisView/webNetworkApiSurfaceView.test.ts +++ b/src/domain/analysisView/webNetworkApiSurfaceView.test.ts @@ -19,6 +19,7 @@ const project = ( const parent = analysisViewCaptureEvidence(capture); return projectAnalysisView( { + predicateType: parent.predicate_type, evidenceId: parent.evidence_id, operation: parent.operation, normalizedResult: parent.normalized_result, @@ -202,6 +203,7 @@ it("rejects native captures and malformed parents with typed errors", () => { expect( projectAnalysisView( { + predicateType: "rea.analysis", evidenceId: `ev_${"a".repeat(64)}`, operation: "inspect_web_network_capture", normalizedResult: {}, @@ -218,6 +220,7 @@ it.each(["api-surface", "environment-manifest", "environment-state"])( const capture = analysisViewCaptureFixture(); const result = projectAnalysisView( { + predicateType: "rea.analysis", evidenceId: `ev_${"a".repeat(64)}`, operation: "inspect_web_network_capture", limitations: [], @@ -357,6 +360,7 @@ it("rejects cyclic captures and accessor members without invoking them", () => { expect( projectAnalysisView( { + predicateType: "rea.analysis", evidenceId: `ev_${"a".repeat(64)}`, operation: "inspect_web_network_capture", normalizedResult, diff --git a/src/domain/analysisView/webNetworkCaptureView.test.ts b/src/domain/analysisView/webNetworkCaptureView.test.ts index 9fabd6d8aa..4aaecad8c6 100644 --- a/src/domain/analysisView/webNetworkCaptureView.test.ts +++ b/src/domain/analysisView/webNetworkCaptureView.test.ts @@ -69,6 +69,7 @@ const capture = () => { return { report, parent: { + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, normalizedResult: report, @@ -126,6 +127,7 @@ it("uses qlog event headlines without promoting HTTP-shaped extensions", () => { ); const preview = projectAnalysisView( { + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, normalizedResult: qlog, @@ -349,6 +351,7 @@ const nativePreview = ( }); const preview = projectAnalysisView( { + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, normalizedResult: report, diff --git a/src/domain/bundleComparison.test.ts b/src/domain/bundleComparison.test.ts index f91aff856e..406389608d 100644 --- a/src/domain/bundleComparison.test.ts +++ b/src/domain/bundleComparison.test.ts @@ -77,39 +77,6 @@ describe("bundle comparison explicit cross-pairs", () => { }); describe("bundle comparison", () => { - it("does not report display-only bundle differences as unexplained changes", () => { - const target = { - path: "/left/artifact.elf", - sha256: "a".repeat(64), - format: "elf" as const, - architecture: "x86_64" as const, - }; - const record = createEvidence(target, PROVIDER, { - operation: "observe", - parameters: {}, - result: { observed: true }, - confidence: "observed", - authority: "shipped-artifact", - }); - const left = createEvidenceBundle([record]); - const rightRecord = { - ...record, - subject: { - ...record.subject!, - name: "renamed.elf", - local_path: "/right/renamed.elf", - }, - }; - const right = { ...left, records: [rightRecord] }; - const result = compareBundles(left, right); - expect(result).toMatchObject({ - status: "unchanged", - summary: { records_unchanged: 1, unresolved: 0 }, - changes: [], - }); - expect(result.left_bundle_sha256).not.toBe(result.right_bundle_sha256); - }); - it("accepts long caller-selected bundle paths", () => { const path = `/${"deep/".repeat(1_000)}bundle.json`; expect( diff --git a/src/domain/changedBehavior.test.ts b/src/domain/changedBehavior.test.ts index 4142623361..049f84da9a 100644 --- a/src/domain/changedBehavior.test.ts +++ b/src/domain/changedBehavior.test.ts @@ -228,15 +228,6 @@ describe("changed behavior", () => { }); expect(aggregate([changed, unknown]).result).toEqual(result); }); - - it("uses complete inline artifact deltas as static candidates", () => { - const complete = comparison("compare_artifacts", artifactResult()); - expect(aggregate([complete]).result.findings.items).toEqual( - expect.arrayContaining([ - expect.objectContaining({ dimension: "artifact:main.js" }), - ]), - ); - }); }); describe("changed-behavior input and Evidence validation", () => { diff --git a/src/domain/evidence.ts b/src/domain/evidence.ts index 14c598c506..9439563979 100644 --- a/src/domain/evidence.ts +++ b/src/domain/evidence.ts @@ -32,6 +32,9 @@ import { type RelationshipCoverage, } from "./relationshipCoverage.js"; +/** Predicate shared by ordinary provider analysis observations. */ +export const ANALYSIS_EVIDENCE_PREDICATE = "rea.analysis"; + /** Provider identity schema shared by evidence-bearing persistence formats. */ export const providerSchema = z.strictObject({ id: z.string().min(1), @@ -453,7 +456,7 @@ const normalizeEvidenceObservation = ( version: provider.version, }, analysis_profile: observation.analysisProfile ?? null, - predicate_type: observation.predicateType ?? "rea.analysis", + predicate_type: observation.predicateType ?? ANALYSIS_EVIDENCE_PREDICATE, operation: observation.operation, parameters: observation.parameters, raw_result: observation.rawResult ?? null, diff --git a/src/domain/evidenceErrors.ts b/src/domain/evidenceErrors.ts index 80d0af2ce0..07d00ec7fe 100644 --- a/src/domain/evidenceErrors.ts +++ b/src/domain/evidenceErrors.ts @@ -31,7 +31,14 @@ export class EvidenceIntegrityError extends AnalysisError { } /** A valid snapshot belongs to a different target, provider, or analysis profile. */ -export class AnalysisSnapshotMismatchError extends EvidenceIntegrityError {} +export class AnalysisSnapshotMismatchError extends EvidenceIntegrityError { + constructor( + message: string, + readonly bindingMismatch?: import("./analysisSnapshot.js").SnapshotBindingMismatch, + ) { + super(message); + } +} /** A resource session cannot import or export a binary analysis snapshot. */ export class ResourceSnapshotMismatchError extends AnalysisSnapshotMismatchError { diff --git a/src/domain/evidenceRecordSummary.ts b/src/domain/evidenceRecordSummary.ts index 47653431f8..01c4ed2278 100644 --- a/src/domain/evidenceRecordSummary.ts +++ b/src/domain/evidenceRecordSummary.ts @@ -1,5 +1,6 @@ import { z } from "zod"; +import { uniqueSorted } from "./canonicalOrdering.js"; import { digestSchema, prefixedDigestSchema } from "./digests.js"; import { evidenceSchema, type Evidence } from "./evidence.js"; import type { EvidenceBundle } from "./evidenceBundle.js"; @@ -107,15 +108,24 @@ const nativeSummary = ( }; }; -/** Discover retained Evidence without reading or copying its complete payloads. */ -export const summarizeEvidenceBundle = ( - bundle: EvidenceBundle, - filters: EvidenceRecordFilters = {}, -): EvidenceBundleSummary => { - const selectedFilters: Record = {}; +/** Caller-selected filters with omitted fields removed, echoed by every selection. */ +const selectedFilters = ( + filters: EvidenceRecordFilters, +): Record => { + const selected: Record = {}; for (const [field, value] of Object.entries(filters)) - if (value !== undefined) selectedFilters[field] = value; - const records: EvidenceBundleSummary["records"] = []; + if (value !== undefined) selected[field] = value; + return selected; +}; + +/** One owner for exact filter semantics, in canonical bundle (Evidence ID) order. */ +function* matchingEvidenceRecords( + bundle: EvidenceBundle, + filters: EvidenceRecordFilters, +): Generator<{ + readonly evidence: Evidence; + readonly native: z.infer; +}> { for (const evidence of bundle.records) { if ( (filters.evidence_id !== undefined && @@ -135,6 +145,17 @@ export const summarizeEvidenceBundle = ( filters.procedure_address !== native.procedure_address) ) continue; + yield { evidence, native }; + } +} + +/** Discover retained Evidence without reading or copying its complete payloads. */ +export const summarizeEvidenceBundle = ( + bundle: EvidenceBundle, + filters: EvidenceRecordFilters = {}, +): EvidenceBundleSummary => { + const records: EvidenceBundleSummary["records"] = []; + for (const { evidence, native } of matchingEvidenceRecords(bundle, filters)) records.push({ evidence_id: evidence.evidence_id, subject: evidence.subject, @@ -147,13 +168,78 @@ export const summarizeEvidenceBundle = ( limitation_count: evidence.limitations.length, native_dossier: native, }); - } return { kind: "evidence-bundle-summary", - filters: selectedFilters, + filters: selectedFilters(filters), total_retained_records: bundle.records.length, matching_records: records.length, unknown_revisions: bundle.unknowns.length, records, }; }; + +/** Default page size for complete record selection; each record may be large. */ +export const EVIDENCE_RECORD_SELECTION_LIMIT = 8; + +/** Labels every record selection as historical and as distinct from an importable bundle. */ +export const EVIDENCE_RECORD_SELECTION_LIMITATIONS = [ + "Selected records are retained historical observations; they do not establish current live provider or database state.", + "A record selection is not an Evidence bundle: it omits manifests and residual unknowns and cannot be imported. Use export_evidence_bundle for a portable bundle.", +] as const; + +/** Complete selected records with their retained identities and direct-link closure. */ +export const evidenceRecordSelectionSchema = z.strictObject({ + kind: z.literal("evidence-record-selection"), + filters: z.record(z.string(), z.string()), + total_retained_records: countSchema, + matching_records: countSchema, + offset: countSchema, + limit: z.int().positive(), + next_offset: countSchema.nullable(), + records: z.array(evidenceSchema), + missing_evidence_links: z + .array(prefixedDigestSchema("ev")) + .describe( + "Direct evidence_links of the returned records that this session or bundle does not retain; links are not traversed.", + ), + limitations: z.array(z.string()), +}); +export type EvidenceRecordSelection = z.infer< + typeof evidenceRecordSelectionSchema +>; + +/** Select one page of complete authenticated records without copying the rest of the ledger. */ +export const selectEvidenceRecords = ( + bundle: EvidenceBundle, + filters: EvidenceRecordFilters = {}, + page: { + readonly offset?: number | undefined; + readonly limit?: number | undefined; + } = {}, +): EvidenceRecordSelection => { + const offset = page.offset ?? 0; + const limit = page.limit ?? EVIDENCE_RECORD_SELECTION_LIMIT; + const records: Evidence[] = []; + let matching = 0; + for (const { evidence } of matchingEvidenceRecords(bundle, filters)) { + if (matching >= offset && records.length < limit) records.push(evidence); + matching += 1; + } + const retained = new Set(bundle.records.map(({ evidence_id: id }) => id)); + const missing = records.flatMap(({ evidence_links: links }) => + links.filter((link) => !retained.has(link)), + ); + const end = offset + records.length; + return { + kind: "evidence-record-selection", + filters: selectedFilters(filters), + total_retained_records: bundle.records.length, + matching_records: matching, + offset, + limit, + next_offset: end < matching ? end : null, + records, + missing_evidence_links: uniqueSorted(missing), + limitations: [...EVIDENCE_RECORD_SELECTION_LIMITATIONS], + }; +}; diff --git a/src/domain/explicitSensitiveFailure.ts b/src/domain/explicitSensitiveFailure.ts index 3237aa96f8..e55be448a7 100644 --- a/src/domain/explicitSensitiveFailure.ts +++ b/src/domain/explicitSensitiveFailure.ts @@ -145,9 +145,22 @@ const redactFailure = ( ? {} : { observed: visit(issue.observed) }), }); + const guidance = error.continuingCallerGuidance; const context = { cause: error, ...(error.limit === undefined ? {} : { limit: error.limit }), + ...(guidance === undefined + ? {} + : { + continuingCallerGuidance: { + ...(guidance.userMessage === undefined + ? {} + : { userMessage: text(guidance.userMessage) }), + ...(guidance.remediationAction === undefined + ? {} + : { remediationAction: text(guidance.remediationAction) }), + }, + }), ...(error.cleanupPrimary === undefined ? {} : { @@ -156,6 +169,9 @@ const redactFailure = ( ...(error.diagnostics === undefined ? {} : { diagnostics: diagnostics(error.diagnostics) }), + ...(error.invocationDiagnostics === undefined + ? {} + : { invocationDiagnostics: diagnostics(error.invocationDiagnostics) }), ...(error.capturedOutput === undefined ? {} : { diff --git a/src/domain/functionDossierEvidence.ts b/src/domain/functionDossierEvidence.ts index ee3ccd92d6..15abd76511 100644 --- a/src/domain/functionDossierEvidence.ts +++ b/src/domain/functionDossierEvidence.ts @@ -1,4 +1,4 @@ -import type { Evidence } from "./evidence.js"; +import { ANALYSIS_EVIDENCE_PREDICATE, type Evidence } from "./evidence.js"; import { AnalysisInputError } from "./analysisErrorCore.js"; import type { EvidenceIntegrityError } from "./evidenceErrors.js"; import { readEvidence, readEvidencePayload } from "./evidenceValidation.js"; @@ -70,7 +70,7 @@ export const parseFunctionEvidence = ( ); if ( evidence.operation !== "analyze_function" || - evidence.predicate_type !== "rea.analysis" + evidence.predicate_type !== ANALYSIS_EVIDENCE_PREDICATE ) { const location = selection.path.map(String).join("."); const subject = @@ -81,7 +81,10 @@ export const parseFunctionEvidence = ( : `Evidence at ${location}`; return invalidSelection( `Function analysis requires analyze_function Evidence with the rea.analysis predicate; the ${subject} has operation ${evidence.operation} and predicate ${evidence.predicate_type}.`, - { operation: "analyze_function", predicate_type: "rea.analysis" }, + { + operation: "analyze_function", + predicate_type: ANALYSIS_EVIDENCE_PREDICATE, + }, ); } if (evidence.subject === null) diff --git a/src/domain/ghidraValues.fixture.ts b/src/domain/ghidraValues.fixture.ts index 255947a1f4..904de4e462 100644 --- a/src/domain/ghidraValues.fixture.ts +++ b/src/domain/ghidraValues.fixture.ts @@ -6,7 +6,7 @@ export const ghidraFunctionClassification = () => ({ }); /** Wire-boundary fixture, not evidence that a real Ghidra run was performed. */ -export const ghidraFunctionBody = () => ({ +const ghidraFunctionBody = () => ({ available: true as const, provenance: "ghidra-function-body-address-set", ranges: [{ start: "0x401000", end: "0x401005" }], diff --git a/src/domain/hopperErrors.ts b/src/domain/hopperErrors.ts index 88a577da9f..fd00aaedce 100644 --- a/src/domain/hopperErrors.ts +++ b/src/domain/hopperErrors.ts @@ -36,6 +36,14 @@ export type HopperLauncherOutcome = { readonly diagnostic_truncated: boolean; }; +/** Startup-timeout recovery that holds after the calling invocation has ended. */ +export const HOPPER_STARTUP_TIMEOUT_REMEDIATION = + "Review details.launcher and inspect Hopper for ongoing initial analysis or a pending loader, demo, or license dialog. For a large target, set REA_HOPPER_STARTUP_TIMEOUT_MS higher in the environment of a new REA invocation, then rerun the command; that invocation starts Hopper again. Resolve retained cleanup before rerunning."; + +/** Startup-timeout recovery for a caller whose REA runtime keeps running. */ +export const HOPPER_STARTUP_TIMEOUT_CONTINUING_REMEDIATION = + "Review details.launcher and inspect Hopper for ongoing initial analysis or a pending loader, demo, or license dialog. For a large target, use update_analysis_settings with setting hopper.startup_timeout_ms and action set for the next startup attempt without restarting REA, or increase REA_HOPPER_STARTUP_TIMEOUT_MS for future REA processes. Allow at least that long in the MCP client. Resolve retained cleanup before reopening."; + /** Hopper did not respond within the configured operation deadline. */ export class HopperTimeoutError extends HopperError { readonly _tag = "HopperTimeoutError"; @@ -53,7 +61,14 @@ export class HopperTimeoutError extends HopperError { ) { super( `Hopper ${operation === undefined ? "startup" : operation} timed out after ${String(timeoutMs)}ms`, - options, + operation === undefined + ? { + ...options, + continuingCallerGuidance: { + remediationAction: HOPPER_STARTUP_TIMEOUT_CONTINUING_REMEDIATION, + }, + } + : options, ); } } diff --git a/src/domain/javascript/htmlArtifactReferences.test.ts b/src/domain/javascript/htmlArtifactReferences.test.ts index 1a3d525288..3381ae95f1 100644 --- a/src/domain/javascript/htmlArtifactReferences.test.ts +++ b/src/domain/javascript/htmlArtifactReferences.test.ts @@ -106,7 +106,7 @@ it.each([ ['', []], ["", []], ["", []], - ["", []], + ["", ["classic"]], ])("classifies inline JavaScript in %s", (text, expected) => { expect( htmlArtifactReferences(text).inlineScripts.map( @@ -115,6 +115,65 @@ it.each([ ).toEqual(expected); }); +it.each([ + ['src="ignored.js"', "classic"], + ['language="vbscript"', "classic"], + ['xlink:type="simple"', "classic"], + ['type=" Text/JavaScript "', "classic"], + ['type="MODULE"', "module"], + ['type=" module "', null], + ['type="application/json"', null], + ['type="text/javascript; charset=utf-8"', null], + ['href="external.js"', null], + ['href=""', null], + ['xlink:href="external.js"', null], +] as const)("classifies SVG inline bodies using %s", (attributes, expected) => { + const result = htmlArtifactReferences( + ``, + ); + expect(result.inlineScripts.map(({ scriptType }) => scriptType)).toEqual( + expected === null ? [] : [expected], + ); + expect(result.inlineScriptCount).toBe(expected === null ? 0 : 1); +}); + +it.each([ + ['href="external.js"', ["external.js"]], + ['xlink:href="external.js"', ["external.js"]], + ['href="first.js" xlink:href="second.js"', ["first.js"]], + ['xlink:href="second.js" href="first.js"', ["first.js"]], + ['href="" xlink:href="ignored.js"', []], + ['other:href="ignored.js"', []], + ['src="ignored.js"', []], +])("extracts SVG script URL from %s", (attributes, paths) => { + const result = htmlArtifactReferences( + ``, + ); + expect(result.scripts.map(({ scriptPath }) => scriptPath)).toEqual(paths); + expect(result.inlineScriptCount).toBe(0); +}); + +it("shares precise body coordinates and location limits across HTML and SVG", () => { + const prefix = '\n`).inlineScripts, + ).toEqual([ + { + scriptType: "classic", + location: { start: { line: 2, column: 30 }, end: { line: 4, column: 0 } }, + }, + ]); + const repeated = "".repeat(60); + const result = htmlArtifactReferences(repeated); + expect(result.inlineScriptCount).toBe(120); + expect(result.inlineScripts).toHaveLength(HTML_INLINE_SCRIPT_LOCATION_LIMIT); + expect( + htmlArtifactReferences("") + .inlineScriptCount, + ).toBe(0); +}); + it("retains exact inline script body coordinates, including unclosed scripts", () => { const prefix = "\n`, + '', + '', + '', + '', + "", + ].join("\n"); + await writeFile(join(root, "index.html"), html); + await writeFile( + join(root, "main.js"), + 'fetch("https://example.test/external");', + ); + const response = await cli.run({ + arguments: [ + "analyze-javascript-application", + root, + "--source-path", + "index.html", + "--source-path", + "main.js", + "--format", + "json", + ], + }); + expect(response.exitCode, response.stderr).toBe(0); + const result = javascriptApplicationAnalysisResultSchema.parse( + parseEvidence(response.json).normalized_result, + ); + const observations = result.graph.nodes.flatMap( + ({ observations }) => observations, + ); + const scopes = observations.filter( + ({ properties }) => properties.operation === "parse-html-inline-scripts", + ); + expect(scopes).toHaveLength(1); + expect(scopes[0]).toMatchObject({ + properties: { + path: "index.html", + inline_script_count: 1, + inline_scripts: [ + { + script_type: "classic", + location: { + start: { line: 2, column: 45 }, + end: { line: 2, column: 86 }, + }, + }, + ], + }, + evidence: { + state: "unavailable", + confidence: "unknown", + coverage: { status: "partial" }, + }, + }); + expect( + result.graph.edges.filter( + ({ relation, properties }) => + relation === "loads" && properties.resolved_path === "main.js", + ), + ).toHaveLength(2); + const endpoints = result.graph.nodes + .filter(({ kind }) => kind === "endpoint") + .flatMap(({ observations }) => observations) + .map(({ properties }) => properties.value); + expect(endpoints).toContain("https://example.test/external"); + expect(endpoints).not.toContain(inlineUrl); + }, +); + cliTest.for([ "invalid-html", "empty-html", diff --git a/tests/acceptance/asarSelectedCompanionCli.test.ts b/tests/acceptance/asarSelectedCompanionCli.test.ts index 0026292553..a31d60d31b 100644 --- a/tests/acceptance/asarSelectedCompanionCli.test.ts +++ b/tests/acceptance/asarSelectedCompanionCli.test.ts @@ -6,6 +6,7 @@ import { symlink, writeFile, } from "node:fs/promises"; +import { createHash } from "node:crypto"; import { join } from "node:path"; import { createPackageWithOptions, extractFile } from "@electron/asar"; import { expect, onTestFinished } from "vitest"; @@ -28,6 +29,7 @@ cliTest.skipIf(process.platform === "win32")( const bytes = Buffer.from("selected native companion"); await mkdir(source); await writeFile(join(source, "native.node"), bytes); + await writeFile(join(source, "packed.txt"), "packed bytes"); await createPackageWithOptions(source, selected, { unpack: "*.node" }); await rename(selected, canonical); await symlink(canonical, selected); @@ -54,11 +56,15 @@ cliTest.skipIf(process.platform === "win32")( const inventory = artifactInventoryResultSchema.parse( inventoryEvidence.normalized_result, ); + const occurrence = inventory.occurrences.find( + ({ logical_path }) => logical_path === "native.node", + ); + expect(occurrence?.hash_status).toBe("verified"); expect( - inventory.occurrences.find( - ({ logical_path }) => logical_path === "native.node", - )?.hash_status, - ).toBe("verified"); + inventory.nodes.find( + ({ artifact_id }) => artifact_id === occurrence?.artifact_id, + )?.sha256, + ).toBe(createHash("sha256").update(bytes).digest("hex")); const extracted = await cli.run({ arguments: ["extract-artifact", selected, "--json"], @@ -74,5 +80,8 @@ cliTest.skipIf(process.platform === "win32")( expect( (await readFile(join(result.output_root, "native.node"))).equals(bytes), ).toBe(true); + expect(await readFile(join(result.output_root, "packed.txt"), "utf8")).toBe( + "packed bytes", + ); }, ); diff --git a/tests/acceptance/bunStandaloneCli.test.ts b/tests/acceptance/bunStandaloneCli.test.ts index 78d0f8a4fc..763b815043 100644 --- a/tests/acceptance/bunStandaloneCli.test.ts +++ b/tests/acceptance/bunStandaloneCli.test.ts @@ -16,7 +16,7 @@ import { } from "../../src/domain/artifactGraph.js"; cliTest( - "inventories and extracts Bun source without executing its carrier", + "inventories and extracts Bun source from its carrier", async ({ cli }) => { const root = await createTestTempDirectory("rea-bun-cli-"); const source = join(root, "app"); diff --git a/tests/acceptance/godotPck.test.ts b/tests/acceptance/godotPck.test.ts index 24a58416fd..d16d4b86b2 100644 --- a/tests/acceptance/godotPck.test.ts +++ b/tests/acceptance/godotPck.test.ts @@ -387,7 +387,7 @@ cliTest( ); cliTest( - "rejects repeated-range expansion before hashing members or creating extraction output", + "rejects repeated-range expansion without creating extraction output", async ({ cli }) => { const root = await createTestTempDirectory("rea-pck-expansion-"); const source = join(root, "aliased.pck"); diff --git a/tests/acceptance/peCertificatesCli.test.ts b/tests/acceptance/peCertificatesCli.test.ts index 860041988e..c068c47ecb 100644 --- a/tests/acceptance/peCertificatesCli.test.ts +++ b/tests/acceptance/peCertificatesCli.test.ts @@ -346,3 +346,46 @@ cliTest.for([ ); }, ); + +cliTest.for([ + [0x13, "REA@Fixture"], + [0x16, "REAéFixture"], +] as const)( + "rejects an invalid certificate name alphabet for tag %i through the public CLI", + async ([tag, name], { cli }) => { + const root = await createTestTempDirectory("rea-pe-cert-name-alphabet-"); + const path = join(root, "name.exe"); + const payload = pkcs7SignedDataFixture(name); + const content = Buffer.from(name, "utf8"); + const marker = Buffer.concat([ + Buffer.from([0x0c, content.length]), + content, + ]); + const at = payload.indexOf(marker); + expect(at).toBeGreaterThanOrEqual(0); + payload[at] = tag; + await writeFile(path, peCertificateFixture(false, payload)); + const invalid = await cli.run({ + arguments: ["inspect-pe-certificates", path, "--json"], + }); + expect(invalid.exitCode).toBe(1); + expect(invalid.json).toMatchObject({ + code: "invalid_request", + details: { operation: "inspect_pe_certificates" }, + }); + // Only restore the string tag: the same bytes are valid UTF8String content. + payload[at] = 0x0c; + await writeFile(path, peCertificateFixture(false, payload)); + const valid = await cli.run({ + arguments: ["inspect-pe-certificates", path, "--json"], + }); + expect(valid.exitCode, valid.stderr).toBe(0); + const evidence = toolContract("inspect_pe_certificates").outputSchema.parse( + valid.json, + ); + expect( + evidence.normalized_result.entries[0]?.pkcs7?.certificates[0]?.issuer + .attributes[0]?.value, + ).toBe(name); + }, +); diff --git a/tests/acceptance/sourcePathFailuresCli.test.ts b/tests/acceptance/sourcePathFailuresCli.test.ts index 1ae2d7c45c..6d894c6205 100644 --- a/tests/acceptance/sourcePathFailuresCli.test.ts +++ b/tests/acceptance/sourcePathFailuresCli.test.ts @@ -6,7 +6,7 @@ import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js"; import { cliTest } from "../support/cli/cliFixture.js"; cliTest.skipIf(process.platform === "win32")( - "reports unresolvable source and scenario selections without permission advice", + "reports unresolvable source and scenario selections with path-specific guidance", async ({ cli }) => { const root = await createTestTempDirectory("rea-source-path-failures-"); await symlink("loop-b", join(root, "loop-a")); diff --git a/tests/boundary/browser/browserFixtureHttpTargets.test.ts b/tests/boundary/browser/browserFixtureHttpTargets.test.ts new file mode 100644 index 0000000000..7619fb9484 --- /dev/null +++ b/tests/boundary/browser/browserFixtureHttpTargets.test.ts @@ -0,0 +1,31 @@ +import { fileURLToPath } from "node:url"; +import { expect } from "vitest"; + +import { processTest } from "../../support/process/processFixture.js"; + +for (const kind of ["proxy", "workers", "storage"] as const) { + processTest( + `${kind} HTTP fixture preserves targets and survives invalid requests`, + async ({ processes }) => { + const result = await processes.run( + process.execPath, + [ + fileURLToPath( + new URL( + "../../fixtures/browserFixtureHttpTargets.mjs", + import.meta.url, + ), + ), + kind, + ], + { timeoutMs: 15_000 }, + ); + expect(result.signal, result.stderr).toBeNull(); + expect(result.exitCode, result.stderr).toBe(0); + expect(result.stderr).toBe(""); + expect(result.stdout).toBe( + `${kind}: target routing, invalid-target recovery and physical close passed\n`, + ); + }, + ); +} diff --git a/tests/boundary/browser/browserStorageListenerScope.test.ts b/tests/boundary/browser/browserStorageListenerScope.test.ts new file mode 100644 index 0000000000..8149ac3cf0 --- /dev/null +++ b/tests/boundary/browser/browserStorageListenerScope.test.ts @@ -0,0 +1,36 @@ +import { networkInterfaces } from "node:os"; +import { fileURLToPath } from "node:url"; +import { expect } from "vitest"; + +import { processTest } from "../../support/process/processFixture.js"; + +const hasNonLoopbackIpv4 = Object.values(networkInterfaces()) + .flatMap((entries) => entries ?? []) + .some((entry) => entry.family === "IPv4" && !entry.internal); + +processTest.skipIf(!hasNonLoopbackIpv4)( + "storage fixture serves localhost and refuses this host's non-loopback interface", + async ({ processes }) => { + const result = await processes.run( + process.execPath, + [ + fileURLToPath( + new URL( + "../../fixtures/browserFixtureHttpTargets.mjs", + import.meta.url, + ), + ), + "storage", + "listener-scope", + ], + { timeoutMs: 15_000 }, + ); + expect(result.signal, result.stderr).toBeNull(); + expect(result.exitCode, result.stderr).toBe(0); + expect(result.stderr).toBe(""); + expect(result.stdout).toBe( + "storage: localhost served; non-loopback connection refused\n" + + "storage: target routing, invalid-target recovery and physical close passed\n", + ); + }, +); diff --git a/tests/boundary/browser/electronActiveHook.test.ts b/tests/boundary/browser/electronActiveHook.test.ts index 667c555751..aa3ae9f527 100644 --- a/tests/boundary/browser/electronActiveHook.test.ts +++ b/tests/boundary/browser/electronActiveHook.test.ts @@ -3305,6 +3305,188 @@ process.stdout.write(JSON.stringify({ }, ); +it("preserves warning-only rejections and records later blocked shell rejections", async () => { + const root = await createTestTempDirectory( + "rea-electron-hook-rejection-warn-", + ); + temporary.push(root); + await writeElectronFixture(root); + const script = join(root, "warnings.cjs"); + await writeFile( + script, + `const { shell } = require("electron"); +Promise.reject(new Error("controlled rejection")); +setImmediate(() => { + shell.openExternal("https://example.invalid"); + setImmediate(() => { + console.log(JSON.stringify({ + listeners: process.listenerCount("unhandledRejection"), + events: globalThis.__reaElectronActiveSnapshot().events.filter(event => + event.kind === "process-lifecycle" && event.event === "unhandled-rejection").map(event => ({ + phase: event.phase, argument_shapes: event.argument_shapes, error: event.error, + })), + })); + }); +}); +`, + ); + const result = await runNode(script, root, undefined, [ + "--unhandled-rejections=warn", + ]); + expect(result.code, result.stderr).toBe(0); + expect(result.stderr).toContain("UnhandledPromiseRejectionWarning"); + expect(JSON.parse(result.stdout)).toEqual({ + listeners: 0, + events: Array.from({ length: 2 }, () => ({ + phase: "observed", + argument_shapes: ["object", "object"], + error: true, + })), + }); +}); + +it("preserves application rejection listeners and borrowed process emit behavior", async () => { + const root = await createTestTempDirectory( + "rea-electron-hook-rejection-listener-", + ); + temporary.push(root); + await writeElectronFixture(root); + const script = join(root, "listeners.cjs"); + await writeFile( + script, + `const { EventEmitter } = require("node:events"); +const reason = new Error("controlled listener reason"); +const promise = Promise.resolve(); +const sentinel = new Error("controlled listener throw"); +let calls = 0; +process.on("unhandledRejection", (actualReason, actualPromise) => { + if (actualReason !== reason || actualPromise !== promise) throw new Error("identity changed"); + calls++; + if (calls === 2) throw sentinel; +}); +const returned = process.emit("unhandledRejection", reason, promise); +let sameThrow = false; +try { process.emit("unhandledRejection", reason, promise); } catch (cause) { sameThrow = cause === sentinel; } +const borrowed = process.emit.call(new EventEmitter(), "unhandledRejection", reason, promise); +console.log(JSON.stringify({ calls, returned, sameThrow, borrowed, + listeners: process.listenerCount("unhandledRejection"), + observed: globalThis.__reaElectronActiveSnapshot().events.filter(event => event.kind === "process-lifecycle" && event.event === "unhandled-rejection").length, +})); +`, + ); + const result = await runNode(script, root); + expect(result.code, result.stderr).toBe(0); + expect(JSON.parse(result.stdout)).toEqual({ + calls: 2, + returned: true, + sameThrow: true, + borrowed: false, + listeners: 1, + observed: 2, + }); +}); + +it("delivers revoked proxy rejections to the application's original handler", async () => { + const root = await createTestTempDirectory( + "rea-electron-hook-revoked-rejection-", + ); + temporary.push(root); + await writeElectronFixture(root); + const script = join(root, "revoked-rejections.cjs"); + await writeFile( + script, + `const entries = [{}, [], () => undefined, []].map((target, index) => { + const { proxy, revoke } = Proxy.revocable(target, {}); + if (index < 3) revoke(); + return { reason: proxy, promise: undefined }; +}); +const received = []; +process.on("unhandledRejection", (reason, promise) => { + received.push(entries.some(entry => entry.reason === reason && entry.promise === promise)); +}); +for (const entry of entries) entry.promise = Promise.reject(entry.reason); +setImmediate(() => console.log(JSON.stringify({ + received, + listeners: process.listenerCount("unhandledRejection"), + shapes: globalThis.__reaElectronActiveSnapshot().events.filter(event => + event.kind === "process-lifecycle" && event.event === "unhandled-rejection").map(event => event.argument_shapes), +}))); +`, + ); + const result = await runNode(script, root); + expect(result.code, result.stderr).toBe(0); + expect(JSON.parse(result.stdout)).toEqual({ + received: [true, true, true, true], + listeners: 1, + shapes: [ + ["object", "object"], + ["object", "object"], + ["function", "object"], + ["array", "object"], + ], + }); +}); + +it("preserves an Electron-style uncaught exception listener policy", async () => { + const root = await createTestTempDirectory( + "rea-electron-hook-uncaught-policy-", + ); + temporary.push(root); + await writeElectronFixture( + root, + electronModuleSource + + ` +process.on("uncaughtException", () => { + if (process.listenerCount("uncaughtException") > 1) return; + console.log(JSON.stringify({ listeners: process.listenerCount("uncaughtException"), + kinds: globalThis.__reaElectronActiveSnapshot().events.filter(event => event.kind === "process-lifecycle").map(event => event.event), + })); +}); +`, + ); + const script = join(root, "uncaught.cjs"); + await writeFile(script, 'throw new Error("controlled uncaught");\n'); + const result = await runNode(script, root); + expect(result.code, result.stderr).toBe(0); + expect(JSON.parse(result.stdout)).toEqual({ + listeners: 1, + kinds: ["uncaught-exception"], + }); +}); + +it.each(["throw", "strict"])( + "preserves fatal rejection policy in %s mode", + async (mode) => { + const root = await createTestTempDirectory( + "rea-electron-hook-rejection-fatal-", + ); + temporary.push(root); + await writeElectronFixture(root); + const script = join(root, "fatal-rejection.cjs"); + await writeFile( + script, + `process.on("exit", () => console.log(JSON.stringify({ + listeners: process.listenerCount("uncaughtException"), + kinds: globalThis.__reaElectronActiveSnapshot().events.filter(event => event.kind === "process-lifecycle").map(event => event.event), +}))); +Promise.reject(new Error("controlled fatal rejection")); +`, + ); + const result = await runNode(script, root, undefined, [ + "--unhandled-rejections=" + mode, + ]); + expect(result.code).toBe(1); + expect(result.stderr).toContain("controlled fatal rejection"); + expect(JSON.parse(result.stdout)).toEqual({ + listeners: 0, + kinds: + mode === "throw" + ? ["unhandled-rejection", "uncaught-exception"] + : ["uncaught-exception"], + }); + }, +); + it("preserves fatal process termination after recording uncaught exceptions", async () => { const root = await createTestTempDirectory("rea-electron-hook-fatal-"); temporary.push(root); @@ -3324,6 +3506,101 @@ it("preserves fatal process termination after recording uncaught exceptions", as expect(result.stderr).toContain("fatal-hook-secret"); }); +it.each([{ sandboxPrefix: [] }, { sandboxPrefix: ["--no-sandbox"] }])( + "restores the exact application argv with Playwright prefix %j", + async ({ sandboxPrefix }) => { + const root = await createTestTempDirectory("rea-electron-argv-"); + temporary.push(root); + await writeElectronFixture(root); + const hook = join(root, "electron-active-hook.cjs"); + const callerArguments = [ + "--", + "/caller/plugin.asar", + "--config=/caller/app.asar/settings", + "-r", + hook, + `--require=${hook}`, + "--inspect=0", + "--remote-debugging-port=0", + "/caller/electron-active-hook.cjs", + "", + "中文", + ]; + const plan = ["-r", hook, "/selected/app.asar", ...callerArguments]; + const script = join(root, "argv.cjs"); + await writeFile( + script, + ` + const argv = ["/runner/electron", ...${JSON.stringify(sandboxPrefix)}, + "--inspect=0", "--remote-debugging-port=0", ...${JSON.stringify(plan)}]; + process.argv = argv; + const plan = ${JSON.stringify(plan)}; + process.env.REA_ELECTRON_CAPTURE_ARGV = JSON.stringify({ length: plan.length, + sha256: require("node:crypto").createHash("sha256").update(JSON.stringify(plan)).digest("hex") }); + require("./electron-active-hook.cjs"); + console.log(JSON.stringify({ argv: process.argv, same: process.argv === argv, + handoff_present: "REA_ELECTRON_CAPTURE_ARGV" in process.env, + hook_error: globalThis.__reaElectronActiveSnapshot().hook_error })); + `, + ); + // The bootstrap script establishes Electron's observed argv before loading the hook. + const result = await runNode(script, root, undefined, [], false); + expect(result.code, result.stderr).toBe(0); + expect(JSON.parse(result.stdout.trim())).toEqual({ + argv: ["/runner/electron", "/selected/app.asar", ...callerArguments], + same: true, + handoff_present: false, + hook_error: false, + }); + }, +); + +it.each([ + "absent", + "invalid-json", + "mismatch", + "foreign-hook", + "unknown-prefix", + "no-entry", +])("preserves observed argv for an %s capture handoff", async (mode) => { + const root = await createTestTempDirectory("rea-electron-argv-guard-"); + temporary.push(root); + await writeElectronFixture(root); + const hook = join(root, "electron-active-hook.cjs"); + const plan = ["-r", hook, "/selected/app", "caller-document"]; + const script = join(root, "guard.cjs"); + await writeFile( + script, + ` + const mode = ${JSON.stringify(mode)}; + const plan = ${JSON.stringify(plan)}; + const argv = ["/runner/electron", "--inspect=0", "--remote-debugging-port=0", ...plan]; + process.argv = argv; + if (mode === "unknown-prefix") argv[1] = "--inspect=1234"; + if (mode === "mismatch") argv[argv.length - 1] = "another-document"; + if (mode === "foreign-hook") plan[1] = "/other/electron-active-hook.cjs"; + if (mode === "no-entry") plan.length = 2; + if (mode === "absent") delete process.env.REA_ELECTRON_CAPTURE_ARGV; + else process.env.REA_ELECTRON_CAPTURE_ARGV = mode === "invalid-json" ? "{" : JSON.stringify({ + length: plan.length, + sha256: require("node:crypto").createHash("sha256").update(JSON.stringify(plan)).digest("hex") }); + const before = JSON.stringify(argv); + require("./electron-active-hook.cjs"); + console.log(JSON.stringify({ unchanged: JSON.stringify(process.argv) === before, + same: process.argv === argv, handoff_present: "REA_ELECTRON_CAPTURE_ARGV" in process.env, + hook_error: globalThis.__reaElectronActiveSnapshot().hook_error })); + `, + ); + const result = await runNode(script, root, undefined, [], false); + expect(result.code, result.stderr).toBe(0); + expect(JSON.parse(result.stdout.trim())).toEqual({ + unchanged: true, + same: true, + handoff_present: false, + hook_error: mode !== "absent", + }); +}); + const writeElectronFixture = async ( root: string, source = electronModuleSource, @@ -3359,6 +3636,8 @@ const runNode = async ( script: string, root: string, sourceHook?: string, + nodeArguments: readonly string[] = [], + preloadHook = true, ): Promise<{ readonly code: number | null; readonly stdout: string; @@ -3366,11 +3645,15 @@ const runNode = async ( }> => { const hook = await stageHook(root, sourceHook); return new Promise((resolve, reject) => { - const child = spawn(process.execPath, ["-r", hook, script], { - cwd: root, - env: process.env, - stdio: ["ignore", "pipe", "pipe"], - }); + const child = spawn( + process.execPath, + [...nodeArguments, ...(preloadHook ? ["-r", hook] : []), script], + { + cwd: root, + env: process.env, + stdio: ["ignore", "pipe", "pipe"], + }, + ); const stdout: Buffer[] = []; const stderr: Buffer[] = []; child.stdout.on("data", (chunk: Buffer) => stdout.push(chunk)); diff --git a/tests/boundary/cli/ampRegistrationPreservation.test.ts b/tests/boundary/cli/ampRegistrationPreservation.test.ts index 97d0842a10..507b8af71f 100644 --- a/tests/boundary/cli/ampRegistrationPreservation.test.ts +++ b/tests/boundary/cli/ampRegistrationPreservation.test.ts @@ -58,6 +58,7 @@ cliTest( expect(await readFile(registrationBackupPath, "utf8")).toBe(original); expect(refreshed.stderr).toContain(registrationBackupPath); expect(after.theme).toBe("dark"); + expect(after["amp.mcpServers"].other).toEqual({ command: "other" }); // The equals form --agent=amp is parsed as a spaced form before Incur sees // it, so the Amp registration lands and keeps the environment map. @@ -69,6 +70,7 @@ cliTest( command: expect.any(String), env: { KEEP_ME: "1" }, }); + expect(afterEquals["amp.mcpServers"].other).toEqual({ command: "other" }); expect(await readFile(registrationBackupPath, "utf8")).toBe(original); const recoveryBackupPath = equals.stderr.match( /Amp MCP registration backup retained at (.+)/, diff --git a/tests/boundary/cli/analysisViewCli.test.ts b/tests/boundary/cli/analysisViewCli.test.ts index 5b462c3b67..c0c10c3246 100644 --- a/tests/boundary/cli/analysisViewCli.test.ts +++ b/tests/boundary/cli/analysisViewCli.test.ts @@ -46,6 +46,72 @@ import { } from "../../../src/domain/evidenceBundle.js"; import { inspectEvidenceBundle } from "../../../src/application/investigation/InspectEvidenceBundle.js"; +cliTest( + "reads selected batch rows and one body from a saved bundle without a provider", + async ({ cli }) => { + const root = await createTestTempDirectory("rea-batch-view-cli-"); + const path = join(root, "bundle.json"); + const body = "int main(void) { return 0; }"; + const parent = createEvidence( + undefined, + { id: "fixture", name: "Fixture", version: "1" }, + { + operation: "batch_decompile", + parameters: { addresses: ["main", "main"] }, + result: { + items: ["main", "main"].map((address) => ({ + address, + procedure: { status: "resolved", address: "0x1", name: "main" }, + status: "ok", + pseudocode: body, + })), + total: 2, + succeeded: 2, + failed: 0, + }, + confidence: "derived", + authority: "shipped-artifact", + }, + ); + await writeFile(path, JSON.stringify(createEvidenceBundle([parent]))); + const inspect = async (view: JsonValue) => { + const result = await cli.run({ + arguments: [ + "inspect-analysis-view", + JSON.stringify({ + source: { + kind: "retained-evidence", + evidence_id: parent.evidence_id, + }, + view, + }), + "--evidence-bundle", + path, + "--json", + ], + }); + expect(result.exitCode).toBe(0); + return parseEvidence(result.json).normalized_result; + }; + expect( + await inspect({ kind: "batch-decompile", collection: "items" }), + ).toMatchObject({ + batch: { total: 2 }, + items: [ + { index: 0, pseudocode_code_units: body.length }, + { index: 1, pseudocode_code_units: body.length }, + ], + }); + expect( + await inspect({ + kind: "batch-decompile", + collection: "pseudocode", + index: 1, + }), + ).toMatchObject({ item: { index: 1, text: body } }); + }, +); + cliTest( "authenticates an exact portable parent for non-managed views and rejects ambiguous or altered sources", async ({ cli }) => { @@ -622,6 +688,24 @@ cliTest( const expected = inspectEvidenceBundle(bundle, options); if (!expected.ok) throw expected.error; expect(result.json).toEqual(expected.value); + // The same shared owner selects the complete record from the portable file. + const records = await cli.run({ + arguments: [ + "inspect-evidence-bundle", + JSON.stringify({ path, ...options, detail: "records", limit: 1 }), + "--json", + ], + environment: { HOME: root, XDG_CONFIG_HOME: root, XDG_CACHE_HOME: root }, + }); + expect(records.exitCode).toBe(0); + expect(records.json).toMatchObject({ + kind: "evidence-record-selection", + matching_records: 1, + limit: 1, + next_offset: null, + records: [parent], + missing_evidence_links: [], + }); const complete = await cli.run({ arguments: [ "inspect-evidence-bundle", diff --git a/tests/boundary/cli/bundleComparisonCli.test.ts b/tests/boundary/cli/bundleComparisonCli.test.ts index 809ae9b8aa..fcf164b79b 100644 --- a/tests/boundary/cli/bundleComparisonCli.test.ts +++ b/tests/boundary/cli/bundleComparisonCli.test.ts @@ -91,7 +91,11 @@ cliTest( const display = await compare(displayPath); expect(display.exitCode).toBe(0); const displayResult = bundleComparisonResultSchema.parse(display.json); - expect(displayResult).toMatchObject({ status: "unchanged", changes: [] }); + expect(displayResult).toMatchObject({ + status: "unchanged", + summary: { records_unchanged: 1, unresolved: 0 }, + changes: [], + }); expect(displayResult.left_bundle_sha256).not.toBe( displayResult.right_bundle_sha256, ); diff --git a/tests/boundary/cli/cliEvidenceCallerPath.test.ts b/tests/boundary/cli/cliEvidenceCallerPath.test.ts index 5c2754d63c..e2c1d08ad3 100644 --- a/tests/boundary/cli/cliEvidenceCallerPath.test.ts +++ b/tests/boundary/cli/cliEvidenceCallerPath.test.ts @@ -55,7 +55,12 @@ it("binds authenticated partial Evidence but preserves raw partial observations" sha256: "a".repeat(64), bytes: 1, }, - { kind: "file-snapshot-hash", path: "/tmp/selected", bytes: 1 }, + { + kind: "file-snapshot-hash", + path: "/tmp/selected", + sha256: "a".repeat(64), + bytes: 1, + }, ], ] as const) { const result = await finishCliEvidenceCommand( @@ -74,5 +79,7 @@ it("binds authenticated partial Evidence but preserves raw partial observations" expect(result.ok).toBe(false); if (result.ok) throw new Error("Expected cancellation"); expect(result.error.details?.partial_observation).toMatchObject(expected); + if ("kind" in expected) + expect(result.error.details?.partial_observation).toEqual(expected); } }); diff --git a/tests/boundary/cli/cliInputSchemaRequired.test.ts b/tests/boundary/cli/cliInputSchemaRequired.test.ts index 1a65c77a51..37d3a0d407 100644 --- a/tests/boundary/cli/cliInputSchemaRequired.test.ts +++ b/tests/boundary/cli/cliInputSchemaRequired.test.ts @@ -85,7 +85,11 @@ const registerParserAdvertisementInventoryTests = (): void => { it("require exactly the inputs whose omission the parser rejects", async () => { const cli = createCli({}); const mismatches: string[] = []; - for (const command of createCliInventory(cli).primary) { + const primary = createCliInventory(cli).primary; + expect(primary).toEqual( + expect.arrayContaining(["inspect-pcap-client", "inspect-pcap-capture"]), + ); + for (const command of primary) { const advertised = await advertisedSchema(cli, command); for (const group of INPUT_GROUPS) { const parser = parserSchema(cli, command, group); diff --git a/tests/boundary/cli/cliWorkerCompletion.test.ts b/tests/boundary/cli/cliWorkerCompletion.test.ts index dc879f65a0..19a6a72f7b 100644 --- a/tests/boundary/cli/cliWorkerCompletion.test.ts +++ b/tests/boundary/cli/cliWorkerCompletion.test.ts @@ -4,6 +4,8 @@ import { fileURLToPath } from "node:url"; import { expect } from "vitest"; import { z } from "zod"; +import { createDeferred } from "../../fixtures/binarySession.js"; +import { awaitAcknowledgement } from "../../fixtures/lifecycleAcknowledgement.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { createCliSessionFrameReader, @@ -41,22 +43,45 @@ process.stdout.write = (chunk, ...arguments_) => { }; `, ); - const result = await processes.run( - process.execPath, - ["--import", preload, worker], - { - input: encodeCliSessionFrame({ - id: 1, - arguments: ["--help"], - cwd: root, - env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent" }, - }), - timeoutMs: 15_000, - }, + const child = processes.spawn(process.execPath, [ + "--import", + preload, + worker, + ]); + const closed = new Promise((resolve) => { + child.once("close", () => resolve()); + }); + let stderr = ""; + child.stderr.on("data", (chunk: Buffer) => { + stderr += chunk.toString("utf8"); + }); + const ready = createDeferred(); + child.stdout.on( + "data", + createCliSessionFrameReader((value: unknown) => { + const frame = z.object({ ready: z.boolean() }).parse(value); + if (frame.ready) ready.resolve(); + }), ); - expect(result.signal, result.stderr).toBeNull(); - expect(result.exitCode, result.stderr).toBe(1); - expect(result.stderr).toContain("selected CLI response frame write failed"); + await awaitAcknowledgement( + ready.promise, + closed, + "CLI worker startup", + 15_000, + ); + child.stdin.end( + encodeCliSessionFrame({ + id: 1, + arguments: ["--help"], + cwd: root, + env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent" }, + }), + ); + expect(await waitForExit(child, 15_000), stderr).toBe(true); + await closed; + expect(child.signalCode, stderr).toBeNull(); + expect(child.exitCode, stderr).toBe(1); + expect(stderr).toContain("selected CLI response frame write failed"); }, ); diff --git a/tests/boundary/cli/cutterVerifierPlatform.test.ts b/tests/boundary/cli/cutterVerifierPlatform.test.ts new file mode 100644 index 0000000000..1bef958449 --- /dev/null +++ b/tests/boundary/cli/cutterVerifierPlatform.test.ts @@ -0,0 +1,81 @@ +import { execFile } from "node:child_process"; +import { readFile, readdir, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { promisify } from "node:util"; +import { expect, it } from "vitest"; +import { parse } from "yaml"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const execute = promisify(execFile); + +it("enforces workflow platform labels before contacting Cutter", async () => { + const workflow = parse( + await readFile(".github/workflows/real-cutter.yml", "utf8"), + ); + const labels = workflow.jobs["cutter-upstream"].strategy.matrix.include.map( + (entry: { platform: string }) => entry.platform, + ); + expect(labels).toEqual(["linux", "macOS", "Windows"]); + const root = await createTestTempDirectory("rea-cutter-platform-"); + const preload = join(root, "preload.mjs"); + await writeFile( + preload, + ` +import childProcess from 'node:child_process'; +import { syncBuiltinESMExports } from 'node:module'; +Object.defineProperty(process, 'platform', { value: process.env.REA_PROBE_PLATFORM }); +childProcess.execFile = () => { throw new Error('fixture discovery reached'); }; +syncBuiltinESMExports(); +`, + ); + const check = async (selected: string | undefined, actual: string) => { + const env: NodeJS.ProcessEnv = { + ...process.env, + TMPDIR: root, + TMP: root, + TEMP: root, + REA_PROBE_PLATFORM: actual, + }; + if (selected === undefined) delete env.EXPECTED_PLATFORM; + else env.EXPECTED_PLATFORM = selected; + const result: unknown = await execute( + process.execPath, + ["--import", preload, "scripts/verify-real-cutter.mjs"], + { env, timeout: 10_000 }, + ).then( + () => { + throw new Error("Fixture discovery should stop the verifier"); + }, + (error: unknown) => error, + ); + if ( + !(result instanceof Error) || + !("stderr" in result) || + typeof result.stderr !== "string" + ) + throw new Error("Missing verifier diagnostics"); + return result.stderr; + }; + for (const [selected, actual] of [ + ["linux", "linux"], + ["macOS", "darwin"], + ["Windows", "win32"], + ] as const) + expect(await check(selected, actual)).toContain( + "fixture discovery reached", + ); + expect(await check("linux", "darwin")).toContain( + "Cutter runner did not match its declared platform", + ); + expect(await check("unknown", "linux")).toContain( + "Unrecognized Cutter runner platform", + ); + expect(await check(undefined, "linux")).toContain( + "fixture discovery reached", + ); + expect( + (await readdir(root)).filter((name) => + name.startsWith("rea-cutter-smoke-"), + ), + ).toEqual([]); +}); diff --git a/tests/boundary/cli/evidenceBundleCommands.test.ts b/tests/boundary/cli/evidenceBundleCommands.test.ts index d21424da9b..778f867692 100644 --- a/tests/boundary/cli/evidenceBundleCommands.test.ts +++ b/tests/boundary/cli/evidenceBundleCommands.test.ts @@ -475,7 +475,7 @@ cliTest( ); cliTest( - "diagnoses an obsolete explicit-null Evidence commitment without claiming the file was modified", + "diagnoses an obsolete explicit-null Evidence commitment", async ({ cli }) => { const root = await createTestTempDirectory("rea-legacy-coverage-identity-"); const path = join(root, "legacy.json"); diff --git a/tests/boundary/cli/pcapCli.test.ts b/tests/boundary/cli/pcapCli.test.ts index aa4986bd3b..e042f4985b 100644 --- a/tests/boundary/cli/pcapCli.test.ts +++ b/tests/boundary/cli/pcapCli.test.ts @@ -2,15 +2,6 @@ import { expect, it } from "vitest"; import { expectConfigurationRefusal, serveCli } from "./cliServeHarness.js"; -it.each(["inspect-pcap-client", "inspect-pcap-capture"])( - "%s advertises a schema through the public CLI", - async (command) => { - const result = await serveCli([command, "--schema", "--json"]); - expect(result.exitCode).toBe(0); - expect(JSON.parse(result.stdout)).toBeTruthy(); - }, -); - it.each([ ["inspect-pcap-client"], ["inspect-pcap-capture", "/nonexistent/capture.pcap"], diff --git a/tests/boundary/cli/procedureAliasSearchCli.test.ts b/tests/boundary/cli/procedureAliasSearchCli.test.ts new file mode 100644 index 0000000000..22a3e29d1a --- /dev/null +++ b/tests/boundary/cli/procedureAliasSearchCli.test.ts @@ -0,0 +1,87 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { describe, expect } from "vitest"; + +import { elf } from "../../../src/domain/binaryTarget.fixture.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { cliTest } from "../../support/cli/cliFixture.js"; +import { writeFakeKuna } from "../providers/kuna/fakeKuna.js"; + +describe("procedure alias search CLI", () => { + cliTest( + "finds and traces a procedure by an alias exact lookup accepts", + async ({ cli }) => { + const root = await createTestTempDirectory("rea-alias-search-cli-"); + const kuna = await writeFakeKuna(root, { + functions: [ + { name: "main", addr: "0x1000" }, + { + name: "audit_v1", + addr: "0x1334", + aliases: ["audit_transform", "sub_1334"], + }, + { + name: "audit_v2", + addr: "0x133a", + aliases: ["audit_transform", "sub_133a"], + }, + ], + }); + const target = join(root, "target.elf"); + await writeFile(target, elf(2, 1, 62)); + const run = async (...arguments_: string[]) => { + const result = await cli.run({ + arguments: [...arguments_, "--provider", "kuna", "--format", "json"], + cwd: root, + environment: { + REA_LOG_LEVEL: "silent", + REA_KUNA_COMMAND: kuna, + REA_KUNA_CACHE_DIR: join(root, "cache"), + }, + }); + expect(result.exitCode, JSON.stringify(result.json)).toBe(0); + return result.json; + }; + + expect( + await run( + "search", + target, + "audit_transform", + "--kind", + "procedures", + "--mode", + "regex", + ), + ).toMatchObject({ + operation: "search_procedures", + normalized_result: [ + { + address: "0x1334", + value: "audit_v1", + matched_aliases: ["audit_transform"], + }, + { + address: "0x133a", + value: "audit_v2", + matched_aliases: ["audit_transform"], + }, + ], + }); + expect(await run("trace", target, "sub_1334")).toMatchObject({ + operation: "trace_feature", + normalized_result: { + matches: [ + { + type: "procedure", + address: "0x1334", + value: "audit_v1", + matched_aliases: ["sub_1334"], + }, + ], + references: [{ target_address: "0x1334", source_address: "0x1004" }], + }, + }); + }, + ); +}); diff --git a/tests/boundary/cli/providerSelectionRemediationCli.test.ts b/tests/boundary/cli/providerSelectionRemediationCli.test.ts new file mode 100644 index 0000000000..53ade79c81 --- /dev/null +++ b/tests/boundary/cli/providerSelectionRemediationCli.test.ts @@ -0,0 +1,51 @@ +import { expect } from "vitest"; + +import { parseConfig } from "../../../src/config/parseConfig.js"; +import { elf } from "../../../src/domain/binaryTarget.fixture.js"; +import { IdaProvider } from "../../../src/ida/IdaProvider.js"; +import { workspaceCliTest } from "../../support/cli/workspaceCliFixture.js"; + +workspaceCliTest( + "returns an unready IDA registration's repair for an explicit --provider selection", + async ({ cli, workspace }) => { + const registration = workspace.path("missing-ida.json"); + const target = await workspace.write("target.elf", elf(2, 1, 62)); + // The readiness owner is the oracle for the repair the caller receives. + const config = parseConfig({ REA_IDA_MCP_CONFIG: registration }); + if (!config.ok) throw config.error; + const ida = new IdaProvider(config.value, {}, () => { + throw new Error("Readiness must not connect to IDA"); + }); + const owner = await ida.inspectAvailability(); + const released = await ida.closeOwnedResources(); + if (!released.ok) throw released.error; + if (owner.status !== "unavailable") + throw new Error("A missing registration must be unavailable"); + + const result = await cli.run({ + arguments: ["decompile", target, "main", "--provider", "ida", "--json"], + environment: { + ...workspace.environment, + REA_IDA_MCP_CONFIG: registration, + REA_LOG_LEVEL: "silent", + }, + }); + expect(result.exitCode, result.stderr).toBe(1); + expect(result.json).toMatchObject({ + remediation: { action: owner.remediation }, + details: { + selection_reason: "provider_unavailable", + requested_provider_id: "ida", + rejections: expect.arrayContaining([ + expect.objectContaining({ + provider_id: "ida", + origin: "availability", + reason: owner.reason, + remediation: owner.remediation, + }), + ]), + }, + }); + expect(owner.remediation).toContain("REA_IDA_MCP_CONFIG"); + }, +); diff --git a/tests/boundary/cli/reverseEngineeringCommands.test.ts b/tests/boundary/cli/reverseEngineeringCommands.test.ts index ee9a7d5eab..77a7ceef67 100644 --- a/tests/boundary/cli/reverseEngineeringCommands.test.ts +++ b/tests/boundary/cli/reverseEngineeringCommands.test.ts @@ -373,8 +373,9 @@ it("does not project cancellation cleanup as incomplete after a successful retry }); it("returns cancellation with the completed observation when the caller interrupts service close", async () => { + const evidence = observation("complete"); const service = { - inspectWithObjdump: vi.fn(async () => ok(observation("complete"))), + inspectWithObjdump: vi.fn(async () => ok(evidence)), close: vi.fn(async () => { process.emit("SIGINT"); }), @@ -382,7 +383,10 @@ it("returns cancellation with the completed observation when the caller interrup const result = await runObjdump(service); - expect(result).toMatchObject({ code: "cancelled" }); + expect(result).toMatchObject({ + code: "cancelled", + details: { partial_observation: evidence }, + }); expect(process.exitCode).toBe(130); }); diff --git a/tests/boundary/cli/traceNativeValuesCompletenessCli.test.ts b/tests/boundary/cli/traceNativeValuesCompletenessCli.test.ts index ee76f1d2c8..787717e85c 100644 --- a/tests/boundary/cli/traceNativeValuesCompletenessCli.test.ts +++ b/tests/boundary/cli/traceNativeValuesCompletenessCli.test.ts @@ -3,23 +3,31 @@ import { join } from "node:path"; import { Cli } from "incur"; import pino from "pino"; import { expect, it } from "vitest"; +import { z } from "zod"; import { createAnalysisExecution, + type AnalysisClient, + type AnalysisOperationPort, type AnalysisProvider, } from "../../../src/application/AnalysisProvider.js"; import { runDirectAnalysis } from "../../../src/application/DirectAnalysis.js"; import type { DirectAnalysisDependencies } from "../../../src/application/DirectAnalysisDependencies.js"; +import { CLEANUP_STOPPED_VALUE_TRACE } from "../../../src/application/native/NativeValueTrace.js"; import { + failingValueTraceReads, valueTraceDiamond, valueTraceProvider, + VALUE_TRACE_DIAMOND_READS, } from "../../../src/application/native/NativeValueTrace.fixture.js"; import { registerCoreNativeCommands } from "../../../src/cli/coreNativeCommands.js"; import { parseConfig } from "../../../src/config/parseConfig.js"; +import { AnalysisCapabilityUnavailableError } from "../../../src/domain/analysisErrorCore.js"; import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js"; import { parseEvidence } from "../../../src/domain/evidence.js"; import { nativeValueTraceSchema } from "../../../src/domain/native/nativeValueTrace.js"; -import { ok } from "../../../src/domain/result.js"; +import { ProviderCleanupError } from "../../../src/domain/providerCleanupError.js"; +import { err, ok } from "../../../src/domain/result.js"; import { createTestBinarySession } from "../../fixtures/binarySession.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; @@ -33,9 +41,11 @@ const effects = { requiresRoot: false, }; -/** One-shot CLI analysis over the shared trace with a simulated provider. */ -const traceCli = async () => { - const graph = valueTraceDiamond({ sessionSubject: true }); +/** One-shot CLI analysis over a simulated value-trace provider; returns parsed stdout. */ +const serveTraceCli = async ( + port: AnalysisOperationPort, + close: AnalysisClient["close"] = async () => ok(null), +) => { const directory = await createTestTempDirectory("rea-value-trace-cli-"); const path = join(directory, "fixture.hop"); await writeFile(path, "synthetic analysis database"); @@ -61,8 +71,8 @@ const traceCli = async () => { execute: async (operation, parameters, options) => operation === "health" ? ok(createAnalysisExecution(null, valueTraceProvider)) - : graph.port.execute(operation, parameters, options), - close: async () => ok(null), + : port.execute(operation, parameters, options), + close, }), }; const createSession = () => createTestBinarySession(provider); @@ -93,12 +103,30 @@ const traceCli = async () => { exit: () => undefined, }, ); - return nativeValueTraceSchema.parse( - parseEvidence(JSON.parse(stdout)).normalized_result, - ); + return { path, output: JSON.parse(stdout) as unknown }; }; }; +const traceCli = async () => { + const run = await serveTraceCli( + valueTraceDiamond({ sessionSubject: true }).port, + ); + return async (options: readonly string[]) => + nativeValueTraceSchema.parse( + parseEvidence((await run(options)).output).normalized_result, + ); +}; + +const cliFailureSchema = z.object({ + code: z.string(), + details: z + .object({ + resources: z.array(z.string()), + partial_observation: z.unknown(), + }) + .loose(), +}); + it("reports the CLI last page and its observed node budget separately", async () => { const trace = await traceCli(); expect( @@ -135,3 +163,66 @@ it("reports the CLI last page and its observed node budget separately", async () reason: "Node budget reached", }); }); + +it.each([ + { close: "confirmed", resources: ["owned worker"] }, + { close: "unconfirmed", resources: ["owned session", "owned worker"] }, +] as const)( + "stops a CLI trace at a callee's unconfirmed cleanup with $close final close", + async ({ close, resources }) => { + const failure = new AnalysisCapabilityUnavailableError( + valueTraceProvider.id, + "analyze_function", + "controlled child failure", + { + cleanup: { + reason: "cleanup proof refused", + resources: ["owned worker"], + }, + }, + ); + const { port, reads } = failingValueTraceReads( + valueTraceDiamond({ sessionSubject: true }).port, + { "analyze_function:0x2000": failure }, + ); + const run = await serveTraceCli( + port, + close === "confirmed" + ? async () => ok(null) + : async () => + err( + new ProviderCleanupError( + valueTraceProvider.id, + ["owned session"], + {}, + ), + ), + ); + + const { path, output } = await run(["--json"]); + + expect(reads).toEqual(VALUE_TRACE_DIAMOND_READS.slice(0, 4)); + const failed = cliFailureSchema.parse(output); + expect(failed.code).toBe("cleanup_incomplete"); + expect([...failed.details.resources].sort()).toEqual(resources); + const partial = parseEvidence(failed.details.partial_observation); + expect(partial).toMatchObject({ + operation: "trace_native_values", + caller_input_path: path, + parameters: { procedure: "0x1000", max_depth: 1 }, + }); + const admitted = nativeValueTraceSchema.parse(partial.normalized_result); + expect(admitted.procedures.map(({ address }) => address)).toEqual([ + "0x1000", + ]); + expect(admitted.traversal).toEqual({ + status: "truncated", + reasons: ["cleanup_incomplete"], + }); + expect(admitted.unknowns).toContainEqual({ + procedure: "0x2000", + address: null, + reason: CLEANUP_STOPPED_VALUE_TRACE, + }); + }, +); diff --git a/tests/boundary/evm/evmoleInterfaceProvider.test.ts b/tests/boundary/evm/evmoleInterfaceProvider.test.ts index 1f79921ba6..9edd10309e 100644 --- a/tests/boundary/evm/evmoleInterfaceProvider.test.ts +++ b/tests/boundary/evm/evmoleInterfaceProvider.test.ts @@ -40,6 +40,7 @@ it.runIf(unsupportedHost)( encoding: "hex", }); if (result.ok) throw new Error("Expected a host refusal"); + expect(result.error._tag).toBe("AnalysisCapabilityUnavailableError"); const projected = projectAnalysisError(result.error); expect(projected).toMatchObject({ code: "capability_unavailable" }); expect(projected.message).toContain("Linux x64"); @@ -715,21 +716,6 @@ it.runIf(!unsupportedHost)( }, ); -it.runIf(unsupportedHost)( - "reports unverified host coverage before acquiring a provider", - async () => { - expect( - await new EvmoleInterfaceProvider({}).inspect({ - path: "/selected.hex", - encoding: "hex", - }), - ).toMatchObject({ - ok: false, - error: { _tag: "AnalysisCapabilityUnavailableError" }, - }); - }, -); - it.runIf(!unsupportedHost)( "preserves actual snapshot-write cancellation and root cleanup", async () => { diff --git a/tests/boundary/filesystem/apkSignatureViewIdentity.test.ts b/tests/boundary/filesystem/apkSignatureViewIdentity.test.ts index c7f8863133..c5f4331d09 100644 --- a/tests/boundary/filesystem/apkSignatureViewIdentity.test.ts +++ b/tests/boundary/filesystem/apkSignatureViewIdentity.test.ts @@ -26,6 +26,7 @@ it("checks parent subject identity again when signing facts are already cached", if (evidence.subject === null || evidence.subject.format !== "apk") throw new Error("selected file subject missing"); const parent = { + predicateType: evidence.predicate_type, evidenceId: evidence.evidence_id, operation: evidence.operation, normalizedResult: freezeJsonSnapshot(evidence.normalized_result), diff --git a/tests/boundary/filesystem/asarSelectedCompanion.test.ts b/tests/boundary/filesystem/asarSelectedCompanion.test.ts index 9fbcc0f3f2..4b500abdba 100644 --- a/tests/boundary/filesystem/asarSelectedCompanion.test.ts +++ b/tests/boundary/filesystem/asarSelectedCompanion.test.ts @@ -38,40 +38,6 @@ const fixture = async () => { return { root, selected, canonical }; }; -it.skipIf(process.platform === "win32")( - "inventories and extracts the same selected ASAR companion as Electron tooling", - async () => { - const { root, selected } = await fixture(); - expect(extractFile(selected, "native.node").equals(bytes)).toBe(true); - const inventory = await inventoryArtifact(selected); - const occurrence = inventory.occurrences.find( - ({ logical_path }) => logical_path === "native.node", - ); - expect(occurrence?.hash_status).toBe("verified"); - expect( - inventory.nodes.find( - ({ artifact_id }) => artifact_id === occurrence?.artifact_id, - )?.sha256, - ).toBe(createHash("sha256").update(bytes).digest("hex")); - const result = await withArtifactResourceScope((resourceScope) => - extractArtifact({ - inputPath: selected, - inputFormat: "asar", - outputRoot: join(root, "output"), - environment: {}, - integrityPolicy: "fail", - resourceScope, - }), - ); - expect( - (await readFile(join(result.output_root, "native.node"))).equals(bytes), - ).toBe(true); - expect(await readFile(join(result.output_root, "packed.txt"), "utf8")).toBe( - "packed bytes", - ); - }, -); - it.skipIf(process.platform === "win32")( "does not substitute the canonical archive companion for an absent selected companion", async () => { diff --git a/tests/boundary/filesystem/bufferDeepEqualityLint.test.ts b/tests/boundary/filesystem/bufferDeepEqualityLint.test.ts index 38c433b070..4637774b26 100644 --- a/tests/boundary/filesystem/bufferDeepEqualityLint.test.ts +++ b/tests/boundary/filesystem/bufferDeepEqualityLint.test.ts @@ -63,14 +63,6 @@ new RuleTester({ expect(bytes).toEqual(value);`, ], invalid: [ - { - // Exact pre-fix ELF test shape: readFile without an encoding returns Buffer. - code: `import { readFile } from "node:fs/promises"; - import { expect } from "vitest"; - const before = await readFile(path); - expect(await readFile(path)).toEqual(before);`, - errors: [{ messageId: "nativeBytes" }], - }, { code: `import { readFileSync as read } from "node:fs"; import { expect } from "vitest"; diff --git a/tests/boundary/filesystem/ciCacheRetention.test.ts b/tests/boundary/filesystem/ciCacheRetention.test.ts index d455b83500..571d05bd22 100644 --- a/tests/boundary/filesystem/ciCacheRetention.test.ts +++ b/tests/boundary/filesystem/ciCacheRetention.test.ts @@ -269,13 +269,20 @@ describe("retention orchestration", () => { it("deletes nothing when a later list is incomplete", async () => { const deleted: number[] = []; - const page = Array.from({ length: 100 }, (_, index) => - cache(index + 1, `${prefix}${String(index)}`, "2026-10-20T01:00:00Z"), - ); + const requests: string[] = []; + const page = [ + cache(1, `${prefix}old`, "2026-10-20T01:00:00Z"), + cache(2, `${prefix}mid`, "2026-10-20T02:00:00Z"), + cache(3, `${prefix}new`, "2026-10-20T03:00:00Z"), + ]; await expect( retainCaches({ repository, request: async (_method, endpoint) => { + requests.push( + new URL(endpoint, "https://example.test").searchParams.get("key") ?? + "", + ); if (endpoint.includes("npm-download-")) throw new Error("list failed after the first family"); return { actions_caches: page }; @@ -283,10 +290,39 @@ describe("retention orchestration", () => { deleteCache: async (id) => { deleted.push(id); }, - families: [{ prefix, keepKey: `${prefix}0` }], + families: [{ prefix, keepKey: `${prefix}new` }], dedupeNpmDownloads: true, }), - ).rejects.toThrow(/list failed|exceeded/u); + ).rejects.toThrow("list failed after the first family"); + expect(requests).toEqual([prefix, "npm-download-"]); + expect(deleted).toEqual([]); + }); +}); + +describe("cache listing and retention", () => { + const repository = "morluto/rea"; + const prefix = "rea-static-v1-Linux-X64-"; + + it("deletes nothing when the cache listing reaches the page cap", async () => { + const deleted: number[] = []; + let requests = 0; + const page = Array.from({ length: 100 }, (_, index) => + cache(index + 1, `${prefix}${String(index)}`, "2026-10-20T01:00:00Z"), + ); + await expect( + retainCaches({ + repository, + request: async () => { + requests += 1; + return { actions_caches: page }; + }, + deleteCache: async (id) => { + deleted.push(id); + }, + families: [{ prefix, keepKey: `${prefix}0` }], + }), + ).rejects.toThrow(/exceeded.*pages/u); + expect(requests).toBe(10); expect(deleted).toEqual([]); }); diff --git a/tests/boundary/filesystem/ciChangeScope.test.ts b/tests/boundary/filesystem/ciChangeScope.test.ts index 1c5b9908a5..03b936b5d0 100644 --- a/tests/boundary/filesystem/ciChangeScope.test.ts +++ b/tests/boundary/filesystem/ciChangeScope.test.ts @@ -1396,6 +1396,14 @@ it.each([ ".github/workflows/pr-readiness.yml", { workflows: true, code: true, package: false, website: false }, ], + [ + ".github/workflows/codeql.yml", + { workflows: true, code: true, runtime: false, package: false }, + ], + [ + ".github/codeql/production.yml", + { workflows: true, code: true, runtime: false, package: false }, + ], [ ".github/workflows/real-ghidra-windows.yml", { workflows: true, code: true, package: false, website: false }, diff --git a/tests/boundary/filesystem/ciCodeqlScope.test.ts b/tests/boundary/filesystem/ciCodeqlScope.test.ts new file mode 100644 index 0000000000..0374672db7 --- /dev/null +++ b/tests/boundary/filesystem/ciCodeqlScope.test.ts @@ -0,0 +1,136 @@ +import { readFile } from "node:fs/promises"; +import { matchesGlob } from "node:path"; +import { expect, it } from "vitest"; +import { parse } from "yaml"; +import { z } from "zod"; + +const github = new URL("../../../.github/", import.meta.url); +const configuration = z + .strictObject({ + name: z.string(), + "threat-models": z.array(z.string()), + paths: z.array(z.string()), + "paths-ignore": z.array(z.string()), + }) + .parse( + parse(await readFile(new URL("codeql/production.yml", github), "utf8")), + ); + +const excluded = (path: string): boolean => + configuration["paths-ignore"].some( + (pattern) => + path === pattern || + path.startsWith(`${pattern}/`) || + matchesGlob(path, pattern), + ); + +it.each([ + "tests/fixtures/fakeHopper.mjs", + "tests/conformance/ghidra/riscv-checks.mjs", + "tests/support/analysis-efficiency/resource-run.py", + "scripts/fixtures/http3-lab.py", + "scripts/fixtures/managed/pe.mjs", + "src/process/ExecFileOutput.test.ts", + "src/process/ProcessOwnership.fixture.ts", + "scripts/lib/http3-verifier-lifecycle.test.mjs", + "bridge/pwndbg/test_core_context.py", + "bridge/example_test.py", +])("excludes test-only input %s", (path) => { + expect(excluded(path)).toBe(true); +}); + +it.each([ + "src/process/ExecFileOutput.ts", + "src/filesystem/RegularFile.ts", + "src/application/ClientConfigurationPublication.ts", + "bridge/pwndbg/launch.py", + "bridge/pwntools/layout_impl.py", + "bridge/native/rea_lldb_tracer.py", + "bridge/ghidra/ReaGhidraBridge.java", + "install.sh", + "scripts/rea.mjs", + "scripts/prepack.mjs", + "scripts/hopper-demo-x11.py", + "scripts/ci/release-recovery.mjs", + "scripts/verify-package-install.mjs", + "scripts/verify-package-update.mjs", + "scripts/verify-windows-native-artifact.mjs", + "docs/.vitepress/config.ts", + ".github/workflows/release.yml", + ".github/actions/publish-mcp/action.yml", +])("retains production or release input %s", (path) => { + expect(excluded(path)).toBe(false); +}); + +it("retains all packaged entrypoints and the existing local-input threat model", async () => { + const manifest = z + .object({ files: z.array(z.string()) }) + .parse( + JSON.parse( + await readFile( + new URL("../../../package.json", import.meta.url), + "utf8", + ), + ), + ); + expect(manifest.files.filter(excluded)).toEqual([]); + expect(configuration["threat-models"]).toEqual(["local"]); + // A narrow allowlist would silently drop new production roots. Explicitly + // traverse the hidden configuration directory without excluding the root. + expect(configuration.paths).toContain("."); + expect(configuration.paths).toContain("docs/.vitepress"); +}); + +it("uses the scope in a bounded workflow gated on administrator activation", async () => { + const workflow = z + .object({ + permissions: z.object({ contents: z.literal("read") }), + jobs: z.object({ + analyze: z.object({ + if: z.string(), + permissions: z.object({ + contents: z.literal("read"), + "security-events": z.literal("write"), + }), + strategy: z.object({ + "max-parallel": z.literal(1), + matrix: z.object({ language: z.array(z.string()) }), + }), + steps: z.array( + z.object({ + uses: z.string(), + with: z.record(z.string(), z.unknown()).optional(), + }), + ), + }), + }), + }) + .parse( + parse(await readFile(new URL("workflows/codeql.yml", github), "utf8")), + ); + const job = workflow.jobs.analyze; + expect(job.if).toBe("vars.REA_CODEQL_ADVANCED == 'true'"); + expect(job.strategy.matrix.language).toEqual([ + "javascript-typescript", + "python", + "actions", + ]); + const init = job.steps.find((step) => + step.uses.startsWith("github/codeql-action/init@"), + ); + expect(init?.with).toMatchObject({ + "config-file": ".github/codeql/production.yml", + "build-mode": "none", + threads: 2, + ram: 4096, + }); + const analyze = job.steps.find((step) => + step.uses.startsWith("github/codeql-action/analyze@"), + ); + expect(analyze?.with).toMatchObject({ + category: "/language:${{ matrix.language }}", + threads: 2, + ram: 4096, + }); + expect(analyze?.with).not.toHaveProperty("upload"); +}); diff --git a/tests/boundary/filesystem/ciFeedback.test.ts b/tests/boundary/filesystem/ciFeedback.test.ts index d05c7c93d4..48530fd75f 100644 --- a/tests/boundary/filesystem/ciFeedback.test.ts +++ b/tests/boundary/filesystem/ciFeedback.test.ts @@ -12,7 +12,6 @@ import { parseStaticTaskContext, recordStaticTaskContext, } from "../../../scripts/ci/static-task-context.mjs"; -import { createGhApiRequest } from "../../../scripts/ci/github-api.mjs"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; const execute = promisify(execFile); @@ -339,17 +338,6 @@ it("records the actual static task graph with explicit source identity and no am expect(() => parseStaticTaskContext(invalid)).toThrow(); }); -it("rejects mutation methods at the shared GitHub transport before invoking the CLI", async () => { - const request = createGhApiRequest({ env: { PATH: "" } }); - await expect( - request("POST", "repos/owner/repo/actions/runs/1/rerun", { - requests: 0, - startedAt: Date.now(), - limitMs: 1000, - }), - ).rejects.toThrow("read-only GET"); -}); - it("collects basic CI telemetry before optional dependency installation can fail", async () => { const { jobs } = z .object({ diff --git a/tests/boundary/filesystem/ciJobTimingsBatch.test.ts b/tests/boundary/filesystem/ciJobTimingsBatch.test.ts index e0dcecc824..c3ea01bb55 100644 --- a/tests/boundary/filesystem/ciJobTimingsBatch.test.ts +++ b/tests/boundary/filesystem/ciJobTimingsBatch.test.ts @@ -91,15 +91,20 @@ it("collects the latest attempt for recent completed CI and Release runs into a headBranch: null, }, ]); - expect(manifest.runs).toEqual( - expect.arrayContaining([ - expect.objectContaining({ - status: "collected", - rawPath: expect.any(String), - reportPath: expect.any(String), - }), - ]), - ); + for (const { id, runAttempt, status, rawPath, reportPath } of manifest.runs) { + const directory = join( + fixture.outputDirectory, + "runs", + `${id}-${runAttempt}`, + ); + expect({ status, rawPath, reportPath }).toEqual({ + status: "collected", + rawPath: join(directory, "jobs.json"), + reportPath: join(directory, "report.json"), + }); + await expect(readFile(rawPath, "utf8")).resolves.toBeTruthy(); + await expect(readFile(reportPath, "utf8")).resolves.toBeTruthy(); + } expect(calls).toContainEqual([ "api", "--paginate", diff --git a/tests/boundary/filesystem/ciPrHealth.test.ts b/tests/boundary/filesystem/ciPrHealth.test.ts index c9d7a968b4..ca1ace52aa 100644 --- a/tests/boundary/filesystem/ciPrHealth.test.ts +++ b/tests/boundary/filesystem/ciPrHealth.test.ts @@ -11,7 +11,6 @@ import { matchRunToPullRequest, writePrCiHealthReport, } from "../../../scripts/ci/pr-ci-health.mjs"; -import { createGhApiRequest } from "../../../scripts/ci/github-api.mjs"; import type { PrCiPullRequest, PrCiWorkflowRun, @@ -453,23 +452,44 @@ it("public CLI uses only GET and produces a report from fixture GitHub responses it("rejects non-GET methods before launching the GitHub client", async () => { const fixture = await createFixture(); - const request = createGhApiRequest({ - env: { - ...process.env, - PATH: `${fixture.fakeBin}${delimiter}${process.env.PATH ?? ""}`, - CI_GH_CALLS: join(fixture.directory, "should-not-exist"), - }, - }); - await expect( - request("POST", "repos/owner/rea/pulls", { - requests: 0, - startedAt: observedAt, - limitMs: 1_000, - }), - ).rejects.toThrow("read-only GET"); - await expect( - readFile(join(fixture.directory, "should-not-exist")), - ).rejects.toMatchObject({ code: "ENOENT" }); + const marker = join(fixture.directory, "gh-invoked"); + const preload = join(fixture.directory, "watch-gh.cjs"); + await writeFile( + preload, + `const fs = require("node:fs"); +const util = require("node:util"); +const child = require("node:child_process"); +child.execFile = () => { throw new Error("Unexpected callback invocation"); }; +child.execFile[util.promisify.custom] = async () => { + fs.appendFileSync(process.env.CI_GH_CALLS, "called\\n"); + return { stdout: "{}", stderr: "" }; +}; +require("node:module").syncBuiltinESMExports(); +`, + ); + const transport = new URL( + "../../../scripts/ci/github-api.mjs", + import.meta.url, + ).href; + await executeWorkflowFixture( + process.execPath, + [ + "--require", + preload, + "--input-type=module", + "-e", + ` + import assert from "node:assert/strict"; + import { createGhApiRequest } from ${JSON.stringify(transport)}; + const request = createGhApiRequest(); + const budget = { requests: 0, startedAt: Date.now(), limitMs: 10_000 }; + await assert.rejects(request("POST", "repos/owner/rea/pulls", budget), /read-only GET/); + await request("GET", "repos/owner/rea/pulls", budget); + `, + ], + { cwd: fixture.directory, env: { ...process.env, CI_GH_CALLS: marker } }, + ); + expect(await readFile(marker, "utf8")).toBe("called\n"); }); function validPull(overrides: Partial = {}): PrCiPullRequest { diff --git a/tests/boundary/filesystem/ciTestHistory.test.ts b/tests/boundary/filesystem/ciTestHistory.test.ts index 0d2093f7cb..9cf82548cb 100644 --- a/tests/boundary/filesystem/ciTestHistory.test.ts +++ b/tests/boundary/filesystem/ciTestHistory.test.ts @@ -132,23 +132,23 @@ it("selects the latest trusted full run that actually published timing history", expect(output).toBe("run-id=11\nartifact-name=test-timings-full\n"); expect(calls).toContain("name=test-timings-full"); expect(calls).toContain("per_page=100"); + expect(calls).not.toContain("actions/runs/13"); expect(calls.match(/actions\/runs\//gu)).toHaveLength(3); expect(calls).not.toContain("actions/workflows"); }); -it("rejects expired, foreign, old, and pull-request candidates", async () => { +it("rejects expired, old, and pull-request candidates", async () => { const { result, output } = await invoke( [ artifact(15, { expired: true }), - artifact(14, { - workflow_run: { ...artifact(14).workflow_run, head_repository_id: 2 }, - }), artifact(13, { created_at: new Date(now - 31 * 24 * 60 * 60 * 1000).toISOString(), }), artifact(12), ], { + 15: run(15), + 13: run(13), 12: run(12, { event: "pull_request" }), }, { tier: "full" }, @@ -197,22 +197,25 @@ it("bounds run lookups to five artifacts even when inventory has more", async () expect(calls.match(/actions\/runs\//gu)).toHaveLength(5); }); -it("returns empty history without querying for an invalid repository identity", async () => { - const { result, output, calls } = await invoke( - [], - {}, - { - repository: "owner/repo?per_page=100", - repositoryId: "not-an-id", - }, - ); - expect(JSON.parse(result.stdout)).toEqual({ - run_id: "", - artifact_name: "", - }); - expect(output).toBe("run-id=\nartifact-name=\n"); - expect(calls).toBe(""); -}); +it.each([ + { repository: "owner/repo?per_page=100", repositoryId: "1" }, + { repository: "example/rea", repositoryId: "not-an-id" }, +])( + "returns empty history without querying for an invalid repository identity: %j", + async (identity) => { + const { result, output, calls } = await invoke( + [], + {}, + { ...identity, tier: "full" }, + ); + expect(JSON.parse(result.stdout)).toEqual({ + run_id: "", + artifact_name: "", + }); + expect(output).toBe("run-id=\nartifact-name=\n"); + expect(calls).toBe(""); + }, +); it("selects fast history without reading the full-tier artifact", async () => { const { result, calls } = await invoke( diff --git a/tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts b/tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts index 5c418312ec..d26a2e7381 100644 --- a/tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts +++ b/tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts @@ -1,6 +1,7 @@ import { ProviderCleanupError } from "../../../src/domain/providerCleanupError.js"; import { AnalysisCancelledError } from "../../../src/domain/analysisErrorCore.js"; import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; +import type { AnalysisError } from "../../../src/domain/analysisErrorBase.js"; import { projectCliAnalysisResult } from "../../../src/cli/projectCliAnalysisResult.js"; import { parseConfig } from "../../../src/config/parseConfig.js"; import { readFile, symlink, writeFile } from "node:fs/promises"; @@ -43,7 +44,6 @@ import { } from "../../fixtures/binarySession.js"; import { routeProviderIdentity } from "../../../src/application/binary/SessionProviderPolicy.js"; import { BinarySession } from "../../../src/application/binary/BinarySession.js"; -import { isProfilelessNativeDispatchSnapshot } from "../../../src/application/binary/ProfilelessNativeWorkflowSnapshot.js"; import { OwnedStagedOutput } from "../../../src/filesystem/OwnedStagedOutput.js"; import { ProviderAdapterError } from "../../../src/domain/providerAdapterError.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; @@ -1194,6 +1194,18 @@ describe("snapshot replay provider identity", () => { expect(changedProfileResult).not.toEqual(first); expect(changedProfileResult).toMatchObject({ code: "evidence_integrity_mismatch", + details: { + binding_mismatch: { + provider: { + snapshot: { version: "launcher-sha256:fixture" }, + selected: { version: "launcher-sha256:changed" }, + }, + profile_digest: { + snapshot: expect.any(String), + selected: expect.any(String), + }, + }, + }, }); expect(fixture.starts).toEqual(startsAfterFirst); expect(fixture.calls).toEqual(callsAfterFirst); @@ -1221,6 +1233,38 @@ describe("snapshot replay provider identity", () => { expect(changedProviderResult).not.toEqual(first); expect(changedProviderResult).toMatchObject({ code: "evidence_integrity_mismatch", + details: { + binding_mismatch: { + provider: { + snapshot: { id: IDENTITY.id }, + selected: { id: "other-provider" }, + }, + }, + }, + }); + expect(fixture.starts).toEqual(startsAfterFirst); + expect(fixture.calls).toEqual(callsAfterFirst); + + const changedTarget = join(dirname(fixture.path), "changed.hop"); + await writeFile(changedTarget, "different target"); + const changedTargetResult = await runDirectAnalysis( + dependencies, + changedTarget, + "search_strings", + { pattern: "fixture", document: "fixture" }, + { snapshotPath: fixture.snapshotPath }, + ).then(projectCliAnalysisResult); + expect(changedTargetResult).toMatchObject({ + code: "evidence_integrity_mismatch", + details: { + binding_mismatch: { + target: { + snapshot: { sha256: firstSnapshot.value.target.sha256 }, + selected: { sha256: expect.any(String) }, + }, + profile_comparison: "not_compared", + }, + }, }); expect(fixture.starts).toEqual(startsAfterFirst); expect(fixture.calls).toEqual(callsAfterFirst); @@ -1356,6 +1400,42 @@ describe("workflow snapshot profile and cancellation binding", () => { }); }); +const expectReaderBindingMismatch = ( + error: AnalysisError, + selectedVersion: string, + snapshotDigest: string, +): void => { + expect(projectAnalysisError(error).details).toMatchObject({ + binding_mismatch: { + provider: { + snapshot: { version: "apple-metadata-reader-older" }, + selected: { version: selectedVersion }, + }, + profile_digest: { snapshot: snapshotDigest }, + }, + }); +}; + +const expectProfilelessOperationMismatch = ( + session: BinarySession, + snapshot: AnalysisSnapshot, +): void => { + const wrongOperation = structuredClone(snapshot); + wrongOperation.workflow_entries[0]!.operation = "inspect_native_api"; + wrongOperation.workflow_entries[0]!.query_id = analysisQueryId( + wrongOperation.target, + wrongOperation.binding, + "inspect_native_api", + wrongOperation.workflow_entries[0]!.parameters, + ); + const rejected = session.importAnalysisSnapshot(wrongOperation); + expect(rejected.ok).toBe(false); + if (!rejected.ok) + expect(projectAnalysisError(rejected.error).details).toMatchObject({ + binding_mismatch: { workflow_scope: "incompatible" }, + }); +}; + const expectProfilelessWorkflowSnapshotRejections = async ( path: string, provider: AnalysisProvider, @@ -1378,37 +1458,17 @@ const expectProfilelessWorkflowSnapshotRejections = async ( provider: otherReaderProfile.provider, analysis_profile: otherReaderProfile, }; - expect( - isProfilelessNativeDispatchSnapshot( - wrongReader, - opened.value, - APPLE_DISPATCH_METADATA_ANALYSIS_PROFILE, - ), - ).toBe(false); const rejectedReader = session.importAnalysisSnapshot(wrongReader); expect(rejectedReader.ok).toBe(false); - if (!rejectedReader.ok) - expect(rejectedReader.error.message).toContain("profile_mismatch"); + if (!rejectedReader.ok) { + expectReaderBindingMismatch( + rejectedReader.error, + readerIdentity.version, + otherReaderProfile.digest, + ); + } - const wrongOperation = structuredClone(snapshot); - wrongOperation.workflow_entries[0]!.operation = "inspect_native_api"; - wrongOperation.workflow_entries[0]!.query_id = analysisQueryId( - wrongOperation.target, - wrongOperation.binding, - "inspect_native_api", - wrongOperation.workflow_entries[0]!.parameters, - ); - expect( - isProfilelessNativeDispatchSnapshot( - wrongOperation, - opened.value, - APPLE_DISPATCH_METADATA_ANALYSIS_PROFILE, - ), - ).toBe(false); - const rejectedOperation = session.importAnalysisSnapshot(wrongOperation); - expect(rejectedOperation.ok).toBe(false); - if (!rejectedOperation.ok) - expect(rejectedOperation.error.message).toContain("profile_mismatch"); + expectProfilelessOperationMismatch(session, snapshot); const mixedOperation = "inspect_native_api"; const mixedProfile = workflowAnalysisProfile( @@ -1437,16 +1497,11 @@ const expectProfilelessWorkflowSnapshotRejections = async ( ], }; const rejectedMixed = session.importAnalysisSnapshot(mixedSnapshot); - expect( - isProfilelessNativeDispatchSnapshot( - mixedSnapshot as AnalysisSnapshot, - opened.value, - APPLE_DISPATCH_METADATA_ANALYSIS_PROFILE, - ), - ).toBe(false); expect(rejectedMixed.ok).toBe(false); if (!rejectedMixed.ok) - expect(rejectedMixed.error.message).toContain("profile_mismatch"); + expect(projectAnalysisError(rejectedMixed.error).details).toMatchObject({ + binding_mismatch: { workflow_scope: "incompatible" }, + }); const deepProfile = createAnalysisProfile( { id: "ghidra", name: "Ghidra", version: "fixture-version" }, @@ -1506,7 +1561,12 @@ const expectProfilelessWorkflowSnapshotRejections = async ( const rejectedDeepOnly = session.importAnalysisSnapshot(deepSnapshot); expect(rejectedDeepOnly.ok).toBe(false); if (!rejectedDeepOnly.ok) - expect(rejectedDeepOnly.error.message).toContain("profile_mismatch"); + expect(projectAnalysisError(rejectedDeepOnly.error).details).toMatchObject({ + binding_mismatch: { + provider: { snapshot: { id: "ghidra" } }, + profile_digest: { snapshot: deepProfile.digest }, + }, + }); }; it("retains a native dispatch workflow snapshot without a deep provider profile", async () => { diff --git a/tests/boundary/filesystem/ghidraBridgeSources.test.ts b/tests/boundary/filesystem/ghidraBridgeSources.test.ts index dc19ac0411..4ef89f268b 100644 --- a/tests/boundary/filesystem/ghidraBridgeSources.test.ts +++ b/tests/boundary/filesystem/ghidraBridgeSources.test.ts @@ -46,6 +46,7 @@ const installation = async ( javac, [ "#!/bin/sh", + 'printf "invoked\\n" >> "$REA_JAVAC_MARKER"', 'if [ "$1" = "-version" ]; then echo "javac 21.0.12"; exit 0; fi', ...(compilerStatus === 0 ? ["exit 0"] @@ -60,8 +61,10 @@ const installation = async ( await mkdir(scratchParent); return { scratchParent, + javacMarker: join(root, "javac-invocations"), environment: { ...process.env, + REA_JAVAC_MARKER: join(root, "javac-invocations"), GHIDRA_INSTALL_DIR: install, JAVA_HOME: javaHome, TMPDIR: scratchParent, @@ -193,6 +196,9 @@ it.skipIf(process.platform === "win32").each([ ), }); expect(await readdir(fixture.scratchParent)).toEqual([]); + await expect(readFile(fixture.javacMarker)).rejects.toMatchObject({ + code: "ENOENT", + }); }, ); diff --git a/tests/boundary/filesystem/ghidraInstallationProperties.test.ts b/tests/boundary/filesystem/ghidraInstallationProperties.test.ts index 97cf2a61da..5f8b147de0 100644 --- a/tests/boundary/filesystem/ghidraInstallationProperties.test.ts +++ b/tests/boundary/filesystem/ghidraInstallationProperties.test.ts @@ -277,6 +277,7 @@ it.each(["read", "capacity", "completed"] as const)( }, }, }); + expect(failure.cause).toBe(reason); } else if (outcome === "capacity") { expect(projectAnalysisError(failure).code).toBe("resource_constraint"); } else { diff --git a/tests/boundary/filesystem/ghidraLanguageCatalogRecovery.test.ts b/tests/boundary/filesystem/ghidraLanguageCatalogRecovery.test.ts index 632832d42d..31adbada80 100644 --- a/tests/boundary/filesystem/ghidraLanguageCatalogRecovery.test.ts +++ b/tests/boundary/filesystem/ghidraLanguageCatalogRecovery.test.ts @@ -218,6 +218,7 @@ it.each(["released", "still failing"] as const)( cause: reason, cleanupIncomplete: false, }); + expect(cancelled.error.cause).toBe(reason); expect(first.error.cleanupIncomplete).toBe(false); expect(registry.hasPendingOwnedResources()).toBe(false); } else { @@ -252,7 +253,7 @@ it("keeps the caller's reason and the catalog owner when cancellation lands duri cleanupIncomplete: true, cleanupResources: [definition], }); - expect(JSON.stringify(resolved.error.cause)).toContain(reason.caller); + expect(resolved.error.cause).toBe(reason); expect(provider.hasPendingOwnedResources()).toBe(true); fault.deny = false; fault.onClose = undefined; diff --git a/tests/boundary/filesystem/javascript-applications/javascriptPackageRendererProvenance.test.ts b/tests/boundary/filesystem/javascript-applications/javascriptPackageRendererProvenance.test.ts index 07bb01a5e1..d35954b05c 100644 --- a/tests/boundary/filesystem/javascript-applications/javascriptPackageRendererProvenance.test.ts +++ b/tests/boundary/filesystem/javascript-applications/javascriptPackageRendererProvenance.test.ts @@ -39,13 +39,6 @@ it.each(["renderer", "browser", "module"] as const)( }), ); const metadata = graph.nodes.find(({ kind }) => kind === "package"); - expect(metadata?.observations).toContainEqual( - expect.objectContaining({ - properties: expect.objectContaining({ - renderer_declarations: { [field]: "entry.js" }, - }), - }), - ); expect(metadata?.observations[0]?.properties).not.toHaveProperty( "renderer", ); diff --git a/tests/boundary/filesystem/macHopperInstallOwnership.test.ts b/tests/boundary/filesystem/macHopperInstallOwnership.test.ts index 41549a9c83..cbd60110c7 100644 --- a/tests/boundary/filesystem/macHopperInstallOwnership.test.ts +++ b/tests/boundary/filesystem/macHopperInstallOwnership.test.ts @@ -288,7 +288,9 @@ it("reports only retained Mac cleanup roots and clears the live report after exa const temporary = join(home, "owned-dmg"); const stage = join(home, "owned-stage"); await Promise.all([mkdir(temporary), mkdir(stage)]); - const host = systemMacHopperInstallHost(home, { HOME: home }); + const host = systemMacHopperInstallHost(home, { HOME: home }, () => + Promise.resolve("14.0"), + ); let allowTemporary = false; let allowStage = false; let temporaryCalls = 0; diff --git a/tests/boundary/filesystem/prReadiness.test.ts b/tests/boundary/filesystem/prReadiness.test.ts index ead81379de..4d3e7273e2 100644 --- a/tests/boundary/filesystem/prReadiness.test.ts +++ b/tests/boundary/filesystem/prReadiness.test.ts @@ -180,6 +180,24 @@ describe("initial author declaration", () => { ); }); + it.each([ + ["longer backticks", "````markdown", "`````", "success"], + ["longer tildes", "~~~~markdown", "~~~~~", "success"], + ["indented closer with whitespace", "```", " ```` \t", "success"], + ["shorter backticks", "````", "```", "failure"], + ["shorter tildes", "~~~~", "~~~", "failure"], + ["different marker", "```", "~~~", "failure"], + ["mixed markers", "```", "``~", "failure"], + ["four-space indentation", "```", " ```", "failure"], + ["text after the closer", "~~~", "~~~ text", "failure"], + ])( + "keeps declarations hidden until a valid closing fence: %s", + (_name, opening, closing, state) => { + const body = `${opening}\n${readyBody}\n${closing}\n${readyBody}`; + expect(evaluateReadiness(makePull({ body }), template).state).toBe(state); + }, + ); + it("fails closed when the trusted template loses a declaration", () => { expect(() => evaluateReadiness( diff --git a/tests/boundary/filesystem/pythonPyc.test.ts b/tests/boundary/filesystem/pythonPyc.test.ts index 736db15458..25226f9f35 100644 --- a/tests/boundary/filesystem/pythonPyc.test.ts +++ b/tests/boundary/filesystem/pythonPyc.test.ts @@ -48,9 +48,6 @@ describe("Python pyc stable filesystem boundary", () => { const tooSmall = await service.inspect({ path, max_file_bytes: 16 }); if (tooSmall.ok) throw new Error("Expected resource budget failure"); expect(tooSmall.error._tag).toBe("AnalysisResourceConstraintError"); - const tooFewObjects = await service.inspect({ path, max_objects: 1 }); - if (tooFewObjects.ok) throw new Error("Expected object budget failure"); - expect(tooFewObjects.error._tag).toBe("AnalysisResourceConstraintError"); for (const [field, selected] of [ ["max_objects", 1], ["max_depth", 1], @@ -58,6 +55,8 @@ describe("Python pyc stable filesystem boundary", () => { ] as const) { const refused = await service.inspect({ path, [field]: selected }); if (refused.ok) throw new Error(`Expected ${field} refusal`); + if (field === "max_objects") + expect(refused.error._tag).toBe("AnalysisResourceConstraintError"); const projected = projectAnalysisError(refused.error); expect(projected).toMatchObject({ code: "resource_constraint", diff --git a/tests/boundary/filesystem/releaseNotes.test.ts b/tests/boundary/filesystem/releaseNotes.test.ts index 95ea310305..4ec741a90c 100644 --- a/tests/boundary/filesystem/releaseNotes.test.ts +++ b/tests/boundary/filesystem/releaseNotes.test.ts @@ -528,16 +528,7 @@ it("accepts Release Please lockfile refreshes without allowing product changes", join(f.directory, "package-lock.json"), '{"lockfileVersion":3}', ); - await writeFile( - join(f.directory, "package-lock.json"), - '{"lockfileVersion":3}', - ); - await git(f.directory, [ - "add", - "CHANGELOG.md", - "package-lock.json", - "package-lock.json", - ]); + await git(f.directory, ["add", "CHANGELOG.md", "package-lock.json"]); await git(f.directory, ["commit", "-m", "chore: release lockfiles"]); await f.invoke(["check", "--manifest", f.manifest, "--source", "HEAD"]); }); diff --git a/tests/boundary/filesystem/undeliveredSourceClosure.test.ts b/tests/boundary/filesystem/undeliveredSourceClosure.test.ts new file mode 100644 index 0000000000..1ce25fdbf2 --- /dev/null +++ b/tests/boundary/filesystem/undeliveredSourceClosure.test.ts @@ -0,0 +1,627 @@ +import { readdir, readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { expect, it, vi } from "vitest"; + +import type { UndeliveredSourceClosure } from "../../../src/application/AnalysisEvidenceSources.js"; +import { runDirectAnalysis } from "../../../src/application/DirectAnalysis.js"; +import { + deliverCliAnalysisEvidence, + publishCliAnalysisEvidence, +} from "../../../src/cli/analysisEvidenceDelivery.js"; +import { + AnalysisCancelledError, + AnalysisOutputError, +} from "../../../src/domain/analysisErrorCore.js"; +import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; +import { createArtifactInspection } from "../../../src/domain/artifactInspection.js"; +import { + createEvidence, + parseEvidence, + type Evidence, +} from "../../../src/domain/evidence.js"; +import { jsonByteLength } from "../../../src/domain/jsonSerialization.js"; +import type { JsonValue } from "../../../src/domain/jsonValue.js"; +import { ProviderCleanupError } from "../../../src/domain/providerCleanupError.js"; +import { err, ok } from "../../../src/domain/result.js"; +import { OwnedStagedOutput } from "../../../src/filesystem/OwnedStagedOutput.js"; +import { inventoryArtifact } from "../../fixtures/artifactInventory.js"; +import { composedWorkflowFixture } from "../../fixtures/composedWorkflowEvidence.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const provider = { id: "fixture", name: "Fixture", version: "1" }; + +const source = (name: string, links: readonly string[] = []) => + createEvidence(undefined, provider, { + operation: "fixture_source", + parameters: { name }, + result: { name }, + evidenceLinks: [...links], + }); + +const observed = (links: readonly string[]) => + createEvidence(undefined, provider, { + operation: "fixture_observation", + parameters: {}, + result: { completed: true }, + evidenceLinks: [...links], + }); + +const cancelledWith = (observation: unknown): JsonValue => + projectAnalysisError( + new AnalysisCancelledError("fixture_observation", { + partialObservation: observation as Evidence, + }), + "invocation", + ); + +const delivery = (root: string, sources?: readonly Evidence[]) => ({ + providerId: "fixture", + operation: "fixture_observation", + signal: new AbortController().signal, + detail: "complete" as const, + evidenceOutput: join(root, "evidence.json"), + ...(sources === undefined ? {} : { sourceEvidence: sources }), +}); + +const closureOf = (output: unknown) => + (output as { details: { source_closure: UndeliveredSourceClosure } }).details + .source_closure; + +it("reports never-captured links as explicit omissions without fabricating records", async () => { + const root = await createTestTempDirectory("rea-closure-missing-"); + const held = source("held", ["ev_".padEnd(67, "a")]); + const neverCaptured = "ev_".padEnd(67, "b"); + const observation = observed([held.evidence_id, neverCaptured]); + const closure = closureOf( + await publishCliAnalysisEvidence( + err(cancelledWith(observation)), + delivery(root, [held]), + ), + ); + expect(closure.records.map((record) => record.evidence_id)).toEqual([ + held.evidence_id, + ]); + expect(parseEvidence(closure.records[0])).toEqual(held); + // The ancestor of a held record is unresolved here, not absent. + expect(closure.omitted).toEqual([ + { evidence_id: neverCaptured, reason: "not_captured" }, + { evidence_id: "ev_".padEnd(67, "a"), reason: "not_captured" }, + ]); + expect(closure).toMatchObject({ + publication: "not_published", + complete: false, + omitted_count: 2, + }); + expect(await readdir(root)).toEqual([]); +}); + +it("reports an unlinked captured source and stops at nothing linked or captured", async () => { + const root = await createTestTempDirectory("rea-closure-unlinked-"); + const selected = source("selected"); + const observation = observed([]); + expect( + closureOf( + await publishCliAnalysisEvidence( + err(cancelledWith(observation)), + delivery(root, [selected]), + ), + ).records, + ).toEqual([selected]); + + const failure = cancelledWith(observation); + expect( + await publishCliAnalysisEvidence(err(failure), delivery(root, [])), + ).toBe(failure); + // A self-reference is the observation, not one of its sources. + expect( + await publishCliAnalysisEvidence( + err(failure), + delivery(root, [observation]), + ), + ).toBe(failure); +}); + +it("leaves failures untouched unless a completed observation requested delivery", async () => { + const root = await createTestTempDirectory("rea-closure-untouched-"); + const held = source("held"); + const observation = observed([held.evidence_id]); + const requested = delivery(root, [held]); + + // Without a selected output no source closure was requested on success either. + const noOutput = cancelledWith(observation); + expect( + await publishCliAnalysisEvidence(err(noOutput), { + ...requested, + evidenceOutput: undefined, + }), + ).toBe(noOutput); + + const noObservation = projectAnalysisError( + new AnalysisCancelledError("fixture_observation"), + "invocation", + ); + expect(await publishCliAnalysisEvidence(err(noObservation), requested)).toBe( + noObservation, + ); + + const notEvidence = cancelledWith({ kind: "file-snapshot-read" }); + expect(await publishCliAnalysisEvidence(err(notEvidence), requested)).toBe( + notEvidence, + ); + + // A failed authentication cannot select which sources an observation cites. + const altered = cancelledWith({ + ...observation, + evidence_links: [], + normalized_result: { completed: false }, + }); + expect(await publishCliAnalysisEvidence(err(altered), requested)).toBe( + altered, + ); +}); + +it("bounds records and listed omissions while counting everything", async () => { + const root = await createTestTempDirectory("rea-closure-bounds-"); + const held = Array.from({ length: 200 }, (_, index) => + createEvidence(undefined, provider, { + operation: "fixture_source", + parameters: { index }, + result: { text: "x".repeat(4 * 1024) }, + }), + ); + const missing = Array.from({ length: 100 }, (_, index) => + `ev_${index.toString(16).padStart(4, "0")}`.padEnd(67, "c"), + ); + const observation = observed([ + ...held.map((record) => record.evidence_id), + ...missing, + ]); + const closure = closureOf( + await publishCliAnalysisEvidence( + err(cancelledWith(observation)), + delivery(root, held), + ), + ); + expect(closure.complete).toBe(false); + expect(closure.records.length).toBeGreaterThan(0); + expect(closure.records.length).toBeLessThan(held.length); + const expectedOmissions = held.length + missing.length; + expect(closure.omitted_count).toBe( + expectedOmissions - closure.records.length, + ); + expect(closure.omitted).toHaveLength(64); + expect(jsonByteLength(closure)).toBeLessThan(80 * 1024); +}); + +/** Above V8's default call-argument limit of about 125,000 on a forked main thread. */ +const DATA_SIZED_LINKS = 200_000; + +const digestLink = (index: number) => + `ev_${index.toString(16).padStart(64, "0")}`; + +const wideSource = (links: readonly string[]) => + createEvidence(undefined, provider, { + operation: "fixture_source", + parameters: {}, + result: { wide: true }, + evidenceLinks: links, + }); + +it.each(["final-close cancellation", "refused publication"] as const)( + "keeps the original failure when a captured source has a data-sized link set: %s", + async (failure) => { + const root = await createTestTempDirectory("rea-closure-wide-"); + const links = Array.from({ length: DATA_SIZED_LINKS }, (_, index) => + digestLink(index), + ); + const wide = wideSource(links); + const observation = observed([wide.evidence_id]); + const requested = delivery(root, [wide]); + if (failure === "refused publication") + await writeFile(requested.evidenceOutput, "original"); + const output = await publishCliAnalysisEvidence( + failure === "refused publication" + ? ok(observation) + : err(cancelledWith(observation)), + requested, + ); + expect(output).toMatchObject( + failure === "refused publication" + ? { details: { reason: "exists" } } + : { code: "cancelled" }, + ); + const closure = closureOf(output); + expect(closure.records).toEqual([]); + expect(closure.omitted_count).toBe(DATA_SIZED_LINKS + 1); + expect(closure.omitted).toHaveLength(64); + expect(closure.omitted[0]).toEqual({ + evidence_id: wide.evidence_id, + reason: "not_inlined", + }); + expect(jsonByteLength(closure)).toBeLessThan(16 * 1024); + }, + 60_000, +); + +it("reports a data-sized link set cited by the observation itself", async () => { + const root = await createTestTempDirectory("rea-closure-wide-observation-"); + const closure = closureOf( + await publishCliAnalysisEvidence( + err( + cancelledWith( + observed( + Array.from({ length: DATA_SIZED_LINKS }, (_, index) => + digestLink(index), + ), + ), + ), + ), + delivery(root), + ), + ); + expect(closure.omitted_count).toBe(DATA_SIZED_LINKS); + expect(closure.omitted).toHaveLength(64); +}, 60_000); + +it("examines repeated links once across every captured source", async () => { + const root = await createTestTempDirectory("rea-closure-dedupe-"); + const shared = source("shared"); + const repeated = Array.from( + { length: DATA_SIZED_LINKS }, + () => shared.evidence_id, + ); + const first = wideSource(repeated); + const second = createEvidence(undefined, provider, { + operation: "fixture_source", + parameters: { second: true }, + result: { wide: true }, + evidenceLinks: repeated, + }); + const closure = closureOf( + await publishCliAnalysisEvidence( + err(cancelledWith(observed([first.evidence_id, second.evidence_id]))), + delivery(root, [first, second, shared]), + ), + ); + expect(closure.records).toEqual([shared]); + expect(closure.omitted_count).toBe(2); +}, 60_000); + +it.each([ + ["a captured source", "captured"], + ["the completed observation", "observation"], +] as const)( + "keeps the original failure when examining %s throws", + async (_, thrower) => { + const root = await createTestTempDirectory("rea-closure-throws-"); + const held = source("held"); + const failing = (record: Evidence) => + Object.defineProperty({ ...record }, "evidence_links", { + enumerable: true, + get: () => { + throw new RangeError("quoted analyzed-value text"); + }, + }) as Evidence; + const observation = observed([held.evidence_id]); + const projected = cancelledWith(observation) as { + details: Record; + }; + const partial = + thrower === "observation" + ? (failing(observation) as unknown as JsonValue) + : projected.details.partial_observation!; + const failure = { + ...projected, + details: { ...projected.details, partial_observation: partial }, + } as JsonValue; + const output = (await publishCliAnalysisEvidence( + err(failure), + delivery(root, thrower === "captured" ? [failing(held)] : [held]), + )) as { + code: string; + message: string; + details: Record; + }; + expect(output.code).toBe("cancelled"); + expect(output.message).toBe((failure as { message: string }).message); + expect(output.details.partial_observation).toBe(partial); + expect(output.details.operation).toBe("fixture_observation"); + expect(output.details.source_closure).toEqual({ + publication: "not_published", + complete: false, + collection_failure: "RangeError", + recovery: expect.any(String), + }); + expect(JSON.stringify(output.details.source_closure)).not.toContain( + "quoted analyzed-value text", + ); + expect(await readdir(root)).toEqual([]); + }, +); + +it("spends the allowance on smaller later records after a record does not fit", async () => { + const root = await createTestTempDirectory("rea-closure-greedy-"); + const large = createEvidence(undefined, provider, { + operation: "fixture_source", + parameters: {}, + result: { text: "x".repeat(128 * 1024) }, + }); + const small = source("small"); + const closure = closureOf( + await publishCliAnalysisEvidence( + err( + cancelledWith(observed([large.evidence_id, small.evidence_id].sort())), + ), + delivery(root, [large, small]), + ), + ); + expect(closure.records.map((record) => record.evidence_id)).toEqual([ + small.evidence_id, + ]); + expect(closure.omitted).toEqual([ + { evidence_id: large.evidence_id, reason: "not_inlined" }, + ]); +}); + +it("keeps captured records out of error output under explicit exclusions", async () => { + const root = await createTestTempDirectory("rea-closure-sensitive-"); + const held = source("literal-secret-value"); + const output = await publishCliAnalysisEvidence( + err(cancelledWith(observed([held.evidence_id]))), + { ...delivery(root, [held]), sensitiveValues: ["literal-secret-value"] }, + ); + const closure = closureOf(output); + expect(closure.records).toEqual([]); + expect(closure.omitted).toEqual([ + { evidence_id: held.evidence_id, reason: "not_inlined" }, + ]); + expect(JSON.stringify(closure)).not.toContain("literal-secret-value"); +}); + +it("does not duplicate sources an artifact inspection already embeds", async () => { + const root = await createTestTempDirectory("rea-closure-embedded-"); + const input = join(root, "input.txt"); + await writeFile(input, "artifact contents"); + const embedded = createEvidence(undefined, provider, { + operation: "inventory_artifact", + parameters: {}, + result: await inventoryArtifact(input), + }); + const observation = createEvidence(undefined, provider, { + operation: "inspect_artifact", + parameters: {}, + result: createArtifactInspection(embedded), + evidenceLinks: [embedded.evidence_id], + }); + const output = await publishCliAnalysisEvidence( + err(cancelledWith(observation)), + { ...delivery(root), evidenceOutput: join(root, "bundle.json") }, + ); + expect(closureOf(output)).toMatchObject({ + publication: "not_published", + complete: true, + records: [], + omitted_count: 0, + }); +}); + +it("reports the closure when cancellation precedes bundle commit", async () => { + const root = await createTestTempDirectory("rea-closure-precommit-"); + const held = source("held"); + const observation = observed([held.evidence_id]); + const controller = new AbortController(); + controller.abort(new Error("cancelled before publication")); + const output = await publishCliAnalysisEvidence(ok(observation), { + ...delivery(root, [held]), + signal: controller.signal, + }); + expect(output).toMatchObject({ code: "cancelled" }); + expect(closureOf(output).records).toEqual([held]); + expect(await readdir(root)).toEqual([]); +}); + +it("reports the closure when an existing output refuses publication", async () => { + const root = await createTestTempDirectory("rea-closure-exists-"); + const held = source("held"); + const requested = delivery(root, [held]); + await writeFile(requested.evidenceOutput, "original"); + const output = await publishCliAnalysisEvidence( + ok(observed([held.evidence_id])), + requested, + ); + expect(output).toMatchObject({ details: { reason: "exists" } }); + expect(closureOf(output).records).toEqual([held]); + expect(await readFile(requested.evidenceOutput, "utf8")).toBe("original"); +}); + +it("defers to the committed bundle receipt after cancellation wins late", async () => { + const root = await createTestTempDirectory("rea-closure-committed-"); + const held = source("held"); + const controller = new AbortController(); + const close = OwnedStagedOutput.prototype.close; + const seam = vi + .spyOn(OwnedStagedOutput.prototype, "close") + .mockImplementationOnce(async function (this: OwnedStagedOutput) { + await close.call(this); + controller.abort(new Error("cancelled after commit")); + }); + const requested = { + ...delivery(root, [held]), + signal: controller.signal, + }; + let output: JsonValue; + try { + output = await publishCliAnalysisEvidence( + ok(observed([held.evidence_id])), + requested, + ); + } finally { + seam.mockRestore(); + } + expect(output).toMatchObject({ + code: "cancelled", + details: { + diagnostics: { evidence_bundle: { path: requested.evidenceOutput } }, + }, + }); + expect(output).not.toHaveProperty("details.source_closure"); +}); + +it("defers to the published file when staging cleanup stays incomplete after commit", async () => { + const root = await createTestTempDirectory("rea-closure-published-"); + const held = source("held"); + const requested = delivery(root, [held]); + const close = OwnedStagedOutput.prototype.close; + const owners = new Set(); + const seam = vi + .spyOn(OwnedStagedOutput.prototype, "close") + .mockImplementation(async function (this: OwnedStagedOutput) { + owners.add(this); + throw new Error("controlled staging cleanup failure"); + }); + let output: JsonValue; + try { + output = await publishCliAnalysisEvidence( + ok(observed([held.evidence_id])), + requested, + ); + } finally { + seam.mockRestore(); + for (const owner of owners) await close.call(owner); + } + expect(output).toMatchObject({ + code: "cleanup_incomplete", + details: { published_path: requested.evidenceOutput }, + }); + expect(output).not.toHaveProperty("details.source_closure"); +}); + +it("does not change ordinary successful delivery", async () => { + const root = await createTestTempDirectory("rea-closure-success-"); + const held = source("held"); + const observation = observed([held.evidence_id]); + const requested = delivery(root, [held]); + const delivered = await publishCliAnalysisEvidence( + ok(observation), + requested, + ); + expect(JSON.stringify(delivered)).toBe(JSON.stringify(observation)); + const withoutOutput = await publishCliAnalysisEvidence(ok(observation), { + ...requested, + evidenceOutput: undefined, + }); + expect(JSON.stringify(withoutOutput)).toBe(JSON.stringify(observation)); +}); + +it.each(["interrupt", "cleanup-failure", "interrupt-cleanup-failure"] as const)( + "states the sources a production direct analysis never captured: %s", + async (scenario) => { + const workflow = await composedWorkflowFixture(); + const controller = new AbortController(); + const close = workflow.session.close.bind(workflow.session); + workflow.session.close = async () => { + // The fixture's own final close retries this owner after the command. + workflow.session.close = close; + if (scenario !== "cleanup-failure") controller.abort(); + const closed = await close(); + if (scenario === "interrupt" || !closed.ok) return closed; + return err( + new ProviderCleanupError( + "fixture", + ["owned-root"], + { reason: "controlled session close failure" }, + { operation: "close_binary" }, + ), + ); + }; + const root = await createTestTempDirectory("rea-closure-direct-"); + const complete = await runDirectAnalysis( + workflow.dependencies, + workflow.path, + "binary_overview", + {}, + { signal: controller.signal }, + ); + const output = await publishCliAnalysisEvidence(complete, { + ...delivery(root), + signal: controller.signal, + }); + const failure = output as { + code: string; + details: { partial_observation: unknown }; + }; + expect(failure.code).toBe( + scenario === "interrupt" ? "cancelled" : "cleanup_incomplete", + ); + const links = parseEvidence( + failure.details.partial_observation, + ).evidence_links; + expect(links.length).toBeGreaterThan(0); + expect(closureOf(output)).toMatchObject({ + publication: "not_published", + complete: false, + records: [], + omitted_count: links.length, + omitted: links.map((evidence_id) => ({ + evidence_id, + reason: "not_captured", + })), + }); + expect(await readdir(root)).toEqual([]); + }, +); + +it("reports through the shared one-shot delivery when cancellation arrives during close", async () => { + const root = await createTestTempDirectory("rea-closure-deliver-"); + const held = source("held"); + const uncaptured = "ev_".padEnd(67, "d"); + const observation = observed([held.evidence_id, uncaptured]); + const controller = new AbortController(); + const output = await deliverCliAnalysisEvidence({ + ...delivery(root, [held]), + signal: controller.signal, + run: async () => ok(observation), + service: { + close: async () => { + controller.abort(new Error("cancelled during final close")); + }, + }, + }); + expect(output).toMatchObject({ + code: "cancelled", + details: { partial_observation: { evidence_id: observation.evidence_id } }, + }); + expect(closureOf(output)).toMatchObject({ + complete: false, + records: [held], + omitted: [{ evidence_id: uncaptured, reason: "not_captured" }], + }); + expect(await readdir(root)).toEqual([]); +}); + +it("reports the closure when delivered view projection refuses a completed observation", async () => { + const root = await createTestTempDirectory("rea-closure-view-"); + const held = source("held"); + const observation = observed([held.evidence_id]); + const output = await publishCliAnalysisEvidence(ok(observation), { + ...delivery(root, [held]), + detail: "summary", + // A view applying to another provider's facts cannot be selected here. + view: { kind: "native", facet: "pseudocode", offset: 0, limit: 1 }, + }); + expect(closureOf(output).records).toEqual([held]); + expect(await readdir(root)).toEqual([]); +}); + +it("leaves a primary failure without a completed observation untouched", async () => { + const root = await createTestTempDirectory("rea-closure-primary-"); + const held = source("held"); + const primary = projectAnalysisError( + new AnalysisOutputError("fixture_observation", "primary reply failure"), + "invocation", + ); + expect( + await publishCliAnalysisEvidence(err(primary), delivery(root, [held])), + ).toBe(primary); +}); diff --git a/tests/boundary/filesystem/webArtifactReadOwnership.test.ts b/tests/boundary/filesystem/webArtifactReadOwnership.test.ts index b7cdbd20a4..eeb2a2a945 100644 --- a/tests/boundary/filesystem/webArtifactReadOwnership.test.ts +++ b/tests/boundary/filesystem/webArtifactReadOwnership.test.ts @@ -356,7 +356,9 @@ it("preserves both web cleanup owners' resources, diagnostics and recovery actio }, }, ); - const projected = [artifacts, resolver].map(projectAnalysisError); + const projected = [artifacts, resolver].map((failure) => + projectAnalysisError(failure), + ); await expect(service.close()).rejects.toMatchObject({ cleanupResources: ["artifact-root", "shared-resource", "resolver-root"], diagnostics: { failures: projected }, diff --git a/tests/boundary/go/goBinaryFinalization.test.ts b/tests/boundary/go/goBinaryFinalization.test.ts index 7522082264..8db9563eba 100644 --- a/tests/boundary/go/goBinaryFinalization.test.ts +++ b/tests/boundary/go/goBinaryFinalization.test.ts @@ -65,6 +65,7 @@ it.each(["control", "close-failure", "close-cancel", "close-both"] as const)( _tag: "AnalysisCancelledError", cause: reason, }); + expect(primary.cause).toBe(reason); } close.mockRestore(); await service.close(); diff --git a/tests/boundary/go/goBinaryImage.test.ts b/tests/boundary/go/goBinaryImage.test.ts index 362abad5a0..07805f1c4a 100644 --- a/tests/boundary/go/goBinaryImage.test.ts +++ b/tests/boundary/go/goBinaryImage.test.ts @@ -119,6 +119,7 @@ it.each([ it("continues past an empty file-backed ELF data load", () => { const fixture = createGoSplitElfFixture({ sectionless: true }); const expected = readGoBinaryImage(fixture.bytes).build_info; + expect(expected).not.toBeNull(); fixture.bytes.writeBigUInt64LE(0n, 128 + 32); expect(readGoBinaryImage(fixture.bytes).build_info).toEqual(expected); }); @@ -135,6 +136,7 @@ it.each([4, 5, 7])( it("prefers the named ELF section to an earlier writable-load marker", () => { const fixture = createGoSplitElfFixture(); const expected = readGoBinaryImage(fixture.bytes).build_info; + expect(expected).not.toBeNull(); fixture.bytes.copy( fixture.bytes, 512, diff --git a/tests/boundary/go/goBinaryP32PointerWidth.test.ts b/tests/boundary/go/goBinaryP32PointerWidth.test.ts deleted file mode 100644 index 0602d8549c..0000000000 --- a/tests/boundary/go/goBinaryP32PointerWidth.test.ts +++ /dev/null @@ -1,32 +0,0 @@ -import { expect, it } from "vitest"; -import { readGoBinaryImage } from "../../../src/go/GoBinaryImage.js"; -import { - createGoBinaryFixture, - GO_MODULE_TEXT, -} from "../../fixtures/go/image.js"; - -it("decodes legacy ELF64 Go strings at the record's four-byte pointer width", () => { - const fixture = createGoBinaryFixture({ - format: "elf", - bits: 64, - encoding: "pointer", - pointerSize: 4, - goVersion: "go1.13.15", - }); - - expect(readGoBinaryImage(fixture.bytes)).toMatchObject({ - format: "elf", - architecture: "amd64", - bits: 64, - build_info: { - encoding: "pointer", - go_version: "go1.13.15", - module_text: GO_MODULE_TEXT, - version_location: { offset: fixture.versionOffset, bytes: 9 }, - module_location: { - offset: fixture.moduleOffset, - bytes: fixture.moduleBytes.length, - }, - }, - }); -}); diff --git a/tests/boundary/gpu/codeObjects.test.ts b/tests/boundary/gpu/codeObjects.test.ts index a508d009ff..73e693d59a 100644 --- a/tests/boundary/gpu/codeObjects.test.ts +++ b/tests/boundary/gpu/codeObjects.test.ts @@ -499,14 +499,17 @@ it.each([ const inventory = await inspect(bytes, { analysis: "inventory" }); if (!inventory.ok) throw inventory.error; const row = inventory.value.objects.find((value) => value.kind === "amdgpu")!; - expect(row.target).not.toBeNull(); - const selected = await inspect(bytes, { targets: [row.target!] }); + const expectedTarget = targetId.endsWith("opaque") + ? "amdgcn-amd-amdhsa--gfx900" + : "gfx900"; + expect(row.target).toBe(expectedTarget); + const selected = await inspect(bytes, { targets: [expectedTarget] }); if (!selected.ok) throw selected.error; expect(selected.value.kernels).toHaveLength(1); expect( selected.value.objects.find((value) => value.kind === "amdgpu"), ).toMatchObject({ - target: row.target, + target: expectedTarget, payload_target: "amdgcn-amd-amdhsa--gfx900", target_consistency: targetId.endsWith("opaque") ? "single-source" diff --git a/tests/boundary/il2cpp/il2cppBinaryFile.test.ts b/tests/boundary/il2cpp/il2cppBinaryFile.test.ts index 81c9e97bde..2f6010e787 100644 --- a/tests/boundary/il2cpp/il2cppBinaryFile.test.ts +++ b/tests/boundary/il2cpp/il2cppBinaryFile.test.ts @@ -1173,7 +1173,9 @@ it.each([1, 0xff])( (padding) => { const bytes = createIl2cppBinaryFixture(); bytes.fill(padding, 0x604, 0x608); - expect(link(bytes)).toEqual(link(createIl2cppBinaryFixture())); + const linked = link(bytes); + expect(linked.method_addresses).toEqual(expectedAddresses); + expect(linked).toEqual(link(createIl2cppBinaryFixture())); bytes.writeUInt32LE(3, 0x600); expect(failure(bytes).reason).toBe("code-registration-not-found"); }, @@ -1834,12 +1836,6 @@ it("keeps a capacity refusal while naming a generic row a refusal", () => { expect((failure as Il2cppBinaryFormatFailure).kind).toBe("capacity"); }); -it("ignores CodeRegistration count alignment padding", () => { - const bytes = createIl2cppBinaryFixture(); - bytes.fill(0xff, 0x604, 0x608); - expect(link(bytes).method_addresses).toEqual(expectedAddresses); -}); - it.each([0, 8, 16])( "finds registration with %s file-backed trailing zero bytes", (tail) => { diff --git a/tests/boundary/mcp/addressContextBookmarks.test.ts b/tests/boundary/mcp/addressContextBookmarks.test.ts index ed97c4a055..dd668c2b17 100644 --- a/tests/boundary/mcp/addressContextBookmarks.test.ts +++ b/tests/boundary/mcp/addressContextBookmarks.test.ts @@ -5,7 +5,12 @@ import { expect, it } from "vitest"; import { createTestBinarySession } from "../../fixtures/binarySession.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js"; -import { AnalysisProtocolError } from "../../../src/domain/analysisErrorCore.js"; +import { + AnalysisCapabilityUnavailableError, + AnalysisCancelledError, + AnalysisProtocolError, +} from "../../../src/domain/analysisErrorCore.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; import { err, ok } from "../../../src/domain/result.js"; import { createServer } from "../../../src/server/createServer.js"; @@ -85,3 +90,68 @@ it.each([ } }, ); + +it.each([false, true])( + "reports each unresolved facet cleanup over MCP (cancellation=%s)", + async (cancelled) => { + const identity = { id: "fixture", name: "Fixture", version: "1" }; + const directory = await createTestTempDirectory("rea-context-cleanup-"); + const target = join(directory, "fixture.com"); + await writeFile(target, Buffer.from([0xb8, 0x00, 0x4c, 0xcd, 0x21])); + const session = createTestBinarySession(() => ({ + execute: async (operation) => { + if (cancelled && operation === "resolve_containing_procedure") + return err(new AnalysisCancelledError(operation)); + if (operation === "comment" || operation === "inline_comment") + return err( + new AnalysisCapabilityUnavailableError( + identity.id, + operation, + "controlled failure", + { + cleanup: { + reason: "controlled cleanup failure", + resources: [operation], + }, + }, + ), + ); + return ok(createAnalysisExecution(null, identity)); + }, + close: async () => ok(null), + })); + const server = createServer({ kind: "session", session }); + const client = new Client({ name: "context-cleanup", version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + try { + await server.connect(serverTransport); + await client.connect(clientTransport); + expect( + ( + await client.callTool({ + name: "open_binary", + arguments: { path: target, format: "dos-com" }, + }) + ).isError, + ).not.toBe(true); + const response = await client.callTool({ + name: "inspect_address_context", + arguments: { address: "0x401001", document: "fixture" }, + }); + expect(response.isError).toBe(true); + expect(parseMcpToolError(response)).toMatchObject({ + error: { + code: "cleanup_incomplete", + details: { + cleanup: "incomplete", + resources: expect.arrayContaining(["comment", "inline_comment"]), + }, + }, + }); + } finally { + await client.close(); + await server.close(); + } + }, +); diff --git a/tests/boundary/mcp/addressContextInputErrors.test.ts b/tests/boundary/mcp/addressContextInputErrors.test.ts new file mode 100644 index 0000000000..b908f51872 --- /dev/null +++ b/tests/boundary/mcp/addressContextInputErrors.test.ts @@ -0,0 +1,183 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { expect, it } from "vitest"; +import { createTestBinarySession } from "../../fixtures/binarySession.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js"; +import { inspectAddressContext } from "../../../src/application/AnalysisContextQueries.js"; +import { + AnalysisInputError, + AnalysisProtocolError, +} from "../../../src/domain/analysisErrorCore.js"; +import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; +import { err, ok } from "../../../src/domain/result.js"; +import { SESSION_TOOL_CONTRACTS } from "../../../src/contracts/sessionToolContracts.js"; +import { createServer } from "../../../src/server/createServer.js"; + +it.each([ + ["compact", false, "comment"], + ["compact", true, "comment"], + ["full", false, "comment"], + ["full", true, "comment"], + ["full", false, "list_bookmarks"], +] as const)( + "preserves a local input refusal alongside successful facets (%s, conforms=%s, operation=%s)", + async (profile, conforms, rejectedOperation) => { + const identity = { id: "fixture", name: "Fixture", version: "1" }; + const directory = await createTestTempDirectory("rea-context-input-"); + const target = join(directory, "fixture.com"); + await writeFile(target, Buffer.from([0xb8, 0x00, 0x4c, 0xcd, 0x21])); + const inputFailure = new AnalysisInputError( + rejectedOperation, + { + userMessage: "Controlled facet refusal", + conformsToInputContract: conforms, + diagnostics: { producer_fact: "retained" }, + }, + [ + { + path: [rejectedOperation === "comment" ? "address" : "document"], + reason: "invalid_value", + expected: "controlled-value", + }, + ], + ); + const session = createTestBinarySession(() => ({ + execute: async (operation) => { + if (operation === rejectedOperation) return err(inputFailure); + if (operation === "inline_comment") + return err(new AnalysisProtocolError("controlled provider failure")); + const result = + operation === "address_name" + ? "entry" + : operation === "resolve_containing_procedure" + ? { + query_address: "0x401001", + found: true, + procedure: { + address: "0x401000", + name: "entry", + classification: null, + body: { available: false, reason: "not recorded" }, + }, + } + : operation === "list_bookmarks" + ? [] + : null; + return ok(createAnalysisExecution(result, identity)); + }, + close: async () => ok(null), + })); + const server = createServer( + { kind: "session", session }, + { + environment: { + ...process.env, + REA_MCP_OUTPUT_SCHEMA_PROFILE: profile, + REA_MCP_TOOL_FAMILIES: "session", + }, + }, + ); + const client = new Client({ name: "context-input-error", version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + try { + await server.connect(serverTransport); + await client.connect(clientTransport); + const definitions = new Map( + (await client.listTools()).tools.map((tool) => [tool.name, tool]), + ); + const contextDefinition = definitions.get("inspect_address_context"); + if (contextDefinition === undefined) + throw new Error("inspect_address_context was not advertised"); + expect( + ( + await client.callTool({ + name: "open_binary", + arguments: { path: target, format: "dos-com" }, + }) + ).isError, + ).not.toBe(true); + const response = await client.callTool( + { + name: "inspect_address_context", + arguments: { address: "0x401001", document: "fixture" }, + }, + { toolDefinition: contextDefinition }, + ); + expect(response.isError).not.toBe(true); + const contract = SESSION_TOOL_CONTRACTS.find( + ( + candidate, + ): candidate is Extract< + (typeof SESSION_TOOL_CONTRACTS)[number], + { readonly name: "inspect_address_context" } + > => candidate.name === "inspect_address_context", + ); + if (contract === undefined) + throw new Error("inspect_address_context contract is missing"); + const parsed = contract.outputSchema.parse( + response.structuredContent, + ).result; + const expected = projectAnalysisError(inputFailure); + expect(parsed.name).toEqual({ state: "available", value: "entry" }); + expect(parsed.procedure).toMatchObject({ + state: "available", + value: { found: true, query_address: "0x401001" }, + }); + const rejected = + rejectedOperation === "comment" ? parsed.comment : parsed.bookmarks; + expect(rejected).toEqual({ + state: "unavailable", + reason: expected.message, + remediation: expected.remediation.action, + error: expected, + }); + if (rejectedOperation === "comment") + expect(parsed.bookmarks).toEqual({ state: "available", value: [] }); + else expect(parsed.comment).toEqual({ state: "available", value: null }); + expect(parsed.inline_comment).toMatchObject({ state: "unavailable" }); + expect(parsed.inline_comment).not.toHaveProperty("error"); + expect(expected.details).toMatchObject({ + issues: [ + { + path: [rejectedOperation === "comment" ? "address" : "document"], + expected: "controlled-value", + }, + ], + diagnostics: { producer_fact: "retained" }, + }); + expect(Object.hasOwn(expected, "recovery")).toBe(!conforms); + } finally { + await client.close(); + await server.close(); + } + }, +); + +it("does not demote an input refusal with incomplete cleanup into a local facet", async () => { + const identity = { id: "fixture", name: "Fixture", version: "1" }; + const failure = new AnalysisInputError( + "comment", + { + cleanup: { reason: "controlled failure", resources: ["owned-resource"] }, + }, + [{ path: ["address"], reason: "invalid_value" }], + ); + const result = await inspectAddressContext( + { + execute: async (operation) => + operation === "comment" + ? err(failure) + : ok(createAnalysisExecution(null, identity)), + }, + { address: "0x401001", document: "fixture" }, + ); + if (result.ok) throw new Error("expected incomplete cleanup"); + expect(result.error).toBe(failure); + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "cleanup_incomplete", + details: { resources: ["owned-resource"] }, + }); +}); diff --git a/tests/boundary/mcp/analyzeFunctionDetail.test.ts b/tests/boundary/mcp/analyzeFunctionDetail.test.ts index 7a88e84892..89b8757670 100644 --- a/tests/boundary/mcp/analyzeFunctionDetail.test.ts +++ b/tests/boundary/mcp/analyzeFunctionDetail.test.ts @@ -309,3 +309,32 @@ it("rejects a summary request before analysis when the server retains no Evidenc expect(reply.isError).toBe(true); expect(execute).not.toHaveBeenCalled(); }); + +it.each([true, false])( + "delivers an unreadable dossier's partial Evidence through the session writer (retained=%s)", + async (retain) => { + const { client, ledger } = await connect(retain, { + not: "a dossier", + } as never); + + const reply = await client.callTool({ + name: "analyze_function", + arguments: { procedure: "main", detail: "complete" }, + }); + + expect(reply.isError).toBe(true); + const partial = JSON.parse( + reply.content.find((item) => item.type === "text")?.text ?? "{}", + ).error.details.partial_observation; + if (!retain) { + expect(parseEvidence(partial).operation).toBe("analyze_function"); + return; + } + expect(partial).toMatchObject({ kind: "retained-evidence" }); + expect(ledger.forAnalysis(partial.evidence_id)).toMatchObject({ + evidence_id: partial.evidence_id, + operation: "analyze_function", + normalized_result: { not: "a dossier" }, + }); + }, +); diff --git a/tests/boundary/mcp/batchDecompileViewsMcp.test.ts b/tests/boundary/mcp/batchDecompileViewsMcp.test.ts new file mode 100644 index 0000000000..ecbe90b383 --- /dev/null +++ b/tests/boundary/mcp/batchDecompileViewsMcp.test.ts @@ -0,0 +1,870 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import type { CallToolResult } from "@modelcontextprotocol/server"; +import { afterAll, expect, it, onTestFinished, vi } from "vitest"; +import { z } from "zod"; +import { ANALYSIS_VIEW_TOOL_CONTRACTS } from "../../../src/contracts/analysisViewToolContracts.js"; +import { analysisViewResultSchema } from "../../../src/domain/analysisView/analysisView.js"; +import { AnalysisOutputError } from "../../../src/domain/analysisErrorCore.js"; +import { + createEvidence, + parseEvidence, + type Evidence, +} from "../../../src/domain/evidence.js"; +import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js"; +import { jsonObjectSchema } from "../../../src/domain/jsonValue.js"; +import { batchDecompileResultSchema } from "../../../src/domain/native/batchDecompile.js"; +import { err, ok } from "../../../src/domain/result.js"; +import { GhidraSessionError } from "../../../src/ghidra/GhidraSessionError.js"; +import { createServer } from "../../../src/server/createServer.js"; +import { createToolResultDelivery } from "../../../src/server/toolResult.js"; +import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; +import { createTestBinarySession } from "../../fixtures/binarySession.js"; +import { createMcpSchemaValidation } from "../../fixtures/mcpSchemaValidation.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { + closeEnhancedToolResources, + connect as connectFixedAnalysis, +} from "./enhancedToolsHarness.js"; +import { connectGhidraMcp, ghidraFixtureResult } from "./ghidraMcpHarness.js"; + +const MAIN_BODY = "int main(void) { return 0; }"; +const EMOJI_BODY = `int emoji(void) { return "${"\u{1F600}".repeat(4)}"; }`; +const NO_BODY_REASON = "Thunk has no decompilable body."; + +/** Selectors exercising a duplicate, an interior address, no body and no identity. */ +const SELECTORS = ["main", "0x401005", "main", "emoji", "bodyless", "missing"]; +const ENTRIES: Readonly> = { + main: "0x401000", + "0x401005": "0x401000", + emoji: "0x402000", + bodyless: "0x403000", + unlabeled: "0x404000", + "label-failure": "0x405000", +}; +const LABELS: Readonly> = { + "0x401000": "main", + "0x402000": "emoji", + "0x403000": "bodyless", +}; +const BODIES: Readonly> = { + "0x401000": MAIN_BODY, + "0x402000": EMOJI_BODY, +}; + +const provider = (calls: { count: number }, bodies: typeof BODIES) => + ((operation, input) => { + calls.count++; + if (operation === "procedure_address") { + const selector = String(input.procedure); + const entry = ENTRIES[selector]; + return Promise.resolve( + entry === undefined + ? err( + new GhidraSessionError( + "remote", + `no function: ${selector}`, + {}, + { remoteCode: "not_found" }, + ), + ) + : ok(entry), + ); + } + if (operation === "address_name") + return Promise.resolve( + input.address === "0x405000" + ? err(new GhidraSessionError("remote", "label lookup failed")) + : ok(LABELS[String(input.address)] ?? null), + ); + if (operation === "procedure_pseudo_code") { + const body = bodies[String(input.procedure)]; + return Promise.resolve( + ok( + body === undefined + ? { value: null, limitations: [NO_BODY_REASON] } + : { value: body, limitations: [] }, + ), + ); + } + return Promise.resolve(ok(ghidraFixtureResult(operation, input))); + }) satisfies NonNullable[1]>; + +const [inspectAnalysisViewContract] = ANALYSIS_VIEW_TOOL_CONTRACTS.filter( + ({ name }) => name === "inspect_analysis_view", +); +if (inspectAnalysisViewContract === undefined) + throw new Error("Missing inspect_analysis_view contract"); + +const schemaValidation = createMcpSchemaValidation(); +afterAll(() => schemaValidation.clear()); + +/** Replies these tools deliver must validate against what they advertise. */ +const ADVERTISED_OUTPUTS = ["batch_decompile", "inspect_analysis_view"]; + +const connect = async ( + profile: "full" | "compact", + options: { + readonly budget?: number; + readonly bodies?: typeof BODIES; + readonly execute?: Parameters[1]; + } = {}, +) => { + const calls = { count: 0 }; + const harness = await connectGhidraMcp( + `batch-decompile-views-${profile}`, + options.execute ?? provider(calls, options.bodies ?? BODIES), + { + environment: { + REA_MCP_INPUT_SCHEMA_PROFILE: profile, + REA_MCP_OUTPUT_SCHEMA_PROFILE: profile, + }, + schemaValidator: schemaValidation, + ...(options.budget === undefined + ? {} + : { delivery: createToolResultDelivery(options.budget) }), + }, + ); + onTestFinished(() => harness.close()); + const { tools } = await harness.mcp.listTools(); + const outputSchemas = new Map( + ADVERTISED_OUTPUTS.map((name) => { + const advertised = tools.find((tool) => tool.name === name)?.outputSchema; + if (advertised === undefined) + throw new Error(`${name} advertises no output schema`); + const schema: Record = advertised; + return [name, schema] as const; + }), + ); + const reply = async (name: string, args: Record) => + z + .custom() + .parse(await harness.mcp.callTool({ name, arguments: args })); + const call = async (name: string, args: Record) => { + const result = await reply(name, args); + expect(result.isError, JSON.stringify(result)).not.toBe(true); + const schema = outputSchemas.get(name); + if (schema !== undefined) + expect( + schemaValidation.assertions.validate(schema, result.structuredContent), + JSON.stringify(schemaValidation.assertions.errors), + ).toBe(true); + return { evidence: parseEvidence(result.structuredContent), result }; + }; + return { ...harness, calls, reply, call }; +}; + +type Harness = Awaited>; + +const viewOf = (evidence: Evidence) => + z.looseObject({}).parse(evidence.normalized_result); + +const retained = (evidenceId: string) => ({ + kind: "retained-evidence", + evidence_id: evidenceId, +}); + +const view = async ( + { call }: Harness, + parentId: string, + selected: Record, +) => + viewOf( + ( + await call("inspect_analysis_view", { + source: retained(parentId), + view: { kind: "batch-decompile", ...selected }, + }) + ).evidence, + ); + +const rowIndices = (page: Record) => + z + .array(z.looseObject({ index: z.number() })) + .parse(page.items) + .map(({ index }) => index); + +/** Read every text page of one retained body in order. */ +const readBody = async ( + harness: Harness, + parentId: string, + index: number, + limit?: number, +) => { + let text = ""; + let pages = 0; + let offset: number | null = 0; + while (offset !== null) { + const page = await view(harness, parentId, { + collection: "pseudocode", + index, + offset, + ...(limit === undefined ? {} : { limit }), + }); + text += z.looseObject({ text: z.string() }).parse(page.item).text; + offset = z + .looseObject({ next_offset: z.number().nullable() }) + .parse(page.coverage).next_offset; + pages++; + } + return { text, pages }; +}; + +/** A typed refusal whose issue names the rejected selection. */ +const invalidIssue = (message: string) => ({ + code: "invalid_request", + details: { + issues: expect.arrayContaining([ + expect.objectContaining({ message: expect.stringContaining(message) }), + ]), + }, +}); + +it.each(["full", "compact"] as const)( + "selects retained batch rows and bodies from a summary without repeating analysis (%s)", + async (profile) => { + const harness = await connect(profile); + const reply = await harness.call("batch_decompile", { + addresses: SELECTORS, + detail: "summary", + }); + const summary = viewOf(reply.evidence); + const parentId = z.string().parse(summary.parent_evidence_id); + const parent = parseEvidence(harness.session.evidenceById(parentId)); + + // The selection never reaches the record, so the parent is the same analysis. + expect(parent.parameters).toEqual({ addresses: SELECTORS }); + expect( + batchDecompileResultSchema.parse(parent.normalized_result), + ).toMatchObject({ total: 6, succeeded: 4, failed: 2 }); + expect(JSON.stringify(reply.result)).not.toContain("int main"); + expect(reply.evidence.evidence_links).toEqual([parentId]); + expect(reply.evidence.limitations).toEqual( + expect.arrayContaining(parent.limitations), + ); + expect(summary).toMatchObject({ + kind: "summary", + parent_operation: "batch_decompile", + summary: { + batch: { total: 6, succeeded: 4, failed: 2 }, + source_provider: parent.provider, + source_analysis_profile_digest: parent.analysis_profile?.digest, + decompiling_provider: { id: "ghidra", version: "12.1.4" }, + procedures: { + distinct_resolved: 3, + unknown_identity: 1, + label_unavailable: 0, + }, + pseudocode: { + code_units: MAIN_BODY.length * 3 + EMOJI_BODY.length, + largest: { index: 3, code_units: EMOJI_BODY.length }, + }, + }, + }); + expect(summary.unknowns).toEqual( + expect.arrayContaining([ + expect.stringContaining("1 items have unknown procedure identity"), + expect.stringContaining("not attributed to one successful item"), + ]), + ); + const callsAfterBatch = harness.calls.count; + + // The same summary is reachable later through the retained parent. + expect( + viewOf( + ( + await harness.call("inspect_analysis_view", { + source: retained(parentId), + view: { kind: "summary" }, + }) + ).evidence, + ).summary, + ).toEqual(summary.summary); + + // The default page is an index without bodies. + const index = await view(harness, parentId, { collection: "items" }); + expect(JSON.stringify(index.items)).not.toContain("int main"); + expect(index).toMatchObject({ + batch: { total: 6, succeeded: 4, failed: 2 }, + coverage: { total: 6, exhausted: true }, + }); + expect(index.items).toMatchObject([ + { + index: 0, + address: "main", + procedure: { status: "resolved", address: "0x401000", name: "main" }, + status: "ok", + pseudocode_code_units: MAIN_BODY.length, + }, + { index: 1, address: "0x401005" }, + { index: 2, address: "main" }, + { index: 3, address: "emoji" }, + { + index: 4, + status: "error", + error: { code: "capability_unavailable" }, + limitations: expect.arrayContaining([NO_BODY_REASON]), + }, + { index: 5, procedure: { status: "unknown" } }, + ]); + + // Filters are exact, compose, and keep original indices. + for (const [where, indices] of [ + [{ addresses: ["main"] }, [0, 2]], + [{ procedure_addresses: ["0x401000"] }, [0, 1, 2]], + // A selector never matches as an observed entry, including an unknown one. + [{ procedure_addresses: ["missing"] }, []], + [{ statuses: ["error"] }, [4, 5]], + [{ indices: [5, 3, 3] }, [3, 5]], + [{ statuses: ["ok"], procedure_addresses: ["0x402000"] }, [3]], + [{ indices: [] }, []], + [{ addresses: [] }, []], + [{ procedure_addresses: [] }, []], + [{ statuses: [] }, []], + ] as const) + expect( + rowIndices( + await view(harness, parentId, { collection: "items", where }), + ), + JSON.stringify(where), + ).toEqual(indices); + + // Coverage counts matches rather than the whole batch. + const paged = await view(harness, parentId, { + collection: "items", + where: { statuses: ["ok"] }, + offset: 1, + limit: 2, + }); + expect(rowIndices(paged)).toEqual([1, 2]); + expect(paged.coverage).toMatchObject({ + status: "page", + examined: 2, + total: 4, + next_offset: 3, + exhausted: false, + }); + + // Bodies are opt-in, and only successful items have one. + const selected = await view(harness, parentId, { + collection: "items", + where: { indices: [3, 4] }, + include_pseudocode: true, + }); + expect(selected.items).toMatchObject([ + { index: 3, pseudocode: EMOJI_BODY }, + { index: 4, status: "error" }, + ]); + expect(JSON.stringify(selected.items)).not.toContain('"pseudocode":null'); + + expect(harness.calls.count).toBe(callsAfterBatch); + expect(harness.session.evidenceById(parentId)).toEqual(parent); + }, +); + +it("counts both forms of an unavailable entry label", async () => { + const harness = await connect("full"); + const parent = ( + await harness.call("batch_decompile", { + addresses: ["main", "unlabeled", "label-failure"], + }) + ).evidence.evidence_id; + const summary = viewOf( + ( + await harness.call("inspect_analysis_view", { + source: retained(parent), + view: { kind: "summary" }, + }) + ).evidence, + ); + expect(summary.summary).toMatchObject({ + procedures: { + distinct_resolved: 3, + unknown_identity: 0, + label_unavailable: 2, + }, + }); + const { items } = await view(harness, parent, { collection: "items" }); + expect(items).toMatchObject([ + { procedure: { name: "main" } }, + { procedure: { name: null } }, + { procedure: { name: null, name_error: { code: expect.any(String) } } }, + ]); + expect(JSON.stringify(z.array(z.unknown()).parse(items)[1])).not.toContain( + "name_error", + ); +}); + +it("pages one body exactly without splitting a Unicode character", async () => { + const harness = await connect("full"); + const { evidence } = await harness.call("batch_decompile", { + addresses: SELECTORS, + }); + const parent = evidence.evidence_id; + // The complete delivery still carries every body by default. + expect(JSON.stringify(evidence.normalized_result)).toContain("int main"); + + // A limit ending inside a surrogate pair stops before it, so pages rebuild the text. + const { text, pages } = await readBody(harness, parent, 3, 30); + expect(text).toBe(EMOJI_BODY); + expect(pages).toBeGreaterThan(1); + expect( + await view(harness, parent, { collection: "pseudocode", index: 3 }), + ).toMatchObject({ + item: { + index: 3, + address: "emoji", + procedure: { status: "resolved", address: "0x402000" }, + pseudocode_code_units: EMOJI_BODY.length, + unit: "utf16-code-units", + }, + coverage: { total: EMOJI_BODY.length, exhausted: true }, + }); + + const refused = async (selected: Record) => + parseMcpToolError( + await harness.reply("inspect_analysis_view", { + source: retained(parent), + view: { + kind: "batch-decompile", + collection: "pseudocode", + ...selected, + }, + }), + ).error; + expect(await refused({ index: 4 })).toMatchObject( + invalidIssue("Item 4 has status error"), + ); + expect(await refused({ index: 6 })).toMatchObject( + invalidIssue("outside this batch of 6"), + ); + expect( + await refused({ index: 3, offset: EMOJI_BODY.indexOf("\u{1F600}") + 1 }), + ).toMatchObject(invalidIssue("splits a UTF-16 surrogate pair")); +}); + +it("recovers oversized batches and pages through smaller selections of the parent", async () => { + const budget = 64 * 1024; + const large = `int main(void) { /*${"x".repeat(budget * 2)}*/ }`; + const harness = await connect("full", { + budget, + bodies: { "0x401000": large, "0x402000": large }, + }); + const constraint = (result: CallToolResult) => { + const { error } = parseMcpToolError(result); + expect(error.code).toBe("resource_constraint"); + const limits = z + .object({ details: z.object({ reported_limits: jsonObjectSchema }) }) + .parse(error).details.reported_limits; + return z + .object({ + name: z.literal("inspect_analysis_view"), + arguments: inspectAnalysisViewContract.inputSchema, + }) + .parse(limits.continuation); + }; + + // The complete batch names a body-free first page of its retained parent. + const first = constraint( + await harness.reply("batch_decompile", { + addresses: ["main", "emoji", "bodyless"], + }), + ); + const source = first.arguments.source; + if (source.kind !== "retained-evidence") throw new Error("Expected parent"); + expect(first.arguments.view).toEqual({ + kind: "batch-decompile", + collection: "items", + where: {}, + include_pseudocode: false, + offset: 0, + limit: 64, + }); + const rows = await harness.call(first.name, first.arguments); + expect(rowIndices(viewOf(rows.evidence))).toEqual([0, 1, 2]); + expect(Buffer.byteLength(JSON.stringify(rows.result))).toBeLessThan(budget); + + // Several bodies that cannot fit continue with fewer rows from the same parent. + const withBodies = { + kind: "batch-decompile", + collection: "items", + include_pseudocode: true, + }; + const fewer = constraint( + await harness.reply("inspect_analysis_view", { + source, + view: withBodies, + }), + ); + expect(fewer.arguments).toMatchObject({ + source, + view: { ...withBodies, limit: 1 }, + }); + + // One body that cannot fit continues with its text pages rather than an export. + const text = constraint(await harness.reply(fewer.name, fewer.arguments)); + expect(text.arguments).toMatchObject({ + source, + view: { kind: "batch-decompile", collection: "pseudocode", index: 0 }, + }); + const oversizedPage = constraint( + await harness.reply("inspect_analysis_view", { + source, + view: { ...text.arguments.view, limit: budget * 2 }, + }), + ); + expect(oversizedPage.arguments).toMatchObject({ + source, + view: { collection: "pseudocode", index: 0, limit: budget }, + }); + expect((await readBody(harness, source.evidence_id, 0)).text).toBe(large); +}); + +it("reads the same rows from a bundle and an Evidence file without a provider", async () => { + const harness = await connect("full"); + const parent = ( + await harness.call("batch_decompile", { addresses: SELECTORS }) + ).evidence; + const selected = { + kind: "batch-decompile", + collection: "items", + where: { addresses: ["emoji"] }, + include_pseudocode: true, + }; + const { items } = viewOf( + ( + await harness.call("inspect_analysis_view", { + source: retained(parent.evidence_id), + view: selected, + }) + ).evidence, + ); + expect(items).toMatchObject([{ index: 3, pseudocode: EMOJI_BODY }]); + + const root = await createTestTempDirectory("rea-batch-decompile-view-"); + const bundlePath = join(root, "bundle.json"); + const filePath = join(root, "batch.json"); + await writeFile(bundlePath, JSON.stringify(createEvidenceBundle([parent]))); + await writeFile(filePath, JSON.stringify(parent)); + const { client, close } = await createApplicationMcpHarness({ + session: createTestBinarySession(() => { + throw new Error("Retained batch views must not open a provider"); + }), + clientIdentity: { name: "batch-decompile-offline", version: "1.0.0" }, + schemaValidator: schemaValidation, + }); + onTestFinished(close); + const imported = await client.callTool({ + name: "import_evidence_bundle", + arguments: { path: bundlePath }, + }); + expect(imported.isError).not.toBe(true); + for (const source of [ + retained(parent.evidence_id), + { kind: "evidence-file", path: filePath }, + ]) { + const reply = await client.callTool({ + name: "inspect_analysis_view", + arguments: { source, view: selected }, + }); + expect(viewOf(parseEvidence(reply.structuredContent)).items).toEqual(items); + } +}); + +it("keeps the caller's own source when a page cannot fit, without retaining its parent", async () => { + const budget = 64 * 1024; + const body = `int f(void) { /*${"x".repeat(budget * 2)}*/ }`; + const row = { + procedure: { status: "resolved", address: "0x1", name: null }, + status: "ok", + pseudocode: body, + }; + const parent = createEvidence( + undefined, + { id: "fixture", name: "Fixture", version: "1" }, + { + operation: "batch_decompile", + parameters: {}, + result: { + items: [ + { address: "a", ...row }, + { address: "b", ...row }, + ], + total: 2, + succeeded: 2, + failed: 0, + }, + predicateType: "rea.analysis", + confidence: "derived", + authority: "shipped-artifact", + }, + ); + const root = await createTestTempDirectory("rea-batch-decompile-source-"); + const path = join(root, "batch.json"); + await writeFile(path, JSON.stringify(parent)); + const session = createTestBinarySession(() => { + throw new Error("Retained batch views must not open a provider"); + }); + const { client, close } = await createApplicationMcpHarness({ + session, + clientIdentity: { name: "batch-decompile-source", version: "1.0.0" }, + schemaValidator: schemaValidation, + serverOptions: { delivery: createToolResultDelivery(budget) }, + }); + onTestFinished(close); + const inspect = (source: unknown, selected: Record) => + client.callTool({ + name: "inspect_analysis_view", + arguments: { source, view: { kind: "batch-decompile", ...selected } }, + }); + const continuationOf = (result: Parameters[0]) => + z + .object({ details: z.object({ reported_limits: jsonObjectSchema }) }) + .parse(parseMcpToolError(result).error).details.reported_limits + .continuation; + const withBodies = { collection: "items", include_pseudocode: true }; + + // A file source continues from that file: its parent was never retained. + const file = { kind: "evidence-file", path }; + expect(continuationOf(await inspect(file, withBodies))).toMatchObject({ + arguments: { source: file, view: { limit: 1 } }, + }); + const oneRow = continuationOf( + await inspect(file, { ...withBodies, limit: 1 }), + ); + expect(oneRow).toMatchObject({ + arguments: { + source: file, + view: { collection: "pseudocode", index: 0 }, + }, + }); + const followed = await client.callTool( + z.object({ name: z.string(), arguments: jsonObjectSchema }).parse(oneRow), + ); + expect(followed.isError, JSON.stringify(followed)).not.toBe(true); + expect(session.hasEvidence(parent.evidence_id)).toBe(false); + + // An inline record cannot be repeated within the reply it overflowed. + const inline = { kind: "inline", evidence: parent }; + const overflowed = await inspect(inline, withBodies); + expect(parseMcpToolError(overflowed).error.code).toBe("resource_constraint"); + expect(continuationOf(overflowed)).toBeUndefined(); + + // A server that retains nothing still reads the file, and can name it again. + const unretained = createServer( + { + kind: "fixed", + analysis: { + execute: () => { + throw new Error("Retained batch views must not call a provider"); + }, + }, + }, + { delivery: createToolResultDelivery(budget) }, + ); + const fixedClient = new Client({ + name: "batch-unretained", + version: "1.0.0", + }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + onTestFinished(async () => { + await fixedClient.close(); + await unretained.close(); + }); + await unretained.connect(serverTransport); + await fixedClient.connect(clientTransport); + const noRecorder = continuationOf( + await fixedClient.callTool({ + name: "inspect_analysis_view", + arguments: { + source: file, + view: { kind: "batch-decompile", ...withBodies }, + }, + }), + ); + expect(noRecorder).toMatchObject({ + arguments: { source: file, view: { limit: 1 } }, + }); + const followedUnretained = await fixedClient.callTool( + z + .object({ name: z.string(), arguments: jsonObjectSchema }) + .parse( + continuationOf( + await fixedClient.callTool( + z + .object({ name: z.string(), arguments: jsonObjectSchema }) + .parse(noRecorder), + ), + ), + ), + ); + expect( + followedUnretained.isError, + JSON.stringify(followedUnretained), + ).not.toBe(true); +}); + +it("delivers no summary without retention, on a ledger failure or after cancellation", async () => { + onTestFinished(closeEnhancedToolResources); + let calls = 0; + const unretained = await connectFixedAnalysis({ + execute: () => { + calls++; + throw new Error("Selected delivery must fail before analysis"); + }, + }); + expect( + parseMcpToolError( + await unretained.callTool({ + name: "batch_decompile", + arguments: { addresses: ["0x1"], detail: "summary" }, + }), + ).error, + ).toMatchObject({ code: "capability_unavailable" }); + expect(calls).toBe(0); + + const ledger = await connect("full"); + vi.spyOn(ledger.session, "recordEvidence").mockReturnValue( + err(new AnalysisOutputError("record_evidence", "ledger refused the batch")), + ); + const refused = await ledger.reply("batch_decompile", { + addresses: ["main"], + detail: "summary", + }); + expect(parseMcpToolError(refused).error.code).toBe("unreadable_output"); + + const cancelled = await connect("full", { + execute: (operation, input) => + Promise.resolve( + operation === "procedure_pseudo_code" + ? err(new GhidraSessionError("cancelled", "Decompile cancelled")) + : ok(ghidraFixtureResult(operation, input)), + ), + }); + const stopped = await cancelled.reply("batch_decompile", { + addresses: ["main"], + detail: "summary", + }); + expect(parseMcpToolError(stopped).error.code).toBe("cancelled"); + const records = await cancelled.reply("get_evidence_bundle", { + detail: "summary", + filters: { operation: "batch_decompile" }, + }); + expect(records.structuredContent).toMatchObject({ result: { records: [] } }); +}); + +it("binds views to their parent and each page to its own collection", async () => { + const harness = await connect("full"); + const summary = viewOf( + ( + await harness.call("batch_decompile", { + addresses: [], + detail: "summary", + }) + ).evidence, + ); + expect(summary.summary).toMatchObject({ + batch: { total: 0, succeeded: 0, failed: 0 }, + procedures: { distinct_resolved: 0, unknown_identity: 0 }, + pseudocode: { code_units: 0, largest: null }, + }); + expect( + await view(harness, z.string().parse(summary.parent_evidence_id), { + collection: "items", + }), + ).toMatchObject({ + items: [], + coverage: { status: "empty", exhausted: true }, + }); + + const parent = ( + await harness.call("batch_decompile", { addresses: ["main"] }) + ).evidence.evidence_id; + const items = await view(harness, parent, { collection: "items" }); + const body = await view(harness, parent, { + collection: "pseudocode", + index: 0, + }); + expect(analysisViewResultSchema.safeParse(items).success).toBe(true); + expect(analysisViewResultSchema.safeParse(body).success).toBe(true); + expect( + analysisViewResultSchema.safeParse({ ...items, view: body.view }).success, + ).toBe(false); + const { item: _item, ...withoutItem } = body; + expect( + analysisViewResultSchema.safeParse({ ...withoutItem, items: items.items }) + .success, + ).toBe(false); + + const inspect = async (source: unknown, selected: unknown) => + harness.reply("inspect_analysis_view", { source, view: selected }); + expect( + parseMcpToolError( + await inspect(retained(parent), { kind: "native-inventory" }), + ).error, + ).toMatchObject(invalidIssue("batch_decompile Evidence supports")); + const inventory = (await harness.call("list_procedures", {})).evidence; + expect( + parseMcpToolError( + await inspect(retained(inventory.evidence_id), { + kind: "batch-decompile", + collection: "items", + }), + ).error.code, + ).toBe("invalid_request"); + + // Inline parents authenticate their predicate and result before selection. + const inline = ( + predicateType: string, + result: Evidence["normalized_result"], + ) => + inspect( + { + kind: "inline", + evidence: createEvidence( + undefined, + { id: "fixture", name: "Fixture", version: "1" }, + { + operation: "batch_decompile", + parameters: {}, + result, + predicateType, + confidence: "derived", + authority: "shipped-artifact", + }, + ), + }, + { kind: "batch-decompile", collection: "items" }, + ); + const valid = { + items: [ + { + address: "main", + procedure: { status: "resolved", address: "0x1", name: null }, + status: "ok", + pseudocode: "x", + }, + ], + total: 1, + succeeded: 1, + failed: 0, + }; + // A parent without a target stays explicit rather than inventing identity. + const accepted = await inline("rea.analysis", valid); + expect(viewOf(parseEvidence(accepted.structuredContent))).toMatchObject({ + artifact: null, + decompiling_provider: null, + source_analysis_profile_digest: null, + }); + for (const rejected of [ + await inline("rea.other", valid), + await inline("rea.analysis", { + ...valid, + items: [{ address: "main", status: "ok" }], + }), + ]) + expect(parseMcpToolError(rejected).error.code).toBe("invalid_request"); +}); diff --git a/tests/boundary/mcp/callGraphCleanupIncomplete.test.ts b/tests/boundary/mcp/callGraphCleanupIncomplete.test.ts new file mode 100644 index 0000000000..e0ec7a107f --- /dev/null +++ b/tests/boundary/mcp/callGraphCleanupIncomplete.test.ts @@ -0,0 +1,428 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { afterEach, describe, expect, it } from "vitest"; +import { z } from "zod"; + +import { + createAnalysisExecution, + type AnalysisOperationPort, +} from "../../../src/application/AnalysisProvider.js"; +import { AnalysisError } from "../../../src/domain/analysisErrorBase.js"; +import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js"; +import { AnalysisCapabilityUnavailableError } from "../../../src/domain/analysisErrorCore.js"; +import { createAnalysisExecutionEvidence } from "../../../src/domain/analysisObservation.js"; +import { parseEvidence } from "../../../src/domain/evidence.js"; +import { evidenceBundleSchema } from "../../../src/domain/evidenceBundle.js"; +import { err, ok } from "../../../src/domain/result.js"; +import { createServer } from "../../../src/server/createServer.js"; +import { cleanupStoppedTraversal } from "../../../src/application/CallRelationshipTraversal.js"; +import { createTestBinarySession } from "../../fixtures/binarySession.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { + closeEnhancedToolResources, + connect, + jsonResult, + resources, +} from "./enhancedToolsHarness.js"; + +afterEach(closeEnhancedToolResources); + +const provider = { + id: "fixture", + name: "Fixture analysis provider", + version: "1", +} as const; +const directions = [ + ["forward", "procedure_callees"], + ["backward", "procedure_callers"], +] as const; +const tools = ["get_call_graph", "trace_call_path"] as const; +type Tool = (typeof tools)[number]; +type Relationships = Readonly< + Record +>; + +const childFailure = (operation: string, cleanup: boolean) => + new AnalysisCapabilityUnavailableError( + provider.id, + operation, + "child failed", + cleanup + ? { + cleanup: { + reason: "worker cleanup failed", + resources: ["owned worker"], + }, + } + : undefined, + ); + +/** Answer each selector's relationship read and record every read in order. */ +const relationshipPort = ( + relationships: Relationships, + reads: string[], +): AnalysisOperationPort => ({ + execute: async (operation, parameters) => { + const procedure = String(parameters.procedure); + if (operation === "procedure_address") + return ok(createAnalysisExecution(procedure, provider)); + if (operation !== "procedure_callees" && operation !== "procedure_callers") + return ok(createAnalysisExecution(null, provider)); + reads.push(`${operation}:${procedure}`); + const answer = relationships[procedure] ?? []; + return answer instanceof AnalysisError + ? err(answer) + : ok( + createAnalysisExecution(answer, provider, { + relationshipCoverage: { status: "complete" }, + }), + ); + }, +}); + +const call = ( + tool: Tool, + direction: (typeof directions)[number][0], + goal = "0x4", +) => ({ + name: tool, + arguments: + tool === "get_call_graph" + ? { address: "0x1", direction } + : { start: "0x1", goal, direction }, +}); + +describe.each(directions)("%s relationship reads", (direction, operation) => { + it.each(tools)( + "%s stops at unconfirmed cleanup and reports it with the admitted result", + async (tool) => { + const reads: string[] = []; + const client = await connect( + relationshipPort( + { + "0x1": ["0x2", "0x3"], + "0x2": childFailure(operation, true), + "0x3": ["0x4"], + }, + reads, + ), + ); + + const response = await client.callTool(call(tool, direction)); + + expect(reads).toEqual([`${operation}:0x1`, `${operation}:0x2`]); + const { error } = parseMcpToolError(response); + expect(error).toMatchObject({ + code: "cleanup_incomplete", + details: { + operation, + cleanup: "incomplete", + resources: ["owned worker"], + cleanup_reason: "worker cleanup failed", + execution_failure: "capability_unavailable", + }, + }); + // Without an Evidence writer the admitted result stays inline. + const partial = parseEvidence(error.details?.partial_observation); + expect(partial).toMatchObject({ + operation: tool, + confidence: "derived", + parameters: call(tool, direction).arguments, + }); + // The stop names 0x2 and its one queued sibling; the formatter owns prose. + const stopped = cleanupStoppedTraversal("0x2", 1); + if (tool === "get_call_graph") + expect(partial.normalized_result).toEqual({ + graph: { + "0": [ + { + address: "0x1", + status: "ok", + calls: ["0x2", "0x3"], + relationship_coverage: { status: "complete" }, + }, + ], + }, + limitations: [stopped], + }); + else + expect(partial.normalized_result).toMatchObject({ + goal: "0x4", + goal_status: "unknown", + nodes: [ + { address: "0x1", depth: 0 }, + { address: "0x2", depth: 1 }, + { address: "0x3", depth: 1 }, + ], + traversal_path: [], + failures: [], + relationship_coverage: { "0x1": { status: "complete" } }, + truncated: true, + residual_unknowns: [stopped], + }); + }, + ); + + it.each(tools)( + "%s keeps an ordinary expansion failure as a usable partial result", + async (tool) => { + const reads: string[] = []; + const client = await connect( + relationshipPort( + { + "0x1": ["0x2", "0x3"], + "0x2": childFailure(operation, false), + "0x3": ["0x4"], + }, + reads, + ), + ); + + const response = await client.callTool(call(tool, direction)); + + expect(response.isError).not.toBe(true); + expect(reads).toEqual([ + `${operation}:0x1`, + `${operation}:0x2`, + `${operation}:0x3`, + ...(tool === "get_call_graph" ? [`${operation}:0x4`] : []), + ]); + const failed = { + address: "0x2", + error: { + code: "capability_unavailable", + details: { operation }, + }, + }; + expect(jsonResult(response)).toMatchObject( + tool === "get_call_graph" + ? { + graph: { + "1": [{ ...failed, status: "error" }, { address: "0x3" }], + }, + } + : { + goal_status: "reached", + traversal_path: ["0x1", "0x3", "0x4"], + failures: [failed], + }, + ); + }, + ); +}); + +it.each(tools)( + "%s keeps earlier ordinary failures and dispatches nothing after a later cleanup failure", + async (tool) => { + const reads: string[] = []; + const client = await connect( + relationshipPort( + { + "0x1": ["0x2", "0x3", "0x4"], + "0x2": childFailure("procedure_callees", false), + "0x3": childFailure("procedure_callees", true), + "0x4": ["0x5"], + }, + reads, + ), + ); + + const response = await client.callTool(call(tool, "forward", "0x5")); + + expect(reads).toEqual([ + "procedure_callees:0x1", + "procedure_callees:0x2", + "procedure_callees:0x3", + ]); + const { error } = parseMcpToolError(response); + expect(error).toMatchObject({ + code: "cleanup_incomplete", + details: { resources: ["owned worker"] }, + }); + const partial = parseEvidence(error.details?.partial_observation); + const stopped = cleanupStoppedTraversal("0x3", 1); + const ordinary = { + address: "0x2", + error: { code: "capability_unavailable" }, + }; + expect(partial.normalized_result).toMatchObject( + tool === "get_call_graph" + ? { + graph: { + "0": [{ address: "0x1", status: "ok" }], + "1": [{ ...ordinary, status: "error" }], + }, + limitations: [stopped], + } + : { + goal_status: "unknown", + failures: [ordinary], + residual_unknowns: [ + "Call relationships were unavailable for 0x2.", + stopped, + ], + }, + ); + }, +); + +/** Serve one fixed provider through a production session with a committed profile. */ +const openSession = async (port: AnalysisOperationPort) => { + const directory = await createTestTempDirectory("rea-call-graph-cleanup-"); + const target = join(directory, "fixture.com"); + await writeFile(target, Buffer.from([0xb8, 0x00, 0x4c, 0xcd, 0x21])); + const session = createTestBinarySession( + () => ({ + execute: (operation, parameters, options) => + port.execute(operation, parameters, options), + close: async () => ok(null), + }), + { + // A committed profile lets the session retain each admitted read. + resolveAnalysisProfile: async () => + ok({ profile: createAnalysisProfile(provider, { fixture: true }) }), + }, + ); + const server = createServer({ kind: "session", session }); + const client = new Client({ name: "call-graph-cleanup", version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + resources.push(client, server); + await server.connect(serverTransport); + await client.connect(clientTransport); + const opened = await client.callTool({ + name: "open_binary", + arguments: { path: target, format: "dos-com" }, + }); + expect(opened.isError).not.toBe(true); + return { client, session }; +}; + +/** Resolve a failure's retained partial and prove a complete export keeps it. */ +const retainedPartial = async ( + client: Client, + session: ReturnType, + error: ReturnType["error"], +) => { + const reference = z + .strictObject({ + kind: z.literal("retained-evidence"), + evidence_id: z.string(), + }) + .parse(error.details?.partial_observation); + const exported = await client.callTool({ + name: "get_evidence_bundle", + arguments: {}, + }); + expect(exported.isError).not.toBe(true); + const bundle = evidenceBundleSchema.parse( + z.object({ result: z.unknown() }).parse(exported.structuredContent).result, + ); + expect(bundle.records.map(({ evidence_id }) => evidence_id)).toContain( + reference.evidence_id, + ); + return session.evidenceForAnalysis(reference.evidence_id); +}; + +it.each(tools)( + "%s retains the admitted result as session Evidence and references it", + async (tool) => { + const reads: string[] = []; + const { client, session } = await openSession( + relationshipPort( + { + "0x1": ["0x2", "0x3"], + "0x2": childFailure("procedure_callees", true), + "0x3": ["0x4"], + }, + reads, + ), + ); + + const response = await client.callTool(call(tool, "forward")); + + expect(reads).toEqual(["procedure_callees:0x1", "procedure_callees:0x2"]); + const { error } = parseMcpToolError(response); + expect(error).toMatchObject({ + code: "cleanup_incomplete", + details: { resources: ["owned worker"] }, + }); + const retained = await retainedPartial(client, session, error); + expect(retained).toMatchObject({ + operation: tool, + subject: { format: "dos-com" }, + normalized_result: + tool === "get_call_graph" + ? { graph: { "0": [{ address: "0x1", status: "ok" }] } } + : { goal_status: "unknown", truncated: true }, + }); + // The admitted read stays linked as the partial result's source. + expect(retained?.evidence_links).toHaveLength(1); + expect( + session.evidenceForAnalysis(retained?.evidence_links?.[0] ?? ""), + ).toMatchObject({ + operation: "procedure_callees", + parameters: { procedure: "0x1" }, + }); + }, +); + +it.each(tools)( + "%s retains a failed read's own partial Evidence instead of the workflow's", + async (tool) => { + const failure = childFailure("procedure_callees", true); + const own = createAnalysisExecutionEvidence( + "procedure_callees", + { procedure: "0x2" }, + createAnalysisExecution(["0x9"], provider), + "observed", + ); + failure.retainPartialObservation(own); + const reads: string[] = []; + const { client, session } = await openSession( + relationshipPort({ "0x1": ["0x2", "0x3"], "0x2": failure }, reads), + ); + + const { error } = parseMcpToolError( + await client.callTool(call(tool, "forward")), + ); + + expect(reads).toEqual(["procedure_callees:0x1", "procedure_callees:0x2"]); + expect(error).toMatchObject({ + code: "cleanup_incomplete", + details: { resources: ["owned worker"] }, + }); + expect(await retainedPartial(client, session, error)).toEqual(own); + }, +); + +it.each(tools)( + "%s stops at a root read cleanup failure without dispatching more reads", + async (tool) => { + const reads: string[] = []; + const client = await connect( + relationshipPort( + { "0x1": childFailure("procedure_callees", true), "0x4": [] }, + reads, + ), + ); + + const { error } = parseMcpToolError( + await client.callTool(call(tool, "forward")), + ); + + expect(reads).toEqual(["procedure_callees:0x1"]); + expect(error.code).toBe("cleanup_incomplete"); + const stopped = cleanupStoppedTraversal("0x1", 0); + expect( + parseEvidence(error.details?.partial_observation).normalized_result, + ).toMatchObject( + tool === "get_call_graph" + ? { graph: {}, limitations: [stopped] } + : { goal_status: "unknown", residual_unknowns: [stopped] }, + ); + }, +); diff --git a/tests/boundary/mcp/capabilityRemediationMcp.test.ts b/tests/boundary/mcp/capabilityRemediationMcp.test.ts new file mode 100644 index 0000000000..54fcf13294 --- /dev/null +++ b/tests/boundary/mcp/capabilityRemediationMcp.test.ts @@ -0,0 +1,197 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import type { CallToolResult } from "@modelcontextprotocol/server"; +import { afterAll, expect, it, onTestFinished } from "vitest"; + +import { AnalysisProviderRegistry } from "../../../src/application/binary/AnalysisProviderRegistry.js"; +import { composeBinarySession } from "../../../src/application/binary/BinarySessionComposition.js"; +import { parseConfig } from "../../../src/config/parseConfig.js"; +import { toolContract } from "../../../src/contracts/toolContracts.js"; +import { elf } from "../../../src/domain/binaryTarget.fixture.js"; +import { IdaProvider } from "../../../src/ida/IdaProvider.js"; +import { silentLogger } from "../../../src/logger.js"; +import { createServer } from "../../../src/server/createServer.js"; +import { + expandToolAvailability, + toolAvailabilityInventory, +} from "../../../src/contracts/toolOutputSchemaPrimitives.js"; +import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; +import { startJebMcpFixture } from "../../fixtures/jebMcp.js"; +import { jebMcpTimeoutGuidance } from "../../../src/jeb/JebMcpDeadline.js"; +import { createMcpSchemaValidation } from "../../fixtures/mcpSchemaValidation.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { + ADB_MISSING_REMEDIATION, + ADB_NOT_EXECUTABLE_REMEDIATION, +} from "../../../src/adb/AdbConfiguration.js"; +import { + APKTOOL_MISSING_REMEDIATION, + APKTOOL_NOT_EXECUTABLE_REMEDIATION, +} from "../../../src/apktool/ApktoolConfiguration.js"; + +const schemaValidation = createMcpSchemaValidation(); +afterAll(() => schemaValidation.clear()); + +/** Read final capability discovery through the production MCP composition. */ +const discoverTools = async (environment: NodeJS.ProcessEnv) => { + const { client, close } = await createApplicationMcpHarness({ + serverOptions: { environment }, + clientIdentity: { name: "capability-remediation", version: "1" }, + schemaValidator: schemaValidation, + }); + try { + const result = await client.callTool({ + name: "binary_session", + arguments: { detail: "complete" }, + }); + expect(result.isError, JSON.stringify(result)).not.toBe(true); + const status = toolContract("binary_session").outputSchema.parse( + result.structuredContent, + ).result; + return new Map( + expandToolAvailability( + toolAvailabilityInventory.parse(status.tool_availability), + ).map((tool) => [tool.name, tool]), + ); + } finally { + await close(); + } +}; + +it("names the JEB discovery deadline setting instead of repeating the expired deadline", async () => { + let release!: () => void; + const gate = new Promise((resolve) => { + release = resolve; + }); + const fixture = await startJebMcpFixture( + () => null, + async () => { + await gate; + return { tools: [] }; + }, + ); + try { + const tools = await discoverTools({ + REA_JEB_MCP_URL: fixture.endpoint, + REA_JEB_MCP_TIMEOUT_MS: "100", + }); + const entry = tools.get("inspect_jeb_client"); + expect(entry).toMatchObject({ + available: false, + reason: "provider_unavailable", + }); + // The deadline owner authors the repair; discovery must not restate the failure. + expect(entry?.remediation).toBe( + jebMcpTimeoutGuidance("discovery", 100).remediationAction, + ); + expect(entry?.remediation).toContain("REA_JEB_MCP_TIMEOUT_MS"); + expect(entry?.remediation).not.toContain("deadlineExpired"); + } finally { + release(); + await fixture.close(); + } +}); + +it.skipIf(process.platform === "win32").each([ + { + selection: "non-executable", + adb: ADB_NOT_EXECUTABLE_REMEDIATION, + apktool: APKTOOL_NOT_EXECUTABLE_REMEDIATION, + }, + { + selection: "missing", + adb: ADB_MISSING_REMEDIATION, + apktool: APKTOOL_MISSING_REMEDIATION, + }, +] as const)( + "keeps the owner's repair for a $selection ADB and Apktool selection", + async ({ selection, adb, apktool }) => { + const root = await createTestTempDirectory("rea-capability-remediation-"); + const selected = { adb: join(root, "adb"), apktool: join(root, "apktool") }; + if (selection === "non-executable") + for (const path of Object.values(selected)) + await writeFile(path, "#!/bin/sh\nexit 0\n", { mode: 0o600 }); + const tools = await discoverTools({ + ...(process.env.PATH === undefined ? {} : { PATH: process.env.PATH }), + REA_ADB_PATH: selected.adb, + REA_APKTOOL_COMMAND: selected.apktool, + }); + const entries = [ + { setting: "REA_ADB_PATH", repair: adb, names: ["inspect_adb_client"] }, + { + setting: "REA_APKTOOL_COMMAND", + repair: apktool, + names: ["inspect_apktool_client", "decode_android_resources"], + }, + ]; + for (const { setting, repair, names } of entries) + for (const name of names) { + const entry = tools.get(name); + expect(entry).toMatchObject({ available: false, remediation: repair }); + // Independent of wording: the repair names the setting and matches + // the observed failure, not the other outcome's repair. + expect(entry?.remediation).toContain(setting); + expect(/execute permission/u.test(entry?.remediation ?? "")).toBe( + selection === "non-executable", + ); + } + }, +); + +it("returns an unready IDA registration's repair from an explicit open_binary selection", async () => { + const root = await createTestTempDirectory("rea-selection-remediation-"); + const target = join(root, "target.elf"); + await writeFile(target, elf(2, 1, 62)); + const config = parseConfig({ + REA_IDA_MCP_CONFIG: join(root, "missing.json"), + }); + if (!config.ok) throw config.error; + const ida = new IdaProvider(config.value, {}, () => { + throw new Error("Selection must not connect to IDA"); + }); + // The readiness owner is the oracle for the repair the caller receives. + const owner = await ida.inspectAvailability(); + if (owner.status !== "unavailable") + throw new Error("A missing registration must be unavailable"); + const session = composeBinarySession(new AnalysisProviderRegistry([ida])); + const server = createServer( + { kind: "session", session }, + { logger: silentLogger }, + ); + const client = new Client({ name: "selection-remediation", version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + onTestFinished(async () => { + await client.close(); + await server.close(); + const closed = await session.close(); + if (!closed.ok) throw closed.error; + const released = await ida.closeOwnedResources(); + if (!released.ok) throw released.error; + }); + await server.connect(serverTransport); + await client.connect(clientTransport); + + const { error } = parseMcpToolError( + (await client.callTool({ + name: "open_binary", + arguments: { path: target, provider_id: "ida" }, + })) as CallToolResult, + ); + expect(error.remediation.action).toBe(owner.remediation); + expect(error.remediation.action).toContain("REA_IDA_MCP_CONFIG"); + expect(error.details).toMatchObject({ + selection_reason: "provider_unavailable", + requested_provider_id: "ida", + rejections: [ + { + provider_id: "ida", + origin: "availability", + reason: owner.reason, + remediation: owner.remediation, + }, + ], + }); +}); diff --git a/tests/boundary/mcp/cutterCancellationMcp.test.ts b/tests/boundary/mcp/cutterCancellationMcp.test.ts index 4a2a5ca589..f8b520e511 100644 --- a/tests/boundary/mcp/cutterCancellationMcp.test.ts +++ b/tests/boundary/mcp/cutterCancellationMcp.test.ts @@ -114,7 +114,7 @@ it.skipIf(process.platform === "win32")( }, ); -it("propagates MCP cancellation into an active Cutter command and reports uncertain completion", async () => { +it("propagates MCP cancellation into an active Cutter command", async () => { let markDispatched: (() => void) | undefined; let observedAbort = false; const dispatched = new Promise((resolve) => { diff --git a/tests/boundary/mcp/deferredRuntime.test.ts b/tests/boundary/mcp/deferredRuntime.test.ts index 61cf7122bd..f52e3f166b 100644 --- a/tests/boundary/mcp/deferredRuntime.test.ts +++ b/tests/boundary/mcp/deferredRuntime.test.ts @@ -11,6 +11,7 @@ import { } from "node:fs/promises"; import { resolve, join, dirname } from "node:path"; import { expect, it, onTestFinished } from "vitest"; +import { artifactMcpCall } from "../../../scripts/lib/artifact-e2e.mjs"; import { closeVerifierResources } from "../../../scripts/lib/verifier-cleanup.mjs"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { GENERATED_MCP_TOOL_CATALOG } from "../../fixtures/mcpToolCatalog.js"; @@ -112,15 +113,20 @@ registerHooks({ const { client, stderr } = await connect("legacy", environment, [ `--import=${hook}`, ]); - await client.listTools(); + const discovery = await client.listTools(); await client.listPrompts(); expect(await loadedModules()).toEqual( new Set(eager ? ["entrypoint", ...modules] : ["entrypoint"]), ); - const result = await client.callTool({ - name: "list_unknowns", - arguments: {}, - }); + // Keep output-off coverage too, using this connection's named definition. + const toolDefinition = discovery.tools.find( + ({ name }) => name === "list_unknowns", + ); + if (toolDefinition === undefined) throw new Error("Missing list_unknowns"); + const result = await client.callTool( + { name: "list_unknowns", arguments: {} }, + { toolDefinition }, + ); expect(result.isError).not.toBe(true); expect(await loadedModules()).toEqual(new Set(["entrypoint", ...modules])); expect(stderr()).toBe(""); @@ -195,9 +201,14 @@ it.each([ prompt, ), ); + // Discovery is checked in full above. Validate each called tool's actual + // schema without compiling unrelated outputs in every fresh SDK client. const results = await Promise.all([ - client.callTool({ name: "list_unknowns", arguments: {} }), - client.callTool({ name: "list_unknowns", arguments: { limit: 1 } }), + artifactMcpCall(client, { name: "list_unknowns", arguments: {} }), + artifactMcpCall(client, { + name: "list_unknowns", + arguments: { limit: 1 }, + }), ]); expect(results.every((result) => result.isError !== true)).toBe(true); expect(results.map((result) => result.structuredContent)).toEqual([ @@ -205,7 +216,7 @@ it.each([ { result: { items: [], total: 0, offset: 0, next_offset: null } }, ]); expect(await client.listTools()).toEqual(before); - const invalid = await client.callTool({ + const invalid = await artifactMcpCall(client, { name: "list_unknowns", arguments: { limit: 1.5 }, }); @@ -238,7 +249,7 @@ it("composes a scoped registration instead of serving the recorded catalog", asy expect(advertised).not.toContain("inspect_macho"); // The recorded catalog cannot verify a scoped identity, so the scope must // serve real calls instead of failing every one after materialization. - const status = await client.callTool({ + const status = await artifactMcpCall(client, { name: "binary_session", arguments: {}, }); @@ -263,7 +274,7 @@ globalThis.Date = class extends NativeDate { const { client } = await connect("legacy", {}, [`--import=${hook}`]); await client.listTools(); await writeFile(clock, "2021-01-01T00:00:00.000Z"); - const first = await client.callTool({ + const first = await artifactMcpCall(client, { name: "binary_session", arguments: {}, }); @@ -271,7 +282,7 @@ globalThis.Date = class extends NativeDate { expect(first.structuredContent).toMatchObject({ result: { server_identity: { server: { started_at: startedAt } } }, }); - const again = await client.callTool({ + const again = await artifactMcpCall(client, { name: "binary_session", arguments: {}, }); @@ -596,10 +607,14 @@ it.each([{}, { REA_MCP_OUTPUT_SCHEMA_PROFILE: "off" }])( ) : catalog.tools; expect(before.tools).toEqual(expected); - const result = await client.callTool({ - name: "list_unknowns", - arguments: {}, - }); + const toolDefinition = before.tools.find( + ({ name }) => name === "list_unknowns", + ); + if (toolDefinition === undefined) throw new Error("Missing list_unknowns"); + const result = await client.callTool( + { name: "list_unknowns", arguments: {} }, + { toolDefinition }, + ); expect(result.isError).not.toBe(true); expect(await client.listTools()).toEqual(before); }, diff --git a/tests/boundary/mcp/enhancedTools.callPathsAndBatches.test.ts b/tests/boundary/mcp/enhancedTools.callPathsAndBatches.test.ts index 6518fbf866..1fb25aa167 100644 --- a/tests/boundary/mcp/enhancedTools.callPathsAndBatches.test.ts +++ b/tests/boundary/mcp/enhancedTools.callPathsAndBatches.test.ts @@ -8,6 +8,7 @@ import { AnalysisOutputError, } from "../../../src/domain/analysisErrorCore.js"; import { parseEvidence } from "../../../src/domain/evidence.js"; +import { batchDecompileResultSchema } from "../../../src/domain/native/batchDecompile.js"; import { ENHANCED_TOOL_CONTRACTS } from "../../../src/contracts/enhancedToolContracts.js"; import { NULL_PSEUDOCODE_GUIDANCE } from "../../../src/application/EnhancedTools.js"; import type { RelationshipCoverage } from "../../../src/domain/relationshipCoverage.js"; @@ -722,7 +723,9 @@ describe("enhanced MCP batch answers without pseudocode", () => { succeeded: 1, failed: 5, }); - const { items } = evidence.normalized_result; + const { items } = batchDecompileResultSchema.parse( + evidence.normalized_result, + ); // Successful text, a caveat-free null and a typed provider failure carry // no item caveats; the batch Evidence aggregates every observation's. for (const index of [0, 2, 5]) diff --git a/tests/boundary/mcp/enhancedTools.swiftTypes.test.ts b/tests/boundary/mcp/enhancedTools.swiftTypes.test.ts index 82fc8d4945..9d6c0f2a95 100644 --- a/tests/boundary/mcp/enhancedTools.swiftTypes.test.ts +++ b/tests/boundary/mcp/enhancedTools.swiftTypes.test.ts @@ -147,15 +147,24 @@ describe("Swift classification through MCP", () => { it("returns cancellation during the symbol join", async () => { const controller = new AbortController(); + const reason = { stage: "symbol-join" }; const tools = new EnhancedTools({ execute: async (operation) => { - if (operation === "list_names") controller.abort(); + if (operation === "list_names") controller.abort(reason); return ok([{ address: "0x1", value: "$s4main4PairVMa" }]); }, }); - expect( - await tools.execute("analyze_swift_types", {}, controller.signal), - ).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" } }); + const result = await tools.execute( + "analyze_swift_types", + {}, + controller.signal, + ); + expect(result).toMatchObject({ + ok: false, + error: { _tag: "AnalysisCancelledError", operation: "list_names" }, + }); + if (result.ok) throw new Error("Expected cancellation"); + expect(result.error.cause).toBe(reason); }); }); diff --git a/tests/boundary/mcp/evidenceRecordSelectionMcp.test.ts b/tests/boundary/mcp/evidenceRecordSelectionMcp.test.ts new file mode 100644 index 0000000000..86d267d16d --- /dev/null +++ b/tests/boundary/mcp/evidenceRecordSelectionMcp.test.ts @@ -0,0 +1,286 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { afterAll, expect, it } from "vitest"; +import { z } from "zod"; +import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js"; +import { MCP_TRANSPORT_EVIDENCE_RECORDS } from "../../../src/domain/analysisErrorPresentation.js"; +import { + EVIDENCE_RECORD_SELECTION_LIMIT, + EVIDENCE_RECORD_SELECTION_LIMITATIONS, + evidenceRecordSelectionSchema, +} from "../../../src/domain/evidenceRecordSummary.js"; +import { createToolResultDelivery } from "../../../src/server/toolResult.js"; +import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; +import { createTestBinarySession } from "../../fixtures/binarySession.js"; +import { createMcpSchemaValidation } from "../../fixtures/mcpSchemaValidation.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const schemaValidation = createMcpSchemaValidation(); +afterAll(() => schemaValidation.clear()); + +const GHIDRA = { id: "ghidra", name: "Ghidra", version: "12.1.4" }; +const target = (letter: string) => ({ + path: `/fixtures/${letter}.macho`, + format: "mach-o" as const, + sha256: letter.repeat(64), +}); + +// Synthetic records with the shapes the issue observed; no provider is run. +const annotation = (letter: string, name: string) => + createEvidence(target(letter), GHIDRA, { + operation: "annotate_native_function", + parameters: { procedure: "0x100000430", name }, + result: { procedure: { address: "0x100000430", name }, comment: "audit" }, + }); +const annotatedA = annotation("a", "audit_entry"); +const annotatedB = annotation("b", "other_entry"); +const pseudocode = createEvidence(target("a"), GHIDRA, { + operation: "procedure_pseudo_code", + parameters: { procedure: "0x100000430" }, + result: "int audit_entry(void) { return 0; }", +}); +const unretainedSource = createEvidence(target("a"), GHIDRA, { + operation: "xrefs", + parameters: { address: "0x100000430" }, + result: [], +}); +const trace = createEvidence(target("a"), GHIDRA, { + operation: "trace_feature", + parameters: { feature: "audit" }, + result: { references: 1 }, + evidenceLinks: [pseudocode.evidence_id, unretainedSource.evidence_id], +}); +const LEDGER = [annotatedA, pseudocode, annotatedB, trace]; +type Record_ = (typeof LEDGER)[number]; +/** Canonical bundle order: exact Evidence ID code-point order. */ +const canonical = (records: readonly Record_[]) => + [...records].sort((left, right) => + left.evidence_id < right.evidence_id ? -1 : 1, + ); + +const connect = async ( + options: { + readonly profile?: "full" | "compact"; + readonly maximumResponseBytes?: number; + readonly records?: readonly ReturnType[]; + } = {}, +) => { + const session = createTestBinarySession(() => { + throw new Error("Record selection must not open a provider"); + }); + const { client } = await createApplicationMcpHarness({ + session, + clientIdentity: { name: "evidence-record-selection", version: "1" }, + schemaValidator: schemaValidation, + serverOptions: { + environment: { + ...process.env, + REA_MCP_OUTPUT_SCHEMA_PROFILE: options.profile ?? "full", + }, + ...(options.maximumResponseBytes === undefined + ? {} + : { delivery: createToolResultDelivery(options.maximumResponseBytes) }), + }, + }); + for (const record of options.records ?? LEDGER) + expect(session.recordEvidence(record).ok).toBe(true); + const reply = (args: Record) => + client.callTool({ name: "get_evidence_bundle", arguments: args }); + const select = async (args: Record) => { + const result = await reply({ detail: "records", ...args }); + expect(result.isError, JSON.stringify(result)).not.toBe(true); + return { + result, + selection: evidenceRecordSelectionSchema.parse( + z.object({ result: z.unknown() }).parse(result.structuredContent) + .result, + ), + }; + }; + return { client, session, reply, select }; +}; + +it.each(["full", "compact"] as const)( + "returns selected complete records with retained identities (%s)", + async (profile) => { + const { client, select } = await connect({ profile }); + const exact = await select({ + filters: { evidence_id: annotatedA.evidence_id }, + }); + expect(exact.selection).toMatchObject({ + kind: "evidence-record-selection", + filters: { evidence_id: annotatedA.evidence_id }, + total_retained_records: LEDGER.length, + matching_records: 1, + offset: 0, + limit: EVIDENCE_RECORD_SELECTION_LIMIT, + next_offset: null, + missing_evidence_links: [], + }); + // The readback is the retained record itself, authenticated under its original ID. + expect(exact.selection.records).toEqual([annotatedA]); + expect(parseEvidence(exact.selection.records[0]).evidence_id).toBe( + annotatedA.evidence_id, + ); + // Historical and non-bundle labels come from their owning constant. + expect(exact.selection.limitations).toEqual([ + ...EVIDENCE_RECORD_SELECTION_LIMITATIONS, + ]); + + const { tools } = await client.listTools(); + const tool = tools.find(({ name }) => name === "get_evidence_bundle")!; + const ajv = schemaValidation.assertions; + expect(ajv.validateSchema(tool.inputSchema)).toBe(true); + expect( + ajv.validate(tool.inputSchema, { + detail: "records", + filters: { operation: "trace_feature" }, + offset: 0, + limit: 1, + }), + ).toBe(true); + const outputSchema: Record = tool.outputSchema!; + expect(ajv.validate(outputSchema, exact.result.structuredContent)).toBe( + true, + ); + + // Pages follow canonical bundle order across distinct targets. + const annotations = canonical([annotatedA, annotatedB]); + const first = await select({ + filters: { operation: "annotate_native_function" }, + limit: 1, + }); + expect(first.selection).toMatchObject({ + matching_records: 2, + next_offset: 1, + records: [annotations[0]], + }); + const second = await select({ + filters: { operation: "annotate_native_function" }, + offset: 1, + limit: 1, + }); + expect(second.selection).toMatchObject({ + next_offset: null, + records: [annotations[1]], + }); + expect( + (await select({ filters: { target_sha256: "a".repeat(64) } })).selection + .records, + ).toEqual(canonical([annotatedA, pseudocode, trace])); + expect( + (await select({ filters: { operation: "absent" } })).selection, + ).toMatchObject({ matching_records: 0, records: [], next_offset: null }); + expect((await select({ offset: LEDGER.length })).selection).toMatchObject({ + matching_records: LEDGER.length, + records: [], + }); + }, +); + +it("reports only the selected records' unretained direct links", async () => { + const { session, select } = await connect(); + const selected = await select({ + filters: { evidence_id: trace.evidence_id }, + }); + expect(selected.selection.records).toEqual([trace]); + expect(selected.selection.missing_evidence_links).toEqual([ + unretainedSource.evidence_id, + ]); + // Unselected records never contribute closure facts. + expect( + (await select({ filters: { operation: "annotate_native_function" } })) + .selection.missing_evidence_links, + ).toEqual([]); + expect(session.recordEvidence(unretainedSource).ok).toBe(true); + expect( + (await select({ filters: { evidence_id: trace.evidence_id } })).selection, + ).toMatchObject({ records: [trace], missing_evidence_links: [] }); +}); + +it("selects the same records after importing an exported bundle", async () => { + const exporter = await connect(); + const root = await createTestTempDirectory("rea-record-selection-"); + const path = join(root, "evidence.json"); + await writeFile( + path, + JSON.stringify(exporter.session.exportEvidenceBundle()), + ); + const importer = await connect({ records: [] }); + const imported = await importer.client.callTool({ + name: "import_evidence_bundle", + arguments: { path }, + }); + expect(imported.isError, JSON.stringify(imported)).not.toBe(true); + expect( + ( + await importer.select({ + filters: { evidence_id: annotatedB.evidence_id }, + }) + ).selection.records, + ).toEqual([annotatedB]); +}); + +it("keeps canonical bundle and summary contracts distinct from record pages", async () => { + const { reply } = await connect(); + for (const [args, path] of [ + [ + { detail: "complete", filters: { operation: "trace_feature" } }, + "filters", + ], + [{ detail: "summary", limit: 1 }, "limit"], + [{ detail: "complete", offset: 0 }, "offset"], + [{ limit: 1 }, "limit"], + ] as const) + expect(parseMcpToolError(await reply(args))).toMatchObject({ + error: { + code: "invalid_request", + details: { issues: [expect.objectContaining({ path: [path] })] }, + }, + }); + expect( + parseMcpToolError(await reply({ detail: "records", limit: 0 })), + ).toMatchObject({ error: { code: "invalid_request" } }); + const summary = await reply({ + detail: "summary", + filters: { evidence_id: trace.evidence_id }, + }); + expect(summary.structuredContent).toMatchObject({ + result: { kind: "evidence-bundle-summary", matching_records: 1 }, + }); +}); + +it("guides an oversized record page toward a smaller selection", async () => { + const large = createEvidence(target("c"), GHIDRA, { + operation: "procedure_pseudo_code", + parameters: { procedure: "0x1000" }, + result: "x".repeat(32 * 1024), + }); + const { reply, select } = await connect({ + maximumResponseBytes: 16 * 1024, + records: [...LEDGER, large], + }); + const failed = parseMcpToolError(await reply({ detail: "records" })).error; + expect(failed.code).toBe("resource_constraint"); + // Guidance comes from its owning value; the limit facts are checked separately. + expect(failed.remediation?.action).toContain(MCP_TRANSPORT_EVIDENCE_RECORDS); + const limits = z + .object({ + details: z.object({ + reported_limits: z.object({ + result_budget_bytes: z.number(), + response_bytes_at_least: z.number(), + }), + }), + }) + .parse(failed).details.reported_limits; + expect(limits.response_bytes_at_least).toBeGreaterThan( + limits.result_budget_bytes, + ); + // An exact selection that excludes the large record still fits. + expect( + (await select({ filters: { evidence_id: annotatedA.evidence_id } })) + .selection.records, + ).toEqual([annotatedA]); +}); diff --git a/tests/boundary/mcp/featureTraceViewsMcp.test.ts b/tests/boundary/mcp/featureTraceViewsMcp.test.ts new file mode 100644 index 0000000000..7d3656115c --- /dev/null +++ b/tests/boundary/mcp/featureTraceViewsMcp.test.ts @@ -0,0 +1,540 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { afterAll, expect, it, onTestFinished } from "vitest"; +import { z } from "zod"; +import { JAVASCRIPT_FEATURE_TRACE_EXAMPLE } from "../../../src/contracts/javascript/javascriptApplicationWorkflowExamples.js"; +import { toolContract } from "../../../src/contracts/toolContracts.js"; +import { analysisViewResultSchema } from "../../../src/domain/analysisView/analysisView.js"; +import { parseEvidence, type Evidence } from "../../../src/domain/evidence.js"; +import { applicationFeatureTraceResultSchema } from "../../../src/domain/javascript/javascriptFeatureTraceSchemas.js"; +import { jsonObjectSchema } from "../../../src/domain/jsonValue.js"; +import { createServer } from "../../../src/server/createServer.js"; +import { deliverApplicationEvidence } from "../../../src/server/registerApplicationTools/helpers.js"; +import { traceApplicationFeatureEvidence } from "../../../src/application/javascript/JavaScriptApplicationWorkflowService.js"; +import { ArtifactResourceScope } from "../../../src/artifacts/ArtifactResourceScope.js"; +import { silentLogger } from "../../../src/logger.js"; +import { createToolResultDelivery } from "../../../src/server/toolResult.js"; +import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; +import { createTestBinarySession } from "../../fixtures/binarySession.js"; +import { createMcpSchemaValidation } from "../../fixtures/mcpSchemaValidation.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { analyzeJavaScriptApplication } from "../../support/javascriptApplicationScope.js"; + +const schemaValidation = createMcpSchemaValidation(); +afterAll(() => schemaValidation.clear()); + +type Trace = z.output; +type Row = Record; + +const CHANNELS = ["save-0", "save-1", "save-2"]; + +/** A small Electron application whose IPC handlers reach storage and network endpoints. */ +const writeApplication = async (): Promise => { + const root = await createTestTempDirectory("rea-feature-trace-view-"); + await Promise.all([ + writeFile( + join(root, "package.json"), + JSON.stringify({ name: "trace-view", version: "1.0.0", main: "main.js" }), + ), + writeFile( + join(root, "main.js"), + [ + 'const {app,BrowserWindow,ipcMain}=require("electron");', + 'const fs=require("fs");', + 'const audio=require("./audio.node");', + 'const crypto=require("./crypto.node");', + ...CHANNELS.map( + (channel) => + `ipcMain.handle("${channel}",async(event,value)=>{fs.writeFileSync("/tmp/${channel}",value);return fetch("https://api.example.test/${channel}");});`, + ), + 'app.whenReady().then(()=>new BrowserWindow({webPreferences:{preload:"./preload.js"}}));', + ].join("\n"), + ), + writeFile( + join(root, "preload.js"), + `const {contextBridge,ipcRenderer}=require("electron"); +contextBridge.exposeInMainWorld("api",{${CHANNELS.map( + (channel, index) => + `save${String(index)}:(value)=>ipcRenderer.invoke("${channel}",value)`, + ).join(",")}});`, + ), + writeFile(join(root, "index.html"), "\n"), + // Distinct opaque addon bytes give two content-identified native handoffs. + writeFile(join(root, "audio.node"), Buffer.from("rea-audio-addon")), + writeFile(join(root, "crypto.node"), Buffer.from("rea-crypto-addon")), + ]); + return root; +}; + +const connect = async ( + profile: "full" | "compact", + maximumResponseBytes?: number, +) => { + const application = await analyzeJavaScriptApplication({ + input_path: await writeApplication(), + }); + if (!application.ok) throw application.error; + const session = createTestBinarySession(() => { + throw new Error("Feature trace views must not open a native provider"); + }); + const { client } = await createApplicationMcpHarness({ + session, + clientIdentity: { name: "feature-trace-view", version: "1" }, + schemaValidator: schemaValidation, + serverOptions: { + environment: { ...process.env, REA_MCP_INPUT_SCHEMA_PROFILE: profile }, + ...(maximumResponseBytes === undefined + ? {} + : { delivery: createToolResultDelivery(maximumResponseBytes) }), + }, + }); + expect(session.recordEvidence(application.value).ok).toBe(true); + const inputs = { + application: { + kind: "retained-evidence", + evidence_id: application.value.evidence_id, + }, + seed: { kind: "channel", value: "save-0", match: "exact" }, + direction: "both", + }; + const reply = async (name: string, args: Record) => + client.callTool({ name, arguments: args }); + const call = async (name: string, args: Record) => { + const result = await reply(name, args); + expect(result.isError, JSON.stringify(result)).not.toBe(true); + return { evidence: parseEvidence(result.structuredContent), result }; + }; + return { client, session, inputs, reply, call }; +}; + +const traced = (evidence: Evidence): Trace => + applicationFeatureTraceResultSchema.parse(evidence.normalized_result); + +const bytes = (value: unknown): number => + Buffer.byteLength(JSON.stringify(value)); + +type Harness = Awaited>; + +it.each(["full", "compact"] as const)( + "returns selected feature-trace facts and retains the complete trace (%s)", + async (profile) => { + const harness = await connect(profile); + const { client, session, inputs, call } = harness; + // Select first: the view call alone must retain the complete parent. + const summary = await call("trace_application_feature", { + ...inputs, + view: { kind: "summary" }, + }); + const parentId = z + .object({ parent_evidence_id: z.string() }) + .parse(summary.evidence.normalized_result).parent_evidence_id; + const retained = session.evidenceById(parentId); + const complete = await call("trace_application_feature", inputs); + const parent = complete.evidence; + expect(parent.evidence_id).toBe(parentId); + expect(retained).toEqual(parent); + const full = traced(parent); + const graph = full.graph!; + expect(full.paths.length).toBeGreaterThan(1); + expect(graph.edges.length).toBeGreaterThan(graph.nodes.length); + expect(summary.evidence.confidence).toBe(parent.confidence); + expect(summary.evidence.normalized_result).toMatchObject({ + kind: "summary", + parent_operation: "trace_application_feature", + artifact: null, + summary: { + trace_id: full.trace_id, + source_evidence_id: full.source_evidence_id, + seed: full.seed, + summary: full.summary, + trace_coverage: full.coverage, + graph_coverage: graph.coverage, + evidence_links: full.evidence_links, + }, + coverage: { status: "complete-within-view", exhausted: true }, + }); + expect(bytes(summary.result)).toBeLessThan(bytes(complete.result) / 10); + + const { tools } = await client.listTools(); + const ajv = schemaValidation.assertions; + const traceTool = tools.find( + ({ name }) => name === "trace_application_feature", + )!; + expect(ajv.validateSchema(traceTool.inputSchema)).toBe(true); + const outputSchema: Record = traceTool.outputSchema!; + expect(ajv.validateSchema(outputSchema)).toBe(true); + const request = { + kind: "feature-trace", + collection: "paths", + where: { end_kinds: ["endpoint"] }, + limit: 2, + }; + expect( + ajv.validate(traceTool.inputSchema, { ...inputs, view: request }), + ).toBe(true); + const selected = await call("trace_application_feature", { + ...inputs, + view: request, + }); + for (const evidence of [summary.evidence, selected.evidence]) + expect(ajv.validate(outputSchema, evidence)).toBe(true); + const endpointPaths = full.paths.filter( + ({ end_kind }) => end_kind === "endpoint", + ); + expect(endpointPaths.length).toBeGreaterThan(2); + expect(selected.evidence.normalized_result).toMatchObject({ + kind: "feature-trace", + parent_evidence_id: parent.evidence_id, + trace_id: full.trace_id, + trace_coverage: full.coverage, + items: endpointPaths.slice(0, 2), + coverage: { total: endpointPaths.length, next_offset: 2 }, + }); + // The advertised page contract binds each collection to its own rows. + const mixed = { + ...jsonObjectSchema.parse(selected.evidence.normalized_result), + items: graph.edges.slice(0, 1), + }; + expect(analysisViewResultSchema.safeParse(mixed).success).toBe(false); + expect( + ajv.validate(outputSchema, { + ...jsonObjectSchema.parse(selected.result.structuredContent), + normalized_result: mixed, + }), + ).toBe(false); + await assertRetainedTracePages(harness, parent.evidence_id, full); + }, +); + +/** Page the retained parent through inspect_analysis_view without re-running the trace. */ +const assertRetainedTracePages = async ( + { call, reply }: Harness, + parentId: string, + full: Trace, +) => { + const graph = full.graph!; + const source = { kind: "retained-evidence", evidence_id: parentId }; + const view = async (selection: Record) => { + const page = analysisViewResultSchema.parse( + (await call("inspect_analysis_view", { source, view: selection })) + .evidence.normalized_result, + ); + if (page.kind !== "feature-trace") throw new Error("Expected trace page"); + return page; + }; + // Successive small pages reproduce each complete collection in retained order. + const collect = async (selection: Record) => { + const rows: Row[] = []; + let offset: number | null = 0; + while (offset !== null) { + const page = await view({ + kind: "feature-trace", + ...selection, + offset, + limit: 7, + }); + rows.push(...(page.items as Row[])); + offset = page.coverage.next_offset; + } + return rows; + }; + const ids = (rows: readonly Row[] | readonly { node_id: string }[]) => + rows.map(({ node_id: id }) => id); + const seeds = new Set(ids(full.seed_matches)); + expect(await collect({ collection: "paths" })).toEqual(full.paths); + expect(await collect({ collection: "edges" })).toEqual(graph.edges); + expect(await collect({ collection: "seed_matches" })).toEqual( + full.seed_matches, + ); + expect(full.native_handoffs).toHaveLength(2); + expect(await collect({ collection: "native_handoffs" })).toEqual( + full.native_handoffs, + ); + for (const [offset, handoff] of full.native_handoffs.entries()) + expect( + await view({ + kind: "feature-trace", + collection: "native_handoffs", + offset, + limit: 1, + }), + ).toMatchObject({ + items: [handoff], + coverage: { + total: 2, + next_offset: offset === 0 ? 1 : null, + }, + }); + const detailed = await collect({ + collection: "nodes", + include_observations: true, + }); + expect(ids(detailed)).toEqual(ids(graph.nodes)); + expect(detailed.map(({ observations }) => observations)).toEqual( + graph.nodes.map(({ observations }) => observations), + ); + expect(ids(detailed.filter(({ seed }) => seed === true))).toEqual( + ids(graph.nodes.filter(({ node_id: id }) => seeds.has(id))), + ); + const compact = await view({ kind: "feature-trace", collection: "nodes" }); + expect(compact.items[0]).not.toHaveProperty("observations"); + expect(compact.items[0]).toMatchObject({ + node_id: graph.nodes[0]!.node_id, + observation_count: graph.nodes[0]!.observations.length, + }); + + const seed = full.seed_matches[0]!.node_id; + expect( + await collect({ collection: "edges", where: { node_ids: [seed] } }), + ).toEqual( + graph.edges.filter( + (edge) => edge.source_node_id === seed || edge.target_node_id === seed, + ), + ); + expect( + await collect({ collection: "paths", where: { node_ids: [seed] } }), + ).toEqual(full.paths.filter(({ node_ids: path }) => path.includes(seed))); + expect( + ids( + await collect({ collection: "nodes", where: { kinds: ["ipc-channel"] } }), + ), + ).toEqual(ids(graph.nodes.filter(({ kind }) => kind === "ipc-channel"))); + expect( + await collect({ collection: "edges", where: { relations: ["handles"] } }), + ).toEqual(graph.edges.filter(({ relation }) => relation === "handles")); + for (const selection of [ + { collection: "paths", where: { node_ids: [] } }, + { collection: "paths", where: { end_kinds: [] } }, + { collection: "nodes", where: { kinds: [] } }, + { collection: "edges", where: { relations: [] } }, + { collection: "paths", offset: full.paths.length }, + ]) + expect(await view({ kind: "feature-trace", ...selection })).toMatchObject({ + items: [], + coverage: { status: "empty", exhausted: true }, + trace_coverage: full.coverage, + }); + + // Views owned by other parents, and unknown filter values, are rejected rather than reinterpreted. + for (const selection of [ + { kind: "page", collection: "modules", offset: 0, limit: 1 }, + { + kind: "feature-trace", + collection: "nodes", + where: { kinds: ["not-a-kind"] }, + }, + { kind: "feature-trace", collection: "graph" }, + ]) + expect( + parseMcpToolError( + await reply("inspect_analysis_view", { source, view: selection }), + ), + ).toMatchObject({ error: { code: "invalid_request" } }); +}; + +it("keeps a no-match trace's discovery and empty collections explicit", async () => { + const { inputs, call } = await connect("full"); + const missing = { + ...inputs, + seed: { kind: "string", value: "absent-literal", match: "exact" }, + }; + const summary = await call("trace_application_feature", { + ...missing, + view: { kind: "summary" }, + }); + expect(summary.evidence.normalized_result).toMatchObject({ + summary: { + summary: { matched_seeds: 0, traced_nodes: 0 }, + trace_coverage: { status: "no-match" }, + graph_coverage: null, + semantic_discovery: { total: 0, next_offset: null }, + }, + }); + const nodes = await call("trace_application_feature", { + ...missing, + view: { kind: "feature-trace", collection: "nodes" }, + }); + expect(nodes.evidence.normalized_result).toMatchObject({ + items: [], + graph_coverage: null, + coverage: { status: "empty", total: 0, exhausted: true }, + }); +}); + +it("refuses a selected view before tracing when the server retains no Evidence", async () => { + const session = createTestBinarySession(() => { + throw new Error("No deep provider"); + }); + const server = createServer({ kind: "fixed", analysis: session }); + const client = new Client({ name: "feature-trace-fixed", version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + onTestFinished(async () => { + await client.close(); + await server.close(); + }); + await server.connect(serverTransport); + await client.connect(clientTransport); + const refused = await client.callTool({ + name: "trace_application_feature", + arguments: { + ...JAVASCRIPT_FEATURE_TRACE_EXAMPLE, + view: { kind: "summary" }, + }, + }); + expect(parseMcpToolError(refused)).toMatchObject({ + error: { + code: "capability_unavailable", + message: expect.stringContaining("does not retain Evidence"), + }, + }); + const complete = await client.callTool({ + name: "trace_application_feature", + arguments: JAVASCRIPT_FEATURE_TRACE_EXAMPLE, + }); + expect(complete.isError, JSON.stringify(complete)).not.toBe(true); + expect( + traced(parseEvidence(complete.structuredContent)).summary, + ).toMatchObject({ matched_seeds: 1 }); +}); + +it("recovers an oversized completed trace through its retained paths page", async () => { + const budget = 64 * 1024; + const { session, inputs, call, reply } = await connect("full", budget); + const failed = await reply("trace_application_feature", inputs); + const error = parseMcpToolError(failed).error; + expect(error.code).toBe("resource_constraint"); + expect(error.remediation?.action).toContain("inspect_analysis_view"); + const limits = z + .object({ details: z.object({ reported_limits: jsonObjectSchema }) }) + .parse(error).details.reported_limits; + const reference = z + .object({ kind: z.literal("retained-evidence"), evidence_id: z.string() }) + .parse(limits.evidence_reference); + const parent = parseEvidence(session.evidenceById(reference.evidence_id)); + const full = traced(parent); + expect(bytes(parent)).toBeGreaterThan(budget); + const continuation = z + .object({ + name: z.literal("inspect_analysis_view"), + arguments: toolContract("inspect_analysis_view").inputSchema, + }) + .parse(limits.continuation); + expect(continuation.arguments).toEqual({ + source: reference, + view: { + kind: "feature-trace", + collection: "paths", + where: {}, + offset: 0, + limit: 64, + }, + }); + const first = await call(continuation.name, continuation.arguments); + expect(first.evidence.normalized_result).toMatchObject({ + kind: "feature-trace", + parent_evidence_id: parent.evidence_id, + trace_id: full.trace_id, + summary: full.summary, + items: full.paths, + coverage: { total: full.paths.length, exhausted: true }, + }); + expect(bytes(first.result)).toBeLessThan(budget); + // A selected initial view also fits where the complete reply cannot. + const summary = await call("trace_application_feature", { + ...inputs, + view: { kind: "summary" }, + }); + expect(summary.evidence.normalized_result).toMatchObject({ + parent_evidence_id: parent.evidence_id, + }); +}); + +it("names the retained trace and a smaller page when a selected page is oversized", async () => { + const budget = 64 * 1024; + const { session, inputs, reply } = await connect("full", budget); + const selection = { + kind: "feature-trace", + collection: "nodes", + include_observations: true, + limit: 64, + }; + const failed = parseMcpToolError( + await reply("trace_application_feature", { ...inputs, view: selection }), + ).error; + expect(failed.code).toBe("resource_constraint"); + const limits = z + .object({ details: z.object({ reported_limits: jsonObjectSchema }) }) + .parse(failed).details.reported_limits; + const reference = z + .object({ kind: z.literal("retained-evidence"), evidence_id: z.string() }) + .parse(limits.evidence_reference); + // The reference names the retained trace, not the page record that did not fit. + const parent = parseEvidence(session.evidenceById(reference.evidence_id)); + expect(parent.operation).toBe("trace_application_feature"); + const nodes = traced(parent).graph!.nodes; + const continuation = z + .object({ + name: z.literal("inspect_analysis_view"), + arguments: toolContract("inspect_analysis_view").inputSchema, + }) + .parse(limits.continuation); + expect(continuation.arguments).toMatchObject({ + source: reference, + view: { ...selection, offset: 0, limit: 32 }, + }); + // The retained parent pages the same selection once the page fits. + const page = await reply(continuation.name, { + ...continuation.arguments, + view: { ...continuation.arguments.view, limit: 2 }, + }); + expect(page.isError, JSON.stringify(page).slice(0, 500)).not.toBe(true); + expect(bytes(page)).toBeLessThan(budget); + const view = analysisViewResultSchema.parse( + parseEvidence(page.structuredContent).normalized_result, + ); + if (view.kind !== "feature-trace") throw new Error("Expected trace page"); + expect((view.items as Row[]).map(({ node_id: id }) => id)).toEqual( + nodes.slice(0, 2).map(({ node_id: id }) => id), + ); +}); + +it("commits no session facts when cancelled after the trace completes", async () => { + const session = createTestBinarySession(() => { + throw new Error("No deep provider"); + }); + const traceExecution = traceApplicationFeatureEvidence( + JAVASCRIPT_FEATURE_TRACE_EXAMPLE, + ); + if (!traceExecution.ok) throw traceExecution.error; + const artifactResources = new ArtifactResourceScope(); + onTestFinished(() => artifactResources.close()); + const before = session.exportEvidenceBundle(); + for (const view of [undefined, { kind: "summary" } as const]) { + const cancelled = new AbortController(); + cancelled.abort(new Error("client cancelled")); + // Cancellation can only arrive between completion and commit through this + // staging step, which the public workflow cannot schedule deterministically. + const delivered = await deliverApplicationEvidence( + { delivery: createToolResultDelivery(undefined) }, + { + artifactResources, + logger: silentLogger, + evidenceById: (id) => session.evidenceById(id), + recordEvidence: (evidence) => session.recordEvidence(evidence), + recordEvidenceWithUnknown: undefined, + }, + toolContract("trace_application_feature"), + traceExecution.value, + view, + cancelled.signal, + ); + // Same contract as compare_source_to_bundle: a cancelled request leaves the session unchanged. + expect(parseMcpToolError(delivered)).toMatchObject({ + error: { code: "cancelled" }, + }); + expect(session.exportEvidenceBundle()).toEqual(before); + } +}); diff --git a/tests/boundary/mcp/flutterMcp.test.ts b/tests/boundary/mcp/flutterMcp.test.ts index d76c022367..dda9ac505c 100644 --- a/tests/boundary/mcp/flutterMcp.test.ts +++ b/tests/boundary/mcp/flutterMcp.test.ts @@ -13,6 +13,7 @@ import { flutterResultSchemas, } from "../../../src/domain/flutter/flutterBuildAnalysis.js"; import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { parseMcpEvidenceBundle } from "../../fixtures/mcpEvidenceBundle.js"; import { aotFixtureImage } from "../../../src/flutter/FlutterBuild.fixture.js"; import { createFlutterApk } from "../../fixtures/flutterApk.js"; import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; @@ -318,6 +319,14 @@ it("advertises an exact valid Flutter schema and records inline evidence through .map((part) => part.text ?? "") .join(""); const inspectedEvidence = parseEvidence(JSON.parse(inspectedText)); + const bundle = await client.callTool({ + name: "get_evidence_bundle", + arguments: {}, + }); + expect(bundle.isError).not.toBe(true); + const records = parseMcpEvidenceBundle(bundle).records; + expect(records).toContainEqual(evidence); + expect(records).toContainEqual(inspectedEvidence); expect(inspectedEvidence.normalized_result).toMatchObject({ abi: "arm64-v8a", libapp: { diff --git a/tests/boundary/mcp/ghidraDecompileDeadlineLifetime.test.ts b/tests/boundary/mcp/ghidraDecompileDeadlineLifetime.test.ts new file mode 100644 index 0000000000..78a23ca334 --- /dev/null +++ b/tests/boundary/mcp/ghidraDecompileDeadlineLifetime.test.ts @@ -0,0 +1,188 @@ +import { access } from "node:fs/promises"; + +import { expect, it } from "vitest"; +import { z } from "zod"; + +import { AnalysisProviderRegistry } from "../../../src/application/binary/AnalysisProviderRegistry.js"; +import { composeBinarySession } from "../../../src/application/binary/BinarySessionComposition.js"; +import { runDirectAnalysis } from "../../../src/application/DirectAnalysis.js"; +import type { DirectAnalysisDependencies } from "../../../src/application/DirectAnalysisDependencies.js"; +import { projectCliAnalysisResult } from "../../../src/cli/projectCliAnalysisResult.js"; +import { parseConfig } from "../../../src/config/parseConfig.js"; +import { analysisCliErrorEnvelopeSchema } from "../../../src/contracts/errorSchemas.js"; +import { parseEvidence } from "../../../src/domain/evidence.js"; +import { GhidraClient } from "../../../src/ghidra/GhidraClient.js"; +import { ghidraProfileInstallationHost } from "../../../src/ghidra/GhidraInstallation.fixture.js"; +import { GhidraProvider } from "../../../src/ghidra/GhidraProvider.js"; +import { + GHIDRA_SESSION_RETAINED, + ghidraDecompileTimeoutRecovery, +} from "../../../src/ghidra/GhidraProviderClient.js"; +import { silentLogger } from "../../../src/logger.js"; +import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { GhidraFixtureLauncher } from "../providers/ghidra/ghidraClient.fixture.js"; + +// Only installation discovery and the Java bridge are simulated. The bridge +// fails one procedure_pseudo_code request with the production deadline code; +// requests cross the real client socket, provider projection and adapters. +const environment = { + GHIDRA_INSTALL_DIR: "/fixture/ghidra", + REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS: "1", +}; +const BRIDGE_MESSAGE = + "Ghidra decompilation of 0x1000 exceeded the 1-second configured decompilation limit (ghidra.decompile_timeout_seconds; launch seed REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS)"; + +const ghidraSession = (launcher: GhidraFixtureLauncher) => { + const config = parseConfig(environment); + if (!config.ok) throw config.error; + return composeBinarySession( + new AnalysisProviderRegistry([ + new GhidraProvider( + config.value, + silentLogger, + {}, + ghidraProfileInstallationHost, + (options) => + new GhidraClient({ + ...options, + platform: process.platform, + launcher, + }), + ), + ]), + ); +}; + +const expectReleased = async (launcher: GhidraFixtureLauncher) => { + expect(launcher.processes.length).toBeGreaterThan(0); + for (const root of launcher.runtimeRoots) + await expect(access(root)).rejects.toMatchObject({ code: "ENOENT" }); + for (const child of launcher.processes) + expect(child.exitCode !== null || child.signalCode !== null).toBe(true); +}; + +const cliFailure = z.object({ message: z.string() }); + +/** Facts about the enforced deadline, independent of the caller's lifetime. */ +const enforcedLimit = { + setting: "ghidra.decompile_timeout_seconds", + effective_value: 1, + unit: "seconds", + minimum: 1, + maximum: 2_147_483, + agent_editable: true, + rea_restart_required: false, +}; +/** A finished invocation can act only through a new invocation's launch value. */ +const invocationLimit = { + ...enforcedLimit, + applies_at: "next_invocation", + scope: "rea-invocation", + environment: "REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS", +}; + +const runId = z.object({ + result: z.object({ analysis_run: z.object({ run_id: z.string() }) }), +}); + +it("tells a finished CLI invocation to rerun with the launch setting, not to reuse its released session", async () => { + const launcher = new GhidraFixtureLauncher("decompile_timed_out"); + const create = () => ghidraSession(launcher); + const dependencies: DirectAnalysisDependencies = { + readConfiguration: () => parseConfig(environment), + createBinarySession: create, + createNativeDispatchBinarySession: create, + createArtifactBinarySession: create, + createNativeBinarySession: create, + createManagedBinarySession: create, + }; + const projected = analysisCliErrorEnvelopeSchema.parse( + projectCliAnalysisResult( + await runDirectAnalysis( + dependencies, + process.execPath, + "procedure_pseudo_code", + { procedure: "fixture_main" }, + { providerId: "ghidra" }, + ), + ), + ); + // The invocation has already released the Program the guidance describes. + await expectReleased(launcher); + const recovery = ghidraDecompileTimeoutRecovery(1, false); + expect(projected).toMatchObject({ + error: "Analysis failed", + code: "provider_timeout", + category: "timeout", + retryable: true, + recovery: { kind: "inspect_timeout_state", timeout_ms: 1000 }, + // The owner's invocation recovery for the effective 1-second deadline. + remediation: { action: recovery.remediationAction }, + details: { + diagnostics: { remote_code: "decompile_timed_out" }, + limit: invocationLimit, + }, + }); + const { message } = cliFailure.parse(projected); + expect(message.startsWith(BRIDGE_MESSAGE)).toBe(true); + expect(message).not.toContain(GHIDRA_SESSION_RETAINED); +}, 30_000); + +it("keeps same-session and runtime-update guidance for a continuing MCP connection", async () => { + const launcher = new GhidraFixtureLauncher("decompile_timed_out"); + const harness = await createApplicationMcpHarness({ + session: ghidraSession(launcher), + serverOptions: { logger: silentLogger }, + clientIdentity: { name: "ghidra-decompile-deadline", version: "1" }, + }); + const tool = (name: string, arguments_: Record = {}) => + harness.client.callTool({ name, arguments: arguments_ }); + try { + const opened = await tool("open_binary", { + path: process.execPath, + provider_id: "ghidra", + }); + expect(opened.isError).not.toBe(true); + const timedOut = parseMcpToolError( + await tool("procedure_pseudo_code", { procedure: "fixture_main" }), + ).error; + expect(timedOut).toMatchObject({ + code: "provider_timeout", + retryable: true, + recovery: { kind: "inspect_timeout_state", timeout_ms: 1000 }, + remediation: { + action: ghidraDecompileTimeoutRecovery(1, false) + .continuingRemediationAction, + }, + details: { + limit: { + ...enforcedLimit, + applies_at: "next_operation", + scope: "rea-runtime", + }, + }, + }); + expect(timedOut.details?.limit).not.toHaveProperty("environment"); + expect(timedOut.message.startsWith(BRIDGE_MESSAGE)).toBe(true); + expect(timedOut.message.endsWith(` ${GHIDRA_SESSION_RETAINED}`)).toBe(true); + // The claim is true here: the same provider run answers the next call. + const run = runId.parse((await tool("binary_session")).structuredContent) + .result.analysis_run.run_id; + const address = await tool("procedure_address", { + procedure: "fixture_main", + }); + expect(address.isError, JSON.stringify(address)).not.toBe(true); + expect(parseEvidence(address.structuredContent).normalized_result).toBe( + "0x1000", + ); + expect( + runId.parse((await tool("binary_session")).structuredContent).result + .analysis_run.run_id, + ).toBe(run); + expect(launcher.processes).toHaveLength(1); + } finally { + await harness.close(); + await expectReleased(launcher); + } +}, 30_000); diff --git a/tests/boundary/mcp/ghidraInventoryMcp.test.ts b/tests/boundary/mcp/ghidraInventoryMcp.test.ts index b16e757820..02faf8ed06 100644 --- a/tests/boundary/mcp/ghidraInventoryMcp.test.ts +++ b/tests/boundary/mcp/ghidraInventoryMcp.test.ts @@ -566,3 +566,134 @@ it("rejects contradictory provider output before emitting MCP Evidence", async ( await harness.close(); } }); + +it("returns a procedure once with the exact-entry names its search matched", async () => { + const primary = { + primary: true, + dynamic: false, + external: false, + type: "function", + source: "imported", + }; + const row = ( + address: string, + value: string, + matched_aliases?: readonly string[], + ) => ({ + address, + value, + symbol: primary, + ...(matched_aliases === undefined ? {} : { matched_aliases }), + }); + let output = jsonValueSchema.parse([ + row("0x1012a0", "audit_alias", ["audit_primary"]), + ]); + const requests: unknown[] = []; + const harness = await connectGhidraMcp( + "ghidra-procedure-aliases", + (operation, input) => { + if (operation !== "search_procedures") + throw new Error(`Unexpected operation: ${operation}`); + requests.push(input); + return Promise.resolve(ok(output)); + }, + ); + const search = async (arguments_: Record) => { + const reply = await harness.mcp.callTool({ + name: "search_procedures", + arguments: arguments_, + }); + if (reply.isError === true) throw new Error(JSON.stringify(reply)); + return sessionEvidence(harness.session, reply.structuredContent); + }; + try { + // The bridge owns exact and regex matching over its entry symbols. + const exact = await search({ + pattern: "audit_primary", + case_sensitive: true, + }); + expect(exact.normalized_result).toEqual(output); + expect(exact.raw_result).toBeNull(); + expect(exact.analysis_profile?.parameters).toMatchObject({ + procedure_search: "primary-and-exact-entry-symbol-names-v1", + }); + + // Node owns Unicode folding: the bridge's non-ASCII candidates are kept + // only when they match, for secondary names as for primary names. + output = jsonValueSchema.parse([ + row("0x1012a0", "audit_alias", ["audit_primary", "Äudit_label"]), + row("0x101300", "ÄUDIT_only", ["ünrelated"]), + row("0x101340", "FUN_00101340", ["AUDİT_x", "AUDIT_label"]), + row("0x101380", "Audit_plain"), + ]); + const folded = await search({ pattern: "audit" }); + expect(folded.normalized_result).toEqual([ + row("0x1012a0", "audit_alias", ["audit_primary"]), + row("0x101340", "FUN_00101340", ["AUDIT_label"]), + row("0x101380", "Audit_plain"), + ]); + expect(folded.raw_result).toEqual(output); + expect(requests).toEqual([ + expect.objectContaining({ pattern: "audit_primary" }), + expect.objectContaining({ pattern: "audit", case_sensitive: false }), + ]); + + // An empty alias list is not a match report. + output = jsonValueSchema.parse([row("0x1012a0", "audit_alias", [])]); + const malformed = await harness.mcp.callTool({ + name: "search_procedures", + arguments: { pattern: "audit_", mode: "regex" }, + }); + expect(parseMcpToolError(malformed)).toMatchObject({ + error: { code: "unreadable_output" }, + }); + } finally { + await harness.close(); + } +}); + +it("keeps a namespaced entry name that matched by its leaf spelling", async () => { + // procedure_address accepts a symbol's leaf or qualified spelling, so the + // bridge matches both and reports the qualified one. + const output = jsonValueSchema.parse([ + { + address: "0x101400", + value: "outer::impl", + matched_aliases: ["ns::audit_alias"], + }, + { address: "0x101440", value: "alpha::same" }, + ]); + const harness = await connectGhidraMcp("ghidra-namespaced-aliases", () => + Promise.resolve(ok(output)), + ); + try { + const [aliasRow, primaryRow] = output as unknown[]; + for (const [arguments_, expected] of [ + [ + { + pattern: "^(audit_alias|same)$", + mode: "regex", + case_sensitive: true, + }, + output, + ], + // The host folding pass checks reported qualified spellings, which end + // with the leaf the bridge matched. + [{ pattern: "AUDIT_ALIAS" }, [aliasRow]], + [{ pattern: "SAME" }, [primaryRow]], + ] as const) { + const reply = await harness.mcp.callTool({ + name: "search_procedures", + arguments: arguments_, + }); + expect(reply.isError, JSON.stringify(arguments_)).not.toBe(true); + expect( + sessionEvidence(harness.session, reply.structuredContent) + .normalized_result, + JSON.stringify(arguments_), + ).toEqual(expected); + } + } finally { + await harness.close(); + } +}); diff --git a/tests/boundary/mcp/ghidraMcpHarness.ts b/tests/boundary/mcp/ghidraMcpHarness.ts index 5b6a5c5b3e..36fe94a0d9 100644 --- a/tests/boundary/mcp/ghidraMcpHarness.ts +++ b/tests/boundary/mcp/ghidraMcpHarness.ts @@ -3,6 +3,7 @@ import type { GhidraJavaObservation } from "../../../src/ghidra/GhidraJavaProbe. import { expect } from "vitest"; import { parseEvidence } from "../../../src/domain/evidence.js"; import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; +import type { createMcpSchemaValidation } from "../../fixtures/mcpSchemaValidation.js"; import { composeBinarySession } from "../../../src/application/binary/BinarySessionComposition.js"; import type { BinarySession } from "../../../src/application/binary/BinarySession.js"; import { AnalysisProviderRegistry } from "../../../src/application/binary/AnalysisProviderRegistry.js"; @@ -15,6 +16,7 @@ import type { GhidraProviderClientFactory } from "../../../src/ghidra/GhidraProv import { GHIDRA_SESSION_CAPABILITIES } from "../../../src/ghidra/GhidraSessionValues.js"; import type { GhidraSessionInfo } from "../../../src/ghidra/GhidraSessionValues.js"; import { silentLogger } from "../../../src/logger.js"; +import type { ToolResultDelivery } from "../../../src/server/ToolResultDelivery.js"; import { ghidraFunctionClassification, ghidraFunctionDossier, @@ -33,6 +35,9 @@ export const connectGhidraMcp = async ( readonly sessionTarget?: GhidraSessionInfo["target"]; readonly attestDosTargetSnapshot?: ReturnType["attestDosTargetSnapshot"]; readonly close?: ReturnType["close"]; + readonly delivery?: ToolResultDelivery; + readonly environment?: Readonly>; + readonly schemaValidator?: ReturnType; } = {}, ) => { const calls: GhidraOperation[] = []; @@ -72,12 +77,17 @@ export const connectGhidraMcp = async ( const { client: mcp, close } = await createApplicationMcpHarness({ session, clientIdentity: { name, version: "1.0.0" }, + ...(fixture.schemaValidator === undefined + ? {} + : { schemaValidator: fixture.schemaValidator }), serverOptions: { logger: silentLogger, environment: { ...process.env, REA_MCP_MAX_INPUT_ELEMENTS: process.env.REA_MCP_MAX_INPUT_ELEMENTS, + ...fixture.environment, }, + ...(fixture.delivery === undefined ? {} : { delivery: fixture.delivery }), }, }); try { diff --git a/tests/boundary/mcp/ghidraReopen.test.ts b/tests/boundary/mcp/ghidraReopen.test.ts index fc2bdc7232..2690a795d6 100644 --- a/tests/boundary/mcp/ghidraReopen.test.ts +++ b/tests/boundary/mcp/ghidraReopen.test.ts @@ -39,6 +39,8 @@ type FixtureLaunch = { class FixtureLauncher implements GhidraLauncher { readonly launches: FixtureLaunch[] = []; readonly requests: string[] = []; + /** Extra handshake fields, one entry per launch in order. */ + readonly handshakeReports: Record[] = []; cleanupAllowed = true; cleanupAttempts = 0; @@ -57,6 +59,7 @@ class FixtureLauncher implements GhidraLauncher { profileDigest: session.profileDigest, targetSha256: session.targetSha256, transport: session.transport, + handshakeReports: this.handshakeReports.shift() ?? {}, }), }, }); @@ -208,6 +211,22 @@ const connectGhidra = async () => { }; }; +const UNCOMMITTED_SEEDS = { + analysis_seeds: { + format: "rea-ghidra-seeds-v1", + sha256: "b".repeat(64), + entries: 1, + function_created: 1, + function_existing: 0, + function_failed: 0, + code_decoded: 0, + code_failed: 0, + label_applied: 0, + label_failed: 0, + unmapped: 0, + }, +}; + const firstLaunch = (launcher: FixtureLauncher): FixtureLaunch => { const first = launcher.launches[0]; if (first === undefined) throw new Error("Expected a running Ghidra fixture"); @@ -347,3 +366,81 @@ describe.skipIf(process.platform === "win32")( }); }, ); + +describe.skipIf(process.platform === "win32")( + "Ghidra MCP handshake rejection over Unix sockets", + () => { + it("refuses an uncommitted handshake report before the first query and replaces that session on reopen", async () => { + const harness = await connectGhidra(); + try { + harness.launcher.handshakeReports.push(UNCOMMITTED_SEEDS); + expect((await harness.open()).isError).not.toBe(true); + expect(harness.launcher.launches).toHaveLength(0); + expect(parseMcpToolError(await harness.query(1))).toMatchObject({ + error: { + details: { + operation: "list_strings", + diagnostics: { + reason: expect.stringContaining( + "reported analysis seeds that the profile did not commit", + ), + }, + }, + }, + }); + const rejected = firstLaunch(harness.launcher); + await expectReleased(rejected); + expect(harness.launcher.requests).not.toContain("list_strings"); + + expect((await harness.query(2)).isError).toBe(true); + expect(harness.launcher.launches).toHaveLength(1); + expect(harness.launcher.requests).not.toContain("list_strings"); + + expect((await harness.open()).isError).not.toBe(true); + expect((await harness.query(3)).isError).not.toBe(true); + expect(harness.clients).toHaveLength(2); + expect(harness.launcher.launches).toHaveLength(2); + await expect + .poll(() => + harness.launcher.requests.filter( + (method) => method === "list_strings", + ), + ) + .toHaveLength(1); + } finally { + await harness.close(); + } + }); + + it("retains a rejected session's failed cleanup and refuses replacement until it can be retried", async () => { + const harness = await connectGhidra(); + try { + harness.launcher.handshakeReports.push(UNCOMMITTED_SEEDS); + harness.launcher.cleanupAllowed = false; + expect((await harness.open()).isError).not.toBe(true); + expect(parseMcpToolError(await harness.query(1))).toMatchObject({ + error: { code: "cleanup_incomplete" }, + }); + expect(harness.launcher.cleanupAttempts).toBe(1); + expect(harness.launcher.requests).not.toContain("list_strings"); + const rejected = firstLaunch(harness.launcher); + await access(rejected.runtimeRoot); + + expect(parseMcpToolError(await harness.open())).toMatchObject({ + error: { code: "cleanup_incomplete" }, + }); + expect(harness.launcher.cleanupAttempts).toBe(2); + expect(harness.launcher.launches).toHaveLength(1); + await access(rejected.runtimeRoot); + + harness.launcher.cleanupAllowed = true; + expect((await harness.open()).isError).not.toBe(true); + await expectReleased(rejected); + expect((await harness.query(2)).isError).not.toBe(true); + expect(harness.launcher.launches).toHaveLength(2); + } finally { + await harness.close(); + } + }); + }, +); diff --git a/tests/boundary/mcp/ghidraRoutingMcp.test.ts b/tests/boundary/mcp/ghidraRoutingMcp.test.ts index ac191dd057..dcca3a5017 100644 --- a/tests/boundary/mcp/ghidraRoutingMcp.test.ts +++ b/tests/boundary/mcp/ghidraRoutingMcp.test.ts @@ -1,3 +1,4 @@ +import { batchDecompileResultSchema } from "../../../src/domain/native/batchDecompile.js"; import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; import { describe, expect, it } from "vitest"; import { connectGhidraMcp, ghidraFixtureResult } from "./ghidraMcpHarness.js"; @@ -112,9 +113,11 @@ describe("Ghidra MCP capability routing", () => { arguments: { addresses: ["0x401000", bodyless] }, }); expect(batch.isError).not.toBe(true); - const result = toolContract("batch_decompile").outputSchema.parse( - batch.structuredContent, - ).normalized_result; + const result = batchDecompileResultSchema.parse( + toolContract("batch_decompile").outputSchema.parse( + batch.structuredContent, + ).normalized_result, + ); expect(result.items[0]).not.toHaveProperty("limitations"); expect(result).toEqual({ items: [ diff --git a/tests/boundary/mcp/ghidraStartupCancellationMcp.test.ts b/tests/boundary/mcp/ghidraStartupCancellationMcp.test.ts index f2f64b9670..cc33a01f10 100644 --- a/tests/boundary/mcp/ghidraStartupCancellationMcp.test.ts +++ b/tests/boundary/mcp/ghidraStartupCancellationMcp.test.ts @@ -7,7 +7,6 @@ import { AnalysisProviderRegistry } from "../../../src/application/binary/Analys import { composeBinarySession } from "../../../src/application/binary/BinarySessionComposition.js"; import { parseConfig } from "../../../src/config/parseConfig.js"; import { GhidraClient } from "../../../src/ghidra/GhidraClient.js"; -import type { GhidraQueuedRequestResult } from "../../../src/ghidra/GhidraRequestQueue.js"; import { ghidraProfileInstallationHost } from "../../../src/ghidra/GhidraInstallation.fixture.js"; import { GhidraProvider } from "../../../src/ghidra/GhidraProvider.js"; import type { GhidraProviderClientFactory } from "../../../src/ghidra/GhidraProviderClient.js"; @@ -26,9 +25,13 @@ it.each([0, 1])( const fixture = new GhidraFixtureLauncher(); const roots = fixture.runtimeRoots; const children = fixture.processes; - const results: (GhidraQueuedRequestResult | undefined)[] = []; + const startResults: ( + | Awaited> + | undefined + )[] = []; let producerSignal: AbortSignal | undefined; - let callCount = 0; + let startCount = 0; + let queryCount = 0; const factory: GhidraProviderClientFactory = (options) => { const raw = new GhidraClient({ ...options, @@ -42,17 +45,21 @@ it.each([0, 1])( }, }); return { - start: raw.start.bind(raw), - close: raw.close.bind(raw), - callTool: (operation, parameters, requestOptions) => { - const index = callCount++; - const pending = raw.callTool(operation, parameters, requestOptions); + // Every query waits here for the checked handshake before it can run. + start: (signal, progress) => { + const index = startCount++; + const pending = raw.start(signal, progress); admitted[index]?.resolve(); return pending.then((result) => { - results[index] = result; + startResults[index] = result; return result; }); }, + close: raw.close.bind(raw), + callTool: (operation, parameters, requestOptions) => { + queryCount++; + return raw.callTool(operation, parameters, requestOptions); + }, }; }; const config = parseConfig({ @@ -107,7 +114,7 @@ it.each([0, 1])( await entered.promise; controllers[cancelledIndex]?.abort(); await vi.waitFor(() => - expect(results[cancelledIndex]).toMatchObject({ + expect(startResults[cancelledIndex]).toMatchObject({ ok: false, error: { kind: "cancelled" }, }), @@ -115,16 +122,18 @@ it.each([0, 1])( expect(await calls[cancelledIndex]).toMatchObject({ ok: false }); expect(producerSignal?.aborted).toBe(false); expect(children).toHaveLength(0); + expect(queryCount).toBe(0); gate.resolve(); const staying = await calls[1 - cancelledIndex]; if (staying === undefined || !staying.ok) throw new Error("Live MCP peer did not complete", { cause: staying }); expect(staying.value.isError).not.toBe(true); - expect(results[1 - cancelledIndex]).toMatchObject({ ok: true }); + expect(startResults[1 - cancelledIndex]).toMatchObject({ ok: true }); expect(controllers[1 - cancelledIndex]?.signal.aborted).toBe(false); const warm = await mcp.callTool({ name: "list_strings", arguments: {} }); expect(warm.isError).not.toBe(true); - expect(callCount).toBe(3); + expect(startCount).toBe(3); + expect(queryCount).toBe(2); expect(children).toHaveLength(1); } finally { for (const controller of controllers) controller.abort(); diff --git a/tests/boundary/mcp/javascriptRuntimeObservationMcp.test.ts b/tests/boundary/mcp/javascriptRuntimeObservationMcp.test.ts index 82edcee870..36c4d736d0 100644 --- a/tests/boundary/mcp/javascriptRuntimeObservationMcp.test.ts +++ b/tests/boundary/mcp/javascriptRuntimeObservationMcp.test.ts @@ -1,5 +1,5 @@ import { ok as resultOk } from "../../../src/domain/result.js"; -import { rm, writeFile } from "node:fs/promises"; +import { writeFile } from "node:fs/promises"; import { join } from "node:path"; import { pathToFileURL } from "node:url"; @@ -20,26 +20,21 @@ import { JAVASCRIPT_RUNTIME_RECONCILIATION_EXAMPLE } from "../../../src/contract import { createJavaScriptRuntimeObservationEvidence } from "../../../src/application/javascript/JavaScriptRuntimeObservationEvidence.js"; import { V8_INSPECTOR_PROVIDER_IDENTITY } from "../../../src/inspector/providerIdentity.js"; import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { parseMcpEvidenceBundle } from "../../fixtures/mcpEvidenceBundle.js"; import { observed } from "../../fixtures/analysisExecution.js"; import { startFakeV8Inspector } from "../../fixtures/inspector/fakeV8Inspector.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; describe("JavaScript runtime observation MCP tools", () => { const resources: Array<{ close(): Promise }> = []; - const temporary: string[] = []; afterEach(async () => { await Promise.all(resources.splice(0).map((item) => item.close())); - await Promise.all( - temporary - .splice(0) - .map((path) => rm(path, { recursive: true, force: true })), - ); }); registerRuntimeParameterContract(resources); registerRuntimeMetadataContract(); - registerRetainedRuntimeEvidence(resources, temporary); + registerRetainedRuntimeEvidence(resources); }); function registerRuntimeParameterContract( @@ -205,11 +200,9 @@ function registerRuntimeMetadataContract(): void { function registerRetainedRuntimeEvidence( resources: Array<{ close(): Promise }>, - temporary: string[], ): void { test("lists and observes one target as retained Evidence", async () => { const root = await createTestTempDirectory("rea-v8-mcp-"); - temporary.push(root); const { entry, inspector, client } = await createObservationClient( root, resources, @@ -271,6 +264,14 @@ function registerRetainedRuntimeEvidence( }, }); const evidenceId = evidenceIdFrom(observedRuntime.structuredContent); + const bundle = await client.callTool({ + name: "get_evidence_bundle", + arguments: {}, + }); + expect(bundle.isError).not.toBe(true); + expect(parseMcpEvidenceBundle(bundle).records).toContainEqual( + observedRuntime.structuredContent, + ); expect(observedRuntime.structuredContent).toMatchObject({ evidence_id: evidenceId, operation: "observe_javascript_runtime", @@ -279,8 +280,10 @@ function registerRetainedRuntimeEvidence( expect(new Set(inspector.commands.map(({ method }) => method))).toEqual( new Set(["Runtime.enable", "Debugger.enable"]), ); - const direct = await observeDirectly(inspector); - expect(direct.ok).toBe(true); + const provider = new V8InspectorProvider(); + resources.push(provider); + const direct = await observeDirectly(inspector, provider); + expect(direct.ok, direct.ok ? undefined : direct.error.message).toBe(true); if (direct.ok) expect(evidenceId).toBe(direct.value.evidence_id); }); } @@ -335,9 +338,10 @@ const createObservationClient = async ( const observeDirectly = ( inspector: Awaited>, + provider: V8InspectorProvider, ) => observeJavaScriptRuntime( - new V8InspectorProvider(), + provider, observeJavaScriptRuntimeInputSchema.parse({ inspector_endpoint: inspector.endpoint, target_id: inspector.targetId, diff --git a/tests/boundary/mcp/jebMcp.test.ts b/tests/boundary/mcp/jebMcp.test.ts index 8509fb7752..d3a69cb028 100644 --- a/tests/boundary/mcp/jebMcp.test.ts +++ b/tests/boundary/mcp/jebMcp.test.ts @@ -32,7 +32,8 @@ it("discovers current JEB operation support and recovery through binary_session" let availability: JebAnalysisAvailability = { status: "unavailable", code: "not_configured", - reason: "Start JEB at REA_JEB_MCP_URL", + reason: "The JEB MCP endpoint did not respond.", + remediation: "Start JEB at REA_JEB_MCP_URL", diagnostics: {}, }; let probes = 0; @@ -71,7 +72,10 @@ it("discovers current JEB operation support and recovery through binary_session" expect(unavailable).toHaveLength(4); expect( unavailable.every( - (tool) => !tool.available && tool.remediation === availability.reason, + (tool) => + !tool.available && + availability.status === "unavailable" && + tool.remediation === availability.remediation, ), ).toBe(true); availability = { diff --git a/tests/boundary/mcp/kunaCloseRetryMcp.test.ts b/tests/boundary/mcp/kunaCloseRetryMcp.test.ts index 9e52ecb78a..3e4e00f809 100644 --- a/tests/boundary/mcp/kunaCloseRetryMcp.test.ts +++ b/tests/boundary/mcp/kunaCloseRetryMcp.test.ts @@ -88,9 +88,21 @@ it("names the same-connection close_binary retry for an unproven Kuna export sto expect(failed.error.remediation.action).toBe( `${analysisErrorRemediationAction(delivered.error)} ${cleanupRetryRemediation({ operation: "close_binary" })}`, ); - expect(await sessionStatus(client)).toMatchObject({ + const pendingStatus = await sessionStatus(client); + expect(pendingStatus).toMatchObject({ open: false, cleanup_pending: true, + cleanup_retry: { operation: "close_binary" }, + }); + const unavailable = pendingStatus.tool_availability.groups.find( + ({ names }) => names.includes("procedure_assembly"), + ); + expect(unavailable).toMatchObject({ + available: false, + reason: "target_required", + remediation: expect.stringContaining( + cleanupRetryRemediation({ operation: "close_binary" }), + ), }); // Reopening stays blocked; only close_binary offers its retry. const blocked = parseMcpToolError( @@ -109,10 +121,12 @@ it("names the same-connection close_binary retry for an unproven Kuna export sto unproven.release(); const recovered = await call("close_binary"); expect(recovered.isError, JSON.stringify(recovered.content)).not.toBe(true); - expect(await sessionStatus(client)).toMatchObject({ + const settledStatus = await sessionStatus(client); + expect(settledStatus).toMatchObject({ open: false, cleanup_pending: false, }); + expect(settledStatus).not.toHaveProperty("cleanup_retry"); expect(await readdir(join(fixture.cacheRoot, "staging"))).toEqual([]); expect(Buffer.compare(await readFile(program), targetBytes)).toBe(0); } finally { diff --git a/tests/boundary/mcp/managedMcp.test.ts b/tests/boundary/mcp/managedMcp.test.ts index 4e45cf91fb..ba370ca045 100644 --- a/tests/boundary/mcp/managedMcp.test.ts +++ b/tests/boundary/mcp/managedMcp.test.ts @@ -344,6 +344,16 @@ const verifyManagedCatalogAndNativeWorkflow = async ( await outputAssertion.assert("verify_managed_native_boundaries", verified); expect(verified).toMatchObject({ evidence_id: expect.stringMatching(/^ev_[a-f0-9]{64}$/u), + operation: "verify_managed_native_boundaries", + provider: { id: "rea-dotnet-workflows" }, + confidence: "inferred", + authority: "analyst-inference", + evidence_links: [ + MANAGED_NATIVE_VERIFICATION_EXAMPLE.managed_boundaries.evidence_id, + ...MANAGED_NATIVE_VERIFICATION_EXAMPLE.native_observations.map( + ({ evidence_id }) => evidence_id, + ), + ], normalized_result: { summary: { verified: 1 }, algorithm: { token_to_address_mapping: "not-inferred" }, diff --git a/tests/boundary/mcp/modelArtifactPartialCancellation.test.ts b/tests/boundary/mcp/modelArtifactPartialReadFailure.test.ts similarity index 95% rename from tests/boundary/mcp/modelArtifactPartialCancellation.test.ts rename to tests/boundary/mcp/modelArtifactPartialReadFailure.test.ts index 58df84f360..857c8fa0fa 100644 --- a/tests/boundary/mcp/modelArtifactPartialCancellation.test.ts +++ b/tests/boundary/mcp/modelArtifactPartialReadFailure.test.ts @@ -52,7 +52,7 @@ mcpTest.for([ { retained: false, fullHash: true }, { retained: true, fullHash: true }, ])( - "projects partial model Evidence through an MCP tool error ($retained, fullHash=$fullHash)", + "projects partial model Evidence after a denied read ($retained, fullHash=$fullHash)", async ({ retained, fullHash }, { mcp }) => { const root = await createTestTempDirectory("rea-model-mcp-partial-"); const path = join(root, fullHash ? "model.safetensors" : "model.gguf"); @@ -133,6 +133,14 @@ mcpTest.for([ }); if (!observedArrayChunk) throw new Error(JSON.stringify(response)); const projected = parseMcpToolError(response); + expect(projected.error).toMatchObject({ + code: "access_denied", + details: { + operation: "inspect_model_artifact", + path, + system_code: "EACCES", + }, + }); const details = projected.error.details; if (details === undefined) throw new Error("MCP error omitted its partial observation"); diff --git a/tests/boundary/mcp/nativeValueTraceCleanupIncomplete.test.ts b/tests/boundary/mcp/nativeValueTraceCleanupIncomplete.test.ts new file mode 100644 index 0000000000..6e28b05750 --- /dev/null +++ b/tests/boundary/mcp/nativeValueTraceCleanupIncomplete.test.ts @@ -0,0 +1,334 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { afterEach, describe, expect, it } from "vitest"; +import { z } from "zod"; + +import type { AnalysisOperationPort } from "../../../src/application/AnalysisProvider.js"; +import { CLEANUP_STOPPED_VALUE_TRACE } from "../../../src/application/native/NativeValueTrace.js"; +import { + failingValueTraceReads, + valueTraceDiamond, + valueTraceProvider as provider, + VALUE_TRACE_DIAMOND_READS as ALL_READS, + type ValueTraceDiamondRead, +} from "../../../src/application/native/NativeValueTrace.fixture.js"; +import type { AnalysisError } from "../../../src/domain/analysisErrorBase.js"; +import { AnalysisCapabilityUnavailableError } from "../../../src/domain/analysisErrorCore.js"; +import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js"; +import { parseEvidence } from "../../../src/domain/evidence.js"; +import { evidenceBundleSchema } from "../../../src/domain/evidenceBundle.js"; +import { nativeValueTraceSchema } from "../../../src/domain/native/nativeValueTrace.js"; +import { ok } from "../../../src/domain/result.js"; +import { createServer } from "../../../src/server/createServer.js"; +import { createTestBinarySession } from "../../fixtures/binarySession.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { + closeEnhancedToolResources, + connect, + jsonResult, + resources, +} from "./enhancedToolsHarness.js"; + +afterEach(closeEnhancedToolResources); + +const CHILD_FAILURE = "controlled child failure"; + +const childFailure = (operation: string, cleanup: boolean) => { + const failure = new AnalysisCapabilityUnavailableError( + provider.id, + operation, + CHILD_FAILURE, + cleanup + ? { + cleanup: { + reason: "cleanup proof refused", + resources: ["owned worker"], + }, + } + : undefined, + ); + // The owner retaining the worker binds this retry; the trace must keep it. + return cleanup + ? failure.bindCleanupRetry({ operation: "close_binary" }) + : failure; +}; + +/** Serve the diamond, failing each listed read, and record every dispatched read. */ +const failingDiamond = ( + failures: Partial>, + options: { readonly sessionSubject?: boolean } = {}, +) => failingValueTraceReads(valueTraceDiamond(options).port, failures); + +const request = { + procedure: "0x1000", + max_depth: 1, + max_functions: 4, + max_call_sites: 20, + max_nodes: 100, + max_edges: 200, + limit: 100, + offset: 0, +}; +const trace = { name: "trace_native_values", arguments: request }; + +const stops = [ + { + read: "resolve_native_call_targets:0x1004", + stop: { procedure: "0x1000", address: "0x1004" }, + procedures: ["0x1000"], + calls: [], + // An unresolved call never discovers 0x2000. + ordinaryReads: [0, 1, 2, 5], + }, + { + read: "analyze_function:0x2000", + stop: { procedure: "0x2000", address: null }, + procedures: ["0x1000"], + calls: ["0x1000/call-b->0x2000", "0x1000/call-c->0x3000"], + // A failed dossier has no call sites to resolve. + ordinaryReads: [0, 1, 2, 3, 5], + }, + { + read: "resolve_native_call_targets:0x2004", + stop: { procedure: "0x2000", address: "0x2004" }, + procedures: ["0x1000", "0x2000"], + calls: ["0x1000/call-b->0x2000", "0x1000/call-c->0x3000"], + ordinaryReads: [0, 1, 2, 3, 4, 5], + }, +] as const; + +const callEdges = (value: z.output) => + value.edges + .filter(({ kind }) => kind === "call") + .map(({ source, target }) => `${source}->${target}`); + +describe("trace_native_values after a validated seed", () => { + it.each(stops)( + "stops at unconfirmed cleanup from $read and reports it with the admitted trace", + async ({ read, stop, procedures, calls }) => { + const operation = read.split(":")[0] ?? ""; + const { port, reads } = failingDiamond({ + [read]: childFailure(operation, true), + }); + const client = await connect(port); + + const { error } = parseMcpToolError(await client.callTool(trace)); + + // Nothing is dispatched after the read whose cleanup is unconfirmed. + expect(reads).toEqual(ALL_READS.slice(0, ALL_READS.indexOf(read) + 1)); + expect(error).toMatchObject({ + code: "cleanup_incomplete", + recovery: { kind: "retry_cleanup", operation: "close_binary" }, + details: { + operation, + cleanup: "incomplete", + resources: ["owned worker"], + cleanup_reason: "cleanup proof refused", + execution_failure: "capability_unavailable", + }, + }); + expect(JSON.stringify(error)).toContain(CHILD_FAILURE); + // Without an Evidence writer the admitted trace stays inline. + const partial = parseEvidence(error.details?.partial_observation); + expect(partial).toMatchObject({ + operation: "trace_native_values", + parameters: request, + }); + const admitted = nativeValueTraceSchema.parse(partial.normalized_result); + expect(admitted.procedures.map(({ address }) => address)).toEqual( + procedures, + ); + expect(callEdges(admitted)).toEqual(calls); + expect(admitted).toMatchObject({ + traversal: { status: "truncated", reasons: ["cleanup_incomplete"] }, + truncated: true, + }); + expect(admitted.unknowns).toContainEqual({ + ...stop, + reason: CLEANUP_STOPPED_VALUE_TRACE, + }); + // The stopping failure is the outcome, not one of the trace's unknowns. + expect(JSON.stringify(admitted.unknowns)).not.toContain(CHILD_FAILURE); + }, + ); + + it.each(stops)( + "keeps an ordinary failure from $read as a located unknown and continues", + async ({ read, stop, ordinaryReads }) => { + const operation = read.split(":")[0] ?? ""; + const { port, reads } = failingDiamond({ + [read]: childFailure(operation, false), + }); + const client = await connect(port); + + const response = await client.callTool(trace); + + expect(response.isError).not.toBe(true); + expect(reads).toEqual(ordinaryReads.map((index) => ALL_READS[index])); + const completed = nativeValueTraceSchema.parse(jsonResult(response)); + expect(completed.traversal).toEqual({ status: "complete" }); + expect(completed.unknowns).toContainEqual({ + ...stop, + reason: expect.stringContaining(CHILD_FAILURE), + }); + }, + ); + + it("keeps an earlier ordinary failure and dispatches nothing after a later cleanup failure", async () => { + const { port, reads } = failingDiamond({ + "resolve_native_call_targets:0x1004": childFailure( + "resolve_native_call_targets", + false, + ), + "resolve_native_call_targets:0x1008": childFailure( + "resolve_native_call_targets", + true, + ), + }); + const client = await connect(port); + + const { error } = parseMcpToolError(await client.callTool(trace)); + + expect(reads).toEqual(ALL_READS.slice(0, 3)); + expect(error).toMatchObject({ + code: "cleanup_incomplete", + details: { resources: ["owned worker"] }, + }); + const admitted = nativeValueTraceSchema.parse( + parseEvidence(error.details?.partial_observation).normalized_result, + ); + expect(admitted.unknowns).toEqual([ + { + procedure: "0x1000", + address: "0x1004", + reason: expect.stringContaining(CHILD_FAILURE), + }, + { + procedure: "0x1000", + address: "0x1008", + reason: CLEANUP_STOPPED_VALUE_TRACE, + }, + ]); + }); + + it("returns a seed cleanup failure without a trace", async () => { + const { port, reads } = failingDiamond({ + "analyze_function:0x1000": childFailure("analyze_function", true), + }); + const client = await connect(port); + + const { error } = parseMcpToolError(await client.callTool(trace)); + + expect(reads).toEqual(["analyze_function:0x1000"]); + expect(error).toMatchObject({ + code: "cleanup_incomplete", + details: { resources: ["owned worker"] }, + }); + expect(error.details?.partial_observation).toBeUndefined(); + }); +}); + +/** Serve the failing diamond through a production session with a committed profile. */ +const openSession = async (port: AnalysisOperationPort) => { + const directory = await createTestTempDirectory("rea-value-trace-cleanup-"); + const target = join(directory, "fixture.com"); + await writeFile(target, Buffer.from([0xb8, 0x00, 0x4c, 0xcd, 0x21])); + const session = createTestBinarySession({ + identity: () => provider, + capabilities: () => + (["analyze_function", "resolve_native_call_targets"] as const).map( + (operation) => ({ + provider, + operation, + available: true, + reason: null, + cachePolicy: "live", + limitations: [], + effects: { + mutatesArtifact: false, + launchesProcess: false, + mayShowUi: false, + mayAccessNetwork: false, + mayWriteFilesystem: false, + changesPermissions: false, + requiresRoot: false, + }, + }), + ), + // A committed profile lets the session retain each admitted read. + resolveAnalysisProfile: async () => + ok({ profile: createAnalysisProfile(provider, { fixture: true }) }), + createClient: () => ({ + execute: (operation, parameters, options) => + port.execute(operation, parameters, options), + close: async () => ok(null), + }), + }); + const server = createServer({ kind: "session", session }); + const client = new Client({ name: "value-trace-cleanup", version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + resources.push(client, server); + await server.connect(serverTransport); + await client.connect(clientTransport); + const opened = await client.callTool({ + name: "open_binary", + arguments: { path: target, format: "dos-com" }, + }); + expect(opened.isError).not.toBe(true); + return { client, session }; +}; + +it("retains the admitted trace and its completed sources as session Evidence", async () => { + const { port, reads } = failingDiamond( + { + "analyze_function:0x3000": childFailure("analyze_function", true), + }, + { sessionSubject: true }, + ); + const { client, session } = await openSession(port); + + const { error } = parseMcpToolError(await client.callTool(trace)); + + expect(reads).toEqual(ALL_READS); + expect(error).toMatchObject({ + code: "cleanup_incomplete", + details: { resources: ["owned worker"] }, + }); + const reference = z + .strictObject({ + kind: z.literal("retained-evidence"), + evidence_id: z.string(), + }) + .parse(error.details?.partial_observation); + const exported = await client.callTool({ + name: "get_evidence_bundle", + arguments: {}, + }); + expect(exported.isError).not.toBe(true); + const bundle = evidenceBundleSchema.parse( + z.object({ result: z.unknown() }).parse(exported.structuredContent).result, + ); + expect(bundle.records.map(({ evidence_id }) => evidence_id)).toContain( + reference.evidence_id, + ); + const retained = session.evidenceForAnalysis(reference.evidence_id); + expect(retained).toMatchObject({ + operation: "trace_native_values", + subject: { format: "dos-com" }, + }); + expect( + nativeValueTraceSchema.parse(retained?.normalized_result).traversal, + ).toEqual({ status: "truncated", reasons: ["cleanup_incomplete"] }); + // Both dossiers completed before the stop stay linked as sources. + const dossiers = (retained?.evidence_links ?? []).flatMap((link) => { + const source = session.evidenceForAnalysis(link); + return source?.operation === "analyze_function" + ? [String(source.parameters.procedure)] + : []; + }); + expect(dossiers.sort()).toEqual(["0x1000", "0x2000"]); +}); diff --git a/tests/boundary/mcp/nativeViewPredicateMcp.test.ts b/tests/boundary/mcp/nativeViewPredicateMcp.test.ts new file mode 100644 index 0000000000..d07ec6bd62 --- /dev/null +++ b/tests/boundary/mcp/nativeViewPredicateMcp.test.ts @@ -0,0 +1,200 @@ +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { expect, it } from "vitest"; +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { FUNCTION_COMPARISON_EXAMPLE } from "../../../src/contracts/functionComparisonExample.js"; +import { + createEvidence, + parseEvidence, + type Evidence, +} from "../../../src/domain/evidence.js"; +import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js"; +import { createServer } from "../../../src/server/createServer.js"; +import { createTestBinarySession } from "../../fixtures/binarySession.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { createMcpSchemaValidation } from "../../fixtures/mcpSchemaValidation.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; + +const fixture = FUNCTION_COMPARISON_EXAMPLE.left; +if (fixture.subject === null) + throw new Error("Expected artifact-bound fixture"); +const subject = { + path: fixture.subject.local_path, + sha256: fixture.subject.digest.sha256, + format: fixture.subject.format, +}; +const selected = ( + operation: string, + result: Evidence["normalized_result"], + predicateType: string, +) => + createEvidence(subject, fixture.provider, { + operation, + parameters: {}, + result, + predicateType, + confidence: "observed", + authority: "shipped-artifact", + }); + +const wrongMatchingPredicate = (matching: Evidence) => + createEvidence(undefined, matching.provider, { + operation: matching.operation, + parameters: matching.parameters, + result: matching.normalized_result, + predicateType: "rea.analysis", + confidence: matching.confidence, + authority: matching.authority, + evidenceLinks: matching.evidence_links, + }); + +it.each(["full", "compact"] as const)( + "validates native parent predicates across public sources (%s schema)", + async (profile) => { + const session = createTestBinarySession(() => { + throw new Error("No provider should start"); + }); + const server = createServer( + { kind: "session", session }, + { + environment: { ...process.env, REA_MCP_OUTPUT_SCHEMA_PROFILE: profile }, + }, + ); + const client = new Client({ name: "native-view-identity", version: "1" }); + const validation = createMcpSchemaValidation(); + const [ct, st] = InMemoryTransport.createLinkedPair(); + const root = await createTestTempDirectory("rea-native-view-identity-"); + try { + await server.connect(st); + await client.connect(ct); + const definitions = new Map( + (await client.listTools()).tools.map((tool) => [tool.name, tool]), + ); + const viewTool = definitions.get("inspect_analysis_view"); + if (viewTool?.outputSchema === undefined) + throw new Error("Missing advertised view schema"); + expect(validation.assertions.validateSchema(viewTool.outputSchema)).toBe( + true, + ); + const call = (name: string, args: Record) => { + const toolDefinition = definitions.get(name); + if (toolDefinition === undefined) throw new Error(`Missing ${name}`); + return client.callTool({ name, arguments: args }, { toolDefinition }); + }; + const inventory = selected( + "list_procedures", + [{ address: "0x1000", value: "main" }], + "rea.analysis", + ); + const matchingResult = await call("match_native_functions", { + left: inventory, + right: inventory, + detail: "complete", + }); + expect( + matchingResult.isError, + JSON.stringify(matchingResult.content), + ).not.toBe(true); + const matching = parseEvidence(matchingResult.structuredContent); + expect(matching.predicate_type).toBe("rea.native-function-matching"); + const wrongMatching = wrongMatchingPredicate(matching); + const graph = { + graph: { + "0": [ + { + address: "0x1000", + status: "ok", + calls: [], + relationship_coverage: { status: "complete" }, + }, + ], + }, + limitations: [], + }; + const cases = [ + { + good: selected( + "analyze_function", + fixture.normalized_result, + "rea.analysis", + ), + bad: selected( + "analyze_function", + fixture.normalized_result, + "rea.unrelated", + ), + view: { kind: "summary" }, + }, + { + good: inventory, + bad: selected( + "list_procedures", + inventory.normalized_result, + "rea.unrelated", + ), + view: { kind: "native-inventory", offset: 0, limit: 1 }, + }, + { + good: selected("procedure_address", "0x1000", "rea.analysis"), + bad: selected("procedure_address", "0x1000", "rea.unrelated"), + view: { kind: "native-observation", offset: 0, limit: 1 }, + }, + { + good: selected("get_call_graph", graph, "rea.analysis"), + bad: selected("get_call_graph", graph, "rea.unrelated"), + view: { kind: "native-observation", offset: 0, limit: 1 }, + }, + { good: matching, bad: wrongMatching, view: { kind: "summary" } }, + ]; + for (const [index, { good, bad, view }] of cases.entries()) { + expect(session.recordEvidence(good).ok).toBe(true); + expect(session.recordEvidence(bad).ok).toBe(true); + const goodReply = await call("inspect_analysis_view", { + source: { kind: "retained-evidence", evidence_id: good.evidence_id }, + view, + }); + expect(goodReply.isError, JSON.stringify(goodReply.content)).not.toBe( + true, + ); + expect( + validation.assertions.validate( + viewTool.outputSchema, + goodReply.structuredContent, + ), + JSON.stringify(validation.assertions.errors), + ).toBe(true); + const goodView = parseEvidence(goodReply.structuredContent); + expect(goodView.evidence_links).toContain(good.evidence_id); + const raw = join(root, `${index}.json`), + bundle = join(root, `${index}-bundle.json`); + await writeFile(raw, JSON.stringify(bad)); + await writeFile(bundle, JSON.stringify(createEvidenceBundle([bad]))); + for (const source of [ + { kind: "inline", evidence: bad }, + { kind: "retained-evidence", evidence_id: bad.evidence_id }, + { kind: "evidence-file", path: raw }, + { kind: "evidence-file", path: bundle }, + ]) { + const reply = await call("inspect_analysis_view", { source, view }); + expect(parseMcpToolError(reply).error).toMatchObject({ + code: "invalid_request", + details: { + issues: [ + expect.objectContaining({ + path: ["source", "predicate_type"], + reason: "invalid_value", + }), + ], + }, + }); + } + } + expect(session.activeTarget()).toBeUndefined(); + } finally { + await client.close(); + await server.close(); + await session.close(); + validation.clear(); + } + }, +); diff --git a/tests/boundary/mcp/pcapMcp.test.ts b/tests/boundary/mcp/pcapMcp.test.ts index 95154eb14e..8678cac2df 100644 --- a/tests/boundary/mcp/pcapMcp.test.ts +++ b/tests/boundary/mcp/pcapMcp.test.ts @@ -8,6 +8,7 @@ import { parseEvidence } from "../../../src/domain/evidence.js"; import { ok } from "../../../src/domain/result.js"; import { createServer } from "../../../src/server/createServer.js"; import { createTestBinarySession } from "../../fixtures/binarySession.js"; +import { parseMcpEvidenceBundle } from "../../fixtures/mcpEvidenceBundle.js"; const identity = { id: "pcap", name: "Packet capture analysis", version: null }; @@ -116,6 +117,12 @@ it("advertises exact valid pcap schemas and records inline evidence through MCP" .map((part) => part.text ?? "") .join(""); const evidence = parseEvidence(JSON.parse(text)); + const bundle = await client.callTool({ + name: "get_evidence_bundle", + arguments: {}, + }); + expect(bundle.isError).not.toBe(true); + expect(parseMcpEvidenceBundle(bundle).records).toContainEqual(evidence); expect(evidence.normalized_result).toMatchObject({ packet_count: 50, capture_start: "2026-10-11T14:13:40.608505Z", diff --git a/tests/boundary/mcp/promptMcp.test.ts b/tests/boundary/mcp/promptMcp.test.ts index 7b732a78d0..c57e0f7d35 100644 --- a/tests/boundary/mcp/promptMcp.test.ts +++ b/tests/boundary/mcp/promptMcp.test.ts @@ -1,5 +1,5 @@ import { ok as resultOk, err, ok } from "../../../src/domain/result.js"; -import { readFile, rm, writeFile } from "node:fs/promises"; +import { readFile, writeFile } from "node:fs/promises"; import { basename, join } from "node:path"; import { @@ -45,17 +45,11 @@ import { observed } from "../../fixtures/analysisExecution.js"; import { serveStreamableHttp } from "../../../src/server/http/serveStreamableHttp.js"; const resources: Array<{ close(): Promise }> = []; -const temporaryDirectories: string[] = []; afterEach(async () => { await Promise.allSettled( resources.splice(0).map((resource) => resource.close()), ); - await Promise.all( - temporaryDirectories - .splice(0) - .map((directory) => rm(directory, { recursive: true, force: true })), - ); }); describe("guided prompts over MCP", () => { @@ -364,7 +358,6 @@ describe("guided prompt family scope", () => { it("preserves caller-selected paths and verbatim crash signals", async () => { const directory = await createTestTempDirectory("rea-prompt-exact-context-"); - temporaryDirectories.push(directory); const targetPath = join(directory, "target with trailing space "); await writeFile(targetPath, "owned prompt target", "utf8"); await expect(readFile(targetPath.trim(), "utf8")).rejects.toThrow(); @@ -493,7 +486,6 @@ describe("guided prompt completion registry", () => { it("guided prompt completion avoids cold startup and cancels warm lookup on close", async () => { const directory = await createTestTempDirectory("rea-prompt-warm-only-"); - temporaryDirectories.push(directory); const target = join(directory, "target.hop"); await writeFile(target, "fixture target"); const fixture = await createWarmCompletionFixture(target); @@ -696,7 +688,6 @@ describe("guided prompt completion lifecycle", () => { it("refreshes wire completions across open, switch, concurrency, and close", async () => { const directory = await createTestTempDirectory("rea-prompt-mcp-"); - temporaryDirectories.push(directory); const first = join(directory, "first.hop"); const second = join(directory, "second.hop"); await Promise.all([writeFile(first, "first"), writeFile(second, "second")]); diff --git a/tests/boundary/mcp/providerSelectionMcp.test.ts b/tests/boundary/mcp/providerSelectionMcp.test.ts index a93013cdfb..a821fb5f30 100644 --- a/tests/boundary/mcp/providerSelectionMcp.test.ts +++ b/tests/boundary/mcp/providerSelectionMcp.test.ts @@ -1,4 +1,5 @@ import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { createHash } from "node:crypto"; import { rm, writeFile } from "node:fs/promises"; import { join } from "node:path"; @@ -21,6 +22,8 @@ import { import { AnalysisProviderRegistry } from "../../../src/application/binary/AnalysisProviderRegistry.js"; import { composeBinarySession } from "../../../src/application/binary/BinarySessionComposition.js"; import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js"; +import { analysisErrorRemediationAction } from "../../../src/domain/analysisErrorPresentation.js"; +import { AnalysisSnapshotMismatchError } from "../../../src/domain/evidenceErrors.js"; import { err, ok } from "../../../src/domain/result.js"; import { AnalysisCancelledError } from "../../../src/domain/analysisErrorCore.js"; import { ProviderAdapterError } from "../../../src/domain/providerAdapterError.js"; @@ -546,6 +549,156 @@ describe("binary_session detail over MCP", () => { }); }); +it("reports confirmed snapshot binding differences and keeps the active session", async () => { + const testDirectory = await createTestTempDirectory( + "rea-snapshot-mismatch-mcp-", + ); + const first = join(testDirectory, "first.hop"); + const second = join(testDirectory, "second.hop"); + const snapshotPath = join(testDirectory, "analysis.json"); + await writeFile(first, "first fixture"); + await writeFile(second, "second fixture"); + const sha = (value: string) => + createHash("sha256").update(value).digest("hex"); + let revision = 1; + const alphaProfile = createAnalysisProfile( + { id: "alpha", name: "alpha provider", version: "1" }, + { revision: 1 }, + ); + const betaProfile = createAnalysisProfile( + { id: "beta", name: "beta provider", version: "1" }, + { fixture: "beta" }, + ); + const alpha = candidate("alpha", []); + const session = composeBinarySession( + new AnalysisProviderRegistry([ + { + ...alpha, + resolveAnalysisProfile: () => + Promise.resolve( + ok({ + profile: createAnalysisProfile( + { id: "alpha", name: "alpha provider", version: "1" }, + { revision }, + ), + }), + ), + }, + candidate("beta", []), + ]), + ); + const server = createServer({ kind: "session", session }); + const mcp = new Client({ name: "snapshot-mismatch", version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + resources.push(mcp, server, { close: () => session.close() }); + await server.connect(serverTransport); + await mcp.connect(clientTransport); + const call = (name: string, args: Record) => + mcp.callTool({ name, arguments: args }); + expect( + (await call("open_binary", { path: first, provider_id: "alpha" })).isError, + ).not.toBe(true); + expect( + (await call("close_binary", { snapshot_path: snapshotPath })).isError, + ).not.toBe(true); + expect( + (await call("open_binary", { path: first, provider_id: "alpha" })).isError, + ).not.toBe(true); + + const mismatch = async (path: string, provider: string) => { + const result = await call("open_binary", { + path, + provider_id: provider, + snapshot_path: snapshotPath, + }); + expect(result.isError).toBe(true); + const error = parseMcpToolError(result).error; + expect(error.code).toBe("evidence_integrity_mismatch"); + expect(error.remediation.action).toBe( + analysisErrorRemediationAction(new AnalysisSnapshotMismatchError("")), + ); + expect((await call("binary_session", {})).structuredContent).toMatchObject({ + result: { + open: true, + path: first, + analysis_provider_binding: { + analysis_profile: { digest: alphaProfile.digest }, + }, + }, + }); + return error.details?.binding_mismatch; + }; + expect(await mismatch(second, "alpha")).toEqual({ + target: { + snapshot: expect.objectContaining({ sha256: sha("first fixture") }), + selected: expect.objectContaining({ sha256: sha("second fixture") }), + }, + }); + const otherProvider = await mismatch(first, "beta"); + expect(otherProvider).toMatchObject({ + provider: { snapshot: { id: "alpha" }, selected: { id: "beta" } }, + profile_digest: { + snapshot: alphaProfile.digest, + selected: betaProfile.digest, + }, + }); + expect(otherProvider).not.toHaveProperty("target"); + revision = 2; + const otherProfile = await mismatch(first, "alpha"); + expect(otherProfile).toMatchObject({ + profile_digest: { + snapshot: expect.any(String), + selected: expect.any(String), + }, + }); + expect(otherProfile).not.toHaveProperty("provider"); + expect(otherProfile).not.toHaveProperty("target"); + const combined = await mismatch(second, "beta"); + expect(combined).toMatchObject({ + target: { + snapshot: { sha256: sha("first fixture") }, + selected: { sha256: sha("second fixture") }, + }, + provider: { snapshot: { id: "alpha" }, selected: { id: "beta" } }, + }); + expect((await call("address_name", { address: "0x10" })).isError).not.toBe( + true, + ); + + revision = 1; + const raw = join(testDirectory, "raw.bin"); + const rawSnapshot = join(testDirectory, "raw-snapshot.json"); + await writeFile(raw, Buffer.from([0, 1, 2, 3])); + const rawTarget = { + path: raw, + provider_id: "alpha", + format: "raw-binary", + architecture: "arm64", + }; + expect( + (await call("open_binary", { ...rawTarget, entry_point: "0x1000" })) + .isError, + ).not.toBe(true); + expect( + (await call("close_binary", { snapshot_path: rawSnapshot })).isError, + ).not.toBe(true); + const differentEntry = await call("open_binary", { + ...rawTarget, + entry_point: "0x2000", + snapshot_path: rawSnapshot, + }); + expect(differentEntry.isError).toBe(true); + expect(parseMcpToolError(differentEntry).error.details).toMatchObject({ + binding_mismatch: { + target: { + snapshot: { entry_point: "0x1000" }, + selected: { entry_point: "0x2000" }, + }, + }, + }); +}); + const candidate = ( id: string, starts: string[], @@ -571,6 +724,7 @@ const candidate = ( status: "unavailable", code: "not_configured", reason: `${id} is not configured`, + remediation: "Repair the fixture provider.", diagnostics: {}, }, inspectTargetSupport: () => ({ diff --git a/tests/boundary/mcp/sessionAvailabilityRecovery.test.ts b/tests/boundary/mcp/sessionAvailabilityRecovery.test.ts index 6686c95868..929379e5d4 100644 --- a/tests/boundary/mcp/sessionAvailabilityRecovery.test.ts +++ b/tests/boundary/mcp/sessionAvailabilityRecovery.test.ts @@ -1,14 +1,33 @@ -import { expect, it } from "vitest"; +import { expect, it, onTestFinished } from "vitest"; +import { + createAnalysisExecution, + type AnalysisExecution, + type AnalysisOperation, + type AnalysisProvider, + type CapabilityDescriptor, + type ProviderIdentity, +} from "../../../src/application/AnalysisProvider.js"; import { toolContract } from "../../../src/contracts/toolContracts.js"; import { expandToolAvailability } from "../../../src/contracts/toolOutputSchemaPrimitives.js"; -import { AnalysisTimeoutError } from "../../../src/domain/analysisErrorCore.js"; +import type { AnalysisError } from "../../../src/domain/analysisErrorBase.js"; +import { + AnalysisCancelledError, + AnalysisCapabilityUnavailableError, + AnalysisTimeoutError, +} from "../../../src/domain/analysisErrorCore.js"; +import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js"; +import { + inferredUnavailabilityReason, + runtimeFailureReason, +} from "../../../src/application/binary/SessionRuntimeAvailability.js"; import { ProviderAdapterError } from "../../../src/domain/providerAdapterError.js"; import { ProviderCleanupError } from "../../../src/domain/providerCleanupError.js"; -import { err, ok } from "../../../src/domain/result.js"; +import { err, ok, type Result } from "../../../src/domain/result.js"; import { silentLogger } from "../../../src/logger.js"; import { observed } from "../../fixtures/analysisExecution.js"; import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; +import { awaitAcknowledgement } from "../../fixtures/lifecycleAcknowledgement.js"; import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; import { createBinarySessionTargets, @@ -309,3 +328,529 @@ it("keeps provider tools available after one request reaches its deadline", asyn await close(); } }); + +type Reply = Result; +type Responder = (operation: string) => Promise; + +const FIXTURE = { + id: "fixture", + name: "Fixture analysis provider", + version: "1", +} as const satisfies ProviderIdentity; +const AUXILIARY = { + id: "fixture-auxiliary", + name: "Auxiliary fixture provider", + version: "1", +} as const satisfies ProviderIdentity; + +const liveCapability = ( + provider: ProviderIdentity, + operation: Exclude, + declaredRefusal?: string, +): CapabilityDescriptor => ({ + provider, + operation, + ...(declaredRefusal === undefined + ? { available: true as const, reason: null } + : { available: false as const, reason: declaredRefusal }), + cachePolicy: "live", + effects: { + mutatesArtifact: false, + launchesProcess: false, + mayShowUi: false, + mayAccessNetwork: false, + mayWriteFilesystem: false, + changesPermissions: false, + requiresRoot: false, + }, + limitations: [], +}); + +const adapterFailure = (provider: ProviderIdentity, operation: string) => + Promise.resolve(err(new ProviderAdapterError(provider.id, operation))); +const recordedReason = (failure: AnalysisError): string => { + const reason = runtimeFailureReason(failure); + if (reason === undefined) throw new Error(`${failure._tag} is not recorded`); + return reason; +}; +// Compare remediation with its owning formatters; the facts are their inputs. +const observedFailure = (operation: string, provider: string = FIXTURE.id) => + recordedReason(new ProviderAdapterError(provider, operation)); +const inferredFailure = (operation: string, provider: string = FIXTURE.id) => + inferredUnavailabilityReason(provider, { + operation, + reason: observedFailure(operation, provider), + }); +const refusal = (operation: string, reason: string) => + recordedReason( + new AnalysisCapabilityUnavailableError(FIXTURE.id, operation, reason), + ); + +/** Hold provider replies; release and join every launched call. */ +const heldCalls = ( + client: Awaited>["client"], +) => { + const held = new Map< + string, + { readonly reply: Promise; readonly received: () => void } + >(); + const launched: { + readonly release: (reply: Reply) => void; + readonly response: Promise; + }[] = []; + return { + /** Serve a held reply when one is registered for this operation. */ + respond: (operation: string): Promise | undefined => { + const entry = held.get(operation); + if (entry === undefined) return undefined; + held.delete(operation); + entry.received(); + return entry.reply; + }, + dispatch: async (name: string, arguments_: Record) => { + const reply = createDeferred(); + const received = createDeferred(); + held.set(name, { reply: reply.promise, received: received.resolve }); + const response = client.callTool({ name, arguments: arguments_ }); + launched.push({ release: reply.resolve, response }); + await awaitAcknowledgement(received.promise, response, name); + return { + release: reply.resolve, + isError: async () => (await response).isError === true, + }; + }, + /** Release every held reply and wait for its MCP response. */ + join: async () => { + for (const { release } of launched) release(observed("released")); + await Promise.allSettled(launched.map(({ response }) => response)); + }, + }; +}; + +/** Connect a deep fixture and an auxiliary provider with scripted replies. */ +const connectScripted = async (options: { + readonly operations: readonly Exclude[]; + readonly auxiliaryOperations?: readonly Exclude< + AnalysisOperation, + "health" + >[]; + readonly respond: { current: Responder }; + readonly respondAuxiliary?: { current: Responder }; + /** Operations the fixture declares unavailable, with their declared reason. */ + readonly declaredRefusals?: Readonly>; + readonly families?: string; +}) => { + const [path] = await createBinarySessionTargets(); + const provider: AnalysisProvider = { + identity: () => FIXTURE, + resolveAnalysisProfile: () => + Promise.resolve( + ok({ profile: createAnalysisProfile(FIXTURE, { fixture: true }) }), + ), + capabilities: () => + options.operations.map((operation) => + liveCapability( + FIXTURE, + operation, + options.declaredRefusals?.[operation], + ), + ), + createClient: () => ({ + execute: (operation) => + operation === "health" + ? Promise.resolve(observed(null)) + : options.respond.current(operation), + close: () => Promise.resolve(ok(null)), + }), + }; + const auxiliary: AnalysisProvider = { + identity: () => AUXILIARY, + capabilities: () => + (options.auxiliaryOperations ?? []).map((operation) => + liveCapability(AUXILIARY, operation), + ), + createClient: () => ({ + execute: (operation) => + options.respondAuxiliary?.current(operation) ?? + Promise.resolve(ok(createAnalysisExecution(null, AUXILIARY))), + close: () => Promise.resolve(ok(null)), + }), + }; + const session = createTestBinarySession(provider, { + auxiliaryProviders: [auxiliary], + }); + const { client, close } = await createApplicationMcpHarness({ + session, + clientIdentity: { name: "provider-recovery", version: "1" }, + serverOptions: { + environment: + options.families === undefined + ? {} + : { REA_MCP_TOOL_FAMILIES: options.families }, + logger: silentLogger, + }, + }); + onTestFinished(close); + expect( + (await client.callTool({ name: "open_binary", arguments: { path } })) + .isError, + ).not.toBe(true); + const call = async (name: string, arguments_: Record) => + (await client.callTool({ name, arguments: arguments_ })).isError === true + ? "error" + : "ok"; + const tools = async () => { + const reply = await client.callTool({ + name: "binary_session", + arguments: {}, + }); + return expandToolAvailability( + toolContract("binary_session").outputSchema.parse(reply.structuredContent) + .result.tool_availability, + ); + }; + const availability = async (names: readonly string[]) => { + const advertised = await tools(); + return Object.fromEntries( + names.map((name) => { + const tool = advertised.find((entry) => entry.name === name); + if (tool === undefined) throw new Error(`${name} is not advertised`); + return [name, tool.available ? "available" : tool.remediation]; + }), + ); + }; + return { client, call, availability, tools }; +}; + +const DIRECT_OPERATIONS = [ + "address_name", + "current_address", + "list_procedures", + "procedure_assembly", + "procedure_info", +] as const; +const ADDRESS = { address: "0x1000", document: "fixture" }; +const PROCEDURE = { procedure: "main" }; + +it("returns provider-inferred siblings to declared availability after one success", async () => { + let broken = false; + const respond = { + current: (operation: string): Promise => { + if (operation === "procedure_info") + return Promise.resolve( + err( + new AnalysisCapabilityUnavailableError( + FIXTURE.id, + operation, + "fixture project is read-only", + ), + ), + ); + if (broken) return adapterFailure(FIXTURE, operation); + return Promise.resolve(observed("fixture-text")); + }, + }; + const declared = "Fixture project was opened read-only."; + const { call, availability } = await connectScripted({ + operations: [...DIRECT_OPERATIONS, "set_comment"], + auxiliaryOperations: ["list_documents"], + declaredRefusals: { set_comment: declared }, + respond, + }); + const refused = refusal("procedure_info", "fixture project is read-only"); + expect(await call("procedure_info", PROCEDURE)).toBe("error"); + broken = true; + expect(await call("address_name", ADDRESS)).toBe("error"); + expect(await call("procedure_assembly", PROCEDURE)).toBe("error"); + const names = [...DIRECT_OPERATIONS, "set_comment", "list_documents"]; + expect(await availability(names)).toEqual({ + address_name: observedFailure("address_name"), + current_address: inferredFailure("procedure_assembly"), + list_procedures: inferredFailure("procedure_assembly"), + procedure_assembly: observedFailure("procedure_assembly"), + // Separately observed and declared refusals keep their own reasons. + procedure_info: refused, + set_comment: declared, + list_documents: "available", + }); + + broken = false; + expect(await call("address_name", ADDRESS)).toBe("ok"); + // One success is evidence the provider answers again, not that every + // operation is healthy: failures observed on other operations remain. + expect(await availability(names)).toEqual({ + address_name: "available", + current_address: "available", + list_procedures: "available", + procedure_assembly: observedFailure("procedure_assembly"), + procedure_info: refused, + set_comment: declared, + list_documents: "available", + }); + expect(await call("procedure_assembly", PROCEDURE)).toBe("ok"); + expect(await availability(["procedure_assembly", "procedure_info"])).toEqual({ + procedure_assembly: "available", + procedure_info: refused, + }); +}); + +it("keeps each provider's failures and ignores cancelled requests", async () => { + let fixtureBroken = false; + let auxiliaryBroken = true; + const respond = { + current: (operation: string): Promise => { + if (operation === "list_procedures") + return Promise.resolve(err(new AnalysisCancelledError(operation))); + return fixtureBroken + ? adapterFailure(FIXTURE, operation) + : Promise.resolve(observed("fixture-text")); + }, + }; + const respondAuxiliary = { + current: (operation: string): Promise => + auxiliaryBroken + ? adapterFailure(AUXILIARY, operation) + : Promise.resolve(ok(createAnalysisExecution("doc", AUXILIARY))), + }; + const { call, availability } = await connectScripted({ + operations: ["address_name", "current_address", "list_procedures"], + auxiliaryOperations: ["current_document", "list_documents"], + respond, + respondAuxiliary, + }); + const names = [ + "address_name", + "current_address", + "list_procedures", + "current_document", + "list_documents", + ]; + expect(await call("list_documents", {})).toBe("error"); + expect(await call("address_name", ADDRESS)).toBe("ok"); + const auxiliaryFailure = { + current_document: inferredFailure("list_documents", AUXILIARY.id), + list_documents: observedFailure("list_documents", AUXILIARY.id), + }; + // Another provider's success does not clear the auxiliary provider's state. + expect(await availability(names)).toEqual({ + address_name: "available", + current_address: "available", + list_procedures: "available", + ...auxiliaryFailure, + }); + + fixtureBroken = true; + expect(await call("address_name", ADDRESS)).toBe("error"); + // A cancelled request is not evidence that the provider recovered. + expect(await call("list_procedures", {})).toBe("error"); + const fixtureFailure = { + address_name: observedFailure("address_name"), + current_address: inferredFailure("address_name"), + list_procedures: inferredFailure("address_name"), + }; + expect(await availability(names)).toEqual({ + ...fixtureFailure, + ...auxiliaryFailure, + }); + + auxiliaryBroken = false; + expect(await call("current_document", {})).toBe("ok"); + expect(await availability(names)).toEqual({ + ...fixtureFailure, + current_document: "available", + list_documents: auxiliaryFailure.list_documents, + }); +}); + +it("keeps newer failures when older successes complete after them", async () => { + let reply: Responder = () => Promise.resolve(observed("text")); + const respond = { current: (operation: string) => reply(operation) }; + const { client, call, availability } = await connectScripted({ + operations: [...DIRECT_OPERATIONS, "current_procedure"], + respond, + }); + const calls = heldCalls(client); + respond.current = (operation) => calls.respond(operation) ?? reply(operation); + const document = { document: "doc" }; + const refuse: Responder = (operation) => + Promise.resolve( + err( + new AnalysisCapabilityUnavailableError( + FIXTURE.id, + operation, + "fixture refusal", + ), + ), + ); + try { + // Successes dispatched before an own refusal, before a provider failure, + // or between the two complete after both. + const olderAddress = await calls.dispatch("address_name", ADDRESS); + const olderCurrent = await calls.dispatch("current_address", document); + reply = refuse; + expect(await call("address_name", ADDRESS)).toBe("error"); + expect(await call("current_procedure", document)).toBe("error"); + const between = await calls.dispatch("current_procedure", document); + reply = (operation) => adapterFailure(FIXTURE, operation); + expect(await call("procedure_assembly", PROCEDURE)).toBe("error"); + olderAddress.release(observed("old-name")); + olderCurrent.release(observed("0x1000")); + between.release(observed("main")); + for (const settled of [olderAddress, olderCurrent, between]) + expect(await settled.isError()).toBe(false); + expect( + await availability([ + "address_name", + "current_address", + "current_procedure", + "list_procedures", + ]), + ).toEqual({ + address_name: refusal("address_name", "fixture refusal"), + current_address: inferredFailure("procedure_assembly"), + // Its own later success superseded the refusal, not the provider failure. + current_procedure: inferredFailure("procedure_assembly"), + list_procedures: inferredFailure("procedure_assembly"), + }); + + // A success dispatched behind a failure executed after it, even though + // the failure was not yet observed when the success was dispatched. + reply = () => Promise.resolve(observed("text")); + const failure = await calls.dispatch("procedure_assembly", PROCEDURE); + const queued = await calls.dispatch("current_address", document); + failure.release( + err(new ProviderAdapterError(FIXTURE.id, "procedure_assembly")), + ); + expect(await failure.isError()).toBe(true); + queued.release(observed("0x1000")); + expect(await queued.isError()).toBe(false); + expect( + await availability([ + "address_name", + "current_address", + "current_procedure", + "list_procedures", + "procedure_assembly", + ]), + ).toEqual({ + address_name: refusal("address_name", "fixture refusal"), + current_address: "available", + current_procedure: "available", + list_procedures: "available", + procedure_assembly: observedFailure("procedure_assembly"), + }); + } finally { + await calls.join(); + } +}); + +it("keeps newer successes when an older failure completes last", async () => { + const respond = { + current: (_operation: string): Promise => + Promise.resolve(observed("text")), + }; + const { client, call, availability } = await connectScripted({ + operations: DIRECT_OPERATIONS, + respond, + }); + const calls = heldCalls(client); + respond.current = (operation) => + calls.respond(operation) ?? Promise.resolve(observed("text")); + const names = ["address_name", "current_address", "list_procedures"]; + try { + // A sibling dispatched later already proved the provider answers. + const older = await calls.dispatch("address_name", ADDRESS); + const newer = await calls.dispatch("current_address", { document: "doc" }); + newer.release(observed("0x1000")); + expect(await newer.isError()).toBe(false); + older.release(err(new ProviderAdapterError(FIXTURE.id, "address_name"))); + expect(await older.isError()).toBe(true); + expect(await availability(names)).toEqual({ + address_name: observedFailure("address_name"), + current_address: "available", + list_procedures: "available", + }); + + // A later success of the same operation supersedes its older failure. + const stale = await calls.dispatch("procedure_assembly", PROCEDURE); + expect(await call("procedure_assembly", PROCEDURE)).toBe("ok"); + stale.release( + err(new ProviderAdapterError(FIXTURE.id, "procedure_assembly")), + ); + expect(await stale.isError()).toBe(true); + expect(await availability([...names, "procedure_assembly"])).toEqual({ + address_name: observedFailure("address_name"), + current_address: "available", + list_procedures: "available", + procedure_assembly: "available", + }); + } finally { + await calls.join(); + } +}); + +it("restores composed tools in a scoped catalog after one composed success", async () => { + let broken = true; + const respond = { + current: (operation: string): Promise => { + if (broken) return adapterFailure(FIXTURE, operation); + if (operation === "procedure_address") + return Promise.resolve(observed("0x1")); + if (operation === "address_name") + return Promise.resolve(observed("fixture_name")); + return Promise.resolve(observed("return 0;")); + }, + }; + const { client, availability, tools } = await connectScripted({ + operations: [ + "address_name", + "procedure_address", + "procedure_callees", + "procedure_callers", + "procedure_pseudo_code", + "search_procedures", + "search_strings", + "xrefs", + ], + respond, + families: "session,enhanced", + }); + // Direct prerequisites are outside this scope; only composed tools can retry. + const batchItemStatus = async () => { + const reply = await client.callTool({ + name: "batch_decompile", + arguments: { addresses: ["0x1"] }, + }); + expect(reply.isError).not.toBe(true); + return reply.structuredContent; + }; + expect(await batchItemStatus()).toMatchObject({ + normalized_result: { items: [{ status: "error" }] }, + }); + expect(await availability(["trace_feature", "get_call_graph"])).toEqual({ + // The batch still decompiles the unresolved selector; its failure is newest. + trace_feature: inferredFailure("procedure_pseudo_code"), + get_call_graph: observedFailure("procedure_address"), + }); + + broken = false; + expect(await batchItemStatus()).toMatchObject({ + normalized_result: { items: [{ status: "ok", pseudocode: "return 0;" }] }, + }); + expect( + await availability(["batch_decompile", "trace_feature", "get_call_graph"]), + ).toEqual({ + batch_decompile: "available", + trace_feature: "available", + get_call_graph: "available", + }); + expect( + (await tools()).find(({ name }) => name === "get_call_graph"), + ).toMatchObject({ + default_mode_available: true, + modes: [ + { name: "forward", available: true, missing_operations: [] }, + { name: "backward", available: true, missing_operations: [] }, + ], + }); +}); diff --git a/tests/boundary/mcp/sessionMcp.test.ts b/tests/boundary/mcp/sessionMcp.test.ts index 2dbe57ee55..5001a9ae23 100644 --- a/tests/boundary/mcp/sessionMcp.test.ts +++ b/tests/boundary/mcp/sessionMcp.test.ts @@ -180,7 +180,8 @@ describe("public provider availability", () => { const unavailable: ProviderAvailability = { status: "unavailable", code: "unsupported_version", - reason: "Use Apktool 2.7.0 or newer.", + reason: "Apktool 2.6.3 is older than the supported minimum 2.7.0.", + remediation: "Use Apktool 2.7.0 or newer.", diagnostics: { minimum_version: "2.7.0" }, }; const adbDeviceAnalysis: AdbDeviceAnalysisPort = { @@ -222,7 +223,11 @@ describe("public provider availability", () => { availability.groups.find(({ names }) => names.includes("inspect_apktool_client"), ), - ).toMatchObject({ available: false, reason: "provider_unavailable" }); + ).toMatchObject({ + available: false, + reason: "provider_unavailable", + remediation: "Use Apktool 2.7.0 or newer.", + }); }); }); @@ -284,8 +289,10 @@ describe("owned-process prerequisite during status discovery", () => { available: false, reason: "provider_unavailable", }); - expect(apktool?.remediation).toContain("procps-compatible ps"); - expect(apktool?.remediation).toContain("spawn ps ENOENT"); + // Discovery carries the prerequisite owner's repair, not its failure text. + expect(apktool?.remediation).toBe( + "Install your distribution's procps or procps-ng package, or put a procps-compatible ps on REA's PATH, then retry.", + ); expect( availability.groups.find(({ names }) => names.includes("inspect_adb_client"), diff --git a/tests/boundary/mcp/sessionMcpHarness.ts b/tests/boundary/mcp/sessionMcpHarness.ts index 28dcd12596..4a5312a38c 100644 --- a/tests/boundary/mcp/sessionMcpHarness.ts +++ b/tests/boundary/mcp/sessionMcpHarness.ts @@ -34,6 +34,7 @@ const HOST_TOOLS_NOT_CONFIGURED: ProviderAvailability = { status: "unavailable", code: "not_configured", reason: "The target-free MCP fixture does not probe host tools.", + remediation: "Repair the fixture provider.", diagnostics: { source: "test_fixture" }, }; diff --git a/tests/boundary/mcp/toolFamiliesMcp.test.ts b/tests/boundary/mcp/toolFamiliesMcp.test.ts index 885dcbd3ce..ed8661c405 100644 --- a/tests/boundary/mcp/toolFamiliesMcp.test.ts +++ b/tests/boundary/mcp/toolFamiliesMcp.test.ts @@ -191,6 +191,7 @@ describe("MCP tool family advertising", () => { status: "unavailable", code: "runtime_missing", reason: `${name} runtime is unavailable`, + remediation: "Repair the fixture provider.", diagnostics: {}, } as const; }; @@ -231,6 +232,7 @@ describe("MCP tool family advertising", () => { status: "unavailable", code: "runtime_missing", reason: "android runtime is unavailable", + remediation: "Repair the fixture provider.", diagnostics: {}, }), }); diff --git a/tests/boundary/mcp/toolResult.test.ts b/tests/boundary/mcp/toolResult.test.ts index 1ee63a03aa..95bbe231b0 100644 --- a/tests/boundary/mcp/toolResult.test.ts +++ b/tests/boundary/mcp/toolResult.test.ts @@ -372,7 +372,7 @@ describe("tool result projection", () => { .details, ).not.toHaveProperty("launch_rejection"); }); - it("projects bounded private-display coordinates without raw stderr", () => { + it("projects private-display diagnostic fields and stderr byte count", () => { const result = delivery.toCallToolResult( err( new HopperProcessError(80, { @@ -405,6 +405,7 @@ describe("tool result projection", () => { mount_read_only: true, wsl: true, strategy: "user-mount-namespace", + xvfb_stderr_bytes: 512, }, }, }, diff --git a/tests/boundary/mcp/universalSliceMcp.test.ts b/tests/boundary/mcp/universalSliceMcp.test.ts index 9fab37a6ec..1009f3c1ad 100644 --- a/tests/boundary/mcp/universalSliceMcp.test.ts +++ b/tests/boundary/mcp/universalSliceMcp.test.ts @@ -355,6 +355,7 @@ sliceTest( ...thinReport.auditable_metadata, file_offset: (thinReport.auditable_metadata?.file_offset ?? 0) + offset, }); + expect(thinReport.rustc_commit?.hashes.length).toBeGreaterThan(0); expect(report.rustc_commit?.hashes).toEqual( thinReport.rustc_commit?.hashes.map((entry) => ({ ...entry, diff --git a/tests/boundary/model/modelArtifact.test.ts b/tests/boundary/model/modelArtifact.test.ts index be6aef9dfc..f181bb6a62 100644 --- a/tests/boundary/model/modelArtifact.test.ts +++ b/tests/boundary/model/modelArtifact.test.ts @@ -1233,7 +1233,7 @@ it("reports a truncated safetensors header whose length resembles a pickle marke }); }); -it("bounds expansion before allocation and distinguishes unsupported pickle, version and paths", async () => { +it("bounds expansion before allocation and distinguishes unsupported pickle and paths", async () => { const fixture = ggufFixture(); fixture.bytes.writeBigUInt64LE(0xffffffffffffffffn, 8); const { provider, path, root } = await providerAt(fixture.bytes); @@ -1245,9 +1245,6 @@ it("bounds expansion before allocation and distinguishes unsupported pickle, ver if (pickle.ok) throw new Error("Pickle admitted"); expect(pickle.error._tag).toBe("AnalysisUnsupportedTargetError"); await writeFile(path, ggufFixture({ version: 999 }).bytes); - const version = await provider.inspect({ path, hash: "header" }); - if (version.ok) throw new Error("Unknown version admitted"); - expect(version.error._tag).toBe("AnalysisUnsupportedTargetError"); const alias = join(root, "link"); await symlink(path, alias); const linked = await provider.inspect({ path: alias, hash: "header" }); diff --git a/tests/boundary/pcap/pcapProvider.test.ts b/tests/boundary/pcap/pcapProvider.test.ts index d690e3e62d..e9ac466a76 100644 --- a/tests/boundary/pcap/pcapProvider.test.ts +++ b/tests/boundary/pcap/pcapProvider.test.ts @@ -7,6 +7,7 @@ import { withVerifierCleanup } from "../../../scripts/lib/verifier-cleanup.mjs"; import { awaitAcknowledgement } from "../../fixtures/lifecycleAcknowledgement.js"; import { PcapProvider } from "../../../src/pcap/PcapProvider.js"; +import { PCAP_LIMITER_REMEDIATION } from "../../../src/pcap/PcapNativeBound.js"; import { CAPINFOS_COMMAND_ENV, TSHARK_COMMAND_ENV, @@ -468,6 +469,12 @@ it("names the prlimit selection when the Linux limiter is unusable", async () => }, }); onTestFinished(() => provider.close()); + // Readiness keeps the limiter owner's repair instead of Wireshark advice. + expect(await provider.inspectAvailability()).toMatchObject({ + status: "unavailable", + reason: expect.stringContaining("Cannot establish the selected Linux"), + remediation: PCAP_LIMITER_REMEDIATION, + }); const result = await provider.execute({ operation: "inspect_pcap_capture", input: { path: "/not-opened/capture.pcap" }, @@ -480,6 +487,7 @@ it("names the prlimit selection when the Linux limiter is unusable", async () => }); expect(projected.message).toContain("must be absolute"); expect(projected.message).toContain("Check that REA_TSHARK_PRLIMIT_COMMAND"); + expect(projected.remediation.action).toBe(PCAP_LIMITER_REMEDIATION); }); it("rejects directory and non-prlimit selections before opening the capture", async () => { diff --git a/tests/boundary/process/adbVerifierCleanup.test.ts b/tests/boundary/process/adbVerifierCleanup.test.ts index 0a8db02b3b..1b56fb29c1 100644 --- a/tests/boundary/process/adbVerifierCleanup.test.ts +++ b/tests/boundary/process/adbVerifierCleanup.test.ts @@ -21,6 +21,7 @@ afterEach(async () => { const runVerifier = async ( settings: { + args?: string[]; closeFails?: boolean; thrown?: "undefined" | "null"; removal?: "fails" | "leaves_probe"; @@ -63,7 +64,8 @@ const runVerifier = async ( await writeFile( join(root, "dist", "composition", "adb.js"), `import { writeFile } from "node:fs/promises"; -export const createAdbDeviceAnalysisProvider = () => ({ +import { writeFileSync } from "node:fs"; +export const createAdbDeviceAnalysisProvider = () => (writeFileSync(process.env.REA_ADB_CREATE_MARKER, "created"), { async close() { await writeFile(process.env.REA_ADB_CLOSE_MARKER, "closed"); if (process.env.REA_ADB_CLOSE_FAIL === "1") { @@ -112,16 +114,18 @@ export class AdbDeviceAnalysisService { }\n`, ); const marker = join(root, "provider-closed"); + const createdMarker = join(root, "provider-created"); const adbLog = join(root, "adb.log"); const run = spawnSync( process.execPath, - [join(root, "scripts", "verify-adb.mjs")], + [join(root, "scripts", "verify-adb.mjs"), ...(settings.args ?? [])], { encoding: "utf8", timeout: 10_000, env: { ...process.env, REA_ADB_CLOSE_MARKER: marker, + REA_ADB_CREATE_MARKER: createdMarker, REA_ADB_CLOSE_FAIL: settings.closeFails ? "1" : "0", REA_FAKE_ADB: adb, REA_FAKE_ADB_LOG: adbLog, @@ -135,11 +139,18 @@ export class AdbDeviceAnalysisService { }, ); const closed = await readFile(marker, "utf8").catch(() => null); + const created = await readFile(createdMarker, "utf8").catch(() => null); const adbCalls = await readFile(adbLog, "utf8").then( (text) => text.trim().split("\n"), () => [], ); - return { ...run, closed, adbCalls, deviceFiles: await readdir(device) }; + return { + ...run, + created, + closed, + adbCalls, + deviceFiles: await readdir(device), + }; }; // The fake adb client is a POSIX shell script. @@ -251,3 +262,19 @@ it("reports both an unexpected verifier throw and provider cleanup failure", asy expect(result.stderr).toContain("fixture/private-root"); expect(result.closed).toBe("closed"); }); + +it.each([["--install-apk", "/tmp/unrelated.apk"], ["--install-apk"]])( + "rejects %s before contacting the device or acquiring provider resources", + async (...args) => { + const result = await runVerifier({ args }); + + expect(result.status).toBe(1); + expect(result.stderr).toContain( + "APK identity and device cleanup ownership cannot be established safely", + ); + expect(result.created).toBeNull(); + expect(result.closed).toBeNull(); + expect(result.adbCalls).toEqual([]); + expect(result.deviceFiles).toEqual([]); + }, +); diff --git a/tests/boundary/process/applicationWorkflowVerifier.test.ts b/tests/boundary/process/applicationWorkflowVerifier.test.ts index f420e817ca..2e10cd563d 100644 --- a/tests/boundary/process/applicationWorkflowVerifier.test.ts +++ b/tests/boundary/process/applicationWorkflowVerifier.test.ts @@ -1,5 +1,5 @@ import { execFile } from "node:child_process"; -import { copyFile, mkdir, rm } from "node:fs/promises"; +import { copyFile, mkdir, readFile, rm, writeFile } from "node:fs/promises"; import { join } from "node:path"; import { promisify } from "node:util"; @@ -107,3 +107,65 @@ describe("local application workflow verifier", () => { }); }, 20_000); }); + +it.each([false, true])( + "joins sibling analysis cleanup after a first failure (cleanup fails: %s)", + async (cleanupFails) => { + const root = await createTestTempDirectory( + "rea-application-verifier-cleanup-", + ); + const left = join(root, "left"); + const right = join(root, "right"); + const marker = join(root, "cleanup-completed"); + const preload = join(root, "instrument.mjs"); + await Promise.all([mkdir(left), mkdir(right)]); + const fake = ` +import { writeFile } from 'node:fs/promises'; +export async function analyzeJavaScriptApplicationWithOwnedScope({input_path}) { + if (input_path === ${JSON.stringify(left)}) throw new Error('authored first failure'); + await new Promise(resolve => setTimeout(resolve, 200)); + await writeFile(${JSON.stringify(marker)}, 'closed'); + if (${JSON.stringify(cleanupFails)}) throw new Error('authored late cleanup failure'); + return {}; +} +`; + await writeFile( + preload, + ` +import { registerHooks } from 'node:module'; +registerHooks({ resolve(specifier, context, next) { + if (specifier === '../../lib/javascript-application-analysis.mjs' && context.parentURL?.endsWith('/scripts/verify/javascript/application-workflows.mjs')) + return { shortCircuit: true, url: 'data:text/javascript,' + encodeURIComponent(${JSON.stringify(fake)}) }; + return next(specifier, context); +}}); +`, + ); + const result: unknown = await execute( + process.execPath, + [ + "--import", + preload, + "scripts/verify/javascript/application-workflows.mjs", + "--left", + left, + "--right", + right, + ], + { timeout: 10_000 }, + ).then( + () => { + throw new Error("Expected analysis failure"); + }, + (error: unknown) => error, + ); + expect(await readFile(marker, "utf8")).toBe("closed"); + expect(result).toMatchObject({ + stderr: expect.stringContaining("authored first failure"), + }); + if (cleanupFails) + expect(result).toMatchObject({ + stderr: expect.stringContaining("authored late cleanup failure"), + }); + }, + 20_000, +); diff --git a/tests/boundary/process/bunVerifierOwnerInitialization.test.ts b/tests/boundary/process/bunVerifierOwnerInitialization.test.ts new file mode 100644 index 0000000000..68c711ebac --- /dev/null +++ b/tests/boundary/process/bunVerifierOwnerInitialization.test.ts @@ -0,0 +1,80 @@ +import { spawnSync } from "node:child_process"; +import { mkdtemp, mkdir, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { expect, it } from "vitest"; + +it.each([ + ["initialization failure", "init", true], + ["initialization and root cleanup failures", "init_close", false], + ["command failure", "run", true], + ["command cleanup failure", "close", false], +] as const)( + "retains the verifier root correctly after %s", + async (_name, failure, rootClosed) => { + const root = await mkdtemp(join(tmpdir(), "rea-bun-owner-test-")); + try { + await mkdir(join(root, "scripts", "lib"), { recursive: true }); + await mkdir(join(root, "dist", "process"), { recursive: true }); + await mkdir(join(root, "dist", "domain"), { recursive: true }); + await writeFile( + join(root, "scripts", "verify-bun-standalone.mjs"), + await readFile( + new URL( + "../../../scripts/verify-bun-standalone.mjs", + import.meta.url, + ), + ), + ); + await writeFile( + join(root, "scripts", "lib", "verifier-cleanup.mjs"), + await readFile( + new URL("../../../scripts/lib/verifier-cleanup.mjs", import.meta.url), + ), + ); + await writeFile( + join(root, "scripts", "lib", "verify-commands.mjs"), + `export async function createVerifierCommands() { + if (process.env.FAULT.startsWith("init")) throw new Error("owner load refused"); + return { async exec() { throw new Error("command refused"); }, async close() { + if (process.env.FAULT === "close") throw new Error("owner close refused"); + } }; + }`, + ); + await writeFile( + join(root, "dist", "process", "PrivateRuntimeRoot.js"), + `import { writeFile } from "node:fs/promises"; + export class PrivateRuntimeRoot { static async create() { return { + path: process.env.ROOT, async close() { if (process.env.FAULT === "init_close") throw new Error("root close refused"); await writeFile(process.env.CLOSED, "closed"); } + }; } }`, + ); + await writeFile( + join(root, "dist", "domain", "evidence.js"), + `export function parseEvidence() { throw new Error("unexpected"); }`, + ); + const marker = join(root, "closed"); + const result = spawnSync( + process.execPath, + [join(root, "scripts", "verify-bun-standalone.mjs")], + { + encoding: "utf8", + timeout: 10_000, + env: { ...process.env, ROOT: root, CLOSED: marker, FAULT: failure }, + }, + ); + expect(result.status).not.toBe(0); + expect(result.stderr).toContain( + failure.startsWith("init") ? "owner load refused" : "command refused", + ); + if (failure === "close") + expect(result.stderr).toContain("owner close refused"); + if (failure === "init_close") + expect(result.stderr).toContain("root close refused"); + expect(await readFile(marker, "utf8").catch(() => null)).toBe( + rootClosed ? "closed" : null, + ); + } finally { + await rm(root, { recursive: true, force: true }); + } + }, +); diff --git a/tests/boundary/process/claudeVerifierSetupTargets.test.ts b/tests/boundary/process/claudeVerifierSetupTargets.test.ts new file mode 100644 index 0000000000..ae065868fe --- /dev/null +++ b/tests/boundary/process/claudeVerifierSetupTargets.test.ts @@ -0,0 +1,98 @@ +import { spawnSync } from "node:child_process"; +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { expect, it } from "vitest"; + +it.each( + ( + [ + ["claude", "relative path", false], + ["claude", "sibling", false], + ["claude", "parent traversal", false], + ["claude", "owned child", true], + ["opencode", "relative path", false], + ["opencode", "sibling", false], + ["opencode", "parent traversal", false], + ["opencode", "owned child", true], + ] as const + ).filter(([client]) => process.platform !== "win32" || client !== "opencode"), +)( + "%s checks a %s setup-plan target before writing", + async (client, kind, allowed) => { + const state = await mkdtemp(join(tmpdir(), "rea-claude-plan-")); + try { + const commands = join(state, "commands.mjs"); + const fixture = join(state, "fixture.mjs"); + const hook = join(state, "hook.mjs"); + await writeFile( + commands, + `import { writeFile } from 'node:fs/promises'; +import { isAbsolute, join, relative } from 'node:path'; +export async function createVerifierCommands(){return { + async exec(command,args,options={}){ + if(command==='git')return {stdout:'',stderr:''}; + if(args[0]==='--version')return {stdout:'fake client',stderr:''}; + if(args.includes('setup') && args.includes('--dry-run')){ + const client=process.env.CLIENT; const profile=client==='claude'?options.env.CLAUDE_CONFIG_DIR:options.env.OPENCODE_CONFIG_DIR; const skill=client==='claude'?join(profile,'skills','reverse-engineer-anything'):join(options.env.USERPROFILE,'.agents','skills','reverse-engineer-anything'); + const candidate=process.env.KIND==='relative path' ? relative(process.cwd(),join(skill,'AGENTS.md')) : process.env.KIND==='sibling' ? skill+'-sibling/AGENTS.md' : process.env.KIND==='parent traversal' ? skill+'/../outside/AGENTS.md' : skill+'/AGENTS.md'; + if (process.env.KIND==='relative path' && (candidate==='' || isAbsolute(candidate))) throw Error('Fixture expected a relative target'); + return {stdout:JSON.stringify({status:'planned',plannedActions:[{target:join(profile,client==='claude'?'.claude.json':'opencode.jsonc')},{target:skill},{target:candidate}]}),stderr:''}; + } + await writeFile(process.env.STATE+'/setup-advanced','yes'); throw Error('intentional stop before setup mutation'); + }, async close(){} };}`, + ); + await writeFile( + fixture, + `export async function createAnthropicModelFixture(){return {baseUrl:'http://127.0.0.1:1',async close(){}};} export const createOpenAiModelFixture=createAnthropicModelFixture;`, + ); + await writeFile( + hook, + `import {registerHooks} from 'node:module';import{pathToFileURL}from'node:url'; +registerHooks({resolve(specifier,context,next){if(context.parentURL?.endsWith('/scripts/verify-'+process.env.CLIENT+'-client.mjs')){let f;if(specifier==='./lib/verify-commands.mjs')f='commands.mjs';if(specifier==='./verify/clients/anthropic-model-fixture.mjs'||specifier==='./verify/clients/model-fixture.mjs')f='fixture.mjs';if(f)return{url:pathToFileURL(process.env.STATE+'/'+f).href,shortCircuit:true};}return next(specifier,context);}});`, + ); + const result = spawnSync( + process.execPath, + [ + "--import", + hook, + fileURLToPath( + new URL( + `../../../scripts/verify-${client}-client.mjs`, + import.meta.url, + ), + ), + ], + { + encoding: "utf8", + cwd: state, + timeout: 10_000, + env: { + ...process.env, + STATE: state, + KIND: kind, + CLIENT: client, + TMPDIR: state, + TMP: state, + TEMP: state, + }, + }, + ); + expect(result.status).not.toBe(0); + const advanced = await readFile( + join(state, "setup-advanced"), + "utf8", + ).then( + () => true, + () => false, + ); + expect(advanced).toBe(allowed); + expect(result.stderr).toContain( + allowed ? "intentional stop before setup mutation" : "AssertionError", + ); + } finally { + await rm(state, { recursive: true, force: true }); + } + }, +); diff --git a/tests/boundary/process/clientVerifierSetupTargets.test.ts b/tests/boundary/process/clientVerifierSetupTargets.test.ts new file mode 100644 index 0000000000..eb2cd6357d --- /dev/null +++ b/tests/boundary/process/clientVerifierSetupTargets.test.ts @@ -0,0 +1,103 @@ +import { spawnSync } from "node:child_process"; +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { expect, it } from "vitest"; + +it.each( + ( + [ + ["gemini", "relative path", false], + ["gemini", "profile traversal", false], + ["gemini", "profile root alias", false], + ["gemini", "skill root alias", false], + ["gemini", "skill traversal", false], + ["gemini", "owned child", true], + ["copilot", "relative path", false], + ["copilot", "sibling", false], + ["copilot", "skill traversal", false], + ["copilot", "owned child", true], + ["qwen", "relative path", false], + ["qwen", "profile traversal", false], + ["qwen", "profile root alias", false], + ["qwen", "skill root alias", false], + ["qwen", "skill traversal", false], + ["qwen", "profile child", true], + ["qwen", "skill child", true], + ] as const + ).filter(([client]) => process.platform !== "win32" || client !== "copilot"), +)( + "%s checks %s setup-plan targets before writing", + async (client, kind, allowed) => { + const state = await mkdtemp(join(tmpdir(), "rea-client-plan-")); + try { + const commands = join(state, "commands.mjs"); + const fixture = join(state, "fixture.mjs"); + const hook = join(state, "hook.mjs"); + await writeFile( + commands, + `import {writeFile} from 'node:fs/promises'; import {join} from 'node:path'; +export async function createVerifierCommands(){return { + async exec(command,args,options={}){ + if(command==='git')return {stdout:'',stderr:''}; + if(args.includes('--version'))return {stdout:'fake client',stderr:''}; + if(args.includes('setup') && args.includes('--dry-run')){ + const client=process.env.CLIENT; const kind=process.env.KIND; + const profile=client==='gemini'?options.env.GEMINI_CLI_HOME:client==='copilot'?options.env.COPILOT_HOME:options.env.QWEN_HOME; + const skills=join(client==='gemini'?profile:options.env.HOME,'.agents','skills'); + const skill=join(skills,'reverse-engineer-anything'); + const config=join(profile,client==='gemini'?'.gemini/settings.json':client==='copilot'?'mcp-config.json':'settings.json'); + const candidate=kind==='relative path'?'relative/AGENTS.md':kind==='profile root alias'?profile+'/child/..':kind==='skill root alias'?skills+'/child/..':kind==='profile traversal'?profile+'/../outside/AGENTS.md':kind==='skill traversal'?(client==='copilot'?skill:skills)+'/../outside/AGENTS.md':kind==='sibling'?skill+'-sibling/AGENTS.md':kind==='profile child'?config:join(skill,'SKILL.md'); + const selectedSkill=(kind==='skill traversal'||kind==='skill root alias')&&client==='gemini'?candidate:join(skill,'SKILL.md'); + const actions=client==='qwen'?[{target:candidate}]:[{target:config},{target:client==='copilot'?skill:selectedSkill},{target:candidate}]; + return {stdout:JSON.stringify({status:'planned',plannedActions:actions}),stderr:''}; + } + await writeFile(process.env.STATE+'/setup-advanced','yes'); throw Error('intentional stop before setup mutation'); + }, async close(){} };}`, + ); + await writeFile( + fixture, + `export async function createOpenAiModelFixture(){return {baseUrl:'http://127.0.0.1:1',async close(){}};}`, + ); + await writeFile( + hook, + `import {registerHooks} from 'node:module';import{pathToFileURL}from'node:url'; +registerHooks({resolve(specifier,context,next){if(context.parentURL?.endsWith('/scripts/verify-'+process.env.CLIENT+'-client.mjs')){let f;if(specifier==='./lib/verify-commands.mjs')f='commands.mjs';if(specifier==='./verify/clients/model-fixture.mjs')f='fixture.mjs';if(f)return{url:pathToFileURL(process.env.STATE+'/'+f).href,shortCircuit:true};}return next(specifier,context);}});`, + ); + const script = fileURLToPath( + new URL( + `../../../scripts/verify-${client}-client.mjs`, + import.meta.url, + ), + ); + const result = spawnSync(process.execPath, ["--import", hook, script], { + encoding: "utf8", + timeout: 10_000, + env: { + ...process.env, + STATE: state, + CLIENT: client, + KIND: kind, + TMPDIR: state, + TMP: state, + TEMP: state, + }, + }); + expect(result.status).not.toBe(0); + const advanced = await readFile( + join(state, "setup-advanced"), + "utf8", + ).then( + () => true, + () => false, + ); + expect(advanced).toBe(allowed); + expect(result.stderr).toContain( + allowed ? "intentional stop before setup mutation" : "AssertionError", + ); + } finally { + await rm(state, { recursive: true, force: true }); + } + }, +); diff --git a/tests/boundary/process/commandcodeVerifierSkillBody.test.ts b/tests/boundary/process/commandcodeVerifierSkillBody.test.ts new file mode 100644 index 0000000000..457f39c8a2 --- /dev/null +++ b/tests/boundary/process/commandcodeVerifierSkillBody.test.ts @@ -0,0 +1,139 @@ +import { spawnSync } from "node:child_process"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { expect, it } from "vitest"; + +const posix = it.skipIf(process.platform === "win32"); +posix.each([ + ["empty", false], + ["empty body", false], + ["wrong role", false], + ["wrong id", false], + ["missing body", false], + ["complete", true], +] as const)( + "requires CommandCode skill proof for %s", + async (mode, allowed) => { + const state = await mkdtemp(join(tmpdir(), "rea-commandcode-skill-")); + try { + const account = join(state, "account"); + await mkdir(account); + await writeFile( + join(account, ".rea-client-verification"), + "Disposable REA client verification account\n", + ); + const commands = String.raw` +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { TOOL_CONTRACTS } from REPLACE_CONTRACTS; + +let setup = 0; +export async function createVerifierCommands() { + return { + async exec(command, args, options = {}) { + if (command === "git") return { stdout: "", stderr: "" }; + if (args.includes("--version")) return { stdout: "fake", stderr: "" }; + const account = options.env.USERPROFILE; + const profile = join(account, ".commandcode"); + const config = join(profile, "mcp.json"); + const skill = join(account, ".agents", "skills", "reverse-engineer-anything"); + if (args.includes("setup") && args.includes("--dry-run")) { + return { stdout: JSON.stringify({ status: "planned", plannedActions: [{ target: config }, { target: skill }] }), stderr: "" }; + } + if (args.includes("setup")) { + if (setup++ === 0) { + await writeFile(config + ".rea.backup", await readFile(config)); + await mkdir(skill, { recursive: true }); + const frontmatter = "---\nname: reverse-engineer-anything\n---\n"; + const body = 'Complete "authored" instructions\nKeep a \\ literal\n'; + await writeFile(join(skill, "SKILL.md"), process.env.MODE === "empty" ? "" : frontmatter + (process.env.MODE === "empty body" ? "" : body)); + return { stdout: JSON.stringify({ status: "ready" }), stderr: "" }; + } + return { stdout: JSON.stringify({ appliedActions: [] }), stderr: "" }; + } + if (args.includes("doctor")) return { stdout: JSON.stringify({ healthy: true }), stderr: "" }; + const tools = [ + { name: "search_tools" }, + { name: "activate_skill", parameters: { properties: { name: { enum: ["reverse-engineer-anything"] } } } }, + ]; + const system = { role: "system", content: TOOL_CONTRACTS.map((t) => "mcp__rea__" + t.name).join(" ") }; + const first = await globalThis.fakeRequest({ body: { messages: [system] }, tools }); + const body = 'Complete "authored" instructions\nKeep a \\ literal'; + const response = { + role: process.env.MODE === "wrong role" ? "user" : "tool", + tool_call_id: process.env.MODE === "wrong id" ? "wrong" : first.tool_calls[0].id, + content: process.env.MODE === "missing body" ? "partial" : body, + }; + const second = await globalThis.fakeRequest({ body: { messages: [system, response] }, tools }); + await writeFile(process.env.STATE + "/observed.json", JSON.stringify({ first, second })); + throw Error("intentional stop before client output"); + }, + async close() {}, + }; +} +`.replace( + "REPLACE_CONTRACTS", + JSON.stringify( + new URL("../../../dist/contracts/toolContracts.js", import.meta.url) + .href, + ), + ); + await writeFile(join(state, "commands.mjs"), commands); + await writeFile( + join(state, "fixture.mjs"), + "export async function createOpenAiModelFixture({onRequest}){globalThis.fakeRequest=onRequest;return {baseUrl:'http://127.0.0.1:1',async close(){}};}\n", + ); + await writeFile( + join(state, "hook.mjs"), + "import{registerHooks}from'node:module';import{pathToFileURL}from'node:url';registerHooks({resolve(specifier,context,next){if(context.parentURL?.endsWith('/scripts/verify-commandcode-client.mjs')){let f;if(specifier==='node:os')f='os.mjs';if(specifier==='./lib/verify-commands.mjs')f='commands.mjs';if(specifier==='./verify/clients/model-fixture.mjs')f='fixture.mjs';if(f)return{url:pathToFileURL(process.env.STATE+'/'+f).href,shortCircuit:true};}return next(specifier,context);}});\n", + ); + await writeFile( + join(state, "os.mjs"), + "export * from 'node:os';export function homedir(){return process.env.REA_VERIFY_COMMANDCODE_ACCOUNT_HOME;}\n", + ); + const script = fileURLToPath( + new URL( + "../../../scripts/verify-commandcode-client.mjs", + import.meta.url, + ), + ); + const result = spawnSync( + process.execPath, + ["--import", join(state, "hook.mjs"), script], + { + encoding: "utf8", + timeout: 10_000, + env: { + ...process.env, + STATE: state, + MODE: mode, + REA_VERIFY_COMMANDCODE_ACCOUNT_HOME: account, + }, + }, + ); + expect(result.status).not.toBe(0); + const observed = await readFile( + join(state, "observed.json"), + "utf8", + ).then(JSON.parse, () => null); + expect(observed !== null).toBe(allowed); + if (allowed) + expect(observed.second.tool_calls[0].function.name).toBe( + "search_tools", + ); + expect(result.stderr).toContain( + allowed + ? "intentional stop before client output" + : mode === "empty" + ? "Installed skill must have authored frontmatter" + : mode === "empty body" + ? "Installed skill body must not be empty" + : "The model must receive the complete activated skill body", + ); + } finally { + await rm(state, { recursive: true, force: true }); + } + }, +); diff --git a/tests/boundary/process/darwinProcessRunTokenReader.test.ts b/tests/boundary/process/darwinProcessRunTokenReader.test.ts index dfd4edc423..247d3e3639 100644 --- a/tests/boundary/process/darwinProcessRunTokenReader.test.ts +++ b/tests/boundary/process/darwinProcessRunTokenReader.test.ts @@ -1389,6 +1389,7 @@ it.skipIf(!onDarwin)( zombieLifecycleIsSeparateFromIdentity: true, effectiveUidComesFromNativeIdentity: true, nativeRelationIsSeparateFromIdentity: true, + nativeNameIsSeparateFromIdentity: true, }); } finally { await rm(root, { recursive: true, force: true }); diff --git a/tests/boundary/process/deepseekVerifierSkillBody.test.ts b/tests/boundary/process/deepseekVerifierSkillBody.test.ts new file mode 100644 index 0000000000..2020878453 --- /dev/null +++ b/tests/boundary/process/deepseekVerifierSkillBody.test.ts @@ -0,0 +1,83 @@ +import { spawnSync } from "node:child_process"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { expect, it } from "vitest"; + +const posix = it.skipIf(process.platform === "win32"); +posix.each([ + ["empty", false], + ["outside wrapper", false], + ["closed wrapper", false], + ["closed early", false], + ["inside resources", false], + ["split messages", false], + ["complete", true], + ["complete escaped", true], + ["complete CRLF", true], + ["complete tags", true], +] as const)( + "requires actual skill body when model receives %s content", + async (body, allowed) => { + const state = await mkdtemp(join(tmpdir(), "rea-deepseek-skill-")); + try { + const commands = + "import {mkdir,readFile,writeFile} from 'node:fs/promises'; import {join} from 'node:path'; import {TOOL_CONTRACTS} from REPLACE_CONTRACTS;\nlet setup=0;\nexport async function createVerifierCommands(){return {async exec(command,args,options={}){\n if(command==='git')return {stdout:'',stderr:''}; if(args.includes('--version'))return {stdout:'fake',stderr:''};\n const profile=options.env.DSH_HOME;const skill=join(options.env.HOME,'.agents','skills','reverse-engineer-anything');const config=join(profile,'cordis.patch.yml');\n if(args.includes('setup')&&args.includes('--dry-run'))return {stdout:JSON.stringify({status:'planned',plannedActions:[{target:config},{target:skill}]}),stderr:''};\n if(args.includes('setup')){if(setup++===0){await writeFile(config+'.rea.backup',await readFile(config));await mkdir(skill,{recursive:true});await writeFile(join(skill,'SKILL.md'),('---\\nname: reverse-engineer-anything\\n---\\n'+(process.env.BODY==='complete tags'?'Literal and \\n':'')+'Essential \"authored\" instructions\\nKeep a \\\\ literal\\n').replaceAll('\\n',process.env.BODY==='complete CRLF'?'\\r\\n':'\\n'));return {stdout:JSON.stringify({status:'ready'}),stderr:''};}return {stdout:JSON.stringify({appliedActions:[]}),stderr:''};}\n if(args.includes('doctor'))return {stdout:JSON.stringify({healthy:true}),stderr:''};\n const tools=[...TOOL_CONTRACTS.map(t=>({name:'mcp__rea__'+t.name,parameters:{type:'object',properties:{v:{type:'string',pattern:'^x$'}}}})),{name:'skill'}];\n const first=await globalThis.fakeRequest({body:{messages:[]},tools,requestIndex:1});\n const body=((process.env.BODY==='complete tags'?'Literal and \\n':'')+'Essential \"authored\" instructions\\nKeep a \\\\ literal').replaceAll('\\n',process.env.BODY==='complete CRLF'?'\\r\\n':'\\n');\n const escapedSkill=skill.replaceAll('&','&').replaceAll('<','<').replaceAll('>','>'); const wrap='\\n\\n'+escapedSkill+(process.env.BODY==='closed early'?'':'')+(process.env.BODY==='inside resources'?body:'')+'\\n\\n\\n';\n const payload=wrap+(process.env.BODY==='closed wrapper'?'':'')+((process.env.BODY.startsWith('complete')||process.env.BODY==='closed wrapper'||process.env.BODY==='closed early')?'\\n'+body+'\\n\\n':'')+''+(process.env.BODY==='outside wrapper'?body:'');\n const messages=[{role:'tool',content:payload},...(process.env.BODY==='split messages'?[{role:'tool',content:body}]:[])];\n const second=await globalThis.fakeRequest({body:{messages},tools,requestIndex:2});\n await writeFile(process.env.STATE+'/observed.json',JSON.stringify({first,second,payload}));throw Error('intentional stop after empty-skill request');\n },async close(){}};}\n".replace( + "REPLACE_CONTRACTS", + JSON.stringify( + new URL("../../../dist/contracts/toolContracts.js", import.meta.url) + .href, + ), + ); + await writeFile(join(state, "commands.mjs"), commands); + await writeFile( + join(state, "fixture.mjs"), + "export async function createOpenAiModelFixture({onRequest}){globalThis.fakeRequest=onRequest;return {baseUrl:'http://127.0.0.1:1',async close(){}};}\n", + ); + await writeFile( + join(state, "hook.mjs"), + "import{registerHooks}from'node:module';import{pathToFileURL}from'node:url';registerHooks({resolve(specifier,context,next){if(context.parentURL?.endsWith('/scripts/verify-deepseek-client.mjs')){let f;if(specifier==='./lib/verify-commands.mjs')f='commands.mjs';if(specifier==='./verify/clients/model-fixture.mjs')f='fixture.mjs';if(f)return{url:pathToFileURL(process.env.STATE+'/'+f).href,shortCircuit:true};}return next(specifier,context);}});\n", + ); + const script = fileURLToPath( + new URL("../../../scripts/verify-deepseek-client.mjs", import.meta.url), + ); + const temporary = + body === "complete escaped" ? join(state, "dir &") : state; + if (temporary !== state) await mkdir(temporary); + const result = spawnSync( + process.execPath, + ["--import", join(state, "hook.mjs"), script], + { + encoding: "utf8", + timeout: 10_000, + env: { + ...process.env, + STATE: state, + BODY: body, + TMPDIR: temporary, + TMP: temporary, + TEMP: temporary, + }, + }, + ); + expect(result.status).not.toBe(0); + const observed = await readFile( + join(state, "observed.json"), + "utf8", + ).then(JSON.parse, () => null); + expect(observed !== null).toBe(allowed); + if (allowed) + expect(observed.second.tool_calls[0].function.name).toBe( + "mcp__rea__analyze_javascript_application", + ); + expect(result.stderr).toContain( + allowed + ? "intentional stop after empty-skill request" + : "Harness must load the installed skill before analysis", + ); + } finally { + await rm(state, { recursive: true, force: true }); + } + }, +); diff --git a/tests/boundary/process/flutterVerifierDefaultAbi.test.ts b/tests/boundary/process/flutterVerifierDefaultAbi.test.ts new file mode 100644 index 0000000000..bf8a1394c8 --- /dev/null +++ b/tests/boundary/process/flutterVerifierDefaultAbi.test.ts @@ -0,0 +1,28 @@ +import { execFile } from "node:child_process"; +import { resolve } from "node:path"; +import { promisify } from "node:util"; +import { expect, it } from "vitest"; + +const execute = promisify(execFile); + +it("requires the default Flutter AOT result to belong to the identified ABI set", async () => { + const invoke = (kind: string) => + execute( + process.execPath, + [ + "--import", + resolve("tests/fixtures/flutterVerifierPreload.mjs"), + "scripts/verify-flutter.mjs", + "--apk", + "/tmp/fixture.apk", + ], + { env: { ...process.env, REA_FLUTTER_PROBE: kind }, timeout: 10_000 }, + ); + await expect(invoke("wrong")).rejects.toMatchObject({ + stderr: expect.stringContaining( + "Default AOT selection must be one of the identified ABIs", + ), + }); + const result = await invoke("correct"); + expect(result.stdout).toContain("all asserted Flutter CLI/MCP checks passed"); +}); diff --git a/tests/boundary/process/fridaVerifier.test.ts b/tests/boundary/process/fridaVerifier.test.ts new file mode 100644 index 0000000000..e55fb24641 --- /dev/null +++ b/tests/boundary/process/fridaVerifier.test.ts @@ -0,0 +1,70 @@ +import { execFile } from "node:child_process"; +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { promisify } from "node:util"; +import { expect, it } from "vitest"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const execute = promisify(execFile); + +it("loads the Frida verifier help without starting instrumentation", async () => { + const result = await execute(process.execPath, [ + "scripts/verify-frida.mjs", + "--help", + ]); + expect(result.stdout).toContain( + "Without flags, verifies a local Node.js process", + ); +}); + +it("requires marker identity to match the selected target", async () => { + const root = await createTestTempDirectory("rea-frida-verifier-"); + const preload = join(root, "provider.mjs"); + await writeFile( + preload, + ` +import { registerHooks } from 'node:module'; +const service = \` +export class FridaInstrumentationService { + async listProcesses() { return { ok: true, value: { processes: [{ pid: 41, name: 'selected' }] } }; } + async instrument() { + const pid = process.env.REA_PROBE_KIND === 'missing' ? undefined : process.env.REA_PROBE_KIND === 'correct' ? 41 : 42; + return { ok: true, value: { cleanupError: null, evidence: { normalized_result: { pid, messages: [{ type: 'send', payload: { rea_frida_verification: 'ok', pid } }] } } } }; + } +}\`; +const manager = \`export class FridaInstrumentationManager { async closeAll() {} }\`; +registerHooks({ resolve(specifier, context, next) { + if (context.parentURL?.endsWith('/scripts/verify-frida.mjs')) { + if (specifier.endsWith('/FridaInstrumentationService.js')) return { shortCircuit: true, url: 'data:text/javascript,' + encodeURIComponent(service) }; + if (specifier.endsWith('/FridaInstrumentationManager.js')) return { shortCircuit: true, url: 'data:text/javascript,' + encodeURIComponent(manager) }; + } + return next(specifier, context); +}}); +`, + ); + const invoke = (kind: string, remote: boolean) => + execute( + process.execPath, + [ + "--import", + preload, + "scripts/verify-frida.mjs", + ...(remote ? ["--address", "127.0.0.1:27042", "--pid", "41"] : []), + ], + { env: { ...process.env, REA_PROBE_KIND: kind }, timeout: 10_000 }, + ); + for (const [kind, remote, message] of [ + ["missing", false, "Frida returned no valid target PID"], + ["wrong", true, "Frida observed a different target PID"], + ] as const) { + await expect(invoke(kind, remote)).rejects.toMatchObject({ + stderr: expect.stringContaining(message), + }); + } + const result = await invoke("correct", true); + expect(JSON.parse(result.stdout)).toMatchObject({ + ok: true, + target_pid: 41, + script_message_observed: true, + }); +}); diff --git a/tests/boundary/process/geminiVerifierSkillBody.test.ts b/tests/boundary/process/geminiVerifierSkillBody.test.ts new file mode 100644 index 0000000000..422b3b3ec8 --- /dev/null +++ b/tests/boundary/process/geminiVerifierSkillBody.test.ts @@ -0,0 +1,77 @@ +import { spawnSync } from "node:child_process"; +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { expect, it } from "vitest"; + +it.each([ + ["empty", false], + ["outside wrapper", false], + ["inside resources", false], + ["closed early", false], + ["split messages", false], + ["complete", true], + ["complete CRLF", true], + ["complete tags", true], +] as const)( + "requires actual Gemini skill body in %s response", + async (body, allowed) => { + const state = await mkdtemp(join(tmpdir(), "rea-gemini-skill-")); + try { + const commands = + "import {mkdir,readFile,writeFile} from 'node:fs/promises';import {join} from 'node:path';import {TOOL_CONTRACTS} from REPLACE_CONTRACTS;\nlet setup=0;export async function createVerifierCommands(){return {async exec(command,args,options={}){\nif(command==='git')return {stdout:'',stderr:''};if(args.includes('--version'))return {stdout:'fake',stderr:''};\nconst profile=options.env.GEMINI_CLI_HOME;const config=join(profile,'.gemini','settings.json');const skill=join(profile,'.agents','skills','reverse-engineer-anything');\nif(args.includes('setup')&&args.includes('--dry-run'))return {stdout:JSON.stringify({status:'planned',plannedActions:[{target:config},{target:join(skill,'SKILL.md')}]}),stderr:''};\nif(args.includes('setup')){if(setup++===0){await writeFile(config+'.rea.backup',await readFile(config));await mkdir(skill,{recursive:true});await writeFile(join(skill,'SKILL.md'),('---\\nname: reverse-engineer-anything\\n---\\n'+(process.env.BODY==='complete tags'?'literal and \\n':'')+'Essential \"instructions\"\\nKeep a \\\\ literal\\n').replaceAll('\\n',process.env.BODY==='complete CRLF'?'\\r\\n':'\\n'));return {stdout:JSON.stringify({status:'ready'}),stderr:''};}return {stdout:JSON.stringify({appliedActions:[]}),stderr:''};}\nif(args.includes('doctor'))return {stdout:JSON.stringify({healthy:true}),stderr:''};\nconst tools=[...TOOL_CONTRACTS.map(t=>({name:'rea__'+t.name,parametersJsonSchema:{type:'object'}})),{name:'activate_skill'}];\nconst first=await globalThis.fakeRequest({body:{contents:[]},tools});\nconst body=((process.env.BODY==='complete tags'?'literal and \\n':'')+'Essential \"instructions\"\\nKeep a \\\\ literal').replaceAll('\\n',process.env.BODY==='complete CRLF'?'\\r\\n':'\\n');\nconst valid=(process.env.BODY.startsWith('complete')||process.env.BODY==='closed early');\nconst instructions=valid?' \\n '+body+'\\n \\n\\n':'';\nconst output='\\n'+instructions+' \\n '+skill+(process.env.BODY==='closed early'?'':'')+(process.env.BODY==='inside resources'?body:'')+'\\n \\n'+(process.env.BODY==='outside wrapper'?body:'');\nconst responses=[{functionResponse:{name:'activate_skill',response:{output}}},...(process.env.BODY==='split messages'?[{functionResponse:{name:'activate_skill',response:{output:body}}}]:[])];\nconst second=await globalThis.fakeRequest({body:{contents:[{parts:responses}]},tools});const payload=output;\nawait writeFile(process.env.STATE+'/observed.json',JSON.stringify({first,second,payload}));throw Error('intentional stop before client mutation');},async close(){}};}\n".replace( + "REPLACE_CONTRACTS", + JSON.stringify( + new URL("../../../dist/contracts/toolContracts.js", import.meta.url) + .href, + ), + ); + await writeFile(join(state, "commands.mjs"), commands); + await writeFile( + join(state, "fixture.mjs"), + "export async function createGeminiModelFixture({onRequest}){globalThis.fakeRequest=onRequest;return {baseUrl:'http://127.0.0.1:1',async close(){}};}\n", + ); + await writeFile( + join(state, "hook.mjs"), + "import{registerHooks}from'node:module';import{pathToFileURL}from'node:url';registerHooks({resolve(specifier,context,next){if(context.parentURL?.endsWith('/scripts/verify-gemini-client.mjs')){let f;if(specifier==='./lib/verify-commands.mjs')f='commands.mjs';if(specifier==='./verify/clients/gemini-model-fixture.mjs')f='fixture.mjs';if(f)return{url:pathToFileURL(process.env.STATE+'/'+f).href,shortCircuit:true};}return next(specifier,context);}});\n", + ); + const script = fileURLToPath( + new URL("../../../scripts/verify-gemini-client.mjs", import.meta.url), + ); + const result = spawnSync( + process.execPath, + ["--import", join(state, "hook.mjs"), script], + { + encoding: "utf8", + timeout: 10_000, + env: { + ...process.env, + STATE: state, + BODY: body, + TMPDIR: state, + TMP: state, + TEMP: state, + }, + }, + ); + expect(result.status).not.toBe(0); + const observed = await readFile( + join(state, "observed.json"), + "utf8", + ).then(JSON.parse, () => null); + expect(observed !== null).toBe(allowed); + if (allowed) + expect(observed.second[0].functionCall.name).toBe( + "rea__analyze_javascript_application", + ); + expect(result.stderr).toContain( + allowed + ? "intentional stop before client mutation" + : "Gemini must load the installed personal REA skill", + ); + } finally { + await rm(state, { recursive: true, force: true }); + } + }, +); diff --git a/tests/boundary/process/ghidraJavaLifecycle.test.ts b/tests/boundary/process/ghidraJavaLifecycle.test.ts index ccf8c53233..d2527d8433 100644 --- a/tests/boundary/process/ghidraJavaLifecycle.test.ts +++ b/tests/boundary/process/ghidraJavaLifecycle.test.ts @@ -176,6 +176,9 @@ posix.each([true, false])( code: "version_unresolved", cancellation: { reason }, }); + if (result.error._tag !== "GhidraJavaProbeFailure") + throw new Error("Expected the invalid Java report"); + expect(result.error.cancellation?.reason).toBe(reason); expect(Object.keys(result.error)).not.toContain("cancellation"); expect(() => JSON.stringify(result.error)).not.toThrow(); } diff --git a/tests/boundary/process/ghidraVerifierAssertions.test.ts b/tests/boundary/process/ghidraVerifierAssertions.test.ts index bd6554bf7e..6f8aebe956 100644 --- a/tests/boundary/process/ghidraVerifierAssertions.test.ts +++ b/tests/boundary/process/ghidraVerifierAssertions.test.ts @@ -66,3 +66,48 @@ processTest( expect(result.exitCode, result.stderr).toBe(0); }, ); + +processTest( + "Ghidra cleanup rejects unreadable paths and accepts only confirmed absence", + async ({ processes }) => { + const assertionsUrl = new URL( + "../../../scripts/verify-real-ghidra-assertions.mjs", + import.meta.url, + ).href; + const source = ` + import assert from "node:assert/strict"; + import { createRequire, syncBuiltinESMExports } from "node:module"; + const fs = createRequire(import.meta.url)("node:fs/promises"); + const { assertCleanup } = await import(${JSON.stringify(assertionsUrl)}); + const root = await fs.mkdtemp((await import("node:os")).tmpdir() + "/rea-ghidra-cleanup-test-"); + try { + const dangling = root + "/dangling"; + if (process.platform !== "win32") { + await fs.symlink(root + "/missing-target", dangling); + await assert.rejects(assertCleanup({ runtime_root: dangling }), /Ghidra cleanup left runtime_root/); + } + } finally { await fs.rm(root, { recursive: true, force: true }); } + let result = "exists"; + fs.lstat = async () => { + if (result === "exists") return; + const error = new Error(result); + error.code = result; + throw error; + }; + syncBuiltinESMExports(); + const coordinates = { runtime_root: "/owned-runtime" }; + await assert.rejects(assertCleanup(coordinates), /Ghidra cleanup left runtime_root/); + result = "ENOENT"; + await assertCleanup(coordinates); + result = "EACCES"; + await assert.rejects(assertCleanup(coordinates), { code: "EACCES" }); + `; + const result = await processes.run( + process.execPath, + ["--input-type=module", "--eval", source], + { timeoutMs: 10_000 }, + ); + expect(result.signal, result.stderr).toBeNull(); + expect(result.exitCode, result.stderr).toBe(0); + }, +); diff --git a/tests/boundary/process/javascriptVerifierBoundedEvidence.test.ts b/tests/boundary/process/javascriptVerifierBoundedEvidence.test.ts new file mode 100644 index 0000000000..1fe3fb7320 --- /dev/null +++ b/tests/boundary/process/javascriptVerifierBoundedEvidence.test.ts @@ -0,0 +1,150 @@ +import { execFile } from "node:child_process"; +import { realpath, truncate, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { promisify } from "node:util"; +import { expect, it } from "vitest"; + +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { + applicationFixture, + electronRuntimeEvidence, + SOURCE, +} from "../../fixtures/javascriptRuntimeReconciliation.js"; + +const execute = promisify(execFile); + +it.each(["once", "always", "physical"] as const)( + "retries the exact Evidence descriptor after close denial: %s", + async (denial) => { + const root = await createTestTempDirectory("rea-verifier-evidence-close-"); + const evidence = join(root, "evidence.json"); + const preload = join(root, "instrument.mjs"); + const application = await applicationFixture(); + await writeFile( + evidence, + JSON.stringify(electronRuntimeEvidence(application, SOURCE)), + ); + const selected = await realpath(evidence); + await writeFile( + preload, + ` +import fs from 'node:fs'; +import { syncBuiltinESMExports } from 'node:module'; +const open = fs.promises.open; +let attempts = 0; +fs.promises.open = async function(path, ...args) { + const handle = await open.call(this, path, ...args); + if (String(path) === ${JSON.stringify(selected)}) { + const close = handle.close.bind(handle); + if (${JSON.stringify(denial)} === 'physical') + handle.read = async () => { throw new Error('authored Evidence read failure'); }; + handle.close = async () => { + attempts++; + if (${JSON.stringify(denial)} === 'always' || attempts === 1) + throw new Error('authored Evidence close denial'); + return close(); + }; + } + return handle; +}; +syncBuiltinESMExports(); +process.on('exit', () => fs.writeSync(2, 'Evidence close attempts: ' + attempts + '\\n')); +`, + ); + const invocation = execute( + process.execPath, + [ + "--import", + preload, + "scripts/verify/javascript/runtime-observation.mjs", + "--application", + application, + "--runtime-evidence", + evidence, + ], + { timeout: 20_000 }, + ); + if (denial === "once") { + const result = await invocation; + expect(JSON.parse(result.stdout)).toMatchObject({ verified: true }); + expect(result.stderr).toContain("Evidence close attempts: 2"); + expect(result.stderr).not.toContain("authored Evidence close denial"); + return; + } + const result: unknown = await invocation.then( + () => { + throw new Error("Expected Evidence failure"); + }, + (error: unknown) => error, + ); + expect(result).toMatchObject({ + stderr: expect.stringContaining("Evidence close attempts: 2"), + }); + if ( + !(result instanceof Error) || + !("stderr" in result) || + typeof result.stderr !== "string" + ) + throw new Error("Missing verifier diagnostics"); + if (denial === "always") { + expect(result.stderr).toContain("authored Evidence close denial"); + expect(result.stderr).toContain( + "AggregateError: Verifier cleanup failed", + ); + expect(result.stderr).toContain("cleanupLeases"); + } else { + expect(result.stderr).toContain("authored Evidence read failure"); + expect(result.stderr).not.toContain("authored Evidence close denial"); + expect(result.stderr).not.toContain( + "AggregateError: Verifier cleanup failed", + ); + } + }, + 30_000, +); + +it("refuses oversized Evidence before a whole-file read", async () => { + const root = await createTestTempDirectory("rea-verifier-evidence-limit-"); + const evidence = join(root, "evidence.json"); + const preload = join(root, "instrument.mjs"); + await writeFile(evidence, ""); + await truncate(evidence, 64 * 1024 * 1024 + 1); + const selected = await realpath(evidence); + await writeFile( + preload, + ` +import fs from 'node:fs'; +import { syncBuiltinESMExports } from 'node:module'; +const readFile = fs.promises.readFile; +fs.promises.readFile = function(path, ...args) { + if (String(path) === ${JSON.stringify(selected)}) throw new Error('oversized whole-file read'); + return readFile.call(this, path, ...args); +}; +syncBuiltinESMExports(); +`, + ); + const application = await applicationFixture(); + const result = await execute( + process.execPath, + [ + "--import", + preload, + "scripts/verify/javascript/runtime-observation.mjs", + "--application", + application, + "--runtime-evidence", + evidence, + ], + { timeout: 20_000 }, + ).then( + () => { + throw new Error("Expected oversized Evidence rejection"); + }, + (error: unknown) => error, + ); + expect(result).toMatchObject({ + stderr: expect.stringContaining( + `Runtime Evidence exceeds 64 MiB: ${evidence}`, + ), + }); +}, 30_000); diff --git a/tests/boundary/process/jebVerifier.test.ts b/tests/boundary/process/jebVerifier.test.ts new file mode 100644 index 0000000000..b13038c95a --- /dev/null +++ b/tests/boundary/process/jebVerifier.test.ts @@ -0,0 +1,65 @@ +import { execFile } from "node:child_process"; +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { promisify } from "node:util"; +import { expect, it } from "vitest"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const execute = promisify(execFile); + +it("reports a decompile marker only when a nonempty requested marker was observed", async () => { + const root = await createTestTempDirectory("rea-jeb-verifier-"); + const preload = join(root, "provider.mjs"); + await writeFile( + preload, + ` +import { registerHooks } from 'node:module'; +const service = \` +export class JebAnalysisService { + async execute(name) { + if (name === 'inspect_jeb_client') return { ok: true, value: { normalized_result: { engine: { version: 'fixture', gui_client: false, endpoint: 'fixture' } } } }; + if (name === 'open_jeb_project') return { ok: false, error: { message: 'not available' } }; + if (name === 'list_jeb_units') return { ok: true, value: { normalized_result: { units: [{}] } } }; + if (name === 'decompile_jeb_item') return { ok: true, value: { normalized_result: { text: process.env.REA_JEB_PROBE_TEXT ?? '' } } }; + throw new Error('unexpected operation'); + } +}\`; +const provider = \`export const createJebAnalysisProvider = () => ({ close: async () => {} });\`; +registerHooks({ resolve(specifier, context, next) { + if (context.parentURL?.endsWith('/scripts/verify-jeb.mjs')) { + if (specifier.endsWith('/JebAnalysisService.js')) return { shortCircuit: true, url: 'data:text/javascript,' + encodeURIComponent(service) }; + if (specifier.endsWith('/composition/jeb.js')) return { shortCircuit: true, url: 'data:text/javascript,' + encodeURIComponent(provider) }; + } + return next(specifier, context); +}}); +`, + ); + const invoke = (text: string, marker?: string) => + execute( + process.execPath, + [ + "--import", + preload, + "scripts/verify-jeb.mjs", + "--item-address", + "Lx;->x()V", + ...(marker === undefined ? [] : ["--marker", marker]), + ], + { env: { ...process.env, REA_JEB_PROBE_TEXT: text }, timeout: 10_000 }, + ); + await expect(invoke("", undefined)).rejects.toMatchObject({ + stderr: expect.stringContaining("Decompiled pseudo-code was empty"), + }); + await expect(invoke("void x() {}", "missing")).rejects.toMatchObject({ + stderr: expect.stringContaining("did not contain the fixture marker"), + }); + await expect(invoke("void x() {}", "")).rejects.toMatchObject({ + stderr: expect.stringContaining("--marker must not be empty"), + }); + expect( + JSON.parse((await invoke("void x() {}", undefined)).stdout), + ).toMatchObject({ ok: true, decompiled_marker_observed: false }); + expect(JSON.parse((await invoke("void x() {}", "x()")).stdout)).toMatchObject( + { ok: true, decompiled_marker_observed: true }, + ); +}); diff --git a/tests/boundary/process/packageUpdateRegistryHttp.test.ts b/tests/boundary/process/packageUpdateRegistryHttp.test.ts new file mode 100644 index 0000000000..9226082d03 --- /dev/null +++ b/tests/boundary/process/packageUpdateRegistryHttp.test.ts @@ -0,0 +1,169 @@ +import { mkdir, writeFile } from "node:fs/promises"; +import { request, type IncomingHttpHeaders } from "node:http"; +import { join } from "node:path"; + +import { expect, it, onTestFinished, vi } from "vitest"; + +import { verifyPackageUpdate } from "../../../scripts/verify-package-update.mjs"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +interface HttpReply { + readonly status: number | undefined; + readonly headers: IncomingHttpHeaders; + readonly body: string; +} + +const httpRequest = ( + registry: URL, + target: string, + method: string, +): Promise => + new Promise((resolve, reject) => { + const pending = request( + { + hostname: registry.hostname, + port: registry.port, + path: target, + method, + timeout: 3_000, + }, + (response) => { + const chunks: Buffer[] = []; + response.on("data", (chunk: Buffer) => chunks.push(chunk)); + response.on("error", reject); + response.on("end", () => + resolve({ + status: response.statusCode, + headers: response.headers, + body: Buffer.concat(chunks).toString("utf8"), + }), + ); + }, + ); + pending.on("error", reject); + pending.on("timeout", () => + pending.destroy(new Error("Registry fixture response timed out")), + ); + pending.end(); + }); + +/** Exercise the actual HTTP owner without installing or launching a package. */ +const registryRequest = async ( + target: string, + method = "GET", + upstreamFailure?: Error, +): Promise => { + const root = await createTestTempDirectory("rea-registry-http-"); + const packageRoot = join(root, "package"); + const tarball = join(root, "release.tgz"); + await mkdir(join(packageRoot, "dist"), { recursive: true }); + await Promise.all([ + writeFile( + join(packageRoot, "package.json"), + JSON.stringify({ name: "rea-agents", version: "6.0.0" }), + ), + writeFile( + join(packageRoot, "dist", "generatedPackageMetadata.js"), + 'export const generated = { version: "6.0.0" };\n', + ), + writeFile(tarball, "archive"), + ]); + const finished = new Error("HTTP observation completed"); + let reply: HttpReply | undefined; + // Assertions run after the owner settles: its retained upstream failure must + // never swallow a failed assertion inside the command callback. + await expect( + verifyPackageUpdate({ + prefix: join(root, "prefix"), + packageRoot, + tarball, + workspace: root, + environment: {}, + commands: { + async exec(_command, _arguments, options) { + const selected = options?.env?.npm_config_registry; + if (selected === undefined) + throw new Error("Verifier did not select its local registry"); + reply = await httpRequest(new URL(selected), target, method); + throw finished; + }, + }, + }), + ).rejects.toBe(upstreamFailure ?? finished); + if (reply === undefined) + throw new Error("Registry response was not observed"); + return reply; +}; + +it.each([ + ["/unlisted?x=a%20b&x=2", "/unlisted?x=a%20b&x=2"], + ["http://other.invalid:9999/unlisted?x=%2F", "/unlisted?x=%2F"], + ["//other.invalid/unlisted?x=1", "//other.invalid/unlisted?x=1"], + ["//other.invalid/rea-agents?x=1", "//other.invalid/rea-agents?x=1"], + ["/%40scope%2fpackage?escaped=%252f", "/%40scope%2fpackage?escaped=%252f"], +])( + "forwards %s to the fixed npm origin with its path and query", + async (target, path) => { + const destinations: string[] = []; + const upstream = vi + .spyOn(globalThis, "fetch") + .mockImplementation(async (input) => { + destinations.push(input instanceof Request ? input.url : String(input)); + return new Response("upstream body", { + status: 206, + headers: { "content-type": "application/json" }, + }); + }); + onTestFinished(() => upstream.mockRestore()); + + const reply = await registryRequest(target); + + expect(destinations).toEqual([`https://registry.npmjs.org${path}`]); + expect(reply).toMatchObject({ + status: 206, + headers: { "content-type": "application/json" }, + body: "upstream body", + }); + }, +); + +it.each([ + ["GET", "http://other.invalid/rea-agents", "metadata"], + ["GET", "/release.tgz", "archive"], + ["POST", "/unlisted", "{}"], +])("keeps the local %s %s fixture route", async (method, target, body) => { + const upstream = vi + .spyOn(globalThis, "fetch") + .mockResolvedValue( + new Response("unexpected upstream request", { status: 502 }), + ); + onTestFinished(() => upstream.mockRestore()); + + const reply = await registryRequest(target, method); + + expect(upstream).not.toHaveBeenCalled(); + expect(reply.status).toBe(200); + if (body === "metadata") + expect(JSON.parse(reply.body) as unknown).toMatchObject({ + name: "rea-agents", + "dist-tags": { latest: "6.0.0" }, + }); + else expect(reply.body).toBe(body); +}); + +it("returns upstream failures as plain text and preserves the original failure", async () => { + const failure = new Error("fixture "); + const upstream = vi.spyOn(globalThis, "fetch").mockRejectedValue(failure); + onTestFinished(() => upstream.mockRestore()); + + const reply = await registryRequest("/unlisted", "GET", failure); + + expect(reply).toMatchObject({ + status: 500, + headers: { + "content-type": "text/plain; charset=utf-8", + "x-content-type-options": "nosniff", + }, + body: String(failure), + }); +}); diff --git a/tests/boundary/process/providerVerifierCleanup.test.ts b/tests/boundary/process/providerVerifierCleanup.test.ts index 037917f2c2..1ec3be5aba 100644 --- a/tests/boundary/process/providerVerifierCleanup.test.ts +++ b/tests/boundary/process/providerVerifierCleanup.test.ts @@ -57,7 +57,7 @@ const verifierFixture = async (provider: Provider) => { if (provider === "frida") { await mkdir(join(root, "dist", "frida")); - await mkdir(join(root, "dist", "contracts")); + await mkdir(join(root, "dist", "domain")); await writeFile( join(root, "dist", "frida", "FridaInstrumentationManager.js"), `import { appendFile } from "node:fs/promises"; @@ -89,7 +89,7 @@ export class FridaInstrumentationManager { `, ); await writeFile( - join(root, "dist", "contracts", "fridaRemoteAddress.js"), + join(root, "dist", "domain", "fridaRemoteAddress.js"), "export const isValidFridaRemoteAddress = () => false;\n", ); } else { diff --git a/tests/boundary/process/qwenVerifierSkillBody.test.ts b/tests/boundary/process/qwenVerifierSkillBody.test.ts new file mode 100644 index 0000000000..e0d2ab6b99 --- /dev/null +++ b/tests/boundary/process/qwenVerifierSkillBody.test.ts @@ -0,0 +1,58 @@ +import { spawnSync } from "node:child_process"; +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { expect, it } from "vitest"; + +it.each([ + ["no skill tool", false], + ["user mention", false], + ["wrong id", false], + ["missing body", false], + ["split", false], + ["complete", true], + ["CRLF", true], +] as const)("binds Qwen skill evidence for %s", async (mode, loaded) => { + const state = await mkdtemp(join(tmpdir(), "rea-qwen-skill-")); + try { + await writeFile( + join(state, "commands.mjs"), + "import {mkdir,readFile,writeFile} from 'node:fs/promises';import {join} from 'node:path';\nexport async function createVerifierCommands(){return {async exec(command,args,options={}){\n if(args.includes('--version'))return {stdout:'fake',stderr:''};\n const profile=options.env.QWEN_HOME;const config=join(profile,'settings.json');\n if(args.includes('setup')&&args.includes('--dry-run'))return {stdout:JSON.stringify({plannedActions:[{target:config}]}),stderr:''};\n if(args.includes('setup')){\n const original=await readFile(config,'utf8');await writeFile(config+'.rea.backup',original);\n const value=JSON.parse(original);value.mcpServers.rea={command:'/fake/rea',args:['mcp']};await writeFile(config,JSON.stringify(value));\n const skill=join(options.env.HOME,'.agents','skills','reverse-engineer-anything');await mkdir(skill,{recursive:true});\n const source='---\\nname: reverse-engineer-anything\\n---\\nQwen \"authored\" body\\nKeep a \\\\ literal\\n';\n await writeFile(join(skill,'SKILL.md'),process.env.MODE==='CRLF'?source.replaceAll('\\n','\\r\\n'):source);\n return {stdout:'{}',stderr:''};\n }\n if(args.includes('doctor'))return {stdout:'{}',stderr:''};\n const skill=join(options.env.HOME,'.agents','skills','reverse-engineer-anything');\n const content='Base directory for this skill: '+skill+'\\nImportant: ALWAYS resolve absolute paths from this base directory when working with skills.\\n\\nQwen \"authored\" body\\nKeep a \\\\ literal\\n';\n const tools=[...(process.env.MODE==='no skill tool'?[]:[{name:'skill'}]),{name:'mcp__rea__analyze_javascript_application'}];\n const first=await globalThis.fakeRequest({body:{messages:[]},tools,requestIndex:1});const callId=first.tool_calls[0].id;\n let messages=(process.env.MODE==='user mention'||process.env.MODE==='no skill tool')?[{role:'user',content:'Unrelated '+skill}]:[{role:'tool',tool_call_id:process.env.MODE==='wrong id'?'other':callId,content:process.env.MODE==='missing body'?'Base directory for this skill: '+skill:content}];\n if(process.env.MODE==='split')messages=[{role:'tool',tool_call_id:callId,content:'Base directory for this skill: '+skill},{role:'tool',tool_call_id:'other',content}];\n const second=await globalThis.fakeRequest({body:{messages},tools,requestIndex:2});await writeFile(process.env.STATE+'/observed.json',JSON.stringify({first,second,messages}));throw Error('intentional stop before client output');\n},async close(){}};}\n", + ); + await writeFile( + join(state, "fixture.mjs"), + "export async function createOpenAiModelFixture({onRequest}){globalThis.fakeRequest=onRequest;return {baseUrl:'http://127.0.0.1:1',async close(){}};}\n", + ); + await writeFile( + join(state, "hook.mjs"), + "import {registerHooks}from'node:module';import{pathToFileURL}from'node:url';registerHooks({resolve(specifier,context,next){if(context.parentURL?.endsWith('/scripts/verify-qwen-client.mjs')){let f;if(specifier==='./lib/verify-commands.mjs')f='commands.mjs';if(specifier==='./verify/clients/model-fixture.mjs')f='fixture.mjs';if(f)return{url:pathToFileURL(process.env.STATE+'/'+f).href,shortCircuit:true};}return next(specifier,context);}});\n", + ); + const script = fileURLToPath( + new URL("../../../scripts/verify-qwen-client.mjs", import.meta.url), + ); + const result = spawnSync( + process.execPath, + ["--import", join(state, "hook.mjs"), script], + { + encoding: "utf8", + timeout: 10_000, + env: { + ...process.env, + STATE: state, + MODE: mode, + TMPDIR: state, + TMP: state, + TEMP: state, + }, + }, + ); + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("intentional stop before client output"); + const receipt = JSON.parse(result.stdout.trim().split("\n")[0] ?? ""); + expect(receipt.selectedSkillLoaded).toBe(loaded); + expect(receipt.passed).toBe(false); + } finally { + await rm(state, { recursive: true, force: true }); + } +}); diff --git a/tests/boundary/providers/RizinDebugSessionCoverage.part1.test.ts b/tests/boundary/providers/RizinDebugSessionCoverage.part1.test.ts index b167e484e5..c5080464f8 100644 --- a/tests/boundary/providers/RizinDebugSessionCoverage.part1.test.ts +++ b/tests/boundary/providers/RizinDebugSessionCoverage.part1.test.ts @@ -159,28 +159,6 @@ it.skipIf(process.platform === "win32")( }, ); -it.skipIf(process.platform === "win32")( - "rejects a non-TTY path before spawning Rizin", - async () => { - let spawned = false; - const manager = createManager( - { REA_RIZIN_COMMAND: "injected-rizin" }, - { - spawnProcess: async () => { - spawned = true; - throw new Error("must not spawn with an invalid TTY"); - }, - }, - ); - const result = await manager.start({ - path: "/tmp/not-a-tty-target", - inferior_tty: "/dev/null", - }); - expect(result.ok).toBe(false); - expect(spawned).toBe(false); - }, -); - it.skipIf(process.platform === "win32")( "closeAll drains concurrent startups and blocks later admission", async () => { diff --git a/tests/boundary/providers/auxiliaryDeclarations.test.ts b/tests/boundary/providers/auxiliaryDeclarations.test.ts index 15e42fc32a..c28137f5af 100644 --- a/tests/boundary/providers/auxiliaryDeclarations.test.ts +++ b/tests/boundary/providers/auxiliaryDeclarations.test.ts @@ -7,7 +7,13 @@ import { parseBinaryTarget } from "../../../src/application/BinaryTargetResolver it("keeps declaration discovery and client creation lazy and independently owned", async () => { const target = await parseBinaryTarget(process.execPath); if (!target.ok) throw target.error; - for (const declaration of auxiliaryAnalysisProviderDeclarations({})) { + const declarations = auxiliaryAnalysisProviderDeclarations({}); + expect(declarations.map(({ identity }) => identity.id).sort()).toEqual([ + "native-macos", + "rea-artifact-graph", + "rea-dotnet-static", + ]); + for (const declaration of declarations) { let loads = 0; const provider = new LazyAnalysisProvider({ ...declaration, diff --git a/tests/boundary/providers/cutterBridgeProducer.test.ts b/tests/boundary/providers/cutterBridgeProducer.test.ts index 721ffc05a1..c81cf904b3 100644 --- a/tests/boundary/providers/cutterBridgeProducer.test.ts +++ b/tests/boundary/providers/cutterBridgeProducer.test.ts @@ -239,10 +239,10 @@ it("retains failed receipt acknowledgements and retries before another command", bridge.client.execute({ sessionId: bridge.session.session_id, expectedGeneration: bridge.session.document_generation, - command: "iI", + command: "fixture-receipt-count", json: false, }), - ).resolves.toMatchObject({ executionState: "complete" }); + ).resolves.toMatchObject({ executionState: "complete", output: "0" }); } finally { await bridge.close(); } @@ -263,10 +263,10 @@ it("retains a receipt when an ACK reply omits its acknowledgement field", async bridge.client.execute({ sessionId: bridge.session.session_id, expectedGeneration: bridge.session.document_generation, - command: "iI", + command: "fixture-receipt-count", json: false, }), - ).resolves.toMatchObject({ executionState: "complete" }); + ).resolves.toMatchObject({ executionState: "complete", output: "0" }); } finally { await bridge.close(); } diff --git a/tests/boundary/providers/ghidra/ghidraClient.fixture.ts b/tests/boundary/providers/ghidra/ghidraClient.fixture.ts index a0102a0954..045a9065db 100644 --- a/tests/boundary/providers/ghidra/ghidraClient.fixture.ts +++ b/tests/boundary/providers/ghidra/ghidraClient.fixture.ts @@ -27,6 +27,7 @@ type FixtureMode = | "future_id" | "analysis_timeout" | "remote_error" + | "decompile_timed_out" | "remote_startup_error" | "shutdown_error" | "invalid_shutdown_ack" diff --git a/tests/boundary/providers/jeb/failureEnvelopes.test.ts b/tests/boundary/providers/jeb/failureEnvelopes.test.ts index ca64c0e9d9..00342bc52c 100644 --- a/tests/boundary/providers/jeb/failureEnvelopes.test.ts +++ b/tests/boundary/providers/jeb/failureEnvelopes.test.ts @@ -170,7 +170,6 @@ it.each( expect(producer.calls).toEqual([ { name: "open_project", arguments: { file_path: input.path } }, ]); - expect(projected.error.code).not.toBe("unsupported_target"); } finally { try { await client.close(); diff --git a/tests/boundary/providers/kuna/fakeKuna.ts b/tests/boundary/providers/kuna/fakeKuna.ts index 98189c276d..0f24f98296 100644 --- a/tests/boundary/providers/kuna/fakeKuna.ts +++ b/tests/boundary/providers/kuna/fakeKuna.ts @@ -6,6 +6,8 @@ export interface FakeKunaFunction { readonly name: string; readonly addr: string; readonly error?: string; + /** Inventory aliases, such as versioned-symbol names or generated `sub_` names. */ + readonly aliases?: readonly string[]; } /** One inventory-only row; Kuna reports a size of 0 for an unmeasured entry. */ @@ -13,6 +15,7 @@ export interface FakeKunaInventoryEntry { readonly name: string; readonly addr: string; readonly size?: number; + readonly aliases?: readonly string[]; } export interface FakeKunaString { @@ -121,7 +124,7 @@ if (args[0] === "--version") { if (mode === "hang") hang(); else if (mode === "not-json") process.stdout.write("functions: 2\n"); else { - const listed = [...functions, ...(behavior.inventoryEntries ?? []), ...(behavior.inventoryImports === false ? [] : [{ name: "puts", addr: "0x2000" }])].map((fn) => ({ name: fn.name, address: parseInt(fn.addr, 16), address_hex: fn.addr, aliases: [], object_location: null, size: fn.size ?? (fn.name === "puts" ? 0 : 16) })); + const listed = [...functions, ...(behavior.inventoryEntries ?? []), ...(behavior.inventoryImports === false ? [] : [{ name: "puts", addr: "0x2000" }])].map((fn) => ({ name: fn.name, address: parseInt(fn.addr, 16), address_hex: fn.addr, aliases: fn.aliases ?? [], object_location: null, size: fn.size ?? (fn.name === "puts" ? 0 : 16) })); setTimeout(() => process.stdout.write(JSON.stringify({ binary: args[1], count: listed.length, total: listed.length, error: null, functions: listed }, null, 2) + "\n"), behavior.inventoryDelayMs ?? 0); } } else if (args[0] === "strings") { diff --git a/tests/boundary/providers/kuna/kunaAliasSearchMcp.test.ts b/tests/boundary/providers/kuna/kunaAliasSearchMcp.test.ts new file mode 100644 index 0000000000..a633278d30 --- /dev/null +++ b/tests/boundary/providers/kuna/kunaAliasSearchMcp.test.ts @@ -0,0 +1,237 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { expect } from "vitest"; + +import { AnalysisProviderRegistry } from "../../../../src/application/binary/AnalysisProviderRegistry.js"; +import { composeBinarySession } from "../../../../src/application/binary/BinarySessionComposition.js"; +import { parseEvidence } from "../../../../src/domain/evidence.js"; +import { KunaProvider } from "../../../../src/kuna/KunaProvider.js"; +import { elfNoteFixture } from "../../../../src/native/elf/ElfNotes.fixture.js"; +import { createServer } from "../../../../src/server/createServer.js"; +import { parseMcpToolError } from "../../../fixtures/mcpToolError.js"; +import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js"; +import { mcpTest, type TestMcp } from "../../../support/mcp/mcpFixture.js"; +import { writeFakeKuna, type FakeKunaBehavior } from "./fakeKuna.js"; +import { + MACHO_FUNCTIONS, + MACHO_INVENTORY_ENTRIES, + machoInventoryImage, +} from "./kunaMachoFixture.js"; + +const test = mcpTest.skipIf(process.platform === "win32"); + +/** + * Kuna's inventory for the versioned ELF in #8330: two entries share the + * source-export alias `audit_transform`, each has a generated `sub_` alias, + * and one repeats its primary name among its aliases. + */ +const VERSIONED = [ + { name: "main", addr: "0x1000" }, + { + name: "audit_v1", + addr: "0x1334", + aliases: ["audit_transform", "sub_1334"], + }, + { + name: "audit_v2", + addr: "0x133a", + aliases: ["audit_transform", "sub_133a", "audit_v2", "sub_133a"], + }, +]; + +const aliasSession = async ( + mcp: TestMcp, + prefix: string, + behavior: FakeKunaBehavior, + image: Uint8Array = elfNoteFixture({ bits: 64, type: 2 }), +) => { + const root = await createTestTempDirectory(prefix); + const kuna = await writeFakeKuna(root, behavior); + const target = join(root, "program"); + await writeFile(target, image); + const session = composeBinarySession( + new AnalysisProviderRegistry( + [ + new KunaProvider( + { REA_KUNA_COMMAND: kuna, REA_KUNA_CACHE_DIR: join(root, "cache") }, + { platform: process.platform }, + ), + ], + "kuna", + ), + ); + const client = await mcp.connect(createServer({ kind: "session", session })); + const call = async (name: string, arguments_: Record) => { + const response = await client.callTool({ name, arguments: arguments_ }); + if (response.isError === true) + throw new Error(`${name} failed: ${JSON.stringify(response)}`); + return response.structuredContent; + }; + const observe = async (name: string, arguments_: Record) => + parseEvidence(await call(name, arguments_)); + await call("open_binary", { path: target }); + const search = async (arguments_: Record) => + (await observe("search_procedures", arguments_)).normalized_result; + return { client, call, observe, search }; +}; + +const audit = ( + version: 1 | 2, + matched_aliases?: readonly string[], +): Record => ({ + address: version === 1 ? "0x1334" : "0x133a", + value: `audit_v${String(version)}`, + ...(matched_aliases === undefined ? {} : { matched_aliases }), +}); + +test("searches the aliases exact lookup accepts without listing them as procedures", async ({ + mcp, +}) => { + const { client, call, observe, search } = await aliasSession( + mcp, + "rea-kuna-alias-search-", + { functions: VERSIONED }, + ); + // The listing keeps one row per entry under its primary name. + expect((await observe("list_procedures", {})).normalized_result).toEqual([ + { address: "0x1000", value: "main" }, + audit(1), + audit(2), + { address: "0x2000", value: "puts" }, + ]); + + const shared = [audit(1, ["audit_transform"]), audit(2, ["audit_transform"])]; + expect(await search({ pattern: "audit_transform" })).toEqual(shared); + expect( + await search({ + pattern: "^audit_transform$", + mode: "regex", + case_sensitive: true, + }), + ).toEqual(shared); + expect(await search({ pattern: "SUB_1334" })).toEqual([ + audit(1, ["sub_1334"]), + ]); + expect(await search({ pattern: "SUB_1334", case_sensitive: true })).toEqual( + [], + ); + // A primary-name match carries no alias; one also matching an alias keeps + // the primary value, and an alias repeating the primary name is not listed. + expect(await search({ pattern: "audit_v1" })).toEqual([audit(1)]); + expect(await search({ pattern: "audit" })).toEqual(shared); + expect(await search({ pattern: "^sub_", mode: "regex" })).toEqual([ + audit(1, ["sub_1334"]), + audit(2, ["sub_133a"]), + ]); + expect(await search({ pattern: "absent_alias" })).toEqual([]); + + // Exact lookup and search agree: a unique alias resolves, and a shared one + // stays ambiguous with both entries. + expect( + (await observe("procedure_address", { procedure: "sub_1334" })) + .normalized_result, + ).toBe("0x1334"); + const ambiguous = await client.callTool({ + name: "procedure_address", + arguments: { procedure: "audit_transform" }, + }); + expect(parseMcpToolError(ambiguous).error.message).toContain( + "0x1334, 0x133a", + ); + + // Retained summary pages carry which name matched. + const summary = (await call("search_procedures", { + pattern: "audit_transform", + detail: "summary", + })) as { + normalized_result: { + parent_evidence_id: string; + summary: { row_count: number; selectable_fields: string[] }; + }; + }; + expect(summary.normalized_result.summary.row_count).toBe(2); + expect(summary.normalized_result.summary.selectable_fields).toContain( + "matched_aliases", + ); + const page = await observe("inspect_analysis_view", { + source: { + kind: "retained-evidence", + evidence_id: summary.normalized_result.parent_evidence_id, + }, + view: { kind: "native-inventory", fields: ["matched_aliases"] }, + }); + expect(page.normalized_result).toMatchObject({ + rows: [ + { + source_index: 0, + record: { address: "0x1334", matched_aliases: ["audit_transform"] }, + }, + { + source_index: 1, + record: { address: "0x133a", matched_aliases: ["audit_transform"] }, + }, + ], + }); +}); + +test("traces a procedure found only by its alias through the composed feature trace", async ({ + mcp, +}) => { + const { observe } = await aliasSession(mcp, "rea-kuna-alias-trace-", { + functions: VERSIONED, + }); + const traced = await observe("trace_feature", { query: "sub_1334" }); + expect(traced.normalized_result).toMatchObject({ + matches: [{ type: "procedure", ...audit(1, ["sub_1334"]) }], + references: [ + { + target_address: "0x1334", + source_address: "0x1004", + containing_procedure: { status: "unavailable" }, + }, + ], + }); + const primary = await observe("trace_feature", { query: "audit_v1" }); + expect(primary.normalized_result).toMatchObject({ + matches: [{ type: "procedure", ...audit(1) }], + }); +}); + +test("omits a data entry an alias names while keeping the code entry that shares it", async ({ + mcp, +}) => { + const { search, observe } = await aliasSession( + mcp, + "rea-kuna-alias-macho-", + { + functions: [ + MACHO_FUNCTIONS[0]!, + { ...MACHO_FUNCTIONS[1]!, aliases: ["shared_symbol"] }, + ], + inventoryEntries: MACHO_INVENTORY_ENTRIES.map((entry) => + entry.name === "global_count" + ? { ...entry, aliases: ["shared_symbol"] } + : entry, + ), + inventoryImports: false, + }, + machoInventoryImage(), + ); + const searched = await observe("search_procedures", { + pattern: "shared_symbol", + }); + expect(searched.normalized_result).toEqual([ + { + address: "0x100000440", + value: "helper", + matched_aliases: ["shared_symbol"], + }, + ]); + expect(searched.limitations).toContainEqual( + expect.stringContaining( + "Kuna's inventory also lists 1 entry in Mach-O data regions (__DATA,__data: 1)", + ), + ); + expect(await search({ pattern: "global_count" })).toEqual([]); +}); diff --git a/tests/boundary/providers/kuna/kunaBatchDecompileMcp.test.ts b/tests/boundary/providers/kuna/kunaBatchDecompileMcp.test.ts index afe215505a..4e8f191faf 100644 --- a/tests/boundary/providers/kuna/kunaBatchDecompileMcp.test.ts +++ b/tests/boundary/providers/kuna/kunaBatchDecompileMcp.test.ts @@ -1,3 +1,4 @@ +import { batchDecompileResultSchema } from "../../../../src/domain/native/batchDecompile.js"; import { writeFile } from "node:fs/promises"; import { join } from "node:path"; @@ -92,7 +93,9 @@ test("keeps each Kuna no-pseudocode observation's limitations on its batch item" throw new Error( "Missing batch_decompile or get_evidence_bundle contract", ); - const result = batchDecompile.outputSchema.parse(batch).normalized_result; + const result = batchDecompileResultSchema.parse( + batchDecompile.outputSchema.parse(batch).normalized_result, + ); expect(result).toMatchObject({ total: 4, succeeded: 1, failed: 3 }); const [main, failed, imported, unknown] = result.items; expect(main).toMatchObject({ address: "main", status: "ok" }); diff --git a/tests/boundary/providers/kuna/kunaCommandFailures.test.ts b/tests/boundary/providers/kuna/kunaCommandFailures.test.ts index c480f52a77..ffea9aa070 100644 --- a/tests/boundary/providers/kuna/kunaCommandFailures.test.ts +++ b/tests/boundary/providers/kuna/kunaCommandFailures.test.ts @@ -1,4 +1,4 @@ -import { expect, it as test } from "vitest"; +import { expect, it as test, vi } from "vitest"; import { projectAnalysisError } from "../../../../src/domain/analysisErrorProjection.js"; import { inspectKunaInstallationOwned as inspectKunaInstallation } from "./kunaResourceScope.js"; @@ -147,3 +147,98 @@ it("says why Kuna cannot answer instead of a generic unavailable message", async expect(await client.close()).toEqual({ ok: true, value: null }); } }); + +it("preserves a caller reason when a target command is already cancelled", async () => { + const { kuna, target, cacheRoot } = await session( + "rea-kuna-command-preabort-", + ); + const installed = await inspectKunaInstallation({ + REA_KUNA_COMMAND: kuna, + REA_KUNA_CACHE_DIR: cacheRoot, + }); + if (!installed.ok) throw new Error(installed.error.reason); + const controller = new AbortController(); + const reason = { caller: "before target command admission" }; + controller.abort(reason); + const callsBefore = await fakeKunaCalls(kuna); + const releaseRun = vi.fn(); + const commands = new KunaTargetCommands(target, { + signal: controller.signal, + }); + try { + const result = await commands.run({ + installation: installed.value, + operation: "list_procedures", + arguments: (input) => ["functions", input, "--json"], + deadline: { setting: "REA_KUNA_TIMEOUT_MS", ms: 10_000 }, + outputBytes: 100, + signal: controller.signal, + releaseRun, + }); + if (result.ok) throw new Error("Expected pre-aborted command to cancel"); + expect(result.error._tag).toBe("AnalysisCancelledError"); + expect(result.error.cause).toBe(reason); + expect(releaseRun).toHaveBeenCalledOnce(); + expect(await fakeKunaCalls(kuna)).toEqual(callsBefore); + } finally { + expect(await commands.close()).toEqual({ ok: true, value: null }); + } +}); + +it("preserves the lifetime reason when workspace preparation is already cancelled", async () => { + const { kuna, target } = await session("rea-kuna-prepare-preabort-"); + const controller = new AbortController(); + const reason = { caller: "before workspace preparation" }; + controller.abort(reason); + const callsBefore = await fakeKunaCalls(kuna); + const commands = new KunaTargetCommands(target, { + signal: controller.signal, + }); + try { + const result = await commands.prepare(); + if (result.ok) throw new Error("Expected cancelled preparation"); + expect(result.error).toMatchObject({ + _tag: "AnalysisCancelledError", + operation: "kuna:workspace", + }); + expect(result.error.cause).toBe(reason); + expect(await fakeKunaCalls(kuna)).toEqual(callsBefore); + } finally { + expect(await commands.close()).toEqual({ ok: true, value: null }); + } +}); + +it("does not invent a caller cause when closed without cancellation", async () => { + const { kuna, target, cacheRoot } = await session("rea-kuna-command-closed-"); + const installed = await inspectKunaInstallation({ + REA_KUNA_COMMAND: kuna, + REA_KUNA_CACHE_DIR: cacheRoot, + }); + if (!installed.ok) throw new Error(installed.error.reason); + const controller = new AbortController(); + const callsBefore = await fakeKunaCalls(kuna); + const releaseRun = vi.fn(); + const commands = new KunaTargetCommands(target, { + signal: controller.signal, + }); + expect(await commands.close()).toEqual({ ok: true, value: null }); + const preparation = await commands.prepare(); + if (preparation.ok) throw new Error("Expected closed preparation to cancel"); + expect(preparation.error._tag).toBe("AnalysisCancelledError"); + expect(preparation.error.cause).toBeUndefined(); + const result = await commands.run({ + installation: installed.value, + operation: "list_procedures", + arguments: (input) => ["functions", input, "--json"], + deadline: { setting: "REA_KUNA_TIMEOUT_MS", ms: 10_000 }, + outputBytes: 100, + signal: controller.signal, + releaseRun, + }); + if (result.ok) throw new Error("Expected closed command to cancel"); + expect(result.error._tag).toBe("AnalysisCancelledError"); + expect(result.error.cause).toBeUndefined(); + expect(releaseRun).toHaveBeenCalledOnce(); + expect(await fakeKunaCalls(kuna)).toEqual(callsBefore); + expect(await commands.close()).toEqual({ ok: true, value: null }); +}); diff --git a/tests/boundary/providers/kuna/kunaDoctor.test.ts b/tests/boundary/providers/kuna/kunaDoctor.test.ts index 89cd3ea401..8c3a5f052a 100644 --- a/tests/boundary/providers/kuna/kunaDoctor.test.ts +++ b/tests/boundary/providers/kuna/kunaDoctor.test.ts @@ -34,7 +34,7 @@ it("reports an unconfigured Kuna as a missing optional engine", async () => { it("publishes the caller's explicit settings once the installation validates", async () => { const root = await createTestTempDirectory("rea-kuna-doctor-"); const kuna = await writeFakeKuna(root, { - version: { stdout: "kuna 1.986\n" }, + version: { stdout: "kuna 1.988\n" }, }); const cache = join(root, "cache"); @@ -48,7 +48,7 @@ it("publishes the caller's explicit settings once the installation validates", a expect(inspection).toMatchObject({ configured: true, available: true, - providerVersion: "1.986", + providerVersion: "1.988", registrationEnvironment: { REA_KUNA_COMMAND: kuna, REA_KUNA_CACHE_DIR: cache, @@ -59,10 +59,10 @@ it("publishes the caller's explicit settings once the installation validates", a name: "installation", ok: true, detail: expect.stringMatching( - /reports Kuna 1\.986 \(sha256 [a-f0-9]{64}\)\.$/u, + /reports Kuna 1\.988 \(sha256 [a-f0-9]{64}\)\.$/u, ), remediation: null, - details: { version: "1.986", verified_version: "1.986" }, + details: { version: "1.988", verified_version: "1.988" }, }, { name: "cache", @@ -88,7 +88,7 @@ it("flags an unverified release without failing the installation", async () => { "You may continue with the selected unverified release.", ), remediation: expect.stringContaining("setting REA_KUNA_COMMAND"), - details: { verified_version: "1.986" }, + details: { verified_version: "1.988" }, }); }); diff --git a/tests/boundary/providers/kuna/kunaInstallation.test.ts b/tests/boundary/providers/kuna/kunaInstallation.test.ts index c4b4dffc82..1f66253993 100644 --- a/tests/boundary/providers/kuna/kunaInstallation.test.ts +++ b/tests/boundary/providers/kuna/kunaInstallation.test.ts @@ -22,7 +22,7 @@ const it = test.skipIf(process.platform === "win32"); it("identifies a caller-selected kuna by version and digest", async () => { const root = await createTestTempDirectory("rea-kuna-install-"); const kuna = await writeFakeKuna(root, { - version: { stdout: "kuna 1.986\n" }, + version: { stdout: "kuna 1.988\n" }, }); const link = join(root, "current-kuna"); await symlink(kuna, link); @@ -37,7 +37,7 @@ it("identifies a caller-selected kuna by version and digest", async () => { value: { configuration: { command: link, cacheRoot: join(root, "cache") }, executable: await realpath(kuna), - version: "1.986", + version: "1.988", verified: true, // A `#!` launcher reports its interpreter; ownership checks must not expect the path. script: true, @@ -104,14 +104,14 @@ fs.renameSync(process.argv[1] + ".new", process.argv[1]); it("accepts another release but reports it as unverified", async () => { const root = await createTestTempDirectory("rea-kuna-unverified-"); const kuna = await writeFakeKuna(root, { - version: { stdout: "kuna 1.833\n" }, + version: { stdout: "kuna 1.986\n" }, }); const result = await inspectKunaInstallation({ REA_KUNA_COMMAND: kuna }); expect(result).toMatchObject({ ok: true, - value: { version: "1.833", verified: false }, + value: { version: "1.986", verified: false }, }); }); diff --git a/tests/boundary/providers/kuna/kunaProjectCache.test.ts b/tests/boundary/providers/kuna/kunaProjectCache.test.ts index 3b949c8024..311a97ea8d 100644 --- a/tests/boundary/providers/kuna/kunaProjectCache.test.ts +++ b/tests/boundary/providers/kuna/kunaProjectCache.test.ts @@ -705,6 +705,28 @@ it("shares one Kuna run between concurrent requests for one export", async () => expect(await callsOf(kuna, "decompile-project")).toHaveLength(1); }); +it("preserves a pre-aborted caller reason without launching an export", async () => { + const { kuna, installation, cache, target } = await fixture( + "rea-kuna-preabort-reason-", + ); + const input = await target("program"); + const controller = new AbortController(); + const reason = { caller: "before admission" }; + controller.abort(reason); + const callsBefore = await fakeKunaCalls(kuna); + const result = await cache.build({ + installation, + ...input, + signal: controller.signal, + }); + if (result.ok) + throw new Error("Expected pre-aborted cache request to cancel"); + expect(result.error._tag).toBe("AnalysisCancelledError"); + expect(result.error.cause).toBe(reason); + expect(await fakeKunaCalls(kuna)).toEqual(callsBefore); + expect(await cache.close()).toEqual({ ok: true, value: null }); +}); + it("stops the run only when every waiting request is cancelled", async () => { const { kuna, installation, cacheRoot, cache, target } = await fixture( "rea-kuna-cancel-", @@ -727,23 +749,31 @@ it("stops the run only when every waiting request is cancelled", async () => { await expect.poll(() => callsOf(kuna, "decompile-project")).toHaveLength(1); const pid = (await callsOf(kuna, "decompile-project"))[0]?.pid ?? 0; - firstCaller.abort(); - expect(await first).toMatchObject({ + const firstReason = { caller: "first waiter" }; + firstCaller.abort(firstReason); + const firstResult = await first; + expect(firstResult).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" }, }); + if (firstResult.ok) throw new Error("Expected first waiter cancellation"); // The other request still owns the run. expect(processGone(pid)).toBe(false); - secondCaller.abort(); - expect(await second).toMatchObject({ + const secondReason = { caller: "last waiter" }; + secondCaller.abort(secondReason); + const secondResult = await second; + expect(secondResult).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" }, }); + if (secondResult.ok) throw new Error("Expected last waiter cancellation"); expect(await cache.close()).toEqual({ ok: true, value: null }); expect(processGone(pid)).toBe(true); expect(await entries(cacheRoot, "staging")).toEqual([]); expect(await entries(cacheRoot, "projects")).toEqual([]); + expect(firstResult.error.cause).toBe(firstReason); + expect(secondResult.error.cause).toBe(secondReason); }); it("applies one deadline to the inventory and the export together", async () => { diff --git a/tests/boundary/providers/kuna/kunaProvider.test.ts b/tests/boundary/providers/kuna/kunaProvider.test.ts index 110d820d8c..8907678718 100644 --- a/tests/boundary/providers/kuna/kunaProvider.test.ts +++ b/tests/boundary/providers/kuna/kunaProvider.test.ts @@ -182,13 +182,13 @@ it("waits for an aborted pre-session hash before retrying its late close owner", it("commits the observed Kuna release and digest to the analysis profile", async () => { const { profile, kuna } = await session("rea-kuna-profile-", { - version: { stdout: "kuna 1.986\n" }, + version: { stdout: "kuna 1.988\n" }, }); expect(profile).toMatchObject({ - provider: { id: "kuna", name: "Kuna", version: "1.986" }, + provider: { id: "kuna", name: "Kuna", version: "1.988" }, parameters: { - kuna_version: "1.986", + kuna_version: "1.988", kuna_sha256: expect.stringMatching(/^[0-9a-f]{64}$/u), kuna_command: kuna, verified_release: true, diff --git a/tests/boundary/providers/kuna/kunaSingleFunctionRuns.test.ts b/tests/boundary/providers/kuna/kunaSingleFunctionRuns.test.ts index e7c22bec3f..468dd9329f 100644 --- a/tests/boundary/providers/kuna/kunaSingleFunctionRuns.test.ts +++ b/tests/boundary/providers/kuna/kunaSingleFunctionRuns.test.ts @@ -1,6 +1,7 @@ import { expect, it as test } from "vitest"; import { projectAnalysisError } from "../../../../src/domain/analysisErrorProjection.js"; +import { kunaTimeoutRecovery } from "../../../../src/kuna/KunaCommand.js"; import { fakeKunaCalls, @@ -311,6 +312,14 @@ it("names the setting that owns a single-function deadline", async () => { }, details: { timeout_ms: 300 }, }); + // A finished invocation cannot read a later export from this session. + const recovery = kunaTimeoutRecovery("REA_KUNA_TIMEOUT_MS", "decompile"); + expect( + projectAnalysisError(result.error, "invocation").remediation.action, + ).toBe(recovery.remediationAction); + expect(projectAnalysisError(result.error).remediation.action).toBe( + recovery.continuingCallerGuidance?.remediationAction, + ); } finally { expect(await client.close()).toEqual({ ok: true, value: null }); } diff --git a/tests/boundary/providers/managed/managedHashCancellation.test.ts b/tests/boundary/providers/managed/managedHashCancellation.test.ts index d917a6e6a2..2681f7a48d 100644 --- a/tests/boundary/providers/managed/managedHashCancellation.test.ts +++ b/tests/boundary/providers/managed/managedHashCancellation.test.ts @@ -94,6 +94,7 @@ it.each([ code: "cleanup_incomplete", details: { diagnostics: { primary_error: { code: "cancelled" } } }, }); + expect(cancelled.error.cleanupPrimary?.cause).toBe(reason); expect(cancelledHandle?.fd).toBeGreaterThanOrEqual(0); const readsBeforeRetry = readCount; const blocked = await client.execute("inspect_managed_members", {}); diff --git a/tests/boundary/providers/managed/managedMemberComparisonFromMetadata.test.ts b/tests/boundary/providers/managed/managedMemberComparisonFromMetadata.test.ts index 51154ab80d..d703a0b58f 100644 --- a/tests/boundary/providers/managed/managedMemberComparisonFromMetadata.test.ts +++ b/tests/boundary/providers/managed/managedMemberComparisonFromMetadata.test.ts @@ -36,8 +36,12 @@ it("selects namespace-bearing nested declarations from parsed type facts and com expect(selected.result.methods).toHaveLength(2); expect(selected.result.fields).toHaveLength(2); const comparison = compareManagedMembers(selected, selected); + expect(comparison.methods).toHaveLength(2); + expect(comparison.fields).toHaveLength(2); expect( - comparison.methods.every(({ match }) => match.status === "matched"), + [...comparison.methods, ...comparison.fields].every( + ({ match }) => match.status === "matched", + ), ).toBe(true); expect(comparison.summary).toMatchObject({ added: 0, removed: 0 }); expect( @@ -2569,62 +2573,43 @@ describe("managed member comparison of ownerless members", () => { } }); - it("keeps a resolved copy out of the declared round when an ownerless competitor is on the left, as main does", () => { - const base = inspect(buildManagedPeFixture(), "/tmp/ownerless-left.dll"); - const [method] = base.result.methods; - if (method === undefined) throw new Error("Expected a fixture method"); - const ownerless = { - ...method, - declaring_type: null, - declaring_type_token: null, - }; - // The left holds the resolved Foo and an ownerless Foo with the same CIL - // and signature; the right holds only the resolved Foo. The ownerless - // Foo's owner may be the resolved one, so the three stay ambiguous. - const withOwnerless = { - ...base, - result: { ...base.result, methods: [method, ownerless], fields: [] }, - }; - const resolvedOnly = { - ...base, - result: { ...base.result, methods: [method], fields: [] }, - }; + it.each(["left", "right"] as const)( + "keeps a resolved copy out of the declared round when an ownerless competitor is on the %s", + (side) => { + const base = inspect( + buildManagedPeFixture(), + `/tmp/ownerless-${side}.dll`, + ); + const [method] = base.result.methods; + if (method === undefined) throw new Error("Expected a fixture method"); + const ownerless = { + ...method, + declaring_type: null, + declaring_type_token: null, + }; + // The ownerless Foo may belong to the same type as the resolved Foo, + // so both comparison directions must leave the three members ambiguous. + const withOwnerless = { + ...base, + result: { ...base.result, methods: [method, ownerless], fields: [] }, + }; + const resolvedOnly = { + ...base, + result: { ...base.result, methods: [method], fields: [] }, + }; - const result = compareManagedMembers(withOwnerless, resolvedOnly); + const result = + side === "left" + ? compareManagedMembers(withOwnerless, resolvedOnly) + : compareManagedMembers(resolvedOnly, withOwnerless); - expect(result.matching.ambiguous).toBe(1); - expect( - result.methods.filter(({ match }) => match.status === "matched"), - ).toEqual([]); - expect(result.summary).toMatchObject({ added: 0, removed: 0 }); - }); - - it("keeps a resolved copy out of the declared round when an ownerless competitor is on the right, as main does", () => { - const base = inspect(buildManagedPeFixture(), "/tmp/ownerless-right.dll"); - const [method] = base.result.methods; - if (method === undefined) throw new Error("Expected a fixture method"); - const ownerless = { - ...method, - declaring_type: null, - declaring_type_token: null, - }; - const withOwnerless = { - ...base, - result: { ...base.result, methods: [method, ownerless], fields: [] }, - }; - const resolvedOnly = { - ...base, - result: { ...base.result, methods: [method], fields: [] }, - }; - - const result = compareManagedMembers(resolvedOnly, withOwnerless); - - expect(result.matching.ambiguous).toBe(1); - expect( - result.methods.filter(({ match }) => match.status === "matched"), - ).toEqual([]); - expect(result.summary).toMatchObject({ added: 0, removed: 0 }); - }); + expect(result.matching.ambiguous).toBe(1); + expect( + result.methods.filter(({ match }) => match.status === "matched"), + ).toEqual([]); + expect(result.summary).toMatchObject({ added: 0, removed: 0 }); + }, + ); it("keeps an ownerless method ambiguous beside a resolved method of the same shape", () => { const base = inspect(buildManagedPeFixture(), "/tmp/ownerless-shape.dll"); diff --git a/tests/boundary/providers/managed/managedStaticProvider.test.ts b/tests/boundary/providers/managed/managedStaticProvider.test.ts index 6e0dd5b9b0..81eecd7970 100644 --- a/tests/boundary/providers/managed/managedStaticProvider.test.ts +++ b/tests/boundary/providers/managed/managedStaticProvider.test.ts @@ -27,11 +27,13 @@ import * as boundedFileBytes from "../../../../src/process/BoundedFileBytes.js"; import { primaryAnalysisError } from "../../../../src/domain/analysisErrorBase.js"; it.each( - [false, true].flatMap((warm) => - [false, true].flatMap((changed) => - [false, true].map((failedClose) => ({ warm, changed, failedClose })), - ), - ), + [false, true] + .flatMap((warm) => + [false, true].flatMap((changed) => + [false, true].map((failedClose) => ({ warm, changed, failedClose })), + ), + ) + .filter(({ changed, failedClose }) => changed || failedClose), )( "preserves selected managed identity through finalization (cached=$warm, changed=$changed, failedClose=$failedClose)", async ({ warm, changed, failedClose }) => { @@ -291,8 +293,9 @@ describe("managed static completed snapshot cancellation", () => { kind: "file-snapshot-hash", path, bytes: bytes.length, - sha256: parsed.value.sha256, + sha256: createHash("sha256").update(bytes).digest("hex"), }; + expect(parsed.value.sha256).toBe(observation.sha256); expect(result.error.partialObservation).toEqual(observation); expect(projectAnalysisError(result.error)).toMatchObject({ code: "cancelled", @@ -302,6 +305,9 @@ describe("managed static completed snapshot cancellation", () => { expect(owners.every((owner) => owner.handle.fd === -1)).toBe(true); if (retryClose) expect(owners[0]).toBe(owners[1]); expect(await client.close()).toEqual({ ok: true, value: null }); + expect(result.error.partialObservation).toEqual(observation); + expect(result.error.cleanupIncomplete).toBe(false); + expect(owners.every((owner) => owner.handle.fd === -1)).toBe(true); } finally { close.mockRestore(); await client.close(); @@ -529,15 +535,11 @@ describe("managed static provider path boundary", () => { field_accesses: [expect.objectContaining({ field_name: "counter" })], }); const memberResult = asManagedMemberResult(members.value); - expect(members.value.locations).toEqual( - expect.arrayContaining( - memberResult.methods.flatMap((method) => - method.body.file_offset === null - ? [] - : [{ kind: "file-offset", offset: method.body.file_offset }], - ), - ), - ); + expect(memberResult.methods[0]?.body.file_offset).toBe(0x0a00); + expect(members.value.locations).toContainEqual({ + kind: "file-offset", + offset: 0x0a00, + }); const boundaries = await client.execute( "inspect_managed_native_boundaries", diff --git a/tests/boundary/providers/readinessRemediationInventory.test.ts b/tests/boundary/providers/readinessRemediationInventory.test.ts new file mode 100644 index 0000000000..198f56428d --- /dev/null +++ b/tests/boundary/providers/readinessRemediationInventory.test.ts @@ -0,0 +1,98 @@ +import { join } from "node:path"; +import { expect, it, vi } from "vitest"; + +import type { CapabilityDescriptor } from "../../../src/application/AnalysisProvider.js"; +import { buildCapabilityInventory } from "../../../src/application/CapabilityInventory.js"; +import { providerCapabilitySets } from "../../../src/application/binary/BinarySessionStatus.js"; +import { parseConfig } from "../../../src/config/parseConfig.js"; +import { IdaProvider } from "../../../src/ida/IdaProvider.js"; +import { readKunaConfiguration } from "../../../src/kuna/KunaConfiguration.js"; +import { KunaProvider } from "../../../src/kuna/KunaProvider.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { inspectKunaProviderOwned } from "./kuna/kunaResourceScope.js"; + +// A controlled open target exercises only the final discovery projection. +const openTargetInventory = (capabilities: readonly CapabilityDescriptor[]) => + new Map( + buildCapabilityInventory( + { + open: true, + kind: "executable", + format: "elf", + capabilities: providerCapabilitySets(capabilities), + }, + { processCaptureEnabled: true }, + ).map((entry) => [entry.name, entry]), + ); + +it("carries an unreadable IDA registration's launch-environment repair into discovery", async () => { + const root = await createTestTempDirectory("rea-ida-discovery-repair-"); + const config = parseConfig({ + REA_IDA_MCP_CONFIG: join(root, "missing.json"), + }); + if (!config.ok) throw config.error; + const connect = vi.fn(() => { + throw new Error("Readiness must not connect to IDA"); + }); + const provider = new IdaProvider(config.value, {}, connect); + try { + const { availability, capabilities } = + await provider.inspectReadinessSnapshot(); + if (availability.status !== "unavailable") + throw new Error("A missing registration must be unavailable"); + expect(availability.remediation).toContain("REA_IDA_MCP_CONFIG"); + expect(availability.remediation).toContain("environment that starts REA"); + expect(availability.remediation).not.toBe(availability.reason); + const inventory = openTargetInventory(capabilities); + for (const name of ["analyze_function", "procedure_pseudo_code"]) + expect(inventory.get(name)).toMatchObject({ + available: false, + reason: "provider_unavailable", + remediation: availability.remediation, + }); + expect(connect).not.toHaveBeenCalled(); + } finally { + const closed = await provider.closeOwnedResources(); + if (!closed.ok) throw closed.error; + } +}); + +it.skipIf(process.platform === "win32").each([ + { setting: "REA_KUNA_JOBS", value: "33" }, + { setting: "REA_KUNA_TIMEOUT_MS", value: "0" }, + { setting: "REA_KUNA_CACHE_DIR", value: "relative" }, +])( + "keeps the Kuna configuration owner's repair for invalid $setting through discovery and doctor", + async ({ setting, value }) => { + // Configuration validation refuses the setting before any command probe. + const environment = { + REA_KUNA_COMMAND: "/audit/not-invoked/kuna", + [setting]: value, + }; + const owner = readKunaConfiguration(environment); + if (owner.ok) throw new Error(`${setting}=${value} must be refused`); + expect(owner.error.remediation).toContain(setting); + expect(owner.error.remediation).not.toBe(owner.error.reason); + + const provider = new KunaProvider(environment); + expect(provider.inspectAvailability()).toMatchObject({ + status: "unavailable", + reason: owner.error.reason, + remediation: owner.error.remediation, + }); + const inventory = openTargetInventory(provider.capabilities()); + for (const name of ["analyze_function", "procedure_pseudo_code"]) + expect(inventory.get(name)).toMatchObject({ + available: false, + reason: "provider_unavailable", + remediation: owner.error.remediation, + }); + + // The doctor path already reports the same owner repair. + const doctor = await inspectKunaProviderOwned(environment); + expect(doctor.checks[0]).toMatchObject({ + ok: false, + remediation: owner.error.remediation, + }); + }, +); diff --git a/tests/boundary/setup/linuxHopperPublisherVerification.test.ts b/tests/boundary/setup/linuxHopperPublisherVerification.test.ts index 29f00a0781..1409f848e9 100644 --- a/tests/boundary/setup/linuxHopperPublisherVerification.test.ts +++ b/tests/boundary/setup/linuxHopperPublisherVerification.test.ts @@ -790,11 +790,19 @@ it("bounds a stalled metadata fetch and preserves caller cancellation", async () }), ), ); - await expect( - downloadHopperPackage("https://www.hopperapp.com/fixture", { + // REA's own deadline is a returned transport fact; only caller cancellation rejects. + expect( + await downloadHopperPackage("https://www.hopperapp.com/fixture", { timeoutMs: 10, }), - ).rejects.toMatchObject({ name: "TimeoutError" }); + ).toEqual({ + ok: false, + error: { + url: "https://www.hopperapp.com/fixture", + kind: "timeout", + timeoutMs: 10, + }, + }); const controller = new AbortController(); const reason = new Error("Caller cancelled metadata inspection"); const pending = downloadHopperPackage("https://www.hopperapp.com/fixture", { @@ -823,7 +831,14 @@ it("cancels an oversized chunked metadata response before retaining it", async ( await downloadHopperPackage("https://www.hopperapp.com/fixture", { maxBytes: 3, }), - ).toEqual({ ok: false, bytes: new Uint8Array() }); + ).toEqual({ + ok: false, + error: { + url: "https://www.hopperapp.com/fixture", + kind: "response_too_large", + maxBytes: 3, + }, + }); expect(cancelled).toHaveBeenCalledOnce(); }); diff --git a/tests/boundary/setup/macosHopperPublisherVerification.test.ts b/tests/boundary/setup/macosHopperPublisherVerification.test.ts index dcb85cca5e..32488043c7 100644 --- a/tests/boundary/setup/macosHopperPublisherVerification.test.ts +++ b/tests/boundary/setup/macosHopperPublisherVerification.test.ts @@ -17,7 +17,7 @@ const run = promisify(execFile); const itDarwin = it.skipIf(process.platform !== "darwin"); const executableSegments = ["Contents", "MacOS", "hopper"]; -const infoPlist = (identifier: string): string => +const infoPlist = (identifier: string, entries = ""): string => ` @@ -27,7 +27,7 @@ const infoPlist = (identifier: string): string => CFBundleExecutable hopper CFBundlePackageType - APPL + APPL${entries} `; @@ -56,8 +56,25 @@ const buildBundle = async ( return app; }; +const minimumEntry = (value: string): string => + `\n LSMinimumSystemVersion\n ${value}`; + +/** An unsigned bundle carrying only an Info.plist, for requirement reads. */ +const infoPlistBundle = async (entries: string): Promise => { + const root = await createTestTempDirectory("rea-hopper-minimum-"); + const app = join(root, "Hopper Disassembler.app"); + await mkdir(join(app, "Contents"), { recursive: true }); + await writeFile( + join(app, "Contents", "Info.plist"), + infoPlist(HOPPER_PUBLISHER.bundleIdentifier, entries), + ); + return app; +}; + const host = () => - systemMacHopperInstallHost("/home/rea-test", { ...process.env }); + systemMacHopperInstallHost("/home/rea-test", { ...process.env }, () => + Promise.resolve(undefined), + ); describe("macOS Hopper publisher verification", () => { itDarwin( @@ -89,6 +106,36 @@ describe("macOS Hopper publisher verification", () => { await expect(host().validBundle(app)).resolves.toBe("invalid_signature"); }); + it.each([ + { entries: minimumEntry("12.4"), declared: "12.4" }, + { entries: "", declared: undefined }, + ])( + "reads the XML Info.plist minimum macOS version $declared", + async ({ entries, declared }) => { + const app = await infoPlistBundle(entries); + await expect(host().minimumSystemVersion(app)).resolves.toBe(declared); + }, + ); + + itDarwin("reads a binary Info.plist minimum macOS version", async () => { + const app = await infoPlistBundle(minimumEntry("13.0.1")); + await run("/usr/bin/plutil", [ + "-convert", + "binary1", + join(app, "Contents", "Info.plist"), + ]); + await expect(host().minimumSystemVersion(app)).resolves.toBe("13.0.1"); + }); + + it("refuses a minimum macOS version that is not text", async () => { + const app = await infoPlistBundle(minimumEntry("12.4")); + const failure = host().minimumSystemVersion(app); + await expect(failure).rejects.toBeInstanceOf(TypeError); + // The refusal names the key and the exact plist it read. + await expect(failure).rejects.toThrow("LSMinimumSystemVersion"); + await expect(failure).rejects.toThrow(join(app, "Contents", "Info.plist")); + }); + it("discloses the exact code requirement the installer enforces", () => { expect(macHopperInstallDisclosure.commands).toContain( `codesign --verify --strict '-R= identifier "${HOPPER_PUBLISHER.bundleIdentifier}" and anchor apple generic and certificate leaf[subject.OU] = "${HOPPER_PUBLISHER.teamIdentifier}"' `, @@ -122,6 +169,9 @@ describe("macOS Hopper publisher verification", () => { "accepts the officially signed Hopper application", async () => { await expect(host().validBundle(realApp ?? "")).resolves.toBe("verified"); + await expect(host().minimumSystemVersion(realApp ?? "")).resolves.toMatch( + /^\d+(?:\.\d+)*$/u, + ); await expect( verifyMacHopperNativeHelper( join(realApp ?? "", ...executableSegments), diff --git a/tests/composition/analysis-sessions/binarySession.test.ts b/tests/composition/analysis-sessions/binarySession.test.ts index 18bcf0c1b0..ca4b34d3e8 100644 --- a/tests/composition/analysis-sessions/binarySession.test.ts +++ b/tests/composition/analysis-sessions/binarySession.test.ts @@ -8,6 +8,7 @@ import { parseBinaryTarget } from "../../../src/application/BinaryTargetResolver import type { JsonValue } from "../../../src/domain/jsonValue.js"; import type { RecordUnknownInput } from "../../../src/domain/residualUnknown.js"; import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js"; +import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; import { HopperStartError } from "../../../src/domain/hopperErrors.js"; import { ProviderAdapterError } from "../../../src/domain/providerAdapterError.js"; import { ProviderCleanupError } from "../../../src/domain/providerCleanupError.js"; @@ -309,9 +310,10 @@ it("reports an external cleanup blocker until its exact owner confirms release", ); expect(await session.status()).toMatchObject({ cleanup_pending: true }); expect(await session.open(target)).toEqual(err(failure)); - await session.close(); + await session.close({ cleanupRetry: { operation: "close_binary" } }); expect(released).toBe(false); expect(await session.status()).toMatchObject({ cleanup_pending: true }); + expect(await session.status()).not.toHaveProperty("cleanup_retry"); await lease.release(); expect(await session.status()).toMatchObject({ cleanup_pending: false }); expect((await session.open(target)).ok).toBe(true); @@ -321,6 +323,63 @@ it("reports an external cleanup blocker until its exact owner confirms release", } }); +it("does not offer close retry while an external owner also blocks cleanup", async () => { + const [target] = await createBinarySessionTargets(); + const externalFailure = new ProviderCleanupError( + "external-debugger", + ["external-run"], + {}, + ); + const clientFailure = new ProviderCleanupError("fixture", ["client-run"], {}); + let clientReleased = false; + const externalLease = new AnalysisCleanupLease(async () => {}, [ + "external-run", + ]); + const clientLease = new AnalysisCleanupLease(async () => { + clientReleased = true; + }, ["client-run"]); + externalLease.bindCleanupError(externalFailure); + clientLease.bindCleanupError(clientFailure); + const provider = createCacheProvider([]); + const session = createTestBinarySession({ + ...provider, + createClient: (...args) => { + const client = provider.createClient(...args); + return { + ...client, + close: () => + clientReleased ? client.close() : Promise.resolve(err(clientFailure)), + }; + }, + }); + try { + expect((await session.open(target)).ok).toBe(true); + await session.execute("address_name", { + address: "0x1000", + document: "fixture", + }); + await session.withAdmittedExternalMutation( + target, + undefined, + async () => err(externalFailure), + (_value, error) => error, + ); + expect( + (await session.close({ cleanupRetry: { operation: "close_binary" } })).ok, + ).toBe(false); + expect(await session.status()).toMatchObject({ + open: false, + cleanup_pending: true, + }); + expect(await session.status()).not.toHaveProperty("cleanup_retry"); + } finally { + await clientLease.release(); + await externalLease.release(); + const closed = await session.close(); + if (!closed.ok) throw closed.error; + } +}); + it("retires analysis when an external mutation uses a hard link", async () => { const [target] = await createBinarySessionTargets(); const alias = join(dirname(target), "hard-link-alias.hop"); @@ -906,7 +965,17 @@ describe("replay of exact immutable calls", () => { expect(mismatch.importAnalysisSnapshot(snapshot.value).ok).toBe(true); const opened = await mismatch.open(second); expect(opened.ok).toBe(false); - if (!opened.ok) expect(opened.error._tag).toBe("EvidenceIntegrityError"); + if (!opened.ok) { + expect(opened.error._tag).toBe("EvidenceIntegrityError"); + expect(projectAnalysisError(opened.error).details).toMatchObject({ + binding_mismatch: { + target: { + snapshot: { sha256: snapshot.value.target.sha256 }, + selected: { sha256: expect.any(String) }, + }, + }, + }); + } await mismatch.close(); const profileMismatchCalls: string[] = []; @@ -933,7 +1002,14 @@ describe("replay of exact immutable calls", () => { expect(profileOpened.ok).toBe(false); if (!profileOpened.ok) { expect(profileOpened.error._tag).toBe("EvidenceIntegrityError"); - expect(profileOpened.error.message).toContain("profile_mismatch"); + expect(projectAnalysisError(profileOpened.error).details).toMatchObject({ + binding_mismatch: { + profile_digest: { + snapshot: snapshot.value.binding.analysis_profile.digest, + selected: expect.any(String), + }, + }, + }); } expect(profileMismatchCalls).toEqual([]); await profileMismatch.close(); @@ -946,8 +1022,16 @@ describe("replay of exact immutable calls", () => { snapshot.value, ); expect(activeImport.ok).toBe(false); - if (!activeImport.ok) - expect(activeImport.error.message).toContain("profile_mismatch"); + if (!activeImport.ok) { + expect(projectAnalysisError(activeImport.error).details).toMatchObject({ + binding_mismatch: { + profile_digest: { + snapshot: snapshot.value.binding.analysis_profile.digest, + selected: expect.any(String), + }, + }, + }); + } await activeProfileMismatch.close(); }); }); diff --git a/tests/composition/analysis-sessions/providerBinding.test.ts b/tests/composition/analysis-sessions/providerBinding.test.ts index afc2ef9b54..efd8bd7808 100644 --- a/tests/composition/analysis-sessions/providerBinding.test.ts +++ b/tests/composition/analysis-sessions/providerBinding.test.ts @@ -307,6 +307,7 @@ const deepProvider = (id: string): DeepProviderFixture => { status: "unavailable", code: "runtime_missing", reason: `${id} runtime is unavailable`, + remediation: "Repair the fixture provider.", diagnostics: { executable_path: `/opt/${id}/bin/analyze` }, }, inspectTargetSupport: () => ({ diff --git a/tests/composition/directAnalysisFactories.test.ts b/tests/composition/directAnalysisFactories.test.ts index b4c58f1411..a10562967e 100644 --- a/tests/composition/directAnalysisFactories.test.ts +++ b/tests/composition/directAnalysisFactories.test.ts @@ -1,7 +1,13 @@ import { projectCliAnalysisResult } from "../../src/cli/projectCliAnalysisResult.js"; -import { EvidenceFileError } from "../../src/domain/evidenceErrors.js"; +import { + AnalysisSnapshotMismatchError, + EvidenceFileError, +} from "../../src/domain/evidenceErrors.js"; +import { analysisErrorRemediationAction } from "../../src/domain/analysisErrorPresentation.js"; import { err, ok as resultOk } from "../../src/domain/result.js"; import { dirname, join } from "node:path"; +import { createHash } from "node:crypto"; +import { readFile } from "node:fs/promises"; import { parseConfig } from "../../src/config/parseConfig.js"; import { describe, expect, it, vi } from "vitest"; import { @@ -312,11 +318,25 @@ it("direct analysis snapshot target mismatch retains actionable snapshot recover {}, { snapshotPath }, ).then(projectCliAnalysisResult); + const sha = async (input: string) => + createHash("sha256") + .update(await readFile(input)) + .digest("hex"); expect(mismatch).toMatchObject({ code: "evidence_integrity_mismatch", remediation: { - action: - "Run analysis without this snapshot, then save a fresh snapshot using the intended artifact, provider, and analysis profile.", + action: analysisErrorRemediationAction( + new AnalysisSnapshotMismatchError(""), + ), + }, + details: { + binding_mismatch: { + target: { + snapshot: { sha256: await sha(path) }, + selected: { sha256: await sha(other) }, + }, + profile_comparison: "not_compared", + }, }, }); expect( diff --git a/tests/composition/idaProvider.test.ts b/tests/composition/idaProvider.test.ts index 2c6c08f934..eda322f510 100644 --- a/tests/composition/idaProvider.test.ts +++ b/tests/composition/idaProvider.test.ts @@ -236,6 +236,8 @@ describe("IDA provider selection cancellation", () => { ok: false, error: { _tag: "AnalysisCancelledError", cause: cancellationReason }, }); + if (replacement.ok) throw new Error("Expected cancellation"); + expect(replacement.error.cause).toBe(cancellationReason); expect(session.activeTarget()?.sha256).toBe(first.target.sha256); expect( (await session.execute("procedure_pseudo_code", { procedure: "main" })) @@ -272,6 +274,8 @@ describe("IDA provider selection cancellation", () => { ok: false, error: { _tag: "AnalysisCancelledError", cause: reason }, }); + if (profile.ok) throw new Error("Expected cancellation"); + expect(profile.error.cause).toBe(reason); expect(runCalls).toBe(0); }); }); diff --git a/tests/conformance/ghidra/README.md b/tests/conformance/ghidra/README.md index 419a3bc7d8..580dc41820 100644 --- a/tests/conformance/ghidra/README.md +++ b/tests/conformance/ghidra/README.md @@ -25,11 +25,15 @@ binaries and Ghidra projects are never committed. multi-block branch. - `entry-aliases.c` preserves secondary imported entry labels, a bare-hex alias, and an interior label. The native CLI/MCP lane verifies entry selection, - annotation, interior-label rejection, and unchanged source bytes. + procedure search and feature tracing by the secondary name, annotation, + interior-label rejection, and unchanged source bytes. - `cross-format.c` is freestanding so the verifier can produce AArch64 ELF, x86-64 PE, and x86-64 Mach-O targets from the same semantics. It preserves an exported entry, direct and indirect calls, a volatile string reference, and a multi-block branch across loaders. +- `relocation-symbol.c` builds an x86-64 ELF with named and unnamed dynamic + relocations. The cross-format verifier checks their Ghidra observations through + the public CLI without executing the artifact. - `no-return.c` and `ReaNoReturnProbe.java` inject incorrect import flags into a real ELF or Mach-O analysis database, verify caller recovery, propagated flag repair and signature preservation, and retain genuine/unknown no-return calls diff --git a/tests/conformance/ghidra/relocation-symbol.c b/tests/conformance/ghidra/relocation-symbol.c new file mode 100644 index 0000000000..30b1fd4fe5 --- /dev/null +++ b/tests/conformance/ghidra/relocation-symbol.c @@ -0,0 +1,7 @@ +/* Static ELF fixture: no target code is executed by the verifier. */ +extern int rea_external(int); +int (*volatile rea_external_slot)(int) = rea_external; +__attribute__((visibility("hidden"), noinline, used)) int rea_local(int value) { + return value + 1; +} +int (*volatile rea_local_slot)(int) = rea_local; diff --git a/tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts b/tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts index 0e936b0624..3ec28e10c4 100644 --- a/tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts +++ b/tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts @@ -19,6 +19,17 @@ describe("real Windows Ghidra workflow trust boundary", () => { repository_dispatch: { types: ["real-ghidra-windows"] }, }); expect(workflow.permissions).toEqual({ contents: "read" }); + for (const name of ["native-artifact", "verify"]) { + const job = workflow.jobs[name]; + expect(job.if).toBe("github.ref == 'refs/heads/main'"); + const checkouts = job.steps.filter((step: { uses?: string }) => + step.uses?.startsWith("actions/checkout@"), + ); + expect(checkouts).toHaveLength(1); + expect(checkouts[0].uses).toMatch(/^actions\/checkout@[0-9a-f]{40}$/); + expect(checkouts[0].with?.ref).toBeUndefined(); + expect(checkouts[0].with?.repository).toBeUndefined(); + } expect(workflow).toMatchObject({ jobs: { verify: { diff --git a/tests/conformance/providers/managed/managedNativeVerification.test.ts b/tests/conformance/providers/managed/managedNativeVerification.test.ts index 60835973c1..e86a3a4318 100644 --- a/tests/conformance/providers/managed/managedNativeVerification.test.ts +++ b/tests/conformance/providers/managed/managedNativeVerification.test.ts @@ -913,22 +913,6 @@ it("retains every observed native candidate", () => { expect(result.pinvoke_imports[0]?.candidates).toHaveLength(2); }); -it("wraps verification in derived workflow Evidence", () => { - const evidence = executeManagedNativeBoundaries(exampleInput()); - - if (!evidence.ok) throw evidence.error; - expect(evidence.value.evidence).toMatchObject({ - operation: "verify_managed_native_boundaries", - provider: { id: "rea-dotnet-workflows" }, - confidence: "inferred", - authority: "analyst-inference", - normalized_result: { - summary: { verified: 1 }, - }, - }); - expect(evidence.value.evidence.evidence_links).toHaveLength(2); -}); - describe("managed/native verification result algebra", () => { it("rejects states without their required native match", () => { const result = verifyManagedNativeVerificationSource( diff --git a/tests/evaluation/evidenceJsonPreadmissionRule.test.ts b/tests/evaluation/evidenceJsonPreadmissionRule.test.ts index fc435b3713..850a927070 100644 --- a/tests/evaluation/evidenceJsonPreadmissionRule.test.ts +++ b/tests/evaluation/evidenceJsonPreadmissionRule.test.ts @@ -216,14 +216,6 @@ new RuleTester({ createEvidence(target, provider, { rawResult: jsonValueSchema.parse(raw) });`, "rawResult", ), - invalid( - "aliased imports retain their owner", - ` - import { createEvidence as record } from "../domain/evidence.js"; - import { jsonValueSchema as json } from "../domain/jsonValue.js"; - record(target, provider, { result: json.parse(input) });`, - "result", - ), invalid( "namespace members retain their owner", ` diff --git a/tests/evaluation/model-evals/agentEvaluationReport.test.ts b/tests/evaluation/model-evals/agentEvaluationReport.test.ts index 8d2f8375cd..9c4566c058 100644 --- a/tests/evaluation/model-evals/agentEvaluationReport.test.ts +++ b/tests/evaluation/model-evals/agentEvaluationReport.test.ts @@ -131,12 +131,6 @@ describe("scoped factual report", () => { epistemicCuePresent: false, }), ).toBe(false); - expect( - agentEvaluationPassed({ - ...routingSuccess, - factualCorrectness: "failed", - }), - ).toBe(false); }); it("reports assessed failures separately from unassessed scenarios", () => { diff --git a/tests/evaluation/testDiscovery.test.ts b/tests/evaluation/testDiscovery.test.ts index d4ef905089..70996f2236 100644 --- a/tests/evaluation/testDiscovery.test.ts +++ b/tests/evaluation/testDiscovery.test.ts @@ -15,7 +15,23 @@ describe("test discovery migration guard", () => { [{ file: resolve(root, first), projectName: "domain" }], root, ), - ).toEqual({ missing: [], unexpected: [], duplicates: [], discovered: 1 }); + ).toEqual({ + missing: [], + unexpected: [], + duplicates: [], + wrongProjects: [], + discovered: 1, + }); + }); + + it("reports a discovered test in the wrong project", () => { + expect( + compareTestDiscovery( + [first], + [{ file: first, projectName: "cli-boundary" }], + root, + ).wrongProjects, + ).toEqual([{ file: first, expected: "domain", actual: "cli-boundary" }]); }); it("reports a new provider test omitted by the current project globs", () => { diff --git a/tests/fixtures/android/jniElf.ts b/tests/fixtures/android/jniElf.ts index b6cdaef0f9..f76686da5b 100644 --- a/tests/fixtures/android/jniElf.ts +++ b/tests/fixtures/android/jniElf.ts @@ -328,5 +328,5 @@ export const jniElfFixture = ( }, writer, ); - return { bytes, base, word, functions, slots, relocationSize }; + return { bytes, base, word }; }; diff --git a/tests/fixtures/binaryDiagnostics/appleIps.ts b/tests/fixtures/binaryDiagnostics/appleIps.ts index 4bb557fee7..61bb9eb4ac 100644 --- a/tests/fixtures/binaryDiagnostics/appleIps.ts +++ b/tests/fixtures/binaryDiagnostics/appleIps.ts @@ -1,4 +1,4 @@ -export interface AppleIpsFixtureOptions { +interface AppleIpsFixtureOptions { readonly bugType?: string; readonly procName?: string; readonly uuid?: string; diff --git a/tests/fixtures/binaryDiagnostics/nativeExports.ts b/tests/fixtures/binaryDiagnostics/nativeExports.ts index ea6a1a644b..c055b48cf8 100644 --- a/tests/fixtures/binaryDiagnostics/nativeExports.ts +++ b/tests/fixtures/binaryDiagnostics/nativeExports.ts @@ -1,5 +1,5 @@ /** One PE export-address-table slot: code RVA, forwarder text, or unused. */ -export type PeFixtureFunction = +type PeFixtureFunction = | { readonly rva: number } | { readonly forwarder: string } | null; @@ -132,7 +132,7 @@ export const peRepeatedExportNamesImage = ( }; /** One `.dynsym` entry; `version` is its raw `.gnu.version` half-word. */ -export interface ElfFixtureSymbol { +interface ElfFixtureSymbol { readonly name: string; readonly binding?: number; readonly type?: number; @@ -145,7 +145,7 @@ export interface ElfFixtureSymbol { readonly nameOffset?: number; } -export interface ElfFixtureVersionRequirement { +interface ElfFixtureVersionRequirement { readonly index: number; readonly name: string; } diff --git a/tests/fixtures/binaryDiagnostics/peImports.ts b/tests/fixtures/binaryDiagnostics/peImports.ts index bdbdce47b2..5c775b72d4 100644 --- a/tests/fixtures/binaryDiagnostics/peImports.ts +++ b/tests/fixtures/binaryDiagnostics/peImports.ts @@ -8,11 +8,11 @@ const SPLIT_GAP_BYTES = 0x200; export const peImportFixtureOffset = (rva: number): number => rva - SECTION_RVA + SECTION_OFFSET; -export type PeImportFixtureFunction = +type PeImportFixtureFunction = | { readonly name: string; readonly hint?: number } | { readonly ordinal: number }; -export interface PeImportFixtureDll { +interface PeImportFixtureDll { readonly name: string; readonly functions: readonly PeImportFixtureFunction[]; } diff --git a/tests/fixtures/browserFixtureHttpTargets.mjs b/tests/fixtures/browserFixtureHttpTargets.mjs new file mode 100644 index 0000000000..022e904f56 --- /dev/null +++ b/tests/fixtures/browserFixtureHttpTargets.mjs @@ -0,0 +1,160 @@ +import assert from "node:assert/strict"; +import { once } from "node:events"; +import { createServer, request } from "node:http"; +import { networkInterfaces } from "node:os"; +import { + closeVerifierResources, + withVerifierCleanup, +} from "../../scripts/lib/verifier-cleanup.mjs"; + +const kind = process.argv[2]; +const owned = []; +let alternateRequests = 0; +let site; + +const upstream = async (name) => { + const server = createServer((incoming, response) => { + if (name === "alternate") alternateRequests += 1; + response.setHeader("Content-Type", "application/json"); + response.end(JSON.stringify({ name, target: incoming.url })); + }); + owned.push(server); + server.listen(0, "127.0.0.1"); + await once(server, "listening"); + return `http://127.0.0.1:${server.address().port}`; +}; + +const send = (origin, target) => + new Promise((resolve, reject) => { + const selected = new URL(origin); + const pending = request( + { + hostname: selected.hostname, + port: selected.port, + path: target, + timeout: 2_000, + }, + (response) => { + const chunks = []; + response.on("data", (chunk) => chunks.push(chunk)); + response.on("error", reject); + response.on("end", () => + resolve({ + status: response.statusCode, + headers: response.headers, + body: Buffer.concat(chunks).toString("utf8"), + }), + ); + }, + ); + pending.on("error", reject); + pending.on("timeout", () => + pending.destroy(new Error("Fixture HTTP request timed out")), + ); + pending.end(); + }); + +await withVerifierCleanup(async () => { + let origin; + let ordinary; + let expected; + if (kind === "proxy") { + const configured = await upstream("configured"); + const alternate = await upstream("alternate"); + const { startProxy } = + await import("../../scripts/lib/browser-scenario-service-worker-setup-e2e.mjs"); + site = await startProxy(configured, "http-discovery"); + origin = site.endpoint; + ordinary = "/probe?x=1"; + expected = JSON.stringify({ name: "configured", target: ordinary }); + for (const [target, path] of [ + ["/probe?x=a%20b&x=2", "/probe?x=a%20b&x=2"], + [`${alternate}/probe?absolute=1`, "/probe?absolute=1"], + [ + `${alternate.replace("http:", "")}/probe?x=1`, + `${alternate.replace("http:", "")}/probe?x=1`, + ], + ["https://unused.invalid/probe?escaped=%2f", "/probe?escaped=%2f"], + ["/%40scope%2fmodule?x=%252f", "/%40scope%2fmodule?x=%252f"], + ]) { + const reply = await send(origin, target); + assert.equal(reply.status, 200); + assert.deepEqual(JSON.parse(reply.body), { + name: "configured", + target: path, + }); + } + assert.equal(alternateRequests, 0); + } else if (kind === "workers") { + const { startSite } = + await import("../../scripts/lib/browser-scenario-service-workers-e2e.mjs"); + site = await startSite(); + origin = site.origin; + ordinary = "/probe"; + expected = "network"; + } else { + assert.equal(kind, "storage"); + const { startStorageSite } = + await import("../../scripts/lib/browser-scenario-storage-e2e.mjs"); + site = await startStorageSite(); + origin = site.origin; + ordinary = "/storage-main"; + } + const checkOrdinary = async () => { + const reply = await send(origin, ordinary); + assert.equal(reply.status, 200); + if (expected !== undefined) assert.equal(reply.body, expected); + else assert(reply.body.includes('id="main-report"')); + }; + await checkOrdinary(); + if (process.argv[3] === "listener-scope") { + assert.equal(kind, "storage"); + const named = new URL(origin); + named.hostname = "localhost"; + const localhost = await send(named.href, ordinary); + assert.equal(localhost.status, 200); + assert(localhost.body.includes('id="main-report"')); + const address = Object.values(networkInterfaces()) + .flatMap((entries) => entries ?? []) + .find((entry) => entry.family === "IPv4" && !entry.internal); + assert( + address, + "Listener scope probe requires an owned non-loopback IPv4 interface", + ); + const nonLoopback = new URL(origin); + nonLoopback.hostname = address.address; + await assert.rejects(send(nonLoopback.href, ordinary), { + code: "ECONNREFUSED", + }); + console.log("storage: localhost served; non-loopback connection refused"); + } + for (const target of [ + "http://[", + "ftp://127.0.0.1/probe", + "ws://127.0.0.1/probe", + ]) { + const reply = await send(origin, target); + assert.equal(reply.status, 400); + assert.equal(reply.headers["content-type"], "text/plain; charset=utf-8"); + assert.equal(reply.headers["x-content-type-options"], "nosniff"); + assert.equal(reply.body, "Invalid HTTP request target"); + await checkOrdinary(); + } + await site.close(); + site = undefined; + await assert.rejects(send(origin, ordinary)); + console.log( + `${kind}: target routing, invalid-target recovery and physical close passed`, + ); +}, [ + () => site?.close(), + () => + closeVerifierResources( + owned.map((server) => async () => { + server.closeAllConnections(); + await new Promise((resolve, reject) => + server.close((cause) => (cause ? reject(cause) : resolve())), + ); + }), + ), +]); diff --git a/tests/fixtures/cutter_bridge_protocol_test.py b/tests/fixtures/cutter_bridge_protocol_test.py index fd605f7cdd..957731cf18 100644 --- a/tests/fixtures/cutter_bridge_protocol_test.py +++ b/tests/fixtures/cutter_bridge_protocol_test.py @@ -143,6 +143,7 @@ class ServeDispatcher: elif request.get("kind") == "ack": if self.malformed_next_ack: self.malformed_next_ack = False + self.hide_receipts_after_ack_failure = True reply.update({"ok": True, "session_id": request["session_id"]}) request["_event"].set() return @@ -198,7 +199,11 @@ class ServeDispatcher: else: reply.update({ "ok": True, - "output": 'round-trip: 雪 ☃️ quote=" slash=\\\\ line=\\n tab=\\t', + "output": ( + str(len(self.completed_commands)) + if request.get("command") == "fixture-receipt-count" + else 'round-trip: 雪 ☃️ quote=" slash=\\\\ line=\\n tab=\\t' + ), "current_file": self.current_file, "document_generation": self.generation, "cutter_version": self.cutter_version, @@ -601,14 +606,17 @@ class ProtocolProducerTest(unittest.TestCase): dispatcher.completed_commands = {} dispatcher._receipt_lock = threading.Lock() dispatcher._refresh_document_identity = lambda: None - bridge.cutter.cmd = lambda command: f"output:{command}" + commands = [] + bridge.cutter.cmd = lambda command: commands.append(command) or f"output:{command}" - def handle(value): + def handle(value, cancelled=False): request = { **value, "_event": threading.Event(), "_cancelled": threading.Event(), } + if cancelled: + request["_cancelled"].set() reply = {} bridge._RequestDispatcher._handle(dispatcher, request, reply) self.assertTrue(request["_event"].is_set()) @@ -616,6 +624,15 @@ class ProtocolProducerTest(unittest.TestCase): request_a = "4b18ce4d-bf31-456e-ae31-e8c547b2f9db" request_b = "8ca93d11-107d-41ed-8b40-31dd53cd26a6" + request_cancelled = "9c8074cf-4906-4cc1-ac92-3fca3d305f8f" + cancelled = handle({ + "kind": "command", + "request_id": request_cancelled, + "expected_generation": 4, + "command": "cancelled", + }, cancelled=True) + self.assertEqual(cancelled["error"], "request-cancelled-before-execution") + self.assertEqual(cancelled["execution_state"], "not_started") for request_id, command in ((request_a, "A"), (request_b, "B")): result = handle({ "kind": "command", @@ -624,7 +641,14 @@ class ProtocolProducerTest(unittest.TestCase): "command": command, }) self.assertTrue(result["ok"]) + self.assertEqual(commands, ["A", "B"]) + status = handle({"kind": "status"}) + self.assertEqual( + {receipt["request_id"]: receipt["execution_state"] for receipt in status["completed_commands"]}, + {request_cancelled: "not_started", request_a: "complete", request_b: "complete"}, + ) + self.assertTrue(handle({"kind": "ack", "request_id": request_cancelled})["acknowledged"]) status = handle({"kind": "status"}) self.assertEqual( {receipt["request_id"] for receipt in status["completed_commands"]}, @@ -757,23 +781,12 @@ class ProtocolProducerTest(unittest.TestCase): class RequestFrameLimitTest(unittest.TestCase): def test_payload_limit_excludes_the_line_terminator(self): - class BufferedSocket: - def __init__(self, data): - self.data = data - - def settimeout(self, _timeout): - pass - - def recv(self, count): - part, self.data = self.data[:count], self.data[count:] - return part - payload = b"{}" + b" " * (bridge.MAX_FRAME_BYTES - 2) self.assertEqual( - bridge._BridgeServer._receive_frame(BufferedSocket(payload + b"\n")), {} + bridge._BridgeServer._receive_frame(_BufferedSocket(payload + b"\n")), {} ) with self.assertRaisesRegex(ValueError, "frame-limit-exceeded"): - bridge._BridgeServer._receive_frame(BufferedSocket(payload + b" " + b"\n")) + bridge._BridgeServer._receive_frame(_BufferedSocket(payload + b" " + b"\n")) def test_delimiter_in_a_later_chunk_is_accepted(self): class ShortReads: diff --git a/tests/fixtures/electron/fuseImage.ts b/tests/fixtures/electron/fuseImage.ts index d618a375f7..6de380a43a 100644 --- a/tests/fixtures/electron/fuseImage.ts +++ b/tests/fixtures/electron/fuseImage.ts @@ -1,20 +1,13 @@ -import { - ELECTRON_FUSE_SENTINEL, - ELECTRON_FUSE_STATE_BYTES, -} from "../../../src/domain/electron/electronFuses.js"; - -export const FUSE_SENTINEL = Buffer.from(ELECTRON_FUSE_SENTINEL, "ascii"); +// Electron's documented on-disk fuse wire, independent of the reader constants: +// https://github.com/electron/electron/blob/main/docs/tutorial/fuses.md#the-hard-way +export const FUSE_SENTINEL = Buffer.from( + "dL7pKGdnNz796PbbjQWNKmHXBZaB9tsX", + "ascii", +); /** Obsidian's observed version-1 states: disabled, enabled, then six more with the last enabled. */ export const OBSIDIAN_FUSE_BYTES = [ - ELECTRON_FUSE_STATE_BYTES.disabled, - ELECTRON_FUSE_STATE_BYTES.enabled, - ELECTRON_FUSE_STATE_BYTES.disabled, - ELECTRON_FUSE_STATE_BYTES.disabled, - ELECTRON_FUSE_STATE_BYTES.disabled, - ELECTRON_FUSE_STATE_BYTES.disabled, - ELECTRON_FUSE_STATE_BYTES.disabled, - ELECTRON_FUSE_STATE_BYTES.enabled, + 0x30, 0x31, 0x30, 0x30, 0x30, 0x30, 0x30, 0x31, ] as const; export const fuseWire = (version: number, fuses: readonly number[]): Buffer => @@ -56,7 +49,7 @@ export const universalFuseImage = (): { Buffer.concat([FUSE_SENTINEL, fuseWire(1, OBSIDIAN_FUSE_BYTES)]), ); const rightFuses = [...OBSIDIAN_FUSE_BYTES]; - rightFuses[3] = ELECTRON_FUSE_STATE_BYTES.enabled; + rightFuses[3] = 0x31; const right = thinMacho( 0x0100000c, Buffer.concat([FUSE_SENTINEL, fuseWire(1, rightFuses)]), diff --git a/tests/fixtures/environmentStateDerivedValues.ts b/tests/fixtures/environmentStateDerivedValues.ts index 76a2f2f006..ca7f419d58 100644 --- a/tests/fixtures/environmentStateDerivedValues.ts +++ b/tests/fixtures/environmentStateDerivedValues.ts @@ -167,7 +167,7 @@ const slugOf = (title: string): string => .replace(/^-|-$/gu, ""); /** A folder named by the uid its creator chose, with an address from that uid and its title, and a time-based numeric id. */ -export const folder = ( +const folder = ( uid: string, title: string, id: number, diff --git a/tests/fixtures/environmentStateItemWrites.ts b/tests/fixtures/environmentStateItemWrites.ts index 203cb03a30..381093e66e 100644 --- a/tests/fixtures/environmentStateItemWrites.ts +++ b/tests/fixtures/environmentStateItemWrites.ts @@ -55,7 +55,7 @@ const TAGS = [ ]; /** An item as the application answers it: its parent location and tags spelled out from the ids a call sent. */ -export const inventoryItem = ( +const inventoryItem = ( id: string, name: string, parent: Entity | null, diff --git a/tests/fixtures/environmentStateWrites.ts b/tests/fixtures/environmentStateWrites.ts index 0838ae6d74..12f8d092d1 100644 --- a/tests/fixtures/environmentStateWrites.ts +++ b/tests/fixtures/environmentStateWrites.ts @@ -283,7 +283,7 @@ export const UNCHAPTERED_ROWS: readonly TimedCaptureRow[] = renamingRows( ); /** A book whose sent handle the application stores as its code and embeds in its link. */ -export const handledBook = (entity: Book, handle: string): Book => ({ +const handledBook = (entity: Book, handle: string): Book => ({ ...entity, code: handle, link: `https://books.example.test/b/${handle}`, @@ -348,7 +348,7 @@ const shelved = (entry: Book, shelf: string): Book => ({ ...entry, shelf }); const SHELVED_FIRST = shelved(FIRST_BOOK, "a"); const SHELVED_SECOND = shelved(SECOND_BOOK, "b"); /** The book the create at exchange 7 of QUERY_ROWS made, on shelf a. */ -export const SHELVED_THIRD = shelved(book(3, "Third Book", bookClock(7)), "a"); +const SHELVED_THIRD = shelved(book(3, "Third Book", bookClock(7)), "a"); /** * One list tool recorded whole, filtered by shelf, searched, sorted and with a flag that changes nothing; item reads; @@ -373,7 +373,7 @@ export const QUERY_ROWS: readonly TimedCaptureRow[] = [ ]; /** A clock spelled with a `+00:00` offset where the books application's item reads spell `Z`. */ -export const offsetClock = (clock: JsonValue | undefined): string => +const offsetClock = (clock: JsonValue | undefined): string => String(clock).replace(/Z$/u, "+00:00"); /** A book as a list that spells its clocks with an offset shows it. */ @@ -455,6 +455,7 @@ export const environmentStateViewOf = ( ): EnvironmentStateView => { const result = projectAnalysisView( { + predicateType: capture.predicate_type, evidenceId: capture.evidence_id, operation: capture.operation, normalizedResult: capture.normalized_result, diff --git a/tests/fixtures/fakeCdpBrowserEvents.ts b/tests/fixtures/fakeCdpBrowserEvents.ts index 7133d6ad32..5ea79d28e5 100644 --- a/tests/fixtures/fakeCdpBrowserEvents.ts +++ b/tests/fixtures/fakeCdpBrowserEvents.ts @@ -67,7 +67,7 @@ const emitMainWebMcpTools = ( description: "Search orders; authorization=Bearer tool-secret", frameId: "frame-main", backendNodeId: 42, - inputSchema: options.webMcpInputSchema ?? { + inputSchema: { type: "object", properties: inputSchemaProperties, required: ["orderId"], diff --git a/tests/fixtures/fakeCdpBrowserTypes.ts b/tests/fixtures/fakeCdpBrowserTypes.ts index 3ece0256db..1ef4e07d1a 100644 --- a/tests/fixtures/fakeCdpBrowserTypes.ts +++ b/tests/fixtures/fakeCdpBrowserTypes.ts @@ -116,7 +116,6 @@ export interface FakeOptions { readonly webMcpSameUrlRegistrations?: "retain" | "remove-second"; readonly webMcpFrameCount?: number; readonly webMcpSchemaPropertyCount?: number; - readonly webMcpInputSchema?: JsonValue; readonly webMcpChildLeavesScope?: boolean; readonly webMcpChildTransientBlank?: boolean; readonly webMcpChildRecoversAfterTransient?: boolean; diff --git a/tests/fixtures/fakeGhidra.mjs b/tests/fixtures/fakeGhidra.mjs index 99cd7310a8..5ef111d92b 100644 --- a/tests/fixtures/fakeGhidra.mjs +++ b/tests/fixtures/fakeGhidra.mjs @@ -13,6 +13,7 @@ const { targetSha256, transport, mode = "success", + handshakeReports = {}, } = configuration ?? {}; if ( @@ -264,6 +265,24 @@ const handleRequest = (socket, server, request, state) => { return; } } + // The bridge's per-procedure deadline: only this request fails. + if ( + state.mode === "decompile_timed_out" && + request.method === "procedure_pseudo_code" + ) { + socket.write( + `${JSON.stringify({ + id: request.id, + ok: false, + error: { + code: "decompile_timed_out", + message: + "Ghidra decompilation of 0x1000 exceeded the 1-second configured decompilation limit (ghidra.decompile_timeout_seconds; launch seed REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS)", + }, + })}\n`, + ); + return; + } if (request.method === "address_to_file_offset") { socket.write( `${JSON.stringify({ id: request.id, ...fileOffsetReply(request.params.address) })}\n`, @@ -368,14 +387,17 @@ const handlePing = (socket, request, state) => { return; } if (state.mode === "hang_after_start" && state.pings.value > 1) return; - const result = sessionInfo({ - runId: state.runId, - providerVersion: - state.mode === "wrong_identity" ? "0.0.0" : state.providerVersion, - profileDigest: state.profileDigest, - targetSha256: state.targetSha256, - timedOut: state.mode === "analysis_timeout", - }); + const result = { + ...sessionInfo({ + runId: state.runId, + providerVersion: + state.mode === "wrong_identity" ? "0.0.0" : state.providerVersion, + profileDigest: state.profileDigest, + targetSha256: state.targetSha256, + timedOut: state.mode === "analysis_timeout", + }), + ...handshakeReports, + }; const response = `${JSON.stringify({ id: request.id, ok: true, result })}\n`; if (state.mode === "fragmented") { const split = Math.floor(response.length / 2); diff --git a/tests/fixtures/flutterVerifierPreload.mjs b/tests/fixtures/flutterVerifierPreload.mjs new file mode 100644 index 0000000000..863c2e0fb4 --- /dev/null +++ b/tests/fixtures/flutterVerifierPreload.mjs @@ -0,0 +1,34 @@ +import { registerHooks } from "node:module"; +const model = ` +const identity={sha256:'a'.repeat(64),bytes:1}; +export function result(operation,input) { + const path=input.path;const target={path,...identity}; + if(operation==='identify_flutter_build') return {target,flutter_detected:true,coverage:'complete',abis:[{abi:'x86_64',libapp:{present:true,elf_header_valid:true,snapshot_hash_sources:2,snapshot_hash:'hash',sha256:'b'.repeat(64),bytes:100},libflutter:{present:true,build_id:'12345678',build_id_candidates:['12345678']}}]}; + const abi=input.abi ?? (process.env.REA_FLUTTER_PROBE === 'wrong' ? 'ghost' : 'x86_64'); + return {target,abi,coverage:'complete',libapp:{coverage:'complete',sha256:'b'.repeat(64),bytes:100,snapshot_hash_sources:2,snapshot_hash:'hash',sections:['_kDartVmSnapshotData','_kDartVmSnapshotInstructions','_kDartIsolateSnapshotData','_kDartIsolateSnapshotInstructions'].map(name=>({name,offset:0,size:64,magic_valid:true,kind:3,header_length:48}))},string_pool:{coverage:'complete',package_uri_count_at_least:1,dart_uri_count_at_least:1}}; +} +export function envelope(op,input){const resultValue=result(op,input);return {normalized_result:resultValue,subject:{local_path:input.path,digest:{sha256:identity.sha256}},limitations:[]};} +`; +const modelUrl = "data:text/javascript," + encodeURIComponent(model); +const mocks = { + "node:child_process": `import {promisify} from 'node:util';import {envelope} from ${JSON.stringify(modelUrl)};export const execFile=Object.assign(()=>{}, {[promisify.custom]:async (_cmd,args)=>{const op=args[1].replaceAll('-','_');const index=args.indexOf('--abi');const input={path:args[2],...(index<0?{}:{abi:args[index+1]})};return {stdout:JSON.stringify(envelope(op,input)),stderr:''};}});`, + "@modelcontextprotocol/client": `import {envelope} from ${JSON.stringify(modelUrl)};export class Client {async connect(){} async close(){} async callTool({name,arguments:input}){return {structuredContent:envelope(name,input)}}}`, + "@modelcontextprotocol/client/stdio": `export class StdioClientTransport {async close(){}}`, + "../dist/contracts/toolContracts.js": `export const toolContract=()=>({outputSchema:{parse:value=>value}});`, + "../dist/artifacts/ArtifactResourceScope.js": `export class ArtifactResourceScope {async close(){}}`, + "./lib/mcp-verifier-results.mjs": `export const requireMcpOutput=response=>response.structuredContent;`, + "./lib/verifier-file-resources.mjs": `export const hashVerifierFile=async()=>({sha256:'a'.repeat(64),bytes:1});`, +}; +registerHooks({ + resolve(specifier, context, next) { + if ( + context.parentURL?.endsWith("/scripts/verify-flutter.mjs") && + Object.hasOwn(mocks, specifier) + ) + return { + shortCircuit: true, + url: "data:text/javascript," + encodeURIComponent(mocks[specifier]), + }; + return next(specifier, context); + }, +}); diff --git a/tests/fixtures/il2cpp/metadata.ts b/tests/fixtures/il2cpp/metadata.ts index 45bc4a8f7a..cb2e490924 100644 --- a/tests/fixtures/il2cpp/metadata.ts +++ b/tests/fixtures/il2cpp/metadata.ts @@ -7,7 +7,7 @@ type SectionName = IL2cppMetadataSectionName; type Width = 1 | 2 | 4; /** Synthetic version 39 metadata small enough to read in a hex dump. */ -export interface Il2cppMetadataFixtureOptions { +interface Il2cppMetadataFixtureOptions { /** Serialized TypeIndex width. */ readonly typeIndexWidth?: Width; /** Omit interfaceOffsets so the TypeIndex width comes from type records. */ @@ -46,7 +46,7 @@ export interface Il2cppMetadataFixtureOptions { * System.Größe, Game.Player.Slot.Add and System.Größe.ToString; * the fixture names each container's parameters T0, T1 and so on. */ -export const FIXTURE_GENERIC_CONTAINERS = [ +const FIXTURE_GENERIC_CONTAINERS = [ [3, 1, 0, 0], [2, 2, 1, 1], [4, 1, 1, 3], @@ -134,7 +134,7 @@ export const FIXTURE_STRING_LITERALS = [ * Per-image method tokens. Row ids restart in each image and do not follow * metadata order, so linking must use the token rather than the position. */ -export const FIXTURE_METHOD_TOKENS = [ +const FIXTURE_METHOD_TOKENS = [ 0x06000002, 0x06000001, 0x06000003, 0x06000001, 0x06000002, ] as const; diff --git a/tests/fixtures/inspector/fakeV8Inspector.ts b/tests/fixtures/inspector/fakeV8Inspector.ts index 35ef3cb69a..544fc52622 100644 --- a/tests/fixtures/inspector/fakeV8Inspector.ts +++ b/tests/fixtures/inspector/fakeV8Inspector.ts @@ -48,6 +48,9 @@ export const startFakeV8Inspector = async ( const sockets = new Set(); let port = 0; const http = createServer((request, response) => { + // The in-process client may block on schema validation past the idle socket + // deadline. Keep discovery independent of pooled-socket expiry races. + response.setHeader("connection", "close"); response.setHeader("content-type", "application/json"); if (request.url === "/json/version") { response.end( diff --git a/tests/fixtures/mcpAdvertisedSchemaValidity.ts b/tests/fixtures/mcpAdvertisedSchemaValidity.ts index 4549cc0b98..30f76b9631 100644 --- a/tests/fixtures/mcpAdvertisedSchemaValidity.ts +++ b/tests/fixtures/mcpAdvertisedSchemaValidity.ts @@ -6,7 +6,7 @@ import { } from "./mcpClientSchemaConstraints.js"; /** Advertised schema pair as a client receives it from `tools/list`. */ -export interface AdvertisedToolSchemas { +interface AdvertisedToolSchemas { readonly name: string; readonly inputSchema: Record; readonly outputSchema?: Record | undefined; diff --git a/tests/fixtures/processRunTokenSnapshotProbe.swift b/tests/fixtures/processRunTokenSnapshotProbe.swift index 77a758d705..393809c9e8 100644 --- a/tests/fixtures/processRunTokenSnapshotProbe.swift +++ b/tests/fixtures/processRunTokenSnapshotProbe.swift @@ -45,7 +45,7 @@ struct ProcessRunTokenSnapshotProbe { let zombieIdentity = readProcessIdentities( pids: [42], snapshot: ProcessSnapshot( - metadata: [42: ProcessMetadata(identity: "start:42", effectiveUid: 501, lifecycle: "zombie", parentPid: 7, processGroupId: 9, pointerSize: 8)], + metadata: [42: ProcessMetadata(identity: "start:42", effectiveUid: 501, lifecycle: "zombie", parentPid: 7, processGroupId: 9, pointerSize: 8, name: "probe-name")], failure: nil ) ) @@ -58,7 +58,8 @@ struct ProcessRunTokenSnapshotProbe { "invalidPidIsUnavailable": invalidIdentity.first?.state == "unavailable" && invalidIdentity.first?.reason == "invalid_process_id", "zombieLifecycleIsSeparateFromIdentity": zombieIdentity.first?.state == "readable" && zombieIdentity.first?.identity == "start:42" && zombieIdentity.first?.lifecycle == "zombie", "effectiveUidComesFromNativeIdentity": zombieIdentity.first?.effective_uid == 501, - "nativeRelationIsSeparateFromIdentity": zombieIdentity.first?.relation?.parent_pid == 7 && zombieIdentity.first?.relation?.process_group_id == 9 + "nativeRelationIsSeparateFromIdentity": zombieIdentity.first?.relation?.parent_pid == 7 && zombieIdentity.first?.relation?.process_group_id == 9, + "nativeNameIsSeparateFromIdentity": zombieIdentity.first?.name == "probe-name" && absentIdentities.first?.name == nil && failedIdentities.first?.name == nil ] let data = try JSONSerialization.data(withJSONObject: output, options: [.sortedKeys]) FileHandle.standardOutput.write(data) diff --git a/tests/fixtures/sqlite/database.ts b/tests/fixtures/sqlite/database.ts index 445ba6b458..07729df8c7 100644 --- a/tests/fixtures/sqlite/database.ts +++ b/tests/fixtures/sqlite/database.ts @@ -12,7 +12,7 @@ export const SQLITE_NATIVE_LIMITS_AVAILABLE = (() => { })(); /** Exact selected SQLite fixture with committed rows and optional live WAL. */ -export interface SqliteDatabaseFixture { +interface SqliteDatabaseFixture { readonly path: string; readonly walPath: string; readonly quotedTable: string; diff --git a/tests/fixtures/uimage.ts b/tests/fixtures/uimage.ts index 5c4f8ec8d0..64cfd5e462 100644 --- a/tests/fixtures/uimage.ts +++ b/tests/fixtures/uimage.ts @@ -1,10 +1,9 @@ import { crc32 } from "node:zlib"; -import { - UIMAGE_HEADER_SIZE, - UIMAGE_MAGIC, - UIMAGE_NAME_LENGTH, -} from "../../src/domain/uimageFormat.js"; +// U-Boot legacy image header layout, independent of the REA reader. +const UIMAGE_HEADER_SIZE = 64; +const UIMAGE_MAGIC = 0x27051956; +const UIMAGE_NAME_LENGTH = 32; export interface UImageFixtureOptions { readonly arch?: number; diff --git a/tests/fixtures/unityfs.ts b/tests/fixtures/unityfs.ts index 27b78770e1..28c65d511a 100644 --- a/tests/fixtures/unityfs.ts +++ b/tests/fixtures/unityfs.ts @@ -8,7 +8,7 @@ export interface UnityFsMember { readonly size?: number; } -export interface UnityFsOptions { +interface UnityFsOptions { readonly formatVersion?: number; readonly playerVersion?: string; readonly engineVersion?: string; @@ -29,7 +29,7 @@ const cstring = (value: string): Buffer => const align16 = (value: number): number => (value + 15) & ~15; /** One final LZ4 literal sequence; separate sequences require intervening matches. */ -export const lz4LiteralBlock = (data: Buffer): Buffer => { +const lz4LiteralBlock = (data: Buffer): Buffer => { const head = [Math.min(data.length, 15) << 4]; if (data.length >= 15) { let extra = data.length - 15; diff --git a/tests/process-global/ghidraLaunchRollback.test.ts b/tests/process-global/ghidraLaunchRollback.test.ts index c38993e100..e5dd4eb663 100644 --- a/tests/process-global/ghidraLaunchRollback.test.ts +++ b/tests/process-global/ghidraLaunchRollback.test.ts @@ -226,9 +226,10 @@ it.skipIf(process.platform === "win32")( expect(result.error.cause).toBeInstanceOf(AggregateError); if (!(result.error.cause instanceof AggregateError)) throw new Error("Expected composite failure"); - expect(result.error.cause.errors[0]).toMatchObject({ - cause: fault.failure, - }); + const primary: unknown = result.error.cause.errors[0]; + expect(primary).toBeInstanceOf(GhidraLaunchError); + if (!(primary instanceof GhidraLaunchError)) throw primary; + expect(primary.cause).toBe(fault.failure); expect(result.error.cause.errors[1]).toBeUndefined(); expect(result.error.partialLaunch).toBeDefined(); denyCleanup = false; diff --git a/tests/process-global/hopperDownloadHttpFailure.test.ts b/tests/process-global/hopperDownloadHttpFailure.test.ts index 5906d215c2..98cd0c707a 100644 --- a/tests/process-global/hopperDownloadHttpFailure.test.ts +++ b/tests/process-global/hopperDownloadHttpFailure.test.ts @@ -1,69 +1,142 @@ +import { createHash } from "node:crypto"; +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; + import { afterEach, expect, it, vi } from "vitest"; import { createDoctorHostFixture } from "../../src/application/Doctor.fixture.js"; -import { downloadHopperPackage } from "../../src/application/HopperDownload.js"; +import { + describeHopperRequestFailure, + downloadHopperPackage, + hopperRequestRecovery, + type HopperRequestFailure, +} from "../../src/application/HopperDownload.js"; +import { + HOPPER_PUBLIC_KEY_URL, + HOPPER_RELEASES_URL, +} from "../../src/application/HopperReleaseFeed.js"; import { systemLinuxHopperInstallHost, type LinuxHopperInstallHost, } from "../../src/application/LinuxHopper.js"; +import { LinuxHopperPublisher } from "../../src/application/LinuxHopperPublisher.js"; import { systemSetupHost } from "../../src/application/SetupHost.js"; +import type { + SetupFailureCode, + SetupHopperInstallResult, +} from "../../src/application/SetupInstallFailure.js"; import { ok } from "../../src/domain/result.js"; +import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js"; -vi.mock("../../src/application/LinuxHopper.js", async (importOriginal) => ({ - ...(await importOriginal< - typeof import("../../src/application/LinuxHopper.js") - >()), - systemLinuxHopperInstallHost: vi.fn(), -})); +vi.mock("../../src/application/LinuxHopper.js", async (importOriginal) => { + const actual = + await importOriginal< + typeof import("../../src/application/LinuxHopper.js") + >(); + return { + ...actual, + systemLinuxHopperInstallHost: vi.fn(actual.systemLinuxHopperInstallHost), + }; +}); + +const nativeFetch = globalThis.fetch; afterEach(() => { vi.unstubAllGlobals(); vi.restoreAllMocks(); }); -const url = - "https://www.hopperapp.com/downloader/public/Hopper-fixture-Linux-demo.deb"; -const generic = - "REA could not download or unpack the official Hopper package. Check network and filesystem access, then retry setup."; +const debUrl = + "https://www.hopperapp.com:443/downloader/public/Hopper-6.6.1-Linux-demo.deb"; +const dmgUrl = + "https://www.hopperapp.com:443/downloader/public/Hopper-6.6.1-demo.dmg"; +const packageBytes = Buffer.from("fixture package"); +const sha1 = (bytes: Uint8Array) => + createHash("sha1").update(bytes).digest("hex"); +const sha256 = (bytes: Uint8Array) => + createHash("sha256").update(bytes).digest("hex"); +const credentialed = + "https://fixture-user:fixture-secret@downloads.hopperapp.com/redirected"; -const installHost = (): LinuxHopperInstallHost => { - const unexpected = vi.fn((): never => { - throw new Error("Unexpected installation effect"); +const linuxRelease = (mainExeHash = "0".repeat(64)) => ({ + version: "6.6.1", + filename: debUrl, + file_length: String(packageBytes.byteLength), + file_hash: sha1(packageBytes), + main_exe_hash: mainExeHash, + signature: "AA==", + public: "1" as const, + beta: "0" as const, +}); +const feed = (mainExeHash?: string) => + JSON.stringify({ + "OS X": { + filename: dmgUrl, + file_length: String(packageBytes.byteLength), + file_hash: sha1(packageBytes), + }, + "Ubuntu / Mint": linuxRelease(mainExeHash), }); - return { - distribution: async () => ({ + +/** Route controlled vendor responses by the exact caller-selected URL. */ +const vendor = ( + responses: Readonly Response | Promise>>, +) => { + const fetch = vi.fn(async (input, init) => { + const respond = responses[String(input)]; + // Native Fetch refuses credentialed URLs offline, echoing them in its error. + if (respond === undefined && String(input) === credentialed) + return nativeFetch(input, init); + if (respond === undefined) + throw new Error(`Unexpected request ${String(input)}`); + return respond(); + }); + vi.stubGlobal("fetch", fetch); + return { fetch }; +}; +const failed = (status: number, cancel?: () => void) => () => + new Response(new ReadableStream({ ...(cancel ? { cancel } : {}) }), { + status, + }); + +/** Delivered guidance comes from the owning formatters; facts are checked independently. */ +const expectRequestFailure = ( + result: SetupHopperInstallResult, + code: SetupFailureCode, + download: HopperRequestFailure, +) => { + expect(result).toEqual({ + status: "failed", + code, + remediation: `${describeHopperRequestFailure(download)} ${hopperRequestRecovery(download)}`, + download, + }); + if (result.status !== "failed") throw new Error("Expected a setup failure"); + expect(result.remediation).toContain(download.url); + if (download.kind === "http_status" && download.httpStatus !== undefined) + expect(result.remediation).toContain(`HTTP ${download.httpStatus}`); +}; + +const linuxDoctor = () => + createDoctorHostFixture({ + platform: "linux", + linuxDistribution: async () => ({ id: "ubuntu", versionId: "24.04", packageFamily: "deb", hopperSetupSupported: true, }), - release: async () => - ok({ - version: "fixture", - filename: url, - file_length: "1", - file_hash: "0".repeat(40), - main_exe_hash: "0".repeat(64), - signature: "AA==", - public: "1", - beta: "0", - }), - download: downloadHopperPackage, - createTemporaryDirectory: unexpected, - writeArchive: unexpected, - installPackage: unexpected, - verifyArchive: unexpected, - retainVerifiedArchive: unexpected, - launcherStatus: unexpected, - cleanup: unexpected, - }; -}; + }); -const setupFailure = async () => { - vi.mocked(systemLinuxHopperInstallHost).mockReturnValue(installHost()); +const installThroughSetup = async ( + platform: "darwin" | "linux", + environment: Readonly = {}, +) => { const host = systemSetupHost( - createDoctorHostFixture({ platform: "linux" }), - {}, + platform === "linux" + ? linuxDoctor() + : createDoctorHostFixture({ platform: "darwin" }), + environment, ); try { return await host.installHopper(false); @@ -72,33 +145,194 @@ const setupFailure = async () => { } }; -it.each([404, 503])( - "preserves HTTP %i from download through Linux setup diagnostics", +it.each([404, 503] as const)( + "names the macOS release feed URL and HTTP %i in the setup result", + async (status) => { + const { fetch } = vendor({ [HOPPER_RELEASES_URL]: failed(status) }); + expectRequestFailure( + await installThroughSetup("darwin"), + "download_failed", + { + request: "release_feed", + url: HOPPER_RELEASES_URL, + kind: "http_status", + httpStatus: status, + }, + ); + expect(fetch).toHaveBeenCalledOnce(); + }, +); + +it.each([404, 503] as const)( + "names the selected macOS DMG URL and HTTP %i in the setup result", async (status) => { const cancel = vi.fn(); - const fetch = vi - .fn() - .mockImplementation( - async () => new Response(new ReadableStream({ cancel }), { status }), + vendor({ + [HOPPER_RELEASES_URL]: () => new Response(feed()), + [dmgUrl]: failed(status, cancel), + }); + expectRequestFailure( + await installThroughSetup("darwin"), + "download_failed", + { + request: "package", + url: dmgUrl, + kind: "http_status", + httpStatus: status, + }, + ); + expect(cancel).toHaveBeenCalledOnce(); + }, +); + +it("keeps a network failure distinct from an HTTP status in setup", async () => { + vendor({ + [HOPPER_RELEASES_URL]: () => new Response(feed()), + [dmgUrl]: () => { + throw new TypeError("fetch failed", { + cause: new Error("getaddrinfo ENOTFOUND www.hopperapp.com"), + }); + }, + }); + expectRequestFailure(await installThroughSetup("darwin"), "download_failed", { + request: "package", + url: dmgUrl, + kind: "network", + detail: "fetch failed: getaddrinfo ENOTFOUND www.hopperapp.com", + }); +}); + +it.each([404, 503] as const)( + "names the Linux publisher release feed URL and HTTP %i in the setup result", + async (status) => { + const { fetch } = vendor({ [HOPPER_RELEASES_URL]: failed(status) }); + expectRequestFailure( + await installThroughSetup("linux"), + "download_failed", + { + request: "release_feed", + url: HOPPER_RELEASES_URL, + kind: "http_status", + httpStatus: status, + }, + ); + expect(fetch).toHaveBeenCalledOnce(); + }, +); + +it.each([404, 503] as const)( + "names the Linux publisher public-key URL and HTTP %i before any package manager runs", + async (status) => { + const root = await createTestTempDirectory("rea-hopper-key-failure-"); + const { fetch } = vendor({ + [HOPPER_RELEASES_URL]: () => new Response(feed()), + [debUrl]: () => new Response(packageBytes), + [HOPPER_PUBLIC_KEY_URL]: failed(status), + }); + expectRequestFailure( + await installThroughSetup("linux", { + HOME: root, + XDG_CACHE_HOME: join(root, "cache"), + }), + "publisher_verification_failed", + { + request: "public_key", + url: HOPPER_PUBLIC_KEY_URL, + kind: "http_status", + httpStatus: status, + }, + ); + expect(fetch.mock.calls.map(([input]) => input)).toEqual([ + HOPPER_RELEASES_URL, + debUrl, + HOPPER_PUBLIC_KEY_URL, + ]); + }, +); + +it.each([404, 503] as const)( + "names the Linux package URL and HTTP %i in the setup result", + async (status) => { + const unexpected = vi.fn((): never => { + throw new Error("Unexpected installation effect"); + }); + const host: LinuxHopperInstallHost = { + distribution: async () => ({ + id: "ubuntu", + versionId: "24.04", + packageFamily: "deb", + hopperSetupSupported: true, + }), + release: async () => ok(linuxRelease()), + download: downloadHopperPackage, + createTemporaryDirectory: unexpected, + writeArchive: unexpected, + installPackage: unexpected, + verifyArchive: unexpected, + retainVerifiedArchive: unexpected, + launcherStatus: unexpected, + cleanup: unexpected, + }; + vi.mocked(systemLinuxHopperInstallHost).mockReturnValueOnce(host); + const cancel = vi.fn(); + vendor({ [debUrl]: failed(status, cancel) }); + expectRequestFailure( + await installThroughSetup("linux"), + "download_failed", + { + request: "package", + url: debUrl, + kind: "http_status", + httpStatus: status, + }, + ); + expect(cancel).toHaveBeenCalledOnce(); + }, +); + +it.each([503, 404] as const)( + "names the package URL and HTTP %i when launch verification downloads the current release", + async (status) => { + const root = await createTestTempDirectory("rea-hopper-launch-download-"); + const launcher = join(root, "Hopper"); + await writeFile(launcher, "fixture launcher"); + const launcherHash = sha256(Buffer.from("fixture launcher")); + vendor({ + [HOPPER_RELEASES_URL]: () => new Response(feed(launcherHash)), + [debUrl]: failed(status), + }); + const publisher = new LinuxHopperPublisher( + { XDG_CACHE_HOME: join(root, "cache") }, + async () => "deb", + ); + try { + const inspected = await publisher.inspectLauncher(launcher); + expect(inspected).toMatchObject({ + ok: false, + error: { + reason: "download", + download: { + request: "package", + url: debUrl, + kind: "http_status", + httpStatus: status, + }, + }, + }); + if (inspected.ok || inspected.error.download === undefined) + throw new Error("Expected the failed package request"); + expect(inspected.error.message).toBe( + describeHopperRequestFailure(inspected.error.download), ); - vi.stubGlobal("fetch", fetch); - expect(await downloadHopperPackage(url)).toEqual({ - ok: false, - bytes: new Uint8Array(), - httpStatus: status, - }); - expect(await setupFailure()).toEqual({ - status: "failed", - code: "download_failed", - remediation: `download: Package download returned HTTP ${status}. ${generic}`, - }); - expect(fetch).toHaveBeenCalledTimes(2); - expect(cancel).toHaveBeenCalledTimes(2); + expect(inspected.error.message).toContain(`HTTP ${status}`); + } finally { + await publisher.close(); + } }, ); it.each([undefined, null, 0, 200, 600, 503.5, "503", Number.NaN])( - "keeps generic diagnostics when the failed response status is %s", + "omits an invalid failed-response status %s instead of reporting it", async (status) => { vi.stubGlobal( "fetch", @@ -108,15 +342,19 @@ it.each([undefined, null, 0, 200, 600, 503.5, "503", Number.NaN])( return response; }), ); - expect(await downloadHopperPackage(url)).toEqual({ + expect(await downloadHopperPackage(debUrl)).toEqual({ ok: false, - bytes: new Uint8Array(), - }); - expect(await setupFailure()).toEqual({ - status: "failed", - code: "download_failed", - remediation: generic, + error: { url: debUrl, kind: "http_status" }, }); + expectRequestFailure( + await installThroughSetup("darwin"), + "download_failed", + { + request: "release_feed", + url: HOPPER_RELEASES_URL, + kind: "http_status", + }, + ); }, ); @@ -125,8 +363,126 @@ it("keeps a successful download unchanged", async () => { "fetch", vi.fn().mockResolvedValue(new Response("package")), ); - expect(await downloadHopperPackage(url)).toEqual({ - ok: true, - bytes: Buffer.from("package"), + expect(await downloadHopperPackage(debUrl)).toEqual( + ok(Buffer.from("package")), + ); +}); + +it("chooses recovery by failure class: missing file, temporary service failure, refusal or unreachable host", () => { + const status = (httpStatus?: number) => + hopperRequestRecovery( + httpStatus === undefined + ? { url: debUrl, kind: "http_status" } + : { url: debUrl, kind: "http_status", httpStatus }, + ); + const unreachable = hopperRequestRecovery({ + url: debUrl, + kind: "network", + detail: "fetch failed", + }); + expect(status(410)).toBe(status(404)); + for (const transient of [408, 429, 500, 502, 504]) + expect(status(transient)).toBe(status(503)); + expect(status(undefined)).toBe(status(403)); + expect( + hopperRequestRecovery({ url: debUrl, kind: "timeout", timeoutMs: 10_000 }), + ).toBe(unreachable); + expect( + new Set([status(404), status(503), status(403), unreachable]).size, + ).toBe(4); +}); + +it.each([ + { platform: "darwin", request: "package", url: dmgUrl }, + { platform: "linux", request: "release_feed", url: HOPPER_RELEASES_URL }, +] as const)( + "refuses a credentialed redirect of the $request request without echoing its credentials", + async ({ platform, request, url }) => { + const { fetch } = vendor({ + [HOPPER_RELEASES_URL]: () => + url === HOPPER_RELEASES_URL + ? new Response(null, { + status: 302, + headers: { location: credentialed }, + }) + : new Response(feed()), + [dmgUrl]: () => + new Response(null, { + status: 302, + headers: { location: credentialed }, + }), + }); + const result = await installThroughSetup(platform); + expectRequestFailure(result, "download_failed", { + request, + url, + kind: "url_refused", + refusal: "credentials", + }); + expect(JSON.stringify(result)).not.toMatch(/fixture-(user|secret)/u); + expect(fetch.mock.calls.map(([input]) => input)).not.toContain( + credentialed, + ); + }, +); + +it("refuses a credentialed request URL before any request and reports it without userinfo", async () => { + const fetch = vi.fn(); + vi.stubGlobal("fetch", fetch); + expect( + await downloadHopperPackage( + "https://fixture-user:fixture-secret@www.hopperapp.com/package", + ), + ).toEqual({ + ok: false, + error: { + url: "https://www.hopperapp.com/package", + kind: "url_refused", + refusal: "credentials", + }, + }); + expect(fetch).not.toHaveBeenCalled(); +}); + +it("keeps an observed HTTP status when its truncated body cannot be cancelled", async () => { + vendor({ + [HOPPER_RELEASES_URL]: () => + new Response( + new ReadableStream({ + start(controller) { + controller.error(new TypeError("terminated")); + }, + }), + { status: 503 }, + ), + }); + expectRequestFailure(await installThroughSetup("darwin"), "download_failed", { + request: "release_feed", + url: HOPPER_RELEASES_URL, + kind: "http_status", + httpStatus: 503, + }); +}); + +it("keeps the size-limit outcome when releasing the oversized body fails", async () => { + vi.stubGlobal( + "fetch", + vi.fn( + async () => + new Response( + new ReadableStream({ + start(controller) { + controller.enqueue(new Uint8Array(4)); + }, + cancel() { + throw new TypeError("terminated"); + }, + }), + ), + ), + ); + expect(await downloadHopperPackage(debUrl, { maxBytes: 3 })).toEqual({ + ok: false, + error: { url: debUrl, kind: "response_too_large", maxBytes: 3 }, }); }); diff --git a/tests/process-global/hopperDownloadRedirect.test.ts b/tests/process-global/hopperDownloadRedirect.test.ts index 0339a5430b..263863a457 100644 --- a/tests/process-global/hopperDownloadRedirect.test.ts +++ b/tests/process-global/hopperDownloadRedirect.test.ts @@ -19,7 +19,14 @@ it("refuses an HTTPS downgrade before sending the redirected request and release vi.stubGlobal("fetch", fetch); expect( await downloadHopperPackage("https://www.hopperapp.com/package"), - ).toEqual({ ok: false, bytes: new Uint8Array() }); + ).toEqual({ + ok: false, + error: { + url: "https://www.hopperapp.com/package", + kind: "url_refused", + refusal: "non_https", + }, + }); expect(fetch).toHaveBeenCalledTimes(1); expect(fetch).toHaveBeenCalledWith( "https://www.hopperapp.com/package", @@ -46,8 +53,8 @@ it("preserves HTTPS relative and cross-origin redirects under one request deadli vi.stubGlobal("fetch", fetch); const selected = "https://www.hopperapp.com:443/start#selected"; const result = await downloadHopperPackage(selected); - expect(result.ok).toBe(true); - expect(new TextDecoder().decode(result.bytes)).toBe("package"); + if (!result.ok) throw new Error("Expected the redirected package"); + expect(new TextDecoder().decode(result.value)).toBe("package"); expect(fetch.mock.calls.map(([input]) => input)).toEqual([ selected, "https://www.hopperapp.com/next#selected", @@ -69,9 +76,16 @@ it("retains Fetch's twenty-redirect bound and releases every redirect response", }), ); vi.stubGlobal("fetch", fetch); - expect( - (await downloadHopperPackage("https://www.hopperapp.com/loop")).ok, - ).toBe(false); + expect(await downloadHopperPackage("https://www.hopperapp.com/loop")).toEqual( + { + ok: false, + error: { + url: "https://www.hopperapp.com/loop", + kind: "url_refused", + refusal: "redirect_limit", + }, + }, + ); expect(fetch).toHaveBeenCalledTimes(21); expect(cancelled).toBe(21); }); @@ -80,7 +94,14 @@ it("refuses an initial non-HTTPS URL before any request", async () => { const fetch = vi.fn(); vi.stubGlobal("fetch", fetch); expect( - (await downloadHopperPackage("http://www.hopperapp.com/package")).ok, - ).toBe(false); + await downloadHopperPackage("http://www.hopperapp.com/package"), + ).toEqual({ + ok: false, + error: { + url: "http://www.hopperapp.com/package", + kind: "url_refused", + refusal: "non_https", + }, + }); expect(fetch).not.toHaveBeenCalled(); }); diff --git a/tests/process-global/macHopperReleaseMinimum.test.ts b/tests/process-global/macHopperReleaseMinimum.test.ts new file mode 100644 index 0000000000..8cb5d7538a --- /dev/null +++ b/tests/process-global/macHopperReleaseMinimum.test.ts @@ -0,0 +1,105 @@ +import { afterEach, expect, it, vi } from "vitest"; + +import { createDoctorHostFixture } from "../../src/application/Doctor.fixture.js"; +import { HOPPER_RELEASES_URL } from "../../src/application/HopperReleaseFeed.js"; +import { macHopperHostVersionDetail } from "../../src/application/MacHopper.js"; +import { setupInstallFailure } from "../../src/application/SetupInstallFailure.js"; +import { runSetup } from "../../src/application/Setup.js"; +import { systemSetupHost } from "../../src/application/SetupHost.js"; + +afterEach(() => vi.unstubAllGlobals()); + +const PACKAGE_URL = + "https://www.hopperapp.com:443/downloader/public/Hopper-6.6.1-demo.dmg"; +// The public macOS release row observed on 2026-11-01, without its changelog +// and Sparkle signatures. +const FEED = JSON.stringify({ + "OS X": { + version: "6.6.1", + filename: PACKAGE_URL, + file_length: "34838169", + file_hash: "b2a74b6c47abc15d0f0ba2cbc36ed98eed8bd5d1", + public: "1", + beta: "0", + deployment_target: "12.4", + }, +}); + +/** Approved Hopper-only setup on a Mac without Hopper; vendor responses are local. */ +const setupHopperOnMacos = async (version: string) => { + const fetch = vi.fn((input) => + Promise.resolve( + String(input) === HOPPER_RELEASES_URL + ? new Response(FEED) + : new Response(null, { status: 404 }), + ), + ); + vi.stubGlobal("fetch", fetch); + const host = { + ...systemSetupHost( + createDoctorHostFixture({ + platform: "darwin", + macosVersion: () => Promise.resolve(version), + executable: () => Promise.resolve(false), + validTarget: () => Promise.resolve(false), + }), + { HOME: "/fixture/home", USERPROFILE: "/fixture/home" }, + ), + // Keep the runtime gate independent of the Node.js running the suite. + nodeVersion: "24.22.1", + }; + try { + const result = await runSetup( + { + approved: true, + installHopper: true, + structured: true, + clientIds: [], + installSkill: false, + }, + host, + ); + return { + result, + requests: fetch.mock.calls.map(([input]) => String(input)), + }; + } finally { + await host.close?.(); + } +}; + +it.each(["12.0", "12.3"])( + "refuses Hopper on macOS %s from the release deployment target before downloading the package", + async (version) => { + const { result, requests } = await setupHopperOnMacos(version); + + expect(result).toMatchObject({ + status: "needs_human", + code: "incompatible_host_version", + appliedActions: [], + }); + // Recovery and installer detail are delivered from their owners; the + // observed facts are checked independently of that wording. + expect(result.remediation).toContain( + setupInstallFailure("incompatible_host_version").remediation, + ); + expect(result.remediation).toContain( + macHopperHostVersionDetail({ + source: "release_feed", + packageUrl: PACKAGE_URL, + minimumVersion: "12.4", + hostVersion: version, + }), + ); + for (const fact of [PACKAGE_URL, "12.4", version, "HOPPER_LAUNCHER_PATH"]) + expect(result.remediation).toContain(fact); + expect(requests).toEqual([HOPPER_RELEASES_URL]); + }, +); + +it("requests the package once macOS meets the release deployment target", async () => { + const { result, requests } = await setupHopperOnMacos("12.4"); + + expect(result).toMatchObject({ code: "download_failed" }); + expect(requests).toEqual([HOPPER_RELEASES_URL, PACKAGE_URL]); +}); diff --git a/tests/process-global/oneShotSourceClosure.test.ts b/tests/process-global/oneShotSourceClosure.test.ts new file mode 100644 index 0000000000..2ce020b86c --- /dev/null +++ b/tests/process-global/oneShotSourceClosure.test.ts @@ -0,0 +1,379 @@ +import { readdir, readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { Cli } from "incur"; +import { afterEach, expect, it, vi } from "vitest"; + +import type { UndeliveredSourceClosure } from "../../src/application/AnalysisEvidenceSources.js"; +import * as matchingService from "../../src/application/native/NativeFunctionMatchingService.js"; +import { ArtifactResourceScope } from "../../src/artifacts/ArtifactResourceScope.js"; +import { registerNativeFunctionMatchingCommands } from "../../src/cli/nativeFunctionMatchingCommands.js"; +import { analysisErrorProjectionSchema } from "../../src/contracts/errorSchemas.js"; +import { createEvidence, parseEvidence } from "../../src/domain/evidence.js"; +import { + createEvidenceBundle, + parseEvidenceBundle, +} from "../../src/domain/evidenceBundle.js"; +import { jsonByteLength } from "../../src/domain/jsonSerialization.js"; +import { ProviderCleanupError } from "../../src/domain/providerCleanupError.js"; +import { silentLogger } from "../../src/logger.js"; +import { + nativeMatchingInventory, + nativeMatchingProvider, +} from "../fixtures/nativeFunctionMatching.js"; +import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js"; + +const host = vi.hoisted(() => ({ + listeners: new Map void>(), + get exitCode() { + return process.exitCode; + }, + set exitCode(value: typeof process.exitCode) { + process.exitCode = value; + }, + on(signal: string, listener: () => void) { + this.listeners.set(signal, listener); + }, + off(signal: string, listener: () => void) { + if (this.listeners.get(signal) === listener) this.listeners.delete(signal); + }, +})); + +vi.mock("../../src/cli/commandCancellation.js", async (importOriginal) => { + const original = + await importOriginal< + typeof import("../../src/cli/commandCancellation.js") + >(); + return { + ...original, + withCommandCancellation: ( + operation: (signal: AbortSignal) => Promise, + ) => original.withCommandCancellation(operation, host), + }; +}); + +afterEach(() => { + vi.restoreAllMocks(); + host.listeners.clear(); +}); + +type FinalClose = + | "control" + | "interrupt" + | "cleanup-failure" + | "interrupt-cleanup-failure"; + +const fixture = async (dependencyText = "Synthetic dependency") => { + const root = await createTestTempDirectory("rea-source-closure-"); + const dependency = createEvidence(undefined, nativeMatchingProvider, { + operation: "fixture_dependency", + parameters: {}, + result: { source: dependencyText }, + }); + const left = nativeMatchingInventory( + "1", + [{ address: "0x10", value: "main" }], + [dependency.evidence_id], + ); + const right = nativeMatchingInventory("2", [ + { address: "0x20", value: "main" }, + ]); + const sources = join(root, "sources.json"); + await writeFile(sources, JSON.stringify(createEvidenceBundle([dependency]))); + const request = join(root, "request.json"); + await writeFile(request, JSON.stringify({ left, right, detail: "complete" })); + return { + root, + dependency, + left, + right, + request, + sources, + output: join(root, "evidence.json"), + }; +}; + +/** + * Run the registered command, faulting only the close of the scope the command + * gave to matching, after that matching completed. Another scope's close cannot + * move the fault, and a command that stops reaching this close fails here. + */ +const runMatching = async ( + input: Awaited>, + finalClose: FinalClose, + options: { + readonly environment?: Readonly>; + readonly earlierScopeClose?: boolean; + } = {}, +) => { + const match = matchingService.matchNativeFunctionsFromBundles; + const close = ArtifactResourceScope.prototype.close; + let commandScope: ArtifactResourceScope | undefined; + let completed = false; + let finalCloses = 0; + const matching = vi + .spyOn(matchingService, "matchNativeFunctionsFromBundles") + .mockImplementation(async (request, paths, resources, matchOptions) => { + commandScope = resources; + if (options.earlierScopeClose === true) + await new ArtifactResourceScope().close(); + const result = await match(request, paths, resources, matchOptions); + completed = result.ok; + return result; + }); + const seam = vi + .spyOn(ArtifactResourceScope.prototype, "close") + .mockImplementation(async function (this: ArtifactResourceScope) { + if (this !== commandScope || !completed) return close.call(this); + finalCloses += 1; + if ( + finalClose === "interrupt" || + finalClose === "interrupt-cleanup-failure" + ) + host.listeners.get("SIGINT")?.(); + await close.call(this); + if ( + finalClose === "cleanup-failure" || + finalClose === "interrupt-cleanup-failure" + ) + throw new ProviderCleanupError( + "fixture", + ["owned-root"], + { reason: "controlled final close failure" }, + { operation: "match_native_functions" }, + ); + }); + const cli = Cli.create("one-shot-source-closure-test", { sync: false }); + registerNativeFunctionMatchingCommands( + cli, + silentLogger, + options.environment ?? {}, + ); + let stdout = ""; + const previous = process.exitCode; + try { + process.exitCode = undefined; + await cli.serve( + [ + "match-native-functions", + input.request, + "--source_bundle", + input.sources, + "--evidence_output", + input.output, + "--json", + ], + { + stdout: (part) => { + stdout += part; + }, + exit: () => {}, + }, + ); + expect(finalCloses, stdout).toBe(1); + return { stdout, exitCode: process.exitCode }; + } finally { + process.exitCode = previous; + seam.mockRestore(); + matching.mockRestore(); + } +}; + +/** Above V8's default call-argument limit of about 125,000 on a forked main thread. */ +const DATA_SIZED_LINKS = 200_000; + +const closureOf = (stdout: string) => { + const failure = analysisErrorProjectionSchema.parse(JSON.parse(stdout)); + const closure = failure.details?.source_closure as unknown as + | UndeliveredSourceClosure + | undefined; + if (closure === undefined) throw new Error(`No source closure: ${stdout}`); + return { failure, closure }; +}; + +const recordIds = (closure: UndeliveredSourceClosure) => + closure.records.map((record) => parseEvidence(record).evidence_id).sort(); + +it.each([ + ["interrupt", "cancelled", 130], + ["cleanup-failure", "cleanup_incomplete", 1], + ["interrupt-cleanup-failure", "cleanup_incomplete", 130], +] as const)( + "keeps captured linked sources through final close: %s", + async (finalClose, code, exitCode) => { + const input = await fixture(); + const { stdout, exitCode: observedExit } = await runMatching( + input, + finalClose, + ); + expect(observedExit).toBe(exitCode); + const { failure, closure } = closureOf(stdout); + expect(failure.code).toBe(code); + const observation = parseEvidence(failure.details?.partial_observation); + expect(observation.evidence_links).toEqual( + [input.left.evidence_id, input.right.evidence_id].sort(), + ); + // Sources are authentic originals, including the transitively linked one. + expect(recordIds(closure)).toEqual( + [ + input.dependency.evidence_id, + input.left.evidence_id, + input.right.evidence_id, + ].sort(), + ); + expect(closure).toMatchObject({ + publication: "not_published", + complete: true, + omitted_count: 0, + omitted: [], + }); + // Late cancellation never turns into successful or silent publication. + expect(await readdir(input.root)).not.toContain("evidence.json"); + expect(host.listeners.size).toBe(0); + }, +); + +it("keeps the fault on the final close when another scope closes first", async () => { + const input = await fixture(); + const { stdout, exitCode } = await runMatching(input, "interrupt", { + earlierScopeClose: true, + }); + expect(exitCode).toBe(130); + const { failure, closure } = closureOf(stdout); + expect(failure.code).toBe("cancelled"); + expect(parseEvidence(failure.details?.partial_observation).operation).toBe( + "match_native_functions", + ); + expect(closure.complete).toBe(true); +}); + +it.each([ + ["interrupt", "cancelled"], + ["cleanup-failure", "cleanup_incomplete"], +] as const)( + "keeps the typed failure for a captured source with a data-sized link set: %s", + async (finalClose, code) => { + // Repeated links are valid Evidence, and only one distinct record backs them. + const ancestor = createEvidence(undefined, nativeMatchingProvider, { + operation: "fixture_ancestor", + parameters: {}, + result: { source: "Synthetic ancestor" }, + }); + const wide = createEvidence(undefined, nativeMatchingProvider, { + operation: "fixture_dependency", + parameters: {}, + result: { source: "Synthetic wide dependency" }, + evidenceLinks: Array.from( + { length: DATA_SIZED_LINKS }, + () => ancestor.evidence_id, + ), + }); + const input = await fixture(); + const left = nativeMatchingInventory( + "1", + [{ address: "0x10", value: "main" }], + [wide.evidence_id], + ); + await writeFile( + input.sources, + JSON.stringify(createEvidenceBundle([ancestor, wide])), + ); + await writeFile( + input.request, + JSON.stringify({ left, right: input.right, detail: "complete" }), + ); + const { stdout } = await runMatching(input, finalClose, { + environment: { + REA_NATIVE_MATCHING_REPRESENTATION_BYTES: String(1024 ** 3), + }, + }); + const { failure, closure } = closureOf(stdout); + expect(failure.code).toBe(code); + expect(parseEvidence(failure.details?.partial_observation).operation).toBe( + "match_native_functions", + ); + // The closure is deduplicated: each record is examined and reported once. + expect(recordIds(closure)).toEqual( + [ancestor.evidence_id, left.evidence_id, input.right.evidence_id].sort(), + ); + expect(closure).toMatchObject({ + complete: false, + omitted_count: 1, + omitted: [{ evidence_id: wide.evidence_id, reason: "not_inlined" }], + }); + }, + 60_000, +); + +it("hands off the same closure for summary delivery and permits a clean retry", async () => { + const input = await fixture(); + await writeFile( + input.request, + JSON.stringify({ + left: input.left, + right: input.right, + detail: "summary", + }), + ); + const cancelled = await runMatching(input, "interrupt"); + const { closure } = closureOf(cancelled.stdout); + expect(closure.complete).toBe(true); + + const retried = await runMatching(input, "control"); + expect(retried.exitCode).toBeUndefined(); + expect(JSON.parse(retried.stdout)).not.toHaveProperty( + "details.source_closure", + ); + const saved = parseEvidenceBundle( + JSON.parse(await readFile(input.output, "utf8")), + ); + const savedIds = saved.records.map((record) => record.evidence_id); + expect(savedIds).toEqual(expect.arrayContaining(recordIds(closure))); +}); + +it("keeps an oversized source explicit instead of expanding the failure", async () => { + const input = await fixture("x".repeat(128 * 1024)); + const { stdout } = await runMatching(input, "interrupt"); + const { failure, closure } = closureOf(stdout); + expect(failure.code).toBe("cancelled"); + expect(recordIds(closure)).toEqual( + [input.left.evidence_id, input.right.evidence_id].sort(), + ); + expect(closure).toMatchObject({ + complete: false, + omitted_count: 1, + omitted: [ + { evidence_id: input.dependency.evidence_id, reason: "not_inlined" }, + ], + }); + expect(jsonByteLength(closure)).toBeLessThan(64 * 1024); + + // The omitted record stays reachable through ordinary delivery. + const retried = await runMatching(input, "control"); + expect(retried.exitCode).toBeUndefined(); + const saved = parseEvidenceBundle( + JSON.parse(await readFile(input.output, "utf8")), + ); + expect(saved.records.map((record) => record.evidence_id)).toContain( + input.dependency.evidence_id, + ); +}); + +it("leaves ordinary successful delivery free of closure reporting", async () => { + const input = await fixture(); + const { stdout, exitCode } = await runMatching(input, "control"); + expect(exitCode).toBeUndefined(); + expect(stdout).not.toContain("source_closure"); + const delivered = parseEvidence(JSON.parse(stdout)); + const saved = parseEvidenceBundle( + JSON.parse(await readFile(input.output, "utf8")), + ); + expect(saved.records.map((record) => record.evidence_id).sort()).toEqual( + [ + delivered.evidence_id, + input.dependency.evidence_id, + input.left.evidence_id, + input.right.evidence_id, + ].sort(), + ); +}); diff --git a/tests/process-global/shortFileReads.test.ts b/tests/process-global/shortFileReads.test.ts index e392cd8e2d..957c1bd46c 100644 --- a/tests/process-global/shortFileReads.test.ts +++ b/tests/process-global/shortFileReads.test.ts @@ -388,10 +388,11 @@ it("preserves cancellation during continued short reads and releases the descrip const path = join(root, "Electron"); await writeFile(path, thinFuseImage(OBSIDIAN_FUSE_BYTES).bytes); const controller = new AbortController(); + const reason = new Error("cancel continued read"); fault.path = path; fault.maximum = 2; fault.onRead = () => { - if (fault.reads === 2) controller.abort(new Error("cancel continued read")); + if (fault.reads === 2) controller.abort(reason); }; const provider = new ElectronFuseProvider(); try { @@ -402,6 +403,7 @@ it("preserves cancellation during continued short reads and releases the descrip expect(result.ok).toBe(false); if (result.ok) throw new Error("Expected cancellation"); expect(result.error).toBeInstanceOf(AnalysisCancelledError); + expect(result.error.cause).toBe(reason); expect(fault.reads).toBe(2); expect(fault.handles.every((handle) => handle.fd === -1)).toBe(true); } finally { diff --git a/tests/support/hostCleanup.ts b/tests/support/hostCleanup.ts index 443ba85817..811c065509 100644 --- a/tests/support/hostCleanup.ts +++ b/tests/support/hostCleanup.ts @@ -50,8 +50,9 @@ const expectHostCleanupReport = (report: { process.platform === "linux" ? "environment_errno_(?:EACCES|EPERM)" : "environment_unavailable"; + // Native identity facts may follow; a JSON-quoted name can hold parentheses. const candidate = - "[1-9][0-9]*=(?:environment_unavailable|environment_errno_(?:EACCES|EPERM))"; + '[1-9][0-9]*=(?:environment_unavailable|environment_errno_(?:EACCES|EPERM))(?: \\((?:[^()"]|"(?:[^"\\\\]|\\\\.)*")*\\))?'; expect(reason).toMatch( new RegExp( `^process ownership token could not be read for [1-9][0-9]* live process\\(es\\): ` +