diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md index fc0fed32f5..f2b143a366 100644 --- a/.agents/skills/reverse-engineer-anything/SKILL.md +++ b/.agents/skills/reverse-engineer-anything/SKILL.md @@ -117,10 +117,11 @@ explicit path or endpoint and do not need it. `inspect_android_package`, then focused Android tools when advertised. Archive member inventory still uses the archive route above. - Managed PE/CLI assembly: `inspect_managed_artifact`. -- Firmware image: `inspect_firmware_regions` when advertised. Use - `extract_firmware` when extraction is requested, with a caller-selected new - absolute output directory. These tools use caller-supplied Binwalk/Unblob on - Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md). +- UEFI firmware volumes and NVAR stores: use bundled `inspect_artifact`. + For external firmware scanning or extraction on Linux, use + `inspect_firmware_regions` or `extract_firmware` with caller-supplied + Binwalk or Unblob. Extraction needs a caller-selected new absolute output + directory. See the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md). - ESP32-family app or bootloader image (`0xE9` magic): `inspect_artifact` without a provider; see the ESP section of `references/native-and-artifacts.md`. - .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode; diff --git a/.cursor-plugin/skills/reverse-engineer-anything/SKILL.md b/.cursor-plugin/skills/reverse-engineer-anything/SKILL.md index fc0fed32f5..f2b143a366 100644 --- a/.cursor-plugin/skills/reverse-engineer-anything/SKILL.md +++ b/.cursor-plugin/skills/reverse-engineer-anything/SKILL.md @@ -117,10 +117,11 @@ explicit path or endpoint and do not need it. `inspect_android_package`, then focused Android tools when advertised. Archive member inventory still uses the archive route above. - Managed PE/CLI assembly: `inspect_managed_artifact`. -- Firmware image: `inspect_firmware_regions` when advertised. Use - `extract_firmware` when extraction is requested, with a caller-selected new - absolute output directory. These tools use caller-supplied Binwalk/Unblob on - Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md). +- UEFI firmware volumes and NVAR stores: use bundled `inspect_artifact`. + For external firmware scanning or extraction on Linux, use + `inspect_firmware_regions` or `extract_firmware` with caller-supplied + Binwalk or Unblob. Extraction needs a caller-selected new absolute output + directory. See the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md). - ESP32-family app or bootloader image (`0xE9` magic): `inspect_artifact` without a provider; see the ESP section of `references/native-and-artifacts.md`. - .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode; diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5f2a3c66b2..5019c3d9c8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -382,7 +382,7 @@ for commands, test placement, pruning, and real-provider prerequisites. Before requesting review, run relevant tests and `npm run check`; add `npm run docs:check` when contracts or generated metadata change. Record the checks and unavailable coverage. CI owns the full deterministic suite and -coverage. `npm run check:pr` is an optional full local gate for broad changes or +coverage. `npm run check:pr` is an optional broad local suite for large changes or CI diagnosis. Packaging/setup/distribution changes additionally require `npm run verify:package` and `npm pack --dry-run`; provider changes require the matching real-provider lane. State which host/provider workflows ran. diff --git a/README.md b/README.md index ff39a01428..6cbd8d77f7 100644 --- a/README.md +++ b/README.md @@ -227,7 +227,7 @@ Additional tools and host support depend on the target: | JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/providers/jeb-analysis.md) | | Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/guides/android/apktool-resource-analysis.md) | | Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/guides/android/flutter-analysis.md) | -| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/guides/firmware/firmware-analysis.md) | +| Firmware | Regions, extraction results and native-analysis handoffs | Bundled UEFI/NVAR; external Binwalk/Unblob on Linux; [firmware guide](docs/guides/firmware/firmware-analysis.md) | | Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/guides/data/packet-captures.md) | | Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/guides/data/pcap-analysis.md) | | Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/guides/signals/signal-recordings.md) | diff --git a/docs/development/testing.md b/docs/development/testing.md index 35cd0342ba..6811994ce2 100644 --- a/docs/development/testing.md +++ b/docs/development/testing.md @@ -44,7 +44,7 @@ that invoke PowerShell themselves remain subject to their own policy checks. | `npm run test:watch:all` | Changed tests from every project; builds at startup, so rebuild after production edits before relying on compiled tests | | `npm run check:fast` | Cached typecheck and lint, then the architecture guards and their counted baselines | | `npm run check:changed` | Cached typecheck/lint and branch-related source feedback | -| `npm run check:pr` | Opt-in complete local deterministic gate and generated-file checks | +| `npm run check:pr` | Opt-in deterministic test suite and generated-file checks | | `npm run docs:check` | Generated-document validation and fresh onboarding example consistency checks | | `npm run docs:build` | Generate and build the VitePress site for local preview | | `npm run verify:docs:routes` | Build the VitePress site and verify rendered local routes, anchors, images, and physical edit targets | @@ -262,11 +262,10 @@ does not cap the complete process family's memory; measure the full invocation when enforcing a host budget. Routine iterations and rebases need focused regressions and relevant checks. -Before handing off a PR, run `npm run check` and generated-document checks when -applicable; CI owns the full suite and coverage. Use the full local gate for -broad changes or diagnosing CI, rather than after every edit. Package/install -changes additionally need package verification; provider changes need actual -provider evidence. +Follow the [contributor checks](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#development-feedback-and-pr-verification) +before handing off a PR. Use `check:pr` for broad changes or CI diagnosis. +Architecture guards run in `check`, `check:fast`, and `check:ci:deep`. +The standalone test-discovery check runs in `check:ci:deep`. Vitest projects use up to two workers, bounded by host parallelism. GitHub Actions jobs own their runner and use up to four; see @@ -1203,15 +1202,14 @@ and job log also list up to 20 reported failed test files and the count of unhandled Vitest errors when reported. Partial reports are marked diagnostic only. Failed, cancelled, or missing selected shards still fail the required gate, and aggregate coverage thresholds remain enforced in full qualification. -Only a successful complete merge on the default branch publishes shard history, -and only from a push, schedule, or manual run. Fast runs publish -`test-timings-fast`. Full runs publish `test-timings-full`. Pull requests do not -publish it: the lookup reads one unpaginated artifact page, and pull-request -artifacts would crowd out the default branch. The build job downloads the -artifact that matches its tier, so fast shards use durations measured without -coverage and full shards keep coverage-weighted durations. A tier with no -qualifying artifact leaves history unset. The download is that small -checkout-relative timing file, not the coverage/JUnit bundle. Timing weights come from Vitest module +A complete shard merge on a default-branch push, schedule, or manual run can +publish timing history. Fast runs publish `test-timings-fast` and full runs +publish `test-timings-full`. The build job accepts only verified artifacts from +successful default-branch CI runs that match its tier. It lists one page of +matching artifacts and checks up to five producer runs individually. When +eligible artifacts remain, it can check them against one page of successful CI +runs. A tier with no qualifying artifact leaves history unset. The build +downloads the small, checkout-relative timing file. Timing weights come from Vitest module diagnostics and include tests and hooks, worker preparation, environment, imports, and setup. Import timings already include transform waits, which are not added again. The weights describe file costs under the measured worker and diff --git a/docs/elf-notes.md b/docs/elf-notes.md index 1694a72877..0f48851e44 100644 --- a/docs/elf-notes.md +++ b/docs/elf-notes.md @@ -42,7 +42,8 @@ For Linux `CORE` notes, `core.prpsinfo`, `core.file`, `core.siginfo`, or `core.auxv` are decoded for process arguments and status (`NT_PRPSINFO`), mapped file memory regions and paths (`NT_FILE`), fault signal and code (`NT_SIGINFO`), and auxiliary vectors (`NT_AUXV`) across 32-bit and 64-bit images. Other owners or unparsed -notes keep `core` null. +notes keep `core` null. `NT_FILE` and `NT_AUXV` words follow the ELF class and +byte order. Addresses and word values retain their unsigned hexadecimal form. An `NT_FILE` descriptor with nonzero bytes after its declared filenames retains its raw note evidence and leaves `core` null. @@ -112,7 +113,8 @@ bytes and up to 64 KiB of projected metadata JSON per report. A repeated descriptor consumes projected bytes for each entry that includes it. Metadata that would exceed the Evidence JSON nesting allowance is also left undecoded. -Zstandard section-name tables also use a 128 MiB decoder window limit. +Zstandard section-name tables also use a 128 MiB decoder window limit and the +shared [`zstd.max_frames` work limit](design/limits-and-recovery.md#runtime-settings-through-mcp). `SHF_COMPRESSED` note sections return `unsupported_target` before their payload is interpreted or decompressed. Their compression header is storage metadata, diff --git a/docs/guides/firmware/firmware-analysis.md b/docs/guides/firmware/firmware-analysis.md index ca4aad037b..5a493a30df 100644 --- a/docs/guides/firmware/firmware-analysis.md +++ b/docs/guides/firmware/firmware-analysis.md @@ -35,7 +35,8 @@ Unblob. FFS file and section discovery applies to FFS2 and FFS3 volumes. Volumes with other file-system GUIDs retain their header and extent, with the body reported as unparsed. REA checks the volume header and block map against the declared volume extent. FFS traversal accounts for volume extensions and -retains valid files after an unusable file header. Malformed FV extension +retains later files when the observed header or state establishes the next +file boundary. Otherwise the remaining bytes stay unparsed. Malformed FV extension entries and conflicts between `UsedSize` and the claimed erased tail appear as volume-scoped gaps. Readable FFS files remain selectable. A truncated file body does not receive a verified checksum verdict. For a file without a body diff --git a/docs/guides/javascript/javascript-application-workflows.md b/docs/guides/javascript/javascript-application-workflows.md index b5c0044f88..134939a517 100644 --- a/docs/guides/javascript/javascript-application-workflows.md +++ b/docs/guides/javascript/javascript-application-workflows.md @@ -214,8 +214,9 @@ candidates. A boundary nested inside another expression or compound assignment does not establish a direct result binding. The query declares backward provenance, forward influence, callers, or ownership and can restrict admitted relation kinds. REA returns every matching -seed and the complete reachable subgraph; callers do not choose node, relation, -depth, function, module, seed-match, or page budgets. +seed and its complete reachable subgraph within the admitted graph. Callers +do not choose node, relation, depth, function, module, seed-match, or page +budgets. Extraction limits and unknowns can still leave coverage partial. The extractor covers lexical definitions and reads, literal seeds, static property slots, object reads/writes/spreads/destructuring, closure captures, @@ -241,6 +242,14 @@ assignment. Computed method keys belong to the enclosing callable; parameter defaults and method bodies belong to the method. Uncertain later property reads remain opt-in through `include_ambiguous_dynamic_edges`. +Destructuring retains dependencies from the source slots proved at the time +of the read. A later write does not replace an earlier proved dependency. +A numeric key in an object pattern selects a named property directly. +Nested array patterns can preserve dependencies for later bindings when +element selection is known and iterator closing cannot invoke a return hook. +Holes, accessors, and unknown iteration keep the affected dependencies +uncertain. + Binary, logical, unary, template, conditional, and sequence expressions retain `influences` edges from supported immediate inputs to the result, then to a direct declaration, assignment, call argument, or return. These are static @@ -248,7 +257,8 @@ dependencies, not aliases or proof that the expression executed; conditional operands and logical-assignment writes carry `conditional: true`. Unsupported operand dependencies, unmodeled receiving containers, dynamic call results, and sequence side effects remain unknown. An applicable -unknown makes the trace status `partial` even when a seed was found. Literal +unknown makes the trace status `partial` when no applicable ambiguity takes +precedence, even when a seed was found. Literal operator operands, direct call arguments and directly returned scalar literals can also be seeds. Older graphs that only recorded a call-site literal remain searchable. The offered per-file node capacity also bounds the producer's @@ -318,7 +328,9 @@ behavioral absence. Semantic relations use static-inference authority. `resolved` means the static target identity was unique within admitted analysis; it does not mean the path executed. Candidate edges are excluded by default, require explicit opt-in, and -keep the result ambiguous. Runtime Evidence can later corroborate an exact +keep the result ambiguous. An exact edge may still be present when other +possible dependencies make the overall trace ambiguous. Runtime Evidence can +later corroborate an exact mapped candidate, but structural reachability, semantic influence, runtime observation, and causal proof remain separate claims. @@ -521,9 +533,10 @@ properties. Dynamic child writes such as `copy[key].value = 2` remain conservati about which original slot was selected. Object rest excludes consumed keys, array rest excludes the consumed prefix for known arrays, and later explicit data properties and methods replace earlier object -spread references. A known array prefix does not expose values from a following -spread. Accessors, custom or uncertain iteration, unknown computed keys, and -unknown spread lengths retain conservative uncertainty. +spread references. A bounded dense inline array spread can supply exact slots +and keep the positions of later elements known. Accessors, sparse or unknown +spread sources, custom or uncertain iteration, and unknown computed keys keep +the affected reads uncertain. Getter-local aliases of `this`, including direct destructuring and assignments, retain their receiver when a returned shared reference is written. These writes invalidate the earlier literal instead of claiming the getter executed or the @@ -545,8 +558,8 @@ Writes through an object getter's returned receiver retain that uncertainty when the object is an array element, nested inside mixed object/array containers, or assigned to a previously declared identifier. Saving the getter result in an alias does not make its old field values immutable. Known array -indices include holes; an element after a spread has only a minimum possible -index. For these returned-reference writes, unrelated properties and known +indices include holes. After a spread with unknown length, a later element +has only a minimum possible index. For these returned-reference writes, unrelated properties and known array prefixes remain precise. Accessor applicability uses the shared object reference owner for literal definition order, own-property shadowing, known array elements and explicit literal prototypes. Getter invocation effects diff --git a/docs/guides/managed/nuget-nupkg.md b/docs/guides/managed/nuget-nupkg.md index 57c14345e6..b97b136cf0 100644 --- a/docs/guides/managed/nuget-nupkg.md +++ b/docs/guides/managed/nuget-nupkg.md @@ -29,12 +29,13 @@ requires an absolute filesystem path and preserves that exact selection: The inline Evidence result contains the artifact path, SHA-256, and byte length. `identity` reports `id`, `version`, `authors`, and `description` as present in -the `.nuspec` metadata. For authors and description, NuGet indexes the first -nonempty leaf with a matching local name, ignoring case. The package ID and -version use their case-sensitive names in the metadata namespace. Missing -fields are `null`. The `.nuspec` and OPC -relationships go through XML end-of-line handling, as NuGet's reader applies -it: CRLF and lone CR read as LF. When no `.nuspec` can be +the `.nuspec` metadata. Authors and description follow NuGet's first eligible +leaf rule with case-insensitive local names. Empty leaves and ordinary +whitespace-only leaves are skipped. Whitespace preserved by CDATA or +`xml:space` can occupy the first slot and yield `null` even when a later leaf +has text. The package ID and version use their case-sensitive names in the +metadata namespace. Missing fields are `null`. In the `.nuspec` and OPC +relationships, XML end-of-line handling maps CRLF and lone CR to LF. When no `.nuspec` can be selected, `identity` is `null`. `license` reports the `.nuspec` license element: `type` is `expression` or diff --git a/docs/guides/native/windows-shortcut.md b/docs/guides/native/windows-shortcut.md index 0c89c45ec4..1eba99e076 100644 --- a/docs/guides/native/windows-shortcut.md +++ b/docs/guides/native/windows-shortcut.md @@ -36,7 +36,9 @@ decoded as `normal`, as specified. file offset, size, and type byte. Root-folder, volume, and file-entry items are decoded when those layouts match documented shell-item headers, including the 8.3 short name on a file entry. Every other item is `raw` with a SHA-256 -of its payload. Unknown items are not guessed. +of its payload. An invalid item extent, missing TerminalID, or bytes after the +TerminalID make coverage partial. Earlier items remain available, and parsing +resumes at the declared end of the list when that extent fits the file. `link_info` reports volume drive type, serial, label, local base path, and common network relative link names, with Unicode variants when the header @@ -61,9 +63,9 @@ birth-droid GUIDs. Version-1 UUID MAC and timestamp values are labelled `evidence_kind: "derived"`. `KnownFolderDataBlock` and `SpecialFolderDataBlock` keep their folder identifiers. `ConsoleFEDataBlock` keeps the code page. `PropertyStoreDataBlock` and unrecognized signatures stay raw with SHA-256. -Unknown signatures are listed and are not dropped. Decoded fixed-size blocks -with a mismatched length are retained as raw payload hashes and reported as -partial. Later blocks remain readable at their declared offsets. +Decoded fixed-size blocks with a mismatched length retain raw payload hashes +and make coverage partial. Later blocks remain readable when the declared +block extent fits the file. When `HasExpString`, `HasDarwinID`, `HasExpIcon`, or `RunWithShimLayer` declares a block, the terminal record lets REA report a missing block and mark coverage partial. If parsing stops before the terminal, an unseen block remains @@ -92,9 +94,10 @@ extension. Raw fields and artifact identity are preserved. The code page in a ConsoleFE block describes the console and does not establish the original system ANSI encoding for these paths. -An unreadable declared `LinkInfo` target path leaves the target extension -unknown and prevents a mismatch claim, unless another complete target matches -the icon. +An unreadable LinkInfo target or rejected environment target leaves the target +extension unknown. An independently established full-path match still gives +`false` for `icon_location_differs_from_target`. Without such a match, the +comparison remains unknown. A missing optional section is observed absence of that structure. It is not proof the shortcut never carried that data in another container. @@ -109,10 +112,10 @@ Malformed `HeaderSize` or `LinkCLSID`, and exhausted budgets, fail without partial success. Truncation after a valid header, ExtraData size overruns, and odd Unicode lengths return partial coverage with `coverage.failing_offset`. A LinkInfo, VolumeID, or network-name string offset that points inside its -structure header or outside its structure is reported at the offset field itself, -so every issue offset lies inside the file. A `LinkInfoHeaderSize` below 0x1C -does not shrink the mandatory 28-byte header, so an offset in that header is -still reported at the offset field and the path stays unreadable. +structure header or outside its structure is reported at the offset field. +A `LinkInfoHeaderSize` below 0x1C does not shrink the mandatory 28-byte +header, so an offset in that header is still reported at the offset field +and the path stays unreadable. Coverage is never `complete` when issues are present. A PropertyStoreDataBlock shorter than 12 bytes or a ShimDataBlock shorter than 136 bytes makes coverage partial. The report retains the block's signature, diff --git a/docs/guides/web/web-source-location.md b/docs/guides/web/web-source-location.md index f98e59b6b7..bac52b7529 100644 --- a/docs/guides/web/web-source-location.md +++ b/docs/guides/web/web-source-location.md @@ -234,17 +234,17 @@ is `--source-map-bytes 8388608`; the other local flags are `--source-map-section-depth` and `--source-map-codec-heap-mib`. `analyze-web-bundle` instead offers `--source-map-fetch-bytes` and the shared output/record/depth flags. Raising only `decoded_records` does not raise the -independent row or segment budgets. Failed local decoding returns no partial -point evidence; completed browser map observations retain their existing -partial/unknown semantics. Wait for retained cleanup to settle before retrying. +independent row or segment budgets. A failure before local decoding produces +a valid report leaves point evidence unavailable. Cancellation or incomplete +cleanup after decoding can retain the finished report in partial Evidence. +Wait for retained cleanup to settle before retrying. -## Verification +## Browser collection results Browser bundle collection uses absolute section offsets for nested inline maps. If caller cancellation stops browser map fetching after some items complete, the failed bundle request retains those items with the captured page in partial -Evidence and marks bundle analysis `not_performed`. This is distinct from the -local source-map decoder's failure semantics above. +Evidence and marks bundle analysis `not_performed`. An enclosing column offset contributes only when the nested section starts on that enclosing section's first generated line. Parent boundaries still constrain preceding browser mappings when a nested section starts with a gap or has no @@ -290,6 +290,8 @@ source-map decoding, and retains the same completed-body identity. Successful and partial maps keep their full original text, including BOM and XSSI framing; only their parsing view removes it. +## Verification + Focused tests cover duplicate mappings, indexed offsets/overlap/local indexes, 32-bit VLQ overflow, nullable content, UTF-16/CRLF positions, immutable artifact identity, cancellation, late output closure, cleanup failures, SDK schemas and diff --git a/docs/providers/ghidra-dos.md b/docs/providers/ghidra-dos.md index 492f86277d..7b72d9ad25 100644 --- a/docs/providers/ghidra-dos.md +++ b/docs/providers/ghidra-dos.md @@ -156,9 +156,12 @@ FileBytes digests, every source mapping and its memory digest, relocation record module partitions, relocation fixups and linear coordinates rather than relying on the imported executable SHA alone. -Expected DOS relocation records retain each fixup's pre-application bytes and -applied value. Their memory-byte windows come from the final relocated image, -including changes made by later overlapping fixups. +REA computes expected MZ relocation records in table order. Each expected +record keeps the word before and after its fixup, and its memory-byte window +comes from the completed image. Ghidra supplies the observed relocation +records and final mapped memory separately. A historical Ghidra 12.1.4 probe +found a relocation-record mismatch despite identical final bytes. The overlap +case is still unverified on Ghidra 12.2. [Verification issue](https://github.com/morluto/rea/issues/12590) The result is `verified`, `mismatch`, or `unsupported`. Every check retains its expected and observed values. A memory-digest mismatch identifies the source @@ -206,15 +209,8 @@ there, and explicit COM is not admitted. macOS x64 DOS verification remains unve npm run verify:ghidra:dos ``` -The lane generates a small public fixture from source at runtime. It verifies -16-bit instruction bytes, a near call, a segment-relocated far call, actual -decompilation, disjoint function ranges, stable CLI/MCP observations, unchanged -input bytes, and session/process cleanup. It does not require external -executable fixtures, an emulator, or an existing analysis database. - -Provider p-code representations can contain process-specific address-space selector -tokens. The lane preserves those reported values and compares stable dossier -observations rather than asserting cross-process identity of raw p-code. +See the [DOS verification lane](../testing-lanes.md#dos-ghidra-analysis) for +fixtures, platform coverage, and provider checks. ## Function annotations diff --git a/docs/providers/ghidra-nativeaot.md b/docs/providers/ghidra-nativeaot.md index 96d45fe7e8..10a625afaa 100644 --- a/docs/providers/ghidra-nativeaot.md +++ b/docs/providers/ghidra-nativeaot.md @@ -103,37 +103,26 @@ Ghidra analysis profile; non-PE profiles do not include that contract. ## Evidence boundaries -The built-in parser uses an initialized non-executable PE signature scan and -requires a unique RTR 9.1 header and exactly one DEHYDRATED_DATA row. Rehydrated -bytes live only in a parser-owned overlay, identified by virtual address, size, -and SHA-256; the caller's source Buffer remains unchanged. Limits derive from -captured section extents and format field bounds. Separate resource policies -admit up to 128 MiB of captured PE bytes, 64 MiB-equivalent parser work, 64 MiB -of exact parser-owned overlay/index storage, and 4 MiB of serialized NativeAOT -report JSON. These are local parser limits, independent of MCP delivery; the -buffer counter does not claim to cap V8 heap or total process memory. Rows that -do not fit the report budget are omitted with explicit partial status and -coverage. Coverage counts describe the pointer scan, recovered MethodTables, -and frozen-string candidates examined, not every possible runtime type or -object. Cancellation is honored during snapshot I/O and checked immediately -before and after parsing; the synchronous parser call itself is bounded but -cannot be interrupted mid-call. Parser output never claims that a Ghidra -database type was defined. +The built-in parser scans readable, non-executable PE sections within their +file-backed virtual extents. It requires a unique RTR 9.1 header and exactly +one DEHYDRATED_DATA row. The directory header and rows must fit the same +section. Rehydrated bytes stay in a parser-owned overlay identified by virtual +address, size, and SHA-256. The source bytes remain unchanged. Evidence +from the overlay establishes parser findings. Ghidra database types require +the optional adapter and a database lookup. -RTR signature omission counts include every eligible section before the work -budget is spent, including later sections that were never visited. MethodTable -closure indexes each validated table's related pointer once and visits a work -queue for inheritance and interface reachability. Its temporary index and queue -consume the existing working-memory budget; the parser does not increase its -work limit. Candidate headers/counts are parsed once; unmapped table bodies are -rejected before charging table entries, and validation charges only each slot or -interface pointer actually examined. Declared counts and unvisited tails do not -become examined coverage. Discovery still must finish before claiming a unique System.Object -root, so a discovery budget stop can return zero recovered tables without -establishing their absence. Performance on the large bflat report has not been -measured. Indexed closure and candidate validation avoid separate repeated or -unperformed work, but do not establish that the fixed discovery budget admits -that artifact. Budget stops remain partial results with unknown type absence. +The parser admits up to 128 MiB of captured PE bytes. Its `work_budget_units` +starts at 64 × 2²⁰ work units and adds eight units per admitted source byte. +Separate ceilings apply to parser-owned overlay and index storage (64 MiB) +and serialized report JSON (4 MiB). These limits count the stated resources. +They do not measure total process memory or MCP response delivery. + +Coverage counts work actually examined. Declared table sizes and unvisited +tails do not become examined records. A budget stop yields partial output. +If discovery stops before it proves a unique System.Object root, zero +recovered tables leaves their presence unknown. Snapshot I/O honors +cancellation. The bounded synchronous parse checks cancellation before and +after it runs. If captured snapshot bytes no longer match their admitted digest, the parser retains both digests and the Ghidra owner reports `artifact_changed`, with the @@ -158,7 +147,7 @@ does not imply that ordinary native disassembly/decompilation is impossible. ## Verification -See [the dedicated lane](../development/testing.md#optional-nativeaot-ghidra-analysis). +See [the dedicated lane](../testing-lanes.md#optional-nativeaot-ghidra-analysis). Fixtures are source-built benign .NET programs with inheritance, an interface, virtual dispatch, representative arithmetic and frozen strings. Independent compiler/linker symbols and raw directory bytes provide the oracle. Stripped @@ -166,8 +155,8 @@ fixtures exercise heuristic detection. Tests do not execute the resulting NativeAOT binaries. Fixture generation and adapter compilation are optional and never prerequisites of ordinary native analysis or setup. -The real MCP lane runs only on a Linux x64 host with the adapter. The -Windows-host behavior above was checked manually over MCP on Windows 11 x64 +The real MCP lane runs on Linux x64 with the adapter. A historical +Windows-host check ran manually over MCP on Windows 11 x64 (non-elevated, Ghidra 12.1.4, JDK 21, no JAR) with the source-built win-x64 symbols fixture. The format, RTR version and frozen strings matched the fixture oracle; the derived MethodTable, its base type, its interface and a virtual diff --git a/docs/release-exposure.md b/docs/release-exposure.md index ec884957c7..f2393af31a 100644 --- a/docs/release-exposure.md +++ b/docs/release-exposure.md @@ -71,6 +71,6 @@ A map read counts against the source-map allowance when it starts, so an oversiz `exposed` means the check's condition was observed. `not_observed` means that condition was absent in a completed scan. `not_applicable` means the check does not apply to this artifact, for example Electron fuses on an app bundle with no Electron Framework, or on a directory whose native images were all read and hold no fuse sentinel. `unknown` means the check could not be decided, including a directory with no ASAR layout where those reads are missing or incomplete (an `asar: false` Electron build has no ASAR to find), safe fuses from an incomplete inventory, an unopened packaged native image, or a dropped image when the sample did not read an executable named as the Electron host or every executable that could be it, a missing fuse sentinel on an Electron target, a universal binary with a slice that has no sentinel or an undecided fuse, a file that changed after inventory, Electron fuses on a selected ASAR (which does not include the executable that holds them), a PE inside a package that was too large to read, Go or Rust metadata on a native image stored inside a package, and Go or Rust metadata that was not present. Absence of that metadata is not proof the binary is something else. `not_assessed` means this audit did not run the check. -Cancellation during a Go, Rust, fuse, or desktop read returns a typed cancellation error. Its partial observation holds the report with completed facts, and checks whose readers did not run are `unknown`. Cancellation during the member scan returns the same typed cancellation error. Its partial observation holds the members examined before the cancellation, with member coverage `unknown` and a limitation saying the scan was cancelled. Cancellation before the scan can be assembled, such as during inventory, returns no partial report. +Cancellation during a Go, Rust, fuse, or desktop read returns a typed cancellation error. Its partial observation holds the report with completed facts, and checks whose readers did not run are `unknown`. Cancellation during the member scan returns the same typed cancellation error. Its partial observation holds the members examined before the cancellation, with member coverage `unknown` and a limitation saying the scan was cancelled. Cancellation before the scan can be assembled, such as during inventory, returns no partial report. If a reader still owns a resource that could not be closed, the workflow returns `cleanup_incomplete` with that owner and any partial report it could assemble. See [lifecycle completion](development/tool-design.md#own-the-lifecycle). Severity is `null` unless the status is `exposed`. diff --git a/docs/testing-lanes.md b/docs/testing-lanes.md index 082cae6d6a..c8daab7938 100644 --- a/docs/testing-lanes.md +++ b/docs/testing-lanes.md @@ -826,8 +826,8 @@ its own complete result at the exact released revision. Package lanes assert the actual Node platform/architecture before verification and record those values with the Node version. The package matrix runs up to four jobs concurrently on independent hosted runners with explicit -timeouts and per-runner Node heap/thread limits; Windows retains its curated -capability lane. Package checks cover installation, CLI/MCP discovery, +timeouts and per-runner Node heap and thread limits. Windows runs its curated +capabilities and installed-package suites in separate jobs. Package checks cover installation, CLI/MCP discovery, target-free analysis, configuration backups/recovery, Evidence and owned lifecycle; Inspector checks execute source-owned loopback targets and special filename cases. @@ -854,13 +854,10 @@ catalog, and managed-conformance metadata. Each shard still installs the locked dependencies and runs its assigned tests; full qualification enables coverage. It no longer regenerates the same metadata independently. -The build job finds the newest trusted timing artifact for its tier -(`test-timings-fast` or `test-timings-full`) from a successful default-branch -push, schedule, or manual CI run, and includes that same report in each shard's -snapshot. The bounded lookup verifies the producer. Pull-request runs and other -branches are not producers. A tier with no qualifying artifact leaves history -unset, and Vitest uses its own assignment. A later schedule that reuses -qualification without uploading timings leaves the previous artifact in place. +The build job includes any +[verified timing history for its tier](development/testing.md#coverage-and-timing) +in each shard's snapshot. A later schedule that reuses qualification without +uploading timings leaves the previous artifact in place. Each tier keeps its own history because stale weights from another tier, or from before a large test change, unbalance the shards.