mirror of
https://github.com/morluto/rea.git
synced 2026-11-10 16:57:19 +00:00
cb4e5cfbff25c2dda8683ef8598c17cec8a73473
31
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 morluto
|
cb4e5cfbff |
feat(analysis): make resource budgets recoverable across CLI and MCP (#9706)
* refactor(analysis): centralize resource budgets and recovery facts * feat(analysis): expose ASAR and Ghidra runtime admission controls * feat(artifacts): expose reader admission and preserve limited recognition * feat(javascript): expose recoverable semantic precision budgets * feat(web): add recoverable source-map budgets and compact retained views * feat(web): make historical capture decoding budgets recoverable * fix(desktop): enforce selected view policies across capture and export * feat(runtime): make Frida and LLDB retention budgets selectable * feat(native): expose diagnostic budgets and preserve exhaustion causes * feat(managed): expose recoverable snapshot and representation budgets * feat(metadata): make Go and Rust inspection budgets recoverable * feat(wasm): expose representation and producer admissions * feat(evm): expose per-call worker resource budgets * feat(android): make resource and string-pool omissions recoverable * feat(producers): expose command and recorded-capture budgets * docs(analysis): record recovery contracts and integrated validation * fix(docs): link calibration oracle from published verification pages * fix(docs): restore desktop task anchors and receipt links * test(analysis): align fixtures with selectable budget contracts * fix(analysis): remove source-location projector dependency cycle * test(analysis): refresh budget recovery and verifier regressions * test(analysis): preserve format and publication recovery expectations * test(windows): align conformance with owned runtime admission * fix(desktop): preserve selected alias kinds in exported state * test(analysis): align provider fixtures with recoverable budgets * test(desktop): validate emitted alias metadata structurally * fix(ghidra): classify bounded Windows endpoint refusals * test(cli): assert selectable artifact admission recovery * fix(cli): preserve capture provenance in environment builds |
||
|
|
8bc01b48b4 |
feat(native): list ELF and Mach-O initializer and finalizer functions (#9574)
* feat(native): list ELF and Mach-O initializer and finalizer functions inspect_native_initializers and inspect-native-initializers list the functions an ELF or Mach-O image runs before and after main: ELF DT_INIT, DT_FINI, preinit, init and fini arrays and .ctors/.dtors, and Mach-O __mod_init_func, __mod_term_func and __init_offsets, with slot addresses, resolved targets, symbols where available and relocation facts for relocatable objects. Nothing is executed. Slot, symbol, relocation and note budgets are shared across tables, sentinels are matched at the image width, and target-derived names are bounded and marked untrusted. * fix(native): keep ELF initializer slots unresolved when relocation coverage is partial A skipped, unreadable or over-budget section relocation table now keeps every slot it could reach unresolved instead of reading the stored word as raw or dropping a .ctors slot as a sentinel. Dynamic writes that only partly overlap a slot, or write it twice, leave it unresolved. Linked images no longer reapply retained non-SHF_ALLOC link-time records. The dynamic relocation walk now pauses at the shared reader's step boundaries and observes cancellation there, resolved targets need only their first byte mapped, and unreadable symbol names make the report incomplete. * fix(native): stop reporting unchecked Mach-O initializer words as resolved Object files now apply their own section relocations: an UNSIGNED entry against a defined symbol gives symbol_addend, one against an undefined symbol gives unresolved_import, and any other entry leaves the slot unresolved. Linked images without a fixup decoder, such as 32-bit ones, keep pointer slots unresolved when an LC_DYLD_INFO bind stream or LC_DYSYMTAB external relocations could replace the stored word. LC_DYLD_INFO rebases now report dyld_info_rebase instead of chained_fixup, and unreadable symbol names make the report incomplete. Section headers keep their declared reloff and nreloc for this. * docs(native): move initializer guide under docs/guides/native The documentation guide assigns analyst workflows to docs/guides/<domain>/. Move the guide there, place its navigation and directory entries in the Native binaries group, and update links from the docs index, CLI guide, spec and authored skill. The public /native-initializers route is unchanged. The guide and spec also describe per-slot relocation coverage, retained link-time records, Mach-O object relocations, undecoded bind streams, the dyld_info_rebase resolution and unreadable names. * fix(native): keep IFUNC and partly relocated ELF initializer slots unresolved An absolute relocation against an STT_GNU_IFUNC symbol no longer reports the resolver address as the target. Section relocation records now use their real write width, so a record that writes only part of a slot, starts inside it or repeats it leaves the slot unresolved instead of reading a raw zero or dropping a .ctors slot as a sentinel. On 32-bit ARM the Thumb bit is cleared when locating a target's section and file offset, while value_va keeps the stored callable value. * fix(native): locate Thumb Mach-O initializer targets and reject arm64_32 with a 64-bit header On CPU_TYPE_ARM the Thumb bit is cleared when locating and naming a target, since nlist values do not carry it. An arm64_32 image that declares a 64-bit header is rejected as malformed rather than read with 8-byte slots. The guide and spec describe IFUNC relocations, partly relocated section slots, Thumb targets and this rejection. * fix(native): index initializer address lookups and keep ambiguous mappings unknown ELF and Mach-O initializer lookups now build the shared file mapping index once per image, so each slot and target lookup is logarithmic in the number of load segments rather than a scan of every mapping. An address that load segments map ambiguously no longer falls back to a section header offset. Its file offset is reported as unknown and the report is incomplete; only genuinely unmapped spans use the section fallback. * fix(native): report Mach-O bind names over the name budget A bound import name longer than the 1024-byte name budget was dropped without a trace, although it is the only identity of an unresolved initializer target. It now counts toward the over-long name limitation and makes the report incomplete. * test(native): count mapping lookups instead of timing the load index regression The load-mapping regression asserted a wall-clock bound, which is sensitive to machine load. It now spies on the shared mapping module and requires the same number of mapping scans and index builds for 8 and 64 slots, so a per-slot scan of every load segment fails deterministically on a small fixture. * fix(native): yield while decoding legacy rebase runs * fix(native): preserve initializer mapping and coverage * fix(native): index legacy fixup spans * fix(native): preserve relocation uncertainty and bound bind lookup work * fix(native): validate ELF slot mappings and yield section relocation scans --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
morluto
|
deae5fbe6d |
docs: correct workflow claims and clarify historical specifications (#9655)
* docs: correct lifecycle and analysis workflow claims * docs: distinguish historical specifications from current guides * docs: align translated setup and MCP recovery guidance |
||
|
Nianze Wuandmorluto
|
759817ccbe |
fix(javascript): distinguish Promise and FileHandle releases (#9584)
* fix(javascript): distinguish Promise and FileHandle releases * fix(javascript): preserve reassigned handle candidates * test(javascript): select unisolated route fixtures explicitly * test(javascript): distinguish package scope in route fixtures * test(apple): separate identity and refusal fixture groups * fix(javascript): retain uncertain releases after opaque receiver writes --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
RRFZandmorluto
|
da3bb5aa7b |
fix(managed): report InternalCall methods as runtime-provided boundaries (#9547)
* fix(managed): report InternalCall methods as runtime-provided boundaries inspect_managed_native_boundaries listed P/Invoke, native, runtime and unmanaged methods, but not methods with MethodImplOptions.InternalCall (0x1000). The runtime implements those bodies; mscorlib and System.Private.CoreLib declare hundreds of them as FCalls. They now appear in native_implementations with boundary_kind runtime-provided. * fix(managed): keep InternalCall bodies out of CIL decoding The native-boundary view reports InternalCall methods as runtime-provided. Body admission still decoded CIL from a nonzero RVA when the row kept IL and managed flags. CoreCLR rejects such a row (BFA_INTERNAL_METHOD_WITH_RVA). Body admission now treats InternalCall like the Runtime code type. The method gets a partial body with no instructions. The InternalCall test covers both a zero and a nonzero RVA. * test(javascript): distinguish package scope in route fixtures * test(apple): separate identity and refusal fixture groups * fix(managed): honor InternalCall during body admission --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
|
43f06eb1e8 |
feat(native): recover ELF function ranges from unwind tables (#9585)
* feat(native): recover ELF function ranges from unwind tables inspect_elf_unwind_ranges and inspect-elf-unwind-ranges read .eh_frame and .eh_frame_hdr from one local ELF32 or ELF64 image and list the function ranges its FDEs name, with CIE facts, LSDA and personality, the containing section and an exact symbol where one starts at the address. Nothing is executed and no disassembler is needed. Pointer encodings follow the DW_EH_PE rules with raw zero as null, discarded FDEs are counted and skipped, the .eh_frame_hdr search table is compared with the decoded FDEs, and record, row and symbol budgets are bounded. verify:elf:unwind-ranges compares the reported starts with llvm-dwarfdump or readelf on a compiled C++ program. * fix(elf): keep unresolved unwind facts out of reported ranges Skip FDEs whose initial location is indirect and do not locate .eh_frame through an indirect eh_frame_ptr, since pointer slots are not functions. Stop an unbounded header-located scan at a CIE with an unrecognized augmentation, report ignored duplicate PT_GNU_EH_FRAME headers as incomplete coverage, and accept a null personality stored under an indirect encoding. Keep only symbols at decoded FDE starts, cache name observations per string-table offset, and decode names only for the returned page under a fixed 16 MiB text allowance, so repeated or long names no longer grow memory with the symbol count. * fix(scripts): run the ELF unwind verifier under owned process trees Launch probes, compiler drivers, the stripper, the frame dump and the inspection through createVerifierCommands, as verify-binary-hardening does, so a timeout or interrupt stops the whole tree. The temporary directory is removed only after the owned commands have closed. * docs(elf): move ELF unwind ranges guide under docs/guides/native The documentation guide assigns analyst workflows to docs/guides/<domain>/. Move the guide there, place its navigation and directory entries in the Native binaries group, and update links from the docs index, CLI guide, spec and authored skill. The public /elf-unwind-ranges route is unchanged. * fix(elf): give unwind text limits their own recovery and skip indirect tables Throw a dedicated failure when a page's symbol names exceed the 16 MiB text allowance, and map it to a resource error that tells the caller to request a smaller limit rather than raise max_ranges. Read a CIE augmentation with one bounded NUL search and keep it as a byte view, decoded only for returned CIEs, so a long augmentation is never built up character by character. Leave an .eh_frame_hdr search table with an indirect encoding unread: the header is incomplete, the comparison is unavailable, and the table no longer bounds a header-located .eh_frame scan. * fix(elf): resolve header count and table forms and name IFUNC resolvers Leave a non-zero .eh_frame_hdr FDE count under an indirect encoding unresolved, since it is the address of a slot that is not followed, and mark the header incomplete without reading the table. Treat a DW_EH_PE_omit table encoding as an omitted search table before testing the indirect bit, so a header that declares a count without a table stays present with complete coverage, and accept that form in output validation. Match STT_GNU_IFUNC symbols at exact FDE starts after function symbols, with a limitation that the name is the resolver, not the implementation it selects at run time. * fix(elf): require allocated unwind pointer bases * fix(elf): map unwind header relative base * fix(elf): validate unwind load mappings * test(elf): cover mapped nonallocated frame agreement --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
|
f21fda0fc0 |
feat(elf): report a Linux kernel image's version banner and embedded build config (#9573)
* feat(elf): report a Linux kernel image's version banner and embedded build config inspect_kernel_build_config and inspect-kernel-build-config read one local vmlinux, raw arm64 Image, gzip bzImage or zImage, or configs.ko and report its Linux version banner and the IKCONFIG .config embedded between IKCFG_ST and IKCFG_ED, without executing it. A config is present only when a gzip member ends exactly at IKCFG_ED, passes its CRC-32 and ISIZE checks and parses as .config text; framing that fails is reported malformed, never absent. bzImage gzip payloads are inflated within a budget and other codecs are named as unsupported, using codec detection now shared with the kernel module parser. The config search, inflate sizes and option counts are bounded, scans honor cancellation, and target-derived text is marked untrusted. * fix(elf): reject a gzip member whose header checksum fails When FHCRC is set, compare the low 16 bits of the header CRC-32 before inflating, so a damaged header no longer yields a present config or an inflated bzImage payload. * fix(elf): detect the bzImage codec inside the declared payload Codec detection now reads the same payload_offset/payload_length slice that is inflated, so bytes past a short payload cannot name its codec. * fix(elf): report a framed config that does not decode as malformed A damaged or cut-off deflate stream between IKCFG_ST and IKCFG_ED was skipped and could end as an absent config with complete coverage. It is now malformed with config_gzip_invalid, and a later valid config still wins. * fix(elf): scan config lines in place and accept long string values Config text is read one line at a time instead of split into an array, and a quoted value is validated before it is decoded once. The 4096-character string cap is removed: Kconfig has no such limit and max_config_bytes already bounds every line, so a long CONFIG_EXTRA_FIRMWARE list is no longer reported as malformed. * fix(elf): let a live cancellation reach the kernel config parser The parser ran synchronously, so an abort raised during parsing could not change the signal before it returned. Inflation now runs on the zlib thread pool and each scan yields to the event loop once per MiB, so the existing cancellation checks observe a real abort. * refactor(elf): carry the refused kernel config budget as a typed fact Budget failures are now KernelBuildConfigBudgetFailure with a knob field, and the provider builds its recovery advice from that field instead of searching the diagnostic message for an option name. * docs(elf): move the kernel build config guide under docs/guides/native The documentation guide assigns analyst workflows to docs/guides/<domain>/. Move the guide there, place its navigation and directory entries in the Native binaries group, and update links from the docs index, CLI guide, spec and authored skill. The public /kernel-build-config route is unchanged. * fix(elf): yield while checking one long config line A config made of one long quoted value never reached the per-line yield, so the control-byte check, value validation and SHA-256 ran in one synchronous block. Each now works one MiB window per event-loop turn, and the line patterns match only a bounded name prefix instead of the whole value. * fix(elf): return paged config entries that do not pin the decoded text Entry names and values were V8 slices of the whole decoded config, so a one-entry page kept the full text alive. Entries in the returned page are now standalone copies, so retained memory follows the page size. * fix(elf): yield while kernel marker searches find nothing IKCFG_ST, IKCFG_ED and banner searches used one Buffer.indexOf over the rest of the image, so a large image with no marker never yielded. They now search overlapping one MiB windows and yield after each window without a match. * fix(elf): preserve long kernel config names * fix(elf): frame kernel configs by gzip member boundaries --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
morluto
|
7b9b6cc78b |
fix(electron): preserve unknown indirect listener ownership (#9653)
* test(javascript): distinguish package scope in route fixtures * test(apple): separate identity and refusal fixture groups * fix(electron): preserve uncertainty in indirect listener ownership |
||
|
morluto
|
b1f473fcf8 | docs: clarify publication recovery and Evidence ownership (#9654) | ||
|
morlutoandSonai Biswas
|
c5ee066022 |
fix(process): preserve semantic comparison across capture-local timing (#9644)
fix(process): stabilize capture comparisons Co-authored-by: Sonai Biswas <sonai.bsws124@gmail.com> |
||
|
|
3230f50b16 |
feat(sqlite)!: page selected rows with a stated order and row offset (#9331)
* feat(sqlite): page selected rows with a stated order and row offset readRows read `NOT INDEXED LIMIT ?` with no ORDER BY and reported order: "unspecified", so an agent could not reach rows past row_limit deterministically: raising row_limit hits the fixed 16 MiB reply budget, and the sampled order depends on whichever index SQLite happened to scan. Read the window in a unique, stated order and expose the position: - rowid tables order by rowid unless all of rowid, _rowid_ and oid are declared columns, in which case the order stays unspecified. - WITHOUT ROWID tables order by the primary-key columns, reusing each key's DESC flag and declared collation from pragma_index_xinfo so the plan keeps index order instead of materialising a sorter. - rows.order, rows.row_offset and rows.next_row_offset report the window; next_row_offset is null once the window reaches the end. - row_offset (CLI --row-offset) requires table and defaults to 0. A non-zero offset is refused when the table declares no usable order, because the skipped prefix would be arbitrary. The schema now checks next_row_offset against truncated and the offset so a worker reply cannot claim a page it did not read. One attribution fix came with it: adding ORDER BY text can push an over-wide projection past SQLite's SQL-text limit, which fails as SQLITE_TOOBIG before the result-column limit is reported. Attribute that case to column capacity too, so a 250k-column request still reads as capacity rather than a byte budget. Verified: npm run check:fast, npm run format:check on the touched files, npm run verify:module-boundaries, npm run docs:check, and 196 tests across the sqlite boundary, application and CLI/MCP suites. A 1002-row inspection and in-memory EXPLAIN QUERY PLAN checks confirmed the WITHOUT ROWID path adds no sorter. Full check:pr and the complete test suite were not run. * fix(sqlite): resolve the rowid alias and keep paging claims reachable Three defects in the row paging added alongside row_offset: - readRowOrder emitted ORDER BY rowid for every rowid table, but a table that declares a rowid column shadows that spelling, so the sort used the stored column and could repeat or drop rows across pages. Order now names the first alias the table leaves unbound, read from raw name bytes so an unrelated malformed identifier is not decoded just to choose an order. - The inventory projection could hide the primary key a WITHOUT ROWID table orders by, silently degrading a paginated read to unspecified order. The order key is now read from the table itself rather than from the columns the caller asked to see. - A truncated table with no stated order reported a next_row_offset that the row_offset rule then refused. It is null there, and the reply limitation no longer claims an unspecified order for windows that have one. rowEvaluationFailure also probed the first rows of the table instead of the requested window, so a failure confined to a later page was attributed to columns that evaluate fine there. * test(sqlite): assert the concatenated pages The paging regression checked only counts, offsets and flags, so a second page that repeated the first page's rows would still pass. Assert the values read across all three pages instead. * fix(sqlite): decode row keys in the database encoding readRowKeyFacts decoded primary-key names as UTF-8 and probed a fixed hidden set, so two supported schemas could not be paged at all: - A UTF-16 database stores key names as UTF-16 bytes, so "k" decoded to NUL-padded text and ORDER BY named an identifier that does not exist. Pass the detected encoding, as readColumns and the index-metadata path already do. - A virtual generated column named rowid (hidden=2) was absent from the shadow check, so ORDER BY rowid evaluated the generated expression instead of the rowid and could sort non-uniquely or fail outright. Include generated columns in the alias probe. Also stop decoding primary-key names for rowid tables: their ORDER BY names an alias, never a declared key, so an unselected malformed key name could abort an otherwise readable selection. The page-concatenation assertion now narrows SqliteValue before reading .value, which the null/blob arms do not carry; this was failing the Static checks typecheck job. * fix(sqlite): order WITHOUT ROWID rows by the index's own key sequence `readRowOrder` filed `pragma_index_xinfo` key terms into a map keyed by column name, so a valid composite key that repeats a column — `PRIMARY KEY(a COLLATE BINARY, a COLLATE NOCASE DESC)` — collapsed to one term. The generated clause then overstated uniqueness: it omitted the BINARY term, so `A`/`a` ties survived, and the `row_offset` paging advertised a `primary_key` order that the actual key does not establish. Build the terms from the index rows in `seqno` order instead, keeping repetitions, and bail to `unspecified` if any declared key column never appears in the index. The declared collation stays explicit so a caller column's own collation cannot re-sort the window. Verified against a real SQLite database that the repeated key is accepted and that the old clause orders A, a, B while the index orders A, B, a. * fix(sqlite): accept a null next offset on unordered truncated rows readRows already withholds next_row_offset when a table shadows rowid, _rowid_ and oid and declares no primary key, because every non-zero offset is refused there. The reply schema still required the offset whenever truncated was set, so the provider rejected its own worker result and the first sample never reached CLI or MCP. Require the offset only when the order is stated. * fix(sqlite): probe row keys only for ordinary tables and name row_offset readRowKeyFacts ran for a selected virtual shadow table before readRows could reject it, so an unavailable module surfaced as SQLite's "no such module" instead of the selection refusal. Probe the key only when the selected kind is an ordinary table and leave the order unspecified otherwise, since no rows are read for that kind. The unordered-offset refusal also reported the generic selection reason, which callers see against table. Give it its own reason so the failure is attributed to row_offset. * feat(sqlite)!: page selected rows with a stated order and row offset Selected-row inspection currently reports only rows.order: "unspecified" and offers no way to reach rows past the first sample deterministically. Main also refuses non-zero row_offset today, so nothing relies on that omission; this change makes the stated order and the offset the contract, reports the next offset the caller feeds back, and keeps the existing byte, heap, statement and deadline limits. BREAKING CHANGE: rows.order becomes rowid | primary_key | unspecified, and rows gains required row_offset and next_row_offset members. A client that validates selected-table output against the previous schema (where order was the only permitted value) rejects a now-valid reply. Migration: accept the two new members and treat order: "rowid" | "primary_key" as the stated order the offset applies to; a null next_row_offset means the window reached the end. * docs(sqlite): say a null next offset also means no stated order The selected-rows limitation described next_row_offset as null only when the window reached the end. An unordered table whose row_limit truncates the sample also publishes null, because an offset there would name a request the row-offset rule refuses, so the Evidence read as if the sample were complete. State the second cause and pin it in the existing regression. * test(sqlite): drop the limitation wording assertion and state the unordered exception The boundary test pinned the sentence a caller reads in `limitations`, so a harmless rewrite of that text failed a paging test whose real subject is `order`, `truncated` and `next_row_offset`. docs/development/testing.md asks for facts over frozen explanatory prose, and the three structured expectations already fail if the paging fact changes. The tool contract promised that retained rows always have a unique order and can be paged. A rowid table that shadows rowid, _rowid_ and oid and declares no primary key has no such order: it returns order=unspecified with a null next offset and refuses a non-zero row_offset. Say so in the description so callers do not plan a continuation the tool always refuses. * docs(sqlite): say unordered tables cannot be paged The description told callers that a truncated table without a stated order needs "the table read without offsets", but omitting row_offset starts at zero again: with an unchanged row_limit the next call repeats the first sample instead of continuing past it. Say that these tables cannot be paged and point at row_limit for a larger first sample. * style(cli): wrap the native data type path description The `inspect-native-data-type` argument description line is 82 columns, over the repository's 80-column `.oxfmtrc.json` limit, so `npm run format:check` fails on main and on every branch merged from it. It came in with #9465 and the following commits to this file did not touch it. Reformat the one call the formatter objects to; no behaviour change. * fix(sqlite): bound rowid alias comparison by name byte length readRowKeyFacts decoded every column name before testing it against three short aliases, so a table declaring a multi-megabyte name could exhaust the worker heap even when the caller projected the column away. Reject names that cannot be an alias by their byte length, accounting for the database encoding, before building any string. --------- Co-authored-by: 2SinceJuly <202121119+2SinceJuly@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
morluto
|
694edef951 |
fix(decompile): retain published output receipts (#9633)
* fix(decompile): retain published index on failure * fix(decompile): preserve publication failure facts * docs(decompile): explain retained publication receipts * fix(decompile): retain receipts across provider cleanup chains * fix(evidence): retain diagnostics through cleanup primary chains * refactor(evidence): retain receipts once on the returned error |
||
|
morluto
|
d4b9a33348 |
fix(native): disclose failed accessibility reads (#9632)
* fix(native): disclose unavailable accessibility actions * fix(native): retain accessibility attribute failures * fix(native): preserve unknown accessibility selection |
||
|
Matt Van HornandMatt Van Horn
|
645735fc0f |
feat(windows): inspect Shell Link (.lnk) shortcut files (#7859)
* feat(windows): inspect Shell Link (.lnk) shortcut files Add inspect_windows_shortcut / inspect-windows-shortcut to read one local Shell Link without resolving or executing the target. The report keeps header flags and FILETIMEs, IDList items, LinkInfo volume and network names, StringData, ExtraData including Tracker droid facts labelled derived, and derived observations such as argument length and a minimized ShowCommand. * fix(windows): tighten Shell Link string reads and coverage - Decode counted StringData over its full CountCharacters payload. - Locate LinkInfo string terminators before charging max_string_bytes and before decoding; an unterminated string is a coverage issue. - Mark an IDList without a TerminalID and an out-of-range LinkInfoHeaderSize as partial coverage. - Report arguments_present and argument_length as null when HasArguments is set but the field could not be read. - Derive target_extension from the joined base path and suffix. - Keep ISO dates for FILETIME and UUIDv1 timestamps before 1970. - Retain the completed file snapshot on parse failures. - Reject an empty CLI path before normalization. - Select the shortcut contract from its family in focused tests. - Fixture StringData no longer writes a NUL into the counted payload. * style: format docs index and Shell Link boundary type Apply oxfmt to the two files format:check reports so Static checks passes. * fix(windows): reject network share paths for Shell Link reads inspect_windows_shortcut is advertised as offline, but the Windows path schema admits UNC paths, and lstat on one can contact an SMB server and offer the caller's credentials. Reject UNC, extended UNC, and other host-prefixed device paths in the provider before any filesystem call. Extended drive paths such as \\?\C:\x stay allowed. A mapped network drive cannot be told from a local drive by its path, and the docs say so. * fix(windows): decode Unicode Shell Link file-entry names File-entry items with bit 0x04 of the type byte, such as 0x36, store a UTF-16LE name, but the parser always read it as Latin-1. The read stopped at the first zero byte, so report.exe became r and an IDList-only shortcut reported the wrong target_extension. Read those names as UTF-16LE. * fix(windows): read volume labels by exact Shell Link discriminator VolumeLabelOffset 0x14 selects VolumeLabelOffsetUnicode and is not itself a label offset, and that Unicode field exists for no other value. The parser treated any offset of 0x14 or more as having both, so a Unicode-only label such as "OS" also produced a false ANSI label "O", and an ANSI label stored past 0x14 had its own bytes read as a Unicode offset. Branch on the exact value in parseVolume, the one place that interprets it, and report a VolumeID too short to hold the selected field. The fixture wrote both encodings for a Unicode volume, which hid the false label, so it now writes one. * fix(windows): limit network share rejection to Windows hosts POSIX hosts read repeated leading slashes as one local root, so an accepted selection such as ///tmp/shortcut.lnk is a local file there. Apply the host-prefixed path check only on Windows, and take the platform through the provider constructor so the Windows path forms stay covered on every test host. * fix(windows): compare shortcut icons only against full paths A stored IDList filename or relative path cannot show that an icon at a full path is elsewhere, so the comparison reported a false difference. Report null unless the icon and a recorded target are both full paths, and assert the missing TerminalID issue by code and offset instead of its message. * fix(windows): keep unread ExtraData blocks unknown A walk that stopped before or inside ExtraData still reported the blocks that HasExpString and HasExpIcon expect as absent, though they were never checked for. Report a block as absent only after the TerminalBlock shows the section was read in full. Without one the block stays unknown and coverage stays partial. * fix(native): ignore LNK network provider type unless ValidNetType is set CommonNetworkRelativeLink NetworkProviderType is only meaningful when the ValidNetType flag (0x2) is set. Return null otherwise. * fix(native): accept any ExtraData TerminalSize below 4 as the terminal MS-SHLLINK requires TerminalSize to be less than 4, not exactly 0. Treat 1, 2 and 3 as the terminal block instead of a malformed block. * fix(native): derive shortcut target facts from the stored environment target Include the EnvironmentVariableDataBlock literal target among the target candidates for target_extension and the icon comparison. Variables are not expanded and the target is not opened. * fix(windows-shortcut): keep unexpanded variable paths unresolved in icon comparison A stored environment target such as C:\Users\%USERNAME%\app.exe could be compared as different from C:\Users\Alice\app.exe. Variable-bearing paths are never expanded, so the icon comparison now returns null unless a fixed target matches the icon exactly. * test: drop redundant ok assertion before typed error guard The testing guide asks for the typed error to be thrown before asserting a successful Result value, and says a preceding .ok assertion adds no coverage. * fix(windows): point out-of-range LinkInfo offsets at their field A LinkInfo, VolumeID or CommonNetworkRelativeLink offset that points past its structure was reported at base + offset. With an offset such as 0xffffffff that position exceeds uint32, so the report failed its schema and the service discarded the whole partial result. Report such issues at the offending offset field instead, which keeps every issue offset and coverage.failing_offset inside the file. Cover each LinkInfo offset field in the parser tests and the LocalBasePathOffset case through the stable filesystem service. * fix(windows): require RFC 4122 variant before deriving Tracker UUID facts A version nibble of 1 alone does not identify a version-1 UUID. Also require the clock_seq_hi_and_reserved top bits to be 10, so other GUID layouts keep their raw value and derive no MAC or timestamp. * fix(windows): ignore DeviceNameOffset fields unless ValidDevice is set MS-SHLLINK 2.3.2 says DeviceNameOffset and DeviceNameOffsetUnicode are ignored when the ValidDevice flag is clear. parseNetwork read both regardless, so garbage in those fields became device-name facts or malformed_structure issues. Read the flags once and only read the device name fields when ValidDevice is set. * fix(windows): keep unsupported Tracker headers as raw data Check TrackerDataBlock Length (0x58) and Version (0) before decoding. A block with any other header is retained as raw bytes with a malformed_structure issue, so it no longer yields MAC and timestamp facts under complete coverage. * fix(windows): compare shortcut icon paths by normalized location Collapse "." and ".." segments and repeated separators in the icon comparison as text only, never opening a path. ".." stops at the drive or UNC share root, and \\?\ and \\.\ paths stay verbatim. Join LocalBasePath with CommonPathSuffix as the target and use the bare base only when the suffix is empty, as the network share branch does. Read the selected shortcut with lstat and open only, so a rejected symlink's destination is never resolved. * fix(windows): keep the UNC share in the root after repeated separators normalizePath took the host and the next split part as the UNC root, so \\server\\share\..\app.exe kept an empty share and let .. remove the real share. Drop empty parts before picking the root so paths on different shares stay different. * fix(cli): parse windows shortcut limits as decimals The four inspect-windows-shortcut limit options now go through cliDecimalField like the other inspection commands, so values such as 0x10 are rejected and cliDecimalArguments.test.ts passes. * fix(native): reject linked parents of windows shortcuts A parent directory symlink or junction pointing at a UNC share was followed by the first lstat, since the network check reads only the path text and the symlink rejection covers only the final component. On Windows, lstat each ancestor from the drive root down and reject the path at the first link, before anything follows it. * docs(native): move Windows shortcut guide and add its feature spec Move the Shell Link guide to docs/guides/native/windows-shortcut.md and list it under Native binaries in the site navigation, keeping the /windows-shortcut public route. Update README, translated README, CLI page and authored skill links to the new physical path. Add docs/specs/native/windows-shortcut/ with spec.md, plan.md and tasks.md, as required by docs/development/documentation.md. * fix(native): admit the windows shortcut parent walk On Windows the ancestor link walk ran before the provider's resource admission, so close() could finish while the walk kept calling lstat. Run the walk inside the same admission as the read, and check the caller's signal before each ancestor lstat. * fix(native): walk the effective parents of windows shortcuts The ancestor link walk checked every prefix of the path as written, so C:\alias\..\real\a.lnk was refused when C:\alias is a directory link, although Windows removes alias\.. before opening the file. Walk the ancestors left after removing . and .. components, keeping the caller's spelling, never climbing above an absolute root, and leaving \\?\ paths as written since Windows does not normalize them. * test(native): assert the linked parent as a structured fact The parent link regression matched the phrase "junction, which" in the refusal message, so a wording edit would fail it. Report the linked ancestor as observed.linked_ancestor on the input issue and assert that fact instead of the explanatory text. * test(native): bound the shortcut close gate and release it in finally Race the parent walk lstat acknowledgement against inspection settling with awaitAcknowledgement, then release the gate and join the inspect and close work in finally so a failure reports its cause instead of waiting for the Vitest timeout. * fix(native): walk Windows shortcut parents lazily windowsAncestors built and kept every parent prefix before the first lstat, so a path with thousands of components allocated memory quadratic in its length. It now keeps only the normalized components and yields each parent from a running prefix as the walk reaches it. --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> |
||
|
吴佳浩Albenandmorluto
|
25885b05c0 |
feat(v8): inspect V8 code cache containers and retained byte patterns (#8993)
* feat(v8): inspect V8 code cache containers and their retained byte patterns Bytenode-compiled `.jsc` files — the V8 code cache container that `vm.Script`, `vm.compileFunction` and bytenode 2.x write, and that Node single-executable applications and Electron apps ship — had no format-native reader. `inspect_binary_layout` shows the surrounding ELF/PE and `inspect_artifact` inventories the file, but neither decodes the V8 structures, so an agent could not tell which V8 line produced the file, could not learn the container is metadata rather than source, and had nothing to point at a version-pinned decompiler. Issue #8543. `inspect_v8_code_cache` joins the format-native family with `inspect_python_pyc`, `inspect_erlang_beam` and `inspect_kernel_module`: one explicit local file, bounded parsing, no execution, no installation, no child process. **The container is version-dependent, so the reader reports roles with confidence instead of assuming an ABI.** All eight header words are reported with byte offset, observed value, the role this reader believes the word has, and whether that role is `confirmed`, `inferred`, or `unconfirmed`. Only three facts gate admission: the header size this reader pins, the V8 code cache magic high half `0xc0de` (the low half is `ExternalReferenceTable::kSize` and varies per build, so it is a build fingerprint rather than a constant), and a declared payload length that reaches exactly the end of the file. I derived the field semantics from the producer rather than assuming them, and two findings changed the design: - **The word V8 calls the source hash holds the source length on the measured producers.** A 54-code-unit source stored `54`, and sources with the same code-unit length but different content stored the same value, including a source whose UTF-8 length was 154. The reader reports it as `source_length` and the docs say that this name is what the measurement supports, not a claim about what another build stores there. - **`vm.compileFunction` sets a high bit in that word.** The reader masks it out and reports the producer family as `script` or `compile_function`, so the source length stays correct for both producers. Embedded strings are reported as **byte-pattern observations with absolute byte ranges**, labelled as such in the field name and in the limitations. A reported sequence is not proof V8 retained it as a string, and a missing literal is not proof it is absent. **Fail-closed boundaries, all covered by tests:** an unknown magic is a typed `unsupported_target` failure because the file is not a V8 code cache; a short header or a payload-length disagreement is a typed `invalid_request` failure. Neither yields a partial parse and neither records Evidence. Non-goals are explicit in the tool description and the docs: the payload is serialized V8 heap data, not source; recovering logic needs a version-pinned View8-style decompiler that this tool neither provides nor depends on. Fixtures are generated from the running V8 instead of committing a version-pinned binary, so the tests exercise the real producer and stay valid across Node upgrades. - Semantic owner and earliest changed stage: format-native V8 code cache container decoding. - CLI and MCP/tool-catalog contract: one new `artifact-provider` tool and one new `inspect-v8-code-cache` CLI command, registered in `TOOL_CONTRACTS`, the `artifact` language family, `TOOL_EFFECTS`, the capability declarations and services, and the capability inventory. `docs:check` passes, so the generated catalog agrees. - Provider, bridge, target-format, or platform compatibility: no provider runs and no external tool is required, so the tool is available on every supported Node host. - Evidence, artifact, provenance, or reconstruction contract: complete Evidence with the payload byte range as its location; no artifact format, reader, or reconstruction change. - Generated metadata, package, or installation impact: catalog and document facts regenerated by the existing checks; no new dependency. Expected effects: - [x] Target content is returned as bounded structured facts. - `src/v8/V8CodeCacheParser.test.ts` (13 cases): real-producer header facts, per-field role and confidence, source length for both producers, an observation's absolute offset verified against the file bytes, the limitation text, foreign magic, payload-length mismatch, short header, missing payload bytes, a different build constant still admitted, and the observation budget. - `tests/boundary/mcp/v8CodeCacheMcp.test.ts` (4 cases): advertisement and annotations, header facts over MCP, relative-path rejection, unsupported magic, malformed container. - `tests/acceptance/v8CodeCacheCli.test.ts` (4 cases): the public CLI, relative-path resolution, byte-range observations, and the typed failure. - `npx vitest run src/contracts src/v8 src/application/v8CodeCache src/domain/v8` — 242 passed - `npx vitest run tests/boundary/mcp/v8CodeCacheMcp.test.ts tests/acceptance/v8CodeCacheCli.test.ts tests/boundary/filesystem/productCatalog.test.ts tests/acceptance/tools/localToolContracts.test.ts` — 20 passed, 4 skipped - `npx tsc --noEmit -p .` — exit 0 - `npm run build` — exit 0 - `npm run lint` — 0 errors - `npm run knip` — exit 0 - `npm run docs:check` — exit 0 - `npx oxfmt --check src tests docs` — exit 0 Gap: the header table is measured on node v24.22.1 / V8 13.6.233.17-node.58 only. No other V8 build is available here, so a different build's field order is untested; that is exactly why only three facts gate admission and the remaining roles carry a confidence level rather than being asserted. Additive. No existing tool, schema, or behaviour changes. The reader never executes the selected file, never launches a process, and never accesses the network; its audited effects are `mutatesSession: true` only, matching the other static format readers. Human owner: @XiaoBinGan - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue: #8543. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. * chore: retrigger title and readiness checks after the title was corrected * fix(v8): bound the payload scan and use the repository path conventions Review feedback from @greptile-apps on four points. **Bound the synchronous payload scan.** The scan is one synchronous pass over the payload, so the 512 MiB ceiling allowed a single request to hold the process for several seconds. The ceiling is now 64 MiB, matching the pinned CPython pyc reader, and the default is 32 MiB. The scan stays synchronous, like the BEAM and ELF note readers in this family, but the budget that bounds it is no longer large enough to matter. The new `max_file_bytes` ceiling and default are documented. **Use the absolute-path schema.** `path` now uses `absoluteLocalPathStringSchema` instead of `localPathStringSchema`, matching `inspect_erlang_beam`. A relative name previously passed validation and was resolved against the server's working directory, so `module.jsc` could select a different file than the caller meant. The rejection is now at the schema boundary. **State the extraction step.** The handoff in `docs/v8-code-cache.md` now says a cache embedded in a single-executable application or an Electron app must be extracted first, because this reader requires the magic at byte zero and returns `unsupported_target` for the surrounding ELF/PE executable. **Load only the owning contract group in tests.** Both tests now import `V8_CODE_CACHE_TOOL_CONTRACTS` directly instead of resolving the tool through the full catalog, so a focused test no longer constructs every other tool's schema. 21 cases pass across the parser, MCP boundary and CLI acceptance files; `tsc` 0; `build` 0; `knip` 0; `docs:check` 0; `oxfmt --check` 0. * fix(v8): select the tool contract by name in the focused tests Reverts the catalog-position selection I added for a review suggestion. This repository has a lint rule (`rea(no-positional-tool-contract)`) that forbids selecting a tool contract by catalog order, and `toolContract("inspect_v8_code_cache")` is how every neighbouring focused test selects one. The positional form I tried introduced two lint errors, so the named form stays. The review concern was that resolving through the full catalog constructs every other tool's schema. That cost is real but it is the repository's chosen trade: catalog ordering must never select an operation or schema, and `toolContract` is the named accessor that enforces it. * test(v8): cover the new tool in the absolute-path boundary test Main added `tests/boundary/mcp/absoluteLocalPathTools.test.ts` after this branch was opened. It enumerates every tool whose MCP `tools/list` schema must reject a relative path, and `inspect_v8_code_cache` uses `absoluteLocalPathStringSchema`, so the tool belongs in that list. Adding it keeps the enumerated field set complete instead of leaving one absolute-path tool untested. `tests/boundary/mcp` + v8 + contracts: 256 passing. `tsc` 0; `build` 0; `knip` 0; `docs:check` 0; `npm run lint` 0 errors. * perf(v8): read the payload by index instead of allocating per byte `for (const [index, byte] of payload.entries())` allocates a two-element array for every byte of the cache payload. On a 1 MiB payload that allocation dominated the scan. Measured with a real cache built from this parser, alternated so the two forms run against the same input: | payload | entries() | indexed reads | | --- | --- | --- | | 1 MiB | 15.5ms | 4.4ms | That is 3.49x on the scan, and the gap widens on smaller payloads where the allocation cost is a larger share (7.3x at 64 KiB in a synthetic run of the same shape). The scan is the whole cost of `inspect_v8_code_cache` once the file is read, so this is the tool's own hot loop. Output is unchanged: the parser returns one retained string for the 64 KiB fixture, and `src/v8` plus the CLI and MCP boundary files stay at 21 passing. `tsc` 0; `build` 0; `npm run lint` 0 errors. * fix(v8): admit only decimal tokens on the numeric CLI options Main added `tests/boundary/cli/cliDecimalArguments.test.ts` after this branch was opened. It enumerates every numeric CLI option and requires `0x10` and `1e3` to be rejected, because plain `Number` coercion would read them as 16 and 1000. This command's three options forwarded the domain fields unchanged, so the test failed on them: inspect-v8-code-cache options.maxFileBytes: Parser.parse accepts 1e3 inspect-v8-code-cache options.maxStrings: Parser.parse accepts 0x10 inspect-v8-code-cache options.maxStrings: Parser.parse accepts 1e3 inspect-v8-code-cache options.minStringBytes: Parser.parse accepts 0x10 `cliDecimalField` publishes the input view that parses the decimal token first and then lets the domain field own bounds, defaults and help, which is what the other numeric commands use. `tests/boundary/cli/cliDecimalArguments.test.ts` now passes. `tsc` 0; `build` 0; `npm run lint` 0 errors; `knip` 0; `docs:check` 0. * fix(v8): recognize module caches, and advise from remaining headroom Two review findings, each measured on Node 24.22.1 before changing anything. **A module cache was reported as a script.** `vm.SourceTextModule.createCachedData()` sets bit 30 of the source-length word, which the source-length mask `0x1fffffff` leaves out, and the producer check only tested the function bit at bit 29: | producer | source word | | --- | --- | | `vm.Script` | `0x00000013` for a 19-unit source | | `vm.compileFunction` | `0x20000009` for a 9-unit source | | `vm.SourceTextModule` | `0x40000013` for a 19-unit source | So a real module cache was labelled `script`, and `vm.Script` rejects that container. The producer enum gains `module` and the module bit is tested first; the source length is unchanged because both flags sit outside the mask. **The budget advice recommended a retry that could not work.** Both limits have ceilings — `max_strings` at 65536 and `min_string_bytes` at 256 — so an admitted file with more than 65536 runs of 256 bytes reached the branch while every suggested retry was a rejected input. `observationBudgetRemediation` now reads the remaining headroom and only recommends a change that is still admissible, falling back to selecting a smaller artifact when neither can move. The parser message states the fact and leaves recovery to the provider, which is the only place that can see both budgets. `MAX_INSPECT_V8_CODE_CACHE_MIN_STRING_BYTES` names the `min_string_bytes` ceiling that the schema previously spelled inline, so the advice and the schema cannot drift. `docs/v8-code-cache.md` describes both flag bits. `src/v8` plus the CLI and MCP boundary files: 28 passing; 7 new cases fail on the base commit. `tsc` 0; `build` 0; `npm run lint` 0 errors; `knip` 0; `docs:check` 0. * refactor(v8): decide the budget recovery as data, and test the decision The remediation tests matched fragments of the advice sentence and banned old phrases. `docs/testing.md` asks for error codes, recorded limits and recovery behaviour instead of frozen prose, so a harmless reword could fail those tests while wrong advice passed them. `observationBudgetRecovery` now returns the decision — which retries remain within the schema's admitted range, and whether a smaller artifact is the only option — and `observationBudgetRemediation` renders one sentence from it. The tests assert the decision, so the wording is free to change and an incorrect choice cannot pass. This also fixes a bug in the previous shape: it took the `min_string_bytes` branch to mean "lower it", but that branch is reachable with the value already at the schema minimum, where lowering is not a legal retry. The decision now carries `canLowerMinStringBytes` and a `requiresSmallerArtifact` flag for that case. 5 cases, all failing on the base commit, covering both headroom states, either ceiling alone, and the run-length floor. `src/v8` plus the CLI and MCP boundary files: 28 passing. `tsc` 0; `build` 0; `npm run lint` 0 errors; `knip` 0; `docs:check` 0. * ci: rerun after the wakaru release download recovered The only failing job on this branch was `javascript_recovery / real-javascript-recovery`, which failed on `curl: (22) The requested URL returned error: 504` while downloading `wakaru-linux-x64.tar.gz`. The release is reachable again (HTTP 200), so this empty commit re-triggers the lane. * docs(v8): place the guide in the tree and record the feature The guide sat at `docs/v8-code-cache.md`, outside the `docs/guides/` tree every other guide uses, and neither index linked it, so it could not be reached from `docs/README.md` or the VitePress sidebar. Moved to `docs/guides/javascript/`, linked from both indexes, and recorded the feature under `docs/specs/v8-code-cache-inspection/` with `spec.md`, `plan.md` and `tasks.md`. The spec keeps only the premises that were measured on Node 24.22.1 — the three producer words and the retained-string layout — rather than restating the implementation. `npm run docs:check` 0; `npm run format:check` 0; `tsc` 0; `build` 0; `lint` 0 errors; `knip` 0. V8 reader, CLI and MCP files: 28 passing. * docs(v8): use the documented spec path and test the advice against its struct Two review findings. **The records were at `docs/specs/v8-code-cache-inspection/`.** `AGENTS.md:73` asks for `docs/specs/<domain>/<feature-slug>/`, so they move to `docs/specs/v8/code-cache-inspection/`. **The verification section claimed more than the tests do.** It said the producer cases were built from real `vm.Script`, `vm.compileFunction` and `vm.SourceTextModule` output, and that the budget advice was compared with its owning formatter. In fact only `vm.Script` and `vm.compileFunction` are real producers — the module case sets the measured bit on a `vm.Script` cache — and only `observationBudgetRecovery` was asserted. Corrected the claim, and closed the second gap rather than only describing it: `observationBudgetRemediation` now has four cases holding the sentence to the struct it renders from. The sentence must not advise a knob the struct closed, and must mention a smaller artifact exactly when the struct requires one. Verified by making the sentence advise `max_strings` after the ceiling was reached — the test fails. Wording stays free; wrong advice does not. `npm run docs:check` 0; `npm run format:check` 0; `tsc` 0; `build` 0; `lint` 0 errors; `knip` 0. V8 reader, CLI, MCP, absolute-path and decimal-argument files: 34 passing. * fix(v8): restore the absolute-path contract entry after the rebase The rebase onto `main` conflicted in `tests/boundary/mcp/absoluteLocalPathTools.test.ts`: `main` gained `inspect_api_contract`, `inspect_devicetree`, `inspect_playwright_trace` and `inspect_python_pyc` in both the contract list and the path-field map while this branch added `inspect_v8_code_cache`. Both sides are additive, so both are kept. Without the resolution the branch carried conflict markers, which made `mergeable_state` `dirty` and every check on it meaningless. Re-verified after the rebase: `tsc` 0, `build` 0, `format:check` 0, `lint` 0 errors, `knip` 0, `docs:check` 0, and the V8 reader plus its CLI, MCP, absolute-path and decimal-argument boundary files at 34 passing. * test(v8): pin the producer from the runtime's own header word An independent reviewer ran these files on Node 22.23.4, 24.22.1 and 26.12.1. Only 22.23.4 failed, on `separates the compileFunction producer bit from the source length`, expecting `compile_function` and reading `script`. The cause is the fixture, not the parser: `vm.compileFunction(...).cachedData` carries `0x20000009` in the source-length word on Node 24.22.1, with bit 29 set, while Node 22.23.4 does not set that bit at all. Whether a given V8 build marks the word is a build detail, so asserting one build's spelling made the check version-dependent. The case now reads the word the runtime produced and asserts the parser maps it correctly, and keeps the part that must hold everywhere: the source length is the low bits, so it equals both `word & 0x1fffffff` and the source's length. The flag meanings themselves stay pinned by the synthetic `withHeaderWord` cases above, which do not depend on any build. Reverting the parser's `compile_function` arm fails this case and `keeps the function and module flags distinct`. `src/v8` plus the CLI, MCP and absolute-path boundary files: 24 and 34 passing. `tsc` 0; `build` 0; `npm run format:check` 0; `npm run lint` 0 errors; `knip` 0. * fix(v8): report an unflagged producer word as unknown, not as a script `vm.compileFunction` marks bit 29 of the source-length word on Node 24.22.1 (`0x20000004` for a 4-unit source) and does not mark it on Node 22.23.4 (`0x00000004` for the same call). Whether a V8 build marks a function cache is therefore a build detail, and an unflagged word is either a `vm.Script` cache or an unmarked function cache. The container carries nothing that separates the two. The reader reported `script` for that case, which is a confident claim the bytes do not support. My previous commit made the test accept whichever value the runtime produced, which recorded the ambiguity instead of resolving it in the contract. `unknown` is now a producer value, the unflagged word maps to it, and the observed word stays in `header.fields` for a caller that trusts a specific build fingerprint. A limitation line says the same in the report itself. `header.producer` widens to `["script", "compile_function", "module", "unknown"]`; `script` is no longer produced, because no unflagged word can be shown to be one. The spec's producer rule and its verification notes are updated to match. Reverting the mapping to `script` fails 2 of the 24 `src/v8` cases. `src/v8` with the CLI, MCP and absolute-path boundary files: 33 passing. `tsc` 0; `build` 0; `docs:check` 0; `npm run format:check` 0; `npm run lint` 0 errors. `knip` reports one unused export (`builtinDataEffectMethod`) on `main` as well, so it is not from this change. * fix(artifacts): retain completed identity across native parser failures * docs(v8): preserve unknown unflagged cache provenance --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
|
2c27c2a43b |
feat(electron): report V8 cachedData compilation boundaries (#9501)
* feat(electron): report V8 cachedData execution boundaries
analyze-javascript-application treated new vm.Script(source, {cachedData}) and vm.compileFunction(..., {cachedData}) as ordinary source. V8 deserializes previously compiled bytecode for those calls, so an Electron preload wrapper that delegates its whole contextBridge and IPC surface to a .cache/.jsc snapshot appeared to have been fully analyzed with zero exposed APIs.
Record each vm compilation with its inert cache path when the cachedData value traces to a readFileSync/readFile of path syntax, and project a cached-data execution unknown plus a loads edge from the owning source. Compilations without a cache stay recorded as no-cached-data so absence remains distinguishable from an unparsed snapshot. Count cachedData executions in the Electron summary and state the coverage limit in the graph limitations.
* style(cli): wrap the native data type path description
The `inspect-native-data-type` argument description line is 82 columns, over the repository's 80-column `.oxfmtrc.json` limit, so `npm run format:check` fails on main and on every branch merged from it. It came in with #9465 and the following commits to this file did not touch it. Reformat the one call the formatter objects to; no behaviour change.
* style(domain): reformat the pending dynamic-code accumulator field
`npm run format:check` fails on this branch, which is what `Static checks` reports: oxfmt wants the `pendingDynamicCode` union on one line. Applying it is the whole change; no behaviour is touched.
* fix(electron): preserve VM callable and cache source uncertainty
* docs(javascript): specify cached-data compilation boundaries
* fix(electron): exclude disabled caches from recovered boundaries
---------
Co-authored-by: 2SinceJuly <202121119+2SinceJuly@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
|
||
|
|
ab1b9e35d0 |
fix(entropy)!: own selected Windows path spelling at admission (#9393)
* fix(evidence)!: keep one Evidence identity across Windows path case On Windows, an existing caller path now takes its filesystem spelling when that spelling differs from the resolved path by letter case alone. One file spelled `C:\Temp\x.bin`, `c:\temp\x.bin` or `C:\TEMP\X.BIN` produces one `parameters.path`, `normalized_result` path and `evidence_id`, matching the existing separator and `.` folding. `caller_input_path` keeps the input. Links, junctions, `subst` drives, 8.3 short names and paths that do not exist yet keep the caller spelling, so no link is followed and requested output names keep their case. BREAKING CHANGE: Windows Evidence produced from a path spelled in a case other than its filesystem spelling now gets a different `evidence_id`. Fixes #9373 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(entropy): keep Windows casing lookup optional * ci(entropy): select Windows identity verification --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
|
0b1a945d69 |
fix(electron): model view constructors as window-like surfaces (#9571)
* fix(electron): model view constructors as window-like surfaces Electron's static pass recognized a window only through the literal `BrowserWindow` constructor, so `BaseWindow`, `BrowserView` and `WebContentsView` produced no node and their literal `webPreferences` were invisible. IPC registrations reached through such a view's own `webContents.ipc` were dropped too, because the receiver resolver had the same single-constructor gate -- VS Code alone registers seven channels this way, and a shipped sample of four applications put 29 percent of window construction sites on an unmodeled constructor. Read webPreferences through one constructor set shared by the construction inspector and the IPC receiver resolver. The finding only carries the new `window_constructor` field when the site is not `BrowserWindow`, so existing projections and their evidence stay byte-identical. A non-BrowserWindow owner is not admitted for `loadFile`/`loadURL` on itself: only BrowserWindow exposes those directly. * fix(electron): keep the view constructor label lookup nullable Observation labels are nullable in the application graph schema, so the byLabel lookup failed tsc with TS18047 and took the Static checks job down with it. --------- Co-authored-by: 2SinceJuly <202121119+2SinceJuly@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
|
908d9c0726 |
fix(javascript): keep export bindings out of module views (#9486)
* fix(javascript): keep export bindings out of module views Exported symbol bindings reuse the javascript-module node kind, so the modules collection counted one row per export alongside the owning module. On a CommonJS or ESM file with N exports, inspect_analysis_view reported N+1 module units, module pages listed indistinguishable rows sharing the module path, and item selection by selector.path failed with "Multiple modules have path". Treat a node whose observations carry semantic_role export-binding as an export of its owning module, not a module unit, and exclude it from module entries. Export bindings remain distinct graph entities for export-shape comparison and version matching. Validation: a domain regression in analysisView.test.ts and a CLI acceptance regression over a real CommonJS analysis both fail on main and pass with this change. Also ran the analysis view, MCP boundary, CLI boundary and JavaScript acceptance suites (125 tests), npm run check:fast, and the MCP catalog schema test after regenerating .cache/mcp-tool-catalog.json. * test(javascript): run the export-binding view case on the shared CLI The case starts a compiled CLI process for each of its three commands, paying startup and tool registration three times per run. The fixture already provides a file-scoped process for calls with no stdin or explicit timeout, so use it. * style(javascript): format the constructed-evidence type in the module-view case The acceptance case's inline graph type did not match what oxfmt emits for the 80-column limit, so `format:check` refused the tree. Reflow only; the assertions are unchanged. * style(cli): wrap the native data type path description The `inspect-native-data-type` argument description line is 82 columns, over the repository's 80-column `.oxfmtrc.json` limit, so `npm run format:check` fails on main and on every branch merged from it. It came in with #9465 and the following commits to this file did not touch it. Reformat the one call the formatter objects to; no behaviour change. --------- Co-authored-by: 2SinceJuly <202121119+2SinceJuly@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
|
512bcf5cdb |
fix(javascript): analyze first-party sources before dependencies (#9341)
* fix(javascript): analyze first-party sources before dependencies Every source in one application analysis charges the same parent result budget, and sources were offered in lexical inventory order. On an Electron app.asar, node_modules/ sorts before an electron-vite out-*/ directory, so dependency files spent the budget and every first-party main, preload, and renderer file had its semantic analysis not started and its application projection rejected. The summary then reported zero windows, preloads, and bridges while electron_semantics still said "recovered". Offer application JavaScript before JavaScript under node_modules; reported files keep inventory order. Count Electron boundary facts only from files whose projection was admitted, and report electron_semantics "partial" (with a limitation naming the count) when an Electron-loading source was not fully examined or reported. Fixes #9321 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(javascript): treat limited Electron examination as partial coverage A static pass can stop early at the Electron wrapper expansion budget or another resource limit without failing the worker. Such a source carries no analysis or projection failure, so its retained IPC or bridge facts still reported electron_semantics "recovered". Read the owned examination status and count a limited Electron-loading source as incomplete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
morluto
|
a874cef9af | fix(adb): report observed output after failed file pull (#9596) | ||
|
2SinceJulyand2SinceJuly
|
ca456efcbf |
fix(wasm): admit compatible 1.0.x WABT releases (#9223)
* fix(wasm): admit compatible 1.0.x WABT releases REA_WABT_BIN_DIRECTORY admission compared each --version banner to the literal 1.0.42, so a Homebrew or rolling-distribution WABT that advanced past that build was refused even though wasm-validate, wasm-objdump and wasm2wat accept the same flags and emit the same output shape. Callers had to keep a hand-extracted 1.0.42 tarball alongside the system package. Admission now accepts the release range ^1.0.42 and records the observed banner, matching how the Wakaru adapter already admits compatible builds. The verified release stays the reference: a banner other than 1.0.42 is retained as tool_profile.release and provider.version and the report states that only 1.0.42 was verified. All three tools must report the same release, since one directory is one installation. Managed downloads stay pinned. Fixes #9197 * fix(wasm): bound WABT admission to releases the report schema accepts WABT_ACCEPTED_RANGE was "^1.0.42", which also admits 1.1.0 and later 1.x releases, while wasmArtifactSchema only accepts 1.0.x banners. A configured tool reporting 1.1.0 therefore passed version admission, ran the inspection, and then failed at report parsing. Use the explicit ">=1.0.42 <1.1.0" range so admission and the schema agree, and test the boundary. Also carry the compatible-release warning into both partial-Evidence builders. Previously only successful reports stated that REA verified 1.0.42, so a validation rejection or later read failure under a compatible release silently dropped the uncertainty that a success reports. Share the text through wabtReleaseLimitation so both paths stay aligned. * fix(wasm): stop claiming the 1.0.42 commit for a newer admitted release A caller-supplied 1.0.43 or 1.0.44 was admitted with the observed banner in tool_profile.release, but source_commit still carried the 1.0.42 revision, which is a claim about a build that did not run. Emit null for any release other than the verified 1.0.42; the observed release and each executable digest still identify what actually ran. Also spell the accepted range as it is defined (>=1.0.42 <1.1.0) in the guide and the failure fixture instead of ^1.0.42, which in semver admits 1.1.0. --------- Co-authored-by: 2SinceJuly <202121119+2SinceJuly@users.noreply.github.com> |
||
|
morluto
|
5d5fe72218 | fix(plist): reserve Evidence framing in nesting allowance | ||
|
Ryan S
|
9d0e87f331 |
feat(browser): reuse admitted browser-socket transport and allow discovery_timeout_ms (#9203)
* feat(browser): reuse admitted browser-socket transport and allow discovery_timeout_ms * fix(browser): close unhealthy shared sockets and clamp max discovery timer * fix(browser): retry failed shared-socket cleanup while peer leases remain active * test(browser): race observation readiness with awaitAcknowledgement and join before mock restore * fix(browser): isolate shared-socket discovery aborts and coalesce cold connects * fix(browser): observe pending cold-connect cleanup failures and isolate fake CDP sessions * refactor(browser): carry typed connect cleanup owners and gate concurrent session test * test(browser): cover loopback socket reuse-key normalization * test(browser): verify distinct browser socket path IDs stay separate |
||
|
Ryan S
|
94182f331e |
feat(go): recover bounded Go string literals from (ptr, len) headers (#9206)
* feat(go): recover bounded Go string literals from (ptr, len) headers * fix(go): deduplicate header scan spans, bound rejected cache, and reject Cf * fix(go): allow U+200C ZWNJ and U+200D ZWJ in Go string literal recovery * fix(go): index Mach-O load regions, validate MCP schema, and add feature specs * test(go): cover Mach-O region-index cancellation and load-region boundaries |
||
|
morluto
|
3602b64c45 |
fix(providers): retain cleanup ownership across probes and queued work (#9468)
* fix(providers): retain cleanup ownership across probes and queued work * fix(jeb): retain primary discovery failures during cleanup * test(providers): bound cleanup and readiness barriers * test(jeb): scope availability cleanup fault injection * test(jeb): retain caller error through cleanup recovery |
||
|
morluto
|
9e78606e2b | fix(inspector)!: retain context lifetimes across protocol ID reuse (#9471) | ||
|
morluto
|
c5ed046d6e |
fix(ghidra)!: qualify memory effect coverage (#8651)
* fix(ghidra): qualify memory effect coverage * fix(native): retain value trace source limitations * fix(native): distinguish value trace policy commitments * fix(ghidra): bind value-flow scope to snapshot profile * test(ghidra): verify legacy value-flow snapshot recovery |
||
|
morlutoandCursor Agent
|
b81da90354 |
feat(native): page retained procedure references (#8588)
* feat(native): page retained feature traces * fix(snapshot): retain linked source evidence for replay * test(snapshot): verify linked foreign evidence stays scoped * feat(native): page retained procedure references * docs(mcp): clarify imported residual questions * docs(mcp): guide trace reference classification * fix(cli): report result sources when Evidence publication fails A trace keeps linked sources on the result. Use those records for the undelivered source closure as well as the bundle, so a refused output does not report them as not captured. * ci: rerun pull request checks The merge of origin/main did not start the CI workflow for this head. * fix(native): document shared procedure endpoints and the required selector Describe kind shared and items.endpoints instead of inline procedure copies. State that procedure-references requires a positional selector or --procedure. * fix(native): count procedure-reference endpoints before keeping a row Include the shared endpoint table in the page budget, with JSON punctuation, and store a new procedure only when its row is accepted. * fix(native): measure a procedure endpoint once before its comparison key Size the candidate before building one canonical identity, and reuse that key for the selected-procedure and shared-endpoint checks. * fix(native): reject an oversized procedure endpoint before canonicalizing it When the JSON size already exceeds every stored identity, skip the canonical string and keep the row out of the page. * fix(native): describe procedure-reference direction and coverage Avoid execution wording in the tool description, and give the CLI direction option the help text the catalog requires. * fix(native): compare endpoint size with UTF-8 identity length Keep one running maximum of stored identity bytes. A multibyte selected procedure is recognized when its JSON no longer fits the page. * test(cli): count procedure-references as a dedicated command The MCP tool has a CLI command, so it is no longer an MCP-only exception. * test(native): catch the string-length endpoint comparison Use a 200,000-character multibyte selected procedure and a shorter ASCII endpoint so the old JavaScript length check drops the selected row. * ci: rerun after the binary-layout toolchain timeout The required check failed because the toolchain install exceeded 15 minutes. The same job completed on the previous head, and this commit does not change the procedure-reference behavior. * test: stabilize capture decoder exhaustion coverage --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> |
||
|
morlutoandCursor Agent
|
2d25a09c0f |
fix(native)!: recognize identical function evidence (#8442)
* fix(native)!: recognize identical function evidence * test(native): verify same-evidence dimension citations * fix(compare): reject impossible same-evidence changes A comparison of one Evidence record cannot report a changed dimension, unequal subject hashes, or unequal digests on an unchanged dimension. * test(compare): admit the partial same-evidence fixture explicitly --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> |
||
|
|
a67525cdf1 |
docs: organize canonical documentation layout and verify new pages (#2791)
* docs: organize canonical documentation layout (URGENTE) Group all existing authored guides by domain and ownership, expose complete navigation, and preserve their public routes. Update physical path consumers and release configuration while retaining historical records. Document domain-based specs and ADR slugs without global numbering. Co-Authored-By: Codex <noreply@openai.com> * fix(docs): honor home layout in all-page route verification * style: format reviewed source changes * fix(docs): preserve routes and navigation cache inputs * fix: repair the static-check failures on the default branch (#2895) * fix: repair the static-check failures on the default branch The default branch fails its own required `Static checks` job three ways, so every pull request reports that job as failed regardless of its own change. - `npm run format:check` reports `src/domain/functionComparison.test.ts` and `src/ida/IdaOperationRunner.ts`. - `npm run lint` reports a duplicate `import { join } from "node:path"` in `tests/boundary/mcp/ghidraInventoryMcp.test.ts` (lines 4 and 17). - `npm run typecheck` reports `format: "devin"` missing from the client format union in `src/application/SupportedClients.ts`, and an unchecked index in `tests/boundary/cli/clientProfileLifecycle.test.ts`, whose pair array is inferred as `string[][]`. Apply the pinned formatter, drop the duplicate import, list the `"devin"` format that its own definition already uses, and mark the pair array `as const`. Validation: `npm run check:fast` and `npm run format:check` pass. * fix(setup): keep Devin on the JSON configuration dialect --------- Co-authored-by: 吴佳浩 <wujiahao@supremind.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> * fix(docs): update links after guide relocation * docs: correct verified documentation route counts * fix(docs): update browser capture guide link after move (#2820) Refresh gettext protected-link comments alongside the authored href. Co-authored-by: Codex <noreply@openai.com> * fix(docs): include http3 lab in stable routes * fix(docs): preserve historical release evidence and links * fix(ghidra): restore Windows verifier identity import * docs(wakaru): preserve pinned upstream policy link * docs(index): retain recently added guide links * docs(mcp): preserve parent retention clarification * fix(cli): preserve standalone import guidance * fix(test): preserve translated guide link drift coverage --------- Co-authored-by: Matheus Breguêz <matbrgz@gmail.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: morluto <17176810+morluto@users.noreply.github.com> Co-authored-by: 吴佳浩Alben <35291372+XiaoBinGan@users.noreply.github.com> Co-authored-by: 吴佳浩 <wujiahao@supremind.com> |