mirror of
https://github.com/morluto/rea.git
synced 2026-11-03 05:16:16 +00:00
8250bf2320ea57aab4e664db4b4d44b8abd2004e
129
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 morluto
|
7bcefead69 |
test: strengthen and prune audited test coverage (#8573)
* test: strengthen and prune audited test coverage * test: require delivered DeepSeek skill instructions * test: keep schema fixture types private * test: verify Gemini receives skill instructions * test: bind Qwen skill evidence to its tool result * test: report NativeAOT verifier checks precisely * fix: verify Ghidra cleanup paths reliably * fix: verify Command Code skill activation * test: tighten cleanup and fixture checks * test: verify packet capture evidence is recorded * fix: report HTTP3 capture provenance accurately * test: preserve resource coverage through MCP * test: prune repeated assertions and retain cancellation identity * fix: validate client verifier fixture paths * test: skip Copilot verifier cases on Windows * test: keep verifier setup target relative |
||
|
gensokyo-dev
|
a989c0e458 |
refactor(evidence): unify retained inputs across domains (#8299)
Declare logical Evidence sources once and resolve exact retained inputs through shared application workflows across 20 composition tools. Preserve domain-owned authentication, source closure, caller choices, coverage and resource admission. Validation includes public CLI/MCP regressions, local Codex terra/high trials and fresh-import real-artifact workflows. Measurements, failed attempts and coverage limits are recorded in docs/verification/evidence-input-composition-2026-10-31.md. Fixes #8293. Fixes #8281. Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning) |
||
|
Entrpiandmorluto
|
b43c329df7 |
feat(artifacts): inventory and extract UF2 and DfuSe firmware containers (#8230)
* feat(artifacts): inventory and extract UF2 and DfuSe regions open_binary refused UF2 and DfuSe firmware containers as an unsupported format, so inspect_artifact and extract_artifact could not show which bytes a drag-and-drop or DFU update writes at which addresses. UF2 is recognized by a complete first block with all three magic numbers. Valid main-flash payloads are joined into address-contiguous regions per familyID, named family-0x<id>/0x<start>-0x<end>.bin, and known IDs are named from a small table copied from Microsoft's uf2families.json at a pinned commit. Numbering is counted per familyID and block count: absent, repeated, out-of-order and out-of-range block numbers are reported, and absent blocks leave coverage unknown. Not-main-flash blocks are excluded and reported; file-container blocks are reported and leave coverage unknown; invalid blocks and a partial final block are excluded with their position. A rewrite of one address with different bytes fails closed, naming both block spans. DfuSe is recognized only by its prefix together with a DFU 1.1a suffix. Each element becomes one region at its file range with its target's alternate setting and name. Structural faults fail closed with a byte offset; overlapping elements stay separate and are reported. A suffix CRC mismatch follows integrity_policy: fail refuses with a dfu-crc32 integrity error, and record-and-continue records a root contradiction that extraction also carries. HEX, UF2 and DfuSe share one retained-region reader and one run merger, so the HEX decoder and reader lose their private copies. * docs(artifacts): document UF2 and DfuSe images and their esptool lane Describe recognition, region names, metadata, numbering, failures, the DfuSe suffix CRC under integrity_policy, and limits, and link the page from the CLI and firmware guides and the agent skill reference. npm run verify:uf2 runs esptool merge-bin --format uf2 on generated inputs at known addresses, with and without MD5, with 256-byte payloads, and on a concatenation of two chips' outputs. Inspection and extraction through the built CLI must return every input at its address under the familyID esptool itself reports. * fix(artifacts): accept both DfuSe image size conventions UM0391 calls DFUImageSize the file length, and dfu-util's dfuse-pack.py writes that value, while ST's DfuFileMgr, MicroPython's dfu.py and pydfu leave the 16-byte suffix out. The decoder accepted only the second form, so it refused dfuse-pack.py output as malformed. Accept either value and keep the declared one in image_size; any other value still fails with the suffix offset and the file length. The tests pin both producers' output for one image, CRC included, from their build() functions transcribed and run under Python 3. * fix(artifacts): keep reused UF2 block numbers from hiding missing blocks esptool numbers every merged input from 0, so two inputs with the same block count share one numbering group. A block lost from one input left its number present in the other, and the file reported complete coverage. Each block number now records the target addresses it was seen at: equal address counts are complete copies, a resend to the same address is one block, and unequal counts make coverage unknown with a limitation and ambiguous_numbering_group_count. sequence_count becomes numbering_group_count, and region block positions become first_file_block and last_file_block. Review follow-ups in the same change: - compare_artifacts compares a DfuSe element's alternate setting and target name, so bytes moved to another device memory are changed; - the shared reader keeps no region bytes outside its map after close; - one domain type replaces four identical decoder failure types; - the UF2 decoder returns a typed limit instead of throwing on bytes beyond its admitted size, and caps retained regions, not runs; - DfuSe caps elements with bytes only, names empty elements and element-less targets, drops the constant bVersion and bcdDFU fields, and says whether the CRC matches when DFUImageSize fits neither convention; - classification evidence carries a suffix only when the probe saw the end, and root classification reads the tail only for a DfuSe prefix; - the UF2 domain region owns family_name, and the root records the family registry revision; the table's MIT notice is preserved; - docs pin the producer revisions (dfu-util ee6b393d and f9a537c9, MicroPython cb99ca98), give the MD5 structure's real meaning, and describe container contradictions in the MCP contract guide; - tests and the esptool lane add the equal-count inputs and a block lost from one of them, accepted boundaries beside each refusal, a DfuSe root larger than the classification prefix, and CLI refusals. * feat(artifacts): name the UF2 families that LibreTiny builds for LibreTiny writes UF2 for Realtek, Beken and Bouffalo Lab modules. The Microsoft registry revision REA already cites (cf3bd7f9) lists eight of those families: RTL8710A, RTL8710B, RTL8720C, RTL8720D, BK7231U, BK7231N, BK7251 and BL602. Their regions now carry those names. LN882H, which that revision omits, keeps its familyID with no name. * fix(artifacts): count DfuSe declarations without bytes instead of listing them Empty DfuSe elements have no entry, so DFUSE_MAX_ELEMENTS does not cap them, and the target walk kept a description string for each one. At 8 bytes per element, a file under the 128 MiB ceiling can declare about 16.7 million of them. The walk now counts every declaration without bytes and builds only the first NAMED_ELEMENTS descriptions, which is all that the limitation shows. The limitation text and empty_element_count do not change. The guide now says that both DfuSe limitations name the first four. On this host, decoding 4,000,000 empty elements (a 32 MB file) ran out of memory under a 128 MB V8 heap before this change, and it peaked at 1.59 GB RSS with the default heap. It now decodes in 114 ms with a 220 MB peak. --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
Entrpi
|
e6d0618979 |
feat(artifacts): inventory and extract ESP flash layouts (#8228)
* feat(artifacts): inventory and extract ESP32 firmware images ESP32-family app and bootloader images (magic 0xE9) had no reader, so open_binary refused OTA downloads, app partitions and bootloaders. Add the esp-image container format behind inspect_artifact and extract_artifact. A pure decoder reads each byte once and keeps only fixed-size fields. Derived facts cover the chip, from a table cited to ESP-IDF v6.1; segment classification as ESP-IDF's should_map and should_load; checksum and appended SHA-256 results; app and bootloader descriptors; Secure Boot v2 signature blocks, observed and unverified; and undescribed trailing byte ranges. Truncation and undescribed bytes make coverage unknown. ESP8266 images are refused with a specific reason, and ciphertext stays unrecognized. Add the format to every target, Evidence, snapshot, live-summary and artifact vocabulary, document it, and add an optional verify:esp-images lane that compares esptool-built images with esptool image-info and espsecure signature-info-v2. Refs #3615 * test(artifacts): run the ESP image lane on esptool v5.6.0 The lane compared descriptor fields only when esptool image-info printed one, so a descriptor that REA reports and esptool does not print passed without comparison. Compare the descriptor kind in both directions. Record that the lane passed for all 13 images with esptool and espsecure v5.6.0, the version the layout facts cite, as well as with 4.12.0. * refactor(artifacts): define ESP image layout sizes once The decoder and the derived facts each defined the segment-header, appended-digest and flash-sector sizes, the erased-byte scan and the offset formatter. Define them once with the other fixed-layout structures and import them in both places. The signature slot count is the one constant that drives the slot scan. Correct the observedEnd comment: it is the end of the last chunk read. * fix(artifacts): report ESP-IDF's segment length limit by chip ESP-IDF's verify_segment_header rejects a segment of 16 MiB or more unless the chip maps flash through 32-bit cache addresses (SOC_SPI_MEM_SUPPORT_CACHE_32BIT_ADDR_MAP: ESP32-S3, P4, C5, C61, H4 and S31 in v6.1). Record that capability in the chip table and add a limitation for such segments; unknown chips get none. `loaded` repeats should_load. It does not mean the bootloader accepts the segment: verify_load_addresses still rejects one outside the chip's RAM, which REA does not model. Say so in the docs, the skill reference and the inventory limitation. * test(artifacts): check ESP lane tools first and verify extraction The lane started work before checking its tools, so a missing RISC-V target, ld.lld, esptool or espsecure surfaced as a raw spawn or command error. Check each one before building and name the variable to set. Warn when esptool is not v5.6.0, the release the layout facts cite. Assert each segment's class for the ESP32-C3 app, its variants and the bootloader, as the Xtensa image already did. Extract the app and the signed image through the CLI and compare every file with the bytes at esptool's segment offsets or in the appended signature sector; the source's SHA-256 must be unchanged and equal the reported root_sha256. * docs(artifacts): state what the ESP image lane does not establish The Verification section listed the lane's checks but not its gaps: only ESP32-C3 and ESP32 images come from a real producer, no real project build, OTA download or flash dump is inspected, no signature is verified, and the lane does not run in CI. Record the timing of the final lane run on esptool v5.6.0. * fix(artifacts): compare ESP segment declarations Segment entries hash only their data, so compare_artifacts reported a segment that moved to another load address with the same bytes as unchanged; only the image itself changed. Compare each segment's load_addr and data_len as its declaration, as Node SEA and PyInstaller members are compared, and count a changed declaration as a metadata change, also when content is unavailable. Header offsets are layout and the classification is derived, so neither is compared. * test(artifacts): run ESP lane commands as owned process trees The lane ran its tools through execFile. A timeout killed only the direct child, so a linker or assembler that Clang or GCC had started could keep running while the lane removed its workspace. Run every command through createVerifierCommands, check the tools inside withVerifierCleanup, and remove the workspace only after commands.close() has stopped every tree; a failed close keeps the workspace. Only a nonzero exit now counts as a refusal or an expected esptool failure; timeouts and spawn failures propagate. Supervising each command roughly doubles the lane's run time. * test(artifacts): cancel ESP decoding after its first decoded chunk The cancellation test requested cancel or close on setImmediate, which can land while the file is still being opened and checked, so it never showed that decoding had started. It also passed against a reader that ignored both requests while decoding. Acknowledge the decoder's first chunk, then cancel or close, join the operation, and assert that no later chunk reaches the decoder and that the reader ends closed. * docs(artifacts): record which ESP metadata comparison ignores Segments are compared by their declaration. The image's and the signature blocks' format_metadata are not: every field comes from their own bytes, which content comparison already covers, and a block's image_digest_matches follows the image's content. * feat(artifacts): inventory and extract ESP flash layouts A whole-flash dump of an ESP32-family chip was read as its bootloader image, with the partition table, apps and data left as undescribed bytes, or refused when it began with erased flash. Read it as an esp-flash-image container instead. The partition table is decoded as esp_partition_table_verify accepts it and is found at the caller's partition_table_offset, at the default 0x8000, or by scanning 0x1000-aligned sectors for a table whose MD5 verifies, which the inventory reports as derived. The bootloader is located at its chip's ROM bootloader offset. Each partition is an entry with its declaration in flat format_metadata; the bootloader and app partitions carry ESP image facts from the existing streaming decoder; the otadata records are reported with a selection derived without rollback or anti-rollback, and the running partition stays unknown. One unreadable partition does not hide the others, and extraction writes every partition, nvs_keys included, as ordinary data. Flash layouts are detected before single ESP images. The scan runs only when an ESP image sits at a ROM bootloader offset, and is refused when a bootloader at another chip's offset shows that the dump does not start at flash address 0. open_binary gains flash_offset and partition_table_offset, which require format esp-flash-image and are part of the snapshot target identity; the CLI takes them with --target-format esp-flash-image. Comparison treats a partition's label, type, subtype, range and flags as its declaration. * test(artifacts): verify ESP flash layouts with gen_esp32part, merge-bin and otatool Add the optional verify:esp-flash lane. It builds default and relocated partition tables with ESP-IDF's gen_esp32part.py, assembles dumps with esptool merge-bin, including one written from flash 0x1000, and writes OTA records with otatool through flash_file_tool.py, which replaces parttool's esptool calls with reads and writes of the dump file. It compares REA's bootloader and app facts with esptool image-info, every partition with gen_esp32part.py's decoding of the extracted table, and the OTA records and derived slot with otatool read_otadata after each switch. It also checks extracted bytes and source integrity, the scan and caller table offsets, flash_offset on the 0x1000 dump, a corrupted app beside valid partitions and a truncated dump. ESP32 linker scripts place the conformance bootloader and app in ESP32 address ranges. * fix(artifacts): read ESP flash layouts as the bootloader does Read the partition table as bootloader_utility_load_partition_table does: the last factory, otadata and same-subtype OTA declaration wins, and a limitation names repeats. Plan each otadata record within its partition and the dump. An otadata partition smaller than two sectors selects none, the bootloader's INVALID_INDEX; a record the dump lacks leaves the selection unknown while the other record stays reported. The fallbacks follow the bootloader's indexes: ota_0-fallback and factory-fallback name no partition when the table declares none, and the assumptions add GPIO holds and deep-sleep wake. Refuse a dump whose bootloader contradicts its origin before the default table offset is tried, whether flash_offset was omitted or given, and keep the ESP image reader's ESP8266 refusal. A table never lies inside the located bootloader image, so single app images are no longer scanned. An explicit esp-flash-image selection is decided before other container signatures, and the native commands' --target-format no longer takes it. Say when no bootloader confirms the default origin. Exempt the primary bootloader and partition_table entries of ESP-IDF 5.3 from placement limits, report every region inside a longer one, and name partitions by index when labels repeat ignoring case. Keep an undecodable label's bytes in label_hex and compare them. Word undescribed ranges as starting at a byte that is not erased, and treat a short erased region as erased. Add an MCP compare_artifacts test for partition declarations, reader identity and close tests, scan edge cases, the 0x2000 ROM offset and a literal otatool CRC. * test(artifacts): verify ESP primary layout entries and the otadata fallback Build a table with the primary bootloader and partition_table entries that gen_esp32part.py fills from --primary-bootloader-offset, and check that REA reads it with complete coverage, no overlap or misplacement limitation, and a table that agrees with gen_esp32part's decoding. Erase the otadata with otatool after the slot switches and check the records against read_otadata and the factory fallback. * fix(artifacts): keep ESP flash coverage complete for a dump cut short An esptool merge-bin dump ends after the last part it wrote, usually inside its last partition. Such a partition made the layout's coverage unknown, so artifact comparison of two merge-bin dumps reported every unchanged partition as unknown. The partition table names every partition whatever the file's length, so a partition cut short or left out by the end of the file hides no entry. As the ESP image reader does for a cut-short segment, its entry holds the bytes the file has and an entry limitation gives the declared size; a partition with no bytes stays in partitions_outside_dump. Coverage stays complete unless undescribed bytes remain or no bootloader is located. An MCP compare_artifacts test of two dumps that end inside ota_0 now classifies every partition, and fails on the old rule. * fix(artifacts): report written bytes in ESP partitions, erased slots included An app slot is erased for ESP-IDF's bootloader when its image header bytes are erased flash, and REA reported only that status. A slot can still hold data further in: a dump read from a device held a complete older app image 1 MiB into an OTA slot reported as erased, with no limitation. Each partition entry now records where the bytes the dump holds are written, relative to the partition: written_offset, the first byte that is not 0xFF, written_end, just past the last, and written_bytes, how many are not 0xFF. They are null for a partition outside the dump. The status stays erased, which is the bootloader's view, and a limitation names an erased app slot that holds written bytes and says they are not read as an image. The reader reads every partition byte once more to find them. * fix(artifacts): derive the ESP boot selection for every partition table The boot selection lived inside otadata, so a table without an otadata partition, the common factory-only layout, had none. ESP-IDF's bootloader_utility_get_selected_boot_partition returns the factory index when the table declares no otadata partition. Move it to a top-level boot_selection that every layout has. Its basis names the bootloader's branch: otadata, factory-fallback and ota_0-fallback from the records, no-otadata for a table without otadata, none for the bootloader's INVALID_INDEX, and unknown when the dump lacks a record the bootloader reads, which replaces the null selection. running_partition moves with it, and otadata keeps its partition index and records. The assumptions name what each branch leaves out: no records assumption without otadata, and none for none or unknown. The real-tool lane checks no-otadata on a table gen_esp32part built without otadata. * fix(artifacts): say when an ESP image is a flash dump read from 0x1000 An ESP32 dump read from flash 0x1000, as read-flash 0x1000 or merge-bin --target-offset 0x1000 writes it, opened automatically as its bootloader image with a megabyte of undescribed bytes and no next probe. The flash reader refuses that origin, but nothing told an agent which flash_offset to pass. espFlashDumpOrigin owns the inference: an image of a chip whose ROM loads the bootloader from flash R, at file offset F, is the bootloader of a dump read from R - F when a partition table sits where that origin puts 0x8000. The ESP image reader asks it for a non-app image at file offset 0, so the result does not depend on how the reader was created, and records flash_dump {flash_offset, table_file_offset}, null otherwise, with a limitation that says to reopen the file as esp-flash-image with that flash_offset. inspect_artifact proposes the same open_binary call as a next probe, and the flash reader's misplaced-origin refusal names the offset. * perf(artifacts): summarize nested ESP images in the flash layout summary view The summary view returned an ESP flash layout's root facts whole, so each app partition and the bootloader carried every segment and signature block, about half of a dump's summary. The summary view now gives each nested image a summary instead of its facts: status, chip, entry point, segment count, load ranges (each mapped, loaded or unclassified segment's address and declared length), the checksum and appended SHA-256 statuses, coverage and limitations. Complete detail keeps the full facts, every segment included, and the retained Evidence is unchanged. On a 16 MiB device dump the summary's root falls from 5,141 to 2,657 bytes, and the summary Evidence from 10,038 to 7,554. * test(artifacts): expect the misplaced image the ESP bootloader search reports |
||
|
gensokyo-devandhhang
|
0ffed7a48c |
feat(android): trace callers of external DEX methods (#8154)
Resolve exact external method targets from directly encoded DEX invokes. Preserve complete caller identity, bounded position evidence and follow-up selectors. Report aggregate position coverage independently of the caller inventory, and reuse bounded prototype facts within the exact session. Absorbs #8096. Fixes #7811. Co-authored-by: hhang <169027046+xu91102@users.noreply.github.com> |
||
|
gensokyo-dev
|
b3d31ed106 |
fix: preserve nullable source-map facts and streamline model tests (#8018)
* test: repair MCP validation assertions and reuse model sessions * test(ci): audit constrained guidance bundle fingerprints * fix(browser): distinguish empty and absent source-map declarations * refactor(browser): encode script metadata groups once * test: assert structured path errors without freezing prose |
||
|
|
860fdd8fbe |
feat(macho): report per-slice encryption status in inspect_macho (#5359)
* refactor(macho): extract non-throwing container and slice load-command readers Split readMachoImage into readMachoContainer, which classifies a file as thin, FAT or not Mach-O and keeps FAT range issues per slice, and readSliceLoadCommands, which reads one slice header and frames its load commands, returning every command framed before a failure. readMachoImage keeps its strict policy on top of them: commands framed before a framing defect still decode first, an unsupported FAT slice still yields to a later malformed one, and the first FAT range issue is still reported. A characterization test pins that precedence for inputs with several defects; it passes against both the previous and the new implementation. Part of #4306. * feat(macho): report per-slice encryption status in inspect_macho inspect_macho now reports the LC_ENCRYPTION_INFO(_64) state of every slice in a new `encryption` collection, read in process from load commands without reading code bytes. - Each slice is encrypted, not-encrypted, absent, malformed or unknown. Absent requires a completed load-command walk; malformed and unknown carry a reason and keep commands read before the failure. - The named range is end-exclusive in slice and input-file coordinates and maps to file-backed segment fragments, VM addresses and the overlapping nonempty sections. - A ciphertext range adds a limitation naming its input-file bytes. - The in-process read and the native tools are bracketed by one regular file descriptor and stat identity checks; a change reports artifact_changed. A missing or inaccessible target keeps the tool facts and reports encryption as non-exhaustive, never absent. - Retained Evidence recorded before this field is parsed by the owning schema and reports encryption as unknown in the macho view and in managed/native verification. Fixes #4306 * fix(macho): keep slice facts and failures under large tables and close errors - Frame load commands through a visitor instead of retaining every command, restoring the pre-refactor memory profile (2M eight-byte commands: peak heap 370 MiB -> 0.9 MiB). - Confirm a large FAT container from a valid slice's Mach-O header so a sibling's range fault stays per slice instead of hiding healthy slices. - Name only sections whose bytes the range and their segment both cover. - Throw inspection failures inside the owned descriptor callback so a close failure keeps them as its primary error. * fix(macho): map encrypted ranges only within segment VM extents mapRange gave every file-backed segment fragment VM addresses, even for a segment with no VM size, whose file bytes dyld never maps, and past vmsize when a segment declared more file bytes than memory. Read vmsize with the segment. A segment with no VM size keeps its fragment with null VM addresses. A file size beyond the VM size, or a VM range past the address space, makes the mapping unknown with a reason while the slice keeps its encryption status. The docs also claimed the encryption read never reads code bytes. Like every Mach-O container read, a 0xcafebabe input of at most 16 MiB is read whole to rule out a JVM class file; say so. * fix(macho): read encryption through the admitted descriptor facts Main's withArtifactRegularFile now hands callers the admitted handle and its initial stats. Use those facts instead of re-stating the handle, and pass the legacy Evidence payload straight to createEvidence. * fix(macho): retain a finished report when the target close fails The file owner keeps the completed callback value when the descriptor close fails or cancellation arrives during close. inspect_macho now validates that value and retains the report as the error's partial observation, as inspect_signature does, instead of returning only the cleanup or cancellation error. * test(native): supply readEncryption in merged dyld refusal cases main's new dyld refusal cases build NativeMachoInspectionContext without the encryption reader this branch requires. * test(views): bind the Mach-O encryption page to its facet main's facet-ownership case builds inspect_macho Evidence without the encryption coverage this branch requires; the fixture now carries one slice row and the loop checks the encryption page like the other facets. * fix(macho): bound encryption commands and range mapping across slices A slice retains at most two encryption load commands, since a second one already makes it malformed; the reason keeps the full count. Range mapping in every slice shares one budget of segment and section records, charged from each segment command's capacity before it is decoded. A slice past the budget keeps its encryption status and reports an unknown mapping. * style(macho): format encryption reader and binding test * fix(macho): stop retaining segments past the mapping budget Segment commands are collected during framing only while their mapping records still fit in the budget shared by all slices, and the budget is charged only when the slice maps its range. * docs(macho): clarify retained mapping allowance * fix(ci): own the Mach-O encryption test in Apple scope --------- Co-authored-by: t <t@t> Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
gensokyo-dev
|
e3d8d20236 |
fix(native): preserve dispatch metadata format refusals (#8013)
* fix(native): preserve dispatch metadata format refusals * test(native): normalize CLI format refusal envelopes * test(native): assert admitted metadata target paths |
||
|
gensokyo-dev
|
45cc935964 |
fix(native): accept walkable Mach-O command tables (#8002)
* fix(native): accept walkable Mach-O command tables * test(native): verify large command tables through macOS CLI and MCP |
||
|
morluto
|
640e19d79d |
fix(native)!: own Mach-O load commands in binary reader (#7952)
* fix(native): retain selected Mach-O slice identity * fix(native)!: own Mach-O load commands in binary reader * fix(native): preserve malformed Mach-O and image base limits * fix(native): preserve leading BOM in Mach-O names * test(native): update Mach-O provenance verification * fix(native): retain completed Mach-O inspection on cleanup failure * fix: bound Mach-O projected section output * fix(native): own retained Mach-O command bytes * fix(native): retain Mach-O outcomes on final cancellation * fix(cli): preserve caller path in partial evidence * fix: reserve Mach-O projection envelope budget |
||
|
morluto
|
af1ccb6506 | fix(android): preserve manifest component spelling (#7818) | ||
|
Entrpiandgensokyo-dev
|
64196b08ca |
feat(artifacts): inventory and extract ESP32 firmware images (#7498)
* feat(artifacts): inventory and extract ESP32 firmware images ESP32-family app and bootloader images (magic 0xE9) had no reader, so open_binary refused OTA downloads, app partitions and bootloaders. Add the esp-image container format behind inspect_artifact and extract_artifact. A pure decoder reads each byte once and keeps only fixed-size fields. Derived facts cover the chip, from a table cited to ESP-IDF v6.1; segment classification as ESP-IDF's should_map and should_load; checksum and appended SHA-256 results; app and bootloader descriptors; Secure Boot v2 signature blocks, observed and unverified; and undescribed trailing byte ranges. Truncation and undescribed bytes make coverage unknown. ESP8266 images are refused with a specific reason, and ciphertext stays unrecognized. Add the format to every target, Evidence, snapshot, live-summary and artifact vocabulary, document it, and add an optional verify:esp-images lane that compares esptool-built images with esptool image-info and espsecure signature-info-v2. Refs #3615 * test(artifacts): run the ESP image lane on esptool v5.6.0 The lane compared descriptor fields only when esptool image-info printed one, so a descriptor that REA reports and esptool does not print passed without comparison. Compare the descriptor kind in both directions. Record that the lane passed for all 13 images with esptool and espsecure v5.6.0, the version the layout facts cite, as well as with 4.12.0. * refactor(artifacts): define ESP image layout sizes once The decoder and the derived facts each defined the segment-header, appended-digest and flash-sector sizes, the erased-byte scan and the offset formatter. Define them once with the other fixed-layout structures and import them in both places. The signature slot count is the one constant that drives the slot scan. Correct the observedEnd comment: it is the end of the last chunk read. * fix(artifacts): report ESP-IDF's segment length limit by chip ESP-IDF's verify_segment_header rejects a segment of 16 MiB or more unless the chip maps flash through 32-bit cache addresses (SOC_SPI_MEM_SUPPORT_CACHE_32BIT_ADDR_MAP: ESP32-S3, P4, C5, C61, H4 and S31 in v6.1). Record that capability in the chip table and add a limitation for such segments; unknown chips get none. `loaded` repeats should_load. It does not mean the bootloader accepts the segment: verify_load_addresses still rejects one outside the chip's RAM, which REA does not model. Say so in the docs, the skill reference and the inventory limitation. * test(artifacts): check ESP lane tools first and verify extraction The lane started work before checking its tools, so a missing RISC-V target, ld.lld, esptool or espsecure surfaced as a raw spawn or command error. Check each one before building and name the variable to set. Warn when esptool is not v5.6.0, the release the layout facts cite. Assert each segment's class for the ESP32-C3 app, its variants and the bootloader, as the Xtensa image already did. Extract the app and the signed image through the CLI and compare every file with the bytes at esptool's segment offsets or in the appended signature sector; the source's SHA-256 must be unchanged and equal the reported root_sha256. * docs(artifacts): state what the ESP image lane does not establish The Verification section listed the lane's checks but not its gaps: only ESP32-C3 and ESP32 images come from a real producer, no real project build, OTA download or flash dump is inspected, no signature is verified, and the lane does not run in CI. Record the timing of the final lane run on esptool v5.6.0. * fix(artifacts): compare ESP segment declarations Segment entries hash only their data, so compare_artifacts reported a segment that moved to another load address with the same bytes as unchanged; only the image itself changed. Compare each segment's load_addr and data_len as its declaration, as Node SEA and PyInstaller members are compared, and count a changed declaration as a metadata change, also when content is unavailable. Header offsets are layout and the classification is derived, so neither is compared. * test(artifacts): run ESP lane commands as owned process trees The lane ran its tools through execFile. A timeout killed only the direct child, so a linker or assembler that Clang or GCC had started could keep running while the lane removed its workspace. Run every command through createVerifierCommands, check the tools inside withVerifierCleanup, and remove the workspace only after commands.close() has stopped every tree; a failed close keeps the workspace. Only a nonzero exit now counts as a refusal or an expected esptool failure; timeouts and spawn failures propagate. Supervising each command roughly doubles the lane's run time. * test(artifacts): cancel ESP decoding after its first decoded chunk The cancellation test requested cancel or close on setImmediate, which can land while the file is still being opened and checked, so it never showed that decoding had started. It also passed against a reader that ignored both requests while decoding. Acknowledge the decoder's first chunk, then cancel or close, join the operation, and assert that no later chunk reaches the decoder and that the reader ends closed. * docs(artifacts): record which ESP metadata comparison ignores Segments are compared by their declaration. The image's and the signature blocks' format_metadata are not: every field comes from their own bytes, which content comparison already covers, and a block's image_digest_matches follows the image's content. * docs(artifacts): state that ESP mmu_page_size is a base-2 logarithm The app descriptor's mmu_page_size keeps ESP-IDF's name and value, the MMU page size in log base 2 as esp_app_desc_t declares it, so 16 means 64 KiB. Say so in docs/esp-image.md and the schema comment; a reader could otherwise take 16 as 16 KiB. * fix(esp-image): retain member consumption and share lifecycle regressions * test(artifacts): scope MCP catalogs and budget full-schema validation * test(artifacts): verify compact output profiles and document bounded catalogs * test(artifacts): close readers when cleanup-test admission fails * test(artifacts): compare ESP image limitations with their owners The decoder and reader tests froze complete limitation sentences, so a harmless rewording failed them. The wording now has two owners: ESP_IMAGE_LIMITATIONS in espImageFacts.ts, which the decoder, the inventory and the reader use, and ESP_DESCRIPTOR_LIMITATIONS beside the descriptor decoder. Tests compare delivered limitations with the owner, called with the facts each case establishes (segment index, declared and available bytes, offsets, chip, page range), next to the structured assertions they already make. Delivered text is unchanged. * test(esp-image): assert structured facts and owned guidance * fix(esp-image): keep spi_flash_mode and descriptor fields before a truncated tail ESP-IDF v6.1's esp_app_desc_t declares spi_flash_mode at byte 181, after mmu_page_size, as the committed reference layout shows; the descriptor now keeps it as stored. The decoder also required all 256 descriptor bytes, though it reads none past byte 181, so a file ending in the reserved tail lost the version, project, digest and revision facts it does hold. The app descriptor now needs only its fields; the segment's truncation is reported as before. * test(esp-image): pin the P-192 public key digest to espsecure's layout espsecure packs ECDSA signature blocks as "<BBBx32sB64s64s1031x" and zero-pads a 48-byte P-192 point to 64 bytes, so the signature starts at byte 101 for every SHA-256 curve. signature_info_v2 hashes sig_block[36:101] and _digest_sbv2_public_key the curve ID with the padded point. No test built a P-192 block; this one checks public_key_sha256 against that definition. * fix(esp-image): keep each app descriptor field the file holds in full A file can end anywhere inside the 256-byte esp_app_desc_t. The decoder needed every field byte, so a cut at byte 176 dropped the version, project and app_elf_sha256 it does hold. Once the 4-byte magic identifies an app descriptor, each field is now kept when the file holds all of its bytes and is null otherwise; observed_length records where the descriptor ends, so a cut field is told apart from text that is not UTF-8. The fields after the magic become nullable in the metadata schema. * fix(esp-image): report a signature block whose image digest does not match A CRC-valid Secure Boot v2 block can carry another image's digest. The block recorded image_digest_matches false, but no limitation said so, unlike a checksum or appended-SHA mismatch, so CLI and MCP summaries showed only the generic unverified note. The mismatch is now a limitation; coverage stays complete and signature_status stays unverified. The three other signature-sector limitations move into ESP_IMAGE_LIMITATIONS with their text unchanged, so all ESP image wording has one owner. * refactor(esp-image): remove redundant descriptor byte guards --------- Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
gensokyo-dev
|
570abe95f2 |
feat(android): link native declarations to static JNI candidates (#7472)
Link original DEX native declarations to per-ABI function exports and relocated static JNI table candidates through shared CLI/MCP workflows. Preserve global attribution ambiguity, exact provenance, partial coverage and retained Evidence; default MCP delivery to bounded summaries with filtered/batch pages. Use one immutable ELF relocation decoder with streaming IL2CPP materialization, cooperative bounded export/version parsing and bounded JNI state. Reuse validated session inventories without loading target libraries or executing JNI code. Validation: relevant type/lint/build and parser/domain/public CLI/MCP regressions; genuine four-ABI NDK fixtures and independent library/slot oracles; actual JDK/JADX Conscrypt workflows with retained-source removal. Full measurements and limitations are recorded in docs/verification/android-jni-bindings-2026-10-27.md. Closes #6403. |
||
|
gensokyo-devand555aaditya
|
03f17cf41d |
feat(android): apply caller-selected R8 mapping evidence (#7456)
Apply caller-selected R8/ProGuard mappings to shared Android inspections, preserving raw identities, exact verification, ambiguity and provenance. Add retained summaries and bounded/cached parsing, with measured real R8/Conscrypt workflows and review fixes. Fixes #6329. Co-authored-by: 555aaditya <107977627+555aaditya@users.noreply.github.com> |
||
|
morluto
|
5aa23c3d17 | refactor(providers): move inspection scope to application | ||
|
gensokyo-devandHarsh
|
7e8f7f6101 |
feat(android): expose selected manifest component declarations (#7379)
Extend static manifest declarations and retain precise source locations and historical unknowns. Share bounded selected component views across CLI and MCP with complete parent Evidence. Verified real ApiDemos, an AAPT2-built fixture, legacy Evidence, default-package metadata and relevant CLI/MCP contracts. Recorded whole-task token estimates, timing/resource limits and independent main-failure controls in the PR. Co-authored-by: Harsh <johiyaharsh@gmail.com> |
||
|
morluto
|
662ec2e450 | fix(verifiers): preserve source Evidence through shared admission | ||
|
morluto
|
e7f986c994 | fix(json): consolidate dictionary admission across analysis and verification | ||
|
morluto
|
ccdeda7a27 |
fix(inputs): centralize admission and preserve caller selections
Move shared request parsing into its application owner, migrate all consumers, and remove six duplicated admission blocks while preserving cancellation and validation policy. Keep operator-selected JavaScript paths in verifier parent Evidence while retaining canonical comparison guards. Validated with 204 passing tests, 3 Linux-x64-only skips, typecheck, lint, and test discovery. |
||
|
morluto
|
554816becc | refactor(verification): centralize native client result discovery | ||
|
morluto
|
f4cc6c85ef | refactor(verification): share native client fixture assertions | ||
|
morluto
|
1fe1f1f8d6 | refactor(managed): retire redundant verification facade | ||
|
Abel YagubyanandAbelo9996
|
955c7c80ec |
fix(android): report DEX static values of static final fields (#6981)
* fix(android): report DEX static values of static final fields Fixes #6397 * fix(android): request DEX static values only for class inspection * fix(android): range-check wide DEX static values * fix(android): move the bridge to version 7 after main took 6 --------- Co-authored-by: Abelo9996 <Abelo9996@users.noreply.github.com> |
||
|
|
b4ba96a9a5 |
feat(firmware): inspect flattened devicetree blobs (#7009)
* feat(firmware): inspect flattened devicetree blobs Adds inspect-devicetree and inspect_devicetree for FDT v16/v17 blobs without target execution or external dependencies. Results include validated headers, reservations, a node/property tree with bounded raw previews, and derived standard values and selected phandle references. Parsing enforces file, tree, and decoded representation budgets and rejects malformed input without partial success. Summary delivery supports retained Evidence pagination, with tests covering parsing, service validation, CLI delivery, and MCP access after the input is removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016y53PdpyMAp9RSW65fatHe * fix(firmware): reject conflicting devicetree phandles and bound page summaries A node whose phandle and linux,phandle disagree no longer resolves under either value; the conflict is reported as a limitation. Finalization failures after a completed read retain the file-snapshot-hash partial observation, matching the Go provider. Devicetree view pages carry fixed-size context and counts instead of repeating summary strings, so a large bootargs no longer pushes every page over the MCP response budget. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016y53PdpyMAp9RSW65fatHe --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
morluto
|
de088fcec6 |
fix(verifiers): join model fixture request finalization
Keep each admitted HTTP handler owned through Evidence writes and error rendering, including after its client disconnects. Close joins graceful transport shutdown and handler work before publishing completion, and propagates exact late failures while preserving ordinary request diagnostics. Share lifecycle ownership across OpenAI, Anthropic, and Gemini fixtures while retaining protocol-specific parsing and responses. Exercise real disconnects, graceful concurrent and repeated close, ordinary 400s, and opaque late failures. |
||
|
morluto
|
43c7255dbd | fix(android): preserve binary class selectors across metadata calls | ||
|
morluto
|
72ba1cd92b |
fix(verifiers): centralize failure and cleanup ownership
Own operation results and exact rejections in withVerifierCleanup, including undefined, null and independent cleanup failures. Remove the optional-primary cleanup contract and migrate every caller, preserving returns, annotations, signal restoration and dependent cleanup order. Make Frida/JEB reporters fail on reasonless rejections and retain JEB fixture files while provider cleanup is incomplete. Own the nested packaged WASM verifier and expose its composite process/root cleanup capability for retry. Update runnable docs and isolated fixture dependencies. Validation: 547 focused cases in 20 files, eight isolated boundary unit checks, static/build/docs/discovery gates and duplication check (3.58%). Node process, SDK and descriptor cleanup are exercised; provider and packaging results in orchestration tests are simulated. Pack/install subprocess ownership remains with the existing shared package runner outside the nested-command change. |
||
|
morluto
|
aa9043f6b6 | fix(native): preserve qualified Swift protocol identities | ||
|
morluto
|
a4e4863029 | test(apple): distinguish root discovery from dylib resolution coverage | ||
|
|
1a48a82020 |
feat(analysis)!: default function MCP output to retained summaries (#5660)
* feat(ghidra): bound one procedure's decompile without closing the session Fixes #5340 * fix(ghidra): report decompile limit as a request timeout Keeps BinarySession from marking every Ghidra operation unavailable after one procedure's decompile reaches the limit, and names the MCP tool-call deadline in the remediation. * fix(ghidra): budget the decompile limit and report cancellation-closed sessions Adds REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS to the derived Codex/Kimi tool deadline and forwards it through setup. Marks a remote failure that settles after caller cancellation so the timeout projection no longer promises session reuse. * feat(ghidra)!: default analyze_function to a retained summary analyze_function was the one analysis tool never wired into REA's detail: summary|complete convention, so it always returned the complete native dossier inline (0.2-10 MB, dominated by value-flow and references). Agents on tool-result-capped harnesses never reached the pseudocode. Add `detail` to analyze_function, defaulting to summary: return a compact dossier (procedure identity, pseudocode size, facet counts, value-flow availability) and retain the complete dossier Evidence in the session. Page pseudocode, references, assembly and value-flow through normalized_result.parent_evidence_id with inspect_analysis_view, which already projects every native facet. detail: complete returns the full dossier inline as before. Delivery detail never reaches the provider or Evidence identity. BREAKING CHANGE: analyze_function now defaults to the compact summary view instead of the complete dossier inline. Pass detail: "complete" for the previous behavior. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * feat(cli): expose analyze_function detail on the function command Wire `rea function` into the shared CLI detail-delivery convention (--detail, --evidence_output) so the CLI and MCP adapters offer the same summary | complete choice for analyze_function. Summary delivery writes the complete dossier Evidence to a bundle and returns the compact summary view; it fails before analysis when --evidence_output is missing, so no provider work is wasted. The CLI default stays complete because a one-shot command has no in-process Evidence retention. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * refactor(ghidra): harden analyze_function summary after independent review FlatlineRoundtable cross-model review (6 lanes) converged on the summary projection's availability semantics being underspecified. Document that facet_availability is authoritative over raw pseudocode length, that code_units are UTF-16 (the unit inspect_analysis_view pages by), and that value_flow.truncated is meaningful only when value-flow is available. Parenthesize the nullish-coalescing/comparison for clarity. No behavior change; declined claims (value-flow "truncated collapse", unresolved_calls dead code) were schema-impossible or already correct. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * test(ghidra): cover analyze_function detail wiring on both adapters Close the roundtable's top convergent finding: the summary path had only domain-projection coverage, not the server/CLI wiring where the breaking behavior lives. MCP server (registerEnhancedTools): the default delivers a retained summary whose complete parent is recorded for paging; delivery detail is stripped before the provider request; detail: complete returns the full dossier inline; a registration without retention rejects a summary request before any provider work. CLI (rea function): the default returns the complete dossier; --detail summary without --evidence_output fails before analysis; with the bundle sink it returns the compact summary and writes the complete Evidence bundle. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(ghidra): keep analyze_function delivery detail out of provider inputs Address the five automated review findings on #5660. P1: `detail` lived in the shared enhancedInputSchemas.analyze_function, so inspect_native_api composed `{procedure, detail: "summary"}` and the strict Ghidra analyze_function schema rejected it. Delivery detail is now owned by the analyze_function MCP tool contract; the handler separates it before any provider call, and executeFunctionAnalysisEvidence parses the provider input only. The MCP regression test validates provider input exactly as the production Ghidra client does and exercises inspect_native_api. P2: the analyze_function summary Evidence now carries the parent's relationship_coverage like native facet views, so partial or unknown caller/callee discovery is not reported as complete. P2: the summary reports the dossier's declared facet_availability, nulls the counts of facets the provider declared unavailable, and lists each reason in unknowns, so Kuna's empty references, comments and basic blocks are not mistaken for observed absence. P2: inspect-analysis-view evidence-file sources accept the Evidence bundle that --evidence_output writes, selecting a record with source.evidence_id when the bundle holds several; the CLI test reads the saved bundle back through that route. Tests that assert the complete dossier over public analyze_function now pass detail complete, and the oversized-dossier test asserts the summary view it previously expected to be unsupported. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(ghidra): keep omitted function facets unknown in the summary The analyze_function summary coerced an omitted unresolved_calls record to a count of 0 while the native facet view reports the same dossier as "not recorded", and it carried no counts at all for the pageable swift_string_literals facets. Count optional records as null when the dossier omits them, name them in unknowns, and add nullable counts for both Swift literal facets. The summary Evidence envelope also dropped the function address: createAnalysisViewEvidence now derives one function-view predicate for both the address location and relationship coverage. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(scripts): request complete analyze_function dossiers in real-provider checks The real-provider check scripts call analyze_function over MCP and then consume the complete dossier: requireFunctionDossier, functionDossierSchema, .procedure/.pseudocode reads, operation assertions, compare_functions inputs and the oversized-dossier rejection path. The summary default broke them. Every such call site now passes detail: "complete". Provider-direct callers are unchanged because detail is an MCP contract field; the admission-error sites stay unchanged because provider failures return before the summary branch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(ghidra): keep unavailable value-flow counts unknown in the summary summarizeDossier coerced the five value_flow_* counts to zero when native_value_flow was null (IDA, Kuna) or declared unavailable (Ghidra), so the summary reported empty collections while value_flow.available said otherwise. The counts are now null unless value flow is available, with the reason already named in unknowns. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(evidence): budget authentication headroom by file shape and serialized size The evidence-file headroom reserved two parsed copies, calibrated for one record, and charged strings at two bytes per character. Bundle authentication additionally holds the schema-parsed bundle, the re-parsed records and two JSON.stringify results, which expand control characters and lone surrogates sixfold: a bundle whose raw_result is a million NUL characters estimated at 2 MB parsed but needed 12 MB of serialized text. The budget now has a parsed term and a serialized term per file shape, owned by EvidenceAuthenticationHeadroom and shared by the analysis-view reader and the bundle importer, which had no guard. Two transient peaks the budget did not name are removed at their producers: the canonical digest hashes a large leaf directly instead of concatenating it into its buffer, and the JSON validator walks containers by cursor so a wide array no longer queues one entry per member. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(mcp): route analyze_function summary replies to dossier consumers in docs and prompts The MCP guide, the version-comparison and crash-triage prompt contracts, the authored skill and the Kuna, IDA and function-matching docs still told agents to pass analyze_function replies straight to compare_functions or build_call_path. Under the summary default those consumers reject the reply. They now name the retained-evidence reference for compare_functions, the observations fields of match_native_functions, and detail: "complete" for build_call_path, which accepts only inline dossiers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * test(evidence): pin validator member order and keep interrupted validation yielding Independent review of the cursor validator found the interrupted-validation fixture relied on the old last-first walk: its over-deep head was reached before the first cooperative yield. The padding now precedes the deep element so the walk yields first, which is the invariant the test guards. A new case pins first-defect-in-member-order reporting, skipped non-enumerable keys no longer count toward the yield cadence, and the import-headroom test loads the reader through a fresh module graph so it holds under the services project's shared workers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(ghidra): align deadline bounds and cancellation outcomes * refactor(ghidra): shorten deadline recovery guidance * fix(mcp): clarify the scope of inferred decompile deadlines * test(ghidra): verify deadline workflows and document measured costs Separate failure and registration cases by their distinct contracts. Record actual Linux Ghidra, CLI and MCP evidence with measured context, elapsed time, CPU and memory, and explicit remaining coverage limits. * docs(ghidra): pin integration and distinguish baseline format failures * fix(ghidra): own failure lifecycle wording in the provider * test(ghidra): budget timeout fixture generation before allocation * fix(analysis): preserve summary provenance and bound bundle guidance * fix(ghidra): align capability and traversal deadline guidance * style(config): align deadline guidance formatting * docs(ghidra): record capability and traversal verification * docs(analysis): use absolute paths for Evidence files * docs(analysis): record completed function summary measurements * docs(analysis): update main integration validation --------- Co-authored-by: Syed Rishad <rishadzzzz@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
morluto
|
6aae8efd35 | fix: preserve selected Electron display authorization | ||
|
morluto
|
0c4bf68ce8 | merge: reconcile concurrent main fixes with issue remediation | ||
|
morluto
|
93de7bc7e3 | fix: preserve mixed Swift demangler record outcomes | ||
|
kiranandgensokyo-dev
|
5d2c8c2b78 |
feat(cli)!: export row-oriented results as JSON Lines (#5629)
* feat(cli): add JSON Lines output for row-oriented results (#5487) - Support opt-in JSON Lines format (--format jsonl) for row-oriented results and collection views with clear record boundaries - Preserve stable source identities and stream records deterministically line-by-line - Emit final coverage and limitation metadata as a distinct line with https://rea.dev/schemas/jsonl-coverage-v1.json to prevent confusion with data rows - Distinguish error outputs with https://rea.dev/schemas/jsonl-error-v1.json - Reject non-row-oriented results that lack record boundaries with an unreadable_output error - Document JSON Lines streaming conventions in docs/cli.md - Add boundary tests covering empty results, multiline strings, partial coverage, large fields, and non-row rejection * fix(cli): preserve diagnostics, provenance, and nested collections in JSONL * fix(cli): frame classified failures and reject unrepresented JSONL facets --------- Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
morluto
|
d137a4cdde |
fix(android): preserve metadata dialect and root manifest identity
Read Apktool 3's producer string dialect before YAML can discard hashes or reinterpret values. Preserve older YAML behavior and decode producer escapes once. Select the exact root AndroidManifest.xml resource in REA's JADX bridge and derive summaries once with namespace-aware XML parsing. Preserve decoded text, summarize before its display cap, invalidate cached facts with the loaded session, and remove superseded upstream manifest handlers. Keep the engine pinned to 0.7.1. Validated Apktool parser/provider and public adapter controls; compiled the bridge against the verified pinned JAR and ran its real Java regressions plus public CLI and stdio MCP manifest parity checks. Real Apktool execution remains unverified; the new manifest fixtures use plain XML. No Android application was executed. Fixes #6804 Fixes #6866 Fixes #6863 Fixes #6819 |
||
|
morluto
|
8d8d85ba22 | refactor(cli): share parsed output controls across consumers | ||
|
morluto
|
b33f68deb7 | refactor(providers): separate identities from workflow profiles | ||
|
morluto
|
da9a580a0d | refactor(javascript): unify application workflow ownership | ||
|
|
8b478d0f15 |
fix(native): preserve Mach-O install-name whitespace (#4979)
* fix(native): preserve whitespace in otool load-command strings otool prints dylib install names and rpaths as `<key> %s (offset %u)` with exactly one separator space around the recorded string. The parser trimmed the value and stripped the offset suffix with `\s+`, so a valid install name such as " leading.dylib " became "leading.dylib" in both the dependency inventory and the raw load-command fields. Parse these offset-suffixed string fields by their exact format and strip only the single-space offset suffix. Fixes #3384 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(native): verify exact Mach-O install names through CLI and MCP --------- Co-authored-by: Bhaaraadwaj <bharadwajdaya@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
Vishal Tambiandgensokyo-dev
|
9c81191b1d |
fix(ghidra): treat format-specific disabled analyzers as not applicable (#5425)
* fix(ghidra): treat format-specific disabled analyzers as not applicable * test(json): assert composed cancellation cleanup for staged output --------- Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
|
6dc0f60faa |
refactor(process): share paired prlimit ceiling ownership (#6352)
* refactor(process): share paired prlimit ceiling ownership Route firmware, Wakaru, and pcap paired ceilings through LinuxResourceLimits without changing their byte or CPU budgets. Keep EVMole's soft-only limits as a dated architecture-guard exception. Co-authored-by: morluto <morluto@users.noreply.github.com> * test(pcap): admit capture limits without host prlimit The ceiling test only checks the shared argument prefix, so it uses a temporary executable instead of /usr/bin/prlimit. Co-authored-by: morluto <morluto@users.noreply.github.com> * style(pcap): format the capture ceiling test oxfmt rejected the object literal wrapping in PcapNativeBound.test.ts. Co-authored-by: morluto <morluto@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <morluto@users.noreply.github.com> |
||
|
Kaoru0822andClaude Opus 5.5
|
c02945bc44 |
feat(native): decode Swift nominal type declarations from __swift5_types (#6257)
* feat(native): decode Swift nominal type declarations from __swift5_types Fill swift_decls from the nominal type context descriptors that survive stripping: kind (class, struct, enum, actor), own and qualified name, module, generic flag, anonymous-context flag, metadata accessor and descriptor location, with evidence citing the type record. Unresolved parent contexts keep qualified_name null and the entry partial. One shared reader walks each context once; class vtable recovery now consumes its descriptors instead of re-reading the records, so a type record that cannot be read keeps both facets partial. Remove the never-populated swift_decls fields that descriptors do not carry. Fixes #5341 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(native): bound Swift context walks and decode vtables in record order Address review of the nominal-type reader: - Cache intrinsic context depth (distance to the module) so a cached parent cannot bypass the 32-context bound, and never cache a stop that depends on where the walk began. Bound qualified names to 4096 characters so nesting cannot multiply name storage. - Visit each descriptor as soon as its kind and name are read, so a class vtable spends the record budget in record order instead of after every later declaration. - Report an accessor outside executable mapped code as null and keep the declaration partial. - Forward a name failure to vtable coverage only when the record is a class or its kind is unknown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Kaoru0822andClaude Opus 5.5
|
ef5ab1f1f2 |
fix(apple): decode NIBArchive booleans with AppKit's polarity (#6255)
The NIBArchive value decoder read type tag 4 as true and 5 as false, the reverse of what ibtool writes and AppKit's decoder returns, so every compiled-nib boolean in decode-interface-builder output was inverted. The test fixture encoder shared the reversed mapping, hiding it in round trips. Name the value-type tags once and use them in both the read and skip paths, fix the fixture encoder, pin raw tags in a unit test, and make verify:interface-builder compare both polarities with ibtool --export. Fixes #5541 Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
gensokyo-dev
|
f86206535f |
fix(ci): validate artifact MCP replies against named schemas (#6021)
Bind artifact lifecycle and analysis calls to their advertised named SDK output schemas, sharing discovery per connection and rebinding on reconnect. Reject schema overrides and ambiguous catalogs, preserve request options, and retain primary failures while attempting all owned cleanup. Add real-stdio boundary regressions for malformed output, discovery reuse, reconnection, schema ambiguity, override rejection and timeout preservation. Assisted-by: Reimu (gensokyo-skills:reimu-incident-triage) Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing) |
||
|
morluto
|
d0067ba34a |
fix(native)!: preserve Swift pointer nullability and import bindings
Resolve relative pointers using field-owned ABI encodings. Keep null, raw, rebased, imported and unknown targets distinct, and retain imported conformance rows with exact Mach-O binding provenance. Decode legacy rebase membership with source-derived memory and work bounds, preserving validated prefix facts when streams are incomplete. Validation: check:ci, docs:check and build passed; 158 source tests and 20 schema/acceptance tests passed. Ten source-built Apple dispatch variants and four issue-specific CLI/MCP workflows passed without executing targets. Paired cold/warm CLI/MCP cost observations retained locally; model tokens and maximum-sized input runtime were not measured. Closes #5683 Closes #5685 BREAKING CHANGE: Swift conformance type_name and protocol_name can be null when names cannot be resolved; consumers must use typed type_binding and protocol_binding evidence. MachoImportBinding replaces the former Objective-C-specific binding type and schema. |
||
|
morluto
|
1bb2fa23de | test(apple): match plist provenance to the selected workflow | ||
|
morluto
|
5518363aa6 | test(artifacts): compare matching resource-root workflows | ||
|
|
23196daa11 |
fix(apple): read LC_LAZY_LOAD_DYLIB_INFO dependency payloads (#5650)
* fix(apple): read LC_LAZY_LOAD_DYLIB_INFO dependency payloads Parse the linkedit install name for lazy-load info commands, resolve those edges without traversing them, and stop inspect-macho from treating an otool dependency list as exhaustive when that payload is omitted. Fixes #5551 Fixes #5649 Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(apple): bound lazy-load payload reads and image offsets Reuse identical LC_LAZY_LOAD_DYLIB_INFO ranges, cap their combined size, and reject flag or chain-start offsets outside the slice VM range. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(apple): bound lazy-load string scans and contained reads Reject payload strings that start in the 24-byte header, validate each file-offset string once, and reuse payload bytes already read for a containing range. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(apple): reject cached lazy-load strings outside a payload Remember each cached string's terminating offset and require that NUL to fall inside the payload being decoded. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(apple): bound lazy-load payload range lookups Count containment checks across earlier payload reads and stop at the existing 16 MiB work limit. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(apple): count every lazy-info string reference Charge repeated and overlapping LC_LAZY_LOAD_DYLIB_INFO scans against the 16 MiB validation budget, and stop a scan once that budget is exhausted. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(apple): cap retained lazy-info dependency results Stop after 1024 LC_LAZY_LOAD_DYLIB_INFO commands so a load-command region full of identical commands cannot become a million traced edges. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(apple): reject a lazy-load path NUL in the final byte dyld stops the load-path scan before the payload's last byte, so a terminator in that byte leaves the image malformed. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(apple): keep may-be-missing lazy loads optional A missing LC_LAZY_LOAD_DYLIB_INFO with the may-be-missing bit is weak, so the lazy finding says dyld continues on first use instead of failing. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(apple): cap lazy-info symbol string references A payload of distinct one-byte symbol strings stayed under the byte budgets while caching one entry per offset. A slice now stops at 65536 symbol references. Co-authored-by: morluto <morluto@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <morluto@users.noreply.github.com> |
||
|
gensokyo-dev
|
1c2c2f1e78 |
fix(process): settle native exit and repair ownership verification (#5249)
* fix(process): classify baseline cleanup with current ownership facts Use the pre-launch baseline only to select candidates for current native identity/token validation. Preserve confirmed absence and verified foreign generations instead of carrying a premature baseline identity failure. Exercise the real Linux enumeration/exit race and retain countercontrols for unknown identities and live token-owned processes. Assisted-by: Reimu (gensokyo-skills:reimu-incident-triage) * fix(ci): distinguish process inventory from historical PID lookups * test(ci): verify decoded historical PID controls before tracing * fix(ci): trace historical PID handles and signal probes * fix(ci): resolve traced directory paths across pathname syscalls Classify producer-decoded FD/path pairs once instead of limiting failed relative lookup detection to openat on /proc. This preserves attempted readlinkat, statx and newfstatat accesses with no returned descriptor. Three regression cases fail against the previous policy and all 33 controls pass with the shared pathname interpretation. Assisted-by: Reimu (gensokyo-skills:reimu-incident-triage) * test(ci): declare unresolved legacy pathname coverage Report the historical-PID access surfaces actually checked and explicitly mark legacy cwd-relative pathname calls unverified. The existing tracer does not provide their cwd identity, and a blanket rejection also rejects legitimate GDB build-ID metadata queries. Retain every committed access guard and document the preexisting limit without claiming absence from missing path evidence. The complete real 44-case CLI/MCP recorded-crash qualification passes with the coverage declaration. Assisted-by: Reimu (gensokyo-skills:reimu-incident-triage) * fix(ci): reject priority access to historical PIDs * fix(ci): preserve C-quoted components in traced path checks strace renders non-ASCII pathname bytes as C octal escapes, not JSON strings. Keep those components opaque when resolving literal ASCII path separators and proc PID selectors, matching the existing absolute-path interpretation. Two controls fail before the correction and all 35 pass afterward. An actual strace observation independently reproduces the rejection of an unrelated Chinese filename and verifies the corrected classification. Assisted-by: Reimu (gensokyo-skills:reimu-incident-triage) * fix(ci): reject historical PID resource and scheduler calls * fix(schema): preserve portable large base64 validation * fix(ci): share explicit PID syscall contracts with tracing Derive tracer selection and reported coverage from the same named target contracts used by the access checker. Cover resource limits, scheduler calls, process metadata, PID groups, capabilities and pidfd operations without mixing UIDs, policy enums, self selectors or output values with target PIDs. Interpret each target once and retain unresolved target facts as a distinct failure. Emit the coverage catalog once per verifier result. The previous oracle misses 28 new controls. All 83 controls pass after the change; real native resource/scheduler probes capture 14 previously missed calls with resource limits and affinity unchanged. All 35 selected syscall names are accepted by the pinned local strace. Assisted-by: Reimu (gensokyo-skills:reimu-incident-triage) * fix(ci): classify I/O priority and qualify PID trace coverage * fix(process): keep ownership retries on the native clock * fix(ci): retain process edges in ownership trace inventory Keep fork/vfork and resumed edges along with clone events so the shared attribution owner receives the complete process relation facts it consumes. Assisted-by: Reimu (gensokyo-skills:reimu-incident-triage) * fix(process): type native exit predicates and ownership fixtures precisely * fix(ci): retain trace coverage limits after integration Restore explicit unverified symlink aliases and same-Node stat/status purpose in the shared coverage contract and guide. The native-exit and trace-attribution integration does not establish those properties. Preserve both limitations rather than silently broadening the reported qualification. Assisted-by: Reimu (gensokyo-skills:reimu-incident-triage) |
||
|
morluto
|
aacf740a04 |
fix(ci): correlate ownership trace threads and preserve setup failures
Follow CLONE_THREAD edges across strace -ff files when excluding the ownership process's stat/status reads. Keep provider child processes outside the allowance and document the same-process attribution limit. Validate setup result array shapes and preserve the original failed-plan JSON instead of masking it with a diagnostic TypeError. |