* fix(flutter): name a missing or non-file APK selection
identify_flutter_build and inspect_dart_aot reported a missing path or a
directory as "The APK could not be read as a package: <path>: ENOENT ...",
which blames the package bytes for a selection mistake. Report
"Selected APK does not exist" for ENOENT/ENOTDIR and "Selected APK is not
a regular file" for a non-regular selection, matching the SQLite
inspector's wording. The issue path and invalid_format reason are
unchanged; package format failures keep their existing message.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(filesystem): classify unopenable non-regular selections at admission
A Unix socket cannot be opened at all: open() fails with ENXIO on Linux
and EOPNOTSUPP on macOS (surfaced by libuv as "Unknown system error
-102") before openRegularFile can stat the descriptor, so such a
selection escaped NonRegularFileReadError and the Flutter provider still
blamed the package bytes.
When open fails, classify the path by its own metadata (lstat when
symlinks are rejected, stat otherwise): an existing non-regular file
becomes NonRegularFileReadError. Missing paths, permission failures,
regular files, and the O_NOFOLLOW ELOOP rejection keep their original
errors. The Flutter selection test now covers a real Unix socket.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(filesystem): reclassify only unopenable file-type errors at admission
Checking the path's metadata after any open failure turned EACCES on an
unreadable directory into NonRegularFileReadError, so callers reported
invalid input instead of access_denied. Reclassify only errors that mean
the file type cannot be opened (ENXIO, EOPNOTSUPP, ENOTSUP, matched by
errno because libuv has no name for macOS EOPNOTSUPP 102); permission,
symlink-rejection and resource failures keep their original errors.
Run the Unix socket selection check in its own test, skipped on Windows
where listen() takes named pipes, and keep the missing-path and
directory checks cross-platform.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(filesystem): keep cancellation that arrives during the non-regular probe
A caller abort while the post-open metadata probe ran was replaced by
NonRegularFileReadError, so callers such as parseBinaryTarget reported an
input-format error instead of cancellation. Recheck the signal before
substituting the non-regular classification.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Inventory expands an ASAR-named filesystem file only when its bytes are
an ASAR. JavaScript reconstruction expanded it on the suffix alone, so
a directory holding any other file named *.asar failed the whole
analysis with an invalid ASAR header.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
* fix(apple): report a connector that cannot name its owner as an omission
A connector is recognized by class name and then dropped by an early
`continue` when its owner cannot be resolved. Nothing recorded the drop, so
connection coverage derived `complete` with `examined: 0` and `omitted: 0`
for a connector that was present, recognized and discarded.
Both producers of that coverage contract are affected and are changed
together, because they share the facet and the reason value:
- the keyed-archive parser records the drop as `unresolvedConnections` and
reports it through the `connections:<path>` facet, the document
`connection_count` and the graph omission total;
- the NIBArchive projection in the artifact adapter recorded nothing at all,
and the projected dictionary carried no trace of the dropped connector for
a downstream consumer to find. Its count is taken where the connector is
discarded and plumbed through the same per-document omission channel that
already carries incomplete hierarchies, so the reason follows the cause.
An unresolved source cannot be represented as a connection at all, because
connection identity requires a non-empty source; that is unlike an unresolved
destination, which the graph already retains as an unresolved edge.
`connection_limit_reached` stays reserved for connections dropped by the hard
cap and `serialized_view_hierarchy_incomplete` for hierarchy omissions.
`connection_count` also includes the unresolved connectors. That field already
counts connectors the parser recognized but could not emit, because
connections dropped by the hard cap are counted there too, so unresolved
connectors follow the existing rule rather than a new one.
Verified with 98 focused Interface Builder and catalog tests, check:fast and
verify:test-discovery on macOS arm64, and through the public CLI: a rooted
archive holding one ownerless UIRuntimeOutletConnection reported complete with
examined 0 and omitted 0 before this change and reports partial with the reason
connection_unresolved and omitted 1 after it.
* fix(apple): count ownerless NIBArchive connectors and keep the limit reason
Greptile found two defects in the NIBArchive half of this change.
The document summary was left unchanged, so the same recognized connector
counted as one in a keyed archive and zero in a NIBArchive. The dictionary
parser only sees connectors that were projected, so the omission recorded
where the connector was dropped is added to the matching document here.
The facet reason was overwritten unconditionally, so a NIBArchive holding both
a connection dropped by the limit and an ownerless connector reported
connection_unresolved and lost the cause a higher limit could recover. The
existing reason is kept when the graph already named one, which is the
precedence the keyed-archive producer applies.
Verified with `npm run test:focused` over the bundle suite (42 passed, 1
skipped), `npm run lint` (0 errors) and `npm run check:fast` on macOS arm64.
Reverting the reason precedence alone fails only the limit case; reverting the
document count fails the count assertion in both cases.
The new case moved into its own describe block because the enclosing callback
crossed the repository's max-lines-per-function bound otherwise.
---------
Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
* fix(native): keep lipo path echoes out of FAT records
Parse FAT records only after separating the exact echoed operand path. Pass the same lipo operand at all three consumers and cover multiline paths through parser, slice-reader and compiled CLI regressions.
* fix(native): use parsed lipo framing for container offsets
* test(native): use explicit FAT fixtures for CLI regressions
inspect_artifact parsed a fresh inventory and inspection report but left
both mutable, so each later Evidence boundary cloned and re-digested the
complete payload: createArtifactInspection re-authenticated the nested
inventory Evidence, createAnalysisExecution re-parsed the report, and
recordEvidence re-authenticated the outer Evidence.
Seal both exclusively owned values with freezeJsonSnapshot so those
boundaries take the existing immutable-snapshot path. Output is
byte-identical.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(artifacts): report malformed ZIP archives as format failures
zip.js signals archive faults (missing end-of-central-directory record,
unrecognized format, overlapping members, corrupt member data,
unsupported compression) as plain Errors. ZipArtifactReader passed them
through untyped, so the artifact provider's fallback reported every one
as `io` with no detail and told the caller to "check file access".
Type these faults at the ZIP reader boundary: structural and data
corruption become `format` failures and unsupported archive features
become `unavailable`, each carrying the zip.js reason as detail. This
covers enumeration, the getData rejection, and the WritableStream abort
path that delivers corrupt-data errors first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(artifacts): share the ZIP inventory failure assertion
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(artifacts): read ASCII control bytes in legacy ZIP names as ASCII
ZIP names without the UTF-8 flag were decoded with zip.js's CP437 table,
which maps bytes 0x01-0x1F and 0x7F to IBM PC display glyphs. A folder
zipped on macOS with a custom icon therefore listed and extracted
`Folder/Icon\r` as `Folder/Icon♪`, a name that is not in the archive and
differs from REA's directory inventory of the same folder. Tabs became `○`
and DEL became `⌂`.
Decode legacy names in REA's existing decodeText hook with the Unicode
CP437 mapping that Python's zipfile also uses: bytes below 0x80 as ASCII,
and the upper half with the same table zip.js uses.
Windows file names cannot contain U+0001-U+001F, so a Windows extraction
destination now refuses such a path with a typed path failure before
creating anything, as it already does for ':'.
* test(artifacts): check extracted legacy ZIP control names on disk
The control-name cases checked only the logical path that inventory
reports. Extraction writes through SafeOutputTree, so a change that
renamed the extracted files would still pass those cases.
Extract `Icon\r` and `é\t\u007F.a`, then compare the reported
relative_path, the directory listing, and the file contents. The new
cases skip on Windows, which refuses these names.
safeOutputTree.test.ts covers that refusal.
Keep failed stable-read handles and WABT runtime roots under retryable resource scopes. Preserve cancellation causes and primary errors, close the same service instances bound to MCP tools, and consolidate CLI operation/cleanup projection.
Validation: fast typecheck and lint, cached runtime build, docs checks, SDK schema validation, focused service and real-file cleanup tests, CLI cleanup regressions, and MCP lifecycle/shutdown workflows. Provider process fixtures simulate engine output; real WABT and Linux EVM execution are not established by this pass.
* fix(native): name the ownership helper prerequisite in command failures
Every supervised native command launch needs the Darwin process-ownership
helper. When that helper cannot be compiled (relative REA_DARWIN_SWIFTC_PATH,
missing SDKROOT, a missing compiler, or an SDK/compiler mismatch), the
runner reduced the typed DarwinProcessOwnershipInspectionError to a bare
`io` failure. inspect_macho then reported `execution_failure` with no hint,
and inspect_artifact on any universal Mach-O said only "lipo could not
enumerate universal Mach-O slices".
The runner now keeps the inspection message as the failure's `detail` at
the launch boundary and carries it through xcrun resolution. The native
provider adds it to its diagnostics and the slice reader appends it to its
message, matching what the SQLite and capture providers already report.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(native): cover the provider's ownership helper diagnostics
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
extract_artifact carried its own copy of the ZIP package format list with five
of the eight members the domain owner declares, and used it both as the guard
that refuses a format without an extraction reader and as the selector for the
reader itself. A .hap, .hsp or App Pack target was therefore refused with
"Artifact format has no extraction reader" even though the next branch builds a
ZipArtifactReader for exactly those formats, while inventory_artifact classifies
the same path through the owner and reads it.
Import isZipPackageFormat from the domain owner and delete the duplicate list
and its predicate, removing the second representation instead of reconciling it.
The reader branch now narrows to ZipPackageFormat, which is what the
ZipArtifactReader constructor already requires.
Verified with 41 boundary extraction, ZIP integrity and detection tests,
check:fast and verify:test-discovery on macOS arm64. Reverting only the source
change fails exactly the three added families and leaves the zip control green.
Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
Use the shared resource scope and private runtime root for signature snapshots. Retain failed file-handle/root cleanup, preserve primary failures alongside cleanup diagnostics, and wait for binding operations during client close before retrying owned resources.
Validate root cleanup failure, blocked retry, repeated client close and in-flight acquisition. Existing provider/MCP/CLI tests and the real Darwin codesign ancestor-swap case pass. Dedicated native file-handle close-failure injection remains unverified.
* fix(artifacts): bound archive decompression by source size
A ZIP/APK/IPA/MSIX member is attacker-controlled and can expand far
beyond the archive that carries it. Inventory hashed every member to EOF
and extraction wrote every member with no ceiling, so a ~2 MiB archive
could drive ~30 s of CPU during inventory and ~150 s plus 2 GiB of disk
writes during extraction.
A fixed per-entry byte cap is the wrong shape: it rejects legitimately
large members inside large packages. The ceiling instead scales with the
archive's own size (500x, with a 256 MiB floor), so large packages keep a
proportionally large budget while a small archive cannot amplify into
unbounded work. The cumulative declared size is charged from the central
directory before any entry is decompressed, and zip.js already verifies
streamed output against each declaration, so the check is exact and
covers both inventory and extraction through the shared reader.
* fix(artifacts): own decompression accounting per traversal
* test(artifacts): verify declared ZIP output enforcement
---------
Co-authored-by: baron <richstyles@gmail.com>
* fix(artifacts): name CPU_SUBTYPE_ARM64E_X1 Mach-O slices arm64e
macOS 27 system binaries such as /bin/ls carry a third FAT slice with
cpusubtype 0x8000000c. The SDK defines subtype 12 as
CPU_SUBTYPE_ARM64E_X1, and CPU_SUBTYPE_HAS_ARM64E puts it in the arm64e
family. architectureName knew only subtype 2, so it named the slice
arm64, lipoNameMatches rejected lipo's arm64e.x1 name, and
inspect-artifact /bin/ls failed with integrity_mismatch.
The arm64e.x1 test fixture now uses the real subtype. It fails without
this change.
Refs #1168, #1765
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(artifacts): classify en-GB hdiutil verify refusals as format failures
hdiutil follows AppleLanguages. With en-GB it writes "image not
recognised", and verifyFailureReason matched only the US spelling, so a
file that is not a disk image got the reason unavailable. Both spellings
now give format. Other languages still give unavailable.
Refs #1765
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(process): read the full process table in the lifecycle cleanup check
The check ran ps -axo command= with the Node default maxBuffer of 1 MiB.
On a host with 1905 processes, the output is 1.52 MB, and the test failed
with "stdout maxBuffer length exceeded".
Refs #1765
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(process): bound cleanup observations to sampled descendants
* fix(apple): retain CPU subtype identity during dylib resolution
* test(process): retain OS PIDs for descendant observations
---------
Co-authored-by: David Lee <davidlee@Dav3Book-Pro-M4-Max.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Classify FAT containers using shared architecture, range, and observed slice-header facts rather than the CAFEBABE signature alone. Preserve typed unsupported and malformed reasons, reject malformed sibling slices consistently, and remove repeated range parsing in consumers.
Validation: valid small and large JVM classes in public JAR/ZIP inspection, four FAT width/byte-order controls, and focused executable/Mach-O boundary tests. The class fixtures were checked with JDK javap; no new JVM analysis capability is advertised.
The Apple selection and Hopper readers retain their existing parsing because they also own oversized-count diagnostics, reserved fields, streaming final-size validation, or source freshness that the shared header result does not represent.
Representative public mixed-ZIP inventory probe, four fresh processes with an already warm filesystem cache: first process elapsed 626.8 -> 548.3 ms, CPU 740 -> 740 ms, sampled family peak RSS 375.5 -> 345.4 MiB. Subsequent three-process medians: elapsed 767.2 -> 454.0 ms, CPU 890 -> 640 ms, peak RSS 368.7 -> 346.2 MiB. Estimated output tokens 2310 -> 2292. Concurrent load was uncontrolled, so these observations do not establish a speed improvement. Whole-agent task tokens and schema/call overhead were not measured.
Fixes#2317
Classify coverage once at inventory production and require it in current outputs. Normalize older retained Evidence at the inventory parser, reject inconsistent page statuses, and remove comparison heuristics over limitation prose.
Validation: public CLI-to-MCP plist comparison, changed-behavior and unknowns workflow; boundary checks for legacy Evidence, partial coverage, page disagreement, content and executable changes; generated docs checks. Fixes#2173.
Select authoritative connection fields by runtime class in a shared owner used by keyed archives and NIBArchive projection. Preserve archive identities, endpoint relationships and coverage, and replace duplicate synthetic outlet assertions with a rooted public CLI fixture.
Validation: compiled CLI acceptance against the issue-sourced rooted UIKit fixture and 41 filesystem tests. Native UIKit framework validation was not rerun. Fixes#2177.
Observe MZ header declarations and signature uncertainty at the domain boundary, then reuse those facts in bounded target resolution, inventory, and Go PE parsing. Preserve DOS relocation ownership and fail closed when the relevant bytes remain unexamined.
Validation: 59 domain tests, 130 target/Go boundary tests, and public CLI inspection of a clang/lld-link PE DLL with a short DOS header. Fixes#2172.
Add offline SQLite database/WAL inspection through the shared CLI/MCP.
Use compact inventory, selected-table schema, exact ordered columns and
bounded row samples while preserving source identity and lossless values.
Retain completed Evidence through late cancellation and private-root cleanup
failures, bound concurrent inspections, and recheck journal validity after
acquisition. Reuse prepared CI runtime artifacts across supported Node lines.
Co-authored-by: Y0UW1SH <29498139+Y0UW1SH@users.noreply.github.com>
Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)
* fix(apple): join recognized keys from NIBArchive field maps
inspect-asset-catalog returned no resource-key matches for a decoded
NIBArchive resource whose recognized image key is present in the archive. The
NIBArchive projection groups an object's decoded fields under the node's
nibValues attribute beside the class, label and ID it derives from the same
archive, while the AppKit keyed-archive parser flattens the same fields onto
the node. The catalog join read only the top level, so a recognized NSImageName
on a decoded resource produced no row, and the empty-catalog limitation still
claimed resource keys were reported as unmatched.
Read an archive object's fields from both shapes, and give the namespace one
owner by exporting NIB_VALUES_ATTRIBUTE from the projection that writes it.
The recognized-field set stays in the catalog analysis, which owns that
contract.
Verified with 5 focused catalog tests, check:fast and verify:test-discovery on
macOS arm64, and through the public CLI against the reported NIBArchive
fixture. The keyed-archive control keeps its existing row, and a NIBArchive
with only an unrecognized title still produces none. The ibtool-dependent
native hierarchy lane requires full Xcode and was unavailable here.
* fix(apple): normalize resource keys at the NIBArchive producer
Fold the complementary implementation from #2256 into the earlier #2248 proposal, retaining the original contribution history.
Co-authored-by: Thang Luu <76550342+thangldw@users.noreply.github.com>
* fix(apple): restrict catalog joins to parsed resource nodes
---------
Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Thang Luu <76550342+thangldw@users.noreply.github.com>
* fix(apple): preflight XML plist nesting before keyed archive decoding
inspect_keyed_archive documented a 128-level nesting limit, but the XML
plist path reached recursive decoding before any depth check: plist and
the prototype-key traversal in parseXmlPropertyList recurse once per
nesting level, so a ~75 KiB, 3000-level XML plist exhausted the call
stack and reported Maximum call stack size exceeded instead of the
bounded-resource limit that shallower over-limit documents (129, 200)
already produced. The CLI classified both as truncated, so no crash or
classification bypass was involved; the diagnostic was the defect.
Reject excessive nesting in parseXmlPropertyList before any recursive
pass, using the projection's own limit (MAX_PLIST_DEPTH) so the error is
identical to the one projectPlistValue raises: callers keep the
limit/truncated classification with no error-handler translation. Depth
counts values the way the projection does - container members nest one
level deeper, dictionary keys name members rather than nesting, and a
self-closing element is the value at its own level - so the guard
rejects exactly the documents the projection rejected, at the same
boundary, and every in-limit document decodes unchanged. The guard sits
at the shared decoder boundary, so Interface Builder, app Info.plist
resolution and native inspect_plist gain the same deterministic limit;
the binary plist path keeps its existing expansion preflight.
Binary plist behavior, ordinary XML decode, __proto__ omission counting,
and archive digest/Evidence provenance are unchanged. OpenStep
autodetection is untouched by this guard, as the existing Interface
Builder preflight also scopes its depth check to XML.
Regression coverage: decoder unit tests at and just past the limit for
dictionary and array nesting with self-closing and opaque markup,
module tests through decodeKeyedArchiveBytes including a valid
NSKeyedArchiver XML envelope at the limit, a production-boundary test
through inspectBundleKeyedArchive, and the compiled public CLI
reporting code truncated / reason limit for a 3000-level plist.
* fix(apple): check XML nesting against parsed element structure
Reuse the iterative XML decoder document for depth preflight and prototype-key lookup. Bound recursive plist wrappers alongside dictionaries and arrays without interpreting markup inside attributes or DOCTYPE declarations as values.
Validated with 104 focused tests including compiled CLI, MCP prototype-key, and Interface Builder coverage; npm run check and verify:test-discovery passed. No real-provider verification was run.
---------
Co-authored-by: Zburgers <Zburgers@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
* feat(flutter): identify Dart snapshot builds by pure APK parsing
Add a flutter-provider family with one operation, identify_flutter_build,
that reads an APK's lib/<abi>/libapp.so and libflutter.so through the
hardened ZIP reader and reports build facts as observations:
- The Dart VM snapshot hash, read 20 bytes after every snapshot magic
(f5f5dcdc). The vm and isolate sections are cross-checked: agreeing
sections collapse to one snapshot_hash, disagreeing sections keep every
candidate with snapshot_hash null and partial coverage.
- Per-ABI library digests and byte sizes, plus the engine library's GNU
build-id note and any labeled 'Dart VM version' or Android clang/LLVM
toolchain strings, verbatim; release builds without a version string
report null rather than a guess.
- The APK is digested before parsing so Evidence binds the observation to
the exact input, and a target with no libapp.so reports not-detected
with complete coverage.
The scanner is streaming with explicit pending-magic and rescan
bookkeeping, so chunk boundaries cannot double-count or drop sections, and
magic-free streams never accumulate. No engine, no external lookups: the
hash-to-SDK mapping stays with external datasets by design.
Verified against Flutter Gallery 2.9.2 (112 MB, three ABIs): every ABI
reports the same agreeing hash 65817c30a78bb44c3dc3771876b6010a from two
sections, per-ABI build-ids, and the clang toolchain line; a plain
aapt2-built APK reports not-detected with complete coverage. Scanner
goldens come from the same libraries. READMEs carry the new 'Flutter apps'
row in every supported language.
* feat(flutter): inspect Dart AOT snapshots through the ELF symbol table
Add inspect_dart_aot to the flutter family: for one ABI's libapp.so it
locates the four _kDart*Snapshot* symbols through the ELF dynamic symbol
table (ELF32 and ELF64 little-endian, bounds-checked reads; a payload
without exported symbols is refused), reports each section's offset and
size, validates data-section headers (magic, kind, length; instructions
sections honestly report nulls for the fields their layout does not
carry), and projects a bounded string pool from the isolate data section:
distinct package: source URIs (the app's dependency inventory), dart: SDK
URIs, and identifier-shaped tokens, each with counts, bounds, and partial
coverage when truncated.
The string pool is labeled as a byte-pattern projection, not deserialized
cluster semantics: typed class and function recovery remains an
SDK-specific parser, which the snapshot hash now unlocks. The ABI is
caller-selected with a deterministic default (arm64-v8a, x86_64, then
first available).
Verified against Flutter Gallery 2.9.2: all four symbols on both ELF64
(arm64-v8a) and ELF32 (armeabi-v7a) images, header validation, hash
agreement with identify_flutter_build, and a 756 package: / 201 dart:
URI pool from the isolate data section; goldens in DartElf.test.ts come
from the same libraries.
* fix(flutter): own snapshots, bound image reads, and preserve cancellation
Mirror the ownership bar the maintainers applied across the merged JEB,
ADB, and Apktool PRs:
- Copy the admitted APK into a provider-owned, read-only snapshot (EXCL
create, chmod 0400) before digesting and parsing; every later read uses
the snapshot, so a concurrently modified original cannot mix into one
observation. The result reports the snapshot's byte size and digest.
- Retain failed snapshot-root cleanup in a provider set and retry it on
close(), throwing an aggregate error when a root is still owned; root
removal is injectable so the retention is testable.
- Allocate the AOT image buffer at exactly the declared library size
before reading (bounded before allocation, not concat-after-stream),
keep the 256 MiB ceiling, and check cancellation between digest and
stream chunks through a captured abort reason.
- Accept APK paths containing spaces (covered by a fixture test) and
escape the ABI pattern's dash for v-mode regex compatibility.
Also add tests for cleanup retention with close retry and for
cancellation before any snapshot root is created.
* docs(flutter): cover the Italian README with the Flutter row
The Italian translation landed between the merged feature PRs; its table
now carries every merged row plus this one, keeping its documentation-link
set identical to README.md.
* fix(flutter): make the ABI pattern portable for v-mode validators
Use a hex-escaped hyphen in the ABI character class, matching the form
the ADB portability fix uses, so clients compiling advertised schemas in
unicodeSets mode accept the inspect_dart_aot input schema.
* refactor(flutter): reuse shared harnesses and deduplicate the vertical
The duplicate-detection gate (jscpd, 2.5% threshold) tipped to 2.51% with
the flutter files added. Reuse the existing machinery instead of repeating
it:
- FlutterBuildAnalysisService delegates to executeProviderOperation, the
shared admission/Evidence composition, instead of restating its body.
- The CLI registers both commands from one table-driven spec.
- Boundary tests use the shared serveCli and assertAdvertisedMcpContracts
harnesses rather than hand-rolled equivalents.
- The provider test consolidates the four payload-shape cases into one
table-driven case; the service test uses one admission table and
distinct phrasings.
Duplicate lines fall from 2.51% to 2.48%, back under the gate.
* fix(flutter): address review findings from real-app verification
CassieFox23 verified the PR against three real F-Droid Flutter apps and
a control app, and found five gaps. All are fixed and re-verified against
cone (info.tangential.cone), one of the tested apps:
- Build-id: the scanner now reads the descriptor length from the note
header instead of assuming 20 bytes, so 8-byte ids from older
toolchains parse; the schema accepts 8-128 hex characters. Verified
against cone's readelf-reported ids on all three ABIs.
- Compiler line: the toolchain capture bound rises from 200 to 500
characters, keeping cone's 262-character clang line verbatim.
- Zero-byte libapp.so: the extraction target is created before streaming,
so an empty entry yields a clean unsupported_target refusal for
inspect_dart_aot and elf_header_valid: false with partial coverage for
identify_flutter_build, instead of a raw ENOENT with code UNKNOWN.
- Truncated library: identify_flutter_build marks fewer than two snapshot
sections as partial coverage with a limitation naming the count, and a
missing ELF magic gets its own limitation.
- Docs: the default ABI preference now matches the code exactly
(arm64-v8a, x86_64, x86, armeabi-v7a, then the rest alphabetically).
Also: every test that read the author's /tmp/opencode goldens now builds
a self-contained synthetic ELF64 image with dynsym, four snapshot
symbols, two snapshot sections, and a string pool, so the suite runs
anywhere; the verify lane now requires a present hex build-id per ABI
instead of accepting null.
* fix(flutter): address the Codex review findings
Eight findings, all applied:
- P1: SnapshotHashScanner keeps candidates in a Set (no quadratic
Array.includes) with a 64-distinct-hash cap, so a hostile payload of
magic+hash records cannot stall the scan or bloat the response.
- P1: projectStringPool stops admitting new distinct entries once each
category reaches twice its projection bound and reports
collectionCapped; allocation and sorting now track the response, not
the payload.
- P2: ZIP reader rejections during entry iteration or entry streaming
map into the typed error algebra (cancellation vs input refusal)
instead of escaping as unhandled exceptions in both operations.
- P2: magic_valid now marks coverage only for the *Data symbols; a
complete four-symbol image with valid data magics reports complete
coverage (the instruction symbols intentionally lack the magic).
- P2: the MCP server closes the Flutter provider during shutdown, so a
retained snapshot root is retried and reported, not leaked.
- P2: LabeledStringScanner commits matches only when a printable run is
delimiter-terminated; the carry grows to 600 bytes so a 500-character
toolchain line survives any chunk boundary untruncated.
- P2: both Flutter tool effects declare writesFilesystem: true; the
owned APK snapshot and extracted libapp.so are temporary writes.
- P2: the verify lane asserts every ABI agrees on exactly one snapshot
hash instead of discarding the computed set.
* fix(flutter): address the second Codex review round
Nine findings on the first-round fixes:
- P1: identify_flutter_build enforces a cumulative extraction budget
(768 MiB) across every library entry, not just per library, so an APK
with many ABIs cannot stream an unbounded total.
- P1: a single NUL-free printable run stops materializing at 1024 bytes
and marks the pool capped, so an adversarial one-run section cannot
allocate unbounded strings.
- P1: ELF cString bounds the NUL-terminator scan to the name maximum
plus one, so malformed symbol tables cannot scan the whole image per
entry.
- P2: mapReaderFailure distinguishes ArtifactReaderFailure texts —
budget/limit rejections map to resource constraints, integrity
failures (CRC/overlap/signature) to their own input refusal — instead
of collapsing all reader failures into one message.
- P2: the labeled-string carry is cleared when a delimiter commits a
run, so consecutive toolchain lines are never concatenated.
- P2: string-pool counts are renamed *_count_at_least and documented as
lower bounds, because capped collection discards values beyond twice
the projection bound.
- P2: inspect_dart_aot marks hash disagreements, non-pair source counts,
and candidate-cap saturation as partial coverage.
- P2: only valid lowercase-hex hash records count as snapshot sources;
complete magic records with non-hex fields do not inflate agreement.
- P2: the per-entry size refusal in inspect_dart_aot moves after ABI
selection, so an oversized unselected ABI no longer refuses a small
request.
* fix(flutter): address the third Codex review round
Five findings:
- P2: the owned snapshot revalidates the source after the copy (size,
mtime, inode), so an in-place replacement or swap mid-copy refuses
the observation instead of publishing mixed bytes.
- P2: a failed ZipArtifactReader close retains the snapshot root for
close() retry instead of silently swallowing the descriptor leak,
in both operations.
- P2: snapshot headers are parsed within their symbol bounds and a
symbol smaller than the 16-byte header reports magic_valid false,
so trailing-image bytes can no longer fake a valid data section.
- P2: the verify lane counts provider.close() cleanup failures as
lane failures instead of certifying a run with retained roots.
- P2: LabeledStringScanner reports toolchain_capped when further
distinct matching lines are observed past the 4-line projection;
identify marks such results partial with a naming limitation.
* fix(flutter): address rounds three and four of the Codex review
Round three (one missed finding) plus round four (five findings):
- P2: the GNU build-id note descriptor size is decoded from all four
bytes of the little-endian field, rejecting nonzero high bytes
instead of masking them to 16 bits.
- P1: the Flutter MCP handlers route through createContractToolHandler
and the session's analysisAdmission, so close_binary and snapshot
operations drain in-flight Flutter calls instead of racing them.
- P2: the owned-snapshot copy is a streamed, abort-checked write (no
copyFile), so cancellation stops the write promptly while the
source-identity revalidation (size, mtime, inode) still guards
against mid-copy replacement.
- P2: identify marks any snapshot_hash_sources value other than exactly
two as partial, matching the inspection path's pair requirement.
- P2: the verify lane refuses to run without --apk (exit 2) instead of
printing a false-green pass, and the engine check requires every
libapp-carrying ABI to have a present libflutter with a valid hex
build-id rather than filtering missing engines away vacuously.
* fix(flutter): address the fifth Codex review round
Six findings plus the upstream duplication baseline:
- P2: the flutter-provider kind reports available in the capability
inventory, so binary_session no longer lists always-registered
Flutter tools as provider_missing.
- P2: printable runs that exceed the 1024-byte cap are discarded at
their delimiter instead of emitting their truncated prefix as a
fabricated observation.
- P2: ELF symbols must fit inside the file-backed portion of their
PT_LOAD segment (p_filesz), so addresses in gaps or BSS are refused
instead of accepting unrelated bytes.
- P2: the build-id carry spans the complete 80-byte maximum note
(16-byte header + 64-byte descriptor) so chunk-boundary splits
cannot drop the first byte.
- P2: the labeled-string carry is sized for the toolchain pattern's
true 605-byte maximum, not the previous 600.
- P2: the AOT completeness predicate requires exactly two sources and
one candidate unconditionally, so zero-source images with valid data
magics report partial instead of complete.
Also: the shared ELF fixture builder and snapshot-section helper moved
to a dedicated FlutterTestFixtures module, eliminating cross-file test
duplication; jscpd now passes at 2.50% while main sits at 2.52%.
* fix(flutter): address the sixth Codex review round
Five findings:
- P2: snapshot-hash scanning is bounded to the exported data symbols'
byte ranges, so decoy or stale magic/hash sequences elsewhere in the
image cannot inflate source counts or contribute candidates.
- P2: labeled-string runs longer than the carry bound are rejected
instead of truncated, preventing a regex prefix from being published
as a verbatim source string.
- P2: the AOT image reread checks cancellation between file reads and
before the synchronous parsing stage, so a cancelled caller cannot
receive a success from a completed 256 MiB read.
- P2: the verify lane requires exactly two snapshot sources per ABI,
matching the provider's pair requirement, instead of accepting any
count >= 2.
- P2: ELF section and program header entry sizes are validated against
the class-defined minimum widths before walking either table, so
undersized entries cannot cause field reads to cross into
neighboring bytes.
* fix(flutter): address the seventh Codex review round
Six findings:
- P2: data-symbol ranges are deduplicated by offset and size, so aliased
symbols mapping to the same bytes are scanned once and the result
marks partial coverage with a limitation naming the shared range.
- P2: openEntrySafely forwards return/throw to the underlying Readable's
iterator, so an early for-await exit closes the ZIP entry stream and
releases its producer instead of leaking a backpressured pipe.
- P2: inspect_dart_aot now reports snapshot_hash_candidates and
snapshot_hash_sources alongside snapshot_hash, so a null hash from
conflicting evidence is distinguishable from missing hash data.
- P2: overlong printable runs only set toolchain_capped when they begin
with a recognized label prefix (Dart VM version: or Android (); other
long strings such as license text are skipped without marking partial.
- P2: readElfSymbol falls back to PT_DYNAMIC/DT_SYMTAB when section
headers are absent (e_shnum 0, ET_DYN), so stripped Flutter libraries
with exported snapshot symbols are no longer refused; the dynamic
table is bounded by DT_STRSZ, DT_SYMENT, and the image end.
- P2: the verify lane exercises inspect_dart_aot for every ABI carrying
libapp.so, not just the preferred default.
* fix(flutter): address the eighth Codex review round
Five findings:
- P2: each data-symbol range gets a fresh SnapshotHashScanner so the
carry from one noncontiguous slice cannot fabricate a record spanning
into the next; results are aggregated after scanning.
- P2: dynsym name lookups are bounded to the linked string table —
st_name offsets outside the table and terminators beyond it are
rejected instead of reading unrelated payload bytes as symbol names,
in both the section-header and PT_DYNAMIC paths.
- P2: identify_flutter_build routes its reader.entries and reader.open
calls through the same collectEntriesSafely and openEntrySafely
wrappers as inspect_dart_aot, so malformed ZIPs surface as input
refusals instead of AnalysisProtocolError.
- P2: mapReaderFailure recognizes ArtifactReaderFailure texts carrying
a cancelled reason, mapping them to AnalysisCancelledError instead
of AnalysisInputError.
- P2: the cumulative extraction cap is enforced inside the chunk loop,
not after each library completes, so refusal happens before the
remaining bytes are streamed and hashed.
* fix(flutter): harden streaming, sizes, and cancellation after self-review
Four issues found in a systematic self-audit across all bug classes
from eight Codex rounds:
- createReadStream in both the digest function and the owned-snapshot
copy now receives the AbortSignal directly, so cancellation aborts
the filesystem read at the source instead of waiting for the next
chunk to arrive on a slow filesystem.
- The owned snapshot enforces a 2 GiB APK-level size cap before any
copy I/O, so a path pointing at an enormous file is refused before
disk usage grows.
- identify_flutter_build preflights the sum of declared entry sizes
against the 768 MiB cumulative budget, refusing before any ZIP
extraction when the declared total already exceeds it.
- The LabeledStringScanner triple result() call is collapsed to a
single invocation (the previous code called result() three times in
the emission, relying on end() being idempotent).
No behavior change for valid inputs; all 36 tests, check:fast, and
the cone verify lane pass unchanged.
* fix(flutter): require structurally parseable ELF headers for complete coverage
CassieFox23's third corrupt-header case: an ELF with valid magic but
zeroed section/program header offsets passes the 4-byte magic check
and the byte scanner still finds hash patterns in the data sections,
so identify_flutter_build reports complete coverage for a structurally
broken file.
The elf_header_valid check now also verifies beyond the magic:
e_type must be a known value (ET_REL/ET_EXEC/ET_DYN), at least one
of e_shoff/e_phoff must be non-zero, and at least one of
e_shnum/e_phnum must be non-zero. A file with valid magic but zeroed
structure reports partial coverage with the existing ELF-magic
limitation.
* fix(flutter): validate ELF and snapshot coverage
Share bounded ELF header parsing between identification and inspection. Accumulate headers across ZIP chunks and validate table entry widths and extents before claiming complete identification coverage.
Bind AOT hashes to the exported VM and isolate headers, validate stored snapshot lengths including the magic bytes, and preserve partial evidence for missing, malformed, aliased, or overlapping data headers.
Preserve native stream cancellations and APK resource constraints across both operations. Bound snapshot copies while streaming and retain useful input-failure diagnostics and lower-bound count wording.
Validation: 72 focused tests; check:fast; check including formatting; docs:check; test discovery; jscpd; real cone APK and non-Flutter control on Linux. Verified CLI/MCP parity for valid and corrupted ELF headers and AOT inspection across three ABIs. Native Windows and macOS were not run.
* fix(flutter): unify artifact ownership and bound ELF evidence
* fix(flutter): preserve ELF provenance and partial coverage
Read GNU build IDs only from bounded ELF note ranges and retain conflicting candidates. Preserve source versus owned storage failures and honor cancellation through parsing and final cleanup.
Separate snapshot structure from string-pool coverage. Verify every Gallery ABI with digest-bound engine labels and URI counts, and document the supported snapshot-kind layout.
---------
Co-authored-by: REA Contributor <contributor@rea.tools>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Keep FAT container format distinct from selected slice architecture through target admission, Evidence, snapshots, and CLI/MCP lifecycle results. Preserve native, Hopper, Ghidra, extraction, and dylib routing for the Mach-O family.
Reject selected FAT ranges overlapping their table or too short for a Mach-O header. Replace empty-slice fixtures with in-bounds executable headers and cover thin, universal, single-slice, FAT64, and swapped containers.
Validation: static and architecture checks, generated docs, targeted producer and public MCP tests. Provider routing uses controlled fixtures; no live Hopper, IDA, JADX, or Ghidra analysis was performed for this change.
* fix(native): apply printed lipo capability bits to CPU subtypes
lipo -detailed_info prints a slice's capability byte separately from its
symbolic subtype: CPU_SUBTYPE_LIB64 for x86_64 and ppc64, and other bits
as hex. Only the arm64e pointer-auth form was applied, so a LIB64 x86_64
slice normalized to subtype 3 while its FAT entry carries 0x80000003,
and slice validation rejected the universal binary as an integrity
mismatch. Fold the printed capability byte into the subtype code.
* fix(native): leave lipo CPU subtypes unknown without capability bits
The lipo shipped with macOS 26 prints an empty capabilities line for
x86_64 slices whose FAT capability byte is 0x80 as well as 0x00, so a
symbolic subtype alone does not identify the slice's CPU subtype. The
parser assumed no capability bits and still reported subtype 3 for
LIB64 slices, which made universal applications such as Sublime Text
fail slice validation. Report the subtype code as unknown when lipo
prints no capability byte, as arm64e already does; slice validation
then relies on the remaining physical facts.
* test(native): cover printed and blank lipo capability bits
Fold CPU_SUBTYPE_LIB64 and hex capability bytes into x86_64 subtype
codes, and keep the code unknown for the blank capabilities line that
macOS 26 lipo prints.
* test(native): match a LIB64 x86_64 slice with blank lipo capabilities
Read a universal image whose x86_64 FAT entry carries 0x80000003 while
lipo reports CPU_SUBTYPE_X86_ALL with a blank capabilities line, as
macOS 26 lipo does, and expect both slices instead of an integrity
failure.
* fix(native): compare named lipo subtype bits without capability bytes
A blank capabilities line leaves the subtype code unknown, and slice
matching then skipped the subtype entirely. A FAT entry whose named
subtype differs, such as CPU_SUBTYPE_ARM64_V8 recorded by lipo as
CPU_SUBTYPE_ARM64_ALL, was accepted. Keep comparing the complete
subtype when lipo printed its capability byte, and otherwise compare
the subtype bits its record names.
* test(native): share LIB64 slice reader setup
Read the LIB64 universal fixture through one helper that takes the
x86_64 subtype lipo reports, so further cases reuse the same image and
lipo record. The existing expectation is unchanged.
* test(native): reject a blank-capability lipo subtype naming other bits
With the capabilities line blank, a lipo record naming
CPU_SUBTYPE_X86_64_H for the x86_64 LIB64 slice still disagrees with
the FAT entry's subtype bits and fails slice validation.
Give atomic Evidence and snapshot writes an explicit resource owner, preserve primary failures and completed publication receipts, and retry retained staging cleanup at the owning lifecycle. Verify inode identity before publishing or deleting; unknown path disappearance remains a cleanup failure. Keep snapshot admission reusable when a session closes and reopens.
Consolidate JavaScript source-scope classification so cancellation cannot fabricate failures for unselected files. Classify configuration failures as invalid input. Remove stale assertions and reuse shared test cleanup ownership.
Validation: 266 focused tests before final ownership tightening, 108 affected tests afterward, static and documentation checks, actual invalid-configuration CLI behavior, and isolated test temporary-directory hygiene.
Refs #1968
Reject malformed UTF-8 ZIP filenames at the decoding boundary while retaining CP437, Unicode Path overrides and leading U+FEFF. Preserve valid CustomAttribute parent and value facts when constructor identity is invalid, with an explicit partial-coverage issue.
Validated ZIP integrity and filesystem-provider workflows, managed producer cases, temporary-file hygiene, package and static checks. Removed the superseded producer-only invalid-constructor regression.
Wait for directory operations, close owned streams and traversal handles, and retain the mounted reader when cleanup fails so close can retry before detach. Consolidate stable-artifact filesystem regressions into the boundary suite.
Validation: 35 artifact tests and temporary-directory hygiene, including real hdiutil diagnostics. Directory-close and traversal races use injected faults.
Give snapshot capture, header admission, packed members, and container streams one file-handle owner. Serialize adoption with reader shutdown, retain failed closes for retry, and remove snapshots only after descriptor cleanup succeeds.
Cover unread members, pending opens, zero-byte members, and paired format/cleanup failures. Keep intentionally unrecoverable test fixtures hygienic and strengthen managed identity and process cleanup-authority expectations.
Validation: full npm run check; 99 targeted tests and temporary-directory hygiene; exact discovery of 877 test files. No real vendor-provider verification was performed.
Capture exact filesystem identity and nanosecond timestamps, validate descriptor/path state before and after streaming, and enforce the captured byte count. Bound growing files to their captured size plus one byte using precise positional reads; keep unrepresentable display sizes explicitly unknown.
Reuse the captured-state comparison around Apple assetutil and retain existing reader cleanup ownership. Validation: focused reader and affected consumer suites, full static/architecture checks, and a compiled directory-read probe on Bun 1.3.14 passed. Includes same-size mutation, growth, truncation, cancellation, FIFO replacement, cleanup retry, ZIP identity forgery, ASAR and Mach-O coverage.
Analyze exact selected sources in first-request order before unrelated
static scopes, preserving full artifact identity and byte verification.
Retain selection in CLI/MCP Evidence and summaries, with explicit unknowns
and partial coverage for unselected sources. Keep default full analysis.
Fixes#1848
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Host-file occurrences derived `executable` from POSIX execute bits, which
Windows never reports, so every Windows host file (including PE .exe
images classified as executable) reported `executable: false`.
Derive the fact once through hostExecutable: POSIX hosts keep the execute
bits; Windows marks regular files with the extensions it launches directly
(.exe, .com, .bat, .cmd), without reading the ambient PATHEXT.
Fixes#1819
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
* fix: order remaining outputs independently of host locale
String.prototype.localeCompare without a pinned collation follows the
host's ICU default locale, so the same input could be ordered differently
on different machines. 614fb768 moved canonical domain projections to
compareUnicodeCodePoints; provider, application, catalog, and setup
orderings still used locale collation.
Order every remaining source site by Unicode code point, including Mach-O
symbols, firmware extraction paths, ASAR entries (previously pinned to
"en" collation), evidence ID sets, browser and Electron target, script,
and resource inventories, reconstruction obligations, capability and
provider listings, and setup environment details.
The current tool and prompt catalog names order identically under both
comparators, so the catalog identity digest is unchanged.
* fix(artifacts): order ASAR entries by their portable path
@electron/asar lists host-spelled paths, so code-point order over the raw
spelling still differed by host: "a\b" sorts after "a0" on Windows while
"a/b" sorts before it on POSIX. Pair each listing with its portable
archive path once, order by that path, and keep the host spelling for
@electron/asar calls and adapter keys.
Bind each ZIP occurrence independently of its filename so engine metadata
inspection can read repeated resources without replacing retained sources.
Keep logical-path collision checks in artifact inventory.
Closes#1860
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Windows has no POSIX execute bit; Node reports file modes from the
read-only attribute alone, so every host file, including PE images,
reported `executable: false` while the same occurrence reported
`artifact_kind: executable`. Derive the host executable marking once:
POSIX permission bits stay authoritative, and Windows marks regular
files with the CreateProcess/cmd launchable extensions.
Fixes#1819
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(harmony): project HarmonyOS package inventory evidence
First HarmonyOS increment from #1431: HAP, HSP, and App Pack roots are
classified through the hardened ZIP reader and projected into an
execution-free application graph with packaging-model hints, exact
component paths and hashes, App Pack children by path, runtime-family
hints, and path-based N-API bridge hypotheses. HarmonyOS .har stays
unclassified by suffix because .har also names HTTP Archive JSON.
Adds the project_harmony_application_graph tool (CLI and MCP), synthetic
composition coverage, a pinned real-package verify lane, and docs.
* refactor(domain): share application graph projection scaffolding
The HarmonyOS projection mirrors the Android and Apple projections by
design, which pushed jscpd duplication over its 2.5 percent budget.
Extract the identical pieces (inventory input schema, projected
component schema and occurrence flattening, bridge candidate schema
factory, coverage assembly, comparator, canonical digest) into
domain/mobileApplicationGraphProjection.ts and route all three
projections plus the composition fixtures through it. Duplication drops
from 2.52 to 2.49 percent with no behavior change: projection ids,
classifications, and limitations are byte-identical.
* fix(harmony): address review, stable app-pack roots, and rotated fixture
- rebased onto main; the shared projection scaffolding now keeps main's
nativeRuntimeConvention loop and the harmony runtime families adopt the
same react-native/flutter/unity path conventions
- the .app App Pack suffix is now a root-input rule only (verified ZIP
magic in classifyRootFormat and BinaryTargetResolver); nested .app ZIP
members inside other packages keep the generic zip classification, with
a regression test, so existing inventories do not change
- the signing limitation now states that an empty signing list does not
mean unsigned because packaging-tool signatures live in the ZIP signing
block, and the app-pack root rule is disclosed as suffix-based
- repinned the real-package fixture to the current VHome 2.6.14-beta asset
after the upstream deleted v5.0.111-beta and replaced the v2.6.14-beta
asset within hours; the pin records both observed digests
---------
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Reject duplicate ZIP filenames before an emitted entry source lookup can be overwritten. Preserve valid repeated enumeration, case-distinct paths and typed inventory path failures.
Closes#1725. Related: #1480.
