Preserve XML, OpenStep and binary dictionary keys at their format owners. Share validated binary reference facts with materialization while retaining scalar precision, UID identity and bounded expansion. Migrate archive, Interface Builder, native and app-bundle consumers, then remove obsolete key-omission accounting and schema copies.
Validated source regressions and compiled CLI/MCP workflows, including native plist fidelity and archive identity. Typecheck, lint, docs and test discovery pass. No performance benchmarks were required.
inspect_macho reported only the symbol-table spelling (`_main`), while
LLDB, dlsym and observe_native_calls use the runtime spelling (`main`).
Passing a static name as a breakpoint silently selected a different
function, so results from the two tools could not be joined by name.
Keep `name` as the exact symbol-table fact and add a derived
`runtime_name` to every export and import row, produced by one shared
Mach-O symbol-name owner. Managed P/Invoke verification now uses the
same owner instead of its own underscore stripping.
Fixes#5298
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Retain the selected catalog path with captured assetutil failures and malformed metadata. Map two explicit selected-path invalid-catalog diagnostics to the existing format error while preserving other failure classifications. Verify the observed diagnostic through the real CLI and stdio MCP using a truncated temporary catalog and an unchanged valid control.
Use one containing-procedure parser beside its declared schema for
navigation, address facets, and feature trace references. Keep the trace
cache and reference type bound to parsed facts, returning typed malformed
output before SDK projection instead of passing unchecked JSON onward.
Retain same-address search alternatives such as Kuna's distinct string
encodings. References identify target addresses, so observe and emit each
target once while retaining duplicate rows from the provider observation.
Remove address-only match collapse and redundant residual deduplication.
Document these semantics.
Validation: regression-first runs reproduced four malformed-resolution
failures and three alternative/reference-replay failures. Focused MCP and
consumer controls passed after repair. Real native engines were not run.
Use one lossless Mach-O name projection for symbols and dependency paths, retaining undecodable bytes and partial binding facts. Keep referenced NIB string failures unknown and preserve their source nodes through the shared lazy projection. Valid Unicode, readable sibling metadata and weak-bind scope remain usable.
Exercise the producer and CLI/MCP consumers, and align date-overlap and object-capacity fixtures with their owning archive contracts without relaxing admission or omission assertions.
Reuse filesystem-aware path selection for resource-only plist sessions so directory symlinks and trailing separators retain their meaning. Parse dyld bind names once as lossless text or exact bytes, preserving partial binding facts and independent valid metadata without invented class identities. Keep coverage issues separate from known opcode applicability and exercise both producer and generated Mach-O consumers.
Classify lipo parse failures once for native and artifact consumers, retain valid read-only ELF ranges independently of optional section names, and map IL2CPP reader architecture through the existing CPU identity owner before Mach-O parsing.
Regression-first checks reproduced malformed lipo output, lost unnamed-section compiler paths, and the amd64/x86_64 adapter mismatch. Focused parser, artifact, Rust CLI/MCP, Go, and IL2CPP controls passed; authored fixtures do not establish real-provider behavior.
Fixes#5982Fixes#3397
* feat(decompile): write function pseudocode to a directory and return a manifest
Add decompile_all and `rea decompile-all` so a caller can write one
provider pseudocode file per function, or a filtered set, and receive
only a small manifest. The result stays bounded when the program is
larger than a local model's tool-result cap.
Resume continues an export for the same target digest, provider,
analysis profile, and layout. Matching files are skipped, conflicting
files stay unchanged, and one function's failure does not stop the
rest. Cancellation leaves files already written. Requesting dwarf
layout reports that compilation-unit paths are unavailable.
Simulated-provider tests cover the manifest, resume, filters, limits,
cancellation, and the failure cap. MCP registration, CLI coverage,
schema validity, the product catalog, check:fast, and docs:check
passed. Real Ghidra was not exercised.
Refs #6090
* fix(decompile): keep unknown export identity unresumable and recover torn journals
Resume treated two missing target or profile digests as a match, so an
export of one binary could continue into another. Resume now requires both
digests, and a decompiler observation without the export's profile, or with
a different target, is recorded as profile_changed.
A crash during a journal append left a torn final line that made the export
unresumable. Text after the last newline is now treated as uncommitted and
truncated before the next append, and stale temporary files are removed.
An inventory row with an empty address aborted the whole export because its
failure record failed the journal schema. Such rows are now excluded and
counted in the limitations.
A limited resume retried the same recorded failures first and never reached
new functions. Functions not yet attempted now run before retries. File path
reservations come from every journal line, so an output_io record no longer
releases a path. The index is rebuilt from the journal in numeric address
order, and journal close and cleanup failures return a Result.
The cancellation test now cancels after a file is committed and checks that
file survives and is skipped on resume. Redundant snapshot-replay checks are
removed, the CLI limit uses the contract constant, and --snapshot no longer
claims decompile-all updates it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): refuse resume of a missing export and repair journals by byte offset
A second review round found that resume against a directory that does not
exist silently started a full new export and reported it as resumed. Resume
now requires an existing export directory.
The torn-line repair measured the committed journal from decoded text, so
damaged UTF-8 could move the truncation point past the end of the file. The
journal is now read as bytes and cut at the last newline byte, and an intact
journal is not truncated, so a concurrent append is kept.
A per-function observation for a different target is now recorded as
target_changed instead of profile_changed. A failure to remove a partly
created output directory is reported instead of hidden. The journal handle is
closed on every path again, including an unexpected throw. The path allocator
no longer gives up after 1,000 collisions and reuses a taken name.
Tests now use createTestTempDirectory, assert journal lines against the
journal schema, and cover resume of a missing directory, resume with an
unknown analysis profile, and a target-mismatched observation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): keep a function's first file name across failed retries
Reserving every journal path stopped other addresses from taking a path, but
an address looked up only its latest record. After its file went missing and
a retry failed without a path, the next resume moved the function to a
hash-suffixed name and orphaned the original. Each address now keeps the
first path the journal gave it.
A matched commit still ran the decompiler but did not count toward limit.
The limit now counts every function taken on after skipping verified files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): never follow a journal symlink and check procedure_info identity
Torn-line repair truncates the journal, and the journal was opened through
any symlink in the export directory, so repair could truncate a file outside
it. The journal must now be a regular file and is opened without following
symlinks where the platform supports it.
Body ranges from procedure_info were written without checking that the
observation belonged to this export or described the requested function. An
observation with a different provider, profile or target, or another entry
point, now leaves the body unavailable with that reason.
File, journal and index writes now use FileHandle.writeFile, which writes the
whole buffer instead of trusting one write call. The stale temporary sweep is
removed: a matching file name does not prove this export created the file.
The inverted-range test now uses one address space so it reaches the
inversion check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): bound returned caveats and keep verified checksums
Ghidra names each terminal call's address in its per-function caveats,
so decompile_all copied one new caveat per function into the returned
Evidence and the answer grew with the program. The returned Evidence
now keeps the eight most often observed distinct per-function caveats,
grouped by provider and text as they are rendered, plus a count of the
rest. Inventory caveats are unchanged, and other workflows are not
bounded. Each function keeps the caveats of the observations it used:
a written file in its header, now including procedure_info caveats,
and a failed attempt in its journal and index record. A procedure_info
observation for another target or profile is foreign and contributes
none.
A failure record for a function that was already written replaced the
record holding its checksum, so after a transient read error a resume
decompiled that function again. The journal now keeps the last
successful write separately from the latest attempt. Resume verifies
against that write and records it again once the file verifies.
A caller abort that arrives while the export finishes now returns
cancellation with the caller's reason instead of a success manifest,
as #6143 does for other late finalization. A publish, index or close
failure stays the primary error.
Also make the missing-journal resume error say what to do, and add
boundary tests: one runs decompile-all through the real CLI against a
fake Kuna to check flag mapping, the relative output directory,
filters and limit; one runs decompile_all through production MCP
registration to check that the caveat bound is applied.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): keep each function's complete caveats in the index
The returned Evidence keeps eight distinct per-function caveats and
counts the rest, pointing readers at the export. A written function's
file header truncates its caveats after 4,000 characters, and its
journal record kept none, so a large function with many terminal calls
could lose caveats from both the answer and the export.
Every journal record, written or failed, now keeps the complete caveats
of the observations its attempt used, and decompile-index.json carries
them. The header stays a readable summary and, when truncated, says
that the index keeps the complete list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): report external functions instead of failing them
Ghidra classifies imports as external functions with an empty body and
returns null pseudocode for them, as its limitations state. decompile_all
recorded each one as an invalid_pseudocode failure ("not text"), so a
dynamically linked program reported every import as a decompiler failure:
34 of 116 functions in /usr/bin/xxd.
procedure_info owns the external classification. When it classifies a
function as external, the export now records status "external", counts it
in counts.external, writes no file and requests no pseudocode. The record
keeps the body fact as observed. The journal header pins the target,
provider and profile, so a resume skips a recorded external function
without asking the provider again; a limit below the external count still
leaves nothing remaining. A null pseudocode result for a function whose
classification is not external is reported as no_pseudocode.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): report finalization failures after a cancelled export
When the caller's abort stopped the publish loop, decompile_all returned
that cancellation before checking the index write and the journal close,
which had already run. A failure to finalize the export was hidden, so
the caller could not tell the saved index or journal was incomplete.
Publish failures stay primary, then index and close failures. A
cancellation counts as the caller's only when the caller's signal is
aborted, and is then reported after a successful finalization with the
caller's own reason. A provider's own cancellation, without a caller
abort, stays a publish failure like any other. Tests cover an index
failure and a close failure after the caller stops the loop, a stopped
loop with a successful finalization keeping the caller's reason, and a
provider cancellation staying primary over an index failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): remove a created output directory that cannot be secured
createPrivateDirectory ran mkdir and chmod as one step. When mkdir
succeeded and chmod failed, it returned an error before #create reached
its cleanup, leaving an empty directory behind. A fresh export then
refused the existing directory, and a resume refused it for having no
journal, so the caller had to remove it by hand.
mkdir and chmod are now separate. A mkdir failure created nothing, so
nothing is removed. A chmod failure after this call created the
directory removes it with a non-recursive rmdir: it is still empty, so a
path another process filled or replaced meanwhile is refused rather than
deleted, and that refusal is reported.
A filesystem boundary test fails chmod for the new root and checks the
directory is gone and a retry succeeds. A module test confirms that a
function written earlier and now classified as external still reports a
changed file at its path as output_exists.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* refactor(process): share the socket lease with owners other than Hopper
The socket lease that keeps two REA processes from opening the same Hopper
document lived only in HopperLease, with Hopper error messages built in.
A decompile export needs the same guarantee for its output directory.
OwnedLease now holds the lease mechanism unchanged: a listening local
socket that the operating system ends with its process, owner identity
answered by the live holder, and recovery of a stale socket serialized
through a recovery directory. Callers map a tagged LeaseFailure to their
own error. HopperLease keeps its exports and maps each failure to the
same HopperStartError message as before. On Windows the lease is a named
pipe, which needs no stale recovery.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): hold the output directory per run and stop on repeated write failures
Two runs could resume the same output directory at once. Each read the
journal once, and the index each wrote reflected only its own view, so
the last writer could drop the other run's records. A run's torn-line
repair could also truncate the other run's append.
DecompileExport now holds a lease on the output directory, named by its
device and inode, from before the journal is read (or, for a new export,
from just after the directory is created and before its journal exists)
until the journal is closed. A second run is refused as a retryable
capacity constraint that reports the holder's process ID. The lease ends
with its process, so a crashed run leaves no lock to clear. The lease is
released on every open failure, and a failed release is reported.
An export also kept decompiling every remaining function when its files
had stopped being writable mid-run, for example on a full disk, failing
each one with output_io. After five consecutive functions whose files
could not be written or read, the export now stops with an output error
that carries the counts and the last failures. A verified file or any
other outcome ends a run of failures. Committed files and records stay,
the index is still written, and a later resume continues.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* docs(native): record directory export verification
Record matched real Ghidra outputs, bounded caveat and cleanup regressions,
actual catalog-inclusive context estimates, runtime measurements and coverage
limits for the pinned integration.
* fix(native): bound export summaries and reserve portable paths
Use one case-folded reservation owner for allocation and resumed journals.
Budget inline failures and provider caveats by encoded JSON bytes while
retaining complete records in the output index. Align the tool contract
with the collector order and verify SDK result-window behavior.
* docs(native): refresh final export measurements
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Use original object-table UIDs for inventory, hierarchy and connector references. Retain authored identity fields as data, refuse forged reference aliases and select UIKit top-level collections without substituting unrelated roots.
BREAKING CHANGE: Keyed archives that previously used authored objectID or object-id graph identities now use decimal table UIDs. Authored values remain attributes; ibtool and native NIBArchive identities retain their existing source semantics.
Fixes#6849Fixes#6850
Retain producer traversal uncertainty through inventory, extraction, Interface Builder decoding and executable-root discovery. Keep known copied files and decoded facts with honest coverage; share omission normalization and bound omitted archive paths with their source reasons.
Fixes#6852Fixes#6825Fixes#6582
Separate AMDGPU processors from feature-qualified note identities. Read Apple hidden ARM64 table rows once and preserve visibility through artifact, dylib and fuse consumers. Keep Objective-C++ template commas within type attributes and incomplete metadata unknown.
Validated provider/parser and production property regressions, Electron CLI/MCP consumers, and native Apple lipo -hideARM64 output through the production slice reader and public artifact inspection. LLVM execution remains unverified.
Fixes#6806Fixes#6799Fixes#6801
Reuse preflight-parsed container references for bounded decoder-copy normalization. Preserve original artifact bytes, scalar provenance and reference order. Verify three-byte offset and reference encodings through CLI and stdio MCP.
Fixes#6805
Use zip.js's parsed symlink fact for inventory and open refusal. Treat an
unmatched Godot trailer as an unconfirmed candidate while preserving failures
for confirmed malformed packs. Report entropy path-resolution failures at the
selected-input boundary.
Separate Go header discovery from inline string extents in ELF and Mach-O,
and apply the aggregate content allowance after validating real source ranges.
Normalize binary plist UID widths through the existing scalar-copy owner,
budget exact clone growth, preserve source identity, and refuse inexact UIDs.
Validation: 389 distinct tests passed, one Linux /proc test skipped on Darwin;
build, typecheck/lint, docs, single-thread formatting, and discovery (2142 files)
passed. CLI/MCP Go fixtures and public ZIP/Godot/keyed-archive workflows ran.
Real producer prevalence and external provider execution were not established.
Repeated ordinary keyed-archive CLI inventory facts and estimated reply tokens
were equal before/after. Warm median elapsed was 0.5166s / 0.5177s; no speed claim.
Fixes#6751Fixes#6763Fixes#6754Fixes#6767Fixes#6762Fixes#6779
morluto