* feat(decompile): write function pseudocode to a directory and return a manifest
Add decompile_all and `rea decompile-all` so a caller can write one
provider pseudocode file per function, or a filtered set, and receive
only a small manifest. The result stays bounded when the program is
larger than a local model's tool-result cap.
Resume continues an export for the same target digest, provider,
analysis profile, and layout. Matching files are skipped, conflicting
files stay unchanged, and one function's failure does not stop the
rest. Cancellation leaves files already written. Requesting dwarf
layout reports that compilation-unit paths are unavailable.
Simulated-provider tests cover the manifest, resume, filters, limits,
cancellation, and the failure cap. MCP registration, CLI coverage,
schema validity, the product catalog, check:fast, and docs:check
passed. Real Ghidra was not exercised.
Refs #6090
* fix(decompile): keep unknown export identity unresumable and recover torn journals
Resume treated two missing target or profile digests as a match, so an
export of one binary could continue into another. Resume now requires both
digests, and a decompiler observation without the export's profile, or with
a different target, is recorded as profile_changed.
A crash during a journal append left a torn final line that made the export
unresumable. Text after the last newline is now treated as uncommitted and
truncated before the next append, and stale temporary files are removed.
An inventory row with an empty address aborted the whole export because its
failure record failed the journal schema. Such rows are now excluded and
counted in the limitations.
A limited resume retried the same recorded failures first and never reached
new functions. Functions not yet attempted now run before retries. File path
reservations come from every journal line, so an output_io record no longer
releases a path. The index is rebuilt from the journal in numeric address
order, and journal close and cleanup failures return a Result.
The cancellation test now cancels after a file is committed and checks that
file survives and is skipped on resume. Redundant snapshot-replay checks are
removed, the CLI limit uses the contract constant, and --snapshot no longer
claims decompile-all updates it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): refuse resume of a missing export and repair journals by byte offset
A second review round found that resume against a directory that does not
exist silently started a full new export and reported it as resumed. Resume
now requires an existing export directory.
The torn-line repair measured the committed journal from decoded text, so
damaged UTF-8 could move the truncation point past the end of the file. The
journal is now read as bytes and cut at the last newline byte, and an intact
journal is not truncated, so a concurrent append is kept.
A per-function observation for a different target is now recorded as
target_changed instead of profile_changed. A failure to remove a partly
created output directory is reported instead of hidden. The journal handle is
closed on every path again, including an unexpected throw. The path allocator
no longer gives up after 1,000 collisions and reuses a taken name.
Tests now use createTestTempDirectory, assert journal lines against the
journal schema, and cover resume of a missing directory, resume with an
unknown analysis profile, and a target-mismatched observation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): keep a function's first file name across failed retries
Reserving every journal path stopped other addresses from taking a path, but
an address looked up only its latest record. After its file went missing and
a retry failed without a path, the next resume moved the function to a
hash-suffixed name and orphaned the original. Each address now keeps the
first path the journal gave it.
A matched commit still ran the decompiler but did not count toward limit.
The limit now counts every function taken on after skipping verified files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): never follow a journal symlink and check procedure_info identity
Torn-line repair truncates the journal, and the journal was opened through
any symlink in the export directory, so repair could truncate a file outside
it. The journal must now be a regular file and is opened without following
symlinks where the platform supports it.
Body ranges from procedure_info were written without checking that the
observation belonged to this export or described the requested function. An
observation with a different provider, profile or target, or another entry
point, now leaves the body unavailable with that reason.
File, journal and index writes now use FileHandle.writeFile, which writes the
whole buffer instead of trusting one write call. The stale temporary sweep is
removed: a matching file name does not prove this export created the file.
The inverted-range test now uses one address space so it reaches the
inversion check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): bound returned caveats and keep verified checksums
Ghidra names each terminal call's address in its per-function caveats,
so decompile_all copied one new caveat per function into the returned
Evidence and the answer grew with the program. The returned Evidence
now keeps the eight most often observed distinct per-function caveats,
grouped by provider and text as they are rendered, plus a count of the
rest. Inventory caveats are unchanged, and other workflows are not
bounded. Each function keeps the caveats of the observations it used:
a written file in its header, now including procedure_info caveats,
and a failed attempt in its journal and index record. A procedure_info
observation for another target or profile is foreign and contributes
none.
A failure record for a function that was already written replaced the
record holding its checksum, so after a transient read error a resume
decompiled that function again. The journal now keeps the last
successful write separately from the latest attempt. Resume verifies
against that write and records it again once the file verifies.
A caller abort that arrives while the export finishes now returns
cancellation with the caller's reason instead of a success manifest,
as #6143 does for other late finalization. A publish, index or close
failure stays the primary error.
Also make the missing-journal resume error say what to do, and add
boundary tests: one runs decompile-all through the real CLI against a
fake Kuna to check flag mapping, the relative output directory,
filters and limit; one runs decompile_all through production MCP
registration to check that the caveat bound is applied.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): keep each function's complete caveats in the index
The returned Evidence keeps eight distinct per-function caveats and
counts the rest, pointing readers at the export. A written function's
file header truncates its caveats after 4,000 characters, and its
journal record kept none, so a large function with many terminal calls
could lose caveats from both the answer and the export.
Every journal record, written or failed, now keeps the complete caveats
of the observations its attempt used, and decompile-index.json carries
them. The header stays a readable summary and, when truncated, says
that the index keeps the complete list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): report external functions instead of failing them
Ghidra classifies imports as external functions with an empty body and
returns null pseudocode for them, as its limitations state. decompile_all
recorded each one as an invalid_pseudocode failure ("not text"), so a
dynamically linked program reported every import as a decompiler failure:
34 of 116 functions in /usr/bin/xxd.
procedure_info owns the external classification. When it classifies a
function as external, the export now records status "external", counts it
in counts.external, writes no file and requests no pseudocode. The record
keeps the body fact as observed. The journal header pins the target,
provider and profile, so a resume skips a recorded external function
without asking the provider again; a limit below the external count still
leaves nothing remaining. A null pseudocode result for a function whose
classification is not external is reported as no_pseudocode.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): report finalization failures after a cancelled export
When the caller's abort stopped the publish loop, decompile_all returned
that cancellation before checking the index write and the journal close,
which had already run. A failure to finalize the export was hidden, so
the caller could not tell the saved index or journal was incomplete.
Publish failures stay primary, then index and close failures. A
cancellation counts as the caller's only when the caller's signal is
aborted, and is then reported after a successful finalization with the
caller's own reason. A provider's own cancellation, without a caller
abort, stays a publish failure like any other. Tests cover an index
failure and a close failure after the caller stops the loop, a stopped
loop with a successful finalization keeping the caller's reason, and a
provider cancellation staying primary over an index failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): remove a created output directory that cannot be secured
createPrivateDirectory ran mkdir and chmod as one step. When mkdir
succeeded and chmod failed, it returned an error before #create reached
its cleanup, leaving an empty directory behind. A fresh export then
refused the existing directory, and a resume refused it for having no
journal, so the caller had to remove it by hand.
mkdir and chmod are now separate. A mkdir failure created nothing, so
nothing is removed. A chmod failure after this call created the
directory removes it with a non-recursive rmdir: it is still empty, so a
path another process filled or replaced meanwhile is refused rather than
deleted, and that refusal is reported.
A filesystem boundary test fails chmod for the new root and checks the
directory is gone and a retry succeeds. A module test confirms that a
function written earlier and now classified as external still reports a
changed file at its path as output_exists.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* refactor(process): share the socket lease with owners other than Hopper
The socket lease that keeps two REA processes from opening the same Hopper
document lived only in HopperLease, with Hopper error messages built in.
A decompile export needs the same guarantee for its output directory.
OwnedLease now holds the lease mechanism unchanged: a listening local
socket that the operating system ends with its process, owner identity
answered by the live holder, and recovery of a stale socket serialized
through a recovery directory. Callers map a tagged LeaseFailure to their
own error. HopperLease keeps its exports and maps each failure to the
same HopperStartError message as before. On Windows the lease is a named
pipe, which needs no stale recovery.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* fix(decompile): hold the output directory per run and stop on repeated write failures
Two runs could resume the same output directory at once. Each read the
journal once, and the index each wrote reflected only its own view, so
the last writer could drop the other run's records. A run's torn-line
repair could also truncate the other run's append.
DecompileExport now holds a lease on the output directory, named by its
device and inode, from before the journal is read (or, for a new export,
from just after the directory is created and before its journal exists)
until the journal is closed. A second run is refused as a retryable
capacity constraint that reports the holder's process ID. The lease ends
with its process, so a crashed run leaves no lock to clear. The lease is
released on every open failure, and a failed release is reported.
An export also kept decompiling every remaining function when its files
had stopped being writable mid-run, for example on a full disk, failing
each one with output_io. After five consecutive functions whose files
could not be written or read, the export now stops with an output error
that carries the counts and the last failures. A verified file or any
other outcome ends a run of failures. Committed files and records stay,
the index is still written, and a later resume continues.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb
* docs(native): record directory export verification
Record matched real Ghidra outputs, bounded caveat and cleanup regressions,
actual catalog-inclusive context estimates, runtime measurements and coverage
limits for the pinned integration.
* fix(native): bound export summaries and reserve portable paths
Use one case-folded reservation owner for allocation and resumed journals.
Budget inline failures and provider caveats by encoded JSON bytes while
retaining complete records in the output index. Align the tool contract
with the collector order and verify SDK result-window behavior.
* docs(native): refresh final export measurements
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
* feat(diagnostics): report a byte entropy profile for one local file
The diff adds the MCP tool inspect_byte_entropy and the CLI command
inspect-byte-entropy, both implemented by ByteEntropyService. One
absolute regular file is read offline into observed Evidence: file-wide
SHA-256 and Shannon entropy, then fixed blocks and adjacent runs for an
optional window, using a power-of-two block size from 256 to 65536
(default 4096), with high at 7.2 or above and low at 1.0 or below. A
window past EOF, a missing path, or a non-file is an input error; a size
or identity change during the read is an artifact-changed error; empty
input reports entropy 0; cancellation is honored before and during the
read. Summary detail retains the complete series and returns file
entropy, counts, and the ten longest runs for inspect_analysis_view,
while the registered effect remains read-only Evidence and the
diagnostics docs record the operation and its errors.
* fix(diagnostics): flag zero-size files as not shown empty
Generated files such as Linux /proc entries report size 0 while holding
content, so inspect_byte_entropy read nothing and returned an empty-looking
report. A file that reports size 0 now carries a limitation that the result
does not show the file is empty. The result stays a success because the
tool reads the declared size, which the file-stability checks depend on.
* fix(mcp): reject unsupported diagnostic summaries before inspection
* fix(ci): repair integrated checks and Hopper test baseline
* test(entropy): use shared MCP schema verification after merge
* fix(catalog): list inspect_byte_entropy under its own provider
Adding inspect_byte_entropy to BINARY_DIAGNOSTICS_TOOL_CONTRACTS put it
under pwntools-elf in the generated product catalog, because the
pwntools group took every binary diagnostics tool except
inspect_pe_resources. The service reports provider byte-entropy.
Add a BYTE_ENTROPY_PROVIDER identity beside PE_RESOURCES_PROVIDER, use
it in ByteEntropyService, give it its own catalog group, and name the
pwntools tools (inspect_binary_layout, inspect_recorded_crash)
explicitly so a new diagnostics tool is not attributed to pwntools by
default. The catalog test pins both groups.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFTE4BHbGYzgf8Uhzcppdh
* fix(cli): resolve the byte-entropy path with absoluteCallerPath
main replaced the node:path resolve import in binaryDiagnosticsCommands.ts
with absoluteCallerPath (#4067), so the byte-entropy command's
resolve(args.path) no longer type-checks once the PR is merged with main.
Use absoluteCallerPath like the other commands, and add
src/filesystem/callerPath.ts matching main so the branch builds on its own.
* fix(diagnostics): retain entropy read cleanup ownership
* docs(diagnostics): explain entropy output capacity
* fix(diagnostics): preserve entropy cancellation outcomes
* fix(diagnostics): close the entropy input and warn on large reports
Public inspect_byte_entropy input now uses publicInputStrictObject before
detail is added, so unknown keys stay rejected on the advertised schema.
The tool description warns that complete output can become very large and
points callers to a shorter length or a larger block_size.
Co-authored-by: morluto <morluto@users.noreply.github.com>
* fix(diagnostics): keep byte-entropy registration mergeable with main
Register inspect_byte_entropy beside layout inspection and reject
unretained summaries before the file read, without editing workflows.
Co-authored-by: morluto <morluto@users.noreply.github.com>
* fix(diagnostics): register byte entropy with the shared tool options
Use toolRegistrationOptions so the handler typechecks on this branch and
on main, and still refuse unretained summaries before the file read.
Co-authored-by: morluto <morluto@users.noreply.github.com>
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
Add a read-only app resource session shared by CLI and MCP. Preserve resource-root provenance without inventing executable identity, reject binary-only snapshots and operations, and retain failed cleanup owners across switches and retries.
Keep filesystem admission failures typed and preserve cancellation ordering from observed cleanup facts. Document resource routing in the packaged investigation skill.
BREAKING CHANGE: resource-only observations may have a null Evidence subject and null executable target digests. Clients must use resource_root and reported per-resource provenance instead.
Refs #5589
* feat(native): search analyzed memory for masked byte patterns
The diff adds search_bytes to the CLI, the shared MCP contract, Ghidra
inventory, and the headless bridge, and advertises the same operation on
Hopper as unavailable until that provider is verified. Patterns are
hexadecimal byte pairs with optional ?? wildcards and an optional
equal-length bit mask; the TypeScript boundary and the bridge both
require a nonempty pattern, equal lengths, and at least one constrained
bit. Ghidra walks initialized blocks in address order within inclusive
bounds, keeps overlapping matches inside a single block, and returns
each hit with its segment, unique source file offset, and containing
procedure, continuing through a cursor bound to the target, profile,
pattern, mask, and bounds. Each page reports the inclusive ranges
examined and the skipped uninitialized blocks; cancellation returns a
typed error and discards the active Ghidra database.
address_to_file_offset now uses the same unique-offset helper, which
rejects a mapping that is missing, ambiguous, or larger than a safe
integer.
* fix(native): limit unbounded byte search to loaded memory
Without start or end, search_bytes walked every Ghidra block, including
non-loaded file parts such as .symtab and overlay blocks, which the docs
call out of scope. Unbounded searches now skip those blocks and list them
in searched.skipped_not_loaded; explicit bounds in their address space
still search them. Document that a match must end at or before end.
* fix(native): preserve implicit address spaces in range consumers
* fix(native): reject present malformed byte search state
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
* feat(evidence): discover retained records without full payloads
* fix(evidence): guide oversized results to retained discovery
* style(tests): format retained discovery recovery assertion
Add optional finalization_ms to keep observing a captured root after SIGTERM, record the actual finalizer exit and each signal attempt, and escalate once when needed. Bind the initial root identity to captured ownership and preserve observations through cleanup uncertainty.
Fixes#1253.
Co-authored-by: Mykola Dementiev <16836724+nikolay-dementiev@users.noreply.github.com>
Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com>
Preserve native document identity, nullable bookmark labels, address diagnostics, segment-end byte prefixes, and verified file mappings. Validate batch rename addresses before mutation.
BREAKING CHANGE: remove set_current_document. Use open_binary to switch targets; document arguments are scoped to the active target.
* fix(contracts)!: require absolute local paths for remaining caller-supplied file inputs
Follow-up to the session snapshot/export tightening: the same invisible server-cwd resolution applied to open_binary target paths (parseBinaryTarget(path, process.cwd())), import_evidence_bundle bundle reads, firmware input files (resolve(raw)), browser executable launch paths (resolve-then-spawn), Electron executable/application/root paths (realpath on raw input), and the expected server path check.
Each field gains the shared isAbsoluteLocalPath refinement plus a catalog-visible description, since JSON Schema cannot express the rule. Examples, scenario fixtures, and contract tests updated to absolute paths; new schema cases cover rejection/acceptance/omission per field.
Explicitly unchanged (verified rooted or out of scope): inspect_plist paths (resolved against the target bundle dir), firmware output_directory (already provider-enforced), filesystem observation paths (resolved against the caller-supplied working_directory), keyed-archive member paths (archive-relative), JS evidence member paths (evidence-relative under root_alias), provider-internal execution schemas, CLI-only doctor flows, and CLI options (own resolve against the operator cwd).
* fix(cli): resolve operator-relative paths before shared absolute-path validation
The contract tightening rejects relative paths, but three CLI commands forward operator-relative input into the shared schemas: firmware commands pass raw args to FirmwareAnalysisService, and the Electron/browser scenario commands validate parsed scenario JSON directly. CLI paths are relative to the operator shell, so resolve them against the working directory first: resolve() in firmwareCommands, and a shared resolveCliJsonPaths JSON key-path helper (cliJsonInput) applied in electronCommands and cliBrowserScenarioCommands. MCP behavior is unchanged — MCP callers must still send absolute paths.
Covers the regression with a relative-path firmware CLI acceptance test (asserts provider-stage capability_unavailable, not a validation error) and unit tests for the helper.
* feat(ghidra): edit function annotations atomically
Return annotation readback and refreshed function analysis through CLI and MCP. Own each database transaction, invalidate caches and immutable snapshots, preserve executable bytes, and retain the Windows mutation boundary.
Assisted-by: Nitori (gensokyo-skills:nitori-reverse-engineering)
* fix(ghidra): preserve qualified function annotation identity
Preserve namespaces during local name writes and match qualified readback to the refreshed dossier. Add a producer-shaped regression and verify a real namespaced C++ function through the CLI.
Assisted-by: Nitori (gensokyo-skills:nitori-reverse-engineering)
* test(ghidra): align verification with annotation authority
Keep strict capability and profile checks while admitting the explicit session metadata operation. Verify packaged transport authority and atomic transaction commitments and preserve a read-only Windows expectation.
Assisted-by: Nitori (gensokyo-skills:nitori-reverse-engineering)
* fix(hopper): report unavailable procedure body coverage
* refactor!: simplify runtime tools and remove replay engines
Keep direct inspection, capture, and comparison tools. Bind handlers to named contracts and let agents compose experiments with commands and local fixture servers.
BREAKING CHANGE: Remove controlled replay, Node characterization, finite replay-machine, managed runtime planning, and redundant string-tracing tools. Retire process replay, shims, reactive scenarios, custom checkpoints, and active browser replay and origin-scope inputs.
Add immutable session bundle snapshots, bounded resource discovery, paginated unknowns, mutable current analysis snapshots, and provider-neutral navigation/address context tools.
BREAKING CHANGE: export_evidence_bundle now requires a filesystem path and no longer returns an inline bundle.
BREAKING CHANGE: comparison tools now require session-owned Evidence IDs or approved bundle paths, and structured Evidence results use compact references.
