feat(diagnostics): report a byte entropy profile for one local file (#2331)

* feat(diagnostics): report a byte entropy profile for one local file

The diff adds the MCP tool inspect_byte_entropy and the CLI command
inspect-byte-entropy, both implemented by ByteEntropyService. One
absolute regular file is read offline into observed Evidence: file-wide
SHA-256 and Shannon entropy, then fixed blocks and adjacent runs for an
optional window, using a power-of-two block size from 256 to 65536
(default 4096), with high at 7.2 or above and low at 1.0 or below. A
window past EOF, a missing path, or a non-file is an input error; a size
or identity change during the read is an artifact-changed error; empty
input reports entropy 0; cancellation is honored before and during the
read. Summary detail retains the complete series and returns file
entropy, counts, and the ten longest runs for inspect_analysis_view,
while the registered effect remains read-only Evidence and the
diagnostics docs record the operation and its errors.

* fix(diagnostics): flag zero-size files as not shown empty

Generated files such as Linux /proc entries report size 0 while holding
content, so inspect_byte_entropy read nothing and returned an empty-looking
report. A file that reports size 0 now carries a limitation that the result
does not show the file is empty. The result stays a success because the
tool reads the declared size, which the file-stability checks depend on.

* fix(mcp): reject unsupported diagnostic summaries before inspection

* fix(ci): repair integrated checks and Hopper test baseline

* test(entropy): use shared MCP schema verification after merge

* fix(catalog): list inspect_byte_entropy under its own provider

Adding inspect_byte_entropy to BINARY_DIAGNOSTICS_TOOL_CONTRACTS put it
under pwntools-elf in the generated product catalog, because the
pwntools group took every binary diagnostics tool except
inspect_pe_resources. The service reports provider byte-entropy.

Add a BYTE_ENTROPY_PROVIDER identity beside PE_RESOURCES_PROVIDER, use
it in ByteEntropyService, give it its own catalog group, and name the
pwntools tools (inspect_binary_layout, inspect_recorded_crash)
explicitly so a new diagnostics tool is not attributed to pwntools by
default. The catalog test pins both groups.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFTE4BHbGYzgf8Uhzcppdh

* fix(cli): resolve the byte-entropy path with absoluteCallerPath

main replaced the node:path resolve import in binaryDiagnosticsCommands.ts
with absoluteCallerPath (#4067), so the byte-entropy command's
resolve(args.path) no longer type-checks once the PR is merged with main.
Use absoluteCallerPath like the other commands, and add
src/filesystem/callerPath.ts matching main so the branch builds on its own.

* fix(diagnostics): retain entropy read cleanup ownership

* docs(diagnostics): explain entropy output capacity

* fix(diagnostics): preserve entropy cancellation outcomes

* fix(diagnostics): close the entropy input and warn on large reports

Public inspect_byte_entropy input now uses publicInputStrictObject before
detail is added, so unknown keys stay rejected on the advertised schema.
The tool description warns that complete output can become very large and
points callers to a shorter length or a larger block_size.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(diagnostics): keep byte-entropy registration mergeable with main

Register inspect_byte_entropy beside layout inspection and reject
unretained summaries before the file read, without editing workflows.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(diagnostics): register byte entropy with the shared tool options

Use toolRegistrationOptions so the handler typechecks on this branch and
on main, and still refuse unretained summaries before the file read.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>

docs/binary-diagnostics.md

@@ -215,3 +215,87 @@ Large Evidence envelopes may require a caller-configured MCP transport receive
 buffer because envelope copies add overhead. Requests honor cancellation during
 snapshot reading, directory traversal and payload hashing. The operation reads
 regular files only and rejects symbolic links.
+
+## Byte entropy triage
+
+`inspect_byte_entropy` / `inspect-byte-entropy` profiles any local regular file
+without an analysis engine, execution, extraction or network access.
+
+```sh
+rea inspect-byte-entropy ./firmware.bin --block-size 4096 --json
+rea inspect-byte-entropy ./firmware.bin --offset 4096 --length 65536 --json
+```
+
+```json
+{
+  "name": "inspect_byte_entropy",
+  "arguments": { "path": "/artifacts/firmware.bin", "detail": "summary" }
+}
+```
+
+Complete Evidence includes `artifact` (original path, byte size and SHA-256),
+`block_size`, `window`, `file_entropy`, `thresholds`, `blocks`, `runs` and
+`limitations`. For a 4096-byte zero-filled file, the entropy fields are:
+
+```json
+{
+  "block_size": 4096,
+  "window": { "offset": 0, "length": 4096 },
+  "file_entropy": 0,
+  "thresholds": { "high": 7.2, "low": 1 },
+  "blocks": [{ "offset": 0, "length": 4096, "entropy": 0 }],
+  "runs": [{ "offset": 0, "length": 4096, "classification": "low" }]
+}
+```
+
+Offsets and lengths are decimal byte counts. Blocks begin at the requested
+`offset` (default zero); omitted `length` extends to EOF. Explicit windows past
+EOF fail rather than silently clipping. The final block may be shorter.
+`block_size` must be a power of two from 256 through 65536, default 4096. A
+window may produce at most 32,768 block rows; larger requests fail before the
+file scan with a capacity error. Choose a shorter window or larger block size.
+SHA-256 and `file_entropy` always cover the entire file, including when a window
+is selected. Empty files and empty windows are accepted; empty entropy is zero
+by convention. The tool reads the size the file reports. A file that reports size
+0, such as a Linux `/proc` entry, returns no blocks with a limitation that the
+result does not show the file is empty. File identity, size and timestamps are checked across the read;
+detected changes produce an error rather than Evidence.
+
+Shannon entropy is reported in bits per byte from 0 to 8, rounded to four decimal
+places. Adjacent blocks with reported entropy at least 7.2 form `high` runs;
+blocks at most 1.0 form `low` runs. Other blocks interrupt runs. Entropy does not
+prove encryption, and this measure cannot distinguish encrypted from compressed
+data. Small blocks can underestimate entropy; the result is a triage clue.
+
+MCP `detail: "summary"` retains complete Evidence in the session and returns
+file entropy, block/run counts, thresholds, the selected window, and the ten
+longest runs (ties ordered by file offset). Use its `parent_evidence_id` to
+retrieve complete Evidence, or request its summary with `inspect_analysis_view`.
+The CLI returns complete Evidence through the same service. Summary detail
+requires a session capable of retaining Evidence.
+
+Reads use one block-sized buffer and a 256-entry histogram. Complete output
+can become very large because it includes one row per selected block, and
+summary delivery still retains that full list. Use a shorter length or a
+larger block size to reduce it. Full Evidence and its serialization require
+memory proportional to the number of selected blocks. The 32,768-row ceiling bounds the actual retained block objects and
+their serialized representation: a cap-sized run below produced 1,514,877
+serialized bytes and increased V8 heap use by 21.2 MiB on the measured host.
+The cap therefore bounds this output independently of file size; it is not a
+portable process-memory guarantee. Narrow the window or increase block size to
+reduce retained rows further.
+
+Local service measurement (macOS arm64, Node 24.18.0, 100,000,000-byte file,
+4096-byte blocks, file already in the OS cache): first process run took 1.113 s
+wall / 1.148 s CPU with 145.3 MiB peak RSS; a second run in the same process took
+1.015 s / 1.035 s CPU, with cumulative peak RSS 155.8 MiB. Each read 100,000,000
+bytes and produced 24,415 blocks. Measurements include Evidence construction,
+summary projection and JSON serialization, with no child processes. Complete
+Evidence serialized to 1,122,303 bytes and its summary to 2,705 bytes. A
+cap-sized 134,217,728-byte sparse file with 4096-byte blocks produced 32,768
+rows and 1,514,877 serialized bytes; V8 heap use increased by 21.2 MiB and RSS
+by 18.1 MiB in that process. This separate measurement used the same macOS
+arm64 host and Node version. Cold disk I/O, token counts and an agent
+investigation were not measured; there was no previous equivalent tool to
+benchmark. These figures do not establish a portable peak-memory bound,
+constant memory for retained Evidence or faster agent task completion.