Commit Graph
46 Commits
Author SHA1 Message Date
morluto 008931a178 refactor(hopper): move verifier assertions out of runtime 2026-10-24 11:19:45 +08:00
1a48a82020 feat(analysis)!: default function MCP output to retained summaries (#5660)
* feat(ghidra): bound one procedure's decompile without closing the session

Fixes #5340

* fix(ghidra): report decompile limit as a request timeout

Keeps BinarySession from marking every Ghidra operation unavailable after one procedure's decompile reaches the limit, and names the MCP tool-call deadline in the remediation.

* fix(ghidra): budget the decompile limit and report cancellation-closed sessions

Adds REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS to the derived Codex/Kimi tool deadline and forwards it through setup. Marks a remote failure that settles after caller cancellation so the timeout projection no longer promises session reuse.

* feat(ghidra)!: default analyze_function to a retained summary

analyze_function was the one analysis tool never wired into REA's
detail: summary|complete convention, so it always returned the complete
native dossier inline (0.2-10 MB, dominated by value-flow and
references). Agents on tool-result-capped harnesses never reached the
pseudocode.

Add `detail` to analyze_function, defaulting to summary: return a
compact dossier (procedure identity, pseudocode size, facet counts,
value-flow availability) and retain the complete dossier Evidence in
the session. Page pseudocode, references, assembly and value-flow
through normalized_result.parent_evidence_id with inspect_analysis_view,
which already projects every native facet. detail: complete returns the
full dossier inline as before. Delivery detail never reaches the
provider or Evidence identity.

BREAKING CHANGE: analyze_function now defaults to the compact summary
view instead of the complete dossier inline. Pass detail: "complete"
for the previous behavior.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* feat(cli): expose analyze_function detail on the function command

Wire `rea function` into the shared CLI detail-delivery convention
(--detail, --evidence_output) so the CLI and MCP adapters offer the same
summary | complete choice for analyze_function. Summary delivery writes
the complete dossier Evidence to a bundle and returns the compact
summary view; it fails before analysis when --evidence_output is
missing, so no provider work is wasted. The CLI default stays complete
because a one-shot command has no in-process Evidence retention.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* refactor(ghidra): harden analyze_function summary after independent review

FlatlineRoundtable cross-model review (6 lanes) converged on the summary
projection's availability semantics being underspecified. Document that
facet_availability is authoritative over raw pseudocode length, that
code_units are UTF-16 (the unit inspect_analysis_view pages by), and that
value_flow.truncated is meaningful only when value-flow is available.
Parenthesize the nullish-coalescing/comparison for clarity. No behavior
change; declined claims (value-flow "truncated collapse", unresolved_calls
dead code) were schema-impossible or already correct.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* test(ghidra): cover analyze_function detail wiring on both adapters

Close the roundtable's top convergent finding: the summary path had only
domain-projection coverage, not the server/CLI wiring where the breaking
behavior lives.

MCP server (registerEnhancedTools): the default delivers a retained
summary whose complete parent is recorded for paging; delivery detail is
stripped before the provider request; detail: complete returns the full
dossier inline; a registration without retention rejects a summary
request before any provider work.

CLI (rea function): the default returns the complete dossier; --detail
summary without --evidence_output fails before analysis; with the bundle
sink it returns the compact summary and writes the complete Evidence
bundle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(ghidra): keep analyze_function delivery detail out of provider inputs

Address the five automated review findings on #5660.

P1: `detail` lived in the shared enhancedInputSchemas.analyze_function, so
inspect_native_api composed `{procedure, detail: "summary"}` and the strict
Ghidra analyze_function schema rejected it. Delivery detail is now owned by
the analyze_function MCP tool contract; the handler separates it before any
provider call, and executeFunctionAnalysisEvidence parses the provider input
only. The MCP regression test validates provider input exactly as the
production Ghidra client does and exercises inspect_native_api.

P2: the analyze_function summary Evidence now carries the parent's
relationship_coverage like native facet views, so partial or unknown
caller/callee discovery is not reported as complete.

P2: the summary reports the dossier's declared facet_availability, nulls the
counts of facets the provider declared unavailable, and lists each reason in
unknowns, so Kuna's empty references, comments and basic blocks are not
mistaken for observed absence.

P2: inspect-analysis-view evidence-file sources accept the Evidence bundle
that --evidence_output writes, selecting a record with source.evidence_id
when the bundle holds several; the CLI test reads the saved bundle back
through that route.

Tests that assert the complete dossier over public analyze_function now pass
detail complete, and the oversized-dossier test asserts the summary view it
previously expected to be unsupported.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(ghidra): keep omitted function facets unknown in the summary

The analyze_function summary coerced an omitted unresolved_calls record to
a count of 0 while the native facet view reports the same dossier as "not
recorded", and it carried no counts at all for the pageable
swift_string_literals facets. Count optional records as null when the
dossier omits them, name them in unknowns, and add nullable counts for both
Swift literal facets. The summary Evidence envelope also dropped the
function address: createAnalysisViewEvidence now derives one function-view
predicate for both the address location and relationship coverage.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(scripts): request complete analyze_function dossiers in real-provider checks

The real-provider check scripts call analyze_function over MCP and then
consume the complete dossier: requireFunctionDossier, functionDossierSchema,
.procedure/.pseudocode reads, operation assertions, compare_functions inputs
and the oversized-dossier rejection path. The summary default broke them.
Every such call site now passes detail: "complete". Provider-direct callers
are unchanged because detail is an MCP contract field; the admission-error
sites stay unchanged because provider failures return before the summary
branch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(ghidra): keep unavailable value-flow counts unknown in the summary

summarizeDossier coerced the five value_flow_* counts to zero when
native_value_flow was null (IDA, Kuna) or declared unavailable (Ghidra),
so the summary reported empty collections while value_flow.available said
otherwise. The counts are now null unless value flow is available, with the
reason already named in unknowns.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(evidence): budget authentication headroom by file shape and serialized size

The evidence-file headroom reserved two parsed copies, calibrated for one
record, and charged strings at two bytes per character. Bundle
authentication additionally holds the schema-parsed bundle, the re-parsed
records and two JSON.stringify results, which expand control characters and
lone surrogates sixfold: a bundle whose raw_result is a million NUL
characters estimated at 2 MB parsed but needed 12 MB of serialized text.

The budget now has a parsed term and a serialized term per file shape, owned
by EvidenceAuthenticationHeadroom and shared by the analysis-view reader and
the bundle importer, which had no guard. Two transient peaks the budget did
not name are removed at their producers: the canonical digest hashes a large
leaf directly instead of concatenating it into its buffer, and the JSON
validator walks containers by cursor so a wide array no longer queues one
entry per member.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(mcp): route analyze_function summary replies to dossier consumers in docs and prompts

The MCP guide, the version-comparison and crash-triage prompt contracts,
the authored skill and the Kuna, IDA and function-matching docs still told
agents to pass analyze_function replies straight to compare_functions or
build_call_path. Under the summary default those consumers reject the reply.
They now name the retained-evidence reference for compare_functions, the
observations fields of match_native_functions, and detail: "complete" for
build_call_path, which accepts only inline dossiers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* test(evidence): pin validator member order and keep interrupted validation yielding

Independent review of the cursor validator found the interrupted-validation
fixture relied on the old last-first walk: its over-deep head was reached
before the first cooperative yield. The padding now precedes the deep
element so the walk yields first, which is the invariant the test guards.
A new case pins first-defect-in-member-order reporting, skipped
non-enumerable keys no longer count toward the yield cadence, and the
import-headroom test loads the reader through a fresh module graph so it
holds under the services project's shared workers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb

* fix(ghidra): align deadline bounds and cancellation outcomes

* refactor(ghidra): shorten deadline recovery guidance

* fix(mcp): clarify the scope of inferred decompile deadlines

* test(ghidra): verify deadline workflows and document measured costs

Separate failure and registration cases by their distinct contracts.
Record actual Linux Ghidra, CLI and MCP evidence with measured context,
elapsed time, CPU and memory, and explicit remaining coverage limits.

* docs(ghidra): pin integration and distinguish baseline format failures

* fix(ghidra): own failure lifecycle wording in the provider

* test(ghidra): budget timeout fixture generation before allocation

* fix(analysis): preserve summary provenance and bound bundle guidance

* fix(ghidra): align capability and traversal deadline guidance

* style(config): align deadline guidance formatting

* docs(ghidra): record capability and traversal verification

* docs(analysis): use absolute paths for Evidence files

* docs(analysis): record completed function summary measurements

* docs(analysis): update main integration validation

---------

Co-authored-by: Syed Rishad <rishadzzzz@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-24 08:18:53 +08:00
morluto b33f68deb7 refactor(providers): separate identities from workflow profiles 2026-10-23 23:14:14 +08:00
70cd4cbfb8 fix(hopper)!: verify current Linux releases with the publisher key
Replace fixed Linux Hopper release pins with current vendor feed metadata,
complete detached-PGP package verification under the stable publisher primary
key, and authenticated package evidence for offline admission. Share parsed
publisher facts across approved setup, doctor and the pre-execution guard.
Preserve existing ownership, cancellation, private-display and cleanup rules.

Fixes #3577

Normalize empty XDG settings, preserve backups during approved cache repair,
and retain publisher cleanup through cancellation and shutdown. Exercise cold
offline verification without reusing process-local proof state. Serialize
approved cache publication across processes, restore old evidence on failure,
and share the existing lease owner from process without changing its identities.

Validation: 438 focused tests (six platform-related skips), full static/documentation checks, real Ubuntu
24.04 official Hopper 6.6.0 setup/CLI/MCP, offline ordinary-user admission,
launcher mutation rejection, and cold/warm verification measurements. Official
RPM metadata mismatch is tracked separately in #3578 and fails closed.

BREAKING CHANGE: Older Linux Hopper builds admitted by the fixed allowlist
now require retained publisher-signed package evidence or an approved update
to the current public demo. Update explicit HOPPER_LAUNCHER_PATH values when
migrating; the live feed exposes only its current release.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-15 12:15:55 +08:00
morluto 02987d8e8c test(hopper): bind string assertions to observed source extents
Remove OS-based assumptions about long and non-ASCII string recognition. Require the dedicated fragmented fixture, independently verify full source bytes, and reject gaps, overlaps, incomplete reads and corrupted decoding. Report unrecognized native strings explicitly and retain the actual procedure count.

Verified through 14 verifier regressions and the complete macOS arm64 Hopper 6.6.0-demo CLI/MCP lane with cleanup.
2026-10-14 07:12:10 +08:00
Ysopwnable c105391db5 fix(hopper): stop Linux demo children when the REA owner exits (#2499)
Bind the owned demo helper to REA's private stdin stream and unwind child
cleanup on EOF during startup or analysis. Verify graceful and SIGKILL owner
exit through real CLI/MCP workflows and preserve unrelated processes.

Assisted-by: Reimu (gensokyo-skills:reimu-incident-triage)
2026-10-13 12:59:22 +08:00
YsopwnableandKaoru0822 6395e8d300 perf(mcp)!: compact repeated session discovery facts (#1988)
Group repeated session tool availability and provider capability facts, preserve complete named contracts and ordered limitations, and remove duplicate catalog descriptors. Update consumers and migration guidance.

Validated by all 31 CI jobs, focused source checks, and actual Claude Code/Linux Hopper/native Windows Ghidra small-target workflows on the current integration tree.

Refs #723.

BREAKING CHANGE: binary_session no longer includes server_identity.catalog.tools; tool_availability and provider capabilities use grouped facts documented in docs/mcp-contracts.md.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-12 07:40:19 +08:00
morluto ad59531a96 refactor: remove forwarding exports and redundant aliases 2026-10-09 22:46:09 +08:00
morluto 63bf91405c refactor!: enforce canonical evidence and provider boundaries
Deliver complete Evidence directly through MCP with required nullable analysis profiles and immutable per-server delivery settings. Bind selected environments at construction and require typed cleanup outcomes with retained retry ownership.

Remove test-only routing, duplicate session and snapshot metadata, producer-field repairs, and ambiguous provider arguments. Separate artifact byte identity from occurrence classification and retain typed source-map and partial native facts across CLI, MCP, and verifier consumers.

BREAKING CHANGE: obsolete Evidence envelopes, session aliases, incomplete snapshots, and missing normalized producer facts are no longer accepted.
2026-10-09 20:18:52 +08:00
morluto 2847311a57 fix(hopper): normalize native block endpoints and retain terminal evidence 2026-10-09 05:31:49 +08:00
morluto 7d7b3f961a fix(hopper): preserve typed string values and verify real searches 2026-10-09 05:12:36 +08:00
morluto 64717ad7da test(hopper): cover real boundary failures and CLI parity 2026-10-09 02:44:28 +08:00
Ysopwnable b743bd20a0 fix(hopper): verify lease cleanup and repair the macOS test lane (#1111)
* fix(hopper): distinguish shared lease roots from leaked sessions

* fix(ci): follow the moved process CLI regression tests
2026-10-09 02:41:34 +08:00
morluto 7005166f58 test(hopper): verify real boundaries and prune redundant happy paths 2026-10-09 02:05:58 +08:00
Ysopwnable 81b5104276 fix(analysis): bind overview document identity to the active target (#1094)
* fix(analysis): bind overview document identity to the active target

* fix(analysis): refresh legacy overview snapshot results
2026-10-09 00:57:28 +08:00
Alan VoandN0zoM1z0 5a67c7fc54 fix(hopper): diagnose MCP provider failures with stage, state, and request context (#1044)
* fix(hopper): diagnose MCP provider failures with stage, state, and request context

* fix(hopper): retain health at the actual provider lifecycle boundary

* test(hopper): distinguish socket loss from later observed process exit

* test(hopper): cover observed exit separately from socket-first replies

---------

Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-09 00:33:56 +08:00
morluto 405732a7f5 fix: correct native inventories and bridge error classification (#503)
* fix(hopper): preserve normalized inventory addresses

* test(hopper): isolate real inventory verification

* fix(native): preserve otool addresses and section alignment

* fix: preserve address spaces in call path evidence

* fix(hopper): distinguish request validation from provider faults

* fix(native): retain tool inventory facts and reject failed signatures
2026-10-04 17:25:09 +08:00
morluto 4fbc501121 feat(native): add inspection primitives, dispatch traces and approved UI observation (#500)
* fix(hopper): normalize string-backed addresses and update verification

* feat(native): add inspection primitives, dispatch traces and approved UI observation

* test: prioritize real artifact E2E and captured golden regressions

* fix(process): queue observed output before reactive rendering actions

* test: prune redundant cases and strengthen boundary assertions

* fix(cli): route native value tracing through shared workflows

* test: replace synthetic success with real workflows and captured data

* fix(catalog): separate portable metadata from native host availability
2026-10-04 16:02:36 +08:00
morluto 0d96275b68 refactor: return complete tool results inline 2026-09-29 07:08:46 +08:00
morluto 281e41f766 fix(verify): follow dynamic Hopper tool availability 2026-07-29 07:18:48 +00:00
morluto 4716fa2ece test: add critical provider workflow coverage 2026-07-25 15:12:51 +00:00
morluto 8e7709d35a feat(analysis): add bounded memory and call tracing 2026-07-24 13:14:01 +00:00
N0zoM1z0 e0bc7fd973 fix(hopper): add bounded analysis recovery 2026-07-24 00:33:29 +09:00
morluto e4ee4c9635 fix(process): protect unrelated Hopper during cleanup 2026-07-22 14:04:00 +00:00
morluto 513fa294a4 feat(process): record provider and verifier run lineage 2026-07-22 14:01:47 +00:00
N0zoM1z0 d25e98c3f9 feat: complete REA remediation program 2026-07-22 22:09:12 +09:00
N0zoM1z0 d7d6bdf474 fix: align session filters and Hopper verification 2026-07-20 17:33:25 +08:00
morluto da1da3bfb8 test(hopper): add semantic runtime conformance 2026-07-18 04:50:23 +00:00
morluto ee1cd40405 feat(mcp)!: harden contracts and evidence references
BREAKING CHANGE: comparison tools now require session-owned Evidence IDs or approved bundle paths, and structured Evidence results use compact references.
2026-07-16 19:49:58 +08:00
N0zoM1z0 8c67dbd49e feat(electron): reconcile static artifacts with passive runtime 2026-07-16 09:55:21 +09:00
N0zoM1z0 3fa2304bc7 feat(electron): map static process and IPC boundaries 2026-07-16 07:56:13 +09:00
N0zoM1z0 941d7101dd feat(session): add explicit provider registry and target binding
Introduce deterministic deep-provider discovery and immutable per-target routing while retaining disjoint artifact and native capability families.

Expose provider selection consistently through MCP, CLI, environment configuration, session status, prompt completion, typed diagnostics, and generated documentation. Preserve exact provider/profile provenance without request racing or transparent fallback.

Cover overlapping providers, ambiguity, rejection details, cancellation, restoration, package installation, and real Hopper target switching.
2026-07-15 22:13:48 +09:00
N0zoM1z0 be76f8090b feat: add web and Electron reverse-engineering workflows 2026-07-14 20:24:12 +09:00
N0zoM1z0 2c1ceabbf1 feat: add passive website reverse engineering 2026-07-14 12:27:13 +08:00
morluto 4c54ff6bca fix(contracts)!: return structured workflow failures
Replace prose-encoded partial failures in batch decompilation, call graphs, and named xref lookup with discriminated structured results.

BREAKING CHANGE: batch_decompile, get_call_graph, and find_xrefs_to_name now return structured discriminated output shapes.
2026-07-13 05:03:47 +00:00
morluto 0d36976277 test(verification): strengthen conformance and real-Hopper checks 2026-07-13 03:15:46 +00:00
morluto 993356b85a test: add package installation end-to-end coverage 2026-07-12 22:09:34 +00:00
morluto 4ff4c11a00 Add evidence-backed investigation workflows 2026-07-12 21:12:18 +00:00
morluto cc8681a856 feat: add evidence-backed process investigations 2026-07-12 20:36:00 +08:00
morluto 4f18e26ee5 feat: evolve REA analysis platform 2026-07-12 19:32:13 +08:00
morluto 1f235cb52f docs: document the 43-tool workflow 2026-07-12 19:00:03 +08:00
morluto b167bd7e13 fix(verify): tolerate exited Hopper helpers 2026-07-12 09:15:44 +08:00
morluto e32906afb7 chore(release): harden package artifacts 2026-07-12 08:45:28 +08:00
morluto ff321dc574 fix(hopper): launch analysis without stealing focus 2026-07-12 08:19:29 +08:00
morluto ee6d539a40 docs: document frictionless binary workflow 2026-07-12 06:34:36 +08:00
morluto af3831bb5b Add real-Hopper verification script 2026-07-12 03:25:30 +08:00