mirror of
https://github.com/morluto/rea.git
synced 2026-10-26 17:35:11 +00:00
main
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
1a48a82020 |
feat(analysis)!: default function MCP output to retained summaries (#5660)
* feat(ghidra): bound one procedure's decompile without closing the session Fixes #5340 * fix(ghidra): report decompile limit as a request timeout Keeps BinarySession from marking every Ghidra operation unavailable after one procedure's decompile reaches the limit, and names the MCP tool-call deadline in the remediation. * fix(ghidra): budget the decompile limit and report cancellation-closed sessions Adds REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS to the derived Codex/Kimi tool deadline and forwards it through setup. Marks a remote failure that settles after caller cancellation so the timeout projection no longer promises session reuse. * feat(ghidra)!: default analyze_function to a retained summary analyze_function was the one analysis tool never wired into REA's detail: summary|complete convention, so it always returned the complete native dossier inline (0.2-10 MB, dominated by value-flow and references). Agents on tool-result-capped harnesses never reached the pseudocode. Add `detail` to analyze_function, defaulting to summary: return a compact dossier (procedure identity, pseudocode size, facet counts, value-flow availability) and retain the complete dossier Evidence in the session. Page pseudocode, references, assembly and value-flow through normalized_result.parent_evidence_id with inspect_analysis_view, which already projects every native facet. detail: complete returns the full dossier inline as before. Delivery detail never reaches the provider or Evidence identity. BREAKING CHANGE: analyze_function now defaults to the compact summary view instead of the complete dossier inline. Pass detail: "complete" for the previous behavior. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * feat(cli): expose analyze_function detail on the function command Wire `rea function` into the shared CLI detail-delivery convention (--detail, --evidence_output) so the CLI and MCP adapters offer the same summary | complete choice for analyze_function. Summary delivery writes the complete dossier Evidence to a bundle and returns the compact summary view; it fails before analysis when --evidence_output is missing, so no provider work is wasted. The CLI default stays complete because a one-shot command has no in-process Evidence retention. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * refactor(ghidra): harden analyze_function summary after independent review FlatlineRoundtable cross-model review (6 lanes) converged on the summary projection's availability semantics being underspecified. Document that facet_availability is authoritative over raw pseudocode length, that code_units are UTF-16 (the unit inspect_analysis_view pages by), and that value_flow.truncated is meaningful only when value-flow is available. Parenthesize the nullish-coalescing/comparison for clarity. No behavior change; declined claims (value-flow "truncated collapse", unresolved_calls dead code) were schema-impossible or already correct. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * test(ghidra): cover analyze_function detail wiring on both adapters Close the roundtable's top convergent finding: the summary path had only domain-projection coverage, not the server/CLI wiring where the breaking behavior lives. MCP server (registerEnhancedTools): the default delivers a retained summary whose complete parent is recorded for paging; delivery detail is stripped before the provider request; detail: complete returns the full dossier inline; a registration without retention rejects a summary request before any provider work. CLI (rea function): the default returns the complete dossier; --detail summary without --evidence_output fails before analysis; with the bundle sink it returns the compact summary and writes the complete Evidence bundle. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(ghidra): keep analyze_function delivery detail out of provider inputs Address the five automated review findings on #5660. P1: `detail` lived in the shared enhancedInputSchemas.analyze_function, so inspect_native_api composed `{procedure, detail: "summary"}` and the strict Ghidra analyze_function schema rejected it. Delivery detail is now owned by the analyze_function MCP tool contract; the handler separates it before any provider call, and executeFunctionAnalysisEvidence parses the provider input only. The MCP regression test validates provider input exactly as the production Ghidra client does and exercises inspect_native_api. P2: the analyze_function summary Evidence now carries the parent's relationship_coverage like native facet views, so partial or unknown caller/callee discovery is not reported as complete. P2: the summary reports the dossier's declared facet_availability, nulls the counts of facets the provider declared unavailable, and lists each reason in unknowns, so Kuna's empty references, comments and basic blocks are not mistaken for observed absence. P2: inspect-analysis-view evidence-file sources accept the Evidence bundle that --evidence_output writes, selecting a record with source.evidence_id when the bundle holds several; the CLI test reads the saved bundle back through that route. Tests that assert the complete dossier over public analyze_function now pass detail complete, and the oversized-dossier test asserts the summary view it previously expected to be unsupported. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(ghidra): keep omitted function facets unknown in the summary The analyze_function summary coerced an omitted unresolved_calls record to a count of 0 while the native facet view reports the same dossier as "not recorded", and it carried no counts at all for the pageable swift_string_literals facets. Count optional records as null when the dossier omits them, name them in unknowns, and add nullable counts for both Swift literal facets. The summary Evidence envelope also dropped the function address: createAnalysisViewEvidence now derives one function-view predicate for both the address location and relationship coverage. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(scripts): request complete analyze_function dossiers in real-provider checks The real-provider check scripts call analyze_function over MCP and then consume the complete dossier: requireFunctionDossier, functionDossierSchema, .procedure/.pseudocode reads, operation assertions, compare_functions inputs and the oversized-dossier rejection path. The summary default broke them. Every such call site now passes detail: "complete". Provider-direct callers are unchanged because detail is an MCP contract field; the admission-error sites stay unchanged because provider failures return before the summary branch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(ghidra): keep unavailable value-flow counts unknown in the summary summarizeDossier coerced the five value_flow_* counts to zero when native_value_flow was null (IDA, Kuna) or declared unavailable (Ghidra), so the summary reported empty collections while value_flow.available said otherwise. The counts are now null unless value flow is available, with the reason already named in unknowns. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(evidence): budget authentication headroom by file shape and serialized size The evidence-file headroom reserved two parsed copies, calibrated for one record, and charged strings at two bytes per character. Bundle authentication additionally holds the schema-parsed bundle, the re-parsed records and two JSON.stringify results, which expand control characters and lone surrogates sixfold: a bundle whose raw_result is a million NUL characters estimated at 2 MB parsed but needed 12 MB of serialized text. The budget now has a parsed term and a serialized term per file shape, owned by EvidenceAuthenticationHeadroom and shared by the analysis-view reader and the bundle importer, which had no guard. Two transient peaks the budget did not name are removed at their producers: the canonical digest hashes a large leaf directly instead of concatenating it into its buffer, and the JSON validator walks containers by cursor so a wide array no longer queues one entry per member. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(mcp): route analyze_function summary replies to dossier consumers in docs and prompts The MCP guide, the version-comparison and crash-triage prompt contracts, the authored skill and the Kuna, IDA and function-matching docs still told agents to pass analyze_function replies straight to compare_functions or build_call_path. Under the summary default those consumers reject the reply. They now name the retained-evidence reference for compare_functions, the observations fields of match_native_functions, and detail: "complete" for build_call_path, which accepts only inline dossiers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * test(evidence): pin validator member order and keep interrupted validation yielding Independent review of the cursor validator found the interrupted-validation fixture relied on the old last-first walk: its over-deep head was reached before the first cooperative yield. The padding now precedes the deep element so the walk yields first, which is the invariant the test guards. A new case pins first-defect-in-member-order reporting, skipped non-enumerable keys no longer count toward the yield cadence, and the import-headroom test loads the reader through a fresh module graph so it holds under the services project's shared workers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AoJZDPcrgmMsCqLG4LfiFb * fix(ghidra): align deadline bounds and cancellation outcomes * refactor(ghidra): shorten deadline recovery guidance * fix(mcp): clarify the scope of inferred decompile deadlines * test(ghidra): verify deadline workflows and document measured costs Separate failure and registration cases by their distinct contracts. Record actual Linux Ghidra, CLI and MCP evidence with measured context, elapsed time, CPU and memory, and explicit remaining coverage limits. * docs(ghidra): pin integration and distinguish baseline format failures * fix(ghidra): own failure lifecycle wording in the provider * test(ghidra): budget timeout fixture generation before allocation * fix(analysis): preserve summary provenance and bound bundle guidance * fix(ghidra): align capability and traversal deadline guidance * style(config): align deadline guidance formatting * docs(ghidra): record capability and traversal verification * docs(analysis): use absolute paths for Evidence files * docs(analysis): record completed function summary measurements * docs(analysis): update main integration validation --------- Co-authored-by: Syed Rishad <rishadzzzz@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
morluto
|
7d79a6a85b |
fix(verify): own asynchronous Ghidra prerequisite inspection
Await installation inspection through one shared helper with a real resource scope at all 13 verifier call sites. Retain exact close owners, complete cleanup, and preserve primary plus cleanup failures. Keep compiler-only loading lazy and use the namespace workflow's selected environment. Add admission contract and native-file lifecycle controls, plus a real script prerequisite rejection check. Controlled Java/header observations do not qualify a real Ghidra engine. Combined verification selection: 122 tests passed across five files; static/format/discovery passed; two scoped source reviews clear. Refs #4912 |
||
|
Aurelio Bernal
|
c462a4ebd3 |
fix(scripts): pass MCP callTool timeouts as request options (#2387)
@modelcontextprotocol/client 2.3.1 declares callTool(params, options).
Three verifiers still used the v1 shape callTool(params, undefined,
{ timeout }), so the third argument was dropped and those calls ran
under the SDK's 60 s default instead of the 150-180 s they request.
Pass the options second.
|
||
|
Aurelio Bernal
|
eaf88aca2a |
test(ghidra): require pending multi-HI16 coverage in the real MIPS relocatable lane (#1974)
* test(ghidra): require pending multi-HI16 coverage in the real MIPS relocatable lane Post-merge review on #1950: the relocation oracle exposes coverage.multiple_hi16, but the real-Ghidra lane never required it, and Clang's C output never produces a LO16 that resolves several pending HI16s. Ghidra's loader could therefore mishandle that stream while the lane still passed, although most GCC-built kernel modules checked contain it. The relocatable fixture adds an assembly block with two lui %hi(rea_mips_rel_carry); LLVM's relocation sort then leaves one HI16 pending until a later LO16 that resolves it together with another HI16, including the carry. The lane now requires multiple_hi16: false on the previous fixture, true for all three variants (LE, BE, soft-float module) on real Ghidra 12.1.4. * test(ghidra): make the MIPS multi-HI16 fixture independent of relocation sorting Review on #1974: the block used two %lo of rea_mips_rel_carry, a symbol with other %lo uses, so the relocation stream depended on the assembler's HI16/LO16 sort. Clang 21 happened to leave a HI16 pending; Clang 17 (per review) and GNU as 2.45 (checked here: HI@4 LO@8 HI@0 LO@10) pair each HI16 with its own LO16, so multiple_hi16 stays false and the lane fails. Use a dedicated symbol with two %hi and exactly one %lo. Any assembler that orders each HI16 before a matching LO16 must then emit both HI16s ahead of that single LO16: Clang 21 gives HI@0 HI@4 LO@8, GNU as 2.45 gives HI@4 HI@0 LO@8. The real-Ghidra lane passes for LE, BE and the soft-float module variant with multiple_hi16 true. |
||
|
Aurelio Bernalandmorluto
|
290d301ec7 |
feat(ghidra): admit relocatable MIPS32r2 ELF objects (#1950)
* feat(ghidra): admit soft-float MIPS32r2 ELF with integer-register float storage Soft-float o32 executables (Linux kernel builds, bare-metal firmware) were refused because their ABI flags record declares no CPR1 and FP ABI 3. Ghidra's MIPS32 compiler specifications model only hard-float o32, so typed float/double signatures (DWARF) would be bound to $f12/$f14/$f0 and the decompiler loses the real dataflow: `return sel ? a : b` decompiled to `return in_f0;`, and callers passed phantom FPU inputs. Admit ELF32 ET_EXEC o32 MIPS32r2 targets whose ABI record declares soft-float with no CPR1, and commit `mips_soft_float_storage: o32-integer-registers-v1` in their analysis profile only (hard-float profile digests are unchanged). ReaGhidraMipsSoftFloat.java runs after auto-analysis and, for each function with a float/double formal type, either keeps the types with the storage Ghidra's o32 model assigns to a same-size integer (pair alignment, stack spill), or, for auto-parameters such as hidden struct-return pointers, swaps float types for same-size integers under dynamic storage. No function is left on hard-float storage. The report is written last; the provider client requires it exactly when the profile commits the policy and refuses the session before any operation otherwise. Verification: real Ghidra 12.1.4 lane (verify:ghidra:mips) now builds hard- and soft-float fixtures in both byte orders and checks exact parameter/return storage through inspect_native_api, caller constants, snapshot profile and Evidence outcome; test:fast, verify:package and an installed-tarball soft-float run pass. * fix(ghidra): model soft-float MIPS as a prototype for every typed callable Review feedback on #1949: rewriting storage per Function left external functions and indirect calls through typed function pointers on Ghidra's hard-float o32 prototype, which still read $f12/$f14/$f0 while the session reported success. A negative control confirms it: without the change, `(*rea_mips_soft_callback)(8.5f, 9.5f, sel)` decompiles to `(*rea_mips_soft_callback)(in_f12, in_f14, sel)`. ReaGhidraMipsSoftFloat.java now derives a `__rea_soft_float` prototype from the program's own default o32 model (encoded, float pentries and float-consuming rules removed, so byte-order-specific aggregate rules and the hidden struct-return pointer are kept), installs it as a specification extension, and assigns it with dynamic storage to every non-thunk function, external function and function-pointer definition whose formal types use float or double. Thunks follow their targets. Float types are kept everywhere, so the integer-retype fallback is removed. The script fails without writing its report if the derived prototype or any reassigned function still uses an FPU register. The policy becomes `o32-soft-float-prototype-v1` with a {functions, definitions} report. The soft-float fixture adds an indirect call through a typed pointer; the real-Ghidra lane requires its constants with no FPU inputs and the struct return's `float scale` in $a1. * test(ghidra): pin ODDSPREG acceptance for soft-float MIPS ABI records Review on #1949 suggested requiring flags1 == 0 for soft-float, since MIPS_AFL_FLAGS1_ODDSPREG names CPR1 registers. GCC 5.2 kernel modules, LLVM/Rust firmware and Clang -msoft-float objects all record flags1 = 1 with FP ABI soft-float: it is the MIPS32 o32 toolchain default and selects nothing without CPR1. Keep accepting it, say why at the check, and test that soft-float with ODDSPREG is admitted while any other general flag is refused. * feat(ghidra): admit relocatable MIPS32r2 ELF objects Linux MIPS kernel modules (.ko) and other ELF32 o32 MIPS32r2 ET_REL objects were refused by the Ghidra MIPS lane. Ghidra's ELF loader already places each allocated section and applies REL relocations; this admits ET_REL alongside ET_EXEC (ET_DYN and PRX stay refused) under its own support lane, `elf32-rel-o32-arch32r2-standard-v1`, so executable targets keep theirs. Relocatable sessions carry limitations: addresses are analysis coordinates, undefined symbols live in Ghidra's EXTERNAL block, and HI16 relocations report PARTIAL because Ghidra applies them with their paired LO16. Verification: new `verify:ghidra:mips-relocatable` real-Ghidra lane builds -c -G0 objects in both byte orders plus a soft-float -mlong-calls module variant. GNU readelf supplies sections, symbols and REL entries; every relocation in an allocated section must encode S + A in the loaded word (S from Ghidra placement, A from original bytes), with HI16/LO16 checked as the effective address each LO16 forms, and coverage of a shared HI16, a LO16 carry, external code/data and R_MIPS_32 is asserted. The executable lane, unit suites pass, and on 30 real .ko files 54,972 relocations matched (93 reference empty sections and are not checkable). * fix(ghidra): verify every MIPS HI16 and archive-typed soft-float externals Review feedback on #1950: - The relocation oracle kept one pending HI16, so a second HI16 before the LO16 that resolves both overwrote the first, and unrelated relocations cleared it; the earlier site's high half and carry were never compared with S + A. Every HI16 now waits for the next LO16 of its symbol, which checks all of them; later LO16s reuse those HI16s, and a HI16 left unresolved at a section end fails. On 30 real soft-float kernel modules, 24 contain several HI16s per LO16; all 54,972 placed relocations match. - External callees with float prototypes: the relocatable fixture now calls powf, whose prototype Ghidra's C library archive applies by name. Every variant must decompile `powf(1.5,2.5)` without FPU inputs, and the soft-float module variant must report that __rea_soft_float covered it (with the external step removed it decompiles as `powf(param_1,param_2)`). `-fno-builtin` keeps the call from being folded. Also updates the module variant's expected soft-float policy to `o32-soft-float-prototype-v1`. --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |