Files
reverse-engineering-anything/tests/conformance/ghidra
8e2853dd7b fix(ghidra): prevent loader relocations from constant folding (#10483)
* fix(ghidra): prevent loader relocations from constant folding

Ghidra imports loader relocation destinations with their file bytes. A
destination in a non-writable loaded block can be populated by the
runtime loader instead, for example a copy-relocated musl `stderr` in
RELRO, so the decompiler folded the on-disk zero and showed
`fprintf(NULL, ...)`.

Before initializing the decompiler, mark each loaded, initialized,
non-writable block containing a relocation destination volatile in the
ephemeral database, and add a limitation to procedures that read such a
destination. ELF shared objects and PIEs are included; relocatable
objects and core files are not. Permissions and source FileBytes are
unchanged.

Sessions without database-mutation authority (Windows P0 and existing
caller projects) record the limitation but do not mark blocks volatile,
and the analysis profile versions the two behaviors separately.

A real Linux Ghidra ELF probe runs in the Ghidra bridge CI lane.

Fixes #4177

Squashed from #4198 and rebased onto main. Rebase adaptations: the
read-only gate, the verifier renamed to verify-ghidra-loader-relocation
so it stays out of the manual-verifier glob, docs moved to
docs/testing-lanes.md and docs/providers/ghidra-provider.md.

Co-Authored-By: gensokyo-dev <gensokyo-dev@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ghidra): await installation inspection in the relocation verifier

inspectGhidraInstallation became async on main after #4198's base, so the
verifier asserted on a pending Promise. Also wait for provider output to
close before reading it, like the sibling Ghidra verifiers, and keep the
workflow reference inside the docs site.

Co-Authored-By: gensokyo-dev <gensokyo-dev@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ghidra): address relocation review findings

- Version the existing-project analysis profile with the read-only
  relocation semantics, since those sessions now add the limitation.
- Inspect the installation through the verifier helper that owns the
  file-reader resources.
- Build the fixture with -fPIC: with -fPIE, GCC 12 emits a copy
  relocation of rea_external_value into writable .bss, so the probe's
  GOT check could not hold.
- Run probe cleanup through withVerifierCleanup so a cleanup failure
  does not hide the probe failure, and keep the workspace when owned
  Ghidra cleanup is incomplete.
- Link the workflow with a full GitHub URL, per the docs guide.

Co-Authored-By: gensokyo-dev <gensokyo-dev@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ghidra): reach the external data through the GOT with a weak extern

Building the whole fixture with -fPIC also turned the reader's direct
load of rea_loader_pointer into a GOT-relaxed lea plus two loads, and
the real-Ghidra lane then reported that the baseline no longer folds
the file constant. Keep -fPIE and declare only rea_external_value weak:
GCC and clang must reach a possibly absent symbol through a GOT entry,
which lands as R_X86_64_GLOB_DAT inside GNU_RELRO (checked with readelf
for GCC 12.2 and clang/lld).

Co-Authored-By: gensokyo-dev <gensokyo-dev@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ghidra): protect only relocations Ghidra could not apply

Running the real-Ghidra lane locally (Ghidra 12.1.4, JDK 21, GCC 12)
showed that marking every non-writable block that holds a relocation
volatile also marks full-RELRO .got. External calls then decompile as
opaque pointer calls (`(*pcVar2)(param_1)` through
`PTR_rea_external_...` instead of `rea_external(param_1)`), which the
probe's GOT-stability check caught.

Ghidra applies GLOB_DAT and RELATIVE relocations, so those bytes already
hold the loader's value and folding them is correct. The #4177 case is
a copy relocation, which Ghidra imports as UNSUPPORTED with the file's
zero in place. Protect only destinations whose relocation status is not
APPLIED or APPLIED_OTHER.

The fixture now reads a provider `const` object that the linker copies
into .data.rel.ro, the #4177 shape. The probe requires that relocation
to be unapplied and requires applied GOT relocations to stay
non-volatile. Ghidra 12.1.4 already shows this x86-64 read by name
without protection, so the probe records whether the baseline folds and
compares before and after only when it does. The profile value becomes
volatile-unapplied-relocation-blocks-v1.

Co-Authored-By: gensokyo-dev <gensokyo-dev@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Arslan Basharat <arslanbasharat414@gmail.com>
Co-authored-by: gensokyo-dev <gensokyo-dev@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 09:32:48 +01:00
..

Ghidra conformance corpus

These source-owned fixtures are compiled at verification time; generated binaries and Ghidra projects are never committed.

For the one-shot function bundle handoff, run npm run build:cached and then node scripts/verify-real-ghidra-function-bundle.mjs with GHIDRA_INSTALL_DIR and JAVA_HOME set. The verifier compiles its own target, creates a project with a synthetic history report, and exercises complete, summary, and no-overwrite CLI delivery. It checks the target and project files for changes.

  • ../c/riscv.c and riscv-probe.S produce source-owned RV32/RV64 ELF images with double-float ABI declarations and an independently known instruction. ../c/riscv-float-abi.c, riscv-probe.S and riscv-isa-probe.S produce typed and stripped RV32 ilp32 images whose calls cover every rule of the ilp32 policy; riscv-checks.mjs holds that policy's expected storage and the call-site oracle. npm run verify:ghidra:riscv exercises real CLI/MCP discovery, decompilation, argument storage, output schemas, profile and snapshot identity, admission and refusal diagnostics, immutable source bytes, and owned cleanup. It requires BYO Ghidra 12.1.4/JDK 21+ and RISC-V Clang/LLD/LLVM readelf/LLVM objdump; REA_RISCV_CLANG, REA_RISCV_LLD, REA_RISCV_READELF and REA_RISCV_OBJDUMP select existing tool commands.

  • ReaInstructionContextProbe.java inspects source-owned ARM, Thumb, MIPS32 and MIPS16 instructions through the production bridge in an ephemeral, read-only Ghidra project. It checks exact decoder-context register names, values and provenance; the ARM lane also injects an unavailable provider value. The ghidra-bridge-sources CI job runs both processor families. The input bytes are never executed.

  • ReaDecodeProbe.java decodes source-owned instruction bytes from a language's default context and checks each mnemonic, length and optional complete operand. The ghidra-bridge-sources CI job runs it for ARM:LE:32:Cortex, ARM:LE:32:v8-m and Xtensa:LE:32:default; the bytes are never executed.

  • ReaComputedCallCoverageProbe.java decodes source-owned x86-64 calls through a writable slot, a register and a direct target, injects only the slot call's recorded COMPUTED_CALL target, and reads callees through the production bridge. The recorded target stays an observed callee while callee coverage is partial and names the call site; the targetless call stays partial without that candidate limitation, and the direct call stays complete. The ghidra-bridge-sources CI job runs it; the bytes are never executed. It does not establish which references Ghidra's own analysis records for other slot shapes.

  • inventory.c produces separate x86-64 debug and stripped ELF executables. It covers imports, an external puts function, linker thunks, source symbols, direct calls, a targetless callback call, two referenced strings, and a multi-block branch.

  • entry-aliases.c preserves secondary imported entry labels, a bare-hex alias, and an interior label. The native CLI/MCP lane verifies entry selection, procedure search and feature tracing by the secondary name, annotation, interior-label rejection, and unchanged source bytes.

  • cross-format.c is freestanding so the verifier can produce AArch64 ELF, x86-64 PE, and x86-64 Mach-O targets from the same semantics. It preserves an exported entry, direct and indirect calls, a volatile string reference, and a multi-block branch across loaders.

  • relocation-symbol.c builds an x86-64 ELF with named and unnamed dynamic relocations. The cross-format verifier checks their Ghidra observations through the public CLI without executing the artifact.

  • no-return.c and ReaNoReturnProbe.java inject incorrect import flags into a real ELF or Mach-O analysis database, verify caller recovery, propagated flag repair and signature preservation, and retain genuine/unknown no-return calls and independently identified entries. Run npm run build:cached, then node scripts/verify-real-ghidra-noreturn.mjs with the Ghidra/JDK environment below on Linux or macOS x64/arm64. On Linux, use a procps-compatible process table that exposes REA's current process for owned cleanup. It also checks named limitations through the real read-only TCP bridge and provider Evidence projection; default Unix-socket transport, Windows, and the reported Amethyst artifact need separate verification. Fixture target code is never executed.

  • ReaDecompilerFailureProbe.java replaces the bridge decompiler after a real import of cross-format.c, so one function's decompilation fails. It checks that analyze_function still returns that function's assembly, callers, callees, references and basic blocks unchanged, marks pseudocode, native API and p-code value flow unavailable with the decompiler's reason, and that cancellation still ends the request. Run npm run build:cached, then node scripts/verify-real-ghidra-decompiler-failure.mjs on Linux or macOS x64/arm64. The lane also parses the degraded dossier with REA's Ghidra output contract and checks the healthy provider path through the real read-only TCP bridge. Decompiler failure marks only that observation as live: retain its Evidence, retry after backend recovery, and keep healthy dossier snapshot reuse. The injection does not establish which natural inputs make the decompiler fail.

  • node scripts/verify-real-ghidra-decompile-timeout.mjs (after npm run build:cached, Linux or macOS x64/arm64) compiles a generated shared library whose rea_slow function has thousands of branches and analyzes it with REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS=1 through the real read-only TCP provider. Ghidra's own decompiler stops at the limit with no injection: analyze_function returns the listing facets with pseudocode unavailable and a live cache policy, procedure_pseudo_code fails with a message naming the setting, and the same session then decompiles rea_small. Set REA_DECOMPILE_TIMEOUT_BRANCHES to change the fixture size. The verifier budgets 1 MiB for generated C source and derives the maximum branch count from the worst-case line size before allocating the source. Larger or invalid counts are rejected before installation inspection or compilation.

  • loader-relocation.c, loader-relocation-provider.c and ReaLoaderRelocationProbe.java exercise the production ReaGhidraBridge against a real imported Linux full-RELRO PIE ELF and a small shared library. The fixture's .data.rel.ro pointer is populated by an importer-created relocation. The probe proves that the ET_DYN program is marked relocatable by Ghidra but is still protected, checks the relocation byte range and read/write reference handling, and compares baseline/protected pseudocode. It also checks that a -fno-plt GOT caller retains identical pseudocode when its relocation blocks become volatile. Source FileBytes and block permissions must remain unchanged. npm run verify:ghidra:loader-relocation requires Linux, BYO Ghidra/JDK with its native decompiler, and a C compiler; the fixture is never executed.

scripts/verify-real-ghidra.mjs uses cc, clang, and lld-link by default; REA_CC, REA_CLANG, and REA_LLD_LINK can select alternate commands. The verifier requires the verified bring-your-own Ghidra 12.1.4 installation through GHIDRA_INSTALL_DIR and validates header classification before starting the provider. Provider admission accepts newer 12.x (>=12.1) builds with an unverified-build limitation; this lane does not.

Conformance compares semantic facts: provider/profile identity, function classification, resolved call edges, typed references, strings/xrefs, and CFG topology. It only requires provider pseudocode and assembly to be non-empty, bounded, and address-bearing; it never compares their text with Hopper. The callback fixture also proves that an unresolved targetless flow is not silently promoted to a direct callee. Every target runs in a separate owned project and must leave no process, socket, project, or runtime root after close.

relative-switch-signed.S covers signed halfword offsets, the CBZ zero-entry route, and selector biases of 0, 2 and 10 in the AArch64 jump-table lane. The nonzero biases exercise overlapping and disjoint typed label/index ranges. REA retains Ghidra's typed labels when the compared selector has an earlier register definition; additional labels remain unknown until their relationship to the typed switch selector can be proved. The oracle rejects extra numeric labels and verifies each retained target against the source instruction immediate.