Files
reverse-engineering-anything/.github/workflows/ghidra-bridge-sources.yml
T
Akshad Jogiandmorluto ae25050dff feat(native): surface inferred prototype instruction conflicts (#8161)
* feat(native): surface inferred prototype instruction conflicts

- Add evidence-backed x86-32 stack-read and stdcall cleanup diagnostics in ReaGhidraBridge.java without rewriting signatures or function bodies.

- Carry prototype conflicts through native API schemas, analysis summaries and retained dossier views so CLI and MCP share the observations.

- Update Ghidra analysis profile digests, documentation and MCP boundary tests for the new diagnostic contract.

* fix(native): follow framed paths and retain zero cleanup conflicts

- Traverse decoded entry-frame paths in ReaGhidraBridge.java instead of stopping at an address-ordered early return, so later reachable reads remain visible.

- Detect explicit, implicit and partial EBP writes and use decoded operands with p-code loads, preventing stale-frame warnings and recognizing real Ghidra stack reads.

- Compare immediate return cleanup with inferred stdcall stack bytes even when either side is zero, and allow zero in the native API contract.

- Add real Ghidra instruction probes and MCP/CI regressions for the reported cases and document the bounded checks.

* fix(native): attribute frame reads before register writes

- Check a decoded stack read before invalidating EBP in ReaGhidraBridge.java so instructions that read and replace the frame still report the conflict.

- Match each candidate operand's address p-code to its LOAD pointer so a POP from ESP into an EBP destination cannot masquerade as an argument read.

- Cover both decoded instruction cases in ReaPrototypeConflictProbe.java and await verifier-owned scratch cleanup on success and failure.

* fix(ghidra): recompute raw profile digests after rebase

- Recompute the 6502 and disabled-analyzer raw-binary profile digests in GhidraRawProfileDigest.test.ts so the pinned values commit to the merged shared policy on current main.

* fix(native): locate the entry frame by address and own verifier cleanup

- Find PUSH EBP; MOV EBP, ESP from the function entry address in ReaGhidraBridge.java so an owned block below the entry cannot hide a reachable argument read.

- Run analyzeHeadless in its own process group and stop that group before removing scratch in verify-ghidra-prototype-conflicts.mjs, so the JVM and decompiler children are addressed as one owned family.

- Cover the lower-owned-block case in ReaPrototypeConflictProbe.java with an entry frame preceded by a decoded lower block.

* fix(native): size prototype fixture and own Ghidra cleanup

- Allocate the instruction fixture through the lower-block offset in verify-ghidra-prototype-conflicts.mjs so the entry frame and decoded block exist for the probe.

- Check the detached process group instead of the leader exit code before cleaning up, so lingering decompiler children receive SIGTERM and SIGKILL before scratch removal.

* fix(native): own Ghidra probe cleanup with shared process owner

- Use REA's ProviderProcessSupervisor and token-verified cleanup in verify-ghidra-prototype-conflicts.mjs so the JVM and descendants are stopped before scratch removal, including failed launches.

- Preserve the source-owned x86 fixture with an entry frame and lower block while asserting verified cleanup.

- Build the pinned verifier dependency graph in the Ghidra bridge CI job and extend its timeout before running the real probe.

* fix(native): end verifier waits on interrupt before cleanup

- Install SIGINT and SIGTERM handlers before spawning Ghidra and abort the wait, so Ctrl-C no longer leaves the detached process group running.

- Keep cleanup handlers installed until the shared process owner reports verified cleanup, then remove scratch only after the process family is stopped.

* fix(native): preserve verifier cleanup interruption errors

- Keep interrupt handlers installed through process cleanup and workspace deletion, then check cancellation before reporting success.

- Preserve and report the failing cleanup step instead of replacing it with an interruption or deleting a tree that may still be unsafe.

* fix(native): retain probe failures across verifier cleanup

- Record the probe error before cleanup so a late SIGINT or SIGTERM cannot replace its stack and captured Ghidra diagnostics.

- Prefer the earlier probe failure when cleanup also fails, while preserving cleanup diagnostics and temporary state for diagnosis.

* test(kuna): raise caller matrix verifier budget for parallel CI

- The multi-target verifier spawn used a 10s timeout while sibling verifier cases use 25s.

- Under 8-way sharded CI load this intermittently exceeded 10s and produced ETIMEDOUT, failing shard 1/8 and portable tests.

- Raise to 30s to match the existing verifier budget and avoid load flakes.

* ci: retrigger checks after transient Ghidra SDK download 403

- The ghidra_bridge lane failed in upstream's untouched SDK acquisition step: curl returned 403 with 0 bytes downloaded in 0.046s from the GitHub release CDN.

- Push an empty commit to start a fresh run because rerunning on the upstream repository requires admin rights.

* fix(native): keep the cleanup lease on combined verifier failures

- Run the prototype-conflict probe through withVerifierFixture so a probe failure and a failed release surface as one AggregateError that keeps the probe error as its cause and attaches the retained fixture as cleanupLeases, instead of throwing the probe error alone and dropping the cleanup owner.

- Report the failed cleanup step and retained project only when the fixture was not released, and keep interrupt handlers installed until release finishes.

* test(native): use shared MCP prototype test deadlines

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-11-19 01:55:13 +00:00

179 lines
10 KiB
YAML

name: Ghidra bridge source compilation
on:
workflow_call:
inputs:
build_cache_artifact:
description: Portable build cache artifact from the calling CI run.
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
permissions:
contents: read
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
jobs:
ghidra-bridge-sources:
name: Compile packaged Ghidra bridge sources
runs-on: ubuntu-24.04
timeout-minutes: 30
env:
GHIDRA_RELEASE_URL: https://github.com/NationalSecurityAgency/ghidra/releases/download/Ghidra_12.2_build/ghidra_12.2_PUBLIC_20261016.zip
GHIDRA_RELEASE_SHA256: 2c2cd776db3c1f6a097ba418922378ac3649cadd5e844a6f5f088ae628a7be67
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
- name: Validate the SDK pin against the provider's verified release
run: node scripts/verify-ghidra-bridge-sources.mjs --check-sdk-pin >> "$GITHUB_ENV"
# The bridge probe imports only source-owned ARM, Thumb and MIPS bytes and never
# executes target code. All SDK/project state stays in runner storage.
- name: Acquire checksum-verified Ghidra SDK in isolated runner storage
shell: bash
run: |
ghidra_root="${RUNNER_TEMP}/rea-ghidra"
mkdir -p "${ghidra_root}"
curl -fsSL --retry 3 --write-out 'ghidra_download_bytes=%{size_download} ghidra_download_seconds=%{time_total}\n' -o "${ghidra_root}/ghidra.zip" "${GHIDRA_RELEASE_URL}"
echo "${GHIDRA_RELEASE_SHA256} ${ghidra_root}/ghidra.zip" | sha256sum -c -
unzip -q "${ghidra_root}/ghidra.zip" -d "${ghidra_root}"
rm "${ghidra_root}/ghidra.zip"
properties="$(find "${ghidra_root}" -path '*/Ghidra/application.properties' -print -quit)"
test -n "${properties}"
echo "GHIDRA_INSTALL_DIR=$(dirname "$(dirname "${properties}")")" >> "${GITHUB_ENV}"
# JAVA_HOME_<major>_X64 comes from the runner image, not the env context.
- name: Compile bridge sources with the verified runner JDK
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
test -n "${!java_home_variable}"
JAVA_HOME="${!java_home_variable}" npm run verify:ghidra:bridge-sources -- --verified-sdk
- name: Install locked dependencies for loader relocation regression
run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- name: Verify loader relocation handling with the native decompiler
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
export JAVA_HOME="${!java_home_variable}"
npm run verify:ghidra:loader-relocation
- name: Check native prototype conflict decoding and frame paths
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
export JAVA_HOME="${!java_home_variable}"
node scripts/verify-ghidra-prototype-conflicts.mjs
- name: Inspect real ARM, Thumb and MIPS instruction context through the bridge
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
export JAVA_HOME="${!java_home_variable}"
probe_root="${RUNNER_TEMP}/rea-arm-context"
mkdir -p "${probe_root}/project"
printf '\036\377\057\341\160\107\000\000' > "${probe_root}/arm-thumb.bin"
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/project" ArmContext \
-import "${probe_root}/arm-thumb.bin" -loader BinaryLoader -processor ARM:LE:32:v7 \
-noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
-postScript ReaInstructionContextProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
cat "${probe_root}/output.log"
grep -F 'REA_ARM_CONTEXT_OK arm=0 thumb=1 unavailable=null (injected provider value)' "${probe_root}/output.log"
mkdir -p "${probe_root}/mips-project"
# MIPS32 addiu v0,a0,3; MIPS16 li v0,90. These bytes are never executed.
printf '\003\000\202\044\132\152\000\000' > "${probe_root}/mips.bin"
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/mips-project" MipsContext \
-import "${probe_root}/mips.bin" -loader BinaryLoader -processor MIPS:LE:32:default \
-noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
-postScript ReaInstructionContextProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" mips \
> "${probe_root}/mips-output.log" 2>&1 || { cat "${probe_root}/mips-output.log"; exit 1; }
cat "${probe_root}/mips-output.log"
grep -F 'REA_MIPS_CONTEXT_OK standard=0 mips16=1' "${probe_root}/mips-output.log"
# Each probe imports source-owned bytes at 0x1000, away from the Cortex-M
# vector symbols that ARMCortex.pspec defines at 0, and decodes them from
# the language's default context, so M-profile Thumb state must hold at
# the first byte. The bytes are never executed.
- name: Decode M-profile ARM and Xtensa bytes in their default context
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
export JAVA_HOME="${!java_home_variable}"
probe_root="${RUNNER_TEMP}/rea-decode"
mkdir -p "${probe_root}"
# mrs r0, msp; bx lr
printf '\357\363\010\200\160\107' > "${probe_root}/cortex.bin"
# mrs r0, msplim (ARMv8-M only); bx lr
printf '\357\363\012\200\160\107' > "${probe_root}/v8m.bin"
# entry a1, 32; retw.n
printf '\066\101\000\035\360' > "${probe_root}/xtensa.bin"
decode() {
local name="$1" language="$2"
shift 2
mkdir -p "${probe_root}/${name}-project"
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/${name}-project" Decode \
-import "${probe_root}/${name}.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
-processor "${language}" -noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra" \
-postScript ReaDecodeProbe.java "${language}" "$@" \
> "${probe_root}/${name}.log" 2>&1 || { cat "${probe_root}/${name}.log"; exit 1; }
cat "${probe_root}/${name}.log"
grep -F "REA_DECODE_OK ${language} $*" "${probe_root}/${name}.log"
}
decode cortex ARM:LE:32:Cortex mrs/4/msp bx/2/lr
decode v8m ARM:LE:32:v8-m mrs/4/msplim bx/2/lr
decode xtensa Xtensa:LE:32:default entry/3/a1 retw.n/2
# A call through a writable slot keeps Ghidra's recorded target as an
# observed callee while callee coverage stays partial; a direct call is
# the complete control. The source-owned x86-64 bytes are never executed.
- name: Keep computed-call targets non-exhaustive in callee coverage
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
export JAVA_HOME="${!java_home_variable}"
probe_root="${RUNNER_TEMP}/rea-computed-call"
mkdir -p "${probe_root}/project"
{
printf '\270\001\000\000\000\303\314\314' # 0x1000: mov eax, 1; ret
printf '\270\002\000\000\000\303\314\314' # 0x1008: mov eax, 2; ret
printf '\377\025\052\000\000\000\303\314' # 0x1010: call [rip + 0x2a]; ret
printf '\377\320\303\314\314\314\314\314' # 0x1018: call rax; ret
printf '\350\333\377\377\377\303\314\314' # 0x1020: call 0x1000; ret
printf '\314%.0s' $(seq 24) # 0x1028: int3 padding
printf '\000\020\000\000\000\000\000\000' # 0x1040: slot = 0x1000
} > "${probe_root}/computed-call.bin"
test "$(wc -c < "${probe_root}/computed-call.bin")" -eq 72
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/project" ComputedCall \
-import "${probe_root}/computed-call.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
-processor x86:LE:64:default -noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
-postScript ReaComputedCallCoverageProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
cat "${probe_root}/output.log"
grep -F 'REA_COMPUTED_CALL_COVERAGE_OK slot=observed-complete-discovery-unknown register=partial direct=complete callers=discovery-unknown' "${probe_root}/output.log"