mirror of
https://github.com/morluto/rea.git
synced 2026-11-19 21:23:45 +00:00
* feat(native): surface inferred prototype instruction conflicts - Add evidence-backed x86-32 stack-read and stdcall cleanup diagnostics in ReaGhidraBridge.java without rewriting signatures or function bodies. - Carry prototype conflicts through native API schemas, analysis summaries and retained dossier views so CLI and MCP share the observations. - Update Ghidra analysis profile digests, documentation and MCP boundary tests for the new diagnostic contract. * fix(native): follow framed paths and retain zero cleanup conflicts - Traverse decoded entry-frame paths in ReaGhidraBridge.java instead of stopping at an address-ordered early return, so later reachable reads remain visible. - Detect explicit, implicit and partial EBP writes and use decoded operands with p-code loads, preventing stale-frame warnings and recognizing real Ghidra stack reads. - Compare immediate return cleanup with inferred stdcall stack bytes even when either side is zero, and allow zero in the native API contract. - Add real Ghidra instruction probes and MCP/CI regressions for the reported cases and document the bounded checks. * fix(native): attribute frame reads before register writes - Check a decoded stack read before invalidating EBP in ReaGhidraBridge.java so instructions that read and replace the frame still report the conflict. - Match each candidate operand's address p-code to its LOAD pointer so a POP from ESP into an EBP destination cannot masquerade as an argument read. - Cover both decoded instruction cases in ReaPrototypeConflictProbe.java and await verifier-owned scratch cleanup on success and failure. * fix(ghidra): recompute raw profile digests after rebase - Recompute the 6502 and disabled-analyzer raw-binary profile digests in GhidraRawProfileDigest.test.ts so the pinned values commit to the merged shared policy on current main. * fix(native): locate the entry frame by address and own verifier cleanup - Find PUSH EBP; MOV EBP, ESP from the function entry address in ReaGhidraBridge.java so an owned block below the entry cannot hide a reachable argument read. - Run analyzeHeadless in its own process group and stop that group before removing scratch in verify-ghidra-prototype-conflicts.mjs, so the JVM and decompiler children are addressed as one owned family. - Cover the lower-owned-block case in ReaPrototypeConflictProbe.java with an entry frame preceded by a decoded lower block. * fix(native): size prototype fixture and own Ghidra cleanup - Allocate the instruction fixture through the lower-block offset in verify-ghidra-prototype-conflicts.mjs so the entry frame and decoded block exist for the probe. - Check the detached process group instead of the leader exit code before cleaning up, so lingering decompiler children receive SIGTERM and SIGKILL before scratch removal. * fix(native): own Ghidra probe cleanup with shared process owner - Use REA's ProviderProcessSupervisor and token-verified cleanup in verify-ghidra-prototype-conflicts.mjs so the JVM and descendants are stopped before scratch removal, including failed launches. - Preserve the source-owned x86 fixture with an entry frame and lower block while asserting verified cleanup. - Build the pinned verifier dependency graph in the Ghidra bridge CI job and extend its timeout before running the real probe. * fix(native): end verifier waits on interrupt before cleanup - Install SIGINT and SIGTERM handlers before spawning Ghidra and abort the wait, so Ctrl-C no longer leaves the detached process group running. - Keep cleanup handlers installed until the shared process owner reports verified cleanup, then remove scratch only after the process family is stopped. * fix(native): preserve verifier cleanup interruption errors - Keep interrupt handlers installed through process cleanup and workspace deletion, then check cancellation before reporting success. - Preserve and report the failing cleanup step instead of replacing it with an interruption or deleting a tree that may still be unsafe. * fix(native): retain probe failures across verifier cleanup - Record the probe error before cleanup so a late SIGINT or SIGTERM cannot replace its stack and captured Ghidra diagnostics. - Prefer the earlier probe failure when cleanup also fails, while preserving cleanup diagnostics and temporary state for diagnosis. * test(kuna): raise caller matrix verifier budget for parallel CI - The multi-target verifier spawn used a 10s timeout while sibling verifier cases use 25s. - Under 8-way sharded CI load this intermittently exceeded 10s and produced ETIMEDOUT, failing shard 1/8 and portable tests. - Raise to 30s to match the existing verifier budget and avoid load flakes. * ci: retrigger checks after transient Ghidra SDK download 403 - The ghidra_bridge lane failed in upstream's untouched SDK acquisition step: curl returned 403 with 0 bytes downloaded in 0.046s from the GitHub release CDN. - Push an empty commit to start a fresh run because rerunning on the upstream repository requires admin rights. * fix(native): keep the cleanup lease on combined verifier failures - Run the prototype-conflict probe through withVerifierFixture so a probe failure and a failed release surface as one AggregateError that keeps the probe error as its cause and attaches the retained fixture as cleanupLeases, instead of throwing the probe error alone and dropping the cleanup owner. - Report the failed cleanup step and retained project only when the fixture was not released, and keep interrupt handlers installed until release finishes. * test(native): use shared MCP prototype test deadlines --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
178 lines
6.3 KiB
JavaScript
178 lines
6.3 KiB
JavaScript
#!/usr/bin/env node
|
|
// Decode source-owned x86 bytes with the installed Ghidra, then run the bridge
|
|
// producer's frame/cleanup checks. The temporary project is isolated outside
|
|
// the checkout so failed verification cannot affect an analyst's database.
|
|
import assert from "node:assert/strict";
|
|
import { copyFile, mkdir, readdir, writeFile } from "node:fs/promises";
|
|
import { join, resolve } from "node:path";
|
|
|
|
import { cleanupOwnedProcessGroup } from "../dist/process/ProcessOwnership.js";
|
|
import {
|
|
ProviderProcessSupervisor,
|
|
spawnOwnedProviderProcess,
|
|
} from "../dist/process/ProviderProcess.js";
|
|
import { verifierFailureReasons } from "./lib/verifier-cleanup.mjs";
|
|
import {
|
|
VerifierFixture,
|
|
withVerifierFixture,
|
|
} from "./lib/verifier-fixture.mjs";
|
|
import { createVerifierRun } from "./lib/verifier-run.mjs";
|
|
|
|
const install = process.env.GHIDRA_INSTALL_DIR;
|
|
if (!install) throw new Error("Set GHIDRA_INSTALL_DIR to Ghidra 12.1.4");
|
|
const root = resolve(import.meta.dirname, "..");
|
|
const run = createVerifierRun();
|
|
|
|
// The fixture owns the workspace root: Ghidra's headless launcher writes the
|
|
// project beneath it, so it is a plain temporary root, and it is removed only
|
|
// after the owned process confirms cleanup — otherwise it is retained and
|
|
// named for diagnosis.
|
|
const fixture = await VerifierFixture.open({
|
|
prefix: "rea-prototype-conflicts-",
|
|
private: false,
|
|
});
|
|
const scratch = fixture.path;
|
|
const controller = new AbortController();
|
|
let interruptHandler;
|
|
const interrupted = new Promise((resolve) => {
|
|
interruptHandler = (signal) => {
|
|
controller.abort(new Error(`${signal} requested`));
|
|
resolve(false);
|
|
};
|
|
});
|
|
const handleSigint = () => interruptHandler("SIGINT");
|
|
const handleSigterm = () => interruptHandler("SIGTERM");
|
|
process.on("SIGINT", handleSigint);
|
|
process.on("SIGTERM", handleSigterm);
|
|
let supervisor;
|
|
let cleanupStep = "owned process cleanup";
|
|
fixture.releaseAfter(async () => {
|
|
if (supervisor !== undefined) {
|
|
const stopped = await supervisor.stop();
|
|
assert.equal(
|
|
stopped.status,
|
|
"verified-cleanup",
|
|
`Ghidra process cleanup could not be verified: ${
|
|
"reason" in stopped ? stopped.reason : String(stopped.status)
|
|
}`,
|
|
);
|
|
}
|
|
cleanupStep = "temporary workspace deletion";
|
|
});
|
|
const probe = async () => {
|
|
const project = join(scratch, "project");
|
|
await mkdir(project);
|
|
const scripts = join(scratch, "scripts");
|
|
await mkdir(scripts);
|
|
for (const name of await readdir(join(root, "bridge/ghidra"))) {
|
|
if (name.endsWith(".java"))
|
|
await copyFile(join(root, "bridge/ghidra", name), join(scripts, name));
|
|
}
|
|
await copyFile(
|
|
join(root, "tests/conformance/ghidra/ReaPrototypeConflictProbe.java"),
|
|
join(scripts, "ReaPrototypeConflictProbe.java"),
|
|
);
|
|
const image = Buffer.alloc(0x0b00, 0xcc);
|
|
for (const [offset, hex] of [
|
|
// JNE skips the early LEAVE; RET and reaches MOV ECX, [EBP + 8].
|
|
[0x000, "5589e583f8007502c9c38b4d08c9c3"],
|
|
[0x100, "5589e587e88b4d08c9c3"], // XCHG EAX, EBP
|
|
[0x200, "5589e56689c58b4d08c9c3"], // MOV BP, AX
|
|
[0x300, "5589e58b4d08c9c3"],
|
|
[0x400, "c20800"], // RET 8
|
|
[0x500, "c20000"], // RET 0
|
|
[0x600, "5589e58b6d088b4d08c9c3"], // MOV EBP,[EBP+8], then a stale-frame read
|
|
[0x700, "5589e58f4508c9c3"], // POP [EBP+8] reads [ESP], not the destination
|
|
[0x800, "5589e58b4d08c9c3"], // Entry frame whose owned block sits below it.
|
|
[0xa00, "90c3"], // A decoded lower block that precedes the entry in the scan.
|
|
]) {
|
|
Buffer.from(hex, "hex").copy(image, offset);
|
|
}
|
|
const imagePath = join(scratch, "instructions.bin");
|
|
await writeFile(imagePath, image);
|
|
const launch = await spawnOwnedProviderProcess({
|
|
command: join(install, "support/analyzeHeadless"),
|
|
arguments: [
|
|
project,
|
|
"PrototypeConflicts",
|
|
"-import",
|
|
imagePath,
|
|
"-loader",
|
|
"BinaryLoader",
|
|
"-loader-baseAddr",
|
|
"0x1000",
|
|
"-processor",
|
|
"x86:LE:32:default",
|
|
"-noanalysis",
|
|
"-readOnly",
|
|
"-scriptPath",
|
|
scripts,
|
|
"-postScript",
|
|
"ReaPrototypeConflictProbe.java",
|
|
],
|
|
runId: run.run_id,
|
|
expectedCommand: null,
|
|
cwd: scratch,
|
|
env: { JAVA_HOME: process.env.JAVA_HOME },
|
|
signal: controller.signal,
|
|
});
|
|
supervisor = new ProviderProcessSupervisor(
|
|
{
|
|
...launch,
|
|
ownsProcessLifetime: true,
|
|
cleanup: () => cleanupOwnedProcessGroup(launch.ownership),
|
|
},
|
|
{ maxDiagnosticBytes: 8 * 1024 * 1024 },
|
|
);
|
|
const exited = await Promise.race([
|
|
supervisor.waitForExit(180_000),
|
|
interrupted,
|
|
]);
|
|
if (controller.signal.aborted)
|
|
throw new Error(
|
|
"Ghidra prototype conflict probe interrupted by SIGINT/SIGTERM",
|
|
);
|
|
assert.ok(exited, "Ghidra prototype conflict probe timed out");
|
|
assert.ok(
|
|
await supervisor.waitForOutputClose(1_000),
|
|
"Ghidra prototype conflict probe output did not close",
|
|
);
|
|
const snapshot = supervisor.snapshot();
|
|
const output = `${snapshot.stdout.text}\n${snapshot.stderr.text}`;
|
|
assert.equal(snapshot.exitCode, 0, output);
|
|
assert.match(output, /REA_PROTOTYPE_CONFLICT_OK/u, output.slice(-6000));
|
|
};
|
|
|
|
try {
|
|
// The shared owner releases the fixture only after the registered cleanup
|
|
// confirms. A combined failure keeps the probe error as its cause and the
|
|
// retained root as its cleanup lease, so neither failure nor the owner of
|
|
// an unsafe tree is dropped from the error the caller receives.
|
|
await withVerifierFixture(fixture, probe);
|
|
} catch (cause) {
|
|
if (!fixture.released) {
|
|
// Retain cleanup diagnostics and possible evidence instead of replacing
|
|
// the cleanup failure with an interruption or deleting an unsafe tree.
|
|
console.error(
|
|
`Ghidra verifier failed during ${cleanupStep}: ${verifierFailureReasons(
|
|
cause,
|
|
).join("; ")}`,
|
|
);
|
|
if (cleanupStep !== "temporary workspace deletion")
|
|
console.error(
|
|
`Retained failed Ghidra verification project at ${scratch}`,
|
|
);
|
|
}
|
|
throw cause;
|
|
} finally {
|
|
// Keep cleanup handlers installed until release has finished, so an
|
|
// interrupt cannot terminate the verifier before its owner resumes.
|
|
process.off("SIGINT", handleSigint);
|
|
process.off("SIGTERM", handleSigterm);
|
|
}
|
|
if (controller.signal.aborted)
|
|
throw new Error(
|
|
"Ghidra prototype conflict probe interrupted by SIGINT/SIGTERM",
|
|
);
|
|
console.log("REA_PROTOTYPE_CONFLICT_OK");
|