mirror of
https://github.com/morluto/rea.git
synced 2026-11-19 21:23:45 +00:00
feat(ghidra)!: verify the bridge on Ghidra 12.2 (#12081)
* feat(ghidra)!: verify the bridge on Ghidra 12.2 Admit 12.x from minor 2 upward, with 12.2 and JDK 25 as the verified pair. The bridge now calls the four-argument storage API added in 12.2, so 12.1.x cannot compile it and is no longer accepted. BREAKING CHANGE: Ghidra 12.1.x is rejected. Use Ghidra 12.2 or another 12.x build at or above minor 2, and a JDK inside that installation's declared range (25 or newer for the verified 12.2 build). * docs(ghidra): point remaining lanes at Ghidra 12.2 The Windows, 6502, and m68k prerequisites still named 12.1.4 and JDK 21, which the verified-build check now rejects. The bridge compile lane names the 12.2 jars it actually compiles against. Co-authored-by: morluto <morluto@users.noreply.github.com> * test(ghidra): admit the JDK the verified install declares The Java home framing probe still reported JDK 21 while the verified installation fixture now declares a minimum of 25, so doctor admission failed before the path framing was checked. Co-authored-by: morluto <morluto@users.noreply.github.com> * test(ghidra): derive admitted install fixtures from the verified release Boundary hosts and assertions still named Ghidra 12.1.4 after admission moved to 12.2, so list_procedures evidence and completed property reads disagreed with the install the provider actually reported. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(ghidra): record the exact 12.2 Clang switch-recovery defect Ghidra 12.2 stays the verified build. Sessions and decompiler results on that exact version name upstream issue 9996 and the 12.2.1 fix commit. Other version strings, including 12.2.1, do not carry the limitation. Co-authored-by: morluto <morluto@users.noreply.github.com> * test(ghidra): check switch-defect facts instead of the warning sentence The policy test keeps the structured defect record and checks that the owning formatter delivers its id, commit, release, and scope fields. Co-authored-by: morluto <morluto@users.noreply.github.com> * test(browser): name the Playwright launch boundary the lifecycle catalog checks Main retitled that case with a platform placeholder. The coverage catalog still required the old literal, so the merged full suite reported a missing title. Co-authored-by: morluto <morluto@users.noreply.github.com> * test(browser): point the lifecycle catalog at main's launch title The merged suite uses main's platform-specific Playwright launch test. The coverage catalog now names that title, and this branch keeps main's test body so the merge stays clean. Co-authored-by: morluto <morluto@users.noreply.github.com> * test(browser): cover the platform launch title on this branch The lifecycle catalog names main's platform launch title. This branch now has that quoted test too, so the check passes before and after the merge. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(ghidra): record the exact 12.2 masked byte-switch defect Ghidra 12.2 leaves an AArch64 and-0xff byte-offset switch unrecovered. Exact 12.2 sessions now cite open upstream issue 10154, and the AArch64 lane expects that jump table to stay unrecovered. Co-authored-by: morluto <morluto@users.noreply.github.com> * test(browser): keep one platform launch environment check The second launch test repeated the selected environment check. The remaining test is the parameterized title, and the launcher forwards SYSTEMROOT on win32 so that assertion holds. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(ghidra): keep the open switch defect on admitted builds The exact-version check cleared ghidra-10154 on every build except 12.2. That defect stays on admitted releases until one is known to fix it. ghidra-9996 still clears after the exact 12.2 build. Co-authored-by: morluto <morluto@users.noreply.github.com> * style(docs): format the Ghidra testing-lane table The AArch64 jump-table row wrapped past the formatter width and failed static checks. Co-authored-by: morluto <morluto@users.noreply.github.com> * test(ghidra): give the masked switch its full byte range The unrecovered oracle now uses 256 readable table bytes, and the same object must still recover its compare-bounded switch. Co-authored-by: morluto <morluto@users.noreply.github.com> * docs(website): point Ghidra setup at 12.2 and JDK 25 The public setup pages still told new users to install Ghidra 12.1.4 and JDK 21, which this release rejects. Changed setup sentences are extracted and left untranslated so those editions use the English page. Co-authored-by: morluto <morluto@users.noreply.github.com> * docs(website): keep catalog edits to the changed setup strings The catalog check compares active message ids, not a full rewrite. Restore the previous catalogs and update only the Ghidra 12.2 setup strings, leaving their translations empty. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(ghidra): apply the closed switch defect from the parsed patch 12.2, 12.2.0, and 12.2.0 with a suffix all precede 12.2.1, so they keep ghidra-9996. The masked-switch table lists all 256 byte entries. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(ghidra): share the AArch64 switch fixture compiler invocation --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <morluto@users.noreply.github.com>
This commit is contained in:
78 files changed
+1156
-776
No files matched your search
@@ -11,6 +11,12 @@ import {
|
||||
} from "node:fs/promises";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import {
|
||||
SUPPORTED_GHIDRA_JAVA_MAJOR,
|
||||
SUPPORTED_GHIDRA_VERSION,
|
||||
ghidraReleaseAccepted,
|
||||
ghidraReleaseLine,
|
||||
} from "../src/ghidra/GhidraReleasePolicy.ts";
|
||||
|
||||
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
||||
const source = resolve(
|
||||
@@ -20,16 +26,16 @@ const source = resolve(
|
||||
const install = process.env.GHIDRA_INSTALL_DIR;
|
||||
if (!install)
|
||||
throw new Error(
|
||||
"NativeAOT build lane requires existing GHIDRA_INSTALL_DIR (Ghidra 12.1.x).",
|
||||
`NativeAOT build lane requires existing GHIDRA_INSTALL_DIR (Ghidra ${ghidraReleaseLine()}, verified with ${SUPPORTED_GHIDRA_VERSION}).`,
|
||||
);
|
||||
const properties = await readFile(
|
||||
join(install, "Ghidra/application.properties"),
|
||||
"utf8",
|
||||
);
|
||||
const ghidraVersion = propertyValue(properties, "application.version");
|
||||
if (!ghidraReleaseLineAccepted(ghidraVersion))
|
||||
if (!ghidraReleaseAccepted(ghidraVersion))
|
||||
throw new Error(
|
||||
`NativeAOT build lane accepts Ghidra 12.1.x (verified with 12.1.4). Observed application.version=${ghidraVersion ?? "missing"}.`,
|
||||
`NativeAOT build lane accepts Ghidra ${ghidraReleaseLine()} (verified with ${SUPPORTED_GHIDRA_VERSION}). Observed application.version=${ghidraVersion ?? "missing"}.`,
|
||||
);
|
||||
const output = resolve(
|
||||
process.env.REA_NATIVEAOT_BUILD_ROOT ??
|
||||
@@ -52,9 +58,9 @@ const run = (command, args) => {
|
||||
const javac = java ? join(java, "javac") : "javac";
|
||||
const javacBanner = capture(javac, ["-version"]);
|
||||
const javacMajor = Number(/^javac (\d+)/mu.exec(javacBanner)?.[1]);
|
||||
if (!Number.isInteger(javacMajor) || javacMajor < 21)
|
||||
if (!Number.isInteger(javacMajor) || javacMajor < SUPPORTED_GHIDRA_JAVA_MAJOR)
|
||||
throw new Error(
|
||||
`NativeAOT build lane requires javac 21 or newer and compiles with --release 21. Observed: ${javacBanner || "no version"}. REA does not install or upgrade Java.`,
|
||||
`NativeAOT build lane requires javac ${String(SUPPORTED_GHIDRA_JAVA_MAJOR)} or newer and compiles with --release ${String(SUPPORTED_GHIDRA_JAVA_MAJOR)}. Observed: ${javacBanner || "no version"}. REA does not install or upgrade Java.`,
|
||||
);
|
||||
const revision = run("git", ["-C", source, "rev-parse", "HEAD"]);
|
||||
if (
|
||||
@@ -99,7 +105,7 @@ try {
|
||||
"-J-Xmx512m",
|
||||
"-J-XX:ActiveProcessorCount=1",
|
||||
"--release",
|
||||
"21",
|
||||
String(SUPPORTED_GHIDRA_JAVA_MAJOR),
|
||||
`@${join(output, "javac.args")}`,
|
||||
]);
|
||||
await writeFile(
|
||||
@@ -157,12 +163,6 @@ function propertyValue(content, name) {
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Release line supported by the separately pinned NativeAOT integration. */
|
||||
function ghidraReleaseLineAccepted(raw) {
|
||||
const match = /^(\d+)\.(\d+)(?:\.(\d+))?([-+].*)?$/u.exec(raw ?? "");
|
||||
return match !== null && Number(match[1]) === 12 && Number(match[2]) === 1;
|
||||
}
|
||||
|
||||
function capture(command, args) {
|
||||
const result = spawnSync(command, args, {
|
||||
cwd: root,
|
||||
|
||||
@@ -139,6 +139,8 @@ await withVerifierFixture(scratchFixture, async () => {
|
||||
"UTF-8",
|
||||
"--release",
|
||||
String(SUPPORTED_GHIDRA_JAVA_MAJOR),
|
||||
"-Xlint:deprecation,removal",
|
||||
"-Werror",
|
||||
"-cp",
|
||||
jars.join(process.platform === "win32" ? ";" : ":"),
|
||||
"-d",
|
||||
|
||||
@@ -14,7 +14,7 @@ import {
|
||||
const installation = process.env.GHIDRA_INSTALL_DIR;
|
||||
if (!installation || !process.env.JAVA_HOME)
|
||||
throw new Error(
|
||||
"Set GHIDRA_INSTALL_DIR and JAVA_HOME to caller-installed Ghidra and JDK 21.",
|
||||
"Set GHIDRA_INSTALL_DIR and JAVA_HOME to caller-installed Ghidra 12.2 and JDK 25.",
|
||||
);
|
||||
const commands = await createVerifierCommands();
|
||||
const workspace = await VerifierFixture.open({
|
||||
|
||||
@@ -650,6 +650,71 @@ export async function verifyNativeTypeLayout(client, typeNames) {
|
||||
return { layout, payload, state, bits, base };
|
||||
}
|
||||
|
||||
/**
|
||||
* Oracle for masked-byte-switch.S. The masked table has 256 readable bytes,
|
||||
* the full `and #0xff` range. Ghidra 12.2 issue #10154 still leaves that
|
||||
* branch unrecovered. The compare-bounded switch in the same object must
|
||||
* recover, so a truncated table cannot explain the miss. A recovered masked
|
||||
* switch fails the lane so the limitation can be removed with the fix.
|
||||
*/
|
||||
export async function verifyMaskedByteSwitch(client) {
|
||||
const names = await inventoryCall(client, "list_names", {
|
||||
document: null,
|
||||
address: null,
|
||||
});
|
||||
const procedure = requireProcedure(names, "rea_masked_byte_switch");
|
||||
const dossier = await functionCall(client, "analyze_function", {
|
||||
procedure: procedure.address,
|
||||
});
|
||||
const pseudocode = await functionCall(client, "procedure_pseudo_code", {
|
||||
document: null,
|
||||
procedure: procedure.address,
|
||||
});
|
||||
const tables = dossier.native_api?.jump_tables ?? [];
|
||||
if (tables.length !== 0)
|
||||
throw new Error(
|
||||
`Masked byte switch recovered a jump table: ${JSON.stringify(tables)}`,
|
||||
);
|
||||
if (
|
||||
typeof pseudocode !== "string" ||
|
||||
!pseudocode.includes("Could not recover jumptable") ||
|
||||
!pseudocode.includes("Too many branches") ||
|
||||
pseudocode.includes("case 0:")
|
||||
)
|
||||
throw new Error(
|
||||
`Masked byte switch did not keep the unrecovered jump table: ${pseudocode}`,
|
||||
);
|
||||
const bounded = requireProcedure(names, "rea_bounded_byte_switch");
|
||||
const boundedDossier = await functionCall(client, "analyze_function", {
|
||||
procedure: bounded.address,
|
||||
});
|
||||
const boundedTable = boundedDossier.native_api?.jump_tables.find((item) =>
|
||||
item.data_sources.some(
|
||||
(source) => source.entry_size_bytes === 1 && source.entry_count === 4,
|
||||
),
|
||||
);
|
||||
if (boundedTable === undefined)
|
||||
throw new Error(
|
||||
`Compare-bounded byte switch did not recover: ${JSON.stringify(boundedDossier.native_api)}`,
|
||||
);
|
||||
for (let value = 0; value < 4; value += 1) {
|
||||
const mapping = boundedTable.mappings.find(
|
||||
(item) => item.case_value === value,
|
||||
);
|
||||
if (mapping === undefined || mapping.confidence !== "high")
|
||||
throw new Error(
|
||||
`Compare-bounded case ${value} was not verified: ${JSON.stringify(boundedTable)}`,
|
||||
);
|
||||
}
|
||||
return {
|
||||
address: procedure.address,
|
||||
jump_tables: tables.length,
|
||||
unrecovered: true,
|
||||
bounded_address: bounded.address,
|
||||
bounded_cases: boundedTable.mappings.length,
|
||||
};
|
||||
}
|
||||
|
||||
export async function verifyRelativeSwitch(client, procedures, entrySize) {
|
||||
// Relocatable loaders may choose an assembler-generated alias as the function name.
|
||||
const names = await inventoryCall(client, "list_names", {
|
||||
|
||||
@@ -16,6 +16,7 @@ import { parseAnalysisSnapshot } from "../dist/domain/analysisSnapshot.js";
|
||||
import { parseEvidence } from "../dist/domain/evidence.js";
|
||||
import { functionDossierSchema } from "../dist/domain/native/functionDossier.js";
|
||||
import { inspectVerifierGhidraInstallation } from "./lib/ghidra-installation-inspection.mjs";
|
||||
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraReleasePolicy.js";
|
||||
import {
|
||||
buildRiscvFixture,
|
||||
stripRiscvFixture,
|
||||
@@ -56,7 +57,7 @@ const installation = await inspectVerifierGhidraInstallation({
|
||||
: { javaHome: process.env.JAVA_HOME }),
|
||||
});
|
||||
assert.equal(installation.status, "available", JSON.stringify(installation));
|
||||
assert.equal(installation.providerVersion, "12.1.4");
|
||||
assert.equal(installation.providerVersion, SUPPORTED_GHIDRA_VERSION);
|
||||
|
||||
const execute = promisify(execFile);
|
||||
const entrypoint = fileURLToPath(new URL("./rea.mjs", import.meta.url));
|
||||
|
||||
@@ -30,7 +30,7 @@ const installation = await inspectVerifierGhidraInstallation({
|
||||
});
|
||||
if (process.platform !== "darwin" || installation.status !== "available")
|
||||
throw new Error(
|
||||
"verify:ghidra:swift requires macOS, Ghidra 12.1.4 with its native decompiler, and JDK 21; set GHIDRA_INSTALL_DIR and JAVA_HOME.",
|
||||
"verify:ghidra:swift requires macOS, Ghidra 12.2 with its native decompiler, and JDK 25; set GHIDRA_INSTALL_DIR and JAVA_HOME.",
|
||||
);
|
||||
try {
|
||||
await execute("swiftc", ["--version"], { timeout: 10_000 });
|
||||
|
||||
@@ -57,7 +57,7 @@ const installDir = process.env.GHIDRA_INSTALL_DIR;
|
||||
const { clientOptions } = resolveGhidraVerifierTimeouts(process.env);
|
||||
if (installDir === undefined || !isAbsolute(installDir))
|
||||
throw new Error(
|
||||
"Set GHIDRA_INSTALL_DIR to the absolute root of Ghidra 12.1.4.",
|
||||
`Set GHIDRA_INSTALL_DIR to the absolute root of Ghidra ${SUPPORTED_GHIDRA_VERSION}.`,
|
||||
);
|
||||
const installation = await inspectVerifierGhidraInstallation({
|
||||
environment: process.env,
|
||||
|
||||
@@ -43,6 +43,7 @@ import { verifyGhidraRelocationSymbols } from "./lib/real-ghidra-relocation-symb
|
||||
import {
|
||||
verifyDenseDefaultReturn,
|
||||
verifyJumpTableDenseSwitch,
|
||||
verifyMaskedByteSwitch,
|
||||
verifyNativeTypeLayout,
|
||||
verifyRelativeSwitch,
|
||||
verifyRelativeSignedSwitch,
|
||||
@@ -54,7 +55,7 @@ let report;
|
||||
const installDir = process.env.GHIDRA_INSTALL_DIR;
|
||||
if (installDir === undefined || !isAbsolute(installDir))
|
||||
throw new Error(
|
||||
"Set GHIDRA_INSTALL_DIR to the absolute root of an extracted Ghidra 12.1.4 release.",
|
||||
`Set GHIDRA_INSTALL_DIR to the absolute root of an extracted Ghidra ${SUPPORTED_GHIDRA_VERSION} release.`,
|
||||
);
|
||||
const installation = await inspectVerifierGhidraInstallation({
|
||||
environment: process.env,
|
||||
@@ -186,32 +187,37 @@ await withVerifierFixture(fixtures, async () => {
|
||||
]);
|
||||
}
|
||||
}
|
||||
const signedRelativeSource = fileURLToPath(
|
||||
new URL(
|
||||
"../tests/conformance/ghidra/relative-switch-signed.S",
|
||||
import.meta.url,
|
||||
),
|
||||
);
|
||||
for (const selectorBias of [0, 2, 10]) {
|
||||
// Compile the assembly fixtures through the same operator-selected Clang.
|
||||
const switchFixtures = [
|
||||
...[0, 2, 10].map((selectorBias) => ({
|
||||
source: "../tests/conformance/ghidra/relative-switch-signed.S",
|
||||
variant: `relative-signed-${selectorBias}`,
|
||||
flags: [`-DREA_SELECTOR_BIAS=${selectorBias}`],
|
||||
})),
|
||||
{
|
||||
source: "../tests/conformance/ghidra/masked-byte-switch.S",
|
||||
variant: "masked-byte-switch",
|
||||
flags: [],
|
||||
},
|
||||
];
|
||||
for (const { source, variant, flags } of switchFixtures) {
|
||||
const sourceFile = fileURLToPath(new URL(source, import.meta.url));
|
||||
for (const format of [
|
||||
"elf",
|
||||
...(expectedNativeTarget.architecture === "arm64" ? ["mach-o"] : []),
|
||||
]) {
|
||||
const targetPath = join(
|
||||
fixtureRoot,
|
||||
`relative-signed-${selectorBias}-${format}.o`,
|
||||
);
|
||||
const targetPath = join(fixtureRoot, `${variant}-${format}.o`);
|
||||
await exec(clang, [
|
||||
...(format === "elf" ? ["--target=aarch64-linux-gnu"] : []),
|
||||
`-DREA_SELECTOR_BIAS=${selectorBias}`,
|
||||
...flags,
|
||||
"-c",
|
||||
signedRelativeSource,
|
||||
sourceFile,
|
||||
"-o",
|
||||
targetPath,
|
||||
]);
|
||||
crossTargets.push([
|
||||
targetPath,
|
||||
`relative-signed-${selectorBias}`,
|
||||
variant,
|
||||
{ format, architecture: "arm64" },
|
||||
]);
|
||||
}
|
||||
@@ -322,7 +328,11 @@ await withVerifierFixture(fixtures, async () => {
|
||||
const crossResults = [];
|
||||
for (const [targetPath, variant, expectedTarget] of crossTargets) {
|
||||
const result = await verifyTarget(targetPath, variant, expectedTarget);
|
||||
if (variant !== "aarch64-jump-table" && !variant.startsWith("relative-"))
|
||||
if (
|
||||
variant !== "aarch64-jump-table" &&
|
||||
variant !== "masked-byte-switch" &&
|
||||
!variant.startsWith("relative-")
|
||||
)
|
||||
assertCrossFixture(result);
|
||||
crossResults.push(result);
|
||||
}
|
||||
@@ -584,30 +594,33 @@ async function verifyTarget(targetPath, variant, expectedTarget = null) {
|
||||
client,
|
||||
})
|
||||
: null;
|
||||
const probes = variant.startsWith("relative-signed-")
|
||||
? await verifyRelativeSignedSwitch(
|
||||
client,
|
||||
Number(variant.split("-").at(-1)),
|
||||
)
|
||||
: variant.startsWith("relative-")
|
||||
? await verifyRelativeSwitch(
|
||||
client,
|
||||
procedures,
|
||||
Number(variant.slice(-1)),
|
||||
)
|
||||
: variant === "type-layout"
|
||||
? await verifyNativeTypeLayout(client, names)
|
||||
: variant === "jump-table"
|
||||
? await verifyJumpTableDenseSwitch(client, procedures)
|
||||
: variant === "custom" || variant === "aarch64-jump-table"
|
||||
? null
|
||||
: await verifyInventoryOperations({
|
||||
client,
|
||||
variant,
|
||||
procedures,
|
||||
names,
|
||||
strings,
|
||||
});
|
||||
const probes =
|
||||
variant === "masked-byte-switch"
|
||||
? await verifyMaskedByteSwitch(client)
|
||||
: variant.startsWith("relative-signed-")
|
||||
? await verifyRelativeSignedSwitch(
|
||||
client,
|
||||
Number(variant.split("-").at(-1)),
|
||||
)
|
||||
: variant.startsWith("relative-")
|
||||
? await verifyRelativeSwitch(
|
||||
client,
|
||||
procedures,
|
||||
Number(variant.slice(-1)),
|
||||
)
|
||||
: variant === "type-layout"
|
||||
? await verifyNativeTypeLayout(client, names)
|
||||
: variant === "jump-table"
|
||||
? await verifyJumpTableDenseSwitch(client, procedures)
|
||||
: variant === "custom" || variant === "aarch64-jump-table"
|
||||
? null
|
||||
: await verifyInventoryOperations({
|
||||
client,
|
||||
variant,
|
||||
procedures,
|
||||
names,
|
||||
strings,
|
||||
});
|
||||
const nativeValues =
|
||||
variant === "debug"
|
||||
? await verifyNativeValueTrace(client, procedures, parsedTarget.value)
|
||||
|
||||
@@ -73,7 +73,7 @@ if (values["compare-ghidra"]) {
|
||||
);
|
||||
assert.ok(
|
||||
process.env.GHIDRA_INSTALL_DIR,
|
||||
"Ghidra comparison prerequisite missing: GHIDRA_INSTALL_DIR (bring your own Ghidra 12.1.4 and JDK 21)",
|
||||
"Ghidra comparison prerequisite missing: GHIDRA_INSTALL_DIR (bring your own Ghidra 12.2 and JDK 25)",
|
||||
);
|
||||
}
|
||||
const run = createVerifierRun();
|
||||
|
||||
Reference in new issue
Block a user