refactor(providers)!: key Ghidra and Hopper analysis profiles on the producer identity instead of hand-bumped markers (#11320)

* refactor(ghidra)!: key analysis profiles on the producer identity instead of hand-bumped markers

Ghidra snapshot validity depended on 53 hand-maintained "-vN" semantic
markers in the analysis profile; a behaviour fix that forgot to bump its
marker left old snapshots admissible (#3664). #10562 added a digest of the
bridge sources, but TypeScript projection changes still needed a bump and
the REA release was not committed.

- src/filesystem/ProducerIdentity.ts owns the producer identity: the REA
  release plus a framed SHA-256 of a producer's packaged scripts, read once
  per process.
- Ghidra profiles commit `producer` in place of `bridge_sources_sha256` and
  the constant revision markers (shared semantics block, load-image, entry
  seed, PDB, jump-table, DOL import, MIPS ISA-mode markers) are removed.
- Runtime-selected values keep their meaning without a revision suffix
  (language_selection, annotation_policy, relocation/no-return/launch
  policy, MIPS/PowerPC/PSP lanes, MIPS ABI basis). Bridge wire policy ids
  shared with the Java scripts (ilp32-prototype-v1,
  o32-soft-float-prototype-v1) are unchanged.

BREAKING CHANGE: Ghidra Evidence `analysis_profile.parameters` replace the
semantic marker keys and `bridge_sources_sha256` with `producer`
({rea_version, sources_sha256}), and selection values lose their "-vN"
suffix (for example `language_selection: "provider-pinned"`). Ghidra
Evidence IDs and profile digests change, and analysis snapshots saved by
earlier builds are refused at open. Migration: open the target without the
old snapshot and save a fresh one; consumers reading the removed marker keys
should read `producer` instead.

* fix(hopper)!: commit the REA release and bridge script in Hopper analysis profiles (#11324)

Hopper analysis profiles committed the launcher digest and one hand-bumped
marker (function_dossier_coverage "...-v1"), but neither the REA release
nor bridge/hopper_bridge.py. A snapshot saved before a bridge or REA change
replayed results the current bridge would not produce.

HopperBridgeSources owns the packaged bridge script URL (used to launch it)
and hopperProducerIdentity() on the shared ProducerIdentity owner; the
profile commits `producer` in place of the marker.

BREAKING CHANGE: Hopper Evidence `analysis_profile.parameters` replace
`function_dossier_coverage` with `producer` ({rea_version, sources_sha256}).
Hopper profile digests and Evidence IDs change, and analysis snapshots saved
by earlier builds are refused at open. Migration: open the target without
the old snapshot and save a fresh one.

* fix(ghidra): build the legacy reference snapshot from another producer and use the shared temp directory

* refactor(providers): key Kuna exports and the Native AOT profile on the producer identity (#11353)

* fix(ghidra): keep the bridge source table import-free for the Node-loaded verifier

scripts/verify-ghidra-bridge-sources.mjs loads GhidraBridgeSources.ts with
Node type stripping, which does not resolve .js specifiers to .ts sources.
The producer identity import made every verifier run fail with
ERR_MODULE_NOT_FOUND; the identity now lives in GhidraProducerIdentity.ts.

* fix(artifacts): import the moved file source and abort check from their resources owner after merging main
This commit is contained in:
morluto authored and GitHub committed 2026-11-16 05:37:57 +08:00
1 parent 960a387731
commit d87be930c4
65 files changed
+441 -419

No files matched your search

+1 -1
View File
@@ -28,7 +28,7 @@ replaces the classic floating-point unit, so it misinterprets ordinary
The lane therefore imports with `PowerPC:BE:32:default` and compiler spec
`default`, commits that choice in the analysis profile as
`language_selection: "provider-pinned-v1"`, and checks the language Ghidra
`language_selection: "provider-pinned"`, and checks the language Ghidra
reports after import. A different language set through
`REA_GHIDRA_LANGUAGE_ID` (and optionally `REA_GHIDRA_COMPILER_SPEC_ID`)
replaces the lane default and is admitted against the installed language
+3 -2
View File
@@ -183,8 +183,9 @@ REA writes Kuna's exports only inside `REA_KUNA_CACHE_DIR`:
The cache key includes the selected launcher and resolved executable paths, plus
the target's native basename: that name determines the export filenames and
appears in generated source. REA accepts one current manifest layout. Older
exports are rebuilt on demand and retired through normal owned-cache cleanup;
appears in generated source. The key and manifest also name the REA release
that wrote the entry, and REA accepts only entries from its own release. Exports
from other releases are rebuilt on demand and retired through normal owned-cache cleanup;
no manual deletion or migration is needed. Manifests must name the producer's
exact flat file layout, with matching basenames and producing inputs. Replaced
entry, project or manifest symlinks are refused before their targets are read;
+3 -3
View File
@@ -48,9 +48,9 @@ the fixed Ghidra load segment `0x1000`, and the linear address convention are
committed in the analysis profile. The handshake rejects language or compiler
drift. Closing the session deletes the temporary project and snapshot.
The profile also commits `function_body_evidence: complete-inclusive-ranges-v1`,
so snapshots from older length-only Ghidra results do not satisfy the current
profile.
The profile also commits the REA release and a digest of the bridge scripts
(`producer`), so snapshots from older Ghidra results, such as length-only
function bodies, do not satisfy the current profile.
## Headerless COM images
+1 -1
View File
@@ -33,7 +33,7 @@ structure, recorded as `mips_abi_basis` in the profile and stated as a
limitation in Evidence, never a declared fact; floating-point ABI semantics
stay unverified. Ghidra's own header auto-selection picks a 64-bit language
for exactly these files, so the lane pins `-processor MIPS:<LE|BE>:32:default`
through a `provider-pinned-v1` profile selection and checks the
through a `provider-pinned` profile selection and checks the
bridge-reported `language_id` after import. A caller-selected
`REA_GHIDRA_LANGUAGE_ID`/`REA_GHIDRA_COMPILER_SPEC_ID` still wins over the
pin; combining the pinned profile with an environment override is a typed
+5
View File
@@ -318,6 +318,11 @@ same function can be retried after raising the limit. See the
[settings design](../design/limits-and-recovery.md) for snapshots, precedence,
and provider integration.
Every Ghidra analysis profile commits `producer`: the REA release and a SHA-256
of the packaged bridge scripts. A snapshot replays only under the same
producer, so after an REA upgrade or a bridge script change an older snapshot is
refused; open the target without it and save a fresh snapshot.
When `inspect_native_instruction` selects a byte inside a listed instruction,
`containing_instruction` reports its separate start, length and Ghidra Listing
text, retaining any non-default address space. The requested address remains
+2 -2
View File
@@ -67,10 +67,10 @@ Ghidra's header-based selection is not reliable for RV32. The 12.1.4
So an ordinary RV32 image lands on an Andes processor model unless the language
is stated. REA therefore pins `RISCV:LE:32:default` or `RISCV:LE:64:default`
with the `gcc` compiler specification per class, records
`language_selection: provider-pinned-v1`, and refuses a caller-selected
`language_selection: provider-pinned`, and refuses a caller-selected
`REA_GHIDRA_LANGUAGE_ID` or `REA_GHIDRA_COMPILER_SPEC_ID` that disagrees with the
pin without changing the requested override. An override that matches the pin is
accepted and recorded as `configured-v1`.
accepted and recorded as `configured`.
The soft-float lane pins the same `RISCV:LE:32:default` language. Every RV32
language in Ghidra 12.1.4, including the deprecated soft-float `RV32I`, `RV32IC`
+1 -1
View File
@@ -63,7 +63,7 @@ original compiler or ABI of a headerless target. Ghidra's x86 languages do not
provide a compiler specification named `default`.
Those settings, the caller-supplied entry, and `language_selection:
caller-supplied-v1` are committed in the analysis profile. A different entry
caller-supplied` are committed in the analysis profile. A different entry
or architecture is a different profile; snapshots from header-detected targets
do not replay.
+10 -10
View File
@@ -39,17 +39,17 @@ export async function verifyLegacyGhidraReferenceSnapshot(
record.parameters.procedure === procedure,
);
assert.ok(observed);
const { function_references: coverage, ...legacyParameters } =
observed.analysis_profile.parameters;
assert.equal(
typeof coverage,
"string",
"Reference semantics must participate in snapshot identity",
);
const legacyProfile = createAnalysisProfile(
observed.provider,
legacyParameters,
// A snapshot from an earlier bridge: same profile, another producer digest.
const { producer } = observed.analysis_profile.parameters;
assert.match(
producer?.sources_sha256 ?? "",
/^[0-9a-f]{64}$/u,
"The producer identity must participate in snapshot identity",
);
const legacyProfile = createAnalysisProfile(observed.provider, {
...observed.analysis_profile.parameters,
producer: { ...producer, sources_sha256: "0".repeat(64) },
});
assert.notEqual(legacyProfile.digest, observed.analysis_profile.digest);
const parsed = await parseBinaryTarget(target.path, {
cwd: process.cwd(),
@@ -20,17 +20,16 @@ import {
requireMcpToolError,
} from "./mcp-verifier-results.mjs";
/** A pre-scope dossier must not be replayed into a newly qualified trace. */
/** A dossier from an earlier bridge must not be replayed into a newly qualified trace. */
export async function verifyLegacyValueScopeSnapshot(
client,
{ target, procedure, observed, env },
) {
const { value_flow_effect_scope: scope, ...legacyParameters } =
observed.analysis_profile.parameters;
const legacyProfile = createAnalysisProfile(
observed.provider,
legacyParameters,
);
const { producer } = observed.analysis_profile.parameters;
const legacyProfile = createAnalysisProfile(observed.provider, {
...observed.analysis_profile.parameters,
producer: { ...producer, sources_sha256: "0".repeat(64) },
});
const parsed = await parseBinaryTarget(target.path, {
cwd: process.cwd(),
hostArchitecture: process.arch,
@@ -95,7 +94,7 @@ export async function verifyLegacyValueScopeSnapshot(
},
});
assert.equal(reply.isError, true);
assert.equal(scope, "pcode-load-store-explicit-v1");
assert.match(producer.sources_sha256, /^[0-9a-f]{64}$/u);
const error = requireMcpToolError(reply);
assert.equal(error.code, "evidence_integrity_mismatch");
assert.deepEqual(error.details.binding_mismatch.profile_digest, {
+2 -9
View File
@@ -546,20 +546,13 @@ function hexBytes(bytes) {
return bytes.replace(/\s/gu, "").toLowerCase();
}
function assertProfile(profile) {
assert.equal(
profile.parameters.load_image_evidence,
"independent-mz-mapping-relocations-v2",
);
assert.equal(
profile.parameters.function_body_evidence,
"complete-inclusive-ranges-v1",
);
assert.match(profile.parameters.producer.sources_sha256, /^[0-9a-f]{64}$/u);
assert.equal(profile.provider.id, "ghidra");
assert.equal(profile.parameters.target_format, "dos-mz");
assert.equal(profile.parameters.import_mode, "ephemeral-source-immutable");
assert.equal(
profile.parameters.annotation_policy,
"atomic-function-entry-metadata-v1",
"atomic-function-entry-metadata",
);
assert.equal(profile.parameters.loader, "MzLoader");
assert.equal(profile.parameters.language_id, "x86:LE:16:Real Mode");
@@ -233,7 +233,7 @@ async function verifyFixture(byteOrder, kernelModule) {
assert.equal(parameters.mips_elf.type, 1);
assert.equal(
parameters.mips_support_lane,
"elf32-rel-o32-arch32r2-standard-v1",
"elf32-rel-o32-arch32r2-standard",
);
assert.equal(
parameters.mips_soft_float_storage,
+1 -1
View File
@@ -275,7 +275,7 @@ try {
);
assert.equal(
snapshot.binding.analysis_profile.parameters.language_selection,
"provider-pinned-v1",
"provider-pinned",
);
assert.equal(
snapshot.binding.analysis_profile.parameters.language_id,
+4 -4
View File
@@ -178,7 +178,7 @@ async function verifyExistingMipsProject(fixture, byteOrder) {
);
assert.ok(Array.isArray(observation.normalized_result));
const parameters = observation.analysis_profile?.parameters;
assert.equal(parameters?.import_mode, "existing-project-private-copy-v1");
assert.equal(parameters?.import_mode, "existing-project-private-copy");
assert.equal(parameters?.mips_elf?.byte_order, byteOrder);
assert.equal(parameters?.mips_support_lane, undefined);
assert.deepEqual(parameters?.original_analysis, {
@@ -353,12 +353,12 @@ try {
parameters.language_id,
`MIPS:${byteOrder === "little" ? "LE" : "BE"}:32:default`,
);
assert.equal(parameters.language_selection, "provider-pinned-v1");
assert.equal(parameters.language_selection, "provider-pinned");
assert.equal(
parameters.mips_support_lane,
"elf32-exec-arch32-legacy-pinned-v1",
"elf32-exec-arch32-legacy-pinned",
);
assert.equal(parameters.mips_abi_basis, "inferred-o32-assumption-v1");
assert.equal(parameters.mips_abi_basis, "inferred-o32-assumption");
assert.ok(
snapshot.evidence_bundle.records.some((record) =>
parseEvidence(record).limitations.some((item) =>
+1 -1
View File
@@ -174,7 +174,7 @@ await withVerifierFixture(workspaceFixture, async () => {
assert.ok(profile.value.profile);
assert.equal(
profile.value.profile.parameters.no_return_repair,
"returning-imports-decoded-return-v2",
"returning-imports-decoded-return",
);
const client = provider.createClient(target.value, profile.value.profile);
cleaned = false;
@@ -224,7 +224,7 @@ export async function verifyProjectCoverage(run) {
assert.equal(
direct.analysis_profile.parameters
.selected_project_relationship_coverage,
selected ? "prior-analysis-unknown-v1" : undefined,
selected ? "prior-analysis-unknown" : undefined,
);
for (const [start, goal, direction] of [
[caller, leaf, "forward"],
@@ -315,7 +315,7 @@ export async function verifyProjectCoverage(run) {
assert.equal(
snapshot.binding.analysis_profile.parameters
.selected_project_relationship_coverage,
"prior-analysis-unknown-v1",
"prior-analysis-unknown",
);
const profile = snapshot.binding.analysis_profile;
const oldParameters = { ...profile.parameters };
+1 -1
View File
@@ -171,7 +171,7 @@ try {
const parameters = profile.parameters;
assert.equal(parameters.language_id, expectedLanguage);
assert.equal(parameters.compiler_spec_id, "gcc");
assert.equal(parameters.language_selection, "provider-pinned-v1");
assert.equal(parameters.language_selection, "provider-pinned");
assert.equal(parameters.riscv_support_lane, want.lane);
assert.equal(parameters.riscv_calling_convention, want.callingConvention);
assert.deepEqual(parameters.riscv_elf, want.riscvElf);
+1 -1
View File
@@ -2,9 +2,9 @@ import { createHash } from "node:crypto";
import {
ArtifactReaderFailure,
type ArtifactResourceScope,
streamChunkToBuffer,
} from "../resources/index.js";
import { ZipArtifactReader } from "../artifacts/ZipArtifactReader.js";
import { streamChunkToBuffer } from "../resources/index.js";
import type {
AndroidJniLibraryObservation,
AndroidJniReport,
+1 -1
View File
@@ -1,5 +1,5 @@
import { readArtifactFileChunks } from "../resources/index.js";
import {
readArtifactFileChunks,
ArtifactReaderFailure,
type ArtifactTraversalOptions,
ArtifactResourceScope,
@@ -6,8 +6,6 @@ import { basename, dirname, isAbsolute } from "node:path";
import {
ArtifactResourceScope,
type ArtifactCleanupAttempt,
} from "../../resources/index.js";
import {
SafeOutputTree,
outputDirectoryAdmissionRefusal,
SafeOutputTreeDestinationRefusal,
@@ -16,15 +16,13 @@ import {
ArtifactFileCleanupFailure,
ArtifactReaderFailure,
ArtifactResourceScope,
} from "../../resources/index.js";
import { RegularFileCleanupFailure } from "../../filesystem/OwnedRegularFile.js";
import { readFileHandleChunks } from "../../filesystem/readFileHandleChunks.js";
import {
streamChunkToBuffer,
SafeOutputTree,
outputDirectoryAdmissionRefusal,
SafeOutputTreeCreationFailure,
} from "../../resources/index.js";
import { RegularFileCleanupFailure } from "../../filesystem/OwnedRegularFile.js";
import { readFileHandleChunks } from "../../filesystem/readFileHandleChunks.js";
import { RegularFileChangedError } from "../../filesystem/RegularFile.js";
import { withRegularFile } from "../../process/RegularFileRead.js";
import {
+1 -1
View File
@@ -25,6 +25,7 @@ import {
type ArtifactEntry,
type ArtifactReader,
type ArtifactTraversalOptions,
readArtifactFileChunks,
} from "../resources/index.js";
import { closeAsarHandle, readValidatedAsarEntry } from "./AsarEntryStream.js";
import {
@@ -34,7 +35,6 @@ import {
} from "./AsarHeader.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import type { FileHandleChunkStream } from "../filesystem/readFileHandleChunks.js";
import { readArtifactFileChunks } from "../resources/index.js";
import {
assertAsarReadActive,
asarCancellationFailure,
+1 -1
View File
@@ -20,11 +20,11 @@ import {
ArtifactFileCancelled,
ArtifactFileCleanupFailure,
AnalysisCleanupLease,
readArtifactFileChunks,
} from "../resources/index.js";
import { hostExecutableMarking } from "./hostExecutableMarking.js";
import { OwnedDirectoryHandle } from "../filesystem/OwnedDirectoryHandle.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import { readArtifactFileChunks } from "../resources/index.js";
import {
withOwnedFileHandle,
RegularFileCleanupFailure,
+2 -4
View File
@@ -6,6 +6,8 @@ import {
type ArtifactEntry,
type ArtifactReader,
type ArtifactTraversalOptions,
readArtifactFileChunks,
streamChunkToBuffer,
} from "../resources/index.js";
import { readFileHandlePrefix } from "../filesystem/readFileHandlePrefix.js";
import { Readable } from "node:stream";
@@ -20,10 +22,6 @@ import {
assertStableRegularFileIdentity,
type StableRegularFileDescriptor,
} from "../filesystem/RegularFile.js";
import {
readArtifactFileChunks,
streamChunkToBuffer,
} from "../resources/index.js";
import type { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import { abortIfNeeded } from "../resources/index.js";
@@ -13,15 +13,15 @@ import {
import { fixedAllowanceOptions } from "../../domain/fixedAllowanceRemediation.js";
import { createHash } from "node:crypto";
import { lstat } from "node:fs/promises";
import { streamChunkToBuffer } from "../../resources/index.js";
import { DirectoryArtifactReader } from "../DirectoryArtifactReader.js";
import {
streamChunkToBuffer,
ArtifactResourceScope,
ArtifactReaderFailure,
sameArtifactSourceIdentity,
type ArtifactEntry,
type ArtifactReaderTraversal,
} from "../../resources/index.js";
import { DirectoryArtifactReader } from "../DirectoryArtifactReader.js";
import { translateArtifactFailure } from "../ArtifactOperationFailure.js";
import { InternalInvariantError } from "../../domain/internalInvariant.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
-2
View File
@@ -13,8 +13,6 @@ import {
annotateEnglishUnicodeCaseCollisions,
ArtifactPathRegistry,
normalizeArtifactPath,
} from "../../resources/index.js";
import {
ArtifactEntryContentUnavailable,
ArtifactReaderFailure,
type ArtifactEntry,
@@ -10,8 +10,6 @@ import {
ArtifactReaderFailure,
type ArtifactEntry,
type ArtifactReader,
} from "../../resources/index.js";
import {
ArtifactPathRegistry,
normalizeArtifactPath,
streamChunkToBuffer,
-2
View File
@@ -5,8 +5,6 @@ import { Readable } from "node:stream";
import {
ArtifactReaderFailure,
ArtifactResourceScope,
} from "../../resources/index.js";
import {
DestinationCaseCollisionFailure,
DestinationPathUnrepresentableFailure,
SafeOutputTree,
-3
View File
@@ -38,9 +38,6 @@ export const NATIVE_AOT_BASE_WORK_UNITS = 64 * 1024 * 1024;
*/
export const NATIVE_AOT_WORK_UNITS_PER_SOURCE_BYTE = 8;
export const NATIVE_AOT_MAX_WORKING_MEMORY_BYTES = 64 * 1024 * 1024;
/** Version of the built-in read-only PE metadata interpretation in profiles. */
export const NATIVE_AOT_PROFILE_CONTRACT_REVISION =
"rtr-9.1-x64-pe-read-only-v11";
export type NativeAotParserLimits = {
readonly sourceBytes: number;
+61
View File
@@ -0,0 +1,61 @@
import { createHash } from "node:crypto";
import { readFileSync } from "node:fs";
import { PRODUCT_IDENTITY } from "../identity.js";
/**
* The code that produced a cached or committed result: the REA release that
* ran it and the packaged scripts it handed to an engine. A cache key or
* analysis profile that includes this identity is invalidated by any REA
* release or script edit, so producers do not hand-maintain revision markers
* for their own behaviour. Markers remain only for runtime-selected semantics.
*/
export type ProducerIdentity = ReleaseProducerIdentity &
Readonly<{ sources_sha256: string }>;
/**
* The identity of a producer implemented entirely in REA's own code (no
* packaged scripts handed to an engine): the REA release that ran it.
*/
export type ReleaseProducerIdentity = Readonly<{ rea_version: string }>;
const RELEASE_PRODUCER: ReleaseProducerIdentity = Object.freeze({
rea_version: PRODUCT_IDENTITY.packageVersion,
});
/** The REA release, as the producer of results computed in REA's own code. */
export const releaseProducerIdentity = (): ReleaseProducerIdentity =>
RELEASE_PRODUCER;
/**
* SHA-256 over the named packaged sources in `directory`, in name order, each
* framed by its name and byte length. A missing source throws: a producer
* whose scripts are not packaged cannot run, so it has no identity either.
*/
export const packagedSourcesDigest = (
directory: URL,
names: readonly string[],
): string => {
const hash = createHash("sha256");
for (const name of [...names].sort()) {
const bytes = readFileSync(new URL(name, directory));
hash.update(JSON.stringify([name, bytes.byteLength])).update(bytes);
}
return hash.digest("hex");
};
/**
* One identity per packaged producer, computed on first use. Packaged sources
* do not change while REA runs, so the digest is read once per process.
*/
export const packagedProducerIdentity = (
directory: URL,
names: readonly string[],
): (() => ProducerIdentity) => {
let identity: ProducerIdentity | undefined;
return () =>
(identity ??= Object.freeze({
...RELEASE_PRODUCER,
sources_sha256: packagedSourcesDigest(directory, names),
}));
};
-2
View File
@@ -30,8 +30,6 @@ import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
import {
type SafeOutputTree,
readArtifactFileChunks,
} from "../resources/index.js";
import {
ArtifactResourceScope,
withArtifactRegularFile,
assertArtifactReadActive,
+6 -6
View File
@@ -1,4 +1,9 @@
import { AnalysisCleanupLease } from "../resources/index.js";
import {
AnalysisCleanupLease,
SafeOutputTree,
outputDirectoryAdmissionRefusal,
SafeOutputTreeCreationFailure,
} from "../resources/index.js";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { analysisErrorWithCleanupDependencies } from "../domain/analysisErrorCleanup.js";
import {
@@ -8,11 +13,6 @@ import {
import { createHash } from "node:crypto";
import { join } from "node:path";
import type { z } from "zod";
import {
SafeOutputTree,
outputDirectoryAdmissionRefusal,
SafeOutputTreeCreationFailure,
} from "../resources/index.js";
import { classifyProviderFailure } from "../providers/classifyProviderFailure.js";
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
+31 -71
View File
@@ -17,7 +17,7 @@ import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { err, ok, type Result } from "../domain/result.js";
import { ghidraDisabledAnalyzerPreset } from "./GhidraDisabledAnalyzers.js";
import { ghidraBridgeSourcesDigest } from "./GhidraBridgeSources.js";
import { ghidraProducerIdentity } from "./GhidraProducerIdentity.js";
import type { GhidraInstallationInspection } from "./GhidraInstallation.js";
import {
isLegacyMipsAdmission,
@@ -51,41 +51,21 @@ import {
NATIVE_AOT_MAX_WORKING_MEMORY_BYTES,
NATIVE_AOT_BASE_WORK_UNITS,
NATIVE_AOT_WORK_UNITS_PER_SOURCE_BYTE,
NATIVE_AOT_PROFILE_CONTRACT_REVISION,
} from "../domain/native/nativeAotPe.js";
/** Shared analysis commitments for raw and header-selected imports. */
const sharedAnalysisSemanticMarkers = {
annotation_policy: "atomic-function-entry-metadata-v1",
byte_search_boundary: "contiguous-eligible-initialized-blocks-v1",
load_image_observations: "source-mappings-entry-context-relocation-symbol-v3",
function_body_evidence: "complete-inclusive-ranges-v1",
function_references: "complete-body-and-entry-reference-manager-v3",
call_relationship_coverage: "observed-call-inventory-and-discovery-v7",
function_dossier_coverage: "observed-facets-and-call-relationships-v1",
native_value_flow_projection: "accepted-runtime-budgets-v1",
value_flow_effect_scope: "pcode-load-store-explicit-v1",
string_inventory_evidence: "defined-data-coverage-v1",
inventory_search_projection: "provider-inventory-facts-v1",
data_type_observations: "bit-field-base-and-lookup-v1",
location_resolution: "explicit-address-exact-entry-symbol-first-v3",
procedure_search: "primary-and-exact-entry-symbol-names-v1",
procedure_file_offset_resolution:
"address-first-unique-target-source-mapping-v2",
instruction_flow_evidence: "decoded-return-pcode-v1",
instruction_status: "unlisted-and-containing-code-data-v3",
instruction_references: "exact-address-incoming-without-synthetic-entry-v3",
byte_search_coverage: "separate-loaded-overlays-v1",
function_boundary_observations: "ghidra-function-boundary-limitations-v5",
} as const;
/** Marker revisions plus the bridge sources that produce every Ghidra result. */
/**
* The producer every Ghidra result depends on. Behaviour REA implements (bridge
* scripts or TypeScript projection) is covered by the producer identity; the
* profile adds only facts selected at runtime, so no revision marker needs a
* hand bump when that behaviour changes.
*/
const sharedAnalysisSemantics = () => ({
...sharedAnalysisSemanticMarkers,
bridge_sources_sha256: ghidraBridgeSourcesDigest(),
producer: ghidraProducerIdentity(),
annotation_policy: "atomic-function-entry-metadata",
});
const loaderRelocationSemantics = "volatile-unapplied-relocation-blocks-v1";
const loaderRelocationSemantics = "volatile-unapplied-relocation-blocks";
// Read-only sessions record relocation limitations without marking blocks volatile.
const readOnlyLoaderRelocationSemantics = "read-only-relocation-limitations-v1";
const readOnlyLoaderRelocationSemantics = "read-only-relocation-limitations";
/**
* The processor language REA pins for this target, or undefined when Ghidra may
@@ -134,12 +114,11 @@ const binaryLoaderPolicy = (
no_return_repair:
installation.platform === "win32"
? "disabled-windows-p0"
: "returning-imports-decoded-return-v2",
: "returning-imports-decoded-return",
process_launch:
installation.platform === "win32"
? "official-headless-script-v1"
: "inspected-jvm-launch-support-v1",
jump_table_evidence: "typed-case-default-blocks-v1",
? "official-headless-script"
: "inspected-jvm-launch-support",
decompiler_jump_loads: true,
});
@@ -214,16 +193,12 @@ export const resolveGhidraAnalysisProfile = async (
ok({
profile: createProfile({
...binaryLoaderPolicy(target, installation),
load_image_evidence: "independent-dol-section-mapping-v1",
loader: "BinaryLoader",
language_id: language.languageId,
compiler_spec_id: language.compilerSpecId ?? "language-default",
language_selection:
languageOverride === undefined
? "provider-pinned-v1"
: "configured-v1",
languageOverride === undefined ? "provider-pinned" : "configured",
entry_point: target.entryPoint,
entry_seed: "external-entry-and-function-before-analysis-v1",
...ghidraDolProfileParameters(target.dol),
...analyzerParameters,
}),
@@ -245,13 +220,11 @@ export const resolveGhidraAnalysisProfile = async (
ok({
profile: createProfile({
...binaryLoaderPolicy(target, installation),
load_image_evidence: "independent-raw-mapping-context-v4",
loader: "BinaryLoader",
language_id: language.languageId,
compiler_spec_id: language.compilerSpecId,
language_selection: "caller-supplied-v1",
language_selection: "caller-supplied",
entry_point: target.entryPoint,
entry_seed: "external-entry-and-function-before-analysis-v1",
...analyzerParameters,
}),
}),
@@ -304,7 +277,6 @@ export const resolveGhidraAnalysisProfile = async (
executable_role: target.executableRole ?? null,
managed: target.managed ?? null,
native_aot_metadata: {
contract_revision: NATIVE_AOT_PROFILE_CONTRACT_REVISION,
source_bytes: NATIVE_AOT_MAX_SOURCE_BYTES,
base_work_units: NATIVE_AOT_BASE_WORK_UNITS,
work_units_per_source_byte:
@@ -319,17 +291,14 @@ export const resolveGhidraAnalysisProfile = async (
...(target.availableArchitectures ?? []),
].sort(),
import_mode: "ephemeral-source-immutable",
...(target.format === "pe"
? { pdb_observations: "program-info-v5" }
: {}),
...(installation.platform === "win32" &&
target.format === "pe" &&
(target.architecture === "arm" || target.architecture === "arm64")
? {
windows_arm_pe_import:
selectedLanguage === undefined
? "loader-selected-language-v1"
: "explicit-language-v1",
? "loader-selected-language"
: "explicit-language",
windows_arm_pe_machine: target.peMachine ?? null,
}
: {}),
@@ -342,19 +311,11 @@ export const resolveGhidraAnalysisProfile = async (
no_return_repair:
installation.platform === "win32"
? "disabled-windows-p0"
: "returning-imports-decoded-return-v2",
: "returning-imports-decoded-return",
process_launch:
installation.platform === "win32"
? "official-headless-script-v1"
: "inspected-jvm-launch-support-v1",
...(dos
? {
load_image_evidence: dosCom
? "independent-com-mapping-context-v1"
: "independent-mz-mapping-relocations-v2",
}
: {}),
jump_table_evidence: "typed-case-default-blocks-v1",
? "official-headless-script"
: "inspected-jvm-launch-support",
decompiler_jump_loads: true,
loader: dosMz
? "MzLoader"
@@ -372,8 +333,8 @@ export const resolveGhidraAnalysisProfile = async (
...(languageOverride === undefined
? pinnedLanguage === undefined
? {}
: { language_selection: "provider-pinned-v1" }
: { language_selection: "configured-v1" }),
: { language_selection: "provider-pinned" }
: { language_selection: "configured" }),
...(dos
? {
load_segment: "0x1000",
@@ -389,7 +350,6 @@ export const resolveGhidraAnalysisProfile = async (
ES: "0x1000",
SS: "0x1000",
},
entry_seed: "external-entry-and-function-before-analysis-v1",
}
: {}),
...analyzerParameters,
@@ -398,7 +358,7 @@ export const resolveGhidraAnalysisProfile = async (
...ghidraPowerPcProfileParameters(target),
...(pspExtension?.ok === true
? {
mips_support_lane: "psp-elf32-exec-eabi32-allegrex-v2",
mips_support_lane: "psp-elf32-exec-eabi32-allegrex",
loader: "PspElfLoader",
language_id: "Allegrex:LE:32:default",
compiler_spec_id: "default",
@@ -415,8 +375,8 @@ export const ghidraProfileLanguageOverride = (
parameters: Readonly<Record<string, unknown>>,
): GhidraLanguageOverride | undefined => {
if (
parameters.language_selection !== "configured-v1" &&
parameters.language_selection !== "provider-pinned-v1"
parameters.language_selection !== "configured" &&
parameters.language_selection !== "provider-pinned"
)
return undefined;
const languageId = parameters.language_id;
@@ -432,7 +392,7 @@ export const ghidraProfileLanguageOverride = (
export const ghidraProfilePinnedLanguage = (
parameters: Readonly<Record<string, unknown>>,
): GhidraLanguageOverride | undefined =>
parameters.language_selection === "provider-pinned-v1"
parameters.language_selection === "provider-pinned"
? ghidraProfileLanguageOverride(parameters)
: undefined;
@@ -502,14 +462,14 @@ export const existingGhidraAnalysisProfile = (
target_format: target.format,
architecture: target.architecture ?? null,
...(mipsElf === undefined ? {} : { mips_elf: mipsElf }),
import_mode: "existing-project-private-copy-v1",
import_mode: "existing-project-private-copy",
...sharedAnalysisSemantics(),
selected_project_relationship_coverage: "prior-analysis-unknown-v1",
annotation_policy: "read-only-existing-project-v1",
selected_project_relationship_coverage: "prior-analysis-unknown",
annotation_policy: "read-only-existing-project",
loader_relocation_semantics: readOnlyLoaderRelocationSemantics,
existing_project: project,
original_analysis: { completeness: "unknown", settings: "unknown" },
no_return_repair: "disabled-existing-project",
process_launch: "inspected-jvm-launch-support-v1",
process_launch: "inspected-jvm-launch-support",
});
};
+13 -13
View File
@@ -1,11 +1,10 @@
import { createHash } from "node:crypto";
import { readdir, readFile } from "node:fs/promises";
import { readdir } from "node:fs/promises";
import { expect, it } from "vitest";
import {
GHIDRA_BRIDGE_SOURCES,
ghidraBridgeSourcesDigest,
} from "./GhidraBridgeSources.js";
import { packagedSourcesDigest } from "../filesystem/ProducerIdentity.js";
import { PRODUCT_IDENTITY } from "../identity.js";
import { GHIDRA_BRIDGE_SOURCES } from "./GhidraBridgeSources.js";
import { ghidraProducerIdentity } from "./GhidraProducerIdentity.js";
const directory = new URL("../../bridge/ghidra/", import.meta.url);
@@ -16,11 +15,12 @@ it("lists every packaged Ghidra bridge source", async () => {
expect(Object.values(GHIDRA_BRIDGE_SOURCES).sort()).toEqual(java.sort());
});
it("commits to the name and bytes of each bridge source", async () => {
const hash = createHash("sha256");
for (const name of Object.values(GHIDRA_BRIDGE_SOURCES).sort()) {
const bytes = await readFile(new URL(name, directory));
hash.update(JSON.stringify([name, bytes.byteLength])).update(bytes);
}
expect(ghidraBridgeSourcesDigest()).toBe(hash.digest("hex"));
it("identifies Ghidra results by the REA release and every bridge source", () => {
expect(ghidraProducerIdentity()).toEqual({
rea_version: PRODUCT_IDENTITY.packageVersion,
sources_sha256: packagedSourcesDigest(
directory,
Object.values(GHIDRA_BRIDGE_SOURCES),
),
});
});
+7 -24
View File
@@ -1,7 +1,10 @@
import { createHash } from "node:crypto";
import { readFileSync } from "node:fs";
/** Required packaged sources, selected by their runtime role rather than order. */
/**
* Required packaged sources, selected by their runtime role rather than order.
* scripts/verify-ghidra-bridge-sources.mjs loads this module directly with
* Node type stripping, which does not resolve `.js` specifiers to `.ts`
* sources, so it keeps no runtime imports. The producer identity built from
* these sources lives in GhidraProducerIdentity.ts.
*/
export const GHIDRA_BRIDGE_SOURCES = Object.freeze({
bridge: "ReaGhidraBridge.java",
bundleActivator: "ReaGhidraBundleActivator.java",
@@ -15,23 +18,3 @@ export const GHIDRA_BRIDGE_SOURCES = Object.freeze({
mipsSoftFloat: "ReaGhidraMipsSoftFloat.java",
riscvCallingConvention: "ReaGhidraRiscvCallingConvention.java",
} as const);
let bridgeSourcesDigest: string | undefined;
/**
* SHA-256 over the packaged bridge sources, keyed by file name. Ghidra results
* depend on these scripts, so the analysis profile commits to them: a changed
* script yields a new profile digest and earlier snapshots are not reused,
* without a hand-bumped semantics marker.
*/
export const ghidraBridgeSourcesDigest = (): string => {
if (bridgeSourcesDigest !== undefined) return bridgeSourcesDigest;
const directory = new URL("../../bridge/ghidra/", import.meta.url);
const hash = createHash("sha256");
for (const name of Object.values(GHIDRA_BRIDGE_SOURCES).sort()) {
const bytes = readFileSync(new URL(name, directory));
hash.update(JSON.stringify([name, bytes.byteLength])).update(bytes);
}
bridgeSourcesDigest = hash.digest("hex");
return bridgeSourcesDigest;
};
+3 -4
View File
@@ -92,9 +92,8 @@ describe("GameCube DOL Ghidra profile", () => {
loader: "BinaryLoader",
language_id: "PowerPC:BE:32:default",
compiler_spec_id: "default",
language_selection: "provider-pinned-v1",
language_selection: "provider-pinned",
entry_point: "0x80003100",
dol_import: "binary-loader-sections-v2",
});
expect(ghidraPinnedLanguage(target)).toEqual(GHIDRA_GAMECUBE_DOL_LANGUAGE);
expect(ghidraProfileLanguageOverride(parameters)).toEqual(
@@ -132,7 +131,7 @@ describe("GameCube DOL Ghidra profile", () => {
expect(parameters).toMatchObject({
language_id: "PowerPC:BE:32:Gekko_Broadway",
compiler_spec_id: "language-default",
language_selection: "configured-v1",
language_selection: "configured",
});
expect(ghidraProfileLanguageOverride(parameters)).toEqual({
languageId: "PowerPC:BE:32:Gekko_Broadway",
@@ -154,7 +153,7 @@ describe("GameCube DOL Ghidra profile", () => {
expect(resolved.value.profile?.parameters).toMatchObject({
language_id: "PowerPC:BE:32:Gekko_Broadway",
compiler_spec_id: "default",
language_selection: "configured-v1",
language_selection: "configured",
});
});
-1
View File
@@ -53,7 +53,6 @@ export const ghidraDolSectionFile = (header: DolHeader): string =>
export const ghidraDolProfileParameters = (
header: DolHeader,
): Readonly<Record<string, JsonValue>> => ({
dol_import: "binary-loader-sections-v2",
dol_sections: header.sections.map((section) => ({
kind: section.kind,
index: section.index,
+4 -2
View File
@@ -1,7 +1,9 @@
import { createHash } from "node:crypto";
import type { FileHandle } from "node:fs/promises";
import { readArtifactFileChunks } from "../resources/index.js";
import { ArtifactReaderFailure } from "../resources/index.js";
import {
readArtifactFileChunks,
ArtifactReaderFailure,
} from "../resources/index.js";
import {
dosMzDeclaredImageByteLength,
readDosMzDeclarations,
+2 -2
View File
@@ -349,7 +349,7 @@ describe("Ghidra language override profile", () => {
loader: "auto-from-header",
language_id: "x86:LE:32:default",
compiler_spec_id: "borlandcpp",
language_selection: "configured-v1",
language_selection: "configured",
});
expect(ghidraProfileLanguageOverride(parameters)).toEqual({
languageId: "x86:LE:32:default",
@@ -374,7 +374,7 @@ describe("Ghidra language override profile", () => {
expect(resolved.value.profile?.parameters).toMatchObject({
language_id: "MIPS:LE:32:default",
compiler_spec_id: "default",
language_selection: "configured-v1",
language_selection: "configured",
});
} finally {
await rm(mipsLanguages, { recursive: true, force: true });
+12 -12
View File
@@ -247,9 +247,9 @@ describe("recordless legacy MIPS admission", () => {
);
it.each([
[{ flags: 0x00000001 }, "elf32-exec-mips1-legacy-pinned-v1"],
[{ flags: 0x50000001 }, "elf32-exec-arch32-legacy-pinned-v1"],
[{ flags: 0x70000001 }, "elf32-exec-arch32-legacy-pinned-v1"],
[{ flags: 0x00000001 }, "elf32-exec-mips1-legacy-pinned"],
[{ flags: 0x50000001 }, "elf32-exec-arch32-legacy-pinned"],
[{ flags: 0x70000001 }, "elf32-exec-arch32-legacy-pinned"],
] satisfies [Partial<MipsElfMetadata>, string][])(
"admits a recordless legacy header (%j) with a pinned 32-bit language",
async (change, lane) => {
@@ -262,10 +262,10 @@ describe("recordless legacy MIPS admission", () => {
if (!resolved.ok) throw resolved.error;
expect(resolved.value.profile?.parameters).toMatchObject({
mips_support_lane: lane,
mips_abi_basis: "inferred-o32-assumption-v1",
mips_abi_basis: "inferred-o32-assumption",
language_id: "MIPS:LE:32:default",
compiler_spec_id: "default",
language_selection: "provider-pinned-v1",
language_selection: "provider-pinned",
});
const resolvedBig = await ghidra.resolveAnalysisProfile(big);
if (!resolvedBig.ok) throw resolvedBig.error;
@@ -336,7 +336,7 @@ describe("recordless legacy MIPS admission", () => {
expect(parameters.language_selection).toBeUndefined();
expect(parameters.language_id).toBeUndefined();
expect(parameters).toMatchObject({
mips_support_lane: "elf32-exec-arch32-legacy-pinned-v1",
mips_support_lane: "elf32-exec-arch32-legacy-pinned",
});
});
@@ -378,9 +378,9 @@ describe("caller-selected MIPS-I language", () => {
await installedMipsConfig(),
);
expect(profile.parameters).toMatchObject({
mips_support_lane: "elf32-exec-mips1-legacy-pinned-v1",
mips_support_lane: "elf32-exec-mips1-legacy-pinned",
language_id: "MIPS:LE:32:16e",
language_selection: "configured-v1",
language_selection: "configured",
});
const health = await client.execute("health", {});
if (!health.ok) throw health.error;
@@ -561,8 +561,8 @@ describe("MIPS16 shared-image profile", () => {
expect(parameters).toMatchObject({
language_id: `MIPS:${byteOrder === "little" ? "LE" : "BE"}:32:default`,
compiler_spec_id: "default",
language_selection: "provider-pinned-v1",
mips_support_lane: "elf32-dyn-o32-arch32r2-mips16-local-rel32-v1",
language_selection: "provider-pinned",
mips_support_lane: "elf32-dyn-o32-arch32r2-mips16-local-rel32",
mips_elf: { abi_flags: null },
});
expect(ghidraProfileLanguageOverride(parameters)).toEqual({
@@ -716,11 +716,11 @@ describe("soft-float MIPS storage", () => {
const relocatable = target({ type: 1 });
const { profile, client } = await session(relocatable, undefined);
expect(profile.parameters.mips_support_lane).toBe(
"elf32-rel-o32-arch32r2-standard-v1",
"elf32-rel-o32-arch32r2-standard",
);
const executable = await session(target(), undefined);
expect(executable.profile.parameters.mips_support_lane).toBe(
"elf32-exec-o32-arch32r2-standard-v3",
"elf32-exec-o32-arch32r2-standard",
);
const health = await client.execute("health", {});
if (!health.ok) throw health.error;
+8 -9
View File
@@ -348,7 +348,7 @@ export const ghidraMipsProfileParameters = (
? {
language_id: language.languageId,
compiler_spec_id: language.compilerSpecId ?? "default",
language_selection: "provider-pinned-v1",
language_selection: "provider-pinned",
}
: {};
return {
@@ -410,17 +410,16 @@ export const ghidraMipsProfileParameters = (
...(isGhidraMipsSoftFloatTarget(target)
? { mips_soft_float_storage: GHIDRA_MIPS_SOFT_FLOAT_STORAGE }
: {}),
instruction_context_evidence: "per-address-isa-mode-v1",
mips_support_lane: legacy
? isMips1Header(target)
? "elf32-exec-mips1-legacy-pinned-v1"
: "elf32-exec-arch32-legacy-pinned-v1"
? "elf32-exec-mips1-legacy-pinned"
: "elf32-exec-arch32-legacy-pinned"
: isGhidraMips16SharedTarget(target)
? "elf32-dyn-o32-arch32r2-mips16-local-rel32-v1"
? "elf32-dyn-o32-arch32r2-mips16-local-rel32"
: target.mips.type === MIPS_ET_REL
? "elf32-rel-o32-arch32r2-standard-v1"
: "elf32-exec-o32-arch32r2-standard-v3",
...(legacy ? { mips_abi_basis: "inferred-o32-assumption-v1" } : {}),
? "elf32-rel-o32-arch32r2-standard"
: "elf32-exec-o32-arch32r2-standard",
...(legacy ? { mips_abi_basis: "inferred-o32-assumption" } : {}),
...pinnedLanguage,
};
};
@@ -441,7 +440,7 @@ export const ghidraMipsProfileMismatchReason = (
const expected = ghidraMipsProfileParameters(target);
// PSP keeps its own loader and lane but still commits target-derived MIPS
// metadata; bind that metadata without demanding the generic lane name.
const pspLane = "psp-elf32-exec-eabi32-allegrex-v2";
const pspLane = "psp-elf32-exec-eabi32-allegrex";
const committedLane = parameters.mips_support_lane;
const expectedLane =
committedLane === pspLane ? pspLane : expected.mips_support_lane;
+3 -3
View File
@@ -64,9 +64,9 @@ describe("bounded Ghidra PowerPC admission", () => {
architecture: "powerpc",
language_id: "PowerPC:BE:32:default",
compiler_spec_id: "default",
language_selection: "provider-pinned-v1",
language_selection: "provider-pinned",
powerpc_elf: { elf_class: 32, byte_order: "big", type: 2, flags: 0 },
powerpc_support_lane: "elf32-be-exec-generic-v1",
powerpc_support_lane: "elf32-be-exec-generic",
});
expect(ghidraPinnedLanguage(target())).toEqual({
languageId: "PowerPC:BE:32:default",
@@ -99,7 +99,7 @@ describe("bounded Ghidra PowerPC admission", () => {
const parameters = resolved.value.profile?.parameters ?? {};
expect(parameters).toMatchObject({
language_id: "PowerPC:BE:32:e500",
language_selection: "configured-v1",
language_selection: "configured",
});
expect(ghidraProfileLanguageOverride(parameters)).toEqual({
languageId: "PowerPC:BE:32:e500",
+1 -1
View File
@@ -65,6 +65,6 @@ export const ghidraPowerPcProfileParameters = (
type: target.powerpc.type,
flags: target.powerpc.flags,
},
powerpc_support_lane: "elf32-be-exec-generic-v1",
powerpc_support_lane: "elf32-be-exec-generic",
};
};
+12
View File
@@ -0,0 +1,12 @@
import { packagedProducerIdentity } from "../filesystem/ProducerIdentity.js";
import { GHIDRA_BRIDGE_SOURCES } from "./GhidraBridgeSources.js";
/**
* The REA release and packaged bridge sources behind every Ghidra result.
* Analysis profiles commit to it, so an edited bridge script or a new REA
* release yields a new profile digest and earlier snapshots are not reused.
*/
export const ghidraProducerIdentity = packagedProducerIdentity(
new URL("../../bridge/ghidra/", import.meta.url),
Object.values(GHIDRA_BRIDGE_SOURCES),
);
+18 -31
View File
@@ -1,5 +1,6 @@
import { failedInspectionDescriptor } from "../../tests/fixtures/processInspectionCleanup.js";
import { ghidraFailureOutput } from "./GhidraDiagnostics.js";
import { ghidraProducerIdentity } from "./GhidraProducerIdentity.js";
import {
ProcessOwnershipInspectionCancelled,
ProcessOwnershipIdentityBatchFailure,
@@ -372,13 +373,11 @@ it.each(["readiness", "client"] as const)(
);
it.each([
"program-info-v1",
"program-info-v2",
"program-info-v3",
"program-info-v4",
{ rea_version: "6.8.0", sources_sha256: "0".repeat(64) },
{ rea_version: "6.9.0", sources_sha256: "0".repeat(64) },
])(
"retains historical PE PDB Evidence for %s without replaying its old projection",
async (pdbObservations) => {
"retains Evidence from an earlier producer $rea_version without replaying its old projection",
async (producer) => {
const target = peTarget("x86_64");
const resolved = await provider().resolveAnalysisProfile(target);
if (!resolved.ok || resolved.value.profile === null)
@@ -386,15 +385,11 @@ it.each([
const current = resolved.value.profile;
const prior = createAnalysisProfile(current.provider, {
...current.parameters,
pdb_observations: pdbObservations,
producer,
});
const operation = "search_procedures";
const parameters = { pattern: "wmain" };
const limitations = [
pdbObservations === "program-info-v4"
? "Legacy projection discarded NB10 declaration when the reported GUID was omitted."
: "Legacy projection collapsed unknown PDB load state.",
];
const limitations = ["An earlier REA producer projected this result."];
const cache = new AnalysisSnapshotCache();
cache.record({
target,
@@ -490,7 +485,6 @@ describe("Ghidra provider", () => {
target_format: "dos-mz",
architecture: "x86",
loader: "MzLoader",
load_image_evidence: "independent-mz-mapping-relocations-v2",
language_id: "x86:LE:16:Real Mode",
compiler_spec_id: "default",
load_segment: "0x1000",
@@ -671,8 +665,7 @@ describe("Ghidra raw-binary profile", () => {
language_id: "6502:LE:16:default",
compiler_spec_id: "default",
entry_point: "0x8001",
language_selection: "caller-supplied-v1",
load_image_evidence: "independent-raw-mapping-context-v4",
language_selection: "caller-supplied",
});
expect(
ghidra
@@ -702,7 +695,7 @@ describe("Ghidra raw-binary profile", () => {
loader: "BinaryLoader",
language_id: "ARM:LE:32:v7",
compiler_spec_id: "default",
language_selection: "caller-supplied-v1",
language_selection: "caller-supplied",
entry_point: "0x8000000",
},
},
@@ -743,7 +736,7 @@ describe("Ghidra raw-binary profile", () => {
loader: "BinaryLoader",
language_id: "68000:BE:32:default",
compiler_spec_id: "default",
language_selection: "caller-supplied-v1",
language_selection: "caller-supplied",
entry_point: "0x1000",
});
expect(
@@ -858,7 +851,7 @@ describe("Ghidra platform support", () => {
);
if (!resolved.ok) throw resolved.error;
expect(resolved.value.profile?.parameters).toMatchObject({
windows_arm_pe_import: "explicit-language-v1",
windows_arm_pe_import: "explicit-language",
windows_arm_pe_machine: peTarget(architecture).peMachine,
language_id: languageId,
compiler_spec_id: "windows",
@@ -883,7 +876,7 @@ describe("Ghidra platform support", () => {
)
expect(parameters).toHaveProperty(
"windows_arm_pe_import",
"loader-selected-language-v1",
"loader-selected-language",
);
else expect(parameters).not.toHaveProperty("windows_arm_pe_import");
if (
@@ -908,11 +901,9 @@ describe("Ghidra platform support", () => {
};
// Windows P0 cannot mark blocks volatile, so its snapshots must not replay
// as volatile-protected pseudocode.
expect(await semantics("win32")).toBe(
"read-only-relocation-limitations-v1",
);
expect(await semantics("win32")).toBe("read-only-relocation-limitations");
expect(await semantics("linux")).toBe(
"volatile-unapplied-relocation-blocks-v1",
"volatile-unapplied-relocation-blocks",
);
// A caller's existing project is read-only too, so its saved results must
// not match a profile that predates the relocation limitation.
@@ -922,7 +913,7 @@ describe("Ghidra platform support", () => {
{ id: "ghidra", name: "Ghidra", version: "12.1.4" },
{ project_name: "fixture" },
).parameters.loader_relocation_semantics,
).toBe("read-only-relocation-limitations-v1");
).toBe("read-only-relocation-limitations");
});
it("keeps Windows annotation mutation unavailable independently of native controls", () => {
@@ -961,7 +952,6 @@ describe("Ghidra PE analysis profile identity", () => {
executable_role: target.executableRole,
managed: target.managed,
native_aot_metadata: {
contract_revision: "rtr-9.1-x64-pe-read-only-v11",
source_bytes: 128 * 1024 * 1024,
base_work_units: 64 * 1024 * 1024,
work_units_per_source_byte: 8,
@@ -1010,7 +1000,6 @@ describe("Ghidra Windows DOS admission", () => {
profile: {
parameters: {
loader: "MzLoader",
load_image_evidence: "independent-mz-mapping-relocations-v2",
language_id: "x86:LE:16:Real Mode",
load_segment: "0x1000",
},
@@ -1518,17 +1507,15 @@ describe("Ghidra client projection", () => {
provider: { id: "ghidra", name: "Ghidra", version: "12.1.4" },
parameters: {
import_mode: "ephemeral-source-immutable",
annotation_policy: "atomic-function-entry-metadata-v1",
annotation_policy: "atomic-function-entry-metadata",
analyzer_preset: "ghidra-default",
},
});
expect(resolved.value.profile.parameters).toMatchObject({
language_id: "auto-from-header",
compiler_spec_id: "auto-default",
call_relationship_coverage: "observed-call-inventory-and-discovery-v7",
function_dossier_coverage: "observed-facets-and-call-relationships-v1",
native_value_flow_projection: "accepted-runtime-budgets-v1",
loader_relocation_semantics: "volatile-unapplied-relocation-blocks-v1",
producer: ghidraProducerIdentity(),
loader_relocation_semantics: "volatile-unapplied-relocation-blocks",
});
const result = await ghidra
+2 -3
View File
@@ -517,7 +517,7 @@ export const createGhidraProviderClient = (input: {
committedProfile.parameters,
project !== undefined &&
committedProfile.parameters.import_mode ===
"existing-project-private-copy-v1",
"existing-project-private-copy",
);
if (mipsProfileMismatch !== null)
return err(ghidraProfileConflict("open_binary", mipsProfileMismatch));
@@ -586,8 +586,7 @@ export const createGhidraProviderClient = (input: {
]
: []),
...(target.architecture === "mips" ? (target.mips.limitations ?? []) : []),
...(committedProfile.parameters.mips_abi_basis ===
"inferred-o32-assumption-v1"
...(committedProfile.parameters.mips_abi_basis === "inferred-o32-assumption"
? [
"The ELF header declares no ABI field and no ABI flags record was observed; o32 calling conventions are an analysis assumption of this lane, not a declared fact.",
"Floating-point ABI semantics are unverified for this recordless legacy admission.",
+7 -21
View File
@@ -5,18 +5,18 @@ import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import { silentLogger } from "../logger.js";
import { ghidraProfileInstallationHost as host } from "./GhidraInstallation.fixture.js";
import { GhidraProvider } from "./GhidraProvider.js";
import { ghidraBridgeSourcesDigest } from "./GhidraBridgeSources.js";
import { ghidraProducerIdentity } from "./GhidraProducerIdentity.js";
import { createAnalysisProfile } from "../domain/analysisProfile.js";
// A profile digest is the digest of its committed parameters; these tests pin
// the parameters (semantic revisions and the bridge sources) rather than a
// literal hash that every bridge or marker change would have to re-record.
// the parameters (runtime selections and the producer identity) rather than a
// literal hash that every bridge or REA release would have to re-record.
const expectCommittedProfile = (
profile: ReturnType<typeof createAnalysisProfile> | null | undefined,
): void => {
if (profile == null) throw new Error("Expected an analysis profile");
expect(profile.parameters).toMatchObject({
bridge_sources_sha256: ghidraBridgeSourcesDigest(),
producer: ghidraProducerIdentity(),
});
expect(profile.digest).toBe(
createAnalysisProfile(profile.provider, profile.parameters).digest,
@@ -51,19 +51,8 @@ describe("raw-binary analysis profile commitments", () => {
target_format: "raw-binary",
loader: "BinaryLoader",
language_id: "6502:LE:16:default",
language_selection: "caller-supplied-v1",
load_image_evidence: "independent-raw-mapping-context-v4",
load_image_observations:
"source-mappings-entry-context-relocation-symbol-v3",
instruction_status: "unlisted-and-containing-code-data-v3",
inventory_search_projection: "provider-inventory-facts-v1",
value_flow_effect_scope: "pcode-load-store-explicit-v1",
native_value_flow_projection: "accepted-runtime-budgets-v1",
byte_search_boundary: "contiguous-eligible-initialized-blocks-v1",
byte_search_coverage: "separate-loaded-overlays-v1",
data_type_observations: "bit-field-base-and-lookup-v1",
instruction_references:
"exact-address-incoming-without-synthetic-entry-v3",
language_selection: "caller-supplied",
entry_point: "0x8001",
analyzer_preset: "ghidra-default",
});
});
@@ -81,10 +70,7 @@ describe("raw-binary analysis profile commitments", () => {
if (!resolved.ok) throw resolved.error;
expectCommittedProfile(resolved.value.profile);
expect(resolved.value.profile?.parameters).toMatchObject({
inventory_search_projection: "provider-inventory-facts-v1",
value_flow_effect_scope: "pcode-load-store-explicit-v1",
native_value_flow_projection: "accepted-runtime-budgets-v1",
byte_search_coverage: "separate-loaded-overlays-v1",
analyzer_preset: expect.any(String),
analysis_options: {
disabled_analyzers: ["Aggressive Instruction Finder"],
},
+2 -2
View File
@@ -306,7 +306,7 @@ const registerRiscvAdmissionAndCapabilityCases = () => {
if (!resolved.ok || resolved.value.profile === null)
throw new Error("expected a configured profile");
expect(resolved.value.profile.parameters.language_selection).toBe(
"configured-v1",
"configured",
);
expect(resolved.value.profile.parameters.language_id).toBe(
"RISCV:LE:64:default",
@@ -486,7 +486,7 @@ describe("RISC-V ilp32 admission", () => {
"elf32-exec-le-soft-float-generic-v1",
);
expect(parameters.language_id).toBe("RISCV:LE:32:default");
expect(parameters.language_selection).toBe("provider-pinned-v1");
expect(parameters.language_selection).toBe("provider-pinned");
const other = await ghidra().resolveAnalysisProfile(
ilp32(`${C3_ARCH.replace("_zmmul1p0", "")}_zaamo1p0`),
);
+19 -96
View File
@@ -12,6 +12,8 @@ import { ghidraFunctionDossier } from "../domain/ghidraValues.fixture.js";
import { silentLogger } from "../logger.js";
import { GhidraProvider } from "./GhidraProvider.js";
import { GhidraSessionError } from "./GhidraSessionError.js";
import { ghidraProducerIdentity } from "./GhidraProducerIdentity.js";
import type { ProducerIdentity } from "../filesystem/ProducerIdentity.js";
import { GHIDRA_SESSION_CAPABILITIES } from "./GhidraSessionValues.js";
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
@@ -315,110 +317,31 @@ describe("Ghidra relationship coverage replay", () => {
describe("Ghidra snapshot profile admission", () => {
it.each([
{
parameter: "value_flow_effect_scope",
current: "pcode-load-store-explicit-v1",
legacy: undefined,
change: "an edited bridge script",
producer: (current: ProducerIdentity) => ({
...current,
sources_sha256: "0".repeat(64),
}),
},
{
parameter: "byte_search_coverage",
current: "separate-loaded-overlays-v1",
legacy: undefined,
},
{
parameter: "inventory_search_projection",
current: "provider-inventory-facts-v1",
legacy: undefined,
},
{
parameter: "data_type_observations",
current: "bit-field-base-and-lookup-v1",
legacy: undefined,
},
{
parameter: "byte_search_boundary",
current: "contiguous-eligible-initialized-blocks-v1",
legacy: undefined,
},
{
parameter: "instruction_status",
current: "unlisted-and-containing-code-data-v3",
legacy: "unlisted-address-without-decode-v1",
},
{
parameter: "instruction_status",
current: "unlisted-and-containing-code-data-v3",
legacy: undefined,
},
{
parameter: "function_boundary_observations",
current: "ghidra-function-boundary-limitations-v5",
legacy: "ghidra-function-boundary-limitations-v4",
},
{
parameter: "instruction_references",
current: "exact-address-incoming-without-synthetic-entry-v3",
legacy: "exact-address-incoming-without-synthetic-entry-v2",
},
{
parameter: "instruction_references",
current: "exact-address-incoming-without-synthetic-entry-v3",
legacy: "exact-address-incoming-v1",
},
{
parameter: "instruction_references",
current: "exact-address-incoming-without-synthetic-entry-v3",
legacy: undefined,
},
{
parameter: "function_references",
current: "complete-body-and-entry-reference-manager-v3",
legacy: "complete-body-and-entry-reference-manager-v2",
},
{
parameter: "native_value_flow_projection",
current: "accepted-runtime-budgets-v1",
legacy: undefined,
},
{
parameter: "call_relationship_coverage",
current: "observed-call-inventory-and-discovery-v7",
legacy: "resolved-call-reference-diagnostics-v1",
},
{
parameter: "call_relationship_coverage",
current: "observed-call-inventory-and-discovery-v7",
legacy: "decoded-call-target-coverage-v2",
},
{
parameter: "call_relationship_coverage",
current: "observed-call-inventory-and-discovery-v7",
legacy: "decoded-call-target-coverage-v3",
},
{
parameter: "call_relationship_coverage",
current: "observed-call-inventory-and-discovery-v7",
legacy: "function-body-call-coverage-v4",
},
{
parameter: "call_relationship_coverage",
current: "observed-call-inventory-and-discovery-v7",
legacy: "function-body-call-coverage-v5",
},
{
parameter: "call_relationship_coverage",
current: "observed-call-inventory-and-discovery-v7",
legacy: "function-body-call-coverage-v6",
change: "another REA release",
producer: (current: ProducerIdentity) => ({
...current,
rea_version: "0.0.0-earlier",
}),
},
{ change: "no producer identity", producer: () => undefined },
])(
"refuses snapshots with outdated $parameter semantics",
async ({ parameter, current, legacy: prior }) => {
"refuses snapshots from $change without a revision marker",
async ({ producer }) => {
const initial = fixture();
await open(initial.session);
const snapshot = initial.session.exportAnalysisSnapshot();
if (!snapshot.ok) throw snapshot.error;
await initial.session.close();
const profile = snapshot.value.binding.analysis_profile;
expect(profile.parameters[parameter]).toBe(current);
expect(profile.parameters.producer).toEqual(ghidraProducerIdentity());
const prior = producer(ghidraProducerIdentity());
const legacy = {
...snapshot.value,
binding: {
@@ -426,10 +349,10 @@ describe("Ghidra snapshot profile admission", () => {
analysis_profile: createAnalysisProfile(profile.provider, {
...Object.fromEntries(
Object.entries(profile.parameters).filter(
([key]) => key !== parameter,
([key]) => key !== "producer",
),
),
...(prior === undefined ? {} : { [parameter]: prior }),
...(prior === undefined ? {} : { producer: prior }),
}),
},
};
+1 -1
View File
@@ -9,8 +9,8 @@ import {
mapArtifactFileFailure,
ArtifactResourceScope,
ArtifactReaderFailure,
readArtifactFileChunks,
} from "../resources/index.js";
import { readArtifactFileChunks } from "../resources/index.js";
import {
NonRegularFileReadError,
sameRegularFileState,
+1 -1
View File
@@ -14,8 +14,8 @@ import {
artifactFileCancellationCause,
bindArtifactFileCleanup,
withArtifactRegularFile,
readArtifactFileChunks,
} from "../resources/index.js";
import { readArtifactFileChunks } from "../resources/index.js";
import { sameRegularFileState } from "../filesystem/RegularFile.js";
import {
GhidraSessionError,
+2 -1
View File
@@ -54,6 +54,7 @@ import {
type HopperMachOImageResolutionFileSystem,
type HopperMachOImage,
} from "./HopperMachOImage.js";
import { hopperProducerIdentity } from "./HopperBridgeSources.js";
interface HopperProfileOptions {
readonly launcherPath: string;
@@ -180,7 +181,7 @@ const resolveHopperAnalysisProfileInScope = async (
: {}),
target_kind: target.kind,
target_format: target.format,
function_dossier_coverage: "observed-facets-and-call-relationships-v1",
producer: hopperProducerIdentity(),
architecture: target.architecture ?? null,
available_architectures: [
...(target.availableArchitectures ?? []),
+20
View File
@@ -0,0 +1,20 @@
import { packagedProducerIdentity } from "../filesystem/ProducerIdentity.js";
const BRIDGE_DIRECTORY = new URL("../../bridge/", import.meta.url);
const BRIDGE_SCRIPT = "hopper_bridge.py";
/** The packaged Python bridge Hopper runs for every REA operation. */
export const HOPPER_BRIDGE_SCRIPT_URL = new URL(
BRIDGE_SCRIPT,
BRIDGE_DIRECTORY,
);
/**
* The REA release and bridge script behind every Hopper result. Analysis
* profiles commit to it, so an edited bridge or a new REA release yields a new
* profile digest and earlier snapshots are not reused.
*/
export const hopperProducerIdentity = packagedProducerIdentity(
BRIDGE_DIRECTORY,
[BRIDGE_SCRIPT],
);
+3 -3
View File
@@ -8,6 +8,8 @@ import {
import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import { fileURLToPath } from "node:url";
import { HOPPER_BRIDGE_SCRIPT_URL } from "./HopperBridgeSources.js";
import { accessSync, constants, statSync } from "node:fs";
import { createAnalysisExecution } from "../ports/AnalysisProvider.js";
@@ -338,9 +340,7 @@ export class HopperProvider implements AnalysisProviderCandidate {
this.config.hopperLoaderArgs.length > 0
? this.config.hopperLoaderArgs
: derivedLoaderArgs.value,
bridgeScriptPath: fileURLToPath(
new URL("../../bridge/hopper_bridge.py", import.meta.url),
),
bridgeScriptPath: fileURLToPath(HOPPER_BRIDGE_SCRIPT_URL),
bridgeMaxRequestBytes: this.config.bridgeMaxRequestBytes,
...(this.platform === "linux"
? {
+1 -1
View File
@@ -5,6 +5,7 @@ import {
withArtifactRegularFile,
ArtifactReaderFailure,
ArtifactResourceScope,
normalizeArtifactPath,
} from "../../resources/index.js";
import { fileSnapshotPartialObservationSchema } from "../../domain/fileSnapshotObservation.js";
import { OwnedDirectoryHandle } from "../../filesystem/OwnedDirectoryHandle.js";
@@ -18,7 +19,6 @@ import {
writeFile,
} from "node:fs/promises";
import { basename, dirname, isAbsolute, join, relative, sep } from "node:path";
import { normalizeArtifactPath } from "../../resources/index.js";
import {
AnalysisAccessDeniedError,
AnalysisArtifactChangedError,
+4 -2
View File
@@ -4,8 +4,10 @@ import {
createAnalysisExecution,
type ExecutionOptions,
} from "../../ports/AnalysisProvider.js";
import type { SafeOutputTree } from "../../resources/index.js";
import type { ArtifactResourceScope } from "../../resources/index.js";
import type {
SafeOutputTree,
ArtifactResourceScope,
} from "../../resources/index.js";
import {
AnalysisArtifactChangedError,
AnalysisCapabilityUnavailableError,
+4 -2
View File
@@ -1,7 +1,9 @@
import { join } from "node:path";
import { Readable } from "node:stream";
import type { SafeOutputTree } from "../../resources/index.js";
import type { ArtifactResourceScope } from "../../resources/index.js";
import type {
SafeOutputTree,
ArtifactResourceScope,
} from "../../resources/index.js";
import { AnalysisOutputError } from "../../domain/analysisErrorCore.js";
import {
javascriptRecoveryResultSchema,
+1 -3
View File
@@ -3,12 +3,10 @@ import { safeParseUtf8Json } from "../../domain/safeJson.js";
import {
ArtifactReaderFailure,
type ArtifactResourceScope,
} from "../../resources/index.js";
import type { AnalysisCapturedOutput } from "../../domain/analysisErrorBase.js";
import {
normalizeArtifactPath,
ArtifactPathRegistry,
} from "../../resources/index.js";
import type { AnalysisCapturedOutput } from "../../domain/analysisErrorBase.js";
import {
AnalysisAccessDeniedError,
AnalysisArtifactChangedError,
+13 -7
View File
@@ -72,9 +72,7 @@ import type {
KunaProjectFunction,
} from "./KunaProjectFiles.js";
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
/** Bumped whenever the entry layout or manifest meaning changes. */
const CACHE_LAYOUT = 3;
import { releaseProducerIdentity } from "../filesystem/ProducerIdentity.js";
/** A manifest names five files and their digests; anything larger is not ours. */
const MANIFEST_MAX_BYTES = 64 * 1024;
@@ -103,7 +101,9 @@ export const kunaProjectKey = (inputs: KunaProjectKeyInputs): string =>
createHash("sha256")
.update(
canonicalize({
layout: CACHE_LAYOUT,
// The entry layout and manifest meaning are REA's code: a new release
// keys new entries, and older ones are retired as foreign manifests.
producer: releaseProducerIdentity(),
target_sha256: inputs.targetSha256,
target_basename: inputs.targetBasename,
kuna_sha256: inputs.kunaSha256,
@@ -123,7 +123,7 @@ const fileSchema = z.strictObject({
const manifestSchema = z
.strictObject({
layout: z.literal(CACHE_LAYOUT),
producer: z.strictObject({ rea_version: z.string().min(1) }),
key: z.string().regex(KEY_PATTERN),
inputs: z.strictObject({
target_sha256: z.string().regex(SHA256_PATTERN),
@@ -147,7 +147,13 @@ const manifestSchema = z
}),
created_at: utcDateTimeSchema,
})
.superRefine(({ files, inputs, key }, context) => {
.superRefine(({ files, inputs, key, producer }, context) => {
if (producer.rea_version !== releaseProducerIdentity().rea_version)
context.addIssue({
code: "custom",
path: ["producer"],
message: "Export cache entry was produced by another REA release",
});
if (
key !==
kunaProjectKey({
@@ -1127,7 +1133,7 @@ export class KunaProjectCache {
signal: AbortSignal,
): Promise<Result<KunaProject, AnalysisError>> {
const manifest: KunaProjectManifest = {
layout: CACHE_LAYOUT,
producer: releaseProducerIdentity(),
key,
inputs: {
target_sha256: request.targetSha256,
@@ -180,7 +180,7 @@ describe.skipIf(process.platform === "win32")(
language_id: "Allegrex:LE:32:default",
compiler_spec_id: "default",
loader: "PspElfLoader",
mips_support_lane: "psp-elf32-exec-eabi32-allegrex-v2",
mips_support_lane: "psp-elf32-exec-eabi32-allegrex",
mips_elf: {
elf_class: 32,
byte_order: "little",
@@ -0,0 +1,60 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { pathToFileURL } from "node:url";
import { describe, expect, it } from "vitest";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { PRODUCT_IDENTITY } from "../../../src/identity.js";
import {
packagedProducerIdentity,
packagedSourcesDigest,
} from "../../../src/filesystem/ProducerIdentity.js";
const sources = async (files: Readonly<Record<string, string>>) => {
const root = await createTestTempDirectory("rea-producer-");
for (const [name, text] of Object.entries(files))
await writeFile(join(root, name), text);
return pathToFileURL(`${root}/`);
};
describe("packaged producer identity", () => {
it("changes when a producer script changes, without a revision marker", async () => {
const directory = await sources({ "a.java": "one", "b.py": "two" });
const before = packagedSourcesDigest(directory, ["a.java", "b.py"]);
await writeFile(new URL("b.py", directory), "two, edited");
expect(packagedSourcesDigest(directory, ["a.java", "b.py"])).not.toBe(
before,
);
});
it("frames each source by name and length so bytes cannot move between files", async () => {
const left = await sources({ "a.java": "ab", "b.py": "c" });
const right = await sources({ "a.java": "a", "b.py": "bc" });
expect(packagedSourcesDigest(left, ["a.java", "b.py"])).not.toBe(
packagedSourcesDigest(right, ["a.java", "b.py"]),
);
});
it("does not depend on the order sources are listed in", async () => {
const directory = await sources({ "a.java": "one", "b.py": "two" });
expect(packagedSourcesDigest(directory, ["b.py", "a.java"])).toBe(
packagedSourcesDigest(directory, ["a.java", "b.py"]),
);
});
it("refuses an identity when a declared source is not packaged", async () => {
const directory = await sources({ "a.java": "one" });
expect(() =>
packagedSourcesDigest(directory, ["a.java", "missing.py"]),
).toThrow(/ENOENT/u);
});
it("commits the REA release beside the source digest", async () => {
const directory = await sources({ "a.java": "one" });
expect(packagedProducerIdentity(directory, ["a.java"])()).toEqual({
rea_version: PRODUCT_IDENTITY.packageVersion,
sources_sha256: packagedSourcesDigest(directory, ["a.java"]),
});
});
});
+3 -3
View File
@@ -4,6 +4,7 @@ import { describe, expect, it } from "vitest";
import { z } from "zod";
import { EnhancedTools } from "../../../src/application/EnhancedTools.js";
import { connectGhidraMcp, sessionEvidence } from "./ghidraMcpHarness.js";
import { ghidraProducerIdentity } from "../../../src/ghidra/GhidraProducerIdentity.js";
import { functionDossierSchema } from "../../../src/domain/native/functionDossier.js";
import { functionComparisonResultSchema } from "../../../src/domain/functionComparisonSchemas.js";
import {
@@ -190,8 +191,7 @@ it("retains terminal-call qualifications across live dossiers and snapshot-backe
expect(evidence.raw_result, name).toBeNull();
expect(evidence.limitations, name).toContain(limitation);
expect(evidence.analysis_profile?.parameters).toMatchObject({
function_boundary_observations:
"ghidra-function-boundary-limitations-v5",
producer: ghidraProducerIdentity(),
});
const replay = await harness.session.execute(name, evidence.parameters);
if (!replay.ok) throw replay.error;
@@ -611,7 +611,7 @@ describe("Ghidra MCP evidence parity", () => {
provider: { id: "ghidra", version: "12.1.4" },
parameters: {
import_mode: "ephemeral-source-immutable",
annotation_policy: "atomic-function-entry-metadata-v1",
annotation_policy: "atomic-function-entry-metadata",
analyzer_preset: "ghidra-default",
},
},
@@ -17,6 +17,7 @@ import { fileURLToPath } from "node:url";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { GhidraClient } from "../../../src/ghidra/GhidraClient.js";
import { GhidraProvider } from "../../../src/ghidra/GhidraProvider.js";
import { ghidraProducerIdentity } from "../../../src/ghidra/GhidraProducerIdentity.js";
import { parseConfig } from "../../../src/config/parseConfig.js";
import { silentLogger } from "../../../src/logger.js";
import { composeBinarySession } from "../../../src/application/binary/BinarySessionComposition.js";
@@ -397,8 +398,8 @@ it("retains missing PDB coverage in empty PE name-search Evidence and snapshot r
expect(reply.isError).not.toBe(true);
const evidence = sessionEvidence(harness.session, reply.structuredContent);
expect(evidence.normalized_result).toEqual([]);
expect(evidence.analysis_profile?.parameters.pdb_observations).toBe(
"program-info-v5",
expect(evidence.analysis_profile?.parameters.producer).toEqual(
ghidraProducerIdentity(),
);
const limitation = evidence.limitations.find((value) =>
value.includes(pdb.file),
@@ -453,7 +454,7 @@ it("retains defined-string coverage in empty and matching search Evidence", asyn
);
expect(evidence.normalized_result).toEqual([]);
expect(evidence.analysis_profile?.parameters).toMatchObject({
string_inventory_evidence: "defined-data-coverage-v1",
producer: ghidraProducerIdentity(),
});
const coverage = evidence.limitations.find((limitation) =>
/Only Ghidra-defined string Data.*empty search does not establish/u.test(
@@ -683,7 +684,7 @@ it("returns a procedure once with the exact-entry names its search matched", asy
expect(exact.normalized_result).toEqual(output);
expect(exact.raw_result).toBeNull();
expect(exact.analysis_profile?.parameters).toMatchObject({
procedure_search: "primary-and-exact-entry-symbol-names-v1",
producer: ghidraProducerIdentity(),
});
// Node owns Unicode folding: the bridge's non-ASCII candidates are kept
@@ -45,11 +45,13 @@ it.each(["normal", "copied", "parameters", "forged", "source"] as const)(
() => ({
execute: async () =>
ok({
...createAnalysisExecution(["0x100"], provider, {
...(mode === "source" && source !== undefined
...createAnalysisExecution(
["0x100"],
provider,
mode === "source" && source !== undefined
? { sourceEvidence: [source] }
: {}),
}),
: {},
),
...(mode === "forged" && forged !== undefined
? { evidenceId: forged.evidence_id }
: {}),
@@ -12,6 +12,9 @@ import {
resolveHopperAnalysisProfile,
} from "../../../../src/hopper/HopperAnalysisProfile.js";
import { HOPPER_PROVIDER_IDENTITY } from "../../../../src/hopper/HopperProviderMetadata.js";
import { hopperProducerIdentity } from "../../../../src/hopper/HopperBridgeSources.js";
import { packagedSourcesDigest } from "../../../../src/filesystem/ProducerIdentity.js";
import { PRODUCT_IDENTITY } from "../../../../src/identity.js";
let inspectionResources = createInspectionResources();
let directory: string | undefined;
@@ -117,8 +120,7 @@ describe("Hopper analysis profiles", () => {
value: {
profile: {
parameters: {
function_dossier_coverage:
"observed-facets-and-call-relationships-v1",
producer: hopperProducerIdentity(),
loader: {
source: scenario.source,
arguments: scenario.arguments,
@@ -130,6 +132,29 @@ describe("Hopper analysis profiles", () => {
},
);
it("commits the REA release and packaged bridge script it runs", async () => {
directory = await createTestTempDirectory("rea-hopper-profile-producer-");
const launcherPath = join(directory, "hopper");
await writeFile(launcherPath, "Hopper build");
const resolved = await resolveHopperAnalysisProfile(
target("elf", "x86_64"),
{
launcherPath,
loaderArgsOverride: [],
provider: HOPPER_PROVIDER_IDENTITY,
resourceScope: inspectionResources.owner,
},
);
if (!resolved.ok) throw resolved.error;
expect(resolved.value.profile?.parameters.producer).toEqual({
rea_version: PRODUCT_IDENTITY.packageVersion,
sources_sha256: packagedSourcesDigest(
new URL("../../../../bridge/", import.meta.url),
["hopper_bridge.py"],
),
});
});
it("stops launcher hashing when profile resolution is cancelled", async () => {
directory = await createTestTempDirectory("rea-hopper-profile-cancel-");
const launcher = join(directory, "hopper");
@@ -15,6 +15,7 @@ import {
} from "node:fs/promises";
import { basename, dirname, join, relative } from "node:path";
import canonicalize from "canonicalize";
import { PRODUCT_IDENTITY } from "../../../../src/identity.js";
import { expect, it as test, vi } from "vitest";
@@ -112,8 +113,8 @@ const processGone = (pid: number): boolean => {
}
};
it.each([1, 2])(
"regenerates layout %i once and retires it without touching unrelated cache resources",
it.each([1, 2, 3, "release"] as const)(
"regenerates layout %s once and retires it without touching unrelated cache resources",
async (layout) => {
const { kuna, installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-cache-format-transition-",
@@ -126,13 +127,21 @@ it.each([1, 2])(
expect((await cache.close()).ok).toBe(true);
// Seed the previously shipped manifest and content-key format using a real
// authored export; omitted launcher identity must never be invented or reused.
// "release": the current entry format written by another REA release.
const marker =
layout === "release"
? { producer: { rea_version: "0.0.0-another-release" } }
: { layout };
const legacyKey = createHash("sha256")
.update(
canonicalize({
layout,
...marker,
target_sha256: input.targetSha256,
...(layout === 3 || layout === "release"
? { target_basename: basename(input.targetPath) }
: {}),
kuna_sha256: installation.sha256,
...(layout === 2
...(layout !== 1
? {
kuna_command: installation.configuration.command,
kuna_executable: installation.executable,
@@ -151,13 +160,18 @@ it.each([1, 2])(
await writeFile(
join(legacyEntry, "manifest.json"),
JSON.stringify({
...initial.value.manifest,
layout,
...(({ producer: _producer, ...manifest }) => manifest)(
initial.value.manifest,
),
...marker,
key: legacyKey,
inputs: {
target_sha256: input.targetSha256,
...(layout === 3 || layout === "release"
? { target_basename: basename(input.targetPath) }
: {}),
kuna_sha256: installation.sha256,
...(layout === 2
...(layout !== 1
? {
kuna_command: installation.configuration.command,
kuna_executable: installation.executable,
@@ -185,7 +199,7 @@ it.each([1, 2])(
});
expect(built.value.key).not.toBe(legacyKey);
expect(built.value.manifest).toMatchObject({
layout: 3,
producer: { rea_version: PRODUCT_IDENTITY.packageVersion },
inputs: {
kuna_command: installation.configuration.command,
kuna_executable: installation.executable,
@@ -256,7 +270,7 @@ it("publishes a verified export and serves later requests from the cache", async
value: {
key: keyOf(installation, input),
manifest: {
layout: 3,
producer: { rea_version: PRODUCT_IDENTITY.packageVersion },
inputs: {
target_sha256: input.targetSha256,
kuna_sha256: installation.sha256,