Files
reverse-engineering-anything/src/domain/binaryTargetTypes.ts
T
6822ebe1f3 feat(ida): support the official Hex-Rays IDA MCP headlessly (#4813)
* feat(ida): support the official Hex-Rays IDA MCP headlessly

Hex-Rays IDA MCP serves database reads only through execute_python, so
REA rejected it as an upstream. A `hexrays` headless stdio registration
now leases a private idalib worker through open_database, answers the
existing database-supervisor reads with a fixed, packaged, read-only
query module, and releases the lease with close_database, verified by
list_databases.

Observations record execute_python, the query and the module digest.
Tool errors are typed so a completed rejection is distinguished from an
unknown outcome; an unconfirmed release is retried and a late lease on
the private input is adopted.

Validated IDA source and bridge tests, static checks, docs, and npm run
verify:ida on Linux x64 against x86-64 and AArch64 ELF fixtures.

* feat(ida): name the Hex-Rays registration for its catalog

A registration without `protocol` that reaches the official Hex-Rays
server failed with only the missing database-supervisor tools. Startup
profile failures now report the advertised catalog, and when it is the
Hex-Rays one the message and recovery name the `hexrays` registration.

Refs #1416

* docs(ida): state Hex-Rays prerequisites and upstream session traces

Upstream needs IDA 9.4 or later with idalib, Python 3.11 or later, uv
and Git. The profile does not analyze databases open in the IDA GUI, and
the server keeps its own session traces outside REA's workspace.

Refs #1416

* fix(ida): skip inline data when scanning Hex-Rays calls

The call scan decoded every head that FuncItems yields. Inline data such
as a jump table can decode as a call and refer to a function or import,
which reported false callees and call-graph edges. Skip non-code heads
before decoding, as disassembly already does.

Refs #1416

* fix(ida): retain Hex-Rays health reads in evidence

The identity and readiness reads around each operation kept only their
parsed result, dropping the bridge digest and any engine output. Keep
them whole as health_observations beside the parsed metadata, in both
successful and partial-failure Evidence.

Refs #1416

* test(ida): list the Hex-Rays bridge in package fixtures

The packed-bridge verifier has required bridge/ida/hexrays_queries.py
since the Hex-Rays profile landed, but three package tests still built
their member lists without it. The verifier stopped at that file, so 21
cases failed or asserted on the wrong omission.

Refs #1416

* feat(ida): open existing IDA databases through the Hex-Rays profile

A headless open re-ran auto-analysis from the input binary, discarding
the names, types and comments of an existing database and taking far
longer on large targets. Target admission now identifies IDA databases
by their IDA1/IDA2 signature, and the hexrays profile opens a private
copy of one; Hopper and the other IDA profiles reject them.

The copy is required, not incidental: idalib repacks a database when
its last lease is released, even without a save. A database reports
the input it was created from, so the session is bound to the opened
copy's idb_path, and health now reports the input SHA-256 it records.

Validated with resolver, provider and session tests, check:fast,
docs:check and test:changed, plus a real Hex-Rays MCP run against a
1.4 GB SC2 .i64 whose original file was left byte-identical.

Refs #1416

* fix(ida): verify the private workspace of a database target by idb_path

* test(ida): fix Hex-Rays health stand-ins and Windows path and runtime teardown

* refactor(ida): resolve the registration once for target support

* docs(ida): record the real Hex-Rays run on an existing .i64

* docs(ida): make the Hex-Rays registration example install-neutral

* chore: keep merge-hook formatting outside the IDA change

* fix(ida): retain live Hex-Rays databases this server does not lease

A list_databases row with instance_id null is a live owner, not a stopped
database. Mapping it to the supervisor's released-worker shape made cleanup
delete files another client still held. Unleased rows stay active, so the
private path must leave inventory before the workspace is removed.

Accept idat inventory rows, hash the query bridge with sha256Hex, and
complete databaseEngine on Hopper fixtures added on main.

* fix(ida): report uncapped Hex-Rays call graphs

* docs(ida): link to the Hex-Rays bridge on GitHub

* fix(ida): verify Hex-Rays worker identity and cleanup

* feat(ida): support Hex-Rays idat headless workers

Refs #12921

* fix(ida): retain workspace after ambiguous Hex-Rays startup

* fix(ida): preserve typed Hex-Rays call edges and deadlines

* test(ida): verify Hex-Rays cleanup with a peer lease

* test(ci): use project deadlines for the Cutter verifier

* test(ida): keep verifier cleanup fixtures scoped to mocked modules

* fix(ida): retain the worker backend in partial evidence

---------

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-11-20 17:58:27 -05:00

367 lines
13 KiB
TypeScript

import type { AnalysisDatabaseEngine } from "./analysisDatabase.js";
import type { DolHeader } from "./gamecubeDol.js";
import type { TargetFormatsOfKind } from "./targetFormats.js";
/** Provider-neutral CPU families detected from headers or explicitly selected raw loads. */
export const BINARY_ARCHITECTURES = [
"x86",
"x86_64",
"arm",
"arm64",
"arm64_32",
"mips",
"6502",
"riscv",
"powerpc",
"m68k",
"hexagon",
"xtensa",
] as const;
export type BinaryArchitecture = (typeof BINARY_ARCHITECTURES)[number];
/**
* Families admitted only by explicit raw-binary selection. Header detection
* never yields them, so detected targets and Evidence subjects exclude them.
*/
export const RAW_ONLY_ARCHITECTURES = [
"6502",
"m68k",
] as const satisfies readonly BinaryArchitecture[];
export type RawOnlyArchitecture = (typeof RAW_ONLY_ARCHITECTURES)[number];
/**
* ELF header facts for a family whose ISA and ABI are not established by the
* family name alone. Consumers admit a lane from these declarations rather than
* re-reading the header.
*/
export interface ElfFamilyMetadata {
readonly elfClass: 32 | 64;
readonly byteOrder: "little" | "big";
readonly type: number;
readonly flags: number;
readonly hasInterpreter?: boolean;
}
/** Prefix shared by CodeView RVA mapping and both payload readers. */
export const PE_CODEVIEW_PREFIX_BYTES = 4096;
export type PeCodeViewPath =
| {
readonly pdbPath: string;
readonly pdbFileName: string;
}
| {
readonly pdbPath: null;
readonly pdbFileName: null;
/** Exact bytes before the record's NUL terminator, with unknown text decoding. */
readonly pdbPathBytesHex: string;
};
/** Referenced CodeView debug symbols from a PE debug directory. */
export type PeCodeViewRecord = PeCodeViewPath & {
readonly guid: string | null;
readonly age: number;
};
/** Families whose parsed target variant carries `ElfFamilyMetadata`. */
export const ELF_METADATA_ARCHITECTURES = [
"mips",
"riscv",
"powerpc",
] as const satisfies readonly BinaryArchitecture[];
/** Families that require a specialized target variant instead of generic headers. */
export const HEADER_ARCHITECTURE_EXCLUSIONS = [
...ELF_METADATA_ARCHITECTURES,
...RAW_ONLY_ARCHITECTURES,
"hexagon",
"xtensa",
"arm64_32",
] as const satisfies readonly BinaryArchitecture[];
/** Recognized family with no specialized target member; the header alone is the fact. */
export type HeaderArchitecture = Exclude<
BinaryArchitecture,
(typeof HEADER_ARCHITECTURE_EXCLUSIONS)[number]
>;
/** Raw ELF facts; family recognition does not establish ISA or ABI support. */
export interface MipsElfMetadata extends ElfFamilyMetadata {
/** Undefined until inspected; null means no ABI record was observed. */
readonly abiFlags?: MipsAbiFlags | null;
/** Undefined until inspected; null means no GNU attribute section was observed. */
readonly gnuAttributes?: MipsGnuAttributes | null;
/** Unavailable declaration sources; missing records there remain unknown. */
readonly limitations?: readonly string[];
/** Complete dynamic REL declarations, or the reason they could not be inspected. */
readonly sharedLoad?:
| {
readonly status: "complete";
readonly relocationCount: number;
/** The dynamic table declares DF_1_PIE. */
readonly declaresPie?: true;
readonly relocationTypes: readonly number[];
readonly symbolRelativeCount: number;
/** Every PT_LOAD virtual address with its program-header index, in header order. */
readonly loadAddresses: readonly {
readonly headerIndex: number;
readonly virtualAddress: number;
}[];
/** Android prelink trailer base; null when the file has no trailer. */
readonly prelinkBase: number | null;
}
| { readonly status: "unavailable"; readonly reason: string };
}
/** Raw ELF facts; family recognition does not establish ISA or ABI support. */
export interface RiscvElfMetadata extends ElfFamilyMetadata {
/** Undefined until `.riscv.attributes` is inspected. */
readonly isa?: RiscvIsaDeclaration;
}
/** The ISA an image declares in `.riscv.attributes`; not proof of every instruction. */
export type RiscvIsaDeclaration =
| {
readonly status: "declared";
/** `Tag_RISCV_arch` exactly as declared. */
readonly arch: string;
readonly xlen: 32 | 64;
readonly base: "i" | "e";
/** Extension names in declaration order, without versions. */
readonly extensions: readonly string[];
}
/** Every section header was examined and none declares `Tag_RISCV_arch`. */
| { readonly status: "absent" }
| {
readonly status: "unavailable";
readonly reason: string;
/** The declared string, when one was read but not understood. */
readonly arch?: string;
};
/** Declared fields of the 24-byte MIPS ABI flags record, not inferred semantics. */
export interface MipsAbiFlags {
readonly version: number;
readonly isaLevel: number;
readonly isaRevision: number;
readonly gprSize: number;
readonly cpr1Size: number;
readonly cpr2Size: number;
readonly fpAbi: number;
readonly isaExtension: number;
readonly ases: number;
readonly flags1: number;
readonly flags2: number;
}
/** File-scope GNU object attributes parsed from `SHT_GNU_ATTRIBUTES`. */
export interface MipsGnuAttributes {
/** Tag_GNU_MIPS_ABI_FP; null means this file did not declare the tag. */
readonly fpAbi: number | null;
/** Tag_GNU_MIPS_ABI_MSA vector-argument ABI; null means undeclared. */
readonly msa: number | null;
}
/** Byte range of the selected slice inside its universal Mach-O container. */
export interface UniversalMachOSlice {
readonly offset: number;
readonly size: number;
}
/** FAT-table identity; CPU family alone cannot identify a universal slice. */
export interface UniversalMachOSliceIdentity extends UniversalMachOSlice {
readonly cpuType: number;
readonly cpuSubtype: number;
}
interface BinaryTargetIdentity {
readonly path: string;
readonly sourcePath?: string;
/** Verbatim selected spelling when it differs from the inspected file path. */
readonly callerInputPath?: string;
/** Selected artifact spelling determines name-based format hints and external companions. */
readonly artifactSelectedPath?: string;
/** Info.plist of the app bundle the target was resolved from. */
readonly bundleInfoPlist?: string;
readonly sha256: string;
}
type NonExecutableMetadata = {
readonly architecture?: never;
readonly availableArchitectures?: never;
readonly executableRole?: never;
readonly managed?: never;
};
type ExecutableTarget = BinaryTargetIdentity & {
readonly kind: "executable";
readonly architecture: HeaderArchitecture;
readonly availableArchitectures: readonly HeaderArchitecture[];
};
/**
* Canonical local target identity and provider-neutral file classification.
* Each variant carries only metadata established for that exact file kind.
*/
export type BinaryTarget =
| (BinaryTargetIdentity & {
readonly kind: "executable";
readonly format: "elf";
readonly architecture: "mips";
readonly availableArchitectures: readonly "mips"[];
readonly mips: MipsElfMetadata;
readonly executableRole?: never;
readonly managed?: never;
})
| (BinaryTargetIdentity & {
readonly kind: "executable";
readonly format: "elf";
readonly architecture: "riscv";
readonly availableArchitectures: readonly "riscv"[];
readonly riscv: RiscvElfMetadata;
readonly executableRole?: never;
readonly managed?: never;
})
| (BinaryTargetIdentity & {
readonly kind: "executable";
readonly format: "elf";
readonly architecture: "powerpc";
readonly availableArchitectures: readonly "powerpc"[];
/** Family identity does not select a processor variant. */
readonly powerpc: ElfFamilyMetadata;
readonly executableRole?: never;
readonly managed?: never;
})
| (BinaryTargetIdentity & {
readonly kind: "executable";
readonly format: "elf";
readonly architecture: "hexagon";
readonly availableArchitectures: readonly "hexagon"[];
readonly executableRole?: never;
readonly managed?: never;
})
| (BinaryTargetIdentity & {
readonly kind: "executable";
readonly format: "elf";
readonly architecture: "xtensa";
readonly availableArchitectures: readonly "xtensa"[];
readonly executableRole?: never;
readonly managed?: never;
})
| (BinaryTargetIdentity & {
readonly kind: "executable";
readonly format: "mach-o";
readonly architecture: "arm64_32";
readonly availableArchitectures: readonly "arm64_32"[];
readonly universalSlice?: never;
readonly executableRole?: never;
readonly managed?: never;
})
| (ExecutableTarget & {
readonly format: "mach-o" | "elf" | "dos-mz" | "dos-com";
readonly universalSlice?: never;
readonly executableRole?: never;
readonly managed?: never;
})
| (ExecutableTarget & {
readonly format: "mach-o-universal";
/** Selected slice range a provider must analyze instead of the container. */
readonly universalSlice: UniversalMachOSlice;
/** Exact validated FAT declarations, when supplied by target admission. */
readonly universalSlices?: readonly UniversalMachOSliceIdentity[];
readonly executableRole?: never;
readonly managed?: never;
})
| (BinaryTargetIdentity & {
readonly kind: "executable";
readonly format: "raw-binary";
readonly architecture: HeaderArchitecture | RawOnlyArchitecture;
readonly availableArchitectures: readonly (
| HeaderArchitecture
| RawOnlyArchitecture
)[];
readonly entryPoint: string;
readonly entryPointSource: string;
readonly executableRole?: never;
readonly managed?: never;
})
| (BinaryTargetIdentity & {
readonly kind: "executable";
readonly format: "gamecube-dol";
/** Fixed by the DOL selection; the header does not carry a CPU family. */
readonly architecture: "powerpc";
readonly availableArchitectures: readonly ["powerpc"];
readonly byteOrder: "big";
readonly bits: 32;
/** Canonical hex entry taken from the DOL header. */
readonly entryPoint: string;
readonly entryPointSource: "dol-header";
/** Validated header: used sections, BSS, and BSS-overlapping data slots. */
readonly dol: DolHeader;
readonly executableRole?: never;
readonly managed?: never;
})
| (ExecutableTarget & {
readonly format: "pe";
/** Original COFF machine; undefined means this header fact is unavailable. */
readonly peMachine?: number;
/** Provider-neutral PE image role observed from COFF characteristics. */
readonly executableRole:
| "application"
| "shared-library"
| "non-executable";
/** Whether the PE declares a non-empty CLI header data-directory entry. */
readonly managed: boolean;
/** Referenced CodeView debug symbols from the PE debug directory, if present. */
readonly codeView?: PeCodeViewRecord;
})
| (BinaryTargetIdentity &
NonExecutableMetadata & {
readonly kind: "database";
readonly format: "analysis-database";
/** Engine identified at admission; null when the database is unidentified. */
readonly databaseEngine: AnalysisDatabaseEngine | null;
})
| (BinaryTargetIdentity &
NonExecutableMetadata & {
readonly kind: "archive";
readonly format: Exclude<
TargetFormatsOfKind<"archive">,
"esp-flash-image"
>;
})
| (BinaryTargetIdentity &
NonExecutableMetadata & {
readonly kind: "archive";
readonly format: "esp-flash-image";
/** How the dump is read; null inputs take their defaults. */
readonly espFlash: EspFlashTargetSelection;
})
| (BinaryTargetIdentity &
NonExecutableMetadata & {
readonly kind: "artifact";
readonly format: TargetFormatsOfKind<"artifact">;
});
/** Caller inputs that select how an ESP flash dump is read. */
export interface EspFlashTargetSelection {
/** The caller selected the format, so the table scan runs without a bootloader. */
readonly selected: boolean;
/** Flash address of the dump's first byte. */
readonly flashOffset: number | null;
/** Flash address of the partition table. */
readonly partitionTableOffset: number | null;
}
/** Whether an executable target contains a thin or universal Mach-O image. */
export const isMachOFormat = (
format: unknown,
): format is "mach-o" | "mach-o-universal" =>
format === "mach-o" || format === "mach-o-universal";
/** Result-affecting artifact selection; canonical packed identity remains target.path. */
export const artifactTargetSelectedPath = (target: BinaryTarget): string =>
target.kind === "archive" || target.kind === "artifact"
? (target.artifactSelectedPath ?? target.sourcePath ?? target.path)
: (target.sourcePath ?? target.path);