diff --git a/docs/guides/native/go-binaries.md b/docs/guides/native/go-binaries.md index 7df4ec5c7d..5ed8bf52ad 100644 --- a/docs/guides/native/go-binaries.md +++ b/docs/guides/native/go-binaries.md @@ -163,15 +163,31 @@ read within individual segments or sections. The historical two-word string descriptor follows the [Go 1.17.13 reader](https://github.com/golang/go/blob/go1.17.13/src/cmd/go/internal/version/version.go#L212). -A universal (FAT) Mach-O needs `architecture` (`x86`, `x86_64`, `arm` -or `arm64`; CLI `--architecture`). REA reads its own FAT table, without `lipo`, -and validates the selected slice's range, alignment and Mach-O header CPU -against the table. The result keeps the universal file as `artifact` (its -SHA-256 and size), adds `selected_slice` with the slice offset, length, raw CPU -type/subtype and the FAT table digest, and reports every source range as a -file offset inside that slice. Without `architecture`, or when the file has no -such slice or several, the request fails on `architecture` and lists the slices -present. `architecture` on any other image is rejected rather than ignored. +A universal (FAT) Mach-O needs `architecture` (`x86`, `x86_64`, `arm`, `arm64` +or `arm64e`; CLI `--architecture`). REA reads its own FAT table, without `lipo`, +rejects a table whose slices overlap, and validates the selected slice's range, +alignment and Mach-O header CPU against the table. The result keeps the +universal file as `artifact` (its SHA-256 and size), adds `selected_slice` with +the slice offset, length, raw CPU type/subtype and the FAT table digest, and +reports every source range as a file offset inside that slice. Without +`architecture`, or when the file has no slice of that CPU family, the request +fails on `architecture` and lists the slices present. A CPU family with one +slice is selected by its family name, so `arm64` also reads a lone `arm64e` +slice; `arm64e` reads only an `arm64e` slice. When several slices share the +family, the value must name the slice's subtype, as `lipo -thin` does: `arm64` +reads the `arm64` (or `arm64v8`) slice, `arm64e` the `arm64e` (or `arm64e.x1`) +slice, and `x86_64` the canonical `x86_64` slice. Capability bits are ignored +when subtypes are compared. REA does not choose a slice by preference. When no +slice or several slices carry the named subtype, the request fails on +`architecture` and names any other value that selects exactly one verified +slice; `arm64e` beside `arm64e.x1`, for example, is refused. Other subtypes, +such as `x86_64h` and the `x86` (i386) and 32-bit `arm` subtypes, have no value +of their own; they are read only as the only slice of their family. +`limitations` lists the same-family slices that were not read and the value, if +any, that selects each. A slice whose Mach-O header does not match its FAT +declaration, such as a different CPU subtype, is listed as unverified, and no +value is offered for it. `architecture` on any other image is rejected rather +than ignored. For ELF images without a named build-info section, discovery checks each nonempty writable, non-executable load segment in program-header order. This diff --git a/docs/guides/native/native-initializers.md b/docs/guides/native/native-initializers.md index acd425439d..9a1813a8a0 100644 --- a/docs/guides/native/native-initializers.md +++ b/docs/guides/native/native-initializers.md @@ -20,8 +20,21 @@ requires an absolute filesystem path: } ``` -A universal Mach-O requires `architecture` to select one slice. ELF files and -thin Mach-O files do not take it. +A universal Mach-O requires `architecture` (`x86`, `x86_64`, `arm`, `arm64` or +`arm64e`) to select one slice. A CPU family with one slice is selected by its +family name; `arm64e` reads only an `arm64e` slice. When several slices share +the family, such as the `arm64` and `arm64e` slices of stock macOS tools, the +value must name the slice's subtype, as `lipo -thin` does: `arm64` reads the +`arm64` (or `arm64v8`) slice, `arm64e` the `arm64e` (or `arm64e.x1`) slice, and +`x86_64` the canonical `x86_64` slice. REA does not choose a slice by +preference. When no slice or several slices carry the named subtype (capability +bits are ignored when comparing), the request fails on `architecture`; `arm64e` +beside `arm64e.x1`, for example, is refused. `slice` reports the selected +subtype, and `limitations` lists the same-family slices that were not read and +the value, if any, that selects each. A slice whose Mach-O header does not match +its FAT declaration, such as a different CPU subtype, is listed as unverified, +and no value is offered for it. ELF files and thin Mach-O files do not take +`architecture`. ## Sources diff --git a/docs/rust-binaries.md b/docs/rust-binaries.md index 53f34882fb..d4274ddf8a 100644 --- a/docs/rust-binaries.md +++ b/docs/rust-binaries.md @@ -161,15 +161,31 @@ packages or the compiler. ELF, PE, thin Mach-O and one selected slice of a universal Mach-O are the supported containers. -A universal (FAT) Mach-O needs `architecture` (`x86`, `x86_64`, `arm` -or `arm64`; CLI `--architecture`). REA reads its own FAT table, without `lipo`, -and validates the selected slice's range, alignment and Mach-O header CPU -against the table. The result keeps the universal file as `artifact` (its -SHA-256 and size), adds `selected_slice` with the slice offset, length, raw CPU -type/subtype and the FAT table digest, and reports the section and rustc-path offsets as a -file offset inside that slice. Without `architecture`, or when the file has no -such slice or several, the request fails on `architecture` and lists the slices -present. `architecture` on any other image is rejected rather than ignored. +A universal (FAT) Mach-O needs `architecture` (`x86`, `x86_64`, `arm`, `arm64` +or `arm64e`; CLI `--architecture`). REA reads its own FAT table, without `lipo`, +rejects a table whose slices overlap, and validates the selected slice's range, +alignment and Mach-O header CPU against the table. The result keeps the +universal file as `artifact` (its SHA-256 and size), adds `selected_slice` with +the slice offset, length, raw CPU type/subtype and the FAT table digest, and +reports the section and rustc-path offsets as a file offset inside that slice. +Without `architecture`, or when the file has no slice of that CPU family, the +request fails on `architecture` and lists the slices present. A CPU family with +one slice is selected by its family name, so `arm64` also reads a lone `arm64e` +slice; `arm64e` reads only an `arm64e` slice. When several slices share the +family, the value must name the slice's subtype, as `lipo -thin` does: `arm64` +reads the `arm64` (or `arm64v8`) slice, `arm64e` the `arm64e` (or `arm64e.x1`) +slice, and `x86_64` the canonical `x86_64` slice. Capability bits are ignored +when subtypes are compared. REA does not choose a slice by preference. When no +slice or several slices carry the named subtype, the request fails on +`architecture` and names any other value that selects exactly one verified +slice; `arm64e` beside `arm64e.x1`, for example, is refused. Other subtypes, +such as `x86_64h` and the `x86` (i386) and 32-bit `arm` subtypes, have no value +of their own; they are read only as the only slice of their family. +`limitations` lists the same-family slices that were not read and the value, if +any, that selects each. A slice whose Mach-O header does not match its FAT +declaration, such as a different CPU subtype, is listed as unverified, and no +value is offered for it. `architecture` on any other image is rejected rather +than ignored. Package and dependency counts are checked against the minimum required JSON representation before graph allocation or schema cloning. Oversized graphs diff --git a/src/application/binaryDiagnostics/NativeInitializersService.test.ts b/src/application/binaryDiagnostics/NativeInitializersService.test.ts index 317b52f62e..5031d89651 100644 --- a/src/application/binaryDiagnostics/NativeInitializersService.test.ts +++ b/src/application/binaryDiagnostics/NativeInitializersService.test.ts @@ -242,6 +242,30 @@ it("refuses a slice architecture label that disagrees with the selected image", }); }); +it("accepts an arm64e request only for a report of an arm64e slice", async () => { + const selected = await selectedArm64Report(); + const inspect = (cpuSubtype: number) => + serviceFor({ + ...selected, + slice: { ...selected.slice, cpu_subtype: cpuSubtype }, + }).inspect({ path: report.artifact.path, architecture: "arm64e" }); + // arm64 (0x0) and arm64v8 (0x1) are not what arm64e names. + for (const cpuSubtype of [0, 1]) + expect(await inspect(cpuSubtype)).toMatchObject({ + ok: false, + error: { _tag: "AnalysisOutputError" }, + }); + // arm64e with capability bits, and arm64e.x1. + for (const cpuSubtype of [0x80000002, 12]) + expect((await inspect(cpuSubtype)).ok).toBe(true); + // A family name may read the family's only slice, such as /bin/ls's arm64e. + const lone = await serviceFor({ + ...selected, + slice: { ...selected.slice, cpu_subtype: 0x80000002 }, + }).inspect({ path: report.artifact.path, architecture: "arm64" }); + expect(lone.ok).toBe(true); +}); + it.each([ { field: "slot_file_offset" as const }, { field: "value_file_offset" as const }, diff --git a/src/application/binaryDiagnostics/NativeInitializersService.ts b/src/application/binaryDiagnostics/NativeInitializersService.ts index af234d3332..8f3525847a 100644 --- a/src/application/binaryDiagnostics/NativeInitializersService.ts +++ b/src/application/binaryDiagnostics/NativeInitializersService.ts @@ -13,7 +13,10 @@ import { import { err, type Result } from "../../domain/result.js"; import { completeProviderEvidence } from "../ProviderOperationExecution.js"; import { parseAnalysisInput } from "../AnalysisInput.js"; -import { MACHO_CPU_TYPES } from "../../domain/apple/machoTargetSelection.js"; +import { + machoSelectionAnswersArchitecture, + machoSliceFamily, +} from "../../domain/apple/machoTargetSelection.js"; const OPERATION = "inspect_native_initializers"; @@ -96,15 +99,16 @@ const validReport = ( ): boolean => { const { coverage } = report; const slice = report.slice; + if (!machoSelectionAnswersArchitecture(slice, input.architecture)) + return false; + // Slice and report labels name the CPU family; arm64e selects an arm64 slice. if ( - slice === null - ? input.architecture !== undefined - : report.format !== "mach-o" || - input.architecture === undefined || - slice.cpu_type !== MACHO_CPU_TYPES[input.architecture] || - slice.architecture !== input.architecture || - report.machine !== slice.cpu_type || - report.architecture !== input.architecture + slice !== null && + input.architecture !== undefined && + (report.format !== "mach-o" || + slice.architecture !== machoSliceFamily(input.architecture) || + report.machine !== slice.cpu_type || + report.architecture !== machoSliceFamily(input.architecture)) ) return false; const inside = insideArtifact(report.artifact.bytes); diff --git a/src/application/go/GoBinaryService.ts b/src/application/go/GoBinaryService.ts index 6f12cdc6f8..09b6cd96b5 100644 --- a/src/application/go/GoBinaryService.ts +++ b/src/application/go/GoBinaryService.ts @@ -1,5 +1,8 @@ import { binaryMetadataArchitecture } from "../../domain/binaryMetadataArchitecture.js"; -import { metadataReportSubjectFormat } from "../../domain/apple/machoTargetSelection.js"; +import { + machoSelectionAnswersArchitecture, + metadataReportSubjectFormat, +} from "../../domain/apple/machoTargetSelection.js"; import type { ExecutionOptions } from "../AnalysisProvider.js"; import type { GoBinaryPort } from "./GoBinaryPort.js"; import type { AnalysisError } from "../../domain/analysisErrorBase.js"; @@ -53,11 +56,18 @@ export class GoBinaryService { const inspected = await this.provider.inspect(providerInput, options); if (!inspected.ok) return inspected; const report = goBinarySchema.safeParse(inspected.value); - if (!report.success || report.data.artifact.path !== input.value.path) + if ( + !report.success || + report.data.artifact.path !== input.value.path || + !machoSelectionAnswersArchitecture( + report.data.selected_slice, + input.value.architecture, + ) + ) return err( new AnalysisOutputError( OPERATION, - "Go build-information reader returned malformed metadata or changed the selected artifact identity.", + "Go build-information reader returned malformed metadata or changed the selected artifact or slice identity.", ), ); const value = report.data; diff --git a/src/application/rust/RustBinaryService.ts b/src/application/rust/RustBinaryService.ts index 015355a313..052ac3ca37 100644 --- a/src/application/rust/RustBinaryService.ts +++ b/src/application/rust/RustBinaryService.ts @@ -1,5 +1,8 @@ import { binaryMetadataArchitecture } from "../../domain/binaryMetadataArchitecture.js"; -import { metadataReportSubjectFormat } from "../../domain/apple/machoTargetSelection.js"; +import { + machoSelectionAnswersArchitecture, + metadataReportSubjectFormat, +} from "../../domain/apple/machoTargetSelection.js"; import type { ExecutionOptions } from "../AnalysisProvider.js"; import type { RustBinaryPort } from "./RustBinaryPort.js"; import type { AnalysisError } from "../../domain/analysisErrorBase.js"; @@ -54,11 +57,18 @@ export class RustBinaryService { const inspected = await this.provider.inspect(parameters, options); if (!inspected.ok) return inspected; const report = rustBinarySchema.safeParse(inspected.value); - if (!report.success || report.data.artifact.path !== input.value.path) + if ( + !report.success || + report.data.artifact.path !== input.value.path || + !machoSelectionAnswersArchitecture( + report.data.selected_slice, + input.value.architecture, + ) + ) return err( new AnalysisOutputError( OPERATION, - "Rust dependency reader returned malformed metadata or changed the selected artifact identity.", + "Rust dependency reader returned malformed metadata or changed the selected artifact or slice identity.", ), ); const completed = completeProviderEvidence( diff --git a/src/artifacts/apple/MachoLoadCommandReader.ts b/src/artifacts/apple/MachoLoadCommandReader.ts index 1d0ee2c744..1001542432 100644 --- a/src/artifacts/apple/MachoLoadCommandReader.ts +++ b/src/artifacts/apple/MachoLoadCommandReader.ts @@ -26,6 +26,7 @@ import { } from "../../domain/apple/machoPlatform.js"; import { classifyFatContainerIdentity, + fatSliceOverlapReason, findOverlappingFatSlices, hasMachoMagic, readFatContainerHeader, @@ -217,7 +218,7 @@ export const readMachoContainer = async ( return { kind: "issue", status: "malformed", - reason: overlapReason(overlap), + reason: fatSliceOverlapReason(overlap), }; // Identity is only malformed through a slice range issue, which the // returned rangeIssues retain. @@ -296,11 +297,6 @@ const rangeValidSlices = ( ): FatSliceDeclaration[] => header.slices.filter((_, index) => header.rangeIssues[index] === null); -const overlapReason = ( - overlap: NonNullable>, -): string => - `Mach-O slices ${overlap[0].index} and ${overlap[1].index} overlap.`; - /** * Apple lipo stores one hidden ARM64 row after the declared 32-bit FAT table. * It is only admitted from the single table-to-first-slice slot cctools @@ -337,7 +333,7 @@ const readAppleHiddenArm64Row = async ( rangeIssue: overlap === null ? validateFatSlice(declaration, fileSize, end) - : overlapReason(overlap), + : fatSliceOverlapReason(overlap), }; }; diff --git a/src/cli/binaryDiagnosticsCommands.ts b/src/cli/binaryDiagnosticsCommands.ts index 0d55fdb1ae..8209bda6cf 100644 --- a/src/cli/binaryDiagnosticsCommands.ts +++ b/src/cli/binaryDiagnosticsCommands.ts @@ -119,7 +119,7 @@ export const registerBinaryDiagnosticsCommands = ( architecture: machoSliceArchitectureSchema .optional() .describe( - "CPU family of the slice to read from a universal Mach-O; required for universal files", + "Slice to read from a universal Mach-O, by CPU family; when several slices share a family, arm64 (arm64 or arm64v8), arm64e (arm64e or arm64e.x1) or x86_64 names the subtype, and a value matching several slices is refused; required for universal files", ), }), run: ({ args, options }) => diff --git a/src/cli/goCommands.ts b/src/cli/goCommands.ts index dae1f82f7a..cff997c286 100644 --- a/src/cli/goCommands.ts +++ b/src/cli/goCommands.ts @@ -30,7 +30,7 @@ export const registerGoCommands = (cli: CliInstance, logger: Logger): void => { architecture: machoSliceArchitectureSchema .optional() .describe( - "CPU family of the slice to read from a universal Mach-O; required for universal files", + "Slice to read from a universal Mach-O, by CPU family; when several slices share a family, arm64 (arm64 or arm64v8), arm64e (arm64e or arm64e.x1) or x86_64 names the subtype, and a value matching several slices is refused; required for universal files", ), includeStringLiterals: z .boolean() diff --git a/src/cli/rustCommands.ts b/src/cli/rustCommands.ts index c353a1b945..a43616ce42 100644 --- a/src/cli/rustCommands.ts +++ b/src/cli/rustCommands.ts @@ -43,7 +43,7 @@ export const registerRustCommands = ( architecture: machoSliceArchitectureSchema .optional() .describe( - "CPU family of the slice to read from a universal Mach-O; required for universal files", + "Slice to read from a universal Mach-O, by CPU family; when several slices share a family, arm64 (arm64 or arm64v8), arm64e (arm64e or arm64e.x1) or x86_64 names the subtype, and a value matching several slices is refused; required for universal files", ), detail: z .enum(["complete", "summary"]) diff --git a/src/domain/apple/machoContainer.ts b/src/domain/apple/machoContainer.ts index ae771b90a1..11c6d16a62 100644 --- a/src/domain/apple/machoContainer.ts +++ b/src/domain/apple/machoContainer.ts @@ -388,6 +388,12 @@ export const findOverlappingFatSlices = ( return null; }; +/** The malformed-container reason for two FAT declarations whose byte ranges overlap. */ +export const fatSliceOverlapReason = ( + overlap: readonly [FatSliceDeclaration, FatSliceDeclaration], +): string => + `Mach-O slices ${overlap[0].index} and ${overlap[1].index} overlap.`; + /** Validate byte-range and table/header identity facts shared by Mach-O readers. */ export const validateFatSlice = ( declaration: FatSliceDeclaration, diff --git a/src/domain/apple/machoImageSelection.ts b/src/domain/apple/machoImageSelection.ts index 081c065159..84f9953dcf 100644 --- a/src/domain/apple/machoImageSelection.ts +++ b/src/domain/apple/machoImageSelection.ts @@ -1,6 +1,8 @@ import { err, ok, type Result } from "../result.js"; import { classifyFatContainerEvidence, + fatSliceOverlapReason, + findOverlappingFatSlices, readFatContainerHeader, readMachoHeaderPrefixFacts, validateFatSlice, @@ -14,7 +16,10 @@ interface MachoImageSelectionFailure { readonly message: string; } -/** Admit a complete in-memory universal image; FAT-shaped JVM data remains unselected. */ +/** + * Admit a complete in-memory universal image whose slices do not overlap; + * FAT-shaped JVM data remains unselected. + */ export const readUniversalMachoImage = ( bytes: Uint8Array, ): Result => { @@ -31,7 +36,13 @@ export const readUniversalMachoImage = ( const identity = classifyFatContainerEvidence(fat, bytes); if (identity.state === "malformed") return err({ kind: "malformed", message: identity.reason }); - return ok(identity.state === "mach-o" ? fat : null); + if (identity.state !== "mach-o") return ok(null); + // Every range is valid here; aliased declarations are a malformed container, + // as the artifact reader also rules, not slices to choose between. + const overlap = findOverlappingFatSlices(fat.slices); + return overlap === null + ? ok(fat) + : err({ kind: "malformed", message: fatSliceOverlapReason(overlap) }); }; /** Validate the selected declaration against the complete embedded CPU identity. */ diff --git a/src/domain/apple/machoTargetSelection.test.ts b/src/domain/apple/machoTargetSelection.test.ts new file mode 100644 index 0000000000..559df5e5d5 --- /dev/null +++ b/src/domain/apple/machoTargetSelection.test.ts @@ -0,0 +1,28 @@ +import { expect, it } from "vitest"; + +import { machoSelectionAnswersArchitecture } from "./machoTargetSelection.js"; + +const CPU_ARM64 = 0x0100000c; +const CPU_X86_64 = 0x01000007; + +it.each([ + ["no selection, no value", undefined, undefined, true], + ["no selection for a value", null, "arm64", false], + ["a selection without a value", [CPU_ARM64, 0], undefined, false], + ["arm64 for arm64", [CPU_ARM64, 0], "arm64", true], + ["a lone arm64e slice for arm64", [CPU_ARM64, 0x80000002], "arm64", true], + ["arm64e for arm64e", [CPU_ARM64, 0x80000002], "arm64e", true], + ["arm64e.x1 for arm64e", [CPU_ARM64, 12], "arm64e", true], + ["arm64 for arm64e", [CPU_ARM64, 0], "arm64e", false], + ["arm64v8 for arm64e", [CPU_ARM64, 1], "arm64e", false], + ["arm64 for x86_64", [CPU_ARM64, 0], "x86_64", false], + ["x86_64h for x86_64", [CPU_X86_64, 8], "x86_64", true], +] as const)("%s", (_, slice, architecture, answers) => { + const selection = + slice === null || slice === undefined + ? slice + : { cpu_type: slice[0], cpu_subtype: slice[1] }; + expect(machoSelectionAnswersArchitecture(selection, architecture)).toBe( + answers, + ); +}); diff --git a/src/domain/apple/machoTargetSelection.ts b/src/domain/apple/machoTargetSelection.ts index 565c2d1f52..82b09f9afb 100644 --- a/src/domain/apple/machoTargetSelection.ts +++ b/src/domain/apple/machoTargetSelection.ts @@ -2,6 +2,11 @@ import { z } from "zod"; import type { AnalysisProfileCommitment } from "../analysisProfile.js"; import type { BinaryArchitecture } from "../binaryTargetTypes.js"; +import { + machoArchitectureFamily, + machoKnownArm64Subtype, + machoKnownX86_64Subtype, +} from "./machoCpuIdentity.js"; /** Provider-neutral observed identity of one Mach-O image selected from a source. */ export const machoTargetSelectionSchema = z @@ -52,16 +57,65 @@ export const MACHO_CPU_TYPES = { arm64: 0x0100000c, } as const satisfies Partial>; -/** Caller-selected CPU family of one universal Mach-O slice. */ +/** Caller-selected architecture of one universal Mach-O slice. */ export const machoSliceArchitectureSchema = z - .enum(["x86", "x86_64", "arm", "arm64"]) + .enum(["x86", "x86_64", "arm", "arm64", "arm64e"]) .describe( - "CPU family of the slice to read from a universal (FAT) Mach-O. Required for universal files and rejected for every other image.", - ) satisfies z.ZodType; + "Architecture of the slice to read from a universal (FAT) Mach-O. Required for universal files and rejected for every other image. A CPU family with one slice is selected by its family name; arm64e reads only an arm64e slice. When several slices share a family, the value must name the slice's subtype, as lipo -thin does: arm64 reads the arm64 or arm64v8 slice, arm64e the arm64e or arm64e.x1 slice, and x86_64 the x86_64 slice but not x86_64h; capability bits are ignored. A value that matches no slice or several, such as arm64e beside arm64e.x1, is refused.", + ) satisfies z.ZodType; export type MachoSliceArchitecture = z.infer< typeof machoSliceArchitectureSchema >; +/** The CPU family whose slices an architecture value selects; arm64e is an arm64 subtype. */ +export const machoSliceFamily = ( + architecture: MachoSliceArchitecture, +): keyof typeof MACHO_CPU_TYPES => + architecture === "arm64e" ? "arm64" : architecture; + +/** + * The architecture value that names a slice's subtype exactly, as + * `lipo -thin` does, or undefined when no accepted value names it. Capability + * bits are ignored; `arm64v8` reads as `arm64` and `arm64e.x1` as `arm64e`. + */ +export const machoSliceSubtypeSelector = ( + cpuType: number, + cpuSubtype: number, +): MachoSliceArchitecture | undefined => { + if (cpuType === MACHO_CPU_TYPES.arm64) { + const name = machoKnownArm64Subtype(cpuSubtype); + const family = + name === undefined ? undefined : machoArchitectureFamily(name); + return family === "arm64" || family === "arm64e" ? family : undefined; + } + if (cpuType === MACHO_CPU_TYPES.x86_64) + return machoKnownX86_64Subtype(cpuSubtype) === "x86_64" + ? "x86_64" + : undefined; + return undefined; +}; + +/** + * Whether a reported universal slice answers the caller's architecture value. + * Only a universal image has a selection, and only one needs a value. The + * slice must be of the value's CPU family; arm64e also requires its subtype, + * while a family name may read the family's only slice whatever its subtype. + */ +export const machoSelectionAnswersArchitecture = ( + selection: + | { readonly cpu_type: number; readonly cpu_subtype: number } + | null + | undefined, + architecture: MachoSliceArchitecture | undefined, +): boolean => + selection === null || selection === undefined || architecture === undefined + ? (selection === null || selection === undefined) === + (architecture === undefined) + : selection.cpu_type === MACHO_CPU_TYPES[machoSliceFamily(architecture)] && + (architecture !== "arm64e" || + machoSliceSubtypeSelector(selection.cpu_type, selection.cpu_subtype) === + "arm64e"); + /** Confirm the selected header's CPU family agrees with the caller's request. */ export const machoTargetSelectionMatchesArchitecture = ( selection: MachoTargetSelection, diff --git a/src/domain/native/nativeInitializers.ts b/src/domain/native/nativeInitializers.ts index 9813b10459..8bf20d40e1 100644 --- a/src/domain/native/nativeInitializers.ts +++ b/src/domain/native/nativeInitializers.ts @@ -22,7 +22,8 @@ export const MAX_NATIVE_INITIALIZER_NAME_BYTES = 1024; export const MAX_NATIVE_INITIALIZER_NOTES = 24; /** * Upper bound on reported limitations: the standing ones, the note budget, a - * line per tallied reason, an omission count and a withheld-name count. + * line per tallied reason, an omission count, a withheld-name count and the + * universal slice selection line. */ const MAX_NATIVE_INITIALIZER_LIMITATIONS = 40; diff --git a/src/go/GoBinaryProvider.ts b/src/go/GoBinaryProvider.ts index 3c028e5a5b..9c66a2de72 100644 --- a/src/go/GoBinaryProvider.ts +++ b/src/go/GoBinaryProvider.ts @@ -353,7 +353,10 @@ export class GoBinaryProvider implements GoBinaryPort { ...(stringLiterals === undefined ? {} : { string_literals: stringLiterals }), - limitations: limitationsFor(image, stringLiterals, budgets), + limitations: [ + ...limitationsFor(image, stringLiterals, budgets), + ...selected.limitations, + ], }; if ( jsonValueByteLength(report, budgets.max_report_bytes) > diff --git a/src/native/initializers/NativeInitializersProvider.ts b/src/native/initializers/NativeInitializersProvider.ts index 8761b9ad2a..a0f488e625 100644 --- a/src/native/initializers/NativeInitializersProvider.ts +++ b/src/native/initializers/NativeInitializersProvider.ts @@ -128,7 +128,11 @@ export const readNativeInitializers = async ( ); const page = { offset: input.offset, limit: input.limit }; const inspection = await inspectSelected(bytes, selected, page, signal); - return { artifact, ...inspection }; + return { + artifact, + ...inspection, + limitations: [...inspection.limitations, ...selected.limitations], + }; }; const inspectSelected = async ( diff --git a/src/native/metadata/MachoSliceSelection.test.ts b/src/native/metadata/MachoSliceSelection.test.ts new file mode 100644 index 0000000000..794c016b04 --- /dev/null +++ b/src/native/metadata/MachoSliceSelection.test.ts @@ -0,0 +1,263 @@ +import { describe, expect, it } from "vitest"; + +import type { MachoSliceArchitecture } from "../../domain/apple/machoTargetSelection.js"; +import { machoArm64Initializers } from "../initializers/NativeInitializers.fixture.js"; +import { selectNativeImage } from "./MachoSliceSelection.js"; + +const CPU_ARM = 12; +const CPU_X86_64 = 0x01000007; +const CPU_ARM64 = 0x0100000c; +const ALIGN = 0x1000; + +/** + * A FAT32 file whose 4 KiB-aligned slices are the arm64 initializer image with + * each declared CPU; a third entry overrides the embedded header's subtype. + */ +const fat = (cpus: readonly (readonly [number, number, number?])[]) => { + const thin = machoArm64Initializers(); + const stride = Math.ceil(thin.length / ALIGN) * ALIGN; + const offsets = cpus.map((_, index) => ALIGN + index * stride); + const bytes = Buffer.alloc(ALIGN + cpus.length * stride); + bytes.writeUInt32BE(0xcafebabe, 0); + bytes.writeUInt32BE(cpus.length, 4); + for (const [index, [cpuType, cpuSubtype, headerSubtype]] of cpus.entries()) { + const entry = 8 + index * 20; + bytes.writeUInt32BE(cpuType, entry); + bytes.writeUInt32BE(cpuSubtype, entry + 4); + bytes.writeUInt32BE(offsets[index]!, entry + 8); + bytes.writeUInt32BE(thin.length, entry + 12); + bytes.writeUInt32BE(12, entry + 16); + const image = Buffer.from(thin); + image.writeUInt32LE(cpuType, 4); + image.writeUInt32LE(headerSubtype ?? cpuSubtype, 8); + image.copy(bytes, offsets[index]); + } + return { bytes, offsets }; +}; + +const select = ( + cpus: readonly (readonly [number, number, number?])[], + architecture: MachoSliceArchitecture, +) => { + const file = fat(cpus); + const result = selectNativeImage(file.bytes, architecture); + return { + offsets: file.offsets, + offset: result.ok ? result.value.fileOffset : undefined, + limitations: result.ok ? result.value.limitations : [], + message: result.ok ? undefined : result.error.message, + }; +}; + +describe("selectNativeImage subtype names", () => { + it("reads arm64v8 as arm64 and arm64e.x1 as arm64e", () => { + const v8 = select( + [ + [CPU_ARM64, 1], + [CPU_ARM64, 2], + ], + "arm64", + ); + expect(v8.offset).toBe(v8.offsets[0]); + const x1 = select( + [ + [CPU_ARM64, 0], + [CPU_ARM64, 12], + ], + "arm64e", + ); + expect(x1.offset).toBe(x1.offsets[1]); + }); + + it("refuses arm64e beside arm64e.x1, which the value names alike", () => { + expect( + select( + [ + [CPU_ARM64, 2], + [CPU_ARM64, 12], + ], + "arm64e", + ).message, + ).toBe( + "Universal Mach-O holds several arm64e slices (arm64 (subtype 0x2), arm64 (subtype 0xc)); no slice was selected.", + ); + }); + + it("ignores capability bits when naming a subtype", () => { + const capable = select( + [ + [CPU_ARM64, 0x80000002], + [CPU_ARM64, 0x80000000], + ], + "arm64", + ); + expect(capable.offset).toBe(capable.offsets[1]); + }); + + it("refuses a subtype name that several slices share and names the value that works", () => { + const cpus = [ + [CPU_ARM64, 2], + [CPU_ARM64, 0x80000002], + [CPU_ARM64, 0], + ] as const; + const arm64e = select(cpus, "arm64e"); + expect(arm64e.message).toBe( + "Universal Mach-O holds several arm64e slices (arm64 (subtype 0x2), arm64 (subtype 0x80000002), arm64 (subtype 0x0)); no slice was selected. Select architecture arm64.", + ); + const arm64 = select(cpus, "arm64"); + expect(arm64.offset).toBe(arm64.offsets[2]); + expect(arm64.limitations).toEqual([ + `Only the selected arm64 slice (arm64 (subtype 0x0) at file offset ${String(arm64.offsets[2])}) was read; this universal Mach-O also holds arm64 (subtype 0x2) at file offset ${String(arm64.offsets[0])} (no architecture value selects it), arm64 (subtype 0x80000002) at file offset ${String(arm64.offsets[1])} (no architecture value selects it).`, + ]); + }); + + it("refuses a subtype that no slice of the family has", () => { + expect( + select( + [ + [CPU_ARM64, 0], + [CPU_ARM64, 1], + ], + "arm64e", + ).message, + ).toBe( + "Universal Mach-O holds arm64 (subtype 0x0), arm64 (subtype 0x1); no slice has the arm64e subtype.", + ); + // A lone family slice is read by its family name, so the refusal names it. + expect( + select( + [ + [CPU_X86_64, 3], + [CPU_ARM64, 0], + ], + "arm64e", + ).message, + ).toBe( + "Universal Mach-O holds x86_64 (subtype 0x3), arm64 (subtype 0x0); no slice has the arm64e subtype. Select architecture arm64.", + ); + }); + + it("refuses slices that differ only in capability bits", () => { + expect( + select( + [ + [CPU_ARM64, 0], + [CPU_ARM64, 0x80000000], + ], + "arm64", + ).message, + ).toBe( + "Universal Mach-O holds several arm64 slices (arm64 (subtype 0x0), arm64 (subtype 0x80000000)); no slice was selected.", + ); + }); + + it("reports a family with no slice of the named subtype rather than several", () => { + expect( + select( + [ + [CPU_ARM64, 2], + [CPU_ARM64, 12], + ], + "arm64", + ).message, + ).toBe( + "Universal Mach-O holds arm64 (subtype 0x2), arm64 (subtype 0xc); no slice has the arm64 subtype.", + ); + }); +}); + +describe("selectNativeImage family reads and unread slices", () => { + it("does not advertise a selector for a slice whose header disagrees with its declaration", () => { + // Declared arm64e, but the embedded header says arm64. + const mismatched = [CPU_ARM64, 2, 0] as const; + const read = select([[CPU_ARM64, 0], mismatched], "arm64"); + expect(read.offset).toBe(read.offsets[0]); + expect(read.limitations).toEqual([ + `Only the selected arm64 slice (arm64 (subtype 0x0) at file offset ${String(read.offsets[0])}) was read; this universal Mach-O also holds arm64 (subtype 0x2) at file offset ${String(read.offsets[1])} (unverified: FAT architecture 1 CPU type/subtype disagrees with its Mach-O header; no architecture value reads it).`, + ]); + expect( + select([[CPU_ARM64, 0], [CPU_ARM64, 0], mismatched], "arm64").message, + ).toBe( + "Universal Mach-O holds several arm64 slices (arm64 (subtype 0x0), arm64 (subtype 0x0), arm64 (subtype 0x2)); no slice was selected.", + ); + }); + + it("fails as malformed when the requested slice's header disagrees with its declaration", () => { + const file = fat([ + [CPU_ARM64, 0], + [CPU_ARM64, 2, 0], + ]); + const result = selectNativeImage(file.bytes, "arm64e"); + expect(result.ok ? undefined : result.error).toEqual({ + kind: "malformed", + message: + "FAT architecture 1 CPU type/subtype disagrees with its Mach-O header", + }); + }); + + it("fails the whole file when a sibling's declared range leaves it", () => { + const file = fat([ + [CPU_ARM64, 0], + [CPU_ARM64, 2], + ]); + // Declare the second slice past the end of the file. + file.bytes.writeUInt32BE(file.bytes.length, 8 + 20 + 12); + const result = selectNativeImage(file.bytes, "arm64"); + expect(result.ok ? undefined : result.error).toEqual({ + kind: "malformed", + message: "FAT architecture 1 slice range extends beyond the file", + }); + }); + + it("reads a lone arm64e slice by the arm64 family name", () => { + const lone = select( + [ + [CPU_X86_64, 3], + [CPU_ARM64, 2], + ], + "arm64", + ); + expect(lone.offset).toBe(lone.offsets[1]); + expect(lone.limitations).toEqual([]); + }); + + it("reads a lone x86_64h slice by its family name", () => { + const lone = select( + [ + [CPU_ARM64, 0], + [CPU_X86_64, 8], + ], + "x86_64", + ); + expect(lone.offset).toBe(lone.offsets[1]); + expect(lone.limitations).toEqual([]); + }); + + it("reads x86_64 beside x86_64h and reports that no value selects x86_64h", () => { + const x86 = select( + [ + [CPU_X86_64, 8], + [CPU_X86_64, 3], + ], + "x86_64", + ); + expect(x86.offset).toBe(x86.offsets[1]); + expect(x86.limitations).toEqual([ + `Only the selected x86_64 slice (x86_64 (subtype 0x3) at file offset ${String(x86.offsets[1])}) was read; this universal Mach-O also holds x86_64 (subtype 0x8) at file offset ${String(x86.offsets[0])} (no architecture value selects it).`, + ]); + }); + + it("refuses several 32-bit arm slices, which no value names", () => { + expect( + select( + [ + [CPU_ARM, 9], + [CPU_ARM, 11], + ], + "arm", + ).message, + ).toBe( + "Universal Mach-O holds several arm slices (arm (subtype 0x9), arm (subtype 0xb)); no slice was selected.", + ); + }); +}); diff --git a/src/native/metadata/MachoSliceSelection.ts b/src/native/metadata/MachoSliceSelection.ts index c3c525b18c..95cdccb7e4 100644 --- a/src/native/metadata/MachoSliceSelection.ts +++ b/src/native/metadata/MachoSliceSelection.ts @@ -6,6 +6,8 @@ import { } from "../../domain/apple/machoImageSelection.js"; import { MACHO_CPU_TYPES, + machoSliceFamily, + machoSliceSubtypeSelector, type MachoSliceArchitecture, type MachoTargetSelection, } from "../../domain/apple/machoTargetSelection.js"; @@ -29,6 +31,8 @@ export interface SelectedNativeImage { readonly fileOffset: number; /** Selected universal slice identity; null when the file is not a universal Mach-O. */ readonly selection: MachoTargetSelection | null; + /** Other slices of the selected CPU family that this inspection does not read, and how to select each. */ + readonly limitations: readonly string[]; } /** Why no image could be selected; `architecture` issues are the caller's to correct. */ @@ -37,10 +41,48 @@ export interface MachoSliceSelectionIssue { readonly message: string; } +interface FatSlice { + readonly cpuType: number; + readonly cpuSubtype: number; + readonly offset: number; +} + +/** Whether a value names a subtype, not only a CPU family. */ +const namesSubtype = (architecture: MachoSliceArchitecture): boolean => + architecture === "arm64" || + architecture === "arm64e" || + architecture === "x86_64"; + +/** The architecture values that address slices of this value's CPU family. */ +const familyValues = ( + architecture: MachoSliceArchitecture, +): readonly MachoSliceArchitecture[] => + architecture === "arm64" || architecture === "arm64e" + ? ["arm64", "arm64e"] + : [architecture]; + +/** + * The slices one value addresses within its CPU family: a family name reads + * the family's only slice, and otherwise the value must name a subtype. + */ +const addressedSlices = ( + family: readonly Slice[], + architecture: MachoSliceArchitecture, +): readonly Slice[] => + architecture !== "arm64e" && family.length === 1 + ? family + : family.filter( + (slice) => + machoSliceSubtypeSelector(slice.cpuType, slice.cpuSubtype) === + architecture, + ); + /** * Select the thin image to inspect without lipo. A universal Mach-O needs the - * caller's architecture and yields exactly one validated slice; any other file - * is returned whole and rejects an architecture rather than ignoring it. + * caller's architecture. A family name selects the family's only slice; when + * several slices share the family, the value must name one slice's subtype + * exactly, and no slice is chosen by preference. Any other file is returned + * whole and rejects an architecture rather than ignoring it. */ export const selectNativeImage = ( bytes: Buffer, @@ -51,7 +93,7 @@ export const selectNativeImage = ( const fat = container.value; if (fat === null) return architecture === undefined - ? ok({ bytes, fileOffset: 0, selection: null }) + ? ok({ bytes, fileOffset: 0, selection: null, limitations: [] }) : err({ kind: "architecture", message: `architecture ${architecture} was selected, but the file is not a universal Mach-O; omit architecture.`, @@ -64,20 +106,61 @@ export const selectNativeImage = ( kind: "architecture", message: `Universal Mach-O holds ${available}; select one architecture.`, }); - const matches = fat.slices.filter( - ({ cpuType }) => cpuType === MACHO_CPU_TYPES[architecture], + const sameFamily = fat.slices.filter( + ({ cpuType }) => + cpuType === MACHO_CPU_TYPES[machoSliceFamily(architecture)], ); + // Selectors are advertised only for slices whose header agrees with their + // FAT declaration; an unverified declaration is not a working choice. + const identityIssues = new Map( + sameFamily.map((declared) => { + const read = readUniversalMachoSlice(bytes, fat, declared); + return [declared, read.ok ? undefined : read.error.message] as const; + }), + ); + const verifiedSliceFor = (value: MachoSliceArchitecture) => { + const addressed = addressedSlices(sameFamily, value); + return addressed.length === 1 && + addressed[0] !== undefined && + identityIssues.get(addressed[0]) === undefined + ? addressed[0] + : undefined; + }; + const matches = addressedSlices(sameFamily, architecture); const [slice] = matches; - if (slice === undefined || matches.length > 1) + if (slice === undefined || matches.length > 1) { + const choices = familyValues(architecture).filter( + (value) => + value !== architecture && verifiedSliceFor(value) !== undefined, + ); + const select = + choices.length === 0 + ? "" + : ` Select architecture ${choices.join(" or ")}.`; return err({ kind: "architecture", message: - slice === undefined - ? `Universal Mach-O holds ${available}, not ${architecture}.` - : `Universal Mach-O holds several ${architecture} slices (${available}); no slice was selected.`, + sameFamily.length === 0 + ? `Universal Mach-O holds ${available}, not ${architecture}.${select}` + : matches.length === 0 && namesSubtype(architecture) + ? `Universal Mach-O holds ${available}; no slice has the ${architecture} subtype.${select}` + : `Universal Mach-O holds several ${architecture} slices (${available}); no slice was selected.${select}`, }); + } const selected = readUniversalMachoSlice(bytes, fat, slice); if (!selected.ok) return selected; + const unread = sameFamily + .filter((other) => other !== slice) + .map((other) => { + const place = `${cpuName(other.cpuType, other.cpuSubtype)} at file offset ${String(other.offset)}`; + const issue = identityIssues.get(other); + if (issue !== undefined) + return `${place} (unverified: ${issue}; no architecture value reads it)`; + const selector = familyValues(architecture).find( + (value) => verifiedSliceFor(value) === other, + ); + return `${place} (${selector === undefined ? "no architecture value selects it" : `architecture ${selector} selects it`})`; + }); return ok({ bytes: selected.value, fileOffset: slice.offset, @@ -92,6 +175,12 @@ export const selectNativeImage = ( .update(bytes.subarray(0, fat.tableEnd)) .digest("hex"), }, + limitations: + unread.length === 0 + ? [] + : [ + `Only the selected ${architecture} slice (${cpuName(slice.cpuType, slice.cpuSubtype)} at file offset ${String(slice.offset)}) was read; this universal Mach-O also holds ${unread.join(", ")}.`, + ], }); }; diff --git a/src/rust/RustBinaryProvider.ts b/src/rust/RustBinaryProvider.ts index 3bf7347783..aa7dbcdbee 100644 --- a/src/rust/RustBinaryProvider.ts +++ b/src/rust/RustBinaryProvider.ts @@ -136,6 +136,7 @@ export class RustBinaryProvider implements RustBinaryPort { limitations: [ ...limitationsFor(image), `Selected producer_limits.input_bytes=${String(limits.input_bytes)} and output_bytes=${String(limits.output_bytes)} bytes; compressed/inflated dependency parser limits remain independent.`, + ...selected.limitations, ], }; retainCompilerPathsWithinReport(report, limits.output_bytes); diff --git a/tests/boundary/goRustSubjectArchitecture.test.ts b/tests/boundary/goRustSubjectArchitecture.test.ts index 7c7ee3b050..b72cdd0ae5 100644 --- a/tests/boundary/goRustSubjectArchitecture.test.ts +++ b/tests/boundary/goRustSubjectArchitecture.test.ts @@ -7,7 +7,16 @@ import { RustBinaryService } from "../../src/application/rust/RustBinaryService. import { elf } from "../../src/domain/binaryTarget.fixture.js"; import { GoBinaryProvider } from "../../src/go/GoBinaryProvider.js"; import { RustBinaryProvider } from "../../src/rust/RustBinaryProvider.js"; +import { AnalysisOutputError } from "../../src/domain/analysisErrorCore.js"; +import type { MachoSliceArchitecture } from "../../src/domain/apple/machoTargetSelection.js"; +import { createGoBinaryFixture } from "../fixtures/go/image.js"; +import { createRustImage } from "../fixtures/rust/image.js"; import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js"; +import { + CPU_ARM64, + CPU_X86_64, + universalMacho, +} from "../fixtures/universalMacho.js"; it.each([ [1, 1, 20, "ppc", "powerpc"], @@ -40,3 +49,68 @@ it.each([ } }, ); + +it("refuses a Go or Rust report whose slice does not answer the requested architecture", async () => { + const root = await createTestTempDirectory("rea-metadata-slice-"); + const write = async (name: string, bytes: Buffer) => { + const path = join(root, name); + await writeFile(path, bytes); + return path; + }; + const universal = (thin: Buffer) => + universalMacho([ + { bytes: thin, cpuType: CPU_X86_64, cpuSubtype: 3 }, + { bytes: thin, cpuType: CPU_ARM64, cpuSubtype: 0 }, + { bytes: thin, cpuType: CPU_ARM64, cpuSubtype: 0x80000002 }, + ]).bytes; + const go = new GoBinaryProvider(); + const rust = new RustBinaryProvider(); + const targets = [ + { + path: await write( + "universal-go", + universal(createGoBinaryFixture({ format: "macho" }).bytes), + ), + service: (read: MachoSliceArchitecture) => + new GoBinaryService({ + identity: go.identity, + close: () => go.close(), + inspect: (input, options) => + go.inspect({ ...input, architecture: read }, options), + }), + }, + { + path: await write( + "universal-rust", + universal(createRustImage({ format: "macho", includeRustc: true })), + ), + service: (read: MachoSliceArchitecture) => + new RustBinaryService({ + identity: rust.identity, + close: () => rust.close(), + inspect: (input, options) => + rust.inspect({ ...input, architecture: read }, options), + }), + }, + ]; + // A reader that returns another slice than the request names is refused: + // the arm64 slice for arm64e, or a slice of another CPU family. + for (const { path, service } of targets) { + for (const [requested, read, accepted] of [ + ["arm64e", "arm64e", true], + ["arm64e", "arm64", false], + ["x86_64", "arm64", false], + ] as const) { + const result = await service(read).inspect({ + path, + architecture: requested, + }); + expect( + result.ok ? true : result.error, + `${path} ${requested} read as ${read}`, + ).toEqual(accepted ? true : expect.any(AnalysisOutputError)); + } + } + await go.close(); + await rust.close(); +}); diff --git a/tests/boundary/il2cpp/il2cppBinaryFile.test.ts b/tests/boundary/il2cpp/il2cppBinaryFile.test.ts index 12b1f55460..d7bff95e11 100644 --- a/tests/boundary/il2cpp/il2cppBinaryFile.test.ts +++ b/tests/boundary/il2cpp/il2cppBinaryFile.test.ts @@ -231,6 +231,22 @@ it("requires an explicit slice of a universal Mach-O and reads only that slice", expect(failure(thin, "arm64").reason).toBe("slice-not-found"); }); +it("rejects a universal Mach-O whose slices overlap before selecting one", () => { + const universal = createUniversalFixture([ + { cpu: 0x01000007, bytes: createIl2cppBinaryFixture() }, + { cpu: 0x0100000c, bytes: createIl2cppBinaryFixture() }, + ]); + expect(link(universal, "arm64").method_addresses).toEqual(expectedAddresses); + // Stretch the first slice into the second one. + const first = universal.readUInt32BE(8 + 8); + const second = universal.readUInt32BE(8 + 20 + 8); + universal.writeUInt32BE(second - first + 16, 8 + 12); + expect(failure(universal, "arm64")).toEqual({ + reason: "malformed-container", + kind: "malformed", + }); +}); + it.each(["thin", "universal"] as const)( "links a %s x86-64 Mach-O while retaining the reader's amd64 spelling", (format) => { diff --git a/tests/boundary/mcp/nativeInitializersMcp.test.ts b/tests/boundary/mcp/nativeInitializersMcp.test.ts index 3a4e49bf78..a818dc959e 100644 --- a/tests/boundary/mcp/nativeInitializersMcp.test.ts +++ b/tests/boundary/mcp/nativeInitializersMcp.test.ts @@ -7,11 +7,17 @@ import { createTestBinarySession } from "../../fixtures/binarySession.js"; import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; import { machoArm64FatInitializers, + machoArm64Initializers, elf64PieInitializers, elf64AliasedInitializers, elfUnmappedInitializerTargets, } from "../../../src/native/initializers/NativeInitializers.fixture.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { + CPU_ARM64, + CPU_X86_64, + universalMacho, +} from "../../fixtures/universalMacho.js"; import { mcpTest, type TestMcp } from "../../support/mcp/mcpFixture.js"; const connect = async (mcp: TestMcp) => { @@ -140,6 +146,59 @@ mcpTest( }, ); +mcpTest( + "reads the arm64 or arm64e slice that the architecture names", + async ({ mcp }) => { + const { client, session } = await connect(mcp); + try { + // The x86_64, arm64 and arm64e layout of stock tools such as /usr/bin/file. + const thin = machoArm64Initializers(); + const universal = universalMacho([ + { bytes: thin, cpuType: CPU_X86_64, cpuSubtype: 3 }, + { bytes: thin, cpuType: CPU_ARM64, cpuSubtype: 0 }, + { bytes: thin, cpuType: CPU_ARM64, cpuSubtype: 0x80000002 }, + ]); + const root = await createTestTempDirectory("rea-native-init-arm64e-mcp-"); + const path = join(root, "universal"); + await writeFile(path, universal.bytes); + const read = async (architecture: string) => { + const response = await client.callTool({ + name: "inspect_native_initializers", + arguments: { path, architecture }, + }); + expect(response.isError, JSON.stringify(response.content)).not.toBe( + true, + ); + return toolContract("inspect_native_initializers").outputSchema.parse( + response.structuredContent, + ).normalized_result; + }; + const arm64 = await read("arm64"); + expect(arm64).toMatchObject({ + format: "mach-o", + slice: { + file_offset: universal.slices[1]?.offset, + cpu_subtype: 0, + architecture: "arm64", + }, + }); + expect(arm64.entries.length).toBeGreaterThan(0); + // The arm64e slice is not read, and the result names the value that reads it. + expect(arm64.limitations).toContain( + `Only the selected arm64 slice (arm64 (subtype 0x0) at file offset ${String(universal.slices[1]?.offset)}) was read; this universal Mach-O also holds arm64 (subtype 0x80000002) at file offset ${String(universal.slices[2]?.offset)} (architecture arm64e selects it).`, + ); + expect(await read("arm64e")).toMatchObject({ + slice: { + file_offset: universal.slices[2]?.offset, + cpu_subtype: 0x80000002, + }, + }); + } finally { + await session.close(); + } + }, +); + mcpTest( "requires an absolute path for inspect_native_initializers", async ({ mcp }) => { diff --git a/tests/boundary/mcp/universalSliceMcp.test.ts b/tests/boundary/mcp/universalSliceMcp.test.ts index c91e030418..ccf4873408 100644 --- a/tests/boundary/mcp/universalSliceMcp.test.ts +++ b/tests/boundary/mcp/universalSliceMcp.test.ts @@ -235,32 +235,6 @@ sliceTest( }, }); - const duplicate = await write( - "duplicate-arm64", - universalMacho([ - { - bytes: createGoBinaryFixture({ format: "macho" }).bytes, - cpuType: CPU_ARM64, - cpuSubtype: 0, - }, - { - bytes: createGoBinaryFixture({ format: "macho" }).bytes, - cpuType: CPU_ARM64, - cpuSubtype: 2, - }, - ]).bytes, - ); - expect( - issues( - await client.callTool({ - name: "inspect_go_binary", - arguments: { path: duplicate, architecture: "arm64" }, - }), - ).details.issues[0]?.message, - ).toBe( - "Universal Mach-O holds several arm64 slices (arm64 (subtype 0x0), arm64 (subtype 0x2)); no slice was selected.", - ); - // A slice that runs past the file is damaged, not a selection to correct. const truncated = universalMacho([ { @@ -320,6 +294,127 @@ sliceTest( }, ); +const arm64AndArm64e = () => + universalMacho([ + { + bytes: createGoBinaryFixture({ + format: "macho", + goVersion: "go1.25.0", + }).bytes, + cpuType: CPU_ARM64, + cpuSubtype: 0x80000002, + }, + { + bytes: createGoBinaryFixture({ + format: "macho", + goVersion: "go1.26.1", + }).bytes, + cpuType: CPU_ARM64, + cpuSubtype: 0, + }, + ]); + +sliceTest( + "selects the slice whose subtype the architecture names when several share a CPU family", + async ({ slices: { client, write } }) => { + // Stock macOS tools ship arm64 and arm64e together. Each value names one + // subtype, as lipo -thin does, wherever it sits in the FAT table. + const universal = arm64AndArm64e(); + const path = await write("arm64-and-arm64e", universal.bytes); + const read = async (architecture: string) => { + const response = await client.callTool({ + name: "inspect_go_binary", + arguments: { path, architecture }, + }); + expect(response.isError, JSON.stringify(response.content)).not.toBe(true); + return goBinarySchema.parse( + toolContract("inspect_go_binary").outputSchema.parse( + response.structuredContent, + ).normalized_result, + ); + }; + const arm64 = await read("arm64"); + expect(arm64).toMatchObject({ + selected_slice: { offset: universal.slices[1]?.offset, cpu_subtype: 0 }, + build_info: { go_version: "go1.26.1" }, + }); + // The other slice is not read, and the result names the value that reads it. + expect(arm64.limitations).toContain( + `Only the selected arm64 slice (arm64 (subtype 0x0) at file offset ${String(universal.slices[1]?.offset)}) was read; this universal Mach-O also holds arm64 (subtype 0x80000002) at file offset ${String(universal.slices[0]?.offset)} (architecture arm64e selects it).`, + ); + expect(await read("arm64e")).toMatchObject({ + selected_slice: { + offset: universal.slices[0]?.offset, + cpu_subtype: 0x80000002, + }, + build_info: { go_version: "go1.25.0" }, + }); + }, +); + +sliceTest( + "refuses a subtype that is missing or shared instead of choosing by preference", + async ({ slices: { client, write } }) => { + const slice = (cpuSubtype: number) => ({ + bytes: createGoBinaryFixture({ format: "macho" }).bytes, + cpuType: CPU_ARM64, + cpuSubtype, + }); + const refusal = async ( + name: string, + subtypes: readonly number[], + architecture: string, + ) => + issues( + await client.callTool({ + name: "inspect_go_binary", + arguments: { + path: await write(name, universalMacho(subtypes.map(slice)).bytes), + architecture, + }, + }), + ).details.issues[0]?.message; + expect(await refusal("duplicate-arm64", [0, 0], "arm64")).toBe( + "Universal Mach-O holds several arm64 slices (arm64 (subtype 0x0), arm64 (subtype 0x0)); no slice was selected.", + ); + // The refusal names the value that does select one slice. + expect( + await refusal("duplicate-arm64-and-arm64e", [0, 0, 2], "arm64"), + ).toBe( + "Universal Mach-O holds several arm64 slices (arm64 (subtype 0x0), arm64 (subtype 0x0), arm64 (subtype 0x2)); no slice was selected. Select architecture arm64e.", + ); + expect(await refusal("arm64-only", [0, 1], "arm64e")).toBe( + "Universal Mach-O holds arm64 (subtype 0x0), arm64 (subtype 0x1); no slice has the arm64e subtype.", + ); + }, +); + +sliceTest( + "rejects overlapping same-family slices as a malformed container", + async ({ slices: { client, write } }) => { + // Each entry is in bounds and its header agrees, but the arm64e entry's + // range runs into the arm64 slice, so no subtype is a valid choice. + const universal = arm64AndArm64e(); + const [first, second] = universal.slices; + if (first === undefined || second === undefined) + throw new Error("expected two slices"); + universal.bytes.writeUInt32BE(second.offset - first.offset + 16, 8 + 12); + const path = await write("overlapping", universal.bytes); + for (const architecture of ["arm64", "arm64e"]) + expect( + issues( + await client.callTool({ + name: "inspect_go_binary", + arguments: { path, architecture }, + }), + ).details.issues[0], + ).toMatchObject({ + path: ["path"], + message: `Selected artifact ${path}: Mach-O slices 0 and 1 overlap.`, + }); + }, +); + sliceTest( "reads the selected Rust slice and keeps section and rustc offsets in the universal file", async ({ slices: { client, write } }) => {