Files
reverse-engineering-anything/tests/boundary/filesystem/elfDebugLink.test.ts
T
7fd50eb361 fix(zstd): share exact-frame Zstandard decoding across ELF and OTA (#12416)
* fix(zstd): share exact-frame Zstandard decoding across ELF and OTA

Walk RFC 8878 frames once so ELF sections, Android OTA REPLACE_ZSTD, and GPU payloads reject truncated headers, blocks, checksums, and trailing bytes on every supported Node runtime. Bound that walk with the runtime zstd.max_frames setting.

* fix(zstd): keep frame output tight and report GPU frame-cap refusals

Copy each Node 22 frame off its decoder chunk before the next frame is decoded. Map a GPU zstd.max_frames refusal through the resource-limit contract instead of a provider failure.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(zstd): keep frame-cap refusals and bad seeds out of malformed results

Project zstd.max_frames through one resource-limit helper for ELF notes, DWARF, GPU, and artifact reads. An invalid REA_ZSTD_MAX_FRAMES seed stays a configuration failure, and frame-header tests cover content size, window, dictionary id, and RLE blocks.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(zstd): yield GPU frame decoding so cancellation can land

The frame-by-frame fallback awaits a caller pause before each data frame. GPU yields and checks the read signal there, so a many-frame payload does not finish before a disconnect is observed.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(zstd): advertise the frame cap and keep strict GPU decode cancellable

List every operation that enforces zstd.max_frames, including GPU, DWARF, and ELF notes. The strict sequence uses the async decoder and checks the caller signal again after it returns, and a single output buffer is not copied a second time.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(zstd): refresh analysis-setting schemas and GPU frame-cap projection

Regenerate the advertised input summaries for inspect_analysis_settings and update_analysis_settings. GPU inventory now throws the projected zstd.max_frames resource limit from payload decode and #retainFailure, so the adjustable setting survives the public failure.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* style(gpu): format the frame-cap projection throw

oxfmt keeps the zstd.max_frames throw in #retainFailure on one line so CI format:check matches the local formatter.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(zstd): yield after the last frame-by-frame GPU decode

The fallback returned success in the same turn as the final synchronous frame, so a queued abort was still invisible to the caller's signal check. Pause once more before that success, matching the strict sequence.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(zstd): yield ELF, OTA, and DWARF frame decoding on the caller signal

The synchronous frame loop ignored abort signals owned by inspect_elf_notes, inspect_dwarf_units, and Android OTA partition reads. Those callers now use the cooperative decoder and yield before Zstandard work, and 32-frame cancellations cover each path.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(zstd): observe cancellation during structural parsing

* fix(zstd): project section-name frame caps at the ELF directory boundary

readElfDirectoryFromBuffer now requires the caller's operation and throws the adjustable zstd.max_frames resource limit. Hardening, unwind, and the other directory consumers keep that typed refusal instead of an adapter or I/O error.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(zstd): preserve optional ELF evidence when frames are capped

* fix(zip): validate Zstandard members through frame completion (#12461)

fix(zip): require complete Zstandard member frames

* fix(elf): yield hardening section name decoding

* fix(elf): yield section names for unwind and export reads

* fix(zstd): yield ELF directory reads that already own a cancellation signal

Hardening, unwind, initializers, exports, kernel modules, debug link, and Android JNI now read section names through the cooperative directory when their caller has a signal. Readers with no signal stay synchronous.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(elf): share one initializer directory and retain its snapshot

* fix(android): reuse the admitted JNI ELF directory

* fix(elf): retain optional debug candidate evidence through cooperative decoding

* fix(zstd): yield ELF section names for Go, Rust, and IL2CPP

These inspections already own an abort signal. Their ELF directory read now uses the cooperative container decoder so a queued cancellation can land during a many-frame section-name table.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(elf): recognize compressed kernel modules before ELF decoding

The cooperative module reader decoded the bytes as ELF before the container check, so gzip, xz, and zstd modules became malformed images. The compression check now returns the unsupported-container report first.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* refactor(elf): classify each container once before reading it

* fix(il2cpp): preserve admission order during cooperative linking

* fix(elf): reuse section-name decoding in debug-link comparison (#12472)

* style(elf): format updated section-reader test calls

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-11-19 19:45:26 +00:00

442 lines
16 KiB
TypeScript

import * as zlib from "node:zlib";
import { createHash } from "node:crypto";
import { mkdir, symlink, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { scheduler } from "node:timers/promises";
import { describe, expect, it, vi } from "vitest";
import { AnalysisResourceConstraintError } from "../../../src/domain/analysisErrorCore.js";
import { createElfDebugLinkService } from "../../../src/composition/binaryDiagnostics.js";
import {
ELF_DEBUG_LINK_CANDIDATE_BUDGET_LIMITATION,
ELF_DEBUG_LINK_CANDIDATE_PREFIX_LIMITATION,
ELF_DEBUG_LINK_UNAVAILABLE_CHECK_LIMITATION,
elfDebugLinkSchema,
} from "../../../src/domain/native/elfDebugLink.js";
import {
compressedElfNoteFixture,
gnuBuildIdNote,
} from "../../../src/native/elf/ElfNotes.fixture.js";
import {
debuglinkSection,
elfDebugCandidateFixture,
elfDebugLinkFixture,
FIXTURE_BUILD_ID_HEX,
FIXTURE_DEBUG_NAME,
} from "../../../src/native/elf/ElfDebugLink.fixture.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { elfFrameAllowanceFixture } from "../../fixtures/binaryDiagnostics/elfFrameAllowance.js";
vi.mock(import("node:zlib"), { spy: true });
it("inflates section names once per artifact while preserving build-ID comparison", async () => {
const root = await createTestTempDirectory("rea-elf-debug-link-name-reuse-");
const path = join(root, "prog");
const candidatePath = join(root, "prog.debug");
const bytes = compressedElfNoteFixture();
await writeFile(path, bytes);
await writeFile(candidatePath, bytes);
const service = createElfDebugLinkService();
const inflate = vi.mocked(zlib.inflateSync).mockClear();
try {
const result = await service.inspect(
{ path, candidate_path: candidatePath },
{ signal: new AbortController().signal },
);
if (!result.ok) throw result.error;
const report = elfDebugLinkSchema.parse(result.value.normalized_result);
expect(report.build_id).toMatchObject({
status: "observed",
sources: [{ table_kind: "section", table_index: 1 }],
});
expect(report.verification?.build_id.status).toBe("match");
expect(inflate).toHaveBeenCalledTimes(2);
} finally {
inflate.mockClear();
await service.close();
}
});
describe("ELF debug-link stable filesystem boundary", () => {
it("reports non-file and malformed ELF selections as caller input errors", async () => {
const root = await createTestTempDirectory("rea-elf-debug-link-files-");
const path = join(root, "prog");
const alias = join(root, "alias");
const malformed = join(root, "malformed");
await writeFile(path, elfDebugLinkFixture());
await writeFile(malformed, Buffer.from("not an elf"));
await symlink(path, alias);
const service = createElfDebugLinkService();
try {
for (const [candidate, reason] of [
[root, "invalid_format"],
[alias, "invalid_format"],
[malformed, "invalid_format"],
[join(root, "missing"), "invalid_value"],
[join(path, "child"), "invalid_value"],
] as const) {
const result = await service.inspect({ path: candidate });
if (result.ok) throw new Error("Expected file acquisition failure");
expect(result.error).toMatchObject({
_tag: "AnalysisInputError",
operation: "inspect_elf_debug_link",
issues: [
{
path: ["path"],
reason,
message: expect.any(String),
},
],
});
}
} finally {
await service.close();
}
});
it("keeps the image linkage for a missing candidate and rejects symlinked or relative candidates", async () => {
const root = await createTestTempDirectory("rea-elf-debug-link-candidate-");
const path = join(root, "prog");
const alias = join(root, "alias.debug");
await writeFile(path, elfDebugLinkFixture());
await symlink(path, alias);
const service = createElfDebugLinkService();
try {
const missing = await service.inspect({
path,
candidate_path: join(root, "missing.debug"),
});
if (!missing.ok) throw missing.error;
const report = elfDebugLinkSchema.parse(missing.value.normalized_result);
expect(report.debuglink.status).toBe("present");
expect(report.build_id.hex).toBe(FIXTURE_BUILD_ID_HEX);
expect(report.lookup_paths.length).toBeGreaterThan(0);
expect(report.candidate).toBeNull();
expect(report.verification).toMatchObject({
status: "cannot_verify",
reason: "candidate_unreadable",
crc32: { status: "unavailable", reason: "candidate_unreadable" },
build_id: { status: "unavailable", reason: "candidate_unreadable" },
});
const linked = await service.inspect({ path, candidate_path: alias });
const relative = await service.inspect({
path,
candidate_path: "prog.debug",
});
if (linked.ok)
throw new Error("Expected a symlinked candidate rejection");
expect(linked.error).toMatchObject({
_tag: "AnalysisInputError",
issues: [{ path: ["candidate_path"] }],
});
if (relative.ok)
throw new Error("Expected a relative candidate rejection");
expect(relative.error).toMatchObject({
_tag: "AnalysisInputError",
issues: [{ path: ["candidate_path"], reason: "invalid_format" }],
});
} finally {
await service.close();
}
});
it("returns typed resource and cancellation failures", async () => {
const root = await createTestTempDirectory("rea-elf-debug-link-budget-");
const path = join(root, "prog");
const notes = Buffer.concat([
gnuBuildIdNote(FIXTURE_BUILD_ID_HEX, true),
gnuBuildIdNote("bb".repeat(20), true),
]);
await writeFile(path, elfDebugLinkFixture({ notes }));
const service = createElfDebugLinkService();
try {
const tooSmall = await service.inspect({ path, max_file_bytes: 64 });
if (tooSmall.ok) throw new Error("Expected resource budget failure");
expect(tooSmall.error._tag).toBe("AnalysisResourceConstraintError");
const tooMany = await service.inspect({ path, max_entries: 1 });
if (tooMany.ok) throw new Error("Expected entry budget failure");
if (!(tooMany.error instanceof AnalysisResourceConstraintError))
throw new Error("Expected entry budget failure");
expect(tooMany.error.remediationAction).toContain("smaller image");
const cancelled = await service.inspect(
{ path },
{ signal: AbortSignal.abort() },
);
if (cancelled.ok) throw new Error("Expected cancellation");
expect(cancelled.error._tag).toBe("AnalysisCancelledError");
} finally {
await service.close();
}
});
it("does not open a conventional lookup path when no candidate is selected", async () => {
const root = await createTestTempDirectory("rea-elf-debug-link-decoy-");
const path = join(root, "prog");
await writeFile(path, elfDebugLinkFixture());
await mkdir(join(root, ".debug"));
await mkdir(join(root, ".debug", "prog.debug"));
const service = createElfDebugLinkService();
try {
const result = await service.inspect({ path });
if (!result.ok) throw result.error;
const report = elfDebugLinkSchema.parse(result.value.normalized_result);
expect(report.verification).toBeNull();
expect(report.candidate).toBeNull();
expect(report.debuglink.status).toBe("present");
} finally {
await service.close();
}
});
});
describe("ELF debug-link streamed candidate boundary", () => {
it("retains streamed candidate identity when cancellation stops ELF comparison", async () => {
const root = await createTestTempDirectory(
"rea-elf-debug-link-cancel-candidate-",
);
const path = join(root, "prog");
const candidatePath = join(root, "prog.debug");
const candidate = elfFrameAllowanceFixture();
await writeFile(path, elfDebugLinkFixture());
await writeFile(candidatePath, candidate);
const service = createElfDebugLinkService();
const controller = new AbortController();
const reason = new Error("stop while decoding candidate names");
const yieldToEventLoop = scheduler.yield.bind(scheduler);
const pause = vi
.spyOn(scheduler, "yield")
.mockImplementationOnce(async () => {
await yieldToEventLoop();
controller.abort(reason);
});
try {
const result = await service.inspect(
{ path, candidate_path: candidatePath },
{ signal: controller.signal },
);
if (result.ok)
throw new Error("Expected candidate comparison cancellation");
expect(result.error).toMatchObject({
_tag: "AnalysisCancelledError",
cause: reason,
partialObservation: {
build_id: { status: "observed", hex: FIXTURE_BUILD_ID_HEX },
candidate: {
path: candidatePath,
sha256: createHash("sha256").update(candidate).digest("hex"),
bytes: candidate.length,
debug_info: "unknown",
symtab: "unknown",
},
verification: null,
},
});
} finally {
pause.mockRestore();
await service.close();
}
});
it("matches the known CRC32 of a non-ELF candidate", async () => {
const root = await createTestTempDirectory("rea-elf-debug-link-crc32-");
const path = join(root, "prog");
const candidatePath = join(root, FIXTURE_DEBUG_NAME);
await writeFile(
path,
elfDebugLinkFixture({
debuglink: debuglinkSection(FIXTURE_DEBUG_NAME, 0xcbf43926, true),
}),
);
await writeFile(candidatePath, "123456789");
const service = createElfDebugLinkService();
try {
const result = await service.inspect({
path,
candidate_path: candidatePath,
});
if (!result.ok) throw result.error;
const report = elfDebugLinkSchema.parse(result.value.normalized_result);
expect(report.candidate).toMatchObject({
format: null,
byte_order: null,
debug_info: "not_elf",
symtab: "not_elf",
});
expect(report.verification).toMatchObject({
status: "matches",
crc32: { status: "match", observed_hex: "cbf43926" },
build_id: { status: "unavailable", reason: "candidate_not_elf" },
});
expect(report.limitations).toContain(
ELF_DEBUG_LINK_UNAVAILABLE_CHECK_LIMITATION,
);
} finally {
await service.close();
}
});
it("streams a candidate larger than the image read budget", async () => {
const root = await createTestTempDirectory("rea-elf-debug-link-stream-");
// Trailing padding keeps the ELF header while exceeding several chunks.
const candidate = Buffer.concat([
elfDebugCandidateFixture(),
Buffer.alloc(200_000, 0x5a),
]);
const image = elfDebugLinkFixture({
debuglink: debuglinkSection(FIXTURE_DEBUG_NAME, 0x5f6d3801, true),
});
const path = join(root, "prog");
const candidatePath = join(root, FIXTURE_DEBUG_NAME);
await writeFile(path, image);
await writeFile(candidatePath, candidate);
const service = createElfDebugLinkService();
try {
const result = await service.inspect({
path,
candidate_path: candidatePath,
max_file_bytes: image.length,
});
if (!result.ok) throw result.error;
const report = elfDebugLinkSchema.parse(result.value.normalized_result);
expect(candidate.length).toBeGreaterThan(image.length);
expect(report.candidate).toMatchObject({
bytes: candidate.length,
format: "elf64",
byte_order: "little",
debug_info: "unknown",
symtab: "unknown",
});
expect(report.verification).toMatchObject({
status: "matches",
crc32: {
status: "match",
expected_hex: "5f6d3801",
observed_hex: "5f6d3801",
},
build_id: {
status: "unavailable",
reason: "candidate_build_id_unknown",
},
});
expect(report.limitations).toEqual(
expect.arrayContaining([
ELF_DEBUG_LINK_CANDIDATE_PREFIX_LIMITATION,
ELF_DEBUG_LINK_UNAVAILABLE_CHECK_LIMITATION,
]),
);
} finally {
await service.close();
}
});
});
describe("ELF debug-link candidate degradation", () => {
it.each([
["with", "candidate_elf_malformed"],
["without", "binary_build_id_absent"],
] as const)(
"reports a candidate with ELF magic and a bad header as malformed for an image %s a build-id",
async (presence, reason) => {
const root = await createTestTempDirectory(
"rea-elf-debug-link-malformed-",
);
// ELF magic, ELFCLASS64, little-endian, then an unsupported EI_VERSION.
const candidate = Buffer.alloc(64);
candidate.set([0x7f, 0x45, 0x4c, 0x46, 2, 1, 9]);
const image = elfDebugLinkFixture({
...(presence === "with" ? {} : { notes: null }),
debuglink: debuglinkSection(
FIXTURE_DEBUG_NAME,
zlib.crc32(candidate),
true,
),
});
const path = join(root, "prog");
const candidatePath = join(root, FIXTURE_DEBUG_NAME);
await writeFile(path, image);
await writeFile(candidatePath, candidate);
const service = createElfDebugLinkService();
try {
const result = await service.inspect({
path,
candidate_path: candidatePath,
});
if (!result.ok) throw result.error;
const report = elfDebugLinkSchema.parse(result.value.normalized_result);
expect(report.candidate).toMatchObject({
format: null,
byte_order: null,
debug_info: "unknown",
symtab: "unknown",
});
expect(report.verification).toMatchObject({
status: "matches",
crc32: { status: "match" },
build_id: { status: "unavailable", reason },
});
} finally {
await service.close();
}
},
);
it("degrades a candidate over the note entry budget without losing the image", async () => {
const root = await createTestTempDirectory("rea-elf-debug-link-entries-");
const candidate = elfDebugLinkFixture({
debuglink: null,
debugInfo: Buffer.from("debug-info", "latin1"),
notes: Buffer.concat([
gnuBuildIdNote(FIXTURE_BUILD_ID_HEX, true),
gnuBuildIdNote("bb".repeat(20), true),
]),
});
const image = elfDebugLinkFixture({
debuglink: debuglinkSection(
FIXTURE_DEBUG_NAME,
zlib.crc32(candidate),
true,
),
});
const path = join(root, "prog");
const candidatePath = join(root, FIXTURE_DEBUG_NAME);
await writeFile(path, image);
await writeFile(candidatePath, candidate);
const service = createElfDebugLinkService();
try {
const result = await service.inspect({
path,
candidate_path: candidatePath,
max_entries: 1,
});
if (!result.ok) throw result.error;
const report = elfDebugLinkSchema.parse(result.value.normalized_result);
expect(report.build_id).toMatchObject({
status: "observed",
hex: FIXTURE_BUILD_ID_HEX,
});
expect(report.debuglink.status).toBe("present");
expect(report.candidate).toMatchObject({
format: "elf64",
debug_info: "present",
symtab: "absent",
});
expect(report.verification).toMatchObject({
status: "matches",
crc32: { status: "match" },
build_id: {
status: "unavailable",
binary_hex: FIXTURE_BUILD_ID_HEX,
candidate_hex: null,
reason: "candidate_build_id_unknown",
},
});
expect(report.limitations).toEqual(
expect.arrayContaining([
ELF_DEBUG_LINK_CANDIDATE_BUDGET_LIMITATION,
ELF_DEBUG_LINK_UNAVAILABLE_CHECK_LIMITATION,
]),
);
} finally {
await service.close();
}
});
});