fix(erlang): disclose BEAM bytes trailing the declared FOR1 form (#9673)

* fix(erlang): disclose BEAM bytes trailing the declared FOR1 form

A conformant FOR1 container sizes its form to the whole file, so bytes
past the form end are an unexamined appended or corrupt region. The
inspector decoded chunks only within the form but still reported
coverage.status "complete" with no mention of the surplus, so a caller
could not distinguish a clean BEAM from one carrying foreign trailing
bytes.

Compute the trailing byte count, report coverage.status "partial" with a
trailing_bytes field, and append a limitation naming the form end and the
unexamined count. Clean containers stay "complete" with trailing_bytes 0.

* fix(erlang): reparse retained BEAM Evidence without the trailing-bytes field

Evidence retained by a build predating coverage.trailing_bytes has no such
property, and projectErlangBeamView reparses that record with the strict
erlangBeamSchema. A required field rejected those records with an input error
before any summary or page could render, breaking the offline retained-Evidence
flow even for clean files.

Default coverage.trailing_bytes to 0 so historical records reparse while every
new parse still emits the concrete count and preserves each record's stored
status.

* fix(erlang): keep retained BEAM trailing coverage unknown, not zero

Defaulting coverage.trailing_bytes to 0 manufactured a "no trailer" reading for
Evidence retained before the field existed, even when its retained bounds
(artifact.bytes versus form.location.bytes) prove a trailer. Make the field
optional so a pre-field record reparses as unknown instead of a fabricated
zero, while every new parse still emits the concrete count.

* fix(erlang): admit chunk extents before applying count limits

* docs(erlang): explain trailing BEAM coverage

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
This commit is contained in:
Muhammad Hamzaandmorluto authored and GitHub committed 2026-11-19 07:44:30 +00:00
1 parent d7aee71bc7
commit 93c8e0c0c8
8 files changed
+157 -36

No files matched your search

+6 -3
View File
@@ -27,9 +27,10 @@ The inline Evidence result contains the artifact path, SHA-256, and byte length.
Repeated names retain separate chunk occurrences. Decoded tables and the Code
header use the last occurrence of each name, matching OTP's loader; the chunk
inventory still exposes earlier payloads and their source ranges.
The BEAM form size bounds the parsed chunks. Bytes after the declared form are
ignored by the parser but remain part of the selected artifact's byte length
and SHA-256.
The BEAM form size bounds the parsed chunks. Bytes after the form remain part
of the artifact's byte length and SHA-256. They are unexamined by the chunk
reader, so the report sets `coverage.status` to `partial` and gives their exact
count in `coverage.trailing_bytes`. Older retained Evidence may omit the count.
When a selected `AtU8` or `Atom` table is decoded, `atoms` is the 1-based name table and
`module` is atom 1. Both the pre-OTP 28 table and the OTP 28+ compact table
@@ -126,6 +127,8 @@ The selected file is limited to 64 MiB by default (512 MiB maximum). At most
64 chunks and 4096 atoms are examined by default, and at most 65536 export,
import, and lambda rows together. Malformed `FOR1`/`BEAM` headers, truncated
chunks, and `AtU8` names that are not valid UTF-8 fail without partial success.
The reader checks each encountered chunk's payload and alignment padding against
the declared form before applying the chunk count limit.
This workflow does not execute bytecode, inflate `LitT` literals, or decode
`Attr`/`CInf` external-term payloads. Complete coverage describes this static
@@ -62,7 +62,6 @@
"src/dotnet/ManagedConstantValue.ts": 12,
"src/dotnet/ManagedMemberInstructionDecoder.ts": 30,
"src/dotnet/ManagedMethodBodyReader.ts": 5,
"src/erlang/BeamParser.ts": 3,
"src/flutter/DartElf.ts": 2,
"src/ghidra/GhidraLoadImageValues.ts": 1,
"src/ghidra/extensions/GhidraExtensions.ts": 1,
+6 -1
View File
@@ -24,7 +24,12 @@ const report: ErlangBeam = {
lambdas: null,
code: null,
module: null,
coverage: { status: "complete", examined_chunks: 0, atoms: 0 },
coverage: {
status: "complete",
examined_chunks: 0,
atoms: 0,
trailing_bytes: 0,
},
limitations: ["No atom table was present."],
};
+34 -1
View File
@@ -68,9 +68,42 @@ it("rejects extra caller arguments and admits a BEAM form with no tables", () =>
lambdas: null,
code: null,
module: null,
coverage: { status: "complete", examined_chunks: 0, atoms: 0 },
coverage: {
status: "complete",
examined_chunks: 0,
atoms: 0,
trailing_bytes: 0,
},
limitations: ["No atom table was present."],
});
expect(report.atoms).toBeNull();
expect(report.module).toBeNull();
});
it("reparses retained BEAM Evidence that predates the trailing-bytes field", () => {
// A record written by an earlier build carries no coverage.trailing_bytes;
// the retained-Evidence view must still reparse it instead of failing.
const legacy = {
artifact: {
path: "/artifacts/app.beam",
sha256: "a".repeat(64),
bytes: 40,
},
form: { location: { offset: 0, bytes: 24 }, size: 16 },
chunks: [],
atoms: null,
atom_encoding: null,
exports: null,
imports: null,
lambdas: null,
code: null,
module: null,
coverage: { status: "complete", examined_chunks: 1, atoms: 0 },
limitations: [],
};
const parsed = erlangBeamSchema.parse(legacy);
// Preserved as unknown, not a fabricated zero, even though the bounds imply a
// trailer; the producing build never measured it.
expect(parsed.coverage.trailing_bytes).toBeUndefined();
expect(parsed.coverage.status).toBe("complete");
});
+6 -1
View File
@@ -1,4 +1,5 @@
import { z } from "zod";
import { knownCoverageStatusSchema } from "../coverage.js";
import { digestSchema } from "../digests.js";
import { absoluteLocalPathStringSchema } from "../localPath.js";
@@ -117,9 +118,13 @@ export const erlangBeamSchema = z.strictObject({
.nullable(),
module: z.string().nullable(),
coverage: z.strictObject({
status: z.literal("complete"),
// "partial" when bytes trail the declared FOR1 form and go unexamined.
status: knownCoverageStatusSchema,
examined_chunks: uint,
atoms: uint,
// Optional so Evidence retained by builds predating this field reparses
// as unknown rather than a fabricated zero; every new parse emits the count.
trailing_bytes: uint.optional(),
}),
limitations: z.array(z.string()),
});
+28
View File
@@ -251,6 +251,34 @@ it("fails a FOR1 form that is not BEAM", () => {
expect(() => parse(bytes)).toThrow(ArtifactReaderFailure);
});
it("discloses bytes that trail the declared FOR1 form", () => {
const valid = erlangBeamFixture("trailing_probe");
const clean = parse(valid);
expect(clean.coverage.status).toBe("complete");
expect(clean.coverage.trailing_bytes).toBe(0);
// Append foreign bytes past the declared form; a conformant file ends there.
const trailing = Buffer.concat([valid, Buffer.from("APPENDEDOVERLAY!")]);
const report = parse(trailing);
expect(report.form.location.bytes).toBe(valid.length);
expect(report.coverage.status).toBe("partial");
expect(report.coverage.trailing_bytes).toBe(16);
// The surplus bytes are not misread as a chunk; examined facts are unchanged.
expect(report.chunks).toEqual(clean.chunks);
expect(report.coverage.examined_chunks).toBe(clean.coverage.examined_chunks);
expect(report.limitations).toEqual([
...clean.limitations,
`The declared FOR1 form ends at offset ${String(valid.length)}; 16 trailing bytes were not examined.`,
]);
const single = parse(Buffer.concat([valid, Buffer.from("!")]));
expect(single.coverage.status).toBe("partial");
expect(single.coverage.trailing_bytes).toBe(1);
expect(single.limitations.at(-1)).toBe(
`The declared FOR1 form ends at offset ${String(valid.length)}; 1 trailing byte was not examined.`,
);
});
it("rejects a short remainder inside the BEAM form", () => {
const bytes = Buffer.alloc(16);
bytes.write("FOR1");
+39 -29
View File
@@ -1,5 +1,6 @@
import { isUtf8 } from "node:buffer";
import { ArtifactReaderFailure } from "../resources/index.js";
import { BinaryReader } from "../domain/binaryContainer.js";
import {
MAX_INSPECT_ERLANG_BEAM_TABLE_ROWS,
type ErlangBeam,
@@ -41,16 +42,6 @@ const requireRange = (bytes: Buffer, offset: number, length: number): void => {
);
};
const u32 = (bytes: Buffer, offset: number): number => {
requireRange(bytes, offset, 4);
return bytes.readUInt32BE(offset);
};
const chunkName = (bytes: Buffer, offset: number): string => {
requireRange(bytes, offset, 4);
return bytes.subarray(offset, offset + 4).toString("latin1");
};
const atomAt = (
atoms: readonly { index: number; name: string }[] | null,
index: number,
@@ -137,8 +128,8 @@ const parseAtoms = (
// Only AtU8 supports OTP 28+'s negated count and compact name lengths.
// Historical Atom uses an unsigned count and single-byte lengths.
const encoding = format === "AtU8" ? "utf8" : "latin1";
const declared =
format === "AtU8" ? payload.readInt32BE(0) : payload.readUInt32BE(0);
const rawCount = new BinaryReader(payload, false, "beam").u32(0);
const declared = format === "AtU8" ? rawCount | 0 : rawCount;
const compact = format === "AtU8" && declared < 0;
const count = compact ? -declared : declared;
// Every entry needs at least one length byte, regardless of the layout.
@@ -171,7 +162,7 @@ const parseAtoms = (
const tableCount = (payload: Buffer, width: number, label: string): number => {
if (payload.length < 4) fail(`BEAM ${label} table is truncated.`);
const count = u32(payload, 0);
const count = new BinaryReader(payload, false, "beam").u32(0);
if (count > Math.floor((payload.length - 4) / (width * 4)))
fail(`BEAM ${label} table is truncated.`);
return count;
@@ -182,12 +173,13 @@ const parseRows = (
width: number,
count: number,
): number[][] => {
const reader = new BinaryReader(payload, false, "beam");
const rows: number[][] = [];
for (let index = 0; index < count; index += 1) {
const row: number[] = [];
const start = 4 + index * width * 4;
for (let field = 0; field < width; field += 1)
row.push(u32(payload, start + field * 4));
row.push(reader.u32(start + field * 4));
rows.push(row);
}
return rows;
@@ -198,15 +190,16 @@ const parseCode = (
chunkOffset: number,
): NonNullable<ErlangBeam["code"]> => {
if (payload.length < 20) fail("BEAM Code chunk is truncated.");
const infoSize = u32(payload, 0);
const reader = new BinaryReader(payload, false, "beam");
const infoSize = reader.u32(0);
if (infoSize < 16 || infoSize > payload.length - 4)
fail("BEAM Code header size is invalid.");
const instructionsOffset = 4 + infoSize;
return {
instruction_set: u32(payload, 4),
opcode_max: u32(payload, 8),
labels: u32(payload, 12),
functions: u32(payload, 16),
instruction_set: reader.u32(4),
opcode_max: reader.u32(8),
labels: reader.u32(12),
functions: reader.u32(16),
extra_header_bytes: infoSize - 16,
instructions: {
offset: chunkOffset + 8 + instructionsOffset,
@@ -224,30 +217,37 @@ export const parseErlangBeam = (
requireRange(bytes, 0, 12);
if (bytes.subarray(0, 4).toString("latin1") !== "FOR1")
fail("Expected a FOR1 IFF container.");
const formSize = u32(bytes, 4);
const carrier = new BinaryReader(bytes, false, "beam");
const formSize = carrier.u32(4);
if (formSize < 4) fail("BEAM form size is shorter than the BEAM tag.");
requireRange(bytes, 8, formSize);
if (bytes.subarray(8, 12).toString("latin1") !== "BEAM")
fail("Expected a BEAM form type.");
const formEnd = 8 + formSize;
const form = new BinaryReader(
carrier.range(0, formEnd, "BEAM form"),
false,
"beam",
);
const chunks: ErlangBeam["chunks"] = [];
const payloads = new Map<string, { payload: Buffer; offset: number }>();
let at = 12;
while (at < formEnd) {
if (formEnd - at < 8) fail("BEAM chunk header is truncated.");
const size = form.u32(at + 4);
const payloadOffset = at + 8;
if (size > formEnd - payloadOffset)
fail("BEAM chunk payload exceeds the form.");
const padded = payloadOffset + size + ((4 - (size % 4)) % 4);
if (padded > formEnd) fail("BEAM chunk padding exceeds the form.");
if (chunks.length >= input.max_chunks)
throw new BeamCapacityFailure(
"chunks",
input.max_chunks,
`BEAM form exceeds the ${String(input.max_chunks)}-chunk budget.`,
);
const name = chunkName(bytes, at);
const size = u32(bytes, at + 4);
const payloadOffset = at + 8;
requireRange(bytes, payloadOffset, size);
const padded = payloadOffset + size + ((4 - (size % 4)) % 4);
if (padded > formEnd) fail("BEAM chunk padding exceeds the form.");
const payload = bytes.subarray(payloadOffset, payloadOffset + size);
const name = form.range(at, 4, "BEAM chunk name").toString("latin1");
const payload = form.range(payloadOffset, size, "BEAM chunk payload");
chunks.push({
index: chunks.length,
name,
@@ -341,6 +341,9 @@ export const parseErlangBeam = (
codeChunk === undefined
? null
: parseCode(codeChunk.payload, codeChunk.offset);
// A conformant FOR1 form sizes itself to the file; any surplus bytes trail an
// unexamined region. Disclose them instead of reporting complete coverage.
const trailingBytes = bytes.length - formEnd;
return {
artifact,
form: { location: { offset: 0, bytes: formEnd }, size: formSize },
@@ -353,10 +356,17 @@ export const parseErlangBeam = (
code,
module: atoms?.[0]?.name ?? null,
coverage: {
status: "complete",
status: trailingBytes > 0 ? "partial" : "complete",
examined_chunks: chunks.length,
atoms: atoms?.length ?? 0,
trailing_bytes: trailingBytes,
},
limitations: [...LIMITATIONS],
limitations:
trailingBytes > 0
? [
...LIMITATIONS,
`The declared FOR1 form ends at offset ${String(formEnd)}; ${String(trailingBytes)} trailing ${trailingBytes === 1 ? "byte was" : "bytes were"} not examined.`,
]
: [...LIMITATIONS],
};
};
+38
View File
@@ -403,6 +403,19 @@ cliTest(
expect(trailing.normalized_result.form.location.bytes).toBe(
formBytes.length,
);
// The declared form parses identically, but the surplus bytes are disclosed
// rather than silently folded into complete coverage.
expect(control.normalized_result.coverage).toMatchObject({
status: "complete",
trailing_bytes: 0,
});
expect(trailing.normalized_result.coverage).toMatchObject({
status: "partial",
trailing_bytes: trailer.length,
});
expect(trailing.normalized_result.limitations).toContain(
`The declared FOR1 form ends at offset ${String(formBytes.length)}; ${String(trailer.length)} trailing bytes were not examined.`,
);
},
);
@@ -425,6 +438,31 @@ cliTest(
},
);
cliTest(
"checks the BEAM chunk extent before reporting the chunk-count limit",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-beam-chunk-extent-");
const path = join(root, "app.beam");
const bytes = erlangBeamFixture();
const firstSize = bytes.readUInt32BE(16);
const secondHeader = 20 + Math.ceil(firstSize / 4) * 4;
await writeFile(path, bytes);
const inspect = () =>
cli.run({
arguments: ["inspect-erlang-beam", path, "--max-chunks", "1", "--json"],
});
const admitted = await inspect();
expect(admitted.json).toMatchObject({ code: "resource_constraint" });
// Retain bytes beyond the declared form. They cannot supply the second
// chunk's payload even though the complete file contains it.
bytes.writeUInt32BE(secondHeader, 4);
await writeFile(path, bytes);
const refused = await inspect();
expect(refused.json).toMatchObject({ code: "invalid_request" });
},
);
cliTest(
"resolves a relative BEAM path against the working directory",
async ({ cli }) => {