fix(artifacts): keep ESP flash coverage complete for a dump cut short

An esptool merge-bin dump ends after the last part it wrote, usually inside
its last partition. Such a partition made the layout's coverage unknown, so
artifact comparison of two merge-bin dumps reported every unchanged
partition as unknown.

The partition table names every partition whatever the file's length, so a
partition cut short or left out by the end of the file hides no entry. As
the ESP image reader does for a cut-short segment, its entry holds the bytes
the file has and an entry limitation gives the declared size; a partition
with no bytes stays in partitions_outside_dump. Coverage stays complete
unless undescribed bytes remain or no bootloader is located. An MCP
compare_artifacts test of two dumps that end inside ota_0 now classifies
every partition, and fails on the old rule.

docs/esp-flash-image.md

@@ -119,10 +119,12 @@ partitions like any other, whose bytes are also `bootloader.bin` and
 `partition-table.bin`, and they are exempt from the rule that partitions
 follow the table, as in `gen_esp32part.py`.
 
-A partition that the file holds only in part is the bytes the file holds,
-with a limitation. One unreadable partition does not hide the others:
-partitions outside the dump, before its first byte or overlapping another
-are each reported while the rest stay readable.
+A partition that the file holds only in part, as when an `esptool merge-bin`
+dump ends inside its last partition, is the bytes the file holds, with an
+entry limitation that gives the size the table declares. One unreadable
+partition does not hide the others: partitions outside the dump, before its
+first byte or overlapping another are each reported while the rest stay
+readable.
 
 ### App partitions and the bootloader
 
@@ -173,8 +175,11 @@ to another partition when an app fails to load.
 Erased flash (`0xFF`) between regions is described. In a gap between
 regions, the first byte that is not erased starts an `undescribed` range to
 the end of the gap, and makes coverage `unknown`; the rest of that range is
-not examined. Coverage is also `unknown` when a partition is truncated or
-outside the dump, or when no bootloader is located.
+not examined. Coverage is also `unknown` when no bootloader is located. A
+partition cut short or left out by the end of the file does not make it
+`unknown`: the table names every partition, so no entry is hidden, and the
+entry or `partitions_outside_dump` says what the file lacks. Artifact
+comparison of two such dumps therefore classifies every partition.
 
 `extract_artifact` writes every entry as an ordinary file, `nvs_keys`
 partitions included. On a device without flash encryption an `nvs_keys`
@@ -216,8 +221,9 @@ offsets, single images that are not scanned, overlaps, primary entries,
 truncation and partitions outside the dump, index names for unsafe labels,
 repeated declarations, the otadata states and records, and each branch of
 the bootloader's selection. CLI and stdio MCP tests exercise the inputs and
-their identity and extraction, and an MCP `compare_artifacts` test compares
-partition declarations, labels that are not UTF-8 included.
+their identity and extraction. MCP `compare_artifacts` tests compare
+partition declarations, labels that are not UTF-8 included, and two dumps
+that end inside their last partition, whose partitions are each classified.
 
 The optional real lane builds its dumps with esptool and ESP-IDF's own tools:
 
@@ -250,7 +256,8 @@ otatool's own OTA logic runs unchanged. The lane then checks:
 - the relocated table, found by scanning and named with
   `partition_table_offset`, and the dump written from `0x1000`, read with
   `flash_offset` and, without it, left an ESP image;
-- an app with a flipped byte beside valid partitions, and a truncated dump.
+- an app with a flipped byte beside valid partitions, and a truncated dump,
+  read with complete coverage and its missing partitions outside the dump.
 
 The lane ran on macOS arm64 with Node 24.22.1, Homebrew Clang and LLD 21.1.8
 and ESP-IDF v5.5.2's `gen_esp32part.py`, `otatool.py` and `parttool.py`,

scripts/verify/firmware/esp-flash.mjs

@@ -357,7 +357,8 @@ const damaged = async (report) => {
   assert.equal(appFacts(rea, "ota_0").image.checksum.status, "valid");
   await writeFile(path("truncated.bin"), bytes.subarray(0, 0x4_0000));
   const cut = await inspect(path("truncated.bin"));
-  assert.equal(cut.inventory.coverage.status, "unknown");
+  // Partitions cut short or left out by the end of the file hide no entry.
+  assert.equal(cut.inventory.coverage.status, "complete");
   assert.deepEqual(
     cut.root.partitions_outside_dump.map(({ label }) => label),
     ["ota_1", "keys"],

src/artifacts/EspFlashImageArtifactReader.test.ts

@@ -166,7 +166,8 @@ describe("ESP flash layout coverage and selection", () => {
   it("keeps usable partitions beside a truncated and a missing one", async () => {
     const cut = DUMP.subarray(0, 0x2_8000);
     const snapshot = await scanCanonicalArtifactInventory(await writeDump(cut));
-    expect(snapshot.coverage.status).toBe("unknown");
+    // The table names every partition, so the end of the file hides no entry.
+    expect(snapshot.coverage.status).toBe("complete");
     const ota0 = snapshot.occurrences.find(
       ({ logical_path }) => logical_path === "partitions/ota_0.bin",
     );

src/domain/firmware/espFlashLayout.ts

@@ -345,17 +345,13 @@ export const finishEspFlashLayout = (
     plan.otadata === undefined
       ? null
       : otaFacts(plan, plan.otadata, reads.otadataRecords);
-  const incomplete = plan.input.table.partitions.some((part) => {
-    const region = plan.regions.find((item) => item.partition === part);
-    return region === undefined || region.available < part.size;
-  });
+  // The table names every partition, so one cut short or left out by the end
+  // of the file hides no entry: its entry says what the file lacks.
   return {
     otadata,
     undescribed,
     coverage:
-      undescribed.length > 0 ||
-      incomplete ||
-      plan.input.bootloader === undefined
+      undescribed.length > 0 || plan.input.bootloader === undefined
         ? "unknown"
         : "complete",
     limitations,

tests/acceptance/artifactComparisonEspFlash.test.ts

@@ -3,12 +3,14 @@ import { join } from "node:path";
 import { expect, it } from "vitest";
 
 import {
+  ESP32C3_APP_SEGMENTS,
   ESP32C3_BOOTLOADER_SEGMENTS,
   espImage,
 } from "../../src/domain/firmware/espImage.fixture.js";
 import {
   TWO_OTA_PARTITIONS,
   espFlashDump,
+  flashImage,
   partitionTable,
 } from "../../src/domain/firmware/espFlash.fixture.js";
 import { artifactComparisonResultSchema } from "../../src/domain/artifactComparison.js";
@@ -33,12 +35,13 @@ const labelBytes = (bytes: readonly number[]): Buffer => {
   return table;
 };
 
-it("reports an ESP partition whose declaration alone changed", async () => {
+/** Inspect files and compare their inventories through one MCP session. */
+const comparisonSession = async () => {
   const root = await createTestTempDirectory("rea-esp-flash-comparison-");
   const { call } = await connectLocalToolsMcp();
-  const inspect = async (name: string, table: Buffer): Promise<Evidence> => {
+  const inspect = async (name: string, bytes: Buffer): Promise<Evidence> => {
     const path = join(root, name);
-    await writeFile(path, espFlashDump({ bootloader: BOOTLOADER, table }));
+    await writeFile(path, bytes);
     const opened = await call("open_binary", { path });
     expect(opened.isError, JSON.stringify(opened)).not.toBe(true);
     const response = await call("inspect_artifact", {});
@@ -58,25 +61,37 @@ it("reports an ESP partition whose declaration alone changed", async () => {
       parseEvidence(response.structuredContent).normalized_result,
     );
   };
-  const occurrence = (evidence: Evidence, path: string) => {
-    const found = artifactInventoryResultSchema
-      .parse(evidence.normalized_result)
-      .occurrences.find(({ logical_path }) => logical_path === path);
-    if (found === undefined) throw new Error(`Missing ${path}`);
-    return found;
-  };
-  const changedPaths = (result: { changes: { logical_path: string }[] }) =>
-    result.changes
-      .map(({ logical_path }) => logical_path)
-      .filter((path) => path !== ".")
-      .sort();
+  return { inspect, compare };
+};
+
+const occurrence = (evidence: Evidence, path: string) => {
+  const found = artifactInventoryResultSchema
+    .parse(evidence.normalized_result)
+    .occurrences.find(({ logical_path }) => logical_path === path);
+  if (found === undefined) throw new Error(`Missing ${path}`);
+  return found;
+};
+
+const changedPaths = (result: { changes: { logical_path: string }[] }) =>
+  result.changes
+    .map(({ logical_path }) => logical_path)
+    .filter((path) => path !== ".")
+    .sort();
+
+it("reports an ESP partition whose declaration alone changed", async () => {
+  const { inspect, compare } = await comparisonSession();
+  const dump = (table: Buffer) =>
+    espFlashDump({ bootloader: BOOTLOADER, table });
 
   // The same partition bytes, with nvs declared read-only.
   const readonly = TWO_OTA_PARTITIONS.map((part) =>
     part.label === "nvs" ? { ...part, flags: 0x2 } : part,
   );
-  const left = await inspect("flash.bin", partitionTable(TWO_OTA_PARTITIONS));
-  const right = await inspect("readonly.bin", partitionTable(readonly));
+  const left = await inspect(
+    "flash.bin",
+    dump(partitionTable(TWO_OTA_PARTITIONS)),
+  );
+  const right = await inspect("readonly.bin", dump(partitionTable(readonly)));
   const leftNvs = occurrence(left, "partitions/nvs.bin");
   const rightNvs = occurrence(right, "partitions/nvs.bin");
   expect(rightNvs.artifact_id).toBe(leftNvs.artifact_id);
@@ -104,8 +119,8 @@ it("reports an ESP partition whose declaration alone changed", async () => {
   ]);
 
   // Labels that are not UTF-8 compare by their bytes.
-  const first = await inspect("label-a.bin", labelBytes([0xff, 0x01]));
-  const second = await inspect("label-b.bin", labelBytes([0xfe, 0x01]));
+  const first = await inspect("label-a.bin", dump(labelBytes([0xff, 0x01])));
+  const second = await inspect("label-b.bin", dump(labelBytes([0xfe, 0x01])));
   expect(occurrence(first, "partitions/0.bin").format_metadata).toMatchObject({
     label: null,
     label_hex: "ff01",
@@ -118,7 +133,45 @@ it("reports an ESP partition whose declaration alone changed", async () => {
   // Equal declarations leave every partition unchanged.
   const same = await compare(
     left,
-    await inspect("copy.bin", partitionTable(TWO_OTA_PARTITIONS)),
+    await inspect("copy.bin", dump(partitionTable(TWO_OTA_PARTITIONS))),
   );
   expect(same.changes).toEqual([]);
 });
+
+it("classifies every partition of two dumps that end inside their last partition", async () => {
+  const { inspect, compare } = await comparisonSession();
+  const app = espImage({ segments: ESP32C3_APP_SEGMENTS });
+  // The same app with its DRAM segment loading 8 KiB higher.
+  const moved = espImage({
+    segments: ESP32C3_APP_SEGMENTS.map((segment, index) =>
+      index === 1 ? { ...segment, address: segment.address + 0x2000 } : segment,
+    ),
+  });
+  // Like esptool merge-bin: the file ends after the last part written, here
+  // inside ota_0, and ota_1 lies past its end.
+  const mergeBin = (factory: Buffer) =>
+    flashImage(0x2_0000 + app.length, [
+      BOOTLOADER,
+      { at: 0x8000, bytes: partitionTable(TWO_OTA_PARTITIONS) },
+      { at: 0x1_0000, bytes: factory },
+      { at: 0x2_0000, bytes: app },
+    ]);
+  const left = await inspect("left.bin", mergeBin(app));
+  const right = await inspect("right.bin", mergeBin(moved));
+  const inventory = artifactInventoryResultSchema.parse(left.normalized_result);
+  expect(inventory.coverage.status).toBe("complete");
+  expect(occurrence(left, "partitions/ota_0.bin").limitations).toEqual([
+    `partitions/ota_0.bin is truncated: the dump holds ${String(app.length)} of its 65536 bytes.`,
+  ]);
+
+  const result = await compare(left, right);
+  expect(result.status).toBe("changed");
+  expect(changedPaths(result)).toEqual(["partitions/factory.bin"]);
+  // The bootloader, the table, nvs, otadata, phy_init and the truncated
+  // ota_0 hold the same bytes on both sides.
+  expect(result.summary).toMatchObject({
+    unchanged: 6,
+    changed: 2,
+    unknown: 0,
+  });
+});