mirror of
https://github.com/morluto/rea.git
synced 2026-11-09 00:07:18 +00:00
fix(artifacts): read ESP flash layouts as the bootloader does
Read the partition table as bootloader_utility_load_partition_table does: the last factory, otadata and same-subtype OTA declaration wins, and a limitation names repeats. Plan each otadata record within its partition and the dump. An otadata partition smaller than two sectors selects none, the bootloader's INVALID_INDEX; a record the dump lacks leaves the selection unknown while the other record stays reported. The fallbacks follow the bootloader's indexes: ota_0-fallback and factory-fallback name no partition when the table declares none, and the assumptions add GPIO holds and deep-sleep wake. Refuse a dump whose bootloader contradicts its origin before the default table offset is tried, whether flash_offset was omitted or given, and keep the ESP image reader's ESP8266 refusal. A table never lies inside the located bootloader image, so single app images are no longer scanned. An explicit esp-flash-image selection is decided before other container signatures, and the native commands' --target-format no longer takes it. Say when no bootloader confirms the default origin. Exempt the primary bootloader and partition_table entries of ESP-IDF 5.3 from placement limits, report every region inside a longer one, and name partitions by index when labels repeat ignoring case. Keep an undecodable label's bytes in label_hex and compare them. Word undescribed ranges as starting at a byte that is not erased, and treat a short erased region as erased. Add an MCP compare_artifacts test for partition declarations, reader identity and close tests, scan edge cases, the 0x2000 ROM offset and a literal otatool CRC.
This commit is contained in:
1 parent
43247a2afb
commit
4561779b58
19 files changed
+1188
-330
No files matched your search
@@ -679,7 +679,8 @@ cannot be opened as raw-binary. See
|
||||
A whole-flash dump opens as `esp-flash-image`. `root_format_metadata` holds
|
||||
`partition_table` (its `method`: `default`, `caller` or a derived
|
||||
`scan`), `bootloader` and `apps` ESP image facts, `otadata` with a derived
|
||||
`selected` slot and its `assumptions`, and `partitions_outside_dump`;
|
||||
`selected` slot (null when a record is missing) and its `assumptions`, and
|
||||
`partitions_outside_dump`;
|
||||
`running_partition` is always `unknown`. Partition rows
|
||||
(`partitions/<label>.bin`) carry the declared `type`, `subtype`, names,
|
||||
`flash_address`, `size` and flags; `encrypted` is a storage flag, not
|
||||
|
||||
+109
-52
@@ -16,28 +16,45 @@ headers, segments, integrity and descriptors appear in the root
|
||||
|
||||
## Recognition
|
||||
|
||||
A file is a flash image when REA finds a partition table that ESP-IDF's
|
||||
bootloader accepts (`esp_partition_table_verify`): 32-byte `0x50AA` entries,
|
||||
at most one `0xEBEB` MD5 entry over the entries before it, and a terminating
|
||||
entry within `0xC00` bytes. The table is located in this order:
|
||||
A file is a flash image when REA finds a partition table in the entry
|
||||
structure ESP-IDF's bootloader verifies (`esp_partition_table_verify`):
|
||||
32-byte `0x50AA` entries, at most one `0xEBEB` MD5 entry over the entries
|
||||
before it, and a terminating entry within `0xC00` bytes. ESP-IDF accepts a
|
||||
table with no entries, but REA requires one so that erased flash is not a
|
||||
table. A table whose MD5 does not match, or that has a second MD5 entry, is
|
||||
still read, with `md5.status` `invalid` and a limitation, because the
|
||||
bootloader would refuse it. The table is located in this order:
|
||||
|
||||
1. at the caller's `partition_table_offset`, when given; no scan follows;
|
||||
2. at the default `0x8000`;
|
||||
3. by scanning every `0x1000`-aligned flash sector for a table whose MD5
|
||||
verifies. This location is derived, and the inventory says so.
|
||||
|
||||
Without a caller selection, the scan runs only when an ESP image sits at a
|
||||
ROM bootloader offset, so ordinary files are not scanned. A C3 or S3 dump
|
||||
begins with its bootloader, so flash layouts are detected before a file is
|
||||
read as one ESP image. An image at another chip's ROM offset, such as an
|
||||
ESP32 bootloader at the start of a dump read from `0x1000`, means the dump's
|
||||
origin is not flash address 0. REA then does not scan, because a table found
|
||||
that way would place every partition at the wrong file offset; pass
|
||||
`flash_offset`.
|
||||
A table never lies inside the located bootloader image, so the default
|
||||
offset and the scan are tried only after the image's last segment. Without
|
||||
a caller selection, the scan runs only when an ESP image sits at a ROM
|
||||
bootloader offset, and a single app or bootloader image ends at its last
|
||||
segment, so neither ordinary files nor single images are scanned. A C3 or
|
||||
S3 dump begins with its bootloader, so flash layouts are detected before a
|
||||
file is read as one ESP image.
|
||||
|
||||
Unless the caller names the table with `partition_table_offset`, REA
|
||||
refuses two kinds of dump before looking for a table:
|
||||
|
||||
- an image of a known chip at another chip's ROM offset, such as an ESP32
|
||||
bootloader at the start of a dump read from `0x1000`. The dump's origin
|
||||
is not the flash address REA assumed, given or default, and a table found
|
||||
there would place every partition at the wrong file offset; pass the
|
||||
`flash_offset` where the dump starts.
|
||||
- an ESP8266 image at a ROM offset. ESP8266 dumps can hold the same table
|
||||
format, and they keep the [ESP image](esp-image.md) reader's ESP8266
|
||||
refusal.
|
||||
|
||||
When no table is found the layout is not recognized. This is an unknown
|
||||
outcome, not a malformed file: flash encryption, which encrypts the
|
||||
bootloader and the table, is one cause.
|
||||
bootloader and the table, is one cause. When a table is found without a
|
||||
located bootloader and `flash_offset` is omitted, a limitation says that
|
||||
each partition's file offset assumes the dump starts at flash `0x0`.
|
||||
|
||||
## Caller inputs
|
||||
|
||||
@@ -72,7 +89,8 @@ The root `format_metadata` (`esp-flash-image`) records:
|
||||
- `apps`: ESP image facts for each app partition the dump holds;
|
||||
- `otadata`: the OTA records and the derived selection;
|
||||
- `partitions_outside_dump`: declared partitions with no bytes in the file;
|
||||
- `undescribed`: byte ranges outside every region that are not erased flash.
|
||||
- `undescribed`: ranges outside every region, each from a byte that is not
|
||||
erased flash to the end of its gap.
|
||||
|
||||
The ROM loads the bootloader from `ESP_ROM_BOOTLOADER_OFFSET_FLASH`:
|
||||
`0x1000` for ESP32 and ESP32-S2; `0x0` for ESP32-S3, C2, C3, C6, C61, H2 and
|
||||
@@ -81,15 +99,25 @@ where its own `chip_id` says the ROM loads it.
|
||||
|
||||
### Partitions
|
||||
|
||||
Each partition entry is `partitions/<label>.bin` when the label is a safe,
|
||||
unique name of letters, digits, `.`, `_` and `-` with at least one letter or
|
||||
underscore, and `partitions/<index>.bin` otherwise; `source_path` keeps the
|
||||
label. Its `format_metadata` (`esp-flash-partition`) records the table
|
||||
`index`, `label` (null when not UTF-8), `type`, `subtype`, ESP-IDF's
|
||||
`type_name` and `subtype_name`, `flash_address`, `size`, the `encrypted` and
|
||||
`readonly` flags and any `unknown_flags`. The `encrypted` flag says how a
|
||||
device with flash encryption stores the partition; it does not establish
|
||||
that the dump's bytes are ciphertext.
|
||||
Each partition entry is `partitions/<label>.bin` when the label is a safe
|
||||
name of up to 16 letters, digits, `.`, `_` and `-` that starts with a
|
||||
letter, digit or underscore and has at least one letter or underscore, and
|
||||
no other label repeats it, ignoring case as some file systems do; it is
|
||||
`partitions/<index>.bin` otherwise. `source_path` keeps the label, the
|
||||
table's own name for the partition. Its `format_metadata`
|
||||
(`esp-flash-partition`) records the table `index`, `label` (null when not
|
||||
UTF-8), `label_hex` (the label's bytes when it is not UTF-8, else null),
|
||||
`type`, `subtype`, ESP-IDF's `type_name` and `subtype_name`,
|
||||
`flash_address`, `size`, the `encrypted` and `readonly` flags and any
|
||||
`unknown_flags`. The `encrypted` flag says how a device with flash
|
||||
encryption stores the partition; it does not establish that the dump's
|
||||
bytes are ciphertext.
|
||||
|
||||
ESP-IDF 5.3 and later may declare primary `bootloader` and `partition_table`
|
||||
entries over the bootloader region and the table sector. They are
|
||||
partitions like any other, whose bytes are also `bootloader.bin` and
|
||||
`partition-table.bin`, and they are exempt from the rule that partitions
|
||||
follow the table, as in `gen_esp32part.py`.
|
||||
|
||||
A partition that the file holds only in part is the bytes the file holds,
|
||||
with a limitation. One unreadable partition does not hide the others:
|
||||
@@ -109,27 +137,44 @@ segments as entries.
|
||||
### OTA data
|
||||
|
||||
The otadata partition holds two `esp_ota_select_entry_t` records, one at the
|
||||
start of each 4 KiB sector. REA reports each record's `ota_seq`,
|
||||
`seq_label`, `ota_state` and its name, `crc`, whether the CRC matches (CRC-32
|
||||
of `ota_seq` seeded with `0xFFFFFFFF`) and whether the bootloader would treat
|
||||
start of each 4 KiB sector. Like the bootloader, REA reads the last otadata
|
||||
partition the table declares, and the last factory app and OTA app of each
|
||||
subtype; a limitation names any repeated declaration. REA reports each
|
||||
record the partition and the dump hold: its `ota_seq`, `seq_label`,
|
||||
`ota_state` and its name, `crc`, whether the CRC matches (CRC-32 of
|
||||
`ota_seq` seeded with `0xFFFFFFFF`) and whether the bootloader would treat
|
||||
it as valid.
|
||||
|
||||
`selected` is derived as `bootloader_utility_get_selected_boot_partition`
|
||||
does without app rollback or anti-rollback: the valid record with the higher
|
||||
`ota_seq` selects slot `(ota_seq - 1) % ota_app_count`, indexed by OTA
|
||||
subtype. With no usable record or no OTA app, the factory app is selected,
|
||||
else `ota_0`. Its `assumptions` list what the derivation leaves out.
|
||||
`running_partition` is always `unknown`: at boot the bootloader may first
|
||||
mark a pending-verify record aborted, check the app's secure version against
|
||||
eFuse, and fall back to another partition when an app fails to load.
|
||||
does without app rollback or anti-rollback. `basis` names its branch:
|
||||
|
||||
- `otadata`: the valid record with the higher `ota_seq` (record 0 on a tie)
|
||||
selects slot `(ota_seq - 1) % ota_app_count`, indexed by OTA subtype;
|
||||
- `factory-fallback`: no record is usable, or the table has no OTA app, and
|
||||
a factory app is declared; or records are set but none is valid, as after
|
||||
a CRC mismatch, which selects the factory index whether or not a factory
|
||||
app is declared;
|
||||
- `ota_0-fallback`: no record is usable and no factory app is declared, so
|
||||
the bootloader selects boot index 0;
|
||||
- `none`: the otadata partition is smaller than two 4 KiB sectors, so the
|
||||
bootloader cannot read it and resets without selecting an app.
|
||||
|
||||
`partition_index` is null when the table declares no partition for the
|
||||
selected slot or fallback; the bootloader then tries the other apps.
|
||||
`selected` is null when the dump lacks a record the bootloader reads. Its
|
||||
`assumptions` list what the derivation leaves out, among them factory-reset
|
||||
and test-app GPIO holds and a wake from deep sleep. `running_partition` is
|
||||
always `unknown`: at boot the bootloader may first mark a pending-verify
|
||||
record aborted, check the app's secure version against eFuse, and fall back
|
||||
to another partition when an app fails to load.
|
||||
|
||||
## Coverage and extraction
|
||||
|
||||
Erased flash (`0xFF`) between regions is described; any other byte outside
|
||||
the bootloader region, the table sector and the partitions is listed in
|
||||
`undescribed` and makes coverage `unknown`. Coverage is also `unknown` when a
|
||||
partition is truncated or outside the dump, or when no bootloader is
|
||||
located.
|
||||
Erased flash (`0xFF`) between regions is described. In a gap between
|
||||
regions, the first byte that is not erased starts an `undescribed` range to
|
||||
the end of the gap, and makes coverage `unknown`; the rest of that range is
|
||||
not examined. Coverage is also `unknown` when a partition is truncated or
|
||||
outside the dump, or when no bootloader is located.
|
||||
|
||||
`extract_artifact` writes every entry as an ordinary file, `nvs_keys`
|
||||
partitions included. On a device without flash encryption an `nvs_keys`
|
||||
@@ -139,31 +184,40 @@ NVS.
|
||||
|
||||
## Comparison
|
||||
|
||||
Artifact comparison uses each partition's `label`, `type`, `subtype`,
|
||||
`flash_address`, `size` and flags as its declaration, so a partition that
|
||||
moves or changes type with unchanged bytes is `changed` in `metadata`.
|
||||
Partitions pair by path. The root facts derive from the dump's bytes, which
|
||||
content comparison covers.
|
||||
Artifact comparison uses each partition's `label`, `label_hex`, `type`,
|
||||
`subtype`, `flash_address`, `size` and flags as its declaration, so a
|
||||
partition that moves, changes type or flags, or is renamed with unchanged
|
||||
bytes is `changed` in `metadata`. Partitions pair by path. The root facts
|
||||
are not compared as a declaration: they derive from the dump's bytes, which
|
||||
content comparison covers, and from the caller's `flash_offset` and
|
||||
`partition_table_offset`, which are part of the target's identity and
|
||||
decide each partition's byte range.
|
||||
|
||||
## Limits
|
||||
|
||||
- A flash dump inside another container, such as a ZIP, is classified from
|
||||
its first bytes and so reads as an ESP image, not as a flash layout; extract
|
||||
it first.
|
||||
- ESP8266 dumps, Secure Boot v1 bootloader digests and flash-encrypted
|
||||
layouts are not read.
|
||||
- ESP8266 dumps are refused; Secure Boot v1 bootloader digests and
|
||||
flash-encrypted layouts are not read.
|
||||
- Partitions are not checked against the flash chip size, which the
|
||||
bootloader also verifies.
|
||||
- NVS, FAT, SPIFFS and other data partitions are extracted, not decoded.
|
||||
- An image nested in a partition is decoded for its facts; its segments are
|
||||
not inventory entries.
|
||||
|
||||
## Verification
|
||||
|
||||
Synthetic fixtures with computed MD5s and CRCs cover the table rules, every
|
||||
way the table is located, the misplaced-origin guard, the ROM bootloader
|
||||
offsets, overlaps, truncation and partitions outside the dump, index names
|
||||
for unsafe labels, the otadata states and the bootloader's selection rules.
|
||||
CLI and stdio MCP tests exercise the inputs, their identity, extraction and
|
||||
comparison.
|
||||
Synthetic fixtures with computed MD5s and CRCs, and one CRC otatool wrote,
|
||||
cover the table rules, every way the table is located, the scan's chunk and
|
||||
sector edges, the misplaced-origin guard with the origin omitted or given,
|
||||
the ESP8266 refusal, the `0x0`, `0x1000` and `0x2000` ROM bootloader
|
||||
offsets, single images that are not scanned, overlaps, primary entries,
|
||||
truncation and partitions outside the dump, index names for unsafe labels,
|
||||
repeated declarations, the otadata states and records, and each branch of
|
||||
the bootloader's selection. CLI and stdio MCP tests exercise the inputs and
|
||||
their identity and extraction, and an MCP `compare_artifacts` test compares
|
||||
partition declarations, labels that are not UTF-8 included.
|
||||
|
||||
The optional real lane builds its dumps with esptool and ESP-IDF's own tools:
|
||||
|
||||
@@ -207,7 +261,10 @@ The lane does not establish:
|
||||
|
||||
- dumps read from a device: its dumps come from `merge-bin`;
|
||||
- chips other than ESP32-C3 and ESP32;
|
||||
- OTA states the bootloader writes itself, app rollback or anti-rollback;
|
||||
- `ota_state` and record validity, which `read_otadata` does not print, or
|
||||
OTA states the bootloader writes itself, app rollback or anti-rollback;
|
||||
- selection with gaps or repeated OTA subtypes, which otatool does not
|
||||
write;
|
||||
- flash-encrypted dumps, which are not recognized by design.
|
||||
|
||||
It does not run in CI.
|
||||
@@ -5,6 +5,7 @@ import { describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { EspFlashImageArtifactReader } from "./EspFlashImageArtifactReader.js";
|
||||
import { ArtifactResourceScope } from "./ArtifactResourceScope.js";
|
||||
import type { ArtifactEntry } from "./ArtifactReader.js";
|
||||
import { extractArtifact } from "./extraction/ArtifactExtraction.js";
|
||||
import * as espImageDecoding from "../domain/firmware/espImage.js";
|
||||
import {
|
||||
@@ -51,6 +52,12 @@ const root = (
|
||||
snapshot.occurrences.find(({ logical_path }) => logical_path === ".")
|
||||
?.format_metadata;
|
||||
|
||||
const list = async (reader: EspFlashImageArtifactReader) => {
|
||||
const entries: ArtifactEntry[] = [];
|
||||
for await (const entry of reader.entries()) entries.push(entry);
|
||||
return entries;
|
||||
};
|
||||
|
||||
describe("ESP flash layout inventory", () => {
|
||||
it("inventories the bootloader, table and partitions with app and otadata facts", async () => {
|
||||
const snapshot = await scanCanonicalArtifactInventory(
|
||||
@@ -219,6 +226,87 @@ describe("ESP flash layout coverage and selection", () => {
|
||||
} finally {
|
||||
await reader.close();
|
||||
}
|
||||
// The caller's selection decides before another container's signature.
|
||||
const zipLike = await writeDump(
|
||||
Buffer.concat([Buffer.from("PK\x03\x04"), pattern(0x1_0000, 3)]),
|
||||
"dump.zip",
|
||||
);
|
||||
await expect(
|
||||
scanCanonicalArtifactInventory(zipLike, { espFlashSelection: {} }),
|
||||
).rejects.toThrow(/ESP flash image layout is not recognized/u);
|
||||
});
|
||||
});
|
||||
|
||||
describe("ESP flash layout reader identity", () => {
|
||||
it("rejects a changed carrier during extraction preparation", async () => {
|
||||
const reader = new EspFlashImageArtifactReader(await writeDump(DUMP));
|
||||
try {
|
||||
await expect(
|
||||
reader.prepareContainer("0".repeat(64)),
|
||||
).rejects.toMatchObject({ reason: "integrity" });
|
||||
} finally {
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("rejects forged entries and changed carrier identity, then closes idempotently", async () => {
|
||||
const path = await writeDump(DUMP);
|
||||
const reader = new EspFlashImageArtifactReader(path);
|
||||
try {
|
||||
const entry = (await list(reader)).find(
|
||||
({ path: name }) => name === "partitions/nvs.bin",
|
||||
);
|
||||
if (entry === undefined) throw new Error("Missing nvs partition");
|
||||
await expect(
|
||||
reader.open({ ...entry, declaredSize: 1 }),
|
||||
).rejects.toMatchObject({ reason: "integrity" });
|
||||
await expect(
|
||||
reader.open({ ...entry, byteOffset: 0 }),
|
||||
).rejects.toMatchObject({ reason: "integrity" });
|
||||
await expect(
|
||||
reader.open({
|
||||
...entry,
|
||||
adapterKey: "esp-flash-image:partitions/x.bin",
|
||||
}),
|
||||
).rejects.toMatchObject({ reason: "integrity" });
|
||||
await writeFile(path, DUMP.subarray(0, 0x1_0000));
|
||||
await expect(reader.open(entry)).rejects.toMatchObject({
|
||||
reason: "integrity",
|
||||
});
|
||||
} finally {
|
||||
await reader.close();
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("settles an active entry stream when closed", async () => {
|
||||
const reader = new EspFlashImageArtifactReader(await writeDump(DUMP));
|
||||
try {
|
||||
const [entry] = await list(reader);
|
||||
if (entry === undefined) throw new Error("Missing entry");
|
||||
const stream = await reader.open(entry);
|
||||
await reader.close();
|
||||
expect(stream.destroyed).toBe(true);
|
||||
await expect(
|
||||
reader.entries()[Symbol.asyncIterator]().next(),
|
||||
).rejects.toMatchObject({ reason: "unavailable" });
|
||||
} finally {
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("honors cancellation before admitting a file with the exact reason", async () => {
|
||||
const reader = new EspFlashImageArtifactReader(await writeDump(DUMP));
|
||||
const controller = new AbortController();
|
||||
const reason = { stage: "before admission" };
|
||||
controller.abort(reason);
|
||||
try {
|
||||
await expect(
|
||||
reader.entries(controller.signal)[Symbol.asyncIterator]().next(),
|
||||
).rejects.toMatchObject({ reason: "cancelled", cause: reason });
|
||||
} finally {
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -265,15 +265,17 @@ export class EspFlashImageArtifactReader implements ArtifactReader {
|
||||
context: EspFlashReadContext,
|
||||
plan: EspFlashPlan,
|
||||
): Promise<EspFlashLayoutFacts> {
|
||||
const record = (position: number) =>
|
||||
readEspFlashRange(context, position, ESP_OTA_SELECT_ENTRY_BYTES);
|
||||
const otadataRecords =
|
||||
plan.otadata === undefined
|
||||
? undefined
|
||||
: ([
|
||||
await record(plan.otadata.records[0]),
|
||||
await record(plan.otadata.records[1]),
|
||||
] as const);
|
||||
const otadataRecords: (Uint8Array | undefined)[] = [];
|
||||
for (const { fileOffset, available } of plan.otadata?.records ?? [])
|
||||
otadataRecords.push(
|
||||
available
|
||||
? await readEspFlashRange(
|
||||
context,
|
||||
fileOffset,
|
||||
ESP_OTA_SELECT_ENTRY_BYTES,
|
||||
)
|
||||
: undefined,
|
||||
);
|
||||
const gapFirstUnerased: (number | undefined)[] = [];
|
||||
for (const gap of plan.gaps) {
|
||||
this.#assertOpen(context.signal);
|
||||
@@ -426,10 +428,13 @@ const layoutMetadata = (
|
||||
: {
|
||||
...facts.otadata,
|
||||
records: facts.otadata.records.map((record) => ({ ...record })),
|
||||
selected: {
|
||||
...facts.otadata.selected,
|
||||
assumptions: [...facts.otadata.selected.assumptions],
|
||||
},
|
||||
selected:
|
||||
facts.otadata.selected === null
|
||||
? null
|
||||
: {
|
||||
...facts.otadata.selected,
|
||||
assumptions: [...facts.otadata.selected.assumptions],
|
||||
},
|
||||
},
|
||||
undescribed: [...facts.undescribed],
|
||||
};
|
||||
|
||||
@@ -99,8 +99,9 @@ export const decodeEspFlashImage = async (
|
||||
region.start,
|
||||
PROBE_BYTES,
|
||||
);
|
||||
// A region shorter than a header is erased when all of its bytes are.
|
||||
if (
|
||||
prefix.length >= HEADER_BYTES &&
|
||||
prefix.length > 0 &&
|
||||
prefix.subarray(0, HEADER_BYTES).every((byte) => byte === 0xff)
|
||||
)
|
||||
return noImage("erased", []);
|
||||
|
||||
@@ -242,17 +242,6 @@ const classifyRootFormat = async (
|
||||
const archiveBytes = (await handle.stat()).size;
|
||||
const evidence = probe.finish(archiveBytes);
|
||||
abortIfNeeded(signal);
|
||||
const classificationPath = artifactNameForClassification(
|
||||
selectedPath ?? path,
|
||||
path,
|
||||
);
|
||||
const lower = classificationPath.toLowerCase();
|
||||
if (hasZipSignature(evidence.prefix)) return zipRootPackageFormat(lower);
|
||||
if (
|
||||
lower.endsWith(".pkg") &&
|
||||
evidence.prefix.subarray(0, 4).equals(Buffer.from("xar!"))
|
||||
)
|
||||
return "pkg";
|
||||
// Fill each range: a short result is end of file, not a partial host read.
|
||||
const read = async (position: number, length: number): Promise<Buffer> => {
|
||||
const bytes = Buffer.alloc(length);
|
||||
@@ -271,6 +260,32 @@ const classifyRootFormat = async (
|
||||
}
|
||||
return bytes.subarray(0, filled);
|
||||
};
|
||||
const locateFlash = (selection: EspFlashSelection, explicit: boolean) =>
|
||||
locateEspFlashLayout(archiveBytes, read, selection, {
|
||||
explicit,
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
});
|
||||
// The caller's format selection decides before any other signature.
|
||||
if (espFlashSelection !== undefined) {
|
||||
const selected = await locateFlash(espFlashSelection, true);
|
||||
if (!selected.ok)
|
||||
throw new ArtifactReaderFailure(
|
||||
"unavailable",
|
||||
`ESP flash image layout is not recognized: ${selected.error}`,
|
||||
);
|
||||
return { kind: "esp-flash-image", source, location: selected.value };
|
||||
}
|
||||
const classificationPath = artifactNameForClassification(
|
||||
selectedPath ?? path,
|
||||
path,
|
||||
);
|
||||
const lower = classificationPath.toLowerCase();
|
||||
if (hasZipSignature(evidence.prefix)) return zipRootPackageFormat(lower);
|
||||
if (
|
||||
lower.endsWith(".pkg") &&
|
||||
evidence.prefix.subarray(0, 4).equals(Buffer.from("xar!"))
|
||||
)
|
||||
return "pkg";
|
||||
if (
|
||||
lower.endsWith(".dmg") &&
|
||||
(await hasDiskImageSignature(archiveBytes, read))
|
||||
@@ -280,22 +295,9 @@ const classifyRootFormat = async (
|
||||
if (pck !== undefined) return { kind: "godot-pck", source, location: pck };
|
||||
// A C3 or S3 dump begins with its bootloader, so the layout is located
|
||||
// before the bytes are read as one ESP image.
|
||||
const flash = await locateEspFlashLayout(
|
||||
archiveBytes,
|
||||
read,
|
||||
espFlashSelection ?? {},
|
||||
{
|
||||
explicit: espFlashSelection !== undefined,
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
},
|
||||
);
|
||||
const flash = await locateFlash({}, false);
|
||||
if (flash.ok)
|
||||
return { kind: "esp-flash-image", source, location: flash.value };
|
||||
if (espFlashSelection !== undefined)
|
||||
throw new ArtifactReaderFailure(
|
||||
"unavailable",
|
||||
`ESP flash image layout is not recognized: ${flash.error}`,
|
||||
);
|
||||
const fatEvidence = await readRootFatSliceHeaders(handle, evidence, signal);
|
||||
const mzWindowsHeaderSignature = await readRootMzWindowsHeaderSignature(
|
||||
handle,
|
||||
|
||||
+5
-1
@@ -33,8 +33,12 @@ export const snapshotSelectionOption = z
|
||||
.optional()
|
||||
.describe("Load and update a local analysis snapshot");
|
||||
|
||||
/** Explicit interpretation for headerless native executable formats. */
|
||||
/**
|
||||
* Explicit interpretation for headerless native executable formats. Flash
|
||||
* dumps take their offsets on the artifact commands instead.
|
||||
*/
|
||||
export const formatSelectionOption = executableFormatHintSchema
|
||||
.exclude(["esp-flash-image"])
|
||||
.optional()
|
||||
.describe(
|
||||
"Explicit interpretation: dos-com or raw-binary (headerless), or gamecube-dol (GameCube/Wii DOL, never auto-detected); omission preserves header-based detection",
|
||||
|
||||
@@ -274,6 +274,11 @@ export const espFlashPartitionMetadataSchema = z.strictObject({
|
||||
index: uint8,
|
||||
/** The label up to its first NUL; null when it is not UTF-8. */
|
||||
label: z.string().nullable(),
|
||||
/** The label's bytes up to its first NUL in hex, when it is not UTF-8. */
|
||||
label_hex: z
|
||||
.string()
|
||||
.regex(/^(?:[0-9a-f]{2}){1,16}$/u)
|
||||
.nullable(),
|
||||
type: uint8,
|
||||
subtype: uint8,
|
||||
/** `gen_esp32part.py` names; null for values it does not name. */
|
||||
@@ -295,6 +300,7 @@ export const espFlashPartitionDeclaration = (
|
||||
metadata: EspFlashPartitionMetadata,
|
||||
) => ({
|
||||
label: metadata.label,
|
||||
label_hex: metadata.label_hex,
|
||||
type: metadata.type,
|
||||
subtype: metadata.subtype,
|
||||
flash_address: metadata.flash_address,
|
||||
@@ -374,28 +380,40 @@ export const espFlashImageMetadataSchema = z.strictObject({
|
||||
apps: z.array(
|
||||
z.strictObject({ partition_index: uint8, facts: espFlashImageFactsSchema }),
|
||||
),
|
||||
/** The otadata partition the bootloader uses: the last one declared. */
|
||||
otadata: z
|
||||
.strictObject({
|
||||
partition_index: uint8,
|
||||
records: z.array(espOtaRecordSchema),
|
||||
/** Derived from the records under the stated assumptions. */
|
||||
selected: z.strictObject({
|
||||
active_record: z.union([z.literal(0), z.literal(1)]).nullable(),
|
||||
slot: uint8.nullable(),
|
||||
partition_index: uint8.nullable(),
|
||||
basis: z.enum([
|
||||
"otadata",
|
||||
"factory-fallback",
|
||||
"ota_0-fallback",
|
||||
"none",
|
||||
]),
|
||||
assumptions: z.array(z.string()),
|
||||
}),
|
||||
/** The records the dump holds, at most one per sector. */
|
||||
records: z.array(espOtaRecordSchema).max(2),
|
||||
/**
|
||||
* Derived from both records under the stated assumptions; null when
|
||||
* the dump lacks one. `partition_index` is null when the table
|
||||
* declares no partition for the selected slot or fallback, and `none`
|
||||
* is the bootloader's INVALID_INDEX, after which it resets.
|
||||
*/
|
||||
selected: z
|
||||
.strictObject({
|
||||
active_record: z.union([z.literal(0), z.literal(1)]).nullable(),
|
||||
slot: uint8.nullable(),
|
||||
partition_index: uint8.nullable(),
|
||||
basis: z.enum([
|
||||
"otadata",
|
||||
"factory-fallback",
|
||||
"ota_0-fallback",
|
||||
"none",
|
||||
]),
|
||||
assumptions: z.array(z.string()),
|
||||
})
|
||||
.nullable(),
|
||||
/** Rollback, anti-rollback and load fallback decide it at boot. */
|
||||
running_partition: z.literal("unknown"),
|
||||
})
|
||||
.nullable(),
|
||||
/** Bytes outside every described region that are not erased flash. */
|
||||
/**
|
||||
* Ranges outside every described region, each from a byte that is not
|
||||
* erased flash to the end of its gap.
|
||||
*/
|
||||
undescribed: z.array(byteRange),
|
||||
});
|
||||
export type EspFlashImageMetadata = z.infer<typeof espFlashImageMetadataSchema>;
|
||||
|
||||
@@ -3,8 +3,13 @@ import {
|
||||
espChipForId,
|
||||
type EspChip,
|
||||
} from "./espChips.js";
|
||||
import { recognizeEspImage } from "./espImage.js";
|
||||
import { offsetText, u16 } from "./espImageStructures.js";
|
||||
import { ESP_IMAGE_HEADER_BYTES, recognizeEspImage } from "./espImage.js";
|
||||
import {
|
||||
ESP_IMAGE_SEGMENT_HEADER_BYTES,
|
||||
offsetText,
|
||||
u16,
|
||||
u32,
|
||||
} from "./espImageStructures.js";
|
||||
|
||||
/** Bytes that recognition needs at a candidate bootloader offset. */
|
||||
export const ESP_BOOTLOADER_PROBE_BYTES = 32;
|
||||
@@ -27,22 +32,31 @@ export interface EspBootloaderLocation {
|
||||
readonly chip: EspChip;
|
||||
}
|
||||
|
||||
export interface EspBootloaderSearch {
|
||||
readonly location: EspBootloaderLocation | undefined;
|
||||
readonly limitations: readonly string[];
|
||||
/** A known chip's image sits at another chip's ROM offset. */
|
||||
readonly misplaced: boolean;
|
||||
/** An ESP8266 image sits at a ROM offset. */
|
||||
readonly esp8266: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* The bootloader is the ESP image at the flash address its chip's ROM loads
|
||||
* from (`ESP_ROM_BOOTLOADER_OFFSET_FLASH`). An image at another chip's ROM
|
||||
* offset is reported, not located: the dump may not start where its flash
|
||||
* offset says.
|
||||
* from (`ESP_ROM_BOOTLOADER_OFFSET_FLASH`). Without one, an image of a known
|
||||
* chip at another chip's ROM offset says the dump may not start where its
|
||||
* flash offset says, and an ESP8266 image says it is an ESP8266 dump.
|
||||
*/
|
||||
export const locateEspBootloader = (
|
||||
probes: readonly EspBootloaderProbe[],
|
||||
): {
|
||||
readonly location: EspBootloaderLocation | undefined;
|
||||
readonly limitations: readonly string[];
|
||||
} => {
|
||||
): EspBootloaderSearch => {
|
||||
const limitations: string[] = [];
|
||||
let misplaced = false;
|
||||
let esp8266 = false;
|
||||
for (const probe of probes) {
|
||||
if (recognizeEspImage(probe.prefix, probe.available) !== "esp-image")
|
||||
continue;
|
||||
const recognized = recognizeEspImage(probe.prefix, probe.available);
|
||||
if (recognized === "esp8266") esp8266 = true;
|
||||
if (recognized !== "esp-image") continue;
|
||||
const chipId = u16(probe.prefix, 12);
|
||||
const chip = espChipForId(chipId);
|
||||
if (chip?.romBootloaderOffset === probe.flashAddress)
|
||||
@@ -53,14 +67,39 @@ export const locateEspBootloader = (
|
||||
chip,
|
||||
},
|
||||
limitations: [],
|
||||
misplaced: false,
|
||||
esp8266: false,
|
||||
};
|
||||
if (chip !== undefined) misplaced = true;
|
||||
limitations.push(
|
||||
chip === undefined
|
||||
? `An ESP image at flash ${offsetText(probe.flashAddress)} has chip_id 0x${chipId.toString(16).padStart(4, "0")}, which is not in the chip table; it is not read as the bootloader.`
|
||||
: `An ${chip.target} image is at flash ${offsetText(probe.flashAddress)}, but its ROM loads the bootloader from ${offsetText(chip.romBootloaderOffset)}; the dump may not start at the given flash offset.`,
|
||||
: `An ${chip.target} image is at flash ${offsetText(probe.flashAddress)}, but its ROM loads the bootloader from ${offsetText(chip.romBootloaderOffset)}; the dump may not start at flash ${offsetText(probe.flashAddress - probe.fileOffset)}.`,
|
||||
);
|
||||
}
|
||||
return { location: undefined, limitations };
|
||||
return { location: undefined, limitations, misplaced, esp8266 };
|
||||
};
|
||||
|
||||
/**
|
||||
* The file offset just past the last segment of the image at `fileOffset`,
|
||||
* found by following its segment headers, or the file size when they run
|
||||
* past the end. No partition table lies before it.
|
||||
*/
|
||||
export const espImageSegmentsEnd = async (
|
||||
read: (position: number, length: number) => Promise<Uint8Array>,
|
||||
fileOffset: number,
|
||||
fileSize: number,
|
||||
): Promise<number> => {
|
||||
const header = await read(fileOffset, ESP_IMAGE_HEADER_BYTES);
|
||||
if (header.length < ESP_IMAGE_HEADER_BYTES) return fileSize;
|
||||
let cursor = fileOffset + ESP_IMAGE_HEADER_BYTES;
|
||||
for (let index = 0; index < (header[1] ?? 0); index += 1) {
|
||||
const segment = await read(cursor, ESP_IMAGE_SEGMENT_HEADER_BYTES);
|
||||
if (segment.length < ESP_IMAGE_SEGMENT_HEADER_BYTES) return fileSize;
|
||||
cursor += ESP_IMAGE_SEGMENT_HEADER_BYTES + u32(segment, 4);
|
||||
if (cursor >= fileSize) return fileSize;
|
||||
}
|
||||
return cursor;
|
||||
};
|
||||
|
||||
/** The file offsets that hold each ROM bootloader offset, given the dump's origin. */
|
||||
|
||||
@@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
decodeEspPartitionTable,
|
||||
espBootPartitions,
|
||||
espPartitionSubtypeName,
|
||||
espPartitionTypeName,
|
||||
type EspPartition,
|
||||
@@ -9,6 +10,7 @@ import {
|
||||
import { decodeEspOtaRecord, selectEspOtaSlot } from "./espOtaData.js";
|
||||
import {
|
||||
espBootloaderProbeOffsets,
|
||||
espImageSegmentsEnd,
|
||||
locateEspBootloader,
|
||||
} from "./espFlashBootloader.js";
|
||||
import {
|
||||
@@ -17,7 +19,11 @@ import {
|
||||
partitionEntry,
|
||||
partitionTable,
|
||||
} from "./espFlash.fixture.js";
|
||||
import { ESP32C3_BOOTLOADER_SEGMENTS, espImage } from "./espImage.fixture.js";
|
||||
import {
|
||||
ESP32C3_BOOTLOADER_SEGMENTS,
|
||||
esp8266Image,
|
||||
espImage,
|
||||
} from "./espImage.fixture.js";
|
||||
|
||||
const table = (bytes: Uint8Array) => {
|
||||
const decoded = decodeEspPartitionTable(bytes);
|
||||
@@ -142,10 +148,37 @@ describe("ESP partition tables", () => {
|
||||
).partitions;
|
||||
expect(decoded).toMatchObject({
|
||||
label: null,
|
||||
// The source bytes stay comparable when the label is not UTF-8.
|
||||
labelHex: "78ff",
|
||||
encrypted: true,
|
||||
unknownFlags: 0x8000_0000,
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps the partitions the bootloader keeps: the last of each role", () => {
|
||||
const [nvs, otadata, , factory, ota0] = TWO_OTA_PARTITIONS;
|
||||
const boot = espBootPartitions(
|
||||
table(
|
||||
partitionTable([
|
||||
...TWO_OTA_PARTITIONS,
|
||||
{ ...factory!, label: "factory2", offset: 0x4_0000 },
|
||||
{ ...otadata!, label: "otadata2", offset: 0x5_0000 },
|
||||
{ ...ota0!, label: "again", offset: 0x6_0000 },
|
||||
{ ...nvs!, label: "nvs2", offset: 0x7_0000 },
|
||||
]),
|
||||
).partitions,
|
||||
);
|
||||
expect(boot.factory?.label).toBe("factory2");
|
||||
expect(boot.otadata?.label).toBe("otadata2");
|
||||
// A same-subtype OTA entry replaces the slot but still counts.
|
||||
expect(boot.otaSlots.get(0)?.label).toBe("again");
|
||||
expect(boot.otaAppCount).toBe(3);
|
||||
expect(boot.limitations).toEqual([
|
||||
"The table declares otadata 2 times (partitions #1, #7); ESP-IDF's bootloader uses the last.",
|
||||
"The table declares factory 2 times (partitions #3, #6); ESP-IDF's bootloader uses the last.",
|
||||
"The table declares ota_0 2 times (partitions #4, #8); ESP-IDF's bootloader uses the last.",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
const partitions = (): readonly EspPartition[] =>
|
||||
@@ -167,6 +200,10 @@ describe("ESP otadata selection", () => {
|
||||
crc_matches: false,
|
||||
valid: false,
|
||||
});
|
||||
// The CRC otatool v5.5.2 wrote for ota_seq 1 in the real-tool lane.
|
||||
expect(
|
||||
decodeEspOtaRecord(otaRecord({ seq: 1, crc: 0x4743_989a }), 0),
|
||||
).toMatchObject({ crc_matches: true, valid: true });
|
||||
expect(decodeEspOtaRecord(Buffer.alloc(32, 0xff), 1)).toMatchObject({
|
||||
ota_seq: 0xffff_ffff,
|
||||
state_name: "undefined",
|
||||
@@ -174,10 +211,16 @@ describe("ESP otadata selection", () => {
|
||||
});
|
||||
});
|
||||
|
||||
const select = (first: Buffer, second: Buffer, parts = partitions()) =>
|
||||
const select = (
|
||||
first: Buffer,
|
||||
second: Buffer,
|
||||
parts = partitions(),
|
||||
otadataSize = 0x2000,
|
||||
) =>
|
||||
selectEspOtaSlot(
|
||||
[decodeEspOtaRecord(first, 0), decodeEspOtaRecord(second, 1)],
|
||||
parts,
|
||||
espBootPartitions(parts),
|
||||
otadataSize,
|
||||
);
|
||||
const erased = Buffer.alloc(32, 0xff);
|
||||
|
||||
@@ -197,6 +240,12 @@ describe("ESP otadata selection", () => {
|
||||
slot: 1,
|
||||
partition_index: 5,
|
||||
});
|
||||
// ota_seq 0 is set; (0 - 1) wraps to 0xFFFFFFFF, and 0xFFFFFFFF % 2 is 1.
|
||||
expect(select(otaRecord({ seq: 0 }), erased)).toMatchObject({
|
||||
active_record: 0,
|
||||
slot: 1,
|
||||
partition_index: 5,
|
||||
});
|
||||
});
|
||||
|
||||
it("falls back as the bootloader does without usable records", () => {
|
||||
@@ -211,11 +260,37 @@ describe("ESP otadata selection", () => {
|
||||
slot: 0,
|
||||
partition_index: 4,
|
||||
});
|
||||
// A set sequence with a bad CRC is not unusable, but no record is valid.
|
||||
// Boot index 0 stands without an ota_0 partition.
|
||||
const onlyOta1 = noFactory.filter(({ label }) => label !== "ota_0");
|
||||
expect(select(erased, erased, onlyOta1)).toMatchObject({
|
||||
basis: "ota_0-fallback",
|
||||
slot: 0,
|
||||
partition_index: null,
|
||||
});
|
||||
// A set sequence with a bad CRC is not unusable, but no record is valid:
|
||||
// the factory index, with no factory partition to name.
|
||||
expect(
|
||||
select(otaRecord({ seq: 2, crc: 1 }), erased, noFactory),
|
||||
).toMatchObject({ basis: "none", partition_index: null });
|
||||
expect(select(otaRecord({ seq: 2 }), erased).assumptions).toHaveLength(3);
|
||||
).toMatchObject({ basis: "factory-fallback", partition_index: null });
|
||||
expect(select(otaRecord({ seq: 2 }), erased)?.assumptions).toHaveLength(4);
|
||||
});
|
||||
|
||||
it("selects nothing from a short otadata partition and nothing known from one record", () => {
|
||||
expect(
|
||||
select(
|
||||
otaRecord({ seq: 1 }),
|
||||
otaRecord({ seq: 2 }),
|
||||
partitions(),
|
||||
0x1000,
|
||||
),
|
||||
).toMatchObject({ basis: "none", slot: null, partition_index: null });
|
||||
expect(
|
||||
selectEspOtaSlot(
|
||||
[decodeEspOtaRecord(otaRecord({ seq: 1 }), 0)],
|
||||
espBootPartitions(partitions()),
|
||||
0x2000,
|
||||
),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("indexes OTA slots by subtype, so a missing subtype selects no partition", () => {
|
||||
@@ -260,11 +335,45 @@ describe("ESP bootloader location", () => {
|
||||
expect(locateEspBootloader([probe(0, esp32)])).toEqual({
|
||||
location: undefined,
|
||||
limitations: [
|
||||
"An esp32 image is at flash 0x0, but its ROM loads the bootloader from 0x1000; the dump may not start at the given flash offset.",
|
||||
"An esp32 image is at flash 0x0, but its ROM loads the bootloader from 0x1000; the dump may not start at flash 0x0.",
|
||||
],
|
||||
misplaced: true,
|
||||
esp8266: false,
|
||||
});
|
||||
});
|
||||
|
||||
it("tells an unknown chip and an ESP8266 image from a misplaced origin", () => {
|
||||
const unknown = espImage({
|
||||
segments: ESP32C3_BOOTLOADER_SEGMENTS,
|
||||
chipId: 0x7777,
|
||||
});
|
||||
expect(locateEspBootloader([probe(0, unknown)])).toMatchObject({
|
||||
location: undefined,
|
||||
misplaced: false,
|
||||
esp8266: false,
|
||||
});
|
||||
expect(locateEspBootloader([probe(0, esp8266Image())])).toMatchObject({
|
||||
location: undefined,
|
||||
limitations: [],
|
||||
misplaced: false,
|
||||
esp8266: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("finds where a located image's segments end by their headers", async () => {
|
||||
const read = async (position: number, length: number) =>
|
||||
bootloader.subarray(position, position + length);
|
||||
const declared = ESP32C3_BOOTLOADER_SEGMENTS.reduce(
|
||||
(end, { data }) => end + 8 + data.length,
|
||||
24,
|
||||
);
|
||||
expect(await espImageSegmentsEnd(read, 0, bootloader.length)).toBe(
|
||||
declared,
|
||||
);
|
||||
// Segments that run past the file end at the file.
|
||||
expect(await espImageSegmentsEnd(read, 0, 40)).toBe(40);
|
||||
});
|
||||
|
||||
it("probes only the ROM offsets the file holds after its flash offset", () => {
|
||||
expect(espBootloaderProbeOffsets(0, 0x10_0000)).toEqual([
|
||||
{ flashAddress: 0, fileOffset: 0 },
|
||||
|
||||
@@ -2,14 +2,18 @@ import { offsetText } from "./espImageStructures.js";
|
||||
import type { EspBootloaderLocation } from "./espFlashBootloader.js";
|
||||
import {
|
||||
ESP_PARTITION_TABLE_SECTOR_BYTES,
|
||||
espBootPartitions,
|
||||
espPartitionSubtypeName,
|
||||
espPartitionTypeName,
|
||||
isEspOtaData,
|
||||
espPrimaryLayoutRole,
|
||||
type EspBootPartitions,
|
||||
type EspPartition,
|
||||
type EspPartitionTable,
|
||||
} from "./espPartitionTable.js";
|
||||
import {
|
||||
ESP_OTA_DATA_MIN_BYTES,
|
||||
ESP_OTA_DATA_SECTOR_BYTES,
|
||||
ESP_OTA_SELECT_ENTRY_BYTES,
|
||||
decodeEspOtaRecord,
|
||||
selectEspOtaSlot,
|
||||
type EspOtaRecord,
|
||||
@@ -46,6 +50,13 @@ export interface EspFlashPlanInput {
|
||||
readonly table: EspPartitionTable;
|
||||
}
|
||||
|
||||
/** One otadata record within its partition, and whether the file holds it. */
|
||||
export interface EspOtaRecordPlan {
|
||||
readonly index: 0 | 1;
|
||||
readonly fileOffset: number;
|
||||
readonly available: boolean;
|
||||
}
|
||||
|
||||
/** Regions to inventory and the byte ranges the adapter must still read. */
|
||||
export interface EspFlashPlan {
|
||||
readonly input: EspFlashPlanInput;
|
||||
@@ -55,11 +66,13 @@ export interface EspFlashPlan {
|
||||
readonly offset: number;
|
||||
readonly length: number;
|
||||
}[];
|
||||
/** File offsets of the two otadata records, when the file holds them. */
|
||||
/** The partitions the bootloader keeps from the table. */
|
||||
readonly boot: EspBootPartitions;
|
||||
/** The otadata partition the bootloader uses and its records. */
|
||||
readonly otadata:
|
||||
| {
|
||||
readonly partition: EspPartition;
|
||||
readonly records: readonly [number, number];
|
||||
readonly records: readonly EspOtaRecordPlan[];
|
||||
}
|
||||
| undefined;
|
||||
readonly limitations: readonly string[];
|
||||
@@ -68,21 +81,26 @@ export interface EspFlashPlan {
|
||||
const SAFE_LABEL = /^(?=.*[A-Za-z_])[A-Za-z0-9_][A-Za-z0-9._-]{0,15}$/u;
|
||||
|
||||
/**
|
||||
* Partition paths use the label when it is a safe, unique path segment
|
||||
* with a letter or underscore, else the table index, which then cannot
|
||||
* collide with a label.
|
||||
* Partition paths use the label when it is a safe path segment with a
|
||||
* letter or underscore that no other label repeats, ignoring case as some
|
||||
* file systems do; else the table index, which cannot collide with a label.
|
||||
*/
|
||||
const partitionPaths = (
|
||||
partitions: readonly EspPartition[],
|
||||
): ReadonlyMap<number, string> => {
|
||||
const counts = new Map<string, number>();
|
||||
for (const { label } of partitions)
|
||||
if (label !== null) counts.set(label, (counts.get(label) ?? 0) + 1);
|
||||
if (label !== null) {
|
||||
const folded = label.toLowerCase();
|
||||
counts.set(folded, (counts.get(folded) ?? 0) + 1);
|
||||
}
|
||||
return new Map(
|
||||
partitions.map((part) => {
|
||||
const label = part.label;
|
||||
const named =
|
||||
label !== null && SAFE_LABEL.test(label) && counts.get(label) === 1;
|
||||
label !== null &&
|
||||
SAFE_LABEL.test(label) &&
|
||||
counts.get(label.toLowerCase()) === 1;
|
||||
return [
|
||||
part.index,
|
||||
`partitions/${named ? label : String(part.index)}.bin`,
|
||||
@@ -104,6 +122,39 @@ const region = (
|
||||
const rangeText = (address: number, length: number): string =>
|
||||
`${offsetText(address)}-${offsetText(address + length)}`;
|
||||
|
||||
const partitionName = (part: EspPartition): string =>
|
||||
part.label ?? `#${String(part.index)}`;
|
||||
|
||||
/** The records the otadata partition holds, each where the file holds it. */
|
||||
const otadataPlan = (
|
||||
input: EspFlashPlanInput,
|
||||
partition: EspPartition,
|
||||
limitations: string[],
|
||||
): NonNullable<EspFlashPlan["otadata"]> => {
|
||||
const name = partitionName(partition);
|
||||
const small = partition.size < ESP_OTA_DATA_MIN_BYTES;
|
||||
if (small)
|
||||
limitations.push(
|
||||
`The otadata partition ${name} is ${offsetText(partition.size)} bytes, smaller than the two 4 KiB sectors ESP-IDF's bootloader reads; the bootloader resets without selecting an app.`,
|
||||
);
|
||||
const records: EspOtaRecordPlan[] = [];
|
||||
for (const index of [0, 1] as const) {
|
||||
const within = index * ESP_OTA_DATA_SECTOR_BYTES;
|
||||
// A record past the declared size belongs to whatever follows.
|
||||
if (within + ESP_OTA_SELECT_ENTRY_BYTES > partition.size) continue;
|
||||
const fileOffset = partition.offset + within - input.origin.value;
|
||||
const available =
|
||||
fileOffset >= 0 &&
|
||||
fileOffset + ESP_OTA_SELECT_ENTRY_BYTES <= input.fileSize;
|
||||
if (!available)
|
||||
limitations.push(
|
||||
`otadata record ${String(index)} of partition ${name} at flash ${offsetText(partition.offset + within)} is outside the dump${small ? "." : "; the OTA selection is unknown."}`,
|
||||
);
|
||||
records.push({ index, fileOffset, available });
|
||||
}
|
||||
return { partition, records };
|
||||
};
|
||||
|
||||
/** Lay the bootloader, table and partitions over the dump. */
|
||||
export const planEspFlashLayout = (input: EspFlashPlanInput): EspFlashPlan => {
|
||||
const limitations: string[] = [...input.table.limitations];
|
||||
@@ -145,8 +196,8 @@ export const planEspFlashLayout = (input: EspFlashPlanInput): EspFlashPlan => {
|
||||
length: part.size,
|
||||
partition: part,
|
||||
});
|
||||
const name = part.label ?? `#${String(part.index)}`;
|
||||
if (part.offset < tableEnd)
|
||||
const name = partitionName(part);
|
||||
if (part.offset < tableEnd && espPrimaryLayoutRole(part) === undefined)
|
||||
limitations.push(
|
||||
`Partition ${name} at ${offsetText(part.offset)} starts before the end of the partition table at ${offsetText(tableEnd)}; gen_esp32part.py rejects such a table.`,
|
||||
);
|
||||
@@ -165,54 +216,59 @@ export const planEspFlashLayout = (input: EspFlashPlanInput): EspFlashPlan => {
|
||||
if (placed.fileOffset >= 0 && placed.available > 0) regions.push(placed);
|
||||
}
|
||||
limitations.push(...overlaps(regions));
|
||||
const otadataPartition = input.table.partitions.find(isEspOtaData);
|
||||
const otadataStart =
|
||||
otadataPartition === undefined
|
||||
? undefined
|
||||
: otadataPartition.offset - input.origin.value;
|
||||
const otadata =
|
||||
otadataPartition === undefined || otadataStart === undefined
|
||||
? undefined
|
||||
: otadataStart >= 0 &&
|
||||
otadataStart + ESP_OTA_DATA_SECTOR_BYTES + 32 <= input.fileSize
|
||||
? {
|
||||
partition: otadataPartition,
|
||||
records: [
|
||||
otadataStart,
|
||||
otadataStart + ESP_OTA_DATA_SECTOR_BYTES,
|
||||
] as const,
|
||||
}
|
||||
: undefined;
|
||||
if (otadataPartition !== undefined && otadata === undefined)
|
||||
limitations.push(
|
||||
`The dump does not hold both otadata records of partition ${otadataPartition.label ?? `#${String(otadataPartition.index)}`}; the OTA selection is unknown.`,
|
||||
);
|
||||
const boot = espBootPartitions(input.table.partitions);
|
||||
limitations.push(...boot.limitations);
|
||||
return {
|
||||
input,
|
||||
regions,
|
||||
gaps: gapsBetween(regions, input.fileSize),
|
||||
otadata,
|
||||
boot,
|
||||
otadata:
|
||||
boot.otadata === undefined
|
||||
? undefined
|
||||
: otadataPlan(input, boot.otadata, limitations),
|
||||
limitations,
|
||||
};
|
||||
};
|
||||
|
||||
const regionEnd = ({ flashAddress, length }: EspFlashRegion): number =>
|
||||
flashAddress + length;
|
||||
|
||||
/** A primary bootloader or table entry declared over exactly its region. */
|
||||
const declaresRegion = (
|
||||
first: EspFlashRegion,
|
||||
second: EspFlashRegion,
|
||||
): boolean => {
|
||||
const [region, entry] =
|
||||
first.partition === undefined ? [first, second] : [second, first];
|
||||
return (
|
||||
region.partition === undefined &&
|
||||
entry.partition !== undefined &&
|
||||
espPrimaryLayoutRole(entry.partition) === region.role &&
|
||||
region.flashAddress === entry.flashAddress &&
|
||||
region.length === entry.length
|
||||
);
|
||||
};
|
||||
|
||||
/** Regions that share bytes, which gen_esp32part.py rejects for partitions. */
|
||||
const overlaps = (regions: readonly EspFlashRegion[]): string[] => {
|
||||
const ordered = [...regions].sort(
|
||||
(left, right) => left.flashAddress - right.flashAddress,
|
||||
);
|
||||
const found: string[] = [];
|
||||
for (let index = 1; index < ordered.length; index += 1) {
|
||||
const previous = ordered[index - 1];
|
||||
const current = ordered[index];
|
||||
// The region reaching furthest so far, so a long one shows every overlap.
|
||||
let furthest: EspFlashRegion | undefined;
|
||||
for (const current of ordered) {
|
||||
if (
|
||||
previous !== undefined &&
|
||||
current !== undefined &&
|
||||
current.flashAddress < previous.flashAddress + previous.length
|
||||
furthest !== undefined &&
|
||||
current.flashAddress < regionEnd(furthest) &&
|
||||
!declaresRegion(furthest, current)
|
||||
)
|
||||
found.push(
|
||||
`${current.path} at ${offsetText(current.flashAddress)} overlaps ${previous.path}, which ends at ${offsetText(previous.flashAddress + previous.length)}.`,
|
||||
`${current.path} at ${offsetText(current.flashAddress)} overlaps ${furthest.path}, which ends at ${offsetText(regionEnd(furthest))}.`,
|
||||
);
|
||||
if (furthest === undefined || regionEnd(current) > regionEnd(furthest))
|
||||
furthest = current;
|
||||
}
|
||||
return found;
|
||||
};
|
||||
@@ -242,16 +298,18 @@ const gapsBetween = (
|
||||
|
||||
export interface EspFlashOtaFacts {
|
||||
readonly partition_index: number;
|
||||
/** The records the dump holds, each with its index. */
|
||||
readonly records: readonly EspOtaRecord[];
|
||||
/** Derived from the records; see its assumptions. */
|
||||
readonly selected: EspOtaSelection;
|
||||
/** Derived from the records; see its assumptions. Null when unknown. */
|
||||
readonly selected: EspOtaSelection | null;
|
||||
/** The bootloader's rollback and fallback decide it at boot. */
|
||||
readonly running_partition: "unknown";
|
||||
}
|
||||
|
||||
/** Facts the adapter read for the plan: the otadata records and the gaps. */
|
||||
export interface EspFlashReads {
|
||||
readonly otadataRecords: readonly [Uint8Array, Uint8Array] | undefined;
|
||||
/** The bytes of each planned otadata record the file holds, in plan order. */
|
||||
readonly otadataRecords: readonly (Uint8Array | undefined)[];
|
||||
/** The first non-erased file offset in each gap, in plan order. */
|
||||
readonly gapFirstUnerased: readonly (number | undefined)[];
|
||||
}
|
||||
@@ -281,12 +339,12 @@ export const finishEspFlashLayout = (
|
||||
const total = undescribed.reduce((sum, { length }) => sum + length, 0);
|
||||
if (undescribed.length > 0)
|
||||
limitations.push(
|
||||
`${String(total)} bytes of the dump, in ${String(undescribed.length)} range(s), lie outside the bootloader region, the partition table and every partition and are not erased flash.`,
|
||||
`${String(undescribed.length)} range(s) of the dump, ${String(total)} bytes in all, lie outside the bootloader region, the partition table and every partition and are not described; each runs from a byte that is not erased flash to the end of its gap.`,
|
||||
);
|
||||
const otadata =
|
||||
plan.otadata === undefined || reads.otadataRecords === undefined
|
||||
plan.otadata === undefined
|
||||
? null
|
||||
: otaFacts(plan, reads.otadataRecords);
|
||||
: otaFacts(plan, plan.otadata, reads.otadataRecords);
|
||||
const incomplete = plan.input.table.partitions.some((part) => {
|
||||
const region = plan.regions.find((item) => item.partition === part);
|
||||
return region === undefined || region.available < part.size;
|
||||
@@ -306,14 +364,21 @@ export const finishEspFlashLayout = (
|
||||
|
||||
const otaFacts = (
|
||||
plan: EspFlashPlan,
|
||||
[first, second]: readonly [Uint8Array, Uint8Array],
|
||||
): EspFlashOtaFacts | null => {
|
||||
if (plan.otadata === undefined) return null;
|
||||
const records = [decodeEspOtaRecord(first, 0), decodeEspOtaRecord(second, 1)];
|
||||
otadata: NonNullable<EspFlashPlan["otadata"]>,
|
||||
bytes: readonly (Uint8Array | undefined)[],
|
||||
): EspFlashOtaFacts => {
|
||||
const records = otadata.records.flatMap(({ index, available }, position) => {
|
||||
const read = bytes[position];
|
||||
return available &&
|
||||
read !== undefined &&
|
||||
read.length >= ESP_OTA_SELECT_ENTRY_BYTES
|
||||
? [decodeEspOtaRecord(read, index)]
|
||||
: [];
|
||||
});
|
||||
return {
|
||||
partition_index: plan.otadata.partition.index,
|
||||
partition_index: otadata.partition.index,
|
||||
records,
|
||||
selected: selectEspOtaSlot(records, plan.input.table.partitions),
|
||||
selected: selectEspOtaSlot(records, plan.boot, otadata.partition.size),
|
||||
running_partition: "unknown",
|
||||
};
|
||||
};
|
||||
@@ -323,6 +388,7 @@ export const espFlashPartitionFacts = (part: EspPartition) => ({
|
||||
kind: "esp-flash-partition" as const,
|
||||
index: part.index,
|
||||
label: part.label,
|
||||
label_hex: part.labelHex,
|
||||
type: part.type,
|
||||
subtype: part.subtype,
|
||||
type_name: espPartitionTypeName(part.type),
|
||||
|
||||
@@ -4,12 +4,18 @@ import {
|
||||
locateEspFlashLayout,
|
||||
type EspFlashSelection,
|
||||
} from "./espFlashLocation.js";
|
||||
import { ESP8266_REFUSAL } from "./espImage.js";
|
||||
import {
|
||||
TWO_OTA_PARTITIONS,
|
||||
flashImage,
|
||||
partitionTable,
|
||||
} from "./espFlash.fixture.js";
|
||||
import { ESP32C3_BOOTLOADER_SEGMENTS, espImage } from "./espImage.fixture.js";
|
||||
import {
|
||||
ESP32C3_APP_SEGMENTS,
|
||||
ESP32C3_BOOTLOADER_SEGMENTS,
|
||||
esp8266Image,
|
||||
espImage,
|
||||
} from "./espImage.fixture.js";
|
||||
|
||||
const c3Bootloader = espImage({ segments: ESP32C3_BOOTLOADER_SEGMENTS });
|
||||
const esp32Bootloader = espImage({
|
||||
@@ -21,11 +27,17 @@ const locate = (
|
||||
bytes: Uint8Array,
|
||||
selection: EspFlashSelection = {},
|
||||
explicit = false,
|
||||
reads: [number, number][] = [],
|
||||
) =>
|
||||
locateEspFlashLayout(
|
||||
bytes.length,
|
||||
async (position, length) =>
|
||||
bytes.subarray(Math.max(0, position), Math.max(0, position) + length),
|
||||
async (position, length) => {
|
||||
reads.push([position, length]);
|
||||
return bytes.subarray(
|
||||
Math.max(0, position),
|
||||
Math.max(0, position) + length,
|
||||
);
|
||||
},
|
||||
selection,
|
||||
{ explicit },
|
||||
);
|
||||
@@ -47,6 +59,15 @@ const dump = (
|
||||
{ at: tableAt, bytes: partitionTable(TWO_OTA_PARTITIONS, { md5 }) },
|
||||
]);
|
||||
|
||||
/** An ESP32 dump read from flash 0x1000: its bootloader is the file's first byte. */
|
||||
const fromSecondSector = (tableAt: number) =>
|
||||
dump(tableAt, { bootloader: esp32Bootloader, bootloaderAt: 0x1000 }).subarray(
|
||||
0x1000,
|
||||
);
|
||||
|
||||
const UNCONFIRMED =
|
||||
"No bootloader at a ROM offset confirms that the dump starts at flash 0x0; each partition's file offset assumes it. Pass flash_offset if the dump starts elsewhere.";
|
||||
|
||||
describe("ESP flash layout location", () => {
|
||||
it("finds the bootloader and the table at their default offsets", async () => {
|
||||
const located = await locate(dump(0x8000));
|
||||
@@ -60,6 +81,20 @@ describe("ESP flash layout location", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("locates a bootloader at the 0x2000 ROM offset of the chips that use it", async () => {
|
||||
const c5 = espImage({
|
||||
segments: ESP32C3_BOOTLOADER_SEGMENTS,
|
||||
chipId: 0x17,
|
||||
});
|
||||
const located = await locate(
|
||||
dump(0x8000, { bootloader: c5, bootloaderAt: 0x2000 }),
|
||||
);
|
||||
expect(located.ok && located.value).toMatchObject({
|
||||
bootloader: { flashAddress: 0x2000, chip: { target: "esp32c5" } },
|
||||
tableMethod: "default",
|
||||
});
|
||||
});
|
||||
|
||||
it("scans for a relocated table only when its MD5 verifies", async () => {
|
||||
const located = await locate(dump(0xa000));
|
||||
expect(located.ok && located.value).toMatchObject({
|
||||
@@ -69,10 +104,48 @@ describe("ESP flash layout location", () => {
|
||||
expect(await locate(dump(0xa000, { md5: "absent" }))).toEqual({
|
||||
ok: false,
|
||||
error:
|
||||
"no partition table at 0x8000 or in any 0x1000-aligned flash sector with a verified MD5; flash encryption is one cause",
|
||||
"no partition table at 0x8000 or in any 0x1000-aligned flash sector after the bootloader image with a verified MD5; flash encryption is one cause",
|
||||
});
|
||||
});
|
||||
|
||||
it.each([0x1_0000, 0x1_1000])(
|
||||
"scans across read chunks to a table at %s",
|
||||
async (tableAt) => {
|
||||
const located = await locate(dump(tableAt));
|
||||
expect(located.ok && located.value).toMatchObject({
|
||||
tableAddress: tableAt,
|
||||
tableMethod: "scan",
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("scans the sectors of a dump whose origin is not sector-aligned", async () => {
|
||||
// Flash 0x9000 is file offset 0x8800 when the dump starts at flash 0x800.
|
||||
const bytes = flashImage(0x2_0000, [
|
||||
{ at: 0x8800, bytes: partitionTable(TWO_OTA_PARTITIONS) },
|
||||
]);
|
||||
const located = await locate(bytes, { flashOffset: 0x800 }, true);
|
||||
expect(located.ok && located.value).toMatchObject({
|
||||
origin: { value: 0x800, source: "caller" },
|
||||
tableAddress: 0x9000,
|
||||
tableMethod: "scan",
|
||||
limitations: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("does not look for a table inside a single app image", async () => {
|
||||
const app = espImage({
|
||||
segments: [
|
||||
...ESP32C3_APP_SEGMENTS,
|
||||
{ address: 0x4038_0000, data: Buffer.alloc(0x2_0000, 0x5a) },
|
||||
],
|
||||
});
|
||||
const reads: [number, number][] = [];
|
||||
expect((await locate(app, {}, false, reads)).ok).toBe(false);
|
||||
// Probes and segment headers only: no table read, no scan chunk.
|
||||
expect(reads.every(([, length]) => length <= 32)).toBe(true);
|
||||
});
|
||||
|
||||
it("uses the caller's table offset and does not scan", async () => {
|
||||
const located = await locate(dump(0xa000), {
|
||||
partitionTableOffset: 0xa000,
|
||||
@@ -90,13 +163,22 @@ describe("ESP flash layout location", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a table offset outside the dump", async () => {
|
||||
expect(
|
||||
await locate(dump(0x8000), { partitionTableOffset: 0x10_0000 }),
|
||||
).toEqual({
|
||||
ok: false,
|
||||
error:
|
||||
"the partition table offset 0x100000 is outside the dump, which holds flash 0x0-0x20000",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("ESP flash layout origins", () => {
|
||||
it("reads a dump that starts after flash address 0 at the caller's origin", async () => {
|
||||
// An ESP32 dump read from 0x1000: its bootloader is the file's first byte.
|
||||
const partial = dump(0x8000, {
|
||||
bootloader: esp32Bootloader,
|
||||
bootloaderAt: 0x1000,
|
||||
}).subarray(0x1000);
|
||||
const located = await locate(partial, { flashOffset: 0x1000 });
|
||||
const located = await locate(fromSecondSector(0x8000), {
|
||||
flashOffset: 0x1000,
|
||||
});
|
||||
expect(located.ok && located.value).toMatchObject({
|
||||
origin: { value: 0x1000, source: "caller" },
|
||||
bootloader: {
|
||||
@@ -106,12 +188,35 @@ describe("ESP flash layout location", () => {
|
||||
},
|
||||
tableAddress: 0x8000,
|
||||
tableMethod: "default",
|
||||
limitations: [],
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
["omitted", {}],
|
||||
["given as 0x0", { flashOffset: 0 }],
|
||||
] as const)(
|
||||
"refuses an origin the bootloader contradicts when it is %s",
|
||||
async (_name, selection) => {
|
||||
// The table sits at flash 0x9000, so the default 0x8000 would find it
|
||||
// at file 0x8000 and misplace every partition by 0x1000.
|
||||
const refused = await locate(fromSecondSector(0x9000), selection);
|
||||
expect(refused).toEqual({
|
||||
ok: false,
|
||||
error:
|
||||
"An esp32 image is at flash 0x0, but its ROM loads the bootloader from 0x1000; the dump may not start at flash 0x0. Pass the flash_offset where the dump starts, or partition_table_offset",
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("finds the table after the bootloader once the origin is right", async () => {
|
||||
const located = await locate(fromSecondSector(0x9000), {
|
||||
flashOffset: 0x1000,
|
||||
});
|
||||
expect(located.ok && located.value).toMatchObject({
|
||||
tableAddress: 0x9000,
|
||||
tableMethod: "scan",
|
||||
});
|
||||
// Without the origin, the bootloader's chip says the dump is misplaced, so
|
||||
// a scan would misplace every partition by 0x1000; none is attempted.
|
||||
const misplaced = await locate(partial);
|
||||
expect(misplaced.ok).toBe(false);
|
||||
expect(!misplaced.ok && misplaced.error).toContain("Pass flash_offset");
|
||||
});
|
||||
|
||||
it("does not scan arbitrary files without a bootloader unless the format was selected", async () => {
|
||||
@@ -127,16 +232,38 @@ describe("ESP flash layout location", () => {
|
||||
bootloader: undefined,
|
||||
tableAddress: 0xa000,
|
||||
tableMethod: "scan",
|
||||
limitations: [UNCONFIRMED],
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a table offset outside the dump", async () => {
|
||||
expect(
|
||||
await locate(dump(0x8000), { partitionTableOffset: 0x10_0000 }),
|
||||
).toEqual({
|
||||
it("reads an unknown chip's dump at its default table without calling it misplaced", async () => {
|
||||
const unknown = espImage({
|
||||
segments: ESP32C3_BOOTLOADER_SEGMENTS,
|
||||
chipId: 0x7777,
|
||||
});
|
||||
const located = await locate(dump(0x8000, { bootloader: unknown }));
|
||||
expect(located.ok && located.value).toMatchObject({
|
||||
bootloader: undefined,
|
||||
tableMethod: "default",
|
||||
limitations: [
|
||||
"An ESP image at flash 0x0 has chip_id 0x7777, which is not in the chip table; it is not read as the bootloader.",
|
||||
UNCONFIRMED,
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses an ESP8266 dump as the ESP image reader does", async () => {
|
||||
const esp8266 = flashImage(0x2_0000, [
|
||||
{ at: 0, bytes: esp8266Image() },
|
||||
{ at: 0x8000, bytes: partitionTable(TWO_OTA_PARTITIONS) },
|
||||
]);
|
||||
expect(await locate(esp8266)).toEqual({
|
||||
ok: false,
|
||||
error:
|
||||
"the partition table offset 0x100000 is outside the dump, which holds flash 0x0-0x20000",
|
||||
error: ESP8266_REFUSAL,
|
||||
});
|
||||
expect(await locate(esp8266, {}, true)).toEqual({
|
||||
ok: false,
|
||||
error: ESP8266_REFUSAL,
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -2,6 +2,7 @@ import { err, ok, type Result } from "../result.js";
|
||||
import {
|
||||
ESP_BOOTLOADER_PROBE_BYTES,
|
||||
espBootloaderProbeOffsets,
|
||||
espImageSegmentsEnd,
|
||||
locateEspBootloader,
|
||||
type EspBootloaderLocation,
|
||||
} from "./espFlashBootloader.js";
|
||||
@@ -9,6 +10,7 @@ import type {
|
||||
EspFlashOrigin,
|
||||
EspPartitionTableMethod,
|
||||
} from "./espFlashLayout.js";
|
||||
import { ESP8266_REFUSAL } from "./espImage.js";
|
||||
import { offsetText } from "./espImageStructures.js";
|
||||
import {
|
||||
ESP_PARTITION_TABLE_DEFAULT_OFFSET,
|
||||
@@ -55,40 +57,40 @@ const tableAt = async (
|
||||
);
|
||||
|
||||
/**
|
||||
* The first sector-aligned table whose MD5 verifies; requiring the MD5
|
||||
* keeps arbitrary bytes that begin with the magic from passing.
|
||||
* The first table whose MD5 verifies at a flash sector boundary from the
|
||||
* file offset `start`; requiring the MD5 keeps arbitrary bytes that begin
|
||||
* with the magic from passing. Chunks start at sector boundaries, so each
|
||||
* holds whole sector heads.
|
||||
*/
|
||||
const scanForTable = async (
|
||||
read: EspFlashRead,
|
||||
fileSize: number,
|
||||
origin: number,
|
||||
{
|
||||
fileSize,
|
||||
origin,
|
||||
start,
|
||||
}: { fileSize: number; origin: number; start: number },
|
||||
signal?: AbortSignal,
|
||||
): Promise<{ address: number; table: EspPartitionTable } | undefined> => {
|
||||
const skew =
|
||||
(ESP_PARTITION_TABLE_SECTOR_BYTES -
|
||||
(origin % ESP_PARTITION_TABLE_SECTOR_BYTES)) %
|
||||
ESP_PARTITION_TABLE_SECTOR_BYTES;
|
||||
for (let chunk = 0; chunk < fileSize; chunk += SCAN_CHUNK_BYTES) {
|
||||
signal?.throwIfAborted();
|
||||
// One extra byte lets a sector head at the chunk's last byte be checked.
|
||||
const bytes = await read(chunk, SCAN_CHUNK_BYTES + 1);
|
||||
const first =
|
||||
chunk +
|
||||
((skew -
|
||||
(chunk % ESP_PARTITION_TABLE_SECTOR_BYTES) +
|
||||
ESP_PARTITION_TABLE_SECTOR_BYTES) %
|
||||
ESP_PARTITION_TABLE_SECTOR_BYTES);
|
||||
for (
|
||||
let position = first;
|
||||
position < chunk + SCAN_CHUNK_BYTES && position < fileSize;
|
||||
position += ESP_PARTITION_TABLE_SECTOR_BYTES
|
||||
) {
|
||||
const at = position - chunk;
|
||||
if (bytes[at] !== MAGIC_LOW || bytes[at + 1] !== MAGIC_HIGH) continue;
|
||||
const decoded = await tableAt(read, position);
|
||||
if (decoded.ok && decoded.value.md5.status === "valid")
|
||||
return { address: origin + position, table: decoded.value };
|
||||
const sector = ESP_PARTITION_TABLE_SECTOR_BYTES;
|
||||
// The file offset of the first sector boundary at or after `start`.
|
||||
const boundary = (sector - (origin % sector)) % sector;
|
||||
let position =
|
||||
start <= boundary
|
||||
? boundary
|
||||
: boundary + Math.ceil((start - boundary) / sector) * sector;
|
||||
let chunkStart = position;
|
||||
let bytes: Uint8Array = new Uint8Array(0);
|
||||
for (; position < fileSize; position += sector) {
|
||||
if (position - chunkStart + 2 > bytes.length) {
|
||||
signal?.throwIfAborted();
|
||||
chunkStart = position;
|
||||
bytes = await read(position, SCAN_CHUNK_BYTES);
|
||||
}
|
||||
const at = position - chunkStart;
|
||||
if (bytes[at] !== MAGIC_LOW || bytes[at + 1] !== MAGIC_HIGH) continue;
|
||||
const decoded = await tableAt(read, position);
|
||||
if (decoded.ok && decoded.value.md5.status === "valid")
|
||||
return { address: origin + position, table: decoded.value };
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
@@ -96,10 +98,12 @@ const scanForTable = async (
|
||||
/**
|
||||
* Locate the bootloader and partition table: the caller's table offset
|
||||
* when given, else the default `0x8000`, else a scan of 0x1000-aligned
|
||||
* flash sectors for an MD5-verified table (a derived location). The scan
|
||||
* runs only when a bootloader sits at its chip's ROM offset or the caller
|
||||
* selected this format, and never when a bootloader at another chip's ROM
|
||||
* offset says the dump's origin is wrong.
|
||||
* flash sectors for an MD5-verified table (a derived location). A table
|
||||
* never lies inside the located bootloader image. The scan runs only when a
|
||||
* bootloader sits at its chip's ROM offset or the caller selected this
|
||||
* format. Without a caller table offset, an ESP8266 image at a ROM offset
|
||||
* refuses the dump, and a known chip's image at another chip's ROM offset
|
||||
* refuses it as read from the wrong flash offset, given or default.
|
||||
*/
|
||||
export const locateEspFlashLayout = async (
|
||||
fileSize: number,
|
||||
@@ -125,6 +129,15 @@ export const locateEspFlashLayout = async (
|
||||
),
|
||||
);
|
||||
const bootloader = locateEspBootloader(probes);
|
||||
// Without a bootloader, nothing in the dump confirms a default origin.
|
||||
const unconfirmed =
|
||||
bootloader.location === undefined &&
|
||||
!bootloader.misplaced &&
|
||||
origin.source === "default"
|
||||
? [
|
||||
`No bootloader at a ROM offset confirms that the dump starts at flash ${offsetText(origin.value)}; each partition's file offset assumes it. Pass flash_offset if the dump starts elsewhere.`,
|
||||
]
|
||||
: [];
|
||||
const located = (
|
||||
tableAddress: number,
|
||||
tableMethod: EspPartitionTableMethod,
|
||||
@@ -136,7 +149,7 @@ export const locateEspFlashLayout = async (
|
||||
tableAddress,
|
||||
tableMethod,
|
||||
table,
|
||||
limitations: bootloader.limitations,
|
||||
limitations: [...bootloader.limitations, ...unconfirmed],
|
||||
});
|
||||
const fileOffsetOf = (address: number): number => address - origin.value;
|
||||
if (selection.partitionTableOffset !== undefined) {
|
||||
@@ -152,8 +165,24 @@ export const locateEspFlashLayout = async (
|
||||
`no partition table at the given offset ${offsetText(address)}: ${decoded.error}`,
|
||||
);
|
||||
}
|
||||
if (bootloader.location === undefined && bootloader.esp8266)
|
||||
return err(ESP8266_REFUSAL);
|
||||
// A table found at an origin the bootloader contradicts would place every
|
||||
// partition at the wrong file offset.
|
||||
if (bootloader.location === undefined && bootloader.misplaced)
|
||||
return err(
|
||||
`${bootloader.limitations.join(" ")} Pass the flash_offset where the dump starts, or partition_table_offset`,
|
||||
);
|
||||
const imageEnd =
|
||||
bootloader.location === undefined
|
||||
? 0
|
||||
: await espImageSegmentsEnd(
|
||||
read,
|
||||
bootloader.location.fileOffset,
|
||||
fileSize,
|
||||
);
|
||||
const defaultOffset = fileOffsetOf(ESP_PARTITION_TABLE_DEFAULT_OFFSET);
|
||||
if (defaultOffset >= 0 && defaultOffset < fileSize) {
|
||||
if (defaultOffset >= imageEnd && defaultOffset < fileSize) {
|
||||
const decoded = await tableAt(read, defaultOffset);
|
||||
if (decoded.ok)
|
||||
return located(
|
||||
@@ -162,20 +191,21 @@ export const locateEspFlashLayout = async (
|
||||
decoded.value,
|
||||
);
|
||||
}
|
||||
const misplaced =
|
||||
bootloader.location === undefined && bootloader.limitations.length > 0;
|
||||
if (misplaced && selection.flashOffset === undefined)
|
||||
return err(
|
||||
`no partition table at ${offsetText(ESP_PARTITION_TABLE_DEFAULT_OFFSET)}, and ${bootloader.limitations.join(" ")} Pass flash_offset to read the dump from another flash address`,
|
||||
);
|
||||
if (bootloader.location === undefined && !explicit)
|
||||
return err(
|
||||
`no partition table at ${offsetText(ESP_PARTITION_TABLE_DEFAULT_OFFSET)} and no bootloader at a ROM offset`,
|
||||
[
|
||||
`no partition table at ${offsetText(ESP_PARTITION_TABLE_DEFAULT_OFFSET)} and no bootloader at a ROM offset`,
|
||||
...bootloader.limitations,
|
||||
].join("; "),
|
||||
);
|
||||
const scanned = await scanForTable(read, fileSize, origin.value, signal);
|
||||
const scanned = await scanForTable(
|
||||
read,
|
||||
{ fileSize, origin: origin.value, start: imageEnd },
|
||||
signal,
|
||||
);
|
||||
return scanned === undefined
|
||||
? err(
|
||||
`no partition table at ${offsetText(ESP_PARTITION_TABLE_DEFAULT_OFFSET)} or in any 0x1000-aligned flash sector with a verified MD5; flash encryption is one cause`,
|
||||
`no partition table at ${offsetText(ESP_PARTITION_TABLE_DEFAULT_OFFSET)} or in any 0x1000-aligned flash sector${bootloader.location === undefined ? "" : " after the bootloader image"} with a verified MD5; flash encryption is one cause`,
|
||||
)
|
||||
: located(scanned.address, "scan", scanned.table);
|
||||
};
|
||||
@@ -60,7 +60,10 @@ describe("ESP flash layout plans", () => {
|
||||
["partitions/ota_1.bin", 0x3_0000, 0x1_0000, 0x1_0000],
|
||||
]);
|
||||
expect(plan.gaps).toEqual([]);
|
||||
expect(plan.otadata?.records).toEqual([0xd000, 0xe000]);
|
||||
expect(plan.otadata?.records).toEqual([
|
||||
{ index: 0, fileOffset: 0xd000, available: true },
|
||||
{ index: 1, fileOffset: 0xe000, available: true },
|
||||
]);
|
||||
expect(plan.limitations).toEqual([]);
|
||||
});
|
||||
|
||||
@@ -70,6 +73,9 @@ describe("ESP flash layout plans", () => {
|
||||
{ label: "dup", type: 1, subtype: 2, offset: 0xa000, size: 0x1000 },
|
||||
{ label: "dup", type: 1, subtype: 2, offset: 0xb000, size: 0x1000 },
|
||||
{ label: "7", type: 1, subtype: 2, offset: 0xc000, size: 0x1000 },
|
||||
// Case-insensitive file systems would extract these to one file.
|
||||
{ label: "Data", type: 1, subtype: 2, offset: 0xd000, size: 0x1000 },
|
||||
{ label: "data", type: 1, subtype: 2, offset: 0xe000, size: 0x1000 },
|
||||
];
|
||||
expect(
|
||||
planEspFlashLayout(input({ table: table(parts) }))
|
||||
@@ -80,6 +86,39 @@ describe("ESP flash layout plans", () => {
|
||||
"partitions/1.bin",
|
||||
"partitions/2.bin",
|
||||
"partitions/3.bin",
|
||||
"partitions/4.bin",
|
||||
"partitions/5.bin",
|
||||
]);
|
||||
});
|
||||
|
||||
it("accepts the primary bootloader and table entries ESP-IDF 5.3 declares", () => {
|
||||
const plan = planEspFlashLayout(
|
||||
input({
|
||||
table: table([
|
||||
{
|
||||
label: "PrimaryBTLDR",
|
||||
type: 2,
|
||||
subtype: 0,
|
||||
offset: 0,
|
||||
size: 0x8000,
|
||||
},
|
||||
{
|
||||
label: "PrimaryPrtTable",
|
||||
type: 3,
|
||||
subtype: 0,
|
||||
offset: 0x8000,
|
||||
size: 0x1000,
|
||||
},
|
||||
...TWO_OTA_PARTITIONS,
|
||||
]),
|
||||
}),
|
||||
);
|
||||
expect(plan.limitations).toEqual([]);
|
||||
expect(plan.regions.map(({ path }) => path).slice(0, 4)).toEqual([
|
||||
"bootloader.bin",
|
||||
"partition-table.bin",
|
||||
"partitions/PrimaryBTLDR.bin",
|
||||
"partitions/PrimaryPrtTable.bin",
|
||||
]);
|
||||
});
|
||||
|
||||
@@ -133,27 +172,93 @@ describe("ESP flash layout plans", () => {
|
||||
"Partition gone at 0x100000-0x101000 lies outside the dump, which ends at flash 0x39000.",
|
||||
]);
|
||||
const facts = finishEspFlashLayout(plan, {
|
||||
otadataRecords: undefined,
|
||||
otadataRecords: [],
|
||||
gapFirstUnerased: erasedGaps(plan.gaps.length),
|
||||
});
|
||||
expect(facts.coverage).toBe("unknown");
|
||||
});
|
||||
|
||||
it("reports overlapping regions", () => {
|
||||
it("reports every region inside a longer one", () => {
|
||||
const plan = planEspFlashLayout(
|
||||
input({
|
||||
table: table([
|
||||
{ label: "a", type: 1, subtype: 2, offset: 0x9000, size: 0x2000 },
|
||||
{ label: "a", type: 1, subtype: 2, offset: 0x9000, size: 0x8000 },
|
||||
{ label: "b", type: 1, subtype: 2, offset: 0xa000, size: 0x1000 },
|
||||
{ label: "c", type: 1, subtype: 2, offset: 0xc000, size: 0x1000 },
|
||||
]),
|
||||
}),
|
||||
);
|
||||
expect(plan.limitations).toEqual([
|
||||
"partitions/b.bin at 0xa000 overlaps partitions/a.bin, which ends at 0xb000.",
|
||||
"partitions/b.bin at 0xa000 overlaps partitions/a.bin, which ends at 0x11000.",
|
||||
"partitions/c.bin at 0xc000 overlaps partitions/a.bin, which ends at 0x11000.",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("ESP flash layout otadata", () => {
|
||||
const [, otadata] = TWO_OTA_PARTITIONS;
|
||||
|
||||
it("reads the otadata partition the bootloader uses: the last declared", () => {
|
||||
const plan = planEspFlashLayout(
|
||||
input({
|
||||
table: table([
|
||||
...TWO_OTA_PARTITIONS,
|
||||
{ ...otadata!, label: "otadata2", offset: 0x3_0000 + 0x1_0000 },
|
||||
]),
|
||||
fileSize: 0x5_0000,
|
||||
}),
|
||||
);
|
||||
expect(plan.otadata?.partition.label).toBe("otadata2");
|
||||
expect(plan.otadata?.records.map(({ fileOffset }) => fileOffset)).toEqual([
|
||||
0x4_0000, 0x4_1000,
|
||||
]);
|
||||
expect(plan.limitations).toContain(
|
||||
"The table declares otadata 2 times (partitions #1, #6); ESP-IDF's bootloader uses the last.",
|
||||
);
|
||||
});
|
||||
|
||||
it("does not read a record past a short otadata partition and selects none", () => {
|
||||
const short = TWO_OTA_PARTITIONS.map((part) =>
|
||||
part.label === "otadata" ? { ...part, size: 0x1000 } : part,
|
||||
);
|
||||
const plan = planEspFlashLayout(input({ table: table(short) }));
|
||||
expect(plan.otadata?.records).toEqual([
|
||||
{ index: 0, fileOffset: 0xd000, available: true },
|
||||
]);
|
||||
expect(plan.limitations).toContain(
|
||||
"The otadata partition otadata is 0x1000 bytes, smaller than the two 4 KiB sectors ESP-IDF's bootloader reads; the bootloader resets without selecting an app.",
|
||||
);
|
||||
const facts = finishEspFlashLayout(plan, {
|
||||
otadataRecords: [otaRecord({ seq: 1 })],
|
||||
gapFirstUnerased: erasedGaps(plan.gaps.length),
|
||||
});
|
||||
expect(facts.otadata).toMatchObject({
|
||||
records: [{ index: 0, ota_seq: 1 }],
|
||||
selected: { basis: "none", partition_index: null },
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps the record a dump holds when it ends between the two", () => {
|
||||
const plan = planEspFlashLayout(input({ fileSize: 0xe000 }));
|
||||
expect(plan.otadata?.records).toEqual([
|
||||
{ index: 0, fileOffset: 0xd000, available: true },
|
||||
{ index: 1, fileOffset: 0xe000, available: false },
|
||||
]);
|
||||
expect(plan.limitations).toContain(
|
||||
"otadata record 1 of partition otadata at flash 0xe000 is outside the dump; the OTA selection is unknown.",
|
||||
);
|
||||
const facts = finishEspFlashLayout(plan, {
|
||||
otadataRecords: [otaRecord({ seq: 3 }), undefined],
|
||||
gapFirstUnerased: erasedGaps(plan.gaps.length),
|
||||
});
|
||||
expect(facts.otadata).toMatchObject({
|
||||
records: [{ index: 0, ota_seq: 3, valid: true }],
|
||||
selected: null,
|
||||
running_partition: "unknown",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("ESP flash layout facts", () => {
|
||||
it("describes erased gaps and reports the rest as undescribed", () => {
|
||||
const plan = planEspFlashLayout(
|
||||
@@ -164,18 +269,18 @@ describe("ESP flash layout facts", () => {
|
||||
);
|
||||
expect(plan.gaps).toEqual([{ offset: 0x1_0000, length: 0x1_0000 }]);
|
||||
const erased = finishEspFlashLayout(plan, {
|
||||
otadataRecords: undefined,
|
||||
otadataRecords: [],
|
||||
gapFirstUnerased: [undefined],
|
||||
});
|
||||
expect(erased).toMatchObject({ coverage: "complete", undescribed: [] });
|
||||
const written = finishEspFlashLayout(plan, {
|
||||
otadataRecords: undefined,
|
||||
otadataRecords: [],
|
||||
gapFirstUnerased: [0x1_8000],
|
||||
});
|
||||
expect(written.undescribed).toEqual([{ offset: 0x1_8000, length: 0x8000 }]);
|
||||
expect(written.coverage).toBe("unknown");
|
||||
expect(written.limitations.at(-1)).toBe(
|
||||
"32768 bytes of the dump, in 1 range(s), lie outside the bootloader region, the partition table and every partition and are not erased flash.",
|
||||
"1 range(s) of the dump, 32768 bytes in all, lie outside the bootloader region, the partition table and every partition and are not described; each runs from a byte that is not erased flash to the end of its gap.",
|
||||
);
|
||||
});
|
||||
|
||||
@@ -205,6 +310,7 @@ describe("ESP flash layout facts", () => {
|
||||
kind: "esp-flash-partition",
|
||||
index: 0,
|
||||
label: "nvs",
|
||||
label_hex: null,
|
||||
type: 1,
|
||||
subtype: 2,
|
||||
type_name: "data",
|
||||
|
||||
@@ -56,11 +56,16 @@ const allZero = (bytes: Uint8Array): boolean =>
|
||||
|
||||
const UTF8 = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
|
||||
|
||||
/** A NUL-terminated C string in a fixed array, or null when not UTF-8. */
|
||||
const cString = (bytes: Uint8Array): string | null => {
|
||||
/** The bytes of a NUL-terminated C string in a fixed array. */
|
||||
export const cStringBytes = (bytes: Uint8Array): Uint8Array => {
|
||||
const end = bytes.indexOf(0);
|
||||
return end < 0 ? bytes : bytes.subarray(0, end);
|
||||
};
|
||||
|
||||
/** A NUL-terminated C string in a fixed array, or null when not UTF-8. */
|
||||
export const cString = (bytes: Uint8Array): string | null => {
|
||||
try {
|
||||
return UTF8.decode(end < 0 ? bytes : bytes.subarray(0, end));
|
||||
return UTF8.decode(cStringBytes(bytes));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -1,11 +1,7 @@
|
||||
import { crc32 } from "node:zlib";
|
||||
|
||||
import { hex, u32 } from "./espImageStructures.js";
|
||||
import {
|
||||
espOtaAppSlot,
|
||||
isEspFactoryApp,
|
||||
type EspPartition,
|
||||
} from "./espPartitionTable.js";
|
||||
import type { EspBootPartitions } from "./espPartitionTable.js";
|
||||
|
||||
/*
|
||||
* OTA data, from ESP-IDF v6.1 `esp_flash_partitions.h`
|
||||
@@ -15,6 +11,8 @@ import {
|
||||
*/
|
||||
/** The otadata partition holds one record at the start of each of two sectors. */
|
||||
export const ESP_OTA_DATA_SECTOR_BYTES = 0x1000;
|
||||
/** `read_otadata` refuses a smaller otadata partition. */
|
||||
export const ESP_OTA_DATA_MIN_BYTES = 2 * ESP_OTA_DATA_SECTOR_BYTES;
|
||||
export const ESP_OTA_SELECT_ENTRY_BYTES = 32;
|
||||
const UNSET = 0xffff_ffff;
|
||||
|
||||
@@ -73,7 +71,8 @@ export const decodeEspOtaRecord = (
|
||||
export const ESP_OTA_SELECTION_ASSUMPTIONS = [
|
||||
"Records are used as written: the bootloader may first mark a pending-verify record aborted when app rollback is enabled.",
|
||||
"Anti-rollback, which compares the app's secure_version with eFuse, is not applied.",
|
||||
"The selected app is not checked for a loadable image; the bootloader falls back to other partitions when it fails to load.",
|
||||
"The selected app is not checked for a loadable image; the bootloader falls back to other partitions when it fails to load or is not declared.",
|
||||
"Factory-reset and test-app GPIO holds and a wake from deep sleep, which can override the selection, are not applied.",
|
||||
] as const;
|
||||
|
||||
export interface EspOtaSelection {
|
||||
@@ -81,8 +80,9 @@ export interface EspOtaSelection {
|
||||
readonly active_record: 0 | 1 | null;
|
||||
/** `(ota_seq - 1) % ota_app_count`, the bootloader's slot arithmetic. */
|
||||
readonly slot: number | null;
|
||||
/** The table index of the app partition for that slot or the fallback. */
|
||||
/** The table index of the selected app; null when none is declared. */
|
||||
readonly partition_index: number | null;
|
||||
/** The bootloader's branch; `none` is its INVALID_INDEX, which resets. */
|
||||
readonly basis: "otadata" | "factory-fallback" | "ota_0-fallback" | "none";
|
||||
readonly assumptions: readonly string[];
|
||||
}
|
||||
@@ -92,58 +92,48 @@ const unusable = ({ ota_seq, ota_state }: EspOtaRecord): boolean =>
|
||||
ota_seq === UNSET || ota_state === 0x3 || ota_state === 0x4;
|
||||
|
||||
/**
|
||||
* Derive the app partition the bootloader would select from these records,
|
||||
* as `bootloader_utility_get_selected_boot_partition` does without app
|
||||
* Derive the app partition the bootloader would select, as
|
||||
* `bootloader_utility_get_selected_boot_partition` does without app
|
||||
* rollback or anti-rollback:
|
||||
* - both records unusable, or no OTA app: the factory app, else `ota_0`;
|
||||
* - an otadata partition smaller than two sectors: none, because
|
||||
* `read_otadata` fails and the bootloader resets;
|
||||
* - both records unusable, or no OTA app: the factory app when one is
|
||||
* declared, else boot index 0 (`ota_0`);
|
||||
* - otherwise the valid record with the higher `ota_seq` (record 0 on a
|
||||
* tie), whose slot is `(ota_seq - 1) % ota_app_count`;
|
||||
* - no valid record (a CRC mismatch): the factory app, else none, because
|
||||
* the bootloader then tries every partition.
|
||||
* - no valid record, as after a CRC mismatch: the factory index, whether
|
||||
* or not a factory app is declared.
|
||||
* Unknown (null) when the dump lacks a record the bootloader would read.
|
||||
*/
|
||||
export const selectEspOtaSlot = (
|
||||
records: readonly EspOtaRecord[],
|
||||
partitions: readonly EspPartition[],
|
||||
): EspOtaSelection => {
|
||||
// The bootloader counts every OTA app but indexes them by subtype, and a
|
||||
// later entry with the same subtype replaces an earlier one.
|
||||
let appCount = 0;
|
||||
const bySlot = new Map<number, EspPartition>();
|
||||
for (const part of partitions) {
|
||||
const slot = espOtaAppSlot(part);
|
||||
if (slot === null) continue;
|
||||
appCount += 1;
|
||||
bySlot.set(slot, part);
|
||||
}
|
||||
const factory = partitions.find(isEspFactoryApp);
|
||||
boot: EspBootPartitions,
|
||||
otadataSize: number,
|
||||
): EspOtaSelection | null => {
|
||||
const assumptions = [...ESP_OTA_SELECTION_ASSUMPTIONS];
|
||||
const none = {
|
||||
const selection = (
|
||||
basis: EspOtaSelection["basis"],
|
||||
fields: Partial<Omit<EspOtaSelection, "basis" | "assumptions">> = {},
|
||||
): EspOtaSelection => ({
|
||||
active_record: null,
|
||||
slot: null,
|
||||
partition_index: null,
|
||||
basis: "none",
|
||||
...fields,
|
||||
basis,
|
||||
assumptions,
|
||||
} as const;
|
||||
const factoryFallback =
|
||||
factory === undefined
|
||||
? undefined
|
||||
: ({
|
||||
...none,
|
||||
partition_index: factory.index,
|
||||
basis: "factory-fallback",
|
||||
} as const);
|
||||
if (records.every(unusable) || appCount === 0) {
|
||||
if (factoryFallback !== undefined) return factoryFallback;
|
||||
const first = bySlot.get(0);
|
||||
return first === undefined
|
||||
? none
|
||||
: {
|
||||
...none,
|
||||
});
|
||||
const factory = selection("factory-fallback", {
|
||||
partition_index: boot.factory?.index ?? null,
|
||||
});
|
||||
if (otadataSize < ESP_OTA_DATA_MIN_BYTES) return selection("none");
|
||||
if (records.length < 2) return null;
|
||||
if (records.every(unusable) || boot.otaAppCount === 0)
|
||||
return boot.factory !== undefined
|
||||
? factory
|
||||
: selection("ota_0-fallback", {
|
||||
slot: 0,
|
||||
partition_index: first.index,
|
||||
basis: "ota_0-fallback",
|
||||
};
|
||||
}
|
||||
partition_index: boot.otaSlots.get(0)?.index ?? null,
|
||||
});
|
||||
const valid = records.filter(({ valid }) => valid);
|
||||
const [first, second] = valid;
|
||||
const active =
|
||||
@@ -152,14 +142,12 @@ export const selectEspOtaSlot = (
|
||||
second.ota_seq > first.ota_seq
|
||||
? second
|
||||
: first;
|
||||
if (active === undefined) return factoryFallback ?? none;
|
||||
const slot = ((active.ota_seq - 1) >>> 0) % appCount;
|
||||
return {
|
||||
if (active === undefined) return factory;
|
||||
const slot = ((active.ota_seq - 1) >>> 0) % boot.otaAppCount;
|
||||
return selection("otadata", {
|
||||
active_record: active.index,
|
||||
slot,
|
||||
// A gap in the OTA subtypes selects a slot with no partition.
|
||||
partition_index: bySlot.get(slot)?.index ?? null,
|
||||
basis: "otadata",
|
||||
assumptions,
|
||||
};
|
||||
partition_index: boot.otaSlots.get(slot)?.index ?? null,
|
||||
});
|
||||
};
|
||||
@@ -1,7 +1,14 @@
|
||||
import { createHash } from "node:crypto";
|
||||
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { hex, offsetText, u16, u32 } from "./espImageStructures.js";
|
||||
import {
|
||||
cString,
|
||||
cStringBytes,
|
||||
hex,
|
||||
offsetText,
|
||||
u16,
|
||||
u32,
|
||||
} from "./espImageStructures.js";
|
||||
|
||||
/*
|
||||
* The partition table, from ESP-IDF v6.1 `esp_flash_partitions.h`,
|
||||
@@ -36,6 +43,8 @@ export interface EspPartition {
|
||||
readonly size: number;
|
||||
/** The label up to its first NUL, or null when it is not UTF-8. */
|
||||
readonly label: string | null;
|
||||
/** The label's bytes up to its first NUL, in hex, when it is not UTF-8. */
|
||||
readonly labelHex: string | null;
|
||||
readonly encrypted: boolean;
|
||||
readonly readonly: boolean;
|
||||
/** Flag bits other than `encrypted` and `readonly`, which ESP-IDF ignores. */
|
||||
@@ -57,23 +66,14 @@ export interface EspPartitionTable {
|
||||
readonly limitations: readonly string[];
|
||||
}
|
||||
|
||||
const UTF8 = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
|
||||
|
||||
const label = (bytes: Uint8Array): string | null => {
|
||||
const end = bytes.indexOf(0);
|
||||
try {
|
||||
return UTF8.decode(end < 0 ? bytes : bytes.subarray(0, end));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
const partition = (
|
||||
entry: Uint8Array,
|
||||
index: number,
|
||||
entryOffset: number,
|
||||
): EspPartition => {
|
||||
const flags = u32(entry, 28);
|
||||
const labelBytes = entry.subarray(12, 28);
|
||||
const label = cString(labelBytes);
|
||||
return {
|
||||
index,
|
||||
entryOffset,
|
||||
@@ -81,7 +81,8 @@ const partition = (
|
||||
subtype: entry[3] ?? 0,
|
||||
offset: u32(entry, 4),
|
||||
size: u32(entry, 8),
|
||||
label: label(entry.subarray(12, 28)),
|
||||
label,
|
||||
labelHex: label === null ? hex(cStringBytes(labelBytes)) : null,
|
||||
encrypted: (flags & 1) !== 0,
|
||||
readonly: (flags & 2) !== 0,
|
||||
unknownFlags: (flags & ~KNOWN_FLAGS) >>> 0,
|
||||
@@ -89,12 +90,14 @@ const partition = (
|
||||
};
|
||||
|
||||
/**
|
||||
* Decode a table as ESP-IDF's bootloader verifies it: 32-byte entries with
|
||||
* the `0x50AA` magic, at most one `0xEBEB` MD5 entry over the entries before
|
||||
* it, and a terminator whose magic, type and subtype are all `0xFF`, which
|
||||
* cannot be the first entry. A structural fault means the bytes are not a
|
||||
* table; an MD5 mismatch, which the bootloader also rejects, is reported as
|
||||
* an integrity fact so that the declared entries stay visible.
|
||||
* Decode a table in the entry structure ESP-IDF's bootloader verifies:
|
||||
* 32-byte entries with the `0x50AA` magic, at most one `0xEBEB` MD5 entry
|
||||
* over the entries before it, and a terminator whose magic, type and subtype
|
||||
* are all `0xFF`. A structural fault means the bytes are not a table. An MD5
|
||||
* mismatch or a second MD5 entry, which the bootloader rejects, is reported
|
||||
* as an integrity fact so that the declared entries stay visible. ESP-IDF
|
||||
* accepts an empty table, but REA requires one entry before the terminator
|
||||
* so that erased flash is not read as a table.
|
||||
*/
|
||||
export const decodeEspPartitionTable = (
|
||||
bytes: Uint8Array,
|
||||
@@ -214,6 +217,9 @@ export const espPartitionSubtypeName = (
|
||||
const APP_TYPE = 0x00;
|
||||
const OTA_DATA_SUBTYPE = 0x00;
|
||||
const DATA_TYPE = 0x01;
|
||||
const BOOTLOADER_TYPE = 0x02;
|
||||
const PARTITION_TABLE_TYPE = 0x03;
|
||||
const PRIMARY_SUBTYPE = 0x00;
|
||||
const OTA_APP_SUBTYPE_FLAG = 0x10;
|
||||
const OTA_APP_SUBTYPE_MASK = 0x0f;
|
||||
|
||||
@@ -224,11 +230,77 @@ export const espOtaAppSlot = (part: EspPartition): number | null =>
|
||||
? part.subtype & OTA_APP_SUBTYPE_MASK
|
||||
: null;
|
||||
|
||||
export const isEspFactoryApp = (part: EspPartition): boolean =>
|
||||
const isEspFactoryApp = (part: EspPartition): boolean =>
|
||||
part.type === APP_TYPE && part.subtype === 0x00;
|
||||
|
||||
export const isEspOtaData = (part: EspPartition): boolean =>
|
||||
const isEspOtaData = (part: EspPartition): boolean =>
|
||||
part.type === DATA_TYPE && part.subtype === OTA_DATA_SUBTYPE;
|
||||
|
||||
export const isEspAppPartition = (part: EspPartition): boolean =>
|
||||
part.type === APP_TYPE;
|
||||
|
||||
/**
|
||||
* The region a primary `bootloader` or `partition_table` entry declares.
|
||||
* ESP-IDF 5.3 and later may declare them over the bootloader region and the
|
||||
* table sector, and `gen_esp32part.py` exempts them from its rule that
|
||||
* partitions follow the table.
|
||||
*/
|
||||
export const espPrimaryLayoutRole = (
|
||||
part: EspPartition,
|
||||
): "bootloader" | "partition-table" | undefined => {
|
||||
if (part.subtype !== PRIMARY_SUBTYPE) return undefined;
|
||||
if (part.type === BOOTLOADER_TYPE) return "bootloader";
|
||||
return part.type === PARTITION_TABLE_TYPE ? "partition-table" : undefined;
|
||||
};
|
||||
|
||||
/** The partitions the bootloader keeps from a table, and the OTA app count. */
|
||||
export interface EspBootPartitions {
|
||||
readonly factory: EspPartition | undefined;
|
||||
readonly otadata: EspPartition | undefined;
|
||||
/** Each OTA slot's partition, indexed by subtype. */
|
||||
readonly otaSlots: ReadonlyMap<number, EspPartition>;
|
||||
/** Every OTA app entry, duplicates included, as `bs->app_count` counts. */
|
||||
readonly otaAppCount: number;
|
||||
readonly limitations: readonly string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the table as `bootloader_utility_load_partition_table` does: a later
|
||||
* factory, otadata or same-subtype OTA entry replaces an earlier one, and
|
||||
* every OTA app entry adds to the app count.
|
||||
*/
|
||||
export const espBootPartitions = (
|
||||
partitions: readonly EspPartition[],
|
||||
): EspBootPartitions => {
|
||||
let factory: EspPartition | undefined;
|
||||
let otadata: EspPartition | undefined;
|
||||
let otaAppCount = 0;
|
||||
const otaSlots = new Map<number, EspPartition>();
|
||||
const roles = new Map<string, EspPartition[]>();
|
||||
for (const part of partitions) {
|
||||
const slot = espOtaAppSlot(part);
|
||||
const role = isEspFactoryApp(part)
|
||||
? "factory"
|
||||
: isEspOtaData(part)
|
||||
? "otadata"
|
||||
: slot === null
|
||||
? undefined
|
||||
: `ota_${String(slot)}`;
|
||||
if (role === undefined) continue;
|
||||
roles.set(role, [...(roles.get(role) ?? []), part]);
|
||||
if (role === "factory") factory = part;
|
||||
else if (role === "otadata") otadata = part;
|
||||
else if (slot !== null) {
|
||||
otaSlots.set(slot, part);
|
||||
otaAppCount += 1;
|
||||
}
|
||||
}
|
||||
const limitations = [...roles].flatMap(([role, parts]) =>
|
||||
parts.length < 2
|
||||
? []
|
||||
: [
|
||||
`The table declares ${role} ${String(parts.length)} times (partitions ${parts.map(({ index }) => `#${String(index)}`).join(", ")}); ESP-IDF's bootloader uses the last.`,
|
||||
],
|
||||
);
|
||||
return { factory, otadata, otaSlots, otaAppCount, limitations };
|
||||
};
|
||||
@@ -0,0 +1,124 @@
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
ESP32C3_BOOTLOADER_SEGMENTS,
|
||||
espImage,
|
||||
} from "../../src/domain/firmware/espImage.fixture.js";
|
||||
import {
|
||||
TWO_OTA_PARTITIONS,
|
||||
espFlashDump,
|
||||
partitionTable,
|
||||
} from "../../src/domain/firmware/espFlash.fixture.js";
|
||||
import { artifactComparisonResultSchema } from "../../src/domain/artifactComparison.js";
|
||||
import { artifactInspectionResultSchema } from "../../src/domain/artifactInspection.js";
|
||||
import { artifactInventoryResultSchema } from "../../src/domain/artifactGraph.js";
|
||||
import { parseEvidence, type Evidence } from "../../src/domain/evidence.js";
|
||||
import { connectLocalToolsMcp } from "../fixtures/localToolsMcp.js";
|
||||
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
|
||||
|
||||
const BOOTLOADER = {
|
||||
at: 0,
|
||||
bytes: espImage({ segments: ESP32C3_BOOTLOADER_SEGMENTS }),
|
||||
};
|
||||
|
||||
/** A table without an MD5 whose first entry has these label bytes. */
|
||||
const labelBytes = (bytes: readonly number[]): Buffer => {
|
||||
const table = Buffer.from(
|
||||
partitionTable(TWO_OTA_PARTITIONS, { md5: "absent" }),
|
||||
);
|
||||
table.fill(0, 12, 28);
|
||||
Buffer.from(bytes).copy(table, 12);
|
||||
return table;
|
||||
};
|
||||
|
||||
it("reports an ESP partition whose declaration alone changed", async () => {
|
||||
const root = await createTestTempDirectory("rea-esp-flash-comparison-");
|
||||
const { call } = await connectLocalToolsMcp();
|
||||
const inspect = async (name: string, table: Buffer): Promise<Evidence> => {
|
||||
const path = join(root, name);
|
||||
await writeFile(path, espFlashDump({ bootloader: BOOTLOADER, table }));
|
||||
const opened = await call("open_binary", { path });
|
||||
expect(opened.isError, JSON.stringify(opened)).not.toBe(true);
|
||||
const response = await call("inspect_artifact", {});
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
const inventory = artifactInspectionResultSchema
|
||||
.parse(parseEvidence(response.structuredContent).normalized_result)
|
||||
.substeps.find(
|
||||
({ operation }) => operation === "inventory_artifact",
|
||||
)?.evidence;
|
||||
if (inventory === undefined) throw new Error("Missing inventory Evidence");
|
||||
return parseEvidence(inventory);
|
||||
};
|
||||
const compare = async (left: Evidence, right: Evidence) => {
|
||||
const response = await call("compare_artifacts", { left, right });
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
return artifactComparisonResultSchema.parse(
|
||||
parseEvidence(response.structuredContent).normalized_result,
|
||||
);
|
||||
};
|
||||
const occurrence = (evidence: Evidence, path: string) => {
|
||||
const found = artifactInventoryResultSchema
|
||||
.parse(evidence.normalized_result)
|
||||
.occurrences.find(({ logical_path }) => logical_path === path);
|
||||
if (found === undefined) throw new Error(`Missing ${path}`);
|
||||
return found;
|
||||
};
|
||||
const changedPaths = (result: { changes: { logical_path: string }[] }) =>
|
||||
result.changes
|
||||
.map(({ logical_path }) => logical_path)
|
||||
.filter((path) => path !== ".")
|
||||
.sort();
|
||||
|
||||
// The same partition bytes, with nvs declared read-only.
|
||||
const readonly = TWO_OTA_PARTITIONS.map((part) =>
|
||||
part.label === "nvs" ? { ...part, flags: 0x2 } : part,
|
||||
);
|
||||
const left = await inspect("flash.bin", partitionTable(TWO_OTA_PARTITIONS));
|
||||
const right = await inspect("readonly.bin", partitionTable(readonly));
|
||||
const leftNvs = occurrence(left, "partitions/nvs.bin");
|
||||
const rightNvs = occurrence(right, "partitions/nvs.bin");
|
||||
expect(rightNvs.artifact_id).toBe(leftNvs.artifact_id);
|
||||
expect(leftNvs.format_metadata).toMatchObject({ readonly: false });
|
||||
expect(rightNvs.format_metadata).toMatchObject({ readonly: true });
|
||||
const changed = await compare(left, right);
|
||||
expect(
|
||||
changed.changes.find(
|
||||
({ logical_path }) => logical_path === "partitions/nvs.bin",
|
||||
),
|
||||
).toEqual({
|
||||
classification: "changed",
|
||||
logical_path: "partitions/nvs.bin",
|
||||
dimensions: ["metadata"],
|
||||
left_occurrence_id: leftNvs.occurrence_id,
|
||||
right_occurrence_id: rightNvs.occurrence_id,
|
||||
left_artifact_id: leftNvs.artifact_id,
|
||||
right_artifact_id: rightNvs.artifact_id,
|
||||
evidence_links: [left.evidence_id, right.evidence_id],
|
||||
});
|
||||
// Only the table's bytes and the declaration it gives nvs changed.
|
||||
expect(changedPaths(changed)).toEqual([
|
||||
"partition-table.bin",
|
||||
"partitions/nvs.bin",
|
||||
]);
|
||||
|
||||
// Labels that are not UTF-8 compare by their bytes.
|
||||
const first = await inspect("label-a.bin", labelBytes([0xff, 0x01]));
|
||||
const second = await inspect("label-b.bin", labelBytes([0xfe, 0x01]));
|
||||
expect(occurrence(first, "partitions/0.bin").format_metadata).toMatchObject({
|
||||
label: null,
|
||||
label_hex: "ff01",
|
||||
});
|
||||
expect(changedPaths(await compare(first, second))).toEqual([
|
||||
"partition-table.bin",
|
||||
"partitions/0.bin",
|
||||
]);
|
||||
|
||||
// Equal declarations leave every partition unchanged.
|
||||
const same = await compare(
|
||||
left,
|
||||
await inspect("copy.bin", partitionTable(TWO_OTA_PARTITIONS)),
|
||||
);
|
||||
expect(same.changes).toEqual([]);
|
||||
});
|
||||
@@ -594,6 +594,22 @@ describe("binary target I/O: ESP flash layouts", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps refusing an ESP8266 dump that has an ESP-IDF-style table", async () => {
|
||||
const directory = await createTestTempDirectory("rea-esp8266-flash-");
|
||||
const dump = join(directory, "esp8266-flash.bin");
|
||||
await writeFile(
|
||||
dump,
|
||||
espFlashDump({ bootloader: { at: 0, bytes: esp8266Image() } }),
|
||||
);
|
||||
expect(await parseBinaryTarget(dump)).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
constraint: "file_format_rejected",
|
||||
reason: ESP8266_REFUSAL,
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("reads a selected flash dump with caller inputs and refuses bad pairings", async () => {
|
||||
const directory = await createTestTempDirectory("rea-esp-flash-select-");
|
||||
const esp32 = espImage({
|
||||
|
||||
Reference in new issue
Block a user