Files
reverse-engineering-anything/src/domain/artifactGraph.ts
T
bartman081523andmorluto d380dc2409 feat(uefi): add pure-TypeScript UEFI firmware analysis tools (#12342)
* feat(uefi): add pure-TS UEFI firmware volume/FFS parser with inspect/extract tools

Add a dependency-free rea-uefi provider that maps AMI/Aptio flash images:
- inspect_uefi_firmware: aligned _FVH discovery, volume-relative FFS walking,
  guided section/compression classification, conservative PSP-directory probing,
  embedded-volume rescan and leftover-region classification; incremental SHA-256
  and explicit integrity notes.
- extract_uefi_firmware: whole-image/volume/FFS-file/section extraction with
  LZMA-custom/LZMAF86 and AMD-zlib decoding, SafeOutputTree staging, derivations
  and extraction_gaps.
- UefiArtifactReader + 'uefi' format wiring across inventory/domain/contracts
  so parsed volumes and extracted PE32/TE files hand off to deep-provider
  analysis (open_binary -> Ghidra).

* feat(uefi): add inspect_uefi_nvram for AMI NVRAM variable stores

Add a pure-TypeScript tool that maps the vendor AMI NVRAM variable stores
(factory-default "StdDefaults" and peers) inside a flash image:
- scan the raw image for NVAR entry chains ("NVAR" + u16 total size +
  u32 attribute 0x8[23]FFFFFF + state + NUL-terminated name + data),
- recurse into a container entry whose data is itself an NVAR chain, so the
  StdDefaults setup variables surface as depth/parent_offset rows,
- report attributes, state, data range, SHA-256 and trailing padding per entry.

Verified against B550 AORUS Elite V2 FId: one StdDefaults store at 0x37090 with
the 11 expected nested variables (Setup, PlatformLang, AMITSESetup, XhciDID,
GcSensorVarName, PCI_COMMON, SioSetupData, UsbSupport, NetworkStackVar,
SecureBootSetup, Setup). Exposed as MCP tool and inspect-uefi-nvram CLI command.

* fix(uefi): address review on checksums, compression, guided sections, NVRAM bounds

- FFS file checksum: verify the two's-complement invariant (payload sum +
  stored byte == 0 mod 256) instead of comparing sums, and encode the correct
  complement in the fixture builder.
- Compression sections (type 0x01): use the 9-byte header and classify EFI
  standard compression instead of 'none', so extraction reports a gap rather
  than publishing compressed bytes as a raw section.
- GUID-defined sections: derive GUID/data-offset/attributes from the parsed
  common header size (compact or extended) and reject out-of-range payload
  offsets.
- AMI NVRAM: bound recursive chains to the enclosing container's data end and
  report partial coverage when a valid nested chain is cut by max_depth.

* fix(uefi): bound decode memory, serialize operations, enforce deadline, mark truncation

- LZMA worker: enforce max_output_bytes while decoding through a bounded output
  sink, so an oversized stream is rejected during expansion instead of after a
  full decompression plus a second Buffer copy.
- Provider: serialize admitted operations (the promised single decode-worker
  lane must not become one worker per overlapping call) and apply one
  whole-request deadline that covers every decode stage, not a fresh per-stream
  timer. close() drains the queue and the resource scope.
- Scan: track truncated sections of known type and report coverage partial
  instead of complete when a section declares more bytes than are present.

* fix(uefi): cancellable queue, timeout surfacing, availability and reader omissions

- Queue: check the caller signal before enqueue and when a job reaches the head,
  and race the queued promise against abort so a cancelled wait returns
  immediately instead of blocking behind the active request.
- Deadline: track the internal deadline and return AnalysisTimeoutError instead
  of the AnalysisCancelledError that runOwnedCommand reports for any external
  abort, preserving the real failure reason for CLI/MCP callers.
- CapabilityInventory: treat inspect_uefi_nvram like the other bundled rea-uefi
  tools so discovery does not demand Unblob for it.
- UefiArtifactReader: report image truncation through onOmission so inventory
  coverage reflects the unread suffix even when no volume was found.

* fix(uefi): limit early cancellation to unstarted jobs and preserve timeout cleanup

- #raceWithAbort only returns early for jobs that have not started; once a
  request is running, await it so worker cleanup and completed output are
  preserved instead of being discarded by a bare cancellation.
- Reclassify a deadline-induced cancellation as AnalysisTimeoutError while
  keeping its cause and cleanup observations, and never override a different
  primary failure.

* docs(licenses): record UEFI decoder dependency notices

* fix(uefi): preserve stream provenance and bound firmware analysis

* feat(uefi): decode LZMAF86 guided sections

* fix(uefi): validate publication before commit and order decoder cleanup

* fix(uefi): preserve unsupported volumes and extraction gaps

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-11-19 21:19:35 +00:00

346 lines
11 KiB
TypeScript

import { completeOrUnknownCoverageSchema } from "./coverage.js";
import {
artifactDfuCrc32PairSchema,
artifactIntegrityClaimSchema,
artifactSha256PairSchema,
artifactMd5PairSchema,
} from "./artifactIntegrity.js";
import { artifactFormatMetadataSchema } from "./artifactFormatMetadata.js";
import { z } from "zod";
import { artifactReaderLimitsSchema } from "./artifactReaderLimits.js";
import { prefixedDigestSchema } from "./../domain/digests.js";
import { inesInventorySchema } from "./inesContainer.js";
import { ZIP_APPLICATION_PACKAGE_FORMATS } from "./zipPackageFormat.js";
/** Entry kinds or subtrees deliberately excluded by an artifact traversal. */
export const artifactTraversalOmissionKindSchema = z.enum([
"fifo",
"socket",
"block-device",
"character-device",
"unsupported",
"directory-descendants",
]);
export type ArtifactTraversalOmissionKind = z.infer<
typeof artifactTraversalOmissionKindSchema
>;
const sha256Schema = z.string().regex(/^[a-f0-9]{64}$/u);
const artifactIdSchema = prefixedDigestSchema("art");
const occurrenceIdSchema = prefixedDigestSchema("occ");
const edgeIdSchema = prefixedDigestSchema("edge");
const manifestIdSchema = prefixedDigestSchema("agm");
const extractionIdSchema = prefixedDigestSchema("aex");
const contradictionIdSchema = prefixedDigestSchema("ic");
const boundedRelativePathSchema = z
.string()
.min(1)
.refine(
(path) =>
!path.startsWith("/") &&
path.split("/").every((part) => part !== "" && part !== ".."),
"Expected a normalized relative POSIX path without traversal",
);
/** Closed artifact categories used to traverse application containers. */
const artifactKindSchema = z.enum([
"container",
"executable",
"dynamic-library",
"framework",
"native-addon",
"javascript",
"source-map",
"plist",
"entitlements",
"resource",
"universal-slice",
"package-metadata",
"unknown",
]);
/** Recognized storage or application formats; unknown bytes remain explicit. */
export const ARTIFACT_FORMAT_IDS = [
"dotnet-bundle",
"bun-standalone",
"deno-standalone",
"dmg",
"zip",
"pkg",
"asar",
"ines",
"godot-pck",
"unityfs",
"wxapkg",
"uimage",
"android-boot-image",
"android-ota-payload",
"uefi",
"javascript-bundle",
"source-map",
"mach-o-universal",
"mach-o",
"elf",
"pe",
"dos-mz",
"dos-com",
"raw-binary",
"gamecube-dol",
"hopper",
"analysis-database",
...ZIP_APPLICATION_PACKAGE_FORMATS,
"plist",
"entitlements",
"directory",
"file",
"pyinstaller",
"node-sea",
"intel-hex",
"srec",
"esp-image",
"uf2",
"dfuse",
"esp-flash-image",
"fit",
"ntfs-mft",
"unknown",
] as const;
export type ArtifactFormatId = (typeof ARTIFACT_FORMAT_IDS)[number];
const artifactFormatSchema = z.enum(ARTIFACT_FORMAT_IDS);
/** Formats established from bytes or a virtual directory tree, independent of path roles. */
const artifactContentFormatSchema = artifactFormatSchema.extract([
"ines",
"zip",
"godot-pck",
"unityfs",
"wxapkg",
"uimage",
"android-boot-image",
"android-ota-payload",
"uefi",
"intel-hex",
"srec",
"esp-image",
"uf2",
"dfuse",
"fit",
"ntfs-mft",
"mach-o-universal",
"mach-o",
"elf",
"pe",
"dos-mz",
"plist",
"directory",
"file",
"unknown",
]);
/** File byte interval for an artifact derived without writing it to disk. */
const artifactByteRangeSchema = z.object({
offset: z.int().min(0),
length: z.int().min(0),
});
/** External command provenance, including observable side effects. */
const artifactCommandSchema = z.object({
tool: z.string().min(1),
arguments: z.array(z.string()),
tool_version: z.string().nullable(),
executable_sha256: sha256Schema.nullable(),
exit_code: z.int().nullable(),
effects: z.array(z.enum(["read", "write", "mount"])),
});
/** Parsed artifact-producing command provenance. */
export type ArtifactCommand = z.infer<typeof artifactCommandSchema>;
/** One content-addressed application artifact in deterministic manifest order. */
export const artifactNodeSchema = z.object({
artifact_id: artifactIdSchema,
format: artifactContentFormatSchema,
sha256: sha256Schema,
size: z.int().min(0).max(Number.MAX_SAFE_INTEGER),
media_type: z.string().min(1).nullable(),
architecture: z.string().min(1).nullable(),
content_state: z.enum(["materialized", "embedded", "virtual"]),
limitations: z.array(z.string()),
});
/** One path or virtual location where an artifact occurs in its parent. */
export const artifactOccurrenceSchema = z.object({
occurrence_id: occurrenceIdSchema,
artifact_id: artifactIdSchema.nullable(),
parent_occurrence_id: occurrenceIdSchema.nullable(),
/** Producer coverage of this occurrence's modeled relationships. Historical omission is unknown. */
relationship_coverage: completeOrUnknownCoverageSchema.exactOptional(),
logical_path: boundedRelativePathSchema,
entry_kind: z.enum(["file", "directory", "symlink", "slice", "metadata"]),
artifact_kind: artifactKindSchema,
artifact_format: artifactFormatSchema,
declared_size: z.int().min(0).max(Number.MAX_SAFE_INTEGER).nullable(),
compressed_size: z.int().min(0).max(Number.MAX_SAFE_INTEGER).nullable(),
executable: z.boolean(),
encrypted: z.boolean(),
/** Reader-observed ASAR storage; omission leaves historical storage unknown. */
unpacked: z.boolean().exactOptional(),
hash_status: z.enum(["verified", "mismatched", "not-hashed", "unavailable"]),
source_location: artifactByteRangeSchema.nullable(),
/** Exact reader-source path or opaque key, including an observed empty key. */
source_path: z.string().exactOptional(),
declared_integrity: artifactIntegrityClaimSchema.exactOptional(),
format_metadata: artifactFormatMetadataSchema.exactOptional(),
limitations: z.array(z.string()),
});
/** Observed bytes that contradict a container's declared integrity identity. */
const integrityContradictionBaseSchema = z.object({
contradiction_id: contradictionIdSchema,
occurrence_id: occurrenceIdSchema,
parent_artifact_id: artifactIdSchema,
logical_path: boundedRelativePathSchema,
entry_kind: z.enum(["file", "slice"]),
unpacked: z.boolean(),
trust: z.literal("observed-untrusted"),
provenance: z.literal("container-integrity-metadata-versus-observed-bytes"),
limitations: z.array(z.string()).min(1),
});
export const integrityContradictionSchema = z.union([
integrityContradictionBaseSchema.extend(artifactSha256PairSchema.shape),
integrityContradictionBaseSchema.extend(artifactMd5PairSchema.shape),
integrityContradictionBaseSchema.extend(artifactDfuCrc32PairSchema.shape),
]);
/**
* Parent-child derivation relation between two content-addressed artifacts.
* The child artifact and its logical path are those of the edge's occurrence.
*/
export const artifactEdgeSchema = z.object({
edge_id: edgeIdSchema,
ordinal: z.int().min(0),
parent_artifact_id: artifactIdSchema,
relation: z.enum([
"contains",
"extracts",
"slice-of",
"embeds",
"loads",
"maps-source",
"derived-from",
]),
occurrence_id: occurrenceIdSchema,
producer: artifactCommandSchema.nullable(),
});
/** Stable cryptographic commitment and counts for a complete artifact graph. */
export const artifactGraphManifestSchema = z.object({
manifest_id: manifestIdSchema,
root_artifact_id: artifactIdSchema,
root_sha256: sha256Schema,
root_format: artifactFormatSchema,
graph_sha256: sha256Schema,
node_count: z.int().min(1),
occurrence_count: z.int().min(1),
edge_count: z.int().min(0),
});
/** Producer-owned traversal coverage; content certainty remains on each occurrence. */
export const artifactInventoryCoverageSchema = z.strictObject({
scope: z.enum(["root", "container-entries"]),
status: completeOrUnknownCoverageSchema,
});
export type ArtifactInventoryCoverage = z.infer<
typeof artifactInventoryCoverageSchema
>;
/** Current inventory observation containing the stable artifact graph. */
export const artifactInventoryResultSchema = z.strictObject({
effective_limits: artifactReaderLimitsSchema
.exactOptional()
.describe(
"Accepted resource policy when selected; each format group applies only if that format is encountered. Not measured process memory or confirmation every reader ran.",
),
manifest: artifactGraphManifestSchema,
nodes: z.array(artifactNodeSchema),
occurrences: z.array(artifactOccurrenceSchema),
edges: z.array(artifactEdgeSchema),
provenance: z.array(artifactCommandSchema),
ines: inesInventorySchema.exactOptional(),
integrity_contradictions: z.array(integrityContradictionSchema),
coverage: artifactInventoryCoverageSchema,
limitations: z.array(z.string()),
});
/** One safely materialized artifact beneath caller-approved output root. */
const extractedArtifactSchema = z.object({
artifact_id: artifactIdSchema,
relative_path: boundedRelativePathSchema,
sha256: sha256Schema,
bytes_written: z.int().min(0).max(Number.MAX_SAFE_INTEGER),
created: z.boolean(),
});
/** Path-independent commitment to one selected extraction operation. */
const artifactExtractionManifestSchema = z.object({
extraction_id: extractionIdSchema,
source_manifest_id: manifestIdSchema,
selected_occurrence_ids: z.array(occurrenceIdSchema),
files_sha256: sha256Schema,
output_root_alias: z.literal("$OUTPUT_ROOT"),
});
/** Safe extraction observation with containment and cleanup verification. */
export const artifactExtractionResultSchema = z.object({
effective_limits: artifactReaderLimitsSchema.exactOptional(),
manifest: artifactGraphManifestSchema,
extraction_manifest: artifactExtractionManifestSchema,
output_root: z.string().min(1),
artifacts: z.array(extractedArtifactSchema),
containment_verified: z.boolean(),
cleanup: z.object({
attempted: z.boolean(),
verified: z.boolean(),
residual_paths: z.array(boundedRelativePathSchema),
}),
coverage: artifactInventoryCoverageSchema,
provenance: z.array(artifactCommandSchema),
integrity_contradictions: z.array(integrityContradictionSchema),
limitations: z.array(z.string()),
});
/** Parsed artifact graph node. */
export type ArtifactNode = z.infer<typeof artifactNodeSchema>;
/** Parsed artifact derivation edge. */
export type ArtifactEdge = z.infer<typeof artifactEdgeSchema>;
/** Parsed artifact occurrence. */
export type ArtifactOccurrence = z.infer<typeof artifactOccurrenceSchema>;
/** Read relationship coverage without changing authenticated historical rows. */
export const artifactOccurrenceRelationshipCoverage = (
occurrence: ArtifactOccurrence,
) => occurrence.relationship_coverage ?? "unknown";
export type IntegrityContradiction = z.infer<
typeof integrityContradictionSchema
>;
/** Parsed deterministic artifact graph manifest. */
export type ArtifactGraphManifest = z.infer<typeof artifactGraphManifestSchema>;
/** Parsed paginated inventory result. */
export type ArtifactInventoryResult = z.infer<
typeof artifactInventoryResultSchema
>;
/** Parsed paginated extraction result. */
export type ArtifactExtractionResult = z.infer<
typeof artifactExtractionResultSchema
>;