mirror of
https://github.com/morluto/rea.git
synced 2026-11-14 18:52:48 +00:00
feat(artifacts): inventory flat installer packages
A .pkg inventory recorded only the root hash. Flat installer packages
are xar archives, and their install scripts and payloads are what an
analyst needs to review.
A pure TypeScript xar reader, which also runs off macOS:
- reads the TOC (zlib XML) and verifies the TOC checksum;
- walks nested product-package members, such as Component.pkg/;
- decodes raw and zlib ("x-gzip") members, verifying each extracted
SHA-1/MD5 checksum as the bytes stream;
- rejects unsafe member names.
Gzip-cpio Scripts and Payload members are expanded as nested containers
by a streaming odc/newc cpio reader. It reads members once, in archive
order, and holds at most one chunk. pbzx (xz) payloads are recorded
with a limitation that names pkgutil --expand-full. Unsupported member
encodings, traversal, and truncation fail with typed reasons.
@xmldom/xmldom (already installed through plist) becomes a direct,
pinned dependency for TOC parsing.
verify:installer-package builds component, product, and pbzx packages
with pkgbuild and productbuild. It checks the expanded Payload against
`pkgutil --payload-files`, script and payload content digests, product
nesting, and CLI/MCP parity, and runs in the macOS CI job.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kaoru0822andClaude Opus 5.5 committed
1 parent
dcde958450
commit
54dd1a7604
22 files changed
+1396
-24
No files matched your search
@@ -303,6 +303,7 @@ jobs:
|
||||
- run: npm run verify:keyed-archive
|
||||
- run: npm run verify:asset-catalog
|
||||
- run: npm run verify:macos-bundle
|
||||
- run: npm run verify:installer-package
|
||||
- name: Native compiled NIB hierarchy regressions
|
||||
run: npm run test:focused -- tests/boundary/filesystem/nibNativeHierarchy.test.ts tests/boundary/filesystem/interfaceBuilderBundle.test.ts
|
||||
- name: Real binary metadata and Interface Builder integration
|
||||
|
||||
@@ -423,7 +423,7 @@ REA supports native application, JavaScript, Electron, .NET, and browser investi
|
||||
Static Android APK inspection supports Linux and macOS; the current metadata bridge is verified on macOS arm64 with headless JADX; see [Android analysis](docs/android-analysis.md) for its separate prerequisites and coverage.
|
||||
|
||||
- **Native binaries:** Open Mach-O, ELF, PE, and Mac `.app` targets through a selected deep provider. Hopper and Ghidra cover broad inventory and function analysis; the IDA adapter supplies its documented read-only function/string operations. Hopper also accepts `.hop` databases and supports annotations.
|
||||
- **Packages and resources:** Inspect directories, ZIP, APK, IPA, ASAR, plists, compiled Interface Builder files, and Apple asset catalogs. Artifact requests name the input and requested extraction or traversal directly; macOS DMG traversal also requires the host's native mounting support.
|
||||
- **Packages and resources:** Inspect directories, ZIP, APK, IPA, ASAR, PKG installers (including their install scripts and payloads), plists, compiled Interface Builder files, and Apple asset catalogs. Artifact requests name the input and requested extraction or traversal directly; macOS DMG traversal also requires the host's native mounting support.
|
||||
- **JavaScript and Electron:** Map modules, imports, source maps, routes, IPC channels, storage, and native add-ons without running the app. Compare builds and trace a feature across the recovered graph. Dynamic and ambiguous relationships remain unresolved. See [JavaScript application workflows](docs/javascript-application-workflows.md).
|
||||
- **Websites:** Inspect a selected page in an existing Chrome-family browser. Capture page structure, network metadata, script evidence, and screenshots requested by the call. Passive observation does not navigate or execute page JavaScript. See [browser observation](docs/browser-observation.md).
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
"client": "2.3.1",
|
||||
"core": "2.3.1"
|
||||
},
|
||||
"skill_version": "27"
|
||||
"skill_version": "28"
|
||||
},
|
||||
"tools": {
|
||||
"total": 131,
|
||||
@@ -669,9 +669,9 @@
|
||||
"mcp_prompts": 6
|
||||
},
|
||||
"digests": {
|
||||
"tools_sha256": "d0b9e5c99d6a2c9bdf065429db901231326a7de5910f2d878cee124891dba57e",
|
||||
"tools_sha256": "620bc21113bcc2ed5775f3bdb0a59a3d6208e8aec9d918f220b5c7c85e9e4dd6",
|
||||
"prompts_sha256": "ef2de8814390b340623c32001610aa2709d90bed51537b6e562ab3abf144ab4d",
|
||||
"combined_sha256": "188d726867808cd63764625b93d89e2936af29d9943dbc0e5d22044294521e1d",
|
||||
"combined_sha256": "d18386ad8f46e7035cfa49f075c42e5c143f5b3f05c9459eb2d1c23c11112871",
|
||||
"providers_sha256": "a7d885531ee25d42c51385444faec544d34cd7eba6250a7dc75c4faedd4454a3"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -71,6 +71,14 @@ an XML graph golden. `verify:asset-catalog` compiles source-owned colors with
|
||||
`actool`, invokes real `assetutil`, then checks CLI/MCP results, exact catalog
|
||||
digest, every raw metadata field, pagination and malformed input rejection.
|
||||
Neither artifact workflow requires Hopper or launches it. Both run in macOS CI.
|
||||
`verify:installer-package` needs only macOS's `pkgbuild`, `productbuild` and
|
||||
`pkgutil`. It builds a component package with install scripts, a product package
|
||||
around it, and a pbzx (`--compression latest`) package. It checks that
|
||||
`inspect-artifact` expands Scripts and Payload with exact content digests, and
|
||||
that the expanded Payload members equal Apple's `pkgutil --payload-files`
|
||||
listing. It also checks that the product package nests its component, that the
|
||||
pbzx payload is recorded with a limitation instead of being expanded, and that
|
||||
CLI and stdio MCP results agree.
|
||||
`verify:macos-bundle` needs only macOS with Command Line Tools. It compiles a
|
||||
source-owned app with `clang`: a versioned framework, XPC services, an app
|
||||
extension, a login item, a privileged helper, launchd plists, and a helper tool,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"manifest_id": "ecm_4bb7d6acd148fffbd7e696cd56b22ed7a031fcb4b51473170058b4a9e902ee44",
|
||||
"manifest_id": "ecm_cba6ab6b66603cd59aa60b327d67efe09bb9d004c73d1c7e8e368d86887c686e",
|
||||
"verifier": {
|
||||
"id": "rea-managed-conformance",
|
||||
"version": "1"
|
||||
@@ -13,7 +13,7 @@
|
||||
"skill_digests": [
|
||||
{
|
||||
"skill_id": "reverse-engineer-anything",
|
||||
"sha256": "1738df3db4b39d4967e40cc147d24393fd8e1e21fc4f7b7b45da30ce8880c2a9"
|
||||
"sha256": "21bd8b88040d593111905bb47863f5145690c4d5582e0975bab76d2b07fb25b9"
|
||||
}
|
||||
],
|
||||
"claims": [
|
||||
|
||||
Generated
+1
@@ -17,6 +17,7 @@
|
||||
"@lydell/node-pty": "1.1.0",
|
||||
"@modelcontextprotocol/client": "2.3.1",
|
||||
"@modelcontextprotocol/server": "2.3.1",
|
||||
"@xmldom/xmldom": "0.9.12",
|
||||
"@xterm/addon-serialize": "0.14.0",
|
||||
"@xterm/headless": "6.0.0",
|
||||
"@zip.js/zip.js": "2.8.26",
|
||||
|
||||
@@ -123,6 +123,7 @@
|
||||
"verify:apple-dispatch": "npm run build:cached && node scripts/verify-apple-dispatch.mjs",
|
||||
"verify:native-ui": "npm run build:cached && node scripts/verify-native-ui.mjs",
|
||||
"verify:keyed-archive": "npm run build:cached && node scripts/verify-keyed-archive.mjs",
|
||||
"verify:installer-package": "npm run build:cached && node scripts/verify-installer-package.mjs",
|
||||
"verify:macos-bundle": "npm run build:cached && node scripts/verify-macos-bundle.mjs",
|
||||
"verify:ghidra": "npm run build:cached && node scripts/verify-real-ghidra.mjs",
|
||||
"verify:ida": "npm run build:cached && node scripts/verify-real-ida.mjs",
|
||||
@@ -204,6 +205,7 @@
|
||||
"@lydell/node-pty": "1.1.0",
|
||||
"@modelcontextprotocol/client": "2.3.1",
|
||||
"@modelcontextprotocol/server": "2.3.1",
|
||||
"@xmldom/xmldom": "0.9.12",
|
||||
"@xterm/addon-serialize": "0.14.0",
|
||||
"@xterm/headless": "6.0.0",
|
||||
"@zip.js/zip.js": "2.8.26",
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { chmod, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import {
|
||||
artifactCli,
|
||||
artifactMcpResult,
|
||||
withArtifactMcp,
|
||||
} from "./lib/artifact-e2e.mjs";
|
||||
|
||||
const exec = promisify(execFile);
|
||||
|
||||
if (process.platform !== "darwin")
|
||||
throw new Error(
|
||||
"Installer package verification requires macOS pkgbuild, productbuild and pkgutil",
|
||||
);
|
||||
for (const tool of [
|
||||
"/usr/bin/pkgbuild",
|
||||
"/usr/bin/productbuild",
|
||||
"/usr/sbin/pkgutil",
|
||||
])
|
||||
try {
|
||||
await exec(tool, ["--help"]);
|
||||
} catch (cause) {
|
||||
if (cause?.code === "ENOENT")
|
||||
throw new Error(`Installer package verification requires ${tool}`, {
|
||||
cause,
|
||||
});
|
||||
}
|
||||
|
||||
const root = await mkdtemp(join(tmpdir(), "rea-installer-package-"));
|
||||
const sha256 = (text) => createHash("sha256").update(text).digest("hex");
|
||||
|
||||
/** Inventory occurrences keyed by logical path, with each node's content digest. */
|
||||
const inventoryOf = (inspection) => {
|
||||
const inventory = inspection.substeps[0].evidence.normalized_result;
|
||||
const nodes = new Map(
|
||||
inventory.nodes.map((node) => [node.artifact_id, node]),
|
||||
);
|
||||
return {
|
||||
inventory,
|
||||
entries: new Map(
|
||||
inventory.occurrences.map((occurrence) => [
|
||||
occurrence.logical_path,
|
||||
{ ...occurrence, sha256: nodes.get(occurrence.artifact_id)?.sha256 },
|
||||
]),
|
||||
),
|
||||
};
|
||||
};
|
||||
|
||||
/** Apple's Bom listing of a component package's payload, without the root. */
|
||||
const bomPaths = async (path) =>
|
||||
(await exec("/usr/sbin/pkgutil", ["--payload-files", path])).stdout
|
||||
.split("\n")
|
||||
.filter((line) => line.length > 0 && line !== ".")
|
||||
.map((line) => line.replace(/^\.\//u, ""))
|
||||
.sort();
|
||||
|
||||
try {
|
||||
const payload = join(root, "payload");
|
||||
const scripts = join(root, "scripts");
|
||||
await mkdir(join(payload, "usr/local/rea"), { recursive: true });
|
||||
await mkdir(scripts);
|
||||
await writeFile(join(payload, "usr/local/rea/hello.txt"), "hello\n");
|
||||
for (const name of ["preinstall", "postinstall"]) {
|
||||
await writeFile(join(scripts, name), `#!/bin/sh\necho ${name}\n`);
|
||||
await chmod(join(scripts, name), 0o755);
|
||||
}
|
||||
const component = join(root, "Component.pkg");
|
||||
const build = (output, extra) =>
|
||||
exec("/usr/bin/pkgbuild", [
|
||||
"--quiet",
|
||||
"--root",
|
||||
payload,
|
||||
"--identifier",
|
||||
"com.example.rea.installer",
|
||||
"--version",
|
||||
"1.0",
|
||||
"--install-location",
|
||||
"/",
|
||||
...extra,
|
||||
output,
|
||||
]);
|
||||
await build(component, ["--scripts", scripts]);
|
||||
const product = join(root, "Product.pkg");
|
||||
await exec("/usr/bin/productbuild", [
|
||||
"--quiet",
|
||||
"--package",
|
||||
component,
|
||||
product,
|
||||
]);
|
||||
const pbzx = join(root, "Latest.pkg");
|
||||
await build(pbzx, ["--compression", "latest", "--min-os-version", "13.0"]);
|
||||
|
||||
const componentInventory = inventoryOf(
|
||||
await artifactCli("inspect-artifact", component),
|
||||
);
|
||||
assert.equal(componentInventory.inventory.manifest.root_format, "pkg");
|
||||
const payloadPaths = [...componentInventory.entries.keys()]
|
||||
.filter((path) => path.startsWith("Payload/"))
|
||||
.map((path) => path.slice("Payload/".length))
|
||||
.sort();
|
||||
assert.deepEqual(
|
||||
payloadPaths,
|
||||
await bomPaths(component),
|
||||
"Payload members differ from pkgutil",
|
||||
);
|
||||
assert.equal(
|
||||
componentInventory.entries.get("Payload/usr/local/rea/hello.txt")?.sha256,
|
||||
sha256("hello\n"),
|
||||
);
|
||||
for (const name of ["preinstall", "postinstall"]) {
|
||||
const script = componentInventory.entries.get(`Scripts/${name}`);
|
||||
assert.equal(script?.sha256, sha256(`#!/bin/sh\necho ${name}\n`));
|
||||
assert.equal(script?.executable, true);
|
||||
}
|
||||
assert.ok(componentInventory.entries.has("PackageInfo"));
|
||||
assert.ok(componentInventory.entries.has("Bom"));
|
||||
|
||||
const productInventory = inventoryOf(
|
||||
await artifactCli("inspect-artifact", product),
|
||||
);
|
||||
assert.ok(productInventory.entries.has("Distribution"));
|
||||
assert.equal(
|
||||
productInventory.entries.get(
|
||||
"Component.pkg/Payload/usr/local/rea/hello.txt",
|
||||
)?.sha256,
|
||||
sha256("hello\n"),
|
||||
);
|
||||
assert.ok(productInventory.entries.has("Component.pkg/Scripts/postinstall"));
|
||||
|
||||
const pbzxInventory = inventoryOf(
|
||||
await artifactCli("inspect-artifact", pbzx),
|
||||
);
|
||||
const pbzxPayload = pbzxInventory.entries.get("Payload");
|
||||
assert.ok(
|
||||
pbzxPayload?.limitations.some((line) => line.includes("pbzx")),
|
||||
JSON.stringify(pbzxPayload),
|
||||
);
|
||||
assert.ok(
|
||||
![...pbzxInventory.entries.keys()].some((path) =>
|
||||
path.startsWith("Payload/"),
|
||||
),
|
||||
);
|
||||
|
||||
await withArtifactMcp(component, async (client) => {
|
||||
const viaMcp = inventoryOf(
|
||||
await artifactMcpResult(client, "inspect_artifact"),
|
||||
);
|
||||
assert.deepEqual(
|
||||
viaMcp.inventory.manifest,
|
||||
componentInventory.inventory.manifest,
|
||||
);
|
||||
});
|
||||
|
||||
process.stdout.write(
|
||||
`${JSON.stringify({
|
||||
ok: true,
|
||||
mocked: false,
|
||||
cli: true,
|
||||
stdio_mcp: true,
|
||||
payload_members: payloadPaths.length,
|
||||
matches_pkgutil_bom: true,
|
||||
product_nested: true,
|
||||
pbzx_payload: "recorded-not-expanded",
|
||||
})}\n`,
|
||||
);
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
@@ -2,9 +2,9 @@
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "27"
|
||||
version: "28"
|
||||
tool_count: 131
|
||||
catalog_digest: "188d726867808cd63764625b93d89e2936af29d9943dbc0e5d22044294521e1d"
|
||||
catalog_digest: "d18386ad8f46e7035cfa49f075c42e5c143f5b3f05c9459eb2d1c23c11112871"
|
||||
---
|
||||
|
||||
# REA
|
||||
|
||||
@@ -65,7 +65,9 @@ header, depending on the binary and analyzer version.
|
||||
## Packages and extraction
|
||||
|
||||
Use `inspect_artifact` for application bundles, archives, ZIP/APK/IPA/MSIX/AppX,
|
||||
ASAR, or DMG inputs when the artifact graph and findings help answer the
|
||||
ASAR, DMG, or PKG inputs. A PKG inventory includes Distribution, PackageInfo,
|
||||
the pre/postinstall scripts, and the gzip payload's files. Pbzx payloads are
|
||||
recorded only. Inspect it when the artifact graph and findings help answer the
|
||||
question. It returns the complete artifact graph inline. Cite graph manifest
|
||||
IDs when using them.
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@ import type { ArtifactReader } from "../../artifacts/ArtifactReader.js";
|
||||
import { DirectoryArtifactReader } from "../../artifacts/DirectoryArtifactReader.js";
|
||||
import { MachOSliceArtifactReader } from "../../artifacts/MachOSliceArtifactReader.js";
|
||||
import { NativeDmgArtifactReader } from "../../artifacts/NativeDmgArtifactReader.js";
|
||||
import { XarArtifactReader } from "../../artifacts/XarArtifactReader.js";
|
||||
import { ZipArtifactReader } from "../../artifacts/ZipArtifactReader.js";
|
||||
import type { ArtifactNode } from "../../domain/artifactGraph.js";
|
||||
|
||||
@@ -22,6 +23,8 @@ export const createReader = async (
|
||||
return new ZipArtifactReader(path, format);
|
||||
case "asar":
|
||||
return new AsarArtifactReader(path);
|
||||
case "pkg":
|
||||
return new XarArtifactReader(path);
|
||||
case "mach-o-universal":
|
||||
return process.platform === "darwin"
|
||||
? new MachOSliceArtifactReader(path)
|
||||
@@ -39,9 +42,9 @@ export const inventoryLimitations = (
|
||||
reader: ArtifactReader | undefined,
|
||||
): string[] => {
|
||||
if (reader !== undefined) return [];
|
||||
if (format === "dmg" || format === "pkg")
|
||||
if (format === "dmg")
|
||||
return [
|
||||
`${format.toUpperCase()} root hash is observed; child inventory requires a native macOS adapter.`,
|
||||
"DMG root hash is observed; child inventory requires a native macOS adapter.",
|
||||
];
|
||||
if (format === "mach-o-universal" && process.platform !== "darwin")
|
||||
return ["Universal Mach-O slices require the native macOS lipo adapter."];
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
type ArtifactReader,
|
||||
} from "../../artifacts/ArtifactReader.js";
|
||||
import { AsarArtifactReader } from "../../artifacts/AsarArtifactReader.js";
|
||||
import { CpioArtifactReader } from "../../artifacts/CpioArtifactReader.js";
|
||||
import type { ArtifactNode } from "../../domain/artifactGraph.js";
|
||||
import {
|
||||
classifyArtifactContent,
|
||||
@@ -123,11 +124,10 @@ const visitArtifactEntries = async (
|
||||
? entry.path
|
||||
: `${frame.prefix}/${entry.path}`,
|
||||
);
|
||||
const expandableAsar = isExpandableAsar(entry, logicalPath);
|
||||
context.registry.add(
|
||||
logicalPath,
|
||||
expandableAsar ? "directory" : entry.kind,
|
||||
);
|
||||
const expandable =
|
||||
isExpandableAsar(entry, logicalPath) ||
|
||||
entry.nestedArchive === "gzip-cpio";
|
||||
context.registry.add(logicalPath, expandable ? "directory" : entry.kind);
|
||||
const occurrence = createOccurrence(entry, logicalPath, null);
|
||||
let digested:
|
||||
| { readonly node: ArtifactNode; readonly mismatched: boolean }
|
||||
@@ -157,9 +157,13 @@ const visitArtifactEntries = async (
|
||||
}
|
||||
context.occurrences.push(occurrence);
|
||||
context.occurrenceByPath.set(logicalPath, occurrence);
|
||||
if (expandableAsar && digested?.mismatched !== true) {
|
||||
const nested = new AsarArtifactReader(entry.adapterKey);
|
||||
// Only traversed containers can own members, not opaque ASAR-named files.
|
||||
if (expandable && digested?.mismatched !== true) {
|
||||
const parent = frame.reader;
|
||||
const nested: ArtifactReader =
|
||||
entry.nestedArchive === "gzip-cpio"
|
||||
? new CpioArtifactReader((signal) => parent.open(entry, signal))
|
||||
: new AsarArtifactReader(entry.adapterKey);
|
||||
// Only traversed containers can own members, not opaque archive-named files.
|
||||
context.expandedContainerIds.add(occurrence.occurrence_id);
|
||||
stack.push({
|
||||
reader: nested,
|
||||
|
||||
@@ -35,7 +35,7 @@ export const artifactCapabilities = (
|
||||
requiresRoot: false,
|
||||
}),
|
||||
limitations: Object.freeze([
|
||||
"DMG child inventory automatically uses a read-only native macOS mount; PKG remains root-hash-only.",
|
||||
"DMG child inventory automatically uses a read-only native macOS mount. PKG (xar) inventory expands gzip-cpio Scripts and Payload members; pbzx payloads are recorded without expansion.",
|
||||
"ASAR files discovered in filesystem-backed inventories are expanded without bulk extraction; other nested containers remain recorded only.",
|
||||
]),
|
||||
};
|
||||
|
||||
@@ -15,6 +15,8 @@ export interface ArtifactEntry {
|
||||
readonly unpacked: boolean;
|
||||
readonly limitations: readonly string[];
|
||||
readonly adapterKey: string;
|
||||
/** Archive format of this member's own bytes, when the scan should expand it. */
|
||||
readonly nestedArchive?: "gzip-cpio";
|
||||
/** Filesystem identity captured during traversal, when supplied by an adapter. */
|
||||
readonly sourceIdentity?: {
|
||||
readonly device: number;
|
||||
@@ -24,7 +26,7 @@ export interface ArtifactEntry {
|
||||
|
||||
/** Read-only adapter over one directory, archive, or virtual container. */
|
||||
export interface ArtifactReader {
|
||||
readonly format: "directory" | ZipPackageFormat | "asar" | "file";
|
||||
readonly format: "directory" | ZipPackageFormat | "asar" | "pkg" | "file";
|
||||
entries(signal?: AbortSignal): AsyncIterable<ArtifactEntry>;
|
||||
open(entry: ArtifactEntry, signal?: AbortSignal): Promise<Readable>;
|
||||
provenance(): readonly ArtifactCommand[];
|
||||
|
||||
@@ -0,0 +1,287 @@
|
||||
import { Readable } from "node:stream";
|
||||
import { createGunzip } from "node:zlib";
|
||||
|
||||
import type { ArtifactCommand } from "../domain/artifactGraph.js";
|
||||
import {
|
||||
ArtifactReaderFailure,
|
||||
type ArtifactEntry,
|
||||
type ArtifactReader,
|
||||
} from "./ArtifactReader.js";
|
||||
|
||||
/** Longest member name accepted, including its terminating NUL. */
|
||||
const MAX_NAME_BYTES = 64 * 1024;
|
||||
const TRAILER = "TRAILER!!!";
|
||||
const S_IFMT = 0o170000;
|
||||
const S_IFDIR = 0o040000;
|
||||
const S_IFREG = 0o100000;
|
||||
const S_IFLNK = 0o120000;
|
||||
|
||||
/** Pull exactly the requested bytes from a stream, holding at most one chunk. */
|
||||
class ByteSource {
|
||||
readonly #iterator: AsyncIterator<Buffer>;
|
||||
#buffer: Buffer = Buffer.alloc(0);
|
||||
|
||||
constructor(stream: Readable) {
|
||||
this.#iterator = stream[Symbol.asyncIterator]();
|
||||
}
|
||||
|
||||
async #fill(): Promise<boolean> {
|
||||
const next = await this.#iterator.next();
|
||||
if (next.done === true) return false;
|
||||
const chunk: unknown = next.value;
|
||||
if (!Buffer.isBuffer(chunk))
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
"cpio stream yielded non-binary data",
|
||||
);
|
||||
this.#buffer =
|
||||
this.#buffer.length === 0 ? chunk : Buffer.concat([this.#buffer, chunk]);
|
||||
return true;
|
||||
}
|
||||
|
||||
async read(length: number, label: string): Promise<Buffer> {
|
||||
while (this.#buffer.length < length)
|
||||
if (!(await this.#fill()))
|
||||
throw new ArtifactReaderFailure("format", `cpio ${label} is truncated`);
|
||||
const bytes = this.#buffer.subarray(0, length);
|
||||
this.#buffer = this.#buffer.subarray(length);
|
||||
return bytes;
|
||||
}
|
||||
|
||||
/** Yield `length` bytes in chunks without buffering them together. */
|
||||
async *take(length: number, label: string): AsyncGenerator<Buffer> {
|
||||
let remaining = length;
|
||||
while (remaining > 0) {
|
||||
if (this.#buffer.length === 0 && !(await this.#fill()))
|
||||
throw new ArtifactReaderFailure("format", `cpio ${label} is truncated`);
|
||||
const size = Math.min(remaining, this.#buffer.length);
|
||||
const chunk = this.#buffer.subarray(0, size);
|
||||
this.#buffer = this.#buffer.subarray(size);
|
||||
remaining -= size;
|
||||
yield chunk;
|
||||
}
|
||||
}
|
||||
|
||||
async skip(length: number, label: string): Promise<void> {
|
||||
for await (const chunk of this.take(length, label)) void chunk;
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
await this.#iterator.return?.();
|
||||
}
|
||||
}
|
||||
|
||||
interface CpioHeader {
|
||||
readonly format: "odc" | "newc";
|
||||
readonly mode: number;
|
||||
readonly fileSize: number;
|
||||
readonly nameSize: number;
|
||||
}
|
||||
|
||||
const field = (
|
||||
bytes: Buffer,
|
||||
start: number,
|
||||
width: number,
|
||||
radix: 8 | 16,
|
||||
): number => {
|
||||
const text = bytes.toString("latin1", start, start + width);
|
||||
const pattern = radix === 8 ? /^[0-7]+$/u : /^[0-9a-fA-F]+$/u;
|
||||
const value = pattern.test(text) ? Number.parseInt(text, radix) : Number.NaN;
|
||||
if (!Number.isSafeInteger(value))
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
"cpio header has a malformed numeric field",
|
||||
);
|
||||
return value;
|
||||
};
|
||||
|
||||
/** odc (`070707`, octal) and newc/crc (`070701`/`070702`, hexadecimal) headers. */
|
||||
const readHeader = async (source: ByteSource): Promise<CpioHeader> => {
|
||||
const magic = (await source.read(6, "header")).toString("latin1");
|
||||
if (magic === "070707") {
|
||||
const rest = await source.read(70, "header");
|
||||
return {
|
||||
format: "odc",
|
||||
mode: field(rest, 12, 6, 8),
|
||||
nameSize: field(rest, 53, 6, 8),
|
||||
fileSize: field(rest, 59, 11, 8),
|
||||
};
|
||||
}
|
||||
if (magic === "070701" || magic === "070702") {
|
||||
const rest = await source.read(104, "header");
|
||||
return {
|
||||
format: "newc",
|
||||
mode: field(rest, 8, 8, 16),
|
||||
fileSize: field(rest, 48, 8, 16),
|
||||
nameSize: field(rest, 88, 8, 16),
|
||||
};
|
||||
}
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
"cpio member has an unsupported header; only odc and newc archives are expanded",
|
||||
);
|
||||
};
|
||||
|
||||
const padding = (format: CpioHeader["format"], length: number): number =>
|
||||
format === "newc" ? (4 - (length % 4)) % 4 : 0;
|
||||
|
||||
/** Normalize a member name; reject absolute and traversing names. */
|
||||
const memberPath = (raw: string): string | undefined => {
|
||||
const segments = raw
|
||||
.split("/")
|
||||
.filter((segment) => segment !== "" && segment !== ".");
|
||||
if (raw.startsWith("/") || segments.includes(".."))
|
||||
throw new ArtifactReaderFailure(
|
||||
"path",
|
||||
`cpio member escapes its archive: ${raw}`,
|
||||
);
|
||||
return segments.length === 0 ? undefined : segments.join("/");
|
||||
};
|
||||
|
||||
/** Project one cpio header as an archive-neutral entry. */
|
||||
const entryOf = (member: {
|
||||
readonly path: string;
|
||||
readonly kind: ArtifactEntry["kind"];
|
||||
readonly key: string;
|
||||
readonly header: CpioHeader;
|
||||
readonly limitations: readonly string[];
|
||||
}): ArtifactEntry => ({
|
||||
path: member.path,
|
||||
kind: member.kind,
|
||||
declaredSize: member.kind === "file" ? member.header.fileSize : null,
|
||||
compressedSize: null,
|
||||
executable: (member.header.mode & 0o111) !== 0,
|
||||
encrypted: false,
|
||||
byteOffset: null,
|
||||
declaredSha256: null,
|
||||
unpacked: false,
|
||||
limitations: member.limitations,
|
||||
adapterKey: member.key,
|
||||
});
|
||||
|
||||
/**
|
||||
* Sequential reader for a gzip-compressed cpio archive, such as an installer
|
||||
* package's Scripts or Payload. Members are decompressed once, in order: open
|
||||
* an entry before advancing to the next one. Device nodes, FIFOs, and sockets
|
||||
* are skipped; hard links appear as separate entries.
|
||||
*/
|
||||
export class CpioArtifactReader implements ArtifactReader {
|
||||
readonly format = "file" as const;
|
||||
#source: ByteSource | undefined;
|
||||
#current:
|
||||
| {
|
||||
readonly key: string;
|
||||
readonly size: number;
|
||||
readonly header: CpioHeader;
|
||||
}
|
||||
| undefined;
|
||||
#consumed = false;
|
||||
|
||||
constructor(
|
||||
private readonly openCompressed: (
|
||||
signal?: AbortSignal,
|
||||
) => Promise<Readable>,
|
||||
) {}
|
||||
|
||||
async *entries(signal?: AbortSignal): AsyncIterable<ArtifactEntry> {
|
||||
const source = new ByteSource(
|
||||
(await this.openCompressed(signal)).pipe(createGunzip()),
|
||||
);
|
||||
this.#source = source;
|
||||
for (let index = 0; ; index++) {
|
||||
signal?.throwIfAborted();
|
||||
await this.#finishCurrent(source);
|
||||
const header = await readHeader(source);
|
||||
if (header.nameSize < 1 || header.nameSize > MAX_NAME_BYTES)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
"cpio member name size is invalid",
|
||||
);
|
||||
const nameBytes = await source.read(header.nameSize, "member name");
|
||||
await source.skip(
|
||||
padding(header.format, 110 + header.nameSize),
|
||||
"name padding",
|
||||
);
|
||||
const raw = nameBytes.toString(
|
||||
"utf8",
|
||||
0,
|
||||
nameBytes.indexOf(0) >= 0 ? nameBytes.indexOf(0) : nameBytes.length,
|
||||
);
|
||||
if (raw === TRAILER) return;
|
||||
const type = header.mode & S_IFMT;
|
||||
const path = memberPath(raw);
|
||||
const key = `${index}:${raw}`;
|
||||
if (type === S_IFLNK) {
|
||||
const target = (
|
||||
await source.read(header.fileSize, "symlink target")
|
||||
).toString("utf8");
|
||||
await source.skip(
|
||||
padding(header.format, header.fileSize),
|
||||
"data padding",
|
||||
);
|
||||
if (path !== undefined)
|
||||
yield entryOf({
|
||||
path,
|
||||
kind: "symlink",
|
||||
key,
|
||||
header,
|
||||
limitations: [`Symlink target: ${target}`],
|
||||
});
|
||||
continue;
|
||||
}
|
||||
this.#current = { key, size: header.fileSize, header };
|
||||
this.#consumed = false;
|
||||
if (path === undefined || (type !== S_IFREG && type !== S_IFDIR))
|
||||
continue;
|
||||
yield entryOf({
|
||||
path,
|
||||
kind: type === S_IFDIR ? "directory" : "file",
|
||||
key,
|
||||
header,
|
||||
limitations: [],
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/** Skip whatever the caller did not read of the current member. */
|
||||
async #finishCurrent(source: ByteSource): Promise<void> {
|
||||
const current = this.#current;
|
||||
if (current === undefined) return;
|
||||
this.#current = undefined;
|
||||
if (!this.#consumed) await source.skip(current.size, "member data");
|
||||
await source.skip(
|
||||
padding(current.header.format, current.size),
|
||||
"data padding",
|
||||
);
|
||||
}
|
||||
|
||||
open(entry: ArtifactEntry): Promise<Readable> {
|
||||
const source = this.#source;
|
||||
const current = this.#current;
|
||||
if (
|
||||
source === undefined ||
|
||||
current === undefined ||
|
||||
current.key !== entry.adapterKey ||
|
||||
this.#consumed
|
||||
)
|
||||
return Promise.reject(
|
||||
new ArtifactReaderFailure(
|
||||
"unavailable",
|
||||
"cpio members can be read only once, in archive order",
|
||||
),
|
||||
);
|
||||
this.#consumed = true;
|
||||
return Promise.resolve(
|
||||
Readable.from(source.take(current.size, "member data")),
|
||||
);
|
||||
}
|
||||
|
||||
provenance(): readonly ArtifactCommand[] {
|
||||
return [];
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
await this.#source?.close();
|
||||
this.#source = undefined;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
/** xar archives and cpio payloads built field by field for installer-package tests. */
|
||||
import { createHash } from "node:crypto";
|
||||
import { deflateSync, gzipSync } from "node:zlib";
|
||||
|
||||
export interface XarFixtureMember {
|
||||
readonly name: string;
|
||||
readonly type?: "file" | "directory" | "symlink";
|
||||
readonly data?: Uint8Array;
|
||||
readonly encoding?: "raw" | "zlib" | "bzip2";
|
||||
readonly link?: string;
|
||||
readonly mode?: string;
|
||||
/** Replace the declared extracted checksum, to model tampering. */
|
||||
readonly extractedSha1?: string;
|
||||
readonly children?: readonly XarFixtureMember[];
|
||||
}
|
||||
|
||||
const sha1 = (bytes: Uint8Array): string =>
|
||||
createHash("sha1").update(bytes).digest("hex");
|
||||
|
||||
const escape = (text: string): string =>
|
||||
text.replaceAll("&", "&").replaceAll("<", "<").replaceAll(">", ">");
|
||||
|
||||
/** Build a xar archive with a SHA-1 TOC checksum and per-member checksums. */
|
||||
export const xarArchive = (
|
||||
members: readonly XarFixtureMember[],
|
||||
options: { readonly corruptTocChecksum?: boolean } = {},
|
||||
): Uint8Array => {
|
||||
const heap: Uint8Array[] = [new Uint8Array(20)];
|
||||
let heapLength = 20;
|
||||
let nextId = 1;
|
||||
const fileXml = (member: XarFixtureMember): string => {
|
||||
const id = nextId++;
|
||||
const type = member.type ?? "file";
|
||||
let data = "";
|
||||
if (type === "file" && member.data !== undefined) {
|
||||
const archived =
|
||||
member.encoding === "zlib" ? deflateSync(member.data) : member.data;
|
||||
const offset = heapLength;
|
||||
heap.push(archived);
|
||||
heapLength += archived.length;
|
||||
const style =
|
||||
member.encoding === "zlib"
|
||||
? "application/x-gzip"
|
||||
: member.encoding === "bzip2"
|
||||
? "application/x-bzip2"
|
||||
: "application/octet-stream";
|
||||
data = `<data><archived-checksum style="sha1">${sha1(archived)}</archived-checksum><extracted-checksum style="sha1">${member.extractedSha1 ?? sha1(member.data)}</extracted-checksum><encoding style="${style}"/><size>${member.data.length}</size><offset>${offset}</offset><length>${archived.length}</length></data>`;
|
||||
}
|
||||
const link =
|
||||
type === "symlink"
|
||||
? `<link type="file">${escape(member.link ?? "")}</link>`
|
||||
: "";
|
||||
const children = (member.children ?? []).map(fileXml).join("");
|
||||
return `<file id="${id}"><name>${escape(member.name)}</name><type>${type}</type><mode>${member.mode ?? (type === "directory" ? "0755" : "0644")}</mode>${data}${link}${children}</file>`;
|
||||
};
|
||||
const files = members.map(fileXml).join("");
|
||||
const toc = Buffer.from(
|
||||
`<?xml version="1.0" encoding="UTF-8"?><xar><toc><checksum style="sha1"><size>20</size><offset>0</offset></checksum>${files}</toc></xar>`,
|
||||
);
|
||||
const compressed = deflateSync(toc);
|
||||
const checksum = createHash("sha1").update(compressed).digest();
|
||||
if (options.corruptTocChecksum === true)
|
||||
checksum[0] = (checksum[0] ?? 0) ^ 0xff;
|
||||
heap[0] = checksum;
|
||||
const header = Buffer.alloc(28);
|
||||
header.writeUInt32BE(0x78617221, 0);
|
||||
header.writeUInt16BE(28, 4);
|
||||
header.writeUInt16BE(1, 6);
|
||||
header.writeBigUInt64BE(BigInt(compressed.length), 8);
|
||||
header.writeBigUInt64BE(BigInt(toc.length), 16);
|
||||
header.writeUInt32BE(1, 24);
|
||||
return Buffer.concat([header, compressed, ...heap]);
|
||||
};
|
||||
|
||||
export interface CpioFixtureMember {
|
||||
readonly name: string;
|
||||
readonly mode: number;
|
||||
readonly data?: Uint8Array | string;
|
||||
}
|
||||
|
||||
const octal = (value: number, width: number): string =>
|
||||
value.toString(8).padStart(width, "0");
|
||||
const hex = (value: number): string => value.toString(16).padStart(8, "0");
|
||||
|
||||
/** Build a gzip-compressed odc or newc cpio archive ending in its trailer. */
|
||||
export const gzipCpio = (
|
||||
members: readonly CpioFixtureMember[],
|
||||
format: "odc" | "newc" = "odc",
|
||||
): Uint8Array => {
|
||||
const parts: Buffer[] = [];
|
||||
const pad = (length: number): void => {
|
||||
if (format === "newc" && length % 4 !== 0)
|
||||
parts.push(Buffer.alloc(4 - (length % 4)));
|
||||
};
|
||||
for (const member of [...members, { name: "TRAILER!!!", mode: 0 }]) {
|
||||
const data =
|
||||
typeof member.data === "string"
|
||||
? Buffer.from(member.data)
|
||||
: Buffer.from(member.data ?? new Uint8Array());
|
||||
const name = Buffer.from(`${member.name}\0`);
|
||||
if (format === "odc")
|
||||
parts.push(
|
||||
Buffer.from(
|
||||
`070707${octal(0, 6)}${octal(1, 6)}${octal(member.mode, 6)}${octal(0, 6)}${octal(0, 6)}${octal(1, 6)}${octal(0, 6)}${octal(0, 11)}${octal(name.length, 6)}${octal(data.length, 11)}`,
|
||||
),
|
||||
name,
|
||||
data,
|
||||
);
|
||||
else {
|
||||
parts.push(
|
||||
Buffer.from(
|
||||
`070701${hex(1)}${hex(member.mode)}${hex(0)}${hex(0)}${hex(1)}${hex(0)}${hex(data.length)}${hex(0)}${hex(0)}${hex(0)}${hex(0)}${hex(name.length)}${hex(0)}`,
|
||||
),
|
||||
name,
|
||||
);
|
||||
pad(110 + name.length);
|
||||
parts.push(data);
|
||||
pad(data.length);
|
||||
}
|
||||
}
|
||||
return gzipSync(Buffer.concat(parts));
|
||||
};
|
||||
|
||||
export const MODE = {
|
||||
directory: 0o040755,
|
||||
file: 0o100644,
|
||||
executable: 0o100755,
|
||||
symlink: 0o120755,
|
||||
fifo: 0o010644,
|
||||
} as const;
|
||||
@@ -0,0 +1,225 @@
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { Readable } from "node:stream";
|
||||
import { buffer } from "node:stream/consumers";
|
||||
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
|
||||
import type { ArtifactEntry } from "./ArtifactReader.js";
|
||||
import { CpioArtifactReader } from "./CpioArtifactReader.js";
|
||||
import { MODE, gzipCpio, xarArchive } from "./InstallerPackage.fixture.js";
|
||||
import { XarArtifactReader } from "./XarArtifactReader.js";
|
||||
|
||||
let directory: string | undefined;
|
||||
afterEach(async () => {
|
||||
if (directory !== undefined)
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
directory = undefined;
|
||||
});
|
||||
|
||||
const writePackage = async (bytes: Uint8Array): Promise<string> => {
|
||||
directory = await mkdtemp(join(tmpdir(), "rea-pkg-reader-"));
|
||||
const path = join(directory, "fixture.pkg");
|
||||
await writeFile(path, bytes);
|
||||
return path;
|
||||
};
|
||||
|
||||
const collect = async (reader: {
|
||||
entries(): AsyncIterable<ArtifactEntry>;
|
||||
}): Promise<ArtifactEntry[]> => {
|
||||
const entries: ArtifactEntry[] = [];
|
||||
for await (const entry of reader.entries()) entries.push(entry);
|
||||
return entries;
|
||||
};
|
||||
|
||||
describe("xar installer package reader", () => {
|
||||
it("lists nested members and marks gzip-cpio scripts and payloads", async () => {
|
||||
const scripts = gzipCpio([
|
||||
{ name: "./postinstall", mode: MODE.executable, data: "#!/bin/sh\n" },
|
||||
]);
|
||||
const reader = new XarArtifactReader(
|
||||
await writePackage(
|
||||
xarArchive([
|
||||
{
|
||||
name: "Distribution",
|
||||
data: Buffer.from("<installer-gui-script/>"),
|
||||
encoding: "zlib",
|
||||
},
|
||||
{
|
||||
name: "App.pkg",
|
||||
type: "directory",
|
||||
children: [
|
||||
{
|
||||
name: "PackageInfo",
|
||||
data: Buffer.from("<pkg-info/>"),
|
||||
encoding: "zlib",
|
||||
},
|
||||
{ name: "Scripts", data: scripts },
|
||||
{ name: "Payload", data: Buffer.from("pbzx\0\0\0\0\x01\0\0\0") },
|
||||
{ name: "Current", type: "symlink", link: "App.pkg" },
|
||||
],
|
||||
},
|
||||
]),
|
||||
),
|
||||
);
|
||||
try {
|
||||
const entries = await collect(reader);
|
||||
expect(entries.map(({ path, kind }) => `${kind}:${path}`)).toEqual([
|
||||
"file:Distribution",
|
||||
"directory:App.pkg",
|
||||
"file:App.pkg/PackageInfo",
|
||||
"file:App.pkg/Scripts",
|
||||
"file:App.pkg/Payload",
|
||||
"symlink:App.pkg/Current",
|
||||
]);
|
||||
const scriptsEntry = entries.find(
|
||||
({ path }) => path === "App.pkg/Scripts",
|
||||
);
|
||||
expect(scriptsEntry?.nestedArchive).toBe("gzip-cpio");
|
||||
expect(
|
||||
entries.find(({ path }) => path === "App.pkg/Payload")?.limitations,
|
||||
).toEqual([expect.stringContaining("pbzx")]);
|
||||
const distribution = entries[0];
|
||||
if (distribution === undefined) throw new Error("missing entry");
|
||||
expect((await buffer(await reader.open(distribution))).toString()).toBe(
|
||||
"<installer-gui-script/>",
|
||||
);
|
||||
if (scriptsEntry === undefined) throw new Error("missing scripts");
|
||||
const nested = new CpioArtifactReader(() => reader.open(scriptsEntry));
|
||||
try {
|
||||
const members: string[] = [];
|
||||
for await (const entry of nested.entries()) {
|
||||
members.push(entry.path);
|
||||
expect((await buffer(await nested.open(entry))).toString()).toBe(
|
||||
"#!/bin/sh\n",
|
||||
);
|
||||
expect(entry.executable).toBe(true);
|
||||
}
|
||||
expect(members).toEqual(["postinstall"]);
|
||||
} finally {
|
||||
await nested.close();
|
||||
}
|
||||
} finally {
|
||||
await reader.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("rejects tampered members, TOCs and unsupported encodings", async () => {
|
||||
const tampered = new XarArtifactReader(
|
||||
await writePackage(
|
||||
xarArchive([
|
||||
{
|
||||
name: "Bom",
|
||||
data: Buffer.from("bom"),
|
||||
extractedSha1: "0".repeat(40),
|
||||
},
|
||||
]),
|
||||
),
|
||||
);
|
||||
try {
|
||||
const [entry] = await collect(tampered);
|
||||
if (entry === undefined) throw new Error("missing entry");
|
||||
await expect(buffer(await tampered.open(entry))).rejects.toMatchObject({
|
||||
reason: "integrity",
|
||||
});
|
||||
} finally {
|
||||
await tampered.close();
|
||||
}
|
||||
const corrupt = new XarArtifactReader(
|
||||
await writePackage(
|
||||
xarArchive([{ name: "Bom", data: Buffer.from("bom") }], {
|
||||
corruptTocChecksum: true,
|
||||
}),
|
||||
),
|
||||
);
|
||||
await expect(collect(corrupt)).rejects.toMatchObject({
|
||||
reason: "integrity",
|
||||
});
|
||||
await corrupt.close();
|
||||
const bzip = new XarArtifactReader(
|
||||
await writePackage(
|
||||
xarArchive([
|
||||
{ name: "Bom", data: Buffer.from("bom"), encoding: "bzip2" },
|
||||
]),
|
||||
),
|
||||
);
|
||||
const [entry] = await collect(bzip);
|
||||
if (entry === undefined) throw new Error("missing entry");
|
||||
await expect(bzip.open(entry)).rejects.toMatchObject({ reason: "format" });
|
||||
await bzip.close();
|
||||
const notXar = new XarArtifactReader(
|
||||
await writePackage(Buffer.from("not an archive at all")),
|
||||
);
|
||||
await expect(collect(notXar)).rejects.toMatchObject({ reason: "format" });
|
||||
await notXar.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("gzip cpio reader", () => {
|
||||
for (const format of ["odc", "newc"] as const)
|
||||
it(`reads ${format} members in order and skips unopened data`, async () => {
|
||||
const archive = gzipCpio(
|
||||
[
|
||||
{ name: ".", mode: MODE.directory },
|
||||
{ name: "./usr", mode: MODE.directory },
|
||||
{
|
||||
name: "./usr/bin/tool",
|
||||
mode: MODE.executable,
|
||||
data: "skipped bytes",
|
||||
},
|
||||
{ name: "./usr/lib/link", mode: MODE.symlink, data: "../bin/tool" },
|
||||
{ name: "./usr/fifo", mode: MODE.fifo },
|
||||
{ name: "./usr/share/readme.txt", mode: MODE.file, data: "read me" },
|
||||
],
|
||||
format,
|
||||
);
|
||||
const reader = new CpioArtifactReader(() =>
|
||||
Promise.resolve(Readable.from([archive])),
|
||||
);
|
||||
const seen: string[] = [];
|
||||
try {
|
||||
for await (const entry of reader.entries()) {
|
||||
seen.push(`${entry.kind}:${entry.path}`);
|
||||
if (entry.path === "usr/share/readme.txt") {
|
||||
expect((await buffer(await reader.open(entry))).toString()).toBe(
|
||||
"read me",
|
||||
);
|
||||
await expect(reader.open(entry)).rejects.toMatchObject({
|
||||
reason: "unavailable",
|
||||
});
|
||||
}
|
||||
if (entry.kind === "symlink")
|
||||
expect(entry.limitations).toEqual(["Symlink target: ../bin/tool"]);
|
||||
}
|
||||
} finally {
|
||||
await reader.close();
|
||||
}
|
||||
expect(seen).toEqual([
|
||||
"directory:usr",
|
||||
"file:usr/bin/tool",
|
||||
"symlink:usr/lib/link",
|
||||
"file:usr/share/readme.txt",
|
||||
]);
|
||||
});
|
||||
|
||||
it("rejects traversal and truncated archives", async () => {
|
||||
for (const name of ["../escape", "/absolute"]) {
|
||||
const reader = new CpioArtifactReader(() =>
|
||||
Promise.resolve(
|
||||
Readable.from([gzipCpio([{ name, mode: MODE.file, data: "x" }])]),
|
||||
),
|
||||
);
|
||||
await expect(collect(reader)).rejects.toMatchObject({ reason: "path" });
|
||||
await reader.close();
|
||||
}
|
||||
const whole = gzipCpio([{ name: "a", mode: MODE.file, data: "abc" }]);
|
||||
const { gunzipSync, gzipSync } = await import("node:zlib");
|
||||
const truncated = gzipSync(gunzipSync(whole).subarray(0, 60));
|
||||
const reader = new CpioArtifactReader(() =>
|
||||
Promise.resolve(Readable.from([truncated])),
|
||||
);
|
||||
await expect(collect(reader)).rejects.toMatchObject({ reason: "format" });
|
||||
await reader.close();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,413 @@
|
||||
import { createHash, type Hash } from "node:crypto";
|
||||
import { open, type FileHandle } from "node:fs/promises";
|
||||
import { Readable, Transform, type TransformCallback } from "node:stream";
|
||||
import { createInflate, inflateSync } from "node:zlib";
|
||||
|
||||
import { DOMParser, type Element, type Node } from "@xmldom/xmldom";
|
||||
|
||||
import type { ArtifactCommand } from "../domain/artifactGraph.js";
|
||||
import {
|
||||
ArtifactReaderFailure,
|
||||
type ArtifactEntry,
|
||||
type ArtifactReader,
|
||||
} from "./ArtifactReader.js";
|
||||
|
||||
const XAR_MAGIC = 0x78617221;
|
||||
/** A TOC describes members only; real installer TOCs are kilobytes. */
|
||||
const MAX_TOC_BYTES = 64 * 1024 * 1024;
|
||||
const READ_CHUNK_BYTES = 1024 * 1024;
|
||||
|
||||
const CHECKSUM_ALGORITHMS: Readonly<Record<string, string>> = {
|
||||
sha1: "sha1",
|
||||
md5: "md5",
|
||||
sha256: "sha256",
|
||||
sha512: "sha512",
|
||||
};
|
||||
|
||||
/** One TOC `<data>` element: where a member's archived bytes live in the heap. */
|
||||
interface XarData {
|
||||
readonly offset: number;
|
||||
readonly length: number;
|
||||
readonly size: number;
|
||||
readonly encoding: string;
|
||||
readonly extractedChecksum:
|
||||
| { readonly algorithm: string; readonly value: string }
|
||||
| undefined;
|
||||
}
|
||||
|
||||
interface XarMember {
|
||||
readonly path: string;
|
||||
readonly kind: "file" | "directory" | "symlink";
|
||||
readonly mode: number | null;
|
||||
readonly data: XarData | undefined;
|
||||
readonly link: string | undefined;
|
||||
}
|
||||
|
||||
const isElement = (node: Node): node is Element => node.nodeType === 1;
|
||||
|
||||
/** Direct child elements with one tag name, in document order. */
|
||||
const childElements = (element: Element, name: string): Element[] =>
|
||||
Array.from(element.childNodes).filter(
|
||||
(child): child is Element => isElement(child) && child.tagName === name,
|
||||
);
|
||||
|
||||
const childElement = (element: Element, name: string): Element | undefined =>
|
||||
childElements(element, name)[0];
|
||||
|
||||
const textOf = (element: Element, name: string): string | undefined =>
|
||||
childElement(element, name)?.textContent ?? undefined;
|
||||
|
||||
const integer = (value: string | undefined, label: string): number => {
|
||||
const parsed = value === undefined ? Number.NaN : Number(value.trim());
|
||||
if (!Number.isSafeInteger(parsed) || parsed < 0)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
`xar TOC has an invalid ${label}`,
|
||||
);
|
||||
return parsed;
|
||||
};
|
||||
|
||||
const parseData = (file: Element): XarData | undefined => {
|
||||
const data = childElement(file, "data");
|
||||
if (data === undefined) return undefined;
|
||||
const checksum = childElement(data, "extracted-checksum");
|
||||
const style = checksum?.getAttribute("style")?.toLowerCase() ?? "";
|
||||
return {
|
||||
offset: integer(textOf(data, "offset"), "data offset"),
|
||||
length: integer(textOf(data, "length"), "data length"),
|
||||
size: integer(textOf(data, "size"), "data size"),
|
||||
encoding:
|
||||
childElement(data, "encoding")?.getAttribute("style") ??
|
||||
"application/octet-stream",
|
||||
extractedChecksum:
|
||||
checksum === undefined || CHECKSUM_ALGORITHMS[style] === undefined
|
||||
? undefined
|
||||
: {
|
||||
algorithm: style,
|
||||
value: (checksum.textContent ?? "").trim().toLowerCase(),
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
/** Walk nested `<file>` elements in document order. */
|
||||
const collectMembers = (toc: Element): XarMember[] => {
|
||||
const members: XarMember[] = [];
|
||||
const pending: Array<{ readonly element: Element; readonly parent: string }> =
|
||||
childElements(toc, "file")
|
||||
.map((element) => ({ element, parent: "" }))
|
||||
.reverse();
|
||||
while (pending.length > 0) {
|
||||
const next = pending.pop();
|
||||
if (next === undefined) break;
|
||||
const name = textOf(next.element, "name");
|
||||
if (
|
||||
name === undefined ||
|
||||
name === "" ||
|
||||
name.includes("/") ||
|
||||
name === "." ||
|
||||
name === ".."
|
||||
)
|
||||
throw new ArtifactReaderFailure("path", "xar member has an unsafe name");
|
||||
const path = next.parent === "" ? name : `${next.parent}/${name}`;
|
||||
const type = textOf(next.element, "type")?.trim();
|
||||
const modeText = textOf(next.element, "mode");
|
||||
const mode =
|
||||
modeText === undefined ? Number.NaN : Number.parseInt(modeText.trim(), 8);
|
||||
const kind =
|
||||
type === "directory"
|
||||
? "directory"
|
||||
: type === "symlink"
|
||||
? "symlink"
|
||||
: "file";
|
||||
members.push({
|
||||
path,
|
||||
kind,
|
||||
mode: Number.isSafeInteger(mode) ? mode : null,
|
||||
data: kind === "file" ? parseData(next.element) : undefined,
|
||||
link: kind === "symlink" ? textOf(next.element, "link") : undefined,
|
||||
});
|
||||
pending.push(
|
||||
...childElements(next.element, "file")
|
||||
.map((element) => ({ element, parent: path }))
|
||||
.reverse(),
|
||||
);
|
||||
}
|
||||
return members;
|
||||
};
|
||||
|
||||
/** Verify a member's extracted checksum as its decoded bytes stream past. */
|
||||
class ChecksumVerifier extends Transform {
|
||||
readonly #hash: Hash;
|
||||
|
||||
constructor(
|
||||
private readonly path: string,
|
||||
private readonly expected: {
|
||||
readonly algorithm: string;
|
||||
readonly value: string;
|
||||
},
|
||||
) {
|
||||
super();
|
||||
this.#hash = createHash(expected.algorithm);
|
||||
}
|
||||
|
||||
override _transform(
|
||||
chunk: Buffer,
|
||||
_encoding: BufferEncoding,
|
||||
done: TransformCallback,
|
||||
): void {
|
||||
this.#hash.update(chunk);
|
||||
done(null, chunk);
|
||||
}
|
||||
|
||||
override _flush(done: TransformCallback): void {
|
||||
const observed = this.#hash.digest("hex");
|
||||
done(
|
||||
observed === this.expected.value
|
||||
? null
|
||||
: new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`xar ${this.expected.algorithm} checksum disagrees with content: ${this.path}`,
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** First bytes that identify an installer archive member's own format. */
|
||||
export type XarNestedArchive = "gzip-cpio";
|
||||
|
||||
/**
|
||||
* Read-only reader for xar archives, the container of flat installer
|
||||
* packages. Members are decoded and checksum-verified on demand. Gzip cpio
|
||||
* `Scripts` and `Payload` members are marked for nested expansion; pbzx
|
||||
* (LZMA) payloads are recorded as files with a limitation.
|
||||
*/
|
||||
export class XarArtifactReader implements ArtifactReader {
|
||||
readonly format = "pkg" as const;
|
||||
#handle: FileHandle | undefined;
|
||||
#heap = 0;
|
||||
#members: readonly XarMember[] = [];
|
||||
|
||||
constructor(private readonly path: string) {}
|
||||
|
||||
async #load(signal?: AbortSignal): Promise<void> {
|
||||
if (this.#handle !== undefined) return;
|
||||
signal?.throwIfAborted();
|
||||
const handle = await open(this.path, "r");
|
||||
this.#handle = handle;
|
||||
const header = Buffer.alloc(28);
|
||||
const { bytesRead } = await handle.read(header, 0, 28, 0);
|
||||
if (bytesRead < 28 || header.readUInt32BE(0) !== XAR_MAGIC)
|
||||
throw new ArtifactReaderFailure("format", "File is not a xar archive");
|
||||
const headerSize = header.readUInt16BE(4);
|
||||
const tocCompressed = header.readBigUInt64BE(8);
|
||||
const tocSize = header.readBigUInt64BE(16);
|
||||
if (headerSize < 28)
|
||||
throw new ArtifactReaderFailure("format", "xar header is too short");
|
||||
if (
|
||||
tocCompressed > BigInt(MAX_TOC_BYTES) ||
|
||||
tocSize > BigInt(MAX_TOC_BYTES)
|
||||
)
|
||||
throw new ArtifactReaderFailure(
|
||||
"limit",
|
||||
`xar TOC exceeds the ${MAX_TOC_BYTES}-byte limit`,
|
||||
);
|
||||
const compressed = Buffer.alloc(Number(tocCompressed));
|
||||
const toc = await handle.read(compressed, 0, compressed.length, headerSize);
|
||||
if (toc.bytesRead < compressed.length)
|
||||
throw new ArtifactReaderFailure("format", "xar TOC is truncated");
|
||||
this.#heap = headerSize + compressed.length;
|
||||
let xml: string;
|
||||
try {
|
||||
xml = inflateSync(compressed, {
|
||||
maxOutputLength: MAX_TOC_BYTES,
|
||||
}).toString("utf8");
|
||||
} catch (cause: unknown) {
|
||||
throw new ArtifactReaderFailure("format", "xar TOC is not zlib data", {
|
||||
cause,
|
||||
});
|
||||
}
|
||||
let document;
|
||||
try {
|
||||
document = new DOMParser({
|
||||
onError: (level, message) => {
|
||||
if (level !== "warning") throw new Error(message);
|
||||
},
|
||||
}).parseFromString(xml, "text/xml");
|
||||
} catch (cause: unknown) {
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
"xar TOC is not well-formed XML",
|
||||
{ cause },
|
||||
);
|
||||
}
|
||||
const root = document.documentElement;
|
||||
const tocElement = root === null ? undefined : childElement(root, "toc");
|
||||
if (tocElement === undefined)
|
||||
throw new ArtifactReaderFailure("format", "xar TOC has no <toc> element");
|
||||
await this.#verifyToc(tocElement, compressed);
|
||||
this.#members = collectMembers(tocElement);
|
||||
}
|
||||
|
||||
/** The TOC checksum in the heap covers the compressed TOC bytes. */
|
||||
async #verifyToc(toc: Element, compressed: Buffer): Promise<void> {
|
||||
const checksum = childElement(toc, "checksum");
|
||||
const algorithm =
|
||||
CHECKSUM_ALGORITHMS[checksum?.getAttribute("style")?.toLowerCase() ?? ""];
|
||||
if (checksum === undefined || algorithm === undefined) return;
|
||||
const stored = await this.#heapBytes(
|
||||
integer(textOf(checksum, "offset"), "checksum offset"),
|
||||
integer(textOf(checksum, "size"), "checksum size"),
|
||||
);
|
||||
if (!createHash(algorithm).update(compressed).digest().equals(stored))
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
"xar TOC checksum disagrees with the TOC",
|
||||
);
|
||||
}
|
||||
|
||||
async #heapBytes(offset: number, length: number): Promise<Buffer> {
|
||||
const handle = this.#handle;
|
||||
if (handle === undefined)
|
||||
throw new ArtifactReaderFailure("unavailable", "xar archive is closed");
|
||||
const bytes = Buffer.alloc(length);
|
||||
const { bytesRead } = await handle.read(
|
||||
bytes,
|
||||
0,
|
||||
length,
|
||||
this.#heap + offset,
|
||||
);
|
||||
if (bytesRead < length)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
"xar member extends beyond the archive",
|
||||
);
|
||||
return bytes;
|
||||
}
|
||||
|
||||
/** Classify a raw-stored installer member by its leading bytes. */
|
||||
async #nestedArchive(member: XarMember): Promise<{
|
||||
readonly nested: XarNestedArchive | undefined;
|
||||
readonly limitations: readonly string[];
|
||||
}> {
|
||||
const name = member.path.split("/").at(-1);
|
||||
const data = member.data;
|
||||
if (
|
||||
(name !== "Payload" && name !== "Scripts") ||
|
||||
data === undefined ||
|
||||
data.length < 4
|
||||
)
|
||||
return { nested: undefined, limitations: [] };
|
||||
if (data.encoding !== "application/octet-stream")
|
||||
return {
|
||||
nested: undefined,
|
||||
limitations: [
|
||||
`${name} is stored with ${data.encoding}; its archive is not expanded.`,
|
||||
],
|
||||
};
|
||||
const head = await this.#heapBytes(data.offset, 4);
|
||||
if (head[0] === 0x1f && head[1] === 0x8b)
|
||||
return { nested: "gzip-cpio", limitations: [] };
|
||||
if (head.toString("latin1") === "pbzx")
|
||||
return {
|
||||
nested: undefined,
|
||||
limitations: [
|
||||
`${name} is a pbzx (LZMA) archive; its members are not expanded. Expand it with pkgutil --expand-full to inspect them.`,
|
||||
],
|
||||
};
|
||||
return {
|
||||
nested: undefined,
|
||||
limitations: [
|
||||
`${name} has an unrecognized archive format and is not expanded.`,
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
async *entries(signal?: AbortSignal): AsyncIterable<ArtifactEntry> {
|
||||
await this.#load(signal);
|
||||
for (const member of this.#members) {
|
||||
signal?.throwIfAborted();
|
||||
const classified =
|
||||
member.kind === "file"
|
||||
? await this.#nestedArchive(member)
|
||||
: { nested: undefined, limitations: [] };
|
||||
yield {
|
||||
path: member.path,
|
||||
kind: member.kind,
|
||||
declaredSize: member.data?.size ?? (member.kind === "file" ? 0 : null),
|
||||
compressedSize: member.data?.length ?? null,
|
||||
executable: member.mode !== null && (member.mode & 0o111) !== 0,
|
||||
encrypted: false,
|
||||
byteOffset:
|
||||
member.data?.encoding === "application/octet-stream"
|
||||
? this.#heap + member.data.offset
|
||||
: null,
|
||||
// xar declares SHA-1/MD5 checksums, which open() verifies itself.
|
||||
declaredSha256: null,
|
||||
unpacked: false,
|
||||
limitations: [
|
||||
...(member.kind === "symlink" && member.link !== undefined
|
||||
? [`Symlink target recorded in the TOC: ${member.link}`]
|
||||
: []),
|
||||
...classified.limitations,
|
||||
],
|
||||
adapterKey: member.path,
|
||||
...(classified.nested === undefined
|
||||
? {}
|
||||
: { nestedArchive: classified.nested }),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
async open(entry: ArtifactEntry, signal?: AbortSignal): Promise<Readable> {
|
||||
await this.#load(signal);
|
||||
const member = this.#members.find(({ path }) => path === entry.adapterKey);
|
||||
if (member === undefined || member.kind !== "file")
|
||||
throw new ArtifactReaderFailure(
|
||||
"path",
|
||||
`xar member is not a file: ${entry.path}`,
|
||||
);
|
||||
const data = member.data;
|
||||
if (data === undefined) return Readable.from([]);
|
||||
const raw = Readable.from(this.#chunks(data.offset, data.length, signal));
|
||||
let decoded: Readable;
|
||||
if (data.encoding === "application/octet-stream") decoded = raw;
|
||||
else if (data.encoding === "application/x-gzip")
|
||||
// xar's "x-gzip" members are zlib streams.
|
||||
decoded = raw.pipe(createInflate());
|
||||
else
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
`xar member ${entry.path} uses unsupported encoding ${data.encoding}`,
|
||||
);
|
||||
const checksum = data.extractedChecksum;
|
||||
if (checksum === undefined) return decoded;
|
||||
const verified = new ChecksumVerifier(entry.path, checksum);
|
||||
decoded.on("error", (cause: unknown) =>
|
||||
verified.destroy(cause instanceof Error ? cause : undefined),
|
||||
);
|
||||
return decoded.pipe(verified);
|
||||
}
|
||||
|
||||
async *#chunks(
|
||||
offset: number,
|
||||
length: number,
|
||||
signal?: AbortSignal,
|
||||
): AsyncGenerator<Buffer> {
|
||||
for (let done = 0; done < length;) {
|
||||
signal?.throwIfAborted();
|
||||
const size = Math.min(READ_CHUNK_BYTES, length - done);
|
||||
yield await this.#heapBytes(offset + done, size);
|
||||
done += size;
|
||||
}
|
||||
}
|
||||
|
||||
provenance(): readonly ArtifactCommand[] {
|
||||
return [];
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
const handle = this.#handle;
|
||||
this.#handle = undefined;
|
||||
await handle?.close();
|
||||
}
|
||||
}
|
||||
@@ -58,7 +58,7 @@ const artifact = <
|
||||
export const ARTIFACT_TOOL_CONTRACTS = [
|
||||
artifact(
|
||||
"inspect_artifact",
|
||||
"Inspect an active archive or application package in one call. Returns the complete content-addressed artifact graph, source Evidence, observations, relationships, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.",
|
||||
"Inspect an active archive or application package in one call. Returns the complete content-addressed artifact graph, source Evidence, observations, relationships, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Read-only DMG mounting is used automatically on supported macOS hosts. Flat installer packages (PKG) expand their gzip-cpio Scripts and Payload; pbzx payloads are recorded without expansion. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.",
|
||||
artifactInventoryInputSchema,
|
||||
),
|
||||
artifact(
|
||||
|
||||
@@ -30,7 +30,7 @@ interface GeneratedPayload {
|
||||
}
|
||||
|
||||
const GENERATED_PAYLOAD_JSON =
|
||||
'{\n "auxiliaryProviders": [\n {\n "capabilities": [\n {\n "available": true,\n "effects": {\n "changesPermissions": false,\n "launchesProcess": true,\n "mayAccessNetwork": false,\n "mayShowUi": false,\n "mayWriteFilesystem": true,\n "mutatesArtifact": false,\n "requiresRoot": false\n },\n "limitations": [\n "DMG child inventory automatically uses a read-only native macOS mount; PKG remains root-hash-only.",\n "ASAR files discovered in filesystem-backed inventories are expanded without bulk extraction; other nested containers remain recorded only."\n ],\n "operation": "inventory_artifact",\n "provider": {\n "id": "rea-artifact-graph",\n "name": "REA safe artifact graph provider",\n "version": "1"\n },\n "reason": null\n },\n {\n "available": true,\n "effects": {\n "changesPermissions": false,\n "launchesProcess": true,\n "mayAccessNetwork": false,\n "mayShowUi": false,\n "mayWriteFilesystem": true,\n "mutatesArtifact": false,\n "requiresRoot": false\n },\n "limitations": [\n "DMG child inventory automatically uses a read-only native macOS mount; PKG remains root-hash-only.",\n "ASAR files discovered in filesystem-backed inventories are expanded without bulk extraction; other nested containers remain recorded only."\n ],\n "operation": "inspect_artifact",\n "provider": {\n "id": "rea-artifact-graph",\n "name": "REA safe artifact graph provider",\n "version": "1"\n },\n "reason": null\n },\n {\n "available": true,\n "effects": {\n "changesPermissions": false,\n "launchesProcess": true,\n "mayAccessNetwork": false,\n "mayShowUi": false,\n "mayWriteFilesystem": true,\n "mutatesArtifact": false,\n "requiresRoot": false\n },\n "limitations": [\n "DMG child inventory automatically uses a read-only native macOS mount; PKG remains root-hash-only.",\n "ASAR files discovered in filesystem-backed inventories are expanded without bulk extraction; other nested containers remain recorded only."\n ],\n "operation": "extract_artifact",\n "provider": {\n "id": "rea-artifact-graph",\n "name": "REA safe artifact graph provider",\n "version": "1"\n },\n "reason": null\n },\n {\n "available": true,\n "effects": {\n "changesPermissions": false,\n "launchesProcess": false,\n "mayAccessNetwork": false,\n "mayShowUi": false,\n "mayWriteFilesystem": false,\n "mutatesArtifact": false,\n "requiresRoot": false\n },\n "limitations": [\n "DMG child inventory automatically uses a read-only native macOS mount; PKG remains root-hash-only.",\n "ASAR files discovered in filesystem-backed inventories are expanded without bulk extraction; other nested containers remain recorded only."\n ],\n "operation": "decode_interface_builder",\n "provider": {\n "id": "rea-artifact-graph",\n "name": "REA safe artifact graph provider",\n "version": "1"\n },\n "reason": null\n },\n {\n "available": true,\n "effects": {\n "changesPermissions": false,\n "launchesProcess": false,\n "mayAccessNetwork": false,\n "mayShowUi": false,\n "mayWriteFilesystem": false,\n "mutatesArtifact": false,\n "requiresRoot": false\n },\n "limitations": [\n "DMG child inventory automatically uses a read-only native macOS mount; PKG remains root-hash-only.",\n "ASAR files discovered in filesystem-backed inventories are expanded without bulk extraction; other nested containers remain recorded only."\n ],\n "operation": "inspect_keyed_archive",\n "provider": {\n "id": "rea-artifact-graph",\n "name": "REA safe artifact graph provider",\n "version": "1"\n },\n "reason": null\n },\n {\n "available": true,\n "effects": {\n "changesPermissions": false,\n "launchesProcess": true,\n "mayAccessNetwork": false,\n "mayShowUi": false,\n "mayWriteFilesystem": false,\n "mutatesArtifact": false,\n "requiresRoot": false\n },\n "limitations": [\n "DLine truncated
|
||||
'{\n "auxiliaryProviders": [\n {\n "capabilities": [\n {\n "available": true,\n "effects": {\n "changesPermissions": false,\n "launchesProcess": true,\n "mayAccessNetwork": false,\n "mayShowUi": false,\n "mayWriteFilesystem": true,\n "mutatesArtifact": false,\n "requiresRoot": false\n },\n "limitations": [\n "DMG child inventory automatically uses a read-only native macOS mount. PKG (xar) inventory expands gzip-cpio Scripts and Payload members; pbzx payloads are recorded without expansion.",\n "ASAR files discovered in filesystem-backed inventories are expanded without bulk extraction; other nested containers remain recorded only."\n ],\n "operation": "inventory_artifact",\n "provider": {\n "id": "rea-artifact-graph",\n "name": "REA safe artifact graph provider",\n "version": "1"\n },\n "reason": null\n },\n {\n "available": true,\n "effects": {\n "changesPermissions": false,\n "launchesProcess": true,\n "mayAccessNetwork": false,\n "mayShowUi": false,\n "mayWriteFilesystem": true,\n "mutatesArtifact": false,\n "requiresRoot": false\n },\n "limitations": [\n "DMG child inventory automatically uses a read-only native macOS mount. PKG (xar) inventory expands gzip-cpio Scripts and Payload members; pbzx payloads are recorded without expansion.",\n "ASAR files discovered in filesystem-backed inventories are expanded without bulk extraction; other nested containers remain recorded only."\n ],\n "operation": "inspect_artifact",\n "provider": {\n "id": "rea-artifact-graph",\n "name": "REA safe artifact graph provider",\n "version": "1"\n },\n "reason": null\n },\n {\n "available": true,\n "effects": {\n "changesPermissions": false,\n "launchesProcess": true,\n "mayAccessNetwork": false,\n "mayShowUi": false,\n "mayWriteFilesystem": true,\n "mutatesArtifact": false,\n "requiresRoot": false\n },\n "limitations": [\n "DMG child inventory automatically uses a read-only native macOS mount. PKG (xar) inventory expands gzip-cpio Scripts and Payload members; pbzx payloads are recorded without expansion.",\n "ASAR files discovered in filesystem-backed inventories are expanded without bulk extraction; other nested containers remain recorded only."\n ],\n "operation": "extract_artifact",\n "provider": {\n "id": "rea-artifact-graph",\n "name": "REA safe artifact graph provider",\n "version": "1"\n },\n "reason": null\n },\n {\n "available": true,\n "effects": {\n "changesPermissions": false,\n "launchesProcess": false,\n "mayAccessNetwork": false,\n "mayShowUi": false,\n "mayWriteFilesystem": false,\n "mutatesArtifact": false,\n "requiresRoot": false\n },\n "limitations": [\n "DMG child inventory automatically uses a read-only native macOS mount. PKG (xar) inventory expands gzip-cpio Scripts and Payload members; pbzx payloads are recorded without expansion.",\n "ASAR files discovered in filesystem-backed inventories are expanded without bulk extraction; other nested containers remain recorded only."\n ],\n "operation": "decode_interface_builder",\n "provider": {\n "id": "rea-artifact-graph",\n "name": "REA safe artifact graph provider",\n "version": "1"\n },\n "reason": null\n },\n {\n "available": true,\n "effects": {\n "changesPermissions": false,\n "launchesProcess": false,\n "mayAccessNetwork": false,\n "mayShowUi": false,\n "mayWriteFilesystem": false,\n "mutatesArtifact": false,\n "requiresRoot": false\n },\n "limitations": [\n "DMG child inventory automatically uses a read-only native macOS mount. PKG (xar) inventory expands gzip-cpio Scripts and Payload members; pbzx payloads are recorded without expansion.",\n "ASAR files discovered in filesystem-backed inventories are expanded without bulk extraction; other nested containers remain recorded only."\n ],\n "operation": "inspect_keyed_archive",\n "provider": {\n "id": "rea-artifact-graph",\n "name": "REA safe artifact graph provider",\n "version": "1"\n },\n
Line truncated
|
||||
const generatedPayload: unknown = JSON.parse(GENERATED_PAYLOAD_JSON);
|
||||
if (!isGeneratedPayload(generatedPayload))
|
||||
throw new TypeError("Generated MCP catalog payload is invalid");
|
||||
|
||||
@@ -5,5 +5,5 @@ export const PACKAGE_METADATA = {
|
||||
serverSdkVersion: "2.3.1",
|
||||
clientSdkVersion: "2.3.1",
|
||||
coreSdkVersion: "2.3.1",
|
||||
skillVersion: "27",
|
||||
skillVersion: "28",
|
||||
} as const;
|
||||
@@ -0,0 +1,120 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
MODE,
|
||||
gzipCpio,
|
||||
xarArchive,
|
||||
} from "../../../src/artifacts/InstallerPackage.fixture.js";
|
||||
import { runProviderAnalysis } from "../../../src/application/DirectAnalysis.js";
|
||||
import { artifactInventoryResultSchema } from "../../../src/domain/artifactGraph.js";
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
const sha256 = (text: string): string =>
|
||||
createHash("sha256").update(text).digest("hex");
|
||||
|
||||
const productPackage = (payloadText: string, extractedSha1?: string) =>
|
||||
xarArchive([
|
||||
{
|
||||
name: "Distribution",
|
||||
data: Buffer.from("<installer-gui-script/>"),
|
||||
encoding: "zlib",
|
||||
},
|
||||
{
|
||||
name: "App.pkg",
|
||||
type: "directory",
|
||||
children: [
|
||||
{
|
||||
name: "PackageInfo",
|
||||
data: Buffer.from('<pkg-info identifier="com.example.app"/>'),
|
||||
encoding: "zlib",
|
||||
...(extractedSha1 === undefined ? {} : { extractedSha1 }),
|
||||
},
|
||||
{
|
||||
name: "Scripts",
|
||||
data: gzipCpio([
|
||||
{ name: ".", mode: MODE.directory },
|
||||
{
|
||||
name: "./postinstall",
|
||||
mode: MODE.executable,
|
||||
data: "#!/bin/sh\n",
|
||||
},
|
||||
]),
|
||||
},
|
||||
{
|
||||
name: "Payload",
|
||||
data: gzipCpio([
|
||||
{ name: ".", mode: MODE.directory },
|
||||
{ name: "./Applications", mode: MODE.directory },
|
||||
{
|
||||
name: "./Applications/App.app/Contents/Info.plist",
|
||||
mode: MODE.file,
|
||||
data: payloadText,
|
||||
},
|
||||
]),
|
||||
},
|
||||
],
|
||||
},
|
||||
]);
|
||||
|
||||
describe("installer package inventory", () => {
|
||||
it("inventories xar members and expands gzip-cpio scripts and payloads", async () => {
|
||||
const directory = await createTestTempDirectory("rea-pkg-inventory-");
|
||||
const path = join(directory, "Installer.pkg");
|
||||
await writeFile(path, productPackage("<plist/>"));
|
||||
const evidence = parseEvidence(
|
||||
await runProviderAnalysis(path, "inventory_artifact", {}),
|
||||
);
|
||||
const inventory = artifactInventoryResultSchema.parse(
|
||||
evidence.normalized_result,
|
||||
);
|
||||
expect(inventory.manifest.root_format).toBe("pkg");
|
||||
expect(inventory.limitations).toEqual([]);
|
||||
const occurrences = new Map(
|
||||
inventory.occurrences.map((occurrence) => [
|
||||
occurrence.logical_path,
|
||||
occurrence,
|
||||
]),
|
||||
);
|
||||
expect([...occurrences.keys()]).toEqual(
|
||||
expect.arrayContaining([
|
||||
"Distribution",
|
||||
"App.pkg/PackageInfo",
|
||||
"App.pkg/Scripts",
|
||||
"App.pkg/Scripts/postinstall",
|
||||
"App.pkg/Payload",
|
||||
"App.pkg/Payload/Applications/App.app/Contents/Info.plist",
|
||||
]),
|
||||
);
|
||||
const plist = occurrences.get(
|
||||
"App.pkg/Payload/Applications/App.app/Contents/Info.plist",
|
||||
);
|
||||
const node = inventory.nodes.find(
|
||||
({ artifact_id: id }) => id === plist?.artifact_id,
|
||||
);
|
||||
expect(node?.sha256).toBe(sha256("<plist/>"));
|
||||
expect(occurrences.get("App.pkg/Scripts/postinstall")?.executable).toBe(
|
||||
true,
|
||||
);
|
||||
const scripts = occurrences.get("App.pkg/Scripts");
|
||||
expect(
|
||||
occurrences.get("App.pkg/Scripts/postinstall")?.parent_occurrence_id,
|
||||
).toBe(scripts?.occurrence_id);
|
||||
});
|
||||
|
||||
it("fails on a member whose xar checksum disagrees with its bytes", async () => {
|
||||
const directory = await createTestTempDirectory("rea-pkg-tampered-");
|
||||
const path = join(directory, "Installer.pkg");
|
||||
await writeFile(path, productPackage("<plist/>", "0".repeat(40)));
|
||||
expect(
|
||||
await runProviderAnalysis(path, "inventory_artifact", {}),
|
||||
).toMatchObject({
|
||||
error: "Analysis failed",
|
||||
details: { reason: "integrity" },
|
||||
});
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user