Merge remote-tracking branch 'fork/feat/environment-later-reads' into feat/environment-call-values

This commit is contained in:
Edgar Arakelyan committed 2026-10-26 16:38:48 -07:00
commit 442ca0f476
291 files changed
+7178 -2555

No files matched your search

@@ -344,6 +344,11 @@ leaves its database open. The modern headless profile creates and closes an
owned private database without saving. Installation and registration reuse
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
For migration to the separate native IDA MCP plugin, follow the guide's
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/ida-provider.md#migrate-a-legacy-attached-registration).
Replace the upstream registration explicitly; modern mrexodia tools are not the
native profile. Native attachment preserves the database but does not provide
direct callers; check session availability and the guide's verification limits.
Proceed directly to `analyze_function`, pseudocode, or function/string searches
after opening; IDA does not supply `binary_overview`. Consult session availability
when composing broader workflows. Modern direct callers and unsupported dossier
+7 -4
View File
@@ -546,6 +546,9 @@ rea compare /absolute/path/to/evidence/left.json /absolute/path/to/evidence/righ
rea compare left.json right.json --record_pairs '[{"left_evidence_id":"ev_<left-sha256>","right_evidence_id":"ev_<right-sha256>"}]' --json
```
Migration: replace `rea compare-bundles` with `rea compare`; the former command
has been removed. The MCP tool remains `compare_bundles`.
`--record_pairs` supplies an explicit JSON array of observation pairs, matching
MCP `compare_bundles.record_pairs`. Pair IDs must exist on their named sides and
be one-to-one. Without pairs, one-sided membership remains unknown; REA does
@@ -982,10 +985,10 @@ Cutter when available. This is separate from `--format json`, which selects the
CLI result representation; the global `--json` flag selects that same CLI
representation.
Cutter still accepts deprecated `--json=true` and `--json=false` compatibility
values. When both forms are present, `--json-output` takes precedence in either
argument order. Bare `--json` retains its global meaning of selecting JSON CLI
output formatting.
Cutter no longer accepts command-local `--json=true` or `--json=false`.
Replace these with `--json-output=true` or `--json-output=false`, respectively.
Combine `--json-output` with bare `--json` when both Cutter command output and
the CLI result should use JSON; either argument order is accepted.
| Status | Meaning |
| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+65
View File
@@ -80,6 +80,71 @@ Native MCP 1.1.2 can navigate in the previously focused pseudocode tab when swit
Native tool observations retain actual tool names and complete JSON text. Upstream `structuredContent` can be a shortened preview even while text contains the full result. REA reads the full native representation and rejects missing/malformed text instead of treating previews as complete observations.
### Migrate a legacy attached registration
Both attached profiles remain supported. Migrating replaces the upstream server;
adding `"protocol": "native"` to a legacy Python proxy registration is insufficient.
The native plugin is separate from mrexodia/ida-pro-mcp, and upgrading mrexodia to
its modern tool catalog does not select this profile.
1. Back up the working registration file. Install and start the native plugin
using its [upstream instructions](https://github.com/0xshlomil/ida-free-mcp).
IDA and the plugin are bring-your-own; REA does not install or build them.
Use a build with the navigation correction described above, and restrict its
listener to loopback or apply the documented host firewall restriction.
Stop the legacy endpoint first if both servers would use the same port.
2. Open the same original input binary in IDA and wait for analysis. Keep that
file readable by REA. The native metadata resource must report its recorded
input SHA-256; the resource's `path` identifies the database, not an input
file to substitute. Universal Mach-O inputs still require a thin extraction.
3. Replace the registration, using the native plugin's actual local endpoint.
For example, this legacy registration:
```json
{
"mode": "attached",
"command": "python",
"args": [
"/absolute/path/to/ida_pro_mcp/server.py",
"--ida-rpc",
"http://127.0.0.1:13337"
]
}
```
becomes:
```json
{
"mode": "attached",
"protocol": "native",
"url": "http://127.0.0.1:13337/mcp"
}
```
Keep `REA_IDA_MCP_CONFIG` pointing at the replacement file and close/reopen
the REA binary session to connect with it. If changing the environment's
configuration path, restart the MCP server as described above. Do not reuse
the legacy RPC URL as an MCP HTTP endpoint.
4. Check `binary_session` availability before resuming workflows. Direct
`procedure_callers` becomes unavailable and dossier callers remain unknown;
incoming address xrefs do not establish direct call edges. Native strings
use C-string rendering. Assembly and instruction reads no longer navigate
the GUI; native pseudocode still changes focus and closes existing
pseudocode tabs. IDA Free decompilation may require cloud access and consent.
5. Run the [real-provider verification](#verification) with two distinct
functions and the required second-function marker. Supply independently
known string and xref oracles for those checks. Existing Windows evidence
covers IDA Pro 9.3 with the documented source build and listener change;
it does not establish stock-plugin, Windows Free, or native macOS behavior.
A valid registration or successful doctor check alone does not verify the
connection, target identity, or function switching.
REA continues to verify identity and image base around requests and closes only
its connection, preserving the attached database. Restore the backed-up legacy
registration and its upstream endpoint to return to the legacy profile.
The legacy producer moves the active GUI as a side effect: assembly and raw function instructions jump to the function, and successful pseudocode opens or reuses a pseudocode view. A function dossier does both. Native GUI decompilation also changes the active view. `binary_session` reports these operations with `effects.may_show_ui: true`; headless operations and the other attached operations report `false`.
## Give REA a target for headless analysis
@@ -72,6 +72,15 @@ code-point order. Dependency manifests remain package observations but do not
declare Electron main or renderer entries. If only dependency manifests are
present, the inventory artifact remains the graph root.
Package observations keep the legacy `renderer` property as a derived entrypoint
candidate: the first nonempty string from `renderer`, `browser`, then `module`.
It does not assert that the source declares a literal `renderer` field.
`renderer_declarations` records the actual present fields: strings, including
empty strings, remain exact; non-string values use `{ "status": "non-string" }`
without copying nested browser mappings. An empty record means no such fields
were present; `null` means package metadata was invalid or unavailable. Renderer
role mechanisms and package load edges name the field that supplied the candidate.
## Select sources before analysis
For a question about known entry sources, supply exact artifact-relative file
+1 -1
View File
@@ -63,7 +63,7 @@ All four preserve target-file bytes. `open_jeb_project` selects or creates an en
- The engine caps `list_units` pages at 100; a full page is reported as `coverage: "partial"` because more units may exist. An empty remainder is `complete` only when fewer units than requested were returned.
- When `decompile_jeb_item` omits `unit_path`, the engine implicitly selects the project's first code unit; REA reports `unit_path: null` and a limitation instead of guessing the effective unit.
- `item_address` is a JEB code-item reference, such as a value returned by the engine's `list_code_items`. Preserve that representation and select its code unit explicitly. For the audited CE SQLite unit, the exported method reference was `sqlite3_libversion_number`; using its raw ELF symbol address did not produce a successful decompilation. File offsets, ELF virtual addresses and engine item references have different semantics. REA currently exposes unit listing, while code-item discovery remains an upstream JEB operation.
- Engine refusals (`success: false` envelopes) are preserved as unsupported-target errors carrying the engine's own message, for example a nonexistent item address.
- Valid engine refusals (`success: false` envelopes) return `execution_failure` with the engine's message when available. Diagnostics preserve the original envelope, including any recovery advice, and the inputs of a refused primary request. If client metadata fails after a primary reply, that reply and the caller's inputs remain in partial Evidence. REA does not infer an unsupported target from a refusal, invoke a recovery tool, or retry the request.
- Successfully parsed client information is reused for the lifetime of the MCP connection, including when the engine omits its version. Explicit client inspection and availability probes refresh it.
- An unreachable endpoint is a provider-availability failure with recovery guidance; REA does not retry or spawn the engine.
+4
View File
@@ -26,6 +26,10 @@ can be passed directly to `compare_functions`, and `inspect_artifact` Evidence
to `compare_artifacts`. Use `get_evidence_bundle` when the task needs broader
retained session history or an explicit bundle for transfer.
`compare_artifacts` can also resolve an artifact summary to its exact retained
parent in the session. Supported historical summaries keep their original
Evidence identity and omissions; a missing or mismatched parent is rejected.
`analyze_function` does not deliver its complete record by default. Without
`detail`, it retains the complete dossier in the session and returns
`inspect_analysis_view` summary Evidence whose
+10 -4
View File
@@ -700,10 +700,16 @@ complete observation.
release their representation reservation for later documents.
Repeated NIBArchive fields retain all serialized values in order. Array
outlet connections resolve each serialized member and retain the collection
object ID and member position on the outlet observation. AppKit object-parent
registries use paired keys and parents; objects paired with `NSRoot` are
top-level, while nested objects keep their recorded parent. Conflicting or
unreadable registry pairs remain incomplete hierarchy evidence.
object ID and member position on the outlet observation. NIB hierarchy uses
recognized child fields, reverse `NSSuperview` links and window content links.
AppKit registry pairs with a resolvable `NSRoot` owner establish top-level
membership; other registry associations alone do not establish visual
parentage and leave hierarchy coverage partial. Known non-view cell ownership
and control-view backreferences remain raw facts. Conflicting explicit parents
remain unknown rather than choosing an archive-order winner. Retained objects
keep their decoded fields; uncertain registry diagnostics retain the source
registry, child and value IDs. Missing, malformed or repeated registry
declarations also qualify hierarchy coverage.
`NSString` objects with the [recognized `NS.bytes` representation](https://github.com/swiftlang/swift-corelibs-foundation/blob/main/Sources/Foundation/NSString.swift#L201-L215) are decoded
as UTF-8 without substituting malformed bytes. A failed conversion retains
the source string object ID and `nibValues.NS.bytes` base64 data; dependent
@@ -30,7 +30,8 @@ natural-language tasks ask for, or withheld them:
matches, which keeps the recorded value as a partial page's total does.
- **Sorted collections.** An update that writes the field its type's creates are sorted by moves its entity to where
the new value sorts, in replay and serving (a bulk update moving each entity in turn), and its plan carries that
`insert`. When a read listing the whole collection fails after an update and the type's creates are not sorted by a
`insert`. When a read listing the whole collection in collection order (not sorted by its query) fails after an
update and the type's creates are not sorted by a
field the update writes, replay tries the collection sorted by such a field when the entities no exercised write
changed already follow it, as serving moves only written entities, and unless a create was placed by a sort
that excludes it or a whole-collection read in collection order (not sorted by its query) that placed a create did
@@ -38,7 +39,7 @@ natural-language tasks ask for, or withheld them:
- **Unloaded answers.** A create's later item read is taken as its observation when it differs from the answer only in
values the answer had not loaded: empty, zero, `false` or Go's zero time, or objects whose members are each the same
or loaded so; the plan points to those values (`observation.read.unloaded`), which a served create's answer keeps as
the recorded answer showed them while the state holds the read's. An unsent update field that changes only between
the recorded answer showed them, unless the call sent one the plan stores as sent, while the state holds the read's. An unsent update field that changes only between
`null`, `[]` and `{}` is unchanged. Write answers, bulk answers and replay's checks of them included, spell empty
lists and maps the call did not send as their recorded answers did, and a sent one as sent.
- **Unruled fields.** A later create that stored a field no rule writes otherwise than the plan keeps it is
+3 -2
View File
@@ -768,7 +768,8 @@ precision, including microseconds. A list read
only adds fields the answer lacks and still checks it. Serving answers a create
in its answer's shape and clock precision, with empty lists and maps the call
did not send spelled (`null`, `[]` or `{}`) as its answer spelled them and the
values it had not loaded as it answered them, and stores the fields of both,
values it had not loaded as it answered them (unless the call sent one the plan
stores as sent), and stores the fields of both,
as the item read-back shows them, with its clock precision. Updates
answer empty lists and maps they did not send as their answers spelled them
too; a member a call sent is answered as sent. An update or delete that answers
@@ -874,7 +875,7 @@ all request completions are known; otherwise it is null. An
update that writes the field its type's creates are sorted by moves its entity
to where the new value sorts (its own `insert`); a bulk update moves each entity
it writes in turn, as replay applies them. When a read listing the whole
collection fails after an update and the type's creates are not sorted by a field
collection in its own order, not one its query sorts, fails after an update and the type's creates are not sorted by a field
that update writes, the view tries the collection sorted by such a field, when
the entities no write it exercised changed already follow it, as serving moves
only the entities a write changes, and unless
-1
View File
@@ -129,7 +129,6 @@
"check:ci:fast:tasks": "node scripts/run-turbo.mjs run typecheck lint:built format:check",
"check:ci:deep": "node scripts/run-exclusive.mjs artifacts npm run check:ci:deep:tasks && npm run verify:architecture-guards && npm run verify:test-discovery && npm run test:python && npm run knip:production && npm run lint:typed",
"check:ci:deep:tasks": "node scripts/run-turbo.mjs run knip metadata:check jscpd scan:todos",
"check:ci:tasks": "node scripts/run-turbo.mjs run typecheck lint:built format:check knip metadata:check jscpd scan:todos",
"ci:static-context": "node scripts/ci/static-task-context.mjs",
"build:cached": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts node scripts/run-turbo.mjs run artifacts:skills",
"build:termux": "node scripts/run-exclusive.mjs artifacts npm run build:termux:run",
@@ -1,4 +1,4 @@
import { functionDossierSchema } from "../../../dist/domain/hopperValues.js";
import { functionDossierSchema } from "../../../dist/domain/native/functionDossier.js";
/** Validated synthetic native observations shared by managed verification lanes. */
export const functionDossier = (name) =>
+1 -1
View File
@@ -1,6 +1,6 @@
import type { z } from "zod";
import type { analysisStringSchema } from "../../src/domain/hopperValues.js";
import type { analysisStringSchema } from "../../src/domain/native/nativeString.js";
type VerifierCall = (
name: string,
+1 -1
View File
@@ -14,7 +14,7 @@ import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { parseEvidence } from "../dist/domain/evidence.js";
import { functionDossierSchema } from "../dist/domain/hopperValues.js";
import { functionDossierSchema } from "../dist/domain/native/functionDossier.js";
import { inspectVerifierGhidraInstallation } from "./lib/ghidra-installation-inspection.mjs";
import { buildDosComFixture } from "../tests/conformance/ghidra/dos-com-fixture.mjs";
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
+1 -1
View File
@@ -14,7 +14,7 @@ import { ArtifactResourceScope } from "../dist/artifacts/ArtifactResourceScope.j
import { readGhidraLanguageCatalog } from "../dist/ghidra/GhidraLanguageCatalog.js";
import { parseEvidence } from "../dist/domain/evidence.js";
import { parseAnalysisSnapshot } from "../dist/domain/analysisSnapshot.js";
import { functionDossierSchema } from "../dist/domain/hopperValues.js";
import { functionDossierSchema } from "../dist/domain/native/functionDossier.js";
import { inspectVerifierGhidraInstallation } from "./lib/ghidra-installation-inspection.mjs";
import { buildGameCubeDolFixture } from "../tests/conformance/ghidra/dol-fixture.mjs";
import {
+2 -4
View File
@@ -21,10 +21,8 @@ import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { parseEvidence } from "../dist/domain/evidence.js";
import {
functionBodySchema,
functionDossierSchema,
} from "../dist/domain/hopperValues.js";
import { functionDossierSchema } from "../dist/domain/native/functionDossier.js";
import { functionBodySchema } from "../dist/domain/native/functionBody.js";
import { inspectVerifierGhidraInstallation } from "./lib/ghidra-installation-inspection.mjs";
import { buildDosMzFixture } from "../tests/conformance/ghidra/dos-mz-fixture.mjs";
import {
@@ -12,7 +12,7 @@ import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
import { parseEvidence } from "../dist/domain/evidence.js";
import { parseAnalysisSnapshot } from "../dist/domain/analysisSnapshot.js";
import { functionDossierSchema } from "../dist/domain/hopperValues.js";
import { functionDossierSchema } from "../dist/domain/native/functionDossier.js";
import { inspectVerifierGhidraInstallation } from "./lib/ghidra-installation-inspection.mjs";
import { buildMipsRelocatableFixture } from "../tests/conformance/ghidra/mips-fixture.mjs";
import {
+1 -1
View File
@@ -21,7 +21,7 @@ import {
} from "../tests/conformance/ghidra/mips-checks.mjs";
import { parseEvidence } from "../dist/domain/evidence.js";
import { parseAnalysisSnapshot } from "../dist/domain/analysisSnapshot.js";
import { functionDossierSchema } from "../dist/domain/hopperValues.js";
import { functionDossierSchema } from "../dist/domain/native/functionDossier.js";
import { inspectVerifierGhidraInstallation } from "./lib/ghidra-installation-inspection.mjs";
import { buildMipsFixture } from "../tests/conformance/ghidra/mips-fixture.mjs";
import {
+1 -1
View File
@@ -18,7 +18,7 @@ import {
} from "../tests/conformance/ghidra/ppc-checks.mjs";
import { parseEvidence } from "../dist/domain/evidence.js";
import { parseAnalysisSnapshot } from "../dist/domain/analysisSnapshot.js";
import { functionDossierSchema } from "../dist/domain/hopperValues.js";
import { functionDossierSchema } from "../dist/domain/native/functionDossier.js";
import { inspectVerifierGhidraInstallation } from "./lib/ghidra-installation-inspection.mjs";
import { buildPowerPcFixture } from "../tests/conformance/ghidra/ppc-fixture.mjs";
import { requireMcpOperationResult } from "./lib/mcp-verifier-results.mjs";
+1 -1
View File
@@ -21,7 +21,7 @@ import addFormats from "ajv-formats";
import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
import { parseAnalysisSnapshot } from "../dist/domain/analysisSnapshot.js";
import { parseEvidence } from "../dist/domain/evidence.js";
import { functionDossierSchema } from "../dist/domain/hopperValues.js";
import { functionDossierSchema } from "../dist/domain/native/functionDossier.js";
import { inspectVerifierGhidraInstallation } from "./lib/ghidra-installation-inspection.mjs";
import { buildRiscvFixture } from "../tests/conformance/ghidra/riscv-fixture.mjs";
import {
+3 -4
View File
@@ -54,13 +54,13 @@ const [
{ toolContract },
{ evidenceSchema },
{ functionDossierSchema },
{ readIdaConfiguration },
{ readIdaConfiguration, idaCompatibilityProfile },
{ createIdaMcpConnection },
{ idaAddressSchema },
] = await Promise.all([
importBuilt("contracts/toolContracts.js"),
importBuilt("domain/evidence.js"),
importBuilt("domain/hopperValues.js"),
importBuilt("domain/native/functionDossier.js"),
importBuilt("ida/IdaConfiguration.js"),
importBuilt("ida/IdaMcpConnection.js"),
importBuilt("ida/IdaProtocolValues.js"),
@@ -91,8 +91,7 @@ const environment = Object.fromEntries(
const report = {
passed: false,
mode: config.mode,
protocol:
config.protocol ?? (config.mode === "attached" ? "legacy" : "modern"),
protocol: idaCompatibilityProfile(config).protocol,
checks: [],
observations: [],
unverified: [],
+2 -4
View File
@@ -6,10 +6,8 @@ import type {
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
import type { JsonValue } from "../domain/jsonValue.js";
import {
analysisBookmarkSchema,
parseContainingProcedureResolution,
} from "../domain/hopperValues.js";
import { analysisBookmarkSchema } from "../domain/native/nativeInventory.js";
import { parseContainingProcedureResolution } from "../domain/native/nativeObservation.js";
import { resolveAnalysisDocument } from "./AnalysisDocument.js";
import { err, ok, type Result } from "../domain/result.js";
+1 -1
View File
@@ -9,7 +9,7 @@ import {
AnalysisOutputError,
} from "../domain/analysisErrorCore.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { parseDocuments } from "../domain/hopperValues.js";
import { parseDocuments } from "../domain/native/nativeDocument.js";
import { err, ok, type Result } from "../domain/result.js";
const executionOptions = (signal: AbortSignal | undefined) =>
+91
View File
@@ -6,6 +6,7 @@ import {
AnalysisCapabilityUnavailableError,
AnalysisOutputError,
} from "../domain/analysisErrorCore.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { err, ok } from "../domain/result.js";
it("uses the provider-owned overview without requesting complete inventories", async () => {
@@ -59,3 +60,93 @@ it.each([
expect(calls).toEqual(["read_binary_inventory_summary"]);
},
);
const fallbackOverview = (procedures: JsonValue, strings: JsonValue) =>
new EnhancedTools({
execute: async (operation) => {
if (
operation === "read_binary_inventory_summary" ||
operation === "list_segments"
)
return err(
new AnalysisCapabilityUnavailableError(
"fixture",
operation,
"Not supported",
),
);
const result =
operation === "current_document"
? "selected"
: operation === "list_procedures"
? procedures
: strings;
return ok(
createAnalysisExecution(result, {
id: "fixture",
name: "Fixture",
version: "1",
}),
);
},
});
it.each(["list_procedures", "list_strings"] as const)(
"rejects malformed successful %s inventories in the public overview workflow",
async (operation) => {
for (const malformed of [
{ error: "inventory unavailable" },
{ "0x1000": "obsolete map" },
{ [operation === "list_procedures" ? "procedures" : "strings"]: [] },
JSON.parse('{"__proto__":"observed"}'),
[null],
[{ address: "0x1000" }],
[{ address: "0x1000", value: 1 }],
]) {
const tools = fallbackOverview(
operation === "list_procedures" ? malformed : [],
operation === "list_strings" ? malformed : [],
);
const result = await tools.execute("binary_overview", {});
expect(result.ok).toBe(false);
if (!result.ok) {
expect(result.error).toBeInstanceOf(AnalysisOutputError);
if (result.error instanceof AnalysisOutputError)
expect(result.error.operation).toBe(operation);
}
}
},
);
it("counts canonical fallback inventories including typed and display-only strings", async () => {
const tools = fallbackOverview(
[{ address: "0x1000", value: "entry" }],
[
{ address: "0x2000", value: "display-only" },
{
address: "0x2000",
value: "typed",
provider_value: "source display",
string: {
encoding: "utf-8",
termination: "present_or_not_required",
byte_length: 5,
},
},
],
);
expect(await tools.execute("binary_overview", {})).toEqual(
ok({
document: "selected",
procedure_count: 1,
string_count: 2,
}),
);
expect(await fallbackOverview([], []).execute("binary_overview", {})).toEqual(
ok({
document: "selected",
procedure_count: 0,
string_count: 0,
}),
);
});
+12 -21
View File
@@ -1,3 +1,7 @@
import {
artifactNameForClassification,
namedArtifactFormat,
} from "../domain/artifactNameSelection.js";
import { readFileHandleRange } from "../filesystem/readFileHandleRange.js";
import { readPeCoffHeader } from "../domain/peCoffHeader.js";
import {
@@ -372,17 +376,11 @@ const readOpenedBinaryTarget = async (
}
// Probe WASM content before suffix-only database routing; an explicit
// database selection keeps the caller's chosen interpretation.
let artifactFormat =
const classificationPath = artifactNameForClassification(selectedPath, path);
const artifactFormat =
targetKind === "database"
? undefined
: await detectArtifactFormat(path, source, signal);
if (
targetKind !== "database" &&
artifactFormat === undefined &&
selectedPath.toLowerCase().endsWith(".asar") &&
!(await claimsLeadingExecutableHeader(handle, metadata.size, signal))
)
artifactFormat = "asar";
: await detectArtifactFormat(classificationPath, source, signal);
if (
targetKind === "database" ||
(targetKind === undefined && path.toLowerCase().endsWith(".hop"))
@@ -398,7 +396,7 @@ const readOpenedBinaryTarget = async (
const identity = {
path,
sourcePath,
...(artifactFormat === "asar" ? { asarSelectedPath: selectedPath } : {}),
artifactSelectedPath: selectedPath,
sha256: await sha256Handle(handle, path, metadata, signal),
};
return isArchiveFormat(artifactFormat)
@@ -424,7 +422,8 @@ const readOpenedBinaryTarget = async (
sourcePath,
sha256: await sha256Handle(handle, path, metadata, signal),
kind: "archive",
format: zipRootPackageFormat(path),
format: zipRootPackageFormat(classificationPath),
artifactSelectedPath: selectedPath,
});
const reason =
typeof detected.error === "string"
@@ -669,7 +668,8 @@ const detectArtifactFormat = async (
// Binary plists, including keyed archives, often use other extensions.
if (hasBinaryPlistSignature(magic)) return "plist";
const named = namedArtifactFormat(lower);
if (named !== undefined) return named;
if (named !== undefined && !claimsExecutableHeader(magic, initial.size))
return named;
if (
lower.endsWith(".pkg") &&
magic.length >= 4 &&
@@ -717,15 +717,6 @@ const detectArtifactFormat = async (
return undefined;
};
const namedArtifactFormat = (
lowerPath: string,
): "asar" | "plist" | "source-map" | "javascript" | undefined => {
if (lowerPath.endsWith(".asar")) return "asar";
if (lowerPath.endsWith(".plist")) return "plist";
if (lowerPath.endsWith(".map")) return "source-map";
return /\.(?:m?js|cjs)$/u.test(lowerPath) ? "javascript" : undefined;
};
const isArchiveFormat = (
format: Exclude<
BinaryTarget["format"],
+1 -1
View File
@@ -8,7 +8,7 @@ import type {
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { parseRelatedAddresses } from "../domain/hopperValues.js";
import { parseRelatedAddresses } from "../domain/native/nativeObservation.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
import type { EnhancedResult } from "./EnhancedToolTypes.js";
+4 -3
View File
@@ -85,11 +85,12 @@ export const readConfigurationFile = async (
}
};
/** Read every applicable user file; malformed lower-priority files also stop setup. */
/** Read each layer; setup may validate its writable layer with proposed overrides. */
export const readClientConfigurationFiles = async (
client: SetupClient,
resources?: ArtifactResourceScope,
platform: NodeJS.Platform = process.platform,
writeOverrides?: Readonly<Record<string, string>>,
): Promise<
Result<readonly ClientConfigurationSource[], ClientConfigurationFileError>
> => {
@@ -104,7 +105,7 @@ export const readClientConfigurationFiles = async (
if (document.value !== undefined) {
const environment = clientRegistrationEnvironment(
document.value,
{},
path === client.configPath ? writeOverrides : undefined,
"fragment",
platform,
);
@@ -115,7 +116,7 @@ export const readClientConfigurationFiles = async (
}
const environment = clientRegistrationEnvironment(
effectiveClientConfiguration(documents, undefined, platform),
{},
writeOverrides,
"effective",
platform,
);
@@ -169,12 +169,12 @@ export class ClientConfigurationPublication {
return outcome.value;
}
async assertCurrent(readback?: string): Promise<void> {
async assertCurrent(readback: string): Promise<void> {
const owned = await this.owner.handle.stat();
const named = await lstat(this.path);
if (
owned.ino === 0 ||
(readback !== undefined && readback !== this.contents) ||
readback !== this.contents ||
named.isSymbolicLink() ||
!sameRegularFileState(owned, named)
)
-11
View File
@@ -138,8 +138,6 @@ export interface DoctorHost {
providerInspections?(): Promise<readonly DoctorProviderInspection[]>;
/** Resolve a parsed upstream request budget from the effective client environment. */
resolveProviderRequestBudget?: ProviderRequestBudgetResolver;
/** Validated provider settings preserved by setup, independent of readiness. */
providerRegistrationEnvironment?(): Readonly<Record<string, string>>;
installationPaths?(): Promise<readonly string[]>;
/** Undefined checks installed skill roots; explicit selections stay scoped. */
installedSkillIdentity?(
@@ -523,9 +521,6 @@ export interface SystemDoctorHostOptions {
resourceScope: RetainedResourceScope,
signal: AbortSignal,
) => Promise<readonly DoctorProviderInspection[]>;
readonly providerRegistrationEnvironment?: () => Readonly<
Record<string, string>
>;
readonly resolveProviderRequestBudget?: (
provider: Parameters<ProviderRequestBudgetResolver>[0],
environment: Parameters<ProviderRequestBudgetResolver>[1],
@@ -726,12 +721,6 @@ export const systemDoctorHost = (
options.providerInspections!(resourceScope, inspectionSignal),
),
}),
...(options.providerRegistrationEnvironment === undefined
? {}
: {
providerRegistrationEnvironment:
options.providerRegistrationEnvironment,
}),
...(resolveProviderRequestBudget === undefined
? {}
: { resolveProviderRequestBudget }),
+6 -9
View File
@@ -1,3 +1,4 @@
import type { LiteralFeatureMatch } from "../domain/native/literalFeatureMatch.js";
import { nativeInventoryRowsSchemas } from "../domain/native/nativeInventory.js";
import type {
AnalysisExecution,
@@ -9,13 +10,9 @@ import type { JsonValue } from "../domain/jsonValue.js";
import {
parseContainingProcedureResolution,
type ContainingProcedureResolution,
} from "../domain/hopperValues.js";
} from "../domain/native/nativeObservation.js";
import { err, ok, type Result } from "../domain/result.js";
import type {
EnhancedResult,
TraceMatch,
TraceReference,
} from "./EnhancedToolTypes.js";
import type { EnhancedResult, TraceReference } from "./EnhancedToolTypes.js";
type AnalysisCall = (
name: AnalysisOperation,
@@ -35,7 +32,7 @@ type LiteralSearchSource = readonly [
];
interface ReferenceTraceRequest {
readonly matches: readonly TraceMatch[];
readonly matches: readonly LiteralFeatureMatch[];
readonly signal?: AbortSignal;
}
@@ -83,8 +80,8 @@ const literalMatches = async (
input: LiteralTraceInput,
sources: readonly LiteralSearchSource[],
signal?: AbortSignal,
): Promise<Result<{ matches: TraceMatch[] }, AnalysisError>> => {
const matches: TraceMatch[] = [];
): Promise<Result<{ matches: LiteralFeatureMatch[] }, AnalysisError>> => {
const matches: LiteralFeatureMatch[] = [];
for (const [tool, type] of sources) {
const result = await call(
tool,
+1 -2
View File
@@ -3,7 +3,7 @@ import type { z } from "zod";
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { JsonValue } from "../domain/jsonValue.js";
import type { ContainingProcedureResolution } from "../domain/hopperValues.js";
import type { ContainingProcedureResolution } from "../domain/native/nativeObservation.js";
import { err, type Result } from "../domain/result.js";
import {
enhancedInputSchemas,
@@ -11,7 +11,6 @@ import {
} from "../contracts/enhancedInputs.js";
export type EnhancedResult = Promise<Result<JsonValue, AnalysisError>>;
export type { LiteralFeatureMatch as TraceMatch } from "../domain/native/literalFeatureMatch.js";
export type TraceReference = {
readonly target_address: string;
readonly source_address: string;
+15 -7
View File
@@ -21,13 +21,16 @@ import {
} from "../domain/analysisErrorCore.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { parseFunctionDossier } from "../domain/native/functionDossier.js";
import {
parseNativeInventoryCount,
addressedValueSchema,
} from "../domain/native/nativeInventory.js";
import { parseRelatedAddresses } from "../domain/native/nativeObservation.js";
import {
parseFunctionDossier,
parseListCount,
parseRelatedAddresses,
parseSegments,
projectBinaryOverview,
} from "../domain/hopperValues.js";
} from "../domain/native/nativeSegments.js";
import { err, ok, type Result } from "../domain/result.js";
import {
categorizeSwiftTypes,
@@ -40,7 +43,6 @@ import { traceNativeValues } from "./native/NativeValueTrace.js";
import { traceNativeUiAction } from "./native/NativeUiActionTrace.js";
import { decompileAll } from "./decompileDirectory/decompileAll.js";
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
import { addressedValueSchema } from "../domain/native/nativeInventory.js";
import {
nativeDispatchMetadataResultSchema,
MAX_DISPATCH_METADATA_RECORDS,
@@ -761,9 +763,15 @@ export class EnhancedTools {
? parseSegments(segmentsResult.value)
: undefined;
if (segments !== undefined && !segments.ok) return segments;
const procedureCount = parseListCount(proceduresResult.value, "procedures");
const procedureCount = parseNativeInventoryCount(
proceduresResult.value,
"list_procedures",
);
if (!procedureCount.ok) return procedureCount;
const stringCount = parseListCount(stringsResult.value, "strings");
const stringCount = parseNativeInventoryCount(
stringsResult.value,
"list_strings",
);
if (!stringCount.ok) return stringCount;
return ok(
+10 -2
View File
@@ -114,7 +114,13 @@ export const authenticateEvidenceBundleValue = (
}),
);
}
const bundle = parsed.value;
return validateBundlePayloads(parsed.value);
};
/** Validate typed payloads without restricting historical workflow or provider identities. */
const validateBundlePayloads = (
bundle: EvidenceBundle,
): Result<EvidenceBundle, EvidenceIntegrityError> => {
for (const record of bundle.records) {
if (record.predicate_type !== "rea.process-capture") continue;
const parsedCapture = safeParseProcessCapture(record.normalized_result);
@@ -158,8 +164,10 @@ export const writeEvidenceBundle = async (
}),
);
}
const validated = validateBundlePayloads(checked.value);
if (!validated.ok) return validated;
return writeTextParts(
bufferedJsonParts(canonicalJsonParts(checked.value)),
bufferedJsonParts(canonicalJsonParts(validated.value)),
path,
overwrite,
resources,
+1 -1
View File
@@ -6,7 +6,7 @@ import type {
import { nativeProcedureInputSchema } from "../domain/native/nativeProcedureInput.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import { createAnalysisExecutionEvidence } from "../domain/analysisObservation.js";
import { parseFunctionDossier } from "../domain/hopperValues.js";
import { parseFunctionDossier } from "../domain/native/functionDossier.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { completeProviderEvidence } from "./ProviderOperationExecution.js";
import type { Evidence } from "../domain/evidence.js";
+2 -2
View File
@@ -13,12 +13,12 @@ import {
LinuxHopperTemporaryCleanupOwner,
systemLinuxHopperInstallHost,
type LinuxDistribution,
type LinuxHopperDownload,
type LinuxHopperInstallHost,
type LinuxHopperLauncherStatus,
type LinuxPackageInstallFailure,
type LinuxPackageInstallOutcome,
} from "./LinuxHopper.js";
import type { HopperDownload } from "./HopperDownload.js";
import { err, ok } from "../domain/result.js";
import {
type LinuxPackageFamily,
@@ -74,7 +74,7 @@ class RecordingLinuxHost implements LinuxHopperInstallHost {
Promise.resolve(this.distributionValue);
release = (family: LinuxPackageFamily) =>
Promise.resolve(ok(releasesFor(new Uint8Array([1, 2, 3]))[family]));
download = (url: string): Promise<LinuxHopperDownload> => {
download = (url: string): Promise<HopperDownload> => {
this.downloads.push(url);
return Promise.resolve({ ok: this.archiveOk, bytes: this.archive });
};
+1 -4
View File
@@ -119,9 +119,6 @@ export const supportsReaHost = (
distribution: LinuxDistribution | undefined,
): boolean => distribution?.packageFamily !== undefined;
/** Download response kept independent of Node's global fetch implementation. */
export type LinuxHopperDownload = HopperDownload;
/** One failed package-manager command, with the output that explains it. */
export interface LinuxPackageInstallFailure {
/** The executed executable and arguments, joined for display. */
@@ -159,7 +156,7 @@ export interface LinuxHopperInstallHost {
download(
url: string,
options: { readonly signal?: AbortSignal },
): Promise<LinuxHopperDownload>;
): Promise<HopperDownload>;
createTemporaryDirectory(): Promise<string>;
writeArchive(path: string, bytes: Uint8Array): Promise<void>;
installPackage(
+7 -3
View File
@@ -1,5 +1,8 @@
import { readElfSectionHeader } from "../domain/elfSectionHeader.js";
import { readElfProgramHeader } from "../domain/elfProgramHeader.js";
import {
readElfProgramHeader,
type ElfProgramHeader,
} from "../domain/elfProgramHeader.js";
import { readElfRawTableDeclarations } from "../domain/elfRawTableDeclarations.js";
import type { FileHandle } from "node:fs/promises";
import type {
@@ -123,7 +126,7 @@ export const readMipsElfMetadata = async (
return err("extended or reserved MIPS ELF table numbering is unsupported");
let record: Buffer | undefined;
let sharedLoad: MipsElfMetadata["sharedLoad"];
let programHeaders: Buffer | undefined;
let programHeaders: ElfProgramHeader[] | undefined;
const relocationSections: MipsElfRelocationSection[] = [];
const limitations: string[] = [];
let gnuAttributes: MipsGnuAttributes | null = null;
@@ -154,7 +157,7 @@ export const readMipsElfMetadata = async (
table.label,
);
if (!contents.ok) return contents;
if (!table.optional && metadata.type === 3) programHeaders = contents.value;
if (!table.optional && metadata.type === 3) programHeaders = [];
let found = false;
for (let i = 0; i < count; i += 1) {
checkCancelled();
@@ -172,6 +175,7 @@ export const readMipsElfMetadata = async (
);
if (!parsedRecord.ok) return err(parsedRecord.error.message);
const declaration = parsedRecord.value;
if ("fileSize" in declaration) programHeaders?.push(declaration);
const { type } = declaration;
const declaredOffset = Number(declaration.offset);
const declaredSize = Number(
+3 -10
View File
@@ -1,6 +1,6 @@
import { readElfRelocationEntry } from "../domain/elfRelocation.js";
import { readElfDynamicEntry } from "../domain/elfDynamicEntry.js";
import { readElfProgramHeader } from "../domain/elfProgramHeader.js";
import type { ElfProgramHeader } from "../domain/elfProgramHeader.js";
import type { MipsElfMetadata } from "../domain/binaryTargetTypes.js";
import { err, ok, type Result } from "../domain/result.js";
@@ -23,7 +23,7 @@ type SharedLoadFacts = Extract<
/** Source ELF facts only; the provider owns interpretation and admission. */
export const readMipsElfSharedLoad = async (
programHeaders: Buffer,
programHeaders: readonly ElfProgramHeader[],
little: boolean,
fileSize: number,
read: ReadRange,
@@ -37,15 +37,8 @@ export const readMipsElfSharedLoad = async (
memorySize: number;
}[] = [];
let dynamic: { offset: number; address: number; size: number } | undefined;
for (let position = 0; position < programHeaders.length; position += 32) {
for (const program of programHeaders) {
checkCancelled();
const parsedProgram = readElfProgramHeader(
programHeaders.subarray(position, position + 32),
32,
little,
);
if (!parsedProgram.ok) return err(parsedProgram.error.message);
const program = parsedProgram.value;
const { type } = program;
const offset = Number(program.offset);
const size = Number(program.fileSize);
+18 -3
View File
@@ -126,7 +126,12 @@ const configureClientDocument = async (
const format = client.format;
if (format === undefined || format === "unsupported")
return { status: "failed", reason: "readback" };
const files = await readClientConfigurationFiles(client);
const files = await readClientConfigurationFiles(
client,
undefined,
undefined,
environment,
);
if (!files.ok) {
throwCleanupFailure(files.error);
return { status: "failed", reason: "readback" };
@@ -420,7 +425,12 @@ export const clientConfigurationAligned = async (
): Promise<boolean> => {
if (client.configPathError !== undefined) return false;
try {
const files = await readClientConfigurationFiles(client);
const files = await readClientConfigurationFiles(
client,
undefined,
undefined,
providerEnvironment,
);
if (!files.ok) {
throwCleanupFailure(files.error);
return false;
@@ -492,7 +502,12 @@ const inspectClientDocument = async (
if (client.configPathError !== undefined)
return { status: "invalid", remediation: client.configPathError };
if (client.format === "unsupported") return { status: "already_current" };
const files = await readClientConfigurationFiles(client);
const files = await readClientConfigurationFiles(
client,
undefined,
undefined,
providerEnvironment,
);
if (!files.ok) {
throwCleanupFailure(files.error);
return {
-1
View File
@@ -231,7 +231,6 @@ export const systemSetupHost = (
const diagnosis = await runDoctor(undefined, doctorHost, scope);
const providerEnvironment: SetupProviderEnvironment = {
...providerRegistrationEnvironment(diagnosis.providerInspections ?? []),
...doctorHost.providerRegistrationEnvironment?.(),
...(diagnosis.hopperPath === undefined
? {}
: { HOPPER_LAUNCHER_PATH: diagnosis.hopperPath }),
@@ -7,6 +7,7 @@ const SELECTED_REGISTRATION_SETTINGS = [
"GHIDRA_MAXMEM",
"JAVA_HOME",
"HOPPER_LAUNCHER_PATH",
"HOPPER_LOADER_ARGS_JSON",
"REA_ILSPY_CMD_PATH",
"REA_ADB_PATH",
"REA_APKTOOL_COMMAND",
@@ -65,6 +66,7 @@ const SELECTED_REGISTRATION_SETTINGS = [
"REA_GHIDRA_PROGRAM_NAME",
"REA_GHIDRA_NATIVEAOT_JAR",
"REA_HOPPER_STARTUP_TIMEOUT_MS",
"REA_JEB_MCP_URL",
"REA_JEB_MCP_TIMEOUT_MS",
"REA_MCP_INPUT_SCHEMA_PROFILE",
"REA_MCP_TOOL_FAMILIES",
+3 -5
View File
@@ -963,11 +963,9 @@ const removeClient = async (
try {
return await publication.verify(async () => {
try {
const readback = parseClientConfiguration(
await fileSystem.readText(publication.path),
client.format,
);
await publication.assertCurrent();
const text = await fileSystem.readText(publication.path);
await publication.assertCurrent(text);
const readback = parseClientConfiguration(text, client.format);
if (
(removeNative && Object.hasOwn(readback.servers, name)) ||
(readback.dialect === "dsh" &&
+1 -1
View File
@@ -16,7 +16,7 @@ import {
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
import { nativeApiBoundarySchema } from "../domain/native/nativeApiBoundary.js";
import { projectNativeApiInspection } from "./native/NativeApiInspection.js";
import { functionDossierSchema } from "../domain/hopperValues.js";
import { functionDossierSchema } from "../domain/native/functionDossier.js";
import { ghidraFunctionDossier } from "../domain/ghidraValues.fixture.js";
import { ANALYSIS_SNAPSHOT_PROFILE } from "../domain/analysisSnapshot.fixture.js";
import { createEvidence } from "../domain/evidence.js";
@@ -20,7 +20,7 @@ import {
rebindEvidenceToTargetPath,
} from "../../domain/evidence.js";
import { inspectAnalysisViewInputSchema } from "../../domain/analysisView/analysisView.js";
import { functionDossierSchema } from "../../domain/hopperValues.js";
import { functionDossierSchema } from "../../domain/native/functionDossier.js";
import { ghidraFunctionDossier } from "../../domain/ghidraValues.fixture.js";
import {
inspectAnalysisView,
@@ -2,10 +2,14 @@ import { expect, it } from "vitest";
import { inspectAnalysisView } from "../analysisView/AnalysisViewService.js";
import { ARTIFACT_COMPARISON_EXAMPLE } from "../../contracts/artifactComparisonExample.js";
import { artifactInventorySummaryViewResultSchema } from "../../domain/analysisView/analysisView.js";
import {
artifactInventorySummaryViewResultSchema,
sealAnalysisView,
} from "../../domain/analysisView/analysisView.js";
import {
createEvidence,
parseEvidence,
rebindEvidenceToTargetPath,
type Evidence,
} from "../../domain/evidence.js";
import {
@@ -13,6 +17,11 @@ import {
EvidenceReferenceError,
} from "../../domain/evidenceErrors.js";
import { resolveArtifactComparisonEvidence } from "./ArtifactComparisonEvidence.js";
import { artifactInventoryResultSchema } from "../../domain/artifactGraph.js";
import {
artifactGraphDigest,
artifactManifestId,
} from "../../domain/artifactIdentity.js";
const artifactSummary = async () => {
const parent = ARTIFACT_COMPARISON_EXAMPLE.left;
@@ -164,3 +173,281 @@ it("accepts display-only subject changes excluded from Evidence identity", async
);
expect(resolved).toEqual({ ok: true, value: parent });
});
const traversalUnknown =
"Producer traversal coverage is unknown; missing matches do not prove source absence.";
const hashesUnknown =
"Content hashes are unknown for some recorded files; missing node IDs do not prove content absence.";
type SummaryEra = "pre-ce1f4b1bb" | "pre-b395190e4" | "current";
/** Recommit an inventory whose two independent unknowns exercise both migrations. */
const unknownInventory = () => {
const original = ARTIFACT_COMPARISON_EXAMPLE.left;
const inventory = artifactInventoryResultSchema.parse(
original.normalized_result,
);
inventory.coverage.status = "unknown";
inventory.occurrences[0]!.hash_status = "unavailable";
const graphSha256 = artifactGraphDigest({
nodes: inventory.nodes,
occurrences: inventory.occurrences,
edges: inventory.edges,
contradictions: inventory.integrity_contradictions,
});
inventory.manifest.graph_sha256 = graphSha256;
inventory.manifest.manifest_id = artifactManifestId(
inventory.manifest.root_artifact_id,
graphSha256,
);
return createEvidence(
{
path: "historical-inventory",
sha256: inventory.manifest.root_sha256,
format: "file",
},
original.provider,
{
operation: "inventory_artifact",
parameters: {},
result: inventory,
confidence: "observed",
authority: "shipped-artifact",
},
);
};
/**
* Exact non-PCK/non-PyInstaller summary deltas checked against the projectors at
* ce1f4b1bb^ and b395190e4^. The first omitted coverage and both caveats; the
* second included coverage/traversal, but still omitted the hash caveat.
*/
const resealSummary = (
result: ReturnType<typeof artifactInventorySummaryViewResultSchema.parse>,
) => {
const { view_digest: _digest, ...unsigned } = result;
return artifactInventorySummaryViewResultSchema.parse(
sealAnalysisView(unsigned),
);
};
const summaryAtEra = async (era: SummaryEra, aliasPath?: string) => {
const parent = unknownInventory();
const selectedParent =
aliasPath === undefined
? parent
: rebindEvidenceToTargetPath(parent, {
path: aliasPath,
sha256: parent.subject!.digest.sha256,
format: parent.subject!.format,
});
const projected = await inspectAnalysisView({
source: { kind: "inline", evidence: selectedParent },
view: { kind: "summary" },
});
if (!projected.ok) throw projected.error;
const current = artifactInventorySummaryViewResultSchema.parse(
projected.value.normalized_result,
);
expect(current.summary.source_coverage).toEqual({
scope: "root",
status: "unknown",
});
expect(current.unknowns).toContain(traversalUnknown);
expect(current.unknowns).toContain(hashesUnknown);
const { source_coverage: _coverage, ...withoutCoverage } = current.summary;
const result = {
...current,
summary: era === "pre-ce1f4b1bb" ? withoutCoverage : current.summary,
unknowns: current.unknowns.filter(
(unknown) =>
era === "current" ||
(unknown !== hashesUnknown &&
(era !== "pre-ce1f4b1bb" || unknown !== traversalUnknown)),
),
};
return {
parent,
summary: recreateSummary(projected.value, resealSummary(result)),
};
};
it.each<SummaryEra>(["pre-ce1f4b1bb", "pre-b395190e4", "current"])(
"authenticates the exact %s summary against its retained inventory",
async (era) => {
const { parent, summary } = await summaryAtEra(era);
expect(resolveArtifactComparisonEvidence(summary, () => parent)).toEqual({
ok: true,
value: parent,
});
},
);
it.each<SummaryEra>(["pre-ce1f4b1bb", "pre-b395190e4", "current"])(
"authenticates a %s summary projected from the caller alias against the original retained parent",
async (era) => {
const { parent, summary } = await summaryAtEra(era, "/selected/alias.bin");
const result = artifactInventorySummaryViewResultSchema.parse(
summary.normalized_result,
);
expect(result.artifact?.path).toBe("/selected/alias.bin");
expect(parent.subject?.local_path).toBe("historical-inventory");
expect(resolveArtifactComparisonEvidence(summary, () => parent)).toEqual({
ok: true,
value: parent,
});
},
);
it.each<SummaryEra>(["pre-ce1f4b1bb", "pre-b395190e4", "current"])(
"authenticates a %s summary after its retained parent is rebound to an alias path",
async (era) => {
const { parent, summary } = await summaryAtEra(era);
const rebound = rebindEvidenceToTargetPath(parent, {
path: "/selected/alias.bin",
sha256: parent.subject!.digest.sha256,
format: parent.subject!.format,
});
expect(rebound.evidence_id).toBe(parent.evidence_id);
expect(resolveArtifactComparisonEvidence(summary, () => rebound)).toEqual({
ok: true,
value: rebound,
});
},
);
it.each([
"coverage-only",
"traversal-only",
"hash-only-without-traversal",
"all-unknowns",
"wrong-count",
"wrong-parent-digest",
"wrong-parameters",
] as const)(
"rejects an unsupported historical summary: %s",
async (variant) => {
const { parent, summary } = await summaryAtEra("current");
const result = artifactInventorySummaryViewResultSchema.parse(
summary.normalized_result,
);
let parameters = summary.parameters;
switch (variant) {
case "coverage-only":
delete result.summary.source_coverage;
break;
case "traversal-only":
result.unknowns = result.unknowns.filter(
(unknown) => unknown !== traversalUnknown,
);
break;
case "hash-only-without-traversal":
delete result.summary.source_coverage;
result.unknowns = result.unknowns.filter(
(unknown) => unknown !== traversalUnknown,
);
break;
case "all-unknowns":
result.unknowns = [];
break;
case "wrong-count":
result.summary.available_counts.occurrences += 1;
break;
case "wrong-parent-digest":
result.parent_digest = "f".repeat(64);
break;
case "wrong-parameters":
parameters = {
...parameters,
parent_evidence_id: `ev_${"f".repeat(64)}`,
};
break;
}
const forged = recreateSummary(summary, resealSummary(result), parameters);
const resolved = resolveArtifactComparisonEvidence(forged, () => parent);
expect(resolved.ok).toBe(false);
if (!resolved.ok)
expect(resolved.error).toBeInstanceOf(EvidenceIntegrityError);
},
);
it.each<SummaryEra>(["pre-ce1f4b1bb", "pre-b395190e4"])(
"rejects a %s summary when lookup returns another inventory",
async (era) => {
const { summary } = await summaryAtEra(era);
const resolved = resolveArtifactComparisonEvidence(
summary,
() => ARTIFACT_COMPARISON_EXAMPLE.right,
);
expect(resolved.ok).toBe(false);
if (!resolved.ok)
expect(resolved.error).toMatchObject({ reason: "identity_mismatch" });
},
);
it.each(["provider", "authority", "locations", "links"] as const)(
"rejects a validly recommitted historical summary with altered %s",
async (field) => {
const { parent, summary } = await summaryAtEra("pre-ce1f4b1bb");
const altered: Evidence = {
...summary,
...(field === "provider"
? { provider: { ...summary.provider, version: "altered" } }
: field === "authority"
? { authority: "analyst-inference" as const }
: field === "locations"
? {
locations: [
{ kind: "artifact-path" as const, path: "different.bin" },
],
}
: {
evidence_links: [ARTIFACT_COMPARISON_EXAMPLE.right.evidence_id],
}),
};
const forged = recreateSummary(altered, summary.normalized_result);
expect(parseEvidence(forged).evidence_id).toBe(forged.evidence_id);
expect(forged.evidence_id).not.toBe(summary.evidence_id);
const resolved = resolveArtifactComparisonEvidence(forged, () => parent);
expect(resolved.ok).toBe(false);
if (!resolved.ok)
expect(resolved.error).toBeInstanceOf(EvidenceIntegrityError);
},
);
it("authenticates a retained parent's content instead of trusting its original ID", async () => {
const { parent, summary } = await summaryAtEra("pre-b395190e4");
const tampered: Evidence = {
...parent,
limitations: ["changed without recomputing the Evidence ID"],
};
expect(tampered.evidence_id).toBe(parent.evidence_id);
const resolved = resolveArtifactComparisonEvidence(summary, () => tampered);
expect(resolved.ok).toBe(false);
if (!resolved.ok)
expect(resolved.error).toBeInstanceOf(EvidenceIntegrityError);
});
it.each<SummaryEra>(["pre-ce1f4b1bb", "pre-b395190e4", "current"])(
"preserves both immutable Evidence commitments when resolving %s",
async (era) => {
const { parent, summary } = await summaryAtEra(era, "/selected/alias.bin");
const before = {
parentId: parent.evidence_id,
summaryId: summary.evidence_id,
parentResult: JSON.stringify(parent.normalized_result),
summaryResult: JSON.stringify(summary.normalized_result),
};
const resolved = resolveArtifactComparisonEvidence(summary, () => parent);
expect(resolved.ok).toBe(true);
if (resolved.ok) expect(resolved.value.evidence_id).toBe(before.parentId);
expect(parent.evidence_id).toBe(before.parentId);
expect(summary.evidence_id).toBe(before.summaryId);
expect(JSON.stringify(parent.normalized_result)).toBe(before.parentResult);
expect(JSON.stringify(summary.normalized_result)).toBe(
before.summaryResult,
);
expect(parseEvidence(parent).evidence_id).toBe(before.parentId);
expect(parseEvidence(summary).evidence_id).toBe(before.summaryId);
},
);
@@ -1,4 +1,7 @@
import type { EvidenceLookup } from "../EvidenceInputResolver.js";
import {
resolveEvidenceInput,
type EvidenceLookup,
} from "../EvidenceInputResolver.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import { inspectAnalysisViewValidated } from "../analysisView/AnalysisViewService.js";
@@ -7,10 +10,18 @@ import {
EvidenceIntegrityError,
EvidenceReferenceError,
} from "../../domain/evidenceErrors.js";
import type { Evidence } from "../../domain/evidence.js";
import {
rebindEvidenceToTargetPath,
type Evidence,
} from "../../domain/evidence.js";
import { readEvidence } from "../../domain/evidenceValidation.js";
import { artifactInventorySummaryProjectionIds } from "../../domain/analysisView/artifactInventorySummaryProjection.js";
import { err, ok, type Result } from "../../domain/result.js";
/** Resolve only an artifact summary's exact retained owner before comparison. */
/**
* Resolve an already-authenticated artifact summary's exact retained owner.
* The caller must authenticate the supplied Evidence before invoking this helper.
*/
export const resolveArtifactComparisonEvidence = (
evidence: Evidence,
evidenceById: EvidenceLookup | undefined,
@@ -43,19 +54,53 @@ export const resolveArtifactComparisonEvidence = (
null,
),
);
const selected = resolveEvidenceInput(
{ kind: "retained-evidence", evidence_id: parentId },
() => parent,
);
if (!selected.ok) return selected;
const authenticated = readEvidence(selected.value);
if (!authenticated.ok) return authenticated;
const owner = authenticated.value;
const subject = evidence.subject;
if (
subject === null ||
owner.subject === null ||
subject.digest.sha256 !== owner.subject.digest.sha256 ||
subject.format !== owner.subject.format ||
subject.architecture !== owner.subject.architecture
)
return mismatch();
// The ledger preserves its original display path; summary delivery may use a
// later caller-selected alias for this same authenticated owner identity.
const presentation = rebindEvidenceToTargetPath(owner, {
path: summary.data.artifact.path,
sha256: subject.digest.sha256,
format: subject.format,
...(subject.architecture === null
? {}
: { architecture: subject.architecture }),
});
const reprojected = inspectAnalysisViewValidated(
{
source: { kind: "retained-evidence", evidence_id: parentId },
view: { kind: "summary" },
},
(evidenceId) => (evidenceId === parentId ? parent : undefined),
(evidenceId) => (evidenceId === parentId ? presentation : undefined),
);
if (!reprojected.ok) return reprojected;
if (reprojected.value.evidence_id !== evidence.evidence_id)
return err(
new EvidenceIntegrityError(
"Artifact summary does not match its retained parent inventory.",
),
);
return ok(parent);
if (
!artifactInventorySummaryProjectionIds(reprojected.value).includes(
evidence.evidence_id,
)
)
return mismatch();
return ok(owner);
};
const mismatch = (): Result<never, EvidenceIntegrityError> =>
err(
new EvidenceIntegrityError(
"Artifact summary does not match its retained parent inventory.",
),
);
@@ -33,6 +33,66 @@ import {
import { EvidenceLedger } from "../investigation/EvidenceLedger.js";
import { enhancedOutputSchemas } from "../../contracts/toolOutputSchemaGroups.js";
it.each(["provider", "workflow"] as const)(
"owns nested %s execution data before returning control to callers",
(kind) => {
const cache = new AnalysisSnapshotCache();
const target = structuredClone(ANALYSIS_SNAPSHOT_TARGET);
const profile = structuredClone(ANALYSIS_SNAPSHOT_PROFILE);
const analysisProfile =
kind === "provider" ? profile : workflowAnalysisProfile(profile);
const execution = createAnalysisExecution(
{ nested: { value: "original" } },
structuredClone(
kind === "provider"
? ANALYSIS_SNAPSHOT_PROVIDER
: REA_WORKFLOW_PROVIDER,
),
{
rawResult: { nested: { value: "raw" } },
subject: target,
analysisProfile,
locations: [{ kind: "address", address: "0x1000" }],
relationshipCoverage: {
status: "partial",
limitations: ["original coverage"],
},
},
);
const input = {
target,
profile,
operation: "procedure_info" as const,
parameters: { address: "0x1000", document: "fixture" },
execution: { ...execution, analysisProfile },
};
if (kind === "provider") cache.record(input);
else cache.recordWorkflow(input);
const entries = () =>
kind === "provider" ? cache.entries() : cache.workflowEntries();
const expected = entries();
for (const payload of [execution.result, execution.rawResult]) {
if (payload === null || typeof payload !== "object")
throw new Error("Expected structured observation payload");
Reflect.set(payload, "nested", { value: "caller edit" });
}
const location = execution.locations[0];
if (location !== undefined) Reflect.set(location, "address", "0x2000");
if (execution.relationshipCoverage?.status === "partial")
execution.relationshipCoverage.limitations.push("caller edit");
Reflect.set(execution.limitations, 0, "caller limitation");
Reflect.set(execution.provider, "name", "caller provider");
Reflect.set(execution.subject, "path", "/tmp/caller-edited-subject");
if (kind === "workflow") {
const upstream = analysisProfile.parameters.upstream_analysis_profile;
if (upstream === null || typeof upstream !== "object")
throw new Error("Expected nested workflow profile");
Reflect.set(upstream, "parameters", { loader: "caller profile" });
}
expect(entries()).toEqual(expected);
},
);
it("exports independent nested query, workflow and Evidence payloads", () => {
const cache = new AnalysisSnapshotCache();
const target = ANALYSIS_SNAPSHOT_TARGET;
@@ -924,7 +984,7 @@ it("does not replay pre-envelope call graphs under the revised workflow profile"
}),
).toBeUndefined();
expect(
enhancedOutputSchemas.get_call_graph?.safeParse({
enhancedOutputSchemas.get_call_graph.safeParse({
normalized_result: oldResult,
}).success,
).toBe(false);
@@ -158,6 +158,7 @@ const isArtifactBundleSubject = (
subject: EvidenceSubjectTarget,
target: BinaryTarget,
): boolean =>
target.bundleInfoPlist !== undefined &&
(operation === "inventory_artifact" ||
operation === "inspect_artifact" ||
operation === "extract_artifact") &&
+1 -1
View File
@@ -152,7 +152,7 @@ const sameSessionTarget = (
current.path === target.path &&
// Native admission is bound to the requested path, not just its realpath.
(current.sourcePath ?? current.path) === (target.sourcePath ?? target.path) &&
// ASAR companion lookup depends on the selected archive spelling.
// Artifact name hints and companions depend on the selected spelling.
artifactTargetSelectedPath(current) === artifactTargetSelectedPath(target) &&
snapshotMatchesTarget(snapshotTarget(current), target);
+13 -10
View File
@@ -1,13 +1,16 @@
import type { BinaryTarget } from "../../domain/binaryTargetTypes.js";
import { liveBinaryTargetSummarySchema } from "../../domain/liveBinaryTargetSummary.js";
/** Preserve the admitted target interpretation in public lifecycle results. */
export const binaryTargetSummary = (target: BinaryTarget) => ({
path: target.path,
format: target.format,
kind: target.kind,
sha256: target.sha256,
architecture: target.architecture ?? null,
...(target.format === "raw-binary" || target.format === "gamecube-dol"
? { entry_point: target.entryPoint }
: {}),
});
export const binaryTargetSummary = (target: BinaryTarget) =>
liveBinaryTargetSummarySchema.parse({
path: target.path,
format: target.format,
kind: target.kind,
sha256: target.sha256,
architecture: target.architecture ?? null,
...(target.format === "raw-binary" || target.format === "gamecube-dol"
? { entry_point: target.entryPoint }
: {}),
});
+2 -2
View File
@@ -12,8 +12,8 @@ import type { Result } from "../../domain/result.js";
/** Replaceable read-only Electron fuse reader with no target-execution authority. */
export interface ElectronFusePort {
readonly identity: ProviderIdentity;
/** Release owned read resources; legacy read-only ports may own none. */
close?(): Promise<void>;
/** Release owned read resources, retrying retained cleanup failures. */
close(): Promise<void>;
inspect(
input: InspectElectronFusesInput,
options?: ExecutionOptions,
@@ -32,6 +32,7 @@ it("retains completed fuse Evidence when the provider completes after abort", as
const controller = new AbortController();
const result = await new ElectronFuseService({
identity,
close: async () => {},
inspect: async () => {
controller.abort();
return ok(report);
@@ -58,6 +59,7 @@ it("preserves malformed fuse output ahead of late cancellation", async () => {
const controller = new AbortController();
const invalid = await new ElectronFuseService({
identity,
close: async () => {},
inspect: async () => {
controller.abort();
return ok(malformed);
@@ -20,7 +20,7 @@ export class ElectronFuseService {
/** Await and retry the reader's exact retained descriptor owners. */
async close(): Promise<void> {
await this.provider.close?.();
await this.provider.close();
}
/** Read fuse wires for one explicitly selected path without launching it. */
@@ -107,9 +107,6 @@ export interface EnvironmentBuildInput {
readonly exclusiveDestination?: boolean | undefined;
}
/** The report environment-build prints and export_environment_bundle returns. */
export type { EnvironmentBuildReport };
const captureOperations = new Set([
"inspect_web_network_capture",
"capture_browser_scenario",
@@ -1,6 +1,7 @@
import { describe, expect, it } from "vitest";
import {
ACTIVE_ON_READ_ROWS,
BULK_RENAMED_LABEL_ROWS,
ECHOED_EMPTY_ROWS,
formLabelRows,
@@ -146,6 +147,18 @@ describe("served creates and updates modelled on form recordings", () => {
});
});
it("answers a sent value its create does not store as the recorded answer did, and stores the read's", () => {
const users = servedFrom(ACTIVE_ON_READ_ROWS);
expect(
users.answer("post_users", { body: { name: "Cy", active: false } }).body,
).toEqual({ id: 2, name: "Cy", active: false });
expect(users.answer("get_users_by_id", { id: "2" }).body).toEqual({
id: 2,
name: "Cy",
active: true,
});
});
it("lists a created tag's id in the list envelope's ids, in list order", () => {
const tags = servedFrom(LISTED_ID_ROWS);
tags.answer("post_tags", { body: { name: "Lamp" } });
@@ -49,6 +49,7 @@ import {
collectionMatch,
decodedSegment,
isObject,
pointerTokens,
scalarText,
type JsonObject,
type StateEntity,
@@ -407,7 +408,18 @@ export const writeAnswer = (
rule.rule === "clock" ? [rule.field] : [],
)
: [],
unloaded: plan.observation?.read?.unloaded ?? [],
// A member the call sent and the plan stores as sent is answered as sent; any other keeps the recorded answer.
unloaded: (plan.observation?.read?.unloaded ?? []).filter(
(path) => {
const [field] = pointerTokens(path);
return !(
rendered.sent?.includes(field!) === true &&
plan.rules.some(
(rule) => rule.field === field && rule.rule === "echo",
)
);
},
),
},
optionalFields(plan),
laterItemFields(
+1 -1
View File
@@ -12,7 +12,7 @@ import type { Result } from "../../domain/result.js";
/** Stable read and BEAM IFF decoding, independent of a disassembler session. */
export interface BeamPort {
readonly identity: ProviderIdentity;
close?(): Promise<void>;
close(): Promise<void>;
inspect(
input: InspectErlangBeamInput,
options?: ExecutionOptions,
@@ -31,6 +31,7 @@ const report: ErlangBeam = {
it("rejects a relative path and extra arguments before reading", async () => {
let calls = 0;
const service = new BeamService({
close: async () => {},
identity,
inspect: () => {
calls += 1;
@@ -62,6 +63,7 @@ it("rejects a relative path and extra arguments before reading", async () => {
it("records the selected BEAM form", async () => {
const result = await new BeamService({
close: async () => {},
identity,
inspect: () => Promise.resolve(ok(report)),
}).inspect({ path: report.artifact.path });
@@ -78,6 +80,7 @@ it("discards an observation if the caller cancels while the provider completes",
const controller = new AbortController();
const reason = { source: "caller-request", operation: "cancel" };
const service = new BeamService({
close: async () => {},
identity,
inspect: () => {
controller.abort(reason);
+1 -1
View File
@@ -19,7 +19,7 @@ export class BeamService {
constructor(readonly provider: BeamPort) {}
close(): Promise<void> {
return this.provider.close?.() ?? Promise.resolve();
return this.provider.close();
}
async inspect(
@@ -1,4 +1,5 @@
import { expect, it } from "vitest";
import { expect, it, vi } from "vitest";
import { ZodError } from "zod";
import { FirmwareAnalysisService } from "./FirmwareAnalysisService.js";
import { createAnalysisExecution } from "../AnalysisProvider.js";
@@ -65,6 +66,44 @@ const fixture = (
return { input, execution };
};
it.each([
{
input: {},
issue: { path: ["path"], reason: "missing_argument" },
},
{
input: { path: 17 },
issue: { path: ["path"], reason: "invalid_type" },
},
{
input: { path: "/firmware.bin", range: { offset: -1, length: 1 } },
issue: { path: ["range", "offset"], reason: "out_of_range" },
},
{
input: { path: "/firmware.bin", unexpected: "caller value" },
issue: { path: ["unexpected"], reason: "unknown_argument" },
},
])(
"rejects malformed firmware inputs with caller-relative issues: $issue",
async ({ input, issue }) => {
const execute = vi.fn<FirmwareAnalysisService["provider"]["execute"]>();
const result = await new FirmwareAnalysisService({ execute }).execute(
"extract_firmware",
{ output_directory: "/output", ...input },
);
expect(execute).not.toHaveBeenCalled();
expect(result).toMatchObject({
ok: false,
error: {
_tag: "AnalysisInputError",
operation: "extract_firmware",
cause: expect.any(ZodError),
issues: expect.arrayContaining([expect.objectContaining(issue)]),
},
});
},
);
it.each(["inspect_firmware_regions", "extract_firmware"] as const)(
"retains %s Evidence after late cancellation and preserves provider failure precedence",
async (operation) => {
@@ -9,12 +9,11 @@ import {
} from "../../domain/firmware/firmwareAnalysis.js";
import {
AnalysisCancelledError,
AnalysisInputError,
AnalysisOutputError,
} from "../../domain/analysisErrorCore.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { Evidence } from "../../domain/evidence.js";
import { projectInputIssues } from "../../domain/inputIssueProjection.js";
import { analysisOperationInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
import { err, type Result } from "../../domain/result.js";
import { completeProviderEvidence } from "../ProviderOperationExecution.js";
@@ -39,17 +38,11 @@ export class FirmwareAnalysisService {
const parsed = firmwareRequestSchema.safeParse({ operation, input });
if (!parsed.success)
return err(
new AnalysisInputError(
analysisOperationInputErrorFromIssues(
operation,
{ cause: parsed.error },
projectInputIssues(parsed.error.issues, { operation, input }).map(
(issue) => ({
...issue,
// `input` is the internal discriminated request envelope.
path:
issue.path[0] === "input" ? issue.path.slice(1) : issue.path,
}),
),
parsed.error.issues,
input,
parsed.error,
),
);
const executed = await this.provider.execute(parsed.data, options);
+1 -1
View File
@@ -12,7 +12,7 @@ import type { Result } from "../../domain/result.js";
/** Replaceable offline build-information reader with no target-execution authority. */
export interface GoBinaryPort {
readonly identity: ProviderIdentity;
close?(): Promise<void>;
close(): Promise<void>;
inspect(
input: Omit<InspectGoBinaryInput, "detail">,
options?: ExecutionOptions,
@@ -28,6 +28,7 @@ const report: GoBinary = {
it("rejects invalid paths and extra caller arguments before invoking the reader", async () => {
let calls = 0;
const service = new GoBinaryService({
close: async () => {},
identity,
inspect: () => {
calls += 1;
@@ -49,6 +50,7 @@ it("rejects invalid paths and extra caller arguments before invoking the reader"
it("requires a complete Evidence retention path before starting summary inspection", async () => {
let calls = 0;
const service = new GoBinaryService({
close: async () => {},
identity,
inspect: async () => {
calls++;
@@ -67,6 +69,7 @@ it("requires a complete Evidence retention path before starting summary inspecti
it("preserves unsupported subject architectures and does not duplicate metadata", async () => {
const service = new GoBinaryService({
close: async () => {},
identity,
inspect: () => Promise.resolve(ok(report)),
});
@@ -113,6 +116,7 @@ it("preserves byte-valued build metadata and validates compiler source lengths",
limitations: ["Some embedded strings contain non-UTF-8 bytes."],
};
const result = await new GoBinaryService({
close: async () => {},
identity,
inspect: () => Promise.resolve(ok(partial)),
}).inspect({ path: report.artifact.path });
@@ -128,6 +132,7 @@ it("preserves byte-valued build metadata and validates compiler source lengths",
},
};
const invalid = await new GoBinaryService({
close: async () => {},
identity,
inspect: () => Promise.resolve(ok(mismatched)),
}).inspect({ path: report.artifact.path });
@@ -163,6 +168,7 @@ it("refuses reader identity changes and invalid original source ranges", async (
outside,
]) {
const result = await new GoBinaryService({
close: async () => {},
identity,
inspect: () => Promise.resolve(ok(value)),
}).inspect({ path: report.artifact.path });
@@ -176,6 +182,7 @@ it("honors cancellation before reading and after a completed observation", async
const reason = { request: "late Go cancellation" };
let calls = 0;
const service = new GoBinaryService({
close: async () => {},
identity,
inspect: () => {
calls += 1;
@@ -212,6 +219,7 @@ it("keeps malformed provider output ahead of a late cancellation", async () => {
artifact: { ...report.artifact, path: "/other" },
};
const result = await new GoBinaryService({
close: async () => {},
identity,
inspect: () => {
controller.abort();
@@ -240,6 +248,7 @@ it.each([
controller.abort(reason);
let calls = 0;
const service = new GoBinaryService({
close: async () => {},
identity,
inspect: async () => {
calls += 1;
+1 -1
View File
@@ -29,7 +29,7 @@ export class GoBinaryService {
constructor(readonly provider: GoBinaryPort) {}
close(): Promise<void> {
return this.provider.close?.() ?? Promise.resolve();
return this.provider.close();
}
/** Inspect one explicitly selected file without acquiring a binary session. */
+1 -1
View File
@@ -12,7 +12,7 @@ import type { Result } from "../../domain/result.js";
/** Replaceable offline metadata reader with no target-execution or decryption authority. */
export interface Il2cppMetadataPort {
readonly identity: ProviderIdentity;
close?(): Promise<void>;
close(): Promise<void>;
inspect(
input: Il2cppMetadataProviderInput,
options?: ExecutionOptions,
@@ -23,6 +23,7 @@ const report = createEmptyIl2cppMetadataFixture();
const serviceReturning = (value: Il2cppMetadata, calls = { count: 0 }) =>
new Il2cppMetadataService({
close: async () => {},
identity,
inspect: () => {
calls.count += 1;
@@ -104,6 +105,7 @@ it("reports cancellation before and after reading", async () => {
const during = new AbortController();
const service = new Il2cppMetadataService({
close: async () => {},
identity,
inspect: () => {
during.abort();
@@ -158,7 +160,11 @@ it("preserves source Evidence and requested output choices when publication fail
path: "/output/seeds.tsv",
});
const service = new Il2cppMetadataService(
{ identity, inspect: async () => ok(linkedReport()) },
{
close: async () => {},
identity,
inspect: async () => ok(linkedReport()),
},
seedWriter(async () => err(failure)),
);
const result = await service.inspectWithGhidraSeeds(selectedPair, {
@@ -192,7 +198,11 @@ it("retains a published seed receipt if cancellation arrives after the atomic wr
generic_instances: null,
};
const service = new Il2cppMetadataService(
{ identity, inspect: async () => ok(linkedReport()) },
{
close: async () => {},
identity,
inspect: async () => ok(linkedReport()),
},
seedWriter(async () => {
controller.abort(reason);
return ok(receipt);
@@ -44,7 +44,7 @@ export class Il2cppMetadataService {
async close(): Promise<void> {
let failure: AnalysisError | undefined;
for (const close of [
() => this.provider.close?.(),
() => this.provider.close(),
() => this.seeds?.close(),
]) {
try {
@@ -2,23 +2,26 @@ import { expect, it } from "vitest";
import { analyzeArtifactFiles } from "../../../tests/fixtures/javascriptArtifactAnalysis.js";
const packageObservation = async (text: string) => {
const packageObservation = async (text: string | null) => {
const { analysis } = await analyzeArtifactFiles({
files: [
{
path: "app/package.json",
container_sha256: "a".repeat(64),
sha256: "b".repeat(64),
bytes: Buffer.byteLength(text),
bytes: Buffer.byteLength(text ?? ""),
inventory_artifact_id: "package-main-fixture",
kind: "package-json",
unpacked: false,
text: { included: true, value: text },
text:
text === null
? { included: false, reason: "invalid-utf8" }
: { included: true, value: text },
},
],
unavailable_relevant_files: [],
containers: [],
text_bytes_read: Buffer.byteLength(text),
text_bytes_read: Buffer.byteLength(text ?? ""),
invalid_utf8_files: 0,
});
return analysis.packages[0];
@@ -36,7 +39,7 @@ it("retains declared package metadata and normal defaults", async () => {
main: null,
main_status: "default",
main_entrypoint: "",
renderer: null,
renderer_declarations: {},
});
expect(declared).toMatchObject({
status: "included",
@@ -45,10 +48,60 @@ it("retains declared package metadata and normal defaults", async () => {
main: null,
main_status: "invalid",
main_entrypoint: "",
renderer: "source-browser.cjs",
renderer_declarations: { browser: "source-browser.cjs" },
renderer_entrypoint: { field: "browser", path: "source-browser.cjs" },
});
});
it.each([
{
fields: {
renderer: "renderer.js",
browser: "browser.js",
module: "module.js",
},
field: "renderer",
path: "renderer.js",
},
{
fields: { renderer: false, browser: "browser.js", module: "module.js" },
field: "browser",
path: "browser.js",
},
{
fields: { renderer: "", browser: {}, module: "module.js" },
field: "module",
path: "module.js",
},
])(
"keeps simultaneous renderer declarations separate from selected $field provenance",
async ({ fields, field, path }) => {
const observation = await packageObservation(JSON.stringify(fields));
expect(observation).toMatchObject({ renderer_entrypoint: { field, path } });
for (const [key, value] of Object.entries(fields))
expect(observation).toMatchObject({
renderer_declarations: {
[key]: typeof value === "string" ? value : { status: "non-string" },
},
});
},
);
it.each([
["{}", {}],
["{", null],
["null", null],
[null, null],
] as const)(
"keeps absent/invalid/unavailable renderer origins distinct for %s",
async (text, declarations) => {
expect(await packageObservation(text)).toMatchObject({
renderer_declarations: declarations,
renderer_entrypoint: null,
});
},
);
it.each([null, false, 123, {}, []].map((main) => ({ main })))(
"retains non-string main $main evidence separately from Node index fallback",
async ({ main }) => {
@@ -379,6 +379,23 @@ const parsePackage = (
if (document.packageFields === null)
return invalidPackage(file, "package.json root is not an object.");
const value = document.packageFields;
const rendererDeclarations: NonNullable<
JavaScriptPackageObservation["renderer_declarations"]
> = {};
let rendererEntrypoint: JavaScriptPackageObservation["renderer_entrypoint"] =
null;
for (const field of ["renderer", "browser", "module"] as const) {
if (!Object.hasOwn(value, field)) continue;
const declaration = value[field];
rendererDeclarations[field] =
typeof declaration === "string" ? declaration : { status: "non-string" };
if (
rendererEntrypoint === null &&
typeof declaration === "string" &&
declaration !== ""
)
rendererEntrypoint = { field, path: declaration };
}
return {
path: file.path,
sha256: file.sha256,
@@ -393,10 +410,8 @@ const parsePackage = (
? "declared"
: "invalid",
main_entrypoint: typeof value.main === "string" ? value.main : "",
renderer:
optionalString(value.renderer) ??
optionalString(value.browser) ??
optionalString(value.module),
renderer_declarations: rendererDeclarations,
renderer_entrypoint: rendererEntrypoint,
limitation: null,
};
};
@@ -421,7 +436,8 @@ const unavailablePackage = (
main: null,
main_status: "unavailable",
main_entrypoint: null,
renderer: null,
renderer_declarations: null,
renderer_entrypoint: null,
limitation,
});
@@ -20,6 +20,14 @@ interface JavaScriptStructuredObservation {
readonly sha256: string;
}
/** Present source declarations only; non-strings stay bounded and retain source ownership. */
export type JavaScriptPackageRendererDeclarations = Partial<
Record<
"renderer" | "browser" | "module",
string | { readonly status: "non-string" }
>
>;
interface JavaScriptPackageValues {
readonly name: string | null;
readonly version: string | null;
@@ -27,7 +35,12 @@ interface JavaScriptPackageValues {
readonly main_status: "default" | "declared" | "invalid";
/** Node launch declaration; an empty string requests the package index fallback. */
readonly main_entrypoint: string;
readonly renderer: string | null;
readonly renderer_declarations: JavaScriptPackageRendererDeclarations;
/** Selected static renderer candidate, retaining the declaration that supplied it. */
readonly renderer_entrypoint: {
readonly field: "renderer" | "browser" | "module";
readonly path: string;
} | null;
}
/** Parsed package metadata used only for static entrypoint discovery. */
@@ -44,7 +57,8 @@ export type JavaScriptPackageObservation = JavaScriptStructuredObservation &
readonly main: null;
readonly main_status: "unavailable";
readonly main_entrypoint: null;
readonly renderer: null;
readonly renderer_declarations: null;
readonly renderer_entrypoint: null;
readonly limitation: string;
}
);
@@ -492,7 +492,7 @@ const addDiscoveredRole = (
changedRoots: ReadonlySet<string> | null,
): void => {
const root = applicationRootPath(context, input.file);
const resolution = resolveArtifactPathByContext({
const resolved = resolveArtifactPathByContext({
declaredPath: input.value.path,
sourcePath: input.file.path,
context: input.value.resolution_context,
@@ -500,7 +500,8 @@ const addDiscoveredRole = (
jsonDocuments: context.analysis.jsonDocuments,
directoryPaths: context.directoryPaths,
unanalyzedFiles: context.unanalyzedFiles,
...(input.value.role === "preload"
...(input.value.role === "preload" ||
input.value.path_base === "source-directory"
? { runtimePathKind: "exact-file" as const }
: input.value.role === "renderer" &&
input.value.resolution_context === "filesystem-expression"
@@ -520,6 +521,16 @@ const addDiscoveredRole = (
? { runtimePathKind: "url-resource" as const }
: {}),
});
const resolution =
input.value.path_base === "source-directory"
? {
...resolved,
limitations: [
...resolved.limitations,
"The source-directory anchor is inferred from authored code; bundled or transformed runtime path behavior was not verified.",
],
}
: resolved;
const role = createElectronRoleNode(context, {
kind:
input.value.role === "preload" ? "electron-preload" : "electron-renderer",
@@ -543,7 +554,12 @@ const addDiscoveredRole = (
range: input.value.location,
coverage: input.coverage,
relation: "loads",
properties: { mechanism: input.value.mechanism },
properties: {
mechanism: input.value.mechanism,
...(input.value.path_base === undefined
? {}
: { path_base: input.value.path_base }),
},
});
linkElectronRoleToAsset(context, {
role,
@@ -53,6 +53,7 @@ interface PackageRoleInput {
readonly packageFile: JavaScriptArtifactFile;
readonly kind: "electron-main" | "electron-renderer";
readonly declaredPath: string | null;
readonly field: "main" | "renderer" | "browser" | "module";
}
/** Create the exact root artifact node from the inventory manifest. */
@@ -552,7 +553,9 @@ export const addJavaScriptPackageNodes = (
version: packageValue.version,
main: packageValue.main,
main_status: packageValue.main_status,
renderer: packageValue.renderer,
// Legacy derived candidate; source fields live in renderer_declarations.
renderer: packageValue.renderer_entrypoint?.path ?? null,
renderer_declarations: packageValue.renderer_declarations,
parse_status: packageValue.status,
},
evidence: artifactObservationEvidence({
@@ -612,13 +615,16 @@ export const addJavaScriptPackageNodes = (
packageFile: file,
kind: "electron-main",
declaredPath: packageValue.main,
field: "main",
});
addPackageRole(context, {
packageNode: node,
packageFile: file,
kind: "electron-renderer",
declaredPath: packageValue.renderer,
});
if (packageValue.renderer_entrypoint !== null)
addPackageRole(context, {
packageNode: node,
packageFile: file,
kind: "electron-renderer",
declaredPath: packageValue.renderer_entrypoint.path,
field: packageValue.renderer_entrypoint.field,
});
}
return roots;
};
@@ -755,16 +761,14 @@ const addPackageRole = (
kind: input.kind,
anchor: input.packageFile,
resolution,
mechanism:
input.kind === "electron-main"
? "package.json:main"
: "package.json:renderer",
mechanism: `package.json:${input.field}`,
});
context.accumulator.addEdge({
source_node_id: input.packageNode.node_id,
target_node_id: role.node_id,
relation: "loads",
properties: {
source_field: input.field,
declared_path: input.declaredPath,
resolution_context: resolution.resolution_context,
resolved_path: resolution.resolved_path,
@@ -1,3 +1,4 @@
import { artifactNameForClassification } from "../../domain/artifactNameSelection.js";
import type { AnalysisSettingValues } from "../../domain/analysisSettings.js";
import { absoluteCallerPath } from "../../filesystem/callerPath.js";
import {
@@ -500,8 +501,9 @@ const resolveFormat = (
const observed = metadata.isDirectory()
? "directory"
: metadata.isFile() &&
(selectedPath.toLowerCase().endsWith(".asar") ||
path.toLowerCase().endsWith(".asar"))
artifactNameForClassification(selectedPath, path)
.toLowerCase()
.endsWith(".asar")
? "asar"
: undefined;
if (observed === undefined)
@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { functionDossierSchema } from "../../domain/hopperValues.js";
import { functionDossierSchema } from "../../domain/native/functionDossier.js";
import { ghidraFunctionDossier } from "../../domain/ghidraValues.fixture.js";
import { nativeApiBoundarySchema } from "../../domain/native/nativeApiBoundary.js";
import { projectNativeApiInspection } from "./NativeApiInspection.js";
@@ -1,4 +1,4 @@
import type { FunctionDossier } from "../../domain/hopperValues.js";
import type { FunctionDossier } from "../../domain/native/functionDossier.js";
import {
nativeApiInspectionResultSchema,
type NativeApiBoundary,
+2 -4
View File
@@ -4,10 +4,8 @@ import type {
ExecutionOptions,
ProviderIdentity,
} from "../AnalysisProvider.js";
import {
parseRelatedAddresses,
referenceKindSchema,
} from "../../domain/hopperValues.js";
import { parseRelatedAddresses } from "../../domain/native/nativeObservation.js";
import { referenceKindSchema } from "../../domain/native/nativeReference.js";
import { AnalysisOutputError } from "../../domain/analysisErrorCore.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import { err, ok, type Result } from "../../domain/result.js";
@@ -5,7 +5,7 @@ import { nativeFunctionMatchingSchema } from "../../domain/native/functionMatchi
import { createEvidence, immutableEvidence } from "../../domain/evidence.js";
import { createAnalysisProfile } from "../../domain/analysisProfile.js";
import { AnalysisResourceConstraintError } from "../../domain/analysisErrorCore.js";
import { functionDossierSchema } from "../../domain/hopperValues.js";
import { functionDossierSchema } from "../../domain/native/functionDossier.js";
import { procedureInfoSchema } from "../../domain/native/procedureInfo.js";
import {
nativeMatchingNamedInventory as nativeMatchingInventory,
@@ -5,7 +5,7 @@ import {
type AnalysisOperationPort,
} from "../AnalysisProvider.js";
import { ghidraFunctionDossier } from "../../domain/ghidraValues.fixture.js";
import { functionDossierSchema } from "../../domain/hopperValues.js";
import { functionDossierSchema } from "../../domain/native/functionDossier.js";
import { jsonValueSchema } from "../../domain/jsonValue.js";
import { nativeValueTraceSchema } from "../../domain/native/nativeValueTrace.js";
import {
+1 -1
View File
@@ -1,4 +1,4 @@
import { functionDossierSchema } from "../../domain/hopperValues.js";
import { functionDossierSchema } from "../../domain/native/functionDossier.js";
import { nativeCallTargetsSchema } from "../../domain/native/nativeInstruction.js";
import {
nativeValueTraceInputSchema,
+1 -1
View File
@@ -12,7 +12,7 @@ import type { Result } from "../../domain/result.js";
/** Stable read and CPython pyc decoding, independent of a disassembler session. */
export interface PythonPycPort {
readonly identity: ProviderIdentity;
close?(): Promise<void>;
close(): Promise<void>;
inspect(
input: InspectPythonPycInput,
options?: ExecutionOptions,
@@ -69,6 +69,7 @@ const report: PythonPyc = {
it("rejects a relative path and extra arguments before reading", async () => {
let calls = 0;
const service = new PythonPycService({
close: async () => {},
identity,
inspect: () => {
calls += 1;
@@ -89,6 +90,7 @@ it("rejects a relative path and extra arguments before reading", async () => {
it("records the selected artifact and the header plus marshal ranges", async () => {
const result = await new PythonPycService({
close: async () => {},
identity,
inspect: () => Promise.resolve(ok(report)),
}).inspect({ path: report.artifact.path });
@@ -110,6 +112,7 @@ it("retains completed pyc Evidence and the opaque caller reason on late cancella
const controller = new AbortController();
const reason = new AnalysisInputError("opaque caller metadata");
const service = new PythonPycService({
close: async () => {},
identity,
inspect: () => {
controller.abort(reason);
@@ -135,6 +138,7 @@ it("retains a pre-admission cancellation reason without calling the pyc provider
const reason = new Error("caller stopped before pyc admission");
let calls = 0;
const service = new PythonPycService({
close: async () => {},
identity,
inspect: async () => {
calls++;
@@ -155,6 +159,7 @@ it("retains a pre-admission cancellation reason without calling the pyc provider
it("preserves malformed provider output instead of relabeling it as a late abort", async () => {
const controller = new AbortController();
const service = new PythonPycService({
close: async () => {},
identity,
inspect: async () => {
controller.abort(new Error("later caller stop"));
+1 -1
View File
@@ -23,7 +23,7 @@ export class PythonPycService {
constructor(readonly provider: PythonPycPort) {}
close(): Promise<void> {
return this.provider.close?.() ?? Promise.resolve();
return this.provider.close();
}
async inspect(
+1 -1
View File
@@ -12,7 +12,7 @@ import type { Result } from "../../domain/result.js";
/** Replaceable offline dependency reader with no target-execution authority. */
export interface RustBinaryPort {
readonly identity: ProviderIdentity;
close?(): Promise<void>;
close(): Promise<void>;
inspect(
input: InspectRustBinaryInput,
options?: ExecutionOptions,
@@ -51,6 +51,7 @@ const report: RustBinary = {
it("rejects a relative path and extra arguments before reading", async () => {
let calls = 0;
const service = new RustBinaryService({
close: async () => {},
identity,
inspect: () => {
calls += 1;
@@ -71,6 +72,7 @@ it("rejects a relative path and extra arguments before reading", async () => {
it("records the selected artifact and the dependency section range", async () => {
const result = await new RustBinaryService({
close: async () => {},
identity,
inspect: () => Promise.resolve(ok(report)),
}).inspect({ path: report.artifact.path });
@@ -90,6 +92,7 @@ it("records the selected artifact and the dependency section range", async () =>
artifact: { ...report.artifact, path: "/artifacts/other" },
};
const invalid = await new RustBinaryService({
close: async () => {},
identity,
inspect: () => Promise.resolve(ok(mismatched)),
}).inspect({ path: report.artifact.path });
@@ -107,6 +110,7 @@ it("rejects a cycle claim the package dependencies do not contain", async () =>
},
};
const result = await new RustBinaryService({
close: async () => {},
identity,
inspect: () => Promise.resolve(ok(invalidReport)),
}).inspect({ path: report.artifact.path });
@@ -118,6 +122,7 @@ it("retains completed Rust Evidence on late cancellation and keeps output failur
const controller = new AbortController();
const reason = { request: "late Rust cancellation" };
const service = new RustBinaryService({
close: async () => {},
identity,
inspect: () => {
controller.abort(reason);
@@ -142,6 +147,7 @@ it("retains completed Rust Evidence on late cancellation and keeps output failur
artifact: { ...report.artifact, path: "/artifacts/other" },
};
const invalid = await new RustBinaryService({
close: async () => {},
identity,
inspect: () => {
invalidController.abort();
@@ -179,6 +185,7 @@ it.each([
},
};
const result = await new RustBinaryService({
close: async () => {},
identity,
inspect: () => Promise.resolve(ok(invalidReport)),
}).inspect({ path: report.artifact.path });
@@ -204,6 +211,7 @@ it.each([
controller.abort(reason);
let calls = 0;
const service = new RustBinaryService({
close: async () => {},
identity,
inspect: async () => {
calls += 1;
@@ -226,6 +234,7 @@ it.each([
it("requires retention and rejects contradictory detail before invoking the reader", async () => {
let calls = 0;
const service = new RustBinaryService({
close: async () => {},
identity,
inspect: async () => {
calls++;
@@ -250,6 +259,7 @@ it("requires retention and rejects contradictory detail before invoking the read
it("retains the unchanged parent and keeps delivery selectors out of provider inputs", async () => {
const inputs: unknown[] = [];
const service = new RustBinaryService({
close: async () => {},
identity,
inspect: async (input) => {
inputs.push(input);
+1 -1
View File
@@ -26,7 +26,7 @@ export class RustBinaryService {
constructor(readonly provider: RustBinaryPort) {}
close(): Promise<void> {
return this.provider.close?.() ?? Promise.resolve();
return this.provider.close();
}
/** Inspect one explicitly selected file without acquiring a binary session. */
+47 -38
View File
@@ -14,9 +14,12 @@ import {
type ProviderIdentity,
type ExecutionOptions,
} from "../application/AnalysisProvider.js";
import { inspectBundleKeyedArchive } from "./apple/KeyedArchiveReader.js";
import {
inspectBundleKeyedArchive,
inspectStandaloneKeyedArchive,
} from "./apple/KeyedArchiveReader.js";
import { traceDylibResolution } from "./apple/DylibResolutionReader.js";
import { basename, dirname } from "node:path";
import { dirname } from "node:path";
import { inventoryArtifact } from "./inventory/ArtifactInventory.js";
import { extractArtifact } from "./extraction/ArtifactExtraction.js";
import { analyzeInterfaceBuilderBundle } from "./apple/InterfaceBuilderAnalysis.js";
@@ -33,7 +36,6 @@ import {
} from "../domain/binaryTargetTypes.js";
import {
AnalysisCapabilityUnavailableError,
AnalysisInputError,
AnalysisUnsupportedTargetError,
} from "../domain/analysisErrorCore.js";
import type { JsonValue } from "../domain/jsonValue.js";
@@ -260,43 +262,42 @@ class ArtifactClient implements AnalysisClient {
if (operation === "inspect_keyed_archive") {
const standalone =
this.target.kind === "artifact" && this.target.format === "plist";
if (
standalone &&
parameters.path !== undefined &&
parameters.path !== "." &&
parameters.path !== basename(this.target.path)
)
throw new AnalysisInputError(operation, undefined, [
{
path: ["path"],
reason: "invalid_value",
message:
"For an active plist, path must select that archive (omit path or use its basename).",
expected: basename(this.target.path),
},
]);
const bundlePath = standalone
? dirname(this.target.path)
: (this.target.sourcePath ?? this.target.path);
const result = await inspectBundleKeyedArchive({
bundlePath,
targetSha256: this.target.sha256,
parameters: standalone
? { ...parameters, path: basename(this.target.path) }
: parameters,
platform: this.platform,
resources: this.#resourceScope,
...(options?.signal === undefined ? {} : { signal: options.signal }),
});
if (standalone && result.archive_sha256 !== this.target.sha256)
throw new ArtifactReaderFailure(
"integrity",
`Active archive digest changed: expected ${this.target.sha256}, observed ${result.archive_sha256}`,
);
const selectedPath = artifactTargetSelectedPath(this.target);
const result = standalone
? await inspectStandaloneKeyedArchive({
path: this.target.path,
selectedPath,
targetSha256: this.target.sha256,
parameters,
platform: this.platform,
resources: this.#resourceScope,
...(options?.signal === undefined
? {}
: { signal: options.signal }),
})
: await inspectBundleKeyedArchive({
bundlePath: this.target.sourcePath ?? this.target.path,
targetSha256: this.target.sha256,
parameters,
platform: this.platform,
resources: this.#resourceScope,
...(options?.signal === undefined
? {}
: { signal: options.signal }),
});
return ok(
createAnalysisExecution(result, ARTIFACT_GRAPH_PROVIDER, {
limitations: result.limitations,
locations: [{ kind: "artifact-path", path: result.archive_path }],
...(standalone
? {
subject: {
path: selectedPath,
sha256: result.archive_sha256,
format: "plist" as const,
},
}
: {}),
}),
);
}
@@ -310,7 +311,11 @@ class ArtifactClient implements AnalysisClient {
const result = await extractArtifact(
{
inputPath: this.target.sourcePath ?? this.target.path,
selectedPath: this.target.asarSelectedPath,
expectedRootSha256:
this.target.bundleInfoPlist === undefined
? this.target.sha256
: undefined,
selectedPath: this.target.artifactSelectedPath,
inputFormat: this.target.format,
outputRoot: parsed.output_root,
environment: this.environment,
@@ -505,7 +510,11 @@ class ArtifactClient implements AnalysisClient {
options?: ExecutionOptions,
) {
return inventoryArtifact(this.target.sourcePath ?? this.target.path, {
selectedPath: this.target.asarSelectedPath,
expectedRootSha256:
this.target.bundleInfoPlist === undefined
? this.target.sha256
: undefined,
selectedPath: this.target.artifactSelectedPath,
resourceScope: this.#resourceScope,
environment: this.environment,
asarHeaderAdmission: runtimeAsarHeaderAdmission(
+33
View File
@@ -0,0 +1,33 @@
import { realpath } from "node:fs/promises";
import { ArtifactReaderFailure } from "./ArtifactReader.js";
/** Verify selection spelling independently of canonical descriptor ownership. */
export const assertArtifactSelectedSource = async (
canonicalPath: string,
selectedPath: string | undefined,
): Promise<void> => {
if (selectedPath === undefined) return;
const changed = (cause?: unknown) =>
new ArtifactReaderFailure(
"integrity",
`Selected artifact no longer resolves to its admitted source: ${selectedPath}`,
{
userMessage:
"The selected artifact path no longer names the opened file. Reopen the selected artifact, then retry.",
...(cause === undefined ? {} : { cause }),
},
);
let current: string;
try {
current = await realpath(selectedPath);
} catch (cause: unknown) {
if (
cause instanceof Error &&
"code" in cause &&
["ENOENT", "ENOTDIR", "ELOOP"].includes(String(cause.code))
)
throw changed(cause);
throw cause;
}
if (current !== canonicalPath) throw changed();
};
+2 -6
View File
@@ -1,3 +1,4 @@
import { assertArtifactSelectedSource } from "./ArtifactSelectedSource.js";
import { appendCallerPath } from "../filesystem/callerPath.js";
import type { AsarHeaderAdmission } from "../config/asarHeaderAdmission.js";
import { artifactIntegrityDetails } from "../domain/artifactIntegrity.js";
@@ -437,12 +438,7 @@ export class AsarArtifactReader implements ArtifactReader {
// The packed pathname is fixed for this reader; member checks only need
// to re-observe the caller's independently selected alias.
this.#selectedCanonicalSource ??= await realpath(this.path);
const current = await realpath(selected);
if (this.#selectedCanonicalSource !== current)
throw new ArtifactReaderFailure(
"integrity",
`Selected ASAR archive no longer resolves to its admitted source: ${selected}`,
);
await assertArtifactSelectedSource(this.#selectedCanonicalSource, selected);
}
async #openUnpackedStream(
+15 -17
View File
@@ -3,8 +3,8 @@ import { readPeCoffHeader } from "../domain/peCoffHeader.js";
import {
hasGodotPckHeader,
readGodotPckHeaderFacts,
readGodotPckTrailerFacts,
} from "../domain/godotPckFormat.js";
import { observeEmbeddedGodotPck } from "./GodotPckLocation.js";
import type { FileHandle } from "node:fs/promises";
import { createHash } from "node:crypto";
import { setImmediate } from "node:timers/promises";
@@ -351,22 +351,20 @@ const inspectPckTrailer = async (
read: DesktopByteRead,
fact: RecordDesktopFact,
): Promise<void> => {
const trailer = await read(size - 12, 12);
const declaration = readGodotPckTrailerFacts(trailer, size);
if (declaration.state === "declared") {
const { start: offset, packBytes } = declaration;
if (offset === null || packBytes < 20n)
throw new ArtifactReaderFailure(
"format",
"Embedded PCK trailer declares an invalid pack extent",
);
const bytes = await read(offset, 20);
if (!hasGodotPckHeader(bytes))
throw new ArtifactReaderFailure(
"format",
"Embedded PCK trailer does not locate a pack header",
);
const header = readGodotPckHeaderFacts(bytes);
const observation = await observeEmbeddedGodotPck(size, 20, read);
if (observation.state === "invalid-extent")
throw new ArtifactReaderFailure(
"format",
"Embedded PCK trailer declares an invalid pack extent",
);
if (observation.state === "header-mismatch")
throw new ArtifactReaderFailure(
"format",
"Embedded PCK trailer does not locate a pack header",
);
if (observation.state === "located") {
const offset = observation.start;
const header = readGodotPckHeaderFacts(observation.header);
fact({
kind: "godot-pck-header",
offset,
+52 -9
View File
@@ -2,6 +2,7 @@ import type { FileHandle } from "node:fs/promises";
import {
hasGodotPckHeader,
readGodotPckTrailerFacts,
type GodotPckTrailerFacts,
} from "../domain/godotPckFormat.js";
import { ArtifactReaderFailure } from "./ArtifactReader.js";
import { assertArtifactReadActive } from "./ArtifactFileOperation.js";
@@ -12,6 +13,46 @@ export interface GodotPckLocation {
readonly embedded: boolean;
}
type DeclaredGodotPckTrailer = Extract<
GodotPckTrailerFacts,
{ readonly state: "declared" }
>;
/** A malformed declaration remains distinct from an examined missing marker. */
export type EmbeddedGodotPckObservation =
| { readonly state: "absent" }
| {
readonly state: "invalid-extent" | "header-mismatch";
readonly declaration: DeclaredGodotPckTrailer;
}
| {
readonly state: "located";
readonly declaration: DeclaredGodotPckTrailer;
readonly start: number;
readonly header: Buffer;
};
/**
* Observe an admitted source of at least 12 bytes. Caller-owned exact reads retain
* admission, cancellation, budgets and range evidence; read failures propagate.
*/
export const observeEmbeddedGodotPck = async (
size: number,
headerBytes: 4 | 20,
read: (offset: number, bytes: number, end: number) => Promise<Buffer>,
): Promise<EmbeddedGodotPckObservation> => {
const trailer = await read(size - 12, 12, size);
const declaration = readGodotPckTrailerFacts(trailer, size);
if (declaration.state === "absent") return declaration;
const { start, end, packBytes } = declaration;
if (start === null || packBytes < BigInt(headerBytes))
return { state: "invalid-extent", declaration };
const header = await read(start, headerBytes, end);
if (!hasGodotPckHeader(header))
return { state: "header-mismatch", declaration };
return { state: "located", declaration, start, header };
};
/** Positional exact reads preserve short-read, cancellation and I/O semantics. */
export const readPckBytes = async (
handle: Pick<FileHandle, "read">,
@@ -67,13 +108,15 @@ export const locateGodotPck = async (
: await readPckBytes(handle, 0, 4, size, signal);
if (hasGodotPckHeader(magic)) return { start: 0, end: size, embedded: false };
if (size < 12) return undefined;
const trailer = await readPckBytes(handle, size - 12, 12, size, signal);
const declaration = readGodotPckTrailerFacts(trailer, size);
// The terminal marker is only a candidate until its range resolves to a PCK header.
if (declaration.state === "absent") return undefined;
const { start, end, packBytes } = declaration;
if (start === null || packBytes < 4n) return undefined;
if (!hasGodotPckHeader(await readPckBytes(handle, start, 4, end, signal)))
return undefined;
return { start, end, embedded: true };
const observation = await observeEmbeddedGodotPck(
size,
4,
(offset, bytes, end) => readPckBytes(handle, offset, bytes, end, signal),
);
if (observation.state !== "located") return undefined;
return {
start: observation.start,
end: observation.declaration.end,
embedded: true,
};
};
+15 -74
View File
@@ -49,11 +49,7 @@ import {
interfaceBuilderLimitsSchema,
type InterfaceBuilderDocumentInput,
} from "../../domain/apple/interfaceBuilderGraph.js";
import {
mergeNibHierarchies,
projectNibViewHierarchy,
type NibHierarchyNode,
} from "./NibViewHierarchy.js";
import { projectNibViewHierarchy } from "./NibViewHierarchy.js";
import { jsonValueSchema, type JsonValue } from "../../domain/jsonValue.js";
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
@@ -806,7 +802,6 @@ const projectNibArchive = (
.map(projectObject),
);
const connections: Record<string, JsonValue[]> = {};
const hierarchy: NibHierarchyNode[] = [];
// A recognized connector whose owner reference cannot be resolved produces no
// connection entry, so the omission has to be counted here: the projected
// dictionary carries no trace of it for a downstream consumer to find.
@@ -904,80 +899,26 @@ const projectNibArchive = (
facets: ["objects", "connections", "classes"],
description: `NSString NS.bytes is not valid UTF-8 for ${unavailableStringIds.size} source objects; their bytes and object IDs are retained, while dependent class names, titles and selectors remain unknown.`,
});
let hierarchyOmitted = 0;
let hierarchyNodeCount = 0;
const hierarchyFor = (
objectId: number,
seen: Set<number>,
depth: number,
): NibHierarchyNode | null => {
if (depth > 32 || seen.has(objectId)) {
hierarchyOmitted += 1;
return null;
}
if (hierarchyNodeCount >= maxHierarchyNodes) {
hierarchyOmitted += 1;
return null;
}
if (!budget.tryReserve(128)) {
hierarchyOmitted += 1;
return null;
}
const object = byId.get(objectId);
if (object === undefined) {
hierarchyOmitted += 1;
return null;
}
hierarchyNodeCount += 1;
const nextSeen = new Set(seen).add(objectId);
const children: NibHierarchyNode[] = [];
for (const [key, value] of Object.entries(object.values)) {
if (!/(?:subviews|contentview|childviewcontrollers|view)$/iu.test(key))
continue;
const direct = reference(value);
if (direct === null) continue;
const target = byId.get(direct);
if (target === undefined) {
hierarchyOmitted += 1;
continue;
}
if (/array|set/iu.test(target.class_name)) {
for (const candidate of nibCollectionMembers(target) ?? []) {
const child = reference(candidate);
const nested =
child === null ? null : hierarchyFor(child, nextSeen, depth + 1);
if (nested !== null) children.push(nested);
}
} else {
const nested = hierarchyFor(direct, nextSeen, depth + 1);
if (nested !== null) children.push(nested);
}
}
return { objectID: String(objectId), children };
};
const ibData = archive.objects.find(
({ class_name }) => normalizeClassName(class_name) === "NSIBObjectData",
);
const rootId = reference(ibData?.values.NSRoot);
const hierarchyRoots = rootId === null ? [] : [rootId];
for (const hierarchyRoot of hierarchyRoots) {
const root = hierarchyFor(hierarchyRoot, new Set(), 0);
if (root !== null) hierarchy.push(root);
}
const viewHierarchy = projectNibViewHierarchy(
archive.objects,
new Set(Object.keys(objects).map(Number)),
() => budget.tryReserve(128),
maxHierarchyNodes,
);
const merged = mergeNibHierarchies(
hierarchy,
viewHierarchy.hierarchy ?? [],
() => budget.tryReserve(96),
maxHierarchyNodes,
(description) => {
budget.reserve(
256,
"NIB registry hierarchy diagnostics exceed the aggregate Interface Builder decode budget",
);
budget.reserveJsonStringOccurrence(description);
unexaminedSources.push({
reason: "nib_registry_parentage_unexamined",
facets: ["hierarchy"],
description,
});
},
);
return {
omitted: viewHierarchy.omitted + hierarchyOmitted + merged.omitted,
omitted: viewHierarchy.omitted,
unexaminedSources,
unresolvedConnections,
nonFiniteRealCount,
@@ -985,7 +926,7 @@ const projectNibArchive = (
raw: jsonValueSchema.parse({
"com.apple.ibtool.document.objects": objects,
"com.apple.ibtool.document.connections": connections,
"com.apple.ibtool.document.hierarchy": merged.hierarchy,
"com.apple.ibtool.document.hierarchy": viewHierarchy.hierarchy ?? [],
"com.apple.ibtool.document.classes": Object.fromEntries(
archive.classes.map((name) => {
const className = normalizeClassName(name);
+157 -7
View File
@@ -1,10 +1,15 @@
import { createHash } from "node:crypto";
import { join } from "node:path";
import { basename, join } from "node:path";
import { realpath } from "node:fs/promises";
import { readStableArtifact } from "../readStableArtifact.js";
import { projectStableArtifactFailure } from "../StableArtifactFailure.js";
import { ARTIFACT_GRAPH_PROVIDER } from "../../domain/investigationProviders.js";
import { z } from "zod";
import { decodeBinaryPlistValue } from "./BinaryPlistMaterializer.js";
import {
AnalysisAccessDeniedError,
AnalysisArtifactChangedError,
AnalysisInputError,
AnalysisUnsupportedTargetError,
} from "../../domain/analysisErrorCore.js";
@@ -33,7 +38,12 @@ import {
} from "../DirectoryArtifactReader.js";
import { ArtifactReaderFailure } from "../ArtifactReader.js";
import type { ArtifactEntry } from "../ArtifactReader.js";
import { bindArtifactFileCleanup } from "../ArtifactFileOperation.js";
import {
artifactSelectedPathErrno,
artifactFilePrimaryFacts,
assertArtifactReadActive,
bindArtifactFileCleanup,
} from "../ArtifactFileOperation.js";
import { ArtifactResourceScope } from "../ArtifactResourceScope.js";
import { AnalysisError } from "../../domain/analysisErrorBase.js";
import { translateArtifactFailure } from "../ArtifactOperationFailure.js";
@@ -216,6 +226,122 @@ export const inspectBundleKeyedArchive = (input: BundleKeyedArchiveInput) => {
});
};
/** Inspect the admitted physical file while retaining its caller-selected identity. */
export const inspectStandaloneKeyedArchive = (input: {
readonly path: string;
readonly selectedPath: string;
readonly targetSha256: string;
readonly parameters: unknown;
readonly platform: NodeJS.Platform;
readonly resources: ArtifactResourceScope;
readonly signal?: AbortSignal;
}) =>
input.resources.run(async () => {
const selected = keyedArchiveInputSchema.parse(input.parameters);
const selectedBasename = basename(input.selectedPath);
const physicalBasename = basename(input.path);
if (
selected.path !== "." &&
selected.path !== selectedBasename &&
selected.path !== physicalBasename
)
throw new AnalysisInputError("inspect_keyed_archive", undefined, [
{
path: ["path"],
reason: "invalid_value",
message:
"For an active plist, path must select that archive (omit path or use its basename).",
expected: [selectedBasename, physicalBasename],
},
]);
await assertStandaloneArchiveSelection(
input.path,
input.selectedPath,
input.signal,
);
const snapshot = await readStableArtifact(
input.path,
MAX_BYTES,
input.signal,
undefined,
input.resources,
).catch((cause: unknown) => {
const failure = artifactFilePrimaryFacts(cause).cause;
if (
failure instanceof Error &&
"code" in failure &&
(failure.code === "EACCES" || failure.code === "EPERM")
)
throw projectStableArtifactFailure(
cause,
new AnalysisAccessDeniedError(
"inspect_keyed_archive",
input.selectedPath,
failure.code,
{ cause: failure },
),
input.selectedPath,
ARTIFACT_GRAPH_PROVIDER.id,
"inspect_keyed_archive",
);
throw cause;
});
await assertStandaloneArchiveSelection(
input.path,
input.selectedPath,
input.signal,
);
if (snapshot.sha256 !== input.targetSha256)
throw new ArtifactReaderFailure(
"integrity",
`Active archive digest changed: expected ${input.targetSha256}, observed ${snapshot.sha256}`,
);
return decodeSelectedArchive(snapshot.bytes, {
targetSha256: input.targetSha256,
archiveSha256: snapshot.sha256,
parameters: selected,
platform: input.platform,
archivePath: selected.path === "." ? selectedBasename : selected.path,
artifactPath: input.selectedPath,
});
});
/** Selected aliases must continue naming the admitted physical archive. */
const assertStandaloneArchiveSelection = async (
path: string,
selectedPath: string,
signal?: AbortSignal,
): Promise<void> => {
assertArtifactReadActive(signal);
const changed = (cause?: unknown) =>
new AnalysisArtifactChangedError(
"inspect_keyed_archive",
selectedPath,
"Selected archive no longer resolves to the admitted file.",
{ cause },
);
let current: string;
try {
current = await realpath(selectedPath);
} catch (cause: unknown) {
if (artifactSelectedPathErrno(cause) !== undefined) throw changed(cause);
if (
cause instanceof Error &&
"code" in cause &&
(cause.code === "EACCES" || cause.code === "EPERM")
)
throw new AnalysisAccessDeniedError(
"inspect_keyed_archive",
selectedPath,
cause.code,
{ cause },
);
throw cause;
}
assertArtifactReadActive(signal);
if (current !== path) throw changed();
};
/** Read exactly one regular, contained bundle entry without following symlinks. */
const readSelectedKeyedArchive = async (
input: BundleKeyedArchiveInput,
@@ -285,22 +411,46 @@ const readSelectedKeyedArchive = async (
chunks.push(bytes);
}
const bytes = Buffer.concat(chunks, size);
return decodeSelectedArchive(bytes, {
targetSha256: input.targetSha256,
archiveSha256: createHash("sha256").update(bytes).digest("hex"),
resourceRoot: input.resourceRoot,
parameters: selected,
platform: input.platform,
archivePath: entry.path,
artifactPath: join(input.bundlePath, entry.path),
});
};
/** Decode and bind one archive at its owning read boundary. */
const decodeSelectedArchive = (
bytes: Buffer,
input: {
readonly targetSha256: BundleKeyedArchiveInput["targetSha256"];
readonly archiveSha256: string;
readonly resourceRoot?: BundleKeyedArchiveInput["resourceRoot"];
readonly parameters: ReturnType<typeof keyedArchiveInputSchema.parse>;
readonly platform?: NodeJS.Platform | undefined;
readonly archivePath: string;
readonly artifactPath: string;
},
) => {
try {
return keyedArchiveResultSchema.parse({
target_sha256: input.targetSha256,
...(input.resourceRoot === undefined
? {}
: { resource_root: input.resourceRoot }),
archive_path: entry.path,
archive_sha256: createHash("sha256").update(bytes).digest("hex"),
...decodeKeyedArchiveBytes(bytes, selected),
archive_path: input.archivePath,
archive_sha256: input.archiveSha256,
...decodeKeyedArchiveBytes(bytes, input.parameters),
});
} catch (cause) {
if (cause instanceof AnalysisInputError) throw cause;
if (cause instanceof UnsupportedBinaryPlistUidError)
throw new AnalysisUnsupportedTargetError(
"inspect_keyed_archive",
join(input.bundlePath, entry.path),
input.artifactPath,
cause.message,
{ cause },
);
@@ -308,7 +458,7 @@ const readSelectedKeyedArchive = async (
if (cause instanceof KeyedArchiveKindError)
throw new AnalysisUnsupportedTargetError(
"inspect_keyed_archive",
join(input.bundlePath, entry.path),
input.artifactPath,
cause.message,
{
cause,
+24 -72
View File
@@ -1,9 +1,6 @@
import { describe, expect, it } from "vitest";
import {
mergeNibHierarchies,
projectNibViewHierarchy,
} from "./NibViewHierarchy.js";
import { projectNibViewHierarchy } from "./NibViewHierarchy.js";
const object = (id: number, parent?: number) => ({
id,
@@ -13,7 +10,7 @@ const object = (id: number, parent?: number) => ({
});
describe("serialized NIB view hierarchy bounds", () => {
it("leaves archives without superview links to the legacy projection", () => {
it("does not invent hierarchy for objects without serialized relationships", () => {
expect(projectNibViewHierarchy([object(0)], new Set([0]))).toEqual({
hierarchy: null,
omitted: 0,
@@ -113,77 +110,32 @@ describe("serialized NIB view hierarchy bounds", () => {
});
});
describe("legacy NIB hierarchy preservation", () => {
it("keeps encoded window parents while merging shared views only once", () => {
const legacy = [
it("does not treat a dangling registry owner as top-level authority", () => {
const result = projectNibViewHierarchy(
[
{
objectID: "window",
children: [
{ objectID: "view", children: [{ objectID: "first", children: [] }] },
],
id: 0,
class_name: "NSIBObjectData",
values: {
NSRoot: { $nib_object_ref: 99 },
NSObjectsKeys: { $nib_object_ref: 2 },
NSObjectsValues: { $nib_object_ref: 3 },
},
},
];
const recovered = [
object(1),
{
objectID: "view",
children: [
{ objectID: "first", children: [] },
{ objectID: "second", children: [] },
],
id: 2,
class_name: "NSArray",
values: { "NS.objects": { $nib_object_ref: 1 } },
},
];
expect(mergeNibHierarchies(legacy, recovered).hierarchy).toEqual([
{
objectID: "window",
children: [
{
objectID: "view",
children: [
{ objectID: "first", children: [] },
{ objectID: "second", children: [] },
],
},
],
id: 3,
class_name: "NSArray",
values: { "NS.objects": { $nib_object_ref: 99 } },
},
]);
});
it("moves an existing root under its encoded parent without duplicate visits", () => {
expect(
mergeNibHierarchies(
[{ objectID: "child", children: [] }],
[
{
objectID: "parent",
children: [{ objectID: "child", children: [] }],
},
],
),
).toEqual({
hierarchy: [
{ objectID: "parent", children: [{ objectID: "child", children: [] }] },
],
omitted: 0,
});
});
it("reports conflicting encoded parents rather than dropping a link silently", () => {
const legacy = [
{ objectID: "first", children: [{ objectID: "child", children: [] }] },
];
const recovered = [
{ objectID: "second", children: [{ objectID: "child", children: [] }] },
];
expect(mergeNibHierarchies(legacy, recovered).omitted).toBe(1);
});
it("does not introduce a cycle when moving an existing root", () => {
const legacy = [
{ objectID: "child", children: [{ objectID: "parent", children: [] }] },
];
const recovered = [
{ objectID: "parent", children: [{ objectID: "child", children: [] }] },
];
expect(mergeNibHierarchies(legacy, recovered)).toEqual({
hierarchy: legacy,
omitted: 1,
});
});
],
new Set([1]),
);
expect(result.hierarchy).toEqual([]);
expect(result.omitted).toBeGreaterThan(0);
});
+178 -183
View File
@@ -6,25 +6,195 @@ export type NibHierarchyNode = {
children: NibHierarchyNode[];
};
/** Project encoded view parents; archive owners are not inferred view parents. */
const childCollections = new Set([
"subviews",
"NSSubviews",
"UISubviews",
"childViewControllers",
"NSChildViewControllers",
"UIChildViewControllers",
]);
const childReferences = new Set([
"view",
"NSView",
"UIView",
"contentView",
"NSContentView",
"UIContentView",
]);
// AppKit declares NSButtonCell : NSActionCell : NSCell : NSObject, not NSView.
// NSActionCell encodes its control-view backreference as NSControlView.
const nonViewCellClasses = new Set(["NSCell", "NSActionCell", "NSButtonCell"]);
const isNonViewCell = (
object: NibArchiveDocument["objects"][number] | undefined,
) =>
object !== undefined &&
nonViewCellClasses.has(object.class_name.replace(/\0+$/u, ""));
/** Interpret serialized relationships once, before bounded forest materialization. */
export const projectNibViewHierarchy = (
objects: NibArchiveDocument["objects"],
retainedIds: ReadonlySet<number>,
reserveNode: () => boolean = () => true,
maxNodes = 20_000,
reportUnknownRegistry: (description: string) => void = () => {},
) => {
const { parents, involved, invalid } = collectNibParents(
objects,
retainedIds,
);
if (involved.size === 0) return { hierarchy: null, omitted: invalid };
const byId = new Map(objects.map((object) => [object.id, object]));
const parents = new Map<number, number | null>();
const involved = new Set<number>();
const registryUnknown = new Set<number>();
let invalid = 0;
let ownerWithoutRegistry = false;
const addParent = (child: number, parent: number) => {
if (retainedIds.has(parent)) involved.add(parent);
if (!retainedIds.has(child)) {
invalid += 1;
return;
}
involved.add(child);
const previous = parents.get(child);
// A disagreement invalidates placement; archive order is not authority.
parents.set(
child,
previous === undefined || previous === parent ? parent : null,
);
};
const fieldValues = (
object: NibArchiveDocument["objects"][number],
key: string,
) =>
object.repeated_values?.[key] ??
(Object.hasOwn(object.values, key) ? [object.values[key]!] : []);
const addReference = (value: JsonValue, parent: number) => {
if (value === null) return;
const child = reference(value);
if (child === null) invalid += 1;
else addParent(child, parent);
};
for (const object of objects) {
if (!retainedIds.has(object.id)) continue;
for (const value of fieldValues(object, "NSSuperview")) {
if (value === null) continue;
const parent = reference(value);
if (parent === null) invalid += 1;
else addParent(object.id, parent);
}
const keys = new Set([
...Object.keys(object.values),
...Object.keys(object.repeated_values ?? {}),
]);
for (const key of keys) {
if (childCollections.has(key)) {
for (const value of fieldValues(object, key)) {
if (value === null) continue;
const collection = reference(value);
const members = nibCollectionMembers(
collection === null ? undefined : byId.get(collection),
);
if (members === undefined) invalid += 1;
else {
involved.add(object.id);
for (const member of members) addReference(member, object.id);
}
}
} else if (
childReferences.has(key) ||
(key === "NSWindowView" &&
object.class_name.replace(/\0+$/u, "") === "NSWindowTemplate")
) {
for (const value of fieldValues(object, key))
addReference(value, object.id);
} else if (
!["NSSuperview", "NSNextKeyView", "NSPreviousKeyView"].includes(key) &&
!(key === "NSControlView" && isNonViewCell(object)) &&
/(?:subviews|contentview|childviewcontrollers|view)$/iu.test(key)
) {
// Preserve formerly admitted fields as raw facts without inventing semantics.
invalid += 1;
}
}
}
for (const object of objects) {
if (object.class_name.replace(/\0+$/u, "") !== "NSIBObjectData") continue;
const registryFields = ["NSRoot", "NSObjectsKeys", "NSObjectsValues"];
const ambiguous = registryFields.filter(
(key) => fieldValues(object, key).length > 1,
);
if (ambiguous.length > 0) {
invalid += 1;
reportUnknownRegistry(
`NSIBObjectData object ${object.id} has repeated ${ambiguous.join(", ")} declarations; registry hierarchy interpretation remains unknown.`,
);
continue;
}
const rootValue = fieldValues(object, "NSRoot")[0];
const keysValue = fieldValues(object, "NSObjectsKeys")[0];
const valuesValue = fieldValues(object, "NSObjectsValues")[0];
const ownerReference = reference(rootValue);
const owner =
ownerReference !== null && byId.has(ownerReference)
? ownerReference
: null;
if (rootValue !== undefined && rootValue !== null && owner === null)
invalid += 1;
const keysRef = reference(keysValue);
const valuesRef = reference(valuesValue);
if (keysValue === undefined && valuesValue === undefined) {
if (rootValue !== undefined) ownerWithoutRegistry = true;
continue;
}
const keys = nibCollectionMembers(
keysRef === null ? undefined : byId.get(keysRef),
);
const values = nibCollectionMembers(
valuesRef === null ? undefined : byId.get(valuesRef),
);
if (
keys === undefined ||
values === undefined ||
keys.length !== values.length
) {
invalid += 1;
continue;
}
for (let index = 0; index < keys.length; index += 1) {
const child = reference(keys[index]);
const parent = reference(values[index]);
if (child === null || parent === null) {
invalid += 1;
continue;
}
if (!retainedIds.has(child) || isNonViewCell(byId.get(child))) continue;
involved.add(child);
// The registry identifies top-level membership through the owner.
// Other registry associations alone do not establish view parentage.
if (owner !== null && parent === owner) {
if (parents.has(child)) parents.set(child, null);
} else {
registryUnknown.add(child);
if (!parents.has(child))
reportUnknownRegistry(
`NSIBObjectData object ${object.id}: NSObjectsKeys/NSObjectsValues pair child ${child} with registry value ${parent}; NSRoot is ${ownerReference === null ? "unresolved" : owner === null ? `${ownerReference} (unresolved)` : owner}. This decoded registry association does not establish visual parentage.`,
);
}
}
}
if (ownerWithoutRegistry) invalid += 1;
for (const child of registryUnknown)
if (!parents.has(child)) parents.set(child, null);
const children = new Map<number, number[]>();
for (const [child, parent] of parents) {
if (!retainedIds.has(parent) || !retainedIds.has(child)) continue;
if (parent === null || !retainedIds.has(parent)) continue;
const siblings = children.get(parent) ?? [];
siblings.push(child);
children.set(parent, siblings);
}
if (involved.size === 0)
return {
hierarchy: null,
omitted: invalid,
};
const roots: NibHierarchyNode[] = [];
const pending: { id: number; depth: number; output: NibHierarchyNode[] }[] =
[];
@@ -47,101 +217,7 @@ export const projectNibViewHierarchy = (
for (const child of [...(children.get(item.id) ?? [])].reverse())
pending.push({ id: child, depth: item.depth + 1, output: node.children });
}
return {
hierarchy: roots,
omitted: involved.size - visited.size + invalid,
};
};
/** Add encoded parent links once; conflicting legacy parents remain unknown. */
export const mergeNibHierarchies = (
legacy: NibHierarchyNode[],
recovered: NibHierarchyNode[],
reserveNode: () => boolean = () => true,
maxNodes = 20_000,
) => {
const existing = new Map<string, NibHierarchyNode>();
const owners = new Map<string, NibHierarchyNode[]>();
const parents = new Map<string, string>();
const pending = legacy.map((node) => ({
node,
output: legacy,
parent: null as string | null,
}));
while (pending.length > 0) {
const entry = pending.pop();
if (entry === undefined || existing.has(entry.node.objectID)) continue;
existing.set(entry.node.objectID, entry.node);
owners.set(entry.node.objectID, entry.output);
if (entry.parent !== null) parents.set(entry.node.objectID, entry.parent);
pending.push(
...entry.node.children.map((node) => ({
node,
output: entry.node.children,
parent: entry.node.objectID,
})),
);
}
let omitted = 0;
const additions = recovered.map((node) => ({
node,
output: legacy,
parent: null as string | null,
}));
while (additions.length > 0) {
const entry = additions.pop();
if (entry === undefined) break;
let target = existing.get(entry.node.objectID);
if (target === undefined) {
if (existing.size >= maxNodes || !reserveNode()) {
omitted += 1;
continue;
}
target = { objectID: entry.node.objectID, children: [] };
existing.set(target.objectID, target);
entry.output.push(target);
owners.set(target.objectID, entry.output);
if (entry.parent !== null) parents.set(target.objectID, entry.parent);
} else if (
entry.parent !== null &&
owners.get(target.objectID) !== entry.output
) {
const owner = owners.get(target.objectID);
if (
owner !== legacy ||
wouldCycle(target.objectID, entry.parent, parents)
) {
omitted += 1;
} else {
owner.splice(owner.indexOf(target), 1);
entry.output.push(target);
owners.set(target.objectID, entry.output);
parents.set(target.objectID, entry.parent);
}
}
for (const node of [...entry.node.children].reverse())
additions.push({
node,
output: target.children,
parent: target.objectID,
});
}
return { hierarchy: legacy, omitted };
};
const wouldCycle = (
child: string,
parent: string,
parents: ReadonlyMap<string, string>,
): boolean => {
const seen = new Set<string>();
let current: string | undefined = parent;
while (current !== undefined && !seen.has(current)) {
if (current === child) return true;
seen.add(current);
current = parents.get(current);
}
return false;
return { hierarchy: roots, omitted: involved.size - visited.size + invalid };
};
const reference = (value: JsonValue | undefined): number | null => {
@@ -151,84 +227,3 @@ const reference = (value: JsonValue | undefined): number | null => {
? value.$nib_object_ref
: null;
};
const collectNibParents = (
objects: NibArchiveDocument["objects"],
retainedIds: ReadonlySet<number>,
) => {
const parents = new Map<number, number>();
const involved = new Set<number>();
let invalid = 0;
for (const object of objects) {
if (!retainedIds.has(object.id)) continue;
const child = reference(object.values.NSWindowView);
if (
object.class_name.replace(/\0+$/u, "") === "NSWindowTemplate" &&
invalidReference(object.values.NSWindowView)
)
invalid += 1;
if (
object.class_name.replace(/\0+$/u, "") === "NSWindowTemplate" &&
child !== null
) {
parents.set(child, object.id);
involved.add(object.id);
involved.add(child);
}
const parent = reference(object.values.NSSuperview);
if (invalidReference(object.values.NSSuperview)) invalid += 1;
if (parent === null) continue;
parents.set(object.id, parent);
involved.add(object.id);
if (retainedIds.has(parent)) involved.add(parent);
}
const byId = new Map(objects.map((object) => [object.id, object]));
for (const object of objects) {
if (object.class_name.replace(/\0+$/u, "") !== "NSIBObjectData") continue;
const keysRef = reference(object.values.NSObjectsKeys);
const valuesRef = reference(object.values.NSObjectsValues);
if (keysRef === null && valuesRef === null) continue;
const keys = nibCollectionMembers(
keysRef === null ? undefined : byId.get(keysRef),
);
const values = nibCollectionMembers(
valuesRef === null ? undefined : byId.get(valuesRef),
);
if (
keys === undefined ||
values === undefined ||
keys.length !== values.length
) {
invalid += 1;
continue;
}
const rootOwner = reference(object.values.NSRoot);
for (let index = 0; index < keys.length; index += 1) {
const child = reference(keys[index]);
const parent = reference(values[index]);
if (child === null || parent === null) {
invalid += 1;
continue;
}
// Registry keys include non-UI records. Only retained objects contribute
// hierarchy, and a paired NSRoot owner identifies a top-level object.
if (!retainedIds.has(child)) continue;
involved.add(child);
const prior = parents.get(child);
if (parent === rootOwner) {
if (prior !== undefined) invalid += 1;
continue;
}
if (prior !== undefined && prior !== parent) {
invalid += 1;
continue;
}
parents.set(child, parent);
if (retainedIds.has(parent)) involved.add(parent);
}
}
return { parents, involved, invalid };
};
const invalidReference = (value: JsonValue | undefined): boolean =>
value !== undefined && value !== null && reference(value) === null;
+31 -5
View File
@@ -1,3 +1,6 @@
import { assertAdmittedRootDigest } from "../inventory/classify.js";
import { assertArtifactSelectedSource } from "../ArtifactSelectedSource.js";
import { hashStableRootArtifactHandle } from "../inventory/hashStableRootArtifact.js";
import { absoluteCallerPath } from "../../filesystem/callerPath.js";
import type { AsarHeaderAdmission } from "../../config/asarHeaderAdmission.js";
import {
@@ -56,7 +59,8 @@ import type { ArtifactIntegrityPolicyName } from "../../domain/artifactIntegrity
export interface ArtifactExtractionInput {
readonly asarHeaderAdmission?: AsarHeaderAdmission;
readonly inputPath: string;
/** Selected ASAR spelling; packed I/O remains on inputPath. */
readonly expectedRootSha256?: string | undefined;
/** Selected artifact spelling; packed I/O remains on inputPath. */
readonly selectedPath?: string | undefined;
readonly inputFormat: BinaryTarget["format"];
readonly outputRoot: string;
@@ -79,6 +83,8 @@ const extractArtifactInScope = async (
// Cancellation wins over every later refusal, including unsupported formats.
abortIfNeeded(signal);
const sourcePath = await realpath(input.inputPath);
if (input.expectedRootSha256 !== undefined)
await assertArtifactSelectedSource(sourcePath, input.selectedPath);
const admission = await requireExtractableFormat(sourcePath, input, signal);
const { snapshot, zipSource } =
await scanArtifactInventoryForExtractionInScope(
@@ -86,6 +92,7 @@ const extractArtifactInScope = async (
{
resourceScope: input.resourceScope,
selectedPath: input.selectedPath,
expectedRootSha256: input.expectedRootSha256,
signal,
environment: input.environment,
asarHeaderAdmission:
@@ -176,7 +183,7 @@ export const materializeArtifactInventoryInScope = async (
)
throw new AnalysisUnsupportedTargetError(
"extract_artifact",
input.inputPath,
input.selectedPath ?? input.inputPath,
`Godot PCK directory is ${containerMetadata.directory_status}; its complete contents cannot be extracted`,
);
const selectedOccurrences = snapshot.occurrences.filter(
@@ -218,7 +225,7 @@ export const materializeArtifactInventoryInScope = async (
if (occurrence.encrypted)
throw new AnalysisUnsupportedTargetError(
"extract_artifact",
input.inputPath,
input.selectedPath ?? input.inputPath,
`Archive entry ${occurrence.logical_path} is encrypted; REA does not decrypt archive entries, so the archive cannot be extracted completely`,
);
// Inventory retains unavailable source occurrences, but extraction cannot
@@ -377,6 +384,8 @@ const materializeSelection = async ({
traversal.snapshot(),
reader.provenance(),
);
if (input.expectedRootSha256 !== undefined)
await assertArtifactSelectedSource(sourcePath, input.selectedPath);
await output.commit(signal);
return completed;
} catch (cause: unknown) {
@@ -550,7 +559,11 @@ const requireExtractableFormat = async (
path: string,
input: Pick<
ArtifactExtractionInput,
"inputPath" | "inputFormat" | "resourceScope"
| "inputPath"
| "selectedPath"
| "inputFormat"
| "resourceScope"
| "expectedRootSha256"
>,
signal?: AbortSignal,
previous?: ArtifactEmbeddedFormatProbeFacts,
@@ -591,13 +604,26 @@ const requireExtractableFormat = async (
((format === "elf" || format === "pe" || format === "mach-o") &&
((await probes.nodeSea()) !== undefined ||
(await probes.pyInstaller())));
if (pack && input.expectedRootSha256 !== undefined) {
const digest = await hashStableRootArtifactHandle(
path,
source.handle,
source.initial,
signal,
);
assertAdmittedRootDigest(
path,
input.expectedRootSha256,
digest.sha256,
);
}
return { pack, probeFacts: probes.completedFacts() };
},
);
if (!detected.pack)
throw new AnalysisUnsupportedTargetError(
"extract_artifact",
input.inputPath,
input.selectedPath ?? input.inputPath,
`Artifact format has no extraction reader: ${format}`,
);
return { directory, probeFacts: detected.probeFacts };
+72 -15
View File
@@ -1,3 +1,5 @@
import { assertArtifactSelectedSource } from "../ArtifactSelectedSource.js";
import { artifactNameForClassification } from "../../domain/artifactNameSelection.js";
import type { NodeSeaAdmission } from "../nodeSea/NodeSeaAdmission.js";
import type { AsarHeaderAdmission } from "../../config/asarHeaderAdmission.js";
import {
@@ -64,7 +66,11 @@ export const classifyAndHashRootForInventory = async (
expectedMetadata: Stats,
options: Pick<
ArtifactInventoryOptions,
"resourceScope" | "signal" | "asarHeaderAdmission" | "selectedPath"
| "resourceScope"
| "signal"
| "asarHeaderAdmission"
| "selectedPath"
| "expectedRootSha256"
>,
probeFacts?: ArtifactEmbeddedFormatProbeFacts,
): Promise<RootInventoryClassification> => {
@@ -75,6 +81,15 @@ export const classifyAndHashRootForInventory = async (
`Root artifact changed before inventory: ${path}`,
);
}
if (directory && options.expectedRootSha256 !== undefined)
throw new ArtifactReaderFailure(
"integrity",
`Artifact root changed after target admission: expected an admitted file at ${path}, observed a directory`,
{
userMessage:
"The opened file was replaced by a directory. Select the intended artifact and reopen it before retrying.",
},
);
if (directory)
return {
format: "directory",
@@ -82,6 +97,8 @@ export const classifyAndHashRootForInventory = async (
rootSource: undefined,
readerAdmission: undefined,
};
if (options.expectedRootSha256 !== undefined)
await assertArtifactSelectedSource(path, options.selectedPath);
const source = await openRootArtifact(
path,
options.resourceScope,
@@ -100,27 +117,46 @@ export const classifyAndHashRootForInventory = async (
"integrity",
`Root artifact changed before inventory: ${path}`,
);
// Prior probe state can prove replacement without interpreting any bytes.
const probes = new ArtifactEmbeddedFormatProbes(
path,
source,
options.signal,
probeFacts,
);
const admittedDigest =
options.expectedRootSha256 === undefined
? undefined
: await hashStableRootArtifactHandle(
path,
handle,
initial,
options.signal,
);
if (admittedDigest !== undefined)
assertAdmittedRootDigest(
path,
options.expectedRootSha256,
admittedDigest.sha256,
);
const classified = await classifyRootFormat(
path,
source,
new ArtifactEmbeddedFormatProbes(
path,
source,
options.signal,
probeFacts,
),
probes,
options.signal,
options.asarHeaderAdmission,
options.selectedPath,
);
const format =
typeof classified === "string" ? classified : classified.kind;
const digest = await hashStableRootArtifactHandle(
path,
handle,
initial,
options.signal,
);
const digest =
admittedDigest ??
(await hashStableRootArtifactHandle(
path,
handle,
initial,
options.signal,
));
const retainSource =
isZipPackageFormat(format) ||
format === "mach-o-universal" ||
@@ -185,7 +221,11 @@ const classifyRootFormat = async (
const archiveBytes = (await handle.stat()).size;
const evidence = probe.finish(archiveBytes);
abortIfNeeded(signal);
const lower = path.toLowerCase();
const classificationPath = artifactNameForClassification(
selectedPath ?? path,
path,
);
const lower = classificationPath.toLowerCase();
if (hasZipSignature(evidence.prefix)) return zipRootPackageFormat(lower);
if (
lower.endsWith(".pkg") &&
@@ -228,7 +268,7 @@ const classifyRootFormat = async (
? fatEvidence
: { ...fatEvidence, mzWindowsHeaderSignature };
const format = classifyArtifactContent(
selectedPath?.toLowerCase().endsWith(".asar") ? selectedPath : path,
classificationPath,
classificationEvidence,
archiveBytes,
).format;
@@ -352,3 +392,20 @@ const readRootMzWindowsHeaderSignature = async (
if (available === 0) return undefined;
return { offset, bytes: window.subarray(0, available) };
};
/** Compare physically completed bytes before interpreting an admitted file root. */
export const assertAdmittedRootDigest = (
path: string,
expected: string | undefined,
observed: string,
): void => {
if (expected !== undefined && observed !== expected)
throw new ArtifactReaderFailure(
"integrity",
`Artifact root changed after target admission at ${path}: expected ${expected}, observed ${observed}`,
{
userMessage:
"The artifact changed after it was opened. Reopen the selected artifact, then retry.",
},
);
};
+2
View File
@@ -13,6 +13,8 @@ export const STRICT_INTEGRITY_POLICY: ArtifactIntegrityPolicy = {
/** Options shared by artifact inventory scans. */
export interface ArtifactInventoryOptions {
readonly resourceScope: ArtifactResourceScope;
/** Identity of a previously admitted file root; directory roots have no file digest. */
readonly expectedRootSha256?: string | undefined;
/** Independently selected root spelling used only for ASAR companion lookup. */
readonly selectedPath?: string | undefined;
readonly signal?: AbortSignal | undefined;
+1 -7
View File
@@ -60,12 +60,6 @@ export const registerCutterCommands = (
.describe(
"Return command-specific JSON output when available (independent of CLI output formatting).",
),
json: z
.boolean()
.optional()
.describe(
"Deprecated compatibility input: use --json-output; pass --json=true or --json=false (bare --json formats CLI output).",
),
}),
run: ({ args, options }) =>
withCommandCancellation((signal) =>
@@ -78,7 +72,7 @@ export const registerCutterCommands = (
session_id: args.sessionId,
expected_generation: args.expectedGeneration,
command: args.command,
json: options.jsonOutput ?? options.json ?? false,
json: options.jsonOutput ?? false,
},
{ signal },
),
-1
View File
@@ -52,7 +52,6 @@ export const registerEvidenceCommands = (
}),
});
cli.command(CLI_COMMANDS.compare, {
aliases: ["compare-bundles"],
description: "Compare two canonical Evidence bundles",
args: z.object({
left: z.string().describe("Left Evidence bundle JSON path"),
+15
View File
@@ -39,6 +39,7 @@ import {
import { mcpToolFamiliesSchema } from "./mcpToolFamilies.js";
import { parseLoaderArgs, parseStringArray } from "./parsers.js";
import { processStartConfigurationSchema } from "./processStart.js";
import { parseJebMcpEndpoint } from "./jebMcpEndpoint.js";
const environmentSchema = z.object({
REA_ANALYSIS_PROVIDER: analysisProviderSelectorSchema.default("auto"),
@@ -62,6 +63,20 @@ const environmentSchema = z.object({
.refine(isAbsolute, "REA_IDA_MCP_CONFIG must be absolute")
.optional(),
REA_JEB_MCP_TIMEOUT_MS: providerStartupTimeoutSchema.optional(),
REA_JEB_MCP_URL: z
.string()
.superRefine((value, context) => {
const parsed = parseJebMcpEndpoint(value);
if (!parsed.ok)
context.addIssue({
code: "custom",
message:
typeof parsed.error.cause === "string"
? parsed.error.cause
: parsed.error.message,
});
})
.optional(),
GHIDRA_INSTALL_DIR: z
.string()
.min(1)
File renamed without changes.
Loaded 100 of 291 files, more files were not shown because too many files have changed in this diff. Show more