mirror of
https://github.com/morluto/rea.git
synced 2026-11-16 11:43:20 +00:00
feat(snapshots)!: persist the target identity as the snapshot target (#11551)
* feat(snapshots)!: persist the target identity as the snapshot target
The snapshot target is now `{ kind, identity }`, where `identity` is the
FileTargetIdentity that `binaryTargetIdentity` produces and `kind` is the
binary kind query IDs commit to. The per-format target projection and
`snapshotTargetIdentity` are deleted; the schema checks the format's
selection, architecture and kind instead of keeping one union arm per format.
Entry execution subjects use the Evidence subject shape (file or explicit
unknown), so a snapshot entry commits to the same identity, including the
selection, as its Evidence record. Workflow entries take the subject of the
workflow's own Evidence. Replay no longer re-derives the selection from the
open target.
`FileTargetSelection` gains `universal-slice` (`offset`, `size`, and the FAT
`cpu_type`/`cpu_subtype` when target admission supplied the FAT table, else
null). `binaryTargetIdentity` produces it for `mach-o-universal` targets, so
Evidence and snapshots for different slices of one container no longer share
an identity.
Refs #11417
BREAKING CHANGE: analysis snapshot files change shape: `target` is
`{ kind, identity: { kind: "file", sha256, format, architecture, selection } }`
and entry `execution.subject` is the Evidence subject
(`{ identity, name, local_path }`, never null). Snapshot binding mismatch
details report `target.snapshot`/`target.selected` in the same shape. Evidence
IDs of universal Mach-O targets change because their subject identity now
carries the selected slice. Recreate saved snapshots and Evidence bundles of
universal Mach-O targets.
* docs(evidence): name the universal Mach-O slice selection in the subject contract
* fix(snapshots): read the snapshot target identity in the Ghidra verification scripts
The real Ghidra snapshot checks (lifecycle, RISC-V, MIPS, PowerPC) read
snapshot.target.sha256 and .architecture, which moved under
snapshot.target.identity, so they would fail after saving a valid snapshot.
* feat(evidence)!: name live browser page Evidence by its page URL (#11559)
Browser page observations (inspect_web_page, analyze_web_bundle,
observe_web_session, discover_webmcp_tools, capture_web_screenshot,
observe_web_execution, inspect_web_event_listeners) named an unknown
(`not-observed`) subject although the provider reports the page URL.
`TargetIdentity` gains a `url` kind: the reported URL with userinfo removed
and its origin (`null` for an opaque origin). The schema refuses a URL that
carries userinfo or an origin the URL does not have.
`browserUrlTargetIdentity` in the browser URL owner builds it from the
provider's reported URL; a withheld or unparseable URL stays unknown
(`not-observed`). Target lists and comparisons still name no single
document. Analysis views keep their parent's URL subject, as they already
keep a file subject.
BREAKING CHANGE: Evidence IDs of live browser page observations and their
analysis views change, and their subject is
`{ "kind": "url", "url", "origin" }` with `name` and `local_path` null.
Recreate stored Evidence for these tools. This lands inside Evidence format
revision 2 under the release hold (#11417); no `evidence_format` bump.
This commit is contained in:
1 parent
f6df353002
commit
ff0176256a
30 files changed
+678
-396
No files matched your search
@@ -1023,12 +1023,17 @@ it, or with another revision, as an integrity mismatch whose recovery is to
|
||||
recreate the observations with the current release; re-exporting an earlier
|
||||
bundle does not update its format or identities. Revision 2 names the subject
|
||||
by its target identity, `subject.identity` (`kind: "file"`, `sha256`, `format`,
|
||||
`architecture` and `selection`: the raw or DOL entry point or the ESP flash
|
||||
offsets that select what was analysed, otherwise `null`), and the Evidence ID
|
||||
`architecture` and `selection`: the raw or DOL entry point, the ESP flash
|
||||
offsets, or the universal Mach-O slice (`offset`, `size`, and the FAT
|
||||
`cpu_type` and `cpu_subtype`, `null` when the FAT table was not admitted) that
|
||||
select what was analysed, otherwise `null`), and the Evidence ID
|
||||
commits to that identity; `subject.name` and `subject.local_path` are display
|
||||
facts outside the ID. The subject has no separate `digest`, `format` or
|
||||
`architecture` members; read them from `subject.identity`. Every record has a
|
||||
subject: one whose target REA did not identify names
|
||||
`architecture` members; read them from `subject.identity`. A live browser page
|
||||
observation names the page's reported URL,
|
||||
`{ "kind": "url", "url": …, "origin": … }`, with userinfo removed and `origin`
|
||||
`null` for an opaque origin; its `name` and `local_path` are `null`. Every
|
||||
record has a subject: one whose target REA did not identify names
|
||||
`{ "kind": "unknown", "reason": … }`, where the reason is
|
||||
`derived-from-linked-evidence` (comparisons and workflows, whose sources are
|
||||
committed through `evidence_links`), `not-observed` or `unsupported-subject`,
|
||||
|
||||
@@ -96,7 +96,7 @@ export async function verifyLegacyGhidraReferenceSnapshot(
|
||||
analysisProfile: legacyProfile,
|
||||
limitations: observed.limitations,
|
||||
locations: observed.locations,
|
||||
subject: nativeTarget,
|
||||
subject: evidence.subject,
|
||||
},
|
||||
}),
|
||||
],
|
||||
|
||||
@@ -70,7 +70,7 @@ export async function verifyLegacyValueScopeSnapshot(
|
||||
analysisProfile: legacyProfile,
|
||||
limitations: observed.limitations,
|
||||
locations: observed.locations,
|
||||
subject: nativeTarget,
|
||||
subject: evidence.subject,
|
||||
},
|
||||
}),
|
||||
],
|
||||
|
||||
@@ -52,7 +52,7 @@ export async function verifyGhidraSnapshotLifecycle(
|
||||
const snapshot = parseAnalysisSnapshot(
|
||||
JSON.parse(snapshotBytes.toString("utf8")),
|
||||
);
|
||||
assert.equal(snapshot.target.sha256, target.sha256);
|
||||
assert.equal(snapshot.target.identity.sha256, target.sha256);
|
||||
assert.notEqual(original.procedure.name, edited.procedure.name);
|
||||
const runId = (await call("binary_session")).analysis_run.run_id;
|
||||
const api = await call("inspect_native_api", { procedure: entry });
|
||||
|
||||
@@ -347,7 +347,7 @@ try {
|
||||
const snapshot = parseAnalysisSnapshot(
|
||||
JSON.parse(await readFile(snapshotPath, "utf8")),
|
||||
);
|
||||
assert.equal(snapshot.target.architecture, "mips");
|
||||
assert.equal(snapshot.target.identity.architecture, "mips");
|
||||
const parameters = snapshot.binding.analysis_profile.parameters;
|
||||
assert.equal(
|
||||
parameters.language_id,
|
||||
@@ -660,7 +660,7 @@ try {
|
||||
const snapshot = parseAnalysisSnapshot(
|
||||
JSON.parse(await readFile(snapshotPath, "utf8")),
|
||||
);
|
||||
assert.equal(snapshot.target.architecture, "mips");
|
||||
assert.equal(snapshot.target.identity.architecture, "mips");
|
||||
assert.deepEqual(
|
||||
snapshot.binding.analysis_profile.parameters.mips_elf,
|
||||
expectedMipsElfProfile(independent.mips),
|
||||
|
||||
@@ -234,7 +234,7 @@ try {
|
||||
const snapshot = parseAnalysisSnapshot(
|
||||
JSON.parse(await readFile(snapshotPath, "utf8")),
|
||||
);
|
||||
assert.equal(snapshot.target.architecture, "powerpc");
|
||||
assert.equal(snapshot.target.identity.architecture, "powerpc");
|
||||
assert.equal(
|
||||
snapshot.binding.analysis_profile.parameters.language_id,
|
||||
"PowerPC:BE:32:default",
|
||||
|
||||
@@ -368,8 +368,8 @@ try {
|
||||
const snapshot = parseAnalysisSnapshot(
|
||||
JSON.parse(await readFile(snapshotPath, "utf8")),
|
||||
);
|
||||
assert.equal(snapshot.target.architecture, "riscv");
|
||||
assert.equal(snapshot.target.sha256, fixture.sha256);
|
||||
assert.equal(snapshot.target.identity.architecture, "riscv");
|
||||
assert.equal(snapshot.target.identity.sha256, fixture.sha256);
|
||||
assertProfile(snapshot.binding.analysis_profile);
|
||||
for (const record of snapshot.evidence_bundle.records)
|
||||
assertEvidence(parseEvidence(record));
|
||||
|
||||
@@ -1,4 +1,7 @@
|
||||
import { unknownTarget } from "../domain/targetIdentity.js";
|
||||
import {
|
||||
unknownTarget,
|
||||
type NonFileTargetIdentity,
|
||||
} from "../domain/targetIdentity.js";
|
||||
import { isDeepStrictEqual } from "node:util";
|
||||
import type { ProviderIdentity } from "../ports/AnalysisProvider.js";
|
||||
import {
|
||||
@@ -6,7 +9,10 @@ import {
|
||||
type Evidence,
|
||||
type EvidenceObservation,
|
||||
} from "../domain/evidence.js";
|
||||
import { sanitizeBrowserUrl } from "../domain/browserObservation.js";
|
||||
import {
|
||||
browserUrlTargetIdentity,
|
||||
sanitizeBrowserUrl,
|
||||
} from "../domain/browserObservation.js";
|
||||
import type {
|
||||
InspectWebPageInput,
|
||||
ListBrowserTargetsInput,
|
||||
@@ -96,7 +102,7 @@ export const createBrowserEvidence = (
|
||||
const comparison =
|
||||
operation === "compare_web_captures" ||
|
||||
operation === "compare_web_screenshots";
|
||||
return createEvidence(unknownTarget("not-observed"), provider, {
|
||||
return createEvidence(browserSubject(operation, observed), provider, {
|
||||
predicateType: browserPredicate(operation),
|
||||
operation,
|
||||
parameters: {
|
||||
@@ -135,6 +141,44 @@ export const createBrowserEvidence = (
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* The document one page observation is about: the URL the provider reported
|
||||
* for it. Target lists and comparisons observe no single document.
|
||||
*/
|
||||
const browserSubject = (
|
||||
operation: BrowserEvidenceOperation,
|
||||
observed: Exclude<
|
||||
BrowserEvidenceResult,
|
||||
WebBundleAcquisitionPartialObservation
|
||||
>,
|
||||
): NonFileTargetIdentity => {
|
||||
switch (operation) {
|
||||
case "list_browser_targets":
|
||||
case "compare_web_captures":
|
||||
case "compare_web_screenshots":
|
||||
return unknownTarget("not-observed");
|
||||
case "analyze_web_bundle":
|
||||
// A completed analysis reports its captured page; an acquisition that
|
||||
// stopped before analysis reports the page inspection itself.
|
||||
if ("capture" in observed && "target_url" in observed.capture)
|
||||
return browserUrlTargetIdentity(observed.capture.target_url);
|
||||
if ("target" in observed && "url" in observed.target)
|
||||
return browserUrlTargetIdentity(observed.target.url);
|
||||
break;
|
||||
case "observe_web_session":
|
||||
if ("target" in observed && "initial_url" in observed.target)
|
||||
return browserUrlTargetIdentity(observed.target.initial_url);
|
||||
break;
|
||||
case "inspect_web_page":
|
||||
case "discover_webmcp_tools":
|
||||
case "capture_web_screenshot":
|
||||
if ("target" in observed && "url" in observed.target)
|
||||
return browserUrlTargetIdentity(observed.target.url);
|
||||
break;
|
||||
}
|
||||
throw new Error(`Browser ${operation} result does not report its document`);
|
||||
};
|
||||
|
||||
const browserParameters = (
|
||||
input: BrowserEvidenceInput,
|
||||
result: BrowserEvidenceResult,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { unknownTarget } from "../domain/targetIdentity.js";
|
||||
import { browserUrlTargetIdentity } from "../domain/browserObservation.js";
|
||||
import { z } from "zod";
|
||||
import type { ExecutionOptions } from "../ports/AnalysisProvider.js";
|
||||
import type { WebRuntimePort } from "../ports/WebRuntimePort.js";
|
||||
@@ -156,15 +156,19 @@ export class WebRuntimeService {
|
||||
),
|
||||
);
|
||||
return completeProviderEvidence(
|
||||
createEvidence(unknownTarget("not-observed"), this.provider.identity(), {
|
||||
predicateType: `rea.${operation.replaceAll("_", "-")}`,
|
||||
operation,
|
||||
parameters: input,
|
||||
result,
|
||||
confidence: "observed",
|
||||
authority: "external-service",
|
||||
limitations: result.limitations,
|
||||
}),
|
||||
createEvidence(
|
||||
browserUrlTargetIdentity(result.target.initial_url),
|
||||
this.provider.identity(),
|
||||
{
|
||||
predicateType: `rea.${operation.replaceAll("_", "-")}`,
|
||||
operation,
|
||||
parameters: input,
|
||||
result,
|
||||
confidence: "observed",
|
||||
authority: "external-service",
|
||||
limitations: result.limitations,
|
||||
},
|
||||
),
|
||||
options?.signal,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -484,7 +484,7 @@ export const workflowSnapshotRecord = (
|
||||
analysisProfile: analysisProfileSchema.parse(evidence.analysis_profile),
|
||||
limitations: evidence.limitations,
|
||||
locations: evidence.locations,
|
||||
subject: evidenceSubjectTarget(evidence.subject) ?? null,
|
||||
subject: evidence.subject,
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
@@ -28,6 +28,7 @@ import {
|
||||
createEvidence,
|
||||
evidenceFileTarget,
|
||||
evidenceSubjectDisplay,
|
||||
evidenceSubjectInput,
|
||||
evidenceSubjectTarget,
|
||||
rebindEvidenceToTargetPath,
|
||||
type Evidence,
|
||||
@@ -373,7 +374,12 @@ const createAnalysisViewEvidence = (
|
||||
parent: Evidence,
|
||||
result: AnalysisViewResult,
|
||||
): Evidence => {
|
||||
const target = evidenceSubjectTarget(parent.subject);
|
||||
// A view names the target its parent observed (a file or a page URL); a
|
||||
// parent that names no target is committed through the view's link.
|
||||
const subject =
|
||||
parent.subject.identity.kind === "unknown"
|
||||
? unknownTarget("derived-from-linked-evidence")
|
||||
: evidenceSubjectInput(parent.subject);
|
||||
const nativeFunctionView =
|
||||
result.kind === "native" ||
|
||||
(result.kind === "summary" &&
|
||||
@@ -387,7 +393,7 @@ const createAnalysisViewEvidence = (
|
||||
(result.kind === "summary" &&
|
||||
isNativeObservationOperation(result.parent_operation));
|
||||
return createEvidence(
|
||||
target ?? unknownTarget("derived-from-linked-evidence"),
|
||||
subject,
|
||||
ANALYSIS_VIEW_PROVIDER,
|
||||
{
|
||||
predicateType: "rea.analysis-view",
|
||||
|
||||
@@ -8,8 +8,13 @@ import {
|
||||
ANALYSIS_SNAPSHOT_PROFILE,
|
||||
ANALYSIS_SNAPSHOT_PROVIDER,
|
||||
ANALYSIS_SNAPSHOT_TARGET,
|
||||
snapshotExecution,
|
||||
} from "../../domain/analysisSnapshot.fixture.js";
|
||||
import { createEvidence, immutableEvidence } from "../../domain/evidence.js";
|
||||
import {
|
||||
createEvidence,
|
||||
evidenceSubject,
|
||||
immutableEvidence,
|
||||
} from "../../domain/evidence.js";
|
||||
import { createAnalysisExecutionEvidence } from "../../domain/analysisObservation.js";
|
||||
import {
|
||||
createEvidenceBundle,
|
||||
@@ -37,6 +42,7 @@ import {
|
||||
isSourceEvidenceRetainable,
|
||||
} from "./AnalysisSnapshotCache.js";
|
||||
import { EvidenceLedger } from "../investigation/EvidenceLedger.js";
|
||||
|
||||
import { nativeCallGraphSchema } from "../../domain/native/nativeCallGraph.js";
|
||||
|
||||
it.each(["provider", "workflow"] as const)(
|
||||
@@ -73,7 +79,11 @@ it.each(["provider", "workflow"] as const)(
|
||||
execution: { ...execution, analysisProfile },
|
||||
};
|
||||
if (kind === "provider") cache.record(input);
|
||||
else cache.recordWorkflow(input);
|
||||
else
|
||||
cache.recordWorkflow({
|
||||
...input,
|
||||
execution: snapshotExecution(input.execution),
|
||||
});
|
||||
const entries = () =>
|
||||
kind === "provider" ? cache.entries() : cache.workflowEntries();
|
||||
const expected = entries();
|
||||
@@ -123,10 +133,12 @@ it("exports independent nested query, workflow and Evidence payloads", () => {
|
||||
operation: "analyze_function",
|
||||
parameters,
|
||||
execution: {
|
||||
...createAnalysisExecution(result, REA_WORKFLOW_PROVIDER, {
|
||||
subject: target,
|
||||
rawResult: result,
|
||||
}),
|
||||
...snapshotExecution(
|
||||
createAnalysisExecution(result, REA_WORKFLOW_PROVIDER, {
|
||||
subject: target,
|
||||
rawResult: result,
|
||||
}),
|
||||
),
|
||||
analysisProfile: workflowProfile,
|
||||
},
|
||||
});
|
||||
@@ -226,7 +238,10 @@ it.each(["provider", "workflow"] as const)(
|
||||
profile,
|
||||
operation,
|
||||
parameters,
|
||||
execution: { ...execution, analysisProfile: evidenceProfile },
|
||||
execution: snapshotExecution({
|
||||
...execution,
|
||||
analysisProfile: evidenceProfile,
|
||||
}),
|
||||
});
|
||||
expect(ledger.record(evidence).ok).toBe(true);
|
||||
const before = cache.export(target, profile, ledger.export());
|
||||
@@ -285,14 +300,14 @@ it.each(["provider", "workflow"] as const)(
|
||||
if (!after.ok) throw after.error;
|
||||
const entries =
|
||||
kind === "provider" ? after.value.entries : after.value.workflow_entries;
|
||||
expect(entries[0]?.execution.subject?.path).toBe("/relocated/fixture");
|
||||
expect(entries[0]?.execution.subject.local_path).toBe("/relocated/fixture");
|
||||
expect(entries[0]?.execution.result).toBe("original");
|
||||
expect(after.value.evidence_bundle.records).toContainEqual(relocated);
|
||||
const originalEntries =
|
||||
kind === "provider"
|
||||
? before.value.entries
|
||||
: before.value.workflow_entries;
|
||||
expect(originalEntries[0]?.execution.subject?.path).toBe(target.path);
|
||||
expect(originalEntries[0]?.execution.subject.local_path).toBe(target.path);
|
||||
},
|
||||
);
|
||||
|
||||
@@ -327,7 +342,11 @@ it.each(["provider", "workflow"] as const)(
|
||||
};
|
||||
if (kind === "provider")
|
||||
cache.record({ ...input, operation: "procedure_info" });
|
||||
else cache.recordWorkflow(input);
|
||||
else
|
||||
cache.recordWorkflow({
|
||||
...input,
|
||||
execution: snapshotExecution(input.execution),
|
||||
});
|
||||
expect(
|
||||
ledger.record(
|
||||
createEvidence(subject ?? unknownTarget("not-observed"), provider, {
|
||||
@@ -356,7 +375,9 @@ it.each(["provider", "workflow"] as const)(
|
||||
const saved =
|
||||
kind === "provider" ? snapshot.entries : snapshot.workflow_entries;
|
||||
expect(saved).toHaveLength(1);
|
||||
expect(saved[0]?.execution.subject?.sha256).toBe(target.sha256);
|
||||
expect(saved[0]?.execution.subject.identity).toMatchObject({
|
||||
sha256: target.sha256,
|
||||
});
|
||||
expect(snapshot.evidence_bundle.records).toHaveLength(1);
|
||||
expect(
|
||||
kind === "provider" ? cache.entries() : cache.workflowEntries(),
|
||||
@@ -402,9 +423,11 @@ it("counts newly retained direct and workflow bindings when staging", () => {
|
||||
operation: "trace_feature",
|
||||
parameters: { query: "feature" },
|
||||
execution: {
|
||||
...createAnalysisExecution("workflow", REA_WORKFLOW_PROVIDER, {
|
||||
subject: target,
|
||||
}),
|
||||
...snapshotExecution(
|
||||
createAnalysisExecution("workflow", REA_WORKFLOW_PROVIDER, {
|
||||
subject: target,
|
||||
}),
|
||||
),
|
||||
analysisProfile: workflowProfile,
|
||||
},
|
||||
});
|
||||
@@ -532,11 +555,13 @@ it("binds an omitted subject to the selected target in snapshot replay", () => {
|
||||
]),
|
||||
);
|
||||
if (!snapshot.ok) throw snapshot.error;
|
||||
expect(snapshot.value.entries[0]?.execution.subject?.path).toBe(target.path);
|
||||
expect(snapshot.value.entries[0]?.execution.subject.local_path).toBe(
|
||||
target.path,
|
||||
);
|
||||
const reparsed = parseAnalysisSnapshot(
|
||||
JSON.parse(serializeAnalysisSnapshot(snapshot.value)),
|
||||
);
|
||||
expect(reparsed.entries[0]?.execution.subject?.path).toBe(target.path);
|
||||
expect(reparsed.entries[0]?.execution.subject.local_path).toBe(target.path);
|
||||
cache.clear();
|
||||
const imported = cache.import(
|
||||
reparsed,
|
||||
@@ -583,9 +608,11 @@ describe("analysis snapshot cache capacity", () => {
|
||||
const target = ANALYSIS_SNAPSHOT_TARGET;
|
||||
const profile = ANALYSIS_SNAPSHOT_PROFILE;
|
||||
const execution = {
|
||||
...createAnalysisExecution("workflow", REA_WORKFLOW_PROVIDER, {
|
||||
subject: target,
|
||||
}),
|
||||
...snapshotExecution(
|
||||
createAnalysisExecution("workflow", REA_WORKFLOW_PROVIDER, {
|
||||
subject: target,
|
||||
}),
|
||||
),
|
||||
analysisProfile: workflowAnalysisProfile(profile),
|
||||
};
|
||||
const workflowInput = {
|
||||
@@ -673,7 +700,7 @@ describe("analysis snapshot cache capacity", () => {
|
||||
cache.recordWorkflow({
|
||||
...workflowInput,
|
||||
target: otherTarget,
|
||||
execution: { ...execution, subject: otherTarget },
|
||||
execution: { ...execution, subject: evidenceSubject(otherTarget) },
|
||||
});
|
||||
expect(cache.entries()).toEqual([]);
|
||||
expect(cache.workflowEntries()).toHaveLength(1);
|
||||
@@ -1233,10 +1260,12 @@ it.each([
|
||||
operation,
|
||||
parameters,
|
||||
execution: {
|
||||
...createAnalysisExecution(oldResult, REA_WORKFLOW_PROVIDER, {
|
||||
subject: target,
|
||||
rawResult: oldResult,
|
||||
}),
|
||||
...snapshotExecution(
|
||||
createAnalysisExecution(oldResult, REA_WORKFLOW_PROVIDER, {
|
||||
subject: target,
|
||||
rawResult: oldResult,
|
||||
}),
|
||||
),
|
||||
analysisProfile: oldProfile,
|
||||
},
|
||||
});
|
||||
@@ -1316,9 +1345,11 @@ it.each([1, 2])(
|
||||
limitations: [],
|
||||
};
|
||||
const execution = {
|
||||
...createAnalysisExecution(oldResult, REA_WORKFLOW_PROVIDER, {
|
||||
subject: target,
|
||||
}),
|
||||
...snapshotExecution(
|
||||
createAnalysisExecution(oldResult, REA_WORKFLOW_PROVIDER, {
|
||||
subject: target,
|
||||
}),
|
||||
),
|
||||
analysisProfile: oldProfile,
|
||||
};
|
||||
cache.recordWorkflow({ target, profile, operation, parameters, execution });
|
||||
|
||||
@@ -10,10 +10,17 @@ import {
|
||||
import { NoBinaryOpenError } from "../../domain/configurationErrors.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import {
|
||||
evidenceSubject,
|
||||
evidenceSubjectDisplay,
|
||||
evidenceSubjectTarget,
|
||||
rebindEvidenceSubjectTargetPath,
|
||||
type Evidence,
|
||||
evidenceSubjectDisplay,
|
||||
type EvidenceSubjectTarget,
|
||||
} from "../../domain/evidence.js";
|
||||
import {
|
||||
sameTargetIdentity,
|
||||
unknownTarget,
|
||||
} from "../../domain/targetIdentity.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import {
|
||||
analysisQueryId,
|
||||
@@ -25,7 +32,6 @@ import {
|
||||
snapshotBindingMismatch,
|
||||
snapshotMatchesProfile,
|
||||
snapshotTarget,
|
||||
snapshotTargetIdentity,
|
||||
parseAnalysisSnapshot,
|
||||
immutableAnalysisSnapshot,
|
||||
type AnalysisSnapshot,
|
||||
@@ -49,10 +55,6 @@ import {
|
||||
snapshotReplayBinding,
|
||||
type SnapshotReplayBinding,
|
||||
} from "./AnalysisSnapshotReplayBinding.js";
|
||||
import {
|
||||
binaryTargetSelection,
|
||||
sameTargetIdentity,
|
||||
} from "../../domain/targetIdentity.js";
|
||||
import { createInvestigationEvidenceClosureChecker } from "../investigation/InvestigationEvidence.js";
|
||||
|
||||
const RELATIONSHIP_OPERATIONS: ReadonlySet<AnalysisOperation> = new Set([
|
||||
@@ -96,6 +98,12 @@ const DOCUMENT_SCOPED_OPERATIONS: ReadonlySet<string> = new Set(
|
||||
/** Maximum duplicated result bindings retained across direct and workflow queries. */
|
||||
const SNAPSHOT_CACHE_ENTRY_CEILING = 10_000;
|
||||
|
||||
/** A file target without its recorded display name, which follows the replayed path. */
|
||||
const withoutDisplayName = ({
|
||||
name: _name,
|
||||
...target
|
||||
}: EvidenceSubjectTarget): EvidenceSubjectTarget => target;
|
||||
|
||||
const replayDocumentSelected = (
|
||||
operation: AnalysisOperation,
|
||||
parameters: Readonly<Record<string, JsonValue>>,
|
||||
@@ -232,10 +240,7 @@ export class AnalysisSnapshotCache {
|
||||
snapshot = immutableAnalysisSnapshot(snapshot);
|
||||
if (
|
||||
(this.#target !== undefined &&
|
||||
!sameTargetIdentity(
|
||||
snapshotTargetIdentity(this.#target),
|
||||
snapshotTargetIdentity(snapshot.target),
|
||||
)) ||
|
||||
!sameTargetIdentity(this.#target.identity, snapshot.target.identity)) ||
|
||||
(this.#binding !== undefined &&
|
||||
!snapshotMatchesProfile(
|
||||
this.#binding,
|
||||
@@ -358,35 +363,31 @@ export class AnalysisSnapshotCache {
|
||||
previous.parameters,
|
||||
);
|
||||
const entry = this.#entries.get(queryId);
|
||||
const entryIdentity = entry?.execution.subject.identity;
|
||||
if (
|
||||
entry !== undefined &&
|
||||
entry.execution.subject !== null &&
|
||||
entryIdentity?.kind === "file" &&
|
||||
evidenceMatchesEntry(previous, entry, header)
|
||||
)
|
||||
this.#entries.set(queryId, {
|
||||
...entry,
|
||||
execution: {
|
||||
...entry.execution,
|
||||
subject: {
|
||||
...entry.execution.subject,
|
||||
path: display.local_path,
|
||||
},
|
||||
subject: { identity: entryIdentity, ...display },
|
||||
},
|
||||
});
|
||||
const workflow = this.#workflowEntries.get(queryId);
|
||||
const workflowIdentity = workflow?.execution.subject.identity;
|
||||
if (
|
||||
workflow !== undefined &&
|
||||
workflow.execution.subject !== null &&
|
||||
workflowIdentity?.kind === "file" &&
|
||||
evidenceMatchesWorkflowEntry(previous, workflow, header)
|
||||
)
|
||||
this.#workflowEntries.set(queryId, {
|
||||
...workflow,
|
||||
execution: {
|
||||
...workflow.execution,
|
||||
subject: {
|
||||
...workflow.execution.subject,
|
||||
path: display.local_path,
|
||||
},
|
||||
subject: { identity: workflowIdentity, ...display },
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -401,18 +402,21 @@ export class AnalysisSnapshotCache {
|
||||
if (target === undefined || profile === undefined)
|
||||
return err(new NoBinaryOpenError());
|
||||
try {
|
||||
const snapshotTargetIdentity = snapshotTarget(target);
|
||||
const persistedTarget = snapshotTarget(target);
|
||||
const namesTarget = (subject: Evidence["subject"]): boolean =>
|
||||
subject.identity.kind === "file" &&
|
||||
subject.identity.sha256 === target.sha256;
|
||||
const binding = snapshotBinding(profile);
|
||||
// In-memory replay can preserve unbound and child-artifact observations.
|
||||
// A persisted target snapshot can authenticate only rows whose Evidence
|
||||
// has that target subject; the record owner applies the same scope to its bundle.
|
||||
const entries = [...this.#entries.values()]
|
||||
.filter((entry) => entry.execution.subject?.sha256 === target.sha256)
|
||||
.filter((entry) => namesTarget(entry.execution.subject))
|
||||
.sort((left, right) =>
|
||||
compareUnicodeCodePoints(left.query_id, right.query_id),
|
||||
);
|
||||
const workflows = [...this.#workflowEntries.values()]
|
||||
.filter((entry) => entry.execution.subject?.sha256 === target.sha256)
|
||||
.filter((entry) => namesTarget(entry.execution.subject))
|
||||
.sort((left, right) =>
|
||||
compareUnicodeCodePoints(left.query_id, right.query_id),
|
||||
);
|
||||
@@ -420,7 +424,7 @@ export class AnalysisSnapshotCache {
|
||||
// payloads here only adds another full materialization.
|
||||
return ok(
|
||||
parseAnalysisSnapshot({
|
||||
target: snapshotTargetIdentity,
|
||||
target: persistedTarget,
|
||||
binding,
|
||||
entries,
|
||||
workflow_entries: workflows,
|
||||
@@ -507,7 +511,7 @@ export class AnalysisSnapshotCache {
|
||||
readonly limitations: readonly string[];
|
||||
readonly relationshipCoverage?: AnalysisExecution["relationshipCoverage"];
|
||||
readonly locations: AnalysisExecution["locations"];
|
||||
readonly subject: NonNullable<AnalysisExecution["subject"]> | null;
|
||||
readonly subject: Evidence["subject"];
|
||||
};
|
||||
}): void {
|
||||
this.select(input.target, input.profile);
|
||||
@@ -556,27 +560,14 @@ export class AnalysisSnapshotCache {
|
||||
this.#replayBindings.set(queryId, restored);
|
||||
binding = restored;
|
||||
}
|
||||
const cachedSubject = cached.execution.subject;
|
||||
// The query ID commits to the snapshot target, including its selection,
|
||||
// so a cached subject naming the target's bytes made the same selection.
|
||||
// Snapshot subjects do not persist it yet (ADR target-identity slice 2d).
|
||||
const selection =
|
||||
cachedSubject?.sha256 === target.sha256
|
||||
? binaryTargetSelection(target)
|
||||
: null;
|
||||
// Replay names the file at the current path; its display name follows
|
||||
// that path, as the live observation's would.
|
||||
const cachedTarget = evidenceSubjectTarget(cached.execution.subject);
|
||||
const subject =
|
||||
cachedSubject == null
|
||||
cachedTarget === undefined
|
||||
? null
|
||||
: rebindEvidenceSubjectTargetPath(
|
||||
{
|
||||
path: cachedSubject.path,
|
||||
sha256: cachedSubject.sha256,
|
||||
format: cachedSubject.format,
|
||||
...(cachedSubject.architecture === null
|
||||
? {}
|
||||
: { architecture: cachedSubject.architecture }),
|
||||
...(selection === null ? {} : { selection }),
|
||||
},
|
||||
withoutDisplayName(cachedTarget),
|
||||
target,
|
||||
);
|
||||
const execution: AnalysisExecution = structuredClone({
|
||||
@@ -656,17 +647,11 @@ export class AnalysisSnapshotCache {
|
||||
parameters,
|
||||
execution: {
|
||||
...execution,
|
||||
subject:
|
||||
subject: evidenceSubject(
|
||||
execution.subject === undefined
|
||||
? {
|
||||
path: target.path,
|
||||
sha256: target.sha256,
|
||||
format: target.format,
|
||||
...(target.architecture === undefined
|
||||
? {}
|
||||
: { architecture: target.architecture }),
|
||||
}
|
||||
: execution.subject,
|
||||
? target
|
||||
: (execution.subject ?? unknownTarget("not-observed")),
|
||||
),
|
||||
},
|
||||
});
|
||||
this.#entries.set(entry.query_id, entry);
|
||||
|
||||
@@ -172,7 +172,10 @@ describe("target format vocabularies", () => {
|
||||
analysisSnapshotSchema.shape.target.parse(
|
||||
roundTrip(snapshotTarget(target)),
|
||||
),
|
||||
).toMatchObject({ format: target.format, kind: target.kind });
|
||||
).toMatchObject({
|
||||
kind: target.kind,
|
||||
identity: { format: target.format },
|
||||
});
|
||||
expect(binaryTargetSummary(target)).toMatchObject({
|
||||
format: target.format,
|
||||
kind: target.kind,
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
import { createAnalysisProfile } from "./analysisProfile.js";
|
||||
import type { BinaryTarget } from "./binaryTargetTypes.js";
|
||||
import { evidenceSubject, type EvidenceSubjectTarget } from "./evidence.js";
|
||||
import { unknownTarget } from "./targetIdentity.js";
|
||||
|
||||
/** Immutable target shared by analysis-snapshot contract tests. */
|
||||
export const ANALYSIS_SNAPSHOT_TARGET: BinaryTarget = {
|
||||
@@ -23,3 +25,13 @@ export const ANALYSIS_SNAPSHOT_PROFILE = createAnalysisProfile(
|
||||
ANALYSIS_SNAPSHOT_PROVIDER,
|
||||
{ loader: "mach-o-arm64" },
|
||||
);
|
||||
|
||||
/** A provider execution as a snapshot entry records it: with its Evidence subject. */
|
||||
export const snapshotExecution = <
|
||||
Execution extends { readonly subject?: EvidenceSubjectTarget | null },
|
||||
>(
|
||||
execution: Execution,
|
||||
) => ({
|
||||
...execution,
|
||||
subject: evidenceSubject(execution.subject ?? unknownTarget("not-observed")),
|
||||
});
|
||||
@@ -16,7 +16,8 @@ import {
|
||||
ANALYSIS_SNAPSHOT_PROVIDER,
|
||||
ANALYSIS_SNAPSHOT_TARGET,
|
||||
} from "./analysisSnapshot.fixture.js";
|
||||
import { createEvidence } from "./evidence.js";
|
||||
import { createEvidence, evidenceSubject } from "./evidence.js";
|
||||
import { unknownTarget } from "./targetIdentity.js";
|
||||
import { createEvidenceBundle } from "./evidenceBundle.js";
|
||||
import {
|
||||
RAW_ONLY_ARCHITECTURES,
|
||||
@@ -46,12 +47,7 @@ const snapshotWithHistoricalEvidence = () => {
|
||||
limitations: [],
|
||||
relationship_coverage: null,
|
||||
locations: [],
|
||||
subject: {
|
||||
path: ANALYSIS_SNAPSHOT_TARGET.path,
|
||||
sha256: ANALYSIS_SNAPSHOT_TARGET.sha256,
|
||||
format: ANALYSIS_SNAPSHOT_TARGET.format,
|
||||
architecture: ANALYSIS_SNAPSHOT_TARGET.architecture ?? null,
|
||||
},
|
||||
subject: current.subject,
|
||||
},
|
||||
};
|
||||
return {
|
||||
@@ -121,22 +117,25 @@ describe("analysis snapshot target identity", () => {
|
||||
it.each(RAW_ONLY_ARCHITECTURES)(
|
||||
"keeps %s confined to executable raw snapshots",
|
||||
(architecture) => {
|
||||
const target = {
|
||||
const identity = {
|
||||
kind: "file",
|
||||
sha256: ANALYSIS_SNAPSHOT_TARGET.sha256,
|
||||
kind: "executable",
|
||||
architecture,
|
||||
};
|
||||
expect(
|
||||
analysisSnapshotSchema.shape.target.safeParse({
|
||||
...target,
|
||||
format: "pe",
|
||||
kind: "executable",
|
||||
identity: { ...identity, format: "pe", selection: null },
|
||||
}).success,
|
||||
).toBe(false);
|
||||
expect(
|
||||
analysisSnapshotSchema.shape.target.safeParse({
|
||||
...target,
|
||||
format: "raw-binary",
|
||||
entry_point: "0x8000",
|
||||
kind: "executable",
|
||||
identity: {
|
||||
...identity,
|
||||
format: "raw-binary",
|
||||
selection: { kind: "entry-point", entry_point: "0x8000" },
|
||||
},
|
||||
}).success,
|
||||
).toBe(true);
|
||||
},
|
||||
@@ -177,16 +176,22 @@ describe("analysis snapshot target identity", () => {
|
||||
workflow_entries: [],
|
||||
evidence_bundle: createEvidenceBundle([]),
|
||||
});
|
||||
expect(parsed.target).toHaveProperty("entry_point", target.entryPoint);
|
||||
const { entry_point: _entry, ...legacy } = {
|
||||
...snapshot,
|
||||
expect(parsed.target.identity.selection).toEqual({
|
||||
kind: "entry-point",
|
||||
entry_point: target.entryPoint,
|
||||
};
|
||||
});
|
||||
// A raw target without its entry selection is not one REA can open.
|
||||
expect(
|
||||
analysisSnapshotSchema.safeParse({ ...parsed, target: legacy }).success,
|
||||
analysisSnapshotSchema.safeParse({
|
||||
...parsed,
|
||||
target: {
|
||||
...parsed.target,
|
||||
identity: { ...parsed.target.identity, selection: null },
|
||||
},
|
||||
}).success,
|
||||
).toBe(false);
|
||||
expect(snapshotTarget(ANALYSIS_SNAPSHOT_TARGET)).not.toHaveProperty(
|
||||
"entry_point",
|
||||
expect(snapshotTarget(ANALYSIS_SNAPSHOT_TARGET).identity.selection).toBe(
|
||||
null,
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -220,8 +225,11 @@ describe("analysis snapshot nested subject contract", () => {
|
||||
if (entry === undefined) throw new Error("Fixture has no snapshot entry");
|
||||
const malformedSubject = {
|
||||
...entry.execution.subject,
|
||||
format: "pe",
|
||||
architecture,
|
||||
identity: {
|
||||
...entry.execution.subject.identity,
|
||||
format: "pe",
|
||||
architecture,
|
||||
},
|
||||
};
|
||||
expect(
|
||||
analysisSnapshotSchema.safeParse({
|
||||
@@ -309,18 +317,20 @@ describe("analysis snapshot contract", () => {
|
||||
limitations: [],
|
||||
relationship_coverage: null,
|
||||
locations: [],
|
||||
subject: target,
|
||||
subject: evidence.subject,
|
||||
},
|
||||
},
|
||||
],
|
||||
workflow_entries: [],
|
||||
evidence_bundle: createEvidenceBundle([evidence]),
|
||||
});
|
||||
expect(parsed.target).toMatchObject({
|
||||
expect(parsed.target.identity).toMatchObject({
|
||||
format,
|
||||
architecture,
|
||||
});
|
||||
expect(parsed.entries[0]?.execution.subject?.format).toBe(format);
|
||||
expect(fileIdentity(parsed.entries[0]?.execution.subject).format).toBe(
|
||||
format,
|
||||
);
|
||||
expect(
|
||||
fileIdentity(parsed.evidence_bundle.records[0]?.subject).format,
|
||||
).toBe(format);
|
||||
@@ -329,7 +339,7 @@ describe("analysis snapshot contract", () => {
|
||||
|
||||
it("rejects a snapshot subject path containing NUL", () => {
|
||||
const binding = snapshotBinding(ANALYSIS_SNAPSHOT_PROFILE);
|
||||
const entry = {
|
||||
const entry = (localPath: string) => ({
|
||||
query_id: `query_${"0".repeat(64)}`,
|
||||
operation: "analyze_function",
|
||||
parameters: {},
|
||||
@@ -341,22 +351,28 @@ describe("analysis snapshot contract", () => {
|
||||
relationship_coverage: null,
|
||||
locations: [],
|
||||
subject: {
|
||||
path: "a\u0000b",
|
||||
sha256: "a".repeat(64),
|
||||
format: "elf" as const,
|
||||
architecture: null,
|
||||
identity: {
|
||||
kind: "file",
|
||||
sha256: "a".repeat(64),
|
||||
format: "elf",
|
||||
architecture: null,
|
||||
selection: null,
|
||||
},
|
||||
name: "b",
|
||||
local_path: localPath,
|
||||
},
|
||||
},
|
||||
};
|
||||
expect(
|
||||
});
|
||||
const parsed = (localPath: string) =>
|
||||
analysisSnapshotSchema.safeParse({
|
||||
target: snapshotTarget(ANALYSIS_SNAPSHOT_TARGET),
|
||||
binding,
|
||||
entries: [entry],
|
||||
entries: [entry(localPath)],
|
||||
workflow_entries: [],
|
||||
evidence_bundle: createEvidenceBundle([]),
|
||||
}).success,
|
||||
).toBe(false);
|
||||
}).success;
|
||||
expect(parsed("/fixture/ab")).toBe(true);
|
||||
expect(parsed("a\u0000b")).toBe(false);
|
||||
});
|
||||
|
||||
it("accepts snapshots with more than ten thousand analysis entries", () => {
|
||||
@@ -372,7 +388,7 @@ describe("analysis snapshot contract", () => {
|
||||
limitations: [],
|
||||
relationship_coverage: null,
|
||||
locations: [],
|
||||
subject: null,
|
||||
subject: evidenceSubject(unknownTarget("not-observed")),
|
||||
},
|
||||
};
|
||||
|
||||
@@ -631,12 +647,7 @@ it("binds raw Evidence only when its entry point matches the snapshot target", (
|
||||
limitations: [],
|
||||
relationship_coverage: null,
|
||||
locations: [],
|
||||
subject: {
|
||||
path: selected.path,
|
||||
sha256: selected.sha256,
|
||||
format: "raw-binary",
|
||||
architecture: "arm",
|
||||
},
|
||||
subject: evidenceSubject(selected),
|
||||
},
|
||||
},
|
||||
],
|
||||
@@ -691,12 +702,7 @@ describe("analysis snapshot Evidence binding", () => {
|
||||
const parameters = { address: "0x1000", document: "main" };
|
||||
const result = { name: "main" };
|
||||
const rawResult = { name: "provider-main" };
|
||||
const subject = {
|
||||
path: ANALYSIS_SNAPSHOT_TARGET.path,
|
||||
sha256: ANALYSIS_SNAPSHOT_TARGET.sha256,
|
||||
format: ANALYSIS_SNAPSHOT_TARGET.format,
|
||||
architecture: ANALYSIS_SNAPSHOT_TARGET.architecture ?? null,
|
||||
};
|
||||
const subject = evidenceSubject(ANALYSIS_SNAPSHOT_TARGET);
|
||||
const entry = {
|
||||
query_id: analysisQueryId(target, binding, "address_name", parameters),
|
||||
operation: "address_name",
|
||||
@@ -769,8 +775,8 @@ describe("analysis snapshot Evidence binding", () => {
|
||||
},
|
||||
(altered: typeof snapshot) => {
|
||||
const subject = altered.entries[0]?.execution.subject;
|
||||
if (subject !== null && subject !== undefined)
|
||||
subject.path = "/different/path";
|
||||
if (subject === undefined) throw new TypeError("missing fixture entry");
|
||||
Reflect.set(subject, "local_path", "/different/path");
|
||||
},
|
||||
(altered: typeof snapshot) => {
|
||||
const entry = altered.entries[0];
|
||||
|
||||
+79
-152
@@ -3,17 +3,15 @@ import { createHash } from "node:crypto";
|
||||
import canonicalize from "canonicalize";
|
||||
import { z } from "zod";
|
||||
|
||||
import { withEvidenceSubjectDeclaration } from "./evidenceSubject.js";
|
||||
|
||||
import {
|
||||
compareUnicodeCodePoints,
|
||||
compositeKey,
|
||||
} from "./unicodeCodePointOrder.js";
|
||||
import { localPathStringSchema } from "./localPath.js";
|
||||
import {
|
||||
binaryTargetIdentity,
|
||||
fileTargetIdentitySchema,
|
||||
sameTargetIdentity,
|
||||
type FileTargetIdentity,
|
||||
type FileTargetSelection,
|
||||
} from "./targetIdentity.js";
|
||||
|
||||
import {
|
||||
@@ -38,11 +36,10 @@ import {
|
||||
import {
|
||||
evidenceFileTarget,
|
||||
evidenceLocationSchema,
|
||||
evidenceSubjectTarget,
|
||||
evidenceSubjectSchema,
|
||||
rebindEvidenceToTargetPath,
|
||||
type Evidence,
|
||||
type EvidenceLocation,
|
||||
type EvidenceSubjectTarget,
|
||||
} from "./evidence.js";
|
||||
import {
|
||||
jsonObjectSchema,
|
||||
@@ -52,10 +49,6 @@ import {
|
||||
import { digestSchema, prefixedDigestSchema } from "./../domain/digests.js";
|
||||
import { canonicalJsonDigestSteps } from "./canonicalJsonDigestSteps.js";
|
||||
import { freezeJsonSnapshot } from "./immutableJson.js";
|
||||
import {
|
||||
binaryArchitectureSchema,
|
||||
rawBinaryEntryPointSchema,
|
||||
} from "./rawBinary.js";
|
||||
import {
|
||||
relationshipCoverageSchema,
|
||||
type RelationshipCoverage,
|
||||
@@ -101,65 +94,66 @@ const formatSchema = z.enum([
|
||||
"uimage",
|
||||
]);
|
||||
const kindSchema = z.enum(["executable", "database", "archive", "artifact"]);
|
||||
const subjectBaseSchema = z.object({
|
||||
path: localPathStringSchema,
|
||||
sha256: digestSchema,
|
||||
});
|
||||
const subjectSchema = withEvidenceSubjectDeclaration(subjectBaseSchema);
|
||||
const targetIdentitySchema = z.object({
|
||||
sha256: digestSchema,
|
||||
kind: kindSchema,
|
||||
architecture: architectureSchema.nullable(),
|
||||
});
|
||||
type SnapshotFormat = z.infer<typeof formatSchema>;
|
||||
/** The selection each format's open targets make inside their bytes. */
|
||||
const SNAPSHOT_SELECTION_KINDS: Partial<
|
||||
Record<SnapshotFormat, FileTargetSelection["kind"]>
|
||||
> = {
|
||||
"raw-binary": "entry-point",
|
||||
"gamecube-dol": "entry-point",
|
||||
// The caller's inputs select how the dump is read, so they are identity.
|
||||
"esp-flash-image": "esp-flash",
|
||||
"mach-o-universal": "universal-slice",
|
||||
};
|
||||
const RAW_ONLY = new Set<string>(RAW_ONLY_ARCHITECTURES);
|
||||
/** Whether a format admits an architecture, as `binaryTargetIdentity` produces it. */
|
||||
const snapshotArchitectureAllowed = (
|
||||
format: SnapshotFormat,
|
||||
architecture: string | null,
|
||||
): boolean =>
|
||||
format === "raw-binary"
|
||||
? architecture !== null
|
||||
: format === "gamecube-dol"
|
||||
? architecture === "powerpc"
|
||||
: format === "esp-flash-image"
|
||||
? architecture === null
|
||||
: architecture === null || !RAW_ONLY.has(architecture);
|
||||
/**
|
||||
* The persisted target: the path-free identity `binaryTargetIdentity`
|
||||
* produces, plus the binary kind that query IDs commit to.
|
||||
*/
|
||||
const targetSchema = z
|
||||
.discriminatedUnion("format", [
|
||||
targetIdentitySchema.extend({
|
||||
format: z.enum(
|
||||
formatSchema.options.filter(
|
||||
(format) =>
|
||||
format !== "raw-binary" &&
|
||||
format !== "gamecube-dol" &&
|
||||
format !== "esp-flash-image",
|
||||
),
|
||||
),
|
||||
architecture: architectureSchema
|
||||
.exclude([...RAW_ONLY_ARCHITECTURES])
|
||||
.nullable(),
|
||||
.strictObject({
|
||||
kind: kindSchema,
|
||||
identity: fileTargetIdentitySchema.extend({
|
||||
format: formatSchema,
|
||||
architecture: architectureSchema.nullable(),
|
||||
}),
|
||||
targetIdentitySchema.extend({
|
||||
format: z.literal("raw-binary"),
|
||||
kind: z.literal("executable"),
|
||||
architecture: binaryArchitectureSchema,
|
||||
entry_point: rawBinaryEntryPointSchema,
|
||||
}),
|
||||
targetIdentitySchema.extend({
|
||||
format: z.literal("gamecube-dol"),
|
||||
kind: z.literal("executable"),
|
||||
architecture: z.literal("powerpc"),
|
||||
entry_point: rawBinaryEntryPointSchema,
|
||||
}),
|
||||
// The caller's inputs select how the dump is read, so they are identity.
|
||||
targetIdentitySchema.extend({
|
||||
format: z.literal("esp-flash-image"),
|
||||
kind: z.literal("archive"),
|
||||
architecture: z.null(),
|
||||
flash_offset: z.number().int().min(0).max(0xffff_ffff).nullable(),
|
||||
partition_table_offset: z
|
||||
.number()
|
||||
.int()
|
||||
.min(0)
|
||||
.max(0xffff_ffff)
|
||||
.nullable(),
|
||||
}),
|
||||
])
|
||||
.superRefine((target, context) => {
|
||||
})
|
||||
.superRefine(({ kind, identity }, context) => {
|
||||
// Query IDs commit to the kind, so a kind the format cannot have would
|
||||
// admit entries that no live lookup or export of this target can reach.
|
||||
if (target.kind !== BINARY_TARGET_FORMAT_KINDS[target.format])
|
||||
if (kind !== BINARY_TARGET_FORMAT_KINDS[identity.format])
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["kind"],
|
||||
message: `Snapshot target kind ${target.kind} does not match format ${target.format}`,
|
||||
message: `Snapshot target kind ${kind} does not match format ${identity.format}`,
|
||||
});
|
||||
if (!snapshotArchitectureAllowed(identity.format, identity.architecture))
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["identity", "architecture"],
|
||||
message: `Snapshot target architecture ${String(identity.architecture)} does not apply to format ${identity.format}`,
|
||||
});
|
||||
const expected = SNAPSHOT_SELECTION_KINDS[identity.format] ?? null;
|
||||
if ((identity.selection?.kind ?? null) !== expected)
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["identity", "selection"],
|
||||
message:
|
||||
expected === null
|
||||
? `Snapshot target format ${identity.format} makes no selection`
|
||||
: `Snapshot target format ${identity.format} requires a ${expected} selection`,
|
||||
});
|
||||
});
|
||||
const bindingSchema = z
|
||||
@@ -191,7 +185,7 @@ const entrySchema = z.object({
|
||||
limitations: z.array(z.string()),
|
||||
relationship_coverage: relationshipCoverageSchema.nullable(),
|
||||
locations: z.array(evidenceLocationSchema),
|
||||
subject: subjectSchema.nullable(),
|
||||
subject: evidenceSubjectSchema,
|
||||
}),
|
||||
});
|
||||
const workflowEntrySchema = entrySchema.extend({
|
||||
@@ -375,7 +369,7 @@ const snapshotContentDigest = (
|
||||
const metadata = jsonObjectSchema.parse({
|
||||
limitations: execution.limitations,
|
||||
locations: execution.locations,
|
||||
subject: executionSubjectContent(execution.subject ?? undefined),
|
||||
subject: subjectContent(execution.subject),
|
||||
relationship_coverage: execution.relationship_coverage,
|
||||
});
|
||||
return contentDigest({
|
||||
@@ -385,25 +379,11 @@ const snapshotContentDigest = (
|
||||
});
|
||||
};
|
||||
|
||||
/** The execution subject facts a snapshot content digest commits to. */
|
||||
const executionSubjectContent = (
|
||||
subject:
|
||||
| {
|
||||
readonly path: string;
|
||||
readonly sha256: string;
|
||||
readonly format: string;
|
||||
readonly architecture?: string | null;
|
||||
}
|
||||
| undefined,
|
||||
) =>
|
||||
subject === undefined
|
||||
? null
|
||||
: {
|
||||
local_path: subject.path,
|
||||
sha256: subject.sha256,
|
||||
format: subject.format,
|
||||
architecture: subject.architecture ?? null,
|
||||
};
|
||||
/** The subject facts a snapshot content digest commits to; the display name is not one. */
|
||||
const subjectContent = (subject: Evidence["subject"]) => ({
|
||||
identity: subject.identity,
|
||||
local_path: subject.local_path,
|
||||
});
|
||||
|
||||
const evidenceContentDigest = (evidence: Evidence): string => {
|
||||
return contentDigest({
|
||||
@@ -425,7 +405,7 @@ interface SnapshotExecution {
|
||||
readonly limitations: readonly string[];
|
||||
readonly relationshipCoverage?: RelationshipCoverage | null | undefined;
|
||||
readonly locations: readonly EvidenceLocation[];
|
||||
readonly subject: EvidenceSubjectTarget | null;
|
||||
readonly subject: Evidence["subject"];
|
||||
}
|
||||
|
||||
type SnapshotExecutionProjection = Pick<
|
||||
@@ -451,13 +431,7 @@ const snapshotExecutionProjection = (
|
||||
limitations: execution.limitations,
|
||||
relationship_coverage: execution.relationshipCoverage ?? null,
|
||||
locations: execution.locations,
|
||||
subject:
|
||||
execution.subject === null
|
||||
? null
|
||||
: {
|
||||
...execution.subject,
|
||||
architecture: execution.subject.architecture ?? null,
|
||||
},
|
||||
subject: execution.subject,
|
||||
});
|
||||
|
||||
const canonicalJson = (value: unknown): string => {
|
||||
@@ -468,39 +442,12 @@ const canonicalJson = (value: unknown): string => {
|
||||
return encoded;
|
||||
};
|
||||
|
||||
/** Project a binary into the path-free identity used for cache invalidation. */
|
||||
/** The persisted snapshot target of an open binary: its target identity and kind. */
|
||||
export const snapshotTarget = (target: BinaryTarget): AnalysisSnapshotTarget =>
|
||||
target.format === "raw-binary"
|
||||
? {
|
||||
sha256: target.sha256,
|
||||
kind: target.kind,
|
||||
architecture: target.architecture,
|
||||
format: target.format,
|
||||
entry_point: target.entryPoint,
|
||||
}
|
||||
: target.format === "gamecube-dol"
|
||||
? {
|
||||
sha256: target.sha256,
|
||||
kind: target.kind,
|
||||
architecture: target.architecture,
|
||||
format: target.format,
|
||||
entry_point: target.entryPoint,
|
||||
}
|
||||
: target.format === "esp-flash-image"
|
||||
? {
|
||||
sha256: target.sha256,
|
||||
kind: target.kind,
|
||||
architecture: null,
|
||||
format: target.format,
|
||||
flash_offset: target.espFlash.flashOffset,
|
||||
partition_table_offset: target.espFlash.partitionTableOffset,
|
||||
}
|
||||
: {
|
||||
sha256: target.sha256,
|
||||
kind: target.kind,
|
||||
architecture: target.architecture ?? null,
|
||||
format: target.format,
|
||||
};
|
||||
targetSchema.parse({
|
||||
kind: target.kind,
|
||||
identity: binaryTargetIdentity(target),
|
||||
});
|
||||
|
||||
/** Build the immutable provider/profile binding used by a cached snapshot. */
|
||||
export const snapshotBinding = (
|
||||
@@ -511,35 +458,12 @@ export const snapshotBinding = (
|
||||
analysis_profile: profile,
|
||||
});
|
||||
|
||||
/** The target identity a persisted snapshot target names. */
|
||||
export const snapshotTargetIdentity = (
|
||||
target: AnalysisSnapshotTarget,
|
||||
): FileTargetIdentity => ({
|
||||
kind: "file",
|
||||
sha256: target.sha256,
|
||||
format: target.format,
|
||||
architecture: target.architecture,
|
||||
selection:
|
||||
target.format === "raw-binary" || target.format === "gamecube-dol"
|
||||
? { kind: "entry-point", entry_point: target.entry_point }
|
||||
: target.format === "esp-flash-image"
|
||||
? {
|
||||
kind: "esp-flash",
|
||||
flash_offset: target.flash_offset,
|
||||
partition_table_offset: target.partition_table_offset,
|
||||
}
|
||||
: null,
|
||||
});
|
||||
|
||||
/** Compare an open binary with the immutable target identity in a snapshot. */
|
||||
export const snapshotMatchesTarget = (
|
||||
snapshot: AnalysisSnapshotTarget,
|
||||
target: BinaryTarget,
|
||||
): boolean =>
|
||||
sameTargetIdentity(
|
||||
snapshotTargetIdentity(snapshot),
|
||||
binaryTargetIdentity(target),
|
||||
);
|
||||
sameTargetIdentity(snapshot.identity, binaryTargetIdentity(target));
|
||||
|
||||
/** Compare a snapshot binding with the selected concrete analysis profile. */
|
||||
export const snapshotMatchesProfile = (
|
||||
@@ -824,7 +748,10 @@ const restoreSnapshot = (
|
||||
providerRows: readonly Extract<PreparedEntry, { kind: "provider" }>[],
|
||||
workflowRows: readonly Extract<PreparedEntry, { kind: "workflow" }>[],
|
||||
): AnalysisSnapshot => {
|
||||
const scopedEvidence = evidenceBundleForTarget(bundle, header.target.sha256);
|
||||
const scopedEvidence = evidenceBundleForTarget(
|
||||
bundle,
|
||||
header.target.identity.sha256,
|
||||
);
|
||||
if (
|
||||
scopedEvidence.records.length !== bundle.records.length ||
|
||||
scopedEvidence.unknowns.length !== bundle.unknowns.length
|
||||
@@ -843,7 +770,7 @@ const restoreSnapshot = (
|
||||
if (
|
||||
evidence.predicate_type !== "rea.analysis" ||
|
||||
evidence.authority !== "shipped-artifact" ||
|
||||
evidenceFileTarget(evidence)?.sha256 !== header.target.sha256 ||
|
||||
evidenceFileTarget(evidence)?.sha256 !== header.target.identity.sha256 ||
|
||||
evidence.analysis_profile === null ||
|
||||
(evidence.confidence !== "observed" && evidence.confidence !== "derived")
|
||||
)
|
||||
@@ -977,8 +904,8 @@ const subjectNamesSnapshotTarget = (
|
||||
target: AnalysisSnapshotTarget,
|
||||
): boolean => {
|
||||
const file = evidenceFileTarget(evidence);
|
||||
if (file === undefined || file.sha256 !== target.sha256) return false;
|
||||
const expected = snapshotTargetIdentity(target);
|
||||
const expected = target.identity;
|
||||
if (file === undefined || file.sha256 !== expected.sha256) return false;
|
||||
return file.format !== expected.format || sameTargetIdentity(file, expected);
|
||||
};
|
||||
|
||||
@@ -1090,7 +1017,7 @@ const evidenceContentMetadata = (evidence: Evidence) => {
|
||||
limitations: evidence.limitations,
|
||||
relationship_coverage: evidence.relationship_coverage,
|
||||
locations: evidence.locations,
|
||||
subject: executionSubjectContent(evidenceSubjectTarget(evidence.subject)),
|
||||
subject: subjectContent(evidence.subject),
|
||||
};
|
||||
};
|
||||
|
||||
|
||||
@@ -6,9 +6,11 @@ import {
|
||||
browserOriginSchema,
|
||||
recordedBrowserOriginSchema,
|
||||
recordedBrowserObservationEndpointSchema,
|
||||
browserUrlTargetIdentity,
|
||||
sanitizeEndpointCandidate,
|
||||
sanitizeBrowserUrl,
|
||||
} from "./browserObservation.js";
|
||||
import { unknownTarget, urlTargetIdentitySchema } from "./targetIdentity.js";
|
||||
import { webPageInspectionSchema } from "./browserObservationSchemas.js";
|
||||
import { createWebPageInspection } from "../../tests/support/browser/webPageInspection.js";
|
||||
|
||||
@@ -345,3 +347,40 @@ describe("page inspection producer facts", () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("browser URL target identity", () => {
|
||||
it("names the reported page URL without userinfo, keeping its query and fragment", () => {
|
||||
const identity = browserUrlTargetIdentity(
|
||||
"https://user:pass@app.example.test/path?token=secret#part",
|
||||
);
|
||||
expect(identity).toEqual({
|
||||
kind: "url",
|
||||
url: "https://app.example.test/path?token=secret#part",
|
||||
origin: "https://app.example.test",
|
||||
});
|
||||
expect(urlTargetIdentitySchema.parse(identity)).toEqual(identity);
|
||||
|
||||
const backslash = browserUrlTargetIdentity(
|
||||
String.raw`https:\\user:pass@app.example.test/path`,
|
||||
);
|
||||
expect(urlTargetIdentitySchema.parse(backslash)).toMatchObject({
|
||||
origin: "https://app.example.test",
|
||||
});
|
||||
expect(JSON.stringify(backslash)).not.toContain("user:pass");
|
||||
});
|
||||
|
||||
it("names an opaque origin as null", () => {
|
||||
expect(browserUrlTargetIdentity("about:blank")).toEqual({
|
||||
kind: "url",
|
||||
url: "about:blank",
|
||||
origin: null,
|
||||
});
|
||||
});
|
||||
|
||||
it("does not identify a document whose URL was withheld or is not a URL", () => {
|
||||
for (const value of ["[unsupported-url]", ""])
|
||||
expect(browserUrlTargetIdentity(value)).toEqual(
|
||||
unknownTarget("not-observed"),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -9,6 +9,11 @@ import {
|
||||
parseRecordedTransportUrl,
|
||||
parseUrlQueryParameterNames,
|
||||
} from "./recordedTransportUrl.js";
|
||||
import {
|
||||
unknownTarget,
|
||||
type UnknownTargetIdentity,
|
||||
type UrlTargetIdentity,
|
||||
} from "./targetIdentity.js";
|
||||
|
||||
const parseExactOrigin = (value: string): string | undefined => {
|
||||
let url: URL;
|
||||
@@ -336,6 +341,25 @@ const credentialFreeHref = (parsed: URL): string => {
|
||||
return parsed.href;
|
||||
};
|
||||
|
||||
/**
|
||||
* The target identity of the live document URL a browser provider reported:
|
||||
* the URL without userinfo, and its origin. A value that is not a URL, such
|
||||
* as the placeholder for a document outside the selected origins, does not
|
||||
* identify the document.
|
||||
*/
|
||||
export const browserUrlTargetIdentity = (
|
||||
value: string,
|
||||
): UrlTargetIdentity | UnknownTargetIdentity => {
|
||||
const observation = observeBrowserUrl(value);
|
||||
return observation.kind === "parsed"
|
||||
? {
|
||||
kind: "url",
|
||||
url: observation.sanitized.url,
|
||||
origin: observation.sanitized.origin,
|
||||
}
|
||||
: unknownTarget("not-observed");
|
||||
};
|
||||
|
||||
/** Normalize endpoint candidates without discarding local query or fragment data. */
|
||||
export const sanitizeEndpointCandidate = (value: string): string => {
|
||||
return sanitizeBrowserUrl(value).url;
|
||||
|
||||
+13
-10
@@ -15,8 +15,8 @@ import { nulFreeStringSchema } from "./localPath.js";
|
||||
import {
|
||||
binaryTargetIdentity,
|
||||
fileTargetSelectionSchema,
|
||||
unknownTargetIdentitySchema,
|
||||
type UnknownTargetIdentity,
|
||||
nonFileTargetIdentitySchema,
|
||||
type NonFileTargetIdentity,
|
||||
sameTargetIdentity,
|
||||
type FileTargetIdentity,
|
||||
type FileTargetSelection,
|
||||
@@ -63,6 +63,7 @@ const SELECTION_FORMATS: Readonly<
|
||||
> = {
|
||||
"entry-point": ["raw-binary", "gamecube-dol"],
|
||||
"esp-flash": ["esp-flash-image"],
|
||||
"universal-slice": ["mach-o-universal"],
|
||||
};
|
||||
/** The file identity an Evidence subject commits to, with its format-owned declaration. */
|
||||
const subjectIdentitySchema = withEvidenceSubjectDeclaration(
|
||||
@@ -88,13 +89,15 @@ const fileSubjectSchema = z.strictObject({
|
||||
name: z.string().min(1),
|
||||
local_path: nulFreeStringSchema,
|
||||
});
|
||||
/** A subject REA did not identify has no display facts to show. */
|
||||
const unknownSubjectSchema = z.strictObject({
|
||||
identity: unknownTargetIdentitySchema,
|
||||
/** A subject that names no file (a URL, or an unknown target) has no file display facts. */
|
||||
const nonFileSubjectSchema = z.strictObject({
|
||||
identity: nonFileTargetIdentitySchema,
|
||||
name: z.null(),
|
||||
local_path: z.null(),
|
||||
});
|
||||
const subjectSchema = z.union([fileSubjectSchema, unknownSubjectSchema]);
|
||||
const subjectSchema = z.union([fileSubjectSchema, nonFileSubjectSchema]);
|
||||
/** The Evidence subject, for persistence formats that embed one. */
|
||||
export const evidenceSubjectSchema = subjectSchema;
|
||||
/** Source location attached to an evidence observation. */
|
||||
/** Source-location schema shared by evidence-bearing persistence formats. */
|
||||
export const evidenceLocationSchema = z.discriminatedUnion("kind", [
|
||||
@@ -239,11 +242,11 @@ export interface EvidenceSubjectTarget {
|
||||
readonly name?: string;
|
||||
}
|
||||
|
||||
/** What a producer names as an Evidence subject: a file target, or an explicit unknown. */
|
||||
/** What a producer names as an Evidence subject: a file target, or a non-file identity such as a URL or an explicit unknown. */
|
||||
export type EvidenceSubjectInput =
|
||||
| EvidenceSubjectTarget
|
||||
| BinaryTarget
|
||||
| UnknownTargetIdentity;
|
||||
| NonFileTargetIdentity;
|
||||
|
||||
type FileEvidenceSubject = z.infer<typeof fileSubjectSchema>;
|
||||
|
||||
@@ -271,13 +274,13 @@ export const evidenceSubjectTarget = (
|
||||
: { selection: subject.identity.selection }),
|
||||
};
|
||||
|
||||
/** Name the same subject in a derived observation: the file target, or the same unknown. */
|
||||
/** Name the same subject in a derived observation: the file target, or the same non-file identity. */
|
||||
export const evidenceSubjectInput = (
|
||||
subject: Evidence["subject"],
|
||||
): EvidenceSubjectInput =>
|
||||
isFileSubject(subject)
|
||||
? (evidenceSubjectTarget(subject) as EvidenceSubjectTarget)
|
||||
: (subject.identity as UnknownTargetIdentity);
|
||||
: (subject.identity as NonFileTargetIdentity);
|
||||
|
||||
/** The one projection of a producer's subject input into its target identity. */
|
||||
export const evidenceSubjectTargetIdentity = (
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { snapshotTarget, snapshotTargetIdentity } from "./analysisSnapshot.js";
|
||||
import type { BinaryTarget } from "./binaryTargetTypes.js";
|
||||
import {
|
||||
binaryTargetIdentity,
|
||||
sameTargetIdentity,
|
||||
targetIdentityKey,
|
||||
urlTargetIdentitySchema,
|
||||
type FileTargetIdentity,
|
||||
} from "./targetIdentity.js";
|
||||
|
||||
@@ -42,7 +43,7 @@ describe("target identity key", () => {
|
||||
entryPoint: "0x8000000",
|
||||
entryPointSource: "0x08000000",
|
||||
};
|
||||
const identity = snapshotTargetIdentity(snapshotTarget(raw));
|
||||
const identity = binaryTargetIdentity(raw);
|
||||
expect(identity).toEqual({
|
||||
kind: "file",
|
||||
sha256: SHA256,
|
||||
@@ -50,9 +51,7 @@ describe("target identity key", () => {
|
||||
architecture: "arm",
|
||||
selection: { kind: "entry-point", entry_point: "0x8000000" },
|
||||
});
|
||||
const moved = snapshotTargetIdentity(
|
||||
snapshotTarget({ ...raw, entryPoint: "0x9000000" }),
|
||||
);
|
||||
const moved = binaryTargetIdentity({ ...raw, entryPoint: "0x9000000" });
|
||||
expect(sameTargetIdentity(identity, moved)).toBe(false);
|
||||
|
||||
const flash: BinaryTarget = {
|
||||
@@ -62,7 +61,7 @@ describe("target identity key", () => {
|
||||
format: "esp-flash-image",
|
||||
espFlash: { selected: true, flashOffset: 0, partitionTableOffset: null },
|
||||
};
|
||||
const flashIdentity = snapshotTargetIdentity(snapshotTarget(flash));
|
||||
const flashIdentity = binaryTargetIdentity(flash);
|
||||
expect(flashIdentity.selection).toEqual({
|
||||
kind: "esp-flash",
|
||||
flash_offset: 0,
|
||||
@@ -71,22 +70,82 @@ describe("target identity key", () => {
|
||||
expect(
|
||||
sameTargetIdentity(
|
||||
flashIdentity,
|
||||
snapshotTargetIdentity(
|
||||
snapshotTarget({
|
||||
...flash,
|
||||
espFlash: { ...flash.espFlash, flashOffset: 0x1000 },
|
||||
}),
|
||||
),
|
||||
binaryTargetIdentity({
|
||||
...flash,
|
||||
espFlash: { ...flash.espFlash, flashOffset: 0x1000 },
|
||||
}),
|
||||
),
|
||||
).toBe(false);
|
||||
// The path is display, not identity.
|
||||
expect(
|
||||
sameTargetIdentity(
|
||||
flashIdentity,
|
||||
snapshotTargetIdentity(
|
||||
snapshotTarget({ ...flash, path: "/elsewhere" }),
|
||||
),
|
||||
binaryTargetIdentity({ ...flash, path: "/elsewhere" }),
|
||||
),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("separates the slices of one universal Mach-O container", () => {
|
||||
const universal: BinaryTarget = {
|
||||
path: "/fixture/universal",
|
||||
sha256: SHA256,
|
||||
kind: "executable",
|
||||
format: "mach-o-universal",
|
||||
architecture: "arm64",
|
||||
availableArchitectures: ["x86_64", "arm64"],
|
||||
universalSlice: { offset: 0x8000, size: 0x4000 },
|
||||
universalSlices: [
|
||||
{ offset: 0x4000, size: 0x2000, cpuType: 0x0100_0007, cpuSubtype: 3 },
|
||||
{ offset: 0x8000, size: 0x4000, cpuType: 0x0100_000c, cpuSubtype: 0 },
|
||||
],
|
||||
};
|
||||
const arm64 = binaryTargetIdentity(universal);
|
||||
expect(arm64.selection).toEqual({
|
||||
kind: "universal-slice",
|
||||
offset: 0x8000,
|
||||
size: 0x4000,
|
||||
cpu_type: 0x0100_000c,
|
||||
cpu_subtype: 0,
|
||||
});
|
||||
const x86 = binaryTargetIdentity({
|
||||
...universal,
|
||||
architecture: "x86_64",
|
||||
universalSlice: { offset: 0x4000, size: 0x2000 },
|
||||
});
|
||||
expect(sameTargetIdentity(arm64, x86)).toBe(false);
|
||||
// Without the admitted FAT table the slice range still selects, and the
|
||||
// CPU declaration stays explicitly unknown.
|
||||
const { universalSlices: _slices, ...unadmitted } = universal;
|
||||
expect(binaryTargetIdentity(unadmitted).selection).toEqual({
|
||||
kind: "universal-slice",
|
||||
offset: 0x8000,
|
||||
size: 0x4000,
|
||||
cpu_type: null,
|
||||
cpu_subtype: null,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("URL target identity", () => {
|
||||
const page = {
|
||||
kind: "url",
|
||||
url: "https://app.example.test/orders?id=1",
|
||||
origin: "https://app.example.test",
|
||||
} as const;
|
||||
|
||||
it("separates two pages of one origin", () => {
|
||||
expect(urlTargetIdentitySchema.parse(page)).toEqual(page);
|
||||
expect(
|
||||
sameTargetIdentity(page, { ...page, url: "https://app.example.test/" }),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["userinfo", { ...page, url: "https://user:pass@app.example.test/" }],
|
||||
["a URL that does not parse", { ...page, url: "not a url" }],
|
||||
["an origin the URL does not have", { ...page, origin: "https://x.test" }],
|
||||
["a null origin for a tuple origin", { ...page, origin: null }],
|
||||
])("refuses %s", (_case, identity) => {
|
||||
expect(urlTargetIdentitySchema.safeParse(identity).success).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -3,11 +3,15 @@ import { createHash } from "node:crypto";
|
||||
import canonicalize from "canonicalize";
|
||||
import { z } from "zod";
|
||||
|
||||
import type { BinaryTarget } from "./binaryTargetTypes.js";
|
||||
import type {
|
||||
BinaryTarget,
|
||||
UniversalMachOSliceIdentity,
|
||||
} from "./binaryTargetTypes.js";
|
||||
import { digestSchema } from "./digests.js";
|
||||
import { rawBinaryEntryPointSchema } from "./rawBinary.js";
|
||||
|
||||
const flashOffsetSchema = z.number().int().min(0).max(0xffff_ffff).nullable();
|
||||
const fatCpuSchema = z.number().int().min(0).max(0xffff_ffff).nullable();
|
||||
|
||||
/** Persisted facts inside a file's bytes that select what was analysed. */
|
||||
export const fileTargetSelectionSchema = z.discriminatedUnion("kind", [
|
||||
@@ -20,6 +24,15 @@ export const fileTargetSelectionSchema = z.discriminatedUnion("kind", [
|
||||
flash_offset: flashOffsetSchema,
|
||||
partition_table_offset: flashOffsetSchema,
|
||||
}),
|
||||
// One slice of a universal Mach-O. The FAT CPU declaration is null when
|
||||
// target admission did not supply the validated FAT table.
|
||||
z.strictObject({
|
||||
kind: z.literal("universal-slice"),
|
||||
offset: z.number().int().min(0).max(Number.MAX_SAFE_INTEGER),
|
||||
size: z.number().int().min(1).max(Number.MAX_SAFE_INTEGER),
|
||||
cpu_type: fatCpuSchema,
|
||||
cpu_subtype: fatCpuSchema,
|
||||
}),
|
||||
]);
|
||||
|
||||
/**
|
||||
@@ -78,11 +91,87 @@ export const unknownTarget = (
|
||||
reason: UnknownTargetIdentity["reason"],
|
||||
): UnknownTargetIdentity => ({ kind: "unknown", reason });
|
||||
|
||||
/** The origin a URL serializes to, or null for an opaque origin. */
|
||||
const urlOrigin = (parsed: URL): string | null =>
|
||||
parsed.origin === "null" ? null : parsed.origin;
|
||||
|
||||
/**
|
||||
* A live document's URL with userinfo removed, and its origin. Userinfo is a
|
||||
* transport credential, so an identity that carries it is refused rather
|
||||
* than committed.
|
||||
*/
|
||||
export const urlTargetIdentitySchema = z
|
||||
.strictObject({
|
||||
kind: z.literal("url"),
|
||||
url: z.string().min(1),
|
||||
origin: z.string().min(1).nullable(),
|
||||
})
|
||||
.superRefine((identity, context) => {
|
||||
const parsed = URL.parse(identity.url);
|
||||
if (parsed === null) {
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["url"],
|
||||
message: "A URL target identity must be a parseable URL",
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (parsed.username !== "" || parsed.password !== "")
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["url"],
|
||||
message: "A URL target identity must not carry userinfo",
|
||||
});
|
||||
if (urlOrigin(parsed) !== identity.origin)
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["origin"],
|
||||
message: "A URL target identity's origin must be its URL's origin",
|
||||
});
|
||||
});
|
||||
|
||||
/** A live document's URL with userinfo removed, and its origin. */
|
||||
export interface UrlTargetIdentity {
|
||||
readonly kind: "url";
|
||||
readonly url: string;
|
||||
readonly origin: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* What an observation or cache entry is about. Tagged by `kind`; see
|
||||
* docs/adr/evidence/target-identity.md for the kinds later slices add.
|
||||
*/
|
||||
export type TargetIdentity = FileTargetIdentity | UnknownTargetIdentity;
|
||||
export type TargetIdentity =
|
||||
| FileTargetIdentity
|
||||
| UrlTargetIdentity
|
||||
| UnknownTargetIdentity;
|
||||
|
||||
/** A target identity that names no file, and so has no file display facts. */
|
||||
export type NonFileTargetIdentity = Exclude<TargetIdentity, FileTargetIdentity>;
|
||||
|
||||
/** The identities a subject without file display facts can commit to. */
|
||||
export const nonFileTargetIdentitySchema = z.union([
|
||||
unknownTargetIdentitySchema,
|
||||
urlTargetIdentitySchema,
|
||||
]);
|
||||
|
||||
/** The universal Mach-O slice a target selects, with its FAT CPU declaration when admitted. */
|
||||
const universalSliceSelection = (
|
||||
slice: { readonly offset: number; readonly size: number },
|
||||
declarations: readonly UniversalMachOSliceIdentity[] | undefined,
|
||||
): FileTargetSelection => {
|
||||
const declared = declarations?.find(
|
||||
(candidate) =>
|
||||
candidate.offset === slice.offset && candidate.size === slice.size,
|
||||
);
|
||||
return {
|
||||
kind: "universal-slice",
|
||||
offset: slice.offset,
|
||||
size: slice.size,
|
||||
cpu_type: declared?.cpuType ?? null,
|
||||
cpu_subtype: declared?.cpuSubtype ?? null,
|
||||
};
|
||||
};
|
||||
|
||||
/** The selection an open binary target makes inside its bytes, if any. */
|
||||
export const binaryTargetSelection = (
|
||||
@@ -96,7 +185,9 @@ export const binaryTargetSelection = (
|
||||
flash_offset: target.espFlash.flashOffset,
|
||||
partition_table_offset: target.espFlash.partitionTableOffset,
|
||||
}
|
||||
: null;
|
||||
: target.format === "mach-o-universal"
|
||||
? universalSliceSelection(target.universalSlice, target.universalSlices)
|
||||
: null;
|
||||
|
||||
/** The one projection of an open binary target into its target identity. */
|
||||
export const binaryTargetIdentity = (
|
||||
|
||||
@@ -1,4 +1,10 @@
|
||||
import { readFile, stat, writeFile } from "node:fs/promises";
|
||||
import {
|
||||
snapshotExecution,
|
||||
ANALYSIS_SNAPSHOT_PROFILE as PROFILE,
|
||||
ANALYSIS_SNAPSHOT_PROVIDER as PROVIDER,
|
||||
ANALYSIS_SNAPSHOT_TARGET as TARGET,
|
||||
} from "../../../src/domain/analysisSnapshot.fixture.js";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { describe, expect, it, onTestFinished, vi } from "vitest";
|
||||
@@ -22,11 +28,6 @@ import {
|
||||
snapshotTarget,
|
||||
type AnalysisSnapshot,
|
||||
} from "../../../src/domain/analysisSnapshot.js";
|
||||
import {
|
||||
ANALYSIS_SNAPSHOT_PROFILE as PROFILE,
|
||||
ANALYSIS_SNAPSHOT_PROVIDER as PROVIDER,
|
||||
ANALYSIS_SNAPSHOT_TARGET as TARGET,
|
||||
} from "../../../src/domain/analysisSnapshot.fixture.js";
|
||||
import { createEvidence } from "../../../src/domain/evidence.js";
|
||||
import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js";
|
||||
import { readJsonObjectFile } from "../../../src/application/JsonInputFile.js";
|
||||
@@ -44,21 +45,23 @@ const snapshotFixture = (): AnalysisSnapshot => {
|
||||
binding,
|
||||
operation: "address_name",
|
||||
parameters: { address: "0x1000" },
|
||||
execution: createAnalysisExecution(
|
||||
{ name: "main", "😀": { "\uE000": "unicode" } },
|
||||
PROVIDER,
|
||||
{
|
||||
analysisProfile: PROFILE,
|
||||
rawResult: { name: "main", "😀": { "\uE000": "unicode" } },
|
||||
subject: {
|
||||
path: TARGET.path,
|
||||
sha256: TARGET.sha256,
|
||||
format: TARGET.format,
|
||||
...(TARGET.architecture === undefined
|
||||
? {}
|
||||
: { architecture: TARGET.architecture }),
|
||||
execution: snapshotExecution(
|
||||
createAnalysisExecution(
|
||||
{ name: "main", "😀": { "\uE000": "unicode" } },
|
||||
PROVIDER,
|
||||
{
|
||||
analysisProfile: PROFILE,
|
||||
rawResult: { name: "main", "😀": { "\uE000": "unicode" } },
|
||||
subject: {
|
||||
path: TARGET.path,
|
||||
sha256: TARGET.sha256,
|
||||
format: TARGET.format,
|
||||
...(TARGET.architecture === undefined
|
||||
? {}
|
||||
: { architecture: TARGET.architecture }),
|
||||
},
|
||||
},
|
||||
},
|
||||
),
|
||||
),
|
||||
});
|
||||
const evidence = createEvidence(TARGET, PROVIDER, {
|
||||
|
||||
@@ -34,6 +34,7 @@ import {
|
||||
} from "../../../src/application/binary/AnalysisSnapshotFiles.js";
|
||||
import {
|
||||
createEvidence,
|
||||
evidenceSubject,
|
||||
parseEvidence,
|
||||
type Evidence,
|
||||
} from "../../../src/domain/evidence.js";
|
||||
@@ -509,11 +510,11 @@ const withAlternateWorkflowProfile = (
|
||||
const result = current.normalized_result;
|
||||
const subject = {
|
||||
path,
|
||||
sha256: snapshot.target.sha256,
|
||||
format: snapshot.target.format,
|
||||
...(snapshot.target.architecture === null
|
||||
sha256: snapshot.target.identity.sha256,
|
||||
format: snapshot.target.identity.format,
|
||||
...(snapshot.target.identity.architecture === null
|
||||
? {}
|
||||
: { architecture: snapshot.target.architecture }),
|
||||
: { architecture: snapshot.target.identity.architecture }),
|
||||
};
|
||||
const alternateEvidence = createEvidence(subject, REA_WORKFLOW_PROVIDER, {
|
||||
operation: current.operation,
|
||||
@@ -537,7 +538,7 @@ const withAlternateWorkflowProfile = (
|
||||
analysisProfile: alternateProfile,
|
||||
limitations: current.limitations,
|
||||
locations: current.locations,
|
||||
subject,
|
||||
subject: alternateEvidence.subject,
|
||||
},
|
||||
});
|
||||
return {
|
||||
@@ -559,7 +560,7 @@ const withEarlierHistoricalEvidence = (
|
||||
createEvidence(
|
||||
{
|
||||
path,
|
||||
sha256: snapshot.target.sha256,
|
||||
sha256: snapshot.target.identity.sha256,
|
||||
format: "analysis-database",
|
||||
},
|
||||
REA_WORKFLOW_PROVIDER,
|
||||
@@ -723,7 +724,7 @@ describe("snapshot output cleanup lifecycle", () => {
|
||||
if (!saved.ok) throw saved.error;
|
||||
const loaded = await readAnalysisSnapshot(path);
|
||||
if (!loaded.ok) throw loaded.error;
|
||||
expect(loaded.value.target.sha256).toBe(opened.value.sha256);
|
||||
expect(loaded.value.target.identity.sha256).toBe(opened.value.sha256);
|
||||
}
|
||||
} finally {
|
||||
const closed = await session.close();
|
||||
@@ -1538,8 +1539,10 @@ describe("snapshot replay provider identity", () => {
|
||||
details: {
|
||||
binding_mismatch: {
|
||||
target: {
|
||||
snapshot: { sha256: firstSnapshot.value.target.sha256 },
|
||||
selected: { sha256: expect.any(String) },
|
||||
snapshot: {
|
||||
identity: { sha256: firstSnapshot.value.target.identity.sha256 },
|
||||
},
|
||||
selected: { identity: { sha256: expect.any(String) } },
|
||||
},
|
||||
profile_comparison: "not_compared",
|
||||
},
|
||||
@@ -1772,7 +1775,7 @@ const expectProfilelessWorkflowSnapshotRejections = async (
|
||||
analysisProfile: mixedProfile,
|
||||
limitations: [],
|
||||
locations: [],
|
||||
subject: null,
|
||||
subject: evidenceSubject(unknownTarget("not-observed")),
|
||||
},
|
||||
}),
|
||||
],
|
||||
@@ -1821,20 +1824,7 @@ const expectProfilelessWorkflowSnapshotRejections = async (
|
||||
originalWorkflow.execution.relationship_coverage,
|
||||
}),
|
||||
locations: originalWorkflow.execution.locations,
|
||||
subject:
|
||||
originalWorkflow.execution.subject === null
|
||||
? null
|
||||
: {
|
||||
path: originalWorkflow.execution.subject.path,
|
||||
sha256: originalWorkflow.execution.subject.sha256,
|
||||
format: originalWorkflow.execution.subject.format,
|
||||
...(originalWorkflow.execution.subject.architecture === null
|
||||
? {}
|
||||
: {
|
||||
architecture:
|
||||
originalWorkflow.execution.subject.architecture,
|
||||
}),
|
||||
},
|
||||
subject: originalWorkflow.execution.subject,
|
||||
},
|
||||
}),
|
||||
],
|
||||
@@ -2059,8 +2049,8 @@ it.each([true, false])(
|
||||
const historical = createEvidence(
|
||||
{
|
||||
path: fixture.path,
|
||||
sha256: saved.value.target.sha256,
|
||||
format: saved.value.target.format,
|
||||
sha256: saved.value.target.identity.sha256,
|
||||
format: saved.value.target.identity.format,
|
||||
},
|
||||
evidence.provider,
|
||||
{
|
||||
|
||||
@@ -442,9 +442,32 @@ it("exposes CLI-equivalent Evidence results and session tools", async () => {
|
||||
observations: { source_maps: { status: "not_requested" } },
|
||||
},
|
||||
});
|
||||
// A page observation names the page URL the inspection reported; a target
|
||||
// list names no single document.
|
||||
const page = {
|
||||
kind: "url",
|
||||
url: webPageInspectionSchema.parse(
|
||||
normalizedResultOf(inspected.structuredContent),
|
||||
).target.url,
|
||||
origin: browser.allowedOrigin,
|
||||
};
|
||||
expect(page.url).toBe(`${browser.allowedOrigin}/app?token=frame-secret`);
|
||||
expect(subjectOf(listed.structuredContent)).toEqual({
|
||||
identity: { kind: "unknown", reason: "not-observed" },
|
||||
name: null,
|
||||
local_path: null,
|
||||
});
|
||||
for (const result of [inspected, analyzed])
|
||||
expect(subjectOf(result.structuredContent)).toEqual({
|
||||
identity: page,
|
||||
name: null,
|
||||
local_path: null,
|
||||
});
|
||||
await verifySessionAndComparisonTools(connected, browser, inspected);
|
||||
});
|
||||
|
||||
const subjectOf = (content: unknown): unknown => parseEvidence(content).subject;
|
||||
|
||||
const verifySessionAndComparisonTools = async (
|
||||
connected: Awaited<ReturnType<typeof connectBrowser>>,
|
||||
browser: FakeCdpBrowser,
|
||||
@@ -550,6 +573,13 @@ const verifySessionAndComparisonTools = async (
|
||||
},
|
||||
});
|
||||
expect(JSON.stringify(malformedComparison.content)).toContain("types");
|
||||
for (const result of [observedSession, webMcp])
|
||||
expect(subjectOf(result.structuredContent)).toMatchObject({
|
||||
identity: { kind: "url", origin: browser.allowedOrigin },
|
||||
});
|
||||
expect(subjectOf(compared.structuredContent)).toMatchObject({
|
||||
identity: { kind: "unknown", reason: "not-observed" },
|
||||
});
|
||||
const incompleteComparison = await connected.client.callTool({
|
||||
name: "compare_web_captures",
|
||||
arguments: { before: { inspection: capture } },
|
||||
@@ -577,6 +607,9 @@ const verifyScreenshotTools = async (
|
||||
},
|
||||
});
|
||||
expect(screenshot.isError).not.toBe(true);
|
||||
expect(subjectOf(screenshot.structuredContent)).toMatchObject({
|
||||
identity: { kind: "url", origin: browser.allowedOrigin },
|
||||
});
|
||||
const screenshotArtifact = artifactOf(screenshot.structuredContent);
|
||||
const visual = await client.callTool({
|
||||
name: "compare_web_screenshots",
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { unknownTarget } from "../../../src/domain/targetIdentity.js";
|
||||
import { browserUrlTargetIdentity } from "../../../src/domain/browserObservation.js";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { afterAll, expect, it } from "vitest";
|
||||
@@ -19,7 +19,7 @@ afterAll(() => schemaValidation.clear());
|
||||
|
||||
const makeParent = () =>
|
||||
createEvidence(
|
||||
unknownTarget("not-observed"),
|
||||
browserUrlTargetIdentity("https://example.test/app"),
|
||||
{ id: "rea-browser", name: "REA Browser", version: "6.8.0" },
|
||||
{
|
||||
operation: "analyze_web_bundle",
|
||||
@@ -64,6 +64,10 @@ it("projects web bundle summary views over MCP using retained and inline sources
|
||||
{ toolDefinition: schemas.toolDefinition },
|
||||
);
|
||||
expect(summaryCall.isError, JSON.stringify(summaryCall)).not.toBe(true);
|
||||
// A view names the page its parent observed.
|
||||
expect(parseEvidence(summaryCall.structuredContent).subject).toEqual(
|
||||
parent.subject,
|
||||
);
|
||||
|
||||
const summaryResult = webBundleSummaryViewResultSchema.parse(
|
||||
parseEvidence(summaryCall.structuredContent).normalized_result,
|
||||
|
||||
@@ -219,6 +219,16 @@ cliTest(
|
||||
initial_url: `${browser.allowedOrigin}/app?caller=retained#details`,
|
||||
},
|
||||
});
|
||||
// The observation names the page it was armed on.
|
||||
expect(evidence.subject).toEqual({
|
||||
identity: {
|
||||
kind: "url",
|
||||
url: `${browser.allowedOrigin}/app?caller=retained#details`,
|
||||
origin: browser.allowedOrigin,
|
||||
},
|
||||
name: null,
|
||||
local_path: null,
|
||||
});
|
||||
expect(session.evidenceById(evidence.evidence_id)).toEqual(evidence);
|
||||
const cliResult = await cli.run({
|
||||
arguments: [
|
||||
|
||||
@@ -782,7 +782,7 @@ describe("Evidence display metadata refresh", () => {
|
||||
const refreshed = session.exportAnalysisSnapshot();
|
||||
if (!refreshed.ok) throw refreshed.error;
|
||||
if (scope === "active") {
|
||||
expect(refreshed.value.entries[0]?.execution.subject?.path).toBe(
|
||||
expect(refreshed.value.entries[0]?.execution.subject.local_path).toBe(
|
||||
relocated.subject.local_path,
|
||||
);
|
||||
expect(refreshed.value.evidence_bundle.records).toContainEqual(
|
||||
@@ -986,8 +986,10 @@ describe("replay of exact immutable calls", () => {
|
||||
expect(projectAnalysisError(opened.error).details).toMatchObject({
|
||||
binding_mismatch: {
|
||||
target: {
|
||||
snapshot: { sha256: snapshot.value.target.sha256 },
|
||||
selected: { sha256: expect.any(String) },
|
||||
snapshot: {
|
||||
identity: { sha256: snapshot.value.target.identity.sha256 },
|
||||
},
|
||||
selected: { identity: { sha256: expect.any(String) } },
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
@@ -108,7 +108,7 @@ it.each(cases)(
|
||||
analysisProfile: profile,
|
||||
limitations: [],
|
||||
locations: [],
|
||||
subject: target,
|
||||
subject: evidence.subject,
|
||||
},
|
||||
});
|
||||
const snapshot = {
|
||||
@@ -121,13 +121,13 @@ it.each(cases)(
|
||||
const restored = parseAnalysisSnapshot(
|
||||
JSON.parse(serializeAnalysisSnapshot(snapshot)),
|
||||
);
|
||||
expect(restored.target.architecture).toBe(architecture);
|
||||
expect(restored.target.identity.architecture).toBe(architecture);
|
||||
expect(
|
||||
fileIdentity(restored.evidence_bundle.records[0]?.subject).architecture,
|
||||
).toBe(architecture);
|
||||
expect(restored.entries[0]?.execution.subject?.architecture).toBe(
|
||||
architecture,
|
||||
);
|
||||
expect(
|
||||
fileIdentity(restored.entries[0]?.execution.subject).architecture,
|
||||
).toBe(architecture);
|
||||
expect(
|
||||
sessionOutputSchemas.open_binary.parse({ result: target }).result
|
||||
.architecture,
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { rm, writeFile } from "node:fs/promises";
|
||||
import { snapshotExecution } from "../../src/domain/analysisSnapshot.fixture.js";
|
||||
import { join } from "node:path";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { projectAnalysisError } from "../../src/domain/analysisErrorProjection.js";
|
||||
@@ -73,7 +74,7 @@ const reobservesExternallyMutableState = async (): Promise<void> => {
|
||||
binding: snapshotBinding(profile),
|
||||
operation: "procedure_pseudo_code",
|
||||
parameters: query,
|
||||
execution: cached,
|
||||
execution: snapshotExecution(cached),
|
||||
}),
|
||||
);
|
||||
snapshot.value.evidence_bundle = createEvidenceBundle([
|
||||
|
||||
Reference in new issue
Block a user