feat(snapshots)!: persist the target identity as the snapshot target (#11551)

* feat(snapshots)!: persist the target identity as the snapshot target

The snapshot target is now `{ kind, identity }`, where `identity` is the
FileTargetIdentity that `binaryTargetIdentity` produces and `kind` is the
binary kind query IDs commit to. The per-format target projection and
`snapshotTargetIdentity` are deleted; the schema checks the format's
selection, architecture and kind instead of keeping one union arm per format.

Entry execution subjects use the Evidence subject shape (file or explicit
unknown), so a snapshot entry commits to the same identity, including the
selection, as its Evidence record. Workflow entries take the subject of the
workflow's own Evidence. Replay no longer re-derives the selection from the
open target.

`FileTargetSelection` gains `universal-slice` (`offset`, `size`, and the FAT
`cpu_type`/`cpu_subtype` when target admission supplied the FAT table, else
null). `binaryTargetIdentity` produces it for `mach-o-universal` targets, so
Evidence and snapshots for different slices of one container no longer share
an identity.

Refs #11417

BREAKING CHANGE: analysis snapshot files change shape: `target` is
`{ kind, identity: { kind: "file", sha256, format, architecture, selection } }`
and entry `execution.subject` is the Evidence subject
(`{ identity, name, local_path }`, never null). Snapshot binding mismatch
details report `target.snapshot`/`target.selected` in the same shape. Evidence
IDs of universal Mach-O targets change because their subject identity now
carries the selected slice. Recreate saved snapshots and Evidence bundles of
universal Mach-O targets.

* docs(evidence): name the universal Mach-O slice selection in the subject contract

* fix(snapshots): read the snapshot target identity in the Ghidra verification scripts

The real Ghidra snapshot checks (lifecycle, RISC-V, MIPS, PowerPC) read
snapshot.target.sha256 and .architecture, which moved under
snapshot.target.identity, so they would fail after saving a valid snapshot.

* feat(evidence)!: name live browser page Evidence by its page URL (#11559)

Browser page observations (inspect_web_page, analyze_web_bundle,
observe_web_session, discover_webmcp_tools, capture_web_screenshot,
observe_web_execution, inspect_web_event_listeners) named an unknown
(`not-observed`) subject although the provider reports the page URL.

`TargetIdentity` gains a `url` kind: the reported URL with userinfo removed
and its origin (`null` for an opaque origin). The schema refuses a URL that
carries userinfo or an origin the URL does not have.
`browserUrlTargetIdentity` in the browser URL owner builds it from the
provider's reported URL; a withheld or unparseable URL stays unknown
(`not-observed`). Target lists and comparisons still name no single
document. Analysis views keep their parent's URL subject, as they already
keep a file subject.

BREAKING CHANGE: Evidence IDs of live browser page observations and their
analysis views change, and their subject is
`{ "kind": "url", "url", "origin" }` with `name` and `local_path` null.
Recreate stored Evidence for these tools. This lands inside Evidence format
revision 2 under the release hold (#11417); no `evidence_format` bump.
This commit is contained in:
morluto authored and GitHub committed 2026-11-16 07:04:34 +08:00
1 parent f6df353002
commit ff0176256a
30 files changed
+678 -396

No files matched your search

+47 -3
View File
@@ -1,4 +1,7 @@
import { unknownTarget } from "../domain/targetIdentity.js";
import {
unknownTarget,
type NonFileTargetIdentity,
} from "../domain/targetIdentity.js";
import { isDeepStrictEqual } from "node:util";
import type { ProviderIdentity } from "../ports/AnalysisProvider.js";
import {
@@ -6,7 +9,10 @@ import {
type Evidence,
type EvidenceObservation,
} from "../domain/evidence.js";
import { sanitizeBrowserUrl } from "../domain/browserObservation.js";
import {
browserUrlTargetIdentity,
sanitizeBrowserUrl,
} from "../domain/browserObservation.js";
import type {
InspectWebPageInput,
ListBrowserTargetsInput,
@@ -96,7 +102,7 @@ export const createBrowserEvidence = (
const comparison =
operation === "compare_web_captures" ||
operation === "compare_web_screenshots";
return createEvidence(unknownTarget("not-observed"), provider, {
return createEvidence(browserSubject(operation, observed), provider, {
predicateType: browserPredicate(operation),
operation,
parameters: {
@@ -135,6 +141,44 @@ export const createBrowserEvidence = (
});
};
/**
* The document one page observation is about: the URL the provider reported
* for it. Target lists and comparisons observe no single document.
*/
const browserSubject = (
operation: BrowserEvidenceOperation,
observed: Exclude<
BrowserEvidenceResult,
WebBundleAcquisitionPartialObservation
>,
): NonFileTargetIdentity => {
switch (operation) {
case "list_browser_targets":
case "compare_web_captures":
case "compare_web_screenshots":
return unknownTarget("not-observed");
case "analyze_web_bundle":
// A completed analysis reports its captured page; an acquisition that
// stopped before analysis reports the page inspection itself.
if ("capture" in observed && "target_url" in observed.capture)
return browserUrlTargetIdentity(observed.capture.target_url);
if ("target" in observed && "url" in observed.target)
return browserUrlTargetIdentity(observed.target.url);
break;
case "observe_web_session":
if ("target" in observed && "initial_url" in observed.target)
return browserUrlTargetIdentity(observed.target.initial_url);
break;
case "inspect_web_page":
case "discover_webmcp_tools":
case "capture_web_screenshot":
if ("target" in observed && "url" in observed.target)
return browserUrlTargetIdentity(observed.target.url);
break;
}
throw new Error(`Browser ${operation} result does not report its document`);
};
const browserParameters = (
input: BrowserEvidenceInput,
result: BrowserEvidenceResult,
+14 -10
View File
@@ -1,4 +1,4 @@
import { unknownTarget } from "../domain/targetIdentity.js";
import { browserUrlTargetIdentity } from "../domain/browserObservation.js";
import { z } from "zod";
import type { ExecutionOptions } from "../ports/AnalysisProvider.js";
import type { WebRuntimePort } from "../ports/WebRuntimePort.js";
@@ -156,15 +156,19 @@ export class WebRuntimeService {
),
);
return completeProviderEvidence(
createEvidence(unknownTarget("not-observed"), this.provider.identity(), {
predicateType: `rea.${operation.replaceAll("_", "-")}`,
operation,
parameters: input,
result,
confidence: "observed",
authority: "external-service",
limitations: result.limitations,
}),
createEvidence(
browserUrlTargetIdentity(result.target.initial_url),
this.provider.identity(),
{
predicateType: `rea.${operation.replaceAll("_", "-")}`,
operation,
parameters: input,
result,
confidence: "observed",
authority: "external-service",
limitations: result.limitations,
},
),
options?.signal,
);
}
+1 -1
View File
@@ -484,7 +484,7 @@ export const workflowSnapshotRecord = (
analysisProfile: analysisProfileSchema.parse(evidence.analysis_profile),
limitations: evidence.limitations,
locations: evidence.locations,
subject: evidenceSubjectTarget(evidence.subject) ?? null,
subject: evidence.subject,
},
};
};
@@ -28,6 +28,7 @@ import {
createEvidence,
evidenceFileTarget,
evidenceSubjectDisplay,
evidenceSubjectInput,
evidenceSubjectTarget,
rebindEvidenceToTargetPath,
type Evidence,
@@ -373,7 +374,12 @@ const createAnalysisViewEvidence = (
parent: Evidence,
result: AnalysisViewResult,
): Evidence => {
const target = evidenceSubjectTarget(parent.subject);
// A view names the target its parent observed (a file or a page URL); a
// parent that names no target is committed through the view's link.
const subject =
parent.subject.identity.kind === "unknown"
? unknownTarget("derived-from-linked-evidence")
: evidenceSubjectInput(parent.subject);
const nativeFunctionView =
result.kind === "native" ||
(result.kind === "summary" &&
@@ -387,7 +393,7 @@ const createAnalysisViewEvidence = (
(result.kind === "summary" &&
isNativeObservationOperation(result.parent_operation));
return createEvidence(
target ?? unknownTarget("derived-from-linked-evidence"),
subject,
ANALYSIS_VIEW_PROVIDER,
{
predicateType: "rea.analysis-view",
@@ -8,8 +8,13 @@ import {
ANALYSIS_SNAPSHOT_PROFILE,
ANALYSIS_SNAPSHOT_PROVIDER,
ANALYSIS_SNAPSHOT_TARGET,
snapshotExecution,
} from "../../domain/analysisSnapshot.fixture.js";
import { createEvidence, immutableEvidence } from "../../domain/evidence.js";
import {
createEvidence,
evidenceSubject,
immutableEvidence,
} from "../../domain/evidence.js";
import { createAnalysisExecutionEvidence } from "../../domain/analysisObservation.js";
import {
createEvidenceBundle,
@@ -37,6 +42,7 @@ import {
isSourceEvidenceRetainable,
} from "./AnalysisSnapshotCache.js";
import { EvidenceLedger } from "../investigation/EvidenceLedger.js";
import { nativeCallGraphSchema } from "../../domain/native/nativeCallGraph.js";
it.each(["provider", "workflow"] as const)(
@@ -73,7 +79,11 @@ it.each(["provider", "workflow"] as const)(
execution: { ...execution, analysisProfile },
};
if (kind === "provider") cache.record(input);
else cache.recordWorkflow(input);
else
cache.recordWorkflow({
...input,
execution: snapshotExecution(input.execution),
});
const entries = () =>
kind === "provider" ? cache.entries() : cache.workflowEntries();
const expected = entries();
@@ -123,10 +133,12 @@ it("exports independent nested query, workflow and Evidence payloads", () => {
operation: "analyze_function",
parameters,
execution: {
...createAnalysisExecution(result, REA_WORKFLOW_PROVIDER, {
subject: target,
rawResult: result,
}),
...snapshotExecution(
createAnalysisExecution(result, REA_WORKFLOW_PROVIDER, {
subject: target,
rawResult: result,
}),
),
analysisProfile: workflowProfile,
},
});
@@ -226,7 +238,10 @@ it.each(["provider", "workflow"] as const)(
profile,
operation,
parameters,
execution: { ...execution, analysisProfile: evidenceProfile },
execution: snapshotExecution({
...execution,
analysisProfile: evidenceProfile,
}),
});
expect(ledger.record(evidence).ok).toBe(true);
const before = cache.export(target, profile, ledger.export());
@@ -285,14 +300,14 @@ it.each(["provider", "workflow"] as const)(
if (!after.ok) throw after.error;
const entries =
kind === "provider" ? after.value.entries : after.value.workflow_entries;
expect(entries[0]?.execution.subject?.path).toBe("/relocated/fixture");
expect(entries[0]?.execution.subject.local_path).toBe("/relocated/fixture");
expect(entries[0]?.execution.result).toBe("original");
expect(after.value.evidence_bundle.records).toContainEqual(relocated);
const originalEntries =
kind === "provider"
? before.value.entries
: before.value.workflow_entries;
expect(originalEntries[0]?.execution.subject?.path).toBe(target.path);
expect(originalEntries[0]?.execution.subject.local_path).toBe(target.path);
},
);
@@ -327,7 +342,11 @@ it.each(["provider", "workflow"] as const)(
};
if (kind === "provider")
cache.record({ ...input, operation: "procedure_info" });
else cache.recordWorkflow(input);
else
cache.recordWorkflow({
...input,
execution: snapshotExecution(input.execution),
});
expect(
ledger.record(
createEvidence(subject ?? unknownTarget("not-observed"), provider, {
@@ -356,7 +375,9 @@ it.each(["provider", "workflow"] as const)(
const saved =
kind === "provider" ? snapshot.entries : snapshot.workflow_entries;
expect(saved).toHaveLength(1);
expect(saved[0]?.execution.subject?.sha256).toBe(target.sha256);
expect(saved[0]?.execution.subject.identity).toMatchObject({
sha256: target.sha256,
});
expect(snapshot.evidence_bundle.records).toHaveLength(1);
expect(
kind === "provider" ? cache.entries() : cache.workflowEntries(),
@@ -402,9 +423,11 @@ it("counts newly retained direct and workflow bindings when staging", () => {
operation: "trace_feature",
parameters: { query: "feature" },
execution: {
...createAnalysisExecution("workflow", REA_WORKFLOW_PROVIDER, {
subject: target,
}),
...snapshotExecution(
createAnalysisExecution("workflow", REA_WORKFLOW_PROVIDER, {
subject: target,
}),
),
analysisProfile: workflowProfile,
},
});
@@ -532,11 +555,13 @@ it("binds an omitted subject to the selected target in snapshot replay", () => {
]),
);
if (!snapshot.ok) throw snapshot.error;
expect(snapshot.value.entries[0]?.execution.subject?.path).toBe(target.path);
expect(snapshot.value.entries[0]?.execution.subject.local_path).toBe(
target.path,
);
const reparsed = parseAnalysisSnapshot(
JSON.parse(serializeAnalysisSnapshot(snapshot.value)),
);
expect(reparsed.entries[0]?.execution.subject?.path).toBe(target.path);
expect(reparsed.entries[0]?.execution.subject.local_path).toBe(target.path);
cache.clear();
const imported = cache.import(
reparsed,
@@ -583,9 +608,11 @@ describe("analysis snapshot cache capacity", () => {
const target = ANALYSIS_SNAPSHOT_TARGET;
const profile = ANALYSIS_SNAPSHOT_PROFILE;
const execution = {
...createAnalysisExecution("workflow", REA_WORKFLOW_PROVIDER, {
subject: target,
}),
...snapshotExecution(
createAnalysisExecution("workflow", REA_WORKFLOW_PROVIDER, {
subject: target,
}),
),
analysisProfile: workflowAnalysisProfile(profile),
};
const workflowInput = {
@@ -673,7 +700,7 @@ describe("analysis snapshot cache capacity", () => {
cache.recordWorkflow({
...workflowInput,
target: otherTarget,
execution: { ...execution, subject: otherTarget },
execution: { ...execution, subject: evidenceSubject(otherTarget) },
});
expect(cache.entries()).toEqual([]);
expect(cache.workflowEntries()).toHaveLength(1);
@@ -1233,10 +1260,12 @@ it.each([
operation,
parameters,
execution: {
...createAnalysisExecution(oldResult, REA_WORKFLOW_PROVIDER, {
subject: target,
rawResult: oldResult,
}),
...snapshotExecution(
createAnalysisExecution(oldResult, REA_WORKFLOW_PROVIDER, {
subject: target,
rawResult: oldResult,
}),
),
analysisProfile: oldProfile,
},
});
@@ -1316,9 +1345,11 @@ it.each([1, 2])(
limitations: [],
};
const execution = {
...createAnalysisExecution(oldResult, REA_WORKFLOW_PROVIDER, {
subject: target,
}),
...snapshotExecution(
createAnalysisExecution(oldResult, REA_WORKFLOW_PROVIDER, {
subject: target,
}),
),
analysisProfile: oldProfile,
};
cache.recordWorkflow({ target, profile, operation, parameters, execution });
+38 -53
View File
@@ -10,10 +10,17 @@ import {
import { NoBinaryOpenError } from "../../domain/configurationErrors.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
evidenceSubject,
evidenceSubjectDisplay,
evidenceSubjectTarget,
rebindEvidenceSubjectTargetPath,
type Evidence,
evidenceSubjectDisplay,
type EvidenceSubjectTarget,
} from "../../domain/evidence.js";
import {
sameTargetIdentity,
unknownTarget,
} from "../../domain/targetIdentity.js";
import { err, ok, type Result } from "../../domain/result.js";
import {
analysisQueryId,
@@ -25,7 +32,6 @@ import {
snapshotBindingMismatch,
snapshotMatchesProfile,
snapshotTarget,
snapshotTargetIdentity,
parseAnalysisSnapshot,
immutableAnalysisSnapshot,
type AnalysisSnapshot,
@@ -49,10 +55,6 @@ import {
snapshotReplayBinding,
type SnapshotReplayBinding,
} from "./AnalysisSnapshotReplayBinding.js";
import {
binaryTargetSelection,
sameTargetIdentity,
} from "../../domain/targetIdentity.js";
import { createInvestigationEvidenceClosureChecker } from "../investigation/InvestigationEvidence.js";
const RELATIONSHIP_OPERATIONS: ReadonlySet<AnalysisOperation> = new Set([
@@ -96,6 +98,12 @@ const DOCUMENT_SCOPED_OPERATIONS: ReadonlySet<string> = new Set(
/** Maximum duplicated result bindings retained across direct and workflow queries. */
const SNAPSHOT_CACHE_ENTRY_CEILING = 10_000;
/** A file target without its recorded display name, which follows the replayed path. */
const withoutDisplayName = ({
name: _name,
...target
}: EvidenceSubjectTarget): EvidenceSubjectTarget => target;
const replayDocumentSelected = (
operation: AnalysisOperation,
parameters: Readonly<Record<string, JsonValue>>,
@@ -232,10 +240,7 @@ export class AnalysisSnapshotCache {
snapshot = immutableAnalysisSnapshot(snapshot);
if (
(this.#target !== undefined &&
!sameTargetIdentity(
snapshotTargetIdentity(this.#target),
snapshotTargetIdentity(snapshot.target),
)) ||
!sameTargetIdentity(this.#target.identity, snapshot.target.identity)) ||
(this.#binding !== undefined &&
!snapshotMatchesProfile(
this.#binding,
@@ -358,35 +363,31 @@ export class AnalysisSnapshotCache {
previous.parameters,
);
const entry = this.#entries.get(queryId);
const entryIdentity = entry?.execution.subject.identity;
if (
entry !== undefined &&
entry.execution.subject !== null &&
entryIdentity?.kind === "file" &&
evidenceMatchesEntry(previous, entry, header)
)
this.#entries.set(queryId, {
...entry,
execution: {
...entry.execution,
subject: {
...entry.execution.subject,
path: display.local_path,
},
subject: { identity: entryIdentity, ...display },
},
});
const workflow = this.#workflowEntries.get(queryId);
const workflowIdentity = workflow?.execution.subject.identity;
if (
workflow !== undefined &&
workflow.execution.subject !== null &&
workflowIdentity?.kind === "file" &&
evidenceMatchesWorkflowEntry(previous, workflow, header)
)
this.#workflowEntries.set(queryId, {
...workflow,
execution: {
...workflow.execution,
subject: {
...workflow.execution.subject,
path: display.local_path,
},
subject: { identity: workflowIdentity, ...display },
},
});
}
@@ -401,18 +402,21 @@ export class AnalysisSnapshotCache {
if (target === undefined || profile === undefined)
return err(new NoBinaryOpenError());
try {
const snapshotTargetIdentity = snapshotTarget(target);
const persistedTarget = snapshotTarget(target);
const namesTarget = (subject: Evidence["subject"]): boolean =>
subject.identity.kind === "file" &&
subject.identity.sha256 === target.sha256;
const binding = snapshotBinding(profile);
// In-memory replay can preserve unbound and child-artifact observations.
// A persisted target snapshot can authenticate only rows whose Evidence
// has that target subject; the record owner applies the same scope to its bundle.
const entries = [...this.#entries.values()]
.filter((entry) => entry.execution.subject?.sha256 === target.sha256)
.filter((entry) => namesTarget(entry.execution.subject))
.sort((left, right) =>
compareUnicodeCodePoints(left.query_id, right.query_id),
);
const workflows = [...this.#workflowEntries.values()]
.filter((entry) => entry.execution.subject?.sha256 === target.sha256)
.filter((entry) => namesTarget(entry.execution.subject))
.sort((left, right) =>
compareUnicodeCodePoints(left.query_id, right.query_id),
);
@@ -420,7 +424,7 @@ export class AnalysisSnapshotCache {
// payloads here only adds another full materialization.
return ok(
parseAnalysisSnapshot({
target: snapshotTargetIdentity,
target: persistedTarget,
binding,
entries,
workflow_entries: workflows,
@@ -507,7 +511,7 @@ export class AnalysisSnapshotCache {
readonly limitations: readonly string[];
readonly relationshipCoverage?: AnalysisExecution["relationshipCoverage"];
readonly locations: AnalysisExecution["locations"];
readonly subject: NonNullable<AnalysisExecution["subject"]> | null;
readonly subject: Evidence["subject"];
};
}): void {
this.select(input.target, input.profile);
@@ -556,27 +560,14 @@ export class AnalysisSnapshotCache {
this.#replayBindings.set(queryId, restored);
binding = restored;
}
const cachedSubject = cached.execution.subject;
// The query ID commits to the snapshot target, including its selection,
// so a cached subject naming the target's bytes made the same selection.
// Snapshot subjects do not persist it yet (ADR target-identity slice 2d).
const selection =
cachedSubject?.sha256 === target.sha256
? binaryTargetSelection(target)
: null;
// Replay names the file at the current path; its display name follows
// that path, as the live observation's would.
const cachedTarget = evidenceSubjectTarget(cached.execution.subject);
const subject =
cachedSubject == null
cachedTarget === undefined
? null
: rebindEvidenceSubjectTargetPath(
{
path: cachedSubject.path,
sha256: cachedSubject.sha256,
format: cachedSubject.format,
...(cachedSubject.architecture === null
? {}
: { architecture: cachedSubject.architecture }),
...(selection === null ? {} : { selection }),
},
withoutDisplayName(cachedTarget),
target,
);
const execution: AnalysisExecution = structuredClone({
@@ -656,17 +647,11 @@ export class AnalysisSnapshotCache {
parameters,
execution: {
...execution,
subject:
subject: evidenceSubject(
execution.subject === undefined
? {
path: target.path,
sha256: target.sha256,
format: target.format,
...(target.architecture === undefined
? {}
: { architecture: target.architecture }),
}
: execution.subject,
? target
: (execution.subject ?? unknownTarget("not-observed")),
),
},
});
this.#entries.set(entry.query_id, entry);
@@ -172,7 +172,10 @@ describe("target format vocabularies", () => {
analysisSnapshotSchema.shape.target.parse(
roundTrip(snapshotTarget(target)),
),
).toMatchObject({ format: target.format, kind: target.kind });
).toMatchObject({
kind: target.kind,
identity: { format: target.format },
});
expect(binaryTargetSummary(target)).toMatchObject({
format: target.format,
kind: target.kind,