feat(snapshots)!: persist the target identity as the snapshot target (#11551)

* feat(snapshots)!: persist the target identity as the snapshot target

The snapshot target is now `{ kind, identity }`, where `identity` is the
FileTargetIdentity that `binaryTargetIdentity` produces and `kind` is the
binary kind query IDs commit to. The per-format target projection and
`snapshotTargetIdentity` are deleted; the schema checks the format's
selection, architecture and kind instead of keeping one union arm per format.

Entry execution subjects use the Evidence subject shape (file or explicit
unknown), so a snapshot entry commits to the same identity, including the
selection, as its Evidence record. Workflow entries take the subject of the
workflow's own Evidence. Replay no longer re-derives the selection from the
open target.

`FileTargetSelection` gains `universal-slice` (`offset`, `size`, and the FAT
`cpu_type`/`cpu_subtype` when target admission supplied the FAT table, else
null). `binaryTargetIdentity` produces it for `mach-o-universal` targets, so
Evidence and snapshots for different slices of one container no longer share
an identity.

Refs #11417

BREAKING CHANGE: analysis snapshot files change shape: `target` is
`{ kind, identity: { kind: "file", sha256, format, architecture, selection } }`
and entry `execution.subject` is the Evidence subject
(`{ identity, name, local_path }`, never null). Snapshot binding mismatch
details report `target.snapshot`/`target.selected` in the same shape. Evidence
IDs of universal Mach-O targets change because their subject identity now
carries the selected slice. Recreate saved snapshots and Evidence bundles of
universal Mach-O targets.

* docs(evidence): name the universal Mach-O slice selection in the subject contract

* fix(snapshots): read the snapshot target identity in the Ghidra verification scripts

The real Ghidra snapshot checks (lifecycle, RISC-V, MIPS, PowerPC) read
snapshot.target.sha256 and .architecture, which moved under
snapshot.target.identity, so they would fail after saving a valid snapshot.

* feat(evidence)!: name live browser page Evidence by its page URL (#11559)

Browser page observations (inspect_web_page, analyze_web_bundle,
observe_web_session, discover_webmcp_tools, capture_web_screenshot,
observe_web_execution, inspect_web_event_listeners) named an unknown
(`not-observed`) subject although the provider reports the page URL.

`TargetIdentity` gains a `url` kind: the reported URL with userinfo removed
and its origin (`null` for an opaque origin). The schema refuses a URL that
carries userinfo or an origin the URL does not have.
`browserUrlTargetIdentity` in the browser URL owner builds it from the
provider's reported URL; a withheld or unparseable URL stays unknown
(`not-observed`). Target lists and comparisons still name no single
document. Analysis views keep their parent's URL subject, as they already
keep a file subject.

BREAKING CHANGE: Evidence IDs of live browser page observations and their
analysis views change, and their subject is
`{ "kind": "url", "url", "origin" }` with `name` and `local_path` null.
Recreate stored Evidence for these tools. This lands inside Evidence format
revision 2 under the release hold (#11417); no `evidence_format` bump.
This commit is contained in:
morluto authored and GitHub committed 2026-11-16 07:04:34 +08:00
1 parent f6df353002
commit ff0176256a
30 files changed
+678 -396

No files matched your search

@@ -8,8 +8,13 @@ import {
ANALYSIS_SNAPSHOT_PROFILE,
ANALYSIS_SNAPSHOT_PROVIDER,
ANALYSIS_SNAPSHOT_TARGET,
snapshotExecution,
} from "../../domain/analysisSnapshot.fixture.js";
import { createEvidence, immutableEvidence } from "../../domain/evidence.js";
import {
createEvidence,
evidenceSubject,
immutableEvidence,
} from "../../domain/evidence.js";
import { createAnalysisExecutionEvidence } from "../../domain/analysisObservation.js";
import {
createEvidenceBundle,
@@ -37,6 +42,7 @@ import {
isSourceEvidenceRetainable,
} from "./AnalysisSnapshotCache.js";
import { EvidenceLedger } from "../investigation/EvidenceLedger.js";
import { nativeCallGraphSchema } from "../../domain/native/nativeCallGraph.js";
it.each(["provider", "workflow"] as const)(
@@ -73,7 +79,11 @@ it.each(["provider", "workflow"] as const)(
execution: { ...execution, analysisProfile },
};
if (kind === "provider") cache.record(input);
else cache.recordWorkflow(input);
else
cache.recordWorkflow({
...input,
execution: snapshotExecution(input.execution),
});
const entries = () =>
kind === "provider" ? cache.entries() : cache.workflowEntries();
const expected = entries();
@@ -123,10 +133,12 @@ it("exports independent nested query, workflow and Evidence payloads", () => {
operation: "analyze_function",
parameters,
execution: {
...createAnalysisExecution(result, REA_WORKFLOW_PROVIDER, {
subject: target,
rawResult: result,
}),
...snapshotExecution(
createAnalysisExecution(result, REA_WORKFLOW_PROVIDER, {
subject: target,
rawResult: result,
}),
),
analysisProfile: workflowProfile,
},
});
@@ -226,7 +238,10 @@ it.each(["provider", "workflow"] as const)(
profile,
operation,
parameters,
execution: { ...execution, analysisProfile: evidenceProfile },
execution: snapshotExecution({
...execution,
analysisProfile: evidenceProfile,
}),
});
expect(ledger.record(evidence).ok).toBe(true);
const before = cache.export(target, profile, ledger.export());
@@ -285,14 +300,14 @@ it.each(["provider", "workflow"] as const)(
if (!after.ok) throw after.error;
const entries =
kind === "provider" ? after.value.entries : after.value.workflow_entries;
expect(entries[0]?.execution.subject?.path).toBe("/relocated/fixture");
expect(entries[0]?.execution.subject.local_path).toBe("/relocated/fixture");
expect(entries[0]?.execution.result).toBe("original");
expect(after.value.evidence_bundle.records).toContainEqual(relocated);
const originalEntries =
kind === "provider"
? before.value.entries
: before.value.workflow_entries;
expect(originalEntries[0]?.execution.subject?.path).toBe(target.path);
expect(originalEntries[0]?.execution.subject.local_path).toBe(target.path);
},
);
@@ -327,7 +342,11 @@ it.each(["provider", "workflow"] as const)(
};
if (kind === "provider")
cache.record({ ...input, operation: "procedure_info" });
else cache.recordWorkflow(input);
else
cache.recordWorkflow({
...input,
execution: snapshotExecution(input.execution),
});
expect(
ledger.record(
createEvidence(subject ?? unknownTarget("not-observed"), provider, {
@@ -356,7 +375,9 @@ it.each(["provider", "workflow"] as const)(
const saved =
kind === "provider" ? snapshot.entries : snapshot.workflow_entries;
expect(saved).toHaveLength(1);
expect(saved[0]?.execution.subject?.sha256).toBe(target.sha256);
expect(saved[0]?.execution.subject.identity).toMatchObject({
sha256: target.sha256,
});
expect(snapshot.evidence_bundle.records).toHaveLength(1);
expect(
kind === "provider" ? cache.entries() : cache.workflowEntries(),
@@ -402,9 +423,11 @@ it("counts newly retained direct and workflow bindings when staging", () => {
operation: "trace_feature",
parameters: { query: "feature" },
execution: {
...createAnalysisExecution("workflow", REA_WORKFLOW_PROVIDER, {
subject: target,
}),
...snapshotExecution(
createAnalysisExecution("workflow", REA_WORKFLOW_PROVIDER, {
subject: target,
}),
),
analysisProfile: workflowProfile,
},
});
@@ -532,11 +555,13 @@ it("binds an omitted subject to the selected target in snapshot replay", () => {
]),
);
if (!snapshot.ok) throw snapshot.error;
expect(snapshot.value.entries[0]?.execution.subject?.path).toBe(target.path);
expect(snapshot.value.entries[0]?.execution.subject.local_path).toBe(
target.path,
);
const reparsed = parseAnalysisSnapshot(
JSON.parse(serializeAnalysisSnapshot(snapshot.value)),
);
expect(reparsed.entries[0]?.execution.subject?.path).toBe(target.path);
expect(reparsed.entries[0]?.execution.subject.local_path).toBe(target.path);
cache.clear();
const imported = cache.import(
reparsed,
@@ -583,9 +608,11 @@ describe("analysis snapshot cache capacity", () => {
const target = ANALYSIS_SNAPSHOT_TARGET;
const profile = ANALYSIS_SNAPSHOT_PROFILE;
const execution = {
...createAnalysisExecution("workflow", REA_WORKFLOW_PROVIDER, {
subject: target,
}),
...snapshotExecution(
createAnalysisExecution("workflow", REA_WORKFLOW_PROVIDER, {
subject: target,
}),
),
analysisProfile: workflowAnalysisProfile(profile),
};
const workflowInput = {
@@ -673,7 +700,7 @@ describe("analysis snapshot cache capacity", () => {
cache.recordWorkflow({
...workflowInput,
target: otherTarget,
execution: { ...execution, subject: otherTarget },
execution: { ...execution, subject: evidenceSubject(otherTarget) },
});
expect(cache.entries()).toEqual([]);
expect(cache.workflowEntries()).toHaveLength(1);
@@ -1233,10 +1260,12 @@ it.each([
operation,
parameters,
execution: {
...createAnalysisExecution(oldResult, REA_WORKFLOW_PROVIDER, {
subject: target,
rawResult: oldResult,
}),
...snapshotExecution(
createAnalysisExecution(oldResult, REA_WORKFLOW_PROVIDER, {
subject: target,
rawResult: oldResult,
}),
),
analysisProfile: oldProfile,
},
});
@@ -1316,9 +1345,11 @@ it.each([1, 2])(
limitations: [],
};
const execution = {
...createAnalysisExecution(oldResult, REA_WORKFLOW_PROVIDER, {
subject: target,
}),
...snapshotExecution(
createAnalysisExecution(oldResult, REA_WORKFLOW_PROVIDER, {
subject: target,
}),
),
analysisProfile: oldProfile,
};
cache.recordWorkflow({ target, profile, operation, parameters, execution });
+38 -53
View File
@@ -10,10 +10,17 @@ import {
import { NoBinaryOpenError } from "../../domain/configurationErrors.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
evidenceSubject,
evidenceSubjectDisplay,
evidenceSubjectTarget,
rebindEvidenceSubjectTargetPath,
type Evidence,
evidenceSubjectDisplay,
type EvidenceSubjectTarget,
} from "../../domain/evidence.js";
import {
sameTargetIdentity,
unknownTarget,
} from "../../domain/targetIdentity.js";
import { err, ok, type Result } from "../../domain/result.js";
import {
analysisQueryId,
@@ -25,7 +32,6 @@ import {
snapshotBindingMismatch,
snapshotMatchesProfile,
snapshotTarget,
snapshotTargetIdentity,
parseAnalysisSnapshot,
immutableAnalysisSnapshot,
type AnalysisSnapshot,
@@ -49,10 +55,6 @@ import {
snapshotReplayBinding,
type SnapshotReplayBinding,
} from "./AnalysisSnapshotReplayBinding.js";
import {
binaryTargetSelection,
sameTargetIdentity,
} from "../../domain/targetIdentity.js";
import { createInvestigationEvidenceClosureChecker } from "../investigation/InvestigationEvidence.js";
const RELATIONSHIP_OPERATIONS: ReadonlySet<AnalysisOperation> = new Set([
@@ -96,6 +98,12 @@ const DOCUMENT_SCOPED_OPERATIONS: ReadonlySet<string> = new Set(
/** Maximum duplicated result bindings retained across direct and workflow queries. */
const SNAPSHOT_CACHE_ENTRY_CEILING = 10_000;
/** A file target without its recorded display name, which follows the replayed path. */
const withoutDisplayName = ({
name: _name,
...target
}: EvidenceSubjectTarget): EvidenceSubjectTarget => target;
const replayDocumentSelected = (
operation: AnalysisOperation,
parameters: Readonly<Record<string, JsonValue>>,
@@ -232,10 +240,7 @@ export class AnalysisSnapshotCache {
snapshot = immutableAnalysisSnapshot(snapshot);
if (
(this.#target !== undefined &&
!sameTargetIdentity(
snapshotTargetIdentity(this.#target),
snapshotTargetIdentity(snapshot.target),
)) ||
!sameTargetIdentity(this.#target.identity, snapshot.target.identity)) ||
(this.#binding !== undefined &&
!snapshotMatchesProfile(
this.#binding,
@@ -358,35 +363,31 @@ export class AnalysisSnapshotCache {
previous.parameters,
);
const entry = this.#entries.get(queryId);
const entryIdentity = entry?.execution.subject.identity;
if (
entry !== undefined &&
entry.execution.subject !== null &&
entryIdentity?.kind === "file" &&
evidenceMatchesEntry(previous, entry, header)
)
this.#entries.set(queryId, {
...entry,
execution: {
...entry.execution,
subject: {
...entry.execution.subject,
path: display.local_path,
},
subject: { identity: entryIdentity, ...display },
},
});
const workflow = this.#workflowEntries.get(queryId);
const workflowIdentity = workflow?.execution.subject.identity;
if (
workflow !== undefined &&
workflow.execution.subject !== null &&
workflowIdentity?.kind === "file" &&
evidenceMatchesWorkflowEntry(previous, workflow, header)
)
this.#workflowEntries.set(queryId, {
...workflow,
execution: {
...workflow.execution,
subject: {
...workflow.execution.subject,
path: display.local_path,
},
subject: { identity: workflowIdentity, ...display },
},
});
}
@@ -401,18 +402,21 @@ export class AnalysisSnapshotCache {
if (target === undefined || profile === undefined)
return err(new NoBinaryOpenError());
try {
const snapshotTargetIdentity = snapshotTarget(target);
const persistedTarget = snapshotTarget(target);
const namesTarget = (subject: Evidence["subject"]): boolean =>
subject.identity.kind === "file" &&
subject.identity.sha256 === target.sha256;
const binding = snapshotBinding(profile);
// In-memory replay can preserve unbound and child-artifact observations.
// A persisted target snapshot can authenticate only rows whose Evidence
// has that target subject; the record owner applies the same scope to its bundle.
const entries = [...this.#entries.values()]
.filter((entry) => entry.execution.subject?.sha256 === target.sha256)
.filter((entry) => namesTarget(entry.execution.subject))
.sort((left, right) =>
compareUnicodeCodePoints(left.query_id, right.query_id),
);
const workflows = [...this.#workflowEntries.values()]
.filter((entry) => entry.execution.subject?.sha256 === target.sha256)
.filter((entry) => namesTarget(entry.execution.subject))
.sort((left, right) =>
compareUnicodeCodePoints(left.query_id, right.query_id),
);
@@ -420,7 +424,7 @@ export class AnalysisSnapshotCache {
// payloads here only adds another full materialization.
return ok(
parseAnalysisSnapshot({
target: snapshotTargetIdentity,
target: persistedTarget,
binding,
entries,
workflow_entries: workflows,
@@ -507,7 +511,7 @@ export class AnalysisSnapshotCache {
readonly limitations: readonly string[];
readonly relationshipCoverage?: AnalysisExecution["relationshipCoverage"];
readonly locations: AnalysisExecution["locations"];
readonly subject: NonNullable<AnalysisExecution["subject"]> | null;
readonly subject: Evidence["subject"];
};
}): void {
this.select(input.target, input.profile);
@@ -556,27 +560,14 @@ export class AnalysisSnapshotCache {
this.#replayBindings.set(queryId, restored);
binding = restored;
}
const cachedSubject = cached.execution.subject;
// The query ID commits to the snapshot target, including its selection,
// so a cached subject naming the target's bytes made the same selection.
// Snapshot subjects do not persist it yet (ADR target-identity slice 2d).
const selection =
cachedSubject?.sha256 === target.sha256
? binaryTargetSelection(target)
: null;
// Replay names the file at the current path; its display name follows
// that path, as the live observation's would.
const cachedTarget = evidenceSubjectTarget(cached.execution.subject);
const subject =
cachedSubject == null
cachedTarget === undefined
? null
: rebindEvidenceSubjectTargetPath(
{
path: cachedSubject.path,
sha256: cachedSubject.sha256,
format: cachedSubject.format,
...(cachedSubject.architecture === null
? {}
: { architecture: cachedSubject.architecture }),
...(selection === null ? {} : { selection }),
},
withoutDisplayName(cachedTarget),
target,
);
const execution: AnalysisExecution = structuredClone({
@@ -656,17 +647,11 @@ export class AnalysisSnapshotCache {
parameters,
execution: {
...execution,
subject:
subject: evidenceSubject(
execution.subject === undefined
? {
path: target.path,
sha256: target.sha256,
format: target.format,
...(target.architecture === undefined
? {}
: { architecture: target.architecture }),
}
: execution.subject,
? target
: (execution.subject ?? unknownTarget("not-observed")),
),
},
});
this.#entries.set(entry.query_id, entry);
@@ -172,7 +172,10 @@ describe("target format vocabularies", () => {
analysisSnapshotSchema.shape.target.parse(
roundTrip(snapshotTarget(target)),
),
).toMatchObject({ format: target.format, kind: target.kind });
).toMatchObject({
kind: target.kind,
identity: { format: target.format },
});
expect(binaryTargetSummary(target)).toMatchObject({
format: target.format,
kind: target.kind,