fix(browser): preserve capture uncertainty and frame ancestry

Parse required resource rows and retain valid siblings. Missing collections and malformed child subtrees keep unknown omission counts. Compare complete frame forests without capture-local IDs, and never infer aggregate DOM/metadata changes from omissions.

Co-authored-by: ShuaixinHuang <1836656514@qq.com>
This commit is contained in:
morlutoandShuaixinHuang committed 2026-10-14 15:20:08 +08:00
1 parent eb00112a3f
commit 27311f88d4
8 files changed
+878 -46

No files matched your search

+23
View File
@@ -20,6 +20,17 @@ Inspect native SAZ or HAR 1.2 offline with REA, or connect Everywhere's separate
- `observe_web_session` / `rea observe-web-session` arms an observation window of the requested duration for an external user action and records ordered reload, SPA navigation, redirect, failure, lifecycle, and target-termination metadata.
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. Declared `readOnly`, `untrustedContent`, `consequential`, and `autosubmit` annotation hints are reported per registration: `true`/`false` are declared values, `null` means the declaration did not provide the hint in that capture, and a missing `consequential` key means the hint was not recorded, either in a historical capture or because the observed annotation was invalid. Consult discovery completeness and exclusions to distinguish these cases. Malformed annotation values retain the registration with incomplete coverage and an `invalid_protocol_value` exclusion. Other malformed protocol payloads can fail discovery explicitly. REA never exposes `WebMCP.invokeTool`.
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
Passive DOM comparison checks complete node inventories and complete frame
parent/child structure independently. Frame IDs and sibling enumeration order
are capture-local and do not establish a change; duplicate IDs, missing parents,
and cyclic ancestry remain unknown. A complete frame difference can still prove
change when DOM nodes are incomplete, and vice versa. Partial DOM or metadata
collection digests do not prove modification: a missing node or link can result
from a protocol omission. Rejected DOM facts also mark derived URL and agent-hint
metadata incomplete, while valid facts from other nodes and documents remain
available in the capture.
- `capture_web_screenshot` / `rea capture-web-screenshot` returns a content-addressed visible-viewport PNG.
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
- The CLI comparison commands accept each capture, screenshot artifact or normalization policy as inline JSON or a JSON file path. Use files for screenshot artifacts and scenario captures, whose embedded PNG bytes can exceed the host's command-line length limit.
@@ -294,6 +305,18 @@ validation and resource bounds. Omitted script metadata makes its affected
section explicitly truncated.
Disallowed-origin entries are filtered before retention.
Resource-tree inventories distinguish a valid empty `resources` array from a
missing or malformed collection. Invalid resource entries are omitted with
incomplete coverage, while valid resources from that frame and other frames
remain available. Comparing an incomplete resource inventory cannot prove
addition, removal, or equality; a complete empty inventory can prove removal.
Resource records require a nonempty type and a string MIME type; absent or
malformed required fields are recorded as omissions instead of invented defaults.
An explicitly empty MIME type is valid. The optional content size may be absent;
an invalid supplied size remains unknown with incomplete resource coverage while
the resource's other facts are retained. Zero and nonnegative fractional sizes
are preserved.
CDP discovery, connection, and commands wait for the peer or caller cancellation;
a stalled peer can leave an operation waiting. WebSocket messages remain subject
to the [64 MiB limit](#request-boundary). REA does not impose a message-count or
+10 -2
View File
@@ -268,7 +268,9 @@ describe("CDP frame capture", () => {
it("walks deeply nested frame trees and returns every frame and resource", () => {
let frameTree: Record<string, unknown> = {
frame: { id: "frame-0", url: `${origin}/0` },
resources: [{ url: `${origin}/0.js`, type: "Script" }],
resources: [
{ url: `${origin}/0.js`, type: "Script", mimeType: "text/javascript" },
],
};
for (let index = 1; index < 2_000; index += 1) {
frameTree = {
@@ -276,7 +278,13 @@ describe("CDP frame capture", () => {
id: `frame-${String(index)}`,
url: `${origin}/${String(index)}`,
},
resources: [{ url: `${origin}/${String(index)}.js`, type: "Script" }],
resources: [
{
url: `${origin}/${String(index)}.js`,
type: "Script",
mimeType: "text/javascript",
},
],
childFrames: [frameTree],
};
}
+82 -23
View File
@@ -14,6 +14,7 @@ import {
cdpSnapshotString,
cdpSnapshotValueString,
} from "./CdpSnapshotStrings.js";
import type { BrowserExclusionReason } from "../domain/browserCompleteness.js";
import { exclusionReasonForUrl } from "./CdpCaptureEventHelpers.js";
import type { CdpCaptureCompleteness } from "./CdpCaptureCompleteness.js";
import {
@@ -43,8 +44,8 @@ export const captureFrames = (
return { items: [] };
}
const items: WebPageInspection["frames"] = [];
for (const tree of walkFrameTrees(root, () =>
completeness?.exclude("frames", "invalid_protocol_value"),
for (const tree of walkFrameTrees(root, (omittedCount) =>
completeness?.exclude("frames", "invalid_protocol_value", omittedCount),
)) {
const frame = recordValue(tree.frame);
const frameId = cdpStringValue(frame?.id);
@@ -61,9 +62,16 @@ export const captureFrames = (
continue;
}
if (maximum !== undefined && items.length >= maximum) continue;
const parentFrameId = cdpStringValue(frame.parentId);
// Only the enclosing tree's root may legitimately omit parent evidence.
if (
(tree !== root || Object.hasOwn(frame, "parentId")) &&
(parentFrameId === undefined || parentFrameId.length === 0)
)
completeness?.exclude("frames", "invalid_protocol_value");
items.push({
frame_id: frameId,
parent_frame_id: cdpStringValue(frame.parentId) ?? null,
parent_frame_id: parentFrameId ?? null,
url: sanitized.url,
origin: sanitized.origin,
});
@@ -91,10 +99,19 @@ export const captureResources = (
return { items: [] };
}
const items: CapturedResource[] = [];
for (const tree of walkFrameTrees(root, () =>
completeness?.exclude("resources", "invalid_protocol_value"),
for (const tree of walkFrameTrees(root, (omittedCount) =>
completeness?.exclude("resources", "invalid_protocol_value", omittedCount),
)) {
for (const resource of recordsValue(tree.resources)) {
if (!Array.isArray(tree.resources)) {
completeness?.exclude("resources", "invalid_protocol_value", null);
continue;
}
for (const entry of tree.resources) {
const resource = recordValue(entry);
if (resource === undefined) {
completeness?.exclude("resources", "invalid_protocol_value");
continue;
}
const url = allowedSanitizedUrl(resource.url, allowedOrigins);
if (url === undefined || url.origin === null) {
completeness?.exclude(
@@ -103,15 +120,26 @@ export const captureResources = (
);
continue;
}
const contentSize = numberValue(resource.contentSize);
const type = cdpStringValue(resource.type);
const mimeType = cdpStringValue(resource.mimeType);
if (type === undefined || type.length === 0 || mimeType === undefined) {
completeness?.exclude("resources", "invalid_protocol_value");
continue;
}
const rawContentSize = numberValue(resource.contentSize);
const contentSize =
rawContentSize !== undefined && rawContentSize >= 0
? rawContentSize
: null;
if (resource.contentSize !== undefined && contentSize === null)
completeness?.exclude("resources", "invalid_protocol_value");
items.push({
rawUrl: cdpStringValue(resource.url) ?? "",
url: url.url,
origin: url.origin,
type: cdpStringValue(resource.type) ?? "Other",
mime_type: cdpStringValue(resource.mimeType) ?? "",
content_size:
contentSize === undefined ? null : Math.max(0, contentSize),
type,
mime_type: mimeType,
content_size: contentSize,
});
}
}
@@ -120,7 +148,7 @@ export const captureResources = (
export const walkFrameTrees = function* (
root: UnknownRecord,
onMalformedChild?: () => void,
onMalformedChild?: (omittedCount: number | null) => void,
): Generator<UnknownRecord> {
const pending = [root];
while (pending.length > 0) {
@@ -128,11 +156,11 @@ export const walkFrameTrees = function* (
if (tree === undefined) return;
const rawChildren = tree.childFrames;
if (rawChildren !== undefined && !Array.isArray(rawChildren))
onMalformedChild?.();
onMalformedChild?.(null);
const children = Array.isArray(rawChildren)
? rawChildren.flatMap((child) => {
const parsed = recordValue(child);
if (parsed === undefined) onMalformedChild?.();
if (parsed === undefined) onMalformedChild?.(null);
return parsed === undefined ? [] : [parsed];
})
: [];
@@ -187,14 +215,16 @@ export function* captureDom(
let excludedUrls = 0;
let total = 0;
let unresolvedSvgNodes = 0;
for (const document of recordsValue(root.documents)) {
if (!Array.isArray(root.strings))
excludeDocumentEvidence(completeness, "invalid_protocol_value", null);
for (const document of snapshotDocuments(root.documents, completeness)) {
const documentUrl = cdpSnapshotString(strings, document.documentURL);
if (documentUrl === undefined) {
completeness?.exclude("dom", "invalid_protocol_value");
excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
if (allowedSanitizedUrl(documentUrl, allowedOrigins) === undefined) {
completeness?.exclude("dom", exclusionReasonForUrl(documentUrl));
excludeDocumentEvidence(completeness, exclusionReasonForUrl(documentUrl));
continue;
}
const baseUrlValue =
@@ -202,12 +232,15 @@ export function* captureDom(
? documentUrl
: cdpSnapshotValueString(strings, document.baseURL);
if (baseUrlValue === undefined) {
completeness?.exclude("dom", "invalid_protocol_value");
excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
const baseUrl = baseUrlValue || documentUrl;
const documentNodes = recordValue(document.nodes);
if (documentNodes === undefined) continue;
if (documentNodes === undefined) {
excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
const rawNodeTypes = numberArray(documentNodes.nodeType);
const rawNodeNames = numberArray(documentNodes.nodeName);
const rawNodeValues = numberArray(documentNodes.nodeValue);
@@ -225,12 +258,12 @@ export function* captureDom(
rawNodeValues.length !== rawNodeTypes.length ||
rawParents.length !== rawNodeTypes.length
) {
completeness?.exclude("dom", "invalid_protocol_value");
excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
if (rawAttributes.length > rawNodeTypes.length)
completeness?.exclude(
"dom",
excludeDocumentEvidence(
completeness,
"invalid_protocol_value",
rawAttributes.length - rawNodeTypes.length,
);
@@ -266,7 +299,7 @@ export function* captureDom(
attributeIndexes === undefined ||
!cdpSnapshotAttributesResolve(strings, attributeIndexes)
) {
completeness?.exclude("dom", "invalid_protocol_value");
excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
const attributes = attributeIndexes as readonly number[];
@@ -346,6 +379,32 @@ export function* captureDom(
};
}
/** Skipped document facts also make their derived URL and hint inventory incomplete. */
const excludeDocumentEvidence = (
completeness: CdpCaptureCompleteness | undefined,
reason: BrowserExclusionReason,
count: number | null = 1,
): void => {
completeness?.exclude("dom", reason, count);
completeness?.exclude("metadata", reason, count);
};
const snapshotDocuments = (
value: unknown,
completeness: CdpCaptureCompleteness | undefined,
): readonly UnknownRecord[] => {
if (!Array.isArray(value) || value.length === 0) {
excludeDocumentEvidence(completeness, "invalid_protocol_value", null);
return [];
}
return value.flatMap((item) => {
const document = recordValue(item);
if (document === undefined)
excludeDocumentEvidence(completeness, "invalid_protocol_value");
return document === undefined ? [] : [document];
});
};
function* captureDomMetadata(
pending: readonly PendingDomMetadata[],
completeness: CdpCaptureCompleteness | undefined,
+64 -21
View File
@@ -9,6 +9,7 @@ import {
type WebCaptureDimension,
} from "./webCaptureDiffSchemas.js";
import type { WebMcpDiscovery } from "./webMcpDiscovery.js";
import { fingerprintWebCaptureFrames } from "./webCaptureFrameComparison.js";
type Dimension = WebCaptureDimension;
type Change = WebCaptureChange;
@@ -172,13 +173,7 @@ const compareWebCaptureDimensions = (
const before = input.before.inspection;
const after = input.after.inspection;
return {
dom_structure: compareDimension(
singleton("document", digestCanonicalValue(domProjection(before))),
singleton("document", digestCanonicalValue(domProjection(after))),
sectionsComplete(before, ["frames", "dom"]) &&
sectionsComplete(after, ["frames", "dom"]),
"DOM or frame capture was incomplete in at least one observation.",
),
dom_structure: domDimension(before, after),
scripts: compareDimension(
keyed(
before.scripts.items.map(scriptProjection),
@@ -200,9 +195,10 @@ const compareWebCaptureDimensions = (
"Resource inventory was incomplete in at least one observation.",
),
network: networkDimension(before, after),
metadata: compareDimension(
singleton("metadata", digestCanonicalValue(metadataProjection(before))),
singleton("metadata", digestCanonicalValue(metadataProjection(after))),
metadata: compareAggregate(
"metadata",
metadataProjection(before),
metadataProjection(after),
sectionsComplete(before, ["metadata"]) &&
sectionsComplete(after, ["metadata"]),
"Safe metadata capture was incomplete in at least one observation.",
@@ -601,17 +597,64 @@ const incompleteSections = (completeness: {
...completeness.unavailable_sections,
]);
const domProjection = (inspection: WebPageInspection) => ({
frames: inspection.frames
.map(({ url, origin }) => ({ url, origin }))
.sort(
(left, right) =>
compareUnicodeCodePoints(left.url, right.url) ||
compareUnicodeCodePoints(left.origin ?? "", right.origin ?? "") ||
Number(left.origin !== null) - Number(right.origin !== null),
),
nodes: inspection.dom.nodes.map(({ index: _index, ...node }) => node),
});
/** Whole-collection digest differences cannot prove a change across omissions. */
const compareAggregate = (
identity: string,
before: unknown,
after: unknown,
complete: boolean,
reason: string,
): Dimension =>
compareDimension(
complete ? singleton(identity, digestCanonicalValue(before)) : new Map(),
complete ? singleton(identity, digestCanonicalValue(after)) : new Map(),
complete,
reason,
);
const domDimension = (
before: WebPageInspection,
after: WebPageInspection,
): Dimension => {
const beforeFrames = fingerprintWebCaptureFrames(before.frames);
const afterFrames = fingerprintWebCaptureFrames(after.frames);
const nodesComplete = [before, after].every(
(inspection) =>
sectionsComplete(inspection, ["dom"]) &&
inspection.dom.total_nodes === inspection.dom.nodes.length,
);
const framesComplete =
beforeFrames !== null &&
afterFrames !== null &&
[before, after].every((inspection) =>
sectionsComplete(inspection, ["frames"]),
);
const changes: Change[] = [];
if (
nodesComplete &&
digestCanonicalValue(before.dom.nodes) !==
digestCanonicalValue(after.dom.nodes)
)
changes.push({ identity: "document", change: "modified" });
if (framesComplete && beforeFrames !== afterFrames)
changes.push({ identity: "frames", change: "modified" });
if (changes.length > 0)
return {
status: "changed",
total_changes: changes.length,
changes,
reason: null,
};
return nodesComplete && framesComplete
? { status: "unchanged", total_changes: 0, changes: [], reason: null }
: {
status: "unknown",
total_changes: 0,
changes: [],
reason:
"DOM or frame capture was incomplete, or frame ancestry was unresolved in at least one observation.",
};
};
const scriptProjection = (
script: WebPageInspection["scripts"]["items"][number],
@@ -0,0 +1,62 @@
import { describe, expect, it } from "vitest";
import type { WebPageInspection } from "./browserObservationSchemas.js";
import { fingerprintWebCaptureFrames } from "./webCaptureFrameComparison.js";
const frame = (
id: string,
parent: string | null = null,
url = "https://app.example.test/frame",
): WebPageInspection["frames"][number] => ({
frame_id: id,
parent_frame_id: parent,
url,
origin: "https://app.example.test",
});
describe("capture-local frame graph normalization", () => {
it.each([
[frame("child", "missing")],
[frame("same"), frame("same")],
[frame("a", "b"), frame("b", "a")],
[frame("self", "self")],
])("keeps unresolved ancestry unknown (%#)", (...frames) => {
expect(fingerprintWebCaptureFrames(frames)).toBeNull();
});
it("preserves multiplicity while ignoring sibling order and frame IDs", () => {
const left = [frame("root"), frame("a", "root"), frame("b", "root")];
const renamed = [frame("y", "new"), frame("new"), frame("x", "new")];
expect(fingerprintWebCaptureFrames(left)).toBe(
fingerprintWebCaptureFrames(renamed),
);
expect(fingerprintWebCaptureFrames(left)).not.toBe(
fingerprintWebCaptureFrames(left.slice(0, 2)),
);
expect(fingerprintWebCaptureFrames(left)).not.toBe(
fingerprintWebCaptureFrames([
frame("root"),
frame("a", "root"),
frame("b", "a"),
]),
);
});
it("handles deep ancestry without recursive serialization", () => {
const frames = Array.from({ length: 5000 }, (_, index) =>
frame(String(index), index === 0 ? null : String(index - 1)),
);
const renamed = frames
.map((item) => ({
...item,
frame_id: "new-" + item.frame_id,
parent_frame_id:
item.parent_frame_id === null ? null : "new-" + item.parent_frame_id,
}))
.reverse();
expect(fingerprintWebCaptureFrames(frames)).toMatch(/^[a-f0-9]{64}$/u);
expect(fingerprintWebCaptureFrames(frames)).toBe(
fingerprintWebCaptureFrames(renamed),
);
});
});
+49
View File
@@ -0,0 +1,49 @@
import type { WebPageInspection } from "./browserObservationSchemas.js";
import { digestCanonicalValue } from "./canonicalDigest.js";
/** Compare complete frame forests without treating capture-local IDs as identity. */
export const fingerprintWebCaptureFrames = (
frames: WebPageInspection["frames"],
): string | null => {
const byId = new Map(frames.map((frame) => [frame.frame_id, frame]));
if (byId.size !== frames.length) return null;
const remaining = new Map(frames.map((frame) => [frame.frame_id, 0]));
for (const frame of frames) {
if (frame.parent_frame_id === null) continue;
const count = remaining.get(frame.parent_frame_id);
if (count === undefined) return null;
remaining.set(frame.parent_frame_id, count + 1);
}
const ready = frames.filter((frame) => remaining.get(frame.frame_id) === 0);
const children = new Map<string, string[]>();
const roots: string[] = [];
let examined = 0;
for (let index = 0; index < ready.length; index += 1) {
const frame = ready[index];
if (frame === undefined) throw new Error("Missing ready frame");
const fingerprint = digestCanonicalValue({
url: frame.url,
origin: frame.origin,
children: (children.get(frame.frame_id) ?? []).sort(),
});
examined += 1;
const parentId = frame.parent_frame_id;
if (parentId === null) {
roots.push(fingerprint);
continue;
}
const siblings = children.get(parentId) ?? [];
siblings.push(fingerprint);
children.set(parentId, siblings);
const count = (remaining.get(parentId) ?? 0) - 1;
remaining.set(parentId, count);
if (count === 0) {
const parent = byId.get(parentId);
if (parent === undefined)
throw new Error("Missing admitted parent frame");
ready.push(parent);
}
}
// Cyclic ancestry never reaches the leaf queue and cannot prove equality.
return examined === frames.length ? digestCanonicalValue(roots.sort()) : null;
};
@@ -0,0 +1,266 @@
import { describe, expect, it } from "vitest";
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js";
import { inspectWebPageInputSchema } from "../../../src/domain/browserObservation.js";
import type { WebPageInspection } from "../../../src/domain/browserObservationSchemas.js";
import { compareWebCaptures } from "../../../src/domain/webCaptureDiff.js";
import { compareWebCapturesInputSchema } from "../../../src/domain/webCaptureDiffSchemas.js";
import { startFakeCdpBrowser } from "../../fixtures/fakeCdpBrowser.js";
import { trackBrowser } from "./cdpBrowserProvider.support.js";
import { runCliJson } from "../../fixtures/cliJsonProcess.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
type SnapshotMode =
| "valid"
| "omitted-node"
| "changed-node"
| "missing-nodes"
| "null-nodes"
| "missing-documents"
| "invalid-documents"
| "empty-documents"
| "invalid-document"
| "mixed-documents"
| "missing-strings";
const snapshot = (origin: string, mode: SnapshotMode) => {
const document = {
documentURL: 0,
nodes: {
nodeType: [9, 1],
nodeName: [1, mode === "changed-node" ? 6 : 3],
nodeValue: [2, 2],
parentIndex: [-1, 0],
attributes: [[], mode === "omitted-node" ? null : [4, 5]],
},
};
let documents: unknown = [document];
if (mode === "missing-nodes") documents = [{ documentURL: 0 }];
if (mode === "null-nodes") documents = [{ documentURL: 0, nodes: null }];
if (mode === "missing-documents") documents = undefined;
if (mode === "invalid-documents") documents = {};
if (mode === "empty-documents") documents = [];
if (mode === "invalid-document") documents = [null];
if (mode === "mixed-documents") documents = [document, null];
return {
strings:
mode === "missing-strings"
? undefined
: [origin + "/app", "#document", "", "A", "href", "/target", "SPAN"],
documents,
};
};
const startBrowser = async () => {
const state = {
mode: "valid" as SnapshotMode,
nested: false,
renamed: false,
parentOverride: undefined as { value: unknown } | undefined,
};
const browser = await startFakeCdpBrowser({
commandResult(command, origin) {
if (command.method === "DOMSnapshot.captureSnapshot")
return snapshot(origin, state.mode);
if (command.method !== "Page.getFrameTree") return undefined;
const id = (name: string) => (state.renamed ? "new-" + name : name);
const child = {
frame: {
id: id("b"),
parentId:
state.parentOverride === undefined
? id(state.nested ? "a" : "main")
: state.parentOverride.value,
loaderId: "loader-b",
url: origin + "/b",
},
};
const parent = {
frame: {
id: id("a"),
parentId: id("main"),
loaderId: "loader-a",
url: origin + "/a",
},
childFrames: state.nested ? [child] : [],
};
return {
frameTree: {
frame: {
id: id("main"),
loaderId: "loader-main",
url: origin + "/app",
},
childFrames: state.nested ? [parent] : [parent, child],
},
};
},
});
trackBrowser(browser);
const inspect = async () => {
const result = await new CdpBrowserProvider().inspectPage(
inspectWebPageInputSchema.parse({
cdp_endpoint: browser.endpoint,
allowed_origins: [browser.allowedOrigin],
target_id: "allowed-page",
observation_ms: 0,
}),
);
if (!result.ok) throw result.error;
return result.value;
};
return { state, inspect };
};
const compare = (before: WebPageInspection, after: WebPageInspection) =>
compareWebCaptures(
compareWebCapturesInputSchema.parse({
before: { inspection: before },
after: { inspection: after },
}),
);
describe("DOM protocol omissions at the production capture boundary", () => {
it.each([
"missing-nodes",
"null-nodes",
"missing-documents",
"invalid-documents",
"empty-documents",
"invalid-document",
"mixed-documents",
"missing-strings",
] as const)("retains unknown coverage for %s", async (mode) => {
const { state, inspect } = await startBrowser();
state.mode = mode;
const capture = await inspect();
for (const section of ["dom", "metadata"])
expect(capture.completeness.unavailable_sections).toContain(section);
if (mode === "mixed-documents") {
expect(capture.dom.nodes.map(({ node_name }) => node_name)).toEqual([
"#document",
"A",
]);
expect(capture.metadata.dom_urls).toHaveLength(1);
}
const comparison = compare(capture, capture);
expect(comparison.dimensions.dom_structure.status).toBe("unknown");
expect(comparison.dimensions.metadata.status).toBe("unknown");
});
it("does not turn a rejected node and its URL into a page change", async () => {
const { state, inspect } = await startBrowser();
const valid = await inspect();
state.mode = "omitted-node";
const partial = await inspect();
expect(partial.dom.total_nodes).toBe(2);
expect(partial.dom.nodes).toHaveLength(1);
for (const [before, after] of [
[valid, partial],
[partial, valid],
]) {
if (before === undefined || after === undefined)
throw new Error("Missing pair");
const comparison = compare(before, after);
expect(comparison.dimensions.dom_structure.status).toBe("unknown");
expect(comparison.dimensions.metadata.status).toBe("unknown");
expect(comparison.overall_status).toBe("unknown");
}
});
});
describe("complete frame and DOM comparison", () => {
it.each([undefined, null, 17, ""])(
"keeps missing or invalid child parent evidence unknown (%#)",
async (parent) => {
const { state, inspect } = await startBrowser();
const before = await inspect();
state.parentOverride = { value: parent };
const after = await inspect();
expect(after.frames).toHaveLength(3);
expect(after.completeness.unavailable_sections).toContain("frames");
expect(compare(before, after).dimensions.dom_structure.status).toBe(
"unknown",
);
expect(compare(after, after).dimensions.dom_structure.status).toBe(
"unknown",
);
},
);
it("retains a complete observed node change", async () => {
const { state, inspect } = await startBrowser();
const before = await inspect();
expect(compare(before, before).dimensions.dom_structure.status).toBe(
"unchanged",
);
state.mode = "changed-node";
expect(
compare(before, await inspect()).dimensions.dom_structure.status,
).toBe("changed");
});
it("compares frame ancestry independently of volatile IDs", async () => {
const { state, inspect } = await startBrowser();
const before = await inspect();
state.renamed = true;
expect(
compare(before, await inspect()).dimensions.dom_structure.status,
).toBe("unchanged");
state.nested = true;
const nested = await inspect();
expect(nested.frames[2]?.parent_frame_id).toBe("new-a");
expect(compare(before, nested).dimensions.dom_structure.status).toBe(
"changed",
);
});
it("retains proven frame changes when DOM node evidence is incomplete", async () => {
const { state, inspect } = await startBrowser();
const before = await inspect();
state.mode = "omitted-node";
state.nested = true;
const comparison = compare(before, await inspect());
expect(comparison.dimensions.dom_structure.status).toBe("changed");
expect(comparison.dimensions.metadata.status).toBe("unknown");
});
});
describe("DOM evidence through the public CLI", () => {
it("preserves omissions and independently proven frame changes", async () => {
const { state, inspect } = await startBrowser();
const root = await createTestTempDirectory("rea-dom-comparison-");
const before = join(root, "before.json");
const after = join(root, "after.json");
await writeFile(before, JSON.stringify({ inspection: await inspect() }));
state.mode = "omitted-node";
for (const nested of [false, true]) {
state.nested = nested;
await writeFile(after, JSON.stringify({ inspection: await inspect() }));
expect(
await runCliJson(["compare-web-captures", before, after, "--json"]),
).toMatchObject({
operation: "compare_web_captures",
normalized_result: {
overall_status: nested ? "changed" : "unknown",
dimensions: {
dom_structure: { status: nested ? "changed" : "unknown" },
metadata: { status: "unknown" },
},
},
});
}
state.mode = "missing-nodes";
await writeFile(after, JSON.stringify({ inspection: await inspect() }));
expect(
await runCliJson(["compare-web-captures", after, after, "--json"]),
).toMatchObject({
operation: "compare_web_captures",
normalized_result: {
overall_status: "unknown",
dimensions: { dom_structure: { status: "unknown" } },
},
});
});
});
@@ -0,0 +1,322 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js";
import { inspectWebPageInputSchema } from "../../../src/domain/browserObservation.js";
import type { WebPageInspection } from "../../../src/domain/browserObservationSchemas.js";
import { compareWebCaptures } from "../../../src/domain/webCaptureDiff.js";
import { compareWebCapturesInputSchema } from "../../../src/domain/webCaptureDiffSchemas.js";
import { startFakeCdpBrowser } from "../../fixtures/fakeCdpBrowser.js";
import { runCliJson } from "../../fixtures/cliJsonProcess.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { trackBrowser } from "./cdpBrowserProvider.support.js";
type ResourceMode =
| "valid"
| "missing"
| "null"
| "wrong-type"
| "invalid-entry"
| "child-missing"
| "child-invalid-entry"
| "child-wrong-type"
| "empty";
const startBrowser = async () => {
const state = {
mode: "valid" as ResourceMode,
peerFields: undefined as Record<string, unknown> | undefined,
};
const browser = await startFakeCdpBrowser({
commandResult(command, origin) {
const malformedChildren =
state.mode === "child-invalid-entry" ||
state.mode === "child-wrong-type";
if (command.method === "Page.getFrameTree" && malformedChildren)
return {
frameTree: {
frame: { id: "frame-main", url: origin + "/app" },
childFrames:
state.mode === "child-wrong-type"
? {}
: [
{
frame: {
id: "child",
parentId: "frame-main",
url: origin + "/child",
},
},
null,
],
},
};
if (command.method !== "Page.getResourceTree") return undefined;
const resource = (name: string) => ({
url: origin + "/" + name,
type: "Script",
mimeType: "text/javascript",
contentSize: 128,
...(name === "peer.js" ? state.peerFields : undefined),
});
let resources: unknown = [resource("keep.js"), resource("peer.js")];
if (state.mode === "missing") resources = undefined;
if (state.mode === "null") resources = null;
if (state.mode === "wrong-type") resources = {};
if (state.mode === "invalid-entry")
resources = [resource("keep.js"), null];
if (state.mode === "empty") resources = [];
const malformedChildFrames =
state.mode === "child-wrong-type"
? {}
: state.mode === "child-invalid-entry"
? [
{
frame: {
id: "child",
parentId: "frame-main",
url: origin + "/child",
},
resources: [resource("child.js")],
},
null,
]
: undefined;
return {
frameTree: {
frame: { id: "frame-main", url: origin + "/app" },
resources,
...(malformedChildFrames === undefined
? {}
: { childFrames: malformedChildFrames }),
...(state.mode === "child-missing"
? {
childFrames: [
{
frame: {
id: "child",
parentId: "frame-main",
url: origin + "/child",
},
},
],
}
: {}),
},
};
},
});
trackBrowser(browser);
const inspect = async () => {
const result = await new CdpBrowserProvider().inspectPage(
inspectWebPageInputSchema.parse({
cdp_endpoint: browser.endpoint,
allowed_origins: [browser.allowedOrigin],
target_id: "allowed-page",
observation_ms: 0,
}),
);
if (!result.ok) throw result.error;
return result.value;
};
return { state, inspect, origin: browser.allowedOrigin };
};
const compareResources = (
before: WebPageInspection,
after: WebPageInspection,
) =>
compareWebCaptures(
compareWebCapturesInputSchema.parse({
before: { inspection: before },
after: { inspection: after },
}),
).dimensions.resources;
describe("resource inventory omissions through CDP", () => {
it.each([
{ fields: { type: undefined }, retained: false },
{ fields: { type: null }, retained: false },
{ fields: { type: 17 }, retained: false },
{ fields: { type: "" }, retained: false },
{ fields: { mimeType: undefined }, retained: false },
{ fields: { mimeType: null }, retained: false },
{ fields: { mimeType: 17 }, retained: false },
{ fields: { contentSize: -1 }, retained: true },
{ fields: { contentSize: null }, retained: true },
{ fields: { contentSize: "128" }, retained: true },
])(
"keeps malformed resource fields uncertain: $fields",
async ({ fields, retained }) => {
const { state, inspect, origin } = await startBrowser();
const before = await inspect();
state.peerFields = fields;
const after = await inspect();
for (const [left, right] of [
[before, after],
[after, before],
[after, after],
] as const)
expect(compareResources(left, right)).toMatchObject({
status: "unknown",
total_changes: 0,
changes: [],
});
expect(after.completeness.excluded).toContainEqual({
section: "resources",
reason: "invalid_protocol_value",
count: 1,
});
expect(after.resources.map(({ url }) => url).sort()).toEqual(
retained
? [origin + "/keep.js", origin + "/peer.js"]
: [origin + "/keep.js"],
);
if (retained)
expect(
after.resources.find(({ url }) => url.endsWith("/peer.js")),
).toMatchObject({
type: "Script",
mime_type: "text/javascript",
content_size: null,
});
},
);
it.each([
{
fields: { type: "Other", mimeType: "", contentSize: undefined },
size: null,
},
{ fields: { contentSize: 0 }, size: 0 },
{ fields: { contentSize: 1.5 }, size: 1.5 },
])(
"accepts valid optional and empty fields: $fields",
async ({ fields, size }) => {
const { state, inspect } = await startBrowser();
state.peerFields = fields;
const capture = await inspect();
expect(capture.resources).toHaveLength(2);
expect(
capture.resources.find(({ url }) => url.endsWith("/peer.js"))
?.content_size,
).toBe(size);
expect(capture.completeness.unavailable_sections).not.toContain(
"resources",
);
expect(compareResources(capture, capture).status).toBe("unchanged");
},
);
it.each([
"missing",
"null",
"wrong-type",
"invalid-entry",
"child-missing",
"child-invalid-entry",
"child-wrong-type",
] as const)(
"does not claim a resource removal or equality for %s",
async (mode) => {
const { state, inspect, origin } = await startBrowser();
const before = await inspect();
expect(before.resources).toHaveLength(2);
expect(before.completeness.unavailable_sections).not.toContain(
"resources",
);
state.mode = mode;
const after = await inspect();
for (const [left, right] of [
[before, after],
[after, before],
[after, after],
] as const)
expect(compareResources(left, right)).toMatchObject({
status: "unknown",
total_changes: 0,
changes: [],
});
expect(after.completeness.unavailable_sections).toContain("resources");
expect(after.completeness.excluded).toContainEqual({
section: "resources",
reason: "invalid_protocol_value",
count: mode === "invalid-entry" ? 1 : null,
});
if (mode === "child-invalid-entry" || mode === "child-wrong-type") {
expect(after.completeness.excluded).toContainEqual({
section: "frames",
reason: "invalid_protocol_value",
count: null,
});
expect(after.frames.map(({ frame_id }) => frame_id)).toContain(
"frame-main",
);
if (mode === "child-invalid-entry")
expect(after.frames.map(({ frame_id }) => frame_id)).toContain(
"child",
);
}
const expected =
mode === "invalid-entry"
? [origin + "/keep.js"]
: mode === "child-invalid-entry"
? [origin + "/child.js", origin + "/keep.js", origin + "/peer.js"]
: mode === "child-missing"
? [origin + "/keep.js", origin + "/peer.js"]
: mode === "child-wrong-type"
? [origin + "/keep.js", origin + "/peer.js"]
: [];
expect(after.resources.map(({ url }) => url).sort()).toEqual(expected);
},
);
it("distinguishes a complete empty inventory from an unavailable collection", async () => {
const { state, inspect } = await startBrowser();
const before = await inspect();
state.mode = "empty";
const after = await inspect();
expect(after.resources).toEqual([]);
expect(after.completeness.unavailable_sections).not.toContain("resources");
expect(compareResources(before, after)).toMatchObject({
status: "changed",
total_changes: 2,
changes: [{ change: "removed" }, { change: "removed" }],
});
expect(compareResources(after, after).status).toBe("unchanged");
});
});
describe("resource uncertainty in public CLI Evidence", () => {
it("preserves partial membership while complete inventories still prove removal", async () => {
const { state, inspect } = await startBrowser();
const root = await createTestTempDirectory("rea-resource-comparison-");
const before = join(root, "before.json");
const after = join(root, "after.json");
await writeFile(before, JSON.stringify({ inspection: await inspect() }));
for (const { mode, fields } of [
{ mode: "invalid-entry", fields: undefined },
{ mode: "valid", fields: { type: undefined } },
{ mode: "valid", fields: { contentSize: -1 } },
{ mode: "empty", fields: undefined },
] as const) {
state.mode = mode;
state.peerFields = fields;
await writeFile(after, JSON.stringify({ inspection: await inspect() }));
expect(
await runCliJson(["compare-web-captures", before, after, "--json"]),
).toMatchObject({
operation: "compare_web_captures",
normalized_result: {
dimensions: {
resources: {
status: mode === "empty" ? "changed" : "unknown",
total_changes: mode === "empty" ? 2 : 0,
},
},
},
});
}
});
});