mirror of
https://github.com/morluto/rea.git
synced 2026-11-15 19:23:24 +00:00
* refactor(browser): parse frame lifecycle events once by a registered event schema CdpConnection admits each event through protocol/CdpEvents.ts, which parses the five Page frame lifecycle events once into identity facts plus a frame URL observation, like a frame-tree node. Screenshot, page capture, the observation timeline, WebMCP, Electron inspection and the execution window read those facts instead of re-parsing the raw payload; CdpFrameFacts no longer carries the raw source record. Playwright CDP session subscriptions name an event from Playwright's protocol table, and an architecture guard counts the remaining raw event payload reads so they can only shrink. * refactor(browser): parse Runtime and Debugger events once by registered schemas Register Debugger.scriptParsed, Debugger.scriptFailedToParse, Runtime.executionContextCreated, Runtime.executionContextDestroyed and Runtime.consoleAPICalled. Script and context payloads carry both the tolerant facts passive capture uses and the active runtime's strict declaration, derived from one parse, plus the declaration's byte size for the runtime retention budget. Page capture, Electron script capture, the V8 Inspector provider and the runtime source inventory now read the registered payload. The cdp-raw-event-params baseline drops from 20 to 7. * refactor(browser): admit transport events through the registered-event owner Remove the unused cdpFrameUrl wrapper; its fragment cases now exercise observeCdpFrameUrl. Add a connection-level test that two subscribers share one parsed payload and the raw params. * fix(browser): report an unserializable event declaration's size instead of throwing JSON.stringify can throw on a declaration JSON.parse accepted (nesting deeper than the serializer's stack). The registry sized script and context declarations inside the connection's WebSocket callback, so such an event escaped as an uncaught exception. The size is now a Result; the runtime source inventory raises the failure inside its own catch, as before. * refactor(browser): parse Network events once by registered schemas (#10743) * refactor(browser): parse Network events once by registered schemas Register Network.requestWillBeSent, responseReceived, loadingFinished, loadingFailed, webSocketCreated and webSocketFrameSent/Received. The request payload carries the tolerant facts page capture and the timeline read plus the active runtime's strict declaration. Page capture's network and WebSocket handlers, the timeline's request tracking and the runtime request inventory now read the registered payload. The cdp-raw-event-params baseline drops from 7 to 5. * refactor(browser): drop payload helpers the capture handlers no longer use * refactor(browser): read every CDP event payload through the registry (#10745) * refactor(browser): read every CDP event payload through the registry Register Target.attachedToTarget, Target.detachedFromTarget, Page.lifecycleEvent and WebMCP.toolsAdded/toolsRemoved, and carry a requested navigation's reason and a commit's transition type in the frame-event payload. The timeline no longer threads raw params into its entries: each entry receives its timestamp, frame and loader identity explicitly. WebMCP discovery, the service-worker policy blocker and the target-event matcher read the registered payload. With no raw reads left, the cdp-raw-event-params guard is enforced at zero instead of counted. * refactor(browser): drop raw payload helpers the timeline and worker blocker no longer use * refactor(browser): type Playwright CDP session events by the registry (#10751) PlaywrightCdpSession.on subscribes only to registered events and hands the listener the payload parsed by its registered schema, returning the unsubscribe function. Private storage seeding reads registered payloads, including the events of non-flat child sessions, and applies its strict seeding policy to the parsed facts. Its local attach, detach, nested message and pause schemas and the storage-only script/context schemas are removed; Debugger.paused, Runtime.executionContextsCleared and Target.receivedMessageFromTarget join the registry. * test(browser): admit fake policy events through the event registry The service-worker policy now reads event.registered, but the PlaywrightScenarioSession fake connection delivered raw events, so three detached-target tests never reached the policy. The fake now admits each event with receiveCdpEvent as CdpConnection does. * fix(browser): fail the CDP session closed when an event listener throws (#10758) * fix(browser): fail the CDP session closed when an event listener throws An exception from a CDP event subscriber escaped CdpTransportCore's socket message callback uncaught (for example the service-worker blocker's protocol_error on a Target event without a sessionId). The transport now stops delivery, fails the connection as a protocol error whose cause is the listener's exception, rejects pending and later commands with it, and notifies protocol-failure subscribers. The Playwright CDP session relay applies the same rule to its event listeners. * fix(browser): report a failed service-worker policy connection at the next policy check The policy's event listener rejects a malformed Target event, which now fails its connection. With no policy command pending, nothing reported it and later iframe targets went unprotected. The policy records its connection's protocol failure and assertBlocked, run before each scenario step, rethrows it. * fix(browser): keep cleanup commands available after a listener failure A listener failure no longer marks the wire as broken. The transport still rejects pending commands, notifies protocol-failure subscribers, stops delivering events and refuses reuse, but it keeps reading replies and admits commands sent in cleanup grace. The service-worker policy can then remove its init scripts, restore the main realm, detach and release its socket. Wire-level protocol errors still refuse every later command. The PlaywrightScenarioSession policy fake gains onProtocolFailure, which the policy now subscribes to. * refactor(browser): keep raw CDP event payloads out of the received event (#10764) * refactor(browser): keep raw CDP event payloads out of the received event CdpReceivedEvent and CdpTransportEvent no longer carry params: receiveCdpEvent hands the raw payload only to the registered schema, so a subscriber cannot read it by property access, destructuring or computed keys. This supersedes the event.params text guard in architecture-guards, which is removed. * refactor(browser): read registered event payloads in the JavaScript verifiers The runtime, storage and URL-userinfo verifiers subscribe to CdpConnection, which no longer hands subscribers raw params. They now read event.registered. A test scans every scripts/ onEvent callback for raw payload reads, since the TypeScript check does not cover them. The storage verifier's pause trace records a malformed pause instead of the unregistered reason member. * refactor(browser): track the timeline's main frame with the document tracker (#10779) TimelineCapture kept its own mutable main frame id and replaced it on a root replacement. It now holds the CdpDocumentTracker that CdpTargetSession owns and observes the replacement document through it; every main-frame check (lifecycle, navigation, document request, redirect) reads the tracker's current frame. A characterization test pins that the replaced root's lifecycle events are no longer recorded. * refactor(browser): select CDP events only by their registered name (#10829) CdpReceivedEvent carried the wire method as a plain string, so a subscriber could compare it with a misspelt or unregistered name and the check compiled but never matched. Three owners selected Inspector.targetCrashed, an unregistered event, that way. Register Inspector.targetCrashed and drop the wire method from the received event. Subscribers select events by registered.method, whose type is the registered event names. The capture coverage start keeps reading the wire domain, now the event's domain field. * fix(browser): retire a policy child session that CDP reports gone After a listener failure the connection stops delivering events, so an iframe's Target.detachedFromTarget never retires its policy target. Cleanup then sent Page.removeScriptToEvaluateOnNewDocument to the dead session, kept the stale child, and every retry failed before detaching the root. The service-worker policy owner now treats CDP's session-not-found rejection as proof that the child is gone and retires it. The classifier moves from CdpTargetSession to CdpCommandRejection, its owner, and the target-session detach path uses the same one. * refactor(browser): route target events through one session owner (#10835) * refactor(browser): route target events through one session owner Page capture, WebMCP discovery, screenshot capture and the runtime sources each compared the envelope session ID themselves, while the observation window and execution collection used cdpTargetEventMatches, which also routes the root Target.detachedFromTarget that names the target's session. onCdpTargetEvent in CdpTargetEvents.ts now subscribes every target consumer with that one rule; the matcher is private and the per-subscriber comparisons are removed. Electron inspection keeps its own filter while its capture isolation awaits a maintainer decision. * fix(browser): report WebMCP subframe tools whose document is unverified (#10839) * fix(browser): report WebMCP subframe tools whose document is unverified A committed subframe navigation kept the frame's tools silently when CDP reported no loader at the same URL, and removed them on any URL change even when the loader proved the same document. Each frame now carries a CdpDocumentTracker, the session's document identity owner. Only a proven change removes the frame's tools. An unknown comparison keeps them visible, marks webmcp_tools attach-limited and names the frames whose document identity could not be established. * test(browser): prove an unverified live subframe limits WebMCP coverage * fix(browser): key a WebMCP registration by its frame and tool name A subframe tool kept across a committed navigation that changed the URL but not the loader was stored under a key built from the old URL, so a later toolsAdded for the same frame and name added a second entry. Registrations are now keyed by frame and name; tool_key still records the registering frame URL. * test(browser): deliver fake CDP events through the transport's admission (#10840) Event fakes each rebuilt the received event: CdpAgentHints typed its listeners as raw wire events behind an unknown cast, and the Electron, scenario-session and routing tests each wrapped receiveCdpEvent by hand. fakeCdpEvents (tests/fixtures) is a CdpConnection.onEvent fake whose emit admits each raw event with receiveCdpEvent and the wire byte count, so a change to the received event shape fails at the fake instead of silently not exercising subscribers. * refactor(browser): own the selected page generation in one target document tracker (#10860) * refactor(browser): own the selected page generation in one target document tracker Page capture, screenshot, the observation window, the runtime session and the execution window each built a document tracker or rule of their own from root-frame navigation events. CdpTargetDocument now follows the target session's root-frame events and the frame-tree replies its consumers read, classifies root identity once, and reports each navigation to consumers that decide only what it means for their evidence. * fix(browser): report a lost scenario policy connection before the next step (#10864) * fix(browser): report a lost scenario policy connection before the next step The service-worker policy recorded only protocol failures. When its CDP socket closed or its page session detached, later documents lost the blocking script, but the scenario learned of it only when a frame check found an unprotected realm. The owner now records the disconnect or detach when CDP reports it; the next policy check and the next step's action report it first. * fix(browser): refuse an unregistered CDP command before it reaches the browser (#10868) JavaScript verifiers can name any command, and the send owners refused an unregistered one only when its reply arrived, after the browser had acted on it. CdpConnection and the Playwright session now refuse the name before sending, and the Playwright session refuses an unregistered event name instead of subscribing to events no listener receives. The verifier text scan stays as an early warning and documents what it cannot see. * fix(browser): keep a proven root change and the loaderless selection's uncertainty Two review findings on the target document owner. A root event naming another frame with a malformed loader set `changed` but was announced as unidentified, so the execution window failed with protocol_error and dropped what it had collected; the owner now announces it as the proven change it is. The runtime session compared each frame-tree reply with the tracker's current document, which can learn a loader from an event, so a loaderless selection followed by an event and a reply with that loader compared as same; it now compares the reply with the selected document through the owner, and reports target_changed as before. * test(browser): refuse the next scenario step after the policy session detaches * refactor(browser): read WebMCP document identity from the target document owner (#10884) * refactor(browser): read WebMCP document identity from the target document owner WebMCP discovery compared the first and last frame-tree roots itself and kept a document tracker per subframe. CdpTargetDocument now follows the selected root and each committed subframe for it: the final frame tree feeds the owner, a root change proven by a navigation event during discovery is target_changed as in page capture, and a subframe whose committed document changed loses its tools. A subframe navigation without a loader still keeps the tools and reports them as unverified. * fix(browser): start a subframe first seen within its document with an unknown loader A subframe absent from the initial frame tree and first seen through Page.navigatedWithinDocument had no document tracker, so a later commit started a fresh one and the tools it registered meanwhile lost the unverified-document limitation. The owner now starts such a subframe with an unknown loader. * fix(browser): refuse a verifier's unregistered CDP event name at subscription (#10885) JavaScript verifiers subscribed to every event with CdpConnection.onEvent and compared registered.method with a string literal. A misspelt or unregistered name never matched and the verifier waited until its timeout. CdpConnection.onRegisteredEvent now takes one event name, refuses an unregistered one before subscribing (the same registry check the Playwright session applies), and hands the listener only that event's registered parse. The verifiers subscribe only through it, and the evaluation test rejects a raw onEvent subscription in scripts. * fix(browser): keep subframe documents by frame and loader without their URLs
748 lines
24 KiB
TypeScript
748 lines
24 KiB
TypeScript
import {
|
|
scopeCdpResponse,
|
|
type CdpRequestWillBeSentFacts,
|
|
} from "./CdpNetworkEventFacts.js";
|
|
import {
|
|
selectCdpScriptFrame,
|
|
type CdpExecutionContextCreatedFacts,
|
|
type CdpExecutionContextIdentity,
|
|
type CdpScriptParsedFacts,
|
|
} from "./CdpScriptContextFacts.js";
|
|
import type { CdpRootNavigation } from "./CdpTargetDocument.js";
|
|
import type { WebPageInspection } from "../domain/browserObservationSchemas.js";
|
|
import { sanitizeBrowserUrl } from "../domain/browserObservation.js";
|
|
import type { CdpEventPayload } from "./protocol/CdpEvents.js";
|
|
import { reportedFrameUrl } from "./protocol/CdpFrameTree.js";
|
|
import {
|
|
inferJsonShape,
|
|
type JsonShapeCapacityFailure,
|
|
} from "../domain/jsonShape.js";
|
|
import { jsonValueByteLength } from "../domain/jsonValueByteLength.js";
|
|
import { err, ok, type Result } from "../domain/result.js";
|
|
import { paddedBase64ByteLength } from "../domain/paddedBase64.js";
|
|
import { WEB_RUNTIME_LIMITS } from "../domain/webRuntime.js";
|
|
import { safeResponseMetadata } from "./CdpSafeMetadata.js";
|
|
import {
|
|
allowedSanitizedUrl,
|
|
isHttpUrl,
|
|
cdpStringValue,
|
|
type UnknownRecord,
|
|
} from "./CdpCaptureValues.js";
|
|
import {
|
|
requestBodyShape,
|
|
releaseBodyShapes,
|
|
updateResponseBodyShape,
|
|
} from "./CdpCaptureEventBodyShapes.js";
|
|
import {
|
|
consolePrimitive,
|
|
cdpSourceCoordinates,
|
|
exclusionReasonForUrl,
|
|
initiatorLocation,
|
|
isJsonMediaType,
|
|
} from "./CdpCaptureEventHelpers.js";
|
|
import type { CdpCaptureEventsState } from "./CdpCaptureEventState.js";
|
|
import {
|
|
CDP_CAPTURE_SCRIPT_METADATA_LIMITS,
|
|
type CapturedScript,
|
|
type NetworkState,
|
|
} from "./CdpCaptureEventTypes.js";
|
|
|
|
export const handleExecutionContextCreated = (
|
|
state: CdpCaptureEventsState,
|
|
facts: CdpExecutionContextCreatedFacts,
|
|
): void => {
|
|
if (facts.contextKey === null) return;
|
|
// Every reuse replaces future authority, including missing or malformed claims.
|
|
state.executionContextFrames.set(facts.contextKey, facts.frameClaim);
|
|
};
|
|
|
|
export const handleExecutionContextDestroyed = (
|
|
state: CdpCaptureEventsState,
|
|
identity: CdpExecutionContextIdentity,
|
|
): void => {
|
|
const key = identity.contextKey;
|
|
if (key !== null) state.executionContextFrames.delete(key);
|
|
};
|
|
|
|
export const handleScriptParsed = (
|
|
state: CdpCaptureEventsState,
|
|
facts: CdpScriptParsedFacts,
|
|
): void => {
|
|
const scriptId = facts.scriptId;
|
|
const rawUrl = facts.url ?? "";
|
|
const sanitized = allowedSanitizedUrl(rawUrl, state.allowedOrigins);
|
|
if (scriptId === undefined || scriptId === "") {
|
|
state.completeness.exclude("scripts", "invalid_protocol_value");
|
|
return;
|
|
}
|
|
const replacedBytes = state.scriptMetadataBytesById.get(scriptId) ?? 0;
|
|
state.scriptMetadataBudgetOmissionsById.delete(scriptId);
|
|
if (replacedBytes > 0) {
|
|
state.scripts.delete(scriptId);
|
|
state.scriptMetadataBytesById.delete(scriptId);
|
|
state.retainedScriptMetadataBytes -= replacedBytes;
|
|
}
|
|
if (sanitized === undefined) {
|
|
state.completeness.exclude("scripts", exclusionReasonForUrl(rawUrl));
|
|
return;
|
|
}
|
|
const executionContext = facts.contextKey;
|
|
const attribution = selectCdpScriptFrame(
|
|
facts.frameClaim,
|
|
executionContext === null
|
|
? undefined
|
|
: state.executionContextFrames.get(executionContext),
|
|
);
|
|
if (attribution.invalid || attribution.conflicting)
|
|
state.completeness.exclude("scripts", "invalid_protocol_value");
|
|
const baseScript = {
|
|
scriptId,
|
|
rawUrl,
|
|
url: sanitized.url,
|
|
origin: sanitized.origin,
|
|
...facts.metadata,
|
|
executionContextKey: executionContext,
|
|
frameId: attribution.frameId,
|
|
};
|
|
const baseBytes = scriptMetadataStringBytes(baseScript);
|
|
const sourceMapDeclared =
|
|
typeof facts.sourceMapUrl === "string" && facts.sourceMapUrl.length > 0;
|
|
if (
|
|
state.scripts.size >= CDP_CAPTURE_SCRIPT_METADATA_LIMITS.scripts ||
|
|
baseBytes >
|
|
CDP_CAPTURE_SCRIPT_METADATA_LIMITS.retainedBytes -
|
|
state.retainedScriptMetadataBytes
|
|
) {
|
|
state.rejectedScriptMetadataBudgetCount += 1;
|
|
if (sourceMapDeclared) state.rejectedSourceMapMetadataBudgetCount += 1;
|
|
return;
|
|
}
|
|
const sourceMap = sourceMapForScript(
|
|
facts.sourceMapUrl,
|
|
rawUrl,
|
|
state,
|
|
CDP_CAPTURE_SCRIPT_METADATA_LIMITS.retainedBytes -
|
|
state.retainedScriptMetadataBytes -
|
|
baseBytes,
|
|
);
|
|
const { budgetExhausted, ...sourceMapFields } = sourceMap;
|
|
const script: CapturedScript = { ...baseScript, ...sourceMapFields };
|
|
if (budgetExhausted)
|
|
state.scriptMetadataBudgetOmissionsById.set(scriptId, true);
|
|
const retainedBytes =
|
|
baseBytes +
|
|
stringByteLength(sourceMap.sourceMapUrl) +
|
|
stringByteLength(sourceMap.sourceMapRawUrl);
|
|
state.retainedScriptMetadataBytes += retainedBytes;
|
|
state.scriptMetadataBytesById.set(scriptId, retainedBytes);
|
|
state.scripts.set(scriptId, script);
|
|
};
|
|
|
|
const scriptMetadataStringBytes = (script: {
|
|
readonly scriptId: string;
|
|
readonly rawUrl: string;
|
|
readonly url: string;
|
|
readonly origin: string | null;
|
|
readonly hash: string | null;
|
|
readonly language: string | null;
|
|
readonly executionContextKey: string | null;
|
|
readonly frameId: string | null;
|
|
}): number =>
|
|
stringByteLength(script.scriptId) +
|
|
stringByteLength(script.rawUrl) +
|
|
stringByteLength(script.url) +
|
|
stringByteLength(script.origin) +
|
|
stringByteLength(script.hash) +
|
|
stringByteLength(script.language) +
|
|
stringByteLength(script.executionContextKey) +
|
|
stringByteLength(script.frameId);
|
|
|
|
const stringByteLength = (value: string | null): number =>
|
|
value === null ? 0 : Buffer.byteLength(value, "utf8");
|
|
|
|
/** Approximate fixed bytes of one retained console entry or frame object, so a
|
|
* stream of minimal entries cannot admit millions of objects into the budget. */
|
|
const RETAINED_EVENT_ENTRY_OVERHEAD_BYTES = 64;
|
|
/** Approximate fixed bytes of one captured console text value wrapper beyond
|
|
* its retained text. */
|
|
const RETAINED_EVENT_TEXT_VALUE_BYTES = 32;
|
|
|
|
const sourceMapForScript = (
|
|
value: unknown,
|
|
scriptUrl: string,
|
|
state: CdpCaptureEventsState,
|
|
remainingBytes: number,
|
|
): {
|
|
readonly sourceMapUrl: string | null;
|
|
readonly sourceMapRawUrl: string | null;
|
|
readonly budgetExhausted: boolean;
|
|
} => {
|
|
const declaredUrl = cdpStringValue(value);
|
|
if (value !== undefined && declaredUrl === undefined)
|
|
state.completeness.exclude("source_maps", "invalid_protocol_value");
|
|
if (declaredUrl === undefined || declaredUrl === "")
|
|
return {
|
|
sourceMapUrl: null,
|
|
sourceMapRawUrl: null,
|
|
budgetExhausted: false,
|
|
};
|
|
let rawUrl: string;
|
|
try {
|
|
rawUrl = new URL(declaredUrl, scriptUrl).href;
|
|
} catch (cause: unknown) {
|
|
// Unparseable URLs are recorded as unsupported, not as failures.
|
|
void cause;
|
|
state.completeness.exclude("source_maps", "unsupported_url");
|
|
return {
|
|
sourceMapUrl: null,
|
|
sourceMapRawUrl: null,
|
|
budgetExhausted: false,
|
|
};
|
|
}
|
|
const sanitized = allowedSanitizedUrl(rawUrl, state.allowedOrigins);
|
|
if (sanitized !== undefined) {
|
|
if (
|
|
stringByteLength(rawUrl) + stringByteLength(sanitized.url) >
|
|
remainingBytes
|
|
) {
|
|
return {
|
|
sourceMapUrl: null,
|
|
sourceMapRawUrl: null,
|
|
budgetExhausted: true,
|
|
};
|
|
}
|
|
return {
|
|
sourceMapUrl: sanitized.url,
|
|
sourceMapRawUrl: rawUrl,
|
|
budgetExhausted: false,
|
|
};
|
|
}
|
|
state.completeness.exclude("source_maps", exclusionReasonForUrl(rawUrl));
|
|
return {
|
|
sourceMapUrl: null,
|
|
sourceMapRawUrl: null,
|
|
budgetExhausted: false,
|
|
};
|
|
};
|
|
|
|
export const handleRequestWillBeSent = (
|
|
state: CdpCaptureEventsState,
|
|
facts: CdpRequestWillBeSentFacts,
|
|
): void => {
|
|
const requestId = facts.requestId;
|
|
const request = facts.request;
|
|
if (requestId === null) {
|
|
state.completeness.exclude("network_requests", "invalid_protocol_value");
|
|
return;
|
|
}
|
|
if (state.malformedRedirectRequestIds.has(requestId)) return;
|
|
const previous = state.network.get(requestId);
|
|
if (
|
|
facts.redirect.kind === "missing" &&
|
|
previous !== undefined &&
|
|
preserveMalformedRedirectEvidence(state, requestId, previous)
|
|
)
|
|
return;
|
|
const method = request?.method;
|
|
const initiator = facts.initiator;
|
|
const initiatorType = initiator?.type;
|
|
if (
|
|
request === undefined ||
|
|
method === undefined ||
|
|
method.length === 0 ||
|
|
initiatorType === undefined ||
|
|
initiatorType.length === 0
|
|
) {
|
|
preserveMalformedRedirectEvidence(state, requestId, previous);
|
|
return;
|
|
}
|
|
const sanitized = allowedSanitizedUrl(request.url, state.allowedOrigins);
|
|
if (sanitized === undefined) {
|
|
state.completeness.exclude(
|
|
"network_requests",
|
|
exclusionReasonForUrl(request.url),
|
|
);
|
|
releaseBodyShapes(state, requestId);
|
|
state.network.delete(requestId);
|
|
state.networkRequestTimestamps.delete(requestId);
|
|
return;
|
|
}
|
|
const redirectResponse =
|
|
facts.redirect.kind === "response" ? facts.redirect.response : undefined;
|
|
if (
|
|
facts.redirect.kind === "malformed" &&
|
|
preserveMalformedRedirectEvidence(state, requestId, previous)
|
|
)
|
|
return;
|
|
const redirects = [...(previous?.redirects ?? [])];
|
|
if (redirectResponse !== undefined) {
|
|
const responseUrl = scopeCdpResponse(
|
|
redirectResponse,
|
|
state.allowedOrigins,
|
|
);
|
|
if (responseUrl.scope === "approved" && !redirectResponse.status.ok)
|
|
state.completeness.exclude("network_requests", "invalid_protocol_value");
|
|
if (previous === undefined) {
|
|
if (responseUrl.scope !== "approved")
|
|
state.completeness.exclude("network_requests", responseUrl.reason);
|
|
else
|
|
// CDP may deliver the continuation after its predecessor was excluded
|
|
// by origin policy or before capture began. Keep the final request but
|
|
// report missing prior coverage without calling it malformed protocol.
|
|
state.completeness.attachLimited("network_requests");
|
|
} else if (responseUrl.scope !== "approved") {
|
|
// redirectResponse belongs to the same request ID and can contain an
|
|
// excluded origin. Drop the entire chain before retaining any details.
|
|
state.completeness.exclude("network_requests", responseUrl.reason);
|
|
releaseBodyShapes(state, requestId);
|
|
state.network.delete(requestId);
|
|
state.networkRequestTimestamps.delete(requestId);
|
|
return;
|
|
} else {
|
|
const encodedDataLength = redirectResponse.encodedDataLength;
|
|
if (encodedDataLength === undefined || encodedDataLength < 0)
|
|
state.completeness.exclude(
|
|
"network_requests",
|
|
"invalid_protocol_value",
|
|
);
|
|
redirects.push({
|
|
url: previous.url,
|
|
response_url: responseUrl.url.url,
|
|
method: previous.method,
|
|
resource_type: previous.resource_type,
|
|
status: redirectResponse.status.ok
|
|
? redirectResponse.status.value
|
|
: null,
|
|
mime_type: redirectResponse.mimeType ?? null,
|
|
encoded_data_length:
|
|
encodedDataLength === undefined || encodedDataLength < 0
|
|
? null
|
|
: encodedDataLength,
|
|
request_timestamp:
|
|
state.networkRequestTimestamps.get(requestId) ?? null,
|
|
// CDP reports redirectResponse on the next requestWillBeSent event.
|
|
// This is the event boundary, not a separately observed response time.
|
|
redirect_event_timestamp: facts.timestamp ?? null,
|
|
});
|
|
}
|
|
}
|
|
const location = initiatorLocation(initiator?.source);
|
|
if (!location.ok)
|
|
state.completeness.exclude("network_initiators", "invalid_protocol_value");
|
|
const initiatorFrame = location.ok ? location.value : undefined;
|
|
const coordinates = cdpSourceCoordinates(initiatorFrame);
|
|
if (coordinates.invalid)
|
|
state.completeness.exclude("network_initiators", "invalid_protocol_value");
|
|
const rawInitiatorUrl = cdpStringValue(initiatorFrame?.url);
|
|
const initiatorUrl = allowedSanitizedUrl(
|
|
rawInitiatorUrl,
|
|
state.allowedOrigins,
|
|
);
|
|
if (rawInitiatorUrl !== undefined && initiatorUrl === undefined)
|
|
state.completeness.exclude(
|
|
"network_initiators",
|
|
exclusionReasonForUrl(rawInitiatorUrl),
|
|
);
|
|
state.network.set(requestId, {
|
|
request_id: requestId,
|
|
url: sanitized.url,
|
|
origin: sanitized.origin ?? "",
|
|
method,
|
|
resource_type: facts.resourceType ?? null,
|
|
status: null,
|
|
mime_type: null,
|
|
encoded_data_length: null,
|
|
redirects,
|
|
initiator: {
|
|
type: initiatorType,
|
|
url: initiatorUrl?.url ?? null,
|
|
line: coordinates.line,
|
|
column: coordinates.column,
|
|
},
|
|
body_shapes: requestBodyShape(state, requestId, request.source),
|
|
});
|
|
const timestamp = facts.timestamp;
|
|
if (timestamp === undefined) state.networkRequestTimestamps.delete(requestId);
|
|
else state.networkRequestTimestamps.set(requestId, timestamp);
|
|
};
|
|
|
|
const preserveMalformedRedirectEvidence = (
|
|
state: CdpCaptureEventsState,
|
|
requestId: string,
|
|
previous: NetworkState | undefined,
|
|
): boolean => {
|
|
state.completeness.exclude("network_requests", "invalid_protocol_value");
|
|
if (previous === undefined) return false;
|
|
state.malformedRedirectRequestIds.add(requestId);
|
|
if (
|
|
state.input.include_json_body_shapes &&
|
|
isJsonMediaType(previous.mime_type) &&
|
|
previous.body_shapes.response === null
|
|
) {
|
|
updateResponseBodyShape(state, requestId, null);
|
|
state.completeness.exclude("json_body_shapes", "invalid_protocol_value");
|
|
}
|
|
return true;
|
|
};
|
|
|
|
export const handleResponseReceived = (
|
|
state: CdpCaptureEventsState,
|
|
facts: CdpEventPayload<"Network.responseReceived">,
|
|
): void => {
|
|
const requestId = facts.requestId;
|
|
if (requestId === null) {
|
|
state.completeness.exclude("network_requests", "invalid_protocol_value");
|
|
return;
|
|
}
|
|
if (state.malformedRedirectRequestIds.has(requestId)) return;
|
|
const current = state.network.get(requestId);
|
|
if (current === undefined) return;
|
|
const scoped = scopeCdpResponse(
|
|
facts.response,
|
|
state.allowedOrigins,
|
|
"unattributed_origin",
|
|
);
|
|
if (scoped.scope !== "approved") {
|
|
state.completeness.exclude("network_requests", scoped.reason);
|
|
releaseBodyShapes(state, requestId);
|
|
state.network.delete(requestId);
|
|
state.networkRequestTimestamps.delete(requestId);
|
|
return;
|
|
}
|
|
const { url: sanitized, response } = scoped;
|
|
if (!response.status.ok)
|
|
state.completeness.exclude("network_requests", "invalid_protocol_value");
|
|
state.network.set(requestId, {
|
|
...current,
|
|
status: response.status.ok ? response.status.value : null,
|
|
mime_type: response.mimeType ?? null,
|
|
});
|
|
const metadata = safeResponseMetadata(
|
|
requestId,
|
|
sanitized.url,
|
|
response,
|
|
state.allowedOrigins,
|
|
);
|
|
state.responseMetadata.push(metadata.response);
|
|
state.agentHints.push(...metadata.agentHints);
|
|
};
|
|
|
|
export const handleLoadingFinished = (
|
|
state: CdpCaptureEventsState,
|
|
facts: CdpEventPayload<"Network.loadingFinished">,
|
|
): void => {
|
|
const requestId = facts.requestId;
|
|
if (requestId === null) return;
|
|
if (state.malformedRedirectRequestIds.has(requestId)) return;
|
|
const current = state.network.get(requestId);
|
|
if (current === undefined) return;
|
|
const encodedDataLength = facts.encodedDataLength;
|
|
if (encodedDataLength === undefined || encodedDataLength < 0) {
|
|
state.completeness.exclude("network_requests", "invalid_protocol_value");
|
|
return;
|
|
}
|
|
state.network.set(requestId, {
|
|
...current,
|
|
encoded_data_length: encodedDataLength,
|
|
});
|
|
};
|
|
|
|
export const handleConsoleAPICalled = (
|
|
state: CdpCaptureEventsState,
|
|
call: CdpEventPayload<"Runtime.consoleAPICalled">,
|
|
): void => {
|
|
const topFrame = call.topFrame;
|
|
if (!topFrame.ok) {
|
|
state.completeness.exclude("console_events", "invalid_protocol_value");
|
|
return;
|
|
}
|
|
const frame = topFrame.value;
|
|
if (frame === undefined) {
|
|
state.completeness.exclude("console_events", "unattributed_origin");
|
|
return;
|
|
}
|
|
const coordinates = cdpSourceCoordinates(frame);
|
|
if (coordinates.invalid)
|
|
state.completeness.exclude("console_events", "invalid_protocol_value");
|
|
const source = allowedSanitizedUrl(frame.url, state.allowedOrigins);
|
|
if (source === undefined) {
|
|
state.completeness.exclude(
|
|
"console_events",
|
|
exclusionReasonForUrl(cdpStringValue(frame.url)),
|
|
);
|
|
return;
|
|
}
|
|
// A missing inventory is unknown, not empty.
|
|
const arguments_ = call.arguments;
|
|
const argumentTypes = arguments_?.map((argument) =>
|
|
cdpStringValue(argument?.type),
|
|
);
|
|
const timestamp = call.timestamp;
|
|
if (
|
|
timestamp === undefined ||
|
|
argumentTypes === undefined ||
|
|
argumentTypes.includes(undefined)
|
|
)
|
|
state.completeness.exclude("console_events", "invalid_protocol_value");
|
|
const type = call.type ?? "unknown";
|
|
const textCapture = captureConsoleText(state, arguments_);
|
|
// The shared retained-event budget bounds the whole console inventory, so a
|
|
// page cannot grow the capture without limit; a refused entry stays counted
|
|
// as a dropped event instead of vanishing.
|
|
const entryBytes =
|
|
RETAINED_EVENT_ENTRY_OVERHEAD_BYTES +
|
|
stringByteLength(type) +
|
|
stringByteLength(source.url) +
|
|
(argumentTypes ?? []).reduce(
|
|
(sum, argumentType) => sum + stringByteLength(argumentType ?? "unknown"),
|
|
0,
|
|
) +
|
|
textCapture.retained_bytes +
|
|
textCapture.values.length * RETAINED_EVENT_TEXT_VALUE_BYTES;
|
|
if (
|
|
entryBytes >
|
|
WEB_RUNTIME_LIMITS.retainedEventBytes - state.retainedEventBytes
|
|
) {
|
|
state.completeness.drop("console_events");
|
|
return;
|
|
}
|
|
state.retainedEventBytes += entryBytes;
|
|
state.console.push({
|
|
type,
|
|
timestamp: timestamp ?? null,
|
|
argument_types:
|
|
argumentTypes?.map((argumentType) => argumentType ?? "unknown") ?? null,
|
|
url: source.url,
|
|
line: coordinates.line,
|
|
column: coordinates.column,
|
|
text_capture: textCapture,
|
|
});
|
|
};
|
|
|
|
const captureConsoleText = (
|
|
state: CdpCaptureEventsState,
|
|
arguments_: readonly (UnknownRecord | undefined)[] | undefined,
|
|
): WebPageInspection["console"]["events"][number]["text_capture"] => {
|
|
if (!state.input.include_console_text)
|
|
return {
|
|
status: "not_approved",
|
|
values: [],
|
|
retained_bytes: 0,
|
|
};
|
|
if (arguments_ === undefined)
|
|
return {
|
|
status: "unavailable",
|
|
values: [],
|
|
retained_bytes: 0,
|
|
};
|
|
const values: WebPageInspection["console"]["events"][number]["text_capture"]["values"] =
|
|
[];
|
|
let retainedBytes = 0;
|
|
for (const [argumentIndex, argument] of arguments_.entries()) {
|
|
if (argument === undefined) continue;
|
|
const primitive = consolePrimitive(argument);
|
|
if (primitive === undefined) continue;
|
|
const text = primitive.text;
|
|
values.push({
|
|
argument_index: argumentIndex,
|
|
type: primitive.type,
|
|
text,
|
|
});
|
|
retainedBytes += Buffer.byteLength(text);
|
|
}
|
|
return {
|
|
status: "included",
|
|
values,
|
|
retained_bytes: retainedBytes,
|
|
};
|
|
};
|
|
|
|
export const handleWebSocketFrame = (
|
|
state: CdpCaptureEventsState,
|
|
frame: CdpEventPayload<"Network.webSocketFrameSent">,
|
|
direction: "sent" | "received",
|
|
): void => {
|
|
const requestId = frame.requestId;
|
|
if (requestId === undefined) return;
|
|
const connection = state.webSocketConnectionsById.get(requestId);
|
|
if (connection === undefined) return;
|
|
const opcodeValue = frame.opcode;
|
|
// RFC 6455 encodes opcodes in four bits; keep reserved values as observed data.
|
|
const opcode =
|
|
opcodeValue !== undefined &&
|
|
Number.isInteger(opcodeValue) &&
|
|
opcodeValue >= 0 &&
|
|
opcodeValue <= 15
|
|
? opcodeValue
|
|
: null;
|
|
const payload = frame.payloadData;
|
|
const payloadBytes =
|
|
opcode === null || payload === undefined
|
|
? null
|
|
: opcode === 1
|
|
? Buffer.byteLength(payload)
|
|
: (paddedBase64ByteLength(payload) ?? null);
|
|
if (opcode === null || payloadBytes === null)
|
|
state.completeness.exclude("websocket_frames", "invalid_protocol_value");
|
|
const availableBytes =
|
|
WEB_RUNTIME_LIMITS.retainedEventBytes - state.retainedEventBytes;
|
|
const baseBytes =
|
|
RETAINED_EVENT_ENTRY_OVERHEAD_BYTES + stringByteLength(direction);
|
|
if (baseBytes > availableBytes) {
|
|
state.completeness.drop("websocket_frames");
|
|
return;
|
|
}
|
|
const captured = captureWebSocketShape(
|
|
state,
|
|
payload,
|
|
opcode,
|
|
availableBytes - baseBytes,
|
|
);
|
|
if (!captured.ok) {
|
|
state.completeness.drop("websocket_frames");
|
|
state.completeness.exclude("websocket_shapes", "resource_budget_exhausted");
|
|
return;
|
|
}
|
|
const entryBytes = baseBytes + captured.value.serializedBytes;
|
|
if (entryBytes > availableBytes) {
|
|
state.completeness.drop("websocket_frames");
|
|
return;
|
|
}
|
|
state.retainedEventBytes += entryBytes;
|
|
connection.events.push({
|
|
direction,
|
|
opcode,
|
|
payload_bytes: payloadBytes,
|
|
payload_shape: captured.value.payloadShape,
|
|
});
|
|
};
|
|
|
|
type WebSocketShape =
|
|
WebPageInspection["network"]["websocket_connections"][number]["events"][number]["payload_shape"];
|
|
const jsonShapeEnvelopeBytes =
|
|
jsonValueByteLength({ format: "json", json_shape: null }) -
|
|
jsonValueByteLength(null);
|
|
|
|
const captureWebSocketShape = (
|
|
state: CdpCaptureEventsState,
|
|
payload: string | undefined,
|
|
opcode: number | null,
|
|
maximumBytes: number,
|
|
): Result<
|
|
{ readonly payloadShape: WebSocketShape; readonly serializedBytes: number },
|
|
JsonShapeCapacityFailure
|
|
> => {
|
|
if (!state.input.include_websocket_shapes || opcode === null)
|
|
return ok({ payloadShape: null, serializedBytes: 0 });
|
|
if (opcode !== 1) {
|
|
const payloadShape: WebSocketShape = { format: "binary", json_shape: null };
|
|
return ok({
|
|
payloadShape,
|
|
serializedBytes: jsonValueByteLength(payloadShape),
|
|
});
|
|
}
|
|
if (payload === undefined)
|
|
return ok({ payloadShape: null, serializedBytes: 0 });
|
|
const parsed = inferJsonShape(
|
|
payload,
|
|
Math.max(0, maximumBytes - jsonShapeEnvelopeBytes),
|
|
);
|
|
if (!parsed.ok) {
|
|
if (parsed.error.kind === "capacity") return err(parsed.error);
|
|
const payloadShape: WebSocketShape = { format: "text", json_shape: null };
|
|
return ok({
|
|
payloadShape,
|
|
serializedBytes: jsonValueByteLength(payloadShape),
|
|
});
|
|
}
|
|
const payloadShape: WebSocketShape = {
|
|
format: "json",
|
|
json_shape: parsed.value.shape,
|
|
};
|
|
return ok({
|
|
payloadShape,
|
|
serializedBytes: jsonShapeEnvelopeBytes + parsed.value.serializedBytes,
|
|
});
|
|
};
|
|
|
|
export const handleWebSocketCreated = (
|
|
state: CdpCaptureEventsState,
|
|
created: CdpEventPayload<"Network.webSocketCreated">,
|
|
): void => {
|
|
const { requestId, url: rawUrl } = created;
|
|
if (requestId === undefined) {
|
|
state.completeness.exclude(
|
|
"websocket_connections",
|
|
"invalid_protocol_value",
|
|
);
|
|
return;
|
|
}
|
|
if (rawUrl === undefined) {
|
|
state.completeness.exclude(
|
|
"websocket_connections",
|
|
"invalid_protocol_value",
|
|
);
|
|
return;
|
|
}
|
|
const sourceUrl = sanitizeBrowserUrl(rawUrl).url;
|
|
let parsed: URL;
|
|
try {
|
|
parsed = new URL(sourceUrl);
|
|
} catch (cause: unknown) {
|
|
// Unparseable URLs are recorded as unsupported, not as failures.
|
|
void cause;
|
|
state.completeness.exclude("websocket_connections", "unsupported_url");
|
|
return;
|
|
}
|
|
if (parsed.protocol === "ws:") parsed.protocol = "http:";
|
|
else if (parsed.protocol === "wss:") parsed.protocol = "https:";
|
|
else {
|
|
state.completeness.exclude("websocket_connections", "unsupported_url");
|
|
return;
|
|
}
|
|
if (!state.allowedOrigins.has(parsed.origin)) {
|
|
state.completeness.exclude("websocket_connections", "disallowed_origin");
|
|
return;
|
|
}
|
|
if (state.webSocketConnectionsById.has(requestId)) {
|
|
state.completeness.exclude(
|
|
"websocket_connections",
|
|
"invalid_protocol_value",
|
|
);
|
|
return;
|
|
}
|
|
state.webSocketConnectionsById.set(requestId, {
|
|
request_id: requestId,
|
|
url: sourceUrl,
|
|
events: [],
|
|
});
|
|
};
|
|
|
|
/**
|
|
* Record what a root navigation means for capture: an unreadable root makes
|
|
* the capture's document unprovable, and a navigation away from the selected
|
|
* document must stay inside the approved origins. The target session's
|
|
* document owner has already compared the document.
|
|
*/
|
|
export const handleRootNavigation = (
|
|
state: CdpCaptureEventsState,
|
|
navigation: CdpRootNavigation,
|
|
): void => {
|
|
if (navigation.kind === "unreadable_child") {
|
|
state.completeness.exclude("frames", "invalid_protocol_value");
|
|
return;
|
|
}
|
|
if (navigation.kind === "unidentified") {
|
|
state.completeness.exclude("frames", "invalid_protocol_value");
|
|
state.unknownRootIdentity = true;
|
|
return;
|
|
}
|
|
if (navigation.comparison === "same") return;
|
|
const rawUrl = reportedFrameUrl(navigation.frame.url);
|
|
if (
|
|
isHttpUrl(rawUrl) &&
|
|
allowedSanitizedUrl(rawUrl, state.allowedOrigins) === undefined
|
|
)
|
|
state.originViolation = true;
|
|
};
|