mirror of
https://github.com/morluto/rea.git
synced 2026-11-16 19:53:50 +00:00
* fix(native): keep unreported value-flow truncation and archive decode coverage unknown * test(verify): count multiline optional-fact defaults and pin the guard's exclusions * fix(domain): keep unknown counts and costs unknown in comparison and coverage summaries (#11325) * fix(domain): keep unknown counts and costs unknown in comparison and coverage summaries * refactor(verify): exempt legitimate optional-fact defaults with a reasoned marker (#11355) * refactor(verify): exempt legitimate optional-fact defaults with a reasoned marker The optional-fact-default guard counted request defaults and absence that is modelled elsewhere as hidden unknowns. A comment `// optional-fact-default: <reason>` on the default's line or the line before now exempts it; the guard reads unstripped source for the marker and ignores matches inside comment lines. Marked: the two evidence paging offsets and four Swift literal reads whose absence swiftLiteralCoverage reports. Removed instead of marked: jsdiff 9 types Change.count as required, so its two `?? 0` defaults were dead; the evidence link traversal now carries retained records, so it no longer re-looks them up behind `?? []`. Baseline 31 -> 22. * fix(verify): blank comment spans instead of skipping comment-led lines in the optional-fact guard Skipping a match whose line starts like a comment hid live code after a leading block comment and counted defaults inside a block comment without leading stars. The counter now blanks comment text (keeping offsets and line breaks, skipping string literals) and matches the blanked code, while exemption markers are still read from the original lines. * refactor(domain): mark optional-fact defaults whose absence is not an observation (#11389) * refactor(domain): mark optional-fact defaults whose absence is not an observation Per-site review of the remaining optional-fact-default entries. Each marked default is a caller option (trace ignore_fields, environment rules), a list whose producer omits the empty case (unknown_size_frames, the schema's min-1 representation_node_ids), caveat text whose completeness another field carries (call diagnostics, unresolved calls, semantic resource limits), or absence that already turns the result unknown (Wasm interface inventory, WebMCP discovery, managed ExportedType rows, legacy integrity mismatch IDs). Left counted: webCaptureDiff redirects (2; the CDP producer always records them, so absence is a legacy record that should compare as unknown, not as no redirects) and HopperCallerReconciliation (2; index-miss semantics not yet verified). Baseline 22 -> 4. * fix(browser): compare redirects as unknown when a capture predates redirect recording (#11402) * fix(browser): compare redirects as unknown when a capture predates redirect recording The CDP producer records every request's redirect chain, empty when there was none; a record without one predates redirect recording (#986). Comparing it against a current capture reported the network dimension as unchanged, treating the missing chain as "no redirects". Redirects are now compared only when both captures recorded them, and the network dimension is otherwise unknown with a reason; proven membership changes are still reported. * refactor(hopper): interpret a caller-index miss once at the lookup (#11403) The forward index holds observations from the examined procedures only, and relationship coverage stays unknown with the scan facts as its reason. A miss therefore adds no callers without showing there are none; the lookup now says so once instead of four optional reads defaulting at each site. The optional-fact-default baseline is now empty. * docs(javascript): state why an omitted representation list adds no nodes * fix(verify): count optional-fact defaults from parsed TypeScript, not comment-blanked text The guard blanked comments with a hand-written scanner and matched a regular expression on the rest, so a default quoted in a string counted and a regular expression holding /* hid the code after it. It now walks the TypeScript syntax tree for property reads defaulted with ?? 0, ?? [] or ?? false, and reads exemption markers only from comment trivia. * fix(verify): tokenize optional-fact defaults without a compiler and count chained defaults * fix(verify): match whole optional-fact default expressions across calls and operators * fix(verify): count optional-fact defaults in template substitutions and after postfix updates, and tie markers to their own expression * refactor(verify): exempt generation and sample caps from the constant-maximum-slice guard by reason marker (#11454) * refactor(verify): exempt generation and sample caps from the constant-maximum-slice guard by reason marker * fix(verify): read reason markers only from real line comments * fix(verify): read reason markers from the shared source tokenizer * fix(verify): read a regular expression after a control head as a statement start The shared tokenizer treated every closing parenthesis as the end of an operand, so in `if (ready) /[/*]/.test(v)` it read `/[` as division and `/*` inside the regex as a block comment, hiding the rest of the file from the marker-aware guards. Parentheses are now tracked: the `)` closing an if/while/for/with head allows a regular expression, while a call's `)` still means division. * fix(verify): keep a keyword-named member call out of control-head detection `policy.if(ready) / (…)` was read as an if-statement head, so the division was taken for a regular expression and the denominator's optional-fact default was not counted. A control keyword after `.` or `?.` is a member call. * refactor(verify): exempt budget-arithmetic cursors and tallies by a not-an-allowance marker (#11493) * refactor(verify): exempt budget-arithmetic cursors and tallies by a not-an-allowance marker The budget-arithmetic guard cannot tell a scan cursor, an observed total, a string being assembled or a tail buffer's retained length from an allowance when each is compared with a limit. A field declaration marked `// not-an-allowance: <reason>` (read through the shared reason-marker scan) is no longer counted. The four remaining sites carry a marker, so the baseline is empty and removed. * refactor(verify): exempt detail levels, lifecycle states and domain vocabularies from the completeness-enum guard by reason marker (#11510) A zod enum containing "complete" is counted as a local coverage vocabulary. Eleven are not coverage: a caller-chosen detail level, how far a command or session ran, and format states (GPU decode depth, WASM name section, SQLite column details). They now carry `// detail-level:`, `// lifecycle-state:` or `// domain-vocabulary:` with a reason, read through the shared reason-marker scan (which gains an optional predicate for the matches a guard counts). local-completeness-enum: 22 -> 11 on this branch. * fix(verify): bind a not-an-allowance marker to its own class field Exemptions were a file-wide set of names, so a marked tally in one class hid an unmarked allowance with the same name in another, and a marked object property could exempt a class field. Markers now exempt only a member declaration (at class member depth) of the top-level class that contains it, and the named- allowance count is taken per class so the same scoping applies to it. * fix(artifacts): keep main's resources import in the Deno standalone reader after merging main * fix(verify): start an expression after a template substitution and a non-member keyword, and accept any marker reason The shared tokenizer read the `/` after `${` as division, so a regular expression holding `/*` in a substitution hid the rest of the file from every tokenizer-based guard, and it treated property names such as `row.return` as keywords. Tokens now record whether template text opens a substitution and whether a name is an expression keyword (not after `.` or `?.`), and the regular-expression rule and the optional-fact-default operand rule read those facts. The optional-fact-default guard reads markers through reason-marker's isReasonMarker, so any non-empty reason exempts, as for the other reason markers. * fix(verify): blank comments with the shared tokenizer before matching architecture guards (#11573) * fix(verify): blank comments with the shared tokenizer before matching architecture guards The guards stripped comments with a regular expression that took a `/*` inside a string, template or regular expression for a comment opener and deleted code up to the next `*/`. src/flutter/FlutterBuildProvider.ts's "lib/*/libapp.so" text hid four --pool-policy remediation strings from cli-option-text-outside-cli. blankComments in source-tokens.mjs now owns comment removal for the architecture guards and the reason-marker scan: it blanks comment tokens in place (newlines kept) and keeps strings, templates and regular expressions. The scan behind it retains only the last tokens it reads, and whitespace runs are skipped at once. The cli-option-text-outside-cli baseline records FlutterBuildProvider.ts at its true count of 4; no other baseline changed. Fixes #11477 * fix(verify): read a regular expression after a statement block and keep comment offsets for wide characters The tokenizer now records whether each `{` opens a statement block or an object literal (from the token before it) and marks the `}` that closes a block, after which a statement, and so a regular expression, can start; an object literal's `}` still ends an operand. blankComments replaces each run of comment text with one space per UTF-16 unit, so a surrogate pair no longer shifts later offsets. * chore(verify): drop the Flutter CLI-text baseline entry now that main names no option there
621 lines
22 KiB
TypeScript
621 lines
22 KiB
TypeScript
import {
|
|
projectProtobufCapturePage,
|
|
protobufViewRequestSchema,
|
|
} from "./protobufCaptureView.js";
|
|
import { analysisViewInputError } from "./analysisViewErrors.js";
|
|
import { z } from "zod";
|
|
import type { AnalysisError } from "../analysisErrorBase.js";
|
|
import { parseAnalysisViewResult } from "./analysisViewParent.js";
|
|
import type { JsonValue } from "../jsonValue.js";
|
|
import { err, ok, type Result } from "../result.js";
|
|
import {
|
|
webNetworkCaptureRecordSchema,
|
|
webNetworkCaptureBinaryDescriptorSchema,
|
|
safeParseOwnedWebNetworkCapture,
|
|
webNetworkCaptureArtifactSchema,
|
|
webNetworkCaptureDecoderSchema,
|
|
protobufCaptureCoverageSchema,
|
|
WEB_NETWORK_CAPTURE_FORMATS,
|
|
type WebNetworkCaptureRecord,
|
|
type WebNetworkCapture,
|
|
} from "../webNetworkCapture.js";
|
|
import type { AnalysisViewParent } from "./analysisViewParent.js";
|
|
import {
|
|
assertOwnedView,
|
|
type AdmittedView,
|
|
type SummaryAnalysisView,
|
|
} from "./analysisViewShape.js";
|
|
import {
|
|
completeWithinViewCoverage,
|
|
pageViewCoverage,
|
|
} from "./analysisViewCoverage.js";
|
|
import {
|
|
qlogSummarySchema,
|
|
qlogViewRequestSchema,
|
|
summarizeQlogCapture,
|
|
projectQlogCapturePage,
|
|
} from "./qlogCaptureView.js";
|
|
import { evidenceFileTarget } from "../evidence.js";
|
|
import { sha256Hex } from "../../kernel/sha256Hex.js";
|
|
|
|
/** Offsets page the parent's retained record order, preserving producer ordinals. */
|
|
export const captureViewRequestSchema = z.strictObject({
|
|
kind: z.literal("capture"),
|
|
// detail-level: how much of the capture the caller asks to see, not what was examined.
|
|
detail: z.enum(["preview", "complete"]).default("preview"),
|
|
offset: z.int().nonnegative().default(0),
|
|
limit: z.int().positive().default(64),
|
|
});
|
|
|
|
const protobufSummarySchema = z.strictObject({
|
|
bodies: z.number().int().nonnegative(),
|
|
frames: z.number().int().nonnegative(),
|
|
fields: z.number().int().nonnegative(),
|
|
incomplete_bodies: z.number().int().nonnegative(),
|
|
});
|
|
const summarizeProtobuf = (records: readonly WebNetworkCaptureRecord[]) => {
|
|
const counts = { bodies: 0, frames: 0, fields: 0, incomplete_bodies: 0 };
|
|
for (const record of records)
|
|
for (const body of record.protobuf ?? []) {
|
|
counts.bodies++;
|
|
counts.frames += body.frames.length;
|
|
counts.fields += body.fields.length;
|
|
counts.incomplete_bodies += Number(body.status !== "complete");
|
|
}
|
|
return counts.bodies === 0 ? undefined : counts;
|
|
};
|
|
const previewSchema = webNetworkCaptureRecordSchema
|
|
.omit({ protobuf: true })
|
|
.extend({
|
|
binary_fields: z.array(webNetworkCaptureBinaryDescriptorSchema),
|
|
protobuf_summary: protobufSummarySchema.exactOptional(),
|
|
});
|
|
export const captureViewItemsSchema = z.array(
|
|
z.union([webNetworkCaptureRecordSchema, previewSchema]),
|
|
);
|
|
|
|
/** Count/identity overview; bodies and the archive member inventory stay in the parent. */
|
|
export const captureSummarySchema = z.strictObject({
|
|
protobuf_coverage: protobufCaptureCoverageSchema.exactOptional(),
|
|
artifact: webNetworkCaptureArtifactSchema,
|
|
decoder: webNetworkCaptureDecoderSchema,
|
|
format: z.enum(WEB_NETWORK_CAPTURE_FORMATS),
|
|
total_records: z.int().nonnegative(),
|
|
retained_records: z.int().nonnegative(),
|
|
runtime_attribution: z.literal("unknown"),
|
|
protobuf: protobufSummarySchema.exactOptional(),
|
|
qlog: qlogSummarySchema.optional(),
|
|
});
|
|
|
|
const object = (
|
|
value: JsonValue,
|
|
): Readonly<Record<string, JsonValue>> | null =>
|
|
value !== null && typeof value === "object" && !Array.isArray(value)
|
|
? value
|
|
: null;
|
|
|
|
// Field names have different meanings across producers. In particular, HAR
|
|
// entry extensions named `name` or `type` are not qlog or mitmproxy headlines.
|
|
const RECORD_HEADLINES: Readonly<Record<string, readonly string[]>> = {
|
|
har: ["time", "startedDateTime"],
|
|
qlog: ["name", "time"],
|
|
saz: ["session_id", "time"],
|
|
mitmproxy: ["type"],
|
|
};
|
|
const HTTP_HEADLINES: Readonly<
|
|
Record<string, Readonly<Record<string, readonly string[]>>>
|
|
> = {
|
|
har: {
|
|
request: ["method", "url", "httpVersion"],
|
|
response: ["httpVersion", "status", "statusText"],
|
|
},
|
|
saz: {
|
|
request: ["method", "url", "http_version"],
|
|
response: ["http_version", "status", "reason"],
|
|
},
|
|
mitmproxy: {
|
|
request: ["method", "http_version"],
|
|
response: ["http_version", "reason"],
|
|
},
|
|
};
|
|
const scalar = (value: JsonValue): boolean =>
|
|
value === null || typeof value !== "object";
|
|
// Native mitmproxy flows report the request target in parts and the status
|
|
// as status_code. They are previewed only for mitmproxy records and only when
|
|
// each value has its field's well-formed shape, which also bounds the
|
|
// serialized size of every record (at most ~4 KiB for an escape-heavy path).
|
|
// The path must be origin or asterisk form without control characters and
|
|
// without any "@": native redaction misses absolute and ambiguous spellings,
|
|
// and URL parsing can fail before it recognises their userinfo, so a preview
|
|
// fails closed on the marker every userinfo needs. Host and authority never
|
|
// carry a userinfo marker either.
|
|
// Target fields come only from the request and the status only from the
|
|
// response, and integer slots require the producer's own integer type.
|
|
// Excluded values stay in complete record pages.
|
|
const MITMPROXY_HEADLINE_FIELDS: Readonly<Record<string, readonly string[]>> = {
|
|
request: ["scheme", "host", "port", "authority", "path"],
|
|
response: ["status_code"],
|
|
};
|
|
const MITMPROXY_PATH_MAX_LENGTH = 2 * 1024;
|
|
const integerIn = (value: JsonValue, min: number, max: number): boolean =>
|
|
typeof value === "number" &&
|
|
Number.isInteger(value) &&
|
|
value >= min &&
|
|
value <= max;
|
|
/** True when a code unit is C0, DEL or (optionally) a space. */
|
|
const hasControl = (value: string, includeSpace: boolean): boolean => {
|
|
for (let index = 0; index < value.length; index += 1) {
|
|
const code = value.charCodeAt(index);
|
|
if (code < 0x20 || code === 0x7f || (includeSpace && code === 0x20))
|
|
return true;
|
|
}
|
|
return false;
|
|
};
|
|
// An unpaired surrogate has no UTF-8 form, so no producer reported it as text.
|
|
const LONE_SURROGATE = /\p{Cs}/u;
|
|
const hostLike = (value: JsonValue, maxLength: number): boolean =>
|
|
typeof value === "string" &&
|
|
value.length <= maxLength &&
|
|
!hasControl(value, true) &&
|
|
!/[@/\\]/u.test(value) &&
|
|
!LONE_SURROGATE.test(value);
|
|
type NumericSidecar = WebNetworkCaptureRecord["numeric_literals"][number];
|
|
type BinaryField = WebNetworkCaptureRecord["binary_fields"][number];
|
|
const MITMPROXY_INTEGER_FIELDS = new Set(["port", "status_code"]);
|
|
/** Retained native producer bytes whose content, length and digest are all the text's UTF-8. */
|
|
const describesText = (descriptor: BinaryField, text: string): boolean => {
|
|
if (
|
|
descriptor.representation !== "producer-bytes" ||
|
|
descriptor.state !== "retained"
|
|
)
|
|
return false;
|
|
// An unpaired surrogate has no UTF-8 form; Buffer.from would invent
|
|
// replacement-character bytes that no producer value could have had.
|
|
const utf8 = Buffer.from(text, "utf8");
|
|
return (
|
|
utf8.toString("utf8") === text &&
|
|
descriptor.bytes === utf8.length &&
|
|
descriptor.content_base64 === utf8.toString("base64") &&
|
|
descriptor.sha256 === sha256Hex(utf8)
|
|
);
|
|
};
|
|
/**
|
|
* Integer slots need exactly one integer sidecar whose literal is the value's
|
|
* canonical decimal, and no byte descriptor. String slots need no sidecar and
|
|
* at most one byte descriptor, which must describe the value's own UTF-8.
|
|
* Selecting a field also selects its sidecars, so this keeps every copied
|
|
* literal small and consistent.
|
|
*/
|
|
const representationMatches = (
|
|
field: string,
|
|
value: JsonValue,
|
|
sidecars: readonly NumericSidecar[],
|
|
bytes: readonly BinaryField[],
|
|
): boolean => {
|
|
if (MITMPROXY_INTEGER_FIELDS.has(field))
|
|
return (
|
|
bytes.length === 0 &&
|
|
typeof value === "number" &&
|
|
sidecars.length === 1 &&
|
|
sidecars[0]?.producer_type === "integer" &&
|
|
sidecars[0].literal === String(value)
|
|
);
|
|
if (sidecars.length !== 0 || bytes.length > 1) return false;
|
|
const [descriptor] = bytes;
|
|
return (
|
|
descriptor === undefined ||
|
|
(typeof value === "string" && describesText(descriptor, value))
|
|
);
|
|
};
|
|
const headlineShape = (field: string, value: JsonValue): boolean => {
|
|
switch (field) {
|
|
case "scheme":
|
|
return (
|
|
typeof value === "string" &&
|
|
/^[A-Za-z][A-Za-z0-9+.-]{0,31}$/u.test(value)
|
|
);
|
|
case "host":
|
|
return hostLike(value, 255);
|
|
case "authority":
|
|
return hostLike(value, 262);
|
|
case "port":
|
|
return integerIn(value, 0, 65_535);
|
|
case "status_code":
|
|
return integerIn(value, 100, 999);
|
|
case "path":
|
|
return (
|
|
typeof value === "string" &&
|
|
value.length <= MITMPROXY_PATH_MAX_LENGTH &&
|
|
!hasControl(value, false) &&
|
|
!value.includes("@") &&
|
|
!LONE_SURROGATE.test(value) &&
|
|
(value === "*" ||
|
|
(value.startsWith("/") &&
|
|
!value.startsWith("//") &&
|
|
!value.startsWith("/\\")))
|
|
);
|
|
default:
|
|
return false;
|
|
}
|
|
};
|
|
/** Strict ancestors of a JSON pointer, from the root down. */
|
|
const ancestorsOf = (pointer: string): string[] => {
|
|
const ancestors: string[] = [];
|
|
for (
|
|
let at = pointer.indexOf("/");
|
|
at !== -1;
|
|
at = pointer.indexOf("/", at + 1)
|
|
)
|
|
ancestors.push(pointer.slice(0, at));
|
|
return ancestors;
|
|
};
|
|
/** The only coordinates a mitmproxy headline check asks about. */
|
|
const HEADLINE_POINTERS: ReadonlySet<string> = new Set([
|
|
"/type",
|
|
...Object.entries(MITMPROXY_HEADLINE_FIELDS).flatMap(([side, fields]) =>
|
|
fields.map((field) => `/${side}/${field}`),
|
|
),
|
|
]);
|
|
const HEADLINE_CHILD_PREFIXES = [...HEADLINE_POINTERS].map(
|
|
(pointer) => [pointer, `${pointer}/`] as const,
|
|
);
|
|
/** Headline coordinates and their ancestors: the only pointers whose own facets matter. */
|
|
const INDEXED_POINTERS: ReadonlySet<string> = new Set(
|
|
[...HEADLINE_POINTERS].flatMap((pointer) => [
|
|
pointer,
|
|
...ancestorsOf(pointer),
|
|
]),
|
|
);
|
|
/** Mark the headline coordinates a facet lies beneath, without deriving strings from its pointer. */
|
|
const markContainers = (pointer: string, into: Set<string>): void => {
|
|
for (const [coordinate, prefix] of HEADLINE_CHILD_PREFIXES)
|
|
if (pointer.startsWith(prefix)) into.add(coordinate);
|
|
};
|
|
/** Two entries already decide every representation check, so keep no more. */
|
|
const keep = <T>(groups: Map<string, T[]>, pointer: string, facet: T): void => {
|
|
const group = groups.get(pointer);
|
|
if (group === undefined) groups.set(pointer, [facet]);
|
|
else if (group.length < 2) group.push(facet);
|
|
};
|
|
|
|
/** A record's facets at headline coordinates and their ancestors, indexed once. */
|
|
type FacetIndex = {
|
|
/** At most two facets per pointer; a second already fails every check. */
|
|
readonly sidecars: ReadonlyMap<string, readonly NumericSidecar[]>;
|
|
readonly bytes: ReadonlyMap<string, readonly BinaryField[]>;
|
|
/** The producer held a number or bytes here. */
|
|
readonly scalars: ReadonlySet<string>;
|
|
/** Headline coordinates with a facet beneath: the producer held a container there. */
|
|
readonly containers: ReadonlySet<string>;
|
|
/**
|
|
* Withheld by a value redaction or a redacted byte descriptor; either facet
|
|
* withholds the value even when the other one is missing.
|
|
*/
|
|
readonly withheld: ReadonlySet<string>;
|
|
/** The parent of an omitted property name; it hides no sibling values. */
|
|
readonly omittedNames: ReadonlySet<string>;
|
|
};
|
|
/**
|
|
* Facets elsewhere in the record only mark containers, so the index holds a
|
|
* fixed number of entries however many facets an admitted record carries.
|
|
*/
|
|
const indexFacets = (record: WebNetworkCaptureRecord): FacetIndex => {
|
|
const sidecars = new Map<string, NumericSidecar[]>();
|
|
const bytes = new Map<string, BinaryField[]>();
|
|
const containers = new Set<string>();
|
|
const withheld = new Set<string>();
|
|
const omittedNames = new Set<string>();
|
|
const indexed = (pointer: string): boolean => {
|
|
markContainers(pointer, containers);
|
|
return INDEXED_POINTERS.has(pointer);
|
|
};
|
|
for (const sidecar of record.numeric_literals)
|
|
if (indexed(sidecar.pointer)) keep(sidecars, sidecar.pointer, sidecar);
|
|
for (const field of record.binary_fields)
|
|
if (indexed(field.pointer)) {
|
|
keep(bytes, field.pointer, field);
|
|
if (field.state === "redacted") withheld.add(field.pointer);
|
|
}
|
|
for (const redaction of record.redactions)
|
|
if (indexed(redaction.pointer))
|
|
("scope" in redaction ? omittedNames : withheld).add(redaction.pointer);
|
|
return {
|
|
sidecars,
|
|
bytes,
|
|
scalars: new Set([...sidecars.keys(), ...bytes.keys()]),
|
|
containers,
|
|
withheld,
|
|
omittedNames,
|
|
};
|
|
};
|
|
/**
|
|
* A value the record marks as redacted, or that lies under a redacted value,
|
|
* is never promoted even if it is still present (imported or caller-supplied
|
|
* Evidence). A scalar is also inconsistent when a numeric sidecar or byte
|
|
* descriptor sits on an ancestor (the producer held a scalar there) or any
|
|
* facet sits beneath it (the producer held a container at it).
|
|
*/
|
|
const usableScalar = (index: FacetIndex, pointer: string): boolean => {
|
|
const ancestors = ancestorsOf(pointer);
|
|
return (
|
|
!index.omittedNames.has(pointer) &&
|
|
!index.containers.has(pointer) &&
|
|
!index.withheld.has(pointer) &&
|
|
!ancestors.some(
|
|
(ancestor) => index.withheld.has(ancestor) || index.scalars.has(ancestor),
|
|
)
|
|
);
|
|
};
|
|
/**
|
|
* The container format alone does not make a flow HTTP; its own type does,
|
|
* when the producer supplied that type as text rather than a number or bytes.
|
|
*/
|
|
const textualFlowType = (index: FacetIndex): boolean =>
|
|
usableScalar(index, "/type") && !index.scalars.has("/type");
|
|
/** Shape checks run first, so the representation check only sees bounded values. */
|
|
const mitmproxyHeadline = (
|
|
index: FacetIndex,
|
|
pointer: string,
|
|
field: string,
|
|
value: JsonValue,
|
|
): boolean =>
|
|
usableScalar(index, pointer) &&
|
|
headlineShape(field, value) &&
|
|
representationMatches(
|
|
field,
|
|
value,
|
|
index.sidecars.get(pointer) ?? [],
|
|
index.bytes.get(pointer) ?? [],
|
|
);
|
|
|
|
/** Copy one message side's headlines; mitmproxy fields need the record's facet index. */
|
|
const selectHeadlines = (
|
|
fields: Readonly<Record<string, JsonValue>>,
|
|
key: string,
|
|
format: string,
|
|
facets: FacetIndex | null,
|
|
selectedPointers: Set<string>,
|
|
): Record<string, JsonValue> => {
|
|
const selected: Record<string, JsonValue> = {};
|
|
const candidates = [
|
|
...(HTTP_HEADLINES[format]?.[key] ?? []).map(
|
|
(field) => [field, null] as const,
|
|
),
|
|
...(facets === null ? [] : (MITMPROXY_HEADLINE_FIELDS[key] ?? [])).map(
|
|
(field) => [field, facets] as const,
|
|
),
|
|
];
|
|
for (const [field, index] of candidates) {
|
|
const value = fields[field];
|
|
if (
|
|
!Object.hasOwn(fields, field) ||
|
|
value === undefined ||
|
|
!scalar(value) ||
|
|
(index !== null &&
|
|
!mitmproxyHeadline(index, `/${key}/${field}`, field, value))
|
|
)
|
|
continue;
|
|
selected[field] = value;
|
|
selectedPointers.add(`/${key}/${field}`);
|
|
}
|
|
return selected;
|
|
};
|
|
|
|
/** Select already parsed producer facts without interpreting HTTP or payload bytes again. */
|
|
const preview = (record: WebNetworkCaptureRecord, format: string) => {
|
|
const source = object(record.reported);
|
|
const reported: Record<string, JsonValue> = {};
|
|
const selectedPointers = new Set<string>();
|
|
if (source !== null) {
|
|
const index =
|
|
format === "mitmproxy" && source.type === "http"
|
|
? indexFacets(record)
|
|
: null;
|
|
const mitmproxyFacets =
|
|
index !== null && textualFlowType(index) ? index : null;
|
|
for (const key of RECORD_HEADLINES[format] ?? [])
|
|
if (
|
|
Object.hasOwn(source, key) &&
|
|
source[key] !== undefined &&
|
|
scalar(source[key])
|
|
) {
|
|
reported[key] = source[key];
|
|
selectedPointers.add(`/${key}`);
|
|
}
|
|
for (const key of Object.keys(HTTP_HEADLINES[format] ?? {})) {
|
|
const value = source[key];
|
|
if (!Object.hasOwn(source, key) || value === undefined) continue;
|
|
const fields = object(value);
|
|
if (fields === null) {
|
|
// Preserve a reported null/scalar instead of inventing HTTP fields.
|
|
reported[key] = value;
|
|
continue;
|
|
}
|
|
reported[key] = selectHeadlines(
|
|
fields,
|
|
key,
|
|
format,
|
|
mitmproxyFacets,
|
|
selectedPointers,
|
|
);
|
|
}
|
|
}
|
|
const { protobuf: _protobuf, ...sourceRecord } = record;
|
|
const protobufSummary = summarizeProtobuf([record]);
|
|
return {
|
|
...sourceRecord,
|
|
...(protobufSummary === undefined
|
|
? {}
|
|
: { protobuf_summary: protobufSummary }),
|
|
reported: source === null ? record.reported : reported,
|
|
binary_fields: record.binary_fields.map(
|
|
({ content_base64: _content, ...field }) => field,
|
|
),
|
|
numeric_literals:
|
|
source === null
|
|
? record.numeric_literals
|
|
: record.numeric_literals.filter(({ pointer }) =>
|
|
selectedPointers.has(pointer),
|
|
),
|
|
};
|
|
};
|
|
|
|
const captureParser = { safeParse: safeParseOwnedWebNetworkCapture };
|
|
|
|
/** Authenticate the retained report and its subject at one owning boundary. */
|
|
const parseCaptureParent = (
|
|
parent: AnalysisViewParent,
|
|
): Result<WebNetworkCapture, AnalysisError> => {
|
|
const parsed = parseAnalysisViewResult(captureParser, parent);
|
|
if (!parsed.ok) return parsed;
|
|
const report = parsed.value;
|
|
const subject = evidenceFileTarget(parent);
|
|
if (
|
|
report.artifact.sha256 !== null &&
|
|
(subject == null || subject.sha256 !== report.artifact.sha256)
|
|
)
|
|
return err(
|
|
analysisViewInputError([
|
|
{
|
|
path: ["source", "subject"],
|
|
reason: "invalid_value",
|
|
message: "Capture parent subject must identify the report's SHA-256.",
|
|
},
|
|
]),
|
|
);
|
|
return ok(report);
|
|
};
|
|
|
|
/** Page immutable retained facts; no capture file, provider or network is consulted. */
|
|
export const projectWebNetworkCaptureView = (
|
|
parent: AnalysisViewParent,
|
|
view: AdmittedView,
|
|
) => {
|
|
if (
|
|
view.kind !== "summary" &&
|
|
view.kind !== "capture" &&
|
|
view.kind !== "protobuf" &&
|
|
view.kind !== "qlog"
|
|
)
|
|
return err(
|
|
analysisViewInputError([
|
|
{
|
|
path: ["view"],
|
|
reason: "invalid_value",
|
|
message:
|
|
"Historical captures support summary, capture record pages, qlog fact pages and admitted api-surface, environment-manifest, environment-state or environment-coverage views.",
|
|
},
|
|
]),
|
|
);
|
|
assertOwnedView<
|
|
| SummaryAnalysisView
|
|
| z.output<typeof captureViewRequestSchema>
|
|
| z.output<typeof protobufViewRequestSchema>
|
|
| z.output<typeof qlogViewRequestSchema>
|
|
>(view, "summary", "capture", "protobuf", "qlog");
|
|
const parsed = parseCaptureParent(parent);
|
|
if (!parsed.ok) return parsed;
|
|
const report = parsed.value;
|
|
const shared = {
|
|
parent_evidence_id: parent.evidenceId,
|
|
parent_operation: "inspect_web_network_capture" as const,
|
|
parent_digest: parent.evidenceId.slice(3),
|
|
artifact: { path: report.artifact.path, sha256: report.artifact.sha256 },
|
|
limitations: [
|
|
...new Set([
|
|
...parent.limitations,
|
|
...report.limitations,
|
|
"Capture record pages cover the records selected into the parent Evidence, in that selected order; total_records counts the original decoded capture.",
|
|
]),
|
|
],
|
|
unknowns: [
|
|
"Historical records do not establish live runtime attribution or deployment authenticity.",
|
|
...(report.artifact.sha256 === null
|
|
? ["The explicitly excluded artifact digest remains unknown."]
|
|
: []),
|
|
],
|
|
};
|
|
if (view.kind === "protobuf")
|
|
return ok({
|
|
...shared,
|
|
...projectProtobufCapturePage(report, view),
|
|
unknowns: [
|
|
...shared.unknowns,
|
|
"Decoding covers selected retained bodies only. Exhausting wire fields does not establish schema completeness or business meaning; compressed bodies and unsupported schema facets remain unknown.",
|
|
],
|
|
});
|
|
if (view.kind === "summary")
|
|
return ok({
|
|
kind: "summary" as const,
|
|
view,
|
|
...shared,
|
|
summary: {
|
|
artifact: report.artifact,
|
|
decoder: report.decoder,
|
|
format: report.format,
|
|
total_records: report.total_records,
|
|
retained_records: report.records.length,
|
|
runtime_attribution: report.runtime_attribution,
|
|
...(report.protobuf_coverage === undefined
|
|
? {}
|
|
: { protobuf_coverage: report.protobuf_coverage }),
|
|
...(summarizeProtobuf(report.records) === undefined
|
|
? {}
|
|
: { protobuf: summarizeProtobuf(report.records)! }),
|
|
...(report.format === "qlog"
|
|
? { qlog: summarizeQlogCapture(report.qlog) }
|
|
: {}),
|
|
},
|
|
coverage: completeWithinViewCoverage(report.records.length),
|
|
});
|
|
if (view.kind === "qlog") {
|
|
if (report.format !== "qlog")
|
|
return err(
|
|
analysisViewInputError([
|
|
{
|
|
path: ["view", "kind"],
|
|
reason: "invalid_value",
|
|
message: "Qlog fact views require retained qlog Evidence.",
|
|
},
|
|
]),
|
|
);
|
|
const projected = projectQlogCapturePage(report.qlog, view);
|
|
return projected.ok
|
|
? ok({
|
|
kind: "qlog" as const,
|
|
view,
|
|
...shared,
|
|
...projected.value,
|
|
limitations: [
|
|
...shared.limitations,
|
|
"Qlog fact pages retain all-recorded-events-before-selection scope. Exhausting a page does not establish network completeness, body recovery, retransmission or actual loss.",
|
|
],
|
|
})
|
|
: projected;
|
|
}
|
|
const records = report.records.slice(view.offset, view.offset + view.limit);
|
|
return ok({
|
|
kind: "capture" as const,
|
|
view,
|
|
...shared,
|
|
items: captureViewItemsSchema.parse(
|
|
view.detail === "complete"
|
|
? records
|
|
: records.map((record) => preview(record, report.format)),
|
|
),
|
|
coverage: pageViewCoverage(
|
|
view.offset,
|
|
records.length,
|
|
report.records.length,
|
|
),
|
|
limitations: [
|
|
...shared.limitations,
|
|
...(view.detail === "preview"
|
|
? [
|
|
"Preview selects recorded headline fields and byte descriptors; headers, XML, bodies, message data and other fields remain in complete record pages. Missing preview fields do not establish their absence from the producer record.",
|
|
]
|
|
: []),
|
|
],
|
|
});
|
|
};
|