Files
reverse-engineering-anything/src/application/environment/environmentCompare.ts
T
morluto 332dea76a6 fix(native): keep unreported value-flow and archive coverage unknown, and exempt generation caps by reason marker (#11288)
* fix(native): keep unreported value-flow truncation and archive decode coverage unknown

* test(verify): count multiline optional-fact defaults and pin the guard's exclusions

* fix(domain): keep unknown counts and costs unknown in comparison and coverage summaries (#11325)

* fix(domain): keep unknown counts and costs unknown in comparison and coverage summaries

* refactor(verify): exempt legitimate optional-fact defaults with a reasoned marker (#11355)

* refactor(verify): exempt legitimate optional-fact defaults with a reasoned marker

The optional-fact-default guard counted request defaults and absence that
is modelled elsewhere as hidden unknowns. A comment
`// optional-fact-default: <reason>` on the default's line or the line
before now exempts it; the guard reads unstripped source for the marker and
ignores matches inside comment lines.

Marked: the two evidence paging offsets and four Swift literal reads whose
absence swiftLiteralCoverage reports. Removed instead of marked: jsdiff 9
types Change.count as required, so its two `?? 0` defaults were dead; the
evidence link traversal now carries retained records, so it no longer
re-looks them up behind `?? []`. Baseline 31 -> 22.

* fix(verify): blank comment spans instead of skipping comment-led lines in the optional-fact guard

Skipping a match whose line starts like a comment hid live code after a
leading block comment and counted defaults inside a block comment without
leading stars. The counter now blanks comment text (keeping offsets and line
breaks, skipping string literals) and matches the blanked code, while
exemption markers are still read from the original lines.

* refactor(domain): mark optional-fact defaults whose absence is not an observation (#11389)

* refactor(domain): mark optional-fact defaults whose absence is not an observation

Per-site review of the remaining optional-fact-default entries. Each marked
default is a caller option (trace ignore_fields, environment rules), a list
whose producer omits the empty case (unknown_size_frames, the schema's
min-1 representation_node_ids), caveat text whose completeness another field
carries (call diagnostics, unresolved calls, semantic resource limits), or
absence that already turns the result unknown (Wasm interface inventory,
WebMCP discovery, managed ExportedType rows, legacy integrity mismatch IDs).

Left counted: webCaptureDiff redirects (2; the CDP producer always records
them, so absence is a legacy record that should compare as unknown, not
as no redirects) and HopperCallerReconciliation (2; index-miss semantics not
yet verified). Baseline 22 -> 4.

* fix(browser): compare redirects as unknown when a capture predates redirect recording (#11402)

* fix(browser): compare redirects as unknown when a capture predates redirect recording

The CDP producer records every request's redirect chain, empty when there was none; a record without one predates redirect recording (#986). Comparing it against a current capture reported the network dimension as unchanged, treating the missing chain as "no redirects". Redirects are now compared only when both captures recorded them, and the network dimension is otherwise unknown with a reason; proven membership changes are still reported.

* refactor(hopper): interpret a caller-index miss once at the lookup (#11403)

The forward index holds observations from the examined procedures only, and relationship coverage stays unknown with the scan facts as its reason. A miss therefore adds no callers without showing there are none; the lookup now says so once instead of four optional reads defaulting at each site. The optional-fact-default baseline is now empty.

* docs(javascript): state why an omitted representation list adds no nodes

* fix(verify): count optional-fact defaults from parsed TypeScript, not comment-blanked text

The guard blanked comments with a hand-written scanner and matched a regular expression on the rest, so a default quoted in a string counted and a regular expression holding /* hid the code after it. It now walks the TypeScript syntax tree for property reads defaulted with ?? 0, ?? [] or ?? false, and reads exemption markers only from comment trivia.

* fix(verify): tokenize optional-fact defaults without a compiler and count chained defaults

* fix(verify): match whole optional-fact default expressions across calls and operators

* fix(verify): count optional-fact defaults in template substitutions and after postfix updates, and tie markers to their own expression

* refactor(verify): exempt generation and sample caps from the constant-maximum-slice guard by reason marker (#11454)

* refactor(verify): exempt generation and sample caps from the constant-maximum-slice guard by reason marker

* fix(verify): read reason markers only from real line comments

* fix(verify): read reason markers from the shared source tokenizer

* fix(verify): read a regular expression after a control head as a statement start

The shared tokenizer treated every closing parenthesis as the end of an operand,
so in `if (ready) /[/*]/.test(v)` it read `/[` as division and `/*` inside the
regex as a block comment, hiding the rest of the file from the marker-aware
guards. Parentheses are now tracked: the `)` closing an if/while/for/with head
allows a regular expression, while a call's `)` still means division.

* fix(verify): keep a keyword-named member call out of control-head detection

`policy.if(ready) / (…)` was read as an if-statement head, so the division was
taken for a regular expression and the denominator's optional-fact default was
not counted. A control keyword after `.` or `?.` is a member call.

* refactor(verify): exempt budget-arithmetic cursors and tallies by a not-an-allowance marker (#11493)

* refactor(verify): exempt budget-arithmetic cursors and tallies by a not-an-allowance marker

The budget-arithmetic guard cannot tell a scan cursor, an observed total, a
string being assembled or a tail buffer's retained length from an allowance
when each is compared with a limit. A field declaration marked
`// not-an-allowance: <reason>` (read through the shared reason-marker scan)
is no longer counted. The four remaining sites carry a marker, so the
baseline is empty and removed.

* refactor(verify): exempt detail levels, lifecycle states and domain vocabularies from the completeness-enum guard by reason marker (#11510)

A zod enum containing "complete" is counted as a local coverage vocabulary. Eleven
are not coverage: a caller-chosen detail level, how far a command or session
ran, and format states (GPU decode depth, WASM name section, SQLite column
details). They now carry `// detail-level:`, `// lifecycle-state:` or
`// domain-vocabulary:` with a reason, read through the shared reason-marker
scan (which gains an optional predicate for the matches a guard counts).
local-completeness-enum: 22 -> 11 on this branch.

* fix(verify): bind a not-an-allowance marker to its own class field

Exemptions were a file-wide set of names, so a marked tally in one class hid an
unmarked allowance with the same name in another, and a marked object property
could exempt a class field. Markers now exempt only a member declaration (at
class member depth) of the top-level class that contains it, and the named-
allowance count is taken per class so the same scoping applies to it.

* fix(artifacts): keep main's resources import in the Deno standalone reader after merging main

* fix(verify): start an expression after a template substitution and a non-member keyword, and accept any marker reason

The shared tokenizer read the `/` after `${` as division, so a regular
expression holding `/*` in a substitution hid the rest of the file from
every tokenizer-based guard, and it treated property names such as
`row.return` as keywords. Tokens now record whether template text opens a
substitution and whether a name is an expression keyword (not after `.` or
`?.`), and the regular-expression rule and the optional-fact-default operand
rule read those facts. The optional-fact-default guard reads markers through
reason-marker's isReasonMarker, so any non-empty reason exempts, as for the
other reason markers.

* fix(verify): blank comments with the shared tokenizer before matching architecture guards (#11573)

* fix(verify): blank comments with the shared tokenizer before matching architecture guards

The guards stripped comments with a regular expression that took a `/*`
inside a string, template or regular expression for a comment opener and
deleted code up to the next `*/`. src/flutter/FlutterBuildProvider.ts's
"lib/*/libapp.so" text hid four --pool-policy remediation strings from
cli-option-text-outside-cli. blankComments in source-tokens.mjs now owns
comment removal for the architecture guards and the reason-marker scan: it
blanks comment tokens in place (newlines kept) and keeps strings, templates
and regular expressions. The scan behind it retains only the last tokens it
reads, and whitespace runs are skipped at once.

The cli-option-text-outside-cli baseline records FlutterBuildProvider.ts at
its true count of 4; no other baseline changed.

Fixes #11477

* fix(verify): read a regular expression after a statement block and keep comment offsets for wide characters

The tokenizer now records whether each `{` opens a statement block or an
object literal (from the token before it) and marks the `}` that closes a
block, after which a statement, and so a regular expression, can start; an
object literal's `}` still ends an operand. blankComments replaces each run
of comment text with one space per UTF-16 unit, so a surrogate pair no longer
shifts later offsets.

* chore(verify): drop the Flutter CLI-text baseline entry now that main names no option there
2026-11-16 07:40:30 +08:00

577 lines
20 KiB
TypeScript

import { boundedList } from "../../domain/boundedList.js";
import {
ownMember,
decodedSegment,
same,
} from "../../domain/analysisView/environmentStateRules.js";
import type { FieldRule } from "../../domain/analysisView/environmentStateWriteRules.js";
import { fromJsonSchema } from "@modelcontextprotocol/server";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { JsonValue } from "../../domain/jsonValue.js";
import { err, ok, type Result } from "../../domain/result.js";
import {
compareAnswers,
DIFFERENCE_CATEGORIES,
STEP_DIFFERENCE_LIMIT,
type Answer,
type Difference,
type DifferenceCategory,
type Normalised,
} from "./environmentCompareDiff.js";
import {
environmentArguments,
IdMap,
learnIds,
translatePath,
} from "./environmentCompareIds.js";
import {
bulkIds,
collectionScope,
literalPositions,
bodyTools,
matchProbeRequest,
probeArguments,
parameterScopes,
type ProbeToolCall,
} from "./environmentCompareMatch.js";
import {
probeRequests,
requestBody,
sendsBodies,
type ProbeStep,
type UnobservedRecord,
} from "./environmentCompareProbe.js";
import {
inputError,
parseCaptureEvidence,
replayedBody,
type EnvironmentManifestTool,
type EnvironmentSource,
type ReplayedBody,
} from "./environmentSource.js";
import {
selectsRecordedLine,
type ReplayArguments,
type ReplayResult,
} from "./environmentReplay.js";
import { prepareServedState, servedCatalog } from "./environmentState.js";
import type { StateAnswer } from "./environmentStateAnswer.js";
import { EnvironmentStateStore } from "./environmentStateStore.js";
import {
environmentToolAnswer,
type EnvironmentToolAnswer,
} from "./environmentToolAnswer.js";
import type { EnvironmentToolDefinition } from "./environmentTools.js";
/** One probe capture file: its label for the report and its parsed JSON. */
export interface CompareProbeInput {
readonly label: string;
readonly value: unknown;
}
/** One probe request, how the environment answered it, and how that differs from the application. */
export interface CompareStep {
readonly ordinal: number;
readonly method: string;
readonly path: string;
readonly tool: string | null;
readonly live_status: number | null;
readonly env_status: number | null;
readonly equal: boolean;
/** Equal but for item order, which an application that returns sets in no fixed order cannot reproduce either. */
readonly equal_ignoring_order: boolean;
readonly categories: DifferenceCategory[];
readonly differences: Difference[];
/** All differences computed for this step, before the returned list is capped. */
readonly differences_total: number;
readonly differences_omitted: number;
readonly differences_truncated: boolean;
readonly ignored_query: string[];
}
export interface CompareProbeReport {
readonly label: string;
readonly evidence_id: string;
readonly steps: number;
readonly equal: number;
readonly ids_mapped: number;
readonly id_conflicts: number;
readonly normalised: Normalised;
readonly results: CompareStep[];
/** Records left uncompared, in probe order, each with its reason. */
readonly unobserved: UnobservedRecord[];
}
export interface CompareReport {
readonly status: "complete";
readonly environment: {
readonly name: string;
readonly manifest_evidence_id: string;
readonly evidence_id: string;
readonly state: "served" | "replay-only";
};
readonly summary: {
readonly probes: number;
readonly steps: number;
readonly equal: number;
readonly agreement: number | null;
readonly agreement_ignoring_order: number | null;
/** Probe records left uncompared, outside the steps agreement counts. */
readonly unobserved: number;
readonly categories: Record<DifferenceCategory, number>;
readonly normalised: Normalised;
};
readonly tools: { tool: string; steps: number; equal: number }[];
readonly probes: CompareProbeReport[];
}
const NORMALISED_KEYS = [
"ids",
"clocks",
"generated",
"large_bodies",
"unobserved_bodies",
] as const;
const emptyNormalised = (): Normalised => ({
ids: 0,
clocks: 0,
generated: 0,
large_bodies: 0,
unobserved_bodies: 0,
});
const isSuccess = (status: number | null): boolean =>
status !== null && status >= 200 && status < 300;
/** Where a call's answer holds its own entities, and the collection they belong to: a read view's entity or list pointer, a write's response pointer, else the root. A write that answers with only the new id learns that scalar. */
const entityOf = (
run: ProbeRun,
tool: EnvironmentManifestTool,
segments: readonly string[],
answer: ReplayResult,
): {
readonly scope: string;
readonly at: string;
readonly identity: boolean;
readonly rules: readonly FieldRule[];
} => {
const view = run.store?.state.views.get(tool.name);
const plan = run.store?.state.writes.get(tool.name);
const response = plan?.response;
const scope = collectionScope(tool, segments);
if (view?.shape === "item" || view?.shape === "list")
return { scope, at: view.pointer, identity: false, rules: [] };
const identity = response?.shape === "identity";
const at = identity
? (response.pointer ?? "")
: (response?.pointer ?? plan?.observation?.pointer ?? "");
const provenance = (answer as Partial<StateAnswer>).state;
const renderedWrite =
provenance?.source === "state" &&
(provenance.view === "create" ||
provenance.view === "update" ||
provenance.view === "document-write" ||
provenance.view === "delete");
// A missing target replays a recorded absence with state provenance too;
// only an applied write can render the plan's entity response.
const rules =
plan?.served === true && response?.shape === "entity" && renderedWrite
? plan.rules
: [];
return { scope, at, identity, rules };
};
const fromState = (answer: ReplayResult): boolean => {
const state = (answer as Partial<StateAnswer>).state;
// Missing-item selection comes from state, but its response is a recording.
return state?.source === "state" && state.view !== "missing";
};
/** Identity facts come from decoded JSON or text scalars, never binary presentations or truncated bodies. */
const identityValue = (body: ReplayedBody): JsonValue | undefined => {
if (body.kind === "json") return body.value;
if (body.kind === "text") return body.text;
return undefined;
};
/** The environment's body as the application's is read: a replayed recording of empty text, unlike a JSON empty string, is no body. */
const envBody = (run: ProbeRun, answer: ReplayResult): ReplayedBody => {
if (fromState(answer)) return { kind: "json", value: answer.body };
const exchange = run.source.exchanges.get(answer.replay.record_ordinal);
if (exchange === undefined)
return answer.body === ""
? { kind: "none" }
: { kind: "json", value: answer.body };
const body = replayedBody(exchange);
// Empty replayed text is no body; a JSON empty string remains a JSON value.
if (body.kind === "text" && body.text === "") return { kind: "none" };
if (body.kind === "json") return { kind: "json", value: answer.body };
if (body.kind === "text")
return {
kind: "text",
text: typeof answer.body === "string" ? answer.body : body.text,
};
if (body.kind === "base64")
return {
...body,
encoded: typeof answer.body === "string" ? answer.body : body.encoded,
whole: body.whole && !answer.replay.truncated,
};
if (body.kind === "cut")
return {
kind: "cut",
text: typeof answer.body === "string" ? answer.body : body.text,
};
return body;
};
/**
* Whether the environment answered this call itself: from its state, or from a recording of exactly this call, every
* recorded query value supplied and the body it sent, rather than replaying another call's recording.
*/
const answersCall = (
run: ProbeRun,
call: ProbeToolCall,
args: Readonly<Record<string, JsonValue>>,
answer: ReplayResult,
): boolean => {
if (fromState(answer)) return true;
const exchange = run.source.exchanges.get(answer.replay.record_ordinal);
if (
!answer.replay.exact_match ||
exchange === undefined ||
!selectsRecordedLine(call.tool, exchange, args)
)
return false;
const body = probeArguments(call.tool, run.bodies).body;
// A tool with no body parameter matches; one that takes a body matches only the body this call sent, including none.
if (body === undefined) return true;
return same(run.recordedBodies.get(exchange.ordinal), ownMember(args, body));
};
/** A served tool's input check: the first problem with a call's arguments, or undefined. */
type InputCheck = (args: ReplayArguments) => string | undefined;
/** The input schema check the MCP server applies before a served tool answers, so a call it refuses is not compared as answered. */
const inputCheck = (definition: EnvironmentToolDefinition): InputCheck => {
const schema = fromJsonSchema<ReplayArguments>(definition.inputSchema)[
"~standard"
];
return (args) => {
const checked = schema.validate(args);
return checked instanceof Promise
? undefined
: checked.issues?.[0]?.message;
};
};
interface ProbeRun {
readonly source: EnvironmentSource;
/** The request body of each capture record, by ordinal, that a replayed answer's call is compared with. */
readonly recordedBodies: ReadonlyMap<number, JsonValue | undefined>;
readonly answer: EnvironmentToolAnswer;
readonly store: EnvironmentStateStore | undefined;
/** The tools the environment offers, by name, with their input checks. */
readonly served: ReadonlyMap<string, InputCheck>;
/** The served tools that take a body their method does not, as a bulk delete names its entities. */
readonly bodies: ReadonlySet<string>;
readonly map: IdMap;
readonly normalised: Normalised;
}
const heldIds = (
store: EnvironmentStateStore | undefined,
): ReadonlySet<string> =>
new Set(
store?.snapshot().rea_state.entities.map((entity) => entity.id) ?? [],
);
const stepOf = (
base: Pick<CompareStep, "ordinal" | "method" | "path" | "live_status">,
call: ProbeToolCall | undefined,
outcome: { env_status: number | null; differences: Difference[] },
): CompareStep => {
const categories = [...new Set(outcome.differences.map((d) => d.category))];
const { items: differences, omitted } = boundedList(
outcome.differences,
STEP_DIFFERENCE_LIMIT,
);
return {
...base,
tool: call?.tool.name ?? null,
env_status: outcome.env_status,
equal: outcome.differences.length === 0,
equal_ignoring_order: outcome.differences.every(
(difference) => difference.category === "order",
),
categories,
differences: [...differences],
differences_total: outcome.differences.length,
differences_omitted: omitted,
differences_truncated: omitted > 0,
// optional-fact-default: with no matched call (tool null) no query was mapped.
ignored_query: [...(call?.ignoredQuery ?? [])],
};
};
/** Answer one probe request as the environment would and compare it with the application's answer; one without an answer is only made. */
const compareStep = (
run: ProbeRun,
manifest: EnvironmentSource["manifest"],
probe: ProbeStep,
): CompareStep | undefined => {
const { request, live } = probe;
const base = {
ordinal: probe.ordinal,
method: request.method,
path: request.path,
live_status: live?.status ?? null,
};
const report = (
call: ProbeToolCall | undefined,
outcome: { env_status: number | null; differences: Difference[] },
): CompareStep | undefined =>
live === undefined ? undefined : stepOf(base, call, outcome);
// Matched in the environment's terms, so a tool whose path holds a seeded id literally still matches. Scopes and
// literals come from the live path: translating a parent id first would name a different collection than learning did.
const liveSegments = request.path.slice(1).split("/");
const fitted = matchProbeRequest(manifest, request, run.bodies);
const scopes =
fitted === undefined
? undefined
: parameterScopes(fitted.tool, liveSegments);
const literals =
fitted === undefined ? undefined : literalPositions(fitted.tool);
const path = translatePath(
request.path,
run.map,
(index) => scopes?.get(index),
(index) => literals?.has(index) === true,
);
const call = matchProbeRequest(manifest, { ...request, path }, run.bodies);
if (call === undefined)
return report(call, {
env_status: null,
differences: [{ category: "no-tool", pointer: "" }],
});
const check = run.served.get(call.tool.name);
if (check === undefined)
return report(call, {
env_status: null,
differences: [{ category: "not-served", pointer: "" }],
});
const held = heldIds(run.store);
const unknownPath = call.pathArguments.some((name) => {
const value = ownMember(call.args, name);
if (typeof value !== "string") return false;
const id = decodedSegment(value);
return !run.map.hasEnv(id) && !held.has(id);
});
const refusedWith = (message: string): CompareStep | undefined =>
report(call, {
env_status: null,
differences: [
{ category: "refused", pointer: "", env: message.slice(0, 200) },
],
});
const bulk = bulkIds(run, call, liveSegments);
const args = environmentArguments(call, run.map, bulk);
const problem = check(args);
if (problem !== undefined)
return refusedWith(`Invalid arguments: ${problem}`);
const answered = run.answer(call.tool, args, call.observedQuery);
if (!answered.ok) return refusedWith(answered.error.message);
if (live === undefined) return undefined;
const env: Answer = {
status: answered.value.status,
body: envBody(run, answered.value),
};
const entity = entityOf(run, call.tool, liveSegments, answered.value);
// Another call's recording holds other entities, whose ids would map wrongly and, the first pairing winning, for good.
if (answersCall(run, call, args, answered.value))
learnIds(identityValue(live.body), identityValue(env.body), run.map, {
heldIds: heldIds(run.store),
scope: entity.scope,
entityAt: entity.at,
identity: entity.identity,
});
const compared = compareAnswers(live, env, run.map, entity);
for (const key of NORMALISED_KEYS)
run.normalised[key] += compared.normalised[key];
const differences = [...compared.differences];
if (unknownPath && isSuccess(live.status) && !isSuccess(env.status))
differences.unshift({ category: "unmapped-id", pointer: "" });
return stepOf(base, call, { env_status: env.status, differences });
};
const probeReport = (
probe: { readonly label: string; readonly evidenceId: string },
run: ProbeRun,
results: CompareStep[],
unobserved: UnobservedRecord[],
): CompareProbeReport => ({
label: probe.label,
evidence_id: probe.evidenceId,
steps: results.length,
equal: results.filter((step) => step.equal).length,
ids_mapped: run.map.size,
id_conflicts: run.map.conflicts,
normalised: run.normalised,
results,
unobserved,
});
const summaryOf = (
probes: readonly CompareProbeReport[],
): CompareReport["summary"] => {
const steps = probes.flatMap((probe) => probe.results);
const categories = Object.fromEntries(
DIFFERENCE_CATEGORIES.map((category) => [category, 0]),
) as Record<DifferenceCategory, number>;
for (const step of steps)
for (const category of step.categories) categories[category] += 1;
const equal = steps.filter((step) => step.equal).length;
const ratio = (count: number): number | null =>
steps.length === 0
? null
: Math.round((count / steps.length) * 1000) / 1000;
const normalised = emptyNormalised();
for (const probe of probes)
for (const key of NORMALISED_KEYS) normalised[key] += probe.normalised[key];
return {
probes: probes.length,
steps: steps.length,
equal,
agreement: ratio(equal),
agreement_ignoring_order: ratio(
steps.filter((step) => step.equal_ignoring_order).length,
),
unobserved: probes.reduce((sum, probe) => sum + probe.unobserved.length, 0),
categories,
normalised,
};
};
const toolsOf = (
probes: readonly CompareProbeReport[],
): CompareReport["tools"] => {
const tools = new Map<
string,
{ tool: string; steps: number; equal: number }
>();
for (const step of probes.flatMap((probe) => probe.results)) {
if (step.tool === null) continue;
const entry = tools.get(step.tool) ?? {
tool: step.tool,
steps: 0,
equal: 0,
};
entry.steps += 1;
if (step.equal) entry.equal += 1;
tools.set(step.tool, entry);
}
return [...tools.values()].sort(
(left, right) =>
left.equal / left.steps - right.equal / right.steps ||
(left.tool < right.tool ? -1 : 1),
);
};
/** Compare an environment with probe captures of the application: each probe from a reset state, every request answered and compared. */
export const compareEnvironment = (input: {
readonly source: EnvironmentSource;
readonly captureValue: unknown;
readonly probes: readonly CompareProbeInput[];
readonly operation: string;
}): Result<CompareReport, AnalysisError> => {
const { source, operation } = input;
const state = prepareServedState(source, input.captureValue, {
operation,
}).state;
const catalog = servedCatalog(source, state, source.manifest.name, operation);
if (!catalog.ok) return catalog;
const served = new Map(
catalog.value.tools.map((definition) => [
definition.name,
inputCheck(definition),
]),
);
const bodies = bodyTools(catalog.value.tools);
const recorded = parseCaptureEvidence(
operation,
input.captureValue,
"manifest",
);
const recordedBodies = new Map(
recorded.ok
? recorded.value.records.map((record) => [
record.ordinal,
requestBody(record),
])
: [],
);
const probes: CompareProbeReport[] = [];
for (const [index, probe] of input.probes.entries()) {
// A probe's request bodies are its call arguments, which a replay projection of a browser capture drops.
const capture = parseCaptureEvidence(operation, probe.value, "manifest");
// Without response bodies only statuses would compare, and without request bodies writes would be made empty.
const problem = !capture.ok
? capture.error.message
: !capture.value.responseBodies
? "a browser probe needs capture.network.response_body selected, or only statuses would compare."
: !capture.value.requestBodies && sendsBodies(capture.value.records)
? "a browser probe that makes POST, PUT or PATCH calls needs capture.network.request_body selected, or those calls would be made without their bodies."
: undefined;
if (!capture.ok || problem !== undefined)
return err(
inputError(
operation,
[
{
path: ["probe", index],
reason: "invalid_value",
message: `${probe.label}: ${problem}`,
},
],
"A probe file cannot be compared.",
),
);
const store =
state === undefined ? undefined : new EnvironmentStateStore(state);
const run: ProbeRun = {
source,
recordedBodies,
answer: environmentToolAnswer(source, store),
store,
served,
bodies,
map: new IdMap(),
normalised: emptyNormalised(),
};
const { steps, unobserved } = probeRequests(capture.value.records);
const results = steps.flatMap(
(step) => compareStep(run, source.manifest, step) ?? [],
);
probes.push(
probeReport(
{ label: probe.label, evidenceId: capture.value.evidenceId },
run,
results,
unobserved,
),
);
}
return ok({
status: "complete",
environment: {
name: source.manifest.name,
manifest_evidence_id: source.manifestEvidenceId,
evidence_id: source.evidenceId,
state: state === undefined ? "replay-only" : "served",
},
summary: summaryOf(probes),
tools: toolsOf(probes),
probes,
});
};