mirror of
https://github.com/morluto/rea.git
synced 2026-10-26 17:35:11 +00:00
feat(environment): time browser scenario records so scenario environments learn a write clock (#6967)
* feat(browser): record when scenario requests were observed and how long they took Request events carry observed_at, the wall-clock time the provider observed the request start, and finished or failed events carry the browser's elapsed_ms when it reports one. Both are optional, so earlier captures still parse, and scenario comparisons leave them out as a committed built-in rule. The real-Chrome network lane checks every request and finished response is timed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(environment): time browser scenario records so their state learns a write clock Scenario records take startedDateTime from when each request was observed and time from the browser's duration, as HAR entries are timed. A timed scenario of the book session learns the same clock and serves the same writes as its HAR; an untimed one keeps record order and learns no clock, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(environment): document scenario request times and timed scenario records Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(environment): start scenario records at the browser's own request start The provider observes a request up to tens of milliseconds after the browser starts it while it reads other responses, so a record starting then and ending after the browser's duration mixed two clocks. Finished and failed events now carry the browser's started_at, records prefer it and fall back to observed_at, and comparisons drop it too. Tests cover the fallback for a request that never ended or ended twice, a partly untimed capture keeping record order, comparing an untimed capture with a timed one, and the real-Chrome lane checks every finished request carries the browser's start and duration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(environment): document the browser start time, its fallback lag and older readers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(environment): record the scenario timing verification Gates, the real-Chrome lane, prompt-recorded and re-recorded published scenario captures built with and without their times, the stale published environments left unrepublished, and the review. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(browser): compare timed completion events in the request-time comparison test The comparison test's captures had no finished or failed events, so dropping started_at and elapsed_ms from them went untested. The captures now end one request and fail the other, the test asserts both carry times, compares them shifted by seven seconds and against an untimed capture, and a control shows a changed failure on a timed completion is still reported. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
This commit is contained in:
20 files changed
+685
-20
No files matched your search
@@ -2,7 +2,7 @@
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "48"
|
||||
version: "49"
|
||||
---
|
||||
|
||||
# REA
|
||||
|
||||
@@ -186,10 +186,11 @@ withheld with its hint:
|
||||
|
||||
Use new scratch names on each attempt; the first attempt's objects may still
|
||||
exist.
|
||||
A browser scenario carries no request times, so the coverage limitations say
|
||||
the state learns no clock: an update the application stamps with a
|
||||
modification time stays `unexplained-change` however it is recorded. Report
|
||||
it rather than re-recording for it.
|
||||
New browser scenario captures include request times, so a fully timed capture
|
||||
can learn a write clock and serve updates that stamp modification times. Older
|
||||
captures or ones missing a request start keep record order and learn no clock.
|
||||
When coverage reports that limitation, record again with the current version
|
||||
before concluding that the timestamp change cannot be served.
|
||||
|
||||
## Export and hand off
|
||||
|
||||
|
||||
@@ -323,11 +323,7 @@ read the list after each create (the create's response or the list shows the
|
||||
new entity), and record updates and deletes in a second scenario that reads
|
||||
the first one's objects by id before editing them, or make them through the
|
||||
UI. The environment coverage view names each
|
||||
write it withholds and what to record next. A scenario records no request
|
||||
times, so the state learns no clock: an update whose response changes a
|
||||
timestamp the server sets, such as a modification time, is withheld as
|
||||
`unexplained-change` however it is recorded, and the coverage view says so; a
|
||||
HAR of the same session times its entries and can serve it. A scenario with write requests
|
||||
write it withholds and what to record next. A scenario with write requests
|
||||
changes the application's data: record against an instance whose data may
|
||||
change.
|
||||
|
||||
@@ -348,6 +344,15 @@ change.
|
||||
}
|
||||
```
|
||||
|
||||
Each record takes its start and duration from the browser's timing (the time
|
||||
the scenario observed the request where the browser reported no start), so the
|
||||
state orders and overlaps
|
||||
records as it does a HAR's and learns a write clock: an update the application
|
||||
stamps with a modification time can be served. A scenario captured before
|
||||
these times were recorded has none; its records keep request order and the
|
||||
state learns no clock, so such updates stay `unexplained-change` until it is
|
||||
recorded again.
|
||||
|
||||
## What a manifest is
|
||||
|
||||
An environment manifest is a projection of one application's recorded request
|
||||
|
||||
@@ -465,6 +465,17 @@ and unfinished-request metadata. Select content independently:
|
||||
All three options default to `false`. Any selected network content also enables
|
||||
network metadata observation; listing `network` in `events` is then optional.
|
||||
Each observed request object receives a capture-scoped `transaction_id`.
|
||||
Each `request` event carries `observed_at`, the wall-clock time REA observed the
|
||||
request start. Playwright reports the browser's own start only once a request
|
||||
ends, so each `request-finished` or `request-failed` event carries `started_at`,
|
||||
the browser's wall-clock start, and `elapsed_ms`, its time from that start to
|
||||
the end of the response, when the browser reports them. The environment views
|
||||
time scenario records from these, as HAR entries are timed: the browser's start
|
||||
where it reported one, else `observed_at`, which trails it by up to tens of
|
||||
milliseconds while REA reads other responses, and a duration of 0 where the
|
||||
browser reported none. Captures made before these fields existed lack them and
|
||||
are read in record order. REA versions before them refuse captures that carry
|
||||
them. Scenario comparisons ignore all three (`ignore_network_request_timing`).
|
||||
Requests sharing a URL remain distinct. Redirects link to the preceding
|
||||
transaction only when it was observed; a null predecessor does not establish
|
||||
that no earlier hop existed. Request events require both transaction fields;
|
||||
@@ -562,8 +573,8 @@ and compared as an unordered collection after caller rules apply, because
|
||||
concurrent responses arrive in any order and can land in a later step; a
|
||||
redirect retains the complete predecessor ancestry, including each hop's own
|
||||
request, response outcome, and selected network content. Each transaction's
|
||||
capture-local IDs, sequence/index fields, and content source-sequence anchor
|
||||
are removed. Caller rules normalize retained strings before hashing. The
|
||||
capture-local IDs, sequence/index fields, content source-sequence anchor and
|
||||
request times (`observed_at`, `started_at`, `elapsed_ms`) are removed. Caller rules normalize retained strings before hashing. The
|
||||
`preserve_redirect_transaction_ancestry` built-in rule fingerprints each hop
|
||||
as canonical SHA-256 of `{ events, redirected_from }`: `events` is the sorted
|
||||
list of its normalized event SHA-256 digests, and `redirected_from` is its
|
||||
@@ -584,7 +595,10 @@ scope/flags, value fingerprints, and duplicate entries even when redaction or
|
||||
literal rules make names collide. Captured storage arrays retain their original
|
||||
order. The `compare_storage_entries_as_unordered_collection` built-in rule
|
||||
participates in the normalization commitment; new storage comparison digests
|
||||
therefore differ from earlier order-sensitive comparisons.
|
||||
therefore differ from earlier order-sensitive comparisons. The
|
||||
`ignore_network_request_timing` built-in rule likewise joins the commitment, so
|
||||
normalization digests differ from comparisons made before it, though captures
|
||||
with and without request times compare alike.
|
||||
|
||||
Each aligned step reports changed, unchanged, or unknown. Changed and unknown
|
||||
action, screenshot, DOM, accessibility, URL, history, storage, and event
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
# Plan
|
||||
|
||||
1. Schema: optional `observed_at` (ISO date-time) on `request` events, and
|
||||
`started_at` (ISO date-time) and `elapsed_ms` (non-negative number) on
|
||||
`request-finished` and `request-failed` events.
|
||||
2. Collector: stamp `observed_at` when a request is observed and copy the
|
||||
browser's `timing().startTime` and `responseEnd` into `started_at` and
|
||||
`elapsed_ms` when it reports them, admitting the strings with the other
|
||||
metadata.
|
||||
3. Records: index each transaction's single finished or failed start and
|
||||
duration and give each attributable record `startedDateTime` and `time`;
|
||||
add a limitation when any record is timed.
|
||||
4. Comparison: drop both fields from network evidence before fingerprinting and
|
||||
commit the rule in the built-in list and the comparison limitation.
|
||||
5. Docs: the scenario contract's network evidence and the environment docs'
|
||||
browser scenario section.
|
||||
6. Tests: the collector stamps both fields, records take them, the state learns
|
||||
the HAR's clock and verdicts from a timed scenario and none from an untimed
|
||||
or partly untimed one, a request that never ended or ended twice falls back
|
||||
to its observation, comparisons ignore the times (also against an untimed
|
||||
capture), and the real-Chrome network lane checks every request is observed
|
||||
and every finished request carries the browser's start and duration.
|
||||
@@ -0,0 +1,44 @@
|
||||
# Time browser scenario records
|
||||
|
||||
## Outcome
|
||||
|
||||
Browser scenario network events record no times, so every record a scenario
|
||||
projects is untimed. The environment-state view then orders records by
|
||||
position, detects no overlapping requests and learns no write clock: an update
|
||||
whose response carries a modification time the application sets is withheld as
|
||||
`unexplained-change` however it is recorded, while a HAR of the same session
|
||||
serves it with a clock rule.
|
||||
|
||||
Scenario network events now record when each request was observed and how long
|
||||
the browser took, and scenario records carry them as HAR entries do, so a
|
||||
scenario capture learns the clock a HAR does.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Each `request` event carries `observed_at`, the ISO wall-clock time the
|
||||
provider observed the request start; Playwright reports the browser's start
|
||||
only once a request ends.
|
||||
- Each `request-finished` and `request-failed` event carries `started_at`, the
|
||||
browser's wall-clock start (`timing().startTime > 0`), and `elapsed_ms`, its
|
||||
time from that start to the end of the response (`timing().responseEnd >= 0`),
|
||||
when the browser reports them.
|
||||
- The fields are optional in the schema, so Evidence captured before them still
|
||||
parses and is read exactly as before. REA versions before them refuse
|
||||
Evidence that carries them.
|
||||
- Scenario records take `startedDateTime` from the transaction's single
|
||||
finished or failed event's `started_at`, else from `observed_at`, and `time`
|
||||
from its `elapsed_ms` (0 without one), and say so in a limitation; a capture
|
||||
whose requests are all timed gets the recorded-start-time timeline and a
|
||||
write clock.
|
||||
- Scenario comparisons ignore both fields, as a committed built-in rule
|
||||
(`ignore_network_request_timing`).
|
||||
- The added metadata is charged to the event budgets as other metadata is.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No change to HAR captures, to how the state uses times, or to what a
|
||||
record's timing means for overlap or clock rules.
|
||||
- No retroactive timing of captures made before this change.
|
||||
- No browser start for a request that never ended: its record starts at
|
||||
`observed_at`, which can trail the browser's start by tens of milliseconds
|
||||
while the provider reads other responses, with no duration.
|
||||
@@ -0,0 +1,13 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the outcome, requirements and non-goals.
|
||||
- [x] Add the optional timing fields to scenario network events.
|
||||
- [x] Stamp them in the Playwright network collector.
|
||||
- [x] Time scenario records from them.
|
||||
- [x] Leave them out of scenario comparisons.
|
||||
- [x] Document the fields and the timed records.
|
||||
- [x] Add unit, view and real-browser checks.
|
||||
- [x] Run the gates.
|
||||
- [x] Re-record stale scenario environments and compare them with their published versions.
|
||||
- [x] Build prompt-recorded environments with and without request times.
|
||||
- [x] Address the independent review.
|
||||
@@ -0,0 +1,73 @@
|
||||
# Verification
|
||||
|
||||
Executed on macOS with Node v24.22.1 at `feat/scenario-request-timing` b92a11b6 (on main d87e73f2), and on Ubuntu 24.04
|
||||
VMs with snap Chromium 155 for the recordings.
|
||||
|
||||
## Gates
|
||||
|
||||
- `npm run -s typecheck`, `lint`, `format:check`, `docs:check`, `mcp-catalog:check`, `verify:test-discovery`,
|
||||
`scan:todos` and `npx knip`: passed.
|
||||
- `npx vitest run --project domain --project services --project adapters`: 16,380 passed and 1 failed, "stops Inspector
|
||||
target authorization after the current stat settles", which fails the same way on main in this environment.
|
||||
- Browser and scenario MCP and CLI boundaries, tool schema validity and the analysis-view MCP boundary: passed.
|
||||
- `npm run -s verify:browser:network` in real Chrome: every one of the 10 requests carries `observed_at`, and every
|
||||
finished request carries the browser's `started_at` and `elapsed_ms` (9 of 10; the tenth is the fixture's deliberately
|
||||
unfinished stream).
|
||||
- Captures made before this change parse and project exactly as before: the published Pinry (568 requests), PhotoPrism
|
||||
(277) and Lychee (134) captures give byte-identical manifest and state views, Evidence ids and digests included, on
|
||||
main and on this branch.
|
||||
|
||||
## The same capture with and without request times
|
||||
|
||||
Each capture below was recorded with this branch, then built twice: as recorded, and with its request times removed and
|
||||
the Evidence sealed again as the provider seals it. The difference is this change alone.
|
||||
|
||||
| Capture | Records | Timeline without / with times | Writes served without / with times |
|
||||
| ---------------------------------------------- | ------- | ----------------------------- | ---------------------------------- |
|
||||
| Linkding, agent-recorded from a prompt | 60 | record order / start time | 1 / 4 |
|
||||
| Linkding, agent-recorded from a prompt (again) | 65 | record order / start time | 3 / 3 |
|
||||
| Homebox, agent-recorded from a prompt | 263 | record order / start time | 2 / 3 |
|
||||
| Homebox, agent-recorded from a prompt (again) | 275 | record order / start time | 2 / 2 |
|
||||
| Pinry, its published scenario re-recorded | 568 | record order / start time | 0 / 0 (1 clock rule learned) |
|
||||
| PhotoPrism, its published scenario re-recorded | 273 | record order / start time | 2 / 2 (2 clock rules learned) |
|
||||
|
||||
- With times, Linkding's bookmark and bundle edits and Homebox's tag edit are served: their responses carry a
|
||||
modification time the application sets, which only a clock explains. PhotoPrism's served album create now stamps
|
||||
`CreatedAt` and `UpdatedAt` from the clock instead of replaying the recorded values. No capture served fewer writes.
|
||||
- The second Linkding and Homebox recordings gain nothing because their gaps are elsewhere (`unconfirmed` and `contradicted`
|
||||
writes): an agent's recording choices move coverage more than timing does.
|
||||
- The prompt-built recordings used a local, unpushed build of this branch merged with the prompt PR (#6946), which adds
|
||||
the `request` scenario action they need. Claude Code (`claude-sonnet-5-5`) recorded each app from a one-paragraph
|
||||
prompt, as in that PR's end-to-end check.
|
||||
- The Linkding bundle with four served writes passes all three of the application's write tasks with
|
||||
`fireworks_ai/gpt-oss-120b` and `fireworks_ai/glm-5p3-flash` (with the tasks' entity counts taken relative to the
|
||||
bundle's own state, which holds active and archived bookmarks as two types); the earlier prompt-built Linkding bundles,
|
||||
recorded without times, passed none. The Homebox bundle passes both of its write tasks with gpt-oss and one with GLM
|
||||
(GLM deleted the wrong item). The final build's prompt-recorded bundles start and answer retrieval: Linkding
|
||||
passed its smoke task and 8/8 and 7/8 retrieval tasks, Homebox its smoke task and 3/3 and 3/3.
|
||||
|
||||
## Stale published environments
|
||||
|
||||
Pinry and PhotoPrism are the oldest published environments recorded as browser scenarios (version 0.1.0, replay only).
|
||||
Both were re-recorded with this branch from their published scenarios on fresh seeded instances and rebuilt with the
|
||||
publishing pipeline:
|
||||
|
||||
| App | Published 0.1.0 | Rebuilt with this branch |
|
||||
| ---------- | ------------------------------------------- | ---------------------------------------------------------------------------------- |
|
||||
| Pinry | 12 tools, replay, retrieval 6/6 and 6/6 | 12 tools, state and clock, no writes served; retrieval 6/6 and 4/6 |
|
||||
| PhotoPrism | 13 tools, replay, retrieval 10/10 and 10/10 | 14 tools, state and clock, album create and rename served; retrieval 8/10 and 5/10 |
|
||||
|
||||
Neither improves, so neither was republished. The retrieval drops do not come from timing: the timed and untimed states
|
||||
are identical in every view these tasks read. PhotoPrism's album list is answered from the state in name order while
|
||||
the recorded response that generated the task listed another album first; Pinry's two GLM misses are tool calls
|
||||
rejected for passing `limit` and `offset` as numbers, under the same tool schemas the published version had.
|
||||
|
||||
## Review
|
||||
|
||||
An independent review rebuilt the branch and tried to refute it. It confirmed byte-identical views for earlier captures,
|
||||
that comparisons with and without times compare alike, that each record's duration comes only from its own
|
||||
transaction, and that both fields are charged to the event budgets. Its findings are addressed: records now start at the
|
||||
browser's own start time rather than the provider's observation, which trailed it by up to 36 ms while other responses
|
||||
were read; the docs give the fallback's lag, that earlier REA versions refuse captures carrying the fields, and that the
|
||||
comparison's normalization digest changes; and tests cover the fallbacks, partly untimed captures and comparisons with
|
||||
untimed captures.
|
||||
@@ -146,7 +146,23 @@ export function assertBrowserNetworkEvidence(
|
||||
);
|
||||
assert.ok(first && next);
|
||||
assert.equal(next.redirected_from_transaction_id, first.transaction_id);
|
||||
// Every request is stamped when it was observed; finished ones carry the browser's duration.
|
||||
for (const request of requestEvents)
|
||||
assert.ok(
|
||||
Number.isFinite(Date.parse(request.observed_at)),
|
||||
`Request ${request.url.url} has no observation time`,
|
||||
);
|
||||
const timed = events.filter((event) => event.kind === "request-finished");
|
||||
for (const finished of timed)
|
||||
assert.ok(
|
||||
typeof finished.elapsed_ms === "number" &&
|
||||
Number.isFinite(Date.parse(finished.started_at)),
|
||||
`Finished request ${finished.url.url} has no browser start or duration`,
|
||||
);
|
||||
assert.ok(timed.length > 0, "No request finished");
|
||||
return {
|
||||
timed_requests: requestEvents.length,
|
||||
timed_completions: timed.length,
|
||||
same_url_markers: markers,
|
||||
binary_bytes: networkFixtureBinary.length,
|
||||
compressed_representation: "playwright-response-body-bytes",
|
||||
|
||||
@@ -944,3 +944,50 @@ it("does not read content when phase metadata was omitted", async () => {
|
||||
);
|
||||
expect(collector.limitations().join(" ")).toContain("incomplete");
|
||||
});
|
||||
|
||||
it("stamps when each request was observed, and the browser's own start and duration once it ends", async () => {
|
||||
vi.useFakeTimers({ now: Date.parse("2026-10-24T05:00:00.250Z") });
|
||||
const { collector, events } = harness({
|
||||
request_body: false,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
});
|
||||
class TimedRequest extends RequestFixture {
|
||||
constructor(
|
||||
marker: string,
|
||||
readonly startTime: number,
|
||||
readonly responseEnd: number,
|
||||
) {
|
||||
super(marker);
|
||||
}
|
||||
timing() {
|
||||
return { startTime: this.startTime, responseEnd: this.responseEnd };
|
||||
}
|
||||
}
|
||||
const finished = new TimedRequest(
|
||||
"timed",
|
||||
Date.parse("2026-10-24T05:00:00.200Z"),
|
||||
12.5,
|
||||
);
|
||||
const failed = new TimedRequest("untimed", 0, -1);
|
||||
collector.request(finished);
|
||||
collector.request(failed);
|
||||
collector.finished(finished);
|
||||
collector.failed(failed);
|
||||
await collector.finish();
|
||||
expect(events.filter(({ kind }) => kind === "request")).toEqual([
|
||||
expect.objectContaining({ observed_at: "2026-10-24T05:00:00.250Z" }),
|
||||
expect.objectContaining({ observed_at: "2026-10-24T05:00:00.250Z" }),
|
||||
]);
|
||||
expect(events).toContainEqual(
|
||||
expect.objectContaining({
|
||||
kind: "request-finished",
|
||||
started_at: "2026-10-24T05:00:00.200Z",
|
||||
elapsed_ms: 12.5,
|
||||
}),
|
||||
);
|
||||
const failure = events.find(({ kind }) => kind === "request-failed");
|
||||
expect(failure).toBeDefined();
|
||||
expect(failure).not.toHaveProperty("started_at");
|
||||
expect(failure).not.toHaveProperty("elapsed_ms");
|
||||
});
|
||||
@@ -26,8 +26,27 @@ export interface ScenarioNetworkRequest extends NetworkHeadersSource {
|
||||
postDataBuffer(): Buffer | null;
|
||||
redirectedFrom(): ScenarioNetworkRequest | null;
|
||||
failure(): { errorText: string } | null;
|
||||
/** Browser resource timing: `startTime` in epoch milliseconds (0 until known), `responseEnd` after it (-1 when unknown). */
|
||||
timing?(): { readonly startTime: number; readonly responseEnd: number };
|
||||
}
|
||||
|
||||
/** The browser's own start and duration for a finished or failed request, as far as it reported them. */
|
||||
const browserTiming = (
|
||||
request: ScenarioNetworkRequest,
|
||||
): { readonly started_at?: string; readonly elapsed_ms?: number } => {
|
||||
const timing = request.timing?.();
|
||||
if (timing === undefined) return {};
|
||||
const { startTime, responseEnd } = timing;
|
||||
return {
|
||||
...(Number.isFinite(startTime) && startTime > 0
|
||||
? { started_at: new Date(startTime).toISOString() }
|
||||
: {}),
|
||||
...(Number.isFinite(responseEnd) && responseEnd >= 0
|
||||
? { elapsed_ms: responseEnd }
|
||||
: {}),
|
||||
};
|
||||
};
|
||||
|
||||
/** A response's body becomes eligible for reading only after requestfinished. */
|
||||
export interface ScenarioNetworkResponse extends NetworkHeadersSource {
|
||||
request(): ScenarioNetworkRequest;
|
||||
@@ -155,13 +174,20 @@ export class PlaywrightScenarioNetwork {
|
||||
return;
|
||||
}
|
||||
if (transaction.requestObserved) return;
|
||||
if (!this.admitMetadata(transaction, "request", request)) return;
|
||||
const observedAt = new Date().toISOString();
|
||||
if (
|
||||
!this.admitMetadata(transaction, "request", request, undefined, [
|
||||
observedAt,
|
||||
])
|
||||
)
|
||||
return;
|
||||
transaction.requestObserved = true;
|
||||
const admission = this.options.emit({
|
||||
...this.metadata(request, transaction),
|
||||
kind: "request",
|
||||
status: null,
|
||||
failure: null,
|
||||
observed_at: observedAt,
|
||||
});
|
||||
if (admission.status === "omitted") {
|
||||
this.options.omitNetwork(transaction.id, "request", admission.reason);
|
||||
@@ -253,12 +279,23 @@ export class PlaywrightScenarioNetwork {
|
||||
return;
|
||||
}
|
||||
transaction.terminal = "finished";
|
||||
if (!this.admitMetadata(transaction, "request-finished", request)) return;
|
||||
const timing = browserTiming(request);
|
||||
if (
|
||||
!this.admitMetadata(
|
||||
transaction,
|
||||
"request-finished",
|
||||
request,
|
||||
undefined,
|
||||
timing.started_at === undefined ? [] : [timing.started_at],
|
||||
)
|
||||
)
|
||||
return;
|
||||
const admission = this.options.emit({
|
||||
...this.metadata(request, transaction),
|
||||
kind: "request-finished",
|
||||
status: null,
|
||||
failure: null,
|
||||
...timing,
|
||||
});
|
||||
if (admission.status === "omitted") {
|
||||
this.options.omitNetwork(
|
||||
@@ -282,9 +319,11 @@ export class PlaywrightScenarioNetwork {
|
||||
}
|
||||
transaction.terminal = "failed";
|
||||
const failure = request.failure()?.errorText || "Unknown request failure";
|
||||
const timing = browserTiming(request);
|
||||
if (
|
||||
!this.admitMetadata(transaction, "request-failed", request, undefined, [
|
||||
failure,
|
||||
...(timing.started_at === undefined ? [] : [timing.started_at]),
|
||||
])
|
||||
)
|
||||
return;
|
||||
@@ -294,6 +333,7 @@ export class PlaywrightScenarioNetwork {
|
||||
kind: "request-failed",
|
||||
status: null,
|
||||
failure: message,
|
||||
...timing,
|
||||
});
|
||||
if (admission.status === "omitted") {
|
||||
this.options.omitNetwork(
|
||||
|
||||
@@ -174,7 +174,7 @@ export const projectEnvironmentCoverageView = (
|
||||
"Coverage summarizes the environment-state view an environment derives at startup; it does not run the environment. An environment whose served catalog exceeds the discovery budget, or whose state exceeds what data/add loads, serves replay instead, which this view does not check. Hints are advice about what to record next.",
|
||||
...(state.timeline === "record-order"
|
||||
? [
|
||||
"Not every record carries a start time (browser scenario captures carry none), so the state learns no clock: a write whose response changes a timestamp the server sets, such as a modification time, is withheld as unexplained-change however often it is recorded.",
|
||||
"Not every record carries a start time, so the state learns no clock: a write whose response changes a timestamp the server sets, such as a modification time, is withheld as unexplained-change however often it is recorded.",
|
||||
]
|
||||
: []),
|
||||
],
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { timedCaptureEvidence } from "../../../tests/fixtures/analysisView.js";
|
||||
import {
|
||||
scenarioCaptureEvidence,
|
||||
scenarioCaptureFixture,
|
||||
type ScenarioExchange,
|
||||
} from "../../../tests/fixtures/browserScenarioCapture.js";
|
||||
import {
|
||||
BOOK_ROWS,
|
||||
BOOK_WRITE_CLOCK,
|
||||
environmentStateViewOf,
|
||||
} from "../../../tests/fixtures/environmentStateWrites.js";
|
||||
import type { BrowserScenarioCapture } from "../browserScenarioCapture.js";
|
||||
import { scenarioCaptureRecords } from "./scenarioCaptureRecords.js";
|
||||
|
||||
const START = Date.parse("2026-10-17T18:00:00.000Z");
|
||||
|
||||
/** The book session the HAR fixture times one second apart, as browser scenario exchanges. */
|
||||
const EXCHANGES: readonly ScenarioExchange[] = BOOK_ROWS.map(
|
||||
([method, path, body = null, status = 200, sent]) => ({
|
||||
method,
|
||||
url: path,
|
||||
status,
|
||||
mediaType: "application/json",
|
||||
body: typeof body === "string" ? body : JSON.stringify(body),
|
||||
...(sent === undefined
|
||||
? {}
|
||||
: {
|
||||
requestMediaType: "application/json",
|
||||
requestBody: JSON.stringify(sent),
|
||||
}),
|
||||
}),
|
||||
);
|
||||
|
||||
const at = (ms: number): string => new Date(ms).toISOString();
|
||||
|
||||
/**
|
||||
* The same capture as the browser reports it: each request started when the HAR fixture starts it and finished 5 ms
|
||||
* later, observed by the provider 15 ms after its start.
|
||||
*/
|
||||
const timed = (capture: BrowserScenarioCapture): BrowserScenarioCapture => {
|
||||
let transaction = 0;
|
||||
const items = capture.events.items.map((event) =>
|
||||
event.kind === "request"
|
||||
? { ...event, observed_at: at(START + transaction++ * 1000 + 15) }
|
||||
: event,
|
||||
);
|
||||
let sequence = Math.max(...items.map((event) => event.sequence));
|
||||
const finished = items.flatMap((event) =>
|
||||
event.kind === "request"
|
||||
? [
|
||||
{
|
||||
...event,
|
||||
observed_at: undefined,
|
||||
sequence: ++sequence,
|
||||
kind: "request-finished" as const,
|
||||
started_at: at(Date.parse(event.observed_at ?? "") - 15),
|
||||
elapsed_ms: 5,
|
||||
},
|
||||
]
|
||||
: [],
|
||||
);
|
||||
return {
|
||||
...capture,
|
||||
events: {
|
||||
...capture.events,
|
||||
retained: items.length + finished.length,
|
||||
items: [
|
||||
...items,
|
||||
...finished.map(({ observed_at: _unset, ...event }) => event),
|
||||
],
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
const verdicts = (state: ReturnType<typeof environmentStateViewOf>) =>
|
||||
Object.fromEntries(
|
||||
state.writes.map((plan) => [
|
||||
plan.tool,
|
||||
plan.served
|
||||
? plan.rules.map(({ field, rule }) => `${field}:${rule}`)
|
||||
: plan.reason,
|
||||
]),
|
||||
);
|
||||
|
||||
it("times scenario records from when each request was observed and how long the browser took", () => {
|
||||
const records = scenarioCaptureRecords(
|
||||
timed(scenarioCaptureFixture(EXCHANGES)),
|
||||
"manifest",
|
||||
);
|
||||
// The browser's start, not the provider's later observation.
|
||||
expect(records.records[1]?.reported).toMatchObject({
|
||||
startedDateTime: "2026-10-17T18:00:01.000Z",
|
||||
time: 5,
|
||||
});
|
||||
expect(records.limitations).toContainEqual(
|
||||
expect.stringContaining("the browser's own"),
|
||||
);
|
||||
const untimed = scenarioCaptureRecords(
|
||||
scenarioCaptureFixture(EXCHANGES),
|
||||
"manifest",
|
||||
);
|
||||
expect(untimed.records[1]?.reported).not.toHaveProperty("startedDateTime");
|
||||
expect(untimed.limitations.join(" ")).not.toContain("the browser's own");
|
||||
});
|
||||
|
||||
it("learns the clock from a timed scenario as from the same session's HAR, and none without times", () => {
|
||||
const har = environmentStateViewOf(timedCaptureEvidence(BOOK_ROWS));
|
||||
const scenario = environmentStateViewOf(
|
||||
scenarioCaptureEvidence(timed(scenarioCaptureFixture(EXCHANGES))),
|
||||
);
|
||||
expect(scenario.timeline).toBe("recorded-start-time");
|
||||
expect(scenario.write_clock).toBe(BOOK_WRITE_CLOCK);
|
||||
expect(verdicts(scenario)).toEqual(verdicts(har));
|
||||
expect(Object.values(verdicts(scenario)).flat()).toContain(
|
||||
"updated_at:clock",
|
||||
);
|
||||
|
||||
const untimed = environmentStateViewOf(
|
||||
scenarioCaptureEvidence(scenarioCaptureFixture(EXCHANGES)),
|
||||
);
|
||||
expect(untimed.timeline).toBe("record-order");
|
||||
expect(untimed.write_clock).toBeNull();
|
||||
expect(Object.values(verdicts(untimed)).flat()).not.toContain(
|
||||
"updated_at:clock",
|
||||
);
|
||||
});
|
||||
|
||||
it("falls back to the observed time with no duration when a request never ended or ended twice", () => {
|
||||
const capture = timed(scenarioCaptureFixture(EXCHANGES));
|
||||
const finished = capture.events.items.filter(
|
||||
(event) => event.kind === "request-finished",
|
||||
);
|
||||
const [first, second] = finished;
|
||||
if (first === undefined || second === undefined)
|
||||
throw new Error("Expected finished events");
|
||||
const items = capture.events.items.filter((event) => event !== first);
|
||||
items.push({ ...second, sequence: items.length + 100 });
|
||||
const records = scenarioCaptureRecords(
|
||||
{ ...capture, events: { ...capture.events, items } },
|
||||
"manifest",
|
||||
);
|
||||
expect(records.records[0]?.reported).toMatchObject({
|
||||
startedDateTime: "2026-10-17T18:00:00.015Z",
|
||||
time: 0,
|
||||
});
|
||||
expect(records.records[1]?.reported).toMatchObject({
|
||||
startedDateTime: "2026-10-17T18:00:01.015Z",
|
||||
time: 0,
|
||||
});
|
||||
expect(records.records[2]?.reported).toMatchObject({
|
||||
startedDateTime: "2026-10-17T18:00:02.000Z",
|
||||
time: 5,
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps record order and learns no clock when any request is untimed", () => {
|
||||
const capture = timed(scenarioCaptureFixture(EXCHANGES));
|
||||
const items = capture.events.items.map((event, index) => {
|
||||
if (index !== 0 || event.kind !== "request") return event;
|
||||
const { observed_at: _dropped, ...untimed } = event;
|
||||
return untimed;
|
||||
});
|
||||
const firstTransaction = capture.events.items.find(
|
||||
(event) => event.kind === "request",
|
||||
)?.transaction_id;
|
||||
const terminal = items.filter(
|
||||
(event) =>
|
||||
!(
|
||||
event.kind === "request-finished" &&
|
||||
event.transaction_id === firstTransaction
|
||||
),
|
||||
);
|
||||
const state = environmentStateViewOf(
|
||||
scenarioCaptureEvidence({
|
||||
...capture,
|
||||
events: { ...capture.events, items: terminal, retained: terminal.length },
|
||||
}),
|
||||
);
|
||||
expect(state.timeline).toBe("record-order");
|
||||
expect(state.write_clock).toBeNull();
|
||||
});
|
||||
@@ -107,11 +107,51 @@ const reportedRequest = (
|
||||
};
|
||||
};
|
||||
|
||||
interface BrowserTiming {
|
||||
readonly started_at?: string | undefined;
|
||||
readonly elapsed_ms?: number | undefined;
|
||||
}
|
||||
|
||||
/** Each transaction's browser-reported start and duration, from its single finished or failed event. */
|
||||
const timingByTransaction = (
|
||||
events: RetainedBrowserScenarioCapture["events"]["items"],
|
||||
): ReadonlyMap<string, BrowserTiming> => {
|
||||
const timing = new Map<string, BrowserTiming>();
|
||||
for (const event of events) {
|
||||
if (event.kind !== "request-finished" && event.kind !== "request-failed")
|
||||
continue;
|
||||
// A transaction ending twice has no single start or duration.
|
||||
timing.set(
|
||||
event.transaction_id,
|
||||
timing.has(event.transaction_id)
|
||||
? {}
|
||||
: { started_at: event.started_at, elapsed_ms: event.elapsed_ms },
|
||||
);
|
||||
}
|
||||
return timing;
|
||||
};
|
||||
|
||||
/**
|
||||
* HAR timing for a record: the browser's start when it reported one, else when the request was observed, and the
|
||||
* browser's duration (0 when it reported none).
|
||||
*/
|
||||
const recordTiming = (
|
||||
request: RequestEvent,
|
||||
timing: ReadonlyMap<string, BrowserTiming>,
|
||||
): { readonly startedDateTime: string; readonly time: number } | undefined => {
|
||||
const browser = timing.get(request.transaction_id);
|
||||
const started = browser?.started_at ?? request.observed_at;
|
||||
return started === undefined
|
||||
? undefined
|
||||
: { startedDateTime: started, time: browser?.elapsed_ms ?? 0 };
|
||||
};
|
||||
|
||||
const record = (
|
||||
ordinal: number,
|
||||
observed: { readonly event: RequestEvent; readonly item: number },
|
||||
index: BrowserScenarioTransactionIndex,
|
||||
purpose: "manifest" | "replay",
|
||||
timing: ReturnType<typeof recordTiming>,
|
||||
): {
|
||||
readonly record: WebNetworkCaptureRecord;
|
||||
readonly responseBody: ScenarioResponseBodyRetention;
|
||||
@@ -151,6 +191,7 @@ const record = (
|
||||
ordinal,
|
||||
location,
|
||||
reported: {
|
||||
...timing,
|
||||
request: reportedRequest(request, requestContent, purpose),
|
||||
...(response === undefined
|
||||
? {}
|
||||
@@ -190,11 +231,21 @@ export const scenarioCaptureRecords = (
|
||||
purpose: "manifest" | "replay",
|
||||
): ScenarioCaptureRecords => {
|
||||
const index = indexBrowserScenarioTransactions(capture.events.items);
|
||||
const timings = timingByTransaction(capture.events.items);
|
||||
const records: WebNetworkCaptureRecord[] = [];
|
||||
const responseBodies = new Map<number, ScenarioResponseBodyRetention>();
|
||||
let timed = false;
|
||||
for (const [item, event] of capture.events.items.entries()) {
|
||||
if (event.kind !== "request") continue;
|
||||
const projected = record(records.length, { event, item }, index, purpose);
|
||||
const timing = recordTiming(event, timings);
|
||||
timed ||= timing !== undefined;
|
||||
const projected = record(
|
||||
records.length,
|
||||
{ event, item },
|
||||
index,
|
||||
purpose,
|
||||
timing,
|
||||
);
|
||||
records.push(projected.record);
|
||||
if (purpose === "manifest")
|
||||
responseBodies.set(projected.record.ordinal, projected.responseBody);
|
||||
@@ -205,6 +256,11 @@ export const scenarioCaptureRecords = (
|
||||
responseBodies,
|
||||
limitations: [
|
||||
"Scenario records are the capture's network transactions in request order; each record ordinal is the transaction's zero-based position.",
|
||||
...(timed
|
||||
? [
|
||||
"Scenario record start times are the browser's own, from when the request ended, or when the provider observed the request where the browser reported none; durations are the browser's own, or 0 where it reported none.",
|
||||
]
|
||||
: []),
|
||||
...(capture.events.dropped > 0
|
||||
? [
|
||||
`The scenario capture omitted ${String(capture.events.dropped)} events; the manifest covers only retained transactions.`,
|
||||
|
||||
@@ -147,6 +147,21 @@ const eventBase = {
|
||||
};
|
||||
|
||||
const eventUrl = sanitizedBrowserUrlSchema.nullable();
|
||||
/** The browser's wall-clock request start, reported once the request ends. */
|
||||
const browserStartSchema = z.iso
|
||||
.datetime()
|
||||
.optional()
|
||||
.describe(
|
||||
"Wall-clock time the browser started the request, as it reported at completion; absent when it reported none.",
|
||||
);
|
||||
/** Browser-reported time from the request start to the end of its response, when the browser reports one. */
|
||||
const elapsedMsSchema = z
|
||||
.number()
|
||||
.nonnegative()
|
||||
.optional()
|
||||
.describe(
|
||||
"Milliseconds from the request start to the end of its response, as the browser timed it; absent when it reported none.",
|
||||
);
|
||||
const networkEventShape = {
|
||||
...eventBase,
|
||||
transaction_id: z.string().min(1),
|
||||
@@ -186,6 +201,12 @@ export const browserScenarioEventSchema = z
|
||||
kind: z.literal("request"),
|
||||
status: z.null(),
|
||||
failure: z.null(),
|
||||
observed_at: z.iso
|
||||
.datetime()
|
||||
.optional()
|
||||
.describe(
|
||||
"Wall-clock time the provider observed the request start; absent in captures made before it was recorded.",
|
||||
),
|
||||
}),
|
||||
z.strictObject({
|
||||
...networkEventShape,
|
||||
@@ -198,12 +219,16 @@ export const browserScenarioEventSchema = z
|
||||
kind: z.literal("request-failed"),
|
||||
status: z.null(),
|
||||
failure: z.string().min(1),
|
||||
started_at: browserStartSchema,
|
||||
elapsed_ms: elapsedMsSchema,
|
||||
}),
|
||||
z.strictObject({
|
||||
...networkEventShape,
|
||||
kind: z.literal("request-finished"),
|
||||
status: z.null(),
|
||||
failure: z.null(),
|
||||
started_at: browserStartSchema,
|
||||
elapsed_ms: elapsedMsSchema,
|
||||
}),
|
||||
z.strictObject({
|
||||
...networkEventShape,
|
||||
|
||||
@@ -584,6 +584,116 @@ describe("browser scenario network comparison stability", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("browser scenario network request times", () => {
|
||||
/** A two-request capture whose `a` request finished and whose `b` request failed with `failure`. */
|
||||
const ended = (failure = "net::ERR_ABORTED"): BrowserScenarioCapture => {
|
||||
const capture = concurrentCapture(["a", "b"], { a: 200, b: 404 });
|
||||
let sequence = Math.max(...capture.events.items.map((e) => e.sequence));
|
||||
const terminal = capture.events.items.flatMap((event) =>
|
||||
event.kind === "request"
|
||||
? [
|
||||
event.url.url.endsWith("/a.js")
|
||||
? {
|
||||
...event,
|
||||
sequence: ++sequence,
|
||||
kind: "request-finished" as const,
|
||||
}
|
||||
: {
|
||||
...event,
|
||||
sequence: ++sequence,
|
||||
kind: "request-failed" as const,
|
||||
failure,
|
||||
},
|
||||
]
|
||||
: [],
|
||||
);
|
||||
return browserScenarioCaptureSchema.parse({
|
||||
...capture,
|
||||
events: {
|
||||
...capture.events,
|
||||
retained: capture.events.items.length + terminal.length,
|
||||
items: [...capture.events.items, ...terminal],
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
/** The same capture with every request observed, started and timed `offset` ms later. */
|
||||
const timed = (
|
||||
capture: BrowserScenarioCapture,
|
||||
offset: number,
|
||||
): BrowserScenarioCapture =>
|
||||
browserScenarioCaptureSchema.parse({
|
||||
...capture,
|
||||
events: {
|
||||
...capture.events,
|
||||
items: capture.events.items.map((event) =>
|
||||
event.kind === "request"
|
||||
? {
|
||||
...event,
|
||||
observed_at: new Date(
|
||||
Date.parse("2026-10-24T05:00:00.000Z") + offset,
|
||||
).toISOString(),
|
||||
}
|
||||
: event.kind === "request-finished" ||
|
||||
event.kind === "request-failed"
|
||||
? {
|
||||
...event,
|
||||
started_at: new Date(
|
||||
Date.parse("2026-10-24T04:59:59.990Z") + offset,
|
||||
).toISOString(),
|
||||
elapsed_ms: 3 + offset,
|
||||
}
|
||||
: event,
|
||||
),
|
||||
},
|
||||
});
|
||||
|
||||
const compare = (
|
||||
before: BrowserScenarioCapture,
|
||||
after: BrowserScenarioCapture,
|
||||
) =>
|
||||
compareBrowserScenarios(
|
||||
compareBrowserScenariosInputSchema.parse({
|
||||
before_scenario: before,
|
||||
after_scenario: after,
|
||||
}),
|
||||
);
|
||||
|
||||
it("ignores when requests were observed, started and how long they took", () => {
|
||||
const early = timed(ended(), 0);
|
||||
const late = timed(ended(), 7_000);
|
||||
const terminal = early.events.items.filter(
|
||||
(event) =>
|
||||
event.kind === "request-finished" || event.kind === "request-failed",
|
||||
);
|
||||
expect(terminal.map(({ kind }) => kind).sort()).toEqual([
|
||||
"request-failed",
|
||||
"request-finished",
|
||||
]);
|
||||
for (const event of terminal)
|
||||
expect(event).toMatchObject({
|
||||
started_at: expect.any(String),
|
||||
elapsed_ms: expect.any(Number),
|
||||
});
|
||||
const compared = compare(early, late);
|
||||
expect(compared.overall_status).toBe("unchanged");
|
||||
expect(compared.normalization.built_in_rules).toContain(
|
||||
"ignore_network_request_timing",
|
||||
);
|
||||
// A capture made before request times were recorded compares with a timed one.
|
||||
expect(compare(ended(), late).overall_status).toBe("unchanged");
|
||||
});
|
||||
|
||||
it("still compares the rest of a timed completion event", () => {
|
||||
expect(
|
||||
compare(
|
||||
timed(ended(), 0),
|
||||
timed(ended("net::ERR_CONNECTION_RESET"), 7_000),
|
||||
).overall_status,
|
||||
).toBe("changed");
|
||||
});
|
||||
});
|
||||
|
||||
describe("scenario comparison recovery guidance", () => {
|
||||
it("retains network guidance beside a changed console event", () => {
|
||||
const before = scenarioCapture({});
|
||||
|
||||
@@ -137,7 +137,7 @@ export const compareBrowserScenarios = (
|
||||
limitations: [
|
||||
"Steps align only by exact, unique step_id; missing, duplicate, or action-mismatched steps are reported and prevent an unchanged claim.",
|
||||
"Only declared literal normalization rules are applied, in canonical rule_id order, to already-redacted durable capture fields.",
|
||||
"Elapsed time, event sequence/index values and capture-local transaction IDs are excluded as volatile; network transactions are compared as an unordered collection in the step that sent their request; cookies, local storage and session storage entries are compared as unordered collections after literal normalization; screenshot bytes are compared by content digest.",
|
||||
"Elapsed time, event sequence/index values, request observation times and durations, and capture-local transaction IDs are excluded as volatile; network transactions are compared as an unordered collection in the step that sent their request; cookies, local storage and session storage entries are compared as unordered collections after literal normalization; screenshot bytes are compared by content digest.",
|
||||
"A changed status proves an observed normalized difference. Missing or truncated evidence, incompatible capture context, or alignment failure prevents an unchanged claim.",
|
||||
],
|
||||
});
|
||||
|
||||
@@ -131,6 +131,7 @@ export const BROWSER_SCENARIO_BUILT_IN_NORMALIZATION_RULES = [
|
||||
"compare_normalized_text_content",
|
||||
"compare_storage_entries_as_unordered_collection",
|
||||
"ignore_capture_local_network_transaction_ids",
|
||||
"ignore_network_request_timing",
|
||||
"attribute_network_transactions_to_request_step",
|
||||
"compare_network_transactions_as_unordered_collection",
|
||||
"preserve_redirect_transaction_ancestry",
|
||||
|
||||
@@ -65,7 +65,7 @@ export const scenarioNetworkTransactions = (
|
||||
redirected_from_transaction_state: _state,
|
||||
...network
|
||||
} = event;
|
||||
evidence = network;
|
||||
evidence = withoutRequestTiming(network);
|
||||
predecessor =
|
||||
request === undefined ? parent : request.redirected_from_transaction_id;
|
||||
predecessorState =
|
||||
@@ -155,6 +155,21 @@ export const scenarioNetworkCoverageReason = (
|
||||
return undefined;
|
||||
};
|
||||
|
||||
/** Request times differ on every run, so comparisons leave them out. */
|
||||
const withoutRequestTiming = (event: object): object => {
|
||||
const {
|
||||
observed_at: _observed,
|
||||
started_at: _started,
|
||||
elapsed_ms: _elapsed,
|
||||
...rest
|
||||
} = event as {
|
||||
observed_at?: unknown;
|
||||
started_at?: unknown;
|
||||
elapsed_ms?: unknown;
|
||||
};
|
||||
return rest;
|
||||
};
|
||||
|
||||
/** Hash normalized own evidence and its complete parent fingerprint in linear graph space. */
|
||||
export const fingerprintScenarioNetworkTransactions = (
|
||||
nodes: readonly ScenarioNetworkTransaction[],
|
||||
|
||||
@@ -6,5 +6,5 @@ export const PACKAGE_METADATA = {
|
||||
serverSdkVersion: "2.3.1",
|
||||
clientSdkVersion: "2.3.1",
|
||||
coreSdkVersion: "2.3.1",
|
||||
skillVersion: "48",
|
||||
skillVersion: "49",
|
||||
} as const;
|
||||
Reference in new issue
Block a user