fix(environment): name entities and weigh interference per write as serving and the recording show

Greptile raised these on this branch, the fidelity branch and the list-lines branch; five follow from the previous
round's changes.
- An entity field whose name looks like a pointer is matched by name; only a document's pointers cover the members under them.
- A failed read contradicts each exercised write that no unmodelled write came after, instead of skipping them all when the oldest pending write preceded one; the shared pending `since` time goes.
- An item read, update or delete names its entity as serving's namedEntity does: held collections its path matches, verbatim spellings first, the id decoded once, then as written; `book%2531` no longer reaches `book1`.
- A plan whose calls use a member both as the identity and as a fixed member, or fix it at different values, is refused, as serving could not take every recorded call.
- A size of zero bounds nothing only on a line whose recording listed items under it (pageBound), in serving and in the gate.
- An update whose observation drops a field the entity held is refused, as serving keeps every field it does not write.
- A later create whose clock takes another format, or whose unique value is not a UUID, is refused.
- A derived state data/get would write past the 32 MiB intake limit is not served (admitServedState); the environment serves replay.

The verdicts over the 50 captures are unchanged plan by plan, and the smoke applies all 31 served write tools.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Edgar ArakelyanandClaude Opus 5.5 committed 2026-10-21 11:53:26 -07:00
1 parent 6257de93a1
commit 466ed8b433
17 files changed
+458 -117

No files matched your search

+2 -1
View File
@@ -404,7 +404,8 @@ A state that is unavailable does not stop the server: it logs a warning naming
the error `code` and message before it binds, then serves replay. The state is
unavailable when it cannot be derived, such as from a
`capture_browser_scenario` capture that selected header values but no response
bodies or over the view's budget, and when its catalog under the served name
bodies or over the view's budget, when `data/get` would write it past the 32 MiB
`data/add` intake limit, and when its catalog under the served name
exceeds the [discovery reservation](#what-a-manifest-is). That check runs
first, so no state is derived for it, and replay is then admitted exactly as
with `--replay-only`. `--replay-only`
@@ -105,7 +105,11 @@ the value a recorded call sent it with, and then stores the value that call
recorded (its `constant` rule carries it). The type's identity field never
takes a rule: sent as the path's id it is the identity member, sent as another
value the application ignored it is accepted with that value and stores
nothing, and a stored id that changed is `unexplained-change`.
nothing, and a stored id that changed is `unexplained-change`. A plan whose
calls use a member both as the identity and as a fixed member, or fix it at
different values, is `unexplained-change`, as serving could not take every
recorded call. So is an update whose observation drops a field the entity
held, since serving keeps every field an update does not write.
`PUT` requires every member the exemplar sent; `PATCH` and `POST` require at
least one. A field the exemplar's update changed without sending it is
explained as one of:
@@ -159,9 +163,10 @@ the initial state.
**Writes.** Only a 2xx write is modelled. A modelled write applies its effect, using the values it recorded
for generated fields and the stored values for fields it transformed. A
modelled write whose call the plan does not explain brings its entity to the
recorded outcome instead, without attribution. A later create whose sequence
fields hold another value than serving would generate makes the plan
`unexplained-change`; identities are left to the id mapping. An update or
recorded outcome instead, without attribution. A later create whose generated
values take another form than serving gives them (a sequence value out of
order, a clock in another format, a non-UUID unique value) makes the plan
`unexplained-change`; integer identities are left to the id mapping. An update or
delete names its entity as an item read does (below).
Any other write that may have changed state, including a redirected or
@@ -183,8 +188,9 @@ members they sent.
from the state:
- an item read of a state-backed item view, of the entity serving would name:
the read's collection and id as written, else a held collection whose path
matches once percent-decoded;
in a held collection the read's path matches, each segment verbatim or once
percent-decoded and verbatim spellings first, the id percent-decoded once,
then as written;
- a list or document read on one of its view's checked request lines,
chosen as replay chooses the line for the read's call: each path parameter
the read's value verbatim or once percent-decoded, preferring verbatim
@@ -194,7 +200,8 @@ from the state:
different values is never chosen, since no call's arguments can name it; one
repeating a name with one value is, as a call naming it matches both. A list
is compared as served from the collection of that line. A read on a page
(`limit`, `size`, `per_page` and the like, a size of zero bounding nothing, or a `_start`–`_end` range) of a
(`limit`, `size`, `per_page` and the like, or a `_start`–`_end` range; a size
of zero bounds nothing on a line whose recording listed items under it) of a
whole list checks nothing once the collection holds more entities than the
page, which serving then answers from its recording.
- A read made while a modelled write to its entity or collection was still
@@ -224,15 +231,17 @@ refuses to answer with.
instead), a recorded outcome every field, and a create or delete the whole
entity. A document write's effect is the members it merged at its
object, or everything under the object it replaces, the whole document at the
root. A write with nothing left on an entity no
root; an entity's fields match by name only, whatever their names look like. A write with nothing left on an entity no
longer counts there, nor in a collection where no entity it changed keeps any
of its effect. A mismatch contradicts them. A pass confirms those it shows
something of: a create's or delete's membership, or a changed field the
read's view shows and does not treat as volatile; a read that shows none of
an update's changes confirms nothing of it, and leaves it for a later check
of that entity or collection.
- A mismatch after an unmodelled write that followed those writes is checked
only on the members they sent: such a write may explain the rest.
- A mismatch contradicts each exercised write that no unmodelled write came
after, whatever earlier write is still pending. When an unmodelled write
followed some of them, an item read checks those only on the members they
sent and a list is not checked again: such a write may explain the rest.
- A mismatch on an entity or collection that no write changed contradicts,
as `side-effect`, the modelled writes since that previous check whose paths
are segment-prefix related to the read's path.
@@ -305,7 +314,9 @@ installs and of the type's checked list lines (so a collection the capture
listed empty keeps its spelling), never those of a state it replaced. When two
spellings fit a call equally, the one holding entities wins. `data/add` refuses a document that `data/get` would write back past the
32 MiB intake limit once it adds the members a document may omit, such as
`types` and the state Evidence id.
`types` and the state Evidence id. A derived state `data/get` would write
past that limit is not served: the environment serves replay and reports the
state `unavailable`.
**Clock and generators.** The store's clock starts at `write_clock` and
advances one second before each served write; clock fields take its value in
@@ -319,3 +319,22 @@ A fourteenth round settled two findings Greptile raised on the list-queries bran
Both tests fail on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke applies
all 31 served write tools, and the gates passed again.
A fifteenth round settled nine findings Greptile raised on this branch, the fidelity branch and the list-lines branch,
five of them on the previous rounds' own changes:
- an entity field whose name looks like a pointer is matched by name; only a document's pointers cover the members
under them;
- a failed read contradicts each exercised write no unmodelled write came after, rather than skipping all of them
when the oldest pending write came before one;
- an item read, update or delete names its entity as serving does, decoding the id once, so `book%2531` no longer
reaches `book1`;
- a plan whose calls use a member both as the identity and as a fixed member, or fix it at different values, is
refused;
- a size of zero bounds nothing only on a line whose recording listed items under it;
- an update whose observation drops a field the entity held is refused;
- a later create whose clock takes another format, or whose unique value is not a UUID, is refused;
- a derived state `data/get` would write past 32 MiB is not served.
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
applies all 31 served write tools, and the gates passed again.
@@ -24,6 +24,7 @@ import {
} from "../analysisView/AnalysisViewService.js";
import { loadEnvironmentSource } from "./environmentSource.js";
import {
admitServedState,
deriveEnvironmentState,
loadEnvironmentState,
prepareServedState,
@@ -272,6 +273,28 @@ describe("prepareServedState", () => {
});
});
describe("admitServedState", () => {
it("refuses a derived state data/get would write past what data/add loads back", () => {
expect(admitServedState(state, "environment_serve").ok).toBe(true);
// A euro sign is one UTF-16 unit but three bytes as data/get writes it.
const notes = "\u20ac".repeat(12_000_000);
const [first] = state.entities;
const large = {
...state,
entities: [
...state.entities,
{ ...first!, id: "large", fields: { ...first!.fields, notes } },
],
};
const admitted = admitServedState(large, "environment_serve");
if (admitted.ok) throw new Error("Expected the state to be refused");
expect(projectAnalysisError(admitted.error)).toMatchObject({
code: "resource_constraint",
message: expect.stringContaining("larger than data/add loads"),
});
});
});
describe("EnvironmentStateStore", () => {
it("reports its state as rea-state/v1, accepts that report back and resets to the initial state", () => {
const store = new EnvironmentStateStore(state);
@@ -8,9 +8,9 @@ import {
} from "../../domain/analysisView/environmentStateRules.js";
import type { StateWritePlan } from "../../domain/analysisView/environmentStateWriteRules.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type {
AnalysisInputIssue,
import {
AnalysisResourceConstraintError,
type AnalysisInputIssue,
} from "../../domain/analysisErrorCore.js";
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
import { err, ok, type Result } from "../../domain/result.js";
@@ -23,6 +23,8 @@ import {
type EnvironmentManifestTool,
type EnvironmentSource,
} from "./environmentSource.js";
import { STATE_INTAKE_LIMIT_BYTES } from "./environmentStateDocument.js";
import { initialStateBytes } from "./environmentStateStore.js";
import { describeWrite } from "./environmentStateWriteDescription.js";
import {
environmentToolDefinitions,
@@ -238,6 +240,29 @@ export interface PreparedState extends ServedState {
>;
}
/** The derived state, unless data/get would write it past what data/add loads back, so its own report could not be restored. */
export const admitServedState = (
state: EnvironmentState,
operation: string,
): Result<EnvironmentState, AnalysisResourceConstraintError> => {
const bytes = initialStateBytes(state);
return bytes > STATE_INTAKE_LIMIT_BYTES
? err(
new AnalysisResourceConstraintError(
operation,
"capacity",
"The derived state is larger than data/add loads, so it cannot be served",
{ max_state_bytes: STATE_INTAKE_LIMIT_BYTES, state_bytes: bytes },
{
retryable: false,
remediationAction:
"Capture fewer or smaller responses, or serve the environment as replay only.",
},
),
)
: ok(state);
};
const unavailable = (error: AnalysisError): PreparedState => {
const { code, message } = projectAnalysisError(error);
return {
@@ -262,6 +287,8 @@ export const prepareServedState = (
if (!reads.ok) return unavailable(reads.error);
const derived = deriveEnvironmentState(source, captureValue, operation);
if (!derived.ok) return unavailable(derived.error);
const admitted = admitServedState(derived.value, operation);
if (!admitted.ok) return unavailable(admitted.error);
// Served write tools join the catalog, so the whole catalog is admitted again.
const catalog = servedCatalog(source, derived.value, name, operation);
return catalog.ok
@@ -16,7 +16,7 @@ import {
type StateView,
} from "../../domain/analysisView/environmentStateRules.js";
import {
pageSize,
pageBound,
pagesOnly,
} from "../../domain/analysisView/environmentStatePaging.js";
import { isAbsence } from "../../domain/analysisView/environmentStateWriteChecks.js";
@@ -428,7 +428,9 @@ const answerList = (
const collection = collectionPathOf(base, false);
const type = store.type(view.type);
// A requested page the collection has outgrown is answered as recorded, never with more items than it asked for.
const size = exact ? pageSize(base.query ?? undefined) : undefined;
const size = exact
? pageBound(base.query ?? undefined, recorded.length)
: undefined;
if (
view.membership === "all" &&
size !== undefined &&
@@ -37,6 +37,27 @@ const itemsBytes = (items: readonly unknown[]): number =>
items.reduce<number>((sum, item) => sum + jsonBytes(item), 0) +
Math.max(0, items.length - 1);
/** What `data/get` reports for some content of a state. */
const snapshotOf = (
state: EnvironmentState,
entities: readonly StateEntity[],
documents: readonly HeldDocument[],
): StateSnapshot => ({
rea_state: {
version: 1,
state_evidence_id: state.evidenceId,
types: state.types,
entities,
documents,
},
});
/** What `data/get` writes for a state as the Evidence holds it, in UTF-8 bytes, summed item by item. */
export const initialStateBytes = (state: EnvironmentState): number =>
jsonBytes(snapshotOf(state, [], [])) +
itemsBytes(state.entities) +
itemsBytes(state.documents);
/**
* The live state one served environment answers from, shared by every MCP
* session and the data plane. Mutations are synchronous and swap the whole
@@ -74,8 +95,8 @@ export class EnvironmentStateStore {
);
this.#clockStart =
state.writeClock === null ? 0 : Date.parse(state.writeClock);
this.#frameBytes = jsonBytes(this.#snapshotOf([], []));
this.#initialBytes = this.#contentBytes(state);
this.#frameBytes = jsonBytes(snapshotOf(state, [], []));
this.#initialBytes = initialStateBytes(state);
this.reset();
}
@@ -303,7 +324,11 @@ export class EnvironmentStateStore {
}
snapshot(): StateSnapshot {
return this.#snapshotOf(this.#content.entities, this.#content.documents);
return snapshotOf(
this.state,
this.#content.entities,
this.#content.documents,
);
}
/** What `data/get` writes for the current state, in UTF-8 bytes. */
@@ -311,21 +336,6 @@ export class EnvironmentStateStore {
return this.#bytes;
}
#snapshotOf(
entities: readonly StateEntity[],
documents: readonly HeldDocument[],
): StateSnapshot {
return {
rea_state: {
version: 1,
state_evidence_id: this.state.evidenceId,
types: this.state.types,
entities,
documents,
},
};
}
/** What `data/get` writes for some content, summed item by item so the whole state is never one string. */
#contentBytes(content: StateContent): number {
return (
@@ -709,6 +709,26 @@ describe("paged lines", () => {
});
});
it("answers a zero limit a whole line recorded empty from that recording once a create fills its collection", () => {
const alpha = { id: 1, name: "Alpha", org: "1" };
const beta = { id: 2, name: "Beta", org: "2" };
const books = servedFrom([
["GET", "/api/orgs/1/books", [alpha]],
["GET", "/api/orgs/1/books/1", alpha],
["GET", "/api/orgs/2/books?limit=0", []],
["POST", "/api/orgs/2/books", beta, 201, { name: "Beta" }],
["GET", "/api/orgs/2/books/2", beta],
]);
books.answer("post_orgs_books_by_id", { id: "2", body: { name: "Gamma" } });
// Only a recording that listed items under it shows that zero lists every entity.
expect(
books.answer("get_orgs_books_by_id", { id: "2", limit: "0" }),
).toMatchObject({
body: [],
state: { source: "recording", reason: "page-exceeded" },
});
});
it("answers a range the collection has outgrown from its recording", () => {
const books = servedFrom([
["GET", "/api/books?_start=0&_end=2", [FIRST_BOOK, SECOND_BOOK]],
@@ -983,3 +983,180 @@ describe("what a later call must follow", () => {
});
});
});
describe("what a write's effect keeps", () => {
it("keeps an entity member a later write's other member does not cover, though both names look like pointers", () => {
const flat = (id: number, prefs: string, lang: string) => ({
id,
name: `Book ${String(id)}`,
"/prefs": prefs,
"/prefs/lang": lang,
});
const rows: TimedCaptureRow[] = [
["GET", "/api/books", [flat(1, "x", "en"), flat(2, "x", "en")]],
["GET", "/api/books/1", flat(1, "x", "en")],
["GET", "/api/books/2", flat(2, "x", "en")],
[
"PATCH",
"/api/books/1",
flat(1, "x", "fr"),
200,
{ "/prefs/lang": "fr" },
],
["PUT", "/api/books/1", flat(1, "y", "fr"), 200, { "/prefs": "y" }],
["GET", "/api/books/1", flat(1, "y", "fr")],
];
expect(gated(rows, "patch_books_by_id")).toMatchObject({
served: true,
checked_record_ordinals: [5],
});
});
it("contradicts a write made after an unmodelled one, though an earlier pending write came before it", () => {
const summary = (name: string) => [
{ id: 1, name },
{ id: 2, name: "B" },
];
const full = (name: string, description: string) => ({
id: 1,
name,
description,
});
const rows: TimedCaptureRow[] = [
["GET", "/api/books", summary("A")],
["GET", "/api/books/1", full("A", "x")],
["GET", "/api/books/2", { id: 2, name: "B", description: "y" }],
["PATCH", "/api/books/1", full("A", "new"), 200, { description: "new" }],
["GET", "/api/books", summary("A")],
["POST", "/api/exports", { queued: true }, 202, { format: "csv" }],
[
"PUT",
"/api/books/1",
full("Z", "new"),
200,
{ name: "Z", description: "new" },
],
["GET", "/api/books/1", full("Z", "new")],
// The export came before the rename, so it cannot explain the list showing the old name.
["GET", "/api/books", summary("A")],
];
expect(gated(rows, "put_books_by_id")).toMatchObject({
served: false,
contradicted_by: 8,
});
});
it("refuses an update whose recorded response dropped a field serving would keep", () => {
const rows: TimedCaptureRow[] = [
[
"GET",
"/api/books",
[
{ id: 1, name: "A" },
{ id: 2, name: "B" },
],
],
["GET", "/api/books/1", { id: 1, name: "A", subtitle: "s" }],
["GET", "/api/books/2", { id: 2, name: "B", subtitle: "t" }],
["PATCH", "/api/books/1", { id: 1, name: "Z" }, 200, { name: "Z" }],
[
"GET",
"/api/books",
[
{ id: 1, name: "Z" },
{ id: 2, name: "B" },
],
],
];
expect(gated(rows, "patch_books_by_id")).toMatchObject({
served: false,
reason: "unexplained-change",
});
});
});
describe("what later calls must keep consistent", () => {
it("refuses an update one call sends an ignored id with and another the path's id", () => {
const renamed = (at: Book, name: string) => ({ ...at, name });
const rows: TimedCaptureRow[] = [
...BOOK_INIT_ROWS,
[
"PATCH",
"/api/books/1",
renamed(FIRST_BOOK, "B"),
200,
{ id: 999, name: "B" },
],
["GET", "/api/books/1", renamed(FIRST_BOOK, "B")],
[
"PATCH",
"/api/books/2",
renamed(SECOND_BOOK, "C"),
200,
{ id: 2, name: "C" },
],
["GET", "/api/books/2", renamed(SECOND_BOOK, "C")],
];
expect(gated(rows, "patch_books_by_id")).toMatchObject({
served: false,
reason: "unexplained-change",
});
});
it("refuses a create whose later call records a clock in another format", () => {
const epoch = (iso: string) => Math.floor(Date.parse(iso) / 1000);
const fourth = {
...book(4, "Fourth Book", bookClock(5)),
created_at: epoch(bookClock(5)),
updated_at: epoch(bookClock(5)),
};
const rows: TimedCaptureRow[] = [
...BOOK_INIT_ROWS,
["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],
["GET", "/api/books/3", THIRD_BOOK],
["POST", "/api/books", fourth, 201, { name: "Fourth Book" }],
["GET", "/api/books/4", fourth],
];
expect(gated(rows, "post_books")).toMatchObject({
served: false,
reason: "unexplained-change",
});
});
it("names no entity by a delete whose path id decodes to it only twice, as serving would not", () => {
const uid = (key: string) => ({ uid: key, name: key });
const rows: TimedCaptureRow[] = [
["GET", "/api/books", [uid("book1"), uid("book2")]],
["GET", "/api/books/uid/book1", uid("book1")],
["GET", "/api/books/uid/book2", uid("book2")],
["DELETE", "/api/books/uid/book%2531", "", 204],
["GET", "/api/books/uid/book1", BOOK_NOT_FOUND, 404],
];
const { writes } = environmentStateViewOf(timedCaptureEvidence(rows), [
"after-uid-literal",
]);
expect(writes.find((plan) => plan.effect === "delete")).toMatchObject({
served: false,
checked_record_ordinals: [],
});
});
it("checks nothing by a whole line recorded empty under a zero limit once a create fills its collection", () => {
const alpha = { id: 1, name: "Alpha", org: "1" };
const beta = { id: 2, name: "Beta", org: "2" };
const rows: TimedCaptureRow[] = [
["GET", "/api/orgs/1/books", [alpha]],
["GET", "/api/orgs/1/books/1", alpha],
["GET", "/api/orgs/2/books?limit=0", []],
["POST", "/api/orgs/2/books", beta, 201, { name: "Beta" }],
["GET", "/api/orgs/2/books/2", beta],
// An empty recording does not show that zero lists every entity.
["GET", "/api/orgs/2/books?limit=0", []],
];
const { writes } = environmentStateViewOf(timedCaptureEvidence(rows));
expect(writes.find((plan) => plan.effect === "create")).toMatchObject({
served: true,
checked_record_ordinals: [4],
});
});
});
@@ -88,7 +88,7 @@ export class EnvironmentStateWriteSimulation {
readonly #touched: TouchedTargets = new Map();
readonly #effects = new WriteEffects();
readonly #lastCheck = new Map<string, number>();
/** The earliest modelled write to each target since its last check, and when the latest of them ended. */
/** When the latest modelled write to each target since its last check ended. */
readonly #touchedSince: PendingTargets = new Map();
/** Timeline indexes of writes the gate does not model. */
readonly #unmodelledAt: number[] = [];
@@ -236,22 +236,18 @@ export class EnvironmentStateWriteSimulation {
result.entity.collection,
result.entity.id,
);
if (result.kind === "applied")
this.#effects.supersede(this.#touched, write.tool, result);
if (result.kind === "observed")
if (result.kind === "observed") {
this.#effects.observed(this.#touched, result.targets);
for (const target of result.targets) {
// A recorded outcome replaces the entity's values, not its collection's membership.
if (result.kind === "observed") {
if (target === result.targets[0]) this.#touched.delete(target);
continue;
}
this.#touched.delete(result.targets[0]!);
return;
}
this.#effects.supersede(this.#touched, write.tool, result);
for (const target of result.targets)
touchTarget(this.#touched, this.#touchedSince, target, {
...write,
fields: result.fields,
});
}
if (result.kind !== "applied") return;
this.#log.push(write);
if (result.inserted !== undefined)
this.#inserted.set(result.inserted, write.tool);
@@ -362,7 +358,7 @@ export class EnvironmentStateWriteSimulation {
return;
const exercised = exercisedBy(this.#touched, keys, borrowers);
if (borrowers !== undefined) {
this.#contradictOthers(outcome.pass, keys, exercised, record);
this.#contradictOthers(outcome.pass, exercised, record);
return;
}
const pass =
@@ -372,11 +368,17 @@ export class EnvironmentStateWriteSimulation {
? this.#confirm(view, record, keys, exercised)
: undefined;
if (!pass) {
if (!this.#interfered(keys))
const interfered = [...exercised].filter((name) =>
this.#interferedWith(name),
);
if (interfered.length === 0)
this.#mismatch({ record, index, keys, exercised, view });
// An unmodelled write since these writes may explain the difference: check their own members only.
// An unmodelled write after some of these writes may explain the difference: those are checked on their own members only.
else if (view.shape === "item") this.#checkMembers(view, record);
else for (const key of keys) this.#taint.fail(key);
else {
this.#contradictOthers(false, exercised, record);
for (const key of keys) this.#taint.fail(key);
}
if (view.shape === "item") this.#adopt(view, record);
}
const pending = {
@@ -390,16 +392,16 @@ export class EnvironmentStateWriteSimulation {
}
}
/** A read a write's effect was learned from checks nothing of that write, yet still contradicts the others. */
/** A failed read contradicts the writes it exercised that no unmodelled write came after, as a read a write's effect was learned from still does the others. */
#contradictOthers(
pass: boolean,
keys: readonly string[],
exercised: ReadonlySet<string>,
record: StateRecordFacts,
): void {
if (pass || this.#interfered(keys)) return;
if (pass) return;
for (const name of exercised)
this.#drafts.get(name)!.contradictedBy ??= record.ordinal;
if (!this.#interferedWith(name))
this.#drafts.get(name)!.contradictedBy ??= record.ordinal;
}
/** A passing read confirms the writes it exercised that it shows something of, and names, per target, those it shows nothing of. */
@@ -426,12 +428,11 @@ export class EnvironmentStateWriteSimulation {
this.#unmodelledAt.push(index);
}
/** Whether a write the gate does not model came after the earliest modelled write a failed check exercised. */
#interfered(keys: readonly string[]): boolean {
const since = Math.min(
...keys.map((key) => this.#touchedSince.get(key)?.since ?? Infinity),
);
return this.#unmodelledAt.some((index) => index > since);
/** Whether a write the gate does not model came after a tool's latest modelled call, so it may explain what a read shows of that call. */
#interferedWith(tool: string): boolean {
const last =
this.#log.findLast((write) => write.tool === tool)?.index ?? -1;
return this.#unmodelledAt.some((index) => index > last);
}
/** Check the writes to an item read's entity on the members they sent, as a resynchronising read does. */
@@ -42,12 +42,19 @@ export const pageSize = (query: string | undefined): number | undefined => {
for (const [name, value] of iterateUrlQueryParameters(query ?? "")) {
const key = name.toLowerCase();
if (!/^\d+$/u.test(value)) continue;
// A size of zero bounds nothing: applications read it as no limit, as a whole line naming it shows.
if (PAGE_SIZE_PARAMETERS.has(key)) {
if (Number(value) > 0) size = Math.min(size, Number(value));
} else if (key === "_start") range.start = Number(value);
if (PAGE_SIZE_PARAMETERS.has(key)) size = Math.min(size, Number(value));
else if (key === "_start") range.start = Number(value);
else if (key === "_end") range.end = Number(value);
}
const width = Math.min(size, Math.max(range.end - range.start, 0));
return Number.isFinite(width) ? width : undefined;
};
/** The most items a line's call lists: its page size, but none for a size of zero the line's own recording listed items under, as an application reading zero as no limit does. */
export const pageBound = (
query: string | undefined,
recorded: number,
): number | undefined => {
const size = pageSize(query);
return size === 0 && recorded > 0 ? undefined : size;
};
@@ -13,8 +13,8 @@ export const shownFields = (
/** Modelled writes per target since its last check, with the fields each wrote (null: all). */
export type TouchedTargets = Map<string, Map<string, readonly string[] | null>>;
/** Per target, the earliest modelled write since its last check and when the latest of them ended. */
export type PendingTargets = Map<string, { since: number; until: number }>;
/** Per target, when the latest modelled write since its last check ended. */
export type PendingTargets = Map<string, { until: number }>;
/** Whether a modelled write to the target was still running when a read began at `start`, so the read may show either state. */
export const runningAt = (
@@ -36,22 +36,19 @@ export const exercisedBy = (
return exercised;
};
/** Record a modelled write on a target: the fields it wrote (undefined: all), when it began and when it ended. */
/** Record a modelled write on a target: the fields it wrote (undefined: all) and when it ended. */
export const touchTarget = (
touched: TouchedTargets,
pending: PendingTargets,
target: string,
write: {
readonly index: number;
readonly tool: string;
readonly end: number;
readonly fields: readonly string[] | undefined;
},
): void => {
const since = pending.get(target);
pending.set(target, {
since: since?.since ?? write.index,
until: Math.max(since?.until ?? -Infinity, write.end),
until: Math.max(pending.get(target)?.until ?? -Infinity, write.end),
});
const tools = touched.get(target) ?? new Map();
const before = tools.get(write.tool);
@@ -71,14 +68,17 @@ type ChangedFields = readonly string[] | null;
interface Overwrite {
readonly whole: boolean;
readonly fields: ReadonlySet<string> | "all";
readonly pointers: boolean;
}
/** Whether a write's fields cover a field: the same one, or, for a document, a pointer above it. */
const covers = (fields: ReadonlySet<string>, field: string): boolean =>
fields.has(field) ||
[...fields].some(
(over) => over.startsWith("/") && field.startsWith(`${over}/`),
/** Whether a write's fields cover a field: the same one, or, for a document's pointers, one above it. */
const covers = (write: Overwrite, field: string): boolean => {
const { fields } = write;
if (fields === "all" || fields.has(field)) return true;
return (
write.pointers && [...fields].some((over) => field.startsWith(`${over}/`))
);
};
/** What an earlier effect keeps after a write: nothing once every field it holds is overwritten; an update leaves a create's or delete's whole effect. */
const remaining = (
@@ -87,9 +87,8 @@ const remaining = (
): ChangedFields | undefined => {
if (write.whole) return undefined;
if (claim === null) return null;
const { fields } = write;
if (fields === "all") return undefined;
const left = claim.filter((field) => !covers(fields, field));
if (write.fields === "all") return undefined;
const left = claim.filter((field) => !covers(write, field));
return left.length > 0 ? left : undefined;
};
@@ -160,6 +159,8 @@ export class WriteEffects {
readonly fields: readonly string[];
readonly accumulated: readonly string[];
};
/** Set when the fields are a document's pointers. */
readonly pointers?: boolean;
},
): void {
const [entity, collection] = effect.targets as [string, string?];
@@ -175,6 +176,7 @@ export class WriteEffects {
(field) => !effect.changed?.accumulated.includes(field),
),
),
pointers: effect.pointers === true,
},
);
this.#changed.set(
@@ -196,7 +198,7 @@ export class WriteEffects {
this.#overwrite(
touched,
{ entity, collection, writer: undefined },
{ whole: false, fields: "all" },
{ whole: false, fields: "all", pointers: false },
);
}
@@ -6,11 +6,9 @@ import {
} from "../unicodeCodePointOrder.js";
import {
at,
collectionPathOf,
decodedSegment,
isObject,
isParam,
lastSegmentValue,
listItems,
missingRequiredFields,
renderStateEntity,
@@ -28,7 +26,7 @@ import {
type StateToolFacts,
type StateView,
} from "./environmentStateRules.js";
import { pagesOnly, pageSize } from "./environmentStatePaging.js";
import { pageBound, pagesOnly } from "./environmentStatePaging.js";
/** A query as replay compares a call's: decoded values by name, in any order; undefined when it repeats a name with another value, which no call's arguments can carry. */
const queryKey = (query: string | undefined): string | undefined => {
@@ -253,35 +251,48 @@ interface ReadBodies {
}
/** An item read: the answer the held entity renders into the served envelope must equal the recording; a recorded 404 or 410 agrees with none held. */
/** How many of a held collection's segments match a path's verbatim, or undefined when one matches neither verbatim nor once percent-decoded. */
const segmentMatch = (
held: readonly string[],
read: readonly string[],
): number | undefined => {
if (held.length !== read.length) return undefined;
let verbatim = 0;
for (const [index, segment] of held.entries())
if (segment === read[index]) verbatim += 1;
else if (decodedSegment(segment) !== read[index]) return undefined;
return verbatim;
};
/**
* The entity an item read names, resolved as serving resolves an item call: its collection and id as written, else a
* held collection whose path matches once percent-decoded and an id tried decoded, then as written.
* The entity an item read or write names, as serving names it: in a held collection its path matches, verbatim
* spellings first, the id percent-decoded once, then as written; else the collection and decoded id as written.
*/
export const itemTarget = (
state: SimulatedState,
type: string,
record: Pick<StateRecordFacts, "path">,
): { readonly collection: string; readonly id: string } => {
const collection = collectionPathOf(record, true);
const id = lastSegmentValue(record.path);
if (state.entity(type, collection, id) !== undefined)
return { collection, id };
const read = segmentsOf(collection);
const ids = [decodedSegment(id), id];
for (const candidate of ids)
for (const entity of state.ofType(type)) {
const held = segmentsOf(entity.collection);
if (
entity.id === candidate &&
held.length === read.length &&
held.every(
(segment, index) =>
segment === read[index] || decodedSegment(segment) === read[index],
)
)
return { collection: entity.collection, id: entity.id };
}
return { collection, id };
const read = segmentsOf(record.path);
const raw = read.pop() ?? "";
const ids = [...new Set([decodedSegment(raw), raw])];
const written = `/${read.join("/")}`;
for (const id of ids)
if (state.entity(type, written, id) !== undefined)
return { collection: written, id };
const collections = [
...new Set(state.ofType(type).map((entity) => entity.collection)),
]
.flatMap((collection) => {
const verbatim = segmentMatch(segmentsOf(collection), read);
return verbatim === undefined ? [] : [{ collection, verbatim }];
})
.sort((left, right) => right.verbatim - left.verbatim);
for (const { collection } of collections)
for (const id of ids)
if (state.entity(type, collection, id) !== undefined)
return { collection, id };
return { collection: written, id: ids[0]! };
};
export const checkItemRead = (
@@ -403,19 +414,19 @@ export const checkListRead = (
check: ReadBodies & { readonly idField: string; readonly collection: string },
): CheckOutcome | undefined => {
if (!succeeded(record)) return undefined;
const { body, base } = check;
const baseItems =
base === undefined ? undefined : listItems(base, view.pointer);
if (body === undefined || base === undefined || baseItems === undefined)
return undefined;
// Serving answers a page too small for the collection from its recording, so such a read checks nothing.
const size = pageSize(record.query);
const size = pageBound(record.query, baseItems.length);
if (
view.membership === "all" &&
size !== undefined &&
state.members(view.type, check.collection).length > size
)
return undefined;
const { body, base } = check;
const baseItems =
base === undefined ? undefined : listItems(base, view.pointer);
if (body === undefined || base === undefined || baseItems === undefined)
return undefined;
// A usable body that no longer holds the list is not what serving answers on this line.
const items = listItems(body, view.pointer);
const bodies = { ...check, body, base, items: items ?? [], baseItems };
@@ -40,7 +40,7 @@ import {
type Observed,
} from "./environmentStateWriteObserve.js";
import type { FieldRule, WriteCall } from "./environmentStateWriteRules.js";
import { integerOf } from "./environmentStateWriteValues.js";
import { integerOf, isUuid, readClock } from "./environmentStateWriteValues.js";
/** The id a response names alone: a scalar body, or an object's id field or single scalar member. */
const respondedId = (
@@ -186,6 +186,25 @@ const followsSequences = (
return integerOf(created.fields[rule.field]) === next;
});
/** Whether a later call's generated values take the forms serving generates them in: clocks in the learned format, unique values and UUID identities as UUIDs, and sequences in order. */
const followsGenerated = (
context: EffectContext,
draft: WriteDraft,
created: Pick<StateEntity, "type" | "collection" | "fields">,
): boolean =>
draft.plan.rules.every((rule) => {
const value = created.fields[rule.field];
if (rule.rule === "clock")
return (
JSON.stringify(readClock(value)?.format) === JSON.stringify(rule.format)
);
return (
(rule.rule !== "unique" &&
(rule.rule !== "identity" || rule.generator !== "uuid")) ||
isUuid(value)
);
}) && followsSequences(context, draft, created);
/** Record the values a create's sequences took, which the next call follows. */
const takeSequences = (
draft: WriteDraft,
@@ -275,7 +294,7 @@ export const applyCreate = (
id === undefined ||
context.state.entity(type, collection, id) !== undefined ||
(later &&
!followsSequences(context, draft, {
!followsGenerated(context, draft, {
type,
collection,
fields: fields.value,
@@ -134,6 +134,6 @@ export const applyDocument = (
return {
kind: "applied",
targets: [documentTarget(tool)],
...(fields === undefined ? {} : { fields }),
...(fields === undefined ? {} : { fields, pointers: true }),
};
};
@@ -122,8 +122,10 @@ export type EffectResult =
readonly kind: "applied";
readonly targets: readonly string[];
readonly entity?: WrittenEntity;
/** The fields an update wrote; a create writes the whole entity. */
/** The fields an update wrote, or the pointers a document write wrote; a create writes the whole entity. */
readonly fields?: readonly string[];
/** Set when the fields are a document's pointers, each covering the members under it. */
readonly pointers?: boolean;
/** The fields an update sent or changed, derived ones included, and those of them its increments add to. */
readonly changed?: {
readonly fields: readonly string[];
@@ -301,6 +301,9 @@ export const explainUpdate = (
value: evidence.after[member]!,
});
}
// Serving keeps every field an update does not write, so one the update removed is a change no rule models.
for (const field of Object.keys(evidence.before))
if (!Object.hasOwn(evidence.after, field)) unexplained.push(field);
for (const field of Object.keys(evidence.after)) {
if (
Object.hasOwn(body, field) ||
@@ -448,13 +451,19 @@ export const conflictsWithPlan = (
explained: ReturnType<typeof explainUpdate>,
): boolean => {
const body = plan.body!;
const fixed = new Set(body.fixed.map((entry) => entry.member));
const fixed = new Map(body.fixed.map((entry) => [entry.member, entry.sent]));
const rules = new Map(plan.rules.map((rule) => [rule.field, rule]));
const { identity } = explained.body;
// Serving checks a call's identity member before its fixed values, so a member both cannot take every recorded call.
return (
explained.body.accepted.some((member) => fixed.has(member)) ||
explained.body.fixed.some((entry) =>
body.accepted.includes(entry.member),
explained.body.fixed.some(
(entry) =>
body.accepted.includes(entry.member) ||
entry.member === body.identity ||
(fixed.has(entry.member) && !same(fixed.get(entry.member), entry.sent)),
) ||
(identity !== null && fixed.has(identity)) ||
explained.rules.some((rule) => {
const held = rules.get(rule.field);
return held !== undefined && !same(held, rule);