fix(environment): keep unseen writes pending, and hold later calls to their sequences and document objects

Greptile raised these on this branch, the fidelity branch and the list-lines branch.
- A passing read that shows nothing of a write leaves it pending for a later check instead of clearing it, so a summary list no longer hides a description PATCH a later item read shows.
- A read begun while a modelled write to an unknown entity still ran neither checks nor resynchronises the entity.
- A later create whose sequence fields hold another value than serving would generate (one past the largest held or started-with value, and past the plan's last) refuses the plan.
- A document replacement below the root overwrites only the claims under its object, and a built document plan writes later calls at its own object, even one sharing no member with the document.
- An update or delete finds its entity as an item read does, so a plain spelling of a percent-encoded held collection no longer leaves the tool without an observation.
- A `_start`-`_end` range bounds a page as `limit` does, in serving and in the gate.

The document write moves to environmentStateWriteDocument.ts, the pending-write bookkeeping and the gate's input type move beside the code that uses them, and the item-read field check takes the list-lines branch's shape, keeping files under the line limit. The verdicts over the 50 captures are unchanged plan by plan, and the smoke applies all 31 served write tools.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Edgar ArakelyanandClaude Opus 5.5 committed 2026-10-21 10:43:01 -07:00
1 parent 43d98ea941
commit 264ce41062
12 files changed
+584 -191

No files matched your search

@@ -123,7 +123,7 @@ response, with the entity re-rendered when the response carries it.
**Documents.** A document write merges the body's members (replaces them, for
`PUT`) into the document object that holds most of them: the root or one
member. Document reads answer only on their recorded line, so a write is
member. Later calls write at the exemplar's object. Document reads answer only on their recorded line, so a write is
served only at the path the state holds the document at, and the gate treats a
recorded write at another path as an unmodelled write. A recorded write whose
body serving would refuse (a member of another JSON type than the document
@@ -156,7 +156,10 @@ the initial state.
**Writes.** Only a 2xx write is modelled. A modelled write applies its effect, using the values it recorded
for generated fields and the stored values for fields it transformed. A
modelled write whose call the plan does not explain brings its entity to the
recorded outcome instead, without attribution.
recorded outcome instead, without attribution. A later create whose sequence
fields hold another value than serving would generate makes the plan
`unexplained-change`; identities are left to the id mapping. An update or
delete names its entity as an item read does (below).
Any other write that may have changed state, including a redirected or
unanswered one, is unmodelled, and what it may have changed
@@ -188,12 +191,13 @@ from the state:
different values is never chosen, since no call's arguments can name it; one
repeating a name with one value is, as a call naming it matches both. A list
is compared as served from the collection of that line. A read on a page
(`limit`, `size`, `per_page` and the like) of a whole list checks nothing
once the collection holds more entities than the page, which serving then
answers from its recording.
(`limit`, `size`, `per_page` and the like, or a `_start`–`_end` range) of a
whole list checks nothing once the collection holds more entities than the
page, which serving then answers from its recording.
- A read made while a modelled write to its entity or collection was still
running, starting before that write ended, checks nothing: it may show the
state before or after the write.
state before or after the write. Nor does such a read show an unknown entity
again.
- A read a write's effect was learned from checks nothing of that write, but
still contradicts the other writes it exercises.
@@ -215,13 +219,15 @@ refuses to answer with.
An update's effect is the fields it sent or changed, derived ones included.
A later update overwrites the fields it sets (an increment adds to its field
instead), a recorded outcome every field, and a create or delete the whole
entity. A document write's effect is the members it merged, or the whole
document when it replaces it. A write with nothing left on an entity no
entity. A document write's effect is the members it merged at its
object, or everything under the object it replaces, the whole document at the
root. A write with nothing left on an entity no
longer counts there, nor in a collection where no entity it changed keeps any
of its effect. A mismatch contradicts them. A pass confirms those it shows
something of: a create's or delete's membership, or a changed field the
read's view shows and does not treat as volatile; a read that shows none of
an update's changes confirms nothing of it.
an update's changes confirms nothing of it, and leaves it for a later check
of that entity or collection.
- A mismatch after an unmodelled write that followed those writes is checked
only on the members they sent: such a write may explain the rest.
- A mismatch on an entity or collection that no write changed contradicts,
@@ -297,3 +297,16 @@ A twelfth round settled four findings Greptile raised on the list-lines branch a
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
applies all 31 served write tools, and the gates passed again.
A thirteenth round settled seven findings Greptile raised on this branch, the fidelity branch and the list-lines branch:
- a passing read that shows nothing of a write leaves it for a later check, instead of clearing it;
- a read begun while a write to an unknown entity still ran neither checks nor shows the entity again;
- a later create whose sequence fields do not follow the learned sequence refuses the plan;
- a document replacement below the root overwrites only what it replaced;
- a built document plan writes later calls at its own object, even one sharing no member with the document;
- an update or delete finds its entity as an item read does, percent-decoded spellings included;
- a `_start`–`_end` range bounds a page as `limit` does, in serving and in the gate.
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
applies all 31 served write tools, and the gates passed again.
@@ -670,4 +670,22 @@ describe("paged lines", () => {
state: { source: "state", membership: "all", listed: 3 },
});
});
it("answers a range the collection has outgrown from its recording", () => {
const books = servedFrom([
["GET", "/api/books?_start=0&_end=2", [FIRST_BOOK, SECOND_BOOK]],
["GET", "/api/books/1", FIRST_BOOK],
["GET", "/api/books/2", SECOND_BOOK],
["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],
["GET", "/api/books?_start=0&_end=2", [FIRST_BOOK, SECOND_BOOK]],
["GET", "/api/books/3", THIRD_BOOK],
]);
books.answer("post_books", CREATE_NEW_BOOK);
expect(books.answer("get_books", { _start: "0", _end: "2" })).toMatchObject(
{
body: [FIRST_BOOK, SECOND_BOOK],
state: { source: "recording", reason: "page-exceeded" },
},
);
});
});
@@ -812,3 +812,174 @@ it("counts the sent values a plan keeps for fixed members against the view's all
retainFixed(context, []);
expect(retained).toBe(0);
});
describe("what a check leaves pending", () => {
const summary = (books: readonly Book[]): JsonValue =>
books.map(({ id, name }) => ({ id: id ?? null, name: name ?? null }));
const described = (id: number, name: string, description: string) => ({
id,
name,
description,
});
it("keeps an update a passing read showed nothing of for a later read to confirm", () => {
const rows: TimedCaptureRow[] = [
["GET", "/api/books", summary([FIRST_BOOK, SECOND_BOOK])],
["GET", "/api/books/1", described(1, "First Book", "x")],
["GET", "/api/books/2", described(2, "Second Book", "y")],
[
"PATCH",
"/api/books/1",
described(1, "First Book", "new"),
200,
{ description: "new" },
],
// The list passes but shows no description; the item read does.
["GET", "/api/books", summary([FIRST_BOOK, SECOND_BOOK])],
["GET", "/api/books/1", described(1, "First Book", "new")],
];
expect(gated(rows, "patch_books_by_id")).toMatchObject({
served: true,
checked_record_ordinals: [5],
});
});
it("neither resynchronises nor checks an entity by a read begun while a write to it still ran", () => {
const rows: TimedCaptureRow[] = [
["GET", "/api/books", [FIRST_BOOK, SECOND_BOOK]],
["GET", "/api/books/1", FIRST_BOOK],
["GET", "/api/books/2", SECOND_BOOK],
// The rename runs from second 3 to 5.5, and an unmodelled action leaves book 1 unknown.
["PATCH", "/api/books/1", RENAMED, 200, { name: "Renamed Book" }, 2500],
["POST", "/api/books/1/archive", null, 204, {}],
["GET", "/api/books/1", FIRST_BOOK],
["GET", "/api/books/1", RENAMED],
];
expect(gated(rows, "patch_books_by_id")).toMatchObject({
served: true,
checked_record_ordinals: [6],
contradicted_by: null,
});
});
it("checks nothing by a range read the collection has outgrown", () => {
const rows: TimedCaptureRow[] = [
["GET", "/api/books?_start=0&_end=2", [FIRST_BOOK, SECOND_BOOK]],
["GET", "/api/books/1", FIRST_BOOK],
["GET", "/api/books/2", SECOND_BOOK],
["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],
// The application still answers the range with two books.
["GET", "/api/books?_start=0&_end=2", [FIRST_BOOK, SECOND_BOOK]],
["GET", "/api/books/3", THIRD_BOOK],
];
expect(gated(rows, "post_books")).toMatchObject({
served: true,
checked_record_ordinals: [5],
});
});
});
describe("what a later call must follow", () => {
it("refuses a create whose later call does not follow the sequence it learned", () => {
const ranked = (at: Book, rank: number) => ({ ...at, rank });
const third = ranked(book(3, "Third Book", bookClock(3)), 3);
const fourth = ranked(book(4, "Fourth Book", bookClock(4)), 3);
const rows: TimedCaptureRow[] = [
["GET", "/api/books", [ranked(FIRST_BOOK, 1), ranked(SECOND_BOOK, 2)]],
["GET", "/api/books/1", ranked(FIRST_BOOK, 1)],
["GET", "/api/books/2", ranked(SECOND_BOOK, 2)],
["POST", "/api/books", third, 201, { name: "Third Book" }],
// Serving ranks the next create 4; the application recorded 3 again.
["POST", "/api/books", fourth, 201, { name: "Fourth Book" }],
[
"GET",
"/api/books",
[ranked(FIRST_BOOK, 1), ranked(SECOND_BOOK, 2), third, fourth],
],
];
expect(gated(rows, "post_books")).toMatchObject({
served: false,
reason: "unexplained-change",
});
});
it("keeps a document write a later replacement of another member left in place", () => {
const rows: TimedCaptureRow[] = [
["GET", "/api/settings", { theme: "light", prefs: { lang: "en" } }],
[
"PATCH",
"/api/settings",
{ theme: "dark", prefs: { lang: "en" } },
200,
{ theme: "dark" },
],
// The replacement holds `lang`, so it replaces `prefs` alone.
[
"PUT",
"/api/settings",
{ theme: "dark", prefs: { lang: "fr" } },
200,
{ lang: "fr" },
],
["GET", "/api/settings", { theme: "dark", prefs: { lang: "fr" } }],
];
expect(gated(rows, "patch_settings")).toMatchObject({
served: true,
checked_record_ordinals: [3],
});
expect(gated(rows, "put_settings")).toMatchObject({
served: true,
checked_record_ordinals: [3],
});
});
it("applies a later document replacement at the plan's pointer, though it shares no member with the document", () => {
const rows: TimedCaptureRow[] = [
["GET", "/api/settings", { theme: "light", lang: "en" }],
[
"PUT",
"/api/settings",
{ theme: "dark", lang: "fr" },
200,
{ theme: "dark", lang: "fr" },
],
["PUT", "/api/settings", { theme: "blue" }, 200, { theme: "blue" }],
["PUT", "/api/settings", { lang: "de" }, 200, { lang: "de" }],
["GET", "/api/settings", { lang: "de" }],
];
expect(gated(rows, "put_settings")).toMatchObject({
served: true,
checked_record_ordinals: [4],
});
});
it("finds an update's entity as serving does when the write spells the collection plainly", () => {
const alpha = { id: 1, org: "acme1", name: "Alpha" };
const beta = { id: 2, org: "beta2", name: "Beta" };
const renamed = { ...alpha, name: "Renamed" };
const renamedBeta = { ...beta, name: "Renamed Beta" };
const rows: TimedCaptureRow[] = [
["GET", "/api/orgs/%61cme1/books", [alpha]],
["GET", "/api/orgs/beta2/books", [beta]],
["GET", "/api/orgs/%61cme1/books/1", alpha],
["GET", "/api/orgs/beta2/books/2", beta],
["PATCH", "/api/orgs/acme1/books/1", renamed, 200, { name: "Renamed" }],
[
"PATCH",
"/api/orgs/beta2/books/2",
renamedBeta,
200,
{ name: "Renamed Beta" },
],
["GET", "/api/orgs/%61cme1/books/1", renamed],
["GET", "/api/orgs/beta2/books/2", renamedBeta],
];
const { writes } = environmentStateViewOf(timedCaptureEvidence(rows), [
"sibling-variance",
]);
expect(writes.find((plan) => plan.effect === "update")).toMatchObject({
served: true,
checked_record_ordinals: [6, 7],
});
});
});
@@ -4,7 +4,7 @@ import {
isObject,
isStateWrite,
normalizedPath,
type StateDocument,
type JsonObject,
type StateEntity,
type StateRecordFacts,
type StateToolFacts,
@@ -32,22 +32,24 @@ import {
} from "./environmentStateWriteChecks.js";
import {
exercisedBy,
forgetChecked,
runningAt,
shownFields,
shownWrites,
touchTarget,
WriteEffects,
type PendingTargets,
type TouchedTargets,
} from "./environmentStateWriteAttribution.js";
import { applyCreate } from "./environmentStateWriteCreate.js";
import { applyDocument } from "./environmentStateWriteDocument.js";
import {
applyDelete,
applyDocument,
applyUpdate,
type EffectContext,
type EffectResult,
type WriteDraft,
type WriteIo,
type WriteSimulationInput,
type WriteStep,
} from "./environmentStateWriteEffects.js";
import { insertIndex } from "./environmentStateWriteExplain.js";
@@ -60,19 +62,6 @@ import {
import { WriteTaint } from "./environmentStateWriteTaint.js";
import { writeVerdict } from "./environmentStateWriteVerdict.js";
/** What the write gate reads: the manifest tools, the timeline, the read views and the initial state. */
export interface WriteSimulationInput {
readonly tools: readonly StateToolFacts[];
readonly records: readonly StateRecordFacts[];
readonly timed: boolean;
readonly views: ReadonlyMap<string, StateView>;
readonly types: readonly StateType[];
readonly entities: readonly StateEntity[];
readonly documents: readonly StateDocument[];
readonly volatileOf: (type: string) => ReadonlySet<string>;
readonly io: WriteIo;
}
const APPLY = {
create: applyCreate,
update: applyUpdate,
@@ -155,6 +144,7 @@ export class EnvironmentStateWriteSimulation {
sideEffect: false,
calls: 0,
updates: [],
sequences: new Map(),
});
}
this.#input.records.forEach((record, index) => this.#step(record, index));
@@ -194,6 +184,8 @@ export class EnvironmentStateWriteSimulation {
this.#taint.unknown(type, collection, id),
initialDocument: (tool) =>
this.#input.documents.find((document) => document.tool === tool),
startedWith: (type) =>
this.#input.entities.filter((entity) => entity.type === type),
};
}
@@ -282,20 +274,9 @@ export class EnvironmentStateWriteSimulation {
this.#state.remove(entity);
return;
}
const object = this.#input.io.withBody(record.ordinal, (body) =>
body === undefined ? undefined : at(body, view.pointer),
);
if (!isObject(object)) return;
const outcomes = writtenFieldChecks(entity, view, object, {
volatile: this.#input.volatileOf(view.type),
writes,
});
for (const [name, pass] of outcomes) {
const draft = this.#drafts.get(name)!;
if (pass) draft.confirmed.push(record.ordinal);
else draft.contradictedBy ??= record.ordinal;
}
this.#state.put(updatedEntity(entity, { ...entity.fields, ...object }));
const object = this.#checkFields(view, record, entity, writes);
if (object !== undefined)
this.#state.put(updatedEntity(entity, { ...entity.fields, ...object }));
}
/** Whether an item read is of an entity left unknown that it shows again rather than checks. */
@@ -303,6 +284,9 @@ export class EnvironmentStateWriteSimulation {
if (view.shape !== "item") return false;
const { collection, id } = itemTarget(this.#state, view.type, record);
if (!this.#taint.unknown(view.type, collection, id)) return false;
// A read begun while a modelled write to the entity still ran may show either state, so it neither checks nor shows it.
const target = entityTarget(view.type, collection, id);
if (runningAt(this.#touchedSince, target, record.start)) return true;
const entity = this.#state.entity(view.type, collection, id);
// A recorded absence of an entity the state no longer holds is still a check of its delete.
if (entity === undefined) {
@@ -384,18 +368,25 @@ export class EnvironmentStateWriteSimulation {
const pass =
outcome.pass ||
(view.shape === "list" && this.#retryInsert(view, record, exercised));
if (pass) this.#confirm(view, record, keys, exercised);
else if (this.#interfered(keys)) {
const unseen = pass
? this.#confirm(view, record, keys, exercised)
: undefined;
if (!pass) {
if (!this.#interfered(keys))
this.#mismatch({ record, index, keys, exercised, view });
// An unmodelled write since these writes may explain the difference: check their own members only.
if (view.shape === "item") this.#checkMembers(view, record);
else if (view.shape === "item") this.#checkMembers(view, record);
else for (const key of keys) this.#taint.fail(key);
} else this.#mismatch({ record, index, keys, exercised, view });
if (!pass && view.shape === "item") this.#adopt(view, record);
if (view.shape === "item") this.#adopt(view, record);
}
const pending = {
touched: this.#touched,
since: this.#touchedSince,
effects: this.#effects,
};
for (const key of keys) {
this.#touched.delete(key);
this.#effects.checked(key);
this.#touchedSince.delete(key);
this.#lastCheck.set(key, index);
forgetChecked(pending, key, unseen?.get(key));
}
}
@@ -411,22 +402,23 @@ export class EnvironmentStateWriteSimulation {
this.#drafts.get(name)!.contradictedBy ??= record.ordinal;
}
/** A passing read confirms the writes it exercised that it shows something of. */
/** A passing read confirms the writes it exercised that it shows something of, and names, per target, those it shows nothing of. */
#confirm(
view: StateView,
record: StateRecordFacts,
keys: readonly string[],
exercised: ReadonlySet<string>,
): void {
const shown = shownFields(view, this.#input.volatileOf);
for (const name of exercised) {
const shows = (key: string) =>
this.#touched.get(key)?.has(name) === true &&
this.#effects.shows(name, key, shown);
if (!keys.some(shows)) continue;
): Map<string, Set<string>> {
const shown = shownWrites(this.#touched, this.#effects, {
keys,
exercised,
fields: shownFields(view, this.#input.volatileOf),
});
for (const name of shown.confirmed) {
this.#drafts.get(name)!.confirmed.push(record.ordinal);
if (view.shape === "list") this.#placed.add(name);
}
return shown.unseen;
}
#unmodelled(path: string, index: number): void {
@@ -450,14 +442,21 @@ export class EnvironmentStateWriteSimulation {
const { collection, id } = itemTarget(this.#state, view.type, record);
const entity = this.#state.entity(view.type, collection, id);
if (entity === undefined) return;
const writes =
this.#touched.get(
entityTarget(entity.type, entity.collection, entity.id),
) ?? new Map();
const target = entityTarget(entity.type, entity.collection, entity.id);
this.#checkFields(view, record, entity, this.#touched.get(target));
}
/** Check the writes to an entity on the fields each wrote against an item read; the read's object, when it shows one. */
#checkFields(
view: Extract<StateView, { shape: "item" }>,
record: StateRecordFacts,
entity: StateEntity,
writes: ReadonlyMap<string, readonly string[] | null> = new Map(),
): JsonObject | undefined {
const object = this.#input.io.withBody(record.ordinal, (body) =>
body === undefined || body === null ? undefined : at(body, view.pointer),
);
if (!isObject(object)) return;
if (!isObject(object)) return undefined;
for (const [name, pass] of writtenFieldChecks(entity, view, object, {
volatile: this.#input.volatileOf(view.type),
writes,
@@ -466,6 +465,7 @@ export class EnvironmentStateWriteSimulation {
if (pass) draft.confirmed.push(record.ordinal);
else draft.contradictedBy ??= record.ordinal;
}
return object;
}
/** After a failed item check the state takes what the read recorded, so later checks start from it. */
@@ -179,13 +179,19 @@ const PAGE_SIZE_PARAMETERS = new Set([
"size",
]);
/** The most items a query's page holds: its smallest whole-number page-size value, or undefined without one. */
/** The most items a query's page holds: its smallest whole-number page-size value or `_start`–`_end` range width, or undefined without one. */
export const pageSize = (query: string | undefined): number | undefined => {
let size: number | undefined;
for (const [name, value] of iterateUrlQueryParameters(query ?? ""))
if (PAGE_SIZE_PARAMETERS.has(name.toLowerCase()) && /^\d+$/u.test(value))
size = Math.min(size ?? Infinity, Number(value));
return size;
let size = Infinity;
const range = { start: 0, end: Infinity };
for (const [name, value] of iterateUrlQueryParameters(query ?? "")) {
const key = name.toLowerCase();
if (!/^\d+$/u.test(value)) continue;
if (PAGE_SIZE_PARAMETERS.has(key)) size = Math.min(size, Number(value));
else if (key === "_start") range.start = Number(value);
else if (key === "_end") range.end = Number(value);
}
const width = Math.min(size, Math.max(range.end - range.start, 0));
return Number.isFinite(width) ? width : undefined;
};
/** Identity field names tried, in order, when no path parameter proves one. */
@@ -67,12 +67,19 @@ export const touchTarget = (
/** The fields of a write's effect on an entity no later write has overwritten: all of them for a create or delete (null). */
type ChangedFields = readonly string[] | null;
/** What a later write overwrites: the whole entity, the fields it sets, or every field (a recorded outcome). */
/** What a later write overwrites: the whole entity, the fields it sets, or every field (a recorded outcome). A document's fields are pointers, each covering the members under it. */
interface Overwrite {
readonly whole: boolean;
readonly fields: ReadonlySet<string> | "all";
}
/** Whether a write's fields cover a field: the same one, or, for a document, a pointer above it. */
const covers = (fields: ReadonlySet<string>, field: string): boolean =>
fields.has(field) ||
[...fields].some(
(over) => over.startsWith("/") && field.startsWith(`${over}/`),
);
/** What an earlier effect keeps after a write: nothing once every field it holds is overwritten; an update leaves a create's or delete's whole effect. */
const remaining = (
claim: ChangedFields,
@@ -82,7 +89,7 @@ const remaining = (
if (claim === null) return null;
const { fields } = write;
if (fields === "all") return undefined;
const left = claim.filter((field) => !fields.has(field));
const left = claim.filter((field) => !covers(fields, field));
return left.length > 0 ? left : undefined;
};
@@ -116,10 +123,13 @@ export class WriteEffects {
Map<string, Map<string, ChangedFields>>
>();
/** Forget a checked target's writes, as the touched targets do. */
checked(key: string): void {
this.#changed.delete(key);
this.#members.delete(key);
/** Forget a checked target's writes, as the touched targets do: all, or all but those a read showed nothing of. */
checked(key: string, unseen?: ReadonlySet<string>): void {
for (const byTool of [this.#changed, this.#members])
if (unseen === undefined) byTool.delete(key);
else
for (const tool of byTool.get(key)?.keys() ?? [])
if (!unseen.has(tool)) byTool.get(key)!.delete(tool);
}
/** Whether a read showing `fields` of a target shows anything of a tool's effect there: a create's or delete's always, an update's through a field it changed; unknown claims count as shown. */
@@ -222,3 +232,50 @@ export class WriteEffects {
}
}
}
/** Of the tools a read exercised, those it shows something of on some target, and per target those it shows nothing of. */
export const shownWrites = (
touched: TouchedTargets,
effects: WriteEffects,
read: {
readonly keys: readonly string[];
readonly exercised: ReadonlySet<string>;
readonly fields: ReadonlySet<string> | "all";
},
): {
readonly confirmed: string[];
readonly unseen: Map<string, Set<string>>;
} => {
const confirmed: string[] = [];
const unseen = new Map<string, Set<string>>();
for (const name of read.exercised) {
const on = read.keys.filter((key) => touched.get(key)?.has(name));
const seen = on.filter((key) => effects.shows(name, key, read.fields));
for (const key of on)
if (!seen.includes(key))
unseen.set(key, (unseen.get(key) ?? new Set()).add(name));
if (seen.length > 0) confirmed.push(name);
}
return { confirmed, unseen };
};
/** A checked target forgets its writes, but for those a passing read showed nothing of, which a later check still takes. */
export const forgetChecked = (
pending: {
readonly touched: TouchedTargets;
readonly since: PendingTargets;
readonly effects: WriteEffects;
},
key: string,
unseen: ReadonlySet<string> | undefined,
): void => {
const writes = pending.touched.get(key);
if (unseen === undefined || writes === undefined) {
pending.touched.delete(key);
pending.effects.checked(key);
pending.since.delete(key);
return;
}
for (const name of writes.keys()) if (!unseen.has(name)) writes.delete(name);
pending.effects.checked(key, unseen);
};
@@ -31,6 +31,7 @@ import {
explainCreate,
insertCandidates,
insertIndex,
nextInteger,
responseShapeOf,
} from "./environmentStateWriteExplain.js";
import {
@@ -38,7 +39,8 @@ import {
newEntityIn,
type Observed,
} from "./environmentStateWriteObserve.js";
import type { WriteCall } from "./environmentStateWriteRules.js";
import type { FieldRule, WriteCall } from "./environmentStateWriteRules.js";
import { integerOf } from "./environmentStateWriteValues.js";
/** The id a response names alone: a scalar body, or an object's id field or single scalar member. */
const respondedId = (
@@ -142,6 +144,59 @@ const buildCreate = (
draft.observed = evidence.observed.object;
};
type SequenceRule = Extract<FieldRule, { rule: "sequence" }>;
/** The sequences a create's plan generates, each with the key serving counts it under. */
const sequencesOf = (
draft: WriteDraft,
collection: string,
): { readonly rule: SequenceRule; readonly key: string }[] =>
draft.plan.rules.flatMap((rule) =>
rule.rule === "sequence"
? [
{
rule,
key:
rule.scope === "collection"
? `${rule.field} ${collection}`
: rule.field,
},
]
: [],
);
/** Whether a later call's sequences hold what serving generates: one past the largest value their scope holds or started with, and past the plan's last. */
const followsSequences = (
context: EffectContext,
draft: WriteDraft,
created: Pick<StateEntity, "type" | "collection" | "fields">,
): boolean =>
sequencesOf(draft, created.collection).every(({ rule, key }) => {
const scope = [
...context.state.ofType(created.type),
...context.startedWith(created.type),
].filter(
(entity) =>
rule.scope === "type" || entity.collection === created.collection,
);
const next = Math.max(
nextInteger(fieldsOf(scope), rule.field) ?? 1,
(draft.sequences.get(key) ?? 0) + 1,
);
return integerOf(created.fields[rule.field]) === next;
});
/** Record the values a create's sequences took, which the next call follows. */
const takeSequences = (
draft: WriteDraft,
created: Pick<StateEntity, "collection" | "fields">,
): void => {
for (const { rule, key } of sequencesOf(draft, created.collection)) {
const value = integerOf(created.fields[rule.field]);
if (value !== undefined) draft.sequences.set(key, value);
}
};
/** Add a created entity to the state at the plan's insert position, or last before the plan has one. */
const insertCreated = (
context: EffectContext,
@@ -205,8 +260,9 @@ export const applyCreate = (
return recorded();
}
const call = { ...callOf(step, body), body };
if (!built(draft))
buildCreate(context, draft, step, { observed, call, collection });
// The exemplar defines the plan's sequences; each later call must follow them.
const later = built(draft);
if (!later) buildCreate(context, draft, step, { observed, call, collection });
const fields = createdFields(
draft.plan,
draft.observed!,
@@ -217,11 +273,18 @@ export const applyCreate = (
if (
!fields.ok ||
id === undefined ||
context.state.entity(type, collection, id) !== undefined
context.state.entity(type, collection, id) !== undefined ||
(later &&
!followsSequences(context, draft, {
type,
collection,
fields: fields.value,
}))
) {
refuse(draft, step, "unexplained-change");
return recorded();
}
takeSequences(draft, { collection, fields: fields.value });
const inserted = insertCreated(context, draft, entity(fields.value, id));
return {
kind: "applied",
@@ -0,0 +1,139 @@
import type { JsonValue } from "../jsonValue.js";
import {
decodedSegment,
pointerTo,
same,
segmentsOf,
type JsonObject,
} from "./environmentStateRules.js";
import {
documentBodyProblem,
writtenDocument,
} from "./environmentStateWriteApply.js";
import { documentTarget } from "./environmentStateWriteChecks.js";
import {
built,
callOf,
fail,
refuse,
requestObject,
UNKNOWN,
type EffectContext,
type EffectResult,
type WriteDraft,
type WriteStep,
} from "./environmentStateWriteEffects.js";
import { documentPointerOf } from "./environmentStateWriteExplain.js";
/** Whether a write's path names the held document's, a segment verbatim or once percent-decoded, as serving matches a call. */
const namesDocument = (path: string, held: string): boolean => {
const written = segmentsOf(path);
const recorded = segmentsOf(held);
return (
written.length === recorded.length &&
recorded.every(
(segment, index) =>
segment === written[index] ||
decodedSegment(segment) === written[index],
)
);
};
/** Where a write goes: a built plan writes where serving does, at its own object; an exemplar at the object holding most of its members. */
const writePointer = (
draft: WriteDraft,
value: JsonValue,
body: JsonObject,
): string | undefined =>
built(draft)
? draft.plan.observation?.pointer
: documentPointerOf(value, body);
/** The fields a document write changed, as pointers: a merge the members it sent at its object, a replacement all under it (undefined: the whole document). */
const writtenPointers = (
draft: WriteDraft,
body: JsonObject,
): readonly string[] | undefined => {
const at = draft.plan.observation?.pointer ?? "";
if (draft.plan.merge === "merge")
return Object.keys(body).map((member) => `${at}${pointerTo(member)}`);
return at === "" ? undefined : [at];
};
/** Build a document plan from its exemplar: its object, merge or replacement by method, and the members it sent. */
const buildDocument = (
draft: WriteDraft,
step: WriteStep,
evidence: { readonly pointer: string; readonly body: JsonObject },
): void => {
const { ordinal } = step.record;
Object.assign(draft.plan, {
exemplar_ordinal: ordinal,
observation: { ordinal, pointer: evidence.pointer },
merge: step.tool.method === "PUT" ? "replace" : "merge",
body: {
accepted: Object.keys(evidence.body),
fixed: [],
required: [],
identity: null,
},
});
};
/** Apply a recorded document write: the body merged into, or replacing, the object holding most of its members. */
export const applyDocument = (
context: EffectContext,
draft: WriteDraft,
step: WriteStep,
): EffectResult => {
const tool = draft.plan.target!;
const initial = context.initialDocument(tool);
// Serving refuses a write at another path, so one here changes no document the state holds.
if (initial !== undefined && !namesDocument(step.record.path, initial.path))
return UNKNOWN;
const value = context.state.document(tool);
const body = requestObject(context, step.record);
const pointer =
value === undefined || body === undefined
? undefined
: writePointer(draft, value, body);
if (step.observeOnly || pointer === undefined) {
fail(draft, step, body === undefined ? "body-unusable" : "no-observation");
return UNKNOWN;
}
if (!built(draft)) buildDocument(draft, step, { pointer, body: body! });
const call = callOf(step, body!);
// Serving refuses a body its check refuses, so such a call cannot confirm the plan.
if (
documentBodyProblem(draft.plan, call, {
initial: initial?.value,
current: value!,
}) !== undefined
) {
refuse(draft, step, "unexplained-change");
return UNKNOWN;
}
const written = writtenDocument(draft.plan, value!, call);
if (!written.ok) {
refuse(draft, step, "no-observation");
return UNKNOWN;
}
const { ordinal } = step.record;
draft.plan.response ??= context.io.withBody(ordinal, (response) => ({
ordinal,
shape: context.io.emptyResponse(ordinal)
? "empty"
: response !== undefined && same(response, written.value)
? "entity"
: "recorded",
pointer: null,
}));
context.io.retain(64 + JSON.stringify(written.value).length * 2);
context.state.setDocument(tool, written.value);
const fields = writtenPointers(draft, body!);
return {
kind: "applied",
targets: [documentTarget(tool)],
...(fields === undefined ? {} : { fields }),
};
};
@@ -1,35 +1,29 @@
import type { JsonValue } from "../jsonValue.js";
import {
collectionPathOf,
decodedSegment,
isObject,
isStateWrite,
lastSegmentValue,
locateItem,
at,
same,
segmentsOf,
type JsonObject,
type StateDocument,
type StateEntity,
type StateRecordFacts,
type StateToolFacts,
type StateType,
type StateView,
} from "./environmentStateRules.js";
import {
documentBodyProblem,
updatedEntity,
updatedFields,
writtenDocument,
} from "./environmentStateWriteApply.js";
import { updatedEntity, updatedFields } from "./environmentStateWriteApply.js";
import {
collectionTarget,
documentTarget,
entityTarget,
itemTarget,
type SimulatedState,
} from "./environmentStateWriteChecks.js";
import {
conflictsWithPlan,
documentPointerOf,
explainUpdate,
responseShapeOf,
} from "./environmentStateWriteExplain.js";
@@ -64,6 +58,8 @@ export interface WriteDraft {
readonly shown: ReadonlyMap<string, JsonValue>;
readonly sent: ReadonlyMap<string, JsonValue>;
}[];
/** The last value each sequence took, by field and, for a collection's sequence, collection; the next call must follow it. */
readonly sequences: Map<string, number>;
}
/** Response bodies, request bodies and budget the effects read. */
@@ -77,6 +73,19 @@ export interface WriteIo {
readonly retain: (bytes: number) => void;
}
/** What the write gate reads: the manifest tools, the timeline, the read views and the initial state. */
export interface WriteSimulationInput {
readonly tools: readonly StateToolFacts[];
readonly records: readonly StateRecordFacts[];
readonly timed: boolean;
readonly views: ReadonlyMap<string, StateView>;
readonly types: readonly StateType[];
readonly entities: readonly StateEntity[];
readonly documents: readonly StateDocument[];
readonly volatileOf: (type: string) => ReadonlySet<string>;
readonly io: WriteIo;
}
/** What applying a recorded write reads besides the draft and the call. */
export interface EffectContext {
readonly state: SimulatedState;
@@ -91,6 +100,8 @@ export interface EffectContext {
readonly unknown: (type: string, collection: string, id: string) => boolean;
/** A document as the initial state holds it: its path, and the value serving also reads member types from. */
readonly initialDocument: (tool: string) => StateDocument | undefined;
/** The entities of a type the initial state held, whose values a sequence never reuses. */
readonly startedWith: (type: string) => readonly StateEntity[];
}
/** One recorded write: the record, its timeline index, its tool, and whether its plan only takes the recorded outcome. */
@@ -187,17 +198,17 @@ export const searchFrom = (context: EffectContext, step: WriteStep) => ({
withBody: context.io.withBody,
});
/** The entity an update's response shows, located by its identity, else a later item read of it. */
/** The entity an update's response shows, located by its identity, else a later item read of it spelled as the call was. */
const updatedObservation = (
context: EffectContext,
step: WriteStep,
target: {
readonly type: string;
readonly collection: string;
readonly idField: string;
readonly id: string;
},
entity: { readonly type: string; readonly idField: string },
): Observed | undefined => {
const target = {
...entity,
collection: collectionPathOf(step.record, true),
id: lastSegmentValue(step.record.path),
};
const own = context.io.withBody(
step.record.ordinal,
(body): Observed | undefined => {
@@ -345,13 +356,13 @@ export const applyUpdate = (
): EffectResult => {
const type = draft.plan.target!;
const idField = context.idField(type);
const collection = collectionPathOf(step.record, true);
const id = lastSegmentValue(step.record.path);
// The entity as serving names it.
const { collection, id } = itemTarget(context.state, type, step.record);
const entity = context.state.entity(type, collection, id);
const after =
entity === undefined
? undefined
: updatedObservation(context, step, { type, collection, idField, id });
: updatedObservation(context, step, { type, idField });
if (entity === undefined || after === undefined) {
fail(draft, step, "no-observation");
return UNKNOWN;
@@ -446,8 +457,7 @@ export const applyDelete = (
step: WriteStep,
): EffectResult => {
const type = draft.plan.target!;
const collection = collectionPathOf(step.record, true);
const id = lastSegmentValue(step.record.path);
const { collection, id } = itemTarget(context.state, type, step.record);
const entity = context.state.entity(type, collection, id);
if (entity === undefined) {
fail(draft, step, "no-observation");
@@ -475,91 +485,3 @@ export const applyDelete = (
entity: { type, collection, id },
};
};
/** Whether a write's path names the held document's, a segment verbatim or once percent-decoded, as serving matches a call. */
const namesDocument = (path: string, held: string): boolean => {
const written = segmentsOf(path);
const recorded = segmentsOf(held);
return (
written.length === recorded.length &&
recorded.every(
(segment, index) =>
segment === written[index] ||
decodedSegment(segment) === written[index],
)
);
};
/** Apply a recorded document write: the body merged into, or replacing, the object holding most of its members. */
export const applyDocument = (
context: EffectContext,
draft: WriteDraft,
step: WriteStep,
): EffectResult => {
const tool = draft.plan.target!;
const initial = context.initialDocument(tool);
// Serving refuses a write at another path, so one here changes no document the state holds.
if (initial !== undefined && !namesDocument(step.record.path, initial.path))
return UNKNOWN;
const value = context.state.document(tool);
const body = requestObject(context, step.record);
const pointer =
value === undefined || body === undefined
? undefined
: documentPointerOf(value, body);
if (
step.observeOnly ||
value === undefined ||
body === undefined ||
pointer === undefined
) {
fail(draft, step, body === undefined ? "body-unusable" : "no-observation");
return UNKNOWN;
}
const { ordinal } = step.record;
if (!built(draft))
Object.assign(draft.plan, {
exemplar_ordinal: ordinal,
observation: { ordinal, pointer },
merge: step.tool.method === "PUT" ? "replace" : "merge",
body: {
accepted: Object.keys(body),
fixed: [],
required: [],
identity: null,
},
});
const call = callOf(step, body);
// Serving refuses a body its check refuses, so such a call cannot confirm the plan.
if (
documentBodyProblem(draft.plan, call, {
initial: initial?.value,
current: value,
}) !== undefined
) {
refuse(draft, step, "unexplained-change");
return UNKNOWN;
}
const written = writtenDocument(draft.plan, value, call);
if (!written.ok) {
refuse(draft, step, "no-observation");
return UNKNOWN;
}
draft.plan.response ??= context.io.withBody(ordinal, (response) => ({
ordinal,
shape: context.io.emptyResponse(ordinal)
? "empty"
: response !== undefined && same(response, written.value)
? "entity"
: "recorded",
pointer: null,
}));
context.io.retain(64 + JSON.stringify(written.value).length * 2);
context.state.setDocument(tool, written.value);
// A merge writes the members it sent; a replacement writes the whole document.
return {
kind: "applied",
targets: [documentTarget(tool)],
...(draft.plan.merge === "merge" ? { fields: Object.keys(body) } : {}),
};
};
@@ -77,7 +77,7 @@ const isClockNear = (
};
/** One more than the largest integer the members hold in the field, or undefined when none holds one. */
const nextInteger = (
export const nextInteger = (
members: readonly JsonObject[],
field: string,
): number | undefined => {
@@ -1,8 +1,6 @@
import type { ExtractedState } from "./environmentStateRules.js";
import {
EnvironmentStateWriteSimulation,
type WriteSimulationInput,
} from "./EnvironmentStateWriteSimulation.js";
import { EnvironmentStateWriteSimulation } from "./EnvironmentStateWriteSimulation.js";
import type { WriteSimulationInput } from "./environmentStateWriteEffects.js";
/** The instant a served clock starts from: the end of the last timed record, to the next whole second. */
const clockStart = (