mirror of
https://github.com/morluto/rea.git
synced 2026-10-26 17:35:11 +00:00
fix(environment): name entities and weigh interference per write as serving and the recording show
Greptile raised these on this branch, the fidelity branch and the list-lines branch; five follow from the previous round's changes. - An entity field whose name looks like a pointer is matched by name; only a document's pointers cover the members under them. - A failed read contradicts each exercised write that no unmodelled write came after, instead of skipping them all when the oldest pending write preceded one; the shared pending `since` time goes. - An item read, update or delete names its entity as serving's namedEntity does: held collections its path matches, verbatim spellings first, the id decoded once, then as written; `book%2531` no longer reaches `book1`. - A plan whose calls use a member both as the identity and as a fixed member, or fix it at different values, is refused, as serving could not take every recorded call. - A size of zero bounds nothing only on a line whose recording listed items under it (pageBound), in serving and in the gate. - An update whose observation drops a field the entity held is refused, as serving keeps every field it does not write. - A later create whose clock takes another format, or whose unique value is not a UUID, is refused. - A derived state data/get would write past the 32 MiB intake limit is not served (admitServedState); the environment serves replay. The verdicts over the 50 captures are unchanged plan by plan, and the smoke applies all 31 served write tools. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
6257de93a1
commit
466ed8b433
17 files changed
+458
-117
No files matched your search
@@ -404,7 +404,8 @@ A state that is unavailable does not stop the server: it logs a warning naming
|
||||
the error `code` and message before it binds, then serves replay. The state is
|
||||
unavailable when it cannot be derived, such as from a
|
||||
`capture_browser_scenario` capture that selected header values but no response
|
||||
bodies or over the view's budget, and when its catalog under the served name
|
||||
bodies or over the view's budget, when `data/get` would write it past the 32 MiB
|
||||
`data/add` intake limit, and when its catalog under the served name
|
||||
exceeds the [discovery reservation](#what-a-manifest-is). That check runs
|
||||
first, so no state is derived for it, and replay is then admitted exactly as
|
||||
with `--replay-only`. `--replay-only`
|
||||
|
||||
@@ -105,7 +105,11 @@ the value a recorded call sent it with, and then stores the value that call
|
||||
recorded (its `constant` rule carries it). The type's identity field never
|
||||
takes a rule: sent as the path's id it is the identity member, sent as another
|
||||
value the application ignored it is accepted with that value and stores
|
||||
nothing, and a stored id that changed is `unexplained-change`.
|
||||
nothing, and a stored id that changed is `unexplained-change`. A plan whose
|
||||
calls use a member both as the identity and as a fixed member, or fix it at
|
||||
different values, is `unexplained-change`, as serving could not take every
|
||||
recorded call. So is an update whose observation drops a field the entity
|
||||
held, since serving keeps every field an update does not write.
|
||||
`PUT` requires every member the exemplar sent; `PATCH` and `POST` require at
|
||||
least one. A field the exemplar's update changed without sending it is
|
||||
explained as one of:
|
||||
@@ -159,9 +163,10 @@ the initial state.
|
||||
**Writes.** Only a 2xx write is modelled. A modelled write applies its effect, using the values it recorded
|
||||
for generated fields and the stored values for fields it transformed. A
|
||||
modelled write whose call the plan does not explain brings its entity to the
|
||||
recorded outcome instead, without attribution. A later create whose sequence
|
||||
fields hold another value than serving would generate makes the plan
|
||||
`unexplained-change`; identities are left to the id mapping. An update or
|
||||
recorded outcome instead, without attribution. A later create whose generated
|
||||
values take another form than serving gives them (a sequence value out of
|
||||
order, a clock in another format, a non-UUID unique value) makes the plan
|
||||
`unexplained-change`; integer identities are left to the id mapping. An update or
|
||||
delete names its entity as an item read does (below).
|
||||
|
||||
Any other write that may have changed state, including a redirected or
|
||||
@@ -183,8 +188,9 @@ members they sent.
|
||||
from the state:
|
||||
|
||||
- an item read of a state-backed item view, of the entity serving would name:
|
||||
the read's collection and id as written, else a held collection whose path
|
||||
matches once percent-decoded;
|
||||
in a held collection the read's path matches, each segment verbatim or once
|
||||
percent-decoded and verbatim spellings first, the id percent-decoded once,
|
||||
then as written;
|
||||
- a list or document read on one of its view's checked request lines,
|
||||
chosen as replay chooses the line for the read's call: each path parameter
|
||||
the read's value verbatim or once percent-decoded, preferring verbatim
|
||||
@@ -194,7 +200,8 @@ from the state:
|
||||
different values is never chosen, since no call's arguments can name it; one
|
||||
repeating a name with one value is, as a call naming it matches both. A list
|
||||
is compared as served from the collection of that line. A read on a page
|
||||
(`limit`, `size`, `per_page` and the like, a size of zero bounding nothing, or a `_start`–`_end` range) of a
|
||||
(`limit`, `size`, `per_page` and the like, or a `_start`–`_end` range; a size
|
||||
of zero bounds nothing on a line whose recording listed items under it) of a
|
||||
whole list checks nothing once the collection holds more entities than the
|
||||
page, which serving then answers from its recording.
|
||||
- A read made while a modelled write to its entity or collection was still
|
||||
@@ -224,15 +231,17 @@ refuses to answer with.
|
||||
instead), a recorded outcome every field, and a create or delete the whole
|
||||
entity. A document write's effect is the members it merged at its
|
||||
object, or everything under the object it replaces, the whole document at the
|
||||
root. A write with nothing left on an entity no
|
||||
root; an entity's fields match by name only, whatever their names look like. A write with nothing left on an entity no
|
||||
longer counts there, nor in a collection where no entity it changed keeps any
|
||||
of its effect. A mismatch contradicts them. A pass confirms those it shows
|
||||
something of: a create's or delete's membership, or a changed field the
|
||||
read's view shows and does not treat as volatile; a read that shows none of
|
||||
an update's changes confirms nothing of it, and leaves it for a later check
|
||||
of that entity or collection.
|
||||
- A mismatch after an unmodelled write that followed those writes is checked
|
||||
only on the members they sent: such a write may explain the rest.
|
||||
- A mismatch contradicts each exercised write that no unmodelled write came
|
||||
after, whatever earlier write is still pending. When an unmodelled write
|
||||
followed some of them, an item read checks those only on the members they
|
||||
sent and a list is not checked again: such a write may explain the rest.
|
||||
- A mismatch on an entity or collection that no write changed contradicts,
|
||||
as `side-effect`, the modelled writes since that previous check whose paths
|
||||
are segment-prefix related to the read's path.
|
||||
@@ -305,7 +314,9 @@ installs and of the type's checked list lines (so a collection the capture
|
||||
listed empty keeps its spelling), never those of a state it replaced. When two
|
||||
spellings fit a call equally, the one holding entities wins. `data/add` refuses a document that `data/get` would write back past the
|
||||
32 MiB intake limit once it adds the members a document may omit, such as
|
||||
`types` and the state Evidence id.
|
||||
`types` and the state Evidence id. A derived state `data/get` would write
|
||||
past that limit is not served: the environment serves replay and reports the
|
||||
state `unavailable`.
|
||||
|
||||
**Clock and generators.** The store's clock starts at `write_clock` and
|
||||
advances one second before each served write; clock fields take its value in
|
||||
|
||||
@@ -319,3 +319,22 @@ A fourteenth round settled two findings Greptile raised on the list-queries bran
|
||||
|
||||
Both tests fail on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke applies
|
||||
all 31 served write tools, and the gates passed again.
|
||||
|
||||
A fifteenth round settled nine findings Greptile raised on this branch, the fidelity branch and the list-lines branch,
|
||||
five of them on the previous rounds' own changes:
|
||||
|
||||
- an entity field whose name looks like a pointer is matched by name; only a document's pointers cover the members
|
||||
under them;
|
||||
- a failed read contradicts each exercised write no unmodelled write came after, rather than skipping all of them
|
||||
when the oldest pending write came before one;
|
||||
- an item read, update or delete names its entity as serving does, decoding the id once, so `book%2531` no longer
|
||||
reaches `book1`;
|
||||
- a plan whose calls use a member both as the identity and as a fixed member, or fix it at different values, is
|
||||
refused;
|
||||
- a size of zero bounds nothing only on a line whose recording listed items under it;
|
||||
- an update whose observation drops a field the entity held is refused;
|
||||
- a later create whose clock takes another format, or whose unique value is not a UUID, is refused;
|
||||
- a derived state `data/get` would write past 32 MiB is not served.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
|
||||
applies all 31 served write tools, and the gates passed again.
|
||||
@@ -24,6 +24,7 @@ import {
|
||||
} from "../analysisView/AnalysisViewService.js";
|
||||
import { loadEnvironmentSource } from "./environmentSource.js";
|
||||
import {
|
||||
admitServedState,
|
||||
deriveEnvironmentState,
|
||||
loadEnvironmentState,
|
||||
prepareServedState,
|
||||
@@ -272,6 +273,28 @@ describe("prepareServedState", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("admitServedState", () => {
|
||||
it("refuses a derived state data/get would write past what data/add loads back", () => {
|
||||
expect(admitServedState(state, "environment_serve").ok).toBe(true);
|
||||
// A euro sign is one UTF-16 unit but three bytes as data/get writes it.
|
||||
const notes = "\u20ac".repeat(12_000_000);
|
||||
const [first] = state.entities;
|
||||
const large = {
|
||||
...state,
|
||||
entities: [
|
||||
...state.entities,
|
||||
{ ...first!, id: "large", fields: { ...first!.fields, notes } },
|
||||
],
|
||||
};
|
||||
const admitted = admitServedState(large, "environment_serve");
|
||||
if (admitted.ok) throw new Error("Expected the state to be refused");
|
||||
expect(projectAnalysisError(admitted.error)).toMatchObject({
|
||||
code: "resource_constraint",
|
||||
message: expect.stringContaining("larger than data/add loads"),
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("EnvironmentStateStore", () => {
|
||||
it("reports its state as rea-state/v1, accepts that report back and resets to the initial state", () => {
|
||||
const store = new EnvironmentStateStore(state);
|
||||
|
||||
@@ -8,9 +8,9 @@ import {
|
||||
} from "../../domain/analysisView/environmentStateRules.js";
|
||||
import type { StateWritePlan } from "../../domain/analysisView/environmentStateWriteRules.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import type {
|
||||
AnalysisInputIssue,
|
||||
import {
|
||||
AnalysisResourceConstraintError,
|
||||
type AnalysisInputIssue,
|
||||
} from "../../domain/analysisErrorCore.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
@@ -23,6 +23,8 @@ import {
|
||||
type EnvironmentManifestTool,
|
||||
type EnvironmentSource,
|
||||
} from "./environmentSource.js";
|
||||
import { STATE_INTAKE_LIMIT_BYTES } from "./environmentStateDocument.js";
|
||||
import { initialStateBytes } from "./environmentStateStore.js";
|
||||
import { describeWrite } from "./environmentStateWriteDescription.js";
|
||||
import {
|
||||
environmentToolDefinitions,
|
||||
@@ -238,6 +240,29 @@ export interface PreparedState extends ServedState {
|
||||
>;
|
||||
}
|
||||
|
||||
/** The derived state, unless data/get would write it past what data/add loads back, so its own report could not be restored. */
|
||||
export const admitServedState = (
|
||||
state: EnvironmentState,
|
||||
operation: string,
|
||||
): Result<EnvironmentState, AnalysisResourceConstraintError> => {
|
||||
const bytes = initialStateBytes(state);
|
||||
return bytes > STATE_INTAKE_LIMIT_BYTES
|
||||
? err(
|
||||
new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
"capacity",
|
||||
"The derived state is larger than data/add loads, so it cannot be served",
|
||||
{ max_state_bytes: STATE_INTAKE_LIMIT_BYTES, state_bytes: bytes },
|
||||
{
|
||||
retryable: false,
|
||||
remediationAction:
|
||||
"Capture fewer or smaller responses, or serve the environment as replay only.",
|
||||
},
|
||||
),
|
||||
)
|
||||
: ok(state);
|
||||
};
|
||||
|
||||
const unavailable = (error: AnalysisError): PreparedState => {
|
||||
const { code, message } = projectAnalysisError(error);
|
||||
return {
|
||||
@@ -262,6 +287,8 @@ export const prepareServedState = (
|
||||
if (!reads.ok) return unavailable(reads.error);
|
||||
const derived = deriveEnvironmentState(source, captureValue, operation);
|
||||
if (!derived.ok) return unavailable(derived.error);
|
||||
const admitted = admitServedState(derived.value, operation);
|
||||
if (!admitted.ok) return unavailable(admitted.error);
|
||||
// Served write tools join the catalog, so the whole catalog is admitted again.
|
||||
const catalog = servedCatalog(source, derived.value, name, operation);
|
||||
return catalog.ok
|
||||
|
||||
@@ -16,7 +16,7 @@ import {
|
||||
type StateView,
|
||||
} from "../../domain/analysisView/environmentStateRules.js";
|
||||
import {
|
||||
pageSize,
|
||||
pageBound,
|
||||
pagesOnly,
|
||||
} from "../../domain/analysisView/environmentStatePaging.js";
|
||||
import { isAbsence } from "../../domain/analysisView/environmentStateWriteChecks.js";
|
||||
@@ -428,7 +428,9 @@ const answerList = (
|
||||
const collection = collectionPathOf(base, false);
|
||||
const type = store.type(view.type);
|
||||
// A requested page the collection has outgrown is answered as recorded, never with more items than it asked for.
|
||||
const size = exact ? pageSize(base.query ?? undefined) : undefined;
|
||||
const size = exact
|
||||
? pageBound(base.query ?? undefined, recorded.length)
|
||||
: undefined;
|
||||
if (
|
||||
view.membership === "all" &&
|
||||
size !== undefined &&
|
||||
|
||||
@@ -37,6 +37,27 @@ const itemsBytes = (items: readonly unknown[]): number =>
|
||||
items.reduce<number>((sum, item) => sum + jsonBytes(item), 0) +
|
||||
Math.max(0, items.length - 1);
|
||||
|
||||
/** What `data/get` reports for some content of a state. */
|
||||
const snapshotOf = (
|
||||
state: EnvironmentState,
|
||||
entities: readonly StateEntity[],
|
||||
documents: readonly HeldDocument[],
|
||||
): StateSnapshot => ({
|
||||
rea_state: {
|
||||
version: 1,
|
||||
state_evidence_id: state.evidenceId,
|
||||
types: state.types,
|
||||
entities,
|
||||
documents,
|
||||
},
|
||||
});
|
||||
|
||||
/** What `data/get` writes for a state as the Evidence holds it, in UTF-8 bytes, summed item by item. */
|
||||
export const initialStateBytes = (state: EnvironmentState): number =>
|
||||
jsonBytes(snapshotOf(state, [], [])) +
|
||||
itemsBytes(state.entities) +
|
||||
itemsBytes(state.documents);
|
||||
|
||||
/**
|
||||
* The live state one served environment answers from, shared by every MCP
|
||||
* session and the data plane. Mutations are synchronous and swap the whole
|
||||
@@ -74,8 +95,8 @@ export class EnvironmentStateStore {
|
||||
);
|
||||
this.#clockStart =
|
||||
state.writeClock === null ? 0 : Date.parse(state.writeClock);
|
||||
this.#frameBytes = jsonBytes(this.#snapshotOf([], []));
|
||||
this.#initialBytes = this.#contentBytes(state);
|
||||
this.#frameBytes = jsonBytes(snapshotOf(state, [], []));
|
||||
this.#initialBytes = initialStateBytes(state);
|
||||
this.reset();
|
||||
}
|
||||
|
||||
@@ -303,7 +324,11 @@ export class EnvironmentStateStore {
|
||||
}
|
||||
|
||||
snapshot(): StateSnapshot {
|
||||
return this.#snapshotOf(this.#content.entities, this.#content.documents);
|
||||
return snapshotOf(
|
||||
this.state,
|
||||
this.#content.entities,
|
||||
this.#content.documents,
|
||||
);
|
||||
}
|
||||
|
||||
/** What `data/get` writes for the current state, in UTF-8 bytes. */
|
||||
@@ -311,21 +336,6 @@ export class EnvironmentStateStore {
|
||||
return this.#bytes;
|
||||
}
|
||||
|
||||
#snapshotOf(
|
||||
entities: readonly StateEntity[],
|
||||
documents: readonly HeldDocument[],
|
||||
): StateSnapshot {
|
||||
return {
|
||||
rea_state: {
|
||||
version: 1,
|
||||
state_evidence_id: this.state.evidenceId,
|
||||
types: this.state.types,
|
||||
entities,
|
||||
documents,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** What `data/get` writes for some content, summed item by item so the whole state is never one string. */
|
||||
#contentBytes(content: StateContent): number {
|
||||
return (
|
||||
|
||||
@@ -709,6 +709,26 @@ describe("paged lines", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("answers a zero limit a whole line recorded empty from that recording once a create fills its collection", () => {
|
||||
const alpha = { id: 1, name: "Alpha", org: "1" };
|
||||
const beta = { id: 2, name: "Beta", org: "2" };
|
||||
const books = servedFrom([
|
||||
["GET", "/api/orgs/1/books", [alpha]],
|
||||
["GET", "/api/orgs/1/books/1", alpha],
|
||||
["GET", "/api/orgs/2/books?limit=0", []],
|
||||
["POST", "/api/orgs/2/books", beta, 201, { name: "Beta" }],
|
||||
["GET", "/api/orgs/2/books/2", beta],
|
||||
]);
|
||||
books.answer("post_orgs_books_by_id", { id: "2", body: { name: "Gamma" } });
|
||||
// Only a recording that listed items under it shows that zero lists every entity.
|
||||
expect(
|
||||
books.answer("get_orgs_books_by_id", { id: "2", limit: "0" }),
|
||||
).toMatchObject({
|
||||
body: [],
|
||||
state: { source: "recording", reason: "page-exceeded" },
|
||||
});
|
||||
});
|
||||
|
||||
it("answers a range the collection has outgrown from its recording", () => {
|
||||
const books = servedFrom([
|
||||
["GET", "/api/books?_start=0&_end=2", [FIRST_BOOK, SECOND_BOOK]],
|
||||
|
||||
@@ -983,3 +983,180 @@ describe("what a later call must follow", () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("what a write's effect keeps", () => {
|
||||
it("keeps an entity member a later write's other member does not cover, though both names look like pointers", () => {
|
||||
const flat = (id: number, prefs: string, lang: string) => ({
|
||||
id,
|
||||
name: `Book ${String(id)}`,
|
||||
"/prefs": prefs,
|
||||
"/prefs/lang": lang,
|
||||
});
|
||||
const rows: TimedCaptureRow[] = [
|
||||
["GET", "/api/books", [flat(1, "x", "en"), flat(2, "x", "en")]],
|
||||
["GET", "/api/books/1", flat(1, "x", "en")],
|
||||
["GET", "/api/books/2", flat(2, "x", "en")],
|
||||
[
|
||||
"PATCH",
|
||||
"/api/books/1",
|
||||
flat(1, "x", "fr"),
|
||||
200,
|
||||
{ "/prefs/lang": "fr" },
|
||||
],
|
||||
["PUT", "/api/books/1", flat(1, "y", "fr"), 200, { "/prefs": "y" }],
|
||||
["GET", "/api/books/1", flat(1, "y", "fr")],
|
||||
];
|
||||
expect(gated(rows, "patch_books_by_id")).toMatchObject({
|
||||
served: true,
|
||||
checked_record_ordinals: [5],
|
||||
});
|
||||
});
|
||||
|
||||
it("contradicts a write made after an unmodelled one, though an earlier pending write came before it", () => {
|
||||
const summary = (name: string) => [
|
||||
{ id: 1, name },
|
||||
{ id: 2, name: "B" },
|
||||
];
|
||||
const full = (name: string, description: string) => ({
|
||||
id: 1,
|
||||
name,
|
||||
description,
|
||||
});
|
||||
const rows: TimedCaptureRow[] = [
|
||||
["GET", "/api/books", summary("A")],
|
||||
["GET", "/api/books/1", full("A", "x")],
|
||||
["GET", "/api/books/2", { id: 2, name: "B", description: "y" }],
|
||||
["PATCH", "/api/books/1", full("A", "new"), 200, { description: "new" }],
|
||||
["GET", "/api/books", summary("A")],
|
||||
["POST", "/api/exports", { queued: true }, 202, { format: "csv" }],
|
||||
[
|
||||
"PUT",
|
||||
"/api/books/1",
|
||||
full("Z", "new"),
|
||||
200,
|
||||
{ name: "Z", description: "new" },
|
||||
],
|
||||
["GET", "/api/books/1", full("Z", "new")],
|
||||
// The export came before the rename, so it cannot explain the list showing the old name.
|
||||
["GET", "/api/books", summary("A")],
|
||||
];
|
||||
expect(gated(rows, "put_books_by_id")).toMatchObject({
|
||||
served: false,
|
||||
contradicted_by: 8,
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses an update whose recorded response dropped a field serving would keep", () => {
|
||||
const rows: TimedCaptureRow[] = [
|
||||
[
|
||||
"GET",
|
||||
"/api/books",
|
||||
[
|
||||
{ id: 1, name: "A" },
|
||||
{ id: 2, name: "B" },
|
||||
],
|
||||
],
|
||||
["GET", "/api/books/1", { id: 1, name: "A", subtitle: "s" }],
|
||||
["GET", "/api/books/2", { id: 2, name: "B", subtitle: "t" }],
|
||||
["PATCH", "/api/books/1", { id: 1, name: "Z" }, 200, { name: "Z" }],
|
||||
[
|
||||
"GET",
|
||||
"/api/books",
|
||||
[
|
||||
{ id: 1, name: "Z" },
|
||||
{ id: 2, name: "B" },
|
||||
],
|
||||
],
|
||||
];
|
||||
expect(gated(rows, "patch_books_by_id")).toMatchObject({
|
||||
served: false,
|
||||
reason: "unexplained-change",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("what later calls must keep consistent", () => {
|
||||
it("refuses an update one call sends an ignored id with and another the path's id", () => {
|
||||
const renamed = (at: Book, name: string) => ({ ...at, name });
|
||||
const rows: TimedCaptureRow[] = [
|
||||
...BOOK_INIT_ROWS,
|
||||
[
|
||||
"PATCH",
|
||||
"/api/books/1",
|
||||
renamed(FIRST_BOOK, "B"),
|
||||
200,
|
||||
{ id: 999, name: "B" },
|
||||
],
|
||||
["GET", "/api/books/1", renamed(FIRST_BOOK, "B")],
|
||||
[
|
||||
"PATCH",
|
||||
"/api/books/2",
|
||||
renamed(SECOND_BOOK, "C"),
|
||||
200,
|
||||
{ id: 2, name: "C" },
|
||||
],
|
||||
["GET", "/api/books/2", renamed(SECOND_BOOK, "C")],
|
||||
];
|
||||
expect(gated(rows, "patch_books_by_id")).toMatchObject({
|
||||
served: false,
|
||||
reason: "unexplained-change",
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses a create whose later call records a clock in another format", () => {
|
||||
const epoch = (iso: string) => Math.floor(Date.parse(iso) / 1000);
|
||||
const fourth = {
|
||||
...book(4, "Fourth Book", bookClock(5)),
|
||||
created_at: epoch(bookClock(5)),
|
||||
updated_at: epoch(bookClock(5)),
|
||||
};
|
||||
const rows: TimedCaptureRow[] = [
|
||||
...BOOK_INIT_ROWS,
|
||||
["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],
|
||||
["GET", "/api/books/3", THIRD_BOOK],
|
||||
["POST", "/api/books", fourth, 201, { name: "Fourth Book" }],
|
||||
["GET", "/api/books/4", fourth],
|
||||
];
|
||||
expect(gated(rows, "post_books")).toMatchObject({
|
||||
served: false,
|
||||
reason: "unexplained-change",
|
||||
});
|
||||
});
|
||||
|
||||
it("names no entity by a delete whose path id decodes to it only twice, as serving would not", () => {
|
||||
const uid = (key: string) => ({ uid: key, name: key });
|
||||
const rows: TimedCaptureRow[] = [
|
||||
["GET", "/api/books", [uid("book1"), uid("book2")]],
|
||||
["GET", "/api/books/uid/book1", uid("book1")],
|
||||
["GET", "/api/books/uid/book2", uid("book2")],
|
||||
["DELETE", "/api/books/uid/book%2531", "", 204],
|
||||
["GET", "/api/books/uid/book1", BOOK_NOT_FOUND, 404],
|
||||
];
|
||||
const { writes } = environmentStateViewOf(timedCaptureEvidence(rows), [
|
||||
"after-uid-literal",
|
||||
]);
|
||||
expect(writes.find((plan) => plan.effect === "delete")).toMatchObject({
|
||||
served: false,
|
||||
checked_record_ordinals: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("checks nothing by a whole line recorded empty under a zero limit once a create fills its collection", () => {
|
||||
const alpha = { id: 1, name: "Alpha", org: "1" };
|
||||
const beta = { id: 2, name: "Beta", org: "2" };
|
||||
const rows: TimedCaptureRow[] = [
|
||||
["GET", "/api/orgs/1/books", [alpha]],
|
||||
["GET", "/api/orgs/1/books/1", alpha],
|
||||
["GET", "/api/orgs/2/books?limit=0", []],
|
||||
["POST", "/api/orgs/2/books", beta, 201, { name: "Beta" }],
|
||||
["GET", "/api/orgs/2/books/2", beta],
|
||||
// An empty recording does not show that zero lists every entity.
|
||||
["GET", "/api/orgs/2/books?limit=0", []],
|
||||
];
|
||||
const { writes } = environmentStateViewOf(timedCaptureEvidence(rows));
|
||||
expect(writes.find((plan) => plan.effect === "create")).toMatchObject({
|
||||
served: true,
|
||||
checked_record_ordinals: [4],
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -88,7 +88,7 @@ export class EnvironmentStateWriteSimulation {
|
||||
readonly #touched: TouchedTargets = new Map();
|
||||
readonly #effects = new WriteEffects();
|
||||
readonly #lastCheck = new Map<string, number>();
|
||||
/** The earliest modelled write to each target since its last check, and when the latest of them ended. */
|
||||
/** When the latest modelled write to each target since its last check ended. */
|
||||
readonly #touchedSince: PendingTargets = new Map();
|
||||
/** Timeline indexes of writes the gate does not model. */
|
||||
readonly #unmodelledAt: number[] = [];
|
||||
@@ -236,22 +236,18 @@ export class EnvironmentStateWriteSimulation {
|
||||
result.entity.collection,
|
||||
result.entity.id,
|
||||
);
|
||||
if (result.kind === "applied")
|
||||
this.#effects.supersede(this.#touched, write.tool, result);
|
||||
if (result.kind === "observed")
|
||||
if (result.kind === "observed") {
|
||||
this.#effects.observed(this.#touched, result.targets);
|
||||
for (const target of result.targets) {
|
||||
// A recorded outcome replaces the entity's values, not its collection's membership.
|
||||
if (result.kind === "observed") {
|
||||
if (target === result.targets[0]) this.#touched.delete(target);
|
||||
continue;
|
||||
}
|
||||
this.#touched.delete(result.targets[0]!);
|
||||
return;
|
||||
}
|
||||
this.#effects.supersede(this.#touched, write.tool, result);
|
||||
for (const target of result.targets)
|
||||
touchTarget(this.#touched, this.#touchedSince, target, {
|
||||
...write,
|
||||
fields: result.fields,
|
||||
});
|
||||
}
|
||||
if (result.kind !== "applied") return;
|
||||
this.#log.push(write);
|
||||
if (result.inserted !== undefined)
|
||||
this.#inserted.set(result.inserted, write.tool);
|
||||
@@ -362,7 +358,7 @@ export class EnvironmentStateWriteSimulation {
|
||||
return;
|
||||
const exercised = exercisedBy(this.#touched, keys, borrowers);
|
||||
if (borrowers !== undefined) {
|
||||
this.#contradictOthers(outcome.pass, keys, exercised, record);
|
||||
this.#contradictOthers(outcome.pass, exercised, record);
|
||||
return;
|
||||
}
|
||||
const pass =
|
||||
@@ -372,11 +368,17 @@ export class EnvironmentStateWriteSimulation {
|
||||
? this.#confirm(view, record, keys, exercised)
|
||||
: undefined;
|
||||
if (!pass) {
|
||||
if (!this.#interfered(keys))
|
||||
const interfered = [...exercised].filter((name) =>
|
||||
this.#interferedWith(name),
|
||||
);
|
||||
if (interfered.length === 0)
|
||||
this.#mismatch({ record, index, keys, exercised, view });
|
||||
// An unmodelled write since these writes may explain the difference: check their own members only.
|
||||
// An unmodelled write after some of these writes may explain the difference: those are checked on their own members only.
|
||||
else if (view.shape === "item") this.#checkMembers(view, record);
|
||||
else for (const key of keys) this.#taint.fail(key);
|
||||
else {
|
||||
this.#contradictOthers(false, exercised, record);
|
||||
for (const key of keys) this.#taint.fail(key);
|
||||
}
|
||||
if (view.shape === "item") this.#adopt(view, record);
|
||||
}
|
||||
const pending = {
|
||||
@@ -390,16 +392,16 @@ export class EnvironmentStateWriteSimulation {
|
||||
}
|
||||
}
|
||||
|
||||
/** A read a write's effect was learned from checks nothing of that write, yet still contradicts the others. */
|
||||
/** A failed read contradicts the writes it exercised that no unmodelled write came after, as a read a write's effect was learned from still does the others. */
|
||||
#contradictOthers(
|
||||
pass: boolean,
|
||||
keys: readonly string[],
|
||||
exercised: ReadonlySet<string>,
|
||||
record: StateRecordFacts,
|
||||
): void {
|
||||
if (pass || this.#interfered(keys)) return;
|
||||
if (pass) return;
|
||||
for (const name of exercised)
|
||||
this.#drafts.get(name)!.contradictedBy ??= record.ordinal;
|
||||
if (!this.#interferedWith(name))
|
||||
this.#drafts.get(name)!.contradictedBy ??= record.ordinal;
|
||||
}
|
||||
|
||||
/** A passing read confirms the writes it exercised that it shows something of, and names, per target, those it shows nothing of. */
|
||||
@@ -426,12 +428,11 @@ export class EnvironmentStateWriteSimulation {
|
||||
this.#unmodelledAt.push(index);
|
||||
}
|
||||
|
||||
/** Whether a write the gate does not model came after the earliest modelled write a failed check exercised. */
|
||||
#interfered(keys: readonly string[]): boolean {
|
||||
const since = Math.min(
|
||||
...keys.map((key) => this.#touchedSince.get(key)?.since ?? Infinity),
|
||||
);
|
||||
return this.#unmodelledAt.some((index) => index > since);
|
||||
/** Whether a write the gate does not model came after a tool's latest modelled call, so it may explain what a read shows of that call. */
|
||||
#interferedWith(tool: string): boolean {
|
||||
const last =
|
||||
this.#log.findLast((write) => write.tool === tool)?.index ?? -1;
|
||||
return this.#unmodelledAt.some((index) => index > last);
|
||||
}
|
||||
|
||||
/** Check the writes to an item read's entity on the members they sent, as a resynchronising read does. */
|
||||
|
||||
@@ -42,12 +42,19 @@ export const pageSize = (query: string | undefined): number | undefined => {
|
||||
for (const [name, value] of iterateUrlQueryParameters(query ?? "")) {
|
||||
const key = name.toLowerCase();
|
||||
if (!/^\d+$/u.test(value)) continue;
|
||||
// A size of zero bounds nothing: applications read it as no limit, as a whole line naming it shows.
|
||||
if (PAGE_SIZE_PARAMETERS.has(key)) {
|
||||
if (Number(value) > 0) size = Math.min(size, Number(value));
|
||||
} else if (key === "_start") range.start = Number(value);
|
||||
if (PAGE_SIZE_PARAMETERS.has(key)) size = Math.min(size, Number(value));
|
||||
else if (key === "_start") range.start = Number(value);
|
||||
else if (key === "_end") range.end = Number(value);
|
||||
}
|
||||
const width = Math.min(size, Math.max(range.end - range.start, 0));
|
||||
return Number.isFinite(width) ? width : undefined;
|
||||
};
|
||||
|
||||
/** The most items a line's call lists: its page size, but none for a size of zero the line's own recording listed items under, as an application reading zero as no limit does. */
|
||||
export const pageBound = (
|
||||
query: string | undefined,
|
||||
recorded: number,
|
||||
): number | undefined => {
|
||||
const size = pageSize(query);
|
||||
return size === 0 && recorded > 0 ? undefined : size;
|
||||
};
|
||||
@@ -13,8 +13,8 @@ export const shownFields = (
|
||||
/** Modelled writes per target since its last check, with the fields each wrote (null: all). */
|
||||
export type TouchedTargets = Map<string, Map<string, readonly string[] | null>>;
|
||||
|
||||
/** Per target, the earliest modelled write since its last check and when the latest of them ended. */
|
||||
export type PendingTargets = Map<string, { since: number; until: number }>;
|
||||
/** Per target, when the latest modelled write since its last check ended. */
|
||||
export type PendingTargets = Map<string, { until: number }>;
|
||||
|
||||
/** Whether a modelled write to the target was still running when a read began at `start`, so the read may show either state. */
|
||||
export const runningAt = (
|
||||
@@ -36,22 +36,19 @@ export const exercisedBy = (
|
||||
return exercised;
|
||||
};
|
||||
|
||||
/** Record a modelled write on a target: the fields it wrote (undefined: all), when it began and when it ended. */
|
||||
/** Record a modelled write on a target: the fields it wrote (undefined: all) and when it ended. */
|
||||
export const touchTarget = (
|
||||
touched: TouchedTargets,
|
||||
pending: PendingTargets,
|
||||
target: string,
|
||||
write: {
|
||||
readonly index: number;
|
||||
readonly tool: string;
|
||||
readonly end: number;
|
||||
readonly fields: readonly string[] | undefined;
|
||||
},
|
||||
): void => {
|
||||
const since = pending.get(target);
|
||||
pending.set(target, {
|
||||
since: since?.since ?? write.index,
|
||||
until: Math.max(since?.until ?? -Infinity, write.end),
|
||||
until: Math.max(pending.get(target)?.until ?? -Infinity, write.end),
|
||||
});
|
||||
const tools = touched.get(target) ?? new Map();
|
||||
const before = tools.get(write.tool);
|
||||
@@ -71,14 +68,17 @@ type ChangedFields = readonly string[] | null;
|
||||
interface Overwrite {
|
||||
readonly whole: boolean;
|
||||
readonly fields: ReadonlySet<string> | "all";
|
||||
readonly pointers: boolean;
|
||||
}
|
||||
|
||||
/** Whether a write's fields cover a field: the same one, or, for a document, a pointer above it. */
|
||||
const covers = (fields: ReadonlySet<string>, field: string): boolean =>
|
||||
fields.has(field) ||
|
||||
[...fields].some(
|
||||
(over) => over.startsWith("/") && field.startsWith(`${over}/`),
|
||||
/** Whether a write's fields cover a field: the same one, or, for a document's pointers, one above it. */
|
||||
const covers = (write: Overwrite, field: string): boolean => {
|
||||
const { fields } = write;
|
||||
if (fields === "all" || fields.has(field)) return true;
|
||||
return (
|
||||
write.pointers && [...fields].some((over) => field.startsWith(`${over}/`))
|
||||
);
|
||||
};
|
||||
|
||||
/** What an earlier effect keeps after a write: nothing once every field it holds is overwritten; an update leaves a create's or delete's whole effect. */
|
||||
const remaining = (
|
||||
@@ -87,9 +87,8 @@ const remaining = (
|
||||
): ChangedFields | undefined => {
|
||||
if (write.whole) return undefined;
|
||||
if (claim === null) return null;
|
||||
const { fields } = write;
|
||||
if (fields === "all") return undefined;
|
||||
const left = claim.filter((field) => !covers(fields, field));
|
||||
if (write.fields === "all") return undefined;
|
||||
const left = claim.filter((field) => !covers(write, field));
|
||||
return left.length > 0 ? left : undefined;
|
||||
};
|
||||
|
||||
@@ -160,6 +159,8 @@ export class WriteEffects {
|
||||
readonly fields: readonly string[];
|
||||
readonly accumulated: readonly string[];
|
||||
};
|
||||
/** Set when the fields are a document's pointers. */
|
||||
readonly pointers?: boolean;
|
||||
},
|
||||
): void {
|
||||
const [entity, collection] = effect.targets as [string, string?];
|
||||
@@ -175,6 +176,7 @@ export class WriteEffects {
|
||||
(field) => !effect.changed?.accumulated.includes(field),
|
||||
),
|
||||
),
|
||||
pointers: effect.pointers === true,
|
||||
},
|
||||
);
|
||||
this.#changed.set(
|
||||
@@ -196,7 +198,7 @@ export class WriteEffects {
|
||||
this.#overwrite(
|
||||
touched,
|
||||
{ entity, collection, writer: undefined },
|
||||
{ whole: false, fields: "all" },
|
||||
{ whole: false, fields: "all", pointers: false },
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -6,11 +6,9 @@ import {
|
||||
} from "../unicodeCodePointOrder.js";
|
||||
import {
|
||||
at,
|
||||
collectionPathOf,
|
||||
decodedSegment,
|
||||
isObject,
|
||||
isParam,
|
||||
lastSegmentValue,
|
||||
listItems,
|
||||
missingRequiredFields,
|
||||
renderStateEntity,
|
||||
@@ -28,7 +26,7 @@ import {
|
||||
type StateToolFacts,
|
||||
type StateView,
|
||||
} from "./environmentStateRules.js";
|
||||
import { pagesOnly, pageSize } from "./environmentStatePaging.js";
|
||||
import { pageBound, pagesOnly } from "./environmentStatePaging.js";
|
||||
|
||||
/** A query as replay compares a call's: decoded values by name, in any order; undefined when it repeats a name with another value, which no call's arguments can carry. */
|
||||
const queryKey = (query: string | undefined): string | undefined => {
|
||||
@@ -253,35 +251,48 @@ interface ReadBodies {
|
||||
}
|
||||
|
||||
/** An item read: the answer the held entity renders into the served envelope must equal the recording; a recorded 404 or 410 agrees with none held. */
|
||||
/** How many of a held collection's segments match a path's verbatim, or undefined when one matches neither verbatim nor once percent-decoded. */
|
||||
const segmentMatch = (
|
||||
held: readonly string[],
|
||||
read: readonly string[],
|
||||
): number | undefined => {
|
||||
if (held.length !== read.length) return undefined;
|
||||
let verbatim = 0;
|
||||
for (const [index, segment] of held.entries())
|
||||
if (segment === read[index]) verbatim += 1;
|
||||
else if (decodedSegment(segment) !== read[index]) return undefined;
|
||||
return verbatim;
|
||||
};
|
||||
|
||||
/**
|
||||
* The entity an item read names, resolved as serving resolves an item call: its collection and id as written, else a
|
||||
* held collection whose path matches once percent-decoded and an id tried decoded, then as written.
|
||||
* The entity an item read or write names, as serving names it: in a held collection its path matches, verbatim
|
||||
* spellings first, the id percent-decoded once, then as written; else the collection and decoded id as written.
|
||||
*/
|
||||
export const itemTarget = (
|
||||
state: SimulatedState,
|
||||
type: string,
|
||||
record: Pick<StateRecordFacts, "path">,
|
||||
): { readonly collection: string; readonly id: string } => {
|
||||
const collection = collectionPathOf(record, true);
|
||||
const id = lastSegmentValue(record.path);
|
||||
if (state.entity(type, collection, id) !== undefined)
|
||||
return { collection, id };
|
||||
const read = segmentsOf(collection);
|
||||
const ids = [decodedSegment(id), id];
|
||||
for (const candidate of ids)
|
||||
for (const entity of state.ofType(type)) {
|
||||
const held = segmentsOf(entity.collection);
|
||||
if (
|
||||
entity.id === candidate &&
|
||||
held.length === read.length &&
|
||||
held.every(
|
||||
(segment, index) =>
|
||||
segment === read[index] || decodedSegment(segment) === read[index],
|
||||
)
|
||||
)
|
||||
return { collection: entity.collection, id: entity.id };
|
||||
}
|
||||
return { collection, id };
|
||||
const read = segmentsOf(record.path);
|
||||
const raw = read.pop() ?? "";
|
||||
const ids = [...new Set([decodedSegment(raw), raw])];
|
||||
const written = `/${read.join("/")}`;
|
||||
for (const id of ids)
|
||||
if (state.entity(type, written, id) !== undefined)
|
||||
return { collection: written, id };
|
||||
const collections = [
|
||||
...new Set(state.ofType(type).map((entity) => entity.collection)),
|
||||
]
|
||||
.flatMap((collection) => {
|
||||
const verbatim = segmentMatch(segmentsOf(collection), read);
|
||||
return verbatim === undefined ? [] : [{ collection, verbatim }];
|
||||
})
|
||||
.sort((left, right) => right.verbatim - left.verbatim);
|
||||
for (const { collection } of collections)
|
||||
for (const id of ids)
|
||||
if (state.entity(type, collection, id) !== undefined)
|
||||
return { collection, id };
|
||||
return { collection: written, id: ids[0]! };
|
||||
};
|
||||
|
||||
export const checkItemRead = (
|
||||
@@ -403,19 +414,19 @@ export const checkListRead = (
|
||||
check: ReadBodies & { readonly idField: string; readonly collection: string },
|
||||
): CheckOutcome | undefined => {
|
||||
if (!succeeded(record)) return undefined;
|
||||
const { body, base } = check;
|
||||
const baseItems =
|
||||
base === undefined ? undefined : listItems(base, view.pointer);
|
||||
if (body === undefined || base === undefined || baseItems === undefined)
|
||||
return undefined;
|
||||
// Serving answers a page too small for the collection from its recording, so such a read checks nothing.
|
||||
const size = pageSize(record.query);
|
||||
const size = pageBound(record.query, baseItems.length);
|
||||
if (
|
||||
view.membership === "all" &&
|
||||
size !== undefined &&
|
||||
state.members(view.type, check.collection).length > size
|
||||
)
|
||||
return undefined;
|
||||
const { body, base } = check;
|
||||
const baseItems =
|
||||
base === undefined ? undefined : listItems(base, view.pointer);
|
||||
if (body === undefined || base === undefined || baseItems === undefined)
|
||||
return undefined;
|
||||
// A usable body that no longer holds the list is not what serving answers on this line.
|
||||
const items = listItems(body, view.pointer);
|
||||
const bodies = { ...check, body, base, items: items ?? [], baseItems };
|
||||
|
||||
@@ -40,7 +40,7 @@ import {
|
||||
type Observed,
|
||||
} from "./environmentStateWriteObserve.js";
|
||||
import type { FieldRule, WriteCall } from "./environmentStateWriteRules.js";
|
||||
import { integerOf } from "./environmentStateWriteValues.js";
|
||||
import { integerOf, isUuid, readClock } from "./environmentStateWriteValues.js";
|
||||
|
||||
/** The id a response names alone: a scalar body, or an object's id field or single scalar member. */
|
||||
const respondedId = (
|
||||
@@ -186,6 +186,25 @@ const followsSequences = (
|
||||
return integerOf(created.fields[rule.field]) === next;
|
||||
});
|
||||
|
||||
/** Whether a later call's generated values take the forms serving generates them in: clocks in the learned format, unique values and UUID identities as UUIDs, and sequences in order. */
|
||||
const followsGenerated = (
|
||||
context: EffectContext,
|
||||
draft: WriteDraft,
|
||||
created: Pick<StateEntity, "type" | "collection" | "fields">,
|
||||
): boolean =>
|
||||
draft.plan.rules.every((rule) => {
|
||||
const value = created.fields[rule.field];
|
||||
if (rule.rule === "clock")
|
||||
return (
|
||||
JSON.stringify(readClock(value)?.format) === JSON.stringify(rule.format)
|
||||
);
|
||||
return (
|
||||
(rule.rule !== "unique" &&
|
||||
(rule.rule !== "identity" || rule.generator !== "uuid")) ||
|
||||
isUuid(value)
|
||||
);
|
||||
}) && followsSequences(context, draft, created);
|
||||
|
||||
/** Record the values a create's sequences took, which the next call follows. */
|
||||
const takeSequences = (
|
||||
draft: WriteDraft,
|
||||
@@ -275,7 +294,7 @@ export const applyCreate = (
|
||||
id === undefined ||
|
||||
context.state.entity(type, collection, id) !== undefined ||
|
||||
(later &&
|
||||
!followsSequences(context, draft, {
|
||||
!followsGenerated(context, draft, {
|
||||
type,
|
||||
collection,
|
||||
fields: fields.value,
|
||||
|
||||
@@ -134,6 +134,6 @@ export const applyDocument = (
|
||||
return {
|
||||
kind: "applied",
|
||||
targets: [documentTarget(tool)],
|
||||
...(fields === undefined ? {} : { fields }),
|
||||
...(fields === undefined ? {} : { fields, pointers: true }),
|
||||
};
|
||||
};
|
||||
@@ -122,8 +122,10 @@ export type EffectResult =
|
||||
readonly kind: "applied";
|
||||
readonly targets: readonly string[];
|
||||
readonly entity?: WrittenEntity;
|
||||
/** The fields an update wrote; a create writes the whole entity. */
|
||||
/** The fields an update wrote, or the pointers a document write wrote; a create writes the whole entity. */
|
||||
readonly fields?: readonly string[];
|
||||
/** Set when the fields are a document's pointers, each covering the members under it. */
|
||||
readonly pointers?: boolean;
|
||||
/** The fields an update sent or changed, derived ones included, and those of them its increments add to. */
|
||||
readonly changed?: {
|
||||
readonly fields: readonly string[];
|
||||
|
||||
@@ -301,6 +301,9 @@ export const explainUpdate = (
|
||||
value: evidence.after[member]!,
|
||||
});
|
||||
}
|
||||
// Serving keeps every field an update does not write, so one the update removed is a change no rule models.
|
||||
for (const field of Object.keys(evidence.before))
|
||||
if (!Object.hasOwn(evidence.after, field)) unexplained.push(field);
|
||||
for (const field of Object.keys(evidence.after)) {
|
||||
if (
|
||||
Object.hasOwn(body, field) ||
|
||||
@@ -448,13 +451,19 @@ export const conflictsWithPlan = (
|
||||
explained: ReturnType<typeof explainUpdate>,
|
||||
): boolean => {
|
||||
const body = plan.body!;
|
||||
const fixed = new Set(body.fixed.map((entry) => entry.member));
|
||||
const fixed = new Map(body.fixed.map((entry) => [entry.member, entry.sent]));
|
||||
const rules = new Map(plan.rules.map((rule) => [rule.field, rule]));
|
||||
const { identity } = explained.body;
|
||||
// Serving checks a call's identity member before its fixed values, so a member both cannot take every recorded call.
|
||||
return (
|
||||
explained.body.accepted.some((member) => fixed.has(member)) ||
|
||||
explained.body.fixed.some((entry) =>
|
||||
body.accepted.includes(entry.member),
|
||||
explained.body.fixed.some(
|
||||
(entry) =>
|
||||
body.accepted.includes(entry.member) ||
|
||||
entry.member === body.identity ||
|
||||
(fixed.has(entry.member) && !same(fixed.get(entry.member), entry.sent)),
|
||||
) ||
|
||||
(identity !== null && fixed.has(identity)) ||
|
||||
explained.rules.some((rule) => {
|
||||
const held = rules.get(rule.field);
|
||||
return held !== undefined && !same(held, rule);
|
||||
|
||||
Reference in new issue
Block a user