Files
reverse-engineering-anything/src/domain/analysisView/EnvironmentStateWriteSimulation.ts
T
51ad84516e feat(environment): serve sorted creates, listed references and toggles (#7600)
* feat(environment): serve the writes callers make without recorded form fields

- A create member a recording sent empty ("", null, false, [] or {}) is no
  longer required; a call that leaves it out is applied as sending that value
  (body.blank).
- List views render envelope members that list the listed entities' ids
  (Paperless's `all`), so creates in such collections are confirmed.
- Updates move their entity in collections kept sorted by a field they write,
  and a failed whole-collection read after an update can teach that order.
- A later create that stores an unruled field otherwise is withheld at the
  create, and an item read may fill values a create's answer had not loaded.
- An unsent field respelled between null, [] and {} is unchanged.
- The skill asks for one create with every field and one with only what a
  person must give.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skill): show how to read signed-in pages before recording

Agents guessed capture keys (artifacts, dom, final) to see pages behind
sign-in, which inspect_web_page cannot reach.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): accept every empty spelling recorded creates sent

A create that fills only what a person must give sends `null` where the
full create sent a string, and `null` or `false` for the same flag. The
merged plan now accepts each recorded empty value, and a call that leaves
the member out is applied as the earliest call sent it.

Also adds the spec folder for this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skill): keep the vary-fields step neutral on UI or API writes

Agents read "the form" as a cue to write through the UI, where some
applications' request bodies are not exposed; a body-unusable UI write
now re-records once with request steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): record verification for blank members and sorted writes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): address review of blank members and sorted writes

- Bulk updates move their entities in turn against the others' earlier
  values and install the final order once, as replay does.
- A member's recorded empty values stay accepted once a later call teaches
  its default; a left-out member with a default keeps the default.
- Write answers spell empty lists and maps as their recorded answers did.
- A rename can teach a sort when the creates chose one by a field the
  update did not write, such as ids that ascend with titles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(environment): serve the writes that later reads explain

- An item read whose object is marked deleted (deleted, isDeleted or
  is_deleted true) agrees with an absent entity, as EspoCRM answers a
  deleted record; such an object never seeds or adopts an entity.
- maxSize, max_results and maxResults are page sizes, for whole-collection
  list lines and the list query's limit role.
- An item view also shows the fields only reads after its type's first write
  show, when no initial entity holds them (Vikunja's task subscription),
  without requiring them; write answers leave such a field out for entities
  without it, and it no longer withholds the write as incomplete-entities.
- The skill asks updates to change the fields a person would change, and
  expects a deleted object to read back as 404 or marked deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): answer sent empty members as sent, and keep a placed create's order

- A write answer spells an empty list or map the call did not send as its
  recorded answer did; one the call sent is answered as sent, so a create
  sending `[]` is not answered `null` because the first recorded create was.
- A read listing the whole collection that placed a create leaves the create
  only the sorted inserts whose order that read follows, so a later rename
  cannot teach a sort an earlier read ruled out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): answer bulk updates without fields only later reads show

A bulk update's answer renders each entity with the same allowance a single
update's answer has: a field only reads after the type's first write showed,
which no initial entity holds, is left out for an entity without it, so a
bulk update of a seeded task is no longer refused for lacking it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): correct how the EspoCRM contact update task passes

The agent trajectories show gpt-oss passes it by creating a replacement contact
and deleting the original, since the served update refuses emailAddress; the
update itself is refused in both builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): answer a read of a served-deleted entity marked deleted

An application that soft-deletes still answers a deleted record by id, marked
deleted. Serving now keeps an entity a served delete removed from a type whose
recorded item reads show that, until a reset or load and within the size
data/add loads, and answers a read of it with its last fields and the flag
true, instead of refusing the read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep unloaded create answers, check bulk answers with sent members, narrow inserts only on collection-order reads

- A create whose item read loaded values its answer had not records them
  (`observation.read.unloaded`), and a served create's answer keeps them as
  the recorded answer showed them, a zero-time owner included, while the state
  holds the read's.
- Replay checks a bulk update's recorded answers with each target's sent
  members, as serving answers them, so echoed `null` and `[]` no longer make
  the plan body-unusable.
- A whole-collection read whose query sorts it shows the query's order, so it
  no longer narrows a create's sorted insert candidates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): record the census with main's unknown-clock rule merged

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(environment): serve query numbers, counted values and made objects

- A query parameter accepts a number or boolean, which serving sends as its
  JSON text, as agents send page sizes and flags.
- A sequence may start at the value its first create took above every value
  its type held (Vikunja's per-project task index), and a later create's
  composed, template or started-sequence rule is adopted when it also spells
  the first create's value.
- A made-object rule explains an object the application makes with each
  created entity: members the request stamped, members holding the entity's
  own id, and counts one past the held objects' values.
- A create's answer names the id of the one object a one-element array member
  holds, so bulk creates link to their type.
- The skill reads each listed object by id before writing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(environment): let a linked create take path values it stores

A create linked to its type's own collection, whose path has no parameters,
accepts any path values when its plan stores each one in a field, as a Vikunja
task stores its project; replay and serving apply it alike.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): learn a sort from a rename only when moving its entities reaches it

Serving moves only the entities a write changes in a collection kept sorted,
so replay now tries a sort after a rename only when the entities no write the
read exercised changed already follow it. Seeded folders listed out of title
order no longer teach a title sort that serving would answer with another
order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep unloaded create answers a sent value did not echo, and learn no sort from query-sorted reads

- A served create keeps a value its recorded answer had not loaded unless the
  call sent that member and the plan stores it as sent, so a create sending
  `active: false` is answered `false` as recorded while the state holds the
  read's `true`.
- A rename no longer teaches a sort from a whole-collection read its query
  sorts, whose ties need not follow the collection's own order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): verify query values, counted values and made objects

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): learn the soft-delete flag in the state view and serve by it

Extraction now records the member later item reads set true on a record the
application soft-deleted, and the item view carries it as `deleted_flag`.
Serving reads that fact instead of scanning recorded bodies itself, so one
owner decides what a soft delete looks like.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): record the call-values checks at #7274 93821024

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): learn the soft-delete flag from item reads with a query too

A client may read a deleted record back only with a query, such as a
cache-busting parameter. Every successful read of the item tool now shows the
flag; only plain later reads still add fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): record the call-values checks at #7274 45832406

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): learn sorts only from orders a list read confirmed

- A sort retry after a failed whole-collection read requires the entities no
  write changed since a list last showed them to follow it already, rather than
  excusing every entity the update's tool ever wrote: an earlier rename a list
  confirmed in place stays where it was.
- An update takes its creates' sort only from a create a list read showed
  placed; replay still tries a sort a create guessed from the initial order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): drop entities read back as deleted and answer soft deletes as deleted

- A held entity a later read shows marked deleted is dropped, not adopted with
  the marked object's fields.
- Bulk-update replay renders answers with the optional and later-only fields
  serving uses, so a bulk update of created and seeded entities replays.
- A served delete of a soft-deleting type answers with the entity marked
  deleted, single and bulk.
- A deleted flag only query reads showed is a field of the item view, so a
  served tombstone keeps it where live records omit it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): check made objects as recorded and keep their ids and clocks honest

- A made object's clock counts as a write clock, so a capture with no clock
  anchor withholds the create instead of stamping 1970.
- Members holding the entity's identity follow the id written, generated or
  sent, through the type's id field.
- A later create's made object is checked as recorded, before its own members
  are rewritten, so one naming another entity withholds the plan.
- A later create's template is adopted only when it spells that create's value
  too, not just when the first create's text holds its source value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): move updated entities only by placed sorts and keep query-sorted reads from confirming order

Replay moved an updated entity by a sort a create only guessed, while the
update plan exported only placed sorts, so a read could confirm an update
by a move serving would not make. Both now use the placed sort. A passing
list read clears its entities from the moved set only when the state
decides the order of every entity it lists, so a read whose query sorts
it no longer hides an earlier move from a later sort check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): count six withheld plans that explain more, not seven

The bulk task create counted among them is served on this branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): try a learned sort by moving each pending write in turn, as serving does

#retrySort sorted the whole collection, which a list could match while
serving, moving each renamed entity in turn against the others' values
before their own writes, gave another order. The state now records the
values each write replaced, and the retry replays the writes since a
list in collection order last showed their entities.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): answer a served delete with the deleted flag its recorded answer showed

An application may answer a delete with the record as it was and mark it
deleted only on later reads. Delete answers, single and bulk, now take
the flag's value from the recorded answer; reads of a served-deleted
entity still show it marked deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(environment): set aside two hunks next to lines main's saved-values change rewrites

Main's #7267 changes the lines beside the blank-member docs and the
empty-spelling check in explainUpdate. Setting them aside lets that merge
apply cleanly; the commit after it puts them back on top of main's text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(environment): restore the set-aside hunks on main and bump the skill to 51

Puts the blank-member docs and the empty-spelling check back beside main's
slug and composed text and heldSlugRule. Main is at skill 50 with
different text, so this branch's skill changes take 51.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(environment): bump the skill to 52 above #7260's 51

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(environment): put back the call-values hunks main's merge took its side for, and bump the skill to 53

The merge of main kept main's side where it overlapped this branch, so it
could commit without a conflict. This restores this branch's side next to
main's: made-object agreement beside the & slug spellings, sequence starts
in the earlier evidence (now built by earlierOf), the combined value
imports, and recording step 2 beside main's UI-first step 3. Main is at
skill 50 and #7274 at 52, so this branch takes 53.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep sorts a list showed in place ruled out, and created answers' nested members as recorded

A whole list in collection order that showed a renamed entity where a
sort by the renamed field would have moved it now rules that sort out, so
a later rename of the same entity cannot teach it. A create's answer
leaves out members its item read added inside an object the answer
showed, as the recorded answer did; the state still stores them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(environment): serve writes whose calls send different members

Recordings now vary what a type's two creates and updates send, and REA
withheld writes where a later call left out what the first sent. An update
plan's required members narrow to those every call sent; a member one create
sent and another left for the application to number is a sequence that
takes the value a call sends; a PUT's left-out member the application
cleared becomes that member's default; and an update's clock is held as
the read after it spells it. The skill spaces creates (skill 54).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep a later create's made object only with clocks its request stamped

followsMadeObjects only checked that a made object's clocks parsed, so a
later create whose object showed an older clock still passed, while serving
stamps the write's clock. Its clocks must now be in the rule's format and
within the stamp window of that create's request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): accept sent values either way, check defaults against every call, and store update clocks as reads spell them

A sequence that takes sent values now accepts one a later create sent
after the first left the member to be numbered. A default an update learned
must agree with every update that left its member out, else the plan is
withheld. An update plan whose read-back spells a stamped clock at a coarser
precision keeps that read as observation.read, and serving stores the
update's clocks in its spelling while answers keep the response's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): check recorded member and sequence values

* fix(environment): compare soft-deleted read fields

* fix(environment): reject contradictory learned sequences

* fix(environment): preserve initially deleted item observations

* fix(environment): index stored deleted entities separately

* fix(environment): replay lists that include deleted records

* feat(environment): serve sorted creates, listed references and toggles

A create a whole listing placed last is placed again by the sort a later
listing shows. A field holding the entities a sent list of ids names is a
reference with `many`, and a create that sent the member empty agrees with
one that named entities. A scenario request whose body Playwright did not
expose and that names no Content-Type sent no body. An item action calls
show flipping a boolean both ways learns a `toggle` rule. Skill 55 asks
recordings to run an action twice on one object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): count a sequence number a create sends, as replay does

Serving now records a number a call sends for a `sequence` with `sent` in the
same count history it generates from, so after that entity is deleted the next
generated number still follows it, as replay's takeSequences does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): record the final gate run in the verification

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): make no count of a made-object member earlier creates held alike

A count only a later create shows starts at the exemplar's value, which is
right only while the exemplar is the one create accepted: creates accepted
after it showed the member unchanged, which a count would have moved. The
merge now refuses such a count, as the top-level sequence check does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): restate the verification after merging #7539

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): check PUT defaults against reads, and respell update clocks by serving's rule

A default a PUT learns must also agree with what the read after an earlier or
the current call showed for the member its answer left out, so a call whose
read showed the member kept withholds a later clearing default.

Replay and serving now choose an update clock's stored spelling by one rule,
readSpellsCoarser: the read after it at a coarser precision in the same zone
and separator. A read in another zone or separator is no longer adopted by
replay, which serving could not store, and a create's plain clocks stay as
answered, as serving makes them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep sorts earlier listings ruled out, and empty added references

A sort a whole listing that placed a create did not follow stays ruled out for
that create, though the entity that showed it is later deleted. A create that
sends a reference's member empty after a later create added the reference
agrees with it, as with a reference the first create showed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): restate the verification after the review round

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): leave out a reference the observed create left out

A create the plan observes that left out a reference's member stored no such
field, so a served call leaving the member out stores none either, though a
later recorded create sent it empty; a call sending it empty still stores that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): restate the verification after #7539 merged main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep the largest sequence value in replay, and store update clocks by the kept read

Replay now keeps the largest value a sequence took, as serving's commitWrite
does, so a lower sent value no longer lowers the next number.

Serving stores every update's clocks in the spelling of the read the plan
keeps as observation.read; replay now does the same through
readSpelledClocks, instead of adopting each update's own read, so a later
update whose read spells its clock otherwise contradicts the plan. The plain
clock branch of restampedFields, which no caller used any more, is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): restate the verification after merging #7539 3b800a10

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep learned sorts, references, and toggles faithful

Do not adopt a sort an earlier update already ruled out. Learn a list
reference only when every element shows the same nonempty fields. Promote
a toggle only when every call started from a boolean.

* test(environment): compare list-reference fields with matching values

The mixed-field case uses the held tag's color, so rejection comes from
the field set rather than a value the recording never stored.

* test(mcp): assert bundle path issues by field

The compare_bundles rejection reports each relative path as an
invalid_format issue instead of one joined diagnostic sentence.

* test(mcp): assert field issues for bundle paths and Dart ABI

Relative bundle paths and rejected ABI selectors are reported as
input issues on those fields, not as one joined diagnostic sentence.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-10-29 18:46:18 +08:00

1196 lines
45 KiB
TypeScript

import { CapturedWriteCompletion } from "./environmentStateWriteCompletion.js";
import type { JsonValue } from "../jsonValue.js";
import {
at,
changesState,
isObject,
isStateWrite,
listItems,
listsAll,
markedDeleted,
normalizedPath,
ownMember,
same,
scalarText,
type JsonObject,
type ParsedRequestBody,
type StateEntity,
type StateRecordFacts,
type StateToolFacts,
type StateType,
type StateView,
} from "./environmentStateRules.js";
import { UnseenEntities } from "./environmentStateWriteAdoption.js";
import { FailedReadBlame } from "./environmentStateWriteBlame.js";
import { addressedWrite } from "./environmentStateWriteAddress.js";
import { withoutMembers } from "./environmentStateResponseMembers.js";
import { linkedTarget } from "./environmentStateWriteLinks.js";
import {
laterItemFields,
updatedEntity,
} from "./environmentStateWriteApply.js";
import { projectedRead } from "./environmentStateWriteObserve.js";
import { OmittedFields } from "./environmentStateWriteOmitted.js";
import { SortedArrays } from "./environmentStateWriteSorted.js";
import { sameQuery } from "./environmentStateWriteStamps.js";
import {
checkDocumentRead,
checkItemRead,
checkListRead,
collectionTarget,
entityTarget,
isAbsence,
itemTarget,
succeeded,
withBodies,
showsPlacement,
SimulatedState,
type CheckOutcome,
type PendingWrite,
} from "./environmentStateWriteChecks.js";
import {
checkedLines,
servedLine,
type CheckedLines,
type ServedLine,
} from "./environmentStateCheckedLines.js";
import {
exercisedBy,
interference,
type WriteHistory,
forgetChecked,
runningAt,
shownFields,
shownWrites,
touchTarget,
WriteEffects,
type PendingTargets,
type TouchedTargets,
writtenFieldChecks,
} from "./environmentStateWriteAttribution.js";
import { applyBulk } from "./environmentStateWriteBulk.js";
import { applyCreate } from "./environmentStateWriteCreate.js";
import { applyDocument } from "./environmentStateWriteDocument.js";
import { applyDelete } from "./environmentStateWriteDelete.js";
import {
applyUpdate,
built,
type EffectContext,
type EffectResult,
type WriteDraft,
type WriteSimulationInput,
type WriteStep,
} from "./environmentStateWriteEffects.js";
import {
classifyWrite,
prefixRelated,
unservedPlan,
type InsertRule,
type StateWritePlan,
} from "./environmentStateWriteRules.js";
import { WriteTaint } from "./environmentStateWriteTaint.js";
import { writeVerdict } from "./environmentStateWriteVerdict.js";
import { compareValues, orderedBy } from "./environmentStateWriteExplain.js";
/** Whether members are in a sorted rule's order, ties kept in place as an insert keeps them. */
const keptInOrder = (
members: readonly JsonObject[],
rule: Extract<InsertRule, { rule: "sorted" }>,
): boolean =>
members.every((member, index) => {
if (index === 0) return true;
const order = compareValues(
ownMember(members[index - 1]!, rule.field),
ownMember(member, rule.field),
);
return (
order !== undefined &&
(rule.direction === "ascending" ? order <= 0 : order >= 0)
);
});
/** The sorts by a field every listed item holds as a string or number, and whether the items follow each. */
const listedSorts = (
items: readonly JsonObject[],
): {
readonly rule: Extract<InsertRule, { rule: "sorted" }>;
readonly follows: boolean;
}[] =>
[...new Set(items.flatMap((item) => Object.keys(item)))]
.filter((field) =>
items.every((item) =>
["string", "number"].includes(typeof ownMember(item, field)),
),
)
.flatMap((field) =>
(["ascending", "descending"] as const).map((direction) => ({
rule: { rule: "sorted", field, direction } as const,
follows: orderedBy(items, field, direction),
})),
);
/** Whether a list read's query sets a parameter the view's query model reads as a sort or its direction. */
const sortedByQuery = (
view: Extract<StateView, { shape: "list" }>,
record: StateRecordFacts,
): boolean => {
const sent = new Set(
[...new URLSearchParams(record.query ?? "").keys()].map((key) =>
key.toLowerCase(),
),
);
return Object.entries(view.query?.parameters ?? {}).some(
([parameter, { role }]) =>
(role === "sort" || role === "direction") &&
sent.has(parameter.toLowerCase()),
);
};
const APPLY = {
create: applyCreate,
update: applyUpdate,
delete: applyDelete,
document: applyDocument,
} as const;
/**
* Replay the capture's timeline on a copy of the initial state: modelled writes apply their
* effects, other writes taint related types, and each read the served state would answer is
* compared with its recording and attributed to the writes it exercises.
*/
export class EnvironmentStateWriteSimulation {
readonly #input: WriteSimulationInput;
readonly #state: SimulatedState;
readonly #drafts = new Map<string, WriteDraft>();
readonly #toolOf = new Map<number, StateToolFacts>();
readonly #types = new Map<string, StateType>();
/** Each list or document view's checked request lines, with the recording a call of the line is answered on. */
readonly #checkedLines: ReadonlyMap<string, CheckedLines>;
/** Reads a write's effect was learned from, with the tools that borrowed each. */
readonly #observations = new Map<number, Set<string>>();
readonly #taint: WriteTaint;
readonly #touched: TouchedTargets = new Map();
readonly #effects = new WriteEffects();
/** Fields reads show the application leaves out, which serving then does not require. */
readonly omitted = new OmittedFields((bytes) => this.#input.io.retain(bytes));
/** Array fields reads showed a view lists sorted. */
readonly #sorted: SortedArrays;
readonly #lastCheck = new Map<string, number>();
/** When the latest modelled write to each target since its last check ended. */
readonly #touchedSince: PendingTargets = new Map();
/** Timeline indexes of writes the gate does not model. */
readonly #unmodelledAt: number[] = [];
readonly #writeCompletion: CapturedWriteCompletion;
readonly #history: WriteHistory = {
touched: this.#touched,
pending: this.#touchedSince,
checked: this.#lastCheck,
unmodelledAt: this.#unmodelledAt,
};
readonly #log: {
readonly index: number;
readonly tool: string;
readonly path: string;
}[] = [];
/** Entities a create inserted, by target, with the create's tool. */
readonly #inserted = new Map<string, string>();
/** Targets a write changed or placed since a passing list read last showed them, whose position no read has confirmed. */
readonly #moved = new Set<string>();
/** The writes to those targets' entities, in order. */
#pending: PendingWrite[] = [];
/** Sorts, by type, that a whole list in collection order ruled out: moving the writes it showed by them gives another order. */
readonly #refutedSorts = new Map<string, Set<string>>();
/** Creates whose insert rule a passing list check relied on, so it is no longer changed. */
readonly #placed = new Set<string>();
/** Placed creates a read other than a whole listing in collection order placed, whose insert stays as it is. */
readonly #settledInserts = new Set<string>();
/** Sorts a whole listing that placed a create did not follow, by create, which no later listing adds back. */
readonly #unfollowedSorts = new Map<string, Set<string>>();
/** The order a read showed a type's collections sorted in after an update, when no create of it showed one. */
readonly #learnedSorts = new Map<string, InsertRule>();
/** Entities writes no plan models made, taken when an item read first shows them. */
readonly #unseen: UnseenEntities;
readonly #blame: FailedReadBlame;
constructor(input: WriteSimulationInput) {
this.#input = input;
this.#writeCompletion = new CapturedWriteCompletion(input.io.retain);
this.#state = new SimulatedState(input.entities, input.documents);
this.#unseen = new UnseenEntities(input.entities, {
state: this.#state,
io: input.io,
lastCheck: this.#lastCheck,
idField: (type) => this.#types.get(type)?.id_field,
});
this.#blame = new FailedReadBlame({
state: this.#state,
io: input.io,
records: input.records,
touched: this.#touched,
pending: this.#touchedSince,
effects: this.#effects,
inserted: this.#inserted,
idField: (type) => this.#types.get(type)?.id_field,
volatileOf: input.volatileOf,
});
for (const type of input.types)
this.#types.set(type.collection_template, type);
for (const tool of input.tools)
for (const ordinal of tool.record_ordinals)
this.#toolOf.set(ordinal, tool);
this.#checkedLines = checkedLines(input);
this.#sorted = new SortedArrays(input.views.values(), input.entities, {
state: this.#state,
io: input.io,
idField: (type) => this.#types.get(type)?.id_field,
});
this.#taint = new WriteTaint(
[...this.#types.keys()],
new Map(
input.documents.map((document) => [document.tool, document.path]),
),
this.#state,
);
this.#state.keepSorted((type) => this.#sortOf(type));
}
/**
* The sorted order a type's placed creates insert by, when they agree on one and nothing contradicted them: updates
* move their entity only by an order a list read showed, in replay as in serving.
*/
#sortOf(type: string): InsertRule | undefined {
const rules = new Map<string, InsertRule>();
for (const { plan, observed, contradictedBy } of this.#drafts.values())
if (
plan.effect === "create" &&
plan.target === type &&
observed !== undefined &&
plan.reason === null &&
contradictedBy === undefined &&
plan.insert?.rule === "sorted" &&
this.#placed.has(plan.tool)
)
rules.set(JSON.stringify(plan.insert), plan.insert);
if (rules.size === 0) return this.#learnedSorts.get(type);
return rules.size === 1 ? [...rules.values()][0] : undefined;
}
/** Every state-changing tool's plan, in manifest order, with its verdict. */
run(): StateWritePlan[] {
const documents = new Map<string, string>();
for (const tool of this.#input.tools)
if (this.#input.views.get(tool.name)?.shape === "document")
documents.set(normalizedPath(tool.path_template), tool.name);
const types = new Set(this.#types.keys());
for (const tool of this.#input.tools) {
if (
!changesState(tool.method, tool.path_template) ||
this.#input.readOnly.has(tool.name)
)
continue;
const link = linkedTarget(tool, classifyWrite(tool, types, documents), {
...this.#input,
isWrite: (record) => this.#isWrite(record),
viewOf: (ordinal) =>
this.#input.views.get(this.#toolOf.get(ordinal)?.name ?? ""),
});
const addressed =
link.effect === "none"
? addressedWrite(tool, {
...this.#input,
types,
idField: (type) => this.#types.get(type)?.id_field,
views: this.#input.views.values(),
})
: undefined;
const { effect, target } = addressed ?? link;
this.#input.io.retain(512);
this.#drafts.set(tool.name, {
plan: {
...unservedPlan(tool.name, effect, target, "no-state-target"),
reason: effect === "none" ? "no-state-target" : null,
...(addressed === undefined ? {} : { address: addressed.address }),
},
observed: undefined,
candidates: [],
confirmed: [],
contradictedBy: undefined,
sideEffect: false,
calls: 0,
updates: [],
sequences: new Map(),
starts: new Map(),
createCollections: [],
sentLargest: new Map(),
steady: new Set(),
unstamped: new Set(),
kept: new Set(),
fixedWidth: new Map(),
ampersands: new Map(),
linked: link.collection,
});
}
// Reads may finish after writes with the same or a later start: index the
// complete capture before replay so ordinal tie-breaking cannot create proof.
for (const record of this.#input.records)
if (this.#isWrite(record))
this.#writeCompletion.write(
record.path,
this.#input.chronology.endMs(record),
this.#input.chronology.startMs(record),
);
this.#input.records.forEach((record, index) => this.#step(record, index));
const views = new Map(
this.views([...this.#input.views.values()]).map((view) => [
view.tool,
view,
]),
);
const sorts = new Map(
[...this.#types.keys()].map((type) => [type, this.#sortOf(type)]),
);
return [...this.#drafts.values()].map((draft) => {
// A view that became recorded cannot supply proof for a served write.
let kept = 0;
for (const ordinal of draft.confirmed) {
const tool = this.#toolOf.get(ordinal)?.name;
if (tool !== undefined && views.get(tool)?.shape !== "recorded")
draft.confirmed[kept++] = ordinal;
}
draft.confirmed.length = kept;
// An update moves its entity in a sorted collection when it writes the field sorted by.
const sort =
draft.plan.effect === "update" && draft.plan.target !== null
? sorts.get(draft.plan.target)
: undefined;
if (
sort?.rule === "sorted" &&
draft.plan.rules.some((rule) => rule.field === sort.field)
)
draft.plan.insert = sort;
return writeVerdict(draft, this.#input);
});
}
#step(record: StateRecordFacts, index: number): void {
const tool = this.#toolOf.get(record.ordinal);
// A redirected or unanswered write may have changed state, even made an entity, but only a 2xx one shows its effect.
if (this.#isWrite(record) && !succeeded(record)) {
this.#unmodelled(record.path, index);
this.#unseen.unplanned(index);
} else if (this.#isWrite(record)) this.#write(record, index, tool);
else if (tool?.method === "GET") this.#read(record, index, tool);
}
/** Whether a record may change state: a state write whose tool the capture does not show reading only. */
#isWrite(record: StateRecordFacts): boolean {
return (
isStateWrite(record) &&
!this.#input.readOnly.has(this.#toolOf.get(record.ordinal)?.name ?? "")
);
}
#context(
writer: string,
request?: { readonly ordinal: number; readonly body: ParsedRequestBody },
): EffectContext {
return {
isWrite: (record) => this.#isWrite(record),
state: this.#state,
io:
request === undefined
? this.#input.io
: {
...this.#input.io,
requestBody: (ordinal) =>
ordinal === request.ordinal
? request.body
: this.#input.io.requestBody(ordinal),
},
records: this.#input.records,
chronology: this.#input.chronology,
viewOf: (ordinal) => {
const view = this.#input.views.get(
this.#toolOf.get(ordinal)?.name ?? "",
);
return view === undefined ? undefined : this.#viewOf(view);
},
idField: (type) => this.#types.get(type)!.id_field,
volatileOf: this.#input.volatileOf,
observed: (ordinal) =>
this.#observations.set(
ordinal,
(this.#observations.get(ordinal) ?? new Set()).add(writer),
),
unknown: (type, collection, id) =>
this.#taint.unknown(type, collection, id),
initialDocument: (tool) =>
this.#input.documents.find((document) => document.tool === tool),
startedWith: (type) =>
this.#input.entities.filter((entity) => entity.type === type),
laterFields: (type) =>
laterItemFields(this.#input.views.values(), this.#input.entities, type),
pending: (target) =>
[...(this.#touched.get(target)?.keys() ?? [])].some(
(name) => name !== writer,
),
};
}
#write(
record: StateRecordFacts,
index: number,
tool: StateToolFacts | undefined,
): void {
const draft = tool === undefined ? undefined : this.#drafts.get(tool.name);
const effect = draft?.plan.effect;
if (
draft === undefined ||
tool === undefined ||
effect === undefined ||
effect === "none"
) {
this.#unmodelled(record.path, index);
this.#unseen.unplanned(index);
return;
}
const observeOnly = draft.plan.reason !== null;
if (!observeOnly) draft.calls += 1;
// Serving pins the exemplar's query, so a call with another may do something else and confirms nothing.
const pinned =
!built(draft) ||
sameQuery(
this.#input.io.queryOf(draft.plan.exemplar_ordinal ?? -1),
record.query,
);
const step: WriteStep = { record, index, tool, observeOnly, pinned };
const { address } = draft.plan;
const bulk = address !== undefined && address.by !== "action";
const apply = (body?: ParsedRequestBody): void => {
this.#state.track();
const context = this.#context(
tool.name,
body === undefined ? undefined : { ordinal: record.ordinal, body },
);
const results = bulk
? applyBulk(context, draft, step)
: [APPLY[effect](context, draft, step)];
// A call of a tool whose plan was never built, or one a linked plan does not place, is no more modelled than one of a tool without a plan.
if (
results.some((result) => result.kind === "unknown") &&
(!built(draft) || draft.linked !== undefined)
)
this.#unseen.unplanned(index);
for (const result of results)
this.#settle(result, {
index,
tool: tool.name,
path: record.path,
end: this.#input.chronology.endMs(record),
});
};
// Item deletes use their path; calls outside a built plan's query never interpret their body.
if (pinned && (effect !== "delete" || bulk))
this.#input.io.withRequestBody(record.ordinal, apply);
else apply();
}
/** Attribute an applied effect to its tool; a recorded outcome is the state's new baseline; an unknown one taints. */
#settle(
result: EffectResult,
write: {
readonly index: number;
readonly tool: string;
readonly path: string;
readonly end: number | undefined;
},
): void {
if (result.kind === "unknown") {
this.#unmodelled(write.path, write.index);
return;
}
if (result.entity !== undefined) {
this.#unseen.touched(result.entity);
const { type, collection, id } = result.entity;
this.#taint.known(type, collection, id);
}
if (result.kind === "observed") {
this.#effects.observed(this.#touched, result.targets);
this.#blame.rebased(result.targets[0]!, write.index);
// A recorded outcome replaces the entity's values, not its collection's membership.
this.#touched.delete(result.targets[0]!);
return;
}
this.#effects.supersede(this.#touched, write.tool, result);
if (result.entity !== undefined) {
const { type, collection, id } = result.entity;
const target = entityTarget(type, collection, id);
const before = this.#state.priorFields(target);
if (before !== undefined) this.#pending.push({ target, before });
}
for (const target of result.targets) {
this.#moved.add(target);
touchTarget(this.#touched, this.#touchedSince, target, {
...write,
fields: result.fields,
});
}
this.#log.push(write);
if (result.inserted === undefined) return;
this.#inserted.set(result.inserted, write.tool);
if (result.entity !== undefined)
this.#blame.inserted(result.entity, write.tool);
}
/**
* An item read of an entity an unmodelled write may have changed shows it again: the writes that
* changed it are checked on their own fields, and the state takes what the read recorded.
*/
#resync(
view: Extract<StateView, { shape: "item" }>,
record: StateRecordFacts,
entity: StateEntity,
): void {
const target = entityTarget(entity.type, entity.collection, entity.id);
const writes = this.#touched.get(target) ?? new Map();
this.#taint.known(entity.type, entity.collection, entity.id);
this.#touched.delete(target);
if (
isAbsence(record.status) ||
this.#input.io.withBody(record.ordinal, (body) =>
markedDeleted(
body === undefined || body === null
? undefined
: at(body, view.pointer),
),
)
) {
this.#removeEntity(entity);
return;
}
const object = this.#checkFields(view, record, entity, writes);
if (object !== undefined)
this.#state.put(updatedEntity(entity, { ...entity.fields, ...object }));
}
/** Whether an item read shows an entity an unmodelled write made, or one left unknown, again rather than checks it. */
#resynced(view: StateView, record: StateRecordFacts, index: number): boolean {
if (view.shape !== "item") return false;
if (this.#writeOverlapsRead(record)) return true;
const adopted = this.#unseen.adopt(view, { record, index });
if (adopted !== undefined) {
this.#taint.known(adopted.type, adopted.collection, adopted.id);
return true;
}
const { collection, id } = itemTarget(this.#state, view.type, record);
if (!this.#taint.unknown(view.type, collection, id)) return false;
// A read begun while a modelled write to the entity still ran may show either state, so it neither checks nor shows it.
const target = entityTarget(view.type, collection, id);
if (
runningAt(
this.#touchedSince,
target,
this.#input.chronology.startMs(record),
)
)
return true;
// A failed read that is no absence shows nothing of the entity, so it stays unknown for a usable read.
if (!succeeded(record) && !isAbsence(record.status)) return true;
const entity = this.#state.entity(view.type, collection, id);
// A recorded absence of an entity the state no longer holds is still a check of its delete.
if (entity === undefined) {
if (isAbsence(record.status)) return false;
this.#taint.known(view.type, collection, id);
return true;
}
this.#resync(view, record, entity);
return true;
}
#check(
view: StateView,
record: StateRecordFacts,
comparisonBody?: (body: JsonValue) => JsonValue,
): CheckOutcome | undefined {
if (view.shape === "recorded") return undefined;
const line = this.#line(view, record);
if (view.shape !== "item" && line === undefined) return undefined;
if (view.shape === "document")
return this.#input.io.withBody(record.ordinal, (body) =>
checkDocumentRead(this.#state, view, record, body),
);
const item = view.shape === "item";
const target = item
? itemTarget(this.#state, view.type, record)
: undefined;
const collection =
target === undefined ? line!.collection : target.collection;
if (this.#taint.under(collection)) return undefined;
if (!item) {
const ids = this.#state
.members(view.type, collection)
.map((entity) => entity.id);
// A recorded-membership list covers no collection target, so a write at the collection is checked here.
if (
this.#taint.tainted(collectionTarget(view.type, collection)) ||
this.#taint.blocksList(view.type, collection, ids)
)
return undefined;
}
// The served answer keeps the envelope of the item view's or the line's base recording.
const base = item ? view.envelope_ordinal : line!.ordinal;
const ordinals = { base, read: record.ordinal };
const minted = view.per_response_members;
return withBodies(this.#input.io.withBody, ordinals, (served, body) => {
const check = {
body:
body === undefined
? body
: withoutMembers(
comparisonBody === undefined ? body : comparisonBody(body),
minted,
),
base: served === undefined ? served : withoutMembers(served, minted),
idField: this.#types.get(view.type)?.id_field ?? "",
volatile: this.#input.volatileOf(view.type),
collection,
...(target === undefined ? {} : { itemId: target.id }),
};
return this.omitted.check(this.#state, view, record, check, (loosened) =>
loosened.shape === "item"
? checkItemRead(this.#state, loosened, record, check)
: checkListRead(this.#state, loosened, record, {
...check,
line: line!,
}),
);
});
}
/** The checked line serving answers a list or document read's call from, as replay chooses it. */
#line(view: StateView, record: StateRecordFacts): ServedLine | undefined {
return servedLine(this.#checkedLines.get(view.tool), record);
}
/** The view as the gate currently knows it: without the fields reads showed omitted, with the arrays they showed sorted. */
#viewOf(view: StateView): StateView {
return this.#sorted.view(this.omitted.view(view));
}
/** The views as reads showed them. */
views(views: readonly StateView[]): StateView[] {
return this.#sorted.views(this.omitted.views(views));
}
#read(record: StateRecordFacts, index: number, tool: StateToolFacts): void {
const original = this.#input.views.get(tool.name);
if (original === undefined) return;
let view = this.#sorted.comparisonView(this.omitted.view(original));
if (projectedRead(view, record, this.#input.io)) return;
if (this.#writeOverlapsRead(record)) return;
const borrowers = this.#observations.get(record.ordinal);
if (
this.#viewOf(original).shape !== "recorded" &&
!borrowers &&
this.#resynced(view, record, index)
)
return;
let outcome = this.#check(view, record);
if (outcome === undefined) return;
let keys = [...outcome.targets, ...outcome.unheld];
const blocked = (targets: readonly string[]) =>
targets.some(
(key) =>
this.#taint.tainted(key) ||
runningAt(
this.#touchedSince,
key,
this.#input.chronology.startMs(record),
),
);
if (blocked(keys)) return;
// A read the gate counts can show the view lists a written array sorted, or rule that out.
outcome = this.#sorted.retry(
view,
{ record, outcome },
(candidate, comparisonBody) =>
this.#check(candidate, record, comparisonBody),
blocked,
);
view = this.#viewOf(original);
keys = [...outcome.targets, ...outcome.unheld];
if (blocked(keys)) return;
let exercised = exercisedBy(this.#touched, keys, borrowers);
if (view.shape === "recorded") {
// Retiring a view withdraws ordering proof, not a remaining mismatch in values.
if (!outcome.pass)
this.#contradictOthers(
exercised,
record,
interference(this.#history, outcome),
);
return;
}
if (borrowers !== undefined) {
this.omitted.keep(view, outcome);
if (!outcome.pass)
this.#contradictOthers(
exercised,
record,
interference(this.#history, outcome),
);
return;
}
if (!outcome.pass && view.shape === "list") {
const retried =
this.#retryInsert(view, record, exercised) ??
this.#retrySort(view, record, exercised);
if (retried !== undefined) {
// Reordering can replace the entities on a query page. Confirmation
// and pending-write cleanup must use the successful check's coverage.
outcome = retried;
keys = [...outcome.targets, ...outcome.unheld];
if (blocked(keys)) return;
exercised = exercisedBy(this.#touched, keys, borrowers);
}
}
// Retain omissions only after the final check's coverage passes the ownership guards.
this.omitted.keep(view, outcome);
const interferes = interference(this.#history, outcome);
const unseen = outcome.pass
? this.#confirm(view, record, outcome, exercised)
: undefined;
// Only a read whose order the state decides for every entity it lists shows them in place.
if (
outcome.pass &&
view.shape === "list" &&
outcome.positional === undefined
) {
this.#refuteSorts(view, record);
const shown = new Set(outcome.targets);
for (const target of shown) this.#moved.delete(target);
this.#pending = this.#pending.filter((write) => !shown.has(write.target));
}
if (!outcome.pass) {
const interfered = new Set([...exercised].filter(interferes));
if (interfered.size === 0)
this.#mismatch({ record, index, keys, exercised, view });
else {
// An unmodelled write after some of these writes may explain the difference: those are checked on their own members only.
this.#contradictOthers(exercised, record, interferes);
if (view.shape === "item") this.#checkMembers(view, record, interfered);
else for (const key of keys) this.#taint.fail(key);
}
if (view.shape === "item") this.#adopt(view, record);
}
const pending = {
touched: this.#touched,
since: this.#touchedSince,
effects: this.#effects,
};
for (const key of keys) {
this.#lastCheck.set(key, index);
forgetChecked(pending, key, unseen?.get(key));
}
}
/** A failed read contradicts the writes it exercised that no unmodelled write came after, as a read a write's effect was learned from still does the others. */
#contradictOthers(
exercised: ReadonlySet<string>,
record: StateRecordFacts,
interferes: (name: string) => boolean,
): void {
for (const name of exercised)
if (!interferes(name))
this.#drafts.get(name)!.contradictedBy ??= record.ordinal;
}
/** A passing read confirms the writes it exercised that it shows something of, and names, per target, those it shows nothing of. */
#confirm(
view: StateView,
record: StateRecordFacts,
outcome: CheckOutcome,
exercised: ReadonlySet<string>,
): Map<string, Set<string>> {
const shown = shownWrites(this.#touched, this.#effects, {
keys: [...outcome.targets, ...outcome.unheld],
exercised,
fields: shownFields(view, this.#input.volatileOf),
});
for (const name of shown.confirmed) {
this.#drafts.get(name)!.confirmed.push(record.ordinal);
if (showsPlacement(view, outcome, this.#inserted, name)) {
this.#placed.add(name);
// Only whole listings in collection order narrow the inserts a later list may still choose among.
if (view.shape !== "list" || !this.#narrowInserts(view, record, name))
this.#settledInserts.add(name);
}
}
return shown.unseen;
}
/** An overlapping captured write cannot supply a stable read baseline. */
#writeOverlapsRead(read: StateRecordFacts): boolean {
return this.#writeCompletion.runningAt(
read.path,
this.#input.chronology.startMs(read),
this.#input.chronology.endMs(read),
);
}
#unmodelled(path: string, index: number): void {
this.#taint.write(path);
this.#unmodelledAt.push(index);
}
/** Check the writes to an item read's entity on the members they sent, as a resynchronising read does. */
#checkMembers(
view: Extract<StateView, { shape: "item" }>,
record: StateRecordFacts,
only: ReadonlySet<string>,
): void {
const { collection, id } = itemTarget(this.#state, view.type, record);
const entity = this.#state.entity(view.type, collection, id);
if (entity === undefined) return;
const target = entityTarget(entity.type, entity.collection, entity.id);
const writes = [...(this.#touched.get(target) ?? [])];
const checked = new Map(writes.filter(([name]) => only.has(name)));
this.#checkFields(view, record, entity, checked);
}
/** Check the writes to an entity on the fields each wrote against an item read; the read's object, when it shows one. */
#checkFields(
view: Extract<StateView, { shape: "item" }>,
record: StateRecordFacts,
entity: StateEntity,
writes: ReadonlyMap<string, readonly string[] | null> = new Map(),
): JsonObject | undefined {
const object = this.#input.io.withBody(record.ordinal, (body) =>
body === undefined || body === null ? undefined : at(body, view.pointer),
);
if (!isObject(object)) return undefined;
for (const [name, pass] of writtenFieldChecks(entity, view, object, {
volatile: this.#input.volatileOf(view.type),
writes,
})) {
const draft = this.#drafts.get(name)!;
if (pass) draft.confirmed.push(record.ordinal);
else draft.contradictedBy ??= record.ordinal;
}
return object;
}
/** After a failed item check the state takes what the read recorded, so later checks start from it. */
#adopt(
view: Extract<StateView, { shape: "item" }>,
record: StateRecordFacts,
): void {
const { collection, id } = itemTarget(this.#state, view.type, record);
const entity = this.#state.entity(view.type, collection, id);
if (entity === undefined) {
this.#taint.fail(entityTarget(view.type, collection, id));
return;
}
const object = this.#input.io.withBody(record.ordinal, (body) =>
body === undefined || body === null ? undefined : at(body, view.pointer),
);
if (isObject(object) && !markedDeleted(object))
this.#state.put(updatedEntity(entity, { ...entity.fields, ...object }));
else this.#removeEntity(entity);
}
/** A read showed the entity gone: the state drops it, and no write keeps a claim on its collection through it. */
#removeEntity(entity: StateEntity): void {
this.#state.remove(entity);
this.#effects.removed(this.#touched, entity);
}
/** Contradict the writes a failed check exercised, or, when none, the related writes since its targets were last checked; a failed list stops being checked. */
#mismatch(failed: {
readonly record: StateRecordFacts;
readonly index: number;
readonly keys: readonly string[];
readonly exercised: ReadonlySet<string>;
readonly view: StateView;
}): void {
const ordinal = failed.record.ordinal;
const since = Math.min(
...failed.keys.map((key) => this.#lastCheck.get(key) ?? -1),
);
const blame = this.#blame.of({ ...failed, since });
for (const name of blame.confirmed)
this.#drafts.get(name)!.confirmed.push(ordinal);
for (const name of blame.contradicted)
this.#drafts.get(name)!.contradictedBy ??= ordinal;
if (failed.exercised.size === 0)
for (const write of this.#log)
if (
write.index > since &&
prefixRelated(write.path, failed.record.path)
)
this.#drafts.get(write.tool)!.sideEffect = true;
if (failed.view.shape !== "item")
for (const key of failed.keys) this.#taint.fail(key);
}
/** What `use` makes of a read's items when the read lists the whole collection the state holds, the only read that shows the order it is kept in. */
#wholeListing<T>(
view: Extract<StateView, { shape: "list" }>,
record: StateRecordFacts,
use: (items: readonly JsonObject[]) => T,
): T | undefined {
const line = this.#line(view, record);
const idField = this.#types.get(view.type)?.id_field;
if (line === undefined || idField === undefined) return undefined;
const held = new Set(
this.#state
.members(view.type, line.collection)
.map((entity) => entity.id),
);
return this.#input.io.withBody(record.ordinal, (body) => {
const items =
body === undefined ? undefined : listItems(body, view.pointer);
if (
items === undefined ||
items.length < 2 ||
items.length !== held.size ||
!items.every((item) =>
held.has(scalarText(ownMember(item, idField)) ?? ""),
)
)
return undefined;
return use(items);
});
}
/** A whole-collection read in collection order that placed a create leaves it only the sorted inserts whose order the read follows, adding those by a field its items all hold, and says whether it did; one its query sorts shows the query's order instead. */
#narrowInserts(
view: Extract<StateView, { shape: "list" }>,
record: StateRecordFacts,
name: string,
): boolean {
if (sortedByQuery(view, record)) return false;
const draft = this.#drafts.get(name)!;
const kept = this.#wholeListing(view, record, (items) => {
const followed = draft.candidates.filter(
(candidate) =>
candidate.rule !== "sorted" ||
same(candidate, draft.plan.insert) ||
orderedBy(items, candidate.field, candidate.direction),
);
const sorts = listedSorts(items);
const unfollowed = this.#unfollowedSorts.get(name) ?? new Set<string>();
for (const { rule, follows } of sorts)
if (!follows) unfollowed.add(JSON.stringify(rule));
this.#unfollowedSorts.set(name, unfollowed);
// A list shown in the order of the field that names its items says less than one in a field people read.
const idField = this.#types.get(view.type)?.id_field;
const learned = sorts
.map(({ rule }) => rule)
.filter(
(rule) =>
!unfollowed.has(JSON.stringify(rule)) &&
!followed.some((candidate) => same(candidate, rule)),
)
.sort(
(left, right) =>
Number(left.field === idField) - Number(right.field === idField),
);
return [...followed, ...learned];
});
if (kept === undefined) return false;
draft.candidates = kept;
return true;
}
/**
* The successful check with the collection sorted by a field an update the read exercised writes, when the read
* lists the whole collection in that order and no create of the type was placed by a sort that excludes it; the
* type's creates then insert by it, and its updates move their entity by it. A sort the creates chose by a field the
* update did not write does not stop this when every whole-collection read that placed them followed this order too.
*/
#retrySort(
view: Extract<StateView, { shape: "list" }>,
record: StateRecordFacts,
exercised: ReadonlySet<string>,
): CheckOutcome | undefined {
const line = this.#line(view, record);
// A read its query sorts shows the query's order, which may break ties otherwise than the collection.
if (line === undefined || sortedByQuery(view, record)) return undefined;
const written = new Set(
[...exercised].flatMap((name) => {
const { plan } = this.#drafts.get(name)!;
return plan.effect === "update" && plan.target === view.type
? plan.rules.map((rule) => rule.field)
: [];
}),
);
// A sort by a field the update wrote has moved its entity already; one by another field may be a guess both fit.
const current = this.#sortOf(view.type);
if (current?.rule === "sorted" && written.has(current.field))
return undefined;
const creates = [...this.#drafts.values()].filter(
(draft) =>
draft.plan.effect === "create" &&
draft.plan.target === view.type &&
draft.observed !== undefined,
);
const rules =
this.#wholeListing(view, record, (items) =>
[...written].flatMap((field) =>
(["ascending", "descending"] as const).flatMap((direction) =>
orderedBy(items, field, direction)
? [{ rule: "sorted", field, direction } as const]
: [],
),
),
) ?? [];
// Serving moves only the entities writes changed since a list last showed them, so the others, including one an
// earlier write moved that a list then showed in place, must already be in the order tried.
const kept = this.#state
.members(view.type, line.collection)
.filter(
(entity) =>
!this.#moved.has(
entityTarget(entity.type, entity.collection, entity.id),
),
)
.map((entity) => entity.fields);
const refuted = this.#refutedSorts.get(view.type);
for (const rule of rules.filter(
(candidate) =>
(current === undefined || !same(candidate, current)) &&
refuted?.has(JSON.stringify(candidate)) !== true &&
keptInOrder(kept, candidate),
)) {
// A create placed by another sort shows the collection is not kept in this order.
if (
creates.some(
({ candidates }) =>
candidates.some((candidate) => candidate.rule === "sorted") &&
!candidates.some((candidate) => same(candidate, rule)),
)
)
continue;
// Serving moves each written entity in turn, against the others' values before their own writes.
const before = this.#state.replay(
view.type,
line.collection,
rule,
this.#pending,
);
const outcome = this.#check(view, record);
if (outcome?.pass === true) {
for (const draft of creates) draft.plan.insert = rule;
this.#learnedSorts.set(view.type, rule);
return outcome;
}
this.#state.restore(view.type, line.collection, before);
}
return undefined;
}
/** Note the sorts by a field a pending write changed that this passing whole list in collection order rules out. */
#refuteSorts(
view: Extract<StateView, { shape: "list" }>,
record: StateRecordFacts,
): void {
const line = this.#line(view, record);
if (
line === undefined ||
this.#wholeListing(view, record, () => true) === undefined
)
return;
const members = new Map(
this.#state
.members(view.type, line.collection)
.map((entity) => [
entityTarget(entity.type, entity.collection, entity.id),
entity.fields,
]),
);
const writes = this.#pending.filter((write) => members.has(write.target));
const fields = new Set<string>();
for (const [index, { target, before }] of writes.entries()) {
if (before === null) continue;
const after =
writes.slice(index + 1).find((later) => later.target === target)
?.before ?? members.get(target)!;
for (const field of Object.keys(after))
if (!same(ownMember(before, field), ownMember(after, field)))
fields.add(field);
}
const order = [...members.keys()];
const refuted = this.#refutedSorts.get(view.type) ?? new Set<string>();
for (const field of fields)
for (const direction of ["ascending", "descending"] as const) {
const rule = { rule: "sorted", field, direction } as const;
const replayed = this.#state.replayedOrder(
view.type,
line.collection,
rule,
writes,
);
if (replayed !== undefined && !same(replayed, order))
refuted.add(JSON.stringify(rule));
}
if (refuted.size > 0) this.#refutedSorts.set(view.type, refuted);
}
/** The successful check under another create insertion candidate, including its possibly different query-page coverage. */
#retryInsert(
view: Extract<StateView, { shape: "list" }>,
record: StateRecordFacts,
exercised: ReadonlySet<string>,
): CheckOutcome | undefined {
const line = this.#line(view, record);
if (line === undefined || !listsAll(view, line.ordinal)) return undefined;
const members = this.#state.members(view.type, line.collection);
const key = (entity: StateEntity) =>
entityTarget(entity.type, entity.collection, entity.id);
const inserted = members.filter((entity) =>
this.#inserted.has(key(entity)),
);
const creates = new Set(
inserted.map((entity) => this.#inserted.get(key(entity))!),
);
if (creates.size !== 1) return undefined;
const [name] = creates;
const draft = this.#drafts.get(name!)!;
// An item read confirms the entity, not where a list places it; a read that placed it otherwise than a whole
// listing did settles where it goes.
if (!exercised.has(name!) || this.#settledInserts.has(name!))
return undefined;
// A create whole listings placed may still take a sort they followed: one placed last fits it before the next shows the order.
const options = this.#placed.has(name!)
? draft.candidates.filter(
(candidate) =>
candidate.rule === "sorted" && !same(candidate, draft.plan.insert),
)
: draft.candidates.slice(1);
// A sort an earlier update's whole list ruled out must not become this create's order: serving would move those updates by it.
const refuted = this.#refutedSorts.get(view.type);
for (const candidate of options) {
if (
candidate.rule === "sorted" &&
refuted?.has(JSON.stringify(candidate)) === true
)
continue;
this.#state.reinsert(view.type, line.collection, inserted, candidate);
const outcome = this.#check(view, record);
if (outcome?.pass === true) {
draft.plan.insert = candidate;
return outcome;
}
}
if (draft.plan.insert !== null)
this.#state.reinsert(
view.type,
line.collection,
inserted,
draft.plan.insert,
);
return undefined;
}
}