diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md index 8d437b23a8..9de416eb0a 100644 --- a/.agents/skills/reverse-engineer-anything/SKILL.md +++ b/.agents/skills/reverse-engineer-anything/SKILL.md @@ -2,7 +2,7 @@ name: reverse-engineer-anything description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. metadata: - version: "52" + version: "53" --- # REA diff --git a/.agents/skills/reverse-engineer-anything/references/environments.md b/.agents/skills/reverse-engineer-anything/references/environments.md index 930b88b80a..f8b3ca69be 100644 --- a/.agents/skills/reverse-engineer-anything/references/environments.md +++ b/.agents/skills/reverse-engineer-anything/references/environments.md @@ -132,8 +132,10 @@ client with it. Match `wait_for` locators to exactly one element. as a secret reference), submit, then `wait_for` an element only a signed-in user sees. 2. Read with `request` steps: each collection the environment needs, twice - before any write, so fields that change on their own show up; one item of - each. + before any write, so fields that change on their own show up; then each + object a list shows by id, up to ten per collection. An object known only + from a list that leaves out fields its own read shows cannot be read by id + or updated in the environment. 3. Write the way the application's UI does: fill its forms and click its buttons, so each request carries the fields the UI sends, including ones the server would otherwise derive. Give the click or key press that submits diff --git a/docs/agent-environments.md b/docs/agent-environments.md index ebf784cd7b..b5f24d175e 100644 --- a/docs/agent-environments.md +++ b/docs/agent-environments.md @@ -406,7 +406,8 @@ returns that record's recorded response. Selection compares the supplied path parameters against the record's path segments at the parameter indexes, then the supplied query values against the record's query string, and keeps the record with the most matches; ties keep manifest order. A query argument accepts -one string or an array of strings for repeated values; the array is matched +one string, a number or boolean sent as its JSON text, or an array of strings for +repeated values; the array is matched against every occurrence in captured order. The raw recorded query remains the replay evidence. The result reports where the answer came from: @@ -613,8 +614,9 @@ shape and carries no annotations. Path parameters are required strings whose description names the segment index, the templating rule and the observed values (sampled within the list budget above; the manifest keeps them all); -query parameter names accept an optional string or array of strings (for -repeated values in captured order); write methods accept an +query parameter names accept an optional string, number or boolean (a number or +boolean is sent as its JSON text) or array of strings (for repeated values in +captured order); write methods accept an optional `body` object described by the recorded top-level key types. Each input keeps its own argument: a query name equal to a path parameter's name is advertised as `query_`, a query name longer than 64 characters as diff --git a/docs/specs/agent-environments/write-call-values/plan.md b/docs/specs/agent-environments/write-call-values/plan.md new file mode 100644 index 0000000000..b0a4da6b4d --- /dev/null +++ b/docs/specs/agent-environments/write-call-values/plan.md @@ -0,0 +1,13 @@ +# Implementation plan + +1. **Query values.** Widen the query property schema and spell numbers and booleans as JSON text where a call enters + serving. +2. **Sequence start.** `start` on sequence rules and their schema; the exemplar records integer fields above every held + value; serving and replay floor the next value at `start`. +3. **Later rules.** Merging a later create adopts its `composed`, `template` or started `sequence` rule when it spells + the exemplar's value. +4. **Made objects.** The `made-object` rule and schema; its explanation, merge, later-call check, counts taken, serving + and own members. +5. **Bulk link.** `respondedIds` reads the object of a one-element array member; `anyLinkedSource` lets a linked + create take any path values its plan stores in fields, in replay and serving. +6. **Skill, tests and docs.** diff --git a/docs/specs/agent-environments/write-call-values/spec.md b/docs/specs/agent-environments/write-call-values/spec.md new file mode 100644 index 0000000000..7ff8f1b88d --- /dev/null +++ b/docs/specs/agent-environments/write-call-values/spec.md @@ -0,0 +1,46 @@ +# Serve the calls agents make and the values applications count + +Issue: https://github.com/morluto/rea/issues/4933 + +## Outcome + +Eval agents' trajectories on rebuilt environments, and the write plans of the recordings they ran on, showed three +gaps REA could close: + +- **Query values.** Served tools typed each query parameter as a string or an array of strings, while agents send page + sizes and flags as JSON numbers and booleans (`maxSize: 50`, `withItems: true`). The MCP server refused those calls + before REA saw them: 63 of 418 stored eval runs hit it, in EspoCRM, Homebox and PhotoPrism, and 18 never got the tool + to answer. +- **Counted and made values.** Vikunja numbers a project's tasks past those deleted earlier, so a created task's + `index` is not one past what the state holds; its `identifier` spells that index; and each created task gets a + `subscription` object the application makes with it. A second create disagreed with the first on all three, so + task creates were withheld. +- **Bulk creates.** Vikunja's UI creates a task through `POST /projects/{id}/tasks/bulk`, sending and answering the + task wrapped in `tasks: [ … ]`. The answer's id sat inside the array, so REA linked the call to no type. + +The diagnosis also found that a recorded update stores only what it sent (EspoCRM's updates never changed an email +address), and that an object known only from a list that leaves out fields its own read shows can be neither read by +id nor updated. Both are recording gaps the skill now addresses. + +## Behaviour + +- **Query values.** A query parameter accepts a number or boolean besides a string or an array of strings, and serving + sends it as its JSON text. +- **Sequences from a start.** A field the first create showed above every value its type held, which a later create + shows one past the first's, is a `sequence` with that `start`: serving never goes below it. +- **Rules that spell the first create.** A field the plan kept as shown takes a later create's rule when that rule also + spells the first create's value: a `composed` or `template` string, or the `sequence` above. +- **Made objects.** A created object no other entity of the type holds otherwise, with a member the request stamped, is + a `made-object`: stamped members take the write's clock, members holding the entity's identity take its id, counts + go one past the largest value held objects show from a `start`, and other members stay as recorded. +- **Linked bulk creates.** A create's answer names the id of the one object a one-element array member holds. A linked + create accepts any path values when it creates into its type's own collection, whose path has no parameters, and + stores each path value in a field, as a Vikunja task stores its project: no path value changes where it creates. +- **Skill.** Recordings read each listed object by id, up to ten per collection, before writing. + +Plans and views that need none of this are unchanged. + +## Acceptance + +- No write tool of the 130 recorded captures goes from served to withheld. +- Rebuilt from the same recordings, the branch passes at least the write tasks #7274 passes. diff --git a/docs/specs/agent-environments/write-call-values/tasks.md b/docs/specs/agent-environments/write-call-values/tasks.md new file mode 100644 index 0000000000..6ceed72189 --- /dev/null +++ b/docs/specs/agent-environments/write-call-values/tasks.md @@ -0,0 +1,13 @@ +# Tasks + +- [x] Accept numbers and booleans for query parameters. +- [x] Start a sequence from the value its first create took. +- [x] Adopt a later create's rule that spells the first create's value. +- [x] Learn, check and serve objects made with an entity. +- [x] Link creates whose answer wraps the entity in a one-element array, and let a linked create take path values + it stores. +- [x] Ask recordings to read each listed object by id. +- [x] Add view, serving, MCP boundary and unit tests. +- [x] Document the rules and the query argument types. +- [x] Measure the recorded captures and served write tasks. +- [x] Run static, formatting, documentation and test gates. diff --git a/docs/specs/agent-environments/write-call-values/verification.md b/docs/specs/agent-environments/write-call-values/verification.md new file mode 100644 index 0000000000..a2aad7b67f --- /dev/null +++ b/docs/specs/agent-environments/write-call-values/verification.md @@ -0,0 +1,88 @@ +# Verification + +Executed on macOS with Node v24.22.1 on this branch merged with #7274 at `45832406` (main `8706068b`), and on Ubuntu +24.04 VMs for the task runs. + +## Diagnosis + +The 418 eval runs stored on the comparison VMs, with their agent trajectories: + +- 63 runs had a call refused before REA saw it because a query value came as a JSON number or boolean (`maxSize`, + `pageSize`, `withItems`, `isLocation`, PhotoPrism's `count`, `offset`, `merged`, `public`), and in 18 the agent never + got that tool to answer. +- EspoCRM's contact update fails in every run: the served update accepts only the members recorded updates stored + (`description`, `name`, `lastName`), and a seeded contact known only from lists can be neither read by id nor + answered for. On #7274, gpt-oss passes the task by creating a replacement contact and deleting the original. +- Vikunja's task creates were withheld by an unexplained `index`, `identifier` and `subscription` (main's recording), + or had no state target, being bulk creates (four recordings). + +## Gates + +- `npm run -s check:fast`, `format:check`, `scan:todos`, `docs:check`, `verify:test-discovery`, `mcp-catalog:check` and + `jscpd`: passed. `npx knip` reports `staticFileUrlPath` and `ObjcClassImportBindingCoverage`, which main `8706068b` + added and this branch does not touch. +- `npx vitest run --project domain --project services`: 11,430 passed, 6 skipped. `--project adapters`: 7,437 passed, 19 + skipped, 3 failed: the Inspector cancellation test and two AppKit asset-catalog tests, which fail the same way on + #7274 at `e29e9c74`. +- `npm run -s test:focused --` the environment build, export and serve CLI boundaries, the analysis-view CLI and MCP + boundaries and the environment serve MCP boundary: passed. `outputSchemaProfile`, `toolSchemaValidity`, + `mcpSchemaValidation` and `outputSchemaSemanticConstraints`: passed; the compact catalog stays under its frame. + +## Tests + +- View: two task creates at a project path learn `index` as a `sequence` from 15, `identifier` as a `template` around + it and `subscription` as a `made-object` whose `id` counts from 21 and whose `entity_id` holds the task's own id; one + create alone keeps `index` as recorded; a bulk create whose body and answer wrap the task in `tasks` links to the + tasks type and is served; an object with no member the request stamped makes no rule. +- Serving: created tasks take index 15 then 16, `#15` then `#16`, and a subscription numbered 22 holding the second + task's id; a bulk create answers in its envelope, and one into another project stores that project. +- Query values: a number or boolean query value is spelled as its JSON text, and a list paged by a number answers as + with the same text; through the MCP server, `maxSize: 2` and `withItems: true` replay their recording exactly. The + served card of the `renamed` fixture grows 637 bytes, 91 for each of its seven query parameters. +- The seven behaviour tests fail on #7274 at `4ede1305`. + +## Write plans and views over recorded captures + +The 130 captures #7274 measured, built by #7274 at `45832406` and by this branch: 615 plans, 332 served and 333. + +- No write goes from served to withheld. Newly served: Vikunja's bulk task create in #7260's final recording. +- Vikunja's task create in main's recording is now explained, by its counter, template and made subscription, but + main's `unknown-clock` withholds it in both builds, as that capture has no clock anchor. Measured on #7274 at + `4ede1305`, before that rule, it was served too (339 to 341). +- Six plans that stay withheld explain more: Homebox entity creates learn `assetId` as a counter from 5 in four + recordings, and Vikunja task creates in two. +- In #7260's two Vikunja recordings whose UI writes lost their bodies, the bulk create now counts as a task write, so + the scratch task it made is no longer held as seeded (35 to 34 and 36 to 35 initial entities), a read tool that read + only that task replays its recordings, and later reads check fewer lines. Every other view is unchanged. + +## The same recordings, built by #7274 and by this branch + +Claude Code (`claude-sonnet-5-5`) recordings from earlier rounds, rebuilt by #7274 at `e839af3f` (its last two merges +change no census) and by this branch, their write tasks run with `fireworks_ai/gpt-oss-120b` then +`fireworks_ai/glm-5p3-flash`, each checked from the environment's state: + +| Recording | Write tasks | #7274 | This branch | +| ---------------------------------------------------------- | ----------- | ----------- | ----------- | +| Vikunja: #7260 final (bulk creates) | 3 + 2 | 1/5 1/5 | 2/5 2/5 | +| Vikunja: main (two), saved-values (two), #7260 (first two) | 6 × (3 + 2) | 6/30 6/30 | 6/30 6/30 | +| EspoCRM: main, #7260 (three), saved-values | 5 × 3 | 15/15 10/15 | 15/15 10/15 | +| Homebox: main, #7260 final, saved-values (two) | 4 × 2 | 6/8 6/8 | 6/8 6/8 | +| Total | 58 | 28/58 23/58 | 29/58 24/58 | + +No recording passes fewer tasks with this branch. + +- **Vikunja, bulk creates:** the create task passes with both models. The served bulk create numbers the task and makes + its subscription, and creates into Lighthouse Rota though the recording created into the inbox. +- **Refused calls:** agents' calls were refused at input validation 34 times with #7274 (30 in Vikunja) and once with + this branch, and agents made 286 tool calls against 317. +- **Main's `unknown-clock`** withholds Vikunja's task create and Homebox's tag create in main's recordings in both + builds, so those tasks fail in both. Rebuilt by #7274 at `4ede1305`, before that rule, this branch without the + linked-path change made main's Vikunja create task pass with both models (1/3 to 2/3) and changed no other result + among the recordings that finished. + +## Not verified + +- A fresh recording with skill 52, which reads each listed object by id. +- Path parameters given as numbers stay refused: they are required properties, which REA keeps free of combinators. +- A counter one create alone shows, and linked creates into a collection whose path has parameters, which still take + only their recorded path values. diff --git a/docs/web-network-captures.md b/docs/web-network-captures.md index 1eeb5a62c5..b0cf4c396b 100644 --- a/docs/web-network-captures.md +++ b/docs/web-network-captures.md @@ -733,8 +733,12 @@ order. Its `effect` is `create` for a `POST` to a type's plus one trailing parameter, or to that item path plus one action segment, `delete` for a `DELETE` to it, `document` for a `PUT`, `PATCH` or `POST` to a document's path, an `update` or `delete` of each entity a write at the -collection names by id in its body, and otherwise `none` with reason -`no-state-target`. A plan is built from the tool's first successful call: its +collection names by id in its body, a `create` for a `POST` whose answer names +an id (at its top level, in an object member or in the one object of a +one-element array member) that a later item read of a type first shows (served +only with the path values its recorded call sent, unless it creates into the +type's own collection, whose path has no parameters, and a `path` rule stores +each of its path values), and otherwise `none` with reason `no-state-target`. A plan is built from the tool's first successful call: its request body must be a usable JSON object (as response bodies are usable), and the written entity must show in that call's response or in a later read before the next write. Each field of a created entity gets the first rule that @@ -743,10 +747,14 @@ value's shape), `echo` of a body member (`sorted` when the application stored its strings in code-point order rather than as sent), `copy` of a body string of 8 or more characters, a `path` value, a `slug` of a body member, a `clock` value within 10 minutes of the request in its recorded format, a `sequence` one past the -largest value of its collection or type, a `unique` UUID, opaque token or +largest value of its collection or type, and never below its `start`, a `unique` UUID, opaque token or integer of 10 or more digits that no entity of the type holds (where every one holds a distinct integer of the same length, as time-based ids do), a -`nested-clock` object whose clocks the request stamped, a `template` around +`nested-clock` object whose clocks the request stamped, a `made-object` no +other entity of the type holds otherwise, whose `clocks` the request stamped and +whose other members hold the entity's own identity (`own`), one past the largest +value held objects show (`counts`, each from a `start`), or what every held +object shows, a `template` around a generated value, a `reference` to an entity, or a `composed` string; other fields keep the observed value. When the first read of a created entity before the next write shows fields the create's answer lacks, its clocks to another @@ -781,7 +789,13 @@ when, explained on its own, it agrees with it: both calls show the same fields but for a reference whose member a call without it did not send (one an earlier create added stays), every field both rule they rule alike (an echo one showed sorted stays sorted, unless a call stored its strings out of order as sent), -and a field neither rules holds one value. Members either call sent are accepted, only those every call sent are +and a field neither rules holds one value, unless the later call's rule for it +spells the earlier calls' value too: a `composed` or `template` string, or a +`sequence` started at that value (`start`), as Vikunja numbers a project's tasks +past those deleted before the capture. Two `made-object` rules agree in their +clocks and own members, and a count only the later call shows starts at the +first object's value; a later create's made object must hold clocks its own +request stamped, in the rule's format. Members either call sent are accepted, only those every call sent are required, and an echoed member a call left out takes the value that call stored (`body.defaults`). A member a create sent empty (`""`, `null`, `false`, `[]` or `{}`), as a form sends a field left as it was, is not required: a call that diff --git a/src/application/environment/environmentQueryText.test.ts b/src/application/environment/environmentQueryText.test.ts new file mode 100644 index 0000000000..1e52df3078 --- /dev/null +++ b/src/application/environment/environmentQueryText.test.ts @@ -0,0 +1,40 @@ +import { describe, expect, it } from "vitest"; + +import { MAX_SIZE_ROWS } from "../../../tests/fixtures/environmentStateLaterReads.js"; +import { servedFrom } from "../../../tests/fixtures/environmentStateServing.js"; +import { withQueryText } from "./environmentTools.js"; + +const contacts = () => servedFrom(MAX_SIZE_ROWS); + +const listTool = (served: ReturnType) => { + const tool = served.source.manifest.tools.find( + (candidate) => candidate.name === "get_contacts", + ); + if (tool === undefined) throw new Error("The manifest lacks get_contacts"); + return tool; +}; + +describe("query values given as numbers or booleans", () => { + it("spells a number or boolean query value as its JSON text and leaves others as given", () => { + const tool = listTool(contacts()); + const args = { + maxSize: 2, + offset: false, + body: { maxSize: 3 }, + }; + expect(withQueryText(tool, args)).toEqual({ + maxSize: "2", + offset: "false", + body: { maxSize: 3 }, + }); + const strings = { maxSize: "2", offset: ["0"] }; + expect(withQueryText(tool, strings)).toBe(strings); + }); + + it("answers a list paged by a number as it answers the same page size given as text", () => { + const served = contacts(); + expect(served.answer("get_contacts", { maxSize: 2, offset: 0 })).toEqual( + served.answer("get_contacts", { maxSize: "2", offset: "0" }), + ); + }); +}); diff --git a/src/application/environment/environmentStateLaterReadsAnswer.test.ts b/src/application/environment/environmentStateLaterReadsAnswer.test.ts index f1c12be2c3..78fa050912 100644 --- a/src/application/environment/environmentStateLaterReadsAnswer.test.ts +++ b/src/application/environment/environmentStateLaterReadsAnswer.test.ts @@ -1,6 +1,7 @@ import { describe, expect, it } from "vitest"; import { + INITIAL_TRASH_ROWS, LATER_BULK_ROWS, LATER_FIELD_ROWS, OMITTED_FLAG_ROWS, @@ -26,6 +27,12 @@ const names = (body: JsonValue | undefined): JsonValue[] => { }; describe("served writes modelled on later reads", () => { + it("replays the recorded list when it includes a deleted row", () => { + const contacts = servedFrom(INITIAL_TRASH_ROWS); + expect( + contacts.answer("get_contacts", { maxSize: "20", offset: "0" }).body, + ).toEqual(INITIAL_TRASH_ROWS[0]![2]); + }); it("keeps an initially deleted item readable without treating it as live", () => { const deleted = { id: 4, name: "Gone", deleted: true }; const rows = [ diff --git a/src/application/environment/environmentStateMadeValuesAnswer.test.ts b/src/application/environment/environmentStateMadeValuesAnswer.test.ts new file mode 100644 index 0000000000..f2dd88ce6b --- /dev/null +++ b/src/application/environment/environmentStateMadeValuesAnswer.test.ts @@ -0,0 +1,77 @@ +import { describe, expect, it } from "vitest"; + +import { + BULK_TASK_ROWS, + CHOSEN_ID_TASK_ROWS, + COUNTED_TASK_ROWS, +} from "../../../tests/fixtures/environmentStateMadeValues.js"; +import { servedFrom } from "../../../tests/fixtures/environmentStateServing.js"; +import { isObject } from "../../domain/analysisView/environmentStateRules.js"; +import type { JsonValue } from "../../domain/jsonValue.js"; + +const objectOf = (body: JsonValue | undefined) => { + if (!isObject(body)) throw new Error("Expected an object body"); + return body; +}; + +describe("served creates that count and make values", () => { + it("numbers created tasks on from the recorded counter and makes each one's subscription", () => { + const tasks = servedFrom(COUNTED_TASK_ROWS); + const first = objectOf( + tasks.answer("post_projects_tasks_by_id", { + id: "5", + body: { title: "Restock the fog-horn spares" }, + }).body, + ); + const second = objectOf( + tasks.answer("post_projects_tasks_by_id", { + id: "5", + body: { title: "Paint the gallery rail" }, + }).body, + ); + expect([first["index"], first["identifier"]]).toEqual([15, "#15"]); + expect([second["index"], second["identifier"]]).toEqual([16, "#16"]); + const read = objectOf( + tasks.answer("get_tasks_by_id", { id: String(second["id"]) }).body, + ); + expect(read["subscription"]).toMatchObject({ + id: 22, + entity: "task", + entity_id: second["id"], + }); + }); + + it("links a made object to the id the caller chose", () => { + const tasks = servedFrom(CHOSEN_ID_TASK_ROWS); + tasks.answer("post_projects_tasks_by_id", { + id: "5", + body: { id: 99, title: "Restock the fog-horn spares" }, + }); + const read = objectOf(tasks.answer("get_tasks_by_id", { id: "99" }).body); + expect(read["subscription"]).toMatchObject({ entity_id: 99 }); + }); + + it("answers a bulk create in the envelope it was sent in", () => { + const tasks = servedFrom(BULK_TASK_ROWS); + const body = objectOf( + tasks.answer("post_projects_tasks_bulk_by_id", { + id: "5", + body: { tasks: [{ title: "Restock the fog-horn spares" }] }, + }).body, + ); + expect(body["tasks"]).toMatchObject([ + { title: "Restock the fog-horn spares", index: 15, identifier: "#15" }, + ]); + }); + + it("creates into another project when the task stores its project and lands in one collection", () => { + const tasks = servedFrom(BULK_TASK_ROWS); + const body = objectOf( + tasks.answer("post_projects_tasks_bulk_by_id", { + id: "7", + body: { tasks: [{ title: "Paint the gallery rail" }] }, + }).body, + ); + expect(body["tasks"]).toMatchObject([{ project_id: 7 }]); + }); +}); diff --git a/src/application/environment/environmentStateWriteAnswer.ts b/src/application/environment/environmentStateWriteAnswer.ts index 0be2912e81..fb997d84f0 100644 --- a/src/application/environment/environmentStateWriteAnswer.ts +++ b/src/application/environment/environmentStateWriteAnswer.ts @@ -18,10 +18,14 @@ import { } from "../../domain/analysisView/environmentStateWriteExplain.js"; import { callRenewal } from "../../domain/analysisView/environmentStateWriteGenerated.js"; import { + heldObjects, + madeObjectValue, nestedClockValue, + nextMadeCount, unenveloped, } from "../../domain/analysisView/environmentStateWriteStamps.js"; import { + anyLinkedSource, linkedCollection, sameLinkedSource, } from "../../domain/analysisView/environmentStateWriteLinks.js"; @@ -38,6 +42,7 @@ import { seededInteger, seededLike, spelledLike, + startedSequence, } from "../../domain/analysisView/environmentStateWriteValues.js"; import { at, @@ -287,16 +292,41 @@ export const generator = ( exemplar, sequence( `${request.tool.name} ${rule.field} ${rule.scope === "collection" ? target.name : ""}`, - nextHeld( - rule.scope === "collection" - ? [...target.collection, ...startedIn(target.name)] - : [...ofType, ...started], - rule.field, - exemplar, + startedSequence( + nextHeld( + rule.scope === "collection" + ? [...target.collection, ...startedIn(target.name)] + : [...ofType, ...started], + rule.field, + exemplar, + ), + rule.start, ), ), ), ); + case "made-object": { + const clock = formatClock(write.clockMs, rule.format); + if (clock === undefined) + return err( + `The generated clock for ${rule.field} is outside its recorded four-digit year format.`, + ); + const base = ownMember(target.observed, rule.field); + const held = heldObjects( + [...ofType, ...started].map((entity) => entity.fields), + rule.field, + ); + return madeObjectValue(base, rule, clock, (count) => + sequence( + `${request.tool.name} ${rule.field}/${count.member}`, + nextMadeCount( + held, + count, + isObject(base) ? ownMember(base, count.member) : undefined, + ), + ), + ); + } case "unique": return unused(rule.field, (attempt) => generatedIntegerDigits(exemplar) === undefined @@ -438,12 +468,13 @@ const writtenCollection = ( return ok( createdCollection(serving, tool, request.args, plan.target ?? ""), ); - // No read showed where other path arguments create. - return sameLinkedSource( - tool.path_template, - exemplarPath(serving, plan), - pathArguments(tool, request.args), - ) + // No read showed where other path arguments create, unless they cannot change it. + return anyLinkedSource(plan, tool.path_template) || + sameLinkedSource( + tool.path_template, + exemplarPath(serving, plan), + pathArguments(tool, request.args), + ) ? ok(linked) : err( refused( @@ -455,7 +486,7 @@ const writtenCollection = ( /** * The path values a create's path fields take, decoded once as the gate learned them: a linked - * create accepts its exemplar's source only, so it takes that; another the collection this call + * create that accepts its exemplar's source only takes that; another the collection this call * selects, else its own arguments. */ const createdPathValues = ( @@ -466,7 +497,8 @@ const createdPathValues = ( ): ReadonlyMap => { const { tool, plan } = request; const exemplar = - linkedCollection(plan, tool.path_template) === undefined + linkedCollection(plan, tool.path_template) === undefined || + anyLinkedSource(plan, tool.path_template) ? undefined : exemplarPath(serving, plan); return ( @@ -521,6 +553,7 @@ const answerCreate = ( created, values.forTarget({ name: collection, collection: members, observed }), heldEntity(serving), + idField, ); if (!fields.ok) return err(refused(tool, fields.error)); const field = unloadableMember(fields.value); diff --git a/src/application/environment/environmentToolAnswer.ts b/src/application/environment/environmentToolAnswer.ts index 241f00f248..db1b35db3a 100644 --- a/src/application/environment/environmentToolAnswer.ts +++ b/src/application/environment/environmentToolAnswer.ts @@ -15,6 +15,7 @@ import { answerBulk } from "./environmentStateBulkAnswer.js"; import type { EnvironmentStateStore } from "./environmentStateStore.js"; import { answerWrite } from "./environmentStateWriteAnswer.js"; import { pinnedRequest } from "./environmentStateWriteQuery.js"; +import { withQueryText } from "./environmentTools.js"; /** How a served environment answers one tool call. */ export type EnvironmentToolAnswer = ( @@ -30,8 +31,14 @@ export const environmentToolAnswer = ( ): EnvironmentToolAnswer => store === undefined ? (tool, args, observedQuery) => - replayEnvironmentTool(source, tool, args, observedQuery) - : (tool, args, observedQuery) => { + replayEnvironmentTool( + source, + tool, + withQueryText(tool, args), + observedQuery, + ) + : (tool, given, observedQuery) => { + const args = withQueryText(tool, given); const plan = store.state.writes.get(tool.name); if (plan?.served !== true) return answerFromState({ source, store }, tool, args, observedQuery); diff --git a/src/application/environment/environmentTools.test.ts b/src/application/environment/environmentTools.test.ts index 85ae00b887..25dd8d4572 100644 --- a/src/application/environment/environmentTools.test.ts +++ b/src/application/environment/environmentTools.test.ts @@ -92,9 +92,14 @@ describe("environmentToolDefinitions", () => { expect(search.additionalProperties).toBe(false); expect(Object.keys(search.properties).sort()).toEqual(["query", "type"]); expect(search.properties.query).toEqual({ - oneOf: [{ type: "string" }, { type: "array", items: { type: "string" } }], + oneOf: [ + { type: "string" }, + { type: "number" }, + { type: "boolean" }, + { type: "array", items: { type: "string" } }, + ], description: - "Query parameter query; optional. Supply one string for one occurrence or an array to preserve repeated values in captured order; repeated values replay their recorded request.", + "Query parameter query; optional. Supply one value for one occurrence, a number or boolean sent as its JSON text, or an array of strings to preserve repeated values in captured order; repeated values replay their recorded request.", }); expect("body" in search.properties).toBe(false); }); @@ -522,10 +527,12 @@ describe("environmentToolInputSchema argument names", () => { ]); expect(Object.hasOwn(schema.properties, "constructor")).toBe(true); expect(schema.properties.query_id?.description).toBe( - "Query parameter id; optional. Supply one string for one occurrence or an array to preserve repeated values in captured order; repeated values replay their recorded request.", + "Query parameter id; optional. Supply one value for one occurrence, a number or boolean sent as its JSON text, or an array of strings to preserve repeated values in captured order; repeated values replay their recorded request.", ); expect(schema.properties.body?.oneOf).toEqual([ { type: "string" }, + { type: "number" }, + { type: "boolean" }, { type: "array", items: { type: "string" } }, ]); expect(schema.properties.request_body?.type).toBe("object"); diff --git a/src/application/environment/environmentTools.ts b/src/application/environment/environmentTools.ts index f334d9b503..62df5fda92 100644 --- a/src/application/environment/environmentTools.ts +++ b/src/application/environment/environmentTools.ts @@ -32,7 +32,7 @@ export const REPLAY_CATALOG_INSTRUCTIONS = /** JSON Schema advertised for one tool argument. */ export interface EnvironmentToolProperty extends EnvironmentToolSchema { readonly oneOf?: readonly ( - | { readonly type: "string" } + | { readonly type: "string" | "number" | "boolean" } | { readonly type: "array"; readonly items: { readonly type: "string" } } )[]; readonly description: string; @@ -325,6 +325,21 @@ export const hasRepeatedQueryArgument = ( return false; }; +/** The arguments with a number or boolean given for a query parameter spelled as its JSON text, as a URL carries it. */ +export const withQueryText = ( + tool: EnvironmentManifestTool, + args: Readonly>, +): Readonly> => { + let spelled: Record | undefined; + for (const argument of environmentToolArguments(tool).query.values()) { + const value = ownMember(args, argument); + if (typeof value !== "number" && typeof value !== "boolean") continue; + spelled ??= { ...args }; + spelled[argument] = JSON.stringify(value); + } + return spelled ?? args; +}; + /** Lazily render properties so admission precedes retaining a whole schema. */ function* environmentToolProperties( tool: EnvironmentManifestTool, @@ -351,9 +366,11 @@ function* environmentToolProperties( ? { oneOf: [ { type: "string" }, + { type: "number" }, + { type: "boolean" }, { type: "array", items: { type: "string" } }, ], - description: `Query parameter ${describeText(name, "query name")}; optional. Supply one string for one occurrence or an array to preserve repeated values in captured order; repeated values replay their recorded request.`, + description: `Query parameter ${describeText(name, "query name")}; optional. Supply one value for one occurrence, a number or boolean sent as its JSON text, or an array of strings to preserve repeated values in captured order; repeated values replay their recorded request.`, } : { type: "string", diff --git a/src/domain/analysisView/EnvironmentStateExtraction.ts b/src/domain/analysisView/EnvironmentStateExtraction.ts index 4a31b44ee7..6a108cd5c0 100644 --- a/src/domain/analysisView/EnvironmentStateExtraction.ts +++ b/src/domain/analysisView/EnvironmentStateExtraction.ts @@ -20,6 +20,7 @@ import { lastSegmentValue, STATE_BODY_LIMIT_BYTES, listItems, + markedDeleted, locateItem, locateList, projectsFields, @@ -862,7 +863,9 @@ export class EnvironmentStateExtraction { if (this.#collections === undefined) { this.#retain(ENTRY_BYTES * (this.#merged.size + 1)); this.#collections = new Map(); + // List serving selects live entities; item observations retain deleted entities separately. for (const entity of this.#merged.values()) { + if (markedDeleted(entity.fields)) continue; const key = JSON.stringify([entity.type, entity.collection]); const members = this.#collections.get(key); if (members === undefined) this.#collections.set(key, [entity]); diff --git a/src/domain/analysisView/EnvironmentStateWriteSimulation.ts b/src/domain/analysisView/EnvironmentStateWriteSimulation.ts index 2a16eb22b1..597e4d0ca3 100644 --- a/src/domain/analysisView/EnvironmentStateWriteSimulation.ts +++ b/src/domain/analysisView/EnvironmentStateWriteSimulation.ts @@ -295,6 +295,8 @@ export class EnvironmentStateWriteSimulation { calls: 0, updates: [], sequences: new Map(), + starts: new Map(), + createCollections: [], steady: new Set(), unstamped: new Set(), kept: new Set(), diff --git a/src/domain/analysisView/environmentStateCreateCalls.test.ts b/src/domain/analysisView/environmentStateCreateCalls.test.ts index 198d8bc78a..26aa4c3f3e 100644 --- a/src/domain/analysisView/environmentStateCreateCalls.test.ts +++ b/src/domain/analysisView/environmentStateCreateCalls.test.ts @@ -215,3 +215,52 @@ describe("merging a later create into a plan", () => { ).toBeUndefined(); }); }); + +describe("merging a later create's made object", () => { + const made = (counts: { member: string; start: number }[]): FieldRule => ({ + field: "subscription", + rule: "made-object", + clocks: ["created"], + format: { kind: "iso", separator: "T", fraction: 0, zone: "Z" }, + own: ["entity_id"], + counts, + }); + const body = { + accepted: ["title"], + fixed: [], + required: ["title"], + identity: null, + kinds: [], + }; + const title: FieldRule = { field: "title", rule: "echo" }; + const subscription = (id: number, task: number) => ({ + id, + entity: "task", + entity_id: task, + created: "2026-10-26T10:00:00Z", + }); + const plan = { rules: [title, made([])], body }; + const exemplar = { title: "First", subscription: subscription(31, 20) }; + const later = { + entity: { title: "Third", subscription: subscription(32, 22) }, + body: { title: "Third" }, + rules: [title, made([{ member: "id", start: 32 }])], + contract: body, + }; + const earlier = (accepted: number) => ({ + kept: new Set(), + clockOf: () => undefined, + createCollections: Array.from({ length: accepted }, () => "/api/tasks"), + }); + + it("counts a member only the later create shows while the exemplar is the one create accepted", () => { + expect( + mergedCreatePlan(plan, exemplar, later, earlier(1))?.rules, + ).toContainEqual(made([{ member: "id", start: 31 }])); + }); + + it("makes no count of a member creates past the exemplar held alike", () => { + // Creates of 31 and 31 then 32 are no count: serving would make 31, 32, 33. + expect(mergedCreatePlan(plan, exemplar, later, earlier(2))).toBeUndefined(); + }); +}); diff --git a/src/domain/analysisView/environmentStateGate.ts b/src/domain/analysisView/environmentStateGate.ts index fb059287d5..4e52a7c01a 100644 --- a/src/domain/analysisView/environmentStateGate.ts +++ b/src/domain/analysisView/environmentStateGate.ts @@ -8,6 +8,7 @@ import { isObject, lastSegmentValue, listItems, + markedDeleted, renderStateEntity, renderStateList, same, @@ -359,7 +360,9 @@ const listReproduces = ( collection, scalarText(ownMember(item, field)) ?? "", ); - return entity === undefined ? [] : [entity]; + return entity === undefined || markedDeleted(entity.fields) + ? [] + : [entity]; })); const rendered = members.map((entity) => { const item = observed.get(entity.id); diff --git a/src/domain/analysisView/environmentStateLaterReads.test.ts b/src/domain/analysisView/environmentStateLaterReads.test.ts index 1771e4abe8..c7a4ff3d74 100644 --- a/src/domain/analysisView/environmentStateLaterReads.test.ts +++ b/src/domain/analysisView/environmentStateLaterReads.test.ts @@ -6,6 +6,7 @@ import { } from "../../../tests/fixtures/analysisView.js"; import { GHOST_DELETED_ROWS, + INITIAL_TRASH_ROWS, LATER_FIELD_ROWS, LATER_MIXED_BULK_ROWS, MAX_SIZE_ROWS, @@ -25,6 +26,13 @@ const planOf = (rows: readonly TimedCaptureRow[], tool: string) => { }; describe("deletes the application answers for afterwards", () => { + it("replays an initial list that includes a deleted entity", () => { + expect( + viewOf(INITIAL_TRASH_ROWS).views.find( + (view) => view.tool === "get_contacts", + ), + ).toMatchObject({ shape: "recorded", reason: "list-differs" }); + }); it("confirms a delete by an item read that shows the entity marked deleted", () => { expect(planOf(SOFT_DELETED_ROWS, "delete_contacts_by_id")).toMatchObject({ served: true, diff --git a/src/domain/analysisView/environmentStateMadeValues.test.ts b/src/domain/analysisView/environmentStateMadeValues.test.ts new file mode 100644 index 0000000000..66375613c8 --- /dev/null +++ b/src/domain/analysisView/environmentStateMadeValues.test.ts @@ -0,0 +1,143 @@ +import { describe, expect, it } from "vitest"; + +import { + timedCaptureEvidence, + type TimedCaptureRow, +} from "../../../tests/fixtures/analysisView.js"; +import { + BULK_TASK_ROWS, + COUNTED_TASK_ROWS, + MISLINKED_TASK_ROWS, + ONE_TASK_ROWS, + RESPELLED_TASK_ROWS, + STALE_MADE_TASK_ROWS, + UNCLOCKED_TASK_ROWS, +} from "../../../tests/fixtures/environmentStateMadeValues.js"; +import { environmentStateViewOf } from "../../../tests/fixtures/environmentStateWrites.js"; +import { respondedIds } from "./environmentStateWriteLinks.js"; +import { createdMadeObject } from "./environmentStateWriteStamps.js"; + +const createOf = (rows: readonly TimedCaptureRow[]) => { + const plan = environmentStateViewOf(timedCaptureEvidence(rows)).writes.find( + (candidate) => candidate.effect === "create", + ); + if (plan === undefined) throw new Error("No create plan"); + return plan; +}; + +const ruleFor = (plan: ReturnType, field: string): unknown => + plan.rules.find((rule) => rule.field === field); + +describe("values an application counts and makes with each created entity", () => { + it("does not adopt a sequence that contradicts an earlier accepted create", () => { + const a = { id: 1, title: "A", index: 1 }; + const b = { id: 2, title: "B", index: 2 }; + const c = { id: 3, title: "C", index: 15 }; + const d = { id: 4, title: "D", index: 15 }; + const e = { id: 5, title: "E", index: 16 }; + const rows: readonly TimedCaptureRow[] = [ + ["GET", "/api/tasks", [a, b]], + ["GET", "/api/tasks/1", a], + ["POST", "/api/tasks", c, 201, { title: "C" }], + ["GET", "/api/tasks/3", c], + ["GET", "/api/tasks", [a, b, c]], + ["POST", "/api/tasks", d, 201, { title: "D" }], + ["GET", "/api/tasks/4", d], + ["GET", "/api/tasks", [a, b, c, d]], + ["POST", "/api/tasks", e, 201, { title: "E" }], + ["GET", "/api/tasks/5", e], + ["GET", "/api/tasks", [a, b, c, d, e]], + ]; + const plan = createOf(rows); + expect(plan).toMatchObject({ served: false }); + expect(ruleFor(plan, "index")).toBeUndefined(); + }); + it("learns a counter from where the first create stood, a template around it and the object made with each task", () => { + const plan = createOf(COUNTED_TASK_ROWS); + expect(plan).toMatchObject({ + served: true, + checked_record_ordinals: [4, 7], + }); + expect(ruleFor(plan, "index")).toEqual({ + field: "index", + rule: "sequence", + scope: "collection", + start: 15, + }); + expect(ruleFor(plan, "identifier")).toEqual({ + field: "identifier", + rule: "template", + from: "index", + }); + expect(ruleFor(plan, "subscription")).toMatchObject({ + rule: "made-object", + clocks: ["created"], + own: ["entity_id"], + counts: [{ member: "id", start: 21 }], + }); + }); + + it("keeps a value one create alone shows past what the type held", () => { + const plan = createOf(ONE_TASK_ROWS); + expect(ruleFor(plan, "index")).toBeUndefined(); + expect(ruleFor(plan, "subscription")).toMatchObject({ + rule: "made-object", + counts: [], + }); + }); + + it("links a create whose answer wraps the entity in a one-element array member", () => { + expect(createOf(BULK_TASK_ROWS)).toMatchObject({ + tool: "post_projects_tasks_bulk_by_id", + served: true, + body: { envelope: "tasks" }, + }); + expect(respondedIds({ tasks: [{ id: 14 }] })).toEqual(["14"]); + expect(respondedIds({ tasks: [{ id: 14 }, { id: 15 }] })).toEqual([]); + }); + + it("withholds a create whose only clock is its made object's when the capture has no clock anchor", () => { + expect(createOf(UNCLOCKED_TASK_ROWS)).toMatchObject({ served: true }); + expect( + createOf([ + ...UNCLOCKED_TASK_ROWS, + ["GET", "/api/health", { ready: true }, 200, undefined, -1], + ]), + ).toMatchObject({ served: false, reason: "unknown-clock" }); + }); + + it("withholds the creates when a later made object names another entity", () => { + expect(createOf(MISLINKED_TASK_ROWS)).toMatchObject({ + served: false, + reason: "unexplained-change", + }); + }); + + it("adopts no template that would not spell the later create's value", () => { + const plan = createOf(RESPELLED_TASK_ROWS); + expect(plan).toMatchObject({ served: false, reason: "unexplained-change" }); + expect(ruleFor(plan, "identifier")).toBeUndefined(); + }); + + it("withholds the creates when a later made object holds a clock its request did not stamp", () => { + expect(createOf(STALE_MADE_TASK_ROWS)).toMatchObject({ + served: false, + reason: "unexplained-change", + }); + }); + + it("makes no object rule without a clock the request stamped", () => { + expect( + createdMadeObject( + "subscription", + { id: 21, entity: "task", entity_id: 20 }, + { + startMs: Date.parse("2026-10-17T18:00:02Z"), + type: [], + entity: { id: 20 }, + idField: "id", + }, + ), + ).toBeUndefined(); + }); +}); diff --git a/src/domain/analysisView/environmentStateView.ts b/src/domain/analysisView/environmentStateView.ts index a1b7d232b2..aa98efe792 100644 --- a/src/domain/analysisView/environmentStateView.ts +++ b/src/domain/analysisView/environmentStateView.ts @@ -245,6 +245,7 @@ const fieldRuleSchema = z.discriminatedUnion("rule", [ ...ruled, rule: z.literal("sequence"), scope: z.enum(["collection", "type"]), + start: z.number().int().exactOptional(), }), z.strictObject({ ...ruled, rule: z.literal("unique") }), z.strictObject({ @@ -263,6 +264,16 @@ const fieldRuleSchema = z.discriminatedUnion("rule", [ members: z.array(z.string()).min(1), format: clockFormatSchema, }), + z.strictObject({ + ...ruled, + rule: z.literal("made-object"), + clocks: z.array(z.string()).min(1), + format: clockFormatSchema, + own: z.array(z.string()), + counts: z.array( + z.strictObject({ member: z.string(), start: z.number().int() }), + ), + }), z.strictObject({ ...ruled, rule: z.literal("held-clock"), diff --git a/src/domain/analysisView/environmentStateWriteApply.test.ts b/src/domain/analysisView/environmentStateWriteApply.test.ts index d8d65d7149..a4fc8133f1 100644 --- a/src/domain/analysisView/environmentStateWriteApply.test.ts +++ b/src/domain/analysisView/environmentStateWriteApply.test.ts @@ -51,6 +51,7 @@ const generator = () => { clock: NOW, "nested-clock": { at: NOW }, "held-clock": NOW, + "made-object": { at: NOW }, }; const generate = (rule: GeneratedRule) => { asked.push(rule); diff --git a/src/domain/analysisView/environmentStateWriteApply.ts b/src/domain/analysisView/environmentStateWriteApply.ts index 399d84f1b6..05a93adefe 100644 --- a/src/domain/analysisView/environmentStateWriteApply.ts +++ b/src/domain/analysisView/environmentStateWriteApply.ts @@ -236,6 +236,7 @@ const ruledValue = ( case "clock": case "nested-clock": case "held-clock": + case "made-object": return generate(rule); case "echo": return ok( @@ -356,6 +357,7 @@ export const createdFields = ( sentCall: WriteCall, generate: GenerateValue, referenced?: ReferencedEntity, + idField?: string, ): Applied => { if (plan.body === null) return err("The tool has no modelled body."); const call = withBlanks(plan.body, sentCall); @@ -377,10 +379,38 @@ export const createdFields = ( else if (rule.rule === "echo" || rule.rule === "reference") delete written[rule.field]; } + ownMadeMembers(plan.rules, written, idField); const unspelled = derive(plan.rules, observed, written, call.body ?? {}); return unspelled === undefined ? ok(written) : err(unspelled); }; +/** Set each made object's members that hold the entity's identity to the identity it was written with, generated or sent. */ +const ownMadeMembers = ( + rules: readonly FieldRule[], + written: JsonObject, + idField: string | undefined, +): void => { + const identity = + idField ?? rules.find((rule) => rule.rule === "identity")?.field; + const id = identity === undefined ? undefined : ownMember(written, identity); + if (id === undefined) return; + for (const rule of rules) { + if (rule.rule !== "made-object" || rule.own.length === 0) continue; + const made = ownMember(written, rule.field); + if (made === undefined || !isObject(made)) continue; + const owned: JsonObject = { ...made }; + for (const member of rule.own) + setOwnMember(owned, member, spelledIdentity(ownMember(made, member), id)); + setOwnMember(written, rule.field, owned); + } +}; + +/** An identity spelled as a recorded member spells it: a string, or a number when the identity is one. */ +const spelledIdentity = ( + exemplar: JsonValue | undefined, + id: JsonValue, +): JsonValue => (typeof exemplar === "string" ? (scalarText(id) ?? id) : id); + /** Rules an update gives a new value on every call. */ const REGENERATED: ReadonlySet = new Set([ "clock", diff --git a/src/domain/analysisView/environmentStateWriteCreate.ts b/src/domain/analysisView/environmentStateWriteCreate.ts index 616bc9b927..694c45d0c5 100644 --- a/src/domain/analysisView/environmentStateWriteCreate.ts +++ b/src/domain/analysisView/environmentStateWriteCreate.ts @@ -20,10 +20,14 @@ import { } from "./environmentStateWriteApply.js"; import { createdToken, + heldObjects, isOpaqueToken, + nextMadeCount, restampedFields, + stampedBy, } from "./environmentStateWriteStamps.js"; import { + anyLinkedSource, recordedPathValues, respondedIds, sameLinkedSource, @@ -91,6 +95,7 @@ import { isUuid, likeGeneratedInteger, nextSequence, + startedSequence, sequenceNumber, spelledLike, } from "./environmentStateWriteValues.js"; @@ -305,6 +310,8 @@ const refineCreate = ( const earlierOf = (draft: WriteDraft) => ({ kept: draft.kept, clockOf: (field: string) => clockEvidence(draft, field), + startOf: (field: string) => draft.starts.get(field), + createCollections: draft.createCollections, ampersands: draft.ampersands, }); @@ -466,6 +473,13 @@ const buildCreate = ( noteHeldAmpersands(draft.ampersands, explained.rules, held); noteAmpersands(draft.ampersands, explained.rules, evidence.observed.object); for (const entity of held) noteFixedWidths(draft.fixedWidth, entity); + const ruled = new Set(explained.rules.map((rule) => rule.field)); + for (const [field, value] of Object.entries(evidence.observed.object)) { + const integer = sequenceNumber(value); + if (ruled.has(field) || integer === undefined) continue; + const next = nextInteger(held, field, value); + if (next === undefined || integer >= next) draft.starts.set(field, integer); + } return true; }; @@ -505,10 +519,13 @@ const followsSequences = ( rule.scope === "type" || entity.collection === created.collection, ); const next = nextSequence( - nextInteger( - fieldsOf(scope), - rule.field, - ownMember(draft.observed, rule.field), + startedSequence( + nextInteger( + fieldsOf(scope), + rule.field, + ownMember(draft.observed, rule.field), + ), + rule.start, ), draft.sequences.get(key), ); @@ -523,9 +540,17 @@ const agreesLater = ( context: EffectContext, draft: WriteDraft, created: Pick, - call: { readonly body: JsonObject; readonly observed: JsonObject }, + call: { + readonly body: JsonObject; + readonly observed: JsonObject; + readonly startMs: number | undefined; + }, ): boolean => - followsCreateRules(context, draft, { ...created, observed: call.observed }) && + followsCreateRules(context, draft, { + ...created, + observed: call.observed, + startMs: call.startMs, + }) && !contradictsDefault(draft.plan.rules, call, created.fields) && !unruledDiffers(draft.plan.rules, call.observed, { written: created.fields, @@ -572,6 +597,7 @@ const followsCreateRules = ( draft: WriteDraft, created: Pick & { readonly observed: JsonObject; + readonly startMs: number | undefined; }, ): boolean => draft.plan.rules.every((rule) => { @@ -607,7 +633,56 @@ const followsCreateRules = ( return ( rule.rule !== "identity" || rule.generator !== "uuid" || isUuid(value) ); - }) && followsSequences(context, draft, created); + }) && + followsSequences(context, draft, created) && + followsMadeObjects(context, draft, created); + +/** The key a made object's count is taken under in a plan's sequences. */ +const madeKey = (field: string, member: string): string => `${field}/${member}`; + +/** Whether a later call's recorded made objects hold what serving makes: the exemplar's members, the entity's identity, clocks, and each count one past what held objects show or started with, past the plan's last. */ +const followsMadeObjects = ( + context: EffectContext, + draft: WriteDraft, + created: Pick & { + readonly observed: JsonObject; + readonly startMs: number | undefined; + }, +): boolean => + draft.plan.rules.every((rule) => { + if (rule.rule !== "made-object") return true; + const made = ownMember(created.observed, rule.field); + const exemplar = ownMember(draft.observed, rule.field); + if (!isObject(made) || !isObject(exemplar)) return false; + const id = scalarText( + ownMember(created.fields, context.idField(created.type)), + ); + const held = heldObjects( + [ + ...context.state.ofType(created.type), + ...context.startedWith(created.type), + ].map((entity) => entity.fields), + rule.field, + ); + return [...new Set([...Object.keys(exemplar), ...Object.keys(made)])].every( + (member) => { + const value = ownMember(made, member); + if (rule.clocks.includes(member)) + return stampedBy(value, rule.format, created.startMs); + if (rule.own.includes(member)) return scalarText(value) === id; + const count = rule.counts.find((entry) => entry.member === member); + if (count === undefined) + return same(value, ownMember(exemplar, member)); + return ( + sequenceNumber(value) === + nextSequence( + nextMadeCount(held, count, ownMember(exemplar, member)), + draft.sequences.get(madeKey(rule.field, member)), + ) + ); + }, + ); + }); /** Widen the plan's clock formats to the trimmed spelling a later call showed, and its other clocks with them unless a value the plan saw shows them at a fixed width. */ const fitClocks = (draft: WriteDraft, fields: JsonObject): void => { @@ -643,6 +718,17 @@ const takeSequences = ( ); if (value !== undefined) draft.sequences.set(key, value); } + for (const rule of draft.plan.rules) { + if (rule.rule !== "made-object") continue; + const made = ownMember(created.fields, rule.field); + for (const { member } of rule.counts) { + const value = isObject(made) + ? sequenceNumber(ownMember(made, member)) + : undefined; + if (value !== undefined) + draft.sequences.set(madeKey(rule.field, member), value); + } + } }; /** Segments of a recorded path as a create call supplies them: template literals, and each parameter's recorded spelling. */ @@ -709,6 +795,7 @@ const fromLinkedSource = ( draft: WriteDraft, step: WriteStep, ): boolean => + anyLinkedSource(draft.plan, step.tool.path_template) || sameLinkedSource( step.tool.path_template, context.records.find( @@ -838,6 +925,7 @@ export const applyCreate = ( ownMember(observed.object, rule.field) ?? null, ), referencedIn(context), + idField, ); // A later call that agrees with the plan widens it, as a later update does: to members it alone sent or left out, an // array it showed sorted, or a clock spelling. One that disagrees leaves the plan as it was. @@ -861,7 +949,11 @@ export const applyCreate = ( context, draft, { type, collection, fields: fields.value }, - { body: call.body, observed: observed.object }, + { + body: call.body, + observed: observed.object, + startMs: startOf(context, step), + }, )) ) { refuse(draft, step, "unexplained-change"); @@ -877,6 +969,8 @@ export const applyCreate = ( draft.plan.rules, ); const inserted = insertCreated(context, draft, entity(stored, id)); + context.io.retain(collection.length * 2 + 16); + draft.createCollections.push(collection); return { kind: "applied", targets: [inserted, collectionTarget(type, collection)], diff --git a/src/domain/analysisView/environmentStateWriteCreateCalls.ts b/src/domain/analysisView/environmentStateWriteCreateCalls.ts index 91e04bc91a..1383545146 100644 --- a/src/domain/analysisView/environmentStateWriteCreateCalls.ts +++ b/src/domain/analysisView/environmentStateWriteCreateCalls.ts @@ -1,5 +1,11 @@ import type { JsonValue } from "../jsonValue.js"; -import { ownMember, same, type JsonObject } from "./environmentStateRules.js"; +import { + isObject, + ownMember, + same, + scalarText, + type JsonObject, +} from "./environmentStateRules.js"; import { storedKinds } from "./environmentStateWriteApply.js"; import { fittedClockFormat } from "./environmentStateWriteClockFormats.js"; import { composedValue } from "./environmentStateWriteComposed.js"; @@ -9,7 +15,13 @@ import { type FieldRule, type WriteBody, } from "./environmentStateWriteRules.js"; -import { slugOf, type Ampersand } from "./environmentStateWriteValues.js"; +import { + containsDelimited, + replaceDelimited, + sequenceNumber, + slugOf, + type Ampersand, +} from "./environmentStateWriteValues.js"; type Default = { readonly member: string; readonly value: JsonValue }; type Blank = NonNullable[number]; @@ -18,6 +30,9 @@ type Blank = NonNullable[number]; export interface EarlierEvidence { readonly kept: ReadonlySet; readonly clockOf: (field: string) => JsonValue | undefined; + /** The integer the exemplar showed in an unruled field above every value its type held, where a sequence may start. */ + readonly startOf?: (field: string) => number | undefined; + readonly createCollections?: readonly string[]; /** How the plan's accepted calls and the type's held entities spelled `&` in each field's slug; absent where none showed one. */ readonly ampersands?: ReadonlyMap; } @@ -25,7 +40,7 @@ export interface EarlierEvidence { /** How a field's slug showed `&`: dropped, as the word, or in a way neither spelling gives. */ export type AmpersandSpelling = "dropped" | Ampersand | "unknown"; -/** The plan's rule for a field and a later call's agree: equal, or clocks one spelling covers given what earlier calls showed. */ +/** The plan's rule for a field and a later call's agree: equal, clocks one spelling covers given what earlier calls showed, slugs that differ only in how they spell `&`, or made objects alike but in the counts only the later call could show. */ const agreeing = ( held: FieldRule, later: FieldRule, @@ -39,6 +54,13 @@ const agreeing = ( ): FieldRule | undefined => { const alike = alikeRules(held, later, values.earlier.kept); if (alike !== undefined) return alike; + if (held.rule === "made-object" && later.rule === "made-object") + return madeAlike( + held, + later, + ownMember(values.exemplar, values.field), + values.earlier, + ); const spelled = ampersandSpelled( held, { rule: later, entity: values.laterEntity }, @@ -54,6 +76,41 @@ const agreeing = ( return format === undefined ? undefined : { ...held, format }; }; +type MadeObject = Extract; + +/** Two made-object rules as one: the same clocks, format and own members, and the earlier counts among the later's; a count only the later call shows starts at the exemplar's value, which no earlier object could show past, and only while the exemplar is the one create accepted, as earlier creates held it still. */ +const madeAlike = ( + held: MadeObject, + later: MadeObject, + exemplar: JsonValue | undefined, + earlier: EarlierEvidence, +): MadeObject | undefined => { + if ( + !same(held.clocks, later.clocks) || + !same(held.own, later.own) || + !same(held.format, later.format) || + held.counts.some( + ({ member }) => !later.counts.some((count) => count.member === member), + ) + ) + return undefined; + const counts: MadeObject["counts"] = []; + for (const { member } of later.counts) { + const kept = held.counts.find((count) => count.member === member); + // Accepted creates past the exemplar showed this member unchanged, which a count would have moved. + if (kept === undefined && (earlier.createCollections ?? []).length > 1) + return undefined; + const start = + kept?.start ?? + sequenceNumber( + isObject(exemplar) ? ownMember(exemplar, member) : undefined, + ); + if (start === undefined) return undefined; + counts.push({ member, start }); + } + return { ...held, counts }; +}; + /** A rule with its slugs spelled the default way. */ const unspelled = (rule: FieldRule): FieldRule => { if (rule.rule === "slug") { @@ -196,6 +253,44 @@ const joinedDefaults = ( return [...joined].map(([member, value]) => ({ member, value })); }; +/** A later call's rule for a field the plan kept as shown, when it spells the exemplar's value too: a string composed from the exemplar's fields or holding the value it templates, or a sequence that started at its value. */ +const spellsExemplar = ( + field: string, + rule: FieldRule, + calls: Pick, +): FieldRule | undefined => { + const value = ownMember(calls.exemplar, field); + if (rule.rule === "composed") + return value !== undefined && + composedValue(rule.parts, calls.exemplar) === value + ? rule + : undefined; + if (rule.rule === "template") { + const token = scalarText(ownMember(calls.exemplar, rule.from)); + const next = scalarText(ownMember(calls.later.entity, rule.from)); + return typeof value === "string" && + token !== undefined && + next !== undefined && + containsDelimited(value, token) && + replaceDelimited(value, token, next) === + ownMember(calls.later.entity, field) + ? rule + : undefined; + } + if (rule.rule !== "sequence") return undefined; + const start = calls.earlier.startOf?.(field); + const collections = calls.earlier.createCollections ?? []; + if ( + (rule.scope === "type" && collections.length > 1) || + (rule.scope === "collection" && + new Set(collections).size !== collections.length) + ) + return undefined; + return start !== undefined && sequenceNumber(value) === start + ? { ...rule, start } + : undefined; +}; + /** What two calls show of one field. */ interface FieldCalls { readonly before: FieldRule | undefined; @@ -273,6 +368,10 @@ const mergedField = ( default: { member: field, value: later.entity[field]! }, } : undefined; + if (before === undefined && after !== undefined) { + const rule = spellsExemplar(field, after, calls); + if (rule !== undefined) return { rule }; + } // This call echoes a member the plan's calls left out: what they stored there is its default. if (after !== undefined) return after.rule === "echo" && !calls.required.includes(field) diff --git a/src/domain/analysisView/environmentStateWriteEffects.ts b/src/domain/analysisView/environmentStateWriteEffects.ts index e0c74c51fe..4ace3c1121 100644 --- a/src/domain/analysisView/environmentStateWriteEffects.ts +++ b/src/domain/analysisView/environmentStateWriteEffects.ts @@ -94,6 +94,10 @@ export interface WriteDraft { readonly updates: UpdateCall[]; /** The last value each sequence took, by field and, for a collection's sequence, collection; the next call must follow it. */ readonly sequences: Map; + /** Integers the exemplar showed in fields no rule wrote, above every value its type held, where a later call's sequence may start. */ + readonly starts: Map; + /** Collections of accepted create calls, used to check late sequence adoption. */ + readonly createCollections: string[]; /** The collection a create writes into when its own path names no state type: the one its returned id was read under. */ readonly linked?: string | undefined; /** Fields an item action's call found already at the value it sets them to. */ diff --git a/src/domain/analysisView/environmentStateWriteExplain.ts b/src/domain/analysisView/environmentStateWriteExplain.ts index f40c00f6b2..6b2563724e 100644 --- a/src/domain/analysisView/environmentStateWriteExplain.ts +++ b/src/domain/analysisView/environmentStateWriteExplain.ts @@ -55,6 +55,7 @@ import { GENERATED, } from "./environmentStateWriteGenerated.js"; import { + createdMadeObject, createdNestedClock, createdToken, isClockNear, @@ -281,7 +282,10 @@ const generatedRule = ( return { field, rule: "unique" }; if (createdToken(field, evidence.entity, evidence.type) === "independent") return { field, rule: "unique" }; - return createdNestedClock(field, value, evidence); + return ( + createdNestedClock(field, value, evidence) ?? + createdMadeObject(field, value, evidence) + ); }; /** Pinned members less those a reference reads: it rebuilds the object they name for any held entity. */ diff --git a/src/domain/analysisView/environmentStateWriteLinks.ts b/src/domain/analysisView/environmentStateWriteLinks.ts index 3c428a20b3..9c751ccb94 100644 --- a/src/domain/analysisView/environmentStateWriteLinks.ts +++ b/src/domain/analysisView/environmentStateWriteLinks.ts @@ -23,6 +23,7 @@ import { import { ENTRY_BYTES, textBytes } from "./webNetworkRouteTemplates.js"; import { pathValuesOf, + type FieldRule, type WriteEffect, } from "./environmentStateWriteRules.js"; @@ -44,9 +45,14 @@ const identityValues = (object: JsonValue | undefined): string[] => }) : []; +/** The one object a one-element array holds, else the value itself. */ +const unwrapped = (value: JsonValue): JsonValue => + Array.isArray(value) && value.length === 1 ? value[0]! : value; + /** * The ids a write's response names for an entity it made: an identity-named member of the body or of - * an object member (an envelope) at any length, or a scalar body or sole scalar member long enough to name one. + * an object member (an envelope, or the one object of a one-element array member) at any length, or a + * scalar body or sole scalar member long enough to name one. */ export const respondedIds = (body: JsonValue | undefined): string[] => { if (body === undefined || body === null) return []; @@ -60,7 +66,7 @@ export const respondedIds = (body: JsonValue | undefined): string[] => { ...new Set([ ...identityValues(body), ...(members.length === 1 ? named(members[0]) : []), - ...members.flatMap(identityValues), + ...members.flatMap((member) => identityValues(unwrapped(member))), ]), ]; }; @@ -395,6 +401,27 @@ export const recordedPathValues = ( ); }; +/** Whether a linked create may take any path arguments: it creates into its type's own collection, whose path has no parameters, and stores each path value in a field, so none changes where it creates. */ +export const anyLinkedSource = ( + plan: { + readonly target: string | null; + readonly collection: string | null; + readonly rules: readonly FieldRule[]; + }, + template: string, +): boolean => + plan.target !== null && + plan.collection !== null && + normalizedPath(plan.collection) === plan.target && + !segmentsOf(plan.target).some(isParam) && + segmentsOf(template) + .filter(isParam) + .every((segment) => + plan.rules.some( + (rule) => rule.rule === "path" && rule.from === segment.slice(1, -1), + ), + ); + /** * Whether a linked create's call sends the path values its exemplar did, each as recorded or in the * other spelling once percent-decoded: the read that linked its target followed only that call. The diff --git a/src/domain/analysisView/environmentStateWriteRules.ts b/src/domain/analysisView/environmentStateWriteRules.ts index a91c60050f..3378f67be8 100644 --- a/src/domain/analysisView/environmentStateWriteRules.ts +++ b/src/domain/analysisView/environmentStateWriteRules.ts @@ -42,7 +42,8 @@ export const usesWriteClock = (plan: Pick): boolean => (rule) => rule.rule === "clock" || rule.rule === "held-clock" || - rule.rule === "nested-clock", + rule.rule === "nested-clock" || + rule.rule === "made-object", ); /** Rules that generate a value when served; the gate takes the recorded value instead. */ @@ -52,10 +53,12 @@ export type GeneratedRule = readonly rule: "identity"; readonly generator: "sequence" | "uuid" | "token"; } + /** One past the largest value its scope holds, and never below `start`, a value the first create took above them. */ | { readonly field: string; readonly rule: "sequence"; readonly scope: "collection" | "type"; + readonly start?: number; } | { readonly field: string; readonly rule: "unique" } /** The value kept with each UUID at the pointers (`*` for any array index) replaced by a fresh one, as rewritten child rows are. */ @@ -81,6 +84,15 @@ export type GeneratedRule = readonly field: string; readonly rule: "held-clock"; readonly format: ClockFormat; + } + /** An object made with the entity, which no initial entity holds: `clocks` take the write's clock, `own` the entity's identity, each of `counts` one past the largest value held objects show and never below its start; other members as the exemplar showed them. */ + | { + readonly field: string; + readonly rule: "made-object"; + readonly clocks: string[]; + readonly format: ClockFormat; + readonly own: string[]; + readonly counts: { readonly member: string; readonly start: number }[]; }; /** How one field of a written entity gets its value; fields no rule names keep the observed value. */ diff --git a/src/domain/analysisView/environmentStateWriteStamps.ts b/src/domain/analysisView/environmentStateWriteStamps.ts index 000b72583d..ae6f49bef9 100644 --- a/src/domain/analysisView/environmentStateWriteStamps.ts +++ b/src/domain/analysisView/environmentStateWriteStamps.ts @@ -20,6 +20,9 @@ import { containsDelimited, isUuid, readClock, + sequenceNumber, + spelledLike, + type ClockFormat, } from "./environmentStateWriteValues.js"; /** A clock value counts as written by the request when it lies this close to the request's start. */ @@ -243,6 +246,147 @@ export const createdNestedClock = ( return stamped ? nestedRule(field, value, members, evidence.type) : undefined; }; +/** Whether a clock is one a request starting at `startMs` stamped, in the given format. */ +export const stampedBy = ( + value: JsonValue | undefined, + format: ClockFormat, + startMs: number | undefined, +): boolean => { + const clock = readClock(value); + return ( + clock !== undefined && + startMs !== undefined && + same(clock.format, format) && + clock.ms >= startMs - TRUNCATION_MS && + clock.ms <= startMs + NESTED_WINDOW_MS + ); +}; + +/** The largest integer the objects hold at a member, spelled as the exemplar spells it. */ +const largestAt = ( + objects: readonly JsonObject[], + member: string, + exemplar: JsonValue | undefined, +): number | undefined => { + let largest: number | undefined; + for (const object of objects) { + const value = sequenceNumber(ownMember(object, member), exemplar); + if (value !== undefined && (largest === undefined || value > largest)) + largest = value; + } + return largest; +}; + +/** The objects held entities keep at a field. */ +export const heldObjects = ( + entities: readonly JsonObject[], + field: string, +): JsonObject[] => + entities.flatMap((entity) => { + const value = ownMember(entity, field); + return value !== undefined && isObject(value) ? [value] : []; + }); + +/** One past the largest integer held objects show at a member, never below a start. */ +export const nextMadeCount = ( + objects: readonly JsonObject[], + count: { readonly member: string; readonly start: number }, + exemplar: JsonValue | undefined, +): number => { + const largest = largestAt(objects, count.member, exemplar); + return Math.max( + largest === undefined ? count.start : largest + 1, + count.start, + ); +}; + +/** + * On a create, an object the application made with the entity: members holding a clock the request stamped, others the + * entity's own identity or one past the largest integer held objects show there, and the rest as every held object + * shows them. Undefined without a stamped clock, since only that shows the request made the object. + */ +export const createdMadeObject = ( + field: string, + value: JsonValue, + evidence: { + readonly startMs: number | undefined; + readonly type: readonly JsonObject[]; + readonly entity: JsonObject; + readonly idField: string; + }, +): FieldRule | undefined => { + const { startMs } = evidence; + if (!isObject(value) || startMs === undefined) return undefined; + const keys = Object.keys(value); + const clocks = keys.filter((key) => { + const ms = readClock(value[key])?.ms; + return ( + ms !== undefined && + ms >= startMs - TRUNCATION_MS && + ms <= startMs + NESTED_WINDOW_MS + ); + }); + const format = + clocks.length === 0 ? undefined : readClock(value[clocks[0]!])!.format; + if ( + format === undefined || + clocks.some((key) => !same(readClock(value[key])!.format, format)) + ) + return undefined; + const id = scalarText(ownMember(evidence.entity, evidence.idField)); + const own = keys.filter( + (key) => + !clocks.includes(key) && + id !== undefined && + scalarText(value[key]) === id, + ); + const held = heldObjects(evidence.type, field); + const counts = keys.flatMap((key) => { + if (clocks.includes(key) || own.includes(key)) return []; + const integer = sequenceNumber(value[key]); + const largest = largestAt(held, key, value[key]); + return integer !== undefined && + largest !== undefined && + integer === largest + 1 + ? [{ member: key, start: integer }] + : []; + }); + const varying = new Set([ + ...clocks, + ...own, + ...counts.map(({ member }) => member), + ]); + const alike = held.every((object) => + [...new Set([...Object.keys(object), ...keys])].every( + (key) => + varying.has(key) || same(ownMember(object, key), ownMember(value, key)), + ), + ); + return alike + ? { field, rule: "made-object", clocks, format, own, counts } + : undefined; +}; + +/** A served made object: the exemplar's, its clocks set to the write's clock and each count to the value `next` gives; its own members follow the entity's identity once written. */ +export const madeObjectValue = ( + base: JsonValue | undefined, + rule: Extract, + clock: JsonValue, + next: (count: { readonly member: string; readonly start: number }) => number, +): Result => { + if (base === undefined || !isObject(base)) + return err(`The plan holds no ${rule.field} object to make.`); + const made: JsonObject = { ...base }; + for (const member of rule.clocks) setOwnMember(made, member, clock); + for (const count of rule.counts) + setOwnMember( + made, + count.member, + spelledLike(ownMember(base, count.member) ?? "", next(count)), + ); + return ok(made); +}; + /** A served nested clock: the object the entity holds, or the observed one for a create, with its stamped members set to the clock. */ export const nestedClockValue = ( base: JsonValue | undefined, diff --git a/src/domain/analysisView/environmentStateWriteValues.ts b/src/domain/analysisView/environmentStateWriteValues.ts index aa62539e65..b2135f7ae0 100644 --- a/src/domain/analysisView/environmentStateWriteValues.ts +++ b/src/domain/analysisView/environmentStateWriteValues.ts @@ -249,6 +249,13 @@ export const nextSequence = ( ): number => taken === undefined ? (held ?? 1) : Math.max(held ?? taken + 1, taken + 1); +/** The next held integer of a sequence that never goes below its `start`. */ +export const startedSequence = ( + held: number | undefined, + start: number | undefined, +): number | undefined => + start === undefined ? held : Math.max(held ?? start, start); + /** An integer spelled as the recorded value spells integers: a number, or a string of digits. */ export const spelledLike = (exemplar: JsonValue, value: number): JsonValue => maskedCount(exemplar) !== undefined diff --git a/src/generatedPackageMetadata.ts b/src/generatedPackageMetadata.ts index 096f718c69..25bb35bb64 100644 --- a/src/generatedPackageMetadata.ts +++ b/src/generatedPackageMetadata.ts @@ -6,5 +6,5 @@ export const PACKAGE_METADATA = { serverSdkVersion: "2.3.1", clientSdkVersion: "2.3.1", coreSdkVersion: "2.3.1", - skillVersion: "52", + skillVersion: "53", } as const; diff --git a/tests/boundary/mcp/environmentServeMcp.test.ts b/tests/boundary/mcp/environmentServeMcp.test.ts index 83831a6496..ba29ccba49 100644 --- a/tests/boundary/mcp/environmentServeMcp.test.ts +++ b/tests/boundary/mcp/environmentServeMcp.test.ts @@ -1,5 +1,6 @@ import { createHash } from "node:crypto"; import { timedCaptureEvidence } from "../../fixtures/analysisView.js"; +import { INITIAL_TRASH_ROWS } from "../../fixtures/environmentStateLaterReads.js"; import { jsonObjectSchema } from "../../../src/domain/jsonValue.js"; import { @@ -23,9 +24,9 @@ import { const HTTP_BOUNDARY_TIMEOUT_MS = 90_000; const CARD_PATH = "/.well-known/agent-env.json"; -/** SHA-256 of the 13,882-byte card served for the fixture named "renamed", its descriptions naming each tool's recorded responses; pure replay must serve these bytes. */ +/** SHA-256 of the 14,519-byte card served for the fixture named "renamed", its descriptions naming each tool's recorded responses; pure replay must serve these bytes. */ const REPLAY_CARD_SHA256 = - "27370fe0a89556dcd0dd1bc6ddd0439d77b628349829afd56e4af4ee393d0a64"; + "995c280f0798f77e76f206c068eb6c2c1f8c4210a03ac1c01fba8cc07b5d7f09"; /** Serve the fixture pair as pure replay through the built CLI. */ const serveReplayOnly = async ( @@ -58,6 +59,73 @@ const connect = async (url: URL) => { return client; }; +processTest( + "replays a recorded list containing an initially deleted entity", + async ({ processes }) => { + const fixture = environmentStateServeFixture( + timedCaptureEvidence(INITIAL_TRASH_ROWS), + ); + const root = await createTestTempDirectory("rea-environment-trash-mcp-"); + const files = await writeEnvironmentServeFiles(root, fixture); + const started = await startListeningProcess(processes, [ + "environment-serve", + files.manifestPath, + "--evidence", + files.capturePath, + "--port", + "0", + ]); + const client = await connect(started.url); + try { + const response = await client.callTool({ + name: "get_contacts", + arguments: { maxSize: "20", offset: "0" }, + }); + expect(response.isError).toBeFalsy(); + expect(response.structuredContent).toMatchObject({ + body: INITIAL_TRASH_ROWS[0]![2], + }); + } finally { + await client.close(); + } + }, + HTTP_BOUNDARY_TIMEOUT_MS, +); + +processTest( + "accepts numbers and booleans for query parameters, sent as their JSON text", + async ({ processes }) => { + const fixture = environmentStateServeFixture( + timedCaptureEvidence([ + [ + "GET", + "/api/contacts?maxSize=2&withItems=true", + [{ id: 1, name: "Ada" }], + ], + ]), + ); + const started = await serveReplayOnly(processes, {}, fixture); + const client = await connect(started.url); + try { + const response = await client.callTool({ + name: "get_contacts", + arguments: { maxSize: 2, withItems: true }, + }); + expect(response.isError).toBeFalsy(); + expect(response.structuredContent).toMatchObject({ + body: [{ id: 1, name: "Ada" }], + replay: { + exact_match: true, + recorded_query: "?maxSize=2&withItems=true", + }, + }); + } finally { + await client.close(); + } + }, + HTTP_BOUNDARY_TIMEOUT_MS, +); + processTest( "replays recorded own JSON keys without interpreting inherited members as fields", async ({ processes }) => { @@ -113,6 +181,8 @@ processTest( ).toMatchObject({ oneOf: [ { type: "string" }, + { type: "number" }, + { type: "boolean" }, { type: "array", items: { type: "string" } }, ], }); diff --git a/tests/fixtures/environmentStateLaterReads.ts b/tests/fixtures/environmentStateLaterReads.ts index 18807c6486..bdac849871 100644 --- a/tests/fixtures/environmentStateLaterReads.ts +++ b/tests/fixtures/environmentStateLaterReads.ts @@ -18,6 +18,18 @@ const ADA = contact(1, "Ada Finch"); const MINA = contact(2, "Mina Okafor"); const RUI = contact(3, "Rui Castellanos"); +/** An initial trash-style list includes an entity already marked deleted. */ +export const INITIAL_TRASH_ROWS: readonly TimedCaptureRow[] = [ + [ + "GET", + "/api/contacts?maxSize=20&offset=0", + page([ADA, MINA, { ...RUI, deleted: true }]), + ], + ["GET", "/api/contacts/1", ADA], + ["GET", "/api/contacts/2", MINA], + ["GET", "/api/contacts/3", { ...RUI, deleted: true }], +]; + /** A contact deleted, then read by id as EspoCRM answers for soft-deleted records: still found, marked `deleted`. */ export const SOFT_DELETED_ROWS: readonly TimedCaptureRow[] = [ ["GET", "/api/contacts?maxSize=20&offset=0", page([ADA, MINA, RUI])], diff --git a/tests/fixtures/environmentStateMadeValues.ts b/tests/fixtures/environmentStateMadeValues.ts new file mode 100644 index 0000000000..2d9f01c31b --- /dev/null +++ b/tests/fixtures/environmentStateMadeValues.ts @@ -0,0 +1,167 @@ +import type { JsonObject } from "../../src/domain/analysisView/environmentStateRules.js"; +import type { TimedCaptureRow } from "./analysisView.js"; +import { bookClock } from "./environmentStateWrites.js"; + +/** How a capture of two task creates departs from Vikunja's. */ +interface TaskOptions { + /** Whether tasks carry a `created` clock of their own besides their subscription's. */ + readonly clocked?: boolean; + /** Whether each create sends the id the task then holds. */ + readonly sentId?: boolean; + /** A third create, whose subscription names this task rather than its own. */ + readonly thirdSubscribes?: number; + /** The first create's `identifier`, when not `#` and its index. */ + readonly firstIdentifier?: string; + /** Whether lists show each task's subscription, as item reads do. */ + readonly listsSubscriptions?: boolean; + /** A third create of its own task whose subscription shows a clock from before its request. */ + readonly staleThird?: boolean; +} + +const seed = (id: number, index: number, clocked: boolean): JsonObject => ({ + id, + title: `Seeded task ${String(id)}`, + project_id: 5, + index, + identifier: `#${String(index)}`, + subscription: null, + ...(clocked ? { created: "2026-10-01T09:00:00Z" } : {}), +}); + +/** Two task creates at a project path, numbered past tasks deleted before the capture and read back with the subscription the application made, then the list. */ +const taskRows = (options: TaskOptions = {}): TimedCaptureRow[] => { + const clocked = options.clocked ?? true; + const seeds = [seed(1, 1, clocked), seed(2, 2, clocked)]; + const made = ( + id: number, + index: number, + title: string, + row: number, + identifier = `#${String(index)}`, + ): JsonObject => ({ + id, + title, + project_id: 5, + index, + identifier, + subscription: null, + ...(clocked ? { created: bookClock(row) } : {}), + }); + const subscribed = ( + task: JsonObject, + subscription: number, + row: number, + entity = task["id"]!, + ): JsonObject => ({ + ...task, + subscription: { + id: subscription, + entity: "task", + entity_id: entity, + created: bookClock(row), + }, + }); + const first = made( + 20, + 15, + "Replace the lens gasket", + 2, + options.firstIdentifier, + ); + const second = made(21, 16, "Log the fog-signal tests", 5); + const sent = (task: JsonObject) => ({ + ...(options.sentId === true ? { id: task["id"]! } : {}), + title: task["title"]!, + }); + const firstRead = subscribed(first, 21, 2); + const secondRead = subscribed(second, 22, 5); + const listed = (task: JsonObject): JsonObject => { + if (options.listsSubscriptions !== false) return task; + const { subscription: _subscription, ...rest } = task; + return rest; + }; + return [ + ["GET", "/api/tasks", seeds.map(listed)], + ["GET", "/api/tasks/1", seeds[0]!], + ["POST", "/api/projects/5/tasks", first, 201, sent(first)], + ["GET", "/api/tasks/20", firstRead], + ["GET", "/api/tasks", [...seeds, firstRead].map(listed)], + ["POST", "/api/projects/5/tasks", second, 201, sent(second)], + ["GET", "/api/tasks/21", secondRead], + ["GET", "/api/tasks", [...seeds, firstRead, secondRead].map(listed)], + ...(options.thirdSubscribes === undefined && options.staleThird !== true + ? [] + : thirdRows( + made(22, 17, "Paint the gallery rail", 8), + subscribed( + made(22, 17, "Paint the gallery rail", 8), + 23, + options.staleThird === true ? 4 : 8, + options.thirdSubscribes, + ), + [...seeds, firstRead, secondRead].map(listed), + listed, + sent, + )), + ]; +}; + +/** A third create, its read-back and the list after it. */ +const thirdRows = ( + third: JsonObject, + read: JsonObject, + before: readonly JsonObject[], + listed: (task: JsonObject) => JsonObject, + sent: (task: JsonObject) => JsonObject, +): TimedCaptureRow[] => [ + ["POST", "/api/projects/5/tasks", third, 201, sent(third)], + ["GET", "/api/tasks/22", read], + ["GET", "/api/tasks", [...before, listed(read)]], +]; + +/** Vikunja's two task creates, each read back with a made subscription, then the list. */ +export const COUNTED_TASK_ROWS: readonly TimedCaptureRow[] = taskRows(); + +/** As COUNTED_TASK_ROWS, through a bulk path whose body and answer wrap the one task in `tasks`. */ +export const BULK_TASK_ROWS: readonly TimedCaptureRow[] = COUNTED_TASK_ROWS.map( + (row) => + row[0] === "POST" + ? [ + "POST", + `${row[1]}/bulk`, + { tasks: [row[2]!] }, + 201, + { tasks: [row[4]!] }, + ] + : row, +); + +/** COUNTED_TASK_ROWS up to the first create's reads: one create alone shows no counter. */ +export const ONE_TASK_ROWS: readonly TimedCaptureRow[] = + COUNTED_TASK_ROWS.slice(0, 5); + +/** Tasks with no clock of their own, so the subscription's is the only one the creates write. */ +export const UNCLOCKED_TASK_ROWS: readonly TimedCaptureRow[] = taskRows({ + clocked: false, +}); + +/** Creates that send the id the task takes. */ +export const CHOSEN_ID_TASK_ROWS: readonly TimedCaptureRow[] = taskRows({ + sentId: true, +}); + +/** A third create whose subscription names the second task, not its own, in a capture whose lists show no subscription. */ +export const MISLINKED_TASK_ROWS: readonly TimedCaptureRow[] = taskRows({ + thirdSubscribes: 21, + listsSubscriptions: false, +}); + +/** A first create spelling its identifier `TASK-15`, where the second spells `#16`. */ +export const RESPELLED_TASK_ROWS: readonly TimedCaptureRow[] = taskRows({ + firstIdentifier: "TASK-15", +}); + +/** A third create read back with a subscription whose clock is from before its request. */ +export const STALE_MADE_TASK_ROWS: readonly TimedCaptureRow[] = taskRows({ + staleThird: true, +});