feat(environment): serve query numbers, counted values and made objects

- A query parameter accepts a number or boolean, which serving sends as its
  JSON text, as agents send page sizes and flags.
- A sequence may start at the value its first create took above every value
  its type held (Vikunja's per-project task index), and a later create's
  composed, template or started-sequence rule is adopted when it also spells
  the first create's value.
- A made-object rule explains an object the application makes with each
  created entity: members the request stamped, members holding the entity's
  own id, and counts one past the held objects' values.
- A create's answer names the id of the one object a one-element array member
  holds, so bulk creates link to their type.
- The skill reads each listed object by id before writing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Edgar ArakelyanandClaude Opus 5.5 committed 2026-10-26 15:48:57 -07:00
1 parent 4ede130550
commit c0cbf95c9c
29 files changed
+831 -38

No files matched your search

@@ -2,7 +2,7 @@
name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "51"
version: "52"
---
# REA
@@ -115,8 +115,10 @@ client with it. Match `wait_for` locators to exactly one element.
as a secret reference), submit, then `wait_for` an element only a signed-in
user sees.
2. Read with `request` steps: each collection the environment needs, twice
before any write, so fields that change on their own show up; one item of
each.
before any write, so fields that change on their own show up; then each
object a list shows by id, up to ten per collection. An object known only
from a list that leaves out fields its own read shows cannot be read by id
or updated in the environment.
3. Use `request` steps only when the API lets the caller choose each scratch
object's ID or a prior read already established it. Request paths are fixed
strings; no action substitutes a value from an earlier response. For
+5 -3
View File
@@ -402,7 +402,8 @@ returns that record's recorded response. Selection compares the supplied path
parameters against the record's path segments at the parameter indexes, then
the supplied query values against the record's query string, and keeps the
record with the most matches; ties keep manifest order. A query argument accepts
one string or an array of strings for repeated values; the array is matched
one string, a number or boolean sent as its JSON text, or an array of strings for
repeated values; the array is matched
against every occurrence in captured order. The raw recorded query remains the
replay evidence. The result reports
where the answer came from:
@@ -609,8 +610,9 @@ shape and carries no annotations.
Path parameters are required strings whose
description names the segment index, the templating rule and the observed
values (sampled within the list budget above; the manifest keeps them all);
query parameter names accept an optional string or array of strings (for
repeated values in captured order); write methods accept an
query parameter names accept an optional string, number or boolean (a number or
boolean is sent as its JSON text) or array of strings (for repeated values in
captured order); write methods accept an
optional `body` object described by the recorded top-level key types. Each
input keeps its own argument: a query name equal to a path parameter's name is
advertised as `query_<name>`, a query name longer than 64 characters as
@@ -0,0 +1,12 @@
# Implementation plan
1. **Query values.** Widen the query property schema and spell numbers and booleans as JSON text where a call enters
serving.
2. **Sequence start.** `start` on sequence rules and their schema; the exemplar records integer fields above every held
value; serving and replay floor the next value at `start`.
3. **Later rules.** Merging a later create adopts its `composed`, `template` or started `sequence` rule when it spells
the exemplar's value.
4. **Made objects.** The `made-object` rule and schema; its explanation, merge, later-call check, counts taken, serving
and own members.
5. **Bulk link.** `respondedIds` reads the object of a one-element array member.
6. **Skill, tests and docs.**
@@ -0,0 +1,44 @@
# Serve the calls agents make and the values applications count
Issue: https://github.com/morluto/rea/issues/4933
## Outcome
Eval agents' trajectories on rebuilt environments, and the write plans of the recordings they ran on, showed three
gaps REA could close:
- **Query values.** Served tools typed each query parameter as a string or an array of strings, while agents send page
sizes and flags as JSON numbers and booleans (`maxSize: 50`, `withItems: true`). The MCP server refused those calls
before REA saw them: 63 of 418 stored eval runs hit it, in EspoCRM, Homebox and PhotoPrism, and 18 never got the tool
to answer.
- **Counted and made values.** Vikunja numbers a project's tasks past those deleted earlier, so a created task's
`index` is not one past what the state holds; its `identifier` spells that index; and each created task gets a
`subscription` object the application makes with it. A second create disagreed with the first on all three, so
task creates were withheld.
- **Bulk creates.** Vikunja's UI creates a task through `POST /projects/{id}/tasks/bulk`, sending and answering the
task wrapped in `tasks: [ … ]`. The answer's id sat inside the array, so REA linked the call to no type.
The diagnosis also found that a recorded update stores only what it sent (EspoCRM's updates never changed an email
address), and that an object known only from a list that leaves out fields its own read shows can be neither read by
id nor updated. Both are recording gaps the skill now addresses.
## Behaviour
- **Query values.** A query parameter accepts a number or boolean besides a string or an array of strings, and serving
sends it as its JSON text.
- **Sequences from a start.** A field the first create showed above every value its type held, which a later create
shows one past the first's, is a `sequence` with that `start`: serving never goes below it.
- **Rules that spell the first create.** A field the plan kept as shown takes a later create's rule when that rule also
spells the first create's value: a `composed` or `template` string, or the `sequence` above.
- **Made objects.** A created object no other entity of the type holds otherwise, with a member the request stamped, is
a `made-object`: stamped members take the write's clock, members holding the entity's identity take its id, counts
go one past the largest value held objects show from a `start`, and other members stay as recorded.
- **Linked bulk creates.** A create's answer names the id of the one object a one-element array member holds.
- **Skill.** Recordings read each listed object by id, up to ten per collection, before writing.
Plans and views that need none of this are unchanged.
## Acceptance
- No write tool of the 130 recorded captures goes from served to withheld.
- Rebuilt from the same recordings, the branch passes at least the write tasks #7274 passes.
@@ -0,0 +1,12 @@
# Tasks
- [x] Accept numbers and booleans for query parameters.
- [x] Start a sequence from the value its first create took.
- [x] Adopt a later create's rule that spells the first create's value.
- [x] Learn, check and serve objects made with an entity.
- [x] Link creates whose answer wraps the entity in a one-element array.
- [x] Ask recordings to read each listed object by id.
- [x] Add view, serving, MCP boundary and unit tests.
- [x] Document the rules and the query argument types.
- [x] Measure the recorded captures and served write tasks.
- [x] Run static, formatting, documentation and test gates.
+16 -5
View File
@@ -732,8 +732,10 @@ order. Its `effect` is `create` for a `POST` to a type's
plus one trailing parameter, or to that item path plus one action segment,
`delete` for a `DELETE` to it, `document` for a `PUT`, `PATCH` or `POST` to a
document's path, an `update` or `delete` of each entity a write at the
collection names by id in its body, and otherwise `none` with reason
`no-state-target`. A plan is built from the tool's first successful call: its
collection names by id in its body, a `create` for a `POST` whose answer names
an id (at its top level, in an object member or in the one object of a
one-element array member) that a later item read of a type first shows, and
otherwise `none` with reason `no-state-target`. A plan is built from the tool's first successful call: its
request body must be a usable JSON object (as response bodies are usable), and
the written entity must show in that call's response or in a later read before
the next write. Each field of a created entity gets the first rule that
@@ -742,10 +744,14 @@ value's shape), `echo` of a body member (`sorted` when the application stored
its strings in code-point order rather than as sent), `copy` of a body string of
8 or more characters, a `path` value, a `slug` of a body member, a `clock` value within 10
minutes of the request in its recorded format, a `sequence` one past the
largest value of its collection or type, a `unique` UUID, opaque token or
largest value of its collection or type, and never below its `start`, a `unique` UUID, opaque token or
integer of 10 or more digits that no entity of the type holds (where every one
holds a distinct integer of the same length, as time-based ids do), a
`nested-clock` object whose clocks the request stamped, a `template` around
`nested-clock` object whose clocks the request stamped, a `made-object` no
other entity of the type holds otherwise, whose `clocks` the request stamped and
whose other members hold the entity's own identity (`own`), one past the largest
value held objects show (`counts`, each from a `start`), or what every held
object shows, a `template` around
a generated value, a `reference` to an entity, or a `composed` string; other
fields keep the observed value. When the first read of a created entity before
the next write shows fields the create's answer lacks, its clocks to another
@@ -776,7 +782,12 @@ when, explained on its own, it agrees with it: both calls show the same fields
but for a reference whose member a call without it did not send (one an earlier
create added stays), every field both rule they rule alike (an echo one showed
sorted stays sorted, unless a call stored its strings out of order as sent),
and a field neither rules holds one value. Members either call sent are accepted, only those every call sent are
and a field neither rules holds one value, unless the later call's rule for it
spells the earlier calls' value too: a `composed` or `template` string, or a
`sequence` started at that value (`start`), as Vikunja numbers a project's tasks
past those deleted before the capture. Two `made-object` rules agree in their
clocks and own members, and a count only the later call shows starts at the
first object's value. Members either call sent are accepted, only those every call sent are
required, and an echoed member a call left out takes the value that call stored
(`body.defaults`). A member a create sent empty (`""`, `null`, `false`, `[]` or
`{}`), as a form sends a field left as it was, is not required: a call that
@@ -0,0 +1,40 @@
import { describe, expect, it } from "vitest";
import { MAX_SIZE_ROWS } from "../../../tests/fixtures/environmentStateLaterReads.js";
import { servedFrom } from "../../../tests/fixtures/environmentStateServing.js";
import { withQueryText } from "./environmentTools.js";
const contacts = () => servedFrom(MAX_SIZE_ROWS);
const listTool = (served: ReturnType<typeof contacts>) => {
const tool = served.source.manifest.tools.find(
(candidate) => candidate.name === "get_contacts",
);
if (tool === undefined) throw new Error("The manifest lacks get_contacts");
return tool;
};
describe("query values given as numbers or booleans", () => {
it("spells a number or boolean query value as its JSON text and leaves others as given", () => {
const tool = listTool(contacts());
const args = {
maxSize: 2,
offset: false,
body: { maxSize: 3 },
};
expect(withQueryText(tool, args)).toEqual({
maxSize: "2",
offset: "false",
body: { maxSize: 3 },
});
const strings = { maxSize: "2", offset: ["0"] };
expect(withQueryText(tool, strings)).toBe(strings);
});
it("answers a list paged by a number as it answers the same page size given as text", () => {
const served = contacts();
expect(served.answer("get_contacts", { maxSize: 2, offset: 0 })).toEqual(
served.answer("get_contacts", { maxSize: "2", offset: "0" }),
);
});
});
@@ -0,0 +1,55 @@
import { describe, expect, it } from "vitest";
import {
BULK_TASK_ROWS,
COUNTED_TASK_ROWS,
} from "../../../tests/fixtures/environmentStateMadeValues.js";
import { servedFrom } from "../../../tests/fixtures/environmentStateServing.js";
import { isObject } from "../../domain/analysisView/environmentStateRules.js";
import type { JsonValue } from "../../domain/jsonValue.js";
const objectOf = (body: JsonValue | undefined) => {
if (!isObject(body)) throw new Error("Expected an object body");
return body;
};
describe("served creates that count and make values", () => {
it("numbers created tasks on from the recorded counter and makes each one's subscription", () => {
const tasks = servedFrom(COUNTED_TASK_ROWS);
const first = objectOf(
tasks.answer("post_projects_tasks_by_id", {
id: "5",
body: { title: "Restock the fog-horn spares" },
}).body,
);
const second = objectOf(
tasks.answer("post_projects_tasks_by_id", {
id: "5",
body: { title: "Paint the gallery rail" },
}).body,
);
expect([first["index"], first["identifier"]]).toEqual([15, "#15"]);
expect([second["index"], second["identifier"]]).toEqual([16, "#16"]);
const read = objectOf(
tasks.answer("get_tasks_by_id", { id: String(second["id"]) }).body,
);
expect(read["subscription"]).toMatchObject({
id: 22,
entity: "task",
entity_id: second["id"],
});
});
it("answers a bulk create in the envelope it was sent in", () => {
const tasks = servedFrom(BULK_TASK_ROWS);
const body = objectOf(
tasks.answer("post_projects_tasks_bulk_by_id", {
id: "5",
body: { tasks: [{ title: "Restock the fog-horn spares" }] },
}).body,
);
expect(body["tasks"]).toMatchObject([
{ title: "Restock the fog-horn spares", index: 15, identifier: "#15" },
]);
});
});
@@ -18,7 +18,10 @@ import {
} from "../../domain/analysisView/environmentStateWriteExplain.js";
import { callRenewal } from "../../domain/analysisView/environmentStateWriteGenerated.js";
import {
heldObjects,
madeObjectValue,
nestedClockValue,
nextMadeCount,
unenveloped,
} from "../../domain/analysisView/environmentStateWriteStamps.js";
import {
@@ -38,6 +41,7 @@ import {
seededInteger,
seededLike,
spelledLike,
startedSequence,
} from "../../domain/analysisView/environmentStateWriteValues.js";
import {
ownMember,
@@ -283,16 +287,41 @@ export const generator = (
exemplar,
sequence(
`${request.tool.name} ${rule.field} ${rule.scope === "collection" ? target.name : ""}`,
nextHeld(
rule.scope === "collection"
? [...target.collection, ...startedIn(target.name)]
: [...ofType, ...started],
rule.field,
exemplar,
startedSequence(
nextHeld(
rule.scope === "collection"
? [...target.collection, ...startedIn(target.name)]
: [...ofType, ...started],
rule.field,
exemplar,
),
rule.start,
),
),
),
);
case "made-object": {
const clock = formatClock(write.clockMs, rule.format);
if (clock === undefined)
return err(
`The generated clock for ${rule.field} is outside its recorded four-digit year format.`,
);
const base = ownMember(target.observed, rule.field);
const held = heldObjects(
[...ofType, ...started].map((entity) => entity.fields),
rule.field,
);
return madeObjectValue(base, rule, clock, (count) =>
sequence(
`${request.tool.name} ${rule.field}/${count.member}`,
nextMadeCount(
held,
count,
isObject(base) ? ownMember(base, count.member) : undefined,
),
),
);
}
case "unique":
return unused(rule.field, (attempt) =>
generatedIntegerDigits(exemplar) === undefined
@@ -14,6 +14,7 @@ import { answerBulk } from "./environmentStateBulkAnswer.js";
import type { EnvironmentStateStore } from "./environmentStateStore.js";
import { answerWrite } from "./environmentStateWriteAnswer.js";
import { pinnedRequest } from "./environmentStateWriteQuery.js";
import { withQueryText } from "./environmentTools.js";
/** How a served environment answers one tool call. */
export type EnvironmentToolAnswer = (
@@ -29,8 +30,14 @@ export const environmentToolAnswer = (
): EnvironmentToolAnswer =>
store === undefined
? (tool, args, observedQuery) =>
replayEnvironmentTool(source, tool, args, observedQuery)
: (tool, args, observedQuery) => {
replayEnvironmentTool(
source,
tool,
withQueryText(tool, args),
observedQuery,
)
: (tool, given, observedQuery) => {
const args = withQueryText(tool, given);
const plan = store.state.writes.get(tool.name);
if (plan?.served !== true)
return answerFromState({ source, store }, tool, args, observedQuery);
@@ -92,9 +92,14 @@ describe("environmentToolDefinitions", () => {
expect(search.additionalProperties).toBe(false);
expect(Object.keys(search.properties).sort()).toEqual(["query", "type"]);
expect(search.properties.query).toEqual({
oneOf: [{ type: "string" }, { type: "array", items: { type: "string" } }],
oneOf: [
{ type: "string" },
{ type: "number" },
{ type: "boolean" },
{ type: "array", items: { type: "string" } },
],
description:
"Query parameter query; optional. Supply one string for one occurrence or an array to preserve repeated values in captured order; repeated values replay their recorded request.",
"Query parameter query; optional. Supply one value for one occurrence, a number or boolean sent as its JSON text, or an array of strings to preserve repeated values in captured order; repeated values replay their recorded request.",
});
expect("body" in search.properties).toBe(false);
});
@@ -522,10 +527,12 @@ describe("environmentToolInputSchema argument names", () => {
]);
expect(Object.hasOwn(schema.properties, "constructor")).toBe(true);
expect(schema.properties.query_id?.description).toBe(
"Query parameter id; optional. Supply one string for one occurrence or an array to preserve repeated values in captured order; repeated values replay their recorded request.",
"Query parameter id; optional. Supply one value for one occurrence, a number or boolean sent as its JSON text, or an array of strings to preserve repeated values in captured order; repeated values replay their recorded request.",
);
expect(schema.properties.body?.oneOf).toEqual([
{ type: "string" },
{ type: "number" },
{ type: "boolean" },
{ type: "array", items: { type: "string" } },
]);
expect(schema.properties.request_body?.type).toBe("object");
@@ -32,7 +32,7 @@ export const REPLAY_CATALOG_INSTRUCTIONS =
/** JSON Schema advertised for one tool argument. */
export interface EnvironmentToolProperty extends EnvironmentToolSchema {
readonly oneOf?: readonly (
| { readonly type: "string" }
| { readonly type: "string" | "number" | "boolean" }
| { readonly type: "array"; readonly items: { readonly type: "string" } }
)[];
readonly description: string;
@@ -325,6 +325,21 @@ export const hasRepeatedQueryArgument = (
return false;
};
/** The arguments with a number or boolean given for a query parameter spelled as its JSON text, as a URL carries it. */
export const withQueryText = (
tool: EnvironmentManifestTool,
args: Readonly<Record<string, unknown>>,
): Readonly<Record<string, unknown>> => {
let spelled: Record<string, unknown> | undefined;
for (const argument of environmentToolArguments(tool).query.values()) {
const value = ownMember(args, argument);
if (typeof value !== "number" && typeof value !== "boolean") continue;
spelled ??= { ...args };
spelled[argument] = JSON.stringify(value);
}
return spelled ?? args;
};
/** Lazily render properties so admission precedes retaining a whole schema. */
function* environmentToolProperties(
tool: EnvironmentManifestTool,
@@ -351,9 +366,11 @@ function* environmentToolProperties(
? {
oneOf: [
{ type: "string" },
{ type: "number" },
{ type: "boolean" },
{ type: "array", items: { type: "string" } },
],
description: `Query parameter ${describeText(name, "query name")}; optional. Supply one string for one occurrence or an array to preserve repeated values in captured order; repeated values replay their recorded request.`,
description: `Query parameter ${describeText(name, "query name")}; optional. Supply one value for one occurrence, a number or boolean sent as its JSON text, or an array of strings to preserve repeated values in captured order; repeated values replay their recorded request.`,
}
: {
type: "string",
@@ -247,6 +247,7 @@ export class EnvironmentStateWriteSimulation {
calls: 0,
updates: [],
sequences: new Map(),
starts: new Map(),
steady: new Set(),
unstamped: new Set(),
kept: new Set(),
@@ -0,0 +1,86 @@
import { describe, expect, it } from "vitest";
import {
timedCaptureEvidence,
type TimedCaptureRow,
} from "../../../tests/fixtures/analysisView.js";
import {
BULK_TASK_ROWS,
COUNTED_TASK_ROWS,
ONE_TASK_ROWS,
} from "../../../tests/fixtures/environmentStateMadeValues.js";
import { environmentStateViewOf } from "../../../tests/fixtures/environmentStateWrites.js";
import { respondedIds } from "./environmentStateWriteLinks.js";
import { createdMadeObject } from "./environmentStateWriteStamps.js";
const createOf = (rows: readonly TimedCaptureRow[]) => {
const plan = environmentStateViewOf(timedCaptureEvidence(rows)).writes.find(
(candidate) => candidate.effect === "create",
);
if (plan === undefined) throw new Error("No create plan");
return plan;
};
const ruleFor = (plan: ReturnType<typeof createOf>, field: string): unknown =>
plan.rules.find((rule) => rule.field === field);
describe("values an application counts and makes with each created entity", () => {
it("learns a counter from where the first create stood, a template around it and the object made with each task", () => {
const plan = createOf(COUNTED_TASK_ROWS);
expect(plan).toMatchObject({
served: true,
checked_record_ordinals: [4, 7],
});
expect(ruleFor(plan, "index")).toEqual({
field: "index",
rule: "sequence",
scope: "collection",
start: 15,
});
expect(ruleFor(plan, "identifier")).toEqual({
field: "identifier",
rule: "template",
from: "index",
});
expect(ruleFor(plan, "subscription")).toMatchObject({
rule: "made-object",
clocks: ["created"],
own: ["entity_id"],
counts: [{ member: "id", start: 21 }],
});
});
it("keeps a value one create alone shows past what the type held", () => {
const plan = createOf(ONE_TASK_ROWS);
expect(ruleFor(plan, "index")).toBeUndefined();
expect(ruleFor(plan, "subscription")).toMatchObject({
rule: "made-object",
counts: [],
});
});
it("links a create whose answer wraps the entity in a one-element array member", () => {
expect(createOf(BULK_TASK_ROWS)).toMatchObject({
tool: "post_projects_tasks_bulk_by_id",
served: true,
body: { envelope: "tasks" },
});
expect(respondedIds({ tasks: [{ id: 14 }] })).toEqual(["14"]);
expect(respondedIds({ tasks: [{ id: 14 }, { id: 15 }] })).toEqual([]);
});
it("makes no object rule without a clock the request stamped", () => {
expect(
createdMadeObject(
"subscription",
{ id: 21, entity: "task", entity_id: 20 },
{
startMs: Date.parse("2026-10-17T18:00:02Z"),
type: [],
entity: { id: 20 },
idField: "id",
},
),
).toBeUndefined();
});
});
@@ -238,6 +238,7 @@ const fieldRuleSchema = z.discriminatedUnion("rule", [
...ruled,
rule: z.literal("sequence"),
scope: z.enum(["collection", "type"]),
start: z.number().int().exactOptional(),
}),
z.strictObject({ ...ruled, rule: z.literal("unique") }),
z.strictObject({
@@ -256,6 +257,16 @@ const fieldRuleSchema = z.discriminatedUnion("rule", [
members: z.array(z.string()).min(1),
format: clockFormatSchema,
}),
z.strictObject({
...ruled,
rule: z.literal("made-object"),
clocks: z.array(z.string()).min(1),
format: clockFormatSchema,
own: z.array(z.string()),
counts: z.array(
z.strictObject({ member: z.string(), start: z.number().int() }),
),
}),
z.strictObject({
...ruled,
rule: z.literal("held-clock"),
@@ -51,6 +51,7 @@ const generator = () => {
clock: NOW,
"nested-clock": { at: NOW },
"held-clock": NOW,
"made-object": { at: NOW },
};
const generate = (rule: GeneratedRule) => {
asked.push(rule);
@@ -236,6 +236,7 @@ const ruledValue = (
case "clock":
case "nested-clock":
case "held-clock":
case "made-object":
return generate(rule);
case "echo":
return ok(
@@ -377,10 +378,36 @@ export const createdFields = (
else if (rule.rule === "echo" || rule.rule === "reference")
delete written[rule.field];
}
ownMadeMembers(plan.rules, written);
const unspelled = derive(plan.rules, observed, written, call.body ?? {});
return unspelled === undefined ? ok(written) : err(unspelled);
};
/** Set each made object's members that hold the entity's identity to the identity it was written with. */
const ownMadeMembers = (
rules: readonly FieldRule[],
written: JsonObject,
): void => {
const identity = rules.find((rule) => rule.rule === "identity")?.field;
const id = identity === undefined ? undefined : ownMember(written, identity);
if (id === undefined) return;
for (const rule of rules) {
if (rule.rule !== "made-object" || rule.own.length === 0) continue;
const made = ownMember(written, rule.field);
if (made === undefined || !isObject(made)) continue;
const owned: JsonObject = { ...made };
for (const member of rule.own)
setOwnMember(owned, member, spelledIdentity(ownMember(made, member), id));
setOwnMember(written, rule.field, owned);
}
};
/** An identity spelled as a recorded member spells it: a string, or a number when the identity is one. */
const spelledIdentity = (
exemplar: JsonValue | undefined,
id: JsonValue,
): JsonValue => (typeof exemplar === "string" ? (scalarText(id) ?? id) : id);
/** Rules an update gives a new value on every call. */
const REGENERATED: ReadonlySet<FieldRule["rule"]> = new Set([
"clock",
@@ -20,7 +20,9 @@ import {
} from "./environmentStateWriteApply.js";
import {
createdToken,
heldObjects,
isOpaqueToken,
nextMadeCount,
restampedFields,
} from "./environmentStateWriteStamps.js";
import {
@@ -84,6 +86,8 @@ import {
isUuid,
likeGeneratedInteger,
nextSequence,
readClock,
startedSequence,
sequenceNumber,
spelledLike,
} from "./environmentStateWriteValues.js";
@@ -283,7 +287,11 @@ const refineCreate = (
: { envelope: evidence.envelope }),
},
},
{ kept: draft.kept, clockOf: (field) => clockEvidence(draft, field) },
{
kept: draft.kept,
clockOf: (field) => clockEvidence(draft, field),
startOf: (field) => draft.starts.get(field),
},
);
if (merged === undefined) return false;
retainFixed(context, merged.body.fixed.slice(draft.plan.body!.fixed.length));
@@ -398,6 +406,13 @@ const buildCreate = (
});
draft.observed = evidence.observed.object;
for (const entity of held) noteFixedWidths(draft.fixedWidth, entity);
const ruled = new Set(explained.rules.map((rule) => rule.field));
for (const [field, value] of Object.entries(evidence.observed.object)) {
const integer = sequenceNumber(value);
if (ruled.has(field) || integer === undefined) continue;
const next = nextInteger(held, field, value);
if (next === undefined || integer >= next) draft.starts.set(field, integer);
}
return true;
};
@@ -437,10 +452,13 @@ const followsSequences = (
rule.scope === "type" || entity.collection === created.collection,
);
const next = nextSequence(
nextInteger(
fieldsOf(scope),
rule.field,
ownMember(draft.observed, rule.field),
startedSequence(
nextInteger(
fieldsOf(scope),
rule.field,
ownMember(draft.observed, rule.field),
),
rule.start,
),
draft.sequences.get(key),
);
@@ -538,7 +556,52 @@ const followsCreateRules = (
return (
rule.rule !== "identity" || rule.generator !== "uuid" || isUuid(value)
);
}) && followsSequences(context, draft, created);
}) &&
followsSequences(context, draft, created) &&
followsMadeObjects(context, draft, created);
/** The key a made object's count is taken under in a plan's sequences. */
const madeKey = (field: string, member: string): string => `${field}/${member}`;
/** Whether a later call's made objects hold what serving makes: the exemplar's members, the entity's identity, clocks, and each count one past what held objects show or started with, past the plan's last. */
const followsMadeObjects = (
context: EffectContext,
draft: WriteDraft,
created: Pick<StateEntity, "type" | "collection" | "fields">,
): boolean =>
draft.plan.rules.every((rule) => {
if (rule.rule !== "made-object") return true;
const made = ownMember(created.fields, rule.field);
const exemplar = ownMember(draft.observed, rule.field);
if (!isObject(made) || !isObject(exemplar)) return false;
const id = scalarText(
ownMember(created.fields, context.idField(created.type)),
);
const held = heldObjects(
[
...context.state.ofType(created.type),
...context.startedWith(created.type),
].map((entity) => entity.fields),
rule.field,
);
return [...new Set([...Object.keys(exemplar), ...Object.keys(made)])].every(
(member) => {
const value = ownMember(made, member);
if (rule.clocks.includes(member)) return readClock(value) !== undefined;
if (rule.own.includes(member)) return scalarText(value) === id;
const count = rule.counts.find((entry) => entry.member === member);
if (count === undefined)
return same(value, ownMember(exemplar, member));
return (
sequenceNumber(value) ===
nextSequence(
nextMadeCount(held, count, ownMember(exemplar, member)),
draft.sequences.get(madeKey(rule.field, member)),
)
);
},
);
});
/** Widen the plan's clock formats to the trimmed spelling a later call showed, and its other clocks with them unless a value the plan saw shows them at a fixed width. */
const fitClocks = (draft: WriteDraft, fields: JsonObject): void => {
@@ -574,6 +637,17 @@ const takeSequences = (
);
if (value !== undefined) draft.sequences.set(key, value);
}
for (const rule of draft.plan.rules) {
if (rule.rule !== "made-object") continue;
const made = ownMember(created.fields, rule.field);
for (const { member } of rule.counts) {
const value = isObject(made)
? sequenceNumber(ownMember(made, member))
: undefined;
if (value !== undefined)
draft.sequences.set(madeKey(rule.field, member), value);
}
}
};
/** Segments of a recorded path as a create call supplies them: template literals, and each parameter's recorded spelling. */
@@ -1,13 +1,24 @@
import type { JsonValue } from "../jsonValue.js";
import { ownMember, same, type JsonObject } from "./environmentStateRules.js";
import {
isObject,
ownMember,
same,
scalarText,
type JsonObject,
} from "./environmentStateRules.js";
import { storedKinds } from "./environmentStateWriteApply.js";
import { fittedClockFormat } from "./environmentStateWriteClockFormats.js";
import { composedValue } from "./environmentStateWriteComposed.js";
import {
alikeRules,
isBlankValue,
type FieldRule,
type WriteBody,
} from "./environmentStateWriteRules.js";
import {
containsDelimited,
sequenceNumber,
} from "./environmentStateWriteValues.js";
type Default = { readonly member: string; readonly value: JsonValue };
type Blank = NonNullable<WriteBody["blank"]>[number];
@@ -16,20 +27,25 @@ type Blank = NonNullable<WriteBody["blank"]>[number];
export interface EarlierEvidence {
readonly kept: ReadonlySet<string>;
readonly clockOf: (field: string) => JsonValue | undefined;
/** The integer the exemplar showed in an unruled field above every value its type held, where a sequence may start. */
readonly startOf?: (field: string) => number | undefined;
}
/** The plan's rule for a field and a later call's agree: equal, or clocks one spelling covers given what earlier calls showed. */
/** The plan's rule for a field and a later call's agree: equal, clocks one spelling covers given what earlier calls showed, or made objects alike but in the counts only the later call could show. */
const agreeing = (
held: FieldRule,
later: FieldRule,
values: {
readonly field: string;
readonly later: JsonValue | undefined;
readonly exemplar: JsonValue | undefined;
readonly earlier: EarlierEvidence;
},
): FieldRule | undefined => {
const alike = alikeRules(held, later, values.earlier.kept);
if (alike !== undefined) return alike;
if (held.rule === "made-object" && later.rule === "made-object")
return madeAlike(held, later, values.exemplar);
if (held.rule !== "clock" || later.rule !== "clock") return undefined;
const format = fittedClockFormat(
held.format,
@@ -39,6 +55,37 @@ const agreeing = (
return format === undefined ? undefined : { ...held, format };
};
type MadeObject = Extract<FieldRule, { rule: "made-object" }>;
/** Two made-object rules as one: the same clocks, format and own members, and the earlier counts among the later's; a count only the later call shows starts at the exemplar's value, which no earlier object could show past. */
const madeAlike = (
held: MadeObject,
later: MadeObject,
exemplar: JsonValue | undefined,
): MadeObject | undefined => {
if (
!same(held.clocks, later.clocks) ||
!same(held.own, later.own) ||
!same(held.format, later.format) ||
held.counts.some(
({ member }) => !later.counts.some((count) => count.member === member),
)
)
return undefined;
const counts: MadeObject["counts"] = [];
for (const { member } of later.counts) {
const kept = held.counts.find((count) => count.member === member);
const start =
kept?.start ??
sequenceNumber(
isObject(exemplar) ? ownMember(exemplar, member) : undefined,
);
if (start === undefined) return undefined;
counts.push({ member, start });
}
return { ...held, counts };
};
/** Defaults joined, undefined when two calls showed one member left out at different values. */
const joinedDefaults = (
held: readonly Default[],
@@ -53,6 +100,33 @@ const joinedDefaults = (
return [...joined].map(([member, value]) => ({ member, value }));
};
/** A later call's rule for a field the plan kept as shown, when it spells the exemplar's value too: a string composed from the exemplar's fields or holding the value it templates, or a sequence that started at its value. */
const spellsExemplar = (
field: string,
rule: FieldRule,
calls: Pick<FieldCalls, "exemplar" | "earlier">,
): FieldRule | undefined => {
const value = ownMember(calls.exemplar, field);
if (rule.rule === "composed")
return value !== undefined &&
composedValue(rule.parts, calls.exemplar) === value
? rule
: undefined;
if (rule.rule === "template") {
const token = scalarText(ownMember(calls.exemplar, rule.from));
return typeof value === "string" &&
token !== undefined &&
containsDelimited(value, token)
? rule
: undefined;
}
if (rule.rule !== "sequence") return undefined;
const start = calls.earlier.startOf?.(field);
return start !== undefined && sequenceNumber(value) === start
? { ...rule, start }
: undefined;
};
/** What two calls show of one field. */
interface FieldCalls {
readonly before: FieldRule | undefined;
@@ -81,6 +155,7 @@ const addedReference = (
: agreeing(added, after, {
field,
later: later.entity[field],
exemplar: ownMember(calls.exemplar, field),
earlier: calls.earlier,
});
return rule === undefined ? undefined : { rule };
@@ -114,6 +189,7 @@ const mergedField = (
const rule = agreeing(before, after, {
field,
later: later.entity[field],
exemplar: ownMember(exemplar, field),
earlier: calls.earlier,
});
return rule === undefined ? undefined : { rule };
@@ -126,6 +202,10 @@ const mergedField = (
default: { member: field, value: later.entity[field]! },
}
: undefined;
if (before === undefined && after !== undefined) {
const rule = spellsExemplar(field, after, calls);
if (rule !== undefined) return { rule };
}
// This call echoes a member the plan's calls left out: what they stored there is its default.
if (after !== undefined)
return after.rule === "echo" && !calls.required.includes(field)
@@ -88,6 +88,8 @@ export interface WriteDraft {
readonly updates: UpdateCall[];
/** The last value each sequence took, by field and, for a collection's sequence, collection; the next call must follow it. */
readonly sequences: Map<string, number>;
/** Integers the exemplar showed in fields no rule wrote, above every value its type held, where a later call's sequence may start. */
readonly starts: Map<string, number>;
/** The collection a create writes into when its own path names no state type: the one its returned id was read under. */
readonly linked?: string | undefined;
/** Fields an item action's call found already at the value it sets them to. */
@@ -51,6 +51,7 @@ import {
GENERATED,
} from "./environmentStateWriteGenerated.js";
import {
createdMadeObject,
createdNestedClock,
createdToken,
isClockNear,
@@ -263,7 +264,10 @@ const generatedRule = (
return { field, rule: "unique" };
if (createdToken(field, evidence.entity, evidence.type) === "independent")
return { field, rule: "unique" };
return createdNestedClock(field, value, evidence);
return (
createdNestedClock(field, value, evidence) ??
createdMadeObject(field, value, evidence)
);
};
/** Pinned members less those a reference reads: it rebuilds the object they name for any held entity. */
@@ -44,9 +44,14 @@ const identityValues = (object: JsonValue | undefined): string[] =>
})
: [];
/** The one object a one-element array holds, else the value itself. */
const unwrapped = (value: JsonValue): JsonValue =>
Array.isArray(value) && value.length === 1 ? value[0]! : value;
/**
* The ids a write's response names for an entity it made: an identity-named member of the body or of
* an object member (an envelope) at any length, or a scalar body or sole scalar member long enough to name one.
* an object member (an envelope, or the one object of a one-element array member) at any length, or a
* scalar body or sole scalar member long enough to name one.
*/
export const respondedIds = (body: JsonValue | undefined): string[] => {
if (body === undefined || body === null) return [];
@@ -60,7 +65,7 @@ export const respondedIds = (body: JsonValue | undefined): string[] => {
...new Set([
...identityValues(body),
...(members.length === 1 ? named(members[0]) : []),
...members.flatMap(identityValues),
...members.flatMap((member) => identityValues(unwrapped(member))),
]),
];
};
@@ -41,10 +41,12 @@ export type GeneratedRule =
readonly rule: "identity";
readonly generator: "sequence" | "uuid" | "token";
}
/** One past the largest value its scope holds, and never below `start`, a value the first create took above them. */
| {
readonly field: string;
readonly rule: "sequence";
readonly scope: "collection" | "type";
readonly start?: number;
}
| { readonly field: string; readonly rule: "unique" }
/** The value kept with each UUID at the pointers (`*` for any array index) replaced by a fresh one, as rewritten child rows are. */
@@ -70,6 +72,15 @@ export type GeneratedRule =
readonly field: string;
readonly rule: "held-clock";
readonly format: ClockFormat;
}
/** An object made with the entity, which no initial entity holds: `clocks` take the write's clock, `own` the entity's identity, each of `counts` one past the largest value held objects show and never below its start; other members as the exemplar showed them. */
| {
readonly field: string;
readonly rule: "made-object";
readonly clocks: string[];
readonly format: ClockFormat;
readonly own: string[];
readonly counts: { readonly member: string; readonly start: number }[];
};
/** How one field of a written entity gets its value; fields no rule names keep the observed value. */
@@ -20,6 +20,8 @@ import {
containsDelimited,
isUuid,
readClock,
sequenceNumber,
spelledLike,
} from "./environmentStateWriteValues.js";
/** A clock value counts as written by the request when it lies this close to the request's start. */
@@ -243,6 +245,131 @@ export const createdNestedClock = (
return stamped ? nestedRule(field, value, members, evidence.type) : undefined;
};
/** The largest integer the objects hold at a member, spelled as the exemplar spells it. */
const largestAt = (
objects: readonly JsonObject[],
member: string,
exemplar: JsonValue | undefined,
): number | undefined => {
let largest: number | undefined;
for (const object of objects) {
const value = sequenceNumber(ownMember(object, member), exemplar);
if (value !== undefined && (largest === undefined || value > largest))
largest = value;
}
return largest;
};
/** The objects held entities keep at a field. */
export const heldObjects = (
entities: readonly JsonObject[],
field: string,
): JsonObject[] =>
entities.flatMap((entity) => {
const value = ownMember(entity, field);
return value !== undefined && isObject(value) ? [value] : [];
});
/** One past the largest integer held objects show at a member, never below a start. */
export const nextMadeCount = (
objects: readonly JsonObject[],
count: { readonly member: string; readonly start: number },
exemplar: JsonValue | undefined,
): number => {
const largest = largestAt(objects, count.member, exemplar);
return Math.max(
largest === undefined ? count.start : largest + 1,
count.start,
);
};
/**
* On a create, an object the application made with the entity: members holding a clock the request stamped, others the
* entity's own identity or one past the largest integer held objects show there, and the rest as every held object
* shows them. Undefined without a stamped clock, since only that shows the request made the object.
*/
export const createdMadeObject = (
field: string,
value: JsonValue,
evidence: {
readonly startMs: number | undefined;
readonly type: readonly JsonObject[];
readonly entity: JsonObject;
readonly idField: string;
},
): FieldRule | undefined => {
const { startMs } = evidence;
if (!isObject(value) || startMs === undefined) return undefined;
const keys = Object.keys(value);
const clocks = keys.filter((key) => {
const ms = readClock(value[key])?.ms;
return (
ms !== undefined &&
ms >= startMs - TRUNCATION_MS &&
ms <= startMs + NESTED_WINDOW_MS
);
});
const format =
clocks.length === 0 ? undefined : readClock(value[clocks[0]!])!.format;
if (
format === undefined ||
clocks.some((key) => !same(readClock(value[key])!.format, format))
)
return undefined;
const id = scalarText(ownMember(evidence.entity, evidence.idField));
const own = keys.filter(
(key) =>
!clocks.includes(key) &&
id !== undefined &&
scalarText(value[key]) === id,
);
const held = heldObjects(evidence.type, field);
const counts = keys.flatMap((key) => {
if (clocks.includes(key) || own.includes(key)) return [];
const integer = sequenceNumber(value[key]);
const largest = largestAt(held, key, value[key]);
return integer !== undefined &&
largest !== undefined &&
integer === largest + 1
? [{ member: key, start: integer }]
: [];
});
const varying = new Set([
...clocks,
...own,
...counts.map(({ member }) => member),
]);
const alike = held.every((object) =>
[...new Set([...Object.keys(object), ...keys])].every(
(key) =>
varying.has(key) || same(ownMember(object, key), ownMember(value, key)),
),
);
return alike
? { field, rule: "made-object", clocks, format, own, counts }
: undefined;
};
/** A served made object: the exemplar's, its clocks set to the write's clock and each count to the value `next` gives; its own members follow the entity's identity once written. */
export const madeObjectValue = (
base: JsonValue | undefined,
rule: Extract<FieldRule, { rule: "made-object" }>,
clock: JsonValue,
next: (count: { readonly member: string; readonly start: number }) => number,
): Result<JsonValue, string> => {
if (base === undefined || !isObject(base))
return err(`The plan holds no ${rule.field} object to make.`);
const made: JsonObject = { ...base };
for (const member of rule.clocks) setOwnMember(made, member, clock);
for (const count of rule.counts)
setOwnMember(
made,
count.member,
spelledLike(ownMember(base, count.member) ?? "", next(count)),
);
return ok(made);
};
/** A served nested clock: the object the entity holds, or the observed one for a create, with its stamped members set to the clock. */
export const nestedClockValue = (
base: JsonValue | undefined,
@@ -249,6 +249,13 @@ export const nextSequence = (
): number =>
taken === undefined ? (held ?? 1) : Math.max(held ?? taken + 1, taken + 1);
/** The next held integer of a sequence that never goes below its `start`. */
export const startedSequence = (
held: number | undefined,
start: number | undefined,
): number | undefined =>
start === undefined ? held : Math.max(held ?? start, start);
/** An integer spelled as the recorded value spells integers: a number, or a string of digits. */
export const spelledLike = (exemplar: JsonValue, value: number): JsonValue =>
maskedCount(exemplar) !== undefined
+1 -1
View File
@@ -6,5 +6,5 @@ export const PACKAGE_METADATA = {
serverSdkVersion: "2.3.1",
clientSdkVersion: "2.3.1",
coreSdkVersion: "2.3.1",
skillVersion: "51",
skillVersion: "52",
} as const;
+38 -2
View File
@@ -23,9 +23,9 @@ import {
const HTTP_BOUNDARY_TIMEOUT_MS = 90_000;
const CARD_PATH = "/.well-known/agent-env.json";
/** SHA-256 of the 13,882-byte card served for the fixture named "renamed", its descriptions naming each tool's recorded responses; pure replay must serve these bytes. */
/** SHA-256 of the 14,519-byte card served for the fixture named "renamed", its descriptions naming each tool's recorded responses; pure replay must serve these bytes. */
const REPLAY_CARD_SHA256 =
"27370fe0a89556dcd0dd1bc6ddd0439d77b628349829afd56e4af4ee393d0a64";
"995c280f0798f77e76f206c068eb6c2c1f8c4210a03ac1c01fba8cc07b5d7f09";
/** Serve the fixture pair as pure replay through the built CLI. */
const serveReplayOnly = async (
@@ -58,6 +58,40 @@ const connect = async (url: URL) => {
return client;
};
processTest(
"accepts numbers and booleans for query parameters, sent as their JSON text",
async ({ processes }) => {
const fixture = environmentStateServeFixture(
timedCaptureEvidence([
[
"GET",
"/api/contacts?maxSize=2&withItems=true",
[{ id: 1, name: "Ada" }],
],
]),
);
const started = await serveReplayOnly(processes, {}, fixture);
const client = await connect(started.url);
try {
const response = await client.callTool({
name: "get_contacts",
arguments: { maxSize: 2, withItems: true },
});
expect(response.isError).toBeFalsy();
expect(response.structuredContent).toMatchObject({
body: [{ id: 1, name: "Ada" }],
replay: {
exact_match: true,
recorded_query: "?maxSize=2&withItems=true",
},
});
} finally {
await client.close();
}
},
HTTP_BOUNDARY_TIMEOUT_MS,
);
processTest(
"replays recorded own JSON keys without interpreting inherited members as fields",
async ({ processes }) => {
@@ -113,6 +147,8 @@ processTest(
).toMatchObject({
oneOf: [
{ type: "string" },
{ type: "number" },
{ type: "boolean" },
{ type: "array", items: { type: "string" } },
],
});
+83
View File
@@ -0,0 +1,83 @@
import type { JsonObject } from "../../src/domain/analysisView/environmentStateRules.js";
import type { TimedCaptureRow } from "./analysisView.js";
import { bookClock } from "./environmentStateWrites.js";
const seed = (id: number, index: number): JsonObject => ({
id,
title: `Seeded task ${String(id)}`,
project_id: 5,
index,
identifier: `#${String(index)}`,
subscription: null,
created: "2026-10-01T09:00:00Z",
});
const SEEDS = [seed(1, 1), seed(2, 2)];
/** A task the application made at row `row`: numbered past tasks deleted before the capture, its subscription not loaded yet. */
const made = (
id: number,
index: number,
title: string,
row: number,
): JsonObject => ({
id,
title,
project_id: 5,
index,
identifier: `#${String(index)}`,
subscription: null,
created: bookClock(row),
});
/** The task as a later read shows it, with the subscription the application made for its creator. */
const subscribed = (
task: JsonObject,
subscription: number,
row: number,
): JsonObject => ({
...task,
subscription: {
id: subscription,
entity: "task",
entity_id: task["id"]!,
created: bookClock(row),
},
});
const FIRST = made(20, 15, "Replace the lens gasket", 2);
const SECOND = made(21, 16, "Log the fog-signal tests", 5);
/** Two task creates at a project path, each read back with a made subscription, then the list. */
export const COUNTED_TASK_ROWS: readonly TimedCaptureRow[] = [
["GET", "/api/tasks", SEEDS],
["GET", "/api/tasks/1", SEEDS[0]!],
["POST", "/api/projects/5/tasks", FIRST, 201, { title: FIRST["title"]! }],
["GET", "/api/tasks/20", subscribed(FIRST, 21, 2)],
["GET", "/api/tasks", [...SEEDS, subscribed(FIRST, 21, 2)]],
["POST", "/api/projects/5/tasks", SECOND, 201, { title: SECOND["title"]! }],
["GET", "/api/tasks/21", subscribed(SECOND, 22, 5)],
[
"GET",
"/api/tasks",
[...SEEDS, subscribed(FIRST, 21, 2), subscribed(SECOND, 22, 5)],
],
];
/** As COUNTED_TASK_ROWS, through a bulk path whose body and answer wrap the one task in `tasks`. */
export const BULK_TASK_ROWS: readonly TimedCaptureRow[] = COUNTED_TASK_ROWS.map(
(row) =>
row[0] === "POST"
? [
"POST",
`${row[1]}/bulk`,
{ tasks: [row[2]!] },
201,
{ tasks: [row[4]!] },
]
: row,
);
/** COUNTED_TASK_ROWS up to the first create's reads: one create alone shows no counter. */
export const ONE_TASK_ROWS: readonly TimedCaptureRow[] =
COUNTED_TASK_ROWS.slice(0, 5);