reverse-engineering-anything/docs/specs/agent-environments/state-view/spec.md

Environment state view from captures

Issue: https://github.com/morluto/rea/issues/4933

Outcome

inspect_analysis_view accepts view: {"kind": "environment-state"} for the parents the environment manifest accepts and extracts the initial state the capture's reads observed: entity types with an identity field, entities with their recorded values, documents for written paths whose reads are not entities, and a render plan per read tool that reproduces the tool's latest initial recordings from the state, or a recorded view with the reason it cannot. This increment adds the view kind only; serving from the state, loading another state through the AgentEnv data plane and modelling write effects are later increments.

Requirements

• Accept inspect_web_network_capture HAR parents and capture_browser_scenario parents that selected response bodies, with the manifest's admission: the same format, subject and sensitive-value refusals.
• Group records exactly as the manifest does for the same template_rules, so tool names match the manifest the state accompanies.
• Order primary-origin records by recorded start time, or by record order when any start time is missing, and say which. Bound each type's state by the first write under its root (API base, version segments, one segment); a query POST is not a write, a read overlapping the write is left out, and a write whose URL was redacted bounds every type, with a limitation counting it. A successful POST elsewhere bounds a type's state too when its response named an id (an identity member, or a bare or sole scalar of four characters or more) that the type's reads show only once it began: the POST made that entity, so it is not in the initial state. A read finished before the POST began shows what was there, and one still running may show its effect. A list read shows the ids its listed entities carry as their own identity members, an item read its path's id only when its body shows the item (a JSON null shows none), and across two or more of them, when no id-named field (id, uuid, …; not key, slug or name) identifies them, the first suffix-named field (projectId) distinct on each, so a foreign key beside an id is none of theirs, nor are the ids of entities they nest. An id shorter than four characters counts only when the type was read before the POST and an item read names it after.
• Prove an item type's identity field from the requested path value its responses repeat; otherwise infer it from a fixed list of identity names that every listed item carries distinctly, which needs at least one listed item. An item read that asked for some fields only (fields, fields[...], select, $select) and does not return the item is left out of the proof and of the view's checks; the view lists it in projected_record_ordinals. Beside a line showing whole entities, a list line asking for some fields only (a projection value other than *, which asks for all) is listed there too, and proves no identity or membership. So is an item line first recorded after the type's initial reads, judged against the lines left once variants are dropped: one that names a shaping parameter none of them named while its object carries or lacks members theirs do not, such as a later ?expand=shelf read, or one that selects some fields and leaves out members they always show, such as a later ?fields=id,name read. On a view with such replayed reads, serving replays a call naming a projection parameter for an entity the state holds unless every value it gives is * or one a checked line gave, and no replayed read gave it. The write gate leaves out exactly the successful reads serving replays, and any read that names one and leaves out a field its view requires, which serving would answer whole.
• A request line of an item or list view is a representation variant when it shows an entity another line shows with other members, or names a parameter the base lines do not (paging aside) while its objects carry a member no base object carries or lack one every base object carries. Base lines are taken first: lines naming no such parameter, then lines asking for every field, then the line most others agree with. A variant leaves the view's checks, the entity observations and the membership changes that would end the type's initial reads when a query key tells it apart: a parameter only variants name, a value no base line gives a parameter, or a parameter every base line names that a variant leaves out. The view lists its latest recordings in projected_record_ordinals and the keys in projected_query_keys; serving replays a call matching a key with the reason variant (projection when it leaves out a projection parameter every base line names; a projection value is tested as above), so the answer is a recording and shows no served write. The write gate neither checks nor learns from a successful read serving replays; a 400, 404 or 410 such a read answers is still the absence it checks.
• A field every recording of a view carried is required: serving refuses an entity lacking it. Once a successful read the write gate checks, naming no projection parameter, shows the application leaves the field out of an entity the state holds (APIs that omit empty or false members), no view of the type requires it. Never treat an envelope that is itself an entity as a list.
• A top-level UUID or clock member that differs on every repeat of a path's successful GET lines with no write between, and across the capture differed on at least two repeats and never stayed the same, is minted per response there (a request id). A view names it in per_response_members only when every concrete path the view answers independently proves it minted. A collection's proof does not establish that its item paths mint the same member. The shared envelope compares without these common members, and serving keeps their recorded values; an unproved envelope difference stays recorded rather than borrowing another path's value.
• Merge observations into one entity per collection and identity: item views' latest values win over list views', differing views keep their own values, and a value that changed between repeats of one request line with no recorded write is volatile and keeps the latest; a change across a write under another root, including between two tools of the type, ends the type's initial reads at the earliest write within the change, and the state is extracted again once; a type whose reads still disagree keeps every tool recorded. Compare list membership, including empty lists, and field presence before selecting each request line's latest recording; a change across a write ends the type's initial reads the same way. Different query strings remain distinct; omitted fields do not erase their earlier observations on other request lines.
• An item or list view whose recordings always spell a clock field's instant otherwise than the entity's stored value (Z against +00:00, another separator or fraction width) names that spelling in clock_formats; a field it also shows as stored, or spells two ways (counting values at another instant), gets none. A stored value the view never recorded, such as one a write made, is rendered in that spelling: its wall clock in the view's zone, under the view's separator, with its own fraction digits padded with zeros to the view's width and none dropped; an epoch spelling is computed from the instant.
• An item or list view names in sorted_arrays the fields it lists as string arrays in code-point order whatever order they were written in (tags kept as a set). Only the write gate learns it. A field is a candidate when every initial entity shows it as a sorted string array or null, one with two distinct strings, and no read the gate counts has listed it unsorted with two distinct strings. It is learned from a counted read that fails with the stored order and passes with the field sorted, and only when that read lists a written array in an order that is neither the written one nor values the initial state held first, then the new ones as written (an application ordering by tag id or creation shows that order too). The stored value keeps the written order, so a write's own answer echoes it; every view of the type for which the field is still a candidate sorts it as well.
• A collection's initial order is the order of the first line that lists every entity any initial read shows in it and names no sort, direction (sort[], _sort, sort-column, sortKey, orderBy and like spellings) or random parameter (random, rand, shuffle, seed); a collection without such a line keeps its entities' earliest sightings.
• A list has all membership only when no item-total envelope member (total, count and their compounds, not page counts), at the top level or in a top-level object beside the list such as pageInfo.totalRows, differs from the number of entities it lists and no has_more, next or next-page token is set. Such a member that equals the number listed is a counter; a nested one is named by its pointer (/pageInfo/totalRows), and members beside it such as isLastPage stay as recorded.
• Keep an item or list plan only when it reproduces each request line's latest initial recording from the state, comparing JSON values without regard to member order; otherwise record the tool with a reason. Parse bodies only when JSON-like, UTF-8, at most 256 KiB and exactly representable. Preserve all own JSON members, including __proto__, through extraction and normalization.
• Mark the Evidence inferred; state unknowns for authorization, response semantics and write effects.
• Charge the manifest's admission and grouping, every record line, every body as temporary bytes before decoding and parsing, and every kept value, within the existing 96 MiB budget. A planning pass that a rewrite cut discards releases its charges before the state is extracted again.
• Read response bodies through one domain owner shared with replay, and parse exact JSON through the owner of the exact-number policy.

Acceptance and limits

The Grafana-shaped HAR fixture with distinct start times drives domain, service, MCP, CLI and acceptance coverage: folders merge their detail and list views under a proved uid, the dashboard entity is read from its /dashboard wrapper, and the annotation list after the annotation write has no initial recording. Synthetic captures cover write bounds, volatile fields, complete lists with counters, each recorded reason and documents; the scenario fixture covers record order and the response-body requirement. The view performs no network activity and serves nothing. Values copied outside entity objects, derived values and cross-type effects are not tied to the state.

Deferred issue criteria

Serving tool calls from the state, data/reset, data/add and data/get on the environment data plane, exporting the state with a bundle, and write effects are later increments of #4933. So are documents for item-shaped written paths (one per identity rather than per template) and for paths that are only deleted; until then their reads stay recorded. This PR must reference rather than close the issue.