OpenSource/reverse-engineering-anything

0 Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-10-27 18:05:45 +00:00

Files

T

3fca2e0ca6 feat(environment): extract the initial state a capture's reads observed (#5319 )

* feat(environment): extract the initial state a capture's reads observed

Add the environment-state analysis view. From the parents the environment
manifest accepts, it extracts the data the capture's reads observed before
the first related write: entity types with an identity field (proved by a
path parameter or inferred from an identity name), entities with their
recorded values and provenance, documents for written non-entity paths, and
a render plan per read tool that is kept only when it reproduces the tool's
latest initial recordings from the state; every other read tool is a
recorded view with the reason. The Evidence is always inferred.

Response-body decoding moves to a domain owner shared with replay, exact
JSON parsing moves to the owner of the exact-number policy, the manifest and
state views share parent admission and record grouping, and
projectAnalysisView takes its accumulators as one object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): bound state at redacted writes; infer identity only from items

A scenario write whose URL was redacted has no request line, so it was
dropped and reads after it counted as initial state. Such a successful
write now joins the timeline with an empty path, whose root relates to
every type, so it ends every type's state; a limitation counts them.

A list whose recordings were all empty passed the "every listed item
carries it" check vacuously and published an identity field nothing
showed; it now needs at least one item, else the view stays recorded
as no-identity.

A value that differed between two request lines of one item tool, such
as a query variant, was called volatile, and the gate then substituted
each recording's own value; only repeats of one request line now show
volatility, so differing lines go through fields-differ.

List membership is compared only while a list can still be the whole
collection, and merged entities are indexed by type and collection once,
so paged captures no longer cost pages times entities (8,000 pages of
40,000 entities: 5.7 s to 0.23 s).

Docs and the tool contract say scenario state keeps the capture's
redaction markers and how redacted writes bound it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep first sightings, one list pointer and linear field unions

Entities took the order of the first plan observed, and item plans run
before list plans, so a complete list read before its detail reads came
out reordered and lost "all" membership; each entity now keeps its
earliest recorded sighting.

A list plan reused the first recording's pointer for later recordings,
so a later response that was itself an entity wrapping a list was read
as that list; every recording must now locate the same list.

The field union scanned its ordered list for every key, quadratic in an
object's width; a set now answers membership.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep post-write values and partial lists out of the initial state

A value that changed between repeats of one request line was named
volatile whenever no write under the type's own root lay between, so a
write under another root (for example POST /api/dashboards/db before a
second GET /api/search) left its effect in the initial state. Any
recorded write between the two observations now leaves every tool of the
type recorded with reason fields-differ instead.

A list was published with membership "all" even when a total or count
envelope member exceeded the number it listed. Such a counter now makes
the membership "recorded".

Also drop a stale doc comment on ObservedEntity.order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): scope item-shaped and deleted written paths out of documents

State documents are held only for list-shaped paths a POST, PUT or PATCH
tool writes. Reads of item-shaped written paths and of paths that are only
deleted stay recorded views; the guide now says so and the spec defers them
to a later increment of #4933.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): count writes in flight when a value changes between reads

The volatility check compared write start positions with the two reads,
so a write already in flight when the earlier read started was missed and
its effect was named volatile. A write now counts when its interval
overlaps the span from the earlier read's start to the later read's end.

The check also ran once per changed field and scanned every write. It now
runs once per pair of observations, answered by a binary search over the
writes in timeline order and the latest end among each prefix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): catch cross-tool changes across writes; read only item totals

A value that changed across a write was caught only between repeats of
one tool's request line, so a list read before a write and an item read
after it put the post-write value into the initial state. Changes are
now classified per entity in timeline order after observation: across
any recorded write, a change between repeats of a line or between two
lines of the type rewrites the type (every tool recorded as
fields-differ); with no write between, a repeat's change is volatile.
The classification lives in environmentStateMerge.ts.

Any numeric member whose name contained "total" or "count" counted as
an item total, so total_pages, account_id or page_count made complete
lists partial. Item totals are now members whose name tokens include
total or count and none of page, pages or per; a string of digits counts
as a number; and a set has_more, next or next-page token member also
marks the list partial.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep post-write observations out of initial state

* fix(environment): retain field provenance and bound state inference work

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <gensokyo-dev@users.noreply.github.com>

2026-10-19 20:33:46 -07:00

4.5 KiB

Raw Permalink Blame History

Environment state view from captures

Issue: https://github.com/morluto/rea/issues/4933

Outcome

inspect_analysis_view accepts view: {"kind": "environment-state"} for the parents the environment manifest accepts and extracts the initial state the capture's reads observed: entity types with an identity field, entities with their recorded values, documents for written paths whose reads are not entities, and a render plan per read tool that reproduces the tool's latest initial recordings from the state, or a recorded view with the reason it cannot. This increment adds the view kind only; serving from the state, loading another state through the AgentEnv data plane and modelling write effects are later increments.

Requirements

Accept inspect_web_network_capture HAR parents and capture_browser_scenario parents that selected response bodies, with the manifest's admission: the same format, subject and sensitive-value refusals.
Group records exactly as the manifest does for the same template_rules, so tool names match the manifest the state accompanies.
Order primary-origin records by recorded start time, or by record order when any start time is missing, and say which. Bound each type's state by the first write under its root (API base, version segments, one segment); a query POST is not a write, a read overlapping the write is left out, and a write whose URL was redacted bounds every type, with a limitation counting it.
Prove an item type's identity field from the requested path value its responses repeat; otherwise infer it from a fixed list of identity names that every listed item carries distinctly, which needs at least one listed item. Never treat an envelope that is itself an entity as a list.
Merge observations into one entity per collection and identity: item views' latest values win over list views', differing views keep their own values, and a value that changed between repeats of one request line with no recorded write is volatile and keeps the latest; a change across a write under another root, including between two tools of the type, leaves every tool of the type recorded. Compare list membership, including empty lists, and field presence before selecting each request line's latest recording; a change across a write also leaves the type recorded. Different query strings remain distinct; omitted fields do not erase their earlier observations on other request lines.
A list has all membership only when no item-total envelope member (total, count and their compounds, not page counts) differs from the number of entities it lists and no has_more, next or next-page token is set.
Keep an item or list plan only when it reproduces each request line's latest initial recording from the state, comparing JSON values without regard to member order; otherwise record the tool with a reason. Parse bodies only when JSON-like, UTF-8, at most 256 KiB, exactly representable and free of __proto__ members.
Mark the Evidence inferred; state unknowns for authorization, response semantics and write effects.
Charge the manifest's admission and grouping, every record line, every body as temporary bytes before decoding and parsing, and every kept value, within the existing 96 MiB budget.
Read response bodies through one domain owner shared with replay, and parse exact JSON through the owner of the exact-number policy.

Acceptance and limits

The Grafana-shaped HAR fixture with distinct start times drives domain, service, MCP, CLI and acceptance coverage: folders merge their detail and list views under a proved uid, the dashboard entity is read from its /dashboard wrapper, and the annotation list after the annotation write has no initial recording. Synthetic captures cover write bounds, volatile fields, complete lists with counters, each recorded reason and documents; the scenario fixture covers record order and the response-body requirement. The view performs no network activity and serves nothing. Values copied outside entity objects, derived values and cross-type effects are not tied to the state.

Deferred issue criteria

Serving tool calls from the state, data/reset, data/add and data/get on the environment data plane, exporting the state with a bundle, and write effects are later increments of #4933. So are documents for item-shaped written paths (one per identity rather than per template) and for paths that are only deleted; until then their reads stay recorded. This PR must reference rather than close the issue.

4.5 KiB Raw Permalink Blame History