mirror of
https://github.com/morluto/rea.git
synced 2026-10-30 19:35:56 +00:00
docs(environment): document environment-build and the coverage view
The agent environments guide opens with the one-step build: the report, the hint for each withheld write reason, and what a bundle ships. The CLI guide, the capture guide and the skill reference name the command and the environment-coverage view. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
cf7eb709e1
commit
1e2c12e5c3
7 files changed
+287
-9
No files matched your search
@@ -104,6 +104,11 @@ each plan states why in its `reason`. It replays the capture instead with
|
||||
`--replay-only` or when the state is unavailable (not derivable, or a catalog
|
||||
over the discovery budget), and its stopped document's `state` says which
|
||||
applied.
|
||||
Use `view: {kind: "environment-coverage"}` with the same rules to see what an
|
||||
environment built from a capture would serve: read tools answered from the
|
||||
state or replayed with their reason, and each write tool served or withheld
|
||||
with a hint about what to record next. `rea environment-build <capture.har>
|
||||
<dir>` inspects, projects and exports in one step and prints the same coverage.
|
||||
|
||||
For a PE/ELF export question, `inspect_native_exports` accepts `detail: "summary"`
|
||||
and retains the complete table. Query `inspect_analysis_view` with
|
||||
|
||||
+143
-9
@@ -1,16 +1,20 @@
|
||||
# Agent environments from recorded captures
|
||||
|
||||
Use `rea environment-serve` to serve an environment manifest as an agent
|
||||
environment: an MCP server whose tools answer from what one retained web
|
||||
network capture observed, plus the environment card the agentenv framework
|
||||
reads to discover it. By default the server derives the
|
||||
[state the capture observed](#the-state-a-capture-observed) at startup and
|
||||
answers read tools from it, and the write tools whose effect the capture
|
||||
confirmed by changing it; `--replay-only` replays the capture's records
|
||||
Use `rea environment-build` to turn one recorded capture into an agent
|
||||
environment bundle in one offline step: a folder `agent-env run` builds and
|
||||
runs, whose MCP server answers from what the capture observed. See
|
||||
[Build a bundle in one step](#build-a-bundle-in-one-step).
|
||||
|
||||
The steps it composes are separate commands too. `rea environment-serve` serves
|
||||
an environment manifest as an agent environment: an MCP server whose tools
|
||||
answer from what one retained web network capture observed, plus the
|
||||
environment card the agentenv framework reads to discover it. By default the
|
||||
server derives the [state the capture observed](#the-state-a-capture-observed)
|
||||
at startup and answers read tools from it, and the write tools whose effect the
|
||||
capture confirmed by changing it; `--replay-only` replays the capture's records
|
||||
instead. Nothing is fetched from the recorded application; every answer comes
|
||||
from retained Evidence. `rea environment-export` writes the same pair as a
|
||||
bundle folder `agent-env run` builds and runs; see
|
||||
[Export a bundle](#export-a-bundle).
|
||||
bundle folder; see [Export a bundle](#export-a-bundle).
|
||||
|
||||
```sh
|
||||
rea inspect-web-network-capture /absolute/session.har har --format json > capture.json
|
||||
@@ -36,6 +40,136 @@ The third command serves that manifest from the state it derives from the
|
||||
capture; see [Serve from the state](#serve-from-the-state). Both served files
|
||||
are complete REA Evidence JSON as the commands write it with `--format json`;
|
||||
the `evidence` member of `view.json` is the whole of `capture.json`.
|
||||
A `view.json` can also name the capture file as
|
||||
`{"kind": "evidence-file", "path": "capture.json"}`, which projects it as
|
||||
retained Evidence instead of charging the whole capture as inline input.
|
||||
|
||||
## Build a bundle in one step
|
||||
|
||||
```sh
|
||||
rea environment-build grafana.har ./grafana --json
|
||||
agent-env run ./grafana
|
||||
```
|
||||
|
||||
`rea environment-build <capture> [directory]` takes a `.har` file, or capture
|
||||
Evidence JSON from `inspect-web-network-capture` (HAR) or
|
||||
`capture-browser-scenario`, and writes the bundle
|
||||
[Export a bundle](#export-a-bundle) describes. It inspects a HAR as
|
||||
`inspect-web-network-capture` does (Evidence input skips this), projects the
|
||||
environment manifest from that Evidence as retained Evidence, authenticates
|
||||
the pair, prepares the state the bundle's server derives at startup, and
|
||||
writes the bundle with the export. For the same capture path and options every
|
||||
file it writes is byte-identical to what `inspect-web-network-capture`,
|
||||
`inspect-analysis-view` and `environment-export` write, Evidence ids included.
|
||||
It reads one file and writes one folder: it fetches nothing and launches no
|
||||
process, so record a browser scenario with `capture-browser-scenario` first
|
||||
and pass its Evidence. Other capture formats are refused, as the manifest view
|
||||
refuses them.
|
||||
|
||||
- `--template-rule <rule>`, repeated, replaces the default opt-in rules
|
||||
`after-uid-literal`, `opaque-token`, `sibling-variance` and
|
||||
`prefix-variance`, the rules the published capture-built environments use;
|
||||
`--format-rules-only` applies none, the views' own default. Rules apply in
|
||||
the views' fixed order, and the report lists them.
|
||||
- `--name` names the env folder and env id. Without it the name is the
|
||||
manifest's, unless the capture's origin is a loopback name or an IP literal:
|
||||
the capture file then names it (`grafana.har` and `grafana.capture.json`
|
||||
become `grafana`) and a `name-from-file` warning says so. A file name that
|
||||
yields no label is refused; pass `--name`.
|
||||
- `--package-tarball` works as for the export.
|
||||
- `--dry-run` runs every stage and prints the report with `status: "planned"`
|
||||
and the files a build would write, but writes nothing and does not check the
|
||||
directory, which may then be omitted.
|
||||
|
||||
The directory must be absent or empty, as for the export; the build never
|
||||
replaces a folder. A refusal before the write leaves nothing behind. Each
|
||||
refusal names its stage: `environment-build` for the build's own checks,
|
||||
`inspect_web_network_capture` and `inspect_analysis_view` for inspection and
|
||||
projection, `environment-export` for the bundle. Stage progress is logged to
|
||||
stderr at info level; the report is the only output on stdout:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "built",
|
||||
"directory": "/work/grafana",
|
||||
"name": "grafana",
|
||||
"capture": {
|
||||
"evidence_id": "ev_…",
|
||||
"operation": "inspect_web_network_capture",
|
||||
"records": 105,
|
||||
"input": "har"
|
||||
},
|
||||
"manifest": {
|
||||
"evidence_id": "ev_…",
|
||||
"tools": 48,
|
||||
"excluded_groups": 0,
|
||||
"template_rules": [
|
||||
"after-uid-literal",
|
||||
"opaque-token",
|
||||
"sibling-variance",
|
||||
"prefix-variance"
|
||||
]
|
||||
},
|
||||
"state": { "served": true, "evidence_id": "ev_…" },
|
||||
"coverage": {
|
||||
"reads": {
|
||||
"state": 36,
|
||||
"recorded": 6,
|
||||
"recorded_reasons": { "query-varies": 3, "…": 1 }
|
||||
},
|
||||
"writes": {
|
||||
"served": [{ "tool": "delete_folders_by_uid", "effect": "delete" }],
|
||||
"withheld": [
|
||||
{
|
||||
"tool": "post_folders",
|
||||
"effect": "create",
|
||||
"reason": "unconfirmed",
|
||||
"hint": "No later read showed the effect. Read the entity and its list after the write.",
|
||||
"exemplar_ordinal": 94,
|
||||
"contradicted_by": null
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"tool_count": 37,
|
||||
"package_source": { "kind": "registry", "specifier": "rea-agents@…" },
|
||||
"files": [{ "path": "README.md", "bytes": 4210 }],
|
||||
"warnings": [{ "code": "name-from-file", "message": "…" }],
|
||||
"next": ["agent-env run /work/grafana"]
|
||||
}
|
||||
```
|
||||
|
||||
`coverage` is the state's read views and write plans summarized, and `null`
|
||||
when the state is unavailable and the bundle serves replay. The
|
||||
[environment coverage](web-network-captures.md#environment-coverage) view
|
||||
reports the same summary for any capture, so an agent can check it after each
|
||||
recording. Each withheld write carries a hint about what to record next; hints
|
||||
never change what is served:
|
||||
|
||||
| Reason | Hint |
|
||||
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `no-state-target` | The write's path names no collection or item the capture reads. Read the collection or item it changes. |
|
||||
| `no-call` | No recorded call of this tool was modelled, for example because every call failed. Record a successful call. |
|
||||
| `body-unusable` | The request body is not a usable JSON object. |
|
||||
| `no-observation` | Nothing showed the written entity. Read it back by id right after the write. |
|
||||
| `unexplained-change` | The write changed fields it did not send, in a way no rule explains. A second, different call of the same tool can show the pattern. |
|
||||
| `contradicted` | A later read disagreed with the modelled effect. Check the read at contradicted_by. |
|
||||
| `side-effect` | The write changed something besides its own entity, which the environment does not model. |
|
||||
| `unconfirmed` | No later read showed the effect. Read the entity and its list after the write. |
|
||||
| `incomplete-entities` | The write is confirmed, but the views that show its entity cannot render it in full. Read the written entity through every view that lists it. |
|
||||
|
||||
The bundle ships `evidence.json`, the whole capture Evidence, response bodies
|
||||
included. Inspection excludes header and cookie authentication values and URL
|
||||
userinfo; tokens an application returns in bodies remain. The environment
|
||||
views refuse captures inspected with `--sensitive-value`, so scrub a capture
|
||||
before building it. The report's `credential-named-members` warning counts
|
||||
recorded body members whose names look like credentials (whole name segments
|
||||
such as `token`, `secret`, `password`, `api_key` or `session`) and hold a
|
||||
string, and lists the first 20 by record and JSON pointer, never by value.
|
||||
Names do not establish sensitivity, so the build never refuses on them. Capture
|
||||
Evidence also records the capture file's absolute path (`subject.local_path`,
|
||||
`parameters.capture_path`, `locations`, `artifact.path`), and the bundle
|
||||
carries it: build from a neutral folder when that path should not ship.
|
||||
|
||||
## From a browser scenario
|
||||
|
||||
|
||||
+27
@@ -702,6 +702,30 @@ like a blank `MCP_HOST`. The bound URL is written to stderr as one JSON log line
|
||||
[agent environments](agent-environments.md) for what the state and replay
|
||||
return and do not establish.
|
||||
|
||||
## Build an environment bundle
|
||||
|
||||
`rea environment-build <capture> [dir] [--name <name>] [--template-rule <rule> ...]
|
||||
[--format-rules-only] [--package-tarball <path>] [--dry-run]` builds the bundle
|
||||
`environment-export` writes from one recorded capture: a `.har` file, which it
|
||||
inspects as `inspect-web-network-capture` does, or capture Evidence JSON from
|
||||
`inspect-web-network-capture` (HAR) or `capture-browser-scenario`. It projects
|
||||
the environment manifest from the in-memory Evidence with the opt-in rules
|
||||
`after-uid-literal`, `opaque-token`, `sibling-variance` and `prefix-variance`
|
||||
unless `--template-rule` or `--format-rules-only` selects others, prepares the
|
||||
state, and writes the bundle; every file is byte-identical to the three-step
|
||||
path's. A loopback or IP-literal capture is named after its file unless `--name`
|
||||
is given. It prints one JSON report: the capture and manifest identities, the
|
||||
state, a `coverage` summary of which tools answer from the state, which replay
|
||||
and why each write is withheld with a hint, the files, warnings and the next
|
||||
command. `--dry-run` writes nothing and may omit the directory.
|
||||
|
||||
```sh
|
||||
rea environment-build grafana.har ./grafana --json
|
||||
```
|
||||
|
||||
See [agent environments](agent-environments.md#build-a-bundle-in-one-step)
|
||||
for the report, the hints and what the bundle ships.
|
||||
|
||||
## Export an environment bundle
|
||||
|
||||
`rea environment-export <manifest.json> --evidence <capture.json> <dir>
|
||||
@@ -877,6 +901,9 @@ server resources. It preserves a primary listener diagnostic, or a
|
||||
tool count and Evidence identifiers. The successful `stopped` document is
|
||||
returned only after cleanup is confirmed. The same in-process server owner
|
||||
retains failed stages for retry; the CLI exposes no separate retry command.
|
||||
`rea environment-build` is one-shot too: it prints its report and exits `0`,
|
||||
or prints the refusing stage's structured error and exits `1`, writing nothing
|
||||
when the refusal comes before the bundle is written.
|
||||
`rea environment-export` is one-shot: it prints the written files and exits
|
||||
`0`, or prints a structured `invalid_request` error and exits `1` for the same
|
||||
pair refusals, a non-empty or file-occupied directory, a missing package
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
# Plan
|
||||
|
||||
1. Add `environmentCoverageView.ts`: the request schema, the result fields, the
|
||||
hint per write reason (`satisfies Record<WriteReason, string>`), a
|
||||
summarizer over state views and write plans, and the projector, which
|
||||
projects the environment-state view and summarizes it. Register the kind in
|
||||
the view unions, the grouping dispatch and the view service (state
|
||||
accumulator, inferred confidence).
|
||||
2. Split the export's input read into `readEnvironmentExportFile` and
|
||||
`environmentExportFile` (parse and seal bytes), both naming the calling
|
||||
operation, and let the export take an already prepared state.
|
||||
3. Add `environmentBuild.ts`: classify the capture, project the manifest from
|
||||
the in-memory Evidence, serialize it as `--format json` writes it to a file,
|
||||
authenticate the pair from those bytes, name the environment, prepare the
|
||||
state as the export does, summarize coverage, scan credential names, then
|
||||
plan (dry run) or export. `environmentBuildReport.ts` holds the scan and the
|
||||
naming helpers.
|
||||
4. Register `environment-build` in the CLI: inspect a HAR through the capture
|
||||
service as `inspect-web-network-capture` does, or read Evidence once as the
|
||||
export does, and log stages at info.
|
||||
5. Switch `scripts/verify-agentenv-environment.mjs` to the build command.
|
||||
6. Document the command, the report, the hints, what ships, and the view.
|
||||
7. Tests: coverage summary and hint completeness, the view against the state
|
||||
view, build equivalence with the three-step path in process and through the
|
||||
CLI, dry run, naming, refusals, the credential scan, and the view over MCP.
|
||||
@@ -0,0 +1,56 @@
|
||||
# Build an environment in one step
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
Building an agent environment bundle takes three commands and a hand-written
|
||||
view request that embeds the whole capture inline, so large captures hit inline
|
||||
input accounting. The views apply no opt-in template rules by default, while the
|
||||
published capture-built environments all use the same four, and a local capture
|
||||
is named after its origin host (`127-0-0-1-3105`). Nothing reports which tools
|
||||
an environment serves from its state, or why a write tool is withheld.
|
||||
|
||||
`rea environment-build <capture> [directory]` builds the bundle in one offline
|
||||
step and reports what it serves. A new `environment-coverage` analysis view
|
||||
reports the same summary for any capture, so an agent can check a recording
|
||||
before building.
|
||||
|
||||
## Requirements
|
||||
|
||||
- The capture is a `.har` file, inspected as `inspect-web-network-capture`
|
||||
inspects it, or capture Evidence JSON from `inspect_web_network_capture` or
|
||||
`capture_browser_scenario`; anything else is refused before it is read.
|
||||
- The manifest is projected from the in-memory capture as retained Evidence,
|
||||
with the default rules `after-uid-literal`, `opaque-token`,
|
||||
`sibling-variance` and `prefix-variance` unless `--template-rule` or
|
||||
`--format-rules-only` selects others; rules apply in the views' order.
|
||||
- For the same capture path and options, every bundle file is byte-identical
|
||||
to what `inspect-web-network-capture --format json`, `inspect-analysis-view
|
||||
--format json` and `environment-export` write, Evidence ids included.
|
||||
- The name is `--name`, else the manifest's, unless the origin is a loopback
|
||||
name or IP literal: then the capture file's name, with a warning; a file name
|
||||
that yields no label is refused.
|
||||
- The report extends the export document with the capture and manifest
|
||||
identities, the state, `coverage`, warnings and the next command. `coverage`
|
||||
counts read tools answered from the state and replayed (per view reason), and
|
||||
lists write tools as served or withheld with their reason, plan record, the
|
||||
contradicting read and a hint; it is null when the state is unavailable.
|
||||
- Every write reason has a hint; a reason without one fails to compile.
|
||||
- Credential-named body members with a string value are reported as a count
|
||||
and the first 20 record pointers, never values, and never refuse a build.
|
||||
- Nothing is written before the bundle stage; `--dry-run` writes nothing and
|
||||
reports the files a build would write. The directory must be absent or empty.
|
||||
- The `environment-coverage` view summarizes the environment-state view with
|
||||
the same rules, refuses what it refuses and is inferred.
|
||||
- The `real-agentenv` lane builds its bundle with `environment-build`.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No change to the three existing commands' contracts, or to what any view
|
||||
derives or any environment serves.
|
||||
- No browser launch or network access: scenarios are recorded with
|
||||
`capture-browser-scenario` first.
|
||||
- No `--replace`, no refusal on credential names, and no rewriting of the
|
||||
capture's absolute path, which Evidence identity includes.
|
||||
- No MCP export tool; that is a separate change.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the outcome, requirements and non-goals.
|
||||
- [x] Add the environment-coverage view and register it.
|
||||
- [x] Split the export's input read and accept a prepared state.
|
||||
- [x] Add the build module, its report helpers and the CLI command.
|
||||
- [x] Switch the real-agentenv lane to the build command.
|
||||
- [x] Document the command, the report, the hints and the view.
|
||||
- [x] Add domain, service, CLI and MCP tests.
|
||||
- [ ] Run static, formatting, duplication, documentation and test gates.
|
||||
- [ ] Build the 50 published captures and compare them with the three-step path.
|
||||
- [ ] Build environments end to end on Linux VMs and run them with agents.
|
||||
@@ -773,6 +773,25 @@ derives it from the capture at startup, serves read tools and the served write
|
||||
plans from it and loads replacement states through the AgentEnv data plane, as
|
||||
[serve from the state](agent-environments.md#serve-from-the-state) describes.
|
||||
|
||||
## Environment coverage
|
||||
|
||||
`view: {"kind":"environment-coverage"}` with the same `template_rules` reports
|
||||
what an environment built from the parent would serve, without the state's
|
||||
entities: `reads.state` counts read tools answered from the state,
|
||||
`reads.recorded` those that replay their recordings with
|
||||
`reads.recorded_reasons` per view reason, `writes.served` lists the write tools
|
||||
whose effect later reads confirmed, and `writes.withheld` lists every other
|
||||
write tool with its reason, the record its plan was built from
|
||||
(`exemplar_ordinal`), the read that disagreed (`contradicted_by`) and a `hint`
|
||||
about what to record next. Hints are advice; they never change what is served.
|
||||
The view projects the environment-state view and summarizes it, so it refuses
|
||||
every parent that view refuses, with the same errors, and it is always
|
||||
inferred. Call it after each recording to see which writes an environment
|
||||
built from it would serve; `rea environment-build` prints the same summary as
|
||||
its `coverage`. See
|
||||
[Build a bundle in one step](agent-environments.md#build-a-bundle-in-one-step)
|
||||
for the hint per reason.
|
||||
|
||||
## Resource and lifecycle boundaries
|
||||
|
||||
Complete evidence has a 32 MiB input, 96 MiB decoder reply and 64-level nesting
|
||||
|
||||
Reference in new issue
Block a user