diff --git a/.agents/skills/reverse-engineer-anything/references/evidence-workflows.md b/.agents/skills/reverse-engineer-anything/references/evidence-workflows.md index fe542fa90d..33b733b865 100644 --- a/.agents/skills/reverse-engineer-anything/references/evidence-workflows.md +++ b/.agents/skills/reverse-engineer-anything/references/evidence-workflows.md @@ -104,6 +104,11 @@ each plan states why in its `reason`. It replays the capture instead with `--replay-only` or when the state is unavailable (not derivable, or a catalog over the discovery budget), and its stopped document's `state` says which applied. +Use `view: {kind: "environment-coverage"}` with the same rules to see what an +environment built from a capture would serve: read tools answered from the +state or replayed with their reason, and each write tool served or withheld +with a hint about what to record next. `rea environment-build +` inspects, projects and exports in one step and prints the same coverage. For a PE/ELF export question, `inspect_native_exports` accepts `detail: "summary"` and retains the complete table. Query `inspect_analysis_view` with diff --git a/docs/agent-environments.md b/docs/agent-environments.md index a929c1abf8..092f77c923 100644 --- a/docs/agent-environments.md +++ b/docs/agent-environments.md @@ -1,16 +1,20 @@ # Agent environments from recorded captures -Use `rea environment-serve` to serve an environment manifest as an agent -environment: an MCP server whose tools answer from what one retained web -network capture observed, plus the environment card the agentenv framework -reads to discover it. By default the server derives the -[state the capture observed](#the-state-a-capture-observed) at startup and -answers read tools from it, and the write tools whose effect the capture -confirmed by changing it; `--replay-only` replays the capture's records +Use `rea environment-build` to turn one recorded capture into an agent +environment bundle in one offline step: a folder `agent-env run` builds and +runs, whose MCP server answers from what the capture observed. See +[Build a bundle in one step](#build-a-bundle-in-one-step). + +The steps it composes are separate commands too. `rea environment-serve` serves +an environment manifest as an agent environment: an MCP server whose tools +answer from what one retained web network capture observed, plus the +environment card the agentenv framework reads to discover it. By default the +server derives the [state the capture observed](#the-state-a-capture-observed) +at startup and answers read tools from it, and the write tools whose effect the +capture confirmed by changing it; `--replay-only` replays the capture's records instead. Nothing is fetched from the recorded application; every answer comes from retained Evidence. `rea environment-export` writes the same pair as a -bundle folder `agent-env run` builds and runs; see -[Export a bundle](#export-a-bundle). +bundle folder; see [Export a bundle](#export-a-bundle). ```sh rea inspect-web-network-capture /absolute/session.har har --format json > capture.json @@ -36,6 +40,136 @@ The third command serves that manifest from the state it derives from the capture; see [Serve from the state](#serve-from-the-state). Both served files are complete REA Evidence JSON as the commands write it with `--format json`; the `evidence` member of `view.json` is the whole of `capture.json`. +A `view.json` can also name the capture file as +`{"kind": "evidence-file", "path": "capture.json"}`, which projects it as +retained Evidence instead of charging the whole capture as inline input. + +## Build a bundle in one step + +```sh +rea environment-build grafana.har ./grafana --json +agent-env run ./grafana +``` + +`rea environment-build [directory]` takes a `.har` file, or capture +Evidence JSON from `inspect-web-network-capture` (HAR) or +`capture-browser-scenario`, and writes the bundle +[Export a bundle](#export-a-bundle) describes. It inspects a HAR as +`inspect-web-network-capture` does (Evidence input skips this), projects the +environment manifest from that Evidence as retained Evidence, authenticates +the pair, prepares the state the bundle's server derives at startup, and +writes the bundle with the export. For the same capture path and options every +file it writes is byte-identical to what `inspect-web-network-capture`, +`inspect-analysis-view` and `environment-export` write, Evidence ids included. +It reads one file and writes one folder: it fetches nothing and launches no +process, so record a browser scenario with `capture-browser-scenario` first +and pass its Evidence. Other capture formats are refused, as the manifest view +refuses them. + +- `--template-rule `, repeated, replaces the default opt-in rules + `after-uid-literal`, `opaque-token`, `sibling-variance` and + `prefix-variance`, the rules the published capture-built environments use; + `--format-rules-only` applies none, the views' own default. Rules apply in + the views' fixed order, and the report lists them. +- `--name` names the env folder and env id. Without it the name is the + manifest's, unless the capture's origin is a loopback name or an IP literal: + the capture file then names it (`grafana.har` and `grafana.capture.json` + become `grafana`) and a `name-from-file` warning says so. A file name that + yields no label is refused; pass `--name`. +- `--package-tarball` works as for the export. +- `--dry-run` runs every stage and prints the report with `status: "planned"` + and the files a build would write, but writes nothing and does not check the + directory, which may then be omitted. + +The directory must be absent or empty, as for the export; the build never +replaces a folder. A refusal before the write leaves nothing behind. Each +refusal names its stage: `environment-build` for the build's own checks, +`inspect_web_network_capture` and `inspect_analysis_view` for inspection and +projection, `environment-export` for the bundle. Stage progress is logged to +stderr at info level; the report is the only output on stdout: + +```json +{ + "status": "built", + "directory": "/work/grafana", + "name": "grafana", + "capture": { + "evidence_id": "ev_…", + "operation": "inspect_web_network_capture", + "records": 105, + "input": "har" + }, + "manifest": { + "evidence_id": "ev_…", + "tools": 48, + "excluded_groups": 0, + "template_rules": [ + "after-uid-literal", + "opaque-token", + "sibling-variance", + "prefix-variance" + ] + }, + "state": { "served": true, "evidence_id": "ev_…" }, + "coverage": { + "reads": { + "state": 36, + "recorded": 6, + "recorded_reasons": { "query-varies": 3, "…": 1 } + }, + "writes": { + "served": [{ "tool": "delete_folders_by_uid", "effect": "delete" }], + "withheld": [ + { + "tool": "post_folders", + "effect": "create", + "reason": "unconfirmed", + "hint": "No later read showed the effect. Read the entity and its list after the write.", + "exemplar_ordinal": 94, + "contradicted_by": null + } + ] + } + }, + "tool_count": 37, + "package_source": { "kind": "registry", "specifier": "rea-agents@…" }, + "files": [{ "path": "README.md", "bytes": 4210 }], + "warnings": [{ "code": "name-from-file", "message": "…" }], + "next": ["agent-env run /work/grafana"] +} +``` + +`coverage` is the state's read views and write plans summarized, and `null` +when the state is unavailable and the bundle serves replay. The +[environment coverage](web-network-captures.md#environment-coverage) view +reports the same summary for any capture, so an agent can check it after each +recording. Each withheld write carries a hint about what to record next; hints +never change what is served: + +| Reason | Hint | +| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| `no-state-target` | The write's path names no collection or item the capture reads. Read the collection or item it changes. | +| `no-call` | No recorded call of this tool was modelled, for example because every call failed. Record a successful call. | +| `body-unusable` | The request body is not a usable JSON object. | +| `no-observation` | Nothing showed the written entity. Read it back by id right after the write. | +| `unexplained-change` | The write changed fields it did not send, in a way no rule explains. A second, different call of the same tool can show the pattern. | +| `contradicted` | A later read disagreed with the modelled effect. Check the read at contradicted_by. | +| `side-effect` | The write changed something besides its own entity, which the environment does not model. | +| `unconfirmed` | No later read showed the effect. Read the entity and its list after the write. | +| `incomplete-entities` | The write is confirmed, but the views that show its entity cannot render it in full. Read the written entity through every view that lists it. | + +The bundle ships `evidence.json`, the whole capture Evidence, response bodies +included. Inspection excludes header and cookie authentication values and URL +userinfo; tokens an application returns in bodies remain. The environment +views refuse captures inspected with `--sensitive-value`, so scrub a capture +before building it. The report's `credential-named-members` warning counts +recorded body members whose names look like credentials (whole name segments +such as `token`, `secret`, `password`, `api_key` or `session`) and hold a +string, and lists the first 20 by record and JSON pointer, never by value. +Names do not establish sensitivity, so the build never refuses on them. Capture +Evidence also records the capture file's absolute path (`subject.local_path`, +`parameters.capture_path`, `locations`, `artifact.path`), and the bundle +carries it: build from a neutral folder when that path should not ship. ## From a browser scenario diff --git a/docs/cli.md b/docs/cli.md index 3eeac08b7c..6f855f81a2 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -702,6 +702,30 @@ like a blank `MCP_HOST`. The bound URL is written to stderr as one JSON log line [agent environments](agent-environments.md) for what the state and replay return and do not establish. +## Build an environment bundle + +`rea environment-build [dir] [--name ] [--template-rule ...] +[--format-rules-only] [--package-tarball ] [--dry-run]` builds the bundle +`environment-export` writes from one recorded capture: a `.har` file, which it +inspects as `inspect-web-network-capture` does, or capture Evidence JSON from +`inspect-web-network-capture` (HAR) or `capture-browser-scenario`. It projects +the environment manifest from the in-memory Evidence with the opt-in rules +`after-uid-literal`, `opaque-token`, `sibling-variance` and `prefix-variance` +unless `--template-rule` or `--format-rules-only` selects others, prepares the +state, and writes the bundle; every file is byte-identical to the three-step +path's. A loopback or IP-literal capture is named after its file unless `--name` +is given. It prints one JSON report: the capture and manifest identities, the +state, a `coverage` summary of which tools answer from the state, which replay +and why each write is withheld with a hint, the files, warnings and the next +command. `--dry-run` writes nothing and may omit the directory. + +```sh +rea environment-build grafana.har ./grafana --json +``` + +See [agent environments](agent-environments.md#build-a-bundle-in-one-step) +for the report, the hints and what the bundle ships. + ## Export an environment bundle `rea environment-export --evidence @@ -877,6 +901,9 @@ server resources. It preserves a primary listener diagnostic, or a tool count and Evidence identifiers. The successful `stopped` document is returned only after cleanup is confirmed. The same in-process server owner retains failed stages for retry; the CLI exposes no separate retry command. +`rea environment-build` is one-shot too: it prints its report and exits `0`, +or prints the refusing stage's structured error and exits `1`, writing nothing +when the refusal comes before the bundle is written. `rea environment-export` is one-shot: it prints the written files and exits `0`, or prints a structured `invalid_request` error and exits `1` for the same pair refusals, a non-empty or file-occupied directory, a missing package diff --git a/docs/specs/agent-environments/environment-build/plan.md b/docs/specs/agent-environments/environment-build/plan.md new file mode 100644 index 0000000000..65b156bbda --- /dev/null +++ b/docs/specs/agent-environments/environment-build/plan.md @@ -0,0 +1,25 @@ +# Plan + +1. Add `environmentCoverageView.ts`: the request schema, the result fields, the + hint per write reason (`satisfies Record`), a + summarizer over state views and write plans, and the projector, which + projects the environment-state view and summarizes it. Register the kind in + the view unions, the grouping dispatch and the view service (state + accumulator, inferred confidence). +2. Split the export's input read into `readEnvironmentExportFile` and + `environmentExportFile` (parse and seal bytes), both naming the calling + operation, and let the export take an already prepared state. +3. Add `environmentBuild.ts`: classify the capture, project the manifest from + the in-memory Evidence, serialize it as `--format json` writes it to a file, + authenticate the pair from those bytes, name the environment, prepare the + state as the export does, summarize coverage, scan credential names, then + plan (dry run) or export. `environmentBuildReport.ts` holds the scan and the + naming helpers. +4. Register `environment-build` in the CLI: inspect a HAR through the capture + service as `inspect-web-network-capture` does, or read Evidence once as the + export does, and log stages at info. +5. Switch `scripts/verify-agentenv-environment.mjs` to the build command. +6. Document the command, the report, the hints, what ships, and the view. +7. Tests: coverage summary and hint completeness, the view against the state + view, build equivalence with the three-step path in process and through the + CLI, dry run, naming, refusals, the credential scan, and the view over MCP. diff --git a/docs/specs/agent-environments/environment-build/spec.md b/docs/specs/agent-environments/environment-build/spec.md new file mode 100644 index 0000000000..48ffd09eda --- /dev/null +++ b/docs/specs/agent-environments/environment-build/spec.md @@ -0,0 +1,56 @@ +# Build an environment in one step + +Issue: https://github.com/morluto/rea/issues/4933 + +## Outcome + +Building an agent environment bundle takes three commands and a hand-written +view request that embeds the whole capture inline, so large captures hit inline +input accounting. The views apply no opt-in template rules by default, while the +published capture-built environments all use the same four, and a local capture +is named after its origin host (`127-0-0-1-3105`). Nothing reports which tools +an environment serves from its state, or why a write tool is withheld. + +`rea environment-build [directory]` builds the bundle in one offline +step and reports what it serves. A new `environment-coverage` analysis view +reports the same summary for any capture, so an agent can check a recording +before building. + +## Requirements + +- The capture is a `.har` file, inspected as `inspect-web-network-capture` + inspects it, or capture Evidence JSON from `inspect_web_network_capture` or + `capture_browser_scenario`; anything else is refused before it is read. +- The manifest is projected from the in-memory capture as retained Evidence, + with the default rules `after-uid-literal`, `opaque-token`, + `sibling-variance` and `prefix-variance` unless `--template-rule` or + `--format-rules-only` selects others; rules apply in the views' order. +- For the same capture path and options, every bundle file is byte-identical + to what `inspect-web-network-capture --format json`, `inspect-analysis-view + --format json` and `environment-export` write, Evidence ids included. +- The name is `--name`, else the manifest's, unless the origin is a loopback + name or IP literal: then the capture file's name, with a warning; a file name + that yields no label is refused. +- The report extends the export document with the capture and manifest + identities, the state, `coverage`, warnings and the next command. `coverage` + counts read tools answered from the state and replayed (per view reason), and + lists write tools as served or withheld with their reason, plan record, the + contradicting read and a hint; it is null when the state is unavailable. +- Every write reason has a hint; a reason without one fails to compile. +- Credential-named body members with a string value are reported as a count + and the first 20 record pointers, never values, and never refuse a build. +- Nothing is written before the bundle stage; `--dry-run` writes nothing and + reports the files a build would write. The directory must be absent or empty. +- The `environment-coverage` view summarizes the environment-state view with + the same rules, refuses what it refuses and is inferred. +- The `real-agentenv` lane builds its bundle with `environment-build`. + +## Non-goals + +- No change to the three existing commands' contracts, or to what any view + derives or any environment serves. +- No browser launch or network access: scenarios are recorded with + `capture-browser-scenario` first. +- No `--replace`, no refusal on credential names, and no rewriting of the + capture's absolute path, which Evidence identity includes. +- No MCP export tool; that is a separate change. diff --git a/docs/specs/agent-environments/environment-build/tasks.md b/docs/specs/agent-environments/environment-build/tasks.md new file mode 100644 index 0000000000..102ac80285 --- /dev/null +++ b/docs/specs/agent-environments/environment-build/tasks.md @@ -0,0 +1,12 @@ +# Tasks + +- [x] Record the outcome, requirements and non-goals. +- [x] Add the environment-coverage view and register it. +- [x] Split the export's input read and accept a prepared state. +- [x] Add the build module, its report helpers and the CLI command. +- [x] Switch the real-agentenv lane to the build command. +- [x] Document the command, the report, the hints and the view. +- [x] Add domain, service, CLI and MCP tests. +- [ ] Run static, formatting, duplication, documentation and test gates. +- [ ] Build the 50 published captures and compare them with the three-step path. +- [ ] Build environments end to end on Linux VMs and run them with agents. diff --git a/docs/web-network-captures.md b/docs/web-network-captures.md index 95df9b8e88..2c1f35a147 100644 --- a/docs/web-network-captures.md +++ b/docs/web-network-captures.md @@ -773,6 +773,25 @@ derives it from the capture at startup, serves read tools and the served write plans from it and loads replacement states through the AgentEnv data plane, as [serve from the state](agent-environments.md#serve-from-the-state) describes. +## Environment coverage + +`view: {"kind":"environment-coverage"}` with the same `template_rules` reports +what an environment built from the parent would serve, without the state's +entities: `reads.state` counts read tools answered from the state, +`reads.recorded` those that replay their recordings with +`reads.recorded_reasons` per view reason, `writes.served` lists the write tools +whose effect later reads confirmed, and `writes.withheld` lists every other +write tool with its reason, the record its plan was built from +(`exemplar_ordinal`), the read that disagreed (`contradicted_by`) and a `hint` +about what to record next. Hints are advice; they never change what is served. +The view projects the environment-state view and summarizes it, so it refuses +every parent that view refuses, with the same errors, and it is always +inferred. Call it after each recording to see which writes an environment +built from it would serve; `rea environment-build` prints the same summary as +its `coverage`. See +[Build a bundle in one step](agent-environments.md#build-a-bundle-in-one-step) +for the hint per reason. + ## Resource and lifecycle boundaries Complete evidence has a 32 MiB input, 96 MiB decoder reply and 64-level nesting