mirror of
https://github.com/morluto/rea.git
synced 2026-10-26 17:35:11 +00:00
02e049744f
* feat(environment): report what an environment serves with an environment-coverage view
A new inspect_analysis_view kind over capture Evidence. It projects the
state an environment derives with the given template rules and reports
each tool as answered from state, replayed with its view reason, a served
write, or a withheld write with its reason, the records behind it and a
hint about what to record next. The hints cover every write reason, so a
new reason without one fails to compile.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(environment): build an environment bundle in one command
rea environment-build <capture> [directory] inspects a HAR (or reads
capture Evidence), projects the environment manifest from the in-memory
Evidence, authenticates the pair, prepares the state and writes the bundle
environment-export writes. Every bundle file is byte-identical to the
three-step path for the same capture and options.
The report adds the capture and manifest identities, the coverage summary,
warnings (a loopback capture named after its file, credential-named body
members by pointer) and the next command. --dry-run writes nothing. The
default template rules are the four the published environments use.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(environment): cover environment-build and the coverage view
Domain tests check the hint table against the state view's reasons, the
summary, and the view against the environment-state view it summarizes.
Service tests build the fixture capture and compare every file with the
three-step path, a dry run, loopback naming and the refusals. A CLI test
compares a HAR built in one step with the three CLI steps byte for byte,
and an MCP test projects the view from every source kind.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(environment): build the real-agentenv lane's bundle with environment-build
The lane now runs the command users run: it builds the Grafana-shaped HAR
with environment-build and the packed checkout, checks the report, then
runs the bundle with agent-env as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(environment): document environment-build and the coverage view
The agent environments guide opens with the one-step build: the report,
the hint for each withheld write reason, and what a bundle ships. The CLI
guide, the capture guide and the skill reference name the command and the
environment-coverage view.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): harden environment-build after review
- The credential scan walks with its own stack, so a deeply nested body
cannot crash the build, and it also reports credential-named header
names, query names, form body members and string arrays.
- The build checks SIGINT between stages: cancelled before the bundle
stage it writes nothing, during the write it names the written bundle.
- A dry run opens the package tarball, so it refuses what a build refuses.
- Repeated template rules are refused, a HAR saved as .json gets a hint,
file-derived names follow the bundle name rules, `localhost.` counts as
a loopback origin, and the next command is shell-quoted.
- --capture-timeout-ms passes through to the HAR inspection, and a
missing directory is refused before inspection.
- The no-observation hint covers updates and deletes of entities the state
never held; the coverage view states that it does not apply the
serve-time admission checks; the inspect_analysis_view description
names the view.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(environment): state the build's conditions and record its verification
The guides now state when the one-step bundle equals the three-step one,
which headers inspection excludes, what the credential warning scans, how
cancellation behaves, and that the coverage view does not apply the
serve-time admission checks. The spec folder records the census, the end
to end runs, the gates and the review.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): dispatch a pending interrupt before each build stage check
The manifest, state and coverage stages run synchronously, so a SIGINT
that arrived during them was only dispatched once the bundle write had
begun. Each check now yields first, so an interrupt before the bundle
stage cancels the build with nothing written.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(browser): send declared requests from a scenario page with a request step
A capture_browser_scenario step `request` sends one declared HTTP request
through the page's own fetch, with its cookies and session, the way the
application's frontend would, and reads the whole response so the capture
records the transaction like any other. The path must resolve to the start
URL's origin, and the page must be on that origin when the step runs.
Query values and header values may be literals or declared secrets, and a
header may read its value from a cookie or a local storage entry, which is
then redacted from later records like a declared secret. `expected_status`
fails the step on any other status. Scenarios without a request step are
unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(browser): verify request steps in a real browser through to a built environment
verify:browser:request records a write session against a fixture item API
that refuses writes without its CSRF cookie header and stored bearer
token. It checks that every request lands in its own step, that the page
values authenticated each write and appear in no record, header or body,
that an unexpected status fails the step and cancels the rest, and that
environment-build serves the recorded create, update and delete.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(environment): export environment bundles over MCP
export_environment_bundle writes the bundle environment-build writes, from
capture Evidence retained in the session or one capture Evidence file,
into an absolute output directory that must not exist and whose parent
must exist, as extract_firmware and export_web_scripts require; it never
creates the parent or reuses a directory. It returns the same report,
whose schema now lives in the domain so the CLI and the tool share it, and
dry_run reports without writing. A loopback capture without a file needs a
name. The build's refusal messages no longer name CLI flags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): scan scenario credentials, own the dry-run tarball and serialize Evidence once
A browser scenario's retained header values and the query parameters of its recorded URLs are scanned for credential
names as a HAR's are, through a review projection of its records. A dry run admits and releases the package tarball
through the export's regular-file scope, so a close it cannot confirm keeps its owner, and checks for cancellation
after that last await. Evidence is written into one exactly sized buffer instead of chunks joined by a copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(environment): record the review round's checks
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(skill): build agent environments from a prompt with request steps and the bundle tool
Add the environments reference to the skill and install it with setup: record
against a disposable instance, sign in through the UI, write with request
steps and read each write back, check coverage, re-record by withheld reason
and export the bundle. Route it from SKILL.md, name agent environments in the
skill description, and document the request step, the export tool and
recording writes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(browser): split the request lane and header resolution into smaller functions
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* style(environment): format the build spec
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): say when an untimed capture learns no write clock
Coverage of a capture whose records are not all timed, which includes every
browser scenario capture, now states that the state learns no clock, so an
update stamped with a server-set modification time stays unexplained-change
however often it is recorded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(skill): give agents a whole scenario shape with step timeouts
Agents guessed the scenario schema field by field and two hung on steps
without timeout_ms. The reference now starts from a complete scenario with a
timeout on every step, and says a scenario capture cannot serve updates the
application stamps with a modification time.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(browser): keep request steps on their origin and their page values redacted
- Follow no redirects: a redirect fails the step and nothing reaches the
location it names (checked in real Chrome), and check the page's origin
again in the page as the request is sent.
- Send nothing once the step's time has run out or it was cancelled while
header values were read; the in-page deadline is what remains of the step's.
- Keep every page value redacted under a repeated or colliding label, keep a
value's first marker when it is read again, and leave values shorter than
8 characters unredacted rather than rewriting ids, ports and hosts.
- Refuse a header name given twice, User-Agent (Chrome drops it) and line
breaks in literal values and prefixes; fail before sending when a secret or
page value holds them.
- Append query entries after the path's literal query without re-encoding it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(environment): document request step limits and word bundle refusals for both surfaces
Document redirects, timeouts, page value redaction, HttpOnly cookie sources
and header refusals for request steps. Refusals shared by environment-build
and export_environment_bundle no longer name CLI flags, and the retained
loopback refusal no longer prints an empty file name. The skill reference
asks for read-backs after each write, shows the coverage view's source, names
package_tarball for unreleased builds and the browser path a scenario ships.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): scan every form occurrence and MIME-less scenario bodies
The build's credential scan collapsed form bodies with Object.fromEntries, so a
repeated field kept only its last occurrence: `api_key=secret&api_key=` warned
about nothing although the Evidence ships the value, and repeated values were
undercounted. Form fields are now scanned as ordered name/value pairs, as
headers and query strings already are.
The review projection of a capture_browser_scenario dropped a request body
recorded without a media type, so the scan never saw it. Review keeps such a
body's text with an empty media type; the manifest projection still reads only
bodies it can classify.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(browser): close request step edge cases from the second review
- Append query entries correctly after a path ending in "?", "&" or "#".
- Refuse 3xx expected statuses, which an unfollowed redirect can never
report, and say a redirected write may already have been applied.
- Redact a page value also as fetch sends it, without surrounding spaces
and tabs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(environment): document page value body costs, redirects and MCP dry runs
Document that reading a page value shrinks the bodies a capture keeps under
the default allocation limit, that redirected writes may have landed, that
the MCP dry_run still checks its output directory and which refusals name
the tool, and keep the .har hint in the HAR-as-JSON refusal for both
surfaces.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(environment): record the prompt-to-environment verification
Gates, the real-browser lanes, five environments built from prompts by an
agent on disposable VMs and evaluated with two models, and the two reviews.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(skill): count rollback failures from the managed skill files
The environments reference made the skill eight files, so the recovery tests now derive the counts they expect from
MANAGED_SKILL_FILES instead of fixing them at six and seven.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(adb): restore the portable Java float pattern a merge rewrote
A lint autofix during a merge unescaped the hyphens in javaFloatValuePattern, which is invalid in a v-mode character
class, so the advertised start_adb_activity schema failed to compile in every JS regex mode. The file now matches main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(browser): show the request step inside a whole scenario
Every JSON example in the browser scenario guide is a capture_browser_scenario input the catalog test parses, so the
request step example now sits in a launch scenario instead of standing alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): scan structured HAR form params and the request side of unattributed scenario transactions
A HAR request recorded with postData.params and no text is scanned by those parameters, each occurrence counting, and
the review projection keeps the request side of a scenario transaction whose response cannot be attributed, so the
credential warning covers what evidence.json ships in both cases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): validate build paths and read dry-run tarballs
* fix(environment): report bodies too large for the credential scan
Text bodies over the scan's 4 MiB limit, including a response decoded only as a prefix, are reported as a
credential-scan-incomplete warning by record and side, never by content, so an absent credential-named-members
warning no longer reads as a complete check. The spec and agent-environments guide also state the path validation and
dry-run tarball read that e52ae1cd added.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(verification): release cold source transform services
* fix(javascript): retain capture context along covered mutation paths
* fix(environment): guard request dispatch and report export limits
* fix(environment): admit serialized exports and scan HAR form fallbacks
* fix(environment): admit in-memory build inputs against heap headroom and scan SaveHar null-text forms
The inspected HAR's Evidence and the projected manifest are checked against live heap headroom before they are
serialized, with the same check the file route uses, so a capture too large to parse and seal is refused as
resource_constraint before anything is written. A request whose postData text is SaveHar's null is scanned by its
structured params.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): measure the credential scan limit in UTF-8 bytes
A body counts against the 4 MiB limit by its UTF-8 length, not its UTF-16 code units, so multibyte text over the
limit is reported as unscanned instead of parsed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(browser): send request steps only before their deadline and keep page reads on the start origin
The page receives the step's absolute deadline and a one-second start window, so a busy page that runs the request
late, or after the step is cancelled, sends nothing; local storage is read only on the start origin; the response is
drained without keeping its body; an unparseable path is a field issue; retained Evidence is refused for heap headroom
before it is serialized; and the skill says how to reach server-assigned ids.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* style: format integrated export admission
* fix(javascript): erase inline type imports before binding values
* fix(browser): run the request lane under the shared verifier command owner
The browser request lane runs each CLI call through createPackageVerifierCommands on a private runtime root, so a
timeout, output overflow or failure stops the CLI and the browser it launched before the root is removed, and a failed
stop is reported with the root retained. The verification record states the lane's current counts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): scan scenario content by event and read bodies by their media type
A browser scenario's credential scan reads every retained network-content event, including a response attached
mid-flight and a transaction no record represents, with event pointers, and each transaction URL's query once; the
review record projection it replaces is removed. A body is parsed as a form or as JSON by its recorded media type,
falling back to its shape, so valid forms such as `password=x&` are scanned and sentences are not read as forms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
7.0 KiB
7.0 KiB
Verification
Executed on macOS with Node v24.22.1 at feat/environment-prompt 8dfd5a4d (stacked on the environment-build branch at
eeab0bda), and on Ubuntu 24.04 VMs for the end-to-end runs.
Gates
npm run -s typecheck,lint,format:check,docs:check,mcp-catalog:check,verify:test-discoveryandnpx knip: passed.npx vitest run --project domain --project services --project adapters: 16,347 passed and 1 failed, "stops Inspector target authorization after the current stat settles", which fails the same way on the basefce1aba8in this environment (16,326 passed, 2 failed there); none is new.- Boundaries: the bundle tool, analysis view, tool schema validity, environment-build CLI, setup skill and client lifecycle suites (123 passed), and every browser and scenario MCP and CLI boundary suite (68 passed).
- Real Chrome:
verify:browser:request(new: 22 request steps, the CSRF cookie and stored token authenticate seven writes and never reach the Evidence, two queued writes are refused, the build serves the three write tools, an unexpected status fails the step and cancels the rest, and a redirect to another origin is not followed, the other origin receiving nothing) andverify:browser:networkpassed.verify:browserstops at "Real Chrome stable capture identities did not reconcile", aninspect_web_pagecomparison this change does not touch, and fails the same way on the base. verify:browser:requestruns each CLI call under the shared package verifier command owner, which stops the CLI and the browser it launched on timeout, output overflow or failure before the lane's private root is removed, and reports a failed stop with the root retained;browserRequestVerifierOwnership.test.tsproves a still-running CLI's separately grouped browser is stopped before the root goes.
Scenarios without request steps are unchanged: the action union gains a member, the secrets refactor compiles the same patterns, the request-step limitation appears only with request steps, and the browser suites and the network lane pass.
End to end: environments from a prompt
Five self-hosted applications were brought up fresh and seeded on four disposable VMs, loopback only. Claude Code 2.1.310
(claude-sonnet-5-5 through a LiteLLM proxy) got this branch's packed CLI, registered with rea setup --client claude_code (the MCP server and the skill), and one prompt per application: build an agent environment of the app at
this URL, cover these entities, the instance is disposable but leave the seeded data as found, sign in as this user, the
password is in APP_PASSWORD, Chromium is here, write the bundle there. The prompt did not mention the skill. The agent
could use the REA tools, the skill and file tools; Bash and web tools were disabled.
Each bundle was then rebuilt with environment-build from its own evidence.json and this branch's tarball (the
registry's rea-agents@6.7.0 has no environment-serve); manifest.json and evidence.json were byte-identical to
the agent's in all five. It ran its smoke task, a retrieval eval generated from its manifest and capture, and the
application's recorded write eval with fireworks_ai/gpt-oss-120b and fireworks_ai/glm-5p3-flash, one run per task.
| App | Agent run | Recordings | Records | Tools (state / replayed reads) | Served writes | Smoke | Retrieval gpt-oss / GLM | Writes gpt-oss / GLM |
|---|---|---|---|---|---|---|---|---|
| Linkding | 182 s | 9 | 66 | 8 (7 / 1) | 0 | passed | 9/9 / 8/9 | 0/3 / 0/3 |
| Grocy | 253 s | 7 | 154 | 22 (16 / 0) | 6 | passed | 10/11 / 10/11 | 3/3 / 3/3 |
| Homebox | 391 s | 9 | 268 | 13 (11 / 2) | 0 | passed | 9/9 / 9/9 | 0/2 / 0/2 |
| Directus | 202 s | 7 | 100 | 25 (16 / 4) | 5 | passed | 12/12 / 11/12 | 2/2 / 2/2 * |
| Grafana | 143 s | 4 | 100 | 15 (12 / 2) | 1 | passed | 5/5 / 5/5 | 2/2 / 2/2 |
* The Directus write eval checks that the state holds exactly the provisions the scripted recording read (2); the agent's recording read all 5, so its count checks fail although every field check passes (0/2 with both models). With the counts relative to the state's own provisions it passes 2/2 with both.
- Write tasks passed exactly when the build served their tool, with both models.
- The skill loaded unprompted in every run. All five agents signed in through the form with the declared secret,
recorded the API with request steps, checked coverage, stopped by the withheld reasons and exported with
export_environment_bundle; each reported the credential-named warning pointers, and none published anything. - The scripted recordings the published environments were built from, built with this branch, read more (11, 39, 21, 31 and 35 state-backed reads) and serve 7, 3, 2, 2 and 0 writes; the prompted recordings serve 0, 6, 0, 5 and 1.
- Linkding's and Homebox's updates are withheld because a browser scenario carries no request times: the state learns
no clock, so an update the application stamps with a modification time is
unexplained-change. Linkding's scripted HAR serves the same PATCH with a clock rule. The coverage view now says so; recording request times on scenario network events would lift it. Linkding derives each bookmark's archive URL from its creation time, so its create iscontradictedin both recordings.
Earlier rounds: in the first, two agents hung on steps without timeout_ms and most guessed the scenario schema, so the
reference now starts from a whole scenario with a timeout on every step; the second (same five apps, before the review
fixes) served 1, 6, 0, 5 and 0 writes with the same eval pattern.
Review
Two independent reviews tried to refute the change.
- The first found a 302 that took a request step to another origin, page values left unredacted under a repeated or
colliding header label, short page values that rewrote ids and hosts across the capture,
User-Agentaccepted but dropped by Chrome, a request that could still be sent after its step timed out, an origin check separated from the send, a literal query re-encoded, line breaks accepted in header values, and CLI wording in MCP refusals. All are fixed and tested; redirects are now checked in real Chrome. - The second confirmed those fixes and found query entries mishandled after a path ending in
?or#, 3xx expected statuses that can never match, and padded stored values escaping redaction (fixed), plus the body cost of reading a page value, short values kept in plain text and the MCP dry run checking its directory (documented). In the prompted captures, only static assets lost bodies to the size limits.