* fix(mcp): name response-byte and startup ceilings in recovery
Name REA_MCP_MAX_RESPONSE_BYTES, its effective value, the 10485760 byte
floor, and process restart when the stdio response budget is hit, and
state that the setting does not raise Node's string ceiling. Name fixed
MCP startup phase budgets when those deadlines fire, and keep Codex's
30 second startup figure as client guidance.
Co-authored-by: morluto <morluto@users.noreply.github.com>
* fix(mcp): distinguish string ceilings and keep dossier recovery
Name REA's reserved string limit separately from Node's ceiling, restore
detail:summary discovery for an oversized complete evidence bundle, and
apply the doctor deadline sentence when binary_session itself times out.
Co-authored-by: morluto <morluto@users.noreply.github.com>
* fix(mcp): keep retained SDK errors inside the text budget
Quoted SDK validation diagnostics expand when recovery JSON is escaped
into MCP text. Shorten only that preview so the retained-evidence
reference still arrives as parseable error content.
Co-authored-by: morluto <morluto@users.noreply.github.com>
* fix(mcp): keep export advice when a summary exceeds the response budget
An oversized get_evidence_bundle summary no longer repeats detail: summary.
export_evidence_bundle still writes the retained bundle outside one MCP frame,
including when a larger byte budget cannot raise the string ceiling.
Co-authored-by: morluto <morluto@users.noreply.github.com>
* style(mcp): format the SDK diagnostic preview return
oxfmt rejected the fitted-preview ternary in EvidenceMcpServer.ts, which failed Static checks and the CI required gate.
Co-authored-by: morluto <morluto@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
Use compact MCP input advertisements by default while preserving explicit full, canonical validation, output contracts and complete Evidence. Require an explicit selected catalog for runtime identity and honor both profiles in CLI status.
Real ripgrep/Zellij SDK tasks, independent ELF oracle, actual client trials, profile parity workflows and CI are recorded in experiments/analysis-efficiency/mcp-input-catalog.md. SDK token estimates improve; actual Codex token savings and general latency/memory gains are not established.
Refs #5861
BREAKING CHANGE: MCP input advertisements default to compact. Set REA_MCP_INPUT_SCHEMA_PROFILE=full for complete advertised guidance and nested constraints.
* feat(environment): extract the initial state a capture's reads observed
Add the environment-state analysis view. From the parents the environment
manifest accepts, it extracts the data the capture's reads observed before
the first related write: entity types with an identity field (proved by a
path parameter or inferred from an identity name), entities with their
recorded values and provenance, documents for written non-entity paths, and
a render plan per read tool that is kept only when it reproduces the tool's
latest initial recordings from the state; every other read tool is a
recorded view with the reason. The Evidence is always inferred.
Response-body decoding moves to a domain owner shared with replay, exact
JSON parsing moves to the owner of the exact-number policy, the manifest and
state views share parent admission and record grouping, and
projectAnalysisView takes its accumulators as one object.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): bound state at redacted writes; infer identity only from items
A scenario write whose URL was redacted has no request line, so it was
dropped and reads after it counted as initial state. Such a successful
write now joins the timeline with an empty path, whose root relates to
every type, so it ends every type's state; a limitation counts them.
A list whose recordings were all empty passed the "every listed item
carries it" check vacuously and published an identity field nothing
showed; it now needs at least one item, else the view stays recorded
as no-identity.
A value that differed between two request lines of one item tool, such
as a query variant, was called volatile, and the gate then substituted
each recording's own value; only repeats of one request line now show
volatility, so differing lines go through fields-differ.
List membership is compared only while a list can still be the whole
collection, and merged entities are indexed by type and collection once,
so paged captures no longer cost pages times entities (8,000 pages of
40,000 entities: 5.7 s to 0.23 s).
Docs and the tool contract say scenario state keeps the capture's
redaction markers and how redacted writes bound it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): keep first sightings, one list pointer and linear field unions
Entities took the order of the first plan observed, and item plans run
before list plans, so a complete list read before its detail reads came
out reordered and lost "all" membership; each entity now keeps its
earliest recorded sighting.
A list plan reused the first recording's pointer for later recordings,
so a later response that was itself an entity wrapping a list was read
as that list; every recording must now locate the same list.
The field union scanned its ordered list for every key, quadratic in an
object's width; a set now answers membership.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): keep post-write values and partial lists out of the initial state
A value that changed between repeats of one request line was named
volatile whenever no write under the type's own root lay between, so a
write under another root (for example POST /api/dashboards/db before a
second GET /api/search) left its effect in the initial state. Any
recorded write between the two observations now leaves every tool of the
type recorded with reason fields-differ instead.
A list was published with membership "all" even when a total or count
envelope member exceeded the number it listed. Such a counter now makes
the membership "recorded".
Also drop a stale doc comment on ObservedEntity.order.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(environment): scope item-shaped and deleted written paths out of documents
State documents are held only for list-shaped paths a POST, PUT or PATCH
tool writes. Reads of item-shaped written paths and of paths that are only
deleted stay recorded views; the guide now says so and the spec defers them
to a later increment of #4933.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): count writes in flight when a value changes between reads
The volatility check compared write start positions with the two reads,
so a write already in flight when the earlier read started was missed and
its effect was named volatile. A write now counts when its interval
overlaps the span from the earlier read's start to the later read's end.
The check also ran once per changed field and scanned every write. It now
runs once per pair of observations, answered by a binary search over the
writes in timeline order and the latest end among each prefix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): catch cross-tool changes across writes; read only item totals
A value that changed across a write was caught only between repeats of
one tool's request line, so a list read before a write and an item read
after it put the post-write value into the initial state. Changes are
now classified per entity in timeline order after observation: across
any recorded write, a change between repeats of a line or between two
lines of the type rewrites the type (every tool recorded as
fields-differ); with no write between, a repeat's change is volatile.
The classification lives in environmentStateMerge.ts.
Any numeric member whose name contained "total" or "count" counted as
an item total, so total_pages, account_id or page_count made complete
lists partial. Item totals are now members whose name tokens include
total or count and none of page, pages or per; a string of digits counts
as a number; and a set has_more, next or next-page token member also
marks the list partial.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(environment): keep post-write observations out of initial state
* fix(environment): retain field provenance and bound state inference work
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <gensokyo-dev@users.noreply.github.com>
Project large local Evidence captures through the authenticated file-source boundary and add guarded opt-in sibling-pair path inference. Preserve request cancellation, resource ownership and file-relative validation diagnostics.
Validated relevant source and compiled CLI/MCP/environment boundaries, static/build/docs/catalog gates, and a real isolated Gitea 28.2.0 capture through CLI, default stdio MCP and HTTP replay. Current-head automated review passed; unrelated long-running platform jobs may still be pending.
Co-authored-by: Edgar Arakelyan <edgar.arakelyan@scale.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <gensokyo-dev@users.noreply.github.com>
* feat(analysis-view): project an environment manifest from web network captures
Add the environment-manifest view kind to inspect_analysis_view for retained
HAR Evidence: the primary origin's recorded request lines become a
deterministic tool surface (names, path templates, parameters with their
rule and basis, observed query names, value-free body shapes, record
pointers), with excluded groups, three fixed unknowns and the shared
projection budget. The four format rules always apply; after-uid-literal,
opaque-token and sibling-variance are opt-in and labelled, and literal
fold-in never touches a format-rule parameter. The api-surface accumulator
moves onto shared route, media-type and budget helpers without behaviour
change.
Refs morluto/rea#4933
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(web-network): check the manifest refusal text like the api-surface refusal
The real-capture verifier stringified the whole MCP result for the
environment-manifest refusal, so a declared "[" literal matched the result
envelope itself; compare the parsed refusal as the api-surface branch does.
Refs morluto/rea#4933
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(specs): record the hosted web-network lane run for the manifest view
Refs morluto/rea#4933
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(specs): reconcile the manifest view verification scope note with the hosted run
Refs morluto/rea#4933
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(analysis-view): bound body, route and name work in the environment manifest
Addresses the automated review findings on the PR: read request bodies at the
top level only instead of walking nested values; admit every literal route
part before materializing it; compare at most 32 segments in sibling-variance
and say so; keep stop words literal after a uid literal; keep the next
collision suffix per tool-name base; preserve a templated exclusion when two
groups print the same row.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(analysis-view): keep manifest folding, naming and media-type reads linear
Addresses the second automated review round: fold literal siblings through
set membership, key collision suffixes by the truncated name they share, and
charge a request media type before it is scanned.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(analysis-view): charge the manifest group key before building it
Addresses the third automated review round: the composite key copies the
method, origin, template and rules, so that copy is reserved before the key
is stringified.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(analysis-view): accept a +json suffix only on application media types
Addresses the fourth automated review round: `image/example+json` or
`text/example+json` no longer counts as JSON for origin selection, exclusion
or body shapes, matching the documented `application/json` and
`application/*+json` contract.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(analysis-view): require the capture parent's exact subject for a manifest
Addresses the fifth automated review round: the manifest cites the report
path and digest, so a parent whose subject is absent or names another path or
SHA-256 is refused at admission, as the capture view already does; the
api-surface view keeps its existing admission.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(analysis-view): share truncated suffix allocation work
Track suffix progress by the actual truncated prefix and suffix width. Distinct full names can collide only after suffix truncation; sharing that progress prevents quadratic rescanning while preserving shorter free names and existing allocation order.
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
* fix(analysis-view): merge only identifier-like siblings under sibling-variance
Addresses the sixth automated review finding: a bucket merged whenever one
literal carried a digit, folding plain words such as `mine` into the
parameter. The rule now partitions the bucket, merges the identifier-like
siblings and leaves the other literals as their own tools.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(analysis-view): combine qualifying sibling cohorts
Keep coded identifiers and a qualifying plural hyphenated cohort in one eligibility Set before checking the minimum merge size. Read each literal once and reuse the same membership at the consumer, preserving non-identifier routes and the three-hyphen minimum.
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
* docs(analysis-view): clarify opt-in sibling folding
State the additional sibling-variance selection in the guide, acceptance specification and MCP parameter description. Preserve the existing single-rule behavior and caller-selected inference rules.
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Expose filtered canonical module identities, retained source-range counts and mixed support through the existing shared CLI/MCP analysis view. Preserve full-item access, source selection, provenance and partial coverage while allocating only the selected page.
Verify exact facts and CLI/MCP identity on shipped Obsidian, record complete-task BPE/resource measurements and reuse owned-process supervision for optional tokenizer cleanup.
Fixes#4092
Part of #4027; decision #4091.
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
* fix(analysis-view): bound limitations and unknowns per view
Views of a JavaScript application or binary layout copied every parent
limitation into both limitations and unknowns, so a parent with tens of
thousands of limitations produced views as large as the complete record
and exceeded the MCP response budget even for a one-item page. Each view
now carries at most 64 distinct messages per field plus one line naming
the omitted count and the parent Evidence that retains the full list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(analysis-view): assert exact bounded message notices
Source inspection and targeted formatting only. Tests, builds, lint, typechecks and provider workloads were not run at the user's explicit request.
* fix(analysis-view): preserve coverage facts in bounded messages
Reserve JavaScript completeness and unknown-count facts before bulk limitations. Direct omission recovery to the source parent without assuming session retention. Preserve exact threshold assertions and add unrun exhausted-page/item and unretained-inline MCP regressions. Source inspection and targeted formatting only; tests/checks/builds/provider workloads/CI reruns skipped by explicit user request.
* fix(analysis-view): bound repeated message serialization bytes
* fix(analysis-view): let byte budget bound messages
* test(analysis-view): exercise byte-based message omissions
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Narrow WASM and EVM execution ports to file subjects and preserve supplied subject types at the shared execution factory. Authenticate format with path and digest before recording or projecting Evidence, using the existing WASM identity owner in both consumers.
Validation: focused WASM/EVM service and public MCP workflows passed, including malformed provider subjects and self-contained retained Evidence with contradictory format. Global typecheck/lint and documentation checks passed. Real WABT/Evmole execution and the full suite were not run.
Reuse the domain ZIP format list for session schemas, verify full Harmony
CLI/MCP lifecycles, and correct native view guidance and audited docs drift.
Refresh locked dependency notices and gate inventory/provenance consistency,
including notice inputs in the cached documentation gate.
Fixes#2454
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* fix(native): retain failed pwntools process and runtime cleanup
Keep failed worker/debugger cleanup owners and their private files until
verified process shutdown and successful directory removal. Retry cleanup
before another inspection and expose shutdown through the shared services,
CLI and MCP server. Serialize acquisition without delaying cancellation of
queued callers, and preserve primary diagnostics when CLI cleanup fails.
The original decoder reproduction left a live owned Node child with its
runtime deleted and no retained cleanup owner; the regression now verifies
retention, admission blocking, retries and eventual process/file cleanup.
Validation: 10 lifecycle/public CLI/MCP tests, 20 service tests, test
discovery 814/814, build:cached and full check including architecture guards.
Actual POSIX process/filesystem boundaries used injected Node workers on
macOS; real Linux pwntools/GDB provider parsing was not exercised.
* test(native): verify retained recorded-crash cleanup lifecycle
* fix(native): finalize diagnostic cleanup before publishing results
* fix(native): retire decoder stages before reading runtime evidence
---------
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
* feat(mcp): derive an API surface view from retained web network captures
* fix(mcp): fail closed on sensitive parents and budget api-surface path expansion
* fix(mcp): keep recorded api-surface paths and budget only templated segments
* fix(mcp): read api-surface paths only from canonical URLs and charge new values once
* fix(mcp): reject authority-less api-surface URLs and key groups by rule positions
* fix(mcp): label api-surface confidence from all groups and cap transient path memory
* fix(mcp): budget api-surface dedup sets and narrow detail outputs
Charge retained deduplication sets before allocation, advertise only
each analysis operation's summary from its detail output, and document
that any templated group makes every API-surface page inferred.
* fix(mcp): bound api-surface path text and record limitations
Charge retained literal path copies before allocation, publish only
valid media-type essences, and deduplicate examined record limitations
onto the view.
* fix(mcp): keep api-surface refusals and path splits inside the reply budget
Sensitive-parent refusals now use the token sensitive_values_unretained so declared literals are not echoed. A later path split is charged against bytes already retained, and the published example includes the HAR record limitation with its matching digest.
* fix(mcp): budget api-surface method text before group keys
Charge method and origin text against the reply budget before the group key is serialized. Document that samples merge only when method, origin, and exact segment rules match.
* fix(mcp): bound api-surface copies and control-character authorities
Reject authorities containing a tab or newline so WHATWG cannot move userinfo into the recorded path. Charge four path copies, two method and origin copies, and each distinct content type against bytes already retained.
* fix(mcp): preflight combined api-surface group charges
Total path, method, origin, and segment charges before retaining a new group. Track content types in a set so distinct MIME essences are not compared quadratically.
* fix(mcp): charge api-surface content-type sets on first use
Create the per-group content-type set when the first essence is retained, and charge it before allocation. Groups with no content type no longer keep an empty set.
* refactor(analysis): centralize API-surface ownership and admission
Own grouping, deduplication, and combined retained/temporary resource admission in one accumulator. Charge fixed group storage before allocation and materialize only the requested page.
Share recorded URL parsing between capture redaction and projection, preserving source paths and excluding ambiguous credential fields. Cover resource admission invariants and the compiled HAR-to-view workflow.
* fix(analysis): admit capture schema clone before parsing
* fix(javascript): share Electron mutation proof with wrapped calls
* test(mcp): align graph and catalog checks with current contracts
* fix(analysis): admit and validate portable capture views
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Supply schema-validated inspect_analysis_view arguments after completed native function Evidence is recorded and exceeds MCP delivery capacity. Preserve the full canonical result and its provenance. Verify exact source paging, export and following-call usability through real Ghidra stdio MCP.
Fixes#1849
Co-authored-by: Minghang Chamling <133689405+cold-atom@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
* feat(evidence): discover retained records without full payloads
* fix(evidence): guide oversized results to retained discovery
* style(tests): format retained discovery recovery assertion
Project authenticated inline Evidence failures at source.evidence while retaining their reasons and messages. Supply the matching request root to input issue projection so nested values remain identifiable.
Validation: both service entrypoints and SDK MCP integrity regressions, advertised-schema consumer suite, check:fast and architecture guards. No real-provider verification.
* fix(mcp): deliver tool errors within advertised schema contracts
Retain complete typed JSON error content while omitting non-conforming
structured errors at the public transport boundary. Recover oversized
diagnostics into Evidence before removing their private structured carrier.
Update schema-validated client readers and cover actual stdio delivery.
Closes#1261
Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)
* test(mcp): read typed errors in canonical Evidence regressions
* test(mcp): validate public error diagnostics in acceptance workflows
Deliver complete Evidence directly through MCP with required nullable analysis profiles and immutable per-server delivery settings. Bind selected environments at construction and require typed cleanup outcomes with retained retry ownership.
Remove test-only routing, duplicate session and snapshot metadata, producer-field repairs, and ambiguous provider arguments. Separate artifact byte identity from occurrence classification and retain typed source-map and partial native facts across CLI, MCP, and verifier consumers.
BREAKING CHANGE: obsolete Evidence envelopes, session aliases, incomplete snapshots, and missing normalized producer facts are no longer accepted.
Add explicit summary, facet, item and paged views of retained analysis Evidence without repeating analysis. Preserve captured observations, graph validation, artifact identity, coverage and complete default results.
Fixes#1050
Verified with focused source, CLI, pinned-SDK schema and MCP tests, serial static checks, and one complete native Windows Notion ASAR workflow. Real Hopper/Ghidra verification was not performed for this provider-independent projection change.
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
