From e9b21aadc5791e37a32a455fc6b6d65050a6a1c8 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Sun, 25 Oct 2026 09:55:29 +0800 Subject: [PATCH] docs(settings): explain runtime design and provider recovery --- .../skills/reverse-engineer-anything/SKILL.md | 5 +- .../references/android-applications.md | 7 +- CONTRIBUTING.md | 4 +- docs/android-analysis.md | 16 +- docs/client-configuration.md | 11 + docs/design/limits-and-recovery.md | 252 +++++++++++++++--- docs/electron-observation.md | 9 +- docs/ghidra-provider.md | 9 +- docs/installation.md | 36 ++- docs/javascript-artifact-reconstruction.md | 27 +- docs/mcp-contracts.md | 15 +- docs/native-investigation.md | 19 +- docs/wasm-artifacts.md | 6 + docs/web-network-captures.md | 15 +- 14 files changed, 352 insertions(+), 79 deletions(-) diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md index 7d25a526ad..d75f5548bb 100644 --- a/.agents/skills/reverse-engineer-anything/SKILL.md +++ b/.agents/skills/reverse-engineer-anything/SKILL.md @@ -384,8 +384,9 @@ selected provider. Do not treat those scopes as interchangeable. For a Ghidra startup timeout on a large native target, follow [deadline recovery](references/native-and-artifacts.md#ghidra-startup-deadline-recovery) before abandoning deep analysis or repeating the same cold import. The default -330 seconds is configurable; changing the server's environment requires a new -REA process. +330 seconds can be changed for new startup attempts through +`update_analysis_settings` using `ghidra.startup_timeout_ms`. Changing the +server's launch environment instead requires a new REA process. ## Work summary-first diff --git a/.agents/skills/reverse-engineer-anything/references/android-applications.md b/.agents/skills/reverse-engineer-anything/references/android-applications.md index 34fb4aa343..e1e324b51d 100644 --- a/.agents/skills/reverse-engineer-anything/references/android-applications.md +++ b/.agents/skills/reverse-engineer-anything/references/android-applications.md @@ -28,8 +28,11 @@ stdio. A full JDK, including `jdk.compiler`, is required on every host; Windows verification coverage. Calls use one worker and provider deadlines; optional `REA_JADX_HEAP_MIB` selects the JVM heap and `REA_JADX_OPERATION_TIMEOUT_MS` the per-operation deadline (integer milliseconds from 1 to 2147483647, default -120000; a change applies on the next process start; the MCP client's tool -timeout must also allow it). The APK is never launched, +120000). To change that deadline on a running MCP server, set +`jadx.operation_timeout_ms` through `update_analysis_settings`, then retry after +work and cleanup settle. Newly accepted operations use the override; accepted +work keeps its budget. Changing the launch environment requires a new process. +The MCP client's tool timeout must also allow the selected deadline. The APK is never launched, and no emulator is required. Manifest entries and decompiled text are static observations, not execution or diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ad5236520e..a153f19e1b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -47,7 +47,9 @@ compare equivalent real tasks before claiming an improvement. When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog. When adding or changing an admission ceiling, budget, or deadline, follow -[admission limits and recovery](docs/design/limits-and-recovery.md). +[analysis settings, limits, and recovery](docs/design/limits-and-recovery.md) +for ownership, operation snapshots, provider integration, and the steps for +adding a runtime setting. ## Reporting issues diff --git a/docs/android-analysis.md b/docs/android-analysis.md index f85a67cb15..d9dbef8027 100644 --- a/docs/android-analysis.md +++ b/docs/android-analysis.md @@ -415,11 +415,17 @@ export REA_JADX_OPERATION_TIMEOUT_MS=600000 The value must be a decimal millisecond count from 1 to 2147483647; anything else is refused before Java starts. REA reads it from the environment captured -at process start. A change applies on the next process start; an already -running server keeps the deadline it started with. The same deadline bounds -each upstream call. A timeout names `REA_JADX_OPERATION_TIMEOUT_MS`, the -effective milliseconds, and that range. A passing JADX configuration check in -`rea doctor` reports the effective `operation_timeout_ms`. Explicit +at process start. Changing that environment requires a new REA process. On a +running MCP server, set `jadx.operation_timeout_ms` through +`update_analysis_settings` to change the default for newly accepted operations; +queued or running operations retain their accepted budget. The same deadline +bounds each upstream call. Timeout recovery reports the effective milliseconds +and range, with `details.limit` identifying the runtime setting. An invalid +provider-local launch value can also be repaired by a runtime override; reset +exposes that invalid launch value again. See the +[settings design](design/limits-and-recovery.md#invalid-launch-configuration). +A passing JADX configuration check in `rea doctor` reports its launch-derived +`operation_timeout_ms`; it does not inspect another running server's overrides. Explicit cancellation still ends the operation immediately, and partial observations and owned-process cleanup are unchanged. Raising the deadline does not extend the bridge's 30-second startup handshake or the upstream **90-second** diff --git a/docs/client-configuration.md b/docs/client-configuration.md index 97faaf85c7..d86a296f6c 100644 --- a/docs/client-configuration.md +++ b/docs/client-configuration.md @@ -6,6 +6,17 @@ for the reviewed setup transaction and [client IDs](installation.md#supported-ag The versioned verification notes below describe their stated runs; live service acceptance is verified only where explicitly recorded. +## Runtime settings and client deadlines + +`update_analysis_settings` changes curated defaults inside the running REA +server. It does not edit the client registration, extend the client's tool-call +deadline, or recompute timeout values previously derived by `rea setup`. When +raising a REA startup or operation deadline, separately ensure the client allows +enough time for the full request. Use `inspect_analysis_settings` for the running +server's defaults; `rea doctor` checks launch configuration and cannot observe +another process's runtime overrides. See the +[settings design](design/limits-and-recovery.md) for ownership and lifetime. + ## Client tool names and discovery REA advertises canonical names such as `binary_session`. The MCP host decides diff --git a/docs/design/limits-and-recovery.md b/docs/design/limits-and-recovery.md index ac49fa1107..45ea762341 100644 --- a/docs/design/limits-and-recovery.md +++ b/docs/design/limits-and-recovery.md @@ -1,11 +1,99 @@ -# Admission limits and recovery +# Analysis settings, limits, and recovery -How to add a REA-owned admission ceiling, budget, or deadline, and what a caller -does when one is hit. This is a design rule, not an inventory. Ceilings that -should become settings are tracked in +This guide explains the runtime settings design for maintainers and developers: +who owns each value, when an update takes effect, how providers enforce it, and +how to extend the system. The public tool contract is in +[MCP contracts](../mcp-contracts.md#editable-analysis-settings); recorded runtime +checks and their limitations are in the +[verification report](../verification/runtime-analysis-settings.md). + +## Problem and design boundary + +A valid workload can exceed a conservative default. An oversized ASAR header +may prevent analysis of the entire archive; a slow Ghidra function may exceed +its decompile deadline. Requiring a new REA process to change either budget +interrupts the investigation and can discard reusable provider state. + +REA exposes a curated set of analysis defaults through MCP so callers can +adjust a reported limit and retry the same workload in the running runtime. +Each setting has a typed value, resource bounds, an enforcement owner, and a +defined application point. The design applies across archive admission, +provider startup, operation deadlines, and worker memory requests. + +The setter changes in-memory defaults. It does not edit arbitrary environment +variables, configuration files, MCP registrations, or provider installations. +Those mechanisms have different owners and lifetimes. It also cannot change a +client deadline, extend accepted work, release retained resources, or remove a +fixed representation limit. Exposing a setting requires a working enforcement +path, not just an entry in the tool schema. + +Further ceilings that should become settings are tracked in [#6176](https://github.com/morluto/rea/issues/6176). Naming honesty for limit errors is [#5910](https://github.com/morluto/rea/issues/5910). +## Ownership and data flow + +| Owner | Responsibility | +| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [Domain definitions](https://github.com/morluto/rea/blob/main/src/domain/analysisSettingDefinitions.ts) and [types](https://github.com/morluto/rea/blob/main/src/domain/analysisSettings.ts) | Exact setting IDs, value types, shared bounds/defaults, units, provider/operation associations, application points, and structured limit metadata. Numeric constants stay with their domain owner. | +| [Launch parser](https://github.com/morluto/rea/blob/main/src/config/analysisSettings.ts) | Parse selected launch values once, preserving an invalid value as a typed configuration failure rather than substituting a default. | +| [AnalysisSettings](https://github.com/morluto/rea/blob/main/src/application/AnalysisSettings.ts) | Own one runtime's launch baseline, immutable override revisions, atomic updates, inspection, and accepted-operation snapshots. It does not own provider lifecycle or cleanup. | +| [Composition](https://github.com/morluto/rea/blob/main/src/main/eagerRuntime.ts) and [typed configuration views](https://github.com/morluto/rea/blob/main/src/config/runtimeAnalysisSettings.ts) | Inject the same settings reader into providers and workflows; translate overrides into the configuration each consumer understands. | +| Provider or resource owner | Resolve the effective budget, enforce it at the applicable boundary, and attach the actual value and known source to failures. | +| [MCP registration boundary](https://github.com/morluto/rea/blob/main/src/server/EvidenceMcpServerCore.ts) | Enter the operation's settings scope before invoking its handler. | +| [Settings contracts](https://github.com/morluto/rea/blob/main/src/contracts/analysisSettingsContracts.ts) and [tool registration](https://github.com/morluto/rea/blob/main/src/server/registerAnalysisSettingsTools.ts) | Validate named requests and deliver inspection/update results without requiring an open target or healthy analysis provider. | + +The normal path is: composition creates the store, an MCP handler captures its +current revision, and providers read that snapshot when resolving budgets. +If an owner rejects work, it reports the effective limit through the shared +error projection. A subsequent settings update publishes a new revision for +later handlers. It does not reach into the failed operation or retry it. + +### Accepted-operation snapshots + +`AnalysisSettings.run()` installs the current immutable snapshot in +`AsyncLocalStorage`. Its `read` function returns that snapshot inside the +operation and the latest revision outside it. Providers depend on the reader +contract rather than on the mutable store or an ambient environment lookup. + +Acceptance here is entry into the registered handler's settings scope, not +arrival of bytes at the MCP transport. Work queued or awaited within that scope +keeps its snapshot. Deferred server initialization must still dispatch through +the canonical scoped handler. + +| Event | Revision used | +| --------------------------------------------------------- | ----------------------------------- | +| Operation A enters its handler, then waits for a provider | Current revision 0 | +| A settings update publishes revision 1 | Future handlers can read revision 1 | +| Operation A resumes and starts its worker | Its accepted revision 0 | +| Operation B enters its handler | Revision 1 | + +This gives each operation stable budget semantics even if another connection +updates settings while it waits. `next_worker` and `next_startup` describe where +an accepted value is applied; they do not cause workers or startups belonging +to an older operation to adopt a newer revision. + +Inspection deliberately reads the store's current defaults, rather than the +calling operation's snapshot. It reports `active_values.status: "not_observed"`: +an inspection result cannot establish the budget or remaining time of an +already-running worker. + +### Composition and embedding + +The MCP runtime creates one store and supplies its reader to provider factories. +`createServer` uses a supplied store or creates one, and wires its default +providers to that store. An embedder supplying a custom binary session or other +provider must also inject the same reader into that provider's composition and +pass the store to `createServer`. Merely exposing the settings tools cannot +make a custom provider consume updates. + +Keep default factories, explicitly supplied providers, transport composition, +and deferred handler dispatch covered separately where their wiring differs. +Resolve settings at the operation boundary; copying a runtime value into a +provider constructor would freeze it at launch. For entrypoints outside the +MCP wrapper, an embedder needing the same snapshot guarantee must establish +the operation scope with `AnalysisSettings.run()`. + ## Name a default, then a setting when operators need headroom Put the number in a named constant with a unit. Do not leave it as a magic @@ -18,9 +106,10 @@ curated runtime IDs to those same bounds, defaults and application points. Environment parsers and recovery text stay under `src/config/`; register launch schemas on `environmentSchema` so startup and doctor agree. Add launch settings to `SELECTED_REGISTRATION_SETTINGS` when setup should forward them. Unset keeps -today's behavior. An invalid launch value is `configuration_invalid`, never a -silent fallback. An invalid runtime write is `invalid_request` and changes -nothing. +today's behavior. An invalid launch value remains a configuration failure, +never a silent fallback; its validation boundary determines whether MCP can +start (see [invalid launch configuration](#invalid-launch-configuration)). An +invalid runtime write is `invalid_request` and changes nothing. For example, `regex.matching_deadline_ms` and its launch seed `REA_REGEX_MATCHING_DEADLINE_MS` share a 5000 ms default, 1 ms floor and @@ -36,7 +125,8 @@ raise REA's reserved string limit or Node's string ceiling. The setup allowlist does not copy this setting. A new headroom setting should, following the regex deadline. -A per-call budget is not an environment setting. JavaScript `max_heap_mb` +A per-call budget can have a runtime default without having an environment +setting. JavaScript `max_heap_mb` (CLI `--max-heap-mb`) is an integer from 128 to 16384 MiB, default 1024. It applies on the next analysis, which starts a new worker, and does not restart REA. It does not raise the parent result budget, transfer ceilings, or fixed @@ -61,6 +151,14 @@ an exact `setting` and either `action: "set"` with a schema-validated `value`, o optional compare-and-set guard; conflicts and duplicate keys change nothing. An unchanged write does not advance the revision. +The revision tracks override state, not only effective numeric values. Setting +an explicit override equal to the launch value still changes its source to +`runtime` and advances the revision. Reset removes that override, even if the +effective number stays the same. Validation, revision checks, and duplicate-key +checks all happen before publishing the replacement snapshot; a rejected batch +cannot partially apply. Clients coordinating writes can supply +`expected_revision`; without it, valid updates apply to the current state. + | Runtime setting | Application point | | ------------------------------------------------------------- | ---------------------------------------------------------------------- | | `asar.max_header_bytes` | Next archive operation, including full JavaScript application analysis | @@ -72,20 +170,30 @@ An unchanged write does not advance the revision. | `javascript.analysis_timeout_ms`, `javascript.max_heap_mb` | Next source analysis / new JavaScript worker | | `wasm.command_timeout_ms` | Next WABT operation | -Explicit per-call arguments take precedence over runtime defaults, then launch -configuration and built-in defaults. Each accepted MCP operation retains one -immutable snapshot, including queued work. A write does not extend active work, -relaunch a provider, clear retained cleanup, or invalidate successful Evidence. -Transient Ghidra decompile failures remain uncached so a raised deadline can -recover on the same Program. Cached ASAR headers are checked against the new -operation's admission, including after a decrease. +### Precedence and lifetime + +Where a setting supports an explicit per-call argument, precedence is: + +1. The caller's explicit argument. +2. The accepted operation's runtime override. +3. Valid launch configuration, where that setting has an environment seed. +4. The built-in default when launch configuration is absent. + +An invalid launch value does not fall through to the built-in default. Preserve +omitted caller arguments in Evidence: resolving an internal default must not +rewrite the request as though the caller supplied it. `null` is the explicit +unbounded value for `ghidra.decompile_timeout_seconds`; it is not a substitute +for an unknown or omitted value elsewhere. + +A write does not extend active work, relaunch a provider, clear retained +cleanup, or invalidate successful Evidence. Overrides belong to one REA runtime, survive target close/open, and disappear when REA exits. Connections sharing that runtime share its overrides. No files, -ambient environment, registrations or upstream software are edited. Embedders -composing their own providers must inject the same `AnalysisSettings.read` into -provider composition and pass that store to `createServer`; custom provider -implementations own their integration with these controls. +ambient environment, registrations or upstream software are edited. Overrides +are runtime-wide, not scoped to a target or an individual connection. + +### Failure and retry An ASAR admission error supplies `details.limit` with the exact setting, effective value, units, bounds, scope, application point, restart requirement @@ -104,8 +212,48 @@ No preliminary inspection or source subset is required. The setter does not retry automatically. If an explicit per-call value caused the failure, correct that argument; raising a runtime default cannot override it. +The resource owner constructs `details.limit` from the registry plus the budget +it actually enforced. Metadata includes `agent_editable: true`, +`scope: "rea-runtime"`, and `rea_restart_required: false`. Optional +`required_value`, `source`, and `parameter` fields are supplied only where known +or applicable. Do not infer a source because a value happens to equal a default. +Error copying, attribution, redaction, and delivery must preserve these facts. +Human-readable remediation explains the same failure; clients should not need +to parse that sentence to identify an editable setting. + +Cache policy is part of enforcement. Successful reusable analysis can survive +a budget update. Transient Ghidra decompile failures remain uncached so a raised +deadline can recover on the same Program. Admission checks still apply to reused +data: lowering an ASAR limit must reject a cached oversized header. Neither +raising a limit nor clearing a transient analysis failure authorizes skipping +cleanup or changing the identity of the provider session. + +### Invalid launch configuration + +Launch validation has two relevant boundaries. Settings validated by global +configuration parsing can prevent REA from starting; an MCP setter cannot +repair a server that is not running. Correct the launch environment or +registration, use `rea doctor` for its configuration checks, and start REA again. + +Provider-local validation can leave MCP available. For example, an invalid +`REA_JADX_OPERATION_TIMEOUT_MS` is inspectable with `source: "invalid"`, a +`configuration_error`, and omitted `value` and `launch_value`. A valid runtime +override repairs the effective JADX timeout without editing that launch seed. +Inspection then reports the runtime value while retaining the launch error. +Reset exposes the invalid launch state again; it does not repair or replace it +with the built-in default. Provider availability and execution must resolve the +same effective setting so readiness changes along with repair and reset. + ## Keep a fixed ceiling honest +Inspection's separate +[constraints](https://github.com/morluto/rea/blob/main/src/server/analysisSettingsConstraints.ts) +describe operator-owned, external, and fixed controls. REA can report its +configured response budget and parent heap limit; client deadlines and receive +buffers remain explicitly unknown. These are not entries accepted by the +runtime setter. Raising a worker heap request cannot raise the parent heap, +and raising a provider deadline cannot make the client wait longer. + If the ceiling must stay fixed, say so in the failure. Name the limit and the effective value, say this release has no runtime override, and name what does not raise it. Do not invent a setting. @@ -127,9 +275,10 @@ The string the caller sees includes: - whether a restart is required - what the change does not raise -Invalid launch settings never take this path. They are `configuration_invalid`. -Doctor reports `runtime-configuration:` and says to correct it in the -shell or the MCP registration, then rerun. +Distinguish an invalid configuration from a valid budget exhausted by work. +Preserve the configuration failure and its setting identity; apply the +[launch validation boundary](#invalid-launch-configuration) when describing +whether live repair is possible. ## What agents do @@ -157,8 +306,10 @@ request to change it. - `resource_constraint` for capacity, when `retryable` is true: wait for the active admission to finish, then retry within the reported limits. Do not restart REA to free the slot. -- `configuration_invalid`: correct the named setting in the environment that - starts REA, then start REA again. Run `rea doctor` for the +- `configuration_invalid`: when MCP remains available and the failure belongs + to a curated provider-local setting, inspect and set a valid runtime override. + Otherwise correct the named setting in the environment that starts REA, then + start REA again. Run `rea doctor` for the `runtime-configuration:` check. Re-run `rea setup` for that client when the setting is on the registration allowlist, then reconnect so the new process sees it. @@ -169,17 +320,58 @@ A restart is required to adopt a changed launch environment or registration. The curated runtime settings above require no REA restart. It is not the first step for a transient MCP drop, a capacity wait, or retained cleanup. -## Tests +## Adding a runtime setting -Lock the unset default, one accepted override, rejected floor, maximum, and -syntax, and the recovery sentence through `parseEnvironment` and `parseConfig`. +1. Identify the enforcing owner and the real resource constraint. Decide whether + callers need a per-call argument, a runtime default, a launch-only control, + or a fixed limit. A registry entry must not advertise authority REA lacks. +2. Define the value type and named numeric bounds with the domain owner. Add the + ID and metadata to `analysisSettings.ts` and `analysisSettingDefinitions.ts`. + Reuse those bounds in runtime validation, launch parsing, and error metadata. + Choose the application point and provider/operation associations from actual + consumers; a shared control need not apply to every provider. +3. If there is a launch seed, add its parser and appropriate configuration/doctor + validation. Decide whether an invalid value blocks the runtime or only the + provider. Add it to setup's `SELECTED_REGISTRATION_SETTINGS` when it should + be forwarded; runtime editability does not automatically propagate it. +4. Thread the shared reader through every applicable composition path. Resolve + the effective value within the accepted operation, honor explicit arguments, + and preserve their omission in Evidence. Keep availability checks consistent + with execution. A worker heap request must not be described as a measured + process memory limit. +5. Enforce the value at its resource boundary and construct structured limit + facts there. Preserve those facts through shared error projection and copies. + Check cache reuse, lowered admission, transient failures, and retained cleanup + against the same ownership rules. +6. Update the public contracts and affected guides, then regenerate catalogs + and packaged skill metadata using the repository commands. Keep externally + owned and fixed controls in the separate inspection constraints, not in the + editable registry. Validate both store semantics and a public recovery path. + +## Tests and verification + +Cover the unset default, accepted overrides, rejected floor/maximum values, +and invalid syntax through the owning parser and configuration boundary. [`regexMatchingDeadline.test.ts`](https://github.com/morluto/rea/blob/main/src/config/regexMatchingDeadline.test.ts) -is the pattern. Project the failure and assert `configuration_invalid` for a -bad value and the recovery wording for a hit. +is one example. Assert error codes, setting IDs, values, units, provenance, +application points, and resulting behavior. Check that adapters deliver the +canonical recovery guidance, with independent assertions for its underlying +facts. Avoid freezing prose or deriving all expected values from the same +registry used by the implementation; either can hide the wrong failure. +See [test design](../testing.md#test-design-and-pruning) for the assertion policy. Runtime checks cover atomic validation, stale writes, reset, runtime isolation, accepted-operation snapshots, and explicit argument precedence. Public MCP acceptance must reject an oversized ASAR, raise admission, and successfully retry the same complete archive on the same connection. Lowering admission must also refuse previously cached headers. The real Ghidra timeout verifier checks -recovery after a live update without replacing its session. +recovery after a live update without replacing its session. Also cover invalid +provider-local launch configuration, repair/reset availability, default and +custom composition paths, and preservation of structured limits through error +copies. Gates used to overlap work and updates need bounded acknowledgements +and release/join in `finally`. + +The [verification report](../verification/runtime-analysis-settings.md) separates +public MCP checks, simulated provider coverage, real Ghidra execution, and +measured task costs. It does not establish real-provider behavior for every +editable setting or active worker telemetry. diff --git a/docs/electron-observation.md b/docs/electron-observation.md index c462b21f78..810b1eb8b2 100644 --- a/docs/electron-observation.md +++ b/docs/electron-observation.md @@ -305,8 +305,13 @@ capture provides them. ASAR observation reads the header through one open file handle and validates its declared lengths against the archive size. Header/index admission is the same -budget used by static ASAR reads: 16 MiB / 16777216 bytes by default, or -`REA_ASAR_MAX_HEADER_BYTES` (at most 256 MiB) when set. An oversized header is +budget used by static ASAR reads: 16 MiB / 16777216 bytes by default, seeded by +`REA_ASAR_MAX_HEADER_BYTES` (at most 256 MiB). In MCP, +`update_analysis_settings` can override `asar.max_header_bytes` for new +operations without restarting REA. Cached headers are checked against each +operation's admission, including after a decrease. See the +[settings design](design/limits-and-recovery.md#failure-and-retry). +An oversized header is not parsed; selecting a member path does not bypass admission, and the page stays unauthorized. The header cache separately retains at most that many bytes of already-admitted encoded headers across eight archives. Evicting that cache does not raise admission. diff --git a/docs/ghidra-provider.md b/docs/ghidra-provider.md index f1322f4b33..8c9013fd0d 100644 --- a/docs/ghidra-provider.md +++ b/docs/ghidra-provider.md @@ -187,7 +187,7 @@ profile stages before retrying their retained descriptors. | Concern | Ghidra contract | | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Decompiler lifetime | One persistent `DecompInterface` is opened for the imported Program and disposed during bridge shutdown. Native decompilation has no fixed per-function deadline unless `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS` sets one; a decompile that reaches it fails only that request with `remote_code` `decompile_timed_out`, the Program and session stay open, and the next decompile uses a fresh decompiler process. External functions or functions without bodies return `null`; cancellation and native failure remain distinct. | +| Decompiler lifetime | One persistent `DecompInterface` is opened for the imported Program and disposed during bridge shutdown. Native decompilation has no fixed per-function deadline unless the launch setting `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS` or runtime override `ghidra.decompile_timeout_seconds` sets one; a decompile that reaches it fails only that request with `remote_code` `decompile_timed_out`, the Program and session stay open, and the next decompile uses a fresh decompiler process. External functions or functions without bodies return `null`; cancellation and native failure remain distinct. | | Serialization | A FIFO sends one Program request at a time. Caller cancellation removes queued work promptly and is passed to active socket waits. This is an adapter safety commitment, not a claim that every Ghidra API is thread-safe. | | Function identity | Every function result carries the entry address and Ghidra FunctionManager classification for external, thunk, and resolved thunk target. These are observations, not proof that unresolved targetless calls have been recovered. | | Assembly and pseudocode | Assembly is complete Ghidra Listing text; pseudocode is Ghidra decompiler output. Neither is original source, and cross-provider comparison never treats Hopper and Ghidra text as equal or unequal semantic facts. | @@ -197,6 +197,13 @@ profile stages before retrying their retained descriptors. | P-code value flow | Function dossiers include up to 3,000 high-p-code operations, at most 64 inputs per operation, and 12,000 def-use edges. Results identify memory reads/writes, branches, and calls by p-code opcode, and mark decompiler-dead operations. Large results report truncation, an omitted-operation lower bound, and known omitted-input/edge counts. The dossier is an intra-function graph; `trace_native_values` composes derived parameter/argument and return/output dependencies across resolved calls. It does not resolve memory aliasing, call side effects, runtime behavior, persistent state, or business meaning. | | Result extent | Function instruction scans and native API boundary observations remain complete. P-code flow is the explicit bounded exception; its count fields distinguish exact retained counts from lower-bound or known-omitted counts. Caller cancellation and provider failures remain distinct. | +Runtime updates apply to newly accepted operations on the existing Program. +They preserve accepted budgets and do not clear cleanup blockers or change the +client deadline. Transient decompile timeout failures remain uncached so the +same function can be retried after raising the limit. See the +[settings design](design/limits-and-recovery.md) for snapshots, precedence, +and provider integration. + Function extents and instruction completeness refer to Ghidra's analyzed body. An inferred no-return call can terminate that body before subsequent bytes. Pseudocode, assembly, procedure info, instruction windows, incoming and outgoing diff --git a/docs/installation.md b/docs/installation.md index 8535e2cec1..62507fec5a 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -809,7 +809,10 @@ Hopper startup allows 120,000 ms for launch, initial analysis, and authenticated bridge readiness. For large targets, set `REA_HOPPER_STARTUP_TIMEOUT_MS` to a decimal integer from 1 to 2,147,483,647 milliseconds in the environment that starts REA. CLI and MCP use the same setting; invalid values fail configuration -validation. An MCP client's own request timeout must also allow enough time. +validation. In MCP, `update_analysis_settings` can set +`hopper.startup_timeout_ms` for the next startup attempt without restarting REA; +accepted work keeps its budget and retained cleanup must still complete. An +MCP client's own request timeout must also allow enough time. `open_binary` validates and selects a target without waiting for native provider readiness. During Hopper startup, `binary_session.provider_operation_health` reports `busy` at stage `launch`; `analysis_activity` describes bridge requests @@ -1179,19 +1182,25 @@ can time out earlier even when Ghidra would complete within its startup deadline See [Ghidra first-query deadlines and recovery](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery) for client options and the close/reopen recovery flow. -The 330-second default is configurable. Configuration is read when REA starts: -changing an environment variable in another shell, or closing and reopening a -binary in the same MCP server, does not change that server's startup allowance. +The 330-second default can be changed on a running MCP server by setting +`ghidra.startup_timeout_ms` through `update_analysis_settings`. The next accepted +operation that starts a provider uses that value; an accepted startup keeps its +original budget. Changing an environment variable in another shell, or closing +and reopening a binary, does not change the launch configuration already read +by that server. For a CLI retry with an example 30-minute allowance: ```sh REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000 rea analyze /absolute/path/to/app.exe --provider ghidra --json ``` -For MCP, export useful Evidence before restarting, close the binary and resolve -any cleanup failure, update the environment in the client's REA server launch -configuration, then restart REA and reconnect before reopening the target. Allow -startup plus the query in the client's own tool-call deadline. Every retry uses +For MCP, update the runtime setting and resolve any retained cleanup before the +next startup attempt. Follow the error's close/reopen guidance where needed; +the settings update does not clear cleanup blockers. To change the launch +environment instead, export useful Evidence, close the binary and resolve any +cleanup failure, update the client's REA server launch configuration, then +restart REA and reconnect. Allow startup plus the query in the client's own +tool-call deadline; runtime updates do not change that deadline. Each new startup attempt uses a fresh ephemeral import; smaller result requests do not reduce the prerequisite auto-analysis. A larger time allowance does not raise the JVM heap limit or guarantee completion. Keep [heap and host CPU controls](#ghidra-heap-and-cpu-controls) @@ -1218,9 +1227,14 @@ fails with `provider_timeout` and that message; other Ghidra tools stay available. If the caller also cancels while the timeout settles, the session still closes and the message says so. Session reuse concerns an active MCP or library session; a standalone CLI command still closes its own session when it -finishes. The limit is read when a Ghidra session -starts, and Codex/Kimi registrations derived by `rea setup` add it to the tool -deadline as a single-decompile estimate. A bulk call can need that limit for +finishes. In MCP, `ghidra.decompile_timeout_seconds` can override the launch +value for newly accepted operations on the existing Program; explicit `null` +removes the deadline. A transient timeout remains uncached so the same function +can be retried after an update. See the +[settings design](design/limits-and-recovery.md#runtime-settings-through-mcp). +Codex/Kimi registrations derived by `rea setup` add the selected launch value +to the tool deadline as a single-decompile estimate; a runtime update does not +change that registration. A bulk call can need that limit for every attempted function plus listing and file work; select the client deadline for the workload, split batches, use `decompile_all` limit/resume, or reduce `trace_native_values` `max_functions` to select fewer serial decompiles. Ghidra's own interactive default is 30 seconds. Unresolved computed calls diff --git a/docs/javascript-artifact-reconstruction.md b/docs/javascript-artifact-reconstruction.md index 6475f90793..4a4af6564a 100644 --- a/docs/javascript-artifact-reconstruction.md +++ b/docs/javascript-artifact-reconstruction.md @@ -140,17 +140,24 @@ returns an unavailable error with the retained inventory, rather than claiming the selected source does not exist. Observed exact file candidates keep their identity; unknown traversal cannot establish a unique application root. -ASAR header/index admission defaults to 16 MiB / 16777216 bytes. Set -`REA_ASAR_MAX_HEADER_BYTES` in the environment that starts REA to a decimal -byte count of at most 268435456 (256 MiB) to raise or lower it; restart REA -after changing it. Invalid values are configuration errors, never a silent -fallback. The archive index is read before any member can be selected, so +ASAR header/index admission defaults to 16 MiB / 16777216 bytes. In MCP, use +`update_analysis_settings` to set `asar.max_header_bytes` to an integer byte +count from 1 to 268435456 (256 MiB), then retry the same full-archive request. +The update applies to new operations without restarting REA; cached headers +must also satisfy the new admission. For CLI or future MCP processes, set +`REA_ASAR_MAX_HEADER_BYTES` in the launch environment. Changing that environment +requires a new REA process. Invalid launch values are configuration errors, +never a silent fallback. See the +[settings design](design/limits-and-recovery.md#failure-and-retry) for recovery +metadata, precedence, and reset behavior. + +The archive index is read before any member can be selected, so `source_paths`, `--source-path`, `detail: "summary"`, `max_heap_mb`, and available host memory do not bypass or raise it. When the header exceeds the -effective allowance, the failure names it, whether it came from the default or -the setting, and the setting's limits. Alternatively, analyze an extracted +effective allowance, the failure reports the value, its known source, and the +setting's limits. Alternatively, analyze an extracted application directory, which does not admit an ASAR header. Repeating the unchanged -request fails the same way. +request without changing admission fails the same way. ## Read the analysis @@ -737,6 +744,10 @@ deadline per source, including startup and fact transfer. Select `max_heap_mb` and `analysis_timeout_ms` through MCP, or `--max-heap-mb` and `--analysis-timeout-ms` on the dedicated CLI command. These choices control the analyzer; the caller's `NODE_OPTIONS` still controls the CLI/MCP process heap. +For defaults across new MCP operations, use `update_analysis_settings` with +`javascript.max_heap_mb` or `javascript.analysis_timeout_ms`. Explicit per-call +arguments take precedence, and accepted work keeps its original settings. See +the [settings design](design/limits-and-recovery.md#precedence-and-lifetime). V8 old-space limits do not impose a process RSS limit. When REA runs on Bun, the analyzer ignores the V8 heap flags; resource failures then report `worker_heap_limit_bytes` as null instead of a requested or V8-reported limit. diff --git a/docs/mcp-contracts.md b/docs/mcp-contracts.md index b7d4774a86..8ff488ee50 100644 --- a/docs/mcp-contracts.md +++ b/docs/mcp-contracts.md @@ -943,6 +943,9 @@ target close/open, and vanish when REA exits. Explicit per-call arguments take precedence. Neither tool changes host deadlines or resolves retained cleanup. See [runtime settings and recovery](design/limits-and-recovery.md#runtime-settings-through-mcp) for the full setting list, reset/revision semantics and ASAR retry example. +For implementation ownership, accepted-operation snapshots, provider wiring, +and extension guidance, see the +[maintainer design](design/limits-and-recovery.md#ownership-and-data-flow). Ghidra and Hopper startup updates apply to the next startup attempt without restarting REA. Ghidra decompile timeout updates apply on the existing Program. @@ -1049,11 +1052,13 @@ the current client, so it is not a fresh-start recovery. On the same connection: `provider_id: "ghidra"`, then retry the first query with an appropriate client deadline. The connection and selected provider do not need to change. -If REA's provider startup allowance also needs raising, preserve Evidence and -close as above, then update `REA_GHIDRA_STARTUP_TIMEOUT_MS` in REA's launch -environment and restart the server. Reconnect the MCP client before reopening -the target. Configuration is read at process startup; close/reopen on the same -connection cannot apply a changed environment. See the +If REA's provider startup allowance also needs raising, set +`ghidra.startup_timeout_ms` through `update_analysis_settings` before the next +startup attempt. This needs no REA restart and does not change accepted work, +clear cleanup blockers, or extend the client's deadline. Alternatively, change +`REA_GHIDRA_STARTUP_TIMEOUT_MS` in REA's launch environment and restart the +server, preserving Evidence and closing as above. Close/reopen on the same +connection cannot apply a changed launch environment. See the [CLI retry example](installation.md#ghidra). This close/reopen flow was exercised on one real Linux stdio connection after a diff --git a/docs/native-investigation.md b/docs/native-investigation.md index 364d213166..d1dec81301 100644 --- a/docs/native-investigation.md +++ b/docs/native-investigation.md @@ -284,17 +284,22 @@ stops matching while leaving the Hopper API available. The deadline is 2147483647 ms. Unset uses 5000 ms. A larger value is refused because Node schedules a delay past that signed 32-bit bound as 1 ms. Invalid values are configuration errors and do not fall back to the default. The setting is read -from the environment that starts REA, so a change needs a restart. A deadline -failure names the effective milliseconds, whether they came from the default -or the setting, the default, and the maximum. The worker old-generation heap -does not raise it. Kuna regex search uses the same deadline. The worker +from the environment that starts REA; changing that environment needs a restart. +In MCP, `regex.matching_deadline_ms` overrides the deadline for newly accepted +searches without restarting REA. A deadline failure names the effective +milliseconds, known source, default, and maximum. The worker old-generation heap +does not raise it. IDA and Kuna use these same regex controls; Ghidra's Java +matcher does not. The worker heap request is `REA_REGEX_WORKER_HEAP_REQUEST_MIB`, a decimal integer from 7 to 16384 MiB. Unset uses 128 MiB. A request below 7 MiB aborts the worker during startup, and a larger value is refused. Invalid values are configuration errors and do not fall back to 128. The setting is read from -the environment that starts REA, so a change needs a restart. A heap failure -names the effective request, whether it came from the default or the setting, -the default, the floor, and the maximum. A positive Node.js +the environment that starts REA, while MCP `regex.worker_heap_request_mib` +can override it for workers belonging to newly accepted searches. Use +`update_analysis_settings` for either runtime control; accepted work keeps its +budget. See the [settings design](design/limits-and-recovery.md). +A heap failure names the effective request, known source, default, floor, and +maximum. A positive Node.js `--max-old-space-size` on the process that starts REA, including `NODE_OPTIONS`, overrides that request and sets the effective heap. `--max-old-space-size-percentage` takes precedence over that flag in diff --git a/docs/wasm-artifacts.md b/docs/wasm-artifacts.md index 8b7f9322eb..429da7689f 100644 --- a/docs/wasm-artifacts.md +++ b/docs/wasm-artifacts.md @@ -115,6 +115,12 @@ actual selected deadline and these recovery controls; increasing only the client deadline does not change the provider budget. No server restart is required to change this per-call setting. Resolve incomplete cleanup before another attempt. +To change the default across new MCP operations, set `wasm.command_timeout_ms` +through `update_analysis_settings`. An explicit `command_timeout_ms` still wins; +correct that argument if it caused the timeout. Runtime defaults do not insert +omitted arguments into Evidence parameters. See the +[settings design](design/limits-and-recovery.md#precedence-and-lifetime). + For example, `rea inspect-wasm-artifact /artifacts/module.wasm --command-timeout-ms 60000 --format json` uses a 60-second per-command deadline; the equivalent MCP input is `{"path":"/artifacts/module.wasm", diff --git a/docs/web-network-captures.md b/docs/web-network-captures.md index 3498b99418..04f37dc425 100644 --- a/docs/web-network-captures.md +++ b/docs/web-network-captures.md @@ -876,9 +876,11 @@ HAR, SAZ and qlog run in an owned Node old-generation heap with one V8 worker. `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` is a decimal integer of mebibytes from 8 to 16384. Unset uses 192, the previous constant. Invalid values fail configuration and name the setting; they do not fall back to 192. The setting -is read from the environment that starts REA, so a change needs a restart. -A memory failure names the effective mebibytes, whether they came from the -default or the setting, the default, the floor, and the maximum. The decoder +seeds the launch value. In MCP, set `capture.decoder_old_generation_mib` +through `update_analysis_settings` for decoders belonging to newly accepted +operations, without restarting REA. Changing the launch environment still +requires a new process. A memory failure names the effective mebibytes, known +source, default, floor, and maximum. The decoder passes that value as `--max-old-space-size` and removes inherited `NODE_OPTIONS`, so additional host memory and `NODE_OPTIONS` do not raise the heap. The 8 MiB semi-space stays fixed; a smaller old generation aborts Node @@ -900,8 +902,11 @@ hosts enforce no decoder heap limit, so raising the setting does not bound them. Observed decoder memory exhaustion returns `resource_constraint`, with the selected capture, applicable limits and retained process diagnostics. Raising -host memory or `NODE_OPTIONS` does not raise the old-generation heap. Set -`REA_CAPTURE_DECODER_OLD_GENERATION_MIB` and restart REA. +host memory or `NODE_OPTIONS` does not raise the old-generation heap. Update +`capture.decoder_old_generation_mib` through MCP, or set +`REA_CAPTURE_DECODER_OLD_GENERATION_MIB` before starting a new REA process. +Retry after owned cleanup completes; the update does not release resources. +See the [settings design](design/limits-and-recovery.md#failure-and-retry). Record selection happens after complete decoding and does not reduce that workload. Use a smaller capture exported by its producer or another decoder with sufficient capacity, retaining the original for provenance. Subsets do