Commit Graph
48 Commits
Author SHA1 Message Date
morluto cef87c498a refactor(architecture): add the ports and resources layers, move ports and shared resource owners there, and ratchet the layer baseline (#11208)
* test(architecture): fail on a stale layer baseline and document the declared layers

verify:module-boundaries reported a layer baseline entry whose imports were
gone but still passed, so main kept process -> artifacts at 20 after it fell
to 17 and a later change could add three imports back unnoticed. A stale
entry now fails like growth; --shrink-layer-baseline lowers it in the change
that removes the imports. The architecture map and CONTRIBUTING name
scripts/lib/module-layers.mjs as the per-directory layer declaration.

* test(architecture): refuse to lower the layer baseline after an inspection failure

A file that cannot be parsed contributes no imports, so a shrink run would record counts that fail as growth once the file is fixed. Whole-command tests cover stale rejection, shrinking, refused growth and the inspection failure.

* refactor(ports): move the progress, work-scope and inspection-scope ports to src/ports (#11214)

* refactor(ports): move the progress, work-scope and inspection-scope ports to src/ports

Pure move with git mv and import specifier updates.

* test(architecture): declare the ports layer and lower the layer baseline

src/ports sits between the foundation and the providers. Moving the three
ports removes 11 provider -> application imports.

* refactor(ports): move AnalysisProvider to src/ports (#11217)

* refactor(ports): move AnalysisProvider to src/ports

Pure move with git mv and import specifier updates in its importers, the
built-fixture path, the CI scope glob and the lint-rule test spellings.

* test(architecture): lower the layer baseline after moving AnalysisProvider

* refactor(ports): move the per-feature provider ports to src/ports (#11226)

* refactor(ports): move the per-feature provider ports to src/ports

Pure move with git mv and import specifier updates: the 47 *Port.ts and
*Ports.ts files that providers implement, plus OptionalObservationProviders,
which only composes the observation ports.

* test(architecture): lower the layer baseline after moving the feature ports

* refactor(providers): move closeAnalysisClient and the Frida session input errors to their owners (#11233)

* refactor(providers): move closeAnalysisClient to src/providers and the Frida session input errors to src/frida

Pure move with git mv and import specifier updates.

* test(architecture): drop the application-providers and application-frida cycles from the layer baseline

* fix(ci): keep provider lanes selecting the ports moved to src/ports

Each lane glob that matched a moved port under src/application gains its src/ports counterpart; the change-scope test pins one moved port per lane.

* refactor(domain): declare the localPath host exception and read HOST_PLATFORM there (#11256)

absoluteLocalPathStringSchema decides which paths are absolute on the running host, so it reads HOST_PLATFORM instead of process.platform. Domain may not import outside domain/contracts; module-layers now declares LAYER_EXCEPTIONS, exact file-to-file imports with a reason, and both the layer count and the pure-layer boundary skip only that import. Any other domain -> platform import still fails.

* refactor(ports): move the doctor check and provider inspection types to src/ports (#11246)

* refactor(ports): move the doctor check and provider inspection types to src/ports

Declaration move from application/Doctor.ts (unchanged text) and import updates; no re-export from Doctor.ts.

* test(architecture): lower the layer baseline after moving the doctor provider types

* refactor(ports): move the packet capture and signal recording ports to src/ports (#11249)

* refactor(ports): move the packet capture and signal recording ports to src/ports

Declaration move (unchanged text) out of the two application services, with import updates.

* test(architecture): lower the layer baseline after moving the capture and signal ports

* refactor(domain): move the analysis error projection schema from contracts to domain (#11251)

* refactor(domain): move the analysis error projection schema from contracts to domain

Declaration move (unchanged text) of the projection schemas that domain modules embed; contracts/errorSchemas.ts keeps the CLI envelope and the published JSON Schema. Import updates only.

* test(architecture): drop the contracts-domain cycle from the layer baseline

* refactor(ports): drop the imports the moved capture, signal and doctor types left behind

Also give the packet capture and signal recording lanes the new port paths, with change-scope cases.

* refactor(resources): add src/resources and move AnalysisCleanupLease and ArtifactReader there (#11259)

* refactor(resources): move AnalysisCleanupLease to src/resources

Pure move: git mv plus import specifier rewrites (source, tests, and the dist paths the verifier scripts load).

* refactor(resources): declare src/resources as a foundation unit

src/resources owns the shared analysis resource modules that process, windows and providers hold; AnalysisCleanupLease is the first. Its tests join the adapters project, the apk signature lane selects it beside src/artifacts, and the layer baseline drops the process and windows imports of the lease.

* test(ci): pin the apk signature lane on the moved cleanup lease

* refactor(resources): move ArtifactReader to src/resources

Pure move: git mv plus import specifier rewrites (source, tests, and the dist path a verifier script loads).

* test(architecture): lower the layer baseline after moving ArtifactReader

windows no longer imports src/artifacts; process -> artifacts 11 -> 9. The CI lanes that selected src/artifacts/ArtifactReader.ts select its new path, pinned in the change-scope test.

* refactor(resources): move ArtifactResourceScope to src/resources (#11265)

* refactor(artifacts): name the output tree and private root capabilities the resource scope releases

ArtifactResourceScope typed its output-tree and private-runtime-root owners with SafeOutputTree (src/artifacts) and PrivateRuntimeRoot (src/process). It only calls rollback() and close(), so it now declares those capabilities itself and names neither unit, which lets it move to src/resources without a resources <-> process cycle. The withheld-export test narrows to SafeOutputTree where it reads the published flag.

* refactor(resources): move ArtifactResourceScope to src/resources

Pure move: git mv plus import specifier rewrites (source, tests, the dist paths verifier scripts load, and the FIFO probe's module-relative URL).

* test(architecture): lower the layer baseline after moving ArtifactResourceScope

process -> artifacts 9 -> 5, and the artifacts <-> dotnet mutual pair is gone. The sqlite and agentenv lanes select the scope at its new path, pinned in the change-scope test.

* refactor(resources): move ArtifactFileOperation to src/resources (#11269)

* refactor(resources): move ArtifactFileOperation to src/resources

Pure move: git mv plus import specifier rewrites (source, tests, and the dist paths verifier scripts load).

* test(architecture): lower the layer baseline after moving ArtifactFileOperation

process -> artifacts 5 -> 1.

* refactor(resources): move readStableArtifact to src/resources (#11271)

* refactor(resources): move readStableArtifact to src/resources

Pure move: git mv plus import specifier rewrites.

* test(architecture): lower the layer baseline after moving readStableArtifact

process no longer imports src/artifacts: process -> artifacts 1 -> 0 and the artifacts <-> process mutual pair is gone. The seven lanes that selected readStableArtifact select its new path, pinned in the change-scope test; the architecture map lists what src/resources owns.

* refactor(hopper): take Linux launcher verification from composition through a port (#11278)

The Hopper launcher imported inspectLinuxHopperLauncher and retryLinuxHopperPublisherCleanup from src/application (the two hopper -> application imports). src/ports/LinuxHopperLauncherVerification.ts now names that capability; src/application/LinuxHopper.ts provides LINUX_HOPPER_LAUNCHER_VERIFICATION, binary composition passes it to HopperProvider, and the provider hands it to the launcher. A launcher built without one refuses a Linux demo launch with reason verification_unavailable before running the launcher. Layer baseline: hopper -> application 2 -> 0.

* refactor(config): move Hopper launcher discovery to a src/hopperPaths foundation unit (#11292)

* refactor(hopper): move the launcher discovery and macOS Hopper paths to src/hopperPaths

Pure move: git mv plus import specifier rewrites.

* refactor(hopper): declare src/hopperPaths as a foundation unit with the Linux launcher candidates

Hopper install paths and launcher discovery are pure path facts that the
Hopper provider, doctor and setup all read. Declare src/hopperPaths in the
foundation layer, add it to the adapters test project, the architecture
map and the package-baseline CI lane, and move the Linux launcher
candidates out of src/application/LinuxHopper.ts into
src/hopperPaths/LinuxHopperPaths.ts so discovery no longer imports
application code.

* refactor(config): leave Hopper launcher discovery to the Hopper provider

parseConfig resolved an unset HOPPER_LAUNCHER_PATH by probing the
filesystem through application-owned discovery, so config imported
application (config -> application 2, application <-> config cycle) and
read process.platform ambiently.

AppConfig.hopperLauncherPath is now the caller's HOPPER_LAUNCHER_PATH
unchanged (string | undefined). src/hopperPaths/HopperLauncherDiscovery.ts
owns the selection (selectedHopperLauncherPath: the configured spelling,
else discovery from the caller environment's home on the provider's
platform); HopperProvider and the two real-Hopper verifiers read it.
Discovery tests move from config.test.ts to src/hopperPaths; a provider
test covers discovery from the provider's own platform and home.

Module layers: config -> application 2 -> 0; application <-> config removed.
Architecture guards: ambient platform reads in parseConfig.ts 2 -> 0.

* refactor(providers): move owned-process prerequisite availability out of application (#11294)

* refactor(providers): move the owned-process prerequisite availability mapping to src/providers

Pure move: git mv plus import specifier rewrites.

* test(architecture): lower the layer baseline after moving owned-process availability

android -> application 1 -> 0; apktool -> application 1 -> 0.

* refactor(process): move the regular-file read owner out of application (#11299)

* refactor(process): move the regular-file read owner from src/application to src/process

Pure move: git mv plus import specifier rewrites.

* test(architecture): lower the layer baseline after moving the regular-file read owner

desktop -> application 5 -> 4; dotnet -> application 2 -> 1.

The agentenv lane listed src/application/RegularFileRead.ts. At its new
path it is a src/process/ shared boundary: fast qualification selects the
code and runtime checks (and the single owning lane, web_source_map, via
src/process/**), and full qualification selects every lane, as for every
other src/process/ module. A second owner entry would only suppress the
web_source_map selection, so the agentenv entry is removed, not renamed.

* refactor(resources): give atomic text writes their own owner (#11302)

* refactor(resources): give atomic text writes their own owner in src/resources

src/application/JsonFiles.ts owned both JSON input reads and the atomic
text write (writeTextFile / writeTextParts and their staged-output
publication). The write side depends only on domain, filesystem and
resource owners, but il2cpp's Ghidra seed writer had to import
application to reach it (il2cpp -> application).

Move the write side verbatim to src/resources/TextFileWrite.ts
(TextWriteCancellation, writeTextFile, writeTextParts and their private
staging, publication and failure classification helpers). JsonFiles keeps
the JSON read owner and reads the shared errno-to-reason classifier
(fileAccessFailureReason) from the write owner instead of a copy.
Importers (il2cpp seed writer, evidence bundle files, desktop build
solves, analysis snapshot files and four boundary tests) read the new
path; no re-export is left in JsonFiles.

* test(architecture): lower the layer baseline after moving the text write owner

il2cpp -> application 1 -> 0.

* refactor(ports): move the desktop build host contract to src/ports (#11303)

* refactor(domain): move the setup cancellation check from src/application to src/domain

Pure move: git mv plus import specifier rewrites.

* refactor(ports): move the desktop build host contract to src/ports

The desktop provider implements doctor's desktop build facts reader and
setup's desktop build host, but the contract it implements lived in
application (DesktopBuildDoctor, SetupDesktopBuild, Doctor's
clientRegistrations signature, ClientRegistrationEnvironment's setting
types), so desktop imported application four times and the two units
were a recorded mutual pair.

src/ports/DesktopBuildHost.ts now holds that contract, moved verbatim:
DesktopBuildCommand, DesktopBuildFacts, DesktopBuildFactsInput,
DesktopBuildSetupFacts, DesktopBuildSetupHost, RecordedProviderSettings
and OwnedProviderSetting. Two names are new: DesktopBuildRegistration,
the registration facts the host reads (client, command and the
recorded-settings names; client registration status satisfies it), and
DesktopBuildRegistrationReader, which replaces
DoctorHost["clientRegistrations"] in the setup host. The facts input's
registration reader returns DesktopBuildRegistration instead of the full
application status; the setup facts' settings are spelled
Readonly<Record<string, string>> (SetupProviderEnvironment's definition).
Application imports the types from the port; no re-export is left.

* test(architecture): lower the layer baseline after the desktop build host port

desktop -> application 4 -> 0; the application <-> desktop mutual pair is gone.

* refactor(domain): define the resolved app bundle resource root beside its schema (#11304)

* refactor(domain): define the resolved app bundle resource root beside its provenance schema

ResolvedAppBundleResourceRoot (an app directory's resource-only identity)
is a pure type over BundleResourceRoot's layout, but it lived in
src/application/AppBundleExecutable.ts, so the native and artifact
providers that receive it imported application.

Move the interface verbatim to src/domain/apple/bundleResourceRoot.ts,
which owns BundleResourceRoot, and point its eight readers there (native
macOS provider and plist source, artifact provider, binary composition,
and the binary session, status, port and bundle-resource session). The
resolver that produces it stays in application.

* test(architecture): lower the layer baseline after moving the resource root type

native -> application 2 -> 0; artifacts -> application 2 -> 1 (the
artifact provider still constructs BundleResourceSession).

* refactor(config): move the pwntools input bound parser to src/config (#11306)

* refactor(config): move the pwntools input bound parser from src/native to src/config

Pure move: git mv plus import specifier rewrites, and the module path in docs/design/rea-limit-inventory.md (table realigned by oxfmt).

* test(architecture): lower the layer baseline after moving the pwntools input bound

config -> native 1 -> 0. The binary_layout and recorded_crash lanes, which
select src/native/pwntools/**, also select src/config/pwntoolsInputLimit*.ts;
ciChangeScope.test.ts pins both.

* refactor(filesystem): move REA cache directory helpers out of config (#11309)

* refactor(filesystem): move the REA cache directory and lexical directory helpers from src/config to src/filesystem

Pure move: git mv plus import specifier rewrites.

* test(architecture): drop the config and filesystem cycle from the layer baseline

config no longer imports filesystem; filesystem still reads the Windows
ChangeTime settle budget from config.

* refactor(windows): break the process and windows import cycle (#11317)

* refactor(domain): move the runtime-control and private-runtime-root refusals from src/process to src/domain

Pure move: git mv plus import specifier rewrites (including the dist path scripts/verify-windows-native.mjs loads).

* refactor(windows): type Job Object cleanup by what a Windows job reports

WindowsOwnedProcess.cleanup() returned process-ownership's
ProcessCleanupResult, so src/windows imported src/process for a type while
process imports windows to supervise Windows trees. A Job Object stop only
ever reports settled-and-signalled or a reason it did not settle; name that
subset WindowsJobCleanupResult in windows. It stays assignable to
ProcessCleanupResult, so the process supervisor reads it unchanged.

* test(architecture): drop the process and windows cycle from the layer baseline

* test(architecture): ratchet cross-provider imports and imports past a provider entry (#11386)

* test(architecture): ratchet cross-provider imports and imports past a provider entry

A provider folder may import the domain, contracts, foundation and ports layers, the shared provider core (src/providers) and its own files; an import of another provider folder is a recorded exception. Application, composition and the CLI/MCP adapters reach a provider only through src/<provider>/index.ts. verify:module-boundaries counts both kinds of import per unit pair in the layer baseline (crossProvider, providerEntry); like the upward counts they may only shrink, and a pair missing from the baseline fails.

* refactor(resources): move StableArtifactFailure and StableArtifactObservation into src/resources (#11387)

* refactor(resources): move StableArtifactFailure and StableArtifactObservation into src/resources

Providers project stable-read failures through StableArtifactFailure next to the other shared provider toolkit owners (ArtifactReader, ArtifactResourceScope, ArtifactFileOperation, AnalysisCleanupLease, readStableArtifact) that already live in src/resources. Under src/artifacts it made every provider that reads a stable artifact import the artifacts provider. Pure move: git mv plus import specifier and baseline updates; the layer baseline drops the provider-boundary counts the move removes.

* refactor(resources): reach the shared provider toolkit through src/resources/index.ts (#11390)

* refactor(resources): import the shared provider toolkit through src/resources/index.ts

Adds src/resources/index.ts, which exports the toolkit names that production code outside src/resources uses, and rewrites those importers (420 files) to import from it, merging each importer's toolkit imports into one declaration. Mechanical: import declarations only.

* test(architecture): reach src/resources only through its entry file

verify:module-boundaries now counts any production import of a src/resources file other than src/resources/index.ts, from any unit, with the imports past a provider entry. The count is renamed from providerEntry to entry ("import past the entry file"); it may only shrink, and src/resources starts at zero.

* test(architecture): cover provider boundary counts through the verifier command

* refactor(resources): move the safe output tree and artifact path owners into src/resources (#11398)

* refactor(resources): move the safe output tree and artifact path owners into src/resources

SafeOutputTree (with its admission, creation failure, readback, cleanup and identity files), StreamBytes, ArtifactPaths and readArtifactFileChunks depend only on domain, filesystem, platform and the toolkit, and browser, firmware, ghidra, javascript, android and apktool use them to publish outputs. Under src/artifacts every such provider imported the artifacts provider. Pure move: git mv, importers moved onto src/resources/index.ts (names added to the entry), the moved files import toolkit modules directly, path keys in the CI scopes and guard baseline follow the move, and the layer baseline drops the removed imports.

* test(architecture): give the shared provider core its own layer below the providers (#11401)

src/providers (deferred client owners and failure attribution) now imports only domain, ports, process, filesystem and the resources toolkit. Declaring it as the providerCore layer, between ports and providers, makes any import from it into a provider folder an upward import that verify:module-boundaries rejects, and lets every unit in the providers layer be a provider folder.

* test(architecture): declare each provider's entry files instead of a provider barrel (#11407)

* chore: ignore the source map parser move (#11348) in blame

* test(architecture): declare inspector's use of the browser CDP client as a provider dependency (#11420)
2026-11-16 03:56:10 +08:00
morluto 5c53bcaed8 feat(mcp)!: retire the full schema profiles and declare schema facts as typed annotations (#10693)
* test(mcp): check advertised schemas against a portable dialect in every profile

* fix(mcp): re-emit advertised patterns in a portable regex grammar

* fix(mcp): present discover_tools input through the selected profile

The hot-cold catalog registered discover_tools directly with the SDK, so
the default compact input profile still advertised its canonical schema,
including page-size defaults clients apply on their own (#5322). Route
every listed input schema, transport discovery included, through one
server-owned presentation, and pass the selected presentation to the
deferred server, which serves recorded tool schemas but not discovery.

* fix(mcp): keep the Evidence union on described borrowed inputs

borrowEvidenceInput installed its advertised projection on one schema
instance. A call site's .describe() rebuilds the schema from its
definition, so match_native_functions advertised left and right as
unconstrained values in every input profile. Install the projection in
the borrowed parser's constructor so derived schemas keep it, for both
borrowed inputs and borrowed arrays.

* fix(mcp): validate advertised dialect by selected profile

* fix(mcp): preserve schema widening through references

* feat(mcp): project compact input schemas into the portable advertised dialect

One projection (advertisedDialectProjection.ts) owns the compact input
advertisement: one type per node, items on every array, string literals
as enums, oneOf widened to anyOf, allOf merged, tuples as element unions,
exclusive bounds made inclusive, and every keyword outside the dialect
dropped. Each rewrite is exact or widening; server validation is
unchanged. The root description and sentences for dropped cross-field
constraints move to the tool description, which Codex and Zed keep.

The compact baseline falls to nullable values only (29), which the
dialect cannot express without narrowing an input. The gateway shape
repairs (typeArrayItemUnions, hasGatewayIncompatibleShape), the second
dependentRequired-to-prose copy and the strict-gateway test denylist are
removed; the dialect checker gates the inline presentation instead.

* feat(mcp): omit output schemas from tools/list by default

No surveyed model API reads an output schema, MCP client validators
repeat REA's own result validation, and complete output schemas were
about 92% of the default tools/list. REA_MCP_OUTPUT_SCHEMA_PROFILE now
resolves to off through one owner (resolveMcpOutputSchemaProfile);
compact and full remain selectable. structuredContent is still
delivered, and the deferred server serves both off and full from the
recorded catalog with the matching recorded identity.

Tests and artifact verifiers that validate results against complete
advertised output contracts select full explicitly.

* docs: describe omitted output schemas as the default in README translations

* fix(mcp): select schemas in verification workflows

* fix(mcp): refuse zero-padded repeat bounds and record host-specific path patterns

* fix(mcp): keep projected schemas permissive

* fix(mcp): keep references valid when presentation widens their targets

* fix(mcp): keep a single union branch's closure scoped to its properties

* fix(mcp): select complete output schemas in the Electron verifier

* fix(mcp): select complete output schemas in standalone verifier probes

* fix(mcp): keep an allOf member's tuple tail with its own positions

* fix(mcp): relax references into negations and conditionals presentation may remove

* refactor(contracts): declare root discriminators as typed schema annotations

* test(mcp): select the borrowed Evidence contract from its family

* fix(mcp): keep astral literals and dynamic references widening in every mode

* fix(mcp): omit astral literals and relax untracked references when presentation widens

* fix(mcp): select complete output schemas in every verifier transport

* fix(mcp): read reference pointers by schema position, not segment spelling

* fix(mcp): protect references into collapsing boolean unions and keep referencing branches out of the collapse

* fix(mcp): keep reduced schemas open when their root admits names by pattern

* test(mcp): keep the reduced-schema pattern case outside the size-limited block

* fix(mcp): omit patterns whose classes name astral characters

* fix(mcp): decide union collapse and pointer protection from canonical branches

* docs(mcp): document non-u astral matching and pin BMP agreement across V8 modes

* fix(mcp): keep prototype-named properties as own properties in the dialect projection

* test(mcp): select full output schemas in packet and signal schema checks

* test(mcp): expect the advertised base64 schema to omit the unbounded pad-bit probe

* fix(mcp): protect references into unevaluated keywords lost evaluation removes

* fix(mcp): validate the doctor probe reply against the canonical output contract (#10655)

* fix(mcp): validate the doctor probe reply against the canonical output contract

* test(mcp): replay doctor probe validation under every output profile

* test(mcp): accept typed-branch nullable unions in the advertised dialect (#10649)

* test(mcp): select full output schemas in HTTP session workflows

* fix(mcp): emit stack-safe classes with non-ASCII members and protect draft-07 reference siblings

* fix(mcp): protect references into literal siblings of relaxed draft-07 references

* feat(mcp): omit output schemas from tools/list by default (#10646)

* feat(mcp): omit output schemas from tools/list by default

No surveyed model API reads an output schema, MCP client validators
repeat REA's own result validation, and complete output schemas were
about 92% of the default tools/list. REA_MCP_OUTPUT_SCHEMA_PROFILE now
resolves to off through one owner (resolveMcpOutputSchemaProfile);
compact and full remain selectable. structuredContent is still
delivered, and the deferred server serves both off and full from the
recorded catalog with the matching recorded identity.

Tests and artifact verifiers that validate results against complete
advertised output contracts select full explicitly.

* fix(mcp): select schemas in verification workflows

* fix(mcp): select complete output schemas in the Electron verifier

* fix(mcp): select complete output schemas in standalone verifier probes

* fix(mcp): select complete output schemas in every verifier transport

* test(mcp): select full output schemas in packet and signal schema checks

* fix(mcp): validate the doctor probe reply against the canonical output contract (#10655)

* fix(mcp): validate the doctor probe reply against the canonical output contract

* test(mcp): replay doctor probe validation under every output profile

* test(mcp): accept typed-branch nullable unions in the advertised dialect (#10649)

* test(mcp): select full output schemas in HTTP session workflows

* fix(mcp): keep the Evidence union on described borrowed inputs (#10597)

* fix(mcp): keep the Evidence union on described borrowed inputs

borrowEvidenceInput installed its advertised projection on one schema
instance. A call site's .describe() rebuilds the schema from its
definition, so match_native_functions advertised left and right as
unconstrained values in every input profile. Install the projection in
the borrowed parser's constructor so derived schemas keep it, for both
borrowed inputs and borrowed arrays.

* test(mcp): select the borrowed Evidence contract from its family

* docs(mcp): admit baseline growth only for restored contract declarations

* fix(mcp): state reduced multi-type properties as typed branches, not type lists

The reduced fallback derived a property's types and emitted them as a type
list, which Gemini, Moonshot and Ark reject. A property whose canonical
schema is anyOf string and number therefore traded a typed union for a
rejected construct when the inline projection fell back. Emit one type, or
anyOf typed branches, so [T, null] becomes the dialect's nullable union.

* feat(mcp)!: retire the full schema profiles and declare complete observations as a typed annotation (#10718)

* refactor(mcp): declare complete observations as a typed annotation applied only on the wire

* test(mcp): compare complete-observation declarations with their owning text

* refactor(schema): couple Zod checks with their JSON Schema constraint keywords (#10725)

* refactor(schema): couple Zod checks with their JSON Schema constraint keywords

* refactor(schema): declare pattern approximations and object output roots as typed annotations (#10754)

* refactor(schema): declare pattern approximations and object output roots as typed annotations

* fix(schema): refuse pattern declarations JSON Schema cannot state

* test(schema): select guided_workflows from its contract family

* feat(mcp)!: retire the full input and output schema profiles (#10841)

* feat(mcp)!: retire the full input and output schema profiles

The compact input presentation is now the only advertised input schema, and
output schemas are either omitted (`off`, the default) or bounded
(`compact`). Catalog identity, the build-recorded digests and the recorded
startup catalog collapse to those presentations: two digest profiles instead
of six, one recorded catalog instead of two, and no input dialect baseline.

Tests and verifiers that validated replies against listed full output
schemas now validate against the canonical output contract, which REA
already enforces server-side; tests that asserted canonical input
restrictions read the canonical input schema instead of a full listing.

BREAKING CHANGE: `REA_MCP_INPUT_SCHEMA_PROFILE=full` and
`REA_MCP_OUTPUT_SCHEMA_PROFILE=full` are refused at startup and by `rea setup`
with a configuration error naming the setting. Remove the setting from the
client registration (or rerun setup with `REA_MCP_INPUT_SCHEMA_PROFILE=compact`
/ `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact`). `REA_MCP_INPUT_SCHEMA_PROFILE`
still accepts `compact`. Server-side validation is unchanged.

* test(mcp): validate replies against canonical output contracts at every listed-schema reader

tools/list omits output schemas by default, so fixtures, verifiers and the
Android analysis runners that read a listed outputSchema now bind the
canonical contract instead. connectLocalToolsMcp and advertisedMcpToolSchemas
hand the SDK a definition carrying the canonical output schema, so the SDK
client and caller-supplied validators check every successful reply.

* test(mcp): resolve Android runners' output contracts from the selected repository

The Android efficiency runners imported the canonical output-schema helper from the harness checkout while --repository selects the revision under test. Import it from the selected repository, as the runners already do for the SDK and Ajv.

* test(mcp): let Android runners compare revisions before the canonical output helper

A selected revision without scripts/lib/canonical-output-schema.mjs could not start the Android runners. Choose the output contracts in one harness helper: the selected revision's canonical contracts when it has them, otherwise its listed schemas under the full output profile it still supports.

* fix(scripts): keep caller launch settings when the real Hopper verifier builds its server environment

The verifier deleted REA_VERIFY_SERVER_COMMAND and REA_VERIFY_SERVER_ARGS_JSON from process.env itself before reading them, so a caller selecting an installed REA command silently ran the checkout's dist/main.js. Copy the environment, as main does, so only the child's environment changes.

* test(mcp): validate whole replies against canonical output contracts wherever a listed definition is held

tools/list omits output schemas by default and the full output profile is gone, so tests that handed the SDK a listed definition no longer checked complete replies. Attach the canonical output schema with withCanonicalOutputSchema at every held REA tool definition. Environment-served tools (not REA contracts), and the deferred-runtime and doctor tests that exercise per-profile listings, keep their listed definitions.

* test(mcp): record the advertised catalog summary without the retired full profiles

Retiring the full input and output profiles leaves only the compact input and compact output listings, so the summary drops the full-profile digests and every pattern only the full listings carried. Regenerated with npm run verify:advertised-schemas -- --update-summary.
2026-11-15 21:51:18 +08:00
morluto a8e472967b refactor(process): read Ghidra and Hopper replies through one bounded JSON-line reader (#10699)
* refactor(process): read Ghidra and Hopper replies through one bounded JSON-line reader

Both clients now share JsonLineBridgeReader: UTF-8 framing with a mandatory
reply line ceiling (bridge.max_reply_bytes, REA_BRIDGE_MAX_REPLY_BYTES,
default 256 MiB), one fatal UTF-8 policy, request-id correlation including
rejections on the reserved id 0, and typed stream failures. An oversized
reply is refused as a transport resource constraint naming the setting and
its bytes are discarded. Hopper rejections on id 0 now reach the request on
the wire instead of being dropped.

Deletes GhidraResponseRouter and HopperResponseStream.

* fix(process): bound each bridge reply by the ceiling its own request admitted

* fix(process): admit the bridge reply ceiling before launching Hopper or Ghidra

* fix(ghidra): keep a refused setting's configuration error through the provider boundary

* fix(process): admit both bridge ceilings per request as a typed configuration refusal

A malformed REA_BRIDGE_MAX_REQUEST_BYTES or REA_BRIDGE_MAX_REPLY_BYTES seed covered by a runtime override at startup is refused by the runtime view once the override is reset. Hopper and Ghidra read those getters in the request path (and Hopper re-read the request ceiling when a queued request reached the wire, outside the caller's context), so the refusal was thrown instead of returned. Both clients now copy both ceilings once at admission through readRuntimeSetting, return the ConfigurationError (Ghidra as an admission failure), and write queued requests under the ceilings they were admitted with.

* refactor(config): read rizin, gdb, Kuna, Ghidra and regex settings through readRuntimeSetting (#10795)

Each site caught the ConfigurationError a setting view throws for a malformed launch seed by hand. They now read through readRuntimeSetting and return its Result. GDB reads both debugger limits once per console command and reuses those accepted values for its limitations, timeout and diagnostics instead of re-reading the views after the command ran. Kuna's single-function read returns the refusal after releasing admission instead of throwing it. A counted architecture guard keeps the remaining hand-caught sites from growing.

* refactor(hopper): return typed bridge replies from HopperClient.callTool (#10833)

* refactor(hopper): return typed bridge replies from HopperClient.callTool

The reply contract table existed, but the provider, caller reconciliation and
regex search each applied it after callTool returned JsonValue. callTool is now
typed by method: it validates the reply against that method's contract and
returns the typed reply, or a HopperReplyContractError that names the method
and keeps the raw reply. The caller scan keeps that raw reply as its unread
page or native result. The provider dispatches list_names, file offsets and
the three reconciled operations through their literal methods, and health has
its own contract. Transport tests inject a JSON-value table for the fake
bridge's own methods.

* fix(hopper): refuse value-changing reply contracts and uninstalled methods

parseHopperReply returns the raw reply, so a contract whose input and output types differ (a coercion, default or transform) would promise a type the reply does not hold. ValueKeepingContracts refuses such a schema at compile time, for the production table and any injected one. A client whose type argument names a table it was not given now returns AnalysisCapabilityUnavailableError for a method with no installed contract instead of throwing.

* refactor(hopper): read the startup handshake through the health reply contract (#10853)

parseHopperServerInfo hand-checked each health field that hopperReplySchemas.health already defines. It now parses through that contract, keeps the zod failure as the protocol error's cause, and binds run_id to the launch.

* refactor(kuna): read the project deadline through readRuntimeSetting at each request (#10837)

* refactor(kuna): read the project deadline through readRuntimeSetting at each request

The string, call-graph and xref scans read kuna.project_timeout_ms inside the background producer they start, and the project export read it inside the shared startup promise. A malformed launch seed without an override was thrown from there and reached the caller only through the generic failure mapper; a concurrent waiter that swallows the startup promise's rejection never saw it. Each site now reads the deadline with readRuntimeSetting before starting its work and returns the refusal as that request's value; the export start returns it to each caller that needs the project, and the next request reads it again.

* fix(kuna): return a call-graph deadline refusal from analyze_function

The dossier reported any relationship failure as unavailable call coverage. A call-graph deadline refusal is the request's own setting failure, so it now returns as that ConfigurationError, as cleanup and cancellation already did.

* refactor(kuna): require the accepted project deadline on every export request

Remove acceptedExportDeadline's fallback read of the throwing configuration view. KunaProjectRequest and KunaExportRequest now require the deadline the starting operation accepted; the session passes the one it admitted through readRuntimeSetting into #lookupOrBuild. Direct cache callers in tests take it from one shared helper, withKunaProjectDeadline.

* test(mcp): carry main's Evidence-format const count (#10995) beside the bridge reply setting's

* fix(process): clean Hopper and Ghidra sessions up under the ceilings their start admitted

Cleanup requests re-admitted bridge.max_request_bytes and bridge.max_reply_bytes from the current settings. Once an override covering a malformed launch seed was reset, Hopper's shutdown_document and Ghidra's closing checkpoint_program and shutdown were refused before writing, leaving Hopper's document and files or Ghidra's JVM behind. Each session now keeps the ceilings it admitted at start, and its cleanup requests use them.

* fix(hopper): reauthenticate for cleanup under the session's admitted ceilings

The cleanup reconnect's health handshake still re-admitted the ceilings from the current settings, so a seed refused after an override reset blocked reauthentication and the shutdown after it. It now uses the same session ceilings as the shutdown requests, through one #cleanupLimits accessor.

* refactor(process): report an NDJSON line above its ceiling as a limit parse failure

Utf8NdjsonOversizeError now extends ParseFailureError<"limit"> (format ndjson, limit ndjson-line-bytes) and keeps the observed bytes the transport error reports, so parseFailureFromCause recognizes it like every other byte-format budget.
2026-11-15 20:37:21 +08:00
morluto db5078b0c3 fix(config): validate analysis seeds in diagnostics and registration 2026-11-11 15:10:31 -05:00
Cursor Agent 5ba464527b fix(limits): reject invalid runtime seeds and keep the value an operation used
A malformed migrated launch seed stays on the config and the operation view reports configuration_invalid unless a snapshot override is active. Kuna, regex, GDB, and Rizin keep the deadline or heap accepted when the work started, and reattributed resource errors copy the structured limit.
2026-11-11 20:16:24 +08:00
Cursor Agent b9be864801 fix(settings): parse runtime launch seeds at the operation boundary
Startup no longer rejects a runtime-owned REA_ seed, so update_analysis_settings can replace it. Debugger construction keeps only the restart-only caps, Kuna stores the launch baseline rather than a cached override, and the advertised operations include GDB close, Rizin close, Kuna list_procedures, and the stable-file ChangeTime callers.
2026-11-11 20:16:24 +08:00
Cursor Agent 2bac42cde5 fix(settings): apply runtime overrides before provider limit validation
JEB, Kuna, Windows, debugger, and Cutter consult an accepted override before rejecting an invalid launch value. JEB and Hopper startup deadlines keep default, setting, or runtime as the limit source, and a Kuna timeout without its own limit uses the selected project deadline source.
2026-11-11 20:16:24 +08:00
morluto cb4e5cfbff feat(analysis): make resource budgets recoverable across CLI and MCP (#9706)
* refactor(analysis): centralize resource budgets and recovery facts

* feat(analysis): expose ASAR and Ghidra runtime admission controls

* feat(artifacts): expose reader admission and preserve limited recognition

* feat(javascript): expose recoverable semantic precision budgets

* feat(web): add recoverable source-map budgets and compact retained views

* feat(web): make historical capture decoding budgets recoverable

* fix(desktop): enforce selected view policies across capture and export

* feat(runtime): make Frida and LLDB retention budgets selectable

* feat(native): expose diagnostic budgets and preserve exhaustion causes

* feat(managed): expose recoverable snapshot and representation budgets

* feat(metadata): make Go and Rust inspection budgets recoverable

* feat(wasm): expose representation and producer admissions

* feat(evm): expose per-call worker resource budgets

* feat(android): make resource and string-pool omissions recoverable

* feat(producers): expose command and recorded-capture budgets

* docs(analysis): record recovery contracts and integrated validation

* fix(docs): link calibration oracle from published verification pages

* fix(docs): restore desktop task anchors and receipt links

* test(analysis): align fixtures with selectable budget contracts

* fix(analysis): remove source-location projector dependency cycle

* test(analysis): refresh budget recovery and verifier regressions

* test(analysis): preserve format and publication recovery expectations

* test(windows): align conformance with owned runtime admission

* fix(desktop): preserve selected alias kinds in exported state

* test(analysis): align provider fixtures with recoverable budgets

* test(desktop): validate emitted alias metadata structurally

* fix(ghidra): classify bounded Windows endpoint refusals

* test(cli): assert selectable artifact admission recovery

* fix(cli): preserve capture provenance in environment builds
2026-11-10 05:11:20 -05:00
gensokyo-dev c07ab4b994 feat(ghidra): checkpoint owned Programs across sessions
Add opt-in REA_GHIDRA_CHECKPOINT_DIR for owned Ghidra Program persistence through the shared CLI/MCP provider. Save completed initial analysis and metadata changes on normal close; verify and restore the last successful packed Program without auto-analysis.

Preserve generation identity, exclusive ownership, typed cancellation and cleanup retry. Refresh checkpoint-dependent profiles after retirement, including failed-switch restoration, while retaining caller selection provenance. Keep source executables and caller projects unchanged.

Validated with 720 focused tests (3 platform skips), 13 real Ghidra CLI/MCP scenarios, static/docs checks and CI. Review findings addressed. No version bump or release publication.

Refs #8165.
2026-11-07 14:52:39 +08:00
gensokyo-dev 0beedcd712 perf(mcp): add opt-in hot/cold tool discovery (#9046)
Add per-connection discovery that loads canonical named tools on demand while preserving the default full catalog. Bound discovery replies before changing visibility, preserve family exclusions and Evidence workflows, and reset visibility on HTTP reconnect.

Verified with focused protocol tests and real Codex CLI artifact investigations; token costs remain task-dependent, so the profile is opt-in.
2026-11-05 20:27:31 +00:00
morlutoandCursor Agent ec4a0a01c4 fix(hopper): reconcile observed caller relationships (#7953)
* fix(hopper): reconcile observed caller relationships

* fix(hopper): retain completed caller scans on failure

Attach the native reply and collected forward facts when reconciliation
fails or is cancelled, and check scan stop codes separately from their
wording. Live observations keep REA evidence ids rather than provider
supplied ones, and the real Hopper verifier uses advertised MCP schemas.

* fix(hopper): answer caller-scan methods in the package fixture

The packaged Hopper stand-in echoed unknown bridge methods, so
analyze_function rejected rea_caller_scan_target as an invalid scan
target and the native package baseline returned unreadable_output.

* fix(hopper): share caller identities in failed scan observations

Failure partials repeated each caller identity under every destination.
The page budget counted compact replies, then schema parsing copied the
expanded form. Identities are stored once and destinations cite addresses.

* refactor(config): own the Hopper caller-scan budget default

The environment schema hard-coded the 2000 ms scan default. The domain
limit owner now names that millisecond budget and the schema reuses it.

* test(mcp): assert bundle path issues by field

compare_bundles reports each relative path as an invalid_format issue
instead of one joined diagnostic sentence.

* test(flutter): assert ABI selector refusals as structured issues

inspect_dart_aot reports the ABI label as an issue on path abi. The
message is the selector contract, without an abi: prefix.

* fix(session): retain caller observations that cannot be replayed

A missing document selector blocks snapshot replay. Backward call-graph
and trace reads still keep the native result and forward observations,
and the workflow Evidence links those records.

* fix(session): keep decompile bodies out of the in-memory ledger

Caller and callee walks still stay in Evidence without a document.
procedure_pseudo_code does not, so decompile_all leaves each body in the
file-backed export instead of retaining the whole export in memory.

* fix(hopper): own caller-scan recovery and the page byte allowance

Scan facts stay in the domain. Environment recovery, including the time
budget bounds and the fixed allowances with no runtime override, is
appended from configuration. Setup forwards the budget, and the bridge
uses one byte allowance for instructions, native callers, and pages.

* fix(hopper): retain compact caller references within the scan budget

Recovered call sites stay in the address form the scan already counted.
Expanded identity copies are measured before allocation and refused when
they exceed the retention budget. Instruction lists are not part of the
retained reply.

* fix(session): omit instruction payloads from retained caller evidence

Documentless caller and callee reads stay in the ledger for the session.
Their evidence keeps call sites and drops instruction lists the scan
already consumed.

* fix(hopper): keep the owning caller address on each compact call site

A page reference marked source_procedure "caller" belongs to that page
item's caller. Saved sites record that address, and an owner missing from
the callee inventory is stored once instead of copied onto every site.

* fix(session): leave Hopper reply shaping to the Hopper adapter

Documentless caller evidence stores the adapter's raw result unchanged.
The session no longer reads native_result.instructions.

* test(hopper): throw typed scan failures before reading results

Successful caller-scan results now throw the typed error instead of
returning after an ok check. Expected failures assert structured tags,
stop codes, and reported limits.

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-10-29 22:54:58 +08:00
gensokyo-dev f60e42a9ae fix(mcp)!: bound validation work before tool dispatch (#7905)
Reuse the pinned SDK's combined array-element and object-member admission
before canonical validation, deferred runtime loading and tool execution.
Share a configurable finite allowance across eager/deferred HTTP and stdio,
and project refusals to bounded, actionable transport resource errors.

Preserve canonical CLI/application contracts, caller-selected input values,
ordinary validation semantics and complete Evidence access. Exercise actual
transport boundaries, real artifact/TShark workflows, packaged installation,
continued service and downstream identity/capacity regressions.

Fixes #7882

BREAKING CHANGE: MCP tool arguments now default to 1,024 combined array
elements and object members per call. Select a larger positive finite
REA_MCP_MAX_INPUT_ELEMENTS allowance and restart for larger inline inputs,
or use retained/file selectors where supported. Inputs are never truncated.

Assisted-by: Reimu (gensokyo-skills:reimu-incident-triage)
Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)
Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)
2026-10-29 05:56:38 +00:00
morluto 08f99dbaf2 fix(setup): preserve and validate selected JEB endpoints 2026-10-27 07:17:10 +08:00
19447516a4 feat(ghidra): reuse existing projects through verified copies (#7236)
Reuse selected Ghidra projects through verified private read-only copies, with shared CLI/MCP configuration, committed project identity, import-SHA verification and explicit historical/unknown analysis semantics. Preserve exact cancellation and platform capability boundaries.

Actual Ghidra 12.1.4/JDK21 Linux CLI/MCP investigations matched fresh import on 652-function ls and 14,338-function libcrypto inventories and pseudocode; authenticated complete Evidence, mismatch/staleness refusal, saved historical reports and shared /tmp projects verified. 958 focused tests passed with four platform skips; build/static/docs/format passed. Real macOS/Windows reuse remains unverified; Windows reuse is unavailable. See PR verification for measured whole-task costs and final CI attribution.

Co-authored-by: akushonkamen <akushonkamen@163.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-26 22:31:13 +08:00
b675073b73 feat(mcp): scope advertised tools by family (#7228)
Scope MCP discovery, setup, doctor, status and recovery guidance by selected tool families using one contract inventory and shared catalog identity. Preserve default discovery, canonical validation and complete Evidence.

Fixes #3246. Builds on #3026. Real libcrypto workflow verifies all 5,367 exports and identical complete Evidence. Static/source/build/docs/Windows/package/Greptile checks pass; ten assertions reproduced identically on clean main and feature across Node 24/22, with cancelled coverage explicitly unverified. Merge uses the user-authorized bypass for baseline failures and unchecked human-readiness declarations, without attesting human review.

Co-authored-by: 0xLLVEL <fakhrezan1@gmail.com>
Co-authored-by: Codebuff <noreply@codebuff.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-10-26 19:50:12 +08:00
morluto 5ea2c280b8 refactor(doctor): retain parsed configuration diagnostics
Return native admission issues and an explicit Hopper timeout result from
the configuration owner. Doctor consumes those parsed facts instead of raw
timeout text plus a separate issue list. Reuse a successful aggregate parse;
validate the individual timeout independently when other settings reject it.

Preserve aggregate error precedence, JEB isolation, omission, constraints,
causes and provider scope. Keep partial diagnostics when unrelated NUL,
loader arguments, language selection or timeout settings are invalid.

Validation: configuration and Doctor workflow regressions, mixed-invalid
settings, full static/docs checks. No installed provider was exercised.
2026-10-25 04:37:11 +08:00
Syed Rishadandgensokyo-dev a062529bb2 feat(ghidra): bound one procedure's decompile without closing the session (#5662)
* feat(ghidra): bound one procedure's decompile without closing the session

Fixes #5340

* fix(ghidra): report decompile limit as a request timeout

Keeps BinarySession from marking every Ghidra operation unavailable after one procedure's decompile reaches the limit, and names the MCP tool-call deadline in the remediation.

* fix(ghidra): budget the decompile limit and report cancellation-closed sessions

Adds REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS to the derived Codex/Kimi tool deadline and forwards it through setup. Marks a remote failure that settles after caller cancellation so the timeout projection no longer promises session reuse.

* fix(ghidra): align deadline bounds and cancellation outcomes

* refactor(ghidra): shorten deadline recovery guidance

* fix(mcp): clarify the scope of inferred decompile deadlines

* test(ghidra): verify deadline workflows and document measured costs

Separate failure and registration cases by their distinct contracts.
Record actual Linux Ghidra, CLI and MCP evidence with measured context,
elapsed time, CPU and memory, and explicit remaining coverage limits.

* docs(ghidra): pin integration and distinguish baseline format failures

* fix(ghidra): own failure lifecycle wording in the provider

* test(ghidra): budget timeout fixture generation before allocation

* fix(ghidra): align capability and traversal deadline guidance

* style(config): align deadline guidance formatting

* docs(ghidra): record capability and traversal verification

---------

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-24 07:42:50 +08:00
Paritosh LahreandClaude Opus 5.5 8f91e60558 fix(doctor): check only the current platform's Hopper launcher candidates (#6253)
* fix(doctor): check only the current platform's Hopper launcher candidates

Doctor built one launcher list for every platform, so on macOS it could
report a Linux-only path as a healthy Hopper ahead of a renamed bundle,
and setup then registered that path. Give per-platform automatic
candidates one owner that runtime configuration and doctor both read,
scan renamed macOS bundles and probe the Homebrew cask only on macOS,
and share runtime's renamed-bundle scan with the doctor host.

Fixes #6228

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(doctor): keep the renamed Hopper bundle scan asynchronous

Sharing runtime's synchronous scan made the doctor host read
~/Applications with readdirSync, which blocks the event loop on a slow
or network-mounted home. Give MacHopperPaths an asynchronous scan for
doctor beside runtime's lazy synchronous one, sharing the directory
filter and candidate builder, and check that both return the same
candidates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(doctor): model renamed launchers with runtime's POSIX joins

Since #5235, runtime builds macOS launcher paths with posix.join, so on
Windows the discovery table's native-join renamed launcher no longer
matched runtime's candidate. Model launchers with posix.join and keep
filesystem setup native, as MacHopperPaths.test.ts does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 23:41:54 +09:00
47353e494d feat(config): make the bridge request ceiling configurable (#6376)
* fix(reconstruction): dedupe residual unknown ids with uniqueSorted

Current main calls uniqueBy, which is not defined, so typecheck fails.
Residual unknown ids are strings, and the adjacent lists already use
uniqueSorted.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* feat(config): make the bridge request ceiling configurable

REA_BRIDGE_MAX_REQUEST_BYTES replaces the hard-coded 16 MiB Hopper and
Cutter request payload ceiling. The default stays 16777216 bytes. Values
outside 7464 to 268435456 bytes are configuration_invalid.

Part of #6176.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): keep cleanup ownership when a sent command's reply is oversized

An authenticated command may already have changed the Cutter document
when its reply exceeds the frame ceiling. Return the existing unknown
execution, with the ceiling diagnostic in the message, so receipt
cleanup still waits. A frame that overflows before the command is sent
remains a resource constraint.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* style: format the bridge request ceiling sources

oxfmt --check failed on the five files that set and test
REA_BRIDGE_MAX_REQUEST_BYTES. Formatting only; an oversized Cutter
reply still stays an unknown execution.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): exclude the LF from the response frame budget

REA_BRIDGE_MAX_REQUEST_BYTES counts payload bytes. The Cutter response
reader and authenticated encoder were reserving one byte for the
terminating LF, so an exact-limit response was rejected. A reply that
exceeds the ceiling after the command was sent still stays an unknown
execution.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(bridge): measure escaped JSON before allocating the request

A command of U+0001 characters can expand sixfold. JSON.stringify of
that expansion can throw past Node's string ceiling before the payload
check. Count the escaped size within REA_BRIDGE_MAX_REQUEST_BYTES
first and refuse it as a resource constraint.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(config): derive the bridge request maximum from JSON expansion

A 256 MiB ceiling still lets a shorter U+0001 command expand past Node's
536870888-character string limit. The maximum is now
floor((536870888 - 296) / 7) = 76695798, the longest such command whose
authenticated frame fits in one string. Requests are still measured
before JSON.stringify.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(bridge): size the Cutter frame deadline and send the measured Hopper payload

The authenticated Cutter command is encoded after the challenge, so a
one-second read rejected an in-budget frame. That read now allows one
second per mebibyte of the ceiling and never waits longer than 25
seconds. Hello stays at one second. Hopper writes the payload it already
measured instead of serializing the same request again.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(config): bound the bridge request maximum by the serialized string

The ceiling already measures escaped JSON, so dividing Node's string limit by the seven-byte expansion rejected encoded payloads that still fit. The maximum is 536870887, one byte under that string ceiling, leaving room for the framing LF.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(hopper): serialize a bridge request when it reaches the wire

The queue kept each encoded frame, plus the caller's params, until Hopper finished the active request. A raised ceiling made a few waiting calls enough to exhaust the heap. The FIFO now keeps the params, and the frame is encoded only for the request being written.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): hash the bridge proof prefix and payload separately

An exact-limit Cutter frame plus the HMAC prefix exceeds Node's single-string ceiling, so the client threw before sending an admitted command. The prefix and payload are now separate HMAC updates, including the response proof.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): keep recovery fields when a status path exceeds the frame

A long current_file made the status reply miss the ceiling, and the fallback then dropped session_id and the completion receipts. The status now omits the path and version and still returns the session, connection proof, and up to 64 receipts. That required recovery frame is smaller than the 7464 floor.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): report an unknown path when status identity does not fit

A bounded status kept the new document generation and omitted the path, so the client filled in the descriptor's older path. The observed path is sent when it fits. When it does not, current_file is null and is not replaced with that stale path.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): count structured replies by their encoded JSON size

The response guard charged 16 bytes per primitive and two bytes per list slot, so a dense array under the configured ceiling was replaced with output_truncated. Numbers, booleans, null, and container punctuation now use their JSON lengths.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): stream authenticated responses in bounded chunks

A fitting reply was signed from retained raw bytes, escaped bytes, and a
joined frame, so a ceiling near 512 MiB could exhaust Cutter before send.
Measure and sign JSON pieces, then write the same frame in 64 KiB chunks.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(bridge): keep Cutter admission from stalling or hiding sessions

Read an unauthenticated frame off the accept loop. A command reply that
does not fit keeps completion and generation and reports an oversized path
as null. Discovery reports a ceiling mismatch instead of an absent session.
The maximum HMAC proof is opt-in, and both bridges reject non-ASCII digits.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): bound frame scans and client response copies

Search only newly read bytes for the framing LF. Unescape each socket
chunk into one response string, check the proof, and then parse it.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): keep completion, admission, and dense replies bounded

Report completed_connection only through connections that have
terminated. Cap connection threads and keep the listener up if one
fails to start. Batch scalar JSON and fail discovery when any ceiling
mismatches.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): hash requests in parts and scan escapes once

Update the client-proof prefix and request separately. Stop the response
decoder at the first quote or backslash instead of rescanning the tail.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): confirm live ceiling mismatches and omitted output

A stale descriptor from a previous ceiling is not a discovery mismatch
until its hello proves a live authenticated bridge. A response discarded
for the frame ceiling after dispatch reports output_truncated.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(cutter): close the unauthenticated ceiling-mismatch fixture

The listener that fails the hello proof is destroyed after its challenge
so discovery cleanup does not wait on that socket.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-23 06:11:21 +08:00
9512a80804 feat(process): make the regex worker heap request configurable (#6301)
* feat(process): make the regex worker heap request configurable

REA_REGEX_WORKER_HEAP_REQUEST_MIB keeps the 128 MiB default and refuses
values outside 8 to 16384 MiB. A positive Node --max-old-space-size still
overrides the request and is not clamped by the setting.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): set the regex worker heap floor to the measured startup bound

On Node 22.23.4 a 5 MiB old-generation request aborts the regex worker
during startup, and 6 MiB starts it and returns a match. The setting floor
is that measured bound.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): raise the regex worker heap floor to 7 MiB

6 MiB passes configuration but returns ERR_WORKER_OUT_OF_MEMORY for a
trivial match on supported Linux Node 24 and Node 26.0.0. 7 MiB returned
that match on the measured 22, 24, and 26 lines. A clean-process test
runs the production worker at the floor.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-22 23:43:13 +08:00
c7efd3a388 feat(config): make the regex matching deadline configurable (#6267)
* feat(config): make the regex matching deadline configurable

REA_REGEX_MATCHING_DEADLINE_MS defaults to 5000 ms and refuses values
outside 1..2147483647. Recovery and capability text name the effective
deadline, its source, and how to change the setting.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(setup): preserve the regex matching deadline in registrations

Setup validated REA_REGEX_MATCHING_DEADLINE_MS but left it out of the
selected registration environment, so a caller override never reached
the launched server and matching fell back to 5000 ms.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(config): restore a stubbed regex matching deadline on failure

Kuna deadline assertions now unstub the environment in finally, so a
thrown construction or expectation cannot leave the setting stubbed.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-22 17:10:51 +08:00
gensokyo-dev aa51e43439 perf(mcp)!: default to compact input schema advertisements (#5975)
Use compact MCP input advertisements by default while preserving explicit full, canonical validation, output contracts and complete Evidence. Require an explicit selected catalog for runtime identity and honor both profiles in CLI status.

Real ripgrep/Zellij SDK tasks, independent ELF oracle, actual client trials, profile parity workflows and CI are recorded in experiments/analysis-efficiency/mcp-input-catalog.md. SDK token estimates improve; actual Codex token savings and general latency/memory gains are not established.

Refs #5861

BREAKING CHANGE: MCP input advertisements default to compact. Set REA_MCP_INPUT_SCHEMA_PROFILE=full for complete advertised guidance and nested constraints.
2026-10-22 00:17:11 +08:00
b718ad166d fix(doctor): reject unusable Kuna caches and invalid native settings (#5647)
* fix(kuna): reject dangling cache roots in doctor

Treat a cache path as creatable only when no directory entry occupies it.
A dangling symbolic link, including one hidden by a trailing separator,
fails the cache check and clears provider availability while the installation
check and caller-selected registration settings remain.

Fixes #5554
Fixes #5105

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(doctor): admit the same settings native analysis rejects

Project parseConfig failures into scoped engine readiness. Invalid provider
preferences, startup timeouts, and other shared settings now fail the
selected provider and the full audit, while a JEB-only timeout stays outside
that admission.

Fixes #5454

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(doctor): report an invalid log level from doctor

The dispatcher rejected REA_LOG_LEVEL before every command, so doctor never
emitted its readiness check. Doctor now runs and names that setting; other
commands still stop before execution.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(doctor): require log-level failures in every scope

An invalid REA_LOG_LEVEL stops the executable and MCP server before a client
or skill can run, so those scopes now fail readiness too. Other native
admission failures stay limited to a selected provider.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(kuna): strip only the platform cache separator

On Darwin and Linux a trailing backslash is a filename character. Doctor now
removes only the selected platform separator before inspecting the cache path.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(doctor): fail every provider on an invalid Hopper timeout

REA_HOPPER_STARTUP_TIMEOUT_MS is rejected before any native engine starts.
A selected non-Hopper provider now includes the existing timeout check.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* style(doctor): format the Hopper timeout regression

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(doctor): fail client scopes on MCP startup errors

A registered client starts the MCP server, and parseConfig rejects every
native admission failure before that server starts. Client readiness now
includes those failures, not only REA_LOG_LEVEL.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-21 03:44:49 +08:00
ShuaixinHuang d67d895c26 fix(jeb): isolate invalid timeout from shared startup (#3964)
Keep malformed JEB-only timeouts isolated from shared CLI/MCP startup while strict registration and JEB configuration checks remain authoritative. Cover built CLI discovery and the production stdio MCP refusal and peer requests.

Original contribution by @ShuaixinHuang.

Co-authored-by: ShuaixinHuang <1836656514@qq.com>
2026-10-16 01:03:39 +08:00
044fb9127a fix(hopper): skip non-file launcher candidates
Require automatically discovered Hopper launchers to be executable regular
files so POSIX directory search permission cannot hide a later valid
application bundle. Preserve file symlinks, explicit caller configuration,
and the shared candidate ordering.

Cover canonical and renamed user bundles through the production filesystem
predicate; respect existing canonical or renamed system installations in
the isolated fixtures. Clarify launcher eligibility in the installation guide.

Follow-up to #3085 and #3054 after their shared path discovery landed on main.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-15 00:09:42 +08:00
7582f0dff2 fix(hopper): share launcher discovery across setup and runtime
Centralize canonical system/user and deterministic renamed-bundle path facts. Reuse the installer destination and honor the same local candidates in doctor and runtime; scan renamed bundles only after canonical candidates fail. Preserve explicit path spelling and exclusive selection. Homebrew command probing remains doctor-only with explicit runtime configuration guidance.

Validation: owner, config, setup and doctor tests cover canonical priority and renamed paths; public CLI verifies explicit path preservation without launching Hopper. No real Hopper analysis claimed.

Fixes #3054
Related: #3085

Co-authored-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-14 17:09:17 +08:00
morluto 80de3145e4 fix(mcp): align advertised contracts and retain workflow evidence
Use one named input-schema projection for clients and catalog identity, preserve dialect declarations, and bound compact schema depth. Keep canonical SDK output validation when output advertisements are disabled. Release request-owned state on cancellation and carry upstream limitations through composed workflows. Preserve doctor full-output envelopes and validate navigation facts at their owner.

Reviewed fixes: #2088, #2108, #2513, #2684, #2693, #2695, #2836, #2864, #2868, #2877, #2933, #2998, #3002, #3015.

Validation: focused schema, SDK output-validation, context, and workflow tests passed; no schema dialect or canonical output validation was relaxed.
2026-10-14 08:32:00 +08:00
9878bb7296 feat(ghidra): apply validated analyzer overrides before analysis
Apply caller-selected analyzer opt-outs before Ghidra analysis, reject unknown or ineffective names, and commit raw/header imports to the same analyzer profile. Use supported x86 raw compiler specs and cover the real analyzer/readback and raw workflows.

Adopts the original implementation from #1800 with validated corrections. Refs #1783, #1836.

Co-authored-by: akushonkamen <74654766+akushonkamen@users.noreply.github.com>
Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-12 03:49:31 +08:00
069308bbe6 fix(hopper): preserve startup failures and recover denied launches
Preserve the primary Hopper startup failure and launcher diagnostics while retaining resources when cleanup cannot be confirmed. Reauthenticate matching external application sessions before document cleanup, keep the Python bridge available for bounded reconnects, and expose the provider-owned Hopper startup deadline.

Real headless Linux Hopper semantic analysis and 65 focused configuration, handshake, cleanup, and bridge tests passed. macOS authorization/crash causes and saved-HOP behavior require their own real-provider verification.

Refs #1808

Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-11 22:35:01 +08:00
sqmagellanandmorluto af88184eee feat(ghidra): configurable import language and analysis seeds (#1667)
* feat(ghidra): configurable import language and analysis seeds

REA_GHIDRA_LANGUAGE_ID and REA_GHIDRA_COMPILER_SPEC_ID set the import
language when the header cannot identify the compiler (for example,
Borland C++ PE images).

REA_GHIDRA_SEED_FILE seeds functions, code and labels before
auto-analysis, for code reached only through computed jumps or data.

Both are validated before launch, recorded in the analysis profile and
verified in the bridge handshake.

* fix(ghidra): parse language XML and preserve seed ordering

* test(ghidra): use explicit platform paths for seed launch checks

* fix(javascript): retain cleanup failure reason in error causes

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-11 09:57:38 +08:00
Haythem Sellamiandmorluto b6ad1f2c3d feat(mcp): compact input schema profile for size-capped providers (kimi-cli/Moonshot) (#1514)
* feat(mcp): add compact input schema profile for size-capped providers

Moonshot's function-calling API rejects a request while any single tool
parameters schema exceeds its literal serialized size cap, so clients
such as kimi-cli fail every request while the default catalog is
attached (#1484).

Add an opt-in REA_MCP_INPUT_SCHEMA_PROFILE=compact server presentation.
Each advertised input schema is rendered within a 13,312-byte budget,
measured to sit below the observed provider cap: annotation prose
(nested descriptions, examples, defaults, titles) is dropped, repeated
annotation-free subschemas are shared through schema-local $defs/$ref
references the provider validates without expansion, and the three
structurally oversized schemas are advertised in a documented reduced
form. The canonical catalog, tool set, output schemas, and server-side
canonical validation are unchanged; the default full profile is
byte-identical to today's advertisement.

Boundary tests pin every compact schema to the budget, keep them valid
JSON Schema that still accepts every canonical example, hold reduced
presentations at three or fewer, and verify canonical rejection still
guards a permissive advertisement. Verified against the real Moonshot
API through kimi-cli: prompts succeed with the compact catalog attached
where the default catalog fails every request with 400.

* fix(mcp): preserve compact schemas through Kimi conversion

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-10 16:58:43 +08:00
morluto 63bf91405c refactor!: enforce canonical evidence and provider boundaries
Deliver complete Evidence directly through MCP with required nullable analysis profiles and immutable per-server delivery settings. Bind selected environments at construction and require typed cleanup outcomes with retained retry ownership.

Remove test-only routing, duplicate session and snapshot metadata, producer-field repairs, and ambiguous provider arguments. Separate artifact byte identity from occurrence classification and retain typed source-map and partial native facts across CLI, MCP, and verifier consumers.

BREAKING CHANGE: obsolete Evidence envelopes, session aliases, incomplete snapshots, and missing normalized producer facts are no longer accepted.
2026-10-09 20:18:52 +08:00
Selimandmorluto d84bc8f11a feat(ghidra): make the startup deadline configurable (#974)
* feat(ghidra): make the startup deadline configurable

Large binaries (an 8 MB PE with about 20,000 functions) need more than the
fixed 330 s for import and full auto-analysis, so every first request timed
out. REA_GHIDRA_STARTUP_TIMEOUT_MS overrides the deadline in milliseconds;
absent, empty or invalid values keep the 330 s default.

* docs(ghidra): document REA_GHIDRA_STARTUP_TIMEOUT_MS

* fix(ghidra): parse startup deadlines through runtime configuration

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-09 07:21:36 +08:00
morluto 81ae838567 refactor(config): prune unused format wrappers and launcher export 2026-10-09 03:13:27 +08:00
NamnarakandYsopwnable 98b2b027af fix(linux): improve Hopper reliability and CachyOS support (#963)
* fix(linux): support CachyOS Hopper hosts

* fix(config): reuse user-local Hopper launcher

* fix(hopper): harden Linux demo dialog handling

* chore: retrigger CI after PR title fix

* fix(linux): align Hopper defaults and support docs

---------

Co-authored-by: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com>
2026-10-07 23:16:48 +08:00
N0zoM1z0 8238232485 feat(ida): add read-only GUI and headless MCP providers (#754) 2026-10-07 01:47:05 +08:00
N0zoM1z0 fb679b63c2 feat(ghidra): integrate optional NativeAOT metadata recovery (#748)
* feat(ghidra): integrate optional NativeAOT metadata recovery

* fix(ghidra): bind extension snapshots to configured artifacts

* test(ghidra): verify lazy NativeAOT startup failures

* fix(ghidra): preserve NativeAOT failure reasons and coverage

* fix(ghidra): preserve NativeAOT x64 calling conventions

* test(ghidra): guard recovered NativeAOT method conventions

* fix(ghidra): reject blocking extension file inputs

* fix(ghidra): diagnose incompatible NativeAOT extension classes

* test(ghidra): verify real extension initialization failures

* fix(ghidra): keep extension descriptor fields optional
2026-10-07 00:57:08 +08:00
morluto 735c10059c refactor(domain): split error taxonomy into thematic modules (#636)
Move each error family out of domain/errors.ts into its own module
(base, core, provider selection/adapter, browser observation,
artifact, evidence, unknown registry, hopper, configuration) and
relocate AnalysisErrorProjection to analysisErrorProjection.
Migrate all 162 importers to the owning modules and delete
errors.ts. No re-export shims; behavior is unchanged.
2026-10-06 05:57:38 +08:00
morluto e07b81649e refactor!: simplify runtime tools and remove replay engines (#572)
* fix(hopper): report unavailable procedure body coverage

* refactor!: simplify runtime tools and remove replay engines

Keep direct inspection, capture, and comparison tools. Bind handlers to named contracts and let agents compose experiments with commands and local fixture servers.

BREAKING CHANGE: Remove controlled replay, Node characterization, finite replay-machine, managed runtime planning, and redundant string-tracing tools. Retire process replay, shims, reactive scenarios, custom checkpoints, and active browser replay and origin-scope inputs.
2026-10-05 21:15:04 +08:00
morluto b5e98915c6 fix(tools)!: simplify direct invocation and harden boundary contracts (#555)
Remove REA permission grants, scope ceilings, approval confirmations, and inert settings. Preserve selected local evidence, accept caller-chosen commands and paths, and report actual runtime failures.

Harden provider results, schema serialization, snapshot evidence binding, archive paths, source maps, native screenshots, managed EH parsing, and cancellation cleanup. Regenerate tool metadata and document direct invocation.

BREAKING CHANGE: Removed permission configuration and policy commands, approval fields, filesystem scope inputs, and fixed confirmation options. Process capture inherits the host environment and accepts command names; callers must refresh tool schemas and migrate removed fields.
2026-10-05 20:16:44 +08:00
morluto c34efca4cd refactor(files): use caller paths for local evidence files 2026-09-29 14:53:20 +08:00
morluto 9e7605482a refactor(coverage): evaluate inline without workspaces 2026-09-29 08:45:00 +08:00
morluto a2a3243ea8 refactor: return complete analysis results inline 2026-09-29 08:07:24 +08:00
morluto 0d96275b68 refactor: return complete tool results inline 2026-09-29 07:08:46 +08:00
morluto b8a9da0c0f refactor(mcp): make analysis states unrepresentable (#468)
* refactor(config): parse capability settings into exact states

* refactor(runtime): encode replay and process outcomes exactly

* refactor: make analysis states unrepresentable

* refactor(mcp): register canonical schemas with SDK

* fix: preserve capability and catalog invariants

* fix(build): remove stale compiled modules

* fix(replay): update real verifier policy dependency

* fix(package): verify the complete MCP catalog

* fix(test): supply complete replay reload policy

* refactor(contracts): centralize output schema lookup

* refactor(mcp): delegate protocol boundaries to SDK

* fix(browser): approve real source capture

* fix(mcp): preserve catalog guidance
2026-08-10 02:28:11 +08:00
morluto 8f229ede70 feat(electron): add autonomous Electron runtime analysis (#464)
* feat: add autonomous Electron runtime analysis

* feat(electron): add lifecycle and boundary telemetry

* feat(electron): capture runtime boundary evidence

* fix(electron): package boundary hook helper

* feat(electron): report active capture coverage

* fix(electron): account for bring-your-own runtime

* feat(electron): expand agent-owned runtime scenarios

* fix(ghidra): keep fixture credentials out of argv

* perf(test): serialize local test and docs workloads

* fix(electron): harden active runtime capture

* fix(inspector): require exact runtime target authorization

* fix(docs): make managed manifest host independent

* fix(ci): make generated catalog portable

* fix: unblock autonomous command and electron flows

* fix(electron): close popup and detached helper gaps
2026-08-05 19:07:46 +08:00
morluto 035e1144b0 feat(server): advertise available tools dynamically 2026-07-24 13:18:58 +00:00
morluto c6ee0049c4 refactor(app): split session and CLI workflows 2026-07-18 05:49:14 +00:00