mirror of
https://github.com/morluto/rea.git
synced 2026-11-13 18:22:42 +00:00
merge: resolve environment changes from main
This commit is contained in:
morluto committed
commit
06cdfe71e9
622 files changed
+24509
-196582
No files matched your search
@@ -286,6 +286,162 @@ instructions into field declarations, leaving an empty or incomplete method
|
||||
body. The selected method's disassembly preserves those instructions and retains
|
||||
the same source byte limit and explicit truncation reporting.
|
||||
|
||||
## Apply an R8/ProGuard mapping
|
||||
|
||||
Release builds shrunk by R8 or ProGuard rename classes and members. Select the
|
||||
build's `mapping.txt` with `mapping_path` on `inspect_android_class` or
|
||||
`inspect_android_method`, or `--mapping-path` on the matching CLI command:
|
||||
|
||||
```sh
|
||||
rea inspect-android-class /path/Example.apk a.b --mapping-path /path/mapping.txt \
|
||||
--detail summary --evidence_output /tmp/class-evidence.json
|
||||
```
|
||||
|
||||
For agent workflows, select `detail: "summary"` on class and method inspection.
|
||||
It preserves the complete normalized member inventory or selected method body,
|
||||
including mapping annotations, while retaining duplicate raw provider calls in
|
||||
the complete parent Evidence. The reply links that parent's identity and digest;
|
||||
export it with `export_evidence_bundle` or read its summary again through
|
||||
`inspect_analysis_view`. CLI summaries require `--evidence_output` and return
|
||||
the portable bundle path. Omitted or `complete` detail preserves complete inline
|
||||
Evidence. Search and reference operations retain their existing contracts.
|
||||
Symbol summary views admit at most **1 MiB**, including view context. An admitted
|
||||
method's source can already consume that amount; a larger combined summary
|
||||
fails explicitly instead of silently truncating facts. MCP refusal includes
|
||||
the retained parent ID. Select complete detail, or inspect one method rather
|
||||
than a whole class, when the requested summary does not fit.
|
||||
|
||||
The mapping path must be absolute. Selectors still take the observed names, such as
|
||||
`a.b`, and observed rows are unchanged. The result adds one `mapping` block with
|
||||
the file's path, SHA-256 and size, the header values the file reports
|
||||
(`compiler`, `compiler_version`, `min_api`, `map_version`, `pg_map_id`,
|
||||
`pg_map_hash`), valid record counts and up to 16 malformed line samples. Its
|
||||
`symbols` list has one entry per observed class, inner class, method and field;
|
||||
`inspect_android_method` lists the class and the selected method. `coverage`
|
||||
counts each resolution status. With summary delivery, these unchanged facts are
|
||||
under the view's `summary` field; the view separately reports source coverage.
|
||||
|
||||
Lookups use the bridge's raw DEX names, not JADX display aliases such as a
|
||||
renamed reserved word. Each symbol resolves as:
|
||||
|
||||
- `mapped`: exactly one record matches. A class needs one record for its raw
|
||||
name (`verified_by: "class"`). A method must also match its DEX prototype
|
||||
(`"descriptor"`) and a field its DEX type (`"type"`). R8 residual signature
|
||||
metadata is used when present; otherwise original types are translated
|
||||
through the file's class records.
|
||||
- `ambiguous`: `multiple-records`, `descriptor-mismatch`, `type-mismatch`,
|
||||
`ambiguous-class` or `residual-signature-conflict`, with `candidate_count` and
|
||||
up to 8 candidates. Conflicting residual metadata never selects a winner. A name match
|
||||
whose prototype or type does not verify is never reported as mapped.
|
||||
- `unmapped`: no valid record matched the raw identity.
|
||||
- `unknown` with reason `projection-budget`: this claim did not fit the combined
|
||||
reply budget. The observed raw symbol remains present; `coverage.unknown`
|
||||
counts these omissions. Inspect that specific class or method separately,
|
||||
or consult the selected local mapping file for complete source records.
|
||||
|
||||
A line continues an inline frame stack when it repeats the previous line's
|
||||
minified range and obfuscated name and gives a single call-site line, such as
|
||||
`:7`. Only the outermost original of a stack becomes a candidate. Original
|
||||
line numbers and source files are not projected. Malformed records are counted
|
||||
and skipped, so an `unmapped` symbol might be described by one of them. A file
|
||||
that is missing, not a regular file, not UTF-8, larger than the byte limit or
|
||||
without any class record, such as `usage.txt` or `seeds.txt`, is refused before
|
||||
JADX is used. Requests read their mappings one at a time, in the same queue as
|
||||
the JADX work and under the same operation deadline. A mapping failure therefore never starts or retires the JADX
|
||||
session, and a different mapping reuses the loaded APK.
|
||||
|
||||
## Link native declarations to JNI candidates
|
||||
|
||||
When advertised, `inspect_android_jni_bindings` links original `ACC_NATIVE`
|
||||
declarations from standard root `classes*.dex` members to bundled ELF libraries.
|
||||
The supplied JAR's DEX reader preserves duplicate declarations before decompiler
|
||||
deduplication. This operation does not load the APK into the decompiler.
|
||||
MCP defaults to retained summary delivery when `detail` is omitted; select
|
||||
`complete` explicitly to receive the full report inline.
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "/absolute/Example.apk",
|
||||
"class_name": "org.conscrypt.NativeCrypto",
|
||||
"method_names": ["EVP_PKEY_new_RSA"],
|
||||
"abis": ["arm64-v8a"],
|
||||
"detail": "summary"
|
||||
}
|
||||
```
|
||||
|
||||
Class/method names and optional `descriptors` select raw DEX identities.
|
||||
`library_paths` selects exact logical APK paths such as
|
||||
`lib/arm64-v8a/libconscrypt_jni.so`; each library's `source_path` preserves its
|
||||
original ZIP spelling. Omitted filters include all scoped facts; `[]` matches
|
||||
none. Class selection never narrows the global inventory used for table-class
|
||||
attribution.
|
||||
|
||||
Independent `declarations`, `libraries`, `candidates` and `bindings` collections
|
||||
have stable IDs and source ordinals. Candidates retain APK/library digests,
|
||||
ABI, function addresses and file offsets. Table candidates also retain section,
|
||||
table and entry identity plus all three name/signature/function pointer slots
|
||||
and their relocation sources. JNI exports follow the
|
||||
[UTF-16 escaping and short-before-long rules](https://docs.oracle.com/en/java/javase/21/docs/specs/jni/design.html#resolving-native-method-names);
|
||||
failed escaping does not acquire an invented export match. Tables require
|
||||
NUL-terminated [Modified UTF-8 strings](https://docs.oracle.com/en/java/javase/21/docs/specs/jni/types.html#modified-utf-8-strings),
|
||||
valid descriptors and executable load mappings. REL, RELA and RELR retain
|
||||
separate provenance; unsupported packed relocations and unresolved pointer
|
||||
spans stay explicit unknowns.
|
||||
|
||||
A static `JNINativeMethod` entry names no class. Attribute it uniquely only
|
||||
when exactly one declaration in the complete recorded native inventory has
|
||||
that `(name, descriptor)` pair. Multiple classes remain ambiguous after a
|
||||
class filter; a partial inventory cannot establish unique attribution.
|
||||
|
||||
Each selected declaration has a row per selected/observed ABI. `matched` means
|
||||
candidate presence was observed; `ambiguous` means multiple observed targets
|
||||
or class associations remain. Binding `coverage` is separate: a matched
|
||||
partial row does not establish complete uniqueness, and an empty partial scan
|
||||
is `unknown`, never `unmatched`. ELF/ABI label mismatches preserve facts without
|
||||
asserting a binding. No result establishes library loading, runtime registration
|
||||
or method invocation.
|
||||
|
||||
Summary returns the first 64 selected binding rows, their declaration/candidate
|
||||
closure and every selected library's identity, coverage and failure reason;
|
||||
counts and page coverage describe the full selected binding set. Selected
|
||||
declaration IDs have separate preview coverage, so native declarations remain
|
||||
visible even when the APK has no bundled ABI libraries. Complete
|
||||
Evidence stays retained. Read exact fields or further pages without reopening
|
||||
files or rerunning analysis:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": { "kind": "retained-evidence", "evidence_id": "ev_<digest>" },
|
||||
"view": {
|
||||
"kind": "android-jni",
|
||||
"collection": "bindings",
|
||||
"offset": 64,
|
||||
"limit": 64,
|
||||
"fields": ["declaration_id", "abi", "status", "coverage", "candidate_ids"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Collections also include `declarations`, `candidates` and `libraries`, with
|
||||
exact filters and batch lookup by declaration/candidate/library IDs. `row_index`
|
||||
is the original parent collection index; DEX and
|
||||
ZIP source ordinals stay separate. Complete detail returns the full report.
|
||||
CLI uses the same selection and delivery contracts. Its common one-shot
|
||||
default remains complete; summary delivery requires an explicit Evidence path:
|
||||
|
||||
```sh
|
||||
rea inspect-android-jni-bindings /absolute/Example.apk \
|
||||
--query '{"class_name":"org.conscrypt.NativeCrypto","method_names":["EVP_PKEY_new_RSA"],"abis":["arm64-v8a"]}' \
|
||||
--detail summary --evidence_output /absolute/fresh-jni-evidence.json
|
||||
```
|
||||
|
||||
Optional `--view` accepts the same JSON page and requires summary delivery.
|
||||
Evidence bundles never overwrite files. Complete declarations and one library
|
||||
selection are cached within the immutable APK session. Changing ABI/library
|
||||
selection replaces the scan; class/method selection reuses it. Fresh analysis
|
||||
still validates current caller APK bytes. Idle/close releases the caches;
|
||||
retained Evidence remains historical and independently readable.
|
||||
|
||||
## Interpret the result
|
||||
|
||||
- Package inspection retains decoded manifest XML, summary declarations,
|
||||
@@ -322,6 +478,11 @@ the same source byte limit and explicit truncation reporting.
|
||||
`dex_descriptor`, for example `(I)V`. Display names can be JADX aliases;
|
||||
the descriptor alone is not a complete raw DEX method ID. Overload indices
|
||||
are tied to this artifact and engine, not stable identities across builds.
|
||||
- A `mapping` block appears only when the caller selected `mapping_path`. Its
|
||||
names are claims from that file, not observations of the APK. REA does not
|
||||
check that the mapping belongs to this build; `pg_map_id` is reported, not
|
||||
compared. Evidence keeps the selected path in its parameters and the file's
|
||||
SHA-256 in the result.
|
||||
- Method inspection reports Java or smali, fallback markers and complete/partial
|
||||
returned text. Decompiled source is a derived representation. A complete text
|
||||
result does not establish complete semantics or a byte-identical reconstruction.
|
||||
@@ -411,6 +572,19 @@ target/configuration changes, failure, timeout, cancellation, MCP disconnect or
|
||||
server shutdown. One-shot CLI commands always join cleanup before returning.
|
||||
REA never writes to the original APK or launches target code.
|
||||
|
||||
JNI scans reuse the analyzer process without loading the APK into the
|
||||
decompiler. They admit at most 64 MiB per DEX member and 256 MiB of DEX bytes,
|
||||
with a 4 MiB native-declaration JSON allowance before MCP string escaping.
|
||||
Native libraries are processed sequentially: one 64 MiB buffer, 256 MiB total
|
||||
selected read work, and a 16 MiB parsed-library-facts allowance. ZIP inventory
|
||||
and library counts also have explicit resource bounds. Unexamined occurrences
|
||||
remain unknown. A complete relationship report has a separate 16 MiB serialized
|
||||
allowance; summaries/pages allow 1 MiB including their provenance envelope.
|
||||
These are resource admissions, not claims about maximum valid Android formats.
|
||||
Select fewer ABI/library occurrences or smaller retained pages when needed.
|
||||
Completed parsed library facts survive failed reader finalization in typed
|
||||
partial observations; the exact reader owner remains available for cleanup retry.
|
||||
|
||||
On Windows, stdin writes preserve stream backpressure and run outside the Node
|
||||
event loop. Cancellation terminates the retained job, including descendants,
|
||||
before workspace cleanup. On Windows, a detached REA helper owns the private
|
||||
@@ -496,6 +670,27 @@ complete, case-sensitive search contract. A protocol frame above **8 MiB**, or c
|
||||
**32 MiB per operation**, fails with a diagnostic instead of returning an incomplete inventory
|
||||
as complete. These are memory-safety boundaries, not undocumented result caps.
|
||||
|
||||
A selected R8/ProGuard mapping is re-read and SHA-256 checked on every call, up
|
||||
to **64 MiB**. The serialized provider keeps one successful parsed index, keyed
|
||||
by byte length and digest with fixed parser options. Identical bytes reuse that
|
||||
index; changed bytes are reparsed, missing files still fail, and each response
|
||||
preserves the currently selected path. The index is released on provider idle
|
||||
retirement or close. Omitting `mapping_path` never applies the cached mapping.
|
||||
During replacement, the old index can coexist with the new parse until it
|
||||
succeeds; these transient allocations count toward process memory.
|
||||
|
||||
Before retaining records, the parser admits an estimated **128 MiB** index
|
||||
representation and **8 MiB** of expanded mapped claims per residual class,
|
||||
including shared header facts. The index allowance accounts for strings and
|
||||
containers; it is not a measured V8 heap cap. Parameter and metadata expansion
|
||||
is admitted before splitting or JSON decoding. These fixed resource bounds
|
||||
have no runtime override; refusal reports `resource_constraint`, the source
|
||||
line and mapping SHA-256. Ambiguous lookups count all candidates while keeping
|
||||
only the first eight in stable file order. A separate **8 MiB** combined claim
|
||||
budget applies to the actual query set, including outer and inner classes,
|
||||
before cloning the report. Claims that do not fit become explicit `unknown`
|
||||
entries; other admitted claims and all observed identities remain available.
|
||||
|
||||
## Real verification
|
||||
|
||||
Download the explicit public fixtures once; this installs no tools:
|
||||
|
||||
@@ -75,7 +75,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Domain Layer"
|
||||
DOMAIN["src/domain/<br/>Pure side-effect-free modules<br/>format declarations, Evidence, comparisons, application graphs"]
|
||||
DOMAIN["src/domain/<br/>Pure side-effect-free modules<br/>format declarations, provider selection, Evidence, comparisons, application graphs"]
|
||||
end
|
||||
|
||||
subgraph "Configuration"
|
||||
|
||||
@@ -497,17 +497,51 @@ comments and unrelated settings. It resolves that file from `HERMES_HOME`, else
|
||||
from the platform default Hermes itself uses — `%LOCALAPPDATA%\hermes` on
|
||||
Windows, `~/.hermes` elsewhere — including any `HERMES_DATA_DIR_SUFFIX`. Hermes
|
||||
connects an entry unless `enabled` is false, so setup states `enabled: true`.
|
||||
Aliases whose anchor is defined in the `rea` entry retain their original resolved
|
||||
values when setup refreshes or uninstall removes that entry. Setup does not
|
||||
redirect another server's alias to REA's new launcher or policy.
|
||||
This also covers an anchor on the removed registration key. References to
|
||||
definitions outside the entry keep sharing the original value, including when
|
||||
another definition later reuses the same anchor name.
|
||||
|
||||
Hermes personal skills install under the active Hermes home's `skills` directory.
|
||||
Setup, doctor and uninstall follow the root's `active_profile` selection, as
|
||||
plain Hermes CLI commands do. An explicit `HERMES_HOME` pointing to
|
||||
`profiles/<name>` selects that named profile directly. An invalid, missing or
|
||||
deleted sticky profile is reported instead of writing into the default home;
|
||||
it also remains an unresolved skill selection, so setup cannot report a
|
||||
successful empty installation. Overrides nested beneath the native Hermes home
|
||||
use that native home's sticky selection and profile root together, matching
|
||||
[Hermes's root rules](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_constants.py#L83).
|
||||
Repair the profile or run `hermes profile use default` before retrying.
|
||||
Setup, doctor and uninstall share REA's sticky-profile resolver.
|
||||
A custom `HERMES_HOME` outside the platform default home
|
||||
reads its own `active_profile`. An override physically inside the default home
|
||||
uses that default home's sticky selection and profile root together, including
|
||||
when the final override directory does not yet exist.
|
||||
|
||||
For example, with `HERMES_HOME=~/.hermes/sub` and `~/.hermes/active_profile` set
|
||||
to `coder`, setup selects `~/.hermes/profiles/coder/config.yaml` and that
|
||||
profile's skills directory. It does not read `~/.hermes/sub/active_profile`.
|
||||
Without an outer named selection, this override uses `~/.hermes/sub/config.yaml`.
|
||||
Use a home outside the platform default tree for an independent sticky selection.
|
||||
|
||||
This describes REA's current routing, not full Hermes CLI equivalence.
|
||||
At Hermes commit `748c79a1`, the
|
||||
[sticky lookup](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_cli/main.py#L612-L632)
|
||||
also reads the outer selection, but
|
||||
[`resolve_profile_env`](https://github.com/NousResearch/hermes-agent/blob/748c79a1c6873124d84071cdabb4062a75913e1f/hermes_cli/profiles.py#L2431-L2461)
|
||||
resolves it beneath the configured `HERMES_HOME`. For nested overrides, confirm
|
||||
the profile Hermes actually uses and point `HERMES_HOME` directly at that
|
||||
`profiles/<name>` directory before setup. Compatibility is tracked in
|
||||
[#3939](https://github.com/morluto/rea/issues/3939).
|
||||
|
||||
REA resolves existing ancestor directories before checking physical containment,
|
||||
including symlinks and junctions, even when the override directory does not exist.
|
||||
An override beneath a symlinked default home therefore uses the default home's
|
||||
sticky selection. If `~/.hermes/link` points outside the default home,
|
||||
`HERMES_HOME=~/.hermes/link/new` instead uses that override root, whether or not
|
||||
`new` exists. Previewed destinations retain the configured path spelling.
|
||||
An unreadable path or dangling alias leaves the home unresolved; repair that path
|
||||
before configuring Hermes. Inspect setup's previewed destinations before applying
|
||||
changes.
|
||||
|
||||
An explicit `HERMES_HOME` pointing to `profiles/<name>` selects that named
|
||||
profile directly, bypassing sticky selection. An invalid, missing or deleted
|
||||
sticky profile is reported instead of writing into the default home; repair the
|
||||
profile or run `hermes profile use default` before retrying. It also remains an
|
||||
unresolved skill selection, so setup cannot report a successful empty installation.
|
||||
REA follows `HERMES_HOME` whitespace trimming, environment-variable expansion,
|
||||
and home expansion before resolving that profile. Setup previews the resolved
|
||||
configuration and skill destinations; doctor and uninstall use those same paths.
|
||||
@@ -524,10 +558,12 @@ installs its skill copy there.
|
||||
## Factory Droid
|
||||
|
||||
For [Factory Droid](https://docs.factory.com/harness/mcp), setup writes
|
||||
`mcpServers.rea` into the user-level `~/.factory/mcp.json`. Entries carry
|
||||
`command`, `args`, and `env`. Droid discovers personal skills from
|
||||
`~/.factory/skills` ([skills](https://docs.factory.com/harness/skills)), so
|
||||
setup installs its skill copy there.
|
||||
`mcpServers.rea` into the user-level `~/.factory/mcp.json` (the
|
||||
`FACTORY_HOME_OVERRIDE` override replaces the home that contains `.factory`).
|
||||
Entries carry `command`, `args`, and `env`. Droid discovers personal skills
|
||||
from `~/.factory/skills` ([skills](https://docs.factory.com/harness/skills),
|
||||
following `FACTORY_HOME_OVERRIDE` when set), so setup installs its skill copy
|
||||
there.
|
||||
|
||||
## Trae
|
||||
|
||||
@@ -703,10 +739,6 @@ can override an inherited disable. Lower files and other server entries remain
|
||||
unchanged; a legacy server named `servers` that conflicts with the native table
|
||||
requires manual repair before setup can write.
|
||||
|
||||
When Hermes aliases refer to an anchored REA entry, refreshing or removing REA
|
||||
retains the aliases' original resolved values. Setup does not redirect another
|
||||
server's alias to REA's new launcher or policy.
|
||||
|
||||
With no client selected, the interactive skill choice targets the shared
|
||||
`~/.agents/skills` copy. Selecting a client targets its personal destination
|
||||
when supported. Use `rea setup --skill-client shared --skill` for the shared
|
||||
|
||||
@@ -36,7 +36,7 @@ The MCP tool `project_harmony_application_graph` takes the same
|
||||
|
||||
- exact component paths, artifact IDs, and hashes (manifests, Ark bytecode,
|
||||
resources, native libraries, JavaScript, signing)
|
||||
- a packaging-model hint: Stage is claimed only from a `module.json` entry
|
||||
- a packaging-model hint: Stage is claimed only from an exactly cased `module.json` entry
|
||||
outside a classified resource path, FA only from `config.json` outside a
|
||||
classified resource path, otherwise `unknown`
|
||||
- FA resources under `assets/<module>/resources/` as well as Stage `resources/`
|
||||
@@ -46,6 +46,9 @@ The MCP tool `project_harmony_application_graph` takes the same
|
||||
- path-based N-API bridge hypotheses between bytecode and `libs/<abi>` native
|
||||
libraries
|
||||
|
||||
An unavailable manifest can still supply a path-based model hint; its content
|
||||
identity remains unknown. Manifest names follow case-sensitive ZIP member names.
|
||||
|
||||
File and slice occurrences supply component and child-package paths; directory
|
||||
trees and symlinks retain their identities in the source inventory without
|
||||
being classified as code files. Ark and JavaScript runtime hints summarize the
|
||||
|
||||
@@ -240,7 +240,7 @@ The verifier requires the resolved procedure in both inventory and procedure sea
|
||||
|
||||
The initial real workflows use stdio registrations for the legacy Windows GUI profile and a Windows x64 headless supervisor with IDA 9.3. The headless workflow has also passed on macOS arm64 with IDA Professional 9.1 (build 9.1.250226) against the same upstream commit `c133c3853faa111a9b00ee615c013b720d0c4acd`, using a source-built arm64 Mach-O fixture; Mach-O C symbols carry a leading underscore, so the procedure name is `_rea_fixture_add`. Streamable HTTP is tested against a loopback MCP fixture with the pinned client SDK; a real IDA HTTP-supervisor workflow remains unverified. Linux headless engine operation, other IDA releases, architectures, and modern attached GUI tools require their corresponding real workflows before support is claimed.
|
||||
|
||||
Native Windows verification uses IDA Pro 9.3 with the public two-function fixture and Windows `whoami.exe`, using upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c` with a loopback-listener change. It covers production CLI/MCP parity, distinct-function switching, identity and connection cleanup. See the [correctness and performance measurements](https://github.com/morluto/rea/blob/main/experiments/analysis-efficiency/ida-native-windows.md). This follow-up did not run Windows IDA Free or native macOS.
|
||||
Native Windows verification uses IDA Pro 9.3 with the public two-function fixture and Windows `whoami.exe`, using upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c` with a loopback-listener change. It covers production CLI/MCP parity, distinct-function switching, identity and connection cleanup. See the [correctness and performance measurements](https://github.com/morluto/rea/blob/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency/ida-native-windows.md). This follow-up did not run Windows IDA Free or native macOS.
|
||||
|
||||
Headless readiness distinguishes an explicit `auto_analysis_ready: false` from
|
||||
unknown readiness (`null`). The former returns a retryable capability refusal
|
||||
|
||||
+107
-7
@@ -3,13 +3,14 @@
|
||||
Run REA, connect your selected agents, and check the tools needed for your task.
|
||||
Setup previews configuration changes and asks for approval before applying them.
|
||||
|
||||
| Task | Section |
|
||||
| --------------------------------- | --------------------------------------------------------------------------------------------------- |
|
||||
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
|
||||
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
|
||||
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), the [IDA guide](ida-provider.md), or [Kuna](kuna-provider.md) |
|
||||
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
|
||||
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
|
||||
| Task | Section |
|
||||
| ---------------------------------- | --------------------------------------------------------------------------------------------------- |
|
||||
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
|
||||
| Install a Claude Code/Codex plugin | [Agent plugins and marketplaces](#agent-plugins-and-marketplaces) |
|
||||
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
|
||||
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), the [IDA guide](ida-provider.md), or [Kuna](kuna-provider.md) |
|
||||
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
|
||||
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
|
||||
|
||||
## Start setup
|
||||
|
||||
@@ -444,6 +445,9 @@ and other MCP servers. Purging removes REA's legacy cache and state under
|
||||
archives under `${XDG_CACHE_HOME:-~/.cache}/rea/hopper-releases`. Other cache
|
||||
namespaces remain intact. A caller-selected `REA_KUNA_CACHE_DIR` is retained
|
||||
and reported because REA cannot establish ownership of its entire contents.
|
||||
Lexically equivalent spellings of that default directory, including a trailing
|
||||
separator, a `.` segment, or a repeated separator, share its purge result.
|
||||
The comparison preserves POSIX `..` traversal and literal backslashes.
|
||||
A client configuration that is malformed, unreadable, or at an unsafe
|
||||
path stops the operation before anything is removed, and a client that fails
|
||||
while being updated stops the remaining removals. Uninstall reports each
|
||||
@@ -1433,6 +1437,21 @@ current setting. For Windows PE resource analysis, the exact name is
|
||||
`WindowsResourceReference`. Disabling an analyzer skips the facts it would have
|
||||
produced.
|
||||
|
||||
Ghidra's Objective-C Message Analyzer can log
|
||||
`Required Objective-C data type not found in data type manager` /
|
||||
`Try adding libobjc.dylib` when a Mach-O links Objective-C runtime symbols but
|
||||
defines no class metadata (thin Cocoa glue in a C++ binary is a common case).
|
||||
That text is non-fatal Ghidra analyzer output: auto-analysis continues, and the
|
||||
analyzer simply skips `objc_msgSend` call-target recovery. REA's headless import
|
||||
does not accept a "add this library" option, and on modern macOS
|
||||
`/usr/lib/libobjc.A.dylib` exists only in the dyld shared cache, so there is no
|
||||
on-disk library to add. Treat the line as a coverage limitation, not an REA
|
||||
failure; check the session's `failure_kind` (and related provider errors) for
|
||||
any real abort. To silence that analyzer, set
|
||||
`REA_GHIDRA_DISABLED_ANALYZERS="Objective-C Message Analyzer"`. For large
|
||||
executables that miss the startup deadline instead, raise
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS` and the [Ghidra heap](#ghidra-heap-and-cpu-controls).
|
||||
|
||||
### Ghidra heap and CPU controls
|
||||
|
||||
Set resource controls in the environment that launches the CLI or MCP server,
|
||||
@@ -1534,6 +1553,87 @@ read-only operations, digest identity, and runtime cleanup. The separate
|
||||
ordinary user. It requires the built Windows native bundle and an existing
|
||||
Ghidra/JDK installation; neither is inferred from startup alone.
|
||||
|
||||
## Agent plugins and marketplaces
|
||||
|
||||
REA's npm package is the shared Claude Code and Codex plugin artifact. It
|
||||
contains `.claude-plugin/plugin.json`, `.codex-plugin/plugin.json`, and the
|
||||
canonical generated `skills/` tree. Both manifests select `./skills` and run a plain Node bootstrap pinned to
|
||||
that exact package version. The client must have a supported standalone `node`
|
||||
on its PATH. Claude expands its documented `CLAUDE_PLUGIN_ROOT` in the script
|
||||
argument; Codex resolves `cwd: "."` from the installed plugin root before
|
||||
launching the relative script. The bootstrap verifies the package version and runs its bundled runtime in
|
||||
the same process, preserving stdio and REA's provider/worker cleanup. It does
|
||||
not start another package runner or download a different runtime. The existing release skill
|
||||
generator also pins CLI recovery commands to the same version.
|
||||
|
||||
The repository's single-plugin marketplaces are declared in
|
||||
`.claude-plugin/marketplace.json` and `.agents/plugins/marketplace.json`.
|
||||
Both select an exact `rea-agents` npm version instead of mutable source-checkout
|
||||
contents. Release Please updates their selectors alongside the plugin manifests.
|
||||
No plugin-only publication or generated skill files committed to Git are needed.
|
||||
The qualified release archive contains a runtime-only manifest and
|
||||
`npm-shrinkwrap.json` plus bundled runtime dependencies, including PTY packages
|
||||
for each supported OS and CPU; development dependencies and development-only overrides
|
||||
stay in the source manifest and lockfile. Both clients receive this same archive.
|
||||
|
||||
This distribution becomes installable after the first qualified npm release
|
||||
containing these plugin manifests. Earlier published packages are not changed
|
||||
by this source proposal. For local source development, run `npm ci && npm run build:cached` to generate
|
||||
the runtime and packaged skill before loading the repository as a plugin.
|
||||
A source plugin uses the locally built runtime and generated skill; rebuild after
|
||||
source changes. Missing builds and mismatched version pins fail with an actionable
|
||||
error. The authored `.agents/skills/` tree is not the plugin's shipped skill layout.
|
||||
Windows source workflows that need native controls still require the
|
||||
[Windows source-checkout build](https://github.com/morluto/rea/blob/main/native/windows/README.md#build).
|
||||
|
||||
In Claude Code:
|
||||
|
||||
```text
|
||||
/plugin marketplace add morluto/rea
|
||||
/plugin install rea@rea
|
||||
```
|
||||
|
||||
In Codex CLI, add the marketplace and install REA:
|
||||
|
||||
```bash
|
||||
codex plugin marketplace add morluto/rea
|
||||
codex plugin add rea@rea
|
||||
```
|
||||
|
||||
Each release updates both manifest versions and their pinned server version
|
||||
together, which is when installed copies refresh the bundled skill and move to
|
||||
the new server. A plugin does not run `rea setup`, so provider
|
||||
configuration still follows the sections above. Use the plugin or a
|
||||
`rea setup` registration for a client, not both, to avoid two `rea` servers.
|
||||
|
||||
`rea doctor` does not see plugin-owned state. Its client and skill checks read
|
||||
only the registrations and skill locations that `rea setup` manages, so for a
|
||||
plugin-only install `rea doctor --client <id>` and `rea doctor --skill` report
|
||||
`missing` and recommend `rea setup`. Do not follow that remediation. Check the
|
||||
plugin with the client instead (`claude plugin list` and `/mcp` in Claude
|
||||
Code, `codex plugin list` in Codex).
|
||||
|
||||
For analysis engines, ask the connected server: have the agent call
|
||||
`binary_session` and read `result.tool_availability`, which reports what the
|
||||
server the plugin started can use, with reasons and remediation. See
|
||||
[MCP contracts](mcp-contracts.md).
|
||||
|
||||
A plugin does not put `rea` on your `PATH`. To check an engine from a
|
||||
terminal, use the package runner at the version the plugin pins:
|
||||
|
||||
<!-- x-release-please-start-version -->
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@6.8.0 doctor --provider ghidra --json
|
||||
```
|
||||
|
||||
<!-- x-release-please-end -->
|
||||
|
||||
If your installed plugin is older, use the version in its manifest instead.
|
||||
That report describes the terminal's environment, not the plugin server's.
|
||||
The server inherits the client's environment, so a setting such as
|
||||
`GHIDRA_INSTALL_DIR` that exists only in your terminal does not reach it.
|
||||
|
||||
## MCP Registry
|
||||
|
||||
REA is published in the official MCP Registry as `io.github.morluto/rea`. Registry
|
||||
|
||||
+13
-2
@@ -1248,11 +1248,22 @@ failures for such entries name this cause and the operation to rerun with
|
||||
Advertised input and output schemas contain no reachable recursive references.
|
||||
Input schemas inline repeated definitions before applying the selected
|
||||
presentation profile. Both `full` and `compact` start from this self-contained
|
||||
projection; output schemas can share schema-local definitions. Full inputs
|
||||
retain nested fields, descriptions and valid literal examples. The compatibility
|
||||
projection; output schemas can share schema-local definitions. Analyst-authored
|
||||
full inputs retain nested fields, descriptions and valid literal examples. The compatibility
|
||||
projections below can relax advertised constraints; canonical runtime validation
|
||||
still enforces the complete input contract.
|
||||
|
||||
`compare_web_captures` declares its complete producer observations as object
|
||||
operands instead of repeating their full output schemas in the input catalog.
|
||||
Pass the original `inspect_web_page` normalized result in `before.inspection`
|
||||
and `after.inspection`, optional complete `discover_webmcp_tools` results in
|
||||
`webmcp`, or complete `capture_browser_scenario` results in the scenario fields.
|
||||
Do not construct partial observations from the abbreviated declaration: the
|
||||
server still validates every canonical field before comparison. Input groups
|
||||
and analyst-authored normalization rules remain explicit. This presentation
|
||||
changes no capture bytes, identity, completeness, or comparison semantics and
|
||||
does not reduce `inspect_analysis_view` selectors or other tools' declarations.
|
||||
|
||||
When a host forwards input schemas to the model, their examples consume context.
|
||||
REA's advertised examples therefore never copy complete producer Evidence.
|
||||
An embedded Evidence record is shown as its exact `{"kind":"retained-evidence","evidence_id":"ev_..."}`
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -1,112 +0,0 @@
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"repository": "morluto/rea",
|
||||
"version": "6.7.0",
|
||||
"base": "2c4338db1fdb7773e05376301c558cff2a0b4b80",
|
||||
"target": "b83b9caf4a5f0241a5fd370792f1aed1bffbb2cf",
|
||||
"commits": [
|
||||
{
|
||||
"sha": "8cbaf6c7021b97dd4de6b5771c35fbed06b240e9",
|
||||
"subject": "fix(tests): isolate command finalization module mocks",
|
||||
"body": "fix(tests): isolate command finalization module mocks\n\nMove the five registered WASM finalization cases into the existing per-file isolated project. Preserve all result, exit-code, observation and cleanup assertions. A preloaded shared CLI graph reproduces three failures before the move; the isolated covered packet passes.\n\nFixes #4959\n",
|
||||
"parents": ["2c4338db1fdb7773e05376301c558cff2a0b4b80"],
|
||||
"authors": [
|
||||
{
|
||||
"name": "morluto",
|
||||
"email": "76467478+morluto@users.noreply.github.com",
|
||||
"user": {
|
||||
"login": "morluto",
|
||||
"__typename": "User"
|
||||
}
|
||||
}
|
||||
],
|
||||
"pullRequests": []
|
||||
},
|
||||
{
|
||||
"sha": "6b180783ed04ebc87548c8385c80e05725804c47",
|
||||
"subject": "fix(tests): budget CLI coverage from parsed options",
|
||||
"body": "fix(tests): budget CLI coverage from parsed options\n\nUse the pinned Vitest parser for coverage and shard flags. Keep ordinary CLI workers at 384 MiB and covered workers at the tested 512 MiB with unchanged concurrency. Retain complete coverage and add actual-config regressions for false, provider-only and spaced-shard options.\n\nValidation: 27 focused cases; 16 covered cases with CLI preloaded; 5 docs checks; typecheck/lint; exact discovery of 1578 files. Measured covered control: 22.18s elapsed, 46.52s CPU, about 2.33 GiB sampled summed process-family RSS on macOS arm64. Full Linux CI pending. No runtime/device verification claimed.\n\nFixes #4960\n",
|
||||
"parents": ["8cbaf6c7021b97dd4de6b5771c35fbed06b240e9"],
|
||||
"authors": [
|
||||
{
|
||||
"name": "morluto",
|
||||
"email": "76467478+morluto@users.noreply.github.com",
|
||||
"user": {
|
||||
"login": "morluto",
|
||||
"__typename": "User"
|
||||
}
|
||||
}
|
||||
],
|
||||
"pullRequests": []
|
||||
},
|
||||
{
|
||||
"sha": "bb212ffecb47e129cc569b9867b2b6e7c9b65685",
|
||||
"subject": "fix(tests): inherit server identity deadline policy",
|
||||
"body": "fix(tests): inherit server identity deadline policy\n\nRemove per-case ten-second overrides from MCP catalog and identity checks. Keep the suite-owned bounded ordinary and coverage deadlines, all assertions, and session cleanup.\n\nFixes #4967\n",
|
||||
"parents": ["6b180783ed04ebc87548c8385c80e05725804c47"],
|
||||
"authors": [
|
||||
{
|
||||
"name": "morluto",
|
||||
"email": "76467478+morluto@users.noreply.github.com",
|
||||
"user": {
|
||||
"login": "morluto",
|
||||
"__typename": "User"
|
||||
}
|
||||
}
|
||||
],
|
||||
"pullRequests": []
|
||||
},
|
||||
{
|
||||
"sha": "b83b9caf4a5f0241a5fd370792f1aed1bffbb2cf",
|
||||
"subject": "fix(setup): edit the duplicate JSON member that clients read (#4957)",
|
||||
"body": "fix(setup): edit the duplicate JSON member that clients read (#4957)\n\nJSON.parse, JSONC parse, and native Claude Code keep the last duplicate\nobject member, and parseClientConfiguration therefore validates the last\none. The shared setup/uninstall serializer instead edited text through\njsonc-parser's modify and findNodeAtLocation, which select the first\nmatching member, so setup and uninstall on configurations with repeated\nserver tables or rea members wrote into a shadowed member and failed\nreadback.\n\nLocate every JSON text edit by the effective (last) member in one place,\nreplacing the first-member modify path. Removing an effective member\nthat shadows an earlier duplicate would activate that duplicate, whatever\nits ownership, so the serializer now returns a tagged shadowed_member\nfailure and setup and uninstall refuse before writing a backup or the\nconfiguration.\n\nFixes #4952\n\nCo-authored-by: Claude Opus 5.5 <noreply@anthropic.com>",
|
||||
"parents": ["bb212ffecb47e129cc569b9867b2b6e7c9b65685"],
|
||||
"authors": [
|
||||
{
|
||||
"name": "Kaoru0822",
|
||||
"email": "kanadekitauji@proton.me",
|
||||
"user": {
|
||||
"login": "kaoru0822-kitauji",
|
||||
"__typename": "User"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "Claude Opus 5.5",
|
||||
"email": "noreply@anthropic.com",
|
||||
"user": {
|
||||
"login": "claude",
|
||||
"__typename": "User"
|
||||
}
|
||||
}
|
||||
],
|
||||
"pullRequests": [4957]
|
||||
}
|
||||
],
|
||||
"pullRequests": [
|
||||
{
|
||||
"number": 4957,
|
||||
"title": "fix(setup): edit the duplicate JSON member that clients read",
|
||||
"url": "https://github.com/morluto/rea/pull/4957",
|
||||
"author": {
|
||||
"login": "kaoru0822-kitauji",
|
||||
"__typename": "User"
|
||||
},
|
||||
"mergeCommit": {
|
||||
"oid": "b83b9caf4a5f0241a5fd370792f1aed1bffbb2cf"
|
||||
},
|
||||
"issues": [
|
||||
{
|
||||
"number": 4952,
|
||||
"url": "https://github.com/morluto/rea/issues/4952",
|
||||
"author": {
|
||||
"login": "xu91102",
|
||||
"__typename": "User"
|
||||
},
|
||||
"repository": {
|
||||
"nameWithOwner": "morluto/rea"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
+19
-1
@@ -349,7 +349,10 @@ tag remains in main's ancestry. Keep the released tag unchanged.
|
||||
|
||||
The synchronization must include `.release-please-manifest.json`,
|
||||
`package.json`, both root versions in `package-lock.json`, `CHANGELOG.md`,
|
||||
`server.json`, and the versioned documentation examples. Run
|
||||
`server.json`, both plugin manifests (`.claude-plugin/plugin.json` and
|
||||
`.codex-plugin/plugin.json`), the Codex server declaration (`.codex-plugin/mcp.json`), both
|
||||
marketplaces (`.claude-plugin/marketplace.json` and
|
||||
`.agents/plugins/marketplace.json`), and the versioned documentation examples. Run
|
||||
`npm run docs:generate`, `npm run docs:check`, and the release configuration
|
||||
tests against the combined tree. After the merge, verify that the released
|
||||
tag is an ancestor of main:
|
||||
@@ -383,3 +386,18 @@ tag. Do not dispatch a fresh publish phase to repair an already-created
|
||||
release: Release Please will not create the same release again. Do not move
|
||||
the tag, delete the release, or unpublish npm as a retry. A defective public
|
||||
package requires a reviewed correction and a new version.
|
||||
|
||||
### Plugin artifact metadata
|
||||
|
||||
Package verification selects files with npm, then packs an owned staging directory.
|
||||
The release manifest omits development dependencies and development-only overrides;
|
||||
its generated `npm-shrinkwrap.json` preserves the runtime dependency resolutions
|
||||
and integrity values from `package-lock.json`. Runtime or unresolved overrides
|
||||
fail packaging rather than being dropped. Source metadata is never rewritten.
|
||||
The archive bundles production dependencies because Codex fetches npm plugins
|
||||
without installing dependencies. The producer includes all locked PTY platform
|
||||
archives after checking their integrity, without re-resolving the runtime tree.
|
||||
Optional Frida stays outside the bundle so ordinary npm installation can retain
|
||||
its host-specific installer. This also lets Claude install locked dependencies under its plugin policy while
|
||||
retaining development security overrides. Both plugin marketplaces select the
|
||||
same qualified npm artifact, which the release workflow publishes unchanged.
|
||||
+20
-5
@@ -5,6 +5,12 @@ lane. Start with the commands below; use [real-toolchain lanes](#real-toolchain-
|
||||
for engine prerequisites and [client verification](client-verification.md) for
|
||||
installed-client and agent runs.
|
||||
|
||||
Historical verification records may refer to `experiments/analysis-efficiency`.
|
||||
Those experiment scripts are available in the [archived source tree](https://github.com/morluto/rea/tree/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency);
|
||||
they are not included in current checkouts. The retained [test support](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/README.md)
|
||||
contains deterministic checkers and fixtures. The [allocation report](verification/analysis-efficiency/mcp-integer-allocation.md)
|
||||
and its original receipt JSON remain documentation, checked by the report producer.
|
||||
|
||||
## Developer commands
|
||||
|
||||
Use source feedback while editing, explicit boundary checks for the changed
|
||||
@@ -204,7 +210,11 @@ changing a callback solely to satisfy its declared `void` return type.
|
||||
Ordinary Knip remains. `check`, `check:pr`, and `check:ci:deep` run reachability
|
||||
then typed lint sequentially after cached work. Typed lint builds the project
|
||||
type graph and stays separate from fast feedback. Its default Go settings are
|
||||
`GOMAXPROCS=2` and `GOMEMLIMIT=1024MiB`; caller-selected settings take precedence.
|
||||
`GOMAXPROCS=1` and `GOMEMLIMIT=1024MiB`; caller-selected settings take precedence.
|
||||
One Go worker reuses checker caches across the file queue without changing the
|
||||
file inventory or TypeScript project scope. The Go memory target is soft and
|
||||
does not cap the complete process family's memory; measure the full invocation
|
||||
when enforcing a host budget.
|
||||
|
||||
Routine iterations and rebases need focused regressions and relevant checks.
|
||||
Before handing off a PR, run `npm run check` and generated-document checks when
|
||||
@@ -1164,8 +1174,8 @@ its metadata regressions against the same checksum-pinned engine and public APK.
|
||||
It runs on branch pushes affecting its bridge, fixture or engine inputs, and can
|
||||
also be dispatched manually or called by another workflow. It uses the runner's
|
||||
JDK with two active processors and a 512 MiB heap; it needs no npm dependency
|
||||
installation. This lane checks exact raw/display class selectors and the existing
|
||||
cold-metadata regressions. It does not establish CLI/MCP parity or application
|
||||
installation. This lane checks exact raw/display class selectors, raw member names and field
|
||||
DEX types used by mapping lookups, and the existing cold-metadata regressions. It does not establish CLI/MCP parity or application
|
||||
execution behavior. The broader `verify:android` lane checks CLI/MCP parity;
|
||||
neither lane runs the APK.
|
||||
|
||||
@@ -1211,7 +1221,7 @@ HarmonyOS `.har` libraries are deliberately not suffix-classified because
|
||||
Synthetic producer regressions run independently:
|
||||
|
||||
```sh
|
||||
npm run test:focused -- tests/boundary/process/jadxIntegration.test.ts tests/boundary/process/androidAnalysisMcp.test.ts
|
||||
npm run test:focused -- tests/boundary/process/jadxIntegration.test.ts tests/boundary/process/androidAnalysisMcp.test.ts tests/boundary/process/androidMappingMcp.test.ts
|
||||
```
|
||||
|
||||
## Apktool resource decoding
|
||||
@@ -1374,7 +1384,7 @@ other engine versions and architectures remain unverified; see the
|
||||
|
||||
Native IDA MCP verification uses `"protocol": "native"` with an already-running native plugin and requires `--second-procedure other_function --second-marker unique_second_body_text` to exercise GUI function switching and a return to the first function. These options remain optional for other profiles. The literal marker must appear in the second function's pseudocode and be absent from the first. Native cloud/widget failures are reported without hidden retries.
|
||||
|
||||
The initial Linux x64 IDA Free 9.4 run used native MCP 1.1.2 with a local navigation correction. Windows x64 verification uses **IDA Pro 9.3**, the two-function public fixture, and the shipped Windows `whoami.exe` application. The native plugin was built from upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c`, including the merged navigation fix, with a local loopback-listener change. Production CLI/MCP parity, switching, identity and cleanup passed. Correctness and performance measurements are recorded in the [Windows native verification notes](https://github.com/morluto/rea/blob/main/experiments/analysis-efficiency/ida-native-windows.md). Stock 1.1.2 switching, native macOS, and Windows IDA Free remain unverified. See the [native bridge prerequisites](ida-provider.md#optional-native-ida-mcp-bridge).
|
||||
The initial Linux x64 IDA Free 9.4 run used native MCP 1.1.2 with a local navigation correction. Windows x64 verification uses **IDA Pro 9.3**, the two-function public fixture, and the shipped Windows `whoami.exe` application. The native plugin was built from upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c`, including the merged navigation fix, with a local loopback-listener change. Production CLI/MCP parity, switching, identity and cleanup passed. Correctness and performance measurements are recorded in the [Windows native verification notes](https://github.com/morluto/rea/blob/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency/ida-native-windows.md). Stock 1.1.2 switching, native macOS, and Windows IDA Free remain unverified. See the [native bridge prerequisites](ida-provider.md#optional-native-ida-mcp-bridge).
|
||||
|
||||
## Kuna lane
|
||||
|
||||
@@ -1624,6 +1634,11 @@ results after a failure. Available failed-shard reports remain diagnostic only:
|
||||
failed, cancelled, skipped, or missing selected shards cannot pass the gate or
|
||||
publish successful timing history. This deliberately avoids relying on an
|
||||
import graph for generated files, filesystem inputs, subprocesses, or bridges.
|
||||
The portable report rejects an entirely skipped module inventory and lists
|
||||
fully skipped modules in its summary. A mixed passed/skipped inventory is
|
||||
allowed: Vitest's skip state does not establish whether a skipped module was
|
||||
required on that runner. Review the listed skips against platform and provider
|
||||
prerequisites; the inventory alone does not prove every required case ran.
|
||||
|
||||
Scheduled qualification is admitted hourly at minute 23. A forced full run at
|
||||
03:23 UTC refreshes runner and dependency drift every day. Other hourly runs
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,150 @@
|
||||
# Integer validator allocation after MCP materialization
|
||||
|
||||
Issue #6236; PR #4360. Linux x64, 2026-10-22.
|
||||
|
||||
<!-- BEGIN:allocation-report:provenance -->
|
||||
|
||||
Baseline `main@4a4bbab3c337701febeb6b532ace49041c4876e0`; candidate `461f342c71a72ef2be60912391df2393aa4e2ea3`. This is a pinned comparison.
|
||||
|
||||
In separate inspector/forced-GC diagnostics, retained REA V8 heap decreases by about 9.6 MiB. The three-pair ranges do not overlap in either profile. Inspector perturbation is included. Normal task timing, CPU and peak memory are reported separately; these observations do not establish a general latency or peak-memory reduction.
|
||||
|
||||
<!-- END:allocation-report:provenance -->
|
||||
|
||||
Directory discovery uses recorded advertisements after #4424; this study separates it from completed tasks that materialize canonical handlers. Context is unchanged.
|
||||
|
||||
## Change and compatibility
|
||||
|
||||
Replace 1,247 argument-free `z.number().int()` constructions with `z.int()` across 175 existing source files. Keep current shared schema owners, refinements, bounds, defaults and metadata. AST-token comparison against the baseline, after normalizing this construction and formatting, is equal for every touched source file. No new tools, provider effects, caches or response contracts.
|
||||
|
||||
The locked Zod 4.4.3 chained form retains a `ZodNumber` with a separate `ZodNumberFormat` check; direct construction uses the format itself. Incur 0.6.1 coerces argv strings only for `ZodNumber`. Existing CLI declarations and shared-input exceptions therefore retain the chained form. The actual compiled CLI registers 160 leaf commands; 104 numeric argument/option fields have identical results/errors in 974 installed-Incur parser cases across both runtimes, each field with an accepted probe. Fields are isolated from unrelated required options and no handlers are invoked. This is consumer-boundary coverage, not a real-engine claim.
|
||||
|
||||
Seven dependency-boundary variants match parsed values and complete error issues for 126 cases, including fractions, nonfinite values, strings/null/undefined and unsafe integers; 14 JSON Schema projections match in draft-07/draft-2020-12. Actual SDK advertisements match byte-for-byte in both profiles before/after materialization and across roles.
|
||||
|
||||
## Tasks, artifacts and bounds
|
||||
|
||||
Reuse the shared SDK task driver and independent ELF/section/zlib/JSON oracles from [the archived catalog study](https://github.com/morluto/rea/blob/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency/mcp-input-catalog.md): official Alpine v3.21 x86_64 ripgrep 14.1.1-r0 and Zellij 0.40.0-r0. All 42/422 packages and 88/1,063 edges match the independent oracles. Select serde name/version/kind, repeat exact retained-parent reads twice, verify a canonical negative-offset refusal, export/check complete Evidence and confirm the unchanged directory. `kind: null` stays unknown. Targets are not executed; dependency presence does not establish runtime use. Acquisition URLs, archive/executable/oracle hashes and complete answers are in [the receipts](https://github.com/morluto/rea/blob/main/docs/verification/analysis-efficiency/mcp-integer-allocation.json). Previously acquired public bytes are rehashed, not redownloaded.
|
||||
|
||||
Normal cohort: three alternating baseline/candidate pairs for each task × cold/warm server compile cache × default compact/full inputs: 72 trials, plus four excluded full-task warm priming runs. Separate diagnostic cohort: three alternating pairs per profile, cold Zellij tasks, 12 trials. All pass, with identical complete parent IDs/hashes and answers.
|
||||
|
||||
Node 24.18.0/npm 11.16.0; fresh client/server and HOME per trial, driver compile cache disabled. Each trial is serial, pinned to one CPU, 4 GiB hard/3 GiB early stop, no swap, at most 192 tasks. Before/after checks bind clean source, all compiled JS/JSON assets (1,990 baseline / 1,990 candidate files), scripts and manifests; producer and resource-observer hashes are also bound. No OOM, resource stop or leftover payload processes. Whole-family measurements include SDK validation, extra calls, source/compiled audits and oracle checking; kernel peak includes file cache. Build/acquisition/oracle generation and priming are excluded. Warm uses a populated role/profile-specific Node cache; hit rate and OS page cache are unmeasured, and audits warm compiled-file pages.
|
||||
|
||||
## Normal task costs
|
||||
|
||||
Medians, baseline → candidate; all ranges are in the JSON. No inspector or diagnostic GC runs in this cohort.
|
||||
|
||||
<!-- BEGIN:allocation-report:normal -->
|
||||
|
||||
| Task | Cache | Input profile | Wall s | Family CPU s | Family peak MiB |
|
||||
| --------- | ----- | ------------- | --------------: | --------------: | --------------: |
|
||||
| directory | cold | default | 2.739 → 2.239 | 2.553 → 2.075 | 247.4 → 248.4 |
|
||||
| directory | cold | full | 2.940 → 2.437 | 2.682 → 2.217 | 348.0 → 346.8 |
|
||||
| directory | warm | default | 2.639 → 2.135 | 2.461 → 1.980 | 251.8 → 246.2 |
|
||||
| directory | warm | full | 2.840 → 2.338 | 2.666 → 2.144 | 348.6 → 347.2 |
|
||||
| ripgrep | cold | default | 7.571 → 7.169 | 7.372 → 6.918 | 798.8 → 791.1 |
|
||||
| zellij | cold | default | 7.876 → 7.373 | 7.623 → 7.134 | 812.2 → 810.5 |
|
||||
| ripgrep | cold | full | 7.876 → 7.475 | 7.710 → 7.295 | 803.7 → 801.2 |
|
||||
| zellij | cold | full | 8.079 → 7.573 | 7.852 → 7.378 | 815.7 → 819.1 |
|
||||
| ripgrep | warm | default | 7.575 → 6.977 | 7.352 → 6.810 | 815.0 → 791.6 |
|
||||
| zellij | warm | default | 7.774 → 7.178 | 7.581 → 6.981 | 820.9 → 822.0 |
|
||||
| ripgrep | warm | full | 7.976 → 7.875 | 7.746 → 7.679 | 812.7 → 789.3 |
|
||||
| zellij | warm | full | 14.995 → 13.978 | 14.742 → 13.718 | 818.5 → 811.5 |
|
||||
|
||||
<!-- END:allocation-report:normal -->
|
||||
|
||||
### SDK span and audit separation
|
||||
|
||||
<!-- BEGIN:allocation-report:sdk -->
|
||||
|
||||
Cold/default directory audit wall medians are 1.273 s baseline / 0.782 s candidate; SDK spans are 0.944 → 0.948 s. Whole-family costs include those experimental audits. Their cause and per-span CPU are unmeasured.
|
||||
|
||||
SDK span from client connect through task checks and client/server close. Excludes pre/post runtime binding and preflight artifact/oracle loading. Diagnostic spans include inspector and forced GC. CPU is measured for the whole family only; per-span CPU is unmeasured.
|
||||
|
||||
| Task | Cache | Inputs | SDK span s, baseline → candidate |
|
||||
| --------- | ----- | ------- | -------------------------------: |
|
||||
| directory | cold | default | 0.944 → 0.948 |
|
||||
| directory | cold | full | 1.067 → 1.102 |
|
||||
| directory | warm | default | 0.888 → 0.878 |
|
||||
| directory | warm | full | 1.038 → 1.042 |
|
||||
| ripgrep | cold | default | 5.778 → 5.752 |
|
||||
| zellij | cold | default | 5.998 → 5.991 |
|
||||
| ripgrep | cold | full | 6.077 → 6.119 |
|
||||
| zellij | cold | full | 6.228 → 6.209 |
|
||||
| ripgrep | warm | default | 5.743 → 5.656 |
|
||||
| zellij | warm | default | 5.965 → 5.788 |
|
||||
| ripgrep | warm | full | 6.100 → 6.492 |
|
||||
| zellij | warm | full | 11.776 → 11.529 |
|
||||
|
||||
<!-- END:allocation-report:sdk -->
|
||||
|
||||
Full-profile warm Zellij is slower than its cold condition in both roles in this cohort. Cache hit rate and the cause of this observation are unmeasured, and cold/warm groups run sequentially. These timings support neither a general cache speedup nor a causal regression attribution.
|
||||
|
||||
## Separate retained-heap diagnostics
|
||||
|
||||
The same completed SDK task launches only its owned REA process with `--inspect=127.0.0.1:0`. Four CDP calls verify the owned transport PID, read heap, force GC and read heap again. The endpoint comes from that child’s stderr; the PID is a cross-check of the owned endpoint, not sole ownership proof. The inspector socket closes before MCP shutdown. These V8 values include inspector perturbation and exclude the client; diagnostic wall/CPU/family peaks include inspector and forced-GC overhead and are not normal-task timing results.
|
||||
|
||||
<!-- BEGIN:allocation-report:heap -->
|
||||
|
||||
| Inputs | REA V8 retained MiB | Baseline range | Candidate range | Diagnostic wall s | CPU s | Family peak MiB |
|
||||
| ------- | ------------------: | --------------: | --------------: | ----------------: | --------------: | --------------: |
|
||||
| default | 189.835 → 180.242 | 189.823–189.855 | 180.208–180.366 | 15.490 → 14.291 | 15.246 → 14.123 | 814.1 → 786.3 |
|
||||
| full | 191.260 → 181.693 | 191.165–191.304 | 181.588–181.848 | 8.979 → 8.377 | 8.769 → 8.142 | 810.3 → 824.9 |
|
||||
|
||||
<!-- END:allocation-report:heap -->
|
||||
|
||||
A mismatched PID is rejected; removing only that guard makes the negative oracle fail. Both connections in that check use the same owned REA endpoint, with no foreign-process attachment. Aggregation rejects diagnostic GC in a normal trial. Historical #4424 receipt reaggregation preserves previous metrics and adds the separately reported SDK-span fields; its aggregator source hash changes.
|
||||
|
||||
## Context accounting
|
||||
|
||||
<!-- BEGIN:allocation-report:context -->
|
||||
|
||||
tiktoken==0.14.0/cl100k_base; estimates, not model usage. Actual model usage is unmeasured.
|
||||
|
||||
| Profile | Input directory once | Directory task | ripgrep task | Zellij task |
|
||||
| ------- | -------------------: | -------------: | -----------: | ----------: |
|
||||
| default | 49,670 | 50,840 | 54,029 | 53,949 |
|
||||
| full | 205,744 | 206,914 | 210,103 | 210,023 |
|
||||
|
||||
Raw SDK directory costs per list: 756,165/912,436 tokens and 2,935,148/3,587,792 bytes for default/full.
|
||||
|
||||
Question, explicit recipe, server instructions, one complete name/description/input-schema directory, every tool argument and text reply, plus prompt directory for discovery. This projection assumes the client retains a single unchanged directory. Raw catalog result costs include every observed tools/list result separately, with output schemas and without JSON-RPC envelopes. SDK initialization/framing, reasoning and actual model usage are unmeasured. The analysis task includes retained-read, rejection, export/oracle and post-materialization directory checks.
|
||||
|
||||
<!-- END:allocation-report:context -->
|
||||
|
||||
## Validation and reproduction
|
||||
|
||||
- Both runtime builds and all six static checks plus architecture guards pass. Before the final main synchronization, 5,972 source/contract/public CLI/MCP tests pass across 323 files; four skip for unavailable/guarded coverage. After synchronization, both builds/static checks repeat successfully, 77 affected producer and CLI/MCP tests pass, and all 974 registered numeric-parameter parser cases match again. Includes real SQLite CLI/MCP record/column selection and JavaScript heap/deadline preservation. A first direct schema-validity invocation lacked generated catalog metadata; the supported focused entry generates it and all 15 cases pass. A later affected-test invocation selected two nonexistent paths; the corrected three-file producer/consumer run is the 77-case result above. Neither setup failure was counted as a passing test, and no production or timeout change was needed.
|
||||
- All 84 measured tasks and four warm priming tasks pass independent oracles, canonical refusal, parent/directory identity and owned cleanup checks. This does not verify external Hopper/Ghidra/IDA, Android devices, all providers or foreign OS behavior. The earlier macOS numbers in #4360 describe a different revision/platform and are not combined with these results.
|
||||
|
||||
The historical cohort commands below use the retired experiment layout. Run them
|
||||
from the matching recorded producer checkout; the [archived source tree](https://github.com/morluto/rea/tree/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency)
|
||||
provides the former files for reference. Current checkouts retain the report and
|
||||
its conformance producer, rather than the full measurement suite. Moving these
|
||||
files does not requalify the historical measurements against current REA.
|
||||
|
||||
Build the pinned baseline/candidate in separate clean worktrees with the locked toolchain. Use the candidate’s producer scripts for both roles. `measurement_inputs` in the JSON records the complete matrix, questions/recipes, runtime bindings and producer hashes; extract it unchanged as the bindings file. Historical receipts require their matching producer revision. One normal task:
|
||||
|
||||
```bash
|
||||
NODE_DISABLE_COMPILE_CACHE=1 node experiments/analysis-efficiency/mcp-input-catalog.mjs \
|
||||
--runtime "$CANDIDATE_RUNTIME" --bindings "$MEASUREMENT_INPUTS" --role candidate \
|
||||
--artifact "$RIPGREP_BINARY" --oracle "$RIPGREP_ORACLE" \
|
||||
--output "$NEW_CASE_DIRECTORY" --profile default --phase analysis --cache cold
|
||||
```
|
||||
|
||||
Run each task through `resource-run.py --observe-payload-exit --` to obtain whole-family counters. For warm runs replace `--cache cold` with `--cache warm --cache-root <role/profile cache>` after one complete priming task. For the separate Zellij diagnostic tasks add `--diagnostic-heap` and use cold cache. Keep their case names/logs aligned with `measurement_inputs`; aggregate both cohorts with:
|
||||
|
||||
```bash
|
||||
python experiments/analysis-efficiency/mcp-startup-costs.py "$RECEIPT_ROOT" \
|
||||
--output mcp-integer-allocation.json
|
||||
```
|
||||
|
||||
The aggregator requires the recorded tiktoken version, exact catalog/parent/runtime/producer/guard identities, clean source, successful payload cleanup and matching matrix coverage. Default driver behavior continues without an inspector; the heap flag is confined to the experiment.
|
||||
|
||||
In a current checkout, regenerate or verify every numerical section directly
|
||||
from the committed JSON:
|
||||
|
||||
```bash
|
||||
node tests/support/analysis-efficiency/mcp-integer-report.mjs
|
||||
node tests/support/analysis-efficiency/mcp-integer-report.mjs --check
|
||||
```
|
||||
|
||||
The documentation conformance gate checks current receipts, rejects stale normal/heap tables, and refuses missing producer markers without writing.
|
||||
@@ -0,0 +1,320 @@
|
||||
# Static Android JNI verification, 2026-10-27
|
||||
|
||||
Issue #6403 asks which bundled native functions could implement original DEX
|
||||
native declarations. The implementation reads immutable APK bytes, preserving
|
||||
declarations, library occurrences, exported/table candidates and selected
|
||||
per-ABI bindings as independently queryable facts. No target, library,
|
||||
`JNI_OnLoad` or `RegisterNatives` code was executed.
|
||||
|
||||
## Revisions and independent oracles
|
||||
|
||||
The full qualification and alternating delivery measurements below use
|
||||
implementation `4a1839de6f091ade19e82557874c358ed34b8e36`, based on main
|
||||
`03f17cf41d3c844f9383d7792d9b1a14330d0bb2`. Subsequent merge
|
||||
`e8a8a50725cd384ae38ce66a94bafdaba1138108` incorporates main
|
||||
`d6f88becd531cabafca1d968b89cf7fc77a80041`; its browser input-presentation and
|
||||
SQLite test changes are separate from JNI analysis. Post-merge confirmation is
|
||||
recorded below. The native parser's built SHA-256 is
|
||||
`3bf0a2c342f1c404f5861b7c913f7f1b931f740a8c808b518474fc8de20821d5`.
|
||||
|
||||
Linux x86-64, Node 24.22.1/npm 11.16.0 and JDK 21.0.12.1 qualified the workflows.
|
||||
JADX MCP 0.7.1 remains the supplied unmodified JAR, SHA-256
|
||||
`6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75`.
|
||||
The REA metadata bridge is version 9. JNI declarations use the JAR's existing
|
||||
dexlib2 reader before decompiler deduplication, with no `load_apk` call.
|
||||
|
||||
The genuine Android NDK r27d package is revision 27.3.13750724, clang/LLD 18.0.4.
|
||||
Its official Linux ZIP is 663,956,036 bytes, publisher SHA-1
|
||||
`22105e410cf29afcf163760cc95522b9fb981121`; observed SHA-256 is
|
||||
`601246087a682d1944e1e16dd85bc6e49560fe8b6d61255be2829178c8ed15d9`.
|
||||
Pins come from the [official repository manifest](https://dl.google.com/android/repository/repository2-3.xml).
|
||||
The package was streamed, hashed and selectively extracted into a private task
|
||||
cache, without installing or changing global tools. The selected compiler,
|
||||
sysroot and four additional official libunwind archives total 319,933,537 bytes.
|
||||
An initial link failure exposed those omitted archives; the corrected fixture
|
||||
build passed in 4.606 seconds, CPU 6.026 seconds, peak family RSS 110.4 MiB.
|
||||
|
||||
[Committed source fixtures and reproduction](https://github.com/morluto/rea/blob/main/tests/fixtures/android/jni-bindings/README.md)
|
||||
build native declarations, javac headers and two libraries for arm64-v8a,
|
||||
armeabi-v7a, x86 and x86_64. Ordinary REL/RELA, RELR and Android packed APS2
|
||||
variants include overloads, BMP/supplementary names, explicit Modified UTF-8,
|
||||
`.data` and `.data.rel.ro` tables, repeated exports in two libraries, an
|
||||
unmatched declaration and two classes sharing one registered pair. Source,
|
||||
`javac -h`, independent DEX flag/index reads, readelf and a separate byte/relocation
|
||||
oracle establish expected facts. Oracle SHA-256 is
|
||||
`fa634493bc8559a80eda3b70f01d44550c2f06e176cfc2a7cc7e10794b1eb793`.
|
||||
|
||||
| Static APK | SHA-256 |
|
||||
| -------------------- | ------------------------------------------------------------------ |
|
||||
| Ordinary | `c4eb50ef8bdecc6f56c454294c0c8f198fa51a4174d101dc720a849976546649` |
|
||||
| RELR | `5a2ce56191f633575e87bd239702f988d0d1ad82795a203e13a4f907f8624658` |
|
||||
| APS2 | `d2f0d613bc8f22ed1a6435f13b5ae1f088e904216aa8a04dd70c27c5cb606778` |
|
||||
| Conscrypt derivative | `5985c40d5a6b14b64c202c80335d851adb7e8a987e6154a1488a194253856385` |
|
||||
|
||||
The Conscrypt derivative packages the public Android 2.8.0 AAR's original
|
||||
library bytes with its class files transformed by D8 8.0.40. It is a static
|
||||
test container, not a vendor APK or a claimed launchable application. The AAR
|
||||
SHA-256 is `752044c289bc98e2dcabc00a1f88ca158b3aa3f505b38f124552bf4b2a0902b1`.
|
||||
The original-name DEX is used, not an R8-renamed DEX paired with unchanged JNI
|
||||
strings. D8's missing platform-adapter dependency warning remains a qualification
|
||||
limit; no full application linkage or signature verification is claimed.
|
||||
|
||||
## Executed correctness and Evidence checks
|
||||
|
||||
The built pure parser passed independent checks over 25 real libraries:
|
||||
24 genuine NDK outputs plus Conscrypt arm64. All 33 golden table entries match
|
||||
their strings, slot addresses/file offsets, relocated values, encodings,
|
||||
function addresses and code offsets. Conscrypt yields 367 structural entries;
|
||||
the requested `EVP_PKEY_new_RSA ([B[B[B[B[B[B[B[B)J` candidate is `0xfa11c`.
|
||||
Its name/signature/function slots are `0x2ada68`, `0x2ada70`, `0x2ada78`,
|
||||
with relocated values `0x749a8`, `0x7affa`, `0xfa11c`. This built-artifact
|
||||
qualification took 0.684 seconds, CPU 0.736 seconds, peak family RSS 99.5 MiB.
|
||||
|
||||
Real SDK/stdio MCP and CLI checks independently validate:
|
||||
|
||||
- All 12 source native declarations across four ABIs, exact library digests,
|
||||
overloads, UTF-16 names and per-library alternatives. Ordinary/RELR return
|
||||
32 matched, 12 ambiguous and four unknown rows. The source-known unmatched
|
||||
method remains unknown because symbolic writes/NOBITS leave scanning partial.
|
||||
- APS2 preserves 20 observed export matches, four ambiguous export rows and
|
||||
24 unknown rows. It explicitly reports unsupported table relocation encoding,
|
||||
retaining each selected library's reason even without a positive candidate.
|
||||
- Class selection never hides the other class sharing a registered pair.
|
||||
All selected native declarations survive independently of available ABI rows;
|
||||
the no-library selection case is covered by a domain regression, not claimed
|
||||
as a separate real SDK artifact qualification.
|
||||
- All four retained collections support exact fields, empty filters and batch
|
||||
ID retrieval. Parent-array `row_index` remains separate from raw DEX/ZIP
|
||||
source ordinals. Advertised converted schemas validate with the real SDK/Ajv.
|
||||
- Complete portable parents preserve the same normalized facts through summary
|
||||
and CLI delivery. Ordinary and each Conscrypt cohort copy the source to an
|
||||
exclusively created owned file, inspect it, remove only that copy, then read
|
||||
retained summary, exact table slots, binding fields and a full export.
|
||||
Original fixture bytes remain unchanged. Post-removal success is observed;
|
||||
hidden internal engine activity is not directly instrumented by this harness.
|
||||
|
||||
The first three four-ABI cohorts execute 107 SDK calls and three cold CLI parity
|
||||
checks. Each Conscrypt cohort adds 28 investigation calls, five separately
|
||||
accounted retained-removal calls and one CLI parity check. Four alternating
|
||||
Conscrypt cohorts preserve the same normalized fact digest:
|
||||
`7e4738d2dd9e518277d5268ce5c334d9d25728198f66b2102cdc3411865f2d46`.
|
||||
|
||||
Raw parent transcripts show one native inventory RPC on the first investigation,
|
||||
then only JVM-status RPCs on later main queries; normalized native facts remain
|
||||
present. No JNI parent contains `load_apk`. Library reuse is owned by the same
|
||||
immutable APK session; ABI/library selection replaces one scan, while class
|
||||
and method selection reuses it. Fresh analysis admission still hashes current
|
||||
caller APK bytes. Historical views never reopen them.
|
||||
|
||||
Type/lint/build and 396 tests across ten exact files pass together: native/domain
|
||||
projection, producer parsing, real ZIP occurrence/cleanup boundaries, unchanged
|
||||
JADX session/idle/cancellation behavior, public contracts and existing IL2CPP
|
||||
decoding/ownership. The gate took 215.198 seconds, CPU 257.470 seconds, peak
|
||||
family RSS 4396.1 MiB. A full real JDK/JADX metadata run additionally passed
|
||||
native reads before any APK load, malformed/duplicate DEX and CRC negatives,
|
||||
raw aliases, constants, absence facts, manifest identity and downstream CLI/MCP
|
||||
bodies/direct-invoke parity from both DEX entries: 154.890 seconds, CPU 233.275
|
||||
seconds, peak family RSS 984.8 MiB.
|
||||
|
||||
Finalization regressions retain already parsed library facts/digests in typed
|
||||
partial observations before ZIP close. Caller/deadline-first cancellation,
|
||||
failed close, blocked admission and retry preserve their first cause and exact
|
||||
resource lease. Interrupted later libraries preserve earlier completed facts.
|
||||
Stable table candidate identity uses physical pointer-slot address/file offset,
|
||||
not inferred run grouping. Known ZIP size claims stay separate from verified
|
||||
read size/hash.
|
||||
|
||||
## Completed-task costs
|
||||
|
||||
Earlier main cannot decode/register JNI tables with equivalent coverage, so
|
||||
manual export lookup is not an equivalent before workflow. The controlled
|
||||
comparison is complete versus summary delivery of the same new investigation,
|
||||
same bytes, query, selected pages and proof export. Conscrypt trials alternate
|
||||
summary/complete/summary/complete, each with one cold and one warm task.
|
||||
|
||||
Each fixture task has four investigations, 12 selected collection pages and a
|
||||
proof export (17 calls). Each Conscrypt task has one investigation, the same
|
||||
12 page checks and proof export (14 calls). Task time includes public calls and
|
||||
local oracle checks; one-time startup/catalog, separate retained-removal
|
||||
qualification, final cleanup and a cold CLI parity run are included in the
|
||||
whole-process columns, not hidden in task timings. Ordinary includes removal
|
||||
qualification; RELR/APS2 do not, so their whole-process costs are not a paired
|
||||
performance comparison. All four Conscrypt cohorts use the same phase flags.
|
||||
|
||||
| Cohort | Cold task s | Warm task s | Whole s | CPU s | Peak family MiB | Warm text tokens | Warm SDK tokens |
|
||||
| -------------------- | ----------: | ----------: | ------: | -----: | --------------: | ---------------: | --------------: |
|
||||
| Ordinary | 11.058 | 0.606 | 32.658 | 45.124 | 1215.7 | 58,319 | 401,681 |
|
||||
| RELR | 12.140 | 0.797 | 29.122 | 38.375 | 1266.2 | 58,231 | 401,502 |
|
||||
| APS2 | 12.578 | 0.917 | 30.892 | 41.071 | 1268.0 | 50,900 | 386,634 |
|
||||
| Conscrypt summary | 11.237 | 0.444 | 33.999 | 46.953 | 1282.0 | 32,265 | 349,128 |
|
||||
| Conscrypt complete | 10.846 | 0.456 | 32.907 | 45.948 | 1398.3 | 224,954 | 737,723 |
|
||||
| Conscrypt summary 2 | 19.205 | 1.200 | 57.545 | 77.644 | 1456.2 | 32,223 | 349,055 |
|
||||
| Conscrypt complete 2 | 20.978 | 0.925 | 64.513 | 81.218 | 1502.0 | 224,526 | 736,950 |
|
||||
|
||||
Warm text estimates are about 85.7% lower for summary delivery. Cold text
|
||||
estimates are 32,087/32,202 versus 247,593/247,912. SDK-inclusive warm estimates
|
||||
are about 52.6% lower. Latency/CPU/peak-memory samples vary substantially and do
|
||||
not establish a general improvement. Cold costs include the first generic
|
||||
view-schema compilation in the client; subsequent tasks reuse it. Startup/
|
||||
catalog alone ranges 2.692–4.955 seconds across these Conscrypt cohorts.
|
||||
|
||||
Token estimates use tiktoken 0.14.0/o200k_base. Every task counts its analyst
|
||||
question, discovered input catalog once, arguments and all text replies,
|
||||
including pages and export receipts. SDK totals additionally count output
|
||||
schemas and structured-content duplication. Qualifications and CLI response
|
||||
costs are reported separately by the cost script. Exported file contents are
|
||||
verified locally and are not counted as model context unless actually returned
|
||||
inline. Actual billing, hidden prompts, client rendering/pruning and OS cache
|
||||
state are unmeasured.
|
||||
|
||||
### Latest-main confirmation
|
||||
|
||||
After merging main `d6f88becd` into `e8a8a5072`, type/lint/build and the eight
|
||||
input-presentation/public-contract cases pass. The first direct schema-suite
|
||||
invocation could not load its generated test catalog; `test:focused` supplied
|
||||
the required artifacts and all 16 schema tests then passed (63.310 seconds,
|
||||
CPU 72.830 seconds, peak family RSS 1943.5 MiB). No schema assertion was relaxed.
|
||||
|
||||
Fresh full Conscrypt SDK/CLI cohorts use the same proof, 12 pages and owned-copy
|
||||
removal flags. Both retain the identical fact digest above:
|
||||
|
||||
| Delivery | Cold task s | Warm task s | Whole s | CPU s | Peak family MiB | Warm text tokens | Warm SDK tokens |
|
||||
| -------- | ----------: | ----------: | ------: | -----: | --------------: | ---------------: | --------------: |
|
||||
| Summary | 11.739 | 0.680 | 36.712 | 51.209 | 1395.1 | 31,932 | 348,522 |
|
||||
| Complete | 11.161 | 0.459 | 34.692 | 48.319 | 1385.2 | 224,954 | 737,723 |
|
||||
|
||||
Summary text estimates remain about 85.8% lower. This pair does not show a
|
||||
latency or peak-memory improvement. Each adds 33 SDK calls and one cold CLI
|
||||
parity check, bringing the executed public qualification to 305 SDK calls and
|
||||
nine CLI checks across the pinned cohorts.
|
||||
|
||||
`docs:check`, `format:check` and `docs:build` pass together: 44.365 seconds,
|
||||
CPU 56.935 seconds, peak sampled family RSS 1300.6 MiB. Generated catalogs,
|
||||
schemas, packaged skills and provider binaries remain uncommitted outputs.
|
||||
|
||||
All expensive checks use CPU affinity 0,1, bounded heaps and one active process
|
||||
family, with an 8 GiB RSS guard and 4 GiB host-availability guard. Peak memory is
|
||||
the sum of family RSS sampled every 100 ms: shared pages can be double-counted
|
||||
and short peaks can be missed. No guard stop or OOM occurred.
|
||||
|
||||
### Default-summary delivery confirmation
|
||||
|
||||
The new MCP operation now defaults to retained `summary` when `detail` is omitted.
|
||||
Explicit `complete` remains supported; the CLI retains its common explicit-detail
|
||||
policy. The actual SDK regression verifies both modes, caller selectors and
|
||||
reading an unselected declaration from the complete parent after source removal.
|
||||
Type/lint/build and all five cases in that MCP boundary file pass: 80.012 seconds,
|
||||
CPU 110.240 seconds, peak sampled family RSS 4747.6 MiB. An earlier attempt stopped
|
||||
at the test-function length lint rule; the fixture helper was extracted before
|
||||
this successful gate.
|
||||
|
||||
A fresh genuine Conscrypt workflow omits `detail` on JNI investigation calls and
|
||||
passes 33 SDK calls, all 12 retained pages, proof export, one cold CLI parity
|
||||
check and the separate owned-copy removal qualification. The same investigation
|
||||
fact digest above is retained. Cold/warm task times are 15.608/0.481 seconds;
|
||||
whole process-family time is 45.239 seconds, CPU 61.038 seconds, peak sampled RSS
|
||||
1431.2 MiB. Warm text/SDK token estimates are 31,928/348,516 using the same cost
|
||||
method. This adds a tenth CLI check and brings the successful public qualification
|
||||
total to 338 SDK calls. Explicit-complete removal qualification remains separate
|
||||
from the investigation task; default summaries do not discard the full parent.
|
||||
|
||||
Final `docs:check`, `format:check` and `docs:build` also pass: 77.269 seconds,
|
||||
CPU 70.506 seconds, peak sampled family RSS 1191.1 MiB.
|
||||
|
||||
The updated harness adds `--omit-detail` (with `--detail summary`), SHA-256
|
||||
`c2a5d090f32429c852e3c974083bb9c21a0dc6a608407687031f155a722b53d7`.
|
||||
The earlier cohort tables use the original frozen harness below. Default-run
|
||||
summary/catalog/call receipt SHA-256 values are respectively
|
||||
`c10e4d702b9694b0ccc69926a4e1f688fa8dea33a3de2e2837ea15190a706b37`,
|
||||
`0c36244f5ff788fabd5491144b0711e99ad211ce3c09e1812d1d4da5753624ac`, and
|
||||
`e1f021f0a7a92f101a037a1467a3428b4c620482eb556d81501a5ea64226ab82`.
|
||||
|
||||
### Review corrections and final qualification
|
||||
|
||||
Four review findings were reproduced or confirmed at their producing boundaries:
|
||||
|
||||
- The shared relocation reader had charged every file-backed write against a
|
||||
sparse retention limit. Its single immutable decoder now streams writes;
|
||||
IL2CPP applies them to its existing private copy and retains only bounded
|
||||
unresolved/zero-fill state and a file-offset bitmap. JNI keeps its bounded
|
||||
sparse provenance index. The two actual IL2CPP registration-read regressions
|
||||
each include 65,537 extra file-backed relative writes (AArch64 and x86-64)
|
||||
and verify the original bytes remain unchanged.
|
||||
- GNU version headers could replay shared auxiliary chains and names. The shared
|
||||
export owner now bounds total record visits from section bytes and the existing
|
||||
export-row allowance, caches decoded/failed string offsets within 16 MiB, and
|
||||
yields checkpoints through the same generator used by synchronous callers.
|
||||
Refusal retains exports with partial coverage and unknown version names.
|
||||
Both GNU table variants exercise 10,000 headers replaying 65,535 auxiliary
|
||||
records and a 65,535-byte name, verifying one decode and cooperative yields.
|
||||
- A named mapped-string read-limit failure now distinguishes budget exhaustion
|
||||
from invalid text. JNI preserves later candidates, marks tables partial and
|
||||
projects absent candidates as unknown. Both long names and descriptors are
|
||||
covered. An additional actual JNI-consumer regression queues cancellation
|
||||
after GNU name decoding begins and stops before export-symbol decoding.
|
||||
- The static verifier now requires the Conscrypt oracle only when expectations
|
||||
advertise it. Executed controls pass with 25 libraries/33 entries when present,
|
||||
and 24 libraries/32 entries for NDK-only expectations; an advertised missing
|
||||
oracle rejects with ENOENT and writes no success receipt.
|
||||
|
||||
Type/lint/build and 330 cases in 13 related domain, parser, archive, IL2CPP and
|
||||
actual CLI/stdio MCP files pass together: 192.437 seconds, CPU 224.376 seconds,
|
||||
peak sampled family RSS 4190.2 MiB. After adding the during-version cancellation
|
||||
case, type/lint and all 38 JNI parser cases pass: 39.842 seconds, CPU 67.581
|
||||
seconds, peak RSS 3928.0 MiB. These overlap earlier gates; counts are not summed
|
||||
as unique tests.
|
||||
|
||||
The final built parser again passes the independent 25-library/33-entry oracle:
|
||||
0.975 seconds, CPU 1.009 seconds, peak RSS 103.9 MiB. Built parser SHA-256 is
|
||||
`7c0638751427b2b282b8e55bb21c95f6c3f05c246b0ea66a31d0280694f9ffeb`;
|
||||
its receipt also fingerprints every shared parsing dependency. The fixed oracle
|
||||
SHA-256 above is unchanged. Large-capacity IL2CPP verification uses valid producer
|
||||
fixtures, not a newly qualified real IL2CPP application.
|
||||
|
||||
A fresh Conscrypt investigation qualification again passes 33 SDK calls,
|
||||
12 pages, proof, a cold CLI check and owned-copy removal. Investigation facts
|
||||
retain the same digest above. Cold/warm task times are 23.529/1.015 seconds;
|
||||
whole time 66.284 seconds, CPU 77.940 seconds, peak RSS 1402.6 MiB. Warm text/SDK
|
||||
estimates are 31,955/348,569. Host/time variation does not establish a general
|
||||
latency improvement. Cumulative successful public qualification is now 371 SDK
|
||||
calls and eleven CLI parity checks, across the explicitly separated revisions.
|
||||
The final cohort summary/calls SHA-256 values are
|
||||
`84df4f4ed97f6d1a931010db8e16420e5f06df371940ed9225eb99a7a7c01094` and
|
||||
`2c22b89e5ec4b26f15f0e9c5f8fdfd4af8583ad0926ad32edb359ecd991cd7fd`;
|
||||
its catalog hash matches the default-summary cohort. No resource guard or OOM
|
||||
occurred. Final `docs:check`, `format:check` and `docs:build` pass in
|
||||
38.602 seconds, CPU 49.448 seconds, peak sampled family RSS 1286.2 MiB.
|
||||
|
||||
## Reproduction and limitations
|
||||
|
||||
Build the runtime, generate the source fixtures using selected existing JDK,
|
||||
NDK and D8 paths, then run the maintained
|
||||
[pure parser verifier](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-jni-library.mjs)
|
||||
against the built parser. The
|
||||
[public workflow harness](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-jni.mjs)
|
||||
accepts `--variant ordinary|relr|aps2|conscrypt`, `--detail complete|summary`,
|
||||
`--proof --pages --cli` and optional `--retained-removal`, explicit selected
|
||||
repository/artifact/JDK/JAR/oracle paths and a fresh output directory. Frozen
|
||||
harness SHA-256:
|
||||
`14836895659de0f161c479b2b013ec39d754d172fd9935898872ab9cb5937035`.
|
||||
The [cost script](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-jni-costs.py)
|
||||
consumes every case's catalog, calls and summary receipts, fingerprints them
|
||||
and keeps qualification costs separate.
|
||||
|
||||
Initial source integration type/lint errors were corrected before qualification.
|
||||
A first CLI verifier attempt incorrectly treated its summary wrapper as flat
|
||||
Evidence; the CLI had written its valid bundle. The corrected harness saves raw
|
||||
stdout/stderr before assertions and validates the unwrapped named contract.
|
||||
One driver used the wrong Conscrypt derivative filename and failed before
|
||||
analysis; corrected cohorts use the pinned file above. A direct schema-suite
|
||||
invocation lacked generated test catalog metadata; artifact-aware verification
|
||||
is recorded separately. These attempts are not counted as successful runs.
|
||||
|
||||
Only Linux/JDK21/JADX0.7.1 and the fixed genuine NDK variants/public Conscrypt
|
||||
arm64 sample are qualified. Cross-ABI ELF fixtures do not establish real
|
||||
Windows/macOS provider lifecycle, device/emulator operation or runtime JNI
|
||||
registration. Existing IL2CPP regressions cover the shared relocation refactor;
|
||||
no new real IL2CPP application qualification is claimed. Runtime-built tables,
|
||||
obfuscated strings, unsupported encodings and missing split/runtime-loaded code
|
||||
remain unknown. Byte digests identify artifacts, not authenticity or build
|
||||
ownership.
|
||||
@@ -0,0 +1,366 @@
|
||||
# Caller-selected Android mapping investigation — 2026-10-27
|
||||
|
||||
## Question, revisions and oracle
|
||||
|
||||
Apply a caller-selected local R8 mapping while preserving observed DEX identities,
|
||||
original mapping-derived names/types, overloads, uncertainty, both artifact
|
||||
identities and complete source Evidence. Inspect the requested inventory and
|
||||
method bodies, then export complete Evidence. APKs and native code are never
|
||||
executed, installed or modified.
|
||||
|
||||
Both controls use main `4e67bc399951768737eeaf539f7bc9fa8373b244`:
|
||||
|
||||
- Before: `3a588662d916f801a8c444dd86f1c65c81c76bf9`, the integrated
|
||||
#7137 mapping implementation, without parsed-index reuse or retained symbol
|
||||
summaries. This is a feature-equivalent optimization control, not a claim
|
||||
that unmodified main already supported mappings.
|
||||
- After: `6b65d3a4a1a8307417365b9a1e9060e607538aa3`, bounded/cached mapping
|
||||
and explicit retained summary delivery. Later documentation changes do not
|
||||
alter the measured implementation. Following main’s experiment archive, the
|
||||
maintained harness/fixtures now live under tests and this report under
|
||||
docs/verification; the original measured scripts remain available at that pinned revision.
|
||||
- The original #7137 author is 555aaditya; integration retains their co-author
|
||||
credit and preserves #7379's shared manifest contracts/default-package tests.
|
||||
|
||||
The [source fixture](https://github.com/morluto/rea/blob/main/tests/fixtures/android/r8-mapping/README.md) has four classes,
|
||||
eight fields and fifteen methods, including constructors, overloads, Unicode
|
||||
names, nested/default-package classes, arrays and renamed types in prototypes.
|
||||
The same actual R8 invocation emits DEX and mapping. Handwritten source
|
||||
expectations are checked against pre-minification `javap -s -p`; an independently
|
||||
compiled dexlib2 reader inventories raw DEX identities without REA's parser,
|
||||
bridge or JADX aliases. All four classes and fifteen selected bodies are checked
|
||||
on each investigation. Seventeen valid method mapping records include three
|
||||
line ranges for one constructor; they are not seventeen distinct DEX methods.
|
||||
|
||||
The public-library task transforms the shipped `classes.jar` from Conscrypt
|
||||
Android 2.8.0 with actual R8. Its 949,641-byte mapping has 530 class, 1,263 field
|
||||
and 10,342 method records. A separate D8 conversion of the original class files
|
||||
provides the unminified descriptor oracle. Every selected `NativeCrypto` field
|
||||
(28) and method (390), plus inner-class identity, is verified; bodies are selected
|
||||
for `EVP_PKEY_new_RSA`, `EVP_DigestSignInit`, `SSL_new`, `SSL_free` and
|
||||
`SSL_get_error`. These are static DEX-only ZIP containers, not runnable APKs.
|
||||
D8 warned that a platform adapter's `com.android.org.conscrypt.OpenSSLSocketImpl`
|
||||
was missing; complete dependency/runtime linkage is unverified. R8 additionally
|
||||
used `-dontwarn **`, with shrinking/optimization disabled and name minification
|
||||
allowed. Stable R8 did not consume an existing APK's DEX.
|
||||
|
||||
## Pinned artifacts
|
||||
|
||||
| Artifact | Observed SHA-256 |
|
||||
| ----------------------------- | ------------------------------------------------------------------ |
|
||||
| R8 8.0.40 Maven JAR | `ab1379835c7d3e5f21f80347c3c81e2f762e0b9b02748ae5232c3afa14adf702` |
|
||||
| JADX MCP 0.7.1 JAR | `6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75` |
|
||||
| Android 36 platform JAR | `d9eb9da824d9e247a352f570f01e1169e725b2954bca9e283a71786c59b59f9a` |
|
||||
| Source fixture input JAR | `88f3d46018ac2effb166ceed79fdd6e8b58b6bb08d2649cd0aa4f90dc260260c` |
|
||||
| Source fixture DEX | `0b4c49cf3d706f5c2c4c15a10ff27dfbabb9b07f463d508f6f348eafb3245909` |
|
||||
| Source fixture static APK | `f8c0849978ed7eceeeb293223cf2139c1749f672148dce1e16d018fa727445c9` |
|
||||
| Source fixture mapping | `5ca162587794af1bd1da00f79ee64fd4ecde6f7c91048ea5596260104df6e483` |
|
||||
| Conscrypt 2.8.0 AAR | `752044c289bc98e2dcabc00a1f88ca158b3aa3f505b38f124552bf4b2a0902b1` |
|
||||
| Conscrypt shipped classes JAR | `2cd9b6bc444cfca69048e8327383b7f5a13fc7978465e6e0fd5111a2000822f6` |
|
||||
| Conscrypt R8 DEX | `cd4d7eb9b05e1a84d064f7f9e4b33d68a8f92a9f58a6a62e4041972964ef6d27` |
|
||||
| Conscrypt static APK | `9a0224400b91b030213e0f6ba63d899a4ad6333f930414c768ce88bbef423e91` |
|
||||
| Conscrypt mapping | `67cd69e706afee50a1cc4e13bd9fd26056932c569d0ea7706dfe6fe8215a2e3c` |
|
||||
|
||||
R8 publisher and independent official Bazel checksum pin are linked from the
|
||||
fixture README. Conscrypt was downloaded from
|
||||
[Maven Central](https://repo.maven.apache.org/maven2/org/conscrypt/conscrypt-android/2.8.0/conscrypt-android-2.8.0.aar)
|
||||
and verified against its publisher SHA-256 sidecar. No generated binary or
|
||||
mapping file is committed. JAR/ZIP timestamps can change a source rebuild's
|
||||
artifact hash; the table identifies these measured bytes.
|
||||
|
||||
## Owning changes and safe bounds
|
||||
|
||||
The provider keeps one successful parsed index. Each call still performs a
|
||||
stable regular-file read and SHA-256 check. Only identical digest/length with
|
||||
fixed parser options reuses parsed facts; current path provenance comes from
|
||||
the new read. Changed content, including same-size/same-mtime rewrites, is
|
||||
reparsed; missing or cancelled input never falls back to stale facts. Idle
|
||||
retirement and close release the index. During replacement, old and new parsed
|
||||
indices can briefly coexist. APK/query results are not cached by this mechanism.
|
||||
|
||||
The domain parser scans lines without a whole-file split, accounts retained
|
||||
strings/containers before admitting records, and admits parameter/metadata
|
||||
expansion before split/JSON parsing. The fixed 64 MiB input, estimated 128 MiB
|
||||
index and 8 MiB per-residual-class claim allowances are resource constraints,
|
||||
not format claims or measured V8 heap caps. Replaced metadata, deduplicated
|
||||
range splits and overwritten inline frames are not cumulatively billed.
|
||||
Ambiguity scans count all alternatives but retain only eight stable candidates;
|
||||
conflicting residual metadata stays ambiguous. A separate combined 8 MiB claim
|
||||
budget covers actual queries including all inner classes; omitted claims become
|
||||
explicit `unknown`, preserving every raw identity and source SHA. It is not a
|
||||
whole-report/wire-byte cap.
|
||||
|
||||
Existing shared retained-Evidence projection now supports class/method summaries
|
||||
through both CLI and MCP. They preserve complete normalized facts and selected
|
||||
bodies, omitting duplicate raw RPC metadata from the inline response. Complete
|
||||
parent Evidence remains exportable. Default/complete delivery is compatible;
|
||||
summary is explicit, and CLI requires a non-overwriting portable output bundle.
|
||||
Whole summary views admit 1 MiB including context: an admitted one-MiB body may
|
||||
therefore require complete detail. Refusal preserves the MCP parent ID and is
|
||||
nonretryable for the same selection; no facts are silently truncated.
|
||||
|
||||
## Completed-task measurements
|
||||
|
||||
Linux x86-64, Node 24.22.1/npm 11.16.0, Temurin JDK 21.0.12.1, actual JADX
|
||||
0.7.1, one analysis worker, 1 GiB JVM heap and two active processors. Harness,
|
||||
server and children were restricted to CPUs 0/1; Node heap was 3 GiB. Expensive
|
||||
checks ran serially under an 8 GiB sampled family-RSS stop and a 4 GiB host-free
|
||||
memory guard. No guard stopped a completed task and no OOM occurred.
|
||||
|
||||
The [public stdio MCP harness](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-mapping.mjs) uses named live tool
|
||||
contracts and Ajv 2020 validation of SDK-converted advertised outputs. All
|
||||
correctness comparisons, hash reads, discovery and cleanup are included in the
|
||||
whole-harness cost. First-provider calls use fresh processes/JVMs; OS file caches
|
||||
were warm, and the host was shared. Conscrypt order was before/after/after/before.
|
||||
Each small task makes 19 inspections and one proof export; each library task
|
||||
makes six inspections and one proof export. Every exported parent is checked
|
||||
against the exact inline normalized facts, source parameters and APK SHA, and
|
||||
must retain raw calls. The complete/summary lanes use the same requested facts,
|
||||
provider, options, source bytes, discovery families (`android,session`) and proof.
|
||||
|
||||
Peak memory is summed process-family RSS sampled every 100 ms, including client,
|
||||
server, JVM and children. Shared pages can be counted twice and short peaks can
|
||||
be missed; this is not PSS, kernel peak accounting or an enforced production
|
||||
heap measurement. CPU is completed-family user+system time.
|
||||
|
||||
| Cohort | First task s | Warm task s | Whole harness s | CPU s | Peak family MiB |
|
||||
| ----------------- | -----------: | -----------: | --------------: | -----: | --------------: |
|
||||
| small-before | 7.342 | 1.611 | 12.060 | 16.680 | 781.7 |
|
||||
| small-after | 7.960 | 1.842 | 12.945 | 17.413 | 791.5 |
|
||||
| conscrypt-before | 8.057 | 1.015, 0.893 | 13.264 | 17.774 | 920.0 |
|
||||
| conscrypt-after | 6.975 | 0.833, 0.817 | 11.743 | 17.242 | 855.0 |
|
||||
| conscrypt-after2 | 6.987 | 0.725, 0.748 | 11.561 | 16.553 | 872.6 |
|
||||
| conscrypt-before2 | 7.046 | 0.917, 0.886 | 11.855 | 17.093 | 949.7 |
|
||||
|
||||
Warm Conscrypt task medians were 0.905 s before and 0.783 s after (about 14%
|
||||
lower across four warm samples per implementation). Cold/whole-task and memory
|
||||
samples vary; no universal startup or memory improvement is established. The
|
||||
small fixture pays extra summary/retention overhead: 12.060 to 12.945 seconds
|
||||
for the whole harness, with CPU 16.680 to 17.413 seconds. Explicit complete
|
||||
delivery remains available.
|
||||
|
||||
An earlier separate **cache-only** cohort used complete responses without proof
|
||||
export, on the same base. Conscrypt warm six-call medians fell from about
|
||||
0.981 to 0.765 s; before ranges were 0.950–0.996 s and after 0.743–0.850 s.
|
||||
Whole harness samples were 12.861–13.876 to 11.641–11.647 s, CPU
|
||||
16.916–18.297 to 16.454–16.893 s, and peak family 910.1–938.1 to
|
||||
853.1–876.8 MiB. Small before/after whole costs were 18.493/16.110 s,
|
||||
17.495/16.589 CPU s and 763.4/776.3 MiB. These samples motivated reuse but
|
||||
are not substituted for the completed proof-export task above.
|
||||
|
||||
## Whole-task token estimates
|
||||
|
||||
[android-mapping-costs.py](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-mapping-costs.py) executed with tiktoken
|
||||
0.14.0/o200k_base. Text projection counts a fixed task question, all discovered
|
||||
name/description/input schemas once per investigation, every RPC argument and
|
||||
all text replies, including proof export. SDK transport additionally counts
|
||||
output schemas and structured-content duplicates. Exported file contents are
|
||||
checked offline rather than injected into model context; export IO/CPU/memory
|
||||
is included above. Prompt replay, host pruning/rendering, hidden prompts,
|
||||
protocol handshake/framing and actual model billing are unmeasured. Estimates
|
||||
are not billed usage. Each table row is one warm completed investigation.
|
||||
|
||||
| Cohort | Estimated text tokens | Estimated SDK transport tokens |
|
||||
| ----------------- | --------------------: | -----------------------------: |
|
||||
| small-before | 43,526 | 167,094 |
|
||||
| small-after | 41,755 | 159,507 |
|
||||
| conscrypt-before | 180,213 | 510,351 |
|
||||
| conscrypt-after | 69,836 | 217,646 |
|
||||
| conscrypt-after2 | 69,908 | 217,789 |
|
||||
| conscrypt-before2 | 177,902 | 505,723 |
|
||||
|
||||
Conscrypt text estimates fall about 61%, from approximately 178–180k to 70k,
|
||||
while preserving all requested facts and complete proof. Small tasks improve
|
||||
only about 4%; summary context has its own cost. The cache-only cohort kept
|
||||
similar token counts (~172–174k library text tokens without proof export),
|
||||
revealing that parse caching alone did not address repeated raw class metadata.
|
||||
Full inventories still cost more context than a direct known-method summary.
|
||||
|
||||
Normalized investigation digests match exactly across all before/after lanes:
|
||||
|
||||
- Source fixture: `ec76d1e086c8561370fc1ed5dfef67bb88b185e8fb57c7e30afe9f1f153634eb`.
|
||||
- Conscrypt: `7ebf70795dfc23bf2b0d0e813837325fa221f15560ca2384fdb06f26d69d78a5`.
|
||||
|
||||
## Post-review ownership and final workflow confirmation
|
||||
|
||||
Greptile found three concrete boundaries subsequently fixed in
|
||||
`160573cb36f478a088b8a5284fa571cd97dc9a86`: translated prototypes could expand
|
||||
before comparison; a warm request without a mapping could bypass a pending
|
||||
mapping descriptor's cleanup; and timeout classification inspected the outer
|
||||
cleanup wrapper. Type matching now compares incrementally within the observed
|
||||
descriptor, retained file cleanup gates shared request admission and idle
|
||||
retirement independently of engine cleanup, and the first interruption is
|
||||
classified before cleanup composition. All 126 related cases pass serially:
|
||||
55 parser, five file-reader, four first-cause/close outcomes, three cleanup,
|
||||
52 session/idle/cleanup and seven MCP mapping cases.
|
||||
|
||||
A separate bounded diagnostic found and corrected source-string retention.
|
||||
On Node 24.22.1/V8 13.6.233.17-node.58, four small selected class/method results
|
||||
from four distinct mappings containing 10 MiB of ignored comments each retained
|
||||
41,944,028 extra external string bytes through the actual immutable Evidence
|
||||
ledger. Clearing it released that backing; detached-string controls retained
|
||||
zero extra external bytes. The parser now owns admitted UTF-16 strings once,
|
||||
sharing section names and copying only retained outermost/deduplicated facts.
|
||||
It preserves lone surrogate code units and never clones the entire index or
|
||||
copies names per lookup. The identical post-fix ledger experiment retained
|
||||
zero extra external bytes; its RSS delta was 151,552 bytes, versus 62,898,176
|
||||
before. These are engine-specific observations, not portable heap guarantees.
|
||||
The unchanged Evidence/ledger modules and changed parser are fingerprinted by
|
||||
[the diagnostic](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-mapping-retention.mjs).
|
||||
Before/after whole diagnostic costs were 2.995/2.582 seconds, CPU 4.976/4.382
|
||||
seconds and peak sampled family RSS 145,903,616/113,287,168 bytes, with explicit
|
||||
256 MiB old-space selection and a 1 GiB family guard.
|
||||
|
||||
Final equivalent Conscrypt investigations compare the integrated unoptimized
|
||||
control `5ecc6ccc662678c00a45bd171f40798634774155` with fixed `160573cb3`, both
|
||||
on main `e99b6c2c3bcc5f4f050b51b432145144d0d47dc0`, using the same artifacts,
|
||||
facts, seven calls including full proof export, and before/after/after/before
|
||||
order. Every normal-fact digest remains
|
||||
`7ebf70795dfc23bf2b0d0e813837325fa221f15560ca2384fdb06f26d69d78a5`.
|
||||
The final source-fixture summary/proof/negative qualification also passes all
|
||||
43 calls from the maintained harness path.
|
||||
|
||||
| Final cohort | First s | Warm s | Whole s | CPU s | Peak family MiB | Warm text estimate |
|
||||
| ----------------- | ------: | -----------: | ------: | -----: | --------------: | -----------------: |
|
||||
| conscrypt-before | 7.187 | 1.027, 1.006 | 12.370 | 17.307 | 917.7 | 177,855 |
|
||||
| conscrypt-after | 7.376 | 0.813, 0.866 | 12.402 | 17.522 | 868.7 | 69,819 |
|
||||
| conscrypt-after2 | 6.878 | 0.707, 0.726 | 11.439 | 16.383 | 883.4 | 69,902 |
|
||||
| conscrypt-before2 | 7.111 | 0.989, 1.186 | 12.566 | 17.479 | 930.1 | 177,719 |
|
||||
|
||||
The final warm ranges were 0.989–1.186 seconds before and 0.707–0.866 after;
|
||||
text estimates remain about 61% lower, ~177.7–177.9k to ~69.8–69.9k. Whole-task
|
||||
and cold samples overlap and are retained alongside the original cohort above;
|
||||
these measurements do not establish a general startup or memory advantage.
|
||||
|
||||
### Follow-up complexity review
|
||||
|
||||
Two additional review findings exposed repeated linear work despite bounded
|
||||
retention. The parsed owner now records one agreed residual alias (or explicit
|
||||
conflict) per original type, while preserving every class record and its count.
|
||||
Method identities have a direct residual-signature index; missing signatures
|
||||
still merge into the first record, including the initial unsigned-to-signed
|
||||
transition. Superseded alias arrays and candidate searches were removed.
|
||||
The representation allowance includes the new index containers/entries.
|
||||
|
||||
The reproducible
|
||||
[complexity diagnostic](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/android-mapping-complexity.mjs)
|
||||
compares the built parser from `160573cb3` with the follow-up parser SHA-256
|
||||
`9c4c24f7a10a17b8ec08e74b2588b9ba87bc30f3e3d16690c3e07c03fbce233b`.
|
||||
The copied before module only rewrites its three unchanged dependency imports
|
||||
to absolute URLs. Each scenario verifies late matches, counts and answer digest;
|
||||
answers match across revisions. Node 24.22.1, two CPUs, 512 MiB old space,
|
||||
8 GiB sampled family guard, one before/after trial per size:
|
||||
|
||||
| Scenario | Revision | Parse ms | Lookup ms | Whole s | CPU s | Peak family MiB |
|
||||
| ----------------------------------------------------- | -------- | -------: | --------: | ------: | ----: | --------------: |
|
||||
| 20k identical aliases, 1k candidates × 255 parameters | before | 212.60 | 5169.61 | 5.635 | 5.690 | 149.3 |
|
||||
| same | after | 207.31 | 22.84 | 0.449 | 0.559 | 117.1 |
|
||||
| 40k distinct signatures of one original method | before | 8032.93 | 5.70 | 8.300 | 8.371 | 117.5 |
|
||||
| same | after | 321.58 | 5.43 | 0.562 | 0.532 | 117.8 |
|
||||
|
||||
The smaller alias cohort (1k aliases, otherwise identical) looked up in
|
||||
289.18/24.64 ms before/after. The smaller signature cohort (10k) parsed in
|
||||
785.65/89.68 ms. These synthetic cases establish removal of the reported
|
||||
growth, not universal latency or memory gains; they involve no agent tokens.
|
||||
All 60 parser, two report and seven public MCP mapping cases pass serially,
|
||||
along with type/lint/build (79.084 s, CPU 110.273 s, peak family 4393.9 MiB).
|
||||
|
||||
The final indexed implementation also passes the real source fixture's
|
||||
43 summary/proof/negative calls (13.838 s, CPU 18.212 s, peak 797.5 MiB),
|
||||
and the Conscrypt 21-call investigation including proof export. The latter's
|
||||
normal-fact digest remains the value above: first task 7.670 s, warm tasks
|
||||
0.860/0.981 s, whole 12.706 s, CPU 17.870 s, peak 868.8 MiB. Warm whole-task
|
||||
text estimates are 69,940, SDK-inclusive 217,975; tokenizer/scope remain the
|
||||
same. Complete class/method facts and provenance remain accessible.
|
||||
|
||||
For CI head `160573cb3`, all failed/cancelled shard logs and source-test logs
|
||||
were audited. Its executed merge `ee2398458bd8c56666ecaf120c1c3c21f350eea4`
|
||||
uses main parent `0676ad254c206871e0f4b120ba300cbbf9383b62`. The 90 observed
|
||||
failed test identities were selected on a pristine worktree at that exact
|
||||
parent: 88 reproduced in the primary run (128.320 s, CPU 147.775 s, peak
|
||||
2593.3 MiB), then the two newline-bearing ZCode titles reproduced in an exact
|
||||
follow-up (2.388 s, CPU 2.788 s, peak 416.9 MiB). No feature code was present
|
||||
in either control. These are baseline failures; cancelled cases without a
|
||||
completed result remain unverified by CI, with local verification above.
|
||||
|
||||
## Executed verification and limits
|
||||
|
||||
- Parser/reader/protocol/provider mapping regressions cover raw names, exact
|
||||
descriptors/types, overloads, inline outermost frames, malformed/partial
|
||||
records, stable candidate samples, preallocation refusal, repeated metadata,
|
||||
cache freshness/path aliases/deletion, cancellation and descriptor ownership.
|
||||
- Real R8 source and public-library stdio MCP tasks verify all selected symbols
|
||||
and advertised schemas. Actual CLI complete and summary inspections verify
|
||||
raw/mapped identity, exact normalized parity and complete portable parents.
|
||||
- Actual Java metadata regression and downstream CLI/MCP body/direct-invoke
|
||||
parity pass, including reserved raw names, frozen aliases, default packages,
|
||||
encoded constants, native absence and classes.dex/classes2.dex identity.
|
||||
- Summary regressions preserve partial/unknown facts and optional historical
|
||||
omissions, validate SDK output contracts, read retained data after APK/mapping
|
||||
deletion, reject missing portable CLI output and exercise the one-MiB boundary.
|
||||
- Type/lint/build, named schema/profile checks and relevant source/boundary checks
|
||||
are executed separately. Final PR validation records the current command set.
|
||||
|
||||
The final combined 12-file Android run passed 219 of 220 cases, including all
|
||||
52 session/idle/cleanup integration cases. One existing package-summary SDK
|
||||
case hit its 30-second limit under combined load. The unchanged four-case file
|
||||
then passed with one worker: its SDK case took 11.479 seconds; whole isolated
|
||||
harness 31.867 seconds, CPU 35.064 seconds and peak family 1,551,081,472 bytes.
|
||||
No assertion or timeout was relaxed. The combined timeout remains a recorded
|
||||
resource/concurrency limitation rather than being reported as an all-green run.
|
||||
|
||||
Earlier failed attempts are retained rather than counted as passes: a resolver
|
||||
refactor used `only.mapped` after `only` already held the mapped fact; its
|
||||
regressions failed and were fixed. Tagged-error migration exposed one stale
|
||||
string-only test. New SDK schema test typing and a redundant JSON preadmission
|
||||
were corrected at their owning boundaries. A first summary MCP case timed out
|
||||
while enumerating unrelated full catalog contracts; selecting the actual
|
||||
Android/application/session families preserves every assertion and passes
|
||||
without raising timeouts. An initial library harness requested absent
|
||||
`SSL_shutdown` and skipped it; the harness now rejects absent requests, and
|
||||
reported final cohorts explicitly verify all five requested methods.
|
||||
|
||||
The mapping cancellation regression distinguishes caller-first from deadline-
|
||||
first finalization; the original implementation relabelled caller cancellation
|
||||
as timeout (one failing caller-first case and one passing deadline-first case
|
||||
on the unchanged optimization control). An initial test's public deadline code was corrected to the actual
|
||||
`provider_timeout` contract. Fixed code preserves the first cause and cleanup.
|
||||
|
||||
Only Linux x86-64/JDK21/JADX0.7.1 and R8 8.0.40 mapping-format 2.1 were qualified.
|
||||
Residual-signature conflicts use derived negative mappings, not a real mapping-
|
||||
format 2.2 producer. No Windows/macOS, emulator/device, app execution, full APK
|
||||
rebuild/linkage, source/line recovery, APK signature verification or model-usage
|
||||
trial is claimed. Mapping/build association remains explicitly caller-selected.
|
||||
|
||||
## Reproduction
|
||||
|
||||
Generate source/DEX/oracle using the fixture README with selected existing tools.
|
||||
Keep separate before/after worktrees, pin their revisions above, build both and
|
||||
use identical target/mapping/oracle bytes. Then run the same harness for each:
|
||||
|
||||
```sh
|
||||
node tests/support/analysis-efficiency/android-mapping.mjs \
|
||||
--repository "$SELECTED_REPOSITORY" --target "$STATIC_APK" \
|
||||
--mapping "$MAPPING_FILE" --output "$FRESH_RESULTS" \
|
||||
--java-home "$SELECTED_JDK" --jar "$JADX_MCP_JAR" \
|
||||
--oracle "$RAW_DEX_ORACLE" --expectations "$ORIGINAL_EXPECTATIONS" \
|
||||
--proof
|
||||
```
|
||||
|
||||
Add `--detail summary` only on the after implementation. The small fixture
|
||||
selects all classes/bodies. For Conscrypt add `--class-name U0 --repeats 3` and
|
||||
`--methods EVP_PKEY_new_RSA,EVP_DigestSignInit,SSL_new,SSL_free,SSL_get_error`;
|
||||
`U0` is the measured artifact's raw class identity, not a universal residual name.
|
||||
Use the original NativeCrypto D8 descriptor inventory for that expectation file.
|
||||
Run `--variants` separately on the source fixture to exercise derived malformed,
|
||||
partial, duplicate and conflicting mappings; those extra qualification calls
|
||||
are not part of the timing/token table's repeated task.
|
||||
|
||||
```sh
|
||||
python tests/support/analysis-efficiency/android-mapping-costs.py \
|
||||
--output "$TOKEN_RESULTS" "$BEFORE_RESULTS" "$AFTER_RESULTS"
|
||||
```
|
||||
|
||||
Capture elapsed/CPU/family RSS with the bounded host runner or an equivalent
|
||||
process-family sampler; alternate trials and keep each label's raw receipts and
|
||||
failed attempts. Complete Evidence stays in the exported per-pass bundles.
|
||||
File diff suppressed because it is too large.
Load diff
@@ -4,10 +4,14 @@ Issue [#4772](https://github.com/morluto/rea/issues/4772). Qualified runtime
|
||||
`b49ee40bf330b7bf1c5180b14dcc104c1c613a8e`, tree `0e5f8fe2feae1796c340c07c26e0e28e5d4328d8`,
|
||||
integrates main `c8ed32853881b08aa83459d4da55a28a4e09b0c8`. Recorded at
|
||||
`2026-10-22T08:30:58.269Z`. Subsequent verification-only
|
||||
commits change these reports. The [JSON record](javascript-route-inventory-2026-10-21.json)
|
||||
commits updated these reports. The [archived JSON record](https://github.com/morluto/rea/blob/7d0efef62d56d0a10134bea62d709302daa1565f/docs/verification/javascript-route-inventory-2026-10-21.json)
|
||||
pins original inputs, exact commands, profiles, failed controls and historical revisions.
|
||||
Current-head GitHub CI and automated review must be checked separately.
|
||||
|
||||
The full JSON archive is 2,681,223 bytes, with SHA-256
|
||||
`9d7ea2582a6f33ff0741578aec9f01b9327b3256ce50c582437164868e544d51`. It remains available in Git
|
||||
at the linked immutable revision; this page retains the historical summary.
|
||||
|
||||
The latest main contracts, selected environment and ASAR admission remain shared
|
||||
by CLI and MCP. The staged-code hook runs lint fixes before formatting, resolving the
|
||||
producing order behind the earlier CI format failure.
|
||||
|
||||
Reference in new issue
Block a user