OpenSource/reverse-engineering-anything
1
0 Fork 0
mirror of https://github.com/morluto/rea.git synced 2026-10-26 17:35:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: return complete tool results inline

Browse Source
This commit is contained in:
morluto committed 2026-09-29 07:08:46 +08:00
1 parent 2160d4fae5
commit 0d96275b68
535 files changed
+4084 -13537
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
-1
View File
@@ -125,7 +125,6 @@ Pre-commit hooks via Husky format then lint staged source files. Pre-push runs `
- `REA_V8_INSPECTOR_ENDPOINTS_JSON` (optional): approved literal-loopback Node/Electron Inspector HTTP endpoints.
- `REA_V8_INSPECTOR_FILE_ROOTS_JSON` (optional): canonical filesystem roots approved for Inspector targets and script locations.
- `REA_V8_INSPECTOR_ALLOWED_ORIGINS_JSON` (optional): exact HTTP(S) origins approved for Electron renderer Inspector targets and scripts.
- `REA_INVESTIGATION_INPUT_ROOTS_JSON` (optional): canonical filesystem roots approved for static JavaScript/Electron application analysis and other investigation inputs.
- `REA_JAVASCRIPT_REPLAY_ENABLED` (optional, default `false`): add controlled extracted-module execution authority to the administrator ceiling.
- `REA_JAVASCRIPT_REPLAY_ROOTS_JSON` (optional): exact canonical source roots approved for controlled replay.
- `REA_JAVASCRIPT_REPLAY_NODE_PATH`, `REA_JAVASCRIPT_REPLAY_BWRAP_PATH`, `REA_JAVASCRIPT_REPLAY_SYSTEMD_RUN_PATH`, `REA_JAVASCRIPT_REPLAY_SYSTEMCTL_PATH`, and `REA_JAVASCRIPT_REPLAY_SHELL_PATH` (optional): absolute paths committed by each replay plan; defaults target the current Node runtime and standard Linux system executables.
README.md
+16 -25
View File
@@ -30,7 +30,7 @@
See a feature in an app that you want in your own product? Give the app to your agent—even without its source code. With REA, the agent can investigate the feature, explain how it works, show its evidence, and build a version adapted to your stack and requirements.
REA gives agents one consistent way to investigate software. Today that includes deep native analysis and function dossiers through Hopper or bring-your-own Ghidra on Linux, plus an experimental Windows x64 Ghidra P0 for approved native PE applications; execution-free managed PE/CLI triage; reproducible Evidence v2 records; controlled process capture; passive website, Electron page, and Node/Electron V8 Inspector observation; bounded JavaScript/source-map reconstruction; and a versioned domain graph for connecting JavaScript application layers without confusing static inference with runtime observation. The longer-term toolkit extends the same agent workflow to APIs, protocols, mobile artifacts, firmware, richer runtime behavior, and differences between versions.
REA gives agents one consistent way to investigate software. Today that includes deep native analysis and function dossiers through Hopper or bring-your-own Ghidra on Linux, plus an experimental Windows x64 Ghidra P0 for approved native PE applications; execution-free managed PE/CLI triage; reproducible Evidence records; controlled process capture; passive website, Electron page, and Node/Electron V8 Inspector observation; bounded JavaScript/source-map reconstruction; and a versioned domain graph for connecting JavaScript application layers without confusing static inference with runtime observation. The longer-term toolkit extends the same agent workflow to APIs, protocols, mobile artifacts, firmware, richer runtime behavior, and differences between versions.
Reverse engineering normally makes the operator choose a tool, learn its API, move evidence between programs, and decide what to inspect next. REA gives that work to the agent through commands, skills, structured results, and repeatable investigation workflows.
@@ -265,7 +265,7 @@ Uninstall preserves Hopper, Node.js, evidence, captures, external evidence roots
| ---------------------------------------------------------------- | ------------------------------------------------------------------------- |
| Ask an agent to investigate an app and build a feature | Install the skill, then talk to your agent |
| Inspect or decompile one part of an app from the Terminal | `rea analyze` or `rea decompile` |
| Validate, canonicalize, or compare Evidence v2 bundles | `rea evidence-import`, `rea evidence-export`, or `rea compare` |
| Validate, canonicalize, or compare Evidence bundles | `rea evidence-import`, `rea evidence-export`, or `rea compare` |
| Run or resume a persistent two-version artifact analysis | `rea investigate-versions` |
| Map a local JavaScript/Electron application without executing it | `rea analyze PATH --approved` or `rea analyze-javascript-application` |
| Reuse immutable analysis results without relaunching a provider | Pass `--snapshot /approved/path/analysis.json` to a deep-analysis command |
@@ -276,7 +276,6 @@ Filesystem evidence commands and MCP file tools are disabled until the operator
```bash
export REA_EVIDENCE_ROOTS_JSON='["/absolute/path/to/evidence"]'
export REA_INVESTIGATION_INPUT_ROOTS_JSON='["/absolute/path/to/releases"]'
rea evidence-import /absolute/path/to/evidence/bundle.json
rea evidence-export /absolute/path/to/evidence/bundle.json /absolute/path/to/evidence/canonical.json
rea compare /absolute/path/to/evidence/left.json /absolute/path/to/evidence/right.json
@@ -288,17 +287,10 @@ rea analyze-javascript-application /absolute/path/to/releases/app.asar --approve
For a directory or `.asar`, generic `rea analyze` automatically selects the
static JavaScript application provider when neither `--provider` nor
`--snapshot` is supplied. The dedicated command remains available for explicit
format and resource-limit controls. Both routes require `--approved` and an
administrator-approved investigation input root.
If REA is registered with an MCP client, run approved `rea setup` while the
explicit investigation-root variable is set, then restart that client. Setup
copies this non-secret policy into the managed registration; changing the shell
environment alone cannot update an already-running MCP process.
format and traversal controls. Both routes read the selected local input path.
`investigate-versions` inventories both versions, checkpoints their observed
Evidence, derives an artifact comparison, and records a changed-behavior report.
Both input paths must resolve beneath `REA_INVESTIGATION_INPUT_ROOTS_JSON`;
workspace files remain independently restricted by `REA_EVIDENCE_ROOTS_JSON`.
The workspace uses deterministic content identities and monotonic CAS-linked
revisions, so the same request resumes an interrupted run or reuses a completed
@@ -314,10 +306,10 @@ export REA_REFERENCE_ROOTS_JSON='["/absolute/path/to/source"]'
rea import-reference-source /absolute/path/to/source
```
Exports never replace an existing file unless `--overwrite` is explicit. Imports are size/depth bounded, validate every Evidence v2 ID and manifest, and never execute bundle content.
Exports never replace an existing file unless `--overwrite` is explicit. Imports are size/depth bounded, validate every Evidence ID and manifest, and never execute bundle content.
Provider-neutral analysis snapshots persist successful, immutable REA calls and
their Evidence v2 records. They are exact caches rather than Hopper databases:
their Evidence records. They are exact caches rather than Hopper databases:
REA reuses a v2 entry only when the binary digest, kind, format, architecture,
operation parameters, concrete provider build, and canonical analysis-profile
digest match. Hopper loader defaults and configured overrides are normalized by
@@ -384,10 +376,10 @@ REA handles the app analysis in steps 1–5. The agent performs step 6 with its
| Artifact graph | 3 | bounded provider-neutral inspection, deterministic directory/ZIP/APK/IPA/MSIX/AppX/ASAR inventory, and explicitly selected extraction into an absent owned tree |
| Managed PE/CLI | 8 | PE/CLI identity, metadata members, CIL hashes, P/Invoke/native-boundary declarations and verification, application-graph projection, decompiler reconstruction import, token remapping, runtime-correlation plans, and version comparison |
| Browser observation | 9 | exact-origin passive CDP capture, bundle and source-map analysis, WebMCP discovery, session timelines, capture diff, visual evidence, and bounded Playwright scenarios |
| Electron analysis | 5 | passive root-confined observation, bounded static application mapping, evidence-backed static/runtime reconciliation, and separately approved provider-owned click/wait scenarios |
| Electron analysis | 5 | passive root-confined observation, static application mapping, evidence-backed static/runtime reconciliation, and provider-owned click/wait scenarios |
| JavaScript runtime | 2 | approved attach-only Node/Electron Inspector target discovery plus bounded script and execution-context observation without evaluation or instrumentation |
| Application workflows | 12 | bounded cross-layer traces, unique-only version matching, historical-source to bundle mapping, static export return-shape comparison, approved Linux-isolated extracted-module replay, managed-runtime characterization, reconstruction coverage closure, deterministic obligation ledgers, and end-to-end readiness conformance |
| Workspace and observation | 23 | target lifecycle, Evidence v2 bundle snapshots, retained-bundle release, aggregate navigation/address context, direct finite replay-machine evaluation, process/artifact/function comparison, evidence-linked residual-unknown lifecycle |
| Workspace and observation | 22 | target lifecycle, inline Evidence bundle retrieval, aggregate navigation/address context, direct finite replay-machine evaluation, process/artifact/function comparison, evidence-linked residual-unknown lifecycle |
The public interface describes what the agent is trying to learn. Providers decide how to answer. macOS utilities handle common semantic inspection without launching Hopper; Hopper handles deeper native analysis; the process harness implements controlled behavioral capture.
@@ -412,10 +404,10 @@ REA is already useful for native application, browser, and Electron investigatio
- Traverse content-addressed artifact graphs without extraction; on macOS, read-only DMG traversal additionally requires `native_mount_approved: true` and `REA_ARTIFACT_NATIVE_MOUNT_ENABLED=true`. Materialize only approved occurrences into absent output roots.
- Build bounded function dossiers with pseudocode, assembly, CFG edges, comments, calls, references, strings, and names.
- Search and trace features across symbols, strings, metadata, references, and call paths.
- Record every successful result as deterministic Evidence v2 with artifact and provider identity, confidence, authority, limitations, and locations.
- Record every successful result as deterministic Evidence with artifact and provider identity, confidence, authority, limitations, and locations.
- Export and import evidence bundles across sessions.
- Persist automatic cross-version artifact runs as canonical, lock-protected workspaces with tamper-evident revision commitments.
- Capture approved PTY scenarios as Process Capture v4 Evidence, including committed run manifests, raw and rendered terminal frames, scripted interactions, descendant settlement, named filesystem checkpoints, deterministic command shims, and loopback HTTP/WebSocket exchanges.
- Capture approved PTY scenarios as Process Capture Evidence, including committed run manifests, raw and rendered terminal frames, scripted interactions, descendant settlement, named filesystem checkpoints, deterministic command shims, and loopback HTTP/WebSocket exchanges.
- Validate finite replay machines without launching a target through `run_replay_machine` or `rea run-replay-machine`; ordered events return typed decisions, actions, captured aliases, transition journals, final state, and exact limit use without echoing request or captured values.
- Compare complete artifact inventories by stable path, content, metadata, and relations; incomplete evidence never implies equivalence.
- Compare explicit function dossiers across text, calls, references, strings, and address-normalized CFG topology with per-facet unknowns.
@@ -443,7 +435,7 @@ rea list-browser-targets http://127.0.0.1:9222 --approved --json
rea inspect-web-page http://127.0.0.1:9222 TARGET_ID --approved --json
```
All eight browser tools expose the same Evidence v2 contracts over CLI and MCP. Inspection is passive: REA does not evaluate page JavaScript, navigate, click, close the page, or close the browser. Query values, credentials, cookies, authorization headers, storage values, and raw JSON or WebSocket values are never retained. Separately approved captures can retain bounded redacted console primitives, value-free JSON/WebSocket shapes, script sources, accessibility text, or screenshot pixels. Existing activity before attach is explicitly unavailable. See [Website observation with CDP](docs/browser-observation.md) for browser startup, schemas, limits, and the threat model.
All eight browser tools expose the same Evidence contracts over CLI and MCP. Inspection is passive: REA does not evaluate page JavaScript, navigate, click, close the page, or close the browser. Query values, credentials, cookies, authorization headers, storage values, and raw JSON or WebSocket values are never retained. Separately approved captures can retain bounded redacted console primitives, value-free JSON/WebSocket shapes, script sources, accessibility text, or screenshot pixels. Existing activity before attach is explicitly unavailable. See [Website observation with CDP](docs/browser-observation.md) for browser startup, schemas, limits, and the threat model.
### Controlled browser scenarios
@@ -494,7 +486,7 @@ require/import edges, EventEmitter activity, Electron IPC, PID identity, and
Electron role identity stay explicit unknowns. See
[passive Node and Electron runtime observation](docs/javascript-runtime-observation.md).
Exact package, tool-family, provider, setup-client, public schema-version, and CLI facts are generated from source in [`docs/product-catalog.json`](docs/product-catalog.json). PR CI verifies this catalog, narrative documentation, generated schemas, and a clean TypeDoc render.
Exact package, tool-family, provider, setup-client, schema, and CLI facts are generated from source in [`docs/product-catalog.json`](docs/product-catalog.json). PR CI verifies this catalog, narrative documentation, generated schemas, and a clean TypeDoc render.
## Roadmap
@@ -508,7 +500,7 @@ REA is growing into a toolkit for understanding software across static artifacts
### Next
1. **Controlled replay conformance growth** — extend the shipped Linux extracted-module sandbox with more source-owned hostile fixtures and cross-kernel conformance; browser and Electron scenarios remain separately permissioned authorities.
2. **Broader application graph evidence** — extend authenticated cross-layer traces with additional static extractors and separately approved runtime authorities.
2. **Broader application graph evidence** — extend authenticated cross-layer traces with additional static extractors and additional runtime authorities.
3. **Professional managed-code analysis** — extend shipped PE/CLI triage, CIL evidence, managed/native declaration inventory, source-owned conformance, and obfuscation-resistant comparisons toward verified native-provider composition under the accepted [managed-code boundary](docs/managed-code-analysis.md).
4. **Deterministic behavior harnesses** — extend process ownership, protocol fixtures, filesystem observation, reconnects, and cross-version behavioral comparison.
@@ -587,7 +579,6 @@ The agent workflow above is the easiest way to use REA. For a one-off overview f
```bash
npx -y rea-agents@latest analyze /Applications/Notes.app
npx -y rea-agents@latest inspect /Applications/Notes.app
npx -y rea-agents@latest inspect /Applications/Notes.app --detail detailed --limit 20
npx -y rea-agents@latest search /Applications/Notes.app "offline"
npx -y rea-agents@latest function /Applications/Notes.app 0x1000
npx -y rea-agents@latest xrefs /Applications/Notes.app 0x1000
@@ -600,7 +591,7 @@ npx -y rea-agents@latest providers
Run `npx -y rea-agents@latest --help` for direct decompilation, bounded search and
other options. `analyze` and `inspect` share the same overview workflow;
`function`, `xrefs`, and `trace` return the same Evidence v2 envelopes as MCP.
`function`, `xrefs`, and `trace` return the same Evidence envelopes as MCP.
Or install the `rea` command globally:
@@ -709,7 +700,7 @@ Set `REA_PROCESS_CAPTURE_AUTO_GRANT=false` to configure those process-capture
limits as a ceiling without implicitly granting them. This mode remains
fail-closed until a narrower grant is established.
Capture a scenario or compare two saved Process Capture v4 Evidence records:
Capture a scenario or compare two saved Process Capture Evidence records:
```bash
rea capture-process ./scenario.json > authority.json
@@ -719,7 +710,7 @@ rea compare-process-captures authority.json reconstruction.json
The comparison reports each observed dimension separately and identifies the
first terminal, interaction, exit, filesystem, protocol, process, or shim
divergence. See [Process Capture v4](docs/process-capture.md) for scenario
divergence. See [Process Capture](docs/process-capture.md) for scenario
fields, command-shim replay, checkpoint triggers, limits, and safety behavior.
REA installs a prebuilt PTY backend for supported macOS, Linux, and Windows
@@ -803,7 +794,7 @@ natural MCP use, first-tool routing, repeated calls, actual Codex token usage,
completion quality, and explicit treatment of authority and unknowns.
`npm run evidence:generate` regenerates the managed conformance manifest and
Evidence v2 completion ledger from live verifier output. `npm run
Evidence completion ledger from live verifier output. `npm run
evidence:check` reruns the verifier and fails when artifacts, scenarios,
providers, schemas, claim counts, Evidence IDs, or the bundled skill have
drifted. Unsupported claims remain explicit and never count as passes.
README_ar.md
+1 -1
View File
@@ -170,7 +170,7 @@ rea uninstall --purge-data
| تحليل Electron | 5 | مراقبة سلبية ضمن جذور canonical، ورسم ثابت ومحدود للتطبيق، ومطابقة ثابتة/وقت تشغيل قائمة على Evidence |
| وقت تشغيل JavaScript | 2 | اكتشاف أهداف Node/Electron Inspector المعتمدة للاتصال فقط، ومراقبة محدودة للبرامج النصية وسياقات التنفيذ دون تقييم أو تجهيز |
| سير عمل التطبيقات | 12 | تتبع محدود ومقارنة إصدارات فريدة ومقارنة ثابتة لأشكال قيم إرجاع الصادرات وإعادة تشغيل وحدات معزولة ومعتمدة على Linux وتوصيف وقت التشغيل المُدار وإغلاق تغطية إعادة البناء ودفاتر التزامات حتمية والتحقق الشامل من جاهزية إعادة البناء |
| جلسة الملف التنفيذي | 23 | فتح الأهداف وحزم الأدلة وتحرير الحزم المحتفظ بها وتشغيل آلة إعادة محدودة ومقارنة العمليات والقطع والدوال وسجل المجهولات المتبقية |
| جلسة الملف التنفيذي | 22 | فتح الأهداف وحزم الأدلة وتشغيل آلة إعادة محدودة ومقارنة العمليات والقطع والدوال وسجل المجهولات المتبقية |
## استخدام REA مع وكلاء برمجة آخرين
README_ja.md
+1 -1
View File
@@ -160,7 +160,7 @@ REA は手順 1〜5 のバイナリ解析を処理し、手順 6 はエージェ
| Electron 解析 | 5 | canonical ファイルルート内の受動観察、有界な静的アプリマッピング、Evidence に基づく静的/実行時の照合 |
| JavaScript ランタイム | 2 | 承認済みのアタッチ専用 Node/Electron Inspector ターゲット検出と、評価や計装を行わない有界なスクリプト／実行コンテキスト観察 |
| アプリケーション | 12 | 有界なクロスレイヤー追跡、一意限定のバージョン比較、静的なエクスポート戻り値形状比較、承認済み Linux 隔離 extracted-module replay、マネージドランタイム特性評価、再構築カバレッジ閉包、決定論的な義務台帳、エンドツーエンドの準備適合性検証 |
| バイナリセッション | 23 | `open_binary`、`binary_session`、証拠バンドル、保持バンドルの解放、有限リプレイ状態機械、プロセス／成果物／関数比較、残余未知レジストリ |
| バイナリセッション | 22 | `open_binary`、`binary_session`、証拠バンドル、有限リプレイ状態機械、プロセス／成果物／関数比較、残余未知レジストリ |
## 他のコーディングエージェントで使う
README_ko.md
+1 -1
View File
@@ -160,7 +160,7 @@ REA는 1–5단계의 바이너리 분석을 처리합니다. 6단계는 에이
| Electron 분석 | 5 | canonical 파일 루트 내 수동 관찰, 제한된 정적 앱 매핑, Evidence 기반 정적/런타임 조정 |
| JavaScript 런타임 | 2 | 승인된 연결 전용 Node/Electron Inspector 대상 탐색과 평가·계측 없는 제한된 스크립트 및 실행 컨텍스트 관찰 |
| 애플리케이션 | 12 | 제한된 교차 계층 추적, 고유 일치 버전 비교, 정적 export 반환 형태 비교, 승인된 Linux 격리 extracted-module replay, 관리형 런타임 특성화, 재구성 커버리지 폐쇄, 결정론적 의무 원장, 엔드투엔드 준비 적합성 검증 |
| 바이너리 세션 | 23 | `open_binary`, `binary_session`, 증거 번들, 보존 번들 해제, 유한 replay 상태 머신, 프로세스·아티팩트·함수 비교, 잔여 미지수 레지스트리 |
| 바이너리 세션 | 22 | `open_binary`, `binary_session`, 증거 번들, 유한 replay 상태 머신, 프로세스·아티팩트·함수 비교, 잔여 미지수 레지스트리 |
## 다른 코딩 에이전트에서 사용하기
README_zh.md
+1 -1
View File
@@ -160,7 +160,7 @@ REA 负责第 1–5 步中的二进制分析。第 6 步由智能体使用其常
| Electron 分析 | 5 | 限定 canonical 文件根的被动观察、有界静态应用映射，以及基于 Evidence 的静态/运行时对账 |
| JavaScript 运行时 | 2 | 经批准的只附加 Node/Electron Inspector 目标发现，以及不执行求值或插桩的有界脚本和执行上下文观察 |
| 应用工作流 | 12 | 有界跨层追踪、唯一匹配跨版本图比较、静态导出返回形状比较、经批准的 Linux 隔离 extracted-module replay、托管运行时表征、重建覆盖闭合、确定性义务账本和端到端就绪性一致性验证 |
| 二进制会话 | 23 | `open_binary`、`binary_session`、证据包、保留包释放、有限重放状态机、进程、制品与函数比较、残余未知项注册表 |
| 二进制会话 | 22 | `open_binary`、`binary_session`、证据包、有限重放状态机、进程、制品与函数比较、残余未知项注册表 |
## 与其他编程智能体一起使用
bridge/ghidra/ReaGhidraBridge.java
+49 -630
View File
@@ -83,21 +83,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
private static final int MAX_RESPONSE_BYTES = 1024 * 1024;
private static final int RESPONSE_ENVELOPE_RESERVE_BYTES = 32 * 1024;
private static final int MAX_NATIVE_API_RESPONSE_BYTES = 192 * 1024;
private static final int MAX_INVENTORY_ITEMS = 1_000_000;
private static final int MAX_FUNCTION_ITEMS = 100_000;
private static final int MAX_FUNCTION_INSTRUCTIONS = 100_000;
private static final int MAX_NATIVE_API_PARAMETERS = 64;
private static final int MAX_NATIVE_API_JUMP_TABLES = 4;
private static final int MAX_NATIVE_API_LOAD_TABLES = 4;
private static final int MAX_NATIVE_API_MAPPINGS = 32;
private static final int DECOMPILE_TIMEOUT_SECONDS = 30;
private static final int DECOMPILE_PAYLOAD_MBYTES = 8;
private static final int MAX_LIST_VALUE_CODE_POINTS = 1_024;
private static final int MAX_SEARCH_VALUE_CODE_POINTS = 4_096;
private static final int MAX_REGEX_CANDIDATE_CHARACTERS = 4_096;
private static final long MAX_REGEX_BACKTRACKING_PATHS = 10_000;
private static final long MAX_LITERAL_SEARCH_WORK_UNITS = 1_000_000;
private static final long MAX_REGEX_SEARCH_WORK_UNITS = 1_000_000;
private static final Gson GSON = new GsonBuilder().serializeNulls().create();
private static final Set<String> DESCRIPTOR_KEYS = Set.of(
"schema_version",
@@ -375,7 +366,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
DecompileOptions options = new DecompileOptions();
options.setDefaultTimeout(DECOMPILE_TIMEOUT_SECONDS);
options.setMaxPayloadMBytes(DECOMPILE_PAYLOAD_MBYTES);
options.setMaxInstructions(MAX_FUNCTION_INSTRUCTIONS);
options.setMaxInstructions(Integer.MAX_VALUE);
decompiler = new DecompInterface();
decompiler.setOptions(options);
decompiler.toggleCCode(true);
@@ -398,43 +389,19 @@ public final class ReaGhidraBridge extends HeadlessScript {
requireKeys(params, Set.of("document", "procedure"));
requireDocument(params);
Function function = resolveProcedure(requireString(params, "procedure"));
InstructionScan scan = scanInstructions(function, MAX_FUNCTION_INSTRUCTIONS);
if (scan.truncated) {
throw functionLimit("Procedure assembly exceeds the 100000-instruction limit");
}
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
return GSON.toJsonTree(renderAssembly(scan.instructions));
}
private JsonObject readFunctionInstructions(JsonObject params) throws Exception {
requireKeys(params, Set.of("document", "procedure", "offset", "limit"));
requireKeys(params, Set.of("document", "procedure"));
requireDocument(params);
Function function = resolveProcedure(requireString(params, "procedure"));
int offset = requireBoundedInteger(params, "offset", 0, 100_000);
int limit = requireBoundedInteger(params, "limit", 1, 500);
InstructionScan scan = scanInstructions(function, offset + limit + 1);
int start = Math.min(offset, scan.instructions.size());
int end = Math.min(scan.instructions.size(), start + limit);
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
JsonArray items = new JsonArray();
for (String line : renderAssemblyLines(scan.instructions.subList(start, end))) {
for (String line : renderAssemblyLines(scan.instructions)) {
items.add(line);
}
boolean hasMore = scan.truncated || end < scan.instructions.size();
JsonObject instructions = new JsonObject();
instructions.add("items", items);
if (scan.truncated) {
instructions.add("total", JsonNull.INSTANCE);
}
else {
instructions.addProperty("total", scan.instructions.size());
}
instructions.addProperty("returned", items.size());
instructions.addProperty("truncated", hasMore);
if (hasMore) {
instructions.addProperty("next_offset", end);
}
else {
instructions.add("next_offset", JsonNull.INSTANCE);
}
JsonArray limitations = new JsonArray();
limitations.add("Instruction text and ordering are Ghidra-specific representations.");
@@ -443,9 +410,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
);
JsonObject result = new JsonObject();
result.add("procedure", procedureIdentity(function));
result.add("instructions", instructions);
result.addProperty("instructions_scanned", scan.instructions.size());
result.addProperty("instruction_scan_truncated", scan.truncated);
result.add("instructions", items);
result.add("limitations", limitations);
return result;
}
@@ -457,9 +422,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
Set<Function> observed = callers
? function.getCallingFunctions(monitor)
: function.getCalledFunctions(monitor);
if (observed.size() > MAX_FUNCTION_ITEMS) {
throw functionLimit("Resolved call set exceeds the 100000-item limit");
}
List<Function> ordered = new ArrayList<>(observed);
ordered.sort(Comparator.comparing(Function::getEntryPoint));
JsonArray result = new JsonArray();
@@ -490,10 +452,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
Set.of(
"document",
"procedure",
"direction",
"offset",
"limit",
"max_instructions"
"direction"
)
);
requireDocument(params);
@@ -502,10 +461,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (!direction.equals("incoming") && !direction.equals("outgoing")) {
throw new RequestFailure("invalid_request", "direction must be incoming or outgoing");
}
int offset = requireBoundedInteger(params, "offset", 0, Integer.MAX_VALUE);
int limit = requireBoundedInteger(params, "limit", 1, 500);
int maximum = requireBoundedInteger(params, "max_instructions", 1, 5_000);
InstructionScan scan = scanInstructions(function, maximum);
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
List<Reference> references = collectReferences(scan.instructions, direction);
JsonArray edges = new JsonArray();
for (Reference reference : references) {
@@ -514,9 +470,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonObject result = new JsonObject();
result.add("procedure", procedureIdentity(function));
result.addProperty("direction", direction);
result.add("references", bounded(edges, offset, limit, !scan.truncated));
result.addProperty("instructions_scanned", scan.instructions.size());
result.addProperty("instruction_scan_truncated", scan.truncated);
result.add("references", edges);
return result;
}
@@ -533,10 +487,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
continue;
}
sources.add(reference.getFromAddress());
if (sources.size() > MAX_FUNCTION_ITEMS) {
throw functionLimit("Cross-reference set exceeds the 100000-item limit");
}
}
List<Address> ordered = new ArrayList<>(sources);
ordered.sort(Address::compareTo);
JsonArray result = new JsonArray();
@@ -547,48 +498,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
}
private JsonObject analyzeFunction(JsonObject params) throws Exception {
requireKeys(
params,
Set.of(
"procedure",
"include_assembly",
"limit",
"max_pseudocode_chars",
"max_instructions",
"pseudocode_offset",
"assembly_offset",
"collection_offset"
)
);
requireKeys(params, Set.of("procedure"));
Function function = resolveProcedure(requireString(params, "procedure"));
boolean includeAssembly = requireBoolean(params, "include_assembly");
int limit = requireBoundedInteger(params, "limit", 1, 500);
int maximumCharacters = requireBoundedInteger(
params,
"max_pseudocode_chars",
1,
100_000
);
int maximumInstructions = requireBoundedInteger(
params,
"max_instructions",
1,
5_000
);
int pseudocodeOffset = requireBoundedInteger(
params,
"pseudocode_offset",
0,
Integer.MAX_VALUE
);
int assemblyOffset = requireBoundedInteger(
params,
"assembly_offset",
0,
Integer.MAX_VALUE
);
JsonObject offsets = requireCollectionOffsets(params);
InstructionScan scan = scanInstructions(function, maximumInstructions);
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
DecompileResults decompilation = decompile(function);
String pseudocode = decompiledText(decompilation);
if (pseudocode == null) {
@@ -607,75 +519,24 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonArray referencedStrings = referencedStrings(outgoingReferences);
JsonArray referencedNames = referencedNames(outgoingReferences);
JsonArray blocks = basicBlockValues(function);
JsonArray assembly = includeAssembly
? GSON.toJsonTree(renderAssemblyLines(scan.instructions)).getAsJsonArray()
: new JsonArray();
JsonArray assembly = GSON.toJsonTree(renderAssemblyLines(scan.instructions))
.getAsJsonArray();
JsonObject procedure = procedureIdentity(function);
procedure.addProperty("signature", function.getPrototypeString(false, true));
procedure.add("locals", functionLocals(function));
JsonObject result = new JsonObject();
result.add("procedure", procedure);
result.add(
"pseudocode",
pseudocodePage(pseudocode, pseudocodeOffset, maximumCharacters)
);
result.add(
"assembly",
includeAssembly
? bounded(assembly, assemblyOffset, maximumInstructions, !scan.truncated)
: bounded(assembly, 0, maximumInstructions, true)
);
result.add(
"comments",
bounded(comments, collectionOffset(offsets, "comments"), limit, !scan.truncated)
);
result.add("callers", bounded(callers, collectionOffset(offsets, "callers"), limit, true));
result.add("callees", bounded(callees, collectionOffset(offsets, "callees"), limit, true));
result.add(
"incoming_references",
bounded(
incoming,
collectionOffset(offsets, "incoming_references"),
limit,
!scan.truncated
)
);
result.add(
"outgoing_references",
bounded(
outgoing,
collectionOffset(offsets, "outgoing_references"),
limit,
!scan.truncated
)
);
result.add(
"referenced_strings",
bounded(
referencedStrings,
collectionOffset(offsets, "referenced_strings"),
limit,
!scan.truncated
)
);
result.add(
"referenced_names",
bounded(
referencedNames,
collectionOffset(offsets, "referenced_names"),
limit,
!scan.truncated
)
);
result.add(
"basic_blocks",
bounded(blocks, collectionOffset(offsets, "basic_blocks"), limit, true)
);
JsonObject instructionScan = new JsonObject();
instructionScan.addProperty("scanned", scan.instructions.size());
instructionScan.addProperty("truncated", scan.truncated);
result.add("instruction_scan", instructionScan);
result.addProperty("pseudocode", pseudocode);
result.add("assembly", assembly);
result.add("comments", comments);
result.add("callers", callers);
result.add("callees", callees);
result.add("incoming_references", incoming);
result.add("outgoing_references", outgoing);
result.add("referenced_strings", referencedStrings);
result.add("referenced_names", referencedNames);
result.add("basic_blocks", blocks);
int nativeApiBudget = Math.min(
MAX_NATIVE_API_RESPONSE_BYTES,
Math.max(
@@ -721,8 +582,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private JsonObject listNames(JsonObject params) throws Exception {
requireKeys(params, Set.of("document", "address", "offset", "limit"));
private JsonArray listNames(JsonObject params) throws Exception {
requireKeys(params, Set.of("document", "address"));
requireDocument(params);
List<InventoryItem> inventory = names();
String requested = optionalString(params, "address");
@@ -732,26 +593,16 @@ public final class ReaGhidraBridge extends HeadlessScript {
.filter(item -> item.address.equals(address))
.toList();
}
return page(
inventory,
requireBoundedInteger(params, "offset", 0, Integer.MAX_VALUE),
requireBoundedInteger(params, "limit", 1, 500),
"symbol"
);
return inventory(inventory, "symbol");
}
private JsonObject listProcedures(JsonObject params) throws Exception {
requireKeys(params, Set.of("document", "offset", "limit"));
private JsonArray listProcedures(JsonObject params) throws Exception {
requireKeys(params, Set.of("document"));
requireDocument(params);
List<InventoryItem> inventory = procedures().stream()
.map(FunctionEntry::item)
.toList();
return page(
inventory,
requireBoundedInteger(params, "offset", 0, Integer.MAX_VALUE),
requireBoundedInteger(params, "limit", 1, 500),
"procedure"
);
return inventory(inventory, "procedure");
}
private JsonArray listSegments(JsonObject params) {
@@ -760,9 +611,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonArray result = new JsonArray();
String imageBase = canonicalAddress(currentProgram.getImageBase());
MemoryBlock[] blocks = currentProgram.getMemory().getBlocks();
if (blocks.length > MAX_INVENTORY_ITEMS) {
throw inventoryLimit();
}
List<MemoryBlock> ordered = new ArrayList<>(List.of(blocks));
ordered.sort(Comparator.comparing(MemoryBlock::getStart));
for (MemoryBlock block : ordered) {
@@ -773,8 +621,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private JsonObject listStrings(JsonObject params) throws Exception {
requireKeys(params, Set.of("document", "address", "offset", "limit"));
private JsonArray listStrings(JsonObject params) throws Exception {
requireKeys(params, Set.of("document", "address"));
requireDocument(params);
List<InventoryItem> inventory = strings();
String requested = optionalString(params, "address");
@@ -784,12 +632,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
.filter(item -> item.address.equals(address))
.toList();
}
return page(
inventory,
requireBoundedInteger(params, "offset", 0, Integer.MAX_VALUE),
requireBoundedInteger(params, "limit", 1, 500),
"string"
);
return inventory(inventory, "string");
}
private JsonElement procedureAddress(JsonObject params) throws Exception {
@@ -823,23 +666,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
private JsonObject search(JsonObject params, boolean procedureSearch) throws Exception {
requireKeys(
params,
Set.of("pattern", "mode", "case_sensitive", "offset", "limit", "document")
Set.of("pattern", "mode", "case_sensitive", "document")
);
requireDocument(params);
String expression = requireString(params, "pattern");
if (expression.codePointCount(0, expression.length()) > 256) {
throw new RequestFailure(
"invalid_request",
"pattern must contain between 1 and 256 characters"
);
}
String mode = requireString(params, "mode");
if (!mode.equals("literal") && !mode.equals("regex")) {
throw new RequestFailure("invalid_request", "mode must be literal or regex");
}
boolean caseSensitive = requireBoolean(params, "case_sensitive");
int offset = requireBoundedInteger(params, "offset", 0, Integer.MAX_VALUE);
int limit = requireBoundedInteger(params, "limit", 1, 100);
List<InventoryItem> inventory = procedureSearch
? procedures().stream().map(FunctionEntry::item).toList()
: strings();
@@ -847,18 +682,13 @@ public final class ReaGhidraBridge extends HeadlessScript {
? literalMatcher(expression, caseSensitive)
: regexMatcher(expression, caseSensitive);
JsonArray items = new JsonArray();
int total = 0;
long pageEnd = (long) offset + limit;
for (InventoryItem item : inventory) {
if (!matcher.matches(item.value)) {
continue;
}
if (total >= offset && total < pageEnd) {
items.add(resultItem(item, null, MAX_SEARCH_VALUE_CODE_POINTS));
}
total += 1;
items.add(resultItem(item, null));
}
return pageResult(items, offset, limit, total);
return items;
}
private List<InventoryItem> names() throws Exception {
@@ -929,9 +759,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
symbol == null ? function.getName() : symbol.getName(true),
facts
);
if (destination.size() >= MAX_INVENTORY_ITEMS) {
throw inventoryLimit();
}
destination.add(new FunctionEntry(function, item));
}
}
@@ -1540,9 +1367,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
reference.getOperandIndex() + "\u0000" +
reference.isPrimary();
observed.putIfAbsent(key, reference);
if (observed.size() > MAX_FUNCTION_ITEMS) {
throw functionLimit("Reference set exceeds the 100000-item limit");
}
}
private JsonArray referenceEdges(List<Reference> references) {
@@ -1623,9 +1447,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
private static JsonArray functionLocals(Function function) {
Variable[] variables = function.getLocalVariables();
if (variables.length > MAX_FUNCTION_ITEMS) {
throw functionLimit("Local-variable set exceeds the 100000-item limit");
}
List<Variable> ordered = new ArrayList<>(List.of(variables));
ordered.sort(
Comparator.comparing(Variable::getName)
@@ -1646,9 +1467,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
}
private JsonArray procedureIdentities(Set<Function> functions) {
if (functions.size() > MAX_FUNCTION_ITEMS) {
throw functionLimit("Resolved call set exceeds the 100000-item limit");
}
List<Function> ordered = new ArrayList<>(functions);
ordered.sort(Comparator.comparing(Function::getEntryPoint));
JsonArray result = new JsonArray();
@@ -1664,9 +1482,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
List<CodeBlock> result = new ArrayList<>();
while (iterator.hasNext()) {
monitor.checkCancelled();
if (result.size() >= MAX_FUNCTION_ITEMS) {
throw functionLimit("Basic-block set exceeds the 100000-item limit");
}
result.add(iterator.next());
}
result.sort(Comparator.comparing(CodeBlock::getFirstStartAddress));
@@ -1723,10 +1538,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
String key = canonicalAddress(instruction.getAddress()) + "\u0000" +
kind + "\u0000" + type.name() + "\u0000" + text;
observed.put(key, item);
if (observed.size() > MAX_FUNCTION_ITEMS) {
throw functionLimit("Comment set exceeds the 100000-item limit");
}
}
}
}
JsonArray result = new JsonArray();
observed.values().forEach(result::add);
@@ -1753,10 +1565,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
String key = canonicalAddress(data.getAddress()) + "\u0000" + value + "\u0000" +
canonicalAddress(reference.getFromAddress());
observed.put(key, item);
if (observed.size() > MAX_FUNCTION_ITEMS) {
throw functionLimit("Referenced-string set exceeds the 100000-item limit");
}
}
JsonArray result = new JsonArray();
observed.values().forEach(result::add);
return result;
@@ -1778,36 +1587,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
String key = canonicalAddress(reference.getToAddress()) + "\u0000" +
symbol.getName(true) + "\u0000" + canonicalAddress(reference.getFromAddress());
observed.put(key, item);
if (observed.size() > MAX_FUNCTION_ITEMS) {
throw functionLimit("Referenced-name set exceeds the 100000-item limit");
}
}
JsonArray result = new JsonArray();
observed.values().forEach(result::add);
return result;
}
private static JsonObject pseudocodePage(String value, int offset, int limit) {
int total = value.codePointCount(0, value.length());
int start = Math.min(offset, total);
int returned = Math.min(limit, total - start);
int startIndex = value.offsetByCodePoints(0, start);
int endIndex = value.offsetByCodePoints(startIndex, returned);
int next = start + returned;
JsonObject result = new JsonObject();
result.addProperty("text", value.substring(startIndex, endIndex));
result.addProperty("total_chars", total);
result.addProperty("returned_chars", returned);
result.addProperty("truncated", next < total);
if (next < total) {
result.addProperty("next_offset", next);
}
else {
result.add("next_offset", JsonNull.INSTANCE);
}
return result;
}
private static JsonObject bounded(
JsonArray values,
int offset,
@@ -1839,35 +1624,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private static JsonObject requireCollectionOffsets(JsonObject params) {
JsonElement value = params.get("collection_offset");
if (value == null || !value.isJsonObject()) {
throw new RequestFailure("invalid_request", "collection_offset must be an object");
}
JsonObject result = value.getAsJsonObject();
requireKeys(
result,
Set.of(
"comments",
"callers",
"callees",
"incoming_references",
"outgoing_references",
"referenced_strings",
"referenced_names",
"basic_blocks"
)
);
for (String name : result.keySet()) {
requireBoundedInteger(result, name, 0, Integer.MAX_VALUE);
}
return result;
}
private static int collectionOffset(JsonObject offsets, String name) {
return requireBoundedInteger(offsets, name, 0, Integer.MAX_VALUE);
}
private static String exclusiveAddress(Address inclusive) {
Address next = inclusive.next();
if (next != null && next.getAddressSpace().equals(inclusive.getAddressSpace())) {
@@ -1884,51 +1640,21 @@ public final class ReaGhidraBridge extends HeadlessScript {
return truncate(value.replaceAll("[\\r\\n]+", " "), 512).value;
}
private static RequestFailure functionLimit(String message) {
return new RequestFailure("limit_exceeded", message);
}
private JsonObject page(
List<InventoryItem> inventory,
int offset,
int limit,
String factsName) {
int start = Math.min(offset, inventory.size());
int end = Math.min(inventory.size(), start + limit);
JsonArray items = new JsonArray();
for (int index = start; index < end; index += 1) {
items.add(resultItem(inventory.get(index), factsName, MAX_LIST_VALUE_CODE_POINTS));
private JsonArray inventory(
List<InventoryItem> inventory,
String factsName) {
JsonArray result = new JsonArray();
for (InventoryItem item : inventory) {
monitor.checkCancelled();
result.add(resultItem(item, factsName));
}
return pageResult(items, offset, limit, inventory.size());
}
private static JsonObject pageResult(JsonArray items, int offset, int limit, int total) {
int next = offset + items.size();
boolean hasMore = next < total;
JsonObject result = new JsonObject();
result.add("items", items);
result.addProperty("offset", offset);
result.addProperty("limit", limit);
result.addProperty("total", total);
if (hasMore) {
result.addProperty("next_offset", next);
}
else {
result.add("next_offset", JsonNull.INSTANCE);
}
result.addProperty("has_more", hasMore);
return result;
}
private static JsonObject resultItem(
InventoryItem item,
String factsName,
int maximumCodePoints) {
TruncatedValue truncated = truncate(item.value, maximumCodePoints);
private static JsonObject resultItem(InventoryItem item, String factsName) {
JsonObject result = new JsonObject();
result.addProperty("address", canonicalAddress(item.address));
result.addProperty("value", truncated.value);
result.addProperty("value_truncated", truncated.truncated);
result.addProperty("value", item.value);
if (factsName != null) {
result.add(factsName, item.facts.deepCopy());
}
@@ -2089,19 +1815,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
}
private static void addBounded(List<InventoryItem> destination, InventoryItem item) {
if (destination.size() >= MAX_INVENTORY_ITEMS) {
throw inventoryLimit();
}
destination.add(item);
}
private static RequestFailure inventoryLimit() {
return new RequestFailure(
"limit_exceeded",
"Ghidra inventory exceeds the 1000000-item safety limit"
);
}
private static TruncatedValue truncate(String value, int maximumCodePoints) {
int codePoints = value.codePointCount(0, value.length());
if (codePoints <= maximumCodePoints) {
@@ -2113,30 +1829,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
private static ValueMatcher literalMatcher(String pattern, boolean caseSensitive) {
String needle = caseSensitive ? pattern : pattern.toLowerCase(Locale.ROOT);
return new ValueMatcher() {
private long remaining = MAX_LITERAL_SEARCH_WORK_UNITS;
@Override
public boolean matches(String value) {
long required = (long) Math.max(value.length(), 1) + needle.length();
if (required > remaining) {
throw new RequestFailure(
"limit_exceeded",
"Literal search exceeds the 1000000-unit work budget"
);
}
remaining -= required;
String candidate = caseSensitive ? value : value.toLowerCase(Locale.ROOT);
return candidate.contains(needle);
}
return value -> {
String candidate = caseSensitive ? value : value.toLowerCase(Locale.ROOT);
return candidate.contains(needle);
};
}
private static ValueMatcher regexMatcher(String expression, boolean caseSensitive) {
RegexMetrics metrics;
Pattern pattern;
try {
metrics = new BoundedRegexParser(expression).parse();
pattern = Pattern.compile(
expression,
caseSensitive ? 0 : Pattern.CASE_INSENSITIVE | Pattern.UNICODE_CASE
@@ -2145,29 +1846,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
catch (PatternSyntaxException exception) {
throw new RequestFailure("invalid_request", "Invalid regex pattern");
}
long workPerCharacter = metrics.paths * Math.max(metrics.steps, 1);
return new ValueMatcher() {
private long remaining = MAX_REGEX_SEARCH_WORK_UNITS;
@Override
public boolean matches(String value) {
if (value.length() > MAX_REGEX_CANDIDATE_CHARACTERS) {
throw new RequestFailure(
"limit_exceeded",
"Regex candidate exceeds the 4096-character safety limit"
);
}
long required = workPerCharacter * Math.max(value.length(), 1);
if (required > remaining) {
throw new RequestFailure(
"limit_exceeded",
"Regex search exceeds the 1000000-unit work budget"
);
}
remaining -= required;
return pattern.matcher(value).find();
}
};
return value -> pattern.matcher(value).find();
}
private static SessionDescriptor readDescriptor(Path path) throws IOException {
@@ -2407,7 +2086,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
String profileDigest
) {}
private record Request(int id, String method, JsonObject params) {}
private record RegexMetrics(long paths, long steps, boolean containsRepeat) {}
private static final class RequestFailure extends RuntimeException {
private final String code;
@@ -2418,263 +2096,4 @@ public final class ReaGhidraBridge extends HeadlessScript {
}
}
private static final class BoundedRegexParser {
private final String expression;
private int index;
BoundedRegexParser(String expression) {
this.expression = expression;
}
RegexMetrics parse() {
RegexMetrics result = parseExpression(false);
if (index != expression.length()) {
throw invalidRegex();
}
return result;
}
private RegexMetrics parseExpression(boolean grouped) {
RegexMetrics result = parseSequence(grouped);
while (peek('|')) {
index += 1;
RegexMetrics branch = parseSequence(grouped);
result = new RegexMetrics(
checkedPaths(result.paths, branch.paths, false),
Math.max(result.steps, branch.steps),
result.containsRepeat || branch.containsRepeat
);
}
return result;
}
private RegexMetrics parseSequence(boolean grouped) {
RegexMetrics result = new RegexMetrics(1, 0, false);
while (index < expression.length() && !peek('|') && !(grouped && peek(')'))) {
RegexMetrics item = parseAtom();
result = new RegexMetrics(
checkedPaths(result.paths, item.paths, true),
result.steps + item.steps,
result.containsRepeat || item.containsRepeat
);
}
return result;
}
private RegexMetrics parseAtom() {
if (index >= expression.length()) {
throw invalidRegex();
}
char item = expression.charAt(index++);
RegexMetrics atom;
if (item == '(') {
if (peek('?')) {
if (index + 1 >= expression.length() || expression.charAt(index + 1) != ':') {
throw new RequestFailure(
"invalid_request",
"Regex lookarounds and backreferences are not supported"
);
}
index += 2;
}
atom = parseExpression(true);
if (!peek(')')) {
throw invalidRegex();
}
index += 1;
}
else if (item == '[') {
consumeCharacterClass();
atom = leaf();
}
else if (item == '\\') {
consumeEscape();
atom = leaf();
}
else if (item == '*' || item == '+') {
throw unboundedRepeat();
}
else if (item == '?' || item == '{' || item == '}' || item == ')') {
throw invalidRegex();
}
else {
atom = leaf();
}
if (index >= expression.length()) {
return atom;
}
if (peek('*') || peek('+')) {
throw unboundedRepeat();
}
int minimum;
int maximum;
if (peek('?')) {
index += 1;
minimum = 0;
maximum = 1;
}
else if (peek('{')) {
int[] bounds = consumeBounds();
minimum = bounds[0];
maximum = bounds[1];
}
else {
return atom;
}
if (atom.containsRepeat) {
throw new RequestFailure(
"invalid_request",
"Nested regex repetitions are not supported"
);
}
if (maximum > 1_000) {
throw unboundedRepeat();
}
return new RegexMetrics(
repeatPaths(atom.paths, minimum, maximum),
maximum * atom.steps,
true
);
}
private void consumeCharacterClass() {
boolean escaped = false;
boolean hasContent = false;
while (index < expression.length()) {
char item = expression.charAt(index++);
if (escaped) {
escaped = false;
hasContent = true;
}
else if (item == '\\') {
escaped = true;
}
else if (item == ']' && hasContent) {
return;
}
else {
hasContent = true;
}
}
throw invalidRegex();
}
private void consumeEscape() {
if (index >= expression.length()) {
throw invalidRegex();
}
char escaped = expression.charAt(index++);
if (Character.isDigit(escaped) || escaped == 'k') {
throw new RequestFailure(
"invalid_request",
"Regex lookarounds and backreferences are not supported"
);
}
if (escaped == 'Q' || escaped == 'E' || escaped == 'p' || escaped == 'P') {
throw new RequestFailure(
"invalid_request",
"Regex operation is not supported by the bounded matcher"
);
}
if (escaped == 'x') {
consumeHexDigits(2);
}
else if (escaped == 'u') {
consumeHexDigits(4);
}
}
private void consumeHexDigits(int count) {
if (index + count > expression.length()) {
throw invalidRegex();
}
for (int current = 0; current < count; current += 1) {
if (Character.digit(expression.charAt(index + current), 16) < 0) {
throw invalidRegex();
}
}
index += count;
}
private int[] consumeBounds() {
index += 1;
int minimum = consumeDecimal();
int maximum = minimum;
if (peek(',')) {
index += 1;
if (peek('}')) {
throw unboundedRepeat();
}
maximum = consumeDecimal();
}
if (!peek('}') || minimum > maximum) {
throw invalidRegex();
}
index += 1;
return new int[] { minimum, maximum };
}
private int consumeDecimal() {
int start = index;
long value = 0;
while (index < expression.length() && Character.isDigit(expression.charAt(index))) {
value = value * 10 + Character.digit(expression.charAt(index), 10);
if (value > Integer.MAX_VALUE) {
throw unboundedRepeat();
}
index += 1;
}
if (index == start) {
throw invalidRegex();
}
return (int) value;
}
private boolean peek(char expected) {
return index < expression.length() && expression.charAt(index) == expected;
}
private static RegexMetrics leaf() {
return new RegexMetrics(1, 1, false);
}
private static long repeatPaths(long childPaths, int minimum, int maximum) {
long result = 0;
long repeated = 1;
for (int count = 0; count <= maximum; count += 1) {
if (count >= minimum) {
result = checkedPaths(result, repeated, false);
}
if (count < maximum) {
repeated = checkedPaths(repeated, childPaths, true);
}
}
return result;
}
private static long checkedPaths(long left, long right, boolean multiply) {
boolean exceeded = multiply
? right != 0 && left > MAX_REGEX_BACKTRACKING_PATHS / right
: left > MAX_REGEX_BACKTRACKING_PATHS - right;
long result = multiply ? left * right : left + right;
if (exceeded || result > MAX_REGEX_BACKTRACKING_PATHS) {
throw new RequestFailure(
"invalid_request",
"Regex exceeds the 10000-path backtracking budget"
);
}
return result;
}
private static RequestFailure unboundedRepeat() {
return new RequestFailure(
"invalid_request",
"Unbounded or excessive regex repetitions are not supported"
);
}
private static RequestFailure invalidRegex() {
return new RequestFailure("invalid_request", "Invalid regex pattern");
}
}
}
bridge/hopper_bridge.py
+90 -307
View File
@@ -9,7 +9,6 @@ import json
import hmac
import os
import re
import sre_parse
import socket
from typing import Any, Optional, Protocol, Sequence
@@ -17,11 +16,10 @@ MAX_LINE_BYTES = 10 * 1024 * 1024
MAX_READ_BYTES = 4096
BAD_ADDRESSES = (-1, 0xFFFFFFFFFFFFFFFF, None)
_selected_document = None
_rea_session_document = None
_search_inventory_cache = {}
_pseudocode_cache = {}
_hopper_api = None
MAX_SEARCH_PATTERN_LENGTH = 256
MAX_SEARCH_VALUE_LENGTH = 4096
class HopperDocument(Protocol):
@@ -69,8 +67,9 @@ class HopperApiFacade:
def _configure_hopper_api(document_provider):
"""Inject Hopper's document provider or a standalone test facade."""
global _hopper_api
global _hopper_api, _rea_session_document
_hopper_api = HopperApiFacade(document_provider)
_rea_session_document = None
def _api():
@@ -88,21 +87,45 @@ def _api():
def _session_document():
"""Find only the document opened for this authenticated REA session."""
"""Return the exact document bound to this authenticated REA session."""
global _rea_session_document
documents = _api().documents()
if _rea_session_document is not None:
return (
_rea_session_document
if any(document is _rea_session_document for document in documents)
else None
)
return _bind_session_document(documents)
def _bind_session_document(documents=None):
"""Bind this bridge to Hopper's current document for its target."""
global _rea_session_document
api = _api()
if documents is None:
documents = api.documents()
target = os.path.realpath(REA_TARGET_PATH)
for document in _api().documents():
def matches_target(document):
paths = (document.getExecutableFilePath(), document.getDatabaseFilePath())
for path in paths:
if path and os.path.realpath(path) == target:
return document
return any(path and os.path.realpath(path) == target for path in paths)
current = api.current_document()
if current is not None and matches_target(current):
_rea_session_document = current
return current
matching = [document for document in documents if matches_target(document)]
if len(matching) == 1:
_rea_session_document = matching[0]
return matching[0]
if len(matching) > 1:
raise CapabilityUnavailableError(
"The REA session document is ambiguous; Hopper did not identify the launched document"
)
return None
MAX_REGEX_BACKTRACKING_PATHS = 10000
MAX_REGEX_CANDIDATE_LENGTH = 4096
MAX_REGEX_SEARCH_WORK_UNITS = 1000000
class CapabilityUnavailableError(Exception):
"""The active Hopper build lacks one admitted public API operation."""
@@ -149,14 +172,22 @@ def _document(name=None):
api = _api()
documents = api.documents()
if name is not None:
for candidate in documents:
if candidate.getDocumentName() == name:
return candidate
matches = [candidate for candidate in documents if candidate.getDocumentName() == name]
if len(matches) == 1:
return matches[0]
if len(matches) > 1:
current = api.current_document()
if current is not None and current.getDocumentName() == name:
return current
raise ValueError("Hopper document name is ambiguous")
raise ValueError("Unknown Hopper document")
if _selected_document is not None:
for candidate in documents:
if candidate.getDocumentName() == _selected_document:
return candidate
session_document = _session_document()
if session_document is not None:
return session_document
current = api.current_document()
if current is None:
raise ValueError("No Hopper document is loaded")
@@ -225,7 +256,7 @@ def _containing_procedure(document, address):
return (procedure, None) if procedure is not None else (None, "not_in_procedure")
def _instruction_addresses(procedure, limit):
def _instruction_addresses(procedure, limit=None):
result = []
seen = set()
segment = procedure.getSegment()
@@ -234,7 +265,7 @@ def _instruction_addresses(procedure, limit):
address = block.getStartingAddress()
end = block.getEndingAddress()
while address < end and address not in seen:
if len(result) >= limit:
if limit is not None and len(result) >= limit:
truncated = True
return result, truncated
seen.add(address)
@@ -252,18 +283,9 @@ def _instruction_addresses(procedure, limit):
def _procedure_references(document, params):
procedure = _procedure(document, params.get("procedure"))
direction = params.get("direction", "outgoing")
offset = params.get("offset", 0)
limit = params.get("limit", 100)
max_instructions = params.get("max_instructions", 500)
if direction not in ("incoming", "outgoing"):
raise ValueError("direction must be incoming or outgoing")
if not isinstance(offset, int) or isinstance(offset, bool) or offset < 0:
raise ValueError("offset must be a non-negative integer")
if not isinstance(limit, int) or isinstance(limit, bool) or limit < 1 or limit > 500:
raise ValueError("limit must be an integer between 1 and 500")
if not isinstance(max_instructions, int) or isinstance(max_instructions, bool) or max_instructions < 1 or max_instructions > 5000:
raise ValueError("max_instructions must be an integer between 1 and 5000")
addresses, scan_truncated = _instruction_addresses(procedure, max_instructions)
addresses, _ = _instruction_addresses(procedure)
edges = set()
for address in addresses:
segment = _segment(document, address)
@@ -272,8 +294,7 @@ def _procedure_references(document, params):
edges.add((address, reference) if direction == "outgoing" else (reference, address))
ordered = sorted(edges)
items = []
selected = ordered[offset:offset + limit]
for source, target in selected:
for source, target in ordered:
source_procedure, _ = _containing_procedure(document, source)
target_procedure, _ = _containing_procedure(document, target)
items.append({
@@ -283,13 +304,9 @@ def _procedure_references(document, params):
"target_procedure": _procedure_identity(target_procedure) if target_procedure is not None else None,
"kind": _unavailable("Hopper's public Python API does not classify reference kinds"),
})
next_offset = offset + len(selected)
has_more = next_offset < len(ordered)
truncated = scan_truncated or has_more
return {
"procedure": _procedure_identity(procedure), "direction": direction,
"references": {"items": items, "total": None if scan_truncated else len(ordered), "returned": len(items), "truncated": truncated, "next_offset": next_offset if has_more and not scan_truncated else None},
"instructions_scanned": len(addresses), "instruction_scan_truncated": scan_truncated,
"references": items,
}
@@ -352,230 +369,39 @@ def _search_inventory(document, kind):
return inventory
def _checked_regex_paths(left, right, operation):
"""Apply one path-count operation without crossing the static work budget."""
if operation == "add":
exceeded = left > MAX_REGEX_BACKTRACKING_PATHS - right
result = left + right
else:
exceeded = right != 0 and left > MAX_REGEX_BACKTRACKING_PATHS // right
result = left * right
if exceeded or result > MAX_REGEX_BACKTRACKING_PATHS:
raise ValueError(
"Regex exceeds the %d-path backtracking budget"
% MAX_REGEX_BACKTRACKING_PATHS
)
return result
def _repeat_regex_paths(child_paths, minimum, maximum):
"""Count every bounded repetition path, including alternative child paths."""
paths = 0
repeated_paths = 1
for count in range(maximum + 1):
if count >= minimum:
paths = _checked_regex_paths(paths, repeated_paths, "add")
if count < maximum:
repeated_paths = _checked_regex_paths(
repeated_paths, child_paths, "multiply"
)
return paths
def _validate_regex_class(items):
"""Accept only constant-time character-class operations."""
allowed = {
sre_parse.CATEGORY,
sre_parse.LITERAL,
sre_parse.NEGATE,
sre_parse.RANGE,
}
if any(operation not in allowed for operation, _ in items):
raise ValueError("Regex operation is not supported by the bounded matcher")
def _validate_regex_node(node, inside_repeat=False):
"""Return capped path and step bounds for Python regex evaluation."""
leaf_operations = {
sre_parse.ANY,
sre_parse.AT,
sre_parse.CATEGORY,
sre_parse.LITERAL,
sre_parse.NOT_LITERAL,
}
forbidden = {
sre_parse.ASSERT,
sre_parse.ASSERT_NOT,
sre_parse.GROUPREF,
sre_parse.GROUPREF_EXISTS,
}
for name in ("GROUPREF_IGNORE", "GROUPREF_LOC_IGNORE", "GROUPREF_UNI_IGNORE"):
operation = getattr(sre_parse, name, None)
if operation is not None:
forbidden.add(operation)
repeat_tokens = {sre_parse.MAX_REPEAT, sre_parse.MIN_REPEAT}
possessive = getattr(sre_parse, "POSSESSIVE_REPEAT", None)
if possessive is not None:
repeat_tokens.add(possessive)
atomic = getattr(sre_parse, "ATOMIC_GROUP", None)
paths = 1
steps = 0
for operation, argument in node:
if operation in forbidden:
raise ValueError("Regex lookarounds and backreferences are not supported")
if operation in leaf_operations:
operation_paths = 1
operation_steps = 1
elif operation == sre_parse.IN:
_validate_regex_class(argument)
operation_paths = 1
operation_steps = 1
elif operation in repeat_tokens:
if inside_repeat:
raise ValueError("Nested regex repetitions are not supported")
minimum, maximum, child = argument
if maximum == sre_parse.MAXREPEAT or maximum > 1000:
raise ValueError(
"Unbounded or excessive regex repetitions are not supported"
)
child_paths, child_steps = _validate_regex_node(child, True)
operation_paths = _repeat_regex_paths(
child_paths, minimum, maximum
)
operation_steps = maximum * child_steps
elif operation == sre_parse.SUBPATTERN:
operation_paths, operation_steps = _validate_regex_node(
argument[-1], inside_repeat
)
elif operation == sre_parse.BRANCH:
operation_paths = 0
operation_steps = 0
for branch in argument[1]:
branch_paths, branch_steps = _validate_regex_node(
branch, inside_repeat
)
operation_paths = _checked_regex_paths(
operation_paths,
branch_paths,
"add",
)
operation_steps = max(operation_steps, branch_steps)
elif atomic is not None and operation == atomic:
operation_paths, operation_steps = _validate_regex_node(
argument, inside_repeat
)
else:
raise ValueError("Regex operation is not supported by the bounded matcher")
paths = _checked_regex_paths(paths, operation_paths, "multiply")
steps += operation_steps
return paths, steps
def _bounded_regex_matcher(expression, backtracking_paths, steps_per_path):
"""Create a matcher with per-candidate and cumulative work bounds."""
remaining_work = MAX_REGEX_SEARCH_WORK_UNITS
work_per_character = backtracking_paths * max(steps_per_path, 1)
def matches(value):
nonlocal remaining_work
if not isinstance(value, str):
raise ValueError("Regex candidates must be strings")
if len(value) > MAX_REGEX_CANDIDATE_LENGTH:
raise ValueError(
"Regex candidate exceeds the %d-character safety limit"
% MAX_REGEX_CANDIDATE_LENGTH
)
required_work = work_per_character * max(len(value), 1)
if required_work > remaining_work:
raise ValueError(
"Regex search exceeds the %d-unit work budget"
% MAX_REGEX_SEARCH_WORK_UNITS
)
remaining_work -= required_work
return expression.search(value) is not None
return matches
def _search_page(document, kind, params):
def _search_results(document, kind, params):
pattern = params.get("pattern")
if not isinstance(pattern, str) or not pattern or len(pattern) > MAX_SEARCH_PATTERN_LENGTH:
raise ValueError("pattern must contain between 1 and 256 characters")
if not isinstance(pattern, str) or not pattern:
raise ValueError("pattern must be a non-empty string")
mode = params.get("mode", "literal")
if mode not in ("literal", "regex"):
raise ValueError("mode must be literal or regex")
case_sensitive = params.get("case_sensitive", False)
if not isinstance(case_sensitive, bool):
raise ValueError("case_sensitive must be a boolean")
offset = params.get("offset", 0)
limit = params.get("limit", 100)
if not isinstance(offset, int) or isinstance(offset, bool) or offset < 0:
raise ValueError("offset must be a non-negative integer")
if not isinstance(limit, int) or isinstance(limit, bool) or limit < 1 or limit > 100:
raise ValueError("limit must be an integer between 1 and 100")
if mode == "literal":
needle = pattern if case_sensitive else pattern.casefold()
matches = lambda value: needle in (value if case_sensitive else value.casefold())
else:
try:
parsed = sre_parse.parse(pattern)
backtracking_paths, steps_per_path = _validate_regex_node(parsed)
expression = re.compile(pattern, 0 if case_sensitive else re.IGNORECASE)
except (re.error, OverflowError) as error:
raise ValueError("Invalid regex pattern") from error
matches = _bounded_regex_matcher(
expression, backtracking_paths, steps_per_path
)
matches = lambda value: expression.search(value) is not None
selected = []
total = 0
page_end = offset + limit
for item in _search_inventory(document, kind):
if not matches(item[1]):
continue
if offset <= total < page_end:
selected.append(item)
total += 1
next_offset = offset + len(selected)
has_more = next_offset < total
return {
"items": [
selected.append(item)
return [
{
"address": address,
"value": value[:MAX_SEARCH_VALUE_LENGTH],
"value_truncated": len(value) > MAX_SEARCH_VALUE_LENGTH,
"value": value,
}
for address, value in selected
],
"offset": offset,
"limit": limit,
"total": total,
"next_offset": next_offset if has_more else None,
"has_more": has_more,
}
def _page(values, offset, limit):
"""Return one deterministically address-sorted page without crossing an unbounded map."""
if not isinstance(offset, int) or isinstance(offset, bool) or offset < 0:
raise ValueError("offset must be a non-negative integer")
if not isinstance(limit, int) or isinstance(limit, bool) or limit < 1 or limit > 500:
raise ValueError("limit must be an integer between 1 and 500")
ordered = sorted(values.items(), key=lambda item: int(item[0], 16))
selected = ordered[offset:offset + limit]
total = len(ordered)
next_offset = offset + len(selected)
has_more = next_offset < total
return {
"items": [{"address": address, "value": value} for address, value in selected],
"offset": offset,
"limit": limit,
"total": total,
"next_offset": next_offset if has_more else None,
"has_more": has_more,
}
]
def _unavailable(reason):
@@ -590,16 +416,6 @@ def _offset(params, name):
return value
def _collection_offset(params, name):
values = params.get("collection_offset", {})
if not isinstance(values, dict):
raise ValueError("collection_offset must be an object")
value = values.get(name, 0)
if not isinstance(value, int) or isinstance(value, bool) or value < 0:
raise ValueError("collection_offset.%s must be a non-negative integer" % name)
return value
def _bounded(items, offset, limit, total=None, scan_truncated=False):
selected = items[offset:offset + limit]
known_total = len(items) if total is None and not scan_truncated else total
@@ -665,37 +481,18 @@ def _assembly(procedure, limit=None):
def _read_function_instructions(document, params):
"""Read one bounded raw-instruction window without invoking decompilation."""
"""Read all raw instructions without invoking decompilation."""
procedure = _procedure(document, params.get("procedure"))
offset = _offset(params, "offset")
limit = params.get("limit", 64)
if offset > 100000:
raise ValueError("offset must not exceed 100000")
if not isinstance(limit, int) or isinstance(limit, bool) or limit < 1 or limit > 500:
raise ValueError("limit must be an integer between 1 and 500")
addresses, scan_truncated = _instruction_addresses(
procedure, offset + limit + 1
)
start = min(offset, len(addresses))
end = min(len(addresses), start + limit)
addresses, _ = _instruction_addresses(procedure)
segment = procedure.getSegment()
items = []
for address in addresses[start:end]:
for address in addresses:
line = _render_instruction(segment, address)
if line is not None:
items.append(line)
has_more = scan_truncated or end < len(addresses)
return {
"procedure": _procedure_identity(procedure),
"instructions": {
"items": items,
"total": None if scan_truncated else len(addresses),
"returned": len(items),
"truncated": has_more,
"next_offset": end if has_more else None,
},
"instructions_scanned": len(addresses),
"instruction_scan_truncated": scan_truncated,
"instructions": items,
"limitations": [
"Instruction text and ordering are Hopper-specific representations.",
"The fast path does not decompile the procedure or scan whole-program names and strings.",
@@ -704,20 +501,9 @@ def _read_function_instructions(document, params):
def _analyze_function(document, params):
"""Collect a bounded, single-pass function dossier for agent callers."""
"""Collect the complete function dossier for agent callers."""
procedure = _procedure(document, params.get("procedure"))
limit = params.get("limit", 100)
max_chars = params.get("max_pseudocode_chars", 20000)
max_instructions = params.get("max_instructions", 500)
pseudocode_offset = _offset(params, "pseudocode_offset")
assembly_offset = _offset(params, "assembly_offset")
if not isinstance(limit, int) or isinstance(limit, bool) or limit < 1 or limit > 500:
raise ValueError("limit must be an integer between 1 and 500")
if not isinstance(max_chars, int) or isinstance(max_chars, bool) or max_chars < 1 or max_chars > 100000:
raise ValueError("max_pseudocode_chars must be between 1 and 100000")
if not isinstance(max_instructions, int) or isinstance(max_instructions, bool) or max_instructions < 1 or max_instructions > 5000:
raise ValueError("max_instructions must be between 1 and 5000")
addresses, instruction_scan_truncated = _instruction_addresses(procedure, max_instructions)
addresses, instruction_scan_truncated = _instruction_addresses(procedure)
blocks = []
all_blocks = list(procedure.basicBlockIterator())
for block in all_blocks:
@@ -732,7 +518,7 @@ def _analyze_function(document, params):
"successors": sorted(set(successors), key=lambda value: int(value, 16)),
})
pseudo = _pseudocode(document, procedure) or ""
assembly_lines = _assembly(procedure, max_instructions).splitlines() if params.get("include_assembly", False) else []
assembly_lines = _assembly(procedure).splitlines()
callers = sorted((_procedure_identity(item) for item in procedure.getAllCallerProcedures()), key=lambda item: int(item["address"], 16))
callees = sorted((_procedure_identity(item) for item in procedure.getAllCalleeProcedures()), key=lambda item: int(item["address"], 16))
comments = []
@@ -785,28 +571,17 @@ def _analyze_function(document, params):
comments.sort(key=lambda item: (int(item["address"], 16), item["kind"]))
referenced_strings.sort(key=lambda item: (int(item["address"], 16), int(item["source_address"], 16)))
referenced_names.sort(key=lambda item: (int(item["address"], 16), int(item["source_address"], 16)))
pseudo_text = pseudo[pseudocode_offset:pseudocode_offset + max_chars]
pseudo_next = pseudocode_offset + len(pseudo_text)
def collection(name, items, scan_limited=False):
return _bounded(items, _collection_offset(params, name), limit, None, scan_limited)
return {
"procedure": {"address": _hex(procedure.getEntryPoint()), "name": _procedure_name(procedure), "classification": None, "signature": procedure.signatureString(), "locals": _procedure_locals(procedure)},
"pseudocode": {"text": pseudo_text, "total_chars": len(pseudo), "returned_chars": len(pseudo_text), "truncated": pseudo_next < len(pseudo), "next_offset": pseudo_next if pseudo_next < len(pseudo) else None},
"assembly": _bounded(
assembly_lines,
assembly_offset,
max_instructions,
None,
instruction_scan_truncated if params.get("include_assembly", False) else False,
),
"comments": collection("comments", comments, instruction_scan_truncated),
"callers": collection("callers", callers), "callees": collection("callees", callees),
"incoming_references": collection("incoming_references", incoming, instruction_scan_truncated),
"outgoing_references": collection("outgoing_references", outgoing, instruction_scan_truncated),
"referenced_strings": collection("referenced_strings", referenced_strings, instruction_scan_truncated),
"referenced_names": collection("referenced_names", referenced_names, instruction_scan_truncated),
"basic_blocks": collection("basic_blocks", blocks),
"instruction_scan": {"scanned": len(addresses), "truncated": instruction_scan_truncated},
"pseudocode": pseudo,
"assembly": assembly_lines,
"comments": comments,
"callers": callers, "callees": callees,
"incoming_references": incoming,
"outgoing_references": outgoing,
"referenced_strings": referenced_strings,
"referenced_names": referenced_names,
"basic_blocks": blocks,
"limitations": [
"Hopper's public Python API does not classify reference kinds.",
"Hopper's public Python API does not expose equivalent external or thunk classification in this dossier.",
@@ -825,6 +600,13 @@ def _dispatch(method, params):
document = _session_document()
if document is None:
return {"shutdown": True, "analysis_stopped": True, "document_closed": True}
if method == "shutdown" and not REA_OWNS_PROCESS_LIFETIME:
return {
"shutdown": True,
"analysis_stopped": not document.backgroundProcessActive(),
"document_closed": False,
"document_retained": True,
}
if document.backgroundProcessActive():
document.requestBackgroundProcessStop()
if method == "shutdown" and REA_OWNS_PROCESS_LIFETIME:
@@ -968,24 +750,24 @@ def _dispatch(method, params):
})
return result
if method == "list_procedures":
return _page(dict(_search_inventory(document, "procedure")), params.get("offset", 0), params.get("limit", 100))
return [{"address": address, "value": value} for address, value in _search_inventory(document, "procedure")]
if method == "list_strings":
values = dict(_search_inventory(document, "string"))
requested = params.get("address")
if requested is not None:
key = _hex(_address(document, requested))
values = {key: values[key]} if key in values else {}
return _page(values, params.get("offset", 0), params.get("limit", 100))
return [{"address": _hex(address), "value": value} for address, value in sorted(values.items())]
if method == "list_names":
result = dict(_search_inventory(document, "name"))
requested = params.get("address")
if requested is not None:
key = _hex(_address(document, requested))
result = {key: result[key]} if key in result else {}
return _page(result, params.get("offset", 0), params.get("limit", 100))
return [{"address": _hex(address), "value": value} for address, value in sorted(result.items(), key=lambda item: int(item[0]))]
if method in ("search_procedures", "search_strings"):
kind = "procedure" if method == "search_procedures" else "string"
return _search_page(document, kind, params)
return _search_results(document, kind, params)
if method.startswith("procedure_"):
procedure = _procedure(document, params.get("procedure"))
if method == "procedure_address":
@@ -1147,6 +929,7 @@ def _run():
# Hopper's public objects are bound to its dedicated Python execution thread.
# Keep dispatch on that thread; moving calls to an arbitrary worker can deadlock.
# Keep dispatch on that thread; moving calls to a worker can deadlock.
if __name__ == "__main__":
_bind_session_document()
_run()
docs/adr/0001-provider-selection-and-analysis-profiles.md
+3 -5
View File
@@ -278,9 +278,7 @@ The adapter returns this provider-neutral envelope:
```json
{
"schema_version": 1,
"provider": { "id": "ghidra", "version": "<concrete version>" },
"provider_profile_schema_version": 1,
"parameters": {},
"digest": "<sha256 of RFC 8785 canonical JSON excluding digest>"
}
@@ -295,8 +293,8 @@ parameters as opaque data.
A selected deep binding requires a concrete provider version. If the adapter
cannot determine it, the candidate is unavailable with `version_unresolved`;
REA does not persist a cache that merely commits `null` or "whatever is
installed". Provider identity, provider-profile schema version, normalized
parameters, or provider version changes produce a different digest.
installed". Provider identity, normalized parameters, or provider version
changes produce a different digest.
The same target, configuration, and provider must resolve the same profile over
CLI and MCP. Profile resolution must be side-effect-free and complete before
@@ -321,7 +319,7 @@ A cached entry is reusable only when all of the following match exactly:
- target digest, generic format and kind, and selected architecture;
- provider ID and provider version;
- profile schema versions and profile digest;
- profile digest;
- operation and canonical parameters; and
- the existing cacheability and immutability rules.
docs/adr/0003-managed-code-evidence-and-provider-boundary.md
+1 -1
View File
@@ -189,7 +189,7 @@ or a CLR load. Complete-coverage claims require every applicable bounded region
to have been admitted; otherwise coverage is partial, unknown, or unavailable
with exact offsets and reasons.
The input path remains under REA's approved investigation roots. Local artifact
The caller selects the local input path directly. Local artifact
paths, digests, MVIDs, tokens, malformed offsets, and analysis metadata are
actionable diagnostics and are not secrets. Credentials, authorization data,
and genuine secrets remain redacted.
docs/browser-observation.md
+5 -1
View File
@@ -13,7 +13,11 @@ REA can attach to a user-owned Chrome-family browser through the Chrome DevTools
- `capture_web_screenshot` / `rea capture-web-screenshot` returns an explicitly approved, bounded, content-addressed visible-viewport PNG.
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
- Every surface has equivalent CLI and MCP contracts and returns Evidence v2 provenance.
- MCP session results are retained as `rea://evidence/{evidenceId}` resources. One-shot CLI output is not retained by a long-lived REA session.
- MCP tools return complete results inline and include the Evidence ID. Session evidence can be exported with `export_evidence_bundle`.
Target discovery returns the complete approved target array in one result. The
CDP endpoint's response-size ceiling remains internal; an oversized discovery
response fails explicitly instead of appearing complete.
Electron `file://` pages use a separate provider, permission capability, and root model; see [electron-observation.md](electron-observation.md).
Existing static application Evidence and passive web/Electron captures can be
docs/electron-observation.md
+4 -1
View File
@@ -5,7 +5,6 @@ REA can attach to a user-owned Electron/Chromium CDP endpoint and inspect existi
This passive runtime surface is distinct from the target-free static
[`analyze_javascript_application`](javascript-artifact-reconstruction.md)
workflow. Static analysis reads an approved ASAR or extracted directory under
`REA_INVESTIGATION_INPUT_ROOTS_JSON`; it does not attach to CDP. Runtime
observations and static inferences are never silently treated as the same fact.
Use the separate
[`reconcile_javascript_runtime`](javascript-runtime-reconciliation.md) workflow
@@ -25,6 +24,10 @@ Endpoints accept only explicit-port loopback HTTP URLs. Roots are canonicalized
## Workflow
`list_electron_targets` returns every target inside the approved roots in one
inline array. CDP response-size limits remain internal and oversized discovery
responses fail explicitly.
```bash
rea list-electron-targets http://127.0.0.1:9223 --approved --json
rea inspect-electron-page http://127.0.0.1:9223 TARGET_ID \
docs/installation.md
+9 -2
View File
@@ -149,6 +149,13 @@ and digest, validates the application bundle, and atomically installs it to
operator can choose its demo mode or activate an existing license. Homebrew and
administrator access are not used.
The Hopper launcher action used by REA creates a document from an executable;
its supported command-line interface does not attach to an already-open
document. While another live REA session owns the same target and loader
profile, a second session reports the owning run ID instead of opening a
duplicate document. Closing the owning REA session releases this guard; Hopper
keeps its document open.
On supported Linux distributions, approved setup verifies Hopper's official
`.deb`, `.rpm`, or Arch package before invoking the native package manager.
REA runs the supported demo build on a private Xvfb display and selects Hopper's
@@ -158,7 +165,7 @@ desktop display. Unattended package-manager access requires
mount, REA first verifies the conflict and then uses an unprivileged user and
mount namespace with a private mode-1777 tmpfs over that directory only. The
host mount and the rest of `/tmp` remain unchanged; this fallback never invokes
`sudo`. `rea doctor --provider hopper --detail full --json` reports the selected
`sudo`. `rea doctor --provider hopper --json` reports the selected
strategy and both host and effective mount facts.
### Hopper in CI
@@ -175,7 +182,7 @@ starting an unattended job. Ephemeral macOS runners therefore need a
pre-provisioned user session or a deliberate interactive bootstrap step.
When Hopper cannot start in CI, run
`rea doctor --provider hopper --detail full --json` in the failing runner.
`rea doctor --provider hopper --json` in the failing runner.
Structured failures distinguish private-display dependencies, an unsupported
demo dialog or build, process-ownership conflicts, and an early lifecycle exit.
Apply the reported remediation rather than exposing the runner's desktop,
docs/investigation-workspaces.md
+3 -7
View File
@@ -11,7 +11,6 @@ the write:
```bash
export REA_EVIDENCE_ROOTS_JSON='["/absolute/path/to/evidence"]'
export REA_INVESTIGATION_INPUT_ROOTS_JSON='["/absolute/path/to/releases"]'
rea investigate-versions \
/absolute/path/to/version-1 \
@@ -22,13 +21,11 @@ rea investigate-versions \
```
The two versions may be regular artifacts or directories supported by the
artifact graph provider. Both must resolve beneath an explicitly configured
`REA_INVESTIGATION_INPUT_ROOTS_JSON` root. The workspace parent must already
exist beneath `REA_EVIDENCE_ROOTS_JSON`. REA never creates an allowlisted root
implicitly.
artifact graph provider. Workspace files remain restricted by
`REA_EVIDENCE_ROOTS_JSON`; input paths are used directly by the local process.
Use `--expected-revision N` when an external coordinator needs an explicit
compare-and-swap guard. Traversal, byte, page, and comparison limits are also
compare-and-swap guard. Traversal, byte, page, and comparison controls are
available as CLI options.
After a completed run returns its stable run ID, `--replay-run-id RUN_ID`
@@ -104,7 +101,6 @@ digest. Parsing verifies:
Writes are canonical JSON, atomic, fsynced, and mode `0600`. REA rejects
workspace paths outside `REA_EVIDENCE_ROOTS_JSON`, symlink destinations,
artifact inputs outside `REA_INVESTIGATION_INPUT_ROOTS_JSON`,
oversized or deeply nested JSON, stale expected revisions, and concurrent lock
holders. Lock or CAS conflicts fail without mutating the workspace; callers may
retry the same idempotent request.
docs/javascript-application-graph.md
+8 -13
View File
@@ -1,6 +1,6 @@
# JavaScript Application Graph
JavaScript Application Graph v1 is REA's provider-neutral domain model for
JavaScript Application Graph is REA's provider-neutral domain model for
connecting shipped application bytes, JavaScript structure, Electron process
boundaries, passive browser observations, and native add-ons. It is a durable,
canonical data contract; it is not an extractor or a new CLI/MCP tool.
@@ -32,13 +32,12 @@ adds capture-scoped targets, frames, scripts, workers, and inferred
verification remains a separate later layer. The domain graph itself never
reaches outward to obtain any of those facts.
## Versioned record
## Stored record
A stored graph has this top-level shape:
```text
schema: "JavaScriptApplicationGraph"
schema_version: 1
graph_id: jag_<sha256 of normalized graph semantics>
root_node_ids: jag_node_<sha256>[]
nodes: application entities with one or more observations
@@ -47,14 +46,14 @@ coverage: complete, partial, unknown, or unavailable
limitations: explicit graph-wide constraints
```
Parsing rejects unknown fields, unsupported versions, non-canonical ordering,
duplicate IDs, stale semantic commitments, missing roots, dangling endpoints,
self-edges, and `changed_from` edges between unlike node kinds. Constructors
normalize set-like arrays before deriving their IDs.
Parsing rejects unknown fields, non-canonical ordering, duplicate IDs, stale
semantic commitments, missing roots, dangling endpoints, self-edges, and
`changed_from` edges between unlike node kinds. Constructors normalize set-like
arrays before deriving their IDs.
## Entities and relationships
The v1 node vocabulary covers:
The node vocabulary covers:
- package, installer, and artifact;
- ASAR entry;
@@ -160,7 +159,7 @@ so callers cannot preserve a stale ID after changing semantic content.
## Bounds
The v1 boundary admits at most 1,000 roots, 100,000 nodes, 200,000 edges, and 64
The graph boundary admits at most 1,000 roots, 100,000 nodes, 200,000 edges, and 64
observations per node. Each properties object is valid bounded JSON with at most
64 keys, depth 6, 512 structural nodes, and 4,096 characters per string. Arrays
of evidence links, limits, and limitations have independent caps.
@@ -184,7 +183,3 @@ The public domain functions are:
- `serializeJavaScriptApplicationGraph` for canonical JSON;
- `computeJavaScriptApplicationGraphSha256` for a byte-stable verified graph
digest.
Schema evolution is explicit. A future incompatible shape receives a new
`schema_version`; v1 parsing fails with an unsupported-version diagnostic rather
than guessing how to reinterpret it.
docs/javascript-application-workflows.md
+5 -5
View File
@@ -1,6 +1,6 @@
# Cross-layer JavaScript application workflows
REA derives a bounded feature trace from one authenticated JavaScript
REA derives a complete reachable feature trace from one authenticated JavaScript
Application Graph and compares two authenticated graph versions. The MCP tools
are `trace_application_feature`, `trace_javascript_semantics`,
`compare_application_versions`, `compare_source_to_bundle`, and
@@ -25,11 +25,11 @@ semantic data as an empty graph.
Select one literal seed kind: node ID, route, string, API, IPC channel, module,
or native export. Matching is exact or literal substring matching; regular
expressions and executable predicates are not accepted. Direction, depth,
nodes, edges, paths, and seed matches all have caller-visible limits.
expressions and executable predicates are not accepted. The trace includes all
matching seeds and the complete graph reachable in the selected direction.
The result contains the matching basis, a bounded subgraph, terminal paths,
authority summaries, truncation frontier, and native handoffs. A handoff binds
The result contains the matching basis, complete reachable subgraph, terminal
paths, authority summaries, and native handoffs. A handoff binds
the exact native artifact digest and requested exports from the application
graph. Existing Hopper or Ghidra Evidence is linked only when its subject digest
matches exactly. Otherwise the result recommends provider-neutral follow-up
docs/javascript-artifact-reconstruction.md
+7 -23
View File
@@ -17,15 +17,11 @@ separate static, runtime, and cross-layer inference authorities.
## Public workflow
The administrator first grants a canonical input root. Every request still
requires explicit per-call approval, and the requested ASAR or directory must
remain beneath that root after canonicalization:
The tool reads the caller-selected local directory or ASAR path directly:
```bash
export REA_INVESTIGATION_INPUT_ROOTS_JSON='["/absolute/path/to/apps"]'
rea analyze /absolute/path/to/apps/app.asar --approved --json
rea analyze /absolute/path/to/apps/app.asar --json
rea analyze-javascript-application /absolute/path/to/apps/app.asar \
--approved \
--json
```
@@ -35,26 +31,18 @@ command when setting explicit format or reconstruction limits. Native targets,
single JavaScript files, `.app` bundles, and explicit deep-provider or snapshot
requests retain the native deep-analysis route.
When configuring an MCP client, set the variable before approved setup so REA
copies the explicit non-secret root policy into the managed registration:
Configure an MCP client with the ordinary REA setup command:
```bash
export REA_INVESTIGATION_INPUT_ROOTS_JSON='["/absolute/path/to/apps"]'
rea setup
```
Restart the configured MCP client after changing this administrator ceiling.
An already-running server cannot acquire a new process environment through
MCP elicitation or `SIGHUP`.
The equivalent MCP input is:
```json
{
"input_path": "/absolute/path/to/apps/app.asar",
"format": "auto",
"approved": true,
"source_map_read_approved": false
"format": "auto"
}
```
@@ -90,7 +78,7 @@ Selected bounded text is then parsed as inert data to recover:
- static imports, dynamic imports, CommonJS `require` calls, workers, and
service workers;
- route, network endpoint, storage, and vendor-marker observations;
- local source-map declarations and, when separately approved, bounded original
- local source-map declarations and original
source names and content digests;
- explicit BrowserWindow options and `webPreferences`, including statically
resolvable preload entrypoints;
@@ -119,11 +107,7 @@ JavaScript file, package record, or source map produces an `unknown` graph scope
with `state: unavailable`; it is not treated as evidence that modules or source
are absent.
Source-map content has a separate `source_map_read_approved` input. Without that
approval, REA inventories the map, links a static declaration when present, and
records partial non-truncated coverage plus an unavailable parse scope. With
approval, the graph stores original source names and optional content digests,
not the raw `sourcesContent` text.
Source-map contents are parsed as local input. The graph stores original source names and optional content digests, not raw `sourcesContent` text.
Only explicitly present BrowserWindow values are observations. REA does not
substitute version-dependent Electron defaults for omitted `webPreferences`.
@@ -183,7 +167,7 @@ knowable, `omitted_count` is `null` rather than a guessed value.
The source-owned fixture covers an extracted directory, a direct ASAR with an
unpacked native add-on, a direct ASAR whose unpacked native companion is
missing, and an ASAR nested beneath a directory. Tests also cover deterministic
reruns, parse-not-execute behavior, source-map approval, global source-map
reruns, parse-not-execute behavior, source-map parsing, global source-map
limits, malformed structured data, invalid containers, traversal, symlink
escape, oversized text, cancellation, AST truncation, and repeated content
identities. A separate synthetic Electron fixture covers explicit safe and
docs/javascript-runtime-observation.md
+4
View File
@@ -31,6 +31,10 @@ origin scope. Targets already marked attached are rejected rather than
displacing another debugger. Canonical file paths are checked after symlink resolution.
Excluded locations are counted but never retained.
`list_javascript_runtime_targets` returns every approved target in one inline
array. The Inspector discovery response-size ceiling remains internal; an
oversized response fails explicitly.
## Passive protocol boundary
The provider sends exactly two protocol commands:
docs/mcp-contracts.md
+9 -55
View File
@@ -2,14 +2,11 @@
## Identity and discovery
`rea://server/identity` and `binary_session` report the package, server, SDK,
negotiated protocol, skill, and schema-sensitive catalog identities separately.
An absent live comparison is `unknown`, never aligned. Callers can supply the
expected package version, catalog digest, or registered server path to
`binary_session`; a mismatch means the registered MCP process must be restarted.
`rea doctor` separately inspects supported JSON/TOML client registrations,
reports their command vectors as aligned, stale, missing, or invalid, and keeps
live-server state `unknown` unless the active connection supplies identity.
`binary_session` reports the active package, server, SDK, and negotiated
protocol details. `rea doctor` separately inspects supported JSON/TOML client
registrations, reports their command vectors as aligned, stale, missing, or
invalid, and keeps live-server state `unknown` unless the active connection
supplies identity.
Canonical tool names remain stable, while `tools/list` advertises only the
operations callable for the current target, provider, policy, host, and
@@ -58,49 +55,10 @@ CLI calls work without a progress token and translate SIGINT into the same
AbortSignal used by providers. Existing controlled-process cleanup and provider
shutdown rules still apply; REA never kills a process it cannot prove it owns.
## Evidence resources
## Tool results
Evidence-producing tools return compact summaries plus `resource_link` content.
Every link is accompanied by an ordinary instruction to copy its opaque URI
unchanged and call MCP `resources/read` (Codex: `read_mcp_resource`). Session
resources are connection-local:
- `rea://evidence/{evidenceId}` returns the complete immutable Evidence v2 record.
- `rea://evidence/{evidenceId}/section/{section}` returns a bounded result section.
Stable sections include `result`, `terminal`, `filesystem`, `process`,
`protocol`, `nodes`, `occurrences`, and `edges` when present.
- `rea://unknown/{unknownId}` returns the current residual-unknown head and its
immutable revision history.
- `rea://evidence-bundle/{bundleDigest}` returns immutable canonical bytes
retained by `snapshot_evidence_bundle`.
- `rea://snapshot/current` returns the mutable native analysis cache as an
`available` or `unavailable` state.
- `rea://artifact/{manifestId}/{collection}` returns a canonical artifact
`nodes`, `occurrences`, or `edges` page with Evidence provenance.
- `rea://function/{targetSha256}/{address}` returns a retained function dossier
for the exact target and address.
- `rea://workspace/{workspaceId}/revision/{revision}` returns an immutable,
CAS-linked investigation workspace revision retained by this session.
Successful automatic cross-version investigations include both Evidence and
workspace `resource_link` blocks. Workspace resources preserve revision and
`previous_revision_digest` commitments; persistent workspace files remain
subject to configured read/write roots.
Retained bundle URIs are session-scoped and are invalidated by
`release_evidence_bundle` or session close. When retention reaches its bounded
capacity, release an unneeded digest and retry; for cross-session recovery use
`export_evidence_bundle` followed by `import_evidence_bundle` and
`snapshot_evidence_bundle` in the new session.
`export_evidence_bundle` is file-only and requires `path`; existing files
require `overwrite: true`. Use `snapshot_evidence_bundle` and then
`resources/read` when the complete bundle is needed within the current session.
An Evidence URI is discoverability, not authorization. It cannot authorize file
access, extraction, mounting, execution, or networking. IDs disappear when the
session ends unless the existing Evidence bundle or workspace persistence flow
explicitly retains them.
Evidence-producing tools return their result and Evidence context inline. A
workflow can retrieve retained Evidence records with `get_evidence_bundle`.
## Aggregate native context
@@ -162,11 +120,7 @@ administrator ceiling. A request outside that ceiling reports
`elicitation_supported: false` and `restart_required: true`; interactive consent
cannot silently widen administrator policy.
`analyze_javascript_application` uses the `investigation_input` capability. Its
absolute `input_path` must be inside `REA_INVESTIGATION_INPUT_ROOTS_JSON` and the
request must set `approved: true` before any artifact read. Reading source-map
contents additionally requires `source_map_read_approved: true`; ordinary input
approval does not imply that separate authority.
`analyze_javascript_application` reads the selected local directory or ASAR directly. It has no per-call approval flag or configured investigation root.
## Integrity record-and-continue
docs/mcp-prompts.md
+9 -9
View File
@@ -6,14 +6,14 @@ invoke tools by themselves.
## Prompt inventory
| Prompt | Purpose |
| --------------------------------- | ------------------------------------------------------------------------------------- |
| `investigate_feature` | Trace a feature from discovery into bounded function evidence. |
| `compare_application_versions` | Compare two shipped artifacts, with optional static and runtime follow-up. |
| `verify_reconstruction` | Evaluate a finite reconstruction specification against retained comparison Evidence. |
| `trace_crash` | Correlate a crash symptom with static paths and optional Process Capture v4 evidence. |
| `audit_residual_unknowns` | Audit current residual-unknown heads and evidence-qualified resolution. |
| `prepare_bounded_process_capture` | Design an approval-gated, bounded process experiment before execution. |
| Prompt | Purpose |
| --------------------------------- | ------------------------------------------------------------------------------------ |
| `investigate_feature` | Trace a feature from discovery into bounded function evidence. |
| `compare_application_versions` | Compare two shipped artifacts, with optional static and runtime follow-up. |
| `verify_reconstruction` | Evaluate a finite reconstruction specification against retained comparison Evidence. |
| `trace_crash` | Correlate a crash symptom with static paths and optional process capture evidence. |
| `audit_residual_unknowns` | Audit current residual-unknown heads and evidence-qualified resolution. |
| `prepare_bounded_process_capture` | Design an approval-gated, bounded process experiment before execution. |
Every rendered prompt provides an ordered list of current REA tool names. It
also requires the agent to keep observations, inferences, and unknowns
@@ -84,7 +84,7 @@ Completion sources are live projections of the current session:
| Procedure | Bounded pagination over `list_procedures`; observed addresses are offered from partial results, while names require exhaustive discovery and a unique address. Optional document context is forwarded. |
| Provider | `auto` plus sorted deep candidates from the current binary session; target-free completion does not start them. |
| Evidence | Evidence IDs retained in the current session ledger. |
| Process capture | Evidence IDs whose operation and validated result identify Process Capture v4. |
| Process capture | Evidence IDs whose operation and validated result identify process capture. |
| Artifact manifest | Manifest IDs from schema-valid retained artifact inventories. |
| Artifact occurrence | Occurrence IDs present in schema-valid retained artifact inventory pages. |
| Residual unknown | Current non-resolved unknown heads only. |
docs/process-capture.md
+3 -3
View File
@@ -1,4 +1,4 @@
# Process Capture v4
# Process capture
Process Capture records one approved command as deterministic Evidence. It is
intended for authority-versus-reconstruction checks where terminal behavior,
@@ -11,7 +11,7 @@ HTTP/WebSocket exchanges, and process exit ownership. Missing or bounded
observations remain explicit; a truncated capture is never treated as
equivalent to another capture.
Every v4 capture includes a run manifest with canonical SHA-256 commitments for
Every capture includes a run manifest with canonical SHA-256 commitments for
the secret-safe full scenario projection, comparison contract, executable,
normalization rules, command-shim plan, and replay plan. The manifest also
records the REA/provider versions, platform, architecture, PTY backend, and UTC
@@ -60,7 +60,7 @@ not a security sandbox: the target runs with the current user's permissions.
## Capture a scenario
`rea capture-process` reads a JSON scenario and writes Process Capture v4
`rea capture-process` reads a JSON scenario and writes process capture
Evidence to stdout:
```bash
docs/product-catalog.json
+8 -175
View File
@@ -1,5 +1,4 @@
{
"catalog_schema_version": 1,
"package": {
"name": "rea-agents",
"version": "3.1.0",
@@ -11,7 +10,7 @@
"skill_version": "23"
},
"tools": {
"total": 116,
"total": 115,
"families": [
{
"id": "direct",
@@ -168,7 +167,7 @@
{
"id": "session",
"surface": "session",
"count": 23,
"count": 22,
"tools": [
"binary_session",
"build_call_path",
@@ -181,15 +180,14 @@
"correlate_static_and_runtime",
"export_evidence_bundle",
"find_changed_behavior",
"get_evidence_bundle",
"get_navigation_context",
"import_evidence_bundle",
"inspect_address_context",
"list_unknowns",
"open_binary",
"record_unknown",
"release_evidence_bundle",
"run_replay_machine",
"snapshot_evidence_bundle",
"update_unknown",
"verify_reconstruction",
"verify_unknown_resolution"
@@ -430,168 +428,6 @@
"configuration": "detect-only"
}
],
"schemas": [
{
"id": "analysis_snapshot",
"version": 2
},
{
"id": "artifact_extraction",
"version": 1
},
{
"id": "artifact_graph",
"version": 1
},
{
"id": "browser_scenario",
"version": 1
},
{
"id": "browser_scenario_capture",
"version": 1
},
{
"id": "browser_scenario_diff",
"version": 1
},
{
"id": "browser_target_list",
"version": 1
},
{
"id": "electron_page_inspection",
"version": 1
},
{
"id": "electron_target_list",
"version": 1
},
{
"id": "evidence",
"version": 2
},
{
"id": "evidence_bundle",
"version": 2
},
{
"id": "evidence_completion_ledger",
"version": 1
},
{
"id": "evidence_completion_manifest",
"version": 1
},
{
"id": "investigation_run",
"version": 1
},
{
"id": "investigation_run_summary",
"version": 1
},
{
"id": "investigation_workspace",
"version": 1
},
{
"id": "javascript_application_analysis",
"version": 2
},
{
"id": "javascript_application_graph",
"version": 1
},
{
"id": "javascript_application_version_comparison",
"version": 1
},
{
"id": "javascript_feature_trace",
"version": 1
},
{
"id": "javascript_runtime_observation",
"version": 1
},
{
"id": "javascript_runtime_target_list",
"version": 1
},
{
"id": "javascript_semantic_graph",
"version": 1
},
{
"id": "javascript_semantic_query",
"version": 1
},
{
"id": "javascript_semantic_trace",
"version": 1
},
{
"id": "managed_artifact_inspection",
"version": 1
},
{
"id": "managed_member_comparison",
"version": 1
},
{
"id": "managed_member_inspection",
"version": 1
},
{
"id": "managed_native_boundary_inspection",
"version": 1
},
{
"id": "managed_native_verification",
"version": 1
},
{
"id": "process_capture",
"version": 4
},
{
"id": "reconstruction_verification",
"version": 1
},
{
"id": "residual_unknown",
"version": 1
},
{
"id": "web_bundle_analysis",
"version": 1
},
{
"id": "web_capture_diff",
"version": 1
},
{
"id": "web_mcp_discovery",
"version": 1
},
{
"id": "web_observation_session",
"version": 1
},
{
"id": "web_page_inspection",
"version": 2
},
{
"id": "web_screenshot",
"version": 1
},
{
"id": "web_screenshot_diff",
"version": 1
}
],
"cli": {
"primary_count": 68,
"commands": [
@@ -674,16 +510,13 @@
"runtime_catalog": {
"counts": {
"cli_commands": 68,
"mcp_tools": 116,
"mcp_prompts": 6,
"mcp_resources": 2,
"mcp_resource_templates": 11
"mcp_tools": 115,
"mcp_prompts": 6
},
"digests": {
"tools_sha256": "b21e7a4794bf82b27931a373a7812978b54f136904a88681094f2c545e4413a6",
"prompts_sha256": "3886a3bf55a448def79e3eb429e6a56eb2ba075423c5cb1e5747642b389ef8fe",
"resources_sha256": "e0e872276ebf6f88e916ef24930a99eed27ce4c93e494e60820e9bd8cc8497c3",
"combined_sha256": "6624247db5bcf7b52ce53173bbc0ac11bd51e9217e9e3be4298b755bcc60ea50",
"tools_sha256": "4722f1a2fe1d16d493e4a1f51450376e2cd0c617b9659d292a61908d8bb05fcf",
"prompts_sha256": "77c76df62f92484a43cebf5ca110bdf562c38fbee2531d293eeb51d6e712d459",
"combined_sha256": "94f7cd1ac629a6bbffa77ba3b3d3f2e208e15df821bc0b14db4f81692d6f8eaa",
"providers_sha256": "7446b51788734ebf6f4f702e3a5a8c9f0b61967490c245f30bd37b2060827e6b"
}
}
docs/provider-evaluation.md
+1 -1
View File
@@ -91,7 +91,7 @@ capability from a successful import.
| Procedures | Both non-external and external functions are listed. A local thunk remains distinct from its resolved target; exact and qualified name lookup fails on ambiguity rather than guessing. |
| Strings | Only Ghidra-defined string `Data` is observed. Items report charset, byte length, and whether a required null terminator is missing. The API cannot distinguish a present terminator from fixed/Pascal layouts when no terminator is missing, so that state is named `present_or_not_required`. |
| Memory | Memory-block end addresses are exclusive. Read/write/execute, initialization, overlay, address space, and image base are direct Ghidra observations. |
| Pagination | List limits are 500 and search limits are 100. Pages commit exact totals and advancing offsets; an inventory above one million items fails rather than returning a partial result labeled exhaustive. |
| Inventory | Procedure, symbol, and string listings plus searches return the complete matching collection in one response; callers do not provide offsets or limits. The bridge still enforces the declared transport response ceiling. |
| Search | Literal search scans the immutable inventory with a 1,000,000-unit cumulative work budget. Regex uses a conservative finite Java-regex subset with 10,000 static paths, 4,096 UTF-16 code units per candidate, and the same cumulative budget. Exceeding a budget fails explicitly; returned values identify truncation. |
| Analysis state | The socket is exposed only after default auto-analysis. A 300-second analysis timeout fails target opening; ordinary established requests have a 10-second client deadline and every response has a 1 MiB ceiling. |
docs/reconstruction-obligation-ledgers.md
+3 -7
View File
@@ -1,13 +1,11 @@
# Reconstruction obligation ledgers
`build_reconstruction_obligation_ledger` turns authenticated Evidence v2 records
into a versioned, deterministic list of reconstruction claims. The same workflow
`build_reconstruction_obligation_ledger` turns authenticated Evidence records
into a deterministic list of reconstruction claims. The same workflow
is available as:
- MCP tool: `build_reconstruction_obligation_ledger`
- CLI command: `rea build-reconstruction-obligation-ledger`
- MCP resource:
`rea://evidence/{evidenceId}/reconstruction-obligation-ledger`
The ledger is conservative by design. Static Application Graph facts create
candidate obligations; they do not prove runtime or process behavior. A required
@@ -29,7 +27,6 @@ never claims closure:
rea build-reconstruction-obligation-ledger \
'{
"evidence_bundle": {
"bundle_version": 2,
"artifacts": [],
"providers": [],
"environments": [],
@@ -40,7 +37,6 @@ rea build-reconstruction-obligation-ledger \
},
"reviewed_obligations": [],
"manifest": {
"schema_version": 1,
"bindings": [],
"contradictions": []
},
@@ -51,7 +47,7 @@ rea build-reconstruction-obligation-ledger \
```
For a real investigation, export an Evidence bundle from the session and replace
the empty bundle. Application Graph and Process Capture v4 records generate
the empty bundle. Application Graph and process capture records generate
conservative candidates automatically. Add reviewed obligations when a required
claim is not represented by those sources.
docs/reconstruction-readiness.md
+5 -11
View File
@@ -10,10 +10,10 @@ npm run verify:readiness
The command builds the redistributable native conformance fixtures, inventories
the native executable through the public CLI, analyzes the generated JavaScript
CLI and Electron fixture through both CLI and MCP, verifies exact advertised
limits, evaluates the same nine-stage report through both surfaces, reads the
full MCP report resource, and proves stale/tampered replay rejection. It prints
one compact JSON summary.
CLI and Electron fixture through both CLI and MCP, evaluates the same nine-stage
report through both surfaces, verifies that MCP returns the complete report
inline, and proves stale/tampered replay rejection. It prints one compact JSON
summary.
Set `REA_READINESS_REPORT_PATH` to retain the complete machine-readable report:
@@ -28,13 +28,7 @@ full report as the `reconstruction-readiness` artifact.
- CLI: `rea evaluate-reconstruction-readiness <json-or-file> --json`
- MCP tool: `evaluate_reconstruction_readiness`
- Full MCP resource:
`rea://evidence/{evidenceId}/reconstruction-readiness-report`
The MCP tool result contains only report identity, status, summary, metrics, and
the resource URI. The full Evidence bundle, stage records, comparisons,
contradictions, obligation ledger, and replay inputs stay in the retained
resource instead of consuming the tool-response context.
The MCP tool returns the complete report inline, including the Evidence bundle, stage records, comparisons, contradictions, obligation ledger, and replay inputs.
## Required stages
docs/user-visible-errors.csv
+1 -2
View File
@@ -73,8 +73,7 @@ harm_rank,location,trigger,current_copy,user_risk,proposed_replacement,implement
62,src/application/ProcessCli.ts:readJson size,Process scenario or evidence file exceeds 64 MiB,Process JSON input exceeds byte limit,Does not state limit or recovery,"Process input file is too large. Reduce it below 64 MiB, then try again.",implemented,verified by focused sparse-file CLI test
63,src/application/ProcessCli.ts:readJson parse,Process scenario or evidence file is malformed JSON,Raw JSON parser exception through CLI framework,Can expose parser implementation and lacks recovery,"Process input file is not valid JSON. Repair the file, then try again.",implemented,verified by focused CLI test
64,src/application/ProcessCli.ts:capture scenario parse,Scenario fails Process Capture schema,Raw Zod validation exception through CLI framework,Exposes schema internals and field paths without stable recovery,"Process scenario is invalid. Check its required fields and limits, then try again.",implemented,verified by focused CLI test
65,src/application/ProcessCli.ts:legacy evidence,Comparison receives Process Capture v3 Evidence,"Process Capture v3 is unsupported. Re-run the scenario with this REA version using capture_process_scenario to produce Process Capture v4; captures cannot be upgraded because v4 requires new manifest and settlement evidence.",Long but accurately explains why recapture is required and gives exact tool recovery,Keep current copy,unchanged,verified by focused CLI test
66,src/application/ProcessCli.ts:wrong evidence,Comparison receives valid Evidence from another workflow,Expected REA Process Capture v4 Evidence,Uses internal schema name and does not explain recovery,"Capture evidence is not from the current process-capture workflow. Create new capture evidence, then try again.",implemented,verified by focused CLI test
66,src/application/ProcessCli.ts:wrong evidence,Comparison receives valid Evidence from another workflow,Expected REA process capture Evidence,Uses internal schema name and does not explain recovery,"Capture evidence is not from the current process-capture workflow. Create new capture evidence, then try again.",implemented,verified by focused CLI test
67,src/application/ProcessCli.ts:unexpected,Unexpected parser/comparison/canonicalization failure escapes expected classifications,Raw thrown exception through CLI framework,Can expose stack or internal invariant,"Process command could not complete. Check the input files and run `rea doctor`, then try again.",implemented,verified by direct unexpected-cause projection test
68,src/server/registerEvidenceTools.ts:object guard,Validated provider-tool input is unexpectedly not an object,Provider tool input was not an object,Exposes provider architecture and implies caller can fix impossible state,Remove the impossible duplicate guard and parse through the declared JSON-object schema.,removed,verified by typecheck and MCP integration tests
69,src/server/registerEnhancedTools.ts:object guard,Validated enhanced-tool input is unexpectedly not an object,Validated enhanced input was not a JSON object,Exposes validation/JSON implementation detail,Remove the impossible duplicate guard and parse through the declared JSON-object schema.,removed,verified by typecheck and enhanced MCP integration tests
1 harm_rank location trigger current_copy user_risk proposed_replacement implementation_status verification_result
73 62 src/application/ProcessCli.ts:readJson size Process scenario or evidence file exceeds 64 MiB Process JSON input exceeds byte limit Does not state limit or recovery Process input file is too large. Reduce it below 64 MiB, then try again. implemented verified by focused sparse-file CLI test
74 63 src/application/ProcessCli.ts:readJson parse Process scenario or evidence file is malformed JSON Raw JSON parser exception through CLI framework Can expose parser implementation and lacks recovery Process input file is not valid JSON. Repair the file, then try again. implemented verified by focused CLI test
75 64 src/application/ProcessCli.ts:capture scenario parse Scenario fails Process Capture schema Raw Zod validation exception through CLI framework Exposes schema internals and field paths without stable recovery Process scenario is invalid. Check its required fields and limits, then try again. implemented verified by focused CLI test
76 65 66 src/application/ProcessCli.ts:legacy evidence src/application/ProcessCli.ts:wrong evidence Comparison receives Process Capture v3 Evidence Comparison receives valid Evidence from another workflow Process Capture v3 is unsupported. Re-run the scenario with this REA version using capture_process_scenario to produce Process Capture v4; captures cannot be upgraded because v4 requires new manifest and settlement evidence. Expected REA process capture Evidence Long but accurately explains why recapture is required and gives exact tool recovery Uses internal schema name and does not explain recovery Keep current copy Capture evidence is not from the current process-capture workflow. Create new capture evidence, then try again. unchanged implemented verified by focused CLI test
66 src/application/ProcessCli.ts:wrong evidence Comparison receives valid Evidence from another workflow Expected REA Process Capture v4 Evidence Uses internal schema name and does not explain recovery Capture evidence is not from the current process-capture workflow. Create new capture evidence, then try again. implemented verified by focused CLI test
77 67 src/application/ProcessCli.ts:unexpected Unexpected parser/comparison/canonicalization failure escapes expected classifications Raw thrown exception through CLI framework Can expose stack or internal invariant Process command could not complete. Check the input files and run `rea doctor`, then try again. implemented verified by direct unexpected-cause projection test
78 68 src/server/registerEvidenceTools.ts:object guard Validated provider-tool input is unexpectedly not an object Provider tool input was not an object Exposes provider architecture and implies caller can fix impossible state Remove the impossible duplicate guard and parse through the declared JSON-object schema. removed verified by typecheck and MCP integration tests
79 69 src/server/registerEnhancedTools.ts:object guard Validated enhanced-tool input is unexpectedly not an object Validated enhanced input was not a JSON object Exposes validation/JSON implementation detail Remove the impossible duplicate guard and parse through the declared JSON-object schema. removed verified by typecheck and enhanced MCP integration tests
docs/verification/managed-conformance-ledger.json
+15 -15
View File
@@ -1,103 +1,103 @@
{
"schema_version": 1,
"ledger_id": "ecl_a56a75b91375c9c9501d1545fee9977e74f57dbd1bf413a370f9cd1ed26987bd",
"ledger_id": "ecl_2d5c1c83c1fd56615a663bcb3a432b7023afa316133b744c64dcaa659cab53c0",
"records": [
{
"claim_id": "managed.application-graph-projection",
"status": "pass",
"evidence_ids": [
"ev_4d6821cdd212a77507dfbdf995fc8b086cd63b3f1d39dee4a2e12f62a6f0609d"
"ev_0c73f33f51048a7c7cc5d908cb951512ee9f3d9dcf4124f0f0eb2c4a88ceb7cc"
]
},
{
"claim_id": "managed.decompiler-reconstruction-import",
"status": "pass",
"evidence_ids": [
"ev_cd9015f39a60251bee1b01baf5189de7427e5810f130aed39770f17ae69cee44"
"ev_0504a7fc4cc8f77ca07a3aa59d62830b5769c7b41e7518677771fe7aa54c7b5d"
]
},
{
"claim_id": "managed.dotnet-framework-x86-pinvoke",
"status": "pass",
"evidence_ids": [
"ev_af02c0218a01138bd1f0f5ca052f95d348dfacb540040ab72d6ddc9889e3a7e4"
"ev_9b63cdeef9d48e567dc8f4de9fe6b424e94929ac1f17aafd0ef1807775f72d96"
]
},
{
"claim_id": "managed.ilspy-reconstruction-oracle",
"status": "unsupported",
"evidence_ids": [
"ev_5df12074aeb26d67bc954fd9230076bd0c3ce9969600e19c59ba02090a9988f2"
"ev_4e3fbc7203c29aab5d9119332a09c14e285e85007b3d7fc7433880b39a507435"
]
},
{
"claim_id": "managed.malformed-metadata",
"status": "pass",
"evidence_ids": [
"ev_5d6aadf199ae090e0ddebc10a51aab8a410fa778a6ae8454fedfb10d240cdac2"
"ev_dbd885adc65f9cc8abacb04642db609d2180b0af71adc0b0732719f06a057063"
]
},
{
"claim_id": "managed.managed-native-verification",
"status": "pass",
"evidence_ids": [
"ev_9f1862276fe8869b86105eb91154603922eeb95fec33a10c354b2d6c9c3fe077"
"ev_e7841bf63d7b83804987d21f4f432752870ff7338862360bbcf54653eb782731"
]
},
{
"claim_id": "managed.modern-dotnet-anycpu",
"status": "pass",
"evidence_ids": [
"ev_c9aef74724cf555bcedce8a9dbf46327b2f92b95b03eb363f966885d429e6882"
"ev_9593b026617002c64d0618ec079d42dbc0ddeef7cd53f2f8b2c8fca9ffebb2a9"
]
},
{
"claim_id": "managed.mvid-and-token-drift",
"status": "pass",
"evidence_ids": [
"ev_55808bf3ee5ec9d87cee751462077a6bce88701c4fcf66822ba1d7d33706706b"
"ev_ba32694a35810acd211745efdf41c88887f4931efb6027e876eea3e8e506aee0"
]
},
{
"claim_id": "managed.not-managed",
"status": "pass",
"evidence_ids": [
"ev_a8166506ca91a4595b825f355836bb4becf35852c8547c55408c09698f4953a6"
"ev_ac700cf6c8efdda7f689b29a9c3d5a679ecb24893f93bb1c76e50e23b2b9dcbb"
]
},
{
"claim_id": "managed.operator-local-manifest",
"status": "unsupported",
"evidence_ids": [
"ev_03d1a151446834fbee7c093e82121b714a0cc1e6167ad84f47a79ffb9e2602ce"
"ev_f96f07bc28f2e6da5a71597f9a38f5eaa7cde0217f0df10d6ef6d655937fc1a2"
]
},
{
"claim_id": "managed.operator-manifest-self-test",
"status": "pass",
"evidence_ids": [
"ev_74d7f9c75a4476196664542d790e2532435f2a1c54349b481967e51e2ccc9453"
"ev_ed0f3ff1749f0927d493609423a366ec6a432f016c767aeeedd9b4a520672eaf"
]
},
{
"claim_id": "managed.runtime-correlation-admission-plan",
"status": "pass",
"evidence_ids": [
"ev_5a80df0b92dc2280361e900ce590bc88ea52eba3dc8e216a0ce282f503e3cb05"
"ev_24a622e16a885e2d21025f51c3f4b5900719c21d9f0d04ba1d55f6093a702c20"
]
},
{
"claim_id": "managed.unicode-obfuscated-identifiers",
"status": "pass",
"evidence_ids": [
"ev_3b61c79ec152c70693278a70ad2ca6528ce0a9c9ab389d3ec9e4520de79082a7"
"ev_2e9bc5213917635c3eb3d094449adf088c450c07d078dcb26312182c04966bc9"
]
},
{
"claim_id": "managed.x64-ready-to-run-native-body",
"status": "pass",
"evidence_ids": [
"ev_81f41b1b3809c2fe1964f7abbcbd2faeb14c723938639c2e0bb16cd58746aa1a"
"ev_bbf3dd8daec6d46d16ac7befc3875572e7103b27b815f81903e3dca61880c70b"
]
}
],
docs/verification/managed-conformance-manifest.json
+2 -2
View File
@@ -1,6 +1,6 @@
{
"schema_version": 1,
"manifest_id": "ecm_1b79ae0f4063ccac065a4eff5dec420f968ea39bc4d93eb3bddcaaf318eb01a6",
"manifest_id": "ecm_3a42d32a96315338e06700b64afc0367603e2c5050b1ec0bf1ce012e227b4df3",
"verifier": {
"id": "rea-managed-conformance",
"version": "1"
@@ -14,7 +14,7 @@
"skill_digests": [
{
"skill_id": "reverse-engineer-anything",
"sha256": "709a1ace114ea7ffa6f36a67c2f2df9ffa1029c2cad8508aebf40c5105862643"
"sha256": "89b09782340415790ecf161a5bc33f60e1a23cd66e0f4c29dd26e959b2baa0a9"
}
],
"claims": [
scripts/build-conformance-fixtures.mjs
+2 -2
View File
@@ -103,9 +103,9 @@ async function buildPortableFixtures() {
join(outputRoot, "large"),
],
expectations: {
symbolPrefix: `${symbolPrefix}rea_page_`,
symbolPrefix: `${symbolPrefix}rea_inventory_`,
symbolCount: LARGE_FIXTURE_COUNT,
stringPrefix: "REA_PAGE_",
stringPrefix: "REA_INVENTORY_",
stringCount: LARGE_FIXTURE_COUNT,
},
});
scripts/lib/catalog-schemas.mjs
-264
View File
@@ -1,264 +0,0 @@
import { z } from "zod";
const valuesAt = (node, path) => {
if (typeof node !== "object" || node === null) return [];
if ("anyOf" in node && Array.isArray(node.anyOf))
return node.anyOf.flatMap((alternative) => valuesAt(alternative, path));
if ("oneOf" in node && Array.isArray(node.oneOf))
return node.oneOf.flatMap((alternative) => valuesAt(alternative, path));
if (path.length === 0)
return "const" in node &&
(typeof node.const === "number" || typeof node.const === "string")
? [node.const]
: [];
if (
!("properties" in node) ||
typeof node.properties !== "object" ||
node.properties === null
)
return [];
const [segment, ...remaining] = path;
return segment !== undefined && segment in node.properties
? valuesAt(node.properties[segment], remaining)
: [];
};
const versionAt = (schema, path) => {
const versions = [
...new Set(
valuesAt(z.toJSONSchema(schema, { unrepresentable: "any" }), path).map(
String,
),
),
];
if (versions.length !== 1)
throw new Error(
`Schema version path ${path.join(".")} has ${String(versions.length)} constant values`,
);
const version = versions[0];
if (version === undefined) throw new Error("Schema version disappeared");
return /^\d+$/u.test(version) ? Number(version) : version;
};
const durableSchemaDefinitions = (sources) => [
[
"analysis_snapshot",
sources.analysisSnapshot.analysisSnapshotSchema,
["snapshot_version"],
],
[
"artifact_graph",
sources.artifactGraph.artifactInventoryResultSchema,
["manifest", "schema_version"],
],
[
"artifact_extraction",
sources.artifactGraph.artifactExtractionResultSchema,
["extraction_manifest", "schema_version"],
],
[
"browser_scenario",
sources.browserScenario.browserScenarioSchema,
["schema_version"],
],
["evidence", sources.evidence.evidenceSchema, ["schema_version"]],
[
"evidence_bundle",
sources.evidenceBundle.evidenceBundleSchema,
["bundle_version"],
],
[
"evidence_completion_ledger",
sources.evidenceCompletion.evidenceCompletionLedgerSchema,
["schema_version"],
],
[
"evidence_completion_manifest",
sources.completionGeneration.completionManifestSchema,
["schema_version"],
],
[
"investigation_run",
sources.investigationWorkspace.investigationRunSchema,
["schema_version"],
],
[
"investigation_run_summary",
sources.investigationWorkspace.investigationRunSummarySchema,
["schema_version"],
],
[
"investigation_workspace",
sources.investigationWorkspace.investigationWorkspaceSchema,
["workspace_version"],
],
[
"javascript_application_graph",
sources.javascriptApplicationGraph.javascriptApplicationGraphSchema,
["schema_version"],
],
[
"javascript_application_version_comparison",
sources.javascriptVersionComparison
.applicationVersionComparisonResultSchema,
["schema_version"],
],
[
"javascript_semantic_graph",
sources.javascriptSemanticGraph.javaScriptSemanticGraphSchema,
["schema_version"],
],
[
"javascript_semantic_query",
sources.javascriptSemanticQuery.javaScriptSemanticQueryResultSchema,
["schema_version"],
],
[
"javascript_semantic_trace",
sources.javascriptSemanticTrace.javaScriptSemanticTraceResultSchema,
["schema_version"],
],
[
"javascript_feature_trace",
sources.javascriptFeatureTrace.applicationFeatureTraceResultSchema,
["schema_version"],
],
[
"process_capture",
sources.processCapture.processCaptureSchema,
["schema_version"],
],
[
"reconstruction_verification",
sources.reconstructionVerification.reconstructionSpecificationSchema,
["schema_version"],
],
[
"residual_unknown",
sources.residualUnknown.residualUnknownSchema,
["registry_version"],
],
];
const browserScenarioObservationSchemaDefinitions = (sources) => [
[
"browser_scenario_capture",
sources.browserScenarioCapture.browserScenarioCaptureSchema,
["schema_version"],
],
[
"browser_scenario_diff",
sources.browserScenarioDiff.browserScenarioDiffSchema,
["schema_version"],
],
];
const observationSchemaDefinitions = (sources) => [
[
"managed_artifact_inspection",
sources.managedArtifact.managedArtifactInspectionSchema,
["schema_version"],
],
[
"managed_member_inspection",
sources.managedArtifact.managedMemberInspectionSchema,
["schema_version"],
],
[
"managed_native_boundary_inspection",
sources.managedArtifact.managedNativeBoundaryInspectionSchema,
["schema_version"],
],
[
"managed_member_comparison",
sources.managedComparison.managedMemberComparisonResultSchema,
["schema_version"],
],
[
"managed_native_verification",
sources.managedNativeVerification.managedNativeVerificationResultSchema,
["schema_version"],
],
[
"browser_target_list",
sources.browserObservation.browserTargetListSchema,
["schema_version"],
],
...browserScenarioObservationSchemaDefinitions(sources),
[
"electron_page_inspection",
sources.electronObservation.electronPageInspectionSchema,
["schema_version"],
],
[
"electron_target_list",
sources.electronObservation.electronTargetListSchema,
["schema_version"],
],
[
"javascript_application_analysis",
sources.javascriptApplicationAnalysis
.javaScriptApplicationAnalysisResultV2Schema,
["schema_version"],
],
[
"javascript_runtime_observation",
sources.javascriptRuntimeObservation.javascriptRuntimeObservationSchema,
["schema_version"],
],
[
"javascript_runtime_target_list",
sources.javascriptRuntimeObservation.javascriptRuntimeTargetListSchema,
["schema_version"],
],
[
"web_bundle_analysis",
sources.webBundleAnalysis.webBundleAnalysisSchema,
["schema_version"],
],
[
"web_capture_diff",
sources.webCaptureDiff.webCaptureDiffSchema,
["schema_version"],
],
[
"web_mcp_discovery",
sources.webMcpDiscovery.webMcpDiscoverySchema,
["schema_version"],
],
[
"web_observation_session",
sources.browserSession.webObservationSessionSchema,
["schema_version"],
],
[
"web_page_inspection",
sources.browserObservation.webPageInspectionSchema,
["schema_version"],
],
[
"web_screenshot",
sources.webScreenshot.webScreenshotSchema,
["schema_version"],
],
[
"web_screenshot_diff",
sources.webScreenshot.webScreenshotDiffSchema,
["schema_version"],
],
];
/** Derive the catalog of durable and observation schema versions. */
export const schemaCatalog = (sources) =>
[
...durableSchemaDefinitions(sources),
...observationSchemaDefinitions(sources),
]
.map(([id, schema, path]) => {
try {
return { id, version: versionAt(schema, path) };
} catch (cause) {
throw new Error(`Could not derive ${id} schema version`, { cause });
}
})
.sort((left, right) => left.id.localeCompare(right.id));
scripts/lib/conformance-fixtures.mjs
+2 -2
View File
@@ -21,12 +21,12 @@ export const generateLargeFixture = (count = LARGE_FIXTURE_COUNT) => {
const declarations = Array.from(
{ length: count },
(_, index) =>
`__attribute__((noinline, used)) int rea_page_${String(index).padStart(4, "0")}(void) { puts("REA_PAGE_${String(index).padStart(4, "0")}"); return ${index}; }`,
`__attribute__((noinline, used)) int rea_inventory_${String(index).padStart(4, "0")}(void) { puts("REA_INVENTORY_${String(index).padStart(4, "0")}"); return ${index}; }`,
);
return [
"#include <stdio.h>",
...declarations,
"int main(void) { return rea_page_0000(); }",
"int main(void) { return rea_inventory_0000(); }",
"",
].join("\n");
};
scripts/lib/docs-facts.mjs
-17
View File
@@ -56,23 +56,6 @@ export const documentationFactIssues = async (root, catalog) => {
requireText(issues, templatePath, template, "docs/product-catalog.json");
const english = await readFile(join(root, "README.md"), "utf8");
const schemaVersion = (id) => {
const schema = catalog.schemas.find((candidate) => candidate.id === id);
if (schema === undefined) throw new Error(`Missing ${id} catalog schema`);
return String(schema.version);
};
requireText(
issues,
"README.md",
english,
`Evidence v${schemaVersion("evidence")}`,
);
requireText(
issues,
"README.md",
english,
`Process Capture v${schemaVersion("process_capture")}`,
);
requireText(issues, "README.md", english, "docs/product-catalog.json");
return issues;
};
scripts/lib/managed-conformance-config.mjs
+10 -32
View File
@@ -41,14 +41,6 @@ export const nativeBoundaryLimits = {
maxHeapItemBytes: 1024 * 1024,
};
export const applicationGraphLimits = {
max_types: 100,
max_methods: 100,
max_fields: 100,
max_pinvoke_imports: 100,
max_native_implementations: 100,
};
export const comparisonLimits = {
max_method_matches: 100,
max_field_matches: 100,
@@ -60,13 +52,6 @@ export const defaultIlBody = Buffer.from([
]);
export function functionDossier(name) {
const emptyPage = {
items: [],
total: 0,
returned: 0,
truncated: false,
next_offset: null,
};
return {
procedure: {
address: "0x401000",
@@ -80,23 +65,16 @@ export function functionDossier(name) {
signature: null,
locals: [],
},
pseudocode: {
text: "",
total_chars: 0,
returned_chars: 0,
truncated: false,
next_offset: null,
},
assembly: emptyPage,
comments: emptyPage,
callers: emptyPage,
callees: emptyPage,
incoming_references: emptyPage,
outgoing_references: emptyPage,
referenced_strings: emptyPage,
referenced_names: emptyPage,
basic_blocks: emptyPage,
instruction_scan: { scanned: 0, truncated: false },
pseudocode: "",
assembly: [],
comments: [],
callers: [],
callees: [],
incoming_references: [],
outgoing_references: [],
referenced_strings: [],
referenced_names: [],
basic_blocks: [],
limitations: [],
};
}
scripts/lib/managed-conformance-manifest.mjs
+3 -14
View File
@@ -202,13 +202,6 @@ function verifyManagedAppManifestApplicationGraph({
case_sensitive: expectedTrace.case_sensitive,
},
direction: "incoming",
limits: {
max_seed_matches: 10,
max_depth: 4,
max_nodes: 100,
max_edges: 200,
max_paths: 20,
},
});
ensure(
traced.ok,
@@ -245,11 +238,8 @@ function verifyManagedAppManifestApplicationGraph({
}
const projectManifestMethod = (state, token) => {
const {
appManifestInspectionLimits,
appManifestMemberLimits,
applicationGraphLimits,
} = state.context;
const { appManifestInspectionLimits, appManifestMemberLimits } =
state.context;
const existing = state.graphByToken.get(token);
if (existing !== undefined) return existing;
const method = state.inspectedMethods.get(token);
@@ -289,7 +279,6 @@ const projectManifestMethod = (state, token) => {
const projected = projectManagedApplicationGraphEvidence({
managed_artifact: artifactEvidence,
managed_members: memberEvidence,
limits: applicationGraphLimits,
});
ensure(
projected.ok,
@@ -316,7 +305,7 @@ function inspectManagedMethodByToken(context, bytes, target, expectedMethod) {
target,
expectedMethod.token,
);
const method = members.methods.items.find(
const method = members.methods.find(
({ token }) => token === expectedMethod.token,
);
ensure(
scripts/lib/managed-conformance-oracles.mjs
+2 -2
View File
@@ -39,7 +39,7 @@ async function runManagedAppManifestSelfTest(context) {
sample.target,
context.memberLimits,
);
const method = members.methods.items[0];
const method = members.methods[0];
assert.ok(method);
const summary = await context.verifyManagedAppManifest(
{
@@ -143,7 +143,7 @@ async function runOptionalIlspyOracle(context) {
sample.target,
context.memberLimits,
);
const method = members.methods.items[0];
const method = members.methods[0];
ensureIlspy(method !== undefined, "source-owned fixture has no method");
const listOutput = await runIlspy(
ilspyPath,
scripts/lib/mcp-verifier-results.mjs
+11 -49
View File
@@ -19,35 +19,29 @@ export const requireMcpResult = (result, operation) => {
return value.result;
};
/** Verify direct-provider provenance through the complete Evidence resource. */
export const requireEvidenceProvider = async (
client,
result,
operation,
providerId,
) => {
const evidence = await readMcpEvidence(client, result, operation);
/** Verify provider identity directly from the tool result. */
export const requireEvidenceProvider = (result, operation, providerId) => {
const evidence = jsonValue(result);
if (
evidence?.provider?.id !== providerId ||
evidence?.analysis_profile?.provider?.id !== providerId ||
typeof evidence.analysis_profile.provider.version !== "string"
evidence?.evidence?.provider?.id !== providerId ||
evidence?.evidence?.analysis_profile?.provider?.id !== providerId ||
typeof evidence.evidence.analysis_profile.provider.version !== "string"
) {
throw new Error(`${operation} omitted its concrete provider provenance`);
}
};
/** Verify composed-workflow and upstream provenance through full Evidence. */
export const requireWorkflowEvidenceProvider = async (
client,
/** Verify composed workflow and upstream provider provenance inline. */
export const requireWorkflowEvidenceProvider = (
result,
operation,
expected,
) => {
const evidence = await readMcpEvidence(client, result, operation);
const profile = evidence?.analysis_profile;
const evidence = jsonValue(result);
const profile = evidence?.evidence?.analysis_profile;
const upstream = profile?.parameters?.upstream_analysis_profile;
if (
evidence?.provider?.id !== expected.workflowProviderId ||
evidence?.evidence?.provider?.id !== expected.workflowProviderId ||
profile?.provider?.id !== expected.workflowProviderId ||
upstream?.provider?.id !== expected.upstreamProviderId ||
typeof upstream.provider.version !== "string"
@@ -57,35 +51,3 @@ export const requireWorkflowEvidenceProvider = async (
);
}
};
const readMcpEvidence = async (client, result, operation) => {
const compact = jsonValue(result);
if (
compact === null ||
typeof compact !== "object" ||
typeof compact.evidence_id !== "string" ||
compact.evidence_uri !== `rea://evidence/${compact.evidence_id}`
) {
throw new Error(`${operation} omitted its Evidence reference`);
}
const resource = await client.readResource({ uri: compact.evidence_uri });
const text = resource.contents.find(
(content) =>
content.uri === compact.evidence_uri && typeof content.text === "string",
)?.text;
if (text === undefined)
throw new Error(`${operation} Evidence resource omitted JSON text`);
const evidence = JSON.parse(text);
if (
evidence === null ||
typeof evidence !== "object" ||
evidence.evidence_id !== compact.evidence_id ||
JSON.stringify(evidence.normalized_result) !==
JSON.stringify(compact.result)
) {
throw new Error(
`${operation} Evidence resource did not match its projection`,
);
}
return evidence;
};
scripts/lib/product-catalog.d.mts
-1
View File
@@ -1,5 +1,4 @@
export interface ProductCatalog {
readonly catalog_schema_version: number;
readonly package: {
readonly name: string;
readonly version: string;
scripts/lib/product-catalog.mjs
-3
View File
@@ -7,7 +7,6 @@ import {
cliCommandOptionNames,
} from "./catalog-cli.mjs";
import { providerCatalog, toolFamilyCatalog } from "./catalog-builders.mjs";
import { schemaCatalog } from "./catalog-schemas.mjs";
export {
createCliInventory,
@@ -31,7 +30,6 @@ export const createProductCatalog = async (root) => {
const providers = providerCatalog(sources);
const metadata = sources.packageMetadata.PACKAGE_METADATA;
return {
catalog_schema_version: 1,
package: {
name: metadata.name,
version: metadata.version,
@@ -52,7 +50,6 @@ export const createProductCatalog = async (root) => {
configuration: format === "unsupported" ? "detect-only" : "managed",
}),
),
schemas: schemaCatalog(sources),
cli: {
primary_count: cli.primary.length,
commands: cli.primary,
scripts/lib/real-hopper-exhaustive-search.d.mts
+21
View File
@@ -0,0 +1,21 @@
interface ToolClient {
callTool(request: unknown, options: unknown): Promise<unknown>;
}
interface SearchSummary {
readonly count: number;
readonly calls: 1;
}
export function openAndVerifyLargeFixture(input: {
readonly client: ToolClient;
readonly options: unknown;
readonly normalizedResult: (value: unknown, operation: string) => unknown;
readonly path: string;
readonly expectedCount: number;
readonly symbolPrefix: string;
readonly stringPrefix: string;
}): Promise<{
readonly procedures: SearchSummary;
readonly strings: SearchSummary;
}>;
scripts/lib/real-hopper-exhaustive-search.mjs
+83
View File
@@ -0,0 +1,83 @@
/** Open the large fixture and verify complete inline search results. */
export async function openAndVerifyLargeFixture({
client,
options,
normalizedResult,
path,
expectedCount,
symbolPrefix,
stringPrefix,
}) {
const opened = await client.callTool(
{ name: "open_binary", arguments: { path } },
options,
);
if (opened.isError === true)
throw new Error("Hopper rejected the large fixture");
const procedures = await collectAllMatches({
client,
options,
normalizedResult,
tool: "search_procedures",
pattern: symbolPrefix.replace(/^_+/u, ""),
expectedPrefix: symbolPrefix.replace(/^_+/u, ""),
expectedCount,
normalize: (value) => value.replace(/^_+/u, ""),
});
const strings = await collectAllMatches({
client,
options,
normalizedResult,
tool: "search_strings",
pattern: stringPrefix,
expectedPrefix: stringPrefix,
expectedCount,
normalize: (value) => value,
});
return { procedures, strings };
}
async function collectAllMatches({
client,
options,
normalizedResult,
tool,
pattern,
expectedPrefix,
expectedCount,
normalize,
}) {
const matches = normalizedResult(
await client.callTool(
{
name: tool,
arguments: { pattern, mode: "literal", case_sensitive: true },
},
options,
),
tool,
);
if (!Array.isArray(matches) || matches.length !== expectedCount)
throw new Error(`${tool} did not return every expected match`);
const addresses = new Set();
const values = new Set();
for (const item of matches) {
const value = normalize(item.value);
if (
typeof item.address !== "string" ||
!/^0x[0-9a-f]+$/iu.test(item.address)
)
throw new Error(`${tool} returned a malformed address`);
if (addresses.has(item.address) || values.has(value))
throw new Error(`${tool} returned duplicate matches`);
addresses.add(item.address);
values.add(value);
}
const expectedValues = Array.from(
{ length: expectedCount },
(_, index) => `${expectedPrefix}${String(index).padStart(4, "0")}`,
);
if (expectedValues.some((value) => !values.has(value)))
throw new Error(`${tool} omitted expected matches`);
return { count: values.size, calls: 1 };
}
scripts/lib/real-hopper-fixture.mjs
+1 -1
View File
@@ -69,7 +69,7 @@ function requireLargeOracle(largeOracle) {
largeOracle.symbolCount !== largeOracle.stringCount ||
largeOracle.symbolCount <= 0
)
throw new Error("Large fixture omitted its exact pagination oracle");
throw new Error("Large fixture omitted its complete-inventory oracle");
return largeOracle;
}
scripts/lib/real-hopper-pagination.d.mts
-33
View File
@@ -1,33 +0,0 @@
interface ToolClient {
callTool(request: unknown, options: unknown): Promise<unknown>;
}
interface PaginationSummary {
readonly count: number;
readonly pages: number;
}
export function verifyLargeFixturePagination(input: {
readonly client: ToolClient;
readonly options: unknown;
readonly normalizedResult: (value: unknown, operation: string) => unknown;
readonly expectedCount: number;
readonly symbolPrefix: string;
readonly stringPrefix: string;
}): Promise<{
readonly procedures: PaginationSummary;
readonly strings: PaginationSummary;
}>;
export function openAndVerifyLargeFixture(input: {
readonly client: ToolClient;
readonly options: unknown;
readonly normalizedResult: (value: unknown, operation: string) => unknown;
readonly path: string;
readonly expectedCount: number;
readonly symbolPrefix: string;
readonly stringPrefix: string;
}): Promise<{
readonly procedures: PaginationSummary;
readonly strings: PaginationSummary;
}>;
scripts/lib/real-hopper-pagination.mjs
-138
View File
@@ -1,138 +0,0 @@
const PAGE_LIMIT = 100;
/** Open the large fixture and exhaustively prove its public search pages. */
export async function openAndVerifyLargeFixture({
client,
options,
normalizedResult,
path,
expectedCount,
symbolPrefix,
stringPrefix,
}) {
const opened = await client.callTool(
{ name: "open_binary", arguments: { path } },
options,
);
if (opened.isError === true)
throw new Error("Hopper rejected the large fixture");
return verifyLargeFixturePagination({
client,
options,
normalizedResult,
expectedCount,
symbolPrefix,
stringPrefix,
});
}
/** Exhaustively prove the source-owned large fixture through public search pages. */
export async function verifyLargeFixturePagination({
client,
options,
normalizedResult,
expectedCount,
symbolPrefix,
stringPrefix,
}) {
const procedures = await collectExactPages({
client,
options,
normalizedResult,
tool: "search_procedures",
pattern: symbolPrefix.replace(/^_+/u, ""),
expectedPrefix: symbolPrefix.replace(/^_+/u, ""),
expectedCount,
normalize: (value) => value.replace(/^_+/u, ""),
});
const strings = await collectExactPages({
client,
options,
normalizedResult,
tool: "search_strings",
pattern: stringPrefix,
expectedPrefix: stringPrefix,
expectedCount,
normalize: (value) => value,
});
return { procedures, strings };
}
async function collectExactPages({
client,
options,
normalizedResult,
tool,
pattern,
expectedPrefix,
expectedCount,
normalize,
}) {
const seenAddresses = new Set();
const seenValues = new Set();
let offset = 0;
let pages = 0;
while (true) {
const page = normalizedResult(
await client.callTool(
{
name: tool,
arguments: {
pattern,
mode: "literal",
case_sensitive: true,
offset,
limit: PAGE_LIMIT,
},
},
options,
),
`${tool} offset ${offset}`,
);
validatePage(page, { offset, expectedCount });
for (const item of page.items) {
const value = normalize(item.value);
if (item.value_truncated || !/^0x[0-9a-f]+$/iu.test(item.address))
throw new Error(`${tool} returned truncated or malformed evidence`);
if (seenAddresses.has(item.address) || seenValues.has(value))
throw new Error(`${tool} returned duplicate evidence`);
seenAddresses.add(item.address);
seenValues.add(value);
}
pages += 1;
if (page.next_offset === null) break;
offset = page.next_offset;
}
const expectedValues = Array.from(
{ length: expectedCount },
(_, index) => `${expectedPrefix}${String(index).padStart(4, "0")}`,
);
if (
seenValues.size !== expectedCount ||
expectedValues.some((value) => !seenValues.has(value))
)
throw new Error(`${tool} did not exhaustively prove the fixture values`);
return { count: seenValues.size, pages };
}
function validatePage(page, { offset, expectedCount }) {
if (
page === null ||
typeof page !== "object" ||
!Array.isArray(page.items) ||
page.offset !== offset ||
page.limit !== PAGE_LIMIT ||
page.total !== expectedCount ||
page.items.length !== Math.min(PAGE_LIMIT, expectedCount - offset)
)
throw new Error("Hopper returned an inconsistent fixture search page");
const expectedNext =
offset + page.items.length < expectedCount
? offset + page.items.length
: null;
if (
page.next_offset !== expectedNext ||
page.has_more !== (expectedNext !== null)
)
throw new Error("Hopper returned inconsistent pagination metadata");
}
scripts/lib/real-hopper-selection.mjs
+33
View File
@@ -0,0 +1,33 @@
import { HOPPER_PROVIDER_IDENTITY } from "../../dist/hopper/HopperProvider.js";
/** Verify the selected Hopper candidate and concrete provider binding. */
export const requireHopperSelection = (status, expected) => {
const candidates = status.analysis_provider_candidates;
const hopper = Array.isArray(candidates)
? candidates.find(
(candidate) => candidate?.provider?.id === HOPPER_PROVIDER_IDENTITY.id,
)
: undefined;
if (
hopper === undefined ||
hopper.availability?.status !== "available" ||
hopper.target_support?.status !== expected.targetSupport ||
hopper.selected !== expected.selected
)
throw new Error("binary_session omitted truthful Hopper candidate status");
if (!expected.selected) {
if (status.analysis_provider_binding !== null)
throw new Error("Target-free status unexpectedly selected a provider");
return null;
}
const binding = status.analysis_provider_binding;
if (
binding?.provider?.id !== HOPPER_PROVIDER_IDENTITY.id ||
typeof binding.provider.version !== "string" ||
binding.selection_source !== "auto-single-candidate" ||
binding.analysis_profile?.provider?.id !== HOPPER_PROVIDER_IDENTITY.id ||
binding.analysis_profile.provider.version !== binding.provider.version
)
throw new Error("binary_session omitted its concrete Hopper binding");
return binding;
};
scripts/lib/real-hopper-semantic.mjs
+15 -12
View File
@@ -152,7 +152,7 @@ export async function verifyRealHopperFixture({
};
}
/** Reject malformed, unsafe, or error-level diagnostics from the MCP runtime. */
/** Reject malformed, unsafe, or unexpected failure diagnostics from MCP. */
export function requireSafeDiagnostics(chunks) {
const lines = chunks
.join("")
@@ -171,7 +171,12 @@ export function requireSafeDiagnostics(chunks) {
typeof diagnostic !== "object" ||
diagnostic.application !== "rea" ||
typeof diagnostic.level !== "number" ||
diagnostic.level >= 40
diagnostic.level >= 50 ||
(diagnostic.level >= 40 &&
diagnostic.errorTag !== "HopperRemoteError" &&
!(
diagnostic.tool === "procedure_info" && diagnostic.status === "error"
))
)
throw new Error(`The MCP runtime emitted unsafe stderr: ${line}`);
}
@@ -204,7 +209,7 @@ export async function resolveFixtureProcedure(
expectedName,
normalizedResult,
) {
const page = normalizedResult(
const matches = normalizedResult(
await client.callTool(
{
name: "search_procedures",
@@ -212,30 +217,28 @@ export async function resolveFixtureProcedure(
pattern: expectedName,
mode: "literal",
case_sensitive: true,
limit: 100,
},
},
options,
),
`search_procedures ${expectedName}`,
);
const matches = Array.isArray(page?.items)
? page.items.filter(
const exactMatches = Array.isArray(matches)
? matches.filter(
(item) =>
typeof item?.value === "string" &&
item.value_truncated === false &&
item.value.replace(/^_+/u, "") === expectedName,
)
: [];
if (
matches.length !== 1 ||
typeof matches[0]?.address !== "string" ||
!/^0x[0-9a-f]+$/iu.test(matches[0].address)
exactMatches.length !== 1 ||
typeof exactMatches[0]?.address !== "string" ||
!/^0x[0-9a-f]+$/iu.test(exactMatches[0].address)
)
throw new Error(
`Expected exactly one Hopper procedure named ${expectedName}`,
);
return { address: matches[0].address, name: matches[0].value };
return { address: exactMatches[0].address, name: exactMatches[0].value };
}
/** Request the assembly evidence required by the fixture oracle. */
@@ -249,7 +252,7 @@ export const analyzeFixtureProcedure = async (
await client.callTool(
{
name: "analyze_function",
arguments: { procedure: procedure.address, include_assembly: true },
arguments: { procedure: procedure.address },
},
options,
),
scripts/lib/verify-package-core.mjs
+11 -25
View File
@@ -32,7 +32,7 @@ export const json = (text) => JSON.parse(text);
export const verifyCompleteToolCatalog = async (client, options) => {
const listed = await client.listTools(undefined, options);
const status = await client.callTool(
{ name: "binary_session", arguments: { detail: "full" } },
{ name: "binary_session", arguments: {} },
options,
);
const availability = status.structuredContent?.result?.tool_availability;
@@ -61,13 +61,6 @@ export const pathExists = async (path) => {
/** Synthetic function dossier for managed/native verification fixtures. */
export const functionDossier = (name) => {
const emptyPage = {
items: [],
total: 0,
returned: 0,
truncated: false,
next_offset: null,
};
return {
procedure: {
address: "0x401000",
@@ -81,23 +74,16 @@ export const functionDossier = (name) => {
signature: null,
locals: [],
},
pseudocode: {
text: "",
total_chars: 0,
returned_chars: 0,
truncated: false,
next_offset: null,
},
assembly: emptyPage,
comments: emptyPage,
callers: emptyPage,
callees: emptyPage,
incoming_references: emptyPage,
outgoing_references: emptyPage,
referenced_strings: emptyPage,
referenced_names: emptyPage,
basic_blocks: emptyPage,
instruction_scan: { scanned: 0, truncated: false },
pseudocode: "",
assembly: [],
comments: [],
callers: [],
callees: [],
incoming_references: [],
outgoing_references: [],
referenced_strings: [],
referenced_names: [],
basic_blocks: [],
limitations: [],
};
};
scripts/verify-agent-experience.mjs
+2 -4
View File
@@ -134,7 +134,7 @@ try {
const results = [];
for (const scenario of scenarios) {
process.stderr.write(`Running Codex agent evaluation: ${scenario.id}\n`);
const execution = await runCodex(scenario.prompt, fixtureRoot);
const execution = await runCodex(scenario.prompt);
const transcriptDirectory = process.env.REA_AGENT_EVAL_TRANSCRIPT_DIR;
if (transcriptDirectory !== undefined) {
const transcriptPath = resolve(
@@ -332,9 +332,8 @@ async function createTargets(root, includeManaged) {
};
}
async function runCodex(prompt, investigationRoot) {
async function runCodex(prompt) {
const mcpEnvironment = [
["REA_INVESTIGATION_INPUT_ROOTS_JSON", JSON.stringify([investigationRoot])],
["REA_BROWSER_OBSERVE_ENABLED", "true"],
[
"REA_BROWSER_CDP_ENDPOINTS_JSON",
@@ -378,7 +377,6 @@ async function runCodex(prompt, investigationRoot) {
cwd: evaluationRoot,
env: {
...process.env,
REA_INVESTIGATION_INPUT_ROOTS_JSON: JSON.stringify([investigationRoot]),
REA_BROWSER_OBSERVE_ENABLED: "true",
REA_BROWSER_CDP_ENDPOINTS_JSON: JSON.stringify(["http://127.0.0.1:9222"]),
REA_BROWSER_ALLOWED_ORIGINS_JSON: JSON.stringify([
scripts/verify-local-application-workflows.mjs
+4 -35
View File
@@ -9,11 +9,9 @@ import {
compareJavaScriptExportShapesEvidence,
traceApplicationFeatureEvidence,
} from "../dist/application/JavaScriptApplicationWorkflowService.js";
import { PermissionAuthority } from "../dist/application/PermissionAuthority.js";
import { applicationVersionComparisonResultSchema } from "../dist/domain/javascriptApplicationVersionComparisonSchemas.js";
import { applicationFeatureTraceResultSchema } from "../dist/domain/javascriptFeatureTraceSchemas.js";
import { javaScriptExportShapeComparisonResultSchema } from "../dist/domain/javascriptExportShapeComparisonSchemas.js";
import { createPermissionPolicy } from "../dist/domain/permissionPolicy.js";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const verifierRun = createVerifierRun();
@@ -24,7 +22,6 @@ const options = parseArgs({
right: { type: "string" },
"seed-kind": { type: "string", default: "module" },
"seed-value": { type: "string" },
"source-map-read-approved": { type: "boolean", default: false },
"left-module-path": { type: "string" },
"left-export-name": { type: "string" },
"right-module-path": { type: "string" },
@@ -44,10 +41,9 @@ const [leftPath, rightPath] = await Promise.all([
]);
if (leftPath === rightPath)
throw new Error("--left and --right must resolve to distinct paths");
const authority = permissionAuthority([leftPath, rightPath]);
const [left, right] = await Promise.all([
analyze(leftPath, authority),
analyze(rightPath, authority),
analyze(leftPath),
analyze(rightPath),
]);
const comparison = compareApplicationVersionsEvidence({ left, right });
if (!comparison.ok) throw comparison.error;
@@ -199,37 +195,10 @@ function selectorKey(selector) {
return `${selector.modulePath}\0${selector.exportName}`;
}
async function analyze(path, authority) {
const result = await analyzeJavaScriptApplication(authority, {
async function analyze(path) {
const result = await analyzeJavaScriptApplication({
input_path: path,
approved: true,
source_map_read_approved: options["source-map-read-approved"],
});
if (!result.ok) throw result.error;
return result.value;
}
function permissionAuthority(roots) {
const scope = {
capability: "investigation_input",
roots,
executables: [],
environment_names: [],
network: "none",
mount: false,
};
return new PermissionAuthority(
createPermissionPolicy(
[scope],
[
{
...scope,
grant_id: "local-application-workflow-verifier",
lifetime: "session",
operation_identity: null,
expires_at: null,
},
],
),
);
}
scripts/verify-local-javascript-runtime.mjs
+1 -32
View File
@@ -4,9 +4,7 @@ import { readFile, realpath } from "node:fs/promises";
import { parseArgs } from "node:util";
import { analyzeJavaScriptApplication } from "../dist/application/JavaScriptApplicationService.js";
import { PermissionAuthority } from "../dist/application/PermissionAuthority.js";
import { reconcileJavaScriptRuntimeEvidence } from "../dist/application/JavaScriptRuntimeReconciliationService.js";
import { createPermissionPolicy } from "../dist/domain/permissionPolicy.js";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const verifierRun = createVerifierRun();
@@ -18,7 +16,6 @@ const options = parseArgs({
assets: { type: "string", multiple: true, default: [] },
"runtime-evidence": { type: "string", multiple: true, default: [] },
mapping: { type: "string", multiple: true, default: [] },
"source-map-read-approved": { type: "boolean", default: false },
},
strict: true,
allowPositionals: false,
@@ -42,14 +39,11 @@ const canonicalLayers = await Promise.all(
path: await realpath(layer.path),
})),
);
const authority = permissionAuthority(canonicalLayers.map(({ path }) => path));
const staticLayers = [];
for (const layer of canonicalLayers) {
const analyzed = await analyzeJavaScriptApplication(authority, {
const analyzed = await analyzeJavaScriptApplication({
input_path: layer.path,
format: "auto",
approved: true,
source_map_read_approved: options["source-map-read-approved"],
});
if (!analyzed.ok) throw analyzed.error;
staticLayers.push({
@@ -104,31 +98,6 @@ process.stdout.write(
})}\n`,
);
function permissionAuthority(roots) {
const scope = {
capability: "investigation_input",
roots,
executables: [],
environment_names: [],
network: "none",
mount: false,
};
return new PermissionAuthority(
createPermissionPolicy(
[scope],
[
{
...scope,
grant_id: "local-javascript-runtime-verifier",
lifetime: "session",
operation_identity: null,
expires_at: null,
},
],
),
);
}
async function readEvidenceFile(path) {
const bytes = await readFile(await realpath(path));
if (bytes.byteLength > 64 * 1024 * 1024)
scripts/verify-managed-conformance.mjs
+11 -37
View File
@@ -21,7 +21,6 @@ import {
} from "./lib/managed-pe-fixture.mjs";
import { createManagedConformanceSupport } from "./lib/managed-conformance-support.mjs";
import {
applicationGraphLimits,
comparisonLimits,
defaultIlBody,
functionDossier,
@@ -45,7 +44,6 @@ const {
workspace,
inspectionLimits,
memberLimits,
applicationGraphLimits,
defaultIlBody,
});
@@ -66,7 +64,7 @@ try {
".NETCoreApp,Version=v8.0",
]);
assert.equal(
modernArtifact.resources.items[0]?.data_sha256,
modernArtifact.resources[0]?.data_sha256,
sha256(Buffer.from("source-owned managed resource")),
);
@@ -94,17 +92,14 @@ try {
framework.target,
nativeBoundaryLimits,
);
assert.equal(frameworkBoundaries.pinvoke_imports.total, 1);
assert.equal(frameworkBoundaries.pinvoke_imports.length, 1);
assert.equal(
frameworkBoundaries.pinvoke_imports.items[0]?.verification,
frameworkBoundaries.pinvoke_imports[0]?.verification,
"managed-declaration-only",
);
assert.equal(frameworkBoundaries.pinvoke_imports[0]?.char_set, "unicode");
assert.equal(
frameworkBoundaries.pinvoke_imports.items[0]?.char_set,
"unicode",
);
assert.equal(
frameworkBoundaries.pinvoke_imports.items[0]?.call_convention,
frameworkBoundaries.pinvoke_imports[0]?.call_convention,
"stdcall",
);
const frameworkMembers = inspectManagedMembersBytes(
@@ -173,10 +168,6 @@ try {
const nativeVerification = verifyManagedNativeBoundariesEvidence({
managed_boundaries: frameworkBoundaryEvidence,
native_observations: [nativeFunctionEvidence],
limits: {
max_native_observations: 20,
max_candidates_per_import: 25,
},
});
assert.equal(nativeVerification.ok, true);
assert.equal(nativeVerification.value.normalized_result.summary.verified, 1);
@@ -184,13 +175,6 @@ try {
managed_artifact: frameworkArtifactEvidence,
managed_members: frameworkMemberEvidence,
managed_native_boundaries: frameworkBoundaryEvidence,
limits: {
max_types: 100,
max_methods: 100,
max_fields: 100,
max_pinvoke_imports: 100,
max_native_implementations: 100,
},
});
assert.equal(applicationGraph.ok, true);
assert.equal(
@@ -213,13 +197,6 @@ try {
case_sensitive: true,
},
direction: "incoming",
limits: {
max_seed_matches: 5,
max_depth: 4,
max_nodes: 50,
max_edges: 100,
max_paths: 10,
},
});
assert.equal(applicationTrace.ok, true);
assert.equal(
@@ -254,12 +231,9 @@ try {
nativeBoundaryLimits,
);
assert.equal(r2rBoundaries.summary.ready_to_run, true);
assert.equal(r2rBoundaries.native_implementations[0]?.code_type, "native");
assert.equal(
r2rBoundaries.native_implementations.items[0]?.code_type,
"native",
);
assert.equal(
r2rBoundaries.native_implementations.items[0]?.boundary_kind,
r2rBoundaries.native_implementations[0]?.boundary_kind,
"native-body",
);
@@ -273,9 +247,9 @@ try {
obfuscated.target,
memberLimits,
);
assert.equal(obfuscatedMembers.types.items[0]?.full_name, "Fixture.ꙮType");
assert.equal(obfuscatedMembers.methods.items[0]?.name, "λ⛧");
assert.equal(obfuscatedMembers.fields.items[0]?.name, "字段");
assert.equal(obfuscatedMembers.types[0]?.full_name, "Fixture.ꙮType");
assert.equal(obfuscatedMembers.methods[0]?.name, "λ⛧");
assert.equal(obfuscatedMembers.fields[0]?.name, "字段");
const runtimeExecutable = join(workspace, "dotnet");
await writeFile(runtimeExecutable, "#!/bin/sh\n");
const runtimeCeiling = {
@@ -299,7 +273,7 @@ try {
],
);
assert.equal(runtimeAuthority.ok, true);
const runtimeMethod = obfuscatedMembers.methods.items[0];
const runtimeMethod = obfuscatedMembers.methods[0];
assert.ok(runtimeMethod);
const obfuscatedMembersEvidence = createEvidence(
undefined,
scripts/verify-package-discovery.mjs
+1 -1
View File
@@ -39,7 +39,7 @@ export async function verifyPackageDiscovery({ cli, environment }) {
);
const doctorExecution = await runWithStatus(
cli,
["doctor", "--detail", "full", "--json"],
["doctor", "--json"],
environment,
);
const summary = json(summaryExecution.stdout);
scripts/verify-package-environment.mjs
-1
View File
@@ -75,7 +75,6 @@ export async function verifyPackageEnvironment({
...packageHopperEnvironment(root),
REA_NPX_LOG: npxLog,
REA_EVIDENCE_ROOTS_JSON: JSON.stringify([evidenceRoot]),
REA_INVESTIGATION_INPUT_ROOTS_JSON: JSON.stringify([workspace]),
REA_REFERENCE_ROOTS_JSON: JSON.stringify([referenceRoot]),
REA_REFERENCE_SECRET_PATTERNS_JSON: JSON.stringify([".env"]),
};
scripts/verify-package-managed.mjs
+1 -1
View File
@@ -327,7 +327,7 @@ export async function verifyManaged({ cli, workspace, environment }) {
managedPath,
environment,
});
const managedMethod = managedMembers.normalized_result.methods.items[0];
const managedMethod = managedMembers.normalized_result.methods[0];
await verifyManagedReconstruction({
cli,
managedMembers,
scripts/verify-package-mcp.mjs
+2 -3
View File
@@ -41,7 +41,7 @@ const verifyMcpReplay = async (client, mcpOptions, investigationReplay) => {
const verifyMcpTargetFree = async (client, mcpOptions) => {
const status = await client.callTool(
{ name: "binary_session", arguments: { detail: "full" } },
{ name: "binary_session", arguments: {} },
mcpOptions,
);
const currentDocument =
@@ -89,7 +89,7 @@ const verifyMcpOpenAndBind = async (client, mcpOptions) => {
const providerStatusEnvelope = json(
prompts.mcpText(
await client.callTool(
{ name: "binary_session", arguments: { detail: "full" } },
{ name: "binary_session", arguments: {} },
mcpOptions,
),
),
@@ -204,7 +204,6 @@ export async function verifyPackageMcp({
args: ["mcp"],
env: {
...environment,
REA_INVESTIGATION_INPUT_ROOTS_JSON: JSON.stringify([]),
},
stderr: "pipe",
});
scripts/verify-package-platform.mjs
+2 -14
View File
@@ -21,23 +21,11 @@ export async function verifyPackagePlatform({ cli, environment }) {
`packaged Hopper-backed analyze CLI failed: ${JSON.stringify(overview)}`,
);
const inspected = json(
await run(
cli,
[
"inspect",
process.execPath,
"--detail",
"detailed",
"--limit",
"1",
"--json",
],
environment,
),
await run(cli, ["inspect", process.execPath, "--json"], environment),
);
if (
inspected.operation !== "binary_overview" ||
inspected.normalized_result?.detail !== "detailed"
inspected.normalized_result?.procedure_count < 1
)
throw new Error("packaged inspect CLI failed");
const functionResult = json(
scripts/verify-package.mjs
-2
View File
@@ -54,7 +54,6 @@ try {
args: ["mcp"],
environment: {
...environmentData.environment,
REA_INVESTIGATION_INPUT_ROOTS_JSON: JSON.stringify([]),
},
policy: MCP_STARTUP_POLICY,
});
@@ -63,7 +62,6 @@ try {
args: ["mcp"],
environment: {
...environmentData.environment,
REA_INVESTIGATION_INPUT_ROOTS_JSON: JSON.stringify([]),
},
policy: MCP_STARTUP_POLICY,
packageName: PRODUCT_IDENTITY.packageName,
scripts/verify-packed-bridge.mjs
+1 -1
View File
@@ -35,7 +35,7 @@ export async function verifyPackedBridge({
const probe = JSON.parse(
(
await exec("python3", [
join(root, "tests/fixtures/bridgeRegexProbe.py"),
join(root, "tests/fixtures/bridgeSearchProbe.py"),
join(workspace, packedHopperBridge),
JSON.stringify({
action: "match",
scripts/verify-real-browser.mjs
+2 -2
View File
@@ -408,8 +408,8 @@ async function pageTarget(provider, endpoint, origin) {
approved: true,
}),
);
if (listed.ok && listed.value.targets.items[0] !== undefined)
return listed.value.targets.items[0].target_id;
if (listed.ok && listed.value.targets[0] !== undefined)
return listed.value.targets[0].target_id;
await delay(25);
}
throw new Error("Real Chrome did not expose the local test page target");
scripts/verify-real-ghidra-assertions.mjs
+14 -27
View File
@@ -145,10 +145,8 @@ function assertDossierProcedure(dossier, address) {
dossier.procedure.classification === null ||
dossier.procedure.classification.provenance !== "ghidra-function-manager" ||
dossier.procedure.classification.external ||
dossier.pseudocode.text.trim().length === 0 ||
dossier.pseudocode.returned_chars !== [...dossier.pseudocode.text].length ||
dossier.instruction_scan.scanned < 1 ||
dossier.instruction_scan.truncated ||
typeof dossier.pseudocode !== "string" ||
dossier.pseudocode.trim().length === 0 ||
!dossier.limitations.some((value) => /indirect|computed/u.test(value)) ||
!dossier.limitations.some((value) => /Ghidra|Hopper|provider/u.test(value))
)
@@ -159,10 +157,7 @@ function assertDossierProcedure(dossier, address) {
function assertDossierReferences(dossier) {
if (
![
...dossier.incoming_references.items,
...dossier.outgoing_references.items,
].every(
![...dossier.incoming_references, ...dossier.outgoing_references].every(
({ kind }) =>
kind.available && kind.provenance === "ghidra-reference-manager",
)
@@ -172,16 +167,14 @@ function assertDossierReferences(dossier) {
function assertDossierCallees(dossier, expectedCallees) {
for (const expected of expectedCallees)
if (!dossier.callees.items.some(({ address: value }) => value === expected))
if (!dossier.callees.some(({ address: value }) => value === expected))
throw new Error(`Ghidra dossier missed callee ${expected}`);
}
function assertDossierStrings(dossier, expectedString) {
if (
expectedString !== null &&
!dossier.referenced_strings.items.some(
({ value }) => value === expectedString,
)
!dossier.referenced_strings.some(({ value }) => value === expectedString)
)
throw new Error(
`Ghidra dossier missed referenced string ${expectedString}`,
@@ -189,20 +182,15 @@ function assertDossierStrings(dossier, expectedString) {
}
function assertDossierAssembly(dossier, requireAssembly) {
if (
requireAssembly &&
(dossier.assembly.items.length === 0 || dossier.assembly.truncated)
)
if (requireAssembly && dossier.assembly.length === 0)
throw new Error("Ghidra dossier assembly was unavailable or truncated");
}
function assertDossierMultiBlock(dossier, requireMultiBlock) {
if (
requireMultiBlock &&
(dossier.basic_blocks.items.length < 2 ||
!dossier.basic_blocks.items.some(
({ successors }) => successors.length > 0,
))
(dossier.basic_blocks.length < 2 ||
!dossier.basic_blocks.some(({ successors }) => successors.length > 0))
)
throw new Error(
"Ghidra dossier CFG missed the multi-block branch structure",
@@ -212,12 +200,12 @@ function assertDossierMultiBlock(dossier, requireMultiBlock) {
export function dossierSummary(dossier) {
return {
address: dossier.procedure.address,
pseudocode_chars: dossier.pseudocode.total_chars,
instructions: dossier.instruction_scan.scanned,
callees: dossier.callees.returned,
outgoing_references: dossier.outgoing_references.returned,
referenced_strings: dossier.referenced_strings.returned,
basic_blocks: dossier.basic_blocks.returned,
pseudocode_chars: [...dossier.pseudocode].length,
instructions: dossier.assembly.length,
callees: dossier.callees.length,
outgoing_references: dossier.outgoing_references.length,
referenced_strings: dossier.referenced_strings.length,
basic_blocks: dossier.basic_blocks.length,
native_parameters:
dossier.native_api.available === true
? dossier.native_api.parameters.length
@@ -350,7 +338,6 @@ function assertStrings(strings) {
const item = strings.find((candidate) => candidate.value === expected);
if (
item === undefined ||
item.value_truncated ||
typeof item.string.encoding !== "string" ||
item.string.byte_length < expected.length
)
scripts/verify-real-ghidra-function.mjs
+2 -33
View File
@@ -118,10 +118,6 @@ export async function verifyDebugFunctionOperations(
const switchDossier = await analyzeSwitchDossier(client, switchProcedure);
const denseSwitchDossier = await functionCall(client, "analyze_function", {
procedure: denseSwitchProcedure.address,
include_assembly: true,
limit: 500,
max_pseudocode_chars: 100_000,
max_instructions: 5_000,
});
assertDenseSwitchDossier(denseSwitchDossier, denseSwitchProcedure.address);
assertIndirectDossier(indirectDossier, leaf.address);
@@ -246,10 +242,6 @@ async function assertStringXrefs(client, entryStringAddress) {
async function analyzeEntryDossier(client, entry, branch, indirect) {
const entryDossier = await functionCall(client, "analyze_function", {
procedure: entry.address,
include_assembly: true,
limit: 500,
max_pseudocode_chars: 100_000,
max_instructions: 5_000,
});
assertDossier(entryDossier, {
address: entry.address,
@@ -263,10 +255,6 @@ async function analyzeEntryDossier(client, entry, branch, indirect) {
async function analyzeSwitchDossier(client, switchProcedure) {
const dossier = await functionCall(client, "analyze_function", {
procedure: switchProcedure.address,
include_assembly: true,
limit: 500,
max_pseudocode_chars: 100_000,
max_instructions: 5_000,
});
assertDossier(dossier, {
address: switchProcedure.address,
@@ -280,9 +268,6 @@ async function analyzeSwitchDossier(client, switchProcedure) {
async function analyzeBranchDossier(client, branch) {
const branchDossier = await functionCall(client, "analyze_function", {
procedure: branch.address,
limit: 500,
max_pseudocode_chars: 100_000,
max_instructions: 5_000,
});
assertDossier(branchDossier, {
address: branch.address,
@@ -294,10 +279,6 @@ async function analyzeBranchDossier(client, branch) {
async function analyzeIndirectDossier(client, indirect) {
const indirectDossier = await functionCall(client, "analyze_function", {
procedure: indirect.address,
include_assembly: true,
limit: 500,
max_pseudocode_chars: 100_000,
max_instructions: 5_000,
});
assertDossier(indirectDossier, {
address: indirect.address,
@@ -307,13 +288,11 @@ async function analyzeIndirectDossier(client, indirect) {
}
function assertIndirectDossier(indirectDossier, leafAddress) {
if (
indirectDossier.callees.items.some(({ address }) => address === leafAddress)
)
if (indirectDossier.callees.some(({ address }) => address === leafAddress))
throw new Error(
"Ghidra dossier falsely resolved a targetless callback as the fixture leaf",
);
if (!/\bcall\b/iu.test(indirectDossier.assembly.items.join("\n")))
if (!/\bcall\b/iu.test(indirectDossier.assembly.join("\n")))
throw new Error("Ghidra indirect-call fixture lost its computed call site");
}
@@ -384,10 +363,6 @@ export async function verifyStrippedFunctionOperations(client, entryString) {
throw new Error("Ghidra did not decompile the stripped function");
const dossier = await functionCall(client, "analyze_function", {
procedure: procedure.address,
include_assembly: true,
limit: 500,
max_pseudocode_chars: 100_000,
max_instructions: 5_000,
});
assertDossier(dossier, {
address: procedure.address,
@@ -449,9 +424,6 @@ export async function verifyCrossFormatOperations({
const messageOwner = await findCrossStringOwner(client, messageXrefs);
const stringDossier = await functionCall(client, "analyze_function", {
procedure: messageOwner.procedure.address,
limit: 500,
max_pseudocode_chars: 100_000,
max_instructions: 5_000,
});
assertDossier(stringDossier, {
address: messageOwner.procedure.address,
@@ -461,9 +433,6 @@ export async function verifyCrossFormatOperations({
const branchAddress = await findCrossBranchAddress(client, variant, callees);
const branchDossier = await functionCall(client, "analyze_function", {
procedure: branchAddress,
limit: 500,
max_pseudocode_chars: 100_000,
max_instructions: 5_000,
});
assertDossier(branchDossier, {
address: branchAddress,
scripts/verify-real-ghidra-windows.mjs
-1
View File
@@ -153,7 +153,6 @@ try {
await functionOperation("xrefs", { address: procedure.address });
await functionOperation("analyze_function", {
procedure: procedure.value,
include_assembly: true,
});
const expected = [
scripts/verify-real-hopper.mjs
+47 -58
View File
@@ -29,8 +29,9 @@ import {
requireWorkflowEvidenceProvider,
} from "./lib/mcp-verifier-results.mjs";
import { verifyHopperFunctionBasics } from "./lib/real-hopper-function-basics.mjs";
import { openAndVerifyLargeFixture } from "./lib/real-hopper-pagination.mjs";
import { openAndVerifyLargeFixture } from "./lib/real-hopper-exhaustive-search.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
import { requireHopperSelection } from "./lib/real-hopper-selection.mjs";
import { startUnrelatedHopperSentinel } from "./lib/unrelated-hopper-sentinel.mjs";
const execFileAsync = promisify(execFile);
const verifierRun = createVerifierRun();
@@ -51,39 +52,6 @@ const sessionsBefore = new Set(
const textValue = mcpTextValue;
const requireSuccessfulTool = requireMcpResult;
const requireHopperSelection = (status, expected) => {
const candidates = status.analysis_provider_candidates;
const hopper = Array.isArray(candidates)
? candidates.find(
(candidate) => candidate?.provider?.id === HOPPER_PROVIDER_IDENTITY.id,
)
: undefined;
if (
hopper === undefined ||
hopper.availability?.status !== "available" ||
hopper.target_support?.status !== expected.targetSupport ||
hopper.selected !== expected.selected
) {
throw new Error("binary_session omitted truthful Hopper candidate status");
}
if (!expected.selected) {
if (status.analysis_provider_binding !== null)
throw new Error("Target-free status unexpectedly selected a provider");
return null;
}
const binding = status.analysis_provider_binding;
if (
binding?.provider?.id !== HOPPER_PROVIDER_IDENTITY.id ||
typeof binding.provider.version !== "string" ||
binding.selection_source !== "auto-single-candidate" ||
binding.analysis_profile?.provider?.id !== HOPPER_PROVIDER_IDENTITY.id ||
binding.analysis_profile.provider.version !== binding.provider.version
) {
throw new Error("binary_session omitted its concrete Hopper binding");
}
return binding;
};
const requireOverview = (value, operation) => {
if (
value === null ||
@@ -230,7 +198,7 @@ const verifyCurrentTarget = async (client, options) => {
procedure: dossier.procedure,
procedureCount: procedures.total,
boundedPseudocodeChars: boundedPseudocode.length,
analyzedPseudocodeChars: [...dossier.pseudocode.text].length,
analyzedPseudocodeChars: [...dossier.pseudocode].length,
callerCount: relationships.procedure_callers.length,
calleeCount: relationships.procedure_callees.length,
xrefCount: relationships.xrefs.length,
@@ -287,24 +255,20 @@ try {
onprogress: (update) => progressUpdates.push(update),
};
const fullSessionStatus = () =>
client.callTool(
{ name: "binary_session", arguments: { detail: "full" } },
options,
);
client.callTool({ name: "binary_session", arguments: {} }, options);
const initialSession = await fullSessionStatus();
const initialStatus = requireMcpResult(initialSession, "binary_session");
const listed = await client.listTools();
const availableTools = initialStatus.tool_availability;
if (!Array.isArray(availableTools))
throw new Error("The real session omitted tool availability");
const expectedNames = availableTools
.filter(({ available }) => available === true)
.map(({ name }) => name)
.sort();
const expectedNames = availableTools.map(({ name }) => name).sort();
const actualNames = listed.tools.map(({ name }) => name).sort();
if (JSON.stringify(actualNames) !== JSON.stringify(expectedNames)) {
const expected = new Set(expectedNames);
const actual = new Set(actualNames);
throw new Error(
"The real server tool inventory diverged from session availability",
`The real server tool catalog diverged from session capability inventory (untracked tools: ${actualNames.filter((name) => !expected.has(name)).join(", ") || "none"}; missing catalog tools: ${expectedNames.filter((name) => !actual.has(name)).join(", ") || "none"})`,
);
}
@@ -345,18 +309,15 @@ try {
{ name: "binary_overview", arguments: {} },
options,
);
await Promise.all([
requireEvidenceProvider(
client,
documents,
"list_documents",
HOPPER_PROVIDER_IDENTITY.id,
),
requireWorkflowEvidenceProvider(client, overview, "binary_overview", {
workflowProviderId: REA_WORKFLOW_PROVIDER.id,
upstreamProviderId: HOPPER_PROVIDER_IDENTITY.id,
}),
]);
requireEvidenceProvider(
documents,
"list_documents",
HOPPER_PROVIDER_IDENTITY.id,
);
requireWorkflowEvidenceProvider(overview, "binary_overview", {
workflowProviderId: REA_WORKFLOW_PROVIDER.id,
upstreamProviderId: HOPPER_PROVIDER_IDENTITY.id,
});
const segments = await client.callTool(
{ name: "list_segments", arguments: {} },
options,
@@ -413,7 +374,33 @@ try {
"binary_overview after target switch",
);
const secondAnalysis = await verifyCurrentTarget(client, options);
const largePagination = await openAndVerifyLargeFixture({
const documentsAfterTargetSwitch = requireMcpResult(
await client.callTool({ name: "list_documents", arguments: {} }, options),
"list_documents after target switch",
);
if (
!Array.isArray(documentsAfterTargetSwitch) ||
documentsAfterTargetSwitch.length !== firstDocuments.length + 1
)
throw new Error("A distinct target did not receive one Hopper document");
const reopenedTarget = await client.callTool(
{ name: "open_binary", arguments: { path: targetB } },
options,
);
if (reopenedTarget.isError === true)
throw new Error(textValue(reopenedTarget));
const documentsAfterReopen = requireMcpResult(
await client.callTool({ name: "list_documents", arguments: {} }, options),
"list_documents after reopening target B",
);
if (
!Array.isArray(documentsAfterReopen) ||
documentsAfterReopen.length !== documentsAfterTargetSwitch.length
)
throw new Error(
"Reopening the same target created a duplicate Hopper document",
);
const largeInventory = await openAndVerifyLargeFixture({
client,
options,
normalizedResult: requireSuccessfulTool,
@@ -454,7 +441,7 @@ try {
segmentCount: firstOverview.segment_count,
analyses: [firstAnalysis, secondAnalysis],
fixtureAnalysis,
largePagination,
largeInventory,
fixtureManifest: fixtureTargets.manifestPath,
bundledMcpRunning,
stderrBytes,
@@ -466,6 +453,8 @@ try {
targetHashes: [targetHashA, targetHashB, fixtureTargets.large.sha256],
switched: true,
secondOverview: verifiedSecondOverview,
documentCountAfterTargetSwitch: documentsAfterTargetSwitch.length,
documentCountAfterSameTargetReopen: documentsAfterReopen.length,
};
} finally {
const keepAlive = setInterval(() => undefined, 100);
scripts/verify-reconstruction-readiness.mjs
+6 -37
View File
@@ -23,7 +23,6 @@ const javascriptPath = join(fixtureRoot, "javascript-cli");
const electronPath = join(fixtureRoot, "electron");
const environment = {
...process.env,
REA_INVESTIGATION_INPUT_ROOTS_JSON: JSON.stringify([fixtureRoot]),
};
const temporaryRoot = await mkdtemp(join(tmpdir(), "rea-readiness-"));
@@ -32,13 +31,11 @@ const nativeEvidence = await runCli(["inspect-artifact", nativePath, "--json"]);
const javascriptCliEvidence = await runCli([
"analyze-javascript-application",
javascriptPath,
"--approved",
"--json",
]);
const electronCliEvidence = await runCli([
"analyze-javascript-application",
electronPath,
"--approved",
"--json",
]);
@@ -62,11 +59,6 @@ try {
catalog.tools,
"evaluate_reconstruction_readiness",
);
const analysisTool = requireTool(
catalog.tools,
"analyze_javascript_application",
);
assertExactAstLimit(analysisTool);
assert.equal(
readinessTool.annotations?.readOnlyHint,
false,
@@ -141,14 +133,7 @@ try {
const projectedReport = requireObject(projection.result);
assert.equal(projectedReport.report_digest, cliReport.report_digest);
assert.equal(projectedReport.status, "pass");
const reportUri = projectedReport.report_resource_uri;
assert.equal(typeof reportUri, "string");
const resource = await client.readResource({ uri: reportUri });
const resourceContent = resource.contents[0];
if (resourceContent === undefined || !("text" in resourceContent))
throw new TypeError("Readiness report resource is missing JSON text");
const retained = JSON.parse(resourceContent.text);
assert.equal(retained.report.report_digest, cliReport.report_digest);
assert.equal(projectedReport.report_digest, cliReport.report_digest);
const tampered = structuredClone(input);
tampered.replay.expected_source_digest = cliReport.source_digest;
@@ -203,20 +188,13 @@ async function runCli(arguments_) {
}
async function analyzeMcp(client_, inputPath) {
const result = await client_.callTool({
const response = await client_.callTool({
name: "analyze_javascript_application",
arguments: { input_path: inputPath, approved: true },
arguments: { input_path: inputPath },
});
assert.notEqual(result.isError, true);
const projected = requireObject(result.structuredContent);
assert.equal(typeof projected.evidence_id, "string");
const evidence = await client_.readResource({
uri: `rea://evidence/${projected.evidence_id}`,
});
const content = evidence.contents[0];
if (content === undefined || !("text" in content))
throw new TypeError("Analysis Evidence resource is missing JSON text");
return parseEvidence(JSON.parse(content.text));
assert.notEqual(response.isError, true);
const projected = requireObject(response.structuredContent);
return { evidence_id: projected.evidence_id };
}
function requireTool(tools, name) {
@@ -225,15 +203,6 @@ function requireTool(tools, name) {
return tool;
}
function assertExactAstLimit(tool) {
const properties = requireObject(tool.inputSchema.properties);
const limits = requireObject(properties.limits);
const limitProperties = requireObject(limits.properties);
const astLimit = requireObject(limitProperties.max_ast_nodes);
assert.equal(astLimit.default, 2_000_000, "effective AST limit is hidden");
assert.equal(astLimit.maximum, 20_000_000, "schema AST limit is hidden");
}
function requireObject(value) {
if (typeof value !== "object" || value === null || Array.isArray(value))
throw new TypeError("Expected an object");
skills/reverse-engineer-anything/SKILL.md
+5 -6
View File
@@ -3,8 +3,8 @@ name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, and browser applications with REA. Use shipped-artifact or approved runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "23"
tool_count: 116
catalog_digest: "6624247db5bcf7b52ce53173bbc0ac11bd51e9217e9e3be4298b755bcc60ea50"
tool_count: 115
catalog_digest: "94f7cd1ac629a6bbffa77ba3b3d3f2e208e15df821bc0b14db4f81692d6f8eaa"
---
# REA
@@ -43,10 +43,9 @@ is unavailable.
## Work summary-first
Start with the default summary projection. Do not repeat an identical tool call.
Do not fetch full Evidence or a full application graph unless a specific claim
requires detail absent from the summary. For JavaScript graphs, follow the
paged resource URIs returned by the summary and fetch only the relevant page.
Start with the default result and use its inline Evidence and graph context.
Do not repeat an identical tool call. Make a focused follow-up only when the
returned result leaves a specific question unanswered.
Every conclusion must distinguish observations, inferences, and unknowns. Cite
Evidence IDs, preserve limitations and incomplete coverage, and never imply
src/application/AnalysisClientCleanup.ts
+3 -1
View File
@@ -7,7 +7,9 @@ import type { AnalysisClient, ExecutionOptions } from "./AnalysisProvider.js";
export const closeAnalysisClient = async (
client: AnalysisClient,
providerId: string,
options: Pick<ExecutionOptions, "progress"> = {},
options: Pick<ExecutionOptions, "progress"> & {
readonly retainDocument?: boolean;
} = {},
): Promise<Result<null, AnalysisError>> => {
try {
if (client.closeWithOutcome !== undefined)
src/application/AnalysisProvider.ts
+3 -3
View File
@@ -102,7 +102,9 @@ export interface AnalysisClient extends AnalysisOperationPort {
requestActivitySnapshots?(): readonly ProviderRequestActivitySnapshot[];
/** Close with a typed result when the provider can verify cleanup. */
closeWithOutcome?(
options?: Pick<ExecutionOptions, "progress">,
options?: Pick<ExecutionOptions, "progress"> & {
readonly retainDocument?: boolean;
},
): Promise<Result<null, AnalysisError>>;
close(): Promise<void>;
}
@@ -166,8 +168,6 @@ interface CapabilityLimits {
export type CapabilityDescriptor = CapabilityAvailability & {
readonly provider: ProviderIdentity;
readonly operation: Exclude<AnalysisOperation, "health">;
readonly inputContractVersion: number;
readonly outputContractVersion: number;
readonly pagination: "none" | "offset" | "cursor";
readonly exhaustive: boolean;
readonly effects: CapabilityEffects;
src/application/AnalysisProviderRegistry.test.ts
-3
View File
@@ -365,7 +365,6 @@ const candidate = (
? null
: createAnalysisProfile(
{ id, name: identity.name, version: "1" },
1,
{ fixture: id },
),
compatibility: {},
@@ -395,8 +394,6 @@ const candidate = (
const capability = (provider: ProviderIdentity): CapabilityDescriptor => ({
provider,
operation: "address_name",
inputContractVersion: 1,
outputContractVersion: 1,
available: true,
reason: null,
pagination: "none",
src/application/AnalysisSnapshotCache.test.ts
+3 -6
View File
@@ -54,11 +54,9 @@ describe("analysis snapshot cache partitioning", () => {
analysisProfile: ANALYSIS_SNAPSHOT_PROFILE,
limitations: [expect.stringContaining("local REA analysis snapshot")],
});
const changedProfile = createAnalysisProfile(
ANALYSIS_SNAPSHOT_PROVIDER,
1,
{ loader: "configured-override" },
);
const changedProfile = createAnalysisProfile(ANALYSIS_SNAPSHOT_PROVIDER, {
loader: "configured-override",
});
expect(
cache.lookup(ANALYSIS_SNAPSHOT_TARGET, changedProfile, "address_name", {
address: "0x1000",
@@ -67,7 +65,6 @@ describe("analysis snapshot cache partitioning", () => {
).toBeUndefined();
const changedProviderProfile = createAnalysisProfile(
{ id: "other", name: "Other", version: "1" },
1,
{ loader: "mach-o-arm64" },
);
expect(
src/application/ApplicationWorkflowEvidenceResolver.ts
+1 -6
View File
@@ -30,11 +30,7 @@ import {
const APPLICATION_GRAPH_IDENTITIES = [
{
operation: "analyze_javascript_application",
predicate: "rea.javascript-application-analysis/v1",
},
{
operation: "analyze_javascript_application",
predicate: "rea.javascript-application-analysis/v2",
predicate: "rea.javascript-application-analysis",
},
{
operation: "reconcile_javascript_runtime",
@@ -82,7 +78,6 @@ export const resolveTraceApplicationFeatureRequestValidated = (
native_observations: [...input.native_observations, ...native.value],
seed: input.seed,
direction: input.direction,
limits: input.limits,
};
const parsed = traceApplicationFeatureInputSchema.safeParse(raw);
return parsed.success
src/application/ArtifactExtraction.ts
-1
View File
@@ -188,7 +188,6 @@ const createExtractionResult = (
extracted: readonly ExtractedOccurrence[],
): ArtifactExtractionResult => {
const extractionSemantic = {
schema_version: 1 as const,
source_manifest_id: inventory.manifest.manifest_id,
selected_occurrence_ids: selected
.map(({ occurrence }) => occurrence.occurrence_id)
src/application/ArtifactGraphConstruction.ts
+1 -1
View File
@@ -149,7 +149,7 @@ export const createArtifactNode = (input: {
readonly contentState: ArtifactNode["content_state"];
readonly limitations?: readonly string[];
}): ArtifactNode => ({
artifact_id: `art_${digestCanonical({ schema_version: 1, sha256: input.sha256 })}`,
artifact_id: `art_${digestCanonical({ sha256: input.sha256 })}`,
kind: input.kind,
format: input.format,
sha256: input.sha256,
src/application/ArtifactInventory.ts
+21
View File
@@ -41,6 +41,27 @@ export const inventoryArtifact = async (
return paginateArtifactInventory(snapshot, page);
};
/** Inventory one artifact and return every graph collection in one response. */
export const inventoryArtifactFully = async (
inputPath: string,
limits: ArtifactLimits,
options: {
readonly signal?: AbortSignal;
readonly nativeMount?: ArtifactNativeMountPolicy;
readonly integrity?: ArtifactIntegrityPolicy;
} = {},
): Promise<ArtifactInventoryResult> => {
const snapshot = await scanArtifactInventory(inputPath, limits, options);
return paginateArtifactInventory(snapshot, {
nodeOffset: 0,
nodeLimit: Math.max(1, snapshot.nodes.length),
occurrenceOffset: 0,
occurrenceLimit: Math.max(1, snapshot.occurrences.length),
edgeOffset: 0,
edgeLimit: Math.max(1, snapshot.edges.length),
});
};
/** Scan an artifact once and retain the complete immutable graph for projection. */
export const scanArtifactInventory = async (
inputPath: string,
src/application/ArtifactInventory/scanCanonical.ts
-2
View File
@@ -258,9 +258,7 @@ const buildManifest = ({
readonly orderedEdges: readonly { edge_id: string }[];
}): ArtifactInventoryResult["manifest"] =>
artifactInventoryResultSchema.shape.manifest.parse({
schema_version: 1,
manifest_id: `agm_${digestCanonical({
schema_version: 1,
root_artifact_id: rootNode.artifact_id,
graph_sha256: graphSha256,
})}`,
src/application/AuthorizedArtifactInventory.ts
+1 -18
View File
@@ -1,35 +1,18 @@
import { realpath } from "node:fs/promises";
import type { ArtifactLimits } from "../artifacts/ArtifactReader.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import { isPathWithinRoot } from "../domain/localPath.js";
import {
scanCanonicalArtifactInventory,
type ArtifactInventoryOptions,
type ArtifactInventorySnapshot,
} from "./ArtifactInventory.js";
import { canonicalizeConfiguredRoots } from "./ConfiguredRoots.js";
/** Resolve, authorize, and scan one artifact without a second path resolution. */
export const scanAuthorizedArtifactInventory = async (
inputPath: string,
roots: readonly string[],
limits: ArtifactLimits,
options: ArtifactInventoryOptions = {},
): Promise<ArtifactInventorySnapshot> => {
if (roots.length === 0)
throw new ArtifactReaderFailure(
"path",
"Artifact input roots are disabled",
);
const [path, approved] = await Promise.all([
realpath(inputPath),
canonicalizeConfiguredRoots(roots),
]);
if (!approved.some((root) => isPathWithinRoot(root, path)))
throw new ArtifactReaderFailure(
"path",
"Artifact path is outside approved input roots",
);
const path = await realpath(inputPath);
return scanCanonicalArtifactInventory(path, limits, options);
};
src/application/BinarySession.ts
+12 -4
View File
@@ -228,7 +228,9 @@ export class BinarySession
/** Close the active target, if any. */
close(
options: Pick<ExecutionOptions, "progress"> = {},
options: Pick<ExecutionOptions, "progress"> & {
readonly retainProviderDocuments?: boolean;
} = {},
): Promise<Result<null, AnalysisError>> {
return this.#serialize(async () => {
const previous = this.#active;
@@ -240,7 +242,14 @@ export class BinarySession
: await closeAnalysisClient(
previous.client,
previous.route.identity.id,
options,
{
...(options.progress === undefined
? {}
: { progress: options.progress }),
...(options.retainProviderDocuments === true
? { retainDocument: true }
: {}),
},
);
this.clearSessionRecords();
this.#clearRuntimeAvailability();
@@ -340,7 +349,6 @@ export class BinarySession
"ProviderAdapterError",
"HopperProcessError",
"HopperStartError",
"HopperRemoteError",
].includes(result.error._tag)
) {
const capabilities = this.#active?.route.capabilities;
@@ -351,7 +359,7 @@ export class BinarySession
changed =
this.#markRuntimeUnavailable(
descriptor.operation,
"Provider became unavailable during this session.",
`${result.error._tag}: ${result.error.message}`,
) || changed;
if (changed) this.#emitAvailabilityChanged();
}
src/application/BinarySessionPort.ts
+3 -16
View File
@@ -27,18 +27,6 @@ import type {
ProviderIdentity,
} from "./AnalysisProvider.js";
/** Compact identity and summary for one retained immutable Evidence bundle. */
export interface EvidenceBundleSnapshot {
readonly bundleDigest: string;
readonly bundleVersion: 2;
readonly bytes: number;
readonly records: number;
readonly unknowns: number;
readonly scope: "session";
readonly survivesSession: false;
readonly uri: string;
}
/** Target lifecycle used by CLI and MCP without exposing a concrete provider. */
export interface BinarySessionPort extends AnalysisOperationPort {
open(
@@ -51,7 +39,9 @@ export interface BinarySessionPort extends AnalysisOperationPort {
},
): Promise<Result<BinaryTarget, AnalysisError>>;
close(
options?: Pick<ExecutionOptions, "progress">,
options?: Pick<ExecutionOptions, "progress"> & {
readonly retainProviderDocuments?: boolean;
},
): Promise<Result<null, AnalysisError>>;
status(): JsonValue;
activeTarget(): BinaryTarget | undefined;
@@ -61,9 +51,6 @@ export interface BinarySessionPort extends AnalysisOperationPort {
hasEvidence(evidenceId: string): boolean;
evidenceById(evidenceId: string): Evidence | undefined;
exportEvidenceBundle(): EvidenceBundle;
snapshotEvidenceBundle(): Result<EvidenceBundleSnapshot, EvidenceLimitError>;
releaseEvidenceBundle(bundleDigest: string): boolean;
retainedEvidenceBundle(digest: string): string | undefined;
importEvidenceBundle(
bundle: unknown,
): Result<number, EvidenceIntegrityError | EvidenceLimitError>;
src/application/BinarySessionRecords.ts
+1 -67
View File
@@ -3,10 +3,7 @@ import type { AnalysisSnapshot } from "../domain/analysisSnapshot.js";
import type { BinaryTarget } from "../domain/binaryTarget.js";
import { createEvidence, type Evidence } from "../domain/evidence.js";
import type { EvidenceBundle } from "../domain/evidenceBundle.js";
import {
evidenceBundleForTarget,
serializeEvidenceBundle,
} from "../domain/evidenceBundle.js";
import { evidenceBundleForTarget } from "../domain/evidenceBundle.js";
import {
EvidenceIntegrityError,
EvidenceLimitError,
@@ -28,7 +25,6 @@ import type {
UpdateUnknownInput,
} from "../domain/residualUnknown.js";
import { err, ok, type Result } from "../domain/result.js";
import type { EvidenceBundleSnapshot } from "./BinarySessionPort.js";
import type {
AnalysisExecution,
AnalysisOperation,
@@ -48,15 +44,11 @@ export interface ActiveAnalysisBinding {
/** Owns session evidence, snapshots, workspaces, and residual unknowns. */
export abstract class BinarySessionRecords {
static readonly MAX_RETAINED_BUNDLES = 16;
static readonly MAX_RETAINED_BUNDLE_BYTES = 64 * 1024 * 1024;
readonly #evidence = new EvidenceLedger({
maxRecords: 10_000,
maxBytes: 64 * 1024 * 1024,
});
readonly #snapshot = new AnalysisSnapshotCache();
readonly #retainedBundles = new Map<string, string>();
#retainedBundleBytes = 0;
readonly #investigationWorkspaces = new Map<string, InvestigationWorkspace>();
readonly #coverageWorkspaces = new Map<
string,
@@ -94,61 +86,6 @@ export abstract class BinarySessionRecords {
return this.#evidence.export();
}
snapshotEvidenceBundle(): Result<EvidenceBundleSnapshot, EvidenceLimitError> {
const bundle = this.#evidence.export();
const encoded = serializeEvidenceBundle(bundle);
const digest = createHash("sha256").update(encoded).digest("hex");
const existing = this.#retainedBundles.get(digest);
if (existing === undefined) {
if (
this.#retainedBundles.size >= BinarySessionRecords.MAX_RETAINED_BUNDLES
)
return err(
new EvidenceLimitError(
"records",
BinarySessionRecords.MAX_RETAINED_BUNDLES,
),
);
const bytes = Buffer.byteLength(encoded);
if (
this.#retainedBundleBytes + bytes >
BinarySessionRecords.MAX_RETAINED_BUNDLE_BYTES
)
return err(
new EvidenceLimitError(
"bytes",
BinarySessionRecords.MAX_RETAINED_BUNDLE_BYTES,
),
);
this.#retainedBundles.set(digest, encoded);
this.#retainedBundleBytes += bytes;
}
return ok({
bundleDigest: digest,
bundleVersion: 2,
bytes: Buffer.byteLength(existing ?? encoded),
records: bundle.records.length,
unknowns: bundle.unknowns.length,
scope: "session",
survivesSession: false,
uri: `rea://evidence-bundle/${digest}`,
});
}
retainedEvidenceBundle(digest: string): string | undefined {
return this.#retainedBundles.get(digest);
}
/** Release one immutable bundle so bounded retention has an explicit recovery path. */
releaseEvidenceBundle(bundleDigest: string): boolean {
const encoded = this.#retainedBundles.get(bundleDigest);
if (encoded === undefined) return false;
this.#retainedBundles.delete(bundleDigest);
this.#retainedBundleBytes -= Buffer.byteLength(encoded);
this.#emitSnapshotChanged();
return true;
}
importEvidenceBundle(
bundle: unknown,
): Result<number, EvidenceIntegrityError | EvidenceLimitError> {
@@ -263,8 +200,6 @@ export abstract class BinarySessionRecords {
this.#evidence.clear();
this.#snapshot.clear();
this.#snapshotInvalidated = false;
this.#retainedBundles.clear();
this.#retainedBundleBytes = 0;
this.#emitSnapshotChanged();
}
@@ -415,4 +350,3 @@ const sortedWorkspaces = <
left.revision - right.revision,
)
.map((workspace) => structuredClone(workspace));
import { createHash } from "node:crypto";
src/application/BinarySessionStatus.ts
-3
View File
@@ -169,7 +169,6 @@ const processLineageStatus = (
: {
status: observation.status,
observed_at: observation.observedAt,
schema_version: observation.lineage.schemaVersion,
launcher_pid: observation.lineage.launcherPid,
launcher_parent_pid: observation.lineage.launcherParentPid,
process_group_id: observation.lineage.processGroupId,
@@ -194,8 +193,6 @@ const capabilityStatus = (
available: descriptor.available,
reason: descriptor.reason,
availability_code: descriptor.availabilityCode ?? null,
input_contract_version: descriptor.inputContractVersion,
output_contract_version: descriptor.outputContractVersion,
pagination: descriptor.pagination,
exhaustive: descriptor.exhaustive,
effects: {
src/application/BrowserEvidence.ts
+2 -3
View File
@@ -63,7 +63,7 @@ type BrowserEvidenceOperation =
| "capture_web_screenshot"
| "compare_web_screenshots";
/** Create Evidence v2 for a policy-scoped external browser observation. */
/** Create Evidence for a policy-scoped external browser observation. */
export const createBrowserEvidence = (
operation: BrowserEvidenceOperation,
input: BrowserEvidenceInput,
@@ -125,8 +125,7 @@ const browserParameters = (
cdp_endpoint: input.cdp_endpoint,
allowed_origins: input.allowed_origins,
};
if (!("target_id" in input))
return { ...scope, offset: input.offset, limit: input.limit };
if (!("target_id" in input)) return scope;
if ("screenshot_approved" in input)
return {
...scope,
src/application/BrowserScenarioEvidence.ts
+1 -1
View File
@@ -33,7 +33,7 @@ const browserScenarioParameters = (
limits: scenario.limits,
});
/** Create Evidence v2 without retaining resolved scenario secret values. */
/** Create Evidence without retaining resolved scenario secret values. */
export const createBrowserScenarioEvidence = (
scenario: BrowserScenario,
capture: BrowserScenarioCapture,
src/application/CallPathTracing.ts
+2 -44
View File
@@ -15,9 +15,6 @@ export interface CallPathTraceInput {
readonly start: string;
readonly goal?: string | undefined;
readonly direction: "forward" | "backward";
readonly max_depth: number;
readonly max_nodes: number;
readonly max_operations: number;
}
interface TraceState {
@@ -38,9 +35,7 @@ interface TraceState {
readonly address: string;
readonly error: ReturnType<typeof projectAnalysisError>;
}>;
readonly frontier: Set<string>;
readonly residual: Set<string>;
operations: number;
queueIndex: number;
traversalPath: string[];
}
@@ -59,7 +54,7 @@ interface DiscoveredEdge {
readonly discoveryDepth: number;
}
/** Trace one deterministic bounded direct caller/callee path. */
/** Trace direct caller/callee relationships until the graph is exhausted or the goal is reached. */
export const traceCallPath = async (
call: AnalysisCall,
input: CallPathTraceInput,
@@ -74,17 +69,6 @@ export const traceCallPath = async (
if (address === undefined) break;
const current = state.visited.get(address);
if (current === undefined) continue;
if (current.depth >= input.max_depth) {
state.frontier.add(address);
state.residual.add("Call-path traversal reached the depth limit.");
continue;
}
if (state.operations >= input.max_operations) {
recordPendingFrontier(state, input);
state.residual.add("Call-path traversal reached the operation budget.");
break;
}
state.operations += 1;
const found = await expandAddress(call, {
input,
state,
@@ -102,9 +86,7 @@ const createTraceState = (input: CallPathTraceInput): TraceState => ({
queue: [input.start],
edges: new Map(),
failures: [],
frontier: new Set(),
residual: new Set(),
operations: 0,
queueIndex: 0,
traversalPath: input.goal === input.start ? [input.start] : [],
});
@@ -152,12 +134,6 @@ const expandAddress = async (
}
continue;
}
if (state.visited.size >= input.max_nodes) {
if (state.frontier.size < input.max_nodes)
state.frontier.add(relatedAddress);
state.residual.add("Call-path traversal reached the node limit.");
continue;
}
state.visited.set(relatedAddress, {
depth: current.depth + 1,
path,
@@ -182,10 +158,6 @@ const recordEdge = (
input.direction === "forward" ? edge.relatedAddress : edge.address;
const edgeKey = `${sourceAddress}\u0000${targetAddress}`;
if (state.edges.has(edgeKey)) return;
if (state.edges.size >= input.max_nodes) {
state.residual.add("Call-path traversal reached the edge limit.");
return;
}
state.edges.set(edgeKey, {
source_address: sourceAddress,
target_address: targetAddress,
@@ -193,14 +165,6 @@ const recordEdge = (
});
};
const recordPendingFrontier = (
state: TraceState,
input: CallPathTraceInput,
): void => {
for (const pending of state.queue.slice(state.queueIndex - 1))
if (state.frontier.size < input.max_nodes) state.frontier.add(pending);
};
const projectTrace = (input: CallPathTraceInput, state: TraceState) => ({
start: input.start,
goal: input.goal ?? null,
@@ -225,19 +189,13 @@ const projectTrace = (input: CallPathTraceInput, state: TraceState) => ({
),
traversal_path: state.traversalPath,
failures: state.failures,
frontier: [...state.frontier].sort(compareAddress),
limits: {
max_depth: input.max_depth,
max_nodes: input.max_nodes,
max_operations: input.max_operations,
traversal: {
nodes_visited: state.visited.size,
operations_used: state.operations,
},
truncated: state.residual.size > 0,
residual_unknowns: [...state.residual],
limitations: [
"Direct provider call relationships may omit unresolved indirect calls.",
"A not_reached goal is only absent from this bounded traversal, not from the complete program.",
],
});
src/application/CapabilityInventory.contract.test.ts
-1
View File
@@ -10,7 +10,6 @@ describe("capability inventory contract", () => {
{
processCaptureEnabled: false,
evidenceFileRoots: 0,
investigationInputRoots: 0,
},
);
const unavailable = inventory.find(({ available }) => !available);
src/application/CapabilityInventory.test.ts
-2
View File
@@ -5,7 +5,6 @@ import { buildCapabilityInventory } from "./CapabilityInventory.js";
const enabledPolicy: Parameters<typeof buildCapabilityInventory>[1] = {
processCaptureEnabled: true,
evidenceFileRoots: 1,
investigationInputRoots: 1,
browserObservationEnabled: true,
browserScenarioEnabled: true,
electronObservationEnabled: true,
@@ -180,7 +179,6 @@ describe("capability inventory: caller guidance", () => {
...enabledPolicy,
processCaptureEnabled: false,
evidenceFileRoots: 0,
investigationInputRoots: 0,
browserObservationEnabled: false,
electronObservationEnabled: false,
v8InspectorObservationEnabled: false,
src/application/CapabilityInventory.ts
+1 -9
View File
@@ -29,7 +29,6 @@ type ProviderDescriptor = ProviderCapability;
export type AvailabilityPolicy = {
readonly processCaptureEnabled: boolean;
readonly evidenceFileRoots: number;
readonly investigationInputRoots: number;
readonly browserObservationEnabled?: boolean;
readonly browserScenarioEnabled?: boolean;
readonly electronObservationEnabled?: boolean;
@@ -190,16 +189,9 @@ const availabilityFor = (context: AvailabilityContext): Availability => {
const javascriptApplicationAvailability = ({
name,
policy,
}: AvailabilityContext): Availability | null => {
if (name === "analyze_javascript_application")
return policy.investigationInputRoots === 0
? {
reason: "policy_disabled",
remediation:
"Configure an exact REA_INVESTIGATION_INPUT_ROOTS_JSON root in the MCP registration, then restart the registered MCP server or client.",
}
: { reason: "available", remediation: null };
return { reason: "available", remediation: null };
return name === "reconcile_javascript_runtime"
? { reason: "available", remediation: null }
: null;
src/application/CrossVersionInventory.ts
+2 -5
View File
@@ -42,7 +42,6 @@ export type VersionInventoryScanner = typeof scanAuthorizedArtifactInventory;
export interface VersionInventoryScanOptions {
readonly leftPath: string;
readonly rightPath: string;
readonly inputRoots: readonly string[];
readonly limits: ArtifactLimits;
readonly signal?: AbortSignal;
readonly integrity: ArtifactIntegrityPolicy;
@@ -54,11 +53,11 @@ export const scanVersionInventories = async (
scanner: VersionInventoryScanner = scanAuthorizedArtifactInventory,
): Promise<VersionSnapshots> => {
const [left, right] = await Promise.all([
scanner(options.leftPath, options.inputRoots, options.limits, {
scanner(options.leftPath, options.limits, {
signal: options.signal,
integrity: options.integrity,
}),
scanner(options.rightPath, options.inputRoots, options.limits, {
scanner(options.rightPath, options.limits, {
signal: options.signal,
integrity: options.integrity,
}),
@@ -69,7 +68,6 @@ export const scanVersionInventories = async (
/** Authorize and scan both inputs once under the same bounded traversal policy. */
export const scanVersions = async (
input: CrossVersionInvestigationInput,
inputRoots: readonly string[],
signal?: AbortSignal,
integrityContinueEnabled = false,
): Promise<Result<VersionSnapshots, AnalysisError>> => {
@@ -90,7 +88,6 @@ export const scanVersions = async (
await scanVersionInventories({
leftPath: input.left_path,
rightPath: input.right_path,
inputRoots,
limits,
...(signal === undefined ? {} : { signal }),
integrity,
src/application/CrossVersionInvestigation.ts
-7
View File
@@ -56,12 +56,10 @@ export interface CrossVersionInvestigationOutcome {
/** Runtime authority and cancellation capabilities for one investigation. */
export interface CrossVersionInvestigationExecution {
readonly inputRoots: readonly string[];
readonly session?: BinarySessionPort;
readonly signal?: AbortSignal;
readonly progress?: ProgressReporter;
readonly integrityContinueEnabled?: boolean;
readonly authorizeInputRead?: () => Promise<Result<null, AnalysisError>>;
readonly authorizeWorkspaceWrite?: () => Promise<Result<null, AnalysisError>>;
}
@@ -106,10 +104,6 @@ export const runCrossVersionInvestigationValidated = async (
execution.session,
);
}
if (execution.authorizeInputRead !== undefined) {
const authorized = await execution.authorizeInputRead();
if (!authorized.ok) return authorized;
}
await execution.progress?.report({
phase: "scan_versions",
completed: 0,
@@ -119,7 +113,6 @@ export const runCrossVersionInvestigationValidated = async (
const snapshots = await scanVersions(
input,
execution.inputRoots,
execution.signal,
execution.integrityContinueEnabled,
);
src/application/DirectAnalysisStatus.ts
+7 -82
View File
@@ -1,10 +1,7 @@
import { parseConfig } from "../config.js";
import { projectAnalysisError } from "../domain/errors.js";
import { jsonObjectSchema, type JsonValue } from "../domain/jsonValue.js";
import {
createServerIdentity,
createServerIdentitySummary,
} from "../serverIdentity.js";
import { createServerIdentity } from "../serverIdentity.js";
import { silentLogger, type Logger } from "../logger.js";
import { createBinarySession } from "./runtime.js";
@@ -26,82 +23,10 @@ const runSessionStatus = async (
}
};
/** List provider identities, selection, and availability without capability bodies. */
export const runProviderStatus = async (
logger: Logger = silentLogger,
detail: "summary" | "full" = "summary",
): Promise<JsonValue> => {
const fullStatus = await runSessionStatus(logger);
if (detail === "full") return fullStatus;
const parsedStatus = jsonObjectSchema.safeParse(fullStatus);
if (!parsedStatus.success || parsedStatus.data.error !== undefined)
return fullStatus;
const status = parsedStatus.data;
return {
open: status.open ?? false,
provider: status.provider ?? null,
providers: status.providers ?? [],
analysis_provider_binding: status.analysis_provider_binding ?? null,
analysis_provider_candidates: projectJsonObjectArray(
status.analysis_provider_candidates,
["provider", "availability", "target_support", "selected"],
),
server_identity: createServerIdentitySummary({
startedAt: new Date().toISOString(),
}),
};
};
/** List complete provider identities, capabilities, and availability. */
export const runProviderStatus = (logger: Logger = silentLogger) =>
runSessionStatus(logger);
/** List concise operation availability, with explicit full descriptor opt-in. */
export const runCapabilityStatus = async (
logger: Logger = silentLogger,
detail: "summary" | "full" = "summary",
): Promise<JsonValue> => {
const fullStatus = await runSessionStatus(logger);
if (detail === "full") return fullStatus;
const parsedStatus = jsonObjectSchema.safeParse(fullStatus);
if (!parsedStatus.success || parsedStatus.data.error !== undefined)
return fullStatus;
const status = parsedStatus.data;
const capabilities = projectJsonObjectArray(status.capabilities, [
"operation",
"available",
"reason",
]);
const available = capabilities.filter(
(candidate) =>
typeof candidate === "object" &&
candidate !== null &&
"available" in candidate &&
candidate.available === true,
).length;
return {
open: status.open ?? false,
summary: {
total: capabilities.length,
available,
unavailable: capabilities.length - available,
},
capabilities,
analysis_provider_binding: status.analysis_provider_binding ?? null,
server_identity: createServerIdentitySummary({
startedAt: new Date().toISOString(),
}),
};
};
const projectJsonObjectArray = (
value: JsonValue | undefined,
keys: readonly string[],
): JsonValue[] =>
Array.isArray(value)
? value.map((candidate) => {
const parsed = jsonObjectSchema.safeParse(candidate);
if (!parsed.success) return candidate;
return Object.fromEntries(
keys.flatMap((key) =>
parsed.data[key] === undefined ? [] : [[key, parsed.data[key]]],
),
);
})
: [];
/** List complete operation descriptors and availability. */
export const runCapabilityStatus = (logger: Logger = silentLogger) =>
runSessionStatus(logger);
src/application/Doctor.ts
+1 -1
View File
@@ -228,7 +228,7 @@ const collectDoctorIdentity = async (
live_server: {
state: "unknown",
remediation:
"Compare rea://server/identity or binary_session from the active client; an on-disk registration cannot prove the running server version.",
"Call binary_session in the active client to inspect the running server identity.",
},
installations: {
paths: installationPaths,
src/application/DoctorProjection.ts
-56
View File
@@ -1,56 +0,0 @@
import type { DoctorCheck, DoctorReport } from "./Doctor.js";
/** Project doctor diagnostics for concise CLI output or explicit full detail. */
export const projectDoctorReport = (
report: DoctorReport,
detail: "summary" | "full",
): DoctorReport | ReturnType<typeof doctorSummary> =>
detail === "full" ? report : doctorSummary(report);
const doctorSummary = (report: DoctorReport) => ({
healthy: report.healthy,
environment_healthy: report.environment_healthy,
scope: report.scope,
...(report.hopperPath === undefined
? {}
: { hopper_path: report.hopperPath }),
providers: (report.providerInspections ?? []).map((provider) => ({
id: provider.id,
configured: provider.configured,
available: provider.available,
provider_version: provider.providerVersion,
failed_checks: provider.checks.filter(({ ok }) => !ok).map(checkSummary),
})),
failed_scope_checks: report.scope_checks
.filter(({ ok }) => !ok)
.map(checkSummary),
informational_drift_count: report.informational_checks.filter(({ ok }) => !ok)
.length,
identity:
report.identity === undefined
? null
: {
cli_package_version: report.identity.cli_package_version,
expected_skill_version: report.identity.expected_skill_version,
catalog: {
counts: report.identity.catalog.counts,
digests: report.identity.catalog.digests,
},
installations: report.identity.installations,
skill: report.identity.skill,
registrations: report.identity.registrations,
},
});
const checkSummary = (
check: Pick<DoctorCheck, "name" | "classification" | "detail"> & {
readonly remediation?: string | null;
},
) => ({
name: check.name,
classification: check.classification,
...(check.detail === undefined ? {} : { detail: check.detail }),
...(check.remediation === undefined || check.remediation === null
? {}
: { remediation: check.remediation }),
});
src/application/ElectronActiveEvidence.ts
+1 -2
View File
@@ -8,7 +8,7 @@ import type {
import { jsonValueSchema } from "../domain/jsonValue.js";
import { digestJson } from "./JavaScriptReplayPlanning.js";
/** Create Evidence v2 without retaining arbitrary runtime argument values. */
/** Create Evidence without retaining arbitrary runtime argument values. */
export const createElectronActiveEvidence = (
input: ElectronActiveObservationInput,
result: ElectronActiveObservationResult,
@@ -34,7 +34,6 @@ const scenarioProjection = (
input: ElectronActiveObservationInput,
): EvidenceObservation["parameters"] => ({
approved: input.approved,
schema_version: input.schema_version,
executable_path: input.executable_path,
application_path: input.application_path,
application_root: input.application_root,
src/application/ElectronEvidence.ts
+2 -2
View File
@@ -14,7 +14,7 @@ import type {
type ElectronOperation = "list_electron_targets" | "inspect_electron_page";
/** Create Evidence v2 for one root-confined Electron observation. */
/** Create Evidence for one Electron observation. */
export const createElectronEvidence = (
operation: ElectronOperation,
input: ListElectronTargetsInput | InspectElectronPageInput,
@@ -56,5 +56,5 @@ const parameters = (
source_capture_approved: input.source_capture_approved,
limits: input.limits,
}
: { offset: input.offset, limit: input.limit }),
: {}),
});
src/application/EnhancedLiteralTracing.ts
+62 -76
View File
@@ -1,6 +1,6 @@
import { z } from "zod";
import type { AnalysisOperation } from "./AnalysisProvider.js";
import { AnalysisOutputError, type AnalysisError } from "../domain/errors.js";
import { parseAddressedPage } from "../domain/hopperValues.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
import type {
@@ -16,33 +16,29 @@ type AnalysisCall = (
) => Promise<Result<JsonValue, AnalysisError>>;
type LiteralSearchSource = readonly [
"list_strings" | "list_procedures",
"search_strings" | "search_procedures",
"string" | "procedure",
];
interface ReferenceTraceRequest {
readonly matches: readonly TraceMatch[];
readonly limit: number;
readonly operationBudget: number;
readonly signal?: AbortSignal;
}
const FEATURE_SEARCH_SOURCES: readonly LiteralSearchSource[] = [
["list_strings", "string"],
["list_procedures", "procedure"],
["search_strings", "string"],
["search_procedures", "procedure"],
];
const STRING_SEARCH_SOURCES: readonly LiteralSearchSource[] = [
["list_strings", "string"],
["search_strings", "string"],
];
export interface LiteralTraceInput {
readonly query: string;
readonly case_sensitive: boolean;
readonly limit: number;
readonly max_operations: number;
}
/** Trace a bounded feature or string literal through provider xrefs. */
/** Trace every matching feature or string literal through provider xrefs. */
export const traceLiteralFeature = async (
call: AnalysisCall,
input: LiteralTraceInput,
@@ -53,29 +49,18 @@ export const traceLiteralFeature = async (
scope === "feature" ? FEATURE_SEARCH_SOURCES : STRING_SEARCH_SOURCES;
const searched = await literalMatches(call, input, sources, signal);
if (!searched.ok) return searched;
const residual = new Set<string>();
const traced = await traceReferences(call, {
matches: searched.value.matches,
limit: input.limit,
operationBudget: input.max_operations - searched.value.operations,
...(signal === undefined ? {} : { signal }),
});
if (!traced.ok) return traced;
for (const reason of traced.value.residual) residual.add(reason);
const operations = searched.value.operations + traced.value.operations;
if (operations >= input.max_operations)
residual.add("Investigation reached the operation budget.");
if (searched.value.matches.length >= input.limit)
residual.add("Literal matches reached the configured limit.");
return ok({
query: input.query,
search_mode: "literal",
operations_used: operations,
operation_budget: input.max_operations,
matches: searched.value.matches,
references: traced.value.references,
truncated: residual.size > 0,
residual_unknowns: [...residual],
truncated: false,
residual_unknowns: [],
});
};
@@ -84,53 +69,70 @@ const literalMatches = async (
input: LiteralTraceInput,
sources: readonly LiteralSearchSource[],
signal?: AbortSignal,
): Promise<
Result<{ matches: TraceMatch[]; operations: number }, AnalysisError>
> => {
let operations = 0;
): Promise<Result<{ matches: TraceMatch[] }, AnalysisError>> => {
const matches: TraceMatch[] = [];
const needle = input.case_sensitive ? input.query : input.query.toLowerCase();
for (const [tool, type] of sources) {
let offset = 0;
while (operations < input.max_operations && matches.length < input.limit) {
operations += 1;
const result = await call(tool, { offset, limit: 500 }, signal);
const needles = [...new Set(input.query.trim().split(/\s+/u))];
const matched = new Set<string>();
for (const needle of needles) {
for (const [tool, type] of sources) {
const result = await call(
tool,
{
pattern: needle,
mode: "literal",
case_sensitive: input.case_sensitive,
},
signal,
);
if (!result.ok) return result;
const page = parseAddressedPage(result.value);
if (!page.ok) return page;
for (const item of page.value.items) {
const haystack = input.case_sensitive
? item.name
: item.name.toLowerCase();
if (haystack.includes(needle))
matches.push({ type, address: item.address, value: item.name });
if (matches.length >= input.limit) break;
}
if (!page.value.hasMore || page.value.nextOffset === null) break;
offset = page.value.nextOffset;
const items = z
.array(z.object({ address: z.string(), value: z.string() }))
.safeParse(result.value);
if (!items.success)
return err(
new AnalysisOutputError(
tool,
"provider returned an invalid search result",
),
);
addPageMatches({
items: items.data.map(({ address, value: name }) => ({
address,
name,
})),
type,
matches,
matched,
});
}
}
return ok({ matches, operations });
return ok({ matches });
};
const addPageMatches = (input: {
readonly items: readonly {
readonly address: string;
readonly name: string;
}[];
readonly type: TraceMatch["type"];
readonly matches: TraceMatch[];
readonly matched: Set<string>;
}): void => {
const { items, type, matches, matched } = input;
for (const item of items) {
const key = `${type}\u0000${item.address}`;
if (matched.has(key)) continue;
matches.push({ type, address: item.address, value: item.name });
matched.add(key);
}
};
const traceReferences = async (
call: AnalysisCall,
request: ReferenceTraceRequest,
): Promise<
Result<
{ references: TraceReference[]; operations: number; residual: string[] },
AnalysisError
>
> => {
let operations = 0;
): Promise<Result<{ references: TraceReference[] }, AnalysisError>> => {
const references: TraceReference[] = [];
const residual = new Set<string>();
for (const match of request.matches) {
if (operations >= request.operationBudget) {
residual.add("Reference traversal stopped at the operation budget.");
break;
}
operations += 1;
const xrefs = await call(
"xrefs",
{ address: match.address },
@@ -152,13 +154,6 @@ const traceReferences = async (
"provider returned a non-address value",
),
);
if (operations >= request.operationBudget) {
residual.add(
"Containing-procedure resolution stopped at the operation budget.",
);
break;
}
operations += 1;
const resolved = await call(
"resolve_containing_procedure",
{ address: source },
@@ -170,16 +165,7 @@ const traceReferences = async (
source_address: source,
containing_procedure: resolved.value,
});
if (references.length >= request.limit) {
residual.add("Reference results reached the configured limit.");
break;
}
}
if (references.length >= request.limit) break;
}
return ok({
references,
operations,
residual: [...residual],
});
return ok({ references });
};
src/application/EnhancedToolPagination.ts
-42
View File
@@ -1,42 +0,0 @@
import type { AnalysisOperation } from "./AnalysisProvider.js";
import { AnalysisOutputError, type AnalysisError } from "../domain/errors.js";
import { parseAddressedPage } from "../domain/hopperValues.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
type PageCall = (
name: AnalysisOperation,
arguments_: Readonly<Record<string, JsonValue>>,
signal?: AbortSignal,
) => Promise<Result<JsonValue, AnalysisError>>;
/** Read all addressed entries while enforcing advancing provider pagination. */
export const readAllAddressed = async (input: {
readonly call: PageCall;
readonly tool: "list_names" | "list_procedures";
readonly signal?: AbortSignal;
}) => {
const entries: Array<{ address: string; name: string }> = [];
let offset = 0;
while (true) {
const result = await input.call(
input.tool,
{ offset, limit: 500 },
input.signal,
);
if (!result.ok) return result;
const page = parseAddressedPage(result.value);
if (!page.ok) return page;
entries.push(...page.value.items);
if (!page.value.hasMore || page.value.nextOffset === null)
return ok(entries);
if (page.value.nextOffset <= offset)
return err(
new AnalysisOutputError(
input.tool,
"provider returned a non-advancing pagination offset",
),
);
offset = page.value.nextOffset;
}
};
src/application/EnhancedTools.ts
+34 -37
View File
@@ -1,4 +1,4 @@
import type { z } from "zod";
import { z } from "zod";
import type {
AnalysisOperation,
@@ -35,7 +35,6 @@ import {
type EnhancedResult,
type ValidatedEnhancedCall,
} from "./EnhancedToolTypes.js";
import { readAllAddressed } from "./EnhancedToolPagination.js";
import { traceCallPath } from "./CallPathTracing.js";
import { traceLiteralFeature } from "./EnhancedLiteralTracing.js";
import { projectNativeApiInspection } from "./NativeApiInspection.js";
@@ -128,7 +127,7 @@ export class EnhancedTools {
case "find_xrefs_to_name":
return this.#findXrefs(call.input.name, signal);
case "binary_overview":
return this.#binaryOverview(call.input, signal);
return this.#binaryOverview(signal);
case "analyze_function":
return this.#analyzeFunction(call.input, signal);
case "inspect_native_api":
@@ -208,9 +207,6 @@ export class EnhancedTools {
): EnhancedResult {
const analysisInput = enhancedInputSchemas.analyze_function.parse({
procedure: input.procedure,
include_assembly: true,
max_pseudocode_chars: input.max_pseudocode_chars,
max_instructions: input.max_instructions,
});
const result = await this.#call("analyze_function", analysisInput, signal);
if (!result.ok) return result;
@@ -281,7 +277,6 @@ export class EnhancedTools {
input: {
readonly address: string;
readonly direction: "forward" | "backward";
readonly depth: number;
},
signal?: AbortSignal,
): EnhancedResult {
@@ -297,9 +292,7 @@ export class EnhancedTools {
while (queueIndex < queue.length) {
const current = queue[queueIndex++];
if (current === undefined || current.depth >= input.depth) {
continue;
}
if (current === undefined) continue;
const level = String(current.depth);
graph[level] ??= [];
@@ -330,12 +323,10 @@ export class EnhancedTools {
status: "ok",
calls: [...related.value],
});
if (current.depth + 1 < input.depth) {
for (const address of related.value) {
if (!discovered.has(address)) {
discovered.add(address);
queue.push({ address, depth: current.depth + 1 });
}
for (const address of related.value) {
if (!discovered.has(address)) {
discovered.add(address);
queue.push({ address, depth: current.depth + 1 });
}
}
}
@@ -380,16 +371,13 @@ export class EnhancedTools {
});
}
async #binaryOverview(
input: { detail: "concise" | "detailed"; limit: number },
signal?: AbortSignal,
): EnhancedResult {
async #binaryOverview(signal?: AbortSignal): EnhancedResult {
const [segmentsResult, documentsResult, proceduresResult, stringsResult] =
await Promise.all([
this.#call("list_segments", {}, signal),
this.#call("list_documents", {}, signal),
this.#call("list_procedures", { offset: 0, limit: 1 }, signal),
this.#call("list_strings", { offset: 0, limit: 1 }, signal),
this.#call("list_procedures", {}, signal),
this.#call("list_strings", {}, signal),
]);
if (!segmentsResult.ok) return segmentsResult;
if (!documentsResult.ok) return documentsResult;
@@ -407,14 +395,12 @@ export class EnhancedTools {
return ok({
document: documents.value[0] ?? "unknown",
detail: input.detail,
segments: segments.value
.slice(0, input.limit)
.map(({ name, start, end }) =>
input.detail === "detailed"
? { name, start, end, length: addressDistance(start, end) }
: { name, start, end },
),
segments: segments.value.map(({ name, start, end }) => ({
name,
start,
end,
length: addressDistance(start, end),
})),
segment_count: segments.value.length,
procedure_count: procedureCount.value,
string_count: stringCount.value,
@@ -425,12 +411,19 @@ export class EnhancedTools {
tool: "list_names" | "list_procedures",
signal?: AbortSignal,
) {
return readAllAddressed({
call: (name, arguments_, operationSignal) =>
this.#call(name, arguments_, operationSignal),
tool,
...(signal === undefined ? {} : { signal }),
});
const result = await this.#call(tool, {}, signal);
if (!result.ok) return result;
const parsed = z
.array(z.object({ address: z.string(), value: z.string() }))
.safeParse(result.value);
return parsed.success
? ok(parsed.data.map(({ address, value: name }) => ({ address, name })))
: err(
new AnalysisOutputError(
tool,
"provider returned an invalid inventory",
),
);
}
async #call(
@@ -438,12 +431,16 @@ export class EnhancedTools {
arguments_: Readonly<Record<string, JsonValue>>,
signal?: AbortSignal,
): Promise<Result<JsonValue, AnalysisError>> {
if (signal?.aborted === true) return err(new AnalysisCancelledError(name));
if (isAborted(signal)) return err(new AnalysisCancelledError(name));
const execution = await this.analysis.execute(
name,
arguments_,
signal === undefined ? {} : { signal },
);
if (isAborted(signal)) return err(new AnalysisCancelledError(name));
return execution.ok ? ok(execution.value.result) : execution;
}
}
const isAborted = (signal: AbortSignal | undefined): boolean =>
signal?.aborted === true;
src/application/EvidenceBundleCommands.ts
+1 -1
View File
@@ -49,7 +49,7 @@ export const exportEvidenceBundleCommand = async (
);
};
/** Compare two validated canonical Evidence v2 bundles without session state. */
/** Compare two validated canonical Evidence bundles without session state. */
export const compareEvidenceBundlesCommand = async (input: {
readonly leftPath: string;
readonly rightPath: string;
src/application/EvidenceBundleFiles.ts
+2 -7
View File
@@ -6,10 +6,7 @@ import {
} from "../domain/evidenceBundle.js";
import { EvidenceFileError, EvidenceIntegrityError } from "../domain/errors.js";
import { err, ok, type Result } from "../domain/result.js";
import {
LEGACY_PROCESS_CAPTURE_MESSAGE,
parseProcessCapture,
} from "../domain/processCapture.js";
import { parseProcessCapture } from "../domain/processCapture.js";
import { readBoundedJson, writeBoundedText } from "./BoundedJsonFiles.js";
type EvidenceReadFailure = EvidenceFileError | EvidenceIntegrityError;
@@ -27,9 +24,7 @@ export const readEvidenceBundle = async (
try {
const bundle = parseEvidenceBundle(loaded.value);
for (const record of bundle.records) {
if (record.predicate_type === "rea.process-capture/v3")
throw new TypeError(LEGACY_PROCESS_CAPTURE_MESSAGE);
if (record.predicate_type === "rea.process-capture/v4")
if (record.predicate_type === "rea.process-capture")
parseProcessCapture(record.normalized_result);
}
return ok(bundle);
src/application/InventoryProjectionEvidence.ts
+1 -1
View File
@@ -39,7 +39,7 @@ interface InventoryProjectionOptions<
readonly protocolError: string;
}
/** Parse one inventory projection and wrap its deterministic result in Evidence v2. */
/** Parse one inventory projection and wrap its deterministic result in Evidence. */
export const projectInventoryEvidence = <
Input extends InventoryProjectionInput,
Output extends InventoryProjectionResult,
src/application/InvestigationProviders.ts
+1 -1
View File
@@ -14,7 +14,7 @@ export const REA_WORKFLOW_PROVIDER = {
export const workflowAnalysisProfile = (
upstream: AnalysisProfileCommitment,
): AnalysisProfileCommitment =>
createAnalysisProfile(REA_WORKFLOW_PROVIDER, 1, {
createAnalysisProfile(REA_WORKFLOW_PROVIDER, {
upstream_analysis_profile: upstream,
});
src/application/JavaScriptApplicationEvidence.ts
+2 -4
View File
@@ -10,7 +10,7 @@ import type {
import { jsonValueSchema } from "../domain/jsonValue.js";
import { JAVASCRIPT_APPLICATION_PROVIDER } from "./InvestigationProviders.js";
/** Create Evidence v2 for one deterministic local JavaScript application graph. */
/** Create Evidence for one deterministic local JavaScript application graph. */
export const createJavaScriptApplicationEvidence = (
input: AnalyzeJavaScriptApplicationInput,
result: JavaScriptApplicationAnalysisResult,
@@ -23,7 +23,7 @@ export const createJavaScriptApplicationEvidence = (
},
JAVASCRIPT_APPLICATION_PROVIDER,
{
predicateType: `rea.javascript-application-analysis/v${String(result.schema_version)}`,
predicateType: "rea.javascript-application-analysis",
operation: "analyze_javascript_application",
parameters: parameters(input),
result: jsonValueSchema.parse(result),
@@ -40,7 +40,5 @@ const parameters = (
input: AnalyzeJavaScriptApplicationInput,
): EvidenceObservation["parameters"] => ({
format: input.format,
approved: input.approved,
source_map_read_approved: input.source_map_read_approved,
limits: input.limits,
});
src/application/JavaScriptApplicationEvidenceGraph.ts
+4 -10
View File
@@ -33,10 +33,7 @@ export const parseApplicationGraphEvidence = (
const evidence = parseEvidence(input);
if (
evidence.operation === "analyze_javascript_application" &&
[
"rea.javascript-application-analysis/v1",
"rea.javascript-application-analysis/v2",
].includes(evidence.predicate_type) &&
evidence.predicate_type === "rea.javascript-application-analysis" &&
providerMatches(evidence, JAVASCRIPT_APPLICATION_PROVIDER)
)
return staticApplicationSource(evidence);
@@ -70,12 +67,9 @@ const staticApplicationSource = (
const result = javascriptApplicationAnalysisResultSchema.parse(
evidence.normalized_result,
);
if (
evidence.predicate_type !==
`rea.javascript-application-analysis/v${String(result.schema_version)}`
)
if (evidence.predicate_type !== "rea.javascript-application-analysis")
throw new TypeError(
"JavaScript application Evidence predicate does not match its result schema version",
"JavaScript application Evidence predicate does not match its result shape",
);
analyzeJavaScriptApplicationInputSchema.parse({
input_path: result.input_path,
@@ -87,7 +81,7 @@ const staticApplicationSource = (
graph: result.graph,
kind: "static-application",
rootArtifactSha256: result.root_artifact_sha256,
semanticGraph: result.schema_version === 2 ? result.semantic_graph : null,
semanticGraph: result.semantic_graph,
};
};
Loaded 100 of 535 files, more files were not shown because too many files have changed in this diff. Show more