diff --git a/docs/frida-instrumentation.md b/docs/frida-instrumentation.md index 69866fb2ec..b08ee99618 100644 --- a/docs/frida-instrumentation.md +++ b/docs/frida-instrumentation.md @@ -6,6 +6,22 @@ For remote devices, pass a host with an optional port (`host[:port]`; bracket IP MCP exposes `list_frida_devices`, `list_frida_processes`, `start_frida_session`, `load_frida_script`, `resume_frida_session`, `unload_frida_script`, `frida_session_status`, and `close_frida_session`. A spawned process starts paused so scripts can load first. Closing unloads session scripts and detaches. If a spawn remains paused, REA resumes it only after script cleanup or session detach confirms the instrumentation is released; if both cleanup paths fail, the target stays paused and the cleanup error is retained. REA never kills the target. MCP sessions end with the server process. The one-shot CLI command `instrument-with-frida` accepts one inline script or script file, observes messages for a bounded duration, and returns Evidence with the source hash and captured messages. +For state and capture-coverage checks, pass `{"session_id":"SESSION_ID","detail":"summary"}` to `frida_session_status`. Each call reads the observed session state and retains the complete observation. The reply is an `inspect_analysis_view` Evidence summary with the session identity, state, scripts, `messages_retained`, and `messages_truncated`; message payloads remain in its `parent_evidence_id`. The count covers retained queued messages, not every target event. Omitted `detail` or `detail=complete` preserves the existing complete reply. + +A summary's selected fields look like this (excerpt): + +```json +{ + "summary": { + "state": "running", + "messages_retained": 50, + "messages_truncated": false + } +} +``` + +Use `get_evidence_bundle` with `detail="summary"` and `filters={"evidence_id":"PARENT_ID"}` to discover parent metadata without message payloads. Filters apply only to summary delivery. Use `export_evidence_bundle` for the complete canonical bundle, then select the record whose `evidence_id` matches `normalized_result.parent_evidence_id` from the status summary. To use the existing CLI `inspect-analysis-view`, extract that record into a standalone Evidence JSON file and select it as an Evidence-file source with `view={"kind":"summary"}`. Reading retained Evidence does not refresh a live session or verify a process after exit or PID reuse. + Failed startup cleanup retains acquired Frida handles, including a paused spawn that has not attached, for later cleanup retries. Its remote connection stays leased until session detach and target resume complete. CLI commands await provider cleanup before returning; MCP retains these resources for cleanup when its server closes. Server shutdown stops admitting new Frida operations and waits for admitted startup, discovery, and queued work before releasing their resources. A failed cleanup remains reported and can be retried by closing the server again, rather than treating an unregistered session as fully released. Frida scripts execute inside the selected target with that process's memory, filesystem, and network authority. Message capture is bounded to 16 MiB and reports truncation. Script source is not stored in Evidence; its selected path and SHA-256 digest are. For a file, `source_sha256` identifies the original selected bytes. Invalid UTF-8 continues to use replacement decoding; Evidence then records that limitation and `raw_result.source_representation` with the encoding, decoding mode, and `loaded_source_sha256` of the UTF-8 string passed to Frida. Valid UTF-8 files and inline scripts do not need this additional representation fact. Evidence retains non-authentication target selection such as device or endpoint, mode, program or PID, and spawn arguments. Authentication values remain excluded. Results report the observed session and messages, while unsupported platform metadata remains unknown. A request cancelled before a Frida operation starts has no provider effects. If cancellation arrives during a native Frida call, REA waits for that call to return, then closes any session it created and returns the observed session as a partial observation. A cancelled script load on an existing session releases only its new script before queued operations can run; the session and previously loaded scripts remain available. If cancellation is observed when the native call returns, partial Evidence retains the source identity and captured messages even if that call rejects, including messages emitted during awaited cleanup. The caller's cancellation reason remains primary and a native rejection is retained as a sanitized diagnostic. A failure already observed before cleanup stays primary if cancellation arrives during cleanup. The cancellation cutoff is REA's observation of native completion, rather than a claim about unobservable native event order. `script_id` is REA's load-attempt identity. When creation rejects without returning a handle, `script_cleanup: "no_handle"` means REA acquired no handle to unload; it does not prove that the engine performed no work. Once the provider commits a successful load, later request cancellation does not unload that script; use `unload_frida_script` to release it. If script cleanup fails, the script remains owned and a paused target cannot resume until the pending instrumentation is released. Queued script loads, resumes, and unloads check cancellation before dispatching native effects. Cancellation Evidence separately reports whether a target resume was observed, whether instrumentation cleanup completed, and that final target liveness remains unknown. REA does not kill the target. Remote discovery retains its observations and reports a separate cleanup error if the temporary connection cannot be removed. diff --git a/experiments/analysis-efficiency/frida-status-costs.py b/experiments/analysis-efficiency/frida-status-costs.py new file mode 100644 index 0000000000..4a58002e23 --- /dev/null +++ b/experiments/analysis-efficiency/frida-status-costs.py @@ -0,0 +1,81 @@ +"""Count complete actual-client requests, not only smaller tool-reply bytes.""" +import argparse +import hashlib +import json +from pathlib import Path +import tiktoken + +parser = argparse.ArgumentParser(description=__doc__) +parser.add_argument('--lab', type=Path, required=True) +parser.add_argument('--output', type=Path, required=True) +args = parser.parse_args() +enc = tiktoken.get_encoding('cl100k_base') + + +def tokens(value): + return len(enc.encode(json.dumps(value, ensure_ascii=False, separators=(',', ':'), allow_nan=False), disallowed_special=())) + + +def resources(root): + receipt = json.loads(Path(str(root) + '.log').read_text().splitlines()[-1]) + payload = receipt['payload_resource_observation'] + assert receipt['abort'] is None and receipt['payload_resource_observation_verified'] + assert payload['payload_exit_code'] == 0 and payload['payload_cleanup_verified'] + for events in payload['resource_events'].values(): + assert not any(events.values()) + return {'wall_seconds': receipt['elapsed_seconds'], 'cpu_seconds': payload['cpu_stat']['usage_usec'] / 1e6, + 'family_peak_mib': payload['memory_peak_bytes'] / 1048576, + 'payload_cleanup_verified': True, 'final_whole_scope_counters_verified': receipt['resource_counters_verified']} + + +wire, sdk = [], [] +for root in sorted(args.lab.glob('wire-v2-*')): + if not root.is_dir(): + continue + report = json.loads((root / 'report.json').read_text()) + assert report['status'] in ['passed', 'expected-incomplete-task'] and not report['cleanup_failures'] + assert report['appserver_exit_code'] == 0 and report['owned_scope_drain']['remaining_processes'] == [] + requests = json.loads((root / 'model-requests.json').read_text()) + details = [] + for captured in requests: + body = captured['body'] + namespace = [n for n in body.get('tools', []) if n.get('type') == 'namespace' and n.get('name') == 'mcp__rea_audit'] + assert len(namespace) == 1 and len(namespace[0]['tools']) == 211, 'Unexpected compaction/catalog: audit before comparing' + details.append({'whole_json_tokens_proxy': tokens(body), 'rea_namespace_tokens_proxy': tokens(namespace[0]), 'rea_functions': 211}) + wire.append({'case': root.name, 'task_completed': report['task_completed'], 'mode': report['mode'], + 'budget': report['observations']['tool_output_budget'], 'requests': len(details), 'compactions': report['compaction_requests'], + 'whole_json_tokens_proxy': sum(r['whole_json_tokens_proxy'] for r in details), 'request_details': details, **resources(root)}) +for root in sorted(args.lab.glob('sdk-*')): + if not root.is_dir(): + continue + report = json.loads((root / 'report.json').read_text()) + assert report['status'] == 'passed' and not report['cleanup_failures'] and not report['receipt_failures'] + transcript = json.loads((root / 'transcript.json').read_text()) + catalog = json.loads((root / 'catalog.json').read_text()) + assert len(catalog['tools']) == 211 and report['original_target_exited'] + sdk.append({'case': root.name, 'detail': report['detail'], 'capture': report['capture'], 'tool_calls': len(report['calls']), + 'exported_bundle_bytes': report['exported_bundle_bytes'], 'answer': report['answer'], + 'status_calls': [c for c in report['calls'] if c['name'] == 'frida_session_status'], + 'one_catalog_plus_all_sdk_calls_tokens_proxy': tokens(catalog) + sum(tokens({'name': t['name'], 'input': t['input'], 'reply': t['reply']}) for t in transcript), **resources(root)}) +assert len(sdk) == len(wire) == 4, 'A partial cohort cannot qualify a comparison' +for capture in ['small', 'libc']: + pair = [r for r in sdk if r['capture'] == capture] + assert len(pair) == 2 and {r['detail'] for r in pair} == {'complete', 'summary'} + assert {k: v for k, v in pair[0]['answer'].items() if k != 'pid'} == {k: v for k, v in pair[1]['answer'].items() if k != 'pid'} + reports = [json.loads((args.lab / r['case'] / 'report.json').read_text()) for r in pair] + assert reports[0]['load']['source_sha256'] == reports[1]['load']['source_sha256'] +completed = [r for r in wire if r['budget'] == 300000] +assert len(completed) == 2 and all(r['task_completed'] for r in completed) +answers = [json.loads((args.lab / r['case'] / 'report.json').read_text())['observations']['answers'] for r in completed] +comparable_answers = [[{k: v for k, v in a.items() if k != 'parent_evidence_id'} for a in run] for run in answers] +assert comparable_answers[0] == comparable_answers[1] +result = { + 'scope': 'Real local Frida binding with controlled sleep target and caller-authored libc export inventory. Actual Codex0.163 app-server uses a deterministic local SSE fixture: no AI inference, model usage, quality or billing claim. Whole compact JSON token estimates include all 211 schemas and every request replay. Incomplete default baseline is excluded from completed-task gains. SDK exchange estimates are not model-input measurements. Fresh processes have unflushed OS caches; both status reads query live session observations. Timings include the eight-second target lifetime and cleanup; single pairs establish no general speedup. Payload-exit cgroup counters are verified; final whole-scope counters may be unavailable.', + 'tokenizer': {'package': tiktoken.__version__, 'encoding': enc.name}, + 'runtime_pins': {v: {k: x for k, x in json.loads((args.lab / (v + '-manifest.json')).read_text()).items() if k != 'files'} for v in ['baseline', 'candidate']}, + 'native': json.loads((args.lab / 'native-prerequisite-manifest.json').read_text()), + 'script_pins': {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in Path(__file__).parent.glob('frida*') if p.suffix in ['.py', '.mjs']}, + 'wire': wire, 'sdk': sdk, +} +args.output.write_text(json.dumps(result, indent=2) + '\n') +print(json.dumps({'wire': [{k: v for k, v in r.items() if k != 'request_details'} for r in wire], 'sdk': [{k: v for k, v in r.items() if k not in ['status_calls', 'answer']} for r in sdk]}, indent=2)) diff --git a/experiments/analysis-efficiency/frida-status-queries.mjs b/experiments/analysis-efficiency/frida-status-queries.mjs new file mode 100644 index 0000000000..34f60a4a9b --- /dev/null +++ b/experiments/analysis-efficiency/frida-status-queries.mjs @@ -0,0 +1,267 @@ +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { createWriteStream } from "node:fs"; +import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises"; +import { createRequire } from "node:module"; +import { join } from "node:path"; +import { tmpdir } from "node:os"; +import { pathToFileURL } from "node:url"; +import { finished } from "node:stream/promises"; +import { parseArgs } from "node:util"; +import { createRetainedQueryCaller } from "./retained-query-calls.mjs"; +import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs"; +const { values } = parseArgs({ + options: { + runtime: { type: "string" }, + output: { type: "string" }, + detail: { type: "string", default: "complete" }, + capture: { type: "string", default: "libc" }, + }, +}); +const { runtime, output, detail, capture } = values; +assert.ok(runtime && output); +assert.ok(["complete", "summary"].includes(detail)); +assert.ok(["libc", "small"].includes(capture)); +const require = createRequire(join(runtime, "package.json")); +const { Client } = await import( + pathToFileURL(require.resolve("@modelcontextprotocol/client")) +); +const { StdioClientTransport } = await import( + pathToFileURL(require.resolve("@modelcontextprotocol/client/stdio")) +); +const { parseLinuxProcessStat } = await import( + pathToFileURL(join(runtime, "dist/process/LinuxProcessStat.js")) +); +await mkdir(output, { mode: 0o700 }); +const account = await mkdtemp(join(tmpdir(), "rea-frida-real-")); +const serverLog = createWriteStream(join(output, "server.log")); +const serverLogFinished = finished(serverLog); +serverLogFinished.catch(() => {}); +const client = new Client({ name: "rea-frida-audit", version: "1" }); +const transport = new StdioClientTransport({ + command: process.execPath, + args: [join(runtime, "scripts/rea.mjs"), "mcp"], + env: { + PATH: process.env.PATH, + HOME: account, + NODE_OPTIONS: "--max-old-space-size=2048", + UV_THREADPOOL_SIZE: "1", + REA_LOG_LEVEL: "silent", + }, + stderr: "pipe", +}); +transport.stderr?.pipe(serverLog, { end: false }); +const report = { + question: + "Retain a complete caller-requested libc export inventory, then check only session identity/state and message capture coverage twice, preserving complete Evidence.", + detail, + capture, + calls: [], + status: "pending", +}; +const transcript = []; +const caller = createRetainedQueryCaller(client, (receipt, event) => { + report.calls.push(receipt); + transcript.push(event); +}); +const call = async (name, input) => { + const r = await caller(name, input, name); + assert.ok(!r.isError, JSON.stringify(r)); + return r.structuredContent; +}; +const delay = (ms) => new Promise((r) => setTimeout(r, ms)); +let session; +let birth; +const stat = async () => { + try { + return parseLinuxProcessStat( + await readFile(`/proc/${session.pid}/stat`, "utf8"), + ); + } catch (e) { + if (e.code === "ENOENT") return null; + throw e; + } +}; +try { + await client.connect(transport); + const catalog = await client.listTools(); + await writeFile(join(output, "catalog.json"), JSON.stringify(catalog)); + report.catalog_tools = catalog.tools.length; + const defs = new Map(catalog.tools.map((t) => [t.name, t])); + const orig = client.callTool.bind(client); + client.callTool = (input, options) => + orig(input, { + ...options, + timeout: 30000, + toolDefinition: defs.get(input.name), + }); + report.devices = await call("list_frida_devices", {}); + session = await call("start_frida_session", { + mode: "spawn", + device_id: "local", + program: "/usr/bin/sleep", + argv: ["8"], + }); + report.session = session; + birth = await stat(); + assert.ok(birth); + report.target_birth = birth; + report.same_owned_cgroup = + (await readFile(`/proc/${session.pid}/cgroup`, "utf8")) === + (await readFile("/proc/self/cgroup", "utf8")); + assert.ok(report.same_owned_cgroup); + const source = `send({kind:'ready',pid:Process.id}); +setTimeout(() => { + if (${JSON.stringify(capture)} === 'small') { send({kind:'complete',total:0}); return; } + const module = Process.enumerateModules().find(m => /(^|\\/)libc\\.so\\.6$/.test(m.path)); + if (!module) { send({kind:'missing-libc'}); return; } + const exports = module.enumerateExports().map(e => ({type:e.type,name:e.name,address:e.address.toString()})); + send({kind:'module',pid:Process.id,path:module.path,base:module.base.toString(),size:module.size,total:exports.length}); + for (let offset = 0; offset < exports.length; offset += 64) send({kind:'exports',offset,items:exports.slice(offset,offset+64)}); + send({kind:'complete',total:exports.length}); +}, 300);`; + await writeFile(join(output, "selected-script.js"), source); + report.source_sha256 = createHash("sha256").update(source).digest("hex"); + const loaded = await call("load_frida_script", { + session_id: session.session_id, + source_kind: "inline", + source, + }); + assert.equal(loaded.normalized_result.source_sha256, report.source_sha256); + report.load = loaded.normalized_result; + await call("resume_frida_session", { session_id: session.session_id }); + await delay(750); + const snapshots = []; + for (let n = 0; n < 2; n++) + snapshots.push( + await call("frida_session_status", { + session_id: session.session_id, + ...(detail === "summary" ? { detail } : {}), + }), + ); + await call("export_evidence_bundle", { path: join(output, "evidence.json") }); + const bundle = JSON.parse( + await readFile(join(output, "evidence.json"), "utf8"), + ); + for (const snapshot of snapshots) { + assert.equal(snapshot.raw_result, null); + const parent = + detail === "summary" + ? bundle.records.find( + (e) => + e.evidence_id === snapshot.normalized_result.parent_evidence_id, + ) + : snapshot; + assert.ok(parent); + assert.equal(parent.authority, "external-service"); + assert.equal(parent.environment, null); + assert.equal(parent.raw_result, null); + assert.equal(parent.operation, "frida_session_status"); + assert.deepEqual(parent.parameters, { session_id: session.session_id }); + const r = parent.normalized_result; + assert.equal(r.pid, session.pid); + assert.equal(r.state, "running"); + assert.equal(r.messages_truncated, false); + const payloads = r.messages + .filter((m) => m.type === "send") + .map((m) => m.payload); + const ready = payloads.find((p) => p?.kind === "ready"); + const complete = payloads.find((p) => p?.kind === "complete"); + assert.equal(ready?.pid, session.pid); + assert.ok(complete); + let count = 0; + let header; + if (capture === "libc") { + header = payloads.find((p) => p?.kind === "module"); + assert.ok(header); + assert.equal(header.pid, session.pid); + for (const chunk of payloads.filter((p) => p?.kind === "exports")) { + assert.equal(chunk.offset, count); + count += chunk.items.length; + } + assert.equal(count, header.total); + assert.ok(count > 1000); + const maps = await readFile(`/proc/${session.pid}/maps`, "utf8"); + assert.ok(maps.includes(header.path)); + } + assert.equal(count, complete.total); + if (detail === "summary") { + const view = snapshot.normalized_result; + assert.equal(view.parent_operation, parent.operation); + assert.equal(view.parent_digest, parent.evidence_id.slice(3)); + assert.equal(view.artifact, null); + assert.deepEqual(view.source_parameters, parent.parameters); + assert.deepEqual(view.source_provider, parent.provider); + const { messages, ...metadata } = r; + assert.deepEqual(view.summary, { + ...metadata, + messages_retained: messages.length, + }); + assert.ok(view.unknowns.length > 0); + assert.ok(view.limitations.some((m) => m.includes("operating system"))); + assert.deepEqual( + bundle.records.find((e) => e.evidence_id === snapshot.evidence_id), + snapshot, + ); + } + report.answer = { + pid: r.pid, + state: r.state, + scripts: r.scripts.length, + messages: r.messages.length, + messages_truncated: r.messages_truncated, + exports: count, + module_path: header?.path ?? null, + }; + } + report.native_oracle = + "Caller-authored script digest and contiguous chunk coverage; PID/birthtime/cgroup and libc path independently checked in procfs. Export addresses and native export completeness not independently qualified."; + report.exported_bundle_bytes = Buffer.byteLength( + await readFile(join(output, "evidence.json")), + ); + await call("close_frida_session", { session_id: session.session_id }); + session.closed = true; + const deadline = performance.now() + 12000; + while (true) { + const current = await stat(); + if ( + current === null || + current.startTime !== birth.startTime || + ["zombie", "dead"].includes(current.lifecycle) + ) { + report.original_target_exited = true; + break; + } + assert.ok( + performance.now() < deadline, + "Original target exit not observed", + ); + await delay(50); + } + report.status = "passed"; +} catch (error) { + report.status = "failed"; + report.failure = String(error?.stack ?? error); + process.exitCode = 1; +} finally { + if (session && !session.closed) + try { + await call("close_frida_session", { session_id: session.session_id }); + } catch (error) { + report.session_cleanup_failure = String(error); + process.exitCode = 1; + } + if ( + !(await finalizeRetainedQueryReceipt({ + report, + client, + transport, + account, + serverLog, + serverLogFinished, + output, + transcript, + })) + ) + process.exitCode = 1; +} diff --git a/experiments/analysis-efficiency/frida-status-views.json b/experiments/analysis-efficiency/frida-status-views.json new file mode 100644 index 0000000000..667bb5137b --- /dev/null +++ b/experiments/analysis-efficiency/frida-status-views.json @@ -0,0 +1,468 @@ +{ + "scope": "Real local Frida binding with controlled sleep target and caller-authored libc export inventory. Actual Codex0.163 app-server uses a deterministic local SSE fixture: no AI inference, model usage, quality or billing claim. Whole compact JSON token estimates include all 211 schemas and every request replay. Incomplete default baseline is excluded from completed-task gains. SDK exchange estimates are not model-input measurements. Fresh processes have unflushed OS caches; both status reads query live session observations. Timings include the eight-second target lifetime and cleanup; single pairs establish no general speedup. Payload-exit cgroup counters are verified; final whole-scope counters may be unavailable.", + "tokenizer": { + "package": "0.12.0", + "encoding": "cl100k_base" + }, + "runtime_pins": { + "baseline": { + "source_base": "17f581ff75d2b6cf643a997275e3e296f6c95760", + "patch_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "archive_sha256": "cd16ca3e36633459bffc10bb90d1fe4ff5887fdd403729765b9a812969013807" + }, + "candidate": { + "source_base": "32a4690059907f09fb29e284ad492d94c1722278", + "patch_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "archive_sha256": "4ac402f1de93f76f71b7ccdb8b8e36d8c8d0a87b1184812f06393bfc6fadccba" + } + }, + "native": { + "host": "Linux-6.1.0-53-cloud-amd64-x86_64-with-glibc2.36", + "toolchain": { + "node": "24.18.0", + "npm": "11.16.0" + }, + "frida_binding_package_version": "17.23.2", + "files": [ + { + "path": "/usr/bin/sleep", + "bytes": 43888, + "sha256": "4add4bb89d8ca0e3b1bd861130ddd7ae0fd9617a8055de0a38c8d2ca1ac95723" + }, + { + "path": "/usr/lib/x86_64-linux-gnu/libc.so.6", + "bytes": 1926232, + "sha256": "6b4a45352fd0c540a9c7c718f35ce8c8e46a4e482f9d3885a910c32d1a0e1421" + }, + { + "path": "/home/yann/yann/rea-perf-next-audit-20261020/node_modules/frida/package.json", + "bytes": 2252, + "sha256": "7efb59d2050b7b6304e1617f463c4eb7b2f7cb7e3e80e677976825a76ee31e8d" + }, + { + "path": "/home/yann/yann/rea-perf-next-audit-20261020/node_modules/frida/build/frida_binding.node", + "bytes": 112775824, + "sha256": "5b2db5ae57c058ad0765e627d9d9c9c98e4e90c09746c6f727b7a80769eed7a5" + } + ], + "qualifications": { + "Inspector": "real REA MCP target, concurrent native metadata only oracle, 776 rows matched hash/length; not global completeness or execution proof", + "Frida": "local native binding and controlled sleep spawn; caller script digest and procfs PID/birthtime/cgroup/libc path verified; native export address/global completeness oracle unavailable", + "ADB": "default executable unavailable in prior public call; no real device qualified" + } + }, + "script_pins": { + "frida_status_delivery.py": "ed19bcd003165c7b522c8cdd4799bcf987178e4b79fc8d5c2442012f3bf60b8d", + "frida-status-costs.py": "2f3196bcc3198a265f7f4842effc8dce31669723b42f07367f065afee0073906", + "frida-status-queries.mjs": "c5919ae8bba046909858e92a945b4472645bbacbfcf88dd75c89ba959a986980", + "frida-status-wire.py": "b4d20e64bf77b7815bd43b52569b3e4756983b254ab44c50ddae874d1964ef69" + }, + "wire": [ + { + "case": "wire-v2-baseline-0", + "task_completed": false, + "mode": "complete", + "budget": "client default", + "requests": 8, + "compactions": 0, + "whole_json_tokens_proxy": 540065, + "request_details": [ + { + "whole_json_tokens_proxy": 62729, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 62917, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 63599, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 63739, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 68261, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 72785, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 72926, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 73109, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + } + ], + "wall_seconds": 15.993, + "cpu_seconds": 8.211179, + "family_peak_mib": 830.6484375, + "payload_cleanup_verified": true, + "final_whole_scope_counters_verified": false + }, + { + "case": "wire-v2-baseline-300000", + "task_completed": true, + "mode": "complete", + "budget": 300000, + "requests": 8, + "compactions": 0, + "whole_json_tokens_proxy": 1049997, + "request_details": [ + { + "whole_json_tokens_proxy": 62715, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 62903, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 63577, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 63714, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 141108, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 218504, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 218646, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 218830, + "rea_namespace_tokens_proxy": 53518, + "rea_functions": 211 + } + ], + "wall_seconds": 15.418, + "cpu_seconds": 8.026944, + "family_peak_mib": 824.28125, + "payload_cleanup_verified": true, + "final_whole_scope_counters_verified": false + }, + { + "case": "wire-v2-candidate-0", + "task_completed": true, + "mode": "summary", + "budget": "client default", + "requests": 8, + "compactions": 0, + "whole_json_tokens_proxy": 515539, + "request_details": [ + { + "whole_json_tokens_proxy": 62791, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 62978, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 63661, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 63803, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 64749, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 65697, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 65837, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 66023, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + } + ], + "wall_seconds": 15.889, + "cpu_seconds": 7.939387, + "family_peak_mib": 823.69921875, + "payload_cleanup_verified": true, + "final_whole_scope_counters_verified": false + }, + { + "case": "wire-v2-candidate-300000", + "task_completed": true, + "mode": "summary", + "budget": 300000, + "requests": 8, + "compactions": 0, + "whole_json_tokens_proxy": 515713, + "request_details": [ + { + "whole_json_tokens_proxy": 62804, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 62990, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 63677, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 63815, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 64772, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 65731, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 65870, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + }, + { + "whole_json_tokens_proxy": 66054, + "rea_namespace_tokens_proxy": 53581, + "rea_functions": 211 + } + ], + "wall_seconds": 15.439, + "cpu_seconds": 8.371664, + "family_peak_mib": 813.6328125, + "payload_cleanup_verified": true, + "final_whole_scope_counters_verified": false + } + ], + "sdk": [ + { + "case": "sdk-baseline-libc", + "detail": "complete", + "capture": "libc", + "tool_calls": 8, + "exported_bundle_bytes": 214214, + "answer": { + "pid": 517427, + "state": "running", + "scripts": 1, + "messages": 50, + "messages_truncated": false, + "exports": 2963, + "module_path": "/usr/lib/x86_64-linux-gnu/libc.so.6" + }, + "status_calls": [ + { + "name": "frida_session_status", + "input": { + "session_id": "ef8982d3-dbf2-43cb-8256-3a4f4bc83cc3" + }, + "phase": "frida_session_status", + "elapsed_ms": 197.58100300000024, + "reply_bytes": 462831, + "is_error": false + }, + { + "name": "frida_session_status", + "input": { + "session_id": "ef8982d3-dbf2-43cb-8256-3a4f4bc83cc3" + }, + "phase": "frida_session_status", + "elapsed_ms": 108.50572199999988, + "reply_bytes": 462831, + "is_error": false + } + ], + "one_catalog_plus_all_sdk_calls_tokens_proxy": 1186065, + "wall_seconds": 13.759, + "cpu_seconds": 6.097489, + "family_peak_mib": 578.5625, + "payload_cleanup_verified": true, + "final_whole_scope_counters_verified": false + }, + { + "case": "sdk-baseline-small", + "detail": "complete", + "capture": "small", + "tool_calls": 8, + "exported_bundle_bytes": 2335, + "answer": { + "pid": 516970, + "state": "running", + "scripts": 1, + "messages": 2, + "messages_truncated": false, + "exports": 0, + "module_path": null + }, + "status_calls": [ + { + "name": "frida_session_status", + "input": { + "session_id": "3f1f778f-ca18-48ba-a7db-b58afbf4710d" + }, + "phase": "frida_session_status", + "elapsed_ms": 79.25232200000028, + "reply_bytes": 2643, + "is_error": false + }, + { + "name": "frida_session_status", + "input": { + "session_id": "3f1f778f-ca18-48ba-a7db-b58afbf4710d" + }, + "phase": "frida_session_status", + "elapsed_ms": 6.851230000000214, + "reply_bytes": 2643, + "is_error": false + } + ], + "one_catalog_plus_all_sdk_calls_tokens_proxy": 882323, + "wall_seconds": 13.039, + "cpu_seconds": 4.953337, + "family_peak_mib": 577.140625, + "payload_cleanup_verified": true, + "final_whole_scope_counters_verified": false + }, + { + "case": "sdk-candidate-libc", + "detail": "summary", + "capture": "libc", + "tool_calls": 8, + "exported_bundle_bytes": 216931, + "answer": { + "pid": 517901, + "state": "running", + "scripts": 1, + "messages": 50, + "messages_truncated": false, + "exports": 2963, + "module_path": "/usr/lib/x86_64-linux-gnu/libc.so.6" + }, + "status_calls": [ + { + "name": "frida_session_status", + "input": { + "session_id": "9af7f188-05f8-4559-b570-98556196090e", + "detail": "summary" + }, + "phase": "frida_session_status", + "elapsed_ms": 159.16798299999937, + "reply_bytes": 5597, + "is_error": false + }, + { + "name": "frida_session_status", + "input": { + "session_id": "9af7f188-05f8-4559-b570-98556196090e", + "detail": "summary" + }, + "phase": "frida_session_status", + "elapsed_ms": 61.96135099999992, + "reply_bytes": 5597, + "is_error": false + } + ], + "one_catalog_plus_all_sdk_calls_tokens_proxy": 885626, + "wall_seconds": 12.843, + "cpu_seconds": 5.012703, + "family_peak_mib": 579.0, + "payload_cleanup_verified": true, + "final_whole_scope_counters_verified": false + }, + { + "case": "sdk-candidate-small", + "detail": "summary", + "capture": "small", + "tool_calls": 8, + "exported_bundle_bytes": 5051, + "answer": { + "pid": 517162, + "state": "running", + "scripts": 1, + "messages": 2, + "messages_truncated": false, + "exports": 0, + "module_path": null + }, + "status_calls": [ + { + "name": "frida_session_status", + "input": { + "session_id": "1bcda0f5-e3cd-45ee-b82c-5886d8312bc6", + "detail": "summary" + }, + "phase": "frida_session_status", + "elapsed_ms": 107.78244100000029, + "reply_bytes": 5595, + "is_error": false + }, + { + "name": "frida_session_status", + "input": { + "session_id": "1bcda0f5-e3cd-45ee-b82c-5886d8312bc6", + "detail": "summary" + }, + "phase": "frida_session_status", + "elapsed_ms": 5.996030000000246, + "reply_bytes": 5595, + "is_error": false + } + ], + "one_catalog_plus_all_sdk_calls_tokens_proxy": 885676, + "wall_seconds": 13.533, + "cpu_seconds": 5.495048, + "family_peak_mib": 576.48046875, + "payload_cleanup_verified": true, + "final_whole_scope_counters_verified": false + } + ] +} diff --git a/experiments/analysis-efficiency/frida-status-views.md b/experiments/analysis-efficiency/frida-status-views.md new file mode 100644 index 0000000000..9a9516c51f --- /dev/null +++ b/experiments/analysis-efficiency/frida-status-views.md @@ -0,0 +1,163 @@ +# Frida status delivery and runtime-observation audit + +Issue: [#5640](https://github.com/morluto/rea/issues/5640). Paired runtime base: +`17f581ff75d2b6cf643a997275e3e296f6c95760`. Final candidate source: `32a4690059907f09fb29e284ad492d94c1722278` (clean tree). +Exact package, patch, native input, +script and resource pins are in [the measurement record](frida-status-views.json). + +The analyst requests a complete libc export inventory as retained evidence, +then checks the session's identity/state, script identity, queued message count +and truncation twice. The caller-authored script enumerates actual native module +exports in a controlled `/usr/bin/sleep` spawn, in chunks of 64. Both workflows +retain all captured messages and export the complete Evidence bundle. + +Start, status and one-shot contracts share one canonical session envelope. +Message JSON is validated at the Evidence owner; the summary validates the +envelope without reinterpreting opaque payloads. + +The owning application service now accepts optional `detail=summary` for +`frida_session_status`. It reads the provider on every call, retains the original +complete Evidence, then uses the existing shared analysis-view projector. +The summary omits arbitrary message payloads and preserves all other status +fields, the retained message count, truncation, exact parent identity/digest, +provider and source parameters. Default and explicit complete delivery keep the +original Evidence identity and payload. Historical CLI/file projections use the +same projector. Source script decoding/representation Evidence and session +resource ownership are unchanged. + +## Completed tasks + +The native Frida Node binding package 17.23.2 captured 2,963 libc exports in 50 queued +messages. Independent checks establish the test-owned process birthtime, cgroup, +PID and libc mapping; the selected script digest and contiguous chunks are +checked against the complete retained parent. This does not independently +validate every native export address or establish global event coverage. + +| Public SDK task | Complete | Summary | +| ------------------------------------------ | -------: | ------: | +| Large-history status reply, bytes per call | 462,831 | 5,597 | +| Small-history status reply, bytes per call | 2,643 | 5,595 | +| Large complete bundle, bytes | 214,214 | 216,931 | +| Small complete bundle, bytes | 2,335 | 5,051 | +| Large task elapsed, seconds | 13.759 | 12.843 | +| Large task CPU, seconds | 6.097 | 5.013 | +| Large family peak, MiB | 578.6 | 579.0 | +| Small task elapsed, seconds | 13.039 | 13.533 | +| Small task CPU, seconds | 4.953 | 5.495 | +| Small family peak, MiB | 577.1 | 576.5 | + +Each SDK task discovers all 211 tools and executes eight public calls, including +two status reads and complete export. Small history contains two readiness/ +completion messages. Fresh processes retain unflushed OS caches. The second +status read is a repeated live observation, not a cache hit. Every task includes +the controlled target's eight-second lifetime and verified natural exit. +Single pairs and this fixed wait establish no general latency/CPU advantage. +Large status calls took 159.2/62.0 ms versus 197.6/108.5 ms. The first small +summary status was slower (107.8 versus 79.3 ms). + +## Actual client context + +Codex app-server 0.163.0 performs the native Frida workflow through the actual +REA MCP package. A deterministic local Responses/SSE fixture selects the calls; +there is no AI inference, paid model, model-quality or billing claim. The capture +contains all 211 tool input schemas on each request and all history replays. `tiktoken` +0.12.0 `cl100k_base` counts compact JSON as a token estimate, not actual model +usage. Both completed paths use the same 300,000 tool-output budget. + +| Completed client task | Complete | Summary | +| -------------------------------------- | --------: | ------: | +| Requests / compactions | 8 / 0 | 8 / 0 | +| Whole request JSON token estimate, sum | 1,049,997 | 515,713 | +| Elapsed seconds | 15.418 | 15.439 | +| CPU seconds | 8.027 | 8.372 | +| Family peak MiB | 824.3 | 813.6 | + +The completed task uses **50.9% fewer estimated request tokens**, including +schemas and continuations. Client elapsed and CPU time increased in this pair. At the client's +default output budget, complete status JSON is truncated and the task is +incomplete; summary status is intact and completes. That failed delivery is +excluded from completed-task gains. Three acceptance tests exercise invalid +JSON, incorrect counts/state/source linkage, missing chunks and wrong sessions. + +The complete catalog plus all SDK exchanges also shrinks for the large case +(1,186,065 to 885,626 estimated tokens), but grows for the small case (882,323 to +885,676). These SDK exchange estimates include advertised output schemas and +are **not** actual model-input measurements. Summary delivery is optional +because its provenance wrapper costs more than a small complete observation. + +## Resource accounting and limits + +Runtime cases use one CPU, a 4 GiB cgroup ceiling, a 3 GiB early stop, no swap, +192 tasks, and host-memory admission/early-stop guards. Payload exit, descendant +cleanup and zero resource-limit/OOM events are verified while the scope observer +remains alive. Final whole-scope counters disappear after teardown and remain +unknown. Runtime family peaks stay below 1 GiB. Build/typecheck use separately +guarded larger limits; they are not runtime performance measurements. + +Full queued messages still undergo validation, Evidence creation and retention; +this change primarily reduces delivery and repeated context. The extra derived +record increases retained bundle size. Long-lived heap retention, many-script +sessions, remote devices, binary messages and saturated capture budgets are not +performance-qualified by these native cases. Boundary tests preserve malformed +input, truncation, cancellation and cleanup semantics; simulated providers do +not qualify real remote behavior. + +## Other providers in this audit + +Inspector was measured against a test-owned real REA MCP process while it +handled a verified GitHub CLI Go build-information request. A concurrent native +CDP metadata subscription matched all 776 retained scripts by hash and reported +length: 565 file locations and 211 built-ins. The reply was 608,719 bytes; +capture retained 996/996 events with no reported drop/truncation. The qualification +run took 20.956 seconds, 18.076 CPU seconds and 987.4 MiB peak family memory, +including the independent oracle. This is not a paired performance cohort, +global completeness proof, source-byte validation or evidence of script +execution. Only `Runtime.enable` and `Debugger.enable` were sent. A prior +post-capture oracle missed 25 historical rows and was rejected; it does not +establish an Inspector regression. Existing raw-null delivery, grouped location +validation and eight-worker projection were already present and were preserved. + +ADB's configured default executable is unavailable on this host, and no real +device is qualified. This audit makes no ADB device-performance claim. Inspector +selection/caching and Frida queue/filter redesign remain unimplemented without +an equivalent completed-task measurement that justifies them. + +The existing bundled Frida verifier also returned functional success, but its +1,500 ms fixture outlived the verifier's 500 ms observation. Its payload-exit +cleanup receipt was therefore unverified and is excluded from qualified +resource/performance results. That unchanged verifier has no process birthtime +stamp to safely wait on an arbitrary later PID. The paired harness instead owns +the paused spawn and records birthtime/cgroup before resume, then waits for the +original target's natural exit. No PID-only cleanup or target-kill policy was +introduced. + +The shared-service import also exposed an existing module-mock load-order +assumption in the Evidence-file cancellation test. That one mocked test now +runs in the existing isolated `process-global` lane, with an assertion that the +mocked read actually ran. Other file/service tests keep the shared module graph. +The final exact ten-file gate passes 225 tests with file parallelism disabled +under one CPU. An earlier parallel gate hit one existing 30-second MCP lifecycle +timeout; its serial retry passed without increasing a timeout or changing the +production lifecycle contract. + +## Reproduction + +Use the pinned Node/npm toolchain and native Frida dependency. Freeze separate +baseline/candidate built packages and install the same locked dependencies. +Run each case independently with the resource observer: + +```sh +python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \ + node experiments/analysis-efficiency/frida-status-queries.mjs \ + --runtime /absolute/built-package --output /absolute/new-case \ + --capture libc --detail summary +``` + +Repeat with `--detail complete` and `--capture small`. For actual-client captures, +`frida-status-wire.py --help` names the runtime, pinned Codex and Node executables, +output directory and selected script file (`selected-script.js` from the SDK +case). Use both `--mode complete`/`summary` and `--budget 300000`/`0`. +`frida-status-costs.py` checks task/resource receipts and counts complete requests. +The first wire attempt used ambiguous suffix matching for load/unload names; +the fixture now selects exact operation names. Failed attempts remain outside +the performance cohort. No production behavior changed to accommodate them. diff --git a/experiments/analysis-efficiency/frida-status-wire.py b/experiments/analysis-efficiency/frida-status-wire.py new file mode 100644 index 0000000000..daf56c96be --- /dev/null +++ b/experiments/analysis-efficiency/frida-status-wire.py @@ -0,0 +1,299 @@ +"""Actual Codex wire capture with a deterministic local SSE backend; no AI inference.""" +import argparse +import http.server +import threading +import asyncio +import hashlib +import json +import os +from pathlib import Path +import re +import time +import traceback +import subprocess +from frida_status_delivery import verify_frida_delivery + + + +async def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--output", type=Path, required=True) + parser.add_argument("--runtime", type=Path, required=True) + parser.add_argument("--codex", type=Path, required=True) + parser.add_argument("--node", type=Path, required=True) + parser.add_argument("--input-profile", choices=["full", "compact"], default="full") + parser.add_argument("--output-profile", choices=["full", "off"], default="full") + parser.add_argument("--script", type=Path, required=True) + parser.add_argument("--expected-tools", type=int, default=211) + parser.add_argument("--mode", choices=["complete", "summary"], required=True) + parser.add_argument("--budget", type=int, default=30000) + args = parser.parse_args() + source=args.script.read_text() + assert args.runtime.is_absolute() and args.output.is_absolute() and args.script.is_absolute() + def delivered(pieces): + assert isinstance(pieces,str) and pieces.startswith("Wall time: "), pieces[:300] + _,sep,payload=pieces.partition("Output:\n");assert sep + try: value=json.loads(payload) + except json.JSONDecodeError: return {"_invalid_json_payload":payload} + return value["result"] if "result" in value and "normalized_result" not in value else value + def outputs_of(body): + return {item["call_id"]:delivered(item["output"]) for item in body.get("input",[]) if item.get("type")=="function_call_output"} + steps=7 + args.output.mkdir(mode=0o700) + audit_home = args.output / "account" + codex_state = audit_home / ".codex" + mcp_home = audit_home / "mcp" + workspace = audit_home / "workspace" + for directory in [codex_state, mcp_home, workspace]: + directory.mkdir(parents=True) + captured = [] + known_outputs = {} + state = {"normal_requests":0,"compactions":0} + class Capture(http.server.BaseHTTPRequestHandler): + def log_message(self, *_): + pass + def do_POST(self): + size = int(self.headers["Content-Length"]) + assert 0 < size <= 16 * 1024 * 1024 + assert self.headers.get("Content-Encoding", "identity") == "identity" + body = json.loads(self.rfile.read(size)) + captured.append({"path": self.path, "body": body}) + (args.output / "model-requests.json").write_text(json.dumps(captured) + "\n") + http_index = len(captured) + known_outputs.update(outputs_of(body)) + names = [t.get("name", "") for ns in body.get("tools", []) if ns.get("type") == "namespace" and ns.get("name") == "mcp__rea_audit" for t in ns.get("tools", [])] + if not names: + last=body.get('input',[])[-1] + assert 'CONTEXT CHECKPOINT COMPACTION' in json.dumps(last) + state['compactions']+=1;assert state['compactions']<=1 + checkpoint={'fixture':'deterministic observed-ID checkpoint, no AI inference','task':'Check Frida session state, queued message count and truncation twice; retain/export complete observations and close instrumentation.','session_id':known_outputs['call_local_1']['session_id'],'delivered_call_ids':list(known_outputs),'remaining':'Continue scripted tools; source message bodies remain in retained Evidence.'} + text=json.dumps(checkpoint) + item={"type":"message","id":f"msg_checkpoint_{state['compactions']}","role":"assistant","status":"completed","content":[{"type":"output_text","text":text,"annotations":[]}]} + captured[-1]['kind']='deterministic-compaction' + else: + state['normal_requests']+=1;index=state['normal_requests'];assert index<=steps+1 + captured[-1]['kind']='tool-planning-or-final' + if index<=steps: + if index==1: + operation='start_frida_session';arguments={'mode':'spawn','device_id':'local','program':'/usr/bin/sleep','argv':['8']} + else: + session=known_outputs['call_local_1']['session_id'] + if index==2: + operation='load_frida_script';arguments={'session_id':session,'source_kind':'inline','source':source} + elif index==3: + operation='resume_frida_session';arguments={'session_id':session} + elif index in [4,5]: + if index==4: time.sleep(0.75) + operation='frida_session_status';arguments={'session_id':session,**({'detail':'summary'} if args.mode=='summary' else {})} + elif index==6: + operation='close_frida_session';arguments={'session_id':session} + else: + operation='export_evidence_bundle';arguments={'path':str(args.output/'evidence.json')} + selected=[name for name in names if name == operation];assert len(selected)==1 + item={"type":"function_call","id":f"fc_local_{index}","call_id":f"call_local_{index}","name":selected[0],"namespace":"mcp__rea_audit","arguments":json.dumps(arguments),"status":"completed"} + else: + item={"type":"message","id":"msg_local_done","role":"assistant","status":"completed","content":[{"type":"output_text","text":"Scripted local Frida state and capture-coverage task complete.","annotations":[]}]} + response_id=f"resp_local_{http_index}" + base = {"id": response_id, "object": "response", "model": body.get("model"), "status": "completed", "output": [item]} + events = [ + {"type": "response.created", "response": {**base, "status": "in_progress", "output": []}}, + {"type": "response.output_item.added", "output_index": 0, "item": {**item, "status": "in_progress"}}, + {"type": "response.output_item.done", "output_index": 0, "item": item}, + {"type": "response.completed", "response": base}, + ] + wire = "".join("event: " + e["type"] + "\ndata: " + json.dumps(e) + "\n\n" for e in events).encode() + self.send_response(200) + self.send_header("Content-Type", "text/event-stream") + self.send_header("Content-Length", str(len(wire))) + self.end_headers() + self.wfile.write(wire) + endpoint = http.server.HTTPServer(("127.0.0.1", 0), Capture) + endpoint_thread = threading.Thread(target=endpoint.serve_forever, daemon=True) + endpoint_thread.start() + task_path = f"{args.node.parent}:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" + mcp_env = {"HOME": str(mcp_home), "PATH": task_path, + "NODE_OPTIONS": "--max-old-space-size=2048", "UV_THREADPOOL_SIZE": "1", + "REA_LOG_LEVEL": "silent", "REA_MCP_INPUT_SCHEMA_PROFILE": args.input_profile, + "REA_MCP_OUTPUT_SCHEMA_PROFILE": args.output_profile} + config = 'model_provider = "rea_capture"\nmodel = "rea-wire-probe"\n[analytics]\nenabled = false\n[mcp_servers.rea_audit]\n' + config += f'command = {json.dumps(str(args.node))}\n' + config += f'args = {json.dumps([str(args.runtime / "scripts/rea.mjs"), "mcp"])}\n' + config += 'startup_timeout_sec = 60\ntool_timeout_sec = 60\n' + config += '[mcp_servers.rea_audit.env]\n' + config += ''.join(f'{key} = {json.dumps(value)}\n' for key, value in mcp_env.items()) + for name in ['start_frida_session','load_frida_script','resume_frida_session','frida_session_status','close_frida_session','export_evidence_bundle']: + config += f'[mcp_servers.rea_audit.tools.{name}]\napproval_mode = "approve"\n' + if args.budget: config += f'output_token_limit = {args.budget}\n' + config += f'\n[model_providers.rea_capture]\nname = "Local scripted capture"\nbase_url = "http://127.0.0.1:{endpoint.server_port}/v1"\nwire_api = "responses"\nrequires_openai_auth = false\nsupports_websockets = false\nrequest_max_retries = 0\nstream_max_retries = 0\n' + (codex_state / "config.toml").write_text(config) + (args.output / "declared-config.toml").write_text(config) + environment = {"HOME": str(audit_home), "CODEX_HOME": str(codex_state), "PATH": task_path, + "NODE_OPTIONS": "--max-old-space-size=2048", "UV_THREADPOOL_SIZE": "1"} + report = {"question": "Retain a caller-requested native libc export inventory, check session identity/state, queued message count and truncation twice, close instrumentation and export complete Evidence.", + "client": "Codex app-server 0.163.0", "model_turns_started": 1, "backend": "local deterministic SSE fixture; no AI model or paid service", + "actual_model_usage": None, "input_profile": args.input_profile, + "output_profile": args.output_profile, "calls": [], "observations": [], + "unexpected_server_requests": [], "cleanup_failures": []} + client_version = subprocess.check_output([str(args.codex), "--version"], env=environment, cwd=workspace, text=True, timeout=10).strip() + assert client_version == "codex-cli 0.163.0", client_version + transcript = [] + pending = {} + completed_turn = asyncio.Event() + next_id = 0 + stderr = (args.output / "server.log").open("wb") + process = await asyncio.create_subprocess_exec(str(args.codex), "app-server", "--stdio", + stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE, stderr=stderr, + cwd=workspace, env=environment, limit=16 * 1024 * 1024) + + async def send(message): + encoded = (json.dumps(message, separators=(",", ":")) + "\n").encode() + transcript.append({"direction": "client", "bytes": len(encoded), "message": message}) + process.stdin.write(encoded) + await process.stdin.drain() + + async def read_messages(): + try: + while line := await process.stdout.readline(): + value = json.loads(line) + transcript.append({"direction": "server", "bytes": len(line), "message": value}) + if value.get("method") == "turn/completed": + report["turn"] = value["params"]["turn"] + completed_turn.set() + if "id" in value and "method" not in value: + future = pending.pop(value["id"], None) + if future and not future.done(): + future.set_result(value) + elif "id" in value: + report["unexpected_server_requests"].append(value["method"]) + await send({"id": value["id"], "error": {"code": -32601, "message": "Unexpected request in scripted metadata probe"}}) + raise RuntimeError("app-server stdout closed") + except Exception as error: + for future in pending.values(): + if not future.done(): + future.set_exception(error) + + reader = asyncio.create_task(read_messages()) + + async def request(method, params): + nonlocal next_id + assert method in ["initialize", "thread/start", "mcpServerStatus/list", "mcpServer/tool/call", "turn/start"] + next_id += 1 + message = {"id": next_id, "method": method, "params": params} + future = asyncio.get_running_loop().create_future() + pending[next_id] = future + start = time.monotonic() + await send(message) + reply = await asyncio.wait_for(future, 90) + report["calls"].append({"method": method, "elapsed_ms": (time.monotonic() - start) * 1000, + "error": reply.get("error")}) + if "error" in reply: + raise RuntimeError(json.dumps(reply["error"])[:4000]) + return reply["result"] + + try: + await request("initialize", {"clientInfo": {"name": "rea_host_probe", "title": "REA host compatibility probe", "version": "1"}, + "capabilities": {"experimentalApi": True, "explicitGatewayOauth": True}}) + await send({"method": "initialized"}) + created = await request("thread/start", {"cwd": str(workspace), "ephemeral": True, + "approvalPolicy": "never", "sandbox": "read-only"}) + thread_id = created["thread"]["id"] + assert not created["thread"].get("turns", []) + inventory = await request("mcpServerStatus/list", {"threadId": thread_id, + "serverName": "rea_audit", "detail": "toolsAndAuthOnly", "limit": 10}) + assert inventory.get("nextCursor") is None + assert len(inventory["data"]) == 1 + server = inventory["data"][0] + assert server["name"] == "rea_audit" and server.get("toolsError") is None + tools = list(server["tools"].values()) + names = [tool["name"] for tool in tools] + assert len(names) == len(set(names)) == args.expected_tools + assert "frida_session_status" in names + output_schemas = sum(tool.get("outputSchema") is not None for tool in tools) + assert output_schemas == (args.expected_tools if args.output_profile == "full" else 0) + raw = json.dumps(inventory, separators=(",", ":"), ensure_ascii=False).encode() + (args.output / "catalog.json").write_bytes(raw) + report["catalog"] = {"tools": len(tools), "output_schemas": output_schemas, + "bytes": len(raw), "sha256": hashlib.sha256(raw).hexdigest(), + "server_info": server.get("serverInfo"), "runtime_status": server.get("runtimeStatus")} + await request("turn/start", {"threadId":thread_id,"input":[{"type":"text","text":"Retain a complete libc export inventory using the selected caller-authored script. Check session state and queued message capture count/truncation twice, close Frida instrumentation and export complete Evidence. The local fixture scripts the tools; no AI inference."}]}) + await asyncio.wait_for(completed_turn.wait(), 90) + assert report["turn"]["status"] == "completed", report["turn"] + assert state["normal_requests"]==steps+1 + assert len(captured)==steps+1+state["compactions"] + outputs=known_outputs + assert len(outputs)==steps + bundle=json.loads((args.output/'evidence.json').read_text()) + delivery_complete=True + answers=[] + for index in [4,5]: + delivered_result=outputs[f'call_local_{index}'] + answer=verify_frida_delivery(delivered_result,bundle,args.mode,args.budget,outputs['call_local_1']) + if answer is None: delivery_complete=False + else: answers.append(answer) + loaded=next(e for e in bundle['records'] if e['operation']=='load_frida_script') + assert loaded['normalized_result']['source_sha256']==hashlib.sha256(source.encode()).hexdigest() + report['task_completed']=delivery_complete + report['observations']={'answers':answers,'exported':True,'exported_bundle_bytes':(args.output/'evidence.json').stat().st_size,'tool_output_budget':args.budget or 'client default','source_sha256':hashlib.sha256(source.encode()).hexdigest()} + report["captured_requests"]=len(captured) + report["compaction_requests"]=state["compactions"] + report["compaction_backend"]="Deterministic checkpoint of observed identifiers and retained-body location; no AI inference or quality claim." + report["mode"]=args.mode + report["request_tool_counts"] = [len(c["body"].get("tools", [])) for c in captured] + assert not report["unexpected_server_requests"] + report["status"] = "passed" if report["task_completed"] else "expected-incomplete-task" + except Exception: + report["status"] = "failed" + report["failure"] = traceback.format_exc()[-6000:] + finally: + process.stdin.close() + try: + await asyncio.wait_for(process.wait(), 15) + except asyncio.TimeoutError: + report["cleanup_failures"].append("app-server did not exit after stdin close; terminated owned process") + process.terminate() + try: + await asyncio.wait_for(process.wait(), 5) + except asyncio.TimeoutError: + process.kill() + await process.wait() + await reader + # app-server can acknowledge exit before its MCP child finishes EOF + # cleanup. Wait on the allocated resource scope; never target a PID. + try: + groups = [line[3:] for line in Path("/proc/self/cgroup").read_text().splitlines() + if line.startswith("0::")] + assert len(groups) == 1 + group = Path("/sys/fs/cgroup") / groups[0].lstrip("/") + assert re.fullmatch(r"rea-efficiency-[0-9a-f]{32}\.scope", group.name) + observer = os.getppid() + observer_args = Path(f"/proc/{observer}/cmdline").read_bytes().split(b"\0") + assert any(Path(os.fsdecode(value)).name == "resource_payload.py" for value in observer_args if value) + expected = {os.getpid(), observer} + until = time.monotonic() + 10 + while True: + actors = {int(value) for value in (group / "cgroup.procs").read_text().splitlines()} + assert expected <= actors + remaining = sorted(actors - expected) + if not remaining or time.monotonic() >= until: + break + await asyncio.sleep(0.05) + report["owned_scope_drain"] = {"remaining_processes": remaining} + if remaining: + report["cleanup_failures"].append("owned MCP child did not settle after app-server exit") + except (OSError, ValueError, AssertionError) as error: + report["owned_scope_drain"] = {"status": "unavailable", "reason": type(error).__name__} + report["cleanup_failures"].append("allocated resource-scope cleanup could not be verified") + endpoint.shutdown() + endpoint.server_close() + endpoint_thread.join(5) + stderr.close() + report["appserver_exit_code"] = process.returncode + if process.returncode != 0: + report["cleanup_failures"].append(f"app-server exit code: {process.returncode}") + (args.output / "transcript.json").write_text(json.dumps(transcript) + "\n") + (args.output / "report.json").write_text(json.dumps(report, indent=2) + "\n") + print(json.dumps({key: report.get(key) for key in ["status", "failure", "catalog", "observations", "cleanup_failures", "appserver_exit_code"]})) + return 0 if report["status"] in ["passed","expected-incomplete-task"] and not report["cleanup_failures"] else 1 + + +raise SystemExit(asyncio.run(main())) diff --git a/experiments/analysis-efficiency/frida_status_delivery.py b/experiments/analysis-efficiency/frida_status_delivery.py new file mode 100644 index 0000000000..2c6f789650 --- /dev/null +++ b/experiments/analysis-efficiency/frida_status_delivery.py @@ -0,0 +1,37 @@ +"""Accept only complete client delivery of the selected Frida status question.""" + + +def verify_frida_delivery(delivered, bundle, mode, budget, session): + if '_invalid_json_payload' in delivered: + assert mode == 'complete' and budget == 0, 'A truncated summary never completes the task' + return None + value = delivered['normalized_result'] + parent = next(e for e in bundle['records'] if e['evidence_id'] == value['parent_evidence_id']) if mode == 'summary' else delivered + assert parent in bundle['records'] and parent['operation'] == 'frida_session_status' + assert parent['authority'] == 'external-service' and parent['environment'] is None + assert parent['raw_result'] is None and parent['subject'] is None + assert parent['parameters'] == {'session_id': session['session_id']} + facts = parent['normalized_result'] + assert facts['state'] == 'running' and not facts['messages_truncated'] + assert facts['session_id'] == session['session_id'] and facts['pid'] == session['pid'] + payloads = [m['payload'] for m in facts['messages'] if m.get('type') == 'send'] + header = next(p for p in payloads if p['kind'] == 'module') + complete = next(p for p in payloads if p['kind'] == 'complete') + count = 0 + for chunk in [p for p in payloads if p['kind'] == 'exports']: + assert chunk['offset'] == count + count += len(chunk['items']) + assert count == header['total'] == complete['total'] and count > 1000 + assert facts['pid'] == header['pid'] + if mode == 'summary': + assert delivered in bundle['records'] + assert delivered['subject'] is None and delivered['raw_result'] is None + assert delivered['authority'] == parent['authority'] + assert value['parent_operation'] == parent['operation'] + assert value['source_parameters'] == parent['parameters'] and value['source_provider'] == parent['provider'] + assert value['parent_digest'] == parent['evidence_id'][3:] and value['artifact'] is None + assert value['summary'] == {**{k: v for k, v in facts.items() if k != 'messages'}, 'messages_retained': len(facts['messages'])} + assert value['unknowns'] and value['limitations'] + return {'state': facts['state'], 'messages_retained': len(facts['messages']), + 'messages_truncated': facts['messages_truncated'], 'exports': count, + 'parent_evidence_id': parent['evidence_id']} diff --git a/experiments/analysis-efficiency/test_frida_status_delivery.py b/experiments/analysis-efficiency/test_frida_status_delivery.py new file mode 100644 index 0000000000..86d403bc95 --- /dev/null +++ b/experiments/analysis-efficiency/test_frida_status_delivery.py @@ -0,0 +1,48 @@ +"""Reject incomplete status delivery and incorrect retained-source claims.""" +from copy import deepcopy +import unittest +from frida_status_delivery import verify_frida_delivery + + +class FridaDelivery(unittest.TestCase): + def fixture(self): + session = {'session_id': 'selected', 'pid': 12} + payloads = [{'kind': 'module', 'pid': 12, 'total': 1001}, {'kind': 'exports', 'offset': 0, 'items': [None] * 1001}, {'kind': 'complete', 'total': 1001}] + facts = {**session, 'state': 'running', 'messages_truncated': False, 'messages': [{'type': 'send', 'payload': p} for p in payloads]} + parent = {'evidence_id': 'ev_parent', 'operation': 'frida_session_status', 'authority': 'external-service', 'environment': None, 'raw_result': None, 'subject': None, 'parameters': {'session_id': 'selected'}, 'provider': {'id': 'frida'}, 'normalized_result': facts} + summary = {'evidence_id': 'ev_summary', 'subject': None, 'raw_result': None, 'authority': 'external-service', 'normalized_result': { + 'parent_evidence_id': 'ev_parent', 'parent_digest': 'parent', 'parent_operation': 'frida_session_status', 'artifact': None, + 'source_parameters': parent['parameters'], 'source_provider': parent['provider'], 'unknowns': ['historical snapshot'], 'limitations': ['parent retained'], + 'summary': {**{k: v for k, v in facts.items() if k != 'messages'}, 'messages_retained': 3}}} + return session, parent, summary + + def test_truncated_summary_never_qualifies(self): + truncated = {'_invalid_json_payload': 'truncated'} + self.assertIsNone(verify_frida_delivery(truncated, {}, 'complete', 0, {})) + for mode, budget in [('summary', 0), ('summary', 300000), ('complete', 300000)]: + with self.subTest(mode=mode, budget=budget), self.assertRaises(AssertionError): + verify_frida_delivery(truncated, {}, mode, budget, {}) + + def test_complete_and_summary_require_correct_facts_and_source(self): + session, parent, summary = self.fixture() + for mode, delivered in [('complete', parent), ('summary', summary)]: + self.assertEqual(verify_frida_delivery(delivered, {'records': [parent, summary]}, mode, 300000, session)['exports'], 1001) + for field in ['messages_retained', 'state', 'messages', 'source_parameters', 'parent_digest', 'source_provider']: + broken = deepcopy(summary) + target = broken['normalized_result']['summary'] if field in ['messages_retained', 'state', 'messages'] else broken['normalized_result'] + target[field] = 'incorrect' + with self.subTest(field=field), self.assertRaises(AssertionError): + verify_frida_delivery(broken, {'records': [parent, broken]}, 'summary', 0, session) + + def test_missing_native_chunk_and_wrong_session_fail(self): + session, parent, _ = self.fixture() + broken = deepcopy(parent) + broken['normalized_result']['messages'][1]['payload']['items'].pop() + with self.assertRaises(AssertionError): + verify_frida_delivery(broken, {'records': [broken]}, 'complete', 300000, session) + with self.assertRaises(AssertionError): + verify_frida_delivery(parent, {'records': [parent]}, 'complete', 300000, {**session, 'session_id': 'other'}) + + +if __name__ == '__main__': + unittest.main() diff --git a/src/application/analysisView/AnalysisViewService.test.ts b/src/application/analysisView/AnalysisViewService.test.ts index 27500e6855..e40ffea2ed 100644 --- a/src/application/analysisView/AnalysisViewService.test.ts +++ b/src/application/analysisView/AnalysisViewService.test.ts @@ -8,7 +8,7 @@ import { environmentManifestCaptureFixture, environmentStateCaptureFixture, } from "../../../tests/fixtures/analysisView.js"; -import { describe, expect, it, vi } from "vitest"; +import { describe, expect, it } from "vitest"; import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js"; import { createEvidence, parseEvidence } from "../../domain/evidence.js"; @@ -38,11 +38,8 @@ import { scenarioCaptureEvidence, scenarioCaptureFixture, } from "../../../tests/fixtures/browserScenarioCapture.js"; -import { readEvidenceFileSource } from "./evidenceFileSource.js"; import { projectAnalysisError } from "../../domain/analysisErrorProjection.js"; -vi.mock("./evidenceFileSource.js", { spy: true }); - it.each(["layout", "javascript"])( "carries the retained %s source snapshot into repeatable shared views", (kind) => { @@ -1077,7 +1074,7 @@ describe("inspectAnalysisViewWithFiles", () => { }); }); -describe("inspectAnalysisViewWithFiles scenario parents and cancellation", () => { +describe("inspectAnalysisViewWithFiles scenario parents", () => { const view = manifestView; it("accepts from a file a scenario capture whose response bodies exceed the inline budget", async () => { @@ -1109,33 +1106,4 @@ describe("inspectAnalysisViewWithFiles scenario parents and cancellation", () => parent_evidence_id: parent.evidence_id, }); }); - - it("reports cancellation rather than the authentication failure of a file aborted while it is authenticated", async () => { - const controller = new AbortController(); - const parent = analysisViewCaptureEvidence( - environmentManifestCaptureFixture(), - ); - const path = await writeEvidenceFile({ - ...parent, - evidence_id: `ev_${"0".repeat(64)}`, - }); - const { readEvidenceFileSource: read } = await vi.importActual< - typeof import("./evidenceFileSource.js") - >("./evidenceFileSource.js"); - // The abort arrives once the read has finished, while the stale ID is being authenticated. - vi.mocked(readEvidenceFileSource).mockImplementationOnce( - async (...args) => { - const loaded = await read(...args); - controller.abort(); - return loaded; - }, - ); - expect( - await inspectAnalysisViewWithFiles( - { source: { kind: "evidence-file", path }, view }, - undefined, - { signal: controller.signal }, - ), - ).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" } }); - }); }); diff --git a/src/application/frida/FridaInstrumentationService.test.ts b/src/application/frida/FridaInstrumentationService.test.ts index d42d98a8a6..4ce961f4c4 100644 --- a/src/application/frida/FridaInstrumentationService.test.ts +++ b/src/application/frida/FridaInstrumentationService.test.ts @@ -12,6 +12,8 @@ import { AnalysisCapabilityUnavailableError, AnalysisInputError, } from "../../domain/analysisErrorCore.js"; +import type { Evidence } from "../../domain/evidence.js"; +import { ok } from "../../domain/result.js"; const createSession = () => ({ sessionId: "00000000-0000-4000-8000-000000000001", @@ -65,6 +67,109 @@ const createProvider = ( ...overrides, }); +it("keeps default Frida status identity and messages and refreshes every summary", () => { + let reads = 0; + const provider = createProvider({ + status() { + reads++; + return { + ...createSession(), + state: reads === 1 ? "paused" : "detached", + targetResumeObserved: false, + scripts: [], + messages: [{ observed: reads }], + messagesTruncated: reads > 1, + }; + }, + }); + const service = new FridaInstrumentationService(provider); + const records = new Map(); + const retention = { + evidenceById: (id: string) => records.get(id), + recordEvidence(evidence: Evidence) { + records.set(evidence.evidence_id, evidence); + return ok("added" as const); + }, + }; + const input = { session_id: createSession().sessionId }; + const complete = service.inspectStatus(input); + if (!complete.ok) throw complete.error; + expect(complete.value.normalized_result).toMatchObject({ + state: "paused", + messages: [{ observed: 1 }], + messages_truncated: false, + }); + const summary = service.inspectStatus( + { ...input, detail: "summary" }, + retention, + ); + if (!summary.ok) throw summary.error; + expect(summary.value.normalized_result).toMatchObject({ + parent_operation: "frida_session_status", + artifact: null, + source_provider: complete.value.provider, + source_parameters: input, + summary: { + state: "detached", + messages_retained: 1, + messages_truncated: true, + }, + }); + expect(JSON.stringify(summary.value)).not.toContain('"messages":'); + expect(reads).toBe(2); + const parent = [...records.values()][0]; + expect(parent?.normalized_result).toMatchObject({ + messages: [{ observed: 2 }], + }); + expect(parent?.authority).toBe("external-service"); + expect(parent?.environment).toBeNull(); + expect(parent?.raw_result).toBeNull(); +}); + +it.each([ + undefined, + { recordEvidence: undefined, evidenceById: () => undefined }, + { recordEvidence: () => ok("added" as const), evidenceById: undefined }, +])( + "rejects missing Frida summary retention before reading live state: %j", + (retention) => { + let reads = 0; + const service = new FridaInstrumentationService( + createProvider({ + status() { + reads++; + return undefined; + }, + }), + ); + const result = service.inspectStatus( + { session_id: createSession().sessionId, detail: "summary" }, + retention, + ); + expect(result.ok).toBe(false); + if (!result.ok) + expect(result.error._tag).toBe("AnalysisCapabilityUnavailableError"); + expect(reads).toBe(0); + }, +); + +it("rejects malformed Frida status selections and preserves unknown-session errors", () => { + const service = new FridaInstrumentationService(createProvider()); + for (const input of [ + { session_id: "invalid" }, + { session_id: createSession().sessionId, detail: "messages" }, + ]) { + const result = service.inspectStatus(input); + expect(result.ok).toBe(false); + if (!result.ok) expect(result.error._tag).toBe("AnalysisInputError"); + } + const unknown = service.inspectStatus({ + session_id: createSession().sessionId, + }); + expect(unknown.ok).toBe(false); + if (!unknown.ok) expect(unknown.error._tag).toBe("AnalysisInputError"); +}); + it.each(["startSession", "instrument"] as const)( "does not start Frida when %s receives an already-cancelled signal", async (method) => { diff --git a/src/application/frida/FridaInstrumentationService.ts b/src/application/frida/FridaInstrumentationService.ts index 28974e90c3..0aac571ade 100644 --- a/src/application/frida/FridaInstrumentationService.ts +++ b/src/application/frida/FridaInstrumentationService.ts @@ -14,6 +14,14 @@ import { AnalysisInputError, } from "../../domain/analysisErrorCore.js"; import { err, ok, type Result } from "../../domain/result.js"; +import { fridaSessionStatusInputSchema } from "../../domain/fridaSessionStatus.js"; +import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js"; +import { + requireAnalysisViewRetention, + summarizeRetainedAnalysis, + type AnalysisViewRetention, +} from "../analysisView/AnalysisViewService.js"; +import { unknownFridaSession } from "./fridaInputErrors.js"; import { FridaScriptLoadCancelledError } from "./FridaInstrumentationPort.js"; import type { FridaDeviceSelector, @@ -237,13 +245,45 @@ export class FridaInstrumentationService { return this.#provider.status(sessionId); } - statusEvidence(sessionId: string): Evidence | undefined { - const status = this.#provider.status(sessionId); - return status === undefined - ? undefined - : fridaEvidence("frida_session_status", sessionStatusOutput(status), { - session_id: sessionId, - }); + inspectStatus( + rawInput: unknown, + retention: AnalysisViewRetention = { + recordEvidence: undefined, + evidenceById: undefined, + }, + ): Result { + const input = fridaSessionStatusInputSchema.safeParse(rawInput); + if (!input.success) + return err( + analysisInputErrorFromIssues( + "frida_session_status", + input.error.issues, + rawInput, + { cause: input.error }, + ), + ); + if (input.data.detail === "summary") { + const ready = requireAnalysisViewRetention( + "frida_session_status", + retention, + ); + if (!ready.ok) return ready; + } + const status = this.#provider.status(input.data.session_id); + if (status === undefined) + return err( + unknownFridaSession("frida_session_status", input.data.session_id), + ); + const evidence = fridaEvidence( + "frida_session_status", + sessionStatusOutput(status), + { + session_id: input.data.session_id, + }, + ); + return input.data.detail === "summary" + ? summarizeRetainedAnalysis(evidence, retention) + : ok(evidence); } async instrument( diff --git a/src/contracts/analysisViewToolContracts.ts b/src/contracts/analysisViewToolContracts.ts index f103a619af..e14192da88 100644 --- a/src/contracts/analysisViewToolContracts.ts +++ b/src/contracts/analysisViewToolContracts.ts @@ -17,7 +17,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [ ...toolContractMetadata("inspect_analysis_view"), kind: "application", description: - 'Inspect selected views of completed GDB/objdump command observations, Go build metadata, binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_macho, decode_interface_builder, inspect_evm_interface, inspect_managed_artifact, inspect_managed_members, inspect_android_package, inspect_dart_aot, inspect_il2cpp_metadata, inspect_model_artifact or inspect_web_network_capture Evidence without repeating analysis. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the first related write and reports entity types with an identity field (proved by a path parameter or inferred from a field name), entities in collection order with per-view values where views differ, documents for non-parameterized non-entity paths written by POST/PUT/PATCH (item-shaped and delete-only paths stay recorded), and a render plan per read tool: item and list views reproduce their latest recordings from the state, and every other read tool is a recorded view with a reason. Types and identity are inferences; values are recorded data, so it refuses parents inspected with sensitive values, and a scenario parent must have selected response bodies; its values keep the capture redaction markers, and a write whose URL it redacted ends the state of every type. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. A completed Evidence JSON file on this host uses {kind: "evidence-file", path: ""}: REA reads and authenticates it, then projects it like retained Evidence, so a large capture is bounded by the file reader rather than by inline-input accounting. For layout, JavaScript, inventory, WASM, Mach-O, Interface Builder, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Typed module references use view {kind: "javascript", collection: "modules", projection: "references", where: {path: "main.js", kind: "javascript-module"}, offset: 0, limit: 64}; where matches exact retained strings (null selects unavailable paths). Rows include canonical identity, retained source-range count and observation support states. Source coverage and selection remain explicit; exhausting matching rows does not prove runtime loading or complete analysis. For exact file/native facts and processing gaps, use view {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"]}. Rows page original observations with canonical identities and Evidence; missing properties remain omitted. Paths match recorded artifact-path/source-range locations; optional kinds filter nodes. To discover field names without values, opt into include_available_property_keys: true with property_keys: []. Canonical identity and Evidence are separate from properties; keep each node_id/observation_id pair together when composing results. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions. Occurrences accept exact logical_paths and occurrence_ids; nodes accept artifact_ids. Batch filters intersect (omitted means all available, [] means none); fields selects columns while retaining identity links. source_complete describes retained graph membership; source_coverage describes producer coverage, including unknown encrypted PCK directories. Page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. Mach-O views use kind macho and facet entrypoints/architectures/build_metadata/load_commands/dependencies/imports/exports/segments/provenance; source_exhaustive false means unreturned rows may exist. Interface Builder views use kind interface-builder and collection documents/nodes/edges/coverage; nodes and edges accept an exact document relative_path. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Managed member summaries keep identity, selection with unmatched names, coverage and per-collection counts; managed-members pages use collection types/exported_types/fields/methods/member_refs/call_edges/field_accesses/user_strings and carry the source selection and coverage. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Protobuf views use kind protobuf, collection bodies/frames/fields, with record_ordinals, body_pointers and fields; field pages also select frame_indices, field_ids, field_numbers or schema_names. Byte identity, offset basis and body status remain present. Text and schema_value are opt-in page columns. Model views use kind model-artifact and collection metadata (keys), tensors (names/types) or types (derived totals), with indices, fields and paging. Header SHA-256 scopes declarations; full-file identity remains unknown unless full hashing was selected. Arrays retain typed eight-element previews with explicit omissions. IL2CPP metadata views use kind il2cpp-metadata and collection sections/images/types/methods/fields/properties/events/parameters/string_literals/code_gen_modules/method_addresses/return_types/field_types/parameter_types. Select indices, fields and pages; named rows accept names, types also accept image_indices/namespaces/declaring_types, and methods/members accept declaring_types. Parameters and parameter_types accept method_indices; code_gen_modules accept image_indices; method_addresses accept addresses/states, preserving every folded alternative. Row index is scoped to the metadata digest; native pages add binary_source. Uncollected native tables have null counts and remain unavailable. Desktop directory views use kind desktop-application and collection files/omissions/candidates/components/next_probes, with fields and pagination; files and omissions accept exact paths or their stable IDs, and candidates accept families. Source coverage and stable row IDs remain available. Native inventory Evidence from list_names, list/search_procedures or list/search_strings uses view: {kind: "native-inventory", offset: 0, limit: 64, fields: ["value"]}. Rows keep source_index and address even with selected fields. Summary counts describe retained rows; provider discovery coverage remains unknown. Native function Evidence requires kind native: view: {kind: "native", facet: "procedure"}. Facets select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', + 'Inspect selected views of completed Frida session status, GDB/objdump command observations, Go build metadata, binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_macho, decode_interface_builder, inspect_evm_interface, inspect_managed_artifact, inspect_managed_members, inspect_android_package, inspect_dart_aot, inspect_il2cpp_metadata, inspect_model_artifact or inspect_web_network_capture Evidence without repeating analysis. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the first related write and reports entity types with an identity field (proved by a path parameter or inferred from a field name), entities in collection order with per-view values where views differ, documents for non-parameterized non-entity paths written by POST/PUT/PATCH (item-shaped and delete-only paths stay recorded), and a render plan per read tool: item and list views reproduce their latest recordings from the state, and every other read tool is a recorded view with a reason. Types and identity are inferences; values are recorded data, so it refuses parents inspected with sensitive values, and a scenario parent must have selected response bodies; its values keep the capture redaction markers, and a write whose URL it redacted ends the state of every type. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. A completed Evidence JSON file on this host uses {kind: "evidence-file", path: ""}: REA reads and authenticates it, then projects it like retained Evidence, so a large capture is bounded by the file reader rather than by inline-input accounting. For layout, JavaScript, inventory, WASM, Mach-O, Interface Builder, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Typed module references use view {kind: "javascript", collection: "modules", projection: "references", where: {path: "main.js", kind: "javascript-module"}, offset: 0, limit: 64}; where matches exact retained strings (null selects unavailable paths). Rows include canonical identity, retained source-range count and observation support states. Source coverage and selection remain explicit; exhausting matching rows does not prove runtime loading or complete analysis. For exact file/native facts and processing gaps, use view {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"]}. Rows page original observations with canonical identities and Evidence; missing properties remain omitted. Paths match recorded artifact-path/source-range locations; optional kinds filter nodes. To discover field names without values, opt into include_available_property_keys: true with property_keys: []. Canonical identity and Evidence are separate from properties; keep each node_id/observation_id pair together when composing results. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions. Occurrences accept exact logical_paths and occurrence_ids; nodes accept artifact_ids. Batch filters intersect (omitted means all available, [] means none); fields selects columns while retaining identity links. source_complete describes retained graph membership; source_coverage describes producer coverage, including unknown encrypted PCK directories. Page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. Mach-O views use kind macho and facet entrypoints/architectures/build_metadata/load_commands/dependencies/imports/exports/segments/provenance; source_exhaustive false means unreturned rows may exist. Interface Builder views use kind interface-builder and collection documents/nodes/edges/coverage; nodes and edges accept an exact document relative_path. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Managed member summaries keep identity, selection with unmatched names, coverage and per-collection counts; managed-members pages use collection types/exported_types/fields/methods/member_refs/call_edges/field_accesses/user_strings and carry the source selection and coverage. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Protobuf views use kind protobuf, collection bodies/frames/fields, with record_ordinals, body_pointers and fields; field pages also select frame_indices, field_ids, field_numbers or schema_names. Byte identity, offset basis and body status remain present. Text and schema_value are opt-in page columns. Model views use kind model-artifact and collection metadata (keys), tensors (names/types) or types (derived totals), with indices, fields and paging. Header SHA-256 scopes declarations; full-file identity remains unknown unless full hashing was selected. Arrays retain typed eight-element previews with explicit omissions. IL2CPP metadata views use kind il2cpp-metadata and collection sections/images/types/methods/fields/properties/events/parameters/string_literals/code_gen_modules/method_addresses/return_types/field_types/parameter_types. Select indices, fields and pages; named rows accept names, types also accept image_indices/namespaces/declaring_types, and methods/members accept declaring_types. Parameters and parameter_types accept method_indices; code_gen_modules accept image_indices; method_addresses accept addresses/states, preserving every folded alternative. Row index is scoped to the metadata digest; native pages add binary_source. Uncollected native tables have null counts and remain unavailable. Desktop directory views use kind desktop-application and collection files/omissions/candidates/components/next_probes, with fields and pagination; files and omissions accept exact paths or their stable IDs, and candidates accept families. Source coverage and stable row IDs remain available. Native inventory Evidence from list_names, list/search_procedures or list/search_strings uses view: {kind: "native-inventory", offset: 0, limit: 64, fields: ["value"]}. Rows keep source_index and address even with selected fields. Summary counts describe retained rows; provider discovery coverage remains unknown. Native function Evidence requires kind native: view: {kind: "native", facet: "procedure"}. Facets select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', inputSchema: inspectAnalysisViewInputSchema, outputSchema: evidenceResultOf(analysisViewResultSchema), examples: [ diff --git a/src/contracts/fridaToolContracts.ts b/src/contracts/fridaToolContracts.ts index d470eb75fd..d7a6c264e2 100644 --- a/src/contracts/fridaToolContracts.ts +++ b/src/contracts/fridaToolContracts.ts @@ -3,8 +3,15 @@ import { z } from "zod"; import { localPathStringSchema } from "../domain/localPath.js"; import type { ToolContract } from "./toolContractTypes.js"; import { evidenceResultOf } from "./toolOutputSchemaPrimitives.js"; +import { analysisDetailOutputSchemaOf } from "./analysisDetail.js"; import { toolContractMetadata } from "./toolEffects.js"; import { isValidFridaRemoteAddress } from "./fridaRemoteAddress.js"; +import { + fridaSessionObservationSchema, + fridaSessionStatusInputSchema, + fridaSessionStatusSchema, +} from "../domain/fridaSessionStatus.js"; +import { fridaSessionStatusSummaryViewResultSchema } from "../domain/analysisView/analysisView.js"; export const FRIDA_KEEPALIVE_INTERVAL_DESCRIPTION = "Remote keepalive interval in seconds"; @@ -45,15 +52,6 @@ const exclusiveDeviceSelector = (schema: Schema) => const sessionId = z.string().uuid(); -const sessionOutput = z.strictObject({ - session_id: sessionId, - device_id: z.string(), - target: z.string(), - pid: z.number().int().nonnegative(), - mode: z.enum(["attach", "spawn"]), - state: z.enum(["paused", "running", "detached"]), -}); - const scriptOutput = z.strictObject({ script_id: z.string().uuid(), source_kind: z.enum(["inline", "file"]), @@ -142,7 +140,7 @@ export const FRIDA_TOOL_CONTRACTS = [ !(value.device_id !== undefined && value.remote !== undefined), "Select one device; attach requires pid; spawn requires program and rejects pid", ), - outputSchema: sessionOutput, + outputSchema: fridaSessionObservationSchema, examples: [ { title: "Attach to a process", @@ -226,22 +224,11 @@ export const FRIDA_TOOL_CONTRACTS = [ ...toolContractMetadata("frida_session_status"), kind: "native-provider", description: - "Return observed Frida session state and queued script messages. Message history is bounded by the provider capture budget and reports truncation.", - inputSchema: z.strictObject({ session_id: sessionId }), - outputSchema: evidenceResultOf( - z.strictObject({ - session_id: sessionId, - device_id: z.string(), - state: z.enum(["paused", "running", "detached"]), - target: z.string(), - pid: z.number().int().nonnegative(), - mode: z.enum(["attach", "spawn"]), - scripts: z.array( - z.strictObject({ script_id: z.string().uuid(), name: z.string() }), - ), - messages: z.array(z.unknown()), - messages_truncated: z.boolean(), - }), + "Read observed Frida session state on each call. Default detail=complete returns queued script messages and capture truncation. detail=summary retains the complete observation in this session and returns identity, state, scripts, retained message count and truncation without payloads. Export the complete parent with export_evidence_bundle; retained summaries are historical and never refresh live state.", + inputSchema: fridaSessionStatusInputSchema, + outputSchema: analysisDetailOutputSchemaOf( + fridaSessionStatusSchema, + fridaSessionStatusSummaryViewResultSchema, ), examples: [ { @@ -302,13 +289,7 @@ export const FRIDA_TOOL_CONTRACTS = [ "Select one device and provide fields matching the mode and source_kind", ), outputSchema: evidenceResultOf( - z.strictObject({ - ...sessionOutput.shape, - scripts: z.array( - z.strictObject({ script_id: sessionId, name: z.string() }), - ), - messages: z.array(z.unknown()), - messages_truncated: z.boolean(), + fridaSessionStatusSchema.extend({ source_kind: z.enum(["inline", "file"]), source_path: z.string().nullable(), source_sha256: z.string().regex(/^[a-f0-9]{64}$/u), diff --git a/src/domain/analysisView/analysisView.ts b/src/domain/analysisView/analysisView.ts index 29ca29cb99..d991e789ed 100644 --- a/src/domain/analysisView/analysisView.ts +++ b/src/domain/analysisView/analysisView.ts @@ -10,6 +10,11 @@ import { gdbCommandSummarySchema, projectCommandObservationView, } from "./commandObservationView.js"; +import { + fridaSessionStatusSummarySchema, + fridaSessionStatusViewContext, + projectFridaSessionStatusView, +} from "./fridaSessionStatusView.js"; import { commandObservationSchema } from "../commandObservation.js"; import { goBinarySummarySchema, @@ -385,6 +390,7 @@ const viewResultBase = { "inspect_binary_layout", "inspect_go_binary", "gdb_console", + "frida_session_status", "inspect_with_objdump", "inspect_desktop_application", "analyze_javascript_application", @@ -437,6 +443,14 @@ export const objdumpSummaryViewResultSchema = z.strictObject({ summary: commandObservationSchema, }); +export const fridaSessionStatusSummaryViewResultSchema = z.strictObject({ + ...summaryViewResultBase, + ...fridaSessionStatusViewContext, + parent_operation: z.literal("frida_session_status"), + artifact: artifactSchema.nullable(), + summary: fridaSessionStatusSummarySchema, +}); + /** Opt-in Go delivery retains original bytes and exposes build identity/counts. */ export const goBinarySummaryViewResultSchema = z.strictObject({ ...summaryViewResultBase, @@ -645,6 +659,7 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ z.discriminatedUnion("parent_operation", [ gdbCommandSummaryViewResultSchema, objdumpSummaryViewResultSchema, + fridaSessionStatusSummaryViewResultSchema, goBinarySummaryViewResultSchema, binaryLayoutSummaryViewResultSchema, desktopApplicationSummaryViewResultSchema, @@ -930,6 +945,7 @@ const parentArtifactPath = (parent: AnalysisViewParent): string => { /** Named source owners for views that need no extra decoder or accumulator. */ const retainedViewProjectors = new Map([ + ["frida_session_status", projectFridaSessionStatusView], ["gdb_console", projectCommandObservationView], ["inspect_with_objdump", projectCommandObservationView], ["inspect_go_binary", projectGoBinaryView], diff --git a/src/domain/analysisView/fridaSessionStatusView.ts b/src/domain/analysisView/fridaSessionStatusView.ts new file mode 100644 index 0000000000..87ee04b696 --- /dev/null +++ b/src/domain/analysisView/fridaSessionStatusView.ts @@ -0,0 +1,86 @@ +import { z } from "zod"; +import type { AnalysisError } from "../analysisErrorBase.js"; +import { providerSchema } from "../evidence.js"; +import { fridaSessionStatusSchema } from "../fridaSessionStatus.js"; +import { err, ok, type Result } from "../result.js"; +import type { + AnalysisViewParent, + AnalysisViewRequest, + UnsignedAnalysisView, +} from "./analysisView.js"; +import { + boundedViewMessages, + completeWithinViewCoverage, +} from "./analysisViewCoverage.js"; +import { analysisViewInputError } from "./analysisViewErrors.js"; +import { parseAnalysisViewResult } from "./analysisViewParent.js"; + +export const fridaSessionStatusSummarySchema = fridaSessionStatusSchema + .omit({ messages: true }) + .extend({ messages_retained: z.number().int().nonnegative() }); + +export const fridaSessionStatusViewContext = { + source_provider: providerSchema.nullable(), + source_parameters: z.strictObject({ session_id: z.string().uuid() }), +}; + +/** Summarize captured facts without interpreting caller-authored message payloads. */ +export const projectFridaSessionStatusView = ( + parent: AnalysisViewParent, + view: AnalysisViewRequest, +): Result => { + if (view.kind !== "summary") + return err( + analysisViewInputError([ + { + path: ["view"], + reason: "invalid_value", + message: "Frida session status supports summary views.", + }, + ]), + ); + const report = parseAnalysisViewResult(fridaSessionStatusSchema, parent); + if (!report.ok) return report; + const parameters = fridaSessionStatusViewContext.source_parameters.safeParse( + parent.parameters, + ); + if ( + !parameters.success || + parameters.data.session_id !== report.value.session_id + ) + return err( + analysisViewInputError([ + { + path: ["source", "parameters", "session_id"], + reason: "invalid_value", + message: + "The Frida status must identify the session selected by its source parameters.", + }, + ]), + ); + const { messages, ...status } = report.value; + return ok({ + kind: "summary", + view, + parent_evidence_id: parent.evidenceId, + parent_digest: parent.evidenceId.slice(3), + parent_operation: "frida_session_status", + artifact: null, + source_provider: parent.provider ?? null, + source_parameters: parameters.data, + summary: { ...status, messages_retained: messages.length }, + coverage: completeWithinViewCoverage(1, 1), + limitations: boundedViewMessages( + [ + ...parent.limitations, + `Message payloads remain in complete parent Evidence ${parent.evidenceId}; export_evidence_bundle preserves them.`, + ], + parent.evidenceId, + "limitations", + ), + unknowns: [ + "This is a captured session observation; querying retained Evidence does not refresh live state or establish target process identity after exit or reuse.", + "messages_retained counts queued messages, not all target events. messages_truncated preserves the provider's capture limitation; payload meanings and global event coverage are unknown.", + ], + }); +}; diff --git a/src/domain/fridaSessionStatus.ts b/src/domain/fridaSessionStatus.ts new file mode 100644 index 0000000000..fbff44043e --- /dev/null +++ b/src/domain/fridaSessionStatus.ts @@ -0,0 +1,27 @@ +import { z } from "zod"; + +/** Delivery selection is separate from the provider's live session lookup. */ +export const fridaSessionStatusInputSchema = z.strictObject({ + session_id: z.string().uuid(), + detail: z.enum(["complete", "summary"]).optional(), +}); + +/** Session identity and state shared by start, status and one-shot observations. */ +export const fridaSessionObservationSchema = z.strictObject({ + session_id: z.string().uuid(), + device_id: z.string(), + state: z.enum(["paused", "running", "detached"]), + target: z.string(), + pid: z.number().int().nonnegative(), + mode: z.enum(["attach", "spawn"]), +}); + +/** Canonical captured facts; message payload meanings remain caller-defined. */ +export const fridaSessionStatusSchema = fridaSessionObservationSchema.extend({ + scripts: z.array( + z.strictObject({ script_id: z.string().uuid(), name: z.string() }), + ), + // Evidence validates JSON payloads; status views validate only this envelope. + messages: z.array(z.unknown()), + messages_truncated: z.boolean(), +}); diff --git a/src/server/createServer.ts b/src/server/createServer.ts index d3eee29988..03fef3ef53 100644 --- a/src/server/createServer.ts +++ b/src/server/createServer.ts @@ -535,6 +535,7 @@ export const createServer = ( session === undefined ? undefined : (evidence) => session.recordEvidence(evidence), + evidenceById, ); registerCutterTools( server, diff --git a/src/server/registerFridaTools.ts b/src/server/registerFridaTools.ts index 016f4afb94..0d9218694a 100644 --- a/src/server/registerFridaTools.ts +++ b/src/server/registerFridaTools.ts @@ -2,6 +2,7 @@ import type { McpServer, ServerContext } from "@modelcontextprotocol/server"; import type { Logger } from "pino"; import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js"; +import type { EvidenceLookup } from "../application/EvidenceInputResolver.js"; import { FridaInstrumentationService } from "../application/frida/FridaInstrumentationService.js"; import type { FridaInstrumentationPort } from "../application/frida/FridaInstrumentationPort.js"; import type { FridaRemoteConnection } from "../application/frida/FridaInstrumentationPort.js"; @@ -14,7 +15,6 @@ import { jsonValueSchema } from "../domain/jsonValue.js"; import { logToolExecution } from "./toolLogging.js"; import { toolRegistrationOptions } from "./toolRegistrationOptions.js"; import type { ToolResultDelivery } from "./toolResult.js"; -import { unknownFridaSession } from "../application/frida/fridaInputErrors.js"; /** Register Frida device discovery and persistent instrumentation operations. */ export const registerFridaTools = ( @@ -23,6 +23,7 @@ export const registerFridaTools = ( logger: Logger, delivery: ToolResultDelivery, recordEvidence?: EvidenceWriter["recordEvidence"], + evidenceById?: EvidenceLookup, ): void => { const service = new FridaInstrumentationService(provider); const listDevices = toolContract("list_frida_devices"); @@ -193,19 +194,15 @@ export const registerFridaTools = ( status.name, toolRegistrationOptions(status), async (input) => { - const evidence = service.statusEvidence(input.session_id); - if (evidence === undefined) - return delivery.toCallToolResult( - { - ok: false as const, - error: unknownFridaSession(status.name, input.session_id), - }, - status, - ); + const result = service.inspectStatus(input, { + recordEvidence, + evidenceById, + }); + if (!result.ok) return delivery.toCallToolResult(result, status); return delivery.toEvidenceToolResult( - evidence, + result.value, status, - recordEvidence?.(evidence), + recordEvidence?.(result.value), ); }, ); diff --git a/tests/boundary/cli/fridaStatusViewCli.test.ts b/tests/boundary/cli/fridaStatusViewCli.test.ts new file mode 100644 index 0000000000..67fb617c5f --- /dev/null +++ b/tests/boundary/cli/fridaStatusViewCli.test.ts @@ -0,0 +1,71 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect } from "vitest"; +import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js"; +import { fridaSessionStatusSummaryViewResultSchema } from "../../../src/domain/analysisView/analysisView.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { cliTest } from "../../support/cli/cliFixture.js"; + +cliTest( + "summarizes historical Frida status from a complete Evidence file without a live session", + async ({ cli }) => { + const id = "00000000-0000-4000-8000-000000000001"; + const parent = createEvidence( + undefined, + { id: "frida", name: "Frida", version: null }, + { + operation: "frida_session_status", + parameters: { session_id: id }, + authority: "external-service", + environment: null, + limitations: ["Target OS and architecture were not observed."], + result: { + session_id: id, + device_id: "remote", + target: "selected target", + pid: 0, + mode: "attach", + state: "detached", + scripts: [], + messages: [{ payload: "retained only" }, null], + messages_truncated: true, + }, + }, + ); + const root = await createTestTempDirectory("rea-frida-status-view-"); + const source = join(root, "parent.json"); + const request = join(root, "request.json"); + await writeFile(source, JSON.stringify(parent)); + await writeFile( + request, + JSON.stringify({ + source: { kind: "evidence-file", path: source }, + view: { kind: "summary" }, + }), + ); + const result = await cli.run({ + arguments: ["inspect-analysis-view", request, "--json"], + }); + expect(result.exitCode).toBe(0); + const evidence = parseEvidence(result.json); + const view = fridaSessionStatusSummaryViewResultSchema.parse( + evidence.normalized_result, + ); + expect(view).toMatchObject({ + parent_evidence_id: parent.evidence_id, + source_provider: parent.provider, + source_parameters: parent.parameters, + artifact: null, + summary: { + pid: 0, + state: "detached", + messages_retained: 2, + messages_truncated: true, + }, + }); + expect(view.unknowns.join(" ")).toContain("does not refresh live state"); + expect(view.limitations).toContain(parent.limitations[0]); + expect(evidence.authority).toBe("external-service"); + expect(JSON.stringify(evidence)).not.toContain("retained only"); + }, +); diff --git a/tests/boundary/mcp/fridaLifecycleMcp.test.ts b/tests/boundary/mcp/fridaLifecycleMcp.test.ts index 328169ca7c..7fa88f259a 100644 --- a/tests/boundary/mcp/fridaLifecycleMcp.test.ts +++ b/tests/boundary/mcp/fridaLifecycleMcp.test.ts @@ -26,6 +26,7 @@ import type { AnalysisPartialObservation } from "../../../src/domain/analysisErr import { AnalysisCancelledError } from "../../../src/domain/analysisErrorCore.js"; import { EvidenceIntegrityError } from "../../../src/domain/evidenceErrors.js"; import { evidenceBundleSchema } from "../../../src/domain/evidenceBundle.js"; +import { fridaSessionStatusSchema } from "../../../src/domain/fridaSessionStatus.js"; import type { Evidence } from "../../../src/domain/evidence.js"; import { jsonObjectSchema } from "../../../src/domain/jsonValue.js"; import { nativeCallPartialObservationSchema } from "../../../src/domain/native/nativeCallPartialObservation.js"; @@ -268,9 +269,11 @@ it.each(["attached", "during-resume", "after-resume"] as const)( name: "frida_session_status", arguments: { session_id: id }, }); - return toolContract("frida_session_status").outputSchema.parse( - result.structuredContent, - ).normalized_result.state; + return fridaSessionStatusSchema.parse( + toolContract("frida_session_status").outputSchema.parse( + result.structuredContent, + ).normalized_result, + ).state; }; const resuming = fixture.client.callTool({ name: "resume_frida_session", @@ -318,9 +321,11 @@ it("retains detached state and resume evidence when remote cleanup needs retry", arguments: { session_id: id }, }); expect( - toolContract("frida_session_status").outputSchema.parse( - result.structuredContent, - ).normalized_result.state, + fridaSessionStatusSchema.parse( + toolContract("frida_session_status").outputSchema.parse( + result.structuredContent, + ).normalized_result, + ).state, ).toBe("detached"); expect(fixture.manager.status(id)?.targetResumeObserved).toBe(true); expect(fixture.controls.detachCalls).toBe(1); diff --git a/tests/boundary/mcp/fridaMcp.test.ts b/tests/boundary/mcp/fridaMcp.test.ts index 8ed3cc8746..52ece8d583 100644 --- a/tests/boundary/mcp/fridaMcp.test.ts +++ b/tests/boundary/mcp/fridaMcp.test.ts @@ -9,6 +9,10 @@ import { expandToolAvailability } from "../../../src/contracts/toolOutputSchemaP import { FridaInstrumentationManager } from "../../../src/frida/FridaInstrumentationManager.js"; import { createServer } from "../../../src/server/createServer.js"; import { createTestBinarySession } from "../../fixtures/binarySession.js"; +import { fridaSessionStatusSummaryViewResultSchema } from "../../../src/domain/analysisView/analysisView.js"; +import { fridaSessionStatusSchema } from "../../../src/domain/fridaSessionStatus.js"; +import { evidenceBundleSummarySchema } from "../../../src/domain/evidenceRecordSummary.js"; +import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js"; const createFridaTestProvider = ( overrides: Partial = {}, @@ -90,6 +94,158 @@ const createFridaTestProvider = ( ...overrides, }); +it.each([false, true])( + "delivers Frida status summaries with complete retained Evidence (truncated=%s)", + async (truncated) => { + const id = "00000000-0000-4000-8000-000000000001"; + const messages = [ + { + type: "send", + payload: { token: "caller-selected-nonsecret", rows: [1, 2] }, + }, + ]; + let reads = 0; + const session = createTestBinarySession(() => { + throw new Error("No binary provider"); + }); + const server = createServer( + { kind: "session", session }, + { + fridaInstrumentation: createFridaTestProvider({ + status(sessionId) { + reads++; + return { + sessionId, + deviceId: "local", + target: "fixture", + pid: 10, + mode: "attach", + state: "running", + targetResumeObserved: false, + scripts: [], + messages, + messagesTruncated: truncated, + }; + }, + }), + }, + ); + const client = new Client({ name: "frida-status-summary", version: "1" }); + onTestFinished(async () => { + await client.close(); + await server.close(); + await session.close(); + }); + const [ct, st] = InMemoryTransport.createLinkedPair(); + await server.connect(st); + await client.connect(ct); + const tools = (await client.listTools()).tools; + const tool = tools.find((t) => t.name === "frida_session_status"); + if (!tool?.outputSchema) throw new Error("Missing Frida status schema"); + const ajv = new Ajv2020({ strict: false, validateFormats: false }); + expect(ajv.validateSchema(tool.inputSchema)).toBe(true); + const outputSchema: Record = tool.outputSchema; + expect(ajv.validateSchema(outputSchema)).toBe(true); + const complete = await client.callTool({ + name: tool.name, + arguments: { session_id: id }, + }); + expect(complete.isError).not.toBe(true); + const parent = parseEvidence( + toolContract("frida_session_status").outputSchema.parse( + complete.structuredContent, + ), + ); + expect( + fridaSessionStatusSchema.parse(parent.normalized_result).messages, + ).toEqual(messages); + const explicit = await client.callTool({ + name: tool.name, + arguments: { session_id: id, detail: "complete" }, + }); + expect(explicit.structuredContent).toEqual(parent); + const summary = await client.callTool({ + name: tool.name, + arguments: { session_id: id, detail: "summary" }, + }); + expect(summary.isError).not.toBe(true); + expect(ajv.validate(outputSchema, summary.structuredContent)).toBe(true); + const evidence = toolContract("frida_session_status").outputSchema.parse( + summary.structuredContent, + ); + const view = fridaSessionStatusSummaryViewResultSchema.parse( + evidence.normalized_result, + ); + expect(view).toMatchObject({ + parent_evidence_id: parent.evidence_id, + source_provider: parent.provider, + source_parameters: parent.parameters, + artifact: null, + summary: { + session_id: id, + state: "running", + scripts: [], + messages_retained: 1, + messages_truncated: truncated, + }, + }); + expect(evidence.authority).toBe("external-service"); + expect(JSON.stringify(view)).not.toContain("caller-selected-nonsecret"); + const retained = await client.callTool({ + name: "inspect_analysis_view", + arguments: { + source: { kind: "retained-evidence", evidence_id: parent.evidence_id }, + view: { kind: "summary" }, + }, + }); + expect(retained.structuredContent).toEqual(evidence); + expect(reads).toBe(3); + const discovered = await client.callTool({ + name: "get_evidence_bundle", + arguments: { + detail: "summary", + filters: { evidence_id: parent.evidence_id }, + }, + }); + expect(discovered.isError).not.toBe(true); + const discovery = evidenceBundleSummarySchema.parse( + toolContract("get_evidence_bundle").outputSchema.parse( + discovered.structuredContent, + ).result, + ); + expect(discovery.matching_records).toBe(1); + expect(discovery.records.map((record) => record.evidence_id)).toEqual([ + parent.evidence_id, + ]); + expect(JSON.stringify(discovered.structuredContent)).not.toContain( + "caller-selected-nonsecret", + ); + const bundle = await client.callTool({ + name: "get_evidence_bundle", + arguments: {}, + }); + expect(bundle.isError).not.toBe(true); + expect(JSON.stringify(bundle.structuredContent)).toContain( + "caller-selected-nonsecret", + ); + expect(session.evidenceById(parent.evidence_id)).toEqual(parent); + // Inline/file projection validates the owning result and exact source selection. + const malformed = createEvidence(undefined, parent.provider ?? undefined, { + operation: parent.operation, + parameters: { session_id: "00000000-0000-4000-8000-000000000099" }, + result: parent.normalized_result, + }); + const rejected = await client.callTool({ + name: "inspect_analysis_view", + arguments: { + source: { kind: "inline", evidence: malformed }, + view: { kind: "summary" }, + }, + }); + expect(rejected.isError).toBe(true); + }, +); + const expectFridaToolsAvailable = async (client: Client) => { const inventory = await client.callTool({ name: "binary_session", diff --git a/tests/process-global/analysisViewFileCancellation.test.ts b/tests/process-global/analysisViewFileCancellation.test.ts new file mode 100644 index 0000000000..1a5f9a8f52 --- /dev/null +++ b/tests/process-global/analysisViewFileCancellation.test.ts @@ -0,0 +1,49 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect, it, vi } from "vitest"; +import { + analysisViewCaptureEvidence, + environmentManifestCaptureFixture, +} from "../fixtures/analysisView.js"; +import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js"; +import { readEvidenceFileSource } from "../../src/application/analysisView/evidenceFileSource.js"; +import { inspectAnalysisViewWithFiles } from "../../src/application/analysisView/AnalysisViewService.js"; + +// This module mock needs an isolated graph even if another service imports the workflow first. +vi.mock("../../src/application/analysisView/evidenceFileSource.js", { + spy: true, +}); + +it("prefers cancellation observed after failed Evidence-file authentication", async () => { + const controller = new AbortController(); + const parent = analysisViewCaptureEvidence( + environmentManifestCaptureFixture(), + ); + const root = await createTestTempDirectory("rea-analysis-view-cancel-"); + const path = join(root, "parent.json"); + await writeFile( + path, + JSON.stringify({ ...parent, evidence_id: `ev_${"0".repeat(64)}` }), + ); + const { readEvidenceFileSource: read } = await vi.importActual< + typeof import("../../src/application/analysisView/evidenceFileSource.js") + >("../../src/application/analysisView/evidenceFileSource.js"); + // Abort at the completed read boundary, before its authentication failure is delivered. + vi.mocked(readEvidenceFileSource).mockImplementationOnce(async (...args) => { + const loaded = await read(...args); + controller.abort(); + return loaded; + }); + expect( + await inspectAnalysisViewWithFiles( + { + source: { kind: "evidence-file", path }, + view: { kind: "environment-manifest" }, + }, + undefined, + { signal: controller.signal }, + ), + ).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" } }); + expect(readEvidenceFileSource).toHaveBeenCalledTimes(1); + expect(controller.signal.aborted).toBe(true); +});