diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md index 87decc26d5..3e1cac44c3 100644 --- a/.agents/skills/reverse-engineer-anything/SKILL.md +++ b/.agents/skills/reverse-engineer-anything/SKILL.md @@ -67,6 +67,13 @@ explicit path or endpoint and do not need it. [historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream profile and credential exclusions. - User-owned Electron runtime already open: `list_electron_targets`. +- Recorded packet captures: `inspect_packet_capture` with selected TShark fields, + display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see + [capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md). + Packet limits count input packets before filtering. Output defaults to 50 matching + frames; use `match_offset` / `match_limit` for later pages, or explicit null + limit for complete matching output. Preserve the source digest across pages. + This does not perform live capture. - Explicit local WebAssembly artifact: `inspect_wasm_artifact`. Supply WABT 1.0.42 through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT. Select JavaScript glue and candidate paths explicitly and preserve URL/path diff --git a/.github/workflows/real-packet-capture.yml b/.github/workflows/real-packet-capture.yml new file mode 100644 index 0000000000..8faf9ba99b --- /dev/null +++ b/.github/workflows/real-packet-capture.yml @@ -0,0 +1,50 @@ +name: Real packet capture +on: + workflow_dispatch: + pull_request: + paths: + - "tests/boundary/captures/**" + - "tests/boundary/cli/captureCleanup.test.ts" + - "tests/boundary/mcp/packetCaptureMcp.test.ts" + - "src/captures/**" + - "src/domain/captures/**" + - "src/application/captures/**" + - "src/contracts/captures/**" + - "src/composition/captures.ts" + - "src/cli/captureCommands.ts" + - "src/server/registerCaptureTools.ts" + - "src/server/createServer.ts" + - "src/server/sessionAvailabilityPolicy.ts" + - "src/application/CapabilityInventory.ts" + - "src/contracts/toolContracts.ts" + - "src/contracts/toolEffects.ts" + - "src/cli.ts" + - "src/cliCommandNames.ts" + - "src/process/**" + - "src/artifacts/readStableArtifact.ts" + - "scripts/verify-packet-capture.mjs" + - ".github/workflows/real-packet-capture.yml" + - "package*.json" + - ".nvmrc" +permissions: + contents: read +concurrency: + group: real-packet-capture-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true +jobs: + tshark: + runs-on: ubuntu-24.04 + timeout-minutes: 15 + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version-file: .nvmrc + cache: npm + - run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3 + - run: npm ci + - run: npm run build:cached + - run: npm run test:focused -- src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts + - run: node scripts/verify-packet-capture.mjs + env: + REA_TSHARK_COMMAND: /usr/bin/tshark diff --git a/README.md b/README.md index 64475070d9..2c30aa8efd 100644 --- a/README.md +++ b/README.md @@ -169,6 +169,7 @@ Additional tools and host support depend on the target: | Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) | | Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) | | Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) | +| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) | | Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) | | Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) | diff --git a/README_ar.md b/README_ar.md index 6b050084f1..51fc7ba6ed 100644 --- a/README_ar.md +++ b/README_ar.md @@ -146,6 +146,7 @@ npx rea-agents@latest setup | موارد Android | البيان المفكوك الترميز وجداول السلاسل واللغات وحقائق الإصدار | مُشغّل Apktool يوفّره المُستدعي؛ [دليل الموارد](docs/apktool-resource-analysis.md) | | تطبيقات Flutter | بصمة لقطة Dart وهشوش المكتبات لكل ABI وأصل بناء المحرك | لا شيء (تحليل APK ساكن)؛ [دليل Flutter](docs/flutter-analysis.md) | | البرامج الثابتة | المناطق، ونتائج الاستخراج، وما يُمرّر إلى تحليل الشيفرة الأصلية | Binwalk / Unblob على Linux؛ [دليل البرامج الثابتة](docs/firmware-analysis.md) | +| الحزم المسجلة | حقول مختارة وقيم متكررة ومرشحات وقواعد فك الترميز | Linux: TShark 4.x + prlimit; [دليل التقاط الحزم](docs/packet-captures.md) | | الحزم والموارد | قوائم الملفات، والبصمات، وملفات plist، وبنية حزم Apple، والموارد المستخرجة | [دليل تحليل الملفات وJavaScript](docs/javascript-artifact-reconstruction.md)، [تطبيقات Apple](docs/apple-application-analysis.md) | | سلوك العمليات | مخرجات الطرفية، والتفاعلات، وملاحظات الخروج ونظام الملفات، ومقارنات التشغيل | Linux/macOS مع PTY أصلي؛ [تسجيل سلوك العمليات](docs/process-capture.md) | diff --git a/README_de.md b/README_de.md index d10f5f4b7b..2917a79847 100644 --- a/README_de.md +++ b/README_de.md @@ -146,6 +146,7 @@ REA benötigt Node.js 22.x (>=22.19), 24.x (>=24.11) oder 26+ sowie npm. Zusätz | Android-Ressourcen | Dekodiertes Manifest, String-Tabellen, Sprachen und Versionsfakten | Vom Aufrufer bereitgestellter Apktool-Starter; [Ressourcen-Leitfaden](docs/apktool-resource-analysis.md) | | Flutter-Apps | Dart-Snapshot-Hash, Per-ABI-Bibliotheksprüfsummen und Engine-Build-Provenienz | Keine (statische APK-Analyse); [Flutter-Leitfaden](docs/flutter-analysis.md) | | Firmware | Bereiche, Extraktionsergebnisse und Übergaben an die native Analyse | Binwalk / Unblob unter Linux; [Firmware-Leitfaden](docs/firmware-analysis.md) | +| Aufgezeichnete Pakete | Ausgewählte Felder, wiederholte Werte, Filter und Decode-as-Regeln | Linux: TShark 4.x + prlimit; [Mitschnitt-Anleitung](docs/packet-captures.md) | | Pakete und Ressourcen | Dateiinventare, Hashwerte, plists, Aufbau von Apple-Bundles und extrahierte Ressourcen | [Artefakt- und JavaScript-Leitfaden](docs/javascript-artifact-reconstruction.md), [Apple-Anwendungen](docs/apple-application-analysis.md) | | Prozessverhalten | Terminalausgabe, Interaktionen, Beobachtungen zu Prozessende und Dateisystem sowie Laufvergleiche | Linux/macOS mit nativer PTY; [Prozessaufzeichnung](docs/process-capture.md) | diff --git a/README_es.md b/README_es.md index 883cef8638..69823b0bfe 100644 --- a/README_es.md +++ b/README_es.md @@ -146,6 +146,7 @@ REA requiere Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, además de npm. Las h | Recursos de Android | Manifiesto decodificado, tablas de cadenas, idiomas y datos de versión | Lanzador de Apktool suministrado por el autor de la llamada; [guía de recursos](docs/apktool-resource-analysis.md) | | Aplicaciones Flutter | Hash de snapshot de Dart, resúmenes de bibliotecas por ABI y procedencia de compilación del motor | Ninguna (análisis APK estático); [guía de Flutter](docs/flutter-analysis.md) | | Firmware | Regiones, resultados de extracción y derivaciones al análisis nativo | Binwalk / Unblob en Linux; [guía de firmware](docs/firmware-analysis.md) | +| Paquetes grabados | Campos seleccionados, valores repetidos, filtros y reglas de decodificación | Linux: TShark 4.x + prlimit; [guía de capturas](docs/packet-captures.md) | | Paquetes y recursos | Inventarios de archivos, resúmenes criptográficos, plists, estructura de paquetes de Apple y recursos extraídos | [Guía de artefactos y JavaScript](docs/javascript-artifact-reconstruction.md), [aplicaciones de Apple](docs/apple-application-analysis.md) | | Comportamiento de procesos | Salida de la terminal, interacciones, observaciones de salida y del sistema de archivos, y comparaciones entre ejecuciones | Linux/macOS con una PTY nativa; [captura de procesos](docs/process-capture.md) | diff --git a/README_fa.md b/README_fa.md index 21ab30faeb..d8be73583e 100644 --- a/README_fa.md +++ b/README_fa.md @@ -138,6 +138,7 @@ REA به Node.js 22.x (>=22.19)، 24.x (>=24.11) یا 26+، به‌همراه np | منابع اندروید | مانیفست رمزگشایی‌شده، جداول رشته‌ها، زبان‌ها و اطلاعات نسخه | راه‌انداز Apktool تأمین‌شده توسط فراخوان‌کننده؛ [راهنمای منابع](docs/apktool-resource-analysis.md) | | برنامه‌های فلاتر | هش اسنپ‌شات دارت، هش کتابخانه‌ها به تفکیک ABI و اصالت بیلد موتور | هیچ (تحلیل ایستا APK)؛ [راهنمای فلاتر](docs/flutter-analysis.md) | | Firmware | ناحیه‌ها، نتایج استخراج و تحویل به ابزارهای تحلیل بومی | Binwalk / Unblob در لینوکس؛ [راهنمای Firmware](docs/firmware-analysis.md) | +| بسته‌های ضبط‌شده | فیلدهای منتخب، مقادیر تکراری، فیلترها و قواعد رمزگشایی | Linux: TShark 4.x + prlimit; [راهنمای ضبط بسته](docs/packet-captures.md) | | بسته‌ها و منابع | فهرست فایل‌ها، digestها، فایل‌های plist، ساختار bundle اپل و منابع استخراج‌شده | [راهنمای آرتیفکت و JavaScript](docs/javascript-artifact-reconstruction.md)، [برنامه‌های اپل](docs/apple-application-analysis.md) | | رفتار فرایند | خروجی ترمینال، تعاملات، وضعیت خروج، مشاهدات سیستم فایل و مقایسهٔ اجراها | Linux/macOS با PTY بومی؛ [ضبط فرایند](docs/process-capture.md) | diff --git a/README_fr.md b/README_fr.md index 5282fdc727..40510a8ac7 100644 --- a/README_fr.md +++ b/README_fr.md @@ -150,6 +150,7 @@ REA nécessite Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, ainsi que npm. Les | Ressources Android | Manifeste décodé, tables de chaînes, langues et faits de version | Lanceur Apktool fourni par l'appelant ; [guide des ressources](docs/apktool-resource-analysis.md) | | Applications Flutter | Empreinte de snapshot Dart, condensés de bibliothèques par ABI et provenance de build du moteur | Aucune (analyse APK statique) ; [guide Flutter](docs/flutter-analysis.md) | | Firmware | Régions, résultats d'extraction et passage de relais vers l'analyse native | Binwalk / Unblob sous Linux ; [guide firmware](docs/firmware-analysis.md) | +| Paquets enregistrés | Champs sélectionnés, valeurs répétées, filtres et règles de décodage | Linux: TShark 4.x + prlimit; [guide des captures](docs/packet-captures.md) | | Paquets et ressources | Inventaires de fichiers, empreintes, plists, anatomie des bundles Apple et ressources extraites | [Guide artefacts et JavaScript](docs/javascript-artifact-reconstruction.md), [applications Apple](docs/apple-application-analysis.md) | | Comportement des processus | Sortie du terminal, interactions, code de sortie, observations du système de fichiers et comparaisons | Linux/macOS avec un PTY natif ; [capture de processus](docs/process-capture.md) | diff --git a/README_id.md b/README_id.md index 7dc588fa0f..ce4461ea60 100644 --- a/README_id.md +++ b/README_id.md @@ -153,6 +153,7 @@ Alat tambahan dan dukungan sistem host bergantung pada target yang dianalisis: | Resource Android | Manifest terdekode, tabel string, lokal, dan fakta versi | Peluncur Apktool buatan pemanggil; [panduan resource](docs/apktool-resource-analysis.md) | | Aplikasi Flutter | Hash snapshot Dart, hash pustaka per ABI, dan asal build engine | Tidak ada (parsing APK statis); [panduan Flutter](docs/flutter-analysis.md) | | Firmware | Wilayah, hasil ekstraksi, dan pengalihan ke analisis native | Binwalk / Unblob di Linux; [panduan firmware](docs/firmware-analysis.md) | +| Paket rekaman | Kolom terpilih, nilai berulang, filter, dan aturan decode-as | Linux: TShark 4.x + prlimit; [panduan tangkapan](docs/packet-captures.md) | | Paket dan sumber daya | Inventaris berkas, digest, plist, struktur paket Apple, dan sumber daya yang diekstrak | [Panduan artefak dan JavaScript](docs/javascript-artifact-reconstruction.md), [aplikasi Apple](docs/apple-application-analysis.md) | | Perilaku proses | Keluaran terminal, interaksi, pengamatan proses keluar dan sistem berkas, serta perbandingan eksekusi | Linux/macOS dengan PTY native; [perekaman proses](docs/process-capture.md) | diff --git a/README_it.md b/README_it.md index fe9a6eb764..2b822f811a 100644 --- a/README_it.md +++ b/README_it.md @@ -150,6 +150,7 @@ REA richiede Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, oltre a npm. Strument | Risorse Android | Manifest decodificato, tabelle stringhe, lingue e dati di versione | Launcher Apktool fornito dal chiamante; [guida alle risorse](docs/apktool-resource-analysis.md) | | App Flutter | Hash dello snapshot Dart, digest delle librerie per ABI e provenienza della build del motore | Nessuno (analisi APK statica); [guida Flutter](docs/flutter-analysis.md) | | Firmware | Regioni, risultati di estrazione e passaggi all'analisi nativa | Binwalk / Unblob su Linux; [guida firmware](docs/firmware-analysis.md) | +| Pacchetti registrati | Campi selezionati, valori ripetuti, filtri e regole di decodifica | Linux: TShark 4.x + prlimit; [guida alle catture](docs/packet-captures.md) | | Pacchetti e risorse | Inventari file, digest, plist, anatomia dei bundle Apple e risorse estratte | [Guida artifact e JavaScript](docs/javascript-artifact-reconstruction.md), [Applicazioni Apple](docs/apple-application-analysis.md) | | Comportamento dei processi | Output del terminale, interazioni, osservazioni su uscita e filesystem ed confronti di esecuzione | Linux/macOS con un PTY nativo; [cattura dei processi](docs/process-capture.md) | diff --git a/README_ja.md b/README_ja.md index 607315297a..ec34d0ceeb 100644 --- a/README_ja.md +++ b/README_ja.md @@ -146,6 +146,7 @@ REA には Node.js 22.x（>=22.19）、24.x（>=24.11）、または 26+ と npm | Android リソース | デコードされたマニフェスト、文字列テーブル、ロケール、バージョン情報 | 呼び出し側が用意した Apktool ランチャー；[リソースガイド](docs/apktool-resource-analysis.md) | | Flutter アプリ | Dart スナップショットハッシュ、ABI ごとのライブラリダイジェスト、エンジンビルドの来歴 | 不要（静的 APK 解析）；[Flutter ガイド](docs/flutter-analysis.md) | | ファームウェア | 領域、抽出結果、ネイティブ分析への引き継ぎ | Linux 上の Binwalk / Unblob；[ファームウェアガイド](docs/firmware-analysis.md) | +| 記録済みパケット | 選択フィールド、繰り返し値、フィルター、デコード指定 | Linux: TShark 4.x + prlimit; [キャプチャガイド](docs/packet-captures.md) | | パッケージとリソース | ファイル一覧、ダイジェスト、plist、Apple バンドルの構造、抽出されたリソース | [アーティファクトと JavaScript のガイド](docs/javascript-artifact-reconstruction.md)、[Apple アプリケーション](docs/apple-application-analysis.md) | | プロセスの動作 | ターミナル出力、対話、終了とファイルシステムの観測結果、実行の比較 | ネイティブ PTY を備えた Linux/macOS；[プロセスキャプチャ](docs/process-capture.md) | diff --git a/README_ko.md b/README_ko.md index d2df0703c7..e8d9bede31 100644 --- a/README_ko.md +++ b/README_ko.md @@ -146,6 +146,7 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니 | Android 리소스 | 디코딩된 매니페스트, 문자열 테이블, 로캘, 버전 정보 | 호출자 제공 Apktool 런처; [리소스 가이드](docs/apktool-resource-analysis.md) | | Flutter 앱 | Dart 스냅샷 해시, ABI별 라이브러리 해시, 엔진 빌드 출처 | 없음 (정적 APK 파싱); [Flutter 가이드](docs/flutter-analysis.md) | | 펌웨어 | 영역, 추출 결과, 네이티브 분석으로 전달할 정보 | Linux의 Binwalk / Unblob; [펌웨어 가이드](docs/firmware-analysis.md) | +| 기록된 패킷 | 선택 필드, 반복 값, 필터 및 디코딩 규칙 | Linux: TShark 4.x + prlimit; [캡처 가이드](docs/packet-captures.md) | | 패키지 및 리소스 | 파일 목록, 다이제스트, plist, Apple 번들 구조, 추출한 리소스 | [아티팩트 및 JavaScript 가이드](docs/javascript-artifact-reconstruction.md), [Apple 애플리케이션](docs/apple-application-analysis.md) | | 프로세스 동작 | 터미널 출력, 상호작용, 종료 및 파일 시스템 관찰 결과, 실행 비교 | 네이티브 PTY를 지원하는 Linux/macOS; [프로세스 캡처](docs/process-capture.md) | diff --git a/README_pl.md b/README_pl.md index 30ea43edaa..efd03514da 100644 --- a/README_pl.md +++ b/README_pl.md @@ -146,6 +146,7 @@ REA wymaga Node.js 22.x (>=22.19), 24.x (>=24.11) lub 26+ oraz npm. Dodatkowe na | Zasoby Androida | Zdekodowany manifest, tablice ciągów, języki i informacje o wersji | Należący do wywołującego launcher Apktool; [przewodnik zasobów](docs/apktool-resource-analysis.md) | | Aplikacje Flutter | Hash migawki Dart, sumy bibliotek per ABI i pochodzenie kompilacji silnika | Brak (statyczna analiza APK); [przewodnik Flutter](docs/flutter-analysis.md) | | Oprogramowanie układowe | Regiony, wyniki wyodrębniania i przekazanie do analizy kodu natywnego | Binwalk / Unblob w systemie Linux; [przewodnik oprogramowania układowego](docs/firmware-analysis.md) | +| Zapisane pakiety | Wybrane pola, powtarzane wartości, filtry i reguły dekodowania | Linux: TShark 4.x + prlimit; [przewodnik po przechwyconych pakietach](docs/packet-captures.md) | | Pakiety i zasoby | Spisy plików, skróty, pliki plist, struktura pakietów Apple i wyodrębnione zasoby | [Przewodnik artefaktów i JavaScript](docs/javascript-artifact-reconstruction.md), [aplikacje Apple](docs/apple-application-analysis.md) | | Zachowanie procesów | Wyjście terminala, interakcje, obserwacje zakończenia i systemu plików oraz porównania uruchomień | Linux/macOS z natywnym PTY; [przechwytywanie procesów](docs/process-capture.md) | diff --git a/README_pt-BR.md b/README_pt-BR.md index a84f6e0db6..5e86b851e3 100644 --- a/README_pt-BR.md +++ b/README_pt-BR.md @@ -146,6 +146,7 @@ O REA requer Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, além do npm. As fer | Recursos do Android | Manifesto decodificado, tabelas de strings, idiomas e fatos de versão | Lançador do Apktool fornecido pelo chamador; [guia de recursos](docs/apktool-resource-analysis.md) | | Aplicativos Flutter | Hash de snapshot Dart, hashes de bibliotecas por ABI e procedência de build do motor | Nenhuma (análise estática de APK); [guia Flutter](docs/flutter-analysis.md) | | Firmware | Regiões, resultados de extração e encaminhamentos à análise nativa | Binwalk / Unblob no Linux; [guia de firmware](docs/firmware-analysis.md) | +| Pacotes gravados | Campos selecionados, valores repetidos, filtros e regras de decodificação | Linux: TShark 4.x + prlimit; [guia de capturas](docs/packet-captures.md) | | Pacotes e recursos | Inventários de arquivos, digests, plists, estrutura de bundles Apple e recursos extraídos | [Guia de artefatos e JavaScript](docs/javascript-artifact-reconstruction.md), [aplicativos Apple](docs/apple-application-analysis.md) | | Comportamento de processos | Saída do terminal, interações, observações de encerramento e do sistema de arquivos, e comparações entre execuções | Linux/macOS com PTY nativo; [captura de processos](docs/process-capture.md) | diff --git a/README_ru.md b/README_ru.md index 914ef09da3..77731f7f0f 100644 --- a/README_ru.md +++ b/README_ru.md @@ -146,6 +146,7 @@ REA требует Node.js 22.x (>=22.19), 24.x (>=24.11) или 26+, а так | Ресурсы Android | Декодированный манифест, таблицы строк, языки и сведения о версии | Запуск Apktool, предоставленный вызывающей стороной; [руководство по ресурсам](docs/apktool-resource-analysis.md) | | Приложения Flutter | Хеш снапшота Dart, дайджесты библиотек по ABI и происхождение сборки движка | Не требуется (статический разбор APK); [руководство по Flutter](docs/flutter-analysis.md) | | Прошивки | Области, результаты извлечения и передача в нативный анализ | Binwalk / Unblob в Linux; [руководство по прошивкам](docs/firmware-analysis.md) | +| Записанные пакеты | Выбранные поля, повторяющиеся значения, фильтры и правила декодирования | Linux: TShark 4.x + prlimit; [руководство по захватам](docs/packet-captures.md) | | Пакеты и ресурсы | Списки файлов, хеши, plist, структура пакетов Apple и извлечённые ресурсы | [Руководство по артефактам и JavaScript](docs/javascript-artifact-reconstruction.md), [приложения Apple](docs/apple-application-analysis.md) | | Поведение процессов | Вывод терминала, взаимодействия, наблюдения за завершением и файловой системой, сравнения запусков | Linux/macOS с нативным PTY; [захват процессов](docs/process-capture.md) | diff --git a/README_th.md b/README_th.md index 2abe5a0782..31ca23a5aa 100644 --- a/README_th.md +++ b/README_th.md @@ -146,6 +146,7 @@ REA ต้องใช้ Node.js 22.x (>=22.19), 24.x (>=24.11) หรือ 26 | ทรัพยากร Android | ไฟล์ manifest ที่ถอดรหัสแล้ว ตารางสตริง ภาษา และข้อมูลเวอร์ชัน | ตัวเรียก Apktool ที่ผู้เรียกจัดเตรียม; [คู่มือทรัพยากร](docs/apktool-resource-analysis.md) | | แอป Flutter | แฮชสแนปชอต Dart, แฮชไลบรารีราย ABI และที่มาของบิลด์เอนจิน | ไม่ต้อง (วิเคราะห์ APK แบบ static); [คู่มือ Flutter](docs/flutter-analysis.md) | | เฟิร์มแวร์ | พื้นที่ข้อมูล ผลการแตกไฟล์ และการส่งต่อไปวิเคราะห์เนทีฟ | Binwalk / Unblob บน Linux; [คู่มือเฟิร์มแวร์](docs/firmware-analysis.md) | +| แพ็กเก็ตที่บันทึกไว้ | ฟิลด์ที่เลือก ค่าซ้ำ ตัวกรอง และกฎการถอดรหัส | Linux: TShark 4.x + prlimit; [คู่มือไฟล์แคปเจอร์](docs/packet-captures.md) | | แพ็กเกจและทรัพยากร | รายการไฟล์ ค่าแฮช plist โครงสร้างบันเดิลของ Apple และทรัพยากรที่แตกออกมา | [คู่มืออาร์ติแฟกต์และ JavaScript](docs/javascript-artifact-reconstruction.md), [แอปพลิเคชัน Apple](docs/apple-application-analysis.md) | | พฤติกรรมของโปรเซส | เอาต์พุตเทอร์มินัล การโต้ตอบ ข้อมูลการสิ้นสุดและระบบไฟล์ที่สังเกตได้ และการเปรียบเทียบการรัน | Linux/macOS ที่มี PTY เนทีฟ; [การบันทึกโปรเซส](docs/process-capture.md) | diff --git a/README_tr.md b/README_tr.md index 28caee2602..5783286bc5 100644 --- a/README_tr.md +++ b/README_tr.md @@ -146,6 +146,7 @@ REA; Node.js 22.x (>=22.19), 24.x (>=24.11) veya 26+ ve npm gerektirir. Ek araç | Android kaynakları | Çözülmüş manifest, dize tabloları, diller ve sürüm bilgileri | Çağıranın sağladığı Apktool başlatıcısı; [kaynak rehberi](docs/apktool-resource-analysis.md) | | Flutter uygulamaları | Dart anlık görüntü karması, ABI başına kitaplık özetleri ve motor derleme kökeni | Yok (statik APK çözümleme); [Flutter rehberi](docs/flutter-analysis.md) | | Donanım yazılımı | Bölgeler, çıkarma sonuçları ve yerel kod analizine devirler | Linux üzerinde Binwalk / Unblob; [donanım yazılımı rehberi](docs/firmware-analysis.md) | +| Kaydedilmiş paketler | Seçilen alanlar, yinelenen değerler, filtreler ve çözümleme kuralları | Linux: TShark 4.x + prlimit; [yakalama kılavuzu](docs/packet-captures.md) | | Paketler ve kaynaklar | Dosya envanterleri, özet değerleri, plist'ler, Apple paket yapısı ve çıkarılmış kaynaklar | [Artefakt ve JavaScript rehberi](docs/javascript-artifact-reconstruction.md), [Apple uygulamaları](docs/apple-application-analysis.md) | | Süreç davranışı | Terminal çıktısı, etkileşimler, sonlanma ve dosya sistemi gözlemleri, çalıştırma karşılaştırmaları | Yerel PTY ile Linux/macOS; [süreç yakalama](docs/process-capture.md) | diff --git a/README_uk.md b/README_uk.md index a97624ec8d..d64c87c0c5 100644 --- a/README_uk.md +++ b/README_uk.md @@ -146,6 +146,7 @@ REA потребує Node.js 22.x (>=22.19), 24.x (>=24.11) або 26+, а та | Ресурси Android | Декодований маніфест, таблиці рядків, мови та відомості про версію | Засіб запуску Apktool, наданий викликаючою стороною; [підручник ресурсів](docs/apktool-resource-analysis.md) | | Додатки Flutter | Хеш знімка Dart, дайджести бібліотек за ABI та походження збірки рушія | Не потрібні (статичний розбір APK); [підручник Flutter](docs/flutter-analysis.md) | | Прошивки | Області, результати видобування й передавання до нативного аналізу | Binwalk / Unblob у Linux; [посібник прошивок](docs/firmware-analysis.md) | +| Записані пакети | Вибрані поля, повторювані значення, фільтри та правила декодування | Linux: TShark 4.x + prlimit; [посібник із записів пакетів](docs/packet-captures.md) | | Пакети й ресурси | Переліки файлів, хеші, plist, структура пакетів Apple та видобуті ресурси | [Посібник артефактів і JavaScript](docs/javascript-artifact-reconstruction.md), [застосунки Apple](docs/apple-application-analysis.md) | | Поведінка процесів | Вивід термінала, взаємодії, спостереження за завершенням і файловою системою та порівняння запусків | Linux/macOS із нативним PTY; [захоплення процесів](docs/process-capture.md) | diff --git a/README_vi.md b/README_vi.md index be4eac7c25..a98cff4dea 100644 --- a/README_vi.md +++ b/README_vi.md @@ -146,6 +146,7 @@ REA yêu cầu Node.js 22.x (>=22.19), 24.x (>=24.11) hoặc 26+, cùng với np | Tài nguyên Android | Manifest đã giải mã, bảng chuỗi, ngôn ngữ và thông tin phiên bản | Trình khởi chạy Apktool do nơi gọi cung cấp; [hướng dẫn tài nguyên](docs/apktool-resource-analysis.md) | | Ứng dụng Flutter | Băm snapshot Dart, hash thư viện theo ABI và nguồn gốc build engine | Không cần (phân tích APK tĩnh); [hướng dẫn Flutter](docs/flutter-analysis.md) | | Phần sụn | Các vùng, kết quả trích xuất và dữ liệu chuyển sang phân tích mã máy | Binwalk / Unblob trên Linux; [hướng dẫn phần sụn](docs/firmware-analysis.md) | +| Gói tin đã ghi | Trường được chọn, giá trị lặp, bộ lọc và quy tắc giải mã | Linux: TShark 4.x + prlimit; [hướng dẫn bản ghi gói tin](docs/packet-captures.md) | | Gói và tài nguyên | Danh sách tệp, giá trị băm, plist, cấu trúc bundle Apple và tài nguyên được trích xuất | [Hướng dẫn hiện vật phần mềm và JavaScript](docs/javascript-artifact-reconstruction.md), [ứng dụng Apple](docs/apple-application-analysis.md) | | Hành vi tiến trình | Đầu ra terminal, tương tác, quan sát khi thoát và hệ thống tệp, cùng so sánh các lần chạy | Linux/macOS có PTY gốc; [ghi lại tiến trình](docs/process-capture.md) | diff --git a/README_zh-TW.md b/README_zh-TW.md index 841e4d4795..6591c40fa0 100644 --- a/README_zh-TW.md +++ b/README_zh-TW.md @@ -146,6 +146,7 @@ REA 需要 Node.js 22.x（>=22.19）、24.x（>=24.11）或 26+，以及 npm。 | Android 資源 | 解碼後的清單、字串表、語系和版本資訊 | 呼叫端提供的 Apktool 啟動器；[資源指南](docs/apktool-resource-analysis.md) | | Flutter 應用 | Dart 快照雜湊、依 ABI 的程式庫雜湊及引擎建置來源 | 無（靜態 APK 解析）；[Flutter 指南](docs/flutter-analysis.md) | | 韌體 | 區域、提取結果和轉交原生分析的內容 | Linux 上的 Binwalk / Unblob；[韌體指南](docs/firmware-analysis.md) | +| 封包記錄 | 指定欄位、重複值、篩選條件與解碼規則 | Linux: TShark 4.x + prlimit; [封包分析指南](docs/packet-captures.md) | | 軟體包與資源 | 檔案清單、摘要、plist、Apple bundle 結構和提取的資源 | [製品與 JavaScript 指南](docs/javascript-artifact-reconstruction.md)、[Apple 應用](docs/apple-application-analysis.md) | | 處理程序行為 | 終端輸出、互動、退出和檔案系統觀察結果，以及執行對比 | 支援原生 PTY 的 Linux/macOS；[處理程序擷取](docs/process-capture.md) | diff --git a/README_zh.md b/README_zh.md index 3d079c9704..e12321d380 100644 --- a/README_zh.md +++ b/README_zh.md @@ -146,6 +146,7 @@ REA 需要 Node.js 22.x（>=22.19）、24.x（>=24.11）或 26+，以及 npm。 | Android 资源 | 解码后的清单、字符串表、语言区域和版本信息 | 调用方提供的 Apktool 启动器；[资源指南](docs/apktool-resource-analysis.md) | | Flutter 应用 | Dart 快照哈希、按 ABI 的库摘要及引擎构建溯源 | 无（静态 APK 解析）；[Flutter 指南](docs/flutter-analysis.md) | | 固件 | 区域、提取结果和转交原生分析的内容 | Linux 上的 Binwalk / Unblob；[固件指南](docs/firmware-analysis.md) | +| 抓包文件 | 指定字段、重复值、过滤条件与解码规则 | Linux: TShark 4.x + prlimit; [抓包分析指南](docs/packet-captures.md) | | 软件包与资源 | 文件清单、摘要、plist、Apple bundle 结构和提取的资源 | [制品与 JavaScript 指南](docs/javascript-artifact-reconstruction.md)、[Apple 应用](docs/apple-application-analysis.md) | | 进程行为 | 终端输出、交互、退出和文件系统观察结果，以及运行对比 | 支持原生 PTY 的 Linux/macOS；[进程捕获](docs/process-capture.md) | diff --git a/docs/.vitepress/config.ts b/docs/.vitepress/config.ts index 07ca2e72df..6031f2fc43 100644 --- a/docs/.vitepress/config.ts +++ b/docs/.vitepress/config.ts @@ -44,6 +44,7 @@ export default defineConfig({ { text: "Apple applications", link: "/apple-application-analysis" }, { text: "Firmware", link: "/firmware-analysis" }, { text: "SQLite databases", link: "/sqlite-databases" }, + { text: "Packet captures", link: "/packet-captures" }, ], }, { diff --git a/docs/packet-captures.md b/docs/packet-captures.md new file mode 100644 index 0000000000..d56a800370 --- /dev/null +++ b/docs/packet-captures.md @@ -0,0 +1,130 @@ +# Recorded packet capture inspection + +`inspect_packet_capture` answers which selected protocol fields TShark observes +in an existing capture. Network, USB or Bluetooth recordings can use the same +field/filter interface when the installed TShark supports their encapsulation +and dissectors. This tool does not acquire traffic, open an interface or control +a device. HAR/mitmproxy web-capture workflows remain separate. + +Supply an existing TShark 4.x installation **on Linux**, with util-linux +`prlimit` available at `/usr/bin/prlimit`. Override its absolute path with +`REA_TSHARK_PRLIMIT_COMMAND` if required. Unsupported hosts, including macOS +and Windows, are refused before inspection because this integration requires +an enforced address-space limit: + +```sh +export REA_TSHARK_COMMAND=/absolute/path/to/tshark +rea inspect-packet-capture /artifacts/device.pcapng \ + --fields ip.addr --fields udp.port --fields data.data \ + --display_filter 'udp' --format json +``` + +The corresponding MCP call is: + +```json +{ + "name": "inspect_packet_capture", + "arguments": { + "path": "/artifacts/device.pcapng", + "fields": ["ip.addr", "udp.port", "data.data"], + "display_filter": "udp" + } +} +``` + +`frame.number` is always included. Repeated values are ordered string arrays, +for example `"ip.addr": ["192.0.2.1", "192.0.2.2"]`. Missing fields remain +absent; they are not converted to zero or empty arrays. Export-only index/score +metadata is discarded. No payload bytes are automatically added; select a +supported byte field such as `data.data` or `@data` when needed. + +Selectors are limited before argv expansion: at most 1,024 fields and 1,024 +`decode_as` rules, 8,192 UTF-16 code units per selector, and 32 KiB total encoded +selector arguments (UTF-8 text, terminators, option names and argv pointers). +This leaves headroom below Linux's [execve argument/environment limits](https://man7.org/linux/man-pages/man2/execve.2.html); +an unusually large inherited environment can still prevent launching. + +Output is paged after filtering: `match_limit` defaults to 50 and `match_offset` +to 0. The result retains original frame numbers and reports `output_window` +with the matching count within the input read scope, returned count and +`has_more`. Request later pages with the offset, or explicit `match_limit: null` +(CLI `--all_matches`) for all remaining matches. Paging does not shorten +input reading or change two-pass reassembly. Producer output must still fit its +resource limit; use selected fields, a display filter or an input-packet limit +when dissection output is too large. Each call checks the recording's current +digest, so pages from different source digests must not be combined. + +Additional selections: + +- `display_filter`: Wireshark display-filter syntax; omitted means no filter. +- `decode_as`: explicit dissector overrides, such as `["udp.port==5555,dns"]`. + Repeat `--decode_as` in the CLI for multiple overrides. +- `packet_limit`: optional maximum **input packets read before filtering**. + A limit of 1 can therefore yield zero matching packets even when later frames + match. Omitted means read to EOF within resource limits. The report records + `to-eof` or `up-to-input-packet-limit`, never inferring total capture size from + the returned packet count. +- `two_pass`: defaults to true for fields that need later packets and reassembly + dependencies. MCP may set false; CLI uses `--no-two_pass` for a single pass. + +See the [TShark manual](https://www.wireshark.org/docs/man-pages/tshark.html) +for field names, display filters, decode-as syntax and pass semantics. Invalid +fields or filters return producer diagnostics. An empty result does not prove +absence of traffic outside the selected recording, filter or read range. +Dissector results are observations, not independently verified device behavior. + +Evidence includes source path/digest/size, exact selections, typed packet fields, +configured and resolved executable paths/digests/full version banner, arguments and stderr. Warnings remain +visible even on exit zero. `tool.version_diagnostics` also retains warnings from +the separate version probe. Post-run identity failures identify the affected +private copy, executable or limiter; resource-signal errors retain the signal +and effective ceilings. Two-pass mode cannot recover missing capture bytes. + +Each call uses a private read-only capture copy and an empty personal +configuration directory (`WIRESHARK_CONFIG_DIR`), with name resolution disabled. +The executable, resource limiter and private input copy are rechecked after execution. Owned +processes, file descriptors and private files are cleaned up; failed cleanup retains ownership and +completed observations for retry. The one-shot CLI awaits a cleanup retry while +preserving the original failure and completed Evidence. Completed observations +remain in the MCP session after cancellation or cleanup failure; bounded +post-run integrity checks finish before honoring a late abort. Source files +are never passed to TShark. +Installed libraries, global configuration and plugins remain trusted, are not +fully fingerprinted, and can influence dissection. This is not an OS sandbox. + +Input is limited to 32 MiB before reading, each command to 30 seconds, and +combined stdout/stderr to 4 MiB during collection. Output overflow fails instead +of returning truncated success. Before execution, `prlimit` enforces **1 GiB +address space per process**, **64 MiB per file**, **30 CPU seconds** and no core +dumps. Stricter inherited soft/hard limits are preserved; Evidence records the effective +ceilings. Both limits are set before execution, so dissector allocations and +compressed-capture expansion cannot exceed the per-process address-space cap. +These are not aggregate process-memory or filesystem quotas. The selected +limiter is trusted; its path, digest and arguments are retained in Evidence. +No arbitrary process flags are accepted. + +## Verification + +```sh +npm run build:cached +REA_TSHARK_COMMAND=/absolute/path/to/tshark node scripts/verify-packet-capture.mjs +``` + +The verifier generates its own Ethernet/IPv4/UDP/DNS and Bluetooth HCI PCAPs. +It exercises real compiled CLI/MCP parity, advertised schemas, repeated/missing +fields, filtering, decode-as, both pass modes, a packet limit before matching +frames, Evidence persistence, failure diagnostics, source identity and cleanup. +An optional positional REA entrypoint verifies an installed package. + +The real-tool lane requires Linux and TShark 4.x. A separate kernel-limit +regression checks the inherited soft/hard limits and verifies an oversized +allocation is refused before it can consume memory. The Ubuntu workflow uses +its distribution package; check its outcome before claiming that build verified. +The earlier macOS check does not establish the current bounded integration, +which deliberately refuses unsupported hosts. Other formats (including PCAPNG), +USB, BLE ATT and actual physical capture are not established by these fixtures. + +Each provider admits one inspection at a time, including acquisition and cleanup. +An occupied provider returns a retryable capacity error before acquiring another +file or worker. Closing the provider stops new admission and drains accepted work; +failed cleanup remains owned and can be retried. diff --git a/scripts/lib/catalog-builders.mjs b/scripts/lib/catalog-builders.mjs index 5cf27d4861..b32269563f 100644 --- a/scripts/lib/catalog-builders.mjs +++ b/scripts/lib/catalog-builders.mjs @@ -30,6 +30,7 @@ export const toolFamilyCatalog = (sources) => { ...sources.goContracts.GO_TOOL_CONTRACTS, ...sources.wasmContracts.WASM_TOOL_CONTRACTS, ...sources.sqliteContracts.SQLITE_TOOL_CONTRACTS, + ...sources.captureContracts.CAPTURE_TOOL_CONTRACTS, ], }, { @@ -196,6 +197,10 @@ export const providerCatalog = (sources) => { ({ name }) => name === "inspect_pe_resources", ), }, + { + identity: sources.captureProvider.TSHARK_PROVIDER_IDENTITY, + contracts: sources.captureContracts.CAPTURE_TOOL_CONTRACTS, + }, { identity: sources.wasmProvider.WABT_PROVIDER_IDENTITY, contracts: sources.wasmContracts.WASM_TOOL_CONTRACTS, diff --git a/scripts/lib/catalog-core.mjs b/scripts/lib/catalog-core.mjs index 14c8bf3f89..9771f4cfdc 100644 --- a/scripts/lib/catalog-core.mjs +++ b/scripts/lib/catalog-core.mjs @@ -47,6 +47,8 @@ const SOURCE_PATHS = { fridaProvider: "dist/frida/FridaInstrumentationManager.js", pwntoolsProvider: "dist/native/pwntools/PwntoolsRelease.js", nativeContracts: "dist/contracts/native/nativeToolContracts.js", + captureContracts: "dist/contracts/captures/captureToolContracts.js", + captureProvider: "dist/captures/TsharkCaptureProvider.js", wasmContracts: "dist/contracts/wasm/wasmToolContracts.js", wasmProvider: "dist/wasm/WabtRelease.js", evmContracts: "dist/contracts/evm/evmToolContracts.js", diff --git a/scripts/lib/module-boundaries.mjs b/scripts/lib/module-boundaries.mjs index d4acb4f00c..e50d22a9b9 100644 --- a/scripts/lib/module-boundaries.mjs +++ b/scripts/lib/module-boundaries.mjs @@ -9,6 +9,7 @@ const MIGRATED_PROVIDER_ROOTS = new Set([ "evm", "go", "wasm", + "captures", "android", "firmware", "ghidra", diff --git a/scripts/verify-packet-capture.mjs b/scripts/verify-packet-capture.mjs new file mode 100644 index 0000000000..371c72367a --- /dev/null +++ b/scripts/verify-packet-capture.mjs @@ -0,0 +1,294 @@ +#!/usr/bin/env node +import assert from "node:assert/strict"; +import { execFile } from "node:child_process"; +import { createHash } from "node:crypto"; +import { readFile, writeFile, readdir } from "node:fs/promises"; +import { isAbsolute, join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { promisify } from "node:util"; +import { Client } from "@modelcontextprotocol/client"; +import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; +import { Ajv2020 } from "ajv/dist/2020.js"; +import { PrivateRuntimeRoot } from "../dist/process/PrivateRuntimeRoot.js"; +import { parseEvidence } from "../dist/domain/evidence.js"; +import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs"; + +if (process.platform !== "linux") + throw new Error( + "Real packet capture verification requires Linux with util-linux prlimit.", + ); +if (!isAbsolute(process.env.REA_TSHARK_COMMAND ?? "")) + throw new Error("Supply absolute REA_TSHARK_COMMAND for TShark 4.x."); +const entrypoint = + process.argv[2] ?? fileURLToPath(new URL("./rea.mjs", import.meta.url)); +const run = createVerifierRun(); +const root = await PrivateRuntimeRoot.create({ + prefix: "rea-capture-verifier-", +}); +const path = join(root.path, "network with spaces.pcap"); +const bluetooth = join(root.path, "bluetooth.pcap"); +const execute = promisify(execFile); +const environment = { + ...process.env, + TMPDIR: root.path, + TMP: root.path, + TEMP: root.path, +}; +const client = new Client({ name: "packet-capture-verifier", version: "1" }); +const transport = new StdioClientTransport({ + command: process.execPath, + args: [entrypoint, "mcp"], + env: environment, + stderr: "pipe", +}); + +// Independently synthesize PCAP bytes; no external captures or device access. +const pcap = (linktype, packets) => { + const header = Buffer.alloc(24); + header.writeUInt32LE(0xa1b2c3d4, 0); + header.writeUInt16LE(2, 4); + header.writeUInt16LE(4, 6); + header.writeUInt32LE(65535, 16); + header.writeUInt32LE(linktype, 20); + return Buffer.concat([ + header, + ...packets.flatMap((bytes, i) => { + const record = Buffer.alloc(16); + record.writeUInt32LE(1700000000 + i, 0); + record.writeUInt32LE(bytes.length, 8); + record.writeUInt32LE(bytes.length, 12); + return [record, bytes]; + }), + ]); +}; +const udp = (destination, payload) => { + const ip = Buffer.from("450000000001000040110000c0000201c0000202", "hex"); + ip.writeUInt16BE(28 + payload.length, 2); + let checksum = 0; + for (let i = 0; i < ip.length; i += 2) checksum += ip.readUInt16BE(i); + while (checksum > 0xffff) checksum = (checksum & 0xffff) + (checksum >>> 16); + ip.writeUInt16BE(~checksum & 0xffff, 10); + const header = Buffer.alloc(8); + header.writeUInt16BE(40000, 0); + header.writeUInt16BE(destination, 2); + header.writeUInt16BE(8 + payload.length, 4); + return Buffer.concat([ + Buffer.from("0200000000020200000000010800", "hex"), + ip, + header, + payload, + ]); +}; +const dnsQuery = Buffer.from( + "123401000001000000000000076578616d706c6504746573740000010001", + "hex", +); +const source = pcap(1, [ + udp(5554, Buffer.from("REA")), + udp(5555, dnsQuery), + udp(5555, dnsQuery), +]); +// LINKTYPE_BLUETOOTH_HCI_H4 = 187; H4 command packet, HCI Reset opcode 0x0c03, no parameters. +const bluetoothSource = pcap(187, [Buffer.from("01030c00", "hex")]); +let cases = 0; +try { + await writeFile(path, source); + await writeFile(bluetooth, bluetoothSource); + await client.connect(transport); + const tool = (await client.listTools()).tools.find( + ({ name }) => name === "inspect_packet_capture", + ); + assert.ok(tool?.outputSchema); + const ajv = new Ajv2020({ strict: false, validateFormats: false }); + assert.ok(ajv.validateSchema(tool.inputSchema)); + assert.ok(ajv.validateSchema(tool.outputSchema)); + const inspect = async (input) => { + const response = await client.callTool({ + name: tool.name, + arguments: input, + }); + assert.notEqual(response.isError, true, JSON.stringify(response)); + assert.ok( + ajv.validate(tool.outputSchema, response.structuredContent), + JSON.stringify(ajv.errors), + ); + const evidence = parseEvidence(response.structuredContent); + const report = evidence.normalized_result; + assert.equal( + report.artifact.sha256, + createHash("sha256") + .update(await readFile(input.path)) + .digest("hex"), + ); + assert.equal(report.hardware_acquisition, "not-performed"); + assert.equal(report.limits.address_space_bytes, 1073741824); + assert.equal(report.tool.configured_path, process.env.REA_TSHARK_COMMAND); + assert.equal( + report.resource_limiter.configured_path, + process.env.REA_TSHARK_PRLIMIT_COMMAND ?? "/usr/bin/prlimit", + ); + assert.ok( + report.resource_limiter.arguments.includes("--as=1073741824:1073741824"), + ); + assert.equal(report.configuration, "private-empty-personal-profile"); + const flags = Object.entries(input) + .filter(([key]) => key !== "path") + .flatMap(([key, value]) => + key === "match_limit" && value === null + ? ["--all_matches"] + : typeof value === "boolean" + ? [value ? `--${key}` : `--no-${key}`] + : (Array.isArray(value) ? value : [value]).flatMap((item) => [ + `--${key}`, + String(item), + ]), + ); + const cli = JSON.parse( + ( + await execute( + process.execPath, + [ + entrypoint, + "inspect-packet-capture", + input.path, + ...flags, + "--format", + "json", + ], + { env: environment, timeout: 60000, maxBuffer: 16 * 1024 * 1024 }, + ) + ).stdout, + ); + assert.deepEqual(parseEvidence(cli), evidence); + const bundle = await client.callTool({ + name: "get_evidence_bundle", + arguments: {}, + }); + assert.ok(JSON.stringify(bundle).includes(evidence.evidence_id)); + cases++; + return report; + }; + for (const two_pass of [true, false]) { + const report = await inspect({ + path, + fields: ["ip.addr", "udp.port", "tcp.port"], + display_filter: "udp.dstport == 5555", + two_pass, + }); + assert.deepEqual( + report.packets.map((p) => p["frame.number"]), + [["2"], ["3"]], + ); + assert.deepEqual(report.packets[0]["ip.addr"], ["192.0.2.1", "192.0.2.2"]); + assert.deepEqual(report.packets[0]["udp.port"], ["40000", "5555"]); + assert.equal(Object.hasOwn(report.packets[0], "tcp.port"), false); + assert.equal(report.read_scope, "to-eof"); + } + const selected = { + path, + fields: ["dns.qry.name"], + decode_as: ["udp.port==5555,dns"], + display_filter: "dns", + }; + const dns = await inspect(selected); + assert.equal(dns.packets.length, 2); + assert.deepEqual(dns.packets[0]["dns.qry.name"], ["example.test"]); + const prefix = await inspect({ ...selected, packet_limit: 1 }); + assert.deepEqual(prefix.packets, []); + assert.equal(prefix.read_scope, "up-to-input-packet-limit"); + const empty = await inspect({ + path, + fields: ["frame.number"], + display_filter: "tcp", + }); + assert.deepEqual(empty.packets, []); + const pagedPath = join(root.path, "paged.pcap"); + const pagedSource = pcap( + 1, + Array.from({ length: 120 }, () => + udp(5555, Buffer.from("owned paging fixture")), + ), + ); + await writeFile(pagedPath, pagedSource); + const page = await inspect({ path: pagedPath, fields: ["ip.addr"] }); + assert.equal(page.packets.length, 50); + assert.deepEqual(page.output_window, { + matching_packets: 120, + returned_packets: 50, + has_more: true, + }); + const nextPage = await inspect({ + path: pagedPath, + fields: ["ip.addr"], + match_offset: 50, + match_limit: 40, + }); + assert.deepEqual(nextPage.packets[0]["frame.number"], ["51"]); + assert.deepEqual(nextPage.packets.at(-1)["frame.number"], ["90"]); + const complete = await inspect({ + path: pagedPath, + fields: ["ip.addr"], + match_limit: null, + }); + assert.equal(complete.packets.length, 120); + assert.equal(complete.output_window.has_more, false); + assert.deepEqual(await readFile(pagedPath), pagedSource); + const hci = await inspect({ path: bluetooth, fields: ["bthci_cmd.opcode"] }); + assert.deepEqual(hci.packets[0]["bthci_cmd.opcode"], ["0x0c03"]); + assert.deepEqual(await readFile(path), source); + assert.deepEqual(await readFile(bluetooth), bluetoothSource); + for (const input of [ + { path, fields: ["frame.number"], display_filter: "this is not a filter" }, + { path, fields: ["no_such_field"] }, + { + path, + fields: ["frame.number"], + decode_as: ["no_such_protocol==1,unknown"], + }, + { path: join(root.path, "absent.pcap"), fields: ["frame.number"] }, + { path, fields: ["frame.number"], interface: "lo0" }, + ]) { + const response = await client.callTool({ + name: tool.name, + arguments: input, + }); + assert.equal(response.isError, true); + await client.ping(); + cases++; + } + await writeFile(path, "not a capture"); + assert.equal( + ( + await client.callTool({ + name: tool.name, + arguments: { path, fields: ["frame.number"] }, + }) + ).isError, + true, + ); + cases++; + await client.close(); + assert.deepEqual((await readdir(root.path)).sort(), [ + "bluetooth.pcap", + "network with spaces.pcap", + "paged.pcap", + ]); +} finally { + await client.close(); + await transport.close(); + await root.close(); +} +console.log( + JSON.stringify( + { + status: "passed", + cases, + host: `${process.platform}/${process.arch}`, + scope: + "real TShark CLI/MCP; synthetic Ethernet/IPv4/UDP/DNS and Bluetooth HCI PCAP; no hardware acquisition", + ...(await completeVerifierRun(run)), + }, + null, + 2, + ), +); diff --git a/src/application/CapabilityInventory.ts b/src/application/CapabilityInventory.ts index 73ffaecf97..751dc57df6 100644 --- a/src/application/CapabilityInventory.ts +++ b/src/application/CapabilityInventory.ts @@ -43,6 +43,7 @@ type ToolAvailabilityReason = "available" | ToolUnavailabilityReason; type ProviderDescriptor = ProviderCapability; export type AvailabilityPolicy = { + readonly packetCaptureEnabled?: boolean; readonly wasmArtifactEnabled?: boolean; readonly evmInterfaceEnabled?: boolean; readonly processCaptureEnabled: boolean; @@ -395,6 +396,14 @@ const workflowAvailabilityFor = ({ remediation: "On Linux x64, set absolute REA_PWNTOOLS_PYTHON to caller-supplied Python with pwntools 4.15.0. No active binary target is required.", }; + if (name === "inspect_packet_capture") + return policy.packetCaptureEnabled === true + ? { reason: "available", remediation: null } + : { + reason: "provider_missing", + remediation: + "On Linux, supply TShark 4.x via absolute REA_TSHARK_COMMAND and util-linux prlimit (REA_TSHARK_PRLIMIT_COMMAND, default /usr/bin/prlimit). No active binary target or hardware is required; the tool is checked at execution.", + }; if (name === "inspect_wasm_artifact") return policy.wasmArtifactEnabled === true ? { reason: "available", remediation: null } diff --git a/src/application/captures/PacketCaptureService.ts b/src/application/captures/PacketCaptureService.ts new file mode 100644 index 0000000000..f2098b49cf --- /dev/null +++ b/src/application/captures/PacketCaptureService.ts @@ -0,0 +1,93 @@ +import { isAbsolute } from "node:path"; +import type { + AnalysisExecution, + ExecutionOptions, +} from "../AnalysisProvider.js"; +import type { AnalysisError } from "../../domain/analysisErrorBase.js"; +import { + AnalysisCancelledError, + AnalysisInputError, + AnalysisOutputError, +} from "../../domain/analysisErrorCore.js"; +import { createEvidence, type Evidence } from "../../domain/evidence.js"; +import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js"; +import { + inspectPacketCaptureInputSchema, + packetCaptureSchema, + type InspectPacketCaptureInput, +} from "../../domain/captures/packetCapture.js"; +import { err, ok, type Result } from "../../domain/result.js"; + +export interface PacketCapturePort { + /** Release per-call resources before returning, or retain failed cleanup for close/retry. */ + inspect( + input: InspectPacketCaptureInput, + options?: ExecutionOptions, + ): Promise>; + close(): Promise; +} + +/** One shared validation and Evidence owner for CLI and MCP. */ +export class PacketCaptureService { + constructor(readonly provider: PacketCapturePort) {} + close(): Promise { + return this.provider.close(); + } + async inspect( + raw: unknown, + options?: ExecutionOptions, + ): Promise> { + const operation = "inspect_packet_capture"; + if (options?.signal?.aborted) + return err(new AnalysisCancelledError(operation)); + const input = inspectPacketCaptureInputSchema.safeParse(raw); + if (!input.success) + return err( + analysisInputErrorFromIssues(operation, input.error.issues, raw, { + cause: input.error, + }), + ); + if (!isAbsolute(input.data.path)) + return err( + new AnalysisInputError(operation, undefined, [ + { + path: ["path"], + reason: "invalid_format", + message: "Expected an absolute local path.", + }, + ]), + ); + const decoded = await this.provider.inspect(input.data, options); + if (!decoded.ok) return decoded; + const execution = decoded.value; + const report = packetCaptureSchema.safeParse(execution.result); + if ( + !report.success || + report.data.artifact.path !== input.data.path || + execution.subject?.path !== input.data.path || + execution.subject.sha256 !== report.data.artifact.sha256 || + JSON.stringify(report.data.selection) !== JSON.stringify(input.data) + ) + return err( + new AnalysisOutputError( + operation, + "Capture analyzer returned malformed data or changed the selected artifact identity/options.", + ), + ); + const evidence = createEvidence(execution.subject, execution.provider, { + operation, + parameters: input.data, + result: report.data, + confidence: "observed", + locations: execution.locations, + limitations: execution.limitations, + }); + return options?.signal?.aborted + ? err( + new AnalysisCancelledError(operation, { + partialObservation: evidence, + }), + ) + : ok(evidence); + } +} diff --git a/src/captures/TsharkCaptureProvider.ts b/src/captures/TsharkCaptureProvider.ts new file mode 100644 index 0000000000..77aae5a6e2 --- /dev/null +++ b/src/captures/TsharkCaptureProvider.ts @@ -0,0 +1,650 @@ +import { + readLinuxResourceLimits, + type LinuxResourceLimits, +} from "../process/LinuxResourceLimits.js"; +import { parsePacketOutput } from "./parsePacketOutput.js"; +import { randomUUID } from "node:crypto"; +import { constants } from "node:fs"; +import { access, mkdir, realpath, writeFile } from "node:fs/promises"; +import { isAbsolute, join } from "node:path"; +import { + createAnalysisExecution, + type AnalysisExecution, + type ExecutionOptions, +} from "../application/AnalysisProvider.js"; +import type { PacketCapturePort } from "../application/captures/PacketCaptureService.js"; +import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js"; +import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; +import { readStableArtifact } from "../artifacts/readStableArtifact.js"; +import { + AnalysisError, + type AnalysisCapturedOutput, +} from "../domain/analysisErrorBase.js"; +import { + AnalysisAccessDeniedError, + AnalysisArtifactChangedError, + AnalysisCancelledError, + AnalysisCapabilityUnavailableError, + AnalysisInputError, + AnalysisOutputError, + AnalysisResourceConstraintError, + AnalysisTimeoutError, +} from "../domain/analysisErrorCore.js"; +import { createEvidence } from "../domain/evidence.js"; +import { ProviderAdapterError } from "../domain/providerAdapterError.js"; +import { ProviderCleanupError } from "../domain/providerCleanupError.js"; +import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; +import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js"; +import { + type InspectPacketCaptureInput, + packetCaptureSchema, +} from "../domain/captures/packetCapture.js"; +import { err, ok, type Result } from "../domain/result.js"; +import { + runOwnedCommand, + OwnedCommandFailure, +} from "../process/OwnedCommand.js"; +import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js"; +import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js"; + +const OPERATION = "inspect_packet_capture"; +export const TSHARK_PROVIDER_IDENTITY = { + id: "tshark-capture", + name: "REA TShark capture adapter", + version: "tshark@4", +} as const; +// Bound the resident input copy and retained text before allocation/collection. +const LIMITS = { + input_bytes: 32 * 1024 * 1024, + output_bytes: 4 * 1024 * 1024, + command_timeout_ms: 30_000, + address_space_bytes: 1024 * 1024 * 1024, + file_bytes: 64 * 1024 * 1024, + cpu_seconds: 30, +} as const; +type Runtime = Pick; +type Launcher = NonNullable[2]>["launcher"]; + +/** Inspect selected capture files only, retaining cleanup ownership across failures. */ +export class TsharkCaptureProvider implements PacketCapturePort { + readonly #artifactResources = new ArtifactResourceScope(); + readonly #pending = new Map(); + #tail: Promise = Promise.resolve(); + #busy = false; + #closed = false; + constructor( + readonly environment: Readonly = process.env, + readonly launcher?: Launcher, + readonly createRuntime: () => Promise = () => + PrivateRuntimeRoot.create({ prefix: "rea-tshark-" }), + readonly platform: NodeJS.Platform = process.platform, + readonly readArtifact: typeof readStableArtifact = readStableArtifact, + readonly readLimits: typeof readLinuxResourceLimits = readLinuxResourceLimits, + ) {} + inspect( + input: InspectPacketCaptureInput, + options?: ExecutionOptions, + ): Promise> { + if (this.#closed || options?.signal?.aborted) + return Promise.resolve(err(new AnalysisCancelledError(OPERATION))); + if (this.#busy) + return Promise.resolve( + err( + new AnalysisResourceConstraintError( + OPERATION, + "capacity", + "A packet inspection is already active; retry after it completes.", + { max_concurrent_inspections: 1 }, + ), + ), + ); + this.#busy = true; + const work = this.#tail + .then(() => + this.#artifactResources.run(() => this.#inspect(input, options)), + ) + .catch((cause: unknown) => + err( + this.#failure( + cause, + "artifact", + input.path, + null, + undefined, + undefined, + ), + ), + ) + .finally(() => { + this.#busy = false; + }); + this.#tail = work.then( + () => undefined, + () => undefined, + ); + return work; + } + close(): Promise { + this.#closed = true; + const work = this.#tail.then(() => this.#closeResources()); + this.#tail = work.catch(() => undefined); + return work; + } + async #closeResources(): Promise { + let descriptorFailure: AnalysisError | undefined; + try { + await this.#artifactResources.close(); + } catch (cause: unknown) { + descriptorFailure = this.#failure( + cause, + "artifact", + "TShark artifact descriptors", + null, + undefined, + undefined, + ); + } + try { + await this.#release(); + } catch (cause: unknown) { + if (descriptorFailure !== undefined && cause instanceof AnalysisError) + throw analysisErrorWithCleanupFailure( + descriptorFailure, + cause, + OPERATION, + ); + throw cause; + } + if (descriptorFailure !== undefined) throw descriptorFailure; + } + #read(path: string, maximumBytes: number, signal?: AbortSignal) { + return this.readArtifact( + path, + maximumBytes, + signal, + undefined, + this.#artifactResources, + ); + } + async #release(): Promise { + for (const [root, owner] of this.#pending) { + try { + if (owner !== undefined) { + const stopped = await owner.stop(); + if (stopped.status === "incomplete") throw new Error(stopped.reason); + this.#pending.set(root, undefined); + } + await root.close(); + this.#pending.delete(root); + } catch (cause) { + throw new ProviderCleanupError( + TSHARK_PROVIDER_IDENTITY.id, + [root.path], + { reason: String(cause) }, + { operation: OPERATION, cause }, + ); + } + } + } + async #inspect( + input: InspectPacketCaptureInput, + options?: ExecutionOptions, + ): Promise> { + let phase: + | "configuration" + | "artifact" + | "workspace" + | "producer" + | "integrity" = "configuration"; + let capturedOutput: AnalysisCapturedOutput | undefined; + let resourcePath = input.path; + let effectiveLimits: LinuxResourceLimits | undefined; + let root: Runtime | undefined; + let result: Result; + try { + if (options?.signal?.aborted) throw new AnalysisCancelledError(OPERATION); + await this.#release(); + if (this.platform !== "linux") + throw new Error( + "Packet inspection requires Linux with util-linux prlimit and REA_TSHARK_COMMAND so expanded captures are processed under an enforced address-space limit.", + ); + const configured = this.environment.REA_TSHARK_COMMAND ?? ""; + if (!isAbsolute(configured)) + throw new Error("Supply absolute REA_TSHARK_COMMAND for TShark 4.x."); + const command = await realpath(configured); + await access(command, constants.X_OK); + const tool = await this.#read(command, 64 * 1024 * 1024, options?.signal); + const configuredLimiter = + this.environment.REA_TSHARK_PRLIMIT_COMMAND ?? "/usr/bin/prlimit"; + if (!isAbsolute(configuredLimiter)) + throw new Error("REA_TSHARK_PRLIMIT_COMMAND must be absolute."); + const limiter = await realpath(configuredLimiter); + await access(limiter, constants.X_OK); + const limiterIdentity = await this.#read( + limiter, + 64 * 1024 * 1024, + options?.signal, + ); + effectiveLimits = await this.readLimits(LIMITS, this.#artifactResources); + const limitArguments = [ + `--as=${effectiveLimits.address_space_bytes}:${effectiveLimits.address_space_bytes}`, + `--fsize=${effectiveLimits.file_bytes}:${effectiveLimits.file_bytes}`, + `--cpu=${effectiveLimits.cpu_seconds}:${effectiveLimits.cpu_seconds}`, + "--core=0:0", + "--", + command, + ]; + phase = "workspace"; + root = await this.createRuntime(); + this.#pending.set(root, undefined); + const snapshotPath = join(root.path, "capture"); + const configurationPath = join(root.path, "configuration"); + await mkdir(configurationPath, { mode: 0o700 }); + const execute = async (args: readonly string[]) => { + const output = await runOwnedCommand( + { + command: limiter, + arguments: [...limitArguments, ...args], + ...(root === undefined ? {} : { cwd: root.path }), + expectedCommand: null, + runId: `rea-tshark-${randomUUID()}`, + hostEnvironment: { + ...this.environment, + WIRESHARK_CONFIG_DIR: configurationPath, + }, + }, + { + timeoutMs: LIMITS.command_timeout_ms, + diagnosticBytes: LIMITS.output_bytes, + acceptNonZeroExit: true, + }, + { + ...(options?.signal === undefined + ? {} + : { signal: options.signal }), + ...(this.launcher === undefined ? {} : { launcher: this.launcher }), + }, + ); + capturedOutput = { + stdout: output.stdout.text, + stderr: output.stderr.text, + truncated: output.diagnosticTruncated ?? false, + exit_code: output.exitCode ?? null, + signal: output.signal ?? null, + }; + // Finish bounded integrity checks after output; service handles late cancellation with Evidence. + const previousPhase = phase; + phase = "integrity"; + resourcePath = command; + const current = await this.#read(command, 64 * 1024 * 1024); + resourcePath = limiter; + const currentLimiter = await this.#read(limiter, 64 * 1024 * 1024); + if (currentLimiter.sha256 !== limiterIdentity.sha256) + throw new AnalysisArtifactChangedError( + OPERATION, + limiter, + "Configured resource limiter changed during analysis.", + { capturedOutput }, + ); + if (current.sha256 !== tool.sha256) + throw new AnalysisArtifactChangedError( + OPERATION, + command, + "Configured TShark executable changed during analysis.", + { capturedOutput }, + ); + phase = previousPhase; + resourcePath = input.path; + if (output.exitCode !== 0) + throw new ProviderAdapterError( + TSHARK_PROVIDER_IDENTITY.id, + OPERATION, + { + diagnostics: { + exit_code: output.exitCode ?? null, + stdout: output.stdout.text, + stderr: output.stderr.text, + }, + }, + ); + return output; + }; + phase = "configuration"; + const version = await execute(["--version"]); + if ( + !/^TShark \(Wireshark\) 4\.[0-9]+\.[0-9]+(?:[ .(\r\n]|$)/u.test( + version.stdout.text, + ) + ) + throw new Error( + "Expected TShark 4.x; inspect the selected installation.", + ); + phase = "artifact"; + const selected = await this.#read( + input.path, + LIMITS.input_bytes, + options?.signal, + ); + phase = "workspace"; + await writeFile(snapshotPath, selected.bytes, { + flag: "wx", + mode: 0o400, + }); + phase = "producer"; + const args = [ + "-n", + "-r", + "capture", + "-T", + "json", + "--no-duplicate-keys", + "--temp-dir=.", + ...(input.two_pass ? ["-2"] : []), + ...(input.packet_limit === null + ? [] + : ["-c", String(input.packet_limit)]), + ...(input.display_filter === null ? [] : ["-Y", input.display_filter]), + ...input.decode_as.flatMap((value) => ["-d", value]), + ...[...new Set(["frame.number", ...input.fields])].flatMap((field) => [ + "-e", + field, + ]), + ]; + const output = await execute(args); + phase = "integrity"; + resourcePath = snapshotPath; + if ( + (await this.#read(snapshotPath, LIMITS.input_bytes)).sha256 !== + selected.sha256 + ) + throw new AnalysisArtifactChangedError( + OPERATION, + snapshotPath, + "TShark changed the private capture copy.", + capturedOutput === undefined ? {} : { capturedOutput }, + ); + phase = "producer"; + resourcePath = input.path; + const matchingPackets = parsePacketOutput( + output.stdout.text, + input.fields, + ); + const start = Math.min(input.match_offset, matchingPackets.length); + const end = + input.match_limit === null + ? matchingPackets.length + : start + Math.min(input.match_limit, matchingPackets.length - start); + const packets = matchingPackets.slice(start, end); + const report = packetCaptureSchema.parse({ + artifact: { + path: input.path, + sha256: selected.sha256, + bytes: selected.bytes.length, + }, + selection: input, + packets, + output_window: { + matching_packets: matchingPackets.length, + returned_packets: packets.length, + has_more: end < matchingPackets.length, + }, + read_scope: + input.packet_limit === null ? "to-eof" : "up-to-input-packet-limit", + configuration: "private-empty-personal-profile", + diagnostics: output.stderr.text, + tool: { + configured_path: configured, + path: command, + sha256: tool.sha256, + version_banner: version.stdout.text, + version_diagnostics: version.stderr.text, + arguments: args, + }, + resource_limiter: { + configured_path: configuredLimiter, + path: limiter, + sha256: limiterIdentity.sha256, + arguments: limitArguments, + }, + limits: { ...LIMITS, ...effectiveLimits }, + hardware_acquisition: "not-performed", + }); + result = ok( + createAnalysisExecution(report, TSHARK_PROVIDER_IDENTITY, { + subject: { + path: input.path, + sha256: selected.sha256, + format: "file", + }, + locations: [{ kind: "artifact-path", path: input.path }], + limitations: [ + "Fields are TShark dissector observations under the selected filter/decode-as/pass settings, not independently verified protocol semantics. Missing fields are absent from the producer output, not known zero or empty values.", + "Packet limits count input packets before display filtering. Filtered results do not establish whole-capture or device traffic absence. Two-pass reassembly cannot recover bytes missing from the recording or beyond the read limit.", + "Repeated field values remain ordered string arrays; raw packet bytes are included only when explicitly selected as a supported field. Export index/score metadata are omitted. All stderr, including warnings on exit zero, is retained.", + "A private empty personal configuration is used. Installed libraries, global configuration and dissector plugins are trusted and not fully fingerprinted. File-only analysis and disabled name resolution do not constitute an OS sandbox.", + "Linux prlimit enforces soft and hard ceilings before execution, preserving stricter inherited limits: at most 1 GiB address space per process, 64 MiB per file, 30 CPU seconds and no core dumps. Effective limits are reported. Resident input is capped at 32 MiB, each command at 30 wall-clock seconds and combined output at 4 MiB. Output overflow fails. These are per-process/per-file limits, not aggregate memory or filesystem quotas; caller-supplied tools and the limiter are trusted.", + ], + }), + ); + } catch (cause) { + if ( + root !== undefined && + cause instanceof OwnedCommandFailure && + cause.cleanupOwner !== undefined + ) + this.#pending.set(root, cause.cleanupOwner); + result = err( + this.#failure( + cause, + phase, + resourcePath, + root?.path ?? null, + effectiveLimits, + capturedOutput, + ), + ); + } + try { + await this.#release(); + } catch (cause) { + const partialObservation = result.ok + ? createEvidence( + result.value.subject ?? undefined, + result.value.provider, + { + operation: OPERATION, + parameters: input, + result: result.value.result, + confidence: "observed", + locations: result.value.locations, + limitations: result.value.limitations, + }, + ) + : result.error.partialObservation; + return err( + new ProviderCleanupError( + TSHARK_PROVIDER_IDENTITY.id, + [...this.#pending.keys()].map((item) => item.path), + { + reason: String(cause), + previous_error: result.ok + ? null + : projectAnalysisError(result.error), + }, + { + operation: OPERATION, + cause, + ...(partialObservation === undefined ? {} : { partialObservation }), + }, + ), + ); + } + return result; + } + #failure( + cause: unknown, + phase: + | "configuration" + | "artifact" + | "workspace" + | "producer" + | "integrity", + path: string, + workspacePath: string | null, + effectiveLimits: LinuxResourceLimits | undefined, + observedOutput: AnalysisCapturedOutput | undefined, + ): AnalysisError { + if (cause instanceof AnalysisError) return cause; + if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined) + return new ProviderCleanupError( + TSHARK_PROVIDER_IDENTITY.id, + cause.cleanup.resources, + { + reason: cause.cleanup.reason, + ...(observedOutput === undefined + ? {} + : { captured_output: { ...observedOutput } }), + }, + { operation: OPERATION, cause }, + ); + if ( + (cause instanceof Error && cause.name === "AbortError") || + (cause instanceof ArtifactReaderFailure && cause.reason === "cancelled") + ) + return new AnalysisCancelledError(OPERATION, { cause }); + const reason = cause instanceof Error ? cause.message : String(cause); + if (cause instanceof OwnedCommandFailure) { + const capturedOutput = + cause.snapshot === null + ? undefined + : { + stdout: cause.snapshot.stdout.text, + stderr: cause.snapshot.stderr.text, + truncated: cause.snapshot.diagnosticTruncated ?? false, + exit_code: cause.snapshot.exitCode ?? null, + signal: cause.snapshot.signal ?? null, + }; + const detail = { + cause, + ...(capturedOutput === undefined ? {} : { capturedOutput }), + }; + if (cause.reason === "cancelled") + return new AnalysisCancelledError(OPERATION, detail); + if (cause.reason === "timeout") + return new AnalysisTimeoutError( + OPERATION, + LIMITS.command_timeout_ms, + detail, + ); + if (cause.reason === "output-limit") + return new AnalysisResourceConstraintError( + OPERATION, + "memory", + reason, + { output_bytes: LIMITS.output_bytes }, + detail, + ); + if ( + cause.reason === "process" && + (cause.snapshot?.signal === "SIGXFSZ" || + cause.snapshot?.signal === "SIGXCPU") + ) + return new AnalysisResourceConstraintError( + OPERATION, + cause.snapshot.signal === "SIGXFSZ" ? "file-size" : "cpu", + `Producer terminated with ${cause.snapshot.signal}; the signal's exact cause is not independently established.`, + effectiveLimits === undefined ? null : { ...effectiveLimits }, + detail, + ); + return new AnalysisOutputError(OPERATION, reason, detail); + } + if (phase === "workspace") { + const code = + cause instanceof Error && + "code" in cause && + typeof cause.code === "string" + ? cause.code + : null; + if (code === "ENOSPC" || code === "EDQUOT" || code === "EFBIG") + return new AnalysisResourceConstraintError( + OPERATION, + "file-size", + `Private analysis workspace failed (${code}): ${reason}`, + { workspace_path: workspacePath, system_code: code }, + { cause }, + ); + return new ProviderAdapterError(TSHARK_PROVIDER_IDENTITY.id, OPERATION, { + cause, + userMessage: `Cannot prepare the private analysis workspace: ${reason}`, + diagnostics: { + stage: "workspace", + workspace_path: workspacePath, + system_code: code, + reason, + }, + }); + } + if (phase === "integrity") { + const detail = { + cause, + ...(observedOutput === undefined + ? {} + : { capturedOutput: observedOutput }), + }; + if ( + cause instanceof ArtifactReaderFailure && + ["limit", "integrity", "path"].includes(cause.reason) + ) + return new AnalysisArtifactChangedError( + OPERATION, + path, + `Post-run identity verification failed for ${path}: ${reason}`, + detail, + ); + return new ProviderAdapterError(TSHARK_PROVIDER_IDENTITY.id, OPERATION, { + ...detail, + diagnostics: { stage: "integrity", resource_path: path, reason }, + }); + } + if (phase === "configuration") + return new AnalysisCapabilityUnavailableError( + TSHARK_PROVIDER_IDENTITY.id, + OPERATION, + reason, + { + cause, + userMessage: `${reason} On Linux, configure TShark 4.x through REA_TSHARK_COMMAND and util-linux prlimit through REA_TSHARK_PRLIMIT_COMMAND (default /usr/bin/prlimit).`, + }, + ); + if ( + phase === "artifact" && + cause instanceof ArtifactReaderFailure && + cause.reason === "limit" + ) + return new AnalysisResourceConstraintError( + OPERATION, + "file-size", + reason, + { input_bytes: LIMITS.input_bytes }, + { cause }, + ); + if (cause instanceof ArtifactReaderFailure && cause.reason === "integrity") + return new AnalysisArtifactChangedError(OPERATION, path, reason, { + cause, + }); + if ( + phase === "artifact" && + cause instanceof Error && + "code" in cause && + (cause.code === "EACCES" || cause.code === "EPERM") + ) + return new AnalysisAccessDeniedError(OPERATION, path, cause.code, { + cause, + }); + if (phase === "artifact") + return new AnalysisInputError(OPERATION, { cause }, [ + { path: ["path"], reason: "invalid_value", message: reason }, + ]); + return new AnalysisOutputError(OPERATION, reason, { cause }); + } +} diff --git a/src/captures/parsePacketOutput.ts b/src/captures/parsePacketOutput.ts new file mode 100644 index 0000000000..0a408b2162 --- /dev/null +++ b/src/captures/parsePacketOutput.ts @@ -0,0 +1,29 @@ +import { z } from "zod"; +import { AnalysisOutputError } from "../domain/analysisErrorCore.js"; +import { packetFieldsSchema } from "../domain/captures/packetCapture.js"; + +// _index/_score are TShark export metadata, not captured packet fields. +const producerSchema = z.array( + z.object({ _source: z.object({ layers: packetFieldsSchema }) }), +); +export const parsePacketOutput = (text: string, fields: readonly string[]) => { + const decoded: unknown = JSON.parse(text); + const packets = producerSchema + .parse(decoded) + .map((packet) => packet._source.layers); + const selected = new Set(["frame.number", ...fields]); + let previous = 0n; + for (const packet of packets) { + const frame = BigInt(packet["frame.number"][0]); + if ( + frame <= previous || + Object.keys(packet).some((name) => !selected.has(name)) + ) + throw new AnalysisOutputError( + "inspect_packet_capture", + "TShark returned unordered frames or unselected fields.", + ); + previous = frame; + } + return packets; +}; diff --git a/src/cli.ts b/src/cli.ts index 238f8387dc..2e3c34c770 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -1,3 +1,4 @@ +import { registerCaptureCommands } from "./cli/captureCommands.js"; import { registerWasmCommands } from "./cli/wasmCommands.js"; import { snapshotEnvironment } from "./process/snapshotEnvironment.js"; import { createDirectAnalysis } from "./composition/directAnalysis.js"; @@ -94,6 +95,7 @@ export const createCli = ( registerCoreAnalysisCommands(cli, logger, analysis.runDirectAnalysis); registerUtilityCommands(cli, logger, environment, analysis); registerWasmCommands(cli, logger, environment); + registerCaptureCommands(cli, logger, environment); registerEvmCommands(cli, logger, environment); registerGoCommands(cli, logger); registerSqliteCommands(cli, logger, environment); diff --git a/src/cli/captureCommands.ts b/src/cli/captureCommands.ts new file mode 100644 index 0000000000..281be7457e --- /dev/null +++ b/src/cli/captureCommands.ts @@ -0,0 +1,99 @@ +import { z } from "incur"; +import type { PacketCaptureService } from "../application/captures/PacketCaptureService.js"; +import { AnalysisError } from "../domain/analysisErrorBase.js"; +import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js"; +import { ProviderCleanupError } from "../domain/providerCleanupError.js"; +import { resolve } from "node:path"; +import { createPacketCaptureService } from "../composition/captures.js"; +import { + packetCaptureOptionsSchema, + packetCaptureSelectorBudgetIssue, + withinPacketCaptureSelectorBudget, +} from "../domain/captures/packetCapture.js"; +import { CLI_COMMANDS } from "../cliCommandNames.js"; +import { logCliCommand } from "../cliLogging.js"; +import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; +import type { Logger } from "pino"; +import type { CliInstance } from "./types.js"; +import { withCommandCancellation } from "./commandCancellation.js"; +export const registerCaptureCommands = ( + cli: CliInstance, + logger: Logger, + environment: Readonly, + service: Pick< + PacketCaptureService, + "inspect" | "close" + > = createPacketCaptureService(environment), +): void => { + cli.command(CLI_COMMANDS.inspectPacketCapture, { + description: + "Inspect an existing packet capture with caller-supplied TShark", + args: z.object({ path: z.string().describe("Local packet capture") }), + options: z + .object({ + ...packetCaptureOptionsSchema.shape, + match_limit: z + .number() + .int() + .positive() + .optional() + .describe("Returned matching frames per page; default 50"), + all_matches: z + .boolean() + .default(false) + .describe( + "Return all remaining matching frames instead of a bounded page", + ), + }) + .superRefine((options, context) => { + if (options.all_matches && options.match_limit !== undefined) + context.addIssue({ + code: "custom", + path: ["all_matches"], + message: "all_matches and match_limit are mutually exclusive", + }); + }) + .refine( + withinPacketCaptureSelectorBudget, + packetCaptureSelectorBudgetIssue, + ), + run: ({ args, options: { all_matches, ...options } }) => + withCommandCancellation((signal) => + logCliCommand(logger, CLI_COMMANDS.inspectPacketCapture, async () => { + const result = await service.inspect( + { + ...options, + ...(all_matches ? { match_limit: null } : {}), + path: resolve(args.path), + }, + { signal }, + ); + // A one-shot command must retry retained cleanup before discarding its owner. + try { + await service.close(); + } catch (cause) { + const cleanup = + cause instanceof AnalysisError + ? cause + : new ProviderCleanupError( + "tshark-capture", + [], + { reason: String(cause) }, + { cause, operation: "inspect_packet_capture" }, + ); + if (result.ok) cleanup.retainPartialObservation(result.value); + return projectAnalysisError( + result.ok + ? cleanup + : analysisErrorWithCleanupFailure( + result.error, + cleanup, + "inspect_packet_capture", + ), + ); + } + return result.ok ? result.value : projectAnalysisError(result.error); + }), + ), + }); +}; diff --git a/src/cliCommandNames.ts b/src/cliCommandNames.ts index 154c49396d..6f5b5e1499 100644 --- a/src/cliCommandNames.ts +++ b/src/cliCommandNames.ts @@ -1,5 +1,6 @@ /** Canonical names used by every one-shot CLI command registration. */ export const CLI_COMMANDS = Object.freeze({ + inspectPacketCapture: "inspect-packet-capture", inspectWasmArtifact: "inspect-wasm-artifact", inspectSqliteDatabase: "inspect-sqlite-database", inspectEvmInterface: "inspect-evm-interface", diff --git a/src/composition/captures.ts b/src/composition/captures.ts new file mode 100644 index 0000000000..264cf6aa59 --- /dev/null +++ b/src/composition/captures.ts @@ -0,0 +1,6 @@ +import { PacketCaptureService } from "../application/captures/PacketCaptureService.js"; +import { TsharkCaptureProvider } from "../captures/TsharkCaptureProvider.js"; +export const createPacketCaptureService = ( + environment: Readonly = process.env, +): PacketCaptureService => + new PacketCaptureService(new TsharkCaptureProvider(environment)); diff --git a/src/contracts/captures/captureToolContracts.ts b/src/contracts/captures/captureToolContracts.ts new file mode 100644 index 0000000000..3e67b3b190 --- /dev/null +++ b/src/contracts/captures/captureToolContracts.ts @@ -0,0 +1,31 @@ +import { + inspectPacketCaptureInputSchema, + packetCaptureSchema, +} from "../../domain/captures/packetCapture.js"; +import type { ToolContract } from "../toolContractTypes.js"; +import { toolContractMetadata } from "../toolEffects.js"; +import { evidenceResultOf } from "../toolOutputSchemaPrimitives.js"; +export const CAPTURE_TOOL_CONTRACTS = [ + { + name: "inspect_packet_capture", + ...toolContractMetadata("inspect_packet_capture"), + kind: "artifact-provider", + description: + "Inspect caller-selected fields in an existing local packet capture with TShark 4.x. Return 50 matching frames by default, with match_offset/match_limit paging and coverage counts; match_limit:null requests all remaining matches. Preserve repeated values, frame numbers, display filter, decode-as rules and pass mode in inline Evidence with source/tool identity and diagnostics. Optional packet_limit counts input packets before filtering; output paging does not shorten dissection or two-pass reassembly. Requires Linux, util-linux prlimit, and absolute REA_TSHARK_COMMAND. File-only analysis; no live capture or device communication. Missing fields or no matches do not establish absence of device traffic.", + inputSchema: inspectPacketCaptureInputSchema, + outputSchema: evidenceResultOf(packetCaptureSchema), + examples: [ + { + title: "Inspect UDP exchanges in a recording", + input: { + path: "/artifacts/device.pcapng", + fields: ["ip.addr", "udp.port", "data.data"], + display_filter: "udp", + decode_as: [], + packet_limit: null, + two_pass: true, + }, + }, + ], + }, +] as const satisfies readonly ToolContract[]; diff --git a/src/contracts/toolContracts.ts b/src/contracts/toolContracts.ts index 216e2b7928..a2e0c0d726 100644 --- a/src/contracts/toolContracts.ts +++ b/src/contracts/toolContracts.ts @@ -1,3 +1,4 @@ +import { CAPTURE_TOOL_CONTRACTS } from "./captures/captureToolContracts.js"; import { WASM_TOOL_CONTRACTS } from "./wasm/wasmToolContracts.js"; import { EVM_TOOL_CONTRACTS } from "./evm/evmToolContracts.js"; import { GO_TOOL_CONTRACTS } from "./go/goToolContracts.js"; @@ -39,6 +40,7 @@ export const TOOL_CONTRACTS = [ ...GO_TOOL_CONTRACTS, ...WASM_TOOL_CONTRACTS, ...SQLITE_TOOL_CONTRACTS, + ...CAPTURE_TOOL_CONTRACTS, ...MANAGED_TOOL_CONTRACTS, ...ANDROID_TOOL_CONTRACTS, ...APKTOOL_TOOL_CONTRACTS, diff --git a/src/contracts/toolEffects.ts b/src/contracts/toolEffects.ts index 74b8a96651..dde8adf188 100644 --- a/src/contracts/toolEffects.ts +++ b/src/contracts/toolEffects.ts @@ -59,6 +59,11 @@ export const TOOL_EFFECTS: Readonly> = { writesFilesystem: true, launchesProcess: true, }), + inspect_packet_capture: effects({ + mutatesSession: true, + launchesProcess: true, + writesFilesystem: true, + }), inspect_wasm_artifact: effects({ mutatesSession: true, writesFilesystem: true, diff --git a/src/domain/captures/packetCapture.ts b/src/domain/captures/packetCapture.ts new file mode 100644 index 0000000000..deb439cadf --- /dev/null +++ b/src/domain/captures/packetCapture.ts @@ -0,0 +1,168 @@ +import { z } from "zod"; +import { localPathStringSchema } from "../localPath.js"; + +const argument = z + .string() + .min(1) + .max(8192) + .refine((value) => !value.includes("\0"), "NUL is not an argument character"); +const captureInputShape = z.strictObject({ + path: localPathStringSchema.describe("Absolute local capture path"), + fields: z + .array(argument) + .max(1024) + .min(1) + .describe( + "TShark field names to return, e.g. ip.addr, usb.endpoint_address, btatt.opcode; frame.number is always included", + ), + display_filter: argument + .nullable() + .default(null) + .describe( + "Optional Wireshark display filter (-Y), applied within the read limit", + ), + decode_as: z + .array(argument) + .max(1024) + .default([]) + .describe("Explicit TShark decode-as selections, e.g. tcp.port==8888,http"), + packet_limit: z + .number() + .int() + .positive() + .max(2147483647) + .nullable() + .default(null) + .describe( + "Maximum input packets to read before filtering; omitted scans to EOF within resource limits", + ), + two_pass: z + .boolean() + .default(true) + .describe( + "Use two-pass analysis for fields requiring later packets; false requests a single pass", + ), + match_offset: z + .number() + .int() + .nonnegative() + .default(0) + .describe("Skip this many matching packets after filtering; default 0"), + match_limit: z + .number() + .int() + .positive() + .nullable() + .default(50) + .describe( + "Return at most this many matching packets after filtering; default 50. Null explicitly returns all remaining matches within producer resource limits.", + ), +}); +// Bound UTF-8 selector text, terminators, option names and 64-bit argv pointers +// before Set/flatMap expansion. 32 KiB reserves headroom below Linux execve's +// 32-page argv+environment floor; ambient environments can still exhaust it. +export const withinPacketCaptureSelectorBudget = (input: { + fields: readonly string[]; + decode_as: readonly string[]; + display_filter: string | null; +}): boolean => { + if (input.fields.length > 1024 || input.decode_as.length > 1024) return false; + let bytes = 0; + for (const selections of [input.fields, input.decode_as]) + for (const value of selections) { + bytes += Buffer.byteLength(value, "utf8") + 20; + if (bytes > 32768) return false; + } + if (input.display_filter !== null) + bytes += Buffer.byteLength(input.display_filter, "utf8") + 20; + return bytes <= 32768; +}; +export const packetCaptureSelectorBudgetIssue = { + message: + "Packet selectors exceed the 32 KiB encoded argument budget; reduce fields, decode-as rules or filter text.", +}; +export const inspectPacketCaptureInputSchema = captureInputShape.refine( + withinPacketCaptureSelectorBudget, + packetCaptureSelectorBudgetIssue, +); +export const packetCaptureOptionsSchema = captureInputShape + .omit({ path: true }) + .refine(withinPacketCaptureSelectorBudget, packetCaptureSelectorBudgetIssue); +export type InspectPacketCaptureInput = z.infer< + typeof inspectPacketCaptureInputSchema +>; + +/** Values remain strings; repeated occurrences remain ordered arrays. */ +export const packetFieldsSchema = z + .object({ + "frame.number": z.tuple([z.string().regex(/^[1-9][0-9]*$/)]), + }) + .catchall(z.array(z.string())); +export const packetCaptureSchema = z + .strictObject({ + artifact: z.strictObject({ + path: z.string(), + sha256: z.string().regex(/^[a-f0-9]{64}$/), + bytes: z.number().int().nonnegative(), + }), + selection: inspectPacketCaptureInputSchema, + packets: z.array(packetFieldsSchema), + output_window: z.strictObject({ + matching_packets: z + .number() + .int() + .nonnegative() + .describe( + "Matches in the completed producer output within the indicated input read scope, not total capture packets", + ), + returned_packets: z.number().int().nonnegative(), + has_more: z.boolean(), + }), + read_scope: z.enum(["to-eof", "up-to-input-packet-limit"]), + diagnostics: z.string(), + tool: z.strictObject({ + configured_path: z.string(), + path: z.string(), + sha256: z.string().regex(/^[a-f0-9]{64}$/), + version_banner: z.string(), + version_diagnostics: z.string(), + arguments: z.array(z.string()), + }), + configuration: z.literal("private-empty-personal-profile"), + resource_limiter: z.strictObject({ + configured_path: z.string(), + path: z.string(), + sha256: z.string().regex(/^[a-f0-9]{64}$/), + arguments: z.array(z.string()), + }), + limits: z.strictObject({ + input_bytes: z.number().int().positive(), + output_bytes: z.number().int().positive(), + command_timeout_ms: z.number().int().positive(), + address_space_bytes: z.number().int().nonnegative(), + file_bytes: z.number().int().nonnegative(), + cpu_seconds: z.number().int().nonnegative(), + }), + hardware_acquisition: z.literal("not-performed"), + }) + .superRefine((report, context) => { + const remaining = Math.max( + report.output_window.matching_packets - report.selection.match_offset, + 0, + ); + const expected = + report.selection.match_limit === null + ? remaining + : Math.min(remaining, report.selection.match_limit); + if ( + report.packets.length !== expected || + report.output_window.returned_packets !== expected || + report.output_window.has_more !== remaining > expected + ) + context.addIssue({ + code: "custom", + path: ["output_window"], + message: + "Returned packet window must match examined matches and caller-selected offset/limit", + }); + }); diff --git a/src/process/LinuxResourceLimits.test.ts b/src/process/LinuxResourceLimits.test.ts new file mode 100644 index 0000000000..69a584d745 --- /dev/null +++ b/src/process/LinuxResourceLimits.test.ts @@ -0,0 +1,43 @@ +import { expect, it } from "vitest"; +import { clampLinuxResourceLimits } from "./LinuxResourceLimits.js"; +const configured = { + address_space_bytes: 1073741824, + file_bytes: 67108864, + cpu_seconds: 30, +}; +const report = ( + memory: string, + file = "unlimited unlimited", + cpu = "unlimited unlimited", +) => + `Limit Soft Limit Hard Limit Units\nMax address space ${memory} bytes\nMax file size ${file} bytes\nMax cpu time ${cpu} seconds\n`; +it("keeps configured ceilings below unlimited or larger inherited limits", () => { + expect( + clampLinuxResourceLimits(configured, report("unlimited unlimited")), + ).toEqual(configured); + expect( + clampLinuxResourceLimits( + configured, + report("18446744073709551615 18446744073709551615"), + ), + ).toEqual(configured); +}); +it("preserves stricter soft limits and lower hard ceilings, including zero", () => { + expect( + clampLinuxResourceLimits( + configured, + report("268435456 2147483648", "0 1048576", "10 20"), + ), + ).toEqual({ address_space_bytes: 268435456, file_bytes: 0, cpu_seconds: 10 }); + expect( + clampLinuxResourceLimits(configured, report("268435456 536870912")), + ).toMatchObject({ address_space_bytes: 268435456 }); +}); +it("refuses missing, ambiguous or malformed kernel limits before launch", () => { + for (const value of [ + "", + report("invalid unlimited"), + report("unlimited unlimited") + "Max address space 1 2 bytes\n", + ]) + expect(() => clampLinuxResourceLimits(configured, value)).toThrow(); +}); diff --git a/src/process/LinuxResourceLimits.ts b/src/process/LinuxResourceLimits.ts new file mode 100644 index 0000000000..e6eefcefec --- /dev/null +++ b/src/process/LinuxResourceLimits.ts @@ -0,0 +1,82 @@ +import { open } from "node:fs/promises"; +import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js"; +import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; +import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js"; + +/** Per-process ceilings inherited by an owned Linux producer. */ +export interface LinuxResourceLimits { + readonly address_space_bytes: number; + readonly file_bytes: number; + readonly cpu_seconds: number; +} +const rows = { + address_space_bytes: "Max address space", + file_bytes: "Max file size", + cpu_seconds: "Max cpu time", +} as const; + +/** Never raise either inherited soft policy or hard ceiling. */ +export const clampLinuxResourceLimits = ( + configured: LinuxResourceLimits, + report: string, +): LinuxResourceLimits => { + const effective = { ...configured }; + for (const key of Object.keys(rows) as (keyof LinuxResourceLimits)[]) { + const matches = report + .split("\n") + .filter((line) => line.startsWith(`${rows[key]} `)); + const values = matches[0]?.slice(rows[key].length).trim().split(/\s+/u); + if (matches.length !== 1 || values === undefined || values.length !== 3) + throw new Error(`Cannot establish inherited Linux limit: ${rows[key]}`); + let ceiling = BigInt(configured[key]); + for (const token of values.slice(0, 2)) { + if (token === "unlimited") continue; + if (!/^[0-9]+$/u.test(token)) + throw new Error(`Invalid inherited Linux limit: ${rows[key]}`); + const value = BigInt(token); + if (value < ceiling) ceiling = value; + } + effective[key] = Number(ceiling); + } + return effective; +}; + +/** Read the trusted kernel report with a fixed allocation before spawning. */ +export const readLinuxResourceLimits = async ( + configured: LinuxResourceLimits, + resourceScope?: ArtifactResourceScope, +): Promise => { + const file = await open("/proc/self/limits", "r"); + const owner = new OwnedFileHandle(file); + let failure: unknown; + try { + const buffer = Buffer.alloc(16 * 1024); + const { bytesRead } = await file.read(buffer, 0, buffer.length, 0); + if (bytesRead === buffer.length) + throw new Error("Inherited Linux limits exceed the report budget."); + return clampLinuxResourceLimits( + configured, + buffer.subarray(0, bytesRead).toString("utf8"), + ); + } catch (cause: unknown) { + failure = cause; + throw cause; + } finally { + if (resourceScope === undefined) await file.close(); + else { + const cleaned = await resourceScope.release({ + kind: "file-handle", + resource: "/proc/self/limits", + handle: owner, + }); + if (cleaned.kind === "failed") + throw ArtifactReaderFailure.withCleanup( + failure ?? cleaned.cause, + ArtifactReaderFailure.cleanupObservation( + cleaned.cause, + "/proc/self/limits", + ), + ); + } + } +}; diff --git a/src/server/createServer.ts b/src/server/createServer.ts index b67a765cac..8823b4ea76 100644 --- a/src/server/createServer.ts +++ b/src/server/createServer.ts @@ -1,3 +1,6 @@ +import type { PacketCaptureService } from "../application/captures/PacketCaptureService.js"; +import { createPacketCaptureService } from "../composition/captures.js"; +import { registerCaptureTools } from "./registerCaptureTools.js"; import type { WasmArtifactService } from "../application/wasm/WasmArtifactService.js"; import { wasmArtifactModule } from "../composition/wasm.js"; import { registerWasmTools } from "./registerWasmTools.js"; @@ -119,6 +122,7 @@ const ACTIVE_TARGET_INSTRUCTIONS = export interface CreateServerOptions { readonly environment?: Readonly; readonly delivery?: ToolResultDelivery; + readonly packetCapture?: PacketCaptureService; readonly wasmArtifact?: WasmArtifactService; readonly evmInterface?: EvmInterfaceService; readonly sqliteDatabase?: SqliteDatabaseService; @@ -161,6 +165,9 @@ const installSessionToolAvailability = ( const basePolicy = sessionAvailabilityPolicy(options.availabilityPolicy, { optionalProviderLoadFailures: options.optionalProviderLoadFailures, optionalFeatures: { + packetCaptureEnabled: + options.packetCapture !== undefined || + (linux && isAbsolute(environment.REA_TSHARK_COMMAND ?? "")), wasmArtifactEnabled: options.wasmArtifact !== undefined || isAbsolute(environment.REA_WABT_BIN_DIRECTORY ?? ""), @@ -292,6 +299,8 @@ export const createServer = ( environment, options.wasmArtifact, ); + const packetCapture = + options.packetCapture ?? createPacketCaptureService(environment); const apktool = options.apktoolAnalysis ?? createApktoolResourceAnalysisProvider(environment); @@ -304,6 +313,7 @@ export const createServer = ( android.close(), fridaInstrumentation.closeAll(), wasmArtifact.close(), + packetCapture.close(), apktool.close(), jeb.close(), adbDevice.close(), @@ -384,6 +394,13 @@ export const createServer = ( for (const result of results) if (result.status === "rejected") throw result.reason; }; + registerCaptureTools( + server, + packetCapture, + toolLogger, + recordEvidence, + analysisAdmission, + ); registerConfiguredAnalysisTools( toolContext, android, diff --git a/src/server/registerCaptureTools.ts b/src/server/registerCaptureTools.ts new file mode 100644 index 0000000000..c097f89be5 --- /dev/null +++ b/src/server/registerCaptureTools.ts @@ -0,0 +1,31 @@ +import type { EvidenceMcpServer } from "./EvidenceMcpServer.js"; +import type { PacketCaptureService } from "../application/captures/PacketCaptureService.js"; +import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js"; +import { toolContract } from "../contracts/toolContracts.js"; +import type { Logger } from "pino"; +import type { WithAdmittedAnalysis } from "./analysisAdmission.js"; +import { createEvidenceToolHandler } from "./contractToolHandler.js"; +import { toolRegistrationOptions } from "./toolRegistrationOptions.js"; + +/** Bind the named capture contract through shared admission and retained Evidence delivery. */ +export const registerCaptureTools = ( + server: EvidenceMcpServer, + service: PacketCaptureService, + logger: Logger, + recordEvidence?: EvidenceWriter["recordEvidence"], + withAdmittedAnalysis?: WithAdmittedAnalysis, +): void => { + const contract = toolContract("inspect_packet_capture"); + const handler = createEvidenceToolHandler( + server, + (input, options) => service.inspect(input, options), + logger, + recordEvidence, + withAdmittedAnalysis, + ); + server.registerTool( + contract.name, + toolRegistrationOptions(contract), + handler(contract), + ); +}; diff --git a/src/server/sessionAvailabilityPolicy.ts b/src/server/sessionAvailabilityPolicy.ts index e3318a1c5f..5797d52e46 100644 --- a/src/server/sessionAvailabilityPolicy.ts +++ b/src/server/sessionAvailabilityPolicy.ts @@ -8,6 +8,7 @@ export interface SessionAvailabilityDefaults { | undefined; readonly optionalFeatures?: Pick< AvailabilityPolicy, + | "packetCaptureEnabled" | "wasmArtifactEnabled" | "evmInterfaceEnabled" | "browserObservationEnabled" @@ -32,6 +33,8 @@ export const sessionAvailabilityPolicy = ( const policy = configured ?? (() => ({ + packetCaptureEnabled: + defaults.optionalFeatures?.packetCaptureEnabled ?? false, wasmArtifactEnabled: defaults.optionalFeatures?.wasmArtifactEnabled ?? false, evmInterfaceEnabled: diff --git a/tests/boundary/captures/captureDescriptorCleanup.test.ts b/tests/boundary/captures/captureDescriptorCleanup.test.ts new file mode 100644 index 0000000000..a1ef5efc19 --- /dev/null +++ b/tests/boundary/captures/captureDescriptorCleanup.test.ts @@ -0,0 +1,107 @@ +import { expect, it, onTestFinished, vi } from "vitest"; +import { writeFile } from "node:fs/promises"; +import type { FileHandle } from "node:fs/promises"; +import { join } from "node:path"; +import { TsharkCaptureProvider } from "../../../src/captures/TsharkCaptureProvider.js"; +import { PacketCaptureService } from "../../../src/application/captures/PacketCaptureService.js"; +import { PrivateRuntimeRoot } from "../../../src/process/PrivateRuntimeRoot.js"; +import { spawnOwnedProviderProcess } from "../../../src/process/ProviderProcess.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const cleanup = vi.hoisted(() => ({ + path: "", + denied: false, + handles: [] as FileHandle[], +})); +vi.mock("node:fs/promises", async (importOriginal) => { + const filesystem = await importOriginal(); + return { + ...filesystem, + open: async (...args: Parameters) => { + const handle = await filesystem.open(...args); + if (args[0] === cleanup.path) { + cleanup.handles.push(handle); + const close = handle.close.bind(handle); + handle.close = async () => { + if (cleanup.denied && handle.fd >= 0) + throw new Error("Descriptor close deliberately denied"); + await close(); + }; + } + return handle; + }, + }; +}); + +it + .runIf(process.platform === "linux") + .each(["capture", "tool", "limiter", "kernel"])( + "retains failed %s descriptors and retries them before further acquisition and shutdown", + async (member) => { + const directory = await createTestTempDirectory("rea-capture-descriptor-"); + const path = join(directory, "recording.pcap"); + const command = join(directory, "tshark.mjs"); + const limiter = join(directory, "limiter"); + await writeFile(path, "recording bytes"); + await writeFile(limiter, "limiter", { mode: 0o700 }); + await writeFile( + command, + `if(process.argv.includes('--version'))console.log('TShark (Wireshark) 4.0.17');else console.log('[{"_source":{"layers":{"frame.number":["1"]}}}]');`, + { mode: 0o700 }, + ); + let launches = 0; + const provider = new TsharkCaptureProvider( + { + ...process.env, + REA_TSHARK_COMMAND: command, + REA_TSHARK_PRLIMIT_COMMAND: limiter, + }, + (input) => { + launches += 1; + return spawnOwnedProviderProcess({ + ...input, + command: process.execPath, + arguments: [command, ...input.arguments], + expectedCommand: process.execPath, + }); + }, + () => PrivateRuntimeRoot.create({ parent: directory }), + ); + cleanup.path = + member === "capture" + ? path + : member === "tool" + ? command + : member === "limiter" + ? limiter + : "/proc/self/limits"; + cleanup.denied = true; + cleanup.handles = []; + onTestFinished(async () => { + cleanup.denied = false; + await provider.close(); + for (const handle of cleanup.handles) await handle.close(); + cleanup.path = ""; + }); + const service = new PacketCaptureService(provider); + const first = await service.inspect({ path, fields: ["frame.number"] }); + expect(first).toMatchObject({ + ok: false, + error: { cleanupIncomplete: true, cleanupResources: [cleanup.path] }, + }); + expect(cleanup.handles).toHaveLength(1); + expect(cleanup.handles[0]?.fd).toBeGreaterThanOrEqual(0); + const launchCount = launches; + expect( + await service.inspect({ path, fields: ["frame.number"] }), + ).toMatchObject({ ok: false, error: { cleanupIncomplete: true } }); + expect(cleanup.handles).toHaveLength(1); + expect(launches).toBe(launchCount); + await expect(provider.close()).rejects.toMatchObject({ + cleanupIncomplete: true, + }); + cleanup.denied = false; + await provider.close(); + expect(cleanup.handles[0]?.fd).toBe(-1); + }, +); diff --git a/tests/boundary/captures/captureProvider.test.ts b/tests/boundary/captures/captureProvider.test.ts new file mode 100644 index 0000000000..9f5fb4afc7 --- /dev/null +++ b/tests/boundary/captures/captureProvider.test.ts @@ -0,0 +1,467 @@ +import { OwnedCommandFailure } from "../../../src/process/OwnedCommand.js"; +import { it, expect, onTestFinished } from "vitest"; +import { writeFile, readdir, chmod, symlink } from "node:fs/promises"; +import { join } from "node:path"; +import { readStableArtifact } from "../../../src/artifacts/readStableArtifact.js"; +import { TsharkCaptureProvider } from "../../../src/captures/TsharkCaptureProvider.js"; +import { PacketCaptureService } from "../../../src/application/captures/PacketCaptureService.js"; +import { packetCaptureSchema } from "../../../src/domain/captures/packetCapture.js"; +import { PrivateRuntimeRoot } from "../../../src/process/PrivateRuntimeRoot.js"; +import { spawnOwnedProviderProcess } from "../../../src/process/ProviderProcess.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { createDeferred } from "../../fixtures/binarySession.js"; + +// A command-boundary simulator: real process ownership/filesystem, not real TShark coverage. +const fixture = async (mode = "ok", failCleanup = false) => { + const directory = await createTestTempDirectory("rea-capture-test-"); + const path = join(directory, "recorded bytes.pcap"); + await writeFile(path, "recording bytes"); + const script = join(directory, "producer.mjs"); + await writeFile( + script, + `import {readFileSync, writeFileSync, chmodSync, truncateSync} from 'node:fs'; +if (process.argv.includes('--version')) { console.log('TShark (Wireshark) 4.6.9 (fixture)'); console.error('version warning'); } +else if (${JSON.stringify(mode)} === 'nonzero') { console.error('invalid decoder'); process.exitCode = 1; } +else if (${JSON.stringify(mode)} === 'malformed-copy') { chmodSync('capture', 0o600); writeFileSync('capture', 'changed'); } +else if (${JSON.stringify(mode)} === 'oversized-copy') { chmodSync('capture', 0o600); truncateSync('capture', 32 * 1024 * 1024 + 1); console.log('[]'); } +else if (${JSON.stringify(mode)} === 'overflow') process.stdout.write('x'.repeat(5 * 1024 * 1024)); +else if (${JSON.stringify(mode)} === 'wait') { console.log('ready'); setInterval(() => {}, 1000); } +else if (${JSON.stringify(mode)} === 'invalid-json') console.log('not json'); +else if (${JSON.stringify(mode)} === 'many') console.log(JSON.stringify(Array.from({length:90},(_,i)=>({_source:{layers:{"frame.number":[String(i+1)],"ip.addr":["192.0.2.1"]}}})))); +else { if(readFileSync('capture', 'utf8') !== 'recording bytes') throw Error('wrong copy'); console.log(JSON.stringify([{_source:{layers:{"frame.number":["1"],"ip.addr":["192.0.2.1","192.0.2.2"]}}}])); console.error('producer warning'); } +`, + ); + await chmod(script, 0o700); + const configuredCommand = + mode === "symlink" ? join(directory, "configured-tool") : script; + if (mode === "symlink") await symlink(script, configuredCommand); + const controller = new AbortController(); + let calls = 0; + let mayClean = !failCleanup; + const acquired = createDeferred(); + const release = createDeferred(); + const provider = new TsharkCaptureProvider( + { + ...process.env, + REA_TSHARK_COMMAND: configuredCommand, + REA_TSHARK_PRLIMIT_COMMAND: configuredCommand, + }, + async (input) => { + calls++; + if (mode === "file-limit" && !input.arguments.includes("--version")) { + const stdout = { text: "", bytes: 0, observedBytes: 0 }; + const stderr = { + text: "before file limit\n", + bytes: 18, + observedBytes: 18, + }; + throw new OwnedCommandFailure("process", "Synthetic SIGXFSZ snapshot", { + stdout, + stderr, + exitCode: null, + signal: "SIGXFSZ", + }); + } + const spawned = await spawnOwnedProviderProcess({ + ...input, + command: process.execPath, + arguments: [script, ...input.arguments], + expectedCommand: process.execPath, + }); + if (mode === "wait" && !input.arguments.includes("--version")) + spawned.process.stdout?.once("data", () => controller.abort()); + return spawned; + }, + async () => { + if (mode === "workspace-full") + throw Object.assign(new Error("No space left on device"), { + code: "ENOSPC", + }); + const root = await PrivateRuntimeRoot.create({ + parent: directory, + prefix: "decoder-", + }); + if (mode === "gated") { + acquired.resolve(); + await release.promise; + } + if (mode === "snapshot-write-failure") + await writeFile(join(root.path, "capture"), "existing"); + return { + path: root.path, + close: async () => { + if (!mayClean) throw new Error("cleanup fixture failure"); + await root.close(); + }, + }; + }, + "linux", + async (...args) => { + if (mode === "oversized-tool" && calls === 2 && args[0] === script) { + const { truncate } = await import("node:fs/promises"); + await truncate(script, 64 * 1024 * 1024 + 1); + } + if (mode === "post-run-abort" && calls === 2) controller.abort(); + return readStableArtifact(...args); + }, + async (limits) => ({ ...limits, file_bytes: 1048576 }), + ); + onTestFinished(async () => { + release.resolve(); + mayClean = true; + await provider.close(); + }); + const service = new PacketCaptureService(provider); + return { + service, + directory, + path, + controller, + calls: () => calls, + acquired: acquired.promise, + release: () => release.resolve(), + allowCleanup: () => { + mayClean = true; + }, + }; +}; + +it("refuses new acquisition after provider shutdown", async () => { + const f = await fixture(); + await f.service.close(); + expect( + await f.service.inspect({ path: f.path, fields: ["ip.addr"] }), + ).toMatchObject({ + ok: false, + error: { _tag: "AnalysisCancelledError" }, + }); + expect(f.calls()).toBe(0); +}); + +it("pages matching observations without changing dissection scope or frame identities", async () => { + const f = await fixture("many"); + const input = { path: f.path, fields: ["ip.addr"] }; + for (const [options, first, count] of [ + [{}, 1, 50], + [{ match_offset: 50, match_limit: 30 }, 51, 30], + [{ match_limit: null }, 1, 90], + [{ match_offset: 100 }, undefined, 0], + ] as const) { + const result = await f.service.inspect({ ...input, ...options }); + if (!result.ok) throw result.error; + const report = packetCaptureSchema.parse(result.value.normalized_result); + expect(report.packets).toHaveLength(count); + expect(report.packets[0]?.["frame.number"][0]).toBe( + first === undefined ? undefined : String(first), + ); + expect(report.output_window.matching_packets).toBe(90); + expect(report.output_window.returned_packets).toBe(count); + expect(report.read_scope).toBe("to-eof"); + expect(report.tool.arguments).not.toContain("-c"); + } +}); + +it("bounds admission without queuing additional snapshots or delaying cancellation", async () => { + const f = await fixture("gated"); + const input = { path: f.path, fields: ["ip.addr"] }; + const first = f.service.inspect(input); + await f.acquired; + const second = f.service.inspect(input); + const cancelled = f.service.inspect(input, { signal: AbortSignal.abort() }); + f.release(); + expect(await first).toMatchObject({ ok: true }); + expect(await second).toMatchObject({ + ok: false, + error: { _tag: "AnalysisResourceConstraintError", resource: "capacity" }, + }); + expect(await cancelled).toMatchObject({ + ok: false, + error: { _tag: "AnalysisCancelledError" }, + }); + expect(f.calls()).toBe(2); + expect(await f.service.inspect(input)).toMatchObject({ ok: true }); +}); + +it("inspects an immutable copy, preserves options and warnings, and releases its directory", async () => { + const f = await fixture(); + const result = await f.service.inspect({ + path: f.path, + fields: ["ip.addr"], + display_filter: "udp", + packet_limit: 10, + decode_as: ["udp.port==5555,dns"], + }); + if (!result.ok) throw result.error; + const report = packetCaptureSchema.parse(result.value.normalized_result); + expect(report.packets).toEqual([ + { "frame.number": ["1"], "ip.addr": ["192.0.2.1", "192.0.2.2"] }, + ]); + expect(report.diagnostics).toBe("producer warning\n"); + expect(report.tool.version_diagnostics).toBe("version warning\n"); + expect(report.read_scope).toBe("up-to-input-packet-limit"); + expect(report.tool.arguments).toEqual([ + "-n", + "-r", + "capture", + "-T", + "json", + "--no-duplicate-keys", + "--temp-dir=.", + "-2", + "-c", + "10", + "-Y", + "udp", + "-d", + "udp.port==5555,dns", + "-e", + "frame.number", + "-e", + "ip.addr", + ]); + expect((await readdir(f.directory)).sort()).toEqual([ + "producer.mjs", + "recorded bytes.pcap", + ]); +}); + +it.each([ + ["invalid-json", "AnalysisOutputError"], + ["nonzero", "ProviderAdapterError"], + ["malformed-copy", "AnalysisArtifactChangedError"], + ["overflow", "AnalysisResourceConstraintError"], + ["wait", "AnalysisCancelledError"], +])( + "preserves %s failure and releases the process and snapshot", + async (mode, tag) => { + const f = await fixture(mode); + const result = await f.service.inspect( + { path: f.path, fields: ["ip.addr"] }, + { signal: f.controller.signal }, + ); + if (result.ok) throw new Error("Expected capture analysis failure"); + expect(result.error._tag).toBe(tag); + expect(result.error.cleanupIncomplete).toBe(false); + expect( + (await readdir(f.directory)).filter((name) => + name.startsWith("decoder-"), + ), + ).toEqual([]); + }, +); + +it("retains a completed observation and cleanup ownership for retry", async () => { + const f = await fixture("ok", true); + const result = await f.service.inspect({ path: f.path, fields: ["ip.addr"] }); + if (result.ok) throw new Error("Expected cleanup failure"); + expect(result.error.cleanupIncomplete).toBe(true); + expect(result.error.partialObservation).toMatchObject({ + operation: "inspect_packet_capture", + }); + const retry = await f.service.inspect({ path: f.path, fields: ["ip.addr"] }); + expect(retry.ok).toBe(false); + expect(f.calls()).toBe(2); + f.allowCleanup(); + await f.service.close(); + expect( + (await readdir(f.directory)).filter((name) => name.startsWith("decoder-")), + ).toEqual([]); +}); + +it("rejects cancelled and invalid requests before any provider work", async () => { + const f = await fixture(); + for (const input of [ + { path: "relative.pcap", fields: ["ip.addr"] }, + { path: f.path, fields: ["ip.addr"], interface: "lo0" }, + ]) { + const result = await f.service.inspect(input); + expect(result.ok).toBe(false); + } + const result = await f.service.inspect( + { path: f.path, fields: ["ip.addr"] }, + { signal: AbortSignal.abort() }, + ); + if (result.ok) throw new Error("Expected cancellation"); + expect(result.error._tag).toBe("AnalysisCancelledError"); + expect(f.calls()).toBe(0); +}); + +it("retains a completed observation when cancellation arrives at the service boundary", async () => { + const f = await fixture(); + const input = { + path: f.path, + fields: ["ip.addr"], + display_filter: null, + decode_as: [], + packet_limit: null, + two_pass: true, + match_offset: 0, + match_limit: 50, + }; + const execution = await f.service.provider.inspect(input); + if (!execution.ok) throw execution.error; + const service = new PacketCaptureService({ + inspect: async () => { + f.controller.abort(); + return execution; + }, + close: async () => {}, + }); + const result = await service.inspect(input, { signal: f.controller.signal }); + if (result.ok) throw new Error("Expected late cancellation"); + expect(result.error._tag).toBe("AnalysisCancelledError"); + expect(result.error.partialObservation).toMatchObject({ + operation: "inspect_packet_capture", + normalized_result: { + packets: [ + { "frame.number": ["1"], "ip.addr": ["192.0.2.1", "192.0.2.2"] }, + ], + }, + }); +}); + +it.each([ + ["workspace-full", "AnalysisResourceConstraintError"], + ["snapshot-write-failure", "ProviderAdapterError"], +])( + "reports %s against the private workspace, not the selected input", + async (mode, tag) => { + const f = await fixture(mode); + const result = await f.service.inspect({ + path: f.path, + fields: ["ip.addr"], + }); + if (result.ok) throw new Error("Expected workspace failure"); + expect(result.error._tag).toBe(tag); + if (result.error._tag === "ProviderAdapterError") + expect(result.error).toMatchObject({ + diagnostics: { + stage: "workspace", + system_code: "EEXIST", + }, + }); + if (result.error._tag === "AnalysisResourceConstraintError") + expect(result.error).toMatchObject({ + reportedLimits: { + system_code: "ENOSPC", + }, + }); + expect(f.calls()).toBe(mode === "workspace-full" ? 0 : 1); + expect( + (await readdir(f.directory)).filter((name) => + name.startsWith("decoder-"), + ), + ).toEqual([]); + }, +); + +it("refuses unsupported hosts before launching or allocating a workspace", async () => { + const provider = new TsharkCaptureProvider( + { REA_TSHARK_COMMAND: process.execPath }, + async () => { + throw new Error("Must not launch"); + }, + async () => { + throw new Error("Must not allocate"); + }, + "darwin", + ); + const result = await new PacketCaptureService(provider).inspect({ + path: "/recording.pcap", + fields: ["ip.addr"], + }); + if (result.ok) throw new Error("Expected host refusal"); + expect(result.error).toMatchObject({ + _tag: "AnalysisCapabilityUnavailableError", + reason: expect.stringContaining("Linux"), + }); +}); + +it("preserves complete Evidence when aborted during post-run integrity checks", async () => { + const f = await fixture("post-run-abort"); + const result = await f.service.inspect( + { path: f.path, fields: ["ip.addr"] }, + { signal: f.controller.signal }, + ); + if (result.ok) throw new Error("Expected late cancellation"); + expect(result.error._tag).toBe("AnalysisCancelledError"); + expect(result.error.partialObservation).toMatchObject({ + normalized_result: { + packets: [ + { "frame.number": ["1"], "ip.addr": ["192.0.2.1", "192.0.2.2"] }, + ], + diagnostics: "producer warning\n", + }, + }); + expect( + (await readdir(f.directory)).filter((name) => name.startsWith("decoder-")), + ).toEqual([]); +}); +it("preserves configured symlink paths separately from resolved tool identities", async () => { + const f = await fixture("symlink"); + const result = await f.service.inspect({ path: f.path, fields: ["ip.addr"] }); + if (!result.ok) throw result.error; + expect(result.value.normalized_result).toMatchObject({ + tool: { + configured_path: join(f.directory, "configured-tool"), + path: join(f.directory, "producer.mjs"), + }, + resource_limiter: { + configured_path: join(f.directory, "configured-tool"), + path: join(f.directory, "producer.mjs"), + }, + }); +}); + +it.each(["oversized-copy", "oversized-tool"])( + "identifies the mutated %s without blaming the admitted source", + async (mode) => { + const f = await fixture(mode); + const result = await f.service.inspect({ + path: f.path, + fields: ["ip.addr"], + }); + if (result.ok) throw new Error("Expected changed artifact"); + expect(result.error).toMatchObject({ + _tag: "AnalysisArtifactChangedError", + path: + mode === "oversized-tool" + ? join(f.directory, "producer.mjs") + : expect.stringMatching(/decoder-.*[/\\]capture$/u), + }); + expect( + (await readdir(f.directory)).filter((name) => + name.startsWith("decoder-"), + ), + ).toEqual([]); + }, +); +it("preserves termination signal and effective file ceiling as a resource failure", async () => { + const f = await fixture("file-limit"); + const result = await f.service.inspect({ path: f.path, fields: ["ip.addr"] }); + if (result.ok) throw new Error("Expected resource failure"); + expect(result.error).toMatchObject({ + _tag: "AnalysisResourceConstraintError", + resource: "file-size", + reportedLimits: { file_bytes: 1048576 }, + capturedOutput: { signal: "SIGXFSZ", stderr: "before file limit\n" }, + }); +}); + +it("rejects oversized selector lists and UTF-8 argv expansion before provider work", async () => { + const f = await fixture(); + for (const selections of [ + { fields: Array.from({ length: 1025 }, () => "ip.addr") }, + { fields: ["x".repeat(8193)] }, + { fields: ["x".repeat(8192)], display_filter: "界".repeat(8192) }, + { + fields: ["ip.addr"], + decode_as: Array.from({ length: 1025 }, () => "udp.port==5555,dns"), + }, + ]) { + const result = await f.service.inspect({ path: f.path, ...selections }); + if (result.ok) throw new Error("Expected invalid selectors"); + expect(result.error._tag).toBe("AnalysisInputError"); + } + expect(f.calls()).toBe(0); +}); diff --git a/tests/boundary/captures/packetOutput.test.ts b/tests/boundary/captures/packetOutput.test.ts new file mode 100644 index 0000000000..b51cd1f670 --- /dev/null +++ b/tests/boundary/captures/packetOutput.test.ts @@ -0,0 +1,41 @@ +import { expect, it } from "vitest"; +import { parsePacketOutput } from "../../../src/captures/parsePacketOutput.js"; + +it("preserves repeated field values and missing fields while discarding export index metadata", () => { + const output = JSON.stringify([ + { + _index: "packets-2026", + _score: null, + _source: { + layers: { + "frame.number": ["2"], + "ip.addr": ["192.0.2.1", "192.0.2.2"], + }, + }, + }, + { _source: { layers: { "frame.number": ["7"] } } }, + ]); + expect(parsePacketOutput(output, ["ip.addr"])).toEqual([ + { "frame.number": ["2"], "ip.addr": ["192.0.2.1", "192.0.2.2"] }, + { "frame.number": ["7"] }, + ]); +}); + +it.each( + [ + [{ "ip.addr": ["192.0.2.1"] }], + [{ "frame.number": ["1"], "ip.addr": "192.0.2.1" }], + [{ "frame.number": ["0"] }], + [{ "frame.number": ["1", "2"] }], + [{ "frame.number": ["1"], "unselected.field": ["hidden"] }], + [{ "frame.number": ["2"] }, { "frame.number": ["2"] }], + [{ "frame.number": ["2"] }, { "frame.number": ["1"] }], + ].map((layers) => [layers]), +)("rejects malformed, repeated or reordered producer frames: %j", (layers) => { + expect(() => + parsePacketOutput( + JSON.stringify(layers.map((value) => ({ _source: { layers: value } }))), + ["ip.addr"], + ), + ).toThrow(); +}); diff --git a/tests/boundary/captures/resourceLimits.test.ts b/tests/boundary/captures/resourceLimits.test.ts new file mode 100644 index 0000000000..c8f1bf2a68 --- /dev/null +++ b/tests/boundary/captures/resourceLimits.test.ts @@ -0,0 +1,156 @@ +import { execFile } from "node:child_process"; +import { promisify } from "node:util"; +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect, it, onTestFinished } from "vitest"; +import { TsharkCaptureProvider } from "../../../src/captures/TsharkCaptureProvider.js"; +import { PacketCaptureService } from "../../../src/application/captures/PacketCaptureService.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +// Real kernel-limit verification with a synthetic producer; real TShark has its own verifier. +it.skipIf(process.platform !== "linux")( + "enforces both address-space limits before any producer allocation", + async () => { + const directory = await createTestTempDirectory("rea-tshark-limits-"); + const command = join(directory, "memory-probe"); + await writeFile( + command, + `#!/usr/bin/python3 +import json, resource, sys +assert resource.getrlimit(resource.RLIMIT_AS) == (1073741824, 1073741824) +assert resource.getrlimit(resource.RLIMIT_FSIZE) == (67108864, 67108864) +assert resource.getrlimit(resource.RLIMIT_CORE) == (0, 0) +if '--version' in sys.argv: + print('TShark (Wireshark) 4.6.9 (resource fixture)') +else: + try: + bytearray(1073741825) + except MemoryError: + print(json.dumps([{'_source': {'layers': {'frame.number': ['1']}}}])) + else: + raise RuntimeError('allocation unexpectedly succeeded') +`, + { mode: 0o700 }, + ); + const path = join(directory, "recording.pcap"); + await writeFile(path, "selected fixture"); + const service = new PacketCaptureService( + new TsharkCaptureProvider({ + ...process.env, + REA_TSHARK_COMMAND: command, + }), + ); + onTestFinished(() => service.close()); + const result = await service.inspect({ path, fields: ["frame.number"] }); + if (!result.ok) throw result.error; + expect(result.value.normalized_result).toMatchObject({ + packets: [{ "frame.number": ["1"] }], + limits: { address_space_bytes: 1073741824 }, + }); + }, +); + +it.skipIf(process.platform !== "linux")( + "preserves lower inherited file/CPU limits in the real launch", + async () => { + const directory = await createTestTempDirectory( + "rea-tshark-inherited-limits-", + ); + const command = join(directory, "limit-probe"); + const path = join(directory, "recording.pcap"); + await writeFile(path, "selected fixture"); + await writeFile( + command, + `#!/usr/bin/python3 +import json, resource, sys +assert resource.getrlimit(resource.RLIMIT_FSIZE) == (1048576, 1048576) +assert resource.getrlimit(resource.RLIMIT_CPU) == (20, 20) +if '--version' in sys.argv: + print('TShark (Wireshark) 4.6.9 (resource fixture)') +else: + print(json.dumps([{'_source': {'layers': {'frame.number': ['1']}}}])) +`, + { mode: 0o700 }, + ); + const runner = join(directory, "runner.mjs"); + const serviceModule = new URL( + "../../../dist/application/captures/PacketCaptureService.js", + import.meta.url, + ).href; + const providerModule = new URL( + "../../../dist/captures/TsharkCaptureProvider.js", + import.meta.url, + ).href; + await writeFile( + runner, + `import { PacketCaptureService } from ${JSON.stringify(serviceModule)}; +import { TsharkCaptureProvider } from ${JSON.stringify(providerModule)}; +const service = new PacketCaptureService(new TsharkCaptureProvider({ ...process.env, REA_TSHARK_COMMAND: ${JSON.stringify(command)} })); +try { + const result = await service.inspect({ path: ${JSON.stringify(path)}, fields: ['frame.number'] }); + if (!result.ok) throw result.error; + console.log(JSON.stringify(result.value.normalized_result.limits)); +} finally { await service.close(); } +`, + ); + const { stdout } = await promisify(execFile)( + "/usr/bin/prlimit", + [ + "--fsize=1048576:2097152", + "--cpu=20:25", + "--", + process.execPath, + runner, + ], + { timeout: 30000, maxBuffer: 65536 }, + ); + expect(JSON.parse(stdout)).toMatchObject({ + file_bytes: 1048576, + cpu_seconds: 20, + }); + }, +); + +it.skipIf(process.platform !== "linux")( + "reports a real kernel file-limit termination with its effective ceiling", + async () => { + const directory = await createTestTempDirectory("rea-tshark-file-limit-"); + const command = join(directory, "file-probe"); + const path = join(directory, "recording.pcap"); + await writeFile(path, "selected fixture"); + await writeFile( + command, + `#!/usr/bin/python3 +import signal, sys +if '--version' in sys.argv: + print('TShark (Wireshark) 4.6.9 (resource fixture)') +else: + signal.signal(signal.SIGXFSZ, signal.SIG_DFL) + print('before file limit', file=sys.stderr, flush=True) + with open('overflow', 'wb', buffering=0) as output: + output.write(b'x' * 4096) + output.write(b'x') +`, + { mode: 0o700 }, + ); + const service = new PacketCaptureService( + new TsharkCaptureProvider( + { ...process.env, REA_TSHARK_COMMAND: command }, + undefined, + undefined, + process.platform, + undefined, + async (limits) => ({ ...limits, file_bytes: 4096 }), + ), + ); + onTestFinished(() => service.close()); + const result = await service.inspect({ path, fields: ["frame.number"] }); + if (result.ok) throw new Error("Expected file-limit failure"); + expect(result.error).toMatchObject({ + _tag: "AnalysisResourceConstraintError", + resource: "file-size", + reportedLimits: { file_bytes: 4096 }, + capturedOutput: { signal: "SIGXFSZ", stderr: "before file limit\n" }, + }); + }, +); diff --git a/tests/boundary/cli/captureCleanup.test.ts b/tests/boundary/cli/captureCleanup.test.ts new file mode 100644 index 0000000000..1ec6adeddd --- /dev/null +++ b/tests/boundary/cli/captureCleanup.test.ts @@ -0,0 +1,157 @@ +import { Cli } from "incur"; +import { expect, it } from "vitest"; +import { registerCaptureCommands } from "../../../src/cli/captureCommands.js"; +import { silentLogger } from "../../../src/logger.js"; +import { createEvidence } from "../../../src/domain/evidence.js"; +import { ProviderCleanupError } from "../../../src/domain/providerCleanupError.js"; +import { err } from "../../../src/domain/result.js"; + +it.each([false, true])( + "awaits cleanup retry and preserves completed output when retry failure is %s", + async (failAgain) => { + const evidence = createEvidence( + undefined, + { id: "fixture", name: "Completed capture fixture", version: "1" }, + { + operation: "inspect_packet_capture", + parameters: {}, + result: { packets: [{ "frame.number": ["1"] }] }, + confidence: "observed", + }, + ); + const original = new ProviderCleanupError( + "tshark-capture", + ["original-workspace"], + { reason: "original cleanup failure" }, + { operation: "inspect_packet_capture", partialObservation: evidence }, + ); + let closed = false; + const cli = Cli.create("capture-cleanup-test", { sync: false }); + registerCaptureCommands( + cli, + silentLogger, + {}, + { + inspect: async () => err(original), + close: async () => { + await Promise.resolve(); + closed = true; + if (failAgain) + throw new ProviderCleanupError( + "tshark-capture", + ["retry-workspace"], + { reason: "retry failed" }, + { operation: "inspect_packet_capture" }, + ); + }, + }, + ); + let stdout = ""; + await cli.serve( + [ + "inspect-packet-capture", + "recording.pcap", + "--fields", + "ip.addr", + "--format", + "json", + ], + { + stdout: (text) => { + expect(closed).toBe(true); + stdout += text; + }, + exit: () => {}, + }, + ); + const output: unknown = JSON.parse(stdout); + expect(output).toMatchObject({ + code: "cleanup_incomplete", + details: { + partial_observation: evidence, + resources: expect.arrayContaining(["original-workspace"]), + }, + }); + if (failAgain) + expect(output).toMatchObject({ + details: { resources: expect.arrayContaining(["retry-workspace"]) }, + }); + }, +); + +it("rejects an oversized selector budget before the CLI invokes its service", async () => { + const cli = Cli.create("capture-selector-test", { sync: false }); + let called = false; + registerCaptureCommands( + cli, + silentLogger, + {}, + { + inspect: async () => { + called = true; + throw new Error("Must not inspect"); + }, + close: async () => {}, + }, + ); + let stdout = ""; + await cli.serve( + [ + "inspect-packet-capture", + "recording.pcap", + "--fields", + "x".repeat(8192), + "--display_filter", + "界".repeat(8192), + "--format", + "json", + ], + { + stdout: (text) => { + stdout += text; + }, + exit: () => {}, + }, + ); + expect(called).toBe(false); + expect(stdout).toContain("32 KiB"); +}); + +it("rejects conflicting output options before the CLI invokes its service", async () => { + const cli = Cli.create("capture-window-test", { sync: false }); + let called = false; + registerCaptureCommands( + cli, + silentLogger, + {}, + { + inspect: async () => { + called = true; + throw new Error("Must not inspect"); + }, + close: async () => {}, + }, + ); + let stdout = ""; + await cli.serve( + [ + "inspect-packet-capture", + "recording.pcap", + "--fields", + "ip.addr", + "--all_matches", + "--match_limit", + "20", + "--format", + "json", + ], + { + stdout: (text) => { + stdout += text; + }, + exit: () => {}, + }, + ); + expect(called).toBe(false); + expect(stdout).toContain("mutually exclusive"); +}); diff --git a/tests/boundary/mcp/packetCaptureMcp.test.ts b/tests/boundary/mcp/packetCaptureMcp.test.ts new file mode 100644 index 0000000000..3d7322baf8 --- /dev/null +++ b/tests/boundary/mcp/packetCaptureMcp.test.ts @@ -0,0 +1,142 @@ +import { createEvidence } from "../../../src/domain/evidence.js"; +import { AnalysisCancelledError } from "../../../src/domain/analysisErrorCore.js"; +import { ProviderCleanupError } from "../../../src/domain/providerCleanupError.js"; +import { PacketCaptureService } from "../../../src/application/captures/PacketCaptureService.js"; +import { err } from "../../../src/domain/result.js"; +import { expect, it, onTestFinished } from "vitest"; +import { Ajv2020 } from "ajv/dist/2020.js"; +import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; + +it("advertises valid target-free packet capture schemas and actionable missing-tool diagnostics", async () => { + const harness = await createApplicationMcpHarness({ + serverOptions: { environment: {} }, + }); + onTestFinished(harness.close); + const tool = (await harness.client.listTools()).tools.find( + ({ name }) => name === "inspect_packet_capture", + ); + if (tool?.outputSchema === undefined) throw new Error("Capture tool missing"); + const ajv = new Ajv2020({ strict: false, validateFormats: false }); + const inputSchema: Record = tool.inputSchema; + const outputSchema: Record = tool.outputSchema; + expect(ajv.validateSchema(inputSchema)).toBe(true); + expect(ajv.validateSchema(outputSchema)).toBe(true); + const input = { + path: + process.platform === "win32" ? "C:\\recording.pcap" : "/recording.pcap", + fields: ["ip.addr"], + }; + expect(ajv.validate(inputSchema, input)).toBe(true); + expect(ajv.validate(inputSchema, { ...input, interface: "lo0" })).toBe(false); + const result = await harness.client.callTool({ + name: tool.name, + arguments: input, + }); + expect(parseMcpToolError(result)).toMatchObject({ + error: { + code: "capability_unavailable", + details: { + operation: tool.name, + reason: expect.stringContaining("REA_TSHARK_COMMAND"), + }, + }, + }); + await harness.client.ping(); +}); + +it.each(["cancelled", "cleanup"])( + "retains completed Evidence after %s failure", + async (kind) => { + const input = { + path: + process.platform === "win32" ? "C:\\recording.pcap" : "/recording.pcap", + fields: ["ip.addr"], + }; + const evidence = createEvidence( + undefined, + { id: "fixture", name: "Completed capture fixture", version: "1" }, + { + operation: "inspect_packet_capture", + parameters: input, + result: { packets: [{ "frame.number": ["1"] }] }, + confidence: "observed", + }, + ); + const failure = + kind === "cancelled" + ? new AnalysisCancelledError("inspect_packet_capture", { + partialObservation: evidence, + }) + : new ProviderCleanupError( + "tshark-capture", + ["private-workspace"], + {}, + { + operation: "inspect_packet_capture", + partialObservation: evidence, + }, + ); + const packetCapture = new PacketCaptureService({ + inspect: async () => err(failure), + close: async () => {}, + }); + const harness = await createApplicationMcpHarness({ + serverOptions: { packetCapture }, + }); + onTestFinished(harness.close); + const response = await harness.client.callTool({ + name: "inspect_packet_capture", + arguments: input, + }); + expect(response.isError).toBe(true); + expect(parseMcpToolError(response)).toMatchObject({ + error: { + details: { + partial_observation: { + kind: "retained-evidence", + evidence_id: evidence.evidence_id, + }, + }, + }, + }); + const bundle = await harness.client.callTool({ + name: "get_evidence_bundle", + arguments: {}, + }); + expect(bundle.isError).not.toBe(true); + expect(JSON.stringify(bundle.structuredContent)).toContain( + evidence.evidence_id, + ); + expect(harness.session.evidenceById(evidence.evidence_id)).toEqual( + evidence, + ); + }, +); + +it("rejects oversized selector expansion through the MCP boundary before invoking the provider", async () => { + let calls = 0; + const packetCapture = new PacketCaptureService({ + inspect: async () => { + calls++; + return err(new AnalysisCancelledError("inspect_packet_capture")); + }, + close: async () => {}, + }); + const harness = await createApplicationMcpHarness({ + serverOptions: { packetCapture }, + }); + onTestFinished(harness.close); + const response = await harness.client.callTool({ + name: "inspect_packet_capture", + arguments: { + path: + process.platform === "win32" ? "C:\\recording.pcap" : "/recording.pcap", + fields: ["x".repeat(8192)], + display_filter: "界".repeat(8192), + }, + }); + expect(response.isError).toBe(true); + expect(JSON.stringify(response)).toContain("32 KiB"); + expect(calls).toBe(0); +});