mirror of
https://github.com/morluto/rea.git
synced 2026-11-07 07:16:13 +00:00
7b6fc6f13de8cde5dfdca0b44f78b9537ddca8da
28
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Edgar ArakelyanandClaude Opus 5.5
|
970969cb53 |
feat(desktop): capture and export apps installed from a downloaded .deb (#8954)
* feat(desktop): capture a Linux desktop application in use Add `capture-desktop-application`, a CLI-only, Linux and X11 capture that launches an installed desktop application, watches it while it is used and writes capture Evidence plus SHA-256 addressed content: - inventory: desktop entry, launch command, executable, package and its apt sources (or snap, flatpak, AppImage), Electron markers and candidate state directories; - session: the launched process tree, client windows from xprop with the first application window time and a window match, and files held open; - state: a stat walk of the home directory before launch, at checkpoints read from stdin, at stop and after exit, with created, modified, deleted and retyped paths and the content of changed files and state directories. It refuses non-Linux hosts, missing DISPLAY or xprop, a non-empty output folder and an application whose window is already open, and signals only processes it started. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): own launched processes, confirm walk coverage and keep cancelled captures - Launch through the owned provider path with a run token; release with SIGTERM, then the token-verified group cleanup, after every session. Processes without the token are never signalled; each records whether it carries it, and the session records whether cleanup was verified. - Report creations and deletions only where a walk listed the nearest existing directory; count the rest as unconfirmed. - Retain a file only when it is still the walked version, through owned no-follow reads; charge the budget only for new content; create files exclusively in a private output folder owned by the caller. - Walk content roots outside the home directory, compare paths in their physical spelling, and run executables from the caller's directory. - Refuse when the window list cannot be read, re-evaluate window ownership each sample, handle one queued request per sample with a bounded stdin queue, and write the capture with end reason `cancelled` on interruption. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep owned cleanup failures and fresh digests - Allow a capture cancelled before any requested checkpoint to hold only the baseline. - Retry an incomplete stop once; if cleanup is still unverified, write the capture and fail with the run token and process group as cleanup resources. - Classify path changes after a walk (ENOTDIR and other selected-file errors) as changed_since_walk instead of failing the capture. - Treat a non-directory ancestor as proof that nothing existed beneath it. - Report a before digest only for the file version it was read from. - Derive stderr_truncated from stderr's own byte counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * wip(desktop): state view over a capture folder Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep failed file-handle closes owned for retry Content reads pass retainFailedClose, so a handle whose close fails stays owned by the content store; the capture retries those closes after it is written and fails with the output folder as the cleanup resource if one still cannot be closed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(desktop): read a captured application's state Add inspect-desktop-application-state, a CLI command that reads a capture folder into the application's state: every retained or changed file with its role and reasons, a typed JSON view of each version (SQLite through the shared worker, dconf through a private dconf dump, JSON, XML, key files, text) and the kinds of change between versions (lines, values, keys, rows). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): report process and handle cleanup failures together When both the launched processes and a retained file's handle remain unresolved after the capture is written, one cleanup error now lists both reasons and both resource sets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(desktop): export a captured application as an agentenv computer app Add export-desktop-application, which writes an agentenv bundle with one computer_app environment from a capture folder and its state view: the apt package, a boot launch matched on the captured window class with a window ready check, owned paths from the state roles, a seed of the baseline files, data hooks that call agentenv's add/get/reset helpers with a typed JSON view on get, a skill, a README and a deploy smoke task. The desktop CLI commands now load their services lazily. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep state views faithful to the capture - Read key files, XML attributes, dconf groups and SQLite tables into dictionaries without inherited properties, so captured keys such as __proto__ stay data. - Version databases with their write-ahead log as of each checkpoint, and stage only digest-verified bytes for the SQLite and dconf readers. - Detect the format per version, report format changes, keep JSON null and empty containers as values, keep XML text and preserved whitespace, and mark truncated views (including XML depth) and comparisons of them. - Number versions and changes by checkpoint index, decide baseline presence from the captured type, and keep tagged SQLite integers, text bytes and blobs distinct. - Carry the capture's limitations and omitted content forward, match state directories by their home path, index changes once per capture, keep failed handle closes owned, and finish the CLI through the shared owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep coverage, logs and handles honest in the state view - Read coverage flags only from formats whose readers write them, so JSON fields named truncated stay data. - Fail a SQLite version whose write-ahead log existed but was not retained or does not match its digest, instead of reading the database alone. - Close retained content on every exit, including cancellation, and report a close that still fails together with the primary failure. - Finish the CLI with the canonical operation name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): export only what the computer can install and seed - Refuse packages not from the Ubuntu 24.04 archive, since the computer installs by name from that archive. - Start the launch command in the captured working directory. - Respell the home path only in files the state view read as text. - Report files that were not retained instead of dropping them silently. - Load captured dconf keys for apps that own no files. - Keep home-level files as owned paths; rename reserved Python class names. - Mark truncated SQLite and text views in the class's get view, and read file types the way the state view does. - Release the output tree through a resource scope, and read seeds through an owner closed on every exit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep staged folders owned, symlinked state roots and XML values - Stage database bytes in a private runtime root released through a resource scope, so a failed removal keeps its owner and is reported with the read's outcome. - Report cleanup facts and owners from desktop failures instead of wrapping them as a plain adapter error. - Record the state directories as walked, in their physical spelling, and match roles against both spellings, so files in a symlinked state directory are not mistaken for selected documents. - Keep whitespace-only XML leaf values; drop only formatting between elements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): read XML in the exported app the way the state view does Keep whitespace-only leaf values and xml:space="preserve" text in the generated get view; drop only formatting between elements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): release retained handles with their owners, read older captures - Release retained-content handles whose close failed through a resource scope on every exit; a release that fails again reports the exact handle owners. A read that finished keeps its bytes when only its close failed. - Accept captures written before state directories were recorded, and say in the state view's limitations that symlinked state directories may be misclassed for them. - Compute each XML element's whitespace context once instead of scanning siblings for every text run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): boot without the working folder, bound XML, accept mirrors - Try the captured working directory only when an argument is relative, and start the app even when that folder is missing. - Cut the generated XML view at REA's node and depth limits, and keep one unreadable file from failing data/get. - Accept Ubuntu 24.04 mirrors, not only the canonical hosts; PPAs and other repositories are still refused. - Read seeds through the retained-content owner's new signature. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): state view review round - Stage a retained -journal beside its database so the SQLite worker accepts an inactive journal and refuses a hot one; a -wal or -journal that existed but was not retained (including one omitted at the baseline) makes the version unreadable. - Keep tables that read when another fails, and let cancellation and unfinished cleanup propagate instead of becoming view errors. - Read text as a key file only when every line parses as one. - Rank a file named in a window title or selected by the caller as a document even when the application holds it open; match titles on file-name boundaries; leave the display name out of dconf matching. - Bound JSON and key file views (4 MiB, 500 levels, exact numbers) and report too-large, missing and altered retained content separately. - Add no version or change while a database stays absent. - Read staged databases in place instead of copying them again. - Give Evidence the same limitations as the result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): export review round - Load captured dconf keys without D-Bus: install dconf-cli, compile the existing and captured key files into the user database, and warn on failure instead of stopping the server. - Start relative-argument launches in the captured working directory, the home included, since the framework launches from the app folder; respell the home itself and option values under it. - Read views in a thread, skip files over 64 MiB, align the generated readers with the state view's, and close SQLite connections. - Never own the home or a shared base folder; map a symlinked state directory's files back to their home spelling. - Report omitted files and unretained opened folders. - Name the module so it cannot shadow another, and keep the class off the module's own names; derive names the bundle check accepts. - Write seeds as they are read, and remove an unpublished bundle on failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): capture review round - Ask the launch process group and token carriers to exit before killing them, on errors too; report incomplete cleanup with its facts. - Count descendants of the launch as started by the capture even when their environment cannot be read; treat a reused pid as a new process. - Follow symlinked desktop entries; name a flatpak's state after its application id; match flatpak windows by name only. - Stop on end of input; refuse empty, duplicate and reserved checkpoint names; require whitespace after the checkpoint keyword. - Stay on each walk root's device, and never walk or retain `/` or a directory containing the home; say so in the limitations. - Refuse a window manager that does not publish _NET_CLIENT_LIST. - Remove retained content when a capture fails before it is written. - Skip reading a file larger than the remaining content budget. - Record apt's archive origin for each package source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): read an unset client list as no windows when supported xfwm4 leaves _NET_CLIENT_LIST unset while it manages no windows, so the capture refused an ordinary desktop with nothing open. Read _NET_SUPPORTED with the list: an unset list means no windows when the window manager supports it, and the capture is refused only when it does not. `xprop`'s "no such atom" output now reads as unset too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): accept packages by apt's archive origin Accept a package source when apt reports its origin as Ubuntu for a noble suite, so mirrors count and PPAs or other repositories that merely have "ubuntu" in their URL do not. Captures made before origins were recorded fall back to Canonical's own archive hosts. Seed reads report why retained content could not be used. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(desktop): capture and export apps from vendor apt repositories A closed-source app installed from its vendor's apt repository, such as Typora, can now be captured and exported as a computer app. - The capture records each non-Ubuntu repository listing the installed version (uri, suite, components from apt's sources files, one-line or deb822) and the key that signs it: the signed-by key, or the trusted key that gpgv shows verifies the repository's signed index. The key is retained with the capture. - The capture records the packages owning the libraries the executable loads (DT_NEEDED with RUNPATH and the linker cache, through REA's ELF readers), since a vendor package may not depend on them. - The export declares a verified vendor repository as an AptSource with its key, installs a vendor package's library packages, and adds --no-sandbox to an Electron app's launch, since a container denies Chromium's sandbox. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): report a locked database at once in data/get A database the running app holds locked made each data/get wait sqlite's five-second busy timeout before reporting it. Record the Typora run in an agentenv computer in the verification. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(desktop): detect key files without splitting the whole text Read lines one at a time and stop at the first that a key file view would drop, instead of splitting a file of up to 64 MiB into arrays before its format is known. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep symlinked homes and state folders under the home in exports - Respell a path under its state folder's logical spelling only when that spelling is under the captured home, so a symlinked HOME keeps its files; pair walked and logical state folders only when the capture lists the same number, and match the innermost folder first. - When two files map to one home path, keep the one present before the capture and name the other in the limitations. - Read key files in the generated data/get with the state view's rule, checked against it by running the helper under python3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): end captures on zombies and keep cleanup owners - A zombie or dead process no longer keeps the session open. - Remove only the partial capture file this run created. - Keep the exact owners of handles and processes whose cleanup failed on the returned error, so a retry can release them; test the published capture's cleanup-failure branch with mixed outcomes and retries. - Release the application once even when the exit checkpoint fails. - A cancellation before the capture is written removes what it retained. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): vendor package review round - Match apt sources through `apt-get indextargets`, which covers flat `./` and `/` suites and gives each release file and sources entry. - Accept a key whose signature verifies even when the release carries another signature without a key here; keep only the verifying key from a keyring holding several (OpenPGP packet splitter). - Install the package from its repository's release (`name/<release>`), declare one source per URI and suite, support only the capture host's architecture, and refuse Ubuntu Pro repositories. - List only library packages the computer can install; walk libraries only for vendor installs, with RPATH inherited as ld.so does, normalized paths, and DT_NEEDED bounded while reading. - Put --no-sandbox after the program an env wrapper launches; keep the gpgv folder's owner when its removal fails; respect --max-file-bytes for keys; wait 0.25 s on a busy SQLite database in data/get. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(desktop): keep the vendor spec's gpgv command on one line A code span broken across lines left `<binary keyring>` for VitePress to read as an unclosed HTML tag, so the docs build failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(desktop): add a validated task to exported desktop applications Each capture can now yield one agentenv task in its exported bundle. An agent writes the task (a prompt plus declarative checks over the app's data/get views); REA validates it and stays model-free. - `describe-desktop-application-task` summarises a capture's net change per file (role, format, views before and after) and drafts checks. - Checks: file presence, lines present/absent/counted, JSON-pointer values, SQLite rows and counts. REA keeps a task only if a check fails on the seed and every check passes on the captured end state; a file that was not retained at the baseline counts as absent, as the computer starts. - `export-desktop-application --task` writes tasks/NAME.json (deploy, agent, skills, recording, prompt, verifier), a compiled verify.py that reads data/get, and an eval. A boundary test runs the verifier under python3 and checks it agrees with REA's evaluator on every check kind. - Docs, spec and skill guidance, with the solve-rate recipe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(desktop): keep the seed test in its own block Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(desktop): record the task verifier driven in an agentenv computer Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(desktop): record the Typora task run in an agentenv computer Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep task checks bounded, complete and attributable - line_count counts lines holding a substring (contains) instead of matching a regular expression, which could stall the verifier on agent-written text and read differently in Python. - value_equals of a whole object or array is undecided on a truncated view. - The draft keeps a database whose rows changed under an unchanged hash, and a file whose truncated views match while it changed. - A task file that cannot be read keeps the reader's detail, and a failed cleanup keeps the read failure as its primary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): leave a file with one version out of the task draft Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(desktop): update the task verification's test counts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(desktop): record the task solve rate with Claude Code and Sonnet 5.5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(desktop): capture and export apps installed from a downloaded .deb capture-desktop-application --package-file reads the file's control fields with dpkg-deb and records its SHA-256, size and dependencies when it holds the installed package; the file itself is never kept. The library walk now also runs for local installs. export-desktop-application --package-url declares agentenv's Download(url, sha256) for that file, its dependencies and library packages as apt packages, and a run script that installs it with dpkg. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): seed app-named dotfiles and respell UTF-16 seeds The capture now treats dotfiles in the home named after the application (.varicad-cfg.ini, .notesrc) as state roots, so a file the app only reads during the capture is still seeded. The exported seeding respells the captured home in UTF-16 text files with a byte-order mark, keeping the mark. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): hold readiness until the app's own window shows An exported app is ready only once a window of its class that the taskbar lists is shown, so a splash screen of the class, which skips the taskbar, no longer counts as the app being up. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(desktop): record VariCAD captured, exported and run in an agentenv computer Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep exported views strict JSON for infinite and inexact numbers The exported data/get spells infinite and negative-zero SQLite reals as REA's SQLite reader does ("Infinity", "-Infinity", "-0"), and refuses a JSON file holding a number that reading would change (1e999, integers past 2^53, -0, rounded fractions) with the state view's own error, instead of a view with an infinity in it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep package-file facts tied to one file and URLs unechoed - A --package-url refusal never repeats the URL, and credentials are checked before the scheme. - The package file's identity must hold between dpkg-deb's read and the hash. - A failed close keeps its cleanup owner even when the capture is cancelled. - Readiness also ignores windows typed as splash screens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): record database logs and keep state changes distinct - A selected path that is not a directory brings its -wal and -journal as content roots, so the capture records whether they exist and retains them. - The state view treats a file as absent only under recorded content roots and refuses a database whose log presence is unknown. - JSON changes list containers too, so an array that becomes an object with the same members is a change. - Key file entries escape the key as a JSON pointer segment, so [a/b] c and [a] b/c stay apart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep watching through unread samples and late workers - An unreadable window list is unknown, not zero windows, and is named in the limitations. - Before ending the session, live run-token carriers no parent link reached are tracked; an unreadable process table leaves the exit unknown. - An env prefix's assignments find the program and name its XDG folders. - flatpak run options that take a value are skipped with it, -- is honoured and a /arch/branch suffix is dropped from the id. - State directories include the entry's full id. - Path options refuse empty values; roots count toward the entry limit. - A capture file whose close failed stays owned and is retried on discard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): bound apt sources, keep older captures and cleanup owners - Sources files parse into one stanza per line or deb822 paragraph with its URIs and suites kept as lists, so no URI x suite product is built. - Repository architectures and release are optional, so captures written before they were recorded parse; the export treats them as unrestricted and unknown. - Library sources are matched against every index of a declared repository's URI, suite and components. - Two failures side by side keep both causes and both cleanup owners, and a host file whose close fails reports a lease on its handle. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): match escaped key file entries and suggest containers through members - The draft's key file lookup uses the state view's escaped `group/key`, so a key holding `/` or `~` still gets its suggestion. - JSON and XML changes now list containers too; the draft suggests an added or removed container through its listed members, and a container only when none of its members is listed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): read relative PATH entries from the launch folder A program, or an empty or relative PATH entry, is resolved from the folder the program starts in, as the launch does, so `env PATH=. app` with the entry's Path still finds the app. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(desktop): compare JSON views on one shared path JSON and XML changes are found by walking both trees with one shared pointer path, building a pointer only for a listed value, so memory follows depth rather than every container's full path. A pointer over 4096 characters is counted but not listed. A 256-level document with 4000-character keys went from about 251 MiB of heap to under 1 MiB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep JSON Lines as text in the exported reader The exported reader parses the whole text before an inexact number makes it JSON with an error, so JSON Lines with an integer above 2^53 stay text as in REA. Over-long integer spellings are unsafe without converting them. The guide no longer claims large integers keep their value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep the state-path help free of dash-led words The CLI alias check reads `-wal` and `-journal` in help text as short flags, so the help names the write-ahead log and rollback journal instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): never record or export apt source credentials - Source URIs lose any user name and password as they are parsed, the way apt splits them, and so do index-target URIs and descriptions and the sources apt-cache policy lists, so matching works whether or not apt prints them. - A repository whose entry held credentials is marked `uri_credentials_removed`, and the export refuses it, or an older capture whose URI still holds them, with a reason that names no URL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): look programs up without normalizing their paths Program paths and PATH entries are joined rather than resolved, so `..` after a symbolic link means what it means to the launch. The lookup moves to its own module. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): note inexact JSON numbers with a flag, not a list The exported reader only needs to know whether any number was inexact, so it keeps a boolean instead of every rejected spelling. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): refuse older captures that still hold apt credentials A capture written before credentials were removed counted the credentialed repository's libraries as installable, so the export refuses it whole and asks for a new capture rather than declaring packages no source offers. Recorded policy sources are redacted in export messages too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(desktop): keep the credential fixture on an example host A login on a real host reads as a secret to the commit scanner, so the older capture's fixture uses apt.example.com like the other credential tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): tidy looked-up paths and search relative entries only for env Empty and `.` segments are dropped from a looked-up program path, which keeps `/snap/bin/` entries recognisable as snaps, while `..` is still left for the kernel. Relative and empty PATH entries are searched only in the PATH an `env` prefix sets, which travels with the exported launch; the host's own PATH does not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): keep excluded and unwalked database logs unknown The capture records the paths every walk left out, and the state view treats a log as absent only under content roots the baseline walked completely and outside those exclusions, so an excluded or unvisited log leaves the database unread rather than read without it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): judge JSON numbers after duplicate keys settle The exported reader marks an inexact number instead of flagging it during the parse, then looks for marks in the parsed value, so a number a later duplicate key overwrote no longer refuses the file, as in REA. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): redact a repository's policy source when it names it An older capture whose policy source matched no sources entry kept that text whole, and the export named the repository by it; the export now names it without any user name or password. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): never signal the launch group on unknown membership When the current read cannot observe a carrier's process group, the carrier is signalled on its own instead of letting an older process-table row grant a group-wide SIGTERM. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): look programs up in the launch's order and refuse host-relative finds The lookup searches PATH as the launch does, an empty entry meaning the launch folder, so inspection and launch name the same program. A program found only through a relative or empty entry of the host's own PATH is refused, since the exported computer would not have it; an env prefix's PATH travels with the command. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): count unchecked paths and keep logs behind them unknown Each walk counts the paths it could not stat and the directories on another device it did not enter, and the checkpoint records it as `unchecked_paths`. The state view treats a log as absent only when the baseline left nothing unchecked, so a log whose stat failed stays unknown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(desktop): scan parsed JSON with one iterator per level The check for marked numbers walks the parsed value with an iterator per nesting level instead of copying each container's children; a million-entry array went from about 15.7 MiB to 8.1 MiB of peak traced memory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(desktop): start a relative program from its captured folder An exported launch whose program path is relative, such as `./app` from an entry's Path, now changes to the captured working directory first, as it already did for relative arguments. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
5990ff0324 |
feat(artifacts): read WeChat mini-program wxapkg containers (#8148)
* feat(artifacts): read WeChat mini-program wxapkg containers Add a read-only reader for plaintext wxapkg packages: the 0xBE/0xED 18-byte big-endian header, the bounded name/offset/size member table, and verbatim member bytes at absolute carrier offsets. - Members retain declared names, absolute offsets and sizes, so source_location reports real carrier positions. - V1MMWX PC-client packages are detected as encrypted coverage with null member facts; extraction refuses with a typed error and no decryption is attempted. - Bounded table and directory budgets are admitted before allocation; malformed marks, truncated tables, invalid UTF-8 names, member extents inside the header or beyond the file, and counts that cannot fit their table are typed format or limit failures. - Member reads stream in bounded 64 KiB chunks with carrier freshness checks; the reader joins outstanding streams before close and bounds active streams and repeated member I/O. Refs #8121. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(artifacts): bound wxapkg members to the declared data region Normalize the two indexLength conventions before the table bounds check and the table read. Member ranges must then lie in that declared data region, which has to fit in the file. Record wxapkg format metadata as not compared: header facts repeat the package bytes, and entry source paths keep a producer root prefix. * fix(artifacts): choose the wxapkg index after member ranges fit Four trailing bytes made the rows-only layout end on the file and dropped the count-inclusive layout. Keep every admitted layout until the parsed rows and member ranges select one. --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> |
||
|
Beqa
|
d892e7861a |
fix(docs): drop duplicate native-inspection guide row (#7480)
Remove the incomplete Choose a guide duplicate that omitted kernel-modules.md so the docs index keeps one complete native-inspection entry. Fixes #7405 |
||
|
Entrpi
|
e6d0618979 |
feat(artifacts): inventory and extract ESP flash layouts (#8228)
* feat(artifacts): inventory and extract ESP32 firmware images ESP32-family app and bootloader images (magic 0xE9) had no reader, so open_binary refused OTA downloads, app partitions and bootloaders. Add the esp-image container format behind inspect_artifact and extract_artifact. A pure decoder reads each byte once and keeps only fixed-size fields. Derived facts cover the chip, from a table cited to ESP-IDF v6.1; segment classification as ESP-IDF's should_map and should_load; checksum and appended SHA-256 results; app and bootloader descriptors; Secure Boot v2 signature blocks, observed and unverified; and undescribed trailing byte ranges. Truncation and undescribed bytes make coverage unknown. ESP8266 images are refused with a specific reason, and ciphertext stays unrecognized. Add the format to every target, Evidence, snapshot, live-summary and artifact vocabulary, document it, and add an optional verify:esp-images lane that compares esptool-built images with esptool image-info and espsecure signature-info-v2. Refs #3615 * test(artifacts): run the ESP image lane on esptool v5.6.0 The lane compared descriptor fields only when esptool image-info printed one, so a descriptor that REA reports and esptool does not print passed without comparison. Compare the descriptor kind in both directions. Record that the lane passed for all 13 images with esptool and espsecure v5.6.0, the version the layout facts cite, as well as with 4.12.0. * refactor(artifacts): define ESP image layout sizes once The decoder and the derived facts each defined the segment-header, appended-digest and flash-sector sizes, the erased-byte scan and the offset formatter. Define them once with the other fixed-layout structures and import them in both places. The signature slot count is the one constant that drives the slot scan. Correct the observedEnd comment: it is the end of the last chunk read. * fix(artifacts): report ESP-IDF's segment length limit by chip ESP-IDF's verify_segment_header rejects a segment of 16 MiB or more unless the chip maps flash through 32-bit cache addresses (SOC_SPI_MEM_SUPPORT_CACHE_32BIT_ADDR_MAP: ESP32-S3, P4, C5, C61, H4 and S31 in v6.1). Record that capability in the chip table and add a limitation for such segments; unknown chips get none. `loaded` repeats should_load. It does not mean the bootloader accepts the segment: verify_load_addresses still rejects one outside the chip's RAM, which REA does not model. Say so in the docs, the skill reference and the inventory limitation. * test(artifacts): check ESP lane tools first and verify extraction The lane started work before checking its tools, so a missing RISC-V target, ld.lld, esptool or espsecure surfaced as a raw spawn or command error. Check each one before building and name the variable to set. Warn when esptool is not v5.6.0, the release the layout facts cite. Assert each segment's class for the ESP32-C3 app, its variants and the bootloader, as the Xtensa image already did. Extract the app and the signed image through the CLI and compare every file with the bytes at esptool's segment offsets or in the appended signature sector; the source's SHA-256 must be unchanged and equal the reported root_sha256. * docs(artifacts): state what the ESP image lane does not establish The Verification section listed the lane's checks but not its gaps: only ESP32-C3 and ESP32 images come from a real producer, no real project build, OTA download or flash dump is inspected, no signature is verified, and the lane does not run in CI. Record the timing of the final lane run on esptool v5.6.0. * fix(artifacts): compare ESP segment declarations Segment entries hash only their data, so compare_artifacts reported a segment that moved to another load address with the same bytes as unchanged; only the image itself changed. Compare each segment's load_addr and data_len as its declaration, as Node SEA and PyInstaller members are compared, and count a changed declaration as a metadata change, also when content is unavailable. Header offsets are layout and the classification is derived, so neither is compared. * test(artifacts): run ESP lane commands as owned process trees The lane ran its tools through execFile. A timeout killed only the direct child, so a linker or assembler that Clang or GCC had started could keep running while the lane removed its workspace. Run every command through createVerifierCommands, check the tools inside withVerifierCleanup, and remove the workspace only after commands.close() has stopped every tree; a failed close keeps the workspace. Only a nonzero exit now counts as a refusal or an expected esptool failure; timeouts and spawn failures propagate. Supervising each command roughly doubles the lane's run time. * test(artifacts): cancel ESP decoding after its first decoded chunk The cancellation test requested cancel or close on setImmediate, which can land while the file is still being opened and checked, so it never showed that decoding had started. It also passed against a reader that ignored both requests while decoding. Acknowledge the decoder's first chunk, then cancel or close, join the operation, and assert that no later chunk reaches the decoder and that the reader ends closed. * docs(artifacts): record which ESP metadata comparison ignores Segments are compared by their declaration. The image's and the signature blocks' format_metadata are not: every field comes from their own bytes, which content comparison already covers, and a block's image_digest_matches follows the image's content. * feat(artifacts): inventory and extract ESP flash layouts A whole-flash dump of an ESP32-family chip was read as its bootloader image, with the partition table, apps and data left as undescribed bytes, or refused when it began with erased flash. Read it as an esp-flash-image container instead. The partition table is decoded as esp_partition_table_verify accepts it and is found at the caller's partition_table_offset, at the default 0x8000, or by scanning 0x1000-aligned sectors for a table whose MD5 verifies, which the inventory reports as derived. The bootloader is located at its chip's ROM bootloader offset. Each partition is an entry with its declaration in flat format_metadata; the bootloader and app partitions carry ESP image facts from the existing streaming decoder; the otadata records are reported with a selection derived without rollback or anti-rollback, and the running partition stays unknown. One unreadable partition does not hide the others, and extraction writes every partition, nvs_keys included, as ordinary data. Flash layouts are detected before single ESP images. The scan runs only when an ESP image sits at a ROM bootloader offset, and is refused when a bootloader at another chip's offset shows that the dump does not start at flash address 0. open_binary gains flash_offset and partition_table_offset, which require format esp-flash-image and are part of the snapshot target identity; the CLI takes them with --target-format esp-flash-image. Comparison treats a partition's label, type, subtype, range and flags as its declaration. * test(artifacts): verify ESP flash layouts with gen_esp32part, merge-bin and otatool Add the optional verify:esp-flash lane. It builds default and relocated partition tables with ESP-IDF's gen_esp32part.py, assembles dumps with esptool merge-bin, including one written from flash 0x1000, and writes OTA records with otatool through flash_file_tool.py, which replaces parttool's esptool calls with reads and writes of the dump file. It compares REA's bootloader and app facts with esptool image-info, every partition with gen_esp32part.py's decoding of the extracted table, and the OTA records and derived slot with otatool read_otadata after each switch. It also checks extracted bytes and source integrity, the scan and caller table offsets, flash_offset on the 0x1000 dump, a corrupted app beside valid partitions and a truncated dump. ESP32 linker scripts place the conformance bootloader and app in ESP32 address ranges. * fix(artifacts): read ESP flash layouts as the bootloader does Read the partition table as bootloader_utility_load_partition_table does: the last factory, otadata and same-subtype OTA declaration wins, and a limitation names repeats. Plan each otadata record within its partition and the dump. An otadata partition smaller than two sectors selects none, the bootloader's INVALID_INDEX; a record the dump lacks leaves the selection unknown while the other record stays reported. The fallbacks follow the bootloader's indexes: ota_0-fallback and factory-fallback name no partition when the table declares none, and the assumptions add GPIO holds and deep-sleep wake. Refuse a dump whose bootloader contradicts its origin before the default table offset is tried, whether flash_offset was omitted or given, and keep the ESP image reader's ESP8266 refusal. A table never lies inside the located bootloader image, so single app images are no longer scanned. An explicit esp-flash-image selection is decided before other container signatures, and the native commands' --target-format no longer takes it. Say when no bootloader confirms the default origin. Exempt the primary bootloader and partition_table entries of ESP-IDF 5.3 from placement limits, report every region inside a longer one, and name partitions by index when labels repeat ignoring case. Keep an undecodable label's bytes in label_hex and compare them. Word undescribed ranges as starting at a byte that is not erased, and treat a short erased region as erased. Add an MCP compare_artifacts test for partition declarations, reader identity and close tests, scan edge cases, the 0x2000 ROM offset and a literal otatool CRC. * test(artifacts): verify ESP primary layout entries and the otadata fallback Build a table with the primary bootloader and partition_table entries that gen_esp32part.py fills from --primary-bootloader-offset, and check that REA reads it with complete coverage, no overlap or misplacement limitation, and a table that agrees with gen_esp32part's decoding. Erase the otadata with otatool after the slot switches and check the records against read_otadata and the factory fallback. * fix(artifacts): keep ESP flash coverage complete for a dump cut short An esptool merge-bin dump ends after the last part it wrote, usually inside its last partition. Such a partition made the layout's coverage unknown, so artifact comparison of two merge-bin dumps reported every unchanged partition as unknown. The partition table names every partition whatever the file's length, so a partition cut short or left out by the end of the file hides no entry. As the ESP image reader does for a cut-short segment, its entry holds the bytes the file has and an entry limitation gives the declared size; a partition with no bytes stays in partitions_outside_dump. Coverage stays complete unless undescribed bytes remain or no bootloader is located. An MCP compare_artifacts test of two dumps that end inside ota_0 now classifies every partition, and fails on the old rule. * fix(artifacts): report written bytes in ESP partitions, erased slots included An app slot is erased for ESP-IDF's bootloader when its image header bytes are erased flash, and REA reported only that status. A slot can still hold data further in: a dump read from a device held a complete older app image 1 MiB into an OTA slot reported as erased, with no limitation. Each partition entry now records where the bytes the dump holds are written, relative to the partition: written_offset, the first byte that is not 0xFF, written_end, just past the last, and written_bytes, how many are not 0xFF. They are null for a partition outside the dump. The status stays erased, which is the bootloader's view, and a limitation names an erased app slot that holds written bytes and says they are not read as an image. The reader reads every partition byte once more to find them. * fix(artifacts): derive the ESP boot selection for every partition table The boot selection lived inside otadata, so a table without an otadata partition, the common factory-only layout, had none. ESP-IDF's bootloader_utility_get_selected_boot_partition returns the factory index when the table declares no otadata partition. Move it to a top-level boot_selection that every layout has. Its basis names the bootloader's branch: otadata, factory-fallback and ota_0-fallback from the records, no-otadata for a table without otadata, none for the bootloader's INVALID_INDEX, and unknown when the dump lacks a record the bootloader reads, which replaces the null selection. running_partition moves with it, and otadata keeps its partition index and records. The assumptions name what each branch leaves out: no records assumption without otadata, and none for none or unknown. The real-tool lane checks no-otadata on a table gen_esp32part built without otadata. * fix(artifacts): say when an ESP image is a flash dump read from 0x1000 An ESP32 dump read from flash 0x1000, as read-flash 0x1000 or merge-bin --target-offset 0x1000 writes it, opened automatically as its bootloader image with a megabyte of undescribed bytes and no next probe. The flash reader refuses that origin, but nothing told an agent which flash_offset to pass. espFlashDumpOrigin owns the inference: an image of a chip whose ROM loads the bootloader from flash R, at file offset F, is the bootloader of a dump read from R - F when a partition table sits where that origin puts 0x8000. The ESP image reader asks it for a non-app image at file offset 0, so the result does not depend on how the reader was created, and records flash_dump {flash_offset, table_file_offset}, null otherwise, with a limitation that says to reopen the file as esp-flash-image with that flash_offset. inspect_artifact proposes the same open_binary call as a next probe, and the flash reader's misplaced-origin refusal names the offset. * perf(artifacts): summarize nested ESP images in the flash layout summary view The summary view returned an ESP flash layout's root facts whole, so each app partition and the bootloader carried every segment and signature block, about half of a dump's summary. The summary view now gives each nested image a summary instead of its facts: status, chip, entry point, segment count, load ranges (each mapped, loaded or unclassified segment's address and declared length), the checksum and appended SHA-256 statuses, coverage and limitations. Complete detail keeps the full facts, every segment included, and the retained Evidence is unchanged. On a 16 MiB device dump the summary's root falls from 5,141 to 2,657 bytes, and the summary Evidence from 10,038 to 7,554. * test(artifacts): expect the misplaced image the ESP bootloader search reports |
||
|
Entrpiandgensokyo-dev
|
64196b08ca |
feat(artifacts): inventory and extract ESP32 firmware images (#7498)
* feat(artifacts): inventory and extract ESP32 firmware images ESP32-family app and bootloader images (magic 0xE9) had no reader, so open_binary refused OTA downloads, app partitions and bootloaders. Add the esp-image container format behind inspect_artifact and extract_artifact. A pure decoder reads each byte once and keeps only fixed-size fields. Derived facts cover the chip, from a table cited to ESP-IDF v6.1; segment classification as ESP-IDF's should_map and should_load; checksum and appended SHA-256 results; app and bootloader descriptors; Secure Boot v2 signature blocks, observed and unverified; and undescribed trailing byte ranges. Truncation and undescribed bytes make coverage unknown. ESP8266 images are refused with a specific reason, and ciphertext stays unrecognized. Add the format to every target, Evidence, snapshot, live-summary and artifact vocabulary, document it, and add an optional verify:esp-images lane that compares esptool-built images with esptool image-info and espsecure signature-info-v2. Refs #3615 * test(artifacts): run the ESP image lane on esptool v5.6.0 The lane compared descriptor fields only when esptool image-info printed one, so a descriptor that REA reports and esptool does not print passed without comparison. Compare the descriptor kind in both directions. Record that the lane passed for all 13 images with esptool and espsecure v5.6.0, the version the layout facts cite, as well as with 4.12.0. * refactor(artifacts): define ESP image layout sizes once The decoder and the derived facts each defined the segment-header, appended-digest and flash-sector sizes, the erased-byte scan and the offset formatter. Define them once with the other fixed-layout structures and import them in both places. The signature slot count is the one constant that drives the slot scan. Correct the observedEnd comment: it is the end of the last chunk read. * fix(artifacts): report ESP-IDF's segment length limit by chip ESP-IDF's verify_segment_header rejects a segment of 16 MiB or more unless the chip maps flash through 32-bit cache addresses (SOC_SPI_MEM_SUPPORT_CACHE_32BIT_ADDR_MAP: ESP32-S3, P4, C5, C61, H4 and S31 in v6.1). Record that capability in the chip table and add a limitation for such segments; unknown chips get none. `loaded` repeats should_load. It does not mean the bootloader accepts the segment: verify_load_addresses still rejects one outside the chip's RAM, which REA does not model. Say so in the docs, the skill reference and the inventory limitation. * test(artifacts): check ESP lane tools first and verify extraction The lane started work before checking its tools, so a missing RISC-V target, ld.lld, esptool or espsecure surfaced as a raw spawn or command error. Check each one before building and name the variable to set. Warn when esptool is not v5.6.0, the release the layout facts cite. Assert each segment's class for the ESP32-C3 app, its variants and the bootloader, as the Xtensa image already did. Extract the app and the signed image through the CLI and compare every file with the bytes at esptool's segment offsets or in the appended signature sector; the source's SHA-256 must be unchanged and equal the reported root_sha256. * docs(artifacts): state what the ESP image lane does not establish The Verification section listed the lane's checks but not its gaps: only ESP32-C3 and ESP32 images come from a real producer, no real project build, OTA download or flash dump is inspected, no signature is verified, and the lane does not run in CI. Record the timing of the final lane run on esptool v5.6.0. * fix(artifacts): compare ESP segment declarations Segment entries hash only their data, so compare_artifacts reported a segment that moved to another load address with the same bytes as unchanged; only the image itself changed. Compare each segment's load_addr and data_len as its declaration, as Node SEA and PyInstaller members are compared, and count a changed declaration as a metadata change, also when content is unavailable. Header offsets are layout and the classification is derived, so neither is compared. * test(artifacts): run ESP lane commands as owned process trees The lane ran its tools through execFile. A timeout killed only the direct child, so a linker or assembler that Clang or GCC had started could keep running while the lane removed its workspace. Run every command through createVerifierCommands, check the tools inside withVerifierCleanup, and remove the workspace only after commands.close() has stopped every tree; a failed close keeps the workspace. Only a nonzero exit now counts as a refusal or an expected esptool failure; timeouts and spawn failures propagate. Supervising each command roughly doubles the lane's run time. * test(artifacts): cancel ESP decoding after its first decoded chunk The cancellation test requested cancel or close on setImmediate, which can land while the file is still being opened and checked, so it never showed that decoding had started. It also passed against a reader that ignored both requests while decoding. Acknowledge the decoder's first chunk, then cancel or close, join the operation, and assert that no later chunk reaches the decoder and that the reader ends closed. * docs(artifacts): record which ESP metadata comparison ignores Segments are compared by their declaration. The image's and the signature blocks' format_metadata are not: every field comes from their own bytes, which content comparison already covers, and a block's image_digest_matches follows the image's content. * docs(artifacts): state that ESP mmu_page_size is a base-2 logarithm The app descriptor's mmu_page_size keeps ESP-IDF's name and value, the MMU page size in log base 2 as esp_app_desc_t declares it, so 16 means 64 KiB. Say so in docs/esp-image.md and the schema comment; a reader could otherwise take 16 as 16 KiB. * fix(esp-image): retain member consumption and share lifecycle regressions * test(artifacts): scope MCP catalogs and budget full-schema validation * test(artifacts): verify compact output profiles and document bounded catalogs * test(artifacts): close readers when cleanup-test admission fails * test(artifacts): compare ESP image limitations with their owners The decoder and reader tests froze complete limitation sentences, so a harmless rewording failed them. The wording now has two owners: ESP_IMAGE_LIMITATIONS in espImageFacts.ts, which the decoder, the inventory and the reader use, and ESP_DESCRIPTOR_LIMITATIONS beside the descriptor decoder. Tests compare delivered limitations with the owner, called with the facts each case establishes (segment index, declared and available bytes, offsets, chip, page range), next to the structured assertions they already make. Delivered text is unchanged. * test(esp-image): assert structured facts and owned guidance * fix(esp-image): keep spi_flash_mode and descriptor fields before a truncated tail ESP-IDF v6.1's esp_app_desc_t declares spi_flash_mode at byte 181, after mmu_page_size, as the committed reference layout shows; the descriptor now keeps it as stored. The decoder also required all 256 descriptor bytes, though it reads none past byte 181, so a file ending in the reserved tail lost the version, project, digest and revision facts it does hold. The app descriptor now needs only its fields; the segment's truncation is reported as before. * test(esp-image): pin the P-192 public key digest to espsecure's layout espsecure packs ECDSA signature blocks as "<BBBx32sB64s64s1031x" and zero-pads a 48-byte P-192 point to 64 bytes, so the signature starts at byte 101 for every SHA-256 curve. signature_info_v2 hashes sig_block[36:101] and _digest_sbv2_public_key the curve ID with the padded point. No test built a P-192 block; this one checks public_key_sha256 against that definition. * fix(esp-image): keep each app descriptor field the file holds in full A file can end anywhere inside the 256-byte esp_app_desc_t. The decoder needed every field byte, so a cut at byte 176 dropped the version, project and app_elf_sha256 it does hold. Once the 4-byte magic identifies an app descriptor, each field is now kept when the file holds all of its bytes and is null otherwise; observed_length records where the descriptor ends, so a cut field is told apart from text that is not UTF-8. The fields after the magic become nullable in the metadata schema. * fix(esp-image): report a signature block whose image digest does not match A CRC-valid Secure Boot v2 block can carry another image's digest. The block recorded image_digest_matches false, but no limitation said so, unlike a checksum or appended-SHA mismatch, so CLI and MCP summaries showed only the generic unverified note. The mismatch is now a limitation; coverage stays complete and signature_status stays unverified. The three other signature-sector limitations move into ESP_IMAGE_LIMITATIONS with their text unchanged, so all ESP image wording has one owner. * refactor(esp-image): remove redundant descriptor byte guards --------- Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
Cipherandgensokyo-dev
|
18a8e71c9f |
feat(artifacts): add U-Boot legacy uImage reader (part of #7129) (#7234)
* feat(artifacts): add U-Boot legacy uImage reader (part of #7129) Recognize legacy uImages by their 0x27051956 magic and inventory them as containers: 64-byte header facts (arch/os/type/compression raw and named, load/entry addresses, name, timestamp) with header and data CRC32 verified, stored payload extractable as payload.bin, and multi-file images split from the stored size table into image-<n>.bin. Compressed payloads stay as stored; nothing is executed or decompressed. Wires the format through classification, the artifact reader dispatch, extraction, and the evidence/snapshot format vocabularies. Related to #7129 (uImage part only); HEX/SREC and FIT are out of scope. * fix(uimage): preserve aligned component ranges and exact header facts * fix(uimage): retain buffered members through consumption and cancellation * fix(uimage): preserve active read and validation failure precedence --------- Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
blankymandgensokyo-dev
|
e78ac1a55b |
feat(artifacts): read UnityFS AssetBundle containers (#7699)
* feat(artifacts): read UnityFS AssetBundle containers inspect-artifact and extract-artifact now recognize the UnityFS signature and treat format versions 6 through 9 as containers. Header facts (format version, player/engine strings, declared size, dataflags, blocks-info bounds and the declared 16-byte data hash) are retained alongside a directory of CAB-* and .resS members. Member bytes stream on demand: only the blocks covering the requested member are read and decompressed with bounded expansion and cancellation. Bundles declaring asset-bundle encryption keep header facts and report unknown directory coverage. LZMA blocks-info reports unknown coverage; LZMA member reads fail as unavailable while the directory stays readable. Malformed counts, out-of-stream members and declared sizes beyond the file fail as format errors. UnityFS classifies by bytes in root detection, inventory graph construction and desktop inspection, which emits a unityfs-header fact. Validation: tsc --noEmit, oxlint (no new findings on touched files), 17 reader tests covering LZ4HC/uncompressed tables, blocks-info at end, format 6 without padding alignment, multi-block members, malformed input, unsupported compression, cancellation and abandoned streams; full artifacts/domain/conformance suites pass. Verified against one real YooAsset bundle (Unity 2021.3.32f1, format 8, LZ4HC): the streamed SerializedFile contained its managed assembly byte-identical to a UnityPy extraction. Other producer variants unverified. * fix(unityfs): own bounded member reads and preserve partial evidence * refactor(unityfs): clarify producer revision flag decisions * fix(artifacts): preserve coverage authority and bound UnityFS block work --------- Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
|
dba8a31978 |
feat(elf): inspect Linux kernel module metadata (#7010)
* feat(elf): inspect Linux kernel module metadata Adds inspect-kernel-module and inspect_kernel_module for ELF32/ELF64 ET_REL Linux kernel modules, read statically without executing them. Reports ordered .modinfo entries with duplicates preserved, __versions symbol CRCs, the presence of .gnu.linkonce.this_module and __ksymtab, .init.text and .exit.text sizes, and an appended module signature trailer with its PKCS#7 byte range and SHA-256. gzip, xz and zstd containers are reported unsupported with external decompression as the next probe. Malformed headers and exhausted entry budgets return no partial success. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016y53PdpyMAp9RSW65fatHe * fix(elf): bound kernel module ksymtab listing and tighten contracts Charge listed __ksymtab* sections against max_entries, omit section names longer than 256 bytes with ksymtab_name_exceeds_limit, and aggregate the non-file-backed limitation so repeated huge names cannot inflate Evidence. Use fixedTupleSchema for the signature pad bytes with projection parity tests, and assert the late-cancellation reason and retained observation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016y53PdpyMAp9RSW65fatHe * fix: resolve merge integration errors --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
|
3d84215bb2 |
feat(pe): inspect TLS callbacks and load configuration (#7008)
* feat(pe): inspect TLS callbacks and load configuration Adds inspect-pe-load-config and inspect_pe_load_config for PE32 and PE32+ images, with artifact hashes and file ranges in the returned evidence. Reports callback addresses and mappings, plus size-covered security-cookie, SafeSEH, CFG, EH-continuation and XFG fields; flag-derived mitigation claims are explicitly labeled inferred. Missing directories remain null, and malformed storage or exhausted read budgets return errors without partial success. Adds documentation and tests for service validation, CLI/MCP access, malformed inputs and stable-file handling. x86 images that declare the legacy 64-byte load-config directory are read to the structure's own Size when those bytes are file-backed, as the Windows loader does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016y53PdpyMAp9RSW65fatHe * fix(pe): retain snapshot facts and bound load-config section lookup Keep the completed stable-read path, SHA-256 and byte count as a file-snapshot-hash partial observation when descriptor close fails or is cancelled. Report the checked Size span as structure_location and add it to Evidence locations beside the declared directory range. Resolve callback sections through a sorted segment index built once per image, keeping the ambiguity failure. Pin every load-config field to offsets transcribed from the PE/COFF spec and llvm COFF.h at its exact Size boundary for PE32 and PE32+. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016y53PdpyMAp9RSW65fatHe --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Ariel Fikru Avicennaandgensokyo-dev
|
cab78de506 |
feat(ghidra): analyze GameCube/Wii DOL executables through Ghidra (#6240)
* feat(domain): parse and validate GameCube/Wii DOL headers Add a pure parser for the 0x100-byte DOL header (7 text and 11 data slots, BSS, entry point). Used slots must lie after the header and inside the file and the 0x80000000..0x81800000 load window; text slots are 4-byte aligned; sections may not overlap in memory or in the file; the entry point must be inside a text section. BSS may overlap data sections, as .sdata/.sdata2 do in shipped DOLs. Refs #1741 * feat(binary): admit caller-selected GameCube/Wii DOL targets * feat(ghidra): import GameCube/Wii DOL targets through a section-map pre-script * test(ghidra): cover DOL launcher arguments, attestation and raw profile digests * test(ghidra): add the real verify:ghidra:dol lane and DOL docs * fix(ghidra): read only the DOL header and section ranges instead of the whole file * test(ghidra): derive DOL lane addresses from readelf and admit Gekko from the install catalog * fix(ghidra): attest DOL block permissions, space, overlay and load state * fix(ghidra): keep load-image permissions optional for older inspections * fix(contracts): accept DOL targets in binary_session output * fix(ghidra): adapt the DOL lane to the current upstream address schema Validate DOL load-image coordinates with the upstream observed-address schema, drop a duplicate import left by the rebase, and re-pin the raw profile digests to the unmodified upstream resolver. * fix(ghidra): preserve bridge inventory symbol provenance * fix(ghidra): align DOL verification with shared workflow contracts * feat(domain): parse and validate GameCube/Wii DOL headers Add a pure parser for the 0x100-byte DOL header (7 text and 11 data slots, BSS, entry point). Used slots must lie after the header and inside the file and the 0x80000000..0x81800000 load window; text slots are 4-byte aligned; sections may not overlap in memory or in the file; the entry point must be inside a text section. BSS may overlap data sections, as .sdata/.sdata2 do in shipped DOLs. Refs #1741 * feat(binary): admit caller-selected GameCube/Wii DOL targets * feat(ghidra): import GameCube/Wii DOL targets through a section-map pre-script * test(ghidra): cover DOL launcher arguments, attestation and raw profile digests * test(ghidra): add the real verify:ghidra:dol lane and DOL docs * fix(ghidra): read only the DOL header and section ranges instead of the whole file * test(ghidra): derive DOL lane addresses from readelf and admit Gekko from the install catalog * fix(ghidra): attest DOL block permissions, space, overlay and load state * fix(ghidra): keep load-image permissions optional for older inspections * fix(contracts): accept DOL targets in binary_session output * fix(ghidra): adapt the DOL lane to the current upstream address schema Validate DOL load-image coordinates with the upstream observed-address schema, drop a duplicate import left by the rebase, and re-pin the raw profile digests to the unmodified upstream resolver. * docs(ghidra): qualify the DOL memory window * refactor(ghidra): own DOL BSS ranges in the parsed header * docs(ghidra): record real DOL workflows and task costs * docs(ghidra): record final DOL integration verification * fix(mcp): share the native procedure search row contract * docs(ghidra): record public procedure search verification --------- Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
|
781760a195 |
feat(apple): inspect IPS crash reports (#6511)
* feat(apple): inspect IPS crash reports Add inspect_apple_ips for caller-selected Apple .ips files. Parse the first-line metadata object and, when bug_type is the string 309, the crash body: process identity, exception and termination facts, threads, usedImages, lossless integer registers, and documented classification rules. Other bug types including 288 stackshots return no crash result. Opened Mach-O and dSYM UUID joins remain unobserved. Related to #4297 * fix(apple): keep IPS register literals exact Use the arm64e fixture integer that matches 0x8527000193ed4944 instead of an IEEE-754-rounded decimal, and count the lossless JSON example length from the source text. * fix(apple): budget IPS parsing and keep typed presence Parse the IPS metadata and crash body through a projection of the documented members. Members the report does not interpret are syntax-checked without being materialized, and thread, frame, image and the new max_registers budgets run before each row is parsed, so a large selected file fails with a typed resource error instead of first expanding every row. Fixed allowances bound JSON nesting, unlisted key names and termination reasons. Present values whose JSON type differs from the documented one are no longer read as absence: they are listed in coverage.type_mismatches and coverage becomes partial, including wrong-type or missing threads and usedImages collections. osVersion keeps the producer dictionary or the string form. Frame roles follow physical frame depth, so the top frame of a non-faulting thread is current_instruction and inlined rows share the following frame's role. SIGKILL classes compare the canonical hex once, so 0xdeadd0cc no longer matches dead10cc. Invalid UTF-8 is refused instead of being replaced, and JSON strings are decoded by copying unescaped spans. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ra7LM2NyQ7DumdssxbUq5g * fix(apple): bound integer text expansion and reconcile coverage * refactor(apple): retain image metadata at its owning row * refactor(apple): validate row capacity through reconciled coverage * test(apple): verify resource recovery at the public error boundary * docs(apple): record complete IPS task and corpus measurements * docs(apple): record integrated checks and stdio corpus evidence * docs(apple): record PE integration and baseline cleanup controls * fix(native): bound IPS string decoding and preserve invalid identities * refactor(native): use exact optional IPS source evidence * docs(native): verify IPS escape capacity and complete-task costs * test(native): verify IPS through production stdio MCP * style(tests): format IPS advertised schema bindings * fix(native): preserve unresolved IPS references and recorded frame semantics * docs(native): record IPS reference review evidence * fix(apple-ips): enforce JSON and report consistency * docs(apple-ips): record final consistency verification --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
|
4ef916da03 |
feat(pe): inspect import and delay-load tables (#6509)
* feat(pe): inspect import and delay-load tables Add inspect_pe_imports / inspect-pe-imports to read PE32/PE32+ import and delay-load directories without executing the image. The report keeps DLL names, named and ordinal functions, ILT/IAT RVAs, and a pefile import hash of the standard import table. A missing directory is observed absence, not proof the image never imported APIs. * fix(pe): default PE import fixture options Allow peImportImage() without an argument so CLI and filesystem tests can use the KERNEL32 sample without repeating an empty options object. * test(pe): keep import path cases on the dedicated MCP contract inspect_pe_imports already rejects relative MCP paths in its own boundary test. Leave the shared absolute-path matrix to tools whose handler issue projection matches that suite's expected shape. * fix(native): index virtual-to-file mapping lookups mappedFileOffset scans every mapping twice per span, so a parser that translates one span per table entry costs sections x entries. Add fileMappingLookup, which builds an O(n log n) interval index once and answers each span in O(log n) with the same accept, reject and translation results. A differential test checks it against mappedFileOffset over randomized overlapping mappings and zero fills. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ra7LM2NyQ7DumdssxbUq5g * fix(pe): report partial import evidence and exact pefile hashes - Withhold imphash with an explicit reason whenever the standard import table is partial (unreadable names or thunks, missing tables), or when pefile 2024.8.26 would hash a different row sequence: invalid or truncated names, ordinal 0, descriptors without functions, tables its bounds or repeat/spread heuristics discard, and its 8192-thunk import symbol cap. Delay-load issues do not withhold it. - Hash ordinal imports with pefile's complete ordlookup tables and its ord<decimal> fallback instead of declining unmapped ordinals. - Report a nonterminal descriptor whose FirstThunk is zero, and a delay-load descriptor whose IAT or INT RVA is zero, as malformed_entry partial coverage instead of a complete empty inventory. - Directory evidence lists the file spans actually consumed for the descriptor array and terminator; discontiguous mappings produce separate spans and separate Evidence ranges. - Carry exhausted limits as PeImportLimitFailure.limit and derive recovery from it; the fixed 16 MiB retained-text ceiling reports a fixed allowance instead of max_file_bytes advice. - Translate RVAs through the indexed mapping lookup and yield with a cancellation check every 1024 table entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ra7LM2NyQ7DumdssxbUq5g * fix(pe): read only own ordinal tables for DLL names DLL names such as constructor and __proto__ pass pefile's name rule but resolved inherited object members, throwing before the ord<n> fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ra7LM2NyQ7DumdssxbUq5g * fix(native): preserve PE import facts and prove hash compatibility * test(native): verify PE import workflows and measured costs * refactor(native): narrow PE facts across the container union * docs(native): record integrated PE import validation * fix(native): bound every PE import RVA span * docs(native): measure RVA guard completed workflows * fix(pe): reconcile import coverage with retained rows * fix(pe): retain partial imports at descriptor mapping gaps --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
|
0d596fa8f7 |
docs(limits): describe how to design admission recovery (#6368)
* fix(reconstruction): dedupe residual unknown ids with uniqueSorted Merged obligation candidates called uniqueBy, which is not defined, so the project did not typecheck. String ids already use uniqueSorted. Co-authored-by: morluto <morluto@users.noreply.github.com> * docs(limits): describe how to design admission recovery Add a short contributor guide for named defaults, settings, recovery text, and the agent recovery ladder, and mirror that ladder into the authored skill. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(hopper): pass distribution readers into install-host tests systemLinuxHopperInstallHost now requires the caller's distribution reader. The Linux install tests still passed only the environment, so typecheck failed. Co-authored-by: morluto <morluto@users.noreply.github.com> * docs(limits): name the parser doctor shares with startup Co-authored-by: morluto <morluto@users.noreply.github.com> * test: align pcap, storage, and regex fixtures with current contracts Source and portable CI failed on fixtures that still described the previous pcap client identity, legacy storage fingerprints, and the old regex deadline sentence. Co-authored-by: morluto <morluto@users.noreply.github.com> * docs: link repository experiments outside the built site --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <morluto@users.noreply.github.com> |
||
|
FerroLx
|
e93e3b1e7d | feat(apk): inspect signing declarations with retained fact queries | ||
|
|
2c3dd946d5 |
feat(python): inspect CPython pyc headers and marshal code objects (#4826)
* feat(python): inspect CPython pyc headers and marshal code objects Add inspect_python_pyc / inspect-python-pyc to read one explicit local CPython 3.7–3.13 .pyc without executing bytecode or recovering source. * chore(python): drop unused PythonPycHeader export Knip reported the type as unused after inspect_python_pyc landed. * fix(python): bound pyc expansion and decode marshal like CPython - Count a nested code object reached through TYPE_REF once per copy the report carries, against max_objects and max_file_bytes, before counting or validating the tree - Charge long digits to max_string_bytes, reject out-of-range and unnormalized digits, build the value in one pass, and report integers above 256 bits by bit length only - Decode TYPE_UNICODE with surrogatepass semantics and reject other ill-formed UTF-8 - Map only 3.11 magics 3494-3495 to the py311 layout - Report non-finite float and complex parts as inf, -inf or nan - Allow an empty co_filename - Require bytes for co_linetable and co_exceptiontable - Bind cleanup-failure errors to their retained descriptor leases * fix(python): list pyc code objects once instead of a recursive tree Report each distinct code object once in code_objects, top-level first, with nested_code as indexes. A child shared through TYPE_REF is no longer copied per reference, and the MCP output schema is no longer recursive. Names repeated through TYPE_REF still count against max_file_bytes for each copy the report carries. * fix(python): bound shared pyc constants and reject malformed tables - Cache byte-string digests so a buffer shared through TYPE_REF is hashed once instead of on every reference. - Charge every reported co_consts entry, including tables shared through TYPE_REF, against max_objects before summaries are built. - Resolve the inspect-python-pyc path with absoluteCallerPath so link/../file.pyc reads the file the filesystem selects. - Reject TYPE_NULL inside marshal sequences, as CPython does. - Reject a co_localspluskinds length that does not match co_localsplusnames in the parser and the report schema. * fix(python): bound shared pyc names and read CPython marshal graphs Charge each reported code-constant name, keep integer bit lengths on the decoded value, and list code objects nested inside constant containers. Accept self-referential lists and dicts, marshal version 0 text floats, empty code names, the original string byte count, and Python set and dict lengths. Pass the CLI cancellation signal into command cleanup. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(python): bound pyc key expansion and shared nested code Charge set and dictionary keys before a shared tuple doubles them, list each nested code object once per owner, and accept code objects as CPython keys. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(python): correct pyc equality, counts, and nested walks Summarize code objects only after the marshal graph is complete, and collect nested code with an iterative walk that shares wrapper results instead of copying every descendant list. Keep tuple boundaries and the original string bytes in equality keys, and reject negative CPython counts. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(python): bound shared pyc names and keep the first dict key Charge every copied name-table entry against the object budget before building the report array, including empty strings that add no bytes. When dictionary keys compare equal, keep the first key object and replace only its value, matching CPython. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(python): walk each pyc dictionary child list once Keep a container's child array on the nested-code frame so a wide dictionary is not copied on every step of the synchronous marshal walk. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(python): reuse stored pyc dictionary children when combining The nested walk keeps each container's child list on its frame. Combining nested code rebuilt that list, and the 8,000-key regression still accepted the old per-key copy. Use the stored list, and require that dictionary to finish well under the copy's runtime while still reporting a nested code object. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(python): keep distinct pyc code objects and reject corrupt ones Encode each name table's length and entry boundaries in the equality key so a different name list cannot share a memo key. Walk tuple and frozenset keys on an explicit stack so a long TYPE_REF chain does not overflow. Treat code objects CPython rejects for an odd co_code or an undersized local table as malformed instead of complete coverage. Co-authored-by: morluto <morluto@users.noreply.github.com> --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <morluto@users.noreply.github.com> |
||
|
|
bfba99827e |
feat(erlang): inspect BEAM IFF chunks and tables (#4833)
* feat(erlang): inspect BEAM IFF chunks and tables Replay the Erlang BEAM inspection workflow, including table bounds, OTP 28+ atom layouts, the shared absolute-path policy, and empty-path CLI coverage, without carrying GitHub workflow edits. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(erlang): use publicInputStrictObject for BEAM tool input inspect_erlang_beam advertised a public input built with bare z.strictObject, so later shared input-policy changes would miss it. Result and internal objects stay on their own constructors. Co-authored-by: morluto <morluto@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <morluto@users.noreply.github.com> |
||
|
|
ca4ca65c5e |
feat(pe): inspect Authenticode certificate tables (#4830)
* feat(pe): inspect Authenticode certificate tables Add inspect_pe_certificates for IMAGE_DIRECTORY_ENTRY_SECURITY WIN_CERTIFICATE entries and structurally decoded PKCS#7 signer and X.509 identity fields. The workflow reads a stable PE image without executing it or verifying signatures, chains, timestamps, or Authenticode digest coverage. * fix(pe): bound and validate certificate DER decoding - Cap decoded DER values with a shared node budget so sibling fan-out cannot exhaust memory before the PKCS#7 check - Require SignedData version, encapContentInfo and a final signerInfos SET in RFC 5652 order, and a signature OCTET STRING in each SignerInfo - Decode OBJECT IDENTIFIER arcs as unbounded base-128 values, including a multi-byte first combined arc - Decode UniversalString names as UTF-32BE and reject invalid values - Share PE header admission with the resource layout so a section table that does not fit SizeOfHeaders is rejected * docs: list PE certificates in the translated READMEs * fix(pe): cap OBJECT IDENTIFIER length and arc size A crafted type-2 certificate with a 262,144-byte contentType OID stalled decodeOid for about 20 seconds, because one arc kept growing a BigInt before the signedData check ran. decodeOid now rejects an OID longer than 256 bytes and an arc longer than 32 bytes before decoding it. Certificate OIDs, including 2.25 UUID arcs, fit under both caps. * fix(pe): reject truncated certificate tables and PKCS#7 SignedData now requires one SignerInfo with a non-empty signature and a complete issuerAndSerialNumber. Certificate names share a 64 KiB decoded-byte budget and are copied once. A security directory that ends in a short WIN_CERTIFICATE header fails instead of reporting a complete table. The certificate CLI forwards the caller abort signal into cleanup. Co-authored-by: morluto <morluto@users.noreply.github.com> * chore(ci): keep fork workflow files unchanged The certificate restack follows current main. Workflow files stay at the fork tip so this branch update does not rewrite .github/workflows. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(pe): charge serial and validity text to the decoded budget Certificate and signer serial numbers and validity times now count against the shared decoded-text budget before they are converted, so a crafted certificate cannot push megabytes of text into the hashed evidence. The budget error now names certificate text rather than only names. Restore the CI workflow to match main: the CI_TIMING_TIER input, the tier-named timing artifacts and their publication conditions, the Windows typecheck, and the path and snapshot suites in the curated Windows lane. * fix(pe): require certificate fields and preserve signer key ids Reject X.509 records that omit subjectPublicKeyInfo or signatureValue, return subjectKeyIdentifier bytes as bounded hex, and advertise an absolute path in the PE certificate input schema. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(pe): reject extra PKCS#7 wrappers and invalid UTF-8 names ContentInfo and its signedData wrapper no longer ignore trailing values, and directory strings must round-trip as UTF-8 instead of becoming U+FFFD. Co-authored-by: morluto <morluto@users.noreply.github.com> --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <morluto@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
Matt Van Horn
|
2edf3b23c1 |
feat(elf): inspect PT_NOTE and SHT_NOTE tables (#4837)
Inspect selected ELF32/ELF64 PT_NOTE and SHT_NOTE declarations without executing the artifact, retaining independent table identities, source ranges, descriptor digests, GNU build-id/ABI fields and explicit coverage limitations. Integrate mvanhorn's implementation and hardening with current main while preserving byte-entropy registration and the Windows curated regression suite. The maintainer follow-up uses ordinary merges/pushes, not history rewriting. Executed validation: 96 focused/shared/docs tests, complete static/docs checks, eleven real CLI/MCP pairs including GCC ELF32 and Clang big-endian PowerPC64, and two actual Codex task traces checked against their Evidence. Full costs, grader correction and verification limits are recorded in PR #4837. Refs #6262. |
||
|
Matt Van Horn
|
afa5da745e | feat(artifacts): inspect and extract PyInstaller archives | ||
|
|
5fa4defcbc |
feat(artifacts): inspect and extract Godot PCK with retained facts (#3634)
* feat(artifacts): read Godot PCK packs Add a Godot Engine PCK artifact reader for standalone packs and packs embedded behind an executable trailer (formats 2 and 3). Members are uncompressed byte ranges verified against their declared MD5 on read. Encrypted directories, sparse bundles, patch removal entries, unknown flags and format 1 fail closed as unavailable; encrypted members are reported without being read. inspect-artifact and extract-artifact recognize standalone packs by their GDPC magic. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(artifacts): bound Godot PCK metadata and own member streams Cap member paths at 4096 bytes and the retained directory at 64 MiB before allocation. Read members with positional chunk reads through pipeline so cancellation is typed, abandoned consumers release their source, and the shared descriptor stays reader-owned. Retain the root descriptor for godot-pck inventory, as for ZIP and universal Mach-O. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(artifacts): complete Godot PCK facts, integrity and selected views * fix(artifacts): preserve atomic-root inventory Evidence * fix(artifacts): bound aggregate PCK member expansion --------- Co-authored-by: Samucahub <151155051+Samucahub@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
gensokyo-dev
|
b4938aed58 |
feat(artifacts): triage desktop installation directories (#5207)
* feat(artifacts): triage desktop installation directories * fix(artifacts): retain completed probes through failed file cleanup * perf(artifacts): decode only assembly reference names for desktop triage * fix(artifacts): enforce exact signatures and responsive directory probes * fix(artifacts): recognize Unity bundles and bound FORM chunks * fix(artifacts): retain bounded desktop omission evidence * fix(artifacts): select ASAR targets before inventory probes * chore: preserve tested upstream HTTP owner during integration * fix(artifacts): reuse Mach-O signatures for FAT64 desktop probes * fix(artifacts): expose bounded desktop probe diagnostics * fix(artifacts): derive desktop snapshots from observed source identities * fix(artifacts): admit desktop facts before JSON expansion |
||
|
|
10f1994fe1 |
feat(environment): serve an environment manifest as a replay environment (#4975)
Serve retained capture Evidence as a bounded MCP replay environment with discovery. Preserve default stdio, caller-selected facts and complete source Evidence. Fix numeric fidelity, argument identity, startup cancellation and bounded discovery, query/path matching and output metadata. Verify real recorded API data and public CLI/MCP workflows; preserve explicit gaps for Windows and third-party hosts. Co-authored-by: Edgar Arakelyan <edgar.arakelyan@scale.com> Co-authored-by: gensokyo-dev <gensokyo-dev@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c1fd496e6a |
feat(kuna): serve procedures, pseudocode and assembly from Kuna exports (#3854)
* feat(kuna): serve procedures, pseudocode and assembly from Kuna exports Register Kuna as a bring-your-own deep-analysis provider. It answers list_documents, list_procedures, search_procedures, procedure_address, address_name, procedure_pseudo_code, procedure_assembly and analyze_function (and batch_decompile through composition) from the verified project export. While a cold export builds in the background, pseudocode comes from a single-function `kuna decompile` run on a private copy and says so; assembly and data names wait for the export. Each answer records its source and file span in raw_result, limitations are scoped per operation, and the session-owned export is stopped on close unless another session still waits for it. The literal/regex procedure search predicate moves from the IDA runner into a shared domain helper. `npm run verify:kuna` adds a real-provider lane with an optional Ghidra comparison. Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(kuna): refuse a Ghidra comparison without caller-built targets Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(kuna): retry owned cleanup and keep answers bound to the opened Kuna - Re-verify the Kuna executable's digest before every launch (memoized by file identity), so no answer names one build while another produced it. - Keep the process supervisor of an unproven stop, and retry it on every later close in both the session and the cache; a failed close keeps failing until the stop is proven instead of reporting success. - Report an abandoned export's retained cleanup from session close. - Forget and rebuild an adopted export whose files another REA process evicted, instead of failing every later read. - Close a private copy whose setup failed, and retry failed removals. - Let the session own the shared quick inventory; each caller only stops waiting when it cancels. - Report a failed export as `failed` with its reason. - Lane: release the MCP client when open_binary rejects a target, and accept either valid source for the first answer. Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(kuna): drain session requests before releasing workspaces Combine caller cancellation with session shutdown and wait for active reads before releasing their process supervisors or private inputs. Retain unproven cleanup for later close attempts. Measure UTF-8 reply bytes without implying model token usage. Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> * perf(kuna): keep document metadata reads free of target analysis Listing the selected path no longer initializes the project cache or starts an export that the request never consumes. Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> * fix(kuna): reject verification matrices with missing functions Validate requested matrix coverage before building the fixture or opening providers. Empty matrices, unmatched names and partially matched function sets must fail instead of reporting an unperformed comparison as successful. Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> * fix(kuna): retain component provenance in function dossiers Record each source facet as one file span and preserve both pseudocode and assembly digests/ranges in combined reports. Keep direct text source fields compatible and verify every combined span through the real MCP lane. Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> * fix(kuna): keep retired file cleanup bound to its export Retain the export key with each failed file removal so a cancelled session reports and retries its own stopped resources. Preserve that association across sweeping and process-cleanup recovery without attributing unrelated cache failures to the session. Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> * docs(kuna): document the Kuna provider and list it in the catalog (#3855) * docs(kuna): document the Kuna provider and list it in the catalog Add the Kuna provider guide (configuration, answered tools, cold and warm paths, provenance, cache ownership, verification lane), link it from the README and its translations, the CLI, installation and index pages and the site navigation, describe `npm run verify:kuna` in the testing guide, credit Kuna in THIRD_PARTY_NOTICES, teach the agent skill when to select Kuna, and register its identity and operations in the generated product catalog. Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(kuna): state the Linux cache fallback, raw_result scope and comparison inputs Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(kuna): note that cache pins hold within one REA process Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(kuna): describe retired staging and retried cleanup Refs #1957 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * fix(kuna): supervise regex and bind reads to one export Share a cancellable ECMAScript matching worker with Hopper while preserving its error contract. Bind C and assembly indexes to immutable project readers, retry whole dossiers after eviction, and enforce the existing buffered-text capacity before allocating cached spans. Document the verified Linux scope and keep complete exports accessible. Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com> Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com> |
||
|
|
bcc141549a |
feat(electron): inspect packaged Electron fuse state without execution (#2399)
Add bounded, read-only inspection of packaged Electron fuse state for ELF, PE, and Mach-O applications through CLI and MCP workflows. Preserve artifact provenance and unknown values, resolve app bundles without execution, reject malformed or overlapping containers before scanning, and keep output compact for agent use. Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com> Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> |
||
|
|
0e6eca4085 |
feat(native): read embedded cargo-auditable Rust dependency lists (#2456)
* feat(native): read embedded cargo-auditable Rust dependency lists * fix(native): trim PE section padding and compute dependency cycles iteratively * fix(native): ignore SHT_NOBITS sections and validate dependency edges linearly * fix(native): bound compiler path scanning and retained hashes * fix(rust): pass package arrays as named parameterized test cases * fix(rust): admit graph expansion and preserve resource units * fix(rust): reject malformed packages, false cycles, and hidden compiler paths Fail the first malformed cargo-auditable package before Zod collects an issue per field. Require cyclic dependency indices to match the cycles in package dependencies. Keep read-only ELF ranges when a section name does not fit the bounded name window, so a rustc path in that range is still reported. * fix(ci): repair integrated checks and Hopper test baseline * fix(go): verify Mach-O sections against segments --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> |
||
|
Ysopwnableandfactory-droid[bot]
|
7ae3c50a9d |
docs: correct drift and reorganize English guides (#2207)
Put setup, developer commands, MCP replies, and investigation recipes before client details and implementation references. Preserve existing guide anchors, examples, provenance, partial-result recovery, and provider restrictions. Correct verified documentation drift, synchronize the Ghidra capability wording, and regenerate metadata for skill version 40. Keep uncertain platform and lifecycle findings unchanged. Refs #2156. Validation: 56 focused tests, check:fast, docs:check, VitePress production build, format and link/example checks. Real isolated CLI/MCP probes cover Ghidra, NativeAOT, HAR, cancellation, Chrome, large ELF delivery, native type limits, and the new JS/managed/process/setup recipes. Heavy work uses two CPUs; a resource-heavy DOM probe was stopped and replaced by a smaller real CDP payload-limit control. Windows, macOS GUI, ADB, and the full CI matrix were not executed. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> |
||
|
Daniel Gonçalves AraujoandN0zoM1z0
|
4a4ae9b2b9 |
feat: add Frida instrumentation support (#1359)
* feat: add Frida instrumentation support * fix: resolve Frida CI checks * test: cover Frida contract validation branches * docs: link Frida instrumentation guide * fix(frida): handle cancellation cleanup safely * fix(frida): address PR review findings * fix(frida): preserve resume and message observations * fix(ci): use canonical registry for rolldown * fix(frida): declare foreign code effects * fix(frida): retain failed startup resources for cleanup retries Keep failed startup session and spawn handles owned until detach, resume, and remote lease cleanup complete. Serialize retries and release each remote reference once. Await CLI provider cleanup while preserving the primary failure and any completed observations. Add regressions for retained startup resources, concurrent cleanup, shared remote leases, and CLI cleanup outcomes. Validation: 41 focused tests pass on the PR head and its latest-main integration; serial static and docs checks pass. Real Linux Frida 17.23.2 workflows and the native spawn/controlled-failure cleanup probe pass. Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing) --------- Co-authored-by: N0zoM1z0 <1705004398@qq.com> |
||
|
morluto
|
c78fe6990b | docs: publish VitePress documentation on GitHub Pages |