From 03a3fcbcea87d6cb7485bffaee77d0338a0ab635 Mon Sep 17 00:00:00 2001 From: Edgar Arakelyan Date: Sun, 8 Nov 2026 06:19:28 -0800 Subject: [PATCH] feat(skills): build a computer-app environment from a request (#9049) * feat(desktop): capture, draft, export and witness a task inside a build computer - start_build_capture runs REA's capture inside the computer through a small controller over computer-exec, after a user-space apt-get update; mark_build_checkpoint returns once the capture confirms the checkpoint; stop_build_capture returns the draft between seed and end. - export_build_environment exports with the requester's task inside the computer and fetches the bundle through data/get in bounded batches. - check_build_task grades a computer's data/get with REA's evaluator and, through seed, after_edit, near_miss and after_reset stages, records a witness with the action log and the screen recording; act_on_build_computer gains reset. - Capture progress lines carry window, checkpoint and refusal events; apps start in the home; a pack script builds the runtime archive; long tools send progress notifications. Co-Authored-By: Claude Opus 5.5 * fix(desktop): make build capture and witness steps retryable, and close the near-miss reset hole - A capture whose start answer is lost is stopped at once and kept unreleased until the stop is confirmed; a lost checkpoint answer is recovered from the capture's own log, and draft and export use the checkpoints its Evidence records; stop can be repeated to resume reading its result. - A witness can check its seed again after a failed read, and keeps its stages in a saving state until the record is written, so a failed save can be retried without reading the app again. - after_edit and near_miss count only click, drag, scroll, type and key, so a reset no longer stands in for the near-miss edit. - A .deb build keeps its package file for capture and passes its URL to export; bundle downloads write each checked file before fetching the next; an apt-get update failure reports its exit code and log tail; reset sends progress while it waits. Co-Authored-By: Claude Opus 5.5 * fix(desktop): keep the Trash out of seeds, bundle folders unwritten and first frames drawn From the blind Inkscape rebuild: - Paths in ~/.local/share/Trash get a desktop_trash reason and the shared_desktop role, so a file trashed before seed is neither owned nor seeded; the export names it in a limitation. - The keep helper runs with PYTHONDONTWRITEBYTECODE=1, so starting a computer from a bundle folder no longer writes __pycache__ into it. - start_build_computer waits, bounded, until the first frame is not one colour before returning it. From review: a .deb that apt also offers exports from apt, since export passes --package-url only when the capture says the package must come from its file. Co-Authored-By: Claude Opus 5.5 * fix(desktop): bound the first-frame wait's retries to the time left, with progress throughout Each retry of the drawn-first-frame wait runs under an abort that fires at the 10 s deadline, and the latest frame is returned when time runs out. connect takes the first frame itself while its progress heartbeat still runs, reporting "waiting for the desktop to draw". The wait moves to DrawnScreenshot.ts. Co-Authored-By: Claude Opus 5.5 * fix(desktop): serialize each build computer's calls, keep one recording id, and separate unreadable checks from failures - One queue per computer: actions, capture steps and witness checks run one at a time, so two overlapping after_edit checks can no longer both be filed and let the witness hold without a near miss. Close cancels the running and queued calls; different computers still run concurrently. - The witness keeps one recording request id per computer, so a seed retried after a lost reply recovers the recording it started. - A check that could not read its file is undecided, not a failure, so a temporary read error can't satisfy seed, near_miss or after_reset; such a stage is refused and can be checked again. - The export's file listing is written to a file and read back in bounded batches, so a large bundle no longer overruns exec's 1 MiB output. Co-Authored-By: Claude Opus 5.5 * fix(desktop): treat failures on cut views as undecided, require a decided seed failure at export, and keep a close's cancellation - A presence check that misses its line, row or pointed value in a view cut short is undecided rather than failed, since the data may sit in the part left out; a witness stage can no longer be proved by such a failure. - Export counts only decided failures on the seed, and refuses a task whose every seed failure is undecided. - A step whose turn was aborted by a close or the caller returns cancelled even when its work finished, so start_build_capture no longer reports success for a computer being removed. Co-Authored-By: Claude Opus 5.5 * fix(desktop): keep a step's saved result when a cancellation arrives during its commit Reporting cancelled for any step whose turn was aborted after its work replaced a completed export or witness save with a bare cancellation, losing what was saved and leaving the witness unreadable. Steps now return what they completed; only start_build_capture, which commits nothing, checks its turn after the post-launch screenshot and reports cancelled. Asking again for a saved witness's final check returns its record. Co-Authored-By: Claude Opus 5.5 * fix(desktop): refuse a saved witness's repeated check for another folder Asking again for the check that ended a witness returned its saved record whatever directory came with it, so a caller naming another folder got success though nothing was saved there. A different folder is now refused, as an unfinished seed's retry already is. Co-Authored-By: Claude Opus 5.5 * test(desktop): assert the facts of cut-view and undecided outcomes, not their sentences The new cut-view, missing-value, undecided-refusal and witness tests froze whole descriptions, so a harmless rewording broke them. They now check each outcome's id, passed and undecided exactly and the facts its message must carry (file, pointer, line, row, stage) separately. Co-Authored-By: Claude Opus 5.5 * feat(desktop): read XML with a plain DOCTYPE, and check an element's whole text - Both readers accept a DOCTYPE with only a name and an optional PUBLIC or SYSTEM identifier, record it as the view's doctype, and never fetch the DTD or expand entities beyond XML's five; an internal subset, an external entity and a parameter entity stay refused alike. - text_equals and text_contains compare the concatenated text of the element (or text node or attribute) a pointer reaches, as XPath string() does, with optional normalize_whitespace; both evaluators agree, and a text check on a truncated element is undecided. - Whitespace-only text between elements is dropped only when the element holds no other text, so text split into runs keeps its spaces. - The draft suggests text_equals for an element whose text changed. So an AbiWord .abw reads as XML, and its paragraph's text is checked whole however AbiWord splits it into runs, as Writer's spans are. Co-Authored-By: Claude Opus 5.5 * fix(desktop): draft whole-text checks on the paragraph, read only text as text, and skip asctime stamps - A changed value is read at both ends, so text that became a run still marks its paragraph, and the drafted text_equals moves up out of mixed content to the nearest ancestor named by an attribute, such as AbiWord's p by its xid. - Text checks accept only a text node or an attribute value; element names and doctype fields fail, in both evaluators. - isTimestamp also recognises asctime dates (Www Mmm dd hh:mm:ss yyyy). - XML files are drafted with the keyed diff the archive entries use, so no suggestion reaches an element by position; the shared logic moves to appTaskDraftNamed.ts. - Text-check tests assert facts, not whole sentences. Co-Authored-By: Claude Opus 5.5 * fix(desktop): keep checks on a changed attribute inside a paragraph whose text check replaces its runs' A drafted whole-text check on a paragraph hid every change inside it, so a link whose label and href both changed drafted only the paragraph's text and the new destination went unchecked. The text check now covers text and runs added or removed whole; a changed name or attribute of an element both versions hold keeps its own check. Co-Authored-By: Claude Opus 5.5 * fix(desktop): read an attribute called name or attributes as an attribute when the draft keeps its check The owner of `.../attributes/name` was taken as `.../attributes`, so a changed `name` attribute inside a paragraph with a text check was dropped again. The owner is now read by segment, attributes first. Co-Authored-By: Claude Opus 5.5 * fix(desktop): draft text checks from text in runs added whole, and name a text element by its own id before its container Text typed into a run the diff lists whole was never read, so the draft kept only a presence check on the run, which wrong text also passed. Such text now marks the nearest element both ends hold, or, when none can take the check, the added element's own text. The climb also names an element by an identifier of its own (id, xml:id, xid) before a larger container, so a headline under a long layer keeps a whole-text check that survives re-split runs. Co-Authored-By: Claude Opus 5.5 * test(desktop): expect the whole-text check where #8988's pairing test meets text checks On this branch a changed element's text is drafted as `text_equals` on the element, so the identity pairing test now expects that check, under the same `[@role=body]` key. Co-Authored-By: Claude Opus 5.5 * feat(skills): build a computer-app environment from a request The product skill gains references/desktop-build.md, a procedure that takes Claude from a request such as "an Inkscape environment where the agent changes a poster's headline" to a witnessed bundle and a report, through start_build_computer, act_on_build_computer, the build capture tools, export_build_environment and check_build_task: questions asked once upfront, pauses only when blocked, a seed made in the app, content checks by selector, a witness with a near miss and a reset in a fresh computer, and solver runs only within a budget. setup installs the new reference; skillVersion 61. Co-Authored-By: Claude Opus 5.5 * docs(skills): fold the blind Inkscape rebuild's friction into the build skill key takes an array of one chord's keys; check for a text cursor before typing in apps with single-key shortcuts; in Save As, click Documents and type only the file name; anything saved before seed, the Trash included, is part of the seed; dismiss first-run dialogs before seed; a bundle's app starts with the seed's files but need not open the document; Inkscape's window class is Inkscape. Co-Authored-By: Claude Opus 5.5 * docs(skills): fold the held-out AbiWord build's friction into the build skill Name the act fields and link the guide; a region screenshot becomes the latest frame; a drawn caret is not focus; confirm dialogs with Return; check the dialog's folder when the app shows only a name; an XML file the draft shows as lines was read as text, so use short line checks; and keep text checks to the value the edit types, since some apps save a reloaded document differently. Later witness stages take no directory. Co-Authored-By: Claude Opus 5.5 * docs(skills): state the export's seed rule, the Trash exception and draft refusals exactly The export refuses a task unless at least one check fails on the seed and every check passes at the end; a file moved to the Trash is left out of the seed, so trash a stray file rather than starting over; a null draft names its cause in draft_refusal; and a line check on a format read as text only approximates a value, so bound it and call it approximate. Co-Authored-By: Claude Opus 5.5 * docs(skills): teach the build skill to use DOCTYPE and whole-text checks The skill now checks text an app splits into runs whole with text_equals or text_contains, finds an AbiWord paragraph by its xid, and drops the Writer revision-id workaround. Co-Authored-By: Claude Opus 5.5 * docs(skills): point text checks at the paragraph, and say what the scrub replaces From the AbiWord rebuild: point a text check at the paragraph or another element the edit keeps, never at a run a reopened document splits; how to name an unchanged element; the export replaces the host name, user@host and home path while a bare user name stays; AbiWord's entry and class. Co-Authored-By: Claude Opus 5.5 * fix(desktop): keep every text check on a cut element undecided, in both evaluators The XML reader drops text deeper than its depth limit but keeps later siblings, and an element's whole text joins both sides, so a text_contains on a cut element could match text the file never holds ("at 3" + deep "WRONG" + ":30 PM" read as "at 3:30 PM"). Any text check on an element of a cut view is now undecided again, as #9050 had it, in REA's evaluator and the exported verifier; a text node or attribute value stays decided. The text check tests assert facts rather than sentences. Co-Authored-By: Claude Opus 5.5 * fix(desktop): keep a reset applied when the screenshot asked after it fails Once data/reset and the ready wait succeed, act_on_build_computer's reset is applied in the action log and returns success; a screenshot asked of it that is then refused or throws is named in the result's screenshot_error, with no frame, so the next after_reset stage is not refused. Cancellation still propagates. Co-Authored-By: Claude Opus 5.5 * fix(desktop): keep checks for elements without text beside a text edit, and normalize DOCTYPE system id line endings A review found two gaps. - A paragraph's whole-text check covered every element added or removed inside it, including ones holding no text: an image frame, a line break. A solver who typed the text but never inserted the image passed. Such an element now keeps its own presence or absence check. - The Python reader kept a DOCTYPE system literal's CRLF where TypeScript normalizes it to LF. Both now report LF. Co-Authored-By: Claude Opus 5.5 * fix(desktop): wait out long build commands, fail a reset that restores nothing, and tie the witness to its bundle A review found five gaps. - Commands in the computer answer only when they end, but fetch stops waiting for headers after 300 s. So a slow export, draft or start always failed. A command with its own deadline now waits on node:http, bounded by that deadline. - after_reset held whenever some check failed, so a reset that changed nothing passed on the near-miss state. Each check must now come out as it did at the seed. - A failed or partial apt-get update left lists that looked complete, so a second start skipped the update. The update is now recorded only when it succeeds, and rerun otherwise. - A witness on a bundle computer graded whatever task it was given. Its seed stage now refuses a task whose steps or checks differ from the bundle's. - A task too large for one command argument (96 KiB) is refused before the export runs, instead of failing with E2BIG in the computer. Co-Authored-By: Claude Opus 5.5 * fix(desktop): advertise a checkpoint-name pattern that compiles in every JS regex mode main's advertised-pattern test compiles every input schema pattern in v mode, which refuses an unescaped hyphen in a character class. The checkpoint name's hyphen is now escaped, and it means the same in u mode. Co-Authored-By: Claude Opus 5.5 * test(desktop): import textSyntaxFormat once in the state format tests main's mixed-case JSON suffix fix and this branch both added the import, and the merge kept both. Co-Authored-By: Claude Opus 5.5 * fix(desktop): keep a completed reset, an unread seed, and a failed apt update A cancel after data/reset no longer drops the restore from the action log, so a near miss cannot treat the restored seed as an edit. An unreadable seed check that later passes leaves after_reset undecided instead of failing the witness. apt-get update is kept only when Error-Mode reports success and a package list was fetched. The controller reply schema lives with the controller, and capture progress lines are read by their domain parser. * fix(desktop): keep a whitespace-only run under the paragraph text check A run whose only text is whitespace is still text the paragraph check reads. It no longer gets its own presence check, which a later save of the same words without that run would fail. Images and breaks, whose text is empty, still keep their own checks. * fix(desktop): keep an unread reset and a cut text check undecided after_reset stays undecided when any check was not read, even if another check still fails and the unread result's boolean matches the seed. A text check for a child past a truncated view is undecided in both evaluators. A reset whose reply is lost blocks a near miss until a later input or a reset whose reply arrives. * fix(desktop): retry a witness recording and a published record's cleanup A recording that fails to end or download is not written into the witness folder, and a manifest already received is fetched again without ending the recording twice. A folder that is published while its handles fail to close keeps that record and retries the same cleanup instead of writing the folder again. * fix(desktop): keep a published witness error and tag an app data refusal A second failure to close a published witness returns the original error, so its record and cleanup lease stay bound. A reset the app itself refuses is an AppDataRefusal; only a reply that is not that refusal stays unsettled. A missing child of a cut view is undecided with the same description in both evaluators, and the shared cases cover it. * fix(desktop): align a cut view's missing text with the Python check A missing child of a truncated view is undecided for the same sentence in both evaluators, including the pointer the view does not contain. * fix(desktop): keep one published witness cleanup across retries A published record and its bound cleanup error stay together. A later close failure returns that error, and the unused path that built a fresh adapter error is gone. --------- Co-authored-by: Claude Opus 5.5 Co-authored-by: Cursor Agent Co-authored-by: morluto Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- .../skills/reverse-engineer-anything/SKILL.md | 10 +- .../references/desktop-build.md | 290 ++++++++++++++++++ .../references/environments.md | 4 +- docs/desktop-application-tasks.md | 5 +- docs/desktop-build-computer.md | 4 +- docs/desktop-build-witness.md | 25 +- .../build-witness/spec.md | 11 +- .../xml-text-checks/spec.md | 8 +- src/application/SetupSkill.ts | 1 + src/domain/desktop/appTaskChecks.fixture.ts | 39 ++- src/domain/desktop/appTaskChecks.test.ts | 107 +++++-- src/domain/desktop/appTaskChecks.ts | 32 +- src/domain/desktop/appTaskDraft.test.ts | 32 ++ src/domain/desktop/appTaskDraftText.ts | 8 +- src/domain/desktop/appTaskPython.ts | 4 +- src/generatedPackageMetadata.ts | 2 +- tests/boundary/filesystem/setupSkill.test.ts | 1 + .../providers/desktopXmlTextPython.test.ts | 67 ++++ 18 files changed, 591 insertions(+), 59 deletions(-) create mode 100644 .agents/skills/reverse-engineer-anything/references/desktop-build.md diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md index e5514ffc16..815c7eeb52 100644 --- a/.agents/skills/reverse-engineer-anything/SKILL.md +++ b/.agents/skills/reverse-engineer-anything/SKILL.md @@ -1,8 +1,8 @@ --- name: reverse-engineer-anything -description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. +description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. metadata: - version: "60" + version: "61" --- # REA @@ -140,6 +140,10 @@ explicit path or endpoint and do not need it. - Task for a desktop application's exported environment, from a `capture-desktop-application` folder: read [desktop application tasks](references/environments.md#tasks-for-a-desktop-application-environment). +- Environment of a Linux desktop app with a task, from a request such as + "build me an Inkscape environment where the agent changes a poster's + headline": read [desktop builds](references/desktop-build.md). You drive the + app on a build computer from `start_build_computer`, then export and witness. - Fiddler Everywhere traffic or a native `.saz` capture: read the [Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging). Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`; @@ -539,6 +543,8 @@ as complete while required questions remain open. [references/environments.md](references/environments.md) - Tasks for a desktop application's exported environment: [references/environments.md](references/environments.md#tasks-for-a-desktop-application-environment) +- A desktop app environment from a request, built and witnessed on build + computers: [references/desktop-build.md](references/desktop-build.md) ## Finish the task diff --git a/.agents/skills/reverse-engineer-anything/references/desktop-build.md b/.agents/skills/reverse-engineer-anything/references/desktop-build.md new file mode 100644 index 0000000000..2a4c739fd1 --- /dev/null +++ b/.agents/skills/reverse-engineer-anything/references/desktop-build.md @@ -0,0 +1,290 @@ +# Desktop app environments from a request + +Use this when someone asks for an environment of a Linux desktop app with a +task, such as "Build me an Inkscape environment where the agent changes a +poster's headline and saves it". You are the building agent: you drive the +app on a build computer through REA's MCP tools, write the task, export the +bundle and witness it in a fresh computer. REA runs no model. The tools are +MCP-only and need Docker, `REA_AGENT_ENV_COMMAND` and +`REA_BUILD_COMPUTER_REA_ARCHIVE` where the REA server starts; when a start +reports one missing, tell the person what to set and stop. + +For the whole build: + +- Drive the app only with `act_on_build_computer`. Never script it, run + commands in the computer or write its files another way. +- Ask the upfront questions once, then work on your own. Pause only when + blocked. +- The seed's content is synthetic. Never put real personal data in it. +- Close every computer you start, on success and on failure. +- Note the time you start; the report gives the build time. + +## Ask once, upfront + +In one message, ask only what the request leaves open, and keep the list for +the report: + +1. The app and the task, if either is unclear: what changes, in which + document. +2. Whether they have a starting file of their own; otherwise you make one. +3. Whether they want solver runs after the witness: how many (the budget) + and with which model. +4. If so, whether they want to review the task before those runs. + +## When you are blocked + +Stop, say what you need and wait only for: + +- a licence the app or its installer asks someone to accept: never accept it + yourself; name it and where to read it, and go on only once the person + says they accept it; +- credentials: they may only be typed as handles you never see, and the + build tools have none yet, so say the app needs a sign-in and stop. Never + ask for a password in chat; +- a signed-in state in the seed: ask before seeding one; +- an install this flow cannot do, below. + +Decide everything else yourself and note it in the report. + +## Choose the install + +The computer is Ubuntu 24.04 (noble) on this host's architecture only. Use +the first source that has the app: + +| Source | `install` | +| ---------------------------- | --------------------------------------------------------------------------------- | +| Ubuntu archive | `{"kind": "archive", "package": "inkscape"}` | +| PPA or vendor apt repository | `{"kind": "apt_repository", "package", "uri", "suite", "components", "key_file"}` | +| `.deb` by URL | `{"kind": "package_file", "url", "sha256", "architecture", "dependencies"}` | + +- A PPA's `uri` is `https://ppa.launchpadcontent.net/OWNER/NAME/ubuntu`, + `suite` `noble`, `components` `["main"]`. `key_file` is the repository's + signing key saved on this host; for a PPA, fetch it from + keyserver.ubuntu.com by the fingerprint its Launchpad page shows. +- For a `.deb`, ask the requester for its URL and SHA-256; do not fetch a + vendor's download page yourself. List its `Depends` as `dependencies`: + it installs with the network off. +- Flatpak, AppImage and snaps are not supported yet: say so and stop. An + archive package that only installs a snap, such as noble's `firefox`, + counts as a snap. A package with no build for this host's architecture + (some PPAs are amd64 only) cannot be installed either. +- `desktop_entry` is the package's `.desktop` file name under + `/usr/share/applications` without `.desktop`, such as + `org.inkscape.Inkscape`. Add `window_class` when you know it; Inkscape's + is `Inkscape`. AbiWord's entry is `abiword` and its class `Abiword`. + +## Drive the app + +- Pass `computer_id` and the action's fields at the top level: click + `{x, y, count}`, scroll `{x, y, direction, amount}`, a screenshot region + `[x0, y0, x1, y1]`. The rest are in + [the build-computer guide](https://github.com/morluto/rea/blob/main/docs/desktop-build-computer.md). +- Take a `screenshot` before pointing: points are pixels of the latest + frame. If a response is too large, ask for a `region`, or `jpeg` with a + lower `quality`. A region crops the latest frame and becomes it: take a + full screenshot before pointing again. +- A drawn caret does not prove keyboard focus. After using a toolbar + control, click into the document, then type a few characters and look + before a long text. +- Confirm a dialog with `key` Return, not a trailing `\n` in `type`. +- An action's own screenshot is taken at once. After opening a menu, a + dialog or a window, look with + `{"action": "wait", "ms": 500, "screenshot": true}`. +- On a computer started from a request the app is not open at first, and + `start_build_capture` opens it: never open it yourself before that. Do not + close it while capturing; the capture ends when the app exits. When the + app is closed at any other time, open it from Applications → Other → the + environment's name. +- Prefer shortcuts and typing to long pointer paths: `key` takes `keys`, an + array holding one chord's keys, such as + `{"action": "key", "keys": ["ctrl", "s"]}`, and `type` turns `\n` into + Enter. +- Before `type`, check in a screenshot that a text cursor or a focused + field is there. In apps with single-key shortcuts, such as Inkscape, text + typed without one runs as commands. +- Save As opens in the home: click Documents in its sidebar and type only + the file name. Answer the app's save questions, such as keeping the + format, then check the saved path in the title or status bar, or the + dialog's folder when the app shows only the name, and that it shows no + unsaved mark, before you mark a checkpoint or check. +- What the app saved under the home before `seed`, such as a document in + `~/Documents` or its settings, is part of the seed. A file you move to the + Trash is left out, and the export lists it in `limitations`. Trash a stray + file rather than starting over. +- Dismiss first-run dialogs, such as a welcome screen, before `seed`. A + choice the app saves only later, such as Inkscape's "show this every + time", may not reach the seed, so expect the dialog in the witness too. +- Give what the task changes a stable name where the app has one, such as an + id in Inkscape's Object Properties, so a check can select it. +- Expect apps to split text into runs: Writer's revision ids split an + edited paragraph into spans and may give it an automatic style such as + `P1`, and AbiWord wraps each insertion in ``. Check such text whole + with `text_equals` (see Write the task), and find the paragraph by + something the edit keeps, such as AbiWord's `xid`. + +## Build the seed and the edit + +1. `start_build_computer` with + `{"source": {"kind": "request", "desktop_entry": ..., "install": ...}}`. + The first start of an app builds its image and can take minutes. A + failed start returns the build's error: fix the request, or stop when the + install cannot work, and close any `computer_id` the error names. +2. `start_build_capture`. It returns once the app's window is open. A + `package_lists_error` may stop the capture finding the package's + sources; the export says when that matters. +3. Make the seed through the app and save it from the app under + `~/Documents`: build the document in the app, or write its source where + the app edits its format as text, then save it again from the app. Make + it look real but synthetic: the thing the task changes, plus a few things + the change must leave alone. For a starting file of the requester's, + read it here and rebuild it in the app with synthetic data in place of + personal data; REA cannot copy a file into the computer. +4. Only once the seed is saved, `mark_build_checkpoint` with name `seed`. +5. Make the change the task asks for, as a user would, and save it. +6. Only once the change is saved, `mark_build_checkpoint` with name `end`. +7. `stop_build_capture`. Read `draft.files` (each file that changed, with + its net change), `draft.task.checks` (suggested checks) and + `limitations`. When `draft` is null, read `draft_refusal` first: a + missing `seed` or `end` checkpoint means capturing again on a new build + computer, while a failed draft command names its own cause to fix. + +## Write the task + +Write `{"name", "prompt", "checks"}` from the draft and what you did. + +- `name`: lowercase letters, digits and hyphens, not `smoke`. +- `prompt`: what the requester wants, in their words and the app's terms, + naming the document as a user would and ending with saving it: "Change the + poster's headline in Documents/poster.svg to Summer Bake Sale and save + it." Never say how the file stores it, and never give the checks. +- Checks are file-based: content checks that prove the edit, never only that + a file exists or changed. Start from the suggestions; keep those that say + what the user did and drop what follows from how the app saves. +- Compare exact values, so a typo or a wrong number fails. +- Select elements by what they hold, never by position: SVG text by id, + `/root/children/[name=g][@inkscape:label=Layer 1]/children/[name=text][@id=headline]/children/[name=tspan]/children/0`, + or an ODF paragraph by style, + `/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:text]/children/[name=text:p][@text:style-name=Title]/children/0`. + A selector must match exactly one element, so give the target a unique id + or style in the seed. The draft may suggest positional pointers; rewrite + them. Only an element's text is read by position, as `children/0`. +- When an app may split an element's text into runs, check it whole: + `text_equals` (or `text_contains` for a value inside a sentence) on the + element compares all its text, however it is split. Add + `normalize_whitespace: true` when the app may reflow spaces. Example, an + AbiWord paragraph by its stable id: + `{"kind": "text_contains", "file": "Documents/memo.abw", "pointer": "/root/children/[name=section]/children/[name=p][@xid=6]", "value": "at 3:30 PM"}`. + Point a text check at the paragraph or another element the edit keeps, + never at a run such as AbiWord's `[name=c]`: a reopened document splits + the text into several runs. The draft lists only what changed; to name an + unchanged element, such as another paragraph's `xid`, use the app's usual + numbering and let the export confirm it, or read it in the exported + bundle's `envs//seed/`. +- Where easy, add one check that the rest is untouched, such as another + text's value the edit must leave alone. It is the one unchanged value + worth checking; it passes on the seed too, which is fine, since the export + needs only one check to fail there. +- Give `value_equals` a `tolerance` for a number the app writes as text. +- When the draft shows an XML document's changes as `lines`, REA read it + as text, for example because its DOCTYPE declares entities. Selectors are + impossible there, and `line_count` with `contains` only approximates a + value: `3:30` also matches `13:30`. Bound it with neighbouring text and + name the check as approximate in the report's limitations. +- Some apps save a document reloaded from disk differently from one typed + in the same session: AbiWord wraps each new insertion in its own markup. + Your build's end state hides this, which is why whole-text checks matter; + expect the witness to show the real format. + +The kinds and pointer rules are in +[the desktop tasks section](environments.md#pick-the-checks). + +## Export + +`export_build_environment` with `task` and `directory`, the absolute path of +a new or empty folder here, such as `/rea-builds/poster/bundle`. The +export seeds what was saved at `seed`, replaces the computer's host name, +`user@host` and home path (a bare user name, such as AbiWord's author +`user`, stays), and checks the task against `seed` and `end`. It refuses a task +unless at least one check fails on the seed and every check passes at the +end, and says why, such as a pointer that names nothing at the end. A +check that the rest is untouched passes on the seed too, which is fine. On +a refusal, fix the task and export again, into another folder if that one +is no longer empty. Keep +`seed_files`, `scrubbed_files`, `unscrubbed_files` and `limitations` for the +report. + +Keep the build computer open until the witness holds: a task fix exports +again from it. + +## Witness + +Witness in a fresh computer of the bundle, from the prompt, as a solving +agent would. Do not replay the build's actions or coordinates, and do not +check between stages to steer the edit: a solving agent never sees the +checks. Give every `check_build_task` the task exactly as exported. + +1. `start_build_computer` with + `{"source": {"kind": "bundle", "path": ""}}`. The app starts + with the seed's files in place but need not open the document: open it + from the prompt's path, as a solving agent would. Take a `wait` + screenshot first, since the desktop may not have drawn yet. +2. `check_build_task` with `stage: "seed"` and `directory`, the absolute + path of a new or empty folder for the record. A check must fail. + Recording starts. Later stages take no `directory`. +3. Do the task from the prompt with `act_on_build_computer` and save. + `check_build_task` with `stage: "after_edit"`: every check must pass. +4. Make a near miss on top of it: a plausible wrong edit, such as a typo or + a wrong number, and save. `check_build_task` with `stage: "near_miss"`: + a check must fail. +5. `act_on_build_computer` with `action: "reset"`, then `check_build_task` + with `stage: "after_reset"`: a check must fail again. +6. The last stage ends the witness: `witness.record` has the verdict, the + path of `witness.json` and the recording's segments in `directory`. + `close_build_computer` on the witness computer. + +A stage that does not hold ends the witness as `failed`. Read each check's +result and decide: + +- `after_edit` failed though the saved file is right: the check is brittle. + Fix it, export again and witness in a new computer. +- `after_edit` failed because your edit was wrong: witness again in a new + computer; a computer is witnessed once. +- `near_miss` passed: the checks are too loose. Tighten them, export again + and witness again. +- `after_reset` passed: the reset did not restore the seed. Report it as a + limitation. + +Stop after three witnesses and report what still fails. + +## Solver runs + +Only if the requester asked for them. If they wanted to review the task +first, show them its prompt and checks and wait for their approval. Then run +the bundle within the budget, one run at a time, with the runner +`REA_AGENT_ENV_COMMAND` names: +`agent-env run --task --agent claude-code --model `. +Count runs reported `passed`. A run `scored below 1` is a miss; one reported +`failed` did not finish, so run it again if the budget allows instead of +counting it. Never run past the budget. + +## Close and report + +`close_build_computer` on every computer still open, also after a failure. +If a close fails, call it again; if it still fails, tell the requester what +is left. + +Report: + +- the bundle path and the environment name; +- the task's name and prompt; +- each check, and why it proves the edit and fails the near miss; +- each witness stage with its result, the verdict, and the paths of + `witness.json` and the recording; +- the seed's contents: `seed_files` and what each holds; +- `scrubbed_files` and `unscrubbed_files`; an unscrubbed file still names + the host or user, so it needs review before sharing; +- limitations: the capture's, the export's and anything you could not do; +- solver runs, if any: the model, the runs and the passes; +- the build time and the questions you asked. + +Never publish or share the bundle yourself. diff --git a/.agents/skills/reverse-engineer-anything/references/environments.md b/.agents/skills/reverse-engineer-anything/references/environments.md index 327ceab171..63dd4e78b6 100644 --- a/.agents/skills/reverse-engineer-anything/references/environments.md +++ b/.agents/skills/reverse-engineer-anything/references/environments.md @@ -294,7 +294,9 @@ Use this when the user has a folder from `rea capture-desktop-application` and wants a task for the environment `rea export-desktop-application` makes from it: a prompt for a computer-use agent and checks that grade what the agent left in the app's files. You write the prompt and the checks; REA validates -the task against the capture and exports it. These commands are CLI only. +the task against the capture and exports it. These commands are CLI only. To +build the environment and its task from a request instead, follow +[desktop builds](desktop-build.md). ### Describe the change diff --git a/docs/desktop-application-tasks.md b/docs/desktop-application-tasks.md index 0ff09bbf3a..bc01fa5118 100644 --- a/docs/desktop-application-tasks.md +++ b/docs/desktop-application-tasks.md @@ -141,7 +141,10 @@ The rules both evaluators follow: - A check decided by what a view leaves out (`lines_absent`, `value_absent`, `row_absent`, the counts, `value_equals` of a whole object or array, and the text checks of an element) fails when that view is truncated; a check - satisfied by what the view holds is decided on it. + satisfied by what the view holds is decided on it. An element's text is not: + the reader may have dropped text from between the runs it kept, so + `text_contains` fails on a truncated view even where the kept runs hold its + value. A text node or attribute value is whole, and decided. The export refuses, naming the check, a file outside the owned paths, a file spelled with a leading slash, `.` or `..`, a pointer into a file that is text diff --git a/docs/desktop-build-computer.md b/docs/desktop-build-computer.md index 3ff1ce029c..979b92651f 100644 --- a/docs/desktop-build-computer.md +++ b/docs/desktop-build-computer.md @@ -14,7 +14,9 @@ agent's computer-use actions to it through three MCP tools: The tools are MCP-only, like the debugger sessions: the computer lives as long as the REA MCP server that started it, or until it is closed. Capturing the app, exporting its bundle and witnessing the task on such a computer are in -[Building and witnessing an environment](desktop-build-witness.md). +[Building and witnessing an environment](desktop-build-witness.md), and REA's +skill for doing all of it from a request is +[desktop builds](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/desktop-build.md). ## Requirements diff --git a/docs/desktop-build-witness.md b/docs/desktop-build-witness.md index b1d8a4e642..f15af6cfaa 100644 --- a/docs/desktop-build-witness.md +++ b/docs/desktop-build-witness.md @@ -18,6 +18,12 @@ grades the witness with the same checks the exported verifier runs. Like the computer tools, these are MCP-only and act on a `computer_id` from `start_build_computer`. +REA's skill walks an agent from a request such as "build me an Inkscape +environment where the agent changes a poster's headline" to a witnessed +bundle with these tools: the questions it asks first, when it stops for the +person, the install, the seed, the task, the witness and its final report. +See [desktop builds](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/desktop-build.md). + ## Build 1. `start_build_computer` from a request. REA is installed in the computer. @@ -137,14 +143,17 @@ SQLite file locked past the reader's wait, or because what it looks for may lie in what its view cut, is `undecided` in its result: it shows nothing about the task. That covers a line `lines_present` misses in a text view cut at 5,000 lines, a row `row_present` misses in a table whose view keeps only -its first rows, a `value_equals` pointer that names nothing in a truncated -view or past a cut zip listing, and the absence checks and counts on a cut -view. A check that passes on what the view kept stays decided, and so does a -value found there that differs from the one expected. The result's `passed` -and description are the exported verifier's, which has no undecided results. -A stage that only undecided checks would settle, such as a near miss whose -one failing check could not read its file, files nothing and is refused; -check it again once the files read whole. +its first rows, a `value_equals` or text check pointer that names nothing +in a truncated view or past a cut zip listing, the absence checks and counts +on a cut view, and `text_equals` or `text_contains` of an element on one, +even where the element's kept text holds the value, since the cut may have +dropped text from between its runs. Any other check that passes on what the +view kept stays decided, and so does a value found there that differs from +the one expected. +The result's `passed` and description are the exported verifier's, which has +no undecided results. A stage that only undecided checks would settle, such +as a near miss whose one failing check could not read its file, files nothing +and is refused; check it again once the files read whole. A stage whose checks do not show what it must ends the witness, and so does `after_reset`. REA then ends the recording, fetches its segments through diff --git a/docs/specs/desktop-app-environments/build-witness/spec.md b/docs/specs/desktop-app-environments/build-witness/spec.md index 2909eae71d..5576b02c66 100644 --- a/docs/specs/desktop-app-environments/build-witness/spec.md +++ b/docs/specs/desktop-app-environments/build-witness/spec.md @@ -94,11 +94,12 @@ and the recording and writes the record as Evidence. want of a readable or whole view is `undecided` and proves nothing; a stage only such checks would settle files nothing and can be checked again. Undecided includes a failure to find a line, row or value that may - lie in what a cut view left out; a check that passes on what the view kept - stays decided. The export likewise keeps a task only when a check fails on - what the seed's view shows. The controller writes a folder's file list to a - file that REA reads through `data/get`, so a large bundle's list is not cut - at the 1 MiB of a command's output. + lie in what a cut view left out, and any text check of an element in a cut + view, which may have lost text from between the element's runs; any other + check that passes on what the view kept stays decided. The export likewise keeps a task only when a check + fails on what the seed's view shows. The controller writes a folder's file + list to a file that REA reads through `data/get`, so a large bundle's list + is not cut at the 1 MiB of a command's output. - The witness reads the build computer's bounded action log: its stage rules use the log's count and the last applied action of each kind, and the record holds the entries the log keeps, `actions_dropped`, and a diff --git a/docs/specs/desktop-app-environments/xml-text-checks/spec.md b/docs/specs/desktop-app-environments/xml-text-checks/spec.md index dfc9aa9e64..34cfdc5996 100644 --- a/docs/specs/desktop-app-environments/xml-text-checks/spec.md +++ b/docs/specs/desktop-app-environments/xml-text-checks/spec.md @@ -55,9 +55,11 @@ verifier's, agree. entry of another format; a value that is neither an element, a text node (a string among an element's `children`) nor an attribute's value (one among its `attributes`), so an element's `name` and the `doctype`'s fields fail; - and, as undecided, an element in a view that leaves something out, or a - selector searching an array such a view may cut. A text node is whole, so - it is decided on a truncated view. + and, as undecided, in a view that leaves something out, either check of an + element, even one whose kept text holds the value, since the cut may have + dropped text from between the runs it kept; a pointer naming nothing; or a + selector searching an array such a view may cut. A text node or an + attribute's value is whole, so it is decided on a truncated view. - Refusals: the task parser refuses the empty pointer, a pointer not starting with `/`, an empty `text_contains` value or one that is only whitespace with `normalize_whitespace`, and a value that looks like a date and time, diff --git a/src/application/SetupSkill.ts b/src/application/SetupSkill.ts index 14883e8105..56947b8d7c 100644 --- a/src/application/SetupSkill.ts +++ b/src/application/SetupSkill.ts @@ -34,6 +34,7 @@ export const MANAGED_SKILL_FILES = [ "references/runtime-observation.md", "references/evidence-workflows.md", "references/environments.md", + "references/desktop-build.md", ] as const; export const isManagedSkillManifest = (content: string): boolean => diff --git a/src/domain/desktop/appTaskChecks.fixture.ts b/src/domain/desktop/appTaskChecks.fixture.ts index cf02dea3f1..777b6a2972 100644 --- a/src/domain/desktop/appTaskChecks.fixture.ts +++ b/src/domain/desktop/appTaskChecks.fixture.ts @@ -814,8 +814,6 @@ const TEXT_CASES: readonly (readonly [DesktopTaskCheckInput, boolean])[] = [ ], ]; -/** Every kind of check against those views, each with the outcome both evaluators must give. */ - /** Failures a cut view may have caused, which both evaluators fail alike and REA marks undecided. */ const CUT_VIEW_CASES: readonly (readonly [DesktopTaskCheckInput, boolean])[] = [ [{ kind: "lines_present", file: "Documents/long.txt", lines: ["z"] }, false], @@ -838,8 +836,45 @@ const CUT_VIEW_CASES: readonly (readonly [DesktopTaskCheckInput, boolean])[] = [ }, false, ], + [ + { + kind: "text_contains", + file: "Documents/cut.xml", + pointer: "/root", + value: "gone", + }, + false, + ], + [ + { + kind: "text_contains", + file: "Documents/cut.xml", + pointer: "/root", + value: "kept", + }, + false, + ], + [ + { + kind: "text_equals", + file: "Documents/cut.xml", + pointer: "/root/children/5", + value: "x", + }, + false, + ], + [ + { + kind: "text_contains", + file: "Documents/many.zip", + pointer: "/entries/1100/view/root", + value: "x", + }, + false, + ], ]; +/** Every kind of check against those views, each with the outcome both evaluators must give. */ export const CHECK_CASES: readonly (readonly [ DesktopTaskCheckInput, boolean, diff --git a/src/domain/desktop/appTaskChecks.test.ts b/src/domain/desktop/appTaskChecks.test.ts index b3d5be855c..c71e687c38 100644 --- a/src/domain/desktop/appTaskChecks.test.ts +++ b/src/domain/desktop/appTaskChecks.test.ts @@ -1,13 +1,15 @@ import { describe, expect, it } from "vitest"; +import type { JsonValue } from "../jsonValue.js"; import { parseDesktopTask } from "./appTask.js"; import { + elementText, evaluateTaskChecks, type TaskCheckOutcome, type TaskFileView, } from "./appTaskChecks.js"; import { CHECK_CASES, CHECK_VIEWS } from "./appTaskChecks.fixture.js"; -import { textView } from "./stateFormats.js"; +import { textView, xmlView } from "./stateFormats.js"; const portable = (name: string) => /^[a-z0-9]+(?:-[a-z0-9]+)*$/u.test(name); @@ -444,6 +446,84 @@ describe("evaluateTaskChecks on cut views", () => { }); }); +describe("evaluateTaskChecks text on a cut view", () => { + /** A paragraph of `before`, then `dropped` nested past the 64 levels an XML view keeps, then `after`. */ + const nested = (before: string, dropped: string, after: string) => + `

${before}${"".repeat(65)}${dropped}${"".repeat(65)}${after}

`; + const views = new Map([ + [ + "page.xml", + { + format: "xml", + view: { + root: { name: "doc", attributes: {}, children: ["kept"] }, + truncated: true, + }, + }, + ], + [ + "time.xml", + { + format: "xml", + view: xmlView(nested("at 3", "WRONG", ":30 PM")) ?? null, + }, + ], + [ + "word.xml", + { format: "xml", view: xmlView(nested("alpha", "zzz", "beta")) ?? null }, + ], + ]); + const text = (kind: string, file: string, pointer: string, value: string) => + evaluateTaskChecks( + [{ id: "c", kind, file, pointer, value } as never], + views, + )[0]; + const cut = { id: "c", passed: false, undecided: true } as const; + + it("leaves both text checks of an element in a cut view undecided, even where the runs it kept hold the value", () => { + // The reader drops the nested text but keeps the run after it, so the kept runs read as text the file never had. + const keptText = (file: string) => + elementText( + (views.get(file)?.view as { root: JsonValue } | undefined)?.root ?? + null, + ); + expect(keptText("time.xml")).toBe("at 3:30 PM"); + expect(keptText("word.xml")).toBe("alphabeta"); + + for (const [file, value] of [ + ["time.xml", "at 3:30 PM"], + ["word.xml", "alphabeta"], + ["page.xml", "kept"], + ["page.xml", "gone"], + ] as const) + for (const kind of ["text_contains", "text_equals"]) + expectOutcome(text(kind, file, "/root", value), cut, [file]); + }); + + it("decides a text node or an attribute value in a cut view, and leaves a pointer naming nothing there undecided", () => { + expectOutcome( + text("text_equals", "time.xml", "/root/children/0", "at 3"), + { id: "c", passed: true }, + ["time.xml", "/root/children/0", quoted("at 3")], + ); + expectOutcome( + text("text_contains", "time.xml", "/root/attributes/id", "t"), + { id: "c", passed: true }, + ["time.xml", "/root/attributes/id", quoted("t")], + ); + expectOutcome( + text("text_equals", "page.xml", "/root/children/0", "gone"), + { id: "c", passed: false }, + ["page.xml", "/root/children/0", quoted("kept"), quoted("gone")], + ); + expectOutcome( + text("text_equals", "page.xml", "/root/children/1", "kept"), + cut, + ["page.xml", "/root/children/1"], + ); + }); +}); + describe("evaluateTaskChecks selectors and tolerance", () => { const plate = "Documents/plate.FCStd"; const objects = "/entries/[name=Document.xml]/view/root/children"; @@ -578,26 +658,17 @@ describe("evaluateTaskChecks whole text", () => { const meeting = "Documents/meeting.xml"; const plate = "Documents/plate.FCStd"; const time = "/root/children/[name=p][@id=time]"; - /** A value as a description shows it: caller data and what the view holds, as JSON. */ - const quoted = (value: unknown) => JSON.stringify(value); - const expectFacts = ( - found: { passed: boolean; description: string } | undefined, - passed: boolean, - facts: readonly string[], - ) => { - expect(found?.passed).toBe(passed); - for (const fact of facts) expect(found?.description).toContain(fact); - }; + const failed = { id: "c", passed: false } as const; it("reports the text an element has, its runs joined, beside the value it was compared with", () => { - expectFacts( + expectOutcome( outcome({ kind: "text_equals", file: meeting, pointer: time, value: "Starts at 3:45 today.", }), - false, + failed, [ meeting, time, @@ -605,14 +676,14 @@ describe("evaluateTaskChecks whole text", () => { quoted("Starts at 3:45 today."), ], ); - expectFacts( + expectOutcome( outcome({ kind: "text_contains", file: meeting, pointer: time, value: "3:45", }), - false, + failed, [quoted("Starts at 3:30 today."), quoted("3:45")], ); const collapsed = outcome({ @@ -622,7 +693,7 @@ describe("evaluateTaskChecks whole text", () => { value: "Doors open", normalize_whitespace: true, }); - expectFacts(collapsed, true, [quoted("Doors open")]); + expectOutcome(collapsed, { id: "c", passed: true }, [quoted("Doors open")]); expect(collapsed?.description).not.toContain(quoted("Doors open")); }); @@ -659,7 +730,7 @@ describe("evaluateTaskChecks whole text", () => { ]; for (const [check, facts] of refused) - expectFacts(outcome({ kind: "text_equals", ...check }), false, facts); + expectOutcome(outcome({ kind: "text_equals", ...check }), failed, facts); }); it("leaves an element of a truncated view undecided as a value check is, while a text node there is decided", () => { @@ -703,7 +774,7 @@ describe("evaluateTaskChecks whole text", () => { expect(missing?.passed).toBe(false); expect(missing?.undecided).toBe(true); expect(missing?.description).toBe( - "the view of Documents/cut.xml is truncated, so what it leaves out is unknown", + "the view of Documents/cut.xml is truncated, so what it leaves out is unknown (/root/children/2)", ); const zipped = outcome({ kind: "text_equals", diff --git a/src/domain/desktop/appTaskChecks.ts b/src/domain/desktop/appTaskChecks.ts index 9598099296..165adbe5a3 100644 --- a/src/domain/desktop/appTaskChecks.ts +++ b/src/domain/desktop/appTaskChecks.ts @@ -240,12 +240,17 @@ const unreadOrPartial = ( const NOTHING: Located = { kind: "none", searched: [], byPosition: [] }; -/** A value found where its pointer leads, or why it is none. */ -const noValue = (at: string, located: Located & { kind: "none" }) => - fail( +/** Why a pointer names no value, undecided when its view is cut, since the value may lie past what the view kept. */ +const noValue = ( + at: string, + located: Located & { kind: "none" }, + cut: boolean, +): Verdict => + notFound( located.reason === undefined ? `${at} has no value` : `${at}: ${located.reason}`, + cut, ); const valueVerdict = ( @@ -272,13 +277,7 @@ const valueVerdict = ( check.value !== null; if (whole && truncated) return undecided(check.file); if (check.kind === "value_equals") { - if (located.kind === "none") - return notFound( - located.reason === undefined - ? `${at} has no value` - : `${at}: ${located.reason}`, - truncated, - ); + if (located.kind === "none") return noValue(at, located, truncated); if (check.tolerance !== undefined && typeof check.value === "number") return nearVerdict(at, located.value, check.value, check.tolerance); return sameJson(located.value, check.value) @@ -383,12 +382,18 @@ const textVerdict = ( const problem = zip ? notXmlEntry(check, read.view) : undefined; if (problem !== undefined) return fail(problem); if (located.kind === "unresolved") return fail(`${at}: ${located.problem}`); + // A zip view's truncation that matters is the entry the pointer goes into. + // A child past the kept nodes of a cut view may still hold the text. const truncated = zip ? zipLeavesOutAt(read.view, check.pointer) : read.truncated; - // A child past the kept nodes of a cut view may still hold the text. + // A missing child of a cut view is the same undecided result in both evaluators. if (located.kind === "none") - return truncated ? undecided(check.file) : noValue(at, located); + return truncated + ? unread( + `the view of ${check.file} is truncated, so what it leaves out is unknown (${check.pointer})`, + ) + : noValue(at, located, false); const found = located.value; if ( typeof found === "string" @@ -398,7 +403,8 @@ const textVerdict = ( return fail( `${at} is ${shown(found)}, not an element, a text node or an attribute value`, ); - // A text node is whole; an element's text is whole only if its view left nothing out. + // A text node is whole. An element's text in a cut view may lack what the cut + // dropped from its middle, so its kept runs can read as text it never had. if (typeof found !== "string" && truncated) return undecided(check.file); return comparedText(check, at, elementText(found)); }; diff --git a/src/domain/desktop/appTaskDraft.test.ts b/src/domain/desktop/appTaskDraft.test.ts index 061fb4e478..5cdd203467 100644 --- a/src/domain/desktop/appTaskDraft.test.ts +++ b/src/domain/desktop/appTaskDraft.test.ts @@ -1027,6 +1027,38 @@ describe("draftTaskFiles for elements without text beside a text edit", () => { }); }); +describe("draftTaskFiles for a whitespace-only run", () => { + it("covers a whitespace-only run with the paragraph text check and does not require that run", () => { + const file = "Documents/note.abw"; + const note = (sixth: JsonValue[]): JsonValue => ({ + root: xmlElement("abiword", {}, [ + xmlElement("section", { xid: "1" }, [ + xmlElement("p", { xid: "6" }, sixth), + ]), + ]), + truncated: false, + }); + const edited = ["Hello", xmlElement("c", {}, [" "]), "there"]; + const [draftFile] = draft({ + [file]: path("document", "xml", note(["Hello"]), note(edited)), + }); + const paragraph = "/root/children/[name=section]/children/[name=p]"; + expect(draftFile?.suggested).toEqual([ + { + kind: "text_equals", + file, + pointer: paragraph, + value: "Hello there", + }, + ]); + expect( + [edited, ["Hello", " ", "there"]].map((saved) => + passing(file, draftFile?.suggested, note(saved)), + ), + ).toEqual([[true], [true]]); + }); +}); + describe("draftTaskFiles with nullable SQLite keys", () => { it("suggests checks for distinct NULL-key rows and ignores a reorder", () => { const file = ".local/share/app/items.db"; diff --git a/src/domain/desktop/appTaskDraftText.ts b/src/domain/desktop/appTaskDraftText.ts index 97a8434fc1..ff1b1bd806 100644 --- a/src/domain/desktop/appTaskDraftText.ts +++ b/src/domain/desktop/appTaskDraftText.ts @@ -83,11 +83,15 @@ const namedByIdentifier = (pointer: string) => { return key !== undefined && isIdentifier(key); }; -/** Whether an end holds an element at a pointer whose text a check reads as nothing, as an image or a break. */ +/** + * Whether an end holds an element whose text a check reads as nothing, as an + * image or a break. A run of only whitespace is text the paragraph check + * already reads, so it stays covered. + */ const textlessAt = (pointer: string, ends: Ends) => [ends.baseline, ends.final].some((view) => { const value = resolvePointer(view, pointer); - return isXmlElement(value) && isWhitespace(elementText(value)); + return isXmlElement(value) && elementText(value) === ""; }); /** Whether both ends hold an element at a pointer. */ diff --git a/src/domain/desktop/appTaskPython.ts b/src/domain/desktop/appTaskPython.ts index d6f6bd7d8e..5939bf4dde 100644 --- a/src/domain/desktop/appTaskPython.ts +++ b/src/domain/desktop/appTaskPython.ts @@ -339,11 +339,11 @@ def _whole_text(check, entry): truncated = _zip_leaves_out(view, check["pointer"]) if kind == "none": if truncated: - return _undecided(check) + return False, f"the view of {check['file']} is truncated, so what it leaves out is unknown ({check['pointer']})" return False, f"{at} has no value" if found is None else f"{at}: {found}" if not (_text_at(view, check["pointer"]) if isinstance(found, str) else _is_element(found)): return False, f"{at} is {_shown(found)}, not an element, a text node or an attribute value" - # A text node is whole; an element's text is whole only if its view left nothing out. + # A text node is whole. An element's text in a cut view may lack what the cut dropped from its middle. if not isinstance(found, str) and truncated: return _undecided(check) return _compared_text(check, at, _element_text(found)) diff --git a/src/generatedPackageMetadata.ts b/src/generatedPackageMetadata.ts index ce2709114c..27795bfbd8 100644 --- a/src/generatedPackageMetadata.ts +++ b/src/generatedPackageMetadata.ts @@ -6,5 +6,5 @@ export const PACKAGE_METADATA = { serverSdkVersion: "2.3.1", clientSdkVersion: "2.3.1", coreSdkVersion: "2.3.1", - skillVersion: "60", + skillVersion: "61", } as const; diff --git a/tests/boundary/filesystem/setupSkill.test.ts b/tests/boundary/filesystem/setupSkill.test.ts index f4166f17c5..fc5f6f7ab0 100644 --- a/tests/boundary/filesystem/setupSkill.test.ts +++ b/tests/boundary/filesystem/setupSkill.test.ts @@ -654,6 +654,7 @@ describe("canonical skill transaction", () => { "SKILL.md", "references/android-applications.md", "references/connection-and-recovery.md", + "references/desktop-build.md", "references/environments.md", "references/evidence-workflows.md", "references/javascript-applications.md", diff --git a/tests/boundary/providers/desktopXmlTextPython.test.ts b/tests/boundary/providers/desktopXmlTextPython.test.ts index a63a4bcd4b..44e444d7c9 100644 --- a/tests/boundary/providers/desktopXmlTextPython.test.ts +++ b/tests/boundary/providers/desktopXmlTextPython.test.ts @@ -5,7 +5,9 @@ import { pathToFileURL } from "node:url"; import { describe, expect, it } from "vitest"; import type { DesktopTaskCheck } from "../../../src/domain/desktop/appTask.js"; +import { elementText } from "../../../src/domain/desktop/appTaskChecks.js"; import { xmlView } from "../../../src/domain/desktop/stateFormats.js"; +import type { JsonValue } from "../../../src/domain/jsonValue.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { expectSameCheckResults, @@ -209,4 +211,69 @@ describe("whole-text checks in both evaluators", () => { // The value searched for is the collapsed one, " 9:30\n" read as "9:30". expect(after?.[1]?.description).toContain(JSON.stringify("9:30")); }); + + it("leaves an element's text undecided where the readers dropped text from between its runs, and its attribute decided", async () => { + const deep = "Documents/deep.xml"; + /** A paragraph of `before`, then `dropped` nested past the 64 levels both readers keep, then `after`. */ + const paragraph = (before: string, dropped: string, after: string) => ({ + name: "deep.xml", + bytes: new TextEncoder().encode( + `

${before}${"".repeat(65)}${dropped}${"".repeat(65)}${after}

`, + ), + }); + const files = [ + paragraph("at 3", "WRONG", ":30 PM"), + paragraph("alpha", "zzz", "beta"), + ]; + await expectSameFileViews(files); + // The kept runs join into text neither file has. + const kept = await Promise.all( + files.map(async (file) => { + const { view } = await typescriptFileView(file); + return elementText((view as { root: JsonValue }).root); + }), + ); + expect(kept).toEqual(["at 3:30 PM", "alphabeta"]); + + const results = await expectSameCheckResults( + deep, + [ + { + id: "time", + kind: "text_contains", + file: deep, + pointer: "/root", + value: "at 3:30 PM", + }, + { + id: "word", + kind: "text_contains", + file: deep, + pointer: "/root", + value: "alphabeta", + }, + { + id: "id", + kind: "text_equals", + file: deep, + pointer: "/root/attributes/id", + value: "t", + }, + ], + files, + ); + + expect(results).toHaveLength(files.length); + for (const outcomes of results) { + expect( + outcomes.map(({ id, passed, undecided }) => [id, passed, undecided]), + ).toEqual([ + ["time", false, true], + ["word", false, true], + ["id", true, undefined], + ]); + for (const { description } of outcomes) + expect(description).toContain(deep); + } + }); });