feat(skills): build a computer-app environment from a request (#9049)

* feat(desktop): capture, draft, export and witness a task inside a build computer

- start_build_capture runs REA's capture inside the computer through a
  small controller over computer-exec, after a user-space apt-get update;
  mark_build_checkpoint returns once the capture confirms the checkpoint;
  stop_build_capture returns the draft between seed and end.
- export_build_environment exports with the requester's task inside the
  computer and fetches the bundle through data/get in bounded batches.
- check_build_task grades a computer's data/get with REA's evaluator and,
  through seed, after_edit, near_miss and after_reset stages, records a
  witness with the action log and the screen recording; act_on_build_computer
  gains reset.
- Capture progress lines carry window, checkpoint and refusal events; apps
  start in the home; a pack script builds the runtime archive; long tools
  send progress notifications.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): make build capture and witness steps retryable, and close the near-miss reset hole

- A capture whose start answer is lost is stopped at once and kept
  unreleased until the stop is confirmed; a lost checkpoint answer is
  recovered from the capture's own log, and draft and export use the
  checkpoints its Evidence records; stop can be repeated to resume
  reading its result.
- A witness can check its seed again after a failed read, and keeps its
  stages in a saving state until the record is written, so a failed save
  can be retried without reading the app again.
- after_edit and near_miss count only click, drag, scroll, type and key,
  so a reset no longer stands in for the near-miss edit.
- A .deb build keeps its package file for capture and passes its URL to
  export; bundle downloads write each checked file before fetching the
  next; an apt-get update failure reports its exit code and log tail;
  reset sends progress while it waits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): keep the Trash out of seeds, bundle folders unwritten and first frames drawn

From the blind Inkscape rebuild:
- Paths in ~/.local/share/Trash get a desktop_trash reason and the
  shared_desktop role, so a file trashed before seed is neither owned nor
  seeded; the export names it in a limitation.
- The keep helper runs with PYTHONDONTWRITEBYTECODE=1, so starting a
  computer from a bundle folder no longer writes __pycache__ into it.
- start_build_computer waits, bounded, until the first frame is not one
  colour before returning it.
From review: a .deb that apt also offers exports from apt, since export
passes --package-url only when the capture says the package must come
from its file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): bound the first-frame wait's retries to the time left, with progress throughout

Each retry of the drawn-first-frame wait runs under an abort that fires at
the 10 s deadline, and the latest frame is returned when time runs out.
connect takes the first frame itself while its progress heartbeat still
runs, reporting "waiting for the desktop to draw". The wait moves to
DrawnScreenshot.ts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): serialize each build computer's calls, keep one recording id, and separate unreadable checks from failures

- One queue per computer: actions, capture steps and witness checks run one
  at a time, so two overlapping after_edit checks can no longer both be filed
  and let the witness hold without a near miss. Close cancels the running and
  queued calls; different computers still run concurrently.
- The witness keeps one recording request id per computer, so a seed retried
  after a lost reply recovers the recording it started.
- A check that could not read its file is undecided, not a failure, so a
  temporary read error can't satisfy seed, near_miss or after_reset; such a
  stage is refused and can be checked again.
- The export's file listing is written to a file and read back in bounded
  batches, so a large bundle no longer overruns exec's 1 MiB output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): treat failures on cut views as undecided, require a decided seed failure at export, and keep a close's cancellation

- A presence check that misses its line, row or pointed value in a view cut
  short is undecided rather than failed, since the data may sit in the part
  left out; a witness stage can no longer be proved by such a failure.
- Export counts only decided failures on the seed, and refuses a task whose
  every seed failure is undecided.
- A step whose turn was aborted by a close or the caller returns cancelled
  even when its work finished, so start_build_capture no longer reports
  success for a computer being removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): keep a step's saved result when a cancellation arrives during its commit

Reporting cancelled for any step whose turn was aborted after its work
replaced a completed export or witness save with a bare cancellation, losing
what was saved and leaving the witness unreadable. Steps now return what they
completed; only start_build_capture, which commits nothing, checks its turn
after the post-launch screenshot and reports cancelled. Asking again for a
saved witness's final check returns its record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): refuse a saved witness's repeated check for another folder

Asking again for the check that ended a witness returned its saved record
whatever directory came with it, so a caller naming another folder got
success though nothing was saved there. A different folder is now refused,
as an unfinished seed's retry already is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(desktop): assert the facts of cut-view and undecided outcomes, not their sentences

The new cut-view, missing-value, undecided-refusal and witness tests froze
whole descriptions, so a harmless rewording broke them. They now check each
outcome's id, passed and undecided exactly and the facts its message must
carry (file, pointer, line, row, stage) separately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(desktop): read XML with a plain DOCTYPE, and check an element's whole text

- Both readers accept a DOCTYPE with only a name and an optional PUBLIC
  or SYSTEM identifier, record it as the view's doctype, and never fetch
  the DTD or expand entities beyond XML's five; an internal subset, an
  external entity and a parameter entity stay refused alike.
- text_equals and text_contains compare the concatenated text of the
  element (or text node or attribute) a pointer reaches, as XPath
  string() does, with optional normalize_whitespace; both evaluators
  agree, and a text check on a truncated element is undecided.
- Whitespace-only text between elements is dropped only when the element
  holds no other text, so text split into runs keeps its spaces.
- The draft suggests text_equals for an element whose text changed.

So an AbiWord .abw reads as XML, and its paragraph's text is checked
whole however AbiWord splits it into <c> runs, as Writer's spans are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): draft whole-text checks on the paragraph, read only text as text, and skip asctime stamps

- A changed value is read at both ends, so text that became a run still
  marks its paragraph, and the drafted text_equals moves up out of mixed
  content to the nearest ancestor named by an attribute, such as
  AbiWord's p by its xid.
- Text checks accept only a text node or an attribute value; element
  names and doctype fields fail, in both evaluators.
- isTimestamp also recognises asctime dates (Www Mmm dd hh:mm:ss yyyy).
- XML files are drafted with the keyed diff the archive entries use, so
  no suggestion reaches an element by position; the shared logic moves to
  appTaskDraftNamed.ts.
- Text-check tests assert facts, not whole sentences.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): keep checks on a changed attribute inside a paragraph whose text check replaces its runs'

A drafted whole-text check on a paragraph hid every change inside it, so a
link whose label and href both changed drafted only the paragraph's text and
the new destination went unchecked. The text check now covers text and runs
added or removed whole; a changed name or attribute of an element both
versions hold keeps its own check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): read an attribute called name or attributes as an attribute when the draft keeps its check

The owner of `.../attributes/name` was taken as `.../attributes`, so a
changed `name` attribute inside a paragraph with a text check was dropped
again. The owner is now read by segment, attributes first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): draft text checks from text in runs added whole, and name a text element by its own id before its container

Text typed into a run the diff lists whole was never read, so the draft kept
only a presence check on the run, which wrong text also passed. Such text now
marks the nearest element both ends hold, or, when none can take the check,
the added element's own text. The climb also names an element by an
identifier of its own (id, xml:id, xid) before a larger container, so a
headline under a long layer keeps a whole-text check that survives re-split
runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(desktop): expect the whole-text check where #8988's pairing test meets text checks

On this branch a changed element's text is drafted as `text_equals` on the
element, so the identity pairing test now expects that check, under the
same `[@role=body]` key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(skills): build a computer-app environment from a request

The product skill gains references/desktop-build.md, a procedure that
takes Claude from a request such as "an Inkscape environment where the
agent changes a poster's headline" to a witnessed bundle and a report,
through start_build_computer, act_on_build_computer, the build capture
tools, export_build_environment and check_build_task: questions asked
once upfront, pauses only when blocked, a seed made in the app, content
checks by selector, a witness with a near miss and a reset in a fresh
computer, and solver runs only within a budget. setup installs the new
reference; skillVersion 61.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): fold the blind Inkscape rebuild's friction into the build skill

key takes an array of one chord's keys; check for a text cursor before
typing in apps with single-key shortcuts; in Save As, click Documents and
type only the file name; anything saved before seed, the Trash included,
is part of the seed; dismiss first-run dialogs before seed; a bundle's app
starts with the seed's files but need not open the document; Inkscape's
window class is Inkscape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): fold the held-out AbiWord build's friction into the build skill

Name the act fields and link the guide; a region screenshot becomes the
latest frame; a drawn caret is not focus; confirm dialogs with Return;
check the dialog's folder when the app shows only a name; an XML file the
draft shows as lines was read as text, so use short line checks; and keep
text checks to the value the edit types, since some apps save a reloaded
document differently. Later witness stages take no directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): state the export's seed rule, the Trash exception and draft refusals exactly

The export refuses a task unless at least one check fails on the seed and
every check passes at the end; a file moved to the Trash is left out of
the seed, so trash a stray file rather than starting over; a null draft
names its cause in draft_refusal; and a line check on a format read as
text only approximates a value, so bound it and call it approximate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): teach the build skill to use DOCTYPE and whole-text checks

The skill now checks text an app splits into runs whole with text_equals or
text_contains, finds an AbiWord paragraph by its xid, and drops the Writer
revision-id workaround.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): point text checks at the paragraph, and say what the scrub replaces

From the AbiWord rebuild: point a text check at the paragraph or another
element the edit keeps, never at a run a reopened document splits; how to
name an unchanged element; the export replaces the host name, user@host
and home path while a bare user name stays; AbiWord's entry and class.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): keep every text check on a cut element undecided, in both evaluators

The XML reader drops text deeper than its depth limit but keeps later
siblings, and an element's whole text joins both sides, so a text_contains
on a cut element could match text the file never holds ("at 3" + deep
"WRONG" + ":30 PM" read as "at 3:30 PM"). Any text check on an element of a
cut view is now undecided again, as #9050 had it, in REA's evaluator and the
exported verifier; a text node or attribute value stays decided. The text
check tests assert facts rather than sentences.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): keep a reset applied when the screenshot asked after it fails

Once data/reset and the ready wait succeed, act_on_build_computer's reset
is applied in the action log and returns success; a screenshot asked of it
that is then refused or throws is named in the result's screenshot_error,
with no frame, so the next after_reset stage is not refused. Cancellation
still propagates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): keep checks for elements without text beside a text edit, and normalize DOCTYPE system id line endings

A review found two gaps.

- A paragraph's whole-text check covered every element added or removed
  inside it, including ones holding no text: an image frame, a line break.
  A solver who typed the text but never inserted the image passed. Such an
  element now keeps its own presence or absence check.
- The Python reader kept a DOCTYPE system literal's CRLF where TypeScript
  normalizes it to LF. Both now report LF.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): wait out long build commands, fail a reset that restores nothing, and tie the witness to its bundle

A review found five gaps.

- Commands in the computer answer only when they end, but fetch stops
  waiting for headers after 300 s. So a slow export, draft or start always
  failed. A command with its own deadline now waits on node:http, bounded
  by that deadline.
- after_reset held whenever some check failed, so a reset that changed
  nothing passed on the near-miss state. Each check must now come out as it
  did at the seed.
- A failed or partial apt-get update left lists that looked complete, so a
  second start skipped the update. The update is now recorded only when it
  succeeds, and rerun otherwise.
- A witness on a bundle computer graded whatever task it was given. Its
  seed stage now refuses a task whose steps or checks differ from the
  bundle's.
- A task too large for one command argument (96 KiB) is refused before the
  export runs, instead of failing with E2BIG in the computer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): advertise a checkpoint-name pattern that compiles in every JS regex mode

main's advertised-pattern test compiles every input schema pattern in v
mode, which refuses an unescaped hyphen in a character class. The
checkpoint name's hyphen is now escaped, and it means the same in u mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(desktop): import textSyntaxFormat once in the state format tests

main's mixed-case JSON suffix fix and this branch both added the import,
and the merge kept both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): keep a completed reset, an unread seed, and a failed apt update

A cancel after data/reset no longer drops the restore from the action log,
so a near miss cannot treat the restored seed as an edit. An unreadable
seed check that later passes leaves after_reset undecided instead of
failing the witness. apt-get update is kept only when Error-Mode reports
success and a package list was fetched. The controller reply schema lives
with the controller, and capture progress lines are read by their domain
parser.

* fix(desktop): keep a whitespace-only run under the paragraph text check

A run whose only text is whitespace is still text the paragraph check
reads. It no longer gets its own presence check, which a later save of
the same words without that run would fail. Images and breaks, whose
text is empty, still keep their own checks.

* fix(desktop): keep an unread reset and a cut text check undecided

after_reset stays undecided when any check was not read, even if another
check still fails and the unread result's boolean matches the seed. A
text check for a child past a truncated view is undecided in both
evaluators. A reset whose reply is lost blocks a near miss until a later
input or a reset whose reply arrives.

* fix(desktop): retry a witness recording and a published record's cleanup

A recording that fails to end or download is not written into the witness
folder, and a manifest already received is fetched again without ending
the recording twice. A folder that is published while its handles fail to
close keeps that record and retries the same cleanup instead of writing
the folder again.

* fix(desktop): keep a published witness error and tag an app data refusal

A second failure to close a published witness returns the original error,
so its record and cleanup lease stay bound. A reset the app itself
refuses is an AppDataRefusal; only a reply that is not that refusal stays
unsettled. A missing child of a cut view is undecided with the same
description in both evaluators, and the shared cases cover it.

* fix(desktop): align a cut view's missing text with the Python check

A missing child of a truncated view is undecided for the same sentence in both evaluators, including the pointer the view does not contain.

* fix(desktop): keep one published witness cleanup across retries

A published record and its bound cleanup error stay together. A later close failure returns that error, and the unused path that built a fresh adapter error is gone.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
This commit is contained in:
authored and GitHub committed 2026-11-08 22:19:28 +08:00
1 parent 72d5e72e6d
commit 03a3fcbcea
18 files changed
+591 -59

No files matched your search

@@ -1,8 +1,8 @@
---
name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "60"
version: "61"
---
# REA
@@ -140,6 +140,10 @@ explicit path or endpoint and do not need it.
- Task for a desktop application's exported environment, from a
`capture-desktop-application` folder: read
[desktop application tasks](references/environments.md#tasks-for-a-desktop-application-environment).
- Environment of a Linux desktop app with a task, from a request such as
"build me an Inkscape environment where the agent changes a poster's
headline": read [desktop builds](references/desktop-build.md). You drive the
app on a build computer from `start_build_computer`, then export and witness.
- Fiddler Everywhere traffic or a native `.saz` capture: read the
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
@@ -539,6 +543,8 @@ as complete while required questions remain open.
[references/environments.md](references/environments.md)
- Tasks for a desktop application's exported environment:
[references/environments.md](references/environments.md#tasks-for-a-desktop-application-environment)
- A desktop app environment from a request, built and witnessed on build
computers: [references/desktop-build.md](references/desktop-build.md)
## Finish the task
@@ -0,0 +1,290 @@
# Desktop app environments from a request
Use this when someone asks for an environment of a Linux desktop app with a
task, such as "Build me an Inkscape environment where the agent changes a
poster's headline and saves it". You are the building agent: you drive the
app on a build computer through REA's MCP tools, write the task, export the
bundle and witness it in a fresh computer. REA runs no model. The tools are
MCP-only and need Docker, `REA_AGENT_ENV_COMMAND` and
`REA_BUILD_COMPUTER_REA_ARCHIVE` where the REA server starts; when a start
reports one missing, tell the person what to set and stop.
For the whole build:
- Drive the app only with `act_on_build_computer`. Never script it, run
commands in the computer or write its files another way.
- Ask the upfront questions once, then work on your own. Pause only when
blocked.
- The seed's content is synthetic. Never put real personal data in it.
- Close every computer you start, on success and on failure.
- Note the time you start; the report gives the build time.
## Ask once, upfront
In one message, ask only what the request leaves open, and keep the list for
the report:
1. The app and the task, if either is unclear: what changes, in which
document.
2. Whether they have a starting file of their own; otherwise you make one.
3. Whether they want solver runs after the witness: how many (the budget)
and with which model.
4. If so, whether they want to review the task before those runs.
## When you are blocked
Stop, say what you need and wait only for:
- a licence the app or its installer asks someone to accept: never accept it
yourself; name it and where to read it, and go on only once the person
says they accept it;
- credentials: they may only be typed as handles you never see, and the
build tools have none yet, so say the app needs a sign-in and stop. Never
ask for a password in chat;
- a signed-in state in the seed: ask before seeding one;
- an install this flow cannot do, below.
Decide everything else yourself and note it in the report.
## Choose the install
The computer is Ubuntu 24.04 (noble) on this host's architecture only. Use
the first source that has the app:
| Source | `install` |
| ---------------------------- | --------------------------------------------------------------------------------- |
| Ubuntu archive | `{"kind": "archive", "package": "inkscape"}` |
| PPA or vendor apt repository | `{"kind": "apt_repository", "package", "uri", "suite", "components", "key_file"}` |
| `.deb` by URL | `{"kind": "package_file", "url", "sha256", "architecture", "dependencies"}` |
- A PPA's `uri` is `https://ppa.launchpadcontent.net/OWNER/NAME/ubuntu`,
`suite` `noble`, `components` `["main"]`. `key_file` is the repository's
signing key saved on this host; for a PPA, fetch it from
keyserver.ubuntu.com by the fingerprint its Launchpad page shows.
- For a `.deb`, ask the requester for its URL and SHA-256; do not fetch a
vendor's download page yourself. List its `Depends` as `dependencies`:
it installs with the network off.
- Flatpak, AppImage and snaps are not supported yet: say so and stop. An
archive package that only installs a snap, such as noble's `firefox`,
counts as a snap. A package with no build for this host's architecture
(some PPAs are amd64 only) cannot be installed either.
- `desktop_entry` is the package's `.desktop` file name under
`/usr/share/applications` without `.desktop`, such as
`org.inkscape.Inkscape`. Add `window_class` when you know it; Inkscape's
is `Inkscape`. AbiWord's entry is `abiword` and its class `Abiword`.
## Drive the app
- Pass `computer_id` and the action's fields at the top level: click
`{x, y, count}`, scroll `{x, y, direction, amount}`, a screenshot region
`[x0, y0, x1, y1]`. The rest are in
[the build-computer guide](https://github.com/morluto/rea/blob/main/docs/desktop-build-computer.md).
- Take a `screenshot` before pointing: points are pixels of the latest
frame. If a response is too large, ask for a `region`, or `jpeg` with a
lower `quality`. A region crops the latest frame and becomes it: take a
full screenshot before pointing again.
- A drawn caret does not prove keyboard focus. After using a toolbar
control, click into the document, then type a few characters and look
before a long text.
- Confirm a dialog with `key` Return, not a trailing `\n` in `type`.
- An action's own screenshot is taken at once. After opening a menu, a
dialog or a window, look with
`{"action": "wait", "ms": 500, "screenshot": true}`.
- On a computer started from a request the app is not open at first, and
`start_build_capture` opens it: never open it yourself before that. Do not
close it while capturing; the capture ends when the app exits. When the
app is closed at any other time, open it from Applications → Other → the
environment's name.
- Prefer shortcuts and typing to long pointer paths: `key` takes `keys`, an
array holding one chord's keys, such as
`{"action": "key", "keys": ["ctrl", "s"]}`, and `type` turns `\n` into
Enter.
- Before `type`, check in a screenshot that a text cursor or a focused
field is there. In apps with single-key shortcuts, such as Inkscape, text
typed without one runs as commands.
- Save As opens in the home: click Documents in its sidebar and type only
the file name. Answer the app's save questions, such as keeping the
format, then check the saved path in the title or status bar, or the
dialog's folder when the app shows only the name, and that it shows no
unsaved mark, before you mark a checkpoint or check.
- What the app saved under the home before `seed`, such as a document in
`~/Documents` or its settings, is part of the seed. A file you move to the
Trash is left out, and the export lists it in `limitations`. Trash a stray
file rather than starting over.
- Dismiss first-run dialogs, such as a welcome screen, before `seed`. A
choice the app saves only later, such as Inkscape's "show this every
time", may not reach the seed, so expect the dialog in the witness too.
- Give what the task changes a stable name where the app has one, such as an
id in Inkscape's Object Properties, so a check can select it.
- Expect apps to split text into runs: Writer's revision ids split an
edited paragraph into spans and may give it an automatic style such as
`P1`, and AbiWord wraps each insertion in `<c>`. Check such text whole
with `text_equals` (see Write the task), and find the paragraph by
something the edit keeps, such as AbiWord's `xid`.
## Build the seed and the edit
1. `start_build_computer` with
`{"source": {"kind": "request", "desktop_entry": ..., "install": ...}}`.
The first start of an app builds its image and can take minutes. A
failed start returns the build's error: fix the request, or stop when the
install cannot work, and close any `computer_id` the error names.
2. `start_build_capture`. It returns once the app's window is open. A
`package_lists_error` may stop the capture finding the package's
sources; the export says when that matters.
3. Make the seed through the app and save it from the app under
`~/Documents`: build the document in the app, or write its source where
the app edits its format as text, then save it again from the app. Make
it look real but synthetic: the thing the task changes, plus a few things
the change must leave alone. For a starting file of the requester's,
read it here and rebuild it in the app with synthetic data in place of
personal data; REA cannot copy a file into the computer.
4. Only once the seed is saved, `mark_build_checkpoint` with name `seed`.
5. Make the change the task asks for, as a user would, and save it.
6. Only once the change is saved, `mark_build_checkpoint` with name `end`.
7. `stop_build_capture`. Read `draft.files` (each file that changed, with
its net change), `draft.task.checks` (suggested checks) and
`limitations`. When `draft` is null, read `draft_refusal` first: a
missing `seed` or `end` checkpoint means capturing again on a new build
computer, while a failed draft command names its own cause to fix.
## Write the task
Write `{"name", "prompt", "checks"}` from the draft and what you did.
- `name`: lowercase letters, digits and hyphens, not `smoke`.
- `prompt`: what the requester wants, in their words and the app's terms,
naming the document as a user would and ending with saving it: "Change the
poster's headline in Documents/poster.svg to Summer Bake Sale and save
it." Never say how the file stores it, and never give the checks.
- Checks are file-based: content checks that prove the edit, never only that
a file exists or changed. Start from the suggestions; keep those that say
what the user did and drop what follows from how the app saves.
- Compare exact values, so a typo or a wrong number fails.
- Select elements by what they hold, never by position: SVG text by id,
`/root/children/[name=g][@inkscape:label=Layer 1]/children/[name=text][@id=headline]/children/[name=tspan]/children/0`,
or an ODF paragraph by style,
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:text]/children/[name=text:p][@text:style-name=Title]/children/0`.
A selector must match exactly one element, so give the target a unique id
or style in the seed. The draft may suggest positional pointers; rewrite
them. Only an element's text is read by position, as `children/0`.
- When an app may split an element's text into runs, check it whole:
`text_equals` (or `text_contains` for a value inside a sentence) on the
element compares all its text, however it is split. Add
`normalize_whitespace: true` when the app may reflow spaces. Example, an
AbiWord paragraph by its stable id:
`{"kind": "text_contains", "file": "Documents/memo.abw", "pointer": "/root/children/[name=section]/children/[name=p][@xid=6]", "value": "at 3:30 PM"}`.
Point a text check at the paragraph or another element the edit keeps,
never at a run such as AbiWord's `[name=c]`: a reopened document splits
the text into several runs. The draft lists only what changed; to name an
unchanged element, such as another paragraph's `xid`, use the app's usual
numbering and let the export confirm it, or read it in the exported
bundle's `envs/<env>/seed/`.
- Where easy, add one check that the rest is untouched, such as another
text's value the edit must leave alone. It is the one unchanged value
worth checking; it passes on the seed too, which is fine, since the export
needs only one check to fail there.
- Give `value_equals` a `tolerance` for a number the app writes as text.
- When the draft shows an XML document's changes as `lines`, REA read it
as text, for example because its DOCTYPE declares entities. Selectors are
impossible there, and `line_count` with `contains` only approximates a
value: `3:30` also matches `13:30`. Bound it with neighbouring text and
name the check as approximate in the report's limitations.
- Some apps save a document reloaded from disk differently from one typed
in the same session: AbiWord wraps each new insertion in its own markup.
Your build's end state hides this, which is why whole-text checks matter;
expect the witness to show the real format.
The kinds and pointer rules are in
[the desktop tasks section](environments.md#pick-the-checks).
## Export
`export_build_environment` with `task` and `directory`, the absolute path of
a new or empty folder here, such as `<home>/rea-builds/poster/bundle`. The
export seeds what was saved at `seed`, replaces the computer's host name,
`user@host` and home path (a bare user name, such as AbiWord's author
`user`, stays), and checks the task against `seed` and `end`. It refuses a task
unless at least one check fails on the seed and every check passes at the
end, and says why, such as a pointer that names nothing at the end. A
check that the rest is untouched passes on the seed too, which is fine. On
a refusal, fix the task and export again, into another folder if that one
is no longer empty. Keep
`seed_files`, `scrubbed_files`, `unscrubbed_files` and `limitations` for the
report.
Keep the build computer open until the witness holds: a task fix exports
again from it.
## Witness
Witness in a fresh computer of the bundle, from the prompt, as a solving
agent would. Do not replay the build's actions or coordinates, and do not
check between stages to steer the edit: a solving agent never sees the
checks. Give every `check_build_task` the task exactly as exported.
1. `start_build_computer` with
`{"source": {"kind": "bundle", "path": "<bundle>"}}`. The app starts
with the seed's files in place but need not open the document: open it
from the prompt's path, as a solving agent would. Take a `wait`
screenshot first, since the desktop may not have drawn yet.
2. `check_build_task` with `stage: "seed"` and `directory`, the absolute
path of a new or empty folder for the record. A check must fail.
Recording starts. Later stages take no `directory`.
3. Do the task from the prompt with `act_on_build_computer` and save.
`check_build_task` with `stage: "after_edit"`: every check must pass.
4. Make a near miss on top of it: a plausible wrong edit, such as a typo or
a wrong number, and save. `check_build_task` with `stage: "near_miss"`:
a check must fail.
5. `act_on_build_computer` with `action: "reset"`, then `check_build_task`
with `stage: "after_reset"`: a check must fail again.
6. The last stage ends the witness: `witness.record` has the verdict, the
path of `witness.json` and the recording's segments in `directory`.
`close_build_computer` on the witness computer.
A stage that does not hold ends the witness as `failed`. Read each check's
result and decide:
- `after_edit` failed though the saved file is right: the check is brittle.
Fix it, export again and witness in a new computer.
- `after_edit` failed because your edit was wrong: witness again in a new
computer; a computer is witnessed once.
- `near_miss` passed: the checks are too loose. Tighten them, export again
and witness again.
- `after_reset` passed: the reset did not restore the seed. Report it as a
limitation.
Stop after three witnesses and report what still fails.
## Solver runs
Only if the requester asked for them. If they wanted to review the task
first, show them its prompt and checks and wait for their approval. Then run
the bundle within the budget, one run at a time, with the runner
`REA_AGENT_ENV_COMMAND` names:
`agent-env run <bundle> --task <name> --agent claude-code --model <model>`.
Count runs reported `passed`. A run `scored below 1` is a miss; one reported
`failed` did not finish, so run it again if the budget allows instead of
counting it. Never run past the budget.
## Close and report
`close_build_computer` on every computer still open, also after a failure.
If a close fails, call it again; if it still fails, tell the requester what
is left.
Report:
- the bundle path and the environment name;
- the task's name and prompt;
- each check, and why it proves the edit and fails the near miss;
- each witness stage with its result, the verdict, and the paths of
`witness.json` and the recording;
- the seed's contents: `seed_files` and what each holds;
- `scrubbed_files` and `unscrubbed_files`; an unscrubbed file still names
the host or user, so it needs review before sharing;
- limitations: the capture's, the export's and anything you could not do;
- solver runs, if any: the model, the runs and the passes;
- the build time and the questions you asked.
Never publish or share the bundle yourself.
@@ -294,7 +294,9 @@ Use this when the user has a folder from `rea capture-desktop-application` and
wants a task for the environment `rea export-desktop-application` makes from
it: a prompt for a computer-use agent and checks that grade what the agent
left in the app's files. You write the prompt and the checks; REA validates
the task against the capture and exports it. These commands are CLI only.
the task against the capture and exports it. These commands are CLI only. To
build the environment and its task from a request instead, follow
[desktop builds](desktop-build.md).
### Describe the change
+4 -1
View File
@@ -141,7 +141,10 @@ The rules both evaluators follow:
- A check decided by what a view leaves out (`lines_absent`, `value_absent`,
`row_absent`, the counts, `value_equals` of a whole object or array, and
the text checks of an element) fails when that view is truncated; a check
satisfied by what the view holds is decided on it.
satisfied by what the view holds is decided on it. An element's text is not:
the reader may have dropped text from between the runs it kept, so
`text_contains` fails on a truncated view even where the kept runs hold its
value. A text node or attribute value is whole, and decided.
The export refuses, naming the check, a file outside the owned paths, a file
spelled with a leading slash, `.` or `..`, a pointer into a file that is text
+3 -1
View File
@@ -14,7 +14,9 @@ agent's computer-use actions to it through three MCP tools:
The tools are MCP-only, like the debugger sessions: the computer lives as long
as the REA MCP server that started it, or until it is closed. Capturing the
app, exporting its bundle and witnessing the task on such a computer are in
[Building and witnessing an environment](desktop-build-witness.md).
[Building and witnessing an environment](desktop-build-witness.md), and REA's
skill for doing all of it from a request is
[desktop builds](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/desktop-build.md).
## Requirements
+17 -8
View File
@@ -18,6 +18,12 @@ grades the witness with the same checks the exported verifier runs.
Like the computer tools, these are MCP-only and act on a `computer_id` from
`start_build_computer`.
REA's skill walks an agent from a request such as "build me an Inkscape
environment where the agent changes a poster's headline" to a witnessed
bundle with these tools: the questions it asks first, when it stops for the
person, the install, the seed, the task, the witness and its final report.
See [desktop builds](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/desktop-build.md).
## Build
1. `start_build_computer` from a request. REA is installed in the computer.
@@ -137,14 +143,17 @@ SQLite file locked past the reader's wait, or because what it looks for may
lie in what its view cut, is `undecided` in its result: it shows nothing
about the task. That covers a line `lines_present` misses in a text view cut
at 5,000 lines, a row `row_present` misses in a table whose view keeps only
its first rows, a `value_equals` pointer that names nothing in a truncated
view or past a cut zip listing, and the absence checks and counts on a cut
view. A check that passes on what the view kept stays decided, and so does a
value found there that differs from the one expected. The result's `passed`
and description are the exported verifier's, which has no undecided results.
A stage that only undecided checks would settle, such as a near miss whose
one failing check could not read its file, files nothing and is refused;
check it again once the files read whole.
its first rows, a `value_equals` or text check pointer that names nothing
in a truncated view or past a cut zip listing, the absence checks and counts
on a cut view, and `text_equals` or `text_contains` of an element on one,
even where the element's kept text holds the value, since the cut may have
dropped text from between its runs. Any other check that passes on what the
view kept stays decided, and so does a value found there that differs from
the one expected.
The result's `passed` and description are the exported verifier's, which has
no undecided results. A stage that only undecided checks would settle, such
as a near miss whose one failing check could not read its file, files nothing
and is refused; check it again once the files read whole.
A stage whose checks do not show what it must ends the witness, and so does
`after_reset`. REA then ends the recording, fetches its segments through
@@ -94,11 +94,12 @@ and the recording and writes the record as Evidence.
want of a readable or whole view is `undecided` and proves nothing; a
stage only such checks would settle files nothing and can be checked
again. Undecided includes a failure to find a line, row or value that may
lie in what a cut view left out; a check that passes on what the view kept
stays decided. The export likewise keeps a task only when a check fails on
what the seed's view shows. The controller writes a folder's file list to a
file that REA reads through `data/get`, so a large bundle's list is not cut
at the 1 MiB of a command's output.
lie in what a cut view left out, and any text check of an element in a cut
view, which may have lost text from between the element's runs; any other
check that passes on what the view kept stays decided. The export likewise keeps a task only when a check
fails on what the seed's view shows. The controller writes a folder's file
list to a file that REA reads through `data/get`, so a large bundle's list
is not cut at the 1 MiB of a command's output.
- The witness reads the build computer's bounded action log: its stage rules
use the log's count and the last applied action of each kind, and the
record holds the entries the log keeps, `actions_dropped`, and a
@@ -55,9 +55,11 @@ verifier's, agree.
entry of another format; a value that is neither an element, a text node
(a string among an element's `children`) nor an attribute's value (one among
its `attributes`), so an element's `name` and the `doctype`'s fields fail;
and, as undecided, an element in a view that leaves something out, or a
selector searching an array such a view may cut. A text node is whole, so
it is decided on a truncated view.
and, as undecided, in a view that leaves something out, either check of an
element, even one whose kept text holds the value, since the cut may have
dropped text from between the runs it kept; a pointer naming nothing; or a
selector searching an array such a view may cut. A text node or an
attribute's value is whole, so it is decided on a truncated view.
- Refusals: the task parser refuses the empty pointer, a pointer not starting
with `/`, an empty `text_contains` value or one that is only whitespace
with `normalize_whitespace`, and a value that looks like a date and time,
+1
View File
@@ -34,6 +34,7 @@ export const MANAGED_SKILL_FILES = [
"references/runtime-observation.md",
"references/evidence-workflows.md",
"references/environments.md",
"references/desktop-build.md",
] as const;
export const isManagedSkillManifest = (content: string): boolean =>
+37 -2
View File
@@ -814,8 +814,6 @@ const TEXT_CASES: readonly (readonly [DesktopTaskCheckInput, boolean])[] = [
],
];
/** Every kind of check against those views, each with the outcome both evaluators must give. */
/** Failures a cut view may have caused, which both evaluators fail alike and REA marks undecided. */
const CUT_VIEW_CASES: readonly (readonly [DesktopTaskCheckInput, boolean])[] = [
[{ kind: "lines_present", file: "Documents/long.txt", lines: ["z"] }, false],
@@ -838,8 +836,45 @@ const CUT_VIEW_CASES: readonly (readonly [DesktopTaskCheckInput, boolean])[] = [
},
false,
],
[
{
kind: "text_contains",
file: "Documents/cut.xml",
pointer: "/root",
value: "gone",
},
false,
],
[
{
kind: "text_contains",
file: "Documents/cut.xml",
pointer: "/root",
value: "kept",
},
false,
],
[
{
kind: "text_equals",
file: "Documents/cut.xml",
pointer: "/root/children/5",
value: "x",
},
false,
],
[
{
kind: "text_contains",
file: "Documents/many.zip",
pointer: "/entries/1100/view/root",
value: "x",
},
false,
],
];
/** Every kind of check against those views, each with the outcome both evaluators must give. */
export const CHECK_CASES: readonly (readonly [
DesktopTaskCheckInput,
boolean,
+89 -18
View File
@@ -1,13 +1,15 @@
import { describe, expect, it } from "vitest";
import type { JsonValue } from "../jsonValue.js";
import { parseDesktopTask } from "./appTask.js";
import {
elementText,
evaluateTaskChecks,
type TaskCheckOutcome,
type TaskFileView,
} from "./appTaskChecks.js";
import { CHECK_CASES, CHECK_VIEWS } from "./appTaskChecks.fixture.js";
import { textView } from "./stateFormats.js";
import { textView, xmlView } from "./stateFormats.js";
const portable = (name: string) => /^[a-z0-9]+(?:-[a-z0-9]+)*$/u.test(name);
@@ -444,6 +446,84 @@ describe("evaluateTaskChecks on cut views", () => {
});
});
describe("evaluateTaskChecks text on a cut view", () => {
/** A paragraph of `before`, then `dropped` nested past the 64 levels an XML view keeps, then `after`. */
const nested = (before: string, dropped: string, after: string) =>
`<p id="t">${before}${"<b>".repeat(65)}${dropped}${"</b>".repeat(65)}${after}</p>`;
const views = new Map<string, TaskFileView>([
[
"page.xml",
{
format: "xml",
view: {
root: { name: "doc", attributes: {}, children: ["kept"] },
truncated: true,
},
},
],
[
"time.xml",
{
format: "xml",
view: xmlView(nested("at 3", "WRONG", ":30 PM")) ?? null,
},
],
[
"word.xml",
{ format: "xml", view: xmlView(nested("alpha", "zzz", "beta")) ?? null },
],
]);
const text = (kind: string, file: string, pointer: string, value: string) =>
evaluateTaskChecks(
[{ id: "c", kind, file, pointer, value } as never],
views,
)[0];
const cut = { id: "c", passed: false, undecided: true } as const;
it("leaves both text checks of an element in a cut view undecided, even where the runs it kept hold the value", () => {
// The reader drops the nested text but keeps the run after it, so the kept runs read as text the file never had.
const keptText = (file: string) =>
elementText(
(views.get(file)?.view as { root: JsonValue } | undefined)?.root ??
null,
);
expect(keptText("time.xml")).toBe("at 3:30 PM");
expect(keptText("word.xml")).toBe("alphabeta");
for (const [file, value] of [
["time.xml", "at 3:30 PM"],
["word.xml", "alphabeta"],
["page.xml", "kept"],
["page.xml", "gone"],
] as const)
for (const kind of ["text_contains", "text_equals"])
expectOutcome(text(kind, file, "/root", value), cut, [file]);
});
it("decides a text node or an attribute value in a cut view, and leaves a pointer naming nothing there undecided", () => {
expectOutcome(
text("text_equals", "time.xml", "/root/children/0", "at 3"),
{ id: "c", passed: true },
["time.xml", "/root/children/0", quoted("at 3")],
);
expectOutcome(
text("text_contains", "time.xml", "/root/attributes/id", "t"),
{ id: "c", passed: true },
["time.xml", "/root/attributes/id", quoted("t")],
);
expectOutcome(
text("text_equals", "page.xml", "/root/children/0", "gone"),
{ id: "c", passed: false },
["page.xml", "/root/children/0", quoted("kept"), quoted("gone")],
);
expectOutcome(
text("text_equals", "page.xml", "/root/children/1", "kept"),
cut,
["page.xml", "/root/children/1"],
);
});
});
describe("evaluateTaskChecks selectors and tolerance", () => {
const plate = "Documents/plate.FCStd";
const objects = "/entries/[name=Document.xml]/view/root/children";
@@ -578,26 +658,17 @@ describe("evaluateTaskChecks whole text", () => {
const meeting = "Documents/meeting.xml";
const plate = "Documents/plate.FCStd";
const time = "/root/children/[name=p][@id=time]";
/** A value as a description shows it: caller data and what the view holds, as JSON. */
const quoted = (value: unknown) => JSON.stringify(value);
const expectFacts = (
found: { passed: boolean; description: string } | undefined,
passed: boolean,
facts: readonly string[],
) => {
expect(found?.passed).toBe(passed);
for (const fact of facts) expect(found?.description).toContain(fact);
};
const failed = { id: "c", passed: false } as const;
it("reports the text an element has, its runs joined, beside the value it was compared with", () => {
expectFacts(
expectOutcome(
outcome({
kind: "text_equals",
file: meeting,
pointer: time,
value: "Starts at 3:45 today.",
}),
false,
failed,
[
meeting,
time,
@@ -605,14 +676,14 @@ describe("evaluateTaskChecks whole text", () => {
quoted("Starts at 3:45 today."),
],
);
expectFacts(
expectOutcome(
outcome({
kind: "text_contains",
file: meeting,
pointer: time,
value: "3:45",
}),
false,
failed,
[quoted("Starts at 3:30 today."), quoted("3:45")],
);
const collapsed = outcome({
@@ -622,7 +693,7 @@ describe("evaluateTaskChecks whole text", () => {
value: "Doors open",
normalize_whitespace: true,
});
expectFacts(collapsed, true, [quoted("Doors open")]);
expectOutcome(collapsed, { id: "c", passed: true }, [quoted("Doors open")]);
expect(collapsed?.description).not.toContain(quoted("Doors open"));
});
@@ -659,7 +730,7 @@ describe("evaluateTaskChecks whole text", () => {
];
for (const [check, facts] of refused)
expectFacts(outcome({ kind: "text_equals", ...check }), false, facts);
expectOutcome(outcome({ kind: "text_equals", ...check }), failed, facts);
});
it("leaves an element of a truncated view undecided as a value check is, while a text node there is decided", () => {
@@ -703,7 +774,7 @@ describe("evaluateTaskChecks whole text", () => {
expect(missing?.passed).toBe(false);
expect(missing?.undecided).toBe(true);
expect(missing?.description).toBe(
"the view of Documents/cut.xml is truncated, so what it leaves out is unknown",
"the view of Documents/cut.xml is truncated, so what it leaves out is unknown (/root/children/2)",
);
const zipped = outcome({
kind: "text_equals",
+19 -13
View File
@@ -240,12 +240,17 @@ const unreadOrPartial = (
const NOTHING: Located = { kind: "none", searched: [], byPosition: [] };
/** A value found where its pointer leads, or why it is none. */
const noValue = (at: string, located: Located & { kind: "none" }) =>
fail(
/** Why a pointer names no value, undecided when its view is cut, since the value may lie past what the view kept. */
const noValue = (
at: string,
located: Located & { kind: "none" },
cut: boolean,
): Verdict =>
notFound(
located.reason === undefined
? `${at} has no value`
: `${at}: ${located.reason}`,
cut,
);
const valueVerdict = (
@@ -272,13 +277,7 @@ const valueVerdict = (
check.value !== null;
if (whole && truncated) return undecided(check.file);
if (check.kind === "value_equals") {
if (located.kind === "none")
return notFound(
located.reason === undefined
? `${at} has no value`
: `${at}: ${located.reason}`,
truncated,
);
if (located.kind === "none") return noValue(at, located, truncated);
if (check.tolerance !== undefined && typeof check.value === "number")
return nearVerdict(at, located.value, check.value, check.tolerance);
return sameJson(located.value, check.value)
@@ -383,12 +382,18 @@ const textVerdict = (
const problem = zip ? notXmlEntry(check, read.view) : undefined;
if (problem !== undefined) return fail(problem);
if (located.kind === "unresolved") return fail(`${at}: ${located.problem}`);
// A zip view's truncation that matters is the entry the pointer goes into.
// A child past the kept nodes of a cut view may still hold the text.
const truncated = zip
? zipLeavesOutAt(read.view, check.pointer)
: read.truncated;
// A child past the kept nodes of a cut view may still hold the text.
// A missing child of a cut view is the same undecided result in both evaluators.
if (located.kind === "none")
return truncated ? undecided(check.file) : noValue(at, located);
return truncated
? unread(
`the view of ${check.file} is truncated, so what it leaves out is unknown (${check.pointer})`,
)
: noValue(at, located, false);
const found = located.value;
if (
typeof found === "string"
@@ -398,7 +403,8 @@ const textVerdict = (
return fail(
`${at} is ${shown(found)}, not an element, a text node or an attribute value`,
);
// A text node is whole; an element's text is whole only if its view left nothing out.
// A text node is whole. An element's text in a cut view may lack what the cut
// dropped from its middle, so its kept runs can read as text it never had.
if (typeof found !== "string" && truncated) return undecided(check.file);
return comparedText(check, at, elementText(found));
};
+32
View File
@@ -1027,6 +1027,38 @@ describe("draftTaskFiles for elements without text beside a text edit", () => {
});
});
describe("draftTaskFiles for a whitespace-only run", () => {
it("covers a whitespace-only run with the paragraph text check and does not require that run", () => {
const file = "Documents/note.abw";
const note = (sixth: JsonValue[]): JsonValue => ({
root: xmlElement("abiword", {}, [
xmlElement("section", { xid: "1" }, [
xmlElement("p", { xid: "6" }, sixth),
]),
]),
truncated: false,
});
const edited = ["Hello", xmlElement("c", {}, [" "]), "there"];
const [draftFile] = draft({
[file]: path("document", "xml", note(["Hello"]), note(edited)),
});
const paragraph = "/root/children/[name=section]/children/[name=p]";
expect(draftFile?.suggested).toEqual([
{
kind: "text_equals",
file,
pointer: paragraph,
value: "Hello there",
},
]);
expect(
[edited, ["Hello", " ", "there"]].map((saved) =>
passing(file, draftFile?.suggested, note(saved)),
),
).toEqual([[true], [true]]);
});
});
describe("draftTaskFiles with nullable SQLite keys", () => {
it("suggests checks for distinct NULL-key rows and ignores a reorder", () => {
const file = ".local/share/app/items.db";
+6 -2
View File
@@ -83,11 +83,15 @@ const namedByIdentifier = (pointer: string) => {
return key !== undefined && isIdentifier(key);
};
/** Whether an end holds an element at a pointer whose text a check reads as nothing, as an image or a break. */
/**
* Whether an end holds an element whose text a check reads as nothing, as an
* image or a break. A run of only whitespace is text the paragraph check
* already reads, so it stays covered.
*/
const textlessAt = (pointer: string, ends: Ends) =>
[ends.baseline, ends.final].some((view) => {
const value = resolvePointer(view, pointer);
return isXmlElement(value) && isWhitespace(elementText(value));
return isXmlElement(value) && elementText(value) === "";
});
/** Whether both ends hold an element at a pointer. */
+2 -2
View File
@@ -339,11 +339,11 @@ def _whole_text(check, entry):
truncated = _zip_leaves_out(view, check["pointer"])
if kind == "none":
if truncated:
return _undecided(check)
return False, f"the view of {check['file']} is truncated, so what it leaves out is unknown ({check['pointer']})"
return False, f"{at} has no value" if found is None else f"{at}: {found}"
if not (_text_at(view, check["pointer"]) if isinstance(found, str) else _is_element(found)):
return False, f"{at} is {_shown(found)}, not an element, a text node or an attribute value"
# A text node is whole; an element's text is whole only if its view left nothing out.
# A text node is whole. An element's text in a cut view may lack what the cut dropped from its middle.
if not isinstance(found, str) and truncated:
return _undecided(check)
return _compared_text(check, at, _element_text(found))
+1 -1
View File
@@ -6,5 +6,5 @@ export const PACKAGE_METADATA = {
serverSdkVersion: "2.3.1",
clientSdkVersion: "2.3.1",
coreSdkVersion: "2.3.1",
skillVersion: "60",
skillVersion: "61",
} as const;
@@ -654,6 +654,7 @@ describe("canonical skill transaction", () => {
"SKILL.md",
"references/android-applications.md",
"references/connection-and-recovery.md",
"references/desktop-build.md",
"references/environments.md",
"references/evidence-workflows.md",
"references/javascript-applications.md",
@@ -5,7 +5,9 @@ import { pathToFileURL } from "node:url";
import { describe, expect, it } from "vitest";
import type { DesktopTaskCheck } from "../../../src/domain/desktop/appTask.js";
import { elementText } from "../../../src/domain/desktop/appTaskChecks.js";
import { xmlView } from "../../../src/domain/desktop/stateFormats.js";
import type { JsonValue } from "../../../src/domain/jsonValue.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import {
expectSameCheckResults,
@@ -209,4 +211,69 @@ describe("whole-text checks in both evaluators", () => {
// The value searched for is the collapsed one, " 9:30\n" read as "9:30".
expect(after?.[1]?.description).toContain(JSON.stringify("9:30"));
});
it("leaves an element's text undecided where the readers dropped text from between its runs, and its attribute decided", async () => {
const deep = "Documents/deep.xml";
/** A paragraph of `before`, then `dropped` nested past the 64 levels both readers keep, then `after`. */
const paragraph = (before: string, dropped: string, after: string) => ({
name: "deep.xml",
bytes: new TextEncoder().encode(
`<p id="t">${before}${"<b>".repeat(65)}${dropped}${"</b>".repeat(65)}${after}</p>`,
),
});
const files = [
paragraph("at 3", "WRONG", ":30 PM"),
paragraph("alpha", "zzz", "beta"),
];
await expectSameFileViews(files);
// The kept runs join into text neither file has.
const kept = await Promise.all(
files.map(async (file) => {
const { view } = await typescriptFileView(file);
return elementText((view as { root: JsonValue }).root);
}),
);
expect(kept).toEqual(["at 3:30 PM", "alphabeta"]);
const results = await expectSameCheckResults(
deep,
[
{
id: "time",
kind: "text_contains",
file: deep,
pointer: "/root",
value: "at 3:30 PM",
},
{
id: "word",
kind: "text_contains",
file: deep,
pointer: "/root",
value: "alphabeta",
},
{
id: "id",
kind: "text_equals",
file: deep,
pointer: "/root/attributes/id",
value: "t",
},
],
files,
);
expect(results).toHaveLength(files.length);
for (const outcomes of results) {
expect(
outcomes.map(({ id, passed, undecided }) => [id, passed, undecided]),
).toEqual([
["time", false, true],
["word", false, true],
["id", true, undefined],
]);
for (const { description } of outcomes)
expect(description).toContain(deep);
}
});
});