Files
reverse-engineering-anything/docs/desktop-application-export.md
T
d2f9b8b7de feat(desktop): build several tasks from one capture, and measure each task's pass rate within a budget (#9216)
* docs(desktop): clarify gzip scrub limits

* fix(setup): own desktop build process cleanup

* fix(doctor): compare desktop settings with resolved defaults

* fix(desktop): refuse a build runtime whose export takes no task ends

Several tasks pass --task-end to the build computer's export, so a runtime
packed before it now fails at the start of the capture with the repack fix,
as a runtime without synthetic seeds already did, rather than at export.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(desktop): give the root-array draft test its own describe

The records describe grew past the 150-line lint limit with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(doctor): inspect client-recorded desktop runners

* fix(doctor): reuse matching runner probes

* fix(setup): propagate cancellation to desktop actions

* fix(desktop): rewrite a download's launch keys written with spaces around =

The desktop entry parser accepts `Exec = notes %F` and an indented
`TryExec =notes`, so the install script now matches those keys the same way
instead of leaving the original program in place. Adds the feature's tasks.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): keep why a measured run failed in its solve record

A solve that returned an outcome dropped the runner's error, so a run that
failed to deploy, solve or verify was retried with no reason recorded. The
record, the measure result and difficulty.json now carry it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(setup): stop Hopper on cancel, never report a cancelled setup ready, and keep a failed action's client setting

- The Hopper install gets setup's signal: Linux already honoured it, and the
  macOS installer now stops its downloads and stops before mounting or copying.
- runSetup checks the signal after the final doctor and close, so a
  cancellation there raises instead of returning ready.
- A failed pack or agentenv install no longer writes its archive or runner to
  client configs, so the setting the client already had stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(setup): keep the completed setup result in a cancellation after cleanup

A cancellation during the final doctor or host close carried only the applied
actions, dropping each client's result and backup path and the doctor report.
It now carries the whole result as setup_result, as doctor's cancellation
carries its report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* fix(xml): avoid scanning selected ancestors per tag

* fix(desktop): exclude covered text runs from draft counts

* fix: preserve text when drafting desktop checks

* fix: restore nested desktop state aliases

* refactor: share UUID recognition across draft checks

* fix: retain multiline XML attribute context

* fix: retain relationship identifiers in task drafts

* docs: clarify desktop draft identifier handling

* test(desktop): type formatting run cases as pairs

* fix(desktop): carry document format into selector evaluation

* test(desktop): separate markup boundary regression group

* fix(desktop): propagate formats through draft selectors

* fix(desktop): switch selector format within archive entries

* test(desktop): remove duplicate seed filename case

* fix(setup): retain diagnostics when cancellation overlaps failure

* refactor(setup): keep command cancellation at its owner

* test(setup): verify cancellation at the application boundary

* test(desktop): pass JSON format in exported index parity

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* test(desktop): run installer scripts at boundary

* style(desktop): remove trailing blank line

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* fix: align desktop numeric seed comparisons

* test(desktop): keep one draft exclusion fixture import

* test(desktop): retain log eligibility across draft test groups

* fix(doctor): resolve empty registered desktop runner

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix: retain quoted identifier text in task drafts

* fix: recognize Unicode XML in desktop state readers

* fix(desktop): bound nested draft checks

* fix(desktop): explain missing arrays in truncated views

* fix(desktop): find XML type references in archives

* test(desktop): remove duplicated capture assertions

* fix(desktop): retain nested omitted text boundaries

* style(desktop): format draft value imports

* test(desktop): group model and draft value checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): retain placed-item sibling guards

* fix(desktop): preserve archive paths and bound doctor reads

* fix(doctor): refuse non-files before descriptor admission

* fix(setup): track remaining cleanup resources across retries

* test(desktop): specify XML format in heap pairing fixture

* fix(setup): skip Hopper launch after cancelled copy

* fix: preserve task measurement limits and cleanup

* fix(desktop): retain reported solve sandbox cleanup

* test(desktop): check retained log seed admission

* fix(doctor): query agentenv for model endpoint

* fix(doctor): preserve uncertainty for empty endpoint override

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* test(desktop): remove duplicate history regression

* fix(desktop): align XML names with exported reader

* fix(desktop): create selected state alias targets

* fix(desktop): inspect supported runtime archives

* fix(desktop): keep ordinary text checks exact

* fix(desktop): verify registered build host paths

* fix(desktop): retain record identity in draft checks

* test(desktop): preserve text reference numbers

* fix(desktop): bound download install input reads

* fix(desktop): match placed notes by whole text

* test(desktop): match multiple task export results

* fix(desktop): retain failed measurement file closes

* fix(setup): retain completed actions on cancellation

* test(setup): retain cancelled installer result

* fix(desktop): scrub XML independently of reader compatibility

* fix(desktop): preserve file aliases and reader diagnostics

* style(desktop): format review fixes

* fix(desktop): inspect archives with registered runner

* test(desktop): verify registered archive interpreter

* style(desktop): format capture review fixes

* style(desktop): format review changes

* feat(desktop): target agentenv-framework 0.9.1464

Desktop bundles, setup's --install-agentenv and the doctor's exact-version
check now target 0.9.1464, which keeps an exported app's data/get call,
holds the computer fixture above boot dialogs, leaves an unresolved setting's
build unverified rather than failed, and mounts lock packages instead of
copying them into the image. The web environment lane stays on 0.9.1431.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): retain per-solve sandbox cleanup roots

* test(desktop): expect the solve cleanup helper among the shipped agentenv helpers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(desktop): clean up a stopped solve's sandboxes through agentenv 0.9.1482

agentenv-framework 0.9.1482 lists the local sandboxes under a root
(local_sandbox_ids) and rebuilds one killed while it wrote its port map for
teardown. The solve cleanup helper now terminates each sandbox agentenv
lists, keeping its Docker and host-process checks, and reports anything else
left under the root. Desktop bundles, setup's --install-agentenv and the
doctor's version check target 0.9.1482.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): verify a killed solve's cleanup once its sandbox root is empty

A solve's sandboxes are all local under its private root, and agentenv now
keeps a sandbox's folder until its teardown is confirmed. So when the helper
ends without a report and REA's cleanup empties that root, the cleanup is
verified: a cancelled solve ends as a cancellation and a solve past its
deadline as a failed run, rather than as cleanup the measure cannot vouch for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(desktop): target agentenv-framework 0.9.1485

0.9.1485 keeps a local sandbox's folder until its teardown is confirmed, which
the solve cleanup's verified result relies on. (0.9.1484 failed to publish.)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): record a solve stopped before it reports as failed, not as a refused request

A helper killed from outside or stopped at its deadline left no report, so the
measure refused with invalid input and told the caller to fix its model
endpoint. A helper a signal ended now gives a failed solve of unknown cost,
which the measure records and stops on; one that exits on its own without a
report still ends the measure as a solve that could not run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(desktop): check verifier behavior after selector changes

* test(windows): verify rejected writes during cleanup by effect

* fix(desktop): preserve selected alias target kinds in exports

* fix(cli): retain caller capture paths in environment builds

* fix(ghidra): classify oversized endpoint records consistently

* test: align model and Evidence expectations with current contracts

* fix(desktop): reconcile upstream sandbox cleanup recovery

* fix(docs): link cleanup qualification as repository evidence

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 05:26:28 -05:00

607 lines
38 KiB
Markdown

# Desktop application export
`rea export-desktop-application <capture-folder> <directory>` turns a folder
written by [desktop application capture](desktop-application-capture.md) into an
agentenv bundle holding one computer app environment: the real application,
installed from the distribution archive, its vendor's apt repository or the
`.deb` file it was installed from on the `ubuntu-24.04` computer base, launched
at boot, seeded with the user's state and serving that state through agentenv's
data operations.
```sh
rea export-desktop-application ./mousepad-capture ./mousepad-bundle
agent-env run ./mousepad-bundle --dry-run
```
The export reads the capture and its [state view](desktop-application-state.md);
it does not run the application. `--name` sets the environment name; otherwise
it comes from the desktop entry id or the program: a reverse-DNS id's last
part (`org.xfce.mousepad` gives `mousepad`), or else the whole id without a
trailing version (`glabels-3.0` gives `glabels`), with an `app-` prefix when
that name would not be a portable bundle name (for example all digits). The
output directory must be new or empty. `--task` adds a task an agent wrote for
the capture, once REA has validated it against the capture; repeat it for
several tasks, each checked at its own end checkpoint, which
`--task-end NAME=CHECKPOINT` names; see
[desktop application tasks](desktop-application-tasks.md#several-tasks). `--package-url`
says where the computer downloads the package of an application installed from
a `.deb` file; see [Applications from a .deb file](#applications-from-a-deb-file).
`--download-install FILE` says which AppImage or tarball a build computer
unpacked the application from; see
[Applications from an AppImage or tarball](#applications-from-an-appimage-or-tarball).
`--extra-package NAME`, repeatable, installs another Ubuntu 24.04 archive
package beside the application, such as a runtime piece its package does not
install; a [build computer](desktop-build-witness.md) passes its request's
`extra_packages`, or a `.deb` request's `dependencies`, this way. A name that
is not a Debian package name is refused (`extra_package`). Names the bundle
installs anyway, the application's own package and the dependencies it names
for a package installed from its `.deb` file, are not repeated; the rest are
listed in the limitations and the README. A package the captured `.deb` lists
as one alternative, such as `openjdk-17-jre | java-runtime`, is kept when apt
installs the application by name, since apt may satisfy it with another.
`--seed-checkpoint` names the checkpoint whose state seeds the bundle, the
baseline by default, and `--end-checkpoint` the one a task must pass on, the
capture's last by default; see [The seed checkpoint](#the-seed-checkpoint).
`--synthetic-seed` says the seed was made for the task, as a build agent makes
it in a [build computer](desktop-build-witness.md), rather than taken from the
user's own data; see [A seed made for the task](#a-seed-made-for-the-task).
The command returns Evidence with operation `export_desktop_application` and
predicate type `rea.desktop-application-export`, listing the environment name,
the seed and end `checkpoints`, owned paths, seed files, the seeds it
rewrote to remove the capture host's name and user (`scrubbed_files`), the
gzip seeds it compressed again only to clear their headers
(`recompressed_files`), the bundle files that still hold them
(`unscrubbed_files`), and written files, the last four by their path in the
bundle. A failed export, cancellation
included, removes the bundle it had not yet published, so no partial copy of the
user's data remains. If the bundle was published but its output handles could
not be closed, the command reports a cleanup error that carries the export
Evidence; the bundle itself is complete.
## The bundle
| Path | Contents |
| --------------------------------- | -------------------------------------------------------------------------------------------------- |
| `envs/<name>/env.toml` | `type = "computer_app"`, the module and class, `computer = "ubuntu-24.04"`, `driver = "reference"` |
| `envs/<name>/<file>.py` | The `ComputerAppEnvironment` class |
| `envs/<name>/SKILL.md` | What the app is, its window class and where its files live |
| `envs/<name>/seed/` | The application's files at the seed checkpoint, by home-relative path |
| `envs/<name>/apt-keys/` | The signing key of each vendor apt repository the app installs from |
| `envs/<name>/install-package.sh` | With `--package-url`: the run script that installs the downloaded package with dpkg |
| `envs/<name>/install-download.sh` | With `--download-install`: the run script that unpacks the downloaded AppImage or tarball |
| `envs/<name>/applications/` | With a tarball's `--download-install`: the desktop entry REA wrote for its program |
| `tasks/smoke.json` | A task that deploys the app |
| `tasks/<task>.json` | With `--task`: each of the agent's tasks, its verifier in `artifacts/` and the eval in `evals/` |
| `README.md` | What was exported, from which capture, and what to keep in mind |
The class file is `<name>_app.py` with hyphens as underscores, for example
`mousepad_app.py`: agentenv imports it under its stem, so a plain `<name>.py`
could shadow a standard library module such as `calendar`. A class name that
would collide with a name the module imports or defines, such as `Path`, takes
an `App` suffix.
The class declares:
- `Packages(apt=[...])` with the package the capture found, then the
`--extra-package` names. A vendor package is named `<package>/<release>`,
its repository's release (the Suite apt reports as `a=`, or else its
Codename), so apt takes it from that repository
even when the Ubuntu archive offers a higher version or epoch, as noble's
transitional `firefox` does over packages.mozilla.org's; it is named alone
when the release names neither, or when the capture was written before
releases were recorded. Its library packages follow, so a package that
leaves out dependencies on the libraries it loads, as Typora's Electron
package does, still starts; library packages the capture found neither the
Ubuntu 24.04 archive nor a declared repository offers are left out, with a
limitation, and so are those only repositories the export leaves out supply
(a repository's `supplies`), such as one whose key the capture could not
retain. An archive package's declared dependencies are trusted as they
are. When the desktop entry's package is not the one that owns the program
it runs, as `vim-common`'s entry runs `vim`'s program, the program's package
(`executable_package`) follows the app's by name, since the entry's package
need not depend on it; a capture that found no archive or declared
repository offering it records why, and the export refuses it with that
reason, as it does when only repositories it leaves out supply it, or when
no package owns the program at all (`executable_package_reason` without
`executable_package`). The package must come from the Ubuntu 24.04 archive
or a mirror of it,
or from a vendor apt repository whose signing key the capture verified and
retained. A source counts as the archive when apt reports its origin as
`Ubuntu` for a noble suite; a capture made before origins were recorded
counts only Canonical's own archive hosts.
- For a vendor repository, `sources=[AptSource(apt=..., suite=...,
components=[...], key="apt-keys/<name>.gpg")]`, with the retained key copied
to that path (`.asc` when it is armored, which agentenv tells by its
content). A PPA added with `add-apt-repository`, whose sources entry holds its
key in `Signed-By`, exports that key as `apt-keys/<name>.asc`. apt takes one
key per URI and suite, so sources that share them, such as one repository
listed for two architectures, are declared once, with their components
merged; one with a different key is left out. A repository
whose key the capture did not verify, an Ubuntu Pro (ESM) repository, one
whose sources entry's URI held a user name or password (which the capture
did not record and the export never writes; the reason names neither them
nor the URI), one not reached over http or https, or one whose `arch=`
restriction leaves out the capture host's architecture is refused with its
reason (a capture
written before restrictions were recorded counts as unrestricted); when
another usable repository lists the same version it is left out instead,
with a limitation. The export also refuses snaps, flatpaks, an AppImage or
tarball not named with `--download-install`, and a
package no recorded repository lists unless it was installed from a `.deb`
file the capture recorded and `--package-url` is given, since the computer
could not install them by name, and a capture written before credentials
were removed whose repository URI still holds them, since it counted that
repository's libraries as installable; capture it again.
- `@computer_app(supports=...)` with both `amd64` and `arm64` for an archive
package, which the archive serves for both, and only the capture host's
architecture for a vendor package, since its repository may serve no other.
A vendor package captured on any other architecture is refused. A package
installed from its `.deb` file supports only that file's architecture.
- A launch at boot with the captured command, ready when a window of the
captured class appears and one of them is listed in the taskbar and is not
typed as a splash: a splash screen or a dialog of the class skips the
taskbar or says it is a splash, and an app whose splash shows first would
otherwise count as ready before its own window opens. Arguments that are the captured home or under it, and
`<option>=<path>` values under it, are respelled under the computer user's
`$HOME`. agentenv starts the app in its own folder, where an open or save
dialog would show the framework's files, so the command first changes to the
captured working directory (`"$HOME"` when that was the home, as for a
desktop entry without `Path`), which a relative argument or program path
such as `./app` also needs, and still starts if that folder is missing. The
export refuses a capture that saw no application window.
- The seed's document, opened at launch, so the app shows it rather than its
start screen: when the seed holds exactly one file the state view gives the
`document` role, and the desktop entry's `Exec` takes files (a `%f`, `%F`,
`%u` or `%U` field code), its path under `$HOME` is appended to the command.
A command that already names a document, absolute or from its working
directory, as one captured with `--argument <file>` does, is kept as it is;
a settings path under the home does not count. A seed with no document launches
as captured. With several, no desktop entry or an `Exec` that takes no file,
the app launches as captured and the README and the result's limitations say
why. A reset
relaunches the app, so it opens the document again.
- For an Electron application (the capture found its fuse sentinel or a
`resources/app.asar`), `--no-sandbox` right after the program unless the
command already has it: Chromium's sandbox needs user namespaces the
computer's container does not grant, and Electron exits without them. The
README and the result's limitations say the sandbox is off.
- `paths`: the state directories holding the application's state files and the
folders holding its documents, nested folders collapsed. A folder beside a
state directory whose name differs from it only in letter case, or that
adds a version, as TreeLine's `~/.treeline-3.1` beside `~/.treeline`, is owned
whole in the files' spelling, so a `~/.config/GIMP` profile GIMP created
during the capture beside the candidate `~/.config/gimp` is owned as
`.config/GIMP`, not as the folder of its first file. The home itself and
the shared `.config`, `.local`, `.local/share`, `.local/state` and `.cache`
folders are never owned whole: a file directly in one is owned as itself.
Files the capture recorded under a symlinked state directory's physical
spelling are mapped back to the innermost such directory's home spelling,
unless that spelling is outside the captured home, as when the home itself is
a symbolic link. When a selected link names an application state directory,
the bundle seeds the application's state path and restores the selected
alias at boot. Targets recorded as directories are created even when the seed
has no files there; file targets and targets of unknown kind are not turned
into directories. Selections nested under that alias are seeded under the same
state path. An existing incompatible alias stops setup without replacing it. When two files map to one home path, the export uses the one present
before the capture, else the first by path, and a limitation names the other.
- Data hooks, because declaring any hook switches off agentenv's default data
plane: the class seeds the home once per boot and keeps it as the initial
state with `keep_initial`, `data/add` and `data/reset` call agentenv's `add`
and `reset`, and `data/get` returns the files under `paths` plus a data part
with a typed JSON view of each, read in a worker thread. An application with
no observed state declares no paths and no hooks.
A file under the owned paths whose bytes at the seed the capture did not
retain is not seeded: one in the state view without retained content at that
checkpoint, or one the capture omitted (for example too large or past the
content budget). The result lists them in `unseeded_files` and the README
names them. When the application held files open in folders whose starting
content the capture did not retain, a limitation names those folders, spelled
from `~/` under the home: a file there is seeded only when the capture
retained it, as it does one written during the capture, and the limitation
names the seed files there or says there are none. The desktop Trash,
`~/.local/share/Trash`, is never the application's, so a file the user moved
there is neither owned nor seeded, even when a window title names it; a
limitation lists the files the Trash held at the seed.
Files that changed during the capture outside the owned paths are neither
owned nor seeded, so the app starts without what it saved there: MuseScore
keeps its settings in `~/.config/MuseScore`, which its name rules miss, and
showed its first-run wizard on every boot. A limitation and the README name
those folders, leaving out desktop services' files, each collapsed to its
folder in `~/.config`, `~/.local/share`, `~/.local/state` or `~/.local`, the
hidden folder of the home it lies in, or else its own folder, up to eight and
a count of the rest. Capture again naming them with `--state-path`, or
`state_paths` on a build computer, to own them.
Lock files the app holds while it runs are left out of the seed: a copied one
makes the app find its profile or document in use, and a LibreOffice Calc
bundle whose seed held its profile's `.lock` showed no window at boot. They
are `lockfile`, `lock`, `.lock`, `.parentlock`, `parent.lock`,
`SingletonLock` and Lucene's index lock `write.lock` anywhere, as Poedit's
translation memory holds, LibreOffice's document lock `.~lock.<name>#`, a
`<file>.lck`, `.<file>.lock`, `.<file>.lck` or `<file>.<ext>.lock` beside a
file the capture recorded in the same folder, such as GnuCash's
`household.gnucash.LCK`, Grisbi's empty `.household.gsb.lock`, which made it
refuse every save, and VYM's `plan.vym.lock` (a project's `Gemfile.lock`
beside its `Gemfile` has no extension to its file, so it is seeded), and
`*.lock` or `*.lck` directly in the home, as wxWidgets' single-instance
`~/poedit.lock`, or directly in one of the application's state directories;
a lone `notes.lck` or `.notes.lock` elsewhere is seeded. A lock
never makes a path the application's own, so `~/poedit.lock` is neither owned
nor named among the folders the bundle does not own.
Elsewhere such a file may be a project's own, such as `Cargo.lock` or
`yarn.lock`, which pin dependencies, and is seeded, even under a folder whose
name starts with a dot. A file the state view makes a document, or one named
itself by `--state-path` or an absolute `--argument`, is never left out,
whatever its name. A limitation and the README list them, up to
eight and a count, after "Left out of the seed". They stay under the owned
paths, so `data/get` returns the lock the running app holds, and `data/reset`
restores the seed without them.
A seed file keeps the permission bits it had at the seed when the capture saw
it change, such as a `0600` file an app checks: the class module lists those
other than `0644` in `SEED_MODES`, and the first boot sets them when it copies
the seed. A file the capture never saw change has no recorded mode and is
written `0644`.
Files in app state are retained regardless of a `.log` extension or `log.<n>`
name. They can hold conversation history or database writes as well as
diagnostics. The normal host-identity scrub applies to retained text; filenames
alone do not identify disposable diagnostics. Locks beside these files follow
the lock rules above.
Other seed files the app keeps for its own
running are seeded as captured and named, collapsed to the folder that holds
them: WebKit or Chromium storage (such as `hsts-storage.sqlite`, `storage/`,
`mediakeys/`, `Local Storage/`, `IndexedDB/`, `Cookies`, `Visited Links`,
`Platform Notifications/`, and a whole `QtWebEngine/`
profile), which may hold the app's own data, and caches (a folder whose name ends in `cache`); a stale one
can change how the app starts. A seeded LibreOffice profile's
`registrymodifications.xcu` ships without its `RecoveryList` items, which
LibreOffice writes for the documents it has open and which would start the
bundle into document recovery instead of its window; a limitation and the
README name the file. REA finds the items with its XML parser, as `item`
elements whose `oor:path` is `/org.openoffice.Office.Recovery/RecoveryList`
however the tags are spelled, and cuts exactly those elements, so text in a
comment or CDATA and every other byte stay as captured. A registry that is
not well-formed XML ships unchanged.
The application's dconf keys at the seed, if any, are written once when the
app first starts, also for an application that owns no files, and the class adds
`dconf-cli` to its apt packages for that. The server starts before the
desktop's D-Bus session, so the class does not use `dconf load`: it dumps any
existing user database through a `file-db:` profile, then runs `dconf compile`
on that dump and the captured keys, in that name order so the captured keys
win, and replaces `~/.config/dconf/user`. If that fails, the server prints a
one-line warning and the app starts without the captured keys. `data/reset` does
not restore them. No file of the bundle holds them, so `data/get` does not
return them and no task check reads them; a draft whose only change between
its checkpoints is to the application's dconf keys says so with
`uncheckable_changes`, and one whose dconf database changed but could not be
read says so with `unreadable_changes`. When a dconf database changed by the
seed checkpoint but its version there could not be read, as without
`dconf-cli`, the export seeds no keys from it and a limitation names it with
the read error.
## The seed checkpoint
By default the seed is the state before the capture, its `baseline`, and a
task's end state is the capture's last checkpoint. `--seed-checkpoint` and
`--end-checkpoint` name others, so one capture can make a seed and then a
change: the user creates and saves a document and takes `checkpoint seed`,
then edits and saves it and takes `checkpoint end`. Exported with
`--seed-checkpoint seed --end-checkpoint end`, the bundle seeds the document
as it was saved at `seed`, owned paths, dconf keys and unseeded files
included, and a task must fail on that seed and pass on the state at `end`.
A file's state at a checkpoint is its last version recorded at or before it.
The export refuses a name the capture does not hold exactly once, and an end
checkpoint taken before the seed checkpoint; `inspect-desktop-application-state`
lists the names in `capture.checkpoints`.
## A seed made for the task
By default the seed is the user's real data: the README and the result's
limitations say to keep the bundle local and not publish it. With
`--synthetic-seed` they say instead that the seed was made for the task, and
to read `envs/<name>/seed/` before publishing the bundle; the export cannot
tell the two apart, so the flag is the caller's word. `export_build_environment`
passes it, since the building agent makes the seed in the build computer. The
Evidence records it as `synthetic_seed`.
## Withheld sign-in state
`--withhold-from <checkpoint> --withhold-to <checkpoint>` withholds every file
under the owned paths that changed after the first checkpoint up to the
second, as the capture recorded its changes; `--withhold-path <file>` adds a
home-relative file, such as one written at first launch. The draft
(`describe-desktop-application-task`) takes the same options, leaves those
files out and lists them as `withheld_files`; the export refuses a check that
names one. Each withheld file present at the seed is left out of
`envs/<name>/seed/` and becomes a secret setting:
- the class declares `Settings` with a `SecretStr` field per file, and
`WITHHELD`, each field's home path;
- `env.toml` sets each field to `env:REA_<APP>_<FILE>`, so agentenv refuses
the deploy, naming the variable, until it is set;
- at first boot, before keeping the initial state, the class writes each file
from its setting, the file's bytes in base64, mode `0600`, so `data/reset`
restores it, and `data/get` serves it neither as a part nor in the views;
- the README names each variable, with
`export REA_<APP>_<FILE>="$(base64 -w0 < ~/<file>)"` from the receiver's
own signed-in copy of the app.
A withheld file larger than 48 KiB is refused, since its base64 must fit one
environment variable. `--withheld-copy <folder>` copies the withheld files,
as captured, into a new or empty folder outside the bundle. The result lists
them as `withheld_files`, each with its `setting`, `variable` and `handle`.
`--consent <file>` ships the person's consents, a `consent.json` that
`record_build_consent` records, as the bundle's own `consent.json`.
`--seed-withheld` seeds the withheld files after all, refused unless that
file holds a `seed_sign_in` consent; the result lists them as
`consented_files`, and the README says they are there with consent.
## Host name and user
A seed can hold the capture host's name and the user's: Geany saved its mail
template as `pref_template_mail=ada@e81232be457d`, the user `ada` on a
container named `e81232be457d`, and many files hold the home path
`/home/ada`. The capture records `host.hostname` and `host.user`, and the
export removes them from what the bundle ships:
- In seeded text files (text, JSON, key files and XML in the state view), in
UTF-16 files that start with a byte-order mark (keeping the mark, byte order
and a following U+FEFF content character), and in the dconf keys,
`user@host` becomes `user@localhost`, the
host name standing alone becomes `localhost`, and the captured home becomes
`/home/user` wherever it is followed by `/`, the end of the text, or a
character that cannot continue a file name. The host name counts only as a
whole name: not inside a longer word, before a `-`, or as a label of a
domain such as `e81232be457d.local`, so a host named `ubuntu` leaves
`ports.ubuntu.com/ubuntu-ports` alone. A host named `localhost` is left
alone, but its `user@localhost` still takes the computer's user. The user
name alone is never replaced, since a short one such as `ada` or `user` is
also ordinary text. A rewritten seed is written with its own SHA-256, and
`scrubbed_files` and the README list it.
- JSON and XML keep their syntax. In JSON only string values change, each
read as its decoded value, so `\u0065` escapes cannot hide the host, and is
written back as JSON only when it changed; numbers and `true`, `false` and
`null` stay as they are even for a host named `true`. Member names are never
rewritten, since `{"ubuntu": 1, "localhost": 2}` on a host named `ubuntu`
would get two `localhost` keys and lose a setting; a file whose names hold
the host is listed in `unscrubbed_files` instead. In XML only character
data, attribute values, comments and CDATA change, character references
decoded the same way; element and attribute names and processing
instructions stay. A UTF-16 file is scrubbed by the format its text would
have in UTF-8, so JSON or XML in UTF-16 follows these rules too.
- In the dconf keys only GVariant strings change, each read as its decoded
value and written back in its own quotes only when it changed. Group names,
keys, dictionary keys, bytestrings and everything outside quotes stay, so a
host named `true` leaves `wrap=true` alone. A value whose string does not
parse, or that holds an object path or signature, ships as captured.
- `SEED_HOME` in the class is `/home/user`, the home the seed spells, which
the class respells to the computer's own home when it seeds the home at
boot, as it did the captured home before. The replacements read as the
computer's own: its user is `user`, with its home in `/home/user`, and
`localhost` is the name every host has for itself.
- A gzip-compressed seed, such as a Gnumeric workbook, is decompressed, up to
256 MiB, scrubbed as the format the state view read its document as, and
compressed again whether or not its document changed, so no file name,
comment or time its header held ships: with no file name, comment or extra
field, modification time 0 and the operating system unknown, so the same
document gives the same bytes on every host. It is deflated at the fastest
level, or stored when deflating would make it expand more than the 100 times
a view reads, so a seed that read still reads. It is listed in
`scrubbed_files` when its document changed, and otherwise in
`recompressed_files`, which the README counts rather than lists, so the
example and template files an app copies on first run do not bury the
document scrubbed; one already in that form ships as captured. The class copies it at
boot as it is, so the home it holds stays `/home/user`. One that does not
decompress within the scrub limit ships as captured, its bytes searched.
A document the state view did not read can still be decompressed and
recompressed under the separate 256 MiB scrub limit; its content is searched
even when its format cannot be scrubbed. One whose document needs no edit but a member of
which has an extra field, where BGZF (`.vcf.gz`, `.bam`) and dictzip
(`.dict.dz`) files index their blocks, also ships as captured, its bytes
searched, so its blocks stay where its index says; one whose document
changed is still compressed again as one member, losing them, as the
export's limitation says.
- Binary files, SQLite databases among them, and text files over 256 MiB are
copied as captured. Then every file the bundle ships, the class module with
its dconf keys and the README included, is searched by a looser rule than
the replacement's: the host name and `user@host` anywhere, as in
`e81232be457d.local` or a seed named `e81232be457d-notes.txt`, and the
captured home as a whole path, in its path and its content, JSON and dconf
strings and XML references decoded and the replacements themselves set
aside. Each one found is listed in `unscrubbed_files`, and in the README
unless it is the README, to check before sharing the bundle. Bytes that are not text are
searched as they are, so a binary file that holds the host in another
encoding is not listed.
- A task check whose values name `user@host` or the host name anywhere, by
that rule, is refused, as one naming the captured home is: the bundle does
not hold them. Its field names, kind and id are not values, so a host named
`file` or `kind` refuses no check. A task's checks must fail on the seed as
the bundle ships it, scrubbed, so a check the scrub itself satisfies, such
as a mail setting the user changed from `ada@e81232be457d` to
`user@localhost`, does not count as failing there.
A capture written before the host's name and user were recorded has only its
home removed, with a limitation saying so; capture again to remove the rest.
## Applications from a .deb file
A closed-source application you installed from a `.deb` file you downloaded,
such as VariCAD from varicad.com, is not in any apt repository the computer can
use. Capture it with `--package-file` naming that file (see
[desktop application capture](desktop-application-capture.md)), then pass
`--package-url` to the export with an `http://` or `https://` URL the computer
can download that same file from:
```sh
rea capture-desktop-application --executable varicad --package-file ~/Downloads/varicad2026-en_2.03_amd64.deb --output ./varicad-capture
rea export-desktop-application ./varicad-capture ./varicad-bundle --package-url https://files.example.com/varicad2026-en_2.03_amd64.deb
```
The class then declares, beside `Packages`:
- `downloads=[Download(url=..., sha256=..., to="vendor/<file>", arch=...)]`
with the captured file's SHA-256. agentenv downloads the URL once, when the
bundle is put, and refuses the file unless its SHA-256 matches, so the
computer installs exactly the captured version. The bundle does not contain
the package.
- `Packages(apt=[...])` with the first alternative of each of the file's
Depends and Pre-Depends entries, without version constraints or `:any`
qualifiers, then the library packages the capture found, since a vendor's
binary may load libraries its Depends leaves out.
- `run=[RunScript(script="install-package.sh")]`, a script the export writes
into the environment folder. agentenv runs it as root from the app's folder
once the apt packages are installed and the download is in place, with the
network off: it runs `dpkg -i` on the download and then removes it.
The bundle supports only the file's architecture. Vendor sites often refuse
automated downloads, as varicad.com does, so the URL may need to point at a
copy you keep, such as a private mirror; you must be allowed to use that copy,
and the package's licence, such as a 30-day trial's terms, is yours to check.
The README and the result's limitations say so.
The export refuses a package no recorded repository lists when `--package-url`
is missing, saying to capture with `--package-file` if the capture lacks it. It
also refuses `--package-url` when the capture recorded no package file, giving
the capture's `package_file_reason` when there is one; when the archive or a
declared vendor repository installs the package already; when the URL is not
written as `http://` or `https://` or holds a user name or password; and when
the file is built for an architecture other than `amd64`, `arm64` or `all`.
## Applications from an AppImage or tarball
A [build computer](desktop-build-computer.md) can install an application from
its project's AppImage or tarball, pinned by URL and SHA-256, under
`/opt/rea-apps/<desktop entry>`. Neither holds a Debian package, so the
capture records the package as `unknown`, and the export takes the install
from `--download-install`, a JSON file the build computer writes from its
request:
```json
{
"kind": "appimage",
"desktop_entry": "org.freecad.FreeCAD",
"url": "https://github.com/FreeCAD/FreeCAD/releases/download/1.1.4/FreeCAD_1.1.4-Linux-aarch64-py311.AppImage",
"sha256": "675d1a4295b2183e0e1d816a4b5557dcb63aeced4e60aaa28d6c3686f4c71dc9",
"architecture": "arm64"
}
```
A tarball's adds `program`, its path inside the archive, and optionally
`name`. The class then declares
`downloads=[Download(url=..., sha256=..., to="vendor/app.AppImage", arch=...)]`
(`vendor/app.tar` for a tarball) and `run=[RunScript(script="install-download.sh")]`,
the same script the build computer ran, with no apt package for the app. The
bundle does not contain the application and supports only the download's
architecture; the URL must keep serving those bytes, and the application's
licence is the operator's to check.
The export refuses the file (`download_install`) unless it names an
`appimage` or `tarball` with an `http://` or `https://` URL and a SHA-256,
the capture launched its `desktop_entry`, the captured program lies under its
folder in `/opt/rea-apps`, and the capture ran on its architecture; it
refuses `--package-url` beside it.
## Grading once the app has saved
A task's verifier, `artifacts/<task>-check/verify.py`, grades the app's files
only once they have settled, because an app can save on its own timer after the
last change. Gnote saves a note 4 s after the last key, so a read taken as
soon as the agent stops finds the text on screen but not yet in the file. The
verifier reads the app's `data/get` every 0.5 s. It compares each file's bytes
between reads by their SHA-256, as `data/get` returns them, and grades once
every file has held for 3 s. The 3 s count from when a read returned the
current bytes, not when it began, so a slow read that brings a change late
does not count its own run as quiet. 15 s after the first read began, the
bound, the verifier grades the last completed read: no read starts at or
after the bound, and a read still running then is cut. Each result records
whether the files settled in `settled`, and whether the bound cut a read in
`read_cut`.
The window has a cost and a limit:
- An app whose files do not change during the check, such as one that saved
at once, is graded 3 s after the first read returns.
- A save that lands more than 3 s after the first read returns is still
missed. An app that saves 4 s after the last key is caught only when the
agent stops at least a second after that key.
- Files the app rewrites more often than every 3 s, such as a log, never
settle, so every check waits until the bound.
- The bound counts the reads too, so a slow `data/get` ends the check at
15 s. Only the first read, which every grade needs, runs to its own
timeout.
The bound and window are `SETTLE_BOUND_S` and `SETTLE_WINDOW_S` in the
verifier. `check_build_task` waits the same way before it grades each
witness stage; see [witnessing](desktop-build-witness.md#witness).
## Typed views
The views follow the [state view](desktop-application-state.md)'s formats and
shapes: SQLite tables (at most 64 ordinary tables, `tables_truncated` past
that, and per table `columns`, `primary_key`, at most 500 `rows` and
`truncated`, with blobs, unsafe integers and text that is not UTF-8 tagged as
REA tags them), JSON, XML as `{root, truncated}` with REA's node, depth and
whitespace rules and its `doctype` when the document declares one, key file
groups, text as
`{lines, line_count, truncated}` split on `\n` and `\r\n`, and zip archives
as their entries, read with Python's `zipfile` from the file in place, with
REA's entry, size, compression ratio and archive limits and its wording for an
entry it does not read. The module reads the central directory itself, as REA
does, and shows `zipfile` only the records it lists, reading each entry in
bounded chunks no further than a byte past its declared size. A gzip file is
the view of the document inside it, with `"compression": "gzip"` beside its
format, decompressed with Python's `zlib` within the same entry size and ratio
limits, its members joined as REA joins them. A JSON file
holding a number JavaScript cannot read exactly, such as an integer above 2^53,
or whose retained value nests more than 500 levels deep, gets an `error` instead of a view, as in
REA. Duplicate keys keep their last value before the depth check. Text that is not one JSON
document, such as JSON Lines, stays text. Format detection follows REA's
rules. Files larger than 64 MiB get an `error` instead of a view
and are not read. The selected `file_bytes` also applies to scrubbed seed task
views before admission, independently of how much seed data the export can
copy. SQLite table views have a shared `sqlite_view_bytes` allowance, default
16 MiB of compact UTF-8 JSON per table, checked before retaining each row.
An oversized table keeps its error while other tables remain readable. Raising
this view allowance does not raise the independent 16 MiB native SQLite
value/record limit or REA worker reply limit; worker facts include additional
schema and typed-cell metadata and may be refused earlier. The TypeScript
and exported Python readers count their respective native compact JSON encodings;
float and exponent spelling can make their exact byte thresholds differ even
when they report equal numeric facts. The Python byte counter quotes large
strings in bounded chunks before retention.
Exported SQLite readers require Python 3.11 or later for the native connection limits; the
AgentEnv compatibility workflow uses Python 3.12.
XML is read with Python's `expat` twice: with namespace
processing, which refuses what REA's parser refuses, such as an undeclared
prefix, then without it, so element and attribute names keep the prefixes the
document spells and namespace declarations stay attributes, as in REA's view;
text splits into nodes where REA's splits it, at CDATA sections, comments and
processing instructions. Both passes read a DOCTYPE as REA does: with
parameter entity parsing off, so the DTD it names is never read, and handlers
that refuse an internal subset, an external entity and an entity expat would
skip; the first pass also checks each start tag for a reference to an
undeclared entity, which expat drops from an attribute value without a word
when a DOCTYPE names a DTD. Known differences remain:
- dconf databases are reported as binary, since REA reads them with `dconf`;
inside an archive, one is `dconf` with an error, as in REA.
- Zip entry names: a name in an Info-ZIP Unicode Path extra field is used only
where Python's `zipfile` reads it (3.12 and later), and an entry whose local
header names it differently from the central directory is damaged rather
than read.
- SQLite databases are read in place, read-only, rather than from a staged copy;
a write lock the running app holds is waited out for a quarter of a second
and then gives the file an `error`, and so does a table or column name that
is not UTF-8.
Unless it was [made for the task](#a-seed-made-for-the-task), the seed is the
user's real data. Keep the bundle local and do not publish it.
The class targets agentenv-framework 0.9.1485.