mirror of
https://github.com/morluto/rea.git
synced 2026-11-04 13:56:06 +00:00
docs(environment): record verification for blank members and sorted writes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
5a3c6762ac
commit
2fe849da0d
3 files changed
+45
-9
No files matched your search
@@ -37,8 +37,9 @@ natural-language tasks ask for, or withheld them:
|
||||
or loaded so. An unsent update field that changes only between `null`, `[]` and `{}` is unchanged.
|
||||
- **Unruled fields.** A later create that stored a field no rule writes otherwise than the plan keeps it is
|
||||
`unexplained-change` at the create, so its recorded entity is held and later reads check other writes as before.
|
||||
- **Skill.** The second scratch create fills only what a person must give, and the skill shows how to read signed-in
|
||||
pages' accessibility trees from a scenario.
|
||||
- **Skill.** The two scratch creates differ in what they send, the second only what a person must give, however they
|
||||
are made; the skill shows how to read signed-in pages' accessibility trees from a scenario, and re-records once with
|
||||
request steps a UI write whose body the browser did not expose.
|
||||
|
||||
Plans and views that need none of this are unchanged.
|
||||
|
||||
|
||||
@@ -7,5 +7,5 @@
|
||||
- [x] Withhold later creates that store an unruled field otherwise.
|
||||
- [x] Add view, serving and unit tests.
|
||||
- [x] Document the rules and view member, and update the skill.
|
||||
- [ ] Measure the recorded captures and served write tasks.
|
||||
- [x] Measure the recorded captures and served write tasks.
|
||||
- [x] Run static, formatting, documentation and test gates.
|
||||
@@ -1,6 +1,6 @@
|
||||
# Verification
|
||||
|
||||
Executed on macOS with Node v24.22.1 at `54993559` (main `99062fea`), and on Ubuntu 24.04 VMs with snap Chromium for
|
||||
Executed on macOS with Node v24.22.1 at `5a3c6762` (main `99062fea`), and on Ubuntu 24.04 VMs with snap Chromium for
|
||||
the agent recordings.
|
||||
|
||||
## Gates
|
||||
@@ -71,14 +71,49 @@ the required members.
|
||||
| Total | 42 | 18 19 | 24 23 |
|
||||
|
||||
No recording passes fewer tasks with this branch. Paperless creates pass once served. The saved-values Grafana
|
||||
recording's rename is served and passes; its create still needs the saved-values branch's composed address. The
|
||||
Vikunja task create (its project path writes a type its list does not show as a list), EspoCRM's leftover contact and
|
||||
the main Homebox recording, which wrote no tags, are unchanged.
|
||||
recording's rename is served and passes; its create still needs the saved-values branch's composed address.
|
||||
Vikunja's task create, EspoCRM's leftover contact (held as seeded data, so every "nothing else changed" check fails)
|
||||
and the main Homebox recording, which wrote no tags, are unchanged.
|
||||
|
||||
**Fresh recordings.** FRESH_RESULTS
|
||||
**Fresh recordings.** Each application was recorded afresh with main (skill 49) and with this branch. The branch was
|
||||
recorded again for the four applications whose first branch recording passed fewer tasks, and a third time after the
|
||||
skill's sixth step was reworded; the final skill's recordings are the last column (served writes, then tasks):
|
||||
|
||||
| Application | Write tasks | Main | Branch, first two recordings | Branch, final skill |
|
||||
| ----------- | ----------- | -------------------- | ---------------------------- | -------------------- |
|
||||
| Paperless | 3 | 2, 1/3 2/3 | 8, 1/3 2/3 | |
|
||||
| Linkding | 2 | 0, 0/2 0/2 | 5, 2/2 2/2 | |
|
||||
| Miniflux | 3 | no bundle | 4, 3/3 3/3 | |
|
||||
| Grafana | 2 | 3, 2/2 2/2 | 1, 0/2 1/2; 3, 1/2 2/2 | 0, 0/2 0/2 |
|
||||
| Vikunja | 3 + 2 | 3, 1/3 1/3 + 2/2 2/2 | 0, 0/5 0/5; 0, 0/5 0/5 | 1, 1/3 1/3 + 0/2 0/2 |
|
||||
| EspoCRM | 3 | 2, 1/3 1/3 | 2, 1/3 1/3; 2, 1/3 1/3 | 2, 1/3 1/3 |
|
||||
| Homebox | 2 | 4, 2/2 2/2 | 1, 0/2 0/2; 1, 0/2 0/2 | 3, 2/2 2/2 |
|
||||
|
||||
Every fresh recording builds the same plans with main and with this branch, except Paperless (main serves 0 of the
|
||||
branch recording's writes and 2 of 3 of its own), Vikunja's main recording (3 to 4) and the EspoCRM recordings below;
|
||||
the per-application differences are what each agent recorded:
|
||||
|
||||
- **Paperless:** the branch recording serves creates, renames and deletes of tags, correspondents and document types.
|
||||
Its two tag creates used a light and a dark color, whose derived `text_color` differs, so the tag create stays
|
||||
withheld and the tasks, which create a tag, score as on main.
|
||||
- **Linkding:** main's agent recorded no writes. The branch's second bookmark create sent only `url` and `title`, so
|
||||
the served create requires only those.
|
||||
- **Miniflux:** main's agent stopped and asked for an API key. The branch's agent created a token in the UI and sent it
|
||||
as a literal header value, which REA recorded as written; the instance and its token were destroyed afterwards.
|
||||
- **EspoCRM:** following the sixth step, the branch's second contact sent only a last name, with `null` for strings the
|
||||
first stored and for flags it sent `false`. That merge was refused until the empty-spelling fix in this PR; with it,
|
||||
the create requires only `lastName` and `name` and its task passes, as on main.
|
||||
- **Vikunja:** the first two branch agents wrote through the UI, whose request bodies Chromium did not expose, so every
|
||||
write was `body-unusable`. The sixth step then read "fill every field the form offers"; reworded to say how the
|
||||
creates differ, not where they are made, the third agent wrote with request steps. It recorded no label writes.
|
||||
- **Homebox:** the first two branch agents recorded item writes only, the third tags as main's did.
|
||||
- **Grafana:** the branch agents created folders with request steps and their own uids. The second create's address
|
||||
needs the saved-values branch's composed-address fixes to be explained, so the create is withheld here.
|
||||
|
||||
## Not verified
|
||||
|
||||
- One or two fresh recordings per application and version; agent runs vary more than the builds do.
|
||||
- One to three fresh recordings per application and version; agent runs vary more than the builds do.
|
||||
- Request bodies Chromium does not expose for some UI writes (Vikunja's); the skill now re-records such a write once
|
||||
with request steps.
|
||||
- Paperless derives `text_color` from `color`, so a create whose two recorded colors differ in brightness stays
|
||||
withheld; EspoCRM's create still requires `name` and `emailAddressData`, which it derives from other members.
|
||||
Reference in new issue
Block a user