mirror of
https://github.com/morluto/rea.git
synced 2026-11-11 01:13:39 +00:00
c8a52fd478bb6a2367cbf9440a32da6f6cc951ab
14
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
3230f50b16 |
feat(sqlite)!: page selected rows with a stated order and row offset (#9331)
* feat(sqlite): page selected rows with a stated order and row offset readRows read `NOT INDEXED LIMIT ?` with no ORDER BY and reported order: "unspecified", so an agent could not reach rows past row_limit deterministically: raising row_limit hits the fixed 16 MiB reply budget, and the sampled order depends on whichever index SQLite happened to scan. Read the window in a unique, stated order and expose the position: - rowid tables order by rowid unless all of rowid, _rowid_ and oid are declared columns, in which case the order stays unspecified. - WITHOUT ROWID tables order by the primary-key columns, reusing each key's DESC flag and declared collation from pragma_index_xinfo so the plan keeps index order instead of materialising a sorter. - rows.order, rows.row_offset and rows.next_row_offset report the window; next_row_offset is null once the window reaches the end. - row_offset (CLI --row-offset) requires table and defaults to 0. A non-zero offset is refused when the table declares no usable order, because the skipped prefix would be arbitrary. The schema now checks next_row_offset against truncated and the offset so a worker reply cannot claim a page it did not read. One attribution fix came with it: adding ORDER BY text can push an over-wide projection past SQLite's SQL-text limit, which fails as SQLITE_TOOBIG before the result-column limit is reported. Attribute that case to column capacity too, so a 250k-column request still reads as capacity rather than a byte budget. Verified: npm run check:fast, npm run format:check on the touched files, npm run verify:module-boundaries, npm run docs:check, and 196 tests across the sqlite boundary, application and CLI/MCP suites. A 1002-row inspection and in-memory EXPLAIN QUERY PLAN checks confirmed the WITHOUT ROWID path adds no sorter. Full check:pr and the complete test suite were not run. * fix(sqlite): resolve the rowid alias and keep paging claims reachable Three defects in the row paging added alongside row_offset: - readRowOrder emitted ORDER BY rowid for every rowid table, but a table that declares a rowid column shadows that spelling, so the sort used the stored column and could repeat or drop rows across pages. Order now names the first alias the table leaves unbound, read from raw name bytes so an unrelated malformed identifier is not decoded just to choose an order. - The inventory projection could hide the primary key a WITHOUT ROWID table orders by, silently degrading a paginated read to unspecified order. The order key is now read from the table itself rather than from the columns the caller asked to see. - A truncated table with no stated order reported a next_row_offset that the row_offset rule then refused. It is null there, and the reply limitation no longer claims an unspecified order for windows that have one. rowEvaluationFailure also probed the first rows of the table instead of the requested window, so a failure confined to a later page was attributed to columns that evaluate fine there. * test(sqlite): assert the concatenated pages The paging regression checked only counts, offsets and flags, so a second page that repeated the first page's rows would still pass. Assert the values read across all three pages instead. * fix(sqlite): decode row keys in the database encoding readRowKeyFacts decoded primary-key names as UTF-8 and probed a fixed hidden set, so two supported schemas could not be paged at all: - A UTF-16 database stores key names as UTF-16 bytes, so "k" decoded to NUL-padded text and ORDER BY named an identifier that does not exist. Pass the detected encoding, as readColumns and the index-metadata path already do. - A virtual generated column named rowid (hidden=2) was absent from the shadow check, so ORDER BY rowid evaluated the generated expression instead of the rowid and could sort non-uniquely or fail outright. Include generated columns in the alias probe. Also stop decoding primary-key names for rowid tables: their ORDER BY names an alias, never a declared key, so an unselected malformed key name could abort an otherwise readable selection. The page-concatenation assertion now narrows SqliteValue before reading .value, which the null/blob arms do not carry; this was failing the Static checks typecheck job. * fix(sqlite): order WITHOUT ROWID rows by the index's own key sequence `readRowOrder` filed `pragma_index_xinfo` key terms into a map keyed by column name, so a valid composite key that repeats a column — `PRIMARY KEY(a COLLATE BINARY, a COLLATE NOCASE DESC)` — collapsed to one term. The generated clause then overstated uniqueness: it omitted the BINARY term, so `A`/`a` ties survived, and the `row_offset` paging advertised a `primary_key` order that the actual key does not establish. Build the terms from the index rows in `seqno` order instead, keeping repetitions, and bail to `unspecified` if any declared key column never appears in the index. The declared collation stays explicit so a caller column's own collation cannot re-sort the window. Verified against a real SQLite database that the repeated key is accepted and that the old clause orders A, a, B while the index orders A, B, a. * fix(sqlite): accept a null next offset on unordered truncated rows readRows already withholds next_row_offset when a table shadows rowid, _rowid_ and oid and declares no primary key, because every non-zero offset is refused there. The reply schema still required the offset whenever truncated was set, so the provider rejected its own worker result and the first sample never reached CLI or MCP. Require the offset only when the order is stated. * fix(sqlite): probe row keys only for ordinary tables and name row_offset readRowKeyFacts ran for a selected virtual shadow table before readRows could reject it, so an unavailable module surfaced as SQLite's "no such module" instead of the selection refusal. Probe the key only when the selected kind is an ordinary table and leave the order unspecified otherwise, since no rows are read for that kind. The unordered-offset refusal also reported the generic selection reason, which callers see against table. Give it its own reason so the failure is attributed to row_offset. * feat(sqlite)!: page selected rows with a stated order and row offset Selected-row inspection currently reports only rows.order: "unspecified" and offers no way to reach rows past the first sample deterministically. Main also refuses non-zero row_offset today, so nothing relies on that omission; this change makes the stated order and the offset the contract, reports the next offset the caller feeds back, and keeps the existing byte, heap, statement and deadline limits. BREAKING CHANGE: rows.order becomes rowid | primary_key | unspecified, and rows gains required row_offset and next_row_offset members. A client that validates selected-table output against the previous schema (where order was the only permitted value) rejects a now-valid reply. Migration: accept the two new members and treat order: "rowid" | "primary_key" as the stated order the offset applies to; a null next_row_offset means the window reached the end. * docs(sqlite): say a null next offset also means no stated order The selected-rows limitation described next_row_offset as null only when the window reached the end. An unordered table whose row_limit truncates the sample also publishes null, because an offset there would name a request the row-offset rule refuses, so the Evidence read as if the sample were complete. State the second cause and pin it in the existing regression. * test(sqlite): drop the limitation wording assertion and state the unordered exception The boundary test pinned the sentence a caller reads in `limitations`, so a harmless rewrite of that text failed a paging test whose real subject is `order`, `truncated` and `next_row_offset`. docs/development/testing.md asks for facts over frozen explanatory prose, and the three structured expectations already fail if the paging fact changes. The tool contract promised that retained rows always have a unique order and can be paged. A rowid table that shadows rowid, _rowid_ and oid and declares no primary key has no such order: it returns order=unspecified with a null next offset and refuses a non-zero row_offset. Say so in the description so callers do not plan a continuation the tool always refuses. * docs(sqlite): say unordered tables cannot be paged The description told callers that a truncated table without a stated order needs "the table read without offsets", but omitting row_offset starts at zero again: with an unchanged row_limit the next call repeats the first sample instead of continuing past it. Say that these tables cannot be paged and point at row_limit for a larger first sample. * style(cli): wrap the native data type path description The `inspect-native-data-type` argument description line is 82 columns, over the repository's 80-column `.oxfmtrc.json` limit, so `npm run format:check` fails on main and on every branch merged from it. It came in with #9465 and the following commits to this file did not touch it. Reformat the one call the formatter objects to; no behaviour change. * fix(sqlite): bound rowid alias comparison by name byte length readRowKeyFacts decoded every column name before testing it against three short aliases, so a table declaring a multi-megabyte name could exhaust the worker heap even when the caller projected the column away. Reject names that cannot be an alias by their byte length, accounting for the database encoding, before building any string. --------- Co-authored-by: 2SinceJuly <202121119+2SinceJuly@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
gensokyo-dev
|
e18594b247 |
refactor(composition): unify capability bindings and cleanup ownership (#8441)
Bind typed service creation, observed availability and retryable cleanup in one composition owner. Reuse selected CLI/MCP factories and canonical contract groups without copying provider operation inventories. Closes #8429. Refs #740. Validated paired real application workflows, actual local Codex CLI Terra/high use, complete Evidence and Unknown checkpoint parity, compiler-backed import boundaries, SDK schemas and final real mixed routing. Automated review findings were fixed at their producing owners. Existing exact-base CodeQL alerts are documented on the PR under the owner-authorized unrelated-check exception; no global alerts or security rules were changed. Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning) |
||
|
morluto
|
60149282fc |
test: strengthen integration oracles and verification (#8295)
* test: strengthen integration oracles and verification * test: bound lifecycle gates and strengthen workflow checks * test: verify complete paged and extracted results * test: preserve bytecode and cancellation evidence * test: strengthen retained evidence and fixture cleanup * test: require cross-projection bridge identities * test: retain fixture owners through cleanup failures * test: verify graph examples and project selectors * test: tighten provider and lifecycle regressions * test: consolidate hex coverage and bound cutter cleanup * test: bound subprocess output waits and strengthen graph checks * test: check mcp recovery and preserve cleanup owners * test: verify structured mcp argument failures * test: preserve artifact cleanup and gdb limit evidence * test: preserve evidence identities and bound provider gates * test: retain browser and comparison cleanup owners * test: bound startup and strengthen fixture oracles * test: bound lifecycle gates and verify recovered evidence * test: verify public outcomes and lifecycle ownership * test: assert precise evidence and release lifecycle gates * test: verify selected facts and bound cleanup waits * test: check retained evidence and decoded records * test: verify final reports and retain cleanup ownership * test: bound reviewed startup gates and retain cleanup * test: verify PE errors and preserve MCP cleanup * test: preserve MCP fixture cleanup ownership * test: release provider gates before late startup |
||
|
morluto
|
ccdeda7a27 |
fix(inputs): centralize admission and preserve caller selections
Move shared request parsing into its application owner, migrate all consumers, and remove six duplicated admission blocks while preserving cancellation and validation policy. Keep operator-selected JavaScript paths in verifier parent Evidence while retaining canonical comparison guards. Validated with 204 passing tests, 3 Linux-x64-only skips, typecheck, lint, and test discovery. |
||
|
morluto
|
6c35d74a7d |
refactor: centralize input validation and schema interpretation
Migrate twelve application service files to the existing provider input owner, preserving schema causes, cancellation precedence, caller selection, and retained Evidence. Keep flows with distinct error policies separate. Replace the DeepSeek verifier's recursive object scan with schema-position traversal using the canonical keyword and local-reference owners. Literal pattern fields no longer cause false regex failures; patternProperties keys are checked alongside actual pattern constraints in native regex modes. Validation: 271 focused workflow/service tests and six verifier regressions passed; static checks, discovery, and actual Node source-loader smoke passed. The baseline scanner reproduced the new regressions. Installed DeepSeek client execution was not run. |
||
|
|
69a2faa120 |
feat: add FreeBuff agent support for REA setup (#2736)
* feat: add FreeBuff agent support for REA setup Register FreeBuff as a supported setup client using the shared user agents directory (~/.agents/mcp.json with mcpServers, skills in ~/.agents/skills), which matches what FreeBuff Desktop loads. Adds the client definition, boundary tests, supported-agent docs, and the skill reference client list. * fix(freebuff): detect FreeBuff from its profile directory, not shared .agents The ~/.agents marker made FreeBuff appear installed for any home where REA had installed shared skills (the default skill destination, also Gemini CLI's), producing false doctor failures. Detect FreeBuff from its own fixed profile directory ~/.config/freebuff-desktop instead, and add a regression test covering a home with only the shared agents skill directory. * fix(setup): require product evidence for FreeBuff discovery Address review findings at their producing boundaries and preserve directly affected consumers. Source inspection and git diff --check completed. Tests and automated checks were not run at the user’s explicit request. * fix(uninstall): retain unverified shared client configuration * fix(clients): preserve scoped removal evidence * fix(doctor): omit undefined selectedClientIds under exactOptionalPropertyTypes Passing selectedClientIds explicitly as undefined fails typecheck on this branch (TS2379), which blocks the focused test workflow before any test can run. Spread the option only when defined. * fix(update): separate ownership evidence from product applicability rea update's maintenance scope inspected only product applicability evidence, so when a FreeBuff profile was absent but REA already owned a version-pinned row in the shared ~/.agents/mcp.json, the plan omitted freebuff and left the integration stale. FreeBuff now declares its registration storage as shared, which proves ownership of an owned row but never product applicability, and maintenance requests ownership evidence through that declaration instead of reinterpreting the shared paths. Doctor and setup keep readiness evidence, foreign rows stay out of maintenance, and the Doctor exactOptionalPropertyTypes fix makes the focused test workflow runnable. * test(dotnet): split oversized operand extent diagnostics describe The describe callback reached 162 lines (max 150), failing max-lines-per-function and blocking the pre-push check for this branch. Move the single-case extent scenarios into their own describe; no test content or order changes. * fix(update): preserve ZCode's disabled shared registration during maintenance The shared user agents file doubles as ZCode's fallback, where `enable: false` disables the registration. Ownership-based maintenance read that row as a FreeBuff registration, whose format does not recognize the flag, so the row was selected and the emitted setup command rewrote it, dropping the flag and silently re-enabling a registration the user had disabled. registrationActivation now honors `enable: false` for any client whose registration storage is shared, because the activation fact binds every reader of that storage. Maintenance leaves such rows out; doctor reports them as stale, matching ZCode's own treatment. * fix(doctor): let explicit client selection authorize the config read When an ACL or an inaccessible shared parent made every access probe fail, a client selected by name was omitted before the configuration read could return its unreadable-config detail, so doctor reported it as not detected with installation remediation: an unresolved selected path collapsed to absence. An explicit selection now authorizes the read outright and the probes serve automatic discovery only. A selected client with an unreadable config reports invalid with the read failure; one with no config reports missing; automatic discovery still probes before inspecting. * fix(uninstall): share shared-storage checks between ZCode and FreeBuff clients() queued ~/.agents/mcp.json twice: once as zcode_fallback and once as freebuff. ZCode has no discoveryPaths, so its strict read still blocked uninstall on malformed shared text, despite FreeBuff's new retain-and-continue behavior. For a valid owned entry, ZCode removed it first, so FreeBuff reported skipped rather than removed. Make the shared file's checks and removal one shared decision: - Add zcodeDiscoveryPaths to ZCode so it has its own applicability evidence. - Gate zcode_fallback push on applicable, not just zcodeFallbackPath presence. - Report shared storage ownership as one decision for both clients: FreeBuff and ZCode now use clientOwnershipEvidencePaths together, so a recognizable REA-owned row proves ownership once for the shared storage, and both clients report the same result consistently. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(uninstall): merge duplicated host and prove shared ownership from owned rows Uninstall.ts declared systemUninstallHost twice, preventing module load, with the exported copy returning only clients and the second copy missing clientWriteBlocker. Merge into one exported host carrying both blockers. hasOwnershipEvidence used access() existence, so an unrelated malformed ~/.agents/mcp.json established ownership and blocked every removal. Ownership now requires a readable, recognizable REA-owned row via the injected filesystem, so unrelated shared data stays retained. Also cover ZCode discoveryPaths in its path test. * fix(uninstall): keep denied product-local configurations fail-closed Unproven product applicability marked every configuration path of that client unverified, so a denied ZCode profile turned an unreadable product-local config into a retained item: uninstall continued and could report complete with REA still registered. The retain-and-continue exception now applies only to storage REA does not own, classified once by clientSharedRegistrationPaths (shared configuration plus another product's same-scope fallback). Covers the Greptile P1 on Uninstall.ts. Validation: the new test fails on the pre-fix Uninstall and passes after it; freebuff/zcode/updateMaintenance/bom/dsh suites green; clientLifecycle, clientRegistrationStatus and claudeDesktop keep the same 8 host failures as the pre-merge baseline (symlink EPERM, stale-vs-aligned on Windows). * fix(update): retain FreeBuff when ZCode reads the shared fallback Maintenance deduplicated clients on the path each inspection read, so ZCode's fallback view of ~/.agents/mcp.json marked that file "covered" and dropped FreeBuff from the scope. ZCode setup writes ~/.zcode/cli/config.json and deliberately leaves the fallback untouched (docs/client-configuration.md), so the emitted command refreshed only ZCode while the shared row both products load stayed pinned to the old package version. Each setup action rewrites only its own configuration path and no other action rewrites the shared row, so every selected client keeps its own maintenance action; the read-path covered/deduped filter is removed. Covers the Codex P2 on UpdateMaintenance.ts:104. Validation: the new regression fails on |
||
|
morluto
|
58fb824105 |
refactor(evidence): centralize JSON payload admission
Give ordinary Evidence creation an explicit unknown-payload input contract while retaining the strong JSON contract for producer helpers and cooperative immutable creation. Keep schema admission, input snapshotting and identity hashing with the Evidence owner. Remove 32 generic JSON preparses that only fed ordinary Evidence construction. Preserve semantic format validation, selected-source bindings, standalone JSON boundaries, typed executions and scenario digest contracts. Invalid JSON now reports its Evidence parameters/normalized_result/raw_result prefix. Validation: fresh runtime/catalog build, 560 focused and README tests passed, 4 platform-specific cases skipped, full static/docs checks, test discovery, duplication and diff checks. New regressions cover omitted semantic optionals, raw-result defaults and nested undefined refusal without input mutation. Installed providers, real browsers/Frida/JADX and Linux-only recovery remain unverified; boundary providers are authored fixtures. |
||
|
morluto
|
bd4f5e9c19 |
fix(parsers): preserve format evidence and caller input identity
Retain bounded FAT and offset-tagged PE observations through archive hashing, classification, and content nodes. Preserve PyInstaller member paths according to the observed carrier dialect and share APK v1 signing-name semantics. Read SQLite's implicit sqlite_schema table without inventing a stored schema declaration. Distinguish unsupported ELF numbering and decoded-name admission from malformed data and fixed scan allowances. Preserve Sigrok basenames and classify matching explicit input-format refusals at their owning boundary. Validation: 554 distinct tests passed, 3 skipped across serial focused runs; build, typecheck/lint, docs, formatting, and discovery (2141 files) passed. Sigrok command-boundary simulators do not establish real-tool decoding. Small repeated public PyInstaller CLI runs had equivalent inventory facts; warm median elapsed 0.5431s before / 0.5426s after, with no speed claim. Fixes #6733 Fixes #6741 Fixes #6722 Fixes #6732 Fixes #6736 Fixes #6734 Fixes #6764 Fixes #6766 Refs #6739 |
||
|
morluto
|
cdd718ffb0 | refactor(inputs): own absolute path validation in shared contracts | ||
|
gensokyo-dev
|
f801b2fbfd |
feat(sqlite): make worker deadlines adjustable with recovery guidance (#5818)
Expose a per-call SQLite worker deadline with exact CLI/MCP recovery controls, parent-owned execution metadata and historical unknown preservation. Keep fixed resource limits and cleanup admission intact. Share typed cleanup composition with one captured-output owner and preserve the timeout after cleanup resolves. Fixes #5812. Follow-up tracking: #5808. Existing CLI test-oracle failure: #5845. Verified 155 SQLite tests (3 skipped), static/docs/discovery checks, six real CLI/MCP pairs and 13 actual Codex sessions including four corrected deadline recoveries. Three pre-dispatch discovery failures remain explicit, not passes. Review findings include real retained-cleanup and fixture ownership regressions. Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing) |
||
|
morluto
|
dceefa22cb | Merge remote-tracking branch 'origin/main' into codex/audit-continuation-20261020-refresh20 | ||
|
morluto
|
6a6adf703f |
fix(resources)!: inspect app resources without an executable
Add a read-only app resource session shared by CLI and MCP. Preserve resource-root provenance without inventing executable identity, reject binary-only snapshots and operations, and retain failed cleanup owners across switches and retries. Keep filesystem admission failures typed and preserve cancellation ordering from observed cleanup facts. Document resource routing in the packaged investigation skill. BREAKING CHANGE: resource-only observations may have a null Evidence subject and null executable target digests. Clients must use resource_root and reported per-resource provenance instead. Refs #5589 |
||
|
|
6b101ff23b |
fix(errors): preserve caller abort reasons at admission (#5671)
* fix(errors): preserve caller abort reasons at admission Keep the opaque caller reason on reverse, pcap, and SQLite cancellation, and leave a recognized physical digest or command failure in place when a late abort is also observed. Fixes #5670 Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(reverse): keep a foreign digest ABORT_ERR as an adapter failure Treat ABORT_ERR as cancellation on the command boundary, where that code already meant an aborted provider process. A digest or stat rejection with the same code stays a physical adapter failure unless the caller signal is aborted. Cover aborted command EACCES, ETIMEDOUT, and ABORT_ERR with and without captured output. Co-authored-by: morluto <morluto@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <morluto@users.noreply.github.com> |
||
|
YsopwnableandY0UW1SH
|
aeee5074d1 |
feat(sqlite): inspect offline snapshots with selective evidence (#2271)
Add offline SQLite database/WAL inspection through the shared CLI/MCP. Use compact inventory, selected-table schema, exact ordered columns and bounded row samples while preserving source identity and lossless values. Retain completed Evidence through late cancellation and private-root cleanup failures, bound concurrent inspections, and recheck journal validity after acquisition. Reuse prepared CI runtime artifacts across supported Node lines. Co-authored-by: Y0UW1SH <29498139+Y0UW1SH@users.noreply.github.com> Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing) |