Commit Graph
23 Commits
Author SHA1 Message Date
morluto 7181e7f515 perf(mcp): advertise large output union alternatives as shared definitions (#11005)
* perf(mcp): advertise large output union alternatives as shared definitions

inspect_analysis_view's advertised output schema held every view result inline in one union definition, which the MCP SDK's default validator compiled into one function: about 1.7 s CPU and 500 MiB peak. Large union alternatives now become definitions of their own and identical definitions are merged, at the owning output projection. Tests that check results against the advertised output contract now compile that schema instead of a separate inline projection, which overflowed the main-thread stack under forked workers.

* perf(tests): run source test projects in forked workers (#11016)

Thread workers of the source lane share one process, which grew to 3.8-5.0 GiB; forked workers keep each heap separate so the largest process stays near 1.2-1.8 GiB.

* test(mcp): select the analysis view contract from its family
2026-11-15 06:13:21 -05:00
morluto babfc87ec0 fix(mcp): present discover_tools input through the selected profile (#10592)
* test(mcp): check advertised schemas against a portable dialect in every profile

* fix(mcp): re-emit advertised patterns in a portable regex grammar

* fix(mcp): present discover_tools input through the selected profile

The hot-cold catalog registered discover_tools directly with the SDK, so
the default compact input profile still advertised its canonical schema,
including page-size defaults clients apply on their own (#5322). Route
every listed input schema, transport discovery included, through one
server-owned presentation, and pass the selected presentation to the
deferred server, which serves recorded tool schemas but not discovery.

* fix(mcp): validate advertised dialect by selected profile

* fix(mcp): preserve schema widening through references

* fix(mcp): refuse zero-padded repeat bounds and record host-specific path patterns

* fix(mcp): keep references valid when presentation widens their targets

* fix(mcp): relax references into negations and conditionals presentation may remove

* fix(mcp): keep astral literals and dynamic references widening in every mode

* fix(mcp): omit astral literals and relax untracked references when presentation widens

* fix(mcp): read reference pointers by schema position, not segment spelling

* fix(mcp): protect references into collapsing boolean unions and keep referencing branches out of the collapse

* fix(mcp): omit patterns whose classes name astral characters

* fix(mcp): decide union collapse and pointer protection from canonical branches

* docs(mcp): document non-u astral matching and pin BMP agreement across V8 modes

* test(mcp): expect the advertised base64 schema to omit the unbounded pad-bit probe

* fix(mcp): protect references into unevaluated keywords lost evaluation removes

* fix(mcp): emit stack-safe classes with non-ASCII members and protect draft-07 reference siblings

* fix(mcp): protect references into literal siblings of relaxed draft-07 references

* fix(mcp): keep the Evidence union on described borrowed inputs (#10597)

* fix(mcp): keep the Evidence union on described borrowed inputs

borrowEvidenceInput installed its advertised projection on one schema
instance. A call site's .describe() rebuilds the schema from its
definition, so match_native_functions advertised left and right as
unconstrained values in every input profile. Install the projection in
the borrowed parser's constructor so derived schemas keep it, for both
borrowed inputs and borrowed arrays.

* test(mcp): select the borrowed Evidence contract from its family

* docs(mcp): admit baseline growth only for restored contract declarations
2026-11-15 18:08:12 +08:00
morluto 26520bd290 refactor(providers): finalize apktool and shared artifact resources through runOwned (#10653)
* refactor(providers): finalize apktool and shared artifact resources through runOwned

runOwned moves to the provider layer so provider sites can use it without depending on application workflows. It gains a completed-value projection (partial) for values that are not themselves partial observations, an optional signal, and cleanupLeases that bind a close failure to the exact leases whose confirmed release resolves it.

ArtifactAnalysisResources.finish is removed; TShark and sigrok finalize through runOwned with the shared release as close. Apktool's hand-written release, cancellation and lease-group composition is replaced by runOwned. A failed primary now keeps a cancellation observed during final cleanup as a secondary failure, as the documented precedence requires.

A counted architecture guard refuses new hand-written completed-result abort reclassification outside runOwned.

* refactor(adb): finalize runtime-root work through runOwned (#10660)

* refactor(adb): finalize runtime-root work through runOwned

The ADB runtime-root finalizer re-derived the precedence by hand and kept its own cancellation-with-cleanup combiner. It now runs through runOwned: the work body maps snapshot failures, close releases the root lease and binds its own cleanup error, and partial projects the completed execution. A cancellation whose cleanup fails is now the shared cleanup composite with the cancellation as its primary, as runOwned produces for every other owner.

* refactor(providers): keep completed values as diagnostics in runOwned and migrate four finalizers (#10691)

* refactor(providers): keep completed values as diagnostics in runOwned and migrate signature, Ghidra, bundle and LLDB finalization

* test(providers): run the lifecycle scenarios over every runOwned adopter shape (#10707)

One table of testing.md lifecycle rows runs over partial, diagnostics, publishing and lease adopters, replacing the hand-written precedence cases it covers.

* chore(lint): move runOwned's classifier baseline entry with the file
2026-11-15 17:44:13 +08:00
morluto d0fee4e58e fix(mcp): re-emit advertised patterns in a portable regex grammar (#10582)
* test(mcp): check advertised schemas against a portable dialect in every profile

* fix(mcp): re-emit advertised patterns in a portable regex grammar

* fix(mcp): validate advertised dialect by selected profile

* fix(mcp): preserve schema widening through references

* fix(mcp): refuse zero-padded repeat bounds and record host-specific path patterns

* fix(mcp): keep references valid when presentation widens their targets

* fix(mcp): relax references into negations and conditionals presentation may remove

* fix(mcp): keep astral literals and dynamic references widening in every mode

* fix(mcp): omit astral literals and relax untracked references when presentation widens

* fix(mcp): read reference pointers by schema position, not segment spelling

* fix(mcp): protect references into collapsing boolean unions and keep referencing branches out of the collapse

* fix(mcp): omit patterns whose classes name astral characters

* fix(mcp): decide union collapse and pointer protection from canonical branches

* docs(mcp): document non-u astral matching and pin BMP agreement across V8 modes

* test(mcp): expect the advertised base64 schema to omit the unbounded pad-bit probe

* fix(mcp): protect references into unevaluated keywords lost evaluation removes

* fix(mcp): emit stack-safe classes with non-ASCII members and protect draft-07 reference siblings

* fix(mcp): protect references into literal siblings of relaxed draft-07 references
2026-11-15 17:06:43 +08:00
morluto ef1b92bf1e test(mcp): check advertised schemas against a portable dialect in every profile (#10567)
* test(mcp): check advertised schemas against a portable dialect in every profile

* fix(mcp): validate advertised dialect by selected profile

* fix(mcp): refuse zero-padded repeat bounds and record host-specific path patterns
2026-11-14 15:20:31 -05:00
morluto 85da0bd4ee refactor(errors): require a reason for every analysis cancellation (#10566)
* refactor(errors): require a reason for every analysis cancellation

Add cancelledBy, the one constructor for AnalysisCancelledError. Its
reason argument is required: an aborted signal, an earlier cancellation,
a reason carried from an inner failure, or an OwnerCancellation when REA
closed the work's owner itself.

Closed adb, apktool, firmware, sqlite, gdb, web-bundle analyzer and
binary-session record owners now report OwnerCancellation.ownerClosed
instead of an absent reason. Firmware's predecessor wait returns its
cancellation as a Result, removing an unreachable reason-less fallback.

An architecture guard rejects new direct constructions, with a counted
per-file baseline that can only shrink.

* refactor(errors): share one aborted-signal predicate with cancelledBy

* refactor(process): return a departing caller's reason from a shared startup wait

* refactor(providers): build decompiler-provider cancellations with a required reason (#10587)

Migrate every AnalysisCancelledError construction under src/ghidra,
src/ida, src/cutter, src/jeb and src/kuna to cancelledBy and lower the
guard baseline from 671 to 572. Closed Kuna target commands report
OwnerCancellation.ownerClosed instead of an absent reason, and Kuna and
JEB close abort their lifetimes with that typed reason.

* fix(errors): read the reason of any signal-shaped cancellation source

* test(ci): cover the counted-baseline comparison behind the architecture guard
2026-11-14 15:20:03 -05:00
gensokyo-devandgensokyo-dev 55c9e8e546 test: close the wait-deadline review gaps
- doctorCancellation measured from the signal, after the probe had
  started, so a probe stopped by its own deadline could still pass. The
  fake probe now records its start, and the check runs from there.
- rea/project-wait-deadline missed literal computed `timeout` keys and
  spread options. It now reports both, as well as computed keys it
  cannot read.
- The testing guide notes that a long-running test's expect.poll keeps
  its project budget, while waitFor follows the tag's deadline.

Co-Authored-By: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-11-14 19:28:38 +08:00
gensokyo-devandgensokyo-dev 07e4b700ac test: let the deadline policy own test waits
Vitest's expect.poll and vi.waitFor stop after one second by default, a
host-speed guess that the project's deadline policy does not see. Under
hosted-CI load, kunaProjectConcurrency failed that way twice on unrelated
PRs. Another 256 waits relied on the default, and 97 passed literals
picked to fit some host.

Waits now take as long as their test may run, less the reporting reserve
that children already keep. Each Vitest project sets expect.poll's
default, and the new waitFor in tests/support/testWait.ts replaces
vi.waitFor. Fixture readiness waits (awaitAcknowledgement and the provider
process waits) default to the same deadline, and their host-speed literal
arguments are removed. One remainingTestDeadline owns the arithmetic for
both children and waits.

Ten waits whose short timeout was the real check now assert it directly
(Kuna version probe, Cutter discovery, doctor and Inspector cancellation,
apktool fixture teardown, JADX disconnect): settlement before the
production deadline that would otherwise also satisfy the wait, or a
deadline pinned beyond every test deadline.

The rea/project-wait-deadline lint rule rejects vi.waitFor and explicit
expect.poll timeouts. The testing guide and the CI tests audit skill now
document the policy.

Co-Authored-By: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-11-14 19:28:38 +08:00
ee3e627efb docs(ci): describe CI lanes without counts that drift
The testing guides stated lane and shard counts in prose. The counts drifted
when the workflow changed: "all eight fast built shards" survived the move to
twelve, "Eight Linux real-system lanes" now covers twelve workflow calls, and
the SQLite guide named a Node 24.15.0 lane that the matrix no longer runs.
Describe each set by its selector and point at the workflow that owns the
exact versions, so later workflow changes cannot make the text wrong.

Fixes #7616
Fixes #7617

Co-Authored-By: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 17:11:50 +08:00
add0aa1293 ci: split fast built tests into twelve shards
Across 29 successful runs, the slowest of eight shards ran 24% above the
mean shard (797 s against 646 s), with no shard index consistently slow:
duration-balanced shards still vary with hosted runner speed. Resampling
the observed per-shard speed factors predicts the slowest of twelve
shards at about 9.2 minutes instead of 13.2, for roughly 5% more runner
time.

Co-Authored-By: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 15:56:55 +08:00
morluto b3657c557f perf(tests): run plain CLI commands on the file's shared process (#10412)
* refactor(cli): run one-shot commands through one shared owner

scripts/rea.mjs and the test CLI session worker each implemented the
one-shot command sequence: argument normalization, output admission,
configuration errors, version documents, the Amp scope refusal, and
result delivery. The worker copy had drifted: it never rendered
formatted version reports or the Amp refusal.

Move the sequence into src/cliOneShot.ts. Both callers supply a host
for their streams, exit and resource ownership. The launcher keeps
closed-pipe status, worker shutdown and process exit; the session
worker captures output per command.

The cliOutput bundle entry is superseded and removed. The dispatcher
test now replaces only the command runtime, so output admission runs
production code. A new parity test compares the file session with a
fresh process.

* test(cli): bundle the one-shot entry in the runtime bundle fixture

* perf(tests): run plain CLI commands on the file's shared process

`cli.run` in `cliTest` files started a fresh compiled CLI for every
command, about 0.45s and 260 MB each before any work. The file-scoped
session already existed, but callers had to opt in through
`runFileCli` or `sharedCliRun`, so 1,300 call sites still spawned.

Make the fixture choose: `cli.run` uses the shared process when it
reports the command exactly as a fresh process would, and starts one
otherwise (stdin, explicit deadlines, MCP and MCP doctor routing,
NODE_OPTIONS, or enabled logging). `freshProcess: true` keeps
explicit process-level checks. The session worker now presents the
launcher's entrypoint, so registration identity matches a fresh
process.

`runFileCli` and `sharedCliRun` are superseded and removed; their
call sites use `cli.run`, and paired fresh/shared checks use
`freshProcess`.

* style(tests): format the managed member summary call site

* test(cli): give each shared-session command an empty stdin

The shared CLI process reads session frames from its own stdin. A
command that reads stdin without supplied input, such as
`compare-wasm-interfaces - --json`, read that pipe and hung, where a
fresh process receives EOF. The worker now presents an ended, empty
stdin to each command and restores its own afterwards.

The parity test compared the routed path with itself after cli.run
began choosing the session. It now compares a fresh process with the
session directly, including the `mcp add` refusal and a command
reading absent stdin.

* test(cli): keep default command logging and launcher checks on fresh processes

environment-serve and environment-build log at info when REA_LOG_LEVEL is
unset, through a logger that writes to stderr directly. The shared CLI
process captures only process.stderr writes, so routed runs lost those
lines. Declare the default-logging commands beside the command names,
have infoLogger read that set, and start a fresh process for them.

The log-level precedence checks compared the shared process with itself
after routing; their fresh side now passes freshProcess: true.
2026-11-14 14:28:08 +08:00
ee1450b5d4 test: bound every test process registry by the running test
Registries created directly with createTestProcessRegistry did not get the
processTest default, so a hung child there had no deadline once its explicit
value was removed; Vitest's test timeout does not settle the awaited run, so
shutdown was never reached. Derive the default in the registry from the
running test instead, using the real clock.

Co-Authored-By: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 13:21:13 +08:00
077bebe49b test: bound fixture child processes by the remaining test deadline
A processTest child run without its own timeoutMs now gets the test's
remaining deadline less a reporting reserve (at least 5 seconds, or a
tenth of the deadline). A hung child still fails with its captured output,
without a fixed per-call value that assumes a host's speed.

Co-Authored-By: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 13:21:13 +08:00
morluto 243e4015cb docs(ci): remove dated shard measurements from the testing guides (#10414)
* docs(ci): move dated shard measurements into a verification record

The testing guide and testing-lanes guide mixed the October 2026 shard
budget and stale-history measurements into current rules. Move them to
docs/verification/ci-shard-scheduling-2026-10-20.md and link it from
the rules they justify.

* docs(ci): drop dated shard measurements instead of keeping a record
2026-11-14 11:47:46 +08:00
87743d67f5 test: check the deadlines Vitest applies to collected tests
Probes in a source and a built project read each collected test's
deadline, including test, suite, and module long-running tags, so a
change in project inheritance or tag resolution fails directly. Both
the policy and the probes read one set of base deadlines.

Co-Authored-By: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 05:27:07 +08:00
5bad48e122 test: let Vitest projects own test deadlines
Source projects allow 30 seconds and built projects 120 seconds, both
doubled under coverage, and hooks share their project's deadline instead
of Vitest's 10-second default. A long-running tag covers bounded
workloads that need more.

Co-Authored-By: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 05:27:07 +08:00
morluto cc7d73d23a test(ci): double test deadlines on hosted runners (#10417)
* test(javascript): give the wide fingerprint regression a capacity deadline

Each case runs about 22 s of CPU-bound projection on a typical hosted
runner. Its explicit 30 s deadline failed on slower runners at both two
and four Vitest workers, and it also overrode the suite's 60 s
coverage-mode allowance.

* test(ci): double test deadlines on hosted runners

Hosted CI jobs run four Vitest workers on a four-core runner. CPU-bound
capacity cases then share every core with other workers and their
subprocesses; three such cases exceeded the 30 s deadline at four
workers (16.8-22.7 s at two). The deadline only needs to bound a hang,
so hosted CI doubles it: 60 s, or 120 s with coverage. Local runs keep
30 s and 60 s.

* test(ci): check worker and deadline policy for local and hosted runners

The policy test loaded vitest.config.ts in a child that inherited the
parent's GITHUB_ACTIONS, so on CI it observed the hosted deadlines and
failed. Select the runner explicitly and check both: two workers and the
base deadline locally, four workers and twice the deadline on hosted CI.
2026-11-13 15:17:24 -05:00
morlutoandClaude Fable 5.2 3403338ea0 perf(ci): use four Vitest workers on hosted runners (#10399)
Hosted CI jobs own a four-core runner but the suite ran on two workers.
Other hosts keep the two-worker bound because they are shared.

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-13 13:51:39 -05:00
morluto 9ba61e8949 fix(ci): let CodeQL main baselines finish (#10146) 2026-11-12 06:48:50 +08:00
morluto ab84ba6839 fix(process)!: retain inspection cancellation and cleanup ownership (#10113)
* fix(process): preserve producer-owned inspection cancellation

Prototype the explicit inspection origin and retained file/batch owner protocol. Downstream adapter migration and lease binding remain required before integration.

* fix(process): type inspection ownership handoffs precisely

* fix(process): retain plain inspection preparation cancellation

* fix(process): retain explicit inspection cleanup and cancellation facts

* fix(process): require explicit inspection facts and cleanup scope

* refactor(process): remove redundant cleanup forwarding wrapper

* fix(firmware): retain inspection cancellation and cleanup ownership

* fix(wakaru): retain typed inspection outcomes and caller scope

* fix(ghidra): preserve inspection facts and launch cleanup scope

* fix(providers): preserve typed process inspection outcomes

* refactor(process): remove obsolete inspection cancellation re-export

* fix(native): preserve inspection preparation cleanup and cancellation

* fix(registration): retain typed inspection cleanup during Git discovery

* fix(process): preserve inspection deadline origin in adapter handoff

* fix(process): consume the canonical inspection selector in binding

* fix(native): preserve inspection deadlines and cleanup retry completion

* test(process): narrow the retained Darwin preparation owner

* fix(providers): preserve preparation ownership and timeout policy

Narrow preparation facts through their canonical owner, preserve actual cleanup capabilities and recorded cancellation, and keep enclosing command deadlines authoritative. Retain raw inspection records through Java refusal and configuration projections without changing availability fallback policy.

* refactor(process): use one inspection failure type owner

* fix(providers): retain recorded secondary failures when rebinding

* fix(ghidra): preserve normalized inspection facts through public projection

* fix(process): normalize native process listing cancellation

* docs(development): clarify failure provenance and cleanup verification

* fix(process): preserve direct inspection cancellation facts

* fix(process): retain inspection owners through termination

* fix(process): preserve command inspection cleanup provenance

* fix(desktop): preserve inspection facts after cleanup retry

* refactor(process)!: migrate desktop capture with inspection producers

* fix(desktop)!: preserve capture failure origins and cleanup owners (#10115)

* fix(desktop): preserve capture failure origins and cleanup

* fix(desktop): type keyring cleanup cancellation

* fix(kuna)!: retain copy cleanup before retiring workspaces (#10114)

* fix(kuna): own copy cancellation and descriptor cleanup

* fix(kuna): close copy descriptors before workspace removal

* test(kuna): type copy lifecycle gates for project library

* docs(kuna): explain retained copy-file cleanup and staging ownership

* docs(kuna): separate cache publication, integrity and cleanup guidance

* fix(desktop): retain sampling inspection owners before release

* fix(process): preserve capture file read cancellation origin

* fix(process): retain capture inspection cancellation and cleanup owners

* fix(process): retain sampling inspection failures in capture scope

* fix(process): retain inspection failures through mandatory cleanup

* fix(process): distinguish sampler shutdown from capture cancellation

* fix(browser): retain startup EOF before later cancellation

* fix(process): bind capture inspection cleanup to the owning run

* test(process): migrate cancellation fixtures to typed owners

Record explicit preparation and command cancellation facts, retain exact opaque reasons and physical causes, and guard the mocked version probe from native preparation. Keep the live Wakaru command control while adding a recording equivalent for bounded source verification.

* fix(process): own sampling reads and settle inspection peers

* fix(errors): retain authored recovery through cleanup composition

* fix(process): retain all sampling inspection leaves for final cleanup

* test(browser): report recording cleanup signal outcome

* fix(process): keep inspection error context out of implicit JSON

* fix(desktop): retain normalized sampling cleanup ownership

* test(process): assert tagged startup cancellation
2026-11-11 16:37:48 -05:00
morluto b1f473fcf8 docs: clarify publication recovery and Evidence ownership (#9654) 2026-11-09 08:39:20 -05:00
morluto 43d9c9d19b fix(uimage): reject impossible component extents before capacity (#9640)
* fix(artifacts): validate uImage extents before count limit

* docs(testing): distinguish malformed and capacity controls
2026-11-09 20:56:10 +08:00
a67525cdf1 docs: organize canonical documentation layout and verify new pages (#2791)
* docs: organize canonical documentation layout (URGENTE)

Group all existing authored guides by domain and ownership, expose complete navigation, and preserve their public routes. Update physical path consumers and release configuration while retaining historical records. Document domain-based specs and ADR slugs without global numbering.

Co-Authored-By: Codex <noreply@openai.com>

* fix(docs): honor home layout in all-page route verification

* style: format reviewed source changes

* fix(docs): preserve routes and navigation cache inputs

* fix: repair the static-check failures on the default branch (#2895)

* fix: repair the static-check failures on the default branch

The default branch fails its own required `Static checks` job three ways, so
every pull request reports that job as failed regardless of its own change.

- `npm run format:check` reports `src/domain/functionComparison.test.ts` and
  `src/ida/IdaOperationRunner.ts`.
- `npm run lint` reports a duplicate `import { join } from "node:path"` in
  `tests/boundary/mcp/ghidraInventoryMcp.test.ts` (lines 4 and 17).
- `npm run typecheck` reports `format: "devin"` missing from the client format
  union in `src/application/SupportedClients.ts`, and an unchecked index in
  `tests/boundary/cli/clientProfileLifecycle.test.ts`, whose pair array is
  inferred as `string[][]`.

Apply the pinned formatter, drop the duplicate import, list the `"devin"` format
that its own definition already uses, and mark the pair array `as const`.

Validation: `npm run check:fast` and `npm run format:check` pass.

* fix(setup): keep Devin on the JSON configuration dialect

---------

Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>

* fix(docs): update links after guide relocation

* docs: correct verified documentation route counts

* fix(docs): update browser capture guide link after move (#2820)

Refresh gettext protected-link comments alongside the authored href.

Co-authored-by: Codex <noreply@openai.com>

* fix(docs): include http3 lab in stable routes

* fix(docs): preserve historical release evidence and links

* fix(ghidra): restore Windows verifier identity import

* docs(wakaru): preserve pinned upstream policy link

* docs(index): retain recently added guide links

* docs(mcp): preserve parent retention clarification

* fix(cli): preserve standalone import guidance

* fix(test): preserve translated guide link drift coverage

---------

Co-authored-by: Matheus Breguêz <matbrgz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: morluto <17176810+morluto@users.noreply.github.com>
Co-authored-by: 吴佳浩Alben <35291372+XiaoBinGan@users.noreply.github.com>
Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
2026-11-08 01:42:28 +08:00