Commit Graph
17 Commits
Author SHA1 Message Date
morluto 5c53bcaed8 feat(mcp)!: retire the full schema profiles and declare schema facts as typed annotations (#10693)
* test(mcp): check advertised schemas against a portable dialect in every profile

* fix(mcp): re-emit advertised patterns in a portable regex grammar

* fix(mcp): present discover_tools input through the selected profile

The hot-cold catalog registered discover_tools directly with the SDK, so
the default compact input profile still advertised its canonical schema,
including page-size defaults clients apply on their own (#5322). Route
every listed input schema, transport discovery included, through one
server-owned presentation, and pass the selected presentation to the
deferred server, which serves recorded tool schemas but not discovery.

* fix(mcp): keep the Evidence union on described borrowed inputs

borrowEvidenceInput installed its advertised projection on one schema
instance. A call site's .describe() rebuilds the schema from its
definition, so match_native_functions advertised left and right as
unconstrained values in every input profile. Install the projection in
the borrowed parser's constructor so derived schemas keep it, for both
borrowed inputs and borrowed arrays.

* fix(mcp): validate advertised dialect by selected profile

* fix(mcp): preserve schema widening through references

* feat(mcp): project compact input schemas into the portable advertised dialect

One projection (advertisedDialectProjection.ts) owns the compact input
advertisement: one type per node, items on every array, string literals
as enums, oneOf widened to anyOf, allOf merged, tuples as element unions,
exclusive bounds made inclusive, and every keyword outside the dialect
dropped. Each rewrite is exact or widening; server validation is
unchanged. The root description and sentences for dropped cross-field
constraints move to the tool description, which Codex and Zed keep.

The compact baseline falls to nullable values only (29), which the
dialect cannot express without narrowing an input. The gateway shape
repairs (typeArrayItemUnions, hasGatewayIncompatibleShape), the second
dependentRequired-to-prose copy and the strict-gateway test denylist are
removed; the dialect checker gates the inline presentation instead.

* feat(mcp): omit output schemas from tools/list by default

No surveyed model API reads an output schema, MCP client validators
repeat REA's own result validation, and complete output schemas were
about 92% of the default tools/list. REA_MCP_OUTPUT_SCHEMA_PROFILE now
resolves to off through one owner (resolveMcpOutputSchemaProfile);
compact and full remain selectable. structuredContent is still
delivered, and the deferred server serves both off and full from the
recorded catalog with the matching recorded identity.

Tests and artifact verifiers that validate results against complete
advertised output contracts select full explicitly.

* docs: describe omitted output schemas as the default in README translations

* fix(mcp): select schemas in verification workflows

* fix(mcp): refuse zero-padded repeat bounds and record host-specific path patterns

* fix(mcp): keep projected schemas permissive

* fix(mcp): keep references valid when presentation widens their targets

* fix(mcp): keep a single union branch's closure scoped to its properties

* fix(mcp): select complete output schemas in the Electron verifier

* fix(mcp): select complete output schemas in standalone verifier probes

* fix(mcp): keep an allOf member's tuple tail with its own positions

* fix(mcp): relax references into negations and conditionals presentation may remove

* refactor(contracts): declare root discriminators as typed schema annotations

* test(mcp): select the borrowed Evidence contract from its family

* fix(mcp): keep astral literals and dynamic references widening in every mode

* fix(mcp): omit astral literals and relax untracked references when presentation widens

* fix(mcp): select complete output schemas in every verifier transport

* fix(mcp): read reference pointers by schema position, not segment spelling

* fix(mcp): protect references into collapsing boolean unions and keep referencing branches out of the collapse

* fix(mcp): keep reduced schemas open when their root admits names by pattern

* test(mcp): keep the reduced-schema pattern case outside the size-limited block

* fix(mcp): omit patterns whose classes name astral characters

* fix(mcp): decide union collapse and pointer protection from canonical branches

* docs(mcp): document non-u astral matching and pin BMP agreement across V8 modes

* fix(mcp): keep prototype-named properties as own properties in the dialect projection

* test(mcp): select full output schemas in packet and signal schema checks

* test(mcp): expect the advertised base64 schema to omit the unbounded pad-bit probe

* fix(mcp): protect references into unevaluated keywords lost evaluation removes

* fix(mcp): validate the doctor probe reply against the canonical output contract (#10655)

* fix(mcp): validate the doctor probe reply against the canonical output contract

* test(mcp): replay doctor probe validation under every output profile

* test(mcp): accept typed-branch nullable unions in the advertised dialect (#10649)

* test(mcp): select full output schemas in HTTP session workflows

* fix(mcp): emit stack-safe classes with non-ASCII members and protect draft-07 reference siblings

* fix(mcp): protect references into literal siblings of relaxed draft-07 references

* feat(mcp): omit output schemas from tools/list by default (#10646)

* feat(mcp): omit output schemas from tools/list by default

No surveyed model API reads an output schema, MCP client validators
repeat REA's own result validation, and complete output schemas were
about 92% of the default tools/list. REA_MCP_OUTPUT_SCHEMA_PROFILE now
resolves to off through one owner (resolveMcpOutputSchemaProfile);
compact and full remain selectable. structuredContent is still
delivered, and the deferred server serves both off and full from the
recorded catalog with the matching recorded identity.

Tests and artifact verifiers that validate results against complete
advertised output contracts select full explicitly.

* fix(mcp): select schemas in verification workflows

* fix(mcp): select complete output schemas in the Electron verifier

* fix(mcp): select complete output schemas in standalone verifier probes

* fix(mcp): select complete output schemas in every verifier transport

* test(mcp): select full output schemas in packet and signal schema checks

* fix(mcp): validate the doctor probe reply against the canonical output contract (#10655)

* fix(mcp): validate the doctor probe reply against the canonical output contract

* test(mcp): replay doctor probe validation under every output profile

* test(mcp): accept typed-branch nullable unions in the advertised dialect (#10649)

* test(mcp): select full output schemas in HTTP session workflows

* fix(mcp): keep the Evidence union on described borrowed inputs (#10597)

* fix(mcp): keep the Evidence union on described borrowed inputs

borrowEvidenceInput installed its advertised projection on one schema
instance. A call site's .describe() rebuilds the schema from its
definition, so match_native_functions advertised left and right as
unconstrained values in every input profile. Install the projection in
the borrowed parser's constructor so derived schemas keep it, for both
borrowed inputs and borrowed arrays.

* test(mcp): select the borrowed Evidence contract from its family

* docs(mcp): admit baseline growth only for restored contract declarations

* fix(mcp): state reduced multi-type properties as typed branches, not type lists

The reduced fallback derived a property's types and emitted them as a type
list, which Gemini, Moonshot and Ark reject. A property whose canonical
schema is anyOf string and number therefore traded a typed union for a
rejected construct when the inline projection fell back. Emit one type, or
anyOf typed branches, so [T, null] becomes the dialect's nullable union.

* feat(mcp)!: retire the full schema profiles and declare complete observations as a typed annotation (#10718)

* refactor(mcp): declare complete observations as a typed annotation applied only on the wire

* test(mcp): compare complete-observation declarations with their owning text

* refactor(schema): couple Zod checks with their JSON Schema constraint keywords (#10725)

* refactor(schema): couple Zod checks with their JSON Schema constraint keywords

* refactor(schema): declare pattern approximations and object output roots as typed annotations (#10754)

* refactor(schema): declare pattern approximations and object output roots as typed annotations

* fix(schema): refuse pattern declarations JSON Schema cannot state

* test(schema): select guided_workflows from its contract family

* feat(mcp)!: retire the full input and output schema profiles (#10841)

* feat(mcp)!: retire the full input and output schema profiles

The compact input presentation is now the only advertised input schema, and
output schemas are either omitted (`off`, the default) or bounded
(`compact`). Catalog identity, the build-recorded digests and the recorded
startup catalog collapse to those presentations: two digest profiles instead
of six, one recorded catalog instead of two, and no input dialect baseline.

Tests and verifiers that validated replies against listed full output
schemas now validate against the canonical output contract, which REA
already enforces server-side; tests that asserted canonical input
restrictions read the canonical input schema instead of a full listing.

BREAKING CHANGE: `REA_MCP_INPUT_SCHEMA_PROFILE=full` and
`REA_MCP_OUTPUT_SCHEMA_PROFILE=full` are refused at startup and by `rea setup`
with a configuration error naming the setting. Remove the setting from the
client registration (or rerun setup with `REA_MCP_INPUT_SCHEMA_PROFILE=compact`
/ `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact`). `REA_MCP_INPUT_SCHEMA_PROFILE`
still accepts `compact`. Server-side validation is unchanged.

* test(mcp): validate replies against canonical output contracts at every listed-schema reader

tools/list omits output schemas by default, so fixtures, verifiers and the
Android analysis runners that read a listed outputSchema now bind the
canonical contract instead. connectLocalToolsMcp and advertisedMcpToolSchemas
hand the SDK a definition carrying the canonical output schema, so the SDK
client and caller-supplied validators check every successful reply.

* test(mcp): resolve Android runners' output contracts from the selected repository

The Android efficiency runners imported the canonical output-schema helper from the harness checkout while --repository selects the revision under test. Import it from the selected repository, as the runners already do for the SDK and Ajv.

* test(mcp): let Android runners compare revisions before the canonical output helper

A selected revision without scripts/lib/canonical-output-schema.mjs could not start the Android runners. Choose the output contracts in one harness helper: the selected revision's canonical contracts when it has them, otherwise its listed schemas under the full output profile it still supports.

* fix(scripts): keep caller launch settings when the real Hopper verifier builds its server environment

The verifier deleted REA_VERIFY_SERVER_COMMAND and REA_VERIFY_SERVER_ARGS_JSON from process.env itself before reading them, so a caller selecting an installed REA command silently ran the checkout's dist/main.js. Copy the environment, as main does, so only the child's environment changes.

* test(mcp): validate whole replies against canonical output contracts wherever a listed definition is held

tools/list omits output schemas by default and the full output profile is gone, so tests that handed the SDK a listed definition no longer checked complete replies. Attach the canonical output schema with withCanonicalOutputSchema at every held REA tool definition. Environment-served tools (not REA contracts), and the deferred-runtime and doctor tests that exercise per-profile listings, keep their listed definitions.

* test(mcp): record the advertised catalog summary without the retired full profiles

Retiring the full input and output profiles leaves only the compact input and compact output listings, so the summary drops the full-profile digests and every pattern only the full listings carried. Regenerated with npm run verify:advertised-schemas -- --update-summary.
2026-11-15 21:51:18 +08:00
morluto d0fee4e58e fix(mcp): re-emit advertised patterns in a portable regex grammar (#10582)
* test(mcp): check advertised schemas against a portable dialect in every profile

* fix(mcp): re-emit advertised patterns in a portable regex grammar

* fix(mcp): validate advertised dialect by selected profile

* fix(mcp): preserve schema widening through references

* fix(mcp): refuse zero-padded repeat bounds and record host-specific path patterns

* fix(mcp): keep references valid when presentation widens their targets

* fix(mcp): relax references into negations and conditionals presentation may remove

* fix(mcp): keep astral literals and dynamic references widening in every mode

* fix(mcp): omit astral literals and relax untracked references when presentation widens

* fix(mcp): read reference pointers by schema position, not segment spelling

* fix(mcp): protect references into collapsing boolean unions and keep referencing branches out of the collapse

* fix(mcp): omit patterns whose classes name astral characters

* fix(mcp): decide union collapse and pointer protection from canonical branches

* docs(mcp): document non-u astral matching and pin BMP agreement across V8 modes

* test(mcp): expect the advertised base64 schema to omit the unbounded pad-bit probe

* fix(mcp): protect references into unevaluated keywords lost evaluation removes

* fix(mcp): emit stack-safe classes with non-ASCII members and protect draft-07 reference siblings

* fix(mcp): protect references into literal siblings of relaxed draft-07 references
2026-11-15 17:06:43 +08:00
morluto ef1b92bf1e test(mcp): check advertised schemas against a portable dialect in every profile (#10567)
* test(mcp): check advertised schemas against a portable dialect in every profile

* fix(mcp): validate advertised dialect by selected profile

* fix(mcp): refuse zero-padded repeat bounds and record host-specific path patterns
2026-11-14 15:20:31 -05:00
gensokyo-dev 6eb37a7749 fix(ci): restore qualification and reduce validation overhead
Use one typed-checker worker by default without changing semantic scope or caller overrides. Reuse the bounded CLI worker for pure argument validation while retaining fresh-process coverage and all upstream cases.

Keep registration tests independent of real package-runner downloads while asserting the actual prewarm command and startup context. Pass the advertised binary_session definition by name to package verification, retaining default SDK validation and complete catalog/contract checks while avoiding compilation of unrelated tool outputs. Preserve all four installed-tarball scenarios and existing deadlines.

Strengthen Hermes preview operation/backup assertions and clarify physical-path routing documentation.

Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)
2026-10-28 00:20:07 +08:00
morluto 2411d60b5a fix(verification): unify complete MCP catalog checks 2026-10-26 16:11:09 +08:00
morluto 48ff37f6ea fix(verifiers): consolidate shared helper ownership
Use JSON.parse directly in package verification and retain one parsed Cursor recovery snapshot. Route native command source loading through the shared transform-service owner so completed command cleanup leaves no transform child.

Preserve JSONC validation and caller inputs. Cover the actual public source-loading lifecycle with an authored command and verified process lineage.
2026-10-24 22:28:39 +08:00
morluto 2250e67bc2 fix(verifiers): consolidate native client command lifetimes
Migrate the remaining native verifier commands to shared ownership with explicit stdin contracts, command-before-fixture cleanup, and failure receipts. Preserve observed catalog counts and unknowns, retained artifacts, native assertions, and selected launch inputs.

Remove the obsolete raw exec export and share output persistence across native verifiers. Exercise actual entrypoints with authored failed preflights and acknowledged descendants, plus filesystem failures and opaque error preservation.
2026-10-24 19:40:50 +08:00
morluto d9660eb032 fix(verifiers): share package command and workspace ownership
Join canonical owned command cleanup before releasing package verifier workspaces. Replace raw package command wrappers, the updater supervisor, WASM cleanup composite, and boolean-only retention with explicit capability ownership.

Preserve selected environments, npm arguments, no-deadline calls, clean exit status, and distinct aggregate/per-stream output budgets. Exercise actual timeout descendants, cancellation, retained cleanup retry, and workspace admission.
2026-10-24 16:58:18 +08:00
Ysopwnable 6ae59f3d19 fix(verifiers): launch npm through Node on Windows (#4153)
* fix(verifiers): launch npm through Node on Windows

* test(verifiers): cover Windows npm fallback
2026-10-15 23:12:29 +08:00
astralyx0129andClaude Opus 4.7 69e097a8ca fix(scripts, install): honor test:fast "without build" promise and orient Node.js engine mismatches (#2026)
* fix(scripts): defer dist/ imports in verifier helpers so test:fast runs without build

Two helpers under scripts/lib/ reached into ../../dist/ at module load even
though the top-level import was only needed by one of their exports. That
turned mocked, dist-free tests into failing tests under `npm run test:fast`,
which docs/testing.md § test scripts explicitly promises runs "without build":

- tests/conformance/providers/hopper/realHopperSemantic.test.ts only calls
  requireCurrentDocument and resolveFixtureProcedure (both mock-only), but
  scripts/lib/real-hopper-semantic.mjs eagerly imported
  requireFunctionDossierOracle from dist/application/RealHopperAssertions.js.
  Moved the dist-dependent dossier oracle into a lazy loader called from
  verifyRealHopperFixture, which is where it is actually used (against the
  real Hopper bridge, after build:cached).

- tests/evaluation/package/packagePlatformDiagnostics.test.ts only uses
  assertLinuxPackageProviderFailure, which depends on `json`, but
  scripts/lib/verify-package-core.mjs eagerly re-exported helpers from
  mcp-verifier-results.mjs, which in turn pulls four dist/ modules. Moved
  that import into verifyCompleteToolCatalog (the only consumer in this
  file). Downstream scripts (verify-recorded-crash.mjs, verify-package-mcp.mjs,
  verify-windows-package.mjs) are unaffected because they already run after
  build:cached.

Reproduction before fix:
  rm -rf dist && npm run test:fast → 2 failed (both import-time errors)

Reproduction after fix:
  rm -rf dist && npm run test:fast → 388 passed, 4924 tests, 0 failed
  npm run build:cached && npm run test:fast → 388 passed, 4924 tests, 0 failed

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(install): orient Node.js version mismatch failures with actionable remediation

install.sh previously told operators "use Node.js 22.x (>=22.19), 24.x (>=24.11),
or 26+" without saying how to install one. The curl|bash onboarding path then
left the operator to guess between nvm, fnm, n, Volta, system packages, or
nodejs.org — a lookup that bounces across search engines for a first-time user.

Replace the one-line failure with a block that:
- Names the detected Node.js version (so the operator confirms it is theirs).
- Shows the three most common install paths (nvm, fnm, direct tarball).
- Pins the Node.js release line (24) that satisfies the engine, so copy-paste
  commands produce a supported runtime instead of whichever "latest" is on top.
- Maps uname -m / uname -s to the Node.js tarball label pair for this host
  (e.g. darwin/arm64, linux/x64), so the operator can grab the right archive
  from https://nodejs.org/dist/latest-v24.x/ without a second lookup.

Verified with four Node.js versions (24.18 compatible, 25.9/20.11 outside the
supported ranges, 22.18 below the minor floor) that remediation triggers only
on real mismatches and names the host labels correctly. shellcheck install.sh
and npm run check:fast remain clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-10-12 13:54:57 +08:00
YsopwnableandKaoru0822 6395e8d300 perf(mcp)!: compact repeated session discovery facts (#1988)
Group repeated session tool availability and provider capability facts, preserve complete named contracts and ordered limitations, and remove duplicate catalog descriptors. Update consumers and migration guidance.

Validated by all 31 CI jobs, focused source checks, and actual Claude Code/Linux Hopper/native Windows Ghidra small-target workflows on the current integration tree.

Refs #723.

BREAKING CHANGE: binary_session no longer includes server_identity.catalog.tools; tool_availability and provider capabilities use grouped facts documented in docs/mcp-contracts.md.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-12 07:40:19 +08:00
morluto ca9010936a test(package): share validated native dossier fixtures 2026-10-10 04:02:40 +08:00
N0zoM1z0 4007b20fc2 fix(mcp): emit valid schemas for empty arrays (#535)
Replace empty tuples with a shared empty-array schema. Validate all input
and output schemas against Draft 2020-12 during catalog generation and
packaged MCP verification, with runtime and catalog regression coverage.

Closes #528

Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)
2026-10-05 15:32:58 +08:00
morluto 0d96275b68 refactor: return complete tool results inline 2026-09-29 07:08:46 +08:00
morluto b8a9da0c0f refactor(mcp): make analysis states unrepresentable (#468)
* refactor(config): parse capability settings into exact states

* refactor(runtime): encode replay and process outcomes exactly

* refactor: make analysis states unrepresentable

* refactor(mcp): register canonical schemas with SDK

* fix: preserve capability and catalog invariants

* fix(build): remove stale compiled modules

* fix(replay): update real verifier policy dependency

* fix(package): verify the complete MCP catalog

* fix(test): supply complete replay reload policy

* refactor(contracts): centralize output schema lookup

* refactor(mcp): delegate protocol boundaries to SDK

* fix(browser): approve real source capture

* fix(mcp): preserve catalog guidance
2026-08-10 02:28:11 +08:00
morluto 1c9e50814e fix(ci): repair cross-platform package verification 2026-07-24 15:22:24 +00:00
morluto 1a99a60a29 refactor(adapters): split oversized provider workflows 2026-07-18 05:49:14 +00:00