diff --git a/README.md b/README.md index 056af5e877..013e80836a 100644 --- a/README.md +++ b/README.md @@ -102,11 +102,12 @@ complete advertised field guidance and nested validation details. Both profiles keep the full tool set and canonical server validation. See "Compact input schema profile" in the MCP contracts guide. -Complete output schemas are advertised by default. If your client caps the -complete `tools/list` frame, set `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact` to -bound each advertised output schema, or `off` to omit output schemas entirely. -Both settings keep canonical server-side validation and complete structured -results. See "Output schemas" in the MCP contracts guide. +Output schemas are omitted from `tools/list` by default: no model API reads +them, and REA validates every result itself. Set +`REA_MCP_OUTPUT_SCHEMA_PROFILE=compact` to advertise each output schema within +a 4,096-byte budget, or `full` for complete output schemas (for MCP clients +that validate results). All settings keep canonical server-side validation and +complete structured results. See "Output schemas" in the MCP contracts guide. ### Ask your agent diff --git a/docs/development/client-verification.md b/docs/development/client-verification.md index 23d9780c2f..da7a232ccb 100644 --- a/docs/development/client-verification.md +++ b/docs/development/client-verification.md @@ -15,7 +15,9 @@ not establish live model-service acceptance. `rea mcp doctor --json` starts the production MCP adapter with the current runtime executable and environment. A healthy result establishes that adapter's -initialization, discovery and target-free request flow; it does not verify a +initialization, discovery and target-free request flow, with the request-flow +reply validated against the canonical `binary_session` output contract whatever +output schema profile the server lists. It does not verify a client's configured launcher or its inherited environment. Test the generated registration through the actual client, including a reduced `PATH` when checking desktop or service startup. Record the client version and launch environment diff --git a/docs/esp-image.md b/docs/esp-image.md index 71cfc57e47..8816ea37ba 100644 --- a/docs/esp-image.md +++ b/docs/esp-image.md @@ -18,10 +18,9 @@ summary's `root_format_metadata` holds the image facts. Use its `extract_artifact` writes one file per segment and per signature block. For agent sessions, advertise the `session,artifact,application` tool families -and consider setting both `REA_MCP_INPUT_SCHEMA_PROFILE` and -`REA_MCP_OUTPUT_SCHEMA_PROFILE` to `compact`. The output profile bounds each -advertised output schema to 4,096 bytes by relaxing nested client validation; -REA still validates complete results against the canonical contracts. Pair +and keep the default `compact` input profile and `off` output profile, which +omits output schemas; REA still validates complete results against the +canonical contracts. Pair this with retained summaries and selected inventory views to limit the context needed for one question. See [MCP contracts](reference/mcp-contracts.md) for the profiles and their client compatibility tradeoffs. diff --git a/docs/getting-started/client-configuration.md b/docs/getting-started/client-configuration.md index 8c0c272a26..3fc0a49a4a 100644 --- a/docs/getting-started/client-configuration.md +++ b/docs/getting-started/client-configuration.md @@ -186,7 +186,8 @@ REA_MCP_TOOL_CATALOG_PROFILE=full This advertises all tools without requiring catalog refresh. Compact output schemas widen client-side validation; REA still validates complete results -against the canonical contracts. See [output schemas](../reference/mcp-contracts.md#output-schemas). +against the canonical contracts. The default output profile, `off`, omits +output schemas and is smaller still. See [output schemas](../reference/mcp-contracts.md#output-schemas). On main `3e0599a94`, a real FastMCP 4.1.6 / Python MCP 2.3.0 HTTP client with the default 1 MiB SSE event limit rejected the 4,108,975-byte full catalog. @@ -370,11 +371,12 @@ and Claude Code reports that autocompact is thrashing. To avoid this: The sizes above come from REA's own `tools/list`. A live Claude Code session against DeepSeek's API is unverified. -For a client whose stdio reader rejects the complete catalog frame, set -`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "compact"` to bound every advertised output -schema, or `"off"` to omit them, in REA's `env` object, then restart the client. -Both keep canonical result validation and complete structured results; `compact` -keeps a widened root shape and `off` advertises nothing. On catalogs large +Output schemas are omitted by default (`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "off"`). +If you selected `"full"` and a client's stdio reader rejects the complete +catalog frame, return to the default or select `"compact"`, which bounds every +advertised output schema, in REA's `env` object, then restart the client. +Every profile keeps canonical result validation and complete structured +results; `compact` keeps a widened root shape and `off` advertises nothing. On catalogs large enough that full input schemas alone approach the frame limit, also keep the input profile at its `compact` default. See [MCP output schema profiles](../reference/mcp-contracts.md#output-schemas). diff --git a/docs/reference/mcp-contracts.md b/docs/reference/mcp-contracts.md index 0dbe1a4823..158df9d18b 100644 --- a/docs/reference/mcp-contracts.md +++ b/docs/reference/mcp-contracts.md @@ -2110,63 +2110,64 @@ Kimi-K3 backend, not the WorkBuddy MCP client. Set the profile on REA's registration in the actual MCP host; the host's name alone does not identify the downstream model or its schema constraints. -The compact profile also protects strict Anthropic-compatible gateways from -known schema-translation failures reported in [#3800](https://github.com/morluto/rea/issues/3800): -primitive `enum` properties without a sibling `type`, and `oneOf`/`allOf` -inside a property listed in `required`. Affected schemas use the same reduced -first-level projection described below, so the gateway receives a plain object -shape while REA continues to validate the complete canonical contract locally. -This is a compatibility projection, not a claim that the gateway reproduces -the full union semantics. +The compact profile also projects every advertised input schema into the +portable advertised dialect declared in `src/contracts/advertisedDialect.ts`: +the subset of JSON Schema that the clients and gateways REA has met accept, +with one rule per observed rejection (for example strict gateways in +[#3800](https://github.com/morluto/rea/issues/3800), Gemini item and literal +rules in [#5201](https://github.com/morluto/rea/issues/5201) and +[#4025](https://github.com/morluto/rea/issues/4025), and VS Code's required +`items`). One projection (`src/contracts/advertisedDialectProjection.ts`) +owns it, and every rewrite is exact or widening, so a canonical-valid call is +never refused by the advertisement: -For array items expressed as an untyped `oneOf`/`anyOf`, the compact profile -adds a type when every branch has the same type. Mixed-type item unions use the -reduced projection; selecting one branch's type would exclude valid calls. -This addresses the emitted shape reported through a Gemini-backed gateway in -[#5201](https://github.com/morluto/rea/issues/5201). Gateway conversion and -live provider acceptance have not been verified for this projection. +- Every node carries one type, every array has `items`, and the root is a + plain object. A string `const` becomes a one-value `enum`; unions of string + literals become one `enum`; other literals keep only their type. +- `oneOf` becomes `anyOf`, `allOf` members merge into their parent, tuples + advertise the union of their positions with their length bounds, and + exclusive numeric bounds become inclusive (exact for integers). +- `$schema`, `format`, `default`, `examples`, `title`, `propertyNames`, + `additionalProperties` schemas, property-count bounds, conditionals and + length bounds of 1,999 or more are dropped. +- A nullable value is advertised in the dialect's one nullable form: `anyOf` + a typed branch and `{"type": "null"}`, below the root. Type lists such as + `["integer", "null"]` are outside the dialect (Gemini's legacy schema + rejects them, and clients rewrite them differently), and advertising only + the non-null type would refuse a valid call such as `limit: null`. -Gemini function declarations accept `items` only beside the single `array` -type. Reduced array properties therefore advertise the element types their -canonical item schemas accept, such as `{"type": "object"}` for a union of -object actions, instead of an any-value type list with nested `items`. An -element whose type is unknown, a tuple, or an array alongside other types uses -the empty item schema, and a property that may be an array or another type -lists typed `anyOf` branches. A boundary test rejects any compact -advertisement that places `items` beside another type or uses an untyped item -union. +Guidance never stays inside the compact schema. The root description (input +groups, Evidence-source guidance, the reduced-form notice) and sentences for +dropped cross-field constraints (`dependentRequired`, `uniqueItems`, excluded +values) follow the tool description. Codex strips every description inside a +schema above 5,000 bytes and Zed drops the root description; neither strips +the tool description. The compact profile renders the same tool set, output schemas, annotations, and canonical server-side validation. It changes only the advertised input -JSON Schema: +JSON Schema and appends input guidance to the tool description: - Annotation prose leaves the advertised form: nested property descriptions, - literal examples, defaults, and titles are dropped. The root keeps its - description because it states the accepted input groups. A root `detail` + literal examples, defaults, and titles are dropped. A root `detail` property keeps its description because it selects between complete Evidence and a bounded summary. - Schema-local references are inlined within the budget. Kimi Code expands references before sending provider requests, so sharing definitions cannot establish the provider-facing size. Expansion stops at the budget before - allocating an oversized presentation. Unconstrained JSON fields advertise - all JSON value types explicitly so Kimi cannot infer string for Evidence - objects or arrays. -- The few schemas whose validation structure alone exceeds the budget - (`compare_web_captures`, `build_reconstruction_obligation_ledger`, - `evaluate_reconstruction_coverage`, - `compare_application_versions`, `capture_browser_scenario`) are advertised - in a reduced form: property names, reference-resolved types (including union - alternatives), first-level guidance, and the root's required-field constraints, - with nested validation detail elided. The - registered canonical schema still rejects malformed calls before invoking a - handler. That rejection can use the MCP SDK's input-validation diagnostic; - it does not guarantee the application's typed input error. -- Schemas deeper than ten nodes use the same reduced projection, with - root-property names, types, guidance, and required fields retained. The - server continues to enforce the full canonical input contract. + allocating an oversized presentation. +- Schemas whose projected validation structure exceeds the budget or ten + levels, or that need a construct the dialect cannot express (an any-value + field), are advertised in a reduced form: property names, + reference-resolved types (including union alternatives), first-level + guidance, and the root's required-field constraints, with nested validation + detail elided. The registered canonical schema still rejects malformed calls + before invoking a handler. That rejection can use the MCP SDK's + input-validation diagnostic; it does not guarantee the application's typed + input error. -Boundary tests pin every advertised compact schema to the budget, keep them -valid JSON Schema that still accepts every canonical example. Client verification +Boundary tests pin every advertised compact schema to the budget and the +dialect checker, keep them valid JSON Schema that still accepts every +canonical example. Client verification also checks provider-facing sizes and canonical examples after conversion by the actual Kimi Code CLI; local captures do not establish Moonshot acceptance. The SDK catalog test checks that full-profile schemas retain nested ADB extra, @@ -2311,14 +2312,19 @@ profile and the advertised form that profile produces. `full`, `compact`, and bounded schemas clients receive; `off` keeps the canonical output contracts in the digest while advertising nothing. -Some clients reject a complete `tools/list` frame or HTTP SSE event before -they load any tools. The opt-in `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact` -setting bounds every advertised output schema to 4,096 bytes; `off` omits `outputSchema` entirely -(the measured pi-web-ui failure and catalog breakdown are in -[#2512](https://github.com/morluto/rea/issues/2512)). The default `full` profile -is unchanged. Both settings keep the same named canonical tool contract for -server-side result validation, and both still return complete -`structuredContent` and text results. +`REA_MCP_OUTPUT_SCHEMA_PROFILE` selects what `tools/list` carries for each +tool's `outputSchema`. The default, `off`, omits it: no model API surveyed +reads an output schema, MCP client validators only repeat REA's own result +validation, and complete output schemas were about 92% of a default +`tools/list` (4.3 of 4.6 MB). Some clients also reject a large `tools/list` +frame or HTTP SSE event before they load any tools (the measured pi-web-ui +failure is in [#2512](https://github.com/morluto/rea/issues/2512)). Select +`compact` to bound every advertised output schema to 4,096 bytes, or `full` +for complete output schemas when an MCP client should validate results. Every +profile keeps the same named canonical tool contract for server-side result +validation and still returns complete `structuredContent` and text results; +the MCP specification does not require an output schema for +`structuredContent`. For the 1 MiB FastMCP SSE limit reported by Unsloth in [#9093](https://github.com/morluto/rea/issues/9093), see the diff --git a/scripts/lib/artifact-e2e.mjs b/scripts/lib/artifact-e2e.mjs index 59cb2a674d..e476760fcf 100644 --- a/scripts/lib/artifact-e2e.mjs +++ b/scripts/lib/artifact-e2e.mjs @@ -4,7 +4,10 @@ import { Client } from "@modelcontextprotocol/client"; import { createVerifierCommands } from "./verify-commands.mjs"; import { createOwnedMcpStdioTransport } from "./owned-mcp-stdio.mjs"; import { withVerifierCleanup } from "./verifier-cleanup.mjs"; -import { INLINE_EVIDENCE_MCP_ENVIRONMENT } from "./mcp-verification-environment.mjs"; +import { + INLINE_EVIDENCE_MCP_ENVIRONMENT, + schemaVerifierEnvironment, +} from "./mcp-verification-environment.mjs"; // Bind parsed definitions to this connection. Hot/cold catalogs may add tools; // refresh a missing name once without throwing away already validated bindings. @@ -95,7 +98,7 @@ const environment = () => { const compileCache = process.env.NODE_COMPILE_CACHE ?? process.env.REA_ARTIFACT_NODE_COMPILE_CACHE; - return { + return schemaVerifierEnvironment({ PATH: process.env.PATH ?? "/usr/bin:/bin", ...(compileCache && process.env.NODE_V8_COVERAGE === undefined && @@ -106,7 +109,7 @@ const environment = () => { REA_ANALYSIS_PROVIDER: "auto", // Artifact workflows have no deep-provider prerequisite or GUI authority. HOPPER_LAUNCHER_PATH: "/rea-unconfigured-deep-provider/hopper", - }; + }); }; /** Execute the compiled CLI with production providers and no test adapters. */ diff --git a/scripts/lib/browser-capture-metadata-budget-e2e.mjs b/scripts/lib/browser-capture-metadata-budget-e2e.mjs index 5e44d6f8ee..fbf5fb91e2 100644 --- a/scripts/lib/browser-capture-metadata-budget-e2e.mjs +++ b/scripts/lib/browser-capture-metadata-budget-e2e.mjs @@ -4,6 +4,7 @@ import { Client } from "@modelcontextprotocol/client"; import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; import { artifactCliEvidence, artifactMcpResult } from "./artifact-e2e.mjs"; import { webSourceMapsSchema } from "../../dist/domain/webBundleAnalysis.js"; +import { schemaVerifierEnvironment } from "./mcp-verification-environment.mjs"; const ANNOTATION_BYTES = 2 * 1024 * 1024; const SCRIPT_COUNT = 5; @@ -106,7 +107,10 @@ export async function verifyBrowserCaptureMetadataBudget(endpoint) { const transport = new StdioClientTransport({ command: process.execPath, args: [fileURLToPath(new URL("../rea.mjs", import.meta.url)), "mcp"], - env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent" }, + env: schemaVerifierEnvironment({ + PATH: process.env.PATH ?? "", + REA_LOG_LEVEL: "silent", + }), stderr: "pipe", }); const client = new Client({ name: "browser-metadata-e2e", version: "1" }); diff --git a/scripts/lib/browser-scenario-storage-fingerprints-e2e.mjs b/scripts/lib/browser-scenario-storage-fingerprints-e2e.mjs index 3d25385477..f2a41181f3 100644 --- a/scripts/lib/browser-scenario-storage-fingerprints-e2e.mjs +++ b/scripts/lib/browser-scenario-storage-fingerprints-e2e.mjs @@ -1,3 +1,4 @@ +import { schemaVerifierEnvironment } from "./mcp-verification-environment.mjs"; import assert from "node:assert/strict"; import { requireMcpEvidenceBundle } from "./mcp-verifier-results.mjs"; import { createHash } from "node:crypto"; @@ -137,7 +138,7 @@ export async function verifyScenarioStorageFingerprints(executable, origin) { const transport = new StdioClientTransport({ command: process.execPath, args: [fileURLToPath(new URL("../rea.mjs", import.meta.url)), "mcp"], - env: process.env, + env: schemaVerifierEnvironment(process.env), stderr: "pipe", }); const client = new Client({ diff --git a/scripts/lib/mcp-verification-environment.d.mts b/scripts/lib/mcp-verification-environment.d.mts index 8bc6d2a8b0..7020c94ec4 100644 --- a/scripts/lib/mcp-verification-environment.d.mts +++ b/scripts/lib/mcp-verification-environment.d.mts @@ -1,3 +1,10 @@ export const INLINE_EVIDENCE_MCP_ENVIRONMENT: { readonly REA_MCP_MAX_INPUT_ELEMENTS: "65536"; }; + +/** Select complete output contracts when validating advertised schemas. */ +export function schemaVerifierEnvironment< + Environment extends Record, +>( + environment: Environment, +): Environment & { REA_MCP_OUTPUT_SCHEMA_PROFILE: "full" }; diff --git a/scripts/lib/mcp-verification-environment.mjs b/scripts/lib/mcp-verification-environment.mjs index 602719f1b6..de028cb049 100644 --- a/scripts/lib/mcp-verification-environment.mjs +++ b/scripts/lib/mcp-verification-environment.mjs @@ -4,3 +4,9 @@ export const INLINE_EVIDENCE_MCP_ENVIRONMENT = Object.freeze({ REA_MCP_MAX_INPUT_ELEMENTS: "65536", }); + +/** Select complete output contracts when validating advertised schemas. */ +export const schemaVerifierEnvironment = (environment) => ({ + ...environment, + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", +}); diff --git a/scripts/lib/native-value-e2e.mjs b/scripts/lib/native-value-e2e.mjs index e26815c1f6..c16ca0763a 100644 --- a/scripts/lib/native-value-e2e.mjs +++ b/scripts/lib/native-value-e2e.mjs @@ -1,3 +1,4 @@ +import { schemaVerifierEnvironment } from "./mcp-verification-environment.mjs"; import assert from "node:assert/strict"; import { mkdir, mkdtemp, readdir, rm } from "node:fs/promises"; import { tmpdir } from "node:os"; @@ -236,7 +237,7 @@ async function verifyNativeValueAdapters( const transport = await createOwnedMcpStdioTransport({ command: process.execPath, arguments: [entrypoint, "mcp"], - hostEnvironment: env, + hostEnvironment: schemaVerifierEnvironment(env), cwd: process.cwd(), expectedCommand: null, sweepTokenOwnedProcesses: true, diff --git a/scripts/verify-apk-signature.mjs b/scripts/verify-apk-signature.mjs index 7bd3c7c22a..81875710ed 100644 --- a/scripts/verify-apk-signature.mjs +++ b/scripts/verify-apk-signature.mjs @@ -1,4 +1,5 @@ #!/usr/bin/env node +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; import assert from "node:assert/strict"; import { execFile } from "node:child_process"; import { createHash } from "node:crypto"; @@ -223,7 +224,7 @@ const client = new Client({ name: "apk-signature-verifier", version: "1" }); const transport = new StdioClientTransport({ command: process.execPath, args: [entrypoint, "mcp"], - env: environment, + env: schemaVerifierEnvironment(environment), stderr: "pipe", }); // The MCP server child reads files under the workspace until it has exited. diff --git a/scripts/verify-claude-client.mjs b/scripts/verify-claude-client.mjs index 94a4366bb5..e5ba1199b8 100644 --- a/scripts/verify-claude-client.mjs +++ b/scripts/verify-claude-client.mjs @@ -10,7 +10,10 @@ import { compactPresentationForMcpInputSchemaProfile, parseMcpInputSchemaProfile, } from "../dist/config/mcpInputSchemaProfile.js"; -import { parseMcpOutputSchemaProfile } from "../dist/config/mcpOutputSchemaProfile.js"; +import { + parseMcpOutputSchemaProfile, + resolveMcpOutputSchemaProfile, +} from "../dist/config/mcpOutputSchemaProfile.js"; import { TOOL_CONTRACTS, toolContract, @@ -201,7 +204,9 @@ await withVerifierCleanup(async () => { compactPresentationForMcpInputSchemaProfile( inputProfile.value, ), - outputSchemaProfile: outputProfile.value ?? "full", + outputSchemaProfile: resolveMcpOutputSchemaProfile( + outputProfile.value, + ), }), "binary_session must return the selected input/output profile's catalog identity", ); diff --git a/scripts/verify-go-binary.mjs b/scripts/verify-go-binary.mjs index cf9bc1327d..eaa8f544dd 100644 --- a/scripts/verify-go-binary.mjs +++ b/scripts/verify-go-binary.mjs @@ -1,4 +1,5 @@ #!/usr/bin/env node +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; import assert from "node:assert/strict"; import { isUtf8 } from "node:buffer"; import { createHash } from "node:crypto"; @@ -81,7 +82,7 @@ try { transport = await createOwnedMcpStdioTransport({ command: process.execPath, arguments: [entrypoint, "mcp"], - hostEnvironment: environment, + hostEnvironment: schemaVerifierEnvironment(environment), cwd: process.cwd(), expectedCommand: null, sweepTokenOwnedProcesses: true, diff --git a/scripts/verify-http3-network-captures.mjs b/scripts/verify-http3-network-captures.mjs index dbf9dedba0..da5fea9f43 100644 --- a/scripts/verify-http3-network-captures.mjs +++ b/scripts/verify-http3-network-captures.mjs @@ -1,4 +1,5 @@ #!/usr/bin/env node +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; import assert from "node:assert/strict"; import { createHash } from "node:crypto"; import { readFile, writeFile } from "node:fs/promises"; @@ -64,7 +65,7 @@ const client = new Client({ const transport = new StdioClientTransport({ command: process.execPath, args: [entrypoint, "mcp"], - env: environment, + env: schemaVerifierEnvironment(environment), stderr: "pipe", }); const contract = toolContract("inspect_web_network_capture"); diff --git a/scripts/verify-packet-capture.mjs b/scripts/verify-packet-capture.mjs index a57ef7a688..e3c3806763 100644 --- a/scripts/verify-packet-capture.mjs +++ b/scripts/verify-packet-capture.mjs @@ -1,4 +1,5 @@ #!/usr/bin/env node +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; import assert from "node:assert/strict"; import { requireMcpEvidenceBundle, @@ -45,7 +46,7 @@ const client = new Client({ name: "packet-capture-verifier", version: "1" }); const transport = new StdioClientTransport({ command: process.execPath, args: [entrypoint, "mcp"], - env: environment, + env: schemaVerifierEnvironment(environment), stderr: "pipe", }); diff --git a/scripts/verify-qlog-captures.mjs b/scripts/verify-qlog-captures.mjs index ee8462fa6a..d0ada1879a 100644 --- a/scripts/verify-qlog-captures.mjs +++ b/scripts/verify-qlog-captures.mjs @@ -1,4 +1,5 @@ #!/usr/bin/env node +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; import { closeVerifierResources, withVerifierCleanup, @@ -40,7 +41,7 @@ const environment = { const transport = new StdioClientTransport({ command: process.execPath, args: [entrypoint, "mcp"], - env: environment, + env: schemaVerifierEnvironment(environment), stderr: "pipe", }); let diagnostics = ""; diff --git a/scripts/verify-real-browser.mjs b/scripts/verify-real-browser.mjs index a169bb94c8..164a2ff987 100644 --- a/scripts/verify-real-browser.mjs +++ b/scripts/verify-real-browser.mjs @@ -51,6 +51,7 @@ import { verifyScenarioStorage } from "./lib/browser-scenario-storage-e2e.mjs"; import { verifyScenarioStorageFingerprints } from "./lib/browser-scenario-storage-fingerprints-e2e.mjs"; import { verifyScenarioServiceWorkers } from "./lib/browser-scenario-service-workers-e2e.mjs"; import { verifyBrowserUrlUserInfo } from "./lib/browser-url-userinfo-e2e.mjs"; +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; const SCENARIO_SECRET_VALUE = "rea-browser-verifier-secret"; const SCENARIO_URL_SECRET_VALUE = "rea-browser-url-verifier-secret"; @@ -447,7 +448,10 @@ async function verifyPublicBundle(endpoint, target, origin) { const transport = new StdioClientTransport({ command: process.execPath, args: [fileURLToPath(new URL("./rea.mjs", import.meta.url)), "mcp"], - env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent" }, + env: schemaVerifierEnvironment({ + PATH: process.env.PATH ?? "", + REA_LOG_LEVEL: "silent", + }), stderr: "pipe", }); const client = new Client({ name: "browser-bundle-e2e", version: "1" }); diff --git a/scripts/verify-real-ghidra-com.mjs b/scripts/verify-real-ghidra-com.mjs index 83605e20a8..7ba7992fe7 100644 --- a/scripts/verify-real-ghidra-com.mjs +++ b/scripts/verify-real-ghidra-com.mjs @@ -1,4 +1,5 @@ #!/usr/bin/env node +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; import { closeVerifierResources, withVerifierCleanup, @@ -95,7 +96,7 @@ const env = { const transport = new StdioClientTransport({ command: process.execPath, args: [entrypoint, "mcp"], - env, + env: schemaVerifierEnvironment(env), stderr: "pipe", }); const client = new Client({ name: "rea-real-com-proof", version: "1" }); diff --git a/scripts/verify-real-ghidra-raw.mjs b/scripts/verify-real-ghidra-raw.mjs index 82d91936a6..229cb16e3b 100644 --- a/scripts/verify-real-ghidra-raw.mjs +++ b/scripts/verify-real-ghidra-raw.mjs @@ -1,3 +1,4 @@ +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; import assert from "node:assert/strict"; import { execFile } from "node:child_process"; import { createHash } from "node:crypto"; @@ -97,7 +98,7 @@ const env = { const transport = new StdioClientTransport({ command: process.execPath, args: [entrypoint, "mcp"], - env, + env: schemaVerifierEnvironment(env), stderr: "pipe", }); const client = new Client({ name: "rea-real-raw-proof", version: "1" }); @@ -736,7 +737,7 @@ async function verifyCrossBlockProject(target, targetSha256) { command: process.execPath, args: [entrypoint, "mcp"], env: { - ...env, + ...schemaVerifierEnvironment(env), REA_GHIDRA_PROJECT_PATH: join(projectDirectory, "Boundary.gpr"), }, stderr: "pipe", diff --git a/scripts/verify-real-ghidra-riscv.mjs b/scripts/verify-real-ghidra-riscv.mjs index 7a83878d66..c33cbfb21e 100644 --- a/scripts/verify-real-ghidra-riscv.mjs +++ b/scripts/verify-real-ghidra-riscv.mjs @@ -1,4 +1,5 @@ #!/usr/bin/env node +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; import assert from "node:assert/strict"; import { execFile } from "node:child_process"; import { createHash } from "node:crypto"; @@ -83,7 +84,7 @@ for (const key of ["GHIDRA_HEADLESS_MAXMEM", "GHIDRA_MAXMEM"]) const transport = new StdioClientTransport({ command: process.execPath, args: [entrypoint, "mcp"], - env, + env: schemaVerifierEnvironment(env), stderr: "pipe", }); const client = new Client({ name: "rea-real-riscv-proof", version: "1" }); diff --git a/scripts/verify-real-hopper-callers.mjs b/scripts/verify-real-hopper-callers.mjs index 3f9cca6414..2aae7af379 100644 --- a/scripts/verify-real-hopper-callers.mjs +++ b/scripts/verify-real-hopper-callers.mjs @@ -1,3 +1,4 @@ +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; // Run from the repository root after `npm run build:cached` on macOS arm64, // with Hopper and the Xcode command-line tools installed. No target is executed. import assert from "node:assert/strict"; @@ -60,12 +61,12 @@ const transport = new StdioClientTransport({ command: process.execPath, args: ["dist/main.js"], cwd: root, - env: { + env: schemaVerifierEnvironment({ ...process.env, REA_ANALYSIS_PROVIDER: "hopper", REA_LOG_LEVEL: "silent", REA_HOPPER_CALLER_SCAN_BUDGET_MS: "5000", - }, + }), stderr: "pipe", }); let stderr = ""; diff --git a/scripts/verify-real-hopper-fat.mjs b/scripts/verify-real-hopper-fat.mjs index 99dd1330f2..9f7ddee323 100644 --- a/scripts/verify-real-hopper-fat.mjs +++ b/scripts/verify-real-hopper-fat.mjs @@ -1,3 +1,4 @@ +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; import assert from "node:assert/strict"; import { createHash } from "node:crypto"; import { execFile } from "node:child_process"; @@ -84,11 +85,11 @@ const client = new Client({ name: "hopper-fat-verifier", version: "1" }); const transport = new StdioClientTransport({ command: process.execPath, args: ["dist/main.js"], - env: { + env: schemaVerifierEnvironment({ ...process.env, REA_ANALYSIS_PROVIDER: "hopper", HOPPER_LOADER_ARGS_JSON: "[]", - }, + }), stderr: "pipe", }); transport.stderr?.on("data", (chunk) => process.stderr.write(chunk)); diff --git a/scripts/verify-real-hopper.mjs b/scripts/verify-real-hopper.mjs index ca183ffd0b..2768d7bf31 100644 --- a/scripts/verify-real-hopper.mjs +++ b/scripts/verify-real-hopper.mjs @@ -1,3 +1,4 @@ +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; import { execFile } from "node:child_process"; import { tmpdir } from "node:os"; import { promisify } from "node:util"; @@ -233,7 +234,7 @@ const targetB = fixtureTargets.secondary.path; const largeTarget = fixtureTargets.large.path; const targetHashA = fixtureTargets.primary.sha256; const targetHashB = fixtureTargets.secondary.sha256; -const serverEnvironment = { ...process.env }; +const serverEnvironment = schemaVerifierEnvironment(process.env); serverEnvironment.REA_ANALYSIS_PROVIDER = "auto"; delete serverEnvironment.HOPPER_TARGET_PATH; delete serverEnvironment.HOPPER_SECOND_TARGET_PATH; diff --git a/scripts/verify-signal-recording.mjs b/scripts/verify-signal-recording.mjs index a2be6affcb..d5526a995d 100644 --- a/scripts/verify-signal-recording.mjs +++ b/scripts/verify-signal-recording.mjs @@ -1,4 +1,5 @@ #!/usr/bin/env node +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; import assert from "node:assert/strict"; import { requireMcpEvidenceBundle, @@ -79,7 +80,7 @@ const assertRejectedSelector = ( const transport = new StdioClientTransport({ command: process.execPath, args: [entrypoint, "mcp"], - env: environment, + env: schemaVerifierEnvironment(environment), stderr: "pipe", }); // The MCP server child reads files under the root until it has exited. diff --git a/scripts/verify-wasm-artifact.mjs b/scripts/verify-wasm-artifact.mjs index 859ef9d07a..75148ded8e 100644 --- a/scripts/verify-wasm-artifact.mjs +++ b/scripts/verify-wasm-artifact.mjs @@ -1,4 +1,5 @@ #!/usr/bin/env node +import { schemaVerifierEnvironment } from "./lib/mcp-verification-environment.mjs"; import assert from "node:assert/strict"; import { execFile } from "node:child_process"; import { createHash } from "node:crypto"; @@ -89,12 +90,12 @@ const callTool = (request) => const transport = new StdioClientTransport({ command: process.execPath, args: [entrypoint, "mcp"], - env: { + env: schemaVerifierEnvironment({ ...environment, TMPDIR: childTempRoot, TMP: childTempRoot, TEMP: childTempRoot, - }, + }), stderr: "pipe", }); let cases = 0; diff --git a/scripts/verify/android/external-invokes.mjs b/scripts/verify/android/external-invokes.mjs index ff4634f815..e05394a87e 100644 --- a/scripts/verify/android/external-invokes.mjs +++ b/scripts/verify/android/external-invokes.mjs @@ -5,7 +5,10 @@ import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; import { parseEvidence } from "../../../dist/domain/evidence.js"; import { artifactMcpCall } from "../../lib/artifact-e2e.mjs"; import { withVerifierCleanup } from "../../lib/verifier-cleanup.mjs"; -import { INLINE_EVIDENCE_MCP_ENVIRONMENT } from "../../lib/mcp-verification-environment.mjs"; +import { + INLINE_EVIDENCE_MCP_ENVIRONMENT, + schemaVerifierEnvironment, +} from "../../lib/mcp-verification-environment.mjs"; import { requireMcpOutput, requireMcpToolError, @@ -104,7 +107,10 @@ const verifyApk = async (options, path, dexEntry) => { command: process.execPath, args: [options.entrypoint, "mcp"], cwd: options.repository, - env: { ...options.environment, ...INLINE_EVIDENCE_MCP_ENVIRONMENT }, + env: schemaVerifierEnvironment({ + ...options.environment, + ...INLINE_EVIDENCE_MCP_ENVIRONMENT, + }), stderr: "pipe", }); const client = new Client({ @@ -250,7 +256,10 @@ const verifySiteBudget = async (options, path, control) => { command: process.execPath, args: [options.entrypoint, "mcp"], cwd: options.repository, - env: { ...options.environment, ...INLINE_EVIDENCE_MCP_ENVIRONMENT }, + env: schemaVerifierEnvironment({ + ...options.environment, + ...INLINE_EVIDENCE_MCP_ENVIRONMENT, + }), stderr: "pipe", }); const client = new Client({ name: "rea-external-site-budget", version: "1" }); diff --git a/scripts/verify/electron/tool-contracts.mjs b/scripts/verify/electron/tool-contracts.mjs index 48009002c2..ffe32210c7 100644 --- a/scripts/verify/electron/tool-contracts.mjs +++ b/scripts/verify/electron/tool-contracts.mjs @@ -3,6 +3,7 @@ import { withVerifierCleanup, } from "../../lib/verifier-cleanup.mjs"; import { requireMcpToolError } from "../../lib/mcp-verifier-results.mjs"; +import { schemaVerifierEnvironment } from "../../lib/mcp-verification-environment.mjs"; import assert from "node:assert/strict"; import { execFile, spawn } from "node:child_process"; import { randomUUID } from "node:crypto"; @@ -77,7 +78,8 @@ bridge.exposeInMainWorld("fixture", { const transport = new StdioClientTransport({ command: process.execPath, args: [entrypoint, "mcp"], - env: { + // Results are validated against the listed complete output contracts. + env: schemaVerifierEnvironment({ ...Object.fromEntries( ["DISPLAY", "XAUTHORITY", "WAYLAND_DISPLAY"].flatMap((name) => process.env[name] === undefined ? [] : [[name, process.env[name]]], @@ -86,7 +88,7 @@ bridge.exposeInMainWorld("fixture", { PATH: process.env.PATH, REA_LOG_LEVEL: "silent", HOPPER_LAUNCHER_PATH: "/rea-unconfigured/hopper", - }, + }), stderr: "pipe", }); const client = new Client({ name: "electron-real-e2e", version: "1" }); diff --git a/src/application/DirectAnalysisStatus.ts b/src/application/DirectAnalysisStatus.ts index 94056d8df5..ce90d6cbd6 100644 --- a/src/application/DirectAnalysisStatus.ts +++ b/src/application/DirectAnalysisStatus.ts @@ -10,6 +10,7 @@ import { CURRENT_ENTRYPOINT_PATH } from "../process/CurrentEntrypointPath.js"; import { catalogIdentityForToolFamilies } from "../catalogIdentity.js"; import { progressiveCatalogIdentity } from "../catalogDiscovery.js"; import { compactPresentationForMcpInputSchemaProfile } from "../config/mcpInputSchemaProfile.js"; +import { resolveMcpOutputSchemaProfile } from "../config/mcpOutputSchemaProfile.js"; import { directAnalysisCleanupFailure, projectInvocationFailure, @@ -36,7 +37,7 @@ export const runSessionStatus = async ( compactPresentationForMcpInputSchemaProfile( config.value.mcpInputSchemaProfile, ), - config.value.mcpOutputSchemaProfile ?? "full", + resolveMcpOutputSchemaProfile(config.value.mcpOutputSchemaProfile), config.value.mcpToolFamilies === undefined ? undefined : new Set(config.value.mcpToolFamilies), diff --git a/src/catalogIdentity.ts b/src/catalogIdentity.ts index 273a721002..3a76481ed2 100644 --- a/src/catalogIdentity.ts +++ b/src/catalogIdentity.ts @@ -12,10 +12,11 @@ import { CLI_COMMAND_NAMES } from "./cliCommandNames.js"; import { compareUnicodeCodePoints } from "./domain/unicodeCodePointOrder.js"; import { advertisedInputJsonSchema, - toolOutputSchemaWithMetadata, + advertisedOutputJsonSchema, } from "./contracts/toolSchemaMetadata.js"; import { - presentAdvertisedInputSchema, + describeAdvertisedTool, + presentAdvertisedInput, type CompactInputSchemaPresentation, } from "./contracts/compactInputSchema.js"; import { @@ -25,6 +26,7 @@ import { import { COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES, compactPresentationForMcpOutputSchemaProfile, + resolveMcpOutputSchemaProfile, type McpOutputSchemaProfile, } from "./config/mcpOutputSchemaProfile.js"; import { @@ -50,11 +52,22 @@ const createToolCatalog = (contracts: readonly ToolContract[]) => returns_target_content: contract.returnsTargetContent, annotations: contract.annotations, input_schema: advertisedInputJsonSchema(contract), - output_schema: toolOutputSchemaWithMetadata(contract)[ - "~standard" - ].jsonSchema.output({ target: "draft-2020-12" }), + output_schema: advertisedOutputJsonSchema(contract), })); +/** The compact input schema and the tool description carrying its guidance. */ +const presentedInput = ( + description: string, + schema: Record, + presentation: CompactInputSchemaPresentation, +) => { + const input = presentAdvertisedInput(schema, presentation); + return { + description: describeAdvertisedTool(description, input.guidance), + input_schema: input.schema, + }; +}; + const presentToolCatalog = ( catalog: ReturnType, presentation: CompactInputSchemaPresentation | undefined, @@ -66,12 +79,7 @@ const presentToolCatalog = ( ...tool, ...(presentation === undefined ? {} - : { - input_schema: presentAdvertisedInputSchema( - tool.input_schema, - presentation, - ), - }), + : presentedInput(tool.description, tool.input_schema, presentation)), ...(outputPresentation === undefined ? {} : { @@ -243,6 +251,7 @@ export const FULL_CATALOG_IDENTITY = createCatalogIdentity(); /** Default MCP advertisement; explicit full/custom selections have their own identity. */ export const DEFAULT_CATALOG_IDENTITY = createCatalogIdentity({ compactPresentation: compactPresentationForMcpInputSchemaProfile(undefined), + outputSchemaProfile: resolveMcpOutputSchemaProfile(undefined), }); /** diff --git a/src/config/mcpOutputSchemaProfile.ts b/src/config/mcpOutputSchemaProfile.ts index 69dc864ab6..bb7a8a3875 100644 --- a/src/config/mcpOutputSchemaProfile.ts +++ b/src/config/mcpOutputSchemaProfile.ts @@ -22,6 +22,16 @@ export type McpOutputSchemaProfile = z.infer< typeof mcpOutputSchemaProfileSchema >; +/** + * Resolve the selected advertisement at the configuration boundary. No model + * API reads an output schema; MCP client validators re-check what REA already + * validates, and complete output schemas were about 92% of `tools/list`. They + * are therefore omitted unless a caller selects `compact` or `full`. + */ +export const resolveMcpOutputSchemaProfile = ( + profile: McpOutputSchemaProfile | undefined, +): McpOutputSchemaProfile => profile ?? "off"; + /** Resolve the compact output advertisement, preserving every other choice. */ export const compactPresentationForMcpOutputSchemaProfile = ( profile: McpOutputSchemaProfile | undefined, @@ -31,7 +41,7 @@ export const compactPresentationForMcpOutputSchemaProfile = ( : undefined; const profileConstraint = - 'REA_MCP_OUTPUT_SCHEMA_PROFILE must be "full" (default), "compact", or "off". The compact profile advertises every output schema within a 4,096-byte budget; the off profile omits output schemas. Both keep canonical server-side validation and complete structured content.'; + 'REA_MCP_OUTPUT_SCHEMA_PROFILE must be "off" (default), "compact", or "full". The off profile omits output schemas; compact advertises every output schema within a 4,096-byte budget; full advertises complete output schemas. All keep canonical server-side validation and complete structured content.'; /** Parse the output schema advertisement selection at server composition. */ export const parseMcpOutputSchemaProfile = ( diff --git a/src/contracts/advertisedDialect.ts b/src/contracts/advertisedDialect.ts index aa1a160c11..b42476f7ec 100644 --- a/src/contracts/advertisedDialect.ts +++ b/src/contracts/advertisedDialect.ts @@ -43,7 +43,8 @@ export type AdvertisedDialectRule = | "root-shape" // Each keyword outside the allowlist is rejected by at least one target. | "keyword" - // Gemini, Moonshot and Ark reject type lists and untyped nodes (#7103, #3800). + // Gemini, Moonshot and Ark reject type lists and untyped nodes (#7103, + // #3800); a value is nullable only as `anyOf` a typed branch and null. | "type" // VS Code and OpenAI reject arrays without `items`; Gemini rejects `items` // beside a non-array type (#3576, #7103). @@ -91,8 +92,10 @@ const PRIMITIVE_TYPES: ReadonlySet = new Set([ const TOOL_NAME = /^[a-z][a-z0-9_]{0,53}$/u; const PROPERTY_NAME = /^[A-Za-z_][A-Za-z0-9_]{0,63}$/u; const TOOL_DESCRIPTION_LIMIT = 4096; -const DESCRIPTION_LIMIT = 1024; -const BOUND_LIMIT = 1999; +/** VS Code truncates a nested description above this many characters. */ +export const ADVERTISED_DESCRIPTION_MAX_CHARACTERS = 1024; +/** llama.cpp grammars fail at length and item bounds of this value or more. */ +export const ADVERTISED_BOUND_LIMIT = 1999; // Moonshot's function-calling API rejects requests while any single tool // parameters schema exceeds its literal serialized size cap. Community // measurements bound the cap between ~13.7 KB (accepted) and ~15 KB @@ -390,6 +393,31 @@ const patternViolations = ( })), ); +const isNullBranch = (branch: unknown): boolean => + isRecord(branch) && + branch.type === "null" && + Object.keys(branch).length === 1; + +/** + * The dialect's one nullable form: `anyOf` a single typed branch and + * `{"type": "null"}`, below the root. Every surveyed target accepts a nested + * `anyOf` with typed branches, while type lists such as `[T, "null"]` are + * rejected by Gemini's legacy schema (#7103) and rewritten differently by + * Codex, Goose, VS Code and Gemini CLI. Dropping the null branch would refuse + * a valid call such as `limit: null`. The union carries its type in its + * branches, so it is not an untyped node. + */ +const isNullableUnion = (node: Readonly>): boolean => + Array.isArray(node.anyOf) && + node.anyOf.length === 2 && + node.anyOf.some(isNullBranch) && + node.anyOf.some( + (branch) => + isRecord(branch) && + typeof branch.type === "string" && + PRIMITIVE_TYPES.has(branch.type), + ); + const nodeViolations = ( tool: string, node: unknown, @@ -418,7 +446,16 @@ const nodeViolations = ( if (!ALLOWED_KEYWORDS.has(key)) add("keyword", key); const { type } = node; if (Array.isArray(type)) add("type", "type-list"); - else if (type === undefined) add("type", "untyped"); + else if (type === undefined) { + if (!isNullableUnion(node)) add("type", "untyped"); + } + // The null branch of a nullable union is typed by its own `type`. + else if ( + type === "null" && + keyword === "anyOf" && + Object.keys(node).length === 1 + ) + return violations; else if (typeof type !== "string" || !PRIMITIVE_TYPES.has(type)) add("type", `type:${String(type)}`); if (type === "array" && node.items === undefined) @@ -447,11 +484,14 @@ const nodeViolations = ( } if ( typeof node.description === "string" && - node.description.length > DESCRIPTION_LIMIT + node.description.length > ADVERTISED_DESCRIPTION_MAX_CHARACTERS ) add("description-length"); for (const bound of ["minLength", "maxLength", "minItems", "maxItems"]) - if (typeof node[bound] === "number" && node[bound] >= BOUND_LIMIT) + if ( + typeof node[bound] === "number" && + node[bound] >= ADVERTISED_BOUND_LIMIT + ) add("bound", bound); if (isRecord(node.properties)) for (const name of Object.keys(node.properties)) diff --git a/src/contracts/advertisedDialectBaseline.ts b/src/contracts/advertisedDialectBaseline.ts index e760aa44cf..4a584f0d52 100644 --- a/src/contracts/advertisedDialectBaseline.ts +++ b/src/contracts/advertisedDialectBaseline.ts @@ -19,23 +19,9 @@ export const ADVERTISED_INPUT_DIALECT_BASELINE: Readonly< Record<"compact" | "full", Readonly>> > = { - compact: { - "input bound maxItems": 2, - "input bound maxLength": 13, - "input keyword $schema": 256, - "input keyword allOf": 10, - "input keyword const": 83, - "input keyword exclusiveMinimum": 61, - "input keyword format": 20, - "input keyword minProperties": 12, - "input keyword oneOf": 3, - "input keyword prefixItems": 2, - "input keyword propertyNames": 3, - "input keyword uniqueItems": 2, - "input type type-list": 11, - "input type type:null": 14, - "input type untyped": 38, - }, + // The compact projection (`advertisedDialectProjection.ts`) leaves nothing + // outside the dialect. + compact: {}, full: { "input any-of fewer-than-two-branches": 176, "input any-of untyped-branch": 142, @@ -63,7 +49,7 @@ export const ADVERTISED_INPUT_DIALECT_BASELINE: Readonly< "input keyword then": 7, "input keyword uniqueItems": 5, "input type type-list": 53, - "input type type:null": 386, - "input type untyped": 1500, + "input type type:null": 6, + "input type untyped": 1256, }, }; diff --git a/src/contracts/advertisedDialectProjection.test.ts b/src/contracts/advertisedDialectProjection.test.ts new file mode 100644 index 0000000000..1dc306dc84 --- /dev/null +++ b/src/contracts/advertisedDialectProjection.test.ts @@ -0,0 +1,260 @@ +import { Ajv2020 } from "ajv/dist/2020.js"; +import { addFormats } from "@modelcontextprotocol/client/validators/ajv"; +import { describe, expect, it } from "vitest"; + +import { advertisedInputDialectViolations } from "./advertisedDialect.js"; +import { projectToAdvertisedDialect } from "./advertisedDialectProjection.js"; + +const project = (properties: Record, root = {}) => + projectToAdvertisedDialect({ + $schema: "https://json-schema.org/draft/2020-12/schema", + type: "object", + properties, + additionalProperties: false, + ...root, + }); + +/** Every canonical-valid value stays valid; the projection only widens. */ +const expectWidening = ( + canonical: Record, + values: readonly unknown[], +) => { + const { schema } = project({ value: canonical }); + const ajv = new Ajv2020({ strict: false }); + addFormats(ajv); + const validateCanonical = ajv.compile({ + type: "object", + properties: { value: canonical }, + }); + const validateAdvertised = ajv.compile(schema); + for (const value of values) + if (validateCanonical({ value })) + expect(validateAdvertised({ value }), JSON.stringify(value)).toBe(true); + return schema.properties as Record; +}; + +it("does not move a closed allOf member's property scope onto another member", () => { + expectWidening( + { + allOf: [ + { type: "object", properties: { a: { type: "string" } } }, + { + type: "object", + properties: { a: { type: "string" }, b: { type: "string" } }, + additionalProperties: false, + }, + ], + }, + [{ a: "x", b: "y" }], + ); +}); + +it("does not move an allOf member's tuple tail beside other positions", () => { + expectWidening( + { + type: "array", + allOf: [ + { prefixItems: [{ type: "number" }] }, + { prefixItems: [{ type: "number" }, { type: "string" }], items: false }, + ], + }, + [[1, "ok"]], + ); +}); + +it("does not move a single union branch's closure onto the parent's properties", () => { + expectWidening( + { + type: "object", + properties: { a: { type: "string" } }, + anyOf: [ + { + type: "object", + properties: { a: { type: "string" }, b: { type: "string" } }, + additionalProperties: false, + }, + ], + }, + [{ a: "x", b: "y" }], + ); +}); + +it("leaves objects open when dropped pattern properties admit names", () => { + expectWidening( + { + type: "object", + patternProperties: { "^b$": { type: "string" } }, + additionalProperties: false, + }, + [{ b: "ok" }], + ); +}); + +it("does not make a union branch's omitted rule into global guidance", () => { + const { guidance } = project({ + value: { + anyOf: [ + { type: "string", not: { enum: ["reserved"] } }, + { type: "string", enum: ["reserved"] }, + ], + }, + }); + expect(guidance.join(" ")).not.toContain('must not be "reserved"'); +}); + +it("keeps a property named __proto__ as an own property", () => { + const canonical = JSON.parse( + '{"type":"object","properties":{"__proto__":{"type":"string"}},"required":["__proto__"],"additionalProperties":false}', + ) as Record; + const { schema } = projectToAdvertisedDialect(canonical); + const properties = schema.properties as Record; + expect(Object.getPrototypeOf(properties)).toBe(Object.prototype); + expect(Object.hasOwn(properties, "__proto__")).toBe(true); + expect(properties.__proto__).toEqual({ type: "string" }); + expect(schema).toMatchObject({ + required: ["__proto__"], + additionalProperties: false, + }); +}); + +describe("portable advertised dialect projection", () => { + it.each([ + ["string const", { const: "a" }, { type: "string", enum: ["a"] }], + ["empty string const", { type: "string", const: "" }, { type: "string" }], + ["boolean const", { type: "boolean", const: true }, { type: "boolean" }], + ["numeric enum", { enum: [1, 2] }, { type: "integer" }], + [ + "integer exclusive bound", + { type: "integer", exclusiveMinimum: 0, exclusiveMaximum: 10 }, + { type: "integer", minimum: 1, maximum: 9 }, + ], + [ + "number exclusive bound", + { type: "number", exclusiveMinimum: 0.5 }, + { type: "number", minimum: 0.5 }, + ], + [ + "pattern-only allOf members", + { type: "string", allOf: [{ pattern: "^a" }, { pattern: "b$" }] }, + { type: "string", pattern: "^a" }, + ], + [ + "string literal union", + { anyOf: [{ type: "string", enum: ["a", "b"] }, { const: "c" }] }, + { type: "string", enum: ["a", "b", "c"] }, + ], + [ + "homogeneous tuple", + { + type: "array", + prefixItems: [{ type: "number" }, { type: "number" }], + minItems: 2, + maxItems: 2, + }, + { type: "array", items: { type: "number" }, minItems: 2, maxItems: 2 }, + ], + [ + "client-unsafe bounds", + { type: "string", minLength: 1, maxLength: 4096, format: "uuid" }, + { type: "string", minLength: 1 }, + ], + [ + "open map", + { + type: "object", + propertyNames: { pattern: "^k" }, + additionalProperties: { type: "string" }, + minProperties: 1, + }, + { type: "object" }, + ], + ])("projects a %s exactly or wider", (_label, canonical, expected) => { + const properties = expectWidening(canonical, [ + "a", + "c", + "", + true, + 0, + 1, + 2, + 5, + 0.5, + 10, + [1, 2], + { k: "v" }, + "ab", + ]); + expect(properties.value).toEqual(expected); + }); + + it("widens oneOf, types the union and keeps the null branch of nullable values", () => { + const { schema } = project({ + view: { + oneOf: [ + { type: "object", properties: { kind: { const: "a" } } }, + { type: "object", properties: { kind: { const: "b" } } }, + ], + }, + limit: { type: ["integer", "null"], minimum: 1 }, + }); + expect(schema.properties).toEqual({ + view: { + type: "object", + anyOf: [ + { + type: "object", + properties: { kind: { type: "string", enum: ["a"] } }, + }, + { + type: "object", + properties: { kind: { type: "string", enum: ["b"] } }, + }, + ], + }, + limit: { anyOf: [{ type: "integer", minimum: 1 }, { type: "null" }] }, + }); + // A nullable value is a typed branch or null, within the dialect. + expect( + advertisedInputDialectViolations({ name: "probe", inputSchema: schema }), + ).toEqual([]); + }); + + it("moves the root description and dropped cross-field constraints to tool guidance", () => { + const projection = project( + { + table: { type: "string" }, + row_limit: { type: "integer" }, + ids: { type: "array", items: { type: "string" }, uniqueItems: true }, + name: { type: "string", not: { enum: ["reserved"] } }, + }, + { + description: "Root guidance.", + dependentRequired: { row_limit: ["table"] }, + if: { required: ["table"] }, + then: { required: ["row_limit"] }, + }, + ); + expect(projection.guidance).toEqual([ + "Root guidance.", + "When row_limit is provided, also provide table.", + "`ids` entries must be distinct.", + '`name` must not be "reserved".', + ]); + expect(projection.schema).not.toHaveProperty("description"); + expect( + advertisedInputDialectViolations({ + name: "probe", + inputSchema: projection.schema, + }), + ).toEqual([]); + }); + + it("bounds nested descriptions without splitting a surrogate pair", () => { + const description = `${"x".repeat(1022)}😀tail`; + const { schema } = project({ value: { type: "string", description } }); + const value = (schema.properties as Record>) + .value!; + expect(value.description!.length).toBeLessThanOrEqual(1024); + expect(value.description!.endsWith("x…")).toBe(true); + }); +}); diff --git a/src/contracts/advertisedDialectProjection.ts b/src/contracts/advertisedDialectProjection.ts new file mode 100644 index 0000000000..6a0b7f6e3f --- /dev/null +++ b/src/contracts/advertisedDialectProjection.ts @@ -0,0 +1,425 @@ +import { isDeepStrictEqual } from "node:util"; + +import { isRecord } from "../domain/isRecord.js"; +import { + ADVERTISED_BOUND_LIMIT, + ADVERTISED_DESCRIPTION_MAX_CHARACTERS, +} from "./advertisedDialect.js"; + +// The one projection from a self-contained (reference-free) input schema into +// the portable advertised dialect declared in `advertisedDialect.ts`. Every +// rewrite is exact or widening: a canonical-valid input always satisfies the +// projection, and the server still validates the canonical contract. A +// constraint the dialect cannot express leaves the schema; when an agent needs +// it to build a valid call, a sentence joins the tool description instead. +// Codex strips every description inside a schema above 5,000 bytes and Zed +// drops the root description, so that guidance never lives in the schema. + +type Schema = Record; + +/** A schema in the portable dialect, plus guidance for the tool description. */ +export interface AdvertisedDialectProjection { + readonly schema: Schema; + /** The root description, then sentences for constraints the schema omits. */ + readonly guidance: readonly string[]; +} + +const isStringList = (value: unknown): value is readonly string[] => + Array.isArray(value) && value.every((item) => typeof item === "string"); + +const literalType = (value: unknown): string | undefined => { + if (typeof value === "string") return "string"; + if (typeof value === "boolean") return "boolean"; + if (typeof value === "number") + return Number.isInteger(value) ? "integer" : "number"; + return undefined; +}; + +/** One type covering every literal, or undefined when none does. */ +const literalsType = (values: readonly unknown[]): string | undefined => { + const types = new Set(values.map(literalType)); + if (types.has(undefined)) return undefined; + if (types.size === 1) return [...types][0]; + return types.size === 2 && types.has("integer") && types.has("number") + ? "number" + : undefined; +}; + +/** Sentences for `dependentRequired` and string-list `dependencies`. */ +export const dependencyGuidance = ( + schema: Readonly>, +): readonly string[] => { + const dependencies = isRecord(schema.dependentRequired) + ? schema.dependentRequired + : isRecord(schema.dependencies) + ? schema.dependencies + : {}; + return Object.entries(dependencies).flatMap(([property, partners]) => + isStringList(partners) && partners.length > 0 + ? [ + `When ${property} is provided, also provide ${partners.join(" and ")}.`, + ] + : [], + ); +}; + +const truncatedDescription = (text: string): string => { + if (text.length <= ADVERTISED_DESCRIPTION_MAX_CHARACTERS) return text; + let kept = text.slice(0, ADVERTISED_DESCRIPTION_MAX_CHARACTERS - 1); + const last = kept.charCodeAt(kept.length - 1); + if (last >= 0xd800 && last <= 0xdbff) kept = kept.slice(0, -1); + return `${kept}…`; +}; + +/** + * Hoist `allOf` members into their parent. A keyword the parent already sets + * differently is dropped, which only widens the advertisement; required names + * are a conjunction, so their union is exact. + */ +const mergeAllOf = (schema: Schema): Schema => { + if (!Array.isArray(schema.allOf)) return schema; + const { allOf, ...merged } = schema; + const tails = new Map(); + for (const key of TUPLE_TAIL_KEYS) + if (isTupleTail(merged, key)) tails.set(key, merged); + for (const member of allOf) { + if (!isRecord(member)) continue; + const flattened = mergeAllOf(member); + for (const [key, value] of Object.entries(flattened)) { + if (!Object.hasOwn(merged, key)) { + merged[key] = value; + if (isTupleTail(flattened, key)) tails.set(key, flattened); + } else if (key === "required" && isStringList(merged.required)) { + if (isStringList(value)) + merged.required = [...new Set([...merged.required, ...value])]; + } + } + } + // A closed object's properties and patterns are scoped to its own schema + // object. Hoisting its closure beside another member's properties can + // reject names accepted by the original conjunction. + delete merged.additionalProperties; + // A tuple tail applies past its own object's positions. Beside a different + // number of positions it would constrain elements the original did not. + for (const [key, owner] of tails) + if (tuplePositions(owner)?.length !== tuplePositions(merged)?.length) + delete merged[key]; + return merged; +}; + +/** Keywords that constrain only the elements past a tuple's positions. */ +const TUPLE_TAIL_KEYS = ["items", "additionalItems"] as const; + +/** Positional element schemas (2020-12 `prefixItems` or draft-07 `items`). */ +const tuplePositions = (schema: Schema): readonly unknown[] | undefined => + Array.isArray(schema.prefixItems) + ? schema.prefixItems + : Array.isArray(schema.items) + ? schema.items + : undefined; + +/** Whether `key` constrains only the elements past `schema`'s positions. */ +const isTupleTail = (schema: Schema, key: string): boolean => + (key === "additionalItems" && Array.isArray(schema.items)) || + (key === "items" && + Array.isArray(schema.prefixItems) && + !Array.isArray(schema.items)); + +const branchType = (branch: unknown): string | undefined => + isRecord(branch) && typeof branch.type === "string" ? branch.type : undefined; + +const onlyKeys = (schema: Schema, keys: readonly string[]): boolean => + Object.keys(schema).every((key) => keys.includes(key)); + +const distinct = (schemas: readonly unknown[]): unknown[] => { + const kept: unknown[] = []; + for (const schema of schemas) + if (!kept.some((known) => isDeepStrictEqual(known, schema))) + kept.push(schema); + return kept; +}; + +const label = (path: string): string => + path === "" ? "The input" : `\`${path}\``; + +/** Exclusive numeric bounds become inclusive: exact for integers. */ +const inclusiveBounds = (source: Schema, out: Schema): void => { + const { type } = source; + const integral = + type === "integer" || + (isStringList(type) && + type.includes("integer") && + !type.includes("number")); + let { minimum, maximum } = source; + if (typeof source.exclusiveMinimum === "number") { + const lower = integral + ? Math.floor(source.exclusiveMinimum) + 1 + : source.exclusiveMinimum; + minimum = typeof minimum === "number" ? Math.max(minimum, lower) : lower; + } + if (typeof source.exclusiveMaximum === "number") { + const upper = integral + ? Math.ceil(source.exclusiveMaximum) - 1 + : source.exclusiveMaximum; + maximum = typeof maximum === "number" ? Math.min(maximum, upper) : upper; + } + if (typeof minimum === "number") out.minimum = minimum; + if (typeof maximum === "number") out.maximum = maximum; +}; + +/** Length and item bounds clients can compile, and the portable pattern. */ +const scalarBounds = (source: Schema, out: Schema): void => { + for (const bound of ["minLength", "maxLength", "minItems", "maxItems"]) + if ( + Number.isSafeInteger(source[bound]) && + (source[bound] as number) < ADVERTISED_BOUND_LIMIT + ) + out[bound] = source[bound]; + if (typeof source.pattern === "string") out.pattern = source.pattern; + inclusiveBounds(source, out); +}; + +/** Only non-empty string enums are portable; other literals keep a type. */ +const literalKeywords = (source: Schema, out: Schema): void => { + const literals = Object.hasOwn(source, "const") + ? [source.const] + : Array.isArray(source.enum) && source.enum.length > 0 + ? source.enum + : undefined; + if (literals === undefined) return; + const literal = literalsType(literals); + if ( + literal === "string" && + !literals.includes("") && + (out.type === undefined || out.type === "string") + ) { + out.type = "string"; + out.enum = [...new Set(literals)]; + } else if (literal !== undefined && out.type === undefined) + out.type = literal; +}; + +const isStringEnum = (branch: unknown): branch is Schema => + isRecord(branch) && + branch.type === "string" && + Array.isArray(branch.enum) && + onlyKeys(branch, ["type", "enum", "description"]); + +/** `[T, "null"]` becomes the single nullable form: a typed branch or null. */ +const nullableForm = (out: Schema): Schema => { + if ( + !isStringList(out.type) || + out.type.length !== 2 || + !out.type.includes("null") + ) + return out; + const { type, description, ...constraints } = out; + const nonNull = (type as readonly string[]).find( + (member) => member !== "null", + )!; + return { + ...(description === undefined ? {} : { description }), + anyOf: [{ type: nonNull, ...constraints }, { type: "null" }], + }; +}; + +/** One projection: guidance sentences accumulate while the schema is built. */ +class DialectProjection { + readonly notes: string[] = []; + + note(sentence: string): void { + if (!this.notes.includes(sentence)) this.notes.push(sentence); + } + + project(value: unknown, path: string): unknown { + if (!isRecord(value)) return value; + const source = mergeAllOf(value); + const out: Schema = {}; + if (typeof source.type === "string" || isStringList(source.type)) + out.type = source.type; + if (typeof source.description === "string") + out.description = truncatedDescription(source.description); + this.objectKeywords(source, out, path); + this.arrayKeywords(source, out, path); + scalarBounds(source, out); + literalKeywords(source, out); + if (isRecord(source.not) && isStringList(source.not.enum)) + this.note( + `${label(path)} must not be ${source.not.enum.map((item) => JSON.stringify(item)).join(" or ")}.`, + ); + this.unionKeywords(source, out, path); + return nullableForm(out); + } + + /** Name and count constraints are dropped; an object's own cross-field + * rules precede those of its properties. */ + private objectKeywords(source: Schema, out: Schema, path: string): void { + for (const sentence of dependencyGuidance(source)) + this.note(path === "" ? sentence : `In ${label(path)}: ${sentence}`); + if (isRecord(source.properties)) { + // `fromEntries` defines each name as an own property, so a property + // named `__proto__` stays a property rather than setting the prototype. + const properties: Schema = Object.fromEntries( + Object.entries(source.properties) + // A never-valid property is omitted; closed objects still refuse it. + .filter(([, child]) => child !== false) + .map(([name, child]) => [ + name, + this.project(child, path === "" ? name : `${path}.${name}`), + ]), + ); + out.properties = properties; + if (isStringList(source.required)) { + const required = source.required.filter((name) => + Object.hasOwn(properties, name), + ); + if (required.length > 0) out.required = required; + } + } + if ( + source.additionalProperties === false && + (!isRecord(source.patternProperties) || + Object.keys(source.patternProperties).length === 0) + ) + out.additionalProperties = false; + } + + /** A tuple advertises the union of its positions. */ + private arrayKeywords(source: Schema, out: Schema, path: string): void { + const tuple = Array.isArray(source.prefixItems) + ? source.prefixItems + : Array.isArray(source.items) + ? source.items + : undefined; + if (tuple !== undefined) { + // Elements past the positions follow `items` (2020-12) or + // `additionalItems` (draft-07), and are unconstrained without either + // unless `maxItems` admits none. + const restSchema = Array.isArray(source.prefixItems) + ? source.items + : source.additionalItems; + const bounded = + typeof source.maxItems === "number" && source.maxItems <= tuple.length; + const rest = + restSchema === false || bounded + ? [] + : [restSchema === undefined ? true : restSchema]; + out.items = this.elementUnion([...tuple, ...rest], `${path}[]`); + } else if (isRecord(source.items)) + out.items = this.project(source.items, `${path}[]`); + if (source.uniqueItems === true) + this.note(`${label(path)} entries must be distinct.`); + } + + /** One element schema accepting every member; any-value members stay open. */ + private elementUnion(members: readonly unknown[], path: string): unknown { + return members.some( + (member) => + member === true || + (isRecord(member) && Object.keys(member).length === 0), + ) + ? {} + : this.project({ anyOf: members.filter(isRecord) }, path); + } + + /** `oneOf` widens to `anyOf`; string-literal unions merge into one enum. */ + private unionKeywords(source: Schema, out: Schema, path: string): void { + const alternatives = Array.isArray(source.anyOf) + ? source.anyOf + : Array.isArray(source.oneOf) + ? source.oneOf + : undefined; + if (alternatives === undefined) return; + const branches = this.unionBranches(alternatives, path, out.type); + if ( + branches.length > 0 && + out.enum === undefined && + (out.type === undefined || out.type === "string") && + branches.every(isStringEnum) + ) { + out.type = "string"; + out.enum = [ + ...new Set(branches.flatMap((branch) => (branch as Schema).enum)), + ]; + return; + } + if (branches.length === 1 && isRecord(branches[0])) { + const branch = branches[0]; + // As with `allOf`, a closure is scoped to its own properties: once the + // parent's properties win, neither closure may reject the branch's. + const scoped = + Object.hasOwn(out, "properties") && Object.hasOwn(branch, "properties"); + for (const [key, keyword] of Object.entries(branch)) + if (!Object.hasOwn(out, key)) out[key] = keyword; + if (scoped) delete out.additionalProperties; + return; + } + if (branches.length < 2) return; + out.anyOf = branches; + const types = new Set(branches.map(branchType)); + if (out.type === undefined && types.size === 1 && !types.has(undefined)) + out.type = [...types][0]; + // Clients require `items` on every array. The union of the branch + // element schemas restates what the branches already enforce. + if ( + out.type === "array" && + out.items === undefined && + branches.every((branch) => isRecord(branch) && isRecord(branch.items)) + ) + out.items = this.elementUnion( + branches.map((branch) => (branch as Schema).items), + `${path}[]`, + ); + } + + /** Flattened, typed from their parent and deduplicated. */ + private unionBranches( + branches: readonly unknown[], + path: string, + parentType: unknown, + ): unknown[] { + const branchNotes: string[][] = []; + const projectedBranches = distinct( + branches.flatMap((branch) => { + const typed = + typeof parentType === "string" && + isRecord(branch) && + branch.type === undefined && + branch.anyOf === undefined && + branch.oneOf === undefined + ? { ...branch, type: parentType } + : branch; + const projection = new DialectProjection(); + const projected = projection.project(typed, path); + branchNotes.push(projection.notes); + return isRecord(projected) && + Array.isArray(projected.anyOf) && + onlyKeys(projected, ["anyOf"]) + ? projected.anyOf + : [projected]; + }), + ); + for (const note of branchNotes[0] ?? []) + if (branchNotes.every((notes) => notes.includes(note))) this.note(note); + return projectedBranches; + } +} + +/** Project one root input schema into the portable advertised dialect. */ +export const projectToAdvertisedDialect = ( + root: Readonly>, +): AdvertisedDialectProjection => { + const projection = new DialectProjection(); + // The root description joins the tool description whole, not truncated. + const { description, ...rest } = root; + const projected = projection.project(rest, ""); + return { + schema: { type: "object", ...(isRecord(projected) ? projected : {}) }, + guidance: [ + ...(typeof description === "string" && description !== "" + ? [description] + : []), + ...projection.notes, + ], + }; +}; diff --git a/src/contracts/compactInputSchema.test.ts b/src/contracts/compactInputSchema.test.ts index c03e2507c3..10cb4f715a 100644 --- a/src/contracts/compactInputSchema.test.ts +++ b/src/contracts/compactInputSchema.test.ts @@ -2,7 +2,9 @@ import { Ajv } from "ajv"; import { Ajv2020 } from "ajv/dist/2020.js"; import { describe, expect, it } from "vitest"; +import { advertisedInputDialectViolations } from "./advertisedDialect.js"; import { compactAdvertisedInputSchema } from "./compactInputSchema.js"; +import { stripAnnotations } from "./compactJsonSchema.js"; const serializedBytes = (value: unknown): number => Buffer.byteLength(JSON.stringify(value), "utf8"); @@ -84,12 +86,17 @@ const oversizedPairSchema = (): Record => { describe("compact advertised input schema", () => { it("drops nested annotation prose while keeping validation keywords", () => { - const { schema, presentation } = compactAdvertisedInputSchema( + const { schema, presentation, guidance } = compactAdvertisedInputSchema( structuredClone(annotatedSchema), 13 * 1024, ); expect(presentation).toBe("inline"); - expect(schema.description).toBe("Root group guidance stays."); + // Root guidance and dependency prose move to the tool description. + expect(schema.description).toBeUndefined(); + expect(guidance).toEqual([ + "Root group guidance stays.", + "When limits is provided, also provide query.", + ]); const query = (schema.properties as Record) .query as Record; expect(query).toEqual({ type: "string", minLength: 1 }); @@ -97,7 +104,7 @@ describe("compact advertised input schema", () => { .limits as Record; expect(limits.description).toBeUndefined(); expect(limits.required).toEqual(["max_results"]); - expect(schema.dependentRequired).toEqual({ limits: ["query"] }); + expect(schema).not.toHaveProperty("dependentRequired"); expect(schema.required).toEqual(["query"]); }); @@ -106,16 +113,13 @@ describe("compact advertised input schema", () => { description: "saved data", default: { keep: "me" }, }; - const { schema } = compactAdvertisedInputSchema( - { - type: "object", - properties: { - mode: { enum: [literalMode] }, - pinned: { const: { description: "literal" } }, - }, + const schema = stripAnnotations({ + type: "object", + properties: { + mode: { enum: [literalMode] }, + pinned: { const: { description: "literal" } }, }, - 13 * 1024, - ); + }); const properties = schema.properties as Record; expect((properties.mode as Record).enum).toEqual([ literalMode, @@ -132,7 +136,7 @@ describe("compact advertised input schema", () => { properties: { before: { $ref: "#/$defs/observation~1~0" }, after: { $ref: "#/$defs/observation~1~0" }, - literal: { const: { $ref: "saved data", $defs: { keep: true } } }, + literal: { type: "string", enum: ["$ref", "$defs"] }, }, required: ["before", "after"], additionalProperties: false, @@ -151,14 +155,13 @@ describe("compact advertised input schema", () => { captured_at: "now", status_code: 200, }; + expect( + advertisedInputDialectViolations({ name: "probe", inputSchema: schema }), + ).toEqual([]); for (const input of [ { before: observation, after: observation }, { before: observation, after: { ...observation, status_code: 99 } }, - { - before: observation, - after: observation, - literal: original.properties.literal.const, - }, + { before: observation, after: observation, literal: "$defs" }, ]) expect(compact(input)).toBe(canonical(input)); }); @@ -167,7 +170,7 @@ describe("compact advertised input schema", () => { let nested: Record = { type: "string" }; for (let level = 0; level < 12; level += 1) nested = { type: "object", properties: { child: nested } }; - const { schema, presentation } = compactAdvertisedInputSchema( + const { schema, presentation, guidance } = compactAdvertisedInputSchema( { $schema: "https://json-schema.org/draft/2020-12/schema", type: "object", @@ -177,24 +180,23 @@ describe("compact advertised input schema", () => { 1_500, ); expect(presentation).toBe("reduced"); - expect(schema.$schema).toBe("https://json-schema.org/draft/2020-12/schema"); + expect(schema).not.toHaveProperty("$schema"); expect(schema).not.toHaveProperty("dependentRequired"); + expect(guidance).toContain("When detail is provided, also provide target."); }); - it("keeps unconstrained JSON values explicit for client type inference", () => { - const { schema } = compactAdvertisedInputSchema( + it("keeps unconstrained JSON values accepting every value", () => { + // The portable dialect has no any-value form; the reduced presentation + // keeps such properties unconstrained rather than narrow them. + const { schema, presentation } = compactAdvertisedInputSchema( { type: "object", properties: { normalized_result: {}, literal: { const: {} } }, }, 13312, ); - const properties = schema.properties as Record; - expect(properties.normalized_result).toEqual({ - type: ["null", "boolean", "object", "array", "number", "string"], - items: {}, - }); - expect(properties.literal).toEqual({ const: {} }); + expect(presentation).toBe("reduced"); + expect(schema.properties).toEqual({ normalized_result: {}, literal: {} }); const validate = new Ajv2020({ strict: false }).compile(schema); for (const normalized_result of [ null, @@ -224,6 +226,25 @@ describe("compact advertised input schema", () => { }); }); +it("keeps pattern-admitted property names open in reduced schemas", () => { + const canonical = { + type: "object", + properties: { a: { type: "string", pattern: "x" } }, + patternProperties: { "^b$": { type: "string" } }, + additionalProperties: false, + }; + const { schema, presentation } = compactAdvertisedInputSchema( + canonical, + 10_000, + ); + expect(presentation).toBe("reduced"); + const compact = new Ajv2020({ strict: false }).compile(schema); + expect(new Ajv2020({ strict: false }).compile(canonical)({ b: "ok" })).toBe( + true, + ); + expect(compact({ b: "ok" })).toBe(true); +}); + it("inlines the Kimi/Moonshot-rejected reference pattern from #9581", () => { // Minimal trigger from morluto/rea#9581: Moonshot rejects a $ref whose // target is a oneOf whose branches contain another $ref ("detected @@ -277,8 +298,8 @@ it("inlines the Kimi/Moonshot-rejected reference pattern from #9581", () => { } }); -describe("strict gateway compatibility", () => { - it("types homogeneous union items without discarding their branches", () => { +describe("portable dialect unions", () => { + it("types homogeneous union items and widens oneOf to anyOf", () => { const original = { type: "object", properties: { @@ -310,9 +331,20 @@ describe("strict gateway compatibility", () => { schema.properties as Record> ).actions; if (actions === undefined) throw new Error("Missing actions property"); - expect(actions.items).toMatchObject({ + expect(actions.items).toEqual({ type: "object", - oneOf: original.properties.actions.items.oneOf, + anyOf: [ + { + type: "object", + properties: { kind: { type: "string", enum: ["click"] } }, + required: ["kind"], + }, + { + type: "object", + properties: { kind: { type: "string", enum: ["type"] } }, + required: ["kind"], + }, + ], }); const canonical = new Ajv2020({ strict: false }).compile(original); const advertised = new Ajv2020({ strict: false }).compile(schema); @@ -327,7 +359,7 @@ describe("strict gateway compatibility", () => { ); }); - it("reduces mixed-type union items instead of selecting one branch type", () => { + it("reduces mixed-type union items to typed branches instead of selecting one branch type", () => { const { schema, presentation } = compactAdvertisedInputSchema( { type: "object", @@ -340,20 +372,74 @@ describe("strict gateway compatibility", () => { }, 13 * 1024, ); - expect(presentation).toBe("reduced"); const values = ( schema.properties as Record> ).values; if (values === undefined) throw new Error("Missing values property"); - expect(values.type).toBe("array"); - expect(values.items).toMatchObject({ - type: expect.arrayContaining(["string", "object"]), + expect(presentation).toBe("reduced"); + expect(values).toEqual({ + type: "array", + items: { anyOf: [{ type: "string" }, { type: "object" }] }, }); const validate = new Ajv2020({ strict: false }).compile(schema); expect(validate({ values: ["text", {}] })).toBe(true); }); - it("reduces gateway-incompatible enums to explicitly typed properties", () => { + it("never trades a typed union for a type list the dialect rejects", () => { + const canonical = { + type: "object", + properties: { + value: { anyOf: [{ type: "string" }, { type: "number" }] }, + }, + required: ["value"], + additionalProperties: false, + }; + const { schema } = compactAdvertisedInputSchema(canonical, 13 * 1024); + expect(schema.properties).toEqual({ + value: { anyOf: [{ type: "string" }, { type: "number" }] }, + }); + expect( + advertisedInputDialectViolations({ name: "probe", inputSchema: schema }) + .map(({ detail }) => detail) + .filter((detail) => detail === "type-list"), + ).toEqual([]); + // A budget that forces reduction states each union as typed branches. + const reduced = compactAdvertisedInputSchema( + { + ...canonical, + description: "x".repeat(400), + properties: { + ...canonical.properties, + label: { anyOf: [{ type: "string" }, { type: "null" }] }, + nested: { + type: "object", + properties: Object.fromEntries( + Array.from({ length: 40 }, (_, index) => [ + `field_${String(index)}`, + { type: "string", minLength: 1 }, + ]), + ), + }, + }, + }, + 1024, + ); + expect(reduced.presentation).toBe("reduced"); + expect(reduced.schema.properties).toMatchObject({ + value: { anyOf: [{ type: "string" }, { type: "number" }] }, + label: { anyOf: [{ type: "string" }, { type: "null" }] }, + }); + expect( + advertisedInputDialectViolations({ + name: "probe", + inputSchema: reduced.schema, + }).filter(({ detail }) => detail === "type-list"), + ).toEqual([]); + }); +}); + +describe("portable dialect literals and combinators", () => { + it("types untyped string enums", () => { const { schema, presentation } = compactAdvertisedInputSchema( { type: "object", @@ -362,11 +448,13 @@ describe("strict gateway compatibility", () => { }, 13 * 1024, ); - expect(presentation).toBe("reduced"); - expect(schema.properties).toEqual({ mode: { type: "string" } }); + expect(presentation).toBe("inline"); + expect(schema.properties).toEqual({ + mode: { type: "string", enum: ["fast", "safe"] }, + }); }); - it("reduces nested combinators instead of advertising them to strict gateways", () => { + it("merges a single-branch union into its parent", () => { const { schema, presentation } = compactAdvertisedInputSchema( { type: "object", @@ -387,9 +475,17 @@ describe("strict gateway compatibility", () => { }, 13 * 1024, ); - expect(presentation).toBe("reduced"); + expect(presentation).toBe("inline"); expect(JSON.stringify(schema)).not.toContain("oneOf"); - expect(schema.properties).toEqual({ query: { type: "object" } }); + expect(schema.properties).toEqual({ + query: { + type: "object", + properties: { + seed: { type: "object", properties: { kind: { type: "string" } } }, + }, + required: ["seed"], + }, + }); }); it("does not confuse keyword-named fields with schema keywords", () => { @@ -464,7 +560,7 @@ describe("compact input schema budget and reduction", () => { expect(schema.properties).toEqual({ capture: { type: "object", description: "Complete capture." }, observations: { type: "array", items: { type: "object" } }, - label: { type: ["string", "null"] }, + label: { anyOf: [{ type: "string" }, { type: "null" }] }, }); const validate = new Ajv2020({ strict: false }).compile(schema); expect(validate({ capture: {}, observations: [], label: null })).toBe(true); @@ -497,10 +593,11 @@ describe("compact input schema budget and reduction", () => { expect(compacted.presentation).toBe("reduced"); expect(serializedBytes(compacted.schema)).toBeLessThanOrEqual(1024); expect(compacted.schema.type).toBe("object"); - expect(compacted.schema.description).toContain( + expect(compacted.schema.description).toBeUndefined(); + expect(compacted.guidance.join(" ")).toContain( "Provide a complete capture pair.", ); - expect(compacted.schema.description).toContain( + expect(compacted.guidance.join(" ")).toContain( "compact input schema profile", ); const properties = compacted.schema.properties as Record; @@ -627,7 +724,7 @@ it.each(["$dynamicRef", "$recursiveRef"])( ); expect(result.presentation).toBe("reduced"); expect(JSON.stringify(result.schema)).not.toContain('"$ref":'); - expect(result.schema.description).toContain("canonical"); + expect(result.guidance.join(" ")).toContain("canonical"); }, ); @@ -699,8 +796,8 @@ it.each(["draft-2020-12", "draft-07"] as const)( }; const result = compactAdvertisedInputSchema(source, 13312); expect(result.presentation).toBe("inline"); - expect(result.schema.$schema).toBe($schema); - expect(JSON.stringify(result.schema).match(/"\$schema"/gu)).toHaveLength(1); + // Dialect declarations are not part of the portable dialect. + expect(JSON.stringify(result.schema)).not.toContain('"$schema"'); const createAjv = () => dialect === "draft-07" ? new Ajv({ strict: false }) @@ -767,8 +864,8 @@ it("bounds an oversized root description while retaining an object input", () => expect(validate("not an object")).toBe(false); }); -// An untyped primitive enum is gateway-incompatible, so these schemas take -// the reduced projection independently of the budget. +// An any-value property has no portable form, so these schemas take the +// reduced projection independently of the budget. const reducedPresentation = ( properties: Record, $defs: Record = {}, @@ -776,7 +873,7 @@ const reducedPresentation = ( const canonical = { type: "object", $defs, - properties: { mode: { enum: ["fast", "safe"] }, ...properties }, + properties: { mode: {}, ...properties }, }; const result = compactAdvertisedInputSchema(canonical, 13 * 1024); expect(result.presentation).toBe("reduced"); @@ -842,7 +939,10 @@ const reducedArrayCases = [ type: "array", items: { anyOf: [{ type: "string" }, { type: "object" }] }, }, - { type: "array", items: { type: ["string", "object"] } }, + { + type: "array", + items: { anyOf: [{ type: "string" }, { type: "object" }] }, + }, [["text", {}]], ], [ @@ -917,7 +1017,7 @@ describe("reduced array item presentation", () => { expect(validateAdvertised({ values: [[[[["deep"]]]]] })).toBe(true); }); - it("keeps an inline forbidden value as the empty schema under not", () => { + it("drops conditional branches, which only widens", () => { const canonical = { type: "object", properties: { view: { type: "string" } }, @@ -930,10 +1030,14 @@ describe("reduced array item presentation", () => { 13 * 1024, ); expect(presentation).toBe("inline"); - expect(schema).toHaveProperty(["else", "properties", "view", "not"], {}); + expect(schema).toEqual({ + type: "object", + properties: { view: { type: "string" } }, + }); const canonicalValidate = new Ajv2020({ strict: false }).compile(canonical); const advertisedValidate = new Ajv2020({ strict: false }).compile(schema); for (const input of [{}, { view: "x" }, { view: "y" }]) - expect(advertisedValidate(input)).toBe(canonicalValidate(input)); + if (canonicalValidate(input)) + expect(advertisedValidate(input)).toBe(true); }); }); diff --git a/src/contracts/compactInputSchema.ts b/src/contracts/compactInputSchema.ts index d9bdafa8ad..065e02255a 100644 --- a/src/contracts/compactInputSchema.ts +++ b/src/contracts/compactInputSchema.ts @@ -1,18 +1,19 @@ +import { advertisedInputDialectViolations } from "./advertisedDialect.js"; +import { projectToAdvertisedDialect } from "./advertisedDialectProjection.js"; import { COMPACT_SCHEMA_MAX_DEPTH, - hasGatewayIncompatibleShape, inlineWithinBudget, isSchemaObject, reduceToShallowProperties, schemaDepth, serializedBytes, stripAnnotations, - typeArrayItemUnions, } from "./compactJsonSchema.js"; -// Kimi expands schema-local references before sending tools to Moonshot. -// Compact advertisements must fit after expansion, independently of client -// reference handling. Canonical server-side validation remains unchanged. +// The compact input profile: every advertised input schema is self-contained +// (clients such as Kimi expand references before sending tools to Moonshot), +// inside the per-tool byte budget, and in the portable advertised dialect. +// Canonical server-side validation remains unchanged. export type AdvertisedInputSchemaPresentation = "inline" | "reduced"; @@ -24,6 +25,8 @@ export interface CompactInputSchemaPresentation { export interface CompactedInputSchema { readonly schema: Record; readonly presentation: AdvertisedInputSchemaPresentation; + /** Input guidance the schema does not carry; it joins the tool description. */ + readonly guidance: readonly string[]; } const INPUT_REDUCED_ANNOTATION = @@ -57,12 +60,17 @@ const retainRootPropertyGuidance = ( return { ...stripped, properties }; }; +const withinDialect = (schema: Record): boolean => + advertisedInputDialectViolations({ + name: "input", + inputSchema: schema, + }).length === 0; + /** * Render an advertised input schema within a per-schema byte budget for - * providers that validate the literal serialized schema size. Validation - * keywords and the canonical contract are preserved; only - * annotation prose and, for structurally oversized schemas, nested - * validation detail leave the advertised form. + * providers that validate the literal serialized schema size. The complete + * validation structure is projected into the portable dialect when it fits; + * otherwise only root properties and their types remain. */ export const compactAdvertisedInputSchema = ( schema: Record, @@ -72,34 +80,76 @@ export const compactAdvertisedInputSchema = ( const inline = expanded === undefined ? undefined - : typeArrayItemUnions(retainRootPropertyGuidance(schema, expanded)); + : projectToAdvertisedDialect( + retainRootPropertyGuidance(schema, expanded), + ); if ( inline !== undefined && - serializedBytes(inline) <= budgetBytes && - schemaDepth(inline) <= COMPACT_SCHEMA_MAX_DEPTH && - !hasGatewayIncompatibleShape(inline) + serializedBytes(inline.schema) <= budgetBytes && + schemaDepth(inline.schema) <= COMPACT_SCHEMA_MAX_DEPTH && + withinDialect(inline.schema) ) - return { schema: inline, presentation: "inline" }; - const reduced = reduceToShallowProperties(schema, INPUT_REDUCED_ANNOTATION); - if (serializedBytes(reduced) <= budgetBytes) - return { schema: reduced, presentation: "reduced" }; + return { ...inline, presentation: "inline" }; + const reduced = projectToAdvertisedDialect( + reduceToShallowProperties(schema, INPUT_REDUCED_ANNOTATION), + ); + if (serializedBytes(reduced.schema) <= budgetBytes) + return { ...reduced, presentation: "reduced" }; // A future contract with an extreme root-property count still advertises // an honest, budget-fitting shape instead of an oversized one. return { - schema: reduceToShallowProperties( - { type: "object" }, - INPUT_REDUCED_ANNOTATION, + ...projectToAdvertisedDialect( + reduceToShallowProperties({ type: "object" }, INPUT_REDUCED_ANNOTATION), ), presentation: "reduced", }; }; +// Registration needs the guidance and the SDK later needs the schema; both +// read one projection of the same memoized canonical schema object. +const compacted = new WeakMap>(); + +/** The advertised input for the selected presentation, with its guidance. */ +export const presentAdvertisedInput = ( + schema: Record, + compactPresentation: CompactInputSchemaPresentation | undefined, +): Pick => { + if (compactPresentation === undefined) return { schema, guidance: [] }; + const { budgetBytes } = compactPresentation; + let byBudget = compacted.get(schema); + if (byBudget === undefined) compacted.set(schema, (byBudget = new Map())); + let presented = byBudget.get(budgetBytes); + if (presented === undefined) + byBudget.set( + budgetBytes, + (presented = compactAdvertisedInputSchema(schema, budgetBytes)), + ); + return presented; +}; + /** Return the JSON schema clients receive for the selected input presentation. */ export const presentAdvertisedInputSchema = ( schema: Record, compactPresentation: CompactInputSchemaPresentation | undefined, ): Record => - compactPresentation === undefined - ? schema - : compactAdvertisedInputSchema(schema, compactPresentation.budgetBytes) - .schema; + presentAdvertisedInput(schema, compactPresentation).schema; + +/** The tool description followed by the input guidance its schema omits. */ +export function describeAdvertisedTool( + description: string, + guidance: readonly string[], +): string; +export function describeAdvertisedTool( + description: string | undefined, + guidance: readonly string[], +): string | undefined; +export function describeAdvertisedTool( + description: string | undefined, + guidance: readonly string[], +): string | undefined { + if (guidance.length === 0) return description; + const input = guidance.join(" "); + return description === undefined || description === "" + ? input + : `${description}\n\n${input}`; +} diff --git a/src/contracts/compactJsonSchema.ts b/src/contracts/compactJsonSchema.ts index 451400a99c..f14bd2d5ca 100644 --- a/src/contracts/compactJsonSchema.ts +++ b/src/contracts/compactJsonSchema.ts @@ -7,7 +7,6 @@ import { schemaChildren, schemaMaps, schemaLiteralValues, - schemaSubschemas, } from "./jsonSchemaKeywords.js"; // Shared compact JSON Schema projection primitives; canonical validation stays separate. @@ -26,9 +25,9 @@ const jsonValueTypes = [ "string", ]; // An empty item schema accepts every JSON value, including arbitrarily nested -// arrays, while satisfying clients that require `items` for array types. -// Gemini rejects `items` beside a type list; the compact catalog test fails -// if a tool would advertise this presentation. +// arrays, while satisfying clients that require `items` for array types. The +// portable advertised dialect has no any-value form, so a compact input that +// needs this presentation is reduced instead. export const jsonValueSchema = () => ({ type: [...jsonValueTypes], items: {} }); const enumValueTypes = (values: readonly unknown[]): readonly string[] => [ @@ -393,6 +392,19 @@ const REDUCED_ITEM_MAX_DEPTH = 3; * unknown, uses the empty schema: it accepts every element and still gives * clients that require `items` an item schema. */ +/** + * One `type`, or `anyOf` typed branches for several. Gemini, Moonshot and Ark + * reject type lists (#7103, #3800); `[T, "null"]` becomes the dialect's + * nullable union, and other unions keep the typed branches every surveyed + * target accepts below the root. + */ +const typedAlternatives = ( + types: readonly string[], +): Record => + types.length === 1 + ? { type: types[0] } + : { anyOf: types.map((type) => ({ type })) }; + const reducedItemSchema = ( alternatives: readonly ScopedSchema[] | undefined, references: JsonSchemaReferences, @@ -411,8 +423,7 @@ const reducedItemSchema = ( types.push(...detected); } const unique = [...new Set(types)]; - if (!unique.includes("array")) - return { type: unique.length === 1 ? unique[0] : unique }; + if (!unique.includes("array")) return typedAlternatives(unique); if (unique.length > 1) return {}; const nested: ScopedSchema[] = []; for (const { schema, scope } of alternatives) { @@ -426,81 +437,6 @@ const reducedItemSchema = ( }; }; -/** Give homogeneous array-item unions the type required by strict gateways. */ -export const typeArrayItemUnions = ( - schema: Record, -): Record => { - const references = new JsonSchemaReferences(schema); - const visit = (node: unknown): unknown => { - const rebuilt = rebuildSchemaPositions(node, visit); - if (!isSchemaObject(rebuilt) || rebuilt.type !== "array") return rebuilt; - const items = rebuilt.items; - if ( - !isSchemaObject(items) || - items.type !== undefined || - (!Array.isArray(items.oneOf) && !Array.isArray(items.anyOf)) - ) - return rebuilt; - const types = detectedTypes(items, references, references.rootScope); - return types?.length === 1 - ? { ...rebuilt, items: { ...items, type: types[0] } } - : rebuilt; - }; - return visit(schema) as Record; -}; - -/** - * Strict gateways reject some nested combinators, primitive enums without a - * type, and array items whose union has no explicit type. Homogeneous item - * unions gain their shared type; remaining incompatible shapes use the shallow - * projection while canonical runtime validation stays intact. - */ -export const hasGatewayIncompatibleShape = (schema: unknown): boolean => { - if (Array.isArray(schema)) return schema.some(hasGatewayIncompatibleShape); - if (!isSchemaObject(schema)) return false; - if (Array.isArray(schema.anyOf) && schema.anyOf.length === 0) return true; - if ( - schema.type === "array" && - isSchemaObject(schema.items) && - schema.items.type === undefined && - (Array.isArray(schema.items.oneOf) || Array.isArray(schema.items.anyOf)) - ) - return true; - if (Array.isArray(schema.required) && isSchemaObject(schema.properties)) { - for (const name of schema.required) { - const property = schema.properties[name]; - if (isSchemaObject(property) && containsGatewayCombinator(property)) - return true; - } - } - if ( - Array.isArray(schema.enum) && - schema.enum.length > 0 && - schema.type === undefined - ) { - const types = enumValueTypes(schema.enum); - if ( - types.length === 1 && - types.every((type) => - ["string", "number", "boolean", "null"].includes(type), - ) - ) - return true; - } - return schemaSubschemas(schema, { includeDefinitions: true }).some( - hasGatewayIncompatibleShape, - ); -}; - -const containsGatewayCombinator = (schema: unknown): boolean => { - if (Array.isArray(schema)) return schema.some(containsGatewayCombinator); - if (!isSchemaObject(schema)) return false; - if (schema.oneOf !== undefined || schema.allOf !== undefined) return true; - return schemaSubschemas(schema, { includeDefinitions: true }).some( - containsGatewayCombinator, - ); -}; - /** * Present a reduced property's reference-resolved types. Reduced inputs omit * nested validation, but clients still require an item schema for an array @@ -513,8 +449,7 @@ const reducedPropertyTypes = ( ): Record => { const types = detectedTypes(value, references, references.rootScope); if (types === undefined || types.length === 0) return {}; - if (!types.includes("array")) - return { type: types.length === 1 ? types[0] : types }; + if (!types.includes("array")) return typedAlternatives(types); const array = { type: "array", items: reducedItemSchema( @@ -541,20 +476,6 @@ export const reduceToShallowProperties = ( const properties = isSchemaObject(schema.properties) ? schema.properties : {}; const description = [ typeof schema.description === "string" ? schema.description : undefined, - ...Object.entries( - isSchemaObject(schema.dependentRequired) - ? schema.dependentRequired - : isSchemaObject(schema.dependencies) - ? schema.dependencies - : {}, - ).flatMap(([property, partners]) => - Array.isArray(partners) && - partners.every((partner) => typeof partner === "string") - ? [ - `When ${property} is provided, also provide ${partners.join(" and ")}.`, - ] - : [], - ), reducedAnnotation, ] .filter((text) => text !== undefined) @@ -583,8 +504,21 @@ export const reduceToShallowProperties = ( ...(typeof schema.minProperties === "number" ? { minProperties: schema.minProperties } : {}), - ...(schema.additionalProperties === false + // The reduced form drops `patternProperties`; closing it would then reject + // names those patterns admitted. + ...(schema.additionalProperties === false && + !( + isSchemaObject(schema.patternProperties) && + Object.keys(schema.patternProperties).length > 0 + ) ? { additionalProperties: false } : {}), + // Root property dependencies still apply to the shallow properties; a + // dialect projection states them as tool guidance. + ...Object.fromEntries( + ["dependentRequired", "dependencies"].flatMap((key) => + isSchemaObject(schema[key]) ? [[key, schema[key]]] : [], + ), + ), }; }; diff --git a/src/contracts/inputSchemaPresentation.ts b/src/contracts/inputSchemaPresentation.ts index 8f50f9279b..0fac9ce34a 100644 --- a/src/contracts/inputSchemaPresentation.ts +++ b/src/contracts/inputSchemaPresentation.ts @@ -5,6 +5,7 @@ import { import { isDeepStrictEqual } from "node:util"; import { advertisedSchemaPatterns } from "./advertisedDialect.js"; +import { dependencyGuidance } from "./advertisedDialectProjection.js"; import { projectPortablePattern } from "./portablePattern.js"; import { schemaArrays, @@ -1016,30 +1017,15 @@ export const presentInputJsonSchema = ( dependentRequired: _dependentRequired, ...presentation } = described; - const rootDependencies = isObject(described.dependentRequired) - ? described.dependentRequired - : isObject(described.dependencies) - ? described.dependencies - : undefined; - const dependencyGuidance = - rootDependencies === undefined - ? [] - : Object.entries(rootDependencies).flatMap(([property, partners]) => - Array.isArray(partners) && - partners.every((partner) => typeof partner === "string") - ? [ - `When ${property} is provided, also provide ${partners.join(" and ")}.`, - ] - : [], - ); - if (dependencyGuidance.length === 0) + const dependencies = dependencyGuidance(described); + if (dependencies.length === 0) return normalizeAdvertisedSchema(presentation, INPUT_NORMALIZATION) .schema as Record; const description = [ typeof presentation.description === "string" ? presentation.description : undefined, - ...dependencyGuidance, + ...dependencies, ].filter((text) => text !== undefined); return normalizeAdvertisedSchema( { ...presentation, description: description.join(" ") }, diff --git a/src/contracts/toolSchemaMetadata.ts b/src/contracts/toolSchemaMetadata.ts index 17c007056c..ca6c917178 100644 --- a/src/contracts/toolSchemaMetadata.ts +++ b/src/contracts/toolSchemaMetadata.ts @@ -136,6 +136,18 @@ export const advertisedInputJsonSchema = ( target: "draft-2020-12", }) as Record; +/** + * Return the canonical output JSON Schema: the complete projection of the + * contract's output, which the full output profile lists. A client that + * validates a reply uses it whatever the selected profile advertises. + */ +export const advertisedOutputJsonSchema = ( + contract: Contract, +): Record => + toolOutputSchemaWithMetadata(contract)["~standard"].jsonSchema.output({ + target: "draft-2020-12", + }) as Record; + /** Share output definitions and content-bound validator identities on the wire. */ export const toolOutputSchemaWithMetadata = ( contract: Contract, diff --git a/src/contracts/tupleSchemaParity.test.ts b/src/contracts/tupleSchemaParity.test.ts index 170254d60a..624cf3820c 100644 --- a/src/contracts/tupleSchemaParity.test.ts +++ b/src/contracts/tupleSchemaParity.test.ts @@ -9,7 +9,8 @@ import { applicationVersionComparisonResultSchema } from "../domain/javascript/j import { kernelModuleSchema } from "../domain/native/kernelModule.js"; import { pinvokeVerificationSchema } from "../domain/managed/managedNativeVerificationSchemas.js"; import { relationshipCoverageSchema } from "../domain/relationshipCoverage.js"; -import { compactAdvertisedInputSchema } from "./compactInputSchema.js"; +import { advertisedInputDialectViolations } from "./advertisedDialect.js"; +import { projectToAdvertisedDialect } from "./advertisedDialectProjection.js"; const directions = [ [], @@ -187,57 +188,101 @@ describe.each(["draft-2020-12", "draft-07"] as const)( ); describe("fixed tuple element guidance in advertised projections", () => { + const tuplesIn = (schema: unknown): Record[] => { + const found: Record[] = []; + const visit = (value: unknown): void => { + if (Array.isArray(value)) { + value.forEach(visit); + } else if (typeof value === "object" && value !== null) { + const candidate = value as Record; + if ( + Array.isArray(candidate.prefixItems) || + (Array.isArray(candidate.items) && "additionalItems" in candidate) + ) + found.push(candidate); + Object.values(candidate).forEach(visit); + } + }; + visit(schema); + return found; + }; + it.each(["draft-2020-12", "draft-07"] as const)( - "keeps literal item guidance in full and compact %s projections", + "keeps literal item guidance in full %s projections", (target) => { const full = z.toJSONSchema(relationshipCoverageSchema, { io: "output", target, }); - for (const schema of [ - full, - compactAdvertisedInputSchema(full, 64 * 1024).schema, - ]) { - const tupleSchemas: Record[] = []; - const visit = (value: unknown): void => { - if (Array.isArray(value)) { - value.forEach(visit); - } else if (typeof value === "object" && value !== null) { - const candidate = value as Record; - if ( - Array.isArray(candidate.prefixItems) || - (Array.isArray(candidate.items) && "additionalItems" in candidate) - ) - tupleSchemas.push(candidate); - Object.values(candidate).forEach(visit); - } - }; - visit(schema); - - expect(tupleSchemas).toHaveLength(6); - for (const tuple of tupleSchemas) { - expect(tuple).toMatchObject({ - type: "array", - minItems: expect.any(Number), - maxItems: expect.any(Number), - }); - const positional = z - .array(z.object({ const: z.string() })) - .parse(tuple.prefixItems ?? tuple.items); - const rest = tuple.prefixItems ? tuple.items : tuple.additionalItems; - const ajv = - target === "draft-2020-12" - ? new Ajv2020({ strict: false }) - : new Ajv({ strict: false }); - const validateItem = ajv.compile( - z.record(z.string(), z.unknown()).parse(rest), + const tupleSchemas = tuplesIn(full); + expect(tupleSchemas).toHaveLength(6); + for (const tuple of tupleSchemas) { + expect(tuple).toMatchObject({ + type: "array", + minItems: expect.any(Number), + maxItems: expect.any(Number), + }); + const positional = z + .array(z.object({ const: z.string() })) + .parse(tuple.prefixItems ?? tuple.items); + const rest = tuple.prefixItems ? tuple.items : tuple.additionalItems; + const ajv = + target === "draft-2020-12" + ? new Ajv2020({ strict: false }) + : new Ajv({ strict: false }); + const validateItem = ajv.compile( + z.record(z.string(), z.unknown()).parse(rest), + ); + for (const value of ["callers", "callees", "incoming", null]) + expect(validateItem(value), JSON.stringify(tuple)).toBe( + positional.some((position) => position.const === value), ); - for (const value of ["callers", "callees", "incoming", null]) - expect(validateItem(value), JSON.stringify(tuple)).toBe( - positional.some((position) => position.const === value), - ); - } } }, ); + + it.each(["draft-2020-12", "draft-07"] as const)( + "advertises every position as one element enum in the portable %s dialect", + (target) => { + // Tuples are outside the portable dialect: the element schema becomes + // the union of the positions, and the length bounds stay. + const { schema } = projectToAdvertisedDialect( + z.toJSONSchema( + z.strictObject({ coverage: relationshipCoverageSchema }), + { + io: "output", + target, + }, + ), + ); + expect(tuplesIn(schema)).toEqual([]); + expect( + advertisedInputDialectViolations({ + name: "probe", + inputSchema: schema, + }).filter(({ rule }) => rule !== "input-depth"), + ).toEqual([]); + const ajv = new Ajv2020({ strict: false }); + const validate = ajv.compile(schema); + for (const limited_directions of directions) + expect( + validate({ + coverage: { + status: "unknown", + reason: "fixture", + limited_directions, + }, + }), + JSON.stringify(limited_directions), + ).toBe( + // Widened: any one or two known directions, in any order. + Array.isArray(limited_directions) && + limited_directions.length > 0 && + limited_directions.length <= 2 && + limited_directions.every( + (direction) => direction === "callers" || direction === "callees", + ), + ); + }, + ); }); diff --git a/src/main.ts b/src/main.ts index 3160649239..d198809b91 100644 --- a/src/main.ts +++ b/src/main.ts @@ -95,8 +95,11 @@ export const run = async ( dependencies.writeStderr(`${projectAnalysisError(config.error).message}\n`); return 1; } - const fullOutputCatalog = + // The recorded catalog carries complete output schemas, which the deferred + // server either lists (full) or omits (off, the default). + const recordedOutputCatalog = environment.REA_MCP_OUTPUT_SCHEMA_PROFILE === undefined || + environment.REA_MCP_OUTPUT_SCHEMA_PROFILE === "off" || environment.REA_MCP_OUTPUT_SCHEMA_PROFILE === "full"; // The recorded catalog holds the complete inventory and its full-catalog // identity, so only an unscoped registration can be served from it. A @@ -107,7 +110,7 @@ export const run = async ( RECORDED_MCP_CATALOG_SHA256 !== undefined && dependencies.transport !== "http" && config.value.hopperTargetPath === undefined && - fullOutputCatalog && + recordedOutputCatalog && !scopedToolFamilies && dependencies.createServer === undefined && dependencies.loadOptionalProviders === undefined diff --git a/src/main/deferredRuntime.ts b/src/main/deferredRuntime.ts index 78ab0363b4..91ef2d823e 100644 --- a/src/main/deferredRuntime.ts +++ b/src/main/deferredRuntime.ts @@ -14,6 +14,8 @@ import { resolveMcpInputSchemaProfile, type McpInputSchemaProfile, } from "../config/mcpInputSchemaProfile.js"; +import { resolveMcpOutputSchemaProfile } from "../config/mcpOutputSchemaProfile.js"; +import { RECORDED_CATALOG_DIGEST_PROFILES } from "../recordedCatalogDigests.js"; import { DeferredEvidenceMcpServer } from "../server/DeferredEvidenceMcpServer.js"; import { ToolResultDelivery } from "../server/ToolResultDelivery.js"; import { createLogger } from "../logger.js"; @@ -137,6 +139,11 @@ export const runDeferred = async ( layer: "server", }); const profile = resolveMcpInputSchemaProfile(config.mcpInputSchemaProfile); + // `main.ts` defers only the profiles the recorded catalog can serve. + const outputSchemaProfile = + resolveMcpOutputSchemaProfile(config.mcpOutputSchemaProfile) === "full" + ? "full" + : "off"; let firstCatalog: RecordedMcpCatalog | undefined = readCatalog(profile); const servers = new OwnedMcpServers(); let handle: McpServerHandle | undefined; @@ -220,6 +227,14 @@ export const runDeferred = async ( maximumInputElements: config.mcpMaxInputElements, compactPresentation: compactPresentationForMcpInputSchemaProfile(profile), + outputSchemaProfile, + identity: { + ...catalog.identity, + digests: + RECORDED_CATALOG_DIGEST_PROFILES?.[profile][ + outputSchemaProfile + ] ?? catalog.identity.digests, + }, }, ); if (config.mcpToolCatalogProfile === "hot-cold") diff --git a/src/mcpDoctor.ts b/src/mcpDoctor.ts index 5de84f9446..db28b59b1e 100644 --- a/src/mcpDoctor.ts +++ b/src/mcpDoctor.ts @@ -16,7 +16,10 @@ import { compactPresentationForMcpInputSchemaProfile, parseMcpInputSchemaProfile, } from "./config/mcpInputSchemaProfile.js"; -import { parseMcpOutputSchemaProfile } from "./config/mcpOutputSchemaProfile.js"; +import { + parseMcpOutputSchemaProfile, + resolveMcpOutputSchemaProfile, +} from "./config/mcpOutputSchemaProfile.js"; import { parseMcpToolFamilies } from "./config/mcpToolFamilies.js"; import { parseMcpToolCatalogProfile } from "./config/mcpToolCatalogProfile.js"; import { @@ -25,6 +28,8 @@ import { progressiveCatalogIdentity, } from "./catalogDiscovery.js"; import { contractsForToolFamilies } from "./contracts/toolFamilies.js"; +import { toolContract } from "./contracts/toolContracts.js"; +import { advertisedOutputJsonSchema } from "./contracts/toolSchemaMetadata.js"; import { promptsForToolFamilies } from "./contracts/promptToolFamilyScope.js"; import { err, ok } from "./domain/result.js"; import { AnalysisError } from "./domain/analysisErrorBase.js"; @@ -270,18 +275,23 @@ const inspectProductionMcpSession = async ( promptsRequest, ]); const sessionTool = tools.tools.find(({ name }) => name === "binary_session"); + // The reply is checked against the canonical output contract, not the + // listing: the default output profile omits output schemas and `compact` + // widens them, so the listed schema cannot establish a valid reply. + const toolDefinition = { + ...(sessionTool ?? { + name: "binary_session", + inputSchema: { type: "object" as const }, + }), + outputSchema: advertisedOutputJsonSchema(toolContract("binary_session")), + }; // A scope without the session family cannot serve request flow at all; // every other case still probes the call so the report names the real cause. const requestFlow = expectation.expectedToolNames.includes("binary_session") ? await client .callTool( { name: "binary_session", arguments: {} }, - { - ...requestOptions(deadline, signal), - ...(sessionTool === undefined - ? {} - : { toolDefinition: sessionTool }), - }, + { ...requestOptions(deadline, signal), toolDefinition }, ) .then(ok, (cause: unknown) => err({ @@ -625,9 +635,9 @@ const doctorExpectationForEnvironment = ( inputProfile.ok ? inputProfile.value : undefined, ), contracts, - outputSchemaProfile: outputProfile.ok - ? (outputProfile.value ?? "full") - : "full", + outputSchemaProfile: resolveMcpOutputSchemaProfile( + outputProfile.ok ? outputProfile.value : undefined, + ), promptContracts, }); return { diff --git a/src/server/DeferredEvidenceMcpServer.ts b/src/server/DeferredEvidenceMcpServer.ts index bc9e325ec2..66ced19ab2 100644 --- a/src/server/DeferredEvidenceMcpServer.ts +++ b/src/server/DeferredEvidenceMcpServer.ts @@ -66,6 +66,10 @@ export class DeferredEvidenceMcpServer extends EvidenceMcpServerCore { // Recorded tools carry their presented schemas. The selected presentation // still applies to transport discovery, which is not recorded. readonly compactPresentation?: CompactInputSchemaPresentation | undefined; + // Recorded tools carry complete output schemas; `off` omits them from + // listings while the runtime still validates structured results. + readonly outputSchemaProfile?: "full" | "off" | undefined; + readonly identity?: RecordedMcpCatalog["identity"] | undefined; } = {}, ) { super( @@ -80,9 +84,8 @@ export class DeferredEvidenceMcpServer extends EvidenceMcpServerCore { undefined, delivery, options.compactPresentation, - // `main.ts` only defers the exact full advertisement catalog. - "full", - catalog.identity, + options.outputSchemaProfile ?? "full", + options.identity ?? catalog.identity, ); for (const name of catalog.evidenceReferenceAlternatives) this.registerEvidenceReferenceAlternative(name); @@ -217,6 +220,21 @@ export class DeferredEvidenceMcpServer extends EvidenceMcpServerCore { return this.partialClose?.() ?? Promise.resolve(); } + /** + * Recorded tools already list their presented description; keep it rather + * than project every canonical input schema when the runtime materializes. + */ + protected override describeTool( + name: string, + description: string | undefined, + guidance: () => readonly string[], + ): { description?: string } { + const recorded = this.tools.get(name)?.registered.description; + if (this.materializing && recorded !== undefined) + return { description: recorded }; + return super.describeTool(name, description, guidance); + } + protected override bindTool( name: string, config: ToolConfig, diff --git a/src/server/EvidenceMcpServerCore.ts b/src/server/EvidenceMcpServerCore.ts index df494860b4..49a73c65d6 100644 --- a/src/server/EvidenceMcpServerCore.ts +++ b/src/server/EvidenceMcpServerCore.ts @@ -30,7 +30,8 @@ import { import { analysisErrorProjectionSchema } from "../contracts/errorSchemas.js"; import { transportClosedByOwner } from "./transportOwnerClose.js"; import { - presentAdvertisedInputSchema, + describeAdvertisedTool, + presentAdvertisedInput, type CompactInputSchemaPresentation, } from "../contracts/compactInputSchema.js"; import { presentAdvertisedOutputSchema } from "../contracts/compactOutputSchema.js"; @@ -350,10 +351,7 @@ export class EvidenceMcpServerCore extends RequestTrackingMcpServer { // advertisement still follows the selected input presentation. super.registerTool( name, - { - ...config, - inputSchema: this.presentInputSchema(config.inputSchema), - }, + { ...config, ...this.presentInput(name, config) }, (input, context) => { context.mcpReq.signal.throwIfAborted(); return catalog.discover(input); @@ -456,17 +454,21 @@ export class EvidenceMcpServerCore extends RequestTrackingMcpServer { ): RegisteredTool { if (canReferenceInlineEvidence(config.inputSchema)) this.registerEvidenceReferenceAlternative(name); - const presentedSchema = this.presentInputSchema(config.inputSchema); - const registeredConfig = - presentedSchema === undefined - ? config + const presented = this.presentInput(name, config); + const registeredConfig = { + ...config, + ...(presented.description === undefined + ? {} + : { description: presented.description }), + ...(presented.inputSchema === undefined + ? {} : { - ...config, inputSchema: captureInputValidation( name, - presentedSchema as StandardSchemaWithJSON, + presented.inputSchema as StandardSchemaWithJSON, ), - }; + }), + }; const scoped = async (input: never, context: never) => { const request = currentMcpToolRequest(); request?.beginHandler(); @@ -510,17 +512,50 @@ export class EvidenceMcpServerCore extends RequestTrackingMcpServer { } /** - * The input schema this server advertises for one registration: the - * selected compact presentation, or the registered schema unchanged. Every - * listed tool, including transport discovery, takes this one path. + * The input schema and tool description this server advertises for one + * registration: the selected compact presentation, whose guidance joins + * the description, or both unchanged. Every listed tool, including + * transport discovery, takes this one path. */ - private presentInputSchema(schema: Schema): Schema { - const budget = this.compactPresentation?.budgetBytes; - if (budget === undefined || !hasAdvertisedInputJsonSchema(schema)) - return schema; - return transformAdvertisedInputJsonSchema(schema, (projected) => - presentAdvertisedInputSchema(projected, { budgetBytes: budget }), - ); + private presentInput( + name: string, + config: { readonly description?: string; readonly inputSchema?: Schema }, + ): { description?: string; inputSchema?: Schema } { + const { description, inputSchema } = config; + const presentation = this.compactPresentation; + if ( + presentation === undefined || + inputSchema === undefined || + !hasAdvertisedInputJsonSchema(inputSchema) + ) + return config; + const present = (projected: Record) => + presentAdvertisedInput(projected, presentation); + return { + ...this.describeTool(name, description, () => { + const projected = (inputSchema as StandardSchemaWithJSON)[ + "~standard" + ].jsonSchema.input({ target: "draft-2020-12" }); + return isRecord(projected) ? present(projected).guidance : []; + }), + inputSchema: transformAdvertisedInputJsonSchema( + inputSchema, + (projected) => present(projected).schema, + ), + }; + } + + /** + * The listed description: guidance the compact input schema omits follows + * the registered description. Reading the guidance projects the schema. + */ + protected describeTool( + _name: string, + description: string | undefined, + guidance: () => readonly string[], + ): { description?: string } { + const described = describeAdvertisedTool(description, guidance()); + return described === undefined ? {} : { description: described }; } /** Whether this process advertises one named operation. */ diff --git a/src/server/createServer.ts b/src/server/createServer.ts index 04d5ef290a..62a5b83f93 100644 --- a/src/server/createServer.ts +++ b/src/server/createServer.ts @@ -56,6 +56,7 @@ import type { ToolResultDelivery } from "./ToolResultDelivery.js"; import { parseMcpResponseBudget } from "../config/mcpResponseBudget.js"; import { parseMcpOutputSchemaProfile, + resolveMcpOutputSchemaProfile, type McpOutputSchemaProfile, } from "../config/mcpOutputSchemaProfile.js"; import { @@ -919,7 +920,7 @@ const outputSchemaProfile = ( environment.REA_MCP_OUTPUT_SCHEMA_PROFILE, ); if (!configured.ok) throw configured.error; - return configured.value ?? "full"; + return resolveMcpOutputSchemaProfile(configured.value); }; const selectCompactInputSchemaPresentation = ( diff --git a/tests/acceptance/artifactExtractionSelection.test.ts b/tests/acceptance/artifactExtractionSelection.test.ts index b891a60a96..b8223a7d5a 100644 --- a/tests/acceptance/artifactExtractionSelection.test.ts +++ b/tests/acceptance/artifactExtractionSelection.test.ts @@ -20,6 +20,7 @@ import { artifactInspectionResultSchema } from "../../src/domain/artifactInspect import { parseEvidence } from "../../src/domain/evidence.js"; import { advertisedMcpToolSchemas } from "../fixtures/advertisedMcpTool.js"; import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js"; +import { schemaVerifierEnvironment } from "../../scripts/lib/mcp-verification-environment.mjs"; const execute = promisify(execFile); const cli = fileURLToPath(new URL("../../scripts/rea.mjs", import.meta.url)); @@ -122,11 +123,11 @@ it("keeps CLI/MCP selection identity, invalid-input refusals and committed outpu const transport = new StdioClientTransport({ command: process.execPath, args: [cli, "mcp"], - env: { + env: schemaVerifierEnvironment({ ...data.environment, REA_MCP_INPUT_SCHEMA_PROFILE: "full", REA_MCP_TOOL_FAMILIES: "artifact,session", - }, + }), stderr: "pipe", }); const client = new Client({ diff --git a/tests/acceptance/harmonyArtifactJourney.test.ts b/tests/acceptance/harmonyArtifactJourney.test.ts index 571e2cd27c..a0f93c5f01 100644 --- a/tests/acceptance/harmonyArtifactJourney.test.ts +++ b/tests/acceptance/harmonyArtifactJourney.test.ts @@ -16,6 +16,7 @@ import { type HarmonyApplicationProjectionResult, } from "../../src/domain/harmony/harmonyApplication.js"; import { workspaceCliTest as test } from "../support/cli/workspaceCliFixture.js"; +import { schemaVerifierEnvironment } from "../../scripts/lib/mcp-verification-environment.mjs"; const expectProjectionParity = ( graph: HarmonyApplicationProjectionResult, @@ -125,7 +126,7 @@ test("preserves Harmony archive formats through the public CLI/MCP lifecycle", a const transport = new StdioClientTransport({ command: process.execPath, args: [resolve("scripts/rea.mjs"), "mcp"], - env: environment, + env: schemaVerifierEnvironment(environment), stderr: "pipe", }); try { diff --git a/tests/acceptance/managedMemberRecovery.test.ts b/tests/acceptance/managedMemberRecovery.test.ts index 904039c4b0..9ec070eb41 100644 --- a/tests/acceptance/managedMemberRecovery.test.ts +++ b/tests/acceptance/managedMemberRecovery.test.ts @@ -26,6 +26,7 @@ const createTransport = () => env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent", + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", ...(process.platform === "win32" ? { SystemRoot: process.env.SystemRoot ?? "", diff --git a/tests/acceptance/setup/dshClientSetup.test.ts b/tests/acceptance/setup/dshClientSetup.test.ts index 3a900b5b1f..f350df1460 100644 --- a/tests/acceptance/setup/dshClientSetup.test.ts +++ b/tests/acceptance/setup/dshClientSetup.test.ts @@ -8,6 +8,7 @@ import { parse } from "yaml"; import { z } from "zod"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { cliTest } from "../../support/cli/cliFixture.js"; +import { schemaVerifierEnvironment } from "../../../scripts/lib/mcp-verification-environment.mjs"; cliTest.for(["setup", "uninstall"] as const)( "DSH CLI %s refuses a multi-document patch without losing caller content", @@ -308,7 +309,10 @@ async function verifyGeneratedLauncher( command: registration.command, args: registration.args, cwd: home, - env: { PATH: process.env.PATH ?? "", ...environment }, + env: schemaVerifierEnvironment({ + PATH: process.env.PATH ?? "", + ...environment, + }), stderr: "pipe", }); const client = new Client({ name: "dsh-setup-lifecycle", version: "1" }); diff --git a/tests/acceptance/setup/launcherAliasesCli.test.ts b/tests/acceptance/setup/launcherAliasesCli.test.ts index 8864b3923f..fe0fff184a 100644 --- a/tests/acceptance/setup/launcherAliasesCli.test.ts +++ b/tests/acceptance/setup/launcherAliasesCli.test.ts @@ -9,6 +9,7 @@ import { z } from "zod"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { cliTest } from "../../support/cli/cliFixture.js"; +import { schemaVerifierEnvironment } from "../../../scripts/lib/mcp-verification-environment.mjs"; cliTest.skipIf(process.platform === "win32")( "migrates an owned shebang registration and starts its generated MCP command without a runtime on PATH", @@ -105,7 +106,7 @@ cliTest.skipIf(process.platform === "win32")( command: registration.command, args: registration.args, cwd: home, - env: { ...environment, ...registration.env }, + env: schemaVerifierEnvironment({ ...environment, ...registration.env }), stderr: "pipe", }); const client = new Client({ name: "setup-interpreter", version: "1" }); diff --git a/tests/boundary/cli/workflowInputIdentities.test.ts b/tests/boundary/cli/workflowInputIdentities.test.ts index 7163712abd..1c092dd8f7 100644 --- a/tests/boundary/cli/workflowInputIdentities.test.ts +++ b/tests/boundary/cli/workflowInputIdentities.test.ts @@ -52,7 +52,10 @@ const runJson = async ( : ["--evidence-bundle", evidenceBundle]), ], cwd: root, - environment: { REA_LOG_LEVEL: "silent" }, + environment: { + REA_LOG_LEVEL: "silent", + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", + }, }); }; const connectMcp = async (root: string) => { @@ -73,6 +76,7 @@ const connectMcp = async (root: string) => { XDG_CONFIG_HOME: join(root, "config"), XDG_CACHE_HOME: join(root, "cache"), REA_LOG_LEVEL: "silent", + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", ...INLINE_EVIDENCE_MCP_ENVIRONMENT, }, }); diff --git a/tests/boundary/mcp/advertisedDialect.test.ts b/tests/boundary/mcp/advertisedDialect.test.ts index e7c0473ccc..7ce89db193 100644 --- a/tests/boundary/mcp/advertisedDialect.test.ts +++ b/tests/boundary/mcp/advertisedDialect.test.ts @@ -67,6 +67,67 @@ describe("portable regex grammar", () => { }); }); +describe("nullable values", () => { + const check = (properties: Record) => + advertisedInputDialectViolations({ + name: "probe", + inputSchema: { type: "object", properties }, + }).map(({ path, rule, detail }) => [path, rule, detail]); + + it("accepts anyOf one typed branch and null below the root", () => { + expect( + check({ + limit: { + description: "Entries; null returns every remaining entry.", + anyOf: [{ type: "integer", minimum: 1 }, { type: "null" }], + }, + names: { + anyOf: [ + { type: "array", items: { type: "string" } }, + { type: "null" }, + ], + }, + }), + ).toEqual([]); + }); + + it.each([ + [ + "a type list", + { type: ["integer", "null"] }, + ["$.properties.value", "type", "type-list"], + ], + [ + "a bare null type", + { type: "null" }, + ["$.properties.value", "type", "type:null"], + ], + [ + "an untyped non-null branch", + { anyOf: [{ minimum: 1 }, { type: "null" }] }, + ["$.properties.value", "type", "untyped"], + ], + ])("refuses %s", (_name, value, violation) => { + expect(check({ value })).toContainEqual(violation); + }); + + it("refuses a nullable root", () => { + expect( + advertisedInputDialectViolations({ + name: "probe", + inputSchema: { + anyOf: [{ type: "object", properties: {} }, { type: "null" }], + }, + }).map(({ rule, detail }) => [rule, detail]), + ).toEqual( + expect.arrayContaining([ + ["root-shape", undefined], + ["any-of", "root"], + ]), + ); + }); +}); + const listTools = async (environment: Readonly) => { const session = createTestBinarySession(() => { throw new Error("No deep provider may start for a schema projection"); diff --git a/tests/boundary/mcp/androidPackageSummaryMcp.test.ts b/tests/boundary/mcp/androidPackageSummaryMcp.test.ts index a84269e9d4..b50fa35a23 100644 --- a/tests/boundary/mcp/androidPackageSummaryMcp.test.ts +++ b/tests/boundary/mcp/androidPackageSummaryMcp.test.ts @@ -177,7 +177,10 @@ const createJniSummaryContractFixture = async () => { close: async () => {}, inspectAvailability: vi.fn(), }, - environment: { REA_MCP_TOOL_FAMILIES: "android,application,session" }, + environment: { + REA_MCP_TOOL_FAMILIES: "android,application,session", + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", + }, }, }); return { ...harness, apk, selection, raw, declarations, execute }; @@ -281,7 +284,10 @@ it.skipIf(process.platform === "win32")( const harness = await createApplicationMcpHarness({ serverOptions: { androidAnalysis: provider, - environment: { REA_MCP_TOOL_FAMILIES: "android,application,session" }, + environment: { + REA_MCP_TOOL_FAMILIES: "android,application,session", + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", + }, }, }); onTestFinished(harness.close); diff --git a/tests/boundary/mcp/compactInputSchemaMcp.test.ts b/tests/boundary/mcp/compactInputSchemaMcp.test.ts index 625dd53343..70ad810b1d 100644 --- a/tests/boundary/mcp/compactInputSchemaMcp.test.ts +++ b/tests/boundary/mcp/compactInputSchemaMcp.test.ts @@ -9,7 +9,11 @@ import { } from "../../../src/contracts/toolContracts.js"; import { PROMPT_CONTRACTS } from "../../../src/contracts/promptContracts.js"; import { advertisedInputJsonSchema } from "../../../src/contracts/toolSchemaMetadata.js"; -import { presentAdvertisedInputSchema } from "../../../src/contracts/compactInputSchema.js"; +import { + describeAdvertisedTool, + presentAdvertisedInput, + presentAdvertisedInputSchema, +} from "../../../src/contracts/compactInputSchema.js"; import { createCatalogDigests } from "../../../src/catalogIdentity.js"; import { COMPACT_INPUT_SCHEMA_BUDGET_BYTES, @@ -22,7 +26,6 @@ import { toolRegistrationOptions } from "../../../src/server/toolRegistrationOpt import { createTestMcp } from "../../support/mcp/mcpFixture.js"; import { arrayItemSchemaError, - strictGatewayItemSchemaErrors, unsupportedInputLiteralSchemaError, } from "../../fixtures/mcpClientSchemaConstraints.js"; @@ -136,21 +139,7 @@ describe("MCP compact input schema profile", () => { }); }); -it("advertises item schemas strict gateways accept", async () => { - const server = compactServer(); - registerCatalog(server); - const tools = await listToolsThroughTransport(server); - expect(tools.map(({ name }) => name).sort()).toEqual( - TOOL_CONTRACTS.map(({ name }) => name).sort(), - ); - expect( - tools.flatMap(({ name, inputSchema }) => - strictGatewayItemSchemaErrors(inputSchema, name), - ), - ).toEqual([]); -}); - -it("keeps output schemas and tool descriptions identical to the default profile", async () => { +it("keeps output schemas and extends tool descriptions only with input guidance", async () => { const compact = compactServer(); const full = fullServer(); registerCatalog(compact); @@ -165,7 +154,18 @@ it("keeps output schemas and tool descriptions identical to the default profile" const reference = fullByName.get(tool.name); if (reference === undefined) throw new Error(`Missing ${tool.name}`); expect(tool.outputSchema, tool.name).toEqual(reference.outputSchema); - expect(tool.description, tool.name).toBe(reference.description); + // Guidance the compact schema omits follows the unchanged description. + const contract = TOOL_CONTRACTS.find(({ name }) => name === tool.name); + if (contract === undefined) throw new Error(`Missing ${tool.name}`); + expect(tool.description, tool.name).toBe( + describeAdvertisedTool( + reference.description, + presentAdvertisedInput(advertisedInputJsonSchema(contract), { + budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES, + }).guidance, + ), + ); + expect(tool.inputSchema.description, tool.name).toBeUndefined(); } }); @@ -189,41 +189,6 @@ it("detects missing client-required items in nested array alternatives", () => { ); }); -it("detects item schemas Gemini declarations reject in schema positions only", () => { - // The #5201 report: `properties[functions].items.items: field predicate - // failed: $type == Type.ARRAY` for an any-value element type list. - expect( - strictGatewayItemSchemaErrors( - { - type: "object", - properties: { - functions: { - type: "array", - items: { - type: ["null", "boolean", "object", "array", "number", "string"], - items: {}, - }, - }, - actions: { - type: "array", - items: { oneOf: [{ type: "object" }, { type: "string" }] }, - }, - nested: { - type: "array", - items: { type: "array", items: { type: "string" } }, - }, - }, - default: { items: {} }, - examples: [{ items: {} }], - }, - "injected", - ), - ).toEqual([ - 'injected.properties.functions.items.items is beside type ["null","boolean","object","array","number","string"]', - "injected.properties.actions.items is an untyped union", - ]); -}); - it("preserves arbitrary nested JSON values at the converted SDK boundary", async () => { const contract = { ...toolContract("binary_session"), @@ -377,11 +342,20 @@ it("states dependency keywords as prose in both catalog profiles", async () => { expect(sqlite?.inputSchema.description).toContain( "When row_limit is provided, also provide table.", ); - for (const tools of [fullTools, compactTools]) - expect( - tools.find(({ name }) => name === "inspect_sqlite_database")?.inputSchema - .description, - ).toContain("When schema_scope is selected_table, also provide table."); + expect(sqlite?.inputSchema.description).toContain( + "When schema_scope is selected_table, also provide table.", + ); + // Compact moves the sentence to the tool description, which Codex keeps + // when it strips descriptions inside a schema above 5,000 bytes. + const compactSqlite = compactTools.find( + ({ name }) => name === "inspect_sqlite_database", + ); + expect(compactSqlite?.description).toContain( + "When schema_scope is selected_table, also provide table.", + ); + expect(JSON.stringify(compactSqlite?.inputSchema)).not.toContain( + "also provide", + ); }); it("retains parameter types and avoids client-side reference expansion", async () => { diff --git a/tests/boundary/mcp/deferredRuntime.test.ts b/tests/boundary/mcp/deferredRuntime.test.ts index 31c0034f76..cfc19e47bf 100644 --- a/tests/boundary/mcp/deferredRuntime.test.ts +++ b/tests/boundary/mcp/deferredRuntime.test.ts @@ -125,7 +125,11 @@ it.each([ ), ).not.toContain('"default"'); const call = async (name: string, args: Record) => { - if (environment.REA_MCP_OUTPUT_SCHEMA_PROFILE === "off") { + // Without a listed output schema, call with the listed definition. + if ( + !("REA_MCP_OUTPUT_SCHEMA_PROFILE" in environment) || + environment.REA_MCP_OUTPUT_SCHEMA_PROFILE === "off" + ) { const definition = (await client.listTools()).tools.find( (tool) => tool.name === name, ); @@ -210,8 +214,13 @@ it.each([ eager: false, }, { - name: "output-off eager", - environment: { REA_MCP_OUTPUT_SCHEMA_PROFILE: "off" }, + name: "output-full deferred", + environment: { REA_MCP_OUTPUT_SCHEMA_PROFILE: "full" }, + eager: false, + }, + { + name: "output-compact eager", + environment: { REA_MCP_OUTPUT_SCHEMA_PROFILE: "compact" }, eager: true, }, ])( @@ -299,6 +308,7 @@ it.each([ ); const { client, stderr } = await connect(mode, { REA_MCP_INPUT_SCHEMA_PROFILE: profile, + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", }); const before = await client.listTools(); // Compare independently composed canonical SDK registrations as well as @@ -306,6 +316,7 @@ it.each([ // with the same inventory, without target analysis or provider acquisition. const eager = await connect(mode, { REA_MCP_INPUT_SCHEMA_PROFILE: profile, + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", REA_MCP_TOOL_FAMILIES: TOOL_FAMILY_DEFINITIONS.map(({ id }) => id).join( ",", ), @@ -445,6 +456,7 @@ it.each(["legacy", "auto"] as const)( it("composes a scoped registration instead of serving the recorded catalog", async () => { const { client, stderr } = await connect("legacy", { REA_MCP_TOOL_FAMILIES: "session", + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", }); const advertised = (await client.listTools()).tools .map(({ name }) => name) @@ -479,7 +491,11 @@ globalThis.Date = class extends NativeDate { } }; `)}`; - const { client } = await connect("legacy", {}, [`--import=${hook}`]); + const { client } = await connect( + "legacy", + { REA_MCP_OUTPUT_SCHEMA_PROFILE: "full" }, + [`--import=${hook}`], + ); await client.listTools(); await writeFile(clock, "2021-01-01T00:00:00.000Z"); const first = await artifactMcpCall(client, { @@ -800,38 +816,39 @@ console.log("LATE_PUBLICATION_REFUSED"); expect(child.stderr).toBe(""); }); -it.each([{}, { REA_MCP_OUTPUT_SCHEMA_PROFILE: "off" }])( - "preserves the configured catalog profile %j", - async (environment) => { - const { client } = await connect("legacy", environment); - const before = await client.listTools(); - const catalog = JSON.parse(await readFile("dist/mcp-catalog.json", "utf8")); - expect(before.tools).toHaveLength(catalog.tools.length); - const expected = - environment.REA_MCP_OUTPUT_SCHEMA_PROFILE === "off" - ? catalog.tools.map( - ({ outputSchema: _output, ...tool }: Record) => - tool, - ) - : catalog.tools.map((tool: { name: string }) => ({ - ...tool, - outputSchema: GENERATED_MCP_TOOL_CATALOG.find( - ({ name }) => name === tool.name, - )?.outputSchema, - })); - expect(before.tools).toEqual(expected); - const toolDefinition = before.tools.find( - ({ name }) => name === "list_unknowns", - ); - if (toolDefinition === undefined) throw new Error("Missing list_unknowns"); - const result = await client.callTool( - { name: "list_unknowns", arguments: {} }, - { toolDefinition }, - ); - expect(result.isError).not.toBe(true); - expect(await client.listTools()).toEqual(before); - }, -); +it.each([ + {}, + { REA_MCP_OUTPUT_SCHEMA_PROFILE: "off" }, + { REA_MCP_OUTPUT_SCHEMA_PROFILE: "full" }, +])("preserves the configured catalog profile %j", async (environment) => { + const { client } = await connect("legacy", environment); + const before = await client.listTools(); + const catalog = JSON.parse(await readFile("dist/mcp-catalog.json", "utf8")); + expect(before.tools).toHaveLength(catalog.tools.length); + // Output schemas are omitted unless `full` is selected. + const expected = + environment.REA_MCP_OUTPUT_SCHEMA_PROFILE !== "full" + ? catalog.tools.map( + ({ outputSchema: _output, ...tool }: Record) => tool, + ) + : catalog.tools.map((tool: { name: string }) => ({ + ...tool, + outputSchema: GENERATED_MCP_TOOL_CATALOG.find( + ({ name }) => name === tool.name, + )?.outputSchema, + })); + expect(before.tools).toEqual(expected); + const toolDefinition = before.tools.find( + ({ name }) => name === "list_unknowns", + ); + if (toolDefinition === undefined) throw new Error("Missing list_unknowns"); + const result = await client.callTool( + { name: "list_unknowns", arguments: {} }, + { toolDefinition }, + ); + expect(result.isError).not.toBe(true); + expect(await client.listTools()).toEqual(before); +}); it("runs the real static artifact workflow after deferred initialization", async () => { const { client } = await connect("legacy"); @@ -1181,7 +1198,10 @@ try { it("distinguishes an exhausted production discovery page from an unmatched query", async () => { const { client } = await connect( "auto", - { REA_MCP_TOOL_CATALOG_PROFILE: "hot-cold" }, + { + REA_MCP_TOOL_CATALOG_PROFILE: "hot-cold", + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", + }, ["--max-old-space-size=1024"], ); const discover = (arguments_: Record) => diff --git a/tests/boundary/mcp/evidenceInputContractsMcp.test.ts b/tests/boundary/mcp/evidenceInputContractsMcp.test.ts index 4e41ba0125..47013815af 100644 --- a/tests/boundary/mcp/evidenceInputContractsMcp.test.ts +++ b/tests/boundary/mcp/evidenceInputContractsMcp.test.ts @@ -1,15 +1,13 @@ -import { McpServer } from "@modelcontextprotocol/server"; +import { STDIO_DEFAULT_MAX_BUFFER_SIZE } from "@modelcontextprotocol/server"; import { Ajv2020 } from "ajv/dist/2020.js"; import { describe, expect, expectTypeOf, it } from "vitest"; import { COMPACT_INPUT_SCHEMA_BUDGET_BYTES } from "../../../src/config/mcpInputSchemaProfile.js"; -import { presentAdvertisedInputSchema } from "../../../src/contracts/compactInputSchema.js"; import { SESSION_TOOL_CONTRACTS } from "../../../src/contracts/sessionToolContracts.js"; import { toolContract } from "../../../src/contracts/toolContracts.js"; -import { - advertisedInputJsonSchema, - transformAdvertisedInputJsonSchema, -} from "../../../src/contracts/toolSchemaMetadata.js"; +import { advertisedInputJsonSchema } from "../../../src/contracts/toolSchemaMetadata.js"; +import { EvidenceMcpServer } from "../../../src/server/EvidenceMcpServer.js"; +import { createToolResultDelivery } from "../../../src/server/toolResult.js"; import { toolRegistrationOptions } from "../../../src/server/toolRegistrationOptions.js"; import { advertisedSchemaErrors } from "../../fixtures/mcpAdvertisedSchemaValidity.js"; import { mcpTest } from "../../support/mcp/mcpFixture.js"; @@ -69,21 +67,20 @@ describe("Evidence input SDK boundaries", () => { "advertises valid reference inputs after actual SDK conversion (%s)", async (profile, { mcp }) => { const contract = toolContract("build_call_path"); - const registration = toolRegistrationOptions(contract); - const inputSchema = + const server = new EvidenceMcpServer( + { name: "evidence-contract", version: "1" }, + { capabilities: {} }, + undefined, + createToolResultDelivery(STDIO_DEFAULT_MAX_BUFFER_SIZE), profile === "compact" - ? transformAdvertisedInputJsonSchema( - registration.inputSchema, - (schema) => - presentAdvertisedInputSchema(schema, { - budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES, - }), - ) - : registration.inputSchema; - const server = new McpServer({ name: "evidence-contract", version: "1" }); - server.registerTool(contract.name, { inputSchema }, async () => ({ - content: [], - })); + ? { budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES } + : undefined, + ); + server.registerTool( + contract.name, + toolRegistrationOptions(contract), + async () => ({ content: [] }), + ); const client = await mcp.connect(server, { name: "evidence-contract", version: "1", @@ -92,10 +89,13 @@ describe("Evidence input SDK boundaries", () => { expect(advertisedSchemaErrors(tools)).toEqual([]); const tool = tools.find(({ name }) => name === contract.name); if (tool === undefined) throw new Error("named contract not advertised"); - expect(tool.inputSchema.description).toContain( + // Compact input guidance is part of the tool description. + const guidance = + profile === "compact" ? tool.description : tool.inputSchema.description; + expect(guidance).toContain( '{"kind":"retained-evidence","evidence_id":"ev_..."}', ); - expect(tool.inputSchema.description).toContain("parent_evidence_id"); + expect(guidance).toContain("parent_evidence_id"); const validate = new Ajv2020({ strict: false, validateFormats: false, diff --git a/tests/boundary/mcp/fridaMcp.test.ts b/tests/boundary/mcp/fridaMcp.test.ts index 49ec795df4..3c5ef01d45 100644 --- a/tests/boundary/mcp/fridaMcp.test.ts +++ b/tests/boundary/mcp/fridaMcp.test.ts @@ -363,7 +363,8 @@ const expectFridaSelectorGuidance = async ( (tool) => tool.name === "list_frida_processes", ); if (processTool === undefined) throw new Error("Missing Frida process tool"); - expect(processTool.inputSchema.description).toMatch( + // Compact input guidance is part of the tool description. + expect(processTool.description).toMatch( /exactly one of device_id or remote/u, ); expect(ajv.validateSchema(processTool.inputSchema)).toBe(true); diff --git a/tests/boundary/mcp/javascriptSourceSelection.test.ts b/tests/boundary/mcp/javascriptSourceSelection.test.ts index cbb2c34896..27665d6b19 100644 --- a/tests/boundary/mcp/javascriptSourceSelection.test.ts +++ b/tests/boundary/mcp/javascriptSourceSelection.test.ts @@ -42,6 +42,7 @@ const createSelectionClient = () => { NODE_OPTIONS: "--max-old-space-size=512 --max-semi-space-size=8", // Exact advertised constraints require the complete schema profile. REA_MCP_INPUT_SCHEMA_PROFILE: "full", + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", }, stderr: "pipe", }); diff --git a/tests/boundary/mcp/mcpDoctor.test.ts b/tests/boundary/mcp/mcpDoctor.test.ts index 2123232db1..d98e31ed6c 100644 --- a/tests/boundary/mcp/mcpDoctor.test.ts +++ b/tests/boundary/mcp/mcpDoctor.test.ts @@ -210,12 +210,23 @@ const catalogProfiles = [ }, ]; +const withoutOutputProfile = ( + environment: NodeJS.ProcessEnv, +): NodeJS.ProcessEnv => { + const { REA_MCP_OUTPUT_SCHEMA_PROFILE: _profile, ...rest } = environment; + return rest; +}; + it("recognizes a hot/cold initial catalog and its inventory identity", async () => { const result = await runProductionMcpDoctor({ command: process.execPath, args: [resolve("scripts/rea.mjs"), "mcp"], cwd: process.cwd(), - environment: { ...process.env, REA_MCP_TOOL_CATALOG_PROFILE: "hot-cold" }, + // The product default output profile, not the test suite's. + environment: { + ...withoutOutputProfile(process.env), + REA_MCP_TOOL_CATALOG_PROFILE: "hot-cold", + }, }); expect(result.healthy).toBe(true); expect(result.inventory?.tools).toMatchObject({ @@ -229,7 +240,7 @@ it("recognizes a hot/cold initial catalog and its inventory identity", async () .combined_sha256, ); const environment = { - ...process.env, + ...withoutOutputProfile(process.env), REA_MCP_TOOL_CATALOG_PROFILE: "hot-cold", }; const identity = progressiveCatalogIdentity(DEFAULT_CATALOG_IDENTITY); @@ -596,108 +607,124 @@ describe("production MCP doctor output", () => { }); describe("production MCP doctor probe validation", () => { - it("reports schema-invalid probes and mismatched inventory from the child", async () => { - const root = await createTestTempDirectory("rea-mcp-doctor-reply-"); - const snapshotPath = join(root, "reply.json"); - const environment = { PATH: process.env.PATH ?? "" }; - const client = new Client( - { name: "doctor-reply-capture", version: "1" }, - { jsonSchemaValidator: schemaValidation }, - ); - const transport = new StdioClientTransport({ - command: process.execPath, - args: [resolve("scripts/rea.mjs"), "mcp"], - cwd: process.cwd(), - env: environment, - stderr: "pipe", - }); - let removedToolName = ""; - transport.stderr?.on("data", () => undefined); - try { - await client.connect(transport); - const tools = await client.listTools(); - removedToolName = - tools.tools.find(({ name }) => name !== "binary_session")?.name ?? ""; - if (removedToolName === "") - throw new Error("Expected a second production tool"); - const prompts = await client.listPrompts(); - const toolDefinition = tools.tools.find( - ({ name }) => name === "binary_session", + // The default lists no output schemas and `compact` lists widened ones; the + // doctor validates the reply against the canonical contract under each. + it.each([undefined, "compact", "full"] as const)( + "reports schema-invalid probes and mismatched inventory from the child (output profile %s)", + async (outputProfile) => { + const root = await createTestTempDirectory("rea-mcp-doctor-reply-"); + const snapshotPath = join(root, "reply.json"); + const environment = { + PATH: process.env.PATH ?? "", + ...(outputProfile === undefined + ? {} + : { REA_MCP_OUTPUT_SCHEMA_PROFILE: outputProfile }), + }; + const client = new Client( + { name: "doctor-reply-capture", version: "1" }, + { jsonSchemaValidator: schemaValidation }, ); - if (toolDefinition === undefined) - throw new Error("Production binary_session contract is missing"); - const reply = await client.callTool( - { name: "binary_session", arguments: {} }, - { toolDefinition }, - ); - await writeFile( - snapshotPath, - JSON.stringify({ - serverInfo: client.getServerVersion(), - protocolVersion: client.getNegotiatedProtocolVersion(), - tools, - prompts, - reply, - }), - ); - } finally { - await client.close(); - await transport.close(); - } - // Replay real production metadata through a synthetic stdio producer. - for (const mode of ["valid", "invalid", "inventory"]) { - const result = await runProductionMcpDoctor({ + const transport = new StdioClientTransport({ command: process.execPath, - args: [ - resolve("tests/fixtures/mcpDoctorReply.mjs"), - snapshotPath, - mode, - ], + args: [resolve("scripts/rea.mjs"), "mcp"], cwd: process.cwd(), - environment, + env: environment, + stderr: "pipe", }); - expect(result.healthy).toBe(mode === "valid"); - expect(result.inventory?.tools.observed).toBe( - FULL_CATALOG_IDENTITY.counts.mcp_tools + (mode === "inventory" ? 1 : 0), - ); - expect(result.inventory?.prompts.observed).toBe( - FULL_CATALOG_IDENTITY.counts.mcp_prompts, - ); - expect(result.checks).toContainEqual({ - name: "initialize", - ok: true, - detail: expect.any(String), - }); - expect(result.checks).toContainEqual({ - name: "server-identity", - ok: mode !== "invalid", - detail: - mode === "invalid" - ? expect.stringMatching(/^identity not observed: .*output schema/iu) - : expect.stringMatching(/^package=.+; catalog=.+$/u), - }); - if (mode === "invalid") - expect(result.checks).toContainEqual({ - name: "request-flow", - ok: false, - detail: expect.stringMatching(/output schema/iu), - }); - if (mode === "inventory") { - expect(result.inventory?.tools).toMatchObject({ - expected: FULL_CATALOG_IDENTITY.counts.mcp_tools, - observed: FULL_CATALOG_IDENTITY.counts.mcp_tools + 1, - missing: [removedToolName], - unexpected: ["doctor_unexpected_fixture"], - duplicates: ["doctor_unexpected_fixture"], - }); - expect(result.checks).toContainEqual({ - name: "inventory:tools", - ok: false, - detail: expect.stringContaining("doctor_unexpected_fixture"), - }); + let removedToolName = ""; + transport.stderr?.on("data", () => undefined); + try { + await client.connect(transport); + const tools = await client.listTools(); + expect( + tools.tools.some(({ outputSchema }) => outputSchema !== undefined), + ).toBe(outputProfile !== undefined); + removedToolName = + tools.tools.find(({ name }) => name !== "binary_session")?.name ?? ""; + if (removedToolName === "") + throw new Error("Expected a second production tool"); + const prompts = await client.listPrompts(); + const toolDefinition = tools.tools.find( + ({ name }) => name === "binary_session", + ); + if (toolDefinition === undefined) + throw new Error("Production binary_session contract is missing"); + const reply = await client.callTool( + { name: "binary_session", arguments: {} }, + { toolDefinition }, + ); + await writeFile( + snapshotPath, + JSON.stringify({ + serverInfo: client.getServerVersion(), + protocolVersion: client.getNegotiatedProtocolVersion(), + tools, + prompts, + reply, + }), + ); + } finally { + await client.close(); + await transport.close(); } - } - }); + // Replay real production metadata through a synthetic stdio producer. + for (const mode of ["valid", "invalid", "inventory"]) { + const result = await runProductionMcpDoctor({ + command: process.execPath, + args: [ + resolve("tests/fixtures/mcpDoctorReply.mjs"), + snapshotPath, + mode, + ], + cwd: process.cwd(), + environment, + }); + expect(result.healthy).toBe(mode === "valid"); + expect(result.inventory?.tools.observed).toBe( + FULL_CATALOG_IDENTITY.counts.mcp_tools + + (mode === "inventory" ? 1 : 0), + ); + expect(result.inventory?.prompts.observed).toBe( + FULL_CATALOG_IDENTITY.counts.mcp_prompts, + ); + expect(result.checks).toContainEqual({ + name: "initialize", + ok: true, + detail: expect.any(String), + }); + expect(result.checks).toContainEqual({ + name: "server-identity", + ok: mode !== "invalid", + detail: + mode === "invalid" + ? expect.stringMatching( + /^identity not observed: .*output schema/iu, + ) + : expect.stringMatching(/^package=.+; catalog=.+$/u), + }); + if (mode === "invalid") + expect(result.checks).toContainEqual({ + name: "request-flow", + ok: false, + detail: expect.stringMatching(/output schema/iu), + }); + if (mode === "inventory") { + expect(result.inventory?.tools).toMatchObject({ + expected: FULL_CATALOG_IDENTITY.counts.mcp_tools, + observed: FULL_CATALOG_IDENTITY.counts.mcp_tools + 1, + missing: [removedToolName], + unexpected: ["doctor_unexpected_fixture"], + duplicates: ["doctor_unexpected_fixture"], + }); + expect(result.checks).toContainEqual({ + name: "inventory:tools", + ok: false, + detail: expect.stringContaining("doctor_unexpected_fixture"), + }); + } + } + }, + ); }); describe("production MCP doctor child lifecycle", () => { diff --git a/tests/boundary/mcp/outputSchemaProfile.test.ts b/tests/boundary/mcp/outputSchemaProfile.test.ts index 943efdd533..14868fd552 100644 --- a/tests/boundary/mcp/outputSchemaProfile.test.ts +++ b/tests/boundary/mcp/outputSchemaProfile.test.ts @@ -1,13 +1,19 @@ import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; import { Ajv2020 } from "ajv/dist/2020.js"; import { z } from "zod"; -import { expect, it } from "vitest"; +import { expect, it, onTestFinished } from "vitest"; import { TOOL_CONTRACTS, toolContract, } from "../../../src/contracts/toolContracts.js"; -import { COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES } from "../../../src/config/mcpOutputSchemaProfile.js"; +import { + COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES, + resolveMcpOutputSchemaProfile, +} from "../../../src/config/mcpOutputSchemaProfile.js"; +import { createServer as createProductionServer } from "../../../src/server/createServer.js"; +import { silentLogger } from "../../../src/logger.js"; +import { createTestBinarySession } from "../../fixtures/binarySession.js"; import { COMPACT_INPUT_SCHEMA_BUDGET_BYTES } from "../../../src/config/mcpInputSchemaProfile.js"; import type { CompactInputSchemaPresentation } from "../../../src/contracts/compactInputSchema.js"; import { EvidenceMcpServer } from "../../../src/server/EvidenceMcpServer.js"; @@ -370,3 +376,40 @@ it("rejects canonical-invalid output that the reduced advertisement accepts", as await server.close(); } }); + +it("omits output schemas by default and still delivers validated structured results", async () => { + expect(resolveMcpOutputSchemaProfile(undefined)).toBe("off"); + const session = createTestBinarySession(() => { + throw new Error("No deep provider may start for a catalog listing"); + }); + // An environment without the setting selects the product default. + const server = createProductionServer( + { kind: "session", session }, + { environment: { HOME: process.env.HOME }, logger: silentLogger }, + ); + const client = new Client({ name: "default-output-profile", version: "1" }); + onTestFinished(async () => { + await client.close(); + await server.close(); + await session.close(); + }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + await server.connect(serverTransport); + await client.connect(clientTransport); + const { tools } = await client.listTools(); + expect(tools).toHaveLength(TOOL_CONTRACTS.length); + expect( + tools.filter(({ outputSchema }) => outputSchema !== undefined), + ).toEqual([]); + // The SDK client accepts structured content from a tool without a listed + // output schema; REA validated it against the canonical contract. + const result = await client.callTool({ + name: "list_unknowns", + arguments: {}, + }); + expect(result.isError).not.toBe(true); + expect(result.structuredContent).toEqual({ + result: { items: [], total: 0, offset: 0, next_offset: null }, + }); +}); diff --git a/tests/boundary/mcp/packetCaptureMcp.test.ts b/tests/boundary/mcp/packetCaptureMcp.test.ts index 9779f16bc7..7aae4197cc 100644 --- a/tests/boundary/mcp/packetCaptureMcp.test.ts +++ b/tests/boundary/mcp/packetCaptureMcp.test.ts @@ -13,7 +13,7 @@ it.each(["inspect_packet_capture", "inspect_packet_fields"])( "advertises valid target-free schemas and missing-tool diagnostics: %s", async (name) => { const harness = await createApplicationMcpHarness({ - serverOptions: { environment: {} }, + serverOptions: { environment: { REA_MCP_OUTPUT_SCHEMA_PROFILE: "full" } }, }); onTestFinished(harness.close); const tool = (await harness.client.listTools()).tools.find( diff --git a/tests/boundary/mcp/serverIdentity.test.ts b/tests/boundary/mcp/serverIdentity.test.ts index 83921ce14f..34739411ba 100644 --- a/tests/boundary/mcp/serverIdentity.test.ts +++ b/tests/boundary/mcp/serverIdentity.test.ts @@ -151,7 +151,7 @@ describe("server and catalog identity", () => { it("binds output schema profile and canonical output contracts into identity", () => { const profiles = createCatalogDigestProfiles(); expect(FULL_CATALOG_IDENTITY.digests).toEqual(profiles.full.full); - expect(DEFAULT_CATALOG_IDENTITY.digests).toEqual(profiles.compact.full); + expect(DEFAULT_CATALOG_IDENTITY.digests).toEqual(profiles.compact.off); const changedOutput = TOOL_CONTRACTS.map((contract, index) => index === 0 ? { diff --git a/tests/boundary/mcp/serverWorkflowMcp.test.ts b/tests/boundary/mcp/serverWorkflowMcp.test.ts index e7b16d33bc..39637e0f94 100644 --- a/tests/boundary/mcp/serverWorkflowMcp.test.ts +++ b/tests/boundary/mcp/serverWorkflowMcp.test.ts @@ -11,6 +11,12 @@ import type { } from "../../../src/application/AnalysisProvider.js"; import { observed as ok } from "../../fixtures/analysisExecution.js"; import { createServer } from "../../../src/server/createServer.js"; +import { COMPACT_INPUT_SCHEMA_BUDGET_BYTES } from "../../../src/config/mcpInputSchemaProfile.js"; +import { + describeAdvertisedTool, + presentAdvertisedInput, +} from "../../../src/contracts/compactInputSchema.js"; +import { advertisedInputJsonSchema } from "../../../src/contracts/toolSchemaMetadata.js"; import { TOOL_CONTRACTS, toolContract, @@ -163,7 +169,17 @@ it("advertises the complete currently available inventory with a session", async const contract = contracts.get(tool.name); expect(contract, tool.name).toBeDefined(); expect(tool.title, tool.name).toBe(contract?.title); - expect(tool.description, tool.name).toBe(contract?.description); + // The default compact input profile moves input guidance from the schema + // to the end of the contract description. + if (contract === undefined) continue; + expect(tool.description, tool.name).toBe( + describeAdvertisedTool( + contract.description, + presentAdvertisedInput(advertisedInputJsonSchema(contract), { + budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES, + }).guidance, + ), + ); expect(tool.annotations, tool.name).toEqual(contract?.annotations); expect(tool.inputSchema, tool.name).toBeDefined(); expect(tool.outputSchema, tool.name).toBeDefined(); diff --git a/tests/boundary/mcp/signalRecordingMcp.test.ts b/tests/boundary/mcp/signalRecordingMcp.test.ts index 0ef153f355..a21ad6e001 100644 --- a/tests/boundary/mcp/signalRecordingMcp.test.ts +++ b/tests/boundary/mcp/signalRecordingMcp.test.ts @@ -13,7 +13,7 @@ it.each(["decode_signal_recording", "inspect_signal_decoders"])( "advertises valid target-free schemas and missing-tool diagnostics: %s", async (name) => { const harness = await createApplicationMcpHarness({ - serverOptions: { environment: {} }, + serverOptions: { environment: { REA_MCP_OUTPUT_SCHEMA_PROFILE: "full" } }, }); onTestFinished(harness.close); const tool = (await harness.client.listTools()).tools.find( diff --git a/tests/boundary/mcp/streamableHttpMcp.test.ts b/tests/boundary/mcp/streamableHttpMcp.test.ts index 78db023402..4a878171a2 100644 --- a/tests/boundary/mcp/streamableHttpMcp.test.ts +++ b/tests/boundary/mcp/streamableHttpMcp.test.ts @@ -51,7 +51,11 @@ describe("rea mcp --http", () => { const started = await startListeningProcess( processes, ["mcp", "--http", "--port", "0"], - { REA_MCP_TOOL_CATALOG_PROFILE: "hot-cold" }, + // Sessions validate advertised output schemas, which default to off. + { + REA_MCP_TOOL_CATALOG_PROFILE: "hot-cold", + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", + }, ); const first = await openSession(started.url, "hot-cold-first"); const initialNames = (await first.client.listTools()).tools.map( @@ -98,7 +102,11 @@ describe("rea mcp --http", () => { processTest( "serves initialize, tools/list, a tool call and DELETE, then keeps serving later clients", async ({ processes }) => { - const started = await startMcpHttpProcess(processes); + const started = await startListeningProcess( + processes, + ["mcp", "--http", "--port", "0"], + { REA_MCP_OUTPUT_SCHEMA_PROFILE: "full" }, + ); expect(started.url.hostname).toBe("127.0.0.1"); expect(started.url.pathname).toBe("/mcp"); let stdout = ""; diff --git a/tests/boundary/mcp/wasmArtifactMcp.test.ts b/tests/boundary/mcp/wasmArtifactMcp.test.ts index 9ea21aad57..78179274fd 100644 --- a/tests/boundary/mcp/wasmArtifactMcp.test.ts +++ b/tests/boundary/mcp/wasmArtifactMcp.test.ts @@ -106,7 +106,7 @@ it.each(["bin", "hop"])( const harness = await createApplicationMcpHarness({ schemaValidator: schemaValidation, session, - serverOptions: { environment: {} }, + serverOptions: { environment: { REA_MCP_OUTPUT_SCHEMA_PROFILE: "full" } }, }); onTestFinished(harness.close); const { client } = harness; @@ -615,7 +615,7 @@ it("rejects inconsistent decoded-WAT digests in portable Evidence without a prov const value = wasmArtifactFixture(); const harness = await createApplicationMcpHarness({ schemaValidator: schemaValidation, - serverOptions: { environment: {} }, + serverOptions: { environment: { REA_MCP_OUTPUT_SCHEMA_PROFILE: "full" } }, }); onTestFinished(harness.close); const inconsistent = createEvidence( @@ -658,7 +658,7 @@ it("checks WASM semantic identity while preserving rebound display paths", async const value = wasmArtifactFixture(); const harness = await createApplicationMcpHarness({ schemaValidator: schemaValidation, - serverOptions: { environment: {} }, + serverOptions: { environment: { REA_MCP_OUTPUT_SCHEMA_PROFILE: "full" } }, }); onTestFinished(harness.close); const views = [ diff --git a/tests/boundary/process/androidMappingMcp.test.ts b/tests/boundary/process/androidMappingMcp.test.ts index a073fa01f4..67b6af712d 100644 --- a/tests/boundary/process/androidMappingMcp.test.ts +++ b/tests/boundary/process/androidMappingMcp.test.ts @@ -31,7 +31,10 @@ const connect = async () => { { kind: "session", session }, { androidAnalysis: fixture.provider, - environment: { REA_MCP_TOOL_FAMILIES: "android,application,session" }, + environment: { + REA_MCP_TOOL_FAMILIES: "android,application,session", + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", + }, }, ); const client = new Client({ diff --git a/tests/conformance/verifiers/clientSchemaItemConstraints.test.ts b/tests/conformance/verifiers/clientSchemaItemConstraints.test.ts index 75a81cd695..0db544ff6f 100644 --- a/tests/conformance/verifiers/clientSchemaItemConstraints.test.ts +++ b/tests/conformance/verifiers/clientSchemaItemConstraints.test.ts @@ -1,8 +1,5 @@ import { expect, it } from "vitest"; -import { - arrayItemSchemaError, - strictGatewayItemSchemaErrors, -} from "../../fixtures/mcpClientSchemaConstraints.js"; +import { arrayItemSchemaError } from "../../fixtures/mcpClientSchemaConstraints.js"; it("checks client array requirements when a reference selects literal-held schemas", () => { const literal = { examples: [{ type: "array" }] }; @@ -20,38 +17,3 @@ it("checks client array requirements when a reference selects literal-held schem ), ).toBeUndefined(); }); - -it.each([ - { - selected: { type: "object", items: { type: "string" } }, - error: 'tool.examples[0].items is beside type "object"', - }, - { - selected: { - type: "array", - items: { anyOf: [{ type: "string" }, { type: "number" }] }, - }, - error: "tool.examples[0].items is an untyped union", - }, -])( - "checks referenced gateway item constraints: $error", - ({ selected, error }) => { - const literal = { examples: [selected] }; - expect(strictGatewayItemSchemaErrors(literal, "tool")).toEqual([]); - expect( - strictGatewayItemSchemaErrors( - { ...literal, $ref: "#/examples/0" }, - "tool", - ), - ).toEqual([error]); - expect( - strictGatewayItemSchemaErrors( - { - $ref: "#/examples/0", - examples: [{ type: "array", items: { type: "string" } }], - }, - "tool", - ), - ).toEqual([]); - }, -); diff --git a/tests/fixtures/artifactMcpVerifierProbe.mjs b/tests/fixtures/artifactMcpVerifierProbe.mjs index 5217e0f68d..ca2c176a66 100644 --- a/tests/fixtures/artifactMcpVerifierProbe.mjs +++ b/tests/fixtures/artifactMcpVerifierProbe.mjs @@ -4,6 +4,7 @@ import { Client } from "@modelcontextprotocol/client"; import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; import * as artifact from "../../scripts/lib/artifact-e2e.mjs"; +import { schemaVerifierEnvironment } from "../../scripts/lib/mcp-verification-environment.mjs"; import { withVerifierCleanup } from "../../scripts/lib/verifier-cleanup.mjs"; const [mode, target] = process.argv.slice(2); @@ -144,10 +145,10 @@ async function standaloneConnections() { fileURLToPath(new URL("../../scripts/rea.mjs", import.meta.url)), "mcp", ], - env: { + env: schemaVerifierEnvironment({ PATH: process.env.PATH ?? "/usr/bin:/bin", REA_LOG_LEVEL: "silent", - }, + }), stderr: "pipe", }); await withVerifierCleanup(async () => { diff --git a/tests/fixtures/localToolsMcp.ts b/tests/fixtures/localToolsMcp.ts index fc785cf473..9d1f0664fd 100644 --- a/tests/fixtures/localToolsMcp.ts +++ b/tests/fixtures/localToolsMcp.ts @@ -20,6 +20,8 @@ export const localToolsMcpEnvironment = ( REA_LOG_LEVEL: "silent", HOPPER_LAUNCHER_PATH: "/rea-unconfigured-deep-provider/hopper", GHIDRA_INSTALL_DIR: process.execPath, + // Validate each result against its complete advertised output contract. + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", ...environment, }); diff --git a/tests/fixtures/mcpClientSchemaConstraints.ts b/tests/fixtures/mcpClientSchemaConstraints.ts index 9c1fb262f1..6f3ff005b8 100644 --- a/tests/fixtures/mcpClientSchemaConstraints.ts +++ b/tests/fixtures/mcpClientSchemaConstraints.ts @@ -1,9 +1,6 @@ import { geminiUnsupportedLiteral } from "../../scripts/lib/gemini-schema-literals.mjs"; import { clientSchemaPositions } from "../../scripts/lib/client-schema-positions.mjs"; -const isRecord = (value: unknown): value is Record => - typeof value === "object" && value !== null && !Array.isArray(value); - /** VS Code Copilot rejects falsy `items` for any type that admits arrays. */ export const arrayItemSchemaError = ( schema: unknown, @@ -38,34 +35,3 @@ export const unsupportedInputLiteralSchemaError = ( : "contains an empty string"; return `${path}${finding.path.slice(1)} ${reason}`; }; - -/** - * Gemini function declarations accept `items` only on a schema whose type is - * the single `array` type (`items: field predicate failed: $type == - * Type.ARRAY`), and strict gateways drop untyped item unions. Return every - * advertised position with either shape so a catalog test names them all. - */ -export const strictGatewayItemSchemaErrors = ( - schema: unknown, - path = "$", -): readonly string[] => { - const errors: string[] = []; - for (const position of clientSchemaPositions(schema)) { - if (position.kind !== "schema") continue; - const value = position.schema; - const selectedPath = `${path}${position.path.slice(1)}`; - if (value.items !== undefined && value.type !== "array") - errors.push( - `${selectedPath}.items is beside type ${JSON.stringify(value.type)}`, - ); - const items = value.items; - if ( - value.type === "array" && - isRecord(items) && - items.type === undefined && - (Array.isArray(items.oneOf) || Array.isArray(items.anyOf)) - ) - errors.push(`${selectedPath}.items is an untyped union`); - } - return errors; -}; diff --git a/tests/support/analysis-efficiency/android-jni.mjs b/tests/support/analysis-efficiency/android-jni.mjs index 5b631a4f66..28d4bbafe7 100644 --- a/tests/support/analysis-efficiency/android-jni.mjs +++ b/tests/support/analysis-efficiency/android-jni.mjs @@ -737,6 +737,8 @@ const environment = { ? "android,session" : "android", REA_MCP_INPUT_SCHEMA_PROFILE: "compact", + // The run validates each result against its advertised output schema. + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", }; const transport = new StdioClientTransport({ command: process.execPath, diff --git a/tests/support/analysis-efficiency/android-mapping.mjs b/tests/support/analysis-efficiency/android-mapping.mjs index 16b3cf0b98..117045f087 100644 --- a/tests/support/analysis-efficiency/android-mapping.mjs +++ b/tests/support/analysis-efficiency/android-mapping.mjs @@ -71,6 +71,8 @@ const transport = new StdioClientTransport({ REA_LOG_LEVEL: "silent", REA_MCP_TOOL_FAMILIES: values.proof ? "android,session" : "android", REA_MCP_INPUT_SCHEMA_PROFILE: "compact", + // The run validates each result against its advertised output schema. + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", }, stderr: "pipe", maxBufferSize: 16 * 1024 * 1024, diff --git a/vitest.config.ts b/vitest.config.ts index c73d8ead12..6d45b9f026 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -170,7 +170,13 @@ const projects = TEST_PROJECTS.map((project) => { export default defineConfig({ test: { - env: { TMPDIR: CANONICAL_TEMPORARY_DIRECTORY }, + env: { + TMPDIR: CANONICAL_TEMPORARY_DIRECTORY, + // In-process servers advertise complete output schemas so SDK clients + // validate every structured result. Tests of the default (`off`) set + // the profile, or pass an environment without it, explicitly. + REA_MCP_OUTPUT_SCHEMA_PROFILE: "full", + }, maxWorkers: MAX_TEST_WORKERS, projects, ...(process.env.REA_CI_TEST_HISTORY