mirror of
https://github.com/morluto/rea.git
synced 2026-10-26 17:35:11 +00:00
feat(mcp): support compact output schema advertisements (#7218)
Add opt-in bounded output-schema advertisements while preserving the full default, canonical server validation and complete structured content. Verify real stdio discovery under a 1 MiB receive cap, all 5,367 libcrypto export coordinates, complete Evidence parity, whole-task serialization estimates and resource bounds. Fixes #2512. Integrates #7054 by Akshad Jogi with original authorship retained. Co-authored-by: Akshad Jogi <akshadjogi@gmail.com>
This commit is contained in:
gensokyo-devandAkshad Jogi authored and GitHub committed
1 parent
e596a48251
commit
9085222ca1
22 files changed
+1831
-771
No files matched your search
@@ -96,6 +96,12 @@ complete advertised field guidance and nested validation details. Both profiles
|
||||
keep the full tool set and canonical server validation. See
|
||||
"Compact input schema profile" in the MCP contracts guide.
|
||||
|
||||
Complete output schemas are advertised by default. If your client caps the
|
||||
complete `tools/list` frame, set `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact` to
|
||||
bound each advertised output schema, or `off` to omit output schemas entirely.
|
||||
Both settings keep canonical server-side validation and complete structured
|
||||
results. See "Output schemas" in the MCP contracts guide.
|
||||
|
||||
### Ask your agent
|
||||
|
||||
```text
|
||||
|
||||
@@ -208,10 +208,13 @@ and Claude Code reports that autocompact is thrashing. To avoid this:
|
||||
The sizes above come from REA's own `tools/list`. A live Claude Code session
|
||||
against DeepSeek's API is unverified.
|
||||
|
||||
For a client whose stdio reader rejects the complete catalog frame, add
|
||||
`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "off"` to REA's `env` object and restart the
|
||||
client. This omits only advertised output schemas; REA still validates complete
|
||||
structured results against its canonical contracts. See [MCP output schema
|
||||
For a client whose stdio reader rejects the complete catalog frame, set
|
||||
`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "compact"` to bound every advertised output
|
||||
schema, or `"off"` to omit them, in REA's `env` object, then restart the client.
|
||||
Both keep canonical result validation and complete structured results; `compact`
|
||||
keeps a widened root shape and `off` advertises nothing. On catalogs large
|
||||
enough that full input schemas alone approach the frame limit, also keep the
|
||||
input profile at its `compact` default. See [MCP output schema
|
||||
profiles](mcp-contracts.md#output-schemas).
|
||||
|
||||
## Kimi Code
|
||||
|
||||
+29
-13
@@ -1367,21 +1367,37 @@ identity. Explicit schema IDs and relative external reference bases are
|
||||
preserved. This keeps every tool and validation rule in discovery and does not
|
||||
change the SDK's catalog invalidation or availability checks.
|
||||
|
||||
The server identity's tool-schema digest includes both the canonical output
|
||||
contracts and the selected output-schema profile. `full` and `off` therefore
|
||||
have distinct identities even though `off` keeps the same canonical output
|
||||
validation on the server.
|
||||
The server identity's tool-schema digest includes the selected output-schema
|
||||
profile and the advertised form that profile produces. `full`, `compact`, and
|
||||
`off` therefore have distinct identities. The `compact` digest covers the
|
||||
bounded schemas clients receive; `off` keeps the canonical output contracts in
|
||||
the digest while advertising nothing.
|
||||
|
||||
Some stdio hosts reject a complete `tools/list` frame before they load any
|
||||
tools. The opt-in `REA_MCP_OUTPUT_SCHEMA_PROFILE=off` setting omits `outputSchema`
|
||||
from the advertised catalog for hosts with a small frame limit (the measured
|
||||
pi-web-ui failure and catalog breakdown are in [#2512](https://github.com/morluto/rea/issues/2512)).
|
||||
The default `full` profile is unchanged. With `off`, REA validates successful
|
||||
structured results against the same named canonical tool contract before
|
||||
delivery; it still returns complete `structuredContent` and text results.
|
||||
Only client-side schema discovery and validation are omitted, so hosts using
|
||||
this profile should rely on REA's validation and retain their usual response
|
||||
size limits.
|
||||
tools. The opt-in `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact` setting bounds every
|
||||
advertised output schema to 4,096 bytes; `off` omits `outputSchema` entirely
|
||||
(the measured pi-web-ui failure and catalog breakdown are in
|
||||
[#2512](https://github.com/morluto/rea/issues/2512)). The default `full` profile
|
||||
is unchanged. Both settings keep the same named canonical tool contract for
|
||||
server-side result validation, and both still return complete
|
||||
`structuredContent` and text results.
|
||||
|
||||
`compact` keeps the complete validation structure when it fits the budget.
|
||||
Otherwise it advertises the root property shape, first-level reference-resolved
|
||||
types, first-level guidance, and the root's `required`/`additionalProperties`
|
||||
constraints, with nested validation detail elided. Local references are inlined
|
||||
within the budget, so the advertised form is self-contained and omits the
|
||||
canonical contract's content-bound `$id`. The projection only widens: a valid
|
||||
canonical result still satisfies the advertised schema, but a client validating
|
||||
against it can accept a result the canonical contract rejects. REA still
|
||||
validates every result before delivery. On catalogs large enough that full
|
||||
input schemas alone approach the frame limit, keep the input profile at its
|
||||
`compact` default as well; with full input schemas the advertisement is
|
||||
dominated by the input side.
|
||||
|
||||
Only client-side schema discovery and validation are reduced or omitted, so
|
||||
hosts using these profiles should rely on REA's validation and retain their
|
||||
usual response size limits.
|
||||
|
||||
### Function comparison limitations
|
||||
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
# Compact MCP output advertisements
|
||||
|
||||
## Question and scope
|
||||
|
||||
Can an SDK client with a 1 MiB receive buffer discover REA and finish an
|
||||
export investigation without weakening canonical result validation or losing
|
||||
Evidence? This verifies #2512 using the opt-in compact output advertisement
|
||||
from #7054. The default output profile remains `full`; inputs are `compact`.
|
||||
|
||||
The implementation was integrated onto main `18dc78fa43175b9628b45b731df06c9321c3f730`.
|
||||
Measured runtime revision: `650d483b98564d8dbe87d55ce6a77ec410608136`.
|
||||
Both profiles use that same runtime, rather than two different analyzers.
|
||||
The complete schema remains the server's validator; the bounded advertisement
|
||||
is a widening and does not replace the delivered `structuredContent`.
|
||||
|
||||
## Completed public workflow and independent oracle
|
||||
|
||||
`mcp-output-profile.mjs` starts the packaged CLI over real stdio with the
|
||||
installed MCP SDK client, an isolated account and an explicit receive cap.
|
||||
It records initialize/discovery, two native export summaries (cold/warm),
|
||||
selected first/last exports, complete Evidence publication, and a second
|
||||
discovery after materialization. Named advertised schemas validate the calls.
|
||||
After closing the server, it authenticates the complete exported bundle and
|
||||
checks every export name/address against GNU `nm`. The complete sets of names
|
||||
must agree; selected rows must retain their original source indices.
|
||||
|
||||
Linux x64, Node 24.22.1, npm 11.16.0. Real target:
|
||||
`/usr/lib/x86_64-linux-gnu/libcrypto.so.3`, SHA-256
|
||||
`aa01e614af634bc7c8b678c9250b5c4f613c61989bacf0a5d6aed6570e7dcabd`.
|
||||
All 5,367 export names/addresses were corroborated. Both profiles produced the
|
||||
same authenticated parent Evidence ID and complete-record digest. GNU `nm`
|
||||
corroborates export coordinates, not behavioral equivalence.
|
||||
|
||||
| Measurement | Full output | Compact output |
|
||||
| -------------------------------------------------- | ----------: | -------------: |
|
||||
| Discovered tools | 229 | 229 |
|
||||
| Serialized catalog result bytes | 3,639,545 | 554,792 |
|
||||
| Catalog cl100k token estimate | 932,629 | 116,962 |
|
||||
| Complete task wire estimate, one discovery | 939,165 | 123,501 |
|
||||
| Executed wire estimate, including second discovery | 1,871,825 | 240,494 |
|
||||
| Discovery after connection, ms | 292 | 1,886 |
|
||||
| Repeated discovery, ms | 333 | 49 |
|
||||
| Whole harness elapsed, s | 9.86 | 6.18 |
|
||||
| Whole harness CPU, s | 11.96 | 7.66 |
|
||||
| Sampled process-family peak RSS, GiB | 1.04 | 0.62 |
|
||||
|
||||
The SDK's real 1,048,576-byte receive cap rejects the full catalog before any
|
||||
analysis call. The compact run succeeds under that cap; the successful full
|
||||
control uses the SDK's normal 10 MiB cap. A small `/usr/bin/ls` pilot also
|
||||
completed under 1 MiB, with all 15 export names corroborated. Neither public
|
||||
workflow uses a fake provider or launches the inspected executable.
|
||||
|
||||
Whole harness costs include the client, server, complete publication, offline
|
||||
bundle parsing/oracle, and cleanup. Each run has a fresh server; the second
|
||||
inspection/discovery is warm. OS file caches were not flushed. Two CPU affinity
|
||||
slots, a 3 GiB Node heap request and an 8 GiB process-family RSS guard bound the
|
||||
runs. RSS is the sum sampled every 100 ms: shared pages can be counted twice
|
||||
and short peaks can be missed. No run hit a resource guard or OOM, and cleanup
|
||||
succeeded. One pair does not establish a general speed or memory improvement;
|
||||
compact startup takes the eager path and its initial discovery is slower.
|
||||
|
||||
## What the token counts establish
|
||||
|
||||
Estimates use `tiktoken`'s `cl100k_base` over serialized JSON messages, including
|
||||
initialization, request arguments, every needed reply and discovery. The
|
||||
one-discovery value excludes only the explicit repeated catalog consistency
|
||||
probe; the executed total includes it. Local complete-bundle bytes stay outside
|
||||
the wire. These are transport serialization estimates, not actual model usage.
|
||||
|
||||
The names/descriptions/input-schema projection is byte-identical across both
|
||||
profiles: 56,909 estimated tokens. A host forwarding only those fields gains no
|
||||
model-context reduction from this output-profile setting. A host retaining
|
||||
output contracts, a typed Code Mode catalog, or the entire wire has a different
|
||||
cost. Native host forwarding and live model behavior were not measured here.
|
||||
The demonstrated benefit is successful bounded-frame discovery with complete
|
||||
analysis, plus the measured reduction in serialized output advertisements.
|
||||
|
||||
## Reproduction and retained failures
|
||||
|
||||
Build once, then use fresh output directories:
|
||||
|
||||
```sh
|
||||
npm run build:cached
|
||||
node experiments/analysis-efficiency/mcp-output-profile.mjs \
|
||||
--runtime . --profile full --target /usr/lib/x86_64-linux-gnu/libcrypto.so.3 \
|
||||
--output /tmp/rea-output-full
|
||||
node experiments/analysis-efficiency/mcp-output-profile.mjs \
|
||||
--runtime . --profile compact --target /usr/lib/x86_64-linux-gnu/libcrypto.so.3 \
|
||||
--receive-bytes 1048576 --output /tmp/rea-output-compact
|
||||
```
|
||||
|
||||
Repeat the full command with `--receive-bytes 1048576` as a negative control;
|
||||
expect a nonzero exit, a receive-buffer error in `report.json`, and no analysis
|
||||
calls. Reports retain the catalog, transcript, complete bundle and server log.
|
||||
The resource guard/profiler is external to this driver; choose host-appropriate
|
||||
limits when reproducing. Token estimates can be reproduced from the saved
|
||||
messages with the tokenizer above, counting each `message` once.
|
||||
|
||||
Two initial small pilot attempts used incomplete source selectors in this new
|
||||
driver. REA correctly returned `invalid_request`; both failed receipts were
|
||||
retained with successful cleanup. The driver was corrected to use the canonical
|
||||
`source: {kind: "retained-evidence", evidence_id}` before the completed runs.
|
||||
|
||||
Initial focused checks ran concurrent files under two CPU slots. Two full
|
||||
catalog compilation assertions exceeded their 30-second test timeouts; 46 other
|
||||
MCP cases and 14 source cases passed. The representative assertion passed alone
|
||||
in 22 seconds, and the complete file passed with one worker, but another run
|
||||
after the main rebase exceeded the deadlines again. CPU contention alone did
|
||||
not fully explain the variability.
|
||||
|
||||
A bounded probe over the five largest output schemas isolated AJV generated-code
|
||||
optimization: compile-only checks took 4.75 seconds with optimization and 2.22
|
||||
seconds without it. The schema-validity fixture now disables that code-generation
|
||||
optimization because it never executes these validators. It still validates the
|
||||
declared dialect, compiles every schema, isolates each reference namespace, and
|
||||
checks malformed/unresolved references. Actual SDK validators retain their normal
|
||||
options. Test deadlines and schema/reference assertions remain unchanged.
|
||||
|
||||
After rebasing onto main `7d36d172a`, both complete real workflows passed again
|
||||
at integration runtime `f735451d9`, retaining the same export parent Evidence
|
||||
and complete-record digest across profiles. The 1 MiB compact cap still passes.
|
||||
The 20 schema/profile cases also pass with the default two workers after the
|
||||
compile-only optimization change; check:fast and repository formatting pass.
|
||||
@@ -0,0 +1,229 @@
|
||||
/** Public stdio investigation with real ELF exports and a selected receive cap. */
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { createWriteStream } from "node:fs";
|
||||
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, resolve } from "node:path";
|
||||
import { finished } from "node:stream/promises";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { parseArgs, promisify } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs";
|
||||
|
||||
const { values } = parseArgs({
|
||||
options: Object.fromEntries(
|
||||
["runtime", "output", "profile", "target", "receive-bytes"].map((key) => [
|
||||
key,
|
||||
{ type: "string" },
|
||||
]),
|
||||
),
|
||||
});
|
||||
for (const key of ["runtime", "output", "target"]) {
|
||||
assert.ok(values[key], `${key} is required`);
|
||||
values[key] = resolve(values[key]);
|
||||
}
|
||||
assert.ok(["full", "compact"].includes(values.profile));
|
||||
const receiveBytes = Number(values["receive-bytes"] ?? 10 * 1024 * 1024);
|
||||
assert.ok(Number.isSafeInteger(receiveBytes) && receiveBytes > 0);
|
||||
await mkdir(values.output);
|
||||
const report = {
|
||||
profile: values.profile,
|
||||
receive_bytes: receiveBytes,
|
||||
runtime: values.runtime,
|
||||
target: values.target,
|
||||
calls: [],
|
||||
};
|
||||
const transcript = [];
|
||||
let account, client, transport, serverLog, serverLogFinished;
|
||||
let phase = "initialize";
|
||||
const began = performance.now();
|
||||
try {
|
||||
account = await mkdtemp(join(tmpdir(), "rea-output-profile-"));
|
||||
report.target_sha256 = createHash("sha256")
|
||||
.update(await readFile(values.target))
|
||||
.digest("hex");
|
||||
report.revision = (
|
||||
await promisify(execFile)("git", [
|
||||
"-C",
|
||||
values.runtime,
|
||||
"rev-parse",
|
||||
"HEAD",
|
||||
])
|
||||
).stdout.trim();
|
||||
transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
|
||||
cwd: values.runtime,
|
||||
maxBufferSize: receiveBytes,
|
||||
env: {
|
||||
PATH: process.env.PATH,
|
||||
HOME: account,
|
||||
USERPROFILE: account,
|
||||
NODE_OPTIONS: process.env.NODE_OPTIONS ?? "--max-old-space-size=3072",
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: values.profile,
|
||||
},
|
||||
stderr: "pipe",
|
||||
});
|
||||
const start = transport.start.bind(transport);
|
||||
transport.start = async () => {
|
||||
const onerror = transport.onerror;
|
||||
transport.onerror = (cause) => {
|
||||
(report.transport_errors ??= []).push(String(cause));
|
||||
onerror?.(cause);
|
||||
};
|
||||
const receive = transport.onmessage;
|
||||
transport.onmessage = (message) => {
|
||||
transcript.push({ direction: "response", phase, message });
|
||||
receive?.(message);
|
||||
};
|
||||
await start();
|
||||
};
|
||||
const send = transport.send.bind(transport);
|
||||
transport.send = async (message) => {
|
||||
transcript.push({ direction: "request", phase, message });
|
||||
await send(message);
|
||||
};
|
||||
serverLog = createWriteStream(join(values.output, "server.log"));
|
||||
serverLogFinished = finished(serverLog);
|
||||
serverLogFinished.catch(() => {});
|
||||
transport.stderr.pipe(serverLog);
|
||||
client = new Client({
|
||||
name: "rea-output-profile-investigation",
|
||||
version: "1",
|
||||
});
|
||||
await client.connect(transport);
|
||||
phase = "discovery-cold";
|
||||
const discoveryBegan = performance.now();
|
||||
const catalog = await client.listTools({}, { timeout: 15000 });
|
||||
report.discovery_ms = performance.now() - discoveryBegan;
|
||||
report.catalog_bytes = Buffer.byteLength(JSON.stringify(catalog));
|
||||
report.tools = catalog.tools.length;
|
||||
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
|
||||
const definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));
|
||||
const call = async (name, args, label) => {
|
||||
phase = label;
|
||||
const at = performance.now();
|
||||
const result = await client.callTool(
|
||||
{ name, arguments: args },
|
||||
{ timeout: 90000, toolDefinition: definitions.get(name) },
|
||||
);
|
||||
assert.notEqual(
|
||||
result.isError,
|
||||
true,
|
||||
`${name}: ${JSON.stringify(result).slice(0, 500)}`,
|
||||
);
|
||||
report.calls.push({ name, phase, elapsed_ms: performance.now() - at });
|
||||
return result.structuredContent;
|
||||
};
|
||||
const first = await call(
|
||||
"inspect_native_exports",
|
||||
{ path: values.target, detail: "summary" },
|
||||
"exports-cold",
|
||||
);
|
||||
const repeated = await call(
|
||||
"inspect_native_exports",
|
||||
{ path: values.target, detail: "summary" },
|
||||
"exports-warm",
|
||||
);
|
||||
const parentId = first.normalized_result.parent_evidence_id;
|
||||
assert.equal(repeated.normalized_result.parent_evidence_id, parentId);
|
||||
report.parent_evidence_id = parentId;
|
||||
const count = first.normalized_result.summary.export_count;
|
||||
assert.ok(count > 1);
|
||||
const page = await call(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source: { kind: "retained-evidence", evidence_id: parentId },
|
||||
view: {
|
||||
kind: "native-exports",
|
||||
collection: "exports",
|
||||
indices: [0, count - 1],
|
||||
fields: ["name"],
|
||||
limit: 2,
|
||||
},
|
||||
},
|
||||
"selected-exports",
|
||||
);
|
||||
assert.deepEqual(
|
||||
page.normalized_result.items.map((item) => item.source_index),
|
||||
[0, count - 1],
|
||||
);
|
||||
const bundlePath = join(values.output, "complete-evidence.json");
|
||||
await call("export_evidence_bundle", { path: bundlePath }, "complete-export");
|
||||
phase = "discovery-warm";
|
||||
const warmAt = performance.now();
|
||||
assert.deepEqual(await client.listTools(), catalog);
|
||||
report.warm_discovery_ms = performance.now() - warmAt;
|
||||
// Release the server ledger before parsing the complete exported records.
|
||||
await client.close();
|
||||
await transport.close();
|
||||
const { parseEvidenceBundle } = await import(
|
||||
pathToFileURL(join(values.runtime, "dist/domain/evidenceBundle.js")).href
|
||||
);
|
||||
const bundle = parseEvidenceBundle(
|
||||
JSON.parse(await readFile(bundlePath, "utf8")),
|
||||
);
|
||||
const parent = bundle.records.find(
|
||||
(record) => record.evidence_id === parentId,
|
||||
);
|
||||
assert.ok(parent);
|
||||
const rows = parent.normalized_result.exports;
|
||||
assert.equal(rows.length, count);
|
||||
for (const row of page.normalized_result.items) {
|
||||
assert.equal(row.name, rows[row.source_index].name);
|
||||
}
|
||||
const nm = await promisify(execFile)(
|
||||
"nm",
|
||||
[
|
||||
"--dynamic",
|
||||
"--defined-only",
|
||||
"--without-symbol-versions",
|
||||
"--format=posix",
|
||||
values.target,
|
||||
],
|
||||
{ maxBuffer: 16 * 1024 * 1024 },
|
||||
);
|
||||
const symbols = nm.stdout
|
||||
.trim()
|
||||
.split("\n")
|
||||
.map((line) => line.split(/\s+/u));
|
||||
const names = new Set(symbols.map(([name]) => name));
|
||||
const coordinates = new Set(
|
||||
symbols.map(([name, , value]) => `${name}:${BigInt(`0x${value}`)}`),
|
||||
);
|
||||
for (const row of rows)
|
||||
assert.ok(
|
||||
coordinates.has(`${row.name}:${BigInt(row.value)}`),
|
||||
`GNU nm did not corroborate ${row.name} at ${row.value}`,
|
||||
);
|
||||
const observed = new Set(rows.map((row) => row.name));
|
||||
for (const name of names)
|
||||
assert.ok(observed.has(name), `REA omitted GNU nm export ${name}`);
|
||||
report.export_count = count;
|
||||
report.nm_unique_names = names.size;
|
||||
report.facts_sha256 = createHash("sha256")
|
||||
.update(JSON.stringify(parent))
|
||||
.digest("hex");
|
||||
report.outcome = "passed";
|
||||
} catch (cause) {
|
||||
report.outcome = "failed";
|
||||
report.failure = String(cause);
|
||||
process.exitCode = 1;
|
||||
} finally {
|
||||
report.elapsed_ms = performance.now() - began;
|
||||
const published = await finalizeRetainedQueryReceipt({
|
||||
report,
|
||||
client,
|
||||
transport,
|
||||
account,
|
||||
serverLog,
|
||||
serverLogFinished,
|
||||
output: values.output,
|
||||
transcript,
|
||||
});
|
||||
if (!published) process.exitCode = 1;
|
||||
}
|
||||
@@ -13,7 +13,7 @@ const { createCatalogDigestProfiles } = await import(
|
||||
pathToFileURL(join(compiledRoot, "catalogIdentity.js")).href
|
||||
);
|
||||
// Retain only hashes, with every shared record frozen. Schemas remain owned by
|
||||
// the compiled contracts and are converted once for all four identities.
|
||||
// the compiled contracts and are converted once for all six identities.
|
||||
const profiles = Object.entries(createCatalogDigestProfiles())
|
||||
.map(
|
||||
([input, outputs]) =>
|
||||
|
||||
+37
-7
@@ -16,7 +16,15 @@ import {
|
||||
presentAdvertisedInputSchema,
|
||||
type CompactInputSchemaPresentation,
|
||||
} from "./contracts/compactInputSchema.js";
|
||||
import type { McpOutputSchemaProfile } from "./config/mcpOutputSchemaProfile.js";
|
||||
import {
|
||||
presentAdvertisedOutputSchema,
|
||||
type CompactOutputSchemaPresentation,
|
||||
} from "./contracts/compactOutputSchema.js";
|
||||
import {
|
||||
COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES,
|
||||
compactPresentationForMcpOutputSchemaProfile,
|
||||
type McpOutputSchemaProfile,
|
||||
} from "./config/mcpOutputSchemaProfile.js";
|
||||
import {
|
||||
COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
|
||||
compactPresentationForMcpInputSchemaProfile,
|
||||
@@ -57,15 +65,28 @@ const createToolCatalog = (contracts: readonly ToolContract[]) =>
|
||||
const presentToolCatalog = (
|
||||
catalog: ReturnType<typeof createToolCatalog>,
|
||||
presentation: CompactInputSchemaPresentation | undefined,
|
||||
outputPresentation: CompactOutputSchemaPresentation | undefined = undefined,
|
||||
) =>
|
||||
presentation === undefined
|
||||
presentation === undefined && outputPresentation === undefined
|
||||
? catalog
|
||||
: catalog.map((tool) => ({
|
||||
...tool,
|
||||
input_schema: presentAdvertisedInputSchema(
|
||||
tool.input_schema,
|
||||
presentation,
|
||||
),
|
||||
...(presentation === undefined
|
||||
? {}
|
||||
: {
|
||||
input_schema: presentAdvertisedInputSchema(
|
||||
tool.input_schema,
|
||||
presentation,
|
||||
),
|
||||
}),
|
||||
...(outputPresentation === undefined
|
||||
? {}
|
||||
: {
|
||||
output_schema: presentAdvertisedOutputSchema(
|
||||
tool.output_schema,
|
||||
outputPresentation,
|
||||
),
|
||||
}),
|
||||
}));
|
||||
|
||||
const createPromptCatalog = (contracts: readonly PromptContract[]) =>
|
||||
@@ -124,6 +145,7 @@ export const createCatalogDigests = (
|
||||
presentToolCatalog(
|
||||
createToolCatalog(sortByName(contracts)),
|
||||
compactPresentation,
|
||||
compactPresentationForMcpOutputSchemaProfile(outputSchemaProfile),
|
||||
),
|
||||
createPromptCatalog(promptContracts),
|
||||
outputSchemaProfile,
|
||||
@@ -137,7 +159,7 @@ export type CatalogDigestProfiles = Readonly<
|
||||
>
|
||||
>;
|
||||
|
||||
/** Build-time identities share schema conversion across the four supported profiles. */
|
||||
/** Build-time identities share schema conversion across the six supported profiles. */
|
||||
export const createCatalogDigestProfiles = (
|
||||
contracts: readonly ToolContract[] = TOOL_CONTRACTS,
|
||||
promptContracts: readonly PromptContract[] = PROMPT_CONTRACTS,
|
||||
@@ -147,8 +169,16 @@ export const createCatalogDigestProfiles = (
|
||||
budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
|
||||
});
|
||||
const prompts = createPromptCatalog(promptContracts);
|
||||
const compactOutput: CompactOutputSchemaPresentation = {
|
||||
budgetBytes: COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES,
|
||||
};
|
||||
const outputProfiles = (catalog: typeof full) => ({
|
||||
full: digestCatalogs(catalog, prompts, "full"),
|
||||
compact: digestCatalogs(
|
||||
presentToolCatalog(catalog, undefined, compactOutput),
|
||||
prompts,
|
||||
"compact",
|
||||
),
|
||||
off: digestCatalogs(catalog, prompts, "off"),
|
||||
});
|
||||
return { full: outputProfiles(full), compact: outputProfiles(compact) };
|
||||
|
||||
@@ -3,15 +3,35 @@ import { z } from "zod";
|
||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
/**
|
||||
* Per-schema budget for compact output advertisements. Output contracts are
|
||||
* larger than input contracts across the catalog, and the complete frame is
|
||||
* what a frame-capped host rejects. Measured on the complete contract catalog
|
||||
* with the default compact input profile, 4 KiB bounds the output
|
||||
* advertisement to ~276 KiB and the complete `tools/list` response to
|
||||
* ~577 KiB, leaving margin under the one-megabyte frame reported in #2512. A
|
||||
* larger budget admits substantially more full contracts; a much smaller one
|
||||
* drops first-level property guidance for nearly every tool.
|
||||
*/
|
||||
export const COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES = 4 * 1024;
|
||||
|
||||
/** Select whether tools/list carries the large output contracts. */
|
||||
export const mcpOutputSchemaProfileSchema = z.enum(["full", "off"]);
|
||||
export const mcpOutputSchemaProfileSchema = z.enum(["full", "compact", "off"]);
|
||||
|
||||
export type McpOutputSchemaProfile = z.infer<
|
||||
typeof mcpOutputSchemaProfileSchema
|
||||
>;
|
||||
|
||||
/** Resolve the compact output advertisement, preserving every other choice. */
|
||||
export const compactPresentationForMcpOutputSchemaProfile = (
|
||||
profile: McpOutputSchemaProfile | undefined,
|
||||
): { readonly budgetBytes: number } | undefined =>
|
||||
profile === "compact"
|
||||
? { budgetBytes: COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES }
|
||||
: undefined;
|
||||
|
||||
const profileConstraint =
|
||||
'REA_MCP_OUTPUT_SCHEMA_PROFILE must be "full" (default) or "off". The off profile omits output schemas from tools/list while REA validates every successful result against its canonical tool contract and returns complete structured content.';
|
||||
'REA_MCP_OUTPUT_SCHEMA_PROFILE must be "full" (default), "compact", or "off". The compact profile advertises every output schema within a 4,096-byte budget; the off profile omits output schemas. Both keep canonical server-side validation and complete structured content.';
|
||||
|
||||
/** Parse the output schema advertisement selection at server composition. */
|
||||
export const parseMcpOutputSchemaProfile = (
|
||||
|
||||
@@ -1,48 +1,18 @@
|
||||
import {
|
||||
JsonSchemaReferences,
|
||||
type JsonSchemaScope,
|
||||
} from "./jsonSchemaReferences.js";
|
||||
import {
|
||||
schemaArrays,
|
||||
schemaChildren,
|
||||
schemaMaps,
|
||||
schemaLiteralValues,
|
||||
} from "./jsonSchemaKeywords.js";
|
||||
COMPACT_SCHEMA_MAX_DEPTH,
|
||||
hasGatewayIncompatibleShape,
|
||||
inlineWithinBudget,
|
||||
reduceToShallowProperties,
|
||||
schemaDepth,
|
||||
serializedBytes,
|
||||
stripAnnotations,
|
||||
typeArrayItemUnions,
|
||||
} from "./compactJsonSchema.js";
|
||||
|
||||
// Kimi expands schema-local references before sending tools to Moonshot.
|
||||
// Compact advertisements must fit after expansion, independently of client
|
||||
// reference handling. Canonical server-side validation remains unchanged.
|
||||
|
||||
/** Annotation keys that carry guidance but no validation semantics. */
|
||||
const annotationKeys = new Set(["description", "examples", "default", "title"]);
|
||||
|
||||
const definitionKeys = new Set(["$defs", "definitions"]);
|
||||
// An empty schema accepts every JSON value. Kimi otherwise infers string.
|
||||
const jsonValueTypes = [
|
||||
"null",
|
||||
"boolean",
|
||||
"object",
|
||||
"array",
|
||||
"number",
|
||||
"string",
|
||||
];
|
||||
// An empty item schema accepts every JSON value, including arbitrarily nested
|
||||
// arrays, while satisfying clients that require `items` for array types.
|
||||
// Gemini rejects `items` beside a type list; the compact catalog test fails
|
||||
// if a tool would advertise this presentation.
|
||||
const jsonValueSchema = () => ({ type: [...jsonValueTypes], items: {} });
|
||||
|
||||
const enumValueTypes = (values: readonly unknown[]): readonly string[] => [
|
||||
...new Set(
|
||||
values.map((value) => {
|
||||
if (value === null) return "null";
|
||||
if (Array.isArray(value)) return "array";
|
||||
if (typeof value === "object") return "object";
|
||||
return typeof value;
|
||||
}),
|
||||
),
|
||||
];
|
||||
|
||||
export type AdvertisedInputSchemaPresentation = "inline" | "reduced";
|
||||
|
||||
/** Budget selected by the caller for compact MCP input-schema presentation. */
|
||||
@@ -55,561 +25,9 @@ export interface CompactedInputSchema {
|
||||
readonly presentation: AdvertisedInputSchemaPresentation;
|
||||
}
|
||||
|
||||
const isSchemaObject = (value: unknown): value is Record<string, unknown> =>
|
||||
typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
|
||||
const serializedBytes = (value: unknown): number =>
|
||||
Buffer.byteLength(JSON.stringify(value), "utf8");
|
||||
|
||||
const schemaDepth = (value: unknown, depth = 0): number => {
|
||||
if (Array.isArray(value))
|
||||
return Math.max(
|
||||
depth,
|
||||
...value.map((child) => schemaDepth(child, depth + 1)),
|
||||
);
|
||||
if (!isSchemaObject(value)) return depth;
|
||||
return Math.max(
|
||||
depth,
|
||||
...Object.values(value).map((child) => schemaDepth(child, depth + 1)),
|
||||
);
|
||||
};
|
||||
|
||||
/** Strict clients commonly reject advertised schemas deeper than ten nodes. */
|
||||
const COMPACT_SCHEMA_MAX_DEPTH = 10;
|
||||
|
||||
/** Rewrite every schema-valued position; literal data stays opaque. */
|
||||
const rebuildSchemaPositions = (
|
||||
node: unknown,
|
||||
rebuild: (child: unknown) => unknown,
|
||||
): unknown => {
|
||||
if (Array.isArray(node)) return node.map((child) => rebuild(child));
|
||||
if (!isSchemaObject(node)) return node;
|
||||
const out: Record<string, unknown> = {};
|
||||
for (const [key, value] of Object.entries(node)) {
|
||||
if (schemaLiteralValues.has(key)) {
|
||||
out[key] = value;
|
||||
continue;
|
||||
}
|
||||
if (schemaMaps.has(key) && isSchemaObject(value)) {
|
||||
out[key] = Object.fromEntries(
|
||||
Object.entries(value).map(([name, child]) => [name, rebuild(child)]),
|
||||
);
|
||||
continue;
|
||||
}
|
||||
if (schemaArrays.has(key) && Array.isArray(value)) {
|
||||
out[key] = value.map((child) => rebuild(child));
|
||||
continue;
|
||||
}
|
||||
if (schemaChildren.has(key)) {
|
||||
out[key] = Array.isArray(value)
|
||||
? value.map((child) => rebuild(child))
|
||||
: rebuild(value);
|
||||
continue;
|
||||
}
|
||||
out[key] = value;
|
||||
}
|
||||
return out;
|
||||
};
|
||||
|
||||
/**
|
||||
* Drop annotation prose from every subschema while preserving all validation
|
||||
* keywords. The root keeps its own description because the root-union
|
||||
* projection encodes accepted input groups there; the root's echoed literal
|
||||
* examples leave with the rest of the annotation prose.
|
||||
*/
|
||||
const stripAnnotations = (
|
||||
schema: Record<string, unknown>,
|
||||
): Record<string, unknown> => {
|
||||
const rebuilt = rebuildSchemaPositions(
|
||||
schema,
|
||||
stripChildAnnotations,
|
||||
) as Record<string, unknown>;
|
||||
const out: Record<string, unknown> = {};
|
||||
for (const [key, value] of Object.entries(rebuilt)) {
|
||||
if (annotationKeys.has(key) && key !== "description") continue;
|
||||
out[key] = value;
|
||||
}
|
||||
return out;
|
||||
};
|
||||
|
||||
const stripChildAnnotations = (child: unknown): unknown => {
|
||||
const rebuilt = rebuildSchemaPositions(child, stripChildAnnotations);
|
||||
if (!isSchemaObject(rebuilt)) return rebuilt;
|
||||
const kept: Record<string, unknown> = {};
|
||||
for (const [key, value] of Object.entries(rebuilt))
|
||||
if (!annotationKeys.has(key)) kept[key] = value;
|
||||
return kept;
|
||||
};
|
||||
|
||||
/** Stop serialization at the budget before allocating an expanded schema. */
|
||||
const inlineWithinBudget = (
|
||||
root: Record<string, unknown>,
|
||||
budgetBytes: number,
|
||||
): Record<string, unknown> | undefined => {
|
||||
const references = new JsonSchemaReferences(root);
|
||||
const identity = Object.fromEntries(
|
||||
["$id", "$anchor", "$schema"].flatMap((key) =>
|
||||
Object.hasOwn(root, key) ? [[key, root[key]]] : [],
|
||||
),
|
||||
);
|
||||
const identityBytes =
|
||||
Object.keys(identity).length === 0 ? 0 : serializedBytes(identity) - 2;
|
||||
if (identityBytes > budgetBytes) return undefined;
|
||||
let resolved = true;
|
||||
function* serialize(
|
||||
value: unknown,
|
||||
position: "schema" | "map" | "data",
|
||||
scope: JsonSchemaScope | undefined,
|
||||
active: ReadonlyMap<JsonSchemaScope, ReadonlySet<string>>,
|
||||
): Generator<string> {
|
||||
if (Array.isArray(value)) {
|
||||
yield "[";
|
||||
for (const [index, child] of value.entries()) {
|
||||
if (index > 0) yield ",";
|
||||
yield* serialize(child, position, scope, active);
|
||||
}
|
||||
yield "]";
|
||||
} else if (isSchemaObject(value)) {
|
||||
const currentScope =
|
||||
position === "schema" ? references.enter(value, scope) : scope;
|
||||
if (
|
||||
position === "schema" &&
|
||||
(currentScope === undefined ||
|
||||
currentScope.dialect !== references.rootScope.dialect ||
|
||||
[
|
||||
"$dynamicRef",
|
||||
"$recursiveRef",
|
||||
"$dynamicAnchor",
|
||||
"$recursiveAnchor",
|
||||
].some((key) => Object.hasOwn(value, key)))
|
||||
) {
|
||||
resolved = false;
|
||||
return;
|
||||
}
|
||||
if (position === "schema" && Object.keys(value).length === 0) {
|
||||
yield JSON.stringify(jsonValueSchema());
|
||||
return;
|
||||
}
|
||||
if (position === "schema" && typeof value.$ref === "string") {
|
||||
const target = references.resolve(currentScope, value.$ref);
|
||||
if (
|
||||
target.kind !== "target" ||
|
||||
currentScope === undefined ||
|
||||
active.get(currentScope)?.has(value.$ref)
|
||||
) {
|
||||
resolved = false;
|
||||
return;
|
||||
}
|
||||
const {
|
||||
$ref,
|
||||
$id: _id,
|
||||
$anchor: _anchor,
|
||||
$schema: _schema,
|
||||
...siblings
|
||||
} = value;
|
||||
const nextActive = new Map(active);
|
||||
nextActive.set(
|
||||
currentScope,
|
||||
new Set([...(active.get(currentScope) ?? []), $ref]),
|
||||
);
|
||||
// Draft 7 ignores siblings, while clients such as Ajv also apply them.
|
||||
// Do not label either interpretation an equivalent inline advertisement.
|
||||
if (
|
||||
currentScope.dialect === "draft-07" &&
|
||||
Object.keys(siblings).length > 0
|
||||
) {
|
||||
resolved = false;
|
||||
return;
|
||||
}
|
||||
const expanded =
|
||||
Object.keys(siblings).length === 0
|
||||
? target.value
|
||||
: { allOf: [target.value, siblings] };
|
||||
yield* serialize(expanded, "schema", currentScope, nextActive);
|
||||
return;
|
||||
}
|
||||
yield "{";
|
||||
let first = true;
|
||||
for (const [key, child] of Object.entries(value)) {
|
||||
if (
|
||||
position === "schema" &&
|
||||
(definitionKeys.has(key) ||
|
||||
key === "$id" ||
|
||||
key === "$anchor" ||
|
||||
key === "$schema")
|
||||
)
|
||||
continue;
|
||||
if (!first) yield ",";
|
||||
first = false;
|
||||
yield JSON.stringify(key) + ":";
|
||||
// `not: {}` forbids a value; no client infers a type for a value it
|
||||
// must omit, and the any-value list would place `items` beside a
|
||||
// non-array type, which Gemini function declarations reject.
|
||||
if (
|
||||
position === "schema" &&
|
||||
key === "not" &&
|
||||
isSchemaObject(child) &&
|
||||
Object.keys(child).length === 0
|
||||
) {
|
||||
yield "{}";
|
||||
continue;
|
||||
}
|
||||
const childPosition =
|
||||
position === "map"
|
||||
? "schema"
|
||||
: position === "schema" && schemaMaps.has(key)
|
||||
? "map"
|
||||
: position === "schema" &&
|
||||
(schemaArrays.has(key) || schemaChildren.has(key))
|
||||
? "schema"
|
||||
: "data";
|
||||
yield* serialize(child, childPosition, currentScope, active);
|
||||
}
|
||||
yield "}";
|
||||
} else {
|
||||
yield JSON.stringify(value);
|
||||
}
|
||||
}
|
||||
const chunks: string[] = [];
|
||||
let bytes = identityBytes;
|
||||
for (const chunk of serialize(
|
||||
root,
|
||||
"schema",
|
||||
references.rootScope,
|
||||
new Map(),
|
||||
)) {
|
||||
bytes += Buffer.byteLength(chunk, "utf8");
|
||||
if (bytes > budgetBytes || !resolved) return undefined;
|
||||
chunks.push(chunk);
|
||||
}
|
||||
if (!resolved) return undefined;
|
||||
const expanded: unknown = JSON.parse(chunks.join(""));
|
||||
if (!isSchemaObject(expanded)) return undefined;
|
||||
// Every emitted dependency is now static and reference-free. Nested resource
|
||||
// identifiers/anchors can leave without duplicating identities at call sites.
|
||||
const schema = { ...identity, ...expanded };
|
||||
return serializedBytes(schema) <= budgetBytes ? schema : undefined;
|
||||
};
|
||||
|
||||
/** Preserve every possible type, including referenced and nullable unions. */
|
||||
const detectedTypes = (
|
||||
schema: unknown,
|
||||
references: JsonSchemaReferences,
|
||||
inherited: JsonSchemaScope | undefined,
|
||||
active: ReadonlySet<object> = new Set(),
|
||||
): readonly string[] | undefined => {
|
||||
if (!isSchemaObject(schema)) return undefined;
|
||||
const scope = references.enter(schema, inherited);
|
||||
if (scope === undefined) return undefined;
|
||||
if (typeof schema.$ref === "string") {
|
||||
if (active.has(schema)) return undefined;
|
||||
const target = references.resolve(scope, schema.$ref);
|
||||
return target.kind === "target"
|
||||
? detectedTypes(
|
||||
target.value,
|
||||
references,
|
||||
scope,
|
||||
new Set([...active, schema]),
|
||||
)
|
||||
: undefined;
|
||||
}
|
||||
if (typeof schema.type === "string") return [schema.type];
|
||||
if (
|
||||
Array.isArray(schema.type) &&
|
||||
schema.type.every((type) => typeof type === "string")
|
||||
)
|
||||
return schema.type;
|
||||
if (Array.isArray(schema.enum) && schema.enum.length > 0) {
|
||||
const enumTypes = enumValueTypes(schema.enum);
|
||||
if (enumTypes.every((type) => jsonValueTypes.includes(type)))
|
||||
return enumTypes;
|
||||
}
|
||||
const alternatives = Array.isArray(schema.anyOf)
|
||||
? schema.anyOf
|
||||
: Array.isArray(schema.oneOf)
|
||||
? schema.oneOf
|
||||
: [];
|
||||
if (alternatives.length === 0) return undefined;
|
||||
const types = alternatives.map((branch) =>
|
||||
detectedTypes(branch, references, scope, active),
|
||||
);
|
||||
if (types.some((type) => type === undefined)) return undefined;
|
||||
return [...new Set(types.flatMap((type) => type ?? []))];
|
||||
};
|
||||
|
||||
interface ScopedSchema {
|
||||
readonly schema: Record<string, unknown>;
|
||||
readonly scope: JsonSchemaScope;
|
||||
}
|
||||
|
||||
/**
|
||||
* Collect the item schema of every array alternative. Tuples, boolean item
|
||||
* schemas and unresolved references are unknown because one item type would
|
||||
* not describe every accepted element.
|
||||
*/
|
||||
const arrayItemAlternatives = (
|
||||
schema: unknown,
|
||||
references: JsonSchemaReferences,
|
||||
inherited: JsonSchemaScope | undefined,
|
||||
active: ReadonlySet<object> = new Set(),
|
||||
): readonly ScopedSchema[] | undefined => {
|
||||
if (!isSchemaObject(schema)) return undefined;
|
||||
const scope = references.enter(schema, inherited);
|
||||
if (scope === undefined) return undefined;
|
||||
if (typeof schema.$ref === "string") {
|
||||
if (active.has(schema)) return undefined;
|
||||
const target = references.resolve(scope, schema.$ref);
|
||||
return target.kind === "target"
|
||||
? arrayItemAlternatives(
|
||||
target.value,
|
||||
references,
|
||||
scope,
|
||||
new Set([...active, schema]),
|
||||
)
|
||||
: undefined;
|
||||
}
|
||||
const types =
|
||||
typeof schema.type === "string"
|
||||
? [schema.type]
|
||||
: Array.isArray(schema.type)
|
||||
? schema.type
|
||||
: undefined;
|
||||
if (types !== undefined) {
|
||||
if (!types.includes("array")) return [];
|
||||
return isSchemaObject(schema.items) && schema.prefixItems === undefined
|
||||
? [{ schema: schema.items, scope }]
|
||||
: undefined;
|
||||
}
|
||||
const alternatives = Array.isArray(schema.anyOf)
|
||||
? schema.anyOf
|
||||
: Array.isArray(schema.oneOf)
|
||||
? schema.oneOf
|
||||
: [];
|
||||
if (alternatives.length === 0) return undefined;
|
||||
const nested = alternatives.map((branch) =>
|
||||
arrayItemAlternatives(branch, references, scope, active),
|
||||
);
|
||||
return nested.some((items) => items === undefined)
|
||||
? undefined
|
||||
: nested.flatMap((items) => items ?? []);
|
||||
};
|
||||
|
||||
/** Nested arrays beyond this level advertise an unconstrained item schema. */
|
||||
const REDUCED_ITEM_MAX_DEPTH = 3;
|
||||
|
||||
/**
|
||||
* Advertise the element types the canonical item schemas accept. Gemini
|
||||
* function declarations permit `items` only beside the single `array` type,
|
||||
* so an element that may be an array alongside other types, or whose type is
|
||||
* unknown, uses the empty schema: it accepts every element and still gives
|
||||
* clients that require `items` an item schema.
|
||||
*/
|
||||
const reducedItemSchema = (
|
||||
alternatives: readonly ScopedSchema[] | undefined,
|
||||
references: JsonSchemaReferences,
|
||||
depth = 1,
|
||||
): Record<string, unknown> => {
|
||||
if (
|
||||
alternatives === undefined ||
|
||||
alternatives.length === 0 ||
|
||||
depth > REDUCED_ITEM_MAX_DEPTH
|
||||
)
|
||||
return {};
|
||||
const types: string[] = [];
|
||||
for (const { schema, scope } of alternatives) {
|
||||
const detected = detectedTypes(schema, references, scope);
|
||||
if (detected === undefined || detected.length === 0) return {};
|
||||
types.push(...detected);
|
||||
}
|
||||
const unique = [...new Set(types)];
|
||||
if (!unique.includes("array"))
|
||||
return { type: unique.length === 1 ? unique[0] : unique };
|
||||
if (unique.length > 1) return {};
|
||||
const nested: ScopedSchema[] = [];
|
||||
for (const { schema, scope } of alternatives) {
|
||||
const items = arrayItemAlternatives(schema, references, scope);
|
||||
if (items === undefined) return { type: "array", items: {} };
|
||||
nested.push(...items);
|
||||
}
|
||||
return {
|
||||
type: "array",
|
||||
items: reducedItemSchema(nested, references, depth + 1),
|
||||
};
|
||||
};
|
||||
|
||||
/** Give homogeneous array-item unions the type required by strict gateways. */
|
||||
const typeArrayItemUnions = (
|
||||
schema: Record<string, unknown>,
|
||||
): Record<string, unknown> => {
|
||||
const references = new JsonSchemaReferences(schema);
|
||||
const visit = (node: unknown): unknown => {
|
||||
const rebuilt = rebuildSchemaPositions(node, visit);
|
||||
if (!isSchemaObject(rebuilt) || rebuilt.type !== "array") return rebuilt;
|
||||
const items = rebuilt.items;
|
||||
if (
|
||||
!isSchemaObject(items) ||
|
||||
items.type !== undefined ||
|
||||
(!Array.isArray(items.oneOf) && !Array.isArray(items.anyOf))
|
||||
)
|
||||
return rebuilt;
|
||||
const types = detectedTypes(items, references, references.rootScope);
|
||||
return types?.length === 1
|
||||
? { ...rebuilt, items: { ...items, type: types[0] } }
|
||||
: rebuilt;
|
||||
};
|
||||
return visit(schema) as Record<string, unknown>;
|
||||
};
|
||||
|
||||
/**
|
||||
* Strict gateways reject some nested combinators, primitive enums without a
|
||||
* type, and array items whose union has no explicit type. Homogeneous item
|
||||
* unions gain their shared type; remaining incompatible shapes use the shallow
|
||||
* projection while canonical runtime validation stays intact.
|
||||
*/
|
||||
const hasGatewayIncompatibleShape = (schema: unknown): boolean => {
|
||||
if (Array.isArray(schema)) return schema.some(hasGatewayIncompatibleShape);
|
||||
if (!isSchemaObject(schema)) return false;
|
||||
if (Array.isArray(schema.anyOf) && schema.anyOf.length === 0) return true;
|
||||
if (
|
||||
schema.type === "array" &&
|
||||
isSchemaObject(schema.items) &&
|
||||
schema.items.type === undefined &&
|
||||
(Array.isArray(schema.items.oneOf) || Array.isArray(schema.items.anyOf))
|
||||
)
|
||||
return true;
|
||||
if (Array.isArray(schema.required) && isSchemaObject(schema.properties)) {
|
||||
for (const name of schema.required) {
|
||||
const property = schema.properties[name];
|
||||
if (isSchemaObject(property) && containsGatewayCombinator(property))
|
||||
return true;
|
||||
}
|
||||
}
|
||||
if (
|
||||
Array.isArray(schema.enum) &&
|
||||
schema.enum.length > 0 &&
|
||||
schema.type === undefined
|
||||
) {
|
||||
const types = enumValueTypes(schema.enum);
|
||||
if (
|
||||
types.length === 1 &&
|
||||
types.every((type) =>
|
||||
["string", "number", "boolean", "null"].includes(type),
|
||||
)
|
||||
)
|
||||
return true;
|
||||
}
|
||||
return schemaChildrenAndMaps(schema).some(hasGatewayIncompatibleShape);
|
||||
};
|
||||
|
||||
const containsGatewayCombinator = (schema: unknown): boolean => {
|
||||
if (Array.isArray(schema)) return schema.some(containsGatewayCombinator);
|
||||
if (!isSchemaObject(schema)) return false;
|
||||
if (schema.oneOf !== undefined || schema.allOf !== undefined) return true;
|
||||
return schemaChildrenAndMaps(schema).some(containsGatewayCombinator);
|
||||
};
|
||||
|
||||
/** Traverse only schema-valued positions; property names remain opaque data. */
|
||||
const schemaChildrenAndMaps = (
|
||||
schema: Readonly<Record<string, unknown>>,
|
||||
): readonly unknown[] =>
|
||||
Object.entries(schema).flatMap(([key, value]) => {
|
||||
if (schemaMaps.has(key) && isSchemaObject(value))
|
||||
return Object.values(value);
|
||||
if (schemaArrays.has(key) && Array.isArray(value)) return value;
|
||||
if (schemaChildren.has(key)) return Array.isArray(value) ? value : [value];
|
||||
return [];
|
||||
});
|
||||
|
||||
/**
|
||||
* Present a reduced property's reference-resolved types. Reduced inputs omit
|
||||
* nested validation, but clients still require an item schema for an array
|
||||
* declaration, and Gemini accepts `items` only beside the single `array` type,
|
||||
* so a property that may be an array or another type lists typed branches.
|
||||
*/
|
||||
const reducedPropertyTypes = (
|
||||
value: unknown,
|
||||
references: JsonSchemaReferences,
|
||||
): Record<string, unknown> => {
|
||||
const types = detectedTypes(value, references, references.rootScope);
|
||||
if (types === undefined || types.length === 0) return {};
|
||||
if (!types.includes("array"))
|
||||
return { type: types.length === 1 ? types[0] : types };
|
||||
const array = {
|
||||
type: "array",
|
||||
items: reducedItemSchema(
|
||||
arrayItemAlternatives(value, references, references.rootScope),
|
||||
references,
|
||||
),
|
||||
};
|
||||
return types.length === 1
|
||||
? array
|
||||
: {
|
||||
anyOf: types.map((type) => (type === "array" ? array : { type })),
|
||||
};
|
||||
};
|
||||
|
||||
const REDUCED_ANNOTATION =
|
||||
const INPUT_REDUCED_ANNOTATION =
|
||||
"Advertised in reduced form by the compact input schema profile; the server still validates complete canonical inputs.";
|
||||
|
||||
/**
|
||||
* Present only the root property shape with per-property guidance. Used for
|
||||
* schemas whose validation structure alone exceeds the compact budget; the
|
||||
* canonical schema still rejects malformed calls server-side.
|
||||
*/
|
||||
const reduceToShallowProperties = (
|
||||
schema: Record<string, unknown>,
|
||||
): Record<string, unknown> => {
|
||||
const properties = isSchemaObject(schema.properties) ? schema.properties : {};
|
||||
const description = [
|
||||
typeof schema.description === "string" ? schema.description : undefined,
|
||||
...Object.entries(
|
||||
isSchemaObject(schema.dependentRequired)
|
||||
? schema.dependentRequired
|
||||
: isSchemaObject(schema.dependencies)
|
||||
? schema.dependencies
|
||||
: {},
|
||||
).flatMap(([property, partners]) =>
|
||||
Array.isArray(partners) &&
|
||||
partners.every((partner) => typeof partner === "string")
|
||||
? [
|
||||
`When ${property} is provided, also provide ${partners.join(" and ")}.`,
|
||||
]
|
||||
: [],
|
||||
),
|
||||
REDUCED_ANNOTATION,
|
||||
]
|
||||
.filter((text) => text !== undefined)
|
||||
.join(" ");
|
||||
const references = new JsonSchemaReferences(schema);
|
||||
const shallow = Object.fromEntries(
|
||||
Object.entries(properties).map(([name, value]) => {
|
||||
const description =
|
||||
isSchemaObject(value) && typeof value.description === "string"
|
||||
? { description: value.description }
|
||||
: {};
|
||||
return [
|
||||
name,
|
||||
{ ...reducedPropertyTypes(value, references), ...description },
|
||||
];
|
||||
}),
|
||||
);
|
||||
return {
|
||||
...(typeof schema.$schema === "string" ? { $schema: schema.$schema } : {}),
|
||||
type: "object",
|
||||
description,
|
||||
properties: shallow,
|
||||
...(Array.isArray(schema.required)
|
||||
? { required: [...schema.required] }
|
||||
: {}),
|
||||
...(typeof schema.minProperties === "number"
|
||||
? { minProperties: schema.minProperties }
|
||||
: {}),
|
||||
...(schema.additionalProperties === false
|
||||
? { additionalProperties: false }
|
||||
: {}),
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* Render an advertised input schema within a per-schema byte budget for
|
||||
* providers that validate the literal serialized schema size. Validation
|
||||
@@ -631,19 +49,22 @@ export const compactAdvertisedInputSchema = (
|
||||
!hasGatewayIncompatibleShape(inline)
|
||||
)
|
||||
return { schema: inline, presentation: "inline" };
|
||||
const reduced = reduceToShallowProperties(schema);
|
||||
const reduced = reduceToShallowProperties(schema, INPUT_REDUCED_ANNOTATION);
|
||||
if (serializedBytes(reduced) <= budgetBytes)
|
||||
return { schema: reduced, presentation: "reduced" };
|
||||
// A future contract with an extreme root-property count still advertises
|
||||
// an honest, budget-fitting shape instead of an oversized one.
|
||||
return {
|
||||
schema: reduceToShallowProperties({
|
||||
type: "object",
|
||||
description:
|
||||
(typeof schema.description === "string"
|
||||
? `${schema.description} `
|
||||
: "") + REDUCED_ANNOTATION,
|
||||
}),
|
||||
schema: reduceToShallowProperties(
|
||||
{
|
||||
type: "object",
|
||||
description:
|
||||
(typeof schema.description === "string"
|
||||
? `${schema.description} `
|
||||
: "") + INPUT_REDUCED_ANNOTATION,
|
||||
},
|
||||
INPUT_REDUCED_ANNOTATION,
|
||||
),
|
||||
presentation: "reduced",
|
||||
};
|
||||
};
|
||||
|
||||
@@ -0,0 +1,597 @@
|
||||
import {
|
||||
JsonSchemaReferences,
|
||||
type JsonSchemaScope,
|
||||
} from "./jsonSchemaReferences.js";
|
||||
import {
|
||||
schemaArrays,
|
||||
schemaChildren,
|
||||
schemaMaps,
|
||||
schemaLiteralValues,
|
||||
} from "./jsonSchemaKeywords.js";
|
||||
|
||||
// Shared compact JSON Schema projection primitives; canonical validation stays separate.
|
||||
|
||||
/** Annotation keys that carry guidance but no validation semantics. */
|
||||
const annotationKeys = new Set(["description", "examples", "default", "title"]);
|
||||
|
||||
const definitionKeys = new Set(["$defs", "definitions"]);
|
||||
// An empty schema accepts every JSON value. Kimi otherwise infers string.
|
||||
const jsonValueTypes = [
|
||||
"null",
|
||||
"boolean",
|
||||
"object",
|
||||
"array",
|
||||
"number",
|
||||
"string",
|
||||
];
|
||||
// An empty item schema accepts every JSON value, including arbitrarily nested
|
||||
// arrays, while satisfying clients that require `items` for array types.
|
||||
// Gemini rejects `items` beside a type list; the compact catalog test fails
|
||||
// if a tool would advertise this presentation.
|
||||
export const jsonValueSchema = () => ({ type: [...jsonValueTypes], items: {} });
|
||||
|
||||
const enumValueTypes = (values: readonly unknown[]): readonly string[] => [
|
||||
...new Set(
|
||||
values.map((value) => {
|
||||
if (value === null) return "null";
|
||||
if (Array.isArray(value)) return "array";
|
||||
if (typeof value === "object") return "object";
|
||||
return typeof value;
|
||||
}),
|
||||
),
|
||||
];
|
||||
|
||||
export const isSchemaObject = (
|
||||
value: unknown,
|
||||
): value is Record<string, unknown> =>
|
||||
typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
|
||||
export const serializedBytes = (value: unknown): number =>
|
||||
Buffer.byteLength(JSON.stringify(value), "utf8");
|
||||
|
||||
export const schemaDepth = (value: unknown, depth = 0): number => {
|
||||
if (Array.isArray(value))
|
||||
return Math.max(
|
||||
depth,
|
||||
...value.map((child) => schemaDepth(child, depth + 1)),
|
||||
);
|
||||
if (!isSchemaObject(value)) return depth;
|
||||
return Math.max(
|
||||
depth,
|
||||
...Object.values(value).map((child) => schemaDepth(child, depth + 1)),
|
||||
);
|
||||
};
|
||||
|
||||
/** Strict clients commonly reject advertised schemas deeper than ten nodes. */
|
||||
export const COMPACT_SCHEMA_MAX_DEPTH = 10;
|
||||
|
||||
/** Rewrite every schema-valued position; literal data stays opaque. */
|
||||
const rebuildSchemaPositions = (
|
||||
node: unknown,
|
||||
rebuild: (child: unknown) => unknown,
|
||||
): unknown => {
|
||||
if (Array.isArray(node)) return node.map((child) => rebuild(child));
|
||||
if (!isSchemaObject(node)) return node;
|
||||
const out: Record<string, unknown> = {};
|
||||
for (const [key, value] of Object.entries(node)) {
|
||||
if (schemaLiteralValues.has(key)) {
|
||||
out[key] = value;
|
||||
continue;
|
||||
}
|
||||
if (schemaMaps.has(key) && isSchemaObject(value)) {
|
||||
out[key] = Object.fromEntries(
|
||||
Object.entries(value).map(([name, child]) => [name, rebuild(child)]),
|
||||
);
|
||||
continue;
|
||||
}
|
||||
if (schemaArrays.has(key) && Array.isArray(value)) {
|
||||
out[key] = value.map((child) => rebuild(child));
|
||||
continue;
|
||||
}
|
||||
if (schemaChildren.has(key)) {
|
||||
out[key] = Array.isArray(value)
|
||||
? value.map((child) => rebuild(child))
|
||||
: rebuild(value);
|
||||
continue;
|
||||
}
|
||||
out[key] = value;
|
||||
}
|
||||
return out;
|
||||
};
|
||||
|
||||
/**
|
||||
* Drop annotation prose from every subschema while preserving all validation
|
||||
* keywords. The root keeps its own description because the root-union
|
||||
* projection encodes accepted input groups there; the root's echoed literal
|
||||
* examples leave with the rest of the annotation prose.
|
||||
*/
|
||||
export const stripAnnotations = (
|
||||
schema: Record<string, unknown>,
|
||||
): Record<string, unknown> => {
|
||||
const rebuilt = rebuildSchemaPositions(
|
||||
schema,
|
||||
stripChildAnnotations,
|
||||
) as Record<string, unknown>;
|
||||
const out: Record<string, unknown> = {};
|
||||
for (const [key, value] of Object.entries(rebuilt)) {
|
||||
if (annotationKeys.has(key) && key !== "description") continue;
|
||||
out[key] = value;
|
||||
}
|
||||
return out;
|
||||
};
|
||||
|
||||
const stripChildAnnotations = (child: unknown): unknown => {
|
||||
const rebuilt = rebuildSchemaPositions(child, stripChildAnnotations);
|
||||
if (!isSchemaObject(rebuilt)) return rebuilt;
|
||||
const kept: Record<string, unknown> = {};
|
||||
for (const [key, value] of Object.entries(rebuilt))
|
||||
if (!annotationKeys.has(key)) kept[key] = value;
|
||||
return kept;
|
||||
};
|
||||
|
||||
/** Stop serialization at the budget before allocating an expanded schema. */
|
||||
export const inlineWithinBudget = (
|
||||
root: Record<string, unknown>,
|
||||
budgetBytes: number,
|
||||
): Record<string, unknown> | undefined => {
|
||||
const references = new JsonSchemaReferences(root);
|
||||
const identity = Object.fromEntries(
|
||||
["$id", "$anchor", "$schema"].flatMap((key) =>
|
||||
Object.hasOwn(root, key) ? [[key, root[key]]] : [],
|
||||
),
|
||||
);
|
||||
const identityBytes =
|
||||
Object.keys(identity).length === 0 ? 0 : serializedBytes(identity) - 2;
|
||||
if (identityBytes > budgetBytes) return undefined;
|
||||
let resolved = true;
|
||||
function* serialize(
|
||||
value: unknown,
|
||||
position: "schema" | "map" | "data",
|
||||
scope: JsonSchemaScope | undefined,
|
||||
active: ReadonlyMap<JsonSchemaScope, ReadonlySet<string>>,
|
||||
): Generator<string> {
|
||||
if (Array.isArray(value)) {
|
||||
yield "[";
|
||||
for (const [index, child] of value.entries()) {
|
||||
if (index > 0) yield ",";
|
||||
yield* serialize(child, position, scope, active);
|
||||
}
|
||||
yield "]";
|
||||
} else if (isSchemaObject(value)) {
|
||||
const currentScope =
|
||||
position === "schema" ? references.enter(value, scope) : scope;
|
||||
if (
|
||||
position === "schema" &&
|
||||
(currentScope === undefined ||
|
||||
currentScope.dialect !== references.rootScope.dialect ||
|
||||
[
|
||||
"$dynamicRef",
|
||||
"$recursiveRef",
|
||||
"$dynamicAnchor",
|
||||
"$recursiveAnchor",
|
||||
].some((key) => Object.hasOwn(value, key)))
|
||||
) {
|
||||
resolved = false;
|
||||
return;
|
||||
}
|
||||
if (position === "schema" && Object.keys(value).length === 0) {
|
||||
yield JSON.stringify(jsonValueSchema());
|
||||
return;
|
||||
}
|
||||
if (position === "schema" && typeof value.$ref === "string") {
|
||||
const target = references.resolve(currentScope, value.$ref);
|
||||
if (
|
||||
target.kind !== "target" ||
|
||||
currentScope === undefined ||
|
||||
active.get(currentScope)?.has(value.$ref)
|
||||
) {
|
||||
resolved = false;
|
||||
return;
|
||||
}
|
||||
const {
|
||||
$ref,
|
||||
$id: _id,
|
||||
$anchor: _anchor,
|
||||
$schema: _schema,
|
||||
...siblings
|
||||
} = value;
|
||||
const nextActive = new Map(active);
|
||||
nextActive.set(
|
||||
currentScope,
|
||||
new Set([...(active.get(currentScope) ?? []), $ref]),
|
||||
);
|
||||
// Draft 7 ignores siblings, while clients such as Ajv also apply them.
|
||||
// Do not label either interpretation an equivalent inline advertisement.
|
||||
if (
|
||||
currentScope.dialect === "draft-07" &&
|
||||
Object.keys(siblings).length > 0
|
||||
) {
|
||||
resolved = false;
|
||||
return;
|
||||
}
|
||||
const expanded =
|
||||
Object.keys(siblings).length === 0
|
||||
? target.value
|
||||
: { allOf: [target.value, siblings] };
|
||||
yield* serialize(expanded, "schema", currentScope, nextActive);
|
||||
return;
|
||||
}
|
||||
yield "{";
|
||||
let first = true;
|
||||
for (const [key, child] of Object.entries(value)) {
|
||||
if (
|
||||
position === "schema" &&
|
||||
(definitionKeys.has(key) ||
|
||||
key === "$id" ||
|
||||
key === "$anchor" ||
|
||||
key === "$schema")
|
||||
)
|
||||
continue;
|
||||
if (!first) yield ",";
|
||||
first = false;
|
||||
yield JSON.stringify(key) + ":";
|
||||
// `not: {}` forbids a value; no client infers a type for a value it
|
||||
// must omit, and the any-value list would place `items` beside a
|
||||
// non-array type, which Gemini function declarations reject.
|
||||
if (
|
||||
position === "schema" &&
|
||||
key === "not" &&
|
||||
isSchemaObject(child) &&
|
||||
Object.keys(child).length === 0
|
||||
) {
|
||||
yield "{}";
|
||||
continue;
|
||||
}
|
||||
const childPosition =
|
||||
position === "map"
|
||||
? "schema"
|
||||
: position === "schema" && schemaMaps.has(key)
|
||||
? "map"
|
||||
: position === "schema" &&
|
||||
(schemaArrays.has(key) || schemaChildren.has(key))
|
||||
? "schema"
|
||||
: "data";
|
||||
yield* serialize(child, childPosition, currentScope, active);
|
||||
}
|
||||
yield "}";
|
||||
} else {
|
||||
yield JSON.stringify(value);
|
||||
}
|
||||
}
|
||||
const chunks: string[] = [];
|
||||
let bytes = identityBytes;
|
||||
for (const chunk of serialize(
|
||||
root,
|
||||
"schema",
|
||||
references.rootScope,
|
||||
new Map(),
|
||||
)) {
|
||||
bytes += Buffer.byteLength(chunk, "utf8");
|
||||
if (bytes > budgetBytes || !resolved) return undefined;
|
||||
chunks.push(chunk);
|
||||
}
|
||||
if (!resolved) return undefined;
|
||||
const expanded: unknown = JSON.parse(chunks.join(""));
|
||||
if (!isSchemaObject(expanded)) return undefined;
|
||||
// Every emitted dependency is now static and reference-free. Nested resource
|
||||
// identifiers/anchors can leave without duplicating identities at call sites.
|
||||
const schema = { ...identity, ...expanded };
|
||||
return serializedBytes(schema) <= budgetBytes ? schema : undefined;
|
||||
};
|
||||
|
||||
/** Preserve every possible type, including referenced and nullable unions. */
|
||||
const detectedTypes = (
|
||||
schema: unknown,
|
||||
references: JsonSchemaReferences,
|
||||
inherited: JsonSchemaScope | undefined,
|
||||
active: ReadonlySet<object> = new Set(),
|
||||
): readonly string[] | undefined => {
|
||||
if (!isSchemaObject(schema)) return undefined;
|
||||
const scope = references.enter(schema, inherited);
|
||||
if (scope === undefined) return undefined;
|
||||
if (typeof schema.$ref === "string") {
|
||||
if (active.has(schema)) return undefined;
|
||||
const target = references.resolve(scope, schema.$ref);
|
||||
return target.kind === "target"
|
||||
? detectedTypes(
|
||||
target.value,
|
||||
references,
|
||||
scope,
|
||||
new Set([...active, schema]),
|
||||
)
|
||||
: undefined;
|
||||
}
|
||||
if (typeof schema.type === "string") return [schema.type];
|
||||
if (
|
||||
Array.isArray(schema.type) &&
|
||||
schema.type.every((type) => typeof type === "string")
|
||||
)
|
||||
return schema.type;
|
||||
if (Array.isArray(schema.enum) && schema.enum.length > 0) {
|
||||
const enumTypes = enumValueTypes(schema.enum);
|
||||
if (enumTypes.every((type) => jsonValueTypes.includes(type)))
|
||||
return enumTypes;
|
||||
}
|
||||
const alternatives = Array.isArray(schema.anyOf)
|
||||
? schema.anyOf
|
||||
: Array.isArray(schema.oneOf)
|
||||
? schema.oneOf
|
||||
: [];
|
||||
if (alternatives.length === 0) return undefined;
|
||||
const types = alternatives.map((branch) =>
|
||||
detectedTypes(branch, references, scope, active),
|
||||
);
|
||||
if (types.some((type) => type === undefined)) return undefined;
|
||||
return [...new Set(types.flatMap((type) => type ?? []))];
|
||||
};
|
||||
|
||||
interface ScopedSchema {
|
||||
readonly schema: Record<string, unknown>;
|
||||
readonly scope: JsonSchemaScope;
|
||||
}
|
||||
|
||||
/**
|
||||
* Collect the item schema of every array alternative. Tuples, boolean item
|
||||
* schemas and unresolved references are unknown because one item type would
|
||||
* not describe every accepted element.
|
||||
*/
|
||||
const arrayItemAlternatives = (
|
||||
schema: unknown,
|
||||
references: JsonSchemaReferences,
|
||||
inherited: JsonSchemaScope | undefined,
|
||||
active: ReadonlySet<object> = new Set(),
|
||||
): readonly ScopedSchema[] | undefined => {
|
||||
if (!isSchemaObject(schema)) return undefined;
|
||||
const scope = references.enter(schema, inherited);
|
||||
if (scope === undefined) return undefined;
|
||||
if (typeof schema.$ref === "string") {
|
||||
if (active.has(schema)) return undefined;
|
||||
const target = references.resolve(scope, schema.$ref);
|
||||
return target.kind === "target"
|
||||
? arrayItemAlternatives(
|
||||
target.value,
|
||||
references,
|
||||
scope,
|
||||
new Set([...active, schema]),
|
||||
)
|
||||
: undefined;
|
||||
}
|
||||
const types =
|
||||
typeof schema.type === "string"
|
||||
? [schema.type]
|
||||
: Array.isArray(schema.type)
|
||||
? schema.type
|
||||
: undefined;
|
||||
if (types !== undefined) {
|
||||
if (!types.includes("array")) return [];
|
||||
return isSchemaObject(schema.items) && schema.prefixItems === undefined
|
||||
? [{ schema: schema.items, scope }]
|
||||
: undefined;
|
||||
}
|
||||
const alternatives = Array.isArray(schema.anyOf)
|
||||
? schema.anyOf
|
||||
: Array.isArray(schema.oneOf)
|
||||
? schema.oneOf
|
||||
: [];
|
||||
if (alternatives.length === 0) return undefined;
|
||||
const nested = alternatives.map((branch) =>
|
||||
arrayItemAlternatives(branch, references, scope, active),
|
||||
);
|
||||
return nested.some((items) => items === undefined)
|
||||
? undefined
|
||||
: nested.flatMap((items) => items ?? []);
|
||||
};
|
||||
|
||||
/** Nested arrays beyond this level advertise an unconstrained item schema. */
|
||||
const REDUCED_ITEM_MAX_DEPTH = 3;
|
||||
|
||||
/**
|
||||
* Advertise the element types the canonical item schemas accept. Gemini
|
||||
* function declarations permit `items` only beside the single `array` type,
|
||||
* so an element that may be an array alongside other types, or whose type is
|
||||
* unknown, uses the empty schema: it accepts every element and still gives
|
||||
* clients that require `items` an item schema.
|
||||
*/
|
||||
const reducedItemSchema = (
|
||||
alternatives: readonly ScopedSchema[] | undefined,
|
||||
references: JsonSchemaReferences,
|
||||
depth = 1,
|
||||
): Record<string, unknown> => {
|
||||
if (
|
||||
alternatives === undefined ||
|
||||
alternatives.length === 0 ||
|
||||
depth > REDUCED_ITEM_MAX_DEPTH
|
||||
)
|
||||
return {};
|
||||
const types: string[] = [];
|
||||
for (const { schema, scope } of alternatives) {
|
||||
const detected = detectedTypes(schema, references, scope);
|
||||
if (detected === undefined || detected.length === 0) return {};
|
||||
types.push(...detected);
|
||||
}
|
||||
const unique = [...new Set(types)];
|
||||
if (!unique.includes("array"))
|
||||
return { type: unique.length === 1 ? unique[0] : unique };
|
||||
if (unique.length > 1) return {};
|
||||
const nested: ScopedSchema[] = [];
|
||||
for (const { schema, scope } of alternatives) {
|
||||
const items = arrayItemAlternatives(schema, references, scope);
|
||||
if (items === undefined) return { type: "array", items: {} };
|
||||
nested.push(...items);
|
||||
}
|
||||
return {
|
||||
type: "array",
|
||||
items: reducedItemSchema(nested, references, depth + 1),
|
||||
};
|
||||
};
|
||||
|
||||
/** Give homogeneous array-item unions the type required by strict gateways. */
|
||||
export const typeArrayItemUnions = (
|
||||
schema: Record<string, unknown>,
|
||||
): Record<string, unknown> => {
|
||||
const references = new JsonSchemaReferences(schema);
|
||||
const visit = (node: unknown): unknown => {
|
||||
const rebuilt = rebuildSchemaPositions(node, visit);
|
||||
if (!isSchemaObject(rebuilt) || rebuilt.type !== "array") return rebuilt;
|
||||
const items = rebuilt.items;
|
||||
if (
|
||||
!isSchemaObject(items) ||
|
||||
items.type !== undefined ||
|
||||
(!Array.isArray(items.oneOf) && !Array.isArray(items.anyOf))
|
||||
)
|
||||
return rebuilt;
|
||||
const types = detectedTypes(items, references, references.rootScope);
|
||||
return types?.length === 1
|
||||
? { ...rebuilt, items: { ...items, type: types[0] } }
|
||||
: rebuilt;
|
||||
};
|
||||
return visit(schema) as Record<string, unknown>;
|
||||
};
|
||||
|
||||
/**
|
||||
* Strict gateways reject some nested combinators, primitive enums without a
|
||||
* type, and array items whose union has no explicit type. Homogeneous item
|
||||
* unions gain their shared type; remaining incompatible shapes use the shallow
|
||||
* projection while canonical runtime validation stays intact.
|
||||
*/
|
||||
export const hasGatewayIncompatibleShape = (schema: unknown): boolean => {
|
||||
if (Array.isArray(schema)) return schema.some(hasGatewayIncompatibleShape);
|
||||
if (!isSchemaObject(schema)) return false;
|
||||
if (Array.isArray(schema.anyOf) && schema.anyOf.length === 0) return true;
|
||||
if (
|
||||
schema.type === "array" &&
|
||||
isSchemaObject(schema.items) &&
|
||||
schema.items.type === undefined &&
|
||||
(Array.isArray(schema.items.oneOf) || Array.isArray(schema.items.anyOf))
|
||||
)
|
||||
return true;
|
||||
if (Array.isArray(schema.required) && isSchemaObject(schema.properties)) {
|
||||
for (const name of schema.required) {
|
||||
const property = schema.properties[name];
|
||||
if (isSchemaObject(property) && containsGatewayCombinator(property))
|
||||
return true;
|
||||
}
|
||||
}
|
||||
if (
|
||||
Array.isArray(schema.enum) &&
|
||||
schema.enum.length > 0 &&
|
||||
schema.type === undefined
|
||||
) {
|
||||
const types = enumValueTypes(schema.enum);
|
||||
if (
|
||||
types.length === 1 &&
|
||||
types.every((type) =>
|
||||
["string", "number", "boolean", "null"].includes(type),
|
||||
)
|
||||
)
|
||||
return true;
|
||||
}
|
||||
return schemaChildrenAndMaps(schema).some(hasGatewayIncompatibleShape);
|
||||
};
|
||||
|
||||
const containsGatewayCombinator = (schema: unknown): boolean => {
|
||||
if (Array.isArray(schema)) return schema.some(containsGatewayCombinator);
|
||||
if (!isSchemaObject(schema)) return false;
|
||||
if (schema.oneOf !== undefined || schema.allOf !== undefined) return true;
|
||||
return schemaChildrenAndMaps(schema).some(containsGatewayCombinator);
|
||||
};
|
||||
|
||||
/** Traverse only schema-valued positions; property names remain opaque data. */
|
||||
const schemaChildrenAndMaps = (
|
||||
schema: Readonly<Record<string, unknown>>,
|
||||
): readonly unknown[] =>
|
||||
Object.entries(schema).flatMap(([key, value]) => {
|
||||
if (schemaMaps.has(key) && isSchemaObject(value))
|
||||
return Object.values(value);
|
||||
if (schemaArrays.has(key) && Array.isArray(value)) return value;
|
||||
if (schemaChildren.has(key)) return Array.isArray(value) ? value : [value];
|
||||
return [];
|
||||
});
|
||||
|
||||
/**
|
||||
* Present a reduced property's reference-resolved types. Reduced inputs omit
|
||||
* nested validation, but clients still require an item schema for an array
|
||||
* declaration, and Gemini accepts `items` only beside the single `array` type,
|
||||
* so a property that may be an array or another type lists typed branches.
|
||||
*/
|
||||
const reducedPropertyTypes = (
|
||||
value: unknown,
|
||||
references: JsonSchemaReferences,
|
||||
): Record<string, unknown> => {
|
||||
const types = detectedTypes(value, references, references.rootScope);
|
||||
if (types === undefined || types.length === 0) return {};
|
||||
if (!types.includes("array"))
|
||||
return { type: types.length === 1 ? types[0] : types };
|
||||
const array = {
|
||||
type: "array",
|
||||
items: reducedItemSchema(
|
||||
arrayItemAlternatives(value, references, references.rootScope),
|
||||
references,
|
||||
),
|
||||
};
|
||||
return types.length === 1
|
||||
? array
|
||||
: {
|
||||
anyOf: types.map((type) => (type === "array" ? array : { type })),
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* Present only the root property shape with per-property guidance. Used for
|
||||
* schemas whose validation structure alone exceeds the compact budget; the
|
||||
* canonical schema still rejects malformed calls server-side.
|
||||
*/
|
||||
export const reduceToShallowProperties = (
|
||||
schema: Record<string, unknown>,
|
||||
reducedAnnotation: string,
|
||||
): Record<string, unknown> => {
|
||||
const properties = isSchemaObject(schema.properties) ? schema.properties : {};
|
||||
const description = [
|
||||
typeof schema.description === "string" ? schema.description : undefined,
|
||||
...Object.entries(
|
||||
isSchemaObject(schema.dependentRequired)
|
||||
? schema.dependentRequired
|
||||
: isSchemaObject(schema.dependencies)
|
||||
? schema.dependencies
|
||||
: {},
|
||||
).flatMap(([property, partners]) =>
|
||||
Array.isArray(partners) &&
|
||||
partners.every((partner) => typeof partner === "string")
|
||||
? [
|
||||
`When ${property} is provided, also provide ${partners.join(" and ")}.`,
|
||||
]
|
||||
: [],
|
||||
),
|
||||
reducedAnnotation,
|
||||
]
|
||||
.filter((text) => text !== undefined)
|
||||
.join(" ");
|
||||
const references = new JsonSchemaReferences(schema);
|
||||
const shallow = Object.fromEntries(
|
||||
Object.entries(properties).map(([name, value]) => {
|
||||
const description =
|
||||
isSchemaObject(value) && typeof value.description === "string"
|
||||
? { description: value.description }
|
||||
: {};
|
||||
return [
|
||||
name,
|
||||
{ ...reducedPropertyTypes(value, references), ...description },
|
||||
];
|
||||
}),
|
||||
);
|
||||
return {
|
||||
...(typeof schema.$schema === "string" ? { $schema: schema.$schema } : {}),
|
||||
type: "object",
|
||||
description,
|
||||
properties: shallow,
|
||||
...(Array.isArray(schema.required)
|
||||
? { required: [...schema.required] }
|
||||
: {}),
|
||||
...(typeof schema.minProperties === "number"
|
||||
? { minProperties: schema.minProperties }
|
||||
: {}),
|
||||
...(schema.additionalProperties === false
|
||||
? { additionalProperties: false }
|
||||
: {}),
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,171 @@
|
||||
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { compactAdvertisedOutputSchema } from "./compactOutputSchema.js";
|
||||
|
||||
const serializedBytes = (value: unknown): number =>
|
||||
Buffer.byteLength(JSON.stringify(value), "utf8");
|
||||
|
||||
const annotatedSchema = {
|
||||
type: "object",
|
||||
description: "Root result guidance stays.",
|
||||
properties: {
|
||||
value: {
|
||||
type: "string",
|
||||
description: "Field guidance leaves.",
|
||||
examples: ["seed"],
|
||||
default: "seed",
|
||||
title: "Value",
|
||||
},
|
||||
nested: {
|
||||
type: "object",
|
||||
description: "Nested guidance leaves.",
|
||||
properties: {
|
||||
count: { type: "integer", description: "Leaves.", minimum: 1 },
|
||||
},
|
||||
required: ["count"],
|
||||
additionalProperties: false,
|
||||
},
|
||||
},
|
||||
required: ["value", "nested"],
|
||||
additionalProperties: false,
|
||||
} as const;
|
||||
|
||||
const referencedSchema = {
|
||||
$schema: "https://json-schema.org/draft/2020-12/schema",
|
||||
$id: "urn:rea:tool-output:sha256:fixture",
|
||||
type: "object",
|
||||
properties: {
|
||||
entries: { type: "array", items: { $ref: "#/$defs/entry" } },
|
||||
},
|
||||
required: ["entries"],
|
||||
$defs: {
|
||||
entry: {
|
||||
type: "object",
|
||||
properties: { name: { type: "string" } },
|
||||
required: ["name"],
|
||||
},
|
||||
},
|
||||
} as const;
|
||||
|
||||
const oversizedSchema = (): Record<string, unknown> => ({
|
||||
type: "object",
|
||||
description: "A complete result envelope.",
|
||||
properties: {
|
||||
envelope: { type: "string" },
|
||||
result: {
|
||||
type: "object",
|
||||
properties: Object.fromEntries(
|
||||
Array.from({ length: 200 }, (_, index) => [
|
||||
`field_${index}`,
|
||||
{ type: "string", description: `Field ${index}` },
|
||||
]),
|
||||
),
|
||||
required: Array.from({ length: 200 }, (_, index) => `field_${index}`),
|
||||
additionalProperties: false,
|
||||
},
|
||||
},
|
||||
required: ["envelope", "result"],
|
||||
additionalProperties: false,
|
||||
});
|
||||
|
||||
describe("compact advertised output schema", () => {
|
||||
it("drops nested annotation prose while keeping the root guidance and validation", () => {
|
||||
const { schema, presentation } = compactAdvertisedOutputSchema(
|
||||
structuredClone(annotatedSchema),
|
||||
4 * 1024,
|
||||
);
|
||||
expect(presentation).toBe("inline");
|
||||
expect(schema.description).toBe("Root result guidance stays.");
|
||||
const value = (schema.properties as Record<string, unknown>)
|
||||
.value as Record<string, unknown>;
|
||||
expect(value).toEqual({ type: "string" });
|
||||
const nested = (schema.properties as Record<string, unknown>)
|
||||
.nested as Record<string, unknown>;
|
||||
expect(nested.description).toBeUndefined();
|
||||
expect(nested.required).toEqual(["count"]);
|
||||
expect(schema.required).toEqual(["value", "nested"]);
|
||||
expect(schema.additionalProperties).toBe(false);
|
||||
});
|
||||
|
||||
it("inlines local definitions and drops the content-bound identity", () => {
|
||||
const { schema, presentation } = compactAdvertisedOutputSchema(
|
||||
structuredClone(referencedSchema),
|
||||
4 * 1024,
|
||||
);
|
||||
expect(presentation).toBe("inline");
|
||||
expect(schema.$defs).toBeUndefined();
|
||||
expect(schema.$id).toBeUndefined();
|
||||
expect(schema.$schema).toBe("https://json-schema.org/draft/2020-12/schema");
|
||||
const entries = (schema.properties as Record<string, unknown>)
|
||||
.entries as Record<string, unknown>;
|
||||
expect(entries.items).toMatchObject({ type: "object" });
|
||||
});
|
||||
|
||||
it("reduces an oversized contract to a widening root shape within budget", () => {
|
||||
const budgetBytes = 4 * 1024;
|
||||
const { schema, presentation } = compactAdvertisedOutputSchema(
|
||||
oversizedSchema(),
|
||||
budgetBytes,
|
||||
);
|
||||
expect(presentation).toBe("reduced");
|
||||
expect(serializedBytes(schema)).toBeLessThanOrEqual(budgetBytes);
|
||||
expect(schema.type).toBe("object");
|
||||
expect(schema.required).toEqual(["envelope", "result"]);
|
||||
expect(schema.additionalProperties).toBe(false);
|
||||
expect(schema.description).toContain("compact output schema profile");
|
||||
const result = (schema.properties as Record<string, unknown>)
|
||||
.result as Record<string, unknown>;
|
||||
expect(result.type).toBe("object");
|
||||
expect(result.properties).toBeUndefined();
|
||||
});
|
||||
|
||||
it("accepts a valid canonical result after reduction", () => {
|
||||
const budgetBytes = 4 * 1024;
|
||||
const { schema } = compactAdvertisedOutputSchema(
|
||||
oversizedSchema(),
|
||||
budgetBytes,
|
||||
);
|
||||
const validate = new Ajv2020({ strict: false }).compile(schema);
|
||||
const result = {
|
||||
envelope: "ok",
|
||||
result: Object.fromEntries(
|
||||
Array.from({ length: 200 }, (_, index) => [
|
||||
`field_${index}`,
|
||||
`value ${index}`,
|
||||
]),
|
||||
),
|
||||
};
|
||||
expect(validate(result)).toBe(true);
|
||||
// The advertisement is wider than the canonical contract: dropping nested
|
||||
// detail must never reject a valid result, but it can accept an invalid one.
|
||||
expect(validate({ envelope: "ok", result: { unknown: 1 } })).toBe(true);
|
||||
});
|
||||
|
||||
it("keeps a non-object root accepting every JSON value when it cannot fit", () => {
|
||||
const arraySchema = {
|
||||
type: "array",
|
||||
items: Object.fromEntries(
|
||||
Array.from({ length: 200 }, (_, index) => [
|
||||
`field_${index}`,
|
||||
{ type: "string", description: `Field ${index}` },
|
||||
]),
|
||||
),
|
||||
};
|
||||
const { schema, presentation } = compactAdvertisedOutputSchema(
|
||||
arraySchema,
|
||||
128,
|
||||
);
|
||||
expect(presentation).toBe("reduced");
|
||||
expect(serializedBytes(schema)).toBeLessThanOrEqual(128);
|
||||
const validate = new Ajv2020({ strict: false }).compile(schema);
|
||||
expect(validate(["anything"])).toBe(true);
|
||||
expect(validate({ also: "anything" })).toBe(true);
|
||||
});
|
||||
|
||||
it("is deterministic across equivalent inputs", () => {
|
||||
expect(compactAdvertisedOutputSchema(oversizedSchema(), 4 * 1024)).toEqual(
|
||||
compactAdvertisedOutputSchema(oversizedSchema(), 4 * 1024),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,95 @@
|
||||
import {
|
||||
COMPACT_SCHEMA_MAX_DEPTH,
|
||||
inlineWithinBudget,
|
||||
isSchemaObject,
|
||||
jsonValueSchema,
|
||||
reduceToShallowProperties,
|
||||
schemaDepth,
|
||||
serializedBytes,
|
||||
stripAnnotations,
|
||||
} from "./compactJsonSchema.js";
|
||||
|
||||
// Output schemas dominate a complete `tools/list` frame. The compact profile
|
||||
// bounds each advertised output contract so frame-capped hosts can finish the
|
||||
// handshake, while canonical result validation and `structuredContent` stay
|
||||
// unchanged. Every projection is a widening: a valid canonical result always
|
||||
// satisfies the advertised schema.
|
||||
|
||||
export type AdvertisedOutputSchemaPresentation = "inline" | "reduced";
|
||||
|
||||
/** Budget selected by the caller for compact MCP output-schema presentation. */
|
||||
export interface CompactOutputSchemaPresentation {
|
||||
readonly budgetBytes: number;
|
||||
}
|
||||
|
||||
export interface CompactedOutputSchema {
|
||||
readonly schema: Record<string, unknown>;
|
||||
readonly presentation: AdvertisedOutputSchemaPresentation;
|
||||
}
|
||||
|
||||
const OUTPUT_REDUCED_ANNOTATION =
|
||||
"Advertised in reduced form by the compact output schema profile; the server still validates complete canonical results.";
|
||||
|
||||
/**
|
||||
* Drop the canonical contract's content-bound `$id` and anchors. The compact
|
||||
* bytes differ from the canonical schema the identity was derived from, so
|
||||
* retaining that identity would misstate what the advertised document is.
|
||||
* References are already inlined, so no identity is needed to resolve them.
|
||||
*/
|
||||
const withoutContentBoundIdentity = (
|
||||
schema: Record<string, unknown>,
|
||||
): Record<string, unknown> => {
|
||||
const { $id: _id, $anchor: _anchor, ...advertised } = schema;
|
||||
return advertised;
|
||||
};
|
||||
|
||||
/**
|
||||
* Render an advertised output schema within a per-schema byte budget. The
|
||||
* complete validation structure is kept when it fits; otherwise the root
|
||||
* property shape and first-level types remain and nested detail is elided.
|
||||
* The registered canonical schema still validates every delivered result.
|
||||
*/
|
||||
export const compactAdvertisedOutputSchema = (
|
||||
schema: Record<string, unknown>,
|
||||
budgetBytes: number,
|
||||
): CompactedOutputSchema => {
|
||||
const inline = inlineWithinBudget(stripAnnotations(schema), budgetBytes);
|
||||
if (inline !== undefined && schemaDepth(inline) <= COMPACT_SCHEMA_MAX_DEPTH)
|
||||
return {
|
||||
schema: withoutContentBoundIdentity(inline),
|
||||
presentation: "inline",
|
||||
};
|
||||
// Every REA output contract is an object at the root. A non-object root still
|
||||
// advertises an accepting shape rather than a projection that could reject
|
||||
// valid canonical output.
|
||||
const reduced =
|
||||
schema.type === "object" || isSchemaObject(schema.properties)
|
||||
? reduceToShallowProperties(schema, OUTPUT_REDUCED_ANNOTATION)
|
||||
: jsonValueSchema();
|
||||
if (serializedBytes(reduced) <= budgetBytes)
|
||||
return {
|
||||
schema: withoutContentBoundIdentity(reduced),
|
||||
presentation: "reduced",
|
||||
};
|
||||
// A root description can itself be large; keep the advertisement inside the
|
||||
// budget instead of carrying it into a shape with no nested detail left.
|
||||
const minimal = reduceToShallowProperties(
|
||||
{ type: "object" },
|
||||
OUTPUT_REDUCED_ANNOTATION,
|
||||
);
|
||||
return {
|
||||
schema:
|
||||
serializedBytes(minimal) <= budgetBytes ? minimal : jsonValueSchema(),
|
||||
presentation: "reduced",
|
||||
};
|
||||
};
|
||||
|
||||
/** Return the JSON schema clients receive for the selected output presentation. */
|
||||
export const presentAdvertisedOutputSchema = (
|
||||
schema: Record<string, unknown>,
|
||||
compactPresentation: CompactOutputSchemaPresentation | undefined,
|
||||
): Record<string, unknown> =>
|
||||
compactPresentation === undefined
|
||||
? schema
|
||||
: compactAdvertisedOutputSchema(schema, compactPresentation.budgetBytes)
|
||||
.schema;
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { CatalogDigestProfiles } from "./catalogIdentity.js";
|
||||
|
||||
/**
|
||||
* Digests of the four built-in input/output schema profiles, recorded
|
||||
* Digests of the six built-in input/output schema profiles, recorded
|
||||
* from the compiled contracts by `scripts/record-catalog-digests.mjs` after
|
||||
* compilation. Source runs leave this undefined and compute them on first use.
|
||||
*/
|
||||
|
||||
@@ -70,7 +70,8 @@ export class DeferredEvidenceMcpServer extends EvidenceMcpServerCore {
|
||||
undefined,
|
||||
delivery,
|
||||
undefined,
|
||||
false,
|
||||
// `main.ts` only defers the exact full advertisement catalog.
|
||||
"full",
|
||||
catalog.identity,
|
||||
);
|
||||
for (const tool of catalog.tools) {
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
FULL_CATALOG_IDENTITY,
|
||||
createCatalogIdentity,
|
||||
} from "../catalogIdentity.js";
|
||||
import type { McpOutputSchemaProfile } from "../config/mcpOutputSchemaProfile.js";
|
||||
import type { ToolResultDelivery } from "./ToolResultDelivery.js";
|
||||
import {
|
||||
EvidenceMcpServerCore,
|
||||
@@ -23,7 +24,7 @@ export class EvidenceMcpServer extends EvidenceMcpServerCore {
|
||||
recordEvidence: EvidenceWriter["recordEvidence"] | undefined,
|
||||
delivery: ToolResultDelivery,
|
||||
compactPresentation: CompactInputSchemaPresentation | undefined = undefined,
|
||||
omitOutputSchemas = false,
|
||||
outputSchemaProfile: McpOutputSchemaProfile = "full",
|
||||
identity?: ReturnType<typeof createCatalogIdentity>,
|
||||
) {
|
||||
super(
|
||||
@@ -32,14 +33,11 @@ export class EvidenceMcpServer extends EvidenceMcpServerCore {
|
||||
recordEvidence,
|
||||
delivery,
|
||||
compactPresentation,
|
||||
omitOutputSchemas,
|
||||
outputSchemaProfile,
|
||||
identity ??
|
||||
(compactPresentation === undefined && !omitOutputSchemas
|
||||
(compactPresentation === undefined && outputSchemaProfile === "full"
|
||||
? FULL_CATALOG_IDENTITY
|
||||
: createCatalogIdentity(
|
||||
compactPresentation,
|
||||
omitOutputSchemas ? "off" : "full",
|
||||
)),
|
||||
: createCatalogIdentity(compactPresentation, outputSchemaProfile)),
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -30,6 +30,11 @@ import {
|
||||
presentAdvertisedInputSchema,
|
||||
type CompactInputSchemaPresentation,
|
||||
} from "../contracts/compactInputSchema.js";
|
||||
import { presentAdvertisedOutputSchema } from "../contracts/compactOutputSchema.js";
|
||||
import {
|
||||
compactPresentationForMcpOutputSchemaProfile,
|
||||
type McpOutputSchemaProfile,
|
||||
} from "../config/mcpOutputSchemaProfile.js";
|
||||
import type { createCatalogIdentity } from "../catalogIdentity.js";
|
||||
import { transformAdvertisedInputJsonSchema } from "../contracts/toolSchemaMetadata.js";
|
||||
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
|
||||
@@ -110,26 +115,48 @@ const replaceToolResultPayload = (
|
||||
};
|
||||
};
|
||||
|
||||
/** Remove only client-facing output schemas; SDK registrations stay canonical. */
|
||||
const omitOutputSchemasFromToolsList = (
|
||||
/**
|
||||
* Rewrite only client-facing output schemas; SDK registrations stay canonical.
|
||||
* `compact` bounds each advertised schema, while `off` omits it entirely.
|
||||
* The SDK memoises each tool's converted schema, so the caller passes a cache
|
||||
* keyed by that object; repeated listings reuse the same projection.
|
||||
*/
|
||||
const presentOutputSchemasForToolsList = (
|
||||
message: JSONRPCMessage,
|
||||
requestId: RequestId,
|
||||
profile: McpOutputSchemaProfile,
|
||||
compactedSchemas: WeakMap<Record<string, unknown>, Record<string, unknown>>,
|
||||
): JSONRPCMessage => {
|
||||
if (
|
||||
profile === "full" ||
|
||||
!isJSONRPCResultResponse(message) ||
|
||||
message.id !== requestId ||
|
||||
!isRecord(message.result) ||
|
||||
!Array.isArray(message.result.tools)
|
||||
)
|
||||
return message;
|
||||
const compactPresentation =
|
||||
compactPresentationForMcpOutputSchemaProfile(profile);
|
||||
return {
|
||||
...message,
|
||||
result: {
|
||||
...message.result,
|
||||
tools: message.result.tools.map((tool: unknown) => {
|
||||
if (!isRecord(tool)) return tool;
|
||||
const { outputSchema: _outputSchema, ...advertisedTool } = tool;
|
||||
return advertisedTool;
|
||||
if (compactPresentation === undefined) {
|
||||
const { outputSchema: _outputSchema, ...advertisedTool } = tool;
|
||||
return advertisedTool;
|
||||
}
|
||||
if (!isRecord(tool.outputSchema)) return tool;
|
||||
let outputSchema = compactedSchemas.get(tool.outputSchema);
|
||||
if (outputSchema === undefined) {
|
||||
outputSchema = presentAdvertisedOutputSchema(
|
||||
tool.outputSchema,
|
||||
compactPresentation,
|
||||
);
|
||||
compactedSchemas.set(tool.outputSchema, outputSchema);
|
||||
}
|
||||
return { ...tool, outputSchema };
|
||||
}),
|
||||
},
|
||||
} as JSONRPCMessage;
|
||||
@@ -214,7 +241,7 @@ export class EvidenceMcpServerCore extends RequestTrackingMcpServer {
|
||||
private recordEvidence: EvidenceWriter["recordEvidence"] | undefined,
|
||||
public delivery: ToolResultDelivery,
|
||||
compactPresentation: CompactInputSchemaPresentation | undefined = undefined,
|
||||
private readonly omitOutputSchemas = false,
|
||||
private readonly outputSchemaProfile: McpOutputSchemaProfile = "full",
|
||||
identity?: ReturnType<typeof createCatalogIdentity>,
|
||||
) {
|
||||
super(info, options);
|
||||
@@ -226,9 +253,9 @@ export class EvidenceMcpServerCore extends RequestTrackingMcpServer {
|
||||
}
|
||||
|
||||
/**
|
||||
* Compact advertised input schemas and optionally omit output-schema
|
||||
* advertisements. Canonical parsers and contracts remain registered; the
|
||||
* output-schema-off path validates successful structured results locally.
|
||||
* Compact advertised input schemas and bound or omit output-schema
|
||||
* advertisements. Canonical parsers and contracts remain registered; both
|
||||
* output profiles validate successful structured results locally.
|
||||
* The signatures mirror the SDK's registrations so call sites keep their
|
||||
* inferred handler argument types.
|
||||
*/
|
||||
@@ -286,7 +313,7 @@ export class EvidenceMcpServerCore extends RequestTrackingMcpServer {
|
||||
transport,
|
||||
() => this.recordEvidence,
|
||||
() => this.delivery,
|
||||
this.omitOutputSchemas,
|
||||
this.outputSchemaProfile,
|
||||
),
|
||||
);
|
||||
}
|
||||
@@ -309,13 +336,19 @@ class ErrorEvidenceTransport implements Transport {
|
||||
return this.currentDelivery();
|
||||
}
|
||||
|
||||
/** Bounded projections already computed for this server's budget. */
|
||||
private readonly compactedOutputSchemas = new WeakMap<
|
||||
Record<string, unknown>,
|
||||
Record<string, unknown>
|
||||
>();
|
||||
|
||||
constructor(
|
||||
private readonly transport: Transport,
|
||||
private readonly currentEvidenceWriter: () =>
|
||||
| EvidenceWriter["recordEvidence"]
|
||||
| undefined,
|
||||
private readonly currentDelivery: () => ToolResultDelivery,
|
||||
private readonly omitOutputSchemas: boolean,
|
||||
private readonly outputSchemaProfile: McpOutputSchemaProfile,
|
||||
) {
|
||||
if (transport.hasPerRequestStream !== undefined)
|
||||
this.hasPerRequestStream = transport.hasPerRequestStream;
|
||||
@@ -429,14 +462,19 @@ class ErrorEvidenceTransport implements Transport {
|
||||
? message.id
|
||||
: undefined;
|
||||
const tool = id === undefined ? undefined : this.pendingTools.get(id);
|
||||
const omitToolSchemas =
|
||||
this.omitOutputSchemas &&
|
||||
const rewritesOutputSchemas =
|
||||
this.outputSchemaProfile !== "full" &&
|
||||
id !== undefined &&
|
||||
this.pendingToolLists.has(id);
|
||||
try {
|
||||
const recovered = this.recoverProtocolError(message, tool);
|
||||
const advertised = omitToolSchemas
|
||||
? omitOutputSchemasFromToolsList(recovered, id)
|
||||
const advertised = rewritesOutputSchemas
|
||||
? presentOutputSchemasForToolsList(
|
||||
recovered,
|
||||
id,
|
||||
this.outputSchemaProfile,
|
||||
this.compactedOutputSchemas,
|
||||
)
|
||||
: recovered;
|
||||
if (
|
||||
isJSONRPCResultResponse(advertised) &&
|
||||
|
||||
+17
-15
@@ -101,7 +101,10 @@ import { EvidenceMcpServer } from "./EvidenceMcpServer.js";
|
||||
import { createToolResultDelivery } from "./toolResult.js";
|
||||
import type { ToolResultDelivery } from "./ToolResultDelivery.js";
|
||||
import { parseMcpResponseBudget } from "../config/mcpResponseBudget.js";
|
||||
import { parseMcpOutputSchemaProfile } from "../config/mcpOutputSchemaProfile.js";
|
||||
import {
|
||||
parseMcpOutputSchemaProfile,
|
||||
type McpOutputSchemaProfile,
|
||||
} from "../config/mcpOutputSchemaProfile.js";
|
||||
import {
|
||||
compactPresentationForMcpInputSchemaProfile,
|
||||
parseMcpInputSchemaProfile,
|
||||
@@ -408,7 +411,7 @@ const createMcpServer = (
|
||||
session: BinarySessionPort | undefined,
|
||||
delivery: ToolResultDelivery,
|
||||
compactPresentation: CompactInputSchemaPresentation | undefined,
|
||||
omitOutputSchemas: boolean,
|
||||
outputSchemaProfile: McpOutputSchemaProfile,
|
||||
): EvidenceMcpServer =>
|
||||
new EvidenceMcpServer(
|
||||
{
|
||||
@@ -428,11 +431,8 @@ const createMcpServer = (
|
||||
: (evidence) => session.recordEvidence(evidence),
|
||||
delivery,
|
||||
compactPresentation,
|
||||
omitOutputSchemas,
|
||||
createCatalogIdentity(
|
||||
compactPresentation,
|
||||
omitOutputSchemas ? "off" : "full",
|
||||
),
|
||||
outputSchemaProfile,
|
||||
createCatalogIdentity(compactPresentation, outputSchemaProfile),
|
||||
);
|
||||
|
||||
/**
|
||||
@@ -449,7 +449,9 @@ export const createServer = (
|
||||
const environment = snapshotEnvironment(options.environment ?? process.env);
|
||||
const analysisSettings =
|
||||
options.analysisSettings ?? new AnalysisSettings(environment);
|
||||
const omitOutputSchemas = outputSchemasOmitted(environment);
|
||||
const compactInputPresentation =
|
||||
selectCompactInputSchemaPresentation(environment);
|
||||
const selectedOutputSchemaProfile = outputSchemaProfile(environment);
|
||||
const providerEnvironment = captureProviderEnvironment(
|
||||
options.providerEnvironment,
|
||||
environment,
|
||||
@@ -516,8 +518,8 @@ export const createServer = (
|
||||
createMcpServer(
|
||||
session,
|
||||
delivery,
|
||||
selectCompactInputSchemaPresentation(environment),
|
||||
omitOutputSchemas,
|
||||
compactInputPresentation,
|
||||
selectedOutputSchemaProfile,
|
||||
);
|
||||
server.analysisSettings = analysisSettings;
|
||||
registerAnalysisSettingsTools(server, analysisSettings);
|
||||
@@ -528,8 +530,8 @@ export const createServer = (
|
||||
: (evidence) => session.recordEvidence(evidence),
|
||||
delivery,
|
||||
createCatalogIdentity(
|
||||
selectCompactInputSchemaPresentation(environment),
|
||||
omitOutputSchemas ? "off" : "full",
|
||||
compactInputPresentation,
|
||||
selectedOutputSchemaProfile,
|
||||
),
|
||||
);
|
||||
}
|
||||
@@ -882,14 +884,14 @@ const selectToolResultDelivery = (
|
||||
return createToolResultDelivery(configured.value);
|
||||
};
|
||||
|
||||
const outputSchemasOmitted = (
|
||||
const outputSchemaProfile = (
|
||||
environment: Readonly<NodeJS.ProcessEnv>,
|
||||
): boolean => {
|
||||
): McpOutputSchemaProfile => {
|
||||
const configured = parseMcpOutputSchemaProfile(
|
||||
environment.REA_MCP_OUTPUT_SCHEMA_PROFILE,
|
||||
);
|
||||
if (!configured.ok) throw configured.error;
|
||||
return configured.value === "off";
|
||||
return configured.value ?? "full";
|
||||
};
|
||||
|
||||
const selectCompactInputSchemaPresentation = (
|
||||
|
||||
@@ -63,6 +63,64 @@ const assertCliCatalogIdentity = async (
|
||||
}
|
||||
};
|
||||
|
||||
const catalogProfiles = [
|
||||
{
|
||||
name: "default compact input",
|
||||
environment: {
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: undefined,
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
|
||||
},
|
||||
identity: createCatalogIdentity(
|
||||
compactPresentationForMcpInputSchemaProfile(undefined),
|
||||
),
|
||||
},
|
||||
{
|
||||
name: "full",
|
||||
environment: {
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
|
||||
},
|
||||
identity: FULL_CATALOG_IDENTITY,
|
||||
},
|
||||
{
|
||||
name: "compact",
|
||||
environment: {
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
|
||||
},
|
||||
identity: createCatalogIdentity(
|
||||
compactPresentationForMcpInputSchemaProfile("compact"),
|
||||
),
|
||||
},
|
||||
{
|
||||
name: "output schemas off",
|
||||
environment: {
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "off",
|
||||
},
|
||||
identity: createCatalogIdentity(undefined, "off"),
|
||||
},
|
||||
{
|
||||
name: "output schemas compact",
|
||||
environment: {
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "compact",
|
||||
},
|
||||
identity: createCatalogIdentity(undefined, "compact"),
|
||||
},
|
||||
{
|
||||
name: "compact input and output schemas off",
|
||||
environment: {
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "off",
|
||||
},
|
||||
identity: createCatalogIdentity(
|
||||
compactPresentationForMcpInputSchemaProfile("compact"),
|
||||
"off",
|
||||
),
|
||||
},
|
||||
];
|
||||
|
||||
describe("production MCP doctor", () => {
|
||||
it("reuses immutable compiled profile digests and preserves custom-budget identity", async () => {
|
||||
// Exercise the generated module with Node, outside Vitest's source loader.
|
||||
@@ -79,7 +137,7 @@ describe("production MCP doctor", () => {
|
||||
assert.ok(Object.isFrozen(profiles));
|
||||
for (const input of ["full", "compact"]) {
|
||||
assert.ok(Object.isFrozen(profiles[input]));
|
||||
for (const output of ["full", "off"]) {
|
||||
for (const output of ["full", "compact", "off"]) {
|
||||
const identity = createCatalogIdentity(input === "compact" ? { budgetBytes } : undefined, output);
|
||||
assert.equal(identity.digests, profiles[input][output]);
|
||||
assert.ok(Object.isFrozen(identity.digests));
|
||||
@@ -124,55 +182,7 @@ describe("production MCP doctor", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
{
|
||||
name: "default compact input",
|
||||
environment: {
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: undefined,
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
|
||||
},
|
||||
identity: createCatalogIdentity(
|
||||
compactPresentationForMcpInputSchemaProfile(undefined),
|
||||
),
|
||||
},
|
||||
{
|
||||
name: "full",
|
||||
environment: {
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
|
||||
},
|
||||
identity: FULL_CATALOG_IDENTITY,
|
||||
},
|
||||
{
|
||||
name: "compact",
|
||||
environment: {
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
|
||||
},
|
||||
identity: createCatalogIdentity(
|
||||
compactPresentationForMcpInputSchemaProfile("compact"),
|
||||
),
|
||||
},
|
||||
{
|
||||
name: "output schemas off",
|
||||
environment: {
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "off",
|
||||
},
|
||||
identity: createCatalogIdentity(undefined, "off"),
|
||||
},
|
||||
{
|
||||
name: "compact input and output schemas off",
|
||||
environment: {
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "off",
|
||||
},
|
||||
identity: createCatalogIdentity(
|
||||
compactPresentationForMcpInputSchemaProfile("compact"),
|
||||
"off",
|
||||
),
|
||||
},
|
||||
])(
|
||||
it.each(catalogProfiles)(
|
||||
"verifies the $name catalog through production stdio and both CLI inventories",
|
||||
async ({ environment, identity }) => {
|
||||
const result = await runProductionMcpDoctor({
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||
import { z } from "zod";
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
@@ -6,24 +7,34 @@ import {
|
||||
TOOL_CONTRACTS,
|
||||
toolContract,
|
||||
} from "../../../src/contracts/toolContracts.js";
|
||||
import { COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES } from "../../../src/config/mcpOutputSchemaProfile.js";
|
||||
import { COMPACT_INPUT_SCHEMA_BUDGET_BYTES } from "../../../src/config/mcpInputSchemaProfile.js";
|
||||
import type { CompactInputSchemaPresentation } from "../../../src/contracts/compactInputSchema.js";
|
||||
import { EvidenceMcpServer } from "../../../src/server/EvidenceMcpServer.js";
|
||||
import { toolRegistrationOptions } from "../../../src/server/toolRegistrationOptions.js";
|
||||
import { createToolResultDelivery } from "../../../src/server/toolResult.js";
|
||||
import type { ToolContract } from "../../../src/contracts/toolContractTypes.js";
|
||||
import { FULL_CATALOG_IDENTITY } from "../../../src/catalogIdentity.js";
|
||||
import type { McpOutputSchemaProfile } from "../../../src/config/mcpOutputSchemaProfile.js";
|
||||
import { advertisedSchemaErrors } from "../../fixtures/mcpAdvertisedSchemaValidity.js";
|
||||
|
||||
const createServer = (omitOutputSchemas: boolean) =>
|
||||
const createServer = (
|
||||
outputSchemaProfile: McpOutputSchemaProfile,
|
||||
compactInputPresentation:
|
||||
| CompactInputSchemaPresentation
|
||||
| undefined = undefined,
|
||||
) =>
|
||||
new EvidenceMcpServer(
|
||||
{ name: "output-schema-fixture", version: "1" },
|
||||
{ capabilities: {} },
|
||||
undefined,
|
||||
createToolResultDelivery(10 * 1024 * 1024),
|
||||
undefined,
|
||||
omitOutputSchemas,
|
||||
compactInputPresentation,
|
||||
outputSchemaProfile,
|
||||
);
|
||||
|
||||
it("omits only output schema advertisements and keeps canonical MCP output validation", async () => {
|
||||
const server = createServer(true);
|
||||
const server = createServer("off");
|
||||
expect(server.catalogIdentity.digests.combined_sha256).not.toBe(
|
||||
FULL_CATALOG_IDENTITY.digests.combined_sha256,
|
||||
);
|
||||
@@ -86,9 +97,97 @@ it("omits only output schema advertisements and keeps canonical MCP output valid
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps a full catalog below a measured one-megabyte frame when output schemas are off", async () => {
|
||||
const measure = async (omitOutputSchemas: boolean): Promise<number> => {
|
||||
const server = createServer(omitOutputSchemas);
|
||||
it("bounds output schema advertisements and keeps canonical MCP output validation", async () => {
|
||||
const server = createServer("compact");
|
||||
expect(server.catalogIdentity.digests.combined_sha256).not.toBe(
|
||||
FULL_CATALOG_IDENTITY.digests.combined_sha256,
|
||||
);
|
||||
const [clientTransport, serverTransport] =
|
||||
InMemoryTransport.createLinkedPair();
|
||||
const client = new Client({ name: "compact-output-test", version: "1" });
|
||||
const contract = {
|
||||
...toolContract("comment"),
|
||||
name: "compact_output_schema_fixture",
|
||||
inputSchema: z.strictObject({}),
|
||||
outputSchema: z.strictObject({
|
||||
value: z.string().describe("Nested guidance leaves."),
|
||||
nested: z
|
||||
.object({ count: z.number().int() })
|
||||
.describe("Nested guidance leaves."),
|
||||
}),
|
||||
} satisfies ToolContract;
|
||||
server.registerTool(
|
||||
contract.name,
|
||||
toolRegistrationOptions(contract),
|
||||
async () => ({
|
||||
content: [{ type: "text" as const, text: "fixture" }],
|
||||
structuredContent: { value: "complete", nested: { count: 1 } },
|
||||
}),
|
||||
);
|
||||
const invalidContract = {
|
||||
...contract,
|
||||
name: "invalid_compact_output_schema_fixture",
|
||||
} satisfies ToolContract;
|
||||
server.registerTool(
|
||||
invalidContract.name,
|
||||
toolRegistrationOptions(invalidContract),
|
||||
async () => ({
|
||||
content: [{ type: "text" as const, text: "invalid fixture" }],
|
||||
structuredContent: { value: 42, nested: { count: 1 } },
|
||||
}),
|
||||
);
|
||||
|
||||
await server.connect(serverTransport);
|
||||
await client.connect(clientTransport);
|
||||
try {
|
||||
const tools = (await client.listTools()).tools;
|
||||
expect(tools).toHaveLength(2);
|
||||
const advertised = tools.find((tool) => tool.name === contract.name)!
|
||||
.outputSchema as Record<string, unknown>;
|
||||
expect(advertised.type).toBe("object");
|
||||
expect(advertised.$id).toBeUndefined();
|
||||
expect((advertised.properties as Record<string, unknown>).value).toEqual({
|
||||
type: "string",
|
||||
});
|
||||
expect(
|
||||
(advertised.properties as Record<string, unknown>).nested,
|
||||
).toMatchObject({ type: "object" });
|
||||
|
||||
const success = await client.callTool({
|
||||
name: contract.name,
|
||||
arguments: {},
|
||||
});
|
||||
expect(success.isError).not.toBe(true);
|
||||
expect(success.structuredContent).toEqual({
|
||||
value: "complete",
|
||||
nested: { count: 1 },
|
||||
});
|
||||
const invalid = await client.callTool({
|
||||
name: invalidContract.name,
|
||||
arguments: {},
|
||||
});
|
||||
expect(invalid.isError).toBe(true);
|
||||
expect(invalid.content).toEqual([
|
||||
expect.objectContaining({
|
||||
type: "text",
|
||||
text: expect.stringContaining("Output validation error"),
|
||||
}),
|
||||
]);
|
||||
} finally {
|
||||
await client.close();
|
||||
await server.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps the default compact-input catalog below a measured one-megabyte frame", async () => {
|
||||
const compactInput: CompactInputSchemaPresentation = {
|
||||
budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
|
||||
};
|
||||
const measure = async (
|
||||
profile: McpOutputSchemaProfile,
|
||||
inputPresentation: CompactInputSchemaPresentation | undefined,
|
||||
): Promise<number> => {
|
||||
const server = createServer(profile, inputPresentation);
|
||||
const [clientTransport, serverTransport] =
|
||||
InMemoryTransport.createLinkedPair();
|
||||
const client = new Client({ name: "catalog-size-test", version: "1" });
|
||||
@@ -104,6 +203,26 @@ it("keeps a full catalog below a measured one-megabyte frame when output schemas
|
||||
await client.connect(clientTransport);
|
||||
try {
|
||||
const listed = await client.listTools();
|
||||
if (profile === "compact") {
|
||||
// Validate the advertised documents as the client receives them, not
|
||||
// just their bytes: a projection that broke a reference or left an
|
||||
// invalid schema would otherwise still pass the budget assertion.
|
||||
expect(
|
||||
advertisedSchemaErrors(
|
||||
listed.tools.map((tool) => ({
|
||||
name: tool.name,
|
||||
inputSchema: tool.inputSchema,
|
||||
outputSchema: tool.outputSchema,
|
||||
})),
|
||||
),
|
||||
).toEqual([]);
|
||||
for (const tool of listed.tools) {
|
||||
expect(tool.outputSchema).toBeDefined();
|
||||
expect(
|
||||
Buffer.byteLength(JSON.stringify(tool.outputSchema), "utf8"),
|
||||
).toBeLessThanOrEqual(COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES);
|
||||
}
|
||||
}
|
||||
return Buffer.byteLength(JSON.stringify(listed), "utf8");
|
||||
} finally {
|
||||
await client.close();
|
||||
@@ -111,9 +230,120 @@ it("keeps a full catalog below a measured one-megabyte frame when output schemas
|
||||
}
|
||||
};
|
||||
|
||||
const fullBytes = await measure(false);
|
||||
const compactBytes = await measure(true);
|
||||
const fullBytes = await measure("full", undefined);
|
||||
const compactBytes = await measure("compact", compactInput);
|
||||
const offBytes = await measure("off", compactInput);
|
||||
const fullInputOffBytes = await measure("off", undefined);
|
||||
expect(fullBytes).toBeGreaterThan(1024 * 1024);
|
||||
expect(compactBytes).toBeLessThan(1024 * 1024);
|
||||
expect(offBytes).toBeLessThan(1024 * 1024);
|
||||
expect(fullInputOffBytes).toBeLessThan(1024 * 1024);
|
||||
expect(compactBytes).toBeLessThan(fullBytes);
|
||||
expect(compactBytes).toBeGreaterThan(offBytes);
|
||||
});
|
||||
|
||||
it("rejects canonical-invalid output that the reduced advertisement accepts", async () => {
|
||||
// Enough nested detail that the compact projection cannot inline the
|
||||
// canonical schema within the budget, so the advertised form is the shallow
|
||||
// root shape rather than the complete validation structure.
|
||||
const nestedDetail = Object.fromEntries(
|
||||
Array.from({ length: 150 }, (_, index) => [
|
||||
`nested_${String(index).padStart(3, "0")}`,
|
||||
z.string().optional(),
|
||||
]),
|
||||
);
|
||||
const outputSchema = z.strictObject({
|
||||
value: z.string(),
|
||||
payload: z.object({
|
||||
inner: z.string(),
|
||||
count: z.number().int(),
|
||||
...nestedDetail,
|
||||
}),
|
||||
});
|
||||
const server = createServer("compact");
|
||||
const [clientTransport, serverTransport] =
|
||||
InMemoryTransport.createLinkedPair();
|
||||
const client = new Client({ name: "reduced-output-test", version: "1" });
|
||||
const contract = {
|
||||
...toolContract("comment"),
|
||||
name: "reduced_output_schema_fixture",
|
||||
inputSchema: z.strictObject({}),
|
||||
outputSchema,
|
||||
} satisfies ToolContract;
|
||||
server.registerTool(
|
||||
contract.name,
|
||||
toolRegistrationOptions(contract),
|
||||
async () => ({
|
||||
content: [{ type: "text" as const, text: "fixture" }],
|
||||
structuredContent: {
|
||||
value: "complete",
|
||||
payload: { inner: "text", count: 1 },
|
||||
},
|
||||
}),
|
||||
);
|
||||
// `payload.inner` is a string canonically. The shallow advertisement keeps
|
||||
// only `payload: {type: "object"}`, so it accepts the malformed value.
|
||||
const malformed = {
|
||||
value: "complete",
|
||||
payload: { inner: 42, count: 1 },
|
||||
};
|
||||
const invalidContract = {
|
||||
...contract,
|
||||
name: "invalid_reduced_output_schema_fixture",
|
||||
} satisfies ToolContract;
|
||||
server.registerTool(
|
||||
invalidContract.name,
|
||||
toolRegistrationOptions(invalidContract),
|
||||
async () => ({
|
||||
content: [{ type: "text" as const, text: "invalid fixture" }],
|
||||
structuredContent: malformed,
|
||||
}),
|
||||
);
|
||||
|
||||
await server.connect(serverTransport);
|
||||
await client.connect(clientTransport);
|
||||
try {
|
||||
const tools = (await client.listTools()).tools;
|
||||
const advertised = tools.find((tool) => tool.name === contract.name)!
|
||||
.outputSchema as Record<string, unknown>;
|
||||
expect(advertised.description).toContain("reduced form");
|
||||
expect((advertised.properties as Record<string, unknown>).payload).toEqual({
|
||||
type: "object",
|
||||
});
|
||||
expect(
|
||||
Buffer.byteLength(JSON.stringify(advertised), "utf8"),
|
||||
).toBeLessThanOrEqual(COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES);
|
||||
// The regression this guards: the advertisement accepts a value the
|
||||
// canonical contract rejects, so validating results against the
|
||||
// advertisement instead of the contract would deliver malformed output.
|
||||
const validateAdvertised = new Ajv2020({
|
||||
strict: false,
|
||||
validateFormats: false,
|
||||
}).compile(advertised);
|
||||
expect(validateAdvertised(malformed)).toBe(true);
|
||||
|
||||
const success = await client.callTool({
|
||||
name: contract.name,
|
||||
arguments: {},
|
||||
});
|
||||
expect(success.isError).not.toBe(true);
|
||||
expect(success.structuredContent).toEqual({
|
||||
value: "complete",
|
||||
payload: { inner: "text", count: 1 },
|
||||
});
|
||||
const invalid = await client.callTool({
|
||||
name: invalidContract.name,
|
||||
arguments: {},
|
||||
});
|
||||
expect(invalid.isError).toBe(true);
|
||||
expect(invalid.content).toEqual([
|
||||
expect.objectContaining({
|
||||
type: "text",
|
||||
text: expect.stringContaining("Output validation error"),
|
||||
}),
|
||||
]);
|
||||
} finally {
|
||||
await client.close();
|
||||
await server.close();
|
||||
}
|
||||
});
|
||||
@@ -13,6 +13,7 @@ import {
|
||||
arrayItemSchemaError,
|
||||
unsupportedInputLiteralSchemaError,
|
||||
} from "../../fixtures/mcpClientSchemaConstraints.js";
|
||||
import { advertisedSchemaErrors } from "../../fixtures/mcpAdvertisedSchemaValidity.js";
|
||||
import { toolRegistrationOptions } from "../../../src/server/toolRegistrationOptions.js";
|
||||
import { servedCatalog } from "../../../src/application/environment/environmentState.js";
|
||||
import { servedFrom } from "../../fixtures/environmentStateServing.js";
|
||||
@@ -31,47 +32,6 @@ interface ToolSchemas {
|
||||
readonly annotations?: Record<string, unknown> | undefined;
|
||||
}
|
||||
|
||||
function schemaErrors(tools: readonly ToolSchemas[]): string[] {
|
||||
// Reuse Ajv's meta-schema validators, then clear each root's reference
|
||||
// namespace, including nested IDs, before validating the next schema.
|
||||
const ajv = new Ajv2020({
|
||||
strict: false,
|
||||
validateFormats: false,
|
||||
addUsedSchema: false,
|
||||
});
|
||||
return tools.flatMap((tool) =>
|
||||
["inputSchema", "outputSchema"].flatMap((kind) => {
|
||||
const schema =
|
||||
kind === "inputSchema" ? tool.inputSchema : tool.outputSchema;
|
||||
if (schema === undefined) return [];
|
||||
try {
|
||||
if (!ajv.validateSchema(schema))
|
||||
return [`${tool.name}.${kind}: ${ajv.errorsText(ajv.errors)}`];
|
||||
ajv.compile(schema);
|
||||
const clientError = arrayItemSchemaError(
|
||||
schema,
|
||||
`${tool.name}.${kind}`,
|
||||
);
|
||||
if (clientError !== undefined) return [clientError];
|
||||
if (kind === "inputSchema") {
|
||||
const literalError = unsupportedInputLiteralSchemaError(
|
||||
schema,
|
||||
`${tool.name}.${kind}`,
|
||||
);
|
||||
if (literalError !== undefined) return [literalError];
|
||||
}
|
||||
return [];
|
||||
} catch (cause: unknown) {
|
||||
return [
|
||||
`${tool.name}.${kind}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
];
|
||||
} finally {
|
||||
ajv.removeSchema();
|
||||
}
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
function expectStrictInputSchemaParity(
|
||||
tools: readonly ToolSchemas[],
|
||||
ajv: Ajv2020,
|
||||
@@ -484,7 +444,7 @@ describe("served environment catalogs", () => {
|
||||
);
|
||||
expect(tools.some((tool) => tool.name === "delete_books")).toBe(true);
|
||||
expect(
|
||||
schemaErrors(
|
||||
advertisedSchemaErrors(
|
||||
tools.map((tool) => ({
|
||||
name: tool.name,
|
||||
inputSchema: tool.inputSchema as unknown as Record<string, unknown>,
|
||||
@@ -534,7 +494,7 @@ describe("MCP JSON Schema validity", () => {
|
||||
expect(tools.map(({ name }) => name).sort()).toEqual(
|
||||
TOOL_CONTRACTS.map(({ name }) => name).sort(),
|
||||
);
|
||||
expect(schemaErrors(tools)).toEqual([]);
|
||||
expect(advertisedSchemaErrors(tools)).toEqual([]);
|
||||
expect(localInputReferences(tools)).toEqual([]);
|
||||
expect(recursiveReferences(tools)).toEqual([]);
|
||||
expectKnownAuthorityHints(tools);
|
||||
@@ -879,7 +839,7 @@ describe("MCP process input JSON Schema", () => {
|
||||
});
|
||||
|
||||
it("ships nonrecursive input and output schemas in the generated catalog", () => {
|
||||
expect(schemaErrors(GENERATED_MCP_TOOL_CATALOG)).toEqual([]);
|
||||
expect(advertisedSchemaErrors(GENERATED_MCP_TOOL_CATALOG)).toEqual([]);
|
||||
expect(recursiveReferences(GENERATED_MCP_TOOL_CATALOG)).toEqual([]);
|
||||
});
|
||||
|
||||
@@ -888,7 +848,7 @@ it("ships input schemas without local references in the generated catalog", () =
|
||||
});
|
||||
|
||||
it("rejects unresolved output-schema references even for uncalled tools", () => {
|
||||
const errors = schemaErrors([
|
||||
const errors = advertisedSchemaErrors([
|
||||
{
|
||||
name: "dangling",
|
||||
inputSchema: { type: "object" },
|
||||
@@ -901,7 +861,7 @@ it("rejects unresolved output-schema references even for uncalled tools", () =>
|
||||
|
||||
it("validates each tool independently when output schema IDs repeat", () => {
|
||||
const outputSchema = { $id: "urn:rea:test:shared-output", type: "string" };
|
||||
const errors = schemaErrors([
|
||||
const errors = advertisedSchemaErrors([
|
||||
{ name: "first", inputSchema: { type: "object" }, outputSchema },
|
||||
{
|
||||
name: "second",
|
||||
@@ -974,7 +934,7 @@ it.each([false, true])(
|
||||
"rejects references to another tool's nested IDs (earlier compilation fails: %s)",
|
||||
(invalidPrevious) => {
|
||||
const nestedId = "urn:rea:test:nested-output";
|
||||
const errors = schemaErrors([
|
||||
const errors = advertisedSchemaErrors([
|
||||
{
|
||||
name: "first",
|
||||
inputSchema: { type: "object" },
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES,
|
||||
compactPresentationForMcpOutputSchemaProfile,
|
||||
parseMcpOutputSchemaProfile,
|
||||
} from "../../../src/config/mcpOutputSchemaProfile.js";
|
||||
|
||||
describe("MCP output schema profile configuration", () => {
|
||||
it("accepts the full, compact, and off presentations", () => {
|
||||
expect(parseMcpOutputSchemaProfile(undefined)).toEqual({
|
||||
ok: true,
|
||||
value: undefined,
|
||||
});
|
||||
for (const value of ["full", "compact", "off"] as const)
|
||||
expect(parseMcpOutputSchemaProfile(value)).toEqual({
|
||||
ok: true,
|
||||
value,
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["", "FULL", "compact ", "off ", "minimal", "0"])(
|
||||
"rejects unknown profiles: %j",
|
||||
(value) => {
|
||||
const parsed = parseMcpOutputSchemaProfile(value);
|
||||
if (parsed.ok) throw new Error("Expected invalid profile");
|
||||
expect(parsed.error.message).toContain("REA_MCP_OUTPUT_SCHEMA_PROFILE");
|
||||
},
|
||||
);
|
||||
|
||||
it("selects a bounded presentation only for compact", () => {
|
||||
expect(compactPresentationForMcpOutputSchemaProfile("compact")).toEqual({
|
||||
budgetBytes: COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES,
|
||||
});
|
||||
expect(compactPresentationForMcpOutputSchemaProfile(undefined)).toBe(
|
||||
undefined,
|
||||
);
|
||||
expect(compactPresentationForMcpOutputSchemaProfile("full")).toBe(
|
||||
undefined,
|
||||
);
|
||||
expect(compactPresentationForMcpOutputSchemaProfile("off")).toBe(undefined);
|
||||
});
|
||||
});
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||
|
||||
import {
|
||||
arrayItemSchemaError,
|
||||
unsupportedInputLiteralSchemaError,
|
||||
} from "./mcpClientSchemaConstraints.js";
|
||||
|
||||
/** Advertised schema pair as a client receives it from `tools/list`. */
|
||||
export interface AdvertisedToolSchemas {
|
||||
readonly name: string;
|
||||
readonly inputSchema: Record<string, unknown>;
|
||||
readonly outputSchema?: Record<string, unknown> | undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate advertised schemas as clients receive them: Ajv's declared-dialect
|
||||
* meta-schema and reference resolution, then the constraints a real client
|
||||
* enforces. Compiling catches a compact projection that drops a referenced
|
||||
* definition or leaves a local `$ref` without its target, which a byte-budget
|
||||
* assertion alone would not.
|
||||
*/
|
||||
export function advertisedSchemaErrors(
|
||||
tools: readonly AdvertisedToolSchemas[],
|
||||
): string[] {
|
||||
// Reuse Ajv's meta-schema validators, then clear each root's reference
|
||||
// namespace, including nested IDs, before validating the next schema.
|
||||
const ajv = new Ajv2020({
|
||||
strict: false,
|
||||
validateFormats: false,
|
||||
addUsedSchema: false,
|
||||
// This check compiles schemas to detect invalid references but never runs
|
||||
// their validators. Generated-code optimization adds substantial work here
|
||||
// without checking another contract; actual SDK validators keep defaults.
|
||||
code: { optimize: false },
|
||||
});
|
||||
return tools.flatMap((tool) =>
|
||||
["inputSchema", "outputSchema"].flatMap((kind) => {
|
||||
const schema =
|
||||
kind === "inputSchema" ? tool.inputSchema : tool.outputSchema;
|
||||
if (schema === undefined) return [];
|
||||
try {
|
||||
if (!ajv.validateSchema(schema))
|
||||
return [`${tool.name}.${kind}: ${ajv.errorsText(ajv.errors)}`];
|
||||
ajv.compile(schema);
|
||||
const clientError = arrayItemSchemaError(
|
||||
schema,
|
||||
`${tool.name}.${kind}`,
|
||||
);
|
||||
if (clientError !== undefined) return [clientError];
|
||||
if (kind === "inputSchema") {
|
||||
const literalError = unsupportedInputLiteralSchemaError(
|
||||
schema,
|
||||
`${tool.name}.${kind}`,
|
||||
);
|
||||
if (literalError !== undefined) return [literalError];
|
||||
}
|
||||
return [];
|
||||
} catch (cause: unknown) {
|
||||
return [
|
||||
`${tool.name}.${kind}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
];
|
||||
} finally {
|
||||
ajv.removeSchema();
|
||||
}
|
||||
}),
|
||||
);
|
||||
}
|
||||
Reference in new issue
Block a user