mirror of
https://github.com/morluto/rea.git
synced 2026-10-26 17:35:11 +00:00
feat(mcp): support compact output schema advertisements (#7218)
Add opt-in bounded output-schema advertisements while preserving the full default, canonical server validation and complete structured content. Verify real stdio discovery under a 1 MiB receive cap, all 5,367 libcrypto export coordinates, complete Evidence parity, whole-task serialization estimates and resource bounds. Fixes #2512. Integrates #7054 by Akshad Jogi with original authorship retained. Co-authored-by: Akshad Jogi <akshadjogi@gmail.com>
This commit is contained in:
1 parent
e596a48251
commit
9085222ca1
22 files changed
+1821
-761
No files matched your search
@@ -96,6 +96,12 @@ complete advertised field guidance and nested validation details. Both profiles
|
|||||||
keep the full tool set and canonical server validation. See
|
keep the full tool set and canonical server validation. See
|
||||||
"Compact input schema profile" in the MCP contracts guide.
|
"Compact input schema profile" in the MCP contracts guide.
|
||||||
|
|
||||||
|
Complete output schemas are advertised by default. If your client caps the
|
||||||
|
complete `tools/list` frame, set `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact` to
|
||||||
|
bound each advertised output schema, or `off` to omit output schemas entirely.
|
||||||
|
Both settings keep canonical server-side validation and complete structured
|
||||||
|
results. See "Output schemas" in the MCP contracts guide.
|
||||||
|
|
||||||
### Ask your agent
|
### Ask your agent
|
||||||
|
|
||||||
```text
|
```text
|
||||||
|
|||||||
@@ -208,10 +208,13 @@ and Claude Code reports that autocompact is thrashing. To avoid this:
|
|||||||
The sizes above come from REA's own `tools/list`. A live Claude Code session
|
The sizes above come from REA's own `tools/list`. A live Claude Code session
|
||||||
against DeepSeek's API is unverified.
|
against DeepSeek's API is unverified.
|
||||||
|
|
||||||
For a client whose stdio reader rejects the complete catalog frame, add
|
For a client whose stdio reader rejects the complete catalog frame, set
|
||||||
`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "off"` to REA's `env` object and restart the
|
`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "compact"` to bound every advertised output
|
||||||
client. This omits only advertised output schemas; REA still validates complete
|
schema, or `"off"` to omit them, in REA's `env` object, then restart the client.
|
||||||
structured results against its canonical contracts. See [MCP output schema
|
Both keep canonical result validation and complete structured results; `compact`
|
||||||
|
keeps a widened root shape and `off` advertises nothing. On catalogs large
|
||||||
|
enough that full input schemas alone approach the frame limit, also keep the
|
||||||
|
input profile at its `compact` default. See [MCP output schema
|
||||||
profiles](mcp-contracts.md#output-schemas).
|
profiles](mcp-contracts.md#output-schemas).
|
||||||
|
|
||||||
## Kimi Code
|
## Kimi Code
|
||||||
|
|||||||
+29
-13
@@ -1367,21 +1367,37 @@ identity. Explicit schema IDs and relative external reference bases are
|
|||||||
preserved. This keeps every tool and validation rule in discovery and does not
|
preserved. This keeps every tool and validation rule in discovery and does not
|
||||||
change the SDK's catalog invalidation or availability checks.
|
change the SDK's catalog invalidation or availability checks.
|
||||||
|
|
||||||
The server identity's tool-schema digest includes both the canonical output
|
The server identity's tool-schema digest includes the selected output-schema
|
||||||
contracts and the selected output-schema profile. `full` and `off` therefore
|
profile and the advertised form that profile produces. `full`, `compact`, and
|
||||||
have distinct identities even though `off` keeps the same canonical output
|
`off` therefore have distinct identities. The `compact` digest covers the
|
||||||
validation on the server.
|
bounded schemas clients receive; `off` keeps the canonical output contracts in
|
||||||
|
the digest while advertising nothing.
|
||||||
|
|
||||||
Some stdio hosts reject a complete `tools/list` frame before they load any
|
Some stdio hosts reject a complete `tools/list` frame before they load any
|
||||||
tools. The opt-in `REA_MCP_OUTPUT_SCHEMA_PROFILE=off` setting omits `outputSchema`
|
tools. The opt-in `REA_MCP_OUTPUT_SCHEMA_PROFILE=compact` setting bounds every
|
||||||
from the advertised catalog for hosts with a small frame limit (the measured
|
advertised output schema to 4,096 bytes; `off` omits `outputSchema` entirely
|
||||||
pi-web-ui failure and catalog breakdown are in [#2512](https://github.com/morluto/rea/issues/2512)).
|
(the measured pi-web-ui failure and catalog breakdown are in
|
||||||
The default `full` profile is unchanged. With `off`, REA validates successful
|
[#2512](https://github.com/morluto/rea/issues/2512)). The default `full` profile
|
||||||
structured results against the same named canonical tool contract before
|
is unchanged. Both settings keep the same named canonical tool contract for
|
||||||
delivery; it still returns complete `structuredContent` and text results.
|
server-side result validation, and both still return complete
|
||||||
Only client-side schema discovery and validation are omitted, so hosts using
|
`structuredContent` and text results.
|
||||||
this profile should rely on REA's validation and retain their usual response
|
|
||||||
size limits.
|
`compact` keeps the complete validation structure when it fits the budget.
|
||||||
|
Otherwise it advertises the root property shape, first-level reference-resolved
|
||||||
|
types, first-level guidance, and the root's `required`/`additionalProperties`
|
||||||
|
constraints, with nested validation detail elided. Local references are inlined
|
||||||
|
within the budget, so the advertised form is self-contained and omits the
|
||||||
|
canonical contract's content-bound `$id`. The projection only widens: a valid
|
||||||
|
canonical result still satisfies the advertised schema, but a client validating
|
||||||
|
against it can accept a result the canonical contract rejects. REA still
|
||||||
|
validates every result before delivery. On catalogs large enough that full
|
||||||
|
input schemas alone approach the frame limit, keep the input profile at its
|
||||||
|
`compact` default as well; with full input schemas the advertisement is
|
||||||
|
dominated by the input side.
|
||||||
|
|
||||||
|
Only client-side schema discovery and validation are reduced or omitted, so
|
||||||
|
hosts using these profiles should rely on REA's validation and retain their
|
||||||
|
usual response size limits.
|
||||||
|
|
||||||
### Function comparison limitations
|
### Function comparison limitations
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,123 @@
|
|||||||
|
# Compact MCP output advertisements
|
||||||
|
|
||||||
|
## Question and scope
|
||||||
|
|
||||||
|
Can an SDK client with a 1 MiB receive buffer discover REA and finish an
|
||||||
|
export investigation without weakening canonical result validation or losing
|
||||||
|
Evidence? This verifies #2512 using the opt-in compact output advertisement
|
||||||
|
from #7054. The default output profile remains `full`; inputs are `compact`.
|
||||||
|
|
||||||
|
The implementation was integrated onto main `18dc78fa43175b9628b45b731df06c9321c3f730`.
|
||||||
|
Measured runtime revision: `650d483b98564d8dbe87d55ce6a77ec410608136`.
|
||||||
|
Both profiles use that same runtime, rather than two different analyzers.
|
||||||
|
The complete schema remains the server's validator; the bounded advertisement
|
||||||
|
is a widening and does not replace the delivered `structuredContent`.
|
||||||
|
|
||||||
|
## Completed public workflow and independent oracle
|
||||||
|
|
||||||
|
`mcp-output-profile.mjs` starts the packaged CLI over real stdio with the
|
||||||
|
installed MCP SDK client, an isolated account and an explicit receive cap.
|
||||||
|
It records initialize/discovery, two native export summaries (cold/warm),
|
||||||
|
selected first/last exports, complete Evidence publication, and a second
|
||||||
|
discovery after materialization. Named advertised schemas validate the calls.
|
||||||
|
After closing the server, it authenticates the complete exported bundle and
|
||||||
|
checks every export name/address against GNU `nm`. The complete sets of names
|
||||||
|
must agree; selected rows must retain their original source indices.
|
||||||
|
|
||||||
|
Linux x64, Node 24.22.1, npm 11.16.0. Real target:
|
||||||
|
`/usr/lib/x86_64-linux-gnu/libcrypto.so.3`, SHA-256
|
||||||
|
`aa01e614af634bc7c8b678c9250b5c4f613c61989bacf0a5d6aed6570e7dcabd`.
|
||||||
|
All 5,367 export names/addresses were corroborated. Both profiles produced the
|
||||||
|
same authenticated parent Evidence ID and complete-record digest. GNU `nm`
|
||||||
|
corroborates export coordinates, not behavioral equivalence.
|
||||||
|
|
||||||
|
| Measurement | Full output | Compact output |
|
||||||
|
| -------------------------------------------------- | ----------: | -------------: |
|
||||||
|
| Discovered tools | 229 | 229 |
|
||||||
|
| Serialized catalog result bytes | 3,639,545 | 554,792 |
|
||||||
|
| Catalog cl100k token estimate | 932,629 | 116,962 |
|
||||||
|
| Complete task wire estimate, one discovery | 939,165 | 123,501 |
|
||||||
|
| Executed wire estimate, including second discovery | 1,871,825 | 240,494 |
|
||||||
|
| Discovery after connection, ms | 292 | 1,886 |
|
||||||
|
| Repeated discovery, ms | 333 | 49 |
|
||||||
|
| Whole harness elapsed, s | 9.86 | 6.18 |
|
||||||
|
| Whole harness CPU, s | 11.96 | 7.66 |
|
||||||
|
| Sampled process-family peak RSS, GiB | 1.04 | 0.62 |
|
||||||
|
|
||||||
|
The SDK's real 1,048,576-byte receive cap rejects the full catalog before any
|
||||||
|
analysis call. The compact run succeeds under that cap; the successful full
|
||||||
|
control uses the SDK's normal 10 MiB cap. A small `/usr/bin/ls` pilot also
|
||||||
|
completed under 1 MiB, with all 15 export names corroborated. Neither public
|
||||||
|
workflow uses a fake provider or launches the inspected executable.
|
||||||
|
|
||||||
|
Whole harness costs include the client, server, complete publication, offline
|
||||||
|
bundle parsing/oracle, and cleanup. Each run has a fresh server; the second
|
||||||
|
inspection/discovery is warm. OS file caches were not flushed. Two CPU affinity
|
||||||
|
slots, a 3 GiB Node heap request and an 8 GiB process-family RSS guard bound the
|
||||||
|
runs. RSS is the sum sampled every 100 ms: shared pages can be counted twice
|
||||||
|
and short peaks can be missed. No run hit a resource guard or OOM, and cleanup
|
||||||
|
succeeded. One pair does not establish a general speed or memory improvement;
|
||||||
|
compact startup takes the eager path and its initial discovery is slower.
|
||||||
|
|
||||||
|
## What the token counts establish
|
||||||
|
|
||||||
|
Estimates use `tiktoken`'s `cl100k_base` over serialized JSON messages, including
|
||||||
|
initialization, request arguments, every needed reply and discovery. The
|
||||||
|
one-discovery value excludes only the explicit repeated catalog consistency
|
||||||
|
probe; the executed total includes it. Local complete-bundle bytes stay outside
|
||||||
|
the wire. These are transport serialization estimates, not actual model usage.
|
||||||
|
|
||||||
|
The names/descriptions/input-schema projection is byte-identical across both
|
||||||
|
profiles: 56,909 estimated tokens. A host forwarding only those fields gains no
|
||||||
|
model-context reduction from this output-profile setting. A host retaining
|
||||||
|
output contracts, a typed Code Mode catalog, or the entire wire has a different
|
||||||
|
cost. Native host forwarding and live model behavior were not measured here.
|
||||||
|
The demonstrated benefit is successful bounded-frame discovery with complete
|
||||||
|
analysis, plus the measured reduction in serialized output advertisements.
|
||||||
|
|
||||||
|
## Reproduction and retained failures
|
||||||
|
|
||||||
|
Build once, then use fresh output directories:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
npm run build:cached
|
||||||
|
node experiments/analysis-efficiency/mcp-output-profile.mjs \
|
||||||
|
--runtime . --profile full --target /usr/lib/x86_64-linux-gnu/libcrypto.so.3 \
|
||||||
|
--output /tmp/rea-output-full
|
||||||
|
node experiments/analysis-efficiency/mcp-output-profile.mjs \
|
||||||
|
--runtime . --profile compact --target /usr/lib/x86_64-linux-gnu/libcrypto.so.3 \
|
||||||
|
--receive-bytes 1048576 --output /tmp/rea-output-compact
|
||||||
|
```
|
||||||
|
|
||||||
|
Repeat the full command with `--receive-bytes 1048576` as a negative control;
|
||||||
|
expect a nonzero exit, a receive-buffer error in `report.json`, and no analysis
|
||||||
|
calls. Reports retain the catalog, transcript, complete bundle and server log.
|
||||||
|
The resource guard/profiler is external to this driver; choose host-appropriate
|
||||||
|
limits when reproducing. Token estimates can be reproduced from the saved
|
||||||
|
messages with the tokenizer above, counting each `message` once.
|
||||||
|
|
||||||
|
Two initial small pilot attempts used incomplete source selectors in this new
|
||||||
|
driver. REA correctly returned `invalid_request`; both failed receipts were
|
||||||
|
retained with successful cleanup. The driver was corrected to use the canonical
|
||||||
|
`source: {kind: "retained-evidence", evidence_id}` before the completed runs.
|
||||||
|
|
||||||
|
Initial focused checks ran concurrent files under two CPU slots. Two full
|
||||||
|
catalog compilation assertions exceeded their 30-second test timeouts; 46 other
|
||||||
|
MCP cases and 14 source cases passed. The representative assertion passed alone
|
||||||
|
in 22 seconds, and the complete file passed with one worker, but another run
|
||||||
|
after the main rebase exceeded the deadlines again. CPU contention alone did
|
||||||
|
not fully explain the variability.
|
||||||
|
|
||||||
|
A bounded probe over the five largest output schemas isolated AJV generated-code
|
||||||
|
optimization: compile-only checks took 4.75 seconds with optimization and 2.22
|
||||||
|
seconds without it. The schema-validity fixture now disables that code-generation
|
||||||
|
optimization because it never executes these validators. It still validates the
|
||||||
|
declared dialect, compiles every schema, isolates each reference namespace, and
|
||||||
|
checks malformed/unresolved references. Actual SDK validators retain their normal
|
||||||
|
options. Test deadlines and schema/reference assertions remain unchanged.
|
||||||
|
|
||||||
|
After rebasing onto main `7d36d172a`, both complete real workflows passed again
|
||||||
|
at integration runtime `f735451d9`, retaining the same export parent Evidence
|
||||||
|
and complete-record digest across profiles. The 1 MiB compact cap still passes.
|
||||||
|
The 20 schema/profile cases also pass with the default two workers after the
|
||||||
|
compile-only optimization change; check:fast and repository formatting pass.
|
||||||
@@ -0,0 +1,229 @@
|
|||||||
|
/** Public stdio investigation with real ELF exports and a selected receive cap. */
|
||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { execFile } from "node:child_process";
|
||||||
|
import { createHash } from "node:crypto";
|
||||||
|
import { createWriteStream } from "node:fs";
|
||||||
|
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join, resolve } from "node:path";
|
||||||
|
import { finished } from "node:stream/promises";
|
||||||
|
import { pathToFileURL } from "node:url";
|
||||||
|
import { parseArgs, promisify } from "node:util";
|
||||||
|
import { Client } from "@modelcontextprotocol/client";
|
||||||
|
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||||
|
import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs";
|
||||||
|
|
||||||
|
const { values } = parseArgs({
|
||||||
|
options: Object.fromEntries(
|
||||||
|
["runtime", "output", "profile", "target", "receive-bytes"].map((key) => [
|
||||||
|
key,
|
||||||
|
{ type: "string" },
|
||||||
|
]),
|
||||||
|
),
|
||||||
|
});
|
||||||
|
for (const key of ["runtime", "output", "target"]) {
|
||||||
|
assert.ok(values[key], `${key} is required`);
|
||||||
|
values[key] = resolve(values[key]);
|
||||||
|
}
|
||||||
|
assert.ok(["full", "compact"].includes(values.profile));
|
||||||
|
const receiveBytes = Number(values["receive-bytes"] ?? 10 * 1024 * 1024);
|
||||||
|
assert.ok(Number.isSafeInteger(receiveBytes) && receiveBytes > 0);
|
||||||
|
await mkdir(values.output);
|
||||||
|
const report = {
|
||||||
|
profile: values.profile,
|
||||||
|
receive_bytes: receiveBytes,
|
||||||
|
runtime: values.runtime,
|
||||||
|
target: values.target,
|
||||||
|
calls: [],
|
||||||
|
};
|
||||||
|
const transcript = [];
|
||||||
|
let account, client, transport, serverLog, serverLogFinished;
|
||||||
|
let phase = "initialize";
|
||||||
|
const began = performance.now();
|
||||||
|
try {
|
||||||
|
account = await mkdtemp(join(tmpdir(), "rea-output-profile-"));
|
||||||
|
report.target_sha256 = createHash("sha256")
|
||||||
|
.update(await readFile(values.target))
|
||||||
|
.digest("hex");
|
||||||
|
report.revision = (
|
||||||
|
await promisify(execFile)("git", [
|
||||||
|
"-C",
|
||||||
|
values.runtime,
|
||||||
|
"rev-parse",
|
||||||
|
"HEAD",
|
||||||
|
])
|
||||||
|
).stdout.trim();
|
||||||
|
transport = new StdioClientTransport({
|
||||||
|
command: process.execPath,
|
||||||
|
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
|
||||||
|
cwd: values.runtime,
|
||||||
|
maxBufferSize: receiveBytes,
|
||||||
|
env: {
|
||||||
|
PATH: process.env.PATH,
|
||||||
|
HOME: account,
|
||||||
|
USERPROFILE: account,
|
||||||
|
NODE_OPTIONS: process.env.NODE_OPTIONS ?? "--max-old-space-size=3072",
|
||||||
|
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
||||||
|
REA_MCP_OUTPUT_SCHEMA_PROFILE: values.profile,
|
||||||
|
},
|
||||||
|
stderr: "pipe",
|
||||||
|
});
|
||||||
|
const start = transport.start.bind(transport);
|
||||||
|
transport.start = async () => {
|
||||||
|
const onerror = transport.onerror;
|
||||||
|
transport.onerror = (cause) => {
|
||||||
|
(report.transport_errors ??= []).push(String(cause));
|
||||||
|
onerror?.(cause);
|
||||||
|
};
|
||||||
|
const receive = transport.onmessage;
|
||||||
|
transport.onmessage = (message) => {
|
||||||
|
transcript.push({ direction: "response", phase, message });
|
||||||
|
receive?.(message);
|
||||||
|
};
|
||||||
|
await start();
|
||||||
|
};
|
||||||
|
const send = transport.send.bind(transport);
|
||||||
|
transport.send = async (message) => {
|
||||||
|
transcript.push({ direction: "request", phase, message });
|
||||||
|
await send(message);
|
||||||
|
};
|
||||||
|
serverLog = createWriteStream(join(values.output, "server.log"));
|
||||||
|
serverLogFinished = finished(serverLog);
|
||||||
|
serverLogFinished.catch(() => {});
|
||||||
|
transport.stderr.pipe(serverLog);
|
||||||
|
client = new Client({
|
||||||
|
name: "rea-output-profile-investigation",
|
||||||
|
version: "1",
|
||||||
|
});
|
||||||
|
await client.connect(transport);
|
||||||
|
phase = "discovery-cold";
|
||||||
|
const discoveryBegan = performance.now();
|
||||||
|
const catalog = await client.listTools({}, { timeout: 15000 });
|
||||||
|
report.discovery_ms = performance.now() - discoveryBegan;
|
||||||
|
report.catalog_bytes = Buffer.byteLength(JSON.stringify(catalog));
|
||||||
|
report.tools = catalog.tools.length;
|
||||||
|
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
|
||||||
|
const definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));
|
||||||
|
const call = async (name, args, label) => {
|
||||||
|
phase = label;
|
||||||
|
const at = performance.now();
|
||||||
|
const result = await client.callTool(
|
||||||
|
{ name, arguments: args },
|
||||||
|
{ timeout: 90000, toolDefinition: definitions.get(name) },
|
||||||
|
);
|
||||||
|
assert.notEqual(
|
||||||
|
result.isError,
|
||||||
|
true,
|
||||||
|
`${name}: ${JSON.stringify(result).slice(0, 500)}`,
|
||||||
|
);
|
||||||
|
report.calls.push({ name, phase, elapsed_ms: performance.now() - at });
|
||||||
|
return result.structuredContent;
|
||||||
|
};
|
||||||
|
const first = await call(
|
||||||
|
"inspect_native_exports",
|
||||||
|
{ path: values.target, detail: "summary" },
|
||||||
|
"exports-cold",
|
||||||
|
);
|
||||||
|
const repeated = await call(
|
||||||
|
"inspect_native_exports",
|
||||||
|
{ path: values.target, detail: "summary" },
|
||||||
|
"exports-warm",
|
||||||
|
);
|
||||||
|
const parentId = first.normalized_result.parent_evidence_id;
|
||||||
|
assert.equal(repeated.normalized_result.parent_evidence_id, parentId);
|
||||||
|
report.parent_evidence_id = parentId;
|
||||||
|
const count = first.normalized_result.summary.export_count;
|
||||||
|
assert.ok(count > 1);
|
||||||
|
const page = await call(
|
||||||
|
"inspect_analysis_view",
|
||||||
|
{
|
||||||
|
source: { kind: "retained-evidence", evidence_id: parentId },
|
||||||
|
view: {
|
||||||
|
kind: "native-exports",
|
||||||
|
collection: "exports",
|
||||||
|
indices: [0, count - 1],
|
||||||
|
fields: ["name"],
|
||||||
|
limit: 2,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"selected-exports",
|
||||||
|
);
|
||||||
|
assert.deepEqual(
|
||||||
|
page.normalized_result.items.map((item) => item.source_index),
|
||||||
|
[0, count - 1],
|
||||||
|
);
|
||||||
|
const bundlePath = join(values.output, "complete-evidence.json");
|
||||||
|
await call("export_evidence_bundle", { path: bundlePath }, "complete-export");
|
||||||
|
phase = "discovery-warm";
|
||||||
|
const warmAt = performance.now();
|
||||||
|
assert.deepEqual(await client.listTools(), catalog);
|
||||||
|
report.warm_discovery_ms = performance.now() - warmAt;
|
||||||
|
// Release the server ledger before parsing the complete exported records.
|
||||||
|
await client.close();
|
||||||
|
await transport.close();
|
||||||
|
const { parseEvidenceBundle } = await import(
|
||||||
|
pathToFileURL(join(values.runtime, "dist/domain/evidenceBundle.js")).href
|
||||||
|
);
|
||||||
|
const bundle = parseEvidenceBundle(
|
||||||
|
JSON.parse(await readFile(bundlePath, "utf8")),
|
||||||
|
);
|
||||||
|
const parent = bundle.records.find(
|
||||||
|
(record) => record.evidence_id === parentId,
|
||||||
|
);
|
||||||
|
assert.ok(parent);
|
||||||
|
const rows = parent.normalized_result.exports;
|
||||||
|
assert.equal(rows.length, count);
|
||||||
|
for (const row of page.normalized_result.items) {
|
||||||
|
assert.equal(row.name, rows[row.source_index].name);
|
||||||
|
}
|
||||||
|
const nm = await promisify(execFile)(
|
||||||
|
"nm",
|
||||||
|
[
|
||||||
|
"--dynamic",
|
||||||
|
"--defined-only",
|
||||||
|
"--without-symbol-versions",
|
||||||
|
"--format=posix",
|
||||||
|
values.target,
|
||||||
|
],
|
||||||
|
{ maxBuffer: 16 * 1024 * 1024 },
|
||||||
|
);
|
||||||
|
const symbols = nm.stdout
|
||||||
|
.trim()
|
||||||
|
.split("\n")
|
||||||
|
.map((line) => line.split(/\s+/u));
|
||||||
|
const names = new Set(symbols.map(([name]) => name));
|
||||||
|
const coordinates = new Set(
|
||||||
|
symbols.map(([name, , value]) => `${name}:${BigInt(`0x${value}`)}`),
|
||||||
|
);
|
||||||
|
for (const row of rows)
|
||||||
|
assert.ok(
|
||||||
|
coordinates.has(`${row.name}:${BigInt(row.value)}`),
|
||||||
|
`GNU nm did not corroborate ${row.name} at ${row.value}`,
|
||||||
|
);
|
||||||
|
const observed = new Set(rows.map((row) => row.name));
|
||||||
|
for (const name of names)
|
||||||
|
assert.ok(observed.has(name), `REA omitted GNU nm export ${name}`);
|
||||||
|
report.export_count = count;
|
||||||
|
report.nm_unique_names = names.size;
|
||||||
|
report.facts_sha256 = createHash("sha256")
|
||||||
|
.update(JSON.stringify(parent))
|
||||||
|
.digest("hex");
|
||||||
|
report.outcome = "passed";
|
||||||
|
} catch (cause) {
|
||||||
|
report.outcome = "failed";
|
||||||
|
report.failure = String(cause);
|
||||||
|
process.exitCode = 1;
|
||||||
|
} finally {
|
||||||
|
report.elapsed_ms = performance.now() - began;
|
||||||
|
const published = await finalizeRetainedQueryReceipt({
|
||||||
|
report,
|
||||||
|
client,
|
||||||
|
transport,
|
||||||
|
account,
|
||||||
|
serverLog,
|
||||||
|
serverLogFinished,
|
||||||
|
output: values.output,
|
||||||
|
transcript,
|
||||||
|
});
|
||||||
|
if (!published) process.exitCode = 1;
|
||||||
|
}
|
||||||
@@ -13,7 +13,7 @@ const { createCatalogDigestProfiles } = await import(
|
|||||||
pathToFileURL(join(compiledRoot, "catalogIdentity.js")).href
|
pathToFileURL(join(compiledRoot, "catalogIdentity.js")).href
|
||||||
);
|
);
|
||||||
// Retain only hashes, with every shared record frozen. Schemas remain owned by
|
// Retain only hashes, with every shared record frozen. Schemas remain owned by
|
||||||
// the compiled contracts and are converted once for all four identities.
|
// the compiled contracts and are converted once for all six identities.
|
||||||
const profiles = Object.entries(createCatalogDigestProfiles())
|
const profiles = Object.entries(createCatalogDigestProfiles())
|
||||||
.map(
|
.map(
|
||||||
([input, outputs]) =>
|
([input, outputs]) =>
|
||||||
|
|||||||
+33
-3
@@ -16,7 +16,15 @@ import {
|
|||||||
presentAdvertisedInputSchema,
|
presentAdvertisedInputSchema,
|
||||||
type CompactInputSchemaPresentation,
|
type CompactInputSchemaPresentation,
|
||||||
} from "./contracts/compactInputSchema.js";
|
} from "./contracts/compactInputSchema.js";
|
||||||
import type { McpOutputSchemaProfile } from "./config/mcpOutputSchemaProfile.js";
|
import {
|
||||||
|
presentAdvertisedOutputSchema,
|
||||||
|
type CompactOutputSchemaPresentation,
|
||||||
|
} from "./contracts/compactOutputSchema.js";
|
||||||
|
import {
|
||||||
|
COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES,
|
||||||
|
compactPresentationForMcpOutputSchemaProfile,
|
||||||
|
type McpOutputSchemaProfile,
|
||||||
|
} from "./config/mcpOutputSchemaProfile.js";
|
||||||
import {
|
import {
|
||||||
COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
|
COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
|
||||||
compactPresentationForMcpInputSchemaProfile,
|
compactPresentationForMcpInputSchemaProfile,
|
||||||
@@ -57,15 +65,28 @@ const createToolCatalog = (contracts: readonly ToolContract[]) =>
|
|||||||
const presentToolCatalog = (
|
const presentToolCatalog = (
|
||||||
catalog: ReturnType<typeof createToolCatalog>,
|
catalog: ReturnType<typeof createToolCatalog>,
|
||||||
presentation: CompactInputSchemaPresentation | undefined,
|
presentation: CompactInputSchemaPresentation | undefined,
|
||||||
|
outputPresentation: CompactOutputSchemaPresentation | undefined = undefined,
|
||||||
) =>
|
) =>
|
||||||
presentation === undefined
|
presentation === undefined && outputPresentation === undefined
|
||||||
? catalog
|
? catalog
|
||||||
: catalog.map((tool) => ({
|
: catalog.map((tool) => ({
|
||||||
...tool,
|
...tool,
|
||||||
|
...(presentation === undefined
|
||||||
|
? {}
|
||||||
|
: {
|
||||||
input_schema: presentAdvertisedInputSchema(
|
input_schema: presentAdvertisedInputSchema(
|
||||||
tool.input_schema,
|
tool.input_schema,
|
||||||
presentation,
|
presentation,
|
||||||
),
|
),
|
||||||
|
}),
|
||||||
|
...(outputPresentation === undefined
|
||||||
|
? {}
|
||||||
|
: {
|
||||||
|
output_schema: presentAdvertisedOutputSchema(
|
||||||
|
tool.output_schema,
|
||||||
|
outputPresentation,
|
||||||
|
),
|
||||||
|
}),
|
||||||
}));
|
}));
|
||||||
|
|
||||||
const createPromptCatalog = (contracts: readonly PromptContract[]) =>
|
const createPromptCatalog = (contracts: readonly PromptContract[]) =>
|
||||||
@@ -124,6 +145,7 @@ export const createCatalogDigests = (
|
|||||||
presentToolCatalog(
|
presentToolCatalog(
|
||||||
createToolCatalog(sortByName(contracts)),
|
createToolCatalog(sortByName(contracts)),
|
||||||
compactPresentation,
|
compactPresentation,
|
||||||
|
compactPresentationForMcpOutputSchemaProfile(outputSchemaProfile),
|
||||||
),
|
),
|
||||||
createPromptCatalog(promptContracts),
|
createPromptCatalog(promptContracts),
|
||||||
outputSchemaProfile,
|
outputSchemaProfile,
|
||||||
@@ -137,7 +159,7 @@ export type CatalogDigestProfiles = Readonly<
|
|||||||
>
|
>
|
||||||
>;
|
>;
|
||||||
|
|
||||||
/** Build-time identities share schema conversion across the four supported profiles. */
|
/** Build-time identities share schema conversion across the six supported profiles. */
|
||||||
export const createCatalogDigestProfiles = (
|
export const createCatalogDigestProfiles = (
|
||||||
contracts: readonly ToolContract[] = TOOL_CONTRACTS,
|
contracts: readonly ToolContract[] = TOOL_CONTRACTS,
|
||||||
promptContracts: readonly PromptContract[] = PROMPT_CONTRACTS,
|
promptContracts: readonly PromptContract[] = PROMPT_CONTRACTS,
|
||||||
@@ -147,8 +169,16 @@ export const createCatalogDigestProfiles = (
|
|||||||
budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
|
budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
|
||||||
});
|
});
|
||||||
const prompts = createPromptCatalog(promptContracts);
|
const prompts = createPromptCatalog(promptContracts);
|
||||||
|
const compactOutput: CompactOutputSchemaPresentation = {
|
||||||
|
budgetBytes: COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES,
|
||||||
|
};
|
||||||
const outputProfiles = (catalog: typeof full) => ({
|
const outputProfiles = (catalog: typeof full) => ({
|
||||||
full: digestCatalogs(catalog, prompts, "full"),
|
full: digestCatalogs(catalog, prompts, "full"),
|
||||||
|
compact: digestCatalogs(
|
||||||
|
presentToolCatalog(catalog, undefined, compactOutput),
|
||||||
|
prompts,
|
||||||
|
"compact",
|
||||||
|
),
|
||||||
off: digestCatalogs(catalog, prompts, "off"),
|
off: digestCatalogs(catalog, prompts, "off"),
|
||||||
});
|
});
|
||||||
return { full: outputProfiles(full), compact: outputProfiles(compact) };
|
return { full: outputProfiles(full), compact: outputProfiles(compact) };
|
||||||
|
|||||||
@@ -3,15 +3,35 @@ import { z } from "zod";
|
|||||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||||
import { err, ok, type Result } from "../domain/result.js";
|
import { err, ok, type Result } from "../domain/result.js";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Per-schema budget for compact output advertisements. Output contracts are
|
||||||
|
* larger than input contracts across the catalog, and the complete frame is
|
||||||
|
* what a frame-capped host rejects. Measured on the complete contract catalog
|
||||||
|
* with the default compact input profile, 4 KiB bounds the output
|
||||||
|
* advertisement to ~276 KiB and the complete `tools/list` response to
|
||||||
|
* ~577 KiB, leaving margin under the one-megabyte frame reported in #2512. A
|
||||||
|
* larger budget admits substantially more full contracts; a much smaller one
|
||||||
|
* drops first-level property guidance for nearly every tool.
|
||||||
|
*/
|
||||||
|
export const COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES = 4 * 1024;
|
||||||
|
|
||||||
/** Select whether tools/list carries the large output contracts. */
|
/** Select whether tools/list carries the large output contracts. */
|
||||||
export const mcpOutputSchemaProfileSchema = z.enum(["full", "off"]);
|
export const mcpOutputSchemaProfileSchema = z.enum(["full", "compact", "off"]);
|
||||||
|
|
||||||
export type McpOutputSchemaProfile = z.infer<
|
export type McpOutputSchemaProfile = z.infer<
|
||||||
typeof mcpOutputSchemaProfileSchema
|
typeof mcpOutputSchemaProfileSchema
|
||||||
>;
|
>;
|
||||||
|
|
||||||
|
/** Resolve the compact output advertisement, preserving every other choice. */
|
||||||
|
export const compactPresentationForMcpOutputSchemaProfile = (
|
||||||
|
profile: McpOutputSchemaProfile | undefined,
|
||||||
|
): { readonly budgetBytes: number } | undefined =>
|
||||||
|
profile === "compact"
|
||||||
|
? { budgetBytes: COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES }
|
||||||
|
: undefined;
|
||||||
|
|
||||||
const profileConstraint =
|
const profileConstraint =
|
||||||
'REA_MCP_OUTPUT_SCHEMA_PROFILE must be "full" (default) or "off". The off profile omits output schemas from tools/list while REA validates every successful result against its canonical tool contract and returns complete structured content.';
|
'REA_MCP_OUTPUT_SCHEMA_PROFILE must be "full" (default), "compact", or "off". The compact profile advertises every output schema within a 4,096-byte budget; the off profile omits output schemas. Both keep canonical server-side validation and complete structured content.';
|
||||||
|
|
||||||
/** Parse the output schema advertisement selection at server composition. */
|
/** Parse the output schema advertisement selection at server composition. */
|
||||||
export const parseMcpOutputSchemaProfile = (
|
export const parseMcpOutputSchemaProfile = (
|
||||||
|
|||||||
@@ -1,48 +1,18 @@
|
|||||||
import {
|
import {
|
||||||
JsonSchemaReferences,
|
COMPACT_SCHEMA_MAX_DEPTH,
|
||||||
type JsonSchemaScope,
|
hasGatewayIncompatibleShape,
|
||||||
} from "./jsonSchemaReferences.js";
|
inlineWithinBudget,
|
||||||
import {
|
reduceToShallowProperties,
|
||||||
schemaArrays,
|
schemaDepth,
|
||||||
schemaChildren,
|
serializedBytes,
|
||||||
schemaMaps,
|
stripAnnotations,
|
||||||
schemaLiteralValues,
|
typeArrayItemUnions,
|
||||||
} from "./jsonSchemaKeywords.js";
|
} from "./compactJsonSchema.js";
|
||||||
|
|
||||||
// Kimi expands schema-local references before sending tools to Moonshot.
|
// Kimi expands schema-local references before sending tools to Moonshot.
|
||||||
// Compact advertisements must fit after expansion, independently of client
|
// Compact advertisements must fit after expansion, independently of client
|
||||||
// reference handling. Canonical server-side validation remains unchanged.
|
// reference handling. Canonical server-side validation remains unchanged.
|
||||||
|
|
||||||
/** Annotation keys that carry guidance but no validation semantics. */
|
|
||||||
const annotationKeys = new Set(["description", "examples", "default", "title"]);
|
|
||||||
|
|
||||||
const definitionKeys = new Set(["$defs", "definitions"]);
|
|
||||||
// An empty schema accepts every JSON value. Kimi otherwise infers string.
|
|
||||||
const jsonValueTypes = [
|
|
||||||
"null",
|
|
||||||
"boolean",
|
|
||||||
"object",
|
|
||||||
"array",
|
|
||||||
"number",
|
|
||||||
"string",
|
|
||||||
];
|
|
||||||
// An empty item schema accepts every JSON value, including arbitrarily nested
|
|
||||||
// arrays, while satisfying clients that require `items` for array types.
|
|
||||||
// Gemini rejects `items` beside a type list; the compact catalog test fails
|
|
||||||
// if a tool would advertise this presentation.
|
|
||||||
const jsonValueSchema = () => ({ type: [...jsonValueTypes], items: {} });
|
|
||||||
|
|
||||||
const enumValueTypes = (values: readonly unknown[]): readonly string[] => [
|
|
||||||
...new Set(
|
|
||||||
values.map((value) => {
|
|
||||||
if (value === null) return "null";
|
|
||||||
if (Array.isArray(value)) return "array";
|
|
||||||
if (typeof value === "object") return "object";
|
|
||||||
return typeof value;
|
|
||||||
}),
|
|
||||||
),
|
|
||||||
];
|
|
||||||
|
|
||||||
export type AdvertisedInputSchemaPresentation = "inline" | "reduced";
|
export type AdvertisedInputSchemaPresentation = "inline" | "reduced";
|
||||||
|
|
||||||
/** Budget selected by the caller for compact MCP input-schema presentation. */
|
/** Budget selected by the caller for compact MCP input-schema presentation. */
|
||||||
@@ -55,561 +25,9 @@ export interface CompactedInputSchema {
|
|||||||
readonly presentation: AdvertisedInputSchemaPresentation;
|
readonly presentation: AdvertisedInputSchemaPresentation;
|
||||||
}
|
}
|
||||||
|
|
||||||
const isSchemaObject = (value: unknown): value is Record<string, unknown> =>
|
const INPUT_REDUCED_ANNOTATION =
|
||||||
typeof value === "object" && value !== null && !Array.isArray(value);
|
|
||||||
|
|
||||||
const serializedBytes = (value: unknown): number =>
|
|
||||||
Buffer.byteLength(JSON.stringify(value), "utf8");
|
|
||||||
|
|
||||||
const schemaDepth = (value: unknown, depth = 0): number => {
|
|
||||||
if (Array.isArray(value))
|
|
||||||
return Math.max(
|
|
||||||
depth,
|
|
||||||
...value.map((child) => schemaDepth(child, depth + 1)),
|
|
||||||
);
|
|
||||||
if (!isSchemaObject(value)) return depth;
|
|
||||||
return Math.max(
|
|
||||||
depth,
|
|
||||||
...Object.values(value).map((child) => schemaDepth(child, depth + 1)),
|
|
||||||
);
|
|
||||||
};
|
|
||||||
|
|
||||||
/** Strict clients commonly reject advertised schemas deeper than ten nodes. */
|
|
||||||
const COMPACT_SCHEMA_MAX_DEPTH = 10;
|
|
||||||
|
|
||||||
/** Rewrite every schema-valued position; literal data stays opaque. */
|
|
||||||
const rebuildSchemaPositions = (
|
|
||||||
node: unknown,
|
|
||||||
rebuild: (child: unknown) => unknown,
|
|
||||||
): unknown => {
|
|
||||||
if (Array.isArray(node)) return node.map((child) => rebuild(child));
|
|
||||||
if (!isSchemaObject(node)) return node;
|
|
||||||
const out: Record<string, unknown> = {};
|
|
||||||
for (const [key, value] of Object.entries(node)) {
|
|
||||||
if (schemaLiteralValues.has(key)) {
|
|
||||||
out[key] = value;
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
if (schemaMaps.has(key) && isSchemaObject(value)) {
|
|
||||||
out[key] = Object.fromEntries(
|
|
||||||
Object.entries(value).map(([name, child]) => [name, rebuild(child)]),
|
|
||||||
);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
if (schemaArrays.has(key) && Array.isArray(value)) {
|
|
||||||
out[key] = value.map((child) => rebuild(child));
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
if (schemaChildren.has(key)) {
|
|
||||||
out[key] = Array.isArray(value)
|
|
||||||
? value.map((child) => rebuild(child))
|
|
||||||
: rebuild(value);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
out[key] = value;
|
|
||||||
}
|
|
||||||
return out;
|
|
||||||
};
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Drop annotation prose from every subschema while preserving all validation
|
|
||||||
* keywords. The root keeps its own description because the root-union
|
|
||||||
* projection encodes accepted input groups there; the root's echoed literal
|
|
||||||
* examples leave with the rest of the annotation prose.
|
|
||||||
*/
|
|
||||||
const stripAnnotations = (
|
|
||||||
schema: Record<string, unknown>,
|
|
||||||
): Record<string, unknown> => {
|
|
||||||
const rebuilt = rebuildSchemaPositions(
|
|
||||||
schema,
|
|
||||||
stripChildAnnotations,
|
|
||||||
) as Record<string, unknown>;
|
|
||||||
const out: Record<string, unknown> = {};
|
|
||||||
for (const [key, value] of Object.entries(rebuilt)) {
|
|
||||||
if (annotationKeys.has(key) && key !== "description") continue;
|
|
||||||
out[key] = value;
|
|
||||||
}
|
|
||||||
return out;
|
|
||||||
};
|
|
||||||
|
|
||||||
const stripChildAnnotations = (child: unknown): unknown => {
|
|
||||||
const rebuilt = rebuildSchemaPositions(child, stripChildAnnotations);
|
|
||||||
if (!isSchemaObject(rebuilt)) return rebuilt;
|
|
||||||
const kept: Record<string, unknown> = {};
|
|
||||||
for (const [key, value] of Object.entries(rebuilt))
|
|
||||||
if (!annotationKeys.has(key)) kept[key] = value;
|
|
||||||
return kept;
|
|
||||||
};
|
|
||||||
|
|
||||||
/** Stop serialization at the budget before allocating an expanded schema. */
|
|
||||||
const inlineWithinBudget = (
|
|
||||||
root: Record<string, unknown>,
|
|
||||||
budgetBytes: number,
|
|
||||||
): Record<string, unknown> | undefined => {
|
|
||||||
const references = new JsonSchemaReferences(root);
|
|
||||||
const identity = Object.fromEntries(
|
|
||||||
["$id", "$anchor", "$schema"].flatMap((key) =>
|
|
||||||
Object.hasOwn(root, key) ? [[key, root[key]]] : [],
|
|
||||||
),
|
|
||||||
);
|
|
||||||
const identityBytes =
|
|
||||||
Object.keys(identity).length === 0 ? 0 : serializedBytes(identity) - 2;
|
|
||||||
if (identityBytes > budgetBytes) return undefined;
|
|
||||||
let resolved = true;
|
|
||||||
function* serialize(
|
|
||||||
value: unknown,
|
|
||||||
position: "schema" | "map" | "data",
|
|
||||||
scope: JsonSchemaScope | undefined,
|
|
||||||
active: ReadonlyMap<JsonSchemaScope, ReadonlySet<string>>,
|
|
||||||
): Generator<string> {
|
|
||||||
if (Array.isArray(value)) {
|
|
||||||
yield "[";
|
|
||||||
for (const [index, child] of value.entries()) {
|
|
||||||
if (index > 0) yield ",";
|
|
||||||
yield* serialize(child, position, scope, active);
|
|
||||||
}
|
|
||||||
yield "]";
|
|
||||||
} else if (isSchemaObject(value)) {
|
|
||||||
const currentScope =
|
|
||||||
position === "schema" ? references.enter(value, scope) : scope;
|
|
||||||
if (
|
|
||||||
position === "schema" &&
|
|
||||||
(currentScope === undefined ||
|
|
||||||
currentScope.dialect !== references.rootScope.dialect ||
|
|
||||||
[
|
|
||||||
"$dynamicRef",
|
|
||||||
"$recursiveRef",
|
|
||||||
"$dynamicAnchor",
|
|
||||||
"$recursiveAnchor",
|
|
||||||
].some((key) => Object.hasOwn(value, key)))
|
|
||||||
) {
|
|
||||||
resolved = false;
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
if (position === "schema" && Object.keys(value).length === 0) {
|
|
||||||
yield JSON.stringify(jsonValueSchema());
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
if (position === "schema" && typeof value.$ref === "string") {
|
|
||||||
const target = references.resolve(currentScope, value.$ref);
|
|
||||||
if (
|
|
||||||
target.kind !== "target" ||
|
|
||||||
currentScope === undefined ||
|
|
||||||
active.get(currentScope)?.has(value.$ref)
|
|
||||||
) {
|
|
||||||
resolved = false;
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const {
|
|
||||||
$ref,
|
|
||||||
$id: _id,
|
|
||||||
$anchor: _anchor,
|
|
||||||
$schema: _schema,
|
|
||||||
...siblings
|
|
||||||
} = value;
|
|
||||||
const nextActive = new Map(active);
|
|
||||||
nextActive.set(
|
|
||||||
currentScope,
|
|
||||||
new Set([...(active.get(currentScope) ?? []), $ref]),
|
|
||||||
);
|
|
||||||
// Draft 7 ignores siblings, while clients such as Ajv also apply them.
|
|
||||||
// Do not label either interpretation an equivalent inline advertisement.
|
|
||||||
if (
|
|
||||||
currentScope.dialect === "draft-07" &&
|
|
||||||
Object.keys(siblings).length > 0
|
|
||||||
) {
|
|
||||||
resolved = false;
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const expanded =
|
|
||||||
Object.keys(siblings).length === 0
|
|
||||||
? target.value
|
|
||||||
: { allOf: [target.value, siblings] };
|
|
||||||
yield* serialize(expanded, "schema", currentScope, nextActive);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
yield "{";
|
|
||||||
let first = true;
|
|
||||||
for (const [key, child] of Object.entries(value)) {
|
|
||||||
if (
|
|
||||||
position === "schema" &&
|
|
||||||
(definitionKeys.has(key) ||
|
|
||||||
key === "$id" ||
|
|
||||||
key === "$anchor" ||
|
|
||||||
key === "$schema")
|
|
||||||
)
|
|
||||||
continue;
|
|
||||||
if (!first) yield ",";
|
|
||||||
first = false;
|
|
||||||
yield JSON.stringify(key) + ":";
|
|
||||||
// `not: {}` forbids a value; no client infers a type for a value it
|
|
||||||
// must omit, and the any-value list would place `items` beside a
|
|
||||||
// non-array type, which Gemini function declarations reject.
|
|
||||||
if (
|
|
||||||
position === "schema" &&
|
|
||||||
key === "not" &&
|
|
||||||
isSchemaObject(child) &&
|
|
||||||
Object.keys(child).length === 0
|
|
||||||
) {
|
|
||||||
yield "{}";
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
const childPosition =
|
|
||||||
position === "map"
|
|
||||||
? "schema"
|
|
||||||
: position === "schema" && schemaMaps.has(key)
|
|
||||||
? "map"
|
|
||||||
: position === "schema" &&
|
|
||||||
(schemaArrays.has(key) || schemaChildren.has(key))
|
|
||||||
? "schema"
|
|
||||||
: "data";
|
|
||||||
yield* serialize(child, childPosition, currentScope, active);
|
|
||||||
}
|
|
||||||
yield "}";
|
|
||||||
} else {
|
|
||||||
yield JSON.stringify(value);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
const chunks: string[] = [];
|
|
||||||
let bytes = identityBytes;
|
|
||||||
for (const chunk of serialize(
|
|
||||||
root,
|
|
||||||
"schema",
|
|
||||||
references.rootScope,
|
|
||||||
new Map(),
|
|
||||||
)) {
|
|
||||||
bytes += Buffer.byteLength(chunk, "utf8");
|
|
||||||
if (bytes > budgetBytes || !resolved) return undefined;
|
|
||||||
chunks.push(chunk);
|
|
||||||
}
|
|
||||||
if (!resolved) return undefined;
|
|
||||||
const expanded: unknown = JSON.parse(chunks.join(""));
|
|
||||||
if (!isSchemaObject(expanded)) return undefined;
|
|
||||||
// Every emitted dependency is now static and reference-free. Nested resource
|
|
||||||
// identifiers/anchors can leave without duplicating identities at call sites.
|
|
||||||
const schema = { ...identity, ...expanded };
|
|
||||||
return serializedBytes(schema) <= budgetBytes ? schema : undefined;
|
|
||||||
};
|
|
||||||
|
|
||||||
/** Preserve every possible type, including referenced and nullable unions. */
|
|
||||||
const detectedTypes = (
|
|
||||||
schema: unknown,
|
|
||||||
references: JsonSchemaReferences,
|
|
||||||
inherited: JsonSchemaScope | undefined,
|
|
||||||
active: ReadonlySet<object> = new Set(),
|
|
||||||
): readonly string[] | undefined => {
|
|
||||||
if (!isSchemaObject(schema)) return undefined;
|
|
||||||
const scope = references.enter(schema, inherited);
|
|
||||||
if (scope === undefined) return undefined;
|
|
||||||
if (typeof schema.$ref === "string") {
|
|
||||||
if (active.has(schema)) return undefined;
|
|
||||||
const target = references.resolve(scope, schema.$ref);
|
|
||||||
return target.kind === "target"
|
|
||||||
? detectedTypes(
|
|
||||||
target.value,
|
|
||||||
references,
|
|
||||||
scope,
|
|
||||||
new Set([...active, schema]),
|
|
||||||
)
|
|
||||||
: undefined;
|
|
||||||
}
|
|
||||||
if (typeof schema.type === "string") return [schema.type];
|
|
||||||
if (
|
|
||||||
Array.isArray(schema.type) &&
|
|
||||||
schema.type.every((type) => typeof type === "string")
|
|
||||||
)
|
|
||||||
return schema.type;
|
|
||||||
if (Array.isArray(schema.enum) && schema.enum.length > 0) {
|
|
||||||
const enumTypes = enumValueTypes(schema.enum);
|
|
||||||
if (enumTypes.every((type) => jsonValueTypes.includes(type)))
|
|
||||||
return enumTypes;
|
|
||||||
}
|
|
||||||
const alternatives = Array.isArray(schema.anyOf)
|
|
||||||
? schema.anyOf
|
|
||||||
: Array.isArray(schema.oneOf)
|
|
||||||
? schema.oneOf
|
|
||||||
: [];
|
|
||||||
if (alternatives.length === 0) return undefined;
|
|
||||||
const types = alternatives.map((branch) =>
|
|
||||||
detectedTypes(branch, references, scope, active),
|
|
||||||
);
|
|
||||||
if (types.some((type) => type === undefined)) return undefined;
|
|
||||||
return [...new Set(types.flatMap((type) => type ?? []))];
|
|
||||||
};
|
|
||||||
|
|
||||||
interface ScopedSchema {
|
|
||||||
readonly schema: Record<string, unknown>;
|
|
||||||
readonly scope: JsonSchemaScope;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Collect the item schema of every array alternative. Tuples, boolean item
|
|
||||||
* schemas and unresolved references are unknown because one item type would
|
|
||||||
* not describe every accepted element.
|
|
||||||
*/
|
|
||||||
const arrayItemAlternatives = (
|
|
||||||
schema: unknown,
|
|
||||||
references: JsonSchemaReferences,
|
|
||||||
inherited: JsonSchemaScope | undefined,
|
|
||||||
active: ReadonlySet<object> = new Set(),
|
|
||||||
): readonly ScopedSchema[] | undefined => {
|
|
||||||
if (!isSchemaObject(schema)) return undefined;
|
|
||||||
const scope = references.enter(schema, inherited);
|
|
||||||
if (scope === undefined) return undefined;
|
|
||||||
if (typeof schema.$ref === "string") {
|
|
||||||
if (active.has(schema)) return undefined;
|
|
||||||
const target = references.resolve(scope, schema.$ref);
|
|
||||||
return target.kind === "target"
|
|
||||||
? arrayItemAlternatives(
|
|
||||||
target.value,
|
|
||||||
references,
|
|
||||||
scope,
|
|
||||||
new Set([...active, schema]),
|
|
||||||
)
|
|
||||||
: undefined;
|
|
||||||
}
|
|
||||||
const types =
|
|
||||||
typeof schema.type === "string"
|
|
||||||
? [schema.type]
|
|
||||||
: Array.isArray(schema.type)
|
|
||||||
? schema.type
|
|
||||||
: undefined;
|
|
||||||
if (types !== undefined) {
|
|
||||||
if (!types.includes("array")) return [];
|
|
||||||
return isSchemaObject(schema.items) && schema.prefixItems === undefined
|
|
||||||
? [{ schema: schema.items, scope }]
|
|
||||||
: undefined;
|
|
||||||
}
|
|
||||||
const alternatives = Array.isArray(schema.anyOf)
|
|
||||||
? schema.anyOf
|
|
||||||
: Array.isArray(schema.oneOf)
|
|
||||||
? schema.oneOf
|
|
||||||
: [];
|
|
||||||
if (alternatives.length === 0) return undefined;
|
|
||||||
const nested = alternatives.map((branch) =>
|
|
||||||
arrayItemAlternatives(branch, references, scope, active),
|
|
||||||
);
|
|
||||||
return nested.some((items) => items === undefined)
|
|
||||||
? undefined
|
|
||||||
: nested.flatMap((items) => items ?? []);
|
|
||||||
};
|
|
||||||
|
|
||||||
/** Nested arrays beyond this level advertise an unconstrained item schema. */
|
|
||||||
const REDUCED_ITEM_MAX_DEPTH = 3;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Advertise the element types the canonical item schemas accept. Gemini
|
|
||||||
* function declarations permit `items` only beside the single `array` type,
|
|
||||||
* so an element that may be an array alongside other types, or whose type is
|
|
||||||
* unknown, uses the empty schema: it accepts every element and still gives
|
|
||||||
* clients that require `items` an item schema.
|
|
||||||
*/
|
|
||||||
const reducedItemSchema = (
|
|
||||||
alternatives: readonly ScopedSchema[] | undefined,
|
|
||||||
references: JsonSchemaReferences,
|
|
||||||
depth = 1,
|
|
||||||
): Record<string, unknown> => {
|
|
||||||
if (
|
|
||||||
alternatives === undefined ||
|
|
||||||
alternatives.length === 0 ||
|
|
||||||
depth > REDUCED_ITEM_MAX_DEPTH
|
|
||||||
)
|
|
||||||
return {};
|
|
||||||
const types: string[] = [];
|
|
||||||
for (const { schema, scope } of alternatives) {
|
|
||||||
const detected = detectedTypes(schema, references, scope);
|
|
||||||
if (detected === undefined || detected.length === 0) return {};
|
|
||||||
types.push(...detected);
|
|
||||||
}
|
|
||||||
const unique = [...new Set(types)];
|
|
||||||
if (!unique.includes("array"))
|
|
||||||
return { type: unique.length === 1 ? unique[0] : unique };
|
|
||||||
if (unique.length > 1) return {};
|
|
||||||
const nested: ScopedSchema[] = [];
|
|
||||||
for (const { schema, scope } of alternatives) {
|
|
||||||
const items = arrayItemAlternatives(schema, references, scope);
|
|
||||||
if (items === undefined) return { type: "array", items: {} };
|
|
||||||
nested.push(...items);
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
type: "array",
|
|
||||||
items: reducedItemSchema(nested, references, depth + 1),
|
|
||||||
};
|
|
||||||
};
|
|
||||||
|
|
||||||
/** Give homogeneous array-item unions the type required by strict gateways. */
|
|
||||||
const typeArrayItemUnions = (
|
|
||||||
schema: Record<string, unknown>,
|
|
||||||
): Record<string, unknown> => {
|
|
||||||
const references = new JsonSchemaReferences(schema);
|
|
||||||
const visit = (node: unknown): unknown => {
|
|
||||||
const rebuilt = rebuildSchemaPositions(node, visit);
|
|
||||||
if (!isSchemaObject(rebuilt) || rebuilt.type !== "array") return rebuilt;
|
|
||||||
const items = rebuilt.items;
|
|
||||||
if (
|
|
||||||
!isSchemaObject(items) ||
|
|
||||||
items.type !== undefined ||
|
|
||||||
(!Array.isArray(items.oneOf) && !Array.isArray(items.anyOf))
|
|
||||||
)
|
|
||||||
return rebuilt;
|
|
||||||
const types = detectedTypes(items, references, references.rootScope);
|
|
||||||
return types?.length === 1
|
|
||||||
? { ...rebuilt, items: { ...items, type: types[0] } }
|
|
||||||
: rebuilt;
|
|
||||||
};
|
|
||||||
return visit(schema) as Record<string, unknown>;
|
|
||||||
};
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Strict gateways reject some nested combinators, primitive enums without a
|
|
||||||
* type, and array items whose union has no explicit type. Homogeneous item
|
|
||||||
* unions gain their shared type; remaining incompatible shapes use the shallow
|
|
||||||
* projection while canonical runtime validation stays intact.
|
|
||||||
*/
|
|
||||||
const hasGatewayIncompatibleShape = (schema: unknown): boolean => {
|
|
||||||
if (Array.isArray(schema)) return schema.some(hasGatewayIncompatibleShape);
|
|
||||||
if (!isSchemaObject(schema)) return false;
|
|
||||||
if (Array.isArray(schema.anyOf) && schema.anyOf.length === 0) return true;
|
|
||||||
if (
|
|
||||||
schema.type === "array" &&
|
|
||||||
isSchemaObject(schema.items) &&
|
|
||||||
schema.items.type === undefined &&
|
|
||||||
(Array.isArray(schema.items.oneOf) || Array.isArray(schema.items.anyOf))
|
|
||||||
)
|
|
||||||
return true;
|
|
||||||
if (Array.isArray(schema.required) && isSchemaObject(schema.properties)) {
|
|
||||||
for (const name of schema.required) {
|
|
||||||
const property = schema.properties[name];
|
|
||||||
if (isSchemaObject(property) && containsGatewayCombinator(property))
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (
|
|
||||||
Array.isArray(schema.enum) &&
|
|
||||||
schema.enum.length > 0 &&
|
|
||||||
schema.type === undefined
|
|
||||||
) {
|
|
||||||
const types = enumValueTypes(schema.enum);
|
|
||||||
if (
|
|
||||||
types.length === 1 &&
|
|
||||||
types.every((type) =>
|
|
||||||
["string", "number", "boolean", "null"].includes(type),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
return schemaChildrenAndMaps(schema).some(hasGatewayIncompatibleShape);
|
|
||||||
};
|
|
||||||
|
|
||||||
const containsGatewayCombinator = (schema: unknown): boolean => {
|
|
||||||
if (Array.isArray(schema)) return schema.some(containsGatewayCombinator);
|
|
||||||
if (!isSchemaObject(schema)) return false;
|
|
||||||
if (schema.oneOf !== undefined || schema.allOf !== undefined) return true;
|
|
||||||
return schemaChildrenAndMaps(schema).some(containsGatewayCombinator);
|
|
||||||
};
|
|
||||||
|
|
||||||
/** Traverse only schema-valued positions; property names remain opaque data. */
|
|
||||||
const schemaChildrenAndMaps = (
|
|
||||||
schema: Readonly<Record<string, unknown>>,
|
|
||||||
): readonly unknown[] =>
|
|
||||||
Object.entries(schema).flatMap(([key, value]) => {
|
|
||||||
if (schemaMaps.has(key) && isSchemaObject(value))
|
|
||||||
return Object.values(value);
|
|
||||||
if (schemaArrays.has(key) && Array.isArray(value)) return value;
|
|
||||||
if (schemaChildren.has(key)) return Array.isArray(value) ? value : [value];
|
|
||||||
return [];
|
|
||||||
});
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Present a reduced property's reference-resolved types. Reduced inputs omit
|
|
||||||
* nested validation, but clients still require an item schema for an array
|
|
||||||
* declaration, and Gemini accepts `items` only beside the single `array` type,
|
|
||||||
* so a property that may be an array or another type lists typed branches.
|
|
||||||
*/
|
|
||||||
const reducedPropertyTypes = (
|
|
||||||
value: unknown,
|
|
||||||
references: JsonSchemaReferences,
|
|
||||||
): Record<string, unknown> => {
|
|
||||||
const types = detectedTypes(value, references, references.rootScope);
|
|
||||||
if (types === undefined || types.length === 0) return {};
|
|
||||||
if (!types.includes("array"))
|
|
||||||
return { type: types.length === 1 ? types[0] : types };
|
|
||||||
const array = {
|
|
||||||
type: "array",
|
|
||||||
items: reducedItemSchema(
|
|
||||||
arrayItemAlternatives(value, references, references.rootScope),
|
|
||||||
references,
|
|
||||||
),
|
|
||||||
};
|
|
||||||
return types.length === 1
|
|
||||||
? array
|
|
||||||
: {
|
|
||||||
anyOf: types.map((type) => (type === "array" ? array : { type })),
|
|
||||||
};
|
|
||||||
};
|
|
||||||
|
|
||||||
const REDUCED_ANNOTATION =
|
|
||||||
"Advertised in reduced form by the compact input schema profile; the server still validates complete canonical inputs.";
|
"Advertised in reduced form by the compact input schema profile; the server still validates complete canonical inputs.";
|
||||||
|
|
||||||
/**
|
|
||||||
* Present only the root property shape with per-property guidance. Used for
|
|
||||||
* schemas whose validation structure alone exceeds the compact budget; the
|
|
||||||
* canonical schema still rejects malformed calls server-side.
|
|
||||||
*/
|
|
||||||
const reduceToShallowProperties = (
|
|
||||||
schema: Record<string, unknown>,
|
|
||||||
): Record<string, unknown> => {
|
|
||||||
const properties = isSchemaObject(schema.properties) ? schema.properties : {};
|
|
||||||
const description = [
|
|
||||||
typeof schema.description === "string" ? schema.description : undefined,
|
|
||||||
...Object.entries(
|
|
||||||
isSchemaObject(schema.dependentRequired)
|
|
||||||
? schema.dependentRequired
|
|
||||||
: isSchemaObject(schema.dependencies)
|
|
||||||
? schema.dependencies
|
|
||||||
: {},
|
|
||||||
).flatMap(([property, partners]) =>
|
|
||||||
Array.isArray(partners) &&
|
|
||||||
partners.every((partner) => typeof partner === "string")
|
|
||||||
? [
|
|
||||||
`When ${property} is provided, also provide ${partners.join(" and ")}.`,
|
|
||||||
]
|
|
||||||
: [],
|
|
||||||
),
|
|
||||||
REDUCED_ANNOTATION,
|
|
||||||
]
|
|
||||||
.filter((text) => text !== undefined)
|
|
||||||
.join(" ");
|
|
||||||
const references = new JsonSchemaReferences(schema);
|
|
||||||
const shallow = Object.fromEntries(
|
|
||||||
Object.entries(properties).map(([name, value]) => {
|
|
||||||
const description =
|
|
||||||
isSchemaObject(value) && typeof value.description === "string"
|
|
||||||
? { description: value.description }
|
|
||||||
: {};
|
|
||||||
return [
|
|
||||||
name,
|
|
||||||
{ ...reducedPropertyTypes(value, references), ...description },
|
|
||||||
];
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
return {
|
|
||||||
...(typeof schema.$schema === "string" ? { $schema: schema.$schema } : {}),
|
|
||||||
type: "object",
|
|
||||||
description,
|
|
||||||
properties: shallow,
|
|
||||||
...(Array.isArray(schema.required)
|
|
||||||
? { required: [...schema.required] }
|
|
||||||
: {}),
|
|
||||||
...(typeof schema.minProperties === "number"
|
|
||||||
? { minProperties: schema.minProperties }
|
|
||||||
: {}),
|
|
||||||
...(schema.additionalProperties === false
|
|
||||||
? { additionalProperties: false }
|
|
||||||
: {}),
|
|
||||||
};
|
|
||||||
};
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Render an advertised input schema within a per-schema byte budget for
|
* Render an advertised input schema within a per-schema byte budget for
|
||||||
* providers that validate the literal serialized schema size. Validation
|
* providers that validate the literal serialized schema size. Validation
|
||||||
@@ -631,19 +49,22 @@ export const compactAdvertisedInputSchema = (
|
|||||||
!hasGatewayIncompatibleShape(inline)
|
!hasGatewayIncompatibleShape(inline)
|
||||||
)
|
)
|
||||||
return { schema: inline, presentation: "inline" };
|
return { schema: inline, presentation: "inline" };
|
||||||
const reduced = reduceToShallowProperties(schema);
|
const reduced = reduceToShallowProperties(schema, INPUT_REDUCED_ANNOTATION);
|
||||||
if (serializedBytes(reduced) <= budgetBytes)
|
if (serializedBytes(reduced) <= budgetBytes)
|
||||||
return { schema: reduced, presentation: "reduced" };
|
return { schema: reduced, presentation: "reduced" };
|
||||||
// A future contract with an extreme root-property count still advertises
|
// A future contract with an extreme root-property count still advertises
|
||||||
// an honest, budget-fitting shape instead of an oversized one.
|
// an honest, budget-fitting shape instead of an oversized one.
|
||||||
return {
|
return {
|
||||||
schema: reduceToShallowProperties({
|
schema: reduceToShallowProperties(
|
||||||
|
{
|
||||||
type: "object",
|
type: "object",
|
||||||
description:
|
description:
|
||||||
(typeof schema.description === "string"
|
(typeof schema.description === "string"
|
||||||
? `${schema.description} `
|
? `${schema.description} `
|
||||||
: "") + REDUCED_ANNOTATION,
|
: "") + INPUT_REDUCED_ANNOTATION,
|
||||||
}),
|
},
|
||||||
|
INPUT_REDUCED_ANNOTATION,
|
||||||
|
),
|
||||||
presentation: "reduced",
|
presentation: "reduced",
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -0,0 +1,597 @@
|
|||||||
|
import {
|
||||||
|
JsonSchemaReferences,
|
||||||
|
type JsonSchemaScope,
|
||||||
|
} from "./jsonSchemaReferences.js";
|
||||||
|
import {
|
||||||
|
schemaArrays,
|
||||||
|
schemaChildren,
|
||||||
|
schemaMaps,
|
||||||
|
schemaLiteralValues,
|
||||||
|
} from "./jsonSchemaKeywords.js";
|
||||||
|
|
||||||
|
// Shared compact JSON Schema projection primitives; canonical validation stays separate.
|
||||||
|
|
||||||
|
/** Annotation keys that carry guidance but no validation semantics. */
|
||||||
|
const annotationKeys = new Set(["description", "examples", "default", "title"]);
|
||||||
|
|
||||||
|
const definitionKeys = new Set(["$defs", "definitions"]);
|
||||||
|
// An empty schema accepts every JSON value. Kimi otherwise infers string.
|
||||||
|
const jsonValueTypes = [
|
||||||
|
"null",
|
||||||
|
"boolean",
|
||||||
|
"object",
|
||||||
|
"array",
|
||||||
|
"number",
|
||||||
|
"string",
|
||||||
|
];
|
||||||
|
// An empty item schema accepts every JSON value, including arbitrarily nested
|
||||||
|
// arrays, while satisfying clients that require `items` for array types.
|
||||||
|
// Gemini rejects `items` beside a type list; the compact catalog test fails
|
||||||
|
// if a tool would advertise this presentation.
|
||||||
|
export const jsonValueSchema = () => ({ type: [...jsonValueTypes], items: {} });
|
||||||
|
|
||||||
|
const enumValueTypes = (values: readonly unknown[]): readonly string[] => [
|
||||||
|
...new Set(
|
||||||
|
values.map((value) => {
|
||||||
|
if (value === null) return "null";
|
||||||
|
if (Array.isArray(value)) return "array";
|
||||||
|
if (typeof value === "object") return "object";
|
||||||
|
return typeof value;
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
];
|
||||||
|
|
||||||
|
export const isSchemaObject = (
|
||||||
|
value: unknown,
|
||||||
|
): value is Record<string, unknown> =>
|
||||||
|
typeof value === "object" && value !== null && !Array.isArray(value);
|
||||||
|
|
||||||
|
export const serializedBytes = (value: unknown): number =>
|
||||||
|
Buffer.byteLength(JSON.stringify(value), "utf8");
|
||||||
|
|
||||||
|
export const schemaDepth = (value: unknown, depth = 0): number => {
|
||||||
|
if (Array.isArray(value))
|
||||||
|
return Math.max(
|
||||||
|
depth,
|
||||||
|
...value.map((child) => schemaDepth(child, depth + 1)),
|
||||||
|
);
|
||||||
|
if (!isSchemaObject(value)) return depth;
|
||||||
|
return Math.max(
|
||||||
|
depth,
|
||||||
|
...Object.values(value).map((child) => schemaDepth(child, depth + 1)),
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Strict clients commonly reject advertised schemas deeper than ten nodes. */
|
||||||
|
export const COMPACT_SCHEMA_MAX_DEPTH = 10;
|
||||||
|
|
||||||
|
/** Rewrite every schema-valued position; literal data stays opaque. */
|
||||||
|
const rebuildSchemaPositions = (
|
||||||
|
node: unknown,
|
||||||
|
rebuild: (child: unknown) => unknown,
|
||||||
|
): unknown => {
|
||||||
|
if (Array.isArray(node)) return node.map((child) => rebuild(child));
|
||||||
|
if (!isSchemaObject(node)) return node;
|
||||||
|
const out: Record<string, unknown> = {};
|
||||||
|
for (const [key, value] of Object.entries(node)) {
|
||||||
|
if (schemaLiteralValues.has(key)) {
|
||||||
|
out[key] = value;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (schemaMaps.has(key) && isSchemaObject(value)) {
|
||||||
|
out[key] = Object.fromEntries(
|
||||||
|
Object.entries(value).map(([name, child]) => [name, rebuild(child)]),
|
||||||
|
);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (schemaArrays.has(key) && Array.isArray(value)) {
|
||||||
|
out[key] = value.map((child) => rebuild(child));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (schemaChildren.has(key)) {
|
||||||
|
out[key] = Array.isArray(value)
|
||||||
|
? value.map((child) => rebuild(child))
|
||||||
|
: rebuild(value);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
out[key] = value;
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Drop annotation prose from every subschema while preserving all validation
|
||||||
|
* keywords. The root keeps its own description because the root-union
|
||||||
|
* projection encodes accepted input groups there; the root's echoed literal
|
||||||
|
* examples leave with the rest of the annotation prose.
|
||||||
|
*/
|
||||||
|
export const stripAnnotations = (
|
||||||
|
schema: Record<string, unknown>,
|
||||||
|
): Record<string, unknown> => {
|
||||||
|
const rebuilt = rebuildSchemaPositions(
|
||||||
|
schema,
|
||||||
|
stripChildAnnotations,
|
||||||
|
) as Record<string, unknown>;
|
||||||
|
const out: Record<string, unknown> = {};
|
||||||
|
for (const [key, value] of Object.entries(rebuilt)) {
|
||||||
|
if (annotationKeys.has(key) && key !== "description") continue;
|
||||||
|
out[key] = value;
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
};
|
||||||
|
|
||||||
|
const stripChildAnnotations = (child: unknown): unknown => {
|
||||||
|
const rebuilt = rebuildSchemaPositions(child, stripChildAnnotations);
|
||||||
|
if (!isSchemaObject(rebuilt)) return rebuilt;
|
||||||
|
const kept: Record<string, unknown> = {};
|
||||||
|
for (const [key, value] of Object.entries(rebuilt))
|
||||||
|
if (!annotationKeys.has(key)) kept[key] = value;
|
||||||
|
return kept;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Stop serialization at the budget before allocating an expanded schema. */
|
||||||
|
export const inlineWithinBudget = (
|
||||||
|
root: Record<string, unknown>,
|
||||||
|
budgetBytes: number,
|
||||||
|
): Record<string, unknown> | undefined => {
|
||||||
|
const references = new JsonSchemaReferences(root);
|
||||||
|
const identity = Object.fromEntries(
|
||||||
|
["$id", "$anchor", "$schema"].flatMap((key) =>
|
||||||
|
Object.hasOwn(root, key) ? [[key, root[key]]] : [],
|
||||||
|
),
|
||||||
|
);
|
||||||
|
const identityBytes =
|
||||||
|
Object.keys(identity).length === 0 ? 0 : serializedBytes(identity) - 2;
|
||||||
|
if (identityBytes > budgetBytes) return undefined;
|
||||||
|
let resolved = true;
|
||||||
|
function* serialize(
|
||||||
|
value: unknown,
|
||||||
|
position: "schema" | "map" | "data",
|
||||||
|
scope: JsonSchemaScope | undefined,
|
||||||
|
active: ReadonlyMap<JsonSchemaScope, ReadonlySet<string>>,
|
||||||
|
): Generator<string> {
|
||||||
|
if (Array.isArray(value)) {
|
||||||
|
yield "[";
|
||||||
|
for (const [index, child] of value.entries()) {
|
||||||
|
if (index > 0) yield ",";
|
||||||
|
yield* serialize(child, position, scope, active);
|
||||||
|
}
|
||||||
|
yield "]";
|
||||||
|
} else if (isSchemaObject(value)) {
|
||||||
|
const currentScope =
|
||||||
|
position === "schema" ? references.enter(value, scope) : scope;
|
||||||
|
if (
|
||||||
|
position === "schema" &&
|
||||||
|
(currentScope === undefined ||
|
||||||
|
currentScope.dialect !== references.rootScope.dialect ||
|
||||||
|
[
|
||||||
|
"$dynamicRef",
|
||||||
|
"$recursiveRef",
|
||||||
|
"$dynamicAnchor",
|
||||||
|
"$recursiveAnchor",
|
||||||
|
].some((key) => Object.hasOwn(value, key)))
|
||||||
|
) {
|
||||||
|
resolved = false;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (position === "schema" && Object.keys(value).length === 0) {
|
||||||
|
yield JSON.stringify(jsonValueSchema());
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (position === "schema" && typeof value.$ref === "string") {
|
||||||
|
const target = references.resolve(currentScope, value.$ref);
|
||||||
|
if (
|
||||||
|
target.kind !== "target" ||
|
||||||
|
currentScope === undefined ||
|
||||||
|
active.get(currentScope)?.has(value.$ref)
|
||||||
|
) {
|
||||||
|
resolved = false;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const {
|
||||||
|
$ref,
|
||||||
|
$id: _id,
|
||||||
|
$anchor: _anchor,
|
||||||
|
$schema: _schema,
|
||||||
|
...siblings
|
||||||
|
} = value;
|
||||||
|
const nextActive = new Map(active);
|
||||||
|
nextActive.set(
|
||||||
|
currentScope,
|
||||||
|
new Set([...(active.get(currentScope) ?? []), $ref]),
|
||||||
|
);
|
||||||
|
// Draft 7 ignores siblings, while clients such as Ajv also apply them.
|
||||||
|
// Do not label either interpretation an equivalent inline advertisement.
|
||||||
|
if (
|
||||||
|
currentScope.dialect === "draft-07" &&
|
||||||
|
Object.keys(siblings).length > 0
|
||||||
|
) {
|
||||||
|
resolved = false;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const expanded =
|
||||||
|
Object.keys(siblings).length === 0
|
||||||
|
? target.value
|
||||||
|
: { allOf: [target.value, siblings] };
|
||||||
|
yield* serialize(expanded, "schema", currentScope, nextActive);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
yield "{";
|
||||||
|
let first = true;
|
||||||
|
for (const [key, child] of Object.entries(value)) {
|
||||||
|
if (
|
||||||
|
position === "schema" &&
|
||||||
|
(definitionKeys.has(key) ||
|
||||||
|
key === "$id" ||
|
||||||
|
key === "$anchor" ||
|
||||||
|
key === "$schema")
|
||||||
|
)
|
||||||
|
continue;
|
||||||
|
if (!first) yield ",";
|
||||||
|
first = false;
|
||||||
|
yield JSON.stringify(key) + ":";
|
||||||
|
// `not: {}` forbids a value; no client infers a type for a value it
|
||||||
|
// must omit, and the any-value list would place `items` beside a
|
||||||
|
// non-array type, which Gemini function declarations reject.
|
||||||
|
if (
|
||||||
|
position === "schema" &&
|
||||||
|
key === "not" &&
|
||||||
|
isSchemaObject(child) &&
|
||||||
|
Object.keys(child).length === 0
|
||||||
|
) {
|
||||||
|
yield "{}";
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const childPosition =
|
||||||
|
position === "map"
|
||||||
|
? "schema"
|
||||||
|
: position === "schema" && schemaMaps.has(key)
|
||||||
|
? "map"
|
||||||
|
: position === "schema" &&
|
||||||
|
(schemaArrays.has(key) || schemaChildren.has(key))
|
||||||
|
? "schema"
|
||||||
|
: "data";
|
||||||
|
yield* serialize(child, childPosition, currentScope, active);
|
||||||
|
}
|
||||||
|
yield "}";
|
||||||
|
} else {
|
||||||
|
yield JSON.stringify(value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const chunks: string[] = [];
|
||||||
|
let bytes = identityBytes;
|
||||||
|
for (const chunk of serialize(
|
||||||
|
root,
|
||||||
|
"schema",
|
||||||
|
references.rootScope,
|
||||||
|
new Map(),
|
||||||
|
)) {
|
||||||
|
bytes += Buffer.byteLength(chunk, "utf8");
|
||||||
|
if (bytes > budgetBytes || !resolved) return undefined;
|
||||||
|
chunks.push(chunk);
|
||||||
|
}
|
||||||
|
if (!resolved) return undefined;
|
||||||
|
const expanded: unknown = JSON.parse(chunks.join(""));
|
||||||
|
if (!isSchemaObject(expanded)) return undefined;
|
||||||
|
// Every emitted dependency is now static and reference-free. Nested resource
|
||||||
|
// identifiers/anchors can leave without duplicating identities at call sites.
|
||||||
|
const schema = { ...identity, ...expanded };
|
||||||
|
return serializedBytes(schema) <= budgetBytes ? schema : undefined;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Preserve every possible type, including referenced and nullable unions. */
|
||||||
|
const detectedTypes = (
|
||||||
|
schema: unknown,
|
||||||
|
references: JsonSchemaReferences,
|
||||||
|
inherited: JsonSchemaScope | undefined,
|
||||||
|
active: ReadonlySet<object> = new Set(),
|
||||||
|
): readonly string[] | undefined => {
|
||||||
|
if (!isSchemaObject(schema)) return undefined;
|
||||||
|
const scope = references.enter(schema, inherited);
|
||||||
|
if (scope === undefined) return undefined;
|
||||||
|
if (typeof schema.$ref === "string") {
|
||||||
|
if (active.has(schema)) return undefined;
|
||||||
|
const target = references.resolve(scope, schema.$ref);
|
||||||
|
return target.kind === "target"
|
||||||
|
? detectedTypes(
|
||||||
|
target.value,
|
||||||
|
references,
|
||||||
|
scope,
|
||||||
|
new Set([...active, schema]),
|
||||||
|
)
|
||||||
|
: undefined;
|
||||||
|
}
|
||||||
|
if (typeof schema.type === "string") return [schema.type];
|
||||||
|
if (
|
||||||
|
Array.isArray(schema.type) &&
|
||||||
|
schema.type.every((type) => typeof type === "string")
|
||||||
|
)
|
||||||
|
return schema.type;
|
||||||
|
if (Array.isArray(schema.enum) && schema.enum.length > 0) {
|
||||||
|
const enumTypes = enumValueTypes(schema.enum);
|
||||||
|
if (enumTypes.every((type) => jsonValueTypes.includes(type)))
|
||||||
|
return enumTypes;
|
||||||
|
}
|
||||||
|
const alternatives = Array.isArray(schema.anyOf)
|
||||||
|
? schema.anyOf
|
||||||
|
: Array.isArray(schema.oneOf)
|
||||||
|
? schema.oneOf
|
||||||
|
: [];
|
||||||
|
if (alternatives.length === 0) return undefined;
|
||||||
|
const types = alternatives.map((branch) =>
|
||||||
|
detectedTypes(branch, references, scope, active),
|
||||||
|
);
|
||||||
|
if (types.some((type) => type === undefined)) return undefined;
|
||||||
|
return [...new Set(types.flatMap((type) => type ?? []))];
|
||||||
|
};
|
||||||
|
|
||||||
|
interface ScopedSchema {
|
||||||
|
readonly schema: Record<string, unknown>;
|
||||||
|
readonly scope: JsonSchemaScope;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Collect the item schema of every array alternative. Tuples, boolean item
|
||||||
|
* schemas and unresolved references are unknown because one item type would
|
||||||
|
* not describe every accepted element.
|
||||||
|
*/
|
||||||
|
const arrayItemAlternatives = (
|
||||||
|
schema: unknown,
|
||||||
|
references: JsonSchemaReferences,
|
||||||
|
inherited: JsonSchemaScope | undefined,
|
||||||
|
active: ReadonlySet<object> = new Set(),
|
||||||
|
): readonly ScopedSchema[] | undefined => {
|
||||||
|
if (!isSchemaObject(schema)) return undefined;
|
||||||
|
const scope = references.enter(schema, inherited);
|
||||||
|
if (scope === undefined) return undefined;
|
||||||
|
if (typeof schema.$ref === "string") {
|
||||||
|
if (active.has(schema)) return undefined;
|
||||||
|
const target = references.resolve(scope, schema.$ref);
|
||||||
|
return target.kind === "target"
|
||||||
|
? arrayItemAlternatives(
|
||||||
|
target.value,
|
||||||
|
references,
|
||||||
|
scope,
|
||||||
|
new Set([...active, schema]),
|
||||||
|
)
|
||||||
|
: undefined;
|
||||||
|
}
|
||||||
|
const types =
|
||||||
|
typeof schema.type === "string"
|
||||||
|
? [schema.type]
|
||||||
|
: Array.isArray(schema.type)
|
||||||
|
? schema.type
|
||||||
|
: undefined;
|
||||||
|
if (types !== undefined) {
|
||||||
|
if (!types.includes("array")) return [];
|
||||||
|
return isSchemaObject(schema.items) && schema.prefixItems === undefined
|
||||||
|
? [{ schema: schema.items, scope }]
|
||||||
|
: undefined;
|
||||||
|
}
|
||||||
|
const alternatives = Array.isArray(schema.anyOf)
|
||||||
|
? schema.anyOf
|
||||||
|
: Array.isArray(schema.oneOf)
|
||||||
|
? schema.oneOf
|
||||||
|
: [];
|
||||||
|
if (alternatives.length === 0) return undefined;
|
||||||
|
const nested = alternatives.map((branch) =>
|
||||||
|
arrayItemAlternatives(branch, references, scope, active),
|
||||||
|
);
|
||||||
|
return nested.some((items) => items === undefined)
|
||||||
|
? undefined
|
||||||
|
: nested.flatMap((items) => items ?? []);
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Nested arrays beyond this level advertise an unconstrained item schema. */
|
||||||
|
const REDUCED_ITEM_MAX_DEPTH = 3;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Advertise the element types the canonical item schemas accept. Gemini
|
||||||
|
* function declarations permit `items` only beside the single `array` type,
|
||||||
|
* so an element that may be an array alongside other types, or whose type is
|
||||||
|
* unknown, uses the empty schema: it accepts every element and still gives
|
||||||
|
* clients that require `items` an item schema.
|
||||||
|
*/
|
||||||
|
const reducedItemSchema = (
|
||||||
|
alternatives: readonly ScopedSchema[] | undefined,
|
||||||
|
references: JsonSchemaReferences,
|
||||||
|
depth = 1,
|
||||||
|
): Record<string, unknown> => {
|
||||||
|
if (
|
||||||
|
alternatives === undefined ||
|
||||||
|
alternatives.length === 0 ||
|
||||||
|
depth > REDUCED_ITEM_MAX_DEPTH
|
||||||
|
)
|
||||||
|
return {};
|
||||||
|
const types: string[] = [];
|
||||||
|
for (const { schema, scope } of alternatives) {
|
||||||
|
const detected = detectedTypes(schema, references, scope);
|
||||||
|
if (detected === undefined || detected.length === 0) return {};
|
||||||
|
types.push(...detected);
|
||||||
|
}
|
||||||
|
const unique = [...new Set(types)];
|
||||||
|
if (!unique.includes("array"))
|
||||||
|
return { type: unique.length === 1 ? unique[0] : unique };
|
||||||
|
if (unique.length > 1) return {};
|
||||||
|
const nested: ScopedSchema[] = [];
|
||||||
|
for (const { schema, scope } of alternatives) {
|
||||||
|
const items = arrayItemAlternatives(schema, references, scope);
|
||||||
|
if (items === undefined) return { type: "array", items: {} };
|
||||||
|
nested.push(...items);
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
type: "array",
|
||||||
|
items: reducedItemSchema(nested, references, depth + 1),
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Give homogeneous array-item unions the type required by strict gateways. */
|
||||||
|
export const typeArrayItemUnions = (
|
||||||
|
schema: Record<string, unknown>,
|
||||||
|
): Record<string, unknown> => {
|
||||||
|
const references = new JsonSchemaReferences(schema);
|
||||||
|
const visit = (node: unknown): unknown => {
|
||||||
|
const rebuilt = rebuildSchemaPositions(node, visit);
|
||||||
|
if (!isSchemaObject(rebuilt) || rebuilt.type !== "array") return rebuilt;
|
||||||
|
const items = rebuilt.items;
|
||||||
|
if (
|
||||||
|
!isSchemaObject(items) ||
|
||||||
|
items.type !== undefined ||
|
||||||
|
(!Array.isArray(items.oneOf) && !Array.isArray(items.anyOf))
|
||||||
|
)
|
||||||
|
return rebuilt;
|
||||||
|
const types = detectedTypes(items, references, references.rootScope);
|
||||||
|
return types?.length === 1
|
||||||
|
? { ...rebuilt, items: { ...items, type: types[0] } }
|
||||||
|
: rebuilt;
|
||||||
|
};
|
||||||
|
return visit(schema) as Record<string, unknown>;
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Strict gateways reject some nested combinators, primitive enums without a
|
||||||
|
* type, and array items whose union has no explicit type. Homogeneous item
|
||||||
|
* unions gain their shared type; remaining incompatible shapes use the shallow
|
||||||
|
* projection while canonical runtime validation stays intact.
|
||||||
|
*/
|
||||||
|
export const hasGatewayIncompatibleShape = (schema: unknown): boolean => {
|
||||||
|
if (Array.isArray(schema)) return schema.some(hasGatewayIncompatibleShape);
|
||||||
|
if (!isSchemaObject(schema)) return false;
|
||||||
|
if (Array.isArray(schema.anyOf) && schema.anyOf.length === 0) return true;
|
||||||
|
if (
|
||||||
|
schema.type === "array" &&
|
||||||
|
isSchemaObject(schema.items) &&
|
||||||
|
schema.items.type === undefined &&
|
||||||
|
(Array.isArray(schema.items.oneOf) || Array.isArray(schema.items.anyOf))
|
||||||
|
)
|
||||||
|
return true;
|
||||||
|
if (Array.isArray(schema.required) && isSchemaObject(schema.properties)) {
|
||||||
|
for (const name of schema.required) {
|
||||||
|
const property = schema.properties[name];
|
||||||
|
if (isSchemaObject(property) && containsGatewayCombinator(property))
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (
|
||||||
|
Array.isArray(schema.enum) &&
|
||||||
|
schema.enum.length > 0 &&
|
||||||
|
schema.type === undefined
|
||||||
|
) {
|
||||||
|
const types = enumValueTypes(schema.enum);
|
||||||
|
if (
|
||||||
|
types.length === 1 &&
|
||||||
|
types.every((type) =>
|
||||||
|
["string", "number", "boolean", "null"].includes(type),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
return schemaChildrenAndMaps(schema).some(hasGatewayIncompatibleShape);
|
||||||
|
};
|
||||||
|
|
||||||
|
const containsGatewayCombinator = (schema: unknown): boolean => {
|
||||||
|
if (Array.isArray(schema)) return schema.some(containsGatewayCombinator);
|
||||||
|
if (!isSchemaObject(schema)) return false;
|
||||||
|
if (schema.oneOf !== undefined || schema.allOf !== undefined) return true;
|
||||||
|
return schemaChildrenAndMaps(schema).some(containsGatewayCombinator);
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Traverse only schema-valued positions; property names remain opaque data. */
|
||||||
|
const schemaChildrenAndMaps = (
|
||||||
|
schema: Readonly<Record<string, unknown>>,
|
||||||
|
): readonly unknown[] =>
|
||||||
|
Object.entries(schema).flatMap(([key, value]) => {
|
||||||
|
if (schemaMaps.has(key) && isSchemaObject(value))
|
||||||
|
return Object.values(value);
|
||||||
|
if (schemaArrays.has(key) && Array.isArray(value)) return value;
|
||||||
|
if (schemaChildren.has(key)) return Array.isArray(value) ? value : [value];
|
||||||
|
return [];
|
||||||
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Present a reduced property's reference-resolved types. Reduced inputs omit
|
||||||
|
* nested validation, but clients still require an item schema for an array
|
||||||
|
* declaration, and Gemini accepts `items` only beside the single `array` type,
|
||||||
|
* so a property that may be an array or another type lists typed branches.
|
||||||
|
*/
|
||||||
|
const reducedPropertyTypes = (
|
||||||
|
value: unknown,
|
||||||
|
references: JsonSchemaReferences,
|
||||||
|
): Record<string, unknown> => {
|
||||||
|
const types = detectedTypes(value, references, references.rootScope);
|
||||||
|
if (types === undefined || types.length === 0) return {};
|
||||||
|
if (!types.includes("array"))
|
||||||
|
return { type: types.length === 1 ? types[0] : types };
|
||||||
|
const array = {
|
||||||
|
type: "array",
|
||||||
|
items: reducedItemSchema(
|
||||||
|
arrayItemAlternatives(value, references, references.rootScope),
|
||||||
|
references,
|
||||||
|
),
|
||||||
|
};
|
||||||
|
return types.length === 1
|
||||||
|
? array
|
||||||
|
: {
|
||||||
|
anyOf: types.map((type) => (type === "array" ? array : { type })),
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Present only the root property shape with per-property guidance. Used for
|
||||||
|
* schemas whose validation structure alone exceeds the compact budget; the
|
||||||
|
* canonical schema still rejects malformed calls server-side.
|
||||||
|
*/
|
||||||
|
export const reduceToShallowProperties = (
|
||||||
|
schema: Record<string, unknown>,
|
||||||
|
reducedAnnotation: string,
|
||||||
|
): Record<string, unknown> => {
|
||||||
|
const properties = isSchemaObject(schema.properties) ? schema.properties : {};
|
||||||
|
const description = [
|
||||||
|
typeof schema.description === "string" ? schema.description : undefined,
|
||||||
|
...Object.entries(
|
||||||
|
isSchemaObject(schema.dependentRequired)
|
||||||
|
? schema.dependentRequired
|
||||||
|
: isSchemaObject(schema.dependencies)
|
||||||
|
? schema.dependencies
|
||||||
|
: {},
|
||||||
|
).flatMap(([property, partners]) =>
|
||||||
|
Array.isArray(partners) &&
|
||||||
|
partners.every((partner) => typeof partner === "string")
|
||||||
|
? [
|
||||||
|
`When ${property} is provided, also provide ${partners.join(" and ")}.`,
|
||||||
|
]
|
||||||
|
: [],
|
||||||
|
),
|
||||||
|
reducedAnnotation,
|
||||||
|
]
|
||||||
|
.filter((text) => text !== undefined)
|
||||||
|
.join(" ");
|
||||||
|
const references = new JsonSchemaReferences(schema);
|
||||||
|
const shallow = Object.fromEntries(
|
||||||
|
Object.entries(properties).map(([name, value]) => {
|
||||||
|
const description =
|
||||||
|
isSchemaObject(value) && typeof value.description === "string"
|
||||||
|
? { description: value.description }
|
||||||
|
: {};
|
||||||
|
return [
|
||||||
|
name,
|
||||||
|
{ ...reducedPropertyTypes(value, references), ...description },
|
||||||
|
];
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
return {
|
||||||
|
...(typeof schema.$schema === "string" ? { $schema: schema.$schema } : {}),
|
||||||
|
type: "object",
|
||||||
|
description,
|
||||||
|
properties: shallow,
|
||||||
|
...(Array.isArray(schema.required)
|
||||||
|
? { required: [...schema.required] }
|
||||||
|
: {}),
|
||||||
|
...(typeof schema.minProperties === "number"
|
||||||
|
? { minProperties: schema.minProperties }
|
||||||
|
: {}),
|
||||||
|
...(schema.additionalProperties === false
|
||||||
|
? { additionalProperties: false }
|
||||||
|
: {}),
|
||||||
|
};
|
||||||
|
};
|
||||||
@@ -0,0 +1,171 @@
|
|||||||
|
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
|
||||||
|
import { compactAdvertisedOutputSchema } from "./compactOutputSchema.js";
|
||||||
|
|
||||||
|
const serializedBytes = (value: unknown): number =>
|
||||||
|
Buffer.byteLength(JSON.stringify(value), "utf8");
|
||||||
|
|
||||||
|
const annotatedSchema = {
|
||||||
|
type: "object",
|
||||||
|
description: "Root result guidance stays.",
|
||||||
|
properties: {
|
||||||
|
value: {
|
||||||
|
type: "string",
|
||||||
|
description: "Field guidance leaves.",
|
||||||
|
examples: ["seed"],
|
||||||
|
default: "seed",
|
||||||
|
title: "Value",
|
||||||
|
},
|
||||||
|
nested: {
|
||||||
|
type: "object",
|
||||||
|
description: "Nested guidance leaves.",
|
||||||
|
properties: {
|
||||||
|
count: { type: "integer", description: "Leaves.", minimum: 1 },
|
||||||
|
},
|
||||||
|
required: ["count"],
|
||||||
|
additionalProperties: false,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
required: ["value", "nested"],
|
||||||
|
additionalProperties: false,
|
||||||
|
} as const;
|
||||||
|
|
||||||
|
const referencedSchema = {
|
||||||
|
$schema: "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
$id: "urn:rea:tool-output:sha256:fixture",
|
||||||
|
type: "object",
|
||||||
|
properties: {
|
||||||
|
entries: { type: "array", items: { $ref: "#/$defs/entry" } },
|
||||||
|
},
|
||||||
|
required: ["entries"],
|
||||||
|
$defs: {
|
||||||
|
entry: {
|
||||||
|
type: "object",
|
||||||
|
properties: { name: { type: "string" } },
|
||||||
|
required: ["name"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
} as const;
|
||||||
|
|
||||||
|
const oversizedSchema = (): Record<string, unknown> => ({
|
||||||
|
type: "object",
|
||||||
|
description: "A complete result envelope.",
|
||||||
|
properties: {
|
||||||
|
envelope: { type: "string" },
|
||||||
|
result: {
|
||||||
|
type: "object",
|
||||||
|
properties: Object.fromEntries(
|
||||||
|
Array.from({ length: 200 }, (_, index) => [
|
||||||
|
`field_${index}`,
|
||||||
|
{ type: "string", description: `Field ${index}` },
|
||||||
|
]),
|
||||||
|
),
|
||||||
|
required: Array.from({ length: 200 }, (_, index) => `field_${index}`),
|
||||||
|
additionalProperties: false,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
required: ["envelope", "result"],
|
||||||
|
additionalProperties: false,
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("compact advertised output schema", () => {
|
||||||
|
it("drops nested annotation prose while keeping the root guidance and validation", () => {
|
||||||
|
const { schema, presentation } = compactAdvertisedOutputSchema(
|
||||||
|
structuredClone(annotatedSchema),
|
||||||
|
4 * 1024,
|
||||||
|
);
|
||||||
|
expect(presentation).toBe("inline");
|
||||||
|
expect(schema.description).toBe("Root result guidance stays.");
|
||||||
|
const value = (schema.properties as Record<string, unknown>)
|
||||||
|
.value as Record<string, unknown>;
|
||||||
|
expect(value).toEqual({ type: "string" });
|
||||||
|
const nested = (schema.properties as Record<string, unknown>)
|
||||||
|
.nested as Record<string, unknown>;
|
||||||
|
expect(nested.description).toBeUndefined();
|
||||||
|
expect(nested.required).toEqual(["count"]);
|
||||||
|
expect(schema.required).toEqual(["value", "nested"]);
|
||||||
|
expect(schema.additionalProperties).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("inlines local definitions and drops the content-bound identity", () => {
|
||||||
|
const { schema, presentation } = compactAdvertisedOutputSchema(
|
||||||
|
structuredClone(referencedSchema),
|
||||||
|
4 * 1024,
|
||||||
|
);
|
||||||
|
expect(presentation).toBe("inline");
|
||||||
|
expect(schema.$defs).toBeUndefined();
|
||||||
|
expect(schema.$id).toBeUndefined();
|
||||||
|
expect(schema.$schema).toBe("https://json-schema.org/draft/2020-12/schema");
|
||||||
|
const entries = (schema.properties as Record<string, unknown>)
|
||||||
|
.entries as Record<string, unknown>;
|
||||||
|
expect(entries.items).toMatchObject({ type: "object" });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reduces an oversized contract to a widening root shape within budget", () => {
|
||||||
|
const budgetBytes = 4 * 1024;
|
||||||
|
const { schema, presentation } = compactAdvertisedOutputSchema(
|
||||||
|
oversizedSchema(),
|
||||||
|
budgetBytes,
|
||||||
|
);
|
||||||
|
expect(presentation).toBe("reduced");
|
||||||
|
expect(serializedBytes(schema)).toBeLessThanOrEqual(budgetBytes);
|
||||||
|
expect(schema.type).toBe("object");
|
||||||
|
expect(schema.required).toEqual(["envelope", "result"]);
|
||||||
|
expect(schema.additionalProperties).toBe(false);
|
||||||
|
expect(schema.description).toContain("compact output schema profile");
|
||||||
|
const result = (schema.properties as Record<string, unknown>)
|
||||||
|
.result as Record<string, unknown>;
|
||||||
|
expect(result.type).toBe("object");
|
||||||
|
expect(result.properties).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("accepts a valid canonical result after reduction", () => {
|
||||||
|
const budgetBytes = 4 * 1024;
|
||||||
|
const { schema } = compactAdvertisedOutputSchema(
|
||||||
|
oversizedSchema(),
|
||||||
|
budgetBytes,
|
||||||
|
);
|
||||||
|
const validate = new Ajv2020({ strict: false }).compile(schema);
|
||||||
|
const result = {
|
||||||
|
envelope: "ok",
|
||||||
|
result: Object.fromEntries(
|
||||||
|
Array.from({ length: 200 }, (_, index) => [
|
||||||
|
`field_${index}`,
|
||||||
|
`value ${index}`,
|
||||||
|
]),
|
||||||
|
),
|
||||||
|
};
|
||||||
|
expect(validate(result)).toBe(true);
|
||||||
|
// The advertisement is wider than the canonical contract: dropping nested
|
||||||
|
// detail must never reject a valid result, but it can accept an invalid one.
|
||||||
|
expect(validate({ envelope: "ok", result: { unknown: 1 } })).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps a non-object root accepting every JSON value when it cannot fit", () => {
|
||||||
|
const arraySchema = {
|
||||||
|
type: "array",
|
||||||
|
items: Object.fromEntries(
|
||||||
|
Array.from({ length: 200 }, (_, index) => [
|
||||||
|
`field_${index}`,
|
||||||
|
{ type: "string", description: `Field ${index}` },
|
||||||
|
]),
|
||||||
|
),
|
||||||
|
};
|
||||||
|
const { schema, presentation } = compactAdvertisedOutputSchema(
|
||||||
|
arraySchema,
|
||||||
|
128,
|
||||||
|
);
|
||||||
|
expect(presentation).toBe("reduced");
|
||||||
|
expect(serializedBytes(schema)).toBeLessThanOrEqual(128);
|
||||||
|
const validate = new Ajv2020({ strict: false }).compile(schema);
|
||||||
|
expect(validate(["anything"])).toBe(true);
|
||||||
|
expect(validate({ also: "anything" })).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("is deterministic across equivalent inputs", () => {
|
||||||
|
expect(compactAdvertisedOutputSchema(oversizedSchema(), 4 * 1024)).toEqual(
|
||||||
|
compactAdvertisedOutputSchema(oversizedSchema(), 4 * 1024),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
import {
|
||||||
|
COMPACT_SCHEMA_MAX_DEPTH,
|
||||||
|
inlineWithinBudget,
|
||||||
|
isSchemaObject,
|
||||||
|
jsonValueSchema,
|
||||||
|
reduceToShallowProperties,
|
||||||
|
schemaDepth,
|
||||||
|
serializedBytes,
|
||||||
|
stripAnnotations,
|
||||||
|
} from "./compactJsonSchema.js";
|
||||||
|
|
||||||
|
// Output schemas dominate a complete `tools/list` frame. The compact profile
|
||||||
|
// bounds each advertised output contract so frame-capped hosts can finish the
|
||||||
|
// handshake, while canonical result validation and `structuredContent` stay
|
||||||
|
// unchanged. Every projection is a widening: a valid canonical result always
|
||||||
|
// satisfies the advertised schema.
|
||||||
|
|
||||||
|
export type AdvertisedOutputSchemaPresentation = "inline" | "reduced";
|
||||||
|
|
||||||
|
/** Budget selected by the caller for compact MCP output-schema presentation. */
|
||||||
|
export interface CompactOutputSchemaPresentation {
|
||||||
|
readonly budgetBytes: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface CompactedOutputSchema {
|
||||||
|
readonly schema: Record<string, unknown>;
|
||||||
|
readonly presentation: AdvertisedOutputSchemaPresentation;
|
||||||
|
}
|
||||||
|
|
||||||
|
const OUTPUT_REDUCED_ANNOTATION =
|
||||||
|
"Advertised in reduced form by the compact output schema profile; the server still validates complete canonical results.";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Drop the canonical contract's content-bound `$id` and anchors. The compact
|
||||||
|
* bytes differ from the canonical schema the identity was derived from, so
|
||||||
|
* retaining that identity would misstate what the advertised document is.
|
||||||
|
* References are already inlined, so no identity is needed to resolve them.
|
||||||
|
*/
|
||||||
|
const withoutContentBoundIdentity = (
|
||||||
|
schema: Record<string, unknown>,
|
||||||
|
): Record<string, unknown> => {
|
||||||
|
const { $id: _id, $anchor: _anchor, ...advertised } = schema;
|
||||||
|
return advertised;
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Render an advertised output schema within a per-schema byte budget. The
|
||||||
|
* complete validation structure is kept when it fits; otherwise the root
|
||||||
|
* property shape and first-level types remain and nested detail is elided.
|
||||||
|
* The registered canonical schema still validates every delivered result.
|
||||||
|
*/
|
||||||
|
export const compactAdvertisedOutputSchema = (
|
||||||
|
schema: Record<string, unknown>,
|
||||||
|
budgetBytes: number,
|
||||||
|
): CompactedOutputSchema => {
|
||||||
|
const inline = inlineWithinBudget(stripAnnotations(schema), budgetBytes);
|
||||||
|
if (inline !== undefined && schemaDepth(inline) <= COMPACT_SCHEMA_MAX_DEPTH)
|
||||||
|
return {
|
||||||
|
schema: withoutContentBoundIdentity(inline),
|
||||||
|
presentation: "inline",
|
||||||
|
};
|
||||||
|
// Every REA output contract is an object at the root. A non-object root still
|
||||||
|
// advertises an accepting shape rather than a projection that could reject
|
||||||
|
// valid canonical output.
|
||||||
|
const reduced =
|
||||||
|
schema.type === "object" || isSchemaObject(schema.properties)
|
||||||
|
? reduceToShallowProperties(schema, OUTPUT_REDUCED_ANNOTATION)
|
||||||
|
: jsonValueSchema();
|
||||||
|
if (serializedBytes(reduced) <= budgetBytes)
|
||||||
|
return {
|
||||||
|
schema: withoutContentBoundIdentity(reduced),
|
||||||
|
presentation: "reduced",
|
||||||
|
};
|
||||||
|
// A root description can itself be large; keep the advertisement inside the
|
||||||
|
// budget instead of carrying it into a shape with no nested detail left.
|
||||||
|
const minimal = reduceToShallowProperties(
|
||||||
|
{ type: "object" },
|
||||||
|
OUTPUT_REDUCED_ANNOTATION,
|
||||||
|
);
|
||||||
|
return {
|
||||||
|
schema:
|
||||||
|
serializedBytes(minimal) <= budgetBytes ? minimal : jsonValueSchema(),
|
||||||
|
presentation: "reduced",
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Return the JSON schema clients receive for the selected output presentation. */
|
||||||
|
export const presentAdvertisedOutputSchema = (
|
||||||
|
schema: Record<string, unknown>,
|
||||||
|
compactPresentation: CompactOutputSchemaPresentation | undefined,
|
||||||
|
): Record<string, unknown> =>
|
||||||
|
compactPresentation === undefined
|
||||||
|
? schema
|
||||||
|
: compactAdvertisedOutputSchema(schema, compactPresentation.budgetBytes)
|
||||||
|
.schema;
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
import type { CatalogDigestProfiles } from "./catalogIdentity.js";
|
import type { CatalogDigestProfiles } from "./catalogIdentity.js";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Digests of the four built-in input/output schema profiles, recorded
|
* Digests of the six built-in input/output schema profiles, recorded
|
||||||
* from the compiled contracts by `scripts/record-catalog-digests.mjs` after
|
* from the compiled contracts by `scripts/record-catalog-digests.mjs` after
|
||||||
* compilation. Source runs leave this undefined and compute them on first use.
|
* compilation. Source runs leave this undefined and compute them on first use.
|
||||||
*/
|
*/
|
||||||
|
|||||||
@@ -70,7 +70,8 @@ export class DeferredEvidenceMcpServer extends EvidenceMcpServerCore {
|
|||||||
undefined,
|
undefined,
|
||||||
delivery,
|
delivery,
|
||||||
undefined,
|
undefined,
|
||||||
false,
|
// `main.ts` only defers the exact full advertisement catalog.
|
||||||
|
"full",
|
||||||
catalog.identity,
|
catalog.identity,
|
||||||
);
|
);
|
||||||
for (const tool of catalog.tools) {
|
for (const tool of catalog.tools) {
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import {
|
|||||||
FULL_CATALOG_IDENTITY,
|
FULL_CATALOG_IDENTITY,
|
||||||
createCatalogIdentity,
|
createCatalogIdentity,
|
||||||
} from "../catalogIdentity.js";
|
} from "../catalogIdentity.js";
|
||||||
|
import type { McpOutputSchemaProfile } from "../config/mcpOutputSchemaProfile.js";
|
||||||
import type { ToolResultDelivery } from "./ToolResultDelivery.js";
|
import type { ToolResultDelivery } from "./ToolResultDelivery.js";
|
||||||
import {
|
import {
|
||||||
EvidenceMcpServerCore,
|
EvidenceMcpServerCore,
|
||||||
@@ -23,7 +24,7 @@ export class EvidenceMcpServer extends EvidenceMcpServerCore {
|
|||||||
recordEvidence: EvidenceWriter["recordEvidence"] | undefined,
|
recordEvidence: EvidenceWriter["recordEvidence"] | undefined,
|
||||||
delivery: ToolResultDelivery,
|
delivery: ToolResultDelivery,
|
||||||
compactPresentation: CompactInputSchemaPresentation | undefined = undefined,
|
compactPresentation: CompactInputSchemaPresentation | undefined = undefined,
|
||||||
omitOutputSchemas = false,
|
outputSchemaProfile: McpOutputSchemaProfile = "full",
|
||||||
identity?: ReturnType<typeof createCatalogIdentity>,
|
identity?: ReturnType<typeof createCatalogIdentity>,
|
||||||
) {
|
) {
|
||||||
super(
|
super(
|
||||||
@@ -32,14 +33,11 @@ export class EvidenceMcpServer extends EvidenceMcpServerCore {
|
|||||||
recordEvidence,
|
recordEvidence,
|
||||||
delivery,
|
delivery,
|
||||||
compactPresentation,
|
compactPresentation,
|
||||||
omitOutputSchemas,
|
outputSchemaProfile,
|
||||||
identity ??
|
identity ??
|
||||||
(compactPresentation === undefined && !omitOutputSchemas
|
(compactPresentation === undefined && outputSchemaProfile === "full"
|
||||||
? FULL_CATALOG_IDENTITY
|
? FULL_CATALOG_IDENTITY
|
||||||
: createCatalogIdentity(
|
: createCatalogIdentity(compactPresentation, outputSchemaProfile)),
|
||||||
compactPresentation,
|
|
||||||
omitOutputSchemas ? "off" : "full",
|
|
||||||
)),
|
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -30,6 +30,11 @@ import {
|
|||||||
presentAdvertisedInputSchema,
|
presentAdvertisedInputSchema,
|
||||||
type CompactInputSchemaPresentation,
|
type CompactInputSchemaPresentation,
|
||||||
} from "../contracts/compactInputSchema.js";
|
} from "../contracts/compactInputSchema.js";
|
||||||
|
import { presentAdvertisedOutputSchema } from "../contracts/compactOutputSchema.js";
|
||||||
|
import {
|
||||||
|
compactPresentationForMcpOutputSchemaProfile,
|
||||||
|
type McpOutputSchemaProfile,
|
||||||
|
} from "../config/mcpOutputSchemaProfile.js";
|
||||||
import type { createCatalogIdentity } from "../catalogIdentity.js";
|
import type { createCatalogIdentity } from "../catalogIdentity.js";
|
||||||
import { transformAdvertisedInputJsonSchema } from "../contracts/toolSchemaMetadata.js";
|
import { transformAdvertisedInputJsonSchema } from "../contracts/toolSchemaMetadata.js";
|
||||||
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
|
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
|
||||||
@@ -110,26 +115,48 @@ const replaceToolResultPayload = (
|
|||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Remove only client-facing output schemas; SDK registrations stay canonical. */
|
/**
|
||||||
const omitOutputSchemasFromToolsList = (
|
* Rewrite only client-facing output schemas; SDK registrations stay canonical.
|
||||||
|
* `compact` bounds each advertised schema, while `off` omits it entirely.
|
||||||
|
* The SDK memoises each tool's converted schema, so the caller passes a cache
|
||||||
|
* keyed by that object; repeated listings reuse the same projection.
|
||||||
|
*/
|
||||||
|
const presentOutputSchemasForToolsList = (
|
||||||
message: JSONRPCMessage,
|
message: JSONRPCMessage,
|
||||||
requestId: RequestId,
|
requestId: RequestId,
|
||||||
|
profile: McpOutputSchemaProfile,
|
||||||
|
compactedSchemas: WeakMap<Record<string, unknown>, Record<string, unknown>>,
|
||||||
): JSONRPCMessage => {
|
): JSONRPCMessage => {
|
||||||
if (
|
if (
|
||||||
|
profile === "full" ||
|
||||||
!isJSONRPCResultResponse(message) ||
|
!isJSONRPCResultResponse(message) ||
|
||||||
message.id !== requestId ||
|
message.id !== requestId ||
|
||||||
!isRecord(message.result) ||
|
!isRecord(message.result) ||
|
||||||
!Array.isArray(message.result.tools)
|
!Array.isArray(message.result.tools)
|
||||||
)
|
)
|
||||||
return message;
|
return message;
|
||||||
|
const compactPresentation =
|
||||||
|
compactPresentationForMcpOutputSchemaProfile(profile);
|
||||||
return {
|
return {
|
||||||
...message,
|
...message,
|
||||||
result: {
|
result: {
|
||||||
...message.result,
|
...message.result,
|
||||||
tools: message.result.tools.map((tool: unknown) => {
|
tools: message.result.tools.map((tool: unknown) => {
|
||||||
if (!isRecord(tool)) return tool;
|
if (!isRecord(tool)) return tool;
|
||||||
|
if (compactPresentation === undefined) {
|
||||||
const { outputSchema: _outputSchema, ...advertisedTool } = tool;
|
const { outputSchema: _outputSchema, ...advertisedTool } = tool;
|
||||||
return advertisedTool;
|
return advertisedTool;
|
||||||
|
}
|
||||||
|
if (!isRecord(tool.outputSchema)) return tool;
|
||||||
|
let outputSchema = compactedSchemas.get(tool.outputSchema);
|
||||||
|
if (outputSchema === undefined) {
|
||||||
|
outputSchema = presentAdvertisedOutputSchema(
|
||||||
|
tool.outputSchema,
|
||||||
|
compactPresentation,
|
||||||
|
);
|
||||||
|
compactedSchemas.set(tool.outputSchema, outputSchema);
|
||||||
|
}
|
||||||
|
return { ...tool, outputSchema };
|
||||||
}),
|
}),
|
||||||
},
|
},
|
||||||
} as JSONRPCMessage;
|
} as JSONRPCMessage;
|
||||||
@@ -214,7 +241,7 @@ export class EvidenceMcpServerCore extends RequestTrackingMcpServer {
|
|||||||
private recordEvidence: EvidenceWriter["recordEvidence"] | undefined,
|
private recordEvidence: EvidenceWriter["recordEvidence"] | undefined,
|
||||||
public delivery: ToolResultDelivery,
|
public delivery: ToolResultDelivery,
|
||||||
compactPresentation: CompactInputSchemaPresentation | undefined = undefined,
|
compactPresentation: CompactInputSchemaPresentation | undefined = undefined,
|
||||||
private readonly omitOutputSchemas = false,
|
private readonly outputSchemaProfile: McpOutputSchemaProfile = "full",
|
||||||
identity?: ReturnType<typeof createCatalogIdentity>,
|
identity?: ReturnType<typeof createCatalogIdentity>,
|
||||||
) {
|
) {
|
||||||
super(info, options);
|
super(info, options);
|
||||||
@@ -226,9 +253,9 @@ export class EvidenceMcpServerCore extends RequestTrackingMcpServer {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Compact advertised input schemas and optionally omit output-schema
|
* Compact advertised input schemas and bound or omit output-schema
|
||||||
* advertisements. Canonical parsers and contracts remain registered; the
|
* advertisements. Canonical parsers and contracts remain registered; both
|
||||||
* output-schema-off path validates successful structured results locally.
|
* output profiles validate successful structured results locally.
|
||||||
* The signatures mirror the SDK's registrations so call sites keep their
|
* The signatures mirror the SDK's registrations so call sites keep their
|
||||||
* inferred handler argument types.
|
* inferred handler argument types.
|
||||||
*/
|
*/
|
||||||
@@ -286,7 +313,7 @@ export class EvidenceMcpServerCore extends RequestTrackingMcpServer {
|
|||||||
transport,
|
transport,
|
||||||
() => this.recordEvidence,
|
() => this.recordEvidence,
|
||||||
() => this.delivery,
|
() => this.delivery,
|
||||||
this.omitOutputSchemas,
|
this.outputSchemaProfile,
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
@@ -309,13 +336,19 @@ class ErrorEvidenceTransport implements Transport {
|
|||||||
return this.currentDelivery();
|
return this.currentDelivery();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Bounded projections already computed for this server's budget. */
|
||||||
|
private readonly compactedOutputSchemas = new WeakMap<
|
||||||
|
Record<string, unknown>,
|
||||||
|
Record<string, unknown>
|
||||||
|
>();
|
||||||
|
|
||||||
constructor(
|
constructor(
|
||||||
private readonly transport: Transport,
|
private readonly transport: Transport,
|
||||||
private readonly currentEvidenceWriter: () =>
|
private readonly currentEvidenceWriter: () =>
|
||||||
| EvidenceWriter["recordEvidence"]
|
| EvidenceWriter["recordEvidence"]
|
||||||
| undefined,
|
| undefined,
|
||||||
private readonly currentDelivery: () => ToolResultDelivery,
|
private readonly currentDelivery: () => ToolResultDelivery,
|
||||||
private readonly omitOutputSchemas: boolean,
|
private readonly outputSchemaProfile: McpOutputSchemaProfile,
|
||||||
) {
|
) {
|
||||||
if (transport.hasPerRequestStream !== undefined)
|
if (transport.hasPerRequestStream !== undefined)
|
||||||
this.hasPerRequestStream = transport.hasPerRequestStream;
|
this.hasPerRequestStream = transport.hasPerRequestStream;
|
||||||
@@ -429,14 +462,19 @@ class ErrorEvidenceTransport implements Transport {
|
|||||||
? message.id
|
? message.id
|
||||||
: undefined;
|
: undefined;
|
||||||
const tool = id === undefined ? undefined : this.pendingTools.get(id);
|
const tool = id === undefined ? undefined : this.pendingTools.get(id);
|
||||||
const omitToolSchemas =
|
const rewritesOutputSchemas =
|
||||||
this.omitOutputSchemas &&
|
this.outputSchemaProfile !== "full" &&
|
||||||
id !== undefined &&
|
id !== undefined &&
|
||||||
this.pendingToolLists.has(id);
|
this.pendingToolLists.has(id);
|
||||||
try {
|
try {
|
||||||
const recovered = this.recoverProtocolError(message, tool);
|
const recovered = this.recoverProtocolError(message, tool);
|
||||||
const advertised = omitToolSchemas
|
const advertised = rewritesOutputSchemas
|
||||||
? omitOutputSchemasFromToolsList(recovered, id)
|
? presentOutputSchemasForToolsList(
|
||||||
|
recovered,
|
||||||
|
id,
|
||||||
|
this.outputSchemaProfile,
|
||||||
|
this.compactedOutputSchemas,
|
||||||
|
)
|
||||||
: recovered;
|
: recovered;
|
||||||
if (
|
if (
|
||||||
isJSONRPCResultResponse(advertised) &&
|
isJSONRPCResultResponse(advertised) &&
|
||||||
|
|||||||
+17
-15
@@ -101,7 +101,10 @@ import { EvidenceMcpServer } from "./EvidenceMcpServer.js";
|
|||||||
import { createToolResultDelivery } from "./toolResult.js";
|
import { createToolResultDelivery } from "./toolResult.js";
|
||||||
import type { ToolResultDelivery } from "./ToolResultDelivery.js";
|
import type { ToolResultDelivery } from "./ToolResultDelivery.js";
|
||||||
import { parseMcpResponseBudget } from "../config/mcpResponseBudget.js";
|
import { parseMcpResponseBudget } from "../config/mcpResponseBudget.js";
|
||||||
import { parseMcpOutputSchemaProfile } from "../config/mcpOutputSchemaProfile.js";
|
import {
|
||||||
|
parseMcpOutputSchemaProfile,
|
||||||
|
type McpOutputSchemaProfile,
|
||||||
|
} from "../config/mcpOutputSchemaProfile.js";
|
||||||
import {
|
import {
|
||||||
compactPresentationForMcpInputSchemaProfile,
|
compactPresentationForMcpInputSchemaProfile,
|
||||||
parseMcpInputSchemaProfile,
|
parseMcpInputSchemaProfile,
|
||||||
@@ -408,7 +411,7 @@ const createMcpServer = (
|
|||||||
session: BinarySessionPort | undefined,
|
session: BinarySessionPort | undefined,
|
||||||
delivery: ToolResultDelivery,
|
delivery: ToolResultDelivery,
|
||||||
compactPresentation: CompactInputSchemaPresentation | undefined,
|
compactPresentation: CompactInputSchemaPresentation | undefined,
|
||||||
omitOutputSchemas: boolean,
|
outputSchemaProfile: McpOutputSchemaProfile,
|
||||||
): EvidenceMcpServer =>
|
): EvidenceMcpServer =>
|
||||||
new EvidenceMcpServer(
|
new EvidenceMcpServer(
|
||||||
{
|
{
|
||||||
@@ -428,11 +431,8 @@ const createMcpServer = (
|
|||||||
: (evidence) => session.recordEvidence(evidence),
|
: (evidence) => session.recordEvidence(evidence),
|
||||||
delivery,
|
delivery,
|
||||||
compactPresentation,
|
compactPresentation,
|
||||||
omitOutputSchemas,
|
outputSchemaProfile,
|
||||||
createCatalogIdentity(
|
createCatalogIdentity(compactPresentation, outputSchemaProfile),
|
||||||
compactPresentation,
|
|
||||||
omitOutputSchemas ? "off" : "full",
|
|
||||||
),
|
|
||||||
);
|
);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -449,7 +449,9 @@ export const createServer = (
|
|||||||
const environment = snapshotEnvironment(options.environment ?? process.env);
|
const environment = snapshotEnvironment(options.environment ?? process.env);
|
||||||
const analysisSettings =
|
const analysisSettings =
|
||||||
options.analysisSettings ?? new AnalysisSettings(environment);
|
options.analysisSettings ?? new AnalysisSettings(environment);
|
||||||
const omitOutputSchemas = outputSchemasOmitted(environment);
|
const compactInputPresentation =
|
||||||
|
selectCompactInputSchemaPresentation(environment);
|
||||||
|
const selectedOutputSchemaProfile = outputSchemaProfile(environment);
|
||||||
const providerEnvironment = captureProviderEnvironment(
|
const providerEnvironment = captureProviderEnvironment(
|
||||||
options.providerEnvironment,
|
options.providerEnvironment,
|
||||||
environment,
|
environment,
|
||||||
@@ -516,8 +518,8 @@ export const createServer = (
|
|||||||
createMcpServer(
|
createMcpServer(
|
||||||
session,
|
session,
|
||||||
delivery,
|
delivery,
|
||||||
selectCompactInputSchemaPresentation(environment),
|
compactInputPresentation,
|
||||||
omitOutputSchemas,
|
selectedOutputSchemaProfile,
|
||||||
);
|
);
|
||||||
server.analysisSettings = analysisSettings;
|
server.analysisSettings = analysisSettings;
|
||||||
registerAnalysisSettingsTools(server, analysisSettings);
|
registerAnalysisSettingsTools(server, analysisSettings);
|
||||||
@@ -528,8 +530,8 @@ export const createServer = (
|
|||||||
: (evidence) => session.recordEvidence(evidence),
|
: (evidence) => session.recordEvidence(evidence),
|
||||||
delivery,
|
delivery,
|
||||||
createCatalogIdentity(
|
createCatalogIdentity(
|
||||||
selectCompactInputSchemaPresentation(environment),
|
compactInputPresentation,
|
||||||
omitOutputSchemas ? "off" : "full",
|
selectedOutputSchemaProfile,
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
@@ -882,14 +884,14 @@ const selectToolResultDelivery = (
|
|||||||
return createToolResultDelivery(configured.value);
|
return createToolResultDelivery(configured.value);
|
||||||
};
|
};
|
||||||
|
|
||||||
const outputSchemasOmitted = (
|
const outputSchemaProfile = (
|
||||||
environment: Readonly<NodeJS.ProcessEnv>,
|
environment: Readonly<NodeJS.ProcessEnv>,
|
||||||
): boolean => {
|
): McpOutputSchemaProfile => {
|
||||||
const configured = parseMcpOutputSchemaProfile(
|
const configured = parseMcpOutputSchemaProfile(
|
||||||
environment.REA_MCP_OUTPUT_SCHEMA_PROFILE,
|
environment.REA_MCP_OUTPUT_SCHEMA_PROFILE,
|
||||||
);
|
);
|
||||||
if (!configured.ok) throw configured.error;
|
if (!configured.ok) throw configured.error;
|
||||||
return configured.value === "off";
|
return configured.value ?? "full";
|
||||||
};
|
};
|
||||||
|
|
||||||
const selectCompactInputSchemaPresentation = (
|
const selectCompactInputSchemaPresentation = (
|
||||||
|
|||||||
@@ -63,6 +63,64 @@ const assertCliCatalogIdentity = async (
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const catalogProfiles = [
|
||||||
|
{
|
||||||
|
name: "default compact input",
|
||||||
|
environment: {
|
||||||
|
REA_MCP_INPUT_SCHEMA_PROFILE: undefined,
|
||||||
|
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
|
||||||
|
},
|
||||||
|
identity: createCatalogIdentity(
|
||||||
|
compactPresentationForMcpInputSchemaProfile(undefined),
|
||||||
|
),
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "full",
|
||||||
|
environment: {
|
||||||
|
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
||||||
|
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
|
||||||
|
},
|
||||||
|
identity: FULL_CATALOG_IDENTITY,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "compact",
|
||||||
|
environment: {
|
||||||
|
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
||||||
|
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
|
||||||
|
},
|
||||||
|
identity: createCatalogIdentity(
|
||||||
|
compactPresentationForMcpInputSchemaProfile("compact"),
|
||||||
|
),
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "output schemas off",
|
||||||
|
environment: {
|
||||||
|
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
||||||
|
REA_MCP_OUTPUT_SCHEMA_PROFILE: "off",
|
||||||
|
},
|
||||||
|
identity: createCatalogIdentity(undefined, "off"),
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "output schemas compact",
|
||||||
|
environment: {
|
||||||
|
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
||||||
|
REA_MCP_OUTPUT_SCHEMA_PROFILE: "compact",
|
||||||
|
},
|
||||||
|
identity: createCatalogIdentity(undefined, "compact"),
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "compact input and output schemas off",
|
||||||
|
environment: {
|
||||||
|
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
||||||
|
REA_MCP_OUTPUT_SCHEMA_PROFILE: "off",
|
||||||
|
},
|
||||||
|
identity: createCatalogIdentity(
|
||||||
|
compactPresentationForMcpInputSchemaProfile("compact"),
|
||||||
|
"off",
|
||||||
|
),
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
describe("production MCP doctor", () => {
|
describe("production MCP doctor", () => {
|
||||||
it("reuses immutable compiled profile digests and preserves custom-budget identity", async () => {
|
it("reuses immutable compiled profile digests and preserves custom-budget identity", async () => {
|
||||||
// Exercise the generated module with Node, outside Vitest's source loader.
|
// Exercise the generated module with Node, outside Vitest's source loader.
|
||||||
@@ -79,7 +137,7 @@ describe("production MCP doctor", () => {
|
|||||||
assert.ok(Object.isFrozen(profiles));
|
assert.ok(Object.isFrozen(profiles));
|
||||||
for (const input of ["full", "compact"]) {
|
for (const input of ["full", "compact"]) {
|
||||||
assert.ok(Object.isFrozen(profiles[input]));
|
assert.ok(Object.isFrozen(profiles[input]));
|
||||||
for (const output of ["full", "off"]) {
|
for (const output of ["full", "compact", "off"]) {
|
||||||
const identity = createCatalogIdentity(input === "compact" ? { budgetBytes } : undefined, output);
|
const identity = createCatalogIdentity(input === "compact" ? { budgetBytes } : undefined, output);
|
||||||
assert.equal(identity.digests, profiles[input][output]);
|
assert.equal(identity.digests, profiles[input][output]);
|
||||||
assert.ok(Object.isFrozen(identity.digests));
|
assert.ok(Object.isFrozen(identity.digests));
|
||||||
@@ -124,55 +182,7 @@ describe("production MCP doctor", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
it.each([
|
it.each(catalogProfiles)(
|
||||||
{
|
|
||||||
name: "default compact input",
|
|
||||||
environment: {
|
|
||||||
REA_MCP_INPUT_SCHEMA_PROFILE: undefined,
|
|
||||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
|
|
||||||
},
|
|
||||||
identity: createCatalogIdentity(
|
|
||||||
compactPresentationForMcpInputSchemaProfile(undefined),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "full",
|
|
||||||
environment: {
|
|
||||||
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
|
||||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
|
|
||||||
},
|
|
||||||
identity: FULL_CATALOG_IDENTITY,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "compact",
|
|
||||||
environment: {
|
|
||||||
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
|
||||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
|
|
||||||
},
|
|
||||||
identity: createCatalogIdentity(
|
|
||||||
compactPresentationForMcpInputSchemaProfile("compact"),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "output schemas off",
|
|
||||||
environment: {
|
|
||||||
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
|
||||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "off",
|
|
||||||
},
|
|
||||||
identity: createCatalogIdentity(undefined, "off"),
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "compact input and output schemas off",
|
|
||||||
environment: {
|
|
||||||
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
|
||||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "off",
|
|
||||||
},
|
|
||||||
identity: createCatalogIdentity(
|
|
||||||
compactPresentationForMcpInputSchemaProfile("compact"),
|
|
||||||
"off",
|
|
||||||
),
|
|
||||||
},
|
|
||||||
])(
|
|
||||||
"verifies the $name catalog through production stdio and both CLI inventories",
|
"verifies the $name catalog through production stdio and both CLI inventories",
|
||||||
async ({ environment, identity }) => {
|
async ({ environment, identity }) => {
|
||||||
const result = await runProductionMcpDoctor({
|
const result = await runProductionMcpDoctor({
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||||
|
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
import { expect, it } from "vitest";
|
import { expect, it } from "vitest";
|
||||||
|
|
||||||
@@ -6,24 +7,34 @@ import {
|
|||||||
TOOL_CONTRACTS,
|
TOOL_CONTRACTS,
|
||||||
toolContract,
|
toolContract,
|
||||||
} from "../../../src/contracts/toolContracts.js";
|
} from "../../../src/contracts/toolContracts.js";
|
||||||
|
import { COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES } from "../../../src/config/mcpOutputSchemaProfile.js";
|
||||||
|
import { COMPACT_INPUT_SCHEMA_BUDGET_BYTES } from "../../../src/config/mcpInputSchemaProfile.js";
|
||||||
|
import type { CompactInputSchemaPresentation } from "../../../src/contracts/compactInputSchema.js";
|
||||||
import { EvidenceMcpServer } from "../../../src/server/EvidenceMcpServer.js";
|
import { EvidenceMcpServer } from "../../../src/server/EvidenceMcpServer.js";
|
||||||
import { toolRegistrationOptions } from "../../../src/server/toolRegistrationOptions.js";
|
import { toolRegistrationOptions } from "../../../src/server/toolRegistrationOptions.js";
|
||||||
import { createToolResultDelivery } from "../../../src/server/toolResult.js";
|
import { createToolResultDelivery } from "../../../src/server/toolResult.js";
|
||||||
import type { ToolContract } from "../../../src/contracts/toolContractTypes.js";
|
import type { ToolContract } from "../../../src/contracts/toolContractTypes.js";
|
||||||
import { FULL_CATALOG_IDENTITY } from "../../../src/catalogIdentity.js";
|
import { FULL_CATALOG_IDENTITY } from "../../../src/catalogIdentity.js";
|
||||||
|
import type { McpOutputSchemaProfile } from "../../../src/config/mcpOutputSchemaProfile.js";
|
||||||
|
import { advertisedSchemaErrors } from "../../fixtures/mcpAdvertisedSchemaValidity.js";
|
||||||
|
|
||||||
const createServer = (omitOutputSchemas: boolean) =>
|
const createServer = (
|
||||||
|
outputSchemaProfile: McpOutputSchemaProfile,
|
||||||
|
compactInputPresentation:
|
||||||
|
| CompactInputSchemaPresentation
|
||||||
|
| undefined = undefined,
|
||||||
|
) =>
|
||||||
new EvidenceMcpServer(
|
new EvidenceMcpServer(
|
||||||
{ name: "output-schema-fixture", version: "1" },
|
{ name: "output-schema-fixture", version: "1" },
|
||||||
{ capabilities: {} },
|
{ capabilities: {} },
|
||||||
undefined,
|
undefined,
|
||||||
createToolResultDelivery(10 * 1024 * 1024),
|
createToolResultDelivery(10 * 1024 * 1024),
|
||||||
undefined,
|
compactInputPresentation,
|
||||||
omitOutputSchemas,
|
outputSchemaProfile,
|
||||||
);
|
);
|
||||||
|
|
||||||
it("omits only output schema advertisements and keeps canonical MCP output validation", async () => {
|
it("omits only output schema advertisements and keeps canonical MCP output validation", async () => {
|
||||||
const server = createServer(true);
|
const server = createServer("off");
|
||||||
expect(server.catalogIdentity.digests.combined_sha256).not.toBe(
|
expect(server.catalogIdentity.digests.combined_sha256).not.toBe(
|
||||||
FULL_CATALOG_IDENTITY.digests.combined_sha256,
|
FULL_CATALOG_IDENTITY.digests.combined_sha256,
|
||||||
);
|
);
|
||||||
@@ -86,9 +97,97 @@ it("omits only output schema advertisements and keeps canonical MCP output valid
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
it("keeps a full catalog below a measured one-megabyte frame when output schemas are off", async () => {
|
it("bounds output schema advertisements and keeps canonical MCP output validation", async () => {
|
||||||
const measure = async (omitOutputSchemas: boolean): Promise<number> => {
|
const server = createServer("compact");
|
||||||
const server = createServer(omitOutputSchemas);
|
expect(server.catalogIdentity.digests.combined_sha256).not.toBe(
|
||||||
|
FULL_CATALOG_IDENTITY.digests.combined_sha256,
|
||||||
|
);
|
||||||
|
const [clientTransport, serverTransport] =
|
||||||
|
InMemoryTransport.createLinkedPair();
|
||||||
|
const client = new Client({ name: "compact-output-test", version: "1" });
|
||||||
|
const contract = {
|
||||||
|
...toolContract("comment"),
|
||||||
|
name: "compact_output_schema_fixture",
|
||||||
|
inputSchema: z.strictObject({}),
|
||||||
|
outputSchema: z.strictObject({
|
||||||
|
value: z.string().describe("Nested guidance leaves."),
|
||||||
|
nested: z
|
||||||
|
.object({ count: z.number().int() })
|
||||||
|
.describe("Nested guidance leaves."),
|
||||||
|
}),
|
||||||
|
} satisfies ToolContract;
|
||||||
|
server.registerTool(
|
||||||
|
contract.name,
|
||||||
|
toolRegistrationOptions(contract),
|
||||||
|
async () => ({
|
||||||
|
content: [{ type: "text" as const, text: "fixture" }],
|
||||||
|
structuredContent: { value: "complete", nested: { count: 1 } },
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
const invalidContract = {
|
||||||
|
...contract,
|
||||||
|
name: "invalid_compact_output_schema_fixture",
|
||||||
|
} satisfies ToolContract;
|
||||||
|
server.registerTool(
|
||||||
|
invalidContract.name,
|
||||||
|
toolRegistrationOptions(invalidContract),
|
||||||
|
async () => ({
|
||||||
|
content: [{ type: "text" as const, text: "invalid fixture" }],
|
||||||
|
structuredContent: { value: 42, nested: { count: 1 } },
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
await server.connect(serverTransport);
|
||||||
|
await client.connect(clientTransport);
|
||||||
|
try {
|
||||||
|
const tools = (await client.listTools()).tools;
|
||||||
|
expect(tools).toHaveLength(2);
|
||||||
|
const advertised = tools.find((tool) => tool.name === contract.name)!
|
||||||
|
.outputSchema as Record<string, unknown>;
|
||||||
|
expect(advertised.type).toBe("object");
|
||||||
|
expect(advertised.$id).toBeUndefined();
|
||||||
|
expect((advertised.properties as Record<string, unknown>).value).toEqual({
|
||||||
|
type: "string",
|
||||||
|
});
|
||||||
|
expect(
|
||||||
|
(advertised.properties as Record<string, unknown>).nested,
|
||||||
|
).toMatchObject({ type: "object" });
|
||||||
|
|
||||||
|
const success = await client.callTool({
|
||||||
|
name: contract.name,
|
||||||
|
arguments: {},
|
||||||
|
});
|
||||||
|
expect(success.isError).not.toBe(true);
|
||||||
|
expect(success.structuredContent).toEqual({
|
||||||
|
value: "complete",
|
||||||
|
nested: { count: 1 },
|
||||||
|
});
|
||||||
|
const invalid = await client.callTool({
|
||||||
|
name: invalidContract.name,
|
||||||
|
arguments: {},
|
||||||
|
});
|
||||||
|
expect(invalid.isError).toBe(true);
|
||||||
|
expect(invalid.content).toEqual([
|
||||||
|
expect.objectContaining({
|
||||||
|
type: "text",
|
||||||
|
text: expect.stringContaining("Output validation error"),
|
||||||
|
}),
|
||||||
|
]);
|
||||||
|
} finally {
|
||||||
|
await client.close();
|
||||||
|
await server.close();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps the default compact-input catalog below a measured one-megabyte frame", async () => {
|
||||||
|
const compactInput: CompactInputSchemaPresentation = {
|
||||||
|
budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
|
||||||
|
};
|
||||||
|
const measure = async (
|
||||||
|
profile: McpOutputSchemaProfile,
|
||||||
|
inputPresentation: CompactInputSchemaPresentation | undefined,
|
||||||
|
): Promise<number> => {
|
||||||
|
const server = createServer(profile, inputPresentation);
|
||||||
const [clientTransport, serverTransport] =
|
const [clientTransport, serverTransport] =
|
||||||
InMemoryTransport.createLinkedPair();
|
InMemoryTransport.createLinkedPair();
|
||||||
const client = new Client({ name: "catalog-size-test", version: "1" });
|
const client = new Client({ name: "catalog-size-test", version: "1" });
|
||||||
@@ -104,6 +203,26 @@ it("keeps a full catalog below a measured one-megabyte frame when output schemas
|
|||||||
await client.connect(clientTransport);
|
await client.connect(clientTransport);
|
||||||
try {
|
try {
|
||||||
const listed = await client.listTools();
|
const listed = await client.listTools();
|
||||||
|
if (profile === "compact") {
|
||||||
|
// Validate the advertised documents as the client receives them, not
|
||||||
|
// just their bytes: a projection that broke a reference or left an
|
||||||
|
// invalid schema would otherwise still pass the budget assertion.
|
||||||
|
expect(
|
||||||
|
advertisedSchemaErrors(
|
||||||
|
listed.tools.map((tool) => ({
|
||||||
|
name: tool.name,
|
||||||
|
inputSchema: tool.inputSchema,
|
||||||
|
outputSchema: tool.outputSchema,
|
||||||
|
})),
|
||||||
|
),
|
||||||
|
).toEqual([]);
|
||||||
|
for (const tool of listed.tools) {
|
||||||
|
expect(tool.outputSchema).toBeDefined();
|
||||||
|
expect(
|
||||||
|
Buffer.byteLength(JSON.stringify(tool.outputSchema), "utf8"),
|
||||||
|
).toBeLessThanOrEqual(COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES);
|
||||||
|
}
|
||||||
|
}
|
||||||
return Buffer.byteLength(JSON.stringify(listed), "utf8");
|
return Buffer.byteLength(JSON.stringify(listed), "utf8");
|
||||||
} finally {
|
} finally {
|
||||||
await client.close();
|
await client.close();
|
||||||
@@ -111,9 +230,120 @@ it("keeps a full catalog below a measured one-megabyte frame when output schemas
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
const fullBytes = await measure(false);
|
const fullBytes = await measure("full", undefined);
|
||||||
const compactBytes = await measure(true);
|
const compactBytes = await measure("compact", compactInput);
|
||||||
|
const offBytes = await measure("off", compactInput);
|
||||||
|
const fullInputOffBytes = await measure("off", undefined);
|
||||||
expect(fullBytes).toBeGreaterThan(1024 * 1024);
|
expect(fullBytes).toBeGreaterThan(1024 * 1024);
|
||||||
expect(compactBytes).toBeLessThan(1024 * 1024);
|
expect(compactBytes).toBeLessThan(1024 * 1024);
|
||||||
|
expect(offBytes).toBeLessThan(1024 * 1024);
|
||||||
|
expect(fullInputOffBytes).toBeLessThan(1024 * 1024);
|
||||||
expect(compactBytes).toBeLessThan(fullBytes);
|
expect(compactBytes).toBeLessThan(fullBytes);
|
||||||
|
expect(compactBytes).toBeGreaterThan(offBytes);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects canonical-invalid output that the reduced advertisement accepts", async () => {
|
||||||
|
// Enough nested detail that the compact projection cannot inline the
|
||||||
|
// canonical schema within the budget, so the advertised form is the shallow
|
||||||
|
// root shape rather than the complete validation structure.
|
||||||
|
const nestedDetail = Object.fromEntries(
|
||||||
|
Array.from({ length: 150 }, (_, index) => [
|
||||||
|
`nested_${String(index).padStart(3, "0")}`,
|
||||||
|
z.string().optional(),
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
const outputSchema = z.strictObject({
|
||||||
|
value: z.string(),
|
||||||
|
payload: z.object({
|
||||||
|
inner: z.string(),
|
||||||
|
count: z.number().int(),
|
||||||
|
...nestedDetail,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
const server = createServer("compact");
|
||||||
|
const [clientTransport, serverTransport] =
|
||||||
|
InMemoryTransport.createLinkedPair();
|
||||||
|
const client = new Client({ name: "reduced-output-test", version: "1" });
|
||||||
|
const contract = {
|
||||||
|
...toolContract("comment"),
|
||||||
|
name: "reduced_output_schema_fixture",
|
||||||
|
inputSchema: z.strictObject({}),
|
||||||
|
outputSchema,
|
||||||
|
} satisfies ToolContract;
|
||||||
|
server.registerTool(
|
||||||
|
contract.name,
|
||||||
|
toolRegistrationOptions(contract),
|
||||||
|
async () => ({
|
||||||
|
content: [{ type: "text" as const, text: "fixture" }],
|
||||||
|
structuredContent: {
|
||||||
|
value: "complete",
|
||||||
|
payload: { inner: "text", count: 1 },
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
// `payload.inner` is a string canonically. The shallow advertisement keeps
|
||||||
|
// only `payload: {type: "object"}`, so it accepts the malformed value.
|
||||||
|
const malformed = {
|
||||||
|
value: "complete",
|
||||||
|
payload: { inner: 42, count: 1 },
|
||||||
|
};
|
||||||
|
const invalidContract = {
|
||||||
|
...contract,
|
||||||
|
name: "invalid_reduced_output_schema_fixture",
|
||||||
|
} satisfies ToolContract;
|
||||||
|
server.registerTool(
|
||||||
|
invalidContract.name,
|
||||||
|
toolRegistrationOptions(invalidContract),
|
||||||
|
async () => ({
|
||||||
|
content: [{ type: "text" as const, text: "invalid fixture" }],
|
||||||
|
structuredContent: malformed,
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
await server.connect(serverTransport);
|
||||||
|
await client.connect(clientTransport);
|
||||||
|
try {
|
||||||
|
const tools = (await client.listTools()).tools;
|
||||||
|
const advertised = tools.find((tool) => tool.name === contract.name)!
|
||||||
|
.outputSchema as Record<string, unknown>;
|
||||||
|
expect(advertised.description).toContain("reduced form");
|
||||||
|
expect((advertised.properties as Record<string, unknown>).payload).toEqual({
|
||||||
|
type: "object",
|
||||||
|
});
|
||||||
|
expect(
|
||||||
|
Buffer.byteLength(JSON.stringify(advertised), "utf8"),
|
||||||
|
).toBeLessThanOrEqual(COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES);
|
||||||
|
// The regression this guards: the advertisement accepts a value the
|
||||||
|
// canonical contract rejects, so validating results against the
|
||||||
|
// advertisement instead of the contract would deliver malformed output.
|
||||||
|
const validateAdvertised = new Ajv2020({
|
||||||
|
strict: false,
|
||||||
|
validateFormats: false,
|
||||||
|
}).compile(advertised);
|
||||||
|
expect(validateAdvertised(malformed)).toBe(true);
|
||||||
|
|
||||||
|
const success = await client.callTool({
|
||||||
|
name: contract.name,
|
||||||
|
arguments: {},
|
||||||
|
});
|
||||||
|
expect(success.isError).not.toBe(true);
|
||||||
|
expect(success.structuredContent).toEqual({
|
||||||
|
value: "complete",
|
||||||
|
payload: { inner: "text", count: 1 },
|
||||||
|
});
|
||||||
|
const invalid = await client.callTool({
|
||||||
|
name: invalidContract.name,
|
||||||
|
arguments: {},
|
||||||
|
});
|
||||||
|
expect(invalid.isError).toBe(true);
|
||||||
|
expect(invalid.content).toEqual([
|
||||||
|
expect.objectContaining({
|
||||||
|
type: "text",
|
||||||
|
text: expect.stringContaining("Output validation error"),
|
||||||
|
}),
|
||||||
|
]);
|
||||||
|
} finally {
|
||||||
|
await client.close();
|
||||||
|
await server.close();
|
||||||
|
}
|
||||||
});
|
});
|
||||||
@@ -13,6 +13,7 @@ import {
|
|||||||
arrayItemSchemaError,
|
arrayItemSchemaError,
|
||||||
unsupportedInputLiteralSchemaError,
|
unsupportedInputLiteralSchemaError,
|
||||||
} from "../../fixtures/mcpClientSchemaConstraints.js";
|
} from "../../fixtures/mcpClientSchemaConstraints.js";
|
||||||
|
import { advertisedSchemaErrors } from "../../fixtures/mcpAdvertisedSchemaValidity.js";
|
||||||
import { toolRegistrationOptions } from "../../../src/server/toolRegistrationOptions.js";
|
import { toolRegistrationOptions } from "../../../src/server/toolRegistrationOptions.js";
|
||||||
import { servedCatalog } from "../../../src/application/environment/environmentState.js";
|
import { servedCatalog } from "../../../src/application/environment/environmentState.js";
|
||||||
import { servedFrom } from "../../fixtures/environmentStateServing.js";
|
import { servedFrom } from "../../fixtures/environmentStateServing.js";
|
||||||
@@ -31,47 +32,6 @@ interface ToolSchemas {
|
|||||||
readonly annotations?: Record<string, unknown> | undefined;
|
readonly annotations?: Record<string, unknown> | undefined;
|
||||||
}
|
}
|
||||||
|
|
||||||
function schemaErrors(tools: readonly ToolSchemas[]): string[] {
|
|
||||||
// Reuse Ajv's meta-schema validators, then clear each root's reference
|
|
||||||
// namespace, including nested IDs, before validating the next schema.
|
|
||||||
const ajv = new Ajv2020({
|
|
||||||
strict: false,
|
|
||||||
validateFormats: false,
|
|
||||||
addUsedSchema: false,
|
|
||||||
});
|
|
||||||
return tools.flatMap((tool) =>
|
|
||||||
["inputSchema", "outputSchema"].flatMap((kind) => {
|
|
||||||
const schema =
|
|
||||||
kind === "inputSchema" ? tool.inputSchema : tool.outputSchema;
|
|
||||||
if (schema === undefined) return [];
|
|
||||||
try {
|
|
||||||
if (!ajv.validateSchema(schema))
|
|
||||||
return [`${tool.name}.${kind}: ${ajv.errorsText(ajv.errors)}`];
|
|
||||||
ajv.compile(schema);
|
|
||||||
const clientError = arrayItemSchemaError(
|
|
||||||
schema,
|
|
||||||
`${tool.name}.${kind}`,
|
|
||||||
);
|
|
||||||
if (clientError !== undefined) return [clientError];
|
|
||||||
if (kind === "inputSchema") {
|
|
||||||
const literalError = unsupportedInputLiteralSchemaError(
|
|
||||||
schema,
|
|
||||||
`${tool.name}.${kind}`,
|
|
||||||
);
|
|
||||||
if (literalError !== undefined) return [literalError];
|
|
||||||
}
|
|
||||||
return [];
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
return [
|
|
||||||
`${tool.name}.${kind}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
|
||||||
];
|
|
||||||
} finally {
|
|
||||||
ajv.removeSchema();
|
|
||||||
}
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
function expectStrictInputSchemaParity(
|
function expectStrictInputSchemaParity(
|
||||||
tools: readonly ToolSchemas[],
|
tools: readonly ToolSchemas[],
|
||||||
ajv: Ajv2020,
|
ajv: Ajv2020,
|
||||||
@@ -484,7 +444,7 @@ describe("served environment catalogs", () => {
|
|||||||
);
|
);
|
||||||
expect(tools.some((tool) => tool.name === "delete_books")).toBe(true);
|
expect(tools.some((tool) => tool.name === "delete_books")).toBe(true);
|
||||||
expect(
|
expect(
|
||||||
schemaErrors(
|
advertisedSchemaErrors(
|
||||||
tools.map((tool) => ({
|
tools.map((tool) => ({
|
||||||
name: tool.name,
|
name: tool.name,
|
||||||
inputSchema: tool.inputSchema as unknown as Record<string, unknown>,
|
inputSchema: tool.inputSchema as unknown as Record<string, unknown>,
|
||||||
@@ -534,7 +494,7 @@ describe("MCP JSON Schema validity", () => {
|
|||||||
expect(tools.map(({ name }) => name).sort()).toEqual(
|
expect(tools.map(({ name }) => name).sort()).toEqual(
|
||||||
TOOL_CONTRACTS.map(({ name }) => name).sort(),
|
TOOL_CONTRACTS.map(({ name }) => name).sort(),
|
||||||
);
|
);
|
||||||
expect(schemaErrors(tools)).toEqual([]);
|
expect(advertisedSchemaErrors(tools)).toEqual([]);
|
||||||
expect(localInputReferences(tools)).toEqual([]);
|
expect(localInputReferences(tools)).toEqual([]);
|
||||||
expect(recursiveReferences(tools)).toEqual([]);
|
expect(recursiveReferences(tools)).toEqual([]);
|
||||||
expectKnownAuthorityHints(tools);
|
expectKnownAuthorityHints(tools);
|
||||||
@@ -879,7 +839,7 @@ describe("MCP process input JSON Schema", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("ships nonrecursive input and output schemas in the generated catalog", () => {
|
it("ships nonrecursive input and output schemas in the generated catalog", () => {
|
||||||
expect(schemaErrors(GENERATED_MCP_TOOL_CATALOG)).toEqual([]);
|
expect(advertisedSchemaErrors(GENERATED_MCP_TOOL_CATALOG)).toEqual([]);
|
||||||
expect(recursiveReferences(GENERATED_MCP_TOOL_CATALOG)).toEqual([]);
|
expect(recursiveReferences(GENERATED_MCP_TOOL_CATALOG)).toEqual([]);
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -888,7 +848,7 @@ it("ships input schemas without local references in the generated catalog", () =
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("rejects unresolved output-schema references even for uncalled tools", () => {
|
it("rejects unresolved output-schema references even for uncalled tools", () => {
|
||||||
const errors = schemaErrors([
|
const errors = advertisedSchemaErrors([
|
||||||
{
|
{
|
||||||
name: "dangling",
|
name: "dangling",
|
||||||
inputSchema: { type: "object" },
|
inputSchema: { type: "object" },
|
||||||
@@ -901,7 +861,7 @@ it("rejects unresolved output-schema references even for uncalled tools", () =>
|
|||||||
|
|
||||||
it("validates each tool independently when output schema IDs repeat", () => {
|
it("validates each tool independently when output schema IDs repeat", () => {
|
||||||
const outputSchema = { $id: "urn:rea:test:shared-output", type: "string" };
|
const outputSchema = { $id: "urn:rea:test:shared-output", type: "string" };
|
||||||
const errors = schemaErrors([
|
const errors = advertisedSchemaErrors([
|
||||||
{ name: "first", inputSchema: { type: "object" }, outputSchema },
|
{ name: "first", inputSchema: { type: "object" }, outputSchema },
|
||||||
{
|
{
|
||||||
name: "second",
|
name: "second",
|
||||||
@@ -974,7 +934,7 @@ it.each([false, true])(
|
|||||||
"rejects references to another tool's nested IDs (earlier compilation fails: %s)",
|
"rejects references to another tool's nested IDs (earlier compilation fails: %s)",
|
||||||
(invalidPrevious) => {
|
(invalidPrevious) => {
|
||||||
const nestedId = "urn:rea:test:nested-output";
|
const nestedId = "urn:rea:test:nested-output";
|
||||||
const errors = schemaErrors([
|
const errors = advertisedSchemaErrors([
|
||||||
{
|
{
|
||||||
name: "first",
|
name: "first",
|
||||||
inputSchema: { type: "object" },
|
inputSchema: { type: "object" },
|
||||||
|
|||||||
@@ -0,0 +1,43 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
|
||||||
|
import {
|
||||||
|
COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES,
|
||||||
|
compactPresentationForMcpOutputSchemaProfile,
|
||||||
|
parseMcpOutputSchemaProfile,
|
||||||
|
} from "../../../src/config/mcpOutputSchemaProfile.js";
|
||||||
|
|
||||||
|
describe("MCP output schema profile configuration", () => {
|
||||||
|
it("accepts the full, compact, and off presentations", () => {
|
||||||
|
expect(parseMcpOutputSchemaProfile(undefined)).toEqual({
|
||||||
|
ok: true,
|
||||||
|
value: undefined,
|
||||||
|
});
|
||||||
|
for (const value of ["full", "compact", "off"] as const)
|
||||||
|
expect(parseMcpOutputSchemaProfile(value)).toEqual({
|
||||||
|
ok: true,
|
||||||
|
value,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(["", "FULL", "compact ", "off ", "minimal", "0"])(
|
||||||
|
"rejects unknown profiles: %j",
|
||||||
|
(value) => {
|
||||||
|
const parsed = parseMcpOutputSchemaProfile(value);
|
||||||
|
if (parsed.ok) throw new Error("Expected invalid profile");
|
||||||
|
expect(parsed.error.message).toContain("REA_MCP_OUTPUT_SCHEMA_PROFILE");
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
it("selects a bounded presentation only for compact", () => {
|
||||||
|
expect(compactPresentationForMcpOutputSchemaProfile("compact")).toEqual({
|
||||||
|
budgetBytes: COMPACT_OUTPUT_SCHEMA_BUDGET_BYTES,
|
||||||
|
});
|
||||||
|
expect(compactPresentationForMcpOutputSchemaProfile(undefined)).toBe(
|
||||||
|
undefined,
|
||||||
|
);
|
||||||
|
expect(compactPresentationForMcpOutputSchemaProfile("full")).toBe(
|
||||||
|
undefined,
|
||||||
|
);
|
||||||
|
expect(compactPresentationForMcpOutputSchemaProfile("off")).toBe(undefined);
|
||||||
|
});
|
||||||
|
});
|
||||||
+67
@@ -0,0 +1,67 @@
|
|||||||
|
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||||
|
|
||||||
|
import {
|
||||||
|
arrayItemSchemaError,
|
||||||
|
unsupportedInputLiteralSchemaError,
|
||||||
|
} from "./mcpClientSchemaConstraints.js";
|
||||||
|
|
||||||
|
/** Advertised schema pair as a client receives it from `tools/list`. */
|
||||||
|
export interface AdvertisedToolSchemas {
|
||||||
|
readonly name: string;
|
||||||
|
readonly inputSchema: Record<string, unknown>;
|
||||||
|
readonly outputSchema?: Record<string, unknown> | undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Validate advertised schemas as clients receive them: Ajv's declared-dialect
|
||||||
|
* meta-schema and reference resolution, then the constraints a real client
|
||||||
|
* enforces. Compiling catches a compact projection that drops a referenced
|
||||||
|
* definition or leaves a local `$ref` without its target, which a byte-budget
|
||||||
|
* assertion alone would not.
|
||||||
|
*/
|
||||||
|
export function advertisedSchemaErrors(
|
||||||
|
tools: readonly AdvertisedToolSchemas[],
|
||||||
|
): string[] {
|
||||||
|
// Reuse Ajv's meta-schema validators, then clear each root's reference
|
||||||
|
// namespace, including nested IDs, before validating the next schema.
|
||||||
|
const ajv = new Ajv2020({
|
||||||
|
strict: false,
|
||||||
|
validateFormats: false,
|
||||||
|
addUsedSchema: false,
|
||||||
|
// This check compiles schemas to detect invalid references but never runs
|
||||||
|
// their validators. Generated-code optimization adds substantial work here
|
||||||
|
// without checking another contract; actual SDK validators keep defaults.
|
||||||
|
code: { optimize: false },
|
||||||
|
});
|
||||||
|
return tools.flatMap((tool) =>
|
||||||
|
["inputSchema", "outputSchema"].flatMap((kind) => {
|
||||||
|
const schema =
|
||||||
|
kind === "inputSchema" ? tool.inputSchema : tool.outputSchema;
|
||||||
|
if (schema === undefined) return [];
|
||||||
|
try {
|
||||||
|
if (!ajv.validateSchema(schema))
|
||||||
|
return [`${tool.name}.${kind}: ${ajv.errorsText(ajv.errors)}`];
|
||||||
|
ajv.compile(schema);
|
||||||
|
const clientError = arrayItemSchemaError(
|
||||||
|
schema,
|
||||||
|
`${tool.name}.${kind}`,
|
||||||
|
);
|
||||||
|
if (clientError !== undefined) return [clientError];
|
||||||
|
if (kind === "inputSchema") {
|
||||||
|
const literalError = unsupportedInputLiteralSchemaError(
|
||||||
|
schema,
|
||||||
|
`${tool.name}.${kind}`,
|
||||||
|
);
|
||||||
|
if (literalError !== undefined) return [literalError];
|
||||||
|
}
|
||||||
|
return [];
|
||||||
|
} catch (cause: unknown) {
|
||||||
|
return [
|
||||||
|
`${tool.name}.${kind}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||||
|
];
|
||||||
|
} finally {
|
||||||
|
ajv.removeSchema();
|
||||||
|
}
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
}
|
||||||
Reference in new issue
Block a user