perf(mcp)!: default to compact input schema advertisements (#5975)

Use compact MCP input advertisements by default while preserving explicit full, canonical validation, output contracts and complete Evidence. Require an explicit selected catalog for runtime identity and honor both profiles in CLI status.

Real ripgrep/Zellij SDK tasks, independent ELF oracle, actual client trials, profile parity workflows and CI are recorded in experiments/analysis-efficiency/mcp-input-catalog.md. SDK token estimates improve; actual Codex token savings and general latency/memory gains are not established.

Refs #5861

BREAKING CHANGE: MCP input advertisements default to compact. Set REA_MCP_INPUT_SCHEMA_PROFILE=full for complete advertised guidance and nested constraints.
This commit is contained in:
gensokyo-dev authored and GitHub committed 2026-10-22 00:17:11 +08:00
1 parent 01ee6551b4
commit aa51e43439
45 files changed
+3887 -331

No files matched your search

+6
View File
@@ -90,6 +90,12 @@ Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
[installation and setup](docs/installation.md) for provider configuration and
manual MCP registration.
MCP uses compact input schemas by default to reduce context overhead. Set
`REA_MCP_INPUT_SCHEMA_PROFILE=full` in your MCP registration when you need the
complete advertised field guidance and nested validation details. Both profiles
keep the full tool set and canonical server validation. See
"Compact input schema profile" in the MCP contracts guide.
### Ask your agent
```text
+2 -2
View File
@@ -17,8 +17,8 @@ shared form would leave later fields unknown instead. The existing
`REA_MCP_INPUT_SCHEMA_PROFILE=compact` profile lets callers stay below
its character limit by advertising reduced nested validation structure, while
REA continues to validate calls against the complete canonical input schema.
Compact applies to the server's advertised inputs and is not an automatic
full-profile fallback; see the [compact input schema profile](mcp-contracts.md#compact-input-schema-profile)
Compact is the default for the server's advertised inputs; an explicit `full`
choice is preserved. See the [compact input schema profile](mcp-contracts.md#compact-input-schema-profile)
for details of its reduced forms.
Custom renderer hosts can raise `inputMaxChars` through `renderToolSignature` or
+6 -5
View File
@@ -1020,14 +1020,15 @@ verify the compact advertisement through the same client boundary.
Some function-calling providers reject a request while any single tool's
`parameters` schema exceeds a literal serialized size cap; Moonshot clients
such as kimi-cli fail every request while the default catalog is attached
such as kimi-cli have reported failures with the full catalog attached
([#1484](https://github.com/morluto/rea/issues/1484)). Setting the server's
`REA_MCP_INPUT_SCHEMA_PROFILE` environment variable to `compact` renders every
advertised input schema within a 13,312-byte budget, measured to sit below
`REA_MCP_INPUT_SCHEMA_PROFILE` environment variable to `compact` (the default)
renders every advertised input schema within a 13,312-byte budget, measured to sit below
Moonshot's observed cap (community measurements: a 13.7 KB schema is accepted
while a ~15 KB one is rejected). The setting is read at startup; restart the
server to apply changes, as with `REA_MCP_MAX_RESPONSE_BYTES`. The default
value `full` keeps the complete advertisement unchanged. Compact presentation
server to apply changes, as with `REA_MCP_MAX_RESPONSE_BYTES`. Explicitly select
`full` to retain complete advertised input guidance and nested validation detail.
Existing explicit profile choices are preserved. Compact presentation
also reduces schemas whose expanded JSON tree exceeds ten levels, the limit
reported by strict-depth MCP clients in [#2813](https://github.com/morluto/rea/issues/2813).
WorkBuddy/CodeBuddy gateways have also rejected full REA catalogs with an
@@ -0,0 +1,90 @@
"""Account for catalog, complete SDK tasks and separately recorded model trials."""
import argparse
import hashlib
import json
from pathlib import Path
import tiktoken
def compact(value):
return json.dumps(value, ensure_ascii=False, separators=(",", ":"), allow_nan=False)
def resource(case):
path = case.with_name(case.name + ".log")
scope = next(json.loads(line) for line in reversed(path.read_text().splitlines())
if line.startswith('{"resource_scope":'))
observation = scope["payload_resource_observation"]
assert scope["abort"] is None and scope["payload_resource_observation_verified"]
assert observation["payload_cleanup_verified"] and observation["payload_exit_code"] == 0
assert all(observation["resource_events"]["memory.events"][key] == 0
for key in ["high", "max", "oom", "oom_kill"])
return {"wall_seconds": scope["elapsed_seconds"],
"cpu_seconds": observation["cpu_stat"]["usage_usec"] / 1e6,
"kernel_family_peak_bytes_including_page_cache": observation["memory_peak_bytes"],
"resource_stop": scope["abort"], "cleanup_verified": True,
"final_whole_scope_counters": scope["resource_events_final"],
"guard_sources_sha256": scope["payload_observer_sources_sha256"],
"log_sha256": hashlib.sha256(path.read_bytes()).hexdigest()}
def sdk(case, count):
report = json.loads((case / "report.json").read_text())
assert report["status"] == "passed" and report["cleanup_verified"]
catalog = json.loads((case / "catalog.json").read_text())
calls = json.loads((case / "transcript.json").read_text())
inputs = [{key: tool[key] for key in ["name", "description", "inputSchema"]}
for tool in catalog["tools"]]
arguments = sum(count(compact({"name": call["name"], "arguments": call["arguments"]}))
for call in calls)
text = sum(count(block["text"]) for call in calls for block in call["reply"]["content"]
if block["type"] == "text")
return {"case": case.name, "kind": "SDK", "profile": report["profile"],
"artifact_sha256": report["artifact_sha256"],
"tool_count": len(catalog["tools"]), "calls": report["calls"],
"parent_evidence_id": report["parent_evidence_id"],
"parent_semantic_sha256": report["parent_semantic_sha256"],
"token_estimates": {"input_catalog_once": count(compact(inputs)),
"arguments": arguments, "text_replies": text,
"whole_task": count(compact(inputs)) + arguments + text,
"raw_catalog_including_output_schemas": count(compact(catalog))},
"actual_model_usage": None, **resource(case),
"receipt_sha256": {name: hashlib.sha256((case / name).read_bytes()).hexdigest()
for name in ["report.json", "catalog.json", "transcript.json"]}}
def model(case):
summary = json.loads((case / "summary.json").read_text())
assert summary["exit_code"] == 0 and not summary["timeout"]
assert summary["selected_artifact_bytes_unchanged"]
assert json.loads((case / "cleanup.json").read_text())["profile_removed"]
return {"case": case.name, "kind": "model", "client": summary["codex_version"],
"requested_model": summary["requested_model"], "reasoning": summary["reasoning"],
"model_identity_scope": summary["model_identity_scope"],
"actual_model_usage": summary["usage"],
"artifact_sha256": summary["selected_artifact_sha256_before"],
"calls": summary["completed_mcp_calls"],
"factual_correctness": "requires independent oracle and answer review",
**resource(case),
"receipt_sha256": {name: hashlib.sha256((case / name).read_bytes()).hexdigest()
for name in ["summary.json", "answer.txt", "events.jsonl",
"trial-settings.json", "context-receipt.json"]}}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("cases", type=Path, nargs="+")
args = parser.parse_args()
encoding = tiktoken.get_encoding("cl100k_base")
count = lambda value: len(encoding.encode(value, disallowed_special=()))
result = {"tokenizer": f"tiktoken {tiktoken.__version__}", "encoding": encoding.name,
"estimate_scope": "One complete name/description/input-schema catalog, every RPC argument and text reply; raw output-schema transport counted separately. Model usage is separate and is not inferred from estimates.",
"records": [sdk(case, count) if (case / "report.json").exists() else model(case)
for case in args.cases]}
args.output.write_text(json.dumps(result, indent=2) + "\n")
if __name__ == "__main__":
main()
@@ -0,0 +1,54 @@
"""Independent ELF64 little-endian / zlib oracle for the pinned Alpine corpus."""
import argparse
import hashlib
import json
from pathlib import Path
import struct
import zlib
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("artifact", type=Path)
parser.add_argument("output", type=Path)
args = parser.parse_args()
assert args.artifact.stat().st_size <= 256 * 1024 ** 2
binary = args.artifact.read_bytes()
assert binary[:6] == b"\x7fELF\x02\x01"
table = struct.unpack_from("<Q", binary, 40)[0]
stride, count, strings_index = struct.unpack_from("<HHH", binary, 58)
assert stride == 64 and 0 < count < 65535 and strings_index < count
sections = [struct.unpack_from("<IIQQQQIIQQ", binary, table + index * stride)
for index in range(count)]
string_row = sections[strings_index]
names = binary[string_row[4]:string_row[4] + string_row[5]]
matches = [row for row in sections
if names[row[0]:names.index(b"\0", row[0])] == b".dep-v0"]
assert len(matches) == 1
row = matches[0]
assert row[5] <= 1024 ** 2
compressed = binary[row[4]:row[4] + row[5]]
decoder = zlib.decompressobj()
decoded = decoder.decompress(compressed, 8 * 1024 ** 2 + 1)
assert len(decoded) <= 8 * 1024 ** 2 and decoder.eof
assert not decoder.unconsumed_tail and not decoder.unused_data
raw = json.loads(decoded)
packages = [{"name": item["name"], "version": item["version"],
"source": item.get("source"), "kind": item.get("kind"),
"dependencies": item.get("dependencies", []), "root": item.get("root", False)}
for item in raw["packages"]]
result = {
"artifact_sha256": hashlib.sha256(binary).hexdigest(),
"section_sha256": hashlib.sha256(compressed).hexdigest(),
"json_sha256": hashlib.sha256(decoded).hexdigest(),
"packages": packages,
"edges": [{"from_index": index, "to_index": destination}
for index, item in enumerate(packages) for destination in item["dependencies"]],
"serde_rows": [{"index": index, **{key: item[key] for key in ["name", "version", "kind"]}}
for index, item in enumerate(packages) if item["name"] == "serde"],
}
args.output.write_text(json.dumps(result, indent=2) + "\n")
if __name__ == "__main__":
main()
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,228 @@
# Default MCP input catalog cost
Continues [#5861](https://github.com/morluto/rea/issues/5861). Baseline is
`705058f3e2df82503f1c601fc464aea7cbb626b8`. The candidate selects the existing
compact input presentation when the setting is omitted. Explicit `full` and
`compact` choices, all 213 tools, descriptions, output contracts, canonical
validation, provider behavior and complete Evidence remain available.
This is a change to the default advertisement. Callers requiring full field
guidance and nested validation declarations can set
`REA_MCP_INPUT_SCHEMA_PROFILE=full`. Compact drops nested annotation prose and
reduces structurally oversized/deep schemas; it is not an equivalent complete
input declaration. The canonical server validator still admits or rejects the
same inputs. No new profile, tool-selection API, cache or discovery protocol is
introduced. Output schemas retain their existing default.
## Completed SDK tasks
The [receipt](mcp-input-catalog.json) pins sources, publisher URLs/digests,
individual calls and resource observations. The unchanged Alpine v3.21 x86_64
releases are ripgrep 14.1.1-r0 (3,416,168 bytes) and Zellij 0.40.0-r0
(17,151,864 bytes). Neither binary was executed. The independent
[oracle](mcp-input-catalog-oracle.py) reads ELF section headers, decompresses
`.dep-v0` with Python zlib, and derives original package indices and dependency
edges from the embedded JSON, independently of REA.
Each [production stdio SDK task](mcp-input-catalog.mjs) discovers and validates
the complete advertised input catalog, requests exact serde package fields,
reads the same retained page twice, probes a negative offset, exports complete
Evidence, compares every package and edge with the oracle, and closes the
transport. RPCs bind the exact advertised named tool definition. Tool metadata
outside input schemas and complete parent Evidence are equal before/after.
The real negative input is refused in every task.
Three fresh-process pairs per artifact alternate order (before/after,
after/before, before/after). Both runtimes use Node 24.18.0, npm 11.16.0 and
MCP SDK 2.3.1 on Linux x86_64. Node compile caching is disabled; OS caches are
not flushed and unrelated host load is uncontrolled. Artifact acquisition,
builds and oracle preparation are outside the measured analysis task.
| Artifact | Default before/after | Whole-task estimated tokens | Wall median seconds | CPU median seconds | Family peak median MiB |
| -------- | -------------------- | --------------------------: | ------------------: | -----------------: | ---------------------: |
| ripgrep | full | 204,358 | 7.379 | 7.159 | 804.3 |
| ripgrep | compact | 51,801 | 7.274 | 7.036 | 815.5 |
| Zellij | full | 204,278 | 7.373 | 7.162 | 839.0 |
| Zellij | compact | 51,721 | 7.374 | 7.131 | 827.0 |
The input-facing catalog alone changes from 200,425 to 47,868 estimated tokens
(76.1% lower). Each whole task includes the catalog once, all five calls'
arguments and text replies: about 74.7% lower. These are offline
`tiktoken==0.12.0` / `cl100k_base` estimates, not model usage. The raw SDK catalog
including output schemas changes from 903,377 to 750,623 estimated tokens;
clients exposing that entire representation have a smaller proportional gain.
JSON-RPC envelopes and client-specific conversion/replay are unmeasured here.
Wall ranges overlap (ripgrep 7.27–7.68 vs 7.07–7.27 seconds; Zellij 7.17–7.48 vs
7.07–7.38). No general elapsed, CPU or memory improvement is established.
Fresh processes include first-use cost and two same-session historical reads;
the latter are not fresh live-state observations.
## Actual model tasks and the limit of the claim
Six real Codex CLI 0.161.0 tasks used fresh isolated profiles, the complete
catalog, no enabled-tool filter, and requested `gpt-6.1-sol` / medium reasoning.
The resolved server-side model identity is unknown. Each profile completed a
direct ripgrep package question, a broad Zellij metadata question, and an
intentional invalid-offset question. Artifact bytes were unchanged and owned
profiles were removed. Answers match independent package/index/count/digest
facts, cite actual returned Evidence or its parent, preserve embedded-metadata
limitations, and accurately report the negative cases without retrying.
| Question | Actual input tokens full → compact | Calls full → compact | Guarded wall seconds full → compact |
| -------------- | ---------------------------------: | -------------------: | ----------------------------------: |
| Direct package | 169,419 → 173,274 | 2 → 2 | 33.854 → 31.448 |
| Broad metadata | 126,522 → 166,082 | 1 → 2 | 28.434 → 34.989 |
| Invalid offset | 167,283 → 140,530 | 1 → 1 | 32.983 → 29.641 |
These are one pair per question on the baseline runtime's explicit full/compact
variants, not repeated model-performance cohorts. Compact used an extra call
for the broad question. **Actual Codex token, latency or task-quality gains are
not established.** Recorded client usage includes cached input; it does not
identify the exact rendered catalog contribution. The change addresses the
default declaration cost for clients carrying the complete catalog; it does not
promise identical savings for clients performing tool search or other schema
conversion. Explicit full remains useful when the additional guidance avoids
extra calls.
All analysis/model runs use one CPU, 4 GiB family hard / 3 GiB early memory
limits, no swap and 192 tasks. CPU and memory.peak are kernel cgroup counters
observed at payload exit, including the controller and charged page cache.
All accepted tasks have authenticated payload-exit observations, successful
exit and cleanup, and zero observed high/max/OOM/PID-limit events. Final
whole-scope counters are unavailable after the transient scope disappears.
The first SDK harness attempt used an incorrect retained-source argument and
was correctly refused. It is retained as a failed attempt and excluded from
accepted comparisons; the harness was corrected before the repeated cohorts.
## Reproduction
Generate an oracle with Python 3 and the pinned extracted binary:
```bash
python3 experiments/analysis-efficiency/mcp-input-catalog-oracle.py /absolute/ripgrep /absolute/oracle.json
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
node experiments/analysis-efficiency/mcp-input-catalog.mjs \
--runtime /absolute/runtime --artifact /absolute/ripgrep \
--oracle /absolute/oracle.json --output /absolute/new-output \
> /absolute/new-output.log 2>&1
# In an analysis environment with tiktoken==0.12.0 installed:
python3 experiments/analysis-efficiency/mcp-input-catalog-costs.py \
--output /absolute/costs.json /absolute/new-output
```
Repeat against baseline/candidate runtimes with the setting omitted. Explicit
`--profile full` and `--profile compact` verify caller overrides. The existing
`agent-trial.py` runs the recorded model question files and explicit provider
settings; authentication/profile files and raw binaries are never committed.
Use [the cost script](mcp-input-catalog-costs.py) with the pinned tokenizer to
account for complete SDK tasks and separately recorded model usage.
Keep the sibling log: accounting requires its authenticated resource receipt.
Its name appends `.log` to the entire case-directory basename, including dots.
A real `reproduce.sdk.1` Zellij task and its accounting completed successfully;
reprocessing every original record with this path fix leaves all costs equal.
This additional check is not part of the original performance cohorts.
## Latest-main integration and validation
Rebased onto `25e1a24a3ed3671a0148757555f18d16c29d936a`. A separate real Zellij
pair on that candidate, explicit full versus omitted/default, passed all facts,
full parent Evidence and negative checks. Catalog input estimates remain
200,425 versus 47,868 tokens; these integration runs do not replace or extend
the earlier repeated cohorts. Explicit full on the candidate also reproduced
the pre-change full catalog exactly.
The initial ten-file local run passed 88 tests, skipped one and failed two: a
live-identity assertion still expected the old default digest, and the complete
canonical-schema suite timed out at 30 seconds. The assertion now checks the
default compact identity. Both affected suites passed all 26 tests with serial
files and unchanged deadlines; the timeout's cause is not established. After
rebasing, 32 profile/identity/production-doctor tests passed. Static checks
passed on both bases. Documentation checks caught a new README link absent from
translated link inventories (including a duplicate-link attempt); the
introduction now names the existing guide without changing the link inventory.
Final documentation and CI results qualify the PR.
## Review follow-up: ownership of catalog identity
The former `CATALOG_IDENTITY` meant the full advertisement, while two consumers
treated it as the default live identity. The owner now exports explicit
`FULL_CATALOG_IDENTITY` and `DEFAULT_CATALOG_IDENTITY`; the ambiguous export is
removed. Canonical build/packaging consumers use full identity, default
diagnostics use default identity, and custom/explicit profiles keep their own
identity. Readiness captures and validates `binary_session` from the actual
server rather than assuming a profile. Claude's verifier derives its expected
identity using the shared input/output profile parsers.
Nine related test files passed 106 tests with two skips. Static and
documentation gates passed again. Actual readiness tasks passed with default,
explicit full, and compact input/output-off settings. The final real Zellij SDK
task reproduced the earlier complete parent and advertised catalog exactly.
Real Claude Code 2.1.296 with the loopback fixture discovered all 213 schemas
and returned `binary_session` for both profiles. It persisted the 55,471-byte
result, preventing the verifier's existing inline-only recognizer from
continuing. The untouched baseline reproduces the same failure (55,473-byte
result) after preparing its matching packaged skill. This is not a passing
end-to-end Claude analysis claim; the unrelated recognizer is left unchanged.
The actual persisted native responses were independently parsed with the named
production output contract: both selected catalog identities match, and the
completed readiness reports record the same digests. Receipts retain the
baseline/candidate failures, cleanup and this narrower successful check.
First-head CI found three remaining tests that assumed a full default
advertisement. The ZIP retained-inventory, browser-endpoint and literal-seed
cases now exercise full and compact: full retains its strict declaration and
guidance assertions; both keep canonical refusal and valid workflow checks.
Four affected files passed all 27 tests serially on main
`6d356b84344659085f2b912857b5e27fe0160c9b`; static/documentation gates passed.
An earlier parallel-file attempt was stopped at the 3 GiB early boundary before
OOM and is incomplete. Serial execution stayed below the same boundary without
raising the 4 GiB family limit. No performance cohort includes these test runs.
Additional CI cases now select full presentation when testing nested schemas
or literal advertised examples. Default source-comparison and JavaScript-fact
workflows still exercise canonical invalid-input refusal. After regenerating
the ignored catalog metadata, four files passed 31 tests; the full-catalog
ordering case hit its 20-second child deadline. That case passed alone in a
fresh process with the same deadline (14.85-second test duration). All five
files have passing checks; the combined run did not pass and its timeout cause
is not established. Family peaks were 2.43 GiB for the combined run and 1.0 GiB
for the standalone run, within unchanged limits. These validation attempts are
separate from the performance cohorts.
The next CI portable aggregate passed 15,703 tests and skipped 29, with one
remaining Godot inventory advertisement assertion failing. That workflow now
runs under both full and compact, preserving canonical invalid-field refusal
and retained facts after source removal. Its three local tests passed within
the same resource limits, including encrypted-directory unknown coverage.
A further review found that CLI status relied on an implicit default catalog,
so explicit full input advertised a different identity through CLI and MCP.
CLI status now derives input/output presentation from parsed configuration;
configuration retains the output profile too. Runtime identity requires an
explicit catalog from each caller. Two suites passed 25 tests, including real
production stdio, `capabilities` and `providers` comparisons for default, full,
compact and both output-off combinations. This target-free verification is
separate from real artifact measurements and does not qualify native engines.
Rebased again onto `8e3bf56e1` after upstream browser Evidence schema changes.
A separate completed real Zellij SDK task preserved its complete parent and
every catalog field except output schemas; whole-task estimate remains 51,721
tokens. Fourteen production CLI/MCP profile/lifecycle tests passed. Two
absolute-path tests also passed: full advertisement conformance explicitly
selects full, while actual relative-path refusals keep the default compact
presentation. This fixes the remaining nested-path assertion found by CI.
These integration checks do not replace the original performance cohorts.
A later unchanged-head CI retry passed the earlier Ghidra Java-cleanup failure
but exposed a full-schema assumption in call-direction coverage. That workflow
now tests full and compact with actual canonical invalid-input refusals. A
fresh-process audit also found a browser scenario cross-field assertion; it
keeps default valid calls and typed refusals while checking full producer
constraints separately. Five isolated suites passed 14 tests across call paths,
ADB start, browser scenarios, source selection and managed analysis. The first
browser attempt failed and remains recorded; no performance cohort changed.
@@ -0,0 +1,176 @@
// Compare complete public MCP tasks with caller-selected input advertisements.
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { parseArgs } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { Ajv2020 } from "ajv/dist/2020.js";
const { values } = parseArgs({
options: {
runtime: { type: "string" },
artifact: { type: "string" },
oracle: { type: "string" },
output: { type: "string" },
profile: { type: "string", default: "default" },
},
});
for (const key of ["runtime", "artifact", "oracle", "output"])
assert.ok(
values[key] && resolve(values[key]) === values[key],
`${key} must be absolute`,
);
assert.ok(["default", "full", "compact"].includes(values.profile));
await mkdir(values.output, { recursive: false });
const hash = (value) => createHash("sha256").update(value).digest("hex");
const oracle = JSON.parse(await readFile(values.oracle, "utf8"));
const artifactSha = hash(await readFile(values.artifact));
assert.equal(artifactSha, oracle.artifact_sha256);
const environment = {
PATH: process.env.PATH,
HOME: values.output,
REA_LOG_LEVEL: "silent",
UV_THREADPOOL_SIZE: "1",
NODE_OPTIONS: "--max-old-space-size=2048",
NODE_DISABLE_COMPILE_CACHE: "1",
...(values.profile === "default"
? {}
: { REA_MCP_INPUT_SCHEMA_PROFILE: values.profile }),
};
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
cwd: values.output,
env: environment,
stderr: "pipe",
});
const client = new Client({ name: "rea-catalog-task", version: "1" });
const report = {
profile: values.profile,
artifact_sha256: artifactSha,
calls: [],
checks: [],
status: "incomplete",
cleanup_verified: false,
};
const transcript = [];
let stderr = "";
transport.stderr?.on("data", (value) => {
stderr += value;
});
const began = performance.now();
try {
await client.connect(transport);
const catalog = await client.listTools();
assert.equal(catalog.nextCursor, undefined);
const definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));
assert.equal(definitions.size, catalog.tools.length);
const ajv = new Ajv2020({ strict: false, validateFormats: false });
for (const tool of catalog.tools)
assert.ok(
ajv.validateSchema(tool.inputSchema),
`Invalid schema: ${tool.name}`,
);
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
report.tool_names = [...definitions.keys()];
const call = async (name, args) => {
const definition = definitions.get(name);
assert.ok(definition, `Missing named tool ${name}`);
const start = performance.now();
const reply = await client.callTool(
{ name, arguments: args },
{ toolDefinition: definition },
);
report.calls.push({
name,
elapsed_ms: performance.now() - start,
is_error: reply.isError === true,
});
transcript.push({ name, arguments: args, reply });
return reply;
};
const input = {
path: values.artifact,
view: {
collection: "packages",
names: ["serde"],
fields: ["name", "version", "kind"],
},
};
const first = await call("inspect_rust_binary", input);
assert.notEqual(first.isError, true);
const selected = first.structuredContent;
assert.ok(selected?.normalized_result?.page);
assert.deepEqual(selected.normalized_result.page.rows, oracle.serde_rows);
assert.equal(selected.subject.digest.sha256, artifactSha);
const parentId = selected.normalized_result.parent_evidence_id;
for (let index = 0; index < 2; index++) {
const reply = await call("inspect_analysis_view", {
source: { kind: "retained-evidence", evidence_id: parentId },
view: {
kind: "rust",
collection: "packages",
names: ["serde"],
fields: ["name", "version", "kind"],
},
});
assert.notEqual(reply.isError, true);
assert.deepEqual(
reply.structuredContent.normalized_result.page.rows,
oracle.serde_rows,
);
assert.equal(
reply.structuredContent.normalized_result.parent_evidence_id,
parentId,
);
}
const malformed = await call("inspect_rust_binary", {
path: values.artifact,
view: { collection: "packages", offset: -1 },
});
assert.equal(malformed.isError, true);
report.checks.push(
"Independent package rows and digest; repeated retained identity; canonical negative-offset refusal",
);
const exported = await call("export_evidence_bundle", {
path: join(values.output, "evidence.json"),
});
assert.notEqual(exported.isError, true);
const bundle = JSON.parse(
await readFile(join(values.output, "evidence.json"), "utf8"),
);
const parent = bundle.records.find(
(record) => record.evidence_id === parentId,
);
assert.ok(parent, "Complete original Evidence is retained");
assert.equal(parent.subject.digest.sha256, artifactSha);
assert.deepEqual(
parent.normalized_result.auditable_metadata.packages,
oracle.packages,
);
assert.deepEqual(
parent.normalized_result.auditable_metadata.dependency_edges,
oracle.edges,
);
report.parent_evidence_id = parentId;
report.parent_semantic_sha256 = hash(JSON.stringify(parent));
report.answer = oracle.serde_rows;
report.status = "passed";
} finally {
await client.close();
await transport.close();
report.cleanup_verified = transport.pid === null;
assert.ok(report.cleanup_verified);
report.elapsed_ms = performance.now() - began;
await writeFile(
join(values.output, "report.json"),
JSON.stringify(report, null, 2) + "\n",
);
await writeFile(
join(values.output, "transcript.json"),
JSON.stringify(transcript),
);
await writeFile(join(values.output, "server.log"), stderr);
}
+2 -2
View File
@@ -17,7 +17,7 @@ const current = (
await readFile(join(sourceRoot, "SKILL.md"), "utf8")
).replaceAll("\r\n", "\n");
const cacheBust = String(Date.now());
const { CATALOG_IDENTITY } = await import(
const { FULL_CATALOG_IDENTITY } = await import(
`${pathToFileURL(join(root, "dist/catalogIdentity.js")).href}?${cacheBust}`
);
const { PRODUCT_IDENTITY } = await import(
@@ -36,7 +36,7 @@ const pinnedSpecifier = PRODUCT_IDENTITY.registrationPackageSpecifier;
const source = current
.replace(
versionLine,
`$&\n tool_count: ${String(CATALOG_IDENTITY.counts.mcp_tools)}`,
`$&\n tool_count: ${String(FULL_CATALOG_IDENTITY.counts.mcp_tools)}`,
)
.replaceAll(latestSpecifier, pinnedSpecifier);
const paths = await filePaths(sourceRoot);
+2 -1
View File
@@ -143,7 +143,8 @@ export const toolFamilyCatalog = (sources) => {
);
if (
familyTotal !== sources.toolContracts.TOOL_CONTRACTS.length ||
familyTotal !== sources.catalogIdentity.CATALOG_IDENTITY.counts.mcp_tools
familyTotal !==
sources.catalogIdentity.FULL_CATALOG_IDENTITY.counts.mcp_tools
)
throw new Error("Tool family inventory does not match TOOL_CONTRACTS");
return { total: familyTotal, families };
+1 -1
View File
@@ -43,7 +43,7 @@ export const createProductCatalog = async (root) => {
aliases: cli.aliases,
},
runtime_catalog: {
counts: sources.catalogIdentity.CATALOG_IDENTITY.counts,
counts: sources.catalogIdentity.FULL_CATALOG_IDENTITY.counts,
digests: {
// Full runtime schema digests belong to server identity, not this facts projection.
providers_sha256: digest(providers),
+2 -2
View File
@@ -5,7 +5,7 @@ import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { promisify } from "node:util";
import { CATALOG_IDENTITY } from "../../dist/catalogIdentity.js";
import { FULL_CATALOG_IDENTITY } from "../../dist/catalogIdentity.js";
import { TOOL_CONTRACTS } from "../../dist/contracts/toolContracts.js";
import {
requireMcpLifecycleResult,
@@ -104,7 +104,7 @@ export async function verifyInstalledCanaryPackage(
);
return {
mcpTools: toolNames.length,
canonicalMcpTools: CATALOG_IDENTITY.counts.mcp_tools,
canonicalMcpTools: FULL_CATALOG_IDENTITY.counts.mcp_tools,
};
} catch (cause) {
throw new Error(`Installed package MCP verification failed: ${stderr}`, {
+19 -3
View File
@@ -5,7 +5,12 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import Ajv2020 from "ajv/dist/2020.js";
import { CATALOG_IDENTITY } from "../dist/catalogIdentity.js";
import { createCatalogIdentity } from "../dist/catalogIdentity.js";
import {
compactPresentationForMcpInputSchemaProfile,
parseMcpInputSchemaProfile,
} from "../dist/config/mcpInputSchemaProfile.js";
import { parseMcpOutputSchemaProfile } from "../dist/config/mcpOutputSchemaProfile.js";
import {
TOOL_CONTRACTS,
toolContract,
@@ -166,10 +171,21 @@ const fixture = await createAnthropicModelFixture({
...parsedStatus,
result: expandMcpSessionStatus(parsedStatus.result),
};
const inputProfile = parseMcpInputSchemaProfile(
environment.REA_MCP_INPUT_SCHEMA_PROFILE,
);
const outputProfile = parseMcpOutputSchemaProfile(
environment.REA_MCP_OUTPUT_SCHEMA_PROFILE,
);
assert.ok(inputProfile.ok);
assert.ok(outputProfile.ok);
assert.deepEqual(
status.result.server_identity.catalog,
CATALOG_IDENTITY,
"binary_session must return the exact compact catalog counts and digests",
createCatalogIdentity(
compactPresentationForMcpInputSchemaProfile(inputProfile.value),
outputProfile.value ?? "full",
),
"binary_session must return the selected input/output profile's catalog identity",
);
assert.equal(
Object.hasOwn(status.result.server_identity.catalog, "tools"),
+2 -2
View File
@@ -1,6 +1,6 @@
import { mkdir } from "node:fs/promises";
import { join } from "node:path";
import { CATALOG_IDENTITY } from "../dist/catalogIdentity.js";
import { FULL_CATALOG_IDENTITY } from "../dist/catalogIdentity.js";
import { exec, execNpm } from "./lib/verify-package-core.mjs";
import { parseTarListing } from "./lib/verify-package-listing.mjs";
@@ -52,7 +52,7 @@ export async function verifyPackagePack({ root, workspace }) {
).stdout;
if (
!skill.includes(
` tool_count: ${String(CATALOG_IDENTITY.counts.mcp_tools)}\n`,
` tool_count: ${String(FULL_CATALOG_IDENTITY.counts.mcp_tools)}\n`,
) ||
skill.includes(" catalog_digest:")
)
+15 -2
View File
@@ -7,7 +7,7 @@ import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { CATALOG_IDENTITY } from "../dist/catalogIdentity.js";
import { toolContract } from "../dist/contracts/toolContracts.js";
import { RECONSTRUCTION_READINESS_EXAMPLE } from "../dist/contracts/reconstructionReadinessExample.js";
import { parseEvidence } from "../dist/domain/evidence.js";
import { createEvidenceBundle } from "../dist/domain/evidenceBundle.js";
@@ -52,6 +52,18 @@ const client = new Client({
try {
await client.connect(transport);
const catalog = await client.listTools();
const sessionTool = catalog.tools.find(
(tool) => tool.name === "binary_session",
);
assert.ok(sessionTool, "The live server must advertise binary_session");
const sessionReply = await client.callTool(
{ name: "binary_session", arguments: {} },
{ toolDefinition: sessionTool },
);
const sessionStatus = toolContract("binary_session").outputSchema.parse(
sessionReply.structuredContent,
).result;
const javascriptMcp = await analyzeMcp(client, javascriptPath);
const electronMcp = await analyzeMcp(client, electronPath);
assert.equal(
@@ -68,7 +80,8 @@ try {
const input = structuredClone(RECONSTRUCTION_READINESS_EXAMPLE);
input.identity.cli_version = PACKAGE_METADATA.version;
input.identity.server_version = PACKAGE_METADATA.version;
input.identity.catalog_digest = CATALOG_IDENTITY.digests.combined_sha256;
input.identity.catalog_digest =
sessionStatus.server_identity.catalog.digests.combined_sha256;
const records = [
...input.evidence_bundle.records,
nativeEvidence,
+8
View File
@@ -8,6 +8,8 @@ import { providerCapabilitySet } from "../contracts/toolOutputSchemaPrimitives.j
import { providerCapabilitySets } from "./binary/BinarySessionStatus.js";
import { sqliteDatabaseCapability } from "./sqlite/SqliteDatabaseCapability.js";
import { CURRENT_ENTRYPOINT_PATH } from "../process/CurrentEntrypointPath.js";
import { createCatalogIdentity } from "../catalogIdentity.js";
import { compactPresentationForMcpInputSchemaProfile } from "../config/mcpInputSchemaProfile.js";
/** Read target-free provider status and operation availability. */
export const runSessionStatus = async (
@@ -33,6 +35,12 @@ export const runSessionStatus = async (
server_identity: createServerIdentity({
startedAt: new Date().toISOString(),
commandPath: CURRENT_ENTRYPOINT_PATH,
catalog: createCatalogIdentity(
compactPresentationForMcpInputSchemaProfile(
config.value.mcpInputSchemaProfile,
),
config.value.mcpOutputSchemaProfile ?? "full",
),
}),
};
} finally {
+2 -2
View File
@@ -1,4 +1,4 @@
import { CATALOG_IDENTITY } from "../catalogIdentity.js";
import { DEFAULT_CATALOG_IDENTITY } from "../catalogIdentity.js";
import { PRODUCT_IDENTITY } from "../identity.js";
import { ok } from "../domain/result.js";
import type { DoctorHost } from "./Doctor.js";
@@ -36,7 +36,7 @@ export const createDoctorHostFixture = (
installedSkillIdentity: () =>
Promise.resolve({
version: PRODUCT_IDENTITY.skillVersion,
toolCount: CATALOG_IDENTITY.counts.mcp_tools,
toolCount: DEFAULT_CATALOG_IDENTITY.counts.mcp_tools,
canonical: true,
}),
...overrides,
+6 -5
View File
@@ -45,7 +45,7 @@ import {
type InstalledSkillIdentity,
type SkillPathIssue,
} from "./SetupSkill.js";
import { CATALOG_IDENTITY } from "../catalogIdentity.js";
import { DEFAULT_CATALOG_IDENTITY } from "../catalogIdentity.js";
import { PRODUCT_IDENTITY, SDK_IDENTITY } from "../identity.js";
import {
readClientRegistrationStatuses,
@@ -178,7 +178,8 @@ interface DoctorIdentity {
readonly cli_package_version: string;
readonly expected_skill_version: string;
readonly sdk: typeof SDK_IDENTITY;
readonly catalog: typeof CATALOG_IDENTITY;
/** Package default advertisement; the active server reports its selected profile. */
readonly catalog: typeof DEFAULT_CATALOG_IDENTITY;
readonly live_server: {
readonly state: "unknown";
readonly remediation: string;
@@ -390,8 +391,8 @@ const collectDoctorIdentity = async (
expected_skill_version: PRODUCT_IDENTITY.skillVersion,
sdk: SDK_IDENTITY,
catalog: {
counts: CATALOG_IDENTITY.counts,
digests: CATALOG_IDENTITY.digests,
counts: DEFAULT_CATALOG_IDENTITY.counts,
digests: DEFAULT_CATALOG_IDENTITY.digests,
},
live_server: {
state: "unknown",
@@ -422,7 +423,7 @@ const skillIdentityAligned = (
identity: InstalledSkillIdentity | undefined,
): boolean =>
identity?.version === PRODUCT_IDENTITY.skillVersion &&
identity.toolCount === CATALOG_IDENTITY.counts.mcp_tools &&
identity.toolCount === DEFAULT_CATALOG_IDENTITY.counts.mcp_tools &&
identity.canonical;
const skillIdentityState = (
+2 -2
View File
@@ -18,7 +18,7 @@ import type {
} from "./Doctor.js";
import type { LinuxDistribution } from "./LinuxHopper.js";
import { PRODUCT_IDENTITY, SDK_IDENTITY } from "../identity.js";
import { CATALOG_IDENTITY } from "../catalogIdentity.js";
import { DEFAULT_CATALOG_IDENTITY } from "../catalogIdentity.js";
import type { ClientRegistrationStatus } from "./ClientRegistrationStatus.js";
import { setupRegistrationCommand } from "./SetupHost.js";
import { CURRENT_ENTRYPOINT_PATH } from "../process/CurrentEntrypointPath.js";
@@ -228,7 +228,7 @@ export class FakeSetupHost implements SetupHost {
cli_package_version: PRODUCT_IDENTITY.packageVersion,
expected_skill_version: PRODUCT_IDENTITY.skillVersion,
sdk: SDK_IDENTITY,
catalog: CATALOG_IDENTITY,
catalog: DEFAULT_CATALOG_IDENTITY,
live_server: {
state: "unknown",
remediation: "Inspect the running server.",
+10 -4
View File
@@ -19,6 +19,7 @@ import {
import type { McpOutputSchemaProfile } from "./config/mcpOutputSchemaProfile.js";
import {
COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
compactPresentationForMcpInputSchemaProfile,
type McpInputSchemaProfile,
} from "./config/mcpInputSchemaProfile.js";
import { RECORDED_CATALOG_DIGEST_PROFILES } from "./recordedCatalogDigests.js";
@@ -111,7 +112,7 @@ const digestCatalogs = (
/**
* Catalog digests over the given tool contracts. Exported so tests can prove
* which contract fields the identity covers; production uses
* {@link CATALOG_IDENTITY}.
* {@link FULL_CATALOG_IDENTITY}.
*/
export const createCatalogDigests = (
contracts: readonly ToolContract[] = TOOL_CONTRACTS,
@@ -191,7 +192,12 @@ export const createCatalogIdentity = (
};
/**
* Stable, schema-sensitive identity; schema digests are recorded at build time
* or computed on first use.
* Full input/output advertisement identity. Digests are recorded at build time
* or computed on first use; this is independent of the configured default.
*/
export const CATALOG_IDENTITY = createCatalogIdentity();
export const FULL_CATALOG_IDENTITY = createCatalogIdentity();
/** Default MCP advertisement; explicit full/custom selections have their own identity. */
export const DEFAULT_CATALOG_IDENTITY = createCatalogIdentity(
compactPresentationForMcpInputSchemaProfile(undefined),
);
+5 -5
View File
@@ -12,16 +12,16 @@ export const COMPACT_INPUT_SCHEMA_BUDGET_BYTES = 13 * 1024;
export const mcpInputSchemaProfileSchema = z.enum(["full", "compact"]);
/** Select the exact compact presentation budget shared by registration and identity. */
/** Select the default compact advertisement, preserving an explicit full choice. */
export const compactPresentationForMcpInputSchemaProfile = (
profile: "full" | "compact" | undefined,
): { readonly budgetBytes: number } | undefined =>
profile === "compact"
? { budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES }
: undefined;
profile === "full"
? undefined
: { budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES };
const profileConstraint =
'REA_MCP_INPUT_SCHEMA_PROFILE must be "full" (default) or "compact". The compact profile advertises bounded input schemas with fewer nested annotations for providers with schema compatibility constraints.';
'REA_MCP_INPUT_SCHEMA_PROFILE must be "compact" (default) or "full". The compact profile advertises bounded input schemas with fewer nested annotations; full preserves complete advertised input guidance.';
export type McpInputSchemaProfile = z.infer<typeof mcpInputSchemaProfileSchema>;
+3
View File
@@ -77,6 +77,9 @@ export const parseConfig = (
...(env.REA_MCP_INPUT_SCHEMA_PROFILE === undefined
? {}
: { mcpInputSchemaProfile: env.REA_MCP_INPUT_SCHEMA_PROFILE }),
...(env.REA_MCP_OUTPUT_SCHEMA_PROFILE === undefined
? {}
: { mcpOutputSchemaProfile: env.REA_MCP_OUTPUT_SCHEMA_PROFILE }),
analysisProvider: env.REA_ANALYSIS_PROVIDER,
...(env.REA_IDA_MCP_CONFIG === undefined
? {}
+2
View File
@@ -4,10 +4,12 @@ import type { AnalysisProviderSelector } from "../contracts/providerSelection.js
import type { GhidraLanguageOverride } from "./ghidraLanguageOverride.js";
import type { McpInputSchemaProfile } from "./mcpInputSchemaProfile.js";
import type { McpOutputSchemaProfile } from "./mcpOutputSchemaProfile.js";
export interface AppConfig {
readonly mcpMaxResponseBytes?: number;
readonly mcpInputSchemaProfile?: McpInputSchemaProfile;
readonly mcpOutputSchemaProfile?: McpOutputSchemaProfile;
readonly analysisProvider: AnalysisProviderSelector;
readonly idaMcpConfigPath?: string;
readonly ghidraInstallDir: string | undefined;
+5 -2
View File
@@ -26,7 +26,10 @@ import {
presentAdvertisedInputSchema,
type CompactInputSchemaPresentation,
} from "../contracts/compactInputSchema.js";
import { CATALOG_IDENTITY, createCatalogIdentity } from "../catalogIdentity.js";
import {
FULL_CATALOG_IDENTITY,
createCatalogIdentity,
} from "../catalogIdentity.js";
import { transformAdvertisedInputJsonSchema } from "../contracts/toolSchemaMetadata.js";
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
import {
@@ -165,7 +168,7 @@ export class EvidenceMcpServer extends RequestTrackingMcpServer {
: { budgetBytes: compactPresentation.budgetBytes };
this.catalogIdentity =
this.compactPresentation === undefined && !this.omitOutputSchemas
? CATALOG_IDENTITY
? FULL_CATALOG_IDENTITY
: createCatalogIdentity(
this.compactPresentation,
this.omitOutputSchemas ? "off" : "full",
+3 -6
View File
@@ -1,10 +1,7 @@
import { dirname, basename, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import {
CATALOG_IDENTITY,
type createCatalogIdentity,
} from "./catalogIdentity.js";
import type { createCatalogIdentity } from "./catalogIdentity.js";
import { PRODUCT_IDENTITY, SDK_IDENTITY } from "./identity.js";
/** Optional observations supplied by a client comparing its expected server. */
@@ -34,9 +31,9 @@ export const createServerIdentity = (input: {
readonly expected?: ExpectedServerIdentity;
readonly client?: ConnectedClientIdentity;
readonly protocolVersion?: string | undefined;
readonly catalog?: ReturnType<typeof createCatalogIdentity>;
readonly catalog: ReturnType<typeof createCatalogIdentity>;
}) => {
const catalog = input.catalog ?? CATALOG_IDENTITY;
const catalog = input.catalog;
const serverPath = input.commandPath || undefined;
const reasons: string[] = [];
if (
@@ -5,7 +5,7 @@ import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { describe, expect, it } from "vitest";
import { z } from "zod";
import { CATALOG_IDENTITY } from "../../../src/catalogIdentity.js";
import { FULL_CATALOG_IDENTITY } from "../../../src/catalogIdentity.js";
import { TOOL_CONTRACTS } from "../../../src/contracts/toolContracts.js";
import {
evidenceResultOf,
@@ -51,7 +51,7 @@ const expectAvailableToolInventory = async (
expect(
availability.length,
"compiled tool inventory is stale; run npm run build:cached",
).toBe(CATALOG_IDENTITY.counts.mcp_tools);
).toBe(FULL_CATALOG_IDENTITY.counts.mcp_tools);
expect(new Set(listed.tools.map(({ name }) => name))).toEqual(
new Set(TOOL_CONTRACTS.map(({ name }) => name)),
);
@@ -231,7 +231,7 @@ const createBuildCacheFixture = async (): Promise<string> => {
await writeFile("resource-controls.json", JSON.stringify(Object.fromEntries(["GOMAXPROCS", "GOMEMLIMIT", "UV_THREADPOOL_SIZE", "REA_TEST_UNLISTED_ENV"].map((name) => [name, process.env[name] ?? null]))));
await mkdir("dist", { recursive: true });
await writeFile("dist/cache-fixture.js", "built");
await writeFile("dist/catalogIdentity.js", "export const CATALOG_IDENTITY = { counts: { mcp_tools: 1 } };\\n");
await writeFile("dist/catalogIdentity.js", "export const FULL_CATALOG_IDENTITY = { counts: { mcp_tools: 1 } };\\n");
await writeFile("dist/identity.js", ${JSON.stringify(`export const PRODUCT_IDENTITY = ${JSON.stringify({ packageSpecifier: `${packageJson.name}@latest`, registrationPackageSpecifier: `${packageJson.name}@${packageJson.version}` })};\n`)});
`,
);
@@ -38,7 +38,7 @@ async function fixture(eol = "\n") {
const identity = async (count: number, digest: string) =>
writeFile(
join(root, "dist/catalogIdentity.js"),
`export const CATALOG_IDENTITY = ${JSON.stringify({ counts: { mcp_tools: count }, digests: { combined_sha256: digest } })};\n`,
`export const FULL_CATALOG_IDENTITY = ${JSON.stringify({ counts: { mcp_tools: count }, digests: { combined_sha256: digest } })};\n`,
);
await identity(139, "a".repeat(64));
const generate = (...args: string[]) =>
@@ -14,7 +14,10 @@ interface PathInputCase {
readonly invalid: Readonly<Record<string, unknown>>;
}
const createMcp = async (firmwareCalls: string[]) => {
const createMcp = async (
firmwareCalls: string[],
profile?: "full" | "compact",
) => {
const session = createTestBinarySession(() => {
throw new Error("Invalid path input must not acquire a binary provider");
});
@@ -22,6 +25,7 @@ const createMcp = async (firmwareCalls: string[]) => {
{ kind: "session", session },
{
logger: silentLogger,
environment: { ...process.env, REA_MCP_INPUT_SCHEMA_PROFILE: profile },
firmwareAnalysis: {
execute: async (request) => {
firmwareCalls.push(request.operation);
@@ -174,9 +178,9 @@ const inputCases = (
];
};
it("advertises host-absolute path constraints after SDK schema conversion", async () => {
it("advertises full host-absolute path constraints after SDK schema conversion", async () => {
const firmwareCalls: string[] = [];
const { client, close } = await createMcp(firmwareCalls);
const { client, close } = await createMcp(firmwareCalls, "full");
try {
const listed = await client.listTools();
const ajv = new Ajv2020({ strict: false, validateFormats: false });
@@ -61,6 +61,8 @@ const connect = async (
const server = createServer(
{ kind: "session", session },
{
// These schema conformance cases exercise full nested constraints.
environment: { ...process.env, REA_MCP_INPUT_SCHEMA_PROFILE: "full" },
...(binaryLayout === undefined ? {} : { binaryLayout }),
...(webNetworkCapture === undefined ? {} : { webNetworkCapture }),
},
@@ -26,6 +26,8 @@ import {
type ApplicationMcpHarness,
} from "../../fixtures/applicationMcpHarness.js";
import { createHistoricalSourceGraph } from "../../../src/domain/referenceSourceGraph.js";
import { advertisedInputJsonSchema } from "../../../src/contracts/toolSchemaMetadata.js";
import { toolContract } from "../../../src/contracts/toolContracts.js";
async function runUnicodeSourceMapComparison(
harness: ApplicationMcpHarness,
@@ -257,7 +259,12 @@ async function runInlineWorkflowScenarios(
exclusions: [{ path: "vendor/removed.ts", reason: "project-ignored" }],
},
};
expect(validate(missingPattern)).toBe(false);
expect(
ajv.validate(
advertisedInputJsonSchema(toolContract("compare_source_to_bundle")),
missingPattern,
),
).toBe(false);
const rejectedExclusion = await harness.client.callTool({
name: "compare_source_to_bundle",
arguments: missingPattern,
@@ -26,7 +26,11 @@ it("executes the advertised export presence example with matching analyzed Evide
);
if (example === undefined)
throw new Error("Missing advertised presence example");
const { client, close } = await createApplicationMcpHarness();
const { client, close } = await createApplicationMcpHarness({
serverOptions: {
environment: { ...process.env, REA_MCP_INPUT_SCHEMA_PROFILE: "full" },
},
});
onTestFinished(close);
const listing = await client.listTools();
const advertised = listing.tools.find(
@@ -188,7 +188,11 @@ it("preserves keyed archive identities and rejects inconsistent inventory carrie
}
});
const createInventoryViewReader = async (client: Client, parent: Evidence) => {
const createInventoryViewReader = async (
client: Client,
parent: Evidence,
profile: "full" | "compact",
) => {
const advertised = (await client.listTools()).tools.find(
(tool) => tool.name === "inspect_analysis_view",
);
@@ -199,12 +203,18 @@ const createInventoryViewReader = async (client: Client, parent: Evidence) => {
const outputSchema: Record<string, unknown> = advertised.outputSchema;
const validateInput = ajv.compile(inputSchema);
const validateOutput = ajv.compile(outputSchema);
expect(
validateInput({
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
view: { kind: "inventory", collection: "nodes", logical_paths: [] },
}),
).toBe(false);
const malformed = {
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
view: { kind: "inventory", collection: "nodes", logical_paths: [] },
};
if (profile === "full") expect(validateInput(malformed)).toBe(false);
const rejected = await client.callTool({
name: "inspect_analysis_view",
arguments: malformed,
});
expect(parseMcpToolError(rejected)).toMatchObject({
error: { code: "invalid_request", category: "invalid_input" },
});
const readView = async (
view: Record<string, unknown>,
source = {
@@ -322,147 +332,155 @@ const assertSummaryDelivery = async (
);
};
it("keeps complete ZIP output and reads retained inventory views after the source is removed", async () => {
const directory = await createTestTempDirectory("rea-artifact-inline-");
const archive = join(directory, "many.zip");
const writer = new ZipWriter(new Uint8ArrayWriter());
const fileCount = 520;
for (let index = 0; index < fileCount; index += 1)
await writer.add(
`files/${String(index)}.txt`,
new TextReader(`file-${String(index)}`),
);
await writeFile(archive, await writer.close());
it.each(["full", "compact"] as const)(
"keeps complete ZIP output and reads retained inventory views after the source is removed (%s)",
async (profile) => {
const directory = await createTestTempDirectory("rea-artifact-inline-");
const archive = join(directory, "many.zip");
const writer = new ZipWriter(new Uint8ArrayWriter());
const fileCount = 520;
for (let index = 0; index < fileCount; index += 1)
await writer.add(
`files/${String(index)}.txt`,
new TextReader(`file-${String(index)}`),
);
await writeFile(archive, await writer.close());
const session = createTestBinarySession(new ArtifactProvider(process.env));
const server = createServer({ kind: "session", session });
const client = new Client({ name: "artifact-inline-test", version: "1" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
try {
await server.connect(serverTransport);
await client.connect(clientTransport);
const opened = await client.callTool({
name: "open_binary",
arguments: { path: archive },
});
expect(opened.isError).not.toBe(true);
const result = await client.callTool({
name: "inspect_artifact",
arguments: {},
});
expect(result.isError, JSON.stringify(result.structuredContent)).not.toBe(
true,
);
const parent = parseEvidence(result.structuredContent);
await assertSummaryDelivery(client, session, parent);
const parsedInspection = artifactInspectionResultSchema.parse(
parent.normalized_result,
);
const inventoryEvidence = parsedInspection.substeps[0]?.evidence;
if (inventoryEvidence === undefined)
throw new Error("Missing inventory Evidence");
const graph = artifactInventoryResultSchema.parse(
inventoryEvidence.normalized_result,
);
expect(graph).toMatchObject({
integrity_contradictions: [],
coverage: { scope: "container-entries", status: "complete" },
});
expect(graph.occurrences).toHaveLength(fileCount + 1);
// Follow-ups must use retained source facts, not reopen this archive.
await unlink(archive);
const readView = await createInventoryViewReader(client, parent);
const summary = await readView({ kind: "summary" });
expect(summary.result).toMatchObject({
kind: "summary",
parent_evidence_id: parent.evidence_id,
artifact: { path: archive, sha256: graph.manifest.root_sha256 },
summary: {
manifest: graph.manifest,
source_complete: true,
available_counts: { occurrences: fileCount + 1 },
observed_entry_kind_counts: { file: fileCount + 1 },
inventory_evidence_ids: [inventoryEvidence.evidence_id],
},
});
expect(summary.evidence.evidence_links).toEqual([parent.evidence_id]);
const paths = ["files/0.txt", "files/519.txt", "missing.txt"];
const selected = await readView({
kind: "inventory",
collection: "occurrences",
logical_paths: paths,
offset: 0,
limit: 1,
});
expect(selected.result).toMatchObject({
kind: "inventory",
source_complete: true,
items: graph.occurrences
.filter((row) => paths.includes(row.logical_path))
.slice(0, 1),
coverage: { examined: 1, total: 2, next_offset: 1, exhausted: false },
});
const next = await readView({
kind: "inventory",
collection: "occurrences",
logical_paths: paths,
offset: 1,
limit: 1,
});
expect(next.result).toMatchObject({
coverage: { examined: 1, next_offset: null, exhausted: true },
});
const none = await readView({
kind: "inventory",
collection: "occurrences",
logical_paths: [],
});
expect(none.result).toMatchObject({
items: [],
coverage: { total: 0, exhausted: true },
});
const partial = createEvidence(
{ path: archive, sha256: graph.manifest.root_sha256, format: "zip" },
inventoryEvidence.provider,
const session = createTestBinarySession(new ArtifactProvider(process.env));
const server = createServer(
{ kind: "session", session },
{
operation: "inventory_artifact",
parameters: {},
result: jsonValueSchema.parse({
...graph,
occurrences: graph.occurrences.slice(0, 1),
}),
environment: { ...process.env, REA_MCP_INPUT_SCHEMA_PROFILE: profile },
},
);
const partialReply = await client.callTool({
name: "inspect_analysis_view",
arguments: {
source: { kind: "inline", evidence: partial },
view: {
kind: "inventory",
collection: "occurrences",
logical_paths: ["missing.txt"],
const client = new Client({ name: "artifact-inline-test", version: "1" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
try {
await server.connect(serverTransport);
await client.connect(clientTransport);
const opened = await client.callTool({
name: "open_binary",
arguments: { path: archive },
});
expect(opened.isError).not.toBe(true);
const result = await client.callTool({
name: "inspect_artifact",
arguments: {},
});
expect(result.isError, JSON.stringify(result.structuredContent)).not.toBe(
true,
);
const parent = parseEvidence(result.structuredContent);
await assertSummaryDelivery(client, session, parent);
const parsedInspection = artifactInspectionResultSchema.parse(
parent.normalized_result,
);
const inventoryEvidence = parsedInspection.substeps[0]?.evidence;
if (inventoryEvidence === undefined)
throw new Error("Missing inventory Evidence");
const graph = artifactInventoryResultSchema.parse(
inventoryEvidence.normalized_result,
);
expect(graph).toMatchObject({
integrity_contradictions: [],
coverage: { scope: "container-entries", status: "complete" },
});
expect(graph.occurrences).toHaveLength(fileCount + 1);
// Follow-ups must use retained source facts, not reopen this archive.
await unlink(archive);
const readView = await createInventoryViewReader(client, parent, profile);
const summary = await readView({ kind: "summary" });
expect(summary.result).toMatchObject({
kind: "summary",
parent_evidence_id: parent.evidence_id,
artifact: { path: archive, sha256: graph.manifest.root_sha256 },
summary: {
manifest: graph.manifest,
source_complete: true,
available_counts: { occurrences: fileCount + 1 },
observed_entry_kind_counts: { file: fileCount + 1 },
inventory_evidence_ids: [inventoryEvidence.evidence_id],
},
},
});
expect(
parseEvidence(partialReply.structuredContent).normalized_result,
).toMatchObject({
source_complete: false,
items: [],
unknowns: expect.arrayContaining([
expect.stringContaining("Missing matches do not establish absence"),
]),
});
});
expect(summary.evidence.evidence_links).toEqual([parent.evidence_id]);
const paths = ["files/0.txt", "files/519.txt", "missing.txt"];
const selected = await readView({
kind: "inventory",
collection: "occurrences",
logical_paths: paths,
offset: 0,
limit: 1,
});
expect(selected.result).toMatchObject({
kind: "inventory",
source_complete: true,
items: graph.occurrences
.filter((row) => paths.includes(row.logical_path))
.slice(0, 1),
coverage: { examined: 1, total: 2, next_offset: 1, exhausted: false },
});
const next = await readView({
kind: "inventory",
collection: "occurrences",
logical_paths: paths,
offset: 1,
limit: 1,
});
expect(next.result).toMatchObject({
coverage: { examined: 1, next_offset: null, exhausted: true },
});
const none = await readView({
kind: "inventory",
collection: "occurrences",
logical_paths: [],
});
expect(none.result).toMatchObject({
items: [],
coverage: { total: 0, exhausted: true },
});
await assertPortableRoundtrip(client, directory, parent, summary.result);
} finally {
await client.close();
await server.close();
await session.close();
}
});
const partial = createEvidence(
{ path: archive, sha256: graph.manifest.root_sha256, format: "zip" },
inventoryEvidence.provider,
{
operation: "inventory_artifact",
parameters: {},
result: jsonValueSchema.parse({
...graph,
occurrences: graph.occurrences.slice(0, 1),
}),
},
);
const partialReply = await client.callTool({
name: "inspect_analysis_view",
arguments: {
source: { kind: "inline", evidence: partial },
view: {
kind: "inventory",
collection: "occurrences",
logical_paths: ["missing.txt"],
},
},
});
expect(
parseEvidence(partialReply.structuredContent).normalized_result,
).toMatchObject({
source_complete: false,
items: [],
unknowns: expect.arrayContaining([
expect.stringContaining("Missing matches do not establish absence"),
]),
});
await assertPortableRoundtrip(client, directory, parent, summary.result);
} finally {
await client.close();
await server.close();
await session.close();
}
},
);
it("refuses summary delivery without a retained Evidence owner and preserves complete delivery", async () => {
const directory = await createTestTempDirectory("rea-inventory-fixed-");
+49 -41
View File
@@ -435,46 +435,53 @@ it("delivers the source-map response URL through authenticated MCP Evidence", as
).toBe(true);
});
it("records a browser-socket endpoint redacted and reconciles its Evidence", async () => {
const browser = await startFakeCdpBrowser();
browsers.push(browser);
const connected = await connectBrowser();
const tool = (await connected.client.listTools()).tools.find(
({ name }) => name === "inspect_web_page",
);
expect(tool?.inputSchema.properties?.cdp_endpoint).toMatchObject({
description: expect.stringContaining("/devtools/browser/ID"),
});
const inspected = await connected.client.callTool({
name: "inspect_web_page",
arguments: {
cdp_endpoint: browser.browserWebSocketUrl,
target_id: "allowed-page",
observation_ms: 0,
include_script_sources: true,
},
});
expect(inspected.isError).not.toBe(true);
expect(inspected.structuredContent).toMatchObject({
parameters: {
cdp_endpoint: browser.browserWebSocketUrl.replace(/fake$/u, "[REDACTED]"),
},
});
expect(JSON.stringify(inspected.structuredContent)).not.toContain(
"/devtools/browser/fake",
);
const reconciled = await connected.client.callTool({
name: "reconcile_javascript_runtime",
arguments: {
static_layers: JAVASCRIPT_RUNTIME_RECONCILIATION_EXAMPLE.static_layers,
runtime_observations: [evidenceFor(inspected.structuredContent)],
},
});
expect(reconciled.isError).not.toBe(true);
expect(reconciled.structuredContent).toMatchObject({
normalized_result: { summary: { runtime_scripts: 1 } },
});
});
it.each(["full", "compact"] as const)(
"records a browser-socket endpoint redacted and reconciles its Evidence (%s)",
async (profile) => {
const browser = await startFakeCdpBrowser();
browsers.push(browser);
const connected = await connectBrowser(profile);
const tool = (await connected.client.listTools()).tools.find(
({ name }) => name === "inspect_web_page",
);
if (profile === "full")
expect(tool?.inputSchema.properties?.cdp_endpoint).toMatchObject({
description: expect.stringContaining("/devtools/browser/ID"),
});
const inspected = await connected.client.callTool({
name: "inspect_web_page",
arguments: {
cdp_endpoint: browser.browserWebSocketUrl,
target_id: "allowed-page",
observation_ms: 0,
include_script_sources: true,
},
});
expect(inspected.isError).not.toBe(true);
expect(inspected.structuredContent).toMatchObject({
parameters: {
cdp_endpoint: browser.browserWebSocketUrl.replace(
/fake$/u,
"[REDACTED]",
),
},
});
expect(JSON.stringify(inspected.structuredContent)).not.toContain(
"/devtools/browser/fake",
);
const reconciled = await connected.client.callTool({
name: "reconcile_javascript_runtime",
arguments: {
static_layers: JAVASCRIPT_RUNTIME_RECONCILIATION_EXAMPLE.static_layers,
runtime_observations: [evidenceFor(inspected.structuredContent)],
},
});
expect(reconciled.isError).not.toBe(true);
expect(reconciled.structuredContent).toMatchObject({
normalized_result: { summary: { runtime_scripts: 1 } },
});
},
);
it("does not attach to a target outside the request's allowed origin scope", async () => {
const browser = await startFakeCdpBrowser();
@@ -538,7 +545,7 @@ it("projects legal oversized PNG dimensions as a memory constraint", async () =>
});
});
const connectBrowser = async () => {
const connectBrowser = async (profile?: "full" | "compact") => {
const session = createTestBinarySession(() => ({
execute: () => Promise.resolve(observed(null)),
close: () => Promise.resolve(resultOk(null)),
@@ -546,6 +553,7 @@ const connectBrowser = async () => {
const server = createServer(
{ kind: "session", session },
{
environment: { ...process.env, REA_MCP_INPUT_SCHEMA_PROFILE: profile },
browserObservation: new CdpBrowserProvider(),
availabilityPolicy: () => ({
processCaptureEnabled: false,
+10 -2
View File
@@ -22,6 +22,8 @@ import { err, ok, type Result } from "../../../src/domain/result.js";
import { createServer } from "../../../src/server/createServer.js";
import { observed } from "../../fixtures/analysisExecution.js";
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
import { advertisedInputJsonSchema } from "../../../src/contracts/toolSchemaMetadata.js";
import { toolContract } from "../../../src/contracts/toolContracts.js";
const providerIdentity: ProviderIdentity = {
id: "test-browser-scenario",
@@ -257,6 +259,10 @@ describe("browser scenario MCP tool", () => {
strict: false,
validateFormats: false,
}).compile(comparisonTool.inputSchema);
const validateFullComparison = new Ajv2020({
strict: false,
validateFormats: false,
}).compile(advertisedInputJsonSchema(toolContract("compare_web_captures")));
expect(
validateComparison(comparisonInput),
JSON.stringify(validateComparison.errors),
@@ -266,12 +272,14 @@ describe("browser scenario MCP tool", () => {
{ before_scenario: captureResult, after: captureResult },
{ ...comparisonInput, before: captureResult, after: captureResult },
]) {
expect(validateComparison(malformed)).toBe(false);
expect(validateFullComparison(malformed)).toBe(false);
const rejected = await client.callTool({
name: "compare_web_captures",
arguments: malformed,
});
expect(rejected.isError).toBe(true);
expect(parseMcpToolError(rejected)).toMatchObject({
error: { code: "invalid_request", category: "invalid_input" },
});
}
const compared = await client.callTool({
name: "compare_web_captures",
@@ -25,8 +25,13 @@ const resultSchema = z.object({
),
});
it("preserves advertised schemas and handler meaning when presentation order changes", async () => {
it("preserves full advertised schemas and handler meaning when presentation order changes", async () => {
const { stdout } = await execute(process.execPath, [fixture, "reversed"], {
env: {
...process.env,
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
},
timeout: 20_000,
maxBuffer: 16 * 1024 * 1024,
});
@@ -8,6 +8,7 @@ import { functionDossierSchema } from "../../../src/domain/hopperValues.js";
import type { RelationshipCoverage } from "../../../src/domain/relationshipCoverage.js";
import { createServer } from "../../../src/server/createServer.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
const observe = (relationshipCoverage: RelationshipCoverage) => {
const template = FUNCTION_COMPARISON_EXAMPLE.left;
@@ -32,11 +33,15 @@ const observe = (relationshipCoverage: RelationshipCoverage) => {
);
};
it("preserves directional gaps through advertised schemas and public path reconstruction", async () => {
const profiles = ["full", "compact"] as const;
it.each(profiles)("preserves directional gaps (%s)", async (profile) => {
const session = createTestBinarySession(() => {
throw new Error("Offline call paths must not launch a provider");
});
const server = createServer({ kind: "session", session });
const server = createServer(
{ kind: "session", session },
{ environment: { ...process.env, REA_MCP_INPUT_SCHEMA_PROFILE: profile } },
);
const client = new Client({ name: "function-call-directions", version: "1" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
@@ -89,21 +94,28 @@ it("preserves directional gaps through advertised schemas and public path recons
["callees", "callees"],
["callers", "callees", "callers"],
["incoming"],
])
expect(
validate({
...input,
functions: [
{
...observation,
relationship_coverage: {
...coverage,
limited_directions,
},
]) {
const malformed = {
...input,
functions: [
{
...observation,
relationship_coverage: {
...coverage,
limited_directions,
},
],
}),
).toBe(false);
},
],
};
if (profile === "full") expect(validate(malformed)).toBe(false);
const rejected = await client.callTool({
name: "build_call_path",
arguments: malformed,
});
expect(parseMcpToolError(rejected)).toMatchObject({
error: { code: "invalid_request", category: "invalid_input" },
});
}
const result = await client.callTool({
name: "build_call_path",
arguments: input,
+35 -12
View File
@@ -2,9 +2,11 @@ import { writeFile, unlink } from "node:fs/promises";
import { join } from "node:path";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { Ajv2020 } from "ajv/dist/2020.js";
import type { ValidateFunction } from "ajv";
import { expect, it } from "vitest";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
import { buildGodotPck } from "../../fixtures/godotPck.js";
import { ArtifactProvider } from "../../../src/artifacts/ArtifactProvider.js";
import { createServer } from "../../../src/server/createServer.js";
@@ -36,7 +38,28 @@ const viewValidators = async (client: Client) => {
return { validateInput, validateOutput };
};
it("validates advertised schemas, batches connected facts and reuses one retained inventory after unlink", async () => {
const assertInvalidNodeField = async (
client: Client,
evidenceId: string,
validateFullInput?: ValidateFunction,
) => {
const arguments_ = {
source: { kind: "retained-evidence", evidence_id: evidenceId },
view: { kind: "inventory", collection: "nodes", fields: ["logical_path"] },
};
if (validateFullInput !== undefined)
expect(validateFullInput(arguments_)).toBe(false);
const rejected = await client.callTool({
name: "inspect_analysis_view",
arguments: arguments_,
});
expect(parseMcpToolError(rejected)).toMatchObject({
error: { code: "invalid_request", category: "invalid_input" },
});
};
const profiles = ["full", "compact"] as const;
it.each(profiles)("reuses retained PCK inventory (%s)", async (profile) => {
const root = await createTestTempDirectory("rea-pck-mcp-");
const path = join(root, "game.pck");
await writeFile(
@@ -47,7 +70,12 @@ it("validates advertised schemas, batches connected facts and reuses one retaine
]),
);
const session = createTestBinarySession(new ArtifactProvider({}));
const server = createServer({ kind: "session", session });
const server = createServer(
{ kind: "session", session },
{
environment: { ...process.env, REA_MCP_INPUT_SCHEMA_PROFILE: profile },
},
);
const client = new Client({ name: "pck-contract-test", version: "1" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
@@ -113,16 +141,11 @@ it("validates advertised schemas, batches connected facts and reuses one retaine
expect(evidence.evidence_links).toContain(parent.evidence_id);
return analysisViewResultSchema.parse(evidence.normalized_result);
};
expect(
validateInput({
source,
view: {
kind: "inventory",
collection: "nodes",
fields: ["logical_path"],
},
}),
).toBe(false);
await assertInvalidNodeField(
client,
parent.evidence_id,
profile === "full" ? validateInput : undefined,
);
const wanted = graph.occurrences.filter((row) => row.logical_path !== ".");
const occurrences = await view({
collection: "occurrences",
@@ -5,6 +5,9 @@ import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarnes
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { advertisedMcpToolSchemas } from "../../fixtures/advertisedMcpTool.js";
import { advertisedInputJsonSchema } from "../../../src/contracts/toolSchemaMetadata.js";
import { toolContract } from "../../../src/contracts/toolContracts.js";
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
import {
analysisViewJavaScriptAnalysis,
analysisViewJavaScriptEvidence,
@@ -79,12 +82,18 @@ it("pages selected real filesystem observations with shared identities and porta
"inspect_analysis_view",
);
expect(ajv.validate(inputSchema, { source, view: query })).toBe(true);
expect(
ajv.validate(inputSchema, {
source,
view: { ...query, property_keys: [""] },
}),
).toBe(false);
const malformed = { source, view: { ...query, property_keys: [""] } };
const fullInputSchema = advertisedInputJsonSchema(
toolContract("inspect_analysis_view"),
);
expect(ajv.validate(fullInputSchema, malformed)).toBe(false);
const rejected = await harness.client.callTool({
name: "inspect_analysis_view",
arguments: malformed,
});
expect(parseMcpToolError(rejected)).toMatchObject({
error: { code: "invalid_request", category: "invalid_input" },
});
const full = await facts(harness, source, query);
expect(ajv.validate(outputSchema, full.wire.structuredContent)).toBe(true);
const portable = await facts(
+39 -6
View File
@@ -15,7 +15,7 @@ import {
runProductionMcpDoctorCli,
} from "../../../src/mcpDoctor.js";
import {
CATALOG_IDENTITY,
FULL_CATALOG_IDENTITY,
createCatalogIdentity,
} from "../../../src/catalogIdentity.js";
import { compactPresentationForMcpInputSchemaProfile } from "../../../src/config/mcpInputSchemaProfile.js";
@@ -23,6 +23,25 @@ import { MCP_STARTUP_POLICY } from "../../../src/mcpStartupPolicy.js";
afterEach(() => vi.useRealTimers());
const assertCliCatalogIdentity = async (
environment: NodeJS.ProcessEnv,
identity: ReturnType<typeof createCatalogIdentity>,
) => {
for (const command of ["capabilities", "providers"]) {
const { stdout } = await promisify(execFile)(
process.execPath,
[resolve("scripts/rea.mjs"), command, "--json"],
{ env: environment, timeout: 15_000, maxBuffer: 1024 * 1024 },
);
const cliStatus: unknown = JSON.parse(stdout);
expect(cliStatus, command).toMatchObject({
server_identity: {
catalog: { counts: identity.counts, digests: identity.digests },
},
});
}
};
describe("production MCP doctor", () => {
it("reuses immutable compiled profile digests and preserves custom-budget identity", async () => {
// Exercise the generated module with Node, outside Vitest's source loader.
@@ -81,13 +100,23 @@ describe("production MCP doctor", () => {
});
it.each([
{
name: "default compact input",
environment: {
REA_MCP_INPUT_SCHEMA_PROFILE: undefined,
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
},
identity: createCatalogIdentity(
compactPresentationForMcpInputSchemaProfile(undefined),
),
},
{
name: "full",
environment: {
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
},
identity: CATALOG_IDENTITY,
identity: FULL_CATALOG_IDENTITY,
},
{
name: "compact",
@@ -119,7 +148,7 @@ describe("production MCP doctor", () => {
),
},
])(
"connects to the production stdio child and verifies the $name catalog",
"verifies the $name catalog through production stdio and both CLI inventories",
async ({ environment, identity }) => {
const result = await runProductionMcpDoctor({
command: process.execPath,
@@ -145,7 +174,7 @@ describe("production MCP doctor", () => {
result.inventory?.tools.observed,
);
expect(result.inventory?.tools.expected).toBe(
CATALOG_IDENTITY.counts.mcp_tools,
FULL_CATALOG_IDENTITY.counts.mcp_tools,
);
expect(result.identity?.expected_catalog_digest).toBe(
identity.digests.combined_sha256,
@@ -153,6 +182,10 @@ describe("production MCP doctor", () => {
expect(result.identity?.observed_catalog_digest).toBe(
identity.digests.combined_sha256,
);
await assertCliCatalogIdentity(
{ ...process.env, ...environment },
identity,
);
},
30_000,
);
@@ -213,10 +246,10 @@ describe("production MCP doctor probe validation", () => {
});
expect(result.healthy).toBe(mode === "valid");
expect(result.inventory?.tools.observed).toBe(
CATALOG_IDENTITY.counts.mcp_tools,
FULL_CATALOG_IDENTITY.counts.mcp_tools,
);
expect(result.inventory?.prompts.observed).toBe(
CATALOG_IDENTITY.counts.mcp_prompts,
FULL_CATALOG_IDENTITY.counts.mcp_prompts,
);
expect(result.checks).toContainEqual({
name: "initialize",
@@ -10,7 +10,7 @@ import { EvidenceMcpServer } from "../../../src/server/EvidenceMcpServer.js";
import { toolRegistrationOptions } from "../../../src/server/toolRegistrationOptions.js";
import { createToolResultDelivery } from "../../../src/server/toolResult.js";
import type { ToolContract } from "../../../src/contracts/toolContractTypes.js";
import { CATALOG_IDENTITY } from "../../../src/catalogIdentity.js";
import { FULL_CATALOG_IDENTITY } from "../../../src/catalogIdentity.js";
const createServer = (omitOutputSchemas: boolean) =>
new EvidenceMcpServer(
@@ -25,7 +25,7 @@ const createServer = (omitOutputSchemas: boolean) =>
it("omits only output schema advertisements and keeps canonical MCP output validation", async () => {
const server = createServer(true);
expect(server.catalogIdentity.digests.combined_sha256).not.toBe(
CATALOG_IDENTITY.digests.combined_sha256,
FULL_CATALOG_IDENTITY.digests.combined_sha256,
);
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
@@ -24,11 +24,14 @@ const application = JAVASCRIPT_FEATURE_TRACE_EXAMPLE.application;
const pair = JAVASCRIPT_APPLICATION_VERSION_COMPARISON_EXAMPLE;
const exportPair = JAVASCRIPT_EXPORT_SHAPE_COMPARISON_EXAMPLE;
async function connect() {
async function connect(profile?: "full" | "compact") {
const session = createTestBinarySession(() => {
throw new Error("references must not launch a provider");
});
const server = createServer({ kind: "session", session });
const server = createServer(
{ kind: "session", session },
{ environment: { ...process.env, REA_MCP_INPUT_SCHEMA_PROFILE: profile } },
);
const client = new Client({ name: "retained-evidence-test", version: "1" });
resources.push(client, server, session);
const [a, b] = InMemoryTransport.createLinkedPair();
@@ -37,33 +40,39 @@ async function connect() {
return { client, session };
}
it("advertises the primitive literal seeds accepted by semantic tracing", async () => {
const { client } = await connect();
const { tools } = await client.listTools();
const tool = tools.find(({ name }) => name === "trace_javascript_semantics");
if (tool === undefined) throw new Error("Semantic trace tool is missing");
const validate = new Ajv2020({
strict: false,
validateFormats: false,
}).compile(tool.inputSchema);
for (const value of ["literal", 42, true, null, {}, []]) {
const primitive = value === null || typeof value !== "object";
const arguments_ = {
application,
query: {
seed: { kind: "literal", value },
direction: "backward-provenance",
},
};
expect(validate(arguments_), JSON.stringify(value)).toBe(primitive);
const result = await client.callTool({
name: tool.name,
arguments: arguments_,
});
if (primitive) expect(result.isError).not.toBe(true);
else expect(result.isError).toBe(true);
}
});
it.each(["full", "compact"] as const)(
"advertises the primitive literal seeds accepted by semantic tracing (%s)",
async (profile) => {
const { client } = await connect(profile);
const { tools } = await client.listTools();
const tool = tools.find(
({ name }) => name === "trace_javascript_semantics",
);
if (tool === undefined) throw new Error("Semantic trace tool is missing");
const validate = new Ajv2020({
strict: false,
validateFormats: false,
}).compile(tool.inputSchema);
for (const value of ["literal", 42, true, null, {}, []]) {
const primitive = value === null || typeof value !== "object";
const arguments_ = {
application,
query: {
seed: { kind: "literal", value },
direction: "backward-provenance",
},
};
if (primitive || profile === "full")
expect(validate(arguments_), JSON.stringify(value)).toBe(primitive);
const result = await client.callTool({
name: tool.name,
arguments: arguments_,
});
if (primitive) expect(result.isError).not.toBe(true);
else expect(result.isError).toBe(true);
}
},
);
it("runs the same trace/compare workflows with inline and retained application Evidence", async () => {
const { client, session } = await connect();
+28 -13
View File
@@ -11,10 +11,13 @@ import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { machoImage } from "../../../src/artifacts/apple/MachoImage.fixture.js";
import {
CATALOG_IDENTITY,
FULL_CATALOG_IDENTITY,
DEFAULT_CATALOG_IDENTITY,
createCatalogDigestProfiles,
createCatalogDigests,
createCatalogIdentity,
} from "../../../src/catalogIdentity.js";
import { COMPACT_INPUT_SCHEMA_BUDGET_BYTES } from "../../../src/config/mcpInputSchemaProfile.js";
import { CLI_COMMAND_NAMES } from "../../../src/cliCommandNames.js";
import { PACKAGE_METADATA } from "../../../src/generatedPackageMetadata.js";
import { PRODUCT_IDENTITY, SDK_IDENTITY } from "../../../src/identity.js";
@@ -112,16 +115,18 @@ describe("server and catalog identity", () => {
packageJson.dependencies["@modelcontextprotocol/server"],
);
expect(new Set(CLI_COMMAND_NAMES).size).toBe(CLI_COMMAND_NAMES.length);
expect(CATALOG_IDENTITY.counts).toEqual({
expect(FULL_CATALOG_IDENTITY.counts).toEqual({
cli_commands: CLI_COMMAND_NAMES.length,
mcp_tools: TOOL_CONTRACTS.length,
mcp_prompts: PROMPT_CONTRACTS.length,
});
expect(CATALOG_IDENTITY.digests.combined_sha256).toMatch(/^[a-f0-9]{64}$/u);
expect(FULL_CATALOG_IDENTITY.digests.combined_sha256).toMatch(
/^[a-f0-9]{64}$/u,
);
});
it("covers each tool's target-content policy in the catalog digest", () => {
expect(createCatalogDigests()).toEqual(CATALOG_IDENTITY.digests);
expect(createCatalogDigests()).toEqual(FULL_CATALOG_IDENTITY.digests);
const [first] = TOOL_CONTRACTS;
if (first === undefined) throw new Error("expected a tool contract");
const toggled = TOOL_CONTRACTS.map((contract) =>
@@ -131,18 +136,20 @@ describe("server and catalog identity", () => {
);
const digests = createCatalogDigests(toggled);
expect(digests.combined_sha256).not.toBe(
CATALOG_IDENTITY.digests.combined_sha256,
FULL_CATALOG_IDENTITY.digests.combined_sha256,
);
expect(digests.tools_sha256).not.toBe(
CATALOG_IDENTITY.digests.tools_sha256,
FULL_CATALOG_IDENTITY.digests.tools_sha256,
);
expect(digests.prompts_sha256).toBe(
CATALOG_IDENTITY.digests.prompts_sha256,
FULL_CATALOG_IDENTITY.digests.prompts_sha256,
);
});
it("binds output schema profile and canonical output contracts into identity", () => {
const profiles = createCatalogDigestProfiles();
expect(FULL_CATALOG_IDENTITY.digests).toEqual(profiles.full.full);
expect(DEFAULT_CATALOG_IDENTITY.digests).toEqual(profiles.compact.full);
const changedOutput = TOOL_CONTRACTS.map((contract, index) =>
index === 0
? {
@@ -187,22 +194,25 @@ describe("server and catalog identity", () => {
it("reports unknown without a live comparison and distinguishes aligned from stale", () => {
const unknown = createServerIdentity({
catalog: DEFAULT_CATALOG_IDENTITY,
startedAt: "2026-07-13T00:00:00.000Z",
commandPath: normalizeEntrypointPath(""),
});
expect(unknown.alignment.state).toBe("unknown");
expect(unknown.server.command_path).toBeNull();
const aligned = createServerIdentity({
catalog: DEFAULT_CATALOG_IDENTITY,
startedAt: "2026-07-13T00:00:00.000Z",
commandPath: "/packaged/rea.mjs",
expected: {
package_version: PRODUCT_IDENTITY.packageVersion,
catalog_digest: CATALOG_IDENTITY.digests.combined_sha256,
catalog_digest: DEFAULT_CATALOG_IDENTITY.digests.combined_sha256,
server_path: "/packaged/rea.mjs",
},
});
expect(aligned.alignment).toMatchObject({ state: "aligned", reasons: [] });
const stale = createServerIdentity({
catalog: DEFAULT_CATALOG_IDENTITY,
startedAt: "2026-07-13T00:00:00.000Z",
commandPath: "/packaged/rea.mjs",
expected: {
@@ -218,6 +228,7 @@ describe("server and catalog identity", () => {
it("keeps registration alignment unknown when the server entrypoint is absent", () => {
const identity = createServerIdentity({
catalog: DEFAULT_CATALOG_IDENTITY,
startedAt: "2026-07-13T00:00:00.000Z",
commandPath: undefined,
expected: { server_path: "/registered/rea.mjs" },
@@ -307,12 +318,14 @@ describe("guided prompt catalog identity", () => {
);
const digests = createCatalogDigests(TOOL_CONTRACTS, changed);
expect(digests.prompts_sha256).not.toBe(
CATALOG_IDENTITY.digests.prompts_sha256,
FULL_CATALOG_IDENTITY.digests.prompts_sha256,
);
expect(digests.combined_sha256).not.toBe(
CATALOG_IDENTITY.digests.combined_sha256,
FULL_CATALOG_IDENTITY.digests.combined_sha256,
);
expect(digests.tools_sha256).toBe(
FULL_CATALOG_IDENTITY.digests.tools_sha256,
);
expect(digests.tools_sha256).toBe(CATALOG_IDENTITY.digests.tools_sha256);
});
});
@@ -390,7 +403,7 @@ describe("live server identity over MCP", () => {
reasons: [],
});
expect(server.catalogIdentity.digests.tools_sha256).not.toBe(
CATALOG_IDENTITY.digests.tools_sha256,
FULL_CATALOG_IDENTITY.digests.tools_sha256,
);
} finally {
await Promise.allSettled([
@@ -538,7 +551,9 @@ const assertLiveIdentity = async (client: Client): Promise<void> => {
},
client: { name: "identity-test", version: "9" },
negotiated_protocol_version: expect.any(String),
catalog: CATALOG_IDENTITY,
catalog: createCatalogIdentity({
budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
}),
alignment: { state: "unknown" },
},
},
@@ -35,7 +35,7 @@ async function verifyListing(listing: string) {
`import childProcess from "node:child_process";
import { syncBuiltinESMExports } from "node:module";
import { promisify } from "node:util";
import { CATALOG_IDENTITY } from ${JSON.stringify(catalogUrl)};
import { FULL_CATALOG_IDENTITY } from ${JSON.stringify(catalogUrl)};
let bridgeParsed = false;
childProcess.execFile = () => { throw new Error("Expected promisified execution"); };
childProcess.execFile[promisify.custom] = async (command, args) => {
@@ -45,7 +45,7 @@ childProcess.execFile[promisify.custom] = async (command, args) => {
else if (command === "tar" && args[0] === "-xOf" && args[2] === "package/package.json")
stdout = JSON.stringify({ bin: { rea: "scripts/rea.mjs", "rea-agents": "scripts/rea.mjs" }, mcpName: "io.github.morluto/rea", dependencies: { "@lydell/node-pty": "1.1.0" } });
else if (command === "tar" && args[0] === "-xOf" && args[2] === "package/skills/reverse-engineer-anything/SKILL.md")
stdout = " tool_count: " + CATALOG_IDENTITY.counts.mcp_tools + "\\n";
stdout = " tool_count: " + FULL_CATALOG_IDENTITY.counts.mcp_tools + "\\n";
else if (command === "tar" && args[0] === "-xf") stdout = "";
else if (command === "python3") { bridgeParsed = true; stdout = ""; }
else throw new Error("Unexpected command: " + command + " " + JSON.stringify(args));
@@ -31,10 +31,10 @@ describe("MCP input schema profile configuration", () => {
},
);
it("selects the same result-affecting budget for the compact profile", () => {
expect(compactPresentationForMcpInputSchemaProfile(undefined)).toBe(
undefined,
);
it("defaults to compact while preserving an explicit full choice", () => {
expect(compactPresentationForMcpInputSchemaProfile(undefined)).toEqual({
budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
});
expect(compactPresentationForMcpInputSchemaProfile("full")).toBe(undefined);
expect(compactPresentationForMcpInputSchemaProfile("compact")).toEqual({
budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES,