mirror of
https://github.com/morluto/rea.git
synced 2026-11-16 03:33:25 +00:00
* test(mcp): check advertised schemas against a portable dialect in every profile * fix(mcp): re-emit advertised patterns in a portable regex grammar * fix(mcp): present discover_tools input through the selected profile The hot-cold catalog registered discover_tools directly with the SDK, so the default compact input profile still advertised its canonical schema, including page-size defaults clients apply on their own (#5322). Route every listed input schema, transport discovery included, through one server-owned presentation, and pass the selected presentation to the deferred server, which serves recorded tool schemas but not discovery. * fix(mcp): keep the Evidence union on described borrowed inputs borrowEvidenceInput installed its advertised projection on one schema instance. A call site's .describe() rebuilds the schema from its definition, so match_native_functions advertised left and right as unconstrained values in every input profile. Install the projection in the borrowed parser's constructor so derived schemas keep it, for both borrowed inputs and borrowed arrays. * fix(mcp): validate advertised dialect by selected profile * fix(mcp): preserve schema widening through references * feat(mcp): project compact input schemas into the portable advertised dialect One projection (advertisedDialectProjection.ts) owns the compact input advertisement: one type per node, items on every array, string literals as enums, oneOf widened to anyOf, allOf merged, tuples as element unions, exclusive bounds made inclusive, and every keyword outside the dialect dropped. Each rewrite is exact or widening; server validation is unchanged. The root description and sentences for dropped cross-field constraints move to the tool description, which Codex and Zed keep. The compact baseline falls to nullable values only (29), which the dialect cannot express without narrowing an input. The gateway shape repairs (typeArrayItemUnions, hasGatewayIncompatibleShape), the second dependentRequired-to-prose copy and the strict-gateway test denylist are removed; the dialect checker gates the inline presentation instead. * fix(mcp): refuse zero-padded repeat bounds and record host-specific path patterns * fix(mcp): keep projected schemas permissive * fix(mcp): keep references valid when presentation widens their targets * fix(mcp): keep a single union branch's closure scoped to its properties * fix(mcp): keep an allOf member's tuple tail with its own positions * fix(mcp): relax references into negations and conditionals presentation may remove * test(mcp): select the borrowed Evidence contract from its family * fix(mcp): keep astral literals and dynamic references widening in every mode * fix(mcp): omit astral literals and relax untracked references when presentation widens * fix(mcp): read reference pointers by schema position, not segment spelling * fix(mcp): protect references into collapsing boolean unions and keep referencing branches out of the collapse * fix(mcp): keep reduced schemas open when their root admits names by pattern * test(mcp): keep the reduced-schema pattern case outside the size-limited block * fix(mcp): omit patterns whose classes name astral characters * fix(mcp): decide union collapse and pointer protection from canonical branches * docs(mcp): document non-u astral matching and pin BMP agreement across V8 modes * fix(mcp): keep prototype-named properties as own properties in the dialect projection * test(mcp): expect the advertised base64 schema to omit the unbounded pad-bit probe * fix(mcp): protect references into unevaluated keywords lost evaluation removes * fix(mcp): emit stack-safe classes with non-ASCII members and protect draft-07 reference siblings * fix(mcp): protect references into literal siblings of relaxed draft-07 references * feat(mcp): omit output schemas from tools/list by default (#10646) * feat(mcp): omit output schemas from tools/list by default No surveyed model API reads an output schema, MCP client validators repeat REA's own result validation, and complete output schemas were about 92% of the default tools/list. REA_MCP_OUTPUT_SCHEMA_PROFILE now resolves to off through one owner (resolveMcpOutputSchemaProfile); compact and full remain selectable. structuredContent is still delivered, and the deferred server serves both off and full from the recorded catalog with the matching recorded identity. Tests and artifact verifiers that validate results against complete advertised output contracts select full explicitly. * fix(mcp): select schemas in verification workflows * fix(mcp): select complete output schemas in the Electron verifier * fix(mcp): select complete output schemas in standalone verifier probes * fix(mcp): select complete output schemas in every verifier transport * test(mcp): select full output schemas in packet and signal schema checks * fix(mcp): validate the doctor probe reply against the canonical output contract (#10655) * fix(mcp): validate the doctor probe reply against the canonical output contract * test(mcp): replay doctor probe validation under every output profile * test(mcp): accept typed-branch nullable unions in the advertised dialect (#10649) * test(mcp): select full output schemas in HTTP session workflows * fix(mcp): keep the Evidence union on described borrowed inputs (#10597) * fix(mcp): keep the Evidence union on described borrowed inputs borrowEvidenceInput installed its advertised projection on one schema instance. A call site's .describe() rebuilds the schema from its definition, so match_native_functions advertised left and right as unconstrained values in every input profile. Install the projection in the borrowed parser's constructor so derived schemas keep it, for both borrowed inputs and borrowed arrays. * test(mcp): select the borrowed Evidence contract from its family * docs(mcp): admit baseline growth only for restored contract declarations * fix(mcp): state reduced multi-type properties as typed branches, not type lists The reduced fallback derived a property's types and emitted them as a type list, which Gemini, Moonshot and Ark reject. A property whose canonical schema is anyOf string and number therefore traded a typed union for a rejected construct when the inline projection fell back. Emit one type, or anyOf typed branches, so [T, null] becomes the dialect's nullable union.
474 lines
15 KiB
JavaScript
474 lines
15 KiB
JavaScript
import {
|
|
closeVerifierResources,
|
|
withVerifierCleanup,
|
|
} from "../../lib/verifier-cleanup.mjs";
|
|
import { requireMcpToolError } from "../../lib/mcp-verifier-results.mjs";
|
|
import { schemaVerifierEnvironment } from "../../lib/mcp-verification-environment.mjs";
|
|
import assert from "node:assert/strict";
|
|
import { execFile, spawn } from "node:child_process";
|
|
import { randomUUID } from "node:crypto";
|
|
import { access, mkdtemp, realpath, rm, writeFile } from "node:fs/promises";
|
|
import { tmpdir } from "node:os";
|
|
import { join } from "node:path";
|
|
import { promisify } from "node:util";
|
|
import { Client } from "@modelcontextprotocol/client";
|
|
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
|
import Ajv2020 from "ajv/dist/2020.js";
|
|
|
|
import {
|
|
cleanupOwnedProcessGroup,
|
|
readProcessRunId,
|
|
} from "../../../dist/process/ProcessOwnership.js";
|
|
|
|
const execute = promisify(execFile);
|
|
|
|
/** Exercise public Electron tools and cancellation using a real owned application. */
|
|
export async function verifyElectronToolContracts({
|
|
executable,
|
|
repositoryRoot,
|
|
}) {
|
|
const root = await realpath(
|
|
await mkdtemp(join(tmpdir(), "rea-electron-e2e-")),
|
|
);
|
|
const main = join(root, "main.cjs");
|
|
const preload = join(root, "preload.cjs");
|
|
const ready = join(root, "ready");
|
|
const entrypoint = join(repositoryRoot, "scripts", "rea.mjs");
|
|
const runId = randomUUID();
|
|
await writeFile(
|
|
main,
|
|
`const electron = require("electron");
|
|
const path = require("node:path");
|
|
const { app, BrowserWindow, ipcMain } = electron;
|
|
const Window = BrowserWindow;
|
|
const mainIpc = ipcMain;
|
|
app.whenReady().then(() => {
|
|
const window = new Window({
|
|
webPreferences: { preload: path.join(__dirname, "preload.cjs") },
|
|
});
|
|
window.loadFile("renderer.html");
|
|
require("node:fs").writeFileSync(${JSON.stringify(ready)}, "ready");
|
|
});
|
|
mainIpc.handle("fixture:echo", (_event, value) => ({ value }));`,
|
|
);
|
|
await writeFile(
|
|
join(root, "renderer.html"),
|
|
`<!doctype html>
|
|
<button id="run">Run</button>
|
|
<output id="result"></output>
|
|
<script src="renderer.js"></script>`,
|
|
);
|
|
await writeFile(
|
|
preload,
|
|
`const electron = require("electron");
|
|
const { contextBridge, ipcRenderer } = electron;
|
|
const bridge = contextBridge;
|
|
const rendererIpc = ipcRenderer;
|
|
bridge.exposeInMainWorld("fixture", {
|
|
echo: (value) => rendererIpc.invoke("fixture:echo", value),
|
|
});`,
|
|
);
|
|
await writeFile(
|
|
join(root, "renderer.js"),
|
|
`document.querySelector("#run").addEventListener("click", async () => {
|
|
const result = await window.fixture.echo("renderer");
|
|
document.querySelector("#result").textContent = result.value;
|
|
});`,
|
|
);
|
|
const transport = new StdioClientTransport({
|
|
command: process.execPath,
|
|
args: [entrypoint, "mcp"],
|
|
// Results are validated against the listed complete output contracts.
|
|
env: schemaVerifierEnvironment({
|
|
...Object.fromEntries(
|
|
["DISPLAY", "XAUTHORITY", "WAYLAND_DISPLAY"].flatMap((name) =>
|
|
process.env[name] === undefined ? [] : [[name, process.env[name]]],
|
|
),
|
|
),
|
|
PATH: process.env.PATH,
|
|
REA_LOG_LEVEL: "silent",
|
|
HOPPER_LAUNCHER_PATH: "/rea-unconfigured/hopper",
|
|
}),
|
|
stderr: "pipe",
|
|
});
|
|
const client = new Client({ name: "electron-real-e2e", version: "1" });
|
|
let serverStderr = "";
|
|
transport.stderr?.on("data", (chunk) => {
|
|
serverStderr += chunk.toString();
|
|
});
|
|
let application;
|
|
let stderr = "";
|
|
await withVerifierCleanup(async () => {
|
|
try {
|
|
await readProcessRunId(process.pid);
|
|
application = spawn(
|
|
executable,
|
|
[
|
|
"--remote-debugging-address=127.0.0.1",
|
|
"--remote-debugging-port=0",
|
|
`--user-data-dir=${join(root, "profile")}`,
|
|
main,
|
|
],
|
|
{
|
|
cwd: root,
|
|
env: { ...process.env, REA_PROCESS_RUN_ID: runId },
|
|
detached: true,
|
|
stdio: ["ignore", "ignore", "pipe"],
|
|
},
|
|
);
|
|
application.stderr.on("data", (chunk) => {
|
|
stderr += chunk.toString();
|
|
});
|
|
const endpoint = await waitForEndpoint(application, () => stderr);
|
|
await client.connect(transport);
|
|
const ajv = new Ajv2020({ strict: false, validateFormats: false });
|
|
const validators = new Map(
|
|
(await client.listTools()).tools.map((tool) => {
|
|
ajv.compile(tool.inputSchema);
|
|
return [
|
|
tool.name,
|
|
tool.outputSchema
|
|
? (tool.outputSchema.$id &&
|
|
ajv.getSchema(tool.outputSchema.$id)) ||
|
|
ajv.compile(tool.outputSchema)
|
|
: undefined,
|
|
];
|
|
}),
|
|
);
|
|
const call = async (name, arguments_) => {
|
|
console.log(`PROBE ${name}`);
|
|
const result = await client.callTool(
|
|
{ name, arguments: arguments_ },
|
|
{ timeout: 150_000 },
|
|
);
|
|
assert.notEqual(result.isError, true, JSON.stringify(result));
|
|
const validate = validators.get(name);
|
|
assert.ok(
|
|
validate?.(result.structuredContent),
|
|
JSON.stringify(validate?.errors),
|
|
);
|
|
return result.structuredContent;
|
|
};
|
|
let targets;
|
|
for (let attempt = 0; attempt < 30; attempt++) {
|
|
targets = await call("list_electron_targets", {
|
|
cdp_endpoint: endpoint,
|
|
});
|
|
if (targets.normalized_result.targets.length > 0) break;
|
|
await pause(100);
|
|
}
|
|
const target = targets.normalized_result.targets.find(
|
|
(item) => item.file_path === join(root, "renderer.html"),
|
|
);
|
|
assert.ok(target, JSON.stringify(targets));
|
|
const page = await call("inspect_electron_page", {
|
|
cdp_endpoint: endpoint,
|
|
target_id: target.target_id,
|
|
include_script_sources: true,
|
|
observation_ms: 50,
|
|
});
|
|
assert.ok(
|
|
page.normalized_result.scripts.items.some(
|
|
(script) =>
|
|
script.file_path === join(root, "renderer.js") &&
|
|
script.source.included,
|
|
),
|
|
);
|
|
const staticAnalysis = await call("analyze_javascript_application", {
|
|
input_path: root,
|
|
});
|
|
assert.equal(staticAnalysis.normalized_result.summary.browser_windows, 1);
|
|
assert.equal(
|
|
staticAnalysis.normalized_result.summary.preload_entrypoints,
|
|
1,
|
|
);
|
|
assert.equal(
|
|
staticAnalysis.normalized_result.summary.context_bridge_apis,
|
|
1,
|
|
);
|
|
assert.equal(
|
|
staticAnalysis.normalized_result.summary.ipc.main_handlers,
|
|
1,
|
|
);
|
|
assert.equal(
|
|
staticAnalysis.normalized_result.summary.ipc.renderer_transmissions,
|
|
1,
|
|
);
|
|
assert.equal(
|
|
staticAnalysis.normalized_result.summary.ipc
|
|
.paired_renderer_transmissions,
|
|
1,
|
|
);
|
|
const reconciled = await call("reconcile_javascript_runtime", {
|
|
static_layers: [
|
|
{
|
|
analysis: staticAnalysis,
|
|
runtime_mappings: [{ kind: "file-root", root }],
|
|
},
|
|
],
|
|
runtime_observations: [page],
|
|
});
|
|
assert.ok(reconciled.normalized_result.runtime_captures.length > 0);
|
|
const cli = await execute(
|
|
process.execPath,
|
|
[
|
|
entrypoint,
|
|
"inspect-electron-page",
|
|
endpoint,
|
|
target.target_id,
|
|
"--include-script-sources",
|
|
"--json",
|
|
],
|
|
{
|
|
env: { ...process.env, REA_LOG_LEVEL: "silent" },
|
|
timeout: 60_000,
|
|
maxBuffer: 32 * 1024 * 1024,
|
|
},
|
|
);
|
|
assert.equal(
|
|
JSON.parse(cli.stdout).normalized_result.target.file_path,
|
|
target.file_path,
|
|
);
|
|
const input = {
|
|
executable_path: executable,
|
|
application_path: main,
|
|
application_root: root,
|
|
actions: [{ step_id: "ipc", kind: "click", selector: "#run" }],
|
|
};
|
|
const active = await client.callTool(
|
|
{ name: "capture_electron_scenario", arguments: input },
|
|
{ timeout: 150_000 },
|
|
);
|
|
const activeError = active.isError
|
|
? requireMcpToolError(active)
|
|
: undefined;
|
|
if (activeError !== undefined)
|
|
assert.equal(
|
|
activeError.code,
|
|
"cleanup_incomplete",
|
|
JSON.stringify(activeError),
|
|
);
|
|
const capture =
|
|
activeError === undefined
|
|
? active.structuredContent.normalized_result
|
|
: activeError.details?.partial_observation?.capture;
|
|
assert.ok(capture, JSON.stringify(activeError ?? active));
|
|
if (activeError !== undefined) {
|
|
assert.equal(capture.application.cleanup, "unverified");
|
|
} else {
|
|
assert.ok(
|
|
validators.get("capture_electron_scenario")(active.structuredContent),
|
|
);
|
|
assert.equal(capture.application.cleanup, "terminated-owned-process");
|
|
}
|
|
assert.equal(capture.actions[0].status, "completed");
|
|
assert.ok(
|
|
capture.ipc.events.some(
|
|
({ kind, channel }) =>
|
|
kind === "main-handler-invocation" && channel === "fixture:echo",
|
|
),
|
|
JSON.stringify(capture.ipc),
|
|
);
|
|
assert.equal(
|
|
capture.coverage.status,
|
|
"partial_attach",
|
|
JSON.stringify(capture),
|
|
);
|
|
assert.ok(
|
|
capture.timeline.events.some(
|
|
(event) =>
|
|
event.kind === "window-lifecycle" && event.event === "created",
|
|
),
|
|
);
|
|
await client.ping();
|
|
console.log(
|
|
`PASS live Electron CLI/MCP discovery, inspection, reconciliation, and active observations; active cleanup=${capture.application.cleanup}`,
|
|
);
|
|
if (process.platform !== "win32")
|
|
await verifyCancellation({ input, ready, entrypoint, root });
|
|
} catch (cause) {
|
|
throw new Error(
|
|
`Electron public-tool verification failed: ${cause.message}\nMCP stderr:\n${serverStderr}`,
|
|
{ cause },
|
|
);
|
|
}
|
|
}, [
|
|
async () => {
|
|
await closeVerifierResources([
|
|
() => client.close(),
|
|
() => transport.close(),
|
|
async () => {
|
|
if (application?.pid) {
|
|
const cleanup = await cleanupOwnedProcessGroup({
|
|
runId,
|
|
leaderPid: application.pid,
|
|
processGroupId: application.pid,
|
|
expectedParentPid: process.pid,
|
|
expectedCommand: executable,
|
|
});
|
|
assert.equal(cleanup.cleaned, true, JSON.stringify(cleanup));
|
|
}
|
|
},
|
|
]);
|
|
await rm(root, { recursive: true, force: true });
|
|
},
|
|
]);
|
|
}
|
|
|
|
const pause = (milliseconds) =>
|
|
new Promise((resolve) => setTimeout(resolve, milliseconds));
|
|
|
|
async function waitForEndpoint(application, diagnostics) {
|
|
const deadline = Date.now() + 30_000;
|
|
while (Date.now() < deadline) {
|
|
const match = /DevTools listening on ws:\/\/127\.0\.0\.1:(\d+)\//u.exec(
|
|
diagnostics(),
|
|
);
|
|
if (match) return `http://127.0.0.1:${match[1]}`;
|
|
assert.equal(application.exitCode, null, diagnostics());
|
|
await pause(100);
|
|
}
|
|
throw new Error(
|
|
`Electron did not expose a loopback endpoint: ${diagnostics()}`,
|
|
);
|
|
}
|
|
|
|
async function verifyCancellation({ input, ready, entrypoint, root }) {
|
|
await rm(ready, { force: true });
|
|
const child = spawn(
|
|
process.execPath,
|
|
[
|
|
entrypoint,
|
|
"capture-electron-scenario",
|
|
JSON.stringify({
|
|
...input,
|
|
actions: [{ step_id: "wait", kind: "wait", duration_ms: 30_000 }],
|
|
}),
|
|
"--json",
|
|
],
|
|
{
|
|
cwd: root,
|
|
env: { ...process.env, REA_LOG_LEVEL: "silent" },
|
|
stdio: ["ignore", "pipe", "pipe"],
|
|
},
|
|
);
|
|
let stdout = "";
|
|
let stderr = "";
|
|
child.stdout.on("data", (chunk) => {
|
|
stdout += chunk.toString();
|
|
});
|
|
child.stderr.on("data", (chunk) => {
|
|
stderr += chunk.toString();
|
|
});
|
|
const exited = new Promise((resolve, reject) => {
|
|
child.once("error", reject);
|
|
child.once("exit", (code, signal) => resolve({ code, signal }));
|
|
});
|
|
let ownership;
|
|
await withVerifierCleanup(async () => {
|
|
const deadline = Date.now() + 30_000;
|
|
while (Date.now() < deadline) {
|
|
const table = (
|
|
await execute("ps", ["-eo", "pid=,ppid=,pgid=,args="], {
|
|
timeout: 5_000,
|
|
maxBuffer: 2 * 1024 * 1024,
|
|
})
|
|
).stdout;
|
|
for (const row of table.split("\n")) {
|
|
const entry = /^\s*(\d+)\s+(\d+)\s+(\d+)\s+(.*)$/u.exec(row);
|
|
if (
|
|
entry &&
|
|
Number(entry[2]) === child.pid &&
|
|
entry[4].includes(input.application_path)
|
|
) {
|
|
const runId = await readProcessRunId(Number(entry[1]));
|
|
assert.ok(runId, "Electron child lacked its ownership token");
|
|
ownership = {
|
|
runId,
|
|
leaderPid: Number(entry[1]),
|
|
processGroupId: Number(entry[3]),
|
|
};
|
|
}
|
|
}
|
|
if (ownership) {
|
|
try {
|
|
await access(ready);
|
|
break;
|
|
} catch {
|
|
/* wait for actual fixture readiness */
|
|
}
|
|
}
|
|
assert.equal(child.exitCode, null, `${stdout}\n${stderr}`);
|
|
await pause(100);
|
|
}
|
|
assert.ok(ownership, "CLI did not acquire an owned Electron application");
|
|
await access(ready);
|
|
// The fixture's readiness precedes Playwright's Inspector acquisition;
|
|
// interrupt the long action rather than its separate startup boundary.
|
|
await pause(1_000);
|
|
child.kill("SIGTERM");
|
|
let timer;
|
|
const result = await Promise.race([
|
|
exited,
|
|
new Promise((_, reject) => {
|
|
timer = setTimeout(
|
|
() => reject(new Error("Electron CLI cancellation timed out")),
|
|
15_000,
|
|
);
|
|
}),
|
|
]).finally(() => clearTimeout(timer));
|
|
assert.equal(result.code, 143, `${stdout}\n${stderr}`);
|
|
assert.equal(result.signal, null);
|
|
const error = JSON.parse(stdout);
|
|
assert.ok(["cancelled", "cleanup_incomplete"].includes(error.code), stdout);
|
|
assert.equal(error.category, "cancelled", stdout);
|
|
assert.doesNotMatch(error.message, /rea doctor/u);
|
|
assert.equal(
|
|
error.details?.partial_observation?.kind,
|
|
"electron-active-observation",
|
|
stdout,
|
|
);
|
|
assert.ok(
|
|
error.details.partial_observation.capture.actions.some(
|
|
(action) => action.status === "cancelled",
|
|
),
|
|
);
|
|
assert.throws(() => process.kill(-ownership.processGroupId, 0), {
|
|
code: "ESRCH",
|
|
});
|
|
console.log(
|
|
`PASS real Electron CLI SIGTERM, retained cancelled actions, and owned group termination; reported code=${error.code}`,
|
|
);
|
|
}, [
|
|
async () => {
|
|
for (const signal of ["SIGTERM", "SIGKILL"]) {
|
|
if (child.exitCode !== null || child.signalCode !== null) return;
|
|
child.kill(signal);
|
|
let timer;
|
|
try {
|
|
if (
|
|
await Promise.race([
|
|
exited.then(() => true),
|
|
new Promise((resolve) => {
|
|
timer = setTimeout(() => resolve(false), 2000);
|
|
}),
|
|
])
|
|
)
|
|
return;
|
|
} finally {
|
|
clearTimeout(timer);
|
|
}
|
|
}
|
|
throw new Error(
|
|
"Electron verifier CLI did not exit after SIGKILL; application cleanup remains independently owned",
|
|
);
|
|
},
|
|
async () => {
|
|
if (ownership) {
|
|
const cleanup = await cleanupOwnedProcessGroup(ownership);
|
|
assert.equal(cleanup.cleaned, true, JSON.stringify(cleanup));
|
|
}
|
|
},
|
|
]);
|
|
}
|