mirror of
https://github.com/morluto/rea.git
synced 2026-10-26 17:35:11 +00:00
feat(config): make the regex matching deadline configurable (#6267)
* feat(config): make the regex matching deadline configurable REA_REGEX_MATCHING_DEADLINE_MS defaults to 5000 ms and refuses values outside 1..2147483647. Recovery and capability text name the effective deadline, its source, and how to change the setting. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(setup): preserve the regex matching deadline in registrations Setup validated REA_REGEX_MATCHING_DEADLINE_MS but left it out of the selected registration environment, so a caller override never reached the launched server and matching fell back to 5000 ms. Co-authored-by: morluto <morluto@users.noreply.github.com> * test(config): restore a stubbed regex matching deadline on failure Kuna deadline assertions now unstub the environment in finally, so a thrown construction or expectation cannot leave the setting stubbed. Co-authored-by: morluto <morluto@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <morluto@users.noreply.github.com>
This commit is contained in:
26 files changed
+588
-95
No files matched your search
@@ -68,7 +68,7 @@ The background export belongs to the session. `close_binary`, or the end of a CL
|
||||
|
||||
The assembly label index is scanned once per export reader. Cancelling a request stops its wait without interrupting other readers; closing the session aborts and drains the shared scan before releasing its files.
|
||||
|
||||
Regex procedure and string search uses ECMAScript Unicode matching in a cancellable worker with a five-second matching deadline and a worker old-generation heap. Literal matching remains the default. Cancellation or a matching deadline leaves the session available. This release has no runtime override for that deadline. The worker requests 128 MiB. A positive Node.js `--max-old-space-size` on the process that starts REA, including `NODE_OPTIONS`, overrides that request and sets the effective heap. `--max-old-space-size-percentage` takes precedence over that flag in either order. REA does not turn the percentage into a MiB, so the effective heap stays unestablished instead of the 128 MiB request or the absolute flag. The reported `NODE_OPTIONS` value is the process initial environment when that block can be read. A later edit to the environment is not the value Node applied, and an unreadable initial value leaves the effective heap unestablished. The launch Node options are the initial command line on Linux when that buffer still has a program entry, not a later edit to `process.execArgv`. A preload can clear that array after V8 has applied the flags. Setting `process.title` or `--title` overwrites the command line, so a buffer without a program entry leaves the effective heap unestablished. `--env-file` or `--env-file-if-exists` can supply `NODE_OPTIONS` when the initial environment has none. That file is not part of the initial environment block, so the effective heap stays unestablished instead of the 128 MiB request. An initial `NODE_OPTIONS` is still used, because Node does not let the file replace it. A positive `--max-heap-size`, including the underscore form and launch `NODE_OPTIONS` on Node versions that allow it, sets the total heap rather than the old generation and can be lower than the 128 MiB request. The effective old generation stays unestablished while that value is in effect. A zero or empty value clears it. `--experimental-config-file` or `--experimental-default-config-file` can set `nodeOptions`, including a heap percentage, from a file that is not read back. The effective heap stays unestablished when either flag is present, including when an absolute `--max-old-space-size` is also on the command line or in the recovered `NODE_OPTIONS`. Node accepts the config path as `=` or the next argument.
|
||||
Regex procedure and string search uses ECMAScript Unicode matching in a cancellable worker with a matching deadline and a worker old-generation heap. Literal matching remains the default. Cancellation or a matching deadline leaves the session available. The deadline is `REA_REGEX_MATCHING_DEADLINE_MS`, a positive decimal integer from 1 to 2147483647 ms. Unset uses 5000 ms. A larger value is refused because Node schedules a delay past that signed 32-bit bound as 1 ms. Invalid values are configuration errors and do not fall back to the default. The setting is read from the environment that starts REA, so a change needs a restart. A deadline failure names the effective milliseconds, whether they came from the default or the setting, the default, and the maximum. The worker old-generation heap does not raise it. The worker requests 128 MiB. A positive Node.js `--max-old-space-size` on the process that starts REA, including `NODE_OPTIONS`, overrides that request and sets the effective heap. `--max-old-space-size-percentage` takes precedence over that flag in either order. REA does not turn the percentage into a MiB, so the effective heap stays unestablished instead of the 128 MiB request or the absolute flag. The reported `NODE_OPTIONS` value is the process initial environment when that block can be read. A later edit to the environment is not the value Node applied, and an unreadable initial value leaves the effective heap unestablished. The launch Node options are the initial command line on Linux when that buffer still has a program entry, not a later edit to `process.execArgv`. A preload can clear that array after V8 has applied the flags. Setting `process.title` or `--title` overwrites the command line, so a buffer without a program entry leaves the effective heap unestablished. `--env-file` or `--env-file-if-exists` can supply `NODE_OPTIONS` when the initial environment has none. That file is not part of the initial environment block, so the effective heap stays unestablished instead of the 128 MiB request. An initial `NODE_OPTIONS` is still used, because Node does not let the file replace it. A positive `--max-heap-size`, including the underscore form and launch `NODE_OPTIONS` on Node versions that allow it, sets the total heap rather than the old generation and can be lower than the 128 MiB request. The effective old generation stays unestablished while that value is in effect. A zero or empty value clears it. `--experimental-config-file` or `--experimental-default-config-file` can set `nodeOptions`, including a heap percentage, from a file that is not read back. The effective heap stays unestablished when either flag is present, including when an absolute `--max-old-space-size` is also on the command line or in the recovered `NODE_OPTIONS`. Node accepts the config path as `=` or the next argument.
|
||||
|
||||
Single-function subprocess output and each cached source facet share a 16 MiB buffered-text capacity. Cached spans are checked before allocation; larger text remains in the local export and the error identifies the file and range. `analyze_function` initially returns the complete dossier, which can still be large; later retained views do not remove that first transfer. Prefer the individual text tools for a focused question.
|
||||
|
||||
|
||||
@@ -202,9 +202,17 @@ Function dossiers retain this same string evidence. Native call edges retain
|
||||
Hopper's partial `CallReference` classification and exact endpoints; detailed
|
||||
reference flags remain unavailable rather than being invented.
|
||||
Regex searches use ECMAScript Unicode syntax in a cancellable worker with a
|
||||
five-second matching deadline and a worker old-generation heap. Deadline or
|
||||
cancellation stops matching while leaving the Hopper API available. This
|
||||
release has no runtime override for that deadline. The worker requests 128
|
||||
matching deadline and a worker old-generation heap. Deadline or cancellation
|
||||
stops matching while leaving the Hopper API available. The deadline is
|
||||
`REA_REGEX_MATCHING_DEADLINE_MS`, a positive decimal integer from 1 to
|
||||
2147483647 ms. Unset uses 5000 ms. A larger value is refused because Node
|
||||
schedules a delay past that signed 32-bit bound as 1 ms. Invalid values are
|
||||
configuration errors and do not fall back to the default. The setting is read
|
||||
from the environment that starts REA, so a change needs a restart. A deadline
|
||||
failure names the effective milliseconds, whether they came from the default
|
||||
or the setting, the default, and the maximum. The worker old-generation heap
|
||||
does not raise it. Kuna regex search uses the same deadline. The worker
|
||||
requests 128
|
||||
MiB. A positive Node.js `--max-old-space-size` on the process that starts
|
||||
REA, including `NODE_OPTIONS`, overrides that request and sets the effective
|
||||
heap. `--max-old-space-size-percentage` takes precedence over that flag in
|
||||
|
||||
@@ -17,7 +17,7 @@ export const SOURCE_TEST_PROJECTS = [
|
||||
group: "source",
|
||||
include: [
|
||||
"src/*.test.ts",
|
||||
"src/{android,apktool,jeb,adb,artifacts,browser,dotnet,electron,evm,firmware,flutter,pcap,ghidra,go,gpu,hopper,ida,inspector,javascript,native,process,reference,rust,server,sqlite,wasm,windows}/**/*.test.ts",
|
||||
"src/{android,apktool,jeb,adb,artifacts,browser,config,dotnet,electron,evm,firmware,flutter,pcap,ghidra,go,gpu,hopper,ida,inspector,javascript,native,process,reference,rust,server,sqlite,wasm,windows}/**/*.test.ts",
|
||||
],
|
||||
},
|
||||
{
|
||||
|
||||
@@ -164,6 +164,7 @@ describe("doctor native analysis configuration", () => {
|
||||
|
||||
it.each([
|
||||
["REA_ANALYSIS_PROVIDER", "Hopper"],
|
||||
["REA_REGEX_MATCHING_DEADLINE_MS", "0"],
|
||||
["REA_GHIDRA_STARTUP_TIMEOUT_MS", "abc"],
|
||||
["REA_SWIFTC_COMPILE_TIMEOUT_MS", "0"],
|
||||
["REA_LOG_LEVEL", "verbose"],
|
||||
|
||||
@@ -7,6 +7,7 @@ const SELECTED_REGISTRATION_SETTINGS = [
|
||||
"REA_HOPPER_STARTUP_TIMEOUT_MS",
|
||||
"REA_JEB_MCP_TIMEOUT_MS",
|
||||
"REA_MCP_INPUT_SCHEMA_PROFILE",
|
||||
"REA_REGEX_MATCHING_DEADLINE_MS",
|
||||
] as const;
|
||||
|
||||
/** Preserve selected setup settings without copying unrelated process values. */
|
||||
|
||||
@@ -53,7 +53,9 @@ const composeDeepBinarySession = (
|
||||
const hopper = new HopperProvider(config, logger, environment);
|
||||
const ghidra = new GhidraProvider(config, logger, environment);
|
||||
const ida = new IdaProvider(config, environment);
|
||||
const kuna = new KunaProvider(environment);
|
||||
const kuna = new KunaProvider(environment, {
|
||||
regexMatchingDeadline: config.regexMatchingDeadline,
|
||||
});
|
||||
const auxiliaryDeclarations =
|
||||
auxiliaryAnalysisProviderDeclarations(environment);
|
||||
const nativeInspectionAvailable =
|
||||
|
||||
@@ -19,12 +19,14 @@ import { mcpInputSchemaProfileSchema } from "./mcpInputSchemaProfile.js";
|
||||
import { mcpOutputSchemaProfileSchema } from "./mcpOutputSchemaProfile.js";
|
||||
import { mcpResponseBudgetSchema } from "./mcpResponseBudget.js";
|
||||
import { asarMaxHeaderBytesSchema } from "./asarHeaderAdmission.js";
|
||||
import { regexMatchingDeadlineMsSchema } from "./regexMatchingDeadline.js";
|
||||
import { parseLoaderArgs, parseStringArray } from "./parsers.js";
|
||||
|
||||
const environmentSchema = z.object({
|
||||
REA_ANALYSIS_PROVIDER: analysisProviderSelectorSchema.default("auto"),
|
||||
REA_MCP_MAX_RESPONSE_BYTES: mcpResponseBudgetSchema.optional(),
|
||||
REA_ASAR_MAX_HEADER_BYTES: asarMaxHeaderBytesSchema.optional(),
|
||||
REA_REGEX_MATCHING_DEADLINE_MS: regexMatchingDeadlineMsSchema.optional(),
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: mcpInputSchemaProfileSchema.optional(),
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: mcpOutputSchemaProfileSchema.optional(),
|
||||
REA_IDA_MCP_CONFIG: z
|
||||
|
||||
@@ -4,6 +4,7 @@ import type { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { ok, type Result } from "../domain/result.js";
|
||||
import { parseEnvironment } from "./environment.js";
|
||||
import { homeDirectoryFromEnvironment } from "./homeDirectory.js";
|
||||
import { DEFAULT_REGEX_MATCHING_DEADLINE } from "./regexMatchingDeadline.js";
|
||||
import type { AppConfig } from "./types.js";
|
||||
import {
|
||||
linuxHopperLauncherCandidates,
|
||||
@@ -121,5 +122,12 @@ export const parseConfig = (
|
||||
referenceSourcePolicy: {
|
||||
secretPatterns: env.referenceSecretPatterns,
|
||||
},
|
||||
regexMatchingDeadline:
|
||||
env.REA_REGEX_MATCHING_DEADLINE_MS === undefined
|
||||
? DEFAULT_REGEX_MATCHING_DEADLINE
|
||||
: {
|
||||
deadlineMs: env.REA_REGEX_MATCHING_DEADLINE_MS,
|
||||
source: "setting",
|
||||
},
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,136 @@
|
||||
import { expect, it, vi } from "vitest";
|
||||
|
||||
import { HopperProvider } from "../hopper/HopperProvider.js";
|
||||
import { KunaProvider } from "../kuna/KunaProvider.js";
|
||||
import { silentLogger } from "../logger.js";
|
||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import { regexMatchingDeadlineError } from "../process/RegexMatcher.js";
|
||||
import { parseEnvironment } from "./environment.js";
|
||||
import { parseConfig } from "./parseConfig.js";
|
||||
import {
|
||||
DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
MAXIMUM_REGEX_MATCHING_DEADLINE_MS,
|
||||
parseRegexMatchingDeadline,
|
||||
REGEX_MATCHING_DEADLINE_SETTING,
|
||||
regexMatchingDeadlineConstraint,
|
||||
regexMatchingDeadlineRecovery,
|
||||
} from "./regexMatchingDeadline.js";
|
||||
|
||||
const configured = (deadlineMs: number) =>
|
||||
({ deadlineMs, source: "setting" }) as const;
|
||||
|
||||
it("keeps the 5000 ms deadline when the setting is unset", () => {
|
||||
expect(parseRegexMatchingDeadline(undefined)).toEqual({
|
||||
ok: true,
|
||||
value: DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
});
|
||||
const parsed = parseConfig({});
|
||||
if (!parsed.ok) throw new Error("unset deadline was rejected");
|
||||
expect(parsed.value.regexMatchingDeadline).toEqual(
|
||||
DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
);
|
||||
const recovery = regexMatchingDeadlineRecovery(
|
||||
DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
);
|
||||
expect(recovery).toContain(
|
||||
"The effective REA_REGEX_MATCHING_DEADLINE_MS value is 5000 ms (source default; default 5000 ms, floor 1 ms, maximum 2147483647 ms).",
|
||||
);
|
||||
const projected = projectAnalysisError(
|
||||
regexMatchingDeadlineError("search_procedures"),
|
||||
);
|
||||
expect(projected.code).toBe("resource_constraint");
|
||||
expect(projected.message).toContain(recovery);
|
||||
expect(projected.remediation.action).toContain(recovery);
|
||||
expect(projected.message).not.toContain("no runtime override");
|
||||
});
|
||||
|
||||
it("uses a valid REA_REGEX_MATCHING_DEADLINE_MS override", () => {
|
||||
expect(parseRegexMatchingDeadline("10000")).toEqual({
|
||||
ok: true,
|
||||
value: configured(10_000),
|
||||
});
|
||||
expect(
|
||||
parseRegexMatchingDeadline(String(MAXIMUM_REGEX_MATCHING_DEADLINE_MS)),
|
||||
).toEqual({
|
||||
ok: true,
|
||||
value: configured(MAXIMUM_REGEX_MATCHING_DEADLINE_MS),
|
||||
});
|
||||
const parsed = parseConfig({ REA_REGEX_MATCHING_DEADLINE_MS: "10000" });
|
||||
if (!parsed.ok) throw new Error("valid deadline was rejected");
|
||||
expect(parsed.value.regexMatchingDeadline).toEqual(configured(10_000));
|
||||
const hopper = new HopperProvider(parsed.value, silentLogger, {
|
||||
REA_REGEX_MATCHING_DEADLINE_MS: "1",
|
||||
});
|
||||
const hopperText = hopper
|
||||
.capabilities()
|
||||
.find((capability) => capability.operation === "search_procedures")
|
||||
?.limitations.join(" ");
|
||||
expect(hopperText).toContain("10000 ms matching deadline (source setting)");
|
||||
expect(hopperText).toContain("REA_REGEX_MATCHING_DEADLINE_MS");
|
||||
expect(hopperText).toContain("default 5000 ms");
|
||||
expect(hopperText).toContain("maximum 2147483647 ms");
|
||||
expect(hopperText).not.toContain("no runtime override");
|
||||
|
||||
vi.stubEnv(REGEX_MATCHING_DEADLINE_SETTING, "1");
|
||||
try {
|
||||
const kuna = new KunaProvider({ REA_REGEX_MATCHING_DEADLINE_MS: "10000" });
|
||||
const kunaText = kuna
|
||||
.capabilities()
|
||||
.find((capability) => capability.operation === "search_strings")
|
||||
?.limitations.join(" ");
|
||||
expect(kunaText).toContain("10000 ms matching deadline (source setting)");
|
||||
expect(kunaText).not.toContain("1 ms matching deadline");
|
||||
} finally {
|
||||
vi.unstubAllEnvs();
|
||||
}
|
||||
|
||||
const ambient = new KunaProvider({});
|
||||
expect(
|
||||
ambient
|
||||
.capabilities()
|
||||
.find((capability) => capability.operation === "search_procedures")
|
||||
?.limitations.join(" "),
|
||||
).toContain("5000 ms matching deadline (source default)");
|
||||
});
|
||||
|
||||
it.each([
|
||||
"",
|
||||
"0",
|
||||
"-1",
|
||||
"05000",
|
||||
"10_000",
|
||||
"1e4",
|
||||
" 5000",
|
||||
"5000 ",
|
||||
"1.5",
|
||||
String(MAXIMUM_REGEX_MATCHING_DEADLINE_MS + 1),
|
||||
"9".repeat(40),
|
||||
])("rejects invalid %j as a configuration error", (value) => {
|
||||
const parsed = parseRegexMatchingDeadline(value);
|
||||
if (parsed.ok) throw new Error(`Accepted ${JSON.stringify(value)}`);
|
||||
expect(parsed.error).toBeInstanceOf(ConfigurationError);
|
||||
expect(parsed.error.settings).toEqual([
|
||||
{
|
||||
setting: "REA_REGEX_MATCHING_DEADLINE_MS",
|
||||
constraint: regexMatchingDeadlineConstraint,
|
||||
},
|
||||
]);
|
||||
expect(parsed.error.message).toBe(regexMatchingDeadlineConstraint);
|
||||
const startup = parseEnvironment({ REA_REGEX_MATCHING_DEADLINE_MS: value });
|
||||
if (startup.ok) throw new Error(`Startup accepted ${JSON.stringify(value)}`);
|
||||
expect(startup.error.settings).toContainEqual({
|
||||
setting: "REA_REGEX_MATCHING_DEADLINE_MS",
|
||||
constraint: expect.stringContaining("from 1 to 2147483647 ms"),
|
||||
});
|
||||
const config = parseConfig({ REA_REGEX_MATCHING_DEADLINE_MS: value });
|
||||
if (config.ok)
|
||||
throw new Error(`parseConfig accepted ${JSON.stringify(value)}`);
|
||||
expect(projectAnalysisError(config.error)).toMatchObject({
|
||||
code: "configuration_invalid",
|
||||
message: expect.stringContaining("REA_REGEX_MATCHING_DEADLINE_MS"),
|
||||
});
|
||||
expect(
|
||||
() => new KunaProvider({ REA_REGEX_MATCHING_DEADLINE_MS: value }),
|
||||
).toThrow(ConfigurationError);
|
||||
});
|
||||
@@ -0,0 +1,92 @@
|
||||
import { z } from "zod";
|
||||
|
||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
export const REGEX_MATCHING_DEADLINE_SETTING = "REA_REGEX_MATCHING_DEADLINE_MS";
|
||||
|
||||
/** Matching deadline used when the setting is unset. Today's fixed constant. */
|
||||
export const DEFAULT_REGEX_MATCHING_DEADLINE_MS = 5_000;
|
||||
|
||||
/**
|
||||
* Largest delay Node schedules exactly. A `setTimeout` delay above this
|
||||
* signed 32-bit bound is clamped to 1 ms, so a matching deadline past it
|
||||
* would expire immediately. The setting stops here instead of growing
|
||||
* without a bound. The floor is 1 ms: zero and negative values are not a
|
||||
* deadline.
|
||||
*/
|
||||
export const MAXIMUM_REGEX_MATCHING_DEADLINE_MS = 2_147_483_647;
|
||||
|
||||
/** Effective regex matching deadline and where its value came from. */
|
||||
export interface RegexMatchingDeadline {
|
||||
readonly deadlineMs: number;
|
||||
readonly source: "default" | "setting";
|
||||
}
|
||||
|
||||
export const DEFAULT_REGEX_MATCHING_DEADLINE: RegexMatchingDeadline = {
|
||||
deadlineMs: DEFAULT_REGEX_MATCHING_DEADLINE_MS,
|
||||
source: "default",
|
||||
};
|
||||
|
||||
export const regexMatchingDeadlineConstraint = `${REGEX_MATCHING_DEADLINE_SETTING} must be a positive decimal integer from 1 to ${String(MAXIMUM_REGEX_MATCHING_DEADLINE_MS)} ms; the default is ${String(DEFAULT_REGEX_MATCHING_DEADLINE_MS)} ms. Node schedules a delay above that maximum as 1 ms, so a larger value is refused.`;
|
||||
|
||||
const decimalInteger = /^[1-9]\d*$/u;
|
||||
|
||||
const withinDeadlineRange = (value: string): boolean => {
|
||||
// The pattern check reports a non-decimal value. Refine only bounds integers.
|
||||
if (!decimalInteger.test(value)) return true;
|
||||
const parsed = BigInt(value);
|
||||
return parsed >= 1n && parsed <= BigInt(MAXIMUM_REGEX_MATCHING_DEADLINE_MS);
|
||||
};
|
||||
|
||||
/** Optional decimal millisecond count for the supervised regex matching deadline. */
|
||||
export const regexMatchingDeadlineMsSchema = z
|
||||
.string()
|
||||
.regex(decimalInteger, { error: regexMatchingDeadlineConstraint })
|
||||
.refine(withinDeadlineRange, { error: regexMatchingDeadlineConstraint })
|
||||
.transform(Number);
|
||||
|
||||
/** Resolve the effective deadline from one environment value. */
|
||||
export const parseRegexMatchingDeadline = (
|
||||
value: string | undefined,
|
||||
): Result<RegexMatchingDeadline, ConfigurationError> => {
|
||||
if (value === undefined) return ok(DEFAULT_REGEX_MATCHING_DEADLINE);
|
||||
const parsed = regexMatchingDeadlineMsSchema.safeParse(value);
|
||||
return parsed.success
|
||||
? ok({ deadlineMs: parsed.data, source: "setting" })
|
||||
: err(
|
||||
new ConfigurationError(regexMatchingDeadlineConstraint, {
|
||||
settings: [
|
||||
{
|
||||
setting: REGEX_MATCHING_DEADLINE_SETTING,
|
||||
constraint: regexMatchingDeadlineConstraint,
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
* Resolve the deadline from a composition root's selected environment. An
|
||||
* invalid setting is refused, not ignored. Callers that already resolved it
|
||||
* at startup pass that value through instead of reading the environment again.
|
||||
*/
|
||||
export const resolveRegexMatchingDeadline = (
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
): RegexMatchingDeadline => {
|
||||
const parsed = parseRegexMatchingDeadline(
|
||||
environment[REGEX_MATCHING_DEADLINE_SETTING],
|
||||
);
|
||||
if (!parsed.ok) throw parsed.error;
|
||||
return parsed.value;
|
||||
};
|
||||
|
||||
/** How to read and change the effective supervised matching deadline. */
|
||||
export const regexMatchingDeadlineRecovery = (
|
||||
deadline: RegexMatchingDeadline,
|
||||
): string =>
|
||||
[
|
||||
`The effective ${REGEX_MATCHING_DEADLINE_SETTING} value is ${String(deadline.deadlineMs)} ms (source ${deadline.source}; default ${String(DEFAULT_REGEX_MATCHING_DEADLINE_MS)} ms, floor 1 ms, maximum ${String(MAXIMUM_REGEX_MATCHING_DEADLINE_MS)} ms).`,
|
||||
`Set ${REGEX_MATCHING_DEADLINE_SETTING} in the environment that starts REA to a positive decimal integer from 1 to ${String(MAXIMUM_REGEX_MATCHING_DEADLINE_MS)}, then restart REA.`,
|
||||
"The worker old-generation heap does not raise this deadline.",
|
||||
].join(" ");
|
||||
@@ -5,6 +5,7 @@ import type { AnalysisProviderSelector } from "../contracts/providerSelection.js
|
||||
import type { GhidraLanguageOverride } from "./ghidraLanguageOverride.js";
|
||||
import type { McpInputSchemaProfile } from "./mcpInputSchemaProfile.js";
|
||||
import type { McpOutputSchemaProfile } from "./mcpOutputSchemaProfile.js";
|
||||
import type { RegexMatchingDeadline } from "./regexMatchingDeadline.js";
|
||||
|
||||
export interface AppConfig {
|
||||
readonly mcpMaxResponseBytes?: number;
|
||||
@@ -27,4 +28,6 @@ export interface AppConfig {
|
||||
readonly hopperLoaderArgs: readonly string[];
|
||||
readonly logLevel: LogLevel;
|
||||
readonly referenceSourcePolicy: ReferenceSourcePolicy;
|
||||
/** Supervised ECMAScript matching deadline resolved at startup. */
|
||||
readonly regexMatchingDeadline: RegexMatchingDeadline;
|
||||
}
|
||||
@@ -19,6 +19,9 @@ import { functionInstructionInputSchema } from "./functionInstructionContract.js
|
||||
import { HOPPER_MEMORY_TOOL_DEFINITIONS } from "./hopperMemoryContracts.js";
|
||||
import { analysisSearchInput } from "./analysisSearchContract.js";
|
||||
|
||||
const hopperRegexDeadline =
|
||||
"Hopper regex mode uses ECMAScript Unicode syntax in a cancellable worker. The matching deadline is REA_REGEX_MATCHING_DEADLINE_MS (a positive decimal integer of milliseconds from 1 to 2147483647, default 5000). Unset keeps the default. A deadline failure names the effective milliseconds, source (default or setting), the default, and the maximum, and leaves the API usable.";
|
||||
|
||||
const official = <Name extends string, Schema extends z.ZodObject>(
|
||||
name: Name,
|
||||
description: string,
|
||||
@@ -200,12 +203,12 @@ export const OFFICIAL_TOOL_CONTRACTS = [
|
||||
),
|
||||
official(
|
||||
"search_procedures",
|
||||
"Search every analyzed procedure name using literal matching by default or regex when requested. Results are deterministic and complete. Hopper regex mode uses ECMAScript Unicode syntax in a cancellable worker with a fixed 5000 ms matching deadline; this release has no runtime override, and a matching deadline leaves its API usable. Select detail summary for counts and retained native-inventory pages; complete remains the default.",
|
||||
`Search every analyzed procedure name using literal matching by default or regex when requested. Results are deterministic and complete. ${hopperRegexDeadline} Select detail summary for counts and retained native-inventory pages; complete remains the default.`,
|
||||
z.object({ ...analysisSearchInput, detail: analysisDetailSchema }),
|
||||
),
|
||||
official(
|
||||
"search_strings",
|
||||
"Search every provider-reported string object or candidate using literal matching by default or regex when requested. Results are deterministic and complete over that provider inventory, not every text sequence in artifact bytes. Kuna scans mapped sequences of at least five characters, including unterminated fragments, without computing references. Ghidra may omit packed or undefined literals, including Rust strings; an empty result does not prove text is absent. If an address is known, read_bytes and xrefs inspect its bytes and references. Hopper retains typed byte decoding, native display, and explicit decoding unknowns. Adjacent long-literal fragments match independently. Hopper regex mode uses ECMAScript Unicode syntax in a cancellable worker with a fixed 5000 ms matching deadline; this release has no runtime override, and a matching deadline leaves its API usable. Select detail summary for counts and retained native-inventory pages; complete remains the default.",
|
||||
`Search every provider-reported string object or candidate using literal matching by default or regex when requested. Results are deterministic and complete over that provider inventory, not every text sequence in artifact bytes. Kuna scans mapped sequences of at least five characters, including unterminated fragments, without computing references. Ghidra may omit packed or undefined literals, including Rust strings; an empty result does not prove text is absent. If an address is known, read_bytes and xrefs inspect its bytes and references. Hopper retains typed byte decoding, native display, and explicit decoding unknowns. Adjacent long-literal fragments match independently. ${hopperRegexDeadline} Select detail summary for counts and retained native-inventory pages; complete remains the default.`,
|
||||
z.object({ ...analysisSearchInput, detail: analysisDetailSchema }),
|
||||
),
|
||||
official(
|
||||
|
||||
@@ -45,13 +45,14 @@ import { projectHopperNameInventory } from "./HopperNameInventory.js";
|
||||
import { hopperMachOImageSchema } from "./HopperMachOImage.js";
|
||||
|
||||
import {
|
||||
CAPABILITIES,
|
||||
hopperCapabilities,
|
||||
HOPPER_PROVIDER_IDENTITY,
|
||||
} from "./HopperProviderCapabilities.js";
|
||||
|
||||
/** Concrete analysis provider backed by REA's private Hopper bridge. */
|
||||
export class HopperProvider implements AnalysisProviderCandidate {
|
||||
private readonly environment: Readonly<NodeJS.ProcessEnv>;
|
||||
readonly #capabilities: readonly CapabilityDescriptor[];
|
||||
readonly #ownedResources = new ArtifactResourceScope();
|
||||
readonly #inspectionOwner: RetainedResourceScope = {
|
||||
retain: (owner, resource) =>
|
||||
@@ -76,6 +77,7 @@ export class HopperProvider implements AnalysisProviderCandidate {
|
||||
private readonly platform: NodeJS.Platform = process.platform,
|
||||
) {
|
||||
this.environment = snapshotEnvironment(environment, platform);
|
||||
this.#capabilities = hopperCapabilities(config.regexMatchingDeadline);
|
||||
}
|
||||
|
||||
identity(): ProviderIdentity {
|
||||
@@ -83,7 +85,7 @@ export class HopperProvider implements AnalysisProviderCandidate {
|
||||
}
|
||||
|
||||
capabilities(): readonly CapabilityDescriptor[] {
|
||||
return CAPABILITIES;
|
||||
return this.#capabilities;
|
||||
}
|
||||
|
||||
inspectAvailability(): ProviderAvailability {
|
||||
@@ -309,7 +311,10 @@ export class HopperProvider implements AnalysisProviderCandidate {
|
||||
onDiagnostic: (diagnostic) =>
|
||||
this.logger.info(diagnostic, "Hopper bridge reported a diagnostic"),
|
||||
});
|
||||
const regexSearch = new HopperRegexSearch(client);
|
||||
const regexSearch = new HopperRegexSearch(
|
||||
client,
|
||||
this.config.regexMatchingDeadline,
|
||||
);
|
||||
const projectExecution = (
|
||||
operation: AnalysisOperation,
|
||||
normalized: JsonValue,
|
||||
@@ -330,7 +335,7 @@ export class HopperProvider implements AnalysisProviderCandidate {
|
||||
: [
|
||||
"REA loaded a verified thin Mach-O image prepared from the original source. Identity and file offsets refer to the original file. The owned temporary image and its Hopper document close together, including on MCP shutdown.",
|
||||
]),
|
||||
...(CAPABILITIES.find(
|
||||
...(this.#capabilities.find(
|
||||
(descriptor) => descriptor.operation === operation,
|
||||
)?.limitations ?? []),
|
||||
...(operation === "read_bytes"
|
||||
|
||||
@@ -1,4 +1,8 @@
|
||||
import { REGEX_MATCHING_LIMITATION } from "../process/RegexMatcher.js";
|
||||
import {
|
||||
DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
type RegexMatchingDeadline,
|
||||
} from "../config/regexMatchingDeadline.js";
|
||||
import { launchRegexMatchingLimitation } from "../process/RegexMatcher.js";
|
||||
import type {
|
||||
CapabilityDescriptor,
|
||||
ProviderIdentity,
|
||||
@@ -62,48 +66,52 @@ const MUTATING_OPERATIONS = new Set<string>([
|
||||
]);
|
||||
|
||||
/** Source-owned capabilities without acquiring or probing a Hopper session. */
|
||||
export const CAPABILITIES: readonly CapabilityDescriptor[] = Object.freeze(
|
||||
[...HOPPER_OPERATIONS, "search_bytes" as const].map((operation) =>
|
||||
Object.freeze({
|
||||
provider: HOPPER_PROVIDER_IDENTITY,
|
||||
operation,
|
||||
...(operation === "search_bytes"
|
||||
? {
|
||||
available: false as const,
|
||||
reason: "Masked byte search has not been verified for Hopper.",
|
||||
}
|
||||
: { available: true as const, reason: null }),
|
||||
effects: Object.freeze({
|
||||
mutatesArtifact: MUTATING_OPERATIONS.has(operation),
|
||||
launchesProcess: true,
|
||||
mayShowUi: true,
|
||||
mayAccessNetwork: false,
|
||||
mayWriteFilesystem: MUTATING_OPERATIONS.has(operation),
|
||||
changesPermissions: false,
|
||||
requiresRoot: false,
|
||||
export const hopperCapabilities = (
|
||||
deadline: RegexMatchingDeadline = DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
): readonly CapabilityDescriptor[] =>
|
||||
Object.freeze(
|
||||
[...HOPPER_OPERATIONS, "search_bytes" as const].map((operation) =>
|
||||
Object.freeze({
|
||||
provider: HOPPER_PROVIDER_IDENTITY,
|
||||
operation,
|
||||
...(operation === "search_bytes"
|
||||
? {
|
||||
available: false as const,
|
||||
reason: "Masked byte search has not been verified for Hopper.",
|
||||
}
|
||||
: { available: true as const, reason: null }),
|
||||
effects: Object.freeze({
|
||||
mutatesArtifact: MUTATING_OPERATIONS.has(operation),
|
||||
launchesProcess: true,
|
||||
mayShowUi: true,
|
||||
mayAccessNetwork: false,
|
||||
mayWriteFilesystem: MUTATING_OPERATIONS.has(operation),
|
||||
changesPermissions: false,
|
||||
requiresRoot: false,
|
||||
}),
|
||||
limitations: Object.freeze([
|
||||
"Results depend on Hopper's completed static analysis.",
|
||||
...(operation === "address_to_file_offset"
|
||||
? [
|
||||
"Original-file coordinates require source bytes matching the selected executable digest. A changed, unavailable, or database-only source retains native mapping facts in partial error evidence.",
|
||||
]
|
||||
: []),
|
||||
...(operation === "search_strings" ||
|
||||
operation === "search_procedures"
|
||||
? [
|
||||
`${launchRegexMatchingLimitation(deadline)} Literal mode retains native casefold semantics.`,
|
||||
]
|
||||
: []),
|
||||
...(operation === "list_strings" ||
|
||||
operation === "search_strings" ||
|
||||
operation === "analyze_function"
|
||||
? [
|
||||
"String values cover native typed objects. Hopper may split long literals into adjacent unterminated fragments; string searches match each object independently.",
|
||||
"Undecodable typed objects retain native display text with an explicit decoding unknown; these values do not establish decoded source literals.",
|
||||
"Reported string encodings are inferred from typed bytes and native display; the original source's intended encoding is unknown.",
|
||||
]
|
||||
: []),
|
||||
]),
|
||||
}),
|
||||
limitations: Object.freeze([
|
||||
"Results depend on Hopper's completed static analysis.",
|
||||
...(operation === "address_to_file_offset"
|
||||
? [
|
||||
"Original-file coordinates require source bytes matching the selected executable digest. A changed, unavailable, or database-only source retains native mapping facts in partial error evidence.",
|
||||
]
|
||||
: []),
|
||||
...(operation === "search_strings" || operation === "search_procedures"
|
||||
? [
|
||||
`${REGEX_MATCHING_LIMITATION} Literal mode retains native casefold semantics.`,
|
||||
]
|
||||
: []),
|
||||
...(operation === "list_strings" ||
|
||||
operation === "search_strings" ||
|
||||
operation === "analyze_function"
|
||||
? [
|
||||
"String values cover native typed objects. Hopper may split long literals into adjacent unterminated fragments; string searches match each object independently.",
|
||||
"Undecodable typed objects retain native display text with an explicit decoding unknown; these values do not establish decoded source literals.",
|
||||
"Reported string encodings are inferred from typed bytes and native display; the original source's intended encoding is unknown.",
|
||||
]
|
||||
: []),
|
||||
]),
|
||||
}),
|
||||
),
|
||||
);
|
||||
),
|
||||
);
|
||||
@@ -12,6 +12,10 @@ import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { analysisSearchInput } from "../contracts/analysisSearchContract.js";
|
||||
import type { HopperClient } from "./HopperClient.js";
|
||||
import {
|
||||
DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
type RegexMatchingDeadline,
|
||||
} from "../config/regexMatchingDeadline.js";
|
||||
import { matchRegexValues } from "../process/RegexMatcher.js";
|
||||
import { analysisClientFailureForOperation } from "../providers/AnalysisClientFailure.js";
|
||||
|
||||
@@ -26,7 +30,10 @@ export class HopperRegexSearch {
|
||||
Promise<Result<JsonValue, AnalysisError>>
|
||||
>();
|
||||
|
||||
constructor(private readonly client: Pick<HopperClient, "callTool">) {}
|
||||
constructor(
|
||||
private readonly client: Pick<HopperClient, "callTool">,
|
||||
private readonly deadline: RegexMatchingDeadline = DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
) {}
|
||||
|
||||
execute(
|
||||
operation: "search_strings" | "search_procedures",
|
||||
@@ -129,6 +136,7 @@ export class HopperRegexSearch {
|
||||
caseSensitive: input.data.case_sensitive,
|
||||
values: parsed.data.map((item) => item.value),
|
||||
signal: options.signal,
|
||||
deadline: this.deadline,
|
||||
invalidPattern: (reason) =>
|
||||
new HopperRemoteError(-32000, `Invalid regex pattern: ${reason}`, {
|
||||
diagnosticType: "invalid_request",
|
||||
|
||||
@@ -16,6 +16,10 @@ import {
|
||||
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
|
||||
import { err, ok } from "../domain/result.js";
|
||||
import { RegularFileCleanupFailure } from "../filesystem/OwnedRegularFile.js";
|
||||
import {
|
||||
resolveRegexMatchingDeadline,
|
||||
type RegexMatchingDeadline,
|
||||
} from "../config/regexMatchingDeadline.js";
|
||||
import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
|
||||
import {
|
||||
kunaFileCleanupError,
|
||||
@@ -53,6 +57,7 @@ const VERIFIED_ARCHITECTURES: ReadonlySet<string> = new Set([
|
||||
*/
|
||||
export class KunaProvider implements AnalysisProviderCandidate {
|
||||
readonly #environment: Readonly<NodeJS.ProcessEnv>;
|
||||
readonly #deadline: RegexMatchingDeadline;
|
||||
readonly #platform: NodeJS.Platform;
|
||||
readonly #launcher: KunaProcessLauncher | undefined;
|
||||
readonly #probeTimeoutMs: number | undefined;
|
||||
@@ -69,10 +74,18 @@ export class KunaProvider implements AnalysisProviderCandidate {
|
||||
readonly probeTimeoutMs?: number;
|
||||
/** Test seam for foreground native runs; production derives it from the host. */
|
||||
readonly singleFunctionRuns?: number;
|
||||
/**
|
||||
* Deadline already resolved at startup. Omitted means resolve the
|
||||
* selected environment; an invalid setting is refused.
|
||||
*/
|
||||
readonly regexMatchingDeadline?: RegexMatchingDeadline;
|
||||
} = {},
|
||||
) {
|
||||
this.#platform = options.platform ?? process.platform;
|
||||
this.#environment = snapshotEnvironment(environment, this.#platform);
|
||||
this.#deadline =
|
||||
options.regexMatchingDeadline ??
|
||||
resolveRegexMatchingDeadline(this.#environment);
|
||||
this.#launcher = options.launcher;
|
||||
this.#probeTimeoutMs = options.probeTimeoutMs;
|
||||
this.#singleFunctionRuns = options.singleFunctionRuns;
|
||||
@@ -92,6 +105,7 @@ export class KunaProvider implements AnalysisProviderCandidate {
|
||||
reason: availability.reason,
|
||||
code: availability.code,
|
||||
},
|
||||
this.#deadline,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -290,6 +304,7 @@ export class KunaProvider implements AnalysisProviderCandidate {
|
||||
cache: this.#projectCache(configuration.value),
|
||||
runSlots: this.#foregroundSlots(configuration.value),
|
||||
environment: this.#environment,
|
||||
deadline: this.#deadline,
|
||||
...(this.#launcher === undefined ? {} : { launcher: this.#launcher }),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -4,7 +4,11 @@ import type {
|
||||
} from "../application/AnalysisProvider.js";
|
||||
import type { ProviderRejectionCode } from "../contracts/providerSelection.js";
|
||||
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
|
||||
import { REGEX_MATCHING_LIMITATION } from "../process/RegexMatcher.js";
|
||||
import {
|
||||
DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
type RegexMatchingDeadline,
|
||||
} from "../config/regexMatchingDeadline.js";
|
||||
import { launchRegexMatchingLimitation } from "../process/RegexMatcher.js";
|
||||
import { KUNA_FUNCTION_TEXT_BYTES } from "./KunaLimits.js";
|
||||
|
||||
/** Static identity; the observed Kuna release is reported in each analysis profile. */
|
||||
@@ -62,6 +66,7 @@ const KUNA_TEXT_CAPACITY_LIMITATION = `Each buffered source facet admits at most
|
||||
/** The limitations that apply to one operation's answer, and only those. */
|
||||
export const kunaLimitations = (
|
||||
operation: KunaOperation,
|
||||
deadline: RegexMatchingDeadline = DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
): readonly string[] => {
|
||||
switch (operation) {
|
||||
case "list_documents":
|
||||
@@ -72,7 +77,7 @@ export const kunaLimitations = (
|
||||
return [
|
||||
KUNA_PROVENANCE,
|
||||
KUNA_STRING_LIMITATION,
|
||||
REGEX_MATCHING_LIMITATION,
|
||||
launchRegexMatchingLimitation(deadline),
|
||||
];
|
||||
case "list_procedures":
|
||||
case "procedure_address":
|
||||
@@ -82,7 +87,7 @@ export const kunaLimitations = (
|
||||
return [
|
||||
KUNA_PROVENANCE,
|
||||
KUNA_ENTRY_LIMITATION,
|
||||
REGEX_MATCHING_LIMITATION,
|
||||
launchRegexMatchingLimitation(deadline),
|
||||
];
|
||||
case "procedure_callers":
|
||||
case "procedure_callees":
|
||||
@@ -113,6 +118,7 @@ export const kunaCapabilities = (
|
||||
readonly reason: string;
|
||||
readonly code: ProviderRejectionCode;
|
||||
},
|
||||
deadline: RegexMatchingDeadline = DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
): readonly CapabilityDescriptor[] =>
|
||||
KUNA_OPERATIONS.map((operation) => ({
|
||||
provider: KUNA_PROVIDER_IDENTITY,
|
||||
@@ -136,5 +142,5 @@ export const kunaCapabilities = (
|
||||
changesPermissions: false,
|
||||
requiresRoot: false,
|
||||
},
|
||||
limitations: kunaLimitations(operation),
|
||||
limitations: kunaLimitations(operation, deadline),
|
||||
}));
|
||||
@@ -5,6 +5,10 @@ import {
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { err, type Result } from "../domain/result.js";
|
||||
import { SharedStartupAttempt } from "../process/SharedStartupAttempt.js";
|
||||
import {
|
||||
DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
type RegexMatchingDeadline,
|
||||
} from "../config/regexMatchingDeadline.js";
|
||||
import { matchRegexValues } from "../process/RegexMatcher.js";
|
||||
import type { KunaJsonDocumentReader } from "./KunaJsonStream.js";
|
||||
import { KUNA_DIAGNOSTIC_BYTES } from "./KunaLimits.js";
|
||||
@@ -36,6 +40,7 @@ export class KunaQueryWork {
|
||||
readonly own: <T>(work: Promise<T>) => Promise<T>;
|
||||
readonly runSlots: KunaRunSlots;
|
||||
},
|
||||
private readonly deadline: RegexMatchingDeadline = DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
) {}
|
||||
|
||||
/** Unproven process cleanup retains both leases in the native command owner. */
|
||||
@@ -132,6 +137,7 @@ export class KunaQueryWork {
|
||||
...input,
|
||||
values: input.values(),
|
||||
signal,
|
||||
deadline: this.deadline,
|
||||
});
|
||||
} finally {
|
||||
release();
|
||||
|
||||
@@ -70,6 +70,7 @@ import { KunaReferenceQueries } from "./KunaReferenceQueries.js";
|
||||
import { KunaQueryWork } from "./KunaQueryWork.js";
|
||||
import { KunaStringQueries } from "./KunaStringQueries.js";
|
||||
import { normalizeKunaAddress } from "./KunaAddress.js";
|
||||
import type { RegexMatchingDeadline } from "../config/regexMatchingDeadline.js";
|
||||
import { KUNA_FUNCTION_TEXT_BYTES } from "./KunaLimits.js";
|
||||
import type { KunaRunRelease, KunaRunSlots } from "./KunaRunSlots.js";
|
||||
import {
|
||||
@@ -251,6 +252,7 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
readonly #queryWork: KunaQueryWork;
|
||||
readonly #references: KunaReferenceQueries;
|
||||
readonly #strings: KunaStringQueries;
|
||||
readonly #deadline: RegexMatchingDeadline;
|
||||
#installation:
|
||||
| SharedStartupAttempt<Result<KunaInstallation, AnalysisError>>
|
||||
| undefined;
|
||||
@@ -282,6 +284,7 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
readonly cache: KunaProjectCache;
|
||||
readonly runSlots: KunaRunSlots;
|
||||
readonly environment: Readonly<NodeJS.ProcessEnv>;
|
||||
readonly deadline: RegexMatchingDeadline;
|
||||
readonly launcher?: KunaProcessLauncher;
|
||||
},
|
||||
) {
|
||||
@@ -297,11 +300,16 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
? {}
|
||||
: { launcher: dependencies.launcher }),
|
||||
});
|
||||
this.#queryWork = new KunaQueryWork(this.#commands, {
|
||||
signal: this.#session.signal,
|
||||
own: (work) => this.#own(work),
|
||||
runSlots: dependencies.runSlots,
|
||||
});
|
||||
this.#deadline = dependencies.deadline;
|
||||
this.#queryWork = new KunaQueryWork(
|
||||
this.#commands,
|
||||
{
|
||||
signal: this.#session.signal,
|
||||
own: (work) => this.#own(work),
|
||||
runSlots: dependencies.runSlots,
|
||||
},
|
||||
dependencies.deadline,
|
||||
);
|
||||
this.#references = new KunaReferenceQueries(this.#queryWork);
|
||||
this.#strings = new KunaStringQueries(this.#queryWork);
|
||||
}
|
||||
@@ -411,7 +419,7 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
);
|
||||
return ok(
|
||||
this.#execution(
|
||||
kunaLimitations(operation),
|
||||
kunaLimitations(operation, this.#deadline),
|
||||
installation.value,
|
||||
answer.value,
|
||||
),
|
||||
@@ -1263,7 +1271,7 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
cfg: unavailable("Kuna exports no basic blocks."),
|
||||
},
|
||||
limitations: [
|
||||
...kunaLimitations("analyze_function"),
|
||||
...kunaLimitations("analyze_function", this.#deadline),
|
||||
...pseudocode.value.limitations,
|
||||
KUNA_RELATIONSHIP_LIMITATION,
|
||||
...(relationships.ok
|
||||
|
||||
@@ -443,16 +443,18 @@ it("recovers launch exec argv from the command line, not process.execArgv", () =
|
||||
}
|
||||
});
|
||||
|
||||
it("describes the effective heap and keeps the deadline fixed", () => {
|
||||
it("describes the effective heap and the configurable matching deadline", () => {
|
||||
for (const heap of [
|
||||
effective(128, false),
|
||||
effective(256, true),
|
||||
{ kind: "unknown" } as const,
|
||||
]) {
|
||||
const limitation = regexMatchingLimitation(heap);
|
||||
expect(limitation).toContain(
|
||||
"This release has no runtime override for the deadline.",
|
||||
);
|
||||
expect(limitation).toContain("5000 ms matching deadline (source default)");
|
||||
expect(limitation).toContain("REA_REGEX_MATCHING_DEADLINE_MS");
|
||||
expect(limitation).toContain("floor 1 ms");
|
||||
expect(limitation).toContain("maximum 2147483647 ms");
|
||||
expect(limitation).not.toContain("no runtime override");
|
||||
expect(limitation).not.toContain("either cap");
|
||||
expect(limitation).not.toContain("no runtime override for the heap");
|
||||
const projected = projectAnalysisError(
|
||||
|
||||
+38
-15
@@ -11,8 +11,12 @@ import {
|
||||
AnalysisResourceConstraintError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import {
|
||||
DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
regexMatchingDeadlineRecovery,
|
||||
type RegexMatchingDeadline,
|
||||
} from "../config/regexMatchingDeadline.js";
|
||||
|
||||
const MATCHING_DEADLINE_MS = 5_000;
|
||||
const WORKER_HEAP_MIB = 128;
|
||||
/**
|
||||
* Node's Worker `resourceLimits.maxOldGenerationSizeMb` request. A positive
|
||||
@@ -393,30 +397,45 @@ const heapBound = (heap: RegexWorkerHeapLimit): string => {
|
||||
};
|
||||
|
||||
/** Capability text for the supervised ECMAScript matching boundary. */
|
||||
export const regexMatchingLimitation = (heap: RegexWorkerHeapLimit): string => {
|
||||
export const regexMatchingLimitation = (
|
||||
heap: RegexWorkerHeapLimit,
|
||||
deadline: RegexMatchingDeadline = DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
): string => {
|
||||
const deadlineText = `${String(deadline.deadlineMs)} ms matching deadline (source ${deadline.source})`;
|
||||
const preface =
|
||||
heap.kind === "effective"
|
||||
? `Regex mode uses ECMAScript Unicode semantics in a cancellable worker with a ${String(MATCHING_DEADLINE_MS / 1000)}-second matching deadline and an effective ${String(heap.mib)} MiB old-generation heap.`
|
||||
: `Regex mode uses ECMAScript Unicode semantics in a cancellable worker with a ${String(MATCHING_DEADLINE_MS / 1000)}-second matching deadline.`;
|
||||
return `${preface} This release has no runtime override for the deadline. ${heapBound(heap)} Matching failure leaves the service available.`;
|
||||
? `Regex mode uses ECMAScript Unicode semantics in a cancellable worker with a ${deadlineText} and an effective ${String(heap.mib)} MiB old-generation heap.`
|
||||
: `Regex mode uses ECMAScript Unicode semantics in a cancellable worker with a ${deadlineText}.`;
|
||||
return `${preface} ${regexMatchingDeadlineRecovery(deadline)} ${heapBound(heap)} Matching failure leaves the service available.`;
|
||||
};
|
||||
|
||||
/** Shared capability text for the supervised ECMAScript matching boundary. */
|
||||
export const REGEX_MATCHING_LIMITATION = regexMatchingLimitation(
|
||||
launchRegexWorkerHeap,
|
||||
);
|
||||
/** Capability text for this process's launch heap and one resolved deadline. */
|
||||
export const launchRegexMatchingLimitation = (
|
||||
deadline: RegexMatchingDeadline = DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
): string => regexMatchingLimitation(launchRegexWorkerHeap, deadline);
|
||||
|
||||
/** Fixed regex deadline. The worker heap is a separate, launch-flag-sensitive cap. */
|
||||
/**
|
||||
* Capability text for the default deadline on this process's launch heap.
|
||||
* Providers that resolved `REA_REGEX_MATCHING_DEADLINE_MS` pass that value to
|
||||
* `launchRegexMatchingLimitation` instead of this constant.
|
||||
*/
|
||||
export const REGEX_MATCHING_LIMITATION = launchRegexMatchingLimitation();
|
||||
|
||||
/** Supervised regex deadline. The worker heap is a separate, launch-flag-sensitive cap. */
|
||||
export const regexMatchingDeadlineError = (
|
||||
operation: string,
|
||||
deadline: RegexMatchingDeadline = DEFAULT_REGEX_MATCHING_DEADLINE,
|
||||
): AnalysisResourceConstraintError =>
|
||||
new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
"cpu",
|
||||
`Regex matching exceeded its supervised deadline of ${String(MATCHING_DEADLINE_MS)} ms. This release has no runtime override. The service remains available.`,
|
||||
{ matching_deadline_ms: MATCHING_DEADLINE_MS },
|
||||
`Regex matching exceeded its supervised deadline of ${String(deadline.deadlineMs)} ms (source ${deadline.source}). ${regexMatchingDeadlineRecovery(deadline)} The service remains available.`,
|
||||
{
|
||||
remediationAction: `The regex matching deadline is ${String(MATCHING_DEADLINE_MS)} ms and this release has no runtime override. Simplify the ECMAScript Unicode regex or use literal mode, then retry in the same session.`,
|
||||
matching_deadline_ms: deadline.deadlineMs,
|
||||
matching_deadline_source: deadline.source,
|
||||
},
|
||||
{
|
||||
remediationAction: `${regexMatchingDeadlineRecovery(deadline)} Simplify the ECMAScript Unicode regex or use literal mode, then retry in the same session.`,
|
||||
},
|
||||
);
|
||||
|
||||
@@ -478,7 +497,10 @@ export const matchRegexValues = async (input: {
|
||||
readonly values: readonly string[];
|
||||
readonly signal?: AbortSignal | undefined;
|
||||
readonly invalidPattern?: (reason: string) => AnalysisError;
|
||||
/** Resolved deadline. Omitted means the 5000 ms default, not ambient `process.env`. */
|
||||
readonly deadline?: RegexMatchingDeadline | undefined;
|
||||
}): Promise<Result<readonly number[], AnalysisError>> => {
|
||||
const deadline = input.deadline ?? DEFAULT_REGEX_MATCHING_DEADLINE;
|
||||
if (input.signal?.aborted)
|
||||
return err(new AnalysisCancelledError(input.operation));
|
||||
let worker: Worker;
|
||||
@@ -512,8 +534,9 @@ export const matchRegexValues = async (input: {
|
||||
const onAbort = () =>
|
||||
finish(err(new AnalysisCancelledError(input.operation)));
|
||||
const timer = setTimeout(
|
||||
() => finish(err(regexMatchingDeadlineError(input.operation))),
|
||||
MATCHING_DEADLINE_MS,
|
||||
() =>
|
||||
finish(err(regexMatchingDeadlineError(input.operation, deadline))),
|
||||
deadline.deadlineMs,
|
||||
);
|
||||
input.signal?.addEventListener("abort", onAbort, { once: true });
|
||||
if (input.signal?.aborted) onAbort();
|
||||
|
||||
@@ -185,6 +185,7 @@ it.each([
|
||||
REA_GHIDRA_STARTUP_TIMEOUT_MS: "2700000",
|
||||
REA_HOPPER_STARTUP_TIMEOUT_MS: "120000",
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
||||
REA_REGEX_MATCHING_DEADLINE_MS: "10000",
|
||||
},
|
||||
providerConfigured: false,
|
||||
installHopper: true,
|
||||
@@ -217,7 +218,12 @@ it.each([
|
||||
REA_GHIDRA_STARTUP_TIMEOUT_MS: "2700000",
|
||||
REA_HOPPER_STARTUP_TIMEOUT_MS: "120000",
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
||||
REA_REGEX_MATCHING_DEADLINE_MS: "10000",
|
||||
});
|
||||
} else {
|
||||
expect(initial.providerEnvironment).not.toHaveProperty(
|
||||
"REA_REGEX_MATCHING_DEADLINE_MS",
|
||||
);
|
||||
}
|
||||
const discovery = await discoverSetupState({
|
||||
host: setupHost,
|
||||
|
||||
@@ -486,6 +486,7 @@ cliTest(
|
||||
REA_HOPPER_STARTUP_TIMEOUT_MS: "2800002",
|
||||
REA_JEB_MCP_TIMEOUT_MS: "2900003",
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
||||
REA_REGEX_MATCHING_DEADLINE_MS: "10000",
|
||||
REA_UNRELATED_MARKER: "do-not-persist",
|
||||
};
|
||||
const run = (arguments_: readonly string[]) =>
|
||||
@@ -514,6 +515,7 @@ cliTest(
|
||||
);
|
||||
expect(codexRegistration).toMatchObject({ tool_timeout_sec: 3021 });
|
||||
expect(codexConfig).toContain('REA_JEB_MCP_TIMEOUT_MS = "2900003"');
|
||||
expect(codexConfig).toContain('REA_REGEX_MATCHING_DEADLINE_MS = "10000"');
|
||||
expect(codexConfig).not.toContain("REA_UNRELATED_MARKER");
|
||||
|
||||
const kimiPath = join(kimiHome, "mcp.json");
|
||||
@@ -529,6 +531,7 @@ cliTest(
|
||||
REA_HOPPER_STARTUP_TIMEOUT_MS: "2800002",
|
||||
REA_JEB_MCP_TIMEOUT_MS: "2900003",
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
|
||||
REA_REGEX_MATCHING_DEADLINE_MS: "10000",
|
||||
},
|
||||
});
|
||||
expect(await readFile(kimiPath, "utf8")).not.toContain(
|
||||
|
||||
@@ -491,6 +491,11 @@ it.each([
|
||||
{ REA_ASAR_MAX_HEADER_BYTES: "268435457" },
|
||||
"REA_ASAR_MAX_HEADER_BYTES",
|
||||
],
|
||||
[
|
||||
"regex matching deadline of zero",
|
||||
{ REA_REGEX_MATCHING_DEADLINE_MS: "0" },
|
||||
"REA_REGEX_MATCHING_DEADLINE_MS",
|
||||
],
|
||||
[
|
||||
"malformed loader JSON",
|
||||
{ HOPPER_LOADER_ARGS_JSON: "{" },
|
||||
|
||||
@@ -5,6 +5,11 @@ import { join } from "node:path";
|
||||
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
DEFAULT_REGEX_MATCHING_DEADLINE_MS,
|
||||
MAXIMUM_REGEX_MATCHING_DEADLINE_MS,
|
||||
REGEX_MATCHING_DEADLINE_SETTING,
|
||||
} from "../../../src/config/regexMatchingDeadline.js";
|
||||
import { ok } from "../../../src/domain/result.js";
|
||||
import { HopperRegexSearch } from "../../../src/hopper/HopperRegexSearch.js";
|
||||
import { OFFICIAL_TOOL_CONTRACTS } from "../../../src/contracts/officialToolContracts.js";
|
||||
@@ -63,16 +68,19 @@ const probeWorkerHeap = (
|
||||
};
|
||||
};
|
||||
|
||||
it("names the fixed regex deadline and the effective worker heap", () => {
|
||||
it("names the configurable regex deadline and the effective worker heap", () => {
|
||||
const heap = resolveLaunchRegexWorkerHeap(
|
||||
readLaunchExecArgv(),
|
||||
readLaunchNodeOptions(),
|
||||
);
|
||||
expect(REGEX_MATCHING_LIMITATION).toBe(regexMatchingLimitation(heap));
|
||||
expect(REGEX_MATCHING_LIMITATION).toContain("5-second matching deadline");
|
||||
expect(REGEX_MATCHING_LIMITATION).toContain(
|
||||
"This release has no runtime override for the deadline.",
|
||||
"5000 ms matching deadline (source default)",
|
||||
);
|
||||
expect(REGEX_MATCHING_LIMITATION).toContain("REA_REGEX_MATCHING_DEADLINE_MS");
|
||||
expect(REGEX_MATCHING_LIMITATION).toContain("floor 1 ms");
|
||||
expect(REGEX_MATCHING_LIMITATION).toContain("maximum 2147483647 ms");
|
||||
expect(REGEX_MATCHING_LIMITATION).not.toContain("no runtime override");
|
||||
expect(REGEX_MATCHING_LIMITATION).not.toContain("either cap");
|
||||
expect(REGEX_MATCHING_LIMITATION).not.toContain("or heap");
|
||||
if (heap.kind === "unknown") {
|
||||
@@ -86,9 +94,14 @@ it("names the fixed regex deadline and the effective worker heap", () => {
|
||||
regexMatchingDeadlineError("search_procedures"),
|
||||
);
|
||||
expect(deadline.message).toContain("5000 ms");
|
||||
expect(deadline.message).toContain("source default");
|
||||
expect(deadline.message).toContain("REA_REGEX_MATCHING_DEADLINE_MS");
|
||||
expect(deadline.message).toContain("maximum 2147483647 ms");
|
||||
expect(deadline.remediation.action).toContain(
|
||||
"this release has no runtime override",
|
||||
"REA_REGEX_MATCHING_DEADLINE_MS",
|
||||
);
|
||||
expect(deadline.remediation.action).toContain("source default");
|
||||
expect(deadline.remediation.action).not.toContain("no runtime override");
|
||||
const failure = projectAnalysisError(
|
||||
regexWorkerHeapError("search_procedures", new Error("heap")),
|
||||
);
|
||||
@@ -112,8 +125,13 @@ it("names the fixed regex deadline and the effective worker heap", () => {
|
||||
const description = OFFICIAL_TOOL_CONTRACTS.find(
|
||||
(tool) => tool.name === name,
|
||||
)?.description;
|
||||
expect(description).toContain("5000 ms");
|
||||
expect(description).toContain("no runtime override");
|
||||
expect(description).toContain(REGEX_MATCHING_DEADLINE_SETTING);
|
||||
expect(description).toContain(
|
||||
`default ${String(DEFAULT_REGEX_MATCHING_DEADLINE_MS)}`,
|
||||
);
|
||||
expect(description).toContain(String(MAXIMUM_REGEX_MATCHING_DEADLINE_MS));
|
||||
expect(description).toContain("source (default or setting)");
|
||||
expect(description).not.toContain("no runtime override");
|
||||
}
|
||||
});
|
||||
|
||||
@@ -411,6 +429,59 @@ it("leaves the heap unknown when --title overwrites the command line", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("arms the worker timer with an overridden matching deadline", async () => {
|
||||
const delays: number[] = [];
|
||||
const original = global.setTimeout;
|
||||
global.setTimeout = ((handler, timeout, ...args) => {
|
||||
if (typeof timeout === "number") delays.push(timeout);
|
||||
return original(handler, timeout, ...args);
|
||||
}) as typeof setTimeout;
|
||||
try {
|
||||
expect(
|
||||
await matchRegexValues({
|
||||
operation: "search_procedures",
|
||||
pattern: "^a",
|
||||
caseSensitive: true,
|
||||
values: ["a"],
|
||||
deadline: { deadlineMs: 10_000, source: "setting" },
|
||||
}),
|
||||
).toEqual(ok([0]));
|
||||
expect(delays).toContain(10_000);
|
||||
expect(delays).not.toContain(5_000);
|
||||
} finally {
|
||||
global.setTimeout = original;
|
||||
}
|
||||
});
|
||||
|
||||
it("stops a slow match at the configured deadline and names the setting", async () => {
|
||||
const started = performance.now();
|
||||
const matched = await matchRegexValues({
|
||||
operation: "search_procedures",
|
||||
pattern: "^(a+)+$",
|
||||
caseSensitive: true,
|
||||
values: [`${"a".repeat(30)}!`],
|
||||
deadline: { deadlineMs: 100, source: "setting" },
|
||||
});
|
||||
expect(performance.now() - started).toBeLessThan(4_000);
|
||||
expect(matched.ok).toBe(false);
|
||||
if (matched.ok) return;
|
||||
const projected = projectAnalysisError(matched.error);
|
||||
expect(projected.message).toContain("100 ms (source setting)");
|
||||
expect(projected.message).toContain("REA_REGEX_MATCHING_DEADLINE_MS");
|
||||
expect(projected.message).toContain("default 5000 ms");
|
||||
expect(projected.message).toContain("maximum 2147483647 ms");
|
||||
expect(projected.remediation.action).toContain(
|
||||
"Set REA_REGEX_MATCHING_DEADLINE_MS in the environment that starts REA",
|
||||
);
|
||||
expect(projected.remediation.action).not.toContain("no runtime override");
|
||||
expect(matched.error).toMatchObject({
|
||||
reportedLimits: {
|
||||
matching_deadline_ms: 100,
|
||||
matching_deadline_source: "setting",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("preserves Unicode matching order and refuses a pre-cancelled match", async () => {
|
||||
expect(
|
||||
await matchRegexValues({
|
||||
|
||||
@@ -45,7 +45,20 @@ try {
|
||||
const matched = await client.execute(${JSON.stringify(operation)}, {pattern:"^(a+)+$",mode:"regex"}, options);
|
||||
assert.equal(matched.ok, false);
|
||||
assert.equal(matched.error._tag, ${JSON.stringify(mode !== "deadline" ? "AnalysisCancelledError" : "AnalysisResourceConstraintError")});
|
||||
if (${JSON.stringify(mode)} === "deadline") assert.equal(matched.error.reportedLimits.matching_deadline_ms, 5000);
|
||||
if (${JSON.stringify(mode)} === "deadline") {
|
||||
assert.equal(matched.error.reportedLimits.matching_deadline_ms, 5000);
|
||||
assert.equal(matched.error.reportedLimits.matching_deadline_source, "default");
|
||||
assert.match(matched.error.reason, /5000 ms \\(source default\\)/);
|
||||
assert.match(matched.error.reason, /REA_REGEX_MATCHING_DEADLINE_MS/);
|
||||
assert.match(matched.error.reason, /maximum 2147483647 ms/);
|
||||
assert.match(matched.error.remediationAction, /Set REA_REGEX_MATCHING_DEADLINE_MS in the environment that starts REA/);
|
||||
assert.equal(matched.error.reason.includes("no runtime override"), false);
|
||||
assert.equal(matched.error.remediationAction.includes("no runtime override"), false);
|
||||
const limitation = provider.capabilities().find((capability) => capability.operation === ${JSON.stringify(operation)}).limitations.join(" ");
|
||||
assert.match(limitation, /5000 ms matching deadline \\(source default\\)/);
|
||||
assert.match(limitation, /REA_REGEX_MATCHING_DEADLINE_MS/);
|
||||
assert.equal(limitation.includes("no runtime override"), false);
|
||||
}
|
||||
if (${JSON.stringify(mode)} === "close") {
|
||||
assert.equal((await closing).ok, true);
|
||||
assert.equal((await client.execute("health", {})).ok, false);
|
||||
@@ -81,3 +94,61 @@ console.log("bounded and healthy");
|
||||
expect(result.stdout.trim()).toBe("bounded and healthy");
|
||||
},
|
||||
);
|
||||
|
||||
it("uses REA_REGEX_MATCHING_DEADLINE_MS from the selected environment", async () => {
|
||||
const { kuna, target, cacheRoot, profile } = await session(
|
||||
"rea-kuna-regex-deadline-",
|
||||
{
|
||||
functions: [
|
||||
{ name: `${"a".repeat(40)}!`, addr: "0x1000" },
|
||||
{ name: "Δelta", addr: "0x1040" },
|
||||
],
|
||||
},
|
||||
);
|
||||
const script = `
|
||||
import assert from "node:assert/strict";
|
||||
import { KunaProvider } from ${JSON.stringify(pathToFileURL(resolve("dist/kuna/KunaProvider.js")).href)};
|
||||
process.env.REA_REGEX_MATCHING_DEADLINE_MS = "10000";
|
||||
const provider = new KunaProvider(${JSON.stringify({
|
||||
REA_KUNA_COMMAND: kuna,
|
||||
REA_KUNA_CACHE_DIR: cacheRoot,
|
||||
REA_REGEX_MATCHING_DEADLINE_MS: "200",
|
||||
})});
|
||||
const limitation = provider.capabilities().find((capability) => capability.operation === "search_procedures").limitations.join(" ");
|
||||
assert.match(limitation, /200 ms matching deadline \\(source setting\\)/);
|
||||
assert.match(limitation, /REA_REGEX_MATCHING_DEADLINE_MS/);
|
||||
assert.match(limitation, /default 5000 ms/);
|
||||
assert.match(limitation, /maximum 2147483647 ms/);
|
||||
assert.equal(limitation.includes("no runtime override"), false);
|
||||
const client = provider.createClient(${JSON.stringify(target)}, ${JSON.stringify(profile)});
|
||||
try {
|
||||
assert.equal((await client.execute("procedure_assembly", {procedure:"0x1000"})).ok, true);
|
||||
const started = performance.now();
|
||||
const matched = await client.execute("search_procedures", {pattern:"^(a+)+$",mode:"regex"});
|
||||
assert.ok(performance.now() - started < 4000, "deadline stayed at the 5000 ms default");
|
||||
assert.equal(matched.ok, false);
|
||||
assert.equal(matched.error._tag, "AnalysisResourceConstraintError");
|
||||
assert.equal(matched.error.reportedLimits.matching_deadline_ms, 200);
|
||||
assert.equal(matched.error.reportedLimits.matching_deadline_source, "setting");
|
||||
assert.match(matched.error.reason, /200 ms \\(source setting\\)/);
|
||||
assert.match(matched.error.remediationAction, /REA_REGEX_MATCHING_DEADLINE_MS/);
|
||||
assert.equal(matched.error.reason.includes("no runtime override"), false);
|
||||
} finally {
|
||||
assert.equal((await client.close()).ok, true);
|
||||
}
|
||||
console.log("override applied");
|
||||
`;
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
["--input-type=module", "-e", script],
|
||||
{
|
||||
encoding: "utf8",
|
||||
timeout: 25_000,
|
||||
killSignal: "SIGKILL",
|
||||
},
|
||||
);
|
||||
expect(result.error).toBeUndefined();
|
||||
expect(result.stderr).toBe("");
|
||||
expect(result.status).toBe(0);
|
||||
expect(result.stdout.trim()).toBe("override applied");
|
||||
});
|
||||
Reference in new issue
Block a user