fix(process): report the effective regex heap and uncertain signal retry (#6099)

* fix(process): report the effective regex worker heap

Node gives --max-old-space-size, including NODE_OPTIONS, precedence over
the worker's 128 MiB resourceLimits request. Name that effective
old-generation heap and stop saying the request has no runtime override.
The matching deadline stays a fixed cap.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): keep SIGXCPU and SIGXFSZ retries uncertain

The signal does not prove the retained rlimit was exhausted, so an
unchanged retry may succeed. Project that retry as possible instead of
saying it fails the same way.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): leave percentage heap overrides unestablished

--max-old-space-size-percentage takes precedence over the absolute flag
in either order. Node computes that MiB from startup memory, which a
later read does not recover, so the effective heap stays unknown instead
of the 128 MiB request or an absolute flag.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): read launch NODE_OPTIONS, not process.env

A preload can rewrite process.env.NODE_OPTIONS after V8 has applied the
original flags, and those flags are not copied into execArgv. Use the
initial environment block when it can be read, and leave the effective
heap unestablished when that provenance cannot be recovered.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): leave env-file heap overrides unestablished

--env-file and --env-file-if-exists can supply NODE_OPTIONS when that
variable was absent from the initial environment. The file is not in
/proc/self/environ, so a missing entry is not the 128 MiB request.
An initial NODE_OPTIONS is still parsed, because Node does not let the
file replace it.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): leave --max-heap-size old generation unestablished

A positive --max-heap-size or --max_heap_size sets the total heap, not the
old generation, and can stay below the 128 MiB worker request when an
old-space flag is also present. The effective old generation stays
unestablished while that value is in effect. Zero or empty clears it.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): leave config-file heap options unestablished

--experimental-config-file and --experimental-default-config-file can
apply a heap percentage from a file that is not launch provenance. The
effective old generation stays unestablished when either flag is present,
including beside an absolute --max-old-space-size.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): read launch exec argv from the command line

process.execArgv can be cleared after V8 applies --max-old-space-size.
Linux recovers those tokens from /proc/self/cmdline and leaves the heap
unestablished when that vector cannot be separated from the program entry.
Heap probes compare pinned absolute limits instead of a percentage of host memory.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): leave a title-overwritten command line unestablished

process.title and --title replace /proc/self/cmdline with the title.
A buffer without a program entry is not an empty option list, so the
heap stays unknown instead of the 128 MiB request. The command-line
probe does not read /proc off Linux.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
This commit is contained in:
authored and GitHub committed 2026-10-22 07:40:14 +08:00
1 parent 603a7d022c
commit 8413ec08f7
7 files changed
+1364 -22

No files matched your search

+1 -1
View File
@@ -68,7 +68,7 @@ The background export belongs to the session. `close_binary`, or the end of a CL
The assembly label index is scanned once per export reader. Cancelling a request stops its wait without interrupting other readers; closing the session aborts and drains the shared scan before releasing its files.
Regex procedure and string search uses ECMAScript Unicode matching in a cancellable worker with a five-second matching deadline and a 128 MiB worker heap. Literal matching remains the default. Cancellation or a matching deadline leaves the session available. This release has no runtime override for that deadline or heap.
Regex procedure and string search uses ECMAScript Unicode matching in a cancellable worker with a five-second matching deadline and a worker old-generation heap. Literal matching remains the default. Cancellation or a matching deadline leaves the session available. This release has no runtime override for that deadline. The worker requests 128 MiB. A positive Node.js `--max-old-space-size` on the process that starts REA, including `NODE_OPTIONS`, overrides that request and sets the effective heap. `--max-old-space-size-percentage` takes precedence over that flag in either order. REA does not turn the percentage into a MiB, so the effective heap stays unestablished instead of the 128 MiB request or the absolute flag. The reported `NODE_OPTIONS` value is the process initial environment when that block can be read. A later edit to the environment is not the value Node applied, and an unreadable initial value leaves the effective heap unestablished. The launch Node options are the initial command line on Linux when that buffer still has a program entry, not a later edit to `process.execArgv`. A preload can clear that array after V8 has applied the flags. Setting `process.title` or `--title` overwrites the command line, so a buffer without a program entry leaves the effective heap unestablished. `--env-file` or `--env-file-if-exists` can supply `NODE_OPTIONS` when the initial environment has none. That file is not part of the initial environment block, so the effective heap stays unestablished instead of the 128 MiB request. An initial `NODE_OPTIONS` is still used, because Node does not let the file replace it. A positive `--max-heap-size`, including the underscore form and launch `NODE_OPTIONS` on Node versions that allow it, sets the total heap rather than the old generation and can be lower than the 128 MiB request. The effective old generation stays unestablished while that value is in effect. A zero or empty value clears it. `--experimental-config-file` or `--experimental-default-config-file` can set `nodeOptions`, including a heap percentage, from a file that is not read back. The effective heap stays unestablished when either flag is present, including when an absolute `--max-old-space-size` is also on the command line or in the recovered `NODE_OPTIONS`. Node accepts the config path as `=` or the next argument.
Single-function subprocess output and each cached source facet share a 16 MiB buffered-text capacity. Cached spans are checked before allocation; larger text remains in the local export and the error identifies the file and range. `analyze_function` initially returns the complete dossier, which can still be large; later retained views do not remove that first transfer. Prefer the individual text tools for a focused question.
+31 -3
View File
@@ -202,10 +202,38 @@ Function dossiers retain this same string evidence. Native call edges retain
Hopper's partial `CallReference` classification and exact endpoints; detailed
reference flags remain unavailable rather than being invented.
Regex searches use ECMAScript Unicode syntax in a cancellable worker with a
five-second matching deadline and a 128 MiB worker heap. Deadline or
five-second matching deadline and a worker old-generation heap. Deadline or
cancellation stops matching while leaving the Hopper API available. This
release has no runtime override for that deadline or heap. Literal mode
retains Unicode casefold matching.
release has no runtime override for that deadline. The worker requests 128
MiB. A positive Node.js `--max-old-space-size` on the process that starts
REA, including `NODE_OPTIONS`, overrides that request and sets the effective
heap. `--max-old-space-size-percentage` takes precedence over that flag in
either order. REA does not turn the percentage into a MiB, so the effective
heap stays unestablished instead of the 128 MiB request or the absolute flag.
The reported `NODE_OPTIONS` value is the process initial environment when that
block can be read. A later edit to the environment is not the value Node
applied, and an unreadable initial value leaves the effective heap
unestablished. The launch Node options are the initial command line on Linux
when that buffer still has a program entry, not a later edit to
`process.execArgv`. A preload can clear that array after V8 has applied the
flags. Setting `process.title` or `--title` overwrites the command line, so a
buffer without a program entry leaves the effective heap unestablished.
`--env-file` or `--env-file-if-exists` can supply `NODE_OPTIONS`
when the initial environment has none. That file is not part of the initial
environment block, so the effective heap stays unestablished instead of the
128 MiB request. An initial `NODE_OPTIONS` is still used, because Node does
not let the file replace it. A positive `--max-heap-size`, including the
underscore form and launch `NODE_OPTIONS` on Node versions that allow it,
sets the total heap rather than the old generation and can be lower than the
128 MiB request. The effective old generation stays unestablished while that
value is in effect. A zero or empty value clears it.
`--experimental-config-file` or `--experimental-default-config-file` can set
`nodeOptions`, including a heap percentage, from a file that is not read
back. The effective heap stays unestablished when either flag is present,
including when an absolute `--max-old-space-size` is also on the command
line or in the recovered `NODE_OPTIONS`. Node accepts the config path as
`=` or the next argument.
Literal mode retains Unicode casefold matching.
## Bounded static traces
@@ -554,6 +554,13 @@ it("says REA retains the tighter inherited limit when a producer signal fires",
expect(projected.remediation.action).toContain(
"This release has no runtime override",
);
expect(projected.remediation.action).toContain(
"an unchanged retry may succeed or may fail the same way",
);
expect(projected.remediation.action).not.toContain(
"Repeating the unchanged request fails the same way.",
);
expect(projected.retryable).toBe(true);
expect(projected.details).toMatchObject({
reported_limits: { cpu_seconds: 30 },
});
@@ -562,6 +569,48 @@ it("says REA retains the tighter inherited limit when a producer signal fires",
}
});
it("keeps an unchanged SIGXFSZ retry uncertain when the inherited limit was not reported", async () => {
const resources = new ArtifactAnalysisResources(
"fixture",
"inspect_packet_fields",
readStableArtifact,
readLinuxResourceLimits,
{ message: "busy", details: {} },
);
const empty = { text: "", bytes: 0, observedBytes: 0 };
const failure = new OwnedCommandFailure("process", "producer signal", {
stdout: empty,
stderr: empty,
exitCode: null,
signal: "SIGXFSZ",
});
try {
const projected = projectAnalysisError(
resources.commandFailure(
failure,
{ command_timeout_ms: 30_000, output_bytes: 4_194_304 },
undefined,
),
);
expect(projected.remediation.action).toContain(
"The effective inherited limit was not reported.",
);
expect(projected.remediation.action).toContain(
"an unchanged retry may succeed or may fail the same way",
);
expect(projected.remediation.action).not.toContain(
"Repeating the unchanged request fails the same way.",
);
expect(projected.retryable).toBe(true);
expect(projected.details).toMatchObject({
resource: "file-size",
reported_limits: null,
});
} finally {
await resources.close();
}
});
it("treats a fixed command-output cap as a non-retryable capture allowance", async () => {
const resources = new ArtifactAnalysisResources(
"fixture",
+4 -1
View File
@@ -381,6 +381,8 @@ export class ArtifactAnalysisResources {
effectiveLimits === undefined
? "The effective inherited limit was not reported."
: `The effective inherited limits are ${String(effectiveLimits.address_space_bytes)} bytes of address space, ${String(effectiveLimits.file_bytes)} bytes per file, and ${String(effectiveLimits.cpu_seconds)} CPU seconds.`;
// A producer can raise SIGXCPU/SIGXFSZ without exhausting the retained
// rlimit, so an unchanged retry is not a proven failure.
return new AnalysisResourceConstraintError(
operation,
signal === "SIGXFSZ" ? "file-size" : "cpu",
@@ -388,7 +390,8 @@ export class ArtifactAnalysisResources {
effectiveLimits === undefined ? null : { ...effectiveLimits },
{
...detail,
remediationAction: `${reported} REA retains the tighter inherited limit. This release has no runtime override for that ceiling. A smaller artifact may finish inside the retained limit. Repeating the unchanged request fails the same way.`,
retryable: true,
remediationAction: `${reported} REA retains the tighter inherited limit. This release has no runtime override for that ceiling. A smaller artifact may finish inside the retained limit. The signal does not prove that ceiling was exhausted, so an unchanged retry may succeed or may fail the same way.`,
},
);
}
+493
View File
@@ -0,0 +1,493 @@
import { expect, it } from "vitest";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import {
execArgvFromCommandLine,
readLaunchExecArgv,
readLaunchNodeOptions,
regexMatchingLimitation,
regexWorkerHeapError,
resolveLaunchRegexWorkerHeap,
resolveRegexWorkerHeap,
type RegexWorkerHeapLimit,
} from "./RegexMatcher.js";
const effective = (mib: number, overridden: boolean): RegexWorkerHeapLimit => ({
kind: "effective",
mib,
overridden,
});
it.each([
{
name: "uses the 128 MiB request when no flag is set",
execArgv: [],
nodeOptions: undefined,
heap: effective(128, false),
},
{
name: "lets a smaller execArgv flag override the request",
execArgv: ["--max-old-space-size=64"],
nodeOptions: undefined,
heap: effective(64, true),
},
{
name: "lets a larger execArgv flag override the request",
execArgv: ["--require", "./preload.js", "--max_old_space_size=256"],
nodeOptions: undefined,
heap: effective(256, true),
},
{
name: "reads a positive NODE_OPTIONS flag that is absent from execArgv",
execArgv: [],
nodeOptions: "--no-warnings --max-old-space-size=192",
heap: effective(192, true),
},
{
name: "lets the last execArgv flag win over NODE_OPTIONS",
execArgv: ["--max-old-space-size=256"],
nodeOptions: "--max-old-space-size=96",
heap: effective(256, true),
},
{
name: "treats a final zero flag as leaving the 128 MiB request in effect",
execArgv: ["--max-old-space-size=0"],
nodeOptions: "--max-old-space-size=256",
heap: effective(128, false),
},
{
name: "accepts quoted NODE_OPTIONS and a leading plus or zeros",
execArgv: ["--max-old-space-size=+0256"],
nodeOptions: '"--max-old-space-size=144"',
heap: effective(256, true),
},
{
name: "uses the last positive flag",
execArgv: ["--max-old-space-size=256", "--max-old-space-size=96"],
nodeOptions: undefined,
heap: effective(96, true),
},
{
name: "leaves the effective heap unknown for an unparsed flag",
execArgv: ["--max-old-space-size=256.5"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "keeps a later positive flag after an unparsed one",
execArgv: ["--max-old-space-size=256.5", "--max-old-space-size=80"],
nodeOptions: undefined,
heap: effective(80, true),
},
{
name: "leaves an untokenizable NODE_OPTIONS value unknown until execArgv decides",
execArgv: [],
nodeOptions: '"--max-old-space-size=256',
heap: { kind: "unknown" } as const,
},
{
name: "lets execArgv decide when NODE_OPTIONS cannot be tokenized",
execArgv: ["--max-old-space-size=80"],
nodeOptions: '"--max-old-space-size=256',
heap: effective(80, true),
},
{
name: "leaves a percentage flag unestablished instead of the 128 MiB request",
execArgv: ["--max-old-space-size-percentage=50"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "lets a percentage flag win over a later absolute flag",
execArgv: ["--max-old-space-size-percentage=50", "--max-old-space-size=64"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "lets a percentage flag win over an earlier absolute flag",
execArgv: ["--max-old-space-size=64", "--max-old-space-size-percentage=50"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "lets a NODE_OPTIONS percentage win over an execArgv absolute flag",
execArgv: ["--max-old-space-size=64"],
nodeOptions: "--max-old-space-size-percentage=50",
heap: { kind: "unknown" } as const,
},
{
name: "lets an execArgv percentage win over a NODE_OPTIONS absolute flag",
execArgv: ["--max-old-space-size-percentage=50"],
nodeOptions: "--max-old-space-size=64",
heap: { kind: "unknown" } as const,
},
{
name: "recognizes an underscore percentage flag",
execArgv: ["--max_old_space_size_percentage=50"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "recognizes a percentage flag split from its value",
execArgv: [
"--max-old-space-size=64",
"--max-old-space-size-percentage",
"50",
],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "does not let a later absolute flag replace an invalid percentage",
execArgv: ["--max-old-space-size-percentage=0", "--max-old-space-size=64"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "leaves a fractional percentage unestablished",
execArgv: ["--max-old-space-size-percentage=50.5"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "does not treat a final zero absolute flag as 128 MiB after a percentage",
execArgv: ["--max-old-space-size-percentage=50", "--max-old-space-size=0"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "leaves an env-file launch unestablished when NODE_OPTIONS was absent",
execArgv: ["--env-file=rea.env"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "leaves an env-file-if-exists launch unestablished when NODE_OPTIONS was absent",
execArgv: ["--env-file-if-exists=.env"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "does not let an absolute flag hide NODE_OPTIONS supplied by an env file",
execArgv: ["--env-file=rea.env", "--max-old-space-size=64"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "recognizes an env-file flag split from its path",
execArgv: ["--env-file", "rea.env"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "keeps an inherited NODE_OPTIONS value when an env file cannot override it",
execArgv: ["--env-file=rea.env"],
nodeOptions: "--max-old-space-size=64",
heap: effective(64, true),
},
{
name: "lets a later absolute flag win when an inherited NODE_OPTIONS blocks the env file",
execArgv: ["--max-old-space-size=32", "--env-file-if-exists=rea.env"],
nodeOptions: "--max-old-space-size=64",
heap: effective(32, true),
},
{
name: "keeps the 128 MiB request when an empty NODE_OPTIONS blocks the env file",
execArgv: ["--env-file=rea.env"],
nodeOptions: "",
heap: effective(128, false),
},
{
name: "leaves a config-file percentage source unestablished",
execArgv: ["--experimental-config-file=node.config.json"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "leaves a default config-file launch unestablished",
execArgv: ["--experimental-default-config-file"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "recognizes a config-file flag split from its path",
execArgv: ["--experimental-config-file", "node.config.json"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "does not let an absolute flag hide a config-file heap option",
execArgv: [
"--experimental-config-file=node.config.json",
"--max-old-space-size=64",
],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "does not let a recovered NODE_OPTIONS hide a config-file heap option",
execArgv: ["--experimental-default-config-file"],
nodeOptions: "--max-old-space-size=64",
heap: { kind: "unknown" } as const,
},
{
name: "leaves a config-file flag in recovered NODE_OPTIONS unestablished",
execArgv: ["--max-old-space-size=256"],
nodeOptions: "--experimental-config-file=node.config.json",
heap: { kind: "unknown" } as const,
},
{
name: "leaves a positive --max-heap-size unestablished instead of the 128 MiB request",
execArgv: ["--max-heap-size=64"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "recognizes an underscore --max-heap-size",
execArgv: ["--max_heap_size=+064"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "leaves a NODE_OPTIONS --max-heap-size unestablished",
execArgv: [],
nodeOptions: "--max-heap-size=64",
heap: { kind: "unknown" } as const,
},
{
name: "does not let an old-space flag hide a positive --max-heap-size",
execArgv: ["--max-heap-size=64", "--max-old-space-size=256"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "does not let a later --max-heap-size hide an old-space flag",
execArgv: ["--max-old-space-size=64"],
nodeOptions: "--max_heap_size=256",
heap: { kind: "unknown" } as const,
},
{
name: "lets a final zero --max-heap-size clear the total-heap cap",
execArgv: ["--max-heap-size=64", "--max-heap-size=0"],
nodeOptions: undefined,
heap: effective(128, false),
},
{
name: "keeps an old-space flag when a final --max-heap-size is zero",
execArgv: ["--max-old-space-size=256", "--max-heap-size="],
nodeOptions: undefined,
heap: effective(256, true),
},
{
name: "leaves an unparsed --max-heap-size unestablished",
execArgv: ["--max-heap-size=64.5", "--max-old-space-size=80"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
{
name: "does not let a later zero clear an unparsed --max-heap-size",
execArgv: ["--max-heap-size=64.5", "--max-heap-size=0"],
nodeOptions: undefined,
heap: { kind: "unknown" } as const,
},
] as const)(
"resolveRegexWorkerHeap $name",
({ execArgv, nodeOptions, heap }) => {
expect(resolveRegexWorkerHeap(execArgv, nodeOptions)).toEqual(heap);
},
);
it("does not treat a rewritten NODE_OPTIONS value as the launch heap", () => {
const previous = process.env.NODE_OPTIONS;
const rewritten = "--max-old-space-size=61-rewritten";
process.env.NODE_OPTIONS = rewritten;
try {
const launch = readLaunchNodeOptions();
if (process.platform === "linux") {
expect(launch.kind).toBe("recovered");
if (launch.kind === "recovered")
expect(launch.value ?? "").not.toContain("61-rewritten");
} else {
expect(launch).toEqual({ kind: "unrecoverable" });
}
expect(
resolveLaunchRegexWorkerHeap(
{ kind: "recovered", value: ["--max-old-space-size=256"] },
{ kind: "unrecoverable" },
),
).toEqual({ kind: "unknown" });
expect(
resolveLaunchRegexWorkerHeap(
{ kind: "recovered", value: [] },
{ kind: "recovered", value: "--max_heap_size=64" },
),
).toEqual({ kind: "unknown" });
expect(
resolveLaunchRegexWorkerHeap(
{ kind: "unrecoverable" },
{ kind: "recovered", value: "--max-old-space-size=64" },
),
).toEqual({ kind: "unknown" });
if (launch.kind === "recovered") {
expect(launch.value).not.toBe(rewritten);
expect(
resolveLaunchRegexWorkerHeap(
{ kind: "recovered", value: ["--max-old-space-size=32"] },
launch,
),
).toEqual(
resolveRegexWorkerHeap(["--max-old-space-size=32"], launch.value),
);
}
} finally {
if (previous === undefined) delete process.env.NODE_OPTIONS;
else process.env.NODE_OPTIONS = previous;
}
});
const commandLine = (...tokens: readonly string[]): Buffer =>
Buffer.from(`${tokens.join("\0")}\0`);
it("recovers launch exec argv from the command line, not process.execArgv", () => {
expect(
execArgvFromCommandLine(
commandLine(
"node",
"--max-old-space-size=64",
"script.js",
"--max-old-space-size=256",
),
),
).toEqual({ kind: "recovered", value: ["--max-old-space-size=64"] });
expect(
resolveLaunchRegexWorkerHeap(
execArgvFromCommandLine(
commandLine("node", "--max-old-space-size=64", "script.js"),
),
{ kind: "recovered", value: undefined },
),
).toEqual({ kind: "effective", mib: 64, overridden: true });
expect(resolveRegexWorkerHeap([], undefined)).toEqual({
kind: "effective",
mib: 128,
overridden: false,
});
expect(
execArgvFromCommandLine(
commandLine(
"node",
"--require",
"preload.cjs",
"--max_old_space_size=64",
"script.js",
),
),
).toEqual({
kind: "recovered",
value: ["--require", "preload.cjs", "--max_old_space_size=64"],
});
expect(
execArgvFromCommandLine(
commandLine("node", "-e", "code", "--max-old-space-size=80"),
),
).toEqual({
kind: "recovered",
value: ["-e", "code", "--max-old-space-size=80"],
});
expect(
execArgvFromCommandLine(
commandLine("node", "--", "--max-old-space-size=64", "script.js"),
),
).toEqual({ kind: "recovered", value: [] });
expect(
execArgvFromCommandLine(
commandLine("node", "--not-a-real-node-option", "script.js"),
),
).toEqual({ kind: "unrecoverable" });
expect(execArgvFromCommandLine(Buffer.alloc(0))).toEqual({
kind: "unrecoverable",
});
expect(execArgvFromCommandLine(Buffer.from([0xff, 0]))).toEqual({
kind: "unrecoverable",
});
expect(execArgvFromCommandLine(commandLine("rea"))).toEqual({
kind: "unrecoverable",
});
expect(
execArgvFromCommandLine(
Buffer.concat([Buffer.from("rea\0"), Buffer.alloc(32)]),
),
).toEqual({ kind: "unrecoverable" });
expect(
resolveLaunchRegexWorkerHeap(
execArgvFromCommandLine(
Buffer.concat([Buffer.from("rea\0"), Buffer.alloc(32)]),
),
{ kind: "recovered", value: undefined },
),
).toEqual({ kind: "unknown" });
expect(execArgvFromCommandLine(commandLine("node", "script.js"))).toEqual({
kind: "recovered",
value: [],
});
const before = readLaunchExecArgv();
const snapshot = [...process.execArgv];
process.execArgv.length = 0;
try {
expect(process.execArgv).toEqual([]);
expect(readLaunchExecArgv()).toEqual(before);
if (before.kind === "recovered") expect(before.value).toEqual(snapshot);
else expect(before).toEqual({ kind: "unrecoverable" });
} finally {
process.execArgv.push(...snapshot);
}
});
it("describes the effective heap and keeps the deadline fixed", () => {
for (const heap of [
effective(128, false),
effective(256, true),
{ kind: "unknown" } as const,
]) {
const limitation = regexMatchingLimitation(heap);
expect(limitation).toContain(
"This release has no runtime override for the deadline.",
);
expect(limitation).not.toContain("either cap");
expect(limitation).not.toContain("no runtime override for the heap");
const projected = projectAnalysisError(
regexWorkerHeapError("search_procedures", new Error("heap"), heap),
);
expect(projected.remediation.action).not.toContain("no runtime override");
expect(projected.remediation.action).toContain("--max-old-space-size");
expect(projected.remediation.action).toContain(
"--max-old-space-size-percentage",
);
if (heap.kind === "unknown") {
expect(limitation).toContain("was not established");
expect(limitation).toContain("--max-old-space-size-percentage");
expect(limitation).toContain("--max-heap-size");
expect(limitation).toContain("--experimental-config-file");
expect(projected.remediation.action).toContain("--max-heap-size");
expect(projected.remediation.action).toContain(
"--experimental-default-config-file",
);
expect(projected.message).toContain("was not established");
expect(projected.details).toMatchObject({
reported_limits: { requested_worker_heap_mib: 128 },
});
expect(projected.details?.reported_limits).toEqual({
requested_worker_heap_mib: 128,
});
} else {
expect(limitation).toContain(`effective ${String(heap.mib)} MiB`);
expect(projected.message).toContain(`effective ${String(heap.mib)} MiB`);
expect(projected.details).toMatchObject({
reported_limits: {
worker_heap_mib: heap.mib,
requested_worker_heap_mib: 128,
},
});
}
}
});
+405 -7
View File
@@ -1,3 +1,4 @@
import { readFileSync } from "node:fs";
import { Worker } from "node:worker_threads";
import { z } from "zod";
@@ -13,10 +14,399 @@ import { err, ok, type Result } from "../domain/result.js";
const MATCHING_DEADLINE_MS = 5_000;
const WORKER_HEAP_MIB = 128;
/** Shared capability text for the supervised ECMAScript matching boundary. */
export const REGEX_MATCHING_LIMITATION = `Regex mode uses ECMAScript Unicode semantics in a cancellable worker with a ${String(MATCHING_DEADLINE_MS / 1000)}-second matching deadline and a ${String(WORKER_HEAP_MIB)} MiB worker heap. This release has no runtime override for either cap; matching failure leaves the service available.`;
/**
* Node's Worker `resourceLimits.maxOldGenerationSizeMb` request. A positive
* `--max-old-space-size` on the process that starts REA, including
* `NODE_OPTIONS`, overrides it. A zero or empty value leaves this request
* in effect. `NODE_OPTIONS` is applied before `process.execArgv`, and the
* last absolute flag wins. That environment value is not copied into `execArgv`.
*
* `--max-old-space-size-percentage` is applied after both sources and takes
* precedence over the absolute flag in either order. Node turns it into a MiB
* from constrained or total memory while parsing options. That base can change
* before user code reads it, and some Node versions also cap total memory at
* 4 GiB when pointer compression is on. A later read is not the value Node
* applied, so a percentage flag leaves the effective heap unestablished
* instead of the 128 MiB request or an absolute flag.
*
* `process.env.NODE_OPTIONS` is not that launch value. A preload can clear or
* rewrite it after V8 has already applied the original flags, and those flags
* are not copied into `process.execArgv`. The launch string is the
* `NODE_OPTIONS` entry in the process initial environment when that block can
* be read. Otherwise the effective heap stays unestablished.
*
* `process.execArgv` is mutable in the same way. A preload can empty it after
* V8 has applied `--max-old-space-size`. Linux keeps the original tokens in
* `/proc/self/cmdline` until `process.title` or `--title` overwrites that
* same memory with the title. The recovered list skips the executable and the
* program entry the way Node builds `execArgv`, and it stops at `--`. A
* buffer without a program entry is not an empty option list. When that block
* cannot be read, the entry cannot be established, or the platform has no
* equivalent block, the effective heap stays unestablished. The mutable array
* is not a fallback.
*
* `--max-heap-size` is the total heap, not the old generation. A positive
* value can be lower than the 128 MiB request and still changes the result
* when an old-space flag is also present. The old generation stays
* unestablished while that value is in effect. Zero or empty clears it.
* Node 24 and later also accept the flag in `NODE_OPTIONS`.
*
* `--experimental-config-file` and `--experimental-default-config-file` can
* set `nodeOptions`, including a heap percentage, from a file. That file is
* not launch provenance, and it can add heap options when `NODE_OPTIONS` was
* already set. Either flag leaves the old generation unestablished. Node
* accepts the config path as `=` or the next argument. Underscore spellings
* are not these flags.
*/
const OLD_SPACE_FLAG = /^--max[-_]old[-_]space[-_]size(?:=(.*))?$/u;
const OLD_SPACE_PERCENT_FLAG =
/^--max[-_]old[-_]space[-_]size[-_]percentage(?:=.*)?$/u;
const MAX_HEAP_FLAG = /^--max[-_]heap[-_]size(?:=(.*))?$/u;
/**
* Node applies `NODE_OPTIONS` from `--env-file` / `--env-file-if-exists` only
* when the variable was absent. The file is not in `/proc/self/environ`, and a
* later read is not the startup contents. Underscore spellings are not this flag.
*/
const ENV_FILE_FLAG = /^--env-file(?:-if-exists)?(?:=.*)?$/u;
/** Path form and the boolean default-file flag. The file itself is not read. */
const CONFIG_FILE_FLAG = /^--experimental-(?:default-)?config-file(?:=.*)?$/u;
/** Fixed regex deadline. The worker heap and this deadline are separate caps. */
/** Old-generation heap the regex worker will actually hit. */
export type RegexWorkerHeapLimit =
| {
readonly kind: "effective";
readonly mib: number;
/** True when a positive `--max-old-space-size` overrides the 128 MiB request. */
readonly overridden: boolean;
}
| { readonly kind: "unknown" };
const requestedHeap = {
kind: "effective",
mib: WORKER_HEAP_MIB,
overridden: false,
} as const satisfies RegexWorkerHeapLimit;
/** Node's `ParseNodeOptionsEnvVar`: quotes group tokens and are not part of them. */
const nodeOptionsArgv = (
nodeOptions: string,
): readonly string[] | undefined => {
const argv: string[] = [];
let inString = false;
let startNew = true;
for (let index = 0; index < nodeOptions.length; index += 1) {
let character = nodeOptions.charAt(index);
if (character === "\\" && inString) {
if (index + 1 >= nodeOptions.length) return undefined;
index += 1;
character = nodeOptions.charAt(index);
} else if (character === " " && !inString) {
startNew = true;
continue;
} else if (character === '"') {
inString = !inString;
continue;
}
if (startNew) {
argv.push(character);
startNew = false;
} else {
const last = argv.length - 1;
argv[last] = `${argv[last] ?? ""}${character}`;
}
}
return inString ? undefined : argv;
};
const parsedFlagMiB = (
token: string,
flag: RegExp,
): number | "absent" | "unrecognized" => {
const match = flag.exec(token);
if (match === null) return "absent";
const raw = match[1];
if (raw === undefined) return "unrecognized";
if (raw === "") return 0;
if (!/^\+?\d+$/u.test(raw)) return "unrecognized";
const mib = Number(raw);
return Number.isSafeInteger(mib) ? mib : "unrecognized";
};
/**
* Derive the worker old-generation heap from launch flags the caller already
* recovered. The raw `NODE_OPTIONS` string is not retained. This does not read
* `process.env`.
*/
export const resolveRegexWorkerHeap = (
execArgv: readonly string[],
nodeOptions: string | undefined,
): RegexWorkerHeapLimit => {
const fromEnvironment =
nodeOptions === undefined ? [] : nodeOptionsArgv(nodeOptions);
const tokens = [...(fromEnvironment ?? []), ...execArgv];
// An absent NODE_OPTIONS lets the env file supply one, including a percentage
// that beats a later absolute flag. The file's text is not launch provenance.
// A Node config file can inject nodeOptions even when NODE_OPTIONS was
// recovered. The file is not read back.
if (
tokens.some((token) => CONFIG_FILE_FLAG.test(token)) ||
(nodeOptions === undefined &&
tokens.some((token) => ENV_FILE_FLAG.test(token)))
)
return { kind: "unknown" };
let limit: RegexWorkerHeapLimit =
fromEnvironment === undefined ? { kind: "unknown" } : requestedHeap;
let percentage = false;
let maxHeap: "clear" | "positive" | "unrecognized" = "clear";
for (const token of tokens) {
if (OLD_SPACE_PERCENT_FLAG.test(token)) {
percentage = true;
limit = { kind: "unknown" };
continue;
}
// A percentage flag wins over every absolute flag, including a later one.
if (percentage) continue;
const totalHeap = parsedFlagMiB(token, MAX_HEAP_FLAG);
if (totalHeap === "unrecognized") {
maxHeap = "unrecognized";
continue;
}
if (totalHeap !== "absent") {
if (maxHeap !== "unrecognized")
maxHeap = totalHeap > 0 ? "positive" : "clear";
continue;
}
const parsed = parsedFlagMiB(token, OLD_SPACE_FLAG);
if (parsed === "absent") continue;
if (parsed === "unrecognized") {
limit = { kind: "unknown" };
continue;
}
limit =
parsed > 0
? { kind: "effective", mib: parsed, overridden: true }
: requestedHeap;
}
// A positive total-heap cap is not an old-generation size. Zero clears it
// and leaves the old-space result. An unparsed value does not.
return maxHeap === "clear" ? limit : { kind: "unknown" };
};
/** Provenance of the `NODE_OPTIONS` value Node applied before user code ran. */
export type LaunchNodeOptions =
| { readonly kind: "recovered"; readonly value: string | undefined }
| { readonly kind: "unrecoverable" };
const nodeOptionsKey = Buffer.from("NODE_OPTIONS=");
/** First `NODE_OPTIONS` entry, matching `getenv`. The surrounding block is not retained. */
const nodeOptionsFromEnviron = (
raw: Buffer,
): string | undefined | "invalid" => {
let offset = 0;
while (offset < raw.length) {
const end = raw.indexOf(0, offset);
const entryEnd = end === -1 ? raw.length : end;
const entry = raw.subarray(offset, entryEnd);
if (
entry.length >= nodeOptionsKey.length &&
entry.subarray(0, nodeOptionsKey.length).equals(nodeOptionsKey)
) {
try {
return new TextDecoder("utf-8", { fatal: true }).decode(
entry.subarray(nodeOptionsKey.length),
);
} catch {
return "invalid";
}
}
if (end === -1) break;
offset = end + 1;
}
return undefined;
};
/**
* Linux keeps the `execve` environment in `/proc/self/environ` after
* `process.env` is rewritten. Other platforms have no equivalent immutable
* block, so the launch value stays unrecoverable there.
*/
export const readLaunchNodeOptions = (): LaunchNodeOptions => {
if (process.platform !== "linux") return { kind: "unrecoverable" };
let raw: Buffer;
try {
raw = readFileSync("/proc/self/environ");
} catch {
return { kind: "unrecoverable" };
}
const value = nodeOptionsFromEnviron(raw);
return value === "invalid"
? { kind: "unrecoverable" }
: { kind: "recovered", value };
};
/**
* Node options that take the next command-line token when it is not `=`.
* Boolean flags and V8 flags do not. Names are the dashed form Node looks up
* after replacing `_`. An option absent from both sets, followed by the
* program entry, leaves the boundary unrecoverable.
*/
const VALUE_OPTION_NAMES =
"--allow-fs-read,--allow-fs-write,--build-sea,--build-snapshot-config,--conditions,--cpu-prof-dir,--cpu-prof-interval,--cpu-prof-name,--debug-port,--diagnostic-dir,--disable-proto,--disable-warning,--dns-result-order,--env-file,--env-file-if-exists,--eval,--experimental-config-file,--experimental-default-type,--experimental-loader,--experimental-sea-config,--experimental-test-isolation,--heap-prof-dir,--heap-prof-interval,--heap-prof-name,--heapsnapshot-near-heap-limit,--heapsnapshot-signal,--icu-data-dir,--import,--input-type,--inspect-port,--inspect-publish-uid,--loader,--localstorage-file,--max-http-header-size,--max-old-space-size-percentage,--network-family-autoselection-attempt-timeout,--openssl-config,--print,--redirect-warnings,--report-dir,--report-directory,--report-filename,--report-signal,--require,--run,--secure-heap,--secure-heap-min,--snapshot-blob,--test-concurrency,--test-coverage-branches,--test-coverage-exclude,--test-coverage-functions,--test-coverage-include,--test-coverage-lines,--test-global-setup,--test-isolation,--test-name-pattern,--test-reporter,--test-reporter-destination,--test-shard,--test-skip-pattern,--test-timeout,--title,--tls-cipher-list,--tls-keylog,--trace-event-categories,--trace-event-file-pattern,--trace-require-module,--unhandled-rejections,--use-largepages,--v8-pool-size,--watch-kill-signal,--watch-path,-C,-e,-p,-pe,-r";
const FLAG_OPTION_NAMES =
"--abort-on-uncaught-exception,--allow-addons,--allow-child-process,--allow-inspector,--allow-net,--allow-wasi,--allow-worker,--build-snapshot,--check,--completion-bash,--cpu-prof,--disable-sigusr1,--disable-wasm-trap-handler,--disallow-code-generation-from-strings,--enable-etw-stack-walking,--enable-fips,--enable-network-family-autoselection,--enable-source-maps,--entry-url,--experimental-addon-modules,--experimental-async-context-frame,--experimental-default-config-file,--experimental-eventsource,--experimental-import-meta-resolve,--experimental-inspector-network-resource,--experimental-network-inspection,--experimental-permission,--experimental-print-required-tla,--experimental-quic,--experimental-storage-inspection,--experimental-stream-iter,--experimental-test-coverage,--experimental-test-module-mocks,--experimental-transform-types,--experimental-vm-modules,--experimental-webstorage,--experimental-worker-inspection,--expose-gc,--force-context-aware,--force-fips,--force-node-api-uncaught-exceptions-policy,--frozen-intrinsics,--heap-prof,--heap-snapshot-on-oom,--help,--huge-max-old-generation-size,--insecure-http-parser,--inspect,--inspect-brk,--inspect-wait,--interactive,--interpreted-frames-native-stack,--jitless,--max-heap-size,--max-old-space-size,--max-semi-space-size,--no-addons,--no-async-context-frame,--no-deprecation,--no-experimental-detect-module,--no-experimental-fetch,--no-experimental-global-customevent,--no-experimental-global-navigator,--no-experimental-global-webcrypto,--no-experimental-repl-await,--no-experimental-require-module,--no-experimental-sqlite,--no-experimental-strip-types,--no-experimental-websocket,--no-experimental-webstorage,--no-extra-info-on-fatal-exception,--no-force-async-hooks-checks,--no-global-search-paths,--no-network-family-autoselection,--no-require-module,--no-strip-types,--no-warnings,--node-memory-debug,--openssl-legacy-provider,--openssl-shared-config,--pending-deprecation,--permission,--permission-audit,--preserve-symlinks,--preserve-symlinks-main,--prof,--prof-process,--report-compact,--report-exclude-env,--report-exclude-network,--report-on-fatalerror,--report-on-signal,--report-uncaught-exception,--test,--test-force-exit,--test-only,--test-rerun-failures,--test-update-snapshots,--throw-deprecation,--tls-max-v1.2,--tls-max-v1.3,--tls-min-v1.0,--tls-min-v1.1,--tls-min-v1.2,--tls-min-v1.3,--trace-atomics-wait,--trace-deprecation,--trace-env,--trace-env-js-stack,--trace-env-native-stack,--trace-exit,--trace-promises,--trace-sigint,--trace-sync-io,--trace-tls,--trace-uncaught,--trace-warnings,--track-heap-objects,--use-bundled-ca,--use-env-proxy,--use-openssl-ca,--use-system-ca,--v8-options,--version,--watch,--watch-preserve-output,--zero-fill-buffers,-c,-h,-i,-v";
const VALUE_OPTIONS = new Set(VALUE_OPTION_NAMES.split(","));
const FLAG_OPTIONS = new Set(FLAG_OPTION_NAMES.split(","));
/** Provenance of the Node option tokens that were `process.execArgv` at start. */
export type LaunchExecArgv =
| { readonly kind: "recovered"; readonly value: readonly string[] }
| { readonly kind: "unrecoverable" };
const commandLineTokens = (raw: Buffer): readonly string[] | "invalid" => {
if (raw.length === 0) return "invalid";
const end = raw[raw.length - 1] === 0 ? raw.length - 1 : raw.length;
const tokens: string[] = [];
const decoder = new TextDecoder("utf-8", { fatal: true });
let offset = 0;
while (offset < end) {
const next = raw.indexOf(0, offset);
const tokenEnd = next === -1 || next > end ? end : next;
try {
tokens.push(decoder.decode(raw.subarray(offset, tokenEnd)));
} catch {
return "invalid";
}
if (next === -1 || next >= end) break;
offset = next + 1;
}
return tokens.length === 0 ? "invalid" : tokens;
};
const normalizedOption = (
token: string,
): { readonly name: string; readonly inline: boolean } => {
if (!token.startsWith("--")) return { name: token, inline: false };
const equals = token.indexOf("=");
const rawName = equals === -1 ? token : token.slice(0, equals);
return {
name: `${rawName.slice(0, 2)}${rawName.slice(2).replaceAll("_", "-")}`,
inline: equals !== -1,
};
};
/** Options whose argument is the program entry rather than a script path. */
const ENTRY_OPTIONS = new Set(["-e", "--eval", "-p", "--print", "-pe"]);
/**
* Node option tokens from an `execve` command line. The executable and the
* program entry are omitted. `--` ends the list and is omitted. A title
* overwrite leaves one token and null padding, which has no program entry.
* The buffer is not retained.
*/
export const execArgvFromCommandLine = (raw: Buffer): LaunchExecArgv => {
const tokens = commandLineTokens(raw);
if (tokens === "invalid" || tokens.length === 0 || tokens[0] === "")
return { kind: "unrecoverable" };
const exec: string[] = [];
let sawEntry = false;
for (let index = 1; index < tokens.length; index += 1) {
const token = tokens[index] ?? "";
if (token.length === 0) return { kind: "unrecoverable" };
if (token === "--") {
const entry = tokens[index + 1];
if (entry === undefined || entry.length === 0)
return { kind: "unrecoverable" };
sawEntry = true;
break;
}
if (token === "-" || !token.startsWith("-")) {
sawEntry = true;
break;
}
exec.push(token);
const option = normalizedOption(token);
const entryOption = ENTRY_OPTIONS.has(option.name);
if (option.inline) {
if (entryOption) sawEntry = true;
continue;
}
if (VALUE_OPTIONS.has(option.name)) {
const value = tokens[index + 1];
if (value === undefined || value.length === 0 || value.startsWith("-"))
return { kind: "unrecoverable" };
exec.push(value);
index += 1;
if (entryOption) sawEntry = true;
continue;
}
if (FLAG_OPTIONS.has(option.name)) continue;
const next = tokens[index + 1];
if (next !== undefined && (next.length === 0 || !next.startsWith("-")))
return { kind: "unrecoverable" };
}
// No script, eval string, or stdin entry. A title-only buffer looks like this.
if (!sawEntry) return { kind: "unrecoverable" };
return { kind: "recovered", value: exec };
};
/**
* Linux keeps the `execve` argument vector in `/proc/self/cmdline` after
* `process.execArgv` is rewritten. `process.title` and `--title` overwrite
* that vector. Other platforms have no equivalent immutable block, so the
* launch options stay unrecoverable there.
*/
export const readLaunchExecArgv = (): LaunchExecArgv => {
if (process.platform !== "linux") return { kind: "unrecoverable" };
let raw: Buffer;
try {
raw = readFileSync("/proc/self/cmdline");
} catch {
return { kind: "unrecoverable" };
}
return execArgvFromCommandLine(raw);
};
/** Heap for this process. Unrecoverable launch provenance stays unknown. */
export const resolveLaunchRegexWorkerHeap = (
launchExecArgv: LaunchExecArgv,
launchNodeOptions: LaunchNodeOptions,
): RegexWorkerHeapLimit =>
launchExecArgv.kind === "unrecoverable" ||
launchNodeOptions.kind === "unrecoverable"
? { kind: "unknown" }
: resolveRegexWorkerHeap(launchExecArgv.value, launchNodeOptions.value);
const launchRegexWorkerHeap = resolveLaunchRegexWorkerHeap(
readLaunchExecArgv(),
readLaunchNodeOptions(),
);
const heapBound = (heap: RegexWorkerHeapLimit): string => {
if (heap.kind === "unknown")
return "The worker requests 128 MiB. The effective old-generation heap was not established from the launch flags. --max-old-space-size, --max-old-space-size-percentage, --max-heap-size, --env-file, --env-file-if-exists, --experimental-config-file, and --experimental-default-config-file can change that request.";
const override =
"A positive Node --max-old-space-size, including NODE_OPTIONS, overrides the 128 MiB worker request";
return heap.overridden
? `${override} and sets this process's effective heap to ${String(heap.mib)} MiB.`
: `${override}. This process has no positive override, so the effective heap is ${String(heap.mib)} MiB.`;
};
/** Capability text for the supervised ECMAScript matching boundary. */
export const regexMatchingLimitation = (heap: RegexWorkerHeapLimit): string => {
const preface =
heap.kind === "effective"
? `Regex mode uses ECMAScript Unicode semantics in a cancellable worker with a ${String(MATCHING_DEADLINE_MS / 1000)}-second matching deadline and an effective ${String(heap.mib)} MiB old-generation heap.`
: `Regex mode uses ECMAScript Unicode semantics in a cancellable worker with a ${String(MATCHING_DEADLINE_MS / 1000)}-second matching deadline.`;
return `${preface} This release has no runtime override for the deadline. ${heapBound(heap)} Matching failure leaves the service available.`;
};
/** Shared capability text for the supervised ECMAScript matching boundary. */
export const REGEX_MATCHING_LIMITATION = regexMatchingLimitation(
launchRegexWorkerHeap,
);
/** Fixed regex deadline. The worker heap is a separate, launch-flag-sensitive cap. */
export const regexMatchingDeadlineError = (
operation: string,
): AnalysisResourceConstraintError =>
@@ -30,19 +420,27 @@ export const regexMatchingDeadlineError = (
},
);
/** Fixed regex worker heap. Inherited limits and host memory do not raise it. */
/** Regex worker heap at the effective old-generation limit, which a launch flag can change. */
export const regexWorkerHeapError = (
operation: string,
cause: Error,
heap: RegexWorkerHeapLimit = launchRegexWorkerHeap,
): AnalysisResourceConstraintError =>
new AnalysisResourceConstraintError(
operation,
"memory",
`Regex worker exhausted its ${String(WORKER_HEAP_MIB)} MiB heap budget. This release has no runtime override.`,
{ worker_heap_mib: WORKER_HEAP_MIB },
heap.kind === "effective"
? `Regex worker exhausted its effective ${String(heap.mib)} MiB old-generation heap.`
: "Regex worker exhausted its old-generation heap. The effective heap was not established.",
heap.kind === "effective"
? {
worker_heap_mib: heap.mib,
requested_worker_heap_mib: WORKER_HEAP_MIB,
}
: { requested_worker_heap_mib: WORKER_HEAP_MIB },
{
cause,
remediationAction: `The regex worker heap is ${String(WORKER_HEAP_MIB)} MiB and this release has no runtime override. Inherited limits and additional host memory do not raise it. Simplify the pattern or use literal mode.`,
remediationAction: `${heapBound(heap)} Restart REA after changing --max-old-space-size. A launch --max-old-space-size-percentage takes precedence over that flag in either order. Simplify the pattern or use literal mode.`,
},
);
const replySchema = z.discriminatedUnion("status", [
+381 -10
View File
@@ -1,3 +1,8 @@
import { spawnSync } from "node:child_process";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { expect, it } from "vitest";
import { ok } from "../../../src/domain/result.js";
@@ -8,15 +13,75 @@ import {
matchRegexValues,
REGEX_MATCHING_LIMITATION,
regexMatchingDeadlineError,
regexMatchingLimitation,
regexWorkerHeapError,
execArgvFromCommandLine,
resolveLaunchRegexWorkerHeap,
resolveRegexWorkerHeap,
readLaunchExecArgv,
readLaunchNodeOptions,
} from "../../../src/process/RegexMatcher.js";
it("names the fixed regex deadline and heap and says neither has an override", () => {
expect(REGEX_MATCHING_LIMITATION).toContain("5-second matching deadline");
expect(REGEX_MATCHING_LIMITATION).toContain("128 MiB");
expect(REGEX_MATCHING_LIMITATION).toContain(
"This release has no runtime override",
const workerHeapProbe = `
import { Worker } from "node:worker_threads";
const worker = new Worker(
\`import { parentPort, resourceLimits } from "node:worker_threads";
import v8 from "node:v8";
parentPort.postMessage({
resourceOld: resourceLimits.maxOldGenerationSizeMb,
heap: v8.getHeapStatistics().heap_size_limit,
});\`,
{ eval: true, resourceLimits: { maxOldGenerationSizeMb: 128 } },
);
worker.once("message", (message) => {
process.stdout.write(JSON.stringify(message));
void worker.terminate();
});
worker.once("error", (error) => {
process.stderr.write(error.message);
process.exitCode = 1;
});
`;
const probeWorkerHeap = (
args: readonly string[],
nodeOptions: string | undefined,
cwd?: string,
): { readonly resourceOld: number; readonly heap: number } => {
const env = { ...process.env };
if (nodeOptions === undefined) delete env.NODE_OPTIONS;
else env.NODE_OPTIONS = nodeOptions;
const result = spawnSync(process.execPath, [...args, "-e", workerHeapProbe], {
cwd,
env,
encoding: "utf8",
});
expect(result.status).toBe(0);
return JSON.parse(result.stdout ?? "") as {
resourceOld: number;
heap: number;
};
};
it("names the fixed regex deadline and the effective worker heap", () => {
const heap = resolveLaunchRegexWorkerHeap(
readLaunchExecArgv(),
readLaunchNodeOptions(),
);
expect(REGEX_MATCHING_LIMITATION).toBe(regexMatchingLimitation(heap));
expect(REGEX_MATCHING_LIMITATION).toContain("5-second matching deadline");
expect(REGEX_MATCHING_LIMITATION).toContain(
"This release has no runtime override for the deadline.",
);
expect(REGEX_MATCHING_LIMITATION).not.toContain("either cap");
expect(REGEX_MATCHING_LIMITATION).not.toContain("or heap");
if (heap.kind === "unknown") {
expect(REGEX_MATCHING_LIMITATION).toContain("was not established");
} else {
expect(REGEX_MATCHING_LIMITATION).toContain(
`effective ${String(heap.mib)} MiB`,
);
}
const deadline = projectAnalysisError(
regexMatchingDeadlineError("search_procedures"),
);
@@ -24,13 +89,25 @@ it("names the fixed regex deadline and heap and says neither has an override", (
expect(deadline.remediation.action).toContain(
"this release has no runtime override",
);
const heap = projectAnalysisError(
const failure = projectAnalysisError(
regexWorkerHeapError("search_procedures", new Error("heap")),
);
expect(heap.message).toContain("128 MiB");
expect(heap.remediation.action).toContain(
"additional host memory do not raise it",
);
expect(failure.remediation.action).not.toContain("no runtime override");
expect(failure.remediation.action).toContain("--max-old-space-size");
if (heap.kind === "effective") {
expect(failure.message).toContain(`${String(heap.mib)} MiB`);
expect(failure.details).toMatchObject({
reported_limits: {
worker_heap_mib: heap.mib,
requested_worker_heap_mib: 128,
},
});
} else {
expect(failure.message).toContain("was not established");
expect(failure.details).toMatchObject({
reported_limits: { requested_worker_heap_mib: 128 },
});
}
for (const name of ["search_procedures", "search_strings"] as const) {
const description = OFFICIAL_TOOL_CONTRACTS.find(
(tool) => tool.name === name,
@@ -40,6 +117,300 @@ it("names the fixed regex deadline and heap and says neither has an override", (
}
});
const relativeHeapGap = (left: number, right: number): number =>
Math.abs(left - right) / Math.max(left, right);
it("leaves a heap percentage unestablished without reading host memory", () => {
const percentage = "--max-old-space-size-percentage=50";
const absolute = "--max-old-space-size=64";
const observed = probeWorkerHeap([percentage, absolute], undefined);
expect(observed.resourceOld).toBe(128);
for (const execArgv of [
[percentage],
[percentage, absolute],
[absolute, percentage],
])
expect(resolveRegexWorkerHeap(execArgv, undefined)).toEqual({
kind: "unknown",
});
});
it("does not report the 128 MiB request for NODE_OPTIONS supplied by an env file", () => {
const directory = mkdtempSync(join(tmpdir(), "rea-regex-env-file-"));
const file = join(directory, "heap.env");
writeFileSync(file, "NODE_OPTIONS=--max-old-space-size=256\n");
try {
const fromFile = probeWorkerHeap([`--env-file=${file}`], undefined);
const ifExists = probeWorkerHeap(
[`--env-file-if-exists=${file}`],
undefined,
);
const small = probeWorkerHeap(["--max-old-space-size=64"], undefined);
const large = probeWorkerHeap(["--max-old-space-size=256"], undefined);
expect(fromFile.resourceOld).toBe(128);
expect(ifExists.resourceOld).toBe(128);
expect(small.resourceOld).toBe(128);
expect(large.resourceOld).toBe(128);
expect(relativeHeapGap(fromFile.heap, large.heap)).toBeLessThan(0.05);
expect(relativeHeapGap(ifExists.heap, large.heap)).toBeLessThan(0.05);
expect(fromFile.heap).toBeGreaterThan(small.heap);
expect(ifExists.heap).toBeGreaterThan(small.heap);
for (const execArgv of [
[`--env-file=${file}`],
[`--env-file-if-exists=${file}`],
[`--env-file=${file}`, "--max-old-space-size=64"],
])
expect(resolveRegexWorkerHeap(execArgv, undefined)).toEqual({
kind: "unknown",
});
expect(
resolveRegexWorkerHeap([`--env-file=${file}`], "--max-old-space-size=64"),
).toEqual({ kind: "effective", mib: 64, overridden: true });
} finally {
rmSync(directory, { recursive: true, force: true });
}
});
it("does not report the 128 MiB request for a heap percentage in a Node config file", () => {
const directory = mkdtempSync(join(tmpdir(), "rea-regex-config-file-"));
const file = join(directory, "heap.config.json");
const config = '{"nodeOptions":{"max-old-space-size-percentage":"1"}}\n';
writeFileSync(file, config);
writeFileSync(join(directory, "node.config.json"), config);
try {
const fromFile = probeWorkerHeap(
[`--experimental-config-file=${file}`],
undefined,
);
const split = probeWorkerHeap(
["--experimental-config-file", file],
undefined,
);
const withAbsolute = probeWorkerHeap(
[`--experimental-config-file=${file}`, "--max-old-space-size=256"],
undefined,
);
const fromDefault = probeWorkerHeap(
["--experimental-default-config-file"],
undefined,
directory,
);
expect(fromFile.resourceOld).toBe(128);
expect(split.resourceOld).toBe(128);
expect(withAbsolute.resourceOld).toBe(128);
expect(fromDefault.resourceOld).toBe(128);
for (const execArgv of [
[`--experimental-config-file=${file}`],
["--experimental-config-file", file],
[`--experimental-config-file=${file}`, "--max-old-space-size=64"],
["--experimental-default-config-file"],
])
expect(resolveRegexWorkerHeap(execArgv, undefined)).toEqual({
kind: "unknown",
});
expect(
resolveRegexWorkerHeap(
["--experimental-config-file=node.config.json"],
"--max-old-space-size=64",
),
).toEqual({ kind: "unknown" });
} finally {
rmSync(directory, { recursive: true, force: true });
}
});
it("does not report the 128 MiB request when --max-heap-size lowers the worker heap", () => {
const heap64 = probeWorkerHeap(["--max-heap-size=64"], undefined);
const underscore = probeWorkerHeap(["--max_heap_size=64"], undefined);
const old256 = probeWorkerHeap(["--max-old-space-size=256"], undefined);
expect(heap64.resourceOld).toBe(128);
expect(underscore.resourceOld).toBe(128);
expect(old256.resourceOld).toBe(128);
expect(heap64.heap).toBeLessThan(old256.heap);
expect(relativeHeapGap(underscore.heap, heap64.heap)).toBeLessThan(0.05);
expect(resolveRegexWorkerHeap(["--max-heap-size=64"], undefined)).toEqual({
kind: "unknown",
});
expect(resolveRegexWorkerHeap([], "--max-heap-size=64")).toEqual({
kind: "unknown",
});
expect(
resolveRegexWorkerHeap(
["--max-heap-size=64", "--max-old-space-size=256"],
undefined,
),
).toEqual({ kind: "unknown" });
});
it("lets a launch --max-old-space-size override the 128 MiB worker request", () => {
const smaller = probeWorkerHeap(["--max-old-space-size=64"], undefined);
const larger = probeWorkerHeap([], "--max-old-space-size=256");
expect(smaller.resourceOld).toBe(128);
expect(larger.resourceOld).toBe(128);
expect(smaller.heap).toBeLessThan(larger.heap);
});
const assertUnrecoverableOffLinux = (): boolean => {
if (process.platform === "linux") return false;
const launchExecArgv = readLaunchExecArgv();
expect(launchExecArgv).toEqual({ kind: "unrecoverable" });
expect(
resolveLaunchRegexWorkerHeap(launchExecArgv, {
kind: "recovered",
value: "--max-old-space-size=64",
}),
).toEqual({ kind: "unknown" });
return true;
};
const spawnCommandLineReport = (
directory: string,
args: readonly string[],
preloadBody: string | undefined,
): {
execLen: number;
cmdline: string;
heap: number;
resourceOld: number;
} => {
const script = join(directory, "report.cjs");
const launchArgs = [...args];
if (preloadBody !== undefined) {
const preload = join(directory, "preload.cjs");
writeFileSync(preload, preloadBody);
launchArgs.push(`--require=${preload}`);
}
writeFileSync(
script,
`const { Worker } = require("node:worker_threads");
const fs = require("node:fs");
const worker = new Worker(
\`const { parentPort, resourceLimits } = require("node:worker_threads");
const v8 = require("node:v8");
parentPort.postMessage({
resourceOld: resourceLimits.maxOldGenerationSizeMb,
heap: v8.getHeapStatistics().heap_size_limit,
});\`,
{ eval: true, resourceLimits: { maxOldGenerationSizeMb: 128 } },
);
worker.once("message", (message) => {
process.stdout.write(JSON.stringify({
execLen: process.execArgv.length,
cmdline: fs.readFileSync("/proc/self/cmdline").toString("base64"),
heap: message.heap,
resourceOld: message.resourceOld,
}));
});
worker.once("error", (error) => {
process.stderr.write(error.message);
process.exitCode = 1;
});
`,
);
launchArgs.push(script);
const result = spawnSync(process.execPath, launchArgs, { encoding: "utf8" });
expect(result.status).toBe(0);
return JSON.parse(result.stdout ?? "") as {
execLen: number;
cmdline: string;
heap: number;
resourceOld: number;
};
};
it("does not report the 128 MiB request after a preload clears process.execArgv", () => {
if (assertUnrecoverableOffLinux()) return;
const directory = mkdtempSync(join(tmpdir(), "rea-regex-exec-argv-"));
try {
const reported = spawnCommandLineReport(
directory,
["--max-old-space-size=64"],
"process.execArgv.length = 0;\n",
);
const preload = join(directory, "preload.cjs");
const pinned = probeWorkerHeap(["--max-old-space-size=64"], undefined);
const larger = probeWorkerHeap(["--max-old-space-size=256"], undefined);
expect(reported.execLen).toBe(0);
expect(reported.resourceOld).toBe(128);
expect(relativeHeapGap(reported.heap, pinned.heap)).toBeLessThan(0.05);
expect(reported.heap).toBeLessThan(larger.heap);
const recovered = execArgvFromCommandLine(
Buffer.from(reported.cmdline, "base64"),
);
expect(recovered).toEqual({
kind: "recovered",
value: ["--max-old-space-size=64", `--require=${preload}`],
});
expect(
resolveLaunchRegexWorkerHeap(recovered, {
kind: "recovered",
value: undefined,
}),
).toEqual({ kind: "effective", mib: 64, overridden: true });
expect(resolveRegexWorkerHeap([], undefined)).toEqual({
kind: "effective",
mib: 128,
overridden: false,
});
} finally {
rmSync(directory, { recursive: true, force: true });
}
});
it("leaves the heap unknown when a preload title overwrites the command line", () => {
if (assertUnrecoverableOffLinux()) return;
const directory = mkdtempSync(join(tmpdir(), "rea-regex-title-"));
try {
const reported = spawnCommandLineReport(
directory,
["--max-old-space-size=64"],
'process.title = "rea";\n',
);
const pinned = probeWorkerHeap(["--max-old-space-size=64"], undefined);
expect(reported.resourceOld).toBe(128);
expect(relativeHeapGap(reported.heap, pinned.heap)).toBeLessThan(0.05);
const recovered = execArgvFromCommandLine(
Buffer.from(reported.cmdline, "base64"),
);
expect(recovered).toEqual({ kind: "unrecoverable" });
expect(
resolveLaunchRegexWorkerHeap(recovered, {
kind: "recovered",
value: undefined,
}),
).toEqual({ kind: "unknown" });
} finally {
rmSync(directory, { recursive: true, force: true });
}
});
it("leaves the heap unknown when --title overwrites the command line", () => {
if (assertUnrecoverableOffLinux()) return;
const directory = mkdtempSync(join(tmpdir(), "rea-regex-title-flag-"));
try {
const reported = spawnCommandLineReport(
directory,
["--title=rea", "--max-old-space-size=64"],
undefined,
);
const pinned = probeWorkerHeap(["--max-old-space-size=64"], undefined);
expect(reported.resourceOld).toBe(128);
expect(relativeHeapGap(reported.heap, pinned.heap)).toBeLessThan(0.05);
const recovered = execArgvFromCommandLine(
Buffer.from(reported.cmdline, "base64"),
);
expect(recovered).toEqual({ kind: "unrecoverable" });
expect(
resolveLaunchRegexWorkerHeap(recovered, {
kind: "recovered",
value: undefined,
}),
).toEqual({ kind: "unknown" });
} finally {
rmSync(directory, { recursive: true, force: true });
}
});
it("preserves Unicode matching order and refuses a pre-cancelled match", async () => {
expect(
await matchRegexValues({