mirror of
https://github.com/morluto/rea.git
synced 2026-10-26 17:35:11 +00:00
fix(bun): report the real runtime and enforce worker memory budgets on Windows (#1730)
* fix(bun): report the real runtime and enforce worker memory budgets on Windows Under Bun, REA recorded Node/V8 identity it did not observe and its JavaScript helper processes ran without a memory bound, because Bun ignores the V8 heap flags REA passes. Source-map trace Evidence claimed runtime node-v8 with Bun's synthetic V8 heap limit (the decoder then rejected that limit, so trace_web_source_location always failed), analysis resource failures reported heap limits Bun does not enforce, the HAR decoder claimed a fixed 192 MiB heap, and doctor reported Bun's Node.js compatibility version as the Node version. Identify the runtime once from process.versions. Source-map reports record id bun with the Bun and Node.js compatibility versions and a null v8_heap_limit_bytes; the decoder checks the heap budget only for V8 reports and rejects a report from a runtime other than the launching one. The analysis worker reports a null heap limit under Bun, HAR limitations and failures omit heap sizes under Bun, and the doctor node check reports Bun and passes when its Node.js compatibility version is supported. On Windows under Bun, each helper's owned Job Object now carries a commit limit of its heap budget plus a measured 512 MiB Bun runtime reserve (704 MiB for the codec and HAR decoder, 1536 MiB for the default analyzer). The native addon accepts an optional page-aligned job memory limit on process_spawn, verifies it by readback, associates a private completion port before the child is created, and reports a sticky memoryLimitReached and peak job commit on each poll; inspect reports jobMemoryLimit. A requested limit with an artifact lacking the control refuses the launch, and non-Windows launchers refuse a limit. Reaching the limit is classified by the job notification rather than Bun's exit status: the analyzer fails the current source with a memory resource failure reporting worker_process_memory_limit_bytes, the codec returns a memory resource failure, and the HAR decoder returns its memory failure with process_memory_limit_bytes. Bun source-map Evidence records process_memory_limit_bytes, verified against the launcher. Node.js launches and outputs are unchanged. * fix(windows): drain memory notifications after observing exit --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
This commit is contained in:
1 parent
d412def722
commit
66c21572de
48 files changed
+1630
-196
No files matched your search
@@ -51,6 +51,23 @@ without this lane reports Windows native controls as unavailable.
|
||||
unnamed, non-inherited Job Object. Membership and kill-on-close policy are
|
||||
checked before resume. Breakaway is disabled, and cleanup waits for all
|
||||
job members and captured output, including detached descendants.
|
||||
- A launch may request a Job Object commit limit (`JOB_OBJECT_LIMIT_JOB_MEMORY`)
|
||||
as a positive, page-aligned safe-integer byte count; invalid values fail
|
||||
with `ERROR_INVALID_PARAMETER` before any process is created. Readback must
|
||||
match the requested flags and limit exactly. A private I/O completion port is
|
||||
associated with that job before the child is created; each poll drains it
|
||||
without waiting and records a sticky `memoryLimitReached` when
|
||||
`JOB_OBJECT_MSG_JOB_MEMORY_LIMIT` arrives, alongside the job's
|
||||
`peakJobMemoryBytes`. Notifications are drained before checking for exit
|
||||
and again after exit is observed, so a memory failure between those steps
|
||||
is included in the first exit report. Runtimes report a reached limit
|
||||
differently (Bun 1.4.2 exits 1
|
||||
after a JavaScript out-of-memory exception or 3 after a panic), so callers
|
||||
classify by the notification. `inspect` reports `jobMemoryLimit` only after
|
||||
creating such a job and port. An artifact without that capability refuses a
|
||||
requested limit rather than launching an unlimited process. REA requests
|
||||
limits only for its JavaScript helpers when running on Bun, which ignores
|
||||
their V8 heap flags; Node.js launches are unchanged.
|
||||
- Only the suspended child's default file owner is normalized. Caller token
|
||||
privileges and ownership remain unchanged. Caller CPU affinity and process
|
||||
priority are preserved in the child. Environment names are sorted and
|
||||
@@ -93,7 +110,13 @@ The native lane checks DACL readback independently, source write/replacement
|
||||
rejection, unrelated sibling renames, reparse rejection, immutable snapshots,
|
||||
asynchronous cancellation, cleanup containment,
|
||||
descendant termination, unrelated-process preservation, normal exit, owner
|
||||
close, and forced owner exit. An optional independently built
|
||||
close, forced owner exit, and Job Object commit limits: invalid limit
|
||||
rejection, a 128 MiB-limited child allocating external Buffer memory that the
|
||||
V8 heap flags cannot bound (limit notification, peak commit at the limit, all
|
||||
members settled, with the notification present in the first native exit report
|
||||
and the wrapper's exit event), and a control under the same limit that
|
||||
completes normally.
|
||||
An optional independently built
|
||||
`tests/fixtures/windows/processBoundary.cc` executable adds breakaway and
|
||||
caller/child token observations and an in-place ancestor reparse attempt:
|
||||
|
||||
|
||||
@@ -96,6 +96,10 @@ struct Runtime : Resource {
|
||||
};
|
||||
struct Process : Resource {
|
||||
Handle job, process, stdoutRead, stderrRead, stdinWrite;
|
||||
// Present only for a job with a commit limit; receives the job's notifications.
|
||||
Handle notifications;
|
||||
ULONGLONG memoryLimit = 0;
|
||||
bool memoryLimitReached = false;
|
||||
bool inputPending = false;
|
||||
DWORD pid = 0;
|
||||
Process() : Resource(Kind::Process) {}
|
||||
@@ -125,7 +129,7 @@ napi_value filesystemCall(napi_env env, const std::wstring& operation,
|
||||
const std::vector<napi_value>& args);
|
||||
std::unique_ptr<Process> spawnProcess(const std::wstring& command, const std::wstring& commandLine,
|
||||
const std::wstring& cwd, const std::vector<std::wstring>& environment,
|
||||
bool protocolStdin = false);
|
||||
bool protocolStdin = false, ULONGLONG memoryLimit = 0);
|
||||
napi_value processCall(napi_env env, const std::wstring& operation,
|
||||
const std::vector<napi_value>& args);
|
||||
napi_value inspect(napi_env env);
|
||||
|
||||
@@ -19,21 +19,55 @@ public:
|
||||
}
|
||||
};
|
||||
|
||||
static Handle createJob() {
|
||||
// A nonzero memory limit is a commit limit for the whole job; zero keeps only
|
||||
// kill-on-owner-close. The policy read back must match exactly.
|
||||
static Handle createJob(ULONGLONG memoryLimit = 0) {
|
||||
Handle job(CreateJobObjectW(nullptr, nullptr));
|
||||
require(job.valid(), "Create owned Job Object failed");
|
||||
const DWORD flags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE | (memoryLimit != 0 ? JOB_OBJECT_LIMIT_JOB_MEMORY : 0);
|
||||
JOBOBJECT_EXTENDED_LIMIT_INFORMATION policy{};
|
||||
policy.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;
|
||||
policy.BasicLimitInformation.LimitFlags = flags;
|
||||
policy.JobMemoryLimit = static_cast<SIZE_T>(memoryLimit);
|
||||
require(SetInformationJobObject(job.get(), JobObjectExtendedLimitInformation, &policy, sizeof(policy)),
|
||||
"Apply kill-on-owner-close Job Object policy failed");
|
||||
JOBOBJECT_EXTENDED_LIMIT_INFORMATION actual{};
|
||||
require(QueryInformationJobObject(job.get(), JobObjectExtendedLimitInformation, &actual, sizeof(actual), nullptr),
|
||||
"Read Job Object policy failed");
|
||||
require(actual.BasicLimitInformation.LimitFlags == JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE,
|
||||
require(actual.BasicLimitInformation.LimitFlags == flags,
|
||||
"Job Object policy permits unexpected breakaway", L"", ERROR_ACCESS_DENIED);
|
||||
require(memoryLimit == 0 || static_cast<ULONGLONG>(actual.JobMemoryLimit) == memoryLimit,
|
||||
"Job Object memory limit was not applied exactly", L"", ERROR_INVALID_PARAMETER);
|
||||
return job;
|
||||
}
|
||||
|
||||
// Associate a private completion port before any process joins the job, so
|
||||
// the job's commit-limit notification cannot be missed.
|
||||
static Handle createNotifications(HANDLE job) {
|
||||
Handle port(CreateIoCompletionPort(INVALID_HANDLE_VALUE, nullptr, 0, 1));
|
||||
require(port.valid(), "Create owned Job Object notification port failed");
|
||||
JOBOBJECT_ASSOCIATE_COMPLETION_PORT association{};
|
||||
association.CompletionKey = job;
|
||||
association.CompletionPort = port.get();
|
||||
require(SetInformationJobObject(job, JobObjectAssociateCompletionPortInformation, &association, sizeof(association)),
|
||||
"Associate owned Job Object notification port failed");
|
||||
return port;
|
||||
}
|
||||
|
||||
// JavaScript numbers cross the boundary as doubles; accept only an exact
|
||||
// positive safe integer that is a whole number of pages, so read-back is exact.
|
||||
static ULONGLONG jobMemoryLimit(napi_env env, napi_value value) {
|
||||
const double requested = numeric(env, value);
|
||||
require(requested >= 1 && requested <= 9007199254740991.0 &&
|
||||
static_cast<double>(static_cast<ULONGLONG>(requested)) == requested,
|
||||
"Job memory limit must be a positive safe integer", L"", ERROR_INVALID_PARAMETER);
|
||||
SYSTEM_INFO system{};
|
||||
GetSystemInfo(&system);
|
||||
const auto limit = static_cast<ULONGLONG>(requested);
|
||||
require(system.dwPageSize != 0 && limit % system.dwPageSize == 0,
|
||||
"Job memory limit must be a whole number of pages", L"", ERROR_INVALID_PARAMETER);
|
||||
return limit;
|
||||
}
|
||||
|
||||
static void pipe(Handle& read, Handle& write, bool parentReads = true) {
|
||||
SECURITY_ATTRIBUTES attributes{sizeof(SECURITY_ATTRIBUTES), nullptr, TRUE};
|
||||
HANDLE rawRead, rawWrite;
|
||||
@@ -44,12 +78,16 @@ static void pipe(Handle& read, Handle& write, bool parentReads = true) {
|
||||
|
||||
std::unique_ptr<Process> spawnProcess(const std::wstring& command, const std::wstring& commandLine,
|
||||
const std::wstring& cwd, const std::vector<std::wstring>& environment,
|
||||
bool protocolStdin) {
|
||||
bool protocolStdin, ULONGLONG memoryLimit) {
|
||||
const auto executable = ordinaryDriveSeparators(command);
|
||||
require(executable.size() >= 3 && executable[1] == L':' && executable[2] == L'\\',
|
||||
"Owned Windows process requires an absolute executable path", command, ERROR_INVALID_PARAMETER);
|
||||
auto result = std::make_unique<Process>();
|
||||
result->job = createJob();
|
||||
result->job = createJob(memoryLimit);
|
||||
if (memoryLimit != 0) {
|
||||
result->notifications = createNotifications(result->job.get());
|
||||
result->memoryLimit = memoryLimit;
|
||||
}
|
||||
Handle stdoutWrite, stderrWrite;
|
||||
pipe(result->stdoutRead, stdoutWrite); pipe(result->stderrRead, stderrWrite);
|
||||
SECURITY_ATTRIBUTES security{sizeof(SECURITY_ATTRIBUTES), nullptr, TRUE};
|
||||
@@ -170,6 +208,29 @@ static DWORD activeProcesses(Process& process) {
|
||||
return state.ActiveProcesses;
|
||||
}
|
||||
|
||||
// Drain queued job notifications without waiting. The commit-limit message is
|
||||
// posted when an allocation fails, before the process can exit. An observed
|
||||
// exit must be followed by another drain before its diagnostics are published.
|
||||
static void observeNotifications(Process& process) {
|
||||
if (!process.notifications.valid()) return;
|
||||
DWORD message = 0;
|
||||
ULONG_PTR key = 0;
|
||||
LPOVERLAPPED detail = nullptr;
|
||||
while (GetQueuedCompletionStatus(process.notifications.get(), &message, &key, &detail, 0)) {
|
||||
if (key == reinterpret_cast<ULONG_PTR>(process.job.get()) && message == JOB_OBJECT_MSG_JOB_MEMORY_LIMIT)
|
||||
process.memoryLimitReached = true;
|
||||
}
|
||||
const auto error = GetLastError();
|
||||
require(detail == nullptr && error == WAIT_TIMEOUT, "Observe owned Job Object notifications failed", L"", error);
|
||||
}
|
||||
|
||||
static ULONGLONG peakJobMemory(Process& process) {
|
||||
JOBOBJECT_EXTENDED_LIMIT_INFORMATION state{};
|
||||
require(QueryInformationJobObject(process.job.get(), JobObjectExtendedLimitInformation, &state, sizeof(state), nullptr),
|
||||
"Observe owned Job Object memory failed");
|
||||
return static_cast<ULONGLONG>(state.PeakJobMemoryUsed);
|
||||
}
|
||||
|
||||
static napi_value callerToken(napi_env env) {
|
||||
HANDLE raw;
|
||||
require(OpenProcessToken(GetCurrentProcess(), TOKEN_QUERY | TOKEN_DUPLICATE, &raw), "Read caller token failed");
|
||||
@@ -284,16 +345,20 @@ napi_value processCall(napi_env env, const std::wstring& operation, const std::v
|
||||
return callerToken(env);
|
||||
}
|
||||
if (operation == L"process_spawn") {
|
||||
require(args.size() == 4 || args.size() == 5, "Wrong native process launch argument count", L"", ERROR_INVALID_PARAMETER);
|
||||
require(args.size() >= 4 && args.size() <= 6, "Wrong native process launch argument count", L"", ERROR_INVALID_PARAMETER);
|
||||
bool protocolStdin = false;
|
||||
if (args.size() == 5) check(napi_get_value_bool(env, args[4], &protocolStdin));
|
||||
if (args.size() >= 5) check(napi_get_value_bool(env, args[4], &protocolStdin));
|
||||
const ULONGLONG memoryLimit = args.size() == 6 ? jobMemoryLimit(env, args[5]) : 0;
|
||||
std::vector<std::wstring> environment;
|
||||
for (const auto& entry : array(env, args[3])) environment.push_back(wide(env, entry));
|
||||
auto process = spawnProcess(wide(env, args[0]), wide(env, args[1]), wide(env, args[2]), environment, protocolStdin);
|
||||
auto process = spawnProcess(wide(env, args[0]), wide(env, args[1]), wide(env, args[2]), environment, protocolStdin,
|
||||
memoryLimit);
|
||||
auto result = object(env);
|
||||
set(env, result, "pid", number(env, process->pid));
|
||||
set(env, result, "jobAssigned", boolean(env, true));
|
||||
set(env, result, "killOnOwnerClose", boolean(env, true));
|
||||
if (process->memoryLimit != 0)
|
||||
set(env, result, "jobMemoryLimitBytes", number(env, static_cast<double>(process->memoryLimit)));
|
||||
set(env, result, "handle", wrap(env, std::move(process)));
|
||||
return result;
|
||||
}
|
||||
@@ -314,12 +379,21 @@ napi_value processCall(napi_env env, const std::wstring& operation, const std::v
|
||||
set(env, result, "stderr", readOutput(env, process.stderrRead, stderrEnded));
|
||||
set(env, result, "stdoutEnded", boolean(env, stdoutEnded));
|
||||
set(env, result, "stderrEnded", boolean(env, stderrEnded));
|
||||
observeNotifications(process);
|
||||
const auto exited = WaitForSingleObject(process.process.get(), 0);
|
||||
require(exited == WAIT_OBJECT_0 || exited == WAIT_TIMEOUT, "Observe owned process exit failed");
|
||||
DWORD code = 0;
|
||||
if (exited == WAIT_OBJECT_0) require(GetExitCodeProcess(process.process.get(), &code), "Read owned process exit code failed");
|
||||
if (exited == WAIT_OBJECT_0) {
|
||||
// The child may have reached its limit and exited after the first drain.
|
||||
observeNotifications(process);
|
||||
require(GetExitCodeProcess(process.process.get(), &code), "Read owned process exit code failed");
|
||||
}
|
||||
set(env, result, "exitCode", exited == WAIT_OBJECT_0 ? number(env, code) : null(env));
|
||||
set(env, result, "activeProcesses", number(env, activeProcesses(process)));
|
||||
if (process.memoryLimit != 0) {
|
||||
set(env, result, "memoryLimitReached", boolean(env, process.memoryLimitReached));
|
||||
set(env, result, "peakJobMemoryBytes", number(env, static_cast<double>(peakJobMemory(process))));
|
||||
}
|
||||
return result;
|
||||
}
|
||||
if (operation == L"process_alive") return number(env, activeProcesses(process));
|
||||
@@ -331,6 +405,7 @@ napi_value processCall(napi_env env, const std::wstring& operation, const std::v
|
||||
if (operation == L"process_close") {
|
||||
// Closing the job is the crash-safe authority; no PID enumeration is used.
|
||||
process.job.reset(); process.process.reset(); process.stdoutRead.reset(); process.stderrRead.reset(); process.stdinWrite.reset();
|
||||
process.notifications.reset();
|
||||
process.closed = true;
|
||||
return null(env);
|
||||
}
|
||||
@@ -348,6 +423,9 @@ napi_value inspect(napi_env env) {
|
||||
Attributes attributes(1);
|
||||
HANDLE raw = job.get();
|
||||
attributes.add(PROC_THREAD_ATTRIBUTE_JOB_LIST, &raw, sizeof(raw));
|
||||
// Establish the commit-limit policy and its notification port on this host.
|
||||
auto limited = createJob(64ULL * 1024 * 1024);
|
||||
auto notifications = createNotifications(limited.get());
|
||||
auto result = object(env);
|
||||
set(env, result, "abiVersion", number(env, 1));
|
||||
set(env, result, "nodeApiVersion", number(env, 8));
|
||||
@@ -357,6 +435,7 @@ napi_value inspect(napi_env env) {
|
||||
set(env, result, "atomicJobAssignment", boolean(env, true));
|
||||
set(env, result, "killOnOwnerClose", boolean(env, true));
|
||||
set(env, result, "protocolStdin", boolean(env, true));
|
||||
set(env, result, "jobMemoryLimit", boolean(env, true));
|
||||
return result;
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user