perf(process): reuse a verified Darwin ownership helper across processes (#8318)

* perf(process): reuse a verified Darwin ownership helper across processes

Every REA process on macOS compiled the Swift ownership helper into a fresh
temporary root and then paid macOS's first-launch assessment of that new
executable: 408 ms + 325 ms of a 1.4 s `rea inspect-macho /bin/ls`.

With the default xcrun compiler and module cache, the reader now keeps one
content-addressed copy under ~/Library/Caches/rea/process-helper/<key>/.
The key covers the helper sources, toolchain selection, architecture and
Darwin release. A copy is executed only after the cache root, entry and
files are verified as current-user owned, not group- or world-writable,
not links, and digest-matched to the entry manifest. Malformed entries
owned by the user are removed and rebuilt; linked or foreign entries are
left untouched and the per-process compile runs as before. Any cache
failure falls back to that compile, so the cache never fails an ownership
operation.

A miss compiles exactly as before, then publishes a copy through an atomic
rename, keeping the first concurrent publisher, and runs the shared copy.
That way its first-launch assessment is paid once. Per-run roots keep their
existing cleanup; published entries are never removed by close or exit.
An explicit REA_DARWIN_SWIFTC_PATH or REA_DARWIN_MODULE_CACHE_PATH keeps the
per-process compile and writes nothing to the cache. `rea uninstall
--purge-data` removes the namespace.

Fixes #2483

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(process): harden the shared Darwin helper cache

Address review findings on the shared ownership-helper cache:

- Verify every real parent of the cache root: each must belong to the user
  or root and not be writable by others (sticky root-owned directories
  excepted), so no other user can rename a verified entry. Execute through
  the real path.
- Open manifest and helper through the owned non-blocking regular-file
  reader, so a FIFO cannot block preparation, and retain handles whose close
  fails.
- Key entries on the swiftc and SDK that xcrun itself resolves, including
  their real paths, sizes and modification times. DEVELOPER_DIR, TOOLCHAINS,
  xcode-select and in-place updates therefore select new entries.
- Never remove or replace an invalid or published entry. Removal could race
  another process that has verified or is running that path, so such an
  entry falls back to the per-process compile.
- Give the cache a close() that retries retained staging removals and
  handle closes and reports ProviderCleanupError while any remain. The
  reader closes it with its per-run root.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(process): keep cache lookups owned and fall back from unlaunchable helpers

Address the second round of review on the shared ownership-helper cache:

- Pass the preparation signal to both xcrun toolchain lookups, and wait for
  every started lookup before reporting the first failure. Cancellation and
  close then stop a stalled lookup instead of waiting out its deadline.
  A failed or cancelled key computation is retried by the next lookup.
- When the verified shared copy cannot launch (EACCES, EPERM, ENOEXEC,
  ENOENT or an unrequested SIGKILL, for example from a noexec home volume),
  the reader stops using the cache. It reuses its own compiled helper or
  compiles one, and retries the read once.
- Anchor the cache's returned cleanup errors as cleanupRoot does, so every
  error returned while cleanup was pending resolves after a retry succeeds.
- Build the cache path with appendCallerPath, so a home with symlinks and
  `..` keeps the caller's meaning instead of being normalized lexically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(process): keep concurrent fallbacks and dual cleanup failures

Each helper read now records before awaiting whether it launched the shared
copy, so a concurrent read's fallback cannot hide its own. When the per-run
root and the cache's retained resources both fail one close, the returned
error composes both. It stays incomplete until both owners recover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(process): refuse shared helpers that an ACL lets others write

macOS ACLs can grant writes that the POSIX mode bits do not show. On
Darwin, lookup and publication list the real ancestors, the entry and its
files with `ls -lde`, and refuse sharing when any ACL entry allows a
write-class right. The default `group:everyone deny delete` entries on
macOS home folders are unaffected. The unlaunchable-helper test now removes
the execute bit from a verified copy, which refuses launch on both Linux
and macOS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

src/process/DarwinProcessRunTokenReader.ts

@@ -20,6 +20,7 @@ import type {
 import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
 import { safeParseJson } from "../domain/safeJson.js";
 import { boundedUtf8Text } from "./boundedUtf8Text.js";
+import { createDarwinHelperCache } from "./DarwinHelperCache.js";
 import { snapshotEnvironment } from "./snapshotEnvironment.js";
 import {
   execFileOutput,
@@ -394,6 +395,19 @@ const waitForCompilation = async (
   });
 };
 
+/** The kernel or policy refused to launch the file, before it could answer. */
+const helperLaunchFailed = (cause: unknown): boolean => {
+  const failure = execFileOutputFailure(cause);
+  // spawn rejects with the bare errno before any output exists.
+  const code =
+    failure?.code ??
+    (cause instanceof Error && "code" in cause ? cause.code : undefined);
+  return (
+    ["EACCES", "EPERM", "ENOEXEC", "ENOENT"].includes(String(code)) ||
+    (failure?.signal === "SIGKILL" && !failure.killed)
+  );
+};
+
 /** Explicit tool selection; direct Swift compilation also needs its matching SDK. */
 export type DarwinProcessReaderCompiler =
   | { readonly kind: "xcrun"; readonly executable: string }
@@ -417,6 +431,13 @@ export const createDarwinProcessRunTokenReader = (
      * reader because cleanup only removes the per-run helper root.
      */
     readonly moduleCachePath?: string;
+    /**
+     * REA-owned per-user directory that reuses a verified compiled helper
+     * across processes with the xcrun compiler. Unset compiles once per
+     * reader, as callers selecting their own compiler, module cache, or write
+     * boundary expect.
+     */
+    readonly helperCacheDirectory?: string;
   } = {},
 ) => {
   const compiler = options.compiler ?? {
@@ -429,8 +450,21 @@ export const createDarwinProcessRunTokenReader = (
     environment,
   });
   const moduleCachePath = options.moduleCachePath ?? "";
+  const helperCache =
+    options.helperCacheDirectory === undefined || compiler.kind !== "xcrun"
+      ? undefined
+      : createDarwinHelperCache({
+          directory: options.helperCacheDirectory,
+          sources: sourceFiles,
+          xcrun: compiler.executable,
+          environment,
+        });
   let root: ProcessReaderRoot | undefined;
   let executable: string | undefined;
+  // The shared copy runs while it launches; this reader's own helper is kept.
+  let sharedExecutable: string | undefined;
+  let perRunExecutable: string | undefined;
+  let bypassCache = false;
   let compilation: ProcessReaderCompilation | undefined;
   let closePromise: Promise<void> | undefined;
   let closed = false;
@@ -489,6 +523,16 @@ export const createDarwinProcessRunTokenReader = (
           "Direct Darwin helper compilation requires absolute REA_DARWIN_SWIFTC_PATH and SDKROOT paths selecting a compatible installed Swift compiler and macOS SDK.",
         );
       }
+      const cached = bypassCache
+        ? undefined
+        : await helperCache?.lookup(signal);
+      if (signal.aborted || closed) throw abortReason(signal);
+      // A verified shared entry needs no per-run root and is never removed.
+      if (cached?.kind === "hit") {
+        sharedExecutable = cached.executable;
+        executable = cached.executable;
+        return cached.executable;
+      }
       operationRoot = {
         path: await mkdtemp(join(tmpdir(), "rea-process-token-")),
       };
@@ -532,8 +576,16 @@ export const createDarwinProcessRunTokenReader = (
         throw compilerInspectionError(cause, compilerTimeoutMs, compiler);
       }
       if (signal.aborted || closed) throw abortReason(signal);
-      executable = output;
-      return output;
+      // The per-run root stays owned until close even when the shared copy runs.
+      const shared =
+        cached?.kind === "miss"
+          ? await helperCache?.publish(cached.key, output, signal)
+          : undefined;
+      if (signal.aborted || closed) throw abortReason(signal);
+      perRunExecutable = output;
+      sharedExecutable = shared;
+      executable = shared ?? output;
+      return executable;
     } catch (cause: unknown) {
       operation.primaryFailure = { cause };
       if (operationRoot !== undefined) {
@@ -644,8 +696,7 @@ export const createDarwinProcessRunTokenReader = (
   ): Promise<string> => {
     if (closed)
       return Promise.reject(new ProcessAdmissionError("reader_closed"));
-    const pending = (async () => {
-      const binary = await compile(signal);
+    const execute = async (binary: string) => {
       const { stdout } = await execFileOutput(binary, arguments_, {
         env: environment,
         timeout: 15_000,
@@ -654,6 +705,22 @@ export const createDarwinProcessRunTokenReader = (
       });
       signal?.throwIfAborted();
       return stdout;
+    };
+    const pending = (async () => {
+      const binary = await compile(signal);
+      // Decide before awaiting: a concurrent fallback may already switch.
+      const usedShared = binary === sharedExecutable;
+      try {
+        return await execute(binary);
+      } catch (cause: unknown) {
+        if (!usedShared || !helperLaunchFailed(cause)) throw cause;
+        // The shared copy cannot run here, for example from a noexec home
+        // volume; this reader keeps its own helper from now on.
+        bypassCache = true;
+        sharedExecutable = undefined;
+        if (executable === binary) executable = perRunExecutable;
+        return execute(await compile(signal));
+      }
     })();
     activeCommands.add(pending);
     return pending.finally(() => {
@@ -774,7 +841,24 @@ export const createDarwinProcessRunTokenReader = (
         await operation.promise;
       }
       await Promise.allSettled(activeCommands);
-      await cleanupRoot();
+      // Each owner keeps its own failed cleanup for the next close attempt;
+      // a failure of both stays incomplete until both owners recover.
+      const [rootCleanup, cacheCleanup] = await Promise.allSettled([
+        cleanupRoot(),
+        helperCache?.close(),
+      ]);
+      if (
+        rootCleanup.status === "rejected" &&
+        cacheCleanup.status === "rejected" &&
+        rootCleanup.reason instanceof ProviderCleanupError &&
+        cacheCleanup.reason instanceof ProviderCleanupError
+      )
+        throw analysisErrorWithCleanupFailure(
+          rootCleanup.reason,
+          cacheCleanup.reason,
+        ).bindCleanupDependencies([rootCleanup.reason, cacheCleanup.reason]);
+      if (rootCleanup.status === "rejected") throw rootCleanup.reason;
+      if (cacheCleanup.status === "rejected") throw cacheCleanup.reason;
     })().catch((cause: unknown) => {
       if (closePromise === pendingClose) closePromise = undefined;
       throw cause;