feat(config): make the bridge request ceiling configurable (#6376)

* fix(reconstruction): dedupe residual unknown ids with uniqueSorted

Current main calls uniqueBy, which is not defined, so typecheck fails.
Residual unknown ids are strings, and the adjacent lists already use
uniqueSorted.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* feat(config): make the bridge request ceiling configurable

REA_BRIDGE_MAX_REQUEST_BYTES replaces the hard-coded 16 MiB Hopper and
Cutter request payload ceiling. The default stays 16777216 bytes. Values
outside 7464 to 268435456 bytes are configuration_invalid.

Part of #6176.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): keep cleanup ownership when a sent command's reply is oversized

An authenticated command may already have changed the Cutter document
when its reply exceeds the frame ceiling. Return the existing unknown
execution, with the ceiling diagnostic in the message, so receipt
cleanup still waits. A frame that overflows before the command is sent
remains a resource constraint.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* style: format the bridge request ceiling sources

oxfmt --check failed on the five files that set and test
REA_BRIDGE_MAX_REQUEST_BYTES. Formatting only; an oversized Cutter
reply still stays an unknown execution.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): exclude the LF from the response frame budget

REA_BRIDGE_MAX_REQUEST_BYTES counts payload bytes. The Cutter response
reader and authenticated encoder were reserving one byte for the
terminating LF, so an exact-limit response was rejected. A reply that
exceeds the ceiling after the command was sent still stays an unknown
execution.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(bridge): measure escaped JSON before allocating the request

A command of U+0001 characters can expand sixfold. JSON.stringify of
that expansion can throw past Node's string ceiling before the payload
check. Count the escaped size within REA_BRIDGE_MAX_REQUEST_BYTES
first and refuse it as a resource constraint.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(config): derive the bridge request maximum from JSON expansion

A 256 MiB ceiling still lets a shorter U+0001 command expand past Node's
536870888-character string limit. The maximum is now
floor((536870888 - 296) / 7) = 76695798, the longest such command whose
authenticated frame fits in one string. Requests are still measured
before JSON.stringify.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(bridge): size the Cutter frame deadline and send the measured Hopper payload

The authenticated Cutter command is encoded after the challenge, so a
one-second read rejected an in-budget frame. That read now allows one
second per mebibyte of the ceiling and never waits longer than 25
seconds. Hello stays at one second. Hopper writes the payload it already
measured instead of serializing the same request again.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(config): bound the bridge request maximum by the serialized string

The ceiling already measures escaped JSON, so dividing Node's string limit by the seven-byte expansion rejected encoded payloads that still fit. The maximum is 536870887, one byte under that string ceiling, leaving room for the framing LF.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(hopper): serialize a bridge request when it reaches the wire

The queue kept each encoded frame, plus the caller's params, until Hopper finished the active request. A raised ceiling made a few waiting calls enough to exhaust the heap. The FIFO now keeps the params, and the frame is encoded only for the request being written.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): hash the bridge proof prefix and payload separately

An exact-limit Cutter frame plus the HMAC prefix exceeds Node's single-string ceiling, so the client threw before sending an admitted command. The prefix and payload are now separate HMAC updates, including the response proof.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): keep recovery fields when a status path exceeds the frame

A long current_file made the status reply miss the ceiling, and the fallback then dropped session_id and the completion receipts. The status now omits the path and version and still returns the session, connection proof, and up to 64 receipts. That required recovery frame is smaller than the 7464 floor.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): report an unknown path when status identity does not fit

A bounded status kept the new document generation and omitted the path, so the client filled in the descriptor's older path. The observed path is sent when it fits. When it does not, current_file is null and is not replaced with that stale path.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): count structured replies by their encoded JSON size

The response guard charged 16 bytes per primitive and two bytes per list slot, so a dense array under the configured ceiling was replaced with output_truncated. Numbers, booleans, null, and container punctuation now use their JSON lengths.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): stream authenticated responses in bounded chunks

A fitting reply was signed from retained raw bytes, escaped bytes, and a
joined frame, so a ceiling near 512 MiB could exhaust Cutter before send.
Measure and sign JSON pieces, then write the same frame in 64 KiB chunks.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(bridge): keep Cutter admission from stalling or hiding sessions

Read an unauthenticated frame off the accept loop. A command reply that
does not fit keeps completion and generation and reports an oversized path
as null. Discovery reports a ceiling mismatch instead of an absent session.
The maximum HMAC proof is opt-in, and both bridges reject non-ASCII digits.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): bound frame scans and client response copies

Search only newly read bytes for the framing LF. Unescape each socket
chunk into one response string, check the proof, and then parse it.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): keep completion, admission, and dense replies bounded

Report completed_connection only through connections that have
terminated. Cap connection threads and keep the listener up if one
fails to start. Batch scalar JSON and fail discovery when any ceiling
mismatches.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): hash requests in parts and scan escapes once

Update the client-proof prefix and request separately. Stop the response
decoder at the first quote or backslash instead of rescanning the tail.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(cutter): confirm live ceiling mismatches and omitted output

A stale descriptor from a previous ceiling is not a discovery mismatch
until its hello proves a live authenticated bridge. A response discarded
for the frame ceiling after dispatch reports output_truncated.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(cutter): close the unauthenticated ceiling-mismatch fixture

The listener that fails the hello proof is destroyed after its challenge
so discovery cleanup does not wait on that socket.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>

bridge/cutter/rea_bridge.py

@@ -11,6 +11,7 @@ import json
 import hashlib
 import hmac
 import os
+import re
 import secrets
 import socket
 import stat
@@ -33,9 +34,151 @@ except ImportError:
     from PySide2.QtCore import QCoreApplication, QObject, Signal
 
 
-MAX_FRAME_BYTES = 16 * 1024 * 1024
+# Request and response frame budget in bytes, excluding the trailing LF.
+# REA_BRIDGE_MAX_REQUEST_BYTES selects it. Unset stays 16777216.
+# 7464 is the authenticated frame of the largest bounded symbol name.
+# The 64-receipt recovery status is smaller with current_file null. A path
+# that fits is kept. A path that does not fit is null, not the descriptor path.
+# 536870887 is Node's 536870888-character string ceiling minus the framing LF.
+# The ceiling is the encoded payload; JSON expansion is measured before it.
+# A fitting response is signed from bounded JSON pieces and written in 64 KiB
+# chunks, so this maximum does not retain several copies of the frame.
+_BRIDGE_MAX_REQUEST_BYTES_SETTING = "REA_BRIDGE_MAX_REQUEST_BYTES"
+DEFAULT_BRIDGE_MAX_REQUEST_BYTES = 16 * 1024 * 1024
+MINIMUM_BRIDGE_MAX_REQUEST_BYTES = 7464
+MAXIMUM_BRIDGE_MAX_REQUEST_BYTES = 536870887
+# The outer authenticated frame around the escaped JSON payload, excluding LF.
+_AUTHENTICATED_PREFIX = b'{"response":"'
+_AUTHENTICATED_PROOF_MARK = b'","proof":"'
+_PROOF_HEX_LENGTH = 64
+_AUTHENTICATED_FRAME_OVERHEAD = (
+    len(_AUTHENTICATED_PREFIX)
+    + len(_AUTHENTICATED_PROOF_MARK)
+    + _PROOF_HEX_LENGTH
+    + len(b'"}')
+)
+_AUTHENTICATED_WRITE_BUFFER_BYTES = 64 * 1024
+# Raw JSON is produced in pieces no larger than this many characters.
+# One character expands to at most six JSON bytes (\u00XX) before the outer
+# string escape, so a piece stays far below the frame ceiling.
+_JSON_CHUNK_CHARS = 64 * 1024
+# Fixed admission cap. It does not follow REA_BRIDGE_MAX_REQUEST_BYTES.
+# A further accept is closed. Starting a worker must not stop the listener.
+MAX_CONNECTION_WORKERS = 8
+
+
+def _close_socket(connection: socket.socket) -> None:
+    try:
+        connection.close()
+    except OSError:
+        pass
+
+
+def _bridge_max_request_bytes_constraint():
+    return (
+        "REA_BRIDGE_MAX_REQUEST_BYTES must be a positive decimal integer "
+        "from %d to %d bytes; the default is %d bytes (16 MiB). "
+        "A value below %d bytes cannot carry the largest admitted "
+        "symbol-name command, and a larger value is refused so the encoded "
+        "payload plus its framing LF stays one string under Node's "
+        "single-string ceiling."
+        % (
+            MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
+            MAXIMUM_BRIDGE_MAX_REQUEST_BYTES,
+            DEFAULT_BRIDGE_MAX_REQUEST_BYTES,
+            MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
+        )
+    )
+
+
+def _parse_bridge_max_request_bytes(raw):
+    """Refuse a non-decimal or out-of-range ceiling. Never fall back."""
+    constraint = _bridge_max_request_bytes_constraint()
+    if isinstance(raw, bool):
+        raise RuntimeError(constraint)
+    if isinstance(raw, int):
+        value = raw
+    elif isinstance(raw, str):
+        # ASCII digits only. str.isdigit() accepts numerals such as U+0668
+        # that the TypeScript setting rejects.
+        if re.fullmatch(r"[1-9][0-9]*", raw) is None:
+            raise RuntimeError(constraint)
+        value = int(raw)
+    else:
+        raise RuntimeError(constraint)
+    if (
+        value < MINIMUM_BRIDGE_MAX_REQUEST_BYTES
+        or value > MAXIMUM_BRIDGE_MAX_REQUEST_BYTES
+    ):
+        raise RuntimeError(constraint)
+    return value
+
+
+def _resolve_bridge_max_request_bytes():
+    """Prefer an injected global, then the process environment, then 16 MiB."""
+    injected = globals().get(_BRIDGE_MAX_REQUEST_BYTES_SETTING)
+    if injected is not None:
+        return _parse_bridge_max_request_bytes(injected), "setting"
+    raw = os.environ.get(_BRIDGE_MAX_REQUEST_BYTES_SETTING)
+    if raw is None:
+        return DEFAULT_BRIDGE_MAX_REQUEST_BYTES, "default"
+    return _parse_bridge_max_request_bytes(raw), "setting"
+
+
+MAX_FRAME_BYTES, BRIDGE_MAX_REQUEST_BYTES_SOURCE = (
+    _resolve_bridge_max_request_bytes()
+)
+
+
+def _bridge_max_request_bytes_recovery():
+    return (
+        "The effective REA_BRIDGE_MAX_REQUEST_BYTES value is %d bytes "
+        "(source %s; default %d, floor %d, maximum %d). "
+        "Set REA_BRIDGE_MAX_REQUEST_BYTES in the environment that starts REA "
+        "to a decimal integer from %d to %d, then restart REA. "
+        "Restart Hopper or Cutter from that same environment so the bridge "
+        "uses this ceiling."
+        % (
+            MAX_FRAME_BYTES,
+            BRIDGE_MAX_REQUEST_BYTES_SOURCE,
+            DEFAULT_BRIDGE_MAX_REQUEST_BYTES,
+            MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
+            MAXIMUM_BRIDGE_MAX_REQUEST_BYTES,
+            MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
+            MAXIMUM_BRIDGE_MAX_REQUEST_BYTES,
+        )
+    )
+
+
+def _frame_limit_exceeded():
+    return ValueError(
+        "frame-limit-exceeded: Bridge request exceeds the %d-byte payload limit. %s"
+        % (MAX_FRAME_BYTES, _bridge_max_request_bytes_recovery())
+    )
+
+
 SOCKET_TIMEOUT_SECONDS = 20
 AUTH_FRAME_TIMEOUT_SECONDS = 1
+# The client request deadline. The authenticated frame read may use up to
+# this long; it does not grow past it when the byte ceiling is raised.
+REQUEST_DEADLINE_SECONDS = 25
+
+
+def _authenticated_frame_timeout_seconds():
+    """Time to hash and transfer one maximum-sized authenticated frame.
+
+    The hello frame stays at one second. The following command frame is
+    built only after that challenge, so its read budget follows the
+    configured payload size, at one second per mebibyte, and never exceeds
+    the 25 second client deadline.
+    """
+    mebibytes = (MAX_FRAME_BYTES + (1024 * 1024) - 1) // (1024 * 1024)
+    return min(
+        REQUEST_DEADLINE_SECONDS,
+        max(AUTH_FRAME_TIMEOUT_SECONDS, mebibytes),
+    )
+
+
 MAX_DESCRIPTOR_BYTES = 64 * 1024
 MAX_COMPLETION_RECEIPTS = 64
 
@@ -480,6 +623,17 @@ class _BridgeServer(threading.Thread):
         self.token = secrets.token_urlsafe(32)
         self.poisoned = False
         self.connection_ordinal = 0
+        # The client proof is inside the authenticated frame, so that read
+        # cannot start after proof. It runs off the accept loop. One
+        # authenticated command still runs at a time. completed_connection
+        # is the longest prefix of these ordinals whose handlers have
+        # terminated, not "this status ordinal minus one".
+        self._command_lock = threading.Lock()
+        self._connections_lock = threading.Lock()
+        self._connections: set[socket.socket] = set()
+        self._workers: list[threading.Thread] = []
+        self._finished_through = 0
+        self._finished_ordinals: set[int] = set()
         self.directory = _bridge_directory()
         self.descriptor = self.directory / f"cutter-{os.getpid()}-{self.session_id}.json"
 
@@ -500,6 +654,7 @@ class _BridgeServer(threading.Thread):
             "current_file": self.dispatcher.current_file,
             "cutter_version": self.dispatcher.cutter_version,
             "identity_status": "partial",
+            "max_request_bytes": MAX_FRAME_BYTES,
         }
         if os.name == "nt":
             _write_windows_descriptor(self.directory, self.descriptor, json.dumps(payload).encode("utf-8"))
@@ -541,8 +696,46 @@ class _BridgeServer(threading.Thread):
                 continue
             except OSError:
                 break
+            with self._connections_lock:
+                self._workers = [
+                    worker for worker in self._workers if worker.is_alive()
+                ]
+                at_capacity = len(self._workers) >= MAX_CONNECTION_WORKERS
+            if at_capacity:
+                _close_socket(connection)
+                continue
             self.connection_ordinal += 1
             connection_ordinal = self.connection_ordinal
+            worker = threading.Thread(
+                target=self._serve_connection,
+                args=(connection, connection_ordinal),
+                name="rea-cutter-bridge-connection",
+                daemon=True,
+            )
+            with self._connections_lock:
+                self._connections.add(connection)
+                self._workers.append(worker)
+            try:
+                worker.start()
+            except RuntimeError:
+                with self._connections_lock:
+                    self._connections.discard(connection)
+                    if worker in self._workers:
+                        self._workers.remove(worker)
+                self._note_connection_finished(connection_ordinal)
+                _close_socket(connection)
+        with self._connections_lock:
+            workers = list(self._workers)
+        for worker in workers:
+            worker.join(timeout=1)
+        self.server.close()
+        try:
+            self.descriptor.unlink(missing_ok=True)
+        except OSError:
+            pass
+
+    def _serve_connection(self, connection: socket.socket, connection_ordinal: int) -> None:
+        try:
             with connection:
                 connection.settimeout(AUTH_FRAME_TIMEOUT_SECONDS)
                 authenticated = False
@@ -550,60 +743,73 @@ class _BridgeServer(threading.Thread):
                     hello = self._receive_frame(connection)
                     if not isinstance(hello, dict) or hello.get("kind") != "rea-auth-v2" or not isinstance(hello.get("nonce"), str) or len(hello["nonce"]) != 64:
                         self._send(connection, {"ok": False, "error": "authentication-failed"})
-                        continue
+                        return
                     client_nonce = hello["nonce"]
                     server_nonce = f"{connection_ordinal:016x}{secrets.token_hex(24)}"
                     server_proof = hmac.new(self.token.encode(), f"rea-cutter-server-v2:{self.session_id}:{client_nonce}:{server_nonce}".encode(), hashlib.sha256).hexdigest()
                     self._send(connection, {"session_id": self.session_id, "nonce": server_nonce, "proof": server_proof})
-                    authenticated_frame = self._receive_frame(connection)
+                    authenticated_frame = self._receive_frame(
+                        connection, _authenticated_frame_timeout_seconds()
+                    )
                     request_bytes = authenticated_frame.get("request") if isinstance(authenticated_frame, dict) else None
-                    expected_proof = hmac.new(self.token.encode(), f"rea-cutter-client-v2:{self.session_id}:{client_nonce}:{server_nonce}:{request_bytes}".encode(), hashlib.sha256).hexdigest()
-                    if not isinstance(request_bytes, str) or not isinstance(authenticated_frame.get("proof"), str) or not hmac.compare_digest(authenticated_frame["proof"], expected_proof):
+                    presented_proof = authenticated_frame.get("proof") if isinstance(authenticated_frame, dict) else None
+                    if not isinstance(request_bytes, str) or not isinstance(presented_proof, str):
                         self._send(connection, {"ok": False, "error": "authentication-failed"})
-                        continue
+                        return
+                    # Update the prefix and the request separately. One joined
+                    # preimage would retain another copy of a maximum-sized command.
+                    expected_proof = hmac.new(self.token.encode(), digestmod=hashlib.sha256)
+                    expected_proof.update(
+                        f"rea-cutter-client-v2:{self.session_id}:{client_nonce}:{server_nonce}:".encode()
+                    )
+                    expected_proof.update(request_bytes.encode())
+                    if not hmac.compare_digest(presented_proof, expected_proof.hexdigest()):
+                        self._send(connection, {"ok": False, "error": "authentication-failed"})
+                        return
                     request = json.loads(request_bytes)
                     if not isinstance(request, dict) or request.get("session_id") != self.session_id:
                         self._send(connection, {"ok": False, "error": "authentication-failed"})
-                        continue
+                        return
                     authenticated = True
-                    connection.settimeout(SOCKET_TIMEOUT_SECONDS)
-                    if self.poisoned and request.get("kind") not in ("status", "ack"):
-                        self._send_authenticated(connection, {
-                            "ok": False,
-                            "error": "session-poisoned-after-ui-timeout",
-                            "execution_state": "not_started",
-                        }, client_nonce, server_nonce)
-                        continue
-                    if (
-                        request.get("kind") == "command"
-                        and not self.dispatcher.can_accept_command()
-                    ):
-                        self._send_authenticated(connection, {
-                            "ok": False,
-                            "error": "completion-receipt-capacity-exceeded",
-                            "execution_state": "not_started",
-                        }, client_nonce, server_nonce)
-                        continue
-                    request["session_id"] = self.session_id
-                    event = threading.Event()
-                    cancelled = threading.Event()
-                    reply: dict[str, Any] = {}
-                    request["_event"] = event
-                    request["_cancelled"] = cancelled
-                    self.dispatcher.dispatch.emit(request, reply)
-                    if not event.wait(SOCKET_TIMEOUT_SECONDS):
-                        cancelled.set()
-                        self.poisoned = True
-                        self._send_authenticated(connection, {
-                            "ok": False,
-                            "error": "ui-thread-request-timeout",
-                            "execution_state": "unknown",
-                            "message": "The UI operation may still complete after this timeout; restart Cutter before issuing further commands.",
-                        }, client_nonce, server_nonce)
-                    else:
-                        if request.get("kind") == "status" and reply.get("ok") is True:
-                            reply["completed_connection"] = f"{connection_ordinal - 1:016x}"
-                        self._send_authenticated(connection, reply, client_nonce, server_nonce)
+                    with self._command_lock:
+                        connection.settimeout(SOCKET_TIMEOUT_SECONDS)
+                        if self.poisoned and request.get("kind") not in ("status", "ack"):
+                            self._send_authenticated(connection, {
+                                "ok": False,
+                                "error": "session-poisoned-after-ui-timeout",
+                                "execution_state": "not_started",
+                            }, client_nonce, server_nonce)
+                            return
+                        if (
+                            request.get("kind") == "command"
+                            and not self.dispatcher.can_accept_command()
+                        ):
+                            self._send_authenticated(connection, {
+                                "ok": False,
+                                "error": "completion-receipt-capacity-exceeded",
+                                "execution_state": "not_started",
+                            }, client_nonce, server_nonce)
+                            return
+                        request["session_id"] = self.session_id
+                        event = threading.Event()
+                        cancelled = threading.Event()
+                        reply: dict[str, Any] = {}
+                        request["_event"] = event
+                        request["_cancelled"] = cancelled
+                        self.dispatcher.dispatch.emit(request, reply)
+                        if not event.wait(SOCKET_TIMEOUT_SECONDS):
+                            cancelled.set()
+                            self.poisoned = True
+                            self._send_authenticated(connection, {
+                                "ok": False,
+                                "error": "ui-thread-request-timeout",
+                                "execution_state": "unknown",
+                                "message": "The UI operation may still complete after this timeout; restart Cutter before issuing further commands.",
+                            }, client_nonce, server_nonce)
+                        else:
+                            if request.get("kind") == "status" and reply.get("ok") is True:
+                                reply["completed_connection"] = self._completion_watermark()
+                            self._send_authenticated(connection, reply, client_nonce, server_nonce)
                 except socket.timeout:
                     if authenticated:
                         self._send_authenticated(connection, {
@@ -614,19 +820,41 @@ class _BridgeServer(threading.Thread):
                         }, client_nonce, server_nonce)
                 except Exception as error:
                     self._send(connection, {"ok": False, "error": str(error)})
-        self.server.close()
-        try:
-            self.descriptor.unlink(missing_ok=True)
-        except OSError:
-            pass
+        finally:
+            self._note_connection_finished(connection_ordinal)
+            with self._connections_lock:
+                self._connections.discard(connection)
+
+    def _note_connection_finished(self, ordinal: int) -> None:
+        """Record one accepted connection as terminated.
+
+        The watermark is the longest prefix 1..W that has terminated. A later
+        status cannot report an earlier connection that is still reading or
+        still waiting to run.
+        """
+        with self._connections_lock:
+            if ordinal <= self._finished_through or ordinal in self._finished_ordinals:
+                return
+            self._finished_ordinals.add(ordinal)
+            while self._finished_through + 1 in self._finished_ordinals:
+                self._finished_through += 1
+                self._finished_ordinals.discard(self._finished_through)
+
+    def _completion_watermark(self) -> str:
+        with self._connections_lock:
+            return f"{self._finished_through:016x}"
 
     @staticmethod
-    def _receive_frame(connection: socket.socket) -> dict[str, Any]:
+    def _receive_frame(
+        connection: socket.socket,
+        timeout_seconds: float = AUTH_FRAME_TIMEOUT_SECONDS,
+    ) -> dict[str, Any]:
         data = bytearray()
-        deadline = time.monotonic() + AUTH_FRAME_TIMEOUT_SECONDS
-        while b"\n" not in data:
+        deadline = time.monotonic() + timeout_seconds
+        newline = -1
+        while newline < 0:
             if len(data) > MAX_FRAME_BYTES:
-                raise ValueError("frame-limit-exceeded")
+                raise _frame_limit_exceeded()
             remaining = deadline - time.monotonic()
             if remaining <= 0:
                 raise socket.timeout("authentication frame timed out")
@@ -636,57 +864,173 @@ class _BridgeServer(threading.Thread):
             part = connection.recv(min(65536, MAX_FRAME_BYTES + 1 - len(data)))
             if not part:
                 raise ValueError("incomplete authentication frame")
+            scanned = len(data)
             data.extend(part)
-        line, remainder = bytes(data).split(b"\n", 1)
-        if len(line) > MAX_FRAME_BYTES or remainder:
+            # The delimiter is one byte, so it cannot straddle a chunk.
+            # Search only the bytes just received.
+            newline = data.find(b"\n", scanned)
+        if newline > MAX_FRAME_BYTES:
+            raise _frame_limit_exceeded()
+        if newline != len(data) - 1:
             raise ValueError("invalid framed request")
-        value = json.loads(line)
+        value = json.loads(bytes(data[:newline]))
         return value if isinstance(value, dict) else {}
 
     def _send_authenticated(self, connection: socket.socket, value: dict[str, Any], client_nonce: str, server_nonce: str) -> None:
-        encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
-        if encoded is None:
-            value = self._truncated_response(value)
-            encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
-        if encoded is None:
-            # This fixed-size fallback is checked through the same signed framing path.
-            value = {
-                "ok": False,
-                "error": "response-limit-exceeded",
-                "execution_state": value.get("execution_state", "unknown"),
-                "output": None,
-                "output_truncated": True,
-                "message": "The result exceeded the bridge frame limit; command completion state is preserved.",
-            }
-            encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
-        if encoded is None:
+        if self._deliver_authenticated(connection, value, client_nonce, server_nonce):
             return
-        try:
-            connection.sendall(encoded)
-        except OSError:
-            pass
+        # Keep the recovery payload. Include the observed path when it
+        # fits. An explicit null path means unknown, so the client does
+        # not pair this generation with the descriptor's older path.
+        bounded = self._bounded_status_response(value)
+        if bounded is not None:
+            observed_path = value.get("current_file")
+            if isinstance(observed_path, str) and observed_path:
+                with_path = dict(bounded)
+                with_path["current_file"] = observed_path
+                with_path.pop("message", None)
+                if self._deliver_authenticated(
+                    connection, with_path, client_nonce, server_nonce
+                ):
+                    return
+            if self._deliver_authenticated(
+                connection, bounded, client_nonce, server_nonce
+            ):
+                return
+        command = self._bounded_command_response(value)
+        if command is not None:
+            observed_path = value.get("current_file")
+            if isinstance(observed_path, str) and observed_path:
+                with_path = dict(command)
+                with_path["current_file"] = observed_path
+                with_path.pop("message", None)
+                if self._deliver_authenticated(
+                    connection, with_path, client_nonce, server_nonce
+                ):
+                    return
+            if self._deliver_authenticated(
+                connection, command, client_nonce, server_nonce
+            ):
+                return
+        value = self._truncated_response(value)
+        if self._deliver_authenticated(connection, value, client_nonce, server_nonce):
+            return
+        # This fixed-size fallback is checked through the same signed framing path.
+        self._deliver_authenticated(connection, {
+            "ok": False,
+            "error": "response-limit-exceeded",
+            "execution_state": value.get("execution_state", "unknown"),
+            "output": None,
+            "output_truncated": True,
+            "message": "The result exceeded the bridge frame limit; command completion state is preserved.",
+        }, client_nonce, server_nonce)
 
-    def _encode_authenticated_response(self, value: dict[str, Any], client_nonce: str, server_nonce: str) -> bytes | None:
+    def _deliver_authenticated(
+        self, connection: socket.socket, value: dict[str, Any], client_nonce: str, server_nonce: str
+    ) -> bool:
+        proof = self._authenticated_proof(value, client_nonce, server_nonce)
+        if proof is None:
+            return False
+        try:
+            self._write_authenticated_frame(connection, value, proof)
+        except OSError:
+            # A disconnected caller must not fall through to another payload.
+            pass
+        return True
+
+    def _authenticated_proof(
+        self, value: dict[str, Any], client_nonce: str, server_nonce: str
+    ) -> str | None:
         if not _within_response_limit(value):
             return None
-        prefix = b'{"response":"'
-        suffix = b'","proof":"' + (b"0" * 64) + b'"}\n'
-        frame_size = len(prefix) + len(suffix)
-        response_parts: list[bytes] = []
-        escaped_parts: list[bytes] = []
-        for fragment in json.JSONEncoder(ensure_ascii=False, separators=(",", ":")).iterencode(value):
-            raw = fragment.encode("utf-8")
-            escaped = json.dumps(fragment, ensure_ascii=False)[1:-1].encode("utf-8")
-            frame_size += len(escaped)
+        # The payload budget excludes the terminating LF. Fragments are
+        # measured and signed here, then discarded. The write pass encodes
+        # them again so a maximum-sized reply is not held in several copies.
+        frame_size = _AUTHENTICATED_FRAME_OVERHEAD
+        proof = hmac.new(self.token.encode(), digestmod=hashlib.sha256)
+        proof.update(
+            f"rea-cutter-response-v2:{self.session_id}:{client_nonce}:{server_nonce}:".encode()
+        )
+        for fragment in _response_fragments(value):
+            frame_size += len(_escape_json_fragment(fragment))
             if frame_size > MAX_FRAME_BYTES:
                 return None
-            response_parts.append(raw)
-            escaped_parts.append(escaped)
-        proof = hmac.new(self.token.encode(), digestmod=hashlib.sha256)
-        proof.update(f"rea-cutter-response-v2:{self.session_id}:{client_nonce}:{server_nonce}:".encode())
-        for part in response_parts:
-            proof.update(part)
-        return prefix + b"".join(escaped_parts) + b'","proof":"' + proof.hexdigest().encode() + b'"}\n'
+            proof.update(fragment.encode("utf-8"))
+        return proof.hexdigest()
+
+    def _encode_authenticated_response(
+        self, value: dict[str, Any], client_nonce: str, server_nonce: str
+    ) -> bytes | None:
+        """Return one exact frame. The socket path writes the same pieces."""
+        proof = self._authenticated_proof(value, client_nonce, server_nonce)
+        if proof is None:
+            return None
+        return b"".join(self._iter_authenticated_frame(value, proof))
+
+    def _iter_authenticated_frame(self, value: dict[str, Any], proof: str):
+        yield _AUTHENTICATED_PREFIX
+        for fragment in _response_fragments(value):
+            yield _escape_json_fragment(fragment)
+        yield _AUTHENTICATED_PROOF_MARK + proof.encode("ascii") + b'"}\n'
+
+    def _write_authenticated_frame(
+        self, connection: socket.socket, value: dict[str, Any], proof: str
+    ) -> None:
+        pending = bytearray()
+        for chunk in self._iter_authenticated_frame(value, proof):
+            pending.extend(chunk)
+            if len(pending) >= _AUTHENTICATED_WRITE_BUFFER_BYTES:
+                connection.sendall(pending)
+                pending.clear()
+        if pending:
+            connection.sendall(pending)
+
+    @staticmethod
+    def _bounded_status_response(value: dict[str, Any]) -> dict[str, Any] | None:
+        if "completed_commands" not in value and "completed_connection" not in value:
+            return None
+        bounded: dict[str, Any] = {
+            "ok": value.get("ok") is True,
+            "session_id": value.get("session_id"),
+            "document_generation": value.get("document_generation"),
+            "identity_status": value.get("identity_status", "partial"),
+            "completed_commands": value.get("completed_commands"),
+            "current_file": None,
+            "message": (
+                "current_file is unknown because the status frame exceeded "
+                "REA_BRIDGE_MAX_REQUEST_BYTES. Do not reuse an older descriptor "
+                "path with this document_generation."
+            ),
+        }
+        if "completed_connection" in value:
+            bounded["completed_connection"] = value["completed_connection"]
+        return bounded
+
+    @staticmethod
+    def _bounded_command_response(value: dict[str, Any]) -> dict[str, Any] | None:
+        if "execution_state" not in value and "document_generation" not in value:
+            return None
+        bounded: dict[str, Any] = {
+            "ok": value.get("ok") is True,
+            "output": None,
+            "execution_state": value.get("execution_state", "unknown"),
+            "output_truncated": True,
+            "current_file": None,
+            "message": (
+                "current_file is unknown because the result frame exceeded "
+                "REA_BRIDGE_MAX_REQUEST_BYTES. The command completion state and "
+                "document_generation are preserved. Do not reuse an older "
+                "descriptor path with this document_generation."
+            ),
+        }
+        generation = value.get("document_generation")
+        if isinstance(generation, int) and not isinstance(generation, bool):
+            bounded["document_generation"] = generation
+        if value.get("identity_status") == "partial":
+            bounded["identity_status"] = "partial"
+        if value.get("ok") is not True and isinstance(value.get("error"), str):
+            bounded["error"] = value["error"]
+        return bounded
 
     @staticmethod
     def _truncated_response(value: dict[str, Any]) -> dict[str, Any]:
@@ -744,6 +1088,17 @@ class _BridgeServer(threading.Thread):
             self.server.close()
         except OSError:
             pass
+        with self._connections_lock:
+            connections = list(self._connections)
+        for connection in connections:
+            try:
+                connection.shutdown(socket.SHUT_RDWR)
+            except OSError:
+                pass
+            try:
+                connection.close()
+            except OSError:
+                pass
 
 
 class ReaCutterPlugin(cutter.CutterPlugin):
@@ -777,6 +1132,128 @@ def create_cutter_plugin() -> ReaCutterPlugin:
     return ReaCutterPlugin()
 
 
+def _json_object_key(key: Any) -> str:
+    """Match CPython's JSON object-key text before quoting."""
+    if isinstance(key, str):
+        return key
+    if key is True:
+        return "true"
+    if key is False:
+        return "false"
+    if key is None:
+        return "null"
+    if isinstance(key, int):
+        return str(key)
+    if isinstance(key, float):
+        return json.dumps(key, separators=(",", ":"))
+    raise TypeError("keys must be str, int, float, bool or None")
+
+
+def _response_fragments(value: Any):
+    """Yield compact JSON pieces whose concatenation is the encoded value.
+
+    String contents are sliced before encoding so a large output never becomes
+    one frame-sized fragment. Each piece is at most ``_JSON_CHUNK_CHARS``
+    characters, or the six-byte ``\\u00XX`` expansion of one such slice.
+    """
+    if isinstance(value, str):
+        yield '"'
+        for start in range(0, len(value), _JSON_CHUNK_CHARS):
+            yield json.dumps(
+                value[start:start + _JSON_CHUNK_CHARS], ensure_ascii=False
+            )[1:-1]
+        yield '"'
+        return
+    if isinstance(value, dict):
+        yield "{"
+        first = True
+        for key, item in value.items():
+            if not first:
+                yield ","
+            first = False
+            yield from _response_fragments(_json_object_key(key))
+            yield ":"
+            yield from _response_fragments(item)
+        yield "}"
+        return
+    if isinstance(value, (list, tuple)):
+        yield from _list_fragments(value)
+        return
+    token = _scalar_token(value)
+    if token is None:
+        raise TypeError("response value is not JSON")
+    yield token
+
+
+def _scalar_token(value: Any) -> str | None:
+    """Compact JSON for one scalar. None means the value needs structure."""
+    if type(value) is int:
+        return str(value)
+    if value is None:
+        return "null"
+    if value is True:
+        return "true"
+    if value is False:
+        return "false"
+    if isinstance(value, float):
+        return json.dumps(value, separators=(",", ":"))
+    return None
+
+
+def _list_fragments(value: list[Any] | tuple[Any, ...]):
+    """Encode a list, batching scalars into bounded pieces.
+
+    One ``json.dumps`` per number makes a dense array miss the client
+    deadline. A batch is at most ``_JSON_CHUNK_CHARS`` of compact JSON.
+    """
+    yield "["
+    first = True
+    batch: list[str] = []
+    batch_len = 0
+
+    def flush_batch() -> str | None:
+        nonlocal first, batch, batch_len
+        if not batch:
+            return None
+        piece = ",".join(batch)
+        batch = []
+        batch_len = 0
+        if first:
+            first = False
+            return piece
+        return "," + piece
+
+    for item in value:
+        token = _scalar_token(item)
+        if token is None:
+            emitted = flush_batch()
+            if emitted is not None:
+                yield emitted
+            if first:
+                first = False
+            else:
+                yield ","
+            yield from _response_fragments(item)
+            continue
+        extra = len(token) if not batch else len(token) + 1
+        if batch and batch_len + extra > _JSON_CHUNK_CHARS:
+            emitted = flush_batch()
+            if emitted is not None:
+                yield emitted
+            extra = len(token)
+        batch.append(token)
+        batch_len += extra
+    emitted = flush_batch()
+    if emitted is not None:
+        yield emitted
+    yield "]"
+
+
+def _escape_json_fragment(fragment: str) -> bytes:
+    """Escape one raw JSON piece for the outer response string."""
+    return json.dumps(fragment, ensure_ascii=False)[1:-1].encode("utf-8")
+
+
 def _within_response_limit(value: Any) -> bool:
     pending = [value]
     seen: set[int] = set()
@@ -800,13 +1277,22 @@ def _within_response_limit(value: Any) -> bool:
                     # Unencodable output also needs the signed omission fallback.
                     return False
             size += current.count('"') + current.count("\\")
-            size += sum(5 for character in current if ord(character) < 0x20)
+            # A short escape is two JSON bytes and was already counted as one.
+            # Other controls expand to a six-byte \\u00XX sequence.
+            size += sum(
+                1 if ord(character) in (0x08, 0x09, 0x0A, 0x0C, 0x0D) else 5
+                for character in current
+                if ord(character) < 0x20
+            )
         elif isinstance(current, dict):
             identity = id(current)
             if identity in seen:
                 return False
             seen.add(identity)
-            size += 2 + 4 * len(current)
+            count = len(current)
+            # Braces, one colon per entry, and commas between entries.
+            # Keys and values are counted when they are visited.
+            size += 2 + count + max(0, count - 1)
             if size > MAX_FRAME_BYTES:
                 return False
             pending.extend(current.keys())
@@ -816,12 +1302,29 @@ def _within_response_limit(value: Any) -> bool:
             if identity in seen:
                 return False
             seen.add(identity)
-            size += 2 + 2 * len(current)
+            count = len(current)
+            # Brackets and commas between items. Each item is counted below.
+            size += 2 + max(0, count - 1)
             if size > MAX_FRAME_BYTES:
                 return False
             pending.extend(current)
+        elif current is None:
+            size += 4
+        elif current is True:
+            size += 4
+        elif current is False:
+            size += 5
+        elif isinstance(current, int):
+            size += len(str(current))
+        elif isinstance(current, float):
+            size += len(json.dumps(current, separators=(",", ":")))
         else:
-            size += 16
+            try:
+                size += len(
+                    json.dumps(current, ensure_ascii=False, separators=(",", ":"))
+                )
+            except (TypeError, ValueError):
+                return False
         if size > MAX_FRAME_BYTES:
             return False
     return True

bridge/hopper_bridge.py

@@ -8,15 +8,100 @@ API access on this thread: moving dispatch to a worker can deadlock Hopper.
 import json
 import hmac
 import os
+import re
 import socket
 from typing import Any, Optional, Protocol, Sequence
 
 BAD_ADDRESSES = (-1, 0xFFFFFFFFFFFFFFFF, None)
 # The native label setter truncates NSString names at this UTF-16 extent.
 HOPPER_SYMBOL_NAME_UTF16_UNITS = 1024
-# Maximum request JSON payload size in bytes, excluding its trailing LF.
-# Matches the Cutter bridge's MAX_FRAME_BYTES payload ceiling.
-MAX_REQUEST_BYTES = 16 * 1024 * 1024
+# Request JSON payload budget in bytes, excluding the trailing LF.
+# REA_BRIDGE_MAX_REQUEST_BYTES selects it. Unset stays 16777216.
+# 7464 is the authenticated Cutter frame of the largest bounded symbol name.
+# Cutter's 64-receipt recovery status is smaller with a null path. A path
+# that does not fit is null, not an older descriptor path.
+# 536870887 is Node's 536870888-character string ceiling minus the framing LF.
+# The ceiling is the encoded payload; JSON expansion is measured before it.
+_BRIDGE_MAX_REQUEST_BYTES_SETTING = "REA_BRIDGE_MAX_REQUEST_BYTES"
+DEFAULT_BRIDGE_MAX_REQUEST_BYTES = 16 * 1024 * 1024
+MINIMUM_BRIDGE_MAX_REQUEST_BYTES = 7464
+MAXIMUM_BRIDGE_MAX_REQUEST_BYTES = 536870887
+
+
+def _bridge_max_request_bytes_constraint():
+    return (
+        "REA_BRIDGE_MAX_REQUEST_BYTES must be a positive decimal integer "
+        "from %d to %d bytes; the default is %d bytes (16 MiB). "
+        "A value below %d bytes cannot carry the largest admitted "
+        "symbol-name command, and a larger value is refused so the encoded "
+        "payload plus its framing LF stays one string under Node's "
+        "single-string ceiling."
+        % (
+            MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
+            MAXIMUM_BRIDGE_MAX_REQUEST_BYTES,
+            DEFAULT_BRIDGE_MAX_REQUEST_BYTES,
+            MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
+        )
+    )
+
+
+def _parse_bridge_max_request_bytes(raw):
+    """Refuse a non-decimal or out-of-range ceiling. Never fall back."""
+    constraint = _bridge_max_request_bytes_constraint()
+    if isinstance(raw, bool):
+        raise RuntimeError(constraint)
+    if isinstance(raw, int):
+        value = raw
+    elif isinstance(raw, str):
+        # ASCII digits only. str.isdigit() accepts numerals such as U+0668
+        # that the TypeScript setting rejects.
+        if re.fullmatch(r"[1-9][0-9]*", raw) is None:
+            raise RuntimeError(constraint)
+        value = int(raw)
+    else:
+        raise RuntimeError(constraint)
+    if (
+        value < MINIMUM_BRIDGE_MAX_REQUEST_BYTES
+        or value > MAXIMUM_BRIDGE_MAX_REQUEST_BYTES
+    ):
+        raise RuntimeError(constraint)
+    return value
+
+
+def _resolve_bridge_max_request_bytes():
+    """Prefer the bootstrap injection, then the process environment, then 16 MiB."""
+    injected = globals().get(_BRIDGE_MAX_REQUEST_BYTES_SETTING)
+    if injected is not None:
+        return _parse_bridge_max_request_bytes(injected), "setting"
+    raw = os.environ.get(_BRIDGE_MAX_REQUEST_BYTES_SETTING)
+    if raw is None:
+        return DEFAULT_BRIDGE_MAX_REQUEST_BYTES, "default"
+    return _parse_bridge_max_request_bytes(raw), "setting"
+
+
+MAX_REQUEST_BYTES, BRIDGE_MAX_REQUEST_BYTES_SOURCE = (
+    _resolve_bridge_max_request_bytes()
+)
+
+
+def _bridge_max_request_bytes_recovery():
+    return (
+        "The effective REA_BRIDGE_MAX_REQUEST_BYTES value is %d bytes "
+        "(source %s; default %d, floor %d, maximum %d). "
+        "Set REA_BRIDGE_MAX_REQUEST_BYTES in the environment that starts REA "
+        "to a decimal integer from %d to %d, then restart REA. "
+        "Restart Hopper or Cutter from that same environment so the bridge "
+        "uses this ceiling."
+        % (
+            MAX_REQUEST_BYTES,
+            BRIDGE_MAX_REQUEST_BYTES_SOURCE,
+            DEFAULT_BRIDGE_MAX_REQUEST_BYTES,
+            MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
+            MAXIMUM_BRIDGE_MAX_REQUEST_BYTES,
+            MINIMUM_BRIDGE_MAX_REQUEST_BYTES,
+            MAXIMUM_BRIDGE_MAX_REQUEST_BYTES,
+        )
+    )
 _rea_session_document = None
 _search_inventory_cache = {}
 _pseudocode_cache = {}
@@ -743,6 +828,7 @@ def _dispatch(method, params):
             "version": "1.0.0",
             "run_id": REA_RUN_ID,
             "process_id": os.getpid(),
+            "max_request_bytes": MAX_REQUEST_BYTES,
         }
     if method in ("shutdown", "shutdown_document"):
         document = _session_document()
@@ -1071,8 +1157,8 @@ def _serve_connection(connection):
                 break
             if not line.endswith(b"\n"):
                 message = (
-                    "Bridge request exceeds the %d-byte payload limit"
-                    % MAX_REQUEST_BYTES
+                    "Bridge request exceeds the %d-byte payload limit. %s"
+                    % (MAX_REQUEST_BYTES, _bridge_max_request_bytes_recovery())
                     if len(line) > MAX_REQUEST_BYTES
                     else "Incomplete bridge request frame"
                 )