fix(environment): measure the credential scan limit in UTF-8 bytes

A body counts against the 4 MiB limit by its UTF-8 length, not its UTF-16 code units, so multibyte text over the
limit is reported as unscanned instead of parsed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Edgar ArakelyanandClaude Opus 5.5 committed 2026-10-24 04:01:30 -07:00
1 parent 312e74e60d
commit 98fe6dea8b
3 files changed
+28 -6

No files matched your search

@@ -53,11 +53,11 @@ before building.
parameters and no text, or the null text mitmproxy's SaveHar records, is
scanned by those parameters, and a scenario transaction that cannot be
attributed to one response still has its request side scanned. Text bodies
over the 4 MiB scan limit, including a response body decoded only as a prefix,
are not scanned; they are reported as a `credential-scan-incomplete` warning
with a count and the first 20 record and side pointers (`record 3 response`),
never contents, and never refuse a build. Binary bodies hold no JSON or form
text and are neither scanned nor reported.
over the 4 MiB scan limit, measured in UTF-8 bytes, including a response body
decoded only as a prefix, are not scanned; they are reported as a
`credential-scan-incomplete` warning with a count and the first 20 record and
side pointers (`record 3 response`), never contents, and never refuse a build.
Binary bodies hold no JSON or form text and are neither scanned nor reported.
- The inspected HAR's Evidence and the projected manifest are admitted against
the live heap headroom for parsing and sealing before either is serialized, as
a capture Evidence file is before it is read; one that does not fit is refused
@@ -287,6 +287,22 @@ describe("scanCredentialNames over structured and unattributed records", () => {
});
describe("scanCredentialNames coverage", () => {
it("measures the scan limit in UTF-8 bytes, not UTF-16 code units", () => {
// 1.5 million three-byte characters: under 4 Mi code units, over 4 MiB of UTF-8.
const wide = JSON.stringify({
token: "t-wide",
pad: "\u754c".repeat(1_500_000),
});
const multibyte = record(9, undefined, undefined);
(
multibyte.reported as { request: { postData?: unknown } }
).request.postData = { mimeType: "application/json", text: wide };
expect(scanCredentialNames([multibyte])).toMatchObject({
count: 0,
skipped: { count: 1, pointers: ["record 9 request"] },
});
});
it("reports text bodies too large to scan by record and side, never by content", () => {
const large = JSON.stringify({
token: "t-large",
@@ -96,7 +96,13 @@ type ScannedBody =
/** A recorded body as JSON or form fields, skipped when it is cut or over the limit, or undefined when it is neither. */
const parsedBody = (text: unknown, cut = false): ScannedBody | undefined => {
if (typeof text !== "string") return undefined;
if (cut || text.length > SCAN_BODY_LIMIT_BYTES) return { kind: "skipped" };
// UTF-8 is never shorter than UTF-16 code units, so the cheap length test only short-circuits.
if (
cut ||
text.length > SCAN_BODY_LIMIT_BYTES ||
Buffer.byteLength(text, "utf8") > SCAN_BODY_LIMIT_BYTES
)
return { kind: "skipped" };
const start = text.trimStart()[0];
if (start === "{" || start === "[") {
try {