refactor(domain): give strict UTF-8 decoding one owner with an explicit BOM policy (#10728)

* refactor(domain): give strict UTF-8 decoding one owner with an explicit BOM policy

* fix(domain): keep the leading-BOM behaviour every migrated UTF-8 decoder had

* refactor(domain): read the domain layer's strict UTF-8 through the BOM-policy owner (#10803)

Twelve domain decoders built their own fatal TextDecoder with ignoreBOM: true. They now use strictUtf8Decoder("preserve"), which keeps that behaviour: each reads bytes where a leading U+FEFF is content (binary-format names and fields, JSON and XML read byte-for-byte, recorded bodies, registry seeds whose kept bytes are re-emitted). The counted baseline drops from 74 to 62.

* refactor(artifacts): read artifact readers' strict UTF-8 through the BOM-policy owner (#10871)

Fourteen artifact reader modules (17 sites) built their own fatal TextDecoder. They now take it from strictUtf8Decoder with the policy each had: every one kept a leading BOM (ignoreBOM: true), so they use "preserve"; JavaScriptArtifactFiles keeps it for JavaScript and HTML entries and strips it otherwise, as before. The strict-utf8-decoder baseline drops those 17 sites.

* refactor(domain): drop TextDecoder imports left by the strict UTF-8 migration

* refactor(application): read MCP registration metadata through the strict UTF-8 owner

* refactor(formats): read binary-format readers' strict UTF-8 through the BOM-policy owner (#10903)

* refactor(formats): read binary-format readers' strict UTF-8 through the BOM-policy owner

* refactor(formats): drop the model readers' unused TextDecoder imports

* refactor(browser): read browser capture and fetch strict UTF-8 through the BOM-policy owner (#10935)

* refactor(browser): read browser capture and fetch strict UTF-8 through the BOM-policy owner

* refactor(native): read LLDB and Mach-O name strict UTF-8 through the BOM-policy owner (#10936)

* refactor(native): read LLDB and Mach-O name strict UTF-8 through the BOM-policy owner

* refactor(process): read MCP and NDJSON transport frames' strict UTF-8 through the BOM-policy owner (#10937)

* refactor(process): read MCP and NDJSON transport frames' strict UTF-8 through the BOM-policy owner

* refactor(application): read JSON and file inputs' strict UTF-8 through the BOM-policy owner (#10938)

* refactor(application): read JSON and file inputs' strict UTF-8 through the BOM-policy owner

* refactor(providers): read IDA, Electron, Ghidra and reference strict UTF-8 through the BOM-policy owner (#10939)

* refactor(providers): read IDA, Electron, Ghidra and reference strict UTF-8 through the BOM-policy owner

* refactor(evm): read the hex bytecode carrier's strict UTF-8 through the owner and refuse new local decoders (#10940)

* refactor(evm): read the hex bytecode carrier's strict UTF-8 through the BOM-policy owner

* chore(architecture): match local strict UTF-8 decoders in any option order
This commit is contained in:
morluto authored and GitHub committed 2026-11-15 19:54:39 +08:00
1 parent abd36dd250
commit cabd56ea71
103 files changed
+322 -251

No files matched your search

+3 -6
View File
@@ -18,7 +18,6 @@ import {
writeFile,
} from "node:fs/promises";
import { basename, dirname, isAbsolute, join, relative, sep } from "node:path";
import { TextDecoder } from "node:util";
import { normalizeArtifactPath } from "../../artifacts/ArtifactPaths.js";
import {
AnalysisAccessDeniedError,
@@ -36,6 +35,7 @@ import {
sameRegularFileState,
} from "../../filesystem/RegularFile.js";
import { RECOVERY_LIMITS, wakaruFixedLimitRecovery } from "./WakaruRelease.js";
import { isStrictUtf8 } from "../../domain/strictUtf8.js";
const OPERATION = "recover_javascript_sources";
@@ -222,10 +222,8 @@ const snapshotRecoveryInput = async (
cause,
);
}
try {
new TextDecoder("utf-8", { fatal: true }).decode(bytes);
} catch (cause: unknown) {
throw new AnalysisInputError(OPERATION, { cause }, [
if (!isStrictUtf8(bytes))
throw new AnalysisInputError(OPERATION, {}, [
{
path: ["path"],
reason: "invalid_format",
@@ -233,7 +231,6 @@ const snapshotRecoveryInput = async (
message: `JavaScript input is not valid UTF-8: ${path}`,
},
]);
}
try {
await mkdir(dirname(snapshotPath), { recursive: true, mode: 0o700 });
await writeFile(snapshotPath, bytes, { flag: "wx", mode: 0o600 });
@@ -16,6 +16,7 @@ import {
sourceMapCodecReplySchema,
} from "./SourceMapCodecProtocol.js";
import { z } from "zod";
import { decodeStrictUtf8 } from "../../domain/strictUtf8.js";
/** Dedicated process entrypoint invoked only for an explicitly selected map. */
const execute = async (): Promise<string> => {
@@ -28,7 +29,8 @@ const execute = async (): Promise<string> => {
);
const limits = selectedSourceMapLimits(selected);
const data = await readStableArtifact(path, limits.outputBytes);
const text = new TextDecoder("utf-8", { fatal: true }).decode(data.bytes);
// Source maps on disk may carry a BOM signature (RFC 8259 8.1).
const text = decodeStrictUtf8(data.bytes, "strip-signature");
const value: unknown = JSON.parse(text);
const input = sourceMapCodecInputSchema.parse(value);
if (Buffer.byteLength(input.text) > limits.mapBytes)