fix(release-exposure): count scan limitations past the cap instead of dropping them silently (#11267)

inspect_release_exposure kept the first eight scan limitations and dropped the rest with no count, at the collector and at two merge points that prepend a reason. Add boundedList/BoundedListCollector (src/domain/boundedList.ts), which always carry the omitted count, and report it as a 'limitations' coverage.omitted entry. A counted constant-maximum-slice guard keeps new .slice(0, MAX) copies from appearing outside the owner.
This commit is contained in:
morluto authored and GitHub committed 2026-11-15 23:54:42 +08:00
1 parent 500c1f46b1
commit b41fe5a238
7 files changed
+190 -21

No files matched your search

+1 -1
View File
@@ -48,7 +48,7 @@ The audit composes existing readers. It does not reread Electron fuse bytes itse
`native.hardening`, `macho.hardened_runtime`, and `android.manifest` stay `not_assessed`. Their next probes are `inspect_binary_hardening`, `inspect_signature`, and `inspect_android_package`.
Source map text is counted and then dropped. A selected file is read through the stable artifact reader. Text and source-map members stay inside the allowances listed under `coverage.allowances`, and each kind has its own allowance, so binary members cannot use up the read that a later source map needs. Native images on disk are registered from inventory classification and are not limited to those allowances. Other files and package members are identified from a bounded header. The bytes a package member's text or map read returns decide before its name does, so a PE called `payload.js.map` is read as a PE. A header with no NUL byte, or one that starts with a UTF-16 byte order mark, is scanned as text, so names such as `.env.production`, `server.key`, and `credentials` do not depend on an extension list. UTF-16 without a byte order mark has NUL bytes and is not recognized this way. A member already known to be text, such as a script, is also scanned when it is UTF-8 holding a stray NUL. One whose header is at least an eighth NUL bytes is taken to be wide text. It is not decoded and is counted as skipped, as is UTF-32. A binary plist is decoded, so its strings, dictionary keys and data are scanned in whichever encoding each object uses. One that cannot be decoded still has its bytes scanned and is counted as skipped. A file whose header could not be read leaves `text.credential_markers` undecided. A packaged ELF, Mach-O, or PE image is read in full and written to a temporary copy, removed afterward, so it can be probed for an embedded package. All packaged images share one `max_file_bytes` allowance. An image that does not fit, or whose bytes cannot be read, is not probed and leaves member coverage `unknown`. A packaged PE is also read for its debug records. An SVG file is read for its scripts as an HTML file is. A script that an HTML file loads, or a Node SEA main script, is examined as a script whatever it is called, so a name such as `boot.map` does not make it a map, and an HTML file loaded this way is still examined as HTML. Its bytes still decide over that role: a `.map` or loaded script whose text parses as a source map, a JSON object with `version` and `mappings`, `sources`, or `sections`, is read as a source map, so `<script src="main.map">` pointing at a map with `sourcesContent` still reports `javascript.source_map_sources` exposed. One read before that HTML file, one that is not found, one at a location the audit cannot place, or one past the 1 MiB of resolved script paths the audit keeps (`MAX_RELEASE_EXPOSURE_SCRIPT_TARGET_BYTES`) leaves `javascript.source_map_sources` undecided. A `.map` read before the HTML file that loads it is not reported as a linker map. Skipped members are listed under `coverage.omitted`. When the selection is a file, the parent directory is listed up to a fixed entry allowance for sibling `.pdb`, `.dSYM`, `.debug`, and linker `.map` debug companions. A sibling source map that the selected file declares, such as `app.js.map`, or a sibling script that a selected HTML file loads, such as `app.js`, is not read as part of the artifact, so it leaves `javascript.source_map_sources` undecided.
Source map text is counted and then dropped. A selected file is read through the stable artifact reader. Text and source-map members stay inside the allowances listed under `coverage.allowances`, and each kind has its own allowance, so binary members cannot use up the read that a later source map needs. Native images on disk are registered from inventory classification and are not limited to those allowances. Other files and package members are identified from a bounded header. The bytes a package member's text or map read returns decide before its name does, so a PE called `payload.js.map` is read as a PE. A header with no NUL byte, or one that starts with a UTF-16 byte order mark, is scanned as text, so names such as `.env.production`, `server.key`, and `credentials` do not depend on an extension list. UTF-16 without a byte order mark has NUL bytes and is not recognized this way. A member already known to be text, such as a script, is also scanned when it is UTF-8 holding a stray NUL. One whose header is at least an eighth NUL bytes is taken to be wide text. It is not decoded and is counted as skipped, as is UTF-32. A binary plist is decoded, so its strings, dictionary keys and data are scanned in whichever encoding each object uses. One that cannot be decoded still has its bytes scanned and is counted as skipped. A file whose header could not be read leaves `text.credential_markers` undecided. A packaged ELF, Mach-O, or PE image is read in full and written to a temporary copy, removed afterward, so it can be probed for an embedded package. All packaged images share one `max_file_bytes` allowance. An image that does not fit, or whose bytes cannot be read, is not probed and leaves member coverage `unknown`. A packaged PE is also read for its debug records. An SVG file is read for its scripts as an HTML file is. A script that an HTML file loads, or a Node SEA main script, is examined as a script whatever it is called, so a name such as `boot.map` does not make it a map, and an HTML file loaded this way is still examined as HTML. Its bytes still decide over that role: a `.map` or loaded script whose text parses as a source map, a JSON object with `version` and `mappings`, `sources`, or `sections`, is read as a source map, so `<script src="main.map">` pointing at a map with `sourcesContent` still reports `javascript.source_map_sources` exposed. One read before that HTML file, one that is not found, one at a location the audit cannot place, or one past the 1 MiB of resolved script paths the audit keeps (`MAX_RELEASE_EXPOSURE_SCRIPT_TARGET_BYTES`) leaves `javascript.source_map_sources` undecided. A `.map` read before the HTML file that loads it is not reported as a linker map. Skipped members are listed under `coverage.omitted`. The report keeps the first eight scan limitations; a `limitations` entry under `coverage.omitted` counts the reports past them, including repeats. When the selection is a file, the parent directory is listed up to a fixed entry allowance for sibling `.pdb`, `.dSYM`, `.debug`, and linker `.map` debug companions. A sibling source map that the selected file declares, such as `app.js.map`, or a sibling script that a selected HTML file loads, such as `app.js`, is not read as part of the artifact, so it leaves `javascript.source_map_sources` undecided.
Inventory stops at 65,536 entries or 2 GiB of hashed bytes, and member coverage then stays `unknown`. A packed ASAR member is read from the captured archive, whose digest must match inventory. An unpacked member is read from its `.unpacked` companion, so its full bytes must also match the digest inventory recorded for that member, and a companion replaced after inventory fails the audit as a changed artifact. That digest check stays inside the read allowance: a companion larger than the allowance is read only up to the allowance, is not hashed to its end, and is treated as over its allowance without a digest verification. A declared integrity value that disagrees with the bytes does not stop the audit, because signing an unpacked native module after packaging is routine. The contradiction is recorded in the limitations and the audit continues.
@@ -519,5 +519,47 @@
"src/dotnet/ManagedDecodeBudget.ts": 1,
"src/javascript/analysis/JavaScriptAnalysisWorker.ts": 4,
"src/python/PythonPycParser.ts": 2
},
"constant-maximum-slice": {
"src/adb/AdbCommand.ts": 1,
"src/apktool/ApktoolProvider.ts": 1,
"src/application/EnhancedTools.ts": 1,
"src/application/GuidanceAccess.ts": 1,
"src/application/desktop/DesktopBuildActionLog.ts": 1,
"src/application/environment/environmentBuildReport.ts": 1,
"src/application/environment/environmentCompare.ts": 1,
"src/application/environment/environmentCompareDiff.ts": 1,
"src/application/environment/environmentReplay.ts": 1,
"src/application/javascript/JavaScriptArtifactAnalysis.ts": 1,
"src/application/releaseExposure/ReleaseExposureService.ts": 1,
"src/artifacts/ArtifactPaths.ts": 1,
"src/artifacts/AsarListing.ts": 1,
"src/desktop/DesktopSessionObservations.ts": 1,
"src/domain/analysisView/environmentStateWriteApply.ts": 2,
"src/domain/apkSignature/apkV1Signature.ts": 1,
"src/domain/apple/plistDuplicateKeys.ts": 1,
"src/domain/desktop/appExportUnowned.ts": 1,
"src/domain/desktop/appTaskDraft.ts": 5,
"src/domain/desktop/appTaskDraftAdded.ts": 3,
"src/domain/desktop/appTaskDraftAlternatives.ts": 1,
"src/domain/desktop/appTaskDraftCounts.ts": 2,
"src/domain/desktop/appTaskDraftFileNotes.ts": 4,
"src/domain/desktop/appTaskDraftGuards.ts": 2,
"src/domain/desktop/appTaskDraftItems.ts": 1,
"src/domain/desktop/appTaskDraftJson.ts": 4,
"src/domain/desktop/appTaskDraftLineGuards.ts": 1,
"src/domain/desktop/appTaskDraftNamed.ts": 2,
"src/domain/desktop/appTaskDraftNested.ts": 2,
"src/domain/desktop/appTaskDraftNotes.ts": 1,
"src/domain/desktop/appTaskDraftText.ts": 1,
"src/domain/desktop/appTaskDraftValues.ts": 1,
"src/domain/desktop/desktopCapture.ts": 1,
"src/domain/desktop/memberPairing.ts": 1,
"src/domain/desktop/selectorCandidates.ts": 1,
"src/domain/desktop/stateChanges.ts": 5,
"src/domain/firmware/dfuse.ts": 1,
"src/domain/firmware/uf2.ts": 1,
"src/domain/native/nativeProviderOutput.ts": 1,
"src/domain/releaseExposure.ts": 2
}
}
+11
View File
@@ -353,6 +353,17 @@ checkCounted(
["src/domain/budget.ts"],
);
// A slice to a named maximum drops items; boundedList in src/domain/boundedList
// keeps the count of what it dropped beside the retained items, so a consumer
// can say that more exist and how many.
checkCounted(
"slice to a named maximum (use boundedList or BoundedListCollector from src/domain/boundedList so the omitted count travels with the items)",
"constant-maximum-slice",
files,
/\.slice\(\s*0\s*,\s*[A-Z][A-Z0-9_]*\s*\)/gu,
["src/domain/boundedList.ts"],
);
// A deadline is a monotonic budget owned by `Deadline` in src/domain/deadline;
// a wall-clock sum moves with clock changes and has no finalize phase. The
// remaining copy converts a monotonic budget for a browser page, which can
@@ -661,6 +661,10 @@ describe("ReleaseExposureReader limits", () => {
try {
const scanned = await inspect(root);
expect(scanned.limitations).toHaveLength(8);
// The reports past the cap are counted, not dropped silently.
expect(scanned.omitted).toContainEqual(
expect.objectContaining({ subject: "limitations", count: 4 }),
);
} finally {
stableRead.impl = real;
}
+64 -20
View File
@@ -13,6 +13,11 @@ import type {
ReleaseExposureTextVisitor,
} from "../application/releaseExposure/ReleaseExposurePort.js";
import type { ExecutionOptions } from "../application/AnalysisProvider.js";
import {
BoundedListCollector,
boundedList,
type BoundedList,
} from "../domain/boundedList.js";
import {
ArtifactReaderFailure,
type ArtifactEntry,
@@ -157,6 +162,47 @@ const OPEN_CONTAINER_LIMIT = 8;
const EMBEDDED_PROBE_LIMIT = 8;
const PDB_READ_LIMIT = 4;
const LIMITATION_CAP = 8;
const LIMITATIONS_SUBJECT = "limitations";
/**
* Bound limitations to LIMITATION_CAP; the count of reports past it is a
* coverage omission, so the report says how many more there were.
*/
const boundedLimitations = (
scan: Pick<ReleaseExposureScan, "omitted">,
limitations: BoundedList<string>,
): Pick<ReleaseExposureScan, "limitations" | "omitted"> => ({
limitations: limitations.items,
omitted: [
...scan.omitted.filter(({ subject }) => subject !== LIMITATIONS_SUBJECT),
...(limitations.omitted > 0
? [
{
subject: LIMITATIONS_SUBJECT,
count: limitations.omitted,
reason: `Only the first ${String(LIMITATION_CAP)} scan limitations are retained; the count includes repeated reports.`,
},
]
: []),
],
});
/** Put `leading` limitations first, counting any this pushes past the cap. */
const prependLimitations = (
scan: ReleaseExposureScan,
leading: readonly string[],
): ReleaseExposureScan => ({
...scan,
...boundedLimitations(
scan,
boundedList(
[...leading, ...scan.limitations],
LIMITATION_CAP,
scan.omitted.find(({ subject }) => subject === LIMITATIONS_SUBJECT)
?.count ?? 0,
),
),
});
const SCAN_INTERRUPTED =
"The member scan stopped on unfinished cleanup. Members it did not reach are unknown.";
const SCAN_CANCELLED =
@@ -355,12 +401,8 @@ const scanFile = async (
inventory,
});
return {
...scanned,
...prependLimitations(scanned, limitations),
coverage: inventory === undefined ? "unknown" : scanned.coverage,
limitations: [...limitations, ...scanned.limitations].slice(
0,
LIMITATION_CAP,
),
};
};
@@ -475,9 +517,8 @@ const reportInterruptedScan = (
): void => {
if (scan === undefined) return;
input.visitInterruptedScan?.({
...scan,
...prependLimitations(scan, [reason]),
coverage: "unknown",
limitations: [reason, ...scan.limitations].slice(0, LIMITATION_CAP),
});
};
@@ -673,7 +714,7 @@ const assembleScan = (
const sha256 = state.nativeDigests.get(path);
return { path, format, ...(sha256 === undefined ? {} : { sha256 }) };
});
return {
const scan = {
target: {
selected_path: input.path,
kind,
@@ -786,10 +827,18 @@ const assembleScan = (
]
: []),
],
limitations: [
...(input.inventory?.limitations ?? []),
...state.limitations,
].slice(0, LIMITATION_CAP),
} satisfies Omit<ReleaseExposureScan, "limitations">;
const collected = state.limitations.toList();
return {
...scan,
...boundedLimitations(
scan,
boundedList(
[...(input.inventory?.limitations ?? []), ...collected.items],
LIMITATION_CAP,
collected.omitted,
),
),
};
};
@@ -833,7 +882,7 @@ interface ScanState {
unprobedImages: number;
native: Map<string, NativeFormat>;
nativeDigests: Map<string, string>;
limitations: string[];
limitations: BoundedListCollector<string>;
mapsRead: number;
/**
* Local targets of source-map declarations, whatever their file name, by
@@ -908,7 +957,7 @@ const createState = (
unprobedImages: 0,
native: new Map(),
nativeDigests: new Map(),
limitations: [],
limitations: new BoundedListCollector(LIMITATION_CAP),
mapsRead: 0,
mapTargets: new Map(),
claimedMapTargets: new Set(),
@@ -3565,12 +3614,7 @@ const pushSample = (samples: string[], value: string): void => {
};
const pushLimitation = (state: ScanState, message: string): void => {
if (
state.limitations.length >= LIMITATION_CAP ||
state.limitations.includes(message)
)
return;
state.limitations.push(message);
if (!state.limitations.retains(message)) state.limitations.push(message);
};
const fileSizeRecovery = (maxFileBytes: number) =>
+24
View File
@@ -0,0 +1,24 @@
import { expect, it } from "vitest";
import { BoundedListCollector, boundedList } from "./boundedList.js";
it("retains up to the maximum and counts every item past it", () => {
expect(boundedList([1, 2], 3)).toEqual({ items: [1, 2], omitted: 0 });
expect(boundedList([1, 2, 3, 4, 5], 3)).toEqual({
items: [1, 2, 3],
omitted: 2,
});
// Re-bounding a list keeps the count an earlier bound recorded.
expect(boundedList([0, 1, 2, 3], 3, 2)).toEqual({
items: [0, 1, 2],
omitted: 3,
});
});
it("collects up to the maximum and counts later pushes", () => {
const collector = new BoundedListCollector<string>(2);
for (const item of ["a", "b", "c", "d"]) collector.push(item);
expect(collector.retains("a")).toBe(true);
expect(collector.retains("c")).toBe(false);
expect(collector.toList()).toEqual({ items: ["a", "b"], omitted: 2 });
});
+44
View File
@@ -0,0 +1,44 @@
/**
* Items retained up to a maximum, with how many more were offered. A bounded
* list never drops an item without counting it, so a consumer can always say
* that more exist and how many.
*/
export interface BoundedList<T> {
readonly items: readonly T[];
readonly omitted: number;
}
/**
* Retain the first `maximum` items and count the rest, adding to `omitted`
* items an earlier bound already left out.
*/
export const boundedList = <T>(
items: readonly T[],
maximum: number,
omitted = 0,
): BoundedList<T> => ({
items: items.slice(0, maximum),
omitted: omitted + Math.max(0, items.length - maximum),
});
/** Collects items up to a maximum and counts each item offered past it. */
export class BoundedListCollector<T> {
readonly #items: T[] = [];
#omitted = 0;
constructor(readonly maximum: number) {}
push(item: T): void {
if (this.#items.length < this.maximum) this.#items.push(item);
else this.#omitted += 1;
}
/** Whether a retained item equals `item`; omitted items are not kept. */
retains(item: T): boolean {
return this.#items.includes(item);
}
toList(): BoundedList<T> {
return { items: [...this.#items], omitted: this.#omitted };
}
}