mirror of
https://github.com/morluto/rea.git
synced 2026-11-11 01:13:39 +00:00
fix(javascript): apply package type scope to application sources (#7883)
* fix(javascript): apply package type scope to application sources Application analysis parsed .js sources without their package scope, so an unbound require in a type-module package became a resolved CommonJS dependency. The parent now observes each source's nearest same-container package.json, stopping at node_modules, and the worker parses with that package type as reference imports already do. Fixes #7323 * fix(javascript): keep package type unknown behind unavailable manifests The package-type lookup indexed only readable files, so a nearer package.json whose bytes were unavailable was skipped and an ancestor's "type" applied to sources it does not own. Index unavailable manifests by container and directory too, and stop the walk with "unknown" when one is the nearest owner, as the Next app-context lookup already does. * fix(javascript): keep Electron preloads outside module package scope Electron's preload loader ignores a package's "type": "module" and gives a .js preload `require`; ESM preloads need the .mjs suffix. Applying the package type to every .js source marked such a preload as an ES module, so its `require("electron")` edges and IPC pairing disappeared. A source that requires an Electron module now keeps the context REA inferred before package scope was applied. Node-loaded sources in a module scope still drop unbound `require`. * fix(javascript): admit Electron script loaders only on renderer evidence The previous commit dropped a module package type whenever a source required an Electron module. A Node-loaded ES module that requires Electron, or one whose Electron call goes through a local `require`, then regained the false require edges the package scope removes. The parser applies the package type unconditionally again. The worker re-reads a package-module script that requires an Electron module once outside the package scope. The source keeps that context only when the static pass there directly observes a contextBridge exposure or a renderer IPC operation, which only Electron's renderer-side loaders provide. * fix(javascript): check Electron renderer evidence without a second static passb54845a6bran the full static pass on an unscoped parse of every package-module script that requires Electron, then discarded it when no renderer operation turned up, so such main-process sources paid for two full passes. A candidate must now also name contextBridge or ipcRenderer, which any renderer evidence needs. The evidence check runs only the static pass's binding and Electron identity preamble with its exposure and IPC descriptors. The full static pass runs once, on the chosen parse. * fix(javascript): confirm Electron renderer evidence with the static pass The focused evidence check from4a86bca9balso counted renderer calls in bundler factory defaults that the bundler convention never runs, which the static pass skips. Such a default switched a type-module file to script context, so its unrelated unbound requires became imports again. The focused check now only screens. It follows the static pass's own traversal, so a negative result still rules evidence out. A positive result is confirmed by the static pass's direct findings on the unscoped parse, and those findings become the file's facts when they hold. * fix(javascript): confirm renderer evidence before wrapper substitutionf71236400confirmed a screened source from the static pass's final contextBridge and IPC findings, without wrapper substitutions. The wrapper pass replaces a directly observed exposure inside a helper with a substituted finding, so such a preload lost its imports again. Before its wrapper pass, the static pass now records whether its source traversal directly observed a contextBridge exposure or a renderer IPC operation, and observedElectronRendererCall exposes that. The traversal still skips bundler factory defaults that never run, and the reported findings are unchanged. * fix(javascript): resolve declared preload loaders before source analysis * fix(javascript): leave preload discovery failures to source analysis A source that failed during preload discovery threw out of the whole application analysis, so one oversized source or a per-source timeout in a module package returned no Evidence. Skip a source that exceeds its worker limits and end discovery on any other failure except cancellation; source analysis then meets the same source, records its failure and keeps partial Evidence as it does without discovery. * fix(javascript): report preload discovery gaps in Evidence limitations When preload discovery skipped a source or stopped early, source analysis could still read that source, so the result looked complete while a .js preload declared there kept its package scope. Return each discovery gap with the discovered paths and add it to the analysis limitations, which both complete and partial Evidence carry.
This commit is contained in:
16 files changed
+844
-32
No files matched your search
@@ -0,0 +1,83 @@
|
||||
import { expect, it } from "vitest";
|
||||
import { analyzeAndAdoptJavaScriptArtifactFiles } from "./JavaScriptArtifactAnalysis.js";
|
||||
import { createJavaScriptSemanticGraphProjection } from "./JavaScriptSemanticGraphBuilder.js";
|
||||
import { analyzeJavaScriptStaticSource } from "../../domain/javascript/javascriptStaticAnalysis.js";
|
||||
import type {
|
||||
JavaScriptArtifactFile,
|
||||
JavaScriptUnanalyzedArtifactFile,
|
||||
} from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||
import type { JavaScriptPackageType } from "../../domain/javascript/javascriptSourceParser.js";
|
||||
import { ok } from "../../domain/result.js";
|
||||
|
||||
const container = "a".repeat(64);
|
||||
|
||||
const included = (
|
||||
path: string,
|
||||
kind: JavaScriptArtifactFile["kind"],
|
||||
text: string,
|
||||
): JavaScriptArtifactFile => ({
|
||||
path,
|
||||
container_sha256: container,
|
||||
sha256: "b".repeat(64),
|
||||
bytes: Buffer.byteLength(text),
|
||||
inventory_artifact_id: path,
|
||||
kind,
|
||||
unpacked: false,
|
||||
text: { included: true, value: text },
|
||||
});
|
||||
|
||||
it("leaves the package type unknown behind a nearer unavailable manifest", async () => {
|
||||
const files = [
|
||||
included("package.json", "package-json", '{"type":"module"}'),
|
||||
included("main.js", "javascript", 'require("./dep.cjs");'),
|
||||
included("nested/main.js", "javascript", 'require("../dep.cjs");'),
|
||||
included(
|
||||
"nested/inner/package.json",
|
||||
"package-json",
|
||||
'{"type":"commonjs"}',
|
||||
),
|
||||
included("nested/inner/main.js", "javascript", 'require("../../dep.cjs");'),
|
||||
];
|
||||
const unavailable: JavaScriptUnanalyzedArtifactFile = {
|
||||
path: "nested/package.json",
|
||||
container_sha256: container,
|
||||
sha256: null,
|
||||
bytes: null,
|
||||
inventory_artifact_id: null,
|
||||
kind: "package-json",
|
||||
limitations: ["Inventoried package bytes are unavailable."],
|
||||
};
|
||||
const observed = new Map<string, JavaScriptPackageType | undefined>();
|
||||
const javascript = analyzeJavaScriptStaticSource("");
|
||||
const worker = {
|
||||
analyze: async (
|
||||
file: JavaScriptArtifactFile,
|
||||
_capacity: unknown,
|
||||
_signal?: AbortSignal,
|
||||
_onPhase?: unknown,
|
||||
packageType?: JavaScriptPackageType,
|
||||
) => {
|
||||
observed.set(file.path, packageType);
|
||||
return ok({ javascript, module: null, projection: null });
|
||||
},
|
||||
};
|
||||
await analyzeAndAdoptJavaScriptArtifactFiles(
|
||||
{
|
||||
files,
|
||||
unavailable_relevant_files: [unavailable],
|
||||
containers: [],
|
||||
text_bytes_read: 0,
|
||||
invalid_utf8_files: 0,
|
||||
},
|
||||
worker,
|
||||
createJavaScriptSemanticGraphProjection(),
|
||||
async () => {},
|
||||
);
|
||||
// The nearest package.json owns a source's scope even when its bytes are
|
||||
// unavailable, so the root type must not reach nested/main.js.
|
||||
expect(Object.fromEntries(observed)).toEqual({
|
||||
"main.js": "module",
|
||||
"nested/main.js": "unknown",
|
||||
"nested/inner/main.js": "commonjs",
|
||||
});
|
||||
});
|
||||
@@ -1,7 +1,10 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { posix } from "node:path";
|
||||
|
||||
import { setImmediate } from "node:timers/promises";
|
||||
|
||||
import type { JavaScriptPackageType } from "../../domain/javascript/javascriptSourceParser.js";
|
||||
|
||||
import { resolveJavaScriptSourceMapReferences } from "../../domain/javascript/javascriptSourceMapPaths.js";
|
||||
import { hasValidSourceMapContents } from "../../domain/sourceMapContents.js";
|
||||
import {
|
||||
@@ -77,12 +80,19 @@ export const analyzeAndAdoptJavaScriptArtifactFiles = async (
|
||||
) => Promise<void>,
|
||||
signal?: AbortSignal,
|
||||
onPhase?: (file: JavaScriptArtifactFile, phase: string) => Promise<void>,
|
||||
preloadPaths: ReadonlySet<string> = new Set(),
|
||||
/** Discovery gaps that may leave a declared preload in its package scope. */
|
||||
preloadLimitations: readonly string[] = [],
|
||||
): Promise<{
|
||||
readonly analysis: JavaScriptArtifactAnalysis;
|
||||
readonly interruption: AnalysisError | null;
|
||||
}> => {
|
||||
const state = emptyArtifactAnalysis();
|
||||
const sourceIndices: number[] = [];
|
||||
const packageTypeFor = createJavaScriptPackageTypeLookup(
|
||||
fileSet,
|
||||
state.jsonDocuments,
|
||||
);
|
||||
let interruption: AnalysisError | null = null;
|
||||
const selectedFileCount =
|
||||
fileSet.files.length - (fileSet.source_selection?.unselected_files ?? 0);
|
||||
@@ -118,6 +128,10 @@ export const analyzeAndAdoptJavaScriptArtifactFiles = async (
|
||||
projection.remainingFileCapacity(),
|
||||
signal,
|
||||
onPhase === undefined ? undefined : (phase) => onPhase(file, phase),
|
||||
preloadPaths.has(`${file.container_sha256}:${file.path}`) &&
|
||||
/\.js$/iu.test(file.path)
|
||||
? "commonjs"
|
||||
: packageTypeFor(file),
|
||||
);
|
||||
const facts = result.ok ? result.value : result.error;
|
||||
for (const revoked of facts.revokedStaticProjections ?? []) {
|
||||
@@ -294,11 +308,15 @@ export const analyzeAndAdoptJavaScriptArtifactFiles = async (
|
||||
);
|
||||
}
|
||||
}
|
||||
return { analysis: finalizeArtifactAnalysis(state), interruption };
|
||||
return {
|
||||
analysis: finalizeArtifactAnalysis(state, preloadLimitations),
|
||||
interruption,
|
||||
};
|
||||
};
|
||||
|
||||
const finalizeArtifactAnalysis = (
|
||||
state: MutableArtifactAnalysis,
|
||||
preloadLimitations: readonly string[],
|
||||
): JavaScriptArtifactAnalysis => {
|
||||
return {
|
||||
jsonDocuments: state.jsonDocuments,
|
||||
@@ -328,6 +346,7 @@ const finalizeArtifactAnalysis = (
|
||||
(failure) => (failure === undefined ? [] : [failure.reason]),
|
||||
),
|
||||
),
|
||||
...preloadLimitations,
|
||||
],
|
||||
};
|
||||
};
|
||||
@@ -386,6 +405,54 @@ const addStructuredObservations = (
|
||||
if (json.status === "invalid") state.parseFailures += 1;
|
||||
};
|
||||
|
||||
/**
|
||||
* Node's package scope for a source is its nearest package.json; the lookup
|
||||
* stops at a `node_modules` directory (Node's package scope resolution). Only
|
||||
* a same-container, readable manifest object establishes the type; any other
|
||||
* nearest manifest, including one whose bytes are unavailable, or none in the
|
||||
* inventory, leaves the scope unknown.
|
||||
*/
|
||||
export const createJavaScriptPackageTypeLookup = (
|
||||
fileSet: JavaScriptArtifactFileSet,
|
||||
jsonDocuments: JavaScriptJsonDocuments,
|
||||
): ((file: JavaScriptArtifactFile) => JavaScriptPackageType) => {
|
||||
const manifests = new Map<string, JavaScriptArtifactFile>();
|
||||
for (const file of fileSet.files)
|
||||
if (posix.basename(file.path) === "package.json")
|
||||
manifests.set(
|
||||
`${file.container_sha256}:${posix.dirname(file.path)}`,
|
||||
file,
|
||||
);
|
||||
const unavailableManifests = new Set(
|
||||
fileSet.unavailable_relevant_files
|
||||
.filter(({ path }) => posix.basename(path) === "package.json")
|
||||
.map(
|
||||
({ path, container_sha256 }) =>
|
||||
`${container_sha256}:${posix.dirname(path)}`,
|
||||
),
|
||||
);
|
||||
return (file) => {
|
||||
for (
|
||||
let directory = posix.dirname(file.path);
|
||||
posix.basename(directory) !== "node_modules";
|
||||
directory = posix.dirname(directory)
|
||||
) {
|
||||
const key = `${file.container_sha256}:${directory}`;
|
||||
if (unavailableManifests.has(key)) return "unknown";
|
||||
const manifest = manifests.get(key);
|
||||
if (manifest !== undefined) {
|
||||
const document = jsonDocuments.read(manifest);
|
||||
if (document.status !== "included" || document.packageFields === null)
|
||||
return "unknown";
|
||||
const type = document.packageFields.type;
|
||||
return type === "module" || type === "commonjs" ? type : "unspecified";
|
||||
}
|
||||
if (directory === "." || directory === "" || directory === "/") break;
|
||||
}
|
||||
return "unknown";
|
||||
};
|
||||
};
|
||||
|
||||
const parsePackage = (
|
||||
file: JavaScriptArtifactFile,
|
||||
document: JavaScriptJsonDocumentFacts,
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { discoverJavaScriptPreloadPaths } from "./JavaScriptPreloadDiscovery.js";
|
||||
import { artifactNameForClassification } from "../../domain/artifactNameSelection.js";
|
||||
import type { AnalysisSettingValues } from "../../domain/analysisSettings.js";
|
||||
import { absoluteCallerPath } from "../../filesystem/callerPath.js";
|
||||
@@ -180,17 +181,24 @@ export const reconstructJavaScriptArtifact = async (
|
||||
? {}
|
||||
: { fileNodes: input.max_semantic_nodes_per_file }),
|
||||
});
|
||||
const sourceLimits = {
|
||||
heapMb:
|
||||
input.max_heap_mb ??
|
||||
overrides?.["javascript.max_heap_mb"] ??
|
||||
JAVASCRIPT_ANALYSIS_RESOURCE_DEFAULTS.heapMb,
|
||||
timeoutMs:
|
||||
input.analysis_timeout_ms ??
|
||||
overrides?.["javascript.analysis_timeout_ms"] ??
|
||||
JAVASCRIPT_ANALYSIS_RESOURCE_DEFAULTS.timeoutMs,
|
||||
};
|
||||
const preloads = await discoverJavaScriptPreloadPaths(
|
||||
files,
|
||||
sourceLimits,
|
||||
scope,
|
||||
signal,
|
||||
);
|
||||
const worker = await createJavaScriptSourceAnalysis(
|
||||
{
|
||||
heapMb:
|
||||
input.max_heap_mb ??
|
||||
overrides?.["javascript.max_heap_mb"] ??
|
||||
JAVASCRIPT_ANALYSIS_RESOURCE_DEFAULTS.heapMb,
|
||||
timeoutMs:
|
||||
input.analysis_timeout_ms ??
|
||||
overrides?.["javascript.analysis_timeout_ms"] ??
|
||||
JAVASCRIPT_ANALYSIS_RESOURCE_DEFAULTS.timeoutMs,
|
||||
},
|
||||
sourceLimits,
|
||||
files.files.filter(
|
||||
(file) => file.kind === "javascript" && file.text.included,
|
||||
).length,
|
||||
@@ -229,6 +237,8 @@ export const reconstructJavaScriptArtifact = async (
|
||||
});
|
||||
abortIfNeeded(signal);
|
||||
},
|
||||
preloads.paths,
|
||||
preloads.limitations,
|
||||
);
|
||||
} finally {
|
||||
try {
|
||||
|
||||
@@ -54,6 +54,7 @@ const PACKAGE_PROPERTIES = [
|
||||
"browser",
|
||||
"exports",
|
||||
"imports",
|
||||
"type",
|
||||
] as const;
|
||||
|
||||
/**
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
import { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisResourceConstraintError,
|
||||
} from "../../domain/analysisErrorCore.js";
|
||||
import { translateArtifactFailure } from "../../artifacts/ArtifactOperationFailure.js";
|
||||
import type { JavaScriptArtifactFileSet } from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||
import type { JavaScriptSourceAnalysisLimits } from "../../domain/javascript/javascriptSourceAnalysis.js";
|
||||
import type { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
|
||||
import { createJavaScriptSourceAnalysis } from "../../artifacts/javascript/JavaScriptSourceAnalysis.js";
|
||||
import { createJavaScriptPackageTypeLookup } from "./JavaScriptArtifactAnalysis.js";
|
||||
import { JavaScriptJsonDocuments } from "./JavaScriptJsonDocuments.js";
|
||||
import { resolveArtifactPathByContext } from "./JavaScriptArtifactPathResolution.js";
|
||||
|
||||
/** Declared preload identities, with the discovery gaps that may hide others. */
|
||||
export interface JavaScriptPreloadDiscovery {
|
||||
readonly paths: ReadonlySet<string>;
|
||||
readonly limitations: readonly string[];
|
||||
}
|
||||
|
||||
/** Resolve declared preload identities before choosing any .js file's loader.
|
||||
* Discovery uses the same bounded worker and path owner as normal analysis.
|
||||
* It never infers a loader from renderer API text in the selected file.
|
||||
*/
|
||||
export const discoverJavaScriptPreloadPaths = async (
|
||||
fileSet: JavaScriptArtifactFileSet,
|
||||
limits: JavaScriptSourceAnalysisLimits,
|
||||
scope: ArtifactResourceScope,
|
||||
signal?: AbortSignal,
|
||||
): Promise<JavaScriptPreloadDiscovery> => {
|
||||
const paths = new Set<string>();
|
||||
const limitations: string[] = [];
|
||||
const jsonDocuments = new JavaScriptJsonDocuments();
|
||||
const packageTypeFor = createJavaScriptPackageTypeLookup(
|
||||
fileSet,
|
||||
jsonDocuments,
|
||||
);
|
||||
if (
|
||||
!fileSet.files.some(
|
||||
(file) =>
|
||||
file.kind === "javascript" &&
|
||||
file.text.included &&
|
||||
/\.js$/iu.test(file.path) &&
|
||||
packageTypeFor(file) === "module",
|
||||
)
|
||||
)
|
||||
return { paths, limitations };
|
||||
const sources = fileSet.files.filter(
|
||||
(file) => file.kind === "javascript" && file.text.included,
|
||||
);
|
||||
const files = new Map(fileSet.files.map((file) => [file.path, file]));
|
||||
const worker = await createJavaScriptSourceAnalysis(limits, sources.length, {
|
||||
scope,
|
||||
});
|
||||
let outcome: { kind: "completed" } | { kind: "failed"; cause: unknown } = {
|
||||
kind: "completed",
|
||||
};
|
||||
try {
|
||||
for (const [index, file] of sources.entries()) {
|
||||
const result = await worker.analyze(
|
||||
file,
|
||||
{ nodes: 0, unknowns: 0 },
|
||||
signal,
|
||||
undefined,
|
||||
packageTypeFor(file),
|
||||
"preload-discovery",
|
||||
);
|
||||
if (!result.ok) {
|
||||
const { error } = result.error;
|
||||
if (signal?.aborted === true || error instanceof AnalysisCancelledError)
|
||||
throw error;
|
||||
// Normal analysis meets the same source and owns its failure and the
|
||||
// partial Evidence. A per-source limit only loses this source's
|
||||
// declarations; any other failure would stop that pass here too. The
|
||||
// gap stays reported even if normal analysis then reads the source.
|
||||
const skipped =
|
||||
error instanceof AnalysisResourceConstraintError &&
|
||||
error.reportedLimits?.parent_result_budget_exhausted !== true;
|
||||
const later = sources.length - index - 1;
|
||||
limitations.push(
|
||||
skipped || later === 0
|
||||
? `Preload discovery could not read ${file.path} (${error.message}); a .js preload declared only there keeps its package scope.`
|
||||
: `Preload discovery stopped at ${file.path} (${error.message}); a .js preload declared only there or in ${String(later)} later sources keeps its package scope.`,
|
||||
);
|
||||
if (skipped) continue;
|
||||
break;
|
||||
}
|
||||
for (const window of result.value.javascript.electron.browser_windows) {
|
||||
if (window.preload_path === null) continue;
|
||||
const resolution = resolveArtifactPathByContext({
|
||||
declaredPath: window.preload_path,
|
||||
sourcePath: file.path,
|
||||
context: window.preload_resolution_context,
|
||||
files,
|
||||
jsonDocuments,
|
||||
unanalyzedFiles: fileSet.unanalyzed_files,
|
||||
runtimePathKind: "exact-file",
|
||||
});
|
||||
if (
|
||||
resolution.resolution_status !== "resolved" ||
|
||||
resolution.resolved_path === null
|
||||
)
|
||||
continue;
|
||||
const target = files.get(resolution.resolved_path);
|
||||
if (target?.container_sha256 === file.container_sha256)
|
||||
paths.add(`${target.container_sha256}:${target.path}`);
|
||||
}
|
||||
}
|
||||
} catch (cause: unknown) {
|
||||
outcome = { kind: "failed", cause };
|
||||
throw cause;
|
||||
} finally {
|
||||
// The scoped port parks a healthy process; the following checkout resets
|
||||
// its per-analysis accounting, and scope cleanup still owns failed closes.
|
||||
const cleanup = await scope.release({
|
||||
kind: "javascript-source-analysis",
|
||||
analysis: worker,
|
||||
resource: "JavaScript preload discovery",
|
||||
});
|
||||
if (cleanup.kind === "failed") {
|
||||
if (outcome.kind === "failed" && outcome.cause instanceof AnalysisError)
|
||||
throw analysisErrorWithCleanupFailure(
|
||||
outcome.cause,
|
||||
translateArtifactFailure(
|
||||
"analyze_javascript_application",
|
||||
scope.cleanupFailure({ kind: "completed" }, [cleanup]),
|
||||
),
|
||||
"analyze_javascript_application",
|
||||
);
|
||||
throw scope.cleanupFailure(outcome, [cleanup]);
|
||||
}
|
||||
}
|
||||
return { paths, limitations };
|
||||
};
|
||||
Reference in new issue
Block a user