mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-10-26 16:55:08 +00:00
fix(ci): decide the dual-release install layout by graph scale
The Dependency layout lane failed whenever the npm child process exceeded a hardcoded 300 s deadline. That deadline measured the runner rather than the graph: the same layout resolves in 137-205 s on an idle developer host and exceeded 300 s on CI while the placement assertion reported no error, so the failure carried no information about what had grown. The gate now bounds the verified graph by resolution work units, dshPackagesPerVersion * checkedDshEdges against MAX_RESOLUTION_WORK_UNITS, and keeps the wall clock only as a hang guard derived from that budget. Growth fails with the graph's own measured counts on any runner. The resolution cost itself is unchanged. Measurement attributed 99.3% of it to npm's peer-conflict placement, and cache reuse, graph splitting and the placement flags were measured and rejected.
This commit is contained in:
1 parent
4e25074651
commit
f45693fbe5
6 files changed
+199
-3
No files matched your search
+22
@@ -0,0 +1,22 @@
|
||||
# Bilingual-pair consistency record for 2026-09-28-dual-release-install-layout-work-budget.md (docs/i18n/README.md): per heading
|
||||
# section, a hash of its English and Chinese blocks outside code blocks and generated regions.
|
||||
# After editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-09-28-dual-release-install-layout-work-budget.md
|
||||
/agent-note-dual-release-install-layout-work-budget:
|
||||
en: 808f7c1d11ce15b5
|
||||
zh: 861c63dd2de66452
|
||||
/agent-note-dual-release-install-layout-work-budget/problem:
|
||||
en: 0b826a202c56f154
|
||||
zh: 547d73c2e488283d
|
||||
/agent-note-dual-release-install-layout-work-budget/decision:
|
||||
en: 9ff96c5b733bc918
|
||||
zh: a2df4a171cefaf51
|
||||
/agent-note-dual-release-install-layout-work-budget/what-the-resolution-costs:
|
||||
en: ca5c1024209a9b96
|
||||
zh: a3ddc2d2ceea1989
|
||||
/agent-note-dual-release-install-layout-work-budget/alternatives-considered:
|
||||
en: 29987b7634c84d28
|
||||
zh: a1469e4b6aaa46e2
|
||||
/agent-note-dual-release-install-layout-work-budget/consequences:
|
||||
en: 4749c1f7aac32fc1
|
||||
zh: 2780c9784058f547
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
# Agent Note: Dual-release install layout work budget
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-09-28-dual-release-install-layout-work-budget.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The `Dependency layout` lane packs two incompatible synthetic DSH releases into a temporary consumer, runs one real npm resolution over them, and asserts the physical placement npm chose. It decided pass or fail with `TIMEOUT_MS = 300_000` on the npm child process. That deadline measures the runner, not the graph: the same commit resolves the same packages and internal edges in 153-190 s on a developer host and exceeded 300 s on the project's CI runner while the placement assertion reported no error. The diagnostic named only elapsed time, so a slow runner and a graph that had grown were indistinguishable.
|
||||
|
||||
## Decision
|
||||
|
||||
`scripts/verify-npm-install-layout.ts` decides growth with `assertResolutionWorkBudget`, which bounds `dshPackagesPerVersion * checkedDshEdges` against `MAX_RESOLUTION_WORK_UNITS = 875_000`. Both counts come from the summary `assertDualDshInstallLayout` returns, so the budget applies to the graph npm actually produced rather than to a prediction of it. A graph above the budget fails with its own counts, the budget, and the instruction to measure the new resolution cost and raise the constant.
|
||||
|
||||
The npm child keeps a wall-clock limit, `NPM_HANG_GUARD_MS = 900_000`. It is a hang guard, not the growth criterion: the budgeted graph costs about 226 s at the measured 2.6e-4 s per work unit, and the guard allows four times that for slower runners and host load.
|
||||
|
||||
## What the resolution costs
|
||||
|
||||
Measured 2026-09-28 on an idle M-series host, against `origin/master` (277 DSH packages per release, 2524 internal edges), by running the lane's own npm invocation with `--timing` and `--cpu-prof`:
|
||||
|
||||
| Phase | Cost |
|
||||
|---|---|
|
||||
| Registry synthesis (`buildRegistryIndex`, 3952 installed manifests plus 319 DSH workspace manifests into 1544 names) | 1.67 s |
|
||||
| npm child process | 180.66 s of 185.07 s end to end |
|
||||
| `idealTree:buildDeps` inside that child | 153.30 s, and nearly all of it lands in 14 of 877 per-node timers |
|
||||
| `checkCanPlace` / `canPlacePeers` peer-conflict analysis | 117.3 s, of which `new URL()` 65.9 s and `SemVer` 27.9 s |
|
||||
| Registry traffic | 682 requests, 383 KB of packument bodies |
|
||||
| Layout assertion and process startup | 4.4 s |
|
||||
|
||||
The same graph resolved in 1.12 s with `--legacy-peer-deps`, so peer placement is 152.7 s of the 153.8 s. `canPlacePeers` re-checks each internal edge against the incoming edges of its peer target, which makes the cost grow with placements times edge count; the nested release's copy of `@deepseek-ai/dsh-base` alone takes 48.8 s against 1.3 s for the identical package at the root of the same tree. Two thirds of the verified internal edges are peer edges (1874 of 2524), so this work is the assertion, not overhead around it.
|
||||
|
||||
Repeated runs of the same graph on one host span 137.8 s to 204.8 s of npm time, a factor of 1.5 with no input change, which is why a wall-clock threshold cannot separate a slow host from a larger graph.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Reuse one npm cache directory across runs.** The lane's `npm_config_cache` lives inside the temporary consumer, and the registry listens on an ephemeral port, so `make-fetch-happen` keys every cache entry as `http://127.0.0.1:<port>/<name>` and a persisted cache cannot hit. Re-running the full graph warm with `--prefer-offline` over the cold run's 4.4 MB cache produced a byte-identical package lock in 189.8 s, against 153.8 s cold: correct, and slower. The profile's 10.2 s of idle time is the ceiling on any network saving, because npm spends the rest of the run in its own placement code.
|
||||
|
||||
**Merge or parallelize the two releases.** The lane already resolves both releases in one npm invocation, which is what makes the co-existence claim checkable, and one release cannot reuse the other's peer analysis because `canPlacePeers` examines the whole tree. Splitting the graph into two independent resolutions would remove the shared-Cordis and cross-release placement facts the assertion exists to check.
|
||||
|
||||
**Raise `TIMEOUT_MS`.** A larger deadline leaves the failure mode in place: the verdict still depends on runner speed, and a graph that grows until it is slow on every machine still fails with no measurement of what grew.
|
||||
|
||||
**`--prefer-dedupe`.** It resolves the same graph in 126.8 s but chooses a different placement (809 lock entries against 815, eight differing entries), so the lane would assert a non-default npm placement.
|
||||
|
||||
**`--legacy-peer-deps`.** It resolves in 1.12 s and drops the 52 peer-installed packages, along with the peer edges that make up most of the assertion's coverage.
|
||||
|
||||
## Consequences
|
||||
|
||||
The lane's failure now depends on the graph, so a pull request that adds a few DSH packages passes on any runner, and a graph that grows by more than about a quarter fails deterministically with its measured counts. The budget is a ratchet: raising it is a deliberate edit that must come with a new resolution measurement, and the constant's comment records the counts it was derived from. A graph inside the budget still spends its full resolution time, because the change moves the decision and not the work; nothing here reduces the 153 s npm needs to place the dual release. Growth that changes the shape of the graph without changing packages times edges stays inside the budget and is caught only if the hang guard expires.
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
# Agent Note: 双发布安装布局的工作量预算
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-09-28-dual-release-install-layout-work-budget.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
`Dependency layout` lane 把两个互不兼容的合成 DSH 发布打进临时消费者目录,对它们跑一次真实 npm 解析,并断言 npm 选定的物理摆放。它用 npm 子进程上的 `TIMEOUT_MS = 300_000` 判定通过或失败。这个期限度量的是 runner,而不是图:同一个 commit 在开发机上用 153–190 秒解析同一批包和内部边,在项目 CI runner 上超过 300 秒,而摆放断言没有报告任何错误。诊断信息只给出耗时,因此 runner 慢和依赖图变大无法区分。
|
||||
|
||||
## Decision
|
||||
|
||||
`scripts/verify-npm-install-layout.ts` 用 `assertResolutionWorkBudget` 判定规模增长,它以 `dshPackagesPerVersion * checkedDshEdges` 为工作量单位,与 `MAX_RESOLUTION_WORK_UNITS = 875_000` 比较。两个计数都来自 `assertDualDshInstallLayout` 返回的汇总,因此预算作用于 npm 实际产出的图,而不是对图的预测。超过预算的图会带着自己的计数、预算值,以及「测量新的解析开销并调高该常量」的指示失败。
|
||||
|
||||
npm 子进程仍保留墙钟上限 `NPM_HANG_GUARD_MS = 900_000`。它是挂起保护,不是增长判据:按实测每个工作量单位 2.6e-4 秒计算,预算内的图约需 226 秒,该保护给出四倍余量以覆盖更慢的 runner 和主机负载。
|
||||
|
||||
## What the resolution costs
|
||||
|
||||
2026-09-28 在空闲的 M 系列主机上,针对 `origin/master`(每个发布 277 个 DSH 包、2524 条内部边),用 `--timing` 和 `--cpu-prof` 运行 lane 自身的 npm 调用测得:
|
||||
|
||||
| 阶段 | 开销 |
|
||||
|---|---|
|
||||
| Registry 合成(`buildRegistryIndex`,3952 个已安装 manifest 加 319 个 DSH workspace manifest,合成 1544 个包名) | 1.67 s |
|
||||
| npm 子进程 | 端到端 185.07 s 中的 180.66 s |
|
||||
| 该子进程内的 `idealTree:buildDeps` | 153.30 s,且几乎全部落在 877 个按节点计时器中的 14 个上 |
|
||||
| `checkCanPlace` / `canPlacePeers` peer 冲突分析 | 117.3 s,其中 `new URL()` 65.9 s、`SemVer` 27.9 s |
|
||||
| Registry 流量 | 682 次请求、383 KB packument 响应体 |
|
||||
| 布局断言与进程启动 | 4.4 s |
|
||||
|
||||
同一个依赖图加上 `--legacy-peer-deps` 后 1.12 s 解析完成,因此 peer 摆放占 153.8 s 中的 152.7 s。`canPlacePeers` 会把每条内部边与其 peer 目标的入边重新核对一遍,这使开销随「摆放次数 × 边数」增长;嵌套发布的 `@deepseek-ai/dsh-base` 副本单独耗时 48.8 s,而同一棵树上根位置的同一包只耗时 1.3 s。被校验的内部边有三分之二是 peer 边(2524 条中的 1874 条),所以这部分工作是断言本身,而不是断言之外的开销。
|
||||
|
||||
同一台主机上对同一张图的重复运行,npm 耗时在 137.8 s 到 204.8 s 之间波动,输入没有任何变化却有 1.5 倍差距,这正是墙钟阈值无法区分「主机慢」和「图更大」的原因。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**跨次复用同一个 npm cache 目录。** lane 的 `npm_config_cache` 位于临时消费者目录内,registry 监听临时端口,因此 `make-fetch-happen` 把每个 cache 条目键为 `http://127.0.0.1:<port>/<name>`,持久化的 cache 无法命中。用冷启动那次留下的 4.4 MB cache 加 `--prefer-offline` 重跑整张图,产出的 package lock 逐字节相同,耗时 189.8 s,而冷启动为 153.8 s:结果正确,但更慢。profile 中 10.2 s 的空闲时间是任何网络节省的上限,因为 npm 把其余时间都花在自己的摆放代码里。
|
||||
|
||||
**合并或并行两个发布。** lane 已经在一次 npm 调用里解析两个发布,这正是共存声明可被校验的原因;而 `canPlacePeers` 会检查整棵树,一个发布无法复用另一个的 peer 分析。把图拆成两次相互独立的解析会丢掉断言存在的理由:共享 Cordis 与跨发布摆放。
|
||||
|
||||
**调大 `TIMEOUT_MS`。** 更大的期限保留了原有失效模式:判定仍取决于 runner 速度,而一个增长到在任何机器上都慢的图仍会失败,且不给出任何关于「什么变大了」的测量。
|
||||
|
||||
**`--prefer-dedupe`。** 它用 126.8 s 解析同一张图,但选择了不同的摆放(809 条 lock 条目对 815 条,8 条条目不同),因此 lane 断言的将是非默认的 npm 摆放。
|
||||
|
||||
**`--legacy-peer-deps`。** 它 1.12 s 完成解析,但丢掉了 52 个由 peer 安装的包,也丢掉了构成断言大部分覆盖面的 peer 边。
|
||||
|
||||
## Consequences
|
||||
|
||||
lane 的失败判定如今取决于依赖图,因此新增少量 DSH 包的 pull request 在任何 runner 上都能通过,而增长超过约四分之一的图会带着实测计数确定性地失败。预算是棘轮:调高它是一次刻意修改,必须附带新的解析实测,该常量注释里记录了推导所用的计数。预算内的图仍会花掉完整的解析时间,因为这次改动移动的是判定位置而不是工作本身;这里没有任何东西能减少 npm 摆放双发布所需的 153 s。若图的增长改变了形状却没有改变「包数乘边数」,它仍留在预算内,只有在挂起保护到期时才被发现。
|
||||
@@ -417,12 +417,14 @@ async function runNpm(
|
||||
* @param index - Package metadata exposed through the local registry.
|
||||
* @param dependencies - Root dependencies whose install layout npm computes.
|
||||
* @param timeoutMs - Hard wall-clock limit for the npm child process.
|
||||
* @param timeoutSubject - What the limit guards, named in the timeout diagnostic.
|
||||
* @returns The package lock plus timing and registry-request observations.
|
||||
*/
|
||||
export async function resolveNpmPackageLock(
|
||||
index: RegistryIndex,
|
||||
dependencies: Readonly<Record<string, string>>,
|
||||
timeoutMs: number,
|
||||
timeoutSubject = 'npm resolution',
|
||||
): Promise<NpmPackageLockResolution> {
|
||||
let registryRequests = 0
|
||||
let archiveRequests = 0
|
||||
@@ -471,7 +473,7 @@ export async function resolveNpmPackageLock(
|
||||
dependencies,
|
||||
}, null, 2)}\n`)
|
||||
const result = await runNpm(consumer, registry, timeoutMs)
|
||||
if (result.timedOut) throw new Error(`npm resolution exceeded ${String(timeoutMs)} ms`)
|
||||
if (result.timedOut) throw new Error(`${timeoutSubject} exceeded ${String(timeoutMs)} ms`)
|
||||
return {
|
||||
durationMs: result.durationMs,
|
||||
registryRequests,
|
||||
|
||||
@@ -2,7 +2,9 @@ import { describe, expect, it } from 'vitest'
|
||||
import type { NpmPackageLock, RegistryIndex } from './benchmark-npm-resolution.ts'
|
||||
import {
|
||||
assertDualDshInstallLayout,
|
||||
assertResolutionWorkBudget,
|
||||
buildDualDshRegistry,
|
||||
MAX_RESOLUTION_WORK_UNITS,
|
||||
} from './verify-npm-install-layout.ts'
|
||||
|
||||
function validLayout(): NpmPackageLock {
|
||||
@@ -113,3 +115,30 @@ describe('npm install layout verifier', () => {
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
describe('resolution work budget', () => {
|
||||
it('accepts the measured dual-release graph', () => {
|
||||
expect(assertResolutionWorkBudget({ dshPackagesPerVersion: 277, checkedDshEdges: 2524 }))
|
||||
.toBe(699_148)
|
||||
})
|
||||
|
||||
it('accepts a graph exactly at the budget and rejects one internal edge above it', () => {
|
||||
const packages = 350
|
||||
const edgesAtBudget = Math.floor(MAX_RESOLUTION_WORK_UNITS / packages)
|
||||
|
||||
expect(packages * edgesAtBudget).toBe(MAX_RESOLUTION_WORK_UNITS)
|
||||
expect(assertResolutionWorkBudget({ dshPackagesPerVersion: packages, checkedDshEdges: edgesAtBudget }))
|
||||
.toBe(MAX_RESOLUTION_WORK_UNITS)
|
||||
expect(() => assertResolutionWorkBudget({
|
||||
dshPackagesPerVersion: packages,
|
||||
checkedDshEdges: edgesAtBudget + 1,
|
||||
})).toThrow(`= ${String(MAX_RESOLUTION_WORK_UNITS + packages)} unit(s), budget ${String(MAX_RESOLUTION_WORK_UNITS)} unit(s)`)
|
||||
})
|
||||
|
||||
it('accepts the headroom and rejects a runaway graph with its own measured counts', () => {
|
||||
expect(assertResolutionWorkBudget({ dshPackagesPerVersion: 300, checkedDshEdges: 2800 }))
|
||||
.toBe(840_000)
|
||||
expect(() => assertResolutionWorkBudget({ dshPackagesPerVersion: 400, checkedDshEdges: 3000 }))
|
||||
.toThrow('400 package(s) per release x 3000 internal edge(s) = 1200000 unit(s), budget 875000 unit(s)')
|
||||
})
|
||||
})
|
||||
@@ -15,7 +15,31 @@ const CORDIS_PACKAGE = '@deepseek-ai/cordis'
|
||||
const NESTED_DSH_ALIAS = 'dsh-previous'
|
||||
const NESTED_DSH_PATH = `node_modules/${NESTED_DSH_ALIAS}`
|
||||
const DEPENDENCY_FIELDS = ['dependencies', 'optionalDependencies', 'peerDependencies'] as const
|
||||
const TIMEOUT_MS = 300_000
|
||||
|
||||
/**
|
||||
* Resolution work accepted from the dual-release graph, in
|
||||
* `dshPackagesPerVersion * checkedDshEdges` units.
|
||||
*
|
||||
* npm's hoisted placement re-checks every internal edge against the incoming
|
||||
* edges of its peer target (`canPlacePeers` calls `checkCanPlaceNoCurrent`,
|
||||
* which walks `depValid` over that target's edges), so resolution time grows
|
||||
* with this product and not with the package count alone. Measured on an idle
|
||||
* M-series host: 277 package(s) x 2524 internal edge(s) = 699,148 unit(s) cost
|
||||
* 180.7 s of npm time out of 185.1 s end to end. The budget adds 25% headroom
|
||||
* for ordinary product growth, so a graph that grows here fails with its own
|
||||
* measured counts instead of failing on whichever runner happens to be slow.
|
||||
* Raising it requires re-measuring the resolution and recording the new counts.
|
||||
*/
|
||||
export const MAX_RESOLUTION_WORK_UNITS = 875_000
|
||||
|
||||
/**
|
||||
* Wall-clock guard for the npm child process. This is a hang guard, not the
|
||||
* gate's growth criterion: {@link MAX_RESOLUTION_WORK_UNITS} decides whether the
|
||||
* graph is too large to verify. A budgeted graph costs about 226 s at the
|
||||
* measured 2.6e-4 s per work unit, and the guard allows four times that, so a
|
||||
* slower runner can never decide this gate's outcome.
|
||||
*/
|
||||
const NPM_HANG_GUARD_MS = 900_000
|
||||
|
||||
/** Synthetic incompatible versions used to expose cross-release placement errors. */
|
||||
export const SYNTHETIC_DSH_VERSIONS = ['0.1.0', '0.2.0'] as const
|
||||
@@ -186,6 +210,27 @@ export function assertDualDshInstallLayout(packageLock: NpmPackageLock): DshInst
|
||||
return { dshPackagesPerVersion: rootNames.size, checkedDshEdges }
|
||||
}
|
||||
|
||||
/**
|
||||
* Reject a verified dual-release graph that exceeds the resolution work budget.
|
||||
* @param summary - Counts returned by {@link assertDualDshInstallLayout}.
|
||||
* @returns The verified resolution work units.
|
||||
*/
|
||||
export function assertResolutionWorkBudget(summary: DshInstallLayoutSummary): number {
|
||||
const units = summary.dshPackagesPerVersion * summary.checkedDshEdges
|
||||
if (units > MAX_RESOLUTION_WORK_UNITS) {
|
||||
throw new Error(
|
||||
'dual-release graph exceeds the resolution work budget: '
|
||||
+ `${String(summary.dshPackagesPerVersion)} package(s) per release x `
|
||||
+ `${String(summary.checkedDshEdges)} internal edge(s) = ${String(units)} unit(s), `
|
||||
+ `budget ${String(MAX_RESOLUTION_WORK_UNITS)} unit(s)\n`
|
||||
+ ' npm re-checks every internal edge against the incoming edges of its peer target, so the resolution\n'
|
||||
+ ' cost grows with this product. Measure the new cost and raise MAX_RESOLUTION_WORK_UNITS in\n'
|
||||
+ ' scripts/verify-npm-install-layout.ts.',
|
||||
)
|
||||
}
|
||||
return units
|
||||
}
|
||||
|
||||
function workspaceVersion(root: string): string {
|
||||
const manifest = JSON.parse(readFileSync(resolve(root, 'apps/cli/package.json'), 'utf8')) as { version?: unknown }
|
||||
if (typeof manifest.version !== 'string') throw new Error('apps/cli/package.json has no string version')
|
||||
@@ -199,12 +244,14 @@ async function main(): Promise<void> {
|
||||
const result = await resolveNpmPackageLock(index, {
|
||||
[DSH_PACKAGE]: rootVersion,
|
||||
[NESTED_DSH_ALIAS]: `npm:${DSH_PACKAGE}@${nestedVersion}`,
|
||||
}, TIMEOUT_MS)
|
||||
}, NPM_HANG_GUARD_MS, 'npm resolution hang guard')
|
||||
if (result.archiveRequests !== 0) throw new Error(`npm requested ${String(result.archiveRequests)} package archive(s)`)
|
||||
const summary = assertDualDshInstallLayout(result.packageLock)
|
||||
const units = assertResolutionWorkBudget(summary)
|
||||
console.log(
|
||||
`verify-npm-install-layout: ${String(summary.dshPackagesPerVersion)} DSH package(s) per release and `
|
||||
+ `${String(summary.checkedDshEdges)} internal edge(s) verified in ${(result.durationMs / 1000).toFixed(2)} s; `
|
||||
+ `${String(units)} of ${String(MAX_RESOLUTION_WORK_UNITS)} budgeted resolution work unit(s); `
|
||||
+ `both releases share one Cordis installation; ${String(result.unknownPackages.length)} unavailable optional `
|
||||
+ 'package name(s) ignored by npm.',
|
||||
)
|
||||
|
||||
Reference in new issue
Block a user