mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-10-26 16:55:08 +00:00
ci: adapt concurrency to available runner CPUs (#4518)
* ci: adapt worker pools to available runner CPUs * ci: preserve local snapshot worker defaults * test: isolate CI worker overrides in gate fixtures * docs: clarify shared Windows worker limits
This commit is contained in:
1 parent
de742f7f45
commit
9d5fc30632
9 files changed
+139
-39
No files matched your search
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md
|
||||
2026-07-26-ci-failover-runbook.md: 10123fe1999c0ad03f977e1cc7c69d788a679c2b
|
||||
2026-07-26-ci-failover-runbook.zh.md: cff99e6f644bf945f8366c2a723b32222ab902b4
|
||||
2026-07-26-ci-failover-runbook.md: c2fd2fa6cf673ec4d202702c16a9a85a3a609b8d
|
||||
2026-07-26-ci-failover-runbook.zh.md: 7fa5887bb552afd98cebd7502202fb5e9f37b1ee
|
||||
@@ -32,7 +32,7 @@ The two switches are independent: flip only the one whose platform is degraded.
|
||||
|
||||
1. Repository **Settings → Secrets and variables → Actions → Variables → New repository variable**: name `DSH_CI_FAILOVER_LINUX` (Linux pool outage) or `DSH_CI_FAILOVER_WINDOWS` (Windows pool outage), value `selfhosted`.
|
||||
2. Retrigger the required jobs so they re-resolve their pool. Jobs already **queued** for the hosted labels do not retarget and cannot be re-run in place, so for the documented indefinite-queue outage, cancel the stuck run and re-run all jobs, or push a new commit; "Re-run failed jobs" only helps once a job has actually failed rather than queued.
|
||||
3. That is the entire switch. Under the `selfhosted` Linux failover value the workflow also drops `DSH_SNAPSHOT_MAX_CONCURRENCY` to 12 for the shared VM and skips the hosted-path pnpm cache restores because the VM's persistent store serves warm installs. Coverage uses the same four single-worker instrumented partitions and two exempt workers on both Linux pools. The Windows switch has no concurrency or cache branches; it only retargets the native Windows jobs' pool.
|
||||
3. That is the entire switch. Under the `selfhosted` Linux failover value the workflow also drops `DSH_SNAPSHOT_MAX_CONCURRENCY` to 12 for the shared VM and skips the hosted-path pnpm cache restores because the VM's persistent store serves warm installs. Self-hosted coverage uses four single-worker instrumented partitions and two exempt workers; isolated hosted runners derive worker defaults from available CPUs. The Windows switch selects the native Windows pool and retains explicit worker limits on shared self-hosted runners.
|
||||
|
||||
**Dependabot exception.** Both switches' `selfhosted` legs deliberately exclude `dependabot[bot]`: under self-hosted failover, Dependabot PRs stay queued for the hosted pool rather than executing dependency-supplied code on the persistent VMs. A Dependabot PR that remains queued during an outage is expected behavior, not a failed switch; it completes when the hosted pool recovers. The `blacksmith` value's branches carry no such exclusion, because Blacksmith runners are ephemeral (see the [blacksmith failover leg note](2026-09-09-blacksmith-failover-leg.md)).
|
||||
|
||||
|
||||
@@ -32,7 +32,7 @@ Status: implemented
|
||||
|
||||
1. 仓库 **Settings → Secrets and variables → Actions → Variables → New repository variable**:名称 `DSH_CI_FAILOVER_LINUX`(Linux 池故障)或 `DSH_CI_FAILOVER_WINDOWS`(Windows 池故障),值 `selfhosted`。
|
||||
2. 重新触发必需作业,使其重新解析运行器池。已经为托管标签**排队**的作业不会重定向,也无法原地 re-run,因此对于本手册所述的无限排队故障,应取消卡住的运行并 re-run all jobs,或推送一个新提交;“Re-run failed jobs”只有在作业真正失败(而非仍在排队)时才有用。
|
||||
3. 切换到此完成。在 `selfhosted` 的 Linux 故障切换取值下,工作流还会把 `DSH_SNAPSHOT_MAX_CONCURRENCY` 降为 12,以限制共享虚拟机上的争抢,并跳过托管路径的 pnpm 缓存恢复,因为虚拟机的持久 store 会直接提供热安装。覆盖率在两个 Linux 池上都使用 4 个单 worker 插桩分区与 2 个豁免 worker。Windows 开关没有并发或缓存分支;它只重定向原生 Windows 作业的运行器池。
|
||||
3. 切换到此完成。在 `selfhosted` 的 Linux 故障切换取值下,工作流还会把 `DSH_SNAPSHOT_MAX_CONCURRENCY` 降为 12,以限制共享虚拟机上的争抢,并跳过托管路径的 pnpm 缓存恢复,因为虚拟机的持久 store 会直接提供热安装。自托管覆盖率使用 4 个单 worker 插桩分区与 2 个豁免 worker;独占的托管运行器根据可用 CPU 计算默认 worker 数。Windows 开关选择原生 Windows 运行器池,并在共享的自托管运行器上保留显式 worker 限制。
|
||||
|
||||
**Dependabot 例外。**两个开关的 `selfhosted` 腿都刻意排除 `dependabot[bot]`:自托管故障切换期间,Dependabot 拉取请求继续在托管池排队,而不是把依赖项提供的代码放到持久化虚拟机上执行。故障期间 Dependabot PR 持续排队是预期行为而非切换失败;托管池恢复后它会自行完成。`blacksmith` 取值下的分支不带此类排除,因为 Blacksmith 运行器是临时的(见 [blacksmith 故障切换支路笔记](2026-09-09-blacksmith-failover-leg.zh.md))。
|
||||
|
||||
|
||||
+18
-16
@@ -49,7 +49,8 @@ jobs:
|
||||
|| 'dsh-ubuntu-24-04-16core' }}
|
||||
name: node 24 / static
|
||||
env:
|
||||
DSH_GATE_CONCURRENCY: '8'
|
||||
# Shared hosts expose host CPUs, not an isolated allocation.
|
||||
DSH_GATE_CONCURRENCY: ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '8' || '' }}
|
||||
# Stop the aggregate at the first blocking gate failure so a red run
|
||||
# does not keep burning enterprise runner time on the remaining gates.
|
||||
DSH_GATE_FAIL_FAST: '1'
|
||||
@@ -116,11 +117,10 @@ jobs:
|
||||
|| 'dsh-ubuntu-24-04-16core' }}
|
||||
name: node 24 / coverage
|
||||
env:
|
||||
# Partitioning replaces the instrumented share; this budget gives the
|
||||
# exempt-heavy gate two workers on both hosted and failover runners.
|
||||
DSH_COVERAGE_MAX_WORKERS: '6'
|
||||
DSH_COVERAGE_PARTITIONS: '4'
|
||||
DSH_GATE_CONCURRENCY: '3'
|
||||
# Shared hosts expose host CPUs, not an isolated allocation.
|
||||
DSH_COVERAGE_MAX_WORKERS: ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '6' || '' }}
|
||||
DSH_COVERAGE_PARTITIONS: ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '4' || '' }}
|
||||
DSH_GATE_CONCURRENCY: ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '3' || '' }}
|
||||
# Managed-scope teardown cases run past the default per-test budget when
|
||||
# this lane's partitions, workers, and sibling gates share one host: the
|
||||
# disposal cases in subprocess-local and bash-sandbox exceeded 5000ms on
|
||||
@@ -246,17 +246,17 @@ jobs:
|
||||
|| 'dsh-ubuntu-24-04-16core' }}
|
||||
name: node 24 / snapshots and artifacts
|
||||
env:
|
||||
DSH_GATE_CONCURRENCY: '10'
|
||||
# Shared hosts expose host CPUs, not an isolated allocation.
|
||||
DSH_GATE_CONCURRENCY: ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '10' || '' }}
|
||||
DSH_OXLINT_THREADS: ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '8' || '' }}
|
||||
DSH_PUBLINT_CONCURRENCY: ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '8' || '' }}
|
||||
DSH_WEB_SNAPSHOT_WORKERS: ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '6' || '' }}
|
||||
DSH_SNAPSHOT_MAX_CONCURRENCY: ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '12' || '' }}
|
||||
DSH_NODE_COMPAT_SKIP_TYPECHECK: '1'
|
||||
DSH_OXLINT_THREADS: '8'
|
||||
DSH_PUBLINT_CONCURRENCY: '8'
|
||||
DSH_WEB_SNAPSHOT_WORKERS: '6'
|
||||
# A failing gate aborts its running siblings: a failing build stops the
|
||||
# independent Node compatibility smoke, and a failing reader (e.g.
|
||||
# publint) stops the remaining artifact consumers.
|
||||
DSH_GATE_FAIL_FAST: '1'
|
||||
# Failover halves snapshot concurrency for the shared 64-core VM.
|
||||
DSH_SNAPSHOT_MAX_CONCURRENCY: ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '12' || '32' }}
|
||||
steps:
|
||||
# Runner cleanup owns tool caches and fixtures, including cancelled runs.
|
||||
- name: Use runner-owned temporary storage
|
||||
@@ -555,10 +555,11 @@ jobs:
|
||||
name: windows node 24 / coverage
|
||||
timeout-minutes: 120
|
||||
env:
|
||||
DSH_COVERAGE_MAX_WORKERS: '6'
|
||||
DSH_COVERAGE_PARTITIONS: '4'
|
||||
# Shared hosts expose host CPUs, not an isolated allocation.
|
||||
DSH_COVERAGE_MAX_WORKERS: ${{ vars.DSH_CI_FAILOVER_WINDOWS == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '6' || '' }}
|
||||
DSH_COVERAGE_PARTITIONS: ${{ vars.DSH_CI_FAILOVER_WINDOWS == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '4' || '' }}
|
||||
DSH_GATE_CONCURRENCY: ${{ vars.DSH_CI_FAILOVER_WINDOWS == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '3' || '' }}
|
||||
DSH_COVERAGE_TEST_TIMEOUT_MS: '90000'
|
||||
DSH_GATE_CONCURRENCY: '3'
|
||||
# A gate failure aborts the sibling gate instead of waiting out its
|
||||
# multi-minute instrumented run.
|
||||
DSH_GATE_FAIL_FAST: '1'
|
||||
@@ -682,7 +683,8 @@ jobs:
|
||||
name: windows node 24 / observational
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
DSH_PUBLINT_CONCURRENCY: '8'
|
||||
# Shared hosts expose host CPUs, not an isolated allocation.
|
||||
DSH_PUBLINT_CONCURRENCY: ${{ vars.DSH_CI_FAILOVER_WINDOWS == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '8' || '' }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
|
||||
@@ -232,9 +232,8 @@ describe('CI workflow', () => {
|
||||
expect(install!.run).not.toContain('$cloneFlag')
|
||||
}
|
||||
|
||||
// windows-coverage uses the lower 4-partition profile.
|
||||
expect(windowsCoverage.name).toBe('windows node 24 / coverage')
|
||||
expect(windowsCoverage.env).toMatchObject({ DSH_COVERAGE_PARTITIONS: '4' })
|
||||
expect(windowsCoverage.env).toMatchObject({ DSH_COVERAGE_PARTITIONS: "${{ vars.DSH_CI_FAILOVER_WINDOWS == 'selfhosted' && github.event.pull_request.user.login != 'dependabot[bot]' && '4' || '' }}" })
|
||||
const coverageSteps = windowsCoverage.steps as unknown[]
|
||||
const coverageCommands = coverageSteps.filter((step): step is Record<string, unknown> & { run: string } => (
|
||||
isRecord(step) && typeof step.run === 'string'
|
||||
@@ -386,8 +385,7 @@ describe('CI workflow', () => {
|
||||
// The observational lane stays complete: it is continue-on-error by design
|
||||
// and exists to collect as much Windows-native evidence per run as
|
||||
// possible, so the first failure must not truncate the rest.
|
||||
expect(windowsObservational.env).toBeDefined()
|
||||
expect(windowsObservational.env).not.toMatchObject({ DSH_GATE_FAIL_FAST: '1' })
|
||||
expect(windowsObservational.env ?? {}).not.toMatchObject({ DSH_GATE_FAIL_FAST: '1' })
|
||||
})
|
||||
|
||||
it('gates standalone keyless blacksmith jobs and benchmark tiers on the failover variables', () => {
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import { readFileSync } from 'node:fs'
|
||||
import { describe, expect, it, vi, type MockInstance } from 'vitest'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi, type MockInstance } from 'vitest'
|
||||
import {
|
||||
cliGateOptions,
|
||||
ciWorkerEnvironment,
|
||||
collectDescendants,
|
||||
defaultConcurrency,
|
||||
formatGateResultReason,
|
||||
@@ -14,6 +15,10 @@ import {
|
||||
type GateResult,
|
||||
} from './run-gates.ts'
|
||||
|
||||
// Graph fixtures select their own browser pool instead of inheriting the CI host's pool.
|
||||
beforeEach(() => vi.stubEnv('DSH_WEB_SNAPSHOT_WORKERS', undefined))
|
||||
afterEach(() => vi.unstubAllEnvs())
|
||||
|
||||
/**
|
||||
* Capture output a gate streams through runGate's streamOutput path.
|
||||
* @returns the accumulated chunks and the stdout spy to restore in finally.
|
||||
@@ -135,6 +140,59 @@ function withEnv<T>(name: string, value: string | undefined, action: () => T): T
|
||||
}
|
||||
}
|
||||
|
||||
describe('CI worker allocation', () => {
|
||||
it.each([1, 2, 4, 8, 16, 64])('shares a %i CPU coverage budget without multiplying pools', (cpus) => {
|
||||
const env = ciWorkerEnvironment('ci-coverage', {}, cpus)
|
||||
const exempt = Math.max(1, Math.floor(cpus / 3))
|
||||
const instrumented = Number(env.DSH_COVERAGE_PARTITIONS ?? 1)
|
||||
expect(Number(env.DSH_COVERAGE_MAX_WORKERS)).toBe(cpus)
|
||||
expect(instrumented + exempt).toBe(Math.max(2, cpus))
|
||||
expect(defaultConcurrency('ci-coverage', 3, cpus).workers).toBe(Math.min(3, cpus))
|
||||
if (cpus <= 2) expect(env.DSH_COVERAGE_PARTITIONS).toBeUndefined()
|
||||
})
|
||||
|
||||
it('bounds overlapping readers and lets the isolated browser pool use the runner', () => {
|
||||
const env = ciWorkerEnvironment('ci-consumers', {}, 16)
|
||||
expect(env).toMatchObject({
|
||||
DSH_OXLINT_THREADS: '8',
|
||||
DSH_PUBLINT_CONCURRENCY: '8',
|
||||
DSH_SNAPSHOT_MAX_WORKERS: '1',
|
||||
DSH_SNAPSHOT_MAX_CONCURRENCY: '8',
|
||||
DSH_WEB_SNAPSHOT_WORKERS: '16',
|
||||
})
|
||||
})
|
||||
|
||||
it('preserves serial reference overrides on large hosts', () => {
|
||||
const inherited = {
|
||||
DSH_GATE_CONCURRENCY: '1',
|
||||
DSH_COVERAGE_MAX_WORKERS: '1',
|
||||
DSH_OXLINT_THREADS: '1',
|
||||
DSH_PUBLINT_CONCURRENCY: '1',
|
||||
DSH_SNAPSHOT_MAX_CONCURRENCY: '1',
|
||||
DSH_WEB_SNAPSHOT_WORKERS: '1',
|
||||
}
|
||||
const additions = ciWorkerEnvironment('ci-primary', inherited, 64)
|
||||
expect(additions).toEqual({ DSH_SNAPSHOT_MAX_WORKERS: '1' })
|
||||
expect(inherited.DSH_COVERAGE_MAX_WORKERS).toBe('1')
|
||||
})
|
||||
|
||||
it('honors an explicit coverage budget and partition override', () => {
|
||||
expect(ciWorkerEnvironment('ci-coverage', { DSH_COVERAGE_MAX_WORKERS: '6' }, 16))
|
||||
.toHaveProperty('DSH_COVERAGE_PARTITIONS', '4')
|
||||
expect(ciWorkerEnvironment('ci-coverage', { DSH_COVERAGE_PARTITIONS: '3' }, 16))
|
||||
.not.toHaveProperty('DSH_COVERAGE_PARTITIONS')
|
||||
})
|
||||
|
||||
it.each(['0', '-1', 'NaN', '2.5'])('rejects invalid worker budget %s', (raw) => {
|
||||
expect(() => ciWorkerEnvironment('ci-coverage', { DSH_COVERAGE_MAX_WORKERS: raw }, 16))
|
||||
.toThrow('DSH_COVERAGE_MAX_WORKERS must be a positive integer')
|
||||
})
|
||||
|
||||
it('keeps local documentation defaults unchanged', () => {
|
||||
expect(ciWorkerEnvironment('doc-sync', {}, 64)).toEqual({})
|
||||
})
|
||||
})
|
||||
|
||||
describe('gate graph validation', () => {
|
||||
it.each([
|
||||
'ci-primary',
|
||||
@@ -601,8 +659,8 @@ describe('Node 24 lane ownership', () => {
|
||||
const subject = withPnpmEntrypoint(() => gatesForMode('ci-consumers'))
|
||||
|
||||
expect(defaultConcurrency('ci-consumers', subject.length, 4)).toEqual({
|
||||
workers: 11,
|
||||
source: 'ci-consumers gate count',
|
||||
workers: 4,
|
||||
source: '4 available CPU(s)',
|
||||
})
|
||||
expect(subject.map(item => item.id)).toEqual([
|
||||
'build',
|
||||
|
||||
+43
-7
@@ -102,6 +102,9 @@ if (import.meta.main) {
|
||||
|
||||
async function main(args: string[]): Promise<number> {
|
||||
const mode = parseMode(args[0])
|
||||
const workerEnv = ciWorkerEnvironment(mode, process.env)
|
||||
Object.assign(process.env, workerEnv)
|
||||
if (Object.keys(workerEnv).length > 0) console.log(`run-gates: worker settings ${JSON.stringify(workerEnv)}`)
|
||||
const gates = gatesForMode(mode)
|
||||
const concurrencyDefault = defaultConcurrency(mode, gates.length)
|
||||
const concurrencyOverride = process.env.DSH_GATE_CONCURRENCY
|
||||
@@ -170,7 +173,6 @@ export function defaultConcurrency(
|
||||
total: number,
|
||||
available = availableParallelism(),
|
||||
): ConcurrencyDefault {
|
||||
if (selectedMode === 'ci-consumers') return { workers: total, source: 'ci-consumers gate count' }
|
||||
// Local modes cap workers: several doc gates each build a full ts.Program,
|
||||
// so an uncapped default on a large host trades wall clock for memory blowups.
|
||||
const localCap = selectedMode === 'check-all'
|
||||
@@ -186,6 +188,41 @@ export function defaultConcurrency(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fill CI worker defaults from the CPU allocation without replacing explicit overrides.
|
||||
* Coverage shares one budget; artifact readers share CPUs with sibling commands.
|
||||
* @param mode - aggregate selected by the caller.
|
||||
* @param env - inherited worker overrides, including serial reference settings.
|
||||
* @param available - CPUs available to this runner process.
|
||||
* @returns environment additions for this aggregate and its children.
|
||||
*/
|
||||
export function ciWorkerEnvironment(
|
||||
mode: Mode,
|
||||
env: NodeJS.ProcessEnv,
|
||||
available = availableParallelism(),
|
||||
): Record<string, string> {
|
||||
if (!mode.startsWith('ci-')) return {}
|
||||
const additions: Record<string, string> = {}
|
||||
const setDefault = (name: string, value: number): void => {
|
||||
if (env[name] === undefined || env[name] === '') additions[name] = String(value)
|
||||
}
|
||||
const shared = Math.max(1, Math.floor(available / 2))
|
||||
setDefault('DSH_OXLINT_THREADS', shared)
|
||||
setDefault('DSH_PUBLINT_CONCURRENCY', shared)
|
||||
setDefault('DSH_SNAPSHOT_MAX_WORKERS', 1)
|
||||
setDefault('DSH_SNAPSHOT_MAX_CONCURRENCY', shared)
|
||||
setDefault('DSH_WEB_SNAPSHOT_WORKERS', env.DSH_GATE_CONCURRENCY === '1' ? 1 : available)
|
||||
setDefault('DSH_COVERAGE_MAX_WORKERS', available)
|
||||
const coverageBudget = env.DSH_COVERAGE_MAX_WORKERS || String(available)
|
||||
const total = Number(coverageBudget)
|
||||
if (!Number.isSafeInteger(total) || total < 1) {
|
||||
throw new Error(`run-gates: DSH_COVERAGE_MAX_WORKERS must be a positive integer, got ${JSON.stringify(coverageBudget)}.`)
|
||||
}
|
||||
const instrumented = Math.max(1, total - Math.max(1, Math.floor(total / 3)))
|
||||
if (instrumented > 1) setDefault(COVERAGE_PARTITIONS_ENV, instrumented)
|
||||
return additions
|
||||
}
|
||||
|
||||
function concurrencyFromEnv(name: string, fallback: number): number {
|
||||
const raw = process.env[name]
|
||||
if (raw === undefined || raw === '') return fallback
|
||||
@@ -498,8 +535,8 @@ function webSnapshotGate(needs: string[], after?: string[]): Gate {
|
||||
const workerRaw = process.env.DSH_WEB_SNAPSHOT_WORKERS
|
||||
if (workerRaw !== undefined && workerRaw !== '') {
|
||||
const workers = Number.parseInt(workerRaw, 10)
|
||||
if (!Number.isSafeInteger(workers) || workers < 2 || String(workers) !== workerRaw) {
|
||||
throw new Error(`run-gates: DSH_WEB_SNAPSHOT_WORKERS must be an integer greater than 1, got ${JSON.stringify(workerRaw)}.`)
|
||||
if (!Number.isSafeInteger(workers) || workers < 1 || String(workers) !== workerRaw) {
|
||||
throw new Error(`run-gates: DSH_WEB_SNAPSHOT_WORKERS must be a positive integer, got ${JSON.stringify(workerRaw)}.`)
|
||||
}
|
||||
return pnpmScript('web-snapshot', 'test:web:ci', {
|
||||
label: 'web browser snapshot',
|
||||
@@ -593,10 +630,9 @@ function lintGate(options: { needs?: string[] } = {}): Gate {
|
||||
// under v8 instrumentation while contributing nothing the thresholds need
|
||||
// (membership rules in scripts/coverage-exempt.ts).
|
||||
//
|
||||
// DSH_COVERAGE_MAX_WORKERS is the ordinary lane's worker budget, so the two
|
||||
// parallel gates split it instead of each claiming it whole. When
|
||||
// DSH_COVERAGE_PARTITIONS is set, its single-worker processes replace the
|
||||
// instrumented share while this budget still sizes the exempt gate. The exempt
|
||||
// DSH_COVERAGE_MAX_WORKERS is shared between the two parallel gates. CI
|
||||
// defaults its partition count to the instrumented share; an explicit
|
||||
// DSH_COVERAGE_PARTITIONS overrides that share independently. The exempt
|
||||
// gate's wall clock is dominated by its longest single file, so it takes the
|
||||
// small share. A budget of 1 gives each gate 1 worker; lanes that need a strict
|
||||
// total of one (the serial reference jobs) also set DSH_GATE_CONCURRENCY=1,
|
||||
|
||||
@@ -7,8 +7,8 @@ const serialFiles = [
|
||||
]
|
||||
const workerRaw = process.env.DSH_WEB_SNAPSHOT_WORKERS
|
||||
const workers = Number.parseInt(workerRaw ?? '', 10)
|
||||
if (!Number.isSafeInteger(workers) || workers < 2 || String(workers) !== workerRaw) {
|
||||
throw new Error(`DSH_WEB_SNAPSHOT_WORKERS must be an integer greater than 1, got ${JSON.stringify(workerRaw)}.`)
|
||||
if (!Number.isSafeInteger(workers) || workers < 1 || String(workers) !== workerRaw) {
|
||||
throw new Error(`DSH_WEB_SNAPSHOT_WORKERS must be a positive integer, got ${JSON.stringify(workerRaw)}.`)
|
||||
}
|
||||
const invocation = pnpmInvocation(['exec', 'vitest', 'run', '--config', 'vitest.web.config.ts'])
|
||||
let serialStatus = 0
|
||||
|
||||
@@ -21,6 +21,10 @@ const snapshotMaxConcurrency = positiveIntFromEnv(
|
||||
Math.min(DEFAULT_SNAPSHOT_MAX_CONCURRENCY, availableParallelism()),
|
||||
)
|
||||
|
||||
const snapshotMaxWorkers = process.env.DSH_SNAPSHOT_MAX_WORKERS
|
||||
? positiveIntFromEnv('DSH_SNAPSHOT_MAX_WORKERS', availableParallelism())
|
||||
: undefined
|
||||
|
||||
// Replay is the keyless default: boot real subprocess paths from recorded model responses and diff
|
||||
// assembled requests, normalized protocol or transcript output, and persisted-log expected outputs.
|
||||
// `record` calls the real API and updates fixtures and expected outputs; `refresh` replays committed scripts
|
||||
@@ -54,14 +58,16 @@ export default defineConfig({
|
||||
// Replay never writes committed outputs and every scenario owns its
|
||||
// mutable runtime state (the subprocess suites use a unique temp dir and
|
||||
// fixture set per scenario), so replay runs the snapshot files in
|
||||
// parallel and bounds in-file concurrency with the environment knob
|
||||
// (value 1 restores fully serial replay on constrained machines). Record
|
||||
// parallel with separate file-worker and in-file concurrency budgets.
|
||||
// Without a file-worker override, in-file concurrency 1 is fully serial. Record
|
||||
// and refresh stay serial: record spends real API quota per scenario, and
|
||||
// refresh write-back harvests volatile values from fixtures already on
|
||||
// disk, so concurrent writers would corrupt expected outputs.
|
||||
testTimeout: 120_000,
|
||||
hookTimeout: 30_000,
|
||||
fileParallelism: (process.env.DSH_SNAPSHOT || 'replay') === 'replay' && snapshotMaxConcurrency > 1,
|
||||
fileParallelism: (process.env.DSH_SNAPSHOT || 'replay') === 'replay'
|
||||
&& (snapshotMaxWorkers ?? snapshotMaxConcurrency) > 1,
|
||||
maxConcurrency: snapshotMaxConcurrency,
|
||||
...snapshotMaxWorkers === undefined ? {} : { maxWorkers: snapshotMaxWorkers },
|
||||
},
|
||||
})
|
||||
Reference in new issue
Block a user