From 362ee7931d9e58d2673d8cc6cb967aa151450ae4 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 6 Sep 2026 22:44:54 +0800 Subject: [PATCH] fix(test): isolate recorded spill and sandbox fixture storage --- ...6-pr-ci-runner-temporary-storage.i18n.yaml | 4 +- ...26-09-06-pr-ci-runner-temporary-storage.md | 4 +- ...09-06-pr-ci-runner-temporary-storage.zh.md | 4 +- .../fs/fs-sandbox/tests/fs-sandbox.spec.ts | 19 +++--- .../session-snapshot/README.i18n.yaml | 4 +- .../test-support/session-snapshot/README.md | 2 +- .../session-snapshot/README.zh.md | 2 +- .../session-snapshot/src/harness.ts | 13 ++-- .../session-snapshot/tests/harness.spec.ts | 15 +++-- scripts/snapshot-shell-path.spec.ts | 42 ++++++++++++ scripts/snapshot-shell-path.ts | 30 +++++++++ scripts/snapshot-spill-locators.spec.ts | 64 +++++++++++++++++++ scripts/snapshot-spill-locators.ts | 59 +++++++++++++++++ scripts/snapshot-workspace-parent.spec.ts | 2 +- .../escalation-approved/cordis.snapshot.yml | 8 +++ .../fs-glob-sampling/cordis.snapshot.yml | 7 ++ snapshots/session/fs-glob-sampling/cordis.yml | 7 ++ snapshots/session/headless.snapshot.ts | 4 +- .../parallel-tool-calls/cordis.snapshot.yml | 7 ++ .../session/parallel-tool-calls/cordis.yml | 7 ++ .../session-query-spill/cordis.snapshot.yml | 15 +++++ .../session/session-query-spill/cordis.yml | 7 ++ .../cordis.snapshot.yml | 7 ++ .../session-reference-spill/cordis.yml | 7 ++ 24 files changed, 303 insertions(+), 37 deletions(-) create mode 100644 scripts/snapshot-shell-path.spec.ts create mode 100644 scripts/snapshot-shell-path.ts create mode 100644 scripts/snapshot-spill-locators.spec.ts create mode 100644 scripts/snapshot-spill-locators.ts diff --git a/.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.i18n.yaml b/.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.i18n.yaml index 4ba9843387..e43b1175b6 100644 --- a/.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.i18n.yaml +++ b/.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.md -2026-09-06-pr-ci-runner-temporary-storage.md: 7c0ee0e6d1b357790061ca5e61a2c871d8669bb1 -2026-09-06-pr-ci-runner-temporary-storage.zh.md: ff4bae48a2681d70e1b32901297ec4b6fae75e3b +2026-09-06-pr-ci-runner-temporary-storage.md: 93a631e3489f9cbf8b3aaee81cfdd7c170f6403b +2026-09-06-pr-ci-runner-temporary-storage.zh.md: 48759b76263005b29399921ecd65878fcd6022b1 diff --git a/.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.md b/.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.md index 7c0ee0e6d1..93a631e348 100644 --- a/.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.md +++ b/.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.md @@ -22,7 +22,7 @@ The [ACP diagnostic scenario](../../../../snapshots/session/subagent-acp-diagnos ## Workspace-grant fixture placement -The headless `session-sandbox-root` fixture declares `workspace.parent: outside-temp`, not a home-filesystem dependency. Its allocator uses a sibling of the canonical platform temp root where the parent is writable and avoids system temporary grants, otherwise home, and rejects a cwd already covered by automatic temporary write grants. On the failover runner this keeps the test on the data volume without making its write succeed through a temporary-directory exemption. Atomic workspace allocation, final cleanup, recorded Session bytes, and the independent expected file remain unchanged. +The headless `session-sandbox-root` fixture declares `workspace.parent: outside-temp`, not a home-filesystem dependency. Its allocator uses a sibling of the canonical platform temp root where the parent is writable and avoids system temporary grants, otherwise home, and rejects a cwd already covered by automatic temporary write grants. On the failover runner this keeps the test on the data volume without making its write succeed through a temporary-directory exemption. The filesystem-sandbox containment tests use the same allocator for their workspace and denied sibling; they register cleanup immediately after successful acquisition. Atomic workspace allocation, recorded Session bytes, and the independent expected file remain unchanged. ## Alternatives considered @@ -36,6 +36,6 @@ The headless `session-sandbox-root` fixture declares `workspace.parent: outside- Output honoring these temporary-directory and cache settings follows the job lifetime instead of accumulating in unmanaged host storage. This does not reclaim existing shared temporary files, guarantee filesystem capacity, or clean files the runner account cannot remove. Operators still own historical residue, disk provisioning, and jobs outside this PR workflow. -Linux bwrap and Landlock workspace-write profiles grant literal `/tmp` and the workspace, not an inherited `TMPDIR` outside it; confined fixtures must place temporary writes in those granted paths. The [snapshot spill helper](../../../../packages/test-support/session-snapshot/src/harness.ts) also uses literal `/tmp/dsh-acp-snap-*` on POSIX for stable path lengths, so that output remains outside runner cleanup. This workflow change neither widens sandbox grants nor rewrites fixed-path fixtures. +Linux bwrap and Landlock workspace-write profiles grant literal `/tmp` and the workspace, not an inherited `TMPDIR` outside it; confined fixtures must place temporary writes in those granted paths. The [snapshot spill helper](../../../../packages/test-support/session-snapshot/src/harness.ts) separates fixed-length logical locators from atomically allocated live storage. A fixture-only adapter delegates saves to the real local spill provider and resolves only locators saved by that run to their live files. Recorded preview lengths, omission counts, and retrieval assertions remain unchanged; no files are allocated at the logical `/tmp/dsh-acp-snap-*` prefix. This change does not widen product sandbox grants. The parsed-workflow cases in [ci-workflow.spec.ts](../../../../scripts/ci-workflow.spec.ts) require the assignment on all three workers and reject step-level overrides. They fail against the unmodified workflow. Independent-process smoke checks and repeated PR runs validate the actual tooling; the YAML assertions alone do not prove host capacity. diff --git a/.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.zh.md b/.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.zh.md index ff4bae48a2..48759b7626 100644 --- a/.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.zh.md +++ b/.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.zh.md @@ -22,7 +22,7 @@ Linux 故障切换池在同一台虚拟机上运行多个 runner 实例。PR 覆 ## Workspace 授权夹具的位置 -Headless 的 `session-sandbox-root` 夹具声明 `workspace.parent: outside-temp`，而不是依赖 home 所在文件系统。分配器在父目录可写且避开系统临时授权时选择规范化平台临时根目录的同级目录，否则使用 home，并拒绝已被自动临时写授权覆盖的 cwd。在故障切换 runner 上，这让测试留在数据卷中，同时不会让写入借助临时目录豁免而成功。原子 workspace 分配、最终清理、录制的 Session 字节以及独立预期文件保持不变。 +Headless 的 `session-sandbox-root` 夹具声明 `workspace.parent: outside-temp`，而不是依赖 home 所在文件系统。分配器在父目录可写且避开系统临时授权时选择规范化平台临时根目录的同级目录，否则使用 home，并拒绝已被自动临时写授权覆盖的 cwd。在故障切换 runner 上，这让测试留在数据卷中，同时不会让写入借助临时目录豁免而成功。文件系统沙箱的包含关系测试使用同一分配器创建 workspace 及被拒绝的同级目录，并在成功获取目录后立即注册清理。原子 workspace 分配、录制的 Session 字节以及独立预期文件保持不变。 ## 考虑过的替代方案 @@ -36,6 +36,6 @@ Headless 的 `session-sandbox-root` 夹具声明 `workspace.parent: outside-temp 遵循临时目录和缓存配置的输出随作业生命周期清理，而不累积于无人管理的主机存储。本方案不回收既有共享临时文件、不保证文件系统容量，也不清理 runner 账号无权删除的文件。历史残留、磁盘配置以及本 PR 工作流以外的作业仍由运维人员负责。 -Linux bwrap 和 Landlock 的 workspace-write profile 允许写入字面路径 `/tmp` 和 workspace，而不允许写入其外部继承的 `TMPDIR`；受限测试夹具必须将临时写入放在这些已授权路径中。[快照 spill helper](../../../../packages/test-support/session-snapshot/src/harness.ts) 在 POSIX 上也使用字面路径 `/tmp/dsh-acp-snap-*` 以保持路径长度稳定，因此这些输出仍不受 runner 清理管理。本工作流变更既不扩大沙箱授权，也不重写固定路径夹具。 +Linux bwrap 和 Landlock 的 workspace-write profile 允许写入字面路径 `/tmp` 和 workspace，而不允许写入其外部继承的 `TMPDIR`；受限测试夹具必须将临时写入放在这些已授权路径中。[快照 spill helper](../../../../packages/test-support/session-snapshot/src/harness.ts) 将固定长度的逻辑定位符与原子分配的实际存储分开。仅用于夹具的适配器将保存操作委托给真实的本地 spill provider，并仅将本次运行已保存的定位符解析到实际文件。录制的预览长度、省略计数及检索断言保持不变；逻辑 `/tmp/dsh-acp-snap-*` 前缀下不分配文件。本变更不扩大产品沙箱授权。 [ci-workflow.spec.ts](../../../../scripts/ci-workflow.spec.ts) 的 YAML 解析用例要求三个 worker 都包含该赋值，并拒绝步骤级别的覆盖。它们在未修改的工作流上失败。独立进程 smoke 检查和重复 PR 运行验证实际工具链；仅有 YAML 断言不能证明主机容量充足。 diff --git a/packages/fs/fs-sandbox/tests/fs-sandbox.spec.ts b/packages/fs/fs-sandbox/tests/fs-sandbox.spec.ts index 95ae891808..e445c47553 100644 --- a/packages/fs/fs-sandbox/tests/fs-sandbox.spec.ts +++ b/packages/fs/fs-sandbox/tests/fs-sandbox.spec.ts @@ -11,7 +11,7 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest' import { mkdir, mkdtemp, readFile, rm, symlink, writeFile } from 'node:fs/promises' import { existsSync } from 'node:fs' -import { homedir, tmpdir } from 'node:os' +import { tmpdir } from 'node:os' import { join, parse } from 'node:path' import { Context } from '@deepseek-ai/cordis' import { FsError, FsTargetKey } from '@deepseek-ai/dsh-fs' @@ -20,6 +20,7 @@ import SandboxPolicyService from '@deepseek-ai/dsh-sandbox-policy' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import type { SandboxMode } from '@deepseek-ai/dsh-sandbox' import { SandboxedFileSystem } from '@deepseek-ai/dsh-fs-sandbox' +import { assertWorkspaceOutsideTemp, outsideTempWorkspaceParent } from '../../../../scripts/snapshot-workspace-parent.ts' let base: string let workspace: string @@ -36,13 +37,12 @@ async function boot(mode: SandboxMode): Promise { fs = ctx.fs as SandboxedFileSystem } -beforeEach(async () => { - // Base under HOME, deliberately NOT tmpdir: `workspace-write` grants /tmp and - // os.tmpdir() (parity with the bash runner), so an "outside" dir under tmpdir - // would be legitimately writable. Sibling dirs under HOME are outside every - // grant, so containment failures are real denials. (The bwrap e2e roots its - // workspaces under HOME for the same reason.) - base = await mkdtemp(join(homedir(), '.dsh-fssbx-')) +beforeEach(async ({ onTestFinished }) => { + // Both siblings must be outside automatic temp grants for containment denials to be meaningful. + const directory = await mkdtemp(join(outsideTempWorkspaceParent(), '.dsh-fssbx-')) + onTestFinished(async () => { await rm(directory, { recursive: true, force: true }) }) + base = directory + assertWorkspaceOutsideTemp(base) workspace = join(base, 'ws') outside = join(base, 'out') await mkdir(workspace) @@ -50,7 +50,6 @@ beforeEach(async () => { }) afterEach(async () => { await fiber?.dispose() - await rm(base, { recursive: true, force: true }) }) /** Resolve a path through the backend and return its target. */ @@ -183,7 +182,7 @@ describe('workspace-write with the filesystem root as the workspace (a root endi const rootFiber = await rootCtx.plugin(SandboxedFileSystem, { cwd: workspace }) const rootFs = rootCtx.fs as SandboxedFileSystem try { - const path = join(base, 'anywhere.txt') // under HOME, outside temp — allowed only via the filesystem root + const path = join(base, 'anywhere.txt') // outside temp — allowed only via the filesystem root await rootFs.writeText(await rootFs.resolve(path), 'anywhere') expect(await readFile(path, 'utf8')).toBe('anywhere') } finally { diff --git a/packages/test-support/session-snapshot/README.i18n.yaml b/packages/test-support/session-snapshot/README.i18n.yaml index c63adee2f5..46d058dadf 100644 --- a/packages/test-support/session-snapshot/README.i18n.yaml +++ b/packages/test-support/session-snapshot/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/test-support/session-snapshot/README.md -README.md: 8d1e4ad7bc7611c4d22a8109dcc0b4993f49bb01 -README.zh.md: 287de8c7f4390e0762d202e5650a2c5e9ba49c1b +README.md: f1126920969cead7844712ad3b2c7213cce19ed9 +README.zh.md: 65e04f252a13c22351b450da2df44f244d484811 diff --git a/packages/test-support/session-snapshot/README.md b/packages/test-support/session-snapshot/README.md index 8d1e4ad7bc..f112692096 100644 --- a/packages/test-support/session-snapshot/README.md +++ b/packages/test-support/session-snapshot/README.md @@ -72,7 +72,7 @@ Each recorded-session directory carries a closed `snapshot.yml` manifest plus ca `normalizeSessionSnapshot` retains the complete Session header and event payloads but omits top-level `seq`/`time` envelopes from committed fixtures after normalizing paths and scrubbing request headers; it also normalizes embedded stream clocks and historical packed-row `seq0`/`time0` envelopes. Replay synthesizes the top-level envelopes in memory, while runtime persistence continues to write complete logs. Multi-session comparison restores every selected persisted or projected fixture through the current build-static Session format catalog before identity redaction and normalization, so retained v0/v1 replay input and fresh `session.v2.jsonl` writer output compare as one v2 logical Session without rewriting or renaming historical files. Expected and harvested logs use the same strict restoration path; source filenames cannot alter format validation. Versionless protocol-adapter unit fixtures remain outside the released Session format corpus. Current v2 fixtures use one row per event; retained v0/v1 fixtures may use canonical packed rows. The [temporary repository migrator](../../../scripts/migrate-packed-session-fixtures.ts) (`pnpm run migrate:packed-session-fixtures`) rewrites older historical layouts, and its [removal proposal](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md) owns its deletion. -Known snapshot spill paths normalize to stable locator tokens, including paths quoted inside JSON omission notices with JSON-escaped Windows separators. Refresh extraction preserves the matched serialized path spelling for literal replacement. Normalization changes only the locator: saved byte lengths and omission counts remain comparison evidence. +Spill scenarios save through the real local provider under a private temporary root. Their fixture adapter exposes fixed-length logical locators and maps only this run’s saved locators back to real files for retrieval, preserving preview budgets without writing into a shared logical path. Known snapshot spill paths normalize to stable locator tokens, including paths quoted inside JSON omission notices with JSON-escaped Windows separators. Refresh extraction preserves the matched serialized path spelling for literal replacement. Normalization changes only the locator: saved byte lengths and omission counts remain comparison evidence. ### Record, replay, and refresh diff --git a/packages/test-support/session-snapshot/README.zh.md b/packages/test-support/session-snapshot/README.zh.md index 287de8c7f4..65e04f252a 100644 --- a/packages/test-support/session-snapshot/README.zh.md +++ b/packages/test-support/session-snapshot/README.zh.md @@ -72,7 +72,7 @@ defineAcpSnapshotSuite({ `normalizeSessionSnapshot` 在规范化路径并清理 request header 后，会保留完整 Session header 与事件 payload，但从已提交 fixture 中省略顶层 `seq`/`time` envelope；它还会规范化嵌入式 stream clock 与历史 packed-row 的 `seq0`/`time0` envelope。Replay 只在内存中合成顶层 envelope，而运行时持久化仍写入完整日志。多 Session 比较会先通过当前构建期静态 Session 格式目录恢复每个选定的持久化或投影 fixture，再进行身份脱敏与规范化，因此保留的 v0/v1 replay 输入与新生成的 `session.v2.jsonl` writer 输出会作为同一个 v2 logical Session 比较，且不会重写或重命名历史文件。预期日志与收集日志使用同一条严格恢复路径；来源文件名不能改变格式校验。无版本的协议适配器单元测试 fixture 不属于已发布 Session 格式语料。当前 v2 fixture 每个事件占一行；保留的 v0/v1 fixture 可以使用规范 packed row。[临时仓库迁移器](../../../scripts/migrate-packed-session-fixtures.ts)（`pnpm run migrate:packed-session-fixtures`）会改写更旧的历史布局，由其[移除提案](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.zh.md)负责删除该迁移器。 -已知的快照 spill 路径会规范化为稳定的定位信息 token，包括 JSON 省略通知中带引号、使用 JSON 转义 Windows 分隔符的路径。刷新提取会保留匹配路径的序列化写法，以便进行字面替换。规范化只改变定位信息：保存字节数与省略计数仍作为比较证据。 +Spill 场景通过真实本地 provider 保存到私有临时根目录。夹具适配器提供固定长度的逻辑定位符，并仅将本次运行已保存的定位符映射回实际文件以供检索，在不写入共享逻辑路径的情况下保留预览预算。已知的快照 spill 路径会规范化为稳定的定位信息 token，包括 JSON 省略通知中带引号、使用 JSON 转义 Windows 分隔符的路径。刷新提取会保留匹配路径的序列化写法，以便进行字面替换。规范化只改变定位信息：保存字节数与省略计数仍作为比较证据。 ### 录制、回放与刷新 diff --git a/packages/test-support/session-snapshot/src/harness.ts b/packages/test-support/session-snapshot/src/harness.ts index 4490e08df9..cd54f516ae 100644 --- a/packages/test-support/session-snapshot/src/harness.ts +++ b/packages/test-support/session-snapshot/src/harness.ts @@ -210,7 +210,7 @@ export interface RunOptions { } /** - * Derive one stable, fixed-length spill root owned by this scenario. + * Derive the stable, fixed-length logical spill prefix; never allocate files here. * Windows uses a two-character-shorter root because drive resolution adds its drive prefix. * @param fixtureFile - The scenario fixture whose parent directory provides the stable identity. * @param platform - the host platform, injectable for unit coverage. @@ -239,17 +239,14 @@ export async function runScenario(input: InputScript, opts: RunOptions): Promise const cwd = await mkdtemp(join(opts.workspaceParent ?? tmpdir(), 'acp-snap-cwd-')) const cwdAliases = [...new Set([realpathSync(cwd), realpathSync.native(cwd)])] const sessionsRoot = await mkdtemp(join(tmpdir(), 'acp-snap-sessions-')) - // Fixed path length: spill-policy budgets the preview against the REAL path - // before stdout normalization, so tmpdir() length differences churn expected outputs. - // Scenario ownership also matters: replay runs concurrently, and one teardown - // must never delete another scenario's in-flight full-output recovery file. - const spillRoot = snapshotSpillRoot(opts.fixtureFile) + let spillRoot: string | undefined // Everything past the temp-dir creation is followed by failure-safe cleanup, // so a failure in workspace seeding, spawn, or any step never leaks resources. let launched: LaunchedAcpTestAgent | undefined let sessionId: string | undefined let sessionLogs: HarvestedLog[] = [] const outcome = await (async (): Promise => { + spillRoot = await mkdtemp(join(tmpdir(), 'acp-snap-spill-')) // Seed the workspace if the scenario ships one (a file the agent reads/edits). // Copied into the generated cwd so the agent's bash tools see it; the expected outputs // normalize the cwd, so the seeded paths stay stable across runs. @@ -272,6 +269,7 @@ export async function runScenario(input: InputScript, opts: RunOptions): Promise DSH_SNAPSHOT_FILE: opts.fixtureFile, DSH_SNAPSHOT_SESSIONS_ROOT: sessionsRoot, DSH_SNAPSHOT_SPILL_ROOT: spillRoot, + DSH_SNAPSHOT_SPILL_LOCATOR_ROOT: snapshotSpillRoot(opts.fixtureFile), DSH_HOME: join(cwd, '.dsh'), DSH_AGENTS_HOME: join(cwd, '.agents'), ...opts.overrideFile !== undefined ? { DSH_SNAPSHOT_OVERRIDE: opts.overrideFile } : {}, @@ -383,7 +381,8 @@ export async function runScenario(input: InputScript, opts: RunOptions): Promise await cleanup(() => launched?.close('SIGKILL') ?? Promise.resolve()) await cleanup(() => rm(cwd, { recursive: true, force: true })) await cleanup(() => rm(sessionsRoot, { recursive: true, force: true })) - await cleanup(() => rm(spillRoot, { recursive: true, force: true })) + const allocatedSpillRoot = spillRoot + if (allocatedSpillRoot !== undefined) await cleanup(() => rm(allocatedSpillRoot, { recursive: true, force: true })) const cleanupFailures = cleanupResults .filter((result): result is PromiseRejectedResult => result.status === 'rejected') diff --git a/packages/test-support/session-snapshot/tests/harness.spec.ts b/packages/test-support/session-snapshot/tests/harness.spec.ts index 508fd2b635..b3707784eb 100644 --- a/packages/test-support/session-snapshot/tests/harness.spec.ts +++ b/packages/test-support/session-snapshot/tests/harness.spec.ts @@ -569,9 +569,9 @@ describe('runScenario', () => { expect(env.childFiles).toBe(childFiles.join(delimiter)) }) - it('gives concurrent scenarios distinct equal-length spill roots', { timeout: 20_000 }, async () => { - const [first, second] = await Promise.all([scenario({ echoEnv: true }), scenario({ echoEnv: true })]) - const results = await Promise.all([first, second].map(({ fixtureFile }) => runScenario( + it('gives concurrent runs of the same scenario private temporary spill roots', { timeout: 20_000 }, async () => { + const fixture = await scenario({ echoEnv: true }) + const results = await Promise.all([fixture, fixture].map(({ fixtureFile }) => runScenario( { steps: [...boot, { op: 'prompt', text: 'env?' }] }, { agent: AGENT, mode: 'replay', fixtureFile }, ))) @@ -579,10 +579,11 @@ describe('runScenario', () => { expect(roots.every(root => typeof root === 'string')).toBe(true) expect(new Set(roots).size).toBe(2) expect((roots[0] as string).length).toBe((roots[1] as string).length) - expect(roots).toEqual([ - snapshotSpillRoot(first.fixtureFile), - snapshotSpillRoot(second.fixtureFile), - ]) + for (const root of roots as string[]) { + expect(relative(tmpdir(), root)).toMatch(/^acp-snap-spill-[^/\\]+$/) + expect(root).not.toBe(snapshotSpillRoot(fixture.fixtureFile)) + await expect(readdir(root)).rejects.toMatchObject({ code: 'ENOENT' }) + } }) it('seeds the workspace dir into the temp cwd before the run', { timeout: 20_000 }, async () => { diff --git a/scripts/snapshot-shell-path.spec.ts b/scripts/snapshot-shell-path.spec.ts new file mode 100644 index 0000000000..7f32e9cbe8 --- /dev/null +++ b/scripts/snapshot-shell-path.spec.ts @@ -0,0 +1,42 @@ +import { Context } from '@deepseek-ai/cordis' +import LocalSubprocessRuntime from '@deepseek-ai/dsh-subprocess-local' +import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local' +import { mkdtemp, readFile, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { expect, it } from 'vitest' +import * as adapter from './snapshot-shell-path.ts' + +it.skipIf(process.platform === 'win32')('translates only the exact fixture command and preserves real shell failures', async () => { + const root = await mkdtemp(join(tmpdir(), 'snapshot-shell-')) + const ctx = new Context() + const disposers: (() => Promise)[] = [] + try { + const subprocess = ctx.plugin(LocalSubprocessRuntime) + disposers.push(() => subprocess.dispose()) + await subprocess + const bash = ctx.plugin(LocalBashExecutor) + disposers.push(() => bash.dispose()) + await bash + const command = "printf 'actual bytes' > /fixture/recorded.txt" + const livePath = join(root, "space and 'quote.txt") + const fork = ctx.plugin(adapter, { command, recordedPath: '/fixture/recorded.txt', livePath }) + disposers.push(() => fork.dispose()) + await fork + const outcome = await ctx.shell.run(ctx.shell.resolve({ command })) + expect(outcome.timedOut).toBe(false) + expect(outcome.exitCode).toBe(0) + expect(await readFile(livePath, 'utf8')).toBe('actual bytes') + const untouched = await ctx.shell.run(ctx.shell.resolve({ command: "printf '/fixture/recorded.txt'; exit 7" })) + expect(untouched.exitCode).toBe(7) + expect(untouched.stdout.text).toBe('/fixture/recorded.txt') + await fork.dispose() + await rm(livePath) + const restored = await ctx.shell.run(ctx.shell.resolve({ command })) + expect(restored.exitCode).not.toBe(0) + await expect(readFile(livePath)).rejects.toMatchObject({ code: 'ENOENT' }) + } finally { + for (const dispose of disposers.reverse()) await dispose() + await rm(root, { recursive: true, force: true }) + } +}) diff --git a/scripts/snapshot-shell-path.ts b/scripts/snapshot-shell-path.ts new file mode 100644 index 0000000000..42287fabdd --- /dev/null +++ b/scripts/snapshot-shell-path.ts @@ -0,0 +1,30 @@ +/** Exact recorded shell-command path translation; execution and reported outcomes remain real. */ +import type { Context } from '@deepseek-ai/cordis' +import type {} from '@deepseek-ai/dsh-shell' + +export const name = 'snapshot-shell-path' +export const inject = ['shell'] + +/** One recorded command and its isolated live filesystem target. */ +export interface Config { + command: string + recordedPath: string + livePath: string +} + +/** + * Translate one exact fixture command after tool logging and approval. + * @param ctx - profile context with its real shell executor. + * @param config - recorded command/path and allocated live target. + */ +export function apply(ctx: Context, config: Config): void { + const shell = ctx.shell + // oxlint-disable-next-line typescript/unbound-method -- preserve method identity for restoration; calls bind the receiver. + const run = shell.run + ctx.effect(() => { + shell.run = spec => run.call(shell, spec.command === config.command + ? { ...spec, command: spec.command.replaceAll(config.recordedPath, "'" + config.livePath.replaceAll("'", "'\"'\"'") + "'") } + : spec) + return () => { shell.run = run } + }) +} diff --git a/scripts/snapshot-spill-locators.spec.ts b/scripts/snapshot-spill-locators.spec.ts new file mode 100644 index 0000000000..03584d8172 --- /dev/null +++ b/scripts/snapshot-spill-locators.spec.ts @@ -0,0 +1,64 @@ +import { Context } from '@deepseek-ai/cordis' +import { LocalSpillStore } from '@deepseek-ai/dsh-spill-local' +import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' +import { SessionId } from '@deepseek-ai/dsh-session' +import { ToolCallId } from '@deepseek-ai/dsh-llm' +import { mkdtemp, readFile, realpath, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join, resolve } from 'node:path' +import { expect, it } from 'vitest' +import * as locators from './snapshot-spill-locators.ts' + +it('keeps concurrent physical spill files private while retaining logical locator length and bytes', async () => { + const roots: string[] = [] + const disposers: (() => Promise)[] = [] + try { + const runs = await Promise.all([0, 1].map(async (index) => { + const root = await mkdtemp(join(tmpdir(), 'snapshot-locator-')) + roots.push(root) + const ctx = new Context() + const storeFiber = ctx.plugin(LocalSpillStore, { root, cleanupPeriodDays: 0 }) + disposers.push(() => storeFiber.dispose()) + await storeFiber + const fsFiber = ctx.plugin(LocalFileSystem, { cwd: root }) + disposers.push(() => fsFiber.dispose()) + await fsFiber + + + const locatorRoot = resolve('/tmp/dsh-acp-snap-123456789') + const fork = ctx.plugin(locators, { root, locatorRoot }) + disposers.push(() => fork.dispose()) + await fork + const content = `physical UTF-8 内容 ${index}` + const ref = await ctx.spillStore.saveText({ + owner: { sessionId: SessionId('same-session') }, + source: { kind: 'tool', toolName: 'bash', callId: ToolCallId('same-call'), label: 'result' }, + suggestedName: 'bash.txt', content, + }) + expect(ref.bytes).toBe(Buffer.byteLength(content)) + expect(ref.locator.startsWith(locatorRoot)).toBe(true) + const target = await ctx.fs.resolve(ref.locator) + expect(target.displayPath).toBe(ref.locator) + const physicalPath = ctx.fs.processPath(target) + expect(physicalPath.startsWith(await realpath(root))).toBe(true) + expect(await readFile(physicalPath, 'utf8')).toBe(content) + expect(await ctx.fs.readText(target)).toBe(content) + await expect(ctx.fs.resolve(join(locatorRoot, 'missing.txt'))).rejects.toThrow('not saved by this run') + const ordinary = join(root, 'ordinary.txt') + await writeFile(ordinary, 'ordinary') + expect(await ctx.fs.readText(await ctx.fs.resolve(ordinary))).toBe('ordinary') + await fork.dispose() + const restored = await ctx.fs.resolve(ref.locator) + expect(ctx.fs.processPath(restored)).not.toBe(physicalPath) + expect(await ctx.fs.stat(restored)).toBeUndefined() + return { physicalPath, locator: ref.locator, content } + })) + expect(runs[0]?.physicalPath).not.toBe(runs[1]?.physicalPath) + expect(runs[0]?.locator.length).toBe(runs[1]?.locator.length) + await rm(roots[0] as string, { recursive: true, force: true }) + expect(await readFile(runs[1]?.physicalPath as string, 'utf8')).toBe(runs[1]?.content) + } finally { + for (const dispose of disposers.reverse()) await dispose() + await Promise.all(roots.map(root => rm(root, { recursive: true, force: true }))) + } +}) diff --git a/scripts/snapshot-spill-locators.ts b/scripts/snapshot-spill-locators.ts new file mode 100644 index 0000000000..7c0762c289 --- /dev/null +++ b/scripts/snapshot-spill-locators.ts @@ -0,0 +1,59 @@ +/** Fixture-only logical locators over real local spill files; preview budgets retain recorded path lengths. */ +import type { Context } from '@deepseek-ai/cordis' +import { join, relative, resolve, sep } from 'node:path' +import type { SpillLocator } from '@deepseek-ai/dsh-spill' +import type {} from '@deepseek-ai/dsh-fs' + +export const name = 'snapshot-spill-locators' +export const inject = ['spillStore', 'fs'] + +/** Live storage and recorded locator prefixes supplied by the snapshot owner. */ +export interface Config { + root: string + locatorRoot: string +} + +/** + * Translate only locators returned by this fixture's real spill backend. + * @param ctx - profile context with real spill and filesystem providers. + * @param config - per-run storage and stable logical prefix. + */ +export function apply(ctx: Context, config: Config): void { + const root = resolve(config.root) + const locatorRoot = resolve(config.locatorRoot) + const paths = new Map() + const store = ctx.spillStore + const fs = ctx.fs + // oxlint-disable-next-line typescript/unbound-method -- preserve method identity for restoration; calls bind the receiver. + const saveText = store.saveText + // oxlint-disable-next-line typescript/unbound-method -- preserve method identity for restoration; calls bind the receiver. + const resolvePath = fs.resolve + ctx.effect(() => { + store.saveText = async (input) => { + const saved = await saveText.call(store, input) + const suffix = relative(root, saved.locator) + if (suffix.startsWith('..') || resolve(root, suffix) !== saved.locator) { + throw new Error('snapshot spill backend returned a locator outside its live root') + } + const locator = join(locatorRoot, suffix) as SpillLocator + paths.set(locator, saved.locator) + return { ...saved, locator } + } + fs.resolve = async (path, opts) => { + const live = paths.get(path) + if (live !== undefined) { + const target = await resolvePath.call(fs, live, opts) + return { ...target, displayPath: path } + } + if (path.startsWith(locatorRoot + sep)) { + throw new Error('snapshot spill locator was not saved by this run') + } + return resolvePath.call(fs, path, opts) + } + return () => { + store.saveText = saveText + fs.resolve = resolvePath + paths.clear() + } + }) +} diff --git a/scripts/snapshot-workspace-parent.spec.ts b/scripts/snapshot-workspace-parent.spec.ts index 4a2b67e1a8..ec4ebf551c 100644 --- a/scripts/snapshot-workspace-parent.spec.ts +++ b/scripts/snapshot-workspace-parent.spec.ts @@ -14,7 +14,7 @@ import { assertWorkspaceOutsideTemp, outsideTempWorkspaceParent } from './snapsh describe('snapshot workspace parent', () => { // Windows directory permissions and root bypass do not enforce POSIX write bits. it.skipIf(process.platform === 'win32' || process.getuid?.() === 0)('uses home when the temp parent is not writable', async () => { - const base = await mkdtemp(join(homedir(), '.dsh-snapshot-readonly-')) + const base = await mkdtemp(join(outsideTempWorkspaceParent(), '.dsh-snapshot-readonly-')) try { const temporary = join(base, '_temp') await mkdir(temporary) diff --git a/snapshots/acp/escalation-approved/cordis.snapshot.yml b/snapshots/acp/escalation-approved/cordis.snapshot.yml index 512708ce64..20f54d6390 100644 --- a/snapshots/acp/escalation-approved/cordis.snapshot.yml +++ b/snapshots/acp/escalation-approved/cordis.snapshot.yml @@ -42,3 +42,11 @@ - id: subagent-settlement-marker name: './subagent-settlement-marker.ts' + +- insert: + - id: snapshot-shell-path + name: ../../../scripts/snapshot-shell-path.ts + config: + command: "printf 'escalated\\n' > /tmp/dsh-escalated.txt && cat /tmp/dsh-escalated.txt && rm /tmp/dsh-escalated.txt" + recordedPath: /tmp/dsh-escalated.txt + livePath: !!js process.env.DSH_SNAPSHOT_SPILL_ROOT + '/dsh-escalated.txt' diff --git a/snapshots/session/fs-glob-sampling/cordis.snapshot.yml b/snapshots/session/fs-glob-sampling/cordis.snapshot.yml index 0fa7e32db9..170ba81e84 100644 --- a/snapshots/session/fs-glob-sampling/cordis.snapshot.yml +++ b/snapshots/session/fs-glob-sampling/cordis.snapshot.yml @@ -79,3 +79,10 @@ name: '@deepseek-ai/dsh-spill-local' config: root: !!js process.env.DSH_SNAPSHOT_SPILL_ROOT ?? './.spill' + +- insert: + - id: snapshot-spill-locators + name: ../../../scripts/snapshot-spill-locators.ts + config: + root: !!js process.env.DSH_SNAPSHOT_SPILL_ROOT + locatorRoot: !!js process.env.DSH_SNAPSHOT_SPILL_LOCATOR_ROOT diff --git a/snapshots/session/fs-glob-sampling/cordis.yml b/snapshots/session/fs-glob-sampling/cordis.yml index 4732f1ecac..01b73d3855 100644 --- a/snapshots/session/fs-glob-sampling/cordis.yml +++ b/snapshots/session/fs-glob-sampling/cordis.yml @@ -71,3 +71,10 @@ name: '@deepseek-ai/dsh-spill-local' config: root: !!js process.env.DSH_SNAPSHOT_SPILL_ROOT ?? './.spill' + +- insert: + - id: snapshot-spill-locators + name: ../../../scripts/snapshot-spill-locators.ts + config: + root: !!js process.env.DSH_SNAPSHOT_SPILL_ROOT + locatorRoot: !!js process.env.DSH_SNAPSHOT_SPILL_LOCATOR_ROOT diff --git a/snapshots/session/headless.snapshot.ts b/snapshots/session/headless.snapshot.ts index f258e4d2b9..e775b689cf 100644 --- a/snapshots/session/headless.snapshot.ts +++ b/snapshots/session/headless.snapshot.ts @@ -847,8 +847,7 @@ describe('headless recorded-session snapshots', () => { let actualLogs: SessionLog[] = [] let initialWorkspace: WorkspaceSnapshotEntry[] | undefined let finalWorkspace: WorkspaceSnapshotEntry[] | undefined - const spillRoot = snapshotSpillRoot(join(scenario.dir, fixtureFiles[0] as string)) - await rm(spillRoot, { recursive: true, force: true }) + const spillRoot = await mkdtemp(join(tmpdir(), 'acp-snap-spill-')) let result: Awaited> try { result = await runLoaderSmoke({ @@ -872,6 +871,7 @@ describe('headless recorded-session snapshots', () => { DSH_SNAPSHOT_PROVIDER: model.provider, DSH_SNAPSHOT_MODEL: model.model, DSH_SNAPSHOT_SPILL_ROOT: spillRoot, + DSH_SNAPSHOT_SPILL_LOCATOR_ROOT: snapshotSpillRoot(join(scenario.dir, fixtureFiles[0] as string)), DSH_SNAPSHOT_FILE: join(scenario.dir, fixtureFiles[0] as string), ...(replaying && fixtureFiles.length > 1 ? { DSH_SNAPSHOT_CHILD_FILES: fixtureFiles.slice(1).map(file => join(scenario.dir, file)).join(delimiter) } diff --git a/snapshots/session/parallel-tool-calls/cordis.snapshot.yml b/snapshots/session/parallel-tool-calls/cordis.snapshot.yml index 7122ae7943..065e1ff523 100644 --- a/snapshots/session/parallel-tool-calls/cordis.snapshot.yml +++ b/snapshots/session/parallel-tool-calls/cordis.snapshot.yml @@ -50,3 +50,10 @@ name: '@deepseek-ai/dsh-spill-policy' config: maxInlineBytes: 800 + +- insert: + - id: snapshot-spill-locators + name: ../../../scripts/snapshot-spill-locators.ts + config: + root: !!js process.env.DSH_SNAPSHOT_SPILL_ROOT + locatorRoot: !!js process.env.DSH_SNAPSHOT_SPILL_LOCATOR_ROOT diff --git a/snapshots/session/parallel-tool-calls/cordis.yml b/snapshots/session/parallel-tool-calls/cordis.yml index 940fb9bd21..3782b6ea8d 100644 --- a/snapshots/session/parallel-tool-calls/cordis.yml +++ b/snapshots/session/parallel-tool-calls/cordis.yml @@ -11,3 +11,10 @@ name: '@deepseek-ai/dsh-spill-policy' config: maxInlineBytes: !!js process.env.DSH_SNAPSHOT && 800 || 50000 + +- insert: + - id: snapshot-spill-locators + name: ../../../scripts/snapshot-spill-locators.ts + config: + root: !!js process.env.DSH_SNAPSHOT_SPILL_ROOT + locatorRoot: !!js process.env.DSH_SNAPSHOT_SPILL_LOCATOR_ROOT diff --git a/snapshots/session/session-query-spill/cordis.snapshot.yml b/snapshots/session/session-query-spill/cordis.snapshot.yml index c4d1fb6641..38f72d9666 100644 --- a/snapshots/session/session-query-spill/cordis.snapshot.yml +++ b/snapshots/session/session-query-spill/cordis.snapshot.yml @@ -58,3 +58,18 @@ - id: timeout-policy name: '@deepseek-ai/dsh-tool-call-timeout-policy' + +- insert: + - id: snapshot-spill-locators + name: ../../../scripts/snapshot-spill-locators.ts + config: + root: !!js process.env.DSH_SNAPSHOT_SPILL_ROOT + locatorRoot: !!js process.env.DSH_SNAPSHOT_SPILL_LOCATOR_ROOT + +- insert: + - id: snapshot-shell-path + name: ../../../scripts/snapshot-shell-path.ts + config: + command: "file=$(find /tmp/dsh-acp-snap-035d1d054 -name '*-session_event_read.txt' -type f); grep -q request/header \"$file\" && grep -q session_event_search \"$file\" && echo SPILL_CANONICAL_OK" + recordedPath: /tmp/dsh-acp-snap-035d1d054 + livePath: !!js process.env.DSH_SNAPSHOT_SPILL_ROOT diff --git a/snapshots/session/session-query-spill/cordis.yml b/snapshots/session/session-query-spill/cordis.yml index ce9d627e93..ef6a8b3a1c 100644 --- a/snapshots/session/session-query-spill/cordis.yml +++ b/snapshots/session/session-query-spill/cordis.yml @@ -20,3 +20,10 @@ - id: timeout-policy name: '@deepseek-ai/dsh-tool-call-timeout-policy' + +- insert: + - id: snapshot-spill-locators + name: ../../../scripts/snapshot-spill-locators.ts + config: + root: !!js process.env.DSH_SNAPSHOT_SPILL_ROOT + locatorRoot: !!js process.env.DSH_SNAPSHOT_SPILL_LOCATOR_ROOT diff --git a/snapshots/session/session-reference-spill/cordis.snapshot.yml b/snapshots/session/session-reference-spill/cordis.snapshot.yml index aa1b3be225..3a57fbac17 100644 --- a/snapshots/session/session-reference-spill/cordis.snapshot.yml +++ b/snapshots/session/session-reference-spill/cordis.snapshot.yml @@ -52,3 +52,10 @@ maxReferenceBytes: 360 - id: reference-source-fixture name: ../../../packages/context/session-reference/tests/fixtures/source-session.ts + +- insert: + - id: snapshot-spill-locators + name: ../../../scripts/snapshot-spill-locators.ts + config: + root: !!js process.env.DSH_SNAPSHOT_SPILL_ROOT + locatorRoot: !!js process.env.DSH_SNAPSHOT_SPILL_LOCATOR_ROOT diff --git a/snapshots/session/session-reference-spill/cordis.yml b/snapshots/session/session-reference-spill/cordis.yml index 91bed4b866..51d0fc224b 100644 --- a/snapshots/session/session-reference-spill/cordis.yml +++ b/snapshots/session/session-reference-spill/cordis.yml @@ -11,3 +11,10 @@ maxReferenceBytes: 360 - id: reference-source-fixture name: ../../../packages/context/session-reference/tests/fixtures/source-session.ts + +- insert: + - id: snapshot-spill-locators + name: ../../../scripts/snapshot-spill-locators.ts + config: + root: !!js process.env.DSH_SNAPSHOT_SPILL_ROOT + locatorRoot: !!js process.env.DSH_SNAPSHOT_SPILL_LOCATOR_ROOT