fix(spill-local): harden startup cleanup
This commit is contained in:
Dudu-0223
parent
545d177911
commit
a268aada8c
18 changed files with 399 additions and 96 deletions
|
|
@ -2,5 +2,5 @@
|
|||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md
|
||||
2026-07-08-tool-output-spill-files.md: 81a292a00af63b0aa9145a0c556a428ab8c49d64
|
||||
2026-07-08-tool-output-spill-files.zh.md: 8d9e60d461297fb11ff2252e91f98a0cfad33f62
|
||||
2026-07-08-tool-output-spill-files.md: e14607e388c634c4e2679c993c1b720be0a3a9f3
|
||||
2026-07-08-tool-output-spill-files.zh.md: 372c9c6cadf3cd64c3de97a8c305b8909f03caab
|
||||
|
|
|
|||
|
|
@ -57,7 +57,7 @@ interface SpillRef {
|
|||
|
||||
`SpillLocator` is a [branded](../../../../packages/util/brand) model-facing handle returned by the backend. The local backend renders it as a filesystem path; a remote or database backend can render a URI, key, or command token. Consumers treat it as opaque and render it with `retrievalHint` instead of assuming `read` is always the right retrieval mechanism. `SpillOwner.sessionId` is the save-time storage namespace: forked sessions inherit existing spill locators from the seeded log without copying or re-owning them, and new spills after the fork use the child session id. A retention-period cleanup may expire old locators with other old session artifacts; the spill seam does not define a per-session cleanup policy.
|
||||
|
||||
`dsh-spill-local` owns only storage details: session-scoped directory selection, safe names, path-traversal protection, the write, and returning `{ locator, bytes, retrievalHint }`. It does not own retention policy, tool-result replacement, search, or file inspection. Files land at `<root>/session-<hash>/<random>-<safeName>`, where `root` is a configured path or a lazily-created private (0700) per-process temp dir, the session subdir is a short `sha256(sessionId)` prefix, and the leaf is a random hex prefix plus the caller's `suggestedName` sanitized to one path segment (mirrors the JSONL backend's `encodeSegment`). The write is `open(path, 'wx', 0o600)` — exclusive and owner-only, so a planted symlink cannot redirect it. The locator is the path, and the retrieval hint tells the model it can use `read` or `grep` on that path.
|
||||
`dsh-spill-local` owns storage details: session-scoped directory selection, safe names, path-traversal protection, the write, local artifact lifetime, and returning `{ locator, bytes, retrievalHint }`. It does not own tool-result replacement, model-facing preview policy, search, file inspection, or a seam-wide/per-session retention policy. Files land at `<root>/session-<hash>/<random>-<safeName>`, where `root` is a configured path or a lazily-created private (0700) per-process temp dir, the session subdir is a short `sha256(sessionId)` prefix, and the leaf is a random hex prefix plus the caller's `suggestedName` sanitized to one path segment (mirrors the JSONL backend's `encodeSegment`). The write is `open(path, 'wx', 0o600)` — exclusive and owner-only, so a planted symlink cannot redirect it. The locator is the path, and the retrieval hint tells the model it can use `read` or `grep` on that path. Its one-shot startup cleanup applies the backend-specific artifact lifetime described in the [local spill cleanup note](./2026-07-17-local-spill-startup-cleanup.md).
|
||||
|
||||
### Spill policy
|
||||
|
||||
|
|
|
|||
|
|
@ -57,7 +57,7 @@ interface SpillRef {
|
|||
|
||||
`SpillLocator` 是一个[品牌化的](../../../../packages/util/brand)模型可见句柄，由后端返回。本地后端将其渲染为文件系统路径；远程或数据库后端可以渲染 URI、键或命令 token。消费方把它视为不透明值，并使用 `retrievalHint` 渲染，而不是假定 `read` 始终是正确的检索机制。`SpillOwner.sessionId` 是保存时的存储命名空间：fork 后的会话会从种子日志继承已有的 spill 定位符，无需复制它们或重新取得所有权；fork 后的新 spill 使用子会话 id。保留期清理可以连同其他旧会话产物一起使旧定位符失效；spill seam 不定义逐会话的清理策略。
|
||||
|
||||
`dsh-spill-local` 只负责存储细节：选择会话作用域的目录、安全名称、防止路径遍历、执行写入，以及返回 `{ locator, bytes, retrievalHint }`。它不负责保留策略、工具结果替换、搜索或文件检查。文件写入 `<root>/session-<hash>/<random>-<safeName>`：`root` 是配置路径，或延迟创建的私有（0700）进程级临时目录；会话子目录是 `sha256(sessionId)` 的短前缀；叶节点由随机十六进制前缀与调用方的 `suggestedName` 组成，后者会被清理成单一路径段（与 JSONL 后端的 `encodeSegment` 一致）。系统使用 `open(path, 'wx', 0o600)` 写入，确保独占且仅所有者可访问，因此预先植入的符号链接无法重定向写入。定位符就是该路径，检索提示则告知模型可以在该路径上使用 `read` 或 `grep`。
|
||||
`dsh-spill-local` 负责存储细节：选择会话作用域的目录、安全名称、防止路径遍历、执行写入、本地产物生命周期，以及返回 `{ locator, bytes, retrievalHint }`。它不负责工具结果替换、模型可见的预览策略、搜索、文件检查，也不定义 seam 级或逐会话保留策略。文件写入 `<root>/session-<hash>/<random>-<safeName>`：`root` 是配置路径，或延迟创建的私有（0700）进程级临时目录；会话子目录是 `sha256(sessionId)` 的短前缀；叶节点由随机十六进制前缀与调用方的 `suggestedName` 组成，后者会被清理成单一路径段（与 JSONL 后端的 `encodeSegment` 一致）。系统使用 `open(path, 'wx', 0o600)` 写入，确保独占且仅所有者可访问，因此预先植入的符号链接无法重定向写入。定位符就是该路径，检索提示则告知模型可以在该路径上使用 `read` 或 `grep`。它的一次性启动清理会应用[本地 spill 清理说明](./2026-07-17-local-spill-startup-cleanup.zh.md)所述的后端专属产物生命周期。
|
||||
|
||||
### spill 策略
|
||||
|
||||
|
|
|
|||
|
|
@ -2,5 +2,5 @@
|
|||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-17-local-spill-startup-cleanup.md
|
||||
2026-07-17-local-spill-startup-cleanup.md: 96378d6ea785d90385f517c1b9a01073ade47fa2
|
||||
2026-07-17-local-spill-startup-cleanup.zh.md: a154cfb824d3a747c2c9acc2b707eb14d703ce2e
|
||||
2026-07-17-local-spill-startup-cleanup.md: fc64938c1af07d9dd0d7ecec379115d22d1e2464
|
||||
2026-07-17-local-spill-startup-cleanup.zh.md: 583a33ead84f552c67e2e770a8b3fabc3ce88120
|
||||
|
|
|
|||
|
|
@ -12,7 +12,9 @@ The local spill backend never deleted the full tool results it wrote. Every over
|
|||
|
||||
`dsh-spill-local` runs one best-effort cleanup sweep after activation. It does not delay service availability, is owned by the plugin fiber (a single `ctx.effect` whose generator launches the sweep and yields an async disposer that awaits it), and is awaited during disposal so no sweep I/O outlives the fiber. There is no recurring timer and no separate process.
|
||||
|
||||
A `cleanupPeriodDays` config defaults to `30`; `0` disables cleanup. An invalid value (negative or fractional) throws at load. The sweep scans the configured/active root plus any prior default `dsh-spill-*` temp roots discovered under the OS temp dir and deletes regular files whose `mtime` is strictly older than `now − cleanupPeriodDays`. It prunes empty session directories and roots only for discovered prior-default roots; the active root keeps its session directories so pruning cannot race a local write, while writes recreate a session directory if another process prunes a discovered root that is still active. It uses `lstat`, so a symlink is never followed or deleted; unrelated entries (non-`session-` directories, special files) are skipped. Every filesystem failure is caught and logged through `ctx.logger.warn`, and a warning-sink exception is also contained — the sweep never throws, so it cannot reject activation or a concurrent spill write. Discovery excludes symlinks and non-directories, returning only real `dsh-spill-*` directories the backend could have created.
|
||||
A `cleanupPeriodDays` config defaults to `30`; `0` disables cleanup. Schemastery rejects a negative or fractional value at load. The sweep scans the configured/active root plus any prior default `dsh-spill-*` temp roots discovered under the OS temp dir and deletes regular files whose `mtime` is strictly older than `now − cleanupPeriodDays`. It prunes every empty session directory but removes the root itself only for a discovered prior-default root; writes recreate a session directory if pruning races them. Root aliases are de-duplicated by device/inode identity, with the configured identity overriding a discovered match as active and non-prunable. It uses `lstat`, so a symlink is never followed or deleted; unrelated entries (non-`session-` directories, special files) are skipped. Every filesystem failure is caught and logged through `ctx.logger.warn`, and a warning-sink exception is also contained — the sweep never throws, so it cannot reject activation or a concurrent spill write.
|
||||
|
||||
Path-based deletion is restricted to directories an untrusted local OS user cannot replace during the scan. On POSIX, every root and session directory must be owned by the current user and not writable by group or others; the root's ancestor path must also be non-writable or protected by a sticky directory such as `/tmp`. Discovery rejects symlinks, while a configured symlink may resolve to a trusted target and participates in identity de-duplication. An unsafe path is skipped with a warning. The same-user account remains the trust boundary, consistent with the backend's private local-storage model.
|
||||
|
||||
The ctx-free sweep mechanics live in `packages/spill/spill-local/src/cleanup.ts` (`sweepSpillRoots`, `discoverDefaultRoots`), unit-testable without a `ctx`; `store.ts` owns root naming, path derivation, and writes, while the service in `src/index.ts` owns the config, cutoff, and fiber-owned launch/await.
|
||||
|
||||
|
|
@ -32,4 +34,4 @@ Cleanup cost the backend a startup sweep and a config knob, and bought a bounded
|
|||
|
||||
## Testing
|
||||
|
||||
`dsh-spill-local` unit tests cover the age boundary (strictly-older expires, boundary kept), `cleanupPeriodDays: 0` disabling, discovered-root pruning, active-directory preservation, symlink/unrelated-entry skipping, configured-plus-discovered-root coverage through the real `gatherRoots`/`discoverDefaultRoots` path, active-root de-duplication, load-time validation of a bad `cleanupPeriodDays`, filesystem- and warning-sink-failure containment both directly and through the service's `ctx.logger.warn` wiring, and the quiescence contract — activation is available while a barrier-held sweep is parked, and disposal only settles after the sweep finishes.
|
||||
`dsh-spill-local` unit tests cover the exact age boundary, `cleanupPeriodDays: 0` disabling, empty-session and discovered-root pruning, symlink/unrelated-entry skipping, configured-plus-discovered-root coverage, filesystem-identity de-duplication through a configured symlink, unsafe POSIX root/session rejection, load-time config validation, filesystem- and warning-sink-failure containment, and the quiescence contract. A separate test boots the plugin through the real Loader and a cordis.yml, then observes configured expiry and directory pruning after disposal.
|
||||
|
|
|
|||
|
|
@ -12,7 +12,9 @@ Status: implemented
|
|||
|
||||
`dsh-spill-local` 在激活后运行一次尽力而为的清理扫描。它不延迟服务可用性，由插件 fiber 拥有（一个 `ctx.effect`，其生成器启动该扫描并让出一个等待它的异步 disposer），并在 dispose 期间被等待，因此没有扫描 I/O 会存活到 fiber 之后。既没有周期性定时器，也没有独立进程。
|
||||
|
||||
`cleanupPeriodDays` 配置默认为 `30`；`0` 会禁用清理。无效值（负数或小数）在加载时抛出。扫描会遍历配置的/活动的根目录，以及在 OS 临时目录下发现的任何先前默认 `dsh-spill-*` 临时根目录，并删除 `mtime` 严格早于 `now − cleanupPeriodDays` 的常规文件。它只修剪发现的先前默认根目录中的空会话目录和空根目录；活动根目录会保留其会话目录，避免修剪操作与本地写入竞争，而当其他进程修剪了一个仍在使用的发现根目录时，写入操作会重新创建会话目录。扫描使用 `lstat`，因此符号链接绝不会被跟随或删除；无关条目（非 `session-` 目录、特殊文件）会被跳过。每一次文件系统失败都会被捕获并通过 `ctx.logger.warn` 记录，警告接收方抛出的异常也会被兜底——扫描绝不抛出，因此它无法让激活失败，也无法影响并发的 spill 写入。发现过程排除符号链接与非目录，只返回后端可能创建过的真实 `dsh-spill-*` 目录。
|
||||
`cleanupPeriodDays` 配置默认为 `30`；`0` 会禁用清理。Schemastery 会在加载时拒绝负数或小数。扫描会遍历配置的/活动的根目录，以及在 OS 临时目录下发现的任何先前默认 `dsh-spill-*` 临时根目录，并删除 `mtime` 严格早于 `now − cleanupPeriodDays` 的常规文件。它会修剪所有空会话目录，但只删除发现的先前默认根目录本身；如果修剪与写入发生竞争，写入操作会重新创建会话目录。根目录别名按设备/inode 身份去重，配置目录的身份会覆盖发现的匹配项，并标记为活动且不可删除。扫描使用 `lstat`，因此符号链接绝不会被跟随或删除；无关条目（非 `session-` 目录、特殊文件）会被跳过。每一次文件系统失败都会被捕获并通过 `ctx.logger.warn` 记录，警告接收方抛出的异常也会被兜底——扫描绝不抛出，因此它无法让激活失败，也无法影响并发的 spill 写入。
|
||||
|
||||
基于路径的删除仅限于不受信任的本地 OS 用户无法在扫描期间替换的目录。在 POSIX 上，每个根目录和会话目录都必须由当前用户拥有，且组用户和其他用户不可写；根目录的祖先路径也必须不可写，或由 `/tmp` 这类 sticky 目录保护。发现过程拒绝符号链接，而配置的符号链接可以解析到可信目标并参与身份去重。不安全路径会被跳过并记录警告。与后端的私有本地存储模型一致，同一用户账号仍是信任边界。
|
||||
|
||||
无 ctx 依赖的扫描机制位于 `packages/spill/spill-local/src/cleanup.ts`（`sweepSpillRoots`、`discoverDefaultRoots`），无需 `ctx` 即可做单元测试；`store.ts` 负责根目录命名、路径推导与写入，而 `src/index.ts` 中的服务负责配置、截止时间以及 fiber 拥有的启动/等待。
|
||||
|
||||
|
|
@ -32,4 +34,4 @@ Status: implemented
|
|||
|
||||
## 验证
|
||||
|
||||
`dsh-spill-local` 单元测试覆盖了年龄边界（严格更旧者过期，边界值保留）、`cleanupPeriodDays: 0` 的禁用、发现根目录的修剪、活动目录的保留、符号链接/无关条目的跳过、通过真实 `gatherRoots`/`discoverDefaultRoots` 路径对配置根加发现根的覆盖、活动根去重、对错误 `cleanupPeriodDays` 的加载期校验、直接测试以及经由服务的 `ctx.logger.warn` 接线测试所覆盖的文件系统与警告接收方失败兜底，以及静止契约：在一个被屏障挂起的扫描停驻期间激活仍然可用，而 dispose 只有在扫描结束后才会完成。
|
||||
`dsh-spill-local` 单元测试覆盖了精确年龄边界、`cleanupPeriodDays: 0` 的禁用、空会话目录与发现根目录的修剪、符号链接/无关条目的跳过、配置根加发现根的覆盖、经配置符号链接验证的文件系统身份去重、不安全 POSIX 根目录/会话目录拒绝、加载期配置校验、文件系统与警告接收方故障兜底，以及静止契约。另一个测试会通过真实 Loader 和 cordis.yml 启动插件，并在 dispose 后观察按配置执行的过期与目录修剪。
|
||||
|
|
|
|||
|
|
@ -2,5 +2,5 @@
|
|||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/config-catalog.md
|
||||
config-catalog.md: 9fc8c333510c568686ce43f4aa1f6d0ae6bc3615
|
||||
config-catalog.zh.md: 4fb689f63631740d485a854e10a12c6d92c6f4ac
|
||||
config-catalog.md: 1b0eceb35a7679e17166d1bb0b9a7a8ae6079613
|
||||
config-catalog.zh.md: 5a698f82db13fe95535ad631636b921819513a74
|
||||
|
|
|
|||
|
|
@ -2067,8 +2067,10 @@ export interface Config {
|
|||
* cleanup sweep. Defaults to `30`; `0` disables cleanup entirely. Files whose
|
||||
* `mtime` is strictly older than the cutoff are deleted and emptied
|
||||
* directories are pruned; fresh files, symlinks, and unrelated entries are
|
||||
* left untouched. Retention is deliberate — a resumed or forked session may
|
||||
* still reference an older locator until it ages out.
|
||||
* left untouched. On POSIX, cleanup skips roots and session directories that
|
||||
* another local user could modify or replace. Retention is deliberate — a
|
||||
* resumed or forked session may still reference an older locator until it
|
||||
* ages out.
|
||||
*/
|
||||
cleanupPeriodDays?: number
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2069,8 +2069,10 @@ export interface Config {
|
|||
* cleanup sweep. Defaults to `30`; `0` disables cleanup entirely. Files whose
|
||||
* `mtime` is strictly older than the cutoff are deleted and emptied
|
||||
* directories are pruned; fresh files, symlinks, and unrelated entries are
|
||||
* left untouched. Retention is deliberate — a resumed or forked session may
|
||||
* still reference an older locator until it ages out.
|
||||
* left untouched. On POSIX, cleanup skips roots and session directories that
|
||||
* another local user could modify or replace. Retention is deliberate — a
|
||||
* resumed or forked session may still reference an older locator until it
|
||||
* ages out.
|
||||
*/
|
||||
cleanupPeriodDays?: number
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2,5 +2,5 @@
|
|||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/spill/spill-local/README.md
|
||||
README.md: 75bde20c423e1d2aa4bba49201b5bb0369d34fd0
|
||||
README.zh.md: 0539969cc4bd4da8dad4f0ac00436476555fd08c
|
||||
README.md: e5a1fb08ba438640e649319f42d31aa10afd24c1
|
||||
README.zh.md: ba6e2b3c4628b7a21de1361fcd2d1e0a1b573f4b
|
||||
|
|
|
|||
|
|
@ -23,7 +23,9 @@ Files land at `<root>/session-<hash>/<random>-<safeName>`:
|
|||
|
||||
The backend never deletes a spill on the write path — a persisted, resumed, or forked session may still reference an older locator, so immediate deletion would break retrieval. Instead, one best-effort sweep runs **once after activation**: it does not delay service availability, is owned by the plugin fiber, and is awaited on disposal (no sweep I/O outlives the fiber). There is no recurring timer and no separate process, so a long-lived deployment is not swept again until its next restart.
|
||||
|
||||
The sweep scans the configured `root` **and** any earlier default `dsh-spill-*` temp roots that prior default-root runs left under the OS temp dir. Within each, it deletes regular files whose `mtime` is strictly older than `now − cleanupPeriodDays`; it prunes empty session directories and roots only for discovered prior-default roots, while the active root keeps its session directories to avoid racing a write. A write recreates its session directory if another process prunes a discovered root that is still active. The sweep never follows or deletes a symlink, skips unrelated entries, and contains every filesystem or warning-sink failure so it cannot fail activation or a concurrent spill write. Retention is deliberate: an old model-visible locator goes stale only once it ages past the cutoff.
|
||||
The sweep scans the configured `root` **and** any earlier default `dsh-spill-*` temp roots that prior default-root runs left under the OS temp dir. It resolves each root to its filesystem identity, so a configured alias of a discovered root remains the active, non-prunable root. Within each root, the sweep deletes regular files whose `mtime` is strictly older than `now − cleanupPeriodDays` and prunes every empty session directory; only an empty discovered prior-default root is itself removed. A write recreates a session directory if cleanup races it. The sweep never follows or deletes a symlink and skips unrelated entries.
|
||||
|
||||
On POSIX, cleanup admits only roots owned by the current user, not writable by group or others, and protected from replacement through their ancestor path; a writable sticky temporary directory such as `/tmp` is permitted. Session directories must satisfy the same ownership and write restrictions. Unsafe paths are skipped with a warning, which prevents an untrusted local process from redirecting path-based deletion outside the spill root. Every filesystem or warning-sink failure is contained, so cleanup cannot fail activation or a concurrent spill write. Retention is deliberate: an old model-visible locator goes stale only once it ages past the cutoff.
|
||||
|
||||
`saveText` rejects on a real storage failure (permissions, ENOSPC); the spill policy treats a rejection as best-effort and keeps the inline result. See the seam README for the vocabulary and the [tool output spill Agent Note](../../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md) for the design, and the [startup-cleanup Agent Note](../../../.agents/notes/implemented/architecture/2026-07-17-local-spill-startup-cleanup.md) for the sweep.
|
||||
|
||||
|
|
|
|||
|
|
@ -23,7 +23,9 @@
|
|||
|
||||
后端不会在写入路径上删除 spill，因为已持久化、已恢复或 fork 后的会话仍可能引用较旧的定位信息，立即删除会使其无法取回。后端会改为在激活后**仅运行一次**尽力而为的扫描：扫描不延迟服务可用性，由插件 fiber 拥有，并在 dispose 期间被等待（不会有扫描 I/O 存活至 fiber 之后）。它既不使用周期性定时器，也不运行独立进程，因此长期运行的部署要到下次重启才会再次扫描。
|
||||
|
||||
扫描会检查配置的 `root` **以及**先前使用默认根目录的运行在操作系统临时目录下留下的所有 `dsh-spill-*` 临时根目录。在每个根目录中，扫描会删除 `mtime` 严格早于 `now − cleanupPeriodDays` 的常规文件；它只修剪发现的先前默认根目录中的空会话目录和空根目录，而活动根目录会保留其会话目录，以避免与写入操作竞争。如果另一个进程修剪了一个仍在使用的发现根目录，写入操作会重新创建其会话目录。扫描绝不会跟随或删除符号链接，会跳过无关条目，并兜底每一次文件系统或警告接收方失败，因此无法使激活或并发 spill 写入失败。保留是刻意的：旧的模型可见定位信息只有超过截止时间后才会失效。
|
||||
扫描会检查配置的 `root` **以及**先前使用默认根目录的运行在操作系统临时目录下留下的所有 `dsh-spill-*` 临时根目录。它会把每个根目录解析为文件系统身份，因此当配置路径是某个已发现根目录的别名时，该目录仍会作为不可删除的活动根目录处理。在每个根目录中，扫描会删除 `mtime` 严格早于 `now − cleanupPeriodDays` 的常规文件并修剪所有空会话目录；只有发现的先前默认根目录会在变空后被删除。如果清理与写入发生竞争，写入操作会重新创建会话目录。扫描绝不会跟随或删除符号链接，并会跳过无关条目。
|
||||
|
||||
在 POSIX 上，清理只接受由当前用户拥有、组用户和其他用户不可写、且祖先路径可防止他人替换的根目录；`/tmp` 这类带 sticky 位的可写临时目录仍然允许使用。会话目录必须满足相同的所有权和写权限限制。不安全路径会被跳过并记录警告，从而防止不受信任的本地进程把基于路径的删除重定向到 spill 根目录之外。所有文件系统故障和警告接收方故障都会被兜底，因此清理无法使激活或并发 spill 写入失败。保留是刻意的：旧的模型可见定位信息只有超过截止时间后才会失效。
|
||||
|
||||
`saveText` 在发生真实存储故障（权限、ENOSPC）时返回拒绝；spill 策略会按尽力而为原则处理该拒绝，并保留内联结果。词汇见 seam README，设计见[工具输出 spill Agent Note](../../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.zh.md)，扫描机制见[启动清理 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-17-local-spill-startup-cleanup.zh.md)。
|
||||
|
||||
|
|
|
|||
|
|
@ -40,6 +40,8 @@
|
|||
"@deepseek-ai/schemastery": "workspace:^"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/cordis-plugin-include": "workspace:^",
|
||||
"@deepseek-ai/cordis-plugin-loader": "workspace:^",
|
||||
"@deepseek-ai/dsh-brand": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
/** Startup cleanup mechanics for local spill roots. */
|
||||
import { lstat, readdir, rmdir, unlink } from 'node:fs/promises'
|
||||
import { join } from 'node:path'
|
||||
import { lstat, readdir, realpath, rmdir, unlink } from 'node:fs/promises'
|
||||
import type { Stats } from 'node:fs'
|
||||
import { dirname, join } from 'node:path'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { DEFAULT_ROOT_PREFIX, isErrno } from './store.ts'
|
||||
|
||||
|
|
@ -21,6 +22,14 @@ const DEFAULT_ROOT_RE = new RegExp(`^${DEFAULT_ROOT_PREFIX}[A-Za-z0-9]{6}$`)
|
|||
*/
|
||||
const SESSION_DIR_RE = /^session-[0-9a-f]{12}$/
|
||||
|
||||
/** An existing root resolved to one stable filesystem identity. */
|
||||
interface ResolvedRoot {
|
||||
/** Canonical absolute path used for the sweep. */
|
||||
path: string
|
||||
/** Device/inode identity used to de-duplicate filesystem aliases. */
|
||||
identity: string
|
||||
}
|
||||
|
||||
/** A one-argument warning sink — the sweep's only side effect on failure (never throws). */
|
||||
export type WarnFn = (message: string) => void
|
||||
|
||||
|
|
@ -34,16 +43,116 @@ function warnSafely(warn: WarnFn, message: string): void {
|
|||
}
|
||||
}
|
||||
|
||||
/** One root to sweep, plus whether its empty session directories and root may be pruned. */
|
||||
/** Whether another local OS user cannot replace children of this directory. */
|
||||
function isTrustedDirectory(stats: Stats): boolean {
|
||||
if (!stats.isDirectory()) return false
|
||||
/* v8 ignore next -- POSIX ownership and mode bits have no Windows equivalent. */
|
||||
if (process.platform === 'win32' || process.geteuid === undefined) return true
|
||||
return stats.uid === process.geteuid() && (stats.mode & 0o022) === 0
|
||||
}
|
||||
|
||||
/** Stable identity for de-duplicating aliases of one root. */
|
||||
function rootIdentity(path: string, stats: Stats): string {
|
||||
/* v8 ignore next -- Windows file indexes are not portable inode identities. */
|
||||
if (process.platform === 'win32') return path.toLowerCase()
|
||||
return `${String(stats.dev)}:${String(stats.ino)}`
|
||||
}
|
||||
|
||||
/**
|
||||
* Check that no ancestor permits another local OS user to replace the selected
|
||||
* child. A sticky writable ancestor is safe because the child is owned by the
|
||||
* current user; this admits normal per-process roots below `/tmp`.
|
||||
*/
|
||||
async function hasProtectedAncestors(path: string): Promise<boolean> {
|
||||
/* v8 ignore next -- POSIX ancestry checks have no Windows ACL equivalent. */
|
||||
if (process.platform === 'win32' || process.geteuid === undefined) return true
|
||||
const currentUid = process.geteuid()
|
||||
let child = path
|
||||
let childStats = await lstat(child)
|
||||
for (;;) {
|
||||
const parent = dirname(child)
|
||||
if (parent === child) return true
|
||||
const stats = await lstat(parent)
|
||||
/* v8 ignore next -- every ancestor of a successfully resolved path is a directory. */
|
||||
if (!stats.isDirectory()) return false
|
||||
const writableByOthers = (stats.mode & 0o022) !== 0
|
||||
const sticky = (stats.mode & 0o1000) !== 0
|
||||
if (writableByOthers && !sticky) return false
|
||||
/* v8 ignore next -- requires an ancestor owned by another OS account inside
|
||||
a writable sticky parent; ordinary test fixtures cannot change uid. */
|
||||
if (writableByOthers && childStats.uid !== currentUid) return false
|
||||
child = parent
|
||||
childStats = stats
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve one existing root without admitting a directory another local user
|
||||
* can replace during the path-based sweep. A configured root may be a symlink;
|
||||
* discovery passes `false` so a symlink cannot impersonate a default root.
|
||||
*
|
||||
* @param path Candidate root path.
|
||||
* @param allowSymlink Whether the candidate itself may be a configured symlink.
|
||||
* @param warn Sink for skipped or failed inspection.
|
||||
* @returns The trusted canonical root, or `undefined` when it is absent or unsafe.
|
||||
*/
|
||||
async function resolveRoot(path: string, allowSymlink: boolean, warn: WarnFn): Promise<ResolvedRoot | undefined> {
|
||||
let initial: Stats
|
||||
try {
|
||||
initial = await lstat(path)
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore start -- non-ENOENT inspection failures depend on host ACL or
|
||||
an entry racing away and cannot be reproduced portably. */
|
||||
if (!isErrno(error, 'ENOENT')) warnSafely(warn, `spill-local: failed to inspect root ${path}: ${String(error)}`)
|
||||
return undefined
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
if (initial.isSymbolicLink()) {
|
||||
if (!allowSymlink) return undefined
|
||||
} else if (!isTrustedDirectory(initial)) {
|
||||
warnSafely(warn, `spill-local: skipped unsafe root ${path}: expected a directory owned by the current user and not writable by group or others`)
|
||||
return undefined
|
||||
}
|
||||
|
||||
let canonical: string
|
||||
let stats: Stats
|
||||
try {
|
||||
canonical = await realpath(path)
|
||||
stats = await lstat(canonical)
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore start -- a root lstat'd above reaches this only by racing away
|
||||
or by a host-specific realpath failure. */
|
||||
if (!isErrno(error, 'ENOENT')) warnSafely(warn, `spill-local: failed to resolve root ${path}: ${String(error)}`)
|
||||
return undefined
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
let protectedAncestors = false
|
||||
try {
|
||||
protectedAncestors = await hasProtectedAncestors(canonical)
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore start -- a canonical ancestor disappears only through a race;
|
||||
other failures depend on host ACLs. */
|
||||
if (!isErrno(error, 'ENOENT')) warnSafely(warn, `spill-local: failed to inspect ancestors of root ${canonical}: ${String(error)}`)
|
||||
return undefined
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
if (!isTrustedDirectory(stats) || !protectedAncestors) {
|
||||
warnSafely(warn, `spill-local: skipped unsafe root ${canonical}: expected a current-user-owned directory with protected write and ancestor permissions`)
|
||||
return undefined
|
||||
}
|
||||
return { path: canonical, identity: rootIdentity(canonical, stats) }
|
||||
}
|
||||
|
||||
/** One root to sweep, plus whether the root itself may be pruned once empty. */
|
||||
export interface SweepRoot {
|
||||
/** Absolute spill root to sweep. */
|
||||
path: string
|
||||
/**
|
||||
* When `true`, prune empty `session-*` children and then remove the root once
|
||||
* empty. Set for DISCOVERED prior-default `dsh-spill-*` roots (one per past
|
||||
* process — otherwise they accumulate empty forever), never for the
|
||||
* active/configured root the live process is still writing into. Writes retry
|
||||
* if another process still using a discovered root races its pruning.
|
||||
* When `true`, remove the root after its empty `session-*` children are
|
||||
* pruned. Set for DISCOVERED prior-default `dsh-spill-*` roots (one per past
|
||||
* process — otherwise they accumulate empty forever), never for the active
|
||||
* root the live process is still writing into. Every root prunes empty session
|
||||
* directories; writes retry if that races their removal.
|
||||
*/
|
||||
pruneWhenEmpty: boolean
|
||||
}
|
||||
|
|
@ -141,26 +250,39 @@ async function sweepSessionDir(dir: string, cutoffMs: number, warn: WarnFn): Pro
|
|||
|
||||
/**
|
||||
* Best-effort one-shot cleanup: across each root, delete expired regular files
|
||||
* under its `session-*` directories, pruning empty directories only in
|
||||
* discovered prior-default roots. The active root keeps its session directories
|
||||
* to avoid racing a local write; writes recreate a directory pruned by another
|
||||
* process. Every filesystem and warning-sink failure is contained, so a caller
|
||||
* can await this during activation/disposal without it ever rejecting.
|
||||
* under its `session-*` directories and prune every empty session directory.
|
||||
* Only a discovered prior-default root is itself removed. Writes recreate a
|
||||
* session directory when pruning races a local write. Every filesystem and
|
||||
* warning-sink failure is contained, so a caller can await this during
|
||||
* activation/disposal without it ever rejecting.
|
||||
*
|
||||
* @param options The roots to sweep, the age cutoff, and the failure sink.
|
||||
* @returns Resolves when the sweep finishes (never rejects).
|
||||
*/
|
||||
export async function sweepSpillRoots(options: SweepOptions): Promise<void> {
|
||||
const { roots, cutoffMs, warn } = options
|
||||
for (const root of roots) {
|
||||
const { cutoffMs, warn } = options
|
||||
const roots = new Map<string, SweepRoot>()
|
||||
for (const candidate of options.roots) {
|
||||
const resolved = await resolveRoot(candidate.path, false, warn)
|
||||
if (resolved === undefined) continue
|
||||
const existing = roots.get(resolved.identity)
|
||||
roots.set(resolved.identity, {
|
||||
path: resolved.path,
|
||||
pruneWhenEmpty: (existing?.pruneWhenEmpty ?? true) && candidate.pruneWhenEmpty,
|
||||
})
|
||||
}
|
||||
for (const root of roots.values()) {
|
||||
let entries: string[]
|
||||
try {
|
||||
entries = await readdir(root.path)
|
||||
} catch (error: unknown) {
|
||||
// A root that does not exist yet (no spill ever written) is the common
|
||||
// case, not an error: ENOENT is silent, anything else is reported.
|
||||
/* v8 ignore start -- the trusted root was resolved immediately above; a
|
||||
read failure now requires a race or host-specific ACL fault. */
|
||||
if (!isErrno(error, 'ENOENT')) warnSafely(warn, `spill-local: failed to read root ${root.path}: ${String(error)}`)
|
||||
continue
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
// Track whether the root holds ANY entry the sweep did not fully reclaim, so
|
||||
// a discovered prior-default root can be pruned only when nothing remains.
|
||||
|
|
@ -185,15 +307,13 @@ export async function sweepSpillRoots(options: SweepOptions): Promise<void> {
|
|||
continue
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
if (!stats.isDirectory()) { rootEmptiable = false; continue }
|
||||
const empty = await sweepSessionDir(dir, cutoffMs, warn)
|
||||
if (!empty) { rootEmptiable = false; continue }
|
||||
if (!root.pruneWhenEmpty) {
|
||||
// The active root remains writable while cleanup runs. Leaving its empty
|
||||
// session directories in place closes the mkdir/rmdir race with saveText.
|
||||
if (!isTrustedDirectory(stats)) {
|
||||
warnSafely(warn, `spill-local: skipped unsafe session directory ${dir}`)
|
||||
rootEmptiable = false
|
||||
continue
|
||||
}
|
||||
const empty = await sweepSessionDir(dir, cutoffMs, warn)
|
||||
if (!empty) { rootEmptiable = false; continue }
|
||||
try {
|
||||
await rmdir(dir)
|
||||
} catch (error: unknown) {
|
||||
|
|
@ -210,8 +330,7 @@ export async function sweepSpillRoots(options: SweepOptions): Promise<void> {
|
|||
}
|
||||
// A discovered prior-default root (one per past process) is removed once its
|
||||
// last session dir is gone — otherwise empty roots accumulate forever and
|
||||
// every future startup rescans them. The active/configured root is never
|
||||
// pruned (the live process is still writing into it).
|
||||
// every future startup rescans them. The active root itself is never pruned.
|
||||
if (root.pruneWhenEmpty && rootEmptiable) {
|
||||
try {
|
||||
await rmdir(root.path)
|
||||
|
|
@ -245,7 +364,7 @@ export async function sweepSpillRoots(options: SweepOptions): Promise<void> {
|
|||
* @param base The directory to scan; defaults to the OS tmpdir (a test seam).
|
||||
* @returns Absolute paths of the discovered default roots (possibly empty).
|
||||
*/
|
||||
export async function discoverDefaultRoots(warn: WarnFn, base: string = tmpdir()): Promise<string[]> {
|
||||
async function discoverDefaultRootRecords(warn: WarnFn, base: string): Promise<ResolvedRoot[]> {
|
||||
let entries: string[]
|
||||
try {
|
||||
entries = await readdir(base)
|
||||
|
|
@ -253,24 +372,48 @@ export async function discoverDefaultRoots(warn: WarnFn, base: string = tmpdir()
|
|||
warnSafely(warn, `spill-local: failed to scan ${base} for default roots: ${String(error)}`)
|
||||
return []
|
||||
}
|
||||
const roots: string[] = []
|
||||
const roots: ResolvedRoot[] = []
|
||||
for (const name of entries) {
|
||||
if (!DEFAULT_ROOT_RE.test(name)) continue
|
||||
const path = join(base, name)
|
||||
let stats
|
||||
try {
|
||||
// lstat, not stat: a symlink named `dsh-spill-*` must not be treated as a
|
||||
// root we then sweep (it could point anywhere).
|
||||
stats = await lstat(path)
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore start -- an entry readdir just returned fails to lstat only by
|
||||
racing away (ENOENT) or a permission/IO fault; not deterministically
|
||||
reproducible. */
|
||||
if (!isErrno(error, 'ENOENT')) warnSafely(warn, `spill-local: failed to stat default root ${path}: ${String(error)}`)
|
||||
continue
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
if (stats.isDirectory()) roots.push(path)
|
||||
const resolved = await resolveRoot(path, false, warn)
|
||||
if (resolved !== undefined) roots.push(resolved)
|
||||
}
|
||||
return roots
|
||||
}
|
||||
|
||||
/**
|
||||
* Discover trusted prior default roots below the OS temporary directory.
|
||||
*
|
||||
* @param warn Sink for contained discovery failures.
|
||||
* @param base Directory to scan; defaults to the OS temporary directory.
|
||||
* @returns Canonical paths of trusted default roots.
|
||||
*/
|
||||
export async function discoverDefaultRoots(warn: WarnFn, base: string = tmpdir()): Promise<string[]> {
|
||||
return (await discoverDefaultRootRecords(warn, base)).map(root => root.path)
|
||||
}
|
||||
|
||||
/**
|
||||
* Gather and de-duplicate the trusted roots for one startup sweep. The active
|
||||
* configured path may be a symlink; its resolved identity overrides a matching
|
||||
* discovered root so the live target is never marked prunable.
|
||||
*
|
||||
* @param activeRoot Active configured root.
|
||||
* @param warn Sink for contained inspection failures.
|
||||
* @param defaultRootsBase Directory holding prior default roots.
|
||||
* @returns Trusted roots with the active identity marked non-prunable.
|
||||
*/
|
||||
export async function gatherSweepRoots(
|
||||
activeRoot: string,
|
||||
warn: WarnFn,
|
||||
defaultRootsBase: string = tmpdir(),
|
||||
): Promise<SweepRoot[]> {
|
||||
const [discovered, active] = await Promise.all([
|
||||
discoverDefaultRootRecords(warn, defaultRootsBase),
|
||||
resolveRoot(activeRoot, true, warn),
|
||||
])
|
||||
const roots = new Map<string, SweepRoot>()
|
||||
for (const root of discovered) roots.set(root.identity, { path: root.path, pruneWhenEmpty: true })
|
||||
if (active !== undefined) roots.set(active.identity, { path: active.path, pruneWhenEmpty: false })
|
||||
return [...roots.values()]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ import { tmpdir } from 'node:os'
|
|||
import z from '@deepseek-ai/schemastery'
|
||||
import { SpillLocator, SpillStore } from '@deepseek-ai/dsh-spill'
|
||||
import type { SaveTextSpill, SpillRef } from '@deepseek-ai/dsh-spill'
|
||||
import { discoverDefaultRoots, sweepSpillRoots } from './cleanup.ts'
|
||||
import { gatherSweepRoots, sweepSpillRoots } from './cleanup.ts'
|
||||
import type { SweepRoot, WarnFn } from './cleanup.ts'
|
||||
import { privateRoot, saveTextFile } from './store.ts'
|
||||
|
||||
|
|
@ -40,8 +40,10 @@ export interface Config {
|
|||
* cleanup sweep. Defaults to `30`; `0` disables cleanup entirely. Files whose
|
||||
* `mtime` is strictly older than the cutoff are deleted and emptied
|
||||
* directories are pruned; fresh files, symlinks, and unrelated entries are
|
||||
* left untouched. Retention is deliberate — a resumed or forked session may
|
||||
* still reference an older locator until it ages out.
|
||||
* left untouched. On POSIX, cleanup skips roots and session directories that
|
||||
* another local user could modify or replace. Retention is deliberate — a
|
||||
* resumed or forked session may still reference an older locator until it
|
||||
* ages out.
|
||||
*/
|
||||
cleanupPeriodDays?: number
|
||||
}
|
||||
|
|
@ -63,7 +65,7 @@ type ResolvedConfig = Required<Omit<Config, 'root'>> & Pick<Config, 'root'>
|
|||
export class LocalSpillStore extends SpillStore {
|
||||
static Config: z<Config> = z.object({
|
||||
root: z.string(),
|
||||
cleanupPeriodDays: z.number().default(30),
|
||||
cleanupPeriodDays: z.number().step(1).min(0).default(30),
|
||||
})
|
||||
|
||||
/** Resolved absolute spill root (config `root`, else the private default), fixed at construction. */
|
||||
|
|
@ -83,9 +85,6 @@ export class LocalSpillStore extends SpillStore {
|
|||
// schemastery (static Config) has already filled `cleanupPeriodDays`; the
|
||||
// cast records that runtime fact for exactOptionalPropertyTypes.
|
||||
this.config = config as ResolvedConfig
|
||||
if (!Number.isInteger(this.config.cleanupPeriodDays) || this.config.cleanupPeriodDays < 0) {
|
||||
throw new Error(`spill-local: cleanupPeriodDays must be a non-negative integer (got ${this.config.cleanupPeriodDays})`)
|
||||
}
|
||||
this.root = config.root !== undefined ? resolve(config.root) : privateRoot()
|
||||
|
||||
// One best-effort startup sweep, owned by the fiber. The generator body runs
|
||||
|
|
@ -120,24 +119,19 @@ export class LocalSpillStore extends SpillStore {
|
|||
/**
|
||||
* The roots the startup sweep covers: each discovered prior-default
|
||||
* `dsh-spill-*` temp root (see {@link discoverDefaultRoots}), pruned when
|
||||
* emptied, plus the active/configured root, whose root and session directories
|
||||
* are NEVER pruned (the live process is still writing into them). The active
|
||||
* root is de-duped out of the discovered set so it is not swept twice or
|
||||
* marked prunable. A test
|
||||
* overrides this to inject an isolated root set — and, being the sweep's one
|
||||
* async gather point, to hold the sweep open across a disposal for the
|
||||
* quiescence check; it is a test seam, not a deployment knob.
|
||||
* emptied, plus the active/configured root, which is never itself pruned while
|
||||
* the live process may write into it. Empty session directories are pruned in
|
||||
* every root. Filesystem identity de-duplicates aliases before the active root
|
||||
* overrides a discovered match as non-prunable. A test overrides this to
|
||||
* inject an isolated root set — and, being the sweep's one async gather point,
|
||||
* to hold the sweep open across a disposal for the quiescence check; it is a
|
||||
* test seam, not a deployment knob.
|
||||
*
|
||||
* @param warn - sink for a contained discovery failure.
|
||||
* @returns The roots to sweep, each flagged for prune-when-empty.
|
||||
*/
|
||||
protected async gatherRoots(warn: WarnFn): Promise<SweepRoot[]> {
|
||||
const discovered = await discoverDefaultRoots(warn, this.defaultRootsBase())
|
||||
const roots: SweepRoot[] = discovered
|
||||
.filter(path => path !== this.root)
|
||||
.map(path => ({ path, pruneWhenEmpty: true }))
|
||||
roots.push({ path: this.root, pruneWhenEmpty: false })
|
||||
return roots
|
||||
return gatherSweepRoots(this.root, warn, this.defaultRootsBase())
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
78
packages/spill/spill-local/tests/loader-composition.spec.ts
Normal file
78
packages/spill/spill-local/tests/loader-composition.spec.ts
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
/**
|
||||
* Real-composition proof: a cordis.yml loaded by the vendored Loader applies
|
||||
* spill-local configuration and completes its fiber-owned startup cleanup.
|
||||
*/
|
||||
|
||||
import { mkdir, mkdtemp, rm, utimes, writeFile } from 'node:fs/promises'
|
||||
import { existsSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { pathToFileURL } from 'node:url'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import Loader from '@deepseek-ai/cordis-plugin-loader'
|
||||
import Include from '@deepseek-ai/cordis-plugin-include'
|
||||
import LocalSpillStore, { sessionDir } from '@deepseek-ai/dsh-spill-local'
|
||||
|
||||
const DAY_MS = 24 * 60 * 60 * 1000
|
||||
|
||||
let root: string | undefined
|
||||
let context: Context | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
await context?.fiber.dispose()
|
||||
context = undefined
|
||||
if (root !== undefined) await rm(root, { recursive: true, force: true })
|
||||
root = undefined
|
||||
})
|
||||
|
||||
describe('spill-local real Loader composition through cordis.yml', () => {
|
||||
it('loads cleanupPeriodDays and prunes only expired session contents', async () => {
|
||||
root = await mkdtemp(join(tmpdir(), 'dsh-spill-loader-'))
|
||||
const oldDir = sessionDir(root, 'old-session')
|
||||
const freshDir = sessionDir(root, 'fresh-session')
|
||||
await mkdir(oldDir, { recursive: true })
|
||||
await mkdir(freshDir, { recursive: true })
|
||||
const old = join(oldDir, 'old.txt')
|
||||
const fresh = join(freshDir, 'fresh.txt')
|
||||
await writeFile(old, 'old')
|
||||
await writeFile(fresh, 'fresh')
|
||||
const now = Date.now()
|
||||
await utimes(old, (now - 40 * DAY_MS) / 1000, (now - 40 * DAY_MS) / 1000)
|
||||
await utimes(fresh, (now - DAY_MS) / 1000, (now - DAY_MS) / 1000)
|
||||
|
||||
const configPath = join(root, 'cordis.yml')
|
||||
await writeFile(configPath, [
|
||||
"- name: '@deepseek-ai/dsh-spill-local'",
|
||||
' config:',
|
||||
` root: ${JSON.stringify(root)}`,
|
||||
' cleanupPeriodDays: 30',
|
||||
'',
|
||||
].join('\n'))
|
||||
|
||||
context = new Context()
|
||||
context.baseUrl = pathToFileURL(root).href + '/'
|
||||
await context.plugin(Loader)
|
||||
context.loader.builtins.include = Include
|
||||
context.loader.internal = {
|
||||
version: 'v2',
|
||||
async import(specifier: string) {
|
||||
if (specifier !== '@deepseek-ai/dsh-spill-local') throw new Error(`unexpected Loader import: ${specifier}`)
|
||||
return LocalSpillStore
|
||||
},
|
||||
} as unknown as NonNullable<typeof context.loader.internal>
|
||||
await context.loader.create({
|
||||
name: 'cordis:include',
|
||||
config: { path: pathToFileURL(configPath).href },
|
||||
})
|
||||
await context.loader.await()
|
||||
await context.fiber.dispose()
|
||||
context = undefined
|
||||
|
||||
expect(existsSync(old)).toBe(false)
|
||||
expect(existsSync(oldDir)).toBe(false)
|
||||
expect(existsSync(fresh)).toBe(true)
|
||||
expect(existsSync(freshDir)).toBe(true)
|
||||
expect(existsSync(root)).toBe(true)
|
||||
}, 30_000)
|
||||
})
|
||||
|
|
@ -11,7 +11,7 @@
|
|||
|
||||
import { describe, expect, it, beforeEach, afterEach, vi } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, statSync, symlinkSync, utimesSync, writeFileSync } from 'node:fs'
|
||||
import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, statSync, symlinkSync, utimesSync, writeFileSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { basename, dirname, isAbsolute, join, normalize } from 'node:path'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
|
|
@ -28,6 +28,7 @@ import LocalSpillStore, {
|
|||
sweepSpillRoots,
|
||||
} from '@deepseek-ai/dsh-spill-local'
|
||||
import type { SweepRoot } from '@deepseek-ai/dsh-spill-local'
|
||||
import { gatherSweepRoots } from '../src/cleanup.ts'
|
||||
|
||||
const DAY_MS = 24 * 60 * 60 * 1000
|
||||
|
||||
|
|
@ -169,9 +170,9 @@ describe('LocalSpillStore service', () => {
|
|||
|
||||
it('rejects a negative or fractional cleanupPeriodDays at load', async () => {
|
||||
await expect(new Context().plugin(LocalSpillStore, { root, cleanupPeriodDays: -1 }))
|
||||
.rejects.toThrow(/cleanupPeriodDays must be a non-negative integer/)
|
||||
.rejects.toThrow()
|
||||
await expect(new Context().plugin(LocalSpillStore, { root, cleanupPeriodDays: 1.5 }))
|
||||
.rejects.toThrow(/cleanupPeriodDays must be a non-negative integer/)
|
||||
.rejects.toThrow()
|
||||
})
|
||||
|
||||
it('defaults cleanupPeriodDays to 30', async () => {
|
||||
|
|
@ -207,8 +208,8 @@ describe('LocalSpillStore service', () => {
|
|||
})
|
||||
|
||||
it('routes a sweep filesystem failure to ctx.logger.warn (service warn wiring)', async () => {
|
||||
// A root that is a FILE, not a directory, makes readdir throw ENOTDIR inside
|
||||
// the real sweep. The service's warn closure must forward it to
|
||||
// A root that is a FILE, not a directory, is rejected by the real sweep.
|
||||
// The service's warn closure must forward that failure to
|
||||
// ctx.logger.warn, and disposal must still settle cleanly.
|
||||
const filePath = join(root, 'not-a-dir'); writeFileSync(filePath, 'x')
|
||||
const ctx = new Context()
|
||||
|
|
@ -218,7 +219,7 @@ describe('LocalSpillStore service', () => {
|
|||
}
|
||||
const fiber = await ctx.plugin(Discovering, { root: filePath, cleanupPeriodDays: 30 })
|
||||
await fiber.dispose()
|
||||
expect(warn).toHaveBeenCalledWith(expect.stringContaining('failed to read root'))
|
||||
expect(warn).toHaveBeenCalledWith(expect.stringContaining('skipped unsafe root'))
|
||||
})
|
||||
})
|
||||
|
||||
|
|
@ -265,10 +266,11 @@ describe('startup cleanup sweep', () => {
|
|||
it('keeps a file exactly at the boundary (only strictly-older expires)', async () => {
|
||||
const dir = sessionDir(root, 'sess-1')
|
||||
mkdirSync(dir, { recursive: true })
|
||||
// mtime == cutoff: mtimeMs >= cutoffMs holds, so it is kept. Age it just
|
||||
// under 30d to avoid the sub-millisecond race of "exactly now - 30d".
|
||||
const boundary = join(dir, 'boundary.txt'); writeAged(boundary, 'x', 29.9)
|
||||
await runSweep([active(root)])
|
||||
const cutoffMs = Date.now() - 30 * DAY_MS
|
||||
const boundary = join(dir, 'boundary.txt')
|
||||
writeFileSync(boundary, 'x')
|
||||
utimesSync(boundary, cutoffMs / 1000, cutoffMs / 1000)
|
||||
await sweepSpillRoots({ roots: [active(root)], cutoffMs, warn: () => {} })
|
||||
expect(existsSync(boundary)).toBe(true)
|
||||
})
|
||||
|
||||
|
|
@ -280,7 +282,7 @@ describe('startup cleanup sweep', () => {
|
|||
expect(existsSync(old)).toBe(true)
|
||||
})
|
||||
|
||||
it('keeps active session directories after deleting expired files', async () => {
|
||||
it('prunes empty active session directories after deleting expired files', async () => {
|
||||
const emptied = sessionDir(root, 'emptied')
|
||||
const kept = sessionDir(root, 'kept')
|
||||
mkdirSync(emptied, { recursive: true })
|
||||
|
|
@ -288,7 +290,7 @@ describe('startup cleanup sweep', () => {
|
|||
writeAged(join(emptied, 'a.txt'), 'x', 40)
|
||||
writeAged(join(kept, 'fresh.txt'), 'y', 1)
|
||||
await runSweep([active(root)])
|
||||
expect(existsSync(emptied)).toBe(true)
|
||||
expect(existsSync(emptied)).toBe(false)
|
||||
expect(existsSync(kept)).toBe(true)
|
||||
})
|
||||
|
||||
|
|
@ -322,6 +324,18 @@ describe('startup cleanup sweep', () => {
|
|||
expect(existsSync(link)).toBe(true)
|
||||
})
|
||||
|
||||
it('skips a POSIX session directory writable by another local user', async () => {
|
||||
if (process.platform === 'win32') return
|
||||
const dir = sessionDir(root, 'sess-1')
|
||||
mkdirSync(dir, { recursive: true })
|
||||
const old = join(dir, 'old.txt'); writeAged(old, 'x', 40)
|
||||
chmodSync(dir, 0o777)
|
||||
const warn = vi.fn()
|
||||
await sweepSpillRoots({ roots: [active(root)], cutoffMs: Date.now(), warn })
|
||||
expect(existsSync(old)).toBe(true)
|
||||
expect(warn).toHaveBeenCalledWith(expect.stringContaining('skipped unsafe session directory'))
|
||||
})
|
||||
|
||||
it('sweeps only exact session-<12hex> names, not lookalikes', async () => {
|
||||
// `session-backup` and `session-<11hex>` match the old startsWith check but
|
||||
// are NOT backend-generated names; their old files must survive.
|
||||
|
|
@ -351,12 +365,29 @@ describe('startup cleanup sweep', () => {
|
|||
await runSweep([{ path: prior, pruneWhenEmpty: true }, active(root)])
|
||||
expect(existsSync(prior)).toBe(false) // discovered root pruned
|
||||
expect(existsSync(root)).toBe(true) // active root kept
|
||||
expect(existsSync(activeDir)).toBe(true) // active session dirs remain writable
|
||||
expect(existsSync(activeDir)).toBe(false) // empty active session dirs are pruned
|
||||
} finally {
|
||||
rmSync(prior, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
it('de-duplicates repeated roots and lets non-prunable status win', async () => {
|
||||
const dir = sessionDir(root, 'sess-1')
|
||||
mkdirSync(dir, { recursive: true })
|
||||
writeAged(join(dir, 'old.txt'), 'x', 40)
|
||||
await sweepSpillRoots({
|
||||
roots: [
|
||||
{ path: root, pruneWhenEmpty: true },
|
||||
{ path: root, pruneWhenEmpty: false },
|
||||
{ path: root, pruneWhenEmpty: true },
|
||||
],
|
||||
cutoffMs: Date.now() - 30 * DAY_MS,
|
||||
warn: () => {},
|
||||
})
|
||||
expect(existsSync(dir)).toBe(false)
|
||||
expect(existsSync(root)).toBe(true)
|
||||
})
|
||||
|
||||
it('does NOT prune a discovered root that still holds a fresh file', async () => {
|
||||
const prior = mkdtempSync(join(tmpdir(), 'dsh-spill-'))
|
||||
const priorDir = sessionDir(prior, 'sess'); mkdirSync(priorDir, { recursive: true })
|
||||
|
|
@ -424,6 +455,43 @@ describe('startup cleanup sweep', () => {
|
|||
}
|
||||
})
|
||||
|
||||
it('de-dups a configured symlink alias by filesystem identity and keeps its target writable', async () => {
|
||||
const fakeTmp = mkdtempSync(join(tmpdir(), 'dsh-faketmp-'))
|
||||
const activeDefault = mkdtempSync(join(fakeTmp, DEFAULT_ROOT_PREFIX))
|
||||
const alias = join(root, 'configured-root')
|
||||
symlinkSync(activeDefault, alias, process.platform === 'win32' ? 'junction' : 'dir')
|
||||
const dir = sessionDir(activeDefault, 'sess-1')
|
||||
mkdirSync(dir, { recursive: true })
|
||||
const old = join(dir, 'old.txt'); writeAged(old, 'x', 40)
|
||||
try {
|
||||
const roots = await gatherSweepRoots(alias, () => {}, fakeTmp)
|
||||
expect(roots).toEqual([{ path: realpathSync(activeDefault), pruneWhenEmpty: false }])
|
||||
await sweepSpillRoots({ roots, cutoffMs: Date.now() - 30 * DAY_MS, warn: () => {} })
|
||||
expect(existsSync(old)).toBe(false)
|
||||
expect(existsSync(activeDefault)).toBe(true)
|
||||
const saved = await saveTextFile({ root: alias, sessionId: 'next', suggestedName: 'ok.txt', content: 'ok' })
|
||||
expect(readFileSync(saved.path, 'utf8')).toBe('ok')
|
||||
} finally {
|
||||
rmSync(fakeTmp, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
it('skips a root that another POSIX user could replace', async () => {
|
||||
if (process.platform === 'win32') return
|
||||
const unsafeParent = join(root, 'unsafe-parent')
|
||||
const unsafeRoot = join(unsafeParent, 'configured')
|
||||
mkdirSync(unsafeRoot, { recursive: true, mode: 0o700 })
|
||||
const dir = sessionDir(unsafeRoot, 'sess-1')
|
||||
mkdirSync(dir, { recursive: true })
|
||||
const old = join(dir, 'old.txt'); writeAged(old, 'x', 40)
|
||||
chmodSync(unsafeParent, 0o777)
|
||||
const warn = vi.fn()
|
||||
const roots = await gatherSweepRoots(unsafeRoot, warn, join(root, 'missing-discovery-base'))
|
||||
expect(roots).toEqual([])
|
||||
expect(existsSync(old)).toBe(true)
|
||||
expect(warn).toHaveBeenCalledWith(expect.stringContaining('skipped unsafe root'))
|
||||
})
|
||||
|
||||
it('does not block activation but is awaited on disposal (quiescence)', async () => {
|
||||
const dir = sessionDir(root, 'sess-1')
|
||||
mkdirSync(dir, { recursive: true })
|
||||
|
|
@ -449,13 +517,13 @@ describe('startup cleanup sweep', () => {
|
|||
expect(existsSync(old)).toBe(false)
|
||||
})
|
||||
|
||||
it('a filesystem failure is contained (logged, never thrown) and does not fail a spill write', async () => {
|
||||
it('an unsafe root is contained (logged, never thrown)', async () => {
|
||||
const warn = vi.fn()
|
||||
// A path that is a FILE, not a directory: readdir(root) throws ENOTDIR. The
|
||||
// A path that is a FILE, not a directory, is not a valid cleanup root. The
|
||||
// sweep must log and return, never reject.
|
||||
const filePath = join(root, 'not-a-dir'); writeFileSync(filePath, 'x')
|
||||
await expect(sweepSpillRoots({ roots: [active(filePath)], cutoffMs: Date.now(), warn })).resolves.toBeUndefined()
|
||||
expect(warn).toHaveBeenCalledWith(expect.stringContaining('failed to read root'))
|
||||
expect(warn).toHaveBeenCalledWith(expect.stringContaining('skipped unsafe root'))
|
||||
})
|
||||
|
||||
it('contains an exception from the warning sink', async () => {
|
||||
|
|
@ -484,7 +552,7 @@ describe('discoverDefaultRoots', () => {
|
|||
writeFileSync(join(base, `${DEFAULT_ROOT_PREFIX}file01`), 'x') // matches shape but is a file
|
||||
symlinkSync(realRoot, join(base, `${DEFAULT_ROOT_PREFIX}link01`)) // matches shape but is a symlink
|
||||
const found = await discoverDefaultRoots(() => {}, base)
|
||||
expect(found).toEqual([realRoot])
|
||||
expect(found).toEqual([realpathSync(realRoot)])
|
||||
} finally {
|
||||
rmSync(base, { recursive: true, force: true })
|
||||
}
|
||||
|
|
|
|||
6
pnpm-lock.yaml
generated
6
pnpm-lock.yaml
generated
|
|
@ -7978,6 +7978,12 @@ importers:
|
|||
'@deepseek-ai/cordis':
|
||||
specifier: workspace:^
|
||||
version: link:../../../vendor/cordis
|
||||
'@deepseek-ai/cordis-plugin-include':
|
||||
specifier: workspace:^
|
||||
version: link:../../../vendor/include
|
||||
'@deepseek-ai/cordis-plugin-loader':
|
||||
specifier: workspace:^
|
||||
version: link:../../../vendor/loader
|
||||
'@deepseek-ai/dsh-brand':
|
||||
specifier: workspace:^
|
||||
version: link:../../util/brand
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue