Merge pull request #3455 from deepseek-harness/merge/projcache-v6-compat-into-master

sync: merge projection cache fix from 0.1.2-alpha.5 to master
This commit is contained in:
imccyu 2026-09-02 17:18:44 +08:00 • committed by GitHub
commit 0a1efddd40
34 changed files with 1526 additions and 84 deletions

View file

@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-19-projection-cache-per-session-files.md
2026-08-19-projection-cache-per-session-files.md: 0fd171649c1d8c7c3be7a8089287d43782b84714
2026-08-19-projection-cache-per-session-files.zh.md: 47decb598cc70233c287feab2dddff46bd7bcbc9
2026-08-19-projection-cache-per-session-files.md: c0ead55557e45905cfd368c70d826bf0c3eaade3
2026-08-19-projection-cache-per-session-files.zh.md: e5582d14d68c1aed76ef8aab08a8d575e8a1842d

View file

@ -20,8 +20,8 @@ Reads and writes share ONE coherent state: every read (`cachedSnapshot`) is a sy
- Listing is a synchronous in-memory read; a session without a record document simply lacks the projection column.
- ACP, headless, SDK, and Web sessions publish cache rows for later consumers. The log-leading durability barrier may flush a covered prefix at the cache cadence and split otherwise coalesced physical JSONL runs; recorded profile snapshots re-pack the logical event stream so cache timing does not define fixture layout.
- The per-record contract scopes failure: a malformed or stale-version document reads as an absent record at open, so one bad file never bricks the cache, and a checkpoint schema bump discards stale sessions per record instead of rejecting the whole domain.
- The json backend bootstraps the per-record tree from the legacy whole-unit cache only when enumeration finds no new-layout document path and the legacy unit name and version match the requested descriptor. A different version remains untouched and the new domain opens empty; storage never relabels its values as the current version. Any new document path, including an unreadable or stale file, suppresses the bootstrap for the whole unit; missing session rows refold from the log.
- The `session_projcache` domain uses version 6. Every version-5 record reads as absent, including a healthy one, so a poisoned version-5 record cannot fail domain validation. Session headers and event logs remain in session persistence; an exact read refolds them and writes a version-6 cache record, while zero-I/O listings lack that projection until the cache returns.
- The json backend bootstraps the per-record tree from the legacy whole-unit cache only when enumeration finds no new-layout document path, the legacy unit name matches, and its version is current or declared compatible. A version outside that accepted set remains untouched and the new domain opens empty; storage never relabels a version the domain owner did not approve. Any new document path, including an unreadable or stale file, suppresses the bootstrap for the whole unit; missing session rows refold from the log. The [cross-version read-compatibility decision](2026-09-02-projcache-cross-version-read-compat.md) owns the version policy.
- The `session_projcache` domain uses version 6 and declares versions 3, 4, and 5 compatible. Vouched-for records retain their listing projections across upgrades, absent lineage fields normalize to an unseeded identity, and a seeded caller rejects that identity and refolds cold. A record that still fails schema validation is backed up and skipped; every subsequent write stamps version 6.
- The cache record is bound to the same log lifecycle as before: the stored `{createdAt, cwd, isSeeded, inheritedEventCount}` identity guards against a recreated id or a mismatched inherited prefix.
## Alternatives considered
@ -30,4 +30,4 @@ Reads and writes share ONE coherent state: every read (`cachedSnapshot`) is a sy
- **Cache-owned per-session files** (`<root>/<session-id>/projection_cache.json`, the first revision of this change). Tried and reverted in review: the cache hand-rolled the medium — paths, per-path write chains, in-flight tracking, owner-only file modes, and a sqlite no-path special case — and its listing read hit the disk directly on every call while writes were throttled, so reads and writes were never consistent.
- **Resolve the path through `sessionPersistence.locate(meta)`** (the file beside the session log). Rejected: the cache would have to guess "beside the log" from a log artifact path (`dirname` + fixed filename), coupling the cache to the persistence service and to a backend's layout.
- **Make `per-record` a mode of the existing unit instead of a separate unit class.** Rejected: the two layouts have genuinely different state models — `single` is memory-authoritative with whole-file publish, `per-record` is stateless (the directory is the state; `loadAll` re-reads the tree) — so they are separate small classes behind one backend, with record keys validated path-safe instead of encoded.
- **Copy legacy values across unit versions.** Rejected: the json backend does not know a domain's record schema and cannot derive session-lineage fields. Copying raw values under the requested version relabels data without migrating it. A domain that requires compatibility owns an explicit migration; the projection cache instead discards old records and rebuilds them from session logs.
- **Copy legacy values across undeclared unit versions.** Rejected: the json backend does not know a domain's record schema and cannot derive session-lineage fields. It copies an older record only when the domain explicitly lists that version in `compatibleVersions` and its current schema accepts the value; otherwise the record stays untouched and reads as absent.

View file

@ -20,8 +20,8 @@ Status: implemented
- 列表读取是同步内存读;没有记录文档的会话只是缺少投影列。
- ACP、headless、SDK 与 Web 会话都会发布缓存行,供后续消费方使用。确保日志领先的持久性屏障可能按缓存节奏 flush 已覆盖的前缀,并拆分原本会合并的物理 JSONL 行;各 profile 的录制快照会重新 pack 逻辑事件流,因此缓存时序不会决定 fixture 布局。
- per-record 契约把故障范围缩小到单记录:畸形或过期版本的文档在打开时读作"无此记录",单个坏文件不会拖垮整个缓存;检查点 schema 升级按会话丢弃过期行,而不是拒绝整个域。
- json 后端仅在枚举时没有发现任何新布局文档路径,且旧单元名称和版本与请求的 descriptor 相同时,才从旧整单元缓存引导 per-record 目录树。版本不同时,旧文件保持不变,新域为空;存储不会把旧值改标为当前版本。只要存在任意新文档路径,即使文件不可读或版本陈旧,也会对整个单元禁用引导;缺失的会话行从日志重折叠。
- `session_projcache` 域使用版本 6。所有版本 5 记录都读作缺失,包括健康记录,因此被污染的版本 5 记录不能再使域校验失败。会话 header 和事件日志仍保存在会话持久化中;精确读取会重折叠这些数据并写入版本 6 缓存,而零 I/O 列表在缓存恢复前缺少对应投影。
- json 后端仅在枚举时没有发现任何新布局文档路径、旧单元名称匹配,且其版本为当前版本或已声明兼容版本时,才从旧整单元缓存引导 per-record 目录树。接受集合之外的版本保持不变,新域为空;存储绝不把域 owner 未批准的版本改标为当前版本。只要存在任意新文档路径,即使文件不可读或版本陈旧,也会对整个单元禁用引导;缺失的会话行从日志重折叠。[跨版本读兼容决策](2026-09-02-projcache-cross-version-read-compat.zh.md)是版本策略的权威说明。
- `session_projcache` 域使用版本 6,并声明版本 3、4、5 兼容。经背书的记录在升级后保留列表投影;缺失的 lineage 字段归一化为 unseeded 身份,seeded 调用方会拒绝该身份并回落冷折叠。仍然通不过 schema 校验的记录会被备份并跳过;后续每次写入都使用版本 6。
- 缓存记录仍绑定同一日志生命周期:存储的 `{createdAt, cwd, isSeeded, inheritedEventCount}` 身份防止被重建的 id 或不匹配的继承前缀误导。
## Alternatives considered
@ -30,4 +30,4 @@ Status: implemented
- **缓存自持的每会话文件**(`<root>/<session-id>/projection_cache.json`,本改动的第一版)。试过并在评审中回退:缓存手搓了介质——路径、按路径的写链、在途跟踪、仅属主文件权限,以及 sqlite 无路径特判——而且它的列表读每次调用都直读磁盘、写却在节流,读写永不一致。
- **经 `sessionPersistence.locate(meta)` 解析路径**(文件放在会话日志旁)。未采用:缓存得从日志 artifact 路径"猜"日志旁边(`dirname` + 固定文件名),把缓存耦合到持久化服务与后端的布局。
- **把 `per-record` 做成既有单元的一种模式而非独立单元类。** 未采用:两种布局的状态模型本质不同——`single` 内存权威、整文件发布;`per-record` 无状态(目录即状态,`loadAll` 重扫目录树)——所以它们是同一后端下的两个小型独立类,记录键做路径安全校验而非编码。
- **跨单元版本复制旧值。** 未采用:json 后端不知道域的记录 schema,也无法推导会话谱系字段。按请求版本复制原始值只会修改数据标签,不会迁移数据。需要兼容性的域负责显式迁移;投影缓存改为丢弃旧记录,并从会话日志重建。
- **跨未声明单元版本复制旧值。** 未采用:json 后端不知道域的记录 schema,也无法推导会话 lineage 字段。只有当域在 `compatibleVersions` 中明确列出旧版本,且当前 schema 接受该值时,后端才复制旧记录;否则记录保持不变并读作不存在。

View file

@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-02-projcache-cross-version-read-compat.md
2026-09-02-projcache-cross-version-read-compat.md: 1d157d90b04fe92e86d858a8e5424a5e517721f6
2026-09-02-projcache-cross-version-read-compat.zh.md: 3948126271a84810abf4fe1c1e6e874b279a4f5e

View file

@ -0,0 +1,74 @@
# Agent Note: Projection-cache cross-version read compatibility (session_projcache v3/v4/v5 → v6)
Status: implemented
English | [中文](2026-09-02-projcache-cross-version-read-compat.zh.md)
## Problem
The `session_projcache` storage domain evolved through three on-disk generations across published releases. An upgraded DSH_HOME failed in two ways:
- **A v3 single-file home bricked startup after the upgrade**: the per-record layout's legacy bootstrap migrated the old whole-unit file without checking its `unit.version`, stamping the old records with the current version into the new tree; the domain layer's per-record zod validation at open then hit the missing now-required fields → `invalid-record` → the whole domain refused to open → the plugin tree failed to load. And because the bootstrap writes before validation runs, **the first boot permanently wrote the bad documents into the new tree** ("poisoning") — every later boot saw a non-empty tree, never took the legacy path again, and the home stayed unusable.
- **A v4 per-record home lost its listing titles after the upgrade**: v4 documents were silently discarded by the version-stamp check (the per-record contract), and SessionList is a zero-I/O cache-only read, so a miss served the row without projections; titles only returned as each session was individually reopened.
The cache domain's own contract is "a stale or unreadable cache costs a longer tail replay, never a wrong value, never a refused load" — the hard failure and the wholesale discard each broke the first half of that contract or the product expectation.
## The three on-disk generations
| domain version | shipped in | layout | on-disk form | identity fields | row fields |
|---|---|---|---|---|---|
| 3 | 0.1.1-rc.2 | single | one file `storages/session_projcache.json` (`{unit:{name,version}, global, tables}`) | `createdAt`, `cwd?` | `ver`, `seq`, `val` |
| 4 | 0.1.2-alpha.3 | per-record | one file per session `storages/session_projcache/sessions/<sessionId>.json` (`{version, record}`) | `createdAt`, `cwd?` | same |
| 5 | 0.1.2-alpha.4 | per-record | same as v4 | + `isSeeded` (shipped required; now optional), `inheritedEventCount` (same) | same (`seq` numbers mean the same as v4; only type brands were added) |
The only substantive v4→v5 difference is the two new lineage identity fields; the `ver/seq/val` row shape is identical across all three generations, and `seq` numbering did not change ([the 2026-08-31 seq/offset brands note](2026-08-31-session-sequence-and-log-offset-brands.md) pins the on-disk numbers as unchanged). v3→v4 was a layout migration with identical record content.
One derived shape also exists: a v3 home that ran the v5 build once (the poisoned state) — its new tree holds documents **stamped 5 whose content is a v3 record** (no lineage fields).
## Decision
Declared read compatibility — reads tolerate vouched-for older versions, writes always stamp the current one:
1. **`DomainSpec.compatibleVersions` (new, optional)**: the domain owner declares "records stored under these older versions are also readable under the current record schemas" (typically by declaring the fields old records lack as optional). `defineDomain` validates each entry as a non-negative integer below the current version; `descriptorOf` projects the set onto the backend `KvUnitDescriptor`.
2. **json backend per-record reads** accept version stamps in "current ∪ compatibleVersions"; anything outside the set is still discarded as foreign. **The write path always stamps the current version** (the first checkpoint after reading an old record naturally advances it). The `single` layout stays exact-version.
3. **Legacy-bootstrap version gate (the actual bug fix)**: the old whole-unit file's `unit.version` must fall inside the accepted set to be migrated; otherwise the file is left alone and the unit reads empty — stamping records the owner never vouched for turns a discardable stale cache into hard schema failures at the domain layer.
4. **The projcache domain declares `version: 6, compatibleVersions: [3, 4, 5]`**, and the two lineage fields become `.optional()`. The single reader of stored identities, `identityMatches`, normalizes absence to the unseeded lineage (`?? false` / `?? 0`): exact for unforked sessions, while a forked session's expectation is seeded → natural mismatch → discard and cold rebuild, so the lineage binding loses none of its protection.
5. **The poisoned state self-heals**: documents stamped 5 without lineage fields are declared compatible and parse under the optional schema (their content is the real pre-upgrade cache data), so the home boots again and titles serve immediately.
6. **Schema-validation backstop: `invalidRecords: 'backup-and-skip'` (declared by this domain only)**. A stored record that still fails to parse beyond read compatibility no longer refuses the whole domain: the domain layer calls the backend's `KvUnit.backupRecord` (json per-record implementation = rename the document to `<key>.json.bak.<YYYYMMDDHHmm>`, bytes kept, never read again), prints the concrete failure with `logger.error` (domain, table, key, destination, zod cause), and continues the open with the record absent; the next cold read rebuilds and rewrites that session's cache. **The policy is an explicit per-domain declaration and the default stays fail-loud** — other domains still refuse the whole load on invalid stored data, and a backend without `backupRecord` (single layout, row stores) also falls back to fail-loud. Naming history: quarantine → backup-and-skip (user ruling: the word must carry both "back up" and "skip", sharing its root with the `.bak` suffix; skip-backup was rejected because the CLI `--skip-X` convention reads it as "do not back up"). For this domain it supersedes the reset/destroy recovery path of the [2026-07-28 storage recovery proposal](../../proposed/architecture/2026-07-28-storage-root-and-derived-medium-recovery.md), which stays live for authoritative and whole-medium damage.
### v5 → v6 compatibility
Version 6 changes only the current write stamp and keeps the v5 record schema. `compatibleVersions: [3, 4, 5]` therefore admits both healthy v5 records and v5-stamped lineage-less records produced by the faulty bootstrap. The current schema accepts absent lineage; `identityMatches` interprets it as unseeded and rejects the record for a seeded session. The next successful checkpoint rewrites an accepted v5 record with a v6 stamp and complete lineage. No separate v5→v6 rewrite runs at startup: an unaccepted version reads as absent, while a schema-invalid accepted record follows `backup-and-skip`.
### Upgrade matrix
| home shape | behavior after the fix |
|---|---|
| v3 single-file (not poisoned) | bootstrap migrates (3 ∈ accepted set) → titles serve immediately |
| v3 + poisoned new tree | new-tree documents read directly (optional tolerance) → boot restored, titles serve immediately |
| v4 per-record | documents read directly (4 ∈ accepted set) → titles serve immediately |
| v5 healthy | documents read directly (5 ∈ accepted set) → titles serve immediately |
| v6 current | unaffected |
| old records of forked (seeded) sessions | identity mismatch → discarded, cold rebuild when the session opens (safe side) |
## Alternatives considered
- **Discard-and-rebuild only** (bootstrap gate without compatible versions): fixes the boot, but every SessionList title is lost after the upgrade until each session is reopened — fails the upgrade-and-go product requirement.
- **Schema `.default()` fills**: behaviorally equivalent to optional + reader normalization, but bakes the "absent = unseeded" interpretation into the durable schema's output type; ruled for optional — the schema honestly describes every accepted on-disk shape and the interpretation lives at the consumer (user ruling, 2026-09-02).
- **Roll the domain version back to 4**: a small diff, but breaks version monotonicity, depends on the "bootstrap skips no versions" bug itself, and drops every poisoned and healthy v5 home's cache.
## Consequences
- A deployment routing this domain to the sqlite backend gets none of the tolerance: sqlite implements neither `compatibleVersions` nor `backupRecord`, so behavior degrades to the old strict-version semantics (a whole-unit version mismatch still refuses with `version-mismatch`; nothing loosens, nothing serves wrong values). Shipped compositions route this domain to json, so this stays a deployment-configuration risk only.
- The optional lineage fields let accepted records omit lineage: a lineage-less record decodes as unseeded. The identity match still refuses it for seeded callers, and the per-row `ver` guard still screens every value, so the residual exposure is an unseeded caller reading an unseeded-shaped record — the same trust extended to genuine pre-lineage records.
- `backupRecord` overwrites a same-minute backup of the same key (the newer bytes win); distinct minutes and distinct keys never collide.
## Testing
- `storage-json` unit tests: compat-stamped reads / out-of-set discards / writes stamping current; legacy bootstrap migrating only accepted versions (including the migrated-documents-stamp-current assertion); `backupRecord` move / absent read / rewrite / closed guard.
- `storage-domain` unit tests: `compatibleVersions` / `invalidRecords` declaration validation; backup-and-skip falling back to fail-loud when the backend has no `backupRecord`.
- `session-projection-cache` unit tests: records without lineage fields serve unseeded sessions verbatim and are discarded for seeded ones.
- **Archived-fixture recovery tests** (`tests/fixtures.spec.ts` + `tests/fixtures/`): four media archives produced by the real released builds — `v3-single-unit.json` (the 0.1.1-rc.2 whole-unit file), `v4-session-doc.json` (0.1.2-alpha.3), `v5-session-doc.json` (0.1.2-alpha.4), `v5-lineageless-doc.json` (the unguarded bootstrap's poisoned shape, synthesized from the v3 record) — each opened through the real storage stack, asserting the listing serves the archived title and that a live write rewrites the document to the current version (v6 stamp + lineage fields + fresh value); plus the backup-and-skip behavior for a schema-failing record (boot survives, `.bak` lands, log is concrete, neighbor records unharmed).
- End-to-end acceptance, executed against the real release artifacts: the published 0.1.1-rc.2 and 0.1.2-alpha.3 npm builds seeded homes through their own web apps (model turns plus a rename RPC), the published 0.1.2-alpha.4 build reproduced both failures (including the poisoned tree), and the fixed build served every home shape — pristine v3, poisoned v3, v4, and fresh — with the SessionList RPC returning the recorded titles verbatim.
Future bump procedure: when a new version's shape can tolerate old records through "optional fields + reader normalization", add the old version to `compatibleVersions`; otherwise bump normally (discard and rebuild) and remove the no-longer-compatible versions from the set. Either way, the package README requires the bump to land with archived fixtures and tests proving the chosen disposition.

View file

@ -0,0 +1,74 @@
# Agent Note: 投影缓存跨版本读兼容(session_projcache v3/v4/v5 → v6)
Status: implemented
[English](2026-09-02-projcache-cross-version-read-compat.md) | 中文
## 问题
`session_projcache` 存储域在已发布版本间演进了三代磁盘结构。升级后的 DSH_HOME 出现两类故障:
- **v3 单文件 home 升级后启动硬失败**:per-record 布局的 legacy bootstrap 迁移旧单文件时不检查其 `unit.version`,把旧记录原样打上当前版本戳写入新树;domain 层开域时逐条 zod 校验,旧记录缺新增必填字段 → `invalid-record` → 整个域拒开 → 插件树加载失败。且 bootstrap 先写盘后校验,**首次启动即把坏文档永久写入新树**("投毒")——此后每次启动新树非空、连 legacy 路径都不再走,home 持续不可用。
- **v4 per-record home 升级后列表丢标题**:v4 文档被版本戳检查静默丢弃(per-record 契约),SessionList 是零 I/O 纯缓存读,miss 后整行不带投影;标题要等每个会话被逐个重新打开后才恢复。
缓存域自身的契约是"过期或不可读的缓存只付出更长的尾部重放,绝不给出错值、绝不拒载"——硬失败与整体丢弃都违背该契约的前半句或产品预期。
## 三代磁盘结构差异
| domain version | 携带发布 | 布局 | 磁盘形态 | identity 字段 | 行字段 |
|---|---|---|---|---|---|
| 3 | 0.1.1-rc.2 | single | 单文件 `storages/session_projcache.json`(`{unit:{name,version}, global, tables}`) | `createdAt`, `cwd?` | `ver`, `seq`, `val` |
| 4 | 0.1.2-alpha.3 | per-record | 每会话一份 `storages/session_projcache/sessions/<sessionId>.json`(`{version, record}`) | `createdAt`, `cwd?` | 同上 |
| 5 | 0.1.2-alpha.4 | per-record | 同 v4 | + `isSeeded`(v5 首发必填;现为 optional)、`inheritedEventCount`(同前) | 同上(`seq` 数值语义与 v4 相同,仅类型加 brand) |
v4→v5 的唯一实质差异是 identity 新增两个 lineage 字段;行内 `ver/seq/val` 三代一致,`seq` 的数值含义未变([2026-08-31 seq/offset brands note](2026-08-31-session-sequence-and-log-offset-brands.zh.md) 明确 on-disk 数值不变)。v3→v4 是布局迁移,记录内容结构一致。
另有一种衍生形态:跑过一次 v5 版本的 v3 home(投毒态)——新树里存在**版本戳为 5 但内容是 v3 记录**(缺 lineage 字段)的文档。
## 决策
声明式读兼容——读容忍 owner 背书过的旧版本,写恒戳当前版本:
1. **`DomainSpec.compatibleVersions`(新增,可选)**:域 owner 声明"这些旧版本的存量记录在当前记录 schema 下也可读"(典型手段:新增字段标 optional)。`defineDomain` 校验各项为小于当前 version 的非负整数;`descriptorOf` 透传到后端 `KvUnitDescriptor`。
2. **json 后端 per-record 读**:接受"当前版本 ∪ compatibleVersions"内的版本戳,集合外照旧视为 foreign 丢弃;**写路径永远戳当前版本**(读到旧记录后的下一次 checkpoint 自然把它推进到当前版本)。single 布局维持 exact-version 不变。
3. **legacy bootstrap 版本把关(bug 修复本体)**:旧单文件的 `unit.version` 必须落在接受集合内才迁移,否则视为空 unit 留在原地——为 owner 未背书的记录打当前版本戳,会把"可丢弃的过期缓存"变成 domain 层的 schema 硬失败。
4. **projcache 域声明 `version: 6, compatibleVersions: [3, 4, 5]`**;两个 lineage 字段改为 `.optional()`。唯一消费 stored identity 的读点 `identityMatches` 把缺失归一化为 unseeded lineage(`?? false` / `?? 0`):对非 fork 会话这是精确值;fork 会话的 expected 是 seeded → 天然 mismatch → 丢弃冷读重建,lineage 绑定的防护不放松。
5. **投毒态自愈**:v5 戳缺 lineage 字段的文档被声明为兼容,并由 optional schema 接受(内容本就是升级前的真实缓存数据),home 恢复可启动且标题立即可服务。
6. **schema 校验兜底:`invalidRecords: 'backup-and-skip'`(仅本域声明)**。读兼容之外仍然解析失败的存量记录不再让整个域拒开:domain 层调用后端的 `KvUnit.backupRecord`(json per-record 实现=把文档改名为 `<key>.json.bak.<YYYYMMDDHHmm>`,字节留档、不再被读取),用 `logger.error` 打印具体失败信息(域名、表、键、移动去向、zod 失败原因),随后当该记录不存在继续启动;下一次冷读会重建并重写该会话的缓存。**该策略是域级显式声明,缺省仍为 fail-loud**——其他业务域的存量数据校验失败照旧整域拒载;后端没有 `backupRecord` 能力(single 布局、行存储)时也回退 fail-loud。命名沿革:quarantine → backup-and-skip(用户裁决:词要同时含"备份"与"跳过"两义,且与 `.bak` 后缀同源;skip-backup 因 CLI `--skip-X` 惯例存在"不备份"反读而弃用)。对本域而言,该策略取代了 [2026-07-28 存储恢复提案](../../proposed/architecture/2026-07-28-storage-root-and-derived-medium-recovery.zh.md)中 reset/destroy 的恢复途径;该提案对权威介质与整介质损坏仍然有效。
### v5 → v6 兼容方式
版本 6 只改变当前写入的版本戳,沿用 v5 记录 schema。`compatibleVersions: [3, 4, 5]` 因此同时接受健康的 v5 记录,以及错误 bootstrap 生成的 v5 戳、缺 lineage 记录。当前 schema 允许 lineage 缺失;`identityMatches` 将其解释为 unseeded,并对 seeded 会话拒绝该记录。下一次成功的 checkpoint 会用 v6 戳和完整 lineage 重写已接受的 v5 记录。启动时不单独运行 v5→v6 重写:未接受的版本读作不存在,schema 校验失败的已接受记录则执行 `backup-and-skip`。
### 升级矩阵
| home 形态 | 修复后行为 |
|---|---|
| v3 单文件(未投毒) | bootstrap 迁移(3 ∈ 接受集)→ 标题立即可服务 |
| v3 + 投毒新树 | 新树文档直接读入(optional 容忍)→ 启动恢复、标题立即可服务 |
| v4 per-record | 文档直接读入(4 ∈ 接受集)→ 标题立即可服务 |
| v5 正常 | 文档直接读入(5 ∈ 接受集)→ 标题立即可服务 |
| v6 当前版本 | 不受影响 |
| fork(seeded)会话的旧记录 | identity mismatch → 丢弃,打开会话时冷读重建(安全侧) |
## 备选方案
- **只丢弃重建**(bootstrap 把关但不声明兼容版本):启动可修,但升级后 SessionList 标题全丢、要逐会话打开才恢复——不满足升级即用的产品要求。
- **schema `.default()` 填缺省**:行为与 optional+读点归一化等价,但把"缺失=unseeded"的解释固化进 durable schema 的输出类型;拍板为 optional——schema 如实描述介质上所有被接受的形态,解释权在消费点(2026-09-02 用户裁决)。
- **域版本回退到 4**:改动很小,但破坏版本单调性、依赖"bootstrap 不查版本"这个 bug 本身、且投毒态与正常 v5 home 的缓存全被丢弃。
## 影响
- 部署方若把本域路由到 sqlite 后端,得不到任何容忍能力:sqlite 既未实现 `compatibleVersions` 也没有 `backupRecord`,行为退化为原有的严格版本语义(整 unit 版本不匹配仍 `version-mismatch` 拒开;不放松、不出错值)。shipped 组合固定路由 json,此风险仅存在于部署配置层面。
- optional lineage 字段允许被接受的记录缺少 lineage:无 lineage 的记录会解码为 unseeded。身份比对仍会对 seeded 调用方拒收,逐行 `ver` 守卫仍筛查每个值,残余暴露面只是 unseeded 调用方读到 unseeded 形态的记录——与真实 pre-lineage 记录享有的信任完全相同。
- `backupRecord` 对同一键的同一分钟内重复备份会覆盖前一份(新字节胜出);不同分钟、不同键永不冲突。
## 测试
- `storage-json` 单测:compat 版本戳读入/集合外丢弃/写恒当前版本;legacy bootstrap 仅在版本被接受时迁移(含迁移后文档戳当前版本断言);`backupRecord` 移档/读缺席/重写/封闭守卫。
- `storage-domain` 单测:`compatibleVersions`/`invalidRecords` 声明校验;后端无 `backupRecord` 时 backup-and-skip 回退 fail-loud。
- `session-projection-cache` 单测:缺 lineage 字段的记录对 unseeded 会话按原值服务、对 seeded 会话丢弃。
- **归档 fixtures 独立恢复测试**(`tests/fixtures.spec.ts` + `tests/fixtures/`):真实发布物产出的四份介质存档——`v3-single-unit.json`(0.1.1-rc.2 整域单文件)、`v4-session-doc.json`(0.1.2-alpha.3)、`v5-session-doc.json`(0.1.2-alpha.4)、`v5-lineageless-doc.json`(无守卫 bootstrap 的投毒形态,由 v3 记录合成)——逐一走真实存储栈开域,断言列表读出归档标题、且 live 写把文档重写为当前版本(v6 戳 + lineage 字段 + 新值);外加 schema 失败记录的 backup-and-skip 行为(启动不失败、`.bak` 落盘、日志具体、邻居记录不受累)。
- 端到端验收,以真实发布物执行:已发布的 0.1.1-rc.2 与 0.1.2-alpha.3 npm 包经各自 web app 造数(真实模型对话 + rename RPC),已发布的 0.1.2-alpha.4 包复现两类故障(含投毒树),修复后构建对纯净 v3、投毒 v3、v4、全新四种 home 形态经 SessionList RPC 原样返回记录在案的标题。
未来 bump 流程:新版本结构若可用"optional 字段 + 读点归一化"容忍旧记录,就把旧版本加入 `compatibleVersions`;否则正常 bump(丢弃重建),并把不再兼容的版本从集合中移除。无论哪条路,包 README 都要求 bump 随附归档 fixture 和论证所选处置方式的测试。

View file

@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/proposed/architecture/2026-07-28-storage-root-and-derived-medium-recovery.md
2026-07-28-storage-root-and-derived-medium-recovery.md: 1505be1c58d5cf829327b2919113bb2e42798ce7
2026-07-28-storage-root-and-derived-medium-recovery.zh.md: 1bab5ab663df1419cc826c3d6acb59bd8bff7de0
2026-07-28-storage-root-and-derived-medium-recovery.md: 68a6792c38a8fe097558de3d5a857ab4ea0d3533
2026-07-28-storage-root-and-derived-medium-recovery.zh.md: 0c712b3b928f4b3df44e706b154c3a90a784403c

View file

@ -10,7 +10,7 @@ The persisted projection cache ([note](2026-07-27-session-projection-and-command
**Where the files actually live (root mismatch closed; resolve-once residual still open).** The shared base defaults the session store to the global harness home (`$DSH_HOME/sessions`, default `~/.dsh/sessions`), while the shipped Web overlay used to give the json backend the relative root `./.storages`: `workspace.json` and `session_projcache.json` landed under `<launch dir>/.storages/` — two launches from different directories shared their sessions yet saw different workspace registries and different projection caches, and the cache exists precisely to serve the cross-session cold listing, which missed for every session last cached under another launch directory. That mismatch is now closed: the overlay anchors `storage-json.root` to `$DSH_HOME/storages` with the same `!!js` expression the session root uses (`apps/cli/config/web.cordis.yml`). The residual hazard: `JsonStorageBackend` still never resolves its root — each unit open joins the path against whatever `process.cwd()` is at that moment (packages/storage/storage-json/src/index.ts); the shipped overlay root is already absolute and unaffected, but any relative root (bare Loader boots, tests) still splits on a later cwd change — the exact hazard the JSONL session backend resolves-once to prevent ("later process.cwd() changes cannot split one backend across roots", packages/session/session-persistence-jsonl/src/index.ts).
**Recovery behavior.** Inside a healthy medium the cache is fully self-healing by design: a `stateVersion`-mismatched row is discarded and refolded, a log shrunk below a row's watermark is detected by the anchored restore floor and answered with one full re-read, and every background write is fail-soft. But at the *medium* level there is no recovery at all: a truncated, hand-edited, or version-bumped `session_projcache.json` fails `openJsonUnit` with `malformed-medium`/`version-mismatch` (packages/storage/storage-json/src/format.ts), a schema-drifted record fails domain open with `invalid-record` (packages/storage/storage-domain/src/index.ts), the rejection propagates through `SessionProjectionCache[Service.init]`, and under the CLI's fail-loud boot the assembly refuses to start. A file whose entire content is rebuildable from session logs can brick boot. This contradicts the cache package's own stated stance ("a stale or unreadable cache costs a longer tail replay, never a wrong value") and the cache domain spec's JSDoc ("version bumps discard the whole medium"), which describes an aspiration, not the implementation. Partially superseded for the projection cache: [the per-session cache files note](../../implemented/architecture/2026-08-19-projection-cache-per-session-files.md) removed the global `session_projcache` domain, so the cache half of this proposal (recovery on that domain) no longer applies; the `workspace.json` half remains current. The same fail-loud path is *correct* for `workspace.json` — workspace records are authoritative, not derivable — so the missing concept is a per-domain declaration of authority, not a global behavior change.
**Recovery behavior.** Inside a healthy medium the cache is fully self-healing by design: a `stateVersion`-mismatched row is discarded and refolded, a log shrunk below a row's watermark is detected by the anchored restore floor and answered with one full re-read, and every background write is fail-soft. But at the *medium* level there is no recovery at all: a truncated, hand-edited, or version-bumped `session_projcache.json` fails `openJsonUnit` with `malformed-medium`/`version-mismatch` (packages/storage/storage-json/src/format.ts), a schema-drifted record fails domain open with `invalid-record` (packages/storage/storage-domain/src/index.ts), the rejection propagates through `SessionProjectionCache[Service.init]`, and under the CLI's fail-loud boot the assembly refuses to start. A file whose entire content is rebuildable from session logs can brick boot. This contradicts the cache package's own stated stance ("a stale or unreadable cache costs a longer tail replay, never a wrong value") and the cache domain spec's JSDoc ("version bumps discard the whole medium"), which describes an aspiration, not the implementation. Partially superseded for the projection cache: [the per-session cache files note](../../implemented/architecture/2026-08-19-projection-cache-per-session-files.md) removed the global `session_projcache` domain, so the cache half of this proposal (recovery on that domain) no longer applies; the `workspace.json` half remains current. The `invalid-record` class for the per-record projection cache is now also superseded: the shipped domain declares `invalidRecords: 'backup-and-skip'` ([cross-version read-compat note](../../implemented/architecture/2026-09-02-projcache-cross-version-read-compat.md)), which backs the failing record up and skips it at open, so the reset/destroy proposal below stays relevant only for whole-medium damage on authoritative or single-document media. The same fail-loud path is *correct* for `workspace.json` — workspace records are authoritative, not derivable — so the missing concept is a per-domain declaration of authority, not a global behavior change.
## Proposal

View file

@ -10,7 +10,7 @@ Status: proposed
**文件到底存在哪(根错位已收口,resolve-once 残余仍开放)。** 共享 base 将会话存储默认为全局 harness home(`$DSH_HOME/sessions`,默认 `~/.dsh/sessions`),而出厂 Web overlay 曾给 json 后端相对根 `./.storages`:`workspace.json` 和 `session_projcache.json` 落在 `<启动目录>/.storages/` 下——从两个不同目录启动,会话相同,工作区注册表和投影缓存却各是一份,而缓存存在的意义恰恰是跨会话冷列表,凡上次在别的启动目录下缓存过的会话全部 miss。这一错位已消除:overlay 现以与会话根同一段 `!!js` 表达式把 `storage-json.root` 锚定到 `$DSH_HOME/storages`(`apps/cli/config/web.cordis.yml`)。残余隐患:`JsonStorageBackend` 仍从不 resolve 根——每次打开 unit 都把路径 join 到当时的 `process.cwd()` 上(packages/storage/storage-json/src/index.ts);出厂 overlay 的根已是绝对路径不受影响,但任何相对根(裸 Loader 启动、测试)仍会被后续 cwd 变化劈开,JSONL 会话后端用「构造时 resolve 一次」防住的正是它("later process.cwd() changes cannot split one backend across roots",packages/session/session-persistence-jsonl/src/index.ts)。
**恢复行为。** 在健康介质内部,缓存按设计完全自愈:`stateVersion` 不匹配的行被丢弃重折,日志缩短到行水位以下由带锚的 restore floor 检出并以一次全量重读回答,每次后台写都是 fail-soft。但在*介质*层面完全没有恢复:被截断、被手改或版本被 bump 的 `session_projcache.json` 会让 `openJsonUnit` 以 `malformed-medium`/`version-mismatch` 失败(packages/storage/storage-json/src/format.ts),schema 漂移的记录让域 open 以 `invalid-record` 失败(packages/storage/storage-domain/src/index.ts),拒绝一路穿过 `SessionProjectionCache[Service.init]`,在 CLI 的 fail-loud 启动下整个组装拒绝启动。一个内容完全可从会话日志重建的文件能把启动搞死。这与缓存包自己声明的立场("a stale or unreadable cache costs a longer tail replay, never a wrong value")和缓存域 spec 的 JSDoc("version bumps discard the whole medium")相矛盾——后者描述的是愿望而非实现。投影缓存半边已被[每会话缓存文件 note](../../implemented/architecture/2026-08-19-projection-cache-per-session-files.zh.md) 部分取代:全局 `session_projcache` domain 已移除,本提案的缓存恢复半边不再适用;`workspace.json` 半边仍然有效。同一条 fail-loud 路径对 `workspace.json` 却是*正确*的——工作区记录是权威数据,不可派生——所以缺的概念是按域声明权威性,而不是全局改行为。
**恢复行为。** 在健康介质内部,缓存按设计完全自愈:`stateVersion` 不匹配的行被丢弃重折,日志缩短到行水位以下由带锚的 restore floor 检出并以一次全量重读回答,每次后台写都是 fail-soft。但在*介质*层面完全没有恢复:被截断、被手改或版本被 bump 的 `session_projcache.json` 会让 `openJsonUnit` 以 `malformed-medium`/`version-mismatch` 失败(packages/storage/storage-json/src/format.ts),schema 漂移的记录让域 open 以 `invalid-record` 失败(packages/storage/storage-domain/src/index.ts),拒绝一路穿过 `SessionProjectionCache[Service.init]`,在 CLI 的 fail-loud 启动下整个组装拒绝启动。一个内容完全可从会话日志重建的文件能把启动搞死。这与缓存包自己声明的立场("a stale or unreadable cache costs a longer tail replay, never a wrong value")和缓存域 spec 的 JSDoc("version bumps discard the whole medium")相矛盾——后者描述的是愿望而非实现。投影缓存半边已被[每会话缓存文件 note](../../implemented/architecture/2026-08-19-projection-cache-per-session-files.zh.md) 部分取代:全局 `session_projcache` domain 已移除,本提案的缓存恢复半边不再适用;`workspace.json` 半边仍然有效。per-record 投影缓存的 `invalid-record` 一类如今也已被取代:已发布的域声明了 `invalidRecords: 'backup-and-skip'`([跨版本读兼容 note](../../implemented/architecture/2026-09-02-projcache-cross-version-read-compat.zh.md)),open 时把失败记录备份后跳过,因此下文 reset/destroy 提案仅对权威介质或单文档介质的整介质损坏仍然相关。同一条 fail-loud 路径对 `workspace.json` 却是*正确*的——工作区记录是权威数据,不可派生——所以缺的概念是按域声明权威性,而不是全局改行为。
## 提案

View file

@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/subsystems/storage.md
storage.md: 1e4141e6ef1c6f8e1c2593e21e788b626d6b1ed7
storage.zh.md: f0433c600674741c3de0ce3e99430297839ce124
storage.md: 03b0fa1c674064994039d37c4fec906dc45794b3
storage.zh.md: ded8783fbfe7f0d7d8a846c7a66d780b48de1445

View file

@ -44,7 +44,7 @@ interface StorageBackend {
}
```
A backend owns one medium (a file-tree root, a database file) and exposes optional operation groups; `kv` is the only shipped group. `KvFacet.open(descriptor)` opens one named unit — `KvUnitDescriptor` carries the name, format version, table names, and whether a global singleton slot exists — and returns a `KvUnit` with `loadAll`, `putRecord`, `deleteRecord`, `setGlobal`, and `close`. Unit and table names must match `UNIT_NAME_RE` (safe as a file name and as a SQL identifier segment); record keys are arbitrary strings that never reach file paths. A unit does not serialize concurrent writes — ordering belongs to the caller — but each single call is atomic on the medium and durable once resolved. A medium stamped with a different version rejects `version-mismatch`; one that cannot be parsed as the unit rejects `malformed-medium` (no migration, pre-release stance). [`backend.ts`](../../packages/storage/storage/src/backend.ts) is the normative clause-by-clause contract, and the shared conformance suite in [`tests/contract.ts`](../../packages/storage/storage/tests/contract.ts) checks every clause against each backend. The [json backend](../../packages/storage/storage-json/README.md) republishes one whole human-readable file per unit atomically; the [sqlite backend](../../packages/storage/storage-sqlite/README.md) stores one document per row in one database for frequently updated data.
A backend owns one medium (a file-tree root, a database file) and exposes optional operation groups; `kv` is the only shipped group. `KvFacet.open(descriptor)` opens one named unit — `KvUnitDescriptor` carries the name, current format version, optional compatible record versions, table names, and whether a global singleton slot exists — and returns a `KvUnit` with `loadAll`, `putRecord`, `deleteRecord`, `setGlobal`, and `close`. Unit and table names must match `UNIT_NAME_RE` (safe as a file name and as a SQL identifier segment); record keys are arbitrary strings that never reach file paths. A unit does not serialize concurrent writes — ordering belongs to the caller — but each single call is atomic on the medium and durable once resolved. A `single` medium stamped with a different version rejects `version-mismatch`; a `per-record` document stamped outside the accepted set reads as absent. A medium that cannot be parsed as the unit rejects `malformed-medium`. [`backend.ts`](../../packages/storage/storage/src/backend.ts) is the normative clause-by-clause contract, and the shared conformance suite in [`tests/contract.ts`](../../packages/storage/storage/tests/contract.ts) checks every clause against each backend. The [json backend](../../packages/storage/storage-json/README.md) republishes one whole human-readable file per unit atomically; the [sqlite backend](../../packages/storage/storage-sqlite/README.md) stores one document per row in one database for frequently updated data.
## Declaring a domain
@ -55,16 +55,36 @@ A domain is declared once by its owning package as a spec object — the single
interface DomainSpec {
/** Domain name; must match `UNIT_NAME_RE` (doubles as the backend unit name). */
readonly name: string
/** Domain format version; a medium stamped with a different version rejects at open. */
/** Current domain format version; reads enforce it according to the selected layout. */
readonly version: number
/**
* Medium layout for the backend unit: `single` (the default) stores the
* whole unit as one document; `per-record` stores each record as its own
* document, for units whose records are large, sparse, or individually
* disposable — the projection cache — and scopes version bumps per record
* (a stale record document is discarded, never migrated).
* disposable — the projection cache — and scopes version checks per record
* (an unaccepted record document is discarded, never migrated).
*/
readonly layout?: 'single' | 'per-record'
/**
* Older domain versions whose stored records the current record schemas
* also accept (the declaring owner vouches for that, typically by
* declaring the fields older records lack as optional). `per-record` backends
* read documents stamped with a listed version instead of discarding them,
* and accept a legacy whole-unit file so stamped for the one-time
* bootstrap; writes always stamp {@link version}.
*/
readonly compatibleVersions?: readonly number[]
/**
* What `open` does with a stored table record that fails its zod schema.
* Absent (the default), the whole open rejects with `invalid-record` —
* right for authoritative data. `'backup-and-skip'` is for domains whose
* records are disposable derived data: the backend moves the record's
* document aside (`KvUnit.backupRecord`), the failure is logged with
* its cause, and the open continues with the record absent. A backend
* without `backupRecord` (no per-record document to move) falls back
* to the rejecting default. The global slot always rejects.
*/
readonly invalidRecords?: 'backup-and-skip'
/** Optional global singleton slot. */
readonly global?: DomainGlobalSpec<unknown>
/** Table declarations keyed by table name; each name must match `UNIT_NAME_RE`. */
@ -180,7 +200,10 @@ The mounted domain facility. Opens declared domains over routed backends; one fa
* (`facet-unsupported`); open the unit projected from the spec (backend
* `version-mismatch`/`malformed-medium` pass through); load and validate
* every stored record against the spec's zod schemas (`invalid-record`
* with the offending table and key); construct the domain.
* with the offending table and key — unless the spec declares
* `invalidRecords: 'backup-and-skip'` and the unit can move documents aside, in
* which case the failing record is backed up, logged, and skipped);
* construct the domain.
*
* Lifecycle: the CALLER owns the returned handle and closes it via
* `Domain.close()` (typically as its own `ctx.effect` disposer) — the

View file

@ -44,7 +44,7 @@ interface StorageBackend {
}
```
一个后端拥有一个介质(一棵文件树的根目录、一个数据库文件),并提供可选的操作组;`kv` 是唯一已交付的操作组。`KvFacet.open(descriptor)` 打开一个具名 unit——`KvUnitDescriptor` 携带名称、格式版本、表名清单,以及是否存在全局单例 slot——并返回提供 `loadAll`、`putRecord`、`deleteRecord`、`setGlobal` 和 `close` 的 `KvUnit`。unit 名与表名必须匹配 `UNIT_NAME_RE`(既可安全用作文件名,也可安全用作 SQL 标识符片段);记录键是任意字符串,绝不进入文件路径。unit 不对并发写入做串行化——顺序由调用方负责——但每次单独调用在介质上都是原子的,且 resolve 后即已持久。介质上记录的版本与之不同时拒绝 `version-mismatch`;无法按该 unit 解析的介质拒绝 `malformed-medium`(不做迁移:预发布立场)。[`backend.ts`](../../packages/storage/storage/src/backend.ts) 是逐条款的规范性约定,[`tests/contract.ts`](../../packages/storage/storage/tests/contract.ts) 中的共享一致性套件会针对每个后端检查每项条款。[json 后端](../../packages/storage/storage-json/README.zh.md)以原子方式为每个 unit 整文件重新发布一份人类可读文件;[sqlite 后端](../../packages/storage/storage-sqlite/README.zh.md)在单个数据库中每行存储一份文档,用于频繁更新的数据。
一个后端拥有一个介质(一棵文件树的根目录、一个数据库文件),并提供可选的操作组;`kv` 是唯一已交付的操作组。`KvFacet.open(descriptor)` 打开一个具名 unit——`KvUnitDescriptor` 携带名称、当前格式版本、可选的兼容记录版本、表名清单,以及是否存在全局单例 slot——并返回提供 `loadAll`、`putRecord`、`deleteRecord`、`setGlobal` 和 `close` 的 `KvUnit`。unit 名与表名必须匹配 `UNIT_NAME_RE`(既可安全用作文件名,也可安全用作 SQL 标识符片段);记录键是任意字符串,绝不进入文件路径。unit 不对并发写入做串行化——顺序由调用方负责——但每次单独调用在介质上都是原子的,且 resolve 后即已持久。`single` 介质上记录的版本不同时拒绝 `version-mismatch`;`per-record` 文档的版本在接受集合之外时读作不存在。无法按该 unit 解析的介质拒绝 `malformed-medium`。[`backend.ts`](../../packages/storage/storage/src/backend.ts) 是逐条款的规范性约定,[`tests/contract.ts`](../../packages/storage/storage/tests/contract.ts) 中的共享一致性套件会针对每个后端检查每项条款。[json 后端](../../packages/storage/storage-json/README.zh.md)以原子方式为每个 unit 整文件重新发布一份人类可读文件;[sqlite 后端](../../packages/storage/storage-sqlite/README.zh.md)在单个数据库中每行存储一份文档,用于频繁更新的数据。
## 声明领域
@ -55,16 +55,36 @@ interface StorageBackend {
interface DomainSpec {
/** Domain name; must match `UNIT_NAME_RE` (doubles as the backend unit name). */
readonly name: string
/** Domain format version; a medium stamped with a different version rejects at open. */
/** Current domain format version; reads enforce it according to the selected layout. */
readonly version: number
/**
* Medium layout for the backend unit: `single` (the default) stores the
* whole unit as one document; `per-record` stores each record as its own
* document, for units whose records are large, sparse, or individually
* disposable — the projection cache — and scopes version bumps per record
* (a stale record document is discarded, never migrated).
* disposable — the projection cache — and scopes version checks per record
* (an unaccepted record document is discarded, never migrated).
*/
readonly layout?: 'single' | 'per-record'
/**
* Older domain versions whose stored records the current record schemas
* also accept (the declaring owner vouches for that, typically by
* declaring the fields older records lack as optional). `per-record` backends
* read documents stamped with a listed version instead of discarding them,
* and accept a legacy whole-unit file so stamped for the one-time
* bootstrap; writes always stamp {@link version}.
*/
readonly compatibleVersions?: readonly number[]
/**
* What `open` does with a stored table record that fails its zod schema.
* Absent (the default), the whole open rejects with `invalid-record` —
* right for authoritative data. `'backup-and-skip'` is for domains whose
* records are disposable derived data: the backend moves the record's
* document aside (`KvUnit.backupRecord`), the failure is logged with
* its cause, and the open continues with the record absent. A backend
* without `backupRecord` (no per-record document to move) falls back
* to the rejecting default. The global slot always rejects.
*/
readonly invalidRecords?: 'backup-and-skip'
/** Optional global singleton slot. */
readonly global?: DomainGlobalSpec<unknown>
/** Table declarations keyed by table name; each name must match `UNIT_NAME_RE`. */
@ -180,7 +200,10 @@ The mounted domain facility. Opens declared domains over routed backends; one fa
* (`facet-unsupported`); open the unit projected from the spec (backend
* `version-mismatch`/`malformed-medium` pass through); load and validate
* every stored record against the spec's zod schemas (`invalid-record`
* with the offending table and key); construct the domain.
* with the offending table and key — unless the spec declares
* `invalidRecords: 'backup-and-skip'` and the unit can move documents aside, in
* which case the failing record is backed up, logged, and skipped);
* construct the domain.
*
* Lifecycle: the CALLER owns the returned handle and closes it via
* `Domain.close()` (typically as its own `ctx.effect` disposer) — the

View file

@ -2109,7 +2109,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
methods: [
{
signature: 'async open<S extends DomainSpec>(spec: S): Promise<Domain<S>>',
description: 'Open one declared domain. Steps, each failing the whole call: reject a name that is already open (`already-open`); resolve the backend route (`backend-not-found` passes through from the hub); require its `kv` facet (`facet-unsupported`); open the unit projected from the spec (backend `version-mismatch`/`malformed-medium` pass through); load and validate every stored record against the spec\'s zod schemas (`invalid-record` with the offending table and key); construct the domain.\n\nLifecycle: the CALLER owns the returned handle and closes it via `Domain.close()` (typically as its own `ctx.effect` disposer) — the facility does not tie the domain to any consumer fiber. Domains still open when the facility unmounts are closed by the plugin disposer.',
description: 'Open one declared domain. Steps, each failing the whole call: reject a name that is already open (`already-open`); resolve the backend route (`backend-not-found` passes through from the hub); require its `kv` facet (`facet-unsupported`); open the unit projected from the spec (backend `version-mismatch`/`malformed-medium` pass through); load and validate every stored record against the spec\'s zod schemas (`invalid-record` with the offending table and key — unless the spec declares `invalidRecords: \'backup-and-skip\'` and the unit can move documents aside, in which case the failing record is backed up, logged, and skipped); construct the domain.\n\nLifecycle: the CALLER owns the returned handle and closes it via `Domain.close()` (typically as its own `ctx.effect` disposer) — the facility does not tie the domain to any consumer fiber. Domains still open when the facility unmounts are closed by the plugin disposer.',
parameters: [{ name: 'spec', description: 'The domain declaration, typically from `defineDomain`.' }],
returns: 'the opened domain handle, typed by the spec.',
},
@ -3910,7 +3910,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'DomainSpec',
declaration: 'export interface DomainSpec {\n readonly name: string;\n readonly version: number;\n readonly layout?: \'single\' | \'per-record\';\n readonly global?: DomainGlobalSpec<unknown>;\n readonly tables: Record<string, DomainTableSpec>;\n}',
declaration: 'export interface DomainSpec {\n readonly name: string;\n readonly version: number;\n readonly layout?: \'single\' | \'per-record\';\n readonly compatibleVersions?: readonly number[];\n readonly invalidRecords?: \'backup-and-skip\';\n readonly global?: DomainGlobalSpec<unknown>;\n readonly tables: Record<string, DomainTableSpec>;\n}',
},
{
name: 'DomainTableSpec',
@ -4218,11 +4218,11 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'KvUnit',
declaration: 'export interface KvUnit {\n loadAll(): Promise<{\n tables: Record<string, Record<string, unknown>>;\n global: unknown;\n }>;\n putRecord(table: string, key: string, value: unknown): Promise<void>;\n deleteRecord(table: string, key: string): Promise<void>;\n setGlobal(value: unknown): Promise<void>;\n close(): Promise<void>;\n}',
declaration: 'export interface KvUnit {\n loadAll(): Promise<{\n tables: Record<string, Record<string, unknown>>;\n global: unknown;\n }>;\n putRecord(table: string, key: string, value: unknown): Promise<void>;\n deleteRecord(table: string, key: string): Promise<void>;\n backupRecord?(table: string, key: string): Promise<string>;\n setGlobal(value: unknown): Promise<void>;\n close(): Promise<void>;\n}',
},
{
name: 'KvUnitDescriptor',
declaration: 'export interface KvUnitDescriptor {\n readonly name: string;\n readonly version: number;\n readonly tables: readonly string[];\n readonly hasGlobal: boolean;\n readonly layout?: \'single\' | \'per-record\';\n}',
declaration: 'export interface KvUnitDescriptor {\n readonly name: string;\n readonly version: number;\n readonly tables: readonly string[];\n readonly hasGlobal: boolean;\n readonly layout?: \'single\' | \'per-record\';\n readonly compatibleVersions?: readonly number[];\n}',
},
{
name: 'LlmAdapter',

View file

@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/session/session-projection-cache/README.md
README.md: 51b9d86724af96304cf09a5c7c1b61b7394d336a
README.zh.md: bb84b67bdde678884fd4f2be1b14b2161da8c2a0
README.md: 9fd9766d75ab3f9a460b2802ac59810839f21ae4
README.zh.md: 98725d9ce76ab44821adf3d43e3807932cf6b675

View file

@ -64,6 +64,8 @@ Three mandatory points always write: session creation persists the seed-derived
The log leads and the cache follows: a live checkpoint flushes the session's buffered events durably before the cache row lands, so a crash can leave the cache behind the log but never ahead of it. Reads and writes share the storage domain's coherent in-memory state; the per-unit write chain mutates memory only after durability. Each version-stamped record must match the live unit schema and complete lifecycle identity (`createdAt`, `cwd`, `isSeeded`, and `inheritedEventCount`), so a row initialized under one fork cut cannot seed another. The JSON backend stores each record at `<root>/session_projcache/sessions/<id>.json` in an owner-only directory tree.
Upgrades never cost the boot or the listing: records stamped with a version in the spec's `compatibleVersions` stay readable (their absent lineage fields decode as the unseeded lineage — exact for unseeded sessions, while a seeded caller fails the identity match and refolds cold), and a stored record that still fails schema validation is moved aside as `<id>.json.bak.<stamp>` under the domain's `invalidRecords: 'backup-and-skip'` policy, logged with its cause, and rebuilt by the next checkpoint.
-----
<a id="understand-the-implementation"></a>
@ -126,6 +128,7 @@ These limits define where the cache needs operational care. They are current pac
- **No eviction or retention surface** — records accumulate per session; pruning stored checkpoints is out-of-band maintenance, same stance as session persistence itself.
- **Interval throttle is per-session coarse** — the timer arms at the first dirty event after a clean write; a steady sub-threshold trickle writes once per interval, not a sliding window.
- **No cache-side cold refold** — the cache serves and refreshes its rows but never reads the session log (it does not depend on the persistence layer); a consumer that needs a guaranteed cold snapshot refolds from the log itself.
- **Every schema or domain-version change must prove its upgrade story** — a change to the stored record schema or the domain version lands in the same PR with an archived fixture of the previously shipped on-disk format under `tests/fixtures/` and test cases in `tests/fixtures.spec.ts` proving the chosen disposition: read-compat recovery (`compatibleVersions`), current-version rewrite, or backup-and-skip salvage. A bump whose old records are simply discarded still proves that the discard neither fails the boot nor poisons the tree.
<a id="dev-note"></a>
### Dev Note

View file

@ -64,6 +64,8 @@ kind: "package-reference"
日志领先,缓存跟随:实时检查点先把会话的缓冲事件持久化,然后才保存缓存记录。因此崩溃可能让缓存落后于日志,但绝不会让缓存领先。读取和写入共享存储域内一致的内存状态;逐单元写入链只在持久化成功后修改内存。每个带版本戳的记录必须匹配实时单元 schema 与完整生命周期身份(`createdAt`、`cwd`、`isSeeded` 和 `inheritedEventCount`),因此在一个 fork 切点下初始化的行不能播种另一个切点。JSON 后端把每条记录存于仅所有者可访问的 `<root>/session_projcache/sessions/<id>.json` 目录树中。
升级绝不拖垮启动或列表:版本戳落在 spec `compatibleVersions` 集合内的记录保持可读(缺失的 lineage 字段解码为 unseeded lineage——对非 fork 会话精确无误,seeded 调用方则通不过身份比对、回落冷折叠),而仍然通不过 schema 校验的存量记录会按域的 `invalidRecords: 'backup-and-skip'` 策略移出为 `<id>.json.bak.<时间戳>`、连同原因写入日志,并由下一次检查点重建。
-----
<a id="understand-the-implementation"></a>
@ -126,6 +128,7 @@ kind: "package-reference"
- **无淘汰或保留接口**——记录按会话持续累积;清理已存储检查点属于带外维护,与会话持久化采用相同策略。
- **间隔节流采用按会话的粗粒度控制**——一次无脏数据的写入完成后,计时器在首个脏事件到达时启动;持续但低于条数阈值的事件流每间隔写入一次,而非滑动窗口。
- **缓存侧不做冷重折叠**——缓存只服务并刷新自己的记录,从不读取会话日志,因为它不依赖持久化层;需要保证冷快照的消费方自行从日志重新折叠。
- **每次 schema 或域版本变更都必须论证升级路径**——改动存储记录 schema 或域版本时,同一 PR 必须在 `tests/fixtures/` 下归档此前已发布的磁盘格式样本,并在 `tests/fixtures.spec.ts` 中用测试论证所选的处置方式:读兼容恢复(`compatibleVersions`)、当前版本重写,或 backup-and-skip 抢救。即便选择直接丢弃旧记录的 bump,也要证明丢弃既不炸启动、也不污染缓存树。
<a id="dev-note"></a>
### 开发备注

View file

@ -367,12 +367,18 @@ function identityOf(
}
}
/** Whether a stored record's bound identity names the caller's lifecycle. */
/**
* Whether a stored record's bound identity names the caller's lifecycle.
* Absent lineage fields (records admitted via `compatibleVersions` predate
* them) read as the unseeded lineage: exact for an unseeded caller, and a
* seeded caller's expectation then fails the match, discarding the record to
* a cold rebuild.
*/
function identityMatches(stored: CheckpointIdentity, expected: CheckpointIdentity): boolean {
return stored.createdAt === expected.createdAt
&& stored.cwd === expected.cwd
&& stored.isSeeded === expected.isSeeded
&& stored.inheritedEventCount === expected.inheritedEventCount
&& (stored.isSeeded ?? false) === expected.isSeeded
&& (stored.inheritedEventCount ?? 0) === expected.inheritedEventCount
}
export default SessionProjectionCache

View file

@ -38,12 +38,19 @@ export const checkpointRow = z.object({
* old record pass every watermark check and seed state folded from an
* unrelated log. Reads validate this against the live header (listing) or
* the stored header (cold read) before accepting any record.
*
* The lineage fields are optional because records admitted through
* `compatibleVersions` predate them. The reader (`identityMatches`)
* interprets their absence as the unseeded lineage — exact for an unseeded
* session, while a seeded expectation fails the match and the record is
* discarded to a cold rebuild. Current-version writes always store both
* fields.
*/
export const checkpointIdentity = z.object({
createdAt: z.number().int().nonnegative(),
cwd: z.string().optional(),
isSeeded: z.boolean(),
inheritedEventCount: z.number().int().nonnegative().transform(SessionLogOffset),
isSeeded: z.boolean().optional(),
inheritedEventCount: z.number().int().nonnegative().transform(SessionLogOffset).optional(),
})
/** The identity fields a record is bound to, inferred from {@link checkpointIdentity}. */
@ -68,11 +75,27 @@ export type CheckpointRecord = z.infer<typeof checkpointRecord>
* bumps per session: after a bump, a stale session document is discarded on
* open (cache semantics — a stale or unreadable cache costs a longer tail
* replay, never a wrong value) while the rest of the domain stays usable,
* instead of rejecting the whole medium.
* instead of rejecting the whole medium. The `compatibleVersions` entries
* are declared because those records differ from the current version only
* by the absent optional lineage fields, so upgraded homes keep serving
* their cached listing projections instead of dropping every title until
* each session is reopened; the per-record version map lives in the
* read-compat Agent Note
* (.agents/notes/implemented/architecture/2026-09-02-projcache-cross-version-read-compat.md).
* The per-row `ver` guard and the identity match still discard anything the
* current fold semantics cannot vouch for.
*
* `invalidRecords: 'backup-and-skip'`: a stored record that fails the schema
* anyway is disposable derived data, so it must never cost the boot — the
* domain layer moves the document aside as `<key>.json.bak.<stamp>`, logs
* the concrete validation failure, and serves the session as uncached (a
* cold read rebuilds and rewrites it).
*/
export const projectionCacheDomainSpec = defineDomain({
name: 'session_projcache',
version: 6,
compatibleVersions: [3, 4, 5],
invalidRecords: 'backup-and-skip',
layout: 'per-record',
tables: { sessions: domainTable<SessionId, CheckpointRecord>(checkpointRecord) },
})

View file

@ -42,9 +42,11 @@ declare module '@deepseek-ai/dsh-session-projection/types' {
'cache-test/marks2': Map<string, string>
'cache-test/count': number
'cache-test/secret': string
'cache-test/marks3': MarksState
}
interface SessionProjectionMap {
'cache-test/marks': { marks: string[] }
'cache-test/marks3': { marks: string[] }
}
}
@ -71,6 +73,18 @@ const marksUnit = (stateVersion = 1) => ({
stateVersion,
}) satisfies ProjectionDefinition<'cache-test/marks', MarksState>
const marks3Unit = {
key: 'cache-test/marks3',
stateSchema: z.object({ marks: z.array(z.string()) }).nullable(),
init: () => null,
apply: state => state,
wire: {
viewSchema: z.object({ marks: z.array(z.string()) }),
view: state => state ?? { marks: [] },
},
stateVersion: 1,
} satisfies ProjectionDefinition<'cache-test/marks3', MarksState>
const secretUnit = {
key: 'cache-test/secret',
stateSchema: z.string(),
@ -376,14 +390,35 @@ describe('SessionProjectionCache listing read', () => {
.toBeUndefined()
})
it('returns undefined when the stored record is version-mismatched', async () => {
it('carries ONE cut across multiple served rows: the lowest watermark wins', async () => {
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-'))
roots.push(root)
// A stale version-stamped document is discarded at open: absent record.
// Equal watermarks: whichever row is visited second cannot lower the cut,
// so the one-cut fold sees both a lowering and a non-lowering row in
// every iteration order.
await seedRecord(root, 'multi-row', {
'cache-test/marks': { ver: 1, seq: SessionSeq(4), val: { marks: ['a'] } },
'cache-test/marks3': { ver: 1, seq: SessionSeq(4), val: { marks: ['b'] } },
})
const { ctx, cache } = await harness({ root })
ctx.sessionProjections.register(marks3Unit)
const block = cache.cachedSnapshot(headerOf(SessionId('multi-row')), SessionLogOffset(0))
expect(block?.values).toEqual({
'cache-test/marks': { marks: ['a'] },
'cache-test/marks3': { marks: ['b'] },
})
expect(block?.asOfSeq).toBe(4)
})
it('returns undefined when the stored record version is not accepted', async () => {
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-'))
roots.push(root)
// Version 2 is neither current nor declared compatible, so the document
// is discarded at open and the record reads as absent.
const path = recordPath(root, SessionId('all-stale'))
await mkdir(dirname(path), { recursive: true })
await writeFile(path, JSON.stringify({
version: projectionCacheDomainSpec.version - 1,
version: 2,
record: {
identity: { createdAt: 0, isSeeded: false, inheritedEventCount: 0 },
rows: { 'cache-test/marks': { ver: 1, seq: 4, val: { marks: ['old'] } } },
@ -394,6 +429,30 @@ describe('SessionProjectionCache listing read', () => {
.toBeUndefined()
})
it('serves a pre-lineage record (accepted old version) to an unseeded caller only', async () => {
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-'))
roots.push(root)
// A document stamped with an accepted older version whose identity
// predates the lineage fields: absent lineage reads as unseeded.
const path = recordPath(root, SessionId('pre-lineage'))
await mkdir(dirname(path), { recursive: true })
await writeFile(path, JSON.stringify({
version: 4,
record: {
identity: { createdAt: 0 },
rows: { 'cache-test/marks': { ver: 1, seq: 4, val: { marks: ['kept'] } } },
},
}))
const { cache } = await harness({ root })
const id = SessionId('pre-lineage')
// Unseeded caller: the absent lineage is exactly its identity — served.
expect(cache.cachedSnapshot(headerOf(id), SessionLogOffset(0)))
.toEqual({ asOfSeq: 4, values: { 'cache-test/marks': { marks: ['kept'] } } })
// Seeded caller: the lineage-less record cannot vouch for the cut — refused.
expect(cache.cachedSnapshot({ ...headerOf(id), isSeeded: true }, SessionLogOffset(2)))
.toBeUndefined()
})
it('returns undefined when every stored row is version-mismatched', async () => {
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-'))
roots.push(root)

View file

@ -0,0 +1,242 @@
/**
* Cross-version recovery over archived on-disk artifacts. `fixtures/` holds
* real `session_projcache` media, each produced by driving the named release
* through its own web app (session created over RPC, real model turns, a
* rename): the v3 whole-unit file (published 0.1.1-rc.2), a v4 per-record
* document (published 0.1.2-alpha.3), a published v5 document, and the
* v5-stamped lineage-less document reproducing byte-for-byte what the
* formerly unguarded legacy bootstrap wrote over v3 records. Each must
* recover through the real storage stack — the domain opens and the listing
* read serves the archived title — and a record that fails schema validation
* anyway is backed up and skipped instead of failing the boot.
*/
import { afterEach, describe, expect, it, vi } from 'vitest'
import { cp, mkdir, mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { dirname, join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { Context } from '@deepseek-ai/cordis'
import { z } from 'zod'
import SessionStore, { SessionId, SessionLogOffset } from '@deepseek-ai/dsh-session'
import type { SessionHeader } from '@deepseek-ai/dsh-session'
import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection'
import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection'
import Storage from '@deepseek-ai/dsh-storage'
import {
apply as storageJsonApply, Config as storageJsonConfig, inject as storageJsonInject, name as storageJsonName,
} from '@deepseek-ai/dsh-storage-json'
import {
apply as storageDomainApply, Config as storageDomainConfig, inject as storageDomainInject, name as storageDomainName,
} from '@deepseek-ai/dsh-storage-domain'
import SessionProjectionCache from '../src/index.ts'
import { projectionCacheDomainSpec } from '../src/spec.ts'
// Declarations must match the shipped title unit's exactly (the repo-wide
// compile face sees both).
declare module '@deepseek-ai/dsh-session-projection/types' {
interface SessionProjectionStateMap {
title: string | null
}
interface SessionProjectionMap {
title: string | null
}
}
declare module '@deepseek-ai/dsh-session/types' {
interface SessionEventMap {
'fixtures-test/set-title': { title: string }
}
interface OutOfBandSessionEventMap {
'fixtures-test/set-title': true
}
}
// Mirrors the shipped title unit's storage face: stateVersion 1, bare-string
// state (the fixture rows carry exactly this shape in every archived
// version), folding a test event so the rewrite path has fresh data.
const titleUnit = {
key: 'title',
stateSchema: z.string().nullable(),
init: () => null,
apply: (state, event) => (event.type === 'fixtures-test/set-title' ? event.data.title : state),
wire: { viewSchema: z.string().nullable(), view: state => state },
stateVersion: 1,
} satisfies ProjectionDefinition<'title', string | null>
const FIXTURES = fileURLToPath(new URL('./fixtures/', import.meta.url))
/** One archived per-record document (`{version, record}`). */
interface FixtureDoc {
version: number
record: {
identity: { createdAt: number; cwd?: string }
rows: Record<string, { ver: number; seq: number; val: unknown }>
}
}
async function fixtureJson<T>(name: string): Promise<T> {
return JSON.parse(await readFile(join(FIXTURES, name), 'utf8')) as T
}
/** Header for the session a fixture record is bound to (identity witness). */
function headerFor(id: SessionId, identity: FixtureDoc['record']['identity']): SessionHeader {
return {
version: 0,
id,
createdAt: identity.createdAt,
isSeeded: false,
...identity.cwd === undefined ? {} : { cwd: identity.cwd },
}
}
const contexts: Context[] = []
const roots: string[] = []
async function harness(root: string) {
roots.push(root)
const ctx = new Context()
contexts.push(ctx)
await ctx.plugin(Storage)
await ctx.plugin({ name: storageJsonName, inject: storageJsonInject, apply: storageJsonApply, Config: storageJsonConfig }, { root })
await ctx.plugin({ name: storageDomainName, inject: storageDomainInject, apply: storageDomainApply, Config: storageDomainConfig }, { backend: 'json' })
await ctx.plugin(SessionStore)
await ctx.plugin(SessionProjectionRegistry)
ctx.sessionProjections.register(titleUnit)
await ctx.plugin(SessionProjectionCache, { writeEveryEvents: 100, writeIntervalMs: 60_000 })
return { ctx, cache: ctx.sessionProjectionCache }
}
/** Lay one per-record fixture document into a fresh backend root. */
async function placeDoc(root: string, id: string, name: string): Promise<FixtureDoc> {
const path = join(root, projectionCacheDomainSpec.name, 'sessions', `${id}.json`)
await mkdir(dirname(path), { recursive: true })
await cp(join(FIXTURES, name), path)
return fixtureJson<FixtureDoc>(name)
}
/**
* Drive a live write over a recovered session id and assert the archived
* document is replaced by a current-version one: v6 stamp, lineage present,
* and the freshly folded title — the write path never keeps the old format.
*/
async function assertRewrite(ctx: Context, root: string, id: SessionId): Promise<void> {
const session = ctx.sessions.create(id)
session.append('fixtures-test/set-title', { title: '重写标题' })
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
const path = join(root, projectionCacheDomainSpec.name, 'sessions', `${id}.json`)
await vi.waitFor(async () => {
const doc = JSON.parse(await readFile(path, 'utf8')) as FixtureDoc
expect(doc.version).toBe(projectionCacheDomainSpec.version)
expect(doc.record.identity).toMatchObject({ isSeeded: false, inheritedEventCount: 0 })
expect(doc.record.rows['title']?.val).toBe('重写标题')
}, { timeout: 5_000 })
}
afterEach(async () => {
await Promise.all(contexts.splice(0).map(ctx => ctx.fiber.dispose()))
await Promise.all(roots.splice(0).map(root => rm(root, { recursive: true, force: true, maxRetries: 10, retryDelay: 100 })))
})
describe('archived version recovery', () => {
it('recovers the v3 whole-unit archive through the legacy bootstrap', async () => {
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-fx-'))
await cp(join(FIXTURES, 'v3-single-unit.json'), join(root, `${projectionCacheDomainSpec.name}.json`))
type SingleUnit = {
unit: { version: number }
tables: { sessions: Record<string, FixtureDoc['record']> }
}
const archive = await fixtureJson<SingleUnit>('v3-single-unit.json')
expect(archive.unit.version).toBe(3) // the fixture IS the old format
const [sid, record] = Object.entries(archive.tables.sessions)[0]!
const { ctx, cache } = await harness(root)
const snapshot = cache.cachedSnapshot(headerFor(SessionId(sid), record.identity), SessionLogOffset(0), ['title'])
expect(snapshot?.values.title).toBe(record.rows['title']!.val)
// The one-time bootstrap materialized a current-version document.
const migrated = JSON.parse(
await readFile(join(root, projectionCacheDomainSpec.name, 'sessions', `${sid}.json`), 'utf8'),
) as { version: number }
expect(migrated.version).toBe(projectionCacheDomainSpec.version)
await assertRewrite(ctx, root, SessionId(sid))
})
for (const [fixture, storedVersion] of [
['v4-session-doc.json', 4],
['v5-session-doc.json', 5],
['v5-lineageless-doc.json', 5],
] as const) {
it(`serves the archived title from ${fixture}, then rewrites it current`, async () => {
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-fx-'))
const id = SessionId('fixture-session')
const doc = await placeDoc(root, id, fixture)
expect(doc.version).toBe(storedVersion)
const { ctx, cache } = await harness(root)
const snapshot = cache.cachedSnapshot(headerFor(id, doc.record.identity), SessionLogOffset(0), ['title'])
expect(snapshot?.values.title).toBe(doc.record.rows['title']!.val)
await assertRewrite(ctx, root, id)
})
}
it('refuses a lineage-less archive for a seeded caller (identity mismatch, cold rebuild)', async () => {
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-fx-'))
const id = SessionId('fixture-seeded')
const doc = await placeDoc(root, id, 'v5-lineageless-doc.json')
const { cache } = await harness(root)
const seeded = { ...headerFor(id, doc.record.identity), isSeeded: true }
expect(cache.cachedSnapshot(seeded, SessionLogOffset(2), ['title'])).toBeUndefined()
})
it('backs up and skips a record that fails schema validation instead of failing the boot', async () => {
const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-fx-'))
roots.push(root)
const sessionsDir = join(root, projectionCacheDomainSpec.name, 'sessions')
await mkdir(sessionsDir, { recursive: true })
// Current-version stamp, hopeless record content: no compat rung can save it.
await writeFile(join(sessionsDir, 'broken.json'), JSON.stringify({
version: projectionCacheDomainSpec.version,
record: { identity: { createdAt: 'not-a-number' }, rows: 'not-an-object' },
}))
const good = await placeDoc(root, SessionId('survivor'), 'v5-session-doc.json')
const ctx = new Context()
contexts.push(ctx)
await ctx.plugin(Storage)
await ctx.plugin({ name: storageJsonName, inject: storageJsonInject, apply: storageJsonApply, Config: storageJsonConfig }, { root })
await ctx.plugin({ name: storageDomainName, inject: storageDomainInject, apply: storageDomainApply, Config: storageDomainConfig }, { backend: 'json' })
await ctx.plugin(SessionStore)
await ctx.plugin(SessionProjectionRegistry)
ctx.sessionProjections.register(titleUnit)
const error = vi.spyOn(ctx.logger, 'error').mockImplementation(() => {})
// The boot survives the broken record — this line rejecting IS the fixed bug.
await ctx.plugin(SessionProjectionCache, { writeEveryEvents: 100, writeIntervalMs: 60_000 })
// Concrete console diagnostics: which record, where it went, and why.
expect(error).toHaveBeenCalledWith(expect.stringContaining("record 'broken'"))
expect(error).toHaveBeenCalledWith(expect.stringContaining('.json.bak.'))
// The document was moved aside as <key>.json.bak.<YYYYMMDDHHmm>, bytes intact.
const entries = await readdir(sessionsDir)
expect(entries).not.toContain('broken.json')
const backup = entries.find(name => /^broken\.json\.bak\.\d{12}$/.test(name))
expect(backup).toBeDefined()
expect(JSON.parse(await readFile(join(sessionsDir, backup!), 'utf8')))
.toMatchObject({ record: { rows: 'not-an-object' } })
// The broken record reads as absent; its neighbors still serve.
const cache = ctx.sessionProjectionCache
expect(cache.cachedSnapshot(headerFor(SessionId('broken'), { createdAt: 0 }), SessionLogOffset(0)))
.toBeUndefined()
expect(cache.cachedSnapshot(
headerFor(SessionId('survivor'), good.record.identity),
SessionLogOffset(0),
['title'],
)?.values.title).toBe(good.record.rows['title']!.val)
})
})

View file

@ -0,0 +1,136 @@
{
"unit": {
"name": "session_projcache",
"version": 3
},
"global": null,
"tables": {
"sessions": {
"session-1374fa81-15da-44ca-be12-b4c6fe8076a3": {
"identity": {
"createdAt": 1788286864454,
"cwd": "/tmp"
},
"rows": {
"sessionStats": {
"ver": 1,
"seq": 54,
"val": {
"turns": 2,
"steps": 2,
"llmMs": 2169,
"toolMs": 0,
"ttftMs": 1872,
"ttftSteps": 2,
"decodeMs": 297,
"decodeTokens": 20,
"lastTurn": 2,
"openStep": null,
"pendingCalls": {}
}
},
"title": {
"ver": 1,
"seq": 54,
"val": "验收标题-rc2"
},
"goal": {
"ver": 4,
"seq": 54,
"val": null
},
"tokenUsage": {
"ver": 1,
"seq": 54,
"val": {
"totals": {
"uncachedInputTokens": 6083,
"outputTokens": 20,
"cacheReadTokens": 9728,
"cacheWriteTokens": 0
},
"last": {
"turn": 2,
"step": 1,
"buckets": {
"uncachedInputTokens": 112,
"outputTokens": 4,
"cacheReadTokens": 7808,
"cacheWriteTokens": 0
}
}
}
},
"contextPressure": {
"ver": 4,
"seq": 54,
"val": {
"surfaceTokens": 178,
"contextWindow": 1000000,
"pressureTokens": 7920,
"sampledSurfaceTokens": 168
}
},
"contextBreakdown": {
"ver": 2,
"seq": 54,
"val": {
"systemTokens": 1620,
"toolsTokens": 6475,
"messageTokens": 178
}
},
"subagentTiming": {
"ver": 2,
"seq": 54,
"val": {
"descriptorSeen": false,
"settledMs": 0
}
},
"subagent": {
"ver": 2,
"seq": 54,
"val": {}
},
"permissions": {
"ver": 1,
"seq": 54,
"val": {
"preset": "workspace-write",
"sandbox": "workspace-write",
"approval": "ask"
}
},
"sessionListMetadata": {
"ver": 1,
"seq": 54,
"val": {
"blank": false,
"lastPromptAt": 1788286867410
}
},
"imageLimits": {
"ver": 1,
"seq": 54,
"val": null
},
"todos": {
"ver": 2,
"seq": 54,
"val": null
},
"plan": {
"ver": 2,
"seq": 54,
"val": {
"active": false,
"wanted": null,
"running": null
}
}
}
}
}
}
}

View file

@ -0,0 +1,212 @@
{
"version": 4,
"record": {
"identity": {
"createdAt": 1788286912530,
"cwd": "/tmp"
},
"rows": {
"title": {
"ver": 1,
"seq": 68,
"val": "验收标题-alpha3"
},
"titleInput": {
"ver": 3,
"seq": 68,
"val": {
"first": {
"seq": 7,
"text": "请只回复一个词:pong"
},
"count": 2,
"lastSeq": 58
}
},
"llmRetry": {
"ver": 1,
"seq": 68,
"val": {}
},
"sandboxMode": {
"ver": 1,
"seq": 68,
"val": "workspace-write"
},
"goal": {
"ver": 6,
"seq": 68,
"val": {
"current": null,
"seenGoalIds": [],
"failure": null
}
},
"tokenUsage": {
"ver": 2,
"seq": 68,
"val": {
"totals": {
"uncachedInputTokens": 8192,
"outputTokens": 35,
"cacheReadTokens": 8064,
"cacheWriteTokens": 0
},
"last": {
"turn": 2,
"step": 1,
"buckets": {
"uncachedInputTokens": 86,
"outputTokens": 4,
"cacheReadTokens": 8064,
"cacheWriteTokens": 0
}
}
}
},
"contextPressure": {
"ver": 4,
"seq": 68,
"val": {
"surfaceTokens": 193,
"contextWindow": 1000000,
"pressureTokens": 8150,
"sampledSurfaceTokens": 183
}
},
"contextBreakdown": {
"ver": 2,
"seq": 68,
"val": {
"systemTokens": 1760,
"toolsTokens": 6579,
"messageTokens": 193
}
},
"turnBoundary": {
"ver": 2,
"seq": 68,
"val": {
"openTurnStartSeq": null,
"lastStepStartSeq": 57,
"lastStepBoundary": {
"kind": "end",
"seq": 67
},
"lastTurn": 2
}
},
"sessionStats": {
"ver": 1,
"seq": 68,
"val": {
"turns": 2,
"steps": 2,
"llmMs": 3486,
"toolMs": 0,
"ttftMs": 3199,
"ttftSteps": 2,
"decodeMs": 287,
"decodeTokens": 35,
"lastTurn": 2,
"openStep": null,
"pendingCalls": {}
}
},
"turnOutline": {
"ver": 2,
"seq": 68,
"val": {
"turns": [
{
"turn": 1,
"seq": 4,
"prompt": "请只回复一个词:pong",
"response": "pong"
},
{
"turn": 2,
"seq": 55,
"prompt": "请只回复一个词:pong2",
"response": "pong2"
}
],
"draft": ""
}
},
"agentPreset": {
"ver": 1,
"seq": 68,
"val": "standard"
},
"subagentTiming": {
"ver": 2,
"seq": 68,
"val": {
"descriptorSeen": false,
"settledMs": 0
}
},
"subagent": {
"ver": 2,
"seq": 68,
"val": {}
},
"permissions": {
"ver": 2,
"seq": 68,
"val": {
"preset": "workspace-write",
"sandbox": "workspace-write",
"approval": "ask",
"seeded": false
}
},
"modelSelection": {
"ver": 2,
"seq": 68,
"val": {
"lastUsed": {
"provider": "deepseek-official",
"model": "deepseek-v4-flash",
"reasoningEffort": "high"
},
"pending": null
}
},
"sessionListMetadata": {
"ver": 1,
"seq": 68,
"val": {
"blank": false,
"lastPromptAt": 1788286917834
}
},
"imageLimits": {
"ver": 1,
"seq": 68,
"val": null
},
"todos": {
"ver": 2,
"seq": 68,
"val": null
},
"plan": {
"ver": 3,
"seq": 68,
"val": {
"active": false,
"wanted": null,
"running": null,
"activeAtLastHeader": false
}
},
"subagentModelSelectionPolicy": {
"ver": 1,
"seq": 68,
"val": null
}
}
}
}

View file

@ -0,0 +1,128 @@
{
"version": 5,
"record": {
"identity": {
"createdAt": 1788286864454,
"cwd": "/tmp"
},
"rows": {
"sessionStats": {
"ver": 1,
"seq": 54,
"val": {
"turns": 2,
"steps": 2,
"llmMs": 2169,
"toolMs": 0,
"ttftMs": 1872,
"ttftSteps": 2,
"decodeMs": 297,
"decodeTokens": 20,
"lastTurn": 2,
"openStep": null,
"pendingCalls": {}
}
},
"title": {
"ver": 1,
"seq": 54,
"val": "\u9a8c\u6536\u6807\u9898-rc2"
},
"goal": {
"ver": 4,
"seq": 54,
"val": null
},
"tokenUsage": {
"ver": 1,
"seq": 54,
"val": {
"totals": {
"uncachedInputTokens": 6083,
"outputTokens": 20,
"cacheReadTokens": 9728,
"cacheWriteTokens": 0
},
"last": {
"turn": 2,
"step": 1,
"buckets": {
"uncachedInputTokens": 112,
"outputTokens": 4,
"cacheReadTokens": 7808,
"cacheWriteTokens": 0
}
}
}
},
"contextPressure": {
"ver": 4,
"seq": 54,
"val": {
"surfaceTokens": 178,
"contextWindow": 1000000,
"pressureTokens": 7920,
"sampledSurfaceTokens": 168
}
},
"contextBreakdown": {
"ver": 2,
"seq": 54,
"val": {
"systemTokens": 1620,
"toolsTokens": 6475,
"messageTokens": 178
}
},
"subagentTiming": {
"ver": 2,
"seq": 54,
"val": {
"descriptorSeen": false,
"settledMs": 0
}
},
"subagent": {
"ver": 2,
"seq": 54,
"val": {}
},
"permissions": {
"ver": 1,
"seq": 54,
"val": {
"preset": "workspace-write",
"sandbox": "workspace-write",
"approval": "ask"
}
},
"sessionListMetadata": {
"ver": 1,
"seq": 54,
"val": {
"blank": false,
"lastPromptAt": 1788286867410
}
},
"imageLimits": {
"ver": 1,
"seq": 54,
"val": null
},
"todos": {
"ver": 2,
"seq": 54,
"val": null
},
"plan": {
"ver": 2,
"seq": 54,
"val": {
"active": false,
"wanted": null,
"running": null
}
}
}
}
}

View file

@ -0,0 +1,220 @@
{
"version": 5,
"record": {
"identity": {
"createdAt": 1788286864454,
"cwd": "/tmp",
"isSeeded": false,
"inheritedEventCount": 0
},
"rows": {
"title": {
"ver": 1,
"seq": 71,
"val": "验收标题-rc2"
},
"titleInput": {
"ver": 3,
"seq": 71,
"val": {
"first": {
"seq": 7,
"text": "请只回复一个词:pong"
},
"count": 3,
"lastSeq": 60
}
},
"llmRetry": {
"ver": 1,
"seq": 71,
"val": {}
},
"sandboxMode": {
"ver": 1,
"seq": 71,
"val": "workspace-write"
},
"goal": {
"ver": 6,
"seq": 71,
"val": {
"current": null,
"seenGoalIds": [],
"failure": null
}
},
"tokenUsage": {
"ver": 2,
"seq": 71,
"val": {
"totals": {
"uncachedInputTokens": 14219,
"outputTokens": 24,
"cacheReadTokens": 9728,
"cacheWriteTokens": 0
},
"last": {
"turn": 3,
"step": 1,
"buckets": {
"uncachedInputTokens": 8136,
"outputTokens": 4,
"cacheReadTokens": 0,
"cacheWriteTokens": 0
}
}
}
},
"contextPressure": {
"ver": 4,
"seq": 71,
"val": {
"surfaceTokens": 200,
"contextWindow": 1000000,
"pressureTokens": 8136,
"sampledSurfaceTokens": 190
}
},
"contextBreakdown": {
"ver": 2,
"seq": 71,
"val": {
"systemTokens": 1729,
"toolsTokens": 6611,
"messageTokens": 200
}
},
"sessionStats": {
"ver": 1,
"seq": 71,
"val": {
"turns": 3,
"steps": 3,
"llmMs": 4368,
"toolMs": 0,
"ttftMs": 4030,
"ttftSteps": 3,
"decodeMs": 338,
"decodeTokens": 24,
"lastTurn": 3,
"openStep": null,
"pendingCalls": {}
}
},
"agentPreset": {
"ver": 1,
"seq": 71,
"val": "standard"
},
"subagentTiming": {
"ver": 2,
"seq": 71,
"val": {
"descriptorSeen": false,
"settledMs": 0
}
},
"subagent": {
"ver": 2,
"seq": 71,
"val": {}
},
"turnBoundary": {
"ver": 2,
"seq": 71,
"val": {
"openTurnStartSeq": null,
"lastStepStartSeq": 59,
"lastStepBoundary": {
"kind": "end",
"seq": 70
},
"lastTurn": 3
}
},
"turnOutline": {
"ver": 2,
"seq": 71,
"val": {
"turns": [
{
"turn": 1,
"seq": 4,
"prompt": "请只回复一个词:pong",
"response": "pong"
},
{
"turn": 2,
"seq": 40,
"prompt": "请只回复一个词:pong2",
"response": "pong2"
},
{
"turn": 3,
"seq": 57,
"prompt": "请只回复一个词:pong3",
"response": "pong3"
}
],
"draft": ""
}
},
"permissions": {
"ver": 2,
"seq": 71,
"val": {
"preset": "workspace-write",
"sandbox": "workspace-write",
"approval": "ask",
"seeded": true
}
},
"modelSelection": {
"ver": 2,
"seq": 71,
"val": {
"lastUsed": {
"provider": "deepseek-official",
"model": "deepseek-v4-flash",
"reasoningEffort": "high"
},
"pending": null
}
},
"sessionListMetadata": {
"ver": 1,
"seq": 71,
"val": {
"blank": false,
"lastPromptAt": 1788287660106
}
},
"imageLimits": {
"ver": 1,
"seq": 71,
"val": null
},
"todos": {
"ver": 2,
"seq": 71,
"val": null
},
"plan": {
"ver": 3,
"seq": 71,
"val": {
"active": false,
"wanted": null,
"running": null,
"activeAtLastHeader": false
}
},
"subagentModelSelectionPolicy": {
"ver": 1,
"seq": 71,
"val": null
}
}
}
}

View file

@ -88,7 +88,10 @@ export class DomainFacility {
* (`facet-unsupported`); open the unit projected from the spec (backend
* `version-mismatch`/`malformed-medium` pass through); load and validate
* every stored record against the spec's zod schemas (`invalid-record`
* with the offending table and key); construct the domain.
* with the offending table and key — unless the spec declares
* `invalidRecords: 'backup-and-skip'` and the unit can move documents aside, in
* which case the failing record is backed up, logged, and skipped);
* construct the domain.
*
* Lifecycle: the CALLER owns the returned handle and closes it via
* `Domain.close()` (typically as its own `ctx.effect` disposer) — the
@ -118,7 +121,23 @@ export class DomainFacility {
for (const [table, tableSpec] of Object.entries(spec.tables)) {
const records = new Map<string, unknown>()
for (const [key, raw] of Object.entries(snapshot.tables[table] ?? {})) {
records.set(key, parseRecord(spec.name, table, key, () => tableSpec.valueSchema.parse(raw)))
let parsed: unknown
try {
parsed = parseRecord(spec.name, table, key, () => tableSpec.valueSchema.parse(raw))
} catch (error) {
// Backup-and-skip policy (disposable derived data): move the record's
// document aside, log the concrete failure, and open without the
// record. Backends that cannot move a document keep the loud path.
if (spec.invalidRecords !== 'backup-and-skip' || unit.backupRecord === undefined) throw error
const moved = await unit.backupRecord(table, key)
// parseRecord always wraps the zod failure as the cause.
this.ctx.logger.error(
`domain '${spec.name}': stored record '${key}' in table '${table}' failed schema validation; `
+ `moved to '${moved}' and treated as absent. Cause: ${String((error as DomainError).cause)}`,
)
continue
}
records.set(key, parsed)
}
tables.set(table, records)
}

View file

@ -35,16 +35,36 @@ export interface DomainTableSpec<K extends string = string, V = unknown> {
export interface DomainSpec {
/** Domain name; must match `UNIT_NAME_RE` (doubles as the backend unit name). */
readonly name: string
/** Domain format version; a medium stamped with a different version rejects at open. */
/** Current domain format version; reads enforce it according to the selected layout. */
readonly version: number
/**
* Medium layout for the backend unit: `single` (the default) stores the
* whole unit as one document; `per-record` stores each record as its own
* document, for units whose records are large, sparse, or individually
* disposable — the projection cache — and scopes version bumps per record
* (a stale record document is discarded, never migrated).
* disposable — the projection cache — and scopes version checks per record
* (an unaccepted record document is discarded, never migrated).
*/
readonly layout?: 'single' | 'per-record'
/**
* Older domain versions whose stored records the current record schemas
* also accept (the declaring owner vouches for that, typically by
* declaring the fields older records lack as optional). `per-record` backends
* read documents stamped with a listed version instead of discarding them,
* and accept a legacy whole-unit file so stamped for the one-time
* bootstrap; writes always stamp {@link version}.
*/
readonly compatibleVersions?: readonly number[]
/**
* What `open` does with a stored table record that fails its zod schema.
* Absent (the default), the whole open rejects with `invalid-record` —
* right for authoritative data. `'backup-and-skip'` is for domains whose
* records are disposable derived data: the backend moves the record's
* document aside (`KvUnit.backupRecord`), the failure is logged with
* its cause, and the open continues with the record absent. A backend
* without `backupRecord` (no per-record document to move) falls back
* to the rejecting default. The global slot always rejects.
*/
readonly invalidRecords?: 'backup-and-skip'
/** Optional global singleton slot. */
readonly global?: DomainGlobalSpec<unknown>
/** Table declarations keyed by table name; each name must match `UNIT_NAME_RE`. */
@ -91,6 +111,13 @@ export function defineDomain<S extends DomainSpec>(spec: S): S {
if (!Number.isInteger(spec.version) || spec.version < 0) {
throw new Error(`domain '${spec.name}' version must be a non-negative integer, got ${spec.version}`)
}
for (const compat of spec.compatibleVersions ?? []) {
if (!Number.isInteger(compat) || compat < 0 || compat >= spec.version) {
throw new Error(
`domain '${spec.name}' compatibleVersions entries must be non-negative integers below version ${spec.version}, got ${compat}`,
)
}
}
if (spec.layout !== undefined) {
// Runtime boundary: the union type is compile-time only — a spec built
// from config could carry any value, and a bad one must fail loud here.
@ -99,6 +126,12 @@ export function defineDomain<S extends DomainSpec>(spec: S): S {
throw new Error(`domain '${spec.name}' layout must be 'single' or 'per-record', got ${layout}`)
}
}
if (spec.invalidRecords !== undefined) {
const policy: string = spec.invalidRecords
if (policy !== 'backup-and-skip') {
throw new Error(`domain '${spec.name}' invalidRecords must be 'backup-and-skip' when present, got ${policy}`)
}
}
for (const table of Object.keys(spec.tables)) {
if (!UNIT_NAME_RE.test(table)) {
throw new Error(`domain '${spec.name}' table name '${table}' must match ${UNIT_NAME_RE}`)
@ -125,5 +158,6 @@ export function descriptorOf(spec: DomainSpec): KvUnitDescriptor {
tables: Object.keys(spec.tables),
hasGlobal: spec.global !== undefined,
...spec.layout === undefined ? {} : { layout: spec.layout },
...spec.compatibleVersions === undefined ? {} : { compatibleVersions: spec.compatibleVersions },
}
}

View file

@ -58,6 +58,25 @@ describe('defineDomain', () => {
})).toThrow(/must not accept null/)
})
it('validates compatibleVersions entries and projects them onto the descriptor', () => {
expect(() => defineDomain({ name: 'ok', version: 2, compatibleVersions: [1.5], tables: {} }))
.toThrow(/compatibleVersions/)
expect(() => defineDomain({ name: 'ok', version: 2, compatibleVersions: [2], tables: {} }))
.toThrow(/below version/)
expect(() => defineDomain({ name: 'ok', version: 2, compatibleVersions: [-1], tables: {} }))
.toThrow(/compatibleVersions/)
expect(descriptorOf(defineDomain({ name: 'ok', version: 2, compatibleVersions: [0, 1], tables: {} })))
.toMatchObject({ compatibleVersions: [0, 1] })
// An undeclared set is absent from the descriptor.
expect(descriptorOf(spec)).not.toHaveProperty('compatibleVersions')
})
it('rejects an unknown invalidRecords policy', () => {
expect(() => defineDomain({
name: 'ok', version: 1, invalidRecords: 'zap' as 'backup-and-skip', tables: {},
})).toThrow(/invalidRecords/)
})
it('rejects an invalid layout and projects the declared one onto the descriptor', () => {
// A spec built from config can carry any value; the union type is
// compile-time only, so the runtime boundary check must reject it.
@ -139,6 +158,28 @@ describe('DomainFacility.open', () => {
})
})
it('keeps the rejecting default under backup-and-skip when the backend cannot move documents', async () => {
// The memory backend has no backupRecord, so the declared policy cannot
// apply and the open falls back to failing loud.
const salvageSpec = defineDomain({
name: 'salvage',
version: 1,
invalidRecords: 'backup-and-skip',
tables: { items: domainTable<string, Item>(itemSchema) },
})
const pool = new MemoryMediaPool()
{
const { facility } = await harness({ pool })
await (await facility.open(salvageSpec)).table('items').put('bad', { label: 'x', count: 2 })
}
pool.media.get('salvage')!.tables.get('items')!.set('bad', { label: 'x', count: 'NaN' })
const { facility } = await harness({ pool })
await expect(facility.open(salvageSpec)).rejects.toMatchObject({
code: 'invalid-record',
detail: { table: 'items', key: 'bad' },
})
})
it('rejects a stored global that fails its schema with the global marker', async () => {
const pool = new MemoryMediaPool()
pool.versions.set('demo', 1)

View file

@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/storage/storage-json/README.md
README.md: 2972a617248c73e6105e8f19635e2325337405b6
README.zh.md: 5c9f52d2a12981f2bff36e4df198270d09f2a8e5
README.md: d36c6333392c3efe4462aebd82cf3ff9e76f66ae
README.zh.md: 9c266537945941d4bc779a2cb1a349e2409f21dc

View file

@ -53,9 +53,9 @@ The generated [configuration catalog](../../../docs/config-catalog.md#deepseek-a
### Observable behavior
A missing `single` file or `per-record` directory opens as an empty unit and materializes on the first write. In `single`, malformed content rejects with `malformed-medium`, and a different stored version rejects with `version-mismatch`. In `per-record`, each malformed, unreadable, or differently versioned document reads as an absent record, so one bad document does not reject the unit. Record keys must match `[a-zA-Z0-9_-]+`; an unsafe key rejects before any file operation. Every resolved write is durable, and operations after close reject with `closed`.
A missing `single` file or `per-record` directory opens as an empty unit and materializes on the first write. In `single`, malformed content rejects with `malformed-medium`, and a different stored version rejects with `version-mismatch`. In `per-record`, each malformed or unreadable document, and each document whose version is outside the descriptor's current and compatible versions, reads as an absent record, so one bad document does not reject the unit. Record keys must match `[a-zA-Z0-9_-]+`; an unsafe key rejects before any file operation. Every resolved write is durable, and operations after close reject with `closed`.
An empty `per-record` tree can initialize its declared tables from a valid `<root>/<unit>.json` whole-unit document only when the source unit name and version match the current descriptor. The backend leaves that source file unchanged. A different source version leaves the new tree empty. Any document path in a declared table, or a declared `global.json`, suppresses this initialization for the complete unit, even if that document is unreadable or stale.
An empty `per-record` tree can initialize its declared tables from a valid `<root>/<unit>.json` whole-unit document only when the source unit name matches and its version is current or declared compatible. The backend leaves that source file unchanged and stamps migrated records with the current version. A source version outside the accepted set leaves the new tree empty. Any document path in a declared table, or a declared `global.json`, suppresses this initialization for the complete unit, even if that document is unreadable or stale.
-----

View file

@ -53,9 +53,9 @@ kind: "package-reference"
### 可观察行为
缺失的 `single` 文件或 `per-record` 目录会作为空单元打开,并在第一次写入时物化。在 `single` 中,畸形内容以 `malformed-medium` 拒绝,不同的已存版本以 `version-mismatch` 拒绝。在 `per-record` 中,每份畸形、不可读或版本不同的文档都读作记录不存在,因此单个坏文档不会使单元被拒绝。记录键必须匹配 `[a-zA-Z0-9_-]+`;不安全的键在任何文件操作前被拒绝。每次已完成的写入都已持久化,关闭后的操作以 `closed` 拒绝。
缺失的 `single` 文件或 `per-record` 目录会作为空单元打开,并在第一次写入时物化。在 `single` 中,畸形内容以 `malformed-medium` 拒绝,不同的已存版本以 `version-mismatch` 拒绝。在 `per-record` 中,每份畸形或不可读的文档,以及版本不在 descriptor 当前版本和兼容版本内的文档,都读作记录不存在,因此单个坏文档不会使单元被拒绝。记录键必须匹配 `[a-zA-Z0-9_-]+`;不安全的键在任何文件操作前被拒绝。每次已完成的写入都已持久化,关闭后的操作以 `closed` 拒绝。
只有当源单元名称和版本与当前描述符相同时,空的 `per-record` 目录树才可以从有效的 `<root>/<unit>.json` 整单元文档初始化其已声明表。后端保持该源文件不变。源版本不同会使新目录树保持为空。已声明表中只要存在任意文档路径,或存在已声明的 `global.json`,就会对整个单元禁止该初始化,即使该文档不可读或版本陈旧。
只有当源单元名称匹配,且源版本为当前版本或已声明的兼容版本时,空的 `per-record` 目录树才可以从有效的 `<root>/<unit>.json` 整单元文档初始化其已声明表。后端保持该源文件不变,并为迁移的记录写入当前版本戳。接受集合之外的源版本会使新目录树保持为空。已声明表中只要存在任意文档路径,或存在已声明的 `global.json`,就会对整个单元禁止该初始化,即使该文档不可读或版本陈旧。
-----

View file

@ -100,16 +100,18 @@ export function serializeRecord(version: number, value: unknown): string {
/**
* Parse one per-record document, validating its version stamp. A document
* that is malformed or stamped with a different version is FOREIGN and reads
* as absent — the per-record contract: one bad or stale record file must not
* brick the whole unit, and a version bump discards stale records instead of
* migrating them (the whole-unit format rejects instead, because there is
* exactly one document).
* that is malformed or stamped with an unaccepted version is FOREIGN and
* reads as absent — the per-record contract: one bad or stale record file
* must not brick the whole unit, and an unaccepted version stamp discards the
* record instead of migrating it (the whole-unit format rejects instead,
* because there is exactly one document).
* @param text - Raw per-record document content.
* @param version - Expected unit version; a mismatch discards the document.
* @param versions - Accepted unit versions (the current one plus the
* descriptor's compatibleVersions); any other stamp discards the
* document.
* @returns the record value, or `undefined` for a foreign document.
*/
export function parseRecord(text: string, version: number): unknown {
export function parseRecord(text: string, versions: readonly number[]): unknown {
let document: unknown
try {
document = JSON.parse(text)
@ -118,6 +120,6 @@ export function parseRecord(text: string, version: number): unknown {
}
if (typeof document !== 'object' || document === null) return undefined
const { version: stamped, record } = document as Record<string, unknown>
if (stamped !== version) return undefined
if (typeof stamped !== 'number' || !versions.includes(stamped)) return undefined
return record
}

View file

@ -10,21 +10,23 @@
* memory unchanged.
*
* Per-record contract: a record document that is malformed or stamped with a
* different version reads as an absent record — one bad or stale file never
* bricks the whole unit, and a version bump discards stale records instead
* of migrating them. Record keys become path segments, so they must be
* path-safe (`[a-zA-Z0-9_-]+`); an unsafe key rejects at write.
* version outside the accepted set (the descriptor's current version plus
* its `compatibleVersions`) reads as an absent record — one bad or stale
* file never bricks the whole unit, and an unaccepted version stamp discards
* the record instead of migrating it. Record keys become path segments, so
* they must be path-safe (`[a-zA-Z0-9_-]+`); an unsafe key rejects at write.
*
* Legacy bootstrap: when the new tree has no document path, a legacy
* whole-unit file `<root>/<name>.json` (the pre-per-record layout) seeds
* per-record documents only when its unit name and version match the current
* descriptor. Any new document path, including one whose contents are
* per-record documents, provided its stored unit version is in the accepted
* set — a legacy file stamped with any other version is left alone and reads
* as the empty unit. Any new document path, including one whose contents are
* unreadable or stale, suppresses the bootstrap for the whole unit. The
* legacy file is never changed or deleted.
* @module @deepseek-ai/dsh-storage-json/src/per-record-unit
*/
import { mkdir, readFile, readdir, rm } from 'node:fs/promises'
import { mkdir, readFile, readdir, rename, rm } from 'node:fs/promises'
import { dirname, join } from 'node:path'
import type { Dirent } from 'node:fs'
import { StorageError } from '@deepseek-ai/dsh-storage'
@ -58,13 +60,14 @@ export async function openPerRecordUnit(
* Read every record document under the unit directory: each declared table's
* `<key>.json` files plus `global.json`. A missing directory is the empty
* unit (materialization defers to the first write); a foreign document
* (missing, malformed, or stamped with another version) reads as an absent
* (missing, malformed, or stamped with an unaccepted version) reads as an absent
* record, per the per-record contract.
* @param descriptor - Static identity and shape of the unit.
* @param dir - Absolute unit directory path.
* @returns the authoritative state reconstructed from the tree.
*/
async function loadPerRecordState(descriptor: KvUnitDescriptor, dir: string): Promise<UnitState> {
const versions = acceptedStamps(descriptor)
const state: UnitState = {
version: descriptor.version,
global: null,
@ -84,11 +87,11 @@ async function loadPerRecordState(descriptor: KvUnitDescriptor, dir: string): Pr
if (entry.isDirectory()) {
const records = state.tables.get(entry.name)
if (records !== undefined) {
return loadTableRecords(records, descriptor.version, join(dir, entry.name))
return loadTableRecords(records, versions, join(dir, entry.name))
}
}
if (entry.name === 'global.json' && descriptor.hasGlobal) {
const global = await readRecord(join(dir, entry.name), descriptor.version)
const global = await readRecord(join(dir, entry.name), versions)
if (global !== undefined) state.global = global
return true
}
@ -98,13 +101,21 @@ async function loadPerRecordState(descriptor: KvUnitDescriptor, dir: string): Pr
return state
}
/** The version stamps this unit reads as its own: current plus declared compatible versions. */
function acceptedStamps(descriptor: KvUnitDescriptor): readonly number[] {
return [descriptor.version, ...descriptor.compatibleVersions ?? []]
}
/**
* Bootstrap an empty per-record tree from a legacy whole-unit file
* (`<root>/<name>.json`, the pre-per-record layout). Every declared-table
* record is copied into a same-version document, while the legacy file is
* retained unchanged. A missing, foreign (another unit name or version),
* malformed, or non-unit legacy file is left alone; other read failures
* propagate.
* record is copied into a current-version document, while the legacy file is
* retained unchanged. A missing, foreign (another unit's name), malformed,
* or non-unit legacy file is left alone, and so is one whose stored unit
* version is outside the accepted set — migrating records the owner never
* vouched for would stamp them with the current version and turn a
* discardable stale cache into schema failures at the domain layer. Other
* read failures propagate.
* @param descriptor - Static identity and shape of the unit.
* @param dir - The per-record unit directory (`<root>/<name>`).
* @param state - The empty tree state; bootstrapped records are added.
@ -118,15 +129,18 @@ async function bootstrapLegacyUnit(descriptor: KvUnitDescriptor, dir: string, st
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error
return
}
// The legacy document is runtime data: its unit identity and the tables map
// shape are checked here; the domain layer's schemas judge record values.
// The legacy document is runtime data: only `unit.name`, `unit.version`,
// and the tables map shape are checked here — the record values are
// migrated as-is and the domain layer's schemas judge them.
let document: { unit?: { name?: unknown; version?: unknown }; tables?: unknown }
try {
document = JSON.parse(text) as { unit?: { name?: unknown; version?: unknown }; tables?: unknown }
} catch {
return // Malformed legacy file: not ours to interpret or delete.
}
if (document.unit?.name !== descriptor.name || document.unit.version !== descriptor.version) return
if (document.unit?.name !== descriptor.name) return
const stamped = document.unit.version
if (typeof stamped !== 'number' || !acceptedStamps(descriptor).includes(stamped)) return
const tables = document.tables
if (typeof tables !== 'object' || tables === null) return
const recordsByTable = tables as Record<string, Record<string, unknown>>
@ -147,14 +161,14 @@ async function bootstrapLegacyUnit(descriptor: KvUnitDescriptor, dir: string, st
* @returns whether the directory contains any `.json` document path,
* independently of key safety, readability, or stored version.
*/
async function loadTableRecords(records: Map<string, unknown>, version: number, dir: string): Promise<boolean> {
async function loadTableRecords(records: Map<string, unknown>, versions: readonly number[], dir: string): Promise<boolean> {
const files = await readdir(dir, { withFileTypes: true })
const hasDocuments = files.some(file => file.name.endsWith('.json'))
const loaded = await Promise.all(files.map(async (file) => {
if (!file.name.endsWith('.json')) return
const key = file.name.slice(0, -'.json'.length)
if (!SAFE_KEY_RE.test(key)) return
const record = await readRecord(join(dir, file.name), version)
const record = await readRecord(join(dir, file.name), versions)
if (record !== undefined) return [key, record] as const
}))
for (const record of loaded) {
@ -164,9 +178,9 @@ async function loadTableRecords(records: Map<string, unknown>, version: number,
}
/** Read one record document; a foreign (unreadable or stale) one reads as absent. */
async function readRecord(path: string, version: number): Promise<unknown> {
async function readRecord(path: string, versions: readonly number[]): Promise<unknown> {
try {
return parseRecord(await readFile(path, 'utf8'), version)
return parseRecord(await readFile(path, 'utf8'), versions)
} catch {
return undefined
}
@ -214,6 +228,22 @@ export class PerRecordJsonUnit implements KvUnit {
await this.tracked(rm(join(this.tableDir(table), `${key}.json`), { force: true }))
}
/**
* Move one record's document aside as `<key>.json.bak.<YYYYMMDDHHmm>`. The
* moved file no longer ends in `.json`, so every later read ignores it; the
* bytes stay on disk for inspection. A same-minute backup of the same
* key overwrites the previous backup (the newer bytes are the ones worth
* keeping).
*/
async backupRecord(table: string, key: string): Promise<string> {
this.assertOpen()
assertSafeKey(this.descriptor.name, key)
const path = join(this.tableDir(table), `${key}.json`)
const moved = `${path}.bak.${backupStamp(new Date())}`
await this.tracked(rename(path, moved))
return moved
}
/** Durably replace the global singleton. Only valid when declared. */
async setGlobal(value: unknown): Promise<void> {
this.assertOpen()
@ -268,6 +298,12 @@ export class PerRecordJsonUnit implements KvUnit {
}
}
/** Local-time `YYYYMMDDHHmm` suffix for backed-up documents. */
function backupStamp(now: Date): string {
const pad = (value: number): string => String(value).padStart(2, '0')
return `${String(now.getFullYear())}${pad(now.getMonth() + 1)}${pad(now.getDate())}${pad(now.getHours())}${pad(now.getMinutes())}`
}
/** Reject a record key that would be unsafe as a path segment. */
function assertSafeKey(unit: string, key: string): void {
if (!SAFE_KEY_RE.test(key)) {

View file

@ -347,10 +347,14 @@ describe('per-record layout', () => {
await backend.close()
})
it('leaves an older-version legacy whole-unit file unconverted', async () => {
it('bootstraps from a legacy file only when its stored version is accepted', async () => {
// Version 3 is neither current (2) nor declared compat: the legacy file
// is left alone and the unit reads empty — migrating unvouched records
// would stamp them current and surface as schema failures at the domain
// layer instead of a discardable stale cache.
const root = await freshRoot()
const legacy = JSON.stringify({
unit: { name: 'recs', version: descriptor.version - 1 },
unit: { name: 'recs', version: 3 },
global: null,
tables: { t: { old: { v: 1 } } },
})
@ -360,6 +364,56 @@ describe('per-record layout', () => {
expect(await unit.loadAll()).toEqual({ tables: { t: {} }, global: null })
await expect(readFile(recordPath(root, 'old'), 'utf8')).rejects.toMatchObject({ code: 'ENOENT' })
await expect(readFile(join(root, 'recs.json'), 'utf8')).resolves.toBe(legacy)
await unit.close()
await backend.close()
// The same file bootstraps once version 3 is declared read-compatible…
const root2 = await freshRoot()
await writeFile(join(root2, 'recs.json'), legacy, 'utf8')
const backend2 = new JsonStorageBackend(root2)
const compat = { ...descriptor, version: 4, compatibleVersions: [3] }
const unit2 = await backend2.kv.open(compat)
expect(await unit2.loadAll()).toEqual({ tables: { t: { old: { v: 1 } } }, global: null })
// …and the migrated documents are stamped with the CURRENT version.
expect(JSON.parse(await readFile(join(root2, 'recs', 't', 'old.json'), 'utf8')))
.toEqual({ version: 4, record: { v: 1 } })
await unit2.close()
await backend2.close()
})
it('backupRecord moves the document aside; reads see it absent and a write recreates it', async () => {
const root = await freshRoot()
const backend = new JsonStorageBackend(root)
const unit = await backend.kv.open(descriptor)
await unit.putRecord('t', 'k', { v: 1 })
const moved = await unit.backupRecord!('t', 'k')
expect(moved).toMatch(/k\.json\.bak\.\d{12}$/)
expect(JSON.parse(await readFile(moved, 'utf8'))).toEqual({ version: 2, record: { v: 1 } })
await expect(readFile(recordPath(root, 'k'), 'utf8')).rejects.toMatchObject({ code: 'ENOENT' })
// The moved file no longer ends in .json, so it reads as absent…
expect(await unit.loadAll()).toEqual({ tables: { t: {} }, global: null })
// …and the key is free for a fresh write.
await unit.putRecord('t', 'k', { v: 2 })
expect(await unit.loadAll()).toEqual({ tables: { t: { k: { v: 2 } } }, global: null })
await expect(unit.backupRecord!('t', 'a/b')).rejects.toThrow(/not path-safe/)
await unit.close()
await expect(unit.backupRecord!('t', 'k')).rejects.toMatchObject({ code: 'closed' })
await backend.close()
})
it('reads per-record documents stamped with a declared compat version and stamps writes current', async () => {
const root = await freshRoot()
const backend = new JsonStorageBackend(root)
const compat = { ...descriptor, compatibleVersions: [1] }
await mkdir(join(root, 'recs', 't'), { recursive: true })
await writeFile(recordPath(root, 'oldrec'), JSON.stringify({ version: 1, record: { v: 'old' } }), 'utf8')
await writeFile(recordPath(root, 'ancient'), JSON.stringify({ version: 0, record: { v: 'no' } }), 'utf8')
const unit = await backend.kv.open(compat)
// Version 1 is declared compat and served; version 0 is not and discards.
expect(await unit.loadAll()).toEqual({ tables: { t: { oldrec: { v: 'old' } } }, global: null })
await unit.putRecord('t', 'oldrec', { v: 'new' })
expect(JSON.parse(await readFile(recordPath(root, 'oldrec'), 'utf8')))
.toEqual({ version: 2, record: { v: 'new' } })
await backend.close()
})
@ -416,6 +470,7 @@ describe('per-record layout', () => {
await backend4.close()
const root5 = await freshRoot()
// A current-version stamp so the shapeless `tables` is what stops the bootstrap.
await writeFile(
join(root5, 'recs.json'),
JSON.stringify({ unit: { name: 'recs', version: descriptor.version }, tables: 'not an object' }),

View file

@ -56,11 +56,21 @@ export interface KvUnitDescriptor {
* Medium layout. `single` (the default) keeps the whole unit in one
* document; `per-record` keeps each record in its own document, so a unit
* whose records are large or sparse never rewrites the rest on one write,
* and a version bump discards stale records instead of rejecting the whole
* unit. Backends that only serve one layout accept the other's units as
* foreign documents.
* and an unaccepted version stamp discards only that record instead of
* rejecting the whole unit. Backends that only serve one layout accept the
* other's units as foreign documents.
*/
readonly layout?: 'single' | 'per-record'
/**
* Older unit versions whose stored records are also readable under the
* declaring owner's current record schemas (the owner vouches for that —
* typically by declaring the fields old records lack as optional). Reads of
* a `per-record` unit accept documents stamped with any listed version, and
* the legacy whole-unit bootstrap accepts a legacy file stamped with one;
* writes always stamp {@link version}. `single`-layout reads stay
* exact-version.
*/
readonly compatibleVersions?: readonly number[]
}
/**
@ -99,6 +109,19 @@ export interface KvUnit {
*/
deleteRecord(table: string, key: string): Promise<void>
/**
* Move one record's stored document out of the unit's readable set,
* preserving its bytes for inspection instead of deleting them. Backends
* whose medium has no per-record document to move (the `single` layout, a
* row store) omit this member, and the caller falls back to its
* reject-loud path. Absent after the move: a later {@link loadAll} reads
* the key as missing and a later {@link putRecord} recreates it fresh.
* @param table - Declared table name.
* @param key - Record key.
* @returns the medium location the document was moved to (diagnostics).
*/
backupRecord?(table: string, key: string): Promise<string>
/**
* Write the global singleton durably. Only valid when the descriptor
* declared `hasGlobal`.