diff --git a/.agents/notes/implemented/architecture/2026-08-20-webworker-node-face.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-20-webworker-node-face.i18n.yaml index 4b5a98a8d5..bf06cd8dee 100644 --- a/.agents/notes/implemented/architecture/2026-08-20-webworker-node-face.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-08-20-webworker-node-face.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-20-webworker-node-face.md -2026-08-20-webworker-node-face.md: 08119cce96eff244f8e9ada3462ce5d35c1b538d -2026-08-20-webworker-node-face.zh.md: 573c0be055d066d2d6d0db11a2476ba528517727 +2026-08-20-webworker-node-face.md: 41a30dedc7df9a882fbc1d8d3e3583c0a3602d81 +2026-08-20-webworker-node-face.zh.md: b57481335808f3e1a764da123a11ea74ba6cf371 diff --git a/.agents/notes/implemented/architecture/2026-08-20-webworker-node-face.md b/.agents/notes/implemented/architecture/2026-08-20-webworker-node-face.md index 08119cce96..41a30dedc7 100644 --- a/.agents/notes/implemented/architecture/2026-08-20-webworker-node-face.md +++ b/.agents/notes/implemented/architecture/2026-08-20-webworker-node-face.md @@ -6,21 +6,21 @@ English | [中文](2026-08-20-webworker-node-face.zh.md) ## Problem -The worker runs the web profile's Cordis configuration byte for byte — no worker-specific rows — so a browser's missing platform must be replaced at the module layer, where a proxied module keeps its identity and changes its implementation. That covers three fronts: the Node builtins the tree imports, the filesystem those builtins answer from, and a process layer for the bash tool, which mounted, advertised itself to the model, and then failed on every call while `node:child_process` was a structural stub. +The worker runs the web profile's Cordis configuration byte for byte — no worker-specific rows — so a browser's missing platform must be replaced at the module layer, where a proxied module keeps its identity and changes its implementation. That covers three fronts: the Node builtins the tree imports, the filesystem those builtins answer from, and a process layer for the bash tool. A structural `node:child_process` stub would let that tool mount and advertise itself to the model while every call fails. ## Decision -**Builtins.** The proxy table replaces Node builtins and external npm packages, never workspace or vendored modules. `./implemented/.ts` carries real semantics over a worker data source; `./mock/.ts` mounts silently and reports the missing capability when a call reaches it. The loader's table holds one memoized thunk per specifier — evaluation happens at first `require`, not at assembly — and each shim's exported face typechecks against Node's own module type, with the narrow, documented exceptions where structural identity (a real class) cannot be satisfied. The worker installs the `process` global itself and fills it into the table at assembly. +**Builtins.** The proxy table replaces Node builtins and external npm packages, never workspace or vendored modules. `./implemented/.ts` carries real semantics over a worker data source; `./mock/.ts` mounts silently and reports the missing capability when a call reaches it. The loader's table holds one memoized thunk per specifier — evaluation happens at first `require`, not at assembly — and each shim's exported face typechecks against Node's own module type, with the narrow, documented exceptions where structural identity (a real class) cannot be satisfied. Its `createRequire` face supplies both `resolve()` and `resolve.paths()` against the image's package root, allowing unchanged packages to discover manifests without loading targets. The worker installs the `process` global itself and fills it into the table at assembly. -**VFS.** Memory is the truth. `statSync(path, { bigint: true })` returns Node's BigInt shape, and two fields carry real information because `dsh-fs-local`'s stale-write guard depends on them: `ino` is per-path identity from a monotonic counter (a recreated path reports a new identity), and `mtimeMs` is strictly increasing per entry (`max(now, previous + 1)`), because in-memory writes routinely land in one millisecond and an equal timestamp would let a stale overwrite pass. The hunt that produced this also fixed the silence around it: cordis's logger verbosity counts UP, so an exporter that declares no level drops every warning — `startWorkerHost` installs a console exporter with `levels: { default: 2 }` before any entry mounts. +**VFS.** Memory is the truth. `statSync(path, { bigint: true })` returns Node's BigInt shape, and two fields carry real information because `dsh-fs-local`'s stale-write guard depends on them: `ino` is per-path identity from a monotonic counter (a recreated path reports a new identity), and `mtimeMs` is strictly increasing per entry (`max(now, previous + 1)`), because in-memory writes routinely land in one millisecond and an equal timestamp would let a stale overwrite pass. Committed mutations also drive the [Node-compatible watcher and confinement implementation](2026-08-23-webworker-vfs-watch-and-landlock.md). Boot diagnostics remain visible because cordis logger verbosity counts UP: `startWorkerHost` installs a console exporter with `levels: { default: 2 }` before any entry mounts, while an exporter with no declared level drops every warning. -**Shell.** `node:child_process` is a real implementation over the VFS. The grammar is bought — `@yarnpkg/parsers`' `parseShell` — and the evaluator and command table are owned, because every candidate interpreter brings its own filesystem: pipelines are strings handed along, and each program is a function over the VFS. The table is the machine's whole `/bin`; an absent name reports `command not found` (127). Each `spawn` starts a child Web Worker from this same bundle, its first frame declaring the shell-process role, so the termination ladder is real: `SIGTERM` asks at the next command boundary, `SIGKILL` terminates the worker mid-loop — the preemption an in-thread interpreter can never have. The filesystem face is asynchronous end to end (child frames to the host VFS); `execSync`, `execFileSync`, and `fork` refuse, and `node-pty` stays a stub. +**Shell.** `node:child_process` is a real implementation over the VFS. The grammar is bought — `@yarnpkg/parsers`' `parseShell` — and the evaluator and command table are owned, because every candidate interpreter brings its own filesystem: pipelines are strings handed along, and each program is a function over the VFS. Ordinary commands resolve from that table; native-package protocols may contribute Worker-owned virtual executable wrappers through the [watcher and confinement decision](2026-08-23-webworker-vfs-watch-and-landlock.md). A name in neither set reports `ENOENT` at direct spawn or `command not found` (127) inside shell source. Each `spawn` starts a child Web Worker from this same bundle, its first frame declaring the shell-process role, so the termination ladder is real: `SIGTERM` asks at the next command boundary, `SIGKILL` terminates the worker mid-loop — the preemption an in-thread interpreter can never have. The filesystem face is asynchronous end to end (child frames to the host VFS); `execSync`, `execFileSync`, and `fork` refuse, and `node-pty` stays a stub. ## Alternatives considered **Replacing `dsh-subprocess-local` or the bash executor.** The first would let the proxy table replace a workspace package against its own classification and invert the layering; the second trips `dsh-permission-presets`' boot-time `sandboxMode` validation and drops tested timeout/output behavior. -**`@yarnpkg/shell`, WASM shells, WebContainer.** The matching interpreter is built on real Node streams (~1.5 MB closure to own); WASM was removed from this deployment by decision and WASI has no `fork`; all of them arrive with their own filesystem, the one part that cannot be reused. +**`@yarnpkg/shell`, WASM shells, WebContainer.** The matching interpreter is built on real Node streams (~1.5 MB closure to own); this deployment excludes WASM and WASI has no `fork`; all of them arrive with their own filesystem, the one part that cannot be reused. **`SharedArrayBuffer` + `Atomics.wait` for a synchronous child filesystem.** Measured on the deployment target: without COOP/COEP headers `SharedArrayBuffer` is not defined, and GitHub Pages cannot set response headers. The asynchronous face is a superset; a SAB backend can slot under it later without touching a program. @@ -28,7 +28,7 @@ The worker runs the web profile's Cordis configuration byte for byte — no work ## Consequences -- Sandbox modes other than `danger-full-access` fail loud: `SandboxEnforcement` has no "nothing was enforced" value and a browser has no kernel, so `ctx.sandbox.confine` fails closed and the command never starts. Real enforcement at the VFS frame gate is a designed follow-up, not this note. +- `read-only` and `workspace-write` interpret the native Landlock launcher protocol and enforce per-process grants at the VFS frame gate; `danger-full-access` keeps the direct process path. The [watcher and confinement decision](2026-08-23-webworker-vfs-watch-and-landlock.md) owns the narrower meaning of `full` in this execution world. - The Node-host ladder test (`tests/node/child-process.spec.ts`) is registered windows-unsupported: the ladder's win32 kill rung is taskkill-by-real-pid, undeliverable to a process-table pid, while the worker itself always reports `linux`. - Output is incremental but not streamed: programs write into sinks forwarded as `data` events, and a pipeline stage completes before the next starts. - The runtime's tests mirror `src/` (`tests/node/`, `tests/shell/`, `tests/storage/`, …), so each shim family owns its behavior cases beside the oracle-diff suites. diff --git a/.agents/notes/implemented/architecture/2026-08-20-webworker-node-face.zh.md b/.agents/notes/implemented/architecture/2026-08-20-webworker-node-face.zh.md index 573c0be055..b574813358 100644 --- a/.agents/notes/implemented/architecture/2026-08-20-webworker-node-face.zh.md +++ b/.agents/notes/implemented/architecture/2026-08-20-webworker-node-face.zh.md @@ -6,21 +6,21 @@ ## 问题 -worker 逐字节运行 web profile 的 Cordis 配置——没有 worker 专属行——因此浏览器缺失的平台必须在模块层被替换：被代理的模块保持身份、更换实现。这覆盖三条战线：树所 import 的 Node builtin、这些 builtin 背后应答的文件系统，以及 bash 工具的进程层——在 `node:child_process` 还是结构桩的时期，工具照常挂载、向模型自我宣告，然后每次调用都失败。 +worker 逐字节运行 web profile 的 Cordis 配置——没有 worker 专属行——因此浏览器缺失的平台必须在模块层被替换：被代理的模块保持身份、更换实现。这覆盖三条战线：树所 import 的 Node builtin、这些 builtin 背后应答的文件系统，以及 bash 工具的进程层。如果 `node:child_process` 只是结构桩，工具仍会照常挂载并向模型自我宣告，但每次调用都会失败。 ## 决定 -**Builtin。** 代理表只替换 Node builtin 与外部 npm 包，绝不替换 workspace 或 vendored 模块。`./implemented/.ts` 在 worker 数据源之上承载真语义；`./mock/.ts` 静默挂载、在调用真正抵达时报告缺失的能力。装载器的表按 specifier 各持一个 memoized thunk——求值发生在首次 `require` 而非装配期——且每个垫片的导出面对 Node 自身的模块类型作类型检查，仅在结构身份（真实类）确不可满足处留最窄的、有说明的例外。`process` 全局由 worker 自装，装配期填入表中。 +**Builtin。** 代理表只替换 Node builtin 与外部 npm 包，绝不替换 workspace 或 vendored 模块。`./implemented/.ts` 在 worker 数据源之上承载真语义；`./mock/.ts` 静默挂载、在调用真正抵达时报告缺失的能力。装载器的表按 specifier 各持一个 memoized thunk——求值发生在首次 `require` 而非装配期——且每个垫片的导出面对 Node 自身的模块类型作类型检查，仅在结构身份（真实类）确不可满足处留最窄的、有说明的例外。它的 `createRequire` 面在镜像 package 根之上同时提供 `resolve()` 与 `resolve.paths()`，使未修改的包无需加载目标即可发现 manifest。`process` 全局由 worker 自装，装配期填入表中。 -**VFS。** 内存为真相。`statSync(path, { bigint: true })` 返回 Node 的 BigInt 形状，其中两个字段承载真实信息，因为 `dsh-fs-local` 的 stale-write guard 依赖它们：`ino` 是按路径的身份（单调计数器分配，路径重建即新身份），`mtimeMs` 按条目严格递增（`max(now, previous + 1)`）——内存写例行落在同一毫秒内，相等的时间戳会放过陈旧覆写。这场排查同时修掉了它周围的静默：cordis 日志器的详细度数值向上计数，未声明等级的 exporter 会丢掉所有 warning——`startWorkerHost` 在任何 entry 挂载前安装 `levels: { default: 2 }` 的 console exporter。 +**VFS。** 内存为真相。`statSync(path, { bigint: true })` 返回 Node 的 BigInt 形状，其中两个字段承载真实信息，因为 `dsh-fs-local` 的 stale-write guard 依赖它们：`ino` 是按路径的身份（单调计数器分配，路径重建即新身份），`mtimeMs` 按条目严格递增（`max(now, previous + 1)`）——内存写例行落在同一毫秒内，相等的时间戳会放过陈旧覆写。已提交的 mutation 还会驱动 [Node 兼容 watcher 与 confinement 实现](2026-08-23-webworker-vfs-watch-and-landlock.zh.md)。Cordis 日志器的详细度数值向上计数，因此 `startWorkerHost` 会在任何 entry 挂载前安装 `levels: { default: 2 }` 的 console exporter，避免未声明等级的 exporter 丢掉所有 warning。 -**Shell。** `node:child_process` 是 VFS 之上的真实现。语法是买来的——`@yarnpkg/parsers` 的 `parseShell`——求值器与命令表是自有的，因为每个候选解释器都自带文件系统：管道是逐段传递的字符串，每个程序是 VFS 上的一个函数。命令表就是这台机器的全部 `/bin`；不存在的名字报告 `command not found`（127）。每次 `spawn` 从同一个 bundle 起一个子 Web Worker，首帧声明 shell 进程角色，因此终止梯是真的：`SIGTERM` 在下一命令边界处请求停止，`SIGKILL` 在任意时刻终止 worker——这是线程内解释器永远没有的抢占。文件系统面端到端异步（子进程经帧到宿主 VFS）；`execSync`、`execFileSync`、`fork` 拒绝，`node-pty` 保持桩。 +**Shell。** `node:child_process` 是 VFS 之上的真实现。语法是买来的——`@yarnpkg/parsers` 的 `parseShell`——求值器与命令表是自有的，因为每个候选解释器都自带文件系统：管道是逐段传递的字符串，每个程序是 VFS 上的一个函数。普通命令从该表解析；native 包协议可以通过 [watcher 与 confinement 决策](2026-08-23-webworker-vfs-watch-and-landlock.zh.md)提供 Worker 自有的虚拟 executable wrapper。两处都没有的名字在直接 spawn 时报告 `ENOENT`，在 shell source 中则报告 `command not found`（127）。每次 `spawn` 从同一个 bundle 起一个子 Web Worker，首帧声明 shell 进程角色，因此终止梯是真的：`SIGTERM` 在下一命令边界处请求停止，`SIGKILL` 在任意时刻终止 worker——这是线程内解释器永远没有的抢占。文件系统面端到端异步（子进程经帧到宿主 VFS）；`execSync`、`execFileSync`、`fork` 拒绝，`node-pty` 保持桩。 ## 曾考虑的替代方案 **整包替换 `dsh-subprocess-local` 或替换 bash 执行器。** 前者让代理表首次替换 workspace 包、违背其自身分类并倒置分层；后者撞上 `dsh-permission-presets` 对 `sandboxMode` 的 boot 期硬校验，并丢掉执行器已被测试钉住的超时/输出行为。 -**`@yarnpkg/shell`、WASM shell、WebContainer。** 配套解释器建立在真实 Node streams 之上（约 1.5 MB 闭包要自养）；WASM 已被本部署的决定排除，WASI 没有 `fork`；且它们全都自带文件系统——恰是无法复用的那部分。 +**`@yarnpkg/shell`、WASM shell、WebContainer。** 配套解释器建立在真实 Node streams 之上（约 1.5 MB 闭包要自养）；本部署排除 WASM，WASI 没有 `fork`；且它们全都自带文件系统——恰是无法复用的那部分。 **`SharedArrayBuffer` + `Atomics.wait` 给子进程同步文件系统。** 在部署目标实测：无 COOP/COEP 头时 `SharedArrayBuffer` 未定义，而 GitHub Pages 无法设置响应头。异步面是超集；SAB 后端将来可垫入其下而不动任何程序。 @@ -28,7 +28,7 @@ worker 逐字节运行 web profile 的 Cordis 配置——没有 worker 专属 ## 后果 -- `danger-full-access` 之外的沙箱档 fail loud：`SandboxEnforcement` 没有「未执法」值、浏览器没有内核，`ctx.sandbox.confine` 落闭、命令零启动。在 VFS 帧闸口做真执法是设计中的后续，不属本条。 +- `read-only` 与 `workspace-write` 解释 native Landlock launcher 协议，并在 VFS 帧闸口执行逐进程授权；`danger-full-access` 保持直接进程路径。[Watcher 与 confinement 决策](2026-08-23-webworker-vfs-watch-and-landlock.zh.md)拥有该执行世界中 `full` 的更窄含义。 - Node 宿主的阶梯测试（`tests/node/child-process.spec.ts`）登记为 windows 不支持：阶梯的 win32 kill 梯级是按真 pid 的 taskkill，对进程表 pid 不可投递，而 worker 自身恒报 `linux`。 - 输出增量但不流式：程序写入的 sink 以 `data` 事件转发，一个管道阶段完成后下一阶段才开始。 - 运行时的测试镜像 `src/`（`tests/node/`、`tests/shell/`、`tests/storage/`……），每个垫片族在 oracle-diff 套件旁拥有自己的行为用例。 diff --git a/.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.i18n.yaml index b93c9676cb..b4cef0f471 100644 --- a/.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.md -2026-08-20-webworker-pack-lowering-and-preview.md: 37c7730c5da664560156a8e4bd9ba594c78369ee -2026-08-20-webworker-pack-lowering-and-preview.zh.md: 09d9356ee8744ba2406bd765a6ca6a060f93853a +2026-08-20-webworker-pack-lowering-and-preview.md: 72a6ccf856c95f835e103bd355223bf3cf42f692 +2026-08-20-webworker-pack-lowering-and-preview.zh.md: 074d44833c34a2999a5c47da809533065201b580 diff --git a/.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.md b/.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.md index 37c7730c5d..72a6ccf856 100644 --- a/.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.md +++ b/.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.md @@ -14,6 +14,8 @@ The browser worker can neither compile modules at load nor be served by the prod **The preview is the served page plus one tag.** One Vite build emits `dist/index.html` and `dist/preview.html` sharing every chunk; the only difference is a prepended bootstrap entry whose module connects the worker host. Startup then converges on one protocol: whichever side applies the injection table settles the `__DSH_BOOT_READY__` deferred — the served renderer resolves it in a tail script after the rendered rows, the worker bootstrap installs it before its first await and settles it after the last row — and the client entry awaits it before reading any injected state, so the chain from the stock entry onward is the served chain verbatim. The build uses a relative base so the output mounts under any static directory; the served form anchors deep SPA-fallback paths by rendering `` at serve time, keeping the on-disk pages byte-shared. +**The repository preview carries selectable filesystem sources.** The packer emits one base image and a small overlay archive for each named built-in fixture. Without a source query, `preview.html` waits at a chooser for an empty filesystem, the built-in fixtures, or the separately owned WebFS provider. A valid `preview-fixture=none|` query selects directly and skips the chooser for deterministic browser runs; its distinct name avoids the Client's existing `fixture` transport switch. The Worker mounts the base and then applies the selected overlays in order, restricted to `home/` and `workspace/`, before it validates the base manifest or boots Cordis. `packages/experimental/webworker-runtime/tests/fixtures/vfs-example/` supplies one built-in overlay without giving the packer Session or Workspace knowledge. Its plaintext JSONL logs use the persistence backend's real project/session directory layout, so Session Persistence reads them cold and Workspace Registry derives the Workspace from their `/dsh/workspace` headers. The main Session exceeds the Client's 50-message page and keeps representative tool results at its tail; persisted one-shot and continuable children exercise the subagent catalog. WebFS authorization and user data remain a separate provider and never share this fixture tree. + Both packages live in `packages/experimental/` as `@deepseek-ai/dsh-experimental-*`, private and outside official releases. The boundary that carries product promises stays in the product packages: the injection table, `__DSH_TRANSPORT__`, and the `/plugins` bundle bytes are owned by `dsh-host-webserver`, `dsh-client-modules`, and `dsh-client-connection`. ## Alternatives considered @@ -26,6 +28,12 @@ Both packages live in `packages/experimental/` as `@deepseek-ai/dsh-experimental **Gating the stock entry on top-level await ordering instead of a deferred.** Sibling module scripts do not wait for one another's top-level awaits; the `??=`-installed deferred makes the handshake order-independent and lets a failed handshake reject into the boot page's failure rendering. +**Generate example state in the Worker at startup.** A preview-only Session or Workspace creation branch would bypass cold persistence loading and make the runtime own test data. Static image files exercise the same discovery and pagination path as existing user data. + +**Seed the example through WebFS.** WebFS owns user-selected durable storage and its lifecycle. Coupling the built-in demonstration to it would make a static preview depend on browser persistence state and would blur which bytes came from the deployment. + +**Pack one complete base image per fixture.** Full-image variants duplicate the runtime package closure and make combinations quadratic. Restricted overlays keep one immutable base, let the chooser compose zero or more data sources, and give future WebFS hydration the same pre-boot application point. + ## Consequences - `lib/worker.js` contains no parser (423.5 kB → 246.3 kB at the time of the cut, before the shell process layer landed). @@ -33,3 +41,4 @@ Both packages live in `packages/experimental/` as `@deepseek-ai/dsh-experimental - The transform corpus imports every built bundle through Node before comparing its lowered exports. Its pinned exemptions name the actual non-importable bundle and fail when one becomes importable: after Win32 process primitives became the Koffi type owner, `win32-process` carries the duplicate-type exemption and `sandbox-windows-acl` does not. - The served `` anchor exists because relative asset URLs would resolve under the request directory on SPA-fallback paths; remove it only together with the relative build base. - The image ships as a deterministically gzip-compressed tar (`vfs-image.tar.gz`; MTIME 0, OS byte 0xff): static hosts do not compress binary content types (type allowlists, CDN size caps), so the compression rides the artifact, and the worker inflates the fetch body through the browser's native `DecompressionStream` while it downloads. +- The preview waits at a pre-boot source chooser. Its built-in example opens a reproducible Workspace and cold Session corpus suitable for inspecting tool cards, subagent navigation, and backward pagination without credentials or model calls; the empty selection preserves first-run coverage. Fixture tests validate the physical logs through production readers, and browser acceptance verifies both selections. diff --git a/.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.zh.md b/.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.zh.md index 09d9356ee8..074d44833c 100644 --- a/.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.zh.md +++ b/.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.zh.md @@ -14,6 +14,8 @@ **preview 就是服务页面加一个标签。** 一次 Vite 构建产出共享全部 chunk 的 `dist/index.html` 与 `dist/preview.html`；唯一差异是前插的一个引导入口，其模块负责连接 worker host。启动随之汇于一个协议：应用注入表的一方 settle `__DSH_BOOT_READY__` deferred——served 渲染器在渲染完的行之后用尾部脚本 resolve，worker 引导段在首个 await 之前安装、末行生效后 settle——client 入口在读取任何注入状态前 await 它，因此从标准入口起的链路逐字就是 served 链路。构建使用相对 base，产物可挂载于任意静态目录；served 形态在 serve 期渲染 `` 锚定深层 SPA fallback 路径，磁盘上的两个页面保持字节共享。 +**仓库 preview 携带可选择的文件系统来源。** Packer 产出一份基础镜像，并为每套具名内置 fixture 产出一份小型 overlay 归档。没有来源 query 时，`preview.html` 会停在选择面板，可选择空文件系统、内置 fixtures，或归另一实现所有的 WebFS provider。合法的 `preview-fixture=none|` query 会直接选择并跳过面板，供确定性的浏览器流程使用；该独立名称避开 Client 既有的 `fixture` transport 开关。Worker 先挂载基础镜像，再按顺序把所选 overlays 应用到仅限 `home/` 和 `workspace/` 的路径，随后才校验基础 manifest 并启动 Cordis。`packages/experimental/webworker-runtime/tests/fixtures/vfs-example/` 提供其中一套内置 overlay，Packer 无需理解 Session 或 Workspace。明文 JSONL 日志使用 persistence backend 的真实 project/session 目录布局，因此 Session Persistence 会冷读取它们，Workspace Registry 则根据其 `/dsh/workspace` header 派生 Workspace。主 Session 超过 Client 的 50-message page，并把代表性工具结果留在尾页；持久化的 one-shot 与 continuable child 用于验证 subagent catalog。WebFS 授权与用户数据仍属于独立 provider，绝不与该 fixture 共用目录。 + 两个包以 `@deepseek-ai/dsh-experimental-*` 名义放在 `packages/experimental/`，私有且在官方发布之外。承载产品承诺的边界仍在产品包里：注入表、`__DSH_TRANSPORT__` 与 `/plugins` bundle 字节由 `dsh-host-webserver`、`dsh-client-modules`、`dsh-client-connection` 拥有。 ## 曾考虑的替代方案 @@ -26,6 +28,12 @@ **用顶层 await 顺序而非 deferred 去闸标准入口。** 兄弟 module script 互不等待对方的顶层 await；`??=` 安装的 deferred 使握手与求值顺序无关，且失败的握手能 reject 进 boot 页的失败呈现。 +**在 Worker 启动时生成示例状态。** Preview 专用的 Session 或 Workspace 创建分支会绕过冷 persistence 读取，还会让 runtime 拥有测试数据。静态镜像文件与既有用户数据经过相同的发现和分页路径。 + +**通过 WebFS 注入示例。** WebFS 拥有用户选定的 durable storage 及其生命周期。让内置演示依赖它，会使静态 preview 受浏览器持久化状态影响，并模糊哪些字节来自部署。 + +**每套 fixture 各打一份完整基础镜像。** 完整镜像变体会重复 runtime package closure，并使组合数量平方增长。受限 overlay 只保留一个不可变基础镜像，选择面板可组合零到多个数据源，未来 WebFS 水合也能复用同一个 pre-boot 应用点。 + ## 后果 - `lib/worker.js` 不含解析器（当刀落时为 423.5 kB → 246.3 kB，早于 shell 进程层落地）。 @@ -33,3 +41,4 @@ - 转换 corpus 会先通过 Node 导入每个已构建 bundle，再比较 lowered export。固定豁免会点名真正不可导入的 bundle，并在其恢复可导入时失败：`win32-process` 是 Koffi 类型 owner 并承担重复类型豁免；`sandbox-windows-acl` 可正常导入，不承担该豁免。 - served 的 `` 锚存在的原因是：相对资产 URL 在 SPA fallback 深路径下会解析进请求目录；只有与相对构建 base 一起才可移除它。 - 镜像以确定性 gzip 压缩的 tar 交付（`vfs-image.tar.gz`；MTIME 0、OS 字节 0xff）：静态托管不压缩二进制 content-type（类型白名单、CDN 尺寸帽），压缩必须随制品走；worker 用浏览器原生 `DecompressionStream` 在下载的同时解压 fetch body。 +- Preview 会停在 pre-boot 来源选择面板。内置示例提供可复现的 Workspace 与冷 Session 语料，无凭据、零模型调用即可检查工具卡、subagent 导航和向前分页；空白选项保留首次启动覆盖。Fixture 测试通过生产 reader 校验物理日志，浏览器验收同时验证两种选择。 diff --git a/.agents/notes/implemented/architecture/2026-08-23-webworker-vfs-watch-and-landlock.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-23-webworker-vfs-watch-and-landlock.i18n.yaml new file mode 100644 index 0000000000..4260ae5238 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-23-webworker-vfs-watch-and-landlock.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-23-webworker-vfs-watch-and-landlock.md +2026-08-23-webworker-vfs-watch-and-landlock.md: 2705c63aa6bf0f2e2de33d00029a4f41e1b1d4af +2026-08-23-webworker-vfs-watch-and-landlock.zh.md: 4470720e1ff968aa06578b231b054c9053f27283 diff --git a/.agents/notes/implemented/architecture/2026-08-23-webworker-vfs-watch-and-landlock.md b/.agents/notes/implemented/architecture/2026-08-23-webworker-vfs-watch-and-landlock.md new file mode 100644 index 0000000000..2705c63aa6 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-23-webworker-vfs-watch-and-landlock.md @@ -0,0 +1,80 @@ +# Agent Note: Web Worker VFS watching and CLI-compatible confinement + +Status: implemented + +English | [中文](2026-08-23-webworker-vfs-watch-and-landlock.zh.md) + +## Problem + +The Web Worker preview boots the same Web profile and Agent presets as the Node host. Without a VFS change source, refusing `node:fs.watchFile` makes `skill-filesystem` return an incomplete observation and re-scan on every lookup, while an inert success leaves an existing root waiting forever for Chokidar's `ready`. Settings and credentials likewise need real external-edit events rather than a package-specific fake. + +The same composition mounts `sandbox-local`, whose Linux chain probes bwrap and then `@deepseek-ai/node-addon-landlock-run`. A Worker cannot execute either binary. Ending the selection there makes `workspace-write` and `read-only` unusable even though every shell filesystem operation already crosses a Host-side VFS call point. + +The filesystem compatibility boundary follows the [Worker Node face decision](2026-08-20-webworker-node-face.md): pure JavaScript watcher packages run unchanged over Node-compatible modules. Native or binary packages may keep their public JavaScript API and executable protocol while replacing the backend. An API that cannot preserve its caller-visible Node behavior remains explicitly unavailable; `node:vm` is outside this decision. + +## Decision + +### VFS mutation source and filesystem watchers + +`MemoryVfs` publishes committed `write`, `mkdir`, `remove`, and `chmod` mutations to any number of subscribers. Publication happens after state changes, failed operations publish nothing, image seeding stays silent, and one throwing subscriber cannot fail the filesystem operation or starve another subscriber. Rename is a source removal plus complete destination mkdir/write records; destination writes mark the directory entry as changed, so watchers report `rename` while a future durable sink receives the bytes needed to materialize the destination. Directory mtimes advance when their immediate entry set changes, so polling detects child creation and removal as Node does. + +The mutation record is shared with WebFS persistence rather than defining a second notification path. Writes carry their complete post-commit bytes and virtual permission bits, plus an append offset when only a tail changed. `MemoryVfs` accepts an optional asynchronous `VfsMutationSink`, sends the same records to that sink and live watcher subscribers, and exposes `flush()` through file-handle `sync()` and `datasync()`. Hydration supplies `{ mode, mtimeMs }` explicitly, so image permissions and durable timestamps cannot occupy the same positional argument. This change mounts no durable sink; it keeps the synchronous in-memory tree authoritative so an OPFS or user-directory mirror can hydrate before publication and write behind without changing `node:fs`. + +The `node:fs` implementation provides callback `stat` and `lstat`, `watch`, `watchFile`, `unwatchFile`, `FSWatcher`, and `StatWatcher`; `node:fs/promises.watch` provides the abortable async iterator. One path shares one `StatWatcher` across listeners, listener-specific unwatching leaves peers active, and missing paths report zero-valued Stats before later creation, deletion, and recreation transitions. Callback dispatch captures the registration-time async context and checks closure before every queued delivery. A pre-aborted callback watch returns its watcher before asynchronously closing it, while a pre-aborted promise watch rejects its first iterator read with `AbortError`. + +`fs.watch` maps entry creation, removal, and rename destinations to `rename`, and maps content or mode changes to `change`. Non-recursive directory watches report immediate child names; recursive watches report paths relative to the watched directory. The VFS has no symlinks, so this implementation does not invent symlink events. + +### Streams and unchanged npm packages + +`node:stream` uses the maintained `readable-stream` browser implementation for `Readable`, `Writable`, `Duplex`, `Transform`, `PassThrough`, pipeline helpers, async iteration, backpressure, aborts, and teardown ordering. The compatibility module sets the byte high-water default to the 64 KiB value used by the repository's Node 22+ engines. VFS-backed `ReadStream` and `WriteStream` supply file descriptors, inclusive ranges, encoding, append or replace behavior, byte accounting, AbortSignal handling, and `open`/`ready`/`finish`/`end`/`close` ordering. Descriptors retain their opened file identity and access mode across rename, replacement, and unlink; hard links share that identity and subsequent content or mode changes, while truncation zero-fills growth. + +Chokidar and readdirp are ordinary image dependencies, not module replacements. Their package code runs unchanged and imports the Worker implementations of `node:fs`, `node:fs/promises`, `node:stream`, `node:events`, `node:path`, and `node:os`. Chokidar therefore retains its own initial scan, `ready`, polling, atomic-write normalization, write-settle delay, shared watcher, and close behavior. + +### Landlock CLI over per-process VFS grants + +`@deepseek-ai/node-addon-landlock-run` is an ordinary image dependency, not a module replacement. Its unchanged JavaScript entry runs through the Worker implementations of `node:child_process`, `node:module`, `node:path`, and `node:url`, so the package remains the sole owner of `LAUNCHER_BIN`, `LAUNCHER_FAILURE_EXIT`, `launcherPath()`, `grantArgs()`, and `probe()`. The image may include the matching Linux optional package, but package resolution does not decide whether the Worker platform supplies Landlock: the entry package's deterministic fallback path reaches the same platform executable implementation when that optional package is absent. + +The process layer has a table of Worker platform executables identified by logical executable name rather than one package-manager path. Its `landlock-run` provider accepts a bare command or an absolute launcher path, parses the native package's unchanged CLI, validates every grant root, and delegates the inner argv to the existing shell process runner. `node:child_process` performs only generic executable lookup, output delivery, and settlement. The unchanged package's synchronous `probe()` therefore observes the provider through `spawnSync` and reports `full`. A usage error, missing grant root, or unknown inner executable prints one `landlock-run: ...` line, exits `125`, and never runs the inner command. The bwrap probe remains unavailable, so the unmodified `sandbox-local` Linux chain selects this Landlock backend. + +Each launched process receives its own `ShellFileSystem` guard. `stat`, `list`, and `readText` require a read-only or read-write grant; `writeText`, `mkdir`, and `remove` require a read-write grant; `rename` requires both source and destination to be writable. Grant roots normalize trailing separators before containment checks. Denials carry `EACCES` and `permission denied`, preserving `bash-sandbox` denial classification. `/tmp` maps to the VFS `/dsh/tmp`, while `/dev/null` is a virtual empty-read and discarded-write file that stores no bytes. + +The Worker's `full` verdict covers every file operation expressible through its shell command table and Host-served VFS protocol. It does not claim Linux kernel Landlock, arbitrary native executable support, or protection against a future shell program that bypasses `ShellFileSystem`. + +### Explicitly deferred behavior + +`node:vm`, `node:worker_threads`, `node:net`, `node:sqlite`, native PTY, Sharp, and ripgrep remain outside this change. The VFS remains POSIX-only, in-memory, and symlink-free. Browser Workers have no libuv-style ref-counted event loop, so watcher `persistent`, `ref()`, and `unref()` preserve the API and observable state but cannot decide Worker lifetime. + +## Alternatives considered + +**Disable watcher and sandbox rows in the Worker profile.** A smaller composition would stop testing the same Host tree and would hide package integration failures specific to preview deployment. + +**Make `watchFile` an inert success.** Missing roots would never advance, and an existing root would wait forever for Chokidar `ready`. + +**Notify watchers only from `node:fs`.** Shell process requests and any direct VFS writer would bypass the notification point. The commit owner, `MemoryVfs`, is the only complete source. + +**Keep a VFS-specific Chokidar replacement.** This duplicates directory scans, ready accounting, write settling, atomic replacement, shared watcher ownership, and teardown already maintained upstream. + +**Replace the Landlock entry package with a Worker module.** Reimplementing its exported constants, grant builder, launcher resolution, and probe would create a second copy of a package contract that already runs over the Worker Node compatibility layer. Only the platform executable implementation differs. + +**Recognize one exact launcher path.** Optional-dependency installation and the entry package's documented fallback produce different absolute paths for the same executable. Package-manager layout is not the identity of a platform capability, so executable dispatch uses the logical `landlock-run` name. + +**Add a Worker branch to `sandbox-local`.** This would copy policy-to-grant mapping into a business package. Interpreting the existing launcher protocol preserves the provider, consumer, configuration, diagnostics, and native package API. + +**Store one active policy on the global VFS.** Concurrent foreground, background, and escalated commands would overwrite one another's authority. Grants belong to one process handle and its filesystem adapter. + +## Verification + +- `fs-watch-stream.spec.ts` compares missing/create/change/remove `watchFile` transitions and file-stream lifecycle, chunking, range, backpressure, byte count, defaults, and abort identity with the running Node version. +- `chokidar.spec.ts` loads both lockfile-selected Chokidar and readdirp dependency pairs through the Worker transformer and module loader, then proves `ready`, callback watching, polling, missing-file creation, removal, and quiescent close over `MemoryVfs`. +- `image-loadable.spec.ts` packs and loads the real `@deepseek-ai/node-addon-landlock-run` JavaScript, proves it is absent from the replacement table, and runs its fallback `launcherPath()` and `probe()` through the Worker platform executable. `child-process.spec.ts` and `sandbox-stack.spec.ts` then prove the launcher failure code, malformed argv and grant failures, `/tmp` and `/dev/null`, rename denial, all three permission modes, and concurrent process-local grants through the production sandbox and subprocess packages. +- `preview-boot.e2e.ts` builds and boots the packed browser deployment, creates a Workspace and Session, advances missing skill roots into a live Chokidar watch, lists the catalog, and completes settings and credential writes without watcher warnings. + +## Consequences + +The preview now runs npm watcher consumers without source forks, and filesystem mutations observed from Host code or shell process Workers share one ordered commit source. A WebFS/OPFS integration remains an asynchronous mirror around this synchronous authority and consumes that same source; it does not add another Chokidar implementation or a competing mutation protocol. + +Worker `read-only` and `workspace-write` preserve the product's permission vocabulary and denial reporting without forking the Landlock npm package. Their security claim is narrower than native Landlock but complete inside the Worker execution world; any new filesystem message or shell program must continue through the guarded `ShellFileSystem`. Native-backed packages follow the same ownership rule: their JavaScript remains upstream, while the Worker platform replaces only the native artifact behind it. + +The worker bundle gains `readable-stream` and its small browser dependency closure. In return, stream state and backpressure remain maintained upstream instead of becoming local compatibility code. + +Watcher event timing is deterministic from VFS commits rather than inherited from an operating-system backend. This stays within Node's watcher contract, which does not guarantee native event coalescing, while tests pin every event distinction the current consumers require. diff --git a/.agents/notes/implemented/architecture/2026-08-23-webworker-vfs-watch-and-landlock.zh.md b/.agents/notes/implemented/architecture/2026-08-23-webworker-vfs-watch-and-landlock.zh.md new file mode 100644 index 0000000000..4470720e1f --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-23-webworker-vfs-watch-and-landlock.zh.md @@ -0,0 +1,80 @@ +# Agent Note: Web Worker VFS 监听与 CLI 兼容 confinement + +Status: implemented + +[English](2026-08-23-webworker-vfs-watch-and-landlock.md) | 中文 + +## Problem + +Web Worker preview 启动与 Node host 相同的 Web profile 和 Agent preset。缺少 VFS 变更源时，拒绝 `node:fs.watchFile` 会让 `skill-filesystem` 返回不完整观测并在每次查询时重新扫描，而无事件的成功调用会让已有根永远等待 Chokidar 的 `ready`。Settings 和 credentials 同样需要真实的外部编辑事件，而不是包专用 fake。 + +同一组合挂载 `sandbox-local`，其 Linux 选择链依次探测 bwrap 和 `@deepseek-ai/node-addon-landlock-run`。Worker 无法执行这两个二进制文件。如果选择链到此结束，`workspace-write` 与 `read-only` 将不可用，尽管 shell 的每项文件系统操作已经经过 Host 侧 VFS 调用点。 + +文件系统兼容边界遵循 [Worker Node face 决策](2026-08-20-webworker-node-face.zh.md)：纯 JavaScript watcher 包在 Node 兼容模块之上保持原样运行。Native 或 binary 包可以保持公开 JavaScript API 与可执行文件协议，同时替换执行后端。无法维持调用方可见 Node 行为的 API 继续明确标记为不可用；`node:vm` 不属于本决策范围。 + +## Decision + +### VFS mutation source 与文件 watcher + +`MemoryVfs` 向任意数量的订阅方发布已提交的 `write`、`mkdir`、`remove` 和 `chmod` mutation。状态改变后才发布，失败操作不发布，镜像 seed 保持无事件，一个抛错的订阅方也不能让文件系统操作失败或阻止其他订阅方。Rename 被表达为源路径删除与包含完整状态的目标 mkdir/write 记录；目标 write 会标记目录项已改变，因此 watcher 报告 `rename`，未来的 durable sink 同时拿到物化目标所需的字节。目录的直接条目集合改变时，其 mtime 会推进，因此 polling 能像 Node 一样发现子项创建和删除。 + +Mutation record 与 WebFS 持久化共用，而不建立第二条通知路径。Write 记录携带提交后的完整字节与虚拟权限位，并在只有尾部变化时携带 append offset。`MemoryVfs` 接受可选的异步 `VfsMutationSink`，把同一批记录交给 sink 与实时 watcher 订阅方，并通过文件句柄的 `sync()` 和 `datasync()` 暴露 `flush()`。水合通过显式的 `{ mode, mtimeMs }` 传入元数据，因此镜像权限与持久化时间戳不会占用同一个位置参数。本次变更不挂载 durable sink；同步内存树继续作为权威，因此 OPFS 或用户目录 mirror 可以先水合、再异步写回，而无需改变 `node:fs`。 + +`node:fs` 实现 callback `stat` 和 `lstat`、`watch`、`watchFile`、`unwatchFile`、`FSWatcher` 与 `StatWatcher`；`node:fs/promises.watch` 提供可由 abort 取消的异步迭代器。同一路径的 listener 共享一个 `StatWatcher`，按 listener 取消监听不会影响其他 listener；缺失路径先报告零值 Stats，随后再报告创建、删除和重建状态。Callback 分发捕获注册时的异步上下文，并在每次排队交付前检查 watcher 是否已经关闭。预先 abort 的 callback watcher 先返回对象、再异步关闭；预先 abort 的 promise watcher 在第一次读取 iterator 时以 `AbortError` 拒绝。 + +`fs.watch` 把条目创建、删除和 rename 目标映射为 `rename`，把内容或 mode 变化映射为 `change`。非递归目录 watcher 报告直接子项名，递归 watcher 报告相对被监听目录的路径。VFS 没有符号链接，因此该实现不会制造符号链接事件。 + +### Stream 与未修改的 NPM 包 + +`node:stream` 使用维护中的 `readable-stream` 浏览器实现来提供 `Readable`、`Writable`、`Duplex`、`Transform`、`PassThrough`、pipeline helper、异步迭代、backpressure、abort 和 teardown 顺序。兼容模块把字节流 high-water mark 默认值设为仓库 Node 22+ 引擎使用的 64 KiB。VFS 支持的 `ReadStream` 与 `WriteStream` 提供文件描述符、闭区间范围、encoding、追加或替换行为、字节计数、AbortSignal 处理，以及 `open`、`ready`、`finish`、`end`、`close` 顺序。Descriptor 在 rename、replacement 和 unlink 后仍保留打开时的文件身份与访问模式；hard link 共享该身份及后续内容和 mode 变化，truncate 增长则用零字节填充。 + +Chokidar 和 readdirp 作为普通镜像依赖运行，不属于模块 replacement。它们的包代码保持原样，并导入 Worker 实现的 `node:fs`、`node:fs/promises`、`node:stream`、`node:events`、`node:path` 与 `node:os`。因此，初次扫描、`ready`、polling、原子写归一化、写入稳定等待、共享 watcher 与关闭行为仍由 Chokidar 自己负责。 + +### 基于逐进程 VFS 授权的 Landlock CLI + +`@deepseek-ai/node-addon-landlock-run` 是普通镜像依赖，不是模块 replacement。其未经修改的 JavaScript 入口通过 Worker 实现的 `node:child_process`、`node:module`、`node:path` 与 `node:url` 运行，因此该包仍是 `LAUNCHER_BIN`、`LAUNCHER_FAILURE_EXIT`、`launcherPath()`、`grantArgs()` 和 `probe()` 的唯一所有者。镜像可以包含匹配的 Linux optional package，但包解析不决定 Worker 平台是否提供 Landlock；缺少该 optional package 时，入口包产生的确定性 fallback 路径仍到达同一个平台可执行文件实现。 + +进程层持有按逻辑可执行文件名识别的 Worker 平台可执行文件表，而不依赖某一个包管理器路径。其 `landlock-run` provider 接受裸命令或绝对 launcher 路径，解析 native 包未经修改的 CLI、校验每个授权根，并把内部 argv 交给既有 shell 进程 runner。`node:child_process` 只负责通用的可执行文件查找、输出投递与结束处理。因此，原包的同步 `probe()` 会通过 `spawnSync` 观察到该 provider 并报告 `full`。用法错误、缺失的授权根或未知内部可执行文件只输出一行 `landlock-run: ...`，以 `125` 退出，并且绝不运行内部命令。bwrap 仍探测为不可用，因此未修改的 `sandbox-local` Linux 选择链会选中该 Landlock 后端。 + +每个已启动进程分别获得一个 `ShellFileSystem` guard。`stat`、`list` 和 `readText` 需要只读或读写授权；`writeText`、`mkdir` 和 `remove` 需要读写授权；`rename` 要求源和目标都可写。Grant root 在 containment 检查前去除尾部分隔符。拒绝错误包含 `EACCES` 与 `permission denied`，从而保持 `bash-sandbox` 的拒绝分类。`/tmp` 映射到 VFS 的 `/dsh/tmp`，`/dev/null` 则是空读、丢弃写入且不保存任何字节的虚拟文件。 + +Worker 的 `full` 结论覆盖 shell 命令表和 Host 服务 VFS 协议能够表达的全部文件操作。它不表示 Linux 内核 Landlock、不支持任意 native 可执行文件，也无法约束未来绕过 `ShellFileSystem` 的 shell 程序。 + +### 明确延后的行为 + +`node:vm`、`node:worker_threads`、`node:net`、`node:sqlite`、native PTY、Sharp 和 ripgrep 不属于本次变更。VFS 仍然只支持 POSIX、内存存储且没有符号链接。Browser Worker 没有 libuv 风格的引用计数事件循环，因此 watcher 的 `persistent`、`ref()` 和 `unref()` 保留 API 与可观察状态，但不能决定 Worker 生存期。 + +## Alternatives considered + +**在 Worker profile 中禁用 watcher 与 sandbox 配置项。** 缩减组合后将不再测试相同的 Host tree，还会隐藏 preview 部署特有的包集成故障。 + +**让 `watchFile` 成为无事件的成功调用。** 缺失根永远无法推进，已有根则会永久等待 Chokidar `ready`。 + +**只从 `node:fs` 通知 watcher。** Shell 进程请求以及直接写 VFS 的实现可以绕过通知点。只有提交状态的 `MemoryVfs` 才是完整真源。 + +**保留 VFS 专用的 Chokidar replacement。** 这会重复实现上游已经维护的目录扫描、ready 计数、写入稳定等待、原子替换、共享 watcher 所有权和 teardown。 + +**用 Worker 模块替换 Landlock 入口包。** 重新实现其导出常量、授权参数构造、launcher 解析和 probe，会为一个已经能在 Worker Node 兼容层上运行的包约定建立第二份副本。只有平台可执行文件实现需要不同。 + +**只识别一个精确 launcher 路径。** Optional dependency 的安装状态与入口包已有的 fallback 会为同一个可执行文件产生不同的绝对路径。包管理器布局不是平台能力的身份，因此可执行文件分发使用逻辑名称 `landlock-run`。 + +**在 `sandbox-local` 中增加 Worker 分支。** 这会把策略到授权的映射复制到业务包中。解释现有 launcher 协议可以保持 provider、consumer、配置、诊断和 native 包 API 不变。 + +**在全局 VFS 上保存一个当前策略。** 并发前台、后台和升权命令会覆盖彼此的权限。授权必须归属于单个进程句柄及其文件系统适配器。 + +## Verification + +- `fs-watch-stream.spec.ts` 对照当前 Node 版本验证缺失、创建、修改、删除的 `watchFile` 状态转换，以及文件流生命周期、分片、范围、backpressure、字节计数、默认值和 abort 身份。 +- `chokidar.spec.ts` 通过 Worker transformer 与模块 loader 加载 lockfile 选定的两组 Chokidar 和 readdirp 依赖，并在 `MemoryVfs` 上验证 `ready`、callback watcher、polling、缺失文件创建、删除和完全停稳的关闭。 +- `image-loadable.spec.ts` 打包并加载真实的 `@deepseek-ai/node-addon-landlock-run` JavaScript，验证它不在 replacement 表中，并让其 fallback `launcherPath()` 与 `probe()` 经过 Worker 平台可执行文件。`child-process.spec.ts` 与 `sandbox-stack.spec.ts` 随后通过生产 sandbox 和 subprocess 包验证 launcher 失败码、错误 argv 与授权失败、`/tmp` 与 `/dev/null`、rename 拒绝、三种权限模式和逐进程并发授权。 +- `preview-boot.e2e.ts` 构建并启动打包后的浏览器部署，创建 Workspace 与 Session，把缺失的 skill 根逐级推进到可用的 Chokidar watch，读取 catalog，并在没有 watcher 警告的情况下完成 settings 与 credential 写入。 + +## Consequences + +Preview 现在可以在不 fork 源码的情况下运行 NPM watcher 消费方；Host 代码与 shell 进程 Worker 产生的文件系统 mutation 共享同一个有序提交源。WebFS/OPFS 集成仍是围绕该同步权威的异步 mirror，并消费同一个变更源；它不会增加另一份 Chokidar 实现或互相竞争的 mutation 协议。 + +Worker `read-only` 与 `workspace-write` 在不 fork Landlock NPM 包的情况下保留产品权限词汇和拒绝报告。其安全结论比 native Landlock 更窄，但完整覆盖 Worker 执行世界；任何新的文件系统消息或 shell 程序都必须继续经过受 guard 保护的 `ShellFileSystem`。Native-backed 包遵循同一所有权规则：其 JavaScript 保持上游实现，Worker 平台只替换背后的 native artifact。 + +Worker bundle 增加 `readable-stream` 及其少量浏览器依赖。相应地，stream 状态和 backpressure 继续由上游维护，不成为本地兼容代码。 + +Watcher 事件时序由 VFS 提交确定，而不是继承操作系统后端。Node watcher 约定本身不保证 native 事件合并方式，因此该实现仍符合约定；测试固定当前消费方依赖的每一种事件区别。 diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index b82665e1c9..f2637fe699 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -87,6 +87,7 @@ External packages that a workspace package resolves at runtime. The tier covers | [`picomatch`](https://github.com/micromatch/picomatch) | MIT | | [`react`](https://github.com/facebook/react) | MIT | | [`react-dom`](https://github.com/facebook/react) | MIT | +| [`readable-stream`](https://github.com/nodejs/readable-stream) | MIT | | [`sharp`](https://github.com/lovell/sharp) | Apache-2.0 | | [`shiki`](https://github.com/shikijs/shiki) | MIT | | [`supports-color`](https://github.com/chalk/supports-color) | MIT | @@ -140,6 +141,7 @@ External packages **directly declared** only by repository tooling, test infrast | [`@types/picomatch`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT | | [`@types/react`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT | | [`@types/react-dom`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT | +| [`@types/readable-stream`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT | | [`@types/spdx-expression-parse`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT | | [`@types/turndown`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT | | [`@types/use-sync-external-store`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT | diff --git a/apps/web/src/preview.ts b/apps/web/src/preview.ts index 586cbcab5d..f3d51f8bbc 100644 --- a/apps/web/src/preview.ts +++ b/apps/web/src/preview.ts @@ -1,12 +1,14 @@ /** * Worker-preview bootstrap: the one module preview.html adds ahead of the - * stock entry tag. Connecting the worker host installs the boot globals and - * settles `__DSH_BOOT_READY__`, where the stock entry's pre-boot await holds, - * so everything after this module is the served startup chain verbatim. A - * failed handshake rejects the deferred into the boot page's failure - * rendering; this module owns no page painting. + * stock entry tag. The runtime's optional source stage owns the pre-Cordis + * chooser; the unchanged Host connector then owns the Worker handshake. + * Everything after those calls is the served startup chain verbatim. */ import DshWorker from '@deepseek-ai/dsh-experimental-webworker-runtime/worker?worker' -import { connectWorkerHost, IMAGE_FILE_NAME } from '@deepseek-ai/dsh-experimental-webworker-runtime/client' +import { + chooseWorkerHostSource, connectWorkerHost, IMAGE_FILE_NAME, +} from '@deepseek-ai/dsh-experimental-webworker-runtime/client' -await connectWorkerHost(new DshWorker({ name: 'dsh-host' }), { image: `preview/${IMAGE_FILE_NAME}` }) +const image = `preview/${IMAGE_FILE_NAME}` +const source = await chooseWorkerHostSource({ image }) +await connectWorkerHost(new DshWorker({ name: 'dsh-host' }), { image, overlays: source.overlays }) diff --git a/apps/web/tests/preview-boot.e2e.ts b/apps/web/tests/preview-boot.e2e.ts index b2d6add83a..8ef63e1067 100644 --- a/apps/web/tests/preview-boot.e2e.ts +++ b/apps/web/tests/preview-boot.e2e.ts @@ -8,25 +8,33 @@ * Two milestones prove that happened — the host's `tree active` boot line, * whose lowering contract must be the one this checkout's packer emits, and the * workspace hero, which paints only after the client tree comes up over the - * tunnel. + * tunnel. The same page opens the seeded Workspace and showcase Session, + * verifies its tool/subagent/history examples, then writes through the + * settings and credentials providers. That keeps the upstream Chokidar + * instances exercised over the Worker filesystem implementation. * * The site is served the way a static host serves it: bytes from `dist/` with * no rewrite rules, so a missing file is a 404 rather than the index page. */ -import { existsSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' import { readFile } from 'node:fs/promises' import { createServer } from 'node:http' import type { IncomingMessage, ServerResponse } from 'node:http' import { tmpdir } from 'node:os' -import { extname, join, normalize } from 'node:path' +import { dirname, extname, join, normalize } from 'node:path' import { fileURLToPath } from 'node:url' import { chromium } from 'playwright' import type { Browser } from 'playwright' import { expect, it } from 'vitest' import { - composeProfile, configTrees, indexWorkspacePackages, packVfsImage, WRAPPER_CONTRACT, + composeProfile, configTrees, indexWorkspacePackages, packVfsImage, packVfsOverlay, + previewFixtures, WRAPPER_CONTRACT, } from '@deepseek-ai/dsh-experimental-webworker-packer' -import { IMAGE_FILE_NAME } from '@deepseek-ai/dsh-experimental-webworker-runtime' +import { + IMAGE_FILE_NAME, PREVIEW_FIXTURE_MANIFEST_FILE, PREVIEW_FIXTURE_MANIFEST_VERSION, + type PreviewFixtureManifest, +} from '@deepseek-ai/dsh-experimental-webworker-runtime' +import { captureStableAria, compareOrRefreshGolden, webSnapshotMode } from './scaffold.ts' import { newEnglishPage, REPO_ROOT, saveFailureShot } from './support.ts' const DIST_ROOT = fileURLToPath(new URL('../dist', import.meta.url)) @@ -34,9 +42,22 @@ const DIST_ROOT = fileURLToPath(new URL('../dist', import.meta.url)) /** Where the client looks for the image: the runtime's own name, beside the page. */ const IMAGE_FILE = join(DIST_ROOT, 'preview', IMAGE_FILE_NAME) +/** Built-in source catalog read by the pre-boot chooser. */ +const FIXTURE_MANIFEST_FILE = join(DIST_ROOT, 'preview', PREVIEW_FIXTURE_MANIFEST_FILE) + +/** Keyless browser golden for the pre-Worker source chooser. */ +const SOURCE_CHOOSER_EXPECTED = fileURLToPath(new URL('./snapshots/preview-boot/source-chooser.expected.md', import.meta.url)) + +const SNAPSHOT_MODE = webSnapshotMode() + /** Profile the preview deployment composes; `build:preview` packs the same one. */ const PROFILE = 'web' +/** Stable labels authored by the deterministic VFS example fixture. */ +const SHOWCASE_TITLE = 'WebWorker Preview Showcase' +const SHOWCASE_TAIL = 'Preview tour complete' +const SHOWCASE_OLDEST = 'History checkpoint 01: verify deterministic preview state.' + /** Pages the preview needs; the Vite build emits both. */ const PAGES = ['index.html', 'preview.html'] @@ -75,6 +96,12 @@ interface Site { close(): Promise } +interface PreviewAssets { + /** Static-host-relative path to a generated file outside `dist/`. */ + readonly overrides: ReadonlyMap + cleanup(): void +} + /** * Fail before the browser opens a page the build never produced. * @throws When either preview page is missing from `dist/`. @@ -87,20 +114,25 @@ function requirePreviewPages(): void { } /** - * The image file to serve, packed here when `dist/` carries none: `pnpm run - * build` emits the pages but only `build:preview` packs, so this lane packs - * for itself rather than skipping the deployment it is here to accept. An - * image already in place is used as it stands — the worker refuses one lowered - * against another wrapper contract, and that refusal names the rebuild. A - * self-packed image lands in a temp directory, never in `dist/`: the + * The base image, fixture manifest, and overlays to serve, packed here when + * `dist/` does not carry the complete set: `pnpm run build` emits the pages but + * only `build:preview` packs these files, so this lane packs for itself rather + * than skipping the deployment it accepts. A complete built set is used as it + * stands — the worker refuses a base lowered against another wrapper contract. + * Self-packed files land in a temp directory, never in `dist/`: the * client-artifact digest record treats `dist/` as build-owned, so a test write * there fails the record check for every later consumer. - * @returns The file to answer `preview/` with, and its teardown. + * @returns Static-path overrides and their teardown. * @throws When the closure leaves dependencies unresolved, which would pack an * incomplete image the tree fails on later and further from the cause. */ -function requireVfsImage(): { path: string; cleanup(): void } { - if (existsSync(IMAGE_FILE)) return { path: IMAGE_FILE, cleanup: () => {} } +function requireVfsAssets(): PreviewAssets { + const fixtureDefinitions = previewFixtures(REPO_ROOT) + const fixtureFiles = fixtureDefinitions.map(fixture => + join(DIST_ROOT, 'preview', 'fixtures', `${fixture.id}.tar.gz`)) + if ([IMAGE_FILE, FIXTURE_MANIFEST_FILE, ...fixtureFiles].every(existsSync)) { + return { overrides: new Map(), cleanup: () => {} } + } const packed = packVfsImage({ config: composeProfile(REPO_ROOT, PROFILE), profile: PROFILE, @@ -112,23 +144,48 @@ function requireVfsImage(): { path: string; cleanup(): void } { throw new Error(`preview boot: ${String(packed.missing.length)} dependencies did not resolve: ${packed.missing.join(', ')}`) } const directory = mkdtempSync(join(tmpdir(), 'dsh-preview-boot-')) - const path = join(directory, IMAGE_FILE_NAME) - writeFileSync(path, packed.image) - return { path, cleanup: () => { rmSync(directory, { recursive: true, force: true }) } } + const overrides = new Map() + const writeAsset = (relativePath: string, bytes: Uint8Array | string): void => { + const path = join(directory, relativePath) + mkdirSync(dirname(path), { recursive: true }) + writeFileSync(path, bytes) + overrides.set(relativePath, path) + } + writeAsset(`preview/${IMAGE_FILE_NAME}`, packed.image) + const fixtures = fixtureDefinitions.map((fixture) => { + const relativePath = `preview/fixtures/${fixture.id}.tar.gz` + writeAsset(relativePath, packVfsOverlay(fixture.trees).image) + return { + id: fixture.id, + label: fixture.label, + description: fixture.description, + overlays: [`fixtures/${fixture.id}.tar.gz`], + } + }) + const manifest: PreviewFixtureManifest = { + version: PREVIEW_FIXTURE_MANIFEST_VERSION, + defaultFixture: fixtures[0]?.id ?? null, + fixtures, + } + writeAsset(`preview/${PREVIEW_FIXTURE_MANIFEST_FILE}`, `${JSON.stringify(manifest, null, 2)}\n`) + return { overrides, cleanup: () => { rmSync(directory, { recursive: true, force: true }) } } } /** - * Answer one request with the file it names under `dist/`; the image path - * answers from wherever {@link requireVfsImage} put the file. + * Answer one request with its generated override or the file under `dist/`. * @param request - Incoming request; only its path is read. * @param response - Response to write the bytes or the 404 to. - * @param imagePath - File behind `preview/`. + * @param overrides - Generated deployment files used when `dist/` has none. */ -async function respond(request: IncomingMessage, response: ServerResponse, imagePath: string): Promise { +async function respond( + request: IncomingMessage, + response: ServerResponse, + overrides: ReadonlyMap, +): Promise { const path = new URL(request.url ?? '/', 'http://127.0.0.1').pathname const relative = normalize(decodeURIComponent(path)).replace(/^\/+/, '') try { - const body = await readFile(relative === `preview/${IMAGE_FILE_NAME}` ? imagePath : join(DIST_ROOT, relative)) + const body = await readFile(overrides.get(relative) ?? join(DIST_ROOT, relative)) response.writeHead(200, { 'content-type': MIME[extname(relative)] ?? 'application/octet-stream' }) response.end(body) } catch { @@ -142,11 +199,11 @@ async function respond(request: IncomingMessage, response: ServerResponse, image /** * Serve `dist/` over loopback with static-host semantics. - * @param imagePath - File behind `preview/`. + * @param overrides - Generated deployment files used when `dist/` has none. * @returns The origin to navigate, and its teardown. */ -async function serveDist(imagePath: string): Promise { - const server = createServer((request, response) => { void respond(request, response, imagePath) }) +async function serveDist(overrides: ReadonlyMap): Promise { + const server = createServer((request, response) => { void respond(request, response, overrides) }) await new Promise((listening) => { server.listen(0, '127.0.0.1', listening) }) const address = server.address() if (address === null || typeof address === 'string') throw new Error('preview boot: the static server bound no port') @@ -186,12 +243,13 @@ async function within(work: Promise, ms: number, stalled: string): Promise it('boots the packed worker deployment to an interactive page', async () => { requirePreviewPages() - const image = requireVfsImage() + const assets = requireVfsAssets() try { - const site = await serveDist(image.path) + const site = await serveDist(assets.overrides) try { const browser = await chromium.launch({ headless: true, args: ['--no-sandbox', '--disable-dev-shm-usage'] }) try { + await bootEmptyPreview(site.origin, browser) await bootPreview(site.origin, browser) } finally { await browser.close() @@ -200,7 +258,7 @@ it('boots the packed worker deployment to an interactive page', async () => { await site.close() } } finally { - image.cleanup() + assets.cleanup() } }, 600_000) @@ -212,6 +270,7 @@ it('boots the packed worker deployment to an interactive page', async () => { async function bootPreview(origin: string, browser: Browser): Promise { const page = await newEnglishPage(browser) const pageErrors: Error[] = [] + const consoleErrors: string[] = [] page.on('pageerror', (error) => { pageErrors.push(error) }) // Registered before navigation: the worker reports its tree long before the // tunnel serves the client, so a listener added later would miss the line. @@ -219,20 +278,136 @@ async function bootPreview(origin: string, browser: Browser): Promise { page.on('console', (message) => { const text = message.text() if (text.includes(TREE_ACTIVE)) reported(text) + if (message.type() === 'error' || message.type() === 'warning') consoleErrors.push(text) }) }) try { await page.goto(`${origin}/preview.html`, { waitUntil: 'domcontentloaded' }) + await page.getByRole('heading', { name: 'Choose Preview data' }).waitFor() + expect(await page.locator('input[name="preview-source"][value="vfs-example"]').isChecked()).toBe(true) + expect(await page.getByText('Empty environment', { exact: true }).count()).toBe(1) + expect(await page.getByText('WebFS directory', { exact: true }).count()).toBe(1) + expect(await page.locator('input[name="preview-source"][value="webfs"]').isDisabled()).toBe(true) + expect(await page.getByRole('textbox', { name: 'Choose workspace' }).count()).toBe(0) + await compareOrRefreshGolden( + SOURCE_CHOOSER_EXPECTED, + await captureStableAria(page, '[data-preview-source-card]', '/__preview_no_workspace__'), + SNAPSHOT_MODE, + ) + await page.getByRole('button', { name: 'Start Preview' }).click() + await page.getByText('Loading plugins…', { exact: true }).waitFor({ timeout: 10_000 }) const bootLine = await within(treeActive, BOOT_TIMEOUT_MS, `preview boot: the worker never reported "${TREE_ACTIVE}"`) // The activated tree ran bodies lowered against the contract this // checkout's packer emits; a dist built before a contract change would // report the older one. expect(bootLine).toContain(`image lowering=${WRAPPER_CONTRACT}`) + expect(bootLine).toContain('data overlays=1') // The hero's workspace picker is the client tree's first interactive // surface, so it appears only once the startup chain completed over the // tunnel. await page.getByRole('textbox', { name: 'Choose workspace' }).waitFor({ timeout: HERO_TIMEOUT_MS }) + const continueButton = page.getByRole('button', { name: 'Continue' }) + await continueButton.waitFor({ timeout: 30_000 }) + await continueButton.click() + const configureLater = page.getByRole('button', { name: 'Configure later' }) + await configureLater.waitFor({ timeout: 30_000 }) + await configureLater.click() + await page.locator('textarea:enabled[placeholder="Describe what you want to build"]') + .waitFor({ timeout: 30_000 }) + + const exercised = await page.evaluate(async () => { + type Result = { result: { ok: true; value: T } | { ok: false; error: { code: string; message: string } } } + interface PreviewApi { + host: { createDirectory(payload: { path: string; name: string }): Promise> } + skills: { list(payload: { sessionId: string }): Promise> } + settings: { + describe(payload: object): Promise }>> + update(payload: { ns: string; patch: object; expectedRevision: number }): Promise> + } + credentials: { + set(payload: { ref: string; value: string }): Promise> + unset(payload: { ref: string }): Promise> + describe(payload: { refs: string[] }): Promise + }>> + } + } + interface PreviewTransport { + fetch(input: string, init: RequestInit): Promise + createApiClient(): PreviewApi + } + const transport = (globalThis as typeof globalThis & { __DSH_TRANSPORT__?: PreviewTransport }).__DSH_TRANSPORT__ + if (transport === undefined) throw new Error('preview transport is absent after boot') + const response = await transport.fetch('/api/session/list', { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ + type: 'client-request', rpcId: 'preview-session-list', method: 'session/list', + payload: { args: { _request: {} } }, + }), + }) + const sessions = await response.json() as Result<{ items: Array<{ sessionId: string }> }> + if (!sessions.result.ok) throw new Error(`session/list failed: ${sessions.result.error.message}`) + const sessionId = sessions.result.value.items[0]?.sessionId + if (sessionId === undefined) throw new Error('workspace adoption created no Session') + + const api = transport.createApiClient() + const skills = await api.skills.list({ sessionId }) + if (!skills.result.ok) throw new Error(`skill.list failed: ${skills.result.error.message}`) + const createDirectory = async (path: string, name: string): Promise => { + const created = await api.host.createDirectory({ path, name }) + if (!created.result.ok) throw new Error(`host.createDirectory failed: ${created.result.error.message}`) + await new Promise((resolve) => { setTimeout(resolve, 250) }) + const refreshed = await api.skills.list({ sessionId }) + if (!refreshed.result.ok) throw new Error(`skill.list refresh failed: ${refreshed.result.error.message}`) + } + await createDirectory('/dsh/workspace/.agents/skills', 'runtime-created') + const settings = await api.settings.describe({}) + if (!settings.result.ok) throw new Error(`settings.describe failed: ${settings.result.error.message}`) + const shell = settings.result.value.namespaces.find(namespace => namespace.ns === 'shell') + if (shell === undefined) throw new Error('settings.describe omitted the shell namespace') + const updated = await api.settings.update({ ns: 'shell', patch: { timeoutMs: 61_000 }, expectedRevision: shell.revision }) + if (!updated.result.ok) throw new Error(`settings.update failed: ${updated.result.error.message}`) + const stored = await api.credentials.set({ ref: 'PREVIEW_TEST_SECRET', value: 'worker-only' }) + if (!stored.result.ok) throw new Error(`credentials.set failed: ${stored.result.error.message}`) + const credentials = await api.credentials.describe({ refs: ['PREVIEW_TEST_SECRET'] }) + if (!credentials.result.ok) throw new Error(`credentials.describe failed: ${credentials.result.error.message}`) + const removed = await api.credentials.unset({ ref: 'PREVIEW_TEST_SECRET' }) + if (!removed.result.ok) throw new Error(`credentials.unset failed: ${removed.result.error.message}`) + await new Promise((resolve) => { setTimeout(resolve, 250) }) + return { + skillCount: skills.result.value.skills.length, + credentialConfigured: credentials.result.value.credentials.PREVIEW_TEST_SECRET?.configured, + } + }) + expect(exercised.skillCount).toBeGreaterThan(0) + expect(exercised.credentialConfigured).toBe(true) + + const sessions = page.getByRole('tree', { name: 'Sessions' }) + const showcase = sessions.getByRole('treeitem').filter({ hasText: SHOWCASE_TITLE }) + await expect.poll(() => showcase.count(), { timeout: 15_000 }).toBe(1) + await showcase.click() + await page.getByText(SHOWCASE_TAIL, { exact: true }).waitFor({ timeout: 30_000 }) + + expect(await page.getByText(SHOWCASE_OLDEST, { exact: true }).count()).toBe(0) + await page.getByText('PREVIEW.md', { exact: true }).waitFor() + await page.getByText('src/preview.ts', { exact: true }).waitFor() + await page.getByText('Update to-do list', { exact: true }).waitFor() + await page.getByText('Error: ENOENT: no such file, open missing.txt', { exact: true }).waitFor() + + const subagents = page.getByRole('button', { name: '2 subagents' }) + await subagents.waitFor({ timeout: 15_000 }) + await subagents.hover() + const catalog = page.getByRole('tree', { name: 'Subagent sessions' }) + await catalog.getByRole('treeitem', { name: /Review preview architecture/ }).waitFor() + await catalog.getByRole('treeitem', { name: /Continue preview verification/ }).waitFor() + await catalog.press('Escape') + + await page.getByRole('button', { name: 'Load earlier', exact: true }).click() + await page.getByText(SHOWCASE_OLDEST, { exact: true }).waitFor({ timeout: 15_000 }) expect(pageErrors.map(error => error.message)).toEqual([]) + expect(consoleErrors.filter(line => + /watchFile|failed to watch|node-addon-landlock-run\.probe|sandbox backend is usable|SANDBOX_UNAVAILABLE/i.test(line))).toEqual([]) } catch (error) { await saveFailureShot(page, 'preview-boot') throw pageErrors.length === 0 @@ -240,3 +415,65 @@ async function bootPreview(origin: string, browser: Browser): Promise { : new AggregateError([error, ...pageErrors], 'preview boot failed, with uncaught page errors') } } + +/** Verify the chooser can boot the untouched base image and reach first-run UI. */ +async function bootEmptyPreview(origin: string, browser: Browser): Promise { + const page = await newEnglishPage(browser) + const pageErrors: Error[] = [] + const consoleErrors: string[] = [] + const failedResponses: string[] = [] + page.on('pageerror', (error) => { pageErrors.push(error) }) + page.on('response', (response) => { + if (response.status() >= 400) failedResponses.push(new URL(response.url()).pathname) + }) + const treeActive = new Promise((reported) => { + page.on('console', (message) => { + const text = message.text() + if (text.includes(TREE_ACTIVE)) reported(text) + if (message.type() === 'error' || message.type() === 'warning') consoleErrors.push(text) + }) + }) + try { + await page.goto(`${origin}/preview.html?preview-fixture=none`, { waitUntil: 'domcontentloaded' }) + expect(await page.getByRole('heading', { name: '选择 Preview 数据源' }).count()).toBe(0) + const bootLine = await within( + treeActive, + BOOT_TIMEOUT_MS, + `empty preview boot: the worker never reported "${TREE_ACTIVE}"`, + ) + expect(bootLine).toContain(`image lowering=${WRAPPER_CONTRACT}`) + expect(bootLine).toContain('data overlays=0') + await page.getByRole('textbox', { name: 'Choose workspace' }).waitFor({ timeout: HERO_TIMEOUT_MS }) + const sessionCount = await page.evaluate(async () => { + const transport = (globalThis as typeof globalThis & { + __DSH_TRANSPORT__?: { fetch(input: string, init: RequestInit): Promise } + }).__DSH_TRANSPORT__ + if (transport === undefined) throw new Error('empty preview transport is absent after boot') + const response = await transport.fetch('/api/session/list', { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ + type: 'client-request', rpcId: 'empty-preview-session-list', method: 'session/list', + payload: { args: { _request: {} } }, + }), + }) + const body = await response.json() as { + result: { ok: true; value: { items: unknown[] } } | { ok: false; error: { message: string } } + } + if (!body.result.ok) throw new Error(`empty session/list failed: ${body.result.error.message}`) + return body.result.value.items.length + }) + expect(sessionCount).toBe(0) + expect(pageErrors.map(error => error.message)).toEqual([]) + expect(failedResponses).toEqual(['/plugins/events']) + expect(consoleErrors.filter(line => !line.includes('Failed to load resource: the server responded with a status of 404'))) + .toEqual([]) + } catch (error) { + await saveFailureShot(page, 'preview-boot-empty') + throw pageErrors.length === 0 + ? error + : new AggregateError([error, ...pageErrors], 'empty preview boot failed, with uncaught page errors') + } finally { + await page.close() + } +} diff --git a/apps/web/tests/snapshots/preview-boot/source-chooser.expected.md b/apps/web/tests/snapshots/preview-boot/source-chooser.expected.md new file mode 100644 index 0000000000..1ba068c6b5 --- /dev/null +++ b/apps/web/tests/snapshots/preview-boot/source-chooser.expected.md @@ -0,0 +1,15 @@ +- form "Choose Preview data": + - heading "Choose Preview data" [level=1] + - paragraph: Data mounts before the Worker and application start. Refresh to choose again. + - group "Filesystem source": + - text: Filesystem source + - radio "Empty environment Load only the base runtime to verify first launch and workspace creation." + - strong: Empty environment + - text: Load only the base runtime to verify first launch and workspace creation. + - radio "Built-in showcase Sample workspace, tool cards, subagents, and paged history." [checked] + - strong: Built-in showcase + - text: Sample workspace, tool cards, subagents, and paged history. + - radio "WebFS directory Requires directory access and will be available after the WebFS provider lands." [disabled] + - strong: WebFS directory + - text: Requires directory access and will be available after the WebFS provider lands. + - button "Start Preview" diff --git a/apps/web/tests/subagent-interrupt-ui.e2e.ts b/apps/web/tests/subagent-interrupt-ui.e2e.ts index 592d55abe6..0eca3016fd 100644 --- a/apps/web/tests/subagent-interrupt-ui.e2e.ts +++ b/apps/web/tests/subagent-interrupt-ui.e2e.ts @@ -203,6 +203,7 @@ describe.skipIf(MODE === 'record')('web e2e: composer interrupt for a running co name: 'Parent session offline; sending is unavailable but you can still stop the run', }) await input.waitFor({ timeout: 15_000 }) + await page.getByText(INITIAL, { exact: true }).waitFor({ timeout: 15_000 }) expect(await input.isDisabled()).toBe(true) const stop = page.getByRole('button', { name: 'Stop generating' }) expect(await stop.count()).toBe(1) @@ -247,7 +248,7 @@ describe.skipIf(MODE === 'record')('web e2e: composer interrupt for a running co await waitFor(() => existsSync(rearmedReadyFile), 'the re-armed child turn to open') expect(scaffold.ctx.agents.get(childId)?.status).toBe('running') } finally { - await page.unroute(pattern) + await page.unrouteAll({ behavior: 'wait' }) } }, 60_000) diff --git a/docs/subsystems/client-modules.i18n.yaml b/docs/subsystems/client-modules.i18n.yaml index 9e4e1a9dd7..de013f0be4 100644 --- a/docs/subsystems/client-modules.i18n.yaml +++ b/docs/subsystems/client-modules.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/subsystems/client-modules.md -client-modules.md: ae767c6a098cb1b7188c06e61efd2798eb93ff66 -client-modules.zh.md: 710453e3e10c298ed8c2b87bbab6e8e9eb154a87 +client-modules.md: 84d673669825495b7e22933da238daba24ec7b13 +client-modules.zh.md: 7cc353007732188f06738a745b750161a11366a5 diff --git a/docs/subsystems/client-modules.md b/docs/subsystems/client-modules.md index ae767c6a09..84d6736698 100644 --- a/docs/subsystems/client-modules.md +++ b/docs/subsystems/client-modules.md @@ -14,11 +14,10 @@ The graph is the wire single source between the Node and browser halves: the hos /** * One composed client entry pushed by the host (a graph row). Wire * single source: the host node half (package root) produces this same shape. - * `immediately` marks stage-one prefetch; `inject` is informational graph - * metadata (the authoritative edges live in each package's `dsh.client` - * declaration and reach fibers through entry creation). `external` carries - * module-graph edges: unlike `inject`, they constrain code arrival because - * `require` is synchronous (see {@link WebBootGraph.entries}). + * `immediately` marks stage-one prefetch. `inject` names package rows whose + * factories must arrive before this row materializes, while Cordis separately + * uses the same package edges to compose entries. `external` carries exact + * non-inject module requests (see {@link WebBootGraph.entries}). */ interface WebBootEntry { /** Entry name == package name. */ @@ -27,7 +26,7 @@ interface WebBootEntry { url: string /** Bundle content hash (cache-busting consistency anchor). */ rev: string - /** Package-name dependency edges, informational (preflight display / HMR diffing). */ + /** Package-name dependency edges used for factory arrival and plugin composition. */ inject?: string[] /** Stage-one prefetch mark: load the script for factory registration during module-face boot. */ immediately?: boolean diff --git a/docs/subsystems/client-modules.zh.md b/docs/subsystems/client-modules.zh.md index 710453e3e1..7cc3530077 100644 --- a/docs/subsystems/client-modules.zh.md +++ b/docs/subsystems/client-modules.zh.md @@ -14,11 +14,10 @@ Web 插件表：[dsh-client-modules](../../packages/client/modules) 中 client /** * One composed client entry pushed by the host (a graph row). Wire * single source: the host node half (package root) produces this same shape. - * `immediately` marks stage-one prefetch; `inject` is informational graph - * metadata (the authoritative edges live in each package's `dsh.client` - * declaration and reach fibers through entry creation). `external` carries - * module-graph edges: unlike `inject`, they constrain code arrival because - * `require` is synchronous (see {@link WebBootGraph.entries}). + * `immediately` marks stage-one prefetch. `inject` names package rows whose + * factories must arrive before this row materializes, while Cordis separately + * uses the same package edges to compose entries. `external` carries exact + * non-inject module requests (see {@link WebBootGraph.entries}). */ interface WebBootEntry { /** Entry name == package name. */ @@ -27,7 +26,7 @@ interface WebBootEntry { url: string /** Bundle content hash (cache-busting consistency anchor). */ rev: string - /** Package-name dependency edges, informational (preflight display / HMR diffing). */ + /** Package-name dependency edges used for factory arrival and plugin composition. */ inject?: string[] /** Stage-one prefetch mark: load the script for factory registration during module-face boot. */ immediately?: boolean diff --git a/knip.json b/knip.json index b457dd54f3..b6c4e33254 100644 --- a/knip.json +++ b/knip.json @@ -241,7 +241,8 @@ "packages/experimental/webworker-runtime": { "entry": [ "tests/**/*.spec.ts", - "tests/compile/transform-corpus-check.ts" + "tests/compile/transform-corpus-check.ts", + "tests/fixtures/vfs-example/workspace/src/preview.ts" ], "project": [ "src/**/*.ts", diff --git a/packages/client/modules/src/client/manifest.ts b/packages/client/modules/src/client/manifest.ts index e5aff58f5a..ba0e521295 100644 --- a/packages/client/modules/src/client/manifest.ts +++ b/packages/client/modules/src/client/manifest.ts @@ -42,11 +42,10 @@ declare module '@deepseek-ai/cordis' { /** * One composed client entry pushed by the host (a graph row). Wire * single source: the host node half (package root) produces this same shape. - * `immediately` marks stage-one prefetch; `inject` is informational graph - * metadata (the authoritative edges live in each package's `dsh.client` - * declaration and reach fibers through entry creation). `external` carries - * module-graph edges: unlike `inject`, they constrain code arrival because - * `require` is synchronous (see {@link WebBootGraph.entries}). + * `immediately` marks stage-one prefetch. `inject` names package rows whose + * factories must arrive before this row materializes, while Cordis separately + * uses the same package edges to compose entries. `external` carries exact + * non-inject module requests (see {@link WebBootGraph.entries}). */ export interface WebBootEntry { /** Entry name == package name. */ @@ -55,7 +54,7 @@ export interface WebBootEntry { url: string /** Bundle content hash (cache-busting consistency anchor). */ rev: string - /** Package-name dependency edges, informational (preflight display / HMR diffing). */ + /** Package-name dependency edges used for factory arrival and plugin composition. */ inject?: string[] /** Stage-one prefetch mark: load the script for factory registration during module-face boot. */ immediately?: boolean @@ -83,6 +82,8 @@ export interface BootModuleRow { url: string /** Bundle content hash. */ rev: string + /** Injected package rows whose factories arrive before this row materializes. */ + inject: string[] /** Module specifiers this row requests from the module table ([] when the wire omits them). */ external: string[] } @@ -176,6 +177,7 @@ export function parseBootManifest(wire: unknown): BootManifest { id: row.id, url: row.url, rev: row.rev, + inject: inject === undefined ? [] : [...inject], external: external === undefined ? [] : [...external], }) plugins.push({ diff --git a/packages/client/modules/src/client/system.ts b/packages/client/modules/src/client/system.ts index d4cbb5597b..4dd5311378 100644 --- a/packages/client/modules/src/client/system.ts +++ b/packages/client/modules/src/client/system.ts @@ -124,8 +124,12 @@ export class ClientModuleSystem implements ClientModuleLoader { return task } - /** Register each unresolved dynamic request before registering its consumer. */ - private async arriveGraphRow(row: BootModuleRow, open: readonly string[] = []): Promise { + /** Register each injected package and unresolved dynamic request before its consumer. */ + private async arriveGraphRow( + row: BootModuleRow, + open: readonly string[] = [], + visited = new Set(), + ): Promise { const cycleStart = open.indexOf(row.id) if (cycleStart !== -1) { throw new Error( @@ -133,12 +137,18 @@ export class ClientModuleSystem implements ClientModuleLoader { + '(the host must reject this graph before serving it)', ) } + if (visited.has(row.id)) return + visited.add(row.id) const next = [...open, row.id] for (const request of row.external) { const id = stripClientSuffix(request) if (this.seed.has(request) || this.loadCache.has(id)) continue const dependency = this.graphRows.get(id) - if (dependency !== undefined) await this.arriveGraphRow(dependency, next) + if (dependency !== undefined) await this.arriveGraphRow(dependency, next, visited) + } + for (const packageName of row.inject) { + const dependency = this.graphRows.get(packageName) + if (dependency !== undefined) await this.arriveGraphRow(dependency, [], visited) } await this.arrive(row) } diff --git a/packages/client/modules/tests/loader.client.spec.ts b/packages/client/modules/tests/loader.client.spec.ts index 747067ba37..07e83e306b 100644 --- a/packages/client/modules/tests/loader.client.spec.ts +++ b/packages/client/modules/tests/loader.client.spec.ts @@ -20,7 +20,7 @@ afterEach(() => { }) const row = (id: string, fields: Partial = {}): BootModuleRow => - ({ id, url: `/plugins/${id}/client.js?rev=0`, rev: '0', external: [], ...fields }) + ({ id, url: `/plugins/${id}/client.js?rev=0`, rev: '0', inject: [], external: [], ...fields }) interface Bench { loader: ClientModuleLoader @@ -147,6 +147,22 @@ describe('lazy CJS arrival', () => { expect(exports.react.marker).toBe('react') }) + it('registers injected package factories before materializing a consumer', async () => { + const b = bench([ + row('consumer', { inject: ['provider'] }), + row('provider', { inject: ['consumer'] }), + ], { + consumer: req => ({ provider: req('provider/client') }), + provider: () => ({ marker: 'provider' }), + }) + const exports = await b.loader.import('consumer', '', {}) as { provider: { marker: string } } + expect(b.fetched).toEqual([ + '/plugins/provider/client.js?rev=0', + '/plugins/consumer/client.js?rev=0', + ]) + expect(exports.provider.marker).toBe('provider') + }) + it('concurrent callers share one in-flight arrival and materialize once', async () => { const ran: string[] = [] const url = '/plugins/a/client.js?rev=0' @@ -306,13 +322,13 @@ describe('boot manifest wire', () => { const manifest = parseBootManifest({ rev: 'graph', entries: [ - { id: 'a', url: '/plugins/a/client.js', rev: '1' }, + { id: 'a', url: '/plugins/a/client.js', rev: '1', inject: ['b'] }, { id: 'b', url: '/plugins/b/client.js', rev: '2', external: ['react'] }, ], }) expect(manifest.modules).toEqual([ - { id: 'a', url: '/plugins/a/client.js', rev: '1', external: [] }, - { id: 'b', url: '/plugins/b/client.js', rev: '2', external: ['react'] }, + { id: 'a', url: '/plugins/a/client.js', rev: '1', inject: ['b'], external: [] }, + { id: 'b', url: '/plugins/b/client.js', rev: '2', inject: [], external: ['react'] }, ]) }) diff --git a/packages/experimental/webworker-packer/README.i18n.yaml b/packages/experimental/webworker-packer/README.i18n.yaml index cab7fecde4..5e96d5f120 100644 --- a/packages/experimental/webworker-packer/README.i18n.yaml +++ b/packages/experimental/webworker-packer/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/experimental/webworker-packer/README.md -README.md: eb6c4c60106ebb7f6bb123116a749f152dea79a0 -README.zh.md: defa06c048125d627002c445aca491b4a33c4026 +README.md: 39d084bc631db387a3b6e526a3a374bc9f766577 +README.zh.md: 11aa04f3db36c09525bc4d4945f77e278602cc0f diff --git a/packages/experimental/webworker-packer/README.md b/packages/experimental/webworker-packer/README.md index eb6c4c6010..39d084bc63 100644 --- a/packages/experimental/webworker-packer/README.md +++ b/packages/experimental/webworker-packer/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The VFS image packer: turns one composed profile into the single gzip-compressed tar the browser worker inflates and mounts as its filesystem ([experimental stance](../../../.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.md)). Nothing is compiled from source — the image carries the repository's real build products, so a preview deployment debugs exactly what the served deployment ships. +The VFS image packer: turns one composed profile into the gzip-compressed base tar the browser worker mounts as its filesystem, and opaque data trees into ordered overlay tars ([experimental stance](../../../.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.md)). Nothing is compiled from source — the base image carries the repository's real build products, so a preview deployment debugs exactly what the served deployment ships. The pack is a three-layer standard stack: @@ -10,7 +10,9 @@ The pack is a three-layer standard stack: 2. **Publish view** — each workspace package contributes the slice npm would publish (`files` through picomatch) minus the rule tables in `src/rules.ts` (no sources, no workspace `dist/`; external packages keep their trees minus the same exclude globs). 3. **Reachability sweep** — the runtime loader's own resolution walks from every workspace export face plus the worker assembly's seeds (`IMAGE_ENTRY_SEEDS`), lowering each reached module to the wrapper contract at pack time. Page assets (`lib/client.js` behind `./client` exports) ship verbatim; an unresolvable request from our own code fails the pack, third-party ones are tolerated to fail loud at require time. -`repository.ts` owns the repo-shaped inputs (workspace scan of `vendor/`, `packages/`, `apps/`; profile composition through the real CLI dump path); `pack.ts` owns none of them, so the same library packs a different tree by being called differently. The CLI is `dsh-pack-vfs-image --out [--profile web]`; `apps/web`'s `build:preview` runs it after the preview shell build. +`repository.ts` owns the repo-shaped inputs (workspace scan of `vendor/`, `packages/`, `native/landlock-run/packages/`, and `apps/`; profile composition through the real CLI dump path); `pack.ts` owns none of them, so the same library packs a different tree by being called differently. The native scan makes the Landlock entry package an ordinary published-view dependency while its executable remains a Worker platform implementation. The CLI is `dsh-pack-vfs-image --out [--profile web]`; `apps/web`'s `build:preview` runs it after the preview shell build. + +The repository adapter also declares the preview-only fixture trees under `webworker-runtime/tests/fixtures/`. The CLI packs each named fixture into a separate deterministic overlay archive plus a browser-readable manifest. Overlay files bypass npm publish-view and module-reachability exclusions, so dot directories and example source files remain intact; their mounts are limited to `home/` and `workspace/`. `pack.ts` treats them as opaque bytes, and Session and Workspace interpretation stays in the runtime packages that own those formats. ## Model Experience diff --git a/packages/experimental/webworker-packer/README.zh.md b/packages/experimental/webworker-packer/README.zh.md index defa06c048..11aa04f3db 100644 --- a/packages/experimental/webworker-packer/README.zh.md +++ b/packages/experimental/webworker-packer/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -VFS 镜像打包器：把一份合成 profile 变成浏览器 worker 解压后当文件系统挂载的单个 gzip 压缩 tar（[experimental 定位](../../../.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.zh.md)）。不做任何源码编译——镜像携带仓库真实构建产物，预览部署调试的正是 served 部署交付的字节。 +VFS 镜像打包器：把一份合成 profile 变成浏览器 worker 挂载为文件系统的 gzip 压缩基础 tar，并把不透明数据目录变成按序应用的 overlay tar（[experimental 定位](../../../.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.zh.md)）。不做任何源码编译——基础镜像携带仓库真实构建产物，预览部署调试的正是 served 部署交付的字节。 打包是三层标准栈： @@ -10,7 +10,9 @@ VFS 镜像打包器：把一份合成 profile 变成浏览器 worker 解压后 2. **发布视图**——每个 workspace 包贡献 npm 会发布的切片（`files` 走 picomatch），再减去 `src/rules.ts` 的规则表（无源码、无 workspace `dist/`；外部包保留整棵减同一套 exclude glob）。 3. **可达性 sweep**——用运行时加载器自己的解析，从全部 workspace 导出面加 worker 装配种子（`IMAGE_ENTRY_SEEDS`）出发，pack 时把每个可达模块降低到包装契约。页面资产（`./client` 导出背后的 `lib/client.js`）原样直发；自家代码的不可解析请求打包即失败，第三方的容忍到 require 时 fail loud。 -`repository.ts` 拥有仓库形态输入（`vendor/`、`packages/`、`apps/` 的 workspace 扫描；经真 CLI dump 路径合成 profile）；`pack.ts` 一概不拥有，同一库换参即可打另一棵树。CLI 为 `dsh-pack-vfs-image --out [--profile web]`；`apps/web` 的 `build:preview` 在预览壳构建后运行它。 +`repository.ts` 拥有仓库形态输入（`vendor/`、`packages/`、`native/landlock-run/packages/` 与 `apps/` 的 workspace 扫描；经真 CLI dump 路径合成 profile）；`pack.ts` 一概不拥有，同一库换参即可打另一棵树。Native 扫描使 Landlock 入口包成为普通发布视图依赖，其可执行文件仍由 Worker 平台实现。CLI 为 `dsh-pack-vfs-image --out [--profile web]`；`apps/web` 的 `build:preview` 在预览壳构建后运行它。 + +仓库适配层还声明 `webworker-runtime/tests/fixtures/` 下仅用于 preview 的 fixture tree。CLI 会把每套具名 fixture 打成一份独立的确定性 overlay 归档，并写出浏览器可读的 manifest。Overlay 文件绕过 NPM 发布视图和模块可达性排除规则，因此点目录与示例源码会完整保留；其挂载位置仅限 `home/` 与 `workspace/`。`pack.ts` 把它们视为不透明字节；Session 与 Workspace 的解释仍归拥有这些格式的 runtime 包。 ## 模型体验 diff --git a/packages/experimental/webworker-packer/package.json b/packages/experimental/webworker-packer/package.json index e9219b35f9..a8155676dd 100644 --- a/packages/experimental/webworker-packer/package.json +++ b/packages/experimental/webworker-packer/package.json @@ -1,6 +1,6 @@ { "name": "@deepseek-ai/dsh-experimental-webworker-packer", - "description": "Build-time packer for the browser runtime's VFS image: materializes a profile's package closure into one gzip-compressed tar the worker mounts, with every module body pre-transformed", + "description": "Build-time packer for the browser runtime's base VFS image and ordered data-overlay archives", "version": "0.1.1-rc.2", "private": true, "repository": { diff --git a/packages/experimental/webworker-packer/src/bin.ts b/packages/experimental/webworker-packer/src/bin.ts index 57a3c77730..093fac23d5 100644 --- a/packages/experimental/webworker-packer/src/bin.ts +++ b/packages/experimental/webworker-packer/src/bin.ts @@ -1,17 +1,23 @@ #!/usr/bin/env node /** - * Pack a VFS image from this repository: compose the profile, materialize the - * closure, lower every module body, write the gzip-compressed tar. + * Pack a Preview deployment from this repository: compose and lower the base + * image, then write each named fixture overlay and their manifest. * * Usage: dsh-pack-vfs-image --out [--profile web] [--root /dsh] * node --import tsx/esm src/bin.ts --out ../../apps/web/dist/preview/vfs-image.tar.gz * @module @deepseek-ai/dsh-experimental-webworker-packer/src/bin */ import { mkdirSync, writeFileSync } from 'node:fs' -import { dirname, isAbsolute, resolve } from 'node:path' +import { dirname, isAbsolute, join, resolve } from 'node:path' import { fileURLToPath } from 'node:url' -import { packVfsImage } from './pack.ts' -import { composeProfile, configTrees, describePack, indexWorkspacePackages } from './repository.ts' +import { + PREVIEW_FIXTURE_MANIFEST_FILE, PREVIEW_FIXTURE_MANIFEST_VERSION, + type PreviewFixtureManifest, +} from '@deepseek-ai/dsh-experimental-webworker-runtime' +import { packVfsImage, packVfsOverlay } from './pack.ts' +import { + composeProfile, configTrees, describePack, indexWorkspacePackages, previewFixtures, +} from './repository.ts' /** * Read one `--flag value` pair. @@ -54,4 +60,30 @@ if (result.missing.length > 0) { mkdirSync(dirname(outputFile), { recursive: true }) writeFileSync(outputFile, result.image) -process.stdout.write(describePack(result, repoRoot, outputFile).join('\n')) + +const fixtureDefinitions = previewFixtures(repoRoot) +const fixtureDirectory = join(dirname(outputFile), 'fixtures') +mkdirSync(fixtureDirectory, { recursive: true }) +const fixtureLines: string[] = [] +const fixtures = fixtureDefinitions.map((fixture) => { + const packed = packVfsOverlay(fixture.trees) + const file = `fixtures/${fixture.id}.tar.gz` + writeFileSync(join(dirname(outputFile), file), packed.image) + fixtureLines.push(` fixture overlay ${fixture.id} (${String(packed.image.byteLength)} B compressed)`) + return { + id: fixture.id, + label: fixture.label, + description: fixture.description, + overlays: [file], + } +}) +const manifest: PreviewFixtureManifest = { + version: PREVIEW_FIXTURE_MANIFEST_VERSION, + defaultFixture: fixtures[0]?.id ?? null, + fixtures, +} +writeFileSync( + join(dirname(outputFile), PREVIEW_FIXTURE_MANIFEST_FILE), + `${JSON.stringify(manifest, null, 2)}\n`, +) +process.stdout.write([...describePack(result, repoRoot, outputFile), ...fixtureLines, ''].join('\n')) diff --git a/packages/experimental/webworker-packer/src/index.ts b/packages/experimental/webworker-packer/src/index.ts index ea054b26b0..ea203a6f7a 100644 --- a/packages/experimental/webworker-packer/src/index.ts +++ b/packages/experimental/webworker-packer/src/index.ts @@ -7,9 +7,10 @@ export { type ImageFiles, type TransformOutcome, } from './transform-image.ts' export { - CONFIG_PATH, DEFAULT_ROOT, MANIFEST_PATH, packVfsImage, - type ConfigTree, type PackOptions, type PackResult, + CONFIG_PATH, DEFAULT_ROOT, MANIFEST_PATH, packVfsImage, packVfsOverlay, + type ConfigTree, type ImageTree, type PackOptions, type PackOverlayResult, type PackResult, } from './pack.ts' export { - composeProfile, configTrees, describePack, indexWorkspacePackages, + composeProfile, configTrees, describePack, indexWorkspacePackages, previewFixtures, + type PreviewFixture, } from './repository.ts' diff --git a/packages/experimental/webworker-packer/src/pack.ts b/packages/experimental/webworker-packer/src/pack.ts index a5dcb00156..3e86610233 100644 --- a/packages/experimental/webworker-packer/src/pack.ts +++ b/packages/experimental/webworker-packer/src/pack.ts @@ -19,6 +19,7 @@ import { gzipSync } from 'node:zlib' import { lowerModuleSource, MemoryVfs, packTar, WorkerModuleLoader, DEFAULT_ROOT, IMAGE_CONFIG_PATH, IMAGE_EMPTY_DIRECTORIES, IMAGE_MANIFEST_PATH, + IMAGE_OVERLAY_DIRECTORIES, } from '@deepseek-ai/dsh-experimental-webworker-runtime' import picomatch from 'picomatch' import yaml from 'js-yaml' @@ -52,12 +53,16 @@ const workspaceExcluded = picomatch([...EXCLUDE, ...EXCLUDE_WORKSPACE], { dot: t /** Page-asset matcher over image paths ({@link PAGE_ASSETS}). */ const pageAsset = picomatch([...PAGE_ASSETS], { dot: true }) -/** One directory tree to copy in verbatim beside the composition. */ -export interface ConfigTree { +/** One directory tree to copy into the image at a caller-selected mount. */ +export interface ImageTree { /** Image path to mount it at, relative to the virtual root. */ readonly mount: string /** Absolute source directory. */ readonly directory: string +} + +/** One configuration tree whose plugin rows may extend the package roster. */ +export interface ConfigTree extends ImageTree { /** * Whether plugin names inside its `.yml` files join the materialization closure. * An agent preset mounts plugins the base composition never lists, and creating a @@ -119,6 +124,14 @@ export interface PackResult { readonly contract: string } +/** One deterministic data-overlay archive and its uncompressed entries. */ +export interface PackOverlayResult { + /** Gzip-compressed ustar bytes consumed by the Worker host. */ + readonly image: Uint8Array + /** Every path in the overlay before compression. */ + readonly files: ImageFiles +} + const readJson = (file: string): Record => JSON.parse(readFileSync(file, 'utf8')) as Record @@ -204,20 +217,28 @@ function resolveDependency(fromDirectory: string, name: string): string | undefi /** * Collect files under one directory. Traversal mechanics live here — nested - * `node_modules` never mounts (the image is flat) and dot directories are - * tooling residue at any depth — while every judgement call comes in through - * `keep` (the {@link EXCLUDE} tables and the npm publish view). + * package/config collection flattens nested `node_modules` and prunes dot + * directories, while seed collection preserves every directory. Every file + * judgement comes in through `keep` (the {@link EXCLUDE} tables and the npm + * publish view, or an unconditional seed predicate). * @param root - Source directory. * @param into - Image entries to add to. * @param prefix - Image path prefix. * @param keep - Filter over root-relative paths. + * @param preserveDirectories - Whether dot directories and nested `node_modules` + * are ordinary fixture content rather than package-manager residue. */ -function collectTree(root: string, into: ImageFiles, prefix: string, keep: (relativePath: string) => boolean): void { +function collectTree( + root: string, + into: ImageFiles, + prefix: string, + keep: (relativePath: string) => boolean, + preserveDirectories = false, +): void { const walk = (directory: string): void => { for (const entry of readdirSync(directory, { withFileTypes: true })) { if (entry.isDirectory()) { - if (entry.name === 'node_modules') continue - if (entry.name.startsWith('.')) continue + if (!preserveDirectories && (entry.name === 'node_modules' || entry.name.startsWith('.'))) continue walk(join(directory, entry.name)) continue } @@ -621,3 +642,32 @@ export function packVfsImage(options: PackOptions): PackResult { contract: WRAPPER_CONTRACT, } } + +/** + * Pack opaque data trees into one ordered VFS overlay. + * + * Overlay mounts are restricted to the runtime-owned data directories, so an + * overlay cannot replace configuration, the lowering manifest, or modules. + * Files bypass package excludes and module reachability processing; later + * trees replace earlier files at the same path. + * @param trees - Absolute source directories and their data-directory mounts. + * @returns Deterministic compressed archive plus its uncompressed entries. + */ +export function packVfsOverlay(trees: readonly ImageTree[]): PackOverlayResult { + const files: ImageFiles = {} + for (const tree of trees) { + if (!existsSync(tree.directory)) { + throw new Error(`vfs overlay: tree ${tree.mount} is missing at ${tree.directory}`) + } + const mount = tree.mount.replace(/^\.\//, '').replace(/\/$/, '') + const first = mount.split('/')[0] + if (mount === '' || first === undefined || !IMAGE_OVERLAY_DIRECTORIES.includes(first) + || mount.split('/').some(segment => segment === '' || segment === '.' || segment === '..')) { + throw new Error( + `vfs overlay: mount ${JSON.stringify(tree.mount)} must stay under ${IMAGE_OVERLAY_DIRECTORIES.join(' or ')}`, + ) + } + collectTree(tree.directory, files, mount, () => true, true) + } + return { image: compressImage(packTar(files)), files } +} diff --git a/packages/experimental/webworker-packer/src/repository.ts b/packages/experimental/webworker-packer/src/repository.ts index 38ec64bd1d..cdfda6e547 100644 --- a/packages/experimental/webworker-packer/src/repository.ts +++ b/packages/experimental/webworker-packer/src/repository.ts @@ -12,15 +12,15 @@ import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync } from 'node import { tmpdir } from 'node:os' import { join, relative } from 'node:path' import { DSH_HOME_ENV } from '@deepseek-ai/dsh-home-paths' -import type { ConfigTree, PackResult } from './pack.ts' +import type { ConfigTree, ImageTree, PackResult } from './pack.ts' /** * Repository directories scanned for workspace and vendored packages. The - * image only ever materializes runtime packages, which all live here; - * examples, python, and native are never on a roster's dependency chain (the - * native addon is a replaced external). + * image only ever materializes runtime packages, which live here. The Landlock + * package family contributes its unchanged JavaScript entry from `native/`; + * examples and python never occur on a roster's dependency chain. */ -const WORKSPACE_SCAN_ROOTS = ['vendor', 'packages', 'apps'] +const WORKSPACE_SCAN_ROOTS = ['vendor', 'packages', 'native/landlock-run/packages', 'apps'] /** Composition entry point package: the `dsh` CLI, run from source. */ const CLI_PACKAGE = 'apps/cli' @@ -28,6 +28,21 @@ const CLI_PACKAGE = 'apps/cli' /** Composition entry point: the `dsh` CLI, run from source. */ const CLI_ENTRY = `${CLI_PACKAGE}/src/bin.ts` +/** Repository-owned deterministic filesystem content offered by the preview. */ +const PREVIEW_EXAMPLE_ROOT = 'packages/experimental/webworker-runtime/tests/fixtures/vfs-example' + +/** One built-in Preview source and the trees packed into its overlay. */ +export interface PreviewFixture { + /** URL/query-safe identifier. */ + readonly id: string + /** User-facing chooser label. */ + readonly label: string + /** User-facing chooser detail. */ + readonly description: string + /** Opaque trees packed into this fixture's overlay archive. */ + readonly trees: readonly ImageTree[] +} + /** * Index every workspace and vendored package by name. * @param repoRoot - Absolute repository root. @@ -130,6 +145,23 @@ export function configTrees(repoRoot: string): ConfigTree[] { }) } +/** + * Built-in filesystem fixtures offered by the repository preview. + * Session and Workspace semantics remain opaque here; the owning runtime tests + * validate those files through their production readers. + * @param repoRoot - Absolute repository root. + * @returns Named chooser entries and their overlay trees. + */ +export function previewFixtures(repoRoot: string): PreviewFixture[] { + const root = join(repoRoot, PREVIEW_EXAMPLE_ROOT) + return [{ + id: 'vfs-example', + label: 'Built-in showcase', + description: 'Sample workspace, tool cards, subagents, and paged history.', + trees: ['home', 'workspace'].map(mount => ({ mount, directory: join(root, mount) })), + }] +} + /** * Render one pack as the lines a build log should carry. * diff --git a/packages/experimental/webworker-packer/tests/image-loadable.spec.ts b/packages/experimental/webworker-packer/tests/image-loadable.spec.ts index 688b21c279..766f974735 100644 --- a/packages/experimental/webworker-packer/tests/image-loadable.spec.ts +++ b/packages/experimental/webworker-packer/tests/image-loadable.spec.ts @@ -20,20 +20,52 @@ import { existsSync } from 'node:fs' import { join } from 'node:path' import { fileURLToPath } from 'node:url' import { describe, expect, it } from 'vitest' +import { FiberState } from '@deepseek-ai/cordis' import { createNodeBuiltins, REPLACED_PREFIXES } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtins.ts' -import { WorkerModuleLoader } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/module-system/module-loader.ts' +import { + setActiveModuleLoader, WorkerModuleLoader, +} from '@deepseek-ai/dsh-experimental-webworker-runtime/src/module-system/module-loader.ts' import { inflateImage } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/image-gzip.ts' import { loadVfsImage } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/memory.ts' -import { indexWorkspacePackages } from '../src/repository.ts' -import { DEFAULT_ROOT, MANIFEST_PATH, packVfsImage } from '../src/pack.ts' +import { setActiveVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/active.ts' +import { indexWorkspacePackages, previewFixtures } from '../src/repository.ts' +import { DEFAULT_ROOT, MANIFEST_PATH, packVfsImage, packVfsOverlay } from '../src/pack.ts' const repoRoot = fileURLToPath(new URL('../../../../', import.meta.url)) /** A leaf workspace package: real build output, no dependencies to drag in. */ const SUBJECT = '@deepseek-ai/dsh-timeout' +const LANDLOCK = '@deepseek-ai/node-addon-landlock-run' +const PLUGIN_INVENTORY = '@deepseek-ai/dsh-plugin-package-inventory-deepseek' const workspaces = indexWorkspacePackages(repoRoot) +describe('preview example overlays', () => { + it('packs source-looking paths and dot directories into a separate overlay', () => { + const fixture = previewFixtures(repoRoot)[0] + expect(fixture?.id).toBe('vfs-example') + const result = packVfsOverlay(fixture?.trees ?? []) + expect(new TextDecoder().decode(result.files['workspace/src/preview.ts'])) + .toContain("previewStatus = 'ready'") + expect(new TextDecoder().decode(result.files['workspace/.agents/skills/preview-tour/SKILL.md'])) + .toContain('name: preview-tour') + expect(Object.keys(result.files).filter(path => path.endsWith('/session.jsonl'))).toHaveLength(3) + }) + + it('fails loud when a declared seed tree is absent', () => { + expect(() => packVfsOverlay([ + { mount: 'workspace', directory: join(repoRoot, 'missing-preview-seed') }, + ])).toThrow(/tree workspace is missing/) + }) + + it('refuses overlays that could replace runtime files', () => { + const fixture = previewFixtures(repoRoot)[0] + expect(() => packVfsOverlay([ + { mount: 'config', directory: fixture?.trees[0]?.directory ?? repoRoot }, + ])).toThrow(/must stay under home or workspace/) + }) +}) + /** * The pack consumes built `lib/` output. An unbuilt checkout (the unit * coverage lane runs before any build) self-skips; the built lanes and every @@ -53,6 +85,24 @@ const packed = (): ReturnType => memo ??= packVfsImage({ entries: [], }) +let landlockMemo: ReturnType | undefined +const packedLandlock = (): ReturnType => landlockMemo ??= packVfsImage({ + config: `- id: subject\n name: '${LANDLOCK}'\n`, + profile: 'landlock-package-check', + workspaces, + resolveFrom: repoRoot, + entries: [], +}) + +let pluginInventoryMemo: ReturnType | undefined +const packedPluginInventory = (): ReturnType => pluginInventoryMemo ??= packVfsImage({ + config: `- id: subject\n name: '${PLUGIN_INVENTORY}'\n`, + profile: 'plugin-inventory-check', + workspaces, + resolveFrom: repoRoot, + entries: [], +}) + /** The image's archive, inflated once: mounting reads the tar, not the gzip member. */ let archiveMemo: Uint8Array | undefined const archive = async (): Promise => @@ -138,6 +188,95 @@ const archive = async (): Promise => expect(loader.usage().modules).toBeGreaterThan(0) }) + it('runs the unchanged Landlock entry package over the Worker platform executable', async () => { + const result = packedLandlock() + expect(workspaces.has(LANDLOCK)).toBe(true) + expect(result.packages.has(LANDLOCK)).toBe(true) + expect(result.missing).toEqual([]) + expect(Object.hasOwn(result.files, `node_modules/${LANDLOCK}/lib/index.js`)).toBe(true) + expect(createNodeBuiltins()[LANDLOCK]).toBeUndefined() + + const vfs = loadVfsImage(await inflateImage(result.image, 'the packed Landlock package'), DEFAULT_ROOT) + const loader = new WorkerModuleLoader({ + vfs, + root: DEFAULT_ROOT, + staticModules: createNodeBuiltins(), + staticModulePrefixes: REPLACED_PREFIXES, + }) + setActiveVfs(vfs) + setActiveModuleLoader(loader) + const landlock = loader.requireFrom(`${DEFAULT_ROOT}/workspace`)(LANDLOCK) as { + LAUNCHER_BIN: string + LAUNCHER_FAILURE_EXIT: number + launcherPath(): string + grantArgs(grants: { readOnly?: readonly string[]; readWrite?: readonly string[] }): string[] + probe(): string + } + + expect(landlock.LAUNCHER_BIN).toBe('landlock-run') + expect(landlock.LAUNCHER_FAILURE_EXIT).toBe(125) + expect(landlock.grantArgs({ readOnly: ['/'], readWrite: ['/tmp'] })).toEqual([ + '--ro', '/', '--rw', '/tmp', + ]) + expect(landlock.launcherPath()).toBe( + `${DEFAULT_ROOT}/node_modules/${LANDLOCK}/node_modules/${LANDLOCK}-${process.platform}-${process.arch}/bin/landlock-run`, + ) + expect(landlock.probe()).toBe('full') + }) + + it('prepares the unchanged plugin-package inventory through Worker createRequire paths', async () => { + const result = packedPluginInventory() + expect(result.missing).toEqual([]) + + const vfs = loadVfsImage(await inflateImage(result.image, 'the packed plugin inventory'), DEFAULT_ROOT) + const loader = new WorkerModuleLoader({ + vfs, + root: DEFAULT_ROOT, + staticModules: createNodeBuiltins(), + staticModulePrefixes: REPLACED_PREFIXES, + }) + setActiveVfs(vfs) + setActiveModuleLoader(loader) + const inventory = loader.requireFrom(`${DEFAULT_ROOT}/workspace`)(PLUGIN_INVENTORY) as { + apply(ctx: unknown, config: unknown): void + } + + type Prepared = { readonly value: { readonly version: number; readonly packages: readonly unknown[] } } + type Prepare = (request: { readonly body: object; readonly signal: AbortSignal }) => Promise + let prepare: Prepare | undefined + const baseUrl = `file://${DEFAULT_ROOT}/config/cordis.yml` + const tree: { readonly ctx: { readonly baseUrl: string }; entries(): readonly unknown[] } = { + ctx: { baseUrl }, + entries: () => [entry], + } + const entry = { + options: { name: PLUGIN_INVENTORY }, + disabled: false, + fiber: { state: FiberState.ACTIVE }, + parent: { tree }, + } + inventory.apply({ + baseUrl, + loader: tree, + deepseekLlmApiExtensions: { + register: (field: string, contribution: { readonly prepare: Prepare }): void => { + expect(field).toBe('dsh_plugin_packages') + prepare = contribution.prepare + }, + }, + }, {}) + + if (prepare === undefined) throw new Error('packed plugin inventory did not register its request contribution') + const prepared = await prepare({ body: {}, signal: new AbortController().signal }) + const manifest = JSON.parse(vfs.readFileSync( + `${DEFAULT_ROOT}/node_modules/${PLUGIN_INVENTORY}/package.json`, 'utf8', + ) as string) as { version: string } + expect(prepared.value).toEqual({ + version: 1, + packages: [{ name: PLUGIN_INVENTORY, version: manifest.version }], + }) + }) + it('refuses a body the packer did not lower, naming the image', async () => { // The case above only proves the packed bytes are wrappable. This is the // other half: the loader has no transform to fall back on, so an entry the diff --git a/packages/experimental/webworker-runtime/README.i18n.yaml b/packages/experimental/webworker-runtime/README.i18n.yaml index 0dced963dc..d0d0d13a6e 100644 --- a/packages/experimental/webworker-runtime/README.i18n.yaml +++ b/packages/experimental/webworker-runtime/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/experimental/webworker-runtime/README.md -README.md: b82c65b981be6a9405ae72e3a24a42c68b52696e -README.zh.md: 97160641a38095026d5103f2e423846bfb41d5a6 +README.md: 3e9b4fffe0b97a97adf218aa12fd1f4342d3bc6c +README.zh.md: 2552c659d1b735b0cf28b9b0d0808276d31d0a2a diff --git a/packages/experimental/webworker-runtime/README.md b/packages/experimental/webworker-runtime/README.md index b82c65b981..3e9b4fffe0 100644 --- a/packages/experimental/webworker-runtime/README.md +++ b/packages/experimental/webworker-runtime/README.md @@ -6,12 +6,12 @@ The browser worker host: the whole harness plugin tree runs inside one dedicated Three artifacts from one tsdown pipeline: -- **`lib/index.js` (assembly library)** — `createWorkerHost`/`startWorkerHost` mount the image (`storage/`), install the module loader (`module-system/`) and the `process` shim, boot the tree through the image's own `dsh-app-boot`, and hand the tunnel its serving seams. The image layout contract (`image-layout.ts`: virtual root, config/manifest paths, empty directories, the `lowered` wrapper-contract gate) is shared with the packer. Boot patches force the deployment-shaped rows: frontend serving off, JSONL session logs on the plaintext path, preset roots onto the image's `config/agent-presets`. -- **`lib/worker.js` (worker bundle)** — the assembly plus this package's Node-compatibility layer as one self-contained ES module. The module proxy table (`module-proxies.ts`) is the only platform fork: `node:*` builtins over VFS/tunnel/browser primitives, structural stubs that fail loud on the console for what a browser cannot do, and replaced externals. AsyncLocalStorage carries sync-stack causality across `await` through the snapshot/restore faces the pack-time lowering injects. The worker holds no compiler: an image the packer did not lower is refused at mount ([note](../../../.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.md)). -- **`src/shell/` (the worker's own process layer)** — a browser worker cannot fork, so `node:child_process` is not a stub but an implementation: `spawn` starts the command in its own Web Worker — this same bundle, told by its first frame to be a shell process — and reports it through the `ChildProcess` surface the subprocess service consumes. The command runs off the host's thread, `SIGKILL` terminates it whatever it is doing, and it reaches the VFS only by message (the host serves those frames). The grammar is `@yarnpkg/parsers`' `parseShell`; this package owns the evaluator (pipelines, `&&`/`||`, subshells, redirections, expansion, globs) and the command table, which is the only `/bin` that exists — a name it does not hold reports `command not found`, and `execSync`/`fork` still refuse, because they need a real process. -- **`lib/client.js` (page half)** — `connectWorkerHost(worker, { image? })` completes the pre-Cordis handshake: the opening `init` frame carries the image URL (the one deployment-shaped input), the boot payload delivers the structured index-injection table, and `applyIndexInjections` executes it before the shell entry runs. The tunnel exposes fetch-shaped transport, the API client, and `loadBundle` for the shell's boot seam. +- **`lib/index.js` (assembly library)** — `createWorkerHost`/`startWorkerHost` mount the base image and any ordered data overlays (`storage/`), install the module loader (`module-system/`) and the `process` shim, boot the tree through the image's own `dsh-app-boot`, and hand the tunnel its serving seams. Overlays may replace files only under `home/` and `workspace/`; they cannot replace the base manifest, configuration, or modules. The image layout contract (`image-layout.ts`: virtual root, config/manifest paths, empty directories, the `lowered` wrapper-contract gate) is shared with the packer. Boot patches force the deployment-shaped rows: frontend serving off, JSONL session logs on the plaintext path, preset roots onto the image's `config/agent-presets`. +- **`lib/worker.js` (worker bundle)** — the assembly plus this package's Node-compatibility layer as one self-contained ES module. The module proxy table (`module-proxies.ts`) is the only platform fork: `node:*` builtins over VFS/tunnel/browser primitives, structural stubs that fail loud on the console for what a browser cannot do, and native/binary package replacements. `node:module` supplies `createRequire().resolve` and `.resolve.paths()` over the image package root, so unchanged packages can discover manifests without evaluating their modules. VFS mutations drive `node:fs` callback, polling, and promise watchers; open descriptors retain file identity and access mode across rename, replacement, and unlink; `readable-stream` supplies the stream state machine used by file streams and unchanged image packages such as Chokidar and readdirp. AsyncLocalStorage carries sync-stack causality across `await` through the snapshot/restore faces the pack-time lowering injects. The worker holds no compiler: an image the packer did not lower is refused at mount ([note](../../../.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.md)). +- **`src/shell/` (the worker's own process layer)** — a browser worker cannot fork, so `node:child_process` is not a stub but an implementation: `spawn` starts the command in its own Web Worker — this same bundle, told by its first frame to be a shell process — and reports it through the `ChildProcess` surface the subprocess service consumes. The command runs off the host's thread, `SIGKILL` terminates it whatever it is doing, and it reaches the VFS only by message (the host serves those frames). Worker platform executables preserve native-package protocols such as Landlock without replacing their JavaScript packages or coupling their implementations to `node:child_process`; ordinary commands use the package's evaluator and coreutils command table. The grammar is `@yarnpkg/parsers`' `parseShell`, while `execSync`/`fork` still refuse because they need a real process. +- **`lib/client.js` (page half)** — startup has two independent stages. `chooseWorkerHostSource({ image?, fixtureManifest? })` optionally owns the boot barrier and fixture manifest: without `preview-fixture` it waits at the source chooser, while a valid query selects directly; either path returns ordered overlays. `connectWorkerHost(worker, { image?, overlays? })` remains the public base-runtime connector; callers that skip the chooser get an empty overlay list. `apps/web` invokes both and supplies its statically bundled Worker. The opening `init` frame carries the base and ordered overlay URLs, the boot payload delivers the structured index-injection table, and `applyIndexInjections` executes it before the shell entry runs. The tunnel exposes fetch-shaped transport, the API client, and `loadBundle` for the shell's boot seam. -Acceptance lives in `apps/web/tests/preview-boot.e2e.ts`, which serves the real built pages and drives the worker boot in headless Chromium. +Acceptance lives in `apps/web/tests/preview-boot.e2e.ts`, which serves the real built pages and drives the pre-boot chooser plus Worker activation in headless Chromium. The empty selection exercises first-run startup. The `vfs-example` overlay supplies ordinary workspace files and plaintext persistence artifacts for cold Workspace/Session discovery, tool presentation, subagent navigation, and history paging without a model request. The chooser reserves WebFS as a separate user-authorized source; that provider does not read the built-in fixture. ## Model Experience @@ -24,9 +24,9 @@ None; this package neither assembles nor sends a provider request. ## Known Limitations and Deferred Work - **The worker composition writes plaintext session logs** (`compression: 'none'` boot patch): it carries no Zstandard codec, so exported logs are `.jsonl`, never `.jsonl.zstd`. -- **The skill catalog is never cached in the worker** — `skill-filesystem` watches its roots through `node:fs.watchFile`, which this package refuses, so every discovery pass returns an incomplete observation and re-scans. Discovery itself stays correct; the cost is a re-scan on every pass. - **`node:vm`, `node:net`, `node:sqlite`, `node:worker_threads` are structural stubs**: every call reports its refusal on the console and throws. Rows needing a real process or realm isolation cannot run here. -- **The bash tool runs only under `danger-full-access`**: a browser has no kernel to confine a command with, so `ctx.sandbox.confine` fails loud in every other permission preset and the command never starts. The mode is the deployment's own user-facing switch, not a worker-specific composition. +- **Filesystem watchers observe only the mounted VFS**: image seeding is silent and the VFS has no symlinks or external writers. `persistent`, `ref()`, and `unref()` preserve the Node API but cannot control a dedicated Worker's lifetime because browsers expose no ref-counted event loop. +- **Worker confinement is a VFS boundary, not kernel Landlock**: `read-only` and `workspace-write` run the unchanged `@deepseek-ai/node-addon-landlock-run` JavaScript and launcher argv, but the process layer implements the logical `landlock-run` executable and enforces its grants on every shell filesystem request. `full` therefore covers the Worker command table and mounted VFS only; it does not claim arbitrary native-process execution or Linux kernel isolation. - **The worker bundle pins a path inside `@yarnpkg/parsers`** — the build resolves the package's own `lib/shell.js` instead of its root, whose barrel also re-exports the Syml parser and so drags js-yaml into a bundle that never parses that format (around 175 kB, plus its module body at worker start). The path is derived from the package manifest, so a layout change fails the build rather than reinstating the barrel; upgrading the dependency means re-checking that the shell parser still lives there. - **The shell is not bash**: no loops, functions, `case`, job control, or process substitution — the grammar stops at pipelines, `&&`/`||`, subshells, groups, redirections, and expansion. `&` runs its command to completion in place, `sed` accepts only substitution scripts, patterns are JavaScript regular expressions, and the command table holds coreutils only (no `git`, no network tools). - **A shell process has no synchronous filesystem**: it reads and writes the host's VFS by message, because blocking on a reply would need `SharedArrayBuffer`, which requires a cross-origin isolation GitHub Pages cannot grant. Directory-walking commands therefore cost one round trip per entry, and two concurrent commands can interleave their writes. diff --git a/packages/experimental/webworker-runtime/README.zh.md b/packages/experimental/webworker-runtime/README.zh.md index 97160641a3..2552c659d1 100644 --- a/packages/experimental/webworker-runtime/README.zh.md +++ b/packages/experimental/webworker-runtime/README.zh.md @@ -6,12 +6,12 @@ 一条 tsdown 管线出三个产物： -- **`lib/index.js`（装配库）**——`createWorkerHost`/`startWorkerHost` 挂载镜像（`storage/`）、安装模块加载器（`module-system/`）与 `process` shim、经镜像自带的 `dsh-app-boot` 启动插件树，并把服务缝隙交给隧道。镜像布局契约（`image-layout.ts`：虚拟根、config/manifest 路径、空目录、`lowered` 包装契约门）与 packer 共享。boot patch 强制部署形态行：关前端静态服务、JSONL 会话日志走明文、preset 根指向镜像内 `config/agent-presets`。 -- **`lib/worker.js`（worker 束）**——装配库加本包的 Node 兼容层，合成一个自含 ES module。模块代理表（`module-proxies.ts`）是唯一平台叉口：`node:*` 内建走 VFS/隧道/浏览器原语，浏览器做不到的走结构化 stub（调用即 console 报错并抛出），外部包整体替换。AsyncLocalStorage 经 pack 时降低注入的 snapshot/restore 面在 `await` 间携带同步栈因果。worker 不带编译器：packer 未降低的镜像在挂载时被拒（[note](../../../.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.zh.md)）。 -- **`src/shell/`（worker 自己的进程层）**——浏览器 worker 无法 fork，所以 `node:child_process` 不是 stub 而是实现：`spawn` 把命令放进它自己的 Web Worker——就是这同一个束，由首帧告诉它「你是 shell 进程」——并以 subprocess 服务消费的 `ChildProcess` 面报告结果。命令不占宿主线程，`SIGKILL` 不管它在干什么都能终止它，而它只能靠消息触达 VFS（由宿主应答这些帧）。语法来自 `@yarnpkg/parsers` 的 `parseShell`；求值器（管道、`&&`/`||`、子 shell、重定向、展开、glob）与命令表由本包自持，而命令表就是这里唯一存在的 `/bin`——表里没有的名字报 `command not found`，`execSync`/`fork` 依然拒绝，因为它们需要真进程。 -- **`lib/client.js`（页面半）**——`connectWorkerHost(worker, { image? })` 完成 pre-Cordis 握手：开局 `init` 帧携带镜像 URL（唯一部署形态输入），boot 载荷送达结构化 index 注入表，`applyIndexInjections` 在壳入口运行前逐行执行。隧道暴露 fetch 形传输、API 客户端与壳启动缝隙用的 `loadBundle`。 +- **`lib/index.js`（装配库）**——`createWorkerHost`/`startWorkerHost` 挂载基础镜像和按序排列的数据 overlays（`storage/`）、安装模块加载器（`module-system/`）与 `process` shim、经镜像自带的 `dsh-app-boot` 启动插件树，并把服务缝隙交给隧道。Overlay 只能替换 `home/` 与 `workspace/` 下的文件，不能替换基础 manifest、配置或模块。镜像布局契约（`image-layout.ts`：虚拟根、config/manifest 路径、空目录、`lowered` 包装契约门）与 packer 共享。boot patch 强制部署形态行：关前端静态服务、JSONL 会话日志走明文、preset 根指向镜像内 `config/agent-presets`。 +- **`lib/worker.js`（worker 束）**——装配库加本包的 Node 兼容层，合成一个自含 ES module。模块代理表（`module-proxies.ts`）是唯一平台叉口：`node:*` 内建走 VFS、隧道和浏览器原语，浏览器做不到的走结构化 stub（调用即在 console 报错并抛出），native/binary 包则替换执行后端。`node:module` 在镜像 package 根之上提供 `createRequire().resolve` 与 `.resolve.paths()`，使未修改的包无需执行目标模块即可发现 manifest。VFS mutation 驱动 `node:fs` 的 callback、polling 和 promise watcher；打开的 descriptor 在 rename、replacement 和 unlink 后仍保留文件身份与访问模式；`readable-stream` 提供文件流以及 Chokidar、readdirp 等未修改镜像包所用的流状态机。AsyncLocalStorage 经 pack 时降低注入的 snapshot/restore 面在 `await` 间携带同步栈因果。worker 不带编译器：packer 未降低的镜像在挂载时被拒（[note](../../../.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.zh.md)）。 +- **`src/shell/`（worker 自己的进程层）**——浏览器 worker 无法 fork，所以 `node:child_process` 不是 stub 而是实现：`spawn` 把命令放进它自己的 Web Worker——就是这同一个束，由首帧告诉它「你是 shell 进程」——并以 subprocess 服务消费的 `ChildProcess` 面报告结果。命令不占宿主线程，`SIGKILL` 不管它在干什么都能终止它，而它只能靠消息触达 VFS（由宿主应答这些帧）。Worker 平台 executable 在不替换 JavaScript 包、也不把具体实现耦合进 `node:child_process` 的情况下保持 Landlock 等 native 包协议；普通命令使用本包的求值器与 coreutils 命令表。语法来自 `@yarnpkg/parsers` 的 `parseShell`，而 `execSync`/`fork` 依然拒绝，因为它们需要真进程。 +- **`lib/client.js`（页面半）**——启动分为相互独立的两段。`chooseWorkerHostSource({ image?, fixtureManifest? })` 可选地拥有 boot barrier 与 fixture manifest：没有 `preview-fixture` 时停在来源选择面板，合法 query 则直接选择；两条路径都返回按序排列的 overlays。`connectWorkerHost(worker, { image?, overlays? })` 仍是公开的基础运行态连接器；调用方跳过选择器时 overlay 列表为空。`apps/web` 调用这两段并提供静态打包的 Worker。开局 `init` 帧携带基础镜像与按序排列的 overlay URL，boot 载荷送达结构化 index 注入表，`applyIndexInjections` 在壳入口运行前逐行执行。隧道暴露 fetch 形传输、API 客户端与壳启动缝隙用的 `loadBundle`。 -验收在 `apps/web/tests/preview-boot.e2e.ts`：静态服务真实构建页面，在 headless Chromium 里驱动 worker 启动。 +验收在 `apps/web/tests/preview-boot.e2e.ts`：静态服务真实构建页面，在 headless Chromium 里驱动 pre-boot 选择面板与 Worker 激活。空白选择验证首次启动；`vfs-example` overlay 提供普通 workspace 文件与明文 persistence 产物，无需模型请求即可验证 Workspace/Session 冷发现、工具呈现、subagent 导航和历史分页。选择面板为 WebFS 保留独立的用户授权来源；该 provider 不读取内置 fixture。 ## 模型体验 @@ -24,9 +24,9 @@ ## Known Limitations and Deferred Work - **worker 组合写明文会话日志**（`compression: 'none'` boot patch）：不带 Zstandard 编解码器，导出日志是 `.jsonl`，不会是 `.jsonl.zstd`。 -- **worker 里的技能目录从不缓存**——`skill-filesystem` 用 `node:fs.watchFile` 监听各个根，而本包拒绝该调用，于是每轮发现都返回不完整观测并重新扫描。发现本身仍然正确，代价是每轮都要重扫。 - **`node:vm`、`node:net`、`node:sqlite`、`node:worker_threads` 是结构化 stub**：每次调用在 console 报告拒绝并抛出。需要真进程或真 realm 隔离的行在此无法运行。 -- **bash 工具只在 `danger-full-access` 下可用**：浏览器没有内核可以约束命令，因此在其余权限档位下 `ctx.sandbox.confine` 会响亮失败、命令根本不会启动。该档位是部署本身的用户面开关，不是 worker 特有的组合差异。 +- **文件 watcher 只能观察已挂载的 VFS**：镜像 seed 不产生事件，VFS 也没有符号链接或外部写入方。`persistent`、`ref()` 和 `unref()` 保留 Node API，但浏览器没有引用计数事件循环，因此这些接口不能控制 dedicated Worker 的生存期。 +- **Worker confinement 是 VFS 边界，不是内核 Landlock**：`read-only` 和 `workspace-write` 运行未经修改的 `@deepseek-ai/node-addon-landlock-run` JavaScript 与 launcher argv，进程层则实现逻辑 `landlock-run` 可执行文件，并在 shell 的每次文件系统请求上执行其授权。`full` 仅覆盖 Worker 命令表和已挂载 VFS，不表示能够执行任意 native 进程，也不表示 Linux 内核隔离。 - **worker 束钉住了 `@yarnpkg/parsers` 的包内路径**——构建解析到该包自己的 `lib/shell.js` 而非包根，因为包根 barrel 还 re-export 了 Syml 解析器，会把 js-yaml 拖进一个从不解析该格式的束（约 175 kB，外加 worker 启动时的模块体求值）。该路径由包 manifest 派生，包内布局一变即构建期失败、不会静默退回 barrel；升级这个依赖时须复核 shell 解析器是否仍在那里。 - **这个 shell 不是 bash**：没有循环、函数、`case`、作业控制或进程替换——语法止步于管道、`&&`/`||`、子 shell、group、重定向与展开。`&` 会就地把命令跑完，`sed` 只接受替换脚本，模式是 JavaScript 正则，命令表只有 coreutils（没有 `git`，没有网络工具）。 - **shell 进程没有同步文件面**：它靠消息读写宿主的 VFS，因为阻塞等待回帧需要 `SharedArrayBuffer`，而那要求 GitHub Pages 给不了的跨源隔离。因此目录遍历类命令每个条目一次往返，并发的两条命令写入可以交错。 diff --git a/packages/experimental/webworker-runtime/package.json b/packages/experimental/webworker-runtime/package.json index c083f13fc1..68643425f8 100644 --- a/packages/experimental/webworker-runtime/package.json +++ b/packages/experimental/webworker-runtime/package.json @@ -35,7 +35,8 @@ "@yarnpkg/parsers": "^3.1.0", "acorn": "^8.17.0", "buffer": "^6.0.3", - "picomatch": "^4.0.4" + "picomatch": "^4.0.4", + "readable-stream": "^4.7.0" }, "peerDependencies": { "@deepseek-ai/cordis": "workspace:^", @@ -49,12 +50,21 @@ "@deepseek-ai/cordis": "workspace:^", "@deepseek-ai/cordis-plugin-loader": "workspace:^", "@deepseek-ai/dsh-api-gateway": "workspace:^", + "@deepseek-ai/dsh-bash-sandbox": "workspace:^", "@deepseek-ai/dsh-client-modules": "workspace:^", "@deepseek-ai/dsh-host-apiproxy": "workspace:^", "@deepseek-ai/dsh-host-webserver": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-sandbox-local": "workspace:^", + "@deepseek-ai/dsh-sandbox-policy": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^", + "@deepseek-ai/dsh-subagent": "workspace:^", "@deepseek-ai/dsh-subprocess-local": "workspace:^", - "@types/picomatch": "^3.0.2" + "@deepseek-ai/node-addon-landlock-run": "workspace:^", + "@types/picomatch": "^3.0.2", + "@types/readable-stream": "^4.0.24", + "chokidar": "^5.0.0" }, "files": [ "lib/index.js", diff --git a/packages/experimental/webworker-runtime/src/client/client.ts b/packages/experimental/webworker-runtime/src/client/client.ts index 7fa05a5355..0300347297 100644 --- a/packages/experimental/webworker-runtime/src/client/client.ts +++ b/packages/experimental/webworker-runtime/src/client/client.ts @@ -156,9 +156,10 @@ export class WorkerTunnel { /** * Open the tunnel: the worker assembles its host from this frame. * @param image - VFS image URL the worker fetches. + * @param overlays - Ordered data overlay URLs applied before boot. */ - init(image: string): void { - this.worker.postMessage({ t: 'init', image }) + init(image: string, overlays: readonly string[] = []): void { + this.worker.postMessage({ t: 'init', image, overlays }) } /** Fetch-shaped entry: one request frame, one Response (streamed when the worker streams). */ diff --git a/packages/experimental/webworker-runtime/src/client/index.ts b/packages/experimental/webworker-runtime/src/client/index.ts index c84f637896..8494e1c06a 100644 --- a/packages/experimental/webworker-runtime/src/client/index.ts +++ b/packages/experimental/webworker-runtime/src/client/index.ts @@ -9,14 +9,20 @@ * @module @deepseek-ai/dsh-experimental-webworker-runtime/client */ import { IMAGE_FILE_NAME } from '../image-layout.ts' +import { PREVIEW_FIXTURE_MANIFEST_FILE } from '../fixture-manifest.ts' import { WorkerApiClient } from './api-client.ts' import { WorkerTunnel, type TunnelFetch } from './client.ts' import { applyIndexInjections } from './apply-injections.ts' +import { choosePreviewSource } from './source-chooser.ts' export { WorkerApiClient } from './api-client.ts' export { WorkerTunnel, type TunnelFetch } from './client.ts' export { applyIndexInjections } from './apply-injections.ts' export { IMAGE_FILE_NAME } from '../image-layout.ts' +export { + parsePreviewFixtureManifest, PREVIEW_FIXTURE_MANIFEST_FILE, PREVIEW_FIXTURE_MANIFEST_VERSION, + type PreviewFixtureManifest, type PreviewFixtureManifestEntry, +} from '../fixture-manifest.ts' /** Transport global the connection plugin reads instead of building an HTTP carrier. */ interface ClientTransportGlobal { @@ -35,9 +41,25 @@ export interface WorkerHostConnectOptions { /** * VFS image URL, the one deployment-shaped input. Defaults to * {@link IMAGE_FILE_NAME} beside the page; a deployment that packs the - * image elsewhere passes its own URL. + * image elsewhere passes its own URL. Data overlays are independent. */ readonly image?: string | URL + /** Ordered data overlay URLs, resolved against the page like the base image. */ + readonly overlays?: readonly (string | URL)[] +} + +/** Inputs for the optional pre-boot filesystem-source chooser. */ +export interface WorkerHostSourceOptions { + /** Base VFS image URL; defaults to {@link IMAGE_FILE_NAME} beside the page. */ + readonly image?: string | URL + /** Fixture catalog URL; defaults to {@link PREVIEW_FIXTURE_MANIFEST_FILE} beside the image. */ + readonly fixtureManifest?: string | URL +} + +/** Filesystem inputs selected before {@link connectWorkerHost}. */ +export interface WorkerHostSource { + /** Ordered data overlays to pass through unchanged to the Host connection. */ + readonly overlays: readonly URL[] } /** A page connected to a worker-hosted harness, ready to run a shell entry. */ @@ -53,12 +75,51 @@ interface BootReadyGlobal { __DSH_BOOT_READY__?: PromiseWithResolvers } +function bootReadyGate(): PromiseWithResolvers { + return (globalThis as BootReadyGlobal).__DSH_BOOT_READY__ ??= Promise.withResolvers() +} + +/** + * Install the page boot barrier before an asynchronous source chooser waits + * for user input. The later {@link connectWorkerHost} call settles the same + * barrier. + */ +function holdWorkerHostBoot(): void { + const ready = bootReadyGate() + // A chooser may remain open indefinitely; if a later connection fails before + // the stock entry subscribes, retain the rejection without browser noise. + void ready.promise.catch(() => {}) +} + +/** + * Run the optional pre-boot source-selection stage. Calling this stage holds + * the stock shell until the caller passes its result to {@link connectWorkerHost}; + * callers that need no chooser call `connectWorkerHost` directly and receive + * the base image with an empty overlay list. + * @param options - Base image and optional fixture-catalog locations. + * @returns The ordered overlays selected by the user. + */ +export async function chooseWorkerHostSource( + options: WorkerHostSourceOptions = {}, +): Promise { + holdWorkerHostBoot() + const image = new URL(options.image ?? IMAGE_FILE_NAME, document.baseURI) + const manifest = new URL(options.fixtureManifest ?? PREVIEW_FIXTURE_MANIFEST_FILE, image) + try { + const overlays = await choosePreviewSource(manifest) + return { overlays } + } catch (reason) { + bootReadyGate().reject(reason) + throw reason + } +} + /** * Connect a spawned host worker and complete the pre-Cordis handshake. * * The caller constructs the Worker so its bundler resolves the bundle URL - * statically; the opening `init` frame then carries the image location, the - * only input the worker takes from outside. + * statically; the opening `init` frame then carries the base image and ordered + * overlay locations. * * Order is fixed by the web boot protocol: the transport global must exist * before any bundle executes; the injection table then reproduces the served @@ -70,17 +131,20 @@ interface BootReadyGlobal { * row has taken effect, and surfaces a failed handshake instead of * proceeding on missing globals. * @param worker - The host worker. - * @param options - Image location override. + * @param options - Base-image and overlay location overrides. * @returns The connection; hand `loadBundle` to the shell entry's boot seam. */ export async function connectWorkerHost(worker: Worker, options?: WorkerHostConnectOptions): Promise { - const ready = (globalThis as BootReadyGlobal).__DSH_BOOT_READY__ ??= Promise.withResolvers() + const ready = bootReadyGate() // The handshake may fail before any entry awaits the promise; this no-op // subscription keeps that from surfacing as an unhandled rejection. void ready.promise.catch(() => {}) try { const tunnel = new WorkerTunnel(worker) - tunnel.init(new URL(options?.image ?? IMAGE_FILE_NAME, document.baseURI).href) + tunnel.init( + new URL(options?.image ?? IMAGE_FILE_NAME, document.baseURI).href, + (options?.overlays ?? []).map(overlay => new URL(overlay, document.baseURI).href), + ) const payload = await tunnel.bootPayload() ;(globalThis as ClientTransportGlobal).__DSH_TRANSPORT__ = { createApiClient: () => new WorkerApiClient(tunnel), diff --git a/packages/experimental/webworker-runtime/src/client/source-chooser.ts b/packages/experimental/webworker-runtime/src/client/source-chooser.ts new file mode 100644 index 0000000000..48d454eb94 --- /dev/null +++ b/packages/experimental/webworker-runtime/src/client/source-chooser.ts @@ -0,0 +1,264 @@ +/** Pre-boot filesystem-source chooser for static WebWorker previews. */ + +import { + parsePreviewFixtureManifest, type PreviewFixtureManifestEntry, +} from '../fixture-manifest.ts' + +const EMPTY_SOURCE = 'none' +const WEBFS_SOURCE = 'webfs' +const PREVIEW_FIXTURE_QUERY = 'preview-fixture' + +interface PreviewSourceChoice { + readonly id: string + readonly label: string + readonly description: string + readonly overlays: readonly URL[] + readonly disabled?: boolean +} + +const CHOOSER_STYLE = ` + [data-preview-source-chooser] { + position: fixed; + inset: 0; + z-index: 1200; + display: grid; + place-items: center; + overflow: auto; + padding: 24px; + box-sizing: border-box; + color: #0f1115; + background: #fff; + font-size: 14px; + line-height: 22px; + } + [data-preview-source-card] { + width: min(600px, 100%); + max-height: calc(100dvh - 48px); + box-sizing: border-box; + padding: 28px; + overflow-y: auto; + border: 1px solid transparent; + border-radius: 24px; + background: #fff; + box-shadow: 0 0 1px rgb(0 0 0 / 20%), 0 12px 32px rgb(0 0 0 / 8%); + } + [data-preview-source-card] h1 { + margin: 0; + font-size: 20px; + line-height: 28px; + font-weight: 500; + } + [data-preview-source-card] > p { + margin: 8px 0 0; + color: #61666b; + } + [data-preview-source-card] fieldset { + display: flex; + flex-direction: column; + gap: 1px; + margin: 24px 0 0; + padding: 0; + border: 0; + } + [data-preview-source-card] legend { + margin: 0 0 8px; + padding: 0 4px; + color: #61666b; + font-size: 13px; + line-height: 20px; + font-weight: 500; + } + [data-preview-source-option] { + position: relative; + display: flex; + align-items: flex-start; + gap: 8px; + min-height: 56px; + padding: 8px 12px 8px 8px; + box-sizing: border-box; + border: 1px solid transparent; + border-radius: 12px; + background: transparent; + cursor: pointer; + transition: background-color 120ms ease, border-color 120ms ease; + } + [data-preview-source-option]:hover:not(:has(input:disabled)), + [data-preview-source-option]:has(input:checked) { + background: rgb(38 49 72 / 6%); + } + [data-preview-source-option]:has(input:checked) { + border-color: rgb(0 0 0 / 10%); + } + [data-preview-source-option]:has(input:disabled) { + cursor: default; + opacity: 0.4; + } + [data-preview-source-option] input { + flex: none; + width: 16px; + height: 16px; + margin: 4px 0 0; + accent-color: #0f1115; + } + [data-preview-source-option] > span { flex: 1; min-width: 0; } + [data-preview-source-option] strong { + display: block; + font-size: 14px; + line-height: 24px; + font-weight: 500; + } + [data-preview-source-option] strong + span { + display: block; + color: #81858c; + font-size: 14px; + line-height: 24px; + } + [data-preview-source-submit] { + display: block; + min-width: 120px; + height: 36px; + margin: 24px 0 0 auto; + padding: 0 14px; + border: 0; + border-radius: 18px; + color: #fff; + background: #0f1115; + font-size: 14px; + line-height: 22px; + cursor: pointer; + transition: background-color 120ms ease; + } + [data-preview-source-submit]:hover:not(:disabled) { + background: #43454a; + } + [data-preview-source-submit]:focus-visible { + outline: 2px solid rgb(0 0 0 / 16%); + outline-offset: 2px; + } + [data-preview-source-submit]:disabled { cursor: not-allowed; opacity: 0.5; } + @media (prefers-color-scheme: dark) { + [data-preview-source-chooser] { + color: #f9fafb; + background: #151517; + } + [data-preview-source-card] { border-color: rgb(255 255 255 / 6%); background: #2c2c2e; } + [data-preview-source-card] > p, [data-preview-source-card] legend { color: #cfd3d6; } + [data-preview-source-option] strong + span { color: #adb2b8; } + [data-preview-source-option]:hover:not(:has(input:disabled)), + [data-preview-source-option]:has(input:checked) { background: rgb(255 255 255 / 8%); } + [data-preview-source-option]:has(input:checked) { border-color: rgb(255 255 255 / 12%); } + [data-preview-source-option] input { accent-color: #f9fafb; } + [data-preview-source-submit] { color: #0f1115; background: #f9fafb; } + [data-preview-source-submit]:hover:not(:disabled) { background: #ebeef2; } + [data-preview-source-submit]:focus-visible { outline-color: rgb(255 255 255 / 20%); } + } + @media (max-width: 560px) { + [data-preview-source-card] { padding: 24px; } + [data-preview-source-submit] { width: 100%; } + } + @media (prefers-reduced-motion: reduce) { + [data-preview-source-option], [data-preview-source-submit] { transition: none; } + } +` + +const ENTITIES: Readonly> = { + '&': '&', '<': '<', '>': '>', '"': '"', "'": ''', +} + +function escapeMarkup(value: string): string { + return value.replace(/[&<>"']/g, character => ENTITIES[character] ?? character) +} + +function optionMarkup(choice: PreviewSourceChoice, selected: string): string { + return `` +} + +function fixtureChoices(entries: readonly PreviewFixtureManifestEntry[], manifestUrl: URL): PreviewSourceChoice[] { + return entries.map(entry => ({ + id: entry.id, + label: entry.label, + description: entry.description, + overlays: entry.overlays.map(overlay => new URL(overlay, manifestUrl)), + })) +} + +/** + * Render the source chooser and wait for an enabled selection. + * @param manifestUrl - Built-in fixture catalog URL. + * @returns Ordered overlay URLs selected for the Worker mount. + */ +export async function choosePreviewSource(manifestUrl: URL): Promise { + const requested = new URL(location.href).searchParams.get(PREVIEW_FIXTURE_QUERY) + if (requested === EMPTY_SOURCE) return [] + + const response = await fetch(manifestUrl) + if (!response.ok) { + throw new Error(`preview source chooser: fixture manifest returned ${String(response.status)}`) + } + const manifest = parsePreviewFixtureManifest(await response.json()) + const choices: PreviewSourceChoice[] = [ + { + id: EMPTY_SOURCE, + label: 'Empty environment', + description: 'Load only the base runtime to verify first launch and workspace creation.', + overlays: [], + }, + ...fixtureChoices(manifest.fixtures, manifestUrl), + { + id: WEBFS_SOURCE, + label: 'WebFS directory', + description: 'Requires directory access and will be available after the WebFS provider lands.', + overlays: [], + disabled: true, + }, + ] + if (requested !== null) { + const requestedChoice = choices.find(choice => choice.id === requested && choice.disabled !== true) + if (requestedChoice === undefined) { + throw new Error(`preview source chooser: unknown or interactive source "${requested}"`) + } + return requestedChoice.overlays + } + + const root = document.getElementById('root') + if (root === null) throw new Error('preview source chooser: missing #root') + const selected = manifest.defaultFixture ?? EMPTY_SOURCE + const style = document.createElement('style') + style.dataset.previewSourceStyle = '' + style.textContent = CHOOSER_STYLE + document.head.append(style) + + const chooser = document.createElement('main') + chooser.dataset.previewSourceChooser = '' + chooser.innerHTML = `
+

Choose Preview data

+

Data mounts before the Worker and application start. Refresh to choose again.

+
+ Filesystem source + ${choices.map(choice => optionMarkup(choice, selected)).join('')} +
+ +
` + root.prepend(chooser) + const form = chooser.querySelector('[data-preview-source-card]') + if (form === null) throw new Error('preview source chooser: form was not rendered') + const sourceId = await new Promise((resolve, reject) => { + form.addEventListener('submit', (event) => { + event.preventDefault() + const value = new FormData(form).get('preview-source') + if (typeof value === 'string') resolve(value) + else reject(new Error('preview source chooser: no source selected')) + }, { once: true }) + }) + const choice = choices.find(candidate => candidate.id === sourceId && candidate.disabled !== true) + if (choice === undefined) throw new Error(`preview source chooser: unavailable source "${sourceId}"`) + chooser.remove() + style.remove() + return choice.overlays +} diff --git a/packages/experimental/webworker-runtime/src/fixture-manifest.ts b/packages/experimental/webworker-runtime/src/fixture-manifest.ts new file mode 100644 index 0000000000..7d2ecf2556 --- /dev/null +++ b/packages/experimental/webworker-runtime/src/fixture-manifest.ts @@ -0,0 +1,68 @@ +/** Browser-readable catalog of built-in Preview filesystem overlays. */ + +/** Manifest format version emitted beside the base VFS image. */ +export const PREVIEW_FIXTURE_MANIFEST_VERSION = 1 + +/** Leaf name resolved beside the base image. */ +export const PREVIEW_FIXTURE_MANIFEST_FILE = 'fixtures.json' + +/** One selectable built-in fixture and its ordered overlay archives. */ +export interface PreviewFixtureManifestEntry { + readonly id: string + readonly label: string + readonly description: string + readonly overlays: readonly string[] +} + +/** Complete built-in fixture catalog consumed before Worker startup. */ +export interface PreviewFixtureManifest { + readonly version: number + /** Required default fixture id, or null when the chooser should default to an empty overlay. */ + readonly defaultFixture: string | null + readonly fixtures: readonly PreviewFixtureManifestEntry[] +} + +function recordOf(value: unknown): Record | undefined { + return typeof value === 'object' && value !== null && !Array.isArray(value) + ? value as Record + : undefined +} + +/** + * Validate the static fixture catalog before it controls Worker fetches. + * @param value - Parsed JSON response. + * @returns A detached manifest with unique ids and non-empty overlay lists. + */ +export function parsePreviewFixtureManifest(value: unknown): PreviewFixtureManifest { + const record = recordOf(value) + if (record?.version !== PREVIEW_FIXTURE_MANIFEST_VERSION || !Array.isArray(record.fixtures)) { + throw new Error(`preview fixture manifest must use version ${String(PREVIEW_FIXTURE_MANIFEST_VERSION)}`) + } + const fixtures: PreviewFixtureManifestEntry[] = [] + const ids = new Set() + for (const value of record.fixtures) { + const fixture = recordOf(value) + const id = fixture?.id + const label = fixture?.label + const description = fixture?.description + const overlays = fixture?.overlays + const overlayUrls = Array.isArray(overlays) + ? overlays.filter((overlay): overlay is string => typeof overlay === 'string' && overlay.length > 0) + : [] + if (typeof id !== 'string' || !/^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$/.test(id) + || id === 'none' || id === 'webfs' + || typeof label !== 'string' || label.length === 0 + || typeof description !== 'string' || description.length === 0 + || !Array.isArray(overlays) || overlays.length === 0 || overlayUrls.length !== overlays.length) { + throw new Error('preview fixture manifest contains an invalid fixture entry') + } + if (ids.has(id)) throw new Error(`preview fixture manifest repeats id "${id}"`) + ids.add(id) + fixtures.push({ id, label, description, overlays: overlayUrls }) + } + const defaultFixture = record.defaultFixture + if (defaultFixture !== null && (typeof defaultFixture !== 'string' || !ids.has(defaultFixture))) { + throw new Error('preview fixture manifest defaultFixture does not name a fixture') + } + return { version: PREVIEW_FIXTURE_MANIFEST_VERSION, defaultFixture, fixtures } +} diff --git a/packages/experimental/webworker-runtime/src/image-layout.ts b/packages/experimental/webworker-runtime/src/image-layout.ts index 83693ea292..3e16f5a529 100644 --- a/packages/experimental/webworker-runtime/src/image-layout.ts +++ b/packages/experimental/webworker-runtime/src/image-layout.ts @@ -9,9 +9,9 @@ export const DEFAULT_ROOT = '/dsh' /** - * Leaf name of the packed image: one gzip member holding the ustar archive. The - * app build writes it beside the page and the page's boot fetches it from there, - * so the extension is part of what a deployment serves. + * Leaf name of the packed base image: one gzip member holding the ustar archive. + * The app build writes it beside the page and the page's boot fetches it from + * there, so the extension is part of what a deployment serves. */ export const IMAGE_FILE_NAME = 'vfs-image.tar.gz' @@ -27,6 +27,12 @@ export const IMAGE_HOME_DIRECTORY = 'home' /** Working directories the host tree expects to exist, empty. */ export const IMAGE_EMPTY_DIRECTORIES: readonly string[] = ['home/', 'workspace/', 'tmp/'] +/** + * Top-level directories an overlay archive may populate. Runtime code, + * configuration, and the lowering manifest remain owned by the base image. + */ +export const IMAGE_OVERLAY_DIRECTORIES: readonly string[] = ['home', 'workspace'] + /** * Identity of the lowered code shape, recorded in the image manifest by the * packer and required by the worker host: an image lowered by an older transform diff --git a/packages/experimental/webworker-runtime/src/index.ts b/packages/experimental/webworker-runtime/src/index.ts index cf39694ca0..e24faa1eb7 100644 --- a/packages/experimental/webworker-runtime/src/index.ts +++ b/packages/experimental/webworker-runtime/src/index.ts @@ -34,9 +34,13 @@ export { } from './worker-host.ts' export { DEFAULT_ROOT, IMAGE_CONFIG_PATH, IMAGE_EMPTY_DIRECTORIES, IMAGE_FILE_NAME, IMAGE_HOME_DIRECTORY, - IMAGE_MANIFEST_PATH, LOWERING_VERSION, WRAPPER_PARAMS, + IMAGE_MANIFEST_PATH, IMAGE_OVERLAY_DIRECTORIES, LOWERING_VERSION, WRAPPER_PARAMS, } from './image-layout.ts' -export { loadVfsImage, MemoryVfs } from './storage/memory.ts' +export { + parsePreviewFixtureManifest, PREVIEW_FIXTURE_MANIFEST_FILE, PREVIEW_FIXTURE_MANIFEST_VERSION, + type PreviewFixtureManifest, type PreviewFixtureManifestEntry, +} from './fixture-manifest.ts' +export { loadVfsImage, loadVfsOverlay, MemoryVfs } from './storage/memory.ts' export { inflateImage, inflateImageStream } from './storage/image-gzip.ts' export { packTar, parseTar, type TarEntry } from './storage/tar.ts' export { requireActiveVfs, setActiveVfs } from './storage/active.ts' diff --git a/packages/experimental/webworker-runtime/src/module-proxies.ts b/packages/experimental/webworker-runtime/src/module-proxies.ts index e30128d293..4e95da027c 100644 --- a/packages/experimental/webworker-runtime/src/module-proxies.ts +++ b/packages/experimental/webworker-runtime/src/module-proxies.ts @@ -56,7 +56,7 @@ export const MODULE_PROXIES: Record = { 'node:child_process': './node/builtin_modules/implemented/child_process.ts', // Structural mocks: every symbol exists, every call throws. 'node:net': './node/builtin_modules/mock/net.ts', - 'node:stream': './node/builtin_modules/mock/stream.ts', + 'node:stream': './node/builtin_modules/implemented/stream.ts', 'node:vm': './node/builtin_modules/mock/vm.ts', 'node:worker_threads': './node/builtin_modules/mock/worker_threads.ts', 'node:sqlite': './node/builtin_modules/mock/sqlite.ts', @@ -66,10 +66,8 @@ export const MODULE_PROXIES: Record = { 'node-pty': './node/external_packages/node-pty.ts', '@vscode/ripgrep': './node/external_packages/ripgrep.ts', '@earendil-works/pi-ai': './node/external_packages/pi-ai.ts', - '@deepseek-ai/node-addon-landlock-run': './node/external_packages/node-addon-landlock-run.ts', // Constructible fakes whose methods are never reached. 'ws': './node/external_packages/ws.ts', - 'chokidar': './node/external_packages/chokidar.ts', } diff --git a/packages/experimental/webworker-runtime/src/module-system/module-loader.ts b/packages/experimental/webworker-runtime/src/module-system/module-loader.ts index b1d0f70f4a..88cdc39ece 100644 --- a/packages/experimental/webworker-runtime/src/module-system/module-loader.ts +++ b/packages/experimental/webworker-runtime/src/module-system/module-loader.ts @@ -1,8 +1,8 @@ /** * CommonJS module loader over the worker VFS. It fills the `loader.internal` * seam Cordis uses for every entry import, and backs the `node:module` - * `createRequire` proxy that `typert-loader` and `client-modules` resolve - * package metadata through. + * `createRequire` proxy that `typert-loader`, `client-modules`, and the plugin + * package inventory resolve package metadata through. * * Resolution is a narrowed Node `require` algorithm: `exports` walk with a * fixed condition order, extension probing, and one cache keyed by resolved @@ -49,10 +49,26 @@ interface ModuleRecord { readonly module: { exports: unknown } } +/** Resolution helpers carried by a Worker-backed CommonJS require. */ +export interface WorkerRequireResolve { + /** + * Resolve one specifier without evaluating its module. + * @param specifier - Module request relative to the require base. + * @returns Static or VFS-backed module identity. + */ + (specifier: string): string + /** + * Return the directories this loader's Node-style package discovery searches. + * @param specifier - Module request whose lookup roots are requested. + * @returns Search roots, or null for a Worker-provided module. + */ + paths(specifier: string): string[] | null +} + /** The `require` function shape the roster consumes through `createRequire`. */ export interface WorkerRequire { (specifier: string): unknown - resolve(specifier: string): string + readonly resolve: WorkerRequireResolve } /** Construction inputs for {@link WorkerModuleLoader}. */ @@ -226,6 +242,16 @@ export class WorkerModuleLoader { return this.fail(`cannot resolve "${specifier}": no file at ${candidates.join(', ')}`) } + /** @returns The Worker-provided implementation of a static specifier. */ + private staticModule(specifier: string): StaticModuleFactory | undefined { + const exact = this.staticModules.get(specifier) + if (exact !== undefined) return exact + for (const [prefix, factory] of this.staticPrefixes) { + if (specifier.startsWith(prefix)) return factory + } + return this.staticModules.get(`node:${specifier}`) + } + /** * Resolve a specifier the way the module that requested it would. * @param specifier - Bare name, relative path, absolute path, or file URL. @@ -233,11 +259,8 @@ export class WorkerModuleLoader { * @returns Static module or the resolved VFS path. */ resolve(specifier: string, fromDirectory: string): Resolution { - const exact = this.staticModules.get(specifier) - if (exact !== undefined) return { kind: 'static', specifier, factory: exact } - for (const [prefix, factory] of this.staticPrefixes) { - if (specifier.startsWith(prefix)) return { kind: 'static', specifier, factory } - } + const staticModule = this.staticModule(specifier) + if (staticModule !== undefined) return { kind: 'static', specifier, factory: staticModule } if (specifier.startsWith('cordis:') || specifier.startsWith('node:')) { return this.fail(`no static module is registered for "${specifier}"`) } @@ -250,9 +273,6 @@ export class WorkerModuleLoader { if (isAbsolute(specifier)) { return { kind: 'file', path: this.probe(specifier, specifier) } } - // Node resolves `fs` and `node:fs` to the same builtin; the proxy table may register either. - const prefixed = this.staticModules.get(`node:${specifier}`) - if (prefixed !== undefined) return { kind: 'static', specifier, factory: prefixed } const segments = specifier.split('/') const packageName = specifier.startsWith('@') ? segments.slice(0, 2).join('/') : segments[0] ?? specifier const rest = specifier.slice(packageName.length).replace(/^\//, '') @@ -356,15 +376,20 @@ export class WorkerModuleLoader { * @returns Callable require with `resolve`. */ requireFrom(fromDirectory: string): WorkerRequire { - const require = ((specifier: string): unknown => this.load(this.resolve(specifier, fromDirectory))) as WorkerRequire - require.resolve = (specifier: string): string => { + const require = (specifier: string): unknown => this.load(this.resolve(specifier, fromDirectory)) + const resolve = ((specifier: string): string => { const resolution = this.resolve(specifier, fromDirectory) if (resolution.kind === 'static') { return this.fail(`"${specifier}" is a worker-provided module and has no VFS path`) } return resolution.path + }) as WorkerRequireResolve + resolve.paths = (specifier: string): string[] | null => { + if (this.staticModule(specifier) !== undefined || specifier.startsWith('node:')) return null + if (specifier.startsWith('.')) return [resolvePath(fromDirectory, '.')] + return [join(this.root, 'node_modules')] } - return require + return Object.assign(require, { resolve }) } /** diff --git a/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/abort-error.ts b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/abort-error.ts new file mode 100644 index 0000000000..a7f188bdc1 --- /dev/null +++ b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/abort-error.ts @@ -0,0 +1,14 @@ +/** Build the Node-style cancellation error shared by abortable builtin APIs. */ + +/** + * Create an `AbortError` carrying Node's stable error code. + * @param reason - Optional AbortSignal reason exposed as the error cause. + * @returns A Node-compatible abort error. + */ +export function abortError(reason?: unknown): Error & { code: string; cause?: unknown } { + const error = new Error('The operation was aborted') as Error & { code: string; cause?: unknown } + error.name = 'AbortError' + error.code = 'ABORT_ERR' + if (reason !== undefined) error.cause = reason + return error +} diff --git a/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/child_process.ts b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/child_process.ts index 2c863ce6bb..e70d1323d1 100644 --- a/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/child_process.ts +++ b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/child_process.ts @@ -5,9 +5,10 @@ * `spawn` starts the argv as a shell process (`src/shell/process/`) — its own * Web Worker, off this thread — and reports it through the `ChildProcess` * surface the subprocess service consumes: pipes, `exit`/`close`, pid, and - * signals, with `SIGKILL` terminating the worker for real. The command table - * is the only `/bin` that exists, so a name it does not hold fails with - * `ENOENT`, exactly as a missing binary does on a real host. + * signals, with `SIGKILL` terminating the worker for real. Worker-owned + * executable wrappers resolve before the shell's command table; anything in + * neither set fails with `ENOENT`, exactly as a missing binary does on a real + * host. * * What stays impossible is what needs a real process: synchronous execution * (`execSync`, and `spawnSync` for a known program) and `fork`. @@ -19,7 +20,11 @@ import { EventEmitter } from './events.ts' import { notImplementedFail } from '../../notImplementedFail.ts' import { registerProcess, releaseProcess, signalProcess } from '../../process-table.ts' import { startProcess } from '../../../shell/process/host.ts' +import { hostFileSystem } from '../../../shell/fs-access.ts' +import { virtualExecutable } from '../../../shell/process/virtual-executables.ts' +import type { VirtualExecutableExit } from '../../../shell/process/virtual-executables.ts' import { standardPrograms } from '../../../shell/programs/index.ts' +import type { ShellFileSystem } from '../../../shell/types.ts' import { DSH_ROOT } from '../../../storage/paths.ts' const MODULE = 'node:child_process' @@ -219,9 +224,6 @@ export function spawn( const entry = registerProcess() const child = new WorkerChildProcess(entry.pid, stdio) - const script = shellScriptOf(argv) - const known = script !== undefined || standardPrograms().has(program) - const emit = (stream: 'stdout' | 'stderr', text: string): void => { if (text === '') return const pipe = stream === 'stdout' ? child.stdout : child.stderr @@ -236,7 +238,10 @@ export function spawn( } } + let settled = false const settle = (exitCode: number): void => { + if (settled) return + settled = true releaseProcess(entry.pid) // A signalled command reports no exit code, which is what makes the // subprocess service classify it as killed rather than finished. @@ -248,31 +253,68 @@ export function spawn( child.emit('exit', child.exitCode, signal) child.emit('close', child.exitCode, signal) } + const failSpawn = (error: Error): void => { + if (settled) return + settled = true + releaseProcess(entry.pid) + child.emit('error', error) + } // The command starts on a microtask, so a caller that attaches listeners and // writes standard input right after `spawn()` — the subprocess service does // exactly that — is never racing the first output. queueMicrotask(() => { - if (!known) { - releaseProcess(entry.pid) - child.emit('error', spawnEnoent(program)) - return - } - entry.process = startProcess({ - script, - argv, - cwd: options.cwd ?? DSH_ROOT, - env: environmentOf(options.env), - stdin: child.stdin?.contents() ?? '', - onOutput: emit, - onExit: settle, + void (async () => { + const cwd = options.cwd ?? DSH_ROOT + let commandArgv: readonly string[] = argv + let filesystem: ShellFileSystem | undefined + let missingExecutable: VirtualExecutableExit | undefined + const executable = virtualExecutable(program) + if (executable !== undefined) { + const prepared = await executable.prepare(args, { cwd, filesystem: hostFileSystem() }) + if (prepared.kind === 'exit') { + emit('stdout', prepared.stdout) + emit('stderr', prepared.stderr) + settle(prepared.exitCode) + return + } + commandArgv = prepared.argv + filesystem = prepared.filesystem + missingExecutable = prepared.missingExecutable + } + + const command = commandArgv[0] as string + const script = shellScriptOf(commandArgv) + const known = script !== undefined || standardPrograms().has(command) + if (!known) { + if (missingExecutable !== undefined) { + emit('stdout', missingExecutable.stdout) + emit('stderr', missingExecutable.stderr) + settle(missingExecutable.exitCode) + } else { + failSpawn(spawnEnoent(program)) + } + return + } + entry.process = startProcess({ + script, + argv: commandArgv, + cwd, + env: environmentOf(options.env), + stdin: child.stdin?.contents() ?? '', + onOutput: emit, + onExit: settle, + ...filesystem === undefined ? {} : { fs: filesystem }, + }) + // A signal that arrived while the process was still starting has to reach + // it now; the table recorded it but had nothing to deliver it to. + if (entry.signal !== undefined) { + if (entry.signal === 'SIGKILL') entry.process.destroy() + else entry.process.interrupt() + } + })().catch((error: unknown) => { + failSpawn(error instanceof Error ? error : new Error(String(error))) }) - // A signal that arrived while the process was still starting has to reach - // it now; the table recorded it but had nothing to deliver it to. - if (entry.signal !== undefined) { - if (entry.signal === 'SIGKILL') entry.process.destroy() - else entry.process.interrupt() - } }) return child @@ -298,10 +340,22 @@ export interface WorkerSpawnSyncResult { * answers in the same shape: absent programs report `ENOENT`, and a program * this shell *does* have reports that only the asynchronous path can run it. * @param program - the program name. + * @param args - arguments passed to the virtual launcher probe. * @returns the Node-shaped synchronous result carrying the failure. */ -export function spawnSync(program: string): WorkerSpawnSyncResult { +export function spawnSync(program: string, args: readonly string[] = []): WorkerSpawnSyncResult { const empty = Buffer.alloc(0) + const executable = virtualExecutable(program) + if (executable !== undefined) { + const result = executable.runSync(args) + if (result.kind === 'asynchronous') { + const error = new Error(`${MODULE}.spawnSync cannot run ${program} in the worker host: commands run asynchronously`) + return { pid: -1, status: null, signal: null, stdout: empty, stderr: empty, output: [null, empty, empty], error } + } + const stdout = Buffer.from(result.stdout) + const stderr = Buffer.from(result.stderr) + return { pid: -1, status: result.exitCode, signal: null, stdout, stderr, output: [null, stdout, stderr] } + } const error = standardPrograms().has(program) ? new Error(`${MODULE}.spawnSync cannot run ${program} in the worker host: commands run asynchronously`) : spawnEnoent(program) diff --git a/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/fs-watch.ts b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/fs-watch.ts new file mode 100644 index 0000000000..a570e54bc2 --- /dev/null +++ b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/fs-watch.ts @@ -0,0 +1,428 @@ +/** Node filesystem watching over the active in-memory VFS. */ +import { Buffer } from 'buffer' +import { EventEmitter } from './events.ts' +import { captureAsyncContext, runWithAsyncContext } from './async_hooks.ts' +import { basename, relative, resolve, sep } from './path.ts' +import { requireActiveVfs } from '../../../storage/active.ts' +import type { VfsBigIntStats, VfsMutation, VfsStats } from '../../../storage/types.ts' +import { abortError } from './abort-error.ts' + +type PathArg = string | URL | Uint8Array +type WatchListener = (eventType: 'rename' | 'change', filename: string | Buffer | null) => void +type WatchStats = VfsStats | VfsBigIntStats +type StatListener = (current: WatchStats, previous: WatchStats) => void + +/** Options shared by the callback and promise watch faces. */ +export interface WatchOptions { + persistent?: boolean + recursive?: boolean + encoding?: BufferEncoding | 'buffer' + signal?: AbortSignal +} +/** Poll-style watch options. */ +export interface WatchFileOptions { + persistent?: boolean + interval?: number + bigint?: boolean +} + +const asPath = (path: PathArg): string => { + if (typeof path === 'string') return resolve(path) + if (path instanceof URL) return resolve(decodeURIComponent(path.pathname)) + return resolve(new TextDecoder().decode(path)) +} + +const missingStats = (bigint: boolean): WatchStats => ({ + size: bigint ? 0n : 0, + ino: bigint ? 0n : 0, + mtimeMs: bigint ? 0n : 0, + ctimeMs: bigint ? 0n : 0, + atimeMs: bigint ? 0n : 0, + birthtimeMs: bigint ? 0n : 0, + mtime: new Date(0), + mode: bigint ? 0n : 0, + ...bigint ? { + dev: 0n, + nlink: 0n, + mtimeNs: 0n, + ctimeNs: 0n, + atimeNs: 0n, + birthtimeNs: 0n, + ctime: new Date(0), + atime: new Date(0), + birthtime: new Date(0), + } : {}, + isFile: () => false, + isDirectory: () => false, + isSymbolicLink: () => false, + isFIFO: () => false, + isSocket: () => false, + isBlockDevice: () => false, + isCharacterDevice: () => false, +}) as WatchStats + +const statOrMissing = (path: string, bigint: boolean): WatchStats => { + try { + return requireActiveVfs().statSync(path, { bigint }) + } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') return missingStats(bigint) + throw error + } +} + +const statsChanged = (left: WatchStats, right: WatchStats): boolean => + left.size !== right.size + || left.mtimeMs !== right.mtimeMs + || left.mode !== right.mode + || left.ino !== right.ino + || left.isFile() !== right.isFile() + || left.isDirectory() !== right.isDirectory() + +const contains = (parent: string, child: string): boolean => + parent === '/' || child === parent || child.startsWith(`${parent}${sep}`) + +const overlaps = (left: string, right: string): boolean => contains(left, right) || contains(right, left) + +/** `fs.FSWatcher` over VFS mutations. */ +export class FSWatcher extends EventEmitter { + private readonly disposeMutation: () => void + private readonly signal: AbortSignal | undefined + private readonly onAbort: (() => void) | undefined + private closed = false + private referenced: boolean + + constructor( + private readonly target: string, + private readonly directory: boolean, + private readonly options: WatchOptions, + listener?: WatchListener, + ) { + super() + this.referenced = options.persistent ?? true + const context = captureAsyncContext() + if (listener !== undefined) this.on('change', listener as (...args: unknown[]) => void) + this.disposeMutation = requireActiveVfs().subscribe((mutation) => { + if (!this.matches(mutation)) return + const eventType = mutation.kind === 'write' && !mutation.entryChanged || mutation.kind === 'chmod' + ? 'change' + : 'rename' + const filename = this.filename(mutation.path) + queueMicrotask(() => { + if (this.closed) return + runWithAsyncContext(context, () => { this.emit('change', eventType, filename) }) + }) + }) + this.signal = options.signal + this.onAbort = options.signal === undefined ? undefined : () => { this.close() } + if (options.signal?.aborted === true) { + this.close() + return + } + options.signal?.addEventListener('abort', this.onAbort as () => void, { once: true }) + } + + private matches(mutation: VfsMutation): boolean { + if (mutation.path === this.target) return true + if (mutation.kind === 'remove' && contains(mutation.path, this.target)) return true + if (!this.directory || !contains(this.target, mutation.path)) return false + if (this.options.recursive === true) return true + const child = relative(this.target, mutation.path) + return child !== '' && !child.startsWith('..') && !child.includes(sep) + } + + private filename(path: string): string | Buffer { + const relativePath = relative(this.target, path) + const value = this.directory && contains(this.target, path) + ? this.options.recursive === true ? relativePath : relativePath.split(sep)[0] ?? '' + : basename(this.target) + return this.options.encoding === 'buffer' ? Buffer.from(value) : value + } + + /** Stop observing and publish `close` once. */ + close(): void { + if (this.closed) return + this.closed = true + this.disposeMutation() + if (this.onAbort !== undefined) this.signal?.removeEventListener('abort', this.onAbort) + queueMicrotask(() => { this.emit('close') }) + } + + /** + * Mark this watcher as process-liveness-bearing. + * @returns This watcher. + */ + ref(): this { + this.referenced = true + return this + } + + /** + * Clear the process-liveness flag; dedicated Workers have no ref-counted event loop. + * @returns This watcher. + */ + unref(): this { + this.referenced = false + return this + } + + /** + * Read the retained process-liveness flag. + * @returns Whether this watcher is marked as keeping its owner alive. + */ + hasRef(): boolean { + return this.referenced + } +} + +/** + * Watch one path through the active VFS. + * @param path - File or directory path. + * @param optionsOrListener - Watch options, encoding, or the change listener. + * @param maybeListener - Change listener when the second argument carries options. + * @returns The closeable watcher. + */ +export function watch( + path: PathArg, + optionsOrListener?: WatchOptions | BufferEncoding | 'buffer' | WatchListener, + maybeListener?: WatchListener, +): FSWatcher { + const options: WatchOptions = typeof optionsOrListener === 'object' + ? optionsOrListener + : typeof optionsOrListener === 'string' ? { encoding: optionsOrListener } : {} + const listener = typeof optionsOrListener === 'function' ? optionsOrListener : maybeListener + const target = asPath(path) + const stats = requireActiveVfs().statSync(target) + return new FSWatcher(target, stats.isDirectory(), options, listener) +} + +/** `fs.StatWatcher` returned from `watchFile`. */ +export class StatWatcher extends EventEmitter { + private readonly disposeMutation: () => void + private timer: ReturnType | undefined + private previous: WatchStats + private stopped = false + private referenced: boolean + private readonly context: ReturnType + private readonly interval: number + private readonly bigint: boolean + + constructor(readonly path: string, options: WatchFileOptions) { + super() + this.referenced = options.persistent ?? true + this.interval = options.interval ?? 5007 + this.bigint = options.bigint ?? false + this.previous = statOrMissing(path, this.bigint) + this.context = captureAsyncContext() + this.disposeMutation = requireActiveVfs().subscribe((mutation) => { + if (overlaps(path, mutation.path)) this.schedule() + }) + if (!this.previous.isFile() && !this.previous.isDirectory()) this.schedule(true) + } + + private schedule(initialMissing = false): void { + if (this.stopped || this.timer !== undefined) return + this.timer = setTimeout(() => { + this.timer = undefined + if (this.stopped) return + const current = statOrMissing(this.path, this.bigint) + const previous = this.previous + this.previous = current + if (initialMissing || statsChanged(current, previous)) { + runWithAsyncContext(this.context, () => { this.emit('change', current, previous) }) + } + }, this.interval) + if (!this.referenced) timerUnref(this.timer) + } + + /** Stop polling and release the VFS subscription. */ + stop(): void { + if (this.stopped) return + this.stopped = true + this.disposeMutation() + if (this.timer !== undefined) clearTimeout(this.timer) + this.timer = undefined + this.emit('stop') + } + + /** Alias used by callers treating the watcher as a closeable handle. */ + close(): void { + this.stop() + } + + /** + * Mark this watcher as process-liveness-bearing. + * @returns This watcher. + */ + ref(): this { + this.referenced = true + if (this.timer !== undefined) timerRef(this.timer) + return this + } + + /** + * Mark this watcher as not keeping its owner alive. + * @returns This watcher. + */ + unref(): this { + this.referenced = false + if (this.timer !== undefined) timerUnref(this.timer) + return this + } + + /** + * Read the retained process-liveness flag. + * @returns Whether this watcher is marked as keeping its owner alive. + */ + hasRef(): boolean { + return this.referenced + } + +} + +type RefTimer = { ref?: () => unknown; unref?: () => unknown } + +/** Browser timers are numeric; Node timers expose optional liveness methods. */ +const timerRef = (timer: ReturnType): void => { + ;(timer as unknown as RefTimer).ref?.() +} + +/** Browser timers are numeric; Node timers expose optional liveness methods. */ +const timerUnref = (timer: ReturnType): void => { + ;(timer as unknown as RefTimer).unref?.() +} + +const statWatchers = new Map() + +/** + * Register a stat-poll watcher for one path. + * @param path - File or directory path, including a currently missing path. + * @param optionsOrListener - Polling options or the change listener. + * @param maybeListener - Change listener when the second argument carries options. + * @returns The path's shared stat watcher. + */ +export function watchFile( + path: PathArg, + optionsOrListener: WatchFileOptions | StatListener, + maybeListener?: StatListener, +): StatWatcher { + const options = typeof optionsOrListener === 'function' ? {} : optionsOrListener + const listener = typeof optionsOrListener === 'function' ? optionsOrListener : maybeListener + if (listener === undefined) throw new TypeError('The "listener" argument must be of type function') + const target = asPath(path) + let watcher = statWatchers.get(target) + if (watcher === undefined) { + watcher = new StatWatcher(target, options) + statWatchers.set(target, watcher) + watcher.once('stop', () => { statWatchers.delete(target) }) + } + watcher.on('change', listener as (...args: unknown[]) => void) + return watcher +} + +/** + * Remove one listener or every listener for a path. + * @param path - Watched path. + * @param listener - Specific registration to remove; omission removes all registrations. + */ +export function unwatchFile(path: PathArg, listener?: StatListener): void { + const target = asPath(path) + const watcher = statWatchers.get(target) + if (watcher === undefined) return + if (listener === undefined) watcher.removeAllListeners('change') + else watcher.removeListener('change', listener as (...args: unknown[]) => void) + if (watcher.listenerCount('change') === 0) watcher.stop() +} + +/** + * Create the promise-based watch iterator over the callback watcher. + * @param path - File or directory path. + * @param options - Watch options and cancellation signal. + * @returns An iterator of change records that closes its watcher on return or failure. + */ +export function watchAsync( + path: PathArg, + options: WatchOptions = {}, +): AsyncIterableIterator<{ eventType: 'rename' | 'change'; filename: string | Buffer | null }> { + type WatchEvent = { eventType: 'rename' | 'change'; filename: string | Buffer | null } + type Waiting = { + resolve(result: IteratorResult): void + reject(reason: unknown): void + } + const queued: WatchEvent[] = [] + const waiting: Waiting[] = [] + let watcher: FSWatcher | undefined + let failure: Error | undefined + let closed = false + + const stopWatcher = (): void => { + options.signal?.removeEventListener('abort', onAbort) + watcher?.close() + } + const settleFailure = (reason: unknown): void => { + if (closed) return + const error = reason instanceof Error ? reason : new Error(String(reason)) + closed = true + queued.length = 0 + stopWatcher() + const failed = waiting.shift() + if (failed === undefined) failure = error + else failed.reject(error) + for (const pending of waiting.splice(0)) pending.resolve({ done: true, value: undefined }) + } + const onAbort = (): void => { settleFailure(abortError(options.signal?.reason)) } + const start = (): void => { + if (watcher !== undefined || closed || failure !== undefined) return + if (options.signal?.aborted === true) { + settleFailure(abortError(options.signal.reason)) + return + } + try { + watcher = watch(path, options, (eventType, filename) => { + const event = { eventType, filename } + const pending = waiting.shift() + if (pending === undefined) queued.push(event) + else pending.resolve({ done: false, value: event }) + }) + watcher.on('error', settleFailure) + options.signal?.addEventListener('abort', onAbort, { once: true }) + } catch (error) { + settleFailure(error) + } + } + const close = (): void => { + const alreadyClosed = closed + closed = true + queued.length = 0 + failure = undefined + if (!alreadyClosed) stopWatcher() + for (const pending of waiting.splice(0)) pending.resolve({ done: true, value: undefined }) + } + + return { + [Symbol.asyncIterator]() { + return this + }, + next(): Promise> { + start() + if (failure !== undefined) { + const reason = failure + failure = undefined + return Promise.reject(reason) + } + const event = queued.shift() + if (event !== undefined) return Promise.resolve({ done: false, value: event }) + if (closed) return Promise.resolve({ done: true, value: undefined }) + return new Promise>((resolve, reject) => { waiting.push({ resolve, reject }) }) + }, + return(): Promise> { + close() + return Promise.resolve({ done: true, value: undefined }) + }, + throw(reason?: unknown): Promise> { + close() + // AsyncIterator.throw forwards the caller's exact reason, including non-Error values. + // oxlint-disable-next-line typescript/prefer-promise-reject-errors + return Promise.reject(reason) + }, + } +} diff --git a/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/fs.ts b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/fs.ts index 848d04436b..b2f4462a50 100644 --- a/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/fs.ts +++ b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/fs.ts @@ -2,19 +2,23 @@ * `node:fs` bridge over the worker's in-memory VFS. `MemoryVfs` owns paths, * bytes, the directory tree, and Node's error codes; this module adds only what * is Node-API-shaped and not VFS business: Buffer results, `Dirent` objects, - * file descriptors, `mkdtemp`, access checks, inert watches, and the promise face. + * file descriptors, `mkdtemp`, access checks, watchers, streams, and the promise face. */ import { requireActiveVfs } from '../../../storage/active.ts' -import type { MemoryVfs } from '../../../storage/memory.ts' -import type { VfsBigIntStats, VfsStatOptions, VfsStats, VfsWriteOptions } from '../../../storage/types.ts' +import type { + Vfs, VfsBigIntStats, VfsOpenFile, VfsStatOptions, VfsStats, VfsWriteOptions, +} from '../../../storage/types.ts' import { Buffer } from 'buffer' +import { Readable, Writable } from './stream.ts' import { dirname } from './path.ts' +import { abortError } from './abort-error.ts' +import { + FSWatcher, StatWatcher, unwatchFile, watch, watchAsync, watchFile, +} from './fs-watch.ts' -const vfs = (): MemoryVfs => requireActiveVfs() +const vfs = (): Vfs => requireActiveVfs() -const notImplemented = (method: string, subject: string): never => { - throw new Error(`web-preview: node:fs.${method} is not implemented in the worker host (${subject})`) -} +export { FSWatcher, StatWatcher, unwatchFile, watch, watchFile } type PathArg = string | URL | Uint8Array @@ -150,6 +154,32 @@ export function statSync(path: PathArg, options?: VfsStatOptions): VfsStats | Vf return vfs().statSync(asPath(path), options) } +/** + * Read stats through Node's callback form. + * @param path - Path to stat. + * @param optionsOrCallback - Stat options or the completion callback. + * @param maybeCallback - Completion callback when options are present. + */ +export function stat( + path: PathArg, + optionsOrCallback: VfsStatOptions | ((error: NodeJS.ErrnoException | null, stats?: VfsStats | VfsBigIntStats) => void), + maybeCallback?: (error: NodeJS.ErrnoException | null, stats?: VfsStats | VfsBigIntStats) => void, +): void { + const options = typeof optionsOrCallback === 'function' ? undefined : optionsOrCallback + const callback = typeof optionsOrCallback === 'function' ? optionsOrCallback : maybeCallback + if (callback === undefined) throw new TypeError('The "callback" argument must be of type function') + queueMicrotask(() => { + let result: VfsStats | VfsBigIntStats + try { + result = statSync(path, options) + } catch (error) { + callback(error as NodeJS.ErrnoException) + return + } + callback(null, result) + }) +} + /** * Change an entry's permission bits; stat reads back exactly what was set. * @param path - the path. @@ -169,6 +199,20 @@ export function lstatSync(path: PathArg, options?: VfsStatOptions): VfsStats | V return statSync(path, options) } +/** + * Read link stats through Node's callback form; this symlink-free VFS delegates to stat. + * @param path - Path to stat. + * @param optionsOrCallback - Stat options or the completion callback. + * @param maybeCallback - Completion callback when options are present. + */ +export function lstat( + path: PathArg, + optionsOrCallback: VfsStatOptions | ((error: NodeJS.ErrnoException | null, stats?: VfsStats | VfsBigIntStats) => void), + maybeCallback?: (error: NodeJS.ErrnoException | null, stats?: VfsStats | VfsBigIntStats) => void, +): void { + stat(path, optionsOrCallback, maybeCallback) +} + /** * Canonical path (normalization only: the image is symlink-free). * @param path - the path. @@ -252,9 +296,8 @@ export function accessSync(path: PathArg): void { } interface OpenFile { - path: string + file: VfsOpenFile position: number - append: boolean } const openFiles = new Map() @@ -265,27 +308,27 @@ let nextFd = 3 * @param path - file path. * @param flags - Node flag string: 'r', 'w', 'a', with optional '+' and the * exclusive 'x' (create-only) modifier. + * @param mode - creation permission bits. * @returns the descriptor. */ -export function openSync(path: PathArg, flags = 'r'): number { +export function openSync(path: PathArg, flags = 'r', mode?: number): number { const target = asPath(path) - const exists = vfs().existsSync(target) - if (flags.includes('x') && exists) { - const error = new Error(`EEXIST: file already exists, open '${target}'`) as Error & { code: string; path: string } - error.code = 'EEXIST' - error.path = target - throw error - } - if (flags.startsWith('r')) vfs().realpathSync(target) - else if (flags.startsWith('w') || !exists) vfs().writeFileSync(target, new Uint8Array(0)) + const file = vfs().openFileSync(target, flags, mode) const fd = nextFd++ - openFiles.set(fd, { path: target, position: 0, append: flags.startsWith('a') }) + openFiles.set(fd, { file, position: 0 }) return fd } +const badFileDescriptor = (syscall: string): never => { + const error = new Error(`EBADF: bad file descriptor, ${syscall}`) as Error & { code: string; syscall: string } + error.code = 'EBADF' + error.syscall = syscall + throw error +} + const fileOf = (fd: number, syscall: string): OpenFile => { const file = openFiles.get(fd) - if (file === undefined) throw new Error(`EBADF: bad file descriptor, ${syscall}`) + if (file === undefined) return badFileDescriptor(syscall) return file } @@ -306,9 +349,8 @@ export function readSync( position: number | null = null, ): number { const file = fileOf(fd, 'read') - const bytes = bytesOf(file.path) const from = position ?? file.position - const slice = bytes.subarray(from, from + length) + const slice = file.file.read(from, length) buffer.set(slice, offset) if (position === null) file.position = from + slice.byteLength return slice.byteLength @@ -323,17 +365,10 @@ export function readSync( export function writeSync(fd: number, data: string | Uint8Array): number { const file = fileOf(fd, 'write') const bytes = typeof data === 'string' ? new TextEncoder().encode(data) : data - if (file.append) { - vfs().appendFileSync(file.path, bytes) - return bytes.byteLength - } - const existing = vfs().existsSync(file.path) ? bytesOf(file.path) : new Uint8Array(0) - const merged = new Uint8Array(Math.max(existing.byteLength, file.position + bytes.byteLength)) - merged.set(existing, 0) - merged.set(bytes, file.position) - vfs().writeFileSync(file.path, merged) - file.position += bytes.byteLength - return bytes.byteLength + const position = file.file.append ? file.file.stat().size : file.position + const bytesWritten = file.file.write(position, bytes) + file.position = position + bytesWritten + return bytesWritten } /** @@ -341,23 +376,22 @@ export function writeSync(fd: number, data: string | Uint8Array): number { * @param fd - descriptor. */ export function closeSync(fd: number): void { - openFiles.delete(fd) + if (!openFiles.delete(fd)) fileOf(fd, 'close') } /** - * Create a second name for one file's contents. Hard links do not exist in the - * VFS, so the bytes are copied. + * Create a second name for one file identity. * @param from - existing path. * @param to - new path. */ export function linkSync(from: PathArg, to: PathArg): void { - writeFileSync(to, bytesOf(asPath(from))) + vfs().linkSync(asPath(from), asPath(to)) } /** * Open file handle (`fs.FileHandle` subset): the atomic-write and durability - * pair the storage backends use. `sync`/`datasync` are no-ops — an in-memory - * filesystem has nothing to flush, and a worker reload loses it either way. + * pair the storage backends use. `sync`/`datasync` settle the active VFS's + * optional write-behind sink. */ export interface FileHandle { readonly fd: number @@ -377,82 +411,310 @@ export interface FileHandle { * helpers do before an fsync. * @param path - file or directory path. * @param flags - Node flag string. + * @param mode - creation permission bits. * @returns the handle. */ -export function openHandleSync(path: PathArg, flags = 'r'): FileHandle { +export function openHandleSync(path: PathArg, flags = 'r', mode?: number): FileHandle { const target = asPath(path) const directory = vfs().existsSync(target) && vfs().statSync(target).isDirectory() - const append = flags.startsWith('a') - const fd = directory ? -1 : openSync(target, flags) + const fd = directory ? -1 : openSync(target, flags, mode) + let closed = false + const descriptor = (syscall: string): OpenFile => fileOf(fd, syscall) return { fd, - readFile: async (options?: EncodingOption) => readFileSync(target, options), - // Node appends when the handle was opened with 'a'. The JSONL session log - // depends on it — `open(path, 'a')` then `writeFile(batch)` — and replacing - // the file there destroys the header frame its reader requires. + readFile: async (options?: EncodingOption) => { + if (directory) return readFileSync(target, options) + const open = descriptor('read') + const bytes = open.file.read(open.position, Math.max(0, open.file.stat().size - open.position)) + open.position += bytes.length + const encoding = encodingOf(options) + return encoding === undefined || encoding === 'utf8' || encoding === 'utf-8' + ? (encoding === undefined ? asBuffer(bytes) : new TextDecoder().decode(bytes)) + : asBuffer(bytes).toString(encoding) + }, writeFile: async (data: string | Uint8Array) => { - if (append) appendFileSync(target, data) - else writeFileSync(target, data) + if (directory) writeFileSync(target, data) + else writeSync(fd, data) }, write: async (data: string | Uint8Array) => ({ bytesWritten: writeSync(fd, data) }), read: async (buffer: Uint8Array, offset = 0, length = buffer.byteLength, position: number | null = null) => ({ bytesRead: readSync(fd, buffer, offset, length, position), buffer, }), - stat: async () => statSync(target) as VfsStats, + stat: async () => directory ? statSync(target) as VfsStats : descriptor('fstat').file.stat(), truncate: async (length = 0) => { - writeFileSync(target, bytesOf(target).subarray(0, length)) + if (directory) writeFileSync(target, new Uint8Array(length)) + else descriptor('ftruncate').file.truncate(length) }, - sync: async () => { /* memory-backed: nothing to flush */ }, - datasync: async () => { /* memory-backed: nothing to flush */ }, + sync: async () => { await vfs().flush() }, + datasync: async () => { await vfs().flush() }, close: async () => { + if (closed) return + closed = true if (fd !== -1) closeSync(fd) }, } } -/** - * Watch registration refuses loudly, and NOT because watching is hard. - * - * An inert watcher would not serve this caller. `skill-filesystem` does not - * merely register a listener — `openStableWatcher` opens a watcher and then - * loops until two consecutive mode probes agree, so a watcher that reports - * success and never fires leaves `observeRoots()` awaiting forever: the skill - * catalog RPC never answers and the worker's single thread stops serving `/api` - * for the rest of the session. A refusal instead fails that path fast, which the - * provider already handles by returning an incomplete observation. - * - * So the family split is about what the CALLER does with the capability, not - * about the capability: a listener registration tolerates absence, a watcher - * whose progress is awaited does not. - * @param path - the path a caller wanted watched, named in the refusal. - * @returns Never — it throws naming the unavailable member. - */ -export function watchFile(path: PathArg): never { - return notImplemented('watchFile', asPath(path)) +/** Options supported by the VFS-backed read stream. */ +export interface ReadStreamOptions { + flags?: string + encoding?: BufferEncoding | null + autoClose?: boolean + emitClose?: boolean + start?: number + end?: number + highWaterMark?: number + signal?: AbortSignal } -/** Watch removal; teardown paths call it unconditionally, and nothing was watched. */ -export function unwatchFile(): void { - // No watch was ever established. +/** Options supported by the VFS-backed write stream. */ +export interface WriteStreamOptions { + flags?: string + encoding?: BufferEncoding | null + mode?: number + autoClose?: boolean + emitClose?: boolean + start?: number + highWaterMark?: number + signal?: AbortSignal +} + +/** Node implements file-stream `autoClose` through the stream's `autoDestroy` state. */ +const streamAutoDestroy = (autoClose: boolean | undefined): boolean => autoClose ?? true + +interface FileStreamState { + fd: number | null + pending: boolean +} + +/** Release the descriptor and abort listener shared by both file-stream directions. */ +function destroyFileStream( + stream: FileStreamState, + signal: AbortSignal | undefined, + onAbort: (() => void) | undefined, + error: Error | null, + callback: (error: Error | null) => void, +): void { + signal?.removeEventListener('abort', onAbort as () => void) + if (stream.fd !== null) closeSync(stream.fd) + stream.fd = null + stream.pending = false + callback(error) +} + +interface ClosableFileStream { + once(event: string, listener: () => void): unknown + destroy(): unknown +} + +/** Register an optional completion callback and explicitly destroy a file stream. */ +function closeFileStream( + stream: ClosableFileStream, + callback?: (error?: NodeJS.ErrnoException | null) => void, +): void { + if (callback !== undefined) stream.once('close', () => { callback(null) }) + stream.destroy() +} + +/** Read stream over one VFS file. */ +export class ReadStream extends Readable { + /** Resolved path opened by this stream. */ + readonly path: string + /** Open descriptor, or null before open and after close. */ + fd: number | null = null + /** Whether the descriptor is still waiting to open. */ + pending = true + /** Bytes delivered by this stream. */ + bytesRead = 0 + private readonly start: number + private readonly end: number + private readonly flags: string + private readonly signal: AbortSignal | undefined + private readonly onAbort: (() => void) | undefined + private position: number + + constructor(path: PathArg, options: ReadStreamOptions = {}) { + super({ + autoDestroy: streamAutoDestroy(options.autoClose), + emitClose: options.emitClose ?? true, + highWaterMark: options.highWaterMark ?? 64 * 1024, + }) + this.path = asPath(path) + this.start = options.start ?? 0 + this.end = options.end ?? Number.POSITIVE_INFINITY + this.flags = options.flags ?? 'r' + this.position = this.start + this.signal = options.signal + this.onAbort = options.signal === undefined ? undefined : () => { this.destroy(abortError(options.signal?.reason)) } + if (options.encoding !== undefined && options.encoding !== null) this.setEncoding(options.encoding) + options.signal?.addEventListener('abort', this.onAbort as () => void, { once: true }) + } + + override _construct(callback: (error?: Error | null) => void): void { + if (this.start < 0 || this.end < this.start) { + callback(new RangeError('The value of "start" is out of range')) + return + } + if (this.signal?.aborted === true) { + callback(abortError(this.signal.reason)) + return + } + let fd: number + try { + fd = openSync(this.path, this.flags) + } catch (error) { + callback(error as Error) + return + } + this.fd = fd + this.pending = false + callback() + this.emit('open', fd) + this.emit('ready') + } + + override _read(size: number): void { + if (this.fd === null) return + const remaining = this.end === Number.POSITIVE_INFINITY ? size : Math.min(size, this.end - this.position + 1) + if (remaining <= 0) { + this.push(null) + return + } + const buffer = Buffer.allocUnsafe(remaining) + let count: number + try { + count = readSync(this.fd, buffer, 0, remaining, this.position) + } catch (error) { + this.destroy(error as Error) + return + } + if (count === 0) { + this.push(null) + return + } + this.position += count + this.bytesRead += count + this.push(buffer.subarray(0, count)) + } + + override _destroy(error: Error | null, callback: (error?: Error | null) => void): void { + destroyFileStream(this, this.signal, this.onAbort, error, callback) + } + + /** + * Close the stream and release its descriptor. + * @param callback - Optional completion callback after `close`. + */ + close(callback?: (error?: NodeJS.ErrnoException | null) => void): void { + closeFileStream(this, callback) + } +} + +/** Writable stream committing chunks through the VFS file-descriptor face. */ +export class WriteStream extends Writable { + /** Resolved path opened by this stream. */ + readonly path: string + /** Open descriptor, or null before open and after close. */ + fd: number | null = null + /** Whether the descriptor is still waiting to open. */ + pending = true + /** Bytes committed by this stream. */ + bytesWritten = 0 + private readonly flags: string + private readonly mode: number | undefined + private readonly start: number | undefined + private readonly signal: AbortSignal | undefined + private readonly onAbort: (() => void) | undefined + + constructor(path: PathArg, options: WriteStreamOptions = {}) { + super({ + autoDestroy: streamAutoDestroy(options.autoClose), + decodeStrings: true, + defaultEncoding: options.encoding ?? 'utf8', + emitClose: options.emitClose ?? true, + highWaterMark: options.highWaterMark ?? 64 * 1024, + }) + this.path = asPath(path) + this.flags = options.flags ?? 'w' + this.mode = options.mode + this.start = options.start + this.signal = options.signal + this.onAbort = options.signal === undefined ? undefined : () => { this.destroy(abortError(options.signal?.reason)) } + options.signal?.addEventListener('abort', this.onAbort as () => void, { once: true }) + } + + override _construct(callback: (error?: Error | null) => void): void { + if (this.start !== undefined && this.start < 0) { + callback(new RangeError('The value of "start" is out of range')) + return + } + if (this.signal?.aborted === true) { + callback(abortError(this.signal.reason)) + return + } + let fd: number + try { + fd = openSync(this.path, this.flags, this.mode) + } catch (error) { + callback(error as Error) + return + } + this.fd = fd + if (this.start !== undefined) fileOf(fd, 'write').position = this.start + this.pending = false + callback() + this.emit('open', fd) + this.emit('ready') + } + + override _write( + chunk: string | Uint8Array, + encoding: BufferEncoding, + callback: (error?: Error | null) => void, + ): void { + try { + const fd = this.fd + if (fd === null) return badFileDescriptor('write') + const data = typeof chunk === 'string' ? Buffer.from(chunk, encoding) : chunk + this.bytesWritten += writeSync(fd, data) + callback() + } catch (error) { + callback(error as Error) + } + } + + override _destroy(error: Error | null, callback: (error: Error | null) => void): void { + destroyFileStream(this, this.signal, this.onAbort, error, callback) + } + + /** + * Close the stream and release its descriptor. + * @param callback - Optional completion callback after `close`. + */ + close(callback?: (error?: NodeJS.ErrnoException | null) => void): void { + closeFileStream(this, callback) + } } /** - * Streaming read is unavailable: node:stream has no implementation here. - * @param path - the path a caller wanted streamed, named in the refusal. - * @returns Never — it throws naming the unavailable member. + * Create a Node-compatible readable file stream over the VFS. + * @param path - File path. + * @param options - Encoding, range, open, buffer, and abort options. + * @returns The readable file stream. */ -export function createReadStream(path: PathArg): never { - return notImplemented('createReadStream', asPath(path)) +export function createReadStream(path: PathArg, options?: ReadStreamOptions | BufferEncoding): ReadStream { + return new ReadStream(path, typeof options === 'string' ? { encoding: options } : options) } /** - * Streaming write counterpart of {@link createReadStream}. - * @param path - the path a caller wanted streamed, named in the refusal. - * @returns Never — it throws naming the unavailable member. + * Create a Node-compatible writable file stream over the VFS. + * @param path - File path. + * @param options - Encoding, open, buffer, and abort options. + * @returns The writable file stream. */ -export function createWriteStream(path: PathArg): never { - return notImplemented('createWriteStream', asPath(path)) +export function createWriteStream(path: PathArg, options?: WriteStreamOptions | BufferEncoding): WriteStream { + return new WriteStream(path, typeof options === 'string' ? { encoding: options } : options) } /** Open directory handle (`fs.Dir` subset): iteration plus the close pair. */ @@ -535,14 +797,14 @@ export const promises = { mkdirSync(dirname(target), { recursive: true }) writeFileSync(target, bytesOf(source)) }, - // The VFS has no inodes, so a hard link is a byte copy: the caller's contract - // is only that both names read the same content until one is removed. + // The VFS keeps both names attached to one file identity until either name is removed. link: async (from: PathArg, to: PathArg): Promise => { linkSync(from, to) }, - open: async (path: PathArg, flags?: string): Promise => openHandleSync(path, flags), + open: async (path: PathArg, flags?: string, mode?: number): Promise => openHandleSync(path, flags, mode), opendir: async (path: PathArg): Promise => opendirSync(path), truncate: async (path: PathArg, length = 0): Promise => { - writeFileSync(path, bytesOf(asPath(path)).subarray(0, length)) + vfs().truncateSync(asPath(path), length) }, + watch: watchAsync, constants, } satisfies Partial> @@ -559,10 +821,11 @@ export const __esModule = true * the subsets the host tree reads. */ type OwnSignature = - | 'constants' | 'promises' | 'Dirent' + | 'constants' | 'promises' | 'Dirent' | 'FSWatcher' | 'StatWatcher' | 'ReadStream' | 'WriteStream' | 'readFileSync' | 'writeFileSync' | 'appendFileSync' | 'statSync' | 'lstatSync' | 'realpathSync' | 'readdirSync' | 'mkdirSync' | 'mkdtempSync' | 'rmSync' | 'opendirSync' - | 'openSync' | 'readSync' | 'writeSync' + | 'openSync' | 'readSync' | 'writeSync' | 'stat' | 'lstat' | 'watch' | 'watchFile' | 'unwatchFile' + | 'createReadStream' | 'createWriteStream' /** * The `node:fs` declarations this module stands in for. Every other member is @@ -574,10 +837,10 @@ type NodeFace = Partial> /** CommonJS default export: the members `require()` hands a caller of this module. */ export default { - constants, promises, Dirent, - readFileSync, writeFileSync, appendFileSync, existsSync, statSync, lstatSync, realpathSync, chmodSync, + constants, promises, Dirent, FSWatcher, StatWatcher, ReadStream, WriteStream, + readFileSync, writeFileSync, appendFileSync, existsSync, statSync, stat, lstatSync, lstat, realpathSync, chmodSync, readdirSync, mkdirSync, mkdtempSync, rmSync, unlinkSync, renameSync, accessSync, opendirSync, openHandleSync, linkSync, - openSync, readSync, writeSync, closeSync, watchFile, unwatchFile, + openSync, readSync, writeSync, closeSync, watch, watchFile, unwatchFile, createReadStream, createWriteStream, } satisfies NodeFace diff --git a/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/fs/promises.ts b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/fs/promises.ts index a49ca4a22f..c83ca26b6d 100644 --- a/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/fs/promises.ts +++ b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/fs/promises.ts @@ -9,7 +9,7 @@ import { Dirent, promises } from '../fs.ts' /** The promise members of the VFS bridge, as `node:fs/promises` names them. */ export const { readFile, writeFile, appendFile, mkdir, mkdtemp, readdir, stat, lstat, realpath, rm, unlink, - rename, access, chmod, cp, link, open, opendir, truncate, constants, + rename, access, chmod, cp, link, open, opendir, truncate, watch, constants, } = promises export { Dirent } diff --git a/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/module.ts b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/module.ts index bf77f12bf6..8ad91b49cd 100644 --- a/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/module.ts +++ b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/module.ts @@ -1,7 +1,8 @@ /** * `node:module` for the worker: `createRequire` hands out the worker module - * loader's synchronous require, so typert's `require.resolve('/package.json') - * + readFileSync + import()` bypass runs unmodified over the VFS. + * loader's synchronous require. Typert can resolve package exports, and package + * inventory can discover manifests through `require.resolve.paths()` without + * either consumer changing for the Worker. */ import { requireActiveModuleLoader, type WorkerRequire } from '../../../module-system/module-loader.ts' @@ -11,7 +12,7 @@ export type NodeRequire = WorkerRequire /** * Build a `require` bound to a base path or file URL. * @param base - directory, file path, or file URL the resolution starts from. - * @returns the synchronous require face. + * @returns the synchronous require face, including `resolve()` and `resolve.paths()`. */ export function createRequire(base: string | URL): NodeRequire { return requireActiveModuleLoader().createRequire(base) diff --git a/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/stream.ts b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/stream.ts new file mode 100644 index 0000000000..19c60afa15 --- /dev/null +++ b/packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/stream.ts @@ -0,0 +1,84 @@ +/** + * `node:stream` compatibility backed by readable-stream's browser build. + * + * readable-stream is the userland copy of Node's stream implementation. The + * worker owns only platform adapters such as VFS file streams; stream state, + * backpressure, async iteration, abort handling, and event ordering stay in + * that maintained implementation. + */ +import Stream from 'readable-stream' + +type StreamRuntime = typeof import('node:stream') & { + compose(...streams: unknown[]): unknown + destroy(stream: unknown, error?: Error): void + isDisturbed(stream: unknown): boolean +} + +type StreamStatics = typeof import('node:stream').Stream & { + getDefaultHighWaterMark(objectMode: boolean): number + isDestroyed(stream: unknown): boolean | null + isWritable(stream: unknown): boolean | null + setDefaultHighWaterMark(objectMode: boolean, value: number): void +} + +const nodeStream = Stream as unknown as StreamRuntime + +/* oxlint-disable typescript/unbound-method -- readable-stream's namespace statics do not read `this`. */ +const { + Duplex, PassThrough, Readable, Stream: StreamBase, Transform, Writable, + addAbortSignal, compose, destroy, finished, isDisturbed, isErrored, isReadable, pipeline, promises, +} = nodeStream +const streamStatics = StreamBase as unknown as StreamStatics +const { + getDefaultHighWaterMark, isDestroyed, isWritable, setDefaultHighWaterMark, +} = streamStatics +/* oxlint-enable typescript/unbound-method */ + +// readable-stream tracks Node 18's 16 KiB byte default; this repository runs +// Node 22+, whose generic and file streams use 64 KiB. +if (getDefaultHighWaterMark(false) !== 64 * 1024) setDefaultHighWaterMark(false, 64 * 1024) + +/** + * Test whether a value is an ArrayBuffer view. + * @param value - Candidate value. + * @returns Whether the value is a typed-array or DataView instance. + */ +const _isArrayBufferView = (value: unknown): value is ArrayBufferView => ArrayBuffer.isView(value) + +/** Default-import namespace carrying Node's stream class and static helpers. */ +const streamDefault = Object.assign(StreamBase, { + _isArrayBufferView, + getDefaultHighWaterMark, + isDestroyed, + isWritable, + setDefaultHighWaterMark, +}) + +export { + Duplex, + PassThrough, + Readable, + StreamBase as Stream, + Transform, + Writable, + addAbortSignal, + compose, + destroy, + finished, + getDefaultHighWaterMark, + _isArrayBufferView, + isDestroyed, + isDisturbed, + isErrored, + isReadable, + isWritable, + pipeline, + promises, + setDefaultHighWaterMark, +} + +/** CommonJS interop marker consumed by the worker module loader. */ +export const __esModule = true + +/** CommonJS-compatible namespace for default imports. */ +export default streamDefault diff --git a/packages/experimental/webworker-runtime/src/node/builtin_modules/mock/stream.ts b/packages/experimental/webworker-runtime/src/node/builtin_modules/mock/stream.ts deleted file mode 100644 index 7c30396bec..0000000000 --- a/packages/experimental/webworker-runtime/src/node/builtin_modules/mock/stream.ts +++ /dev/null @@ -1,38 +0,0 @@ -/** - * `node:stream` stub. Every harness import of this module in the reachable tree - * is type-only (`Duplex`/`Readable`/`Writable` annotations), so nothing here runs - * unless a value import appears; then it says so. - */ -import { notImplementedFail } from '../../notImplementedFail.ts' - -const MODULE = 'node:stream' - -/** Readable stream (unavailable; use WHATWG ReadableStream). */ -export const Readable: typeof import('node:stream').Readable = notImplementedFail(MODULE, 'Readable') - -/** Writable stream (unavailable). */ -export const Writable: typeof import('node:stream').Writable = notImplementedFail(MODULE, 'Writable') - -/** Duplex stream (unavailable). */ -export const Duplex: typeof import('node:stream').Duplex = notImplementedFail(MODULE, 'Duplex') - -/** Transform stream (unavailable). */ -export const Transform: typeof import('node:stream').Transform = notImplementedFail(MODULE, 'Transform') - -/** PassThrough stream (unavailable). */ -export const PassThrough: typeof import('node:stream').PassThrough = notImplementedFail(MODULE, 'PassThrough') - -/** Pipeline helper (unavailable). */ -export const pipeline: typeof import('node:stream').pipeline = notImplementedFail(MODULE, 'pipeline') - -/** Finished helper (unavailable). */ -export const finished: typeof import('node:stream').finished = notImplementedFail(MODULE, 'finished') - -/** CommonJS interop marker: the worker loader hands `default` to default imports (see ./builtins.ts). */ -export const __esModule = true - -/** The `node:stream` declarations this module stands in for. */ -type NodeFace = Partial - -/** CommonJS default export: the members `require()` hands a caller of this module. */ -export default { Readable, Writable, Duplex, Transform, PassThrough, pipeline, finished } satisfies NodeFace diff --git a/packages/experimental/webworker-runtime/src/node/builtins.ts b/packages/experimental/webworker-runtime/src/node/builtins.ts index f7008b518d..a2260d9831 100644 --- a/packages/experimental/webworker-runtime/src/node/builtins.ts +++ b/packages/experimental/webworker-runtime/src/node/builtins.ts @@ -32,6 +32,7 @@ import * as nodeModule from './builtin_modules/implemented/module.ts' import * as nodeOs from './builtin_modules/implemented/os.ts' import * as nodePath from './builtin_modules/implemented/path.ts' import * as nodePerfHooks from './builtin_modules/implemented/perf_hooks.ts' +import * as nodeStream from './builtin_modules/implemented/stream.ts' import * as nodeTimersPromises from './builtin_modules/implemented/timers/promises.ts' import * as nodeUrl from './builtin_modules/implemented/url.ts' import * as nodeUtil from './builtin_modules/implemented/util.ts' @@ -40,12 +41,9 @@ import * as nodeZlib from './builtin_modules/implemented/zlib.ts' import * as nodeChildProcess from './builtin_modules/implemented/child_process.ts' import * as nodeNet from './builtin_modules/mock/net.ts' import * as nodeSqlite from './builtin_modules/mock/sqlite.ts' -import * as nodeStream from './builtin_modules/mock/stream.ts' import * as nodeVm from './builtin_modules/mock/vm.ts' import * as nodeWorkerThreads from './builtin_modules/mock/worker_threads.ts' -import * as chokidar from './external_packages/chokidar.ts' import * as koffi from './external_packages/koffi.ts' -import * as landlockRun from './external_packages/node-addon-landlock-run.ts' import * as nodePty from './external_packages/node-pty.ts' import * as piAi from './external_packages/pi-ai.ts' import * as ripgrep from './external_packages/ripgrep.ts' @@ -83,14 +81,12 @@ const BUILTINS: Record = { /** External npm packages replaced wholesale (structural not-implemented stubs and fakes). */ const EXTERNALS: Record = { - 'chokidar': () => chokidar, 'koffi': () => koffi, 'sharp': () => sharp, 'node-pty': () => nodePty, 'ws': () => ws, '@vscode/ripgrep': () => ripgrep, '@earendil-works/pi-ai': () => piAi, - '@deepseek-ai/node-addon-landlock-run': () => landlockRun, } /** diff --git a/packages/experimental/webworker-runtime/src/node/external_packages/chokidar.ts b/packages/experimental/webworker-runtime/src/node/external_packages/chokidar.ts deleted file mode 100644 index b8c96ccd65..0000000000 --- a/packages/experimental/webworker-runtime/src/node/external_packages/chokidar.ts +++ /dev/null @@ -1,68 +0,0 @@ -/** - * `chokidar` stub: a constructible watcher that never fires. Settings and - * credentials call `watch()` unconditionally in `[Service.init]`, and the - * in-memory VFS has no external writer, so "no events" is the truth here rather - * than a degradation. - */ - -/** No-op watcher with chokidar's chainable face. */ -export class FSWatcher { - /** - * Register a listener; no event is ever emitted. - * @returns this watcher. - */ - on(): this { - return this - } - - /** - * Register a one-shot listener; no event is ever emitted. - * @returns this watcher. - */ - once(): this { - return this - } - - /** - * Add paths to the (inert) watch set. - * @returns this watcher. - */ - add(): this { - return this - } - - /** - * Remove paths from the (inert) watch set. - * @returns this watcher. - */ - unwatch(): this { - return this - } - - /** - * Watched paths, as chokidar reports them. - * @returns An empty record; nothing is ever watched. - */ - getWatched(): Record { - return {} - } - - /** Close the watcher. */ - async close(): Promise { - // Nothing was ever watched. - } -} - -/** - * Create an inert watcher. - * @returns the watcher. - */ -export function watch(): FSWatcher { - return new FSWatcher() -} - -/** CommonJS interop marker: the worker loader hands `default` to default imports (see ./builtins.ts). */ -export const __esModule = true - -/** CommonJS default export: the members `require()` hands a caller of this module. */ -export default { watch, FSWatcher } diff --git a/packages/experimental/webworker-runtime/src/node/external_packages/node-addon-landlock-run.ts b/packages/experimental/webworker-runtime/src/node/external_packages/node-addon-landlock-run.ts deleted file mode 100644 index b963837cb4..0000000000 --- a/packages/experimental/webworker-runtime/src/node/external_packages/node-addon-landlock-run.ts +++ /dev/null @@ -1,31 +0,0 @@ -/** - * `@deepseek-ai/node-addon-landlock-run` stub: the Landlock launcher. Sandboxing - * is part of the declared excluded surface, so `sandbox-local` mounts with the - * launcher path and probe present and fails when it tries to confine a process. - */ -import { notImplementedFail } from '../notImplementedFail.ts' - -const MODULE = '@deepseek-ai/node-addon-landlock-run' - -/** Launcher executable name, read at module scope by sandbox-local. */ -export const LAUNCHER_BIN = 'landlock-run' - -/** Exit code the launcher reports when confinement itself fails. */ -export const LAUNCHER_FAILURE_EXIT = 126 - -/** - * Path of the launcher binary; nothing in a browser can execute it. - * @returns The image path consumers read before failing on their own terms. - */ -export function launcherPath(): string { - return `/dsh/bin/${LAUNCHER_BIN}` -} - -/** Landlock availability probe (unavailable). */ -export const probe = notImplementedFail(MODULE, 'probe') - -/** CommonJS interop marker: the worker loader hands `default` to default imports (see ./builtins.ts). */ -export const __esModule = true - -/** CommonJS default export: the members `require()` hands a caller of this module. */ -export default { LAUNCHER_BIN, LAUNCHER_FAILURE_EXIT, launcherPath, probe } diff --git a/packages/experimental/webworker-runtime/src/node/external_packages/replaced-externals.ts b/packages/experimental/webworker-runtime/src/node/external_packages/replaced-externals.ts index 6332f8af02..528e6f6885 100644 --- a/packages/experimental/webworker-runtime/src/node/external_packages/replaced-externals.ts +++ b/packages/experimental/webworker-runtime/src/node/external_packages/replaced-externals.ts @@ -8,10 +8,8 @@ /** External packages served from the worker bundle instead of the VFS. */ export const REPLACED_EXTERNAL_PACKAGES: readonly string[] = [ - '@deepseek-ai/node-addon-landlock-run', '@earendil-works/pi-ai', '@vscode/ripgrep', - 'chokidar', 'koffi', 'node-pty', 'sharp', diff --git a/packages/experimental/webworker-runtime/src/shell/fs-access.ts b/packages/experimental/webworker-runtime/src/shell/fs-access.ts index 64d2a6a895..9099c43559 100644 --- a/packages/experimental/webworker-runtime/src/shell/fs-access.ts +++ b/packages/experimental/webworker-runtime/src/shell/fs-access.ts @@ -57,7 +57,8 @@ export function describeFailure(program: string, path: string, error: unknown): * @returns the error to throw. */ export function filesystemError(code: string, syscall: string, path: string): VfsError { - const error = new Error(`${code}: ${syscall} failed, ${syscall} '${path}'`) as VfsError + const reason = code === 'EACCES' ? 'permission denied' : `${syscall} failed` + const error = new Error(`${code}: ${reason}, ${syscall} '${path}'`) as VfsError error.code = code error.path = path error.syscall = syscall @@ -75,19 +76,17 @@ function statsOf(stats: VfsStats): ShellStats { */ export function hostFileSystem(): ShellFileSystem { const vfs = (): ReturnType => requireActiveVfs() - // oxlint-disable-next-line typescript/require-await -- async face, in-memory backend; see the note below. - const stat = async (path: string): Promise => { + const stat = (path: string): Promise => { try { - return statsOf(vfs().statSync(path) as VfsStats) + return Promise.resolve(statsOf(vfs().statSync(path) as VfsStats)) } catch { // Absence is the answer callers branch on; every other failure mode of // the in-memory backend is also "this path holds nothing readable". - return undefined + return Promise.resolve(undefined) } } // Several members take no await: the face is asynchronous because a process // worker's filesystem is, while this backend answers from memory. - /* oxlint-disable typescript/require-await -- see the note above. */ return { stat, list: async (path: string): Promise => { @@ -102,19 +101,22 @@ export function hostFileSystem(): ShellFileSystem { if ((await stat(path))?.directory === true) throw filesystemError('EISDIR', 'read', path) return vfs().readFileSync(path, 'utf8') as string }, - writeText: async (path: string, text: string, append = false): Promise => { + writeText: (path: string, text: string, append = false): Promise => { if (append) vfs().appendFileSync(path, text) else vfs().writeFileSync(path, text) + return Promise.resolve() }, - mkdir: async (path: string, recursive: boolean): Promise => { + mkdir: (path: string, recursive: boolean): Promise => { vfs().mkdirSync(path, { recursive }) + return Promise.resolve() }, - remove: async (path: string, options: { recursive: boolean; force: boolean }): Promise => { + remove: (path: string, options: { recursive: boolean; force: boolean }): Promise => { vfs().rmSync(path, options) + return Promise.resolve() }, - rename: async (from: string, to: string): Promise => { + rename: (from: string, to: string): Promise => { vfs().renameSync(from, to) + return Promise.resolve() }, } - /* oxlint-enable typescript/require-await */ } diff --git a/packages/experimental/webworker-runtime/src/shell/process/landlock.ts b/packages/experimental/webworker-runtime/src/shell/process/landlock.ts new file mode 100644 index 0000000000..5d8c8c3ca0 --- /dev/null +++ b/packages/experimental/webworker-runtime/src/shell/process/landlock.ts @@ -0,0 +1,194 @@ +/** Landlock launcher parsing and per-process VFS enforcement for the worker shell. */ +import { resolve } from '../../module-system/posix-path.ts' +import { DSH_TMP } from '../../storage/paths.ts' +import { filesystemError } from '../fs-access.ts' +import type { ShellDirent, ShellFileSystem, ShellStats } from '../types.ts' +import type { VirtualExecutable, VirtualExecutableExit } from './virtual-executables.ts' + +/** Parsed invocation of the native launcher's unchanged argv grammar. */ +export type LandlockInvocation = + | { readonly kind: 'probe' } + | { + readonly kind: 'run' + readonly readOnly: readonly string[] + readonly readWrite: readonly string[] + readonly argv: readonly string[] + } + +/** Launcher-owned failure; callers print its message with the `landlock-run:` prefix. */ +export class LandlockLauncherError extends Error {} + +/** + * Parse the native launcher's argv grammar. + * @param args - Arguments after the launcher executable. + * @returns A probe or confined-run request. + */ +export function parseLandlockArguments(args: readonly string[]): LandlockInvocation { + const readOnly: string[] = [] + const readWrite: string[] = [] + for (let index = 0; index < args.length;) { + const argument = args[index] as string + if (argument === '--probe') { + if (args.length !== 1) throw new LandlockLauncherError('usage error: --probe takes no other arguments') + return { kind: 'probe' } + } + if (argument === '--ro' || argument === '--rw') { + const path = args[index + 1] + if (path === undefined) throw new LandlockLauncherError(`usage error: ${argument} requires a path`) + ;(argument === '--ro' ? readOnly : readWrite).push(path) + index += 2 + continue + } + if (argument === '--') { + const argv = args.slice(index + 1) + if (argv.length === 0) throw new LandlockLauncherError('usage error: missing `-- ...` command') + return { kind: 'run', readOnly, readWrite, argv } + } + throw new LandlockLauncherError(`usage error: unknown argument: ${argument}`) + } + throw new LandlockLauncherError('usage error: missing `-- ...` command') +} + +/** Map the host launcher's temp path into the Worker VFS. */ +function vfsPath(path: string, cwd: string): string { + const resolved = resolve(cwd, path) + const absolute = resolved.length > 1 ? resolved.replace(/\/+$/u, '') : resolved + if (absolute === '/tmp') return DSH_TMP + if (absolute.startsWith('/tmp/')) return `${DSH_TMP}${absolute.slice('/tmp'.length)}` + return absolute +} + +/** Whether a normalized path is the root itself or one of its descendants. */ +function contains(root: string, path: string): boolean { + return root === '/' || path === root || path.startsWith(`${root}/`) +} + +/** Throw the denial dialect consumed by `dsh-bash-sandbox`. */ +function deny(syscall: string, path: string): never { + throw filesystemError('EACCES', syscall, path) +} + +/** Stats for the virtual `/dev/null` file. */ +const NULL_STATS: ShellStats = { directory: false, size: 0, mtimeMs: 0 } +const DEV_ROOT = '/dev' +const NULL_PATH = '/dev/null' + +/** Build one launcher-owned terminal result. */ +function launcherExit(exitCode: number, stdout = '', stderr = ''): VirtualExecutableExit { + return { kind: 'exit', exitCode, stdout, stderr } +} + +/** Convert a parser or grant failure into the native launcher's fatal dialect. */ +function launcherFailure(error: unknown): VirtualExecutableExit { + const detail = error instanceof LandlockLauncherError ? error.message : String(error) + return launcherExit(125, '', `landlock-run: ${detail}\n`) +} + +/** + * Validate grant roots and create one process-local filesystem guard. + * @param base - Host-side VFS adapter all permitted calls delegate to. + * @param invocation - Parsed confined-run request. + * @param cwd - Launcher's working directory for relative grant paths. + * @returns A filesystem enforcing only this invocation's grants. + */ +export async function landlockFileSystem( + base: ShellFileSystem, + invocation: Extract, + cwd: string, +): Promise { + const normalizeGrant = async (path: string): Promise => { + if (path === '') throw new LandlockLauncherError('cannot open rule path: : No such file or directory') + const target = vfsPath(path, cwd) + if (target !== DEV_ROOT && target !== NULL_PATH && await base.stat(target) === undefined) { + throw new LandlockLauncherError(`cannot open rule path: ${path}: No such file or directory`) + } + return target + } + const readOnly = await Promise.all(invocation.readOnly.map(normalizeGrant)) + const readWrite = await Promise.all(invocation.readWrite.map(normalizeGrant)) + const readable = [...readOnly, ...readWrite] + + const checkedPath = (path: string, syscall: string): string => { + const target = vfsPath(path, cwd) + if (target.startsWith(`${NULL_PATH}/`)) throw filesystemError('ENOTDIR', syscall, path) + return target + } + const readPath = (path: string, syscall: string): string => { + const target = checkedPath(path, syscall) + if (!readable.some(root => contains(root, target))) deny(syscall, path) + return target + } + const writePath = (path: string, syscall: string): string => { + const target = checkedPath(path, syscall) + if (!readWrite.some(root => contains(root, target))) deny(syscall, path) + return target + } + + return { + stat: async (path: string): Promise => { + const target = readPath(path, 'stat') + if (target === NULL_PATH) return NULL_STATS + if (target === DEV_ROOT && !await base.stat(target)) return { directory: true, size: 0, mtimeMs: 0 } + return await base.stat(target) + }, + list: async (path: string): Promise => { + const target = readPath(path, 'scandir') + if (target === DEV_ROOT) return [{ name: 'null', directory: false }] + if (target === NULL_PATH) throw filesystemError('ENOTDIR', 'scandir', path) + return await base.list(target) + }, + readText: async (path: string): Promise => { + const target = readPath(path, 'open') + return target === NULL_PATH ? '' : await base.readText(target) + }, + writeText: async (path: string, text: string, append = false): Promise => { + const target = writePath(path, 'open') + if (target !== NULL_PATH) await base.writeText(target, text, append) + }, + mkdir: async (path: string, recursive: boolean): Promise => { + const target = writePath(path, 'mkdir') + if (target === NULL_PATH) throw filesystemError('EEXIST', 'mkdir', path) + await base.mkdir(target, recursive) + }, + remove: async (path: string, options: { recursive: boolean; force: boolean }): Promise => { + const target = writePath(path, 'rm') + if (target === NULL_PATH) deny('rm', path) + await base.remove(target, options) + }, + rename: async (from: string, to: string): Promise => { + const source = writePath(from, 'rename') + const destination = writePath(to, 'rename') + if (source === NULL_PATH || destination === NULL_PATH) deny('rename', source === NULL_PATH ? from : to) + await base.rename(source, destination) + }, + } +} + +/** Virtual executable implementing the native launcher's CLI over VFS grants. */ +export const LANDLOCK_EXECUTABLE: VirtualExecutable = { + name: 'landlock-run', + async prepare(args, context) { + try { + const invocation = parseLandlockArguments(args) + if (invocation.kind === 'probe') return launcherExit(0, 'landlock: fully enforced\n') + return { + kind: 'delegate', + argv: invocation.argv, + filesystem: await landlockFileSystem(context.filesystem, invocation, context.cwd), + missingExecutable: launcherExit(125, '', 'landlock-run: exec failed: No such file or directory\n'), + } + } catch (error) { + return launcherFailure(error) + } + }, + runSync(args) { + try { + const invocation = parseLandlockArguments(args) + return invocation.kind === 'probe' + ? launcherExit(0, 'landlock: fully enforced\n') + : { kind: 'asynchronous' } + } catch (error) { + return launcherFailure(error) + } + }, +} diff --git a/packages/experimental/webworker-runtime/src/shell/process/virtual-executables.ts b/packages/experimental/webworker-runtime/src/shell/process/virtual-executables.ts new file mode 100644 index 0000000000..6f04cf1865 --- /dev/null +++ b/packages/experimental/webworker-runtime/src/shell/process/virtual-executables.ts @@ -0,0 +1,61 @@ +/** Virtual executable registry used by the Worker process launcher. */ +import { basename } from '../../module-system/posix-path.ts' +import type { ShellFileSystem } from '../types.ts' +import { LANDLOCK_EXECUTABLE } from './landlock.ts' + +/** Completed virtual executable invocation. */ +export interface VirtualExecutableExit { + readonly kind: 'exit' + readonly exitCode: number + readonly stdout: string + readonly stderr: string +} + +/** Invocation delegated to the normal Worker command runner after preparation. */ +export interface VirtualExecutableDelegate { + readonly kind: 'delegate' + readonly argv: readonly string[] + readonly filesystem: ShellFileSystem + readonly missingExecutable: VirtualExecutableExit +} + +/** Result of preparing an asynchronous virtual executable invocation. */ +export type VirtualExecutablePreparation = VirtualExecutableExit | VirtualExecutableDelegate + +/** Result available to the synchronous child-process face. */ +export type VirtualExecutableSyncResult = VirtualExecutableExit | { readonly kind: 'asynchronous' } + +/** One executable implemented by the Worker instead of an operating-system binary. */ +export interface VirtualExecutable { + /** Platform executable name, independent of package-manager installation path. */ + readonly name: string + /** + * Prepare an invocation or complete it without entering the command runner. + * @param args - Arguments after the executable path. + * @param context - Working directory and ambient Worker filesystem. + * @returns The completed result or delegated command and filesystem. + */ + prepare( + args: readonly string[], + context: { readonly cwd: string; readonly filesystem: ShellFileSystem }, + ): Promise + /** + * Handle the subset that can complete synchronously. + * @param args - Arguments after the executable path. + * @returns A completed result or the asynchronous marker. + */ + runSync(args: readonly string[]): VirtualExecutableSyncResult +} + +const EXECUTABLES: ReadonlyMap = new Map([ + [LANDLOCK_EXECUTABLE.name, LANDLOCK_EXECUTABLE], +]) + +/** + * Resolve a Worker platform executable by logical name. + * @param path - Bare name or executable path passed to `spawn`. + * @returns Its implementation, or undefined for the normal command table. + */ +export function virtualExecutable(path: string): VirtualExecutable | undefined { + return EXECUTABLES.get(basename(path)) +} diff --git a/packages/experimental/webworker-runtime/src/storage/active.ts b/packages/experimental/webworker-runtime/src/storage/active.ts index ce54337b21..74cfc63b41 100644 --- a/packages/experimental/webworker-runtime/src/storage/active.ts +++ b/packages/experimental/webworker-runtime/src/storage/active.ts @@ -4,15 +4,15 @@ * which backend the worker entry mounted. * @module @deepseek-ai/dsh-experimental-webworker-runtime/src/storage/active */ -import type { MemoryVfs } from './memory.ts' +import type { Vfs } from './types.ts' -let active: MemoryVfs | undefined +let active: Vfs | undefined /** * Publish the filesystem the `node:fs` proxy reads. * @param vfs - Filesystem mounted by the worker entry. */ -export function setActiveVfs(vfs: MemoryVfs): void { +export function setActiveVfs(vfs: Vfs): void { active = vfs } @@ -20,7 +20,7 @@ export function setActiveVfs(vfs: MemoryVfs): void { * Read the mounted filesystem. * @returns The active filesystem. */ -export function requireActiveVfs(): MemoryVfs { +export function requireActiveVfs(): Vfs { if (active === undefined) { throw new Error('webworker vfs: no filesystem is mounted; the worker entry must call setActiveVfs before any node:fs access') } diff --git a/packages/experimental/webworker-runtime/src/storage/memory.ts b/packages/experimental/webworker-runtime/src/storage/memory.ts index f4a544c001..24d12a688e 100644 --- a/packages/experimental/webworker-runtime/src/storage/memory.ts +++ b/packages/experimental/webworker-runtime/src/storage/memory.ts @@ -1,14 +1,15 @@ /** * In-memory filesystem behind the worker's `node:fs` proxy. Contents come from - * the build-time image (see {@link loadVfsImage}); writes stay in memory and - * vanish with the worker. + * the build-time image (see {@link loadVfsImage}); this remains the synchronous + * authority when an asynchronous durable sink mirrors selected subtrees. * @module @deepseek-ai/dsh-experimental-webworker-runtime/src/storage/memory */ import { dirname, join, normalize, resolve, SEP } from '../module-system/posix-path.ts' +import { IMAGE_OVERLAY_DIRECTORIES } from '../image-layout.ts' import { parseTar } from './tar.ts' import type { - VfsBigIntStats, VfsDir, VfsDirent, VfsEncoding, VfsError, VfsFileHandle, VfsReadOptions, VfsStatOptions, - VfsStats, VfsWriteOptions, + Vfs, VfsBigIntStats, VfsDir, VfsDirent, VfsEncoding, VfsError, VfsFileHandle, VfsMutation, VfsOpenFile, + VfsMutationListener, VfsMutationSink, VfsReadOptions, VfsSeedOptions, VfsStatOptions, VfsStats, VfsWriteOptions, } from './types.ts' const decoder = new TextDecoder() @@ -19,6 +20,10 @@ interface FileNode { mtimeMs: number /** Permission bits (`0o777` mask), set at creation and changed only by `chmod`. */ mode: number + /** Stable identity shared by hard links and retained by open descriptors. */ + identity?: bigint + /** One path normally, a Set only for hard links, or undefined after the final unlink. */ + paths: string | Set | undefined } /** Creation default for files, Node's `0o666` under the classic `022` umask. */ @@ -46,10 +51,14 @@ function encodingOf(options: VfsReadOptions): VfsEncoding | undefined { // stored value — the round-trip consumers like dsh-credentials-local's // owner-only check rely on. The bits are never enforced: a single-owner // filesystem reads and writes as its owner regardless, like root. -function statsOf(size: number, mtimeMs: number, directory: boolean, mode: number): VfsStats { +function statsOf(size: number, mtimeMs: number, directory: boolean, ino: bigint, mode: number): VfsStats { return { size, + ino: Number(ino), mtimeMs, + ctimeMs: mtimeMs, + atimeMs: mtimeMs, + birthtimeMs: mtimeMs, mtime: new Date(mtimeMs), mode: (directory ? 0o040000 : 0o100000) | (mode & 0o777), isFile: () => !directory, @@ -75,7 +84,14 @@ function statsOf(size: number, mtimeMs: number, directory: boolean, mode: number * @param mode - Stored permission bits of the entry. * @returns Stats in the shape Node returns under `{ bigint: true }`. */ -function bigIntStatsOf(size: number, mtimeMs: number, directory: boolean, ino: bigint, mode: number): VfsBigIntStats { +function bigIntStatsOf( + size: number, + mtimeMs: number, + directory: boolean, + ino: bigint, + mode: number, + nlink = 1, +): VfsBigIntStats { const milliseconds = BigInt(Math.trunc(mtimeMs)) const nanoseconds = milliseconds * 1_000_000n const time = new Date(mtimeMs) @@ -84,7 +100,7 @@ function bigIntStatsOf(size: number, mtimeMs: number, directory: boolean, ino: b mode: BigInt((directory ? 0o040000 : 0o100000) | (mode & 0o777)), dev: 1n, ino, - nlink: 1n, + nlink: BigInt(nlink), mtimeMs: milliseconds, mtimeNs: nanoseconds, ctimeMs: milliseconds, @@ -107,23 +123,116 @@ function bigIntStatsOf(size: number, mtimeMs: number, directory: boolean, ino: b } } +interface OpenMode { + readonly readable: boolean + readonly writable: boolean + readonly append: boolean + readonly create: boolean + readonly truncate: boolean + readonly exclusive: boolean +} + +/** Parse the Node string flags supported by the compatibility filesystem. */ +function openMode(flags: string): OpenMode { + const base = flags[0] + const suffix = flags.slice(1).split('') + const validSuffix = suffix.every(flag => flag === '+' || flag === 'x' || flag === 's') + const uniqueSuffix = new Set(suffix).size === suffix.length + if ((base !== 'r' && base !== 'w' && base !== 'a') || !validSuffix || !uniqueSuffix + || base === 'r' && flags.includes('x')) { + const error = new TypeError(`The argument 'flags' is invalid. Received '${flags}'`) as TypeError & { code: string } + error.code = 'ERR_INVALID_ARG_VALUE' + throw error + } + return { + readable: base === 'r' || flags.includes('+'), + writable: base !== 'r' || flags.includes('+'), + append: base === 'a', + create: base === 'w' || base === 'a', + truncate: base === 'w', + exclusive: flags.includes('x'), + } +} + +/** Resize bytes exactly, preserving the prefix and zero-filling growth. */ +function resize(bytes: Uint8Array, length: number): Uint8Array { + if (!Number.isSafeInteger(length) || length < 0) { + const error = new RangeError(`The value of "len" is out of range. It must be >= 0. Received ${String(length)}`) as RangeError & { code: string } + error.code = 'ERR_OUT_OF_RANGE' + throw error + } + const resized = new Uint8Array(length) + resized.set(bytes.subarray(0, length)) + return resized +} + +/** Construction inputs for {@link MemoryVfs}. */ +export interface MemoryVfsOptions { + /** Durable write-behind observer; absent leaves the filesystem ephemeral. */ + readonly sink?: VfsMutationSink +} + /** * Filesystem held in two maps: one for file bytes, one for directories. * Every path is normalized to an absolute POSIX path without a trailing * separator, so callers may pass either form. */ -export class MemoryVfs { +export class MemoryVfs implements Vfs { private readonly files = new Map() private readonly directories = new Set([SEP]) /** Directory permission bits; absence means {@link DEFAULT_DIRECTORY_MODE}. */ private readonly directoryModes = new Map() + /** Directory mtimes advance when their immediate entry set changes. */ + private readonly directoryMtimes = new Map() + private readonly mutationListeners = new Set() + private readonly sink: VfsMutationSink | undefined private temporaries = 0 - // Identity per path, assigned on first stat and dropped when the path goes: - // the filesystem service builds its version token from `ino` plus the - // timestamp, so a recreated path must not look like the entry it replaced. + // Directories retain path identities. File identities live on FileNode so + // descriptors, renames, and hard links continue to address the same file. private readonly identities = new Map() private lastIdentity = 0n + /** + * Build the synchronous filesystem authority. + * @param options - Optional durable write-behind sink. + */ + constructor(options: MemoryVfsOptions = {}) { + this.sink = options.sink + } + + /** + * Settle the durable sink without changing in-memory success. + * @returns A promise that resolves when all recorded mutations are stored. + */ + async flush(): Promise { + await this.sink?.flush() + } + + /** + * Observe committed runtime mutations. Image seeding is deliberately silent. + * @param listener - Consumer called after each successful mutation. + * @returns A disposer that prevents future calls. + */ + subscribe(listener: VfsMutationListener): () => void { + this.mutationListeners.add(listener) + return () => { this.mutationListeners.delete(listener) } + } + + /** Publish after state changes; one faulty observer cannot roll back a write. */ + private publish(mutation: VfsMutation): void { + const observers: VfsMutationListener[] = [ + ...(this.sink === undefined ? [] : [(change: VfsMutation): void => { this.sink?.record(change) }]), + ...this.mutationListeners, + ] + for (const listener of observers) { + try { + listener(mutation) + } catch (error) { + console.error('webworker vfs: mutation observer failed', error) + } + } + } + /** Promise face mirroring `node:fs/promises` for the methods the roster uses. */ readonly promises = { readFile: async (path: string, options?: VfsReadOptions): Promise => this.readFileSync(path, options), @@ -198,11 +307,12 @@ export class MemoryVfs { const [size, mtimeMs, directory, mode] = node !== undefined ? [node.bytes.length, node.mtimeMs, false, node.mode] as const : this.directories.has(target) - ? [0, 0, true, this.directoryModes.get(target) ?? DEFAULT_DIRECTORY_MODE] as const + ? [0, this.directoryMtimes.get(target) ?? 0, true, this.directoryModes.get(target) ?? DEFAULT_DIRECTORY_MODE] as const : fail('ENOENT', 'stat', target) + const identity = node === undefined ? this.identityOf(target) : this.identityOfFile(node) return options?.bigint === true - ? bigIntStatsOf(size, mtimeMs, directory, this.identityOf(target), mode) - : statsOf(size, mtimeMs, directory, mode) + ? bigIntStatsOf(size, mtimeMs, directory, identity, mode, node === undefined ? 1 : this.fileLinkCount(node)) + : statsOf(size, mtimeMs, directory, identity, mode) } /** @returns Stats in the plain shape, for internal callers that read `size`/`mtimeMs`. */ @@ -219,7 +329,109 @@ export class MemoryVfs { return this.lastIdentity } - /** Forget a removed path's identity, so a recreated path reports a new one. */ + /** @returns The inode-like identity retained by a file node across names. */ + private identityOfFile(node: FileNode): bigint { + if (node.identity !== undefined) return node.identity + this.lastIdentity += 1n + node.identity = this.lastIdentity + return node.identity + } + + /** @returns The number of names currently linked to one file node. */ + private fileLinkCount(node: FileNode): number { + return typeof node.paths === 'string' ? 1 : node.paths?.size ?? 0 + } + + /** Add one map name, promoting the rare hard-link case to a Set. */ + private addFilePath(node: FileNode, path: string): void { + if (node.paths === undefined) { + node.paths = path + } else if (typeof node.paths === 'string') { + node.paths = new Set([node.paths, path]) + } else { + node.paths.add(path) + } + } + + /** Remove one map name, collapsing a remaining single link back to a string. */ + private removeFilePath(node: FileNode, path: string): void { + if (typeof node.paths === 'string') { + node.paths = undefined + return + } + if (node.paths === undefined) return + node.paths.delete(path) + if (node.paths.size === 1) { + const [remaining] = node.paths + node.paths = remaining + } + } + + /** Set one file-map entry while maintaining both nodes' reverse path indexes. */ + private setFile(path: string, node: FileNode): void { + const previous = this.files.get(path) + if (previous === node) return + if (previous !== undefined) this.removeFilePath(previous, path) + this.files.set(path, node) + this.addFilePath(node, path) + } + + /** Delete one file-map entry while retaining an unlinked node held by a descriptor. */ + private deleteFile(path: string): FileNode | undefined { + const node = this.files.get(path) + if (node === undefined) return undefined + this.files.delete(path) + this.removeFilePath(node, path) + return node + } + + /** Publish one linked name after a content or metadata write. */ + private publishFilePath(node: FileNode, path: string, appendedFrom?: number): void { + this.publish({ + kind: 'write', path, bytes: node.bytes, mode: node.mode, entryChanged: false, + ...appendedFrom === undefined ? {} : { appendedFrom }, + }) + } + + /** Publish a content or metadata write for every hard link to one node. */ + private publishFile(node: FileNode, appendedFrom?: number): void { + if (typeof node.paths === 'string') { + this.publishFilePath(node, node.paths, appendedFrom) + return + } + if (node.paths === undefined) return + for (const path of node.paths) this.publishFilePath(node, path, appendedFrom) + } + + /** Replace bytes on one file identity and notify all linked paths. */ + private replaceFile(node: FileNode, bytes: Uint8Array, appendedFrom?: number): void { + node.bytes = bytes + node.mtimeMs = this.touchNode(node) + this.publishFile(node, appendedFrom) + } + + /** Write at one offset, zero-filling any gap. */ + private writeFileNode(node: FileNode, position: number, data: Uint8Array): number { + const offset = Math.max(0, position) + const previousLength = node.bytes.length + const bytes = new Uint8Array(Math.max(previousLength, offset + data.length)) + bytes.set(node.bytes) + bytes.set(data, offset) + this.replaceFile(node, bytes, offset === previousLength ? previousLength : undefined) + return data.length + } + + /** Resize one file identity and notify all linked paths. */ + private truncateFile(node: FileNode, length: number): void { + this.replaceFile(node, resize(node.bytes, length)) + } + + /** @returns Plain stats for an open file, including after its last name is removed. */ + private fileStats(node: FileNode): VfsStats { + return statsOf(node.bytes.length, node.mtimeMs, false, this.identityOfFile(node), node.mode) + } + + /** Forget removed directory identities, so recreated paths report new ones. */ private forgetIdentity(target: string): void { this.identities.delete(target) const prefix = `${target}${SEP}` @@ -239,11 +451,23 @@ export class MemoryVfs { * @returns Now, or one millisecond past the entry's current time. */ private touch(target: string): number { - const previous = this.files.get(target)?.mtimeMs + return this.touchNode(this.files.get(target)) + } + + /** @returns A modification time strictly newer than one file node's current value. */ + private touchNode(node?: FileNode): number { + const previous = node?.mtimeMs const now = Date.now() return previous === undefined ? now : Math.max(now, previous + 1) } + /** Advance a directory's mtime after its immediate children change. */ + private touchDirectory(target: string): void { + const previous = this.directoryMtimes.get(target) + const now = Date.now() + this.directoryMtimes.set(target, previous === undefined ? now : Math.max(now, previous + 1)) + } + /** * List a directory. * @param path - Directory path. @@ -312,7 +536,11 @@ export class MemoryVfs { this.mkdirSync(parent, options) } this.directories.add(target) - if (options?.mode !== undefined) this.directoryModes.set(target, options.mode & 0o777) + this.touchDirectory(target) + this.touchDirectory(parent) + const mode = (options?.mode ?? DEFAULT_DIRECTORY_MODE) & 0o777 + if (mode !== DEFAULT_DIRECTORY_MODE) this.directoryModes.set(target, mode) + this.publish({ kind: 'mkdir', path: target, mode }) return target } @@ -331,8 +559,17 @@ export class MemoryVfs { if (flag.startsWith('a')) { this.appendFileSync(target, data); return } // POSIX open(O_CREAT): the mode applies at creation only; a rewrite keeps // the entry's bits. - const mode = this.files.get(target)?.mode ?? (options?.mode !== undefined ? options.mode & 0o777 : DEFAULT_FILE_MODE) - this.files.set(target, { bytes: typeof data === 'string' ? encoder.encode(data) : data, mtimeMs: this.touch(target), mode }) + const previous = this.files.get(target) + const mode = previous?.mode ?? (options?.mode !== undefined ? options.mode & 0o777 : DEFAULT_FILE_MODE) + const bytes = typeof data === 'string' ? encoder.encode(data) : data + if (previous !== undefined) { + this.replaceFile(previous, bytes) + return + } + const node: FileNode = { bytes, mtimeMs: this.touch(target), mode, paths: undefined } + this.setFile(target, node) + this.touchDirectory(dirname(target)) + this.publish({ kind: 'write', path: target, bytes, mode, entryChanged: true }) } /** @@ -379,52 +616,97 @@ export class MemoryVfs { ...this.handleTail(target), } } - const exists = this.files.has(target) - if (flags.startsWith('r') && !exists) fail('ENOENT', 'open', target) - if (flags.startsWith('wx') && exists) fail('EEXIST', 'open', target) - if (!flags.startsWith('r') && !this.directories.has(dirname(target))) fail('ENOENT', 'open', target) - const creation = mode === undefined ? {} : { mode } - if (flags.startsWith('w') && !flags.startsWith('wx')) this.writeFileSync(target, new Uint8Array(), creation) - if (flags.startsWith('wx')) this.writeFileSync(target, new Uint8Array(), { flag: 'wx', ...creation }) - if (flags.startsWith('a') && !exists) this.writeFileSync(target, new Uint8Array(), creation) - const appending = flags.startsWith('a') + const file = this.openFileSync(target, flags, mode) + let position = 0 + let closed = false + const current = (syscall: string): VfsOpenFile => { + if (closed) fail('EBADF', syscall, target) + return file + } return { write: async (data: string | Uint8Array): Promise<{ bytesWritten: number }> => { const bytes = typeof data === 'string' ? encoder.encode(data) : data - this.appendFileSync(target, bytes) - return { bytesWritten: bytes.length } + const descriptor = current('write') + const offset = descriptor.append ? descriptor.stat().size : position + const bytesWritten = descriptor.write(offset, bytes) + position = offset + bytesWritten + return { bytesWritten } }, - // A handle opened for append must append here too: session persistence - // opens the log with `a` and writes each batch through this method, so a - // truncating write would replace the whole log with the newest batch. writeFile: async (data: string | Uint8Array): Promise => { - if (appending) this.appendFileSync(target, data) - else this.writeFileSync(target, data) + const bytes = typeof data === 'string' ? encoder.encode(data) : data + const descriptor = current('write') + const offset = descriptor.append ? descriptor.stat().size : position + position = offset + descriptor.write(offset, bytes) + }, + readFile: async (options?: VfsReadOptions): Promise => { + const descriptor = current('read') + const bytes = descriptor.read(position, Math.max(0, descriptor.stat().size - position)) + position += bytes.length + return encodingOf(options) === undefined ? bytes : decoder.decode(bytes) }, - readFile: async (options?: VfsReadOptions): Promise => this.readFileSync(target, options), truncate: async (length = 0): Promise => { - const node = this.files.get(target) - if (node === undefined) fail('ENOENT', 'ftruncate', target) - this.files.set(target, { bytes: node.bytes.slice(0, length), mtimeMs: this.touch(target), mode: node.mode }) + current('ftruncate').truncate(length) }, - ...this.handleTail(target), + stat: async (): Promise => current('fstat').stat(), + sync: async (): Promise => { current('fsync'); await this.flush() }, + datasync: async (): Promise => { current('fdatasync'); await this.flush() }, + close: async (): Promise => { closed = true }, } } /** - * The handle members that do not depend on how the file was opened. - * - * `sync`/`datasync` have nothing to flush — the bytes are already the stored - * ones — and `close` releases nothing, so both directory and file handles - * share this tail. + * Open one synchronous descriptor over a stable file identity. + * @param path - File path. + * @param flags - Node open flags. + * @param mode - Permission bits applied only when a file is created. + * @returns An open file that survives path rename, replacement, and unlink. + */ + openFileSync(path: string, flags = 'r', mode?: number): VfsOpenFile { + const target = this.key(path) + const access = openMode(flags) + const existing = this.files.get(target) + if (this.directories.has(target)) fail('EISDIR', 'open', target) + if (access.exclusive && existing !== undefined) fail('EEXIST', 'open', target) + if (!access.create && existing === undefined) fail('ENOENT', 'open', target) + if (access.create && existing === undefined) { + this.writeFileSync(target, new Uint8Array(), mode === undefined ? undefined : { mode }) + } else if (access.truncate && existing !== undefined) { + this.truncateFile(existing, 0) + } + const node = this.files.get(target) + if (node === undefined) fail('ENOENT', 'open', target) + return { + readable: access.readable, + writable: access.writable, + append: access.append, + read: (position, length) => { + if (!access.readable) fail('EBADF', 'read', target) + return node.bytes.subarray(position, position + length) + }, + write: (position, data) => { + if (!access.writable) fail('EBADF', 'write', target) + return this.writeFileNode(node, access.append ? node.bytes.length : position, data) + }, + truncate: (length) => { + if (!access.writable) fail('EINVAL', 'ftruncate', target) + this.truncateFile(node, length) + }, + stat: () => this.fileStats(node), + } + } + + /** + * Directory-handle members for metadata, durability, and release. + * `sync`/`datasync` settle an attached durable sink; an ephemeral filesystem + * resolves immediately and `close` releases nothing. * @param target - Normalized path the handle was opened on. * @returns Metadata plus the no-op durability and release calls. */ private handleTail(target: string): Pick { return { stat: async (): Promise => this.plainStats(target), - sync: async (): Promise => {}, - datasync: async (): Promise => {}, + sync: async (): Promise => { await this.flush() }, + datasync: async (): Promise => { await this.flush() }, close: async (): Promise => {}, } } @@ -439,10 +721,7 @@ export class MemoryVfs { const existing = this.files.get(target) const addition = typeof data === 'string' ? encoder.encode(data) : data if (existing === undefined) { this.writeFileSync(target, addition); return } - const merged = new Uint8Array(existing.bytes.length + addition.length) - merged.set(existing.bytes) - merged.set(addition, existing.bytes.length) - this.files.set(target, { bytes: merged, mtimeMs: this.touch(target), mode: existing.mode }) + this.writeFileNode(existing, existing.bytes.length, addition) } /** @@ -453,22 +732,41 @@ export class MemoryVfs { renameSync(from: string, to: string): void { const source = this.key(from) const destination = this.key(to) + if (source === destination) return const node = this.files.get(source) if (node !== undefined) { + if (this.directories.has(destination)) fail('EISDIR', 'rename', destination) if (!this.directories.has(dirname(destination))) fail('ENOENT', 'rename', destination) - this.files.delete(source) - this.files.set(destination, node) + if (this.files.get(destination) === node) return + this.deleteFile(source) + this.setFile(destination, node) this.forgetIdentity(source) this.forgetIdentity(destination) + this.touchDirectory(dirname(source)) + this.touchDirectory(dirname(destination)) + this.publish({ kind: 'remove', path: source }) + this.publish({ kind: 'write', path: destination, bytes: node.bytes, mode: node.mode, entryChanged: true }) return } if (!this.directories.has(source)) fail('ENOENT', 'rename', source) + if (this.files.has(destination)) fail('ENOTDIR', 'rename', destination) + if (!this.directories.has(dirname(destination))) fail('ENOENT', 'rename', destination) + if (this.directories.has(destination)) { + if (this.readdirSync(destination).length > 0) fail('ENOTEMPTY', 'rename', destination) + this.directories.delete(destination) + this.directoryModes.delete(destination) + this.directoryMtimes.delete(destination) + } const prefix = `${source}${SEP}` + const movedFiles: Array<{ path: string; bytes: Uint8Array; mode: number }> = [] for (const [candidate, value] of [...this.files]) { if (!candidate.startsWith(prefix)) continue - this.files.delete(candidate) - this.files.set(join(destination, candidate.slice(prefix.length)), value) + this.deleteFile(candidate) + const target = join(destination, candidate.slice(prefix.length)) + this.setFile(target, value) + movedFiles.push({ path: target, bytes: value.bytes, mode: value.mode }) } + const movedDirectories: Array<{ path: string; mode: number }> = [] for (const candidate of [...this.directories]) { if (!candidate.startsWith(prefix) && candidate !== source) continue const moved = candidate === source ? destination : join(destination, candidate.slice(prefix.length)) @@ -477,17 +775,31 @@ export class MemoryVfs { const bits = this.directoryModes.get(candidate) this.directoryModes.delete(candidate) if (bits !== undefined) this.directoryModes.set(moved, bits) + movedDirectories.push({ path: moved, mode: bits ?? DEFAULT_DIRECTORY_MODE }) + const mtime = this.directoryMtimes.get(candidate) + this.directoryMtimes.delete(candidate) + if (mtime !== undefined) this.directoryMtimes.set(moved, mtime) } this.forgetIdentity(source) this.forgetIdentity(destination) + this.touchDirectory(dirname(source)) + this.touchDirectory(dirname(destination)) + this.publish({ kind: 'remove', path: source }) + for (const directory of movedDirectories) { + this.publish({ kind: 'mkdir', path: directory.path, mode: directory.mode }) + } + for (const entry of movedFiles) { + this.publish({ + kind: 'write', path: entry.path, bytes: entry.bytes, mode: entry.mode, entryChanged: true, + }) + } } /** * Give existing bytes a second name. * - * There are no inodes here, so the two names share the bytes present at link - * time and diverge on the next write through either name; session persistence - * links a finished file to a stable name, which this satisfies. + * Both names retain one file identity, so writes and metadata changes through + * either name remain visible through the other until that name is removed. * @param existing - Source file path. * @param next - Additional path; its parent must exist and it must be free. */ @@ -498,7 +810,9 @@ export class MemoryVfs { if (node === undefined) fail('ENOENT', 'link', source) if (this.files.has(target) || this.directories.has(target)) fail('EEXIST', 'link', target) if (!this.directories.has(dirname(target))) fail('ENOENT', 'link', target) - this.files.set(target, node) + this.setFile(target, node) + this.touchDirectory(dirname(target)) + this.publish({ kind: 'write', path: target, bytes: node.bytes, mode: node.mode, entryChanged: true }) } /** @@ -510,7 +824,7 @@ export class MemoryVfs { const target = this.key(path) const node = this.files.get(target) if (node === undefined) fail('ENOENT', 'truncate', target) - this.files.set(target, { bytes: node.bytes.slice(0, length), mtimeMs: this.touch(target), mode: node.mode }) + this.truncateFile(node, length) } /** @@ -523,10 +837,17 @@ export class MemoryVfs { const node = this.files.get(target) if (node !== undefined) { node.mode = mode & 0o777 + if (typeof node.paths === 'string') { + this.publish({ kind: 'chmod', path: node.paths, mode: node.mode }) + } else if (node.paths !== undefined) { + for (const path of node.paths) this.publish({ kind: 'chmod', path, mode: node.mode }) + } return } if (this.directories.has(target)) { - this.directoryModes.set(target, mode & 0o777) + const bits = mode & 0o777 + this.directoryModes.set(target, bits) + this.publish({ kind: 'chmod', path: target, mode: bits }) return } fail('ENOENT', 'chmod', target) @@ -538,8 +859,10 @@ export class MemoryVfs { */ unlinkSync(path: string): void { const target = this.key(path) - if (!this.files.delete(target)) fail('ENOENT', 'unlink', target) + if (this.deleteFile(target) === undefined) fail('ENOENT', 'unlink', target) this.forgetIdentity(target) + this.touchDirectory(dirname(target)) + this.publish({ kind: 'remove', path: target }) } /** @@ -549,22 +872,28 @@ export class MemoryVfs { */ rmSync(path: string, options?: { recursive?: boolean; force?: boolean }): void { const target = this.key(path) - if (this.files.delete(target)) { + if (this.deleteFile(target) !== undefined) { this.forgetIdentity(target) + this.touchDirectory(dirname(target)) + this.publish({ kind: 'remove', path: target }) return } if (this.directories.has(target)) { if (options?.recursive !== true) fail('ERR_FS_EISDIR', 'rm', target) const prefix = `${target}${SEP}` - for (const candidate of [...this.files.keys()]) if (candidate.startsWith(prefix)) this.files.delete(candidate) + for (const candidate of [...this.files.keys()]) if (candidate.startsWith(prefix)) this.deleteFile(candidate) for (const candidate of [...this.directories]) { if (!candidate.startsWith(prefix)) continue this.directories.delete(candidate) this.directoryModes.delete(candidate) + this.directoryMtimes.delete(candidate) } this.directories.delete(target) this.directoryModes.delete(target) + this.directoryMtimes.delete(target) this.forgetIdentity(target) + this.touchDirectory(dirname(target)) + this.publish({ kind: 'remove', path: target }) return } if (options?.force !== true) fail('ENOENT', 'rm', target) @@ -586,23 +915,37 @@ export class MemoryVfs { * Seed a file and its parent directories, for image loading and tests. * @param path - File path. * @param data - Text or bytes. - * @param mode - Permission bits recorded for the entry. + * @param options - Permission bits and modification time supplied by the image or durable store. */ - seed(path: string, data: string | Uint8Array, mode = DEFAULT_FILE_MODE): void { + seed(path: string, data: string | Uint8Array, options: VfsSeedOptions = {}): void { const target = this.key(path) - this.mkdirSync(dirname(target), { recursive: true }) - this.files.set(target, { bytes: typeof data === 'string' ? encoder.encode(data) : data, mtimeMs: this.touch(target), mode: mode & 0o777 }) + this.seedDirectory(dirname(target)) + this.setFile(target, { + bytes: typeof data === 'string' ? encoder.encode(data) : data, + mtimeMs: options.mtimeMs ?? this.touch(target), + mode: (options.mode ?? DEFAULT_FILE_MODE) & 0o777, + paths: undefined, + }) + this.touchDirectory(dirname(target)) } /** * Create a directory and its parents. * @param path - Directory path. - * @param mode - Permission bits recorded for the directory itself. + * @param options - Permission bits and modification time supplied by the image or durable store. */ - seedDirectory(path: string, mode = DEFAULT_DIRECTORY_MODE): void { + seedDirectory(path: string, options: VfsSeedOptions = {}): void { const target = this.key(path) - this.mkdirSync(target, { recursive: true }) - if (mode !== DEFAULT_DIRECTORY_MODE) this.directoryModes.set(target, mode & 0o777) + if (!this.directories.has(target)) { + const parent = dirname(target) + if (parent !== target) this.seedDirectory(parent) + if (this.files.has(target)) fail('EEXIST', 'mkdir', target) + this.directories.add(target) + this.directoryMtimes.set(target, options.mtimeMs ?? Date.now()) + this.touchDirectory(parent) + } + if (options.mode !== undefined) this.directoryModes.set(target, options.mode & 0o777) + if (options.mtimeMs !== undefined) this.directoryMtimes.set(target, options.mtimeMs) } /** @@ -636,10 +979,45 @@ export function loadVfsImage(image: Uint8Array, root = '/dsh', vfs = new MemoryV } const target = join(root, relativeName) if (entry.directory) { - vfs.seedDirectory(target, entry.mode) + vfs.seedDirectory(target, { mode: entry.mode }) continue } - vfs.seed(target, entry.bytes, entry.mode) + vfs.seed(target, entry.bytes, { mode: entry.mode }) + } + return vfs +} + +/** + * Apply one ordered data overlay to an already mounted base image. + * + * Overlay entries may replace files only under the layout's data directories; + * module code, configuration, and the lowering manifest cannot be shadowed. + * Paths containing traversal segments are refused before normalization. Later + * overlays win for files, while file/directory type conflicts fail loud. + * @param image - Uncompressed ustar overlay archive. + * @param root - Virtual root shared with the base image. + * @param vfs - Mounted filesystem to update. + * @returns The same filesystem after applying the overlay. + */ +export function loadVfsOverlay(image: Uint8Array, root: string, vfs: MemoryVfs): MemoryVfs { + for (const entry of parseTar(image)) { + const relativeName = entry.name.startsWith('./') ? entry.name.slice(2) : entry.name + const path = relativeName.endsWith('/') ? relativeName.slice(0, -1) : relativeName + const segments = path.split('/') + if (path === '' || relativeName.startsWith(SEP) + || segments.some(segment => segment === '' || segment === '.' || segment === '..') + || !IMAGE_OVERLAY_DIRECTORIES.includes(segments[0] ?? '')) { + throw new Error(`webworker vfs: overlay entry must stay under ${IMAGE_OVERLAY_DIRECTORIES.join('/ or ')}, received "${entry.name}"`) + } + const target = join(root, path) + if (entry.directory) { + vfs.seedDirectory(target, { mode: entry.mode }) + continue + } + if (vfs.existsSync(target) && vfs.statSync(target).isDirectory()) { + throw new Error(`webworker vfs: overlay file cannot replace directory "${target}"`) + } + vfs.seed(target, entry.bytes, { mode: entry.mode }) } return vfs } diff --git a/packages/experimental/webworker-runtime/src/storage/types.ts b/packages/experimental/webworker-runtime/src/storage/types.ts index 9208761407..f1482e07d6 100644 --- a/packages/experimental/webworker-runtime/src/storage/types.ts +++ b/packages/experimental/webworker-runtime/src/storage/types.ts @@ -1,8 +1,8 @@ /** * Filesystem interfaces shared by every VFS backend. The shipped implementation - * is in memory; a browser-persistent backend would implement the same faces. Errors carry - * Node's `code` values because roster plugins branch on them (`ENOENT` for - * optional files, `EACCES` for read-only trees). + * is in memory; browser persistence hydrates it and consumes its committed + * mutation stream. Errors carry Node's `code` values because roster plugins + * branch on them (`ENOENT` for optional files, `EACCES` for read-only trees). * @module @deepseek-ai/dsh-experimental-webworker-runtime/src/storage/types */ @@ -22,7 +22,12 @@ export interface VfsError extends Error { /** Subset of `fs.Stats` the roster reads. */ export interface VfsStats { readonly size: number + /** Stable file identity across rename and hard links; recreation receives another value. */ + readonly ino: number readonly mtimeMs: number + readonly ctimeMs: number + readonly atimeMs: number + readonly birthtimeMs: number readonly mtime: Date readonly mode: number isFile(): boolean @@ -49,7 +54,7 @@ export interface VfsBigIntStats { readonly mode: bigint /** One virtual device holds the whole image. */ readonly dev: bigint - /** Identity of the entry at this path; a removed and recreated path gets a new one. */ + /** File identity retained across rename and hard links; recreation gets a new one. */ readonly ino: bigint readonly nlink: bigint readonly mtimeMs: bigint @@ -85,6 +90,12 @@ export interface VfsWriteOptions { readonly flag?: string } +/** Explicit metadata for image or durable-store hydration. */ +export interface VfsSeedOptions { + readonly mode?: number + readonly mtimeMs?: number +} + /** Directory entry as `readdir` with `withFileTypes` reports it. */ export interface VfsDirent { readonly name: string @@ -113,3 +124,126 @@ export interface VfsFileHandle { datasync(): Promise close(): Promise } + +/** Open-file identity used by synchronous Node-style descriptors. */ +export interface VfsOpenFile { + /** Whether reads are allowed by the flags used at open time. */ + readonly readable: boolean + /** Whether writes and truncation are allowed by the flags used at open time. */ + readonly writable: boolean + /** Whether each write targets the current end of the opened file. */ + readonly append: boolean + /** + * Read bytes from the opened file identity. + * @param position - Absolute byte offset. + * @param length - Maximum byte count. + * @returns A view of the available bytes. + */ + read(position: number, length: number): Uint8Array + /** + * Write bytes to the opened file identity. + * @param position - Absolute byte offset, ignored for append descriptors. + * @param data - Bytes to write. + * @returns Number of bytes written. + */ + write(position: number, data: Uint8Array): number + /** + * Resize the opened file, zero-filling growth. + * @param length - Target byte length. + */ + truncate(length: number): void + /** + * Read metadata from the opened file identity. + * @returns Current file metadata, including after rename or unlink. + */ + stat(): VfsStats +} + +/** + * One completed change to the authoritative in-memory filesystem. + * + * A durable mirror receives the post-write bytes, virtual permission bits, and optional append offset; + * live watchers use `entryChanged` to distinguish directory-entry replacement + * from content writes. Rename is represented as source removal plus complete + * destination mkdir/write records, so a sink never receives a path without the + * state needed to materialize it. + */ +export type VfsMutation = + | { + readonly kind: 'write' + readonly path: string + readonly bytes: Uint8Array + readonly mode: number + readonly entryChanged: boolean + readonly appendedFrom?: number + } + | { readonly kind: 'mkdir'; readonly path: string; readonly mode: number } + | { readonly kind: 'remove'; readonly path: string } + | { readonly kind: 'chmod'; readonly path: string; readonly mode: number } + +/** Receives one committed VFS mutation. */ +export type VfsMutationListener = (mutation: VfsMutation) => void + +/** Durable observer attached to the synchronous VFS. */ +export interface VfsMutationSink { + /** + * Record one completed mutation without delaying its caller. + * @param mutation - Post-commit state to mirror. + */ + record(mutation: VfsMutation): void + /** + * Settle all previously recorded mutations. + * Implementations report persistence failures and stop mirroring rather than + * rejecting, because the in-memory mutation has already committed. + * @returns A promise that resolves when the sink has no pending work. + */ + flush(): Promise +} + +/** Synchronous filesystem used by the worker's Node compatibility modules. */ +export interface Vfs { + readonly promises: { + readFile(path: string, options?: VfsReadOptions): Promise + writeFile(path: string, data: string | Uint8Array, options?: VfsWriteOptions): Promise + appendFile(path: string, data: string | Uint8Array): Promise + mkdir(path: string, options?: { recursive?: boolean; mode?: number }): Promise + readdir(path: string, options?: { withFileTypes?: boolean }): Promise + stat(path: string, options?: VfsStatOptions): Promise + lstat(path: string, options?: VfsStatOptions): Promise + realpath(path: string): Promise + rename(from: string, to: string): Promise + unlink(path: string): Promise + rm(path: string, options?: { recursive?: boolean; force?: boolean }): Promise + mkdtemp(prefix: string): Promise + link(existing: string, next: string): Promise + truncate(path: string, length?: number): Promise + chmod(path: string, mode: number): Promise + opendir(path: string): Promise + open(path: string, flags?: string, mode?: number): Promise + access(path: string): Promise + } + readFileSync(path: string, options?: VfsReadOptions): string | Uint8Array + existsSync(path: string): boolean + statSync(path: string, options?: VfsStatOptions): VfsStats | VfsBigIntStats + readdirSync(path: string, options?: { withFileTypes?: boolean }): string[] & VfsDirent[] + realpathSync(path: string): string + mkdirSync(path: string, options?: { recursive?: boolean; mode?: number }): string | undefined + writeFileSync(path: string, data: string | Uint8Array, options?: VfsWriteOptions): void + appendFileSync(path: string, data: string | Uint8Array): void + renameSync(from: string, to: string): void + linkSync(existing: string, next: string): void + truncateSync(path: string, length?: number): void + chmodSync(path: string, mode: number): void + unlinkSync(path: string): void + rmSync(path: string, options?: { recursive?: boolean; force?: boolean }): void + mkdtempSync(prefix: string): string + /** Open and retain one file identity until its Node descriptor closes. */ + openFileSync(path: string, flags?: string, mode?: number): VfsOpenFile + seed(path: string, data: string | Uint8Array, options?: VfsSeedOptions): void + seedDirectory(path: string, options?: VfsSeedOptions): void + usage(): { files: number; directories: number; bytes: number } + /** Register one observer and return its synchronous disposer. */ + subscribe(listener: VfsMutationListener): () => void + /** Settle the attached durable mutation sink, if any. */ + flush(): Promise +} diff --git a/packages/experimental/webworker-runtime/src/transport/frames.ts b/packages/experimental/webworker-runtime/src/transport/frames.ts index 67d89b73fd..a04a2cec5a 100644 --- a/packages/experimental/webworker-runtime/src/transport/frames.ts +++ b/packages/experimental/webworker-runtime/src/transport/frames.ts @@ -33,12 +33,13 @@ export interface TunnelAbortFrame { /** Frames the worker accepts. */ /** - * First inbound frame: the image URL, the one input the worker assembly - * takes from outside. + * First inbound frame: the base image URL and ordered data overlays selected + * before the worker assembly starts. */ export interface TunnelInitFrame { readonly t: 'init' readonly image: string + readonly overlays: readonly string[] } /** Every frame the page sends the worker. */ @@ -142,7 +143,10 @@ export function parseInboundFrame(data: unknown): TunnelInboundFrame { if (typeof frame.image !== 'string') { throw new Error('webworker tunnel: init frame needs a string image url') } - return { t: 'init', image: frame.image } + if (!Array.isArray(frame.overlays) || frame.overlays.some(overlay => typeof overlay !== 'string')) { + throw new Error('webworker tunnel: init frame needs an array of string overlay urls') + } + return { t: 'init', image: frame.image, overlays: frame.overlays as string[] } } const id = frame.id if (typeof id !== 'string' && typeof id !== 'number') { diff --git a/packages/experimental/webworker-runtime/src/worker-host.ts b/packages/experimental/webworker-runtime/src/worker-host.ts index f73f1d2e5c..bbbf6a3900 100644 --- a/packages/experimental/webworker-runtime/src/worker-host.ts +++ b/packages/experimental/webworker-runtime/src/worker-host.ts @@ -29,7 +29,7 @@ import { installProcessGlobal } from './node/globals/process.ts' import type { RequestListener } from './transport/synthetic-http.ts' import { TunnelServer, type TunnelPort } from './transport/tunnel.ts' import { inflateImage, inflateImageStream } from './storage/image-gzip.ts' -import { loadVfsImage, MemoryVfs } from './storage/memory.ts' +import { loadVfsImage, loadVfsOverlay, MemoryVfs } from './storage/memory.ts' import { setActiveVfs } from './storage/active.ts' import { DEFAULT_ROOT, IMAGE_CONFIG_PATH, IMAGE_EMPTY_DIRECTORIES, IMAGE_HOME_DIRECTORY, IMAGE_MANIFEST_PATH, @@ -87,6 +87,8 @@ export interface WorkerHostOptions { readonly requestListener: () => Promise /** Image bytes, or the URL the worker fetches them from. */ readonly image: Uint8Array | string + /** Ordered data overlays applied after the base image and before boot. */ + readonly overlays?: readonly (Uint8Array | string)[] /** Virtual root; defaults to {@link DEFAULT_ROOT}. */ readonly root?: string /** Composed configuration inside the image; defaults to `/config/cordis.yml`. */ @@ -182,8 +184,12 @@ export function createWorkerHost(options: WorkerHostOptions): WorkerHost { const home = join(root, IMAGE_HOME_DIRECTORY) installProcessGlobal({ cwd: root, env: { DSH_HOME: home, HOME: home, ...options.env } }) - const bytes = await readImage(options.image) + const [bytes, overlays] = await Promise.all([ + readImage(options.image), + Promise.all((options.overlays ?? []).map(readImage)), + ]) const mounted = loadVfsImage(bytes, root) + for (const overlay of overlays) loadVfsOverlay(overlay, root, mounted) // Belt and braces over the image's own empty-directory entries: a hand // -built image without them still boots. for (const directory of IMAGE_EMPTY_DIRECTORIES) { @@ -248,7 +254,7 @@ export function createWorkerHost(options: WorkerHostOptions): WorkerHost { const shared = ctx.get('connection') !== undefined const handler = directFetchHandler(ctx, toFetchHandler(apiProxy)) const usage = loader.usage() - console.info(`webworker host: tree active (modules=${String(usage.modules)}, preset root overlay=${presetOverlay ? 'applied' : 'already in roster'}, direct lane=${shared ? 'connection.createSharedFetchHandler (interceptors kept)' : 'api surface only'}, als causality=${options.alsCausality === undefined ? 'inert' : 'snapshot/restore'}, image lowering=${LOWERING_VERSION})`) + console.info(`webworker host: tree active (modules=${String(usage.modules)}, data overlays=${String(overlays.length)}, preset root overlay=${presetOverlay ? 'applied' : 'already in roster'}, direct lane=${shared ? 'connection.createSharedFetchHandler (interceptors kept)' : 'api surface only'}, als causality=${options.alsCausality === undefined ? 'inert' : 'snapshot/restore'}, image lowering=${LOWERING_VERSION})`) tunnel.serve({ directFetch: (request: Request) => handler.fetch(request), diff --git a/packages/experimental/webworker-runtime/src/worker.ts b/packages/experimental/webworker-runtime/src/worker.ts index f82c5fd1fb..f0912aef93 100644 --- a/packages/experimental/webworker-runtime/src/worker.ts +++ b/packages/experimental/webworker-runtime/src/worker.ts @@ -4,9 +4,9 @@ * listener; the assembly owns everything else (process global, VFS image, * Cordis tree, tunnel server). * - * The assembly needs the image location before it can exist, and it arrives in - * the tunnel's opening `init` frame — this bundle reads nothing from its own - * URL, so the deployment decides where both the bundle and the image live. + * The assembly needs the base image and selected overlays before it can exist; + * they arrive in the tunnel's opening `init` frame. This bundle reads nothing + * from its own URL, so the deployment decides where every archive lives. * Messages before `init` queue here; requests during boot queue inside the * host, which attaches its handler before its first await. */ @@ -52,12 +52,16 @@ self.addEventListener('message', (event: MessageEvent) => { if (typeof data.image !== 'string') { throw new Error('webworker: init frame needs a string image url') } + if (!Array.isArray(data.overlays) || data.overlays.some(overlay => typeof overlay !== 'string')) { + throw new Error('webworker: init frame needs an array of string overlay urls') + } const created = createWorkerHost({ staticModules: createNodeBuiltins(), staticModulePrefixes: REPLACED_PREFIXES, requestListener: whenRequestListener, alsCausality, image: data.image, + overlays: data.overlays as string[], }) host = created for (const queued of pending) { diff --git a/packages/experimental/webworker-runtime/tests/client/source-chooser.spec.ts b/packages/experimental/webworker-runtime/tests/client/source-chooser.spec.ts new file mode 100644 index 0000000000..f636c32ebe --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/client/source-chooser.spec.ts @@ -0,0 +1,158 @@ +// @vitest-environment jsdom + +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import type { PreviewFixtureManifest } from '../../src/fixture-manifest.ts' +import { choosePreviewSource } from '../../src/client/source-chooser.ts' + +const MANIFEST_URL = new URL('https://preview.test/preview/fixtures.json') + +const MANIFEST: PreviewFixtureManifest = { + version: 1, + defaultFixture: 'example', + fixtures: [{ + id: 'example', + label: 'Example & "quoted" \'single\'', + description: 'A deterministic example.', + overlays: ['fixtures/base.tar.gz', 'fixtures/tail.tar.gz'], + }], +} + +function setLocation(search = ''): void { + history.replaceState({}, '', `/preview.html${search}`) +} + +function installManifest(manifest: PreviewFixtureManifest = MANIFEST): ReturnType { + const fetch = vi.fn(async () => Response.json(manifest)) + vi.stubGlobal('fetch', fetch) + return fetch +} + +function submitChooser(): void { + const form = document.querySelector('[data-preview-source-card]') + if (form === null) throw new Error('test chooser form was not rendered') + form.dispatchEvent(new Event('submit', { bubbles: true, cancelable: true })) +} + +describe('Preview source chooser', () => { + beforeEach(() => { + document.head.replaceChildren() + document.body.innerHTML = '
' + setLocation() + }) + + afterEach(() => { + vi.restoreAllMocks() + vi.unstubAllGlobals() + }) + + it('bypasses the chooser and manifest for an explicit empty source', async () => { + setLocation('?preview-fixture=none') + const fetch = installManifest() + + await expect(choosePreviewSource(MANIFEST_URL)).resolves.toEqual([]) + expect(fetch).not.toHaveBeenCalled() + expect(document.querySelector('[data-preview-source-chooser]')).toBeNull() + }) + + it('bypasses the chooser and resolves an explicit built-in fixture', async () => { + document.body.replaceChildren() + setLocation('?preview-fixture=example') + const fetch = installManifest() + + await expect(choosePreviewSource(MANIFEST_URL)).resolves.toEqual([ + new URL('https://preview.test/preview/fixtures/base.tar.gz'), + new URL('https://preview.test/preview/fixtures/tail.tar.gz'), + ]) + expect(fetch).toHaveBeenCalledOnce() + expect(document.querySelector('[data-preview-source-chooser]')).toBeNull() + }) + + it.each(['', 'missing', 'webfs'])( + 'fails loud for the explicit unavailable source %j without opening the chooser', + async (source) => { + setLocation(`?preview-fixture=${source}`) + installManifest() + + await expect(choosePreviewSource(MANIFEST_URL)).rejects.toThrow(/unknown or interactive source/) + expect(document.querySelector('[data-preview-source-chooser]')).toBeNull() + }, + ) + + it('shows the chooser only when the query is absent and returns its default selection', async () => { + installManifest() + const root = document.getElementById('root') + if (root === null) throw new Error('test root is missing') + const bootPage = document.createElement('div') + bootPage.dataset.dshBoot = '' + root.append(bootPage) + + const selected = choosePreviewSource(MANIFEST_URL) + await vi.waitFor(() => { + expect(document.querySelector('[data-preview-source-chooser]')).not.toBeNull() + }) + expect(document.querySelector('input[value="example"]')?.checked).toBe(true) + expect(document.querySelector('input[value="webfs"]')?.disabled).toBe(true) + expect(document.querySelector('[data-preview-source-card]')?.textContent) + .toContain(MANIFEST.fixtures[0]?.label) + expect(document.querySelector('[data-preview-source-card] script')).toBeNull() + + submitChooser() + + await expect(selected).resolves.toEqual([ + new URL('https://preview.test/preview/fixtures/base.tar.gz'), + new URL('https://preview.test/preview/fixtures/tail.tar.gz'), + ]) + expect(root.contains(bootPage)).toBe(true) + expect(root.childElementCount).toBe(1) + expect(document.querySelector('[data-preview-source-chooser]')).toBeNull() + expect(document.querySelector('[data-preview-source-style]')).toBeNull() + }) + + it('selects the empty source when the manifest has no default', async () => { + installManifest({ ...MANIFEST, defaultFixture: null }) + + const selected = choosePreviewSource(MANIFEST_URL) + await vi.waitFor(() => { + expect(document.querySelector('input[value="none"]')?.checked).toBe(true) + }) + submitChooser() + + await expect(selected).resolves.toEqual([]) + }) + + it('reports manifest, mount, form, selection, and catalog failures', async () => { + vi.stubGlobal('fetch', vi.fn(async () => new Response('missing', { status: 404 }))) + await expect(choosePreviewSource(MANIFEST_URL)).rejects.toThrow(/returned 404/) + + installManifest() + document.body.replaceChildren() + await expect(choosePreviewSource(MANIFEST_URL)).rejects.toThrow(/missing #root/) + + document.body.innerHTML = '
' + const root = document.getElementById('root') + if (root === null) throw new Error('test root is missing') + const querySelector = vi.spyOn(HTMLElement.prototype, 'querySelector').mockReturnValueOnce(null) + await expect(choosePreviewSource(MANIFEST_URL)).rejects.toThrow(/form was not rendered/) + querySelector.mockRestore() + + document.head.replaceChildren() + document.body.innerHTML = '
' + const missingSelection = choosePreviewSource(MANIFEST_URL) + await vi.waitFor(() => { expect(document.querySelector('form')).not.toBeNull() }) + document.querySelectorAll('input[name="preview-source"]').forEach((input) => { + input.removeAttribute('name') + }) + submitChooser() + await expect(missingSelection).rejects.toThrow(/no source selected/) + + document.head.replaceChildren() + document.body.innerHTML = '
' + const unavailableSelection = choosePreviewSource(MANIFEST_URL) + await vi.waitFor(() => { expect(document.querySelector('form')).not.toBeNull() }) + const selected = document.querySelector('input:checked') + if (selected === null) throw new Error('test selection is missing') + selected.value = 'missing' + submitChooser() + await expect(unavailableSelection).rejects.toThrow(/unavailable source/) + }) +}) diff --git a/packages/experimental/webworker-runtime/tests/fixture-manifest.spec.ts b/packages/experimental/webworker-runtime/tests/fixture-manifest.spec.ts new file mode 100644 index 0000000000..01b3d00e9a --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/fixture-manifest.spec.ts @@ -0,0 +1,48 @@ +import { describe, expect, it } from 'vitest' +import { + parsePreviewFixtureManifest, PREVIEW_FIXTURE_MANIFEST_VERSION, +} from '../src/fixture-manifest.ts' + +describe('Preview fixture manifest', () => { + it('accepts a unique named fixture with ordered overlays', () => { + expect(parsePreviewFixtureManifest({ + version: PREVIEW_FIXTURE_MANIFEST_VERSION, + defaultFixture: 'example', + fixtures: [{ + id: 'example', + label: 'Example', + description: 'A deterministic example.', + overlays: ['fixtures/base.tar.gz', 'fixtures/tail.tar.gz'], + }], + })).toEqual({ + version: PREVIEW_FIXTURE_MANIFEST_VERSION, + defaultFixture: 'example', + fixtures: [{ + id: 'example', + label: 'Example', + description: 'A deterministic example.', + overlays: ['fixtures/base.tar.gz', 'fixtures/tail.tar.gz'], + }], + }) + }) + + it.each([ + [{ version: 2, defaultFixture: null, fixtures: [] }, /must use version/], + [{ version: 1, defaultFixture: 'missing', fixtures: [] }, /defaultFixture/], + [{ + version: 1, + defaultFixture: 'duplicate', + fixtures: [ + { id: 'duplicate', label: 'One', description: 'First.', overlays: ['one.tar.gz'] }, + { id: 'duplicate', label: 'Two', description: 'Second.', overlays: ['two.tar.gz'] }, + ], + }, /repeats id/], + [{ + version: 1, + defaultFixture: 'none', + fixtures: [{ id: 'none', label: 'None', description: 'Reserved.', overlays: ['none.tar.gz'] }], + }, /invalid fixture entry/], + ])('rejects malformed catalogs', (value, error) => { + expect(() => parsePreviewFixtureManifest(value)).toThrow(error) + }) +}) diff --git a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-architecture-review/session.jsonl b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-architecture-review/session.jsonl new file mode 100644 index 0000000000..291a18ba1e --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-architecture-review/session.jsonl @@ -0,0 +1,178 @@ +{"type":"session","version":0,"id":"preview-architecture-review","createdAt":1787472100000,"cwd":"/dsh/workspace","parentSession":"preview-showcase","seedLength":169,"origin":"subagent","delegationDepth":1,"agentPreset":"standard"} +{"type":"turn/start","data":{"turn":1},"seq":0,"time":1787472000000} +{"type":"user/message","data":{"id":"preview-user-01","role":"user","content":[{"type":"text","text":"History checkpoint 01: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":1,"time":1787472000001} +{"type":"session/title","data":{"title":"WebWorker Preview Showcase","messageSeqs":[],"source":{"kind":"user"}},"seq":2,"time":1787472000002} +{"type":"step/start","data":{"turn":1,"step":1},"seq":3,"time":1787472000003} +{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"id":"preview-assistant-01","role":"assistant","content":[{"type":"text","text":"Checkpoint 01 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":4,"time":1787472000004} +{"type":"step/end","data":{"turn":1,"step":1},"seq":5,"time":1787472000005} +{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}},"seq":6,"time":1787472000006} +{"type":"turn/start","data":{"turn":2},"seq":7,"time":1787472000007} +{"type":"user/message","data":{"id":"preview-user-02","role":"user","content":[{"type":"text","text":"History checkpoint 02: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":8,"time":1787472000008} +{"type":"step/start","data":{"turn":2,"step":1},"seq":9,"time":1787472000009} +{"type":"assistant/message","data":{"turn":2,"step":1,"message":{"id":"preview-assistant-02","role":"assistant","content":[{"type":"text","text":"Checkpoint 02 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":10,"time":1787472000010} +{"type":"step/end","data":{"turn":2,"step":1},"seq":11,"time":1787472000011} +{"type":"turn/end","data":{"turn":2,"reason":{"kind":"completed"}},"seq":12,"time":1787472000012} +{"type":"turn/start","data":{"turn":3},"seq":13,"time":1787472000013} +{"type":"user/message","data":{"id":"preview-user-03","role":"user","content":[{"type":"text","text":"History checkpoint 03: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":14,"time":1787472000014} +{"type":"step/start","data":{"turn":3,"step":1},"seq":15,"time":1787472000015} +{"type":"assistant/message","data":{"turn":3,"step":1,"message":{"id":"preview-assistant-03","role":"assistant","content":[{"type":"text","text":"Checkpoint 03 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":16,"time":1787472000016} +{"type":"step/end","data":{"turn":3,"step":1},"seq":17,"time":1787472000017} +{"type":"turn/end","data":{"turn":3,"reason":{"kind":"completed"}},"seq":18,"time":1787472000018} +{"type":"turn/start","data":{"turn":4},"seq":19,"time":1787472000019} +{"type":"user/message","data":{"id":"preview-user-04","role":"user","content":[{"type":"text","text":"History checkpoint 04: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":20,"time":1787472000020} +{"type":"step/start","data":{"turn":4,"step":1},"seq":21,"time":1787472000021} +{"type":"assistant/message","data":{"turn":4,"step":1,"message":{"id":"preview-assistant-04","role":"assistant","content":[{"type":"text","text":"Checkpoint 04 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":22,"time":1787472000022} +{"type":"step/end","data":{"turn":4,"step":1},"seq":23,"time":1787472000023} +{"type":"turn/end","data":{"turn":4,"reason":{"kind":"completed"}},"seq":24,"time":1787472000024} +{"type":"turn/start","data":{"turn":5},"seq":25,"time":1787472000025} +{"type":"user/message","data":{"id":"preview-user-05","role":"user","content":[{"type":"text","text":"History checkpoint 05: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":26,"time":1787472000026} +{"type":"step/start","data":{"turn":5,"step":1},"seq":27,"time":1787472000027} +{"type":"assistant/message","data":{"turn":5,"step":1,"message":{"id":"preview-assistant-05","role":"assistant","content":[{"type":"text","text":"Checkpoint 05 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":28,"time":1787472000028} +{"type":"step/end","data":{"turn":5,"step":1},"seq":29,"time":1787472000029} +{"type":"turn/end","data":{"turn":5,"reason":{"kind":"completed"}},"seq":30,"time":1787472000030} +{"type":"turn/start","data":{"turn":6},"seq":31,"time":1787472000031} +{"type":"user/message","data":{"id":"preview-user-06","role":"user","content":[{"type":"text","text":"History checkpoint 06: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":32,"time":1787472000032} +{"type":"step/start","data":{"turn":6,"step":1},"seq":33,"time":1787472000033} +{"type":"assistant/message","data":{"turn":6,"step":1,"message":{"id":"preview-assistant-06","role":"assistant","content":[{"type":"text","text":"Checkpoint 06 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":34,"time":1787472000034} +{"type":"step/end","data":{"turn":6,"step":1},"seq":35,"time":1787472000035} +{"type":"turn/end","data":{"turn":6,"reason":{"kind":"completed"}},"seq":36,"time":1787472000036} +{"type":"turn/start","data":{"turn":7},"seq":37,"time":1787472000037} +{"type":"user/message","data":{"id":"preview-user-07","role":"user","content":[{"type":"text","text":"History checkpoint 07: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":38,"time":1787472000038} +{"type":"step/start","data":{"turn":7,"step":1},"seq":39,"time":1787472000039} +{"type":"assistant/message","data":{"turn":7,"step":1,"message":{"id":"preview-assistant-07","role":"assistant","content":[{"type":"text","text":"Checkpoint 07 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":40,"time":1787472000040} +{"type":"step/end","data":{"turn":7,"step":1},"seq":41,"time":1787472000041} +{"type":"turn/end","data":{"turn":7,"reason":{"kind":"completed"}},"seq":42,"time":1787472000042} +{"type":"turn/start","data":{"turn":8},"seq":43,"time":1787472000043} +{"type":"user/message","data":{"id":"preview-user-08","role":"user","content":[{"type":"text","text":"History checkpoint 08: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":44,"time":1787472000044} +{"type":"step/start","data":{"turn":8,"step":1},"seq":45,"time":1787472000045} +{"type":"assistant/message","data":{"turn":8,"step":1,"message":{"id":"preview-assistant-08","role":"assistant","content":[{"type":"text","text":"Checkpoint 08 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":46,"time":1787472000046} +{"type":"step/end","data":{"turn":8,"step":1},"seq":47,"time":1787472000047} +{"type":"turn/end","data":{"turn":8,"reason":{"kind":"completed"}},"seq":48,"time":1787472000048} +{"type":"turn/start","data":{"turn":9},"seq":49,"time":1787472000049} +{"type":"user/message","data":{"id":"preview-user-09","role":"user","content":[{"type":"text","text":"History checkpoint 09: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":50,"time":1787472000050} +{"type":"step/start","data":{"turn":9,"step":1},"seq":51,"time":1787472000051} +{"type":"assistant/message","data":{"turn":9,"step":1,"message":{"id":"preview-assistant-09","role":"assistant","content":[{"type":"text","text":"Checkpoint 09 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":52,"time":1787472000052} +{"type":"step/end","data":{"turn":9,"step":1},"seq":53,"time":1787472000053} +{"type":"turn/end","data":{"turn":9,"reason":{"kind":"completed"}},"seq":54,"time":1787472000054} +{"type":"turn/start","data":{"turn":10},"seq":55,"time":1787472000055} +{"type":"user/message","data":{"id":"preview-user-10","role":"user","content":[{"type":"text","text":"History checkpoint 10: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":56,"time":1787472000056} +{"type":"step/start","data":{"turn":10,"step":1},"seq":57,"time":1787472000057} +{"type":"assistant/message","data":{"turn":10,"step":1,"message":{"id":"preview-assistant-10","role":"assistant","content":[{"type":"text","text":"Checkpoint 10 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":58,"time":1787472000058} +{"type":"step/end","data":{"turn":10,"step":1},"seq":59,"time":1787472000059} +{"type":"turn/end","data":{"turn":10,"reason":{"kind":"completed"}},"seq":60,"time":1787472000060} +{"type":"turn/start","data":{"turn":11},"seq":61,"time":1787472000061} +{"type":"user/message","data":{"id":"preview-user-11","role":"user","content":[{"type":"text","text":"History checkpoint 11: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":62,"time":1787472000062} +{"type":"step/start","data":{"turn":11,"step":1},"seq":63,"time":1787472000063} +{"type":"assistant/message","data":{"turn":11,"step":1,"message":{"id":"preview-assistant-11","role":"assistant","content":[{"type":"text","text":"Checkpoint 11 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":64,"time":1787472000064} +{"type":"step/end","data":{"turn":11,"step":1},"seq":65,"time":1787472000065} +{"type":"turn/end","data":{"turn":11,"reason":{"kind":"completed"}},"seq":66,"time":1787472000066} +{"type":"turn/start","data":{"turn":12},"seq":67,"time":1787472000067} +{"type":"user/message","data":{"id":"preview-user-12","role":"user","content":[{"type":"text","text":"History checkpoint 12: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":68,"time":1787472000068} +{"type":"step/start","data":{"turn":12,"step":1},"seq":69,"time":1787472000069} +{"type":"assistant/message","data":{"turn":12,"step":1,"message":{"id":"preview-assistant-12","role":"assistant","content":[{"type":"text","text":"Checkpoint 12 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":70,"time":1787472000070} +{"type":"step/end","data":{"turn":12,"step":1},"seq":71,"time":1787472000071} +{"type":"turn/end","data":{"turn":12,"reason":{"kind":"completed"}},"seq":72,"time":1787472000072} +{"type":"turn/start","data":{"turn":13},"seq":73,"time":1787472000073} +{"type":"user/message","data":{"id":"preview-user-13","role":"user","content":[{"type":"text","text":"History checkpoint 13: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":74,"time":1787472000074} +{"type":"step/start","data":{"turn":13,"step":1},"seq":75,"time":1787472000075} +{"type":"assistant/message","data":{"turn":13,"step":1,"message":{"id":"preview-assistant-13","role":"assistant","content":[{"type":"text","text":"Checkpoint 13 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":76,"time":1787472000076} +{"type":"step/end","data":{"turn":13,"step":1},"seq":77,"time":1787472000077} +{"type":"turn/end","data":{"turn":13,"reason":{"kind":"completed"}},"seq":78,"time":1787472000078} +{"type":"turn/start","data":{"turn":14},"seq":79,"time":1787472000079} +{"type":"user/message","data":{"id":"preview-user-14","role":"user","content":[{"type":"text","text":"History checkpoint 14: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":80,"time":1787472000080} +{"type":"step/start","data":{"turn":14,"step":1},"seq":81,"time":1787472000081} +{"type":"assistant/message","data":{"turn":14,"step":1,"message":{"id":"preview-assistant-14","role":"assistant","content":[{"type":"text","text":"Checkpoint 14 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":82,"time":1787472000082} +{"type":"step/end","data":{"turn":14,"step":1},"seq":83,"time":1787472000083} +{"type":"turn/end","data":{"turn":14,"reason":{"kind":"completed"}},"seq":84,"time":1787472000084} +{"type":"turn/start","data":{"turn":15},"seq":85,"time":1787472000085} +{"type":"user/message","data":{"id":"preview-user-15","role":"user","content":[{"type":"text","text":"History checkpoint 15: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":86,"time":1787472000086} +{"type":"step/start","data":{"turn":15,"step":1},"seq":87,"time":1787472000087} +{"type":"assistant/message","data":{"turn":15,"step":1,"message":{"id":"preview-assistant-15","role":"assistant","content":[{"type":"text","text":"Checkpoint 15 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":88,"time":1787472000088} +{"type":"step/end","data":{"turn":15,"step":1},"seq":89,"time":1787472000089} +{"type":"turn/end","data":{"turn":15,"reason":{"kind":"completed"}},"seq":90,"time":1787472000090} +{"type":"turn/start","data":{"turn":16},"seq":91,"time":1787472000091} +{"type":"user/message","data":{"id":"preview-user-16","role":"user","content":[{"type":"text","text":"History checkpoint 16: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":92,"time":1787472000092} +{"type":"step/start","data":{"turn":16,"step":1},"seq":93,"time":1787472000093} +{"type":"assistant/message","data":{"turn":16,"step":1,"message":{"id":"preview-assistant-16","role":"assistant","content":[{"type":"text","text":"Checkpoint 16 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":94,"time":1787472000094} +{"type":"step/end","data":{"turn":16,"step":1},"seq":95,"time":1787472000095} +{"type":"turn/end","data":{"turn":16,"reason":{"kind":"completed"}},"seq":96,"time":1787472000096} +{"type":"turn/start","data":{"turn":17},"seq":97,"time":1787472000097} +{"type":"user/message","data":{"id":"preview-user-17","role":"user","content":[{"type":"text","text":"History checkpoint 17: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":98,"time":1787472000098} +{"type":"step/start","data":{"turn":17,"step":1},"seq":99,"time":1787472000099} +{"type":"assistant/message","data":{"turn":17,"step":1,"message":{"id":"preview-assistant-17","role":"assistant","content":[{"type":"text","text":"Checkpoint 17 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":100,"time":1787472000100} +{"type":"step/end","data":{"turn":17,"step":1},"seq":101,"time":1787472000101} +{"type":"turn/end","data":{"turn":17,"reason":{"kind":"completed"}},"seq":102,"time":1787472000102} +{"type":"turn/start","data":{"turn":18},"seq":103,"time":1787472000103} +{"type":"user/message","data":{"id":"preview-user-18","role":"user","content":[{"type":"text","text":"History checkpoint 18: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":104,"time":1787472000104} +{"type":"step/start","data":{"turn":18,"step":1},"seq":105,"time":1787472000105} +{"type":"assistant/message","data":{"turn":18,"step":1,"message":{"id":"preview-assistant-18","role":"assistant","content":[{"type":"text","text":"Checkpoint 18 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":106,"time":1787472000106} +{"type":"step/end","data":{"turn":18,"step":1},"seq":107,"time":1787472000107} +{"type":"turn/end","data":{"turn":18,"reason":{"kind":"completed"}},"seq":108,"time":1787472000108} +{"type":"turn/start","data":{"turn":19},"seq":109,"time":1787472000109} +{"type":"user/message","data":{"id":"preview-user-19","role":"user","content":[{"type":"text","text":"History checkpoint 19: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":110,"time":1787472000110} +{"type":"step/start","data":{"turn":19,"step":1},"seq":111,"time":1787472000111} +{"type":"assistant/message","data":{"turn":19,"step":1,"message":{"id":"preview-assistant-19","role":"assistant","content":[{"type":"text","text":"Checkpoint 19 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":112,"time":1787472000112} +{"type":"step/end","data":{"turn":19,"step":1},"seq":113,"time":1787472000113} +{"type":"turn/end","data":{"turn":19,"reason":{"kind":"completed"}},"seq":114,"time":1787472000114} +{"type":"turn/start","data":{"turn":20},"seq":115,"time":1787472000115} +{"type":"user/message","data":{"id":"preview-user-20","role":"user","content":[{"type":"text","text":"History checkpoint 20: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":116,"time":1787472000116} +{"type":"step/start","data":{"turn":20,"step":1},"seq":117,"time":1787472000117} +{"type":"assistant/message","data":{"turn":20,"step":1,"message":{"id":"preview-assistant-20","role":"assistant","content":[{"type":"text","text":"Checkpoint 20 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":118,"time":1787472000118} +{"type":"step/end","data":{"turn":20,"step":1},"seq":119,"time":1787472000119} +{"type":"turn/end","data":{"turn":20,"reason":{"kind":"completed"}},"seq":120,"time":1787472000120} +{"type":"turn/start","data":{"turn":21},"seq":121,"time":1787472000121} +{"type":"user/message","data":{"id":"preview-user-21","role":"user","content":[{"type":"text","text":"History checkpoint 21: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":122,"time":1787472000122} +{"type":"step/start","data":{"turn":21,"step":1},"seq":123,"time":1787472000123} +{"type":"assistant/message","data":{"turn":21,"step":1,"message":{"id":"preview-assistant-21","role":"assistant","content":[{"type":"text","text":"Checkpoint 21 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":124,"time":1787472000124} +{"type":"step/end","data":{"turn":21,"step":1},"seq":125,"time":1787472000125} +{"type":"turn/end","data":{"turn":21,"reason":{"kind":"completed"}},"seq":126,"time":1787472000126} +{"type":"turn/start","data":{"turn":22},"seq":127,"time":1787472000127} +{"type":"user/message","data":{"id":"preview-user-22","role":"user","content":[{"type":"text","text":"History checkpoint 22: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":128,"time":1787472000128} +{"type":"step/start","data":{"turn":22,"step":1},"seq":129,"time":1787472000129} +{"type":"assistant/message","data":{"turn":22,"step":1,"message":{"id":"preview-assistant-22","role":"assistant","content":[{"type":"text","text":"Checkpoint 22 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":130,"time":1787472000130} +{"type":"step/end","data":{"turn":22,"step":1},"seq":131,"time":1787472000131} +{"type":"turn/end","data":{"turn":22,"reason":{"kind":"completed"}},"seq":132,"time":1787472000132} +{"type":"turn/start","data":{"turn":23},"seq":133,"time":1787472000133} +{"type":"user/message","data":{"id":"preview-user-23","role":"user","content":[{"type":"text","text":"History checkpoint 23: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":134,"time":1787472000134} +{"type":"step/start","data":{"turn":23,"step":1},"seq":135,"time":1787472000135} +{"type":"assistant/message","data":{"turn":23,"step":1,"message":{"id":"preview-assistant-23","role":"assistant","content":[{"type":"text","text":"Checkpoint 23 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":136,"time":1787472000136} +{"type":"step/end","data":{"turn":23,"step":1},"seq":137,"time":1787472000137} +{"type":"turn/end","data":{"turn":23,"reason":{"kind":"completed"}},"seq":138,"time":1787472000138} +{"type":"turn/start","data":{"turn":24},"seq":139,"time":1787472000139} +{"type":"user/message","data":{"id":"preview-user-24","role":"user","content":[{"type":"text","text":"History checkpoint 24: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":140,"time":1787472000140} +{"type":"step/start","data":{"turn":24,"step":1},"seq":141,"time":1787472000141} +{"type":"assistant/message","data":{"turn":24,"step":1,"message":{"id":"preview-assistant-24","role":"assistant","content":[{"type":"text","text":"Checkpoint 24 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":142,"time":1787472000142} +{"type":"step/end","data":{"turn":24,"step":1},"seq":143,"time":1787472000143} +{"type":"turn/end","data":{"turn":24,"reason":{"kind":"completed"}},"seq":144,"time":1787472000144} +{"type":"turn/start","data":{"turn":25},"seq":145,"time":1787472000145} +{"type":"user/message","data":{"id":"preview-user-25","role":"user","content":[{"type":"text","text":"History checkpoint 25: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":146,"time":1787472000146} +{"type":"step/start","data":{"turn":25,"step":1},"seq":147,"time":1787472000147} +{"type":"assistant/message","data":{"turn":25,"step":1,"message":{"id":"preview-assistant-25","role":"assistant","content":[{"type":"text","text":"Checkpoint 25 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":148,"time":1787472000148} +{"type":"step/end","data":{"turn":25,"step":1},"seq":149,"time":1787472000149} +{"type":"turn/end","data":{"turn":25,"reason":{"kind":"completed"}},"seq":150,"time":1787472000150} +{"type":"turn/start","data":{"turn":26},"seq":151,"time":1787472000151} +{"type":"user/message","data":{"id":"preview-user-26","role":"user","content":[{"type":"text","text":"History checkpoint 26: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":152,"time":1787472000152} +{"type":"step/start","data":{"turn":26,"step":1},"seq":153,"time":1787472000153} +{"type":"assistant/message","data":{"turn":26,"step":1,"message":{"id":"preview-assistant-26","role":"assistant","content":[{"type":"text","text":"Checkpoint 26 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":154,"time":1787472000154} +{"type":"step/end","data":{"turn":26,"step":1},"seq":155,"time":1787472000155} +{"type":"turn/end","data":{"turn":26,"reason":{"kind":"completed"}},"seq":156,"time":1787472000156} +{"type":"turn/start","data":{"turn":27},"seq":157,"time":1787472000157} +{"type":"user/message","data":{"id":"preview-user-27","role":"user","content":[{"type":"text","text":"History checkpoint 27: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":158,"time":1787472000158} +{"type":"step/start","data":{"turn":27,"step":1},"seq":159,"time":1787472000159} +{"type":"assistant/message","data":{"turn":27,"step":1,"message":{"id":"preview-assistant-27","role":"assistant","content":[{"type":"text","text":"Checkpoint 27 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":160,"time":1787472000160} +{"type":"step/end","data":{"turn":27,"step":1},"seq":161,"time":1787472000161} +{"type":"turn/end","data":{"turn":27,"reason":{"kind":"completed"}},"seq":162,"time":1787472000162} +{"type":"turn/start","data":{"turn":28},"seq":163,"time":1787472000163} +{"type":"user/message","data":{"id":"preview-user-28","role":"user","content":[{"type":"text","text":"History checkpoint 28: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":164,"time":1787472000164} +{"type":"step/start","data":{"turn":28,"step":1},"seq":165,"time":1787472000165} +{"type":"assistant/message","data":{"turn":28,"step":1,"message":{"id":"preview-assistant-28","role":"assistant","content":[{"type":"text","text":"Checkpoint 28 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":166,"time":1787472000166} +{"type":"step/end","data":{"turn":28,"step":1},"seq":167,"time":1787472000167} +{"type":"turn/end","data":{"turn":28,"reason":{"kind":"completed"}},"seq":168,"time":1787472000168} +{"type":"session/end-seed","data":{},"seq":169,"time":1787472100000} +{"type":"turn/start","data":{"turn":29},"seq":170,"time":1787472100001} +{"type":"user/message","data":{"id":"preview-review-user","role":"user","content":[{"type":"text","text":"Review whether the preview fixture is isolated from future WebFS data."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":171,"time":1787472100002} +{"type":"subagent/descriptor","data":{"version":2,"mode":"one-shot","provider":"fork","label":"Review preview architecture"},"seq":172,"time":1787472100003} +{"type":"step/start","data":{"turn":29,"step":1},"seq":173,"time":1787472100004} +{"type":"assistant/message","data":{"turn":29,"step":1,"message":{"id":"preview-review-assistant","role":"assistant","content":[{"type":"text","text":"The bundled fixture is static image content; future WebFS state remains user-owned."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":174,"time":1787472100005} +{"type":"step/end","data":{"turn":29,"step":1},"seq":175,"time":1787472100006} +{"type":"turn/end","data":{"turn":29,"reason":{"kind":"completed"}},"seq":176,"time":1787472100007} diff --git a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-follow-up-builder/session.jsonl b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-follow-up-builder/session.jsonl new file mode 100644 index 0000000000..6353e854c4 --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-follow-up-builder/session.jsonl @@ -0,0 +1,8 @@ +{"type":"session","version":0,"id":"preview-follow-up-builder","createdAt":1787472200000,"cwd":"/dsh/workspace","parentSession":"preview-showcase","origin":"subagent","delegationDepth":1,"agentPreset":"standard"} +{"type":"turn/start","data":{"turn":1},"seq":0,"time":1787472200000} +{"type":"user/message","data":{"id":"preview-builder-user","role":"user","content":[{"type":"text","text":"Check that the Preview workspace can support follow-up tasks."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":1,"time":1787472200001} +{"type":"subagent/descriptor","data":{"version":2,"mode":"continuable","provider":"spawn","label":"Continue preview verification"},"seq":2,"time":1787472200002} +{"type":"step/start","data":{"turn":1,"step":1},"seq":3,"time":1787472200003} +{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"id":"preview-builder-assistant","role":"assistant","content":[{"type":"text","text":"This child is continuable and ready for another verification turn."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":4,"time":1787472200004} +{"type":"step/end","data":{"turn":1,"step":1},"seq":5,"time":1787472200005} +{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}},"seq":6,"time":1787472200006} diff --git a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-showcase/session.jsonl b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-showcase/session.jsonl new file mode 100644 index 0000000000..7b15d63fb3 --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-showcase/session.jsonl @@ -0,0 +1,200 @@ +{"type":"session","version":0,"id":"preview-showcase","createdAt":1787472000000,"cwd":"/dsh/workspace","delegationDepth":0,"agentPreset":"standard"} +{"type":"turn/start","data":{"turn":1},"seq":0,"time":1787472000000} +{"type":"user/message","data":{"id":"preview-user-01","role":"user","content":[{"type":"text","text":"History checkpoint 01: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":1,"time":1787472000001} +{"type":"session/title","data":{"title":"WebWorker Preview Showcase","messageSeqs":[],"source":{"kind":"user"}},"seq":2,"time":1787472000002} +{"type":"step/start","data":{"turn":1,"step":1},"seq":3,"time":1787472000003} +{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"id":"preview-assistant-01","role":"assistant","content":[{"type":"text","text":"Checkpoint 01 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":4,"time":1787472000004} +{"type":"step/end","data":{"turn":1,"step":1},"seq":5,"time":1787472000005} +{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}},"seq":6,"time":1787472000006} +{"type":"turn/start","data":{"turn":2},"seq":7,"time":1787472000007} +{"type":"user/message","data":{"id":"preview-user-02","role":"user","content":[{"type":"text","text":"History checkpoint 02: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":8,"time":1787472000008} +{"type":"step/start","data":{"turn":2,"step":1},"seq":9,"time":1787472000009} +{"type":"assistant/message","data":{"turn":2,"step":1,"message":{"id":"preview-assistant-02","role":"assistant","content":[{"type":"text","text":"Checkpoint 02 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":10,"time":1787472000010} +{"type":"step/end","data":{"turn":2,"step":1},"seq":11,"time":1787472000011} +{"type":"turn/end","data":{"turn":2,"reason":{"kind":"completed"}},"seq":12,"time":1787472000012} +{"type":"turn/start","data":{"turn":3},"seq":13,"time":1787472000013} +{"type":"user/message","data":{"id":"preview-user-03","role":"user","content":[{"type":"text","text":"History checkpoint 03: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":14,"time":1787472000014} +{"type":"step/start","data":{"turn":3,"step":1},"seq":15,"time":1787472000015} +{"type":"assistant/message","data":{"turn":3,"step":1,"message":{"id":"preview-assistant-03","role":"assistant","content":[{"type":"text","text":"Checkpoint 03 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":16,"time":1787472000016} +{"type":"step/end","data":{"turn":3,"step":1},"seq":17,"time":1787472000017} +{"type":"turn/end","data":{"turn":3,"reason":{"kind":"completed"}},"seq":18,"time":1787472000018} +{"type":"turn/start","data":{"turn":4},"seq":19,"time":1787472000019} +{"type":"user/message","data":{"id":"preview-user-04","role":"user","content":[{"type":"text","text":"History checkpoint 04: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":20,"time":1787472000020} +{"type":"step/start","data":{"turn":4,"step":1},"seq":21,"time":1787472000021} +{"type":"assistant/message","data":{"turn":4,"step":1,"message":{"id":"preview-assistant-04","role":"assistant","content":[{"type":"text","text":"Checkpoint 04 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":22,"time":1787472000022} +{"type":"step/end","data":{"turn":4,"step":1},"seq":23,"time":1787472000023} +{"type":"turn/end","data":{"turn":4,"reason":{"kind":"completed"}},"seq":24,"time":1787472000024} +{"type":"turn/start","data":{"turn":5},"seq":25,"time":1787472000025} +{"type":"user/message","data":{"id":"preview-user-05","role":"user","content":[{"type":"text","text":"History checkpoint 05: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":26,"time":1787472000026} +{"type":"step/start","data":{"turn":5,"step":1},"seq":27,"time":1787472000027} +{"type":"assistant/message","data":{"turn":5,"step":1,"message":{"id":"preview-assistant-05","role":"assistant","content":[{"type":"text","text":"Checkpoint 05 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":28,"time":1787472000028} +{"type":"step/end","data":{"turn":5,"step":1},"seq":29,"time":1787472000029} +{"type":"turn/end","data":{"turn":5,"reason":{"kind":"completed"}},"seq":30,"time":1787472000030} +{"type":"turn/start","data":{"turn":6},"seq":31,"time":1787472000031} +{"type":"user/message","data":{"id":"preview-user-06","role":"user","content":[{"type":"text","text":"History checkpoint 06: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":32,"time":1787472000032} +{"type":"step/start","data":{"turn":6,"step":1},"seq":33,"time":1787472000033} +{"type":"assistant/message","data":{"turn":6,"step":1,"message":{"id":"preview-assistant-06","role":"assistant","content":[{"type":"text","text":"Checkpoint 06 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":34,"time":1787472000034} +{"type":"step/end","data":{"turn":6,"step":1},"seq":35,"time":1787472000035} +{"type":"turn/end","data":{"turn":6,"reason":{"kind":"completed"}},"seq":36,"time":1787472000036} +{"type":"turn/start","data":{"turn":7},"seq":37,"time":1787472000037} +{"type":"user/message","data":{"id":"preview-user-07","role":"user","content":[{"type":"text","text":"History checkpoint 07: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":38,"time":1787472000038} +{"type":"step/start","data":{"turn":7,"step":1},"seq":39,"time":1787472000039} +{"type":"assistant/message","data":{"turn":7,"step":1,"message":{"id":"preview-assistant-07","role":"assistant","content":[{"type":"text","text":"Checkpoint 07 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":40,"time":1787472000040} +{"type":"step/end","data":{"turn":7,"step":1},"seq":41,"time":1787472000041} +{"type":"turn/end","data":{"turn":7,"reason":{"kind":"completed"}},"seq":42,"time":1787472000042} +{"type":"turn/start","data":{"turn":8},"seq":43,"time":1787472000043} +{"type":"user/message","data":{"id":"preview-user-08","role":"user","content":[{"type":"text","text":"History checkpoint 08: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":44,"time":1787472000044} +{"type":"step/start","data":{"turn":8,"step":1},"seq":45,"time":1787472000045} +{"type":"assistant/message","data":{"turn":8,"step":1,"message":{"id":"preview-assistant-08","role":"assistant","content":[{"type":"text","text":"Checkpoint 08 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":46,"time":1787472000046} +{"type":"step/end","data":{"turn":8,"step":1},"seq":47,"time":1787472000047} +{"type":"turn/end","data":{"turn":8,"reason":{"kind":"completed"}},"seq":48,"time":1787472000048} +{"type":"turn/start","data":{"turn":9},"seq":49,"time":1787472000049} +{"type":"user/message","data":{"id":"preview-user-09","role":"user","content":[{"type":"text","text":"History checkpoint 09: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":50,"time":1787472000050} +{"type":"step/start","data":{"turn":9,"step":1},"seq":51,"time":1787472000051} +{"type":"assistant/message","data":{"turn":9,"step":1,"message":{"id":"preview-assistant-09","role":"assistant","content":[{"type":"text","text":"Checkpoint 09 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":52,"time":1787472000052} +{"type":"step/end","data":{"turn":9,"step":1},"seq":53,"time":1787472000053} +{"type":"turn/end","data":{"turn":9,"reason":{"kind":"completed"}},"seq":54,"time":1787472000054} +{"type":"turn/start","data":{"turn":10},"seq":55,"time":1787472000055} +{"type":"user/message","data":{"id":"preview-user-10","role":"user","content":[{"type":"text","text":"History checkpoint 10: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":56,"time":1787472000056} +{"type":"step/start","data":{"turn":10,"step":1},"seq":57,"time":1787472000057} +{"type":"assistant/message","data":{"turn":10,"step":1,"message":{"id":"preview-assistant-10","role":"assistant","content":[{"type":"text","text":"Checkpoint 10 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":58,"time":1787472000058} +{"type":"step/end","data":{"turn":10,"step":1},"seq":59,"time":1787472000059} +{"type":"turn/end","data":{"turn":10,"reason":{"kind":"completed"}},"seq":60,"time":1787472000060} +{"type":"turn/start","data":{"turn":11},"seq":61,"time":1787472000061} +{"type":"user/message","data":{"id":"preview-user-11","role":"user","content":[{"type":"text","text":"History checkpoint 11: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":62,"time":1787472000062} +{"type":"step/start","data":{"turn":11,"step":1},"seq":63,"time":1787472000063} +{"type":"assistant/message","data":{"turn":11,"step":1,"message":{"id":"preview-assistant-11","role":"assistant","content":[{"type":"text","text":"Checkpoint 11 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":64,"time":1787472000064} +{"type":"step/end","data":{"turn":11,"step":1},"seq":65,"time":1787472000065} +{"type":"turn/end","data":{"turn":11,"reason":{"kind":"completed"}},"seq":66,"time":1787472000066} +{"type":"turn/start","data":{"turn":12},"seq":67,"time":1787472000067} +{"type":"user/message","data":{"id":"preview-user-12","role":"user","content":[{"type":"text","text":"History checkpoint 12: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":68,"time":1787472000068} +{"type":"step/start","data":{"turn":12,"step":1},"seq":69,"time":1787472000069} +{"type":"assistant/message","data":{"turn":12,"step":1,"message":{"id":"preview-assistant-12","role":"assistant","content":[{"type":"text","text":"Checkpoint 12 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":70,"time":1787472000070} +{"type":"step/end","data":{"turn":12,"step":1},"seq":71,"time":1787472000071} +{"type":"turn/end","data":{"turn":12,"reason":{"kind":"completed"}},"seq":72,"time":1787472000072} +{"type":"turn/start","data":{"turn":13},"seq":73,"time":1787472000073} +{"type":"user/message","data":{"id":"preview-user-13","role":"user","content":[{"type":"text","text":"History checkpoint 13: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":74,"time":1787472000074} +{"type":"step/start","data":{"turn":13,"step":1},"seq":75,"time":1787472000075} +{"type":"assistant/message","data":{"turn":13,"step":1,"message":{"id":"preview-assistant-13","role":"assistant","content":[{"type":"text","text":"Checkpoint 13 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":76,"time":1787472000076} +{"type":"step/end","data":{"turn":13,"step":1},"seq":77,"time":1787472000077} +{"type":"turn/end","data":{"turn":13,"reason":{"kind":"completed"}},"seq":78,"time":1787472000078} +{"type":"turn/start","data":{"turn":14},"seq":79,"time":1787472000079} +{"type":"user/message","data":{"id":"preview-user-14","role":"user","content":[{"type":"text","text":"History checkpoint 14: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":80,"time":1787472000080} +{"type":"step/start","data":{"turn":14,"step":1},"seq":81,"time":1787472000081} +{"type":"assistant/message","data":{"turn":14,"step":1,"message":{"id":"preview-assistant-14","role":"assistant","content":[{"type":"text","text":"Checkpoint 14 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":82,"time":1787472000082} +{"type":"step/end","data":{"turn":14,"step":1},"seq":83,"time":1787472000083} +{"type":"turn/end","data":{"turn":14,"reason":{"kind":"completed"}},"seq":84,"time":1787472000084} +{"type":"turn/start","data":{"turn":15},"seq":85,"time":1787472000085} +{"type":"user/message","data":{"id":"preview-user-15","role":"user","content":[{"type":"text","text":"History checkpoint 15: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":86,"time":1787472000086} +{"type":"step/start","data":{"turn":15,"step":1},"seq":87,"time":1787472000087} +{"type":"assistant/message","data":{"turn":15,"step":1,"message":{"id":"preview-assistant-15","role":"assistant","content":[{"type":"text","text":"Checkpoint 15 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":88,"time":1787472000088} +{"type":"step/end","data":{"turn":15,"step":1},"seq":89,"time":1787472000089} +{"type":"turn/end","data":{"turn":15,"reason":{"kind":"completed"}},"seq":90,"time":1787472000090} +{"type":"turn/start","data":{"turn":16},"seq":91,"time":1787472000091} +{"type":"user/message","data":{"id":"preview-user-16","role":"user","content":[{"type":"text","text":"History checkpoint 16: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":92,"time":1787472000092} +{"type":"step/start","data":{"turn":16,"step":1},"seq":93,"time":1787472000093} +{"type":"assistant/message","data":{"turn":16,"step":1,"message":{"id":"preview-assistant-16","role":"assistant","content":[{"type":"text","text":"Checkpoint 16 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":94,"time":1787472000094} +{"type":"step/end","data":{"turn":16,"step":1},"seq":95,"time":1787472000095} +{"type":"turn/end","data":{"turn":16,"reason":{"kind":"completed"}},"seq":96,"time":1787472000096} +{"type":"turn/start","data":{"turn":17},"seq":97,"time":1787472000097} +{"type":"user/message","data":{"id":"preview-user-17","role":"user","content":[{"type":"text","text":"History checkpoint 17: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":98,"time":1787472000098} +{"type":"step/start","data":{"turn":17,"step":1},"seq":99,"time":1787472000099} +{"type":"assistant/message","data":{"turn":17,"step":1,"message":{"id":"preview-assistant-17","role":"assistant","content":[{"type":"text","text":"Checkpoint 17 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":100,"time":1787472000100} +{"type":"step/end","data":{"turn":17,"step":1},"seq":101,"time":1787472000101} +{"type":"turn/end","data":{"turn":17,"reason":{"kind":"completed"}},"seq":102,"time":1787472000102} +{"type":"turn/start","data":{"turn":18},"seq":103,"time":1787472000103} +{"type":"user/message","data":{"id":"preview-user-18","role":"user","content":[{"type":"text","text":"History checkpoint 18: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":104,"time":1787472000104} +{"type":"step/start","data":{"turn":18,"step":1},"seq":105,"time":1787472000105} +{"type":"assistant/message","data":{"turn":18,"step":1,"message":{"id":"preview-assistant-18","role":"assistant","content":[{"type":"text","text":"Checkpoint 18 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":106,"time":1787472000106} +{"type":"step/end","data":{"turn":18,"step":1},"seq":107,"time":1787472000107} +{"type":"turn/end","data":{"turn":18,"reason":{"kind":"completed"}},"seq":108,"time":1787472000108} +{"type":"turn/start","data":{"turn":19},"seq":109,"time":1787472000109} +{"type":"user/message","data":{"id":"preview-user-19","role":"user","content":[{"type":"text","text":"History checkpoint 19: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":110,"time":1787472000110} +{"type":"step/start","data":{"turn":19,"step":1},"seq":111,"time":1787472000111} +{"type":"assistant/message","data":{"turn":19,"step":1,"message":{"id":"preview-assistant-19","role":"assistant","content":[{"type":"text","text":"Checkpoint 19 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":112,"time":1787472000112} +{"type":"step/end","data":{"turn":19,"step":1},"seq":113,"time":1787472000113} +{"type":"turn/end","data":{"turn":19,"reason":{"kind":"completed"}},"seq":114,"time":1787472000114} +{"type":"turn/start","data":{"turn":20},"seq":115,"time":1787472000115} +{"type":"user/message","data":{"id":"preview-user-20","role":"user","content":[{"type":"text","text":"History checkpoint 20: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":116,"time":1787472000116} +{"type":"step/start","data":{"turn":20,"step":1},"seq":117,"time":1787472000117} +{"type":"assistant/message","data":{"turn":20,"step":1,"message":{"id":"preview-assistant-20","role":"assistant","content":[{"type":"text","text":"Checkpoint 20 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":118,"time":1787472000118} +{"type":"step/end","data":{"turn":20,"step":1},"seq":119,"time":1787472000119} +{"type":"turn/end","data":{"turn":20,"reason":{"kind":"completed"}},"seq":120,"time":1787472000120} +{"type":"turn/start","data":{"turn":21},"seq":121,"time":1787472000121} +{"type":"user/message","data":{"id":"preview-user-21","role":"user","content":[{"type":"text","text":"History checkpoint 21: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":122,"time":1787472000122} +{"type":"step/start","data":{"turn":21,"step":1},"seq":123,"time":1787472000123} +{"type":"assistant/message","data":{"turn":21,"step":1,"message":{"id":"preview-assistant-21","role":"assistant","content":[{"type":"text","text":"Checkpoint 21 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":124,"time":1787472000124} +{"type":"step/end","data":{"turn":21,"step":1},"seq":125,"time":1787472000125} +{"type":"turn/end","data":{"turn":21,"reason":{"kind":"completed"}},"seq":126,"time":1787472000126} +{"type":"turn/start","data":{"turn":22},"seq":127,"time":1787472000127} +{"type":"user/message","data":{"id":"preview-user-22","role":"user","content":[{"type":"text","text":"History checkpoint 22: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":128,"time":1787472000128} +{"type":"step/start","data":{"turn":22,"step":1},"seq":129,"time":1787472000129} +{"type":"assistant/message","data":{"turn":22,"step":1,"message":{"id":"preview-assistant-22","role":"assistant","content":[{"type":"text","text":"Checkpoint 22 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":130,"time":1787472000130} +{"type":"step/end","data":{"turn":22,"step":1},"seq":131,"time":1787472000131} +{"type":"turn/end","data":{"turn":22,"reason":{"kind":"completed"}},"seq":132,"time":1787472000132} +{"type":"turn/start","data":{"turn":23},"seq":133,"time":1787472000133} +{"type":"user/message","data":{"id":"preview-user-23","role":"user","content":[{"type":"text","text":"History checkpoint 23: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":134,"time":1787472000134} +{"type":"step/start","data":{"turn":23,"step":1},"seq":135,"time":1787472000135} +{"type":"assistant/message","data":{"turn":23,"step":1,"message":{"id":"preview-assistant-23","role":"assistant","content":[{"type":"text","text":"Checkpoint 23 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":136,"time":1787472000136} +{"type":"step/end","data":{"turn":23,"step":1},"seq":137,"time":1787472000137} +{"type":"turn/end","data":{"turn":23,"reason":{"kind":"completed"}},"seq":138,"time":1787472000138} +{"type":"turn/start","data":{"turn":24},"seq":139,"time":1787472000139} +{"type":"user/message","data":{"id":"preview-user-24","role":"user","content":[{"type":"text","text":"History checkpoint 24: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":140,"time":1787472000140} +{"type":"step/start","data":{"turn":24,"step":1},"seq":141,"time":1787472000141} +{"type":"assistant/message","data":{"turn":24,"step":1,"message":{"id":"preview-assistant-24","role":"assistant","content":[{"type":"text","text":"Checkpoint 24 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":142,"time":1787472000142} +{"type":"step/end","data":{"turn":24,"step":1},"seq":143,"time":1787472000143} +{"type":"turn/end","data":{"turn":24,"reason":{"kind":"completed"}},"seq":144,"time":1787472000144} +{"type":"turn/start","data":{"turn":25},"seq":145,"time":1787472000145} +{"type":"user/message","data":{"id":"preview-user-25","role":"user","content":[{"type":"text","text":"History checkpoint 25: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":146,"time":1787472000146} +{"type":"step/start","data":{"turn":25,"step":1},"seq":147,"time":1787472000147} +{"type":"assistant/message","data":{"turn":25,"step":1,"message":{"id":"preview-assistant-25","role":"assistant","content":[{"type":"text","text":"Checkpoint 25 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":148,"time":1787472000148} +{"type":"step/end","data":{"turn":25,"step":1},"seq":149,"time":1787472000149} +{"type":"turn/end","data":{"turn":25,"reason":{"kind":"completed"}},"seq":150,"time":1787472000150} +{"type":"turn/start","data":{"turn":26},"seq":151,"time":1787472000151} +{"type":"user/message","data":{"id":"preview-user-26","role":"user","content":[{"type":"text","text":"History checkpoint 26: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":152,"time":1787472000152} +{"type":"step/start","data":{"turn":26,"step":1},"seq":153,"time":1787472000153} +{"type":"assistant/message","data":{"turn":26,"step":1,"message":{"id":"preview-assistant-26","role":"assistant","content":[{"type":"text","text":"Checkpoint 26 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":154,"time":1787472000154} +{"type":"step/end","data":{"turn":26,"step":1},"seq":155,"time":1787472000155} +{"type":"turn/end","data":{"turn":26,"reason":{"kind":"completed"}},"seq":156,"time":1787472000156} +{"type":"turn/start","data":{"turn":27},"seq":157,"time":1787472000157} +{"type":"user/message","data":{"id":"preview-user-27","role":"user","content":[{"type":"text","text":"History checkpoint 27: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":158,"time":1787472000158} +{"type":"step/start","data":{"turn":27,"step":1},"seq":159,"time":1787472000159} +{"type":"assistant/message","data":{"turn":27,"step":1,"message":{"id":"preview-assistant-27","role":"assistant","content":[{"type":"text","text":"Checkpoint 27 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":160,"time":1787472000160} +{"type":"step/end","data":{"turn":27,"step":1},"seq":161,"time":1787472000161} +{"type":"turn/end","data":{"turn":27,"reason":{"kind":"completed"}},"seq":162,"time":1787472000162} +{"type":"turn/start","data":{"turn":28},"seq":163,"time":1787472000163} +{"type":"user/message","data":{"id":"preview-user-28","role":"user","content":[{"type":"text","text":"History checkpoint 28: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":164,"time":1787472000164} +{"type":"step/start","data":{"turn":28,"step":1},"seq":165,"time":1787472000165} +{"type":"assistant/message","data":{"turn":28,"step":1,"message":{"id":"preview-assistant-28","role":"assistant","content":[{"type":"text","text":"Checkpoint 28 is recorded."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":166,"time":1787472000166} +{"type":"step/end","data":{"turn":28,"step":1},"seq":167,"time":1787472000167} +{"type":"turn/end","data":{"turn":28,"reason":{"kind":"completed"}},"seq":168,"time":1787472000168} +{"type":"turn/start","data":{"turn":29},"seq":169,"time":1787472000169} +{"type":"user/message","data":{"id":"preview-gallery-user","role":"user","content":[{"type":"text","text":"Show the seeded workspace, tool cards, subagents, and pagination in one tour."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":170,"time":1787472000170} +{"type":"step/start","data":{"turn":29,"step":1},"seq":171,"time":1787472000171} +{"type":"assistant/message","data":{"turn":29,"step":1,"message":{"id":"preview-gallery-tools","role":"assistant","content":[{"type":"reasoning","text":"I will inspect the deterministic workspace and collect each preview surface."},{"type":"tool-call","id":"preview-read","name":"read","arguments":"{\"file_path\":\"PREVIEW.md\"}"},{"type":"tool-call","id":"preview-write","name":"write","arguments":"{\"file_path\":\"src/preview.ts\",\"content\":\"export const previewStatus = 'ready'\\n\\nexport const previewFeatures = ['tools', 'subagents', 'pagination'] as const\\n\"}"},{"type":"tool-call","id":"preview-bash","name":"bash","arguments":"{\"command\":\"printf 'preview ready\\\\n'\",\"description\":\"Print the preview readiness marker\"}"},{"type":"tool-call","id":"preview-glob","name":"glob","arguments":"{\"pattern\":\"**/*\",\"path\":\".\"}"},{"type":"tool-call","id":"preview-grep","name":"grep","arguments":"{\"pattern\":\"preview\",\"path\":\".\",\"include\":\"*.{md,ts,json}\"}"},{"type":"tool-call","id":"preview-web-search","name":"web_search","arguments":"{\"queries\":[\"Web Worker filesystem compatibility\"]}"},{"type":"tool-call","id":"preview-todo","name":"todo_write","arguments":"{\"todos\":[{\"content\":\"Inspect tool cards\",\"status\":\"completed\"},{\"content\":\"Open both subagents\",\"status\":\"completed\"},{\"content\":\"Load earlier history\",\"status\":\"in_progress\"}]}"},{"type":"tool-call","id":"preview-subagent","name":"subagent","arguments":"{\"description\":\"Continue preview verification\",\"prompt\":\"Check the remaining preview cases.\",\"run_in_background\":true}"},{"type":"tool-call","id":"preview-subagent-fork","name":"subagent_fork","arguments":"{\"description\":\"Review preview architecture\",\"prompt\":\"Review the fixture architecture.\",\"run_in_background\":false}"},{"type":"tool-call","id":"preview-failure","name":"read","arguments":"{\"file_path\":\"missing.txt\"}"}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":172,"time":1787472000172} +{"type":"tool/call","data":{"turn":29,"step":1,"callId":"preview-read","name":"read","arguments":"{\"file_path\":\"PREVIEW.md\"}"},"seq":173,"time":1787472000173} +{"type":"tool/result","data":{"turn":29,"step":1,"message":{"id":"preview-read-result","role":"user","content":[{"type":"tool-result","toolCallId":"preview-read","content":[{"type":"text","text":"PREVIEW.md\nfile\n\n1: # Preview Workspace\n2: \n3: This deterministic workspace is bundled with the browser-only preview.\n4: \n5: - `src/preview.ts` is the file changed by the example write result.\n6: - `data/tasks.json` mirrors the completed preview checklist.\n7: - `.agents/skills/preview-tour/SKILL.md` proves dot directories survive image packing.\n8: \n9: Refresh the preview to restore these image bytes.\n\n(End of file - total 9 lines)\n"}],"isError":false}],"source":{"kind":"tool","callId":"preview-read"}},"meta":{"path":"PREVIEW.md","offset":1,"lines":[{"number":1,"text":"# Preview Workspace"},{"number":2,"text":""},{"number":3,"text":"This deterministic workspace is bundled with the browser-only preview."},{"number":4,"text":""},{"number":5,"text":"- `src/preview.ts` is the file changed by the example write result."},{"number":6,"text":"- `data/tasks.json` mirrors the completed preview checklist."},{"number":7,"text":"- `.agents/skills/preview-tour/SKILL.md` proves dot directories survive image packing."},{"number":8,"text":""},{"number":9,"text":"Refresh the preview to restore these image bytes."}],"totalLines":9,"lang":"md"}},"surfaceOp":"append","seq":174,"time":1787472000174} +{"type":"tool/call","data":{"turn":29,"step":1,"callId":"preview-write","name":"write","arguments":"{\"file_path\":\"src/preview.ts\",\"content\":\"export const previewStatus = 'ready'\\n\\nexport const previewFeatures = ['tools', 'subagents', 'pagination'] as const\\n\"}"},"seq":175,"time":1787472000175} +{"type":"tool/result","data":{"turn":29,"step":1,"message":{"id":"preview-write-result","role":"user","content":[{"type":"tool-result","toolCallId":"preview-write","content":[{"type":"text","text":"src/preview.ts\nfile\n\nUpdated file\n"}],"isError":false}],"source":{"kind":"tool","callId":"preview-write"}},"meta":{"diffs":[{"path":"src/preview.ts","oldText":"export const previewStatus = 'draft'\n","newText":"export const previewStatus = 'ready'\n\nexport const previewFeatures = ['tools', 'subagents', 'pagination'] as const\n"}]}},"surfaceOp":"append","seq":176,"time":1787472000176} +{"type":"tool/call","data":{"turn":29,"step":1,"callId":"preview-bash","name":"bash","arguments":"{\"command\":\"printf 'preview ready\\\\n'\",\"description\":\"Print the preview readiness marker\"}"},"seq":177,"time":1787472000177} +{"type":"tool/result","data":{"turn":29,"step":1,"message":{"id":"preview-bash-result","role":"user","content":[{"type":"tool-result","toolCallId":"preview-bash","content":[{"type":"text","text":"preview ready\n"}],"isError":false}],"source":{"kind":"tool","callId":"preview-bash"}}},"surfaceOp":"append","seq":178,"time":1787472000178} +{"type":"tool/call","data":{"turn":29,"step":1,"callId":"preview-glob","name":"glob","arguments":"{\"pattern\":\"**/*\",\"path\":\".\"}"},"seq":179,"time":1787472000179} +{"type":"tool/result","data":{"turn":29,"step":1,"message":{"id":"preview-glob-result","role":"user","content":[{"type":"tool-result","toolCallId":"preview-glob","content":[{"type":"text","text":"PREVIEW.md\ndata/tasks.json\nsrc/preview.ts"}],"isError":false}],"source":{"kind":"tool","callId":"preview-glob"}},"meta":{"shape":"paths","paths":["PREVIEW.md","data/tasks.json","src/preview.ts"],"truncated":false,"total":3}},"surfaceOp":"append","seq":180,"time":1787472000180} +{"type":"tool/call","data":{"turn":29,"step":1,"callId":"preview-grep","name":"grep","arguments":"{\"pattern\":\"preview\",\"path\":\".\",\"include\":\"*.{md,ts,json}\"}"},"seq":181,"time":1787472000181} +{"type":"tool/result","data":{"turn":29,"step":1,"message":{"id":"preview-grep-result","role":"user","content":[{"type":"tool-result","toolCallId":"preview-grep","content":[{"type":"text","text":"PREVIEW.md:3:This deterministic workspace is bundled with the browser-only preview.\nsrc/preview.ts:1:export const previewStatus = 'ready'"}],"isError":false}],"source":{"kind":"tool","callId":"preview-grep"}},"meta":{"shape":"matches","files":[{"path":"PREVIEW.md","matches":[{"lineNumber":3,"line":"This deterministic workspace is bundled with the browser-only preview."}]},{"path":"src/preview.ts","matches":[{"lineNumber":1,"line":"export const previewStatus = 'ready'"}]}],"truncated":false,"total":2}},"surfaceOp":"append","seq":182,"time":1787472000182} +{"type":"tool/call","data":{"turn":29,"step":1,"callId":"preview-web-search","name":"web_search","arguments":"{\"queries\":[\"Web Worker filesystem compatibility\"]}"},"seq":183,"time":1787472000183} +{"type":"tool/result","data":{"turn":29,"step":1,"message":{"id":"preview-web-search-result","role":"user","content":[{"type":"tool-result","toolCallId":"preview-web-search","content":[{"type":"text","text":"Browser workers can host deterministic in-memory filesystems.\n\nSources:\n1. MDN Web Workers API — https://developer.mozilla.org/docs/Web/API/Web_Workers_API"}],"isError":false}],"source":{"kind":"tool","callId":"preview-web-search"}},"meta":{"sources":[{"url":"https://developer.mozilla.org/docs/Web/API/Web_Workers_API","title":"Web Workers API","snippet":"Web Workers run scripts in background threads."}],"truncated":false,"answer":"Browser workers can host deterministic in-memory filesystems."}},"surfaceOp":"append","seq":184,"time":1787472000184} +{"type":"tool/call","data":{"turn":29,"step":1,"callId":"preview-todo","name":"todo_write","arguments":"{\"todos\":[{\"content\":\"Inspect tool cards\",\"status\":\"completed\"},{\"content\":\"Open both subagents\",\"status\":\"completed\"},{\"content\":\"Load earlier history\",\"status\":\"in_progress\"}]}"},"seq":185,"time":1787472000185} +{"type":"todo/write","data":{"todos":[{"content":"Inspect tool cards","status":"completed"},{"content":"Open both subagents","status":"completed"},{"content":"Load earlier history","status":"in_progress"}]},"seq":186,"time":1787472000186} +{"type":"tool/result","data":{"turn":29,"step":1,"message":{"id":"preview-todo-result","role":"user","content":[{"type":"tool-result","toolCallId":"preview-todo","content":[{"type":"text","text":"Updated todo list: 0 pending, 1 in progress, 2 completed."}],"isError":false}],"source":{"kind":"tool","callId":"preview-todo"}}},"surfaceOp":"append","seq":187,"time":1787472000187} +{"type":"tool/call","data":{"turn":29,"step":1,"callId":"preview-subagent","name":"subagent","arguments":"{\"description\":\"Continue preview verification\",\"prompt\":\"Check the remaining preview cases.\",\"run_in_background\":true}"},"seq":188,"time":1787472000188} +{"type":"tool/result","data":{"turn":29,"step":1,"message":{"id":"preview-subagent-result","role":"user","content":[{"type":"tool-result","toolCallId":"preview-subagent","content":[{"type":"text","text":"started subagent preview-follow-up-builder"}],"isError":false}],"source":{"kind":"tool","callId":"preview-subagent"}}},"surfaceOp":"append","seq":189,"time":1787472000189} +{"type":"tool/call","data":{"turn":29,"step":1,"callId":"preview-subagent-fork","name":"subagent_fork","arguments":"{\"description\":\"Review preview architecture\",\"prompt\":\"Review the fixture architecture.\",\"run_in_background\":false}"},"seq":190,"time":1787472000190} +{"type":"tool/result","data":{"turn":29,"step":1,"message":{"id":"preview-subagent-fork-result","role":"user","content":[{"type":"tool-result","toolCallId":"preview-subagent-fork","content":[{"type":"text","text":"The preview fixture remains separate from user-owned WebFS data."}],"isError":false}],"source":{"kind":"tool","callId":"preview-subagent-fork"}}},"surfaceOp":"append","seq":191,"time":1787472000191} +{"type":"tool/call","data":{"turn":29,"step":1,"callId":"preview-failure","name":"read","arguments":"{\"file_path\":\"missing.txt\"}"},"seq":192,"time":1787472000192} +{"type":"tool/result","data":{"turn":29,"step":1,"message":{"id":"preview-failure-result","role":"user","content":[{"type":"tool-result","toolCallId":"preview-failure","content":[{"type":"text","text":"Error: ENOENT: no such file, open missing.txt"}],"isError":true}],"source":{"kind":"tool","callId":"preview-failure"}},"error":{"name":"FsError","code":"ENOENT"}},"surfaceOp":"append","seq":193,"time":1787472000193} +{"type":"step/end","data":{"turn":29,"step":1},"seq":194,"time":1787472000194} +{"type":"step/start","data":{"turn":29,"step":2},"seq":195,"time":1787472000195} +{"type":"assistant/message","data":{"turn":29,"step":2,"message":{"id":"preview-gallery-final","role":"assistant","content":[{"type":"text","text":"## Preview tour complete\n\nThe workspace, specialized tool cards, two subagent histories, and an earlier history page are ready to inspect."}],"source":{"kind":"model","provider":"preview-fixture","model":"deterministic"}}},"sourceEventSeqs":[],"surfaceOp":"append","seq":196,"time":1787472000196} +{"type":"step/end","data":{"turn":29,"step":2},"seq":197,"time":1787472000197} +{"type":"turn/end","data":{"turn":29,"reason":{"kind":"completed"}},"seq":198,"time":1787472000198} diff --git a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/storages/session_projcache.json b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/storages/session_projcache.json new file mode 100644 index 0000000000..9248dd7ae7 --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/storages/session_projcache.json @@ -0,0 +1,24 @@ +{ + "unit": { + "name": "session_projcache", + "version": 3 + }, + "global": null, + "tables": { + "sessions": { + "preview-showcase": { + "identity": { + "createdAt": 1787472000000, + "cwd": "/dsh/workspace" + }, + "rows": { + "title": { + "ver": 1, + "seq": 198, + "val": "WebWorker Preview Showcase" + } + } + } + } + } +} diff --git a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/workspace/.agents/skills/preview-tour/SKILL.md b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/workspace/.agents/skills/preview-tour/SKILL.md new file mode 100644 index 0000000000..4d4f875565 --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/workspace/.agents/skills/preview-tour/SKILL.md @@ -0,0 +1,8 @@ +--- +name: preview-tour +description: Inspect the bundled Preview workspace and its deterministic Session examples. +--- + +# Preview tour + +Read the workspace files, inspect the tool gallery, open both subagent histories, and load the earlier conversation page. diff --git a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/workspace/PREVIEW.md b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/workspace/PREVIEW.md new file mode 100644 index 0000000000..ae96a96efe --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/workspace/PREVIEW.md @@ -0,0 +1,9 @@ +# Preview Workspace + +This deterministic workspace is bundled with the browser-only preview. + +- `src/preview.ts` is the file changed by the example write result. +- `data/tasks.json` mirrors the completed preview checklist. +- `.agents/skills/preview-tour/SKILL.md` proves dot directories survive image packing. + +Refresh the preview to restore these image bytes. diff --git a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/workspace/data/tasks.json b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/workspace/data/tasks.json new file mode 100644 index 0000000000..65ad487ce7 --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/workspace/data/tasks.json @@ -0,0 +1,17 @@ +{ + "title": "Preview verification", + "tasks": [ + { + "name": "Inspect tool cards", + "status": "completed" + }, + { + "name": "Open both subagents", + "status": "completed" + }, + { + "name": "Load earlier history", + "status": "completed" + } + ] +} diff --git a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/workspace/src/preview.ts b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/workspace/src/preview.ts new file mode 100644 index 0000000000..1494e6ea20 --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/workspace/src/preview.ts @@ -0,0 +1,3 @@ +export const previewStatus = 'ready' + +export const previewFeatures = ['tools', 'subagents', 'pagination'] as const diff --git a/packages/experimental/webworker-runtime/tests/node/builtins-table.spec.ts b/packages/experimental/webworker-runtime/tests/node/builtins-table.spec.ts index b302bbc382..3283f9387e 100644 --- a/packages/experimental/webworker-runtime/tests/node/builtins-table.spec.ts +++ b/packages/experimental/webworker-runtime/tests/node/builtins-table.spec.ts @@ -15,11 +15,11 @@ */ import { describe, expect, it } from 'vitest' import { createNodeBuiltins, REPLACED_PREFIXES } from '../../src/node/builtins.ts' -import { WorkerModuleLoader } from '../../src/module-system/module-loader.ts' +import { WorkerModuleLoader, type WorkerRequire } from '../../src/module-system/module-loader.ts' import { MemoryVfs } from '../../src/storage/memory.ts' /** A loader over an empty image: every specifier below resolves from the table. */ -function loaderRequire(): (specifier: string) => unknown { +function loaderRequire(): WorkerRequire { const vfs = new MemoryVfs() vfs.seedDirectory('/dsh') const loader = new WorkerModuleLoader({ vfs, root: '/dsh', staticModules: createNodeBuiltins() }) @@ -84,4 +84,12 @@ describe('module identity through the loader', () => { const require = loaderRequire() expect(() => require('node:dns')).toThrow() }) + + it('exposes the package search paths used by the VFS resolver', () => { + const require = loaderRequire() + expect(require.resolve.paths('node:fs')).toBeNull() + expect(require.resolve.paths('node:dns')).toBeNull() + expect(require.resolve.paths('workspace-package')).toEqual(['/dsh/node_modules']) + expect(require.resolve.paths('./local.js')).toEqual(['/dsh']) + }) }) diff --git a/packages/experimental/webworker-runtime/tests/node/child-process.spec.ts b/packages/experimental/webworker-runtime/tests/node/child-process.spec.ts index 619f95f3ce..a1e6fd3e9c 100644 --- a/packages/experimental/webworker-runtime/tests/node/child-process.spec.ts +++ b/packages/experimental/webworker-runtime/tests/node/child-process.spec.ts @@ -16,13 +16,22 @@ import { afterEach, beforeEach, expect, it, vi } from 'vitest' import { MemoryVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/memory.ts' import { setActiveVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/active.ts' import { spawn, spawnSync } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtin_modules/implemented/child_process.ts' +import { + LAUNCHER_FAILURE_EXIT, grantArgs, launcherPath, probe, +} from '@deepseek-ai/node-addon-landlock-run' import { processAlive, signalProcess } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/node/process-table.ts' +import { hostFileSystem } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/fs-access.ts' +import { + LANDLOCK_EXECUTABLE, landlockFileSystem, parseLandlockArguments, +} from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/process/landlock.ts' import { spawnSubprocess } from '@deepseek-ai/dsh-subprocess-local/src/spawn.ts' vi.mock('node:child_process', async () => await import('@deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtin_modules/implemented/child_process.ts')) const WORKSPACE = '/dsh/workspace' +const HOME = '/dsh/home' +const TMP = '/dsh/tmp' let vfs: MemoryVfs @@ -30,6 +39,8 @@ beforeEach(() => { vfs = new MemoryVfs() setActiveVfs(vfs) vfs.mkdirSync(WORKSPACE, { recursive: true }) + vfs.mkdirSync(HOME, { recursive: true }) + vfs.mkdirSync(TMP, { recursive: true }) vi.spyOn(process, 'kill').mockImplementation((pid: number, signal?: string | number): true => { if (signal === 0) { if (processAlive(pid)) return true @@ -91,6 +102,191 @@ it('refuses a command name that is not a string, as Node does', () => { it('reports that a synchronous run cannot happen, without throwing at the probe', () => { expect(spawnSync('bwrap').error?.code).toBe('ENOENT') expect(spawnSync('echo').error?.message).toContain('commands run asynchronously') + expect(spawnSync(launcherPath(), ['--probe'])).toMatchObject({ + status: 0, + stdout: Buffer.from('landlock: fully enforced\n'), + }) + expect(spawnSync(launcherPath(), ['--ro', '/', '--', 'echo', 'x']).error?.message) + .toContain('commands run asynchronously') + const failedProbe = spawnSync(launcherPath(), ['--probe', '--']) + expect(failedProbe.status).toBe(LAUNCHER_FAILURE_EXIT) + expect(Buffer.isBuffer(failedProbe.stderr)).toBe(true) +}) + +it('keeps the native Landlock package API and CLI failure contract', async () => { + expect(probe()).toBe('full') + expect(probe('/not-the-worker-launcher')).toBe('unusable') + expect(probe('/another-package-layout/bin/landlock-run')).toBe('full') + expect(launcherPath(() => '/ignored/package.json')).toBe('/ignored/bin/landlock-run') + expect(LAUNCHER_FAILURE_EXIT).toBe(125) + expect(await collect(spawn(launcherPath(), ['--probe']))).toEqual({ + stdout: 'landlock: fully enforced\n', stderr: '', code: 0, + }) + const malformed = spawn(launcherPath(), ['--rw'], { cwd: WORKSPACE }) + expect(await collect(malformed)).toEqual({ + stdout: '', + stderr: 'landlock-run: usage error: --rw requires a path\n', + code: 125, + }) + const missingGrant = spawn(launcherPath(), ['--rw', '/dsh/missing', '--', 'touch', `${WORKSPACE}/never`], { cwd: WORKSPACE }) + expect(await collect(missingGrant)).toEqual({ + stdout: '', + stderr: 'landlock-run: cannot open rule path: /dsh/missing: No such file or directory\n', + code: 125, + }) + expect(vfs.existsSync(`${WORKSPACE}/never`)).toBe(false) + const missingCommand = spawn(launcherPath(), ['--ro', '/', '--', 'not-a-program'], { cwd: WORKSPACE }) + expect(await collect(missingCommand)).toEqual({ + stdout: '', + stderr: 'landlock-run: exec failed: No such file or directory\n', + code: 125, + }) +}) + +it('enforces every ShellFileSystem operation and virtual device edge', async () => { + vfs.writeFileSync(`${HOME}/private.txt`, 'private\n') + const invocation = parseLandlockArguments([ + ...grantArgs({ readOnly: ['/dev'], readWrite: [WORKSPACE, '/dev/null'] }), '--', 'true', + ]) + if (invocation.kind !== 'run') throw new Error('expected a confined run invocation') + const guarded = await landlockFileSystem(hostFileSystem(), invocation, WORKSPACE) + + expect(await guarded.stat('/dev/null')).toEqual({ directory: false, size: 0, mtimeMs: 0 }) + expect(await guarded.stat('/dev')).toEqual({ directory: true, size: 0, mtimeMs: 0 }) + expect(await guarded.list('/dev')).toEqual([{ name: 'null', directory: false }]) + await expect(guarded.list('/dev/null')).rejects.toMatchObject({ code: 'ENOTDIR' }) + expect(await guarded.readText('/dev/null')).toBe('') + await guarded.writeText('/dev/null', 'discarded') + await expect(guarded.mkdir('/dev/null', false)).rejects.toMatchObject({ code: 'EEXIST' }) + await expect(guarded.remove('/dev/null', { recursive: false, force: false })).rejects.toMatchObject({ code: 'EACCES' }) + await expect(guarded.rename('/dev/null', `${WORKSPACE}/null`)).rejects.toMatchObject({ code: 'EACCES' }) + await expect(guarded.stat('/dev/null/child')).rejects.toMatchObject({ code: 'ENOTDIR' }) + await expect(guarded.writeText('/dev/null/child', 'not written')).rejects.toMatchObject({ code: 'ENOTDIR' }) + await expect(guarded.mkdir('/dev/null/child', true)).rejects.toMatchObject({ code: 'ENOTDIR' }) + expect(vfs.existsSync('/dev')).toBe(false) + await expect(guarded.readText(`${HOME}/private.txt`)).rejects.toMatchObject({ code: 'EACCES' }) + + await guarded.mkdir('created', false) + await guarded.writeText('created/file', 'one') + await guarded.writeText('created/file', ' two', true) + expect(await guarded.readText(`${WORKSPACE}/created/file`)).toBe('one two') + expect(await guarded.list(`${WORKSPACE}/created`)).toEqual([{ name: 'file', directory: false }]) + await guarded.rename('created/file', 'created/moved') + await expect(guarded.rename('created/moved', '/dev/null')).rejects.toMatchObject({ code: 'EACCES' }) + await guarded.remove('created', { recursive: true, force: false }) + expect(vfs.existsSync(`${WORKSPACE}/created`)).toBe(false) +}) + +it('turns an unexpected virtual-launcher preparation failure into exit 125', async () => { + const base = hostFileSystem() + const result = await LANDLOCK_EXECUTABLE.prepare( + ['--ro', '/', '--', 'true'], + { + cwd: WORKSPACE, + filesystem: { ...base, stat: () => Promise.reject(new Error('storage unavailable')) }, + }, + ) + expect(result).toEqual({ + kind: 'exit', exitCode: 125, stdout: '', stderr: 'landlock-run: Error: storage unavailable\n', + }) +}) + +it.each([ + { args: [], message: 'missing `-- ...` command' }, + { args: ['--unknown', '--', 'true'], message: 'unknown argument: --unknown' }, + { args: ['--probe', '--'], message: '--probe takes no other arguments' }, + { args: ['--'], message: 'missing `-- ...` command' }, + { args: ['--rw', '', '--', 'true'], message: 'cannot open rule path' }, +])('rejects malformed Landlock argv before execution: $message', async ({ args, message }) => { + const child = spawn(launcherPath(), args, { cwd: WORKSPACE }) + const result = await collect(child) + expect(result.code).toBe(LAUNCHER_FAILURE_EXIT) + expect(result.stderr).toContain(message) +}) + +it('enforces read-only and workspace-write grants over the VFS', async () => { + vfs.writeFileSync(`${HOME}/readable.txt`, 'visible\n') + const readOnly = spawn(launcherPath(), [ + ...grantArgs({ readOnly: ['/'], readWrite: ['/dev/null'] }), + '--', 'bash', '-c', `cat ${HOME}/readable.txt; echo discarded > /dev/null; echo denied > ${WORKSPACE}/denied.txt`, + ], { cwd: WORKSPACE }) + const strict = await collect(readOnly) + expect(strict.code).toBe(1) + expect(strict.stdout).toBe('visible\n') + expect(strict.stderr.toLowerCase()).toContain('permission denied') + expect(vfs.existsSync(`${WORKSPACE}/denied.txt`)).toBe(false) + + const workspaceWrite = spawn(launcherPath(), [ + ...grantArgs({ readOnly: ['/'], readWrite: ['/dev/null', '/tmp', WORKSPACE] }), + '--', 'bash', '-c', `echo workspace > ${WORKSPACE}/allowed.txt; echo temporary > /tmp/temp.txt; cat /tmp/temp.txt`, + ], { cwd: WORKSPACE }) + expect(await collect(workspaceWrite)).toEqual({ stdout: 'temporary\n', stderr: '', code: 0 }) + expect(vfs.readFileSync(`${WORKSPACE}/allowed.txt`, 'utf8')).toBe('workspace\n') + expect(vfs.readFileSync(`${TMP}/temp.txt`, 'utf8')).toBe('temporary\n') + expect(vfs.existsSync('/dev/null')).toBe(false) +}) + +it('normalizes relative grants and denies sibling-prefix escapes and unreadable paths', async () => { + vfs.mkdirSync(`${WORKSPACE}/nested`) + vfs.mkdirSync(`${WORKSPACE}-other`) + vfs.writeFileSync(`${HOME}/private.txt`, 'private\n') + const child = spawn(launcherPath(), [ + ...grantArgs({ readOnly: [WORKSPACE], readWrite: ['.'] }), + '--', 'bash', '-c', `echo kept > nested/relative.txt; echo escaped > ${WORKSPACE}-other/escape.txt; cat ${HOME}/private.txt`, + ], { cwd: WORKSPACE }) + const result = await collect(child) + expect(result.code).toBe(1) + expect(result.stderr.toLowerCase()).toContain('permission denied') + expect(vfs.readFileSync(`${WORKSPACE}/nested/relative.txt`, 'utf8')).toBe('kept\n') + expect(vfs.existsSync(`${WORKSPACE}-other/escape.txt`)).toBe(false) + expect(result.stdout).not.toContain('private') +}) + +it('treats trailing-slash grants as the same subtree', async () => { + const invocation = parseLandlockArguments(['--rw', '/tmp/', '--', 'true']) + if (invocation.kind !== 'run') throw new Error('expected a confined run invocation') + const guarded = await landlockFileSystem(hostFileSystem(), invocation, WORKSPACE) + await guarded.writeText('/tmp/nested.txt', 'allowed') + expect(vfs.readFileSync(`${TMP}/nested.txt`, 'utf8')).toBe('allowed') +}) + +it('presents the virtual device directory without storing it in the VFS', async () => { + const child = spawn(launcherPath(), [ + ...grantArgs({ readOnly: ['/'], readWrite: ['/dev/null'] }), + '--', 'bash', '-c', 'ls /dev; cat /dev/null', + ], { cwd: WORKSPACE }) + expect(await collect(child)).toEqual({ stdout: 'null\n', stderr: '', code: 0 }) + expect(vfs.existsSync('/dev')).toBe(false) +}) + +it('requires both rename paths to be writable', async () => { + vfs.writeFileSync(`${WORKSPACE}/source.txt`, 'kept\n') + const child = spawn(launcherPath(), [ + ...grantArgs({ readOnly: ['/'], readWrite: [WORKSPACE] }), + '--', 'mv', `${WORKSPACE}/source.txt`, `${HOME}/moved.txt`, + ], { cwd: WORKSPACE }) + const result = await collect(child) + expect(result.code).toBe(1) + expect(result.stderr.toLowerCase()).toContain('permission denied') + expect(vfs.readFileSync(`${WORKSPACE}/source.txt`, 'utf8')).toBe('kept\n') + expect(vfs.existsSync(`${HOME}/moved.txt`)).toBe(false) +}) + +it('keeps concurrent Landlock grants process-local', async () => { + const strict = spawn(launcherPath(), [ + ...grantArgs({ readOnly: ['/'], readWrite: ['/dev/null'] }), + '--', 'bash', '-c', `sleep 0.02; echo denied > ${WORKSPACE}/strict.txt`, + ], { cwd: WORKSPACE }) + const writable = spawn(launcherPath(), [ + ...grantArgs({ readOnly: ['/'], readWrite: ['/dev/null', WORKSPACE] }), + '--', 'bash', '-c', `echo allowed > ${WORKSPACE}/writable.txt`, + ], { cwd: WORKSPACE }) + const [strictResult, writableResult] = await Promise.all([collect(strict), collect(writable)]) + expect(strictResult.code).toBe(1) + expect(strictResult.stderr.toLowerCase()).toContain('permission denied') + expect(writableResult).toEqual({ stdout: '', stderr: '', code: 0 }) + expect(vfs.existsSync(`${WORKSPACE}/strict.txt`)).toBe(false) + expect(vfs.readFileSync(`${WORKSPACE}/writable.txt`, 'utf8')).toBe('allowed\n') }) it('carries a command through the real local subprocess service', async () => { diff --git a/packages/experimental/webworker-runtime/tests/node/chokidar.spec.ts b/packages/experimental/webworker-runtime/tests/node/chokidar.spec.ts new file mode 100644 index 0000000000..82066b810a --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/node/chokidar.spec.ts @@ -0,0 +1,210 @@ +/** Upstream Chokidar running unchanged through the shipped Worker module loader. */ +import { existsSync, readFileSync } from 'node:fs' +import { createRequire } from 'node:module' +import { dirname, join } from 'node:path' +import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { lowerModuleSource } from '../../src/compile/transform.ts' +import { WorkerModuleLoader } from '../../src/module-system/module-loader.ts' +import { createNodeBuiltins } from '../../src/node/builtins.ts' +import { MemoryVfs } from '../../src/storage/memory.ts' +import { setActiveVfs } from '../../src/storage/active.ts' + +const ROOT = '/dsh/workspace/skills' +let vfs: MemoryVfs +let chokidar: typeof import('chokidar') +const openWatchers: import('chokidar').FSWatcher[] = [] + +interface ChokidarFixture { + readonly label: string + readonly consumerManifest: string + readonly chokidarFiles: readonly string[] + readonly readdirpFiles: readonly string[] +} + +const CHOKIDAR_FIXTURES: readonly ChokidarFixture[] = [ + { + label: 'Chokidar 4 from settings and credentials', + consumerManifest: 'packages/settings/settings-file/package.json', + chokidarFiles: ['package.json', 'esm/package.json', 'esm/index.js', 'esm/handler.js'], + readdirpFiles: ['package.json', 'esm/package.json', 'esm/index.js'], + }, + { + label: 'Chokidar 5 from skill-filesystem', + consumerManifest: 'packages/skill/skill-filesystem/package.json', + chokidarFiles: ['package.json', 'index.js', 'handler.js'], + readdirpFiles: ['package.json', 'index.js'], + }, +] + +/** Copy one installed JavaScript package into the VFS exactly as the packer does. */ +function packageRoot(name: string, entry: string): string { + for (let directory = dirname(entry);;) { + const manifest = join(directory, 'package.json') + if (existsSync(manifest)) { + const parsed = JSON.parse(readFileSync(manifest, 'utf8')) as { name?: unknown } + if (parsed.name === name) return directory + } + const parent = dirname(directory) + if (parent === directory) throw new Error(`cannot locate package root for ${name}`) + directory = parent + } +} + +/** Copy the package files selected by the packer's import condition. */ +function mountPackage(name: string, directory: string, files: readonly string[]): void { + for (const file of files) { + const source = readFileSync(join(directory, file), 'utf8') + const path = `/dsh/node_modules/${name}/${file}` + vfs.seed(path, file.endsWith('.js') ? lowerModuleSource({ filename: path, source }).code : source) + } +} + +/** Load one consumer's exact Chokidar and readdirp versions through the Worker loader. */ +function loadChokidar(fixture: ChokidarFixture): typeof import('chokidar') { + const consumerManifest = join(process.cwd(), fixture.consumerManifest) + const chokidarEntry = createRequire(consumerManifest).resolve('chokidar') + const readdirpEntry = createRequire(chokidarEntry).resolve('readdirp') + mountPackage('chokidar', packageRoot('chokidar', chokidarEntry), fixture.chokidarFiles) + mountPackage('readdirp', packageRoot('readdirp', readdirpEntry), fixture.readdirpFiles) + const loader = new WorkerModuleLoader({ vfs, staticModules: createNodeBuiltins() }) + return loader.createRequire('/dsh/')('chokidar') as typeof import('chokidar') +} + +beforeEach(() => { + vfs = new MemoryVfs() + setActiveVfs(vfs) + vfs.mkdirSync(ROOT, { recursive: true }) +}) + +afterEach(async () => { + await Promise.all(openWatchers.splice(0).map(async (watcher) => { await watcher.close() })) +}) + +/** Await one emitter event while rejecting hangs deterministically. */ +function onceEvent(watcher: import('chokidar').FSWatcher, event: string): Promise { + return new Promise((resolve, reject) => { + const timeout = setTimeout(() => { reject(new Error(`timed out waiting for chokidar ${event}`)) }, 2_000) + const emitter = watcher as unknown as { + once(name: string, listener: (...args: unknown[]) => void): void + } + emitter.once(event, (...args: unknown[]) => { + clearTimeout(timeout) + resolve(args[0] as T) + }) + }) +} + +/** Let watcher timers and promise-based stats reach a stable point. */ +async function delay(ms: number): Promise { + await new Promise((resolve) => { setTimeout(resolve, ms) }) +} + +/** Construct one tracked watcher with deterministic event normalization. */ +function watchPath(path: string, options: import('chokidar').ChokidarOptions = {}): import('chokidar').FSWatcher { + const watcher = chokidar.watch(path, { + ignoreInitial: true, + atomic: false, + awaitWriteFinish: false, + ...options, + }) + openWatchers.push(watcher) + return watcher +} + +describe.each(CHOKIDAR_FIXTURES)('$label running unchanged', (fixture) => { + beforeEach(() => { + chokidar = loadChokidar(fixture) + }) + + it('reaches ready and reports a file lifecycle through fs.watch', async () => { + const watcher = watchPath(ROOT, { depth: 1 }) + await onceEvent(watcher, 'ready') + + const directory = `${ROOT}/sample` + const file = `${directory}/SKILL.md` + const addDirectory = onceEvent(watcher, 'addDir') + const addFile = onceEvent(watcher, 'add') + vfs.mkdirSync(directory) + vfs.writeFileSync(file, '# sample\n') + await expect(addDirectory).resolves.toBe(directory) + await expect(addFile).resolves.toBe(file) + + const changed = onceEvent(watcher, 'change') + vfs.writeFileSync(file, '# changed\n') + await expect(changed).resolves.toBe(file) + + await new Promise((resolve) => { setTimeout(resolve, 10) }) + const removed = onceEvent(watcher, 'unlink') + vfs.rmSync(file) + await expect(removed).resolves.toBe(file) + }) + + it('watches a missing file through its existing parent', async () => { + const path = '/dsh/home/settings.yaml' + vfs.mkdirSync('/dsh/home', { recursive: true }) + const watcher = watchPath(path) + await onceEvent(watcher, 'ready') + + const added = onceEvent(watcher, 'add') + vfs.writeFileSync(path, 'theme: dark\n') + await expect(added).resolves.toBe(path) + + const removed = onceEvent(watcher, 'unlink') + vfs.rmSync(path) + await expect(removed).resolves.toBe(path) + }) + + it('discovers directory children through watchFile polling mode', async () => { + const watcher = watchPath(ROOT, { usePolling: true, interval: 5 }) + await onceEvent(watcher, 'ready') + const path = `${ROOT}/standalone.md` + const added = onceEvent(watcher, 'add') + vfs.writeFileSync(path, '# standalone\n') + await expect(added).resolves.toBe(path) + }) + + it('normalizes a short unlink/add replacement into one atomic change', async () => { + const path = `${ROOT}/atomic.md` + vfs.writeFileSync(path, 'before') + const watcher = watchPath(path, { atomic: 40 }) + await onceEvent(watcher, 'ready') + const events: string[] = [] + watcher.on('all', (event) => { events.push(event) }) + const changed = onceEvent(watcher, 'change') + vfs.rmSync(path) + await delay(5) + vfs.writeFileSync(path, 'after') + await expect(changed).resolves.toBe(path) + await delay(60) + expect(events).toEqual(['change']) + }) + + it('waits for a write burst to stabilize before publishing one add', async () => { + const path = `${ROOT}/settling.md` + const watcher = watchPath(ROOT, { + awaitWriteFinish: { stabilityThreshold: 30, pollInterval: 5 }, + }) + await onceEvent(watcher, 'ready') + const events: string[] = [] + watcher.on('all', (event) => { events.push(event) }) + const added = onceEvent(watcher, 'add') + vfs.writeFileSync(path, 'a') + await delay(10) + vfs.appendFileSync(path, 'b') + await delay(10) + vfs.appendFileSync(path, 'c') + await expect(added).resolves.toBe(path) + expect(events).toEqual(['add']) + }) + + it('emits nothing after close has reached quiescence', async () => { + const watcher = watchPath(ROOT) + const events: string[] = [] + watcher.on('all', (event) => { events.push(event) }) + await onceEvent(watcher, 'ready') + await watcher.close() + vfs.writeFileSync(`${ROOT}/after.md`, '# after\n') + await Promise.resolve() + expect(events).toEqual([]) + }) +}) diff --git a/packages/experimental/webworker-runtime/tests/node/fs-watch-stream.spec.ts b/packages/experimental/webworker-runtime/tests/node/fs-watch-stream.spec.ts new file mode 100644 index 0000000000..de84fcce85 --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/node/fs-watch-stream.spec.ts @@ -0,0 +1,812 @@ +/** Node differential checks for the Worker filesystem watcher and stream faces. */ +import { + closeSync as closeNodeSync, + createReadStream as createNodeReadStream, + createWriteStream as createNodeWriteStream, + mkdtempSync, + openSync as openNodeSync, + readSync as readNodeSync, + readFileSync, + renameSync as renameNodeSync, + rmSync, + unwatchFile as unwatchNodeFile, + watchFile as watchNodeFile, + writeSync as writeNodeSync, + writeFileSync, +} from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { MemoryVfs } from '../../src/storage/memory.ts' +import { setActiveVfs } from '../../src/storage/active.ts' +import * as workerFs from '../../src/node/builtin_modules/implemented/fs.ts' +import * as workerFsp from '../../src/node/builtin_modules/implemented/fs/promises.ts' +import * as workerStream from '../../src/node/builtin_modules/implemented/stream.ts' + +const VFS_ROOT = '/dsh/watch-stream' +const nativeRoots: string[] = [] +let vfs: MemoryVfs + +beforeEach(() => { + vfs = new MemoryVfs() + setActiveVfs(vfs) + vfs.mkdirSync(VFS_ROOT, { recursive: true }) +}) + +afterEach(() => { + for (const root of nativeRoots.splice(0)) rmSync(root, { recursive: true, force: true }) + vi.restoreAllMocks() +}) + +/** Await the next callback value with a bounded failure instead of an open watcher. */ +function nextValue(install: (resolve: (value: T) => void) => void): Promise { + return new Promise((resolve, reject) => { + const timeout = setTimeout(() => { reject(new Error('timed out waiting for filesystem event')) }, 2_000) + install((value) => { + clearTimeout(timeout) + resolve(value) + }) + }) +} + +interface ReadableFileStream { + readonly bytesRead: number + on(event: string, listener: (...args: unknown[]) => void): ReadableFileStream +} + +/** Collect byte chunks and lifecycle events from one read stream implementation. */ +async function readScenario(create: () => ReadableFileStream): Promise<{ + chunks: string[] + events: string[] + bytesRead: number +}> { + const stream = create() + const chunks: string[] = [] + const events: string[] = [] + stream.on('open', () => { events.push('open') }) + stream.on('ready', () => { events.push('ready') }) + stream.on('data', (chunk: unknown) => { + events.push('data') + chunks.push(Buffer.from(chunk as Uint8Array).toString('utf8')) + }) + stream.on('end', () => { events.push('end') }) + await new Promise((resolve, reject) => { + stream.on('error', reject) + stream.on('close', () => { + events.push('close') + resolve() + }) + }) + return { chunks, events, bytesRead: stream.bytesRead } +} + +interface WritableFileStream { + readonly bytesWritten: number + on(event: string, listener: (...args: unknown[]) => void): WritableFileStream + write(chunk: string): boolean + end(chunk?: string): void +} + +/** Write the same chunks and record backpressure plus lifecycle ordering. */ +async function writeScenario(create: () => WritableFileStream): Promise<{ + writes: boolean[] + events: string[] + bytesWritten: number +}> { + const stream = create() + const events: string[] = [] + for (const event of ['open', 'ready', 'drain', 'finish'] as const) { + stream.on(event, () => { events.push(event) }) + } + const writes = [stream.write('ab'), stream.write('cd')] + stream.end('ef') + await new Promise((resolve, reject) => { + stream.on('error', reject) + stream.on('close', () => { + events.push('close') + resolve() + }) + }) + return { writes, events, bytesWritten: stream.bytesWritten } +} + +describe('file streams', () => { + it('keeps an opened file identity across rename, replacement, and unlink', () => { + const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-stream-diff-')) + nativeRoots.push(nativeRoot) + const nativePath = join(nativeRoot, 'identity.txt') + const workerPath = `${VFS_ROOT}/identity.txt` + + const nativeScenario = (): string[] => { + writeFileSync(nativePath, 'original') + const fd = openNodeSync(nativePath, 'r') + renameNodeSync(nativePath, `${nativePath}.moved`) + writeFileSync(nativePath, 'replacement') + const beforeUnlink = Buffer.alloc(16) + const firstCount = readNodeSync(fd, beforeUnlink, 0, beforeUnlink.length, 0) + rmSync(`${nativePath}.moved`) + const afterUnlink = Buffer.alloc(16) + const secondCount = readNodeSync(fd, afterUnlink, 0, afterUnlink.length, 0) + closeNodeSync(fd) + return [beforeUnlink.subarray(0, firstCount).toString(), afterUnlink.subarray(0, secondCount).toString()] + } + const workerScenario = (): string[] => { + vfs.writeFileSync(workerPath, 'original') + const fd = workerFs.openSync(workerPath, 'r') + vfs.renameSync(workerPath, `${workerPath}.moved`) + vfs.writeFileSync(workerPath, 'replacement') + const beforeUnlink = Buffer.alloc(16) + const firstCount = workerFs.readSync(fd, beforeUnlink, 0, beforeUnlink.length, 0) + vfs.rmSync(`${workerPath}.moved`) + const afterUnlink = Buffer.alloc(16) + const secondCount = workerFs.readSync(fd, afterUnlink, 0, afterUnlink.length, 0) + workerFs.closeSync(fd) + return [beforeUnlink.subarray(0, firstCount).toString(), afterUnlink.subarray(0, secondCount).toString()] + } + + expect(workerScenario()).toEqual(nativeScenario()) + }) + + it('keeps a read stream on the file opened before an atomic replacement', async () => { + const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-stream-diff-')) + nativeRoots.push(nativeRoot) + const nativePath = join(nativeRoot, 'stream-identity.txt') + const workerPath = `${VFS_ROOT}/stream-identity.txt` + writeFileSync(nativePath, 'original') + vfs.writeFileSync(workerPath, 'original') + + const readAfterReplacement = async ( + stream: AsyncIterable & { once(event: string, listener: () => void): unknown }, + replace: () => void, + ): Promise => { + stream.once('open', replace) + const chunks: Uint8Array[] = [] + for await (const chunk of stream) chunks.push(chunk) + return Buffer.concat(chunks).toString() + } + const native = await readAfterReplacement(createNodeReadStream(nativePath, { highWaterMark: 2 }), () => { + renameNodeSync(nativePath, `${nativePath}.moved`) + writeFileSync(nativePath, 'replacement') + }) + const worker = await readAfterReplacement(workerFs.createReadStream(workerPath, { highWaterMark: 2 }), () => { + vfs.renameSync(workerPath, `${workerPath}.moved`) + vfs.writeFileSync(workerPath, 'replacement') + }) + expect(worker).toBe(native) + }) + + it('rejects descriptor operations that conflict with the open mode', () => { + const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-stream-diff-')) + nativeRoots.push(nativeRoot) + const nativePath = join(nativeRoot, 'mode.txt') + const workerPath = `${VFS_ROOT}/mode.txt` + writeFileSync(nativePath, 'content') + vfs.writeFileSync(workerPath, 'content') + const codeOf = (run: () => unknown): string | undefined => { + try { + run() + return undefined + } catch (error) { + return (error as NodeJS.ErrnoException).code + } + } + + const nativeReadOnly = openNodeSync(nativePath, 'r') + const workerReadOnly = workerFs.openSync(workerPath, 'r') + expect(codeOf(() => workerFs.writeSync(workerReadOnly, 'x'))) + .toBe(codeOf(() => writeNodeSync(nativeReadOnly, 'x'))) + closeNodeSync(nativeReadOnly) + workerFs.closeSync(workerReadOnly) + + const nativeWriteOnly = openNodeSync(nativePath, 'w') + const workerWriteOnly = workerFs.openSync(workerPath, 'w') + expect(codeOf(() => workerFs.readSync(workerWriteOnly, Buffer.alloc(1), 0, 1, 0))) + .toBe(codeOf(() => readNodeSync(nativeWriteOnly, Buffer.alloc(1), 0, 1, 0))) + closeNodeSync(nativeWriteOnly) + workerFs.closeSync(workerWriteOnly) + }) + + it('keeps hard-link identity and content shared through the Node face', () => { + const source = `${VFS_ROOT}/linked-source.txt` + const alias = `${VFS_ROOT}/linked-alias.txt` + workerFs.writeFileSync(source, 'one') + workerFs.linkSync(source, alias) + expect(workerFs.statSync(alias, { bigint: true }).ino) + .toBe(workerFs.statSync(source, { bigint: true }).ino) + workerFs.appendFileSync(alias, '-two') + expect(workerFs.readFileSync(source, 'utf8')).toBe('one-two') + }) + + it('reports incompatible read and write stream flags as EBADF', async () => { + const path = `${VFS_ROOT}/stream-mode.txt` + vfs.writeFileSync(path, 'content') + const writeError = nextValue((resolve) => { + const stream = workerFs.createWriteStream(path, { flags: 'r' }) + stream.once('error', resolve) + stream.end('x') + }) + await expect(writeError).resolves.toMatchObject({ code: 'EBADF' }) + + const read = workerFs.createReadStream(path, { flags: 'w' }) + const readError = nextValue((resolve) => { read.once('error', resolve) }) + read.resume() + await expect(readError).resolves.toMatchObject({ code: 'EBADF' }) + }) + + it('zero-extends through promise and file-handle truncate', async () => { + const path = `${VFS_ROOT}/truncate.txt` + vfs.writeFileSync(path, new Uint8Array([1, 2])) + await workerFsp.truncate(path, 4) + expect([...workerFs.readFileSync(path) as Uint8Array]).toEqual([1, 2, 0, 0]) + const handle = await workerFsp.open(path, 'r+') + await handle.truncate(6) + await handle.close() + expect([...workerFs.readFileSync(path) as Uint8Array]).toEqual([1, 2, 0, 0, 0, 0]) + }) + + it('matches Node chunking, inclusive ranges, and read lifecycle ordering', async () => { + const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-stream-diff-')) + nativeRoots.push(nativeRoot) + const nativePath = join(nativeRoot, 'input.txt') + const workerPath = `${VFS_ROOT}/input.txt` + writeFileSync(nativePath, '0123456789') + vfs.writeFileSync(workerPath, '0123456789') + + const native = await readScenario(() => createNodeReadStream(nativePath, { start: 2, end: 7, highWaterMark: 2 })) + const worker = await readScenario(() => workerFs.createReadStream(workerPath, { start: 2, end: 7, highWaterMark: 2 })) + expect(worker).toEqual(native) + }) + + it('matches Node write backpressure, lifecycle ordering, and byte accounting', async () => { + const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-stream-diff-')) + nativeRoots.push(nativeRoot) + const nativePath = join(nativeRoot, 'output.txt') + const workerPath = `${VFS_ROOT}/output.txt` + + const native = await writeScenario(() => createNodeWriteStream(nativePath, { highWaterMark: 2 })) + const worker = await writeScenario(() => workerFs.createWriteStream(workerPath, { highWaterMark: 2 })) + expect(worker).toEqual(native) + expect(workerFs.readFileSync(workerPath, 'utf8')).toBe('abcdef') + }) + + it('uses the maintained stream implementation for backpressure and async iteration', async () => { + const values: string[] = [] + for await (const value of workerStream.Readable.from(['one', 'two'])) values.push(String(value)) + expect(values).toEqual(['one', 'two']) + expect(workerStream.default).toBe(workerStream.Stream) + expect(new workerStream.Writable({ write: (_chunk, _encoding, callback) => { callback() } })) + .toBeInstanceOf(workerStream.default) + expect(typeof workerStream.pipeline).toBe('function') + expect(typeof workerStream.finished).toBe('function') + expect(workerStream.getDefaultHighWaterMark(false)).toBe(64 * 1024) + expect(workerStream.default._isArrayBufferView(new Uint8Array())).toBe(true) + }) + + it('uses Node 22 Linux file-stream defaults and abort error identity', async () => { + const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-stream-diff-')) + nativeRoots.push(nativeRoot) + const nativePath = join(nativeRoot, 'input.txt') + const workerPath = `${VFS_ROOT}/input.txt` + writeFileSync(nativePath, 'content') + vfs.writeFileSync(workerPath, 'content') + const nativeRead = createNodeReadStream(nativePath) + const nativeWrite = createNodeWriteStream(join(nativeRoot, 'output.txt')) + const workerRead = workerFs.createReadStream(workerPath) + const workerWrite = workerFs.createWriteStream(`${VFS_ROOT}/output.txt`) + expect([workerRead.readableHighWaterMark, workerWrite.writableHighWaterMark]).toEqual([ + nativeRead.readableHighWaterMark, + nativeWrite.writableHighWaterMark, + ]) + interface CloseableStream { + once(event: string, listener: (...args: unknown[]) => void): unknown + destroy(): unknown + } + const streams = [nativeRead, nativeWrite, workerRead, workerWrite] as unknown as CloseableStream[] + const closed = streams.map(stream => new Promise((resolve) => { + stream.once('error', () => {}) + stream.once('close', () => { resolve() }) + })) + for (const stream of streams) stream.destroy() + await Promise.all(closed) + + const controller = new AbortController() + controller.abort(new Error('stop')) + const aborted = workerFs.createReadStream(workerPath, { signal: controller.signal }) + const error = await nextValue((resolve) => { aborted.once('error', resolve) }) + expect(error).toMatchObject({ name: 'AbortError', code: 'ABORT_ERR' }) + }) + + it('keeps autoClose false descriptors open until explicit stream close', async () => { + const readPath = `${VFS_ROOT}/manual-read-close.txt` + vfs.writeFileSync(readPath, 'content') + const read = workerFs.createReadStream(readPath, { autoClose: false }) + read.resume() + await nextValue((resolve) => { read.once('end', () => { resolve(undefined) }) }) + const readFd = read.fd + expect(readFd).not.toBeNull() + expect(read.destroyed).toBe(false) + expect(() => workerFs.readSync(readFd as number, Buffer.alloc(1), 0, 1, 0)).not.toThrow() + const readClosed = nextValue((resolve) => { read.once('close', () => { resolve(undefined) }) }) + read.close() + await readClosed + expect(() => workerFs.readSync(readFd as number, Buffer.alloc(1), 0, 1, 0)).toThrow(/EBADF/) + + const write = workerFs.createWriteStream(`${VFS_ROOT}/manual-write-close.txt`, { autoClose: false }) + write.end('a') + await nextValue((resolve) => { write.once('finish', () => { resolve(undefined) }) }) + const writeFd = write.fd + expect(writeFd).not.toBeNull() + expect(write.destroyed).toBe(false) + expect(workerFs.writeSync(writeFd as number, 'b')).toBe(1) + const writeClosed = nextValue((resolve) => { write.once('close', () => { resolve(undefined) }) }) + write.close() + await writeClosed + expect(workerFs.readFileSync(`${VFS_ROOT}/manual-write-close.txt`, 'utf8')).toBe('ab') + + vfs.writeFileSync(`${VFS_ROOT}/manual-error-close.txt`, 'content') + const errored = workerFs.createWriteStream(`${VFS_ROOT}/manual-error-close.txt`, { + flags: 'r', + autoClose: false, + }) + const error = nextValue((resolve) => { errored.once('error', resolve) }) + errored.end('rejected') + await expect(error).resolves.toMatchObject({ code: 'EBADF' }) + const errorFd = errored.fd + expect(errorFd).not.toBeNull() + expect(errored.destroyed).toBe(false) + expect(() => workerFs.readSync(errorFd as number, Buffer.alloc(1), 0, 1, 0)).not.toThrow() + const errorClosed = nextValue((resolve) => { errored.once('close', () => { resolve(undefined) }) }) + errored.destroy() + await errorClosed + expect(() => workerFs.readSync(errorFd as number, Buffer.alloc(1), 0, 1, 0)).toThrow(/EBADF/) + }) + + it('matches Node positional overwrite and missing-file failure', async () => { + const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-stream-diff-')) + nativeRoots.push(nativeRoot) + const nativePath = join(nativeRoot, 'position.txt') + const workerPath = `${VFS_ROOT}/position.txt` + writeFileSync(nativePath, 'abcdef') + vfs.writeFileSync(workerPath, 'abcdef') + + const writeAt = async (stream: WritableFileStream): Promise => { + stream.end('XY') + await new Promise((resolve) => { stream.on('close', () => { resolve() }) }) + } + await writeAt(createNodeWriteStream(nativePath, { flags: 'r+', start: 2 })) + await writeAt(workerFs.createWriteStream(workerPath, { flags: 'r+', start: 2 })) + expect(workerFs.readFileSync(workerPath, 'utf8')).toBe(readFileSync(nativePath, 'utf8')) + + const missing = workerFs.createReadStream(`${VFS_ROOT}/missing.txt`) + const events: string[] = [] + missing.on('error', () => { events.push('error') }) + await new Promise((resolve) => { + missing.on('close', () => { + events.push('close') + resolve() + }) + }) + expect(events).toEqual(['error', 'close']) + }) + + it('publishes descriptors before open and ready listener exceptions escape', () => { + const readPath = `${VFS_ROOT}/listener-read.txt` + vfs.writeFileSync(readPath, 'content') + const readCallback = vi.fn() + const readFailure = new Error('read open listener failed') + const readReceiver: { + path: string + flags: string + start: number + end: number + signal: undefined + pending: boolean + fd: number | null + emit(event: string): boolean + } = { + path: readPath, + flags: 'r', + start: 0, + end: Number.POSITIVE_INFINITY, + signal: undefined, + pending: true, + fd: null, + emit(event) { + expect(readCallback).toHaveBeenCalledOnce() + if (event === 'open') throw readFailure + return true + }, + } + expect(() => { + workerFs.ReadStream.prototype._construct.call( + readReceiver as unknown as workerFs.ReadStream, + readCallback, + ) + }).toThrow(readFailure) + expect(readReceiver.pending).toBe(false) + expect(readReceiver.fd).not.toBeNull() + workerFs.closeSync(readReceiver.fd as number) + + const writeCallback = vi.fn() + const writeFailure = new Error('write ready listener failed') + const writeReceiver: { + path: string + flags: string + mode: undefined + start: undefined + signal: undefined + pending: boolean + fd: number | null + emit(event: string): boolean + } = { + path: `${VFS_ROOT}/listener-write.txt`, + flags: 'w', + mode: undefined, + start: undefined, + signal: undefined, + pending: true, + fd: null, + emit(event) { + expect(writeCallback).toHaveBeenCalledOnce() + if (event === 'ready') throw writeFailure + return true + }, + } + expect(() => { + workerFs.WriteStream.prototype._construct.call( + writeReceiver as unknown as workerFs.WriteStream, + writeCallback, + ) + }).toThrow(writeFailure) + expect(writeReceiver.pending).toBe(false) + expect(writeReceiver.fd).not.toBeNull() + workerFs.closeSync(writeReceiver.fd as number) + }) + + it('codes a write before descriptor publication as EBADF', () => { + let failure: Error | null | undefined + workerFs.WriteStream.prototype._write.call( + { fd: null } as unknown as workerFs.WriteStream, + Buffer.from('x'), + 'utf8', + (error) => { failure = error }, + ) + expect(failure).toMatchObject({ code: 'EBADF', syscall: 'write' }) + }) +}) + +interface StatTransition { + currentExists: boolean + previousExists: boolean + currentSize: number + previousSize: number + currentOtherKinds: boolean[] +} + +/** Observe missing, creation, rewrite, and deletion through one watchFile implementation. */ +async function watchFileScenario( + path: string, + watchFile: typeof watchNodeFile, + unwatchFile: typeof unwatchNodeFile, + write: (text: string) => void, + remove: () => void, +): Promise { + const waiting: Array<(value: StatTransition) => void> = [] + const queued: StatTransition[] = [] + const listener = (current: import('node:fs').Stats, previous: import('node:fs').Stats): void => { + const transition = { + currentExists: current.isFile(), + previousExists: previous.isFile(), + currentSize: current.size, + previousSize: previous.size, + currentOtherKinds: [ + current.isDirectory(), current.isSymbolicLink(), current.isFIFO(), + current.isSocket(), current.isBlockDevice(), current.isCharacterDevice(), + ], + } + const resolve = waiting.shift() + if (resolve === undefined) queued.push(transition) + else resolve(transition) + } + const next = async (): Promise => { + const queuedValue = queued.shift() + if (queuedValue !== undefined) return queuedValue + return await nextValue((resolve) => { waiting.push(resolve) }) + } + watchFile(path, { interval: 10, persistent: false }, listener) + try { + const missing = await next() + write('a') + const created = await next() + write('longer') + const changed = await next() + remove() + const removed = await next() + return [missing, created, changed, removed] + } finally { + unwatchFile(path, listener) + } +} + +describe('watchers', () => { + it('does not catch exceptions thrown by a successful stat callback', () => { + const path = `${VFS_ROOT}/callback.txt` + vfs.writeFileSync(path, 'value') + const failure = new Error('callback failed') + let calls = 0 + const dispatch = vi.spyOn(globalThis, 'queueMicrotask').mockImplementation((callback) => { callback() }) + expect(() => { + workerFs.stat(path, () => { + calls += 1 + throw failure + }) + }).toThrow(failure) + expect(calls).toBe(1) + dispatch.mockRestore() + }) + + it('matches Node watchFile state transitions for a missing and recreated file', async () => { + const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-watch-diff-')) + nativeRoots.push(nativeRoot) + const nativePath = join(nativeRoot, 'watched.txt') + const workerPath = `${VFS_ROOT}/watched.txt` + const native = await watchFileScenario( + nativePath, + watchNodeFile, + unwatchNodeFile, + (text) => { writeFileSync(nativePath, text) }, + () => { rmSync(nativePath) }, + ) + const worker = await watchFileScenario( + workerPath, + workerFs.watchFile as unknown as typeof watchNodeFile, + workerFs.unwatchFile as unknown as typeof unwatchNodeFile, + (text) => { vfs.writeFileSync(workerPath, text) }, + () => { vfs.rmSync(workerPath) }, + ) + expect(worker).toEqual(native) + }) + + it('shares one StatWatcher and removes only the named listener', async () => { + const path = `${VFS_ROOT}/shared.txt` + vfs.writeFileSync(path, 'a') + const firstEvents: number[] = [] + const secondEvents: number[] = [] + const first = (): void => { firstEvents.push(1) } + const second = (): void => { secondEvents.push(1) } + const firstWatcher = workerFs.watchFile(path, { interval: 1, persistent: false }, first) + const secondWatcher = workerFs.watchFile(path, { interval: 1, persistent: false }, second) + expect(secondWatcher).toBe(firstWatcher) + workerFs.unwatchFile(path, first) + vfs.writeFileSync(path, 'bb') + await nextValue((resolve) => { + const poll = setInterval(() => { + if (secondEvents.length === 0) return + clearInterval(poll) + resolve(undefined) + }, 1) + }) + expect(firstEvents).toEqual([]) + expect(secondEvents).toEqual([1]) + workerFs.unwatchFile(path) + }) + + it('reports direct and recursive names, then reaches quiescence on close', async () => { + const root = `${VFS_ROOT}/tree` + vfs.mkdirSync(`${root}/nested`, { recursive: true }) + const directEvents: Array<[string, string]> = [] + const recursiveEvents: Array<[string, string]> = [] + const direct = workerFs.watch(root, (_event, _filename) => {}) + direct.on('change', (event, filename) => { directEvents.push([String(event), String(filename)]) }) + const recursive = workerFs.watch(root, { recursive: true }, (event, filename) => { + recursiveEvents.push([event, String(filename)]) + }) + vfs.writeFileSync(`${root}/top.txt`, 'top') + vfs.writeFileSync(`${root}/nested/deep.txt`, 'deep') + await Promise.resolve() + expect(directEvents).toEqual([['rename', 'top.txt']]) + expect(recursiveEvents).toEqual([ + ['rename', 'top.txt'], + ['rename', 'nested/deep.txt'], + ]) + direct.close() + recursive.close() + vfs.writeFileSync(`${root}/after.txt`, 'after') + await Promise.resolve() + expect(directEvents).toHaveLength(1) + expect(recursiveEvents).toHaveLength(2) + }) + + it('supports Buffer filenames, file targets, abort closure, and ref state', async () => { + const path = `${VFS_ROOT}/encoded.txt` + vfs.writeFileSync(path, 'before') + const controller = new AbortController() + const event = nextValue<[string, Buffer]>((resolve) => { + const watcher = workerFs.watch( + new TextEncoder().encode(path), + { encoding: 'buffer', persistent: false, signal: controller.signal }, + (eventType, filename) => { resolve([eventType, filename as Buffer]) }, + ) + expect(watcher.hasRef()).toBe(false) + expect(watcher.ref().hasRef()).toBe(true) + expect(watcher.unref().hasRef()).toBe(false) + }) + vfs.writeFileSync(path, 'after') + const [eventType, filename] = await event + expect(eventType).toBe('change') + expect(Buffer.isBuffer(filename)).toBe(true) + expect(filename.toString()).toBe('encoded.txt') + + const watcher = workerFs.watch(path, { signal: controller.signal }) + let closes = 0 + const closed = nextValue((resolve) => { + watcher.on('close', () => { + closes += 1 + resolve(undefined) + }) + }) + controller.abort(new Error('stop')) + await closed + watcher.close() + await Promise.resolve() + expect(closes).toBe(1) + }) + + it('supports the string encoding overload and suppresses queued delivery after close', async () => { + const encoded = nextValue((resolve) => { + const watcher = workerFs.watch(VFS_ROOT, 'buffer', (_eventType, filename) => { + watcher.close() + resolve(filename as Buffer) + }) + }) + vfs.writeFileSync(`${VFS_ROOT}/buffer-name.txt`, 'x') + await expect(encoded).resolves.toEqual(Buffer.from('buffer-name.txt')) + + let calls = 0 + const closed = workerFs.watch(VFS_ROOT, () => { calls += 1 }) + vfs.writeFileSync(`${VFS_ROOT}/queued.txt`, 'x') + closed.close() + await Promise.resolve() + expect(calls).toBe(0) + }) + + it('reports removal of an ancestor to a watched file', async () => { + const directory = `${VFS_ROOT}/removed-parent` + const path = `${directory}/file.txt` + vfs.mkdirSync(directory) + vfs.writeFileSync(path, 'x') + const event = nextValue<[string, string]>((resolve) => { + const watcher = workerFs.watch(path, (eventType, filename) => { + watcher.close() + resolve([eventType, String(filename)]) + }) + }) + vfs.rmSync(directory, { recursive: true }) + await expect(event).resolves.toEqual(['rename', 'file.txt']) + }) + + it('returns an asynchronously closing watcher for a pre-aborted signal', async () => { + const controller = new AbortController() + controller.abort(new Error('already stopped')) + const order: string[] = [] + const watcher = workerFs.watch(VFS_ROOT, { signal: controller.signal }) + const closed = nextValue((resolve) => { + watcher.once('close', () => { + order.push('close') + resolve(undefined) + }) + }) + order.push('return') + await closed + expect(order).toEqual(['return', 'close']) + expect(() => { vfs.writeFileSync(`${VFS_ROOT}/after-abort.txt`, 'x') }).not.toThrow() + }) + + it('reports an atomic replacement destination as rename even when it existed', async () => { + const target = `${VFS_ROOT}/target.txt` + const replacement = `${VFS_ROOT}/replacement.txt` + vfs.writeFileSync(target, 'old') + vfs.writeFileSync(replacement, 'new') + const event = nextValue<[string, string]>((resolve) => { + const watcher = workerFs.watch(VFS_ROOT, (eventType, filename) => { + if (String(filename) !== 'target.txt') return + watcher.close() + resolve([eventType, String(filename)]) + }) + }) + vfs.renameSync(replacement, target) + await expect(event).resolves.toEqual(['rename', 'target.txt']) + }) + + it('supports BigInt watchFile state, default options, and idempotent stop', async () => { + const path = `${VFS_ROOT}/bigint.txt` + const states = nextValue<[bigint, bigint]>((resolve) => { + const watcher = workerFs.watchFile(new URL(`file://${path}`), { bigint: true, interval: 1 }, (current, previous) => { + resolve([current.size as bigint, previous.size as bigint]) + }) + expect(watcher.hasRef()).toBe(true) + expect(watcher.unref().hasRef()).toBe(false) + expect(watcher.ref().hasRef()).toBe(true) + }) + vfs.writeFileSync(path, 'big') + await expect(states).resolves.toEqual([3n, 0n]) + workerFs.unwatchFile(path) + workerFs.unwatchFile(path) + + vfs.writeFileSync(`${VFS_ROOT}/default.txt`, 'x') + const listener = (): void => {} + const defaultWatcher = workerFs.watchFile(`${VFS_ROOT}/default.txt`, listener) + expect(defaultWatcher.hasRef()).toBe(true) + defaultWatcher.close() + defaultWatcher.close() + expect(() => workerFs.watchFile(`${VFS_ROOT}/default.txt`, {})).toThrow(/listener/) + + let cancelledCalls = 0 + const cancelled = workerFs.watchFile(`${VFS_ROOT}/never-created`, { interval: 1 }, () => { cancelledCalls += 1 }) + cancelled.close() + cancelled.close() + await new Promise((resolve) => { setTimeout(resolve, 5) }) + expect(cancelledCalls).toBe(0) + }) + + it('propagates non-absence stat failures from watchFile', () => { + const failure = Object.assign(new Error('denied'), { code: 'EACCES' }) + vi.spyOn(vfs, 'statSync').mockImplementationOnce(() => { throw failure }) + expect(() => workerFs.watchFile(`${VFS_ROOT}/denied`, () => {})).toThrow(failure) + }) + + it('exposes promise watch as an abortable async iterator', async () => { + const controller = new AbortController() + const iterator = workerFsp.watch(VFS_ROOT, { signal: controller.signal })[Symbol.asyncIterator]() + const event = iterator.next() + vfs.writeFileSync(`${VFS_ROOT}/async.txt`, 'x') + await expect(event).resolves.toEqual({ done: false, value: { eventType: 'rename', filename: 'async.txt' } }) + const failed = iterator.next() + const completed = iterator.next() + controller.abort() + await expect(failed).rejects.toMatchObject({ name: 'AbortError', code: 'ABORT_ERR' }) + await expect(completed).resolves.toEqual({ done: true, value: undefined }) + await expect(iterator.next()).resolves.toEqual({ done: true, value: undefined }) + }) + + it('rejects the first promise-watch read for a pre-aborted signal', async () => { + const controller = new AbortController() + const reason = new Error('already stopped') + controller.abort(reason) + const iterator = workerFsp.watch(VFS_ROOT, { signal: controller.signal })[Symbol.asyncIterator]() + await expect(iterator.next()).rejects.toMatchObject({ name: 'AbortError', code: 'ABORT_ERR', cause: reason }) + await expect(iterator.next()).resolves.toEqual({ done: true, value: undefined }) + }) + + it('lets promise-watch return interrupt a pending next call', async () => { + const iterator = workerFsp.watch(VFS_ROOT)[Symbol.asyncIterator]() + const pending = iterator.next() + await expect(iterator.return?.()).resolves.toEqual({ done: true, value: undefined }) + await expect(pending).resolves.toEqual({ done: true, value: undefined }) + vfs.writeFileSync(`${VFS_ROOT}/after-return.txt`, 'x') + await expect(iterator.next()).resolves.toEqual({ done: true, value: undefined }) + }) + + it('propagates promise-watch startup and throw failures', async () => { + const missing = workerFsp.watch(`${VFS_ROOT}/missing`)[Symbol.asyncIterator]() + await expect(missing.next()).rejects.toMatchObject({ code: 'ENOENT' }) + await expect(missing.next()).resolves.toEqual({ done: true, value: undefined }) + + const iterator = workerFsp.watch(VFS_ROOT)[Symbol.asyncIterator]() + const reason = { reason: 'caller stopped iteration' } + if (iterator.throw === undefined) throw new Error('watch iterator has no throw method') + await expect(iterator.throw(reason)).rejects.toBe(reason) + await expect(iterator.next()).resolves.toEqual({ done: true, value: undefined }) + }) + + it('queues promise-watch events when no next call is waiting', async () => { + const iterator = workerFsp.watch(VFS_ROOT)[Symbol.asyncIterator]() + const first = iterator.next() + vfs.writeFileSync(`${VFS_ROOT}/one.txt`, 'one') + vfs.writeFileSync(`${VFS_ROOT}/two.txt`, 'two') + await expect(first).resolves.toEqual({ done: false, value: { eventType: 'rename', filename: 'one.txt' } }) + await expect(iterator.next()).resolves.toEqual({ done: false, value: { eventType: 'rename', filename: 'two.txt' } }) + await expect(iterator.return?.()).resolves.toEqual({ done: true, value: undefined }) + await expect(iterator.return?.()).resolves.toEqual({ done: true, value: undefined }) + }) +}) diff --git a/packages/experimental/webworker-runtime/tests/node/fs.spec.ts b/packages/experimental/webworker-runtime/tests/node/fs.spec.ts index 400e01ee92..ce9e6fd916 100644 --- a/packages/experimental/webworker-runtime/tests/node/fs.spec.ts +++ b/packages/experimental/webworker-runtime/tests/node/fs.spec.ts @@ -12,9 +12,17 @@ import { MemoryVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/s import { setActiveVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/active.ts' import * as fs from '@deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtin_modules/implemented/fs.ts' import * as fsp from '@deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtin_modules/implemented/fs/promises.ts' -import type { VfsBigIntStats, VfsStats } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/types.ts' +import type { VfsBigIntStats, VfsMutationSink, VfsStats } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/types.ts' -const vfs = new MemoryVfs() +let flushes = 0 +const sink: VfsMutationSink = { + record: () => {}, + flush: () => { + flushes += 1 + return Promise.resolve() + }, +} +const vfs = new MemoryVfs({ sink }) setActiveVfs(vfs) // Identity precondition: the bridge must read this exact mounted VFS; successful @@ -76,9 +84,6 @@ throws('readFileSync missing', () => fs.readFileSync('/dsh/missing'), 'ENOENT') throws('statSync missing', () => fs.statSync('/dsh/missing'), 'ENOENT') throws('accessSync missing', () =>{ fs.accessSync('/dsh/missing') }, 'ENOENT') throws('readdirSync missing', () => fs.readdirSync('/dsh/missing'), 'ENOENT') -throws('watchFile is loud', () => fs.watchFile('/dsh/config/cordis.yml'), 'not implemented') -throws('createReadStream is loud', () => fs.createReadStream('/dsh/config/cordis.yml'), 'not implemented') - const appendFd = fs.openSync('/dsh/log.jsonl', 'a') fs.writeSync(appendFd, '{"a":1}\n') fs.writeSync(appendFd, '{"a":2}\n') @@ -112,6 +117,7 @@ const appendHandle = await fsp.open('/dsh/log-handle.jsonl', 'a') check('append handle sees the existing size', (await appendHandle.stat()).size, 7) await appendHandle.writeFile('batch-1\n') await appendHandle.sync() +check('handle.sync flushes the active VFS', flushes, 1) await appendHandle.close() const secondHandle = await fsp.open('/dsh/log-handle.jsonl', 'a') await secondHandle.writeFile('batch-2\n') diff --git a/packages/experimental/webworker-runtime/tests/node/node-stubs.spec.ts b/packages/experimental/webworker-runtime/tests/node/node-stubs.spec.ts index ef52920292..35d6f32460 100644 --- a/packages/experimental/webworker-runtime/tests/node/node-stubs.spec.ts +++ b/packages/experimental/webworker-runtime/tests/node/node-stubs.spec.ts @@ -16,17 +16,14 @@ import { notAvailableError, notImplementedFail } from '../../src/node/notImpleme import * as childProcess from '../../src/node/builtin_modules/implemented/child_process.ts' import * as net from '../../src/node/builtin_modules/mock/net.ts' import * as sqlite from '../../src/node/builtin_modules/mock/sqlite.ts' -import * as stream from '../../src/node/builtin_modules/mock/stream.ts' +import * as stream from '../../src/node/builtin_modules/implemented/stream.ts' import * as vm from '../../src/node/builtin_modules/mock/vm.ts' import * as workerThreads from '../../src/node/builtin_modules/mock/worker_threads.ts' -import * as chokidar from '../../src/node/external_packages/chokidar.ts' -import * as landlock from '../../src/node/external_packages/node-addon-landlock-run.ts' import * as nodePty from '../../src/node/external_packages/node-pty.ts' import * as piAi from '../../src/node/external_packages/pi-ai.ts' import * as ripgrep from '../../src/node/external_packages/ripgrep.ts' import * as ws from '../../src/node/external_packages/ws.ts' import { REPLACED_EXTERNAL_PACKAGES } from '../../src/node/external_packages/replaced-externals.ts' -import * as fs from '../../src/node/builtin_modules/implemented/fs.ts' import * as os from '../../src/node/builtin_modules/implemented/os.ts' import * as perfHooks from '../../src/node/builtin_modules/implemented/perf_hooks.ts' import { DSH_HOME, DSH_TMP } from '../../src/storage/paths.ts' @@ -43,9 +40,7 @@ const CALLED: [string, Record, readonly string[]][] = [ // The rest of `node:child_process` runs commands (see child-process.spec.ts); // these three need a real process, so they stay refusals. ['node:child_process', childProcess, ['execFileSync', 'execSync', 'fork']], - ['node:stream', stream, ['Readable', 'Writable', 'Duplex', 'Transform', 'PassThrough', 'pipeline', 'finished']], ['node-pty', nodePty, ['spawn', 'open']], - ['@deepseek-ai/node-addon-landlock-run', landlock, ['probe']], ['@deepseek-ai/pi-ai', piAi, [ 'createProvider', 'createModels', 'openAICompletionsApi', 'openAIResponsesApi', 'anthropicMessagesApi', 'isContextOverflow', 'getSupportedThinkingLevels', @@ -95,7 +90,7 @@ describe('not-implemented stubs', () => { } it('keeps the CommonJS interop marker and a default export on every replaced module', () => { - for (const namespace of [net, sqlite, vm, workerThreads, childProcess, stream, chokidar, ws, nodePty, piAi, os, perfHooks]) { + for (const namespace of [net, sqlite, vm, workerThreads, childProcess, stream, ws, nodePty, piAi, os, perfHooks]) { const holder = namespace as { __esModule?: unknown; default?: unknown } expect(holder.__esModule).toBe(true) expect(holder.default).toBeDefined() @@ -104,19 +99,6 @@ describe('not-implemented stubs', () => { }) describe('constructible-but-inert fakes', () => { - // These two are constructed in `[Service.init]` bodies and field initializers, - // so construction must succeed; only the members that would move bytes refuse. - it('chokidar watches nothing and says so by never emitting', async () => { - const watcher = chokidar.watch() - expect(watcher).toBeInstanceOf(chokidar.FSWatcher) - expect(watcher.on()).toBe(watcher) - expect(watcher.once()).toBe(watcher) - expect(watcher.add()).toBe(watcher) - expect(watcher.unwatch()).toBe(watcher) - expect(watcher.getWatched()).toEqual({}) - await expect(watcher.close()).resolves.toBeUndefined() - }) - it('a ws server constructs, accepts listeners, and refuses to carry an upgrade', () => { quiet() expect(ws.Server).toBe(ws.WebSocketServer) @@ -133,16 +115,14 @@ describe('constructible-but-inert fakes', () => { describe('replaced external packages', () => { it('lists the packages the loader serves from the bundle', () => { - expect(REPLACED_EXTERNAL_PACKAGES).toContain('chokidar') + expect(REPLACED_EXTERNAL_PACKAGES).not.toContain('chokidar') + expect(REPLACED_EXTERNAL_PACKAGES).not.toContain('@deepseek-ai/node-addon-landlock-run') expect(REPLACED_EXTERNAL_PACKAGES).toContain('ws') }) it('answers the values callers read without invoking anything', () => { - // The ripgrep binary path and the landlock launcher are read as data by - // consumers that then fail on their own terms. + // The ripgrep binary path is read as data by its consumer. expect(typeof ripgrep.rgPath).toBe('string') - expect(typeof landlock.LAUNCHER_BIN).toBe('string') - expect(typeof landlock.LAUNCHER_FAILURE_EXIT).toBe('number') }) }) @@ -190,17 +170,3 @@ describe('node:perf_hooks', () => { expect(perfHooks.performance.now()).toBeGreaterThan(0) }) }) - -describe('watching', () => { - // Watching stays a loud refusal because `skill-filesystem` AWAITS watcher - // progress rather than merely registering a listener; an inert watcher left - // its discovery hanging. `fs.ts` records the experiment and the mechanism. - it('refuses, naming the member, so an awaiting caller fails fast', () => { - quiet() - expect(() => fs.watchFile('/dsh/config/cordis.yml')).toThrow(/watchFile is not implemented in the worker host/) - }) - - it('accepts the unconditional teardown call, since nothing was watched', () => { - expect(() => { fs.unwatchFile() }).not.toThrow() - }) -}) diff --git a/packages/experimental/webworker-runtime/tests/node/sandbox-stack.spec.ts b/packages/experimental/webworker-runtime/tests/node/sandbox-stack.spec.ts new file mode 100644 index 0000000000..599608d598 --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/node/sandbox-stack.spec.ts @@ -0,0 +1,98 @@ +/** The unchanged sandbox-local → bash-sandbox → subprocess stack over the Worker Node layer. */ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { Context } from '@deepseek-ai/cordis' +import { SandboxBashExecutor } from '@deepseek-ai/dsh-bash-sandbox' +import LocalSandboxProvider from '@deepseek-ai/dsh-sandbox-local' +import { SandboxPolicyService } from '@deepseek-ai/dsh-sandbox-policy' +import LocalSubprocessRuntime from '@deepseek-ai/dsh-subprocess-local' +import { MemoryVfs } from '../../src/storage/memory.ts' +import { setActiveVfs } from '../../src/storage/active.ts' +import { processAlive, signalProcess } from '../../src/node/process-table.ts' + +vi.mock('node:child_process', async () => await import('../../src/node/builtin_modules/implemented/child_process.ts')) + +const WORKSPACE = '/dsh/workspace' +const OUTSIDE = '/dsh/home' +let vfs: MemoryVfs +const contexts: Context[] = [] + +beforeEach(() => { + vfs = new MemoryVfs() + setActiveVfs(vfs) + vfs.mkdirSync(WORKSPACE, { recursive: true }) + vfs.mkdirSync(OUTSIDE, { recursive: true }) + vfs.mkdirSync('/dsh/tmp', { recursive: true }) + vi.spyOn(process, 'kill').mockImplementation((pid: number, signal?: string | number): true => { + if (signal === 0) { + if (processAlive(pid)) return true + const error = new Error('kill ESRCH') as NodeJS.ErrnoException + error.code = 'ESRCH' + throw error + } + signalProcess(pid, (signal ?? 'SIGTERM') as NodeJS.Signals) + return true + }) +}) + +afterEach(async () => { + await Promise.all(contexts.splice(0).map(async (ctx) => { await ctx.fiber.dispose() })) + vi.restoreAllMocks() +}) + +/** Boot the production providers while only their platform primitives are replaced. */ +async function setup(mode: 'read-only' | 'workspace-write' | 'danger-full-access'): Promise { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(LocalSandboxProvider) + await ctx.plugin(SandboxPolicyService, { mode, workspaceRoot: WORKSPACE }) + await ctx.plugin(LocalSubprocessRuntime) + await ctx.plugin(SandboxBashExecutor, { cwd: WORKSPACE }) + return ctx.shell as SandboxBashExecutor +} + +describe('Worker Landlock through the production sandbox stack', () => { + it('allows workspace and temp writes while classifying an outside write as denied', async () => { + const bash = await setup('workspace-write') + const allowed = await bash.run(bash.resolve({ + command: `echo workspace > ${WORKSPACE}/allowed.txt; echo temp > /tmp/allowed.txt`, + })) + expect(allowed.sandbox).toEqual({ mode: 'workspace-write', denied: false, enforcement: 'full' }) + expect(vfs.readFileSync(`${WORKSPACE}/allowed.txt`, 'utf8')).toBe('workspace\n') + expect(vfs.readFileSync('/dsh/tmp/allowed.txt', 'utf8')).toBe('temp\n') + + const denied = await bash.run(bash.resolve({ command: `echo denied > ${OUTSIDE}/denied.txt` })) + expect(denied.exitCode).toBe(1) + expect(denied.sandbox).toEqual({ mode: 'workspace-write', denied: true, enforcement: 'full' }) + expect(vfs.existsSync(`${OUTSIDE}/denied.txt`)).toBe(false) + }) + + it('keeps read-only confined and danger-full-access unwrapped', async () => { + const readOnly = await setup('read-only') + const strict = await readOnly.run(readOnly.resolve({ + command: `echo discarded > /dev/null; echo denied > ${WORKSPACE}/strict.txt`, + })) + expect(strict.sandbox).toEqual({ mode: 'read-only', denied: true, enforcement: 'full' }) + expect(vfs.existsSync(`${WORKSPACE}/strict.txt`)).toBe(false) + + const unrestricted = await setup('danger-full-access') + const result = await unrestricted.run(unrestricted.resolve({ command: `echo allowed > ${OUTSIDE}/full.txt` })) + expect(result.sandbox).toEqual({ mode: 'danger-full-access', denied: false }) + expect(vfs.readFileSync(`${OUTSIDE}/full.txt`, 'utf8')).toBe('allowed\n') + }) + + it('does not leak a concurrent command policy into another process', async () => { + const bash = await setup('read-only') + const strict = bash.run(bash.resolve({ + command: `sleep 0.02; echo denied > ${WORKSPACE}/strict.txt`, + })) + const writable = bash.run(bash.resolve({ + command: `echo allowed > ${WORKSPACE}/writable.txt`, + sandboxPolicy: { mode: 'workspace-write', workspaceRoot: WORKSPACE }, + })) + const [strictResult, writableResult] = await Promise.all([strict, writable]) + expect(strictResult.sandbox).toEqual({ mode: 'read-only', denied: true, enforcement: 'full' }) + expect(writableResult.sandbox).toEqual({ mode: 'workspace-write', denied: false, enforcement: 'full' }) + expect(vfs.existsSync(`${WORKSPACE}/strict.txt`)).toBe(false) + expect(vfs.readFileSync(`${WORKSPACE}/writable.txt`, 'utf8')).toBe('allowed\n') + }) +}) diff --git a/packages/experimental/webworker-runtime/tests/storage/memory-vfs.spec.ts b/packages/experimental/webworker-runtime/tests/storage/memory-vfs.spec.ts index 1068cfdf21..b80987cd4c 100644 --- a/packages/experimental/webworker-runtime/tests/storage/memory-vfs.spec.ts +++ b/packages/experimental/webworker-runtime/tests/storage/memory-vfs.spec.ts @@ -1,6 +1,7 @@ /** - * The identity, timestamp, and link guarantees MemoryVfs owes its consumers, - * asserted on the filesystem directly rather than through the `node:fs` bridge. + * The identity, timestamp, link, mutation, and durability-sink guarantees + * MemoryVfs owes its consumers, asserted directly rather than through the + * `node:fs` bridge. * * `dsh-fs-local` builds a version token from `dev:ino:size:mtimeNs:ctimeNs` and * refuses a write whose token moved since it read. Two properties carry that: @@ -11,11 +12,14 @@ */ import { afterEach, describe, expect, it, vi } from 'vitest' import { MemoryVfs } from '../../src/storage/memory.ts' -import type { VfsBigIntStats, VfsStats } from '../../src/storage/types.ts' +import type { VfsBigIntStats, VfsMutation, VfsMutationSink, VfsStats } from '../../src/storage/types.ts' const identity = (vfs: MemoryVfs, path: string): bigint => (vfs.statSync(path, { bigint: true }) as VfsBigIntStats).ino +const linkCount = (vfs: MemoryVfs, path: string): bigint => + (vfs.statSync(path, { bigint: true }) as VfsBigIntStats).nlink + const modified = (vfs: MemoryVfs, path: string): number => (vfs.statSync(path) as VfsStats).mtimeMs afterEach(() => { vi.restoreAllMocks() }) @@ -39,10 +43,7 @@ describe('entry identity', () => { expect(identity(vfs, '/dsh/skills/git/SKILL.md')).not.toBe(before) }) - it('assigns the destination of a rename an identity of its own', () => { - // Identity belongs to the path, not to the bytes: a renamed-over path must - // stop looking like the entry it replaced, which is the property the guard - // reads. The source identity deliberately does not follow the move. + it('moves the source identity when a file replaces another path', () => { const vfs = new MemoryVfs() vfs.seed('/dsh/from.txt', 'moved') vfs.seed('/dsh/to.txt', 'replaced') @@ -50,11 +51,21 @@ describe('entry identity', () => { vfs.renameSync('/dsh/from.txt', '/dsh/to.txt') const renamed = identity(vfs, '/dsh/to.txt') expect(vfs.readFileSync('/dsh/to.txt', 'utf8')).toBe('moved') - expect([renamed === source, renamed === destination]).toEqual([false, false]) + expect([renamed === source, renamed === destination]).toEqual([true, false]) }) }) describe('modification time', () => { + it('hydrates explicit metadata without confusing timestamps with permission bits', () => { + const vfs = new MemoryVfs() + vfs.seed('/dsh/restored', 'value', { mode: 0o600, mtimeMs: 1_600_000_000_000 }) + vfs.seedDirectory('/dsh/restored-directory', { mode: 0o700, mtimeMs: 1_600_000_000_001 }) + const stats = vfs.statSync('/dsh/restored') as VfsStats + const directory = vfs.statSync('/dsh/restored-directory') as VfsStats + expect([stats.mode & 0o777, stats.mtimeMs]).toEqual([0o600, 1_600_000_000_000]) + expect([directory.mode & 0o777, directory.mtimeMs]).toEqual([0o700, 1_600_000_000_001]) + }) + it('advances on every write even while the clock stands still', () => { vi.spyOn(Date, 'now').mockReturnValue(1_700_000_000_000) const vfs = new MemoryVfs() @@ -80,16 +91,257 @@ describe('modification time', () => { vfs.writeFileSync('/dsh/log.jsonl', 'second\n') expect(modified(vfs, '/dsh/log.jsonl')).toBe(1_700_000_005_000) }) + + it('extends truncation with zero bytes', async () => { + const vfs = new MemoryVfs() + vfs.seed('/dsh/file', new Uint8Array([1, 2])) + vfs.truncateSync('/dsh/file', 5) + expect([...vfs.readFileSync('/dsh/file') as Uint8Array]).toEqual([1, 2, 0, 0, 0]) + const handle = vfs.open('/dsh/file', 'r+') + await handle.truncate(7) + expect([...vfs.readFileSync('/dsh/file') as Uint8Array]).toEqual([1, 2, 0, 0, 0, 0, 0]) + }) + + it('advances a directory only when its immediate entry set changes', () => { + vi.spyOn(Date, 'now').mockReturnValue(1_700_000_000_000) + const vfs = new MemoryVfs() + vfs.seedDirectory('/dsh/workspace') + const empty = modified(vfs, '/dsh/workspace') + vfs.writeFileSync('/dsh/workspace/file.txt', 'one') + const created = modified(vfs, '/dsh/workspace') + vfs.writeFileSync('/dsh/workspace/file.txt', 'two') + const rewritten = modified(vfs, '/dsh/workspace') + vfs.rmSync('/dsh/workspace/file.txt') + const removed = modified(vfs, '/dsh/workspace') + expect([created > empty, rewritten === created, removed > rewritten]).toEqual([true, true, true]) + }) +}) + +describe('mutation publication', () => { + it('publishes only committed runtime changes and keeps image seeding silent', () => { + const vfs = new MemoryVfs() + const mutations: VfsMutation[] = [] + vfs.subscribe((mutation) => { mutations.push(mutation) }) + vfs.seed('/dsh/seeded.txt', 'seeded') + expect(mutations).toEqual([]) + vfs.writeFileSync('/dsh/seeded.txt', 'changed') + vfs.mkdirSync('/dsh/created') + vfs.chmodSync('/dsh/created', 0o700) + vfs.renameSync('/dsh/seeded.txt', '/dsh/renamed.txt') + vfs.rmSync('/dsh/created', { recursive: true }) + expect(mutations.map(mutation => ({ + kind: mutation.kind, + path: mutation.path, + ...mutation.kind === 'write' ? { entryChanged: mutation.entryChanged } : {}, + ...mutation.kind === 'chmod' ? { mode: mutation.mode } : {}, + }))).toEqual([ + { kind: 'write', path: '/dsh/seeded.txt', entryChanged: false }, + { kind: 'mkdir', path: '/dsh/created' }, + { kind: 'chmod', path: '/dsh/created', mode: 0o700 }, + { kind: 'remove', path: '/dsh/seeded.txt' }, + { kind: 'write', path: '/dsh/renamed.txt', entryChanged: true }, + { kind: 'remove', path: '/dsh/created' }, + ]) + const renamed = mutations[4] + expect(renamed?.kind === 'write' && new TextDecoder().decode(renamed.bytes)).toBe('changed') + expect(() => { vfs.writeFileSync('/missing/file', 'no') }).toThrow(/ENOENT/) + expect(mutations).toHaveLength(6) + }) + + it('contains a faulty observer and lets disposal stop later notifications', () => { + const vfs = new MemoryVfs() + vfs.seedDirectory('/dsh') + const reported = vi.spyOn(console, 'error').mockImplementation(() => {}) + const first = vfs.subscribe(() => { throw new Error('observer failed') }) + const seen: string[] = [] + const second = vfs.subscribe((mutation) => { seen.push(mutation.path) }) + vfs.writeFileSync('/dsh/one', '1') + first() + second() + vfs.writeFileSync('/dsh/two', '2') + expect(seen).toEqual(['/dsh/one']) + expect(reported).toHaveBeenCalledOnce() + }) + + it('feeds the same complete mutations to a durable sink and live subscribers', async () => { + const recorded: VfsMutation[] = [] + let flushes = 0 + const sink: VfsMutationSink = { + record: (mutation) => { recorded.push(mutation) }, + flush: async () => { flushes += 1 }, + } + const vfs = new MemoryVfs({ sink }) + vfs.seedDirectory('/dsh') + const observed: VfsMutation[] = [] + vfs.subscribe((mutation) => { observed.push(mutation) }) + vfs.writeFileSync('/dsh/log', 'a') + vfs.appendFileSync('/dsh/log', 'bc') + await vfs.flush() + expect(observed).toEqual(recorded) + expect(observed[0]).toBe(recorded[0]) + expect(recorded[0]).toMatchObject({ kind: 'write', path: '/dsh/log', mode: 0o644, entryChanged: true }) + expect(recorded[1]).toMatchObject({ kind: 'write', path: '/dsh/log', mode: 0o644, entryChanged: false, appendedFrom: 1 }) + expect(recorded[1]?.kind === 'write' && new TextDecoder().decode(recorded[1].bytes)).toBe('abc') + expect(flushes).toBe(1) + }) + + it('publishes descriptor writes at the file identity current path', () => { + const mutations: VfsMutation[] = [] + const vfs = new MemoryVfs() + vfs.seed('/dsh/source', 'old') + const descriptor = vfs.openFileSync('/dsh/source', 'r+') + vfs.subscribe((mutation) => { mutations.push(mutation) }) + vfs.renameSync('/dsh/source', '/dsh/destination') + mutations.length = 0 + descriptor.write(0, new TextEncoder().encode('new')) + expect(mutations.map(mutation => mutation.path)).toEqual(['/dsh/destination']) + expect(vfs.readFileSync('/dsh/destination', 'utf8')).toBe('new') + vfs.unlinkSync('/dsh/destination') + mutations.length = 0 + descriptor.write(0, new TextEncoder().encode('detached')) + expect(mutations).toEqual([]) + expect(new TextDecoder().decode(descriptor.read(0, descriptor.stat().size))).toBe('detached') + }) + + it('decomposes a directory rename into replayable destination state', () => { + const recorded: VfsMutation[] = [] + const vfs = new MemoryVfs({ + sink: { record: (mutation) => { recorded.push(mutation) }, flush: () => Promise.resolve() }, + }) + vfs.seedDirectory('/dsh/staging/nested', { mode: 0o700 }) + vfs.seed('/dsh/staging/nested/file', 'value', { mode: 0o600 }) + vfs.renameSync('/dsh/staging', '/dsh/published') + + expect(recorded.map(mutation => [mutation.kind, mutation.path])).toEqual([ + ['remove', '/dsh/staging'], + ['mkdir', '/dsh/published'], + ['mkdir', '/dsh/published/nested'], + ['write', '/dsh/published/nested/file'], + ]) + expect(recorded[3]).toMatchObject({ kind: 'write', mode: 0o600, entryChanged: true }) + expect(recorded[3]?.kind === 'write' && new TextDecoder().decode(recorded[3].bytes)).toBe('value') + }) +}) + +describe('directory rename', () => { + it('rejects file, non-empty directory, and missing-parent destinations before mutation', () => { + const vfs = new MemoryVfs() + vfs.seed('/dsh/source/nested/file', 'source') + vfs.seed('/dsh/file', 'destination') + vfs.seed('/dsh/non-empty/child', 'destination') + const mutations: VfsMutation[] = [] + vfs.subscribe((mutation) => { mutations.push(mutation) }) + + expect(() => { vfs.renameSync('/dsh/source', '/dsh/file') }) + .toThrow(expect.objectContaining({ code: 'ENOTDIR' })) + expect(() => { vfs.renameSync('/dsh/source', '/dsh/non-empty') }) + .toThrow(expect.objectContaining({ code: 'ENOTEMPTY' })) + expect(() => { vfs.renameSync('/dsh/source', '/missing/destination') }) + .toThrow(expect.objectContaining({ code: 'ENOENT' })) + + expect(vfs.readFileSync('/dsh/source/nested/file', 'utf8')).toBe('source') + expect(vfs.readFileSync('/dsh/file', 'utf8')).toBe('destination') + expect(vfs.readFileSync('/dsh/non-empty/child', 'utf8')).toBe('destination') + expect(mutations).toEqual([]) + }) + + it('replaces an empty directory with the source subtree', () => { + const vfs = new MemoryVfs() + vfs.seedDirectory('/dsh/source/nested', { mode: 0o700 }) + vfs.seed('/dsh/source/nested/file', 'source') + vfs.seedDirectory('/dsh/destination', { mode: 0o711 }) + + vfs.renameSync('/dsh/source', '/dsh/destination') + + expect(vfs.existsSync('/dsh/source')).toBe(false) + expect(vfs.readFileSync('/dsh/destination/nested/file', 'utf8')).toBe('source') + expect((vfs.statSync('/dsh/destination') as VfsStats).mode & 0o777).toBe(0o755) + expect((vfs.statSync('/dsh/destination/nested') as VfsStats).mode & 0o777).toBe(0o700) + }) }) describe('hard links', () => { - it('shares the bytes present at link time and diverges on the next write', () => { + it('shares identity, bytes, and mode until one name is removed', () => { const vfs = new MemoryVfs() vfs.seed('/dsh/session.jsonl', 'committed\n') vfs.linkSync('/dsh/session.jsonl', '/dsh/session-latest.jsonl') + vfs.linkSync('/dsh/session-latest.jsonl', '/dsh/session-archive.jsonl') + expect(identity(vfs, '/dsh/session-latest.jsonl')).toBe(identity(vfs, '/dsh/session.jsonl')) + expect(linkCount(vfs, '/dsh/session.jsonl')).toBe(3n) expect(vfs.readFileSync('/dsh/session-latest.jsonl', 'utf8')).toBe('committed\n') + const changedPaths: string[] = [] + vfs.subscribe((mutation) => { changedPaths.push(mutation.path) }) vfs.appendFileSync('/dsh/session.jsonl', 'appended\n') + expect(changedPaths).toEqual([ + '/dsh/session.jsonl', + '/dsh/session-latest.jsonl', + '/dsh/session-archive.jsonl', + ]) expect(vfs.readFileSync('/dsh/session.jsonl', 'utf8')).toBe('committed\nappended\n') - expect(vfs.readFileSync('/dsh/session-latest.jsonl', 'utf8')).toBe('committed\n') + expect(vfs.readFileSync('/dsh/session-latest.jsonl', 'utf8')).toBe('committed\nappended\n') + vfs.chmodSync('/dsh/session-latest.jsonl', 0o600) + expect((vfs.statSync('/dsh/session.jsonl') as VfsStats).mode & 0o777).toBe(0o600) + vfs.unlinkSync('/dsh/session-latest.jsonl') + expect(linkCount(vfs, '/dsh/session.jsonl')).toBe(2n) + vfs.unlinkSync('/dsh/session-archive.jsonl') + expect(linkCount(vfs, '/dsh/session.jsonl')).toBe(1n) + expect(vfs.readFileSync('/dsh/session.jsonl', 'utf8')).toBe('committed\nappended\n') + }) + + it('treats rename between names of the same node as a no-op', () => { + const vfs = new MemoryVfs() + vfs.seed('/dsh/source', 'value') + vfs.linkSync('/dsh/source', '/dsh/alias') + const mutations: VfsMutation[] = [] + vfs.subscribe((mutation) => { mutations.push(mutation) }) + + vfs.renameSync('/dsh/source', '/dsh/alias') + + expect(vfs.readFileSync('/dsh/source', 'utf8')).toBe('value') + expect(vfs.readFileSync('/dsh/alias', 'utf8')).toBe('value') + expect(linkCount(vfs, '/dsh/source')).toBe(2n) + expect(mutations).toEqual([]) + }) + + it('retargets linked names through file replacement and directory moves', () => { + const vfs = new MemoryVfs() + vfs.seed('/dsh/replacement', 'replacement') + vfs.seed('/dsh/target', 'old') + vfs.linkSync('/dsh/target', '/dsh/target-alias') + const replaced = vfs.openFileSync('/dsh/target', 'r+') + vfs.renameSync('/dsh/replacement', '/dsh/target') + const mutations: VfsMutation[] = [] + vfs.subscribe((mutation) => { mutations.push(mutation) }) + + replaced.write(0, new TextEncoder().encode('changed')) + expect(mutations.map(mutation => mutation.path)).toEqual(['/dsh/target-alias']) + expect(vfs.readFileSync('/dsh/target', 'utf8')).toBe('replacement') + expect(vfs.readFileSync('/dsh/target-alias', 'utf8')).toBe('changed') + expect(linkCount(vfs, '/dsh/target-alias')).toBe(1n) + + vfs.seed('/dsh/tree/file', 'tree') + vfs.linkSync('/dsh/tree/file', '/dsh/outside') + const moved = vfs.openFileSync('/dsh/tree/file', 'r+') + vfs.renameSync('/dsh/tree', '/dsh/moved') + mutations.length = 0 + moved.write(0, new TextEncoder().encode('moved')) + expect(mutations.map(mutation => mutation.path)).toEqual(['/dsh/outside', '/dsh/moved/file']) + expect(linkCount(vfs, '/dsh/moved/file')).toBe(2n) + + vfs.rmSync('/dsh/moved', { recursive: true }) + mutations.length = 0 + moved.write(0, new TextEncoder().encode('kept!')) + expect(mutations.map(mutation => mutation.path)).toEqual(['/dsh/outside']) + expect(vfs.readFileSync('/dsh/outside', 'utf8')).toBe('kept!') + expect(linkCount(vfs, '/dsh/outside')).toBe(1n) + }) + + it('rejects renaming a file over an existing directory', () => { + const vfs = new MemoryVfs() + vfs.seed('/dsh/file', 'value') + vfs.seedDirectory('/dsh/directory') + expect(() => { vfs.renameSync('/dsh/file', '/dsh/directory') }).toThrow(expect.objectContaining({ code: 'EISDIR' })) + expect(vfs.readFileSync('/dsh/file', 'utf8')).toBe('value') + expect(vfs.statSync('/dsh/directory').isDirectory()).toBe(true) }) }) diff --git a/packages/experimental/webworker-runtime/tests/storage/tar.spec.ts b/packages/experimental/webworker-runtime/tests/storage/tar.spec.ts index 47aee5ec6b..efde651e9c 100644 --- a/packages/experimental/webworker-runtime/tests/storage/tar.spec.ts +++ b/packages/experimental/webworker-runtime/tests/storage/tar.spec.ts @@ -4,7 +4,7 @@ */ import { describe, expect, it } from 'vitest' import { packTar, parseTar } from '../../src/storage/tar.ts' -import { loadVfsImage } from '../../src/storage/memory.ts' +import { loadVfsImage, loadVfsOverlay } from '../../src/storage/memory.ts' const encoder = new TextEncoder() @@ -38,4 +38,21 @@ describe('tar codec', () => { expect(vfs.existsSync('/dsh/workspace')).toBe(true) expect(vfs.existsSync('/dsh/absent')).toBe(false) }) + + it('applies ordered data overlays without exposing runtime paths', () => { + const vfs = loadVfsImage(packTar({ + 'config/cordis.yml': encoder.encode('- id: subject\n'), + 'workspace/status.txt': encoder.encode('base'), + }), '/dsh') + loadVfsOverlay(packTar({ + 'workspace/status.txt': encoder.encode('fixture'), + 'home/sessions/example/session.jsonl': encoder.encode('{}\n'), + }), '/dsh', vfs) + expect(vfs.readFileSync('/dsh/workspace/status.txt', 'utf8')).toBe('fixture') + expect(vfs.readFileSync('/dsh/home/sessions/example/session.jsonl', 'utf8')).toBe('{}\n') + expect(() => loadVfsOverlay(packTar({ + 'config/cordis.yml': encoder.encode('replaced'), + }), '/dsh', vfs)).toThrow(/overlay entry must stay under home\/ or workspace/) + expect(vfs.readFileSync('/dsh/config/cordis.yml', 'utf8')).toBe('- id: subject\n') + }) }) diff --git a/packages/experimental/webworker-runtime/tests/transport/frames.spec.ts b/packages/experimental/webworker-runtime/tests/transport/frames.spec.ts new file mode 100644 index 0000000000..9519962309 --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/transport/frames.spec.ts @@ -0,0 +1,22 @@ +import { describe, expect, it } from 'vitest' +import { parseInboundFrame } from '../../src/transport/frames.ts' + +describe('tunnel init frame', () => { + it('retains the selected overlay order', () => { + expect(parseInboundFrame({ + t: 'init', + image: 'base.tar.gz', + overlays: ['workspace.tar.gz', 'session.tar.gz'], + })).toEqual({ + t: 'init', + image: 'base.tar.gz', + overlays: ['workspace.tar.gz', 'session.tar.gz'], + }) + }) + + it('rejects a missing or non-string overlay list', () => { + expect(() => parseInboundFrame({ t: 'init', image: 'base.tar.gz' })).toThrow(/array of string overlay urls/) + expect(() => parseInboundFrame({ t: 'init', image: 'base.tar.gz', overlays: [1] })) + .toThrow(/array of string overlay urls/) + }) +}) diff --git a/packages/experimental/webworker-runtime/tests/transport/tunnel-client.spec.ts b/packages/experimental/webworker-runtime/tests/transport/tunnel-client.spec.ts index ceacba25df..64e550dc48 100644 --- a/packages/experimental/webworker-runtime/tests/transport/tunnel-client.spec.ts +++ b/packages/experimental/webworker-runtime/tests/transport/tunnel-client.spec.ts @@ -54,6 +54,27 @@ function stubWorker(): { } } +// The opening frame preserves overlay order for deterministic pre-boot mounts. +{ + const { worker, sent } = stubWorker() + const tunnel = new WorkerTunnel(worker) + tunnel.init('https://preview.test/base.tar.gz', [ + 'https://preview.test/first.tar.gz', + 'https://preview.test/second.tar.gz', + ]) + check('the init frame carries ordered overlays', sent[0], { + t: 'init', + image: 'https://preview.test/base.tar.gz', + overlays: ['https://preview.test/first.tar.gz', 'https://preview.test/second.tar.gz'], + }) + + const direct = stubWorker() + new WorkerTunnel(direct.worker).init('https://preview.test/base.tar.gz') + check('the direct init path defaults to no overlays', direct.sent[0], { + t: 'init', image: 'https://preview.test/base.tar.gz', overlays: [], + }) +} + // A normal reply resolves and says nothing on the console. { const { worker, sent, deliver } = stubWorker() diff --git a/packages/experimental/webworker-runtime/tests/vfs-example-fixture.spec.ts b/packages/experimental/webworker-runtime/tests/vfs-example-fixture.spec.ts new file mode 100644 index 0000000000..561cf04699 --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/vfs-example-fixture.spec.ts @@ -0,0 +1,118 @@ +import { readFileSync, readdirSync } from 'node:fs' +import { join, relative } from 'node:path' +import { describe, expect, it } from 'vitest' +import { Session, SessionId, type SessionEvent } from '@deepseek-ai/dsh-session' +import { scanLog } from '@deepseek-ai/dsh-session-persistence-jsonl/src/format.ts' +import { foldSubagentDescriptor } from '@deepseek-ai/dsh-subagent' +import { + buildVfsExampleFiles, + VFS_EXAMPLE_OLDEST_MESSAGE, + VFS_EXAMPLE_ROOT, + VFS_EXAMPLE_SESSION_IDS, + VFS_EXAMPLE_TAIL_MESSAGE, + VFS_EXAMPLE_TITLE, +} from './vfs-example-fixture.ts' + +function filesUnder(root: string): string[] { + const files: string[] = [] + const visit = (directory: string): void => { + for (const entry of readdirSync(directory, { withFileTypes: true })) { + const path = join(directory, entry.name) + if (entry.isDirectory()) visit(path) + else if (entry.isFile()) files.push(relative(root, path).replaceAll('\\', '/')) + } + } + visit(root) + return files.sort() +} + +function readSession(id: string): ReturnType { + return scanLog(readFileSync( + join(VFS_EXAMPLE_ROOT, 'home/sessions/--dsh-workspace--', id, 'session.jsonl'), + )) +} + +function textOf(event: SessionEvent): string { + if (event.type === 'user/message') { + return event.data.content.flatMap(block => block.type === 'text' ? [block.text] : []).join('\n') + } + if (event.type === 'assistant/message') { + return event.data.message.content.flatMap(block => block.type === 'text' ? [block.text] : []).join('\n') + } + return '' +} + +describe('WebWorker preview VFS example', () => { + it('matches its deterministic source byte for byte', () => { + const expected = buildVfsExampleFiles() + expect(filesUnder(VFS_EXAMPLE_ROOT)).toEqual([...expected.keys()].sort()) + for (const [path, content] of expected) { + expect(readFileSync(join(VFS_EXAMPLE_ROOT, path), 'utf8'), path).toBe(content) + } + }) + + it('seeds the cold-list title cache against the main log identity', () => { + const cache = JSON.parse(readFileSync( + join(VFS_EXAMPLE_ROOT, 'home/storages/session_projcache.json'), + 'utf8', + )) as { + unit: { name: string; version: number } + tables: { sessions: Record } + } + expect(cache.unit).toEqual({ name: 'session_projcache', version: 3 }) + expect(cache.tables.sessions[VFS_EXAMPLE_SESSION_IDS.main]).toMatchObject({ + identity: { createdAt: 1_787_472_000_000, cwd: '/dsh/workspace' }, + rows: { title: { ver: 1, val: VFS_EXAMPLE_TITLE } }, + }) + }) + + it('restores the main production log with paging and tool coverage', () => { + const { meta, events } = readSession(VFS_EXAMPLE_SESSION_IDS.main) + expect(meta).toMatchObject({ + id: VFS_EXAMPLE_SESSION_IDS.main, + cwd: '/dsh/workspace', + delegationDepth: 0, + agentPreset: 'standard', + }) + expect(events.map(event => event.seq)).toEqual(events.map((_, index) => index)) + expect(events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } }) + expect(() => Session.fromRestore(SessionId(meta.id), events, meta)).not.toThrow() + + const messages = events.filter(event => + (event.type === 'user/message' || event.type === 'assistant/message') && event.surfaceOp === 'append') + expect(messages.length).toBeGreaterThan(50) + expect(messages.some(event => textOf(event).includes(VFS_EXAMPLE_OLDEST_MESSAGE))).toBe(true) + expect(messages.some(event => textOf(event).includes(VFS_EXAMPLE_TAIL_MESSAGE))).toBe(true) + expect(events.some(event => event.type === 'session/title' + && (event.data as { title?: unknown }).title === VFS_EXAMPLE_TITLE)).toBe(true) + + const tools = events.flatMap(event => event.type === 'tool/call' ? [event.data.name] : []) + expect(new Set(tools)).toEqual(new Set([ + 'read', 'write', 'bash', 'glob', 'grep', 'web_search', 'todo_write', 'subagent', 'subagent_fork', + ])) + expect(events.some(event => event.type === 'todo/write')).toBe(true) + expect(events.some(event => event.type === 'tool/result' && event.data.message.content[0].isError === true)).toBe(true) + }) + + it('restores one-shot and continuable child Sessions with durable descriptors', () => { + const expected = [ + [VFS_EXAMPLE_SESSION_IDS.oneShot, 'one-shot'], + [VFS_EXAMPLE_SESSION_IDS.continuable, 'continuable'], + ] as const + for (const [id, mode] of expected) { + const { meta, events } = readSession(id) + expect(meta).toMatchObject({ + id, + cwd: '/dsh/workspace', + parentSession: VFS_EXAMPLE_SESSION_IDS.main, + origin: 'subagent', + delegationDepth: 1, + agentPreset: 'standard', + }) + expect(events.map(event => event.seq)).toEqual(events.map((_, index) => index)) + expect(events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } }) + expect(foldSubagentDescriptor(events.slice(meta.seedLength ?? 0))).toMatchObject({ mode }) + expect(() => Session.fromRestore(SessionId(meta.id), events, meta)).not.toThrow() + } + }) +}) diff --git a/packages/experimental/webworker-runtime/tests/vfs-example-fixture.ts b/packages/experimental/webworker-runtime/tests/vfs-example-fixture.ts new file mode 100644 index 0000000000..dbc151eb86 --- /dev/null +++ b/packages/experimental/webworker-runtime/tests/vfs-example-fixture.ts @@ -0,0 +1,432 @@ +/** Deterministic source for the filesystem tree bundled into the WebWorker preview. */ + +import { fileURLToPath } from 'node:url' +import { SessionId, type SessionEvent, type SessionHeader } from '@deepseek-ai/dsh-session' +import { + eventLines, projectKey, toHeaderLine, +} from '@deepseek-ai/dsh-session-persistence-jsonl/src/format.ts' +import { snapshotSubagentDescriptor } from '@deepseek-ai/dsh-subagent' + +/** Root copied by the preview image's repository adapter. */ +export const VFS_EXAMPLE_ROOT = fileURLToPath(new URL('./fixtures/vfs-example', import.meta.url)) + +/** Durable ids used by browser assertions and subagent parent links. */ +export const VFS_EXAMPLE_SESSION_IDS = { + main: SessionId('preview-showcase'), + oneShot: SessionId('preview-architecture-review'), + continuable: SessionId('preview-follow-up-builder'), +} as const + +/** Stable title rendered in the root Session list. */ +export const VFS_EXAMPLE_TITLE = 'WebWorker Preview Showcase' + +/** Oldest prompt, intentionally outside the first 50-message history page. */ +export const VFS_EXAMPLE_OLDEST_MESSAGE = 'History checkpoint 01: verify deterministic preview state.' + +/** Settled tail marker used by browser acceptance and the demonstration GIF. */ +export const VFS_EXAMPLE_TAIL_MESSAGE = 'Preview tour complete' + +const WORKSPACE = '/dsh/workspace' +const CREATED_AT = 1_787_472_000_000 +const HISTORICAL_TURNS = 28 + +const PREVIEW_GUIDE = `# Preview Workspace + +This deterministic workspace is bundled with the browser-only preview. + +- \`src/preview.ts\` is the file changed by the example write result. +- \`data/tasks.json\` mirrors the completed preview checklist. +- \`.agents/skills/preview-tour/SKILL.md\` proves dot directories survive image packing. + +Refresh the preview to restore these image bytes. +` + +const PREVIEW_SOURCE_BEFORE = 'export const previewStatus = \'draft\'\n' + +const PREVIEW_SOURCE = `export const previewStatus = 'ready' + +export const previewFeatures = ['tools', 'subagents', 'pagination'] as const +` + +const TASKS = `${JSON.stringify({ + title: 'Preview verification', + tasks: [ + { name: 'Inspect tool cards', status: 'completed' }, + { name: 'Open both subagents', status: 'completed' }, + { name: 'Load earlier history', status: 'completed' }, + ], +}, null, 2)}\n` + +const SKILL = `--- +name: preview-tour +description: Inspect the bundled Preview workspace and its deterministic Session examples. +--- + +# Preview tour + +Read the workspace files, inspect the tool gallery, open both subagent histories, and load the earlier conversation page. +` + +interface EventDraft { + readonly type: string + readonly data: unknown + readonly surfaceOp?: 'append' + readonly sourceEventSeqs?: number[] + readonly ignorable?: true +} + +class EventLog { + readonly events: SessionEvent[] + private nextTime: number + + constructor(time: number, seed: readonly SessionEvent[] = []) { + this.events = seed.map(event => structuredClone(event)) + this.nextTime = Math.max(time, (this.events.at(-1)?.time ?? time - 1) + 1) + } + + add(draft: EventDraft): number { + const seq = this.events.length + this.events.push({ ...draft, seq, time: this.nextTime++ } as unknown as SessionEvent) + return seq + } +} + +function userMessage(id: string, text: string): EventDraft { + return { + type: 'user/message', + data: { + id, + role: 'user', + content: [{ type: 'text', text }], + source: { kind: 'user' }, + }, + surfaceOp: 'append', + } +} + +function assistantMessage(id: string, turn: number, step: number, content: unknown[]): EventDraft { + return { + type: 'assistant/message', + data: { + turn, + step, + message: { + id, + role: 'assistant', + content, + source: { kind: 'model', provider: 'preview-fixture', model: 'deterministic' }, + }, + }, + sourceEventSeqs: [], + surfaceOp: 'append', + } +} + +interface GalleryCall { + readonly id: string + readonly name: string + readonly args: Record + readonly result: string + readonly meta?: unknown + readonly error?: { readonly name: string; readonly code: string } + readonly todos?: Array<{ readonly content: string; readonly status: 'pending' | 'in_progress' | 'completed' }> +} + +function readResult(): { text: string; meta: unknown } { + const lines = PREVIEW_GUIDE.trimEnd().split('\n').map((text, index) => ({ number: index + 1, text })) + return { + text: `PREVIEW.md\nfile\n\n${lines.map(line => `${String(line.number)}: ${line.text}`).join('\n')}\n\n(End of file - total ${String(lines.length)} lines)\n`, + meta: { path: 'PREVIEW.md', offset: 1, lines, totalLines: lines.length, lang: 'md' }, + } +} + +function galleryCalls(): GalleryCall[] { + const read = readResult() + return [ + { + id: 'preview-read', + name: 'read', + args: { file_path: 'PREVIEW.md' }, + result: read.text, + meta: read.meta, + }, + { + id: 'preview-write', + name: 'write', + args: { file_path: 'src/preview.ts', content: PREVIEW_SOURCE }, + result: 'src/preview.ts\nfile\n\nUpdated file\n', + meta: { diffs: [{ path: 'src/preview.ts', oldText: PREVIEW_SOURCE_BEFORE, newText: PREVIEW_SOURCE }] }, + }, + { + id: 'preview-bash', + name: 'bash', + args: { command: "printf 'preview ready\\n'", description: 'Print the preview readiness marker' }, + result: 'preview ready\n', + }, + { + id: 'preview-glob', + name: 'glob', + args: { pattern: '**/*', path: '.' }, + result: 'PREVIEW.md\ndata/tasks.json\nsrc/preview.ts', + meta: { + shape: 'paths', + paths: ['PREVIEW.md', 'data/tasks.json', 'src/preview.ts'], + truncated: false, + total: 3, + }, + }, + { + id: 'preview-grep', + name: 'grep', + args: { pattern: 'preview', path: '.', include: '*.{md,ts,json}' }, + result: 'PREVIEW.md:3:This deterministic workspace is bundled with the browser-only preview.\nsrc/preview.ts:1:export const previewStatus = \'ready\'', + meta: { + shape: 'matches', + files: [ + { path: 'PREVIEW.md', matches: [{ lineNumber: 3, line: 'This deterministic workspace is bundled with the browser-only preview.' }] }, + { path: 'src/preview.ts', matches: [{ lineNumber: 1, line: "export const previewStatus = 'ready'" }] }, + ], + truncated: false, + total: 2, + }, + }, + { + id: 'preview-web-search', + name: 'web_search', + args: { queries: ['Web Worker filesystem compatibility'] }, + result: 'Browser workers can host deterministic in-memory filesystems.\n\nSources:\n1. MDN Web Workers API — https://developer.mozilla.org/docs/Web/API/Web_Workers_API', + meta: { + sources: [{ + url: 'https://developer.mozilla.org/docs/Web/API/Web_Workers_API', + title: 'Web Workers API', + snippet: 'Web Workers run scripts in background threads.', + }], + truncated: false, + answer: 'Browser workers can host deterministic in-memory filesystems.', + }, + }, + { + id: 'preview-todo', + name: 'todo_write', + args: { + todos: [ + { content: 'Inspect tool cards', status: 'completed' }, + { content: 'Open both subagents', status: 'completed' }, + { content: 'Load earlier history', status: 'in_progress' }, + ], + }, + result: 'Updated todo list: 0 pending, 1 in progress, 2 completed.', + todos: [ + { content: 'Inspect tool cards', status: 'completed' }, + { content: 'Open both subagents', status: 'completed' }, + { content: 'Load earlier history', status: 'in_progress' }, + ], + }, + { + id: 'preview-subagent', + name: 'subagent', + args: { description: 'Continue preview verification', prompt: 'Check the remaining preview cases.', run_in_background: true }, + result: `started subagent ${VFS_EXAMPLE_SESSION_IDS.continuable}`, + }, + { + id: 'preview-subagent-fork', + name: 'subagent_fork', + args: { description: 'Review preview architecture', prompt: 'Review the fixture architecture.', run_in_background: false }, + result: 'The preview fixture remains separate from user-owned WebFS data.', + }, + { + id: 'preview-failure', + name: 'read', + args: { file_path: 'missing.txt' }, + result: 'Error: ENOENT: no such file, open missing.txt', + error: { name: 'FsError', code: 'ENOENT' }, + }, + ] +} + +function addClosedTextTurn(log: EventLog, turn: number): void { + const checkpoint = String(turn).padStart(2, '0') + log.add({ type: 'turn/start', data: { turn } }) + log.add(userMessage(`preview-user-${checkpoint}`, `History checkpoint ${checkpoint}: verify deterministic preview state.`)) + if (turn === 1) { + log.add({ + type: 'session/title', + data: { title: VFS_EXAMPLE_TITLE, messageSeqs: [], source: { kind: 'user' } }, + }) + } + log.add({ type: 'step/start', data: { turn, step: 1 } }) + log.add(assistantMessage( + `preview-assistant-${checkpoint}`, + turn, + 1, + [{ type: 'text', text: `Checkpoint ${checkpoint} is recorded.` }], + )) + log.add({ type: 'step/end', data: { turn, step: 1 } }) + log.add({ type: 'turn/end', data: { turn, reason: { kind: 'completed' } } }) +} + +function mainLog(): { readonly events: SessionEvent[]; readonly forkSeedLength: number } { + const log = new EventLog(CREATED_AT) + for (let turn = 1; turn <= HISTORICAL_TURNS; turn++) addClosedTextTurn(log, turn) + const forkSeedLength = log.events.length + const turn = HISTORICAL_TURNS + 1 + const calls = galleryCalls() + + log.add({ type: 'turn/start', data: { turn } }) + log.add(userMessage('preview-gallery-user', 'Show the seeded workspace, tool cards, subagents, and pagination in one tour.')) + log.add({ type: 'step/start', data: { turn, step: 1 } }) + log.add(assistantMessage('preview-gallery-tools', turn, 1, [ + { type: 'reasoning', text: 'I will inspect the deterministic workspace and collect each preview surface.' }, + ...calls.map(call => ({ type: 'tool-call', id: call.id, name: call.name, arguments: JSON.stringify(call.args) })), + ])) + for (const call of calls) { + log.add({ + type: 'tool/call', + data: { turn, step: 1, callId: call.id, name: call.name, arguments: JSON.stringify(call.args) }, + }) + if (call.todos !== undefined) log.add({ type: 'todo/write', data: { todos: call.todos } }) + log.add({ + type: 'tool/result', + data: { + turn, + step: 1, + message: { + id: `${call.id}-result`, + role: 'user', + content: [{ + type: 'tool-result', + toolCallId: call.id, + content: [{ type: 'text', text: call.result }], + isError: call.error !== undefined, + }], + source: { kind: 'tool', callId: call.id }, + }, + ...call.meta === undefined ? {} : { meta: call.meta }, + ...call.error === undefined ? {} : { error: call.error }, + }, + surfaceOp: 'append', + }) + } + log.add({ type: 'step/end', data: { turn, step: 1 } }) + log.add({ type: 'step/start', data: { turn, step: 2 } }) + log.add(assistantMessage('preview-gallery-final', turn, 2, [{ + type: 'text', + text: `## ${VFS_EXAMPLE_TAIL_MESSAGE}\n\nThe workspace, specialized tool cards, two subagent histories, and an earlier history page are ready to inspect.`, + }])) + log.add({ type: 'step/end', data: { turn, step: 2 } }) + log.add({ type: 'turn/end', data: { turn, reason: { kind: 'completed' } } }) + return { events: log.events, forkSeedLength } +} + +function oneShotLog(seed: readonly SessionEvent[]): SessionEvent[] { + const log = new EventLog(CREATED_AT + 100_000, seed) + log.add({ type: 'session/end-seed', data: {} }) + const turn = HISTORICAL_TURNS + 1 + log.add({ type: 'turn/start', data: { turn } }) + log.add(userMessage('preview-review-user', 'Review whether the preview fixture is isolated from future WebFS data.')) + log.add({ + type: 'subagent/descriptor', + data: snapshotSubagentDescriptor({ + mode: 'one-shot', provider: 'fork', label: 'Review preview architecture', + }), + }) + log.add({ type: 'step/start', data: { turn, step: 1 } }) + log.add(assistantMessage('preview-review-assistant', turn, 1, [{ + type: 'text', + text: 'The bundled fixture is static image content; future WebFS state remains user-owned.', + }])) + log.add({ type: 'step/end', data: { turn, step: 1 } }) + log.add({ type: 'turn/end', data: { turn, reason: { kind: 'completed' } } }) + return log.events +} + +function continuableLog(): SessionEvent[] { + const log = new EventLog(CREATED_AT + 200_000) + log.add({ type: 'turn/start', data: { turn: 1 } }) + log.add(userMessage('preview-builder-user', 'Check that the Preview workspace can support follow-up tasks.')) + log.add({ + type: 'subagent/descriptor', + data: snapshotSubagentDescriptor({ + mode: 'continuable', provider: 'spawn', label: 'Continue preview verification', + }), + }) + log.add({ type: 'step/start', data: { turn: 1, step: 1 } }) + log.add(assistantMessage('preview-builder-assistant', 1, 1, [{ + type: 'text', + text: 'This child is continuable and ready for another verification turn.', + }])) + log.add({ type: 'step/end', data: { turn: 1, step: 1 } }) + log.add({ type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }) + return log.events +} + +function header( + id: SessionHeader['id'], + createdAt: number, + child?: { readonly parentSession: SessionHeader['id']; readonly mode: 'one-shot' | 'continuable'; readonly seedLength?: number }, +): SessionHeader { + return { + version: 0, + id, + createdAt, + cwd: WORKSPACE, + delegationDepth: child === undefined ? 0 : 1, + agentPreset: 'standard', + ...child === undefined ? {} : { + parentSession: child.parentSession, + origin: 'subagent' as const, + ...child.seedLength === undefined ? {} : { seedLength: child.seedLength }, + }, + } +} + +function renderLog(meta: SessionHeader, events: readonly SessionEvent[]): string { + return `${JSON.stringify(toHeaderLine(meta))}\n${eventLines(events, true)}\n` +} + +/** Build every committed fixture file as repository-relative UTF-8 text. */ +export function buildVfsExampleFiles(): ReadonlyMap { + const main = mainLog() + const project = projectKey(WORKSPACE) + const sessionPath = (id: string): string => `home/sessions/${project}/${id}/session.jsonl` + const projectionCache = `${JSON.stringify({ + unit: { name: 'session_projcache', version: 3 }, + global: null, + tables: { + sessions: { + [VFS_EXAMPLE_SESSION_IDS.main]: { + identity: { createdAt: CREATED_AT, cwd: WORKSPACE }, + rows: { + title: { ver: 1, seq: main.events.at(-1)?.seq ?? -1, val: VFS_EXAMPLE_TITLE }, + }, + }, + }, + }, + }, null, 2)}\n` + return new Map([ + ['workspace/PREVIEW.md', PREVIEW_GUIDE], + ['workspace/src/preview.ts', PREVIEW_SOURCE], + ['workspace/data/tasks.json', TASKS], + ['workspace/.agents/skills/preview-tour/SKILL.md', SKILL], + ['home/storages/session_projcache.json', projectionCache], + [sessionPath(VFS_EXAMPLE_SESSION_IDS.main), renderLog( + header(VFS_EXAMPLE_SESSION_IDS.main, CREATED_AT), + main.events, + )], + [sessionPath(VFS_EXAMPLE_SESSION_IDS.oneShot), renderLog( + header(VFS_EXAMPLE_SESSION_IDS.oneShot, CREATED_AT + 100_000, { + parentSession: VFS_EXAMPLE_SESSION_IDS.main, + mode: 'one-shot', + seedLength: main.forkSeedLength, + }), + oneShotLog(main.events.slice(0, main.forkSeedLength)), + )], + [sessionPath(VFS_EXAMPLE_SESSION_IDS.continuable), renderLog( + header(VFS_EXAMPLE_SESSION_IDS.continuable, CREATED_AT + 200_000, { + parentSession: VFS_EXAMPLE_SESSION_IDS.main, + mode: 'continuable', + }), + continuableLog(), + )], + ]) +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index f5f3e827dd..e763188fd0 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -4814,6 +4814,9 @@ importers: picomatch: specifier: ^4.0.4 version: 4.0.4 + readable-stream: + specifier: ^4.7.0 + version: 4.7.0 devDependencies: '@deepseek-ai/cordis': specifier: workspace:^ @@ -4824,6 +4827,9 @@ importers: '@deepseek-ai/dsh-api-gateway': specifier: workspace:^ version: link:../../api/gateway + '@deepseek-ai/dsh-bash-sandbox': + specifier: workspace:^ + version: link:../../shell/bash-sandbox '@deepseek-ai/dsh-client-modules': specifier: workspace:^ version: link:../../client/modules @@ -4836,12 +4842,36 @@ importers: '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../runtime-diagnostics/invariants + '@deepseek-ai/dsh-sandbox-local': + specifier: workspace:^ + version: link:../../sandbox/sandbox-local + '@deepseek-ai/dsh-sandbox-policy': + specifier: workspace:^ + version: link:../../sandbox/sandbox-policy + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + '@deepseek-ai/dsh-session-persistence-jsonl': + specifier: workspace:^ + version: link:../../session/session-persistence-jsonl + '@deepseek-ai/dsh-subagent': + specifier: workspace:^ + version: link:../../subagent/subagent '@deepseek-ai/dsh-subprocess-local': specifier: workspace:^ version: link:../../subprocess/subprocess-local + '@deepseek-ai/node-addon-landlock-run': + specifier: workspace:^ + version: link:../../../native/landlock-run/packages/entry '@types/picomatch': specifier: ^3.0.2 version: 3.0.2 + '@types/readable-stream': + specifier: ^4.0.24 + version: 4.0.24 + chokidar: + specifier: ^5.0.0 + version: 5.0.0 packages/extensions/cordis-client-runner: devDependencies: @@ -12641,6 +12671,9 @@ packages: '@types/react@18.3.31': resolution: {integrity: sha512-vfEqpXTvwT91yhmwdfouStN2hSKwTvyRs8qpLfADyrq/kxDw0hZM7Wk9Ug1FELj8hIby+S/+kQCSRFF32nv2Qw==} + '@types/readable-stream@4.0.24': + resolution: {integrity: sha512-NRvUNC/JFGPJvqdAfEve8oginbM6V08u5NzLWpG8MwA2kTPOLnqk+wpwuPT+mp3aUsxyuT6m2gnrPuHYCruzEg==} + '@types/retry@0.12.0': resolution: {integrity: sha512-wWKOClTTiizcZhXnPY4wikVAwmdYHp8q6DmC+EJUzAMsycb7HB32Kh9RN4+0gExjmPmZSAQjgURXIGATPegAvA==} @@ -12892,6 +12925,10 @@ packages: resolution: {integrity: sha512-WoxUM/Be4hfsX06FxsvpGgfYqwgivMV7/Ol7aFuSfSmY6rRaiju4QxOEe9RUS0iYcSHWl5i9AhB1cMoE0p+XiA==} engines: {node: '>=18.12.0'} + abort-controller@3.0.0: + resolution: {integrity: sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg==} + engines: {node: '>=6.5'} + accepts@2.0.0: resolution: {integrity: sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng==} engines: {node: '>= 0.6'} @@ -13583,9 +13620,17 @@ packages: resolution: {integrity: sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg==} engines: {node: '>= 0.6'} + event-target-shim@5.0.1: + resolution: {integrity: sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ==} + engines: {node: '>=6'} + eventemitter3@4.0.7: resolution: {integrity: sha512-8guHBZCwKnFhYdHr2ysuRWErTwhoN2X8XELRlrRwpmfeY2jjuUN4taQMsULKUVo1K4DvZl+0pgfyoysHxvmvEw==} + events@3.3.0: + resolution: {integrity: sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==} + engines: {node: '>=0.8.x'} + eventsource-parser@3.1.0: resolution: {integrity: sha512-kJezFj9YFAMLeORyi7aCLxLbD5/qWMQnoMVlVPyHIll7lgRJCc3JVln9Vgl9nwQi0YkMnhdGTMNn7CkRRAptMg==} engines: {node: '>=18.0.0'} @@ -14818,6 +14863,10 @@ packages: process-nextick-args@2.0.1: resolution: {integrity: sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==} + process@0.11.10: + resolution: {integrity: sha512-cdGef/drWFoydD1JsMzuFf8100nZl+GT+yacc2bEced5f9Rjk4z+WtFUTBu9PhOi9j/jfmBPu0mMEY4wIdAF8A==} + engines: {node: '>= 0.6.0'} + property-information@7.2.0: resolution: {integrity: sha512-IAtzIB6sUiWaJYrX9smp3V46pBGbBeLFRGdh25kg1334VcBlD8HzhPeNIWQH9zhGmo2itIe25EHt9dQP7G5hmg==} @@ -14875,6 +14924,10 @@ packages: readable-stream@2.3.8: resolution: {integrity: sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA==} + readable-stream@4.7.0: + resolution: {integrity: sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg==} + engines: {node: ^12.22.0 || ^14.17.0 || >=16.0.0} + readdirp@4.1.2: resolution: {integrity: sha512-GDhwkLfywWL2s6vEjyhri+eXmfH6j1L7JE27WhqLeYzoh/A3DBaYGEj2H/HFZCn/kMfim73FXxEJTw06WtxQwg==} engines: {node: '>= 14.18.0'} @@ -15122,6 +15175,9 @@ packages: string_decoder@1.1.1: resolution: {integrity: sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==} + string_decoder@1.3.0: + resolution: {integrity: sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==} + stringify-entities@4.0.4: resolution: {integrity: sha512-IwfBptatlO+QCJUo19AqvrPNqlVMpW9YEL2LIVY+Rpv2qsjCGxaDLNRgeGsQWJhfItebuJhsGSLjaBbNSQ+ieg==} @@ -17914,6 +17970,10 @@ snapshots: '@types/prop-types': 15.7.15 csstype: 3.2.3 + '@types/readable-stream@4.0.24': + dependencies: + '@types/node': 22.20.0 + '@types/retry@0.12.0': {} '@types/spdx-expression-parse@4.0.0': {} @@ -18191,6 +18251,10 @@ snapshots: js-yaml: 4.3.1 tslib: 2.8.1 + abort-controller@3.0.0: + dependencies: + event-target-shim: 5.0.1 + accepts@2.0.0: dependencies: mime-types: 3.0.2 @@ -18951,8 +19015,12 @@ snapshots: etag@1.8.1: {} + event-target-shim@5.0.1: {} + eventemitter3@4.0.7: {} + events@3.3.0: {} + eventsource-parser@3.1.0: {} eventsource@3.0.7: @@ -20426,6 +20494,8 @@ snapshots: process-nextick-args@2.0.1: {} + process@0.11.10: {} + property-information@7.2.0: {} protobufjs@7.6.4: @@ -20498,6 +20568,14 @@ snapshots: string_decoder: 1.1.1 util-deprecate: 1.0.2 + readable-stream@4.7.0: + dependencies: + abort-controller: 3.0.0 + buffer: 6.0.3 + events: 3.3.0 + process: 0.11.10 + string_decoder: 1.3.0 + readdirp@4.1.2: {} readdirp@5.0.0: {} @@ -20840,6 +20918,10 @@ snapshots: dependencies: safe-buffer: 5.1.2 + string_decoder@1.3.0: + dependencies: + safe-buffer: 5.2.1 + stringify-entities@4.0.4: dependencies: character-entities-html4: 2.1.0 diff --git a/scripts/session-fixture-layout.spec.ts b/scripts/session-fixture-layout.spec.ts index c0b87953b7..05dfd7a4d0 100644 --- a/scripts/session-fixture-layout.spec.ts +++ b/scripts/session-fixture-layout.spec.ts @@ -1,7 +1,7 @@ import { describe, expect, it } from 'vitest' import { type SessionEvent } from '@deepseek-ai/dsh-session' import { parseSessionLog } from '@deepseek-ai/dsh-llm-replay' -import { canonicalSessionFixture } from './session-fixture-layout.ts' +import { canonicalSessionFixture, isPhysicalSessionFixture } from './session-fixture-layout.ts' const HEADER = ' {"type":"session","version":0,"id":"fixture","createdAt":1,"delegationDepth":0} ' @@ -68,3 +68,15 @@ describe('canonicalSessionFixture', () => { .toThrow(/broken\.jsonl: session snapshot line 2: malformed text-chunks storage row/) }) }) + +describe('isPhysicalSessionFixture', () => { + it('excludes only persisted logs under the WebWorker example root', () => { + expect(isPhysicalSessionFixture( + 'packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/main/session.jsonl', + )).toBe(true) + expect(isPhysicalSessionFixture( + 'packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/README.jsonl', + )).toBe(false) + expect(isPhysicalSessionFixture('apps/web/tests/snapshots/example/session.jsonl')).toBe(false) + }) +}) diff --git a/scripts/session-fixture-layout.ts b/scripts/session-fixture-layout.ts index 996b40fbca..e26cb7f90b 100644 --- a/scripts/session-fixture-layout.ts +++ b/scripts/session-fixture-layout.ts @@ -7,6 +7,10 @@ import { resolve } from 'node:path' import { packChunkRuns, type SessionEvent } from '@deepseek-ai/dsh-session' import { parseSessionLog } from '@deepseek-ai/dsh-llm-replay' +/** Physical persistence artifacts validated by the WebWorker runtime fixture spec. */ +const PHYSICAL_SESSION_FIXTURE_ROOT = + 'packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/' + /** One repository session fixture and its canonical projected representation. */ export interface SessionFixtureLayout { /** Repository-relative path with `/` separators. */ @@ -17,6 +21,16 @@ export interface SessionFixtureLayout { canonical: string } +/** + * Whether a repository JSONL is a production-layout persistence artifact rather + * than an envelope-free replay snapshot owned by this script. + * @param path - Repository-relative path with `/` separators. + * @returns True only for Session logs under the WebWorker VFS example root. + */ +export function isPhysicalSessionFixture(path: string): boolean { + return path.startsWith(PHYSICAL_SESSION_FIXTURE_ROOT) && path.endsWith('/session.jsonl') +} + function isSessionHeader(value: unknown): boolean { return value !== null && typeof value === 'object' && (value as { type?: unknown }).type === 'session' } @@ -109,6 +123,7 @@ function discoverJsonlFiles(root: string): string[] { */ export function inspectSessionFixtureLayouts(root: string): SessionFixtureLayout[] { return discoverJsonlFiles(root).flatMap((path) => { + if (isPhysicalSessionFixture(path)) return [] const source = readFileSync(resolve(root, path), 'utf8') const canonical = canonicalSessionFixture(source, path) return canonical === undefined ? [] : [{ path, source, canonical }] diff --git a/vitest.config.ts b/vitest.config.ts index f24c8c552e..6e870fcbe4 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -57,6 +57,17 @@ const windowsUnsupportedTests = process.platform === 'win32' ] : [] +// These suites compare against or assemble the Worker's fixed Linux platform. +// Host-native Windows and macOS behavior is not their oracle. +const nonLinuxWebWorkerTests = process.platform === 'linux' + ? [] + : [ + 'packages/experimental/webworker-runtime/tests/node/fs-watch-stream.spec.ts', + 'packages/experimental/webworker-runtime/tests/node/sandbox-stack.spec.ts', + ] + +const platformUnsupportedTests = [...windowsUnsupportedTests, ...nonLinuxWebWorkerTests] + const windowsUnsupportedCoveragePackages = process.platform === 'win32' ? [...windowsUnsupportedPackages, 'packages/subprocess/*'] : [] @@ -142,7 +153,7 @@ export default defineConfig({ setupFiles: ['./scripts/test-invariants.ts'], // .tsx: client component specs (jsdom via per-file @vitest-environment pragma). include: testIncludes, - exclude: windowsUnsupportedTests, + exclude: platformUnsupportedTests, // One coverage invocation aggregates both projects. Every suite forks for // Node stability; process-bound suites stay separate for inventory control. projects: [ @@ -158,7 +169,7 @@ export default defineConfig({ setupFiles: ['./scripts/test-invariants.ts'], include: testIncludes, exclude: [ - ...windowsUnsupportedTests, + ...platformUnsupportedTests, ...processBoundTests, ...coverageExemptExcludes, ], @@ -173,7 +184,7 @@ export default defineConfig({ setupFiles: ['./scripts/test-invariants.ts'], include: processBoundTests, exclude: [ - ...windowsUnsupportedTests, + ...platformUnsupportedTests, ...coverageExemptExcludes, ], },