diff --git a/.agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.i18n.yaml new file mode 100644 index 0000000000..10a236e852 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.md +2026-08-27-steer-followup-image-delivery.md: 3a3d985e1dd09e17937d260253f3594b9a27a842 +2026-08-27-steer-followup-image-delivery.zh.md: 8015960eb899b1566cc1d738067acf3b318cdea6 diff --git a/.agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.md b/.agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.md new file mode 100644 index 0000000000..3a3d985e1d --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.md @@ -0,0 +1,43 @@ +# Agent Note: Steer and follow-up image delivery + +Status: implemented + +English | [中文](2026-08-27-steer-followup-image-delivery.zh.md) + +## Problem + +Images submitted while an agent is running did not reliably reach the model context or retain their intended browser placement (#3186), for three addressed reasons and one deferred agent-loop race. + +First, a steer or follow-up spliced into a live driver latched no wake: the live driver was expected to claim it, but a turn that finished or failed between the splice and the claim exited without re-checking, stranding the accepted message until an unrelated waking send. Image admission widens this window because the Host awaits attachment normalization before `agent.steer()`/`agent.followup()` runs. + +Second, continuable-subagent follow-ups rejected images in the Client (`SUBAGENT_IMAGE_UNSUPPORTED`) before any RPC, and stripped image parts from the text-only call. The Host route had no admission at all, and its wire content was `ContentBlock[]`, so lifting the Client rejection alone would have let a browser cite any `attachmentId` it never uploaded. + +Third, the browser queue projection reduced a queued image to the text `[image]` even though the durable reference was already present and readable through the session attachment authorization. + +Fourth, every local submission echo rendered at the Chat flow tail while the browser serialized image bytes. A direct steer therefore appeared as an ordinary chat message during the pre-admission wait, then moved to the pending-steering position when the Host queue snapshot arrived. Busy Queue sends had the same transition into QueueDock. + +## Decision + +**Host-side subagent image admission.** `SubagentPromptRequest.content` is now upload-shaped `PromptContentPart[]` (updating the wire contract in [Web subagent conversations](../feature/2026-07-27-web-subagent-conversations.md)). `dsh-attachment` owns the shared upload vocabulary and the `admitPromptContent()` conversion used by both the Session prompt endpoint and `SubagentRuntime.prompt`; Session Controller's shared request types retain a structurally identical Client wire declaration so the generated Client Cordis catalog contains the complete prompt-part fields, with a compile-time equality test preventing drift. The subagent route admits and persists image batches through `ctx.attachments` before `followup()`, and the continuation manager refuses delivery inside the per-child lock when the child's `agent.options` route resolves to a model without image input (`MODEL_DOES_NOT_SUPPORT_IMAGES`, surfaced as `subagent/attachment-invalid` with the same reason vocabulary as the Session route). A child without a fixed options route, or a deployment without the LLM registry, delivers and relies on the LLM layer's text-only projection. The Client forwards image parts unchanged and the `SUBAGENT_IMAGE_UNSUPPORTED` copy is gone. + +**Queue presentation.** The queue mirror's text preview excludes image blocks, and the queue dock renders each durable image part as a thumbnail resolved through `ctx.uiConversation.imageUrl` — the same session-authorized read the transcript uses. Editing queued image messages stays refused (#3072). + +**Stable optimistic placement.** Session derives a `PendingSubmission` placement synchronously from its running state and the requested delivery mode: `transcript` for an idle send, `queued` for a busy Queue send, and `steering` for a busy Steer send. The captured placement remains stable while serialization is in flight. Chat renders transcript and steering echoes on their respective surfaces, while QueueDock renders queued echoes with browser-owned image previews. The existing `rpcId` correlation suppresses the local echo in the same render that introduces the Host queue occurrence or durable user node. If the turn closes while images serialize and the Host places a requested steer in the next-turn queue, the later move from steering to QueueDock reflects the authoritative delivery decision. + +## Alternatives considered + +**Keep the wire content `ContentBlock[]` and admit refs on the Host.** Rejected: a reference-shaped wire lets a Client fabricate `attachmentId` citations; an upload-shaped wire makes Host admission the only way an attachment reference can exist in a child message. + +**Check child image capability in `SubagentRuntime.prompt`.** Rejected: the route may address a cold child whose agent does not exist yet; the continuation manager sees the live or freshly materialized agent in both arms and inside the per-child delivery lock, so the check cannot race a concurrent delivery. + +## Testing + +Host tests cover `mode: 'steer'` image admission; subagent control tests cover ordered admission, batch refusal, non-canonical base64, and the capability refusal mapping; continuation tests cover refusal without a partial message, capable delivery, and the routeless deferral. Client tests cover unstripped forwarding, the catalog-visible upload declaration, queue thumbnails (load, failure placeholder, unmount), the image-free preview, Session-owned placement derivation and capture, local steering presentation, queued echo presentation, and `rpcId` handoff on both surfaces. + +## Deferred + +A steer or follow-up inserted after a running driver's final inbox check and before it becomes idle can remain pending until another waking send starts the driver. Image admission performs asynchronous work before insertion, so image submissions can reach this timing window more often. This change leaves the agent-loop lifecycle unchanged; the wake race requires a separate lifecycle change and review. + +## Consequences + +Slow image serialization leaves optimistic messages on their selected transcript, QueueDock, or pending-steering surface until the Host handoff. The subagent package depends on `dsh-attachment` and reads `ctx.llm` optionally. Images persisted by a batch whose delivery is later refused stay as unreachable content-addressed objects under the existing retention rules. Queue thumbnails add one authorized attachment read per queued image, shared with the transcript cache. The deferred closing-turn race can leave an accepted message pending as described above. diff --git a/.agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.zh.md b/.agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.zh.md new file mode 100644 index 0000000000..8015960eb8 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.zh.md @@ -0,0 +1,43 @@ +# Agent Note: steer 与 follow-up 的图片投递 + +Status: implemented + +[English](2026-08-27-steer-followup-image-delivery.md) | 中文 + +## Problem + +agent 运行期间提交的图片没有可靠进入模型上下文，也没有保持预期的浏览器显示位置（#3186）。本次处理了其中三个原因，延后处理一个 agent-loop 竞态。 + +第一，splice 进在线 driver 的 steer 或 follow-up 不会锁存唤醒：预期由在线 driver 自行认领，但轮次在 splice 与认领之间正常结束或失败时，退出路径不再复查，已接受的消息就滞留到下一次无关的唤醒发送。图片准入放大了这个窗口，因为 Host 在执行 `agent.steer()`/`agent.followup()` 之前要先等待附件规范化完成。 + +第二，可继续子代理的 follow-up 在客户端就拒绝图片（`SUBAGENT_IMAGE_UNSUPPORTED`），并把图片部分从纯文本调用中剥掉。Host 路由完全没有准入，wire 内容又是 `ContentBlock[]`，单独放开客户端拒绝会允许浏览器引用任何它从未上传过的 `attachmentId`。 + +第三，浏览器队列投影把已排队的图片折叠成文本 `[image]`，尽管持久化引用已经存在，并且可以通过会话附件授权读取。 + +第四，浏览器序列化图片字节期间，所有本地提交回显都位于 Chat 消息流末尾。直接 steer 会在准入前等待阶段显示为普通聊天消息，Host queue snapshot 到达后才移到 pending-steering 位置。繁忙时 Queue 发送也会发生同类跳动，最终进入 QueueDock。 + +## Decision + +**Host 侧子代理图片准入。** `SubagentPromptRequest.content` 改为上传形态的 `PromptContentPart[]`（同步更新 [Web 子代理会话](../feature/2026-07-27-web-subagent-conversations.zh.md) 的 wire 契约）。`dsh-attachment` 负责共享上传词汇，以及 Session prompt 端点与 `SubagentRuntime.prompt` 共用的 `admitPromptContent()` 转换；Session Controller 的共享请求类型保留结构相同的 Client wire 声明，使生成的 Client Cordis 目录包含完整的 prompt part 字段，并用编译期等价测试防止两处定义偏离。子代理路由在 `followup()` 之前经 `ctx.attachments` 完成整批图片的准入与持久化；continuation 管理器在逐子级锁内，当子级 `agent.options` 路由解析到不接受图片输入的模型时拒绝投递（`MODEL_DOES_NOT_SUPPORT_IMAGES`，以与 Session 路由一致的 `subagent/attachment-invalid` 词汇表上抛）。子级没有固定 options 路由，或部署未挂载 LLM 注册表时照常投递，交给 LLM 层的纯文本投影。客户端原样转发图片部分，`SUBAGENT_IMAGE_UNSUPPORTED` 文案删除。 + +**队列展示。** 队列镜像的文本预览不再包含图片块，queue dock 把每个持久化图片部分渲染为缩略图，经 `ctx.uiConversation.imageUrl` 解析，与会话记录使用同一个会话授权读取。已排队图片消息的编辑仍然拒绝（#3072）。 + +**稳定的乐观显示位置。** Session 根据运行状态和请求的投递模式同步推导 `PendingSubmission` 位置：空闲发送是 `transcript`，繁忙时 Queue 发送是 `queued`，繁忙时 Steer 发送是 `steering`。该位置在序列化期间保持不变。Chat 分别在 transcript 与 steering 区域渲染对应回显，QueueDock 用浏览器持有的图片预览渲染 queued 回显。现有 `rpcId` 关联会在 Host queue occurrence 或持久化 user node 出现的同一次渲染中隐藏本地回显。如果图片序列化期间轮次关闭，Host 把请求的 steer 放入 next-turn queue，消息随后从 steering 移到 QueueDock，反映实际投递决定。 + +## Alternatives considered + +**wire 内容保持 `ContentBlock[]`，由 Host 准入引用。** 拒绝：引用形态的 wire 允许客户端伪造 `attachmentId`；上传形态的 wire 使 Host 准入成为子级消息里附件引用的唯一来源。 + +**在 `SubagentRuntime.prompt` 里做子级图片能力检查。** 拒绝：该路由可能寻址冷的子级，其 agent 尚不存在；continuation 管理器在两条分支里都拿得到在线或刚物化的 agent，并且处于逐子级投递锁内，检查不会与并发投递竞态。 + +## Testing + +Host 测试覆盖 `mode: 'steer'` 的图片准入；subagent control 测试覆盖有序准入、整批拒绝、非规范 base64 与能力拒绝映射；continuation 测试覆盖拒绝时不留半条消息、能力通过时投递、无路由时的顺延。客户端测试覆盖不剥离的转发、目录可见的上传声明、队列缩略图（加载、失败占位、卸载）、无图片占位的预览、Session 负责的位置推导与捕获、steering 本地显示、queued 回显显示，以及两个区域的 `rpcId` 交接。 + +## Deferred + +如果 steer 或 follow-up 在运行中 driver 最后一次检查 inbox 之后、转为 idle 之前插入，消息可能保持 pending，直到另一条唤醒消息重新启动 driver。图片准入会在插入前执行异步工作，因此图片提交更容易落入这个时序窗口。本次变更不修改 agent-loop 生命周期；该唤醒竞态需要单独的生命周期变更与审查。 + +## Consequences + +图片序列化较慢时，乐观消息停留在选定的 transcript、QueueDock 或 pending-steering 区域，直到与 Host 状态交接。subagent 包依赖 `dsh-attachment`，并可选读取 `ctx.llm`。整批持久化后投递被拒绝的图片按现有保留规则保持为不可达的内容寻址对象。队列缩略图对每张排队图片增加一次授权附件读取，与会话记录缓存共享。上述延后处理的轮次收尾竞态可能使已接受的消息保持 pending。 diff --git a/.agents/notes/implemented/bug-fix/2026-08-28-read-image-extensionless-paths.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-08-28-read-image-extensionless-paths.i18n.yaml new file mode 100644 index 0000000000..0f9fa28a2a --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-08-28-read-image-extensionless-paths.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-28-read-image-extensionless-paths.md +2026-08-28-read-image-extensionless-paths.md: cf7d82b2ed86208bcb8ef6287dfcfc7c4391ccc4 +2026-08-28-read-image-extensionless-paths.zh.md: 8d4cd15f6c414bb28307863bfe61ea3a7181ed02 diff --git a/.agents/notes/implemented/bug-fix/2026-08-28-read-image-extensionless-paths.md b/.agents/notes/implemented/bug-fix/2026-08-28-read-image-extensionless-paths.md new file mode 100644 index 0000000000..cf7d82b2ed --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-08-28-read-image-extensionless-paths.md @@ -0,0 +1,29 @@ +# Agent Note: read_image accepts extension-less image paths + +Status: implemented + +English | [中文](2026-08-28-read-image-extensionless-paths.zh.md) + +## Problem + +`read_image` mapped `file_path` to a media type by extension alone and refused a path with no extension. Valid extension-less images therefore required a renamed copy before the model could inspect them. Normalized local attachment objects exposed to the model use content digests without extensions, so their published read-only paths triggered the same refusal. + +## Decision + +`read_image` treats a file extension as a media-type declaration. PNG, JPEG, WebP, and GIF extensions select their declared types; another non-empty extension is refused before filesystem I/O, and the attachment store's full decode rejects a declaration that does not match the bytes. A path with no extension is read through `ctx.fs` under the existing `maxImageBytes` and tighter `maxMessageImageBytes` cap, then a tool-local `sniffImageMediaType` helper identifies one of the four supported file signatures. The detected type passes through the same deployment media-type policy and `saveImage` admission, whose full decode remains authoritative. This narrows the sniffing rejection in [the minimal read_image tool note](../feature/2026-08-10-minimal-read-image-tool.md) to extension-bearing paths. + +The mounted `ctx.fs` backend is the complete path-authorization authority for `read_image`. Extensions and file signatures decide only whether the tool accepts bytes that the backend returned. Any valid extension-less image readable through that backend can enter the current session, including a normalized attachment object; the tool performs no session-reference proof and the attachment service exposes no reverse path lookup. + +Admission failures name the offending path. An extension-less mismatch names the signature that supplied the declaration, while unsupported bytes report no file content. + +## Alternatives considered + +**Export signature identification from the attachment Service Definition package.** Only `read_image` needs this pre-admission declaration. Publishing the helper would make one Consumer's filename policy part of the provider-independent attachment API while the store already owns authoritative decoding. + +**Special-case normalized attachment object paths.** Resolving a path back to a Session reference would make two files readable through the same `ctx.fs` behave differently according to their origin and would leave ordinary extension-less images unsupported. Filesystem access remains the read authorization decision. + +**Add extensions to stored attachment objects.** This would change the storage layout and every object-path consumer to satisfy one tool's media-type declaration rule. + +## Consequences + +The model can read ordinary extension-less images and normalized attachment paths directly in native and PTC modes. Wrong extensions retain their pre-I/O refusal and mismatch repair. A non-image path without an extension is read up to the image byte cap before rejection, and a normalized object re-enters source admission instead of bypassing the current deployment limits. The behavior changes only `dsh-tool-fs`; the attachment Service Definition and local provider keep their existing APIs and storage behavior. diff --git a/.agents/notes/implemented/bug-fix/2026-08-28-read-image-extensionless-paths.zh.md b/.agents/notes/implemented/bug-fix/2026-08-28-read-image-extensionless-paths.zh.md new file mode 100644 index 0000000000..8d4cd15f6c --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-08-28-read-image-extensionless-paths.zh.md @@ -0,0 +1,29 @@ +# Agent Note: read_image 接受无扩展名图片路径 + +Status: implemented + +[English](2026-08-28-read-image-extensionless-paths.md) | 中文 + +## 问题 + +`read_image` 只按扩展名把 `file_path` 映射到媒体类型，并拒绝没有扩展名的路径。因此，模型必须先创建一份改名副本，才能查看合法的无扩展名图片。向模型公开的规范化本地附件对象以内容摘要命名，不带扩展名，所以其已发布的只读路径也会触发同一项拒绝。 + +## 决定 + +`read_image` 把文件扩展名视为媒体类型声明。PNG、JPEG、WebP 与 GIF 扩展名选择各自声明的类型；其他非空扩展名在文件系统 I/O 前被拒绝，附件存储的完整解码会拒绝与字节不匹配的声明。对于无扩展名路径，工具通过 `ctx.fs` 在既有 `maxImageBytes` 和更严格的 `maxMessageImageBytes` 上限内读取文件，再由工具内部的 `sniffImageMediaType` 辅助函数识别四种受支持的文件签名。识别结果经过同一套部署媒体类型策略和 `saveImage` 准入，后者的完整解码保持权威。这把[最小 read_image 工具 Agent Note](../feature/2026-08-10-minimal-read-image-tool.zh.md)中对嗅探的拒绝收窄到带扩展名的路径。 + +挂载的 `ctx.fs` 后端是 `read_image` 路径授权的完整依据。扩展名和文件签名只决定工具是否接受后端返回的字节。该后端可读的每个合法无扩展名图片都能进入当前会话，包括规范化附件对象；工具不证明 Session 引用，附件服务也不提供反向路径查找。 + +准入失败会指出出错路径。无扩展名路径的类型不匹配会指出提供声明的文件签名，而不受支持的字节不会出现在错误消息中。 + +## 考虑过的替代方案 + +**从附件 Service Definition 包导出文件签名识别。** 只有 `read_image` 需要这项准入前声明。公开该辅助函数会把单个消费方的文件名策略加入提供方无关的附件 API，而存储已经负责权威解码。 + +**特殊处理规范化附件对象路径。** 把路径反查为 Session 引用，会使 `ctx.fs` 以相同方式提供的两个文件根据来源产生不同读取结果，而且普通无扩展名图片仍然不受支持。文件系统访问保持读取授权决定。 + +**为存储的附件对象增加扩展名。** 这会为了满足一个工具的媒体类型声明规则而修改存储布局和每个对象路径消费方。 + +## 影响 + +模型可以在 native 和 PTC 模式下直接读取普通无扩展名图片与规范化附件路径。错误扩展名保留 I/O 前拒绝和类型不匹配修复提示。无扩展名非图片路径会在拒绝前读取到图片字节上限，规范化对象也会重新经过来源准入，而不会绕过当前部署限额。行为改动只位于 `dsh-tool-fs`；附件 Service Definition 与本地提供方保持现有 API 和存储行为。 diff --git a/.agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.i18n.yaml b/.agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.i18n.yaml index 0838d23d51..b4a945267c 100644 --- a/.agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.md -2026-07-27-web-subagent-conversations.md: 5ae4c22627a5f39547f1ca7f22bb9794b74e4340 -2026-07-27-web-subagent-conversations.zh.md: 79065872837ff3dd9e22f4be660991e9c540c7c0 +2026-07-27-web-subagent-conversations.md: 5a4d3f78c4a23077078cbab17d66e98f76e94d31 +2026-07-27-web-subagent-conversations.zh.md: 39a044f92b7ce6495410da3d826cebb666382c7c diff --git a/.agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.md b/.agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.md index 5ae4c22627..5a4d3f78c4 100644 --- a/.agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.md +++ b/.agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.md @@ -55,9 +55,9 @@ Agent-bound auxiliary controls are unavailable in addressed child views. In part - `subagent.list` takes `parentSessionId`, calls `ctx.subagents.listChildren(parentSessionId, signal)`, returns the complete ordered entries with each healthy row's boolean `hasChildren` snapshot, replaces each healthy row's corpus activity with whether its exact Agent driver is running, and includes whether the exact parent currently resolves from `ctx.agents`. - `subagent.history` takes the full mode-bearing address plus ordinary page arguments. It verifies the child and mode against the direct catalog, reads through `ctx.sessionQuery.readSession()`, rechecks direct lineage, and returns the ordinary raw-event, render-intent, pagination, and host-computed session-projection baseline without publishing an Agent. -- `subagent.prompt` accepts only a `mode: 'continuable'` address and `ContentBlock[]`. It requires the exact live parent, revalidates the catalog address, calls `ctx.subagents.followup(parent, childId, content, { source, signal })`, and returns the accepted `MessageId`. +- `subagent.prompt` accepts only a `mode: 'continuable'` address and upload-shaped `PromptContentPart[]`; the Host admits and persists image parts into durable references before delivery ([image delivery](../bug-fix/2026-08-27-steer-followup-image-delivery.md)). It requires the exact live parent, revalidates the catalog address, calls `ctx.subagents.followup(parent, childId, content, { source, signal })`, and returns the accepted `MessageId`. -The gateway maps missing parent, missing or diagnostic catalog entries, not-resumable and unauthorized children, request cancellation, and temporarily unavailable continuation admission to typed RPC errors. It does not expose descriptor or provider details. A list/prompt race is normal: the prompt result, not the earlier availability or activity snapshot, is authoritative. +The gateway maps missing parent, missing or diagnostic catalog entries, not-resumable and unauthorized children, request cancellation, image admission and image-capability refusals (`subagent/attachment-invalid`), and temporarily unavailable continuation admission to typed RPC errors. It does not expose descriptor or provider details. A list/prompt race is normal: the prompt result, not the earlier availability or activity snapshot, is authoritative. Viewing persisted history creates no mux subscription by itself. When a follow-up materializes a cold child Activation, the existing Host and mux streams publish its lifecycle and events. Reconnect rebuilds the addressed window through `subagent.history`. diff --git a/.agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.zh.md b/.agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.zh.md index 7906587283..39a044f92b 100644 --- a/.agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.zh.md +++ b/.agents/notes/implemented/feature/2026-07-27-web-subagent-conversations.zh.md @@ -55,9 +55,9 @@ one-shot 行始终会用文案替代输入框，说明执行记录为只读。 - `subagent.list` 接受 `parentSessionId`，调用 `ctx.subagents.listChildren(parentSessionId, signal)`，返回完整有序的条目以及每个健康行的布尔 `hasChildren` 快照，把每个健康行的语料活动状态替换为其确切 Agent driver 是否正在运行，并说明当前能否从 `ctx.agents` 解析出确切 parent。 - `subagent.history` 接受包含 mode 的完整地址与普通页参数。它对照直接目录校验 child 与 mode，通过 `ctx.sessionQuery.readSession()` 读取，再次检查直接谱系，并在不发布 agent 的情况下返回普通原始事件、渲染意图、分页与由 Host 计算的会话投影基线。 -- `subagent.prompt` 只接受 `mode: 'continuable'` 地址与 `ContentBlock[]`。它要求确切的存活 parent，重新校验目录地址，调用 `ctx.subagents.followup(parent, childId, content, { source, signal })`，并返回已接受的 `MessageId`。 +- `subagent.prompt` 只接受 `mode: 'continuable'` 地址与上传形态的 `PromptContentPart[]`；Host 在投递前把图片部分准入并持久化为持久引用（[图片投递](../bug-fix/2026-08-27-steer-followup-image-delivery.zh.md)）。它要求确切的存活 parent，重新校验目录地址，调用 `ctx.subagents.followup(parent, childId, content, { source, signal })`，并返回已接受的 `MessageId`。 -网关会将 parent 缺失、目录条目缺失或为 diagnostic、child 不可恢复或未授权、请求取消以及继续执行准入暂时不可用等失败映射为类型化 RPC 错误。它不会公开描述符或提供方细节。list／prompt 竞态属于正常情况：权威依据是提示词操作的结果，而不是更早的可用性或活动快照。 +网关会将 parent 缺失、目录条目缺失或为 diagnostic、child 不可恢复或未授权、请求取消、图片准入或图片能力拒绝（`subagent/attachment-invalid`）以及继续执行准入暂时不可用等失败映射为类型化 RPC 错误。它不会公开描述符或提供方细节。list／prompt 竞态属于正常情况：权威依据是提示词操作的结果，而不是更早的可用性或活动快照。 查看持久化历史本身不会创建 mux 订阅。当后续消息物化冷态 child Activation 时，现有 Host 与 mux 流会发布其生命周期与事件。重新连接时，系统通过 `subagent.history` 重建已寻址窗口。 diff --git a/.agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.i18n.yaml b/.agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.i18n.yaml index 4c276c6b0d..d57485473f 100644 --- a/.agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.md -2026-08-10-minimal-read-image-tool.md: 8880032b2648846df679ea8fa3301d182a95c06b -2026-08-10-minimal-read-image-tool.zh.md: aec34e19fc58037b031f7d4116d2fa664b2b45b5 +2026-08-10-minimal-read-image-tool.md: be7c24965ee256e4062b5caf32ce7f8c62d9c1ae +2026-08-10-minimal-read-image-tool.zh.md: f1d09a320e0f44145e4c8681668996d66fbb4898 diff --git a/.agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.md b/.agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.md index 8880032b26..be7c24965e 100644 --- a/.agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.md +++ b/.agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.md @@ -22,7 +22,7 @@ Both image-reading operations live in `dsh-tool-fs` and publish ordinary logged - **PR #598's route-scoped design** used a request-ready extension point, per-route schema visibility, reversible projection, and three durable concepts. Shared LLM request projection now handles text-only routes without putting tool registration or session formats into agent-loop. - **`agent.inject()` instead of the image-bearing tool result** — routes the image around the tool result as a separate injected user message. Rejected: the image *is* the tool's result; splitting them adds a second logged message with no gain, and the tool-result path already works end to end. -- **Magic-byte sniffing instead of extension declaration** — sniffing duplicates detection the attachment store already owns (sharp-backed, authoritative). The extension is only a *declaration*; a mismatch fails closed with a rename remedy rather than being silently accepted, which also keeps the model's mental map (file name ↔ content) honest. +- **Magic-byte sniffing instead of extension declaration** — sniffing duplicates detection the attachment store already owns (sharp-backed, authoritative). The extension is only a *declaration*; a mismatch fails closed with a rename remedy rather than being silently accepted, which also keeps the model's mental map (file name ↔ content) honest. This rejection covers extension-bearing paths; [extension-less image paths](../bug-fix/2026-08-28-read-image-extensionless-paths.md) narrows it — a path that declares nothing is identified from its file signature. - **Registering unconditionally and failing on a missing store** — rejected; a deployment without an attachment store cannot ever satisfy the tool, so its schema would be a standing lie. The route gate, by contrast, is per-call state and correctly lives at the execution boundary. ## Consequences diff --git a/.agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.zh.md b/.agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.zh.md index aec34e19fc..f1d09a320e 100644 --- a/.agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.zh.md +++ b/.agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.zh.md @@ -22,7 +22,7 @@ Status: implemented - **PR #598 的路由作用域设计**使用 request-ready 扩展点、按路由控制 schema 可见性、可逆投影和三个持久概念。共享 LLM 请求投影现在可以处理纯文本路由，无需把工具注册或会话格式放进 agent-loop。 - **用 `agent.inject()` 代替带图像的工具结果**——把图像绕过工具结果，作为单独注入的用户消息。拒绝：图像就是工具的结果；拆开只会多一条无收益的日志消息，而工具结果路径本就端到端可用。 -- **用魔数嗅探代替扩展名声明**——嗅探重复了附件存储已拥有的检测（基于 sharp，权威）。扩展名只是声明；不匹配时按改名修复提示失败关闭，而不是被静默接受，这也让模型对文件名与内容的对应保持诚实。 +- **用魔数嗅探代替扩展名声明**——嗅探重复了附件存储已拥有的检测（基于 sharp，权威）。扩展名只是声明；不匹配时按改名修复提示失败关闭，而不是被静默接受，这也让模型对文件名与内容的对应保持诚实。这一拒绝覆盖带扩展名的路径；[无扩展名图片路径](../bug-fix/2026-08-28-read-image-extensionless-paths.zh.md)将其收窄，什么也没声明的路径按文件签名识别。 - **无条件注册、缺存储时执行报错**——拒绝；没有附件存储的部署永远无法满足该工具，其 schema 会是常态谎言。相反，路由门禁是逐调用状态，正确的位置就是执行边界。 ## 后果 diff --git a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.i18n.yaml b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.i18n.yaml index f8e8ce4bee..0a43a197b4 100644 --- a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.i18n.yaml +++ b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md -2026-07-26-ci-failover-runbook.md: bfed4e6e15311d0191c1379a5822b0daf46f4ed3 -2026-07-26-ci-failover-runbook.zh.md: 86007d5b189ccc883dc96d68bc9e54f38bb09e2a +2026-07-26-ci-failover-runbook.md: 7579e6ca4da5207f3d308c7606edc6d885ab25c7 +2026-07-26-ci-failover-runbook.zh.md: eba74831572252ecbbde388e83458161cad0f696 diff --git a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md index bfed4e6e15..7579e6ca4d 100644 --- a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md +++ b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md @@ -24,7 +24,7 @@ The decision belongs at workflow level because cancellation applies to the whole #### Windows pool -`dsh-win-ci`: 32 always-on runner instances (scheduled tasks `GH-Runner-01`…`GH-Runner-32`) on the in-house Windows CI server (one 96-core / 580 GB machine). Labels: `[self-hosted, dsh-win-ci, windows]`. The image must preinstall Node 24, pnpm, Git (with Git Bash on `PATH`, i.e. `C:\Program Files\Git\bin` — the `bash` tool spawns `bash` by name), PowerShell 7, and enable Developer Mode for symlink support. Check the latest `serial / windows (self-hosted standby)` run before switching: a green standby verifies the pool can execute `check:ci:windows-complete` end-to-end. +`dsh-win-ci`: 32 always-on runner instances (scheduled tasks `GH-Runner-01`…`GH-Runner-32`) on the in-house Windows CI server (one 96-core / 580 GB machine). Labels: `[self-hosted, dsh-win-ci, windows]`. The image must preinstall Node 24, pnpm, Git (with Git Bash on `PATH`, i.e. `C:\Program Files\Git\bin` — the `bash` tool spawns `bash` by name), PowerShell 7, and enable Developer Mode for symlink support. The workspaces and the pnpm store must both live on a ReFS volume (`F:`): the Windows installs pass `--package-import-method=clone` on ReFS, which needs that volume layout and the `@reflink/reflink` native module that the system corepack pnpm carries (see [the Windows ReFS store note](2026-08-30-windows-refs-store-block-clone-install.md)); a rebuilt runner without this layout fails the Windows build gates with TS6231. Check the latest `serial / windows (self-hosted standby)` run before switching: a green standby verifies the pool can execute `check:ci:windows-complete` end-to-end. ### Switch (any repository writer, ~1 minute, no merge) diff --git a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.zh.md b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.zh.md index 86007d5b18..eba7483157 100644 --- a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.zh.md +++ b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.zh.md @@ -24,7 +24,7 @@ Status: implemented #### Windows 池 -`dsh-win-ci`：公司内部 Windows CI 服务器（一台 96 核 / 580 GB 机器）上 32 个常驻运行器实例（计划任务 `GH-Runner-01`…`GH-Runner-32`）。标签：`[self-hosted, dsh-win-ci, windows]`。镜像必须预装 Node 24、pnpm、Git（Git Bash 在 `PATH` 上，即 `C:\Program Files\Git\bin`——`bash` 工具按名称 spawn `bash`）、PowerShell 7，并为符号链接支持启用开发人员模式。切换前先看 `serial / windows (self-hosted standby)` 最近一次运行：绿色热备验证该池能端到端执行 `check:ci:windows-complete`。 +`dsh-win-ci`：公司内部 Windows CI 服务器（一台 96 核 / 580 GB 机器）上 32 个常驻运行器实例（计划任务 `GH-Runner-01`…`GH-Runner-32`）。标签：`[self-hosted, dsh-win-ci, windows]`。镜像必须预装 Node 24、pnpm、Git（Git Bash 在 `PATH` 上，即 `C:\Program Files\Git\bin`——`bash` 工具按名称 spawn `bash`）、PowerShell 7，并为符号链接支持启用开发人员模式。工作区与 pnpm store 必须都位于 ReFS 卷（`F:`）上：Windows 安装步骤在 ReFS 上传递 `--package-import-method=clone`，这需要该卷布局以及系统 corepack pnpm 携带的 `@reflink/reflink` 原生模块（见 [Windows ReFS store note](2026-08-30-windows-refs-store-block-clone-install.zh.md)）；没有此布局的重建运行器会在 Windows 构建门禁阶段以 TS6231 失败。切换前先看 `serial / windows (self-hosted standby)` 最近一次运行：绿色热备验证该池能端到端执行 `check:ci:windows-complete`。 ### 切换步骤（任何具备写权限的协作者，约 1 分钟，无需合并） diff --git a/.agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.i18n.yaml b/.agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.i18n.yaml index aec2498598..6fe27f1317 100644 --- a/.agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.i18n.yaml +++ b/.agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.md -2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.md: d485098f7ee04596e77322089fa0f6f45020024a -2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.zh.md: 47bd525377d6c358891b238373410049987f5ee1 +2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.md: 2519b9e8bc565790acbd02f0a01491fcc136bbe6 +2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.zh.md: 2157541c4b503d4acd38073263b6b69050a239bc diff --git a/.agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.md b/.agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.md index d485098f7e..2519b9e8bc 100644 --- a/.agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.md +++ b/.agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.md @@ -10,7 +10,7 @@ Outside `landlock-run.yml`, each workflow that installed pnpm hand-provisioned i ## Decision -`pnpm/action-setup@v4` is the only pnpm provisioning mechanism in CI: no workflow runs `corepack enable`. The root dev dependency on `@yarnpkg/cli-dist` separately supplies the modern Yarn CLI exercised by the generated-project e2e; package-manager coverage therefore does not inherit the runner image's Yarn Classic. Caching remains per-job policy on top of pnpm provisioning, in three deliberate shapes: +`pnpm/action-setup@v4` is the pnpm provisioning mechanism across CI: no workflow runs `corepack enable`. The self-hosted Windows install steps are the deliberate exception — they invoke `corepack pnpm` because clone-mode installs need the `@reflink/reflink` native module that the system corepack pnpm carries but `pnpm/action-setup`'s dest build omits (see [the Windows ReFS store note](2026-08-30-windows-refs-store-block-clone-install.md)). The root dev dependency on `@yarnpkg/cli-dist` separately supplies the modern Yarn CLI exercised by the generated-project e2e; package-manager coverage therefore does not inherit the runner image's Yarn Classic. Caching remains per-job policy on top of pnpm provisioning, in three deliberate shapes: - **Symmetric cache** (restore and save): `actions/setup-node` with `cache: pnpm` — `e2e.yml`, `docs-pages.yml`, `pi-ai-provider-e2e.yml`, `build-exe-for-python-sdk.yml`, the node-compat job of `ci.yml`, and the two benchmark jobs of `ci-master.yml`. The larger-runner benchmark keeps its store cache Linux-only through a conditional `cache:` input; the consolidated benchmark caches on both platforms. - **Restore-only caching** (hand-rolled `actions/cache` steps): the three enterprise-runner PR jobs and the Wine-based required Windows job restore without saving, keeping cache compression/upload off their latency-sensitive paths — an asymmetry `setup-node`'s cache cannot express. Each configures a store outside the action's replaceable install directory and resolves that path. No master job produces these hosted caches, so these restores hit matching archived entries until they evict. The enterprise jobs skip restore during self-hosted failover because that VM's persistent store is already warm. @@ -27,7 +27,7 @@ Outside `landlock-run.yml`, each workflow that installed pnpm hand-provisioned i ## Consequences -- The corepack dependency is gone from CI entirely; pnpm arrives via the pnpm team's official action everywhere, and the version pin stays single-sourced in `package.json`'s `packageManager` field. +- The corepack dependency is gone from CI except the self-hosted Windows install steps, which invoke `corepack pnpm` for the ReFS block-clone native module; pnpm otherwise arrives via the pnpm team's official action, and the version pin stays single-sourced in `package.json`'s `packageManager` field. - The generated-project e2e runs the root-pinned Yarn 4 CLI instead of inheriting or silently skipping the runner image's Yarn version. - The cache-key format changed once for converted lanes; one cold run repopulated it, after which hit rates match the old steps. The built-in key spans platform, arch, and the lockfile hash but not the Node version, so the node-compat matrix legs share one store entry — safe, because the pnpm store is Node-version-independent. - `setup-node`'s built-in pnpm cache restores by exact key only, with no `restore-keys` prefix fallback: a `pnpm-lock.yaml` change starts a converted lane from a cold store instead of seeding from the previous entry. diff --git a/.agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.zh.md b/.agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.zh.md index 47bd525377..2157541c4b 100644 --- a/.agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.zh.md +++ b/.agents/notes/implemented/process/2026-07-26-pnpm-action-setup-for-symmetric-ci-caching.zh.md @@ -10,7 +10,7 @@ Status: implemented ## 决策 -`pnpm/action-setup@v4` 是 CI 中提供 pnpm 的唯一机制：没有任何工作流运行 `corepack enable`。根目录的 `@yarnpkg/cli-dist` 开发依赖另行提供 generated-project e2e 所运行的现代 Yarn CLI（命令行界面）；因此，用于包管理器覆盖率的 Yarn 不会沿用 runner 镜像里的 Yarn Classic。缓存仍是叠加在 pnpm 提供机制上的按作业策略，保留三种有意采用的形态： +`pnpm/action-setup@v4` 是 CI 中提供 pnpm 的机制：没有任何工作流运行 `corepack enable`。自托管 Windows 安装步骤是刻意的例外——它们调用 `corepack pnpm`，因为 clone 模式安装需要系统 corepack pnpm 携带、而 `pnpm/action-setup` 的 dest 构建缺少的 `@reflink/reflink` 原生模块（见 [Windows ReFS store note](2026-08-30-windows-refs-store-block-clone-install.zh.md)）。根目录的 `@yarnpkg/cli-dist` 开发依赖另行提供 generated-project e2e 所运行的现代 Yarn CLI（命令行界面）；因此，用于包管理器覆盖率的 Yarn 不会沿用 runner 镜像里的 Yarn Classic。缓存仍是叠加在 pnpm 提供机制上的按作业策略，保留三种有意采用的形态： - **对称缓存**（既恢复也保存）：带 `cache: pnpm` 的 `actions/setup-node`——`e2e.yml`、`docs-pages.yml`、`pi-ai-provider-e2e.yml`、`build-exe-for-python-sdk.yml`、`ci.yml` 的 node-compat 作业，以及 `ci-master.yml` 的两个 benchmark 作业。larger-runner benchmark 通过条件化的 `cache:` 输入让 store 缓存仅限 Linux；consolidated benchmark 在两个平台上都启用缓存。 - **只恢复不上传**（手写的 `actions/cache` 步骤）：企业 runner 上的三个 PR（Pull Request）作业和基于 Wine 的必需 Windows 作业只恢复不保存，把缓存压缩／上传挡在它们的延迟敏感路径之外——这种不对称是 `setup-node` 的缓存无法表达的。每个作业都在 action 可替换的安装目录之外配置 store，并解析该路径。没有任何 master 作业生产这些 hosted 缓存，这些恢复步骤只能命中仍有归档的旧条目，直至其被逐出；企业作业在自托管故障切换期间跳过恢复，因为该 VM 的持久 store 已经预热。 @@ -27,7 +27,7 @@ Status: implemented ## 后果 -- corepack 依赖已从 CI 中彻底消失；pnpm 在所有工作流中都经由 pnpm 团队的官方 action 提供，版本锁定继续单一来源于 `package.json` 的 `packageManager` 字段。 +- corepack 依赖已从 CI 中消失，唯独自托管 Windows 安装步骤例外——它们为 ReFS 块克隆原生模块调用 `corepack pnpm`；pnpm 在其他工作流中都经由 pnpm 团队的官方 action 提供，版本锁定继续单一来源于 `package.json` 的 `packageManager` 字段。 - generated-project e2e 运行根目录锁定的 Yarn 4 CLI，既不再沿用 runner 镜像中的 Yarn 版本，也不会因此悄然跳过。 - 已转换泳道的缓存键格式变更了一次；各跑一次冷运行重建缓存后，命中率与旧步骤持平。内建缓存键涵盖平台、架构与锁文件哈希，但不含 Node 版本，因此 node-compat 的各个矩阵任务共享同一条 store 缓存记录——这是安全的，因为 pnpm store 与 Node 版本无关。 - `setup-node` 内建的 pnpm 缓存只按精确键恢复，没有 `restore-keys` 前缀回退：`pnpm-lock.yaml` 一旦变更，已转换泳道会从冷 store 起步，而不是利用上一条缓存记录预填充。 diff --git a/.agents/notes/implemented/process/2026-08-30-windows-refs-store-block-clone-install.i18n.yaml b/.agents/notes/implemented/process/2026-08-30-windows-refs-store-block-clone-install.i18n.yaml new file mode 100644 index 0000000000..b80e7d4a72 --- /dev/null +++ b/.agents/notes/implemented/process/2026-08-30-windows-refs-store-block-clone-install.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-30-windows-refs-store-block-clone-install.md +2026-08-30-windows-refs-store-block-clone-install.md: 086c00453fffada7faef1631e7c270f3270b82d6 +2026-08-30-windows-refs-store-block-clone-install.zh.md: f813ae15a16c9f5cd61b1f7b98a66d74af2115e9 diff --git a/.agents/notes/implemented/process/2026-08-30-windows-refs-store-block-clone-install.md b/.agents/notes/implemented/process/2026-08-30-windows-refs-store-block-clone-install.md new file mode 100644 index 0000000000..086c00453f --- /dev/null +++ b/.agents/notes/implemented/process/2026-08-30-windows-refs-store-block-clone-install.md @@ -0,0 +1,47 @@ +# Agent Note: Windows self-hosted ReFS store and block-clone installs + +Status: implemented + +English | [中文](2026-08-30-windows-refs-store-block-clone-install.zh.md) + +## Problem + +The self-hosted Windows VM's workspaces moved from the NTFS `E:` volume to the ReFS `F:` volume. `git clean -ffdx` on the NTFS volume deleted the ~70k-file node_modules tree in tens of minutes and forced a full reinstall on every run, driving disk writes past the volume's sustained bandwidth. ReFS metadata operations are orders of magnitude faster, so the workspace move restored fast checkout, but it exposed a second failure. + +The pnpm store also lives on `F:` (`F:\.pnpm-store`), so pnpm links node_modules files to the store with hardlinks (its default `package-import-method=auto` on a same-volume layout). TypeScript resolves module files with the native realpath (`fs.realpathSync.native`), which on Windows resolves a hardlink to the store's content-addressed path (`F:/.pnpm-store/v11/files//`). The compiler then resolves bare imports from that store path, where no `node_modules` exists, and fails with TS6231 (`Could not resolve the path 'F:/.pnpm-store/...'`) during `tsc -b` and vite's module resolution. The JS `realpathSync` does not leak the store path; only the native variant does, so this only appears in compiler tooling. + +A related install failure appears when `package-import-method=clone` runs on a volume that does not support copy-on-write: pnpm reports `ERR_PNPM_LINKING_FAILED ... Source volume does not support copy-on-write` on NTFS volumes (hosted runners). + +The pnpm build that `pnpm/action-setup` installs into its `dest` omits the `@reflink/reflink` native module that clone mode requires, so even on ReFS, clone fails with `Cannot find module './reflink.win32-x64-msvc-*.node'`. The system corepack pnpm carries the complete `@reflink` platform set, including `reflink.win32-x64-msvc.node`. + +## Decision + +The Windows install steps in [ci.yml](../../../../.github/workflows/ci.yml) (the four pull-request native jobs) and [ci-master.yml](../../../../.github/workflows/ci-master.yml) (`serial-windows`) branch on the workspace filesystem, using clone only on ReFS: + +```pwsh +$drive = (Split-Path -Qualifier $env:GITHUB_WORKSPACE).TrimEnd(':') +$fs = (Get-Volume -DriveLetter $drive).FileSystem +if ($fs -eq 'ReFS') { + corepack pnpm install --frozen-lockfile --package-import-method=clone +} else { + pnpm install --frozen-lockfile +} +``` + +- `--package-import-method=clone` on ReFS uses block cloning: each node_modules file gets an independent path (so native realpath cannot resolve it back to a store path, eliminating TS6231) while sharing physical blocks with the store (no copy cost). ReFS supports block cloning and hardlinks (verified with `fsutil fsinfo volumeinfo` and hardlink listing). +- The flag is passed only when the workspace volume is ReFS. Hosted runners (NTFS, fresh VM per job) keep the default import method, because NTFS rejects block clone. +- `corepack pnpm` is used because clone mode needs the `@reflink/reflink` native module, which the system corepack pnpm carries but `pnpm/action-setup`'s dest build omits. +- `.npmrc` and `npm_config_*` environment variables do not drive `package-import-method` in pnpm 11.7.0 on Windows; only the CLI flag is honored, so the flag is explicit in the command. + +The self-hosted VM's store lives on `F:\.pnpm-store` (ReFS, machine-level `PNPM_CONFIG_STORE_DIR`), and the workspaces live on `F:\ci\_work-NN`. The F: volume is 200 GB ReFS after rebuild. `DSH_CI_FAILOVER_WINDOWS=selfhosted` routes the four pull-request native jobs to the self-hosted pool. + +## Alternatives considered + +- **Keep workspaces on NTFS `E:`** - rejected because `git clean -ffdx` deleted the node_modules tree in tens of minutes on NTFS, the original write-storm cause; ReFS reduced it to ~23 seconds. +- **`--package-import-method=copy`** - avoids the store-path leak (files are independent copies) and needs no native module, but copies every file from the store on every install, restoring most of the write cost the workspace move removed. +- **Fix the action-setup pnpm's reflink** - rejected because `pnpm/action-setup` installs a fresh pnpm into a per-job `dest` directory; adding the native module there is fragile and per-job. +- **`.npmrc` `package-import-method=clone`** - rejected because pnpm 11.7.0 on Windows ignores it (verified: files remain hardlinks with `nlink=2` and native realpath still leaks the store path). + +## Consequences + +The self-hosted Windows installs use block cloning, giving independent file paths (no TS6231) with shared physical blocks (no copy). Hosted runners keep the default import method. The `serial-windows` standby drill and the pull-request native jobs on the self-hosted pool depend on the ReFS volume layout; a runner rebuilt from the [failover runbook](2026-07-26-ci-failover-runbook.md) without the ReFS store-and-workspace layout would fail the Windows build gates with TS6231 (or the install with reflink errors). diff --git a/.agents/notes/implemented/process/2026-08-30-windows-refs-store-block-clone-install.zh.md b/.agents/notes/implemented/process/2026-08-30-windows-refs-store-block-clone-install.zh.md new file mode 100644 index 0000000000..f813ae15a1 --- /dev/null +++ b/.agents/notes/implemented/process/2026-08-30-windows-refs-store-block-clone-install.zh.md @@ -0,0 +1,47 @@ +# Agent Note：Windows 自托管 ReFS store 与块克隆安装 + +Status: implemented + +[English](2026-08-30-windows-refs-store-block-clone-install.md) | 中文 + +## Problem + +自托管 Windows 虚拟机的工作区从 NTFS 的 `E:` 卷迁到了 ReFS 的 `F:` 卷。在 NTFS 卷上，`git clean -ffdx` 删除约 7 万个文件的 node_modules 树需要几十分钟，并迫使每次运行全量重装，把磁盘写入推到该卷持续带宽以上。ReFS 的元数据操作快几个数量级，因此工作区迁移恢复了快速 checkout，但暴露了第二个失败。 + +pnpm store 也在 `F:` 上（`F:\.pnpm-store`），因此 pnpm 用硬链接把 node_modules 文件链接到 store（同卷布局下的默认 `package-import-method=auto`）。TypeScript 用原生 realpath（`fs.realpathSync.native`）解析模块文件，在 Windows 上会把硬链接解析到 store 的内容寻址路径（`F:/.pnpm-store/v11/files//`）。编译器随后从那个 store 路径解析裸导入，而那里没有 `node_modules`，于是在 `tsc -b` 和 vite 的模块解析期间以 TS6231（`Could not resolve the path 'F:/.pnpm-store/...'`）失败。JS 的 `realpathSync` 不泄漏 store 路径；只有原生变体会泄漏，所以这只出现在编译器工具链里。 + +当 `package-import-method=clone` 运行在不支持 copy-on-write 的卷上时，会出现相关的安装失败：pnpm 在 NTFS 卷（托管 runner）上报告 `ERR_PNPM_LINKING_FAILED ... Source volume does not support copy-on-write`。 + +`pnpm/action-setup` 装到其 `dest` 的 pnpm 构建缺少 clone 模式所需的 `@reflink/reflink` 原生模块，所以即使在 ReFS 上，clone 也会以 `Cannot find module './reflink.win32-x64-msvc-*.node'` 失败。系统 corepack pnpm 带有完整的 `@reflink` 平台集合，包括 `reflink.win32-x64-msvc.node`。 + +## Decision + +[ci.yml](../../../../.github/workflows/ci.yml)（四个 pull-request 原生作业）和 [ci-master.yml](../../../../.github/workflows/ci-master.yml)（`serial-windows`）中的 Windows 安装步骤按工作区文件系统分支，仅在 ReFS 上使用 clone： + +```pwsh +$drive = (Split-Path -Qualifier $env:GITHUB_WORKSPACE).TrimEnd(':') +$fs = (Get-Volume -DriveLetter $drive).FileSystem +if ($fs -eq 'ReFS') { + corepack pnpm install --frozen-lockfile --package-import-method=clone +} else { + pnpm install --frozen-lockfile +} +``` + +- ReFS 上的 `--package-import-method=clone` 使用块克隆：每个 node_modules 文件获得独立路径（因此原生 realpath 无法把它解析回 store 路径，消除了 TS6231），同时与 store 共享物理块（无复制代价）。ReFS 支持块克隆和硬链接（已用 `fsutil fsinfo volumeinfo` 和硬链接列表验证）。 +- 仅当工作区卷是 ReFS 时才传该 flag。托管 runner（NTFS，每个 job 全新 VM）保留默认导入方式，因为 NTFS 拒绝块克隆。 +- 使用 `corepack pnpm` 是因为 clone 模式需要 `@reflink/reflink` 原生模块，系统 corepack pnpm 带有它，而 `pnpm/action-setup` 的 dest 构建缺少。 +- `.npmrc` 与 `npm_config_*` 环境变量在 Windows 的 pnpm 11.7.0 上不驱动 `package-import-method`；只有 CLI flag 生效，因此命令中显式传 flag。 + +自托管虚拟机的 store 位于 `F:\.pnpm-store`（ReFS，机器级 `PNPM_CONFIG_STORE_DIR`），工作区位于 `F:\ci\_work-NN`。重建后 F: 卷为 200 GB ReFS。`DSH_CI_FAILOVER_WINDOWS=selfhosted` 把四个 pull-request 原生作业路由到自托管池。 + +## Alternatives considered + +- **把工作区留在 NTFS 的 `E:`** - 不采纳，因为 NTFS 上 `git clean -ffdx` 删除 node_modules 树需要几十分钟，即最初的写风暴根因；ReFS 把它降到约 23 秒。 +- **`--package-import-method=copy`** - 避免 store 路径泄漏（文件是独立副本）且不需要原生模块，但每次安装都从 store 复制每个文件，恢复了工作区迁移移除的大部分写代价。 +- **修复 action-setup 的 pnpm 的 reflink** - 不采纳，因为 `pnpm/action-setup` 把全新 pnpm 装进每 job 的 `dest` 目录；在那里补原生模块脆弱且按 job 生效。 +- **`.npmrc` 的 `package-import-method=clone`** - 不采纳，因为 Windows 的 pnpm 11.7.0 忽略它（已验证：文件保持 `nlink=2` 的硬链接，原生 realpath 仍泄漏 store 路径）。 + +## Consequences + +自托管 Windows 安装使用块克隆，既得到独立文件路径（无 TS6231），又共享物理块（无复制）。托管 runner 保留默认导入方式。`serial-windows` standby drill 与自托管池上的 pull-request 原生作业依赖 ReFS 卷布局；若按 [failover runbook](2026-07-26-ci-failover-runbook.zh.md) 重建 runner 而没有 ReFS store 与工作区布局，Windows 构建门禁会以 TS6231 失败（或安装阶段以 reflink 错误失败）。 diff --git a/.github/workflows/ci-master.yml b/.github/workflows/ci-master.yml index b86720a5d3..e16fd624d9 100644 --- a/.github/workflows/ci-master.yml +++ b/.github/workflows/ci-master.yml @@ -184,13 +184,25 @@ jobs: - name: Configure persistent pnpm store shell: pwsh + # The store must share the ReFS workspace volume for the clone + # import method below; LOCALAPPDATA (C:) would cross volumes and + # break block clone. See 2026-08-30-windows-refs-store-block-clone-install. run: | - $storeRoot = "$env:LOCALAPPDATA\pnpm\store" + $storeRoot = "F:\.pnpm-store" echo "PNPM_CONFIG_STORE_DIR=$storeRoot" >> $env:GITHUB_ENV - name: Install (immutable) shell: pwsh - run: pnpm install --frozen-lockfile + # See 2026-08-30-windows-refs-store-block-clone-install for the + # ReFS block-clone rationale; use clone only on ReFS. + run: >- + $drive = (Split-Path -Qualifier $env:GITHUB_WORKSPACE).TrimEnd(':'); + $fs = (Get-Volume -DriveLetter $drive).FileSystem; + if ($fs -eq 'ReFS') { + corepack pnpm install --frozen-lockfile --package-import-method=clone + } else { + pnpm install --frozen-lockfile + } - name: Run complete unsharded Windows gate inventory serially shell: pwsh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 40eab816f8..86ba683bee 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -458,7 +458,17 @@ jobs: node-version: ${{ env.PRIMARY_NODE_VERSION }} - name: Install (immutable) shell: pwsh - run: pnpm install --frozen-lockfile + # See 2026-08-30-windows-refs-store-block-clone-install for the + # ReFS block-clone rationale; detect the workspace filesystem and + # pass --package-import-method=clone only on ReFS. + run: >- + $drive = (Split-Path -Qualifier $env:GITHUB_WORKSPACE).TrimEnd(':'); + $fs = (Get-Volume -DriveLetter $drive).FileSystem; + if ($fs -eq 'ReFS') { + corepack pnpm install --frozen-lockfile --package-import-method=clone + } else { + pnpm install --frozen-lockfile + } - name: Run blocking Windows builds shell: pwsh run: pnpm run check:ci:windows-blocking @@ -510,7 +520,17 @@ jobs: node-version: ${{ env.PRIMARY_NODE_VERSION }} - name: Install (immutable) shell: pwsh - run: pnpm install --frozen-lockfile + # See 2026-08-30-windows-refs-store-block-clone-install for the + # ReFS block-clone rationale; detect the workspace filesystem and + # pass --package-import-method=clone only on ReFS. + run: >- + $drive = (Split-Path -Qualifier $env:GITHUB_WORKSPACE).TrimEnd(':'); + $fs = (Get-Volume -DriveLetter $drive).FileSystem; + if ($fs -eq 'ReFS') { + corepack pnpm install --frozen-lockfile --package-import-method=clone + } else { + pnpm install --frozen-lockfile + } - name: Build before coverage shell: pwsh run: pnpm run build @@ -553,7 +573,17 @@ jobs: node-version: ${{ env.PRIMARY_NODE_VERSION }} - name: Install (immutable) shell: pwsh - run: pnpm install --frozen-lockfile + # See 2026-08-30-windows-refs-store-block-clone-install for the + # ReFS block-clone rationale; detect the workspace filesystem and + # pass --package-import-method=clone only on ReFS. + run: >- + $drive = (Split-Path -Qualifier $env:GITHUB_WORKSPACE).TrimEnd(':'); + $fs = (Get-Volume -DriveLetter $drive).FileSystem; + if ($fs -eq 'ReFS') { + corepack pnpm install --frozen-lockfile --package-import-method=clone + } else { + pnpm install --frozen-lockfile + } - name: Run Windows-specific native tests shell: pwsh run: >- @@ -595,7 +625,17 @@ jobs: node-version: ${{ env.PRIMARY_NODE_VERSION }} - name: Install (immutable) shell: pwsh - run: pnpm install --frozen-lockfile + # See 2026-08-30-windows-refs-store-block-clone-install for the + # ReFS block-clone rationale; detect the workspace filesystem and + # pass --package-import-method=clone only on ReFS. + run: >- + $drive = (Split-Path -Qualifier $env:GITHUB_WORKSPACE).TrimEnd(':'); + $fs = (Get-Volume -DriveLetter $drive).FileSystem; + if ($fs -eq 'ReFS') { + corepack pnpm install --frozen-lockfile --package-import-method=clone + } else { + pnpm install --frozen-lockfile + } - name: Run Windows observational gates shell: pwsh run: pnpm run check:ci:windows-observational diff --git a/apps/web/tests/queue-image.e2e.ts b/apps/web/tests/queue-image.e2e.ts new file mode 100644 index 0000000000..2f57a72c48 --- /dev/null +++ b/apps/web/tests/queue-image.e2e.ts @@ -0,0 +1,155 @@ +// Keyless browser coverage for image attachments submitted while a turn is +// running, through the shipped Web composition and real HTTP/SSE wire. A +// text-plus-image submission queues as one occurrence whose dock row renders +// the durable thumbnail, survives a stop as parked work, and delivers as the +// next turn's user message with its image intact — while the session log holds +// only durable attachment references, never base64. +import { existsSync } from 'node:fs' +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { fileURLToPath } from 'node:url' +import { join } from 'node:path' +import type { Browser, Page } from 'playwright' +import { chromium } from 'playwright' +import { afterEach, describe, expect, it, onTestFailed } from 'vitest' +import { deriveReplayScript, parseSessionLog, type ReplayEntry } from '@deepseek-ai/dsh-llm-replay' +import type { SessionEvent } from '@deepseek-ai/dsh-session' +import { + captureStableAria, compareOrRefreshGolden, + launchWebScaffold, watchConsole, webSnapshotMode, type WebScaffold, +} from './scaffold.ts' +import { connectFreshWorkspace, newEnglishPage, saveFailureShot } from './support.ts' + +const SNAPSHOT_DIR = fileURLToPath(new URL('../../../snapshots/web/queued-image', import.meta.url)) +const FIXTURE = fileURLToPath(new URL('../../../snapshots/web/live-interactions/session.jsonl', import.meta.url)) +const PNG = fileURLToPath(new URL('../../../snapshots/session/read-image/workspace/red.png', import.meta.url)) +const QUEUED_EXPECTED = join(SNAPSHOT_DIR, 'queued.expected.md') +const DELIVERED_EXPECTED = join(SNAPSHOT_DIR, 'delivered.expected.md') +const MODE = webSnapshotMode() + +const ACTIVE_PROMPT = 'Reply with a one-sentence description of event sourcing, then stop.' +const QUEUED_TEXT = 'Compare with this screenshot' + +/** Paste one real PNG into the composer through a genuine clipboard event. */ +async function pasteImage(page: Page, bytes: Uint8Array): Promise { + await page.locator('[data-composer-input]').first().evaluate((surface, data) => { + const transfer = new DataTransfer() + transfer.items.add(new File([new Uint8Array(data)], 'queued.png', { type: 'image/png' })) + surface.dispatchEvent(new ClipboardEvent('paste', { + clipboardData: transfer, bubbles: true, cancelable: true, + })) + }, [...bytes]) +} + +describe('web e2e: queued image submission', () => { + let scaffold: WebScaffold | undefined + let browser: Browser | undefined + let page: Page + let overrideDir: string | undefined + + afterEach(async () => { + const failures: unknown[] = [] + await browser?.close().catch((error: unknown) => failures.push(error)) + browser = undefined + const closing = scaffold + scaffold = undefined + await closing?.close().catch((error: unknown) => failures.push(error)) + if (overrideDir !== undefined) { + await rm(overrideDir, { recursive: true, force: true }) + .catch((error: unknown) => failures.push(error)) + } + overrideDir = undefined + if (failures.length === 1) throw failures[0] + if (failures.length > 1) throw new AggregateError(failures, 'queued-image teardown failed') + }) + + it.skipIf(MODE === 'record')('queues a text-plus-image submission with a thumbnail and delivers it as the next turn', async () => { + overrideDir = await mkdtemp(join(tmpdir(), 'dsh-web-queued-image-')) + const readyFile = join(overrideDir, '.hang-ready') + const overridePath = join(overrideDir, 'replay.override.json') + const recorded = deriveReplayScript(parseSessionLog(await readFile(FIXTURE, 'utf8'))) + expect(recorded).toHaveLength(1) + const replay: ReplayEntry[] = [ + { kind: 'hang', readyFile }, + recorded[0]!, + recorded[0]!, + ] + await writeFile(overridePath, JSON.stringify(replay)) + + const sessionEvents: SessionEvent[] = [] + scaffold = await launchWebScaffold({ replayFixture: FIXTURE, replayOverride: overridePath, compareReplaySession: false }) + scaffold.ctx.on('session/event', (_session, event: SessionEvent) => { sessionEvents.push(event) }) + browser = await chromium.launch() + page = await newEnglishPage(browser) + const tripwire = watchConsole(page) + await page.goto(scaffold.authenticatedUrl, { waitUntil: 'load' }) + await page.waitForSelector('[class*="frame"]', { timeout: 30_000 }) + await connectFreshWorkspace(page, scaffold.workspaceCwd) + onTestFailed(() => saveFailureShot(page, 'web-e2e-queued-image')) + + const input = page.locator('[data-composer-input]').first() + const firstSettled = scaffold.whenTurnSettled() + await input.fill(ACTIVE_PROMPT) + await input.press('Enter') + await expect.poll(() => existsSync(readyFile), { timeout: 15_000 }).toBe(true) + + // A just-submitted composer is read-only for the prompt round-trip. + await page.locator('[data-composer-input][contenteditable="true"]').first().waitFor({ timeout: 10_000 }) + await pasteImage(page, await readFile(PNG)) + await page.getByRole('img', { name: 'queued.png' }).waitFor({ timeout: 10_000 }) + await input.fill(QUEUED_TEXT) + await input.press('Enter') + + // The queued row renders the durable thumbnail beside the text preview. + const dockThumb = page.locator('[data-queue-dock] img[alt="Queued message image"]') + await dockThumb.waitFor({ timeout: 15_000 }) + await expect.poll(() => dockThumb.getAttribute('src')).toMatch(/^blob:/) + await page.getByText(QUEUED_TEXT, { exact: true }).waitFor() + await page.getByRole('button', { name: 'Remove queued message' }).waitFor({ timeout: 15_000 }) + const queuedSnapshot = await captureStableAria(page, '[class*="centerCol"]', scaffold.workspaceCwd) + await compareOrRefreshGolden(QUEUED_EXPECTED, queuedSnapshot, MODE) + + // Stop parks the accepted queue; the next waking send delivers the image + // message first (FIFO), then its own text as the following turn. + await page.getByRole('button', { name: 'Stop generating' }).click() + await firstSettled + await expect.poll(() => page.getByRole('button', { name: 'Stop generating' }).count()).toBe(0) + await dockThumb.waitFor({ timeout: 10_000 }) + + const settled = scaffold.whenTurnSettled() + await input.fill('Continue with the queued comparison') + await input.press('Enter') + await settled + // The queued image message and the waking text run as two further turns; + // wait for both to end so the final snapshot never captures a mid-reply + // frame (the aborted first turn precedes them). + await expect.poll( + () => sessionEvents.flatMap(event => event.type === 'turn/end' ? [event.data.reason.kind] : []), + { timeout: 15_000 }, + ).toEqual(['aborted', 'completed', 'completed']) + + // The delivered user message renders its image in Chat from the durable + // reference, and the dock row is gone. + await expect.poll( + () => page.locator('[data-queue-dock]').count(), + { timeout: 15_000 }, + ).toBe(0) + const chatImage = page.locator('[class*="userRow"] img') + await chatImage.first().waitFor({ timeout: 15_000 }) + const deliveredSnapshot = await captureStableAria(page, '[class*="centerCol"]', scaffold.workspaceCwd) + await compareOrRefreshGolden(DELIVERED_EXPECTED, deliveredSnapshot, MODE) + + // Model-visible means logged: the delivered message carries the durable + // reference (never base64), in the composer's canonical images-then-text order. + const delivered = sessionEvents.find(event => event.type === 'user/message' + && event.data.content.some(block => block.type === 'image')) + expect(delivered?.type === 'user/message' && delivered.data.content.map(block => block.type)).toEqual(['image', 'text']) + const imageBlock = delivered?.type === 'user/message' + ? delivered.data.content.find(block => block.type === 'image') + : undefined + expect(imageBlock?.type === 'image' && imageBlock.attachment.name).toBe('queued.png') + expect(JSON.stringify(sessionEvents)).not.toContain('base64') + expect(tripwire.pageErrors).toEqual([]) + expect(tripwire.warnings).toEqual([]) + }, 120_000) +}) diff --git a/apps/web/tsconfig.json b/apps/web/tsconfig.json index 2357de9d14..23760125d3 100644 --- a/apps/web/tsconfig.json +++ b/apps/web/tsconfig.json @@ -73,6 +73,7 @@ "tests/markdown-cjk-strong.e2e.ts", "tests/markdown-inline-code-links.e2e.ts", "tests/queue-actions.e2e.ts", + "tests/queue-image.e2e.ts", "tests/skill-invocation-policy.e2e.ts", "tests/skill-user-invoke.e2e.ts", "tests/permission-policy-context.e2e.ts", diff --git a/docs/event-producer-consumer.i18n.yaml b/docs/event-producer-consumer.i18n.yaml index 2ba670a4f6..c276a299e4 100644 --- a/docs/event-producer-consumer.i18n.yaml +++ b/docs/event-producer-consumer.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/event-producer-consumer.md -event-producer-consumer.md: ab677bcf094c557ca89b085a92cad6f291183b53 -event-producer-consumer.zh.md: 921234c2a76675241213446842746b24613de6f3 +event-producer-consumer.md: 0f93b10c698637b6db1b9688f9bed82b4a11eeb8 +event-producer-consumer.zh.md: 76b0b9c71387666bed57c6ec8a431262caa7bc8b diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index ab677bcf09..0f93b10c69 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -52,10 +52,10 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `settings/document-updated` | `emit` | [`packages/settings/settings/src/types.ts:105`](../packages/settings/settings/src/types.ts) | [`settings`](../packages/settings/settings) (`events.dispatch`) | `remotes` | | `settings/updated` | `emit` | [`packages/settings/settings/src/types.ts:92`](../packages/settings/settings/src/types.ts) | [`settings`](../packages/settings/settings) (`events.dispatch`) | [`settings`](../packages/settings/settings) | | `skills/change` | `emit` | [`packages/skill/skill/src/index.ts:298`](../packages/skill/skill/src/index.ts) | [`skill`](../packages/skill/skill) (`events.dispatch`) | - | -| `subagent/end` | `emit` | [`packages/subagent/subagent/src/index.ts:178`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude-code`](../packages/hooks/hooks-claude-code), `server`, [`subagent`](../packages/subagent/subagent) | -| `subagent/provider-added` | `emit` | [`packages/subagent/subagent/src/index.ts:152`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`emit`) | [`subagent`](../packages/subagent/subagent), [`tool-subagent`](../packages/subagent/tool-subagent) | -| `subagent/provider-removed` | `emit` | [`packages/subagent/subagent/src/index.ts:158`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`subagent`](../packages/subagent/subagent), [`tool-subagent`](../packages/subagent/tool-subagent) | -| `subagent/start` | `emit` | [`packages/subagent/subagent/src/index.ts:169`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude-code`](../packages/hooks/hooks-claude-code), [`subagent`](../packages/subagent/subagent) | +| `subagent/end` | `emit` | [`packages/subagent/subagent/src/index.ts:179`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude-code`](../packages/hooks/hooks-claude-code), `server`, [`subagent`](../packages/subagent/subagent) | +| `subagent/provider-added` | `emit` | [`packages/subagent/subagent/src/index.ts:153`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`emit`) | [`subagent`](../packages/subagent/subagent), [`tool-subagent`](../packages/subagent/tool-subagent) | +| `subagent/provider-removed` | `emit` | [`packages/subagent/subagent/src/index.ts:159`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`subagent`](../packages/subagent/subagent), [`tool-subagent`](../packages/subagent/tool-subagent) | +| `subagent/start` | `emit` | [`packages/subagent/subagent/src/index.ts:170`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude-code`](../packages/hooks/hooks-claude-code), [`subagent`](../packages/subagent/subagent) | | `system-prompt/assemble` | `waterfall` | [`packages/core/system-prompt/src/index.ts:31`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`waterfall`) | [`agent`](../packages/core/agent), [`agent-presets`](../packages/preset/agent-presets), [`system-prompt`](../packages/core/system-prompt) | | `system-prompt/change` | `emit` | [`packages/core/system-prompt/src/index.ts:37`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`emit`) | - | | `tools/change` | `emit` | [`packages/core/tools/src/index.ts:199`](../packages/core/tools/src/index.ts) | [`agent-presets`](../packages/preset/agent-presets) (`emit`), [`tools`](../packages/core/tools) (`emit`) | [`tool-subagent`](../packages/subagent/tool-subagent) | diff --git a/docs/event-producer-consumer.zh.md b/docs/event-producer-consumer.zh.md index 921234c2a7..76b0b9c713 100644 --- a/docs/event-producer-consumer.zh.md +++ b/docs/event-producer-consumer.zh.md @@ -54,10 +54,10 @@ | `settings/document-updated` | `emit` | [`packages/settings/settings/src/types.ts:105`](../packages/settings/settings/src/types.ts) | [`settings`](../packages/settings/settings) (`events.dispatch`) | `remotes` | | `settings/updated` | `emit` | [`packages/settings/settings/src/types.ts:92`](../packages/settings/settings/src/types.ts) | [`settings`](../packages/settings/settings) (`events.dispatch`) | [`settings`](../packages/settings/settings) | | `skills/change` | `emit` | [`packages/skill/skill/src/index.ts:298`](../packages/skill/skill/src/index.ts) | [`skill`](../packages/skill/skill) (`events.dispatch`) | - | -| `subagent/end` | `emit` | [`packages/subagent/subagent/src/index.ts:178`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude-code`](../packages/hooks/hooks-claude-code), `server`, [`subagent`](../packages/subagent/subagent) | -| `subagent/provider-added` | `emit` | [`packages/subagent/subagent/src/index.ts:152`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`emit`) | [`subagent`](../packages/subagent/subagent), [`tool-subagent`](../packages/subagent/tool-subagent) | -| `subagent/provider-removed` | `emit` | [`packages/subagent/subagent/src/index.ts:158`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`subagent`](../packages/subagent/subagent), [`tool-subagent`](../packages/subagent/tool-subagent) | -| `subagent/start` | `emit` | [`packages/subagent/subagent/src/index.ts:169`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude-code`](../packages/hooks/hooks-claude-code), [`subagent`](../packages/subagent/subagent) | +| `subagent/end` | `emit` | [`packages/subagent/subagent/src/index.ts:179`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude-code`](../packages/hooks/hooks-claude-code), `server`, [`subagent`](../packages/subagent/subagent) | +| `subagent/provider-added` | `emit` | [`packages/subagent/subagent/src/index.ts:153`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`emit`) | [`subagent`](../packages/subagent/subagent), [`tool-subagent`](../packages/subagent/tool-subagent) | +| `subagent/provider-removed` | `emit` | [`packages/subagent/subagent/src/index.ts:159`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`subagent`](../packages/subagent/subagent), [`tool-subagent`](../packages/subagent/tool-subagent) | +| `subagent/start` | `emit` | [`packages/subagent/subagent/src/index.ts:170`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude-code`](../packages/hooks/hooks-claude-code), [`subagent`](../packages/subagent/subagent) | | `system-prompt/assemble` | `waterfall` | [`packages/core/system-prompt/src/index.ts:31`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`waterfall`) | [`agent`](../packages/core/agent), [`agent-presets`](../packages/preset/agent-presets), [`system-prompt`](../packages/core/system-prompt) | | `system-prompt/change` | `emit` | [`packages/core/system-prompt/src/index.ts:37`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`emit`) | - | | `tools/change` | `emit` | [`packages/core/tools/src/index.ts:199`](../packages/core/tools/src/index.ts) | [`agent-presets`](../packages/preset/agent-presets) (`emit`), [`tools`](../packages/core/tools) (`emit`) | [`tool-subagent`](../packages/subagent/tool-subagent) | diff --git a/docs/subsystems/subagent.i18n.yaml b/docs/subsystems/subagent.i18n.yaml index 363ee89e12..cfb4151ab2 100644 --- a/docs/subsystems/subagent.i18n.yaml +++ b/docs/subsystems/subagent.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/subsystems/subagent.md -subagent.md: cedbc8544cb86db9feda4b0a2bcf5a65cf99bfe9 -subagent.zh.md: 4eabe673b0e750e0b3ed5480299b93a377886b13 +subagent.md: 095d73f43e75add0fab68aee97a6e6c0d27dbed2 +subagent.zh.md: e3df4c982cec86cf1701bb807715a20e18978677 diff --git a/docs/subsystems/subagent.md b/docs/subsystems/subagent.md index cedbc8544c..095d73f43e 100644 --- a/docs/subsystems/subagent.md +++ b/docs/subsystems/subagent.md @@ -669,10 +669,12 @@ listDescendants(rootSessionId: SessionId, signal?: AbortSignal): Promise block.type !== 'image') .map(block => (block.type === 'text' ? block.text : `[${block.type}]`)) .join(' ').replace(/\s+/g, ' ').trim() const chars = Array.from(flat) diff --git a/packages/api/session-controller/src/client/sessions/session.ts b/packages/api/session-controller/src/client/sessions/session.ts index 8af47b5aad..25d5e61082 100644 --- a/packages/api/session-controller/src/client/sessions/session.ts +++ b/packages/api/session-controller/src/client/sessions/session.ts @@ -184,6 +184,9 @@ export class Session implements SessionFace { const requestId = randomUUID() as SessionRequestId this.pendingSubmissions = [...this.pendingSubmissions, { requestId, + placement: this.running + ? input.mode === 'steer' ? 'steering' : 'queued' + : 'transcript', time: Date.now(), text: input.text, images: input.images, diff --git a/packages/api/session-controller/src/commands.ts b/packages/api/session-controller/src/commands.ts index 1ad9127925..0988660659 100644 --- a/packages/api/session-controller/src/commands.ts +++ b/packages/api/session-controller/src/commands.ts @@ -4,12 +4,12 @@ import { randomUUID } from 'node:crypto' import type { Context } from '@deepseek-ai/cordis' import { brandString } from '@deepseek-ai/dsh-brand' import type { Agent, ModelSelection as AgentModelSelection } from '@deepseek-ai/dsh-agent' -import { AttachmentError, admitEncodedImages } from '@deepseek-ai/dsh-attachment' +import { AttachmentError, admitPromptContent } from '@deepseek-ai/dsh-attachment' import type { ImageAttachmentRef } from '@deepseek-ai/dsh-attachment' import { ReasoningEffortId, createUserMessage, freezeMessage, } from '@deepseek-ai/dsh-llm' -import type { ContentBlock, MessageSource } from '@deepseek-ai/dsh-llm' +import type { MessageSource } from '@deepseek-ai/dsh-llm' import type { SessionEvent, SessionHeader, SessionId, UserMessage } from '@deepseek-ai/dsh-session' import { SessionQueryError, type SessionObservation } from '@deepseek-ai/dsh-session-query' import { SessionTitleInvalidError } from '@deepseek-ai/dsh-session-title' @@ -319,7 +319,7 @@ export class SessionCommandController { ) } } - const content = await durablePromptContent(this.ctx, request.content) + const content = await admitPromptContent(this.ctx.attachments, request.content) const message: UserMessage = createUserMessage({ content, source }) if (request.mode === 'steer') agent.steer(message) else agent.followup(message) @@ -492,21 +492,6 @@ export class SessionCommandController { } } -async function durablePromptContent( - ctx: Context, - content: readonly SessionPromptRequest['content'][number][], -): Promise { - if (content.every(part => part.type === 'text')) { - return content.map(part => ({ type: 'text', text: part.text })) - } - const refs = await admitEncodedImages(ctx.attachments, content.filter(part => part.type === 'image')) - let next = 0 - return content.map(part => part.type === 'text' - ? { type: 'text', text: part.text } - // admitEncodedImages returns one reference per image part in order. - : { type: 'image', attachment: refs[next++] as ImageAttachmentRef }) -} - function imageBlockIn( content: unknown, match: (ref: ImageAttachmentRef) => boolean, diff --git a/packages/api/session-controller/tests/client-contract.client.spec.ts b/packages/api/session-controller/tests/client-contract.client.spec.ts index 796ba1727c..bb9cd172ce 100644 --- a/packages/api/session-controller/tests/client-contract.client.spec.ts +++ b/packages/api/session-controller/tests/client-contract.client.spec.ts @@ -1,7 +1,9 @@ -import { describe, expect, it, vi } from 'vitest' +import { describe, expect, expectTypeOf, it, vi } from 'vitest' +import type { PromptContentPart as AttachmentPromptContentPart } from '@deepseek-ai/dsh-attachment/types' import { MutableSessionEventSource, type SessionLiveEventEntry, } from '../src/client/contract/events.ts' +import type { PromptContentPart as SessionPromptContentPart } from '../src/types.ts' function entry(seq: number): SessionLiveEventEntry { return { @@ -16,6 +18,10 @@ function entry(seq: number): SessionLiveEventEntry { } describe('Client Session contracts', () => { + it('keeps its catalog-visible prompt parts identical to attachment intake', () => { + expectTypeOf().toEqualTypeOf() + }) + it('publishes exact replace, prepend, and append event-window changes', () => { const feed = new MutableSessionEventSource() const listener = vi.fn() diff --git a/packages/api/session-controller/tests/queue-store.client.spec.ts b/packages/api/session-controller/tests/queue-store.client.spec.ts index 6fc045cf70..856a861d5a 100644 --- a/packages/api/session-controller/tests/queue-store.client.spec.ts +++ b/packages/api/session-controller/tests/queue-store.client.spec.ts @@ -73,7 +73,7 @@ describe('Session queue snapshot intake', () => { ]) }) - it('marks mixed-content messages non-editable while retaining their preview', () => { + it('marks mixed-content messages non-editable and keeps image blocks out of the text preview', () => { const session = makeSession() session.handleControlFrame(queueFrame([{ id: 'q-image', @@ -86,7 +86,9 @@ describe('Session queue snapshot intake', () => { { id: 'q-image', placement: 'queued', content: [{ type: 'text', text: 'hi' }, { type: 'image', data: 'x' }], - preview: 'hi [image]', text: null, + // Image blocks render as thumbnails from `content`, so the preview + // carries only the text; non-image foreign blocks keep their marker. + preview: 'hi', text: null, }, ]) }) diff --git a/packages/api/session-controller/tests/session-models.host.spec.ts b/packages/api/session-controller/tests/session-models.host.spec.ts index f0d208e821..cd26dfb39d 100644 --- a/packages/api/session-controller/tests/session-models.host.spec.ts +++ b/packages/api/session-controller/tests/session-models.host.spec.ts @@ -227,6 +227,58 @@ describe('Web session model selection', () => { await ctx.fiber.dispose() }) + it('delivers an admitted image batch through steer with the same ordered content as queue', async () => { + const { ctx, agent, sessionId } = await harness() + const attachments = { + imageLimits: { + maxImageBytes: 4, + maxImagesPerMessage: 2, + maxMessageImageBytes: 4, + maxImagePixels: 4, + maxImageDimension: 2000, + mediaTypes: ['image/png'], + }, + validateImage: vi.fn(() => Promise.resolve()), + saveImage: vi.fn((input: { data: Uint8Array; mediaType: 'image/png'; name?: string }) => Promise.resolve({ + attachmentId: `att-${String(input.data[0])}`, + mediaType: input.mediaType, + bytes: input.data.byteLength, + width: 1, + height: 1, + ...input.name === undefined ? {} : { name: input.name }, + })), + } + ctx.provide('attachments', Object.setPrototypeOf(attachments, AttachmentStore.prototype) as never) + const steer = vi.fn() + const followup = vi.fn() + Object.assign(agent, { steer, followup }) + const remote = createSessionTestRemote(ctx, { + defaultModelSelection: () => ({ provider: 'deepseek-official', model: 'deepseek-chat' }), + cwd: '/tmp', + }) + + const result = await remote.prompt(promptRequest({ + sessionId, + mode: 'steer' as const, + content: [ + { type: 'text' as const, text: 'look at this' }, + { type: 'image' as const, mediaType: 'image/png' as const, data: 'AQ==', name: 'mid-turn.png' }, + ], + })) + expect(result.ok).toBe(true) + expect(followup).not.toHaveBeenCalled() + expect((steer.mock.calls[0]?.[0] as UserMessage).content).toEqual([ + { type: 'text', text: 'look at this' }, + { + type: 'image', + attachment: { + attachmentId: 'att-1', mediaType: 'image/png', bytes: 1, width: 1, height: 1, name: 'mid-turn.png', + }, + }, + ]) + await ctx.fiber.dispose() + }) + it('allows a text-only selection while durable or pending images remain available for later models', async () => { const { ctx, agent, sessionId } = await harness() registerTextOnly(ctx) diff --git a/packages/api/session-controller/tests/session-pending-submissions.client.spec.ts b/packages/api/session-controller/tests/session-pending-submissions.client.spec.ts index 6805745666..e5aad4564b 100644 --- a/packages/api/session-controller/tests/session-pending-submissions.client.spec.ts +++ b/packages/api/session-controller/tests/session-pending-submissions.client.spec.ts @@ -70,21 +70,38 @@ describe('beginSubmission', () => { const { session } = makeSession() expect(session.getSnapshot()).toMatchObject({ pendingSubmissions: [], promptAttempted: false }) const handle = session.beginSubmission({ + mode: 'queue', text: '你好', images: [{ previewUrl: 'blob:p1', name: 'a.png', width: 4, height: 3 }], }) expect(session.getSnapshot().promptAttempted).toBe(true) expect(session.getSnapshot().pendingSubmissions).toMatchObject([{ requestId: handle.requestId, + placement: 'transcript', text: '你好', images: [{ previewUrl: 'blob:p1', name: 'a.png', width: 4, height: 3 }], }]) }) + it('derives and captures the echo placement from running state and delivery mode', () => { + const { session } = makeSession() + session.beginSubmission({ mode: 'queue', text: '空闲', images: [] }) + session.handleRunning(true) + session.beginSubmission({ mode: 'queue', text: '排队', images: [] }) + session.beginSubmission({ mode: 'steer', text: '纠偏', images: [] }) + session.handleRunning(false) + expect(session.getSnapshot().pendingSubmissions.map(({ text, placement }) => ({ text, placement }))).toEqual([ + { text: '空闲', placement: 'transcript' }, + { text: '排队', placement: 'queued' }, + { text: '纠偏', placement: 'steering' }, + ]) + }) + it('abandon retires the echo as failed exactly once', () => { const { session } = makeSession() const retirements: PendingSubmissionRetirement[] = [] const handle = session.beginSubmission({ + mode: 'queue', text: '放弃', images: [], onRetire: retirement => retirements.push(retirement), @@ -102,6 +119,7 @@ describe('prompt-coupled retirement', () => { api.onPrompt = () => Promise.resolve(err(new RemoteError('session/agent-busy', '忙', { reason: 'busy' }))) const retirements: PendingSubmissionRetirement[] = [] const handle = session.beginSubmission({ + mode: 'queue', text: '失败的', images: [], onRetire: retirement => retirements.push(retirement), @@ -115,7 +133,7 @@ describe('prompt-coupled retirement', () => { it('sends the echo identity as the prompt requestId', async () => { const { api, session } = makeSession() - const handle = session.beginSubmission({ text: '带 id', images: [] }) + const handle = session.beginSubmission({ mode: 'queue', text: '带 id', images: [] }) await session.prompt([{ type: 'text', text: '带 id' }], 'queue', undefined, handle.requestId) expect(api.callsOf('session.prompt')).toMatchObject([{ requestId: handle.requestId }]) }) @@ -123,7 +141,7 @@ describe('prompt-coupled retirement', () => { it('an unidentified prompt failure leaves registered echoes alone', async () => { const { api, session } = makeSession() api.onPrompt = () => Promise.resolve(err(new RemoteError('session/agent-busy', '忙', { reason: 'busy' }))) - session.beginSubmission({ text: '还在', images: [] }) + session.beginSubmission({ mode: 'queue', text: '还在', images: [] }) await session.prompt([{ type: 'text', text: '另一个' }], 'queue') expect(session.getSnapshot().pendingSubmissions).toHaveLength(1) }) @@ -136,6 +154,7 @@ describe('observed retirement', () => { await session.open() const retirements: PendingSubmissionRetirement[] = [] const handle = session.beginSubmission({ + mode: 'queue', text: '发送', images: [{ previewUrl: 'blob:p1' }], onRetire: retirement => retirements.push(retirement), @@ -153,7 +172,9 @@ describe('observed retirement', () => { it('a queue occurrence carrying the rpcId retires the echo (running-turn submissions)', async () => { const { session } = makeSession() const retirements: PendingSubmissionRetirement[] = [] + session.handleRunning(true) const handle = session.beginSubmission({ + mode: 'queue', text: '排队', images: [{ previewUrl: 'blob:p1' }], onRetire: retirement => retirements.push(retirement), @@ -169,7 +190,7 @@ describe('observed retirement', () => { it('a full-window install (reconnect resync) retires echoes observed in the window', async () => { const { api, session } = makeSession() - const handle = session.beginSubmission({ text: '重连', images: [] }) + const handle = session.beginSubmission({ mode: 'queue', text: '重连', images: [] }) api.onHistory = () => Promise.resolve(ok(historyValue([promptEvent(12, handle.requestId)]))) await session.open() await settleFrames() @@ -182,6 +203,7 @@ describe('observed retirement', () => { await session.open() const retirements: PendingSubmissionRetirement[] = [] const handle = session.beginSubmission({ + mode: 'queue', text: '先观察', images: [], onRetire: retirement => retirements.push(retirement), @@ -198,6 +220,7 @@ describe('observed retirement', () => { await session.open() const retirements: PendingSubmissionRetirement[] = [] const handle = session.beginSubmission({ + mode: 'queue', text: '同一请求', images: [], onRetire: retirement => retirements.push(retirement), @@ -222,7 +245,7 @@ describe('observed retirement', () => { const { api, session } = makeSession() api.onHistory = () => Promise.resolve(ok(historyValue([]))) await session.open() - const handle = session.beginSubmission({ text: '帧', images: [] }) + const handle = session.beginSubmission({ mode: 'queue', text: '帧', images: [] }) await api.pushFollow(SID, { type: 'event', event: promptEvent(0, handle.requestId) as never }) expect(session.getSnapshot().pendingSubmissions).toHaveLength(1) expect(frames).toHaveLength(1) @@ -238,11 +261,13 @@ describe('disposal', () => { await session.open() const retirements: { text: string; retirement: PendingSubmissionRetirement }[] = [] const observed = session.beginSubmission({ + mode: 'queue', text: '已观察', images: [], onRetire: retirement => retirements.push({ text: '已观察', retirement }), }) session.beginSubmission({ + mode: 'queue', text: '未settle', images: [], onRetire: retirement => retirements.push({ text: '未settle', retirement }), diff --git a/packages/api/session-controller/tests/session.client.spec.ts b/packages/api/session-controller/tests/session.client.spec.ts index 8d7e4319d3..319c315873 100644 --- a/packages/api/session-controller/tests/session.client.spec.ts +++ b/packages/api/session-controller/tests/session.client.spec.ts @@ -318,6 +318,32 @@ describe('prompt and cancel errors', () => { }) }) + it('forwards continuation image parts to the subagent prompt Remote unstripped', async () => { + const api = new FakeApiClient() + const session = new Session(SID, fakeRemote(api), { + address: { parentSessionId: PARENT, childSessionId: SID, mode: 'continuable' }, + parentAvailable: true, + }) + await session.open() + const content = [ + { type: 'text' as const, text: '看这张图' }, + { type: 'image' as const, mediaType: 'image/png' as const, data: 'aGk=', name: 'shot.png' }, + ] + const prompted = await session.prompt(content, 'queue') + + expect(prompted).toEqual({ ok: true, value: { accepted: true } }) + expect(api.callsOf('subagents.prompt')).toEqual([ + { + requestId: expect.any(String) as unknown as string, + parentSessionId: PARENT, childSessionId: SID, + mode: 'continuable', + content, + clientTimeZone: new Intl.DateTimeFormat().resolvedOptions().timeZone, + }, + ]) + expect(session.getSnapshot().promptError).toBeNull() + }) + it('lands an interrupt business failure in promptError with op=stop', async () => { const api = new FakeApiClient() api.onSubagentInterrupt = () => Promise.resolve(err(new RemoteError('subagent/unauthorized', 'nope', { childSessionId: SID }))) @@ -366,13 +392,8 @@ describe('prompt and cancel errors', () => { expect(api.callsOf('session.cancel')).toEqual([]) }) - it('delivers an image continuation to the Host, which refuses it', async () => { + it('delivers an image continuation to the Host without narrowing its upload parts', async () => { const api = new FakeApiClient() - api.onSubagentPrompt = () => Promise.resolve(err(new RemoteError( - 'subagent/attachment-unsupported', - 'subagent continuation does not accept images', - { childSessionId: SID, reason: 'SUBAGENT_IMAGE_UNSUPPORTED' }, - ))) const session = new Session(SID, fakeRemote(api), { address: { parentSessionId: PARENT, childSessionId: SID, mode: 'continuable' }, }) @@ -382,11 +403,7 @@ describe('prompt and cancel errors', () => { 'queue', ) - expect(prompted).toMatchObject({ - ok: false, - error: { code: 'subagent/attachment-unsupported', details: { reason: 'SUBAGENT_IMAGE_UNSUPPORTED' } }, - }) - // The image reaches the wire unfiltered: refusing it is the Host's call. + expect(prompted).toEqual({ ok: true, value: { accepted: true } }) expect(api.callsOf('subagents.prompt')).toMatchObject([ { content: [{ type: 'text' }, { type: 'image', mediaType: 'image/png', data: 'AA==' }] }, ]) diff --git a/packages/attachment/attachment/README.i18n.yaml b/packages/attachment/attachment/README.i18n.yaml index e67d95604d..6cc6a74eeb 100644 --- a/packages/attachment/attachment/README.i18n.yaml +++ b/packages/attachment/attachment/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/attachment/attachment/README.md -README.md: 01a5d6ee143ff53175dd7327cd6d419af61938a3 -README.zh.md: 1a1d297297a6815ce5338391af0182e471f99000 +README.md: 709189ac8dd9591d89c0b0082ea0ae0247c97f97 +README.zh.md: 24eee83874d2ad12025d4457f1fad74aadb73446 diff --git a/packages/attachment/attachment/README.md b/packages/attachment/attachment/README.md index 01a5d6ee14..709189ac8d 100644 --- a/packages/attachment/attachment/README.md +++ b/packages/attachment/attachment/README.md @@ -75,7 +75,7 @@ The service family runs one admission-and-storage flow: every entry point enforc |---|---| | [`src/index.ts`](src/index.ts) | Plugin entry: abstract `AttachmentStore` service and re-exports | | [`src/types.ts`](src/types.ts) | Durable vocabulary: references, limits, upload and store payloads | -| [`src/admission.ts`](src/admission.ts) | `admitEncodedImages`: canonical-base64 enforcement, then `saveImages` delegation | +| [`src/admission.ts`](src/admission.ts) | Browser prompt admission: canonical-base64 enforcement, `saveImages` delegation, and durable prompt-part projection | | [`src/error.ts`](src/error.ts) | `AttachmentError` class and the `isImageAdmissionError` runtime subset | | [`src/brand.ts`](src/brand.ts) | `AttachmentId` branded opaque identifier | | [`src/invariant.ts`](src/invariant.ts) | Invariant companion (no runtime invariant; implementations enforce immutable-store checks) | diff --git a/packages/attachment/attachment/README.zh.md b/packages/attachment/attachment/README.zh.md index 1a1d297297..24eee83874 100644 --- a/packages/attachment/attachment/README.zh.md +++ b/packages/attachment/attachment/README.zh.md @@ -75,7 +75,7 @@ kind: "package-reference" |---|---| | [`src/index.ts`](src/index.ts) | 插件入口：抽象 `AttachmentStore` 服务与再导出 | | [`src/types.ts`](src/types.ts) | 持久词汇：引用、限额、上传与存储载荷 | -| [`src/admission.ts`](src/admission.ts) | `admitEncodedImages`：规范 base64 强制，随后委托 `saveImages` | +| [`src/admission.ts`](src/admission.ts) | 浏览器 prompt 准入：强制规范 base64、委托 `saveImages` 并投影持久 prompt part | | [`src/error.ts`](src/error.ts) | `AttachmentError` 类与 `isImageAdmissionError` 运行时子集 | | [`src/brand.ts`](src/brand.ts) | `AttachmentId` 带类型标记的不透明标识符 | | [`src/invariant.ts`](src/invariant.ts) | 不变式伴生插件（无运行时不变式；实现负责强制不可变存储检查） | diff --git a/packages/attachment/attachment/src/admission.ts b/packages/attachment/attachment/src/admission.ts index d31bc6d831..155b3644d8 100644 --- a/packages/attachment/attachment/src/admission.ts +++ b/packages/attachment/attachment/src/admission.ts @@ -3,7 +3,13 @@ import { Buffer } from 'node:buffer' import { AttachmentError } from './error.ts' import type { AttachmentStore } from './index.ts' -import type { EncodedImageAttachment, ImageAttachmentRef, SaveImageAttachment } from './types.ts' +import type { + AdmittedPromptContentPart, + EncodedImageAttachment, + ImageAttachmentRef, + PromptContentPart, + SaveImageAttachment, +} from './types.ts' /** Decode one upload payload while rejecting non-canonical base64 forms. */ function decodeBase64(data: string): Uint8Array { @@ -39,3 +45,26 @@ export async function admitEncodedImages( ): Promise { return attachments.saveImages(images.map(saveInput)) } + +/** + * Admit one browser prompt and replace each uploaded image with its durable reference. + * Text-only prompts do not access the attachment store. + * @param attachments - the deployment attachment store owning batch policy. + * @param content - browser prompt parts in message order. + * @returns admitted prompt parts in the same order as `content`. + * @throws AttachmentError when the image batch is refused. + */ +export async function admitPromptContent( + attachments: AttachmentStore, + content: readonly PromptContentPart[], +): Promise { + if (content.every(part => part.type === 'text')) { + return content.map(part => ({ type: 'text', text: part.text })) + } + const refs = await admitEncodedImages(attachments, content.filter(part => part.type === 'image')) + let next = 0 + return content.map(part => part.type === 'text' + ? { type: 'text', text: part.text } + // admitEncodedImages returns one reference per image part in order. + : { type: 'image', attachment: refs[next++] as ImageAttachmentRef }) +} diff --git a/packages/attachment/attachment/src/index.ts b/packages/attachment/attachment/src/index.ts index 4ee001b86c..9dcddeae65 100644 --- a/packages/attachment/attachment/src/index.ts +++ b/packages/attachment/attachment/src/index.ts @@ -14,15 +14,17 @@ import type { export { AttachmentId, ImageVariantId } from './brand.ts' export { AttachmentError, isImageAdmissionError } from './error.ts' export type { AttachmentErrorCode, ImageAdmissionErrorCode } from './error.ts' -export { admitEncodedImages } from './admission.ts' +export { admitEncodedImages, admitPromptContent } from './admission.ts' export { requestImageDimensions } from './request-projection.ts' export type { AttachmentId as AttachmentIdType, + AdmittedPromptContentPart, EncodedImageAttachment, ImageAttachmentLimits, ImageAttachmentRef, ImageRequestPolicy, ImageMediaType, + PromptContentPart, RequestImageAttachment, SaveImageAttachment, StoredImageAttachment, diff --git a/packages/attachment/attachment/src/types.ts b/packages/attachment/attachment/src/types.ts index 046444cd76..7a55c6a68f 100644 --- a/packages/attachment/attachment/src/types.ts +++ b/packages/attachment/attachment/src/types.ts @@ -52,6 +52,26 @@ export interface EncodedImageAttachment { name?: string } +/** + * Browser-submitted prompt content accepted by Host prompt endpoints; the + * accepting Host promotes image parts to durable references through + * `admitPromptContent` before any message is created, so a wire caller can + * never cite an attachment it did not upload. + */ +export type PromptContentPart = + | { readonly type: 'text'; readonly text: string } + | { + readonly type: 'image' + readonly mediaType: ImageMediaType + readonly data: string + readonly name?: string + } + +/** Host-admitted prompt content with each uploaded image replaced by its durable reference. */ +export type AdmittedPromptContentPart = + | { readonly type: 'text'; readonly text: string } + | { readonly type: 'image'; readonly attachment: ImageAttachmentRef } + /** Request to validate and durably commit one image. */ export interface SaveImageAttachment { data: Uint8Array diff --git a/packages/attachment/attachment/tests/admission.spec.ts b/packages/attachment/attachment/tests/admission.spec.ts index 4c929b6d5c..ba24d8fa20 100644 --- a/packages/attachment/attachment/tests/admission.spec.ts +++ b/packages/attachment/attachment/tests/admission.spec.ts @@ -1,6 +1,6 @@ import { describe, expect, it, vi } from 'vitest' import type { AttachmentStore } from '@deepseek-ai/dsh-attachment' -import { admitEncodedImages } from '@deepseek-ai/dsh-attachment' +import { admitEncodedImages, admitPromptContent } from '@deepseek-ai/dsh-attachment' import type { ImageAttachmentRef, SaveImageAttachment } from '@deepseek-ai/dsh-attachment/types' const PNG = 'AAAA' // canonical base64, 3 bytes @@ -64,3 +64,25 @@ describe('admitEncodedImages', () => { await expect(admitEncodedImages(store, [{ mediaType: 'image/png', data: PNG }])).rejects.toBe(refused) }) }) + +describe('admitPromptContent', () => { + it('converts text-only prompts without touching the attachment store', async () => { + const store = { saveImages: () => { throw new Error('text-only prompts must not reach the store') } } + await expect(admitPromptContent(store as unknown as AttachmentStore, [ + { type: 'text', text: 'hello' }, + ])).resolves.toEqual([{ type: 'text', text: 'hello' }]) + }) + + it('replaces image parts with admitted references in part order', async () => { + const { store } = storeOf() + await expect(admitPromptContent(store, [ + { type: 'image', mediaType: 'image/png', data: 'AQ==' }, + { type: 'text', text: 'between' }, + { type: 'image', mediaType: 'image/png', data: 'Ag==' }, + ])).resolves.toEqual([ + { type: 'image', attachment: { attachmentId: 'att-1', mediaType: 'image/png', bytes: 1, width: 1, height: 1 } }, + { type: 'text', text: 'between' }, + { type: 'image', attachment: { attachmentId: 'att-2', mediaType: 'image/png', bytes: 1, width: 1, height: 1 } }, + ]) + }) +}) diff --git a/packages/client/ui-chat/README.i18n.yaml b/packages/client/ui-chat/README.i18n.yaml index a38a72ef3e..ea2377aa37 100644 --- a/packages/client/ui-chat/README.i18n.yaml +++ b/packages/client/ui-chat/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/client/ui-chat/README.md -README.md: 5c7549da07e8101c6b08b4cad57afd5158980b7f -README.zh.md: a558bf94f5436eebdf035ca7507ecd25ec3c8d8a +README.md: 4f5e969197453ee030e2deb8a09f36d3327b5ed0 +README.zh.md: 47ed92366f6c9e0c5b54c8d5095d30c8561bad47 diff --git a/packages/client/ui-chat/README.md b/packages/client/ui-chat/README.md index 5c7549da07..4f5e969197 100644 --- a/packages/client/ui-chat/README.md +++ b/packages/client/ui-chat/README.md @@ -8,7 +8,7 @@ English | [中文](README.zh.md) ## Summary -The browser Chat target for Conversation assembly. It registers Chat event definitions and snapshot construction, supplies `useChat`, renders transcript nodes and details, and owns Chat-specific stores, actions, localization, and scroll restoration; historical image URLs resolve through the Conversation-owned per-session cache (`ctx.uiConversation.imageUrl`). Its Assistant and Turn Tail definitions fold packed historical Assistant runs without expanding their members. The flow tail renders the session's local submission echoes (`SessionSnapshot.pendingSubmissions`) with the same bubble as their eventual durable user nodes, hidden per render once a user/steering node or queue occurrence carries the echo's prompt `rpcId`, so the echo-to-durable swap is atomic. +The browser Chat target for Conversation assembly. It registers Chat event definitions and snapshot construction, supplies `useChat`, renders transcript nodes and details, and owns Chat-specific stores, actions, localization, and scroll restoration; historical image URLs resolve through the Conversation-owned per-session cache (`ctx.uiConversation.imageUrl`). Its Assistant and Turn Tail definitions fold packed historical Assistant runs without expanding their members. Local submission echoes (`SessionSnapshot.pendingSubmissions`) retain the surface selected when the submit begins: transcript echoes render at the flow tail, steering echoes render with the pending-steering marker, and queued echoes stay out of Chat. Each echo is hidden per render once a user/steering node or queue occurrence carries its prompt `rpcId`, so the handoff is atomic. ## Table of Contents diff --git a/packages/client/ui-chat/README.zh.md b/packages/client/ui-chat/README.zh.md index a558bf94f5..47ed92366f 100644 --- a/packages/client/ui-chat/README.zh.md +++ b/packages/client/ui-chat/README.zh.md @@ -8,7 +8,7 @@ kind: "package-reference" ## 概述 -Conversation 组装的浏览器 Chat target。本包注册 Chat event definition 与 snapshot 构造、提供 `useChat`、渲染 transcript node 和详情，并拥有 Chat 专属 store、action、本地化与滚动位置恢复；历史图片 URL 通过 Conversation 持有的按会话缓存（`ctx.uiConversation.imageUrl`）解析。其中 Assistant 与 Turn Tail definition 会直接 fold packed Assistant 历史 run，不展开其成员。消息流尾部渲染 session 的本地提交回显（`SessionSnapshot.pendingSubmissions`），气泡与其最终的 durable user 节点一致；一旦某个 user/steering 节点或 queue occurrence 携带回显的 prompt `rpcId`，该回显即在同一渲染中隐藏，因此回显到 durable 的替换是原子的。 +Conversation 组装的浏览器 Chat target。本包注册 Chat event definition 与 snapshot 构造、提供 `useChat`、渲染 transcript node 和详情，并拥有 Chat 专属 store、action、本地化与滚动位置恢复；历史图片 URL 通过 Conversation 持有的按会话缓存（`ctx.uiConversation.imageUrl`）解析。其中 Assistant 与 Turn Tail definition 会直接 fold packed Assistant 历史 run，不展开其成员。本地提交回显（`SessionSnapshot.pendingSubmissions`）保留提交开始时选定的区域：transcript 回显位于消息流末尾，steering 回显带 pending-steering 标记，queued 回显不进入 Chat。一旦 user/steering 节点或 queue occurrence 携带回显的 prompt `rpcId`，该回显即在同一渲染中隐藏，因此交接是原子的。 ## 目录 diff --git a/packages/client/ui-chat/src/client/chat/ChatView.tsx b/packages/client/ui-chat/src/client/chat/ChatView.tsx index b5f0313f15..b2b07d4229 100644 --- a/packages/client/ui-chat/src/client/chat/ChatView.tsx +++ b/packages/client/ui-chat/src/client/chat/ChatView.tsx @@ -271,7 +271,9 @@ export function ChatView({ const visibleSubmissions = useMemo(() => { if (pendingSubmissions.length === 0) return pendingSubmissions const observed = observedRpcIds(order, nodeStore, inbox) - return pendingSubmissions.filter(submission => !observed.has(submission.requestId)) + return pendingSubmissions.filter(submission => ( + submission.placement !== 'queued' && !observed.has(submission.requestId) + )) }, [pendingSubmissions, order, nodeStore, inbox]) const renderMessageImages = useCallback( owner => renderSlot('conversation.message.images', { ...owner, loadImage }), diff --git a/packages/client/ui-chat/src/client/chat/MessageItem.tsx b/packages/client/ui-chat/src/client/chat/MessageItem.tsx index cb8084fe41..9846a472d6 100644 --- a/packages/client/ui-chat/src/client/chat/MessageItem.tsx +++ b/packages/client/ui-chat/src/client/chat/MessageItem.tsx @@ -224,10 +224,10 @@ export function PendingSteeringBubble({ content, renderMessageImages, t }: { } /** - * Render one local submission echo with the exact visual language of the - * durable user node that replaces it: draft text plus object-URL previews, - * visible from the submit click until the durable `user/message` (or its - * queue occurrence) renders. + * Render one local transcript or steering submission echo with the same + * visual language and surface marker as the Host occurrence that replaces + * it: draft text plus object-URL previews, visible from the submit click + * until the durable `user/message` or steering occurrence renders. * @param props - the session snapshot's pending submission and render seats. * @returns the echoed user bubble. */ @@ -256,6 +256,7 @@ export function PendingSubmissionBubble({ submission, renderMessageImages, t }: content={content} previewImages={previewImages} renderMessageImages={renderMessageImages} + pending={submission.placement === 'steering'} echo t={t} actions={text => ( diff --git a/packages/client/ui-chat/tests/chat-view.client.spec.tsx b/packages/client/ui-chat/tests/chat-view.client.spec.tsx index 2acc40853e..d95a3a4099 100644 --- a/packages/client/ui-chat/tests/chat-view.client.spec.tsx +++ b/packages/client/ui-chat/tests/chat-view.client.spec.tsx @@ -857,7 +857,10 @@ describe('ChatView', () => { { nodes: [assistant(1, 'working')] }, { pendingSubmissions: [ - { requestId: 'req-1' as never, time: 5_000, text: '即发即显', images: [] }, + { + requestId: 'req-1' as never, placement: 'transcript', + time: 5_000, text: '即发即显', images: [], + }, ], }, ) @@ -886,18 +889,57 @@ describe('ChatView', () => { expect(view.getAllByText('即发即显')).toHaveLength(1) }) - it('hides an echo once its queue occurrence carries the rpcId (running-turn submission)', () => { + it('renders a local steer echo as pending steering before Host image admission completes', () => { + const h = makeHarness( + { nodes: [assistant(1, 'working')] }, + { + running: true, + pendingSubmissions: [{ + requestId: 'req-steer' as never, + placement: 'steering', + time: 5_500, + text: '带图纠偏', + images: [{ previewUrl: 'blob:steer-preview', name: 'steer.png' }], + }], + }, + ) + const view = render() + const local = view.getByText('带图纠偏').closest('[data-submission-echo]') + expect(local?.hasAttribute('data-pending-steering')).toBe(true) + + act(() => { + h.setSession({ + queue: [{ + id: 'steer-occurrence' as never, + messageId: 'steer-message' as never, + placement: 'steering', + rpcId: 'req-steer' as never, + content: [{ type: 'text', text: '带图纠偏' }], + preview: '带图纠偏', + text: '带图纠偏', + }], + }) + }) + expect(view.getAllByText('带图纠偏')).toHaveLength(1) + expect(view.container.querySelector('[data-submission-echo]')).toBeNull() + expect(view.container.querySelector('[data-pending-steering]')).not.toBeNull() + }) + + it('keeps a queued echo out of the Chat flow before and after Host admission', () => { const h = makeHarness( { nodes: [assistant(1, 'working')] }, { running: true, pendingSubmissions: [ - { requestId: 'req-q' as never, time: 6_000, text: '排队中', images: [] }, + { + requestId: 'req-q' as never, placement: 'queued', + time: 6_000, text: '排队中', images: [], + }, ], }, ) const view = render() - expect(view.getByText('排队中')).toBeTruthy() + expect(view.queryByText('排队中')).toBeNull() act(() => { h.setSession({ queue: [{ @@ -911,8 +953,8 @@ describe('ChatView', () => { }], }) }) - // The queued occurrence renders in the queue dock, not the flow; the - // flow-tail echo yields to it in the same snapshot. + // The queued occurrence and its local predecessor both belong to the + // queue dock, never the Chat flow. expect(view.queryByText('排队中')).toBeNull() }) @@ -922,6 +964,7 @@ describe('ChatView', () => { { pendingSubmissions: [{ requestId: 'req-img' as never, + placement: 'transcript', time: 7_000, text: '', images: [ diff --git a/packages/client/ui-conversation/README.i18n.yaml b/packages/client/ui-conversation/README.i18n.yaml index a293070d39..26499140f2 100644 --- a/packages/client/ui-conversation/README.i18n.yaml +++ b/packages/client/ui-conversation/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/client/ui-conversation/README.md -README.md: bcce1e77c45cfa75c6a7fe9c45a98b1036d9ea63 -README.zh.md: 1d24141945c68d7e948a730ff5b93e8d2e38e9bb +README.md: 629b8b4f7987fc072066e58692396354f7b6ad6a +README.zh.md: cb622f0308ddb0a978cfb665105d79aadfb3135a diff --git a/packages/client/ui-conversation/README.md b/packages/client/ui-conversation/README.md index bcce1e77c4..629b8b4f79 100644 --- a/packages/client/ui-conversation/README.md +++ b/packages/client/ui-conversation/README.md @@ -38,9 +38,9 @@ The package registers the optional-Session `conversation` shell, strict Session View selection is deterministic: a registered persisted selection wins, otherwise registered `chat` wins, otherwise no View renders. It never chooses the first registered View. Shell phase combines Session lifecycle with the active-target set; no target-specific snapshot is read by the shell. -The resident composer survives no-Session and Session transitions. The no-Session state keeps the same composer surface mounted but inert while the Workspace picker connects a blank Session. The surface is a shell-owned Lexical editor: reference chips are atomic decorator nodes carrying the owner's serialization identity (submission expands them through the owner codec), claimed slash commands stay styled leading text, folder text references carry the folder glyph as an icon prefix, and the draft's clipboard projection is mirrored into the per-Session Conversation store. Queue operations address exact queue occurrences through the scoped `ctx.conversation` service; queue previews render sent text through the shared inline reference projection from `ui-primitives` (wire session forms fold to their label), while an edit exposes the literal sent text. Busy Enter behavior is stored in the Host-backed `ui-conversation` settings namespace. +The resident composer survives no-Session and Session transitions. The no-Session state keeps the same composer surface mounted but inert while the Workspace picker connects a blank Session. The surface is a shell-owned Lexical editor: reference chips are atomic decorator nodes carrying the owner's serialization identity (submission expands them through the owner codec), claimed slash commands stay styled leading text, folder text references carry the folder glyph as an icon prefix, and the draft's clipboard projection is mirrored into the per-Session Conversation store. Queue operations address exact queue occurrences through the scoped `ctx.conversation` service; queue previews render sent text through the shared inline reference projection from `ui-primitives` (wire session forms fold to their label) and show local image previews or durable image parts as thumbnails, while an edit exposes the literal sent text. Durable thumbnails resolve through the session image URL cache. Busy Enter behavior is stored in the Host-backed `ui-conversation` settings namespace. -Default sends commit optimistically: Enter clears the draft, occurrence table, and undo history in the same transaction, keeps the composer in `plain`, and runs the send as a detached attempt, so typing and further sends continue during the flight. `sendSession` registers a Session submission echo (`session.beginSubmission`) before serializing, yields one paint so the echo renders on the click's own frame, and encodes images through the browser's native `FileReader` data-URL path. Concurrent failures are restored together in submission order until the user edits the restored content; command submissions keep the frozen `submitting` phase. Detached attempts retain their image ids through admission and Session scope disposal. When an echo retires as observed, the durable image cache exposes its preview immediately, fetches the admitted attachment, replaces the preview with the canonical URL, and revokes each URL after its use ends. Direct subagent continuations skip local echoes because their transport does not preserve the browser request id. +Default sends commit optimistically: Enter clears the draft, occurrence table, and undo history in the same transaction, keeps the composer in `plain`, and runs the send as a detached attempt, so typing and further sends continue during the flight. `sendSession` registers a Session submission echo (`session.beginSubmission`) with the delivery mode before serializing; Session derives the placement from that mode and its current running state, so idle sends use the transcript, busy Queue sends use QueueDock, and busy Steer sends use the pending-steering surface. It then yields one paint and encodes images through the browser's native `FileReader` data-URL path. Concurrent failures are restored together in submission order until the user edits the restored content; command submissions keep the frozen `submitting` phase. Detached attempts retain their image ids through admission and Session scope disposal. When an echo retires as observed, the durable image cache exposes its preview immediately, fetches the admitted attachment, replaces the preview with the canonical URL, and revokes each URL after its use ends. Direct subagent continuations skip local echoes because their transport does not preserve the browser request id. While a normal composer is running, its primary pointer action remains Stop when the draft is empty or input is unavailable. Actionable text or attachments switch the same seat to Queue Send; clearing or successfully submitting the draft restores Stop. The busy-Enter setting continues to select the Queue or Steer keyboard action. Continuable subagents keep separate Send and Stop actions ([decision](../../../.agents/notes/implemented/bug-fix/2026-08-20-running-draft-primary-send.md)). diff --git a/packages/client/ui-conversation/README.zh.md b/packages/client/ui-conversation/README.zh.md index 1d24141945..cb622f0308 100644 --- a/packages/client/ui-conversation/README.zh.md +++ b/packages/client/ui-conversation/README.zh.md @@ -38,9 +38,9 @@ target package 通过 declaration merge 扩展 snapshot 与 Location data map， View 选择规则固定：有效且已注册的持久化选择优先，其次是已注册的 `chat`，否则不渲染 View；绝不选择第一个已注册 View。Shell phase 只组合 Session lifecycle 与 active-target set，不读取任何 target-specific snapshot。 -常驻 composer 在无 Session 与有 Session 之间保持挂载。无 Session 时，同一个编辑器表面保持 inert，Workspace picker 连接 blank Session。该表面是 shell 所有的 Lexical 编辑器：引用 chip 是携带 owner 序列化身份的原子 decorator 节点（提交时经 owner codec 展开），已认领的 slash command 保持为带样式的行首文本，文件夹文本引用以图标前缀携带文件夹图形，草稿的剪贴板投影镜像到逐 Session Conversation store。Queue 操作通过 scoped `ctx.conversation` service 寻址准确的 queue occurrence；queue 预览经 `ui-primitives` 的共享行内引用投影渲染已发送文本（wire 会话形式折叠为其标签），编辑态则展示字面发送文本。繁忙时 Enter 行为保存在 Host-backed `ui-conversation` settings namespace。 +常驻 composer 在无 Session 与有 Session 之间保持挂载。无 Session 时，同一个编辑器表面保持 inert，Workspace picker 连接 blank Session。该表面是 shell 所有的 Lexical 编辑器：引用 chip 是携带 owner 序列化身份的原子 decorator 节点（提交时经 owner codec 展开），已认领的 slash command 保持为带样式的行首文本，文件夹文本引用以图标前缀携带文件夹图形，草稿的剪贴板投影镜像到逐 Session Conversation store。Queue 操作通过 scoped `ctx.conversation` service 寻址准确的 queue occurrence；queue 预览经 `ui-primitives` 的共享行内引用投影渲染已发送文本（wire 会话形式折叠为其标签），并把本地图片预览或持久化图片部分显示为缩略图，编辑态则展示字面发送文本。持久化缩略图通过会话图片 URL 缓存解析。繁忙时 Enter 行为保存在 Host-backed `ui-conversation` settings namespace。 -默认发送采用乐观提交：Enter 在同一事务里清空草稿、occurrence 表和撤销历史，composer 保持 `plain`，发送作为 detached attempt 运行，发送期间可以继续输入和提交。`sendSession` 在序列化之前注册 Session 提交回显（`session.beginSubmission`），让出一帧使回显在点击当帧渲染，图片经浏览器原生 `FileReader` data-URL 路径编码。多个并发发送失败时，在用户编辑还原内容之前按提交顺序合并还原；命令提交保持冻结的 `submitting` 阶段。Detached attempt 持有图片 id，直到 admission 完成或 Session scope 销毁。回显以 observed 退休时，durable 图片缓存立即公开预览 URL，同时读取 admitted 附件，随后用规范化 URL 替换预览，并在两个 URL 各自停止使用后撤销。直接 subagent continuation 不创建本地回显，因为其 transport 不保留浏览器 request id。 +默认发送采用乐观提交：Enter 在同一事务里清空草稿、occurrence 表和撤销历史，composer 保持 `plain`，发送作为 detached attempt 运行，发送期间可以继续输入和提交。`sendSession` 在序列化之前用投递模式注册 Session 提交回显（`session.beginSubmission`）；Session 根据该模式与当前运行状态推导位置，因此空闲发送进入 transcript，繁忙时 Queue 进入 QueueDock，繁忙时 Steer 进入 pending-steering 区域。随后让出一帧，图片经浏览器原生 `FileReader` data-URL 路径编码。多个并发发送失败时，在用户编辑还原内容之前按提交顺序合并还原；命令提交保持冻结的 `submitting` 阶段。Detached attempt 持有图片 id，直到 admission 完成或 Session scope 销毁。回显以 observed 退休时，durable 图片缓存立即公开预览 URL，同时读取 admitted 附件，随后用规范化 URL 替换预览，并在两个 URL 各自停止使用后撤销。直接 subagent continuation 不创建本地回显，因为其 transport 不保留浏览器 request id。 普通 composer 运行时，如果草稿为空或输入不可用，主指针操作保持为 Stop。可提交的文字或附件会把同一位置切换为 Queue Send；清空或成功提交草稿后恢复 Stop。繁忙态 Enter 设置继续选择 Queue 或 Steer 键盘操作。可继续 subagent 保留独立的 Send 与 Stop 操作（[决策](../../../.agents/notes/implemented/bug-fix/2026-08-20-running-draft-primary-send.zh.md)）。 diff --git a/packages/client/ui-conversation/src/client/image-labels.ts b/packages/client/ui-conversation/src/client/image-labels.ts index 03940533c5..a783d4f840 100644 --- a/packages/client/ui-conversation/src/client/image-labels.ts +++ b/packages/client/ui-conversation/src/client/image-labels.ts @@ -16,7 +16,7 @@ export function imageSizeText(bytes: number): string { /** * Product copy for a host attachment rejection (the `details.reason` of - * `session/attachment-invalid` or `subagent/attachment-unsupported`). + * `session/attachment-invalid` or `subagent/attachment-invalid`). * User-solvable reasons name the limit and the way out; * reasons the user cannot act on fold into one send-failed line carrying the * reason code for a bug report. @@ -32,7 +32,6 @@ export function attachmentErrorText( ): string { switch (reason) { case 'MODEL_DOES_NOT_SUPPORT_IMAGES': return t('image.modelUnsupported') - case 'SUBAGENT_IMAGE_UNSUPPORTED': return t('image.subagentUnsupported') case 'IMAGE_TOO_MANY_PIXELS': return t('image.tooManyPixels') case 'IMAGE_DIMENSION_TOO_LARGE': if (limits !== undefined) return t('image.dimensionTooLarge', { size: limits.maxImageDimension }) diff --git a/packages/client/ui-conversation/src/client/locales.ts b/packages/client/ui-conversation/src/client/locales.ts index 3b1f9c1fc8..0e8ba381f4 100644 --- a/packages/client/ui-conversation/src/client/locales.ts +++ b/packages/client/ui-conversation/src/client/locales.ts @@ -45,7 +45,6 @@ export const zh = { 'image.tooManyPixels': '图片分辨率过大，请压缩后重试', 'image.dimensionTooLarge': '图片宽高不能超过 {size}px，请缩小后重试', 'image.modelUnsupported': '当前模型不支持图片，请切换支持图片的模型', - 'image.subagentUnsupported': '子智能体会话暂不支持图片', 'image.sendFailed': '图片发送失败（{reason}），请重新添加图片后再试', 'context.aria': '上下文已用 {percent}', 'context.used': '上下文已用', @@ -131,6 +130,7 @@ export const zh = { 'web.contentTruncated': '内容已截断', 'details.running': '运行中…', 'queue.count': '{n} 条排队消息', + 'queue.image': '排队消息图片', 'queue.edit': '编辑排队消息', 'queue.edit.unsupported': '包含非文本内容，暂不支持编辑', 'queue.save': '保存排队消息', @@ -195,7 +195,6 @@ export const en = { 'image.tooManyPixels': 'Image resolution is too high; compress it and try again', 'image.dimensionTooLarge': 'Image sides must be at most {size}px; downscale it and try again', 'image.modelUnsupported': 'The current model does not support images; switch to a model that does', - 'image.subagentUnsupported': 'Subagent sessions do not support images yet', 'image.sendFailed': 'Sending images failed ({reason}); re-add them and try again', 'context.aria': '{percent} of context used', 'context.used': 'of context used', @@ -281,6 +280,7 @@ export const en = { 'web.contentTruncated': 'Content truncated', 'details.running': 'Running…', 'queue.count': '{n} queued messages', + 'queue.image': 'Queued message image', 'queue.edit': 'Edit queued message', 'queue.edit.unsupported': 'Contains non-text content; editing is not supported yet', 'queue.save': 'Save queued message', diff --git a/packages/client/ui-conversation/src/client/queue/QueueDock.module.css b/packages/client/ui-conversation/src/client/queue/QueueDock.module.css index eca51941ca..81ea205b5e 100644 --- a/packages/client/ui-conversation/src/client/queue/QueueDock.module.css +++ b/packages/client/ui-conversation/src/client/queue/QueueDock.module.css @@ -122,6 +122,21 @@ box-shadow: inset 0 1px 0 var(--dsw-alias-border-l1); } +.thumbs { + display: flex; + flex: none; + gap: 4px; +} + +.thumb { + width: 24px; + height: 24px; + border: 1px solid var(--dsw-alias-border-l1); + border-radius: 4px; + background: var(--dsw-alias-bg-base); + object-fit: cover; +} + .preview, .editor { flex: 1 1 auto; diff --git a/packages/client/ui-conversation/src/client/queue/QueueDock.tsx b/packages/client/ui-conversation/src/client/queue/QueueDock.tsx index aadedd7d29..2840268034 100644 --- a/packages/client/ui-conversation/src/client/queue/QueueDock.tsx +++ b/packages/client/ui-conversation/src/client/queue/QueueDock.tsx @@ -1,12 +1,13 @@ import type { Context } from '@deepseek-ai/cordis' import { useEffect, useId, useMemo, useState } from 'react' +import type { ImageAttachmentRef } from '@deepseek-ai/dsh-attachment' import type { PropsLocale, PropsRuntime } from '@deepseek-ai/dsh-client-ui-slots' import type { SessionId } from '@deepseek-ai/dsh-session/types' import { IconCheckOutline16, IconChevronDownOutline14, IconChevronUpOutline14, IconCloseOutline16, IconEditOutline16, IconQueueOutline14, IconSendOutline14, IconTrashOutline16, projectUserText, Tooltip, } from '@deepseek-ai/dsh-client-ui-primitives' -import type { QueueAction, QueueItemId } from '../contract/queue.ts' +import type { QueueAction, QueueItemId, QueueRow } from '../contract/queue.ts' import { NS } from '../locales.ts' import css from './QueueDock.module.css' @@ -14,6 +15,43 @@ import css from './QueueDock.module.css' export interface QueueDockInjected { updateQueue: (itemId: QueueItemId, action: QueueAction) => Promise notify: (level: 'info' | 'error', text: string) => void + /** Resolve one durable queued image into a session-scoped browser URL. */ + loadImage: (attachment: ImageAttachmentRef) => Promise +} + +/** + * Durable references carried by one queued row. Queue frames are wire data + * despite their typed face, so an image block without a reference is skipped + * rather than trusted. + * @param content - the row's wire content blocks. + * @returns the row's durable image references in block order. + */ +function queueImageRefs(content: QueueRow['content']): ImageAttachmentRef[] { + return content.flatMap((block) => { + if (block.type !== 'image') return [] + const { attachment } = block as { attachment?: ImageAttachmentRef } + return attachment === undefined ? [] : [attachment] + }) +} + +/** One durable queued image as a fixed-size thumbnail; a load failure keeps the empty placeholder. */ +function QueueThumb({ attachment, loadImage, label }: { + attachment: ImageAttachmentRef + loadImage: QueueDockInjected['loadImage'] + label: string +}) { + const [url, setUrl] = useState(null) + useEffect(() => { + let alive = true + loadImage(attachment).then( + (resolved) => { if (alive) setUrl(resolved) }, + () => { /* placeholder retained; the durable transcript surfaces read errors */ }, + ) + return () => { alive = false } + }, [attachment, loadImage]) + return url === null + ? + : {label} } /** Full props of a dock entry: InputZone owner share + session standard kit + global seat + the locale seat. */ @@ -23,9 +61,17 @@ export type QueueDockProps = PropsRuntime<'conversation.input.dock'> & QueueDock * Queue strip: one item renders directly; multiple items default to a * collapsible count header; an empty queue renders nothing. */ -export function QueueDock({ useSession, updateQueue, notify, t }: QueueDockProps) { +export function QueueDock({ useSession, updateQueue, notify, loadImage, t }: QueueDockProps) { const inbox = useSession(s => s.queue) const queue = useMemo(() => inbox.filter(row => row.placement === 'queued'), [inbox]) + const pendingSubmissions = useSession(s => s.pendingSubmissions) + const pendingQueue = useMemo(() => { + const admitted = new Set(queue.flatMap(row => row.rpcId === undefined ? [] : [row.rpcId])) + return pendingSubmissions.filter(submission => ( + submission.placement === 'queued' && !admitted.has(submission.requestId) + )) + }, [pendingSubmissions, queue]) + const rowCount = queue.length + pendingQueue.length const running = useSession(s => s.running) const queueMutable = useSession(s => s.subagent === null) const [editing, setEditing] = useState<{ id: QueueItemId; text: string } | null>(null) @@ -34,15 +80,15 @@ export function QueueDock({ useSession, updateQueue, notify, t }: QueueDockProps const listId = useId() useEffect(() => { - if (queue.length === 0 && !collapsed) setCollapsed(true) + if (rowCount === 0 && !collapsed) setCollapsed(true) if (editing !== null && (!queueMutable || !queue.some(row => row.id === editing.id))) setEditing(null) - }, [collapsed, editing, queue, queueMutable]) + }, [collapsed, editing, queue, queueMutable, rowCount]) - if (queue.length === 0) return null + if (rowCount === 0) return null const interactionActive = queueMutable && (editing !== null || busy !== null) const expanded = !collapsed || interactionActive - const listVisible = queue.length === 1 || expanded + const listVisible = rowCount === 1 || expanded const applyAction = async ( itemId: QueueItemId, @@ -73,7 +119,7 @@ export function QueueDock({ useSession, updateQueue, notify, t }: QueueDockProps return (
- {queue.length > 1 && ( + {rowCount > 1 && ( )}
} + {queueMutable &&
+ {editing?.id === row.id + ? ( + <> + + + + + + + + ) + : ( + <> + + + + + + + + + + + )} +
} + + ) + })} + {listVisible && pendingQueue.map(submission => ( +
  • + {rowCount === 1 && } + {submission.images.length > 0 && ( + + {submission.images.map((image, index) => ( + {t('queue.image')} + ))} + + )} + {projectUserText(submission.text, [])}
    • ))} @@ -210,7 +293,7 @@ export function QueueDock({ useSession, updateQueue, notify, t }: QueueDockProps /** Registers queue actions backed by the session-scoped conversation service. */ export const queueDockEntry = { name: 'conversation-queue-dock', - inject: ['slots', 'conversation', 'sessions'], + inject: ['slots', 'conversation', 'sessions', 'uiConversation'], apply(ctx: Context): void { ctx.slots.inject('conversation.input.dock', () => ctx.slots.register({ name: 'conversation.input.dock', @@ -225,6 +308,7 @@ export const queueDockEntry = { return { updateQueue: (itemId, action) => conversation.updateQueue(itemId, action), notify: (level, text) => { conversation.input.for(actx).notify(level, text) }, + loadImage: attachment => ctx.uiConversation.imageUrl(sessionId, attachment), } }, }, QueueDock)) diff --git a/packages/client/ui-conversation/src/client/service.ts b/packages/client/ui-conversation/src/client/service.ts index fd59a492f0..27a4c3e629 100644 --- a/packages/client/ui-conversation/src/client/service.ts +++ b/packages/client/ui-conversation/src/client/service.ts @@ -207,7 +207,8 @@ export class ConversationController extends Service implements IConversation { if (attachments.length !== imageIds.length) { throw new Error('conversation.sendSession: one or more draft images are no longer available') } - if (session.getSnapshot().subagent !== null) { + const snapshot = session.getSnapshot() + if (snapshot.subagent !== null) { const uploaded = await this.serializeImages(attachments.map(attachment => attachment.file)) const content = [...uploaded, ...(text === '' ? [] : [{ type: 'text' as const, text }])] const result = await session.prompt(content, mode, signal) @@ -218,6 +219,7 @@ export class ConversationController extends Service implements IConversation { ? undefined : new Promise((resolve) => { finishRetirement = resolve }) const submission = session.beginSubmission({ + mode, text, images: attachments.map(attachment => ({ previewUrl: attachment.previewUrl, diff --git a/packages/client/ui-conversation/src/client/skeleton/InputBar.tsx b/packages/client/ui-conversation/src/client/skeleton/InputBar.tsx index d4c06d8bf3..6c8c46cad4 100644 --- a/packages/client/ui-conversation/src/client/skeleton/InputBar.tsx +++ b/packages/client/ui-conversation/src/client/skeleton/InputBar.tsx @@ -94,7 +94,7 @@ export function InputBar({ useEffect(() => { if (promptError === null) return const { error } = promptError - showToast(error.code === 'session/attachment-invalid' || error.code === 'subagent/attachment-unsupported' + showToast(error.code === 'session/attachment-invalid' || error.code === 'subagent/attachment-invalid' ? attachmentErrorText(t, error.details.reason, imageLimits) : `${error.message} (${error.code})`) }, [promptError, showToast, t, imageLimits]) diff --git a/packages/client/ui-conversation/tests/image-labels.client.spec.ts b/packages/client/ui-conversation/tests/image-labels.client.spec.ts index 8e1ff4409d..0244c134db 100644 --- a/packages/client/ui-conversation/tests/image-labels.client.spec.ts +++ b/packages/client/ui-conversation/tests/image-labels.client.spec.ts @@ -24,7 +24,6 @@ describe('attachment rejection copy', () => { it('maps user-solvable reasons to limit-naming copy', () => { expect(attachmentErrorText(t, 'MODEL_DOES_NOT_SUPPORT_IMAGES')).toBe('当前模型不支持图片，请切换支持图片的模型') - expect(attachmentErrorText(t, 'SUBAGENT_IMAGE_UNSUPPORTED')).toBe('子智能体会话暂不支持图片') expect(attachmentErrorText(t, 'IMAGE_TOO_MANY_PIXELS')).toBe('图片分辨率过大，请压缩后重试') expect(attachmentErrorText(t, 'INVALID_IMAGE')).toBe('仅支持 PNG、JPG、WebP、GIF 格式的图片') expect(attachmentErrorText(t, 'IMAGE_TYPE_MISMATCH')).toBe('仅支持 PNG、JPG、WebP、GIF 格式的图片') diff --git a/packages/client/ui-conversation/tests/input-bar.client.spec.tsx b/packages/client/ui-conversation/tests/input-bar.client.spec.tsx index fafa85283b..d616c34a43 100644 --- a/packages/client/ui-conversation/tests/input-bar.client.spec.tsx +++ b/packages/client/ui-conversation/tests/input-bar.client.spec.tsx @@ -350,17 +350,16 @@ describe('image draft rail', () => { const unknown = bench({ promptError: attachmentError('ATTACHMENT_NOT_REFERENCED') }) expect(unknown.view.getByRole('alert').textContent).toContain('图片发送失败（ATTACHMENT_NOT_REFERENCED）') cleanup() - // A subagent turn refuses images under its own code; the reason keys the - // same product copy, because the user cannot act on which domain refused. + // A subagent refusal uses the same product copy for the same reason. const subagent = bench({ promptError: { op: 'send', - error: new RemoteError('subagent/attachment-unsupported', 'raw wire text', { - childSessionId: SID, reason: 'SUBAGENT_IMAGE_UNSUPPORTED', + error: new RemoteError('subagent/attachment-invalid', 'raw wire text', { + reason: 'MODEL_DOES_NOT_SUPPORT_IMAGES', }), }, }) - expect(subagent.view.getByRole('alert').textContent).toContain('子智能体会话暂不支持图片') + expect(subagent.view.getByRole('alert').textContent).toContain('当前模型不支持图片，请切换支持图片的模型') cleanup() const other = bench({ promptError: { op: 'send', error: new RemoteError('gateway/internal', 'boom', {}) }, diff --git a/packages/client/ui-conversation/tests/queue-dock.client.spec.tsx b/packages/client/ui-conversation/tests/queue-dock.client.spec.tsx index 3cd61a0b5b..492bc16151 100644 --- a/packages/client/ui-conversation/tests/queue-dock.client.spec.tsx +++ b/packages/client/ui-conversation/tests/queue-dock.client.spec.tsx @@ -88,10 +88,26 @@ function kitFor(snapshot: SessionSnapshot, injected: Partial input: INPUT_STATE, updateQueue: vi.fn(() => Promise.resolve()), notify: vi.fn(), + loadImage: vi.fn(() => Promise.resolve('blob:unused')), ...injected, } } +/** One queued row carrying a durable image reference (plus optional leading text). */ +function imageRow(id: string, refId: string, text = ''): QueuedMessage { + return { + id: iid(id), messageId: `message-${id}` as never, placement: 'queued', + content: [ + ...text === '' ? [] : [{ type: 'text' as const, text }], + { + type: 'image', + attachment: { attachmentId: refId, mediaType: 'image/png', bytes: 1, width: 1, height: 1 }, + } as never, + ], + preview: text, text: null, + } +} + describe('QueueDock', () => { it('renders null while the queue is empty', () => { const snap = snapshotWith([]) @@ -100,6 +116,32 @@ describe('QueueDock', () => { expect(container.innerHTML).toBe('') }) + it('renders a queued local echo in the dock and hands off by rpcId', () => { + const pending = { + ...snapshotWith([]), + pendingSubmissions: [{ + requestId: 'req-local-queue' as never, + placement: 'queued' as const, + time: 1, + text: '等待上传', + images: [{ previewUrl: 'blob:queue-preview', name: 'queue.png' }], + }], + } + const source = liveSession(pending) + const view = render() + expect(view.getByText('等待上传').closest('[data-submission-echo]')).not.toBeNull() + expect(view.getByRole('img', { name: '排队消息图片' }).getAttribute('src')).toBe('blob:queue-preview') + + act(() => { + source.push({ + ...pending, + queue: [{ ...row('accepted', '等待上传'), rpcId: 'req-local-queue' as never }], + }) + }) + expect(view.getAllByText('等待上传')).toHaveLength(1) + expect(view.container.querySelector('[data-submission-echo]')).toBeNull() + }) + it('leaves pending steering to the conversation flow', () => { const steering = { ...row('s-1', 'interrupt'), placement: 'steering' as const } const snap = snapshotWith([steering]) @@ -224,6 +266,52 @@ describe('QueueDock', () => { .toBe('包含非文本内容，暂不支持编辑') }) + it('renders queued image thumbnails from durable references beside the text preview', async () => { + const loadImage = vi.fn(() => Promise.resolve('blob:thumb-1')) + const snap = snapshotWith([imageRow('i-img', 'att-9', '带图消息')]) + const source = liveSession(snap) + const { container } = render( + , + ) + + await waitFor(() => { + expect(container.querySelector('img')?.getAttribute('src')).toBe('blob:thumb-1') + }) + expect(loadImage).toHaveBeenCalledWith(expect.objectContaining({ attachmentId: 'att-9' })) + expect(container.querySelector('img')?.getAttribute('alt')).toBe('排队消息图片') + expect(container.querySelector('li')?.textContent).toBe('带图消息') + }) + + it('keeps the empty thumbnail placeholder when the image read fails', async () => { + const loadImage = vi.fn(() => Promise.reject(new Error('read denied'))) + const snap = snapshotWith([imageRow('i-broken', 'att-x')]) + const source = liveSession(snap) + const { container } = render( + , + ) + + await act(async () => { await Promise.resolve() }) + expect(loadImage).toHaveBeenCalled() + expect(container.querySelector('img')).toBeNull() + }) + + it('ignores a thumbnail resolution landing after unmount', async () => { + let resolveUrl: ((url: string) => void) | undefined + const loadImage = vi.fn(() => new Promise((resolve) => { resolveUrl = resolve })) + const snap = snapshotWith([imageRow('i-late', 'att-late')]) + const source = liveSession(snap) + const { unmount } = render( + , + ) + + unmount() + await act(async () => { + resolveUrl?.('blob:late') + await Promise.resolve() + }) + expect(loadImage).toHaveBeenCalledTimes(1) + }) + it('edits text inline with save and cancel controls, then saves with the same item identity', async () => { const snap = snapshotWith([row('i-edit', 'before')]) const source = liveSession(snap) @@ -389,7 +477,7 @@ describe('QueueDock', () => { it('registers as the terminal composer-context entry', () => { expect(queueDockEntry.name).toBe('conversation-queue-dock') - expect(queueDockEntry.inject).toEqual(['slots', 'conversation', 'sessions']) + expect(queueDockEntry.inject).toEqual(['slots', 'conversation', 'sessions', 'uiConversation']) const register = vi.fn(() => () => undefined) const inject = vi.fn((_name: string, callback: () => () => void) => callback()) queueDockEntry.apply({ slots: { inject, register } } as never) diff --git a/packages/client/ui-conversation/tests/service-orchestration.client.spec.ts b/packages/client/ui-conversation/tests/service-orchestration.client.spec.ts index 9fbcaf92aa..7ff7924133 100644 --- a/packages/client/ui-conversation/tests/service-orchestration.client.spec.ts +++ b/packages/client/ui-conversation/tests/service-orchestration.client.spec.ts @@ -185,6 +185,7 @@ describe('sendSession submission echo', () => { const sending = b.root.sendSession(session, '带图', [attachment!.id], 'queue') // Synchronous: the echo is registered before any encoding starts. expect(b.beginSubmission).toHaveBeenCalledWith(expect.objectContaining({ + mode: 'queue', text: '带图', images: [expect.objectContaining({ previewUrl: 'blob:echo-1', name: 'a.png' })], })) @@ -211,6 +212,29 @@ describe('sendSession submission echo', () => { await b.runtime.dispose() }) + it('passes each delivery mode before image serialization', async () => { + const b = await echoBench() + try { + await b.runtime.sessions.updateSessionSnapshot('s1', (draft) => { draft.running = true }) + const session = b.runtime.sessions.binding('s1')!.session + await expect(b.root.sendSession(session, '立即纠偏', [], 'steer')) + .resolves.toEqual({ kind: 'success' }) + expect(b.beginSubmission).toHaveBeenLastCalledWith(expect.objectContaining({ + mode: 'steer', + text: '立即纠偏', + })) + await expect(b.root.sendSession(session, '稍后处理', [], 'queue')) + .resolves.toEqual({ kind: 'success' }) + expect(b.beginSubmission).toHaveBeenLastCalledWith(expect.objectContaining({ + mode: 'queue', + text: '稍后处理', + })) + } finally { + b.restore() + } + await b.runtime.dispose() + }) + it('hands the preview URL to the image cache on observed retirement instead of revoking it', async () => { const b = await echoBench() try { diff --git a/packages/context/agent-instructions/tests/agent-instructions.spec.ts b/packages/context/agent-instructions/tests/agent-instructions.spec.ts index 5c9c3e7f2f..1533709640 100644 --- a/packages/context/agent-instructions/tests/agent-instructions.spec.ts +++ b/packages/context/agent-instructions/tests/agent-instructions.spec.ts @@ -48,6 +48,7 @@ import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent const sk = (directory: string, candidateName: string): string => candidateScopeKey(directory, candidateName) const testToolSignal = new AbortController().signal +const requestTimeoutMs = process.platform === 'win32' ? 5_000 : 1_000 async function tempRepo(): Promise { return mkdtemp(join(tmpdir(), 'dsh-workspace-context-')) @@ -1376,7 +1377,7 @@ describe('workspace context request injection', () => { const original = stubAgent(root) await agentEvents(ctx, original).waterfall( 'agent/pre-step', - { messages: [], turn: 1, step: 1, signal: AbortSignal.timeout(1000) }, + { messages: [], turn: 1, step: 1, signal: AbortSignal.timeout(requestTimeoutMs) }, () => Promise.resolve({ kind: 'enter' as const, messages: [] }), ) const inserted = original.inbox.nextStep[0] @@ -1389,7 +1390,7 @@ describe('workspace context request injection', () => { const claimed = resumed.inbox.claim('next-step', 1) const decision = await agentEvents(ctx, resumed).waterfall( 'agent/pre-step', - { messages: claimed, turn: 1, step: 1, signal: AbortSignal.timeout(1000) }, + { messages: claimed, turn: 1, step: 1, signal: AbortSignal.timeout(requestTimeoutMs) }, () => Promise.resolve({ kind: 'enter' as const, messages: claimed }), ) if (decision.kind !== 'enter') throw new Error('recovered baseline was rejected') @@ -1421,7 +1422,7 @@ describe('workspace context request injection', () => { const original = stubAgent(root) await agentEvents(ctx, original).waterfall( 'agent/pre-step', - { messages: [], turn: 1, step: 1, signal: AbortSignal.timeout(1000) }, + { messages: [], turn: 1, step: 1, signal: AbortSignal.timeout(requestTimeoutMs) }, () => Promise.resolve({ kind: 'enter' as const, messages: [] }), ) const stale = original.inbox.nextStep[0] @@ -1435,7 +1436,7 @@ describe('workspace context request injection', () => { const staleClaim = resumed.inbox.claim('next-step', 1) const staleDecision = await agentEvents(ctx, resumed).waterfall( 'agent/pre-step', - { messages: staleClaim, turn: 1, step: 1, signal: AbortSignal.timeout(1000) }, + { messages: staleClaim, turn: 1, step: 1, signal: AbortSignal.timeout(requestTimeoutMs) }, () => Promise.resolve({ kind: 'enter' as const, messages: staleClaim }), ) @@ -1474,7 +1475,7 @@ describe('workspace context request injection', () => { const original = stubAgent(root) await agentEvents(originalCtx, original).waterfall( 'agent/pre-step', - { messages: [], turn: 1, step: 1, signal: AbortSignal.timeout(1000) }, + { messages: [], turn: 1, step: 1, signal: AbortSignal.timeout(requestTimeoutMs) }, () => Promise.resolve({ kind: 'enter' as const, messages: [] }), ) const stale = original.inbox.nextStep[0] @@ -1488,7 +1489,7 @@ describe('workspace context request injection', () => { const claimed = resumed.inbox.claim('next-step', 1) const decision = await agentEvents(resumedCtx, resumed).waterfall( 'agent/pre-step', - { messages: claimed, turn: 1, step: 1, signal: AbortSignal.timeout(1000) }, + { messages: claimed, turn: 1, step: 1, signal: AbortSignal.timeout(requestTimeoutMs) }, () => Promise.resolve({ kind: 'enter' as const, messages: claimed }), ) @@ -1592,7 +1593,7 @@ describe('workspace context request injection', () => { const decision = await agentEvents(ctx, agent).waterfall( 'agent/pre-step', - { messages: [prompt], turn: 1, step: 1, signal: AbortSignal.timeout(1000) }, + { messages: [prompt], turn: 1, step: 1, signal: AbortSignal.timeout(requestTimeoutMs) }, () => Promise.resolve(downstream), ) @@ -1649,7 +1650,7 @@ describe('workspace context request injection', () => { const decision = await agentEvents(ctx, agent).waterfall( 'agent/pre-step', - { messages: [], turn: 1, step: 1, signal: AbortSignal.timeout(1000) }, + { messages: [], turn: 1, step: 1, signal: AbortSignal.timeout(requestTimeoutMs) }, () => Promise.resolve(downstream), ) @@ -1760,7 +1761,7 @@ describe('workspace context request injection', () => { const decision = await agentEvents(ctx, agent).waterfall( 'agent/pre-step', - { messages: [prompt], turn: 2, step: 1, signal: AbortSignal.timeout(1000) }, + { messages: [prompt], turn: 2, step: 1, signal: AbortSignal.timeout(requestTimeoutMs) }, () => Promise.resolve({ kind: 'enter' as const, messages: [prompt] }), ) diff --git a/packages/context/file-reference-local/src/search.ts b/packages/context/file-reference-local/src/search.ts index ba5d30c2f3..ad4571cc97 100644 --- a/packages/context/file-reference-local/src/search.ts +++ b/packages/context/file-reference-local/src/search.ts @@ -295,10 +295,12 @@ async function readDirectory(absolute: string, signal: AbortSignal) { signal.throwIfAborted() return entries.sort((left, right) => compareText(left.name, right.name)) } catch (_error: unknown) { + /* v8 ignore start -- Windows chmod cannot make the unreadable-directory fixture fail readdir; POSIX behavior covers this fallback. */ signal.throwIfAborted() // An unreadable/missing subtree contributes no candidates; other readable // branches remain useful and autocomplete is advisory. return [] + /* v8 ignore stop */ } } diff --git a/packages/extensions/cordis-client-runner/src/client/api-catalog.ts b/packages/extensions/cordis-client-runner/src/client/api-catalog.ts index bef6356509..e10e956a12 100644 --- a/packages/extensions/cordis-client-runner/src/client/api-catalog.ts +++ b/packages/extensions/cordis-client-runner/src/client/api-catalog.ts @@ -439,7 +439,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'BeginSubmissionInput', - declaration: 'export interface BeginSubmissionInput {\n readonly text: string;\n readonly images: readonly PendingSubmissionImage[];\n readonly onRetire?: (retirement: PendingSubmissionRetirement) => void;\n}', + declaration: 'export interface BeginSubmissionInput {\n readonly mode: \'queue\' | \'steer\';\n readonly text: string;\n readonly images: readonly PendingSubmissionImage[];\n readonly onRetire?: (retirement: PendingSubmissionRetirement) => void;\n}', }, { name: 'BoundActions', @@ -615,12 +615,16 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'PendingSubmission', - declaration: 'export interface PendingSubmission {\n readonly requestId: SessionRequestId;\n readonly time: number;\n readonly text: string;\n readonly images: readonly PendingSubmissionImage[];\n}', + declaration: 'export interface PendingSubmission {\n readonly requestId: SessionRequestId;\n readonly placement: PendingSubmissionPlacement;\n readonly time: number;\n readonly text: string;\n readonly images: readonly PendingSubmissionImage[];\n}', }, { name: 'PendingSubmissionImage', declaration: 'export interface PendingSubmissionImage {\n readonly previewUrl: string;\n readonly name?: string;\n readonly width?: number;\n readonly height?: number;\n}', }, + { + name: 'PendingSubmissionPlacement', + declaration: 'export type PendingSubmissionPlacement = \'transcript\' | \'queued\' | \'steering\';', + }, { name: 'PendingSubmissionRetirement', declaration: 'export type PendingSubmissionRetirement = {\n readonly reason: \'observed\';\n readonly attachments: readonly ImageAttachmentRef[];\n} | {\n readonly reason: \'failed\';\n};', diff --git a/packages/extensions/cordis-client-runner/tests/api-catalog.client.spec.ts b/packages/extensions/cordis-client-runner/tests/api-catalog.client.spec.ts index 3dbfa80b05..9e2e106b9e 100644 --- a/packages/extensions/cordis-client-runner/tests/api-catalog.client.spec.ts +++ b/packages/extensions/cordis-client-runner/tests/api-catalog.client.spec.ts @@ -29,9 +29,11 @@ describe('Client Cordis inspect catalog', () => { it('includes the current referenced type closure for the Sessions service', () => { const result = queryServiceApi('sessions') as { - referencedTypes: readonly { name: string }[] + referencedTypes: readonly { name: string; declaration: string }[] } expect(result.referencedTypes.length).toBeGreaterThan(0) + const promptContentPart = result.referencedTypes.find(type => type.name === 'PromptContentPart') + expect(promptContentPart?.declaration).toContain("readonly type: 'image'") expect(result.referencedTypes.map(type => type.name)).not.toEqual(expect.arrayContaining([ 'ConversationSnapshot', 'PendingInteraction', diff --git a/packages/extensions/tool-cordis/src/api-catalog.ts b/packages/extensions/tool-cordis/src/api-catalog.ts index 2d2ee97188..ef3419db8f 100644 --- a/packages/extensions/tool-cordis/src/api-catalog.ts +++ b/packages/extensions/tool-cordis/src/api-catalog.ts @@ -2257,10 +2257,10 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ }, { signature: '@Remote(\'prompt\') async prompt(request: SubagentPromptRequest, signal: AbortSignal): Promise', - description: 'Deliver one browser-authored message to a continuable child through the exact live direct parent, retaining the caller-minted request identity and validated browser zone on the accepted message. Success identifies the message the child\'s FIFO inbox accepted; later execution is independent of this call.', + description: 'Deliver one browser-authored message to a continuable child through the exact live direct parent, retaining the caller-minted request identity and validated browser zone on the accepted message. Success identifies the message the child\'s FIFO inbox accepted; later execution is independent of this call. Image parts are admitted and persisted through the attachment store before delivery, and the child\'s model must accept image input.', parameters: [{ name: 'request', description: 'durable address, minted identity, content, and optional browser zone.' }, { name: 'signal', description: 'carrier cancellation, owning the call until inbox acceptance.' }], returns: 'the accepted message\'s inbox identity.', - throws: ['{RemoteError} `gateway/bad-request`, `subagent/attachment-unsupported`, `subagent/invalid-time-zone`, `subagent/parent-unavailable`, `subagent/not-resumable`, `subagent/unauthorized`, `subagent/delivery-unavailable`, `gateway/cancelled`, or `gateway/internal`.'], + throws: ['{RemoteError} `gateway/bad-request`, `subagent/attachment-invalid`, `subagent/invalid-time-zone`, `subagent/parent-unavailable`, `subagent/not-resumable`, `subagent/unauthorized`, `subagent/delivery-unavailable`, `gateway/cancelled`, or `gateway/internal`.'], }, { signature: '@Remote(\'interruptByParent\') interruptByParent( childSessionId: SessionId, parentSessionId: SessionId, mode: \'continuable\', ): SubagentInterruptReceipt', @@ -4003,10 +4003,6 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'EncodedImageAttachment', declaration: 'export interface EncodedImageAttachment {\n mediaType: ImageMediaType;\n data: string;\n name?: string;\n}', }, - { - name: 'EncodedImagePromptBlock', - declaration: 'export interface EncodedImagePromptBlock extends EncodedImageAttachment {\n readonly type: \'image\';\n}', - }, { name: 'EpochHeader', declaration: 'export interface EpochHeader {\n config: LlmCallConfig;\n adapterDefaults?: LlmCallConfigAdapterDefaults;\n system?: string;\n tools?: ToolSchema[];\n}', @@ -4603,10 +4599,6 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'PromptAssembly', declaration: 'export interface PromptAssembly {\n sections: AssembledSection[];\n contexts: AssembledContext[];\n tools: ToolSchema[];\n variables: Record;\n}', }, - { - name: 'PromptContentPart', - declaration: 'export type PromptContentPart = {\n readonly type: \'text\';\n readonly text: string;\n} | {\n readonly type: \'image\';\n readonly mediaType: ImageMediaType;\n readonly data: string;\n readonly name?: string;\n};', - }, { name: 'PromptContext', declaration: 'export interface PromptContext {\n readonly name: string;\n readonly order: number;\n readonly text: string | ((context: AssembleContext) => string);\n}', @@ -5439,17 +5431,13 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'SubagentListEntry', declaration: 'export type SubagentListEntry = {\n readonly kind: \'child\';\n readonly id: SessionId;\n readonly activity: \'running\' | \'inactive\';\n readonly hasChildren: boolean;\n} & ({\n readonly mode: \'one-shot\';\n readonly label?: string;\n} | {\n readonly mode: \'continuable\';\n readonly label: string;\n}) | {\n readonly kind: \'diagnostic\';\n readonly id: SessionId;\n readonly reason: \'corrupt\' | \'unsupported\' | \'unavailable\';\n};', }, - { - name: 'SubagentPromptContentPart', - declaration: 'export type SubagentPromptContentPart = ContentBlock | EncodedImagePromptBlock;', - }, { name: 'SubagentPromptReceipt', declaration: 'export interface SubagentPromptReceipt {\n readonly messageId: MessageId;\n}', }, { name: 'SubagentPromptRequest', - declaration: 'export interface SubagentPromptRequest {\n readonly requestId: SubagentPromptRequestId;\n readonly parentSessionId: SessionId;\n readonly childSessionId: SessionId;\n readonly mode: \'continuable\';\n readonly content: readonly SubagentPromptContentPart[];\n readonly clientTimeZone?: string;\n}', + declaration: 'export interface SubagentPromptRequest {\n readonly requestId: SubagentPromptRequestId;\n readonly parentSessionId: SessionId;\n readonly childSessionId: SessionId;\n readonly mode: \'continuable\';\n readonly content: readonly PromptContentPart[];\n readonly clientTimeZone?: string;\n}', }, { name: 'SubagentPromptRequestId', diff --git a/packages/fs/tool-fs/README.i18n.yaml b/packages/fs/tool-fs/README.i18n.yaml index cd2106cbbc..8705144163 100644 --- a/packages/fs/tool-fs/README.i18n.yaml +++ b/packages/fs/tool-fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/tool-fs/README.md -README.md: 193f842aa76b7d3f4df717d1d23960ef4d3ee329 -README.zh.md: 3cbd6bec6d9a4b200d90c15b32291cacc2e41aa8 +README.md: f00c96d18619e0fb59609eb66729ebfce445dbc6 +README.zh.md: 5915e5a07de10775469d0ebe301564948b357569 diff --git a/packages/fs/tool-fs/README.md b/packages/fs/tool-fs/README.md index 193f842aa7..f00c96d186 100644 --- a/packages/fs/tool-fs/README.md +++ b/packages/fs/tool-fs/README.md @@ -44,7 +44,7 @@ The policy plugin is optional: without it the tools run against the bare provide | Tool | Arguments | Behavior | |---|---|---| | `read` | `file_path`, `offset?`, `limit?` | Line-numbered UTF-8 content with a pagination footer; `offset` is 1-based and `limit` defaults to and caps at the configured `readLimit` | -| `read_image` | `file_path` | Reads and persists a PNG/JPEG/WebP/GIF source; normalization can downscale it before the next model request, so the model need not create a thumbnail first | +| `read_image` | `file_path` | Reads and persists a PNG/JPEG/WebP/GIF source; an extension-less path (normalized attachment object paths included) is identified from its file signature; normalization can downscale it before the next model request, so the model need not create a thumbnail first | | `write` | `file_path`, `content` | Creates or fully replaces a file; with the policy plugin, overwriting requires a prior `read` at the unchanged version, creating does not | | `edit` | `file_path`, `old_string`, `new_string`, `replace_all?` | Literal replacement requiring a unique match unless `replace_all` is true; with the policy plugin, requires a prior `read` and an unchanged file | @@ -65,6 +65,8 @@ The generated [configuration catalog](../../../docs/config-catalog.md#deepseek-a ### Policy and sandbox behavior +Path authorization for `read` and `read_image` belongs entirely to `ctx.fs`; media-type declarations and file signatures only decide whether `read_image` accepts the bytes returned by that backend. + With the policy plugin mounted, `write` and `edit` obtain their guard from the `fs/*` intent slots, so an unread target or a stale observation fails with `FS_NOT_OBSERVED` or `FS_STALE_VERSION` and a recovery instruction. Under a confining backend (`fs-sandbox`), `write`/`edit` additionally advertise `sandbox_permissions` and `justification`; a denied mutation returns the `[sandbox: file access denied under mode]` marker with the same-turn escalation hint, and an approved retry may stamp a strictly wider mode for that one call. ### Failures and recovery @@ -219,7 +221,7 @@ Append-only; newly visible content follows the reusable request prefix and does #### What the model sees -Failures are normalized as `Error: `. This package's stable validation and read messages are `file_path must be a non-empty string`, `limit must be less than or equal to `, `old_string must be a non-empty string`, `old_string and new_string must differ`, `cannot read "": not found`, `cannot read "": not a regular file`, `offset is out of range for "" ( lines)`, `cannot read "": read_image only accepts PNG/JPEG/WebP/GIF paths`, `cannot read "" as an image: model "" does not declare image input; switch to an image-capable model to read images`, and the mismatch repair `cannot read "": the extension declares , but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats`. A failed 16-bit conversion reports `cannot read "": the 16-bit PNG could not be converted to the normalized 8-bit sRGB form; convert it to an 8-bit PNG/JPEG/WebP and retry`. Provider and policy templates are quoted in their package READMEs. Guarded-mutation failures additionally carry their recovery instruction in the message, appended by this package's model-facing error wrapper: `FS_STALE_VERSION` gets `— re-read the file, then retry`, and `FS_NOT_OBSERVED` gets `— read the file, then retry`; the structured code is preserved. After that reread confirms absence, `edit` reports `FS_NOT_FOUND` instead of repeating a stale remedy, while `write` uses guarded creation. +Failures are normalized as `Error: `. This package's stable validation and read messages are `file_path must be a non-empty string`, `limit must be less than or equal to `, `old_string must be a non-empty string`, `old_string and new_string must differ`, `cannot read "": not found`, `cannot read "": not a regular file`, `offset is out of range for "" ( lines)`, `cannot read "": the extension does not declare a supported image format; read_image accepts PNG/JPEG/WebP/GIF files, including extension-less files in those formats`, `cannot read "": the file content is not a supported image format; read_image accepts PNG/JPEG/WebP/GIF`, `cannot read "": the bytes do not decode as a supported PNG/JPEG/WebP/GIF image; the file may be truncated or corrupt`, `cannot read "" as an image: model "" does not declare image input; switch to an image-capable model to read images`, and the mismatch repair `cannot read "": the extension declares , but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats` (an extension-less mismatch reports `cannot read "": the file signature claims , but the bytes decode as a different image format; the file may be corrupt`). A failed 16-bit conversion reports `cannot read "": the 16-bit PNG could not be converted to the normalized 8-bit sRGB form; convert it to an 8-bit PNG/JPEG/WebP and retry`. Provider and policy templates are quoted in their package READMEs. Guarded-mutation failures additionally carry their recovery instruction in the message, appended by this package's model-facing error wrapper: `FS_STALE_VERSION` gets `— re-read the file, then retry`, and `FS_NOT_OBSERVED` gets `— read the file, then retry`; the structured code is preserved. After that reread confirms absence, `edit` reports `FS_NOT_FOUND` instead of repeating a stale remedy, while `write` uses guarded creation. #### Token effect @@ -237,8 +239,9 @@ Append-only; newly visible content follows the reusable request prefix and does These limits define when the tool suite is a poor fit or needs special operational care. They are current package constraints, not a general filesystem comparison or a task backlog. - **No model-facing directory listing ships** — `ctx.fs.listDir` serves provider code such as skill discovery, while the sibling `dsh-tool-fs-search` package supplies ripgrep-backed `glob` and `grep` rather than extending the filesystem seam. -- **`read` handles UTF-8 text files only** — images use the separate extension-routed `read_image` tool; PDF, audio, and video remain deferred. A directory target is `FS_NOT_REGULAR_FILE`. -- **Extension-declared media type** — the extension selects the declared type and the attachment store's magic-byte validation stays authoritative; a correctly formatted image under a wrong extension is refused with the rename remedy rather than sniffed. +- **`read` handles UTF-8 text files only** — images use the separate `read_image` tool; PDF, audio, and video remain deferred. A directory target is `FS_NOT_REGULAR_FILE`. +- **Extension-declared media type** — an extension selects the declared type and the attachment store's magic-byte validation stays authoritative; a correctly formatted image under a wrong extension is refused with the rename remedy rather than sniffed. Only a path with no extension is identified from its file signature. +- **Object paths re-enter source admission** — `read_image` on a normalized attachment object re-admits its bytes as a new source, so a deployment whose `maxImageBytes`/`maxMessageImageBytes` sit below the normalized-image byte budget can refuse an object path that `ctx.attachments.readImage` still serves; shipped defaults keep the normalized budget (4 MiB) far under the source caps (20 MiB). - **No inline image preview on the tool-result card** — UI surfaces render the image result generically (the durable reference, not pixels); inline rendering is deferred to the UI packages. - **No attachment-region tool** — an agent may crop an image through another available tool when it has a filesystem path; a pasted or dragged image without a path cannot be re-read at higher resolution. - **No timeout surface** — `read`/`write`/`edit` take no timeout argument and declare no timeout budget; cancellation rides `exec.signal` only ([provider rationale](../README.md)). diff --git a/packages/fs/tool-fs/README.zh.md b/packages/fs/tool-fs/README.zh.md index 3cbd6bec6d..5915e5a07d 100644 --- a/packages/fs/tool-fs/README.zh.md +++ b/packages/fs/tool-fs/README.zh.md @@ -44,7 +44,7 @@ kind: "package-reference" | 工具 | 参数 | 行为 | |---|---|---| | `read` | `file_path`、`offset?`、`limit?` | 带行号的 UTF-8 内容与分页 footer；`offset` 从 1 开始，`limit` 默认为配置的 `readLimit`，上限也为该值 | -| `read_image` | `file_path` | 读取并持久保存 PNG/JPEG/WebP/GIF 源图；规范化可在下一次模型请求前缩小图片，因此模型无需先创建缩略图 | +| `read_image` | `file_path` | 读取并持久保存 PNG/JPEG/WebP/GIF 源图；无扩展名路径（包括规范化附件对象路径）按文件签名识别格式；规范化可在下一次模型请求前缩小图片，因此模型无需先创建缩略图 | | `write` | `file_path`、`content` | 创建或完整替换文件；有策略插件时，覆盖要求先在未变版本上执行 `read`，创建不需要 | | `edit` | `file_path`、`old_string`、`new_string`、`replace_all?` | 字面量替换，除非 `replace_all` 为 true 否则要求唯一匹配；有策略插件时，要求先执行 `read` 且文件未变 | @@ -65,6 +65,8 @@ kind: "package-reference" ### 策略与沙箱行为 +`read` 与 `read_image` 的路径授权完全由 `ctx.fs` 负责；媒体类型声明和文件签名只决定 `read_image` 是否接受该后端返回的字节。 + 挂载策略插件后，`write` 与 `edit` 从 `fs/*` 意图槽位取得防护，因此未读目标或陈旧观察会以 `FS_NOT_OBSERVED` 或 `FS_STALE_VERSION` 及恢复指令失败。使用施加沙箱限制的后端（`fs-sandbox`）时，`write`/`edit` 还会公开 `sandbox_permissions` 与 `justification`；被拒绝的变更返回 `[sandbox: file access denied under mode]` 标记与同轮次升级提示，获批的重试可以在该次调用中加盖严格更宽的模式。 ### 失败与恢复 @@ -219,7 +221,7 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces #### 模型看到的内容 -失败会规范化为 `Error: `。本包稳定的校验和读取消息是 `file_path must be a non-empty string`、`limit must be less than or equal to `、`old_string must be a non-empty string`、`old_string and new_string must differ`、`cannot read "": not found`、`cannot read "": not a regular file`、`offset is out of range for "" ( lines)`、`cannot read "": read_image only accepts PNG/JPEG/WebP/GIF paths`、`cannot read "" as an image: model "" does not declare image input; switch to an image-capable model to read images`，以及类型不匹配的修复消息 `cannot read "": the extension declares , but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats`。16-bit 转换失败会报告 `cannot read "": the 16-bit PNG could not be converted to the normalized 8-bit sRGB form; convert it to an 8-bit PNG/JPEG/WebP and retry`。提供方和策略模板在各自包的 README 中逐字列出。防护变更失败还会在消息中携带恢复指令，由本包面向模型的错误包装追加：`FS_STALE_VERSION` 追加 `— re-read the file, then retry`，`FS_NOT_OBSERVED` 追加 `— read the file, then retry`；结构化错误码保持不变。该次重新读取确认缺失后，`edit` 会报告 `FS_NOT_FOUND`，而不会重复陈旧恢复指令；`write` 则使用带防护的创建。 +失败会规范化为 `Error: `。本包稳定的校验和读取消息是 `file_path must be a non-empty string`、`limit must be less than or equal to `、`old_string must be a non-empty string`、`old_string and new_string must differ`、`cannot read "": not found`、`cannot read "": not a regular file`、`offset is out of range for "" ( lines)`、`cannot read "": the extension does not declare a supported image format; read_image accepts PNG/JPEG/WebP/GIF files, including extension-less files in those formats`、`cannot read "": the file content is not a supported image format; read_image accepts PNG/JPEG/WebP/GIF`、`cannot read "": the bytes do not decode as a supported PNG/JPEG/WebP/GIF image; the file may be truncated or corrupt`、`cannot read "" as an image: model "" does not declare image input; switch to an image-capable model to read images`，以及类型不匹配的修复消息 `cannot read "": the extension declares , but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats`（无扩展名路径的不匹配报告 `cannot read "": the file signature claims , but the bytes decode as a different image format; the file may be corrupt`）。16-bit 转换失败会报告 `cannot read "": the 16-bit PNG could not be converted to the normalized 8-bit sRGB form; convert it to an 8-bit PNG/JPEG/WebP and retry`。提供方和策略模板在各自包的 README 中逐字列出。防护变更失败还会在消息中携带恢复指令，由本包面向模型的错误包装追加：`FS_STALE_VERSION` 追加 `— re-read the file, then retry`，`FS_NOT_OBSERVED` 追加 `— read the file, then retry`；结构化错误码保持不变。该次重新读取确认缺失后，`edit` 会报告 `FS_NOT_FOUND`，而不会重复陈旧恢复指令；`write` 则使用带防护的创建。 #### Token 影响 @@ -237,8 +239,9 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces 这些限制说明工具套件何时不合适，或何时需要特别的运维注意。它们是当前包约束，不是通用文件系统对比或任务积压。 - **未交付面向模型的目录列表工具**：`ctx.fs.listDir` 服务于 skill（技能）发现等提供方代码，同级 `dsh-tool-fs-search` 包则提供基于 ripgrep 的 `glob` 与 `grep`，而不是扩展文件系统 seam。 -- **`read` 只处理 UTF-8 文本文件**：图像使用独立的、按扩展名路由的 `read_image` 工具；PDF、音频和视频仍延期处理。目录目标为 `FS_NOT_REGULAR_FILE`。 -- **媒体类型按扩展名声明**：扩展名选择声明类型，附件存储的魔数校验保持权威；扩展名错误但格式正确的图像会得到改名修复提示，而不是被嗅探接受。 +- **`read` 只处理 UTF-8 文本文件**：图像使用独立的 `read_image` 工具；PDF、音频和视频仍延期处理。目录目标为 `FS_NOT_REGULAR_FILE`。 +- **媒体类型按扩展名声明**：扩展名选择声明类型，附件存储的魔数校验保持权威；扩展名错误但格式正确的图像会得到改名修复提示，而不是被嗅探接受。只有没有扩展名的路径按文件签名识别格式。 +- **对象路径重新走源准入**：对规范化附件对象调用 `read_image` 会把其字节作为新来源重新准入，因此把 `maxImageBytes`/`maxMessageImageBytes` 配置得低于规范化图片字节预算的部署可能拒绝 `ctx.attachments.readImage` 仍可读取的对象路径；默认配置下规范化预算（4 MiB）远低于源上限（20 MiB）。 - **工具结果卡片没有内嵌图像预览**：UI 表面以通用形式渲染图像结果（持久引用而非像素）；内嵌渲染延后到 UI 包处理。 - **没有附件区域工具**：agent 在拥有文件系统路径时可以通过其他可用工具裁剪图片；没有路径的粘贴或拖入图片无法按更高分辨率重新读取。 - **没有超时接口**：`read`/`write`/`edit` 不接受超时参数，也不声明超时预算；取消只通过 `exec.signal` 传递（见[提供方理由](../README.zh.md)）。 diff --git a/packages/fs/tool-fs/src/read-image.ts b/packages/fs/tool-fs/src/read-image.ts index b1cbad9bb9..508d18de7a 100644 --- a/packages/fs/tool-fs/src/read-image.ts +++ b/packages/fs/tool-fs/src/read-image.ts @@ -1,5 +1,8 @@ /** - * The model-facing `read_image` tool commits a PNG/JPEG/WebP/GIF file. + * The model-facing `read_image` tool commits a PNG/JPEG/WebP/GIF file. A path + * without a file extension is identified from its file signature, while the + * attachment service's full decode stays authoritative. The mounted `ctx.fs` + * backend owns path resolution and read access; names only declare media type. * * The route gate is deliberately stricter than the host upload preflight. An * image-reading tool is useful only when the exact calling route can inspect @@ -11,7 +14,7 @@ import { basename, extname } from 'node:path' import type { Context } from '@deepseek-ai/cordis' import { AttachmentError, AttachmentId } from '@deepseek-ai/dsh-attachment' -import type { ImageAttachmentRef, ImageMediaType } from '@deepseek-ai/dsh-attachment' +import type { AttachmentStore, ImageAttachmentRef, ImageMediaType } from '@deepseek-ai/dsh-attachment' import type { ContentBlock } from '@deepseek-ai/dsh-llm' import { defineTool } from '@deepseek-ai/dsh-tools' import type { GenericCallView, ToolExecution } from '@deepseek-ai/dsh-tools' @@ -27,6 +30,35 @@ const IMAGE_EXTENSIONS: Readonly> = { '.gif': 'image/gif', } +const PNG_SIGNATURE = [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a] as const +const JPEG_SIGNATURE = [0xff, 0xd8, 0xff] as const + +function matchesBytes(data: Uint8Array, offset: number, expected: readonly number[]): boolean { + if (data.byteLength < offset + expected.length) return false + return expected.every((byte, index) => data[offset + index] === byte) +} + +function matchesAscii(data: Uint8Array, offset: number, value: string): boolean { + if (data.byteLength < offset + value.length) return false + for (let index = 0; index < value.length; index += 1) { + if (data[offset + index] !== value.charCodeAt(index)) return false + } + return true +} + +/** + * Identify the media type declared by a supported image file signature. + * @param data - file bytes read through the current filesystem backend. + * @returns the detected supported media type, or undefined for other bytes. + */ +export function sniffImageMediaType(data: Uint8Array): ImageMediaType | undefined { + if (matchesBytes(data, 0, PNG_SIGNATURE)) return 'image/png' + if (matchesBytes(data, 0, JPEG_SIGNATURE)) return 'image/jpeg' + if (matchesAscii(data, 0, 'GIF87a') || matchesAscii(data, 0, 'GIF89a')) return 'image/gif' + if (matchesAscii(data, 0, 'RIFF') && matchesAscii(data, 8, 'WEBP')) return 'image/webp' + return undefined +} + const IMAGE_VALUE_SCHEMA = { type: 'object', additionalProperties: false, @@ -98,6 +130,13 @@ export async function assertImageCapableRoute(ctx: Context, exec: ToolExecution, } } +/** Refuse a media type outside the deployment's accepted set, naming the offending path. */ +function assertDeploymentAccepts(attachments: AttachmentStore, mediaType: ImageMediaType, displayPath: string): void { + if (!attachments.imageLimits.mediaTypes.includes(mediaType)) { + throw new Error(`cannot read "${displayPath}": ${mediaType} images are not accepted by this deployment`) + } +} + /** * Re-brand a structured image outcome into the durable attachment reference an * `ImageBlock` carries. @@ -170,6 +209,7 @@ export function applyReadImageTool(ctx: Context): void { ctx.tools.register(defineTool({ name: 'read_image', description: 'Read a PNG/JPEG/WebP/GIF file and return the image itself. ' + + 'A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. ' + 'Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. ' + 'Independent files may be read concurrently in small batches. Requires the current model to accept image input.', parameters: { @@ -192,19 +232,20 @@ export function applyReadImageTool(ctx: Context): void { async execute(args, exec) { if (args.file_path.trim().length === 0) throw new Error('file_path must be a non-empty string') - // Every gate runs before any filesystem I/O so a refusal never leaks - // partial reads or attachment writes. - const mediaType = imageMediaTypeForPath(args.file_path) - if (mediaType === undefined) { - throw new Error(`cannot read "${args.file_path}": read_image only accepts PNG/JPEG/WebP/GIF paths`) + // Every pre-read gate runs before any filesystem I/O so a refusal never + // leaks partial reads or attachment writes. An extension-less path + // declares no format, so only its format and deployment media-type + // checks wait for the bytes. + const extension = extname(args.file_path).toLowerCase() + const declared = imageMediaTypeForPath(args.file_path) + if (declared === undefined && extension !== '') { + throw new Error(`cannot read "${args.file_path}": the ${extension} extension does not declare a supported image format; read_image accepts PNG/JPEG/WebP/GIF files, including extension-less files in those formats`) } const attachments = ctx.get('attachments') if (attachments === undefined) { throw new Error(`cannot read "${args.file_path}" as an image: no attachment service is mounted`) } - if (!attachments.imageLimits.mediaTypes.includes(mediaType)) { - throw new Error(`cannot read "${args.file_path}": ${mediaType} images are not accepted by this deployment`) - } + if (declared !== undefined) assertDeploymentAccepts(attachments, declared, args.file_path) await assertImageCapableRoute(ctx, exec, args.file_path) const { target, info } = await resolveRegularReadTarget(ctx, exec, args.file_path) @@ -213,6 +254,11 @@ export function applyReadImageTool(ctx: Context): void { // aggregate bound applies beside the per-image bound. const byteCap = Math.min(attachments.imageLimits.maxImageBytes, attachments.imageLimits.maxMessageImageBytes) const data = await ctx.fs.readBytes(target, exec.signal, byteCap) + const mediaType = declared ?? sniffImageMediaType(data) + if (mediaType === undefined) { + throw new Error(`cannot read "${target.displayPath}": the file content is not a supported image format; read_image accepts PNG/JPEG/WebP/GIF`) + } + if (declared === undefined) assertDeploymentAccepts(attachments, mediaType, target.displayPath) // Persist before returning: the image block must reference a durably // committed object by the time the tool/result event is appended. let ref: ImageAttachmentRef @@ -247,8 +293,19 @@ export function applyReadImageTool(ctx: Context): void { { cause: error }, ) } + if (error.code === 'INVALID_IMAGE' && declared === undefined) { + throw new Error( + `cannot read "${target.displayPath}": the bytes do not decode as a supported PNG/JPEG/WebP/GIF image; the file may be truncated or corrupt`, + { cause: error }, + ) + } if (error.code !== 'IMAGE_TYPE_MISMATCH') throw error - const extension = extname(target.displayPath).toLowerCase() + if (declared === undefined) { + throw new Error( + `cannot read "${target.displayPath}": the file signature claims ${mediaType}, but the bytes decode as a different image format; the file may be corrupt`, + { cause: error }, + ) + } throw new Error( `cannot read "${target.displayPath}": the ${extension} extension declares ${mediaType}, but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats`, { cause: error }, diff --git a/packages/fs/tool-fs/tests/read-image.spec.ts b/packages/fs/tool-fs/tests/read-image.spec.ts index 7244444245..2ac096429e 100644 --- a/packages/fs/tool-fs/tests/read-image.spec.ts +++ b/packages/fs/tool-fs/tests/read-image.spec.ts @@ -1,8 +1,9 @@ /** * The `read_image` tool over the REAL local filesystem and attachment store: - * extension routing, the strict image-modality gate (every refusal arm), - * durable commit + image-block rendering, attachment admission failures, and - * the regression that `read` keeps its text-only contract. + * extension routing, extension-less content sniffing (attachment object paths + * included), the strict image-modality gate (every refusal arm), durable + * commit + image-block rendering, attachment admission failures, and the + * regression that `read` keeps its text-only contract. */ import { afterEach, beforeEach, describe, expect, it } from 'vitest' @@ -28,6 +29,7 @@ import { formatImageReadOutput, imageMediaTypeForPath, imageRefFromValue, + sniffImageMediaType, } from '../src/read-image.ts' /** 1x1 red PNG (valid signature, IHDR, IDAT). */ @@ -165,6 +167,30 @@ describe('imageMediaTypeForPath', () => { }) }) +function ascii(value: string): Uint8Array { + return new TextEncoder().encode(value) +} + +describe('sniffImageMediaType', () => { + it('identifies each supported container from its complete signature', () => { + expect(sniffImageMediaType(Uint8Array.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00]))).toBe('image/png') + expect(sniffImageMediaType(Uint8Array.from([0xff, 0xd8, 0xff, 0xe0]))).toBe('image/jpeg') + expect(sniffImageMediaType(ascii('GIF87a...'))).toBe('image/gif') + expect(sniffImageMediaType(ascii('GIF89a...'))).toBe('image/gif') + expect(sniffImageMediaType(ascii('RIFF\0\0\0\0WEBPVP8 '))).toBe('image/webp') + }) + + it('returns undefined for other bytes, incomplete signatures, and non-WebP RIFF containers', () => { + expect(sniffImageMediaType(new Uint8Array())).toBeUndefined() + expect(sniffImageMediaType(ascii('plain text'))).toBeUndefined() + expect(sniffImageMediaType(Uint8Array.from([0x89, 0x50, 0x4e]))).toBeUndefined() + expect(sniffImageMediaType(Uint8Array.from([0xff, 0xd8]))).toBeUndefined() + expect(sniffImageMediaType(ascii('GIF90a'))).toBeUndefined() + expect(sniffImageMediaType(ascii('RIFF\0\0\0\0WAVE'))).toBeUndefined() + expect(sniffImageMediaType(ascii('RIFF\0\0\0'))).toBeUndefined() + }) +}) + describe('imageRefFromValue', () => { it('re-brands with and without the optional display name', () => { const base = { attachmentId: 'sha256:00', mediaType: 'image/png' as const, bytes: 1, width: 1, height: 1 } @@ -254,6 +280,143 @@ describe('read_image happy path', () => { }) }) +/** The mounted attachment service, asserted present for direct store calls. */ +function mountedStore(ctx: Context): AttachmentStore { + const attachments = ctx.get('attachments') + if (attachments === undefined) throw new Error('expected the attachment service') + return attachments +} + +/** The host object path behind a reference, asserted present for the local store. */ +function objectPathOf(attachments: AttachmentStore, ref: ImageAttachmentRef): string { + const hostPath = attachments.imageHostPath(ref) + if (hostPath === undefined) throw new Error('expected a host-file-backed store') + return hostPath +} + +describe('extension-less paths', () => { + it('reads a normalized attachment object path directly and dedups to the stored reference', async () => { + await writeFile(join(dir, 'red.png'), PNG_1X1) + const ctx = await setup() + const first = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model')) + expect(first.isError).toBe(false) + const ref = (first.content[1] as { attachment: ImageAttachmentRef }).attachment + const attachments = mountedStore(ctx) + const objectPath = objectPathOf(attachments, ref) + + const second = await readImage(ctx, { file_path: objectPath }, agentOn('vision-model')) + expect(second.isError).toBe(false) + const reread = (second.content[1] as { attachment: ImageAttachmentRef }).attachment + expect(reread.attachmentId).toBe(ref.attachmentId) + expect(reread.mediaType).toBe('image/png') + expect(text(second)).toContain(`${objectPath}`) + }) + + it('reads an ordinary extension-less image file by sniffing its content', async () => { + await writeFile(join(dir, 'avatar'), PNG_1X1) + const ctx = await setup() + const result = await readImage(ctx, { file_path: 'avatar' }, agentOn('vision-model')) + expect(result.isError).toBe(false) + const image = result.content[1] as { attachment: ImageAttachmentRef } + expect(image.attachment.mediaType).toBe('image/png') + expect(image.attachment.name).toBe('avatar') + }) + + it('pins the trailing-dot refusal and reads a dotfile through sniffing', async () => { + await writeFile(join(dir, '.hidden'), PNG_1X1) + const ctx = await setup() + const trailingDot = await readImage(ctx, { file_path: 'foo.' }, agentOn('vision-model')) + expect(trailingDot.isError).toBe(true) + expect(text(trailingDot)).toContain('cannot read "foo.": the . extension does not declare a supported image format') + + const dotfile = await readImage(ctx, { file_path: '.hidden' }, agentOn('vision-model')) + expect(dotfile.isError).toBe(false) + const image = dotfile.content[1] as { attachment: ImageAttachmentRef } + expect(image.attachment.mediaType).toBe('image/png') + expect(image.attachment.name).toBe('.hidden') + }) + + it('refuses extension-less bytes that are not a supported image', async () => { + await writeFile(join(dir, 'notes'), 'plain text, not an image') + const ctx = await setup() + const result = await readImage(ctx, { file_path: 'notes' }, agentOn('vision-model')) + expect(result.isError).toBe(true) + expect(text(result)).toContain(`cannot read "${join(dir, 'notes')}": the file content is not a supported image format`) + }) + + it('explains extension-less bytes that sniff as an image but do not decode', async () => { + await writeFile(join(dir, 'broken'), PNG_1X1.subarray(0, 16)) + const ctx = await setup() + const result = await readImage(ctx, { file_path: 'broken' }, agentOn('vision-model')) + expect(result.isError).toBe(true) + expect(text(result)).toContain('do not decode as a supported PNG/JPEG/WebP/GIF image') + }) + + it('applies the deployment media-type policy to the sniffed format', async () => { + /** Store whose deployment accepts JPEG only; sniffed PNG bytes must refuse before any save. */ + class JpegOnlySniffStore extends AttachmentStore { + readonly imageLimits: ImageAttachmentLimits = Object.freeze({ + maxImageBytes: 1024, + maxImagesPerMessage: 1, + maxMessageImageBytes: 1024, + maxImagePixels: 100, + maxImageDimension: 2000, + mediaTypes: Object.freeze(['image/jpeg'] as const), + }) + + validateImage(_input: SaveImageAttachment): Promise { + throw new Error('unreachable: the sniffed-format policy refuses before validation') + } + + saveImage(_input: SaveImageAttachment): Promise { + throw new Error('unreachable: the sniffed-format policy refuses before save') + } + + readImage(_ref: ImageAttachmentRef): Promise { + throw new Error('unreachable in this test') + } + } + await writeFile(join(dir, 'avatar'), PNG_1X1) + const ctx = await setup({ attachments: false }) + await ctx.plugin(JpegOnlySniffStore) + const result = await readImage(ctx, { file_path: 'avatar' }, agentOn('vision-model')) + expect(result.isError).toBe(true) + expect(text(result)).toContain('image/png images are not accepted by this deployment') + }) + + it('names a signature/decoded-format disagreement on an extension-less path', async () => { + /** Store whose admission reports a media-type mismatch; the tool cannot blame an extension. */ + class MismatchStore extends AttachmentStore { + readonly imageLimits: ImageAttachmentLimits = Object.freeze({ + maxImageBytes: 1024, + maxImagesPerMessage: 1, + maxMessageImageBytes: 1024, + maxImagePixels: 100, + maxImageDimension: 2000, + mediaTypes: Object.freeze(['image/png'] as const), + }) + + validateImage(_input: SaveImageAttachment): Promise { + return Promise.resolve() + } + + saveImage(_input: SaveImageAttachment): Promise { + throw new AttachmentError('Declared image type does not match its bytes.', 'IMAGE_TYPE_MISMATCH') + } + + readImage(_ref: ImageAttachmentRef): Promise { + throw new Error('unreachable in this test') + } + } + await writeFile(join(dir, 'sniffed'), PNG_1X1) + const ctx = await setup({ attachments: false }) + await ctx.plugin(MismatchStore) + const result = await readImage(ctx, { file_path: 'sniffed' }, agentOn('vision-model')) + expect(result.isError).toBe(true) + expect(text(result)).toContain('the file signature claims image/png, but the bytes decode as a different image format') + }) +}) + describe('strict image-modality gate', () => { it('accepts an exact visual route even when the advisory model catalog omits it', async () => { await writeFile(join(dir, 'red.png'), PNG_1X1) @@ -309,7 +472,7 @@ describe('argument and service preconditions', () => { const nonImage = await readImage(ctx, { file_path: 'notes.txt' }, agentOn('vision-model')) expect(nonImage.isError).toBe(true) - expect(text(nonImage)).toContain('only accepts PNG/JPEG/WebP/GIF paths') + expect(text(nonImage)).toContain('the .txt extension does not declare a supported image format') }) it('refuses when no attachment service is mounted', async () => { @@ -373,10 +536,10 @@ describe('image admission failures', () => { expect(text(result)).toContain('rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats') }) - it('fails with FS_TOO_LARGE before reading a file past maxImageBytes', async () => { - await writeFile(join(dir, 'red.png'), PNG_1X1) + it('caps an extension-less read at maxImageBytes before format detection', async () => { + await writeFile(join(dir, 'red'), PNG_1X1) const ctx = await setup({ storeConfig: { maxImageBytes: PNG_1X1.length - 1 } }) - const result = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model')) + const result = await readImage(ctx, { file_path: 'red' }, agentOn('vision-model')) expect(result.isError).toBe(true) expect(text(result)).toContain('exceeds') }) diff --git a/packages/subagent/subagent/README.i18n.yaml b/packages/subagent/subagent/README.i18n.yaml index e807931e2c..9277bc689f 100644 --- a/packages/subagent/subagent/README.i18n.yaml +++ b/packages/subagent/subagent/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subagent/subagent/README.md -README.md: 76df70351d711d580d3ab1a89d0f929b85eab172 -README.zh.md: c4deb001c47435d29a5ca18cc0f8c0d26e48941e +README.md: f13de661015f3aa59b776273376f07653f6bbba5 +README.zh.md: 42923504ed11b8eff7c0bdcd9d0c9505d741f42e diff --git a/packages/subagent/subagent/README.md b/packages/subagent/subagent/README.md index 76df70351d..f13de66101 100644 --- a/packages/subagent/subagent/README.md +++ b/packages/subagent/subagent/README.md @@ -48,7 +48,7 @@ One-shot children run once and settle with a single result, plus an optional str ### Following up, interrupting, and discovering -Continuable children answer follow-up messages as their next turns, and the parent can interrupt a running turn or list its children at any time. Discovery covers both shapes: the service lists direct children and the full descendant tree — mode, activity, and lineage — reading live session state and optional persistence, without loading any child. +Continuable children answer follow-up messages as their next turns, and the parent can interrupt a running turn or list its children at any time. A browser continuation prompt may carry image parts: the Host admits and persists each image batch through the attachment store before the child inbox accepts the message, and refuses delivery when the child's declared model does not accept image input. Discovery covers both shapes: the service lists direct children and the full descendant tree — mode, activity, and lineage — reading live session state and optional persistence, without loading any child. ### Failure and recovery diff --git a/packages/subagent/subagent/README.zh.md b/packages/subagent/subagent/README.zh.md index c4deb001c4..42923504ed 100644 --- a/packages/subagent/subagent/README.zh.md +++ b/packages/subagent/subagent/README.zh.md @@ -48,7 +48,7 @@ kind: "package-reference" ### 后续消息、中断与发现 -可继续子 agent 把后续消息作为下一个轮次回答，父级随时可以中断运行中的轮次或列举自己的子级。发现覆盖两种形态：服务列举直接子级与完整后代树——模式、活动状态与血缘——直接读取在线会话状态与可选持久化，不加载任何子 agent。 +可继续子 agent 把后续消息作为下一个轮次回答，父级随时可以中断运行中的轮次或列举自己的子级。浏览器发出的继续执行 prompt 可以携带图片部分：Host 先通过附件存储完成整批图片的准入与持久化，子级 inbox 才接受这条消息；当子级声明的模型不接受图片输入时拒绝投递。发现覆盖两种形态：服务列举直接子级与完整后代树——模式、活动状态与血缘——直接读取在线会话状态与可选持久化，不加载任何子 agent。 ### 失败与恢复 diff --git a/packages/subagent/subagent/src/continuation.ts b/packages/subagent/subagent/src/continuation.ts index 2cbd3fb846..52033d80f2 100644 --- a/packages/subagent/subagent/src/continuation.ts +++ b/packages/subagent/subagent/src/continuation.ts @@ -31,7 +31,7 @@ import type { AgentSetupCommit, CreateAgentOptions, } from '@deepseek-ai/dsh-agent' -import { ReasoningEffortId, boundContextSummary, createUserMessage, errorChain } from '@deepseek-ai/dsh-llm' +import { ReasoningEffortId, boundContextSummary, contentHasImage, createUserMessage, errorChain } from '@deepseek-ai/dsh-llm' import type { ContentBlock, MessageId, MessageSource } from '@deepseek-ai/dsh-llm' import type { SessionEvent, SessionId } from '@deepseek-ai/dsh-session' import type { SessionPersistence } from '@deepseek-ai/dsh-session-persistence' @@ -516,12 +516,25 @@ export class SubagentContinuationManager { if (activation === undefined) return this.coldResume(parent, childId, content, options) // A delivery that arrives after the disposal transaction began must not // reach a handle being torn down; wait for release, then cold-resume. + const disposal = activation.disposal /* v8 ignore next 3 -- the send-versus-dispose cutoff: reaching this arm needs a * delivery to observe the transaction inside the same critical section that opened it, * which no test can schedule deterministically. The behavior is covered end-to-end by * "cold-resumes a delivery that lost the race with final disposal". */ - if (activation.disposal !== undefined) { - return activation.disposal.then(() => undefined, () => undefined) + if (disposal !== undefined) { + return disposal.then(() => undefined, () => undefined) + } + // Text-only delivery stays await-free, so the disposal-cutoff check + // above and the submit share one critical window. The image path + // awaits a capability read, so it re-checks the cutoff afterwards; a + // disposal that began during the read is waited out and retried like + // one observed on entry. + if (contentHasImage(content)) { + await this.assertImageCapable(activation.handle.agent, options.signal) + if (activation.disposal !== undefined) { + await Promise.allSettled([activation.disposal]) + return undefined + } } return this.submitAdmitted(activation, content, options.source, parent, options.signal) }) @@ -1021,6 +1034,15 @@ export class SubagentContinuationManager { signal: AbortSignal, ): Promise { try { + if (contentHasImage(content)) { + // The capability read awaits with the activation already published, so + // the disposal cutoff is re-checked before the submit; a drain that + // began during the read turns into a clean closing rejection. + await this.assertImageCapable(activation.handle.agent, signal) + if (activation.disposal !== undefined) { + throw new SubagentError(`subagent "${activation.childId}" is closing`, 'ACTIVATION_CLOSING') + } + } return this.submitAdmitted(activation, content, source, parent, signal) } catch (error: unknown) { /* v8 ignore next -- rollback disposal failures must not mask the @@ -1030,6 +1052,37 @@ export class SubagentContinuationManager { } } + /** + * Refuse image content addressed to a child whose model accepts text only. + * Callers guard with `contentHasImage`, so text-only delivery never awaits. + * The check runs inside the per-child delivery lock, before the message + * exists, so a rejection leaves no partial user message. When the child's + * route is not fixed by its options (a request-waterfall listener owns it) + * or no LLM registry is composed, delivery proceeds and the LLM layer's + * text-only projection replaces each image with its stable placeholder. + * @param agent - the live or freshly materialized child agent. + * @param signal - caller cancellation bounding the model-info read. + * @throws {SubagentError} `MODEL_DOES_NOT_SUPPORT_IMAGES` when the child's resolved model declines image input. + */ + private async assertImageCapable( + agent: Agent, + signal: AbortSignal, + ): Promise { + const { provider, model } = agent.options + if (provider === undefined || model === undefined) return + const llm = this.ctx.get('llm') + /* v8 ignore next -- a deployment without the LLM registry serves no model + * to refuse against; delivery then defers to the text-only projection. */ + if (llm === undefined) return + const info = await llm.resolveModelInfo(provider, model, signal) + if (info.inputModalities !== undefined && !info.inputModalities.includes('image')) { + throw new SubagentError( + `Model "${model}" does not support image input.`, + 'MODEL_DOES_NOT_SUPPORT_IMAGES', + ) + } + } + /** * Create or resume the child Agent through the private activation-owner * scope, install the handle in a fresh Activation, and register ownership on diff --git a/packages/subagent/subagent/src/control-types.ts b/packages/subagent/subagent/src/control-types.ts index d40aa1d9f4..aaf69de0f8 100644 --- a/packages/subagent/subagent/src/control-types.ts +++ b/packages/subagent/subagent/src/control-types.ts @@ -6,10 +6,9 @@ * @module @deepseek-ai/dsh-subagent/control-types */ -import type { EncodedImageAttachment } from '@deepseek-ai/dsh-attachment/types' +import type { PromptContentPart } from '@deepseek-ai/dsh-attachment/types' import type { Branded } from '@deepseek-ai/dsh-brand' import type { MessageId } from '@deepseek-ai/dsh-llm/brand' -import type { ContentBlock } from '@deepseek-ai/dsh-llm/types' import type { SessionId } from '@deepseek-ai/dsh-session/types' /** @@ -95,21 +94,6 @@ export type SubagentAddress = | { readonly mode: 'continuable' } ) -/** - * One browser-encoded upload as the Session prompt wire carries it: the shared - * attachment vocabulary under the content-block tag. - */ -export interface EncodedImagePromptBlock extends EncodedImageAttachment { - readonly type: 'image' -} - -/** - * One block a browser prompt may carry. The encoded upload is accepted by the - * wire and refused by the Host, so the Client narrows nothing: a caller that - * attaches an image is answered, not silently stripped. - */ -export type SubagentPromptContentPart = ContentBlock | EncodedImagePromptBlock - /** One human message addressed to a continuable direct child. */ export interface SubagentPromptRequest { /** Identity persisted on the accepted message, minted before the call. */ @@ -118,8 +102,12 @@ export interface SubagentPromptRequest { readonly childSessionId: SessionId /** Required discriminator retained from the browser control address. */ readonly mode: 'continuable' - /** Content proposed as the child's user message; images are refused. */ - readonly content: readonly SubagentPromptContentPart[] + /** + * Browser prompt parts delivered as the child's user message. The Host + * admits and persists image parts before delivery, so the wire never + * carries a durable attachment reference the caller could fabricate. + */ + readonly content: readonly PromptContentPart[] /** Optional browser zone sampled for this exact human prompt. */ readonly clientTimeZone?: string } @@ -135,10 +123,8 @@ export interface SubagentInterruptReceipt { } /** - * Failure details the control surface answers with. The catalog read, the - * prompt, and the interrupt produce these codes; a Client fabricates - * `subagent/not-resumable` and `subagent/delivery-unavailable` for a one-shot - * address it refuses before the call, so both planes read one vocabulary. + * Failure details the control surface answers with. Catalog reads, prompts, + * and interrupts share this vocabulary with the Client Remote result. */ declare module '@deepseek-ai/dsh-typert-protocol' { interface RemoteErrorDetailsMap { @@ -150,11 +136,8 @@ declare module '@deepseek-ai/dsh-typert-protocol' { 'subagent/not-resumable': { readonly childSessionId: SessionId } /** The claimed parent does not own the addressed child. */ 'subagent/unauthorized': { readonly childSessionId: SessionId } - /** - * The continuation admits no attachment. `reason` names the refused plane - * for the caller's copy, as the Session prompt's attachment refusals do. - */ - 'subagent/attachment-unsupported': { readonly childSessionId: SessionId; readonly reason: string } + /** Image admission or model image-capability refusal. */ + 'subagent/attachment-invalid': { readonly reason: string } /** The child exists but its inbox cannot admit the message now. */ 'subagent/delivery-unavailable': { readonly childSessionId: SessionId } /** The deployment mounts no session-projection registry. */ diff --git a/packages/subagent/subagent/src/control.ts b/packages/subagent/subagent/src/control.ts index d3e68aeb39..41893ae93c 100644 --- a/packages/subagent/subagent/src/control.ts +++ b/packages/subagent/subagent/src/control.ts @@ -7,13 +7,11 @@ */ import type { Context } from '@deepseek-ai/cordis' -import type { ContentBlock } from '@deepseek-ai/dsh-llm' +import { AttachmentError } from '@deepseek-ai/dsh-attachment' import type { SessionId } from '@deepseek-ai/dsh-session' import { RemoteError } from '@deepseek-ai/dsh-typert-protocol' import { z } from 'zod' -import type { - SubagentCatalog, SubagentListEntry, SubagentPromptContentPart, -} from './control-types.ts' +import type { SubagentCatalog, SubagentListEntry } from './control-types.ts' import { SubagentError } from './error.ts' const SESSION_ID_SCHEMA = z.string().min(1) @@ -48,38 +46,6 @@ export function validateControlRequest( } } -/** - * Admit the content one continuation may deliver, refusing every image. - * - * The blocks become the child's user message verbatim, and this surface admits - * no attachment: nothing here registers encoded bytes with the attachment - * service, so an image would reach the child as a reference nothing resolves. - * The wire accepts the encoded upload so this refusal — not a Client that - * strips the block — is what the caller is answered with. Other block types - * still cross unnarrowed. - * @param childSessionId - the addressed child, named by the refusal. - * @param content - blocks the caller asked to deliver. - * @returns the admitted blocks, in order, as the durable content vocabulary. - * @throws {RemoteError} `subagent/attachment-unsupported` when any block is an image. - */ -export function admitPromptContent( - childSessionId: SessionId, - content: readonly SubagentPromptContentPart[], -): ContentBlock[] { - const admitted: ContentBlock[] = [] - for (const block of content) { - if (block.type === 'image') { - throw new RemoteError( - 'subagent/attachment-unsupported', - 'subagent continuation does not accept images', - { childSessionId, reason: 'SUBAGENT_IMAGE_UNSUPPORTED' }, - ) - } - admitted.push(block) - } - return admitted -} - /** * Project one durable listing onto the catalog view, replacing each row's * store-derived activity with the live Agent driver's status and reporting @@ -141,8 +107,18 @@ export function rejectPrompt(error: unknown, childSessionId: SessionId, signal: if (isCancellation(error, signal)) { throw new RemoteError('gateway/cancelled', 'subagent prompt was cancelled', {}, { cause: error }) } + if (error instanceof AttachmentError) { + throw new RemoteError('subagent/attachment-invalid', error.message, { reason: error.code }, { cause: error }) + } if (error instanceof SubagentError) { switch (error.code) { + case 'MODEL_DOES_NOT_SUPPORT_IMAGES': + throw new RemoteError( + 'subagent/attachment-invalid', + error.message, + { reason: error.code }, + { cause: error }, + ) case 'NOT_RESUMABLE': throw new RemoteError( 'subagent/not-resumable', diff --git a/packages/subagent/subagent/src/index.ts b/packages/subagent/subagent/src/index.ts index bce712991f..5d3a254075 100644 --- a/packages/subagent/subagent/src/index.ts +++ b/packages/subagent/subagent/src/index.ts @@ -30,6 +30,7 @@ */ import { Context } from '@deepseek-ai/cordis' +import { admitPromptContent } from '@deepseek-ai/dsh-attachment' import { scopeTarget } from '@deepseek-ai/dsh-scope' import type { Scoped } from '@deepseek-ai/dsh-scope' import { assertObjectJsonSchema } from '@deepseek-ai/dsh-tools' @@ -39,7 +40,7 @@ import type { SessionId } from '@deepseek-ai/dsh-session' import { canonicalClientTimeZone } from '@deepseek-ai/dsh-util-time' import { Remote, RemoteError, TypertRemoteService } from '@deepseek-ai/dsh-typert-protocol' import { - admitPromptContent, catalogView, rejectCatalogRead, rejectPrompt, validateControlRequest, + catalogView, rejectCatalogRead, rejectPrompt, validateControlRequest, } from './control.ts' import type { SubagentCatalog, @@ -418,10 +419,12 @@ export class SubagentRuntime extends TypertRemoteService { * validated browser zone on the accepted message. Success identifies the * message the child's FIFO inbox accepted; later execution is independent of * this call. + * Image parts are admitted and persisted through the attachment store + * before delivery, and the child's model must accept image input. * @param request - durable address, minted identity, content, and optional browser zone. * @param signal - carrier cancellation, owning the call until inbox acceptance. * @returns the accepted message's inbox identity. - * @throws {RemoteError} `gateway/bad-request`, `subagent/attachment-unsupported`, + * @throws {RemoteError} `gateway/bad-request`, `subagent/attachment-invalid`, * `subagent/invalid-time-zone`, `subagent/parent-unavailable`, * `subagent/not-resumable`, `subagent/unauthorized`, * `subagent/delivery-unavailable`, `gateway/cancelled`, or `gateway/internal`. @@ -430,7 +433,6 @@ export class SubagentRuntime extends TypertRemoteService { async prompt(request: SubagentPromptRequest, signal: AbortSignal): Promise { const { parentSessionId, childSessionId, clientTimeZone } = request validateControlRequest('subagent.prompt', request) - const content = admitPromptContent(childSessionId, request.content) const canonicalTimeZone = clientTimeZone === undefined ? undefined : canonicalClientTimeZone(clientTimeZone) @@ -455,6 +457,16 @@ export class SubagentRuntime extends TypertRemoteService { ...(canonicalTimeZone === undefined ? {} : { clientTimeZone: canonicalTimeZone }), } try { + // Admission precedes delivery: image parts become durable references + // here, so the child inbox only ever accepts Host-persisted attachments. + let content: ContentBlock[] + if (request.content.every((part): part is { readonly type: 'text'; readonly text: string } => part.type === 'text')) { + content = request.content.map(part => ({ type: 'text', text: part.text })) + } else { + const attachments = this.ctx.get('attachments') + if (attachments === undefined) throw new Error('subagent image prompt requires an attachment store') + content = await admitPromptContent(attachments, request.content) + } return { messageId: await this.followup(parent, childSessionId, content, { source, signal }) } } catch (error: unknown) { return rejectPrompt(error, childSessionId, signal) diff --git a/packages/subagent/subagent/tests/continuation.spec.ts b/packages/subagent/subagent/tests/continuation.spec.ts index 31b5bdbcb5..64d6bd21c3 100644 --- a/packages/subagent/subagent/tests/continuation.spec.ts +++ b/packages/subagent/subagent/tests/continuation.spec.ts @@ -528,6 +528,126 @@ describe('SubagentRuntime.startContinuable', () => { }) }) +describe('continuable image follow-ups', () => { + const imageBlock = { + type: 'image' as const, + attachment: { + attachmentId: 'att-1' as never, mediaType: 'image/png' as const, bytes: 1, width: 1, height: 1, + }, + } + + it('refuses an image follow-up when the child model declines image input, leaving no partial message', async () => { + const { ctx, parent } = await setup([textResponse('child work')]) + const started = await ctx.subagents.startContinuable(startSpec(parent)) + await waitNoActivation(ctx, started.childId) + const resolve = vi.spyOn(ctx.llm, 'resolveModelInfo') + .mockResolvedValue({ inputModalities: ['text'] } as never) + + await expect(ctx.subagents.followup(parent, started.childId, [ + { type: 'text' as const, text: 'see this' }, + imageBlock, + ], { source: { kind: 'user' }, signal: testSignal })) + .rejects.toMatchObject({ code: 'MODEL_DOES_NOT_SUPPORT_IMAGES' }) + + expect(resolve).toHaveBeenCalledWith('mock', 'mock', testSignal) + const loaded = await ctx.sessionPersistence.load(started.childId) + expect(hasUserText(loaded.events, 'see this')).toBe(false) + await drainManager(ctx) + }) + + it('delivers an image follow-up to a resident child when its model accepts image input', async () => { + const releaseFirst = Promise.withResolvers() + const adapter = new GatedAdapter([ + { chunks: textResponse('child work'), gate: releaseFirst.promise }, + { chunks: textResponse('image reply') }, + ]) + const { ctx, parent } = await setupWith(adapter) + const started = await ctx.subagents.startContinuable(startSpec(parent)) + await vi.waitFor(() => { + expect(adapter.requests).toHaveLength(1) + }) + vi.spyOn(ctx.llm, 'resolveModelInfo') + .mockResolvedValue({ inputModalities: ['text', 'image'] } as never) + + await ctx.subagents.followup(parent, started.childId, [ + { type: 'text' as const, text: 'compare' }, + imageBlock, + ], { source: { kind: 'user' }, signal: testSignal }) + releaseFirst.resolve(undefined) + await waitNoActivation(ctx, started.childId) + + const loaded = await ctx.sessionPersistence.load(started.childId) + const delivered = loaded.events.find(event => event.type === 'user/message' + && event.data.content.some(block => block.type === 'image')) + expect(delivered?.type === 'user/message' && delivered.data.content).toEqual([ + { type: 'text', text: 'compare' }, + imageBlock, + ]) + await drainManager(ctx) + }) + + it('re-checks the disposal cutoff when a drain begins during a live image capability read', async () => { + const releaseFirst = Promise.withResolvers() + const adapter = new GatedAdapter([{ chunks: textResponse('child work'), gate: releaseFirst.promise }]) + const { ctx, parent } = await setupWith(adapter) + const started = await ctx.subagents.startContinuable(startSpec(parent)) + await vi.waitFor(() => { expect(adapter.requests).toHaveLength(1) }) + const capability = Promise.withResolvers<{ inputModalities: string[] }>() + const resolve = vi.spyOn(ctx.llm, 'resolveModelInfo').mockReturnValue(capability.promise as never) + + const delivery = ctx.subagents.followup(parent, started.childId, [imageBlock], { + source: { kind: 'user' }, signal: testSignal, + }) + delivery.catch(() => undefined) + await vi.waitFor(() => { expect(resolve).toHaveBeenCalled() }) + releaseFirst.resolve(undefined) + const draining = drainManager(ctx) + capability.resolve({ inputModalities: ['text', 'image'] }) + + await expect(delivery).rejects.toMatchObject({ code: 'DRAINING' }) + await draining + }) + + it('rejects a materialized image follow-up whose capability read raced a drain', async () => { + const { ctx, parent } = await setup([textResponse('child work')]) + const started = await ctx.subagents.startContinuable(startSpec(parent)) + await waitNoActivation(ctx, started.childId) + const capability = Promise.withResolvers<{ inputModalities: string[] }>() + const resolve = vi.spyOn(ctx.llm, 'resolveModelInfo').mockReturnValue(capability.promise as never) + + const delivery = ctx.subagents.followup(parent, started.childId, [imageBlock], { + source: { kind: 'user' }, signal: testSignal, + }) + delivery.catch(() => undefined) + await vi.waitFor(() => { expect(resolve).toHaveBeenCalled() }) + const draining = drainManager(ctx) + capability.resolve({ inputModalities: ['text', 'image'] }) + + await expect(delivery).rejects.toMatchObject({ code: 'ACTIVATION_CLOSING' }) + await draining + const loaded = await ctx.sessionPersistence.load(started.childId) + expect(loaded.events.some(event => event.type === 'user/message' + && event.data.content.some(block => block.type === 'image'))).toBe(false) + }) + + it('defers to the text-only projection when the descriptor declares no model route', async () => { + const { ctx } = await setup([]) + const routeless = ctx.agentLoop.create(SessionId('routeless-image'), {}) + const started = await ctx.subagents.startContinuable(startSpec(routeless)) + await waitNoActivation(ctx, started.childId) + const resolve = vi.spyOn(ctx.llm, 'resolveModelInfo') + + // Acceptance is the success boundary: with no declared route there is no + // model to refuse against, so the image message enters the child inbox. + await ctx.subagents.followup(routeless, started.childId, [imageBlock], { + source: { kind: 'user' }, signal: testSignal, + }) + + expect(resolve).not.toHaveBeenCalled() + await drainManager(ctx) + }) +}) + describe('SubagentRuntime.followup residency routing', () => { it('fails a cold follow-up when Session query is unavailable', async () => { const { ctx, parent } = await setupWith(new MockAdapter([]), { diff --git a/packages/subagent/subagent/tests/control.spec.ts b/packages/subagent/subagent/tests/control.spec.ts index 97641511f4..bd29ccfccb 100644 --- a/packages/subagent/subagent/tests/control.spec.ts +++ b/packages/subagent/subagent/tests/control.spec.ts @@ -5,6 +5,7 @@ import { describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' +import { AttachmentError } from '@deepseek-ai/dsh-attachment' import type { MessageId } from '@deepseek-ai/dsh-llm' import { SessionId } from '@deepseek-ai/dsh-session' import SubagentRuntime, { @@ -19,6 +20,8 @@ const OTHER = SessionId('other') const BROKEN = SessionId('broken') const REQUEST_ID = 'req-1' as SubagentPromptRequestId const signal = new AbortController().signal +/** Durable-reference base for the fake store; per-test ids and media types override. */ +const IMAGE_REF = { attachmentId: 'att', mediaType: 'image/png', bytes: 2, width: 1, height: 1 } /** The runtime plus a programmable live-Agent registry, omitted to compose none. */ async function bench(live?: Record) { @@ -156,33 +159,83 @@ describe('subagent prompt Remote', () => { expect(followup).not.toHaveBeenCalled() }) - it('forwards non-text content blocks without narrowing them', async () => { - const { subagents } = await bench({ [PARENT]: { status: 'idle' } }) + it('admits ordered image parts into durable references before delivery', async () => { + const { ctx, subagents } = await bench({ [PARENT]: { status: 'idle' } }) + const saveImages = vi.fn(async (inputs: readonly { mediaType: string }[]) => + inputs.map((input, index) => ({ ...IMAGE_REF, attachmentId: `att-${index}`, mediaType: input.mediaType }))) + ctx.provide('attachments', { saveImages } as never) const followup = vi.spyOn(subagents, 'followup').mockResolvedValue('m-content' as MessageId) - const content = [{ type: 'reasoning' as const, text: 'retain this block' }] + const content = [ + { type: 'text' as const, text: 'before' }, + { type: 'image' as const, mediaType: 'image/png' as const, data: 'aGk=' }, + { type: 'text' as const, text: 'after' }, + ] await expect(subagents.prompt({ ...promptRequest(), content }, signal)) .resolves.toEqual({ messageId: 'm-content' }) - expect(followup.mock.calls[0]?.[2]).toEqual(content) + expect(followup.mock.calls[0]?.[2]).toEqual([ + { type: 'text', text: 'before' }, + { type: 'image', attachment: { ...IMAGE_REF, attachmentId: 'att-0', mediaType: 'image/png' } }, + { type: 'text', text: 'after' }, + ]) }) - it('refuses an image the wire accepts, whatever else the content carries', async () => { + it('maps a refused image batch to subagent/attachment-invalid and delivers nothing', async () => { + const { ctx, subagents } = await bench({ [PARENT]: { status: 'idle' } }) + ctx.provide('attachments', { + saveImages: async () => { + throw new AttachmentError('Image batch exceeds the configured image-count limit.', 'TOO_MANY_IMAGES') + }, + } as never) + const followup = vi.spyOn(subagents, 'followup') + + await expect(subagents.prompt({ + ...promptRequest(), + content: [{ type: 'image' as const, mediaType: 'image/png' as const, data: 'aGk=' }], + }, signal)).rejects.toMatchObject({ + code: 'subagent/attachment-invalid', details: { reason: 'TOO_MANY_IMAGES' }, + }) + expect(followup).not.toHaveBeenCalled() + }) + + it('maps non-canonical base64 to subagent/attachment-invalid without touching the store', async () => { + const { ctx, subagents } = await bench({ [PARENT]: { status: 'idle' } }) + const saveImages = vi.fn() + ctx.provide('attachments', { saveImages } as never) + const followup = vi.spyOn(subagents, 'followup') + + await expect(subagents.prompt({ + ...promptRequest(), + content: [{ type: 'image' as const, mediaType: 'image/png' as const, data: 'not base64!' }], + }, signal)).rejects.toMatchObject({ + code: 'subagent/attachment-invalid', details: { reason: 'INVALID_IMAGE_BASE64' }, + }) + expect(saveImages).not.toHaveBeenCalled() + expect(followup).not.toHaveBeenCalled() + }) + + it('rejects an image prompt when no attachment store is composed', async () => { const { subagents } = await bench({ [PARENT]: { status: 'idle' } }) const followup = vi.spyOn(subagents, 'followup') - // Both variants the widened wire admits: the browser's encoded upload and a - // durable reference. The Client narrows neither, so the Host answers both. - const encoded = { type: 'image' as const, mediaType: 'image/png' as const, data: 'AAA=' } - const durable = { type: 'image' as const, attachment: { attachmentId: 'att-1' } as never } - for (const image of [encoded, durable]) { - const content = [{ type: 'text' as const, text: 'look at this' }, image] - await expect(subagents.prompt({ ...promptRequest(), content }, signal)).rejects.toMatchObject({ - code: 'subagent/attachment-unsupported', - details: { childSessionId: CHILD, reason: 'SUBAGENT_IMAGE_UNSUPPORTED' }, - }) - } + + await expect(subagents.prompt({ + ...promptRequest(), + content: [{ type: 'image' as const, mediaType: 'image/png' as const, data: 'aGk=' }], + }, signal)).rejects.toMatchObject({ code: 'gateway/internal', message: 'subagent prompt failed' }) expect(followup).not.toHaveBeenCalled() }) + it('maps a text-only child model refusal to subagent/attachment-invalid', async () => { + const { subagents } = await bench({ [PARENT]: { status: 'idle' } }) + vi.spyOn(subagents, 'followup').mockRejectedValue( + new SubagentError('Model "text-only" does not support image input.', 'MODEL_DOES_NOT_SUPPORT_IMAGES'), + ) + + await expect(subagents.prompt(promptRequest(), signal)).rejects.toMatchObject({ + code: 'subagent/attachment-invalid', details: { reason: 'MODEL_DOES_NOT_SUPPORT_IMAGES' }, + }) + }) + it('delivers the content under the caller-minted identity and canonical browser zone', async () => { const { subagents } = await bench({ [PARENT]: { status: 'idle' } }) const followup = vi.spyOn(subagents, 'followup').mockResolvedValue('m-1' as MessageId) diff --git a/packages/test-support/session-snapshot/tests/harness.spec.ts b/packages/test-support/session-snapshot/tests/harness.spec.ts index 647c67d662..4c88111de5 100644 --- a/packages/test-support/session-snapshot/tests/harness.spec.ts +++ b/packages/test-support/session-snapshot/tests/harness.spec.ts @@ -66,6 +66,8 @@ async function scenario(behavior: object): Promise<{ dir: string; fixtureFile: s } const boot: InputStep[] = [{ op: 'initialize' }, { op: 'newSession' }] +// A Windows coverage shard can spend more than 20ms harvesting logs before vi.waitFor records the diagnostic error. +const titleDiagnosticTimeoutMs = process.platform === 'win32' ? 5_000 : 20 it('keeps scenario-owned snapshot spill root length stable across platforms', () => { const fixtureFile = '/fixtures/scenario/session.jsonl' @@ -1057,11 +1059,11 @@ describe('runScenario', () => { steps: [ ...boot, { op: 'promptAndCancel', text: 'hang' }, - { op: 'waitForTitleAfterTurnEnd', timeoutMs: 20 }, + { op: 'waitForTitleAfterTurnEnd', timeoutMs: titleDiagnosticTimeoutMs }, ], }, { agent: AGENT, mode: 'replay', fixtureFile }, - )).rejects.toThrow(/did not persist session\/title after turn\/end within 20ms/) + )).rejects.toThrow(new RegExp(`did not persist session/title after turn/end within ${titleDiagnosticTimeoutMs}ms`)) }) it('waitForEventAfterTurnEnd holds the app for a typed post-boundary record and times out otherwise', { timeout: 20_000 }, async () => { diff --git a/scripts/ci-workflow.spec.ts b/scripts/ci-workflow.spec.ts index 65a5c0b9d5..e33df9aeb3 100644 --- a/scripts/ci-workflow.spec.ts +++ b/scripts/ci-workflow.spec.ts @@ -119,6 +119,35 @@ describe('CI workflow', () => { )) expect(buildCommands.map(step => step.run)).toContain('pnpm run check:ci:windows-blocking') + // The four native Windows installs branch on the workspace filesystem: + // clone (ReFS block clone) only on ReFS, plain install elsewhere. This + // keeps the TS6231 store-path leak (see the Windows ReFS store note) out + // of the self-hosted pool without forcing clone onto hosted NTFS, which + // rejects copy-on-write. The branch must stay, or a hosted fallback would + // fail installs with ERR_PNPM_LINKING_FAILED. + for (const [jobName, job] of [['windows-build', windowsBuild], ['windows-coverage', windowsCoverage], ['windows-native-tests', windowsNativeTests], ['windows-observational', windowsObservational]] as const) { + const steps = job.steps as unknown[] + const install = steps.find((step): step is Record & { run: string } => ( + isRecord(step) && step.name === 'Install (immutable)' && typeof step.run === 'string' + )) + expect(install, `${jobName} must define the filesystem-branched install`).toBeDefined() + expect(install!.run).toContain("$fs -eq 'ReFS'") + expect(install!.run).toContain('--package-import-method=clone') + expect(install!.run).toContain('corepack pnpm install') + // The else branch must keep the plain hosted install as a distinct line + // (not the corepack clone line, which contains the same substring); + // dropping it or making both branches clone would force clone onto + // NTFS, which rejects copy-on-write (ERR_PNPM_LINKING_FAILED). The + // YAML folded block keeps the first statement on line 1 and folds the + // rest with leading two-space indents. + const installLines = install!.run.split('\n').map(line => line.trim()) + expect(installLines).toContain('} else {') + expect(installLines.some(line => line === 'pnpm install --frozen-lockfile'), `${jobName} else branch must keep the plain hosted install`).toBe(true) + // The ReFS branch must not use the interpolated empty-flag form, which + // passes a stray "" positional argument to pnpm. + expect(install!.run).not.toContain('$cloneFlag') + } + // windows-coverage uses the lower 4-partition profile. expect(windowsCoverage.name).toBe('windows node 24 / coverage') expect(windowsCoverage.env).toMatchObject({ DSH_COVERAGE_PARTITIONS: '4' }) @@ -152,6 +181,26 @@ describe('CI workflow', () => { expect(serialWindows.if).toBe("github.event_name == 'push' && github.ref == 'refs/heads/master'") expect(serialWindows['runs-on']).toEqual(['self-hosted', 'dsh-win-ci', 'windows']) expect(serialWindows.name).toBe('serial / windows (self-hosted standby)') + // Its store must share the ReFS workspace volume for clone; the install + // must carry the same filesystem branch as the PR jobs. + const serialSteps = serialWindows.steps as unknown[] + const serialStore = serialSteps.find((step): step is Record & { run: string } => ( + isRecord(step) && step.name === 'Configure persistent pnpm store' && typeof step.run === 'string' + )) + expect(serialStore).toBeDefined() + expect(serialStore!.run).toContain('F:\\.pnpm-store') + const serialInstall = serialSteps.find((step): step is Record & { run: string } => ( + isRecord(step) && step.name === 'Install (immutable)' && typeof step.run === 'string' + )) + expect(serialInstall).toBeDefined() + expect(serialInstall!.run).toContain("$fs -eq 'ReFS'") + expect(serialInstall!.run).toContain('--package-import-method=clone') + expect(serialInstall!.run).toContain('corepack pnpm install') + // Distinct else-branch line, as for the PR jobs: the corepack clone line + // contains the plain-install substring too. + expect(serialInstall!.run.split('\n').map(line => line.trim())).toContain('} else {') + expect(serialInstall!.run.split('\n').map(line => line.trim())).toContain('pnpm install --frozen-lockfile') + expect(serialInstall!.run).not.toContain('$cloneFlag') // Aggregate: Wine and the required split native jobs are needed; // windows-coverage is temporarily non-blocking while Windows ACP diff --git a/snapshots/sdk/bash-tool/tool-schemas.expected.json b/snapshots/sdk/bash-tool/tool-schemas.expected.json index e8fd1b5981..e8b70b6c13 100644 --- a/snapshots/sdk/bash-tool/tool-schemas.expected.json +++ b/snapshots/sdk/bash-tool/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/sdk/subagent-continuable-inheritance/tool-schemas.1.expected.json b/snapshots/sdk/subagent-continuable-inheritance/tool-schemas.1.expected.json index a3c5b2e531..b3dee1a21b 100644 --- a/snapshots/sdk/subagent-continuable-inheritance/tool-schemas.1.expected.json +++ b/snapshots/sdk/subagent-continuable-inheritance/tool-schemas.1.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/sdk/subagent-continuable/tool-schemas.1.expected.json b/snapshots/sdk/subagent-continuable/tool-schemas.1.expected.json index a3c5b2e531..b3dee1a21b 100644 --- a/snapshots/sdk/subagent-continuable/tool-schemas.1.expected.json +++ b/snapshots/sdk/subagent-continuable/tool-schemas.1.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/sdk/subagent-dsh-sdk-diagnostic/tool-schemas.expected.json b/snapshots/sdk/subagent-dsh-sdk-diagnostic/tool-schemas.expected.json index b3a1813e8b..a95ea12f74 100644 --- a/snapshots/sdk/subagent-dsh-sdk-diagnostic/tool-schemas.expected.json +++ b/snapshots/sdk/subagent-dsh-sdk-diagnostic/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/sdk/subagent-dsh-sdk-dynamic-route/tool-schemas.1.expected.json b/snapshots/sdk/subagent-dsh-sdk-dynamic-route/tool-schemas.1.expected.json index fe0882fe53..5b0728889d 100644 --- a/snapshots/sdk/subagent-dsh-sdk-dynamic-route/tool-schemas.1.expected.json +++ b/snapshots/sdk/subagent-dsh-sdk-dynamic-route/tool-schemas.1.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/sdk/subagent-dsh-sdk-dynamic-route/tool-schemas.expected.json b/snapshots/sdk/subagent-dsh-sdk-dynamic-route/tool-schemas.expected.json index 3d92e885eb..3000f6cb46 100644 --- a/snapshots/sdk/subagent-dsh-sdk-dynamic-route/tool-schemas.expected.json +++ b/snapshots/sdk/subagent-dsh-sdk-dynamic-route/tool-schemas.expected.json @@ -323,7 +323,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/sdk/subagent-list-agents/tool-schemas.1.expected.json b/snapshots/sdk/subagent-list-agents/tool-schemas.1.expected.json index a3c5b2e531..b3dee1a21b 100644 --- a/snapshots/sdk/subagent-list-agents/tool-schemas.1.expected.json +++ b/snapshots/sdk/subagent-list-agents/tool-schemas.1.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/sdk/subagent-report/tool-schemas.1.expected.json b/snapshots/sdk/subagent-report/tool-schemas.1.expected.json index a3c5b2e531..b3dee1a21b 100644 --- a/snapshots/sdk/subagent-report/tool-schemas.1.expected.json +++ b/snapshots/sdk/subagent-report/tool-schemas.1.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/sdk/text-turn/tool-schemas.expected.json b/snapshots/sdk/text-turn/tool-schemas.expected.json index e8fd1b5981..e8b70b6c13 100644 --- a/snapshots/sdk/text-turn/tool-schemas.expected.json +++ b/snapshots/sdk/text-turn/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/agent-instructions/tool-schemas.expected.json b/snapshots/session/agent-instructions/tool-schemas.expected.json index 36df7a0481..ebdc1383f5 100644 --- a/snapshots/session/agent-instructions/tool-schemas.expected.json +++ b/snapshots/session/agent-instructions/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { @@ -1007,7 +1007,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/both-mode-turn/system-prompt.expected.md b/snapshots/session/both-mode-turn/system-prompt.expected.md index 8511e0eff3..4ac177a40e 100644 --- a/snapshots/session/both-mode-turn/system-prompt.expected.md +++ b/snapshots/session/both-mode-turn/system-prompt.expected.md @@ -154,7 +154,7 @@ interface ToolArgsMap { /** Maximum number of lines to return. Defaults to 2000. */ limit?: number; } & Record; - /** Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input. */ + /** Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input. */ read_image: { /** Path to the image file, resolved by the filesystem backend. */ file_path: string; diff --git a/snapshots/session/both-mode-turn/tool-schemas.expected.json b/snapshots/session/both-mode-turn/tool-schemas.expected.json index 9893105262..c0426f9673 100644 --- a/snapshots/session/both-mode-turn/tool-schemas.expected.json +++ b/snapshots/session/both-mode-turn/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/compaction-recovery/tool-schemas.expected.json b/snapshots/session/compaction-recovery/tool-schemas.expected.json index 36df7a0481..ebdc1383f5 100644 --- a/snapshots/session/compaction-recovery/tool-schemas.expected.json +++ b/snapshots/session/compaction-recovery/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { @@ -1007,7 +1007,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/cordis-inspect-jsdoc/system-prompt.expected.md b/snapshots/session/cordis-inspect-jsdoc/system-prompt.expected.md index 93ad87e990..556669a52a 100644 --- a/snapshots/session/cordis-inspect-jsdoc/system-prompt.expected.md +++ b/snapshots/session/cordis-inspect-jsdoc/system-prompt.expected.md @@ -321,7 +321,7 @@ interface ToolArgsMap { /** Maximum number of lines to return. Defaults to 2000. */ limit?: number; } & Record; - /** Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input. */ + /** Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input. */ read_image: { /** Path to the image file, resolved by the filesystem backend. */ file_path: string; diff --git a/snapshots/session/cordis-inspect-jsdoc/tool-schemas.expected.json b/snapshots/session/cordis-inspect-jsdoc/tool-schemas.expected.json index 3a0073f42a..bf0d4e5f04 100644 --- a/snapshots/session/cordis-inspect-jsdoc/tool-schemas.expected.json +++ b/snapshots/session/cordis-inspect-jsdoc/tool-schemas.expected.json @@ -503,7 +503,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/fs-glob-sampling/tool-schemas.expected.json b/snapshots/session/fs-glob-sampling/tool-schemas.expected.json index 47769041bd..8a61071acd 100644 --- a/snapshots/session/fs-glob-sampling/tool-schemas.expected.json +++ b/snapshots/session/fs-glob-sampling/tool-schemas.expected.json @@ -226,7 +226,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/lsp-definition/tool-schemas.expected.json b/snapshots/session/lsp-definition/tool-schemas.expected.json index 05543b4012..6c691f70fc 100644 --- a/snapshots/session/lsp-definition/tool-schemas.expected.json +++ b/snapshots/session/lsp-definition/tool-schemas.expected.json @@ -343,7 +343,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/product-subagent-both/tool-schemas.expected.json b/snapshots/session/product-subagent-both/tool-schemas.expected.json index e457070e01..22dc3e2126 100644 --- a/snapshots/session/product-subagent-both/tool-schemas.expected.json +++ b/snapshots/session/product-subagent-both/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/product-subagent-codex/tool-schemas.expected.json b/snapshots/session/product-subagent-codex/tool-schemas.expected.json index bc41f13b88..69094cb8ae 100644 --- a/snapshots/session/product-subagent-codex/tool-schemas.expected.json +++ b/snapshots/session/product-subagent-codex/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/product-subagent-result-diagnostic/tool-schemas.expected.json b/snapshots/session/product-subagent-result-diagnostic/tool-schemas.expected.json index 2072fb3d15..29836f8c17 100644 --- a/snapshots/session/product-subagent-result-diagnostic/tool-schemas.expected.json +++ b/snapshots/session/product-subagent-result-diagnostic/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/ptc-read-image-attachment-path/replay.override.json b/snapshots/session/ptc-read-image-attachment-path/replay.override.json new file mode 100644 index 0000000000..c1b0ca939f --- /dev/null +++ b/snapshots/session/ptc-read-image-attachment-path/replay.override.json @@ -0,0 +1,29 @@ +[ + { + "kind": "chunks", + "chunks": [ + { "type": "block-start", "index": 0, "blockType": "tool-call" }, + { "type": "block-end", "index": 0, "block": { "type": "tool-call", "id": "code-image-source", "name": "run_code", "arguments": "{\"code\":\"return await tools.read_image({ file_path: 'red.png' })\",\"description\":\"Read the source image\"}" } }, + { "type": "usage", "usage": { "inputTokens": 3, "outputTokens": 3 } }, + { "type": "finish", "reason": { "kind": "tool-calls" } } + ] + }, + { + "kind": "chunks", + "chunks": [ + { "type": "block-start", "index": 0, "blockType": "tool-call" }, + { "type": "block-end", "index": 0, "block": { "type": "tool-call", "id": "code-image-object", "name": "run_code", "arguments": "{\"code\":\"return await tools.read_image({ file_path: '{{fromRequest:(.+)/red\\.png}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640' })\",\"description\":\"Read the normalized attachment object directly\"}" } }, + { "type": "usage", "usage": { "inputTokens": 3, "outputTokens": 3 } }, + { "type": "finish", "reason": { "kind": "tool-calls" } } + ] + }, + { + "kind": "chunks", + "chunks": [ + { "type": "block-start", "index": 0, "blockType": "text" }, + { "type": "block-end", "index": 0, "block": { "type": "text", "text": "DONE" } }, + { "type": "usage", "usage": { "inputTokens": 3, "outputTokens": 3 } }, + { "type": "finish", "reason": { "kind": "stop" } } + ] + } +] diff --git a/snapshots/session/ptc-read-image-attachment-path/session.jsonl b/snapshots/session/ptc-read-image-attachment-path/session.jsonl new file mode 100644 index 0000000000..09da819ab9 --- /dev/null +++ b/snapshots/session/ptc-read-image-attachment-path/session.jsonl @@ -0,0 +1,48 @@ +{"type":"session","version":0,"id":"{{session:1}}","createdAt":1783954000000,"cwd":"{{cwd}}","delegationDepth":0} +{"type":"permission/preset","data":{"preset":"danger-full-access"}} +{"type":"sandbox/mode","data":{"mode":"danger-full-access"}} +{"type":"approval/policy","data":{"policy":"never"}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Using run_code, call read_image on red.png, then call read_image again with the extension-less normalized attachment object path it produced, and reply with exactly the single word DONE."}],"source":{"kind":"user"},"role":"user","id":"{{message:1}}"}]}} +{"type":"turn/start","data":{"turn":1}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} +{"type":"step/start","data":{"turn":1,"step":1}} +{"type":"user/message","data":{"content":[{"type":"text","text":"Using run_code, call read_image on red.png, then call read_image again with the extension-less normalized attachment object path it produced, and reply with exactly the single word DONE."}],"source":{"kind":"user"},"role":"user","id":"{{message:1}}"},"surfaceOp":"append"} +{"type":"user/message","data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nCurrent DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations.\n\nApproval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt","form":"snapshot","sections":[{"name":"sandbox:policy","text":"Current DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations."},{"name":"approval:policy","text":"Approval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}]},"role":"user","id":"{{message:2}}"},"surfaceOp":"append"} +{"type":"session/title","data":{"title":"Using run_code, call read_image on","messageSeqs":[7],"source":{"kind":"fallback"}}} +{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/context","data":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"code-image-source","name":"run_code","arguments":"{\"code\":\"return await tools.read_image({ file_path: 'red.png' })\",\"description\":\"Read the source image\"}"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"tool-call","id":"code-image-source","name":"run_code","arguments":"{\"code\":\"return await tools.read_image({ file_path: 'red.png' })\",\"description\":\"Read the source image\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:3}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[12,13,14,15],"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":1,"callId":"code-image-source","name":"run_code","arguments":"{\"code\":\"return await tools.read_image({ file_path: 'red.png' })\",\"description\":\"Read the source image\"}"}} +{"type":"tool/code-dispatch-start","data":{"rootCallId":"code-image-source","parentCallId":"code-image-source","subCallId":"code-image-source:code:1","name":"read_image","arguments":{"file_path":"red.png"}}} +{"type":"tool/code-dispatch","data":{"rootCallId":"code-image-source","parentCallId":"code-image-source","subCallId":"code-image-source:code:1","name":"read_image","arguments":{"file_path":"red.png"},"isError":false,"content":[{"type":"text","text":"{{cwd}}/red.png\nimage\n\nimage/png image, 1x1 px, 69 bytes\n"},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","bytes":69,"width":1,"height":1,"name":"red.png"}}]}} +{"type":"tool/result","data":{"turn":1,"step":1,"message":{"source":{"kind":"tool","callId":"code-image-source"},"content":[{"type":"tool-result","toolCallId":"code-image-source","content":[{"type":"text","text":"{\n \"path\": \"{{cwd}}/red.png\",\n \"image\": {\n \"attachmentId\": \"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640\",\n \"mediaType\": \"image/png\",\n \"bytes\": 69,\n \"width\": 1,\n \"height\": 1,\n \"name\": \"red.png\"\n }\n}"}],"isError":false}],"role":"user","id":"{{message:4}}"}},"sourceEventSeqs":[17],"surfaceOp":"append"} +{"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"inserted":[{"content":[{"type":"text","text":"{{cwd}}/red.png\nimage\n\nimage/png image, 1x1 px, 69 bytes\n"},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","bytes":69,"width":1,"height":1,"name":"red.png"}}],"source":{"kind":"plugin","plugin":"tools-code-mode"},"role":"user","id":"{{message:5}}"}]}} +{"type":"step/end","data":{"turn":1,"step":1}} +{"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"removedCount":1,"inserted":[]}} +{"type":"step/start","data":{"turn":1,"step":2}} +{"type":"user/message","data":{"content":[{"type":"text","text":"{{cwd}}/red.png\nimage\n\nimage/png image, 1x1 px, 69 bytes\n"},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","bytes":69,"width":1,"height":1,"name":"red.png"}}],"source":{"kind":"plugin","plugin":"tools-code-mode"},"role":"user","id":"{{message:5}}"},"surfaceOp":"append"} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"code-image-object","name":"run_code","arguments":"{\"code\":\"return await tools.read_image({ file_path: '{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640' })\",\"description\":\"Read the normalized attachment object directly\"}"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"tool-call","id":"code-image-object","name":"run_code","arguments":"{\"code\":\"return await tools.read_image({ file_path: '{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640' })\",\"description\":\"Read the normalized attachment object directly\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:6}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[26,27,28,29],"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":2,"callId":"code-image-object","name":"run_code","arguments":"{\"code\":\"return await tools.read_image({ file_path: '{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640' })\",\"description\":\"Read the normalized attachment object directly\"}"}} +{"type":"tool/code-dispatch-start","data":{"rootCallId":"code-image-object","parentCallId":"code-image-object","subCallId":"code-image-object:code:1","name":"read_image","arguments":{"file_path":"{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640"}}} +{"type":"tool/code-dispatch","data":{"rootCallId":"code-image-object","parentCallId":"code-image-object","subCallId":"code-image-object:code:1","name":"read_image","arguments":{"file_path":"{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640"},"isError":false,"content":[{"type":"text","text":"{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640\nimage\n\nimage/png image, 1x1 px, 69 bytes\n"},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","bytes":69,"width":1,"height":1,"name":"b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640"}}]}} +{"type":"tool/result","data":{"turn":1,"step":2,"message":{"source":{"kind":"tool","callId":"code-image-object"},"content":[{"type":"tool-result","toolCallId":"code-image-object","content":[{"type":"text","text":"{\n \"path\": \"{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640\",\n \"image\": {\n \"attachmentId\": \"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640\",\n \"mediaType\": \"image/png\",\n \"bytes\": 69,\n \"width\": 1,\n \"height\": 1,\n \"name\": \"b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640\"\n }\n}"}],"isError":false}],"role":"user","id":"{{message:7}}"}},"sourceEventSeqs":[31],"surfaceOp":"append"} +{"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"inserted":[{"content":[{"type":"text","text":"{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640\nimage\n\nimage/png image, 1x1 px, 69 bytes\n"},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","bytes":69,"width":1,"height":1,"name":"b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640"}}],"source":{"kind":"plugin","plugin":"tools-code-mode"},"role":"user","id":"{{message:8}}"}]}} +{"type":"step/end","data":{"turn":1,"step":2}} +{"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"removedCount":1,"inserted":[]}} +{"type":"step/start","data":{"turn":1,"step":3}} +{"type":"user/message","data":{"content":[{"type":"text","text":"{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640\nimage\n\nimage/png image, 1x1 px, 69 bytes\n"},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","bytes":69,"width":1,"height":1,"name":"b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640"}}],"source":{"kind":"plugin","plugin":"tools-code-mode"},"role":"user","id":"{{message:8}}"},"surfaceOp":"append"} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"DONE"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","data":{"turn":1,"step":3,"message":{"role":"assistant","content":[{"type":"text","text":"DONE"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:9}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[40,41,42,43],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":3}} +{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/snapshots/session/ptc-read-image-attachment-path/snapshot.yml b/snapshots/session/ptc-read-image-attachment-path/snapshot.yml new file mode 100644 index 0000000000..dc929c458a --- /dev/null +++ b/snapshots/session/ptc-read-image-attachment-path/snapshot.yml @@ -0,0 +1,10 @@ +version: 1 +scenario: ptc-read-image-attachment-path +profile: headless +composition: ptc-image +recording: authored +header: + class: ptc-image +replay: + override: true +platform: posix diff --git a/snapshots/session/ptc-read-image-attachment-path/workspace/red.png b/snapshots/session/ptc-read-image-attachment-path/workspace/red.png new file mode 100644 index 0000000000..62a5f8f47f Binary files /dev/null and b/snapshots/session/ptc-read-image-attachment-path/workspace/red.png differ diff --git a/snapshots/session/ptc-read-image/system-prompt.expected.md b/snapshots/session/ptc-read-image/system-prompt.expected.md index 42106b19ac..7b20dd24e2 100644 --- a/snapshots/session/ptc-read-image/system-prompt.expected.md +++ b/snapshots/session/ptc-read-image/system-prompt.expected.md @@ -156,7 +156,7 @@ interface ToolArgsMap { /** Maximum number of lines to return. Defaults to 2000. */ limit?: number; } & Record; - /** Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input. */ + /** Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input. */ read_image: { /** Path to the image file, resolved by the filesystem backend. */ file_path: string; diff --git a/snapshots/session/ptc-turn/system-prompt.expected.md b/snapshots/session/ptc-turn/system-prompt.expected.md index ed2959eb01..1b5d70f27d 100644 --- a/snapshots/session/ptc-turn/system-prompt.expected.md +++ b/snapshots/session/ptc-turn/system-prompt.expected.md @@ -156,7 +156,7 @@ interface ToolArgsMap { /** Maximum number of lines to return. Defaults to 2000. */ limit?: number; } & Record; - /** Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input. */ + /** Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input. */ read_image: { /** Path to the image file, resolved by the filesystem backend. */ file_path: string; diff --git a/snapshots/session/pty-tools-sandbox-backend/tool-schemas.expected.json b/snapshots/session/pty-tools-sandbox-backend/tool-schemas.expected.json index 144d2309e9..41b07174f1 100644 --- a/snapshots/session/pty-tools-sandbox-backend/tool-schemas.expected.json +++ b/snapshots/session/pty-tools-sandbox-backend/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/ralph-loop/tool-schemas.1.expected.json b/snapshots/session/ralph-loop/tool-schemas.1.expected.json index dbcc5636e5..adf4451608 100644 --- a/snapshots/session/ralph-loop/tool-schemas.1.expected.json +++ b/snapshots/session/ralph-loop/tool-schemas.1.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/ralph-loop/tool-schemas.2.expected.json b/snapshots/session/ralph-loop/tool-schemas.2.expected.json index dbcc5636e5..adf4451608 100644 --- a/snapshots/session/ralph-loop/tool-schemas.2.expected.json +++ b/snapshots/session/ralph-loop/tool-schemas.2.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/read-image-attachment-path/replay.override.json b/snapshots/session/read-image-attachment-path/replay.override.json new file mode 100644 index 0000000000..513c2b63eb --- /dev/null +++ b/snapshots/session/read-image-attachment-path/replay.override.json @@ -0,0 +1,29 @@ +[ + { + "kind": "chunks", + "chunks": [ + { "type": "block-start", "index": 0, "blockType": "tool-call" }, + { "type": "block-end", "index": 0, "block": { "type": "tool-call", "id": "read-image-source", "name": "read_image", "arguments": "{\"file_path\":\"red.png\"}" } }, + { "type": "usage", "usage": { "inputTokens": 3, "outputTokens": 3 } }, + { "type": "finish", "reason": { "kind": "tool-calls" } } + ] + }, + { + "kind": "chunks", + "chunks": [ + { "type": "block-start", "index": 0, "blockType": "tool-call" }, + { "type": "block-end", "index": 0, "block": { "type": "tool-call", "id": "read-image-object", "name": "read_image", "arguments": "{\"file_path\":\"{{fromRequest:(.+)/red\\.png}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640\"}" } }, + { "type": "usage", "usage": { "inputTokens": 3, "outputTokens": 3 } }, + { "type": "finish", "reason": { "kind": "tool-calls" } } + ] + }, + { + "kind": "chunks", + "chunks": [ + { "type": "block-start", "index": 0, "blockType": "text" }, + { "type": "block-end", "index": 0, "block": { "type": "text", "text": "DONE" } }, + { "type": "usage", "usage": { "inputTokens": 3, "outputTokens": 3 } }, + { "type": "finish", "reason": { "kind": "stop" } } + ] + } +] diff --git a/snapshots/session/read-image-attachment-path/session.jsonl b/snapshots/session/read-image-attachment-path/session.jsonl new file mode 100644 index 0000000000..aabb815515 --- /dev/null +++ b/snapshots/session/read-image-attachment-path/session.jsonl @@ -0,0 +1,38 @@ +{"type":"session","version":0,"id":"{{session:1}}","createdAt":1783953000000,"cwd":"{{cwd}}","delegationDepth":0} +{"type":"permission/preset","data":{"preset":"danger-full-access"}} +{"type":"sandbox/mode","data":{"mode":"danger-full-access"}} +{"type":"approval/policy","data":{"policy":"never"}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Use read_image to look at red.png in the current directory, then call read_image again with the extension-less normalized attachment object path under .dsh, and reply with exactly the single word DONE."}],"source":{"kind":"user"},"role":"user","id":"{{message:1}}"}]}} +{"type":"turn/start","data":{"turn":1}} +{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} +{"type":"step/start","data":{"turn":1,"step":1}} +{"type":"user/message","data":{"content":[{"type":"text","text":"Use read_image to look at red.png in the current directory, then call read_image again with the extension-less normalized attachment object path under .dsh, and reply with exactly the single word DONE."}],"source":{"kind":"user"},"role":"user","id":"{{message:1}}"},"surfaceOp":"append"} +{"type":"user/message","data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nCurrent DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations.\n\nApproval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt","form":"snapshot","sections":[{"name":"sandbox:policy","text":"Current DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations."},{"name":"approval:policy","text":"Approval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}]},"role":"user","id":"{{message:2}}"},"surfaceOp":"append"} +{"type":"session/title","data":{"title":"Use read_image to look at","messageSeqs":[7],"source":{"kind":"fallback"}}} +{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/context","data":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"read-image-source","name":"read_image","arguments":"{\"file_path\":\"red.png\"}"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"tool-call","id":"read-image-source","name":"read_image","arguments":"{\"file_path\":\"red.png\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:3}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[12,13,14,15],"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":1,"callId":"read-image-source","name":"read_image","arguments":"{\"file_path\":\"red.png\"}"}} +{"type":"tool/result","data":{"turn":1,"step":1,"message":{"source":{"kind":"tool","callId":"read-image-source"},"content":[{"type":"tool-result","toolCallId":"read-image-source","content":[{"type":"text","text":"{{cwd}}/red.png\nimage\n\nimage/png image, 1x1 px, 69 bytes\n"},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","bytes":69,"width":1,"height":1,"name":"red.png"}}],"isError":false}],"role":"user","id":"{{message:4}}"}},"sourceEventSeqs":[17],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":1}} +{"type":"step/start","data":{"turn":1,"step":2}} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"read-image-object","name":"read_image","arguments":"{\"file_path\":\"{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640\"}"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"tool-call","id":"read-image-object","name":"read_image","arguments":"{\"file_path\":\"{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:5}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[21,22,23,24],"surfaceOp":"append"} +{"type":"tool/call","data":{"turn":1,"step":2,"callId":"read-image-object","name":"read_image","arguments":"{\"file_path\":\"{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640\"}"}} +{"type":"tool/result","data":{"turn":1,"step":2,"message":{"source":{"kind":"tool","callId":"read-image-object"},"content":[{"type":"tool-result","toolCallId":"read-image-object","content":[{"type":"text","text":"{{cwd}}/.dsh/attachments/v1/objects/b1/b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640\nimage\n\nimage/png image, 1x1 px, 69 bytes\n"},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","bytes":69,"width":1,"height":1,"name":"b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640"}}],"isError":false}],"role":"user","id":"{{message:6}}"}},"sourceEventSeqs":[26],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":2}} +{"type":"step/start","data":{"turn":1,"step":3}} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"DONE"}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","data":{"turn":1,"step":3,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","data":{"turn":1,"step":3,"message":{"role":"assistant","content":[{"type":"text","text":"DONE"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:7}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[30,31,32,33],"surfaceOp":"append"} +{"type":"step/end","data":{"turn":1,"step":3}} +{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/snapshots/session/read-image-attachment-path/snapshot.yml b/snapshots/session/read-image-attachment-path/snapshot.yml new file mode 100644 index 0000000000..724699b789 --- /dev/null +++ b/snapshots/session/read-image-attachment-path/snapshot.yml @@ -0,0 +1,9 @@ +version: 1 +scenario: read-image-attachment-path +profile: headless +composition: image +recording: authored +header: + class: image +replay: + override: true diff --git a/snapshots/session/read-image-attachment-path/workspace/red.png b/snapshots/session/read-image-attachment-path/workspace/red.png new file mode 100644 index 0000000000..62a5f8f47f Binary files /dev/null and b/snapshots/session/read-image-attachment-path/workspace/red.png differ diff --git a/snapshots/session/session-query-spill/tool-schemas.expected.json b/snapshots/session/session-query-spill/tool-schemas.expected.json index 3e28d4cfcb..929cd0fa8d 100644 --- a/snapshots/session/session-query-spill/tool-schemas.expected.json +++ b/snapshots/session/session-query-spill/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/subagent-acp-diagnostic/tool-schemas.expected.json b/snapshots/session/subagent-acp-diagnostic/tool-schemas.expected.json index 796bac1779..ab4af3db64 100644 --- a/snapshots/session/subagent-acp-diagnostic/tool-schemas.expected.json +++ b/snapshots/session/subagent-acp-diagnostic/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/subagent-child-question-rejection/tool-schemas.expected.json b/snapshots/session/subagent-child-question-rejection/tool-schemas.expected.json index 7e4dfe696c..4b6b5b10f4 100644 --- a/snapshots/session/subagent-child-question-rejection/tool-schemas.expected.json +++ b/snapshots/session/subagent-child-question-rejection/tool-schemas.expected.json @@ -369,7 +369,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/text-turn/tool-schemas.expected.json b/snapshots/session/text-turn/tool-schemas.expected.json index c93fb6d65f..df890396ac 100644 --- a/snapshots/session/text-turn/tool-schemas.expected.json +++ b/snapshots/session/text-turn/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/session/web-fetch/tool-schemas.expected.json b/snapshots/session/web-fetch/tool-schemas.expected.json index ee18836349..5381c3bc24 100644 --- a/snapshots/session/web-fetch/tool-schemas.expected.json +++ b/snapshots/session/web-fetch/tool-schemas.expected.json @@ -306,7 +306,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/web/cordis-tool-round/tool-schemas.expected.json b/snapshots/web/cordis-tool-round/tool-schemas.expected.json index b558e1d094..4b21fbb5b5 100644 --- a/snapshots/web/cordis-tool-round/tool-schemas.expected.json +++ b/snapshots/web/cordis-tool-round/tool-schemas.expected.json @@ -566,7 +566,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/web/fresh-round-trip/tool-schemas.expected.json b/snapshots/web/fresh-round-trip/tool-schemas.expected.json index 8232bc9e23..c07e7ea3f3 100644 --- a/snapshots/web/fresh-round-trip/tool-schemas.expected.json +++ b/snapshots/web/fresh-round-trip/tool-schemas.expected.json @@ -369,7 +369,7 @@ }, { "name": "read_image", - "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", + "description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.", "parameters": { "type": "object", "properties": { diff --git a/snapshots/web/ptc-round/system-prompt.expected.md b/snapshots/web/ptc-round/system-prompt.expected.md index 6cda9dc20d..b74423d3c7 100644 --- a/snapshots/web/ptc-round/system-prompt.expected.md +++ b/snapshots/web/ptc-round/system-prompt.expected.md @@ -184,7 +184,7 @@ interface ToolArgsMap { /** Maximum number of lines to return. Defaults to 2000. */ limit?: number; } & Record; - /** Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input. */ + /** Read a PNG/JPEG/WebP/GIF file and return the image itself. A path without a file extension is accepted; the format is detected from the file content, so normalized attachment paths can be passed directly without copying or renaming. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input. */ read_image: { /** Path to the image file, resolved by the filesystem backend. */ file_path: string; diff --git a/snapshots/web/queued-image/delivered.expected.md b/snapshots/web/queued-image/delivered.expected.md new file mode 100644 index 0000000000..b5dc457a4e --- /dev/null +++ b/snapshots/web/queued-image/delivered.expected.md @@ -0,0 +1,88 @@ +- banner: + - navigation "Session hierarchy": + - button "Reply with a one-sentence description" [disabled] + - img + - text: Standard mode + - button "Session log": + - text: Session log + - img + - tablist: + - tab "Chat" [selected] + - tab "Trajectory" +- navigation "Turn navigation": + - button "Jump to turn 1" + - button "Jump to turn 2" + - button "Jump to turn 3" +- button "System prompt": + - img + - img + - text: System prompt +- text: Reply with a one-sentence description of event sourcing, then stop. {{clock}} +- button "Copy": + - img +- button "Thought for a while": + - text: Thought for a while + - img +- paragraph: partial +- text: Stopped +- button "Copy": + - img +- button "Good response": + - img +- button "Bad response": + - img +- button "Branch into a new conversation": + - img +- button "Ran for {{duration}}": + - img + - text: Ran for {{duration}} +- text: {{clock}} +- button "queued.png, click to view original": + - img "queued.png" +- text: Compare with this screenshot {{clock}} +- button "Copy": + - img +- button "Thought for a while": + - text: Thought for a while + - img +- paragraph: Event sourcing is a pattern where all changes to an application's state are stored as an immutable, append-only sequence of events, rather than persisting only the current state, enabling full auditability, temporal queries, and event-driven architectures. +- button "Copy": + - img +- button "Good response": + - img +- button "Bad response": + - img +- button "Branch into a new conversation": + - img +- button "Ran for {{duration}}": + - img + - text: Ran for {{duration}} +- text: {{clock}} Continue with the queued comparison {{clock}} +- button "Copy": + - img +- button "Thought for a while": + - text: Thought for a while + - img +- paragraph: Event sourcing is a pattern where all changes to an application's state are stored as an immutable, append-only sequence of events, rather than persisting only the current state, enabling full auditability, temporal queries, and event-driven architectures. +- button "Copy": + - img +- button "Good response": + - img +- button "Bad response": + - img +- button "Branch into a new conversation": + - img +- button "Ran for {{duration}}": + - img + - text: Ran for {{duration}} +- text: {{clock}} +- textbox "Message or run a task... / commands, @ files or sessions" +- button "Commands": + - img +- 'button "Access mode, current: Workspace Write"': Workspace Write +- button "Select model, current DeepSeek-V4-Flash": + - text: DeepSeek-V4-Flash + - img +- button "6% of context used" +- button "Send message" [disabled] +- text: 3 turns · 3 steps LLM {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 99% Input 15.6K tok · Output 158 tok diff --git a/snapshots/web/queued-image/queued.expected.md b/snapshots/web/queued-image/queued.expected.md new file mode 100644 index 0000000000..462b6d03d1 --- /dev/null +++ b/snapshots/web/queued-image/queued.expected.md @@ -0,0 +1,42 @@ +- banner: + - navigation "Session hierarchy": + - button "Reply with a one-sentence description" [disabled] + - img + - text: Standard mode + - button "Session log": + - text: Session log + - img + - tablist: + - tab "Chat" [selected] + - tab "Trajectory" +- button "System prompt": + - img + - img + - text: System prompt +- text: Reply with a one-sentence description of event sourcing, then stop. {{clock}} +- button "Copy": + - img +- button "Context injection @deepseek-ai/dsh-system-prompt": + - img + - img + - text: Context injection @deepseek-ai/dsh-system-prompt +- paragraph: partial +- status: Deep diving... +- list: + - listitem: + - img "Queued message image" + - text: Compare with this screenshot + - button "Edit queued message" [disabled]: + - img + - button "Remove queued message": + - img + - button "Steer queued message": + - img +- textbox "Cmd/Ctrl+Enter steers all queued messages" +- button "Commands": + - img +- 'button "Access mode, current: Workspace Write"': Workspace Write +- button "Select model, current DeepSeek-V4-Flash": + - text: DeepSeek-V4-Flash + - img +- button "Stop generating" diff --git a/snapshots/web/queued-image/snapshot.yml b/snapshots/web/queued-image/snapshot.yml new file mode 100644 index 0000000000..69c492d2b2 --- /dev/null +++ b/snapshots/web/queued-image/snapshot.yml @@ -0,0 +1,9 @@ +version: 1 +scenario: queued-image +profile: web +composition: web-default +recording: authored +header: + class: web-default +session: + source: ../live-interactions/session.jsonl diff --git a/tsconfig.host.json b/tsconfig.host.json index f816664290..76b512521f 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -60,6 +60,7 @@ "apps/web/tests/markdown-cjk-strong.e2e.ts", "apps/web/tests/markdown-inline-code-links.e2e.ts", "apps/web/tests/queue-actions.e2e.ts", + "apps/web/tests/queue-image.e2e.ts", "apps/web/tests/skill-invocation-policy.e2e.ts", "apps/web/tests/skill-user-invoke.e2e.ts", "apps/web/tests/permission-policy-context.e2e.ts",