fix(llm): keep streamed tool-call identity across empty deltas
A continuation SSE delta that repeats a tool call's `id` or `name` as an empty string — or as `null`, which some OpenAI-compatible gateways send — erased the identity established by the call's first delta. The assembled block reached the loop with an empty name and failed as `unknown tool ""`, and the empty `callId` persisted into `tool/result`, which the session reader refuses on reopen. `acceptIdentity` accepts only a non-empty string, so a repeated empty or null field means "no update". A tool call still missing `id` or `name` at `[DONE]` ends the response with the new retryable `MALFORMED_TOOL_CALL` code instead of closing an unusable block.
This commit is contained in:
Yichen Jiang
parent
3efd4b51e0
commit
a1271a4903
16 changed files with 276 additions and 41 deletions
|
|
@ -0,0 +1,6 @@
|
|||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-01-streamed-tool-call-identity.md
|
||||
2026-09-01-streamed-tool-call-identity.md: a59337bc8f4d0126584822cc2e01155d377b5603
|
||||
2026-09-01-streamed-tool-call-identity.zh.md: 5185377adf9e8fb366dab3934c10f87e22be7830
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
# Agent Note: Streamed tool-call identity survives empty continuation deltas
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-09-01-streamed-tool-call-identity.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The DeepSeek SSE translator assigned `id` and `name` on every tool-call delta that carried the field, so a continuation delta repeating either as an empty string erased the identity established by the call's first delta. The assembled block reached the loop with an empty name, which the tool registry refuses as `unknown tool ""`. Gateways that fill those fields with `null` erased the identity the same way, and `WireToolCallDelta` declared both as `string | undefined`, keeping the observed `null` out of the compiler's reach.
|
||||
|
||||
The empty identity outlived the turn. `appendToolCall` and `appendToolResult` write the block's id verbatim and no write path validates it, while `adoptSessionEvent` refuses a `tool/result` whose `callId` is empty. A session that recorded one such call was writable and no longer loadable: the persistence coordinator wrapped that refusal in `SessionPersistenceCorruptionError`.
|
||||
|
||||
## Decision
|
||||
|
||||
`acceptIdentity` accepts only a non-empty string for a tool call's `id` and `name`; `undefined`, `null`, `''`, and any non-string leave the established value in place. `WireToolCallDelta` widens `id`, `function.name`, and `function.arguments` to admit `null`, so the values gateways actually send are in the type system and the runtime guard is load-bearing rather than speculative.
|
||||
|
||||
A tool call still lacking `id` or `name` when the stream reaches `[DONE]` is not closed. The translator reports any pending usage, then ends the response with an error finish carrying the new `MALFORMED_TOOL_CALL` code, and emits no `block-end` at all. `closeBlock` returns which field is missing instead of substituting an empty string, so no path can assemble an unidentified tool call.
|
||||
|
||||
`MALFORMED_TOOL_CALL` joins the default retryable codes. The failure must arrive as an error `finish` rather than a thrown `LlmError`: the agent loop derives `agent/request-error` — the only extension point `dsh-llm-retry` listens on — from `BlockAssembler.finish`, and rethrows whatever the stream throws straight out of the turn. A thrown failure ends the turn with no retry whatever the policy says.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Concatenate `id` and `name` across deltas.** Rejected: they are identity, not accumulation. Concatenation produces `Globnull` against a gateway that sends `null`, and a doubled name against one that repeats a non-empty value.
|
||||
|
||||
**Refuse a conflicting non-empty identity mid-stream.** Deferred: a gateway that fragments a long tool name would be refused for it, and the `[DONE]` check already keeps an unusable call away from the loop.
|
||||
|
||||
**Relax the session reader's empty-`callId` refusal.** Rejected: an empty `callId` cannot be paired back to the provider on the next request, so accepting it moves the failure into the model request. That refusal is the durable-boundary gate; the producer was the defect.
|
||||
|
||||
**Refuse as soon as a call's first delta carries no `id`.** Rejected: "first delta only" is a claim about the remote encoder, so a gateway that sends `id` one delta later would be refused for nothing, and the `[DONE]` check covers every case an early check would.
|
||||
|
||||
## Consequences
|
||||
|
||||
A continuation delta repeating identity empty or null is inert, so a call keeps the identity its first delta established. A provider that never identifies a call costs a retry instead of an `unknown tool ""` result and a session that cannot be reopened. Sessions that already recorded an empty `callId` stay unreadable; recovering them is outside this change.
|
||||
|
||||
## Testing
|
||||
|
||||
`translate.spec.ts` covers empty and null continuation deltas, a repeated identical identity, parallel calls holding separate identities under empty continuations, and refusal when `id` or `name` never arrives — including that usage is reported before the failure and that no `block-end` precedes it. The two cases that documented lenient empty-identity output now assert refusal. `retry-policy.spec.ts` pins the new default retryable set.
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
# Agent Note：流式工具调用身份不被空续传分片抹除
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-09-01-streamed-tool-call-identity.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
DeepSeek SSE 翻译器对每个携带该字段的工具调用分片都直接赋值 `id` 与 `name`，因此续传分片把其中任一字段重复发送为空串时，会抹掉该调用首个分片已建立的身份。组装出的块带着空名字进入循环，工具注册表以 `unknown tool ""` 拒绝它。把这些字段填成 `null` 的网关会造成同样的抹除，而 `WireToolCallDelta` 把两者都声明为 `string | undefined`，让实际观察到的 `null` 落在编译器视野之外。
|
||||
|
||||
空身份还会活过本轮。`appendToolCall` 与 `appendToolResult` 原样写入块的 id 且没有任何写入路径校验它，而 `adoptSessionEvent` 拒绝 `callId` 为空的 `tool/result`。记录过一次这种调用的会话可写但不再可读：持久化协调器把该拒绝包装成 `SessionPersistenceCorruptionError`。
|
||||
|
||||
## 决定
|
||||
|
||||
`acceptIdentity` 对工具调用的 `id` 与 `name` 只接受非空字符串；`undefined`、`null`、`''` 以及任何非字符串都保留已建立的值。`WireToolCallDelta` 把 `id`、`function.name` 与 `function.arguments` 放宽到允许 `null`，使网关实际发送的值进入类型系统，运行时守卫因此是承重的而非臆测的。
|
||||
|
||||
流到达 `[DONE]` 时仍缺少 `id` 或 `name` 的工具调用不会被闭合。翻译器先报告待发的用量，再以携带新增 `MALFORMED_TOOL_CALL` code 的错误 finish 结束响应，并且完全不发出 `block-end`。`closeBlock` 返回缺失的是哪个字段，而不是替换成空串，因此没有任何路径能组装出无身份的工具调用。
|
||||
|
||||
`MALFORMED_TOOL_CALL` 加入默认可重试 code 集。该失败必须以错误 `finish` 而非抛出的 `LlmError` 抵达：agent 循环从 `BlockAssembler.finish` 派生 `agent/request-error`——`dsh-llm-retry` 唯一监听的扩展点——并把流抛出的任何东西直接重抛出本轮。抛出的失败无论策略如何都会终结本轮且不重试。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**跨分片拼接 `id` 与 `name`。** 否决：它们是身份而非累积。面对发送 `null` 的网关，拼接产生 `Globnull`；面对重复发送非空值的网关，产生重复的名字。
|
||||
|
||||
**流中途拒绝冲突的非空身份。** 推迟：分片发送长工具名的网关会因此被拒，而 `[DONE]` 处的检查已经能让不可用的调用到不了循环。
|
||||
|
||||
**放宽会话读取端对空 `callId` 的拒绝。** 否决：空 `callId` 无法在下一次请求中与提供方配对，接受它只是把失败推进模型请求。该拒绝是持久化边界的闸门；缺陷在生产方。
|
||||
|
||||
**在调用的首个分片不带 `id` 时立即拒绝。** 否决："仅首个分片"是对远端编码器的声明，晚一个分片才发送 `id` 的网关会被无谓拒绝，而 `[DONE]` 处的检查覆盖了早检查能覆盖的全部情况。
|
||||
|
||||
## 后果
|
||||
|
||||
重复发送空或 null 身份的续传分片不产生作用，调用因此保有其首个分片建立的身份。始终不给出调用身份的提供方现在的代价是一次重试，而不是一个 `unknown tool ""` 结果加一个无法重新打开的会话。已经记录了空 `callId` 的会话仍不可读；恢复它们不在本次改动范围内。
|
||||
|
||||
## 测试
|
||||
|
||||
`translate.spec.ts` 覆盖空与 null 续传分片、重复的相同身份、空续传下并行调用各自保有身份，以及 `id` 或 `name` 始终未抵达时的拒绝——包括失败前先报告用量、且其前没有 `block-end`。原先记录空身份输出的两个用例现在断言拒绝。`retry-policy.spec.ts` 钉住新的默认可重试集。
|
||||
|
|
@ -2,5 +2,5 @@
|
|||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
|
||||
README.md: 58639be34cb116f3103f579b277504818cb82278
|
||||
README.zh.md: d5a5be0847bc8f3ad069e7bb1fff3a42e1662e5d
|
||||
README.md: 9ec80aa2c0f2190fa691bdf28a6e0ed07c805c8a
|
||||
README.zh.md: fe2b39059760854c1df16390233158e3432b65d6
|
||||
|
|
|
|||
|
|
@ -92,7 +92,7 @@ When `ctx.deepseekLlmApiExtensions` is present, the adapter prepares its registe
|
|||
|
||||
### Failures and recovery
|
||||
|
||||
Non-2xx responses fail with stable codes: `AUTH` (401/403), `QUOTA`, `RATE_LIMIT`, `CONTEXT_WINDOW_EXCEEDED`, `INVALID_REQUEST`, `SERVER`, and `HTTP_<status>` otherwise; pre-response transport failures throw `TRANSPORT`, caller aborts throw `ABORTED`, and stream-idle expiry throws `TIMEOUT`. Request-extension preparation, field collision, or post-2xx acceptance fails with `REQUEST_EXTENSION`. A normalized-image rejection names every plausible attachment and its durable position when the provider does not identify a file id. Stale-file rejection invalidates the named mappings (or every mapping used by the attempt) and permits one replacement chat attempt. Protocol violations throw `STREAM_CLOSED` or `MALFORMED_RESPONSE`, and a terminal `stop` with no content blocks becomes `EMPTY_RESPONSE`, which the default retry policy retries. A request with no key anywhere fails with `MISSING_CREDENTIAL`, and a malformed credential fails with `INVALID_CREDENTIAL` naming the reference to fix — never any part of the key.
|
||||
Non-2xx responses fail with stable codes: `AUTH` (401/403), `QUOTA`, `RATE_LIMIT`, `CONTEXT_WINDOW_EXCEEDED`, `INVALID_REQUEST`, `SERVER`, and `HTTP_<status>` otherwise; pre-response transport failures throw `TRANSPORT`, caller aborts throw `ABORTED`, and stream-idle expiry throws `TIMEOUT`. Request-extension preparation, field collision, or post-2xx acceptance fails with `REQUEST_EXTENSION`. A normalized-image rejection names every plausible attachment and its durable position when the provider does not identify a file id. Stale-file rejection invalidates the named mappings (or every mapping used by the attempt) and permits one replacement chat attempt. Protocol violations throw `STREAM_CLOSED` or `MALFORMED_RESPONSE`, a terminal `stop` with no content blocks becomes `EMPTY_RESPONSE`, and a streamed tool call still missing its `id` or `name` when the stream ends becomes `MALFORMED_TOOL_CALL`; the default retry policy retries the last two. A request with no key anywhere fails with `MISSING_CREDENTIAL`, and a malformed credential fails with `INVALID_CREDENTIAL` naming the reference to fix — never any part of the key.
|
||||
|
||||
-----
|
||||
|
||||
|
|
@ -117,7 +117,7 @@ The plugin is built on one explicit resolve step and one registration fact. `res
|
|||
| [`src/file-store.ts`](src/file-store.ts) + [`src/files-api.ts`](src/files-api.ts) | Scoped upload caching, expiry, stale-id recovery, quota cleanup, and remote file operations |
|
||||
| [`src/serialize.ts`](src/serialize.ts) | Wire serialization: thinking defaults, Files or inline image blocks, history rules |
|
||||
| [`src/sse.ts`](src/sse.ts) | `eventsource-parser` SSE framing for the direct `fetch` stream |
|
||||
| [`src/translate.ts`](src/translate.ts) | SSE payload translation into harness `StreamChunk` values |
|
||||
| [`src/translate.ts`](src/translate.ts) | SSE payload translation into harness `StreamChunk` values; tool-call `id` and `name` are identity, so a continuation delta repeating them empty or null leaves the established value alone |
|
||||
| [`src/types.ts`](src/types.ts) | Wire-level types shared by the modules above |
|
||||
|
||||
### Wire flow
|
||||
|
|
|
|||
|
|
@ -92,7 +92,7 @@ Files 模式通过 `maxRequestFilesBytes` 与 `maxImagesPerRequest` 限制保留
|
|||
|
||||
### 失败与恢复
|
||||
|
||||
非 2xx 响应以稳定 code 失败：`AUTH`（401/403）、`QUOTA`、`RATE_LIMIT`、`CONTEXT_WINDOW_EXCEEDED`、`INVALID_REQUEST`、`SERVER` 以及其他情况的 `HTTP_<status>`；响应前传输失败抛出 `TRANSPORT`，调用方中止抛出 `ABORTED`，流空闲超时抛出 `TIMEOUT`。请求扩展准备、字段冲突或 2xx 后接受失败使用 `REQUEST_EXTENSION`。当提供方未指出 file id 时，规范化图片拒绝会列出所有可能附件及其持久位置。陈旧文件拒绝会使点名映射（或该次尝试使用的全部映射）失效，并允许一次替换 chat 尝试。协议违规抛出 `STREAM_CLOSED` 或 `MALFORMED_RESPONSE`；不带内容块的终止 `stop` 变成 `EMPTY_RESPONSE`，默认重试策略会重试它。任何位置都没有密钥的请求以 `MISSING_CREDENTIAL` 失败；格式错误的凭据以 `INVALID_CREDENTIAL` 失败，并点名需要修复的引用——绝不包含密钥的任何部分。
|
||||
非 2xx 响应以稳定 code 失败：`AUTH`（401/403）、`QUOTA`、`RATE_LIMIT`、`CONTEXT_WINDOW_EXCEEDED`、`INVALID_REQUEST`、`SERVER` 以及其他情况的 `HTTP_<status>`；响应前传输失败抛出 `TRANSPORT`，调用方中止抛出 `ABORTED`，流空闲超时抛出 `TIMEOUT`。请求扩展准备、字段冲突或 2xx 后接受失败使用 `REQUEST_EXTENSION`。当提供方未指出 file id 时，规范化图片拒绝会列出所有可能附件及其持久位置。陈旧文件拒绝会使点名映射（或该次尝试使用的全部映射）失效，并允许一次替换 chat 尝试。协议违规抛出 `STREAM_CLOSED` 或 `MALFORMED_RESPONSE`；不带内容块的终止 `stop` 变成 `EMPTY_RESPONSE`，流结束时仍缺少 `id` 或 `name` 的流式工具调用变成 `MALFORMED_TOOL_CALL`，默认重试策略会重试后两者。任何位置都没有密钥的请求以 `MISSING_CREDENTIAL` 失败；格式错误的凭据以 `INVALID_CREDENTIAL` 失败，并点名需要修复的引用——绝不包含密钥的任何部分。
|
||||
|
||||
-----
|
||||
|
||||
|
|
@ -117,7 +117,7 @@ Files 模式通过 `maxRequestFilesBytes` 与 `maxImagesPerRequest` 限制保留
|
|||
| [`src/file-store.ts`](src/file-store.ts) + [`src/files-api.ts`](src/files-api.ts) | 限定作用域的上传缓存、到期、陈旧 id 恢复、配额清理与远程文件操作 |
|
||||
| [`src/serialize.ts`](src/serialize.ts) | 协议序列化：thinking 默认值、Files 或内联图片块、历史规则 |
|
||||
| [`src/sse.ts`](src/sse.ts) | 直接 `fetch` 流的 `eventsource-parser` SSE 分帧 |
|
||||
| [`src/translate.ts`](src/translate.ts) | 把 SSE 载荷翻译为 harness `StreamChunk` 值 |
|
||||
| [`src/translate.ts`](src/translate.ts) | 把 SSE 载荷翻译为 harness `StreamChunk` 值；工具调用的 `id` 与 `name` 是身份，后续分片重复发送空串或 null 时保留已建立的值 |
|
||||
| [`src/types.ts`](src/types.ts) | 上述模块共享的协议级类型 |
|
||||
|
||||
### 协议流程
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
*/
|
||||
|
||||
import { brandString } from '@deepseek-ai/dsh-brand'
|
||||
import { EMPTY_RESPONSE_CODE, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { EMPTY_RESPONSE_CODE, LlmError, MALFORMED_TOOL_CALL_CODE } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, FinishReason, StreamChunk, TokenUsage, ToolCallId } from '@deepseek-ai/dsh-llm'
|
||||
import { DONE } from './sse.ts'
|
||||
import type { WireChunk, WireUsage } from './types.ts'
|
||||
|
|
@ -19,9 +19,9 @@ interface OpenBlock {
|
|||
index: number
|
||||
kind: 'text' | 'reasoning' | 'tool-call'
|
||||
text: string
|
||||
/** tool-call only */
|
||||
callId?: string
|
||||
name?: string
|
||||
/** tool-call only, absent until a delta carries a non-empty value. */
|
||||
callId?: string | undefined
|
||||
name?: string | undefined
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -71,16 +71,43 @@ export function mapUsage(usage: WireUsage): TokenUsage {
|
|||
}
|
||||
}
|
||||
|
||||
/** Assemble the final ContentBlock for one open block. */
|
||||
function closeBlock(block: OpenBlock): ContentBlock {
|
||||
/**
|
||||
* Accept one streamed identity field for a tool call. `id` and `name` are
|
||||
* identity, not accumulation: the wire sends each once, on the call's first
|
||||
* delta. A continuation delta that re-sends the field empty — or `null`, which
|
||||
* some OpenAI-compatible gateways fill in — means "no update", never "clear".
|
||||
* @param current - the identity established by an earlier delta of this call.
|
||||
* @param incoming - the field as parsed from this delta. The wire type is a
|
||||
* claim about a remote encoder, so anything but a non-empty string leaves the
|
||||
* established value alone rather than overwriting it.
|
||||
* @returns the identity in force after this delta.
|
||||
*/
|
||||
function acceptIdentity(current: string | undefined, incoming: unknown): string | undefined {
|
||||
return typeof incoming === 'string' && incoming.length > 0 ? incoming : current
|
||||
}
|
||||
|
||||
/** The identity field a streamed tool call never received. */
|
||||
interface Unidentified {
|
||||
unidentified: 'id' | 'name'
|
||||
}
|
||||
|
||||
/**
|
||||
* Assemble the final ContentBlock for one open block.
|
||||
* @param block - the block to close.
|
||||
* @returns the assembled block, or which identity field a tool call is missing.
|
||||
* A tool call without both fields cannot be dispatched, and its result cannot
|
||||
* be paired back to the provider, so the caller rejects the whole response
|
||||
* rather than closing the block.
|
||||
*/
|
||||
function closeBlock(block: OpenBlock): ContentBlock | Unidentified {
|
||||
switch (block.kind) {
|
||||
case 'text': return { type: 'text', text: block.text }
|
||||
case 'reasoning': return { type: 'reasoning', text: block.text }
|
||||
case 'tool-call': return {
|
||||
type: 'tool-call',
|
||||
id: brandString<ToolCallId>(block.callId ?? ''),
|
||||
name: block.name ?? '',
|
||||
arguments: block.text,
|
||||
case 'tool-call': {
|
||||
const { callId, name } = block
|
||||
if (callId === undefined) return { unidentified: 'id' }
|
||||
if (name === undefined) return { unidentified: 'name' }
|
||||
return { type: 'tool-call', id: brandString<ToolCallId>(callId), name, arguments: block.text }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -91,7 +118,9 @@ function closeBlock(block: OpenBlock): ContentBlock {
|
|||
* @param payloads - SSE data payloads from {@link parseSse}, `[DONE]`-terminated.
|
||||
* @returns deltas as they arrive; `block-end`s, `usage`, and `finish` are all deferred to the `[DONE]` sentinel.
|
||||
* A `stop` (or absent) finish with no opened blocks is a degenerate provider completion and maps to an
|
||||
* `EMPTY_RESPONSE` error finish instead of a successful empty message.
|
||||
* `EMPTY_RESPONSE` error finish instead of a successful empty message. A tool call left without an `id` or
|
||||
* `name` is unusable, so the response ends in a `MALFORMED_TOOL_CALL` error finish, after any usage and
|
||||
* without a `block-end` for any block.
|
||||
*/
|
||||
export async function* translate(payloads: AsyncIterable<string>): AsyncGenerator<StreamChunk> {
|
||||
let nextIndex = 0
|
||||
|
|
@ -110,9 +139,28 @@ export async function* translate(payloads: AsyncIterable<string>): AsyncGenerato
|
|||
|
||||
for await (const payload of payloads) {
|
||||
if (payload === DONE) {
|
||||
const ends: StreamChunk[] = []
|
||||
for (const block of order) {
|
||||
yield { type: 'block-end', index: block.index, block: closeBlock(block) }
|
||||
const closed = closeBlock(block)
|
||||
if ('unidentified' in closed) {
|
||||
// Nothing durable is written for a rejected response, so the usage
|
||||
// the attempt already burned is still reported before the failure.
|
||||
if (pendingUsage) yield { type: 'usage', usage: pendingUsage }
|
||||
yield {
|
||||
type: 'finish',
|
||||
reason: {
|
||||
kind: 'error',
|
||||
failure: {
|
||||
message: `model streamed a tool call with no ${closed.unidentified}`,
|
||||
code: MALFORMED_TOOL_CALL_CODE,
|
||||
},
|
||||
},
|
||||
}
|
||||
return
|
||||
}
|
||||
ends.push({ type: 'block-end', index: block.index, block: closed })
|
||||
}
|
||||
yield* ends
|
||||
if (pendingUsage) yield { type: 'usage', usage: pendingUsage }
|
||||
const reason = pendingFinish ?? { kind: 'stop' as const }
|
||||
yield {
|
||||
|
|
@ -166,8 +214,8 @@ export async function* translate(payloads: AsyncIterable<string>): AsyncGenerato
|
|||
toolBlocks.set(call.index, block)
|
||||
yield { type: 'block-start', index: block.index, blockType: 'tool-call' }
|
||||
}
|
||||
if (call.id !== undefined) block.callId = call.id
|
||||
if (call.function?.name !== undefined) block.name = call.function.name
|
||||
block.callId = acceptIdentity(block.callId, call.id)
|
||||
block.name = acceptIdentity(block.name, call.function?.name)
|
||||
const fragment = call.function?.arguments ?? ''
|
||||
block.text += fragment
|
||||
yield {
|
||||
|
|
|
|||
|
|
@ -145,14 +145,17 @@ export interface WireDelta {
|
|||
export interface WireToolCallDelta {
|
||||
/** Disambiguates parallel tool calls; stable across a call's deltas. */
|
||||
index: number
|
||||
/** Present on the first delta of each call only. */
|
||||
id?: string
|
||||
/**
|
||||
* Carried by the first delta of each call. Gateways observed in the wild
|
||||
* repeat it on continuation deltas as `''` or `null`; both mean "unchanged".
|
||||
*/
|
||||
id?: string | null
|
||||
type?: 'function'
|
||||
function?: {
|
||||
/** Present on the first delta of each call only. */
|
||||
name?: string
|
||||
/** Carried by the first delta of each call, with the same `''`/`null` repetition as {@link WireToolCallDelta.id}. */
|
||||
name?: string | null
|
||||
/** Argument JSON fragment (concatenate across deltas). */
|
||||
arguments?: string
|
||||
arguments?: string | null
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
import { describe, expect, it } from 'vitest'
|
||||
import { BlockAssembler, EMPTY_RESPONSE_CODE, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { BlockAssembler, EMPTY_RESPONSE_CODE, LlmError, MALFORMED_TOOL_CALL_CODE } from '@deepseek-ai/dsh-llm'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { DONE } from '../src/sse.ts'
|
||||
import { mapFinishReason, mapUsage, translate } from '../src/translate.ts'
|
||||
|
|
@ -328,19 +328,24 @@ describe('mapUsage', () => {
|
|||
})
|
||||
|
||||
describe('translate: defensive tool-call branches', () => {
|
||||
it('handles deltas that never carry id or name (empty-string fallbacks)', async () => {
|
||||
it('rejects a stream whose tool call never carries an id, reporting usage first', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
// Hypothetical lenient wire: argument fragments with no id/name at all.
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, function: { arguments: '{}' } }] } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'tool_calls' }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'tool_calls' }], usage: { prompt_tokens: 12, completion_tokens: 4 } },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks).toEqual([
|
||||
{ type: 'block-start', index: 0, blockType: 'tool-call' },
|
||||
{ type: 'tool-call-delta', index: 0, id: '', argumentsDelta: '{}' },
|
||||
{ type: 'block-end', index: 0, block: { type: 'tool-call', id: '', name: '', arguments: '{}' } },
|
||||
{ type: 'finish', reason: { kind: 'tool-calls' } },
|
||||
{ type: 'usage', usage: { inputTokens: 12, outputTokens: 4, totalTokens: 16 } },
|
||||
{
|
||||
type: 'finish',
|
||||
reason: {
|
||||
kind: 'error',
|
||||
failure: { message: 'model streamed a tool call with no id', code: MALFORMED_TOOL_CALL_CODE },
|
||||
},
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
|
|
@ -354,7 +359,7 @@ describe('translate: defensive tool-call branches', () => {
|
|||
expect(chunks[1]).toEqual({ type: 'tool-call-delta', index: 0, id: 'c', name: 'f', argumentsDelta: '' })
|
||||
})
|
||||
|
||||
it('handles tool_call deltas with no function object at all', async () => {
|
||||
it('rejects a stream whose tool call never carries a name', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: 'c' }] } }] },
|
||||
|
|
@ -362,5 +367,93 @@ describe('translate: defensive tool-call branches', () => {
|
|||
DONE,
|
||||
)))
|
||||
expect(chunks[1]).toEqual({ type: 'tool-call-delta', index: 0, id: 'c', argumentsDelta: '' })
|
||||
expect(chunks.some(chunk => chunk.type === 'block-end')).toBe(false)
|
||||
expect(chunks.at(-1)).toEqual({
|
||||
type: 'finish',
|
||||
reason: {
|
||||
kind: 'error',
|
||||
failure: { message: 'model streamed a tool call with no name', code: MALFORMED_TOOL_CALL_CODE },
|
||||
},
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('translate: tool-call identity across deltas', () => {
|
||||
it('keeps the established identity when continuation deltas re-send it empty', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: 'call_00_x', type: 'function', function: { name: 'get_weather', arguments: '' } }] } }] },
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: '', type: 'function', function: { name: '', arguments: '{"city"' } }] } }] },
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: '', type: 'function', function: { name: '', arguments: ': "Paris"}' } }] } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'tool_calls' }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks.filter(chunk => chunk.type === 'block-end')).toEqual([{
|
||||
type: 'block-end',
|
||||
index: 0,
|
||||
block: { type: 'tool-call', id: 'call_00_x', name: 'get_weather', arguments: '{"city": "Paris"}' },
|
||||
}])
|
||||
})
|
||||
|
||||
it('keeps the established identity when continuation deltas re-send it null', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: 'call_1', type: 'function', function: { name: 'Glob', arguments: '' } }] } }] },
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: null, function: { name: null, arguments: '{}' } }] } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'tool_calls' }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks.filter(chunk => chunk.type === 'block-end')).toEqual([{
|
||||
type: 'block-end',
|
||||
index: 0,
|
||||
block: { type: 'tool-call', id: 'call_1', name: 'Glob', arguments: '{}' },
|
||||
}])
|
||||
})
|
||||
|
||||
it('re-sending the same non-empty identity does not duplicate it', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: 'call_1', type: 'function', function: { name: 'Glob', arguments: '' } }] } }] },
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: 'call_1', type: 'function', function: { name: 'Glob', arguments: '{}' } }] } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'tool_calls' }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks.filter(chunk => chunk.type === 'block-end')).toEqual([{
|
||||
type: 'block-end',
|
||||
index: 0,
|
||||
block: { type: 'tool-call', id: 'call_1', name: 'Glob', arguments: '{}' },
|
||||
}])
|
||||
})
|
||||
|
||||
it('maintains each parallel call identity separately under empty continuation deltas', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{
|
||||
choices: [{
|
||||
delta: {
|
||||
tool_calls: [
|
||||
{ index: 0, id: 'a', type: 'function', function: { name: 'one', arguments: '' } },
|
||||
{ index: 1, id: 'b', type: 'function', function: { name: 'two', arguments: '' } },
|
||||
],
|
||||
},
|
||||
}],
|
||||
},
|
||||
{
|
||||
choices: [{
|
||||
delta: {
|
||||
tool_calls: [
|
||||
{ index: 1, id: '', function: { name: '', arguments: '{"b":1}' } },
|
||||
{ index: 0, id: '', function: { name: '', arguments: '{"a":1}' } },
|
||||
],
|
||||
},
|
||||
}],
|
||||
},
|
||||
{ choices: [{ delta: {}, finish_reason: 'tool_calls' }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks.filter(chunk => chunk.type === 'block-end')).toEqual([
|
||||
{ type: 'block-end', index: 0, block: { type: 'tool-call', id: 'a', name: 'one', arguments: '{"a":1}' } },
|
||||
{ type: 'block-end', index: 1, block: { type: 'tool-call', id: 'b', name: 'two', arguments: '{"b":1}' } },
|
||||
])
|
||||
})
|
||||
})
|
||||
|
|
|
|||
|
|
@ -2,5 +2,5 @@
|
|||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-retry/README.md
|
||||
README.md: 0ea815130212a165582540e43aad7d59fad32d2c
|
||||
README.zh.md: 43a0187c68d272e2764d34db35fa26e51a9d3a83
|
||||
README.md: 84e4a731e60619e8332d339f51276ac31d615661
|
||||
README.zh.md: b5f38e40321af0a9f04803fbef30a8b73cd27cfc
|
||||
|
|
|
|||
|
|
@ -47,7 +47,7 @@ Choose it when a composition runs the agent loop and wants durable request recov
|
|||
- name: '@deepseek-ai/dsh-llm-retry'
|
||||
```
|
||||
|
||||
Omission of `retryPolicy` uses normal mode: five retries for `EMPTY_RESPONSE`, `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, with bounded exponential backoff from 500 ms to 10 seconds and 10 percent jitter. Normal mode can change its finite budget, eligible codes, and backoff; always mode asks downstream recovery first, then retries every model-request failure without an attempt limit, stopping only on success, cancellation, or plugin disposal.
|
||||
Omission of `retryPolicy` uses normal mode: five retries for `EMPTY_RESPONSE`, `MALFORMED_TOOL_CALL`, `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, with bounded exponential backoff from 500 ms to 10 seconds and 10 percent jitter. Normal mode can change its finite budget, eligible codes, and backoff; always mode asks downstream recovery first, then retries every model-request failure without an attempt limit, stopping only on success, cancellation, or plugin disposal.
|
||||
|
||||
### What you can observe
|
||||
|
||||
|
|
|
|||
|
|
@ -47,7 +47,7 @@ kind: "package-reference"
|
|||
- name: '@deepseek-ai/dsh-llm-retry'
|
||||
```
|
||||
|
||||
省略 `retryPolicy` 时使用 normal mode：对 `EMPTY_RESPONSE`、`RATE_LIMIT`、`SERVER`、`TIMEOUT` 与 `TRANSPORT` 最多重试五次，退避从 500 毫秒到 10 秒、带 10% 抖动。normal mode 可以更改其有界预算、合格 code 与退避；always mode 先询问下游恢复，然后无尝试上限地重试每个模型请求失败，只在成功、取消或插件释放时停止。
|
||||
省略 `retryPolicy` 时使用 normal mode：对 `EMPTY_RESPONSE`、`MALFORMED_TOOL_CALL`、`RATE_LIMIT`、`SERVER`、`TIMEOUT` 与 `TRANSPORT` 最多重试五次，退避从 500 毫秒到 10 秒、带 10% 抖动。normal mode 可以更改其有界预算、合格 code 与退避；always mode 先询问下游恢复，然后无尝试上限地重试每个模型请求失败，只在成功、取消或插件释放时停止。
|
||||
|
||||
### 你可以观察到什么
|
||||
|
||||
|
|
|
|||
|
|
@ -38,6 +38,16 @@ export const QUOTA_EXCEEDED_CODE = 'QUOTA'
|
|||
*/
|
||||
export const EMPTY_RESPONSE_CODE = 'EMPTY_RESPONSE'
|
||||
|
||||
/**
|
||||
* Canonical provider-neutral code for a streamed tool call the provider never
|
||||
* identified: its `id` or `name` was absent or empty by the end of the stream.
|
||||
* Such a call cannot be dispatched, and its result cannot be paired back to the
|
||||
* provider on the next request, so adapters classify it as this failure instead
|
||||
* of emitting a tool call the loop would reject as unknown. Nothing durable is
|
||||
* written for the attempt, so retry policy treats it as safe to repeat.
|
||||
*/
|
||||
export const MALFORMED_TOOL_CALL_CODE = 'MALFORMED_TOOL_CALL'
|
||||
|
||||
/**
|
||||
* Canonical provider-neutral code for a credential that was supplied but
|
||||
* cannot be used — malformed rather than absent. Distinct from
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
|
||||
import z from '@deepseek-ai/schemastery'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { EMPTY_RESPONSE_CODE } from './error.ts'
|
||||
import { EMPTY_RESPONSE_CODE, MALFORMED_TOOL_CALL_CODE } from './error.ts'
|
||||
|
||||
const DEFAULT_MAX_RETRIES = 5
|
||||
const DEFAULT_INITIAL_DELAY_MS = 500
|
||||
|
|
@ -17,6 +17,7 @@ const DEFAULT_MAX_DELAY_MS = 10_000
|
|||
const DEFAULT_JITTER_RATIO = 0.1
|
||||
const DEFAULT_RETRYABLE_CODES = Object.freeze([
|
||||
EMPTY_RESPONSE_CODE,
|
||||
MALFORMED_TOOL_CALL_CODE,
|
||||
'RATE_LIMIT',
|
||||
'SERVER',
|
||||
'TIMEOUT',
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ describe('provider retry policy', () => {
|
|||
expect(policy).toEqual({
|
||||
mode: 'normal',
|
||||
maxRetries: 5,
|
||||
retryableCodes: ['EMPTY_RESPONSE', 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT'],
|
||||
retryableCodes: ['EMPTY_RESPONSE', 'MALFORMED_TOOL_CALL', 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT'],
|
||||
initialDelayMs: 500,
|
||||
maxDelayMs: 10_000,
|
||||
jitterRatio: 0.1,
|
||||
|
|
|
|||
|
|
@ -13,13 +13,13 @@
|
|||
{"type":"request/context","data":{"provider":"deepseek-official","model":"deepseek-v4-flash"}}
|
||||
{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":0,"outputTokens":0}}}}
|
||||
{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"error","failure":{"message":"model returned a completed response with no content","code":"EMPTY_RESPONSE"}}}}}
|
||||
{"type":"llm/retry","data":{"retryId":"{{retry:1}}","turn":1,"step":1,"provider":"deepseek-official","mode":"normal","policyKey":"[\"normal\",2,[\"EMPTY_RESPONSE\",\"RATE_LIMIT\",\"SERVER\",\"TIMEOUT\",\"TRANSPORT\"],1,1,0]","retry":1,"maxRetries":2,"delayMs":1,"failure":{"message":"model returned a completed response with no content","code":"EMPTY_RESPONSE"}}}
|
||||
{"type":"llm/retry","data":{"retryId":"{{retry:1}}","turn":1,"step":1,"provider":"deepseek-official","mode":"normal","policyKey":"[\"normal\",2,[\"EMPTY_RESPONSE\",\"MALFORMED_TOOL_CALL\",\"RATE_LIMIT\",\"SERVER\",\"TIMEOUT\",\"TRANSPORT\"],1,1,0]","retry":1,"maxRetries":2,"delayMs":1,"failure":{"message":"model returned a completed response with no content","code":"EMPTY_RESPONSE"}}}
|
||||
{"type":"llm/retry-started","data":{"retryId":"{{retry:1}}","turn":1,"step":1,"retry":1}}
|
||||
{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}}
|
||||
{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"Recovered."}}}
|
||||
{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"Recovered."}}}}
|
||||
{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":12,"outputTokens":3}}}}
|
||||
{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}}
|
||||
{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"Recovered."}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash"},"id":"{{message:3}}"},"usage":{"inputTokens":12,"outputTokens":3}},"sourceEventSeqs":[16,17,18,19,20],"surfaceOp":"append"}
|
||||
{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"Recovered."}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash"},"id":"{{message:3}}"},"usage":{"inputTokens":12,"outputTokens":3}},"sourceEventSeqs":[[16,20]],"surfaceOp":"append"}
|
||||
{"type":"step/end","data":{"turn":1,"step":1}}
|
||||
{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}}
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue