feat(code-runtime-python): add the CPython subprocess backend
Land the PythonCodeRuntime implementation on top of the fd-3 protocol seam: python3 -I per run, binding namespace over fd 3, RLIMIT_CPU/AS, wall-clock timer, and SIGTERM->grace->SIGKILL process-group teardown, with the real-subprocess integration suite. Fixes three defects surfaced on the source PR's review before they ship: - boot-write failure resolved a worker-exit through finish()/settle() that read wallTimer/onAbort/live in their TDZ, rejecting run() instead; the boot write now runs after those bindings and the v8-ignore that hid the branch is removed. - log capture serialized against settlement with no lock while model daemon threads keep writing; LogBuffer now owns one shared re-entrant lock taken by write/flush_line/push. - the fd-3 line residual was a subarray view pinning the whole joined frame; it is copied into a right-sized Buffer via detachResidual so pendingBytes measures what is retained.
This commit is contained in:
Chinesezjc
Tianyi Cui
parent
817c67d799
commit
c388169cff
18 changed files with 6475 additions and 9 deletions
|
|
@ -0,0 +1,6 @@
|
|||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.md
|
||||
2026-07-31-code-runtime-python-settlement-fixes.md: ef7772c10cece314bc6e77da55525f2521101cec
|
||||
2026-07-31-code-runtime-python-settlement-fixes.zh.md: 07110b18783988c69a5e1e1c976db2fec9e234c1
|
||||
|
|
@ -0,0 +1,105 @@
|
|||
# Agent Note: Three settlement and framing fixes in the CPython backend
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-31-code-runtime-python-settlement-fixes.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The [CPython subprocess backend](2026-07-17-code-runtime-python.md) for Code Mode
|
||||
resolves every program outcome as a `CodeRunResult` and rejects `run()` only for
|
||||
seam misuse. Three defects broke that contract in ways unit coverage did not
|
||||
surface, because each hid behind a `/* v8 ignore */`, a captured-callable
|
||||
comment that read as a fix but was not, or a memory effect invisible through the
|
||||
seam. They were found by review of the backend as it stood, not by a failing
|
||||
test, so each fix ships with a test that fails without it.
|
||||
|
||||
## Decision
|
||||
|
||||
Three independent corrections, each in the package that owns the defect.
|
||||
|
||||
### Boot-write failure no longer rejects run()
|
||||
|
||||
In [`src/index.ts`](../../../../packages/code-runtime/code-runtime-python/src/index.ts)
|
||||
the fd-3 boot-frame write is the last statement of `run()`'s synchronous setup.
|
||||
Its `catch` calls `finish()`, and `finish()` reads `wallTimer` and `onAbort` and
|
||||
— through `settle()` — `live`. Those bindings are `const` and were declared
|
||||
AFTER the boot-write, so on a synchronous write failure `finish()` touched them
|
||||
in their temporal dead zone and threw a `ReferenceError`. That escaped the
|
||||
Promise executor and REJECTED `run()`, violating the seam's "outcomes resolve"
|
||||
contract: the caller saw a thrown error instead of the `worker-exit` the catch
|
||||
constructs. The boot-write block is now emitted after `wallTimer`, `onAbort`, and
|
||||
`live` are initialized, and the `/* v8 ignore */` that had hidden the branch from
|
||||
coverage is removed so the catch is measured.
|
||||
|
||||
### Log capture is serialized against settlement
|
||||
|
||||
In [`py/bootstrap.py`](../../../../packages/code-runtime/code-runtime-python/py/bootstrap.py)
|
||||
the settlement `flush_out()`/`flush_err()` on the main coroutine read and clear
|
||||
each stream's `_pending` list and mutate the shared `LogBuffer` ledger. Model
|
||||
code may start daemon threads whose `print`/`write` mutate the same state
|
||||
concurrently. Capturing the bound method (`out_stream.flush_line`) fixed only
|
||||
WHICH callable settlement invokes, not what it reads mid-flight: an interleaved
|
||||
flush could join a `_pending` list being mutated under it, corrupting the ledger
|
||||
and costing the `done` frame — stranding the run to the wall clock. `LogBuffer`
|
||||
now owns one re-entrant lock shared by both streams; `_LogStream.write` and
|
||||
`flush_line`, and `LogBuffer.push`, take it, so the whole read-modify-write is
|
||||
atomic across threads.
|
||||
|
||||
### Fd-3 residual is copied, not viewed
|
||||
|
||||
Also in `src/index.ts`, after the newline loop over a `Buffer.concat` of the
|
||||
pending fd-3 chunks, the leftover partial line was carried forward as the
|
||||
`subarray` VIEW it was sliced to. A view keeps the entire concat backing
|
||||
allocation alive, so a large frame followed by a tiny trailing fragment pinned a
|
||||
whole frame's worth of memory while `pendingBytes` — set to the fragment's
|
||||
length — reported far less than was retained. The residual is now detached into a
|
||||
fresh right-sized `Buffer` via the exported `detachResidual` helper, letting the
|
||||
concat allocation be collected and keeping `pendingBytes` an honest measure.
|
||||
|
||||
## Testing
|
||||
|
||||
- `tests/boot-write-failure.spec.ts` mocks `spawn` so the fd-3 pipe throws on the
|
||||
boot write — the one path a real subprocess cannot be coerced into — and
|
||||
asserts `run()` resolves a `worker-exit` rather than rejecting. Isolated in its
|
||||
own spec so the real-subprocess suite is untouched.
|
||||
- `tests/residual-detach.spec.ts` unit-tests `detachResidual`: the carried copy
|
||||
equals the residual, owns a backing store sized to its own length, and does not
|
||||
share the source frame's `ArrayBuffer`.
|
||||
- `tests/runtime.spec.ts` adds a real-subprocess case where four daemon threads
|
||||
emit unterminated writes up to the moment the body returns and settlement
|
||||
flushes, repeated so the interleave lands; the run must complete cleanly. A
|
||||
pure data race has no single bad input to reject, so this maximizes overlap
|
||||
rather than asserting a deterministic rejection.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Leave the boot-write `/* v8 ignore */` and fix only the ordering.** Rejected:
|
||||
the ignore is what let the TDZ regression ship uncaught. Removing it makes the
|
||||
catch a measured branch, so per-file 100% coverage now proves the failure path
|
||||
is exercised.
|
||||
|
||||
**Fix the flush race by capturing more bound methods.** Rejected: this is the
|
||||
approach that already failed. Binding a callable fixes reference resolution, not
|
||||
concurrent access to the mutable state the callable reads. Only mutual exclusion
|
||||
over the shared ledger closes the race.
|
||||
|
||||
**Guard the residual with a size threshold (copy only large frames).** Rejected:
|
||||
the branch runs once per newline-bearing read, the copy is bounded by the
|
||||
residual's own length (always a partial line), and a threshold adds a tunable
|
||||
and a second code path for no measurable saving. An unconditional right-sized
|
||||
copy is simpler and always correct.
|
||||
|
||||
**Assert the residual memory effect through the seam.** Rejected: the retained
|
||||
allocation is not observable through `CodeRunResult`, so a black-box test could
|
||||
not distinguish fixed from unfixed. Extracting `detachResidual` makes the
|
||||
backing-store invariant a deterministic unit test instead.
|
||||
|
||||
## Consequences
|
||||
|
||||
The seam's resolve-don't-reject contract now holds on the boot-write path, and
|
||||
its coverage is measured rather than ignored. Log capture is thread-safe at the
|
||||
cost of one re-entrant lock acquisition per write and flush — negligible against
|
||||
the os.write already on that path. Fd-3 residual memory is bounded by the actual
|
||||
retained bytes, and `pendingBytes` measures what it claims. Each fix carries a
|
||||
test that fails without it, so a future regression on any of the three goes red.
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
# Agent Note: CPython 后端的三处结算与分帧修复
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-31-code-runtime-python-settlement-fixes.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
用于 Code Mode 的 [CPython 子进程后端](2026-07-17-code-runtime-python.md)把每个程序结果都 resolve 成一个 `CodeRunResult`，仅在 seam 被误用时才 reject `run()`。三个缺陷以单元测试覆盖率无法暴露的方式破坏了这一契约，因为它们各自藏在一处 `/* v8 ignore */` 之后、藏在一条读起来像修复但实际并非修复的"捕获可调用对象"注释之后，或藏在一处透过 seam 不可见的内存效应之后。这些缺陷是通过审查当时的后端代码发现的，而非由某个失败的测试发现，因此每处修复都附带一个在缺少该修复时会失败的测试。
|
||||
|
||||
## Decision
|
||||
|
||||
三处相互独立的修正，各自位于拥有对应缺陷的包中。
|
||||
|
||||
### Boot-write failure no longer rejects run()
|
||||
|
||||
在 [`src/index.ts`](../../../../packages/code-runtime/code-runtime-python/src/index.ts) 中，fd-3 引导帧写入是 `run()` 同步初始化阶段的最后一条语句。它的 `catch` 会调用 `finish()`，而 `finish()` 读取 `wallTimer` 和 `onAbort`，并通过 `settle()` 读取 `live`。这些绑定是 `const`，且声明在引导写入之后，因此在同步写入失败时，`finish()` 会在它们处于暂时性死区（temporal dead zone）时访问它们，从而抛出一个 `ReferenceError`。该错误逃出了 Promise executor 并 reject 了 `run()`，违反了 seam 的"结果一律 resolve"契约：调用方看到的是一个被抛出的错误，而不是 catch 构造的 `worker-exit`。现在引导写入代码块被放到 `wallTimer`、`onAbort` 和 `live` 初始化之后，并且那处曾把该分支从覆盖率中隐藏的 `/* v8 ignore */` 已被移除，从而使该 catch 被纳入度量。
|
||||
|
||||
### Log capture is serialized against settlement
|
||||
|
||||
在 [`py/bootstrap.py`](../../../../packages/code-runtime/code-runtime-python/py/bootstrap.py) 中，主协程上的结算 `flush_out()`／`flush_err()` 会读取并清空各个流的 `_pending` 列表，并修改共享的 `LogBuffer` 账本。模型代码可能启动一些 daemon 线程，其 `print`／`write` 会并发地修改同一状态。捕获绑定方法（`out_stream.flush_line`）只解决了结算调用哪个可调用对象的问题，而没有解决它在执行途中读取什么的问题：一次交错的 flush 可能拼接一个正在其下被修改的 `_pending` 列表，从而破坏账本并丢失 `done` 帧，使该次运行一直拖到墙钟超时。现在 `LogBuffer` 持有一把由两个流共享的可重入锁；`_LogStream.write` 和 `flush_line`，以及 `LogBuffer.push`，都会获取该锁，因此整个读-改-写过程在多线程间是原子的。
|
||||
|
||||
### Fd-3 residual is copied, not viewed
|
||||
|
||||
同样在 `src/index.ts` 中，在对待处理 fd-3 分片的 `Buffer.concat` 结果按换行符做循环之后，剩余的不完整行被以它被切出的 `subarray` 视图形式向前传递。视图会使整个 concat 的底层分配保持存活，因此一个大帧后面跟着一个极小的尾部片段，会钉住整整一帧大小的内存，而 `pendingBytes`（被设为该片段的长度）报告的值远小于实际保留的内存。现在，残余数据通过导出的 `detachResidual` 辅助函数被分离到一个大小恰当的新 `Buffer` 中，从而让 concat 分配得以被回收，并使 `pendingBytes` 成为一个诚实的度量值。
|
||||
|
||||
## Testing
|
||||
|
||||
- `tests/boot-write-failure.spec.ts` 对 `spawn` 做 mock，使 fd-3 管道在引导写入时抛出异常（这是真实子进程无法被迫进入的唯一路径），并断言 `run()` resolve 出一个 `worker-exit` 而非 reject。它被隔离在自己的 spec 中，因此真实子进程测试套件不受影响。
|
||||
- `tests/residual-detach.spec.ts` 对 `detachResidual` 做单元测试：向前传递的副本与残余数据相等、拥有一个大小与其自身长度一致的底层存储，并且不与源帧的 `ArrayBuffer` 共享。
|
||||
- `tests/runtime.spec.ts` 新增一个真实子进程用例：四个 daemon 线程持续发出未结束的写入，直到函数体返回、结算执行 flush 的那一刻，并反复运行以让交错真正出现；该次运行必须干净地完成。纯数据竞态没有单一的坏输入可供 reject，因此该测试最大化重叠而非断言一个确定性的 reject。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**保留引导写入处的 `/* v8 ignore */`，只修复顺序。** 已否决：正是那处 ignore 让这个 TDZ 回归得以未被发现地进入代码库。移除它使该 catch 成为被度量的分支，因此按文件计的 100% 覆盖率现在能证明该失败路径确实被执行。
|
||||
|
||||
**通过捕获更多绑定方法来修复 flush 竞态。** 已否决：这正是已经失败过的做法。绑定一个可调用对象解决的是引用解析，而不是对该可调用对象所读取的可变状态的并发访问。只有对共享账本施加互斥才能消除该竞态。
|
||||
|
||||
**用大小阈值来保护残余数据（只复制大帧）。** 已否决：该分支在每次包含换行符的读取时运行一次，复制的规模受残余数据自身长度约束（始终是一个不完整行），而阈值会引入一个可调参数和第二条代码路径，却换不来任何可度量的节省。无条件地做大小恰当的复制更简单，且始终正确。
|
||||
|
||||
**通过 seam 断言残余数据的内存效应。** 已否决：被保留的分配透过 `CodeRunResult` 不可观测，因此黑盒测试无法区分已修复与未修复。转而抽取出 `detachResidual`，把底层存储的不变量变成一个确定性的单元测试。
|
||||
|
||||
## Consequences
|
||||
|
||||
现在 seam 的"只 resolve、不 reject"契约在引导写入路径上得以成立，且其覆盖率是被度量而非被忽略的。日志捕获现在是线程安全的，代价是每次写入和 flush 都要获取一次可重入锁，这相对于该路径上已有的 os.write 可以忽略不计。fd-3 残余数据的内存现在受实际保留的字节数约束，且 `pendingBytes` 度量的正是它所声称的值。每处修复都附带一个在缺少它时会失败的测试，因此这三处中任何一处未来若发生回归都会变红。
|
||||
|
|
@ -356,6 +356,47 @@ export interface Config {
|
|||
|
||||
Source: [`packages/client/hmr/src/index.ts:31`](../packages/client/hmr/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-code-runtime-python"></a>
|
||||
|
||||
## `@deepseek-ai/dsh-code-runtime-python`
|
||||
|
||||
```ts config-catalog
|
||||
/** Plugin config: every cap, changeable from `cordis.yml` (no hardcoded tunables). */
|
||||
export interface Config {
|
||||
/**
|
||||
* RLIMIT_CPU in whole seconds (a positive integer — `setrlimit` in the child
|
||||
* rejects a float). The child sets the soft limit to `cpuSeconds` and the
|
||||
* hard limit to `cpuSeconds + 1`: the kernel delivers SIGXCPU at the soft
|
||||
* limit, which the host classifies as a `timeout`; the +1s hard limit is a
|
||||
* SIGKILL backstop for a program that traps SIGXCPU. Granularity is seconds —
|
||||
* a coarser counterpart to the worker backend's millisecond `computeMs`.
|
||||
*/
|
||||
cpuSeconds?: number
|
||||
/** Wall-clock ceiling in milliseconds; backstops CPU time for programs awaiting a promise nobody resolves. */
|
||||
maxWallMs?: number
|
||||
/**
|
||||
* RLIMIT_AS in mebibytes; caps address space so a runaway allocation fails
|
||||
* cleanly. Not applied on Darwin, where the dyld shared cache mapped into
|
||||
* every process at exec exceeds any practical cap and the kernel rejects
|
||||
* the call; `cpuSeconds` and `maxWallMs` still bound the run there.
|
||||
*/
|
||||
addressSpaceMb?: number
|
||||
/** Shared byte budget for captured log text (host-side ledger). */
|
||||
maxLogBytes?: number
|
||||
/** Byte cap for the completion value. */
|
||||
maxValueBytes?: number
|
||||
/** SIGTERM→SIGKILL grace period on kill, matching bash-local's default. */
|
||||
graceMs?: number
|
||||
/**
|
||||
* Absolute path or basename of the CPython interpreter to spawn. Resolved
|
||||
* through `PATH` when a basename is given.
|
||||
*/
|
||||
pythonBin?: string
|
||||
}
|
||||
```
|
||||
|
||||
Source: [`packages/code-runtime/code-runtime-python/src/index.ts:44`](../packages/code-runtime/code-runtime-python/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-code-runtime-worker-thread"></a>
|
||||
|
||||
## `@deepseek-ai/dsh-code-runtime-worker-thread`
|
||||
|
|
@ -3403,7 +3444,6 @@ Imported as libraries by other packages; a `cordis.yml` cannot load them.
|
|||
- `@deepseek-ai/dsh-client-ui-slots` ([`packages/client/ui-slots/src/index.ts`](../packages/client/ui-slots/src/index.ts))
|
||||
- `@deepseek-ai/dsh-client-web` ([`packages/client/web/src/index.ts`](../packages/client/web/src/index.ts))
|
||||
- `@deepseek-ai/dsh-cmdline` ([`packages/boot/cmdline/src/index.ts`](../packages/boot/cmdline/src/index.ts))
|
||||
- `@deepseek-ai/dsh-code-runtime-python` ([`packages/code-runtime/code-runtime-python/src/index.ts`](../packages/code-runtime/code-runtime-python/src/index.ts))
|
||||
- `@deepseek-ai/dsh-deque` ([`packages/util/deque/src/index.ts`](../packages/util/deque/src/index.ts))
|
||||
- `@deepseek-ai/dsh-experimental-agent-team-profile` ([`packages/experimental/agent-team-profile/src/index.ts`](../packages/experimental/agent-team-profile/src/index.ts))
|
||||
- `@deepseek-ai/dsh-experimental-agent-team-web-profile` ([`packages/experimental/agent-team-web-profile/src/index.ts`](../packages/experimental/agent-team-web-profile/src/index.ts))
|
||||
|
|
|
|||
|
|
@ -2,5 +2,5 @@
|
|||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/module-graph.md
|
||||
module-graph.md: 2229efcd7e76b3b224eb307ee7de9ecea0ad85d7
|
||||
module-graph.zh.md: 3edca4ea261f68593973149b21e72e4a25d7a504
|
||||
module-graph.md: e4bbd01f7000e88a4dc982f5f8c7986176968ec6
|
||||
module-graph.zh.md: f046b59e7412bb3db0ea66fccc39294e90c66f1d
|
||||
|
|
|
|||
|
|
@ -378,7 +378,6 @@ flowchart TD
|
|||
pkg_sdk_app --> pkg_invariants
|
||||
pkg_sdk_minimal --> pkg_invariants
|
||||
pkg_code_runtime --> pkg_invariants
|
||||
pkg_code_runtime_python --> pkg_invariants
|
||||
pkg_credentials --> pkg_invariants
|
||||
pkg_e2b --> pkg_invariants
|
||||
pkg_experimental_agent_team_profile --> pkg_invariants
|
||||
|
|
@ -472,6 +471,9 @@ flowchart TD
|
|||
pkg_app_boot --> pkg_invariants
|
||||
pkg_app_boot --> pkg_launch_environment
|
||||
pkg_app_boot --> pkg_system_prompt
|
||||
pkg_code_runtime_python --> pkg_invariants
|
||||
pkg_code_runtime_python --> pkg_session
|
||||
pkg_code_runtime_python --> pkg_timeout
|
||||
pkg_code_runtime_worker_thread --> pkg_code_runtime
|
||||
pkg_code_runtime_worker_thread --> pkg_invariants
|
||||
pkg_code_runtime_worker_thread --> pkg_session
|
||||
|
|
@ -1372,7 +1374,6 @@ flowchart TD
|
|||
| [`sdk-app`](../packages/bundle/sdk-app) | `bundle` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`sdk-minimal`](../packages/bundle/sdk-minimal) | `bundle` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`code-runtime`](../packages/code-runtime/code-runtime) | `code-runtime` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`code-runtime-python`](../packages/code-runtime/code-runtime-python) | `code-runtime` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`credentials`](../packages/credentials/credentials) | `credentials` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`e2b`](../packages/e2b/e2b) | `e2b` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`experimental-agent-team-profile`](../packages/experimental/agent-team-profile) | `experimental` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
|
|
@ -1415,6 +1416,7 @@ flowchart TD
|
|||
| [`spill`](../packages/spill/spill) | `spill` | [`brand`](../packages/util/brand), [`invariants`](../packages/runtime-diagnostics/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
|
||||
| [`attachment-local`](../packages/attachment/attachment-local) | `attachment` | [`attachment`](../packages/attachment/attachment), [`home-paths`](../packages/util/home-paths), [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`app-boot`](../packages/boot/app-boot) | `boot` | [`home-paths`](../packages/util/home-paths), [`invariants`](../packages/runtime-diagnostics/invariants), [`launch-environment`](../packages/util/launch-environment), [`system-prompt`](../packages/core/system-prompt) |
|
||||
| [`code-runtime-python`](../packages/code-runtime/code-runtime-python) | `code-runtime` | [`invariants`](../packages/runtime-diagnostics/invariants), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) |
|
||||
| [`code-runtime-worker-thread`](../packages/code-runtime/code-runtime-worker-thread) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime), [`invariants`](../packages/runtime-diagnostics/invariants), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) |
|
||||
| [`persona`](../packages/preset/persona) | `preset` | [`invariants`](../packages/runtime-diagnostics/invariants), [`system-prompt`](../packages/core/system-prompt) |
|
||||
| [`sandbox`](../packages/sandbox/sandbox) | `sandbox` | [`invariants`](../packages/runtime-diagnostics/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
|
||||
|
|
|
|||
|
|
@ -380,7 +380,6 @@ flowchart TD
|
|||
pkg_sdk_app --> pkg_invariants
|
||||
pkg_sdk_minimal --> pkg_invariants
|
||||
pkg_code_runtime --> pkg_invariants
|
||||
pkg_code_runtime_python --> pkg_invariants
|
||||
pkg_credentials --> pkg_invariants
|
||||
pkg_e2b --> pkg_invariants
|
||||
pkg_experimental_agent_team_profile --> pkg_invariants
|
||||
|
|
@ -474,6 +473,9 @@ flowchart TD
|
|||
pkg_app_boot --> pkg_invariants
|
||||
pkg_app_boot --> pkg_launch_environment
|
||||
pkg_app_boot --> pkg_system_prompt
|
||||
pkg_code_runtime_python --> pkg_invariants
|
||||
pkg_code_runtime_python --> pkg_session
|
||||
pkg_code_runtime_python --> pkg_timeout
|
||||
pkg_code_runtime_worker_thread --> pkg_code_runtime
|
||||
pkg_code_runtime_worker_thread --> pkg_invariants
|
||||
pkg_code_runtime_worker_thread --> pkg_session
|
||||
|
|
@ -1374,7 +1376,6 @@ flowchart TD
|
|||
| [`sdk-app`](../packages/bundle/sdk-app) | `bundle` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`sdk-minimal`](../packages/bundle/sdk-minimal) | `bundle` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`code-runtime`](../packages/code-runtime/code-runtime) | `code-runtime` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`code-runtime-python`](../packages/code-runtime/code-runtime-python) | `code-runtime` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`credentials`](../packages/credentials/credentials) | `credentials` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`e2b`](../packages/e2b/e2b) | `e2b` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`experimental-agent-team-profile`](../packages/experimental/agent-team-profile) | `experimental` | [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
|
|
@ -1417,6 +1418,7 @@ flowchart TD
|
|||
| [`spill`](../packages/spill/spill) | `spill` | [`brand`](../packages/util/brand), [`invariants`](../packages/runtime-diagnostics/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
|
||||
| [`attachment-local`](../packages/attachment/attachment-local) | `attachment` | [`attachment`](../packages/attachment/attachment), [`home-paths`](../packages/util/home-paths), [`invariants`](../packages/runtime-diagnostics/invariants) |
|
||||
| [`app-boot`](../packages/boot/app-boot) | `boot` | [`home-paths`](../packages/util/home-paths), [`invariants`](../packages/runtime-diagnostics/invariants), [`launch-environment`](../packages/util/launch-environment), [`system-prompt`](../packages/core/system-prompt) |
|
||||
| [`code-runtime-python`](../packages/code-runtime/code-runtime-python) | `code-runtime` | [`invariants`](../packages/runtime-diagnostics/invariants), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) |
|
||||
| [`code-runtime-worker-thread`](../packages/code-runtime/code-runtime-worker-thread) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime), [`invariants`](../packages/runtime-diagnostics/invariants), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) |
|
||||
| [`persona`](../packages/preset/persona) | `preset` | [`invariants`](../packages/runtime-diagnostics/invariants), [`system-prompt`](../packages/core/system-prompt) |
|
||||
| [`sandbox`](../packages/sandbox/sandbox) | `sandbox` | [`invariants`](../packages/runtime-diagnostics/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
|
||||
|
|
|
|||
|
|
@ -33,10 +33,17 @@
|
|||
"license": "MIT",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-timeout": "workspace:^",
|
||||
"@deepseek-ai/cordis": "workspace:^"
|
||||
},
|
||||
"dependencies": {
|
||||
"@deepseek-ai/schemastery": "workspace:^"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-timeout": "workspace:^",
|
||||
"@deepseek-ai/cordis": "workspace:^"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
1711
packages/code-runtime/code-runtime-python/py/bootstrap.py
Normal file
1711
packages/code-runtime/code-runtime-python/py/bootstrap.py
Normal file
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,67 @@
|
|||
import { EventEmitter } from 'node:events'
|
||||
import { PassThrough } from 'node:stream'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
|
||||
/**
|
||||
* A synchronous `proto.write` throw on the fd-3 pipe is the one boot path a real
|
||||
* subprocess cannot be coerced into from a test: the pipe accepts queued bytes
|
||||
* until the kernel buffer fills, and a same-tick EPIPE needs fd 3 already closed
|
||||
* before the first write. `spawn` is mocked so fd 3 throws on the boot frame,
|
||||
* which is exactly the branch that regressed. The mock is confined to this file
|
||||
* so the real-subprocess suite in runtime.spec.ts is untouched.
|
||||
*/
|
||||
const { spawnMock } = vi.hoisted(() => ({ spawnMock: vi.fn() }))
|
||||
vi.mock('node:child_process', async importOriginal => ({
|
||||
...(await importOriginal<typeof import('node:child_process')>()),
|
||||
spawn: spawnMock,
|
||||
}))
|
||||
|
||||
const { PythonCodeRuntime } = await import('../src/index.ts')
|
||||
|
||||
/** A `child_process.ChildProcess` stand-in whose fd-3 pipe rejects every write. */
|
||||
function fakeChildWithThrowingFd3(): EventEmitter {
|
||||
const child = new EventEmitter() as EventEmitter & {
|
||||
pid?: number
|
||||
stdout: PassThrough
|
||||
stderr: PassThrough
|
||||
stdio: unknown[]
|
||||
}
|
||||
// Leave `pid` absent: `finish()` still runs its `clearTimeout(wallTimer)` /
|
||||
// `removeEventListener(onAbort)` prologue (the TDZ site) before short-
|
||||
// circuiting on `child.pid === undefined` to `settle` instead of waiting on a
|
||||
// `close` this fake never emits, so the run resolves promptly.
|
||||
child.stdout = new PassThrough()
|
||||
child.stderr = new PassThrough()
|
||||
// A duplex whose `write` throws synchronously, standing in for an fd-3 pipe
|
||||
// that fails the moment the boot frame is issued.
|
||||
const proto = new PassThrough()
|
||||
proto.write = () => { throw Object.assign(new Error('EPIPE: broken pipe, write'), { code: 'EPIPE' }) }
|
||||
child.stdio = [new PassThrough(), child.stdout, child.stderr, proto]
|
||||
return child
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
spawnMock.mockReset()
|
||||
})
|
||||
|
||||
describe('PythonCodeRuntime — boot-write failure', () => {
|
||||
it('resolves a worker-exit when the fd-3 boot write throws (no TDZ ReferenceError)', async () => {
|
||||
// Before the fix, the boot-write block ran BEFORE `wallTimer`, `onAbort`,
|
||||
// and `live` were initialized, so its `finish()` (which clears `wallTimer`,
|
||||
// removes `onAbort`, and — through `settle` — deletes `live`) hit the
|
||||
// temporal dead zone and threw a ReferenceError. That escaped the Promise
|
||||
// executor and REJECTED run() instead of resolving the worker-exit the catch
|
||||
// constructs. This test would see that rejection; the fix makes it resolve.
|
||||
spawnMock.mockImplementation(() => fakeChildWithThrowingFd3())
|
||||
const ctx = new Context()
|
||||
const fiber = await ctx.plugin(PythonCodeRuntime)
|
||||
const runtime = ctx.codeRuntime as InstanceType<typeof PythonCodeRuntime>
|
||||
|
||||
const result = await runtime.run({ program: 'return 1', bindings: [] })
|
||||
|
||||
expect(result.error?.kind).toBe('worker-exit')
|
||||
expect(result.error?.message).toContain('failed to boot python subprocess')
|
||||
await fiber.dispose()
|
||||
})
|
||||
})
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
import { describe, expect, it } from 'vitest'
|
||||
import { detachResidual } from '../src/index.ts'
|
||||
|
||||
describe('detachResidual — fd-3 residual detachment', () => {
|
||||
it('returns a copy that does NOT share the source frame allocation', () => {
|
||||
// Simulate the data handler's state: one large joined frame from
|
||||
// Buffer.concat, sliced past its newline to leave a small residual VIEW.
|
||||
const joined = Buffer.alloc(1024 * 1024, 0x61) // 1 MiB backing allocation
|
||||
joined[512] = 0x0a // a newline partway through
|
||||
const residual = joined.subarray(513) // a view onto `joined`'s backing store
|
||||
|
||||
// Before the fix the handler carried this view forward verbatim, pinning the
|
||||
// whole 1 MiB `joined` allocation behind a residual that reports far fewer
|
||||
// bytes. A right-sized copy must not point back into `joined`.
|
||||
const [carried] = detachResidual(residual)
|
||||
|
||||
expect(carried).toBeDefined()
|
||||
expect(carried!.length).toBe(residual.length)
|
||||
expect(carried!.equals(residual)).toBe(true)
|
||||
// The copy's backing store is its own, sized to its content — not the 1 MiB
|
||||
// frame. A subarray view would report the source's full byteLength here.
|
||||
expect(carried!.buffer.byteLength).toBe(carried!.length)
|
||||
expect(carried!.buffer).not.toBe(joined.buffer)
|
||||
})
|
||||
|
||||
it('carries nothing forward for an empty residual', () => {
|
||||
expect(detachResidual(Buffer.alloc(0))).toEqual([])
|
||||
})
|
||||
})
|
||||
3287
packages/code-runtime/code-runtime-python/tests/runtime.spec.ts
Normal file
3287
packages/code-runtime/code-runtime-python/tests/runtime.spec.ts
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -14,8 +14,20 @@
|
|||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../code-runtime"
|
||||
},
|
||||
{
|
||||
"path": "../../core/session"
|
||||
},
|
||||
{
|
||||
"path": "../../runtime-diagnostics/invariants"
|
||||
},
|
||||
{
|
||||
"path": "../../util/timeout"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
|
|||
13
pnpm-lock.yaml
generated
13
pnpm-lock.yaml
generated
|
|
@ -3954,6 +3954,10 @@ importers:
|
|||
version: link:../../runtime-diagnostics/invariants
|
||||
|
||||
packages/code-runtime/code-runtime-python:
|
||||
dependencies:
|
||||
'@deepseek-ai/schemastery':
|
||||
specifier: link:../../../vendor/schemastery
|
||||
version: link:../../../vendor/schemastery
|
||||
devDependencies:
|
||||
'@deepseek-ai/cordis':
|
||||
specifier: workspace:^
|
||||
|
|
@ -3961,6 +3965,12 @@ importers:
|
|||
'@deepseek-ai/dsh-invariants':
|
||||
specifier: workspace:^
|
||||
version: link:../../runtime-diagnostics/invariants
|
||||
'@deepseek-ai/dsh-session':
|
||||
specifier: workspace:^
|
||||
version: link:../../core/session
|
||||
'@deepseek-ai/dsh-timeout':
|
||||
specifier: workspace:^
|
||||
version: link:../../util/timeout
|
||||
|
||||
packages/code-runtime/code-runtime-worker-thread:
|
||||
dependencies:
|
||||
|
|
@ -10331,6 +10341,9 @@ importers:
|
|||
'@deepseek-ai/dsh-code-runtime':
|
||||
specifier: workspace:^
|
||||
version: link:../../packages/code-runtime/code-runtime
|
||||
'@deepseek-ai/dsh-code-runtime-python':
|
||||
specifier: workspace:^
|
||||
version: link:../../packages/code-runtime/code-runtime-python
|
||||
'@deepseek-ai/dsh-code-runtime-worker-thread':
|
||||
specifier: workspace:^
|
||||
version: link:../../packages/code-runtime/code-runtime-worker-thread
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@
|
|||
"@deepseek-ai/dsh-bash-local": "workspace:^",
|
||||
"@deepseek-ai/dsh-brand": "workspace:^",
|
||||
"@deepseek-ai/dsh-code-runtime": "workspace:^",
|
||||
"@deepseek-ai/dsh-code-runtime-python": "workspace:^",
|
||||
"@deepseek-ai/dsh-code-runtime-worker-thread": "workspace:^",
|
||||
"@deepseek-ai/dsh-command-compact": "workspace:^",
|
||||
"@deepseek-ai/dsh-command-goal": "workspace:^",
|
||||
|
|
|
|||
|
|
@ -38,7 +38,10 @@ const DEPLOY_ONLY_DOCS = ['README.md', 'README.zh.md', 'README.i18n.yaml']
|
|||
/**
|
||||
* Whole-tree assets cover Cordis's runtime bare-package imports, which pkg's
|
||||
* static analysis cannot see. Package manifests are explicit because bare-name
|
||||
* resolution depends on them.
|
||||
* resolution depends on them. `*.py` carries the CPython code-runtime backend's
|
||||
* bootstrap and protocol scripts into the executable; the backend copies them
|
||||
* out to a real filesystem path before spawning, since the interpreter is an
|
||||
* external process that cannot read pkg's virtual filesystem.
|
||||
*/
|
||||
const ASSET_GLOBS = [
|
||||
'package.json',
|
||||
|
|
@ -55,6 +58,7 @@ const ASSET_GLOBS = [
|
|||
'node_modules/**/*.so',
|
||||
'node_modules/**/*.so.*',
|
||||
'node_modules/**/*.wasm',
|
||||
'node_modules/**/*.py',
|
||||
'node_modules/**/*.yaml',
|
||||
'node_modules/**/*.yml',
|
||||
// web-app builds this path dynamically, so pkg cannot discover the static frontend.
|
||||
|
|
|
|||
|
|
@ -31,6 +31,7 @@ const windowsUnsupportedPackages = process.platform === 'win32'
|
|||
'packages/shell/tool-bash',
|
||||
'packages/hooks/*',
|
||||
'packages/terminal/terminal-bash',
|
||||
'packages/code-runtime/code-runtime-python',
|
||||
'packages/sandbox/sandbox-local',
|
||||
]
|
||||
: []
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue