deepseek-harness/docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md

67 lines
17 KiB
Markdown
Raw Normal View History

workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
# RFC: Dynamic workflows — a script-driven multi-agent orchestration seam
Merge remote-tracking branch 'origin/master' into worktree-dynamic-workflows Beyond the mechanical conflicts (provider capability lines vs master's new inheritsParentContext field; generated catalogs regenerated rather than hand-merged; knip/lockfile), three master-side reworks required semantic adaptation of this branch: - The persona rework removed AgentOptions.systemPrompt, which was the structured-output instruction's channel. The instruction now rides the SAME final-request enforcement listener that injects the schema'd tool: appended per request to final.system (per-request wire state, not agent prompt state). Tests assert the wire request (adapter.requests) instead of child.options; the bare-direct-dispatch test pins the no-system arm. - Tool guidance moved out of deployment prompts into per-tool prompt sections; the examples' workflow paragraph became a tool:<toolName> section contributed by dsh-tool-workflow (explicit-ask-only policy), and both example personas resolve to master's minimal identity+behavior form. tool-workflow gains inject: systemPrompt (+ peer dep, tsconfig ref); the export-shape guard updated. - The uniform-RFC-format gate: the dynamic-workflows RFC restructured to the implemented/ skeleton (bare Status line; Proposal -> Decision; What-was-rejected -> Alternatives considered; new Consequences), and the overall-run-timeout deferral is now recorded in the RFC's Deferred list. The doc-graphs atlas classification gains the workflows seam (workflow-vm implementation, tool-workflow consumer). Master's harness-identity section made "empty assembled prompt" states unreachable through the loop, so the instruction-append is a plain undefined-ternary and the structured tests assert append-not-replace. All snapshot goldens (including workflow-run) replay unchanged. Full local CI-equivalent gate sequence green on the merged tree.
2026-07-06 03:14:07 +08:00
Status: implemented
workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
## Problem
The harness can delegate ONE task to ONE child (`dsh-tool-subagent`), but work that fans out across many independent pieces — an audit over many files, a migration, multi-angle research, adversarial verification of findings — forces the model to orchestrate turn by turn: every intermediate result lands in the parent context, the plan lives nowhere durable, and coordination costs a model round-trip per step. Claude Code ships this capability as [dynamic workflows](https://code.claude.com/docs/en/workflows): the model writes a JavaScript orchestration script, a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results.
Merge remote-tracking branch 'origin/master' into worktree-dynamic-workflows Beyond the mechanical conflicts (provider capability lines vs master's new inheritsParentContext field; generated catalogs regenerated rather than hand-merged; knip/lockfile), three master-side reworks required semantic adaptation of this branch: - The persona rework removed AgentOptions.systemPrompt, which was the structured-output instruction's channel. The instruction now rides the SAME final-request enforcement listener that injects the schema'd tool: appended per request to final.system (per-request wire state, not agent prompt state). Tests assert the wire request (adapter.requests) instead of child.options; the bare-direct-dispatch test pins the no-system arm. - Tool guidance moved out of deployment prompts into per-tool prompt sections; the examples' workflow paragraph became a tool:<toolName> section contributed by dsh-tool-workflow (explicit-ask-only policy), and both example personas resolve to master's minimal identity+behavior form. tool-workflow gains inject: systemPrompt (+ peer dep, tsconfig ref); the export-shape guard updated. - The uniform-RFC-format gate: the dynamic-workflows RFC restructured to the implemented/ skeleton (bare Status line; Proposal -> Decision; What-was-rejected -> Alternatives considered; new Consequences), and the overall-run-timeout deferral is now recorded in the RFC's Deferred list. The doc-graphs atlas classification gains the workflows seam (workflow-vm implementation, tool-workflow consumer). Master's harness-identity section made "empty assembled prompt" states unreachable through the loop, so the instruction-append is a plain undefined-ternary and the structured tests assert append-not-replace. All snapshot goldens (including workflow-run) replay unchanged. Full local CI-equivalent gate sequence green on the merged tree.
2026-07-06 03:14:07 +08:00
## Decision
workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
A workflow capability family at `packages/workflow/` in the bash seam shape (interface / implementation / consumer), plus the structured-output foundation it needs on the subagent seam.
### The script contract (Claude Code-compatible)
A script is `export const meta = {...}` (a PURE object literal: `name`, `description`, optional `whenToUse`/`phases`) followed by a plain-JS body with top-level `await`, ending in `return <json-value>`. The body sees exactly: `agent(prompt, {label, phase, schema, model})`, `parallel(thunks)`, `pipeline(items, ...stages)` (NO cross-stage barrier; `(prev, item, index)` callbacks), `phase(title)`, `log(message)`, and `args`. CC semantics are preserved where they matter to script authors: a failed child resolves `null` (scripts `.filter(Boolean)`); an ordinary stage throw nulls the ITEM and skips its remaining stages. CC's determinism bans (`Date.now()`/`Math.random()`/argless `new Date()` throwing) are NOT enforced — they exist for CC's journaling/resume, which this cut defers — so a CC-authored script runs unchanged while scripts written here may freely read the clock.
workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
One deliberate strictness DIVERGENCE from CC: hook misuse — unknown or deferred options (`effort`/`isolation`/`agentType`), malformed arguments, schemas outside the supported subset, tripped caps, seam start failures — throws a `WorkflowError` with `fatal: true`, and the combinators RE-THROW fatal errors instead of nulling the item. Without this, a typo'd option dissolves into a `null` indistinguishable from a child failure — the accepted-then-ignored failure mode this repo bans. One addition: the tool's `args` parameter is a JSON OBJECT (a bare list is wrapped as a field) so the wire schema stays honest.
### The seam (dsh-workflow)
`ctx.workflows` is an abstract `WorkflowService` in the bash shape — one engine per context, no named-provider registry (engines are deployment swaps, not co-residents). `start(request)` throws synchronously for a script that cannot begin; a returned `WorkflowRun`'s `result` NEVER rejects (failures resolve as `stopReason: 'error' | 'cancelled'`). The `workflow/*` events are observe-only emits carrying DATA SNAPSHOTS (id + meta; `workflow/end` omits the result value), per-listener contained, mirroring `subagent/start`/`subagent/end` — control stays with the run's holder. Vocabulary details: [core-data-structures/workflow.md](../../../core-data-structures/workflow.md).
### The engine (dsh-workflow-workerthread): one worker thread per run
workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
**Trust premise (governs every engine decision below)**: workflow scripts are MODEL-WRITTEN — the same trust level as the model's existing bash access — so the engine defends against BUGGY scripts, never hostile ones. In scope: `result` never rejects, no unhandled rejections from dropped hook promises, loud rejection of values JSON cannot carry, fatal-vs-null hook discipline, cancellation that always frees the caller. Out of scope, deliberately: adversarial values (throwing/spinning accessors, proxies with hostile traps, prototype forgery, `prepareStackTrace` hijack) AND Node-API escape from the script's context — the vm context shares object machinery with its surrounding realm, so a script can reach the `Function` constructor (`globalThis.constructor.constructor`) and from it `process` and every Node builtin; the absent globals are API surface, not containment, and a worker thread is NOT a security boundary (an escapee holds process-wide privileges — Node's permission model is per-process). Worker-side code MAY run script code while reading script values, and that is accepted: a synchronous spin costs the script its OWN thread (terminated at the post-cancel grace), never the host loop, so containing error VALUES would be cost without a threat model. Genuine sandboxing (isolated-vm, a separate process) remains an engine swap behind the seam, not incremental defenses here.
workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
**Why node:worker_threads**: one run = one worker thread, no pooling — a run is heavyweight (many children), so thread spin-up (~tens of ms) is noise. The script runs in a vm context INSIDE the worker, keeping the script-visible surface exactly the hook contract above (a bare worker realm would leak `setTimeout`/`fetch`/`process` as accidental API), and every `agent()` bridges to `ctx.subagents` by message-port RPC — children are I/O-bound LLM loops and stay on the host loop; the thread isolates the SCRIPT, the only part that can spin. What the thread buys: `start()` never blocks the host (an in-process engine runs the initial synchronous slice inline and cannot kill a spin past the first await — it could only ABANDON such a script, leaving the spin on the host loop), the post-cancel grace ends in a REAL `worker.terminate()`, and the value boundary is serialization by construction. isolated-vm was rejected for actual sandboxing: maintenance mode, `--no-node-snapshot` on EVERY consumer process (including published bins) on Node ≥ 20, node-gyp source-build fallback. Key mechanics (details in the package README): meta extraction and a body pre-parse stay HOST-side (preserving the seam's synchronous throws), a ready→go handshake keeps a run cancelled before start from ever executing the body, `cancel()` drives both child-cancel channels host-side (the shared request signal AND each child's explicit `cancel()` — a wedged worker cannot relay its own cancel RPCs), a host-side child registry backs worker-death reaping and `dispose()` quiescence, the wire protocol is enum-keyed payload maps private to the package, and on a termination path `agentsStarted` degrades to the host-observed count. Coverage puts the worker-side session on an in-process `MessageChannel` (real-Worker code is invisible to main-process v8) and proves the built `lib/worker.js` — a second tsdown entry, sanctioned in the workspace-constraints gate by the `"./worker"` subpath export — under plain node in the built-bin smoke gate.
workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
workflow: simplify to the trust premise; settle result on cancellation Two review responses that belong together — the same review argued the engine was defending the wrong threat while a benign-input bug wedged the product. 1) Drop hostile-value containment; state the trust premise. Scripts are model-written — the same trust level as the model's bash access — yet successive pre-push review rounds had ratcheted in defenses that only matter against an adversarial author: trap-free proxy rejection, accessor-never-invoked descriptor walks, realm-side pre-rendering of thrown values, realm-built promises/arrays/error clones with structural fatal recognition. That same author keeps a documented, accepted, unkillable event-loop spin, so containing its error VALUES is cost without a threat model — and the planned hardened engine (worker/isolated-vm) gets value isolation by serialization and deletes all of this machinery anyway. What stays, because benign scripts hit it constantly: result never rejects; dropped hook promises cannot become unhandled rejections; the value boundary rejects LOUD everything JSON cannot carry (now a plain recursive walk — getters are read ordinarily and their result is what crosses; a throwing read fails loud); a "__proto__" key still copies as a data property; the fatal-vs-null combinator discipline (now host instanceof — unforgeable from the realm and simpler than clone-shape recognition). What changes for scripts (documented in the engine README): hooks hand back host values and host errors — in-script `instanceof Error` on a hook failure is false (branch on e.name/e.code) — and args are host-cloned once so a script cannot mutate the caller's object. realm.ts drops 289 → 173 lines; the hostile-value test tables go with it. The premise now leads the engine module doc, the README, and the RFC's engine section, with the removed machinery recorded under What was rejected. 2) result settles within the dispose grace of a cancellation. Review finding (verified through the real registry + tool + engine): a script parked on a promise no hook owns — `await new Promise(() => {})`, `await Promise.race([])`, a returned never-settling thenable — could not be settled by cancel(): hooks reject and children abort, but nothing touches a promise the engine does not own, so `result` stayed pending FOREVER (the previous cut even pinned that as intended). The tool awaits run.result BEFORE its disposing finally, the registry awaits the tool, the loop awaits the registry — one such script wedged the whole agent turn past any abort, unrecoverable in-process; the mock engine in the tool's abort test settles result on cancel, which is exactly the behavior the real engine lacked, so no existing test could see it. The seam contract now says it out loud: once a run is cancelled, result SETTLES within the implementation's bounded grace even if the script never does. The vm engine arms an abandon channel in cancel(); drive() races the script against it, force-settling 'cancelled' at the grace (the abandoned settlement stays contained; a post-slice synchronous spin remains the documented limitation). dispose()'s outer race now exists for child quiescence only, and `workflow/end` again fires exactly once per started run. The old 'result stays pending' pin is FLIPPED to the new contract (the pinned behavior was the bug); new regressions cover cancel-then-settle on a parked script, a never-settling returned thenable, and the full composition through the REAL registry + tool + vm engine (tool-workflow gains workflow-vm/subagent devDeps for it). agentsStarted JSDoc clarified while touching the vocabulary (accepted calls, including ones still queued at cancellation).
2026-07-06 00:48:49 +08:00
**Meta extraction**: a string/comment-aware brace scanner (template interpolation rejected) finds the literal; it is evaluated ALONE in an empty, timed vm context; the result must materialize to plain JSON data and pass shape validation (unknown fields rejected loud); the statement is blanked line-preservingly so stacks keep script line numbers.
workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
**Value boundary**: values leaving the script (meta, hook options, schemas, the return value) go through `materializeFromRealm` — a plain recursive walk that rejects loud everything JSON cannot carry (exotic prototypes, functions, symbols, cycles, sparse arrays, non-finite numbers, nested `undefined`), copying via `Object.defineProperty` so a `"__proto__"` key becomes a data property, never a prototype mutation; getters are read ordinarily and their RESULT crosses (a throwing read fails loud) — which is also what makes every later postMessage hop total. Values entering the realm (`args`, `agent()` results, hook promises and failures, combinator arrays) are handed over directly as worker-realm values — the script is trusted, so outer prototypes are not a leak; `args` rides the `workerData` structured clone (the caller-isolation copy) and is cloned once more so a script scribbling on it cannot mutate the session's init object. Hook failures are `WorkflowError`s built OUTSIDE the script's context: the combinators recognize fatality by `instanceof` against the engine's own class (unforgeable from the script), and the script-visible consequence — in-script `instanceof Error` is `false` for hook errors; branch on `e.name`/`e.code` — is documented in the engine README. Realm functions (stages, thunks) are called, never materialized. Thrown script values are rendered by a total renderer (stack → message → `String()`, fixed label if rendering throws), so `result` cannot reject. Caps (`maxConcurrentAgents` auto = `min(16, max(1, availableParallelism() - 2))`, `maxTotalAgents` 1000, `maxItemsPerCall` 4096) and timeouts are validated Config, not literals.
workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
### The consumer (dsh-tool-workflow)
A `workflow` tool mirroring `dsh-tool-subagent`'s synchronous shape: start, await, `try/finally` dispose, abort-bridge `exec.signal`, non-`completed` → `isError`. Render intent: a `generic` card titled by a textual `meta.name` sniff (presentation is a pure function of args). The tool description IS the model-facing authoring spec. The usage policy ships with the tool as its own `tool:<toolName>` prompt section (explicit-ask-only guidance — tool guidance lives in tool plugins, never in the deployment persona); the harness has no ultracode-style effort gate.
workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
### The foundation: structured output on the subagent seam
`agent({schema})` needs `SubagentStartRequest.outputSchema` to actually work; it was vocabulary without an implementation (`outputSchema: false` everywhere). Implemented in `dsh-subagent-inprocess` for both in-process backends: a globally registered `structured_output` capture tool whose per-child schema is enforced by a `prepend: true` `system-prompt/assemble` listener doing FINAL-ASSEMBLY enforcement (post-processing `await next()` — cooperative mutation would not survive a downstream listener returning a replacement assembly; the calling instruction rides as a trailing prompt section, since `AgentOptions` carries no per-agent prompt field, and the loop logs the result as the step's `request/header`, keeping the injection reconstructable), a `prepend: true` `agent/turn-continuation` veto after capture (no wasted extra model step) plus a `tools/pre-execute` deny for calls arriving after the capture (terminal within the step, not only at its end), and validation-retry in-turn via `ToolArgsError`. The schema is `structuredClone`d at `start()` (caller mutation cannot drift enforcement). Deliberately NO re-prompt: a child that finishes cleanly without calling the tool settles `error` to the parent. Lifetime is refcounted by backends (plugin lifetime) AND live runs (start → settle). The seam's `outputSchema` type became the raw JSON-Schema SUBSET (`StructuredOutputSchema` in dsh-tools: single-string `type`, `properties`/`required`/`additionalProperties`, `items`, scalar `enum`/`const`; anything unenforced is rejected loud) — the schema travels verbatim to the model as the forced tool's parameters, so the wire format, not the author DSL, is the right vocabulary.
workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
## Deferred (documented non-goals of this cut)
- **Background collection** (start tool → run id → completion notice → collect), designed alongside bash/subagent background unification.
- **Journaling + resume** (`resumeFromRunId`, cached agent() prefixes) — implementing it reintroduces CC's determinism bans as a script-contract tightening (scripts may read the clock today).
workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
- **Saved/bundled workflows** (a `.deepseek/workflows/` registry, slash-command surface) and **script persistence to a run directory** (the tool-call event already records the script durably).
- **Nested `workflow()`**, **token `budget`**, and the `effort`/`isolation`/`agentType` agent options (each rejects loud with a message naming it deferred).
Merge remote-tracking branch 'origin/master' into worktree-dynamic-workflows Beyond the mechanical conflicts (provider capability lines vs master's new inheritsParentContext field; generated catalogs regenerated rather than hand-merged; knip/lockfile), three master-side reworks required semantic adaptation of this branch: - The persona rework removed AgentOptions.systemPrompt, which was the structured-output instruction's channel. The instruction now rides the SAME final-request enforcement listener that injects the schema'd tool: appended per request to final.system (per-request wire state, not agent prompt state). Tests assert the wire request (adapter.requests) instead of child.options; the bare-direct-dispatch test pins the no-system arm. - Tool guidance moved out of deployment prompts into per-tool prompt sections; the examples' workflow paragraph became a tool:<toolName> section contributed by dsh-tool-workflow (explicit-ask-only policy), and both example personas resolve to master's minimal identity+behavior form. tool-workflow gains inject: systemPrompt (+ peer dep, tsconfig ref); the export-shape guard updated. - The uniform-RFC-format gate: the dynamic-workflows RFC restructured to the implemented/ skeleton (bare Status line; Proposal -> Decision; What-was-rejected -> Alternatives considered; new Consequences), and the overall-run-timeout deferral is now recorded in the RFC's Deferred list. The doc-graphs atlas classification gains the workflows seam (workflow-vm implementation, tool-workflow consumer). Master's harness-identity section made "empty assembled prompt" states unreachable through the loop, so the instruction-append is a plain undefined-ternary and the structured tests assert append-not-replace. All snapshot goldens (including workflow-run) replay unchanged. Full local CI-equivalent gate sequence green on the merged tree.
2026-07-06 03:14:07 +08:00
- **An overall run wall-clock timeout** — cancellation always frees the caller (result settles within the grace), so a cap on total run time is a policy knob for the background redesign, not a correctness need here.
- **Engine hardening beyond worker threads**: an isolated-vm or separate-process engine behind the same seam (actual sandboxing; memory limits).
workflow: dynamic workflows — script-driven multi-agent orchestration A new capability family at packages/workflow/ in the bash seam shape, modeled on Claude Code's dynamic workflows: the model writes a JavaScript orchestration script (export const meta = {...} + plain-JS body), a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results. - dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events carrying data snapshots (id + meta, never the live run), per-listener contained like subagent/*. - dsh-workflow-vm: in-process node:vm engine. Meta extraction via a string/comment-aware scanner (template interpolation rejected; literal evaluated alone in an empty timed context; statement blanked line- preservingly so stacks keep script line numbers). Hooks: agent(prompt, {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline() (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline: hook misuse (unknown/deferred options, bad arguments, unsupported schemas, tripped caps, seam start failures, cancellation) throws fatal WorkflowErrors the combinators RE-THROW — never dissolved into the per-item null reserved for child failures. Realm boundary: inbound values materialized by descriptor walks that never invoke accessors (defineProperty copies, __proto__-safe); outbound values rebuilt in-realm via the context's own JSON.parse. Determinism bans (Date.now/Math.random/argless new Date) kept so future resume support cannot break scripts. Caps and timeouts are validated Config. Every hook promise carries a no-op rejection consumer (app-boot exits on unhandled rejections). - dsh-tool-workflow: the model-facing workflow tool, synchronous like dsh-tool-subagent (start → await → try/finally dispose; abort bridged; non-completed → isError). Generic render card titled by a textual meta.name sniff. The tool description carries the authoring contract. Wired into examples/{coding-agent,acp-agent} with explicit-ask-only guidance. Coverage at every tier: unit (meta scanner, materializer incl. counting-getter and __proto__ regressions, combinator semantics, concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon), integration over the real spawn stack, with-key e2e (real two-phase run + the tool through the registry pipeline), and a recorded ACP snapshot scenario (workflow-run, 1 child session). RFC: docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's layout line.
2026-07-05 13:29:35 +08:00
- **ACP progress UI** over the `workflow/*` events (a `/workflows`-style view); the events exist for it.
- **ACP-backend structured output** and **`toolFilter`** (both still capability-gated `false`).
Merge remote-tracking branch 'origin/master' into worktree-dynamic-workflows Beyond the mechanical conflicts (provider capability lines vs master's new inheritsParentContext field; generated catalogs regenerated rather than hand-merged; knip/lockfile), three master-side reworks required semantic adaptation of this branch: - The persona rework removed AgentOptions.systemPrompt, which was the structured-output instruction's channel. The instruction now rides the SAME final-request enforcement listener that injects the schema'd tool: appended per request to final.system (per-request wire state, not agent prompt state). Tests assert the wire request (adapter.requests) instead of child.options; the bare-direct-dispatch test pins the no-system arm. - Tool guidance moved out of deployment prompts into per-tool prompt sections; the examples' workflow paragraph became a tool:<toolName> section contributed by dsh-tool-workflow (explicit-ask-only policy), and both example personas resolve to master's minimal identity+behavior form. tool-workflow gains inject: systemPrompt (+ peer dep, tsconfig ref); the export-shape guard updated. - The uniform-RFC-format gate: the dynamic-workflows RFC restructured to the implemented/ skeleton (bare Status line; Proposal -> Decision; What-was-rejected -> Alternatives considered; new Consequences), and the overall-run-timeout deferral is now recorded in the RFC's Deferred list. The doc-graphs atlas classification gains the workflows seam (workflow-vm implementation, tool-workflow consumer). Master's harness-identity section made "empty assembled prompt" states unreachable through the loop, so the instruction-append is a plain undefined-ternary and the structured tests assert append-not-replace. All snapshot goldens (including workflow-run) replay unchanged. Full local CI-equivalent gate sequence green on the merged tree.
2026-07-06 03:14:07 +08:00
## Alternatives considered
- **Hostile-value containment in the host** (trap-free proxy rejection, accessor-never-invoked descriptor walks, realm-side pre-rendering of thrown values, realm-built promises/arrays/error clones with structural fatal recognition): an earlier revision built all of it, and review showed the cost was real while the threat model was not — every one of those defenses guards against an author the premise already trusts. Removed in favor of the plain boundary above; the thread boundary makes such machinery redundant anyway (serialization by construction).
- **In-process `node:vm` execution** (the first cut of this RFC shipped it): mechanically simplest — no RPC, no thread — but `start()` blocks the caller for the script's initial synchronous slice, a synchronous spin past the first await cannot be killed in-process (the vm `timeout` covers only that first slice), and `dispose()` could only ABANDON an unsettling script, leaving the spin on the host loop. Superseded by the worker-thread engine, which keeps the same vm-context script surface while unblocking the host and making termination real.
Merge remote-tracking branch 'origin/master' into worktree-dynamic-workflows Beyond the mechanical conflicts (provider capability lines vs master's new inheritsParentContext field; generated catalogs regenerated rather than hand-merged; knip/lockfile), three master-side reworks required semantic adaptation of this branch: - The persona rework removed AgentOptions.systemPrompt, which was the structured-output instruction's channel. The instruction now rides the SAME final-request enforcement listener that injects the schema'd tool: appended per request to final.system (per-request wire state, not agent prompt state). Tests assert the wire request (adapter.requests) instead of child.options; the bare-direct-dispatch test pins the no-system arm. - Tool guidance moved out of deployment prompts into per-tool prompt sections; the examples' workflow paragraph became a tool:<toolName> section contributed by dsh-tool-workflow (explicit-ask-only policy), and both example personas resolve to master's minimal identity+behavior form. tool-workflow gains inject: systemPrompt (+ peer dep, tsconfig ref); the export-shape guard updated. - The uniform-RFC-format gate: the dynamic-workflows RFC restructured to the implemented/ skeleton (bare Status line; Proposal -> Decision; What-was-rejected -> Alternatives considered; new Consequences), and the overall-run-timeout deferral is now recorded in the RFC's Deferred list. The doc-graphs atlas classification gains the workflows seam (workflow-vm implementation, tool-workflow consumer). Master's harness-identity section made "empty assembled prompt" states unreachable through the loop, so the instruction-append is a plain undefined-ternary and the structured tests assert append-not-replace. All snapshot goldens (including workflow-run) replay unchanged. Full local CI-equivalent gate sequence green on the merged tree.
2026-07-06 03:14:07 +08:00
- **Background execution as the default** (CC's shape): deferred; foreground-synchronous matches `dsh-tool-subagent`'s cut, and background semantics should be designed ONCE across bash/subagent/workflow rather than per-tool.
- **Workflow-layer JSON parsing for `agent({schema})`**: duplicating a seam concern at one consumer while the seam's capability flag stayed dishonestly `false`.
- **Meta as tool parameters instead of `export const meta`**: zero parsing, but scripts stop being self-contained artifacts and CC-authored scripts stop being drop-in.
- **`SchemaSpec` as the outputSchema type**: the author-facing DSL cannot express what arrives as data and cannot be validated against without conversion loss.
- **A schema-object library (zod, or the repo's schemastery) for the structured-output subset**: the schema is wire data — plain JSON that crosses the vm realm boundary in `agent({schema})` and lands verbatim in the forced tool's parameters — exactly where live schema objects cannot sit; consuming raw JSON Schema at runtime would need a third-party converter on top (zod core only emits JSON Schema, not the reverse), and it would put a second schema language beside schemastery's config role.
- **ajv for value validation**: it validates FULL JSON Schema, so the subset gate — the module's actual point, since every accepted keyword must be one the harness enforces — would remain hand-written regardless; it compiles validators through `new Function`; and it would be dsh-tools' first runtime dependency, all to replace the ~70-line value walker while the path-qualified, every-violation error reporting stays custom either way.
- **Provider JSON mode (`response_format: {type: json_object}`) instead of the forced capture tool**: the official API guarantees valid JSON, not schema-conforming JSON (no `json_schema` type; the docs' own guidance is to validate client-side, with the schema riding in the prompt), so both walkers survive untouched and only the capture-tool mechanics could go — at the cost of tools during a structured child's run (whether `response_format` composes with tool calling is undocumented), the in-turn validation retry (`ToolArgsError` keeps recovery inside the turn; a JSON-mode empty body — a documented failure mode — ends the turn, and the only recovery is the re-prompt loop this design rejects), and a new per-adapter `LlmCallConfig` surface. The accepted upgrade path is strict TOOL schemas (provider-side constrained decoding on tool parameters) when available: the same forced tool and subset gate, with the gate narrowed to the provider's strict subset.
Merge remote-tracking branch 'origin/master' into worktree-dynamic-workflows Beyond the mechanical conflicts (provider capability lines vs master's new inheritsParentContext field; generated catalogs regenerated rather than hand-merged; knip/lockfile), three master-side reworks required semantic adaptation of this branch: - The persona rework removed AgentOptions.systemPrompt, which was the structured-output instruction's channel. The instruction now rides the SAME final-request enforcement listener that injects the schema'd tool: appended per request to final.system (per-request wire state, not agent prompt state). Tests assert the wire request (adapter.requests) instead of child.options; the bare-direct-dispatch test pins the no-system arm. - Tool guidance moved out of deployment prompts into per-tool prompt sections; the examples' workflow paragraph became a tool:<toolName> section contributed by dsh-tool-workflow (explicit-ask-only policy), and both example personas resolve to master's minimal identity+behavior form. tool-workflow gains inject: systemPrompt (+ peer dep, tsconfig ref); the export-shape guard updated. - The uniform-RFC-format gate: the dynamic-workflows RFC restructured to the implemented/ skeleton (bare Status line; Proposal -> Decision; What-was-rejected -> Alternatives considered; new Consequences), and the overall-run-timeout deferral is now recorded in the RFC's Deferred list. The doc-graphs atlas classification gains the workflows seam (workflow-vm implementation, tool-workflow consumer). Master's harness-identity section made "empty assembled prompt" states unreachable through the loop, so the instruction-append is a plain undefined-ternary and the structured tests assert append-not-replace. All snapshot goldens (including workflow-run) replay unchanged. Full local CI-equivalent gate sequence green on the merged tree.
2026-07-06 03:14:07 +08:00
## Consequences
The harness gains CC-compatible script orchestration: fan-out plans live in a rerunnable artifact instead of the parent context, and the structured-output half of the subagent seam is now real (the vocabulary stopped lying about `outputSchema`). What it cost, all bounded by the trust premise: a worker thread per run (~tens-of-ms spin-up), every hook crossing a message port as RPC, and a termination-path `agentsStarted` that degrades to the host-observed count; in exchange `start()` never blocks the host, a post-cancel grace ends in a real `worker.terminate()`, and the value boundary is serialization by construction. A worker thread is still NOT a security boundary — scripts share the model's trust level, and actual sandboxing names its exit (the isolated-vm/separate-process engine swap behind the seam). The fatal-vs-null strictness divergence from CC means a CC-authored script that RELIES on option typos dissolving to `null` behaves differently here — judged worth it to keep the repo's no-accepted-then-ignored rule. Consumers must hold the run handle for control (`cancel`/`dispose`); observers get data snapshots only, so no listener can extend a run's lifetime or corrupt another's view.