deepseek-harness/packages/subagent/subagent-spawn/tests/spawn.e2e.ts

46 lines
2 KiB
TypeScript
Raw Normal View History

import { createUserMessage } from '@deepseek-ai/dsh-llm'
Add in-process subagent backends: spawn (fresh) and fork (seeded) The second PR of the subagent seam: the two in-process backends that run a child agent on the same cordis context, reusing the agent factory's quiescent AgentHandle teardown. Both register on ctx.subagents (PR1's named-provider registry) and share one run driver. - dsh-subagent-spawn: a FRESH child via ctx.agents.create — own session, the parent's model by default (overridable), zero inherited conversation. Also exports the shared in-process run driver (startInProcessRun): mint ids, stamp cwd/parentSession-lineage/depth, drive the one-shot (send → whenIdle), read the last assistant/message + turn/end reason, dispose to quiescence. - dsh-subagent-fork: a child SEEDED with the parent's balanced completed-turn prefix (the log up to and including its last turn/end), so the child inherits context. The in-flight unbalanced turn is excluded — a raw seed would fail the invariants replay. Proven: a regression test goes red if the boundary seeds the open turn. - Seam extension: CreateAgentOptions.seed, threaded through AgentLoop.createAgent → ctx.sessions.prepare({ seed }) (the primitive resume already used). This is the fork-lineage path the TODO(sub-agents) markers anticipated. - Depth: a merge-extensible AgentOptions.subagentDepth (0 top-level, parent+1 for a child); the depthLimit capability refuses a spawn past request.maxDepth. Tests: real-loop unit tests for both backends (mock MODEL only, real loop + invariants), a multi-subagent test (one parent drives a fork AND a spawn child then keeps working), and a with-key e2e (a real parent delegates via the `subagent` tool to a real child that writes a file on disk — world-verified). 100% per-file coverage. The coding-agent demo wires the spawn backend + tool. Snapshot coverage of nested agents is deferred to a stacked follow-up (TODO(subagent-snapshots)): dsh-llm-replay is a single global positional cursor that cannot route calls to a parent vs. a child on one context. Recorded in the RFC's deferrals and a new AGENTS.md rule: designing a subsystem must design its test infrastructure END TO END up front, verifying the snapshot/e2e harness can express the new shape — a gap this plan hit.
2026-06-22 05:58:40 +08:00
import { mkdtemp, readFile, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, describe, expect, it } from 'vitest'
import type { Context } from 'cordis'
import { spawnHarness, waitForIdle } from './harness.ts'
import { SessionId } from '@deepseek-ai/dsh-session'
Add in-process subagent backends: spawn (fresh) and fork (seeded) The second PR of the subagent seam: the two in-process backends that run a child agent on the same cordis context, reusing the agent factory's quiescent AgentHandle teardown. Both register on ctx.subagents (PR1's named-provider registry) and share one run driver. - dsh-subagent-spawn: a FRESH child via ctx.agents.create — own session, the parent's model by default (overridable), zero inherited conversation. Also exports the shared in-process run driver (startInProcessRun): mint ids, stamp cwd/parentSession-lineage/depth, drive the one-shot (send → whenIdle), read the last assistant/message + turn/end reason, dispose to quiescence. - dsh-subagent-fork: a child SEEDED with the parent's balanced completed-turn prefix (the log up to and including its last turn/end), so the child inherits context. The in-flight unbalanced turn is excluded — a raw seed would fail the invariants replay. Proven: a regression test goes red if the boundary seeds the open turn. - Seam extension: CreateAgentOptions.seed, threaded through AgentLoop.createAgent → ctx.sessions.prepare({ seed }) (the primitive resume already used). This is the fork-lineage path the TODO(sub-agents) markers anticipated. - Depth: a merge-extensible AgentOptions.subagentDepth (0 top-level, parent+1 for a child); the depthLimit capability refuses a spawn past request.maxDepth. Tests: real-loop unit tests for both backends (mock MODEL only, real loop + invariants), a multi-subagent test (one parent drives a fork AND a spawn child then keeps working), and a with-key e2e (a real parent delegates via the `subagent` tool to a real child that writes a file on disk — world-verified). 100% per-file coverage. The coding-agent demo wires the spawn backend + tool. Snapshot coverage of nested agents is deferred to a stacked follow-up (TODO(subagent-snapshots)): dsh-llm-replay is a single global positional cursor that cannot route calls to a parent vs. a child on one context. Recorded in the RFC's deferrals and a new AGENTS.md rule: designing a subsystem must design its test infrastructure END TO END up front, verifying the snapshot/e2e harness can express the new shape — a gap this plan hit.
2026-06-22 05:58:40 +08:00
2026-07-25 13:02:37 +08:00
/** Key-gated smoke for a real parent delegating filesystem work to a real child. */
Add in-process subagent backends: spawn (fresh) and fork (seeded) The second PR of the subagent seam: the two in-process backends that run a child agent on the same cordis context, reusing the agent factory's quiescent AgentHandle teardown. Both register on ctx.subagents (PR1's named-provider registry) and share one run driver. - dsh-subagent-spawn: a FRESH child via ctx.agents.create — own session, the parent's model by default (overridable), zero inherited conversation. Also exports the shared in-process run driver (startInProcessRun): mint ids, stamp cwd/parentSession-lineage/depth, drive the one-shot (send → whenIdle), read the last assistant/message + turn/end reason, dispose to quiescence. - dsh-subagent-fork: a child SEEDED with the parent's balanced completed-turn prefix (the log up to and including its last turn/end), so the child inherits context. The in-flight unbalanced turn is excluded — a raw seed would fail the invariants replay. Proven: a regression test goes red if the boundary seeds the open turn. - Seam extension: CreateAgentOptions.seed, threaded through AgentLoop.createAgent → ctx.sessions.prepare({ seed }) (the primitive resume already used). This is the fork-lineage path the TODO(sub-agents) markers anticipated. - Depth: a merge-extensible AgentOptions.subagentDepth (0 top-level, parent+1 for a child); the depthLimit capability refuses a spawn past request.maxDepth. Tests: real-loop unit tests for both backends (mock MODEL only, real loop + invariants), a multi-subagent test (one parent drives a fork AND a spawn child then keeps working), and a with-key e2e (a real parent delegates via the `subagent` tool to a real child that writes a file on disk — world-verified). 100% per-file coverage. The coding-agent demo wires the spawn backend + tool. Snapshot coverage of nested agents is deferred to a stacked follow-up (TODO(subagent-snapshots)): dsh-llm-replay is a single global positional cursor that cannot route calls to a parent vs. a child on one context. Recorded in the RFC's deferrals and a new AGENTS.md rule: designing a subsystem must design its test infrastructure END TO END up front, verifying the snapshot/e2e harness can express the new shape — a gap this plan hit.
2026-06-22 05:58:40 +08:00
let ctx: Context | undefined
let workdir: string | undefined
afterEach(async () => {
await ctx?.fiber.dispose()
ctx = undefined
if (workdir !== undefined) await rm(workdir, { recursive: true, force: true })
workdir = undefined
})
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('spawn backend with-key smoke', () => {
it('a parent delegates to a child that writes a file on disk', async () => {
workdir = await mkdtemp(join(tmpdir(), 'dsh-subagent-spawn-e2e-'))
ctx = await spawnHarness(workdir)
const parent = ctx.agentLoop.create(SessionId('e2e-parent'), { provider: 'deepseek-official', model: 'deepseek-v4-flash' })
Add in-process subagent backends: spawn (fresh) and fork (seeded) The second PR of the subagent seam: the two in-process backends that run a child agent on the same cordis context, reusing the agent factory's quiescent AgentHandle teardown. Both register on ctx.subagents (PR1's named-provider registry) and share one run driver. - dsh-subagent-spawn: a FRESH child via ctx.agents.create — own session, the parent's model by default (overridable), zero inherited conversation. Also exports the shared in-process run driver (startInProcessRun): mint ids, stamp cwd/parentSession-lineage/depth, drive the one-shot (send → whenIdle), read the last assistant/message + turn/end reason, dispose to quiescence. - dsh-subagent-fork: a child SEEDED with the parent's balanced completed-turn prefix (the log up to and including its last turn/end), so the child inherits context. The in-flight unbalanced turn is excluded — a raw seed would fail the invariants replay. Proven: a regression test goes red if the boundary seeds the open turn. - Seam extension: CreateAgentOptions.seed, threaded through AgentLoop.createAgent → ctx.sessions.prepare({ seed }) (the primitive resume already used). This is the fork-lineage path the TODO(sub-agents) markers anticipated. - Depth: a merge-extensible AgentOptions.subagentDepth (0 top-level, parent+1 for a child); the depthLimit capability refuses a spawn past request.maxDepth. Tests: real-loop unit tests for both backends (mock MODEL only, real loop + invariants), a multi-subagent test (one parent drives a fork AND a spawn child then keeps working), and a with-key e2e (a real parent delegates via the `subagent` tool to a real child that writes a file on disk — world-verified). 100% per-file coverage. The coding-agent demo wires the spawn backend + tool. Snapshot coverage of nested agents is deferred to a stacked follow-up (TODO(subagent-snapshots)): dsh-llm-replay is a single global positional cursor that cannot route calls to a parent vs. a child on one context. Recorded in the RFC's deferrals and a new AGENTS.md rule: designing a subsystem must design its test infrastructure END TO END up front, verifying the snapshot/e2e harness can express the new shape — a gap this plan hit.
2026-06-22 05:58:40 +08:00
parent.followup(createUserMessage({
content: [{ type: 'text', text:
Add in-process subagent backends: spawn (fresh) and fork (seeded) The second PR of the subagent seam: the two in-process backends that run a child agent on the same cordis context, reusing the agent factory's quiescent AgentHandle teardown. Both register on ctx.subagents (PR1's named-provider registry) and share one run driver. - dsh-subagent-spawn: a FRESH child via ctx.agents.create — own session, the parent's model by default (overridable), zero inherited conversation. Also exports the shared in-process run driver (startInProcessRun): mint ids, stamp cwd/parentSession-lineage/depth, drive the one-shot (send → whenIdle), read the last assistant/message + turn/end reason, dispose to quiescence. - dsh-subagent-fork: a child SEEDED with the parent's balanced completed-turn prefix (the log up to and including its last turn/end), so the child inherits context. The in-flight unbalanced turn is excluded — a raw seed would fail the invariants replay. Proven: a regression test goes red if the boundary seeds the open turn. - Seam extension: CreateAgentOptions.seed, threaded through AgentLoop.createAgent → ctx.sessions.prepare({ seed }) (the primitive resume already used). This is the fork-lineage path the TODO(sub-agents) markers anticipated. - Depth: a merge-extensible AgentOptions.subagentDepth (0 top-level, parent+1 for a child); the depthLimit capability refuses a spawn past request.maxDepth. Tests: real-loop unit tests for both backends (mock MODEL only, real loop + invariants), a multi-subagent test (one parent drives a fork AND a spawn child then keeps working), and a with-key e2e (a real parent delegates via the `subagent` tool to a real child that writes a file on disk — world-verified). 100% per-file coverage. The coding-agent demo wires the spawn backend + tool. Snapshot coverage of nested agents is deferred to a stacked follow-up (TODO(subagent-snapshots)): dsh-llm-replay is a single global positional cursor that cannot route calls to a parent vs. a child on one context. Recorded in the RFC's deferrals and a new AGENTS.md rule: designing a subsystem must design its test infrastructure END TO END up front, verifying the snapshot/e2e harness can express the new shape — a gap this plan hit.
2026-06-22 05:58:40 +08:00
'Use the subagent tool to delegate this exact task: "Use the bash tool to write the text '
+ 'SUBAGENT_WAS_HERE into a file named proof.txt in the current directory." '
+ 'After the subagent finishes, tell me it is done.' }], source: { kind: 'user' } }))
Add in-process subagent backends: spawn (fresh) and fork (seeded) The second PR of the subagent seam: the two in-process backends that run a child agent on the same cordis context, reusing the agent factory's quiescent AgentHandle teardown. Both register on ctx.subagents (PR1's named-provider registry) and share one run driver. - dsh-subagent-spawn: a FRESH child via ctx.agents.create — own session, the parent's model by default (overridable), zero inherited conversation. Also exports the shared in-process run driver (startInProcessRun): mint ids, stamp cwd/parentSession-lineage/depth, drive the one-shot (send → whenIdle), read the last assistant/message + turn/end reason, dispose to quiescence. - dsh-subagent-fork: a child SEEDED with the parent's balanced completed-turn prefix (the log up to and including its last turn/end), so the child inherits context. The in-flight unbalanced turn is excluded — a raw seed would fail the invariants replay. Proven: a regression test goes red if the boundary seeds the open turn. - Seam extension: CreateAgentOptions.seed, threaded through AgentLoop.createAgent → ctx.sessions.prepare({ seed }) (the primitive resume already used). This is the fork-lineage path the TODO(sub-agents) markers anticipated. - Depth: a merge-extensible AgentOptions.subagentDepth (0 top-level, parent+1 for a child); the depthLimit capability refuses a spawn past request.maxDepth. Tests: real-loop unit tests for both backends (mock MODEL only, real loop + invariants), a multi-subagent test (one parent drives a fork AND a spawn child then keeps working), and a with-key e2e (a real parent delegates via the `subagent` tool to a real child that writes a file on disk — world-verified). 100% per-file coverage. The coding-agent demo wires the spawn backend + tool. Snapshot coverage of nested agents is deferred to a stacked follow-up (TODO(subagent-snapshots)): dsh-llm-replay is a single global positional cursor that cannot route calls to a parent vs. a child on one context. Recorded in the RFC's deferrals and a new AGENTS.md rule: designing a subsystem must design its test infrastructure END TO END up front, verifying the snapshot/e2e harness can express the new shape — a gap this plan hit.
2026-06-22 05:58:40 +08:00
await waitForIdle(ctx, parent)
2026-07-25 13:02:37 +08:00
// Assert the filesystem effect independently of the model response.
Add in-process subagent backends: spawn (fresh) and fork (seeded) The second PR of the subagent seam: the two in-process backends that run a child agent on the same cordis context, reusing the agent factory's quiescent AgentHandle teardown. Both register on ctx.subagents (PR1's named-provider registry) and share one run driver. - dsh-subagent-spawn: a FRESH child via ctx.agents.create — own session, the parent's model by default (overridable), zero inherited conversation. Also exports the shared in-process run driver (startInProcessRun): mint ids, stamp cwd/parentSession-lineage/depth, drive the one-shot (send → whenIdle), read the last assistant/message + turn/end reason, dispose to quiescence. - dsh-subagent-fork: a child SEEDED with the parent's balanced completed-turn prefix (the log up to and including its last turn/end), so the child inherits context. The in-flight unbalanced turn is excluded — a raw seed would fail the invariants replay. Proven: a regression test goes red if the boundary seeds the open turn. - Seam extension: CreateAgentOptions.seed, threaded through AgentLoop.createAgent → ctx.sessions.prepare({ seed }) (the primitive resume already used). This is the fork-lineage path the TODO(sub-agents) markers anticipated. - Depth: a merge-extensible AgentOptions.subagentDepth (0 top-level, parent+1 for a child); the depthLimit capability refuses a spawn past request.maxDepth. Tests: real-loop unit tests for both backends (mock MODEL only, real loop + invariants), a multi-subagent test (one parent drives a fork AND a spawn child then keeps working), and a with-key e2e (a real parent delegates via the `subagent` tool to a real child that writes a file on disk — world-verified). 100% per-file coverage. The coding-agent demo wires the spawn backend + tool. Snapshot coverage of nested agents is deferred to a stacked follow-up (TODO(subagent-snapshots)): dsh-llm-replay is a single global positional cursor that cannot route calls to a parent vs. a child on one context. Recorded in the RFC's deferrals and a new AGENTS.md rule: designing a subsystem must design its test infrastructure END TO END up front, verifying the snapshot/e2e harness can express the new shape — a gap this plan hit.
2026-06-22 05:58:40 +08:00
const proof = await readFile(join(workdir, 'proof.txt'), 'utf8')
expect(proof).toContain('SUBAGENT_WAS_HERE')
// The parent's log records the subagent tool/call + its result (not the
// child's internal steps).
const events = [...parent.session.events]
const subagentCalls = events.filter(e => e.type === 'tool/call' && e.data.name === 'subagent')
expect(subagentCalls.length).toBeGreaterThan(0)
}, 180_000)
})