deepseek-harness/examples/acp-agent/tests/hooks.e2e.ts

73 lines
3 KiB
TypeScript
Raw Normal View History

2026-07-14 14:11:33 +08:00
import { mkdtemp, writeFile, access } from 'node:fs/promises'
feat(hooks): dsh-hooks-claude + dsh-hooks-codex bridges (hooks stack PR-F) The two bridge plugins that run a user's existing Claude Code / Codex hook config on the harness's typed interception seams, built on the shared dsh-hook-protocol library. A bridge is a faithfulness adapter, not a power tool: anything it does a native cordis plugin does more powerfully — the bridge exists only to run UNMODIFIED external hooks. - dsh-hooks-claude: CC dialect. Seven hook points (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SubagentStart, SubagentStop), CC per-event stdin payloads, env + ${CLAUDE_PLUGIN_ROOT}/ ${CLAUDE_PROJECT_DIR} substitution, literal-or-regex matcher. - dsh-hooks-codex: Codex dialect — a deliberate subset. Five hook points, always-regex matcher, snake_case payloads (turn_id/model, no trailing newline), no env/substitution, block-only decisions. Both map the neutral merged outcome onto the seam's typed Decision and stamp an explicit {kind:'plugin'} source on injected context (so it is never mislabeled as a user prompt). Config parse-failure is contained; only command hooks run. updatedInput is logged+warned (input rewrite deferred); the Stop loop-guard is deferred (TODO). Tests: per-file 100% — config-parse unit branches + per-seam mappings end-to-end through the REAL loop + REAL bash + REAL shell scripts (scripted mock model only) + a real-Loader export-shape guard. A keyless ACP snapshot scenario (hook-prompt-block) proves a UserPromptSubmit hook blocks a prompt end-to-end (rejected turn -> ACP cancelled, hook/* events in the log); a with-key e2e (hooks.e2e.ts) proves a PreToolUse hook blocks real bash (verified on disk). The snapshot normalizer now scrubs hook/result.durationMs. RFC: docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
2026-07-01 04:22:00 +08:00
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { afterEach, describe, expect, it } from 'vitest'
2026-07-14 00:18:00 +08:00
import { PROTOCOL_VERSION } from '@agentclientprotocol/sdk'
feat(hooks): dsh-hooks-claude + dsh-hooks-codex bridges (hooks stack PR-F) The two bridge plugins that run a user's existing Claude Code / Codex hook config on the harness's typed interception seams, built on the shared dsh-hook-protocol library. A bridge is a faithfulness adapter, not a power tool: anything it does a native cordis plugin does more powerfully — the bridge exists only to run UNMODIFIED external hooks. - dsh-hooks-claude: CC dialect. Seven hook points (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SubagentStart, SubagentStop), CC per-event stdin payloads, env + ${CLAUDE_PLUGIN_ROOT}/ ${CLAUDE_PROJECT_DIR} substitution, literal-or-regex matcher. - dsh-hooks-codex: Codex dialect — a deliberate subset. Five hook points, always-regex matcher, snake_case payloads (turn_id/model, no trailing newline), no env/substitution, block-only decisions. Both map the neutral merged outcome onto the seam's typed Decision and stamp an explicit {kind:'plugin'} source on injected context (so it is never mislabeled as a user prompt). Config parse-failure is contained; only command hooks run. updatedInput is logged+warned (input rewrite deferred); the Stop loop-guard is deferred (TODO). Tests: per-file 100% — config-parse unit branches + per-seam mappings end-to-end through the REAL loop + REAL bash + REAL shell scripts (scripted mock model only) + a real-Loader export-shape guard. A keyless ACP snapshot scenario (hook-prompt-block) proves a UserPromptSubmit hook blocks a prompt end-to-end (rejected turn -> ACP cancelled, hook/* events in the log); a with-key e2e (hooks.e2e.ts) proves a PreToolUse hook blocks real bash (verified on disk). The snapshot normalizer now scrubs hook/result.durationMs. RFC: docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
2026-07-01 04:22:00 +08:00
import {
2026-07-14 00:18:00 +08:00
launchAcpTestAgent,
type AgentUnderTest,
type LaunchedAcpTestAgent,
} from '@deepseek-ai/dsh-acp-snapshot'
2026-07-14 14:11:33 +08:00
import { cleanupAcpExampleTest } from './cleanup.ts'
feat(hooks): dsh-hooks-claude + dsh-hooks-codex bridges (hooks stack PR-F) The two bridge plugins that run a user's existing Claude Code / Codex hook config on the harness's typed interception seams, built on the shared dsh-hook-protocol library. A bridge is a faithfulness adapter, not a power tool: anything it does a native cordis plugin does more powerfully — the bridge exists only to run UNMODIFIED external hooks. - dsh-hooks-claude: CC dialect. Seven hook points (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SubagentStart, SubagentStop), CC per-event stdin payloads, env + ${CLAUDE_PLUGIN_ROOT}/ ${CLAUDE_PROJECT_DIR} substitution, literal-or-regex matcher. - dsh-hooks-codex: Codex dialect — a deliberate subset. Five hook points, always-regex matcher, snake_case payloads (turn_id/model, no trailing newline), no env/substitution, block-only decisions. Both map the neutral merged outcome onto the seam's typed Decision and stamp an explicit {kind:'plugin'} source on injected context (so it is never mislabeled as a user prompt). Config parse-failure is contained; only command hooks run. updatedInput is logged+warned (input rewrite deferred); the Stop loop-guard is deferred (TODO). Tests: per-file 100% — config-parse unit branches + per-seam mappings end-to-end through the REAL loop + REAL bash + REAL shell scripts (scripted mock model only) + a real-Loader export-shape guard. A keyless ACP snapshot scenario (hook-prompt-block) proves a UserPromptSubmit hook blocks a prompt end-to-end (rejected turn -> ACP cancelled, hook/* events in the log); a with-key e2e (hooks.e2e.ts) proves a PreToolUse hook blocks real bash (verified on disk). The snapshot normalizer now scrubs hook/result.durationMs. RFC: docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
2026-07-01 04:22:00 +08:00
/**
* With-key e2e for the Claude hook bridge. The process-level `./hooks.json` is
* resolved from a temporary launch cwd and blocks all PreToolUse calls; a real
* model is asked to write there, and absence of the file proves interception.
* The test owns and disposes the ACP subprocess.
feat(hooks): dsh-hooks-claude + dsh-hooks-codex bridges (hooks stack PR-F) The two bridge plugins that run a user's existing Claude Code / Codex hook config on the harness's typed interception seams, built on the shared dsh-hook-protocol library. A bridge is a faithfulness adapter, not a power tool: anything it does a native cordis plugin does more powerfully — the bridge exists only to run UNMODIFIED external hooks. - dsh-hooks-claude: CC dialect. Seven hook points (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SubagentStart, SubagentStop), CC per-event stdin payloads, env + ${CLAUDE_PLUGIN_ROOT}/ ${CLAUDE_PROJECT_DIR} substitution, literal-or-regex matcher. - dsh-hooks-codex: Codex dialect — a deliberate subset. Five hook points, always-regex matcher, snake_case payloads (turn_id/model, no trailing newline), no env/substitution, block-only decisions. Both map the neutral merged outcome onto the seam's typed Decision and stamp an explicit {kind:'plugin'} source on injected context (so it is never mislabeled as a user prompt). Config parse-failure is contained; only command hooks run. updatedInput is logged+warned (input rewrite deferred); the Stop loop-guard is deferred (TODO). Tests: per-file 100% — config-parse unit branches + per-seam mappings end-to-end through the REAL loop + REAL bash + REAL shell scripts (scripted mock model only) + a real-Loader export-shape guard. A keyless ACP snapshot scenario (hook-prompt-block) proves a UserPromptSubmit hook blocks a prompt end-to-end (rejected turn -> ACP cancelled, hook/* events in the log); a with-key e2e (hooks.e2e.ts) proves a PreToolUse hook blocks real bash (verified on disk). The snapshot normalizer now scrubs hook/result.durationMs. RFC: docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
2026-07-01 04:22:00 +08:00
*/
2026-07-14 00:18:00 +08:00
const AGENT: AgentUnderTest = {
binScript: fileURLToPath(new URL('../../../packages/examples/acp-demo/src/bin.ts', import.meta.url)),
2026-07-14 00:18:00 +08:00
configPath: fileURLToPath(new URL('../cordis.yml', import.meta.url)),
tsconfigPath: fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)),
feat(hooks): dsh-hooks-claude + dsh-hooks-codex bridges (hooks stack PR-F) The two bridge plugins that run a user's existing Claude Code / Codex hook config on the harness's typed interception seams, built on the shared dsh-hook-protocol library. A bridge is a faithfulness adapter, not a power tool: anything it does a native cordis plugin does more powerfully — the bridge exists only to run UNMODIFIED external hooks. - dsh-hooks-claude: CC dialect. Seven hook points (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SubagentStart, SubagentStop), CC per-event stdin payloads, env + ${CLAUDE_PLUGIN_ROOT}/ ${CLAUDE_PROJECT_DIR} substitution, literal-or-regex matcher. - dsh-hooks-codex: Codex dialect — a deliberate subset. Five hook points, always-regex matcher, snake_case payloads (turn_id/model, no trailing newline), no env/substitution, block-only decisions. Both map the neutral merged outcome onto the seam's typed Decision and stamp an explicit {kind:'plugin'} source on injected context (so it is never mislabeled as a user prompt). Config parse-failure is contained; only command hooks run. updatedInput is logged+warned (input rewrite deferred); the Stop loop-guard is deferred (TODO). Tests: per-file 100% — config-parse unit branches + per-seam mappings end-to-end through the REAL loop + REAL bash + REAL shell scripts (scripted mock model only) + a real-Loader export-shape guard. A keyless ACP snapshot scenario (hook-prompt-block) proves a UserPromptSubmit hook blocks a prompt end-to-end (rejected turn -> ACP cancelled, hook/* events in the log); a with-key e2e (hooks.e2e.ts) proves a PreToolUse hook blocks real bash (verified on disk). The snapshot normalizer now scrubs hook/result.durationMs. RFC: docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
2026-07-01 04:22:00 +08:00
}
2026-07-14 00:18:00 +08:00
let spawned: LaunchedAcpTestAgent | undefined
feat(hooks): dsh-hooks-claude + dsh-hooks-codex bridges (hooks stack PR-F) The two bridge plugins that run a user's existing Claude Code / Codex hook config on the harness's typed interception seams, built on the shared dsh-hook-protocol library. A bridge is a faithfulness adapter, not a power tool: anything it does a native cordis plugin does more powerfully — the bridge exists only to run UNMODIFIED external hooks. - dsh-hooks-claude: CC dialect. Seven hook points (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SubagentStart, SubagentStop), CC per-event stdin payloads, env + ${CLAUDE_PLUGIN_ROOT}/ ${CLAUDE_PROJECT_DIR} substitution, literal-or-regex matcher. - dsh-hooks-codex: Codex dialect — a deliberate subset. Five hook points, always-regex matcher, snake_case payloads (turn_id/model, no trailing newline), no env/substitution, block-only decisions. Both map the neutral merged outcome onto the seam's typed Decision and stamp an explicit {kind:'plugin'} source on injected context (so it is never mislabeled as a user prompt). Config parse-failure is contained; only command hooks run. updatedInput is logged+warned (input rewrite deferred); the Stop loop-guard is deferred (TODO). Tests: per-file 100% — config-parse unit branches + per-seam mappings end-to-end through the REAL loop + REAL bash + REAL shell scripts (scripted mock model only) + a real-Loader export-shape guard. A keyless ACP snapshot scenario (hook-prompt-block) proves a UserPromptSubmit hook blocks a prompt end-to-end (rejected turn -> ACP cancelled, hook/* events in the log); a with-key e2e (hooks.e2e.ts) proves a PreToolUse hook blocks real bash (verified on disk). The snapshot normalizer now scrubs hook/result.durationMs. RFC: docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
2026-07-01 04:22:00 +08:00
let workdir: string | undefined
afterEach(async () => {
2026-07-14 14:11:33 +08:00
const ownedSpawned = spawned
const ownedWorkdir = workdir
spawned = undefined
workdir = undefined
await cleanupAcpExampleTest(ownedSpawned, ownedWorkdir)
feat(hooks): dsh-hooks-claude + dsh-hooks-codex bridges (hooks stack PR-F) The two bridge plugins that run a user's existing Claude Code / Codex hook config on the harness's typed interception seams, built on the shared dsh-hook-protocol library. A bridge is a faithfulness adapter, not a power tool: anything it does a native cordis plugin does more powerfully — the bridge exists only to run UNMODIFIED external hooks. - dsh-hooks-claude: CC dialect. Seven hook points (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SubagentStart, SubagentStop), CC per-event stdin payloads, env + ${CLAUDE_PLUGIN_ROOT}/ ${CLAUDE_PROJECT_DIR} substitution, literal-or-regex matcher. - dsh-hooks-codex: Codex dialect — a deliberate subset. Five hook points, always-regex matcher, snake_case payloads (turn_id/model, no trailing newline), no env/substitution, block-only decisions. Both map the neutral merged outcome onto the seam's typed Decision and stamp an explicit {kind:'plugin'} source on injected context (so it is never mislabeled as a user prompt). Config parse-failure is contained; only command hooks run. updatedInput is logged+warned (input rewrite deferred); the Stop loop-guard is deferred (TODO). Tests: per-file 100% — config-parse unit branches + per-seam mappings end-to-end through the REAL loop + REAL bash + REAL shell scripts (scripted mock model only) + a real-Loader export-shape guard. A keyless ACP snapshot scenario (hook-prompt-block) proves a UserPromptSubmit hook blocks a prompt end-to-end (rejected turn -> ACP cancelled, hook/* events in the log); a with-key e2e (hooks.e2e.ts) proves a PreToolUse hook blocks real bash (verified on disk). The snapshot normalizer now scrubs hook/result.durationMs. RFC: docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
2026-07-01 04:22:00 +08:00
})
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('acp-agent e2e: a PreToolUse hook blocks bash (real model)', () => {
it('denies every bash command, so the requested file is never written (verified on disk)', async () => {
workdir = await mkdtemp(join(tmpdir(), 'acp-hooks-e2e-'))
// `configPath` is process-relative, so placing the match-all hook in the
// launch cwd selects it; hook commands themselves run in the session cwd.
feat(hooks): dsh-hooks-claude + dsh-hooks-codex bridges (hooks stack PR-F) The two bridge plugins that run a user's existing Claude Code / Codex hook config on the harness's typed interception seams, built on the shared dsh-hook-protocol library. A bridge is a faithfulness adapter, not a power tool: anything it does a native cordis plugin does more powerfully — the bridge exists only to run UNMODIFIED external hooks. - dsh-hooks-claude: CC dialect. Seven hook points (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SubagentStart, SubagentStop), CC per-event stdin payloads, env + ${CLAUDE_PLUGIN_ROOT}/ ${CLAUDE_PROJECT_DIR} substitution, literal-or-regex matcher. - dsh-hooks-codex: Codex dialect — a deliberate subset. Five hook points, always-regex matcher, snake_case payloads (turn_id/model, no trailing newline), no env/substitution, block-only decisions. Both map the neutral merged outcome onto the seam's typed Decision and stamp an explicit {kind:'plugin'} source on injected context (so it is never mislabeled as a user prompt). Config parse-failure is contained; only command hooks run. updatedInput is logged+warned (input rewrite deferred); the Stop loop-guard is deferred (TODO). Tests: per-file 100% — config-parse unit branches + per-seam mappings end-to-end through the REAL loop + REAL bash + REAL shell scripts (scripted mock model only) + a real-Loader export-shape guard. A keyless ACP snapshot scenario (hook-prompt-block) proves a UserPromptSubmit hook blocks a prompt end-to-end (rejected turn -> ACP cancelled, hook/* events in the log); a with-key e2e (hooks.e2e.ts) proves a PreToolUse hook blocks real bash (verified on disk). The snapshot normalizer now scrubs hook/result.durationMs. RFC: docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
2026-07-01 04:22:00 +08:00
await writeFile(join(workdir, 'hooks.json'), JSON.stringify({
hooks: { PreToolUse: [{ hooks: [{ type: 'command', command: 'echo "bash blocked by policy" >&2; exit 2' }] }] },
}))
spawned = launchAcpTestAgent({
agent: AGENT,
cwd: workdir,
env: { DSH_PERMISSION_MODE: 'danger-full-access' },
})
feat(hooks): dsh-hooks-claude + dsh-hooks-codex bridges (hooks stack PR-F) The two bridge plugins that run a user's existing Claude Code / Codex hook config on the harness's typed interception seams, built on the shared dsh-hook-protocol library. A bridge is a faithfulness adapter, not a power tool: anything it does a native cordis plugin does more powerfully — the bridge exists only to run UNMODIFIED external hooks. - dsh-hooks-claude: CC dialect. Seven hook points (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SubagentStart, SubagentStop), CC per-event stdin payloads, env + ${CLAUDE_PLUGIN_ROOT}/ ${CLAUDE_PROJECT_DIR} substitution, literal-or-regex matcher. - dsh-hooks-codex: Codex dialect — a deliberate subset. Five hook points, always-regex matcher, snake_case payloads (turn_id/model, no trailing newline), no env/substitution, block-only decisions. Both map the neutral merged outcome onto the seam's typed Decision and stamp an explicit {kind:'plugin'} source on injected context (so it is never mislabeled as a user prompt). Config parse-failure is contained; only command hooks run. updatedInput is logged+warned (input rewrite deferred); the Stop loop-guard is deferred (TODO). Tests: per-file 100% — config-parse unit branches + per-seam mappings end-to-end through the REAL loop + REAL bash + REAL shell scripts (scripted mock model only) + a real-Loader export-shape guard. A keyless ACP snapshot scenario (hook-prompt-block) proves a UserPromptSubmit hook blocks a prompt end-to-end (rejected turn -> ACP cancelled, hook/* events in the log); a with-key e2e (hooks.e2e.ts) proves a PreToolUse hook blocks real bash (verified on disk). The snapshot normalizer now scrubs hook/result.durationMs. RFC: docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
2026-07-01 04:22:00 +08:00
const { client, updates } = spawned
await client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} })
const { sessionId } = await client.newSession({ cwd: workdir, mcpServers: [] })
const res = await client.prompt({
sessionId,
prompt: [{ type: 'text', text: 'Use the bash tool to write the exact text HOOK_FAIL into a file named proof.txt in the current directory. Then stop.' }],
})
// The turn completes normally (the block is a tool-result error fed back to
// the model, not a turn failure).
expect(['end_turn', 'max_tokens']).toContain(res.stopReason)
2026-07-25 13:02:37 +08:00
// Assert the denied operation independently of the model response.
feat(hooks): dsh-hooks-claude + dsh-hooks-codex bridges (hooks stack PR-F) The two bridge plugins that run a user's existing Claude Code / Codex hook config on the harness's typed interception seams, built on the shared dsh-hook-protocol library. A bridge is a faithfulness adapter, not a power tool: anything it does a native cordis plugin does more powerfully — the bridge exists only to run UNMODIFIED external hooks. - dsh-hooks-claude: CC dialect. Seven hook points (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SubagentStart, SubagentStop), CC per-event stdin payloads, env + ${CLAUDE_PLUGIN_ROOT}/ ${CLAUDE_PROJECT_DIR} substitution, literal-or-regex matcher. - dsh-hooks-codex: Codex dialect — a deliberate subset. Five hook points, always-regex matcher, snake_case payloads (turn_id/model, no trailing newline), no env/substitution, block-only decisions. Both map the neutral merged outcome onto the seam's typed Decision and stamp an explicit {kind:'plugin'} source on injected context (so it is never mislabeled as a user prompt). Config parse-failure is contained; only command hooks run. updatedInput is logged+warned (input rewrite deferred); the Stop loop-guard is deferred (TODO). Tests: per-file 100% — config-parse unit branches + per-seam mappings end-to-end through the REAL loop + REAL bash + REAL shell scripts (scripted mock model only) + a real-Loader export-shape guard. A keyless ACP snapshot scenario (hook-prompt-block) proves a UserPromptSubmit hook blocks a prompt end-to-end (rejected turn -> ACP cancelled, hook/* events in the log); a with-key e2e (hooks.e2e.ts) proves a PreToolUse hook blocks real bash (verified on disk). The snapshot normalizer now scrubs hook/result.durationMs. RFC: docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
2026-07-01 04:22:00 +08:00
await expect(access(join(workdir, 'proof.txt'))).rejects.toThrow()
// ACP publishes only the committed answer; hook/tool trace stays in the session log.
expect(updates.length).toBeGreaterThan(0)
expect(updates.every(update => update.sessionUpdate === 'agent_message_chunk')).toBe(true)
feat(hooks): dsh-hooks-claude + dsh-hooks-codex bridges (hooks stack PR-F) The two bridge plugins that run a user's existing Claude Code / Codex hook config on the harness's typed interception seams, built on the shared dsh-hook-protocol library. A bridge is a faithfulness adapter, not a power tool: anything it does a native cordis plugin does more powerfully — the bridge exists only to run UNMODIFIED external hooks. - dsh-hooks-claude: CC dialect. Seven hook points (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SubagentStart, SubagentStop), CC per-event stdin payloads, env + ${CLAUDE_PLUGIN_ROOT}/ ${CLAUDE_PROJECT_DIR} substitution, literal-or-regex matcher. - dsh-hooks-codex: Codex dialect — a deliberate subset. Five hook points, always-regex matcher, snake_case payloads (turn_id/model, no trailing newline), no env/substitution, block-only decisions. Both map the neutral merged outcome onto the seam's typed Decision and stamp an explicit {kind:'plugin'} source on injected context (so it is never mislabeled as a user prompt). Config parse-failure is contained; only command hooks run. updatedInput is logged+warned (input rewrite deferred); the Stop loop-guard is deferred (TODO). Tests: per-file 100% — config-parse unit branches + per-seam mappings end-to-end through the REAL loop + REAL bash + REAL shell scripts (scripted mock model only) + a real-Loader export-shape guard. A keyless ACP snapshot scenario (hook-prompt-block) proves a UserPromptSubmit hook blocks a prompt end-to-end (rejected turn -> ACP cancelled, hook/* events in the log); a with-key e2e (hooks.e2e.ts) proves a PreToolUse hook blocks real bash (verified on disk). The snapshot normalizer now scrubs hook/result.durationMs. RFC: docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
2026-07-01 04:22:00 +08:00
}, 180_000)
})