deepseek-harness/vitest.e2e.config.ts

40 lines
1.7 KiB
TypeScript
Raw Normal View History

Add two DeepSeek LLM adapters: dsh-llm-deepseek and dsh-llm-pi-ai The first real LlmAdapter implementations, shipped as a deliberate pair: same models and wire protocol, completely different internals, so the StreamChunk protocol is verified across independent implementations. - dsh-llm-deepseek: hand-rolled fetch + SSE parser + chunk-translation state machine against the official chat-completions format (thinking mode via top-level thinking/reasoning_effort; the empty-string reasoning_content first chunk; usage attached to the finish chunk or trailing; reasoning_content passback on tool-call turns; disjoint cache-token accounting). - dsh-llm-pi-ai: the same endpoint through @earendil-works/pi-ai, mapping its event vocabulary (parsed tool arguments, in-stream error events, folded reasoning tokens) onto the same chunks. The agent loop now honors the in-band error path: an adapter that ends its stream with finish {kind:error|aborted} (the only option for adapters that can't throw mid-stream, like pi-ai) is translated into a step error, so the turn ends error/aborted with a logged error event instead of a normal completed assistant message. This makes the StreamChunk error contract real for both adapters; docs/architecture.md and the StreamChunk doc are updated accordingly. New yarn test:e2e (vitest.e2e.config.ts, *.e2e.ts) runs key-gated real-API matrices for both adapters across V4 Flash/Pro and all thinking/effort levels; it self-skips without DEEPSEEK_API_KEY. Unit suites run against local node:http mock SSE servers at 100% per-file coverage.
2026-06-13 00:28:29 +08:00
import tsconfigPaths from 'vite-tsconfig-paths'
import { defineConfig } from 'vitest/config'
2026-06-16 14:55:37 +08:00
// Real-API end-to-end tests: `pnpm run test:e2e`, file pattern *.e2e.ts.
// Separate from the default suite (`pnpm run test`, *.spec.ts) on purpose —
Add two DeepSeek LLM adapters: dsh-llm-deepseek and dsh-llm-pi-ai The first real LlmAdapter implementations, shipped as a deliberate pair: same models and wire protocol, completely different internals, so the StreamChunk protocol is verified across independent implementations. - dsh-llm-deepseek: hand-rolled fetch + SSE parser + chunk-translation state machine against the official chat-completions format (thinking mode via top-level thinking/reasoning_effort; the empty-string reasoning_content first chunk; usage attached to the finish chunk or trailing; reasoning_content passback on tool-call turns; disjoint cache-token accounting). - dsh-llm-pi-ai: the same endpoint through @earendil-works/pi-ai, mapping its event vocabulary (parsed tool arguments, in-stream error events, folded reasoning tokens) onto the same chunks. The agent loop now honors the in-band error path: an adapter that ends its stream with finish {kind:error|aborted} (the only option for adapters that can't throw mid-stream, like pi-ai) is translated into a step error, so the turn ends error/aborted with a logged error event instead of a normal completed assistant message. This makes the StreamChunk error contract real for both adapters; docs/architecture.md and the StreamChunk doc are updated accordingly. New yarn test:e2e (vitest.e2e.config.ts, *.e2e.ts) runs key-gated real-API matrices for both adapters across V4 Flash/Pro and all thinking/effort levels; it self-skips without DEEPSEEK_API_KEY. Unit suites run against local node:http mock SSE servers at 100% per-file coverage.
2026-06-13 00:28:29 +08:00
// these hit the live DeepSeek API, spend tokens, and need a key.
//
// Secrets: tests gate themselves with
// `describe.skipIf(!process.env.DEEPSEEK_API_KEY)`, so the suite passes
// (all-skipped) without credentials — CI has none and stays green. Put the
// key in the environment or in a gitignored `.env` at the repo root:
//
// DEEPSEEK_API_KEY=sk-…
// DEEPSEEK_BASE_URL=https://… # optional, defaults to the public API
try {
// Node >= 21.7 native; throws when the file does not exist.
process.loadEnvFile(new URL('.env', import.meta.url).pathname)
} catch {
// No .env — fine, the environment may already carry the variables.
}
export default defineConfig({
// Same resolution note as vitest.config.ts: bare workspace names resolve
// through the root tsconfig paths map; the native option cannot do this.
plugins: [tsconfigPaths({ projects: ['./tsconfig.test.json'] })],
test: {
include: ['packages/*/tests/**/*.e2e.ts', 'examples/*/tests/**/*.e2e.ts'],
// Real model calls: generous timeouts, and retries for transient flakes
// (the shared internal key hits concurrency quotas). No coverage — the
// unit suites own the coverage gate.
Add two DeepSeek LLM adapters: dsh-llm-deepseek and dsh-llm-pi-ai The first real LlmAdapter implementations, shipped as a deliberate pair: same models and wire protocol, completely different internals, so the StreamChunk protocol is verified across independent implementations. - dsh-llm-deepseek: hand-rolled fetch + SSE parser + chunk-translation state machine against the official chat-completions format (thinking mode via top-level thinking/reasoning_effort; the empty-string reasoning_content first chunk; usage attached to the finish chunk or trailing; reasoning_content passback on tool-call turns; disjoint cache-token accounting). - dsh-llm-pi-ai: the same endpoint through @earendil-works/pi-ai, mapping its event vocabulary (parsed tool arguments, in-stream error events, folded reasoning tokens) onto the same chunks. The agent loop now honors the in-band error path: an adapter that ends its stream with finish {kind:error|aborted} (the only option for adapters that can't throw mid-stream, like pi-ai) is translated into a step error, so the turn ends error/aborted with a logged error event instead of a normal completed assistant message. This makes the StreamChunk error contract real for both adapters; docs/architecture.md and the StreamChunk doc are updated accordingly. New yarn test:e2e (vitest.e2e.config.ts, *.e2e.ts) runs key-gated real-API matrices for both adapters across V4 Flash/Pro and all thinking/effort levels; it self-skips without DEEPSEEK_API_KEY. Unit suites run against local node:http mock SSE servers at 100% per-file coverage.
2026-06-13 00:28:29 +08:00
testTimeout: 120_000,
hookTimeout: 30_000,
retry: 2,
// Run e2e files one at a time: the shared internal API key has a small
// concurrency quota, and parallel files issue enough simultaneous requests
// to trip it (manifesting as flaky rate-limit errors).
fileParallelism: false,
Add two DeepSeek LLM adapters: dsh-llm-deepseek and dsh-llm-pi-ai The first real LlmAdapter implementations, shipped as a deliberate pair: same models and wire protocol, completely different internals, so the StreamChunk protocol is verified across independent implementations. - dsh-llm-deepseek: hand-rolled fetch + SSE parser + chunk-translation state machine against the official chat-completions format (thinking mode via top-level thinking/reasoning_effort; the empty-string reasoning_content first chunk; usage attached to the finish chunk or trailing; reasoning_content passback on tool-call turns; disjoint cache-token accounting). - dsh-llm-pi-ai: the same endpoint through @earendil-works/pi-ai, mapping its event vocabulary (parsed tool arguments, in-stream error events, folded reasoning tokens) onto the same chunks. The agent loop now honors the in-band error path: an adapter that ends its stream with finish {kind:error|aborted} (the only option for adapters that can't throw mid-stream, like pi-ai) is translated into a step error, so the turn ends error/aborted with a logged error event instead of a normal completed assistant message. This makes the StreamChunk error contract real for both adapters; docs/architecture.md and the StreamChunk doc are updated accordingly. New yarn test:e2e (vitest.e2e.config.ts, *.e2e.ts) runs key-gated real-API matrices for both adapters across V4 Flash/Pro and all thinking/effort levels; it self-skips without DEEPSEEK_API_KEY. Unit suites run against local node:http mock SSE servers at 100% per-file coverage.
2026-06-13 00:28:29 +08:00
},
})