fix(ci): serial-windows notices timeout and generator store-scan cost

render() loaded the workspace manifests once per external dependency
name through workspaceLinkedManifest, an O(names x manifests) file
read on the cold path; on the loaded self-hosted Windows host with
coverage instrumentation the freshness spec crossed Vitest's default
5000ms budget and failed the serial-windows standby gate four times in
a week. Load the manifests once in render() and thread the map through
the collectors instead.

The serial-windows lane also ran the coverage inventory at the strictest
budget of any lane: add DSH_COVERAGE_TEST_TIMEOUT_MS=90000 to match the
pull-request windows-coverage lane, pinned by ci-workflow.spec.ts.
This commit is contained in:
Chinesezjc 2026-08-31 14:59:42 +08:00
parent cf2d0986cf
commit a074e6131f
9 changed files with 98 additions and 18 deletions

View file

@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-31-serial-windows-notices-timeout-budget.md
2026-08-31-serial-windows-notices-timeout-budget.md: 184004122bf8050fa00038b7c57ae680748463f8
2026-08-31-serial-windows-notices-timeout-budget.zh.md: 4fc00ed25cc24ef76e43a0099a362b8be0e59e3e

View file

@ -0,0 +1,30 @@
# Agent Note: serial-windows notices timeout budget and generator store-scan cost
Status: implemented
English | [中文](2026-08-31-serial-windows-notices-timeout-budget.zh.md)
## Problem
The `serial / windows (self-hosted standby)` master lane failed its `test:coverage` gate four times in a week (runs 33333033178, 33311481884, 33352293522, 33353113100), always on the same case: `scripts/gen-third-party-notices.spec.ts > THIRD_PARTY_NOTICES.md > matches what the generator produces from the current manifests`, with `Error: Test timed out in 5000ms`. Measured test wall times on the shared Windows host were 4149–8853 ms against Vitest's default 5000 ms per-test budget. All other 26 cases in the file finished in 0–3 ms, and the passing run two hours later (33360033028) had the same code green.
The lane runs the complete unsharded Windows gate inventory serially with `DSH_COVERAGE_MAX_WORKERS=1`, so `render()` regenerates `THIRD_PARTY_NOTICES.md` from the workspace manifests and the installed pnpm store on a host shared by 32 runners. The cold path is dominated by `workspaceLinkedManifest`, which re-ran `loadWorkspaceManifests()` — a glob plus reads and JSON-parses of every workspace `package.json` — once per cache-missing external dependency name: 130 names × 270 manifests ≈ 35k file operations, on top of a `.pnpm` store scan per name. Under v8 coverage instrumentation and shared-host I/O contention that crossed the 5 s default.
The lane also had no `DSH_COVERAGE_TEST_TIMEOUT_MS`, unlike the pull-request `windows-coverage` lane ([ci.yml](../../../../.github/workflows/ci.yml)) which grants 90000 ms, so the serial reference ran the same coverage inventory at the strictest budget of any lane.
## Decision
Two changes:
1. [scripts/gen-third-party-notices.ts](../../../../scripts/gen-third-party-notices.ts) loads the workspace manifests once in `render()` and threads the map through `collectNpmDeps` → `installedMetadata` → `installedManifest` → `workspaceLinkedManifest` instead of reloading it per external dependency name. The cold `render()` wall time on the same checkout fell from ~893 ms to ~86 ms with byte-identical output (verified by diffing the rendered documents before and after the change).
2. [ci-master.yml](../../../../.github/workflows/ci-master.yml) `serial-windows` step "Run complete unsharded Windows gate inventory serially" gains `DSH_COVERAGE_TEST_TIMEOUT_MS: '90000'`, matching the pull-request `windows-coverage` lane budget. This extends the per-test, expect.poll, and hook budget mechanism defined by [the Windows lane hook and Lefthook budget note](../testing/2026-08-29-windows-lane-hook-and-lefthook-budget.md) to a second lane; that note records which lanes set the env. `scripts/ci-workflow.spec.ts` pins this env with a `toMatchObject` assertion; removing the env turns the spec red (negative control exercised).
## Alternatives considered
- **Raise the lane budget only** - rejected as the sole fix: it would mask the O(names × manifests) reload for every lane that runs the generator, including the pre-commit hook and the standalone `--check` path.
- **Module-level cache for `loadWorkspaceManifests()`** - rejected in favor of explicit threading, which keeps the single-load contract visible at the call site and avoids a second hidden cache next to `workspaceLinkedManifestCache`.
## Consequences
The generator resolves installed metadata from one manifest load per `render()` call, and clears the name-keyed linked-manifest cache at the start of each call so the cache cannot outlive the map it was resolved from. The serial-windows lane runs the coverage inventory at the same 90000 ms per-test budget as the pull-request coverage lane. `THIRD_PARTY_NOTICES.md` bytes are unchanged; the freshness spec still compares `render()` against the committed document.

View file

@ -0,0 +1,30 @@
# Agent Note:serial-windows 的 notices 超时预算与 generator store 扫描成本
Status: implemented
[English](2026-08-31-serial-windows-notices-timeout-budget.md) | 中文
## Problem
`serial / windows (self-hosted standby)` master lane 一周内四次失败在 `test:coverage` gate(run 33333033178、33311481884、33352293522、33353113100),失败用例每次都相同:`scripts/gen-third-party-notices.spec.ts > THIRD_PARTY_NOTICES.md > matches what the generator produces from the current manifests`,报 `Error: Test timed out in 5000ms`。共享 Windows 主机上该用例实测 4149–8853 ms,超出 Vitest 默认的 5000 ms 单测预算。文件其余 26 个用例全部 0–3 ms 通过,两小时后的 passing run(33360033028)用同一份代码全绿。
该 lane 以 `DSH_COVERAGE_MAX_WORKERS=1` 串行跑完整的无分片 Windows gate 清单,`render()` 要从 workspace manifest 和已安装的 pnpm store 全量重生成 `THIRD_PARTY_NOTICES.md`,而主机被 32 个 runner 共享。冷路径的代价集中在 `workspaceLinkedManifest`:它对每个未缓存的外部依赖名重跑一遍 `loadWorkspaceManifests()`——glob 并读取、解析全部 workspace `package.json`——即 130 名 × 270 manifest ≈ 3.5 万次文件操作,另加每个名字一次 `.pnpm` store 扫描。叠加 v8 覆盖率插桩与共享主机 I/O 争抢后越过 5 秒默认值。
该 lane 还没有 `DSH_COVERAGE_TEST_TIMEOUT_MS`,而 pull-request 的 `windows-coverage` lane([ci.yml](../../../../.github/workflows/ci.yml))给的是 90000 ms——于是这条 serial 参考 lane 用全仓库最紧的预算跑同一份 coverage 清单。
## Decision
两处改动:
1. [scripts/gen-third-party-notices.ts](../../../../scripts/gen-third-party-notices.ts) 在 `render()` 里只加载一次 workspace manifest,把 map 沿 `collectNpmDeps` → `installedMetadata` → `installedManifest` → `workspaceLinkedManifest` 显式传递,不再按外部依赖名逐个重载。同一 checkout 下冷 `render()` 墙钟从约 893 ms 降到约 86 ms,输出逐字节一致(改动前后渲染结果 diff 验证)。
2. [ci-master.yml](../../../../.github/workflows/ci-master.yml) `serial-windows` 的 "Run complete unsharded Windows gate inventory serially" 步骤增加 `DSH_COVERAGE_TEST_TIMEOUT_MS: '90000'`,与 pull-request `windows-coverage` lane 对齐。这是把 [Windows 覆盖率 lane 的 hook 预算与 Lefthook 套件预算 note](../testing/2026-08-29-windows-lane-hook-and-lefthook-budget.zh.md) 定义的 per-test、expect.poll 与 hook 预算机制扩展到第二个 lane;该 note 记录了哪些 lane 设置此 env。`scripts/ci-workflow.spec.ts` 用 `toMatchObject` 断言钉住该 env;删掉 env 会让 spec 变红(已做负例验证)。
## Alternatives considered
- **只放宽 lane 预算** - 否决作为唯一修复:会掩盖所有运行 generator 的 lane 上的 O(名×manifest) 重载成本,包括 pre-commit hook 与独立 `--check` 路径。
- **给 `loadWorkspaceManifests()` 加模块级缓存** - 否决,改用显式传递:把「单次加载」契约留在调用点可见,避免在 `workspaceLinkedManifestCache` 之外再加一层隐藏缓存。
## Consequences
generator 每次 `render()` 调用只加载一次 manifest 来解析已安装元数据,并在调用开头清空按名字作键的 linked-manifest 缓存,使缓存不会活过它解析自的那份 map。serial-windows lane 与 pull-request coverage lane 一样按 90000 ms 单测预算跑 coverage 清单。`THIRD_PARTY_NOTICES.md` 字节不变;新鲜度 spec 仍把 `render()` 与已提交文档逐字节比较。

View file

@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority; # side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with: # after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-08-29-windows-lane-hook-and-lefthook-budget.md # pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-08-29-windows-lane-hook-and-lefthook-budget.md
2026-08-29-windows-lane-hook-and-lefthook-budget.md: 6886e3ad4958d20a88a66df6a9e02f5a60a36a6a 2026-08-29-windows-lane-hook-and-lefthook-budget.md: 40ebba25e459abd6d9ae755f831e703aa826ae39
2026-08-29-windows-lane-hook-and-lefthook-budget.zh.md: 56c1e625d92f01e24f2268deb5d03040278da6af 2026-08-29-windows-lane-hook-and-lefthook-budget.zh.md: c14be5da098920dbb7360c01bbb94f0bc22f700b

View file

@ -24,7 +24,7 @@ A `git` or `node` spawn spike on the shared-volume runners no longer decides eit
Both budgets widen what counts as an acceptable duration, so a real slowdown into tens of seconds now passes where the previous ceilings would have caught it. That detection is traded away deliberately: those ceilings were firing on host contention rather than on regressions. Both budgets widen what counts as an acceptable duration, so a real slowdown into tens of seconds now passes where the previous ceilings would have caught it. That detection is traded away deliberately: those ceilings were firing on host contention rather than on regressions.
The hook change applies wherever `DSH_COVERAGE_TEST_TIMEOUT_MS` is set, which today is the Windows coverage lane alone. Lanes that leave it unset keep every Vitest default, including the 10 s hook budget. The hook change applies wherever `DSH_COVERAGE_TEST_TIMEOUT_MS` is set: the Windows coverage lane in [ci.yml](../../../../.github/workflows/ci.yml) and the `serial-windows` master standby in [ci-master.yml](../../../../.github/workflows/ci-master.yml) ([the serial-windows notices timeout note](../process/2026-08-31-serial-windows-notices-timeout-budget.md) records the second lane's adoption). Lanes that leave it unset keep every Vitest default, including the 10 s hook budget.
## Alternatives considered ## Alternatives considered

View file

@ -24,7 +24,7 @@ Lefthook 套件取 `{ timeout: 90_000 }`,与 [`.github/workflows/ci.yml`](../.
两份预算都放宽了「多长算可接受」,因此一个退化到几十秒的真实变慢现在会通过,而此前的上限会拦住它。这项检测能力是有意换掉的:那些上限触发的是宿主机争抢,不是回归。 两份预算都放宽了「多长算可接受」,因此一个退化到几十秒的真实变慢现在会通过,而此前的上限会拦住它。这项检测能力是有意换掉的:那些上限触发的是宿主机争抢,不是回归。
hook 的改动在所有设置了 `DSH_COVERAGE_TEST_TIMEOUT_MS` 的地方生效,目前仅 Windows 覆盖率 lane 一处。不设置它的 lane 保持全部 Vitest 默认值,包括 10 秒的 hook 预算。 hook 的改动在所有设置了 `DSH_COVERAGE_TEST_TIMEOUT_MS` 的地方生效:[ci.yml](../../../../.github/workflows/ci.yml) 的 Windows 覆盖率 lane,以及 [ci-master.yml](../../../../.github/workflows/ci-master.yml) 的 `serial-windows` master standby([serial-windows notices 超时 note](../process/2026-08-31-serial-windows-notices-timeout-budget.zh.md) 记录了第二个 lane 的采用)。不设置它的 lane 保持全部 Vitest 默认值,包括 10 秒的 hook 预算。
## 备选方案 ## 备选方案

View file

@ -208,6 +208,7 @@ jobs:
shell: pwsh shell: pwsh
env: env:
DSH_COVERAGE_MAX_WORKERS: '1' DSH_COVERAGE_MAX_WORKERS: '1'
DSH_COVERAGE_TEST_TIMEOUT_MS: '90000'
DSH_GATE_CONCURRENCY: '1' DSH_GATE_CONCURRENCY: '1'
DSH_PUBLINT_CONCURRENCY: '1' DSH_PUBLINT_CONCURRENCY: '1'
run: pnpm run check:ci:windows-complete run: pnpm run check:ci:windows-complete

View file

@ -201,6 +201,14 @@ describe('CI workflow', () => {
expect(serialInstall!.run.split('\n').map(line => line.trim())).toContain('} else {') expect(serialInstall!.run.split('\n').map(line => line.trim())).toContain('} else {')
expect(serialInstall!.run.split('\n').map(line => line.trim())).toContain('pnpm install --frozen-lockfile') expect(serialInstall!.run.split('\n').map(line => line.trim())).toContain('pnpm install --frozen-lockfile')
expect(serialInstall!.run).not.toContain('$cloneFlag') expect(serialInstall!.run).not.toContain('$cloneFlag')
// The unsharded reference runs the whole coverage inventory at the same
// per-test budget the PR coverage lane grants; the default 5000ms times
// out load-sensitive store scans (e.g. gen-third-party-notices).
const serialGate = serialSteps.find((step): step is Record<string, unknown> & { env?: Record<string, unknown> } => (
isRecord(step) && step.name === 'Run complete unsharded Windows gate inventory serially'
))
expect(serialGate).toBeDefined()
expect(serialGate!.env).toMatchObject({ DSH_COVERAGE_TEST_TIMEOUT_MS: '90000' })
// Aggregate: Wine and the required split native jobs are needed; // Aggregate: Wine and the required split native jobs are needed;
// windows-coverage is temporarily non-blocking while Windows ACP // windows-coverage is temporarily non-blocking while Windows ACP

View file

@ -279,11 +279,13 @@ const workspaceLinkedManifestCache = new Map<string, VirtualManifest | undefined
* Resolve the package version selected for a declaring workspace instead of an * Resolve the package version selected for a declaring workspace instead of an
* unrelated historical version that still occupies the shared virtual store. * unrelated historical version that still occupies the shared virtual store.
* @param name - external package identity. * @param name - external package identity.
* @param manifests - workspace manifests already loaded by the caller, so one
* load serves every dependency instead of a full re-read per name.
* @returns the first current workspace link for that package, when installed. * @returns the first current workspace link for that package, when installed.
*/ */
function workspaceLinkedManifest(name: string): VirtualManifest | undefined { function workspaceLinkedManifest(name: string, manifests: Map<string, Manifest>): VirtualManifest | undefined {
if (workspaceLinkedManifestCache.has(name)) return workspaceLinkedManifestCache.get(name) if (workspaceLinkedManifestCache.has(name)) return workspaceLinkedManifestCache.get(name)
for (const [path, manifest] of loadWorkspaceManifests().manifests) { for (const [path, manifest] of manifests) {
if (!ALL_KINDS.some(kind => name in (manifest[kind] ?? {}))) continue if (!ALL_KINDS.some(kind => name in (manifest[kind] ?? {}))) continue
const linked = resolve(root, dirname(path), 'node_modules', name, 'package.json') const linked = resolve(root, dirname(path), 'node_modules', name, 'package.json')
if (!existsSync(linked)) continue if (!existsSync(linked)) continue
@ -296,8 +298,8 @@ function workspaceLinkedManifest(name: string): VirtualManifest | undefined {
} }
/** Resolve one installed external package manifest from either pnpm store. */ /** Resolve one installed external package manifest from either pnpm store. */
function installedManifest(name: string, expectedVersion?: string): VirtualManifest | undefined { function installedManifest(name: string, manifests: Map<string, Manifest>, expectedVersion?: string): VirtualManifest | undefined {
const linked = workspaceLinkedManifest(name) const linked = workspaceLinkedManifest(name, manifests)
if (linked !== undefined && (expectedVersion === undefined || linked.version === expectedVersion)) return linked if (linked !== undefined && (expectedVersion === undefined || linked.version === expectedVersion)) return linked
let manifest: (Manifest & { license?: string; repository?: string | { url?: string }; homepage?: string }) | undefined let manifest: (Manifest & { license?: string; repository?: string | { url?: string }; homepage?: string }) | undefined
// Workspace-local link farms can expose a dependency that is not linked at // Workspace-local link farms can expose a dependency that is not linked at
@ -320,9 +322,9 @@ function installedManifest(name: string, expectedVersion?: string): VirtualManif
} }
/** License and repository URL for an installed external package, from the pnpm store. */ /** License and repository URL for an installed external package, from the pnpm store. */
function installedMetadata(name: string): { license: string; repo: string } { function installedMetadata(name: string, manifests: Map<string, Manifest>): { license: string; repo: string } {
const override = OVERRIDES[name] const override = OVERRIDES[name]
const manifest = installedManifest(name) const manifest = installedManifest(name, manifests)
const license = override?.license ?? manifest?.license const license = override?.license ?? manifest?.license
const rawRepo = typeof manifest?.repository === 'string' ? manifest.repository : manifest?.repository?.url ?? manifest?.homepage const rawRepo = typeof manifest?.repository === 'string' ? manifest.repository : manifest?.repository?.url ?? manifest?.homepage
const repo = override?.repo ?? normalizeRepo(rawRepo) const repo = override?.repo ?? normalizeRepo(rawRepo)
@ -332,8 +334,8 @@ function installedMetadata(name: string): { license: string; repo: string } {
return { license, repo } return { license, repo }
} }
function collectClaudeDistribution(): ClaudeDistribution { function collectClaudeDistribution(manifests: Map<string, Manifest>): ClaudeDistribution {
const manifest = installedManifest(CLAUDE_AGENT_SDK_PACKAGE) const manifest = installedManifest(CLAUDE_AGENT_SDK_PACKAGE, manifests)
if (manifest === undefined) { if (manifest === undefined) {
throw new Error( throw new Error(
`gen-third-party-notices: cannot resolve ${CLAUDE_AGENT_SDK_PACKAGE}; run \`pnpm install\`.`, `gen-third-party-notices: cannot resolve ${CLAUDE_AGENT_SDK_PACKAGE}; run \`pnpm install\`.`,
@ -342,7 +344,7 @@ function collectClaudeDistribution(): ClaudeDistribution {
const distribution = claudeDistributionFromManifest(manifest) const distribution = claudeDistributionFromManifest(manifest)
let installedPayloads = 0 let installedPayloads = 0
for (const payload of distribution.payloads) { for (const payload of distribution.payloads) {
const installed = installedManifest(payload.name, payload.version) const installed = installedManifest(payload.name, manifests, payload.version)
if (installed === undefined) continue if (installed === undefined) continue
installedPayloads += 1 installedPayloads += 1
if ( if (
@ -383,12 +385,11 @@ function normalizeRepo(raw: string | undefined): string | undefined {
* by tooling, test infrastructure, the website, or the demo leaves — whatever * by tooling, test infrastructure, the website, or the demo leaves — whatever
* the declaring section is called — is development-only. * the declaring section is called — is development-only.
*/ */
function collectNpmDeps(): ExternalDep[] { function collectNpmDeps(manifests: Map<string, Manifest>, names: Set<string>): ExternalDep[] {
const { manifests, names } = loadWorkspaceManifests()
return [...tierExternalDeps(manifests, names)] return [...tierExternalDeps(manifests, names)]
.filter(([name]) => !FIRST_PARTY.has(name)) .filter(([name]) => !FIRST_PARTY.has(name))
.sort(([a], [b]) => a.localeCompare(b)) .sort(([a], [b]) => a.localeCompare(b))
.map(([name, runtime]) => ({ name, ...installedMetadata(name), runtime })) .map(([name, runtime]) => ({ name, ...installedMetadata(name, manifests), runtime }))
} }
/** /**
@ -692,7 +693,11 @@ ${rows.join('\n')}
*/ */
export function render(): string { export function render(): string {
verifyBuildTimePins() verifyBuildTimePins()
const npm = collectNpmDeps() // The linked-manifest cache is keyed by name only, so it must not outlive
// the manifests map it was resolved from; render() owns that single load.
workspaceLinkedManifestCache.clear()
const { manifests, names } = loadWorkspaceManifests()
const npm = collectNpmDeps(manifests, names)
const runtimeDeps = npm.filter(dep => dep.runtime) const runtimeDeps = npm.filter(dep => dep.runtime)
const devDeps = npm.filter(dep => !dep.runtime) const devDeps = npm.filter(dep => !dep.runtime)
const vendored = collectVendored() const vendored = collectVendored()
@ -701,7 +706,7 @@ export function render(): string {
const claudeDistribution = runtimeDeps.some( const claudeDistribution = runtimeDeps.some(
dep => dep.name === CLAUDE_AGENT_SDK_PACKAGE, dep => dep.name === CLAUDE_AGENT_SDK_PACKAGE,
) )
? collectClaudeDistribution() ? collectClaudeDistribution(manifests)
: undefined : undefined
const nonPermissiveDev = devDeps.filter(dep => !isPermissive(dep.license)) const nonPermissiveDev = devDeps.filter(dep => !isPermissive(dep.license))
// A copyleft license reaching a shipped surface is a distribution decision, // A copyleft license reaching a shipped surface is a distribution decision,