fix(ci): serial-windows notices timeout and generator store-scan cost
render() loaded the workspace manifests once per external dependency name through workspaceLinkedManifest, an O(names x manifests) file read on the cold path; on the loaded self-hosted Windows host with coverage instrumentation the freshness spec crossed Vitest's default 5000ms budget and failed the serial-windows standby gate four times in a week. Load the manifests once in render() and thread the map through the collectors instead. The serial-windows lane also ran the coverage inventory at the strictest budget of any lane: add DSH_COVERAGE_TEST_TIMEOUT_MS=90000 to match the pull-request windows-coverage lane, pinned by ci-workflow.spec.ts.
This commit is contained in:
parent
cf2d0986cf
commit
a074e6131f
9 changed files with 98 additions and 18 deletions
|
|
@ -0,0 +1,6 @@
|
|||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-31-serial-windows-notices-timeout-budget.md
|
||||
2026-08-31-serial-windows-notices-timeout-budget.md: 184004122bf8050fa00038b7c57ae680748463f8
|
||||
2026-08-31-serial-windows-notices-timeout-budget.zh.md: 4fc00ed25cc24ef76e43a0099a362b8be0e59e3e
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
# Agent Note: serial-windows notices timeout budget and generator store-scan cost
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-31-serial-windows-notices-timeout-budget.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The `serial / windows (self-hosted standby)` master lane failed its `test:coverage` gate four times in a week (runs 33333033178, 33311481884, 33352293522, 33353113100), always on the same case: `scripts/gen-third-party-notices.spec.ts > THIRD_PARTY_NOTICES.md > matches what the generator produces from the current manifests`, with `Error: Test timed out in 5000ms`. Measured test wall times on the shared Windows host were 4149–8853 ms against Vitest's default 5000 ms per-test budget. All other 26 cases in the file finished in 0–3 ms, and the passing run two hours later (33360033028) had the same code green.
|
||||
|
||||
The lane runs the complete unsharded Windows gate inventory serially with `DSH_COVERAGE_MAX_WORKERS=1`, so `render()` regenerates `THIRD_PARTY_NOTICES.md` from the workspace manifests and the installed pnpm store on a host shared by 32 runners. The cold path is dominated by `workspaceLinkedManifest`, which re-ran `loadWorkspaceManifests()` — a glob plus reads and JSON-parses of every workspace `package.json` — once per cache-missing external dependency name: 130 names × 270 manifests ≈ 35k file operations, on top of a `.pnpm` store scan per name. Under v8 coverage instrumentation and shared-host I/O contention that crossed the 5 s default.
|
||||
|
||||
The lane also had no `DSH_COVERAGE_TEST_TIMEOUT_MS`, unlike the pull-request `windows-coverage` lane ([ci.yml](../../../../.github/workflows/ci.yml)) which grants 90000 ms, so the serial reference ran the same coverage inventory at the strictest budget of any lane.
|
||||
|
||||
## Decision
|
||||
|
||||
Two changes:
|
||||
|
||||
1. [scripts/gen-third-party-notices.ts](../../../../scripts/gen-third-party-notices.ts) loads the workspace manifests once in `render()` and threads the map through `collectNpmDeps` → `installedMetadata` → `installedManifest` → `workspaceLinkedManifest` instead of reloading it per external dependency name. The cold `render()` wall time on the same checkout fell from ~893 ms to ~86 ms with byte-identical output (verified by diffing the rendered documents before and after the change).
|
||||
|
||||
2. [ci-master.yml](../../../../.github/workflows/ci-master.yml) `serial-windows` step "Run complete unsharded Windows gate inventory serially" gains `DSH_COVERAGE_TEST_TIMEOUT_MS: '90000'`, matching the pull-request `windows-coverage` lane budget. This extends the per-test, expect.poll, and hook budget mechanism defined by [the Windows lane hook and Lefthook budget note](../testing/2026-08-29-windows-lane-hook-and-lefthook-budget.md) to a second lane; that note records which lanes set the env. `scripts/ci-workflow.spec.ts` pins this env with a `toMatchObject` assertion; removing the env turns the spec red (negative control exercised).
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Raise the lane budget only** - rejected as the sole fix: it would mask the O(names × manifests) reload for every lane that runs the generator, including the pre-commit hook and the standalone `--check` path.
|
||||
- **Module-level cache for `loadWorkspaceManifests()`** - rejected in favor of explicit threading, which keeps the single-load contract visible at the call site and avoids a second hidden cache next to `workspaceLinkedManifestCache`.
|
||||
|
||||
## Consequences
|
||||
|
||||
The generator resolves installed metadata from one manifest load per `render()` call, and clears the name-keyed linked-manifest cache at the start of each call so the cache cannot outlive the map it was resolved from. The serial-windows lane runs the coverage inventory at the same 90000 ms per-test budget as the pull-request coverage lane. `THIRD_PARTY_NOTICES.md` bytes are unchanged; the freshness spec still compares `render()` against the committed document.
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
# Agent Note:serial-windows 的 notices 超时预算与 generator store 扫描成本
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-31-serial-windows-notices-timeout-budget.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
`serial / windows (self-hosted standby)` master lane 一周内四次失败在 `test:coverage` gate(run 33333033178、33311481884、33352293522、33353113100),失败用例每次都相同:`scripts/gen-third-party-notices.spec.ts > THIRD_PARTY_NOTICES.md > matches what the generator produces from the current manifests`,报 `Error: Test timed out in 5000ms`。共享 Windows 主机上该用例实测 4149–8853 ms,超出 Vitest 默认的 5000 ms 单测预算。文件其余 26 个用例全部 0–3 ms 通过,两小时后的 passing run(33360033028)用同一份代码全绿。
|
||||
|
||||
该 lane 以 `DSH_COVERAGE_MAX_WORKERS=1` 串行跑完整的无分片 Windows gate 清单,`render()` 要从 workspace manifest 和已安装的 pnpm store 全量重生成 `THIRD_PARTY_NOTICES.md`,而主机被 32 个 runner 共享。冷路径的代价集中在 `workspaceLinkedManifest`:它对每个未缓存的外部依赖名重跑一遍 `loadWorkspaceManifests()`——glob 并读取、解析全部 workspace `package.json`——即 130 名 × 270 manifest ≈ 3.5 万次文件操作,另加每个名字一次 `.pnpm` store 扫描。叠加 v8 覆盖率插桩与共享主机 I/O 争抢后越过 5 秒默认值。
|
||||
|
||||
该 lane 还没有 `DSH_COVERAGE_TEST_TIMEOUT_MS`,而 pull-request 的 `windows-coverage` lane([ci.yml](../../../../.github/workflows/ci.yml))给的是 90000 ms——于是这条 serial 参考 lane 用全仓库最紧的预算跑同一份 coverage 清单。
|
||||
|
||||
## Decision
|
||||
|
||||
两处改动:
|
||||
|
||||
1. [scripts/gen-third-party-notices.ts](../../../../scripts/gen-third-party-notices.ts) 在 `render()` 里只加载一次 workspace manifest,把 map 沿 `collectNpmDeps` → `installedMetadata` → `installedManifest` → `workspaceLinkedManifest` 显式传递,不再按外部依赖名逐个重载。同一 checkout 下冷 `render()` 墙钟从约 893 ms 降到约 86 ms,输出逐字节一致(改动前后渲染结果 diff 验证)。
|
||||
|
||||
2. [ci-master.yml](../../../../.github/workflows/ci-master.yml) `serial-windows` 的 "Run complete unsharded Windows gate inventory serially" 步骤增加 `DSH_COVERAGE_TEST_TIMEOUT_MS: '90000'`,与 pull-request `windows-coverage` lane 对齐。这是把 [Windows 覆盖率 lane 的 hook 预算与 Lefthook 套件预算 note](../testing/2026-08-29-windows-lane-hook-and-lefthook-budget.zh.md) 定义的 per-test、expect.poll 与 hook 预算机制扩展到第二个 lane;该 note 记录了哪些 lane 设置此 env。`scripts/ci-workflow.spec.ts` 用 `toMatchObject` 断言钉住该 env;删掉 env 会让 spec 变红(已做负例验证)。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **只放宽 lane 预算** - 否决作为唯一修复:会掩盖所有运行 generator 的 lane 上的 O(名×manifest) 重载成本,包括 pre-commit hook 与独立 `--check` 路径。
|
||||
- **给 `loadWorkspaceManifests()` 加模块级缓存** - 否决,改用显式传递:把「单次加载」契约留在调用点可见,避免在 `workspaceLinkedManifestCache` 之外再加一层隐藏缓存。
|
||||
|
||||
## Consequences
|
||||
|
||||
generator 每次 `render()` 调用只加载一次 manifest 来解析已安装元数据,并在调用开头清空按名字作键的 linked-manifest 缓存,使缓存不会活过它解析自的那份 map。serial-windows lane 与 pull-request coverage lane 一样按 90000 ms 单测预算跑 coverage 清单。`THIRD_PARTY_NOTICES.md` 字节不变;新鲜度 spec 仍把 `render()` 与已提交文档逐字节比较。
|
||||
|
|
@ -2,5 +2,5 @@
|
|||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-08-29-windows-lane-hook-and-lefthook-budget.md
|
||||
2026-08-29-windows-lane-hook-and-lefthook-budget.md: 6886e3ad4958d20a88a66df6a9e02f5a60a36a6a
|
||||
2026-08-29-windows-lane-hook-and-lefthook-budget.zh.md: 56c1e625d92f01e24f2268deb5d03040278da6af
|
||||
2026-08-29-windows-lane-hook-and-lefthook-budget.md: 40ebba25e459abd6d9ae755f831e703aa826ae39
|
||||
2026-08-29-windows-lane-hook-and-lefthook-budget.zh.md: c14be5da098920dbb7360c01bbb94f0bc22f700b
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ A `git` or `node` spawn spike on the shared-volume runners no longer decides eit
|
|||
|
||||
Both budgets widen what counts as an acceptable duration, so a real slowdown into tens of seconds now passes where the previous ceilings would have caught it. That detection is traded away deliberately: those ceilings were firing on host contention rather than on regressions.
|
||||
|
||||
The hook change applies wherever `DSH_COVERAGE_TEST_TIMEOUT_MS` is set, which today is the Windows coverage lane alone. Lanes that leave it unset keep every Vitest default, including the 10 s hook budget.
|
||||
The hook change applies wherever `DSH_COVERAGE_TEST_TIMEOUT_MS` is set: the Windows coverage lane in [ci.yml](../../../../.github/workflows/ci.yml) and the `serial-windows` master standby in [ci-master.yml](../../../../.github/workflows/ci-master.yml) ([the serial-windows notices timeout note](../process/2026-08-31-serial-windows-notices-timeout-budget.md) records the second lane's adoption). Lanes that leave it unset keep every Vitest default, including the 10 s hook budget.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ Lefthook 套件取 `{ timeout: 90_000 }`,与 [`.github/workflows/ci.yml`](../.
|
|||
|
||||
两份预算都放宽了「多长算可接受」,因此一个退化到几十秒的真实变慢现在会通过,而此前的上限会拦住它。这项检测能力是有意换掉的:那些上限触发的是宿主机争抢,不是回归。
|
||||
|
||||
hook 的改动在所有设置了 `DSH_COVERAGE_TEST_TIMEOUT_MS` 的地方生效,目前仅 Windows 覆盖率 lane 一处。不设置它的 lane 保持全部 Vitest 默认值,包括 10 秒的 hook 预算。
|
||||
hook 的改动在所有设置了 `DSH_COVERAGE_TEST_TIMEOUT_MS` 的地方生效:[ci.yml](../../../../.github/workflows/ci.yml) 的 Windows 覆盖率 lane,以及 [ci-master.yml](../../../../.github/workflows/ci-master.yml) 的 `serial-windows` master standby([serial-windows notices 超时 note](../process/2026-08-31-serial-windows-notices-timeout-budget.zh.md) 记录了第二个 lane 的采用)。不设置它的 lane 保持全部 Vitest 默认值,包括 10 秒的 hook 预算。
|
||||
|
||||
## 备选方案
|
||||
|
||||
|
|
|
|||
1
.github/workflows/ci-master.yml
vendored
1
.github/workflows/ci-master.yml
vendored
|
|
@ -208,6 +208,7 @@ jobs:
|
|||
shell: pwsh
|
||||
env:
|
||||
DSH_COVERAGE_MAX_WORKERS: '1'
|
||||
DSH_COVERAGE_TEST_TIMEOUT_MS: '90000'
|
||||
DSH_GATE_CONCURRENCY: '1'
|
||||
DSH_PUBLINT_CONCURRENCY: '1'
|
||||
run: pnpm run check:ci:windows-complete
|
||||
|
|
|
|||
|
|
@ -201,6 +201,14 @@ describe('CI workflow', () => {
|
|||
expect(serialInstall!.run.split('\n').map(line => line.trim())).toContain('} else {')
|
||||
expect(serialInstall!.run.split('\n').map(line => line.trim())).toContain('pnpm install --frozen-lockfile')
|
||||
expect(serialInstall!.run).not.toContain('$cloneFlag')
|
||||
// The unsharded reference runs the whole coverage inventory at the same
|
||||
// per-test budget the PR coverage lane grants; the default 5000ms times
|
||||
// out load-sensitive store scans (e.g. gen-third-party-notices).
|
||||
const serialGate = serialSteps.find((step): step is Record<string, unknown> & { env?: Record<string, unknown> } => (
|
||||
isRecord(step) && step.name === 'Run complete unsharded Windows gate inventory serially'
|
||||
))
|
||||
expect(serialGate).toBeDefined()
|
||||
expect(serialGate!.env).toMatchObject({ DSH_COVERAGE_TEST_TIMEOUT_MS: '90000' })
|
||||
|
||||
// Aggregate: Wine and the required split native jobs are needed;
|
||||
// windows-coverage is temporarily non-blocking while Windows ACP
|
||||
|
|
|
|||
|
|
@ -279,11 +279,13 @@ const workspaceLinkedManifestCache = new Map<string, VirtualManifest | undefined
|
|||
* Resolve the package version selected for a declaring workspace instead of an
|
||||
* unrelated historical version that still occupies the shared virtual store.
|
||||
* @param name - external package identity.
|
||||
* @param manifests - workspace manifests already loaded by the caller, so one
|
||||
* load serves every dependency instead of a full re-read per name.
|
||||
* @returns the first current workspace link for that package, when installed.
|
||||
*/
|
||||
function workspaceLinkedManifest(name: string): VirtualManifest | undefined {
|
||||
function workspaceLinkedManifest(name: string, manifests: Map<string, Manifest>): VirtualManifest | undefined {
|
||||
if (workspaceLinkedManifestCache.has(name)) return workspaceLinkedManifestCache.get(name)
|
||||
for (const [path, manifest] of loadWorkspaceManifests().manifests) {
|
||||
for (const [path, manifest] of manifests) {
|
||||
if (!ALL_KINDS.some(kind => name in (manifest[kind] ?? {}))) continue
|
||||
const linked = resolve(root, dirname(path), 'node_modules', name, 'package.json')
|
||||
if (!existsSync(linked)) continue
|
||||
|
|
@ -296,8 +298,8 @@ function workspaceLinkedManifest(name: string): VirtualManifest | undefined {
|
|||
}
|
||||
|
||||
/** Resolve one installed external package manifest from either pnpm store. */
|
||||
function installedManifest(name: string, expectedVersion?: string): VirtualManifest | undefined {
|
||||
const linked = workspaceLinkedManifest(name)
|
||||
function installedManifest(name: string, manifests: Map<string, Manifest>, expectedVersion?: string): VirtualManifest | undefined {
|
||||
const linked = workspaceLinkedManifest(name, manifests)
|
||||
if (linked !== undefined && (expectedVersion === undefined || linked.version === expectedVersion)) return linked
|
||||
let manifest: (Manifest & { license?: string; repository?: string | { url?: string }; homepage?: string }) | undefined
|
||||
// Workspace-local link farms can expose a dependency that is not linked at
|
||||
|
|
@ -320,9 +322,9 @@ function installedManifest(name: string, expectedVersion?: string): VirtualManif
|
|||
}
|
||||
|
||||
/** License and repository URL for an installed external package, from the pnpm store. */
|
||||
function installedMetadata(name: string): { license: string; repo: string } {
|
||||
function installedMetadata(name: string, manifests: Map<string, Manifest>): { license: string; repo: string } {
|
||||
const override = OVERRIDES[name]
|
||||
const manifest = installedManifest(name)
|
||||
const manifest = installedManifest(name, manifests)
|
||||
const license = override?.license ?? manifest?.license
|
||||
const rawRepo = typeof manifest?.repository === 'string' ? manifest.repository : manifest?.repository?.url ?? manifest?.homepage
|
||||
const repo = override?.repo ?? normalizeRepo(rawRepo)
|
||||
|
|
@ -332,8 +334,8 @@ function installedMetadata(name: string): { license: string; repo: string } {
|
|||
return { license, repo }
|
||||
}
|
||||
|
||||
function collectClaudeDistribution(): ClaudeDistribution {
|
||||
const manifest = installedManifest(CLAUDE_AGENT_SDK_PACKAGE)
|
||||
function collectClaudeDistribution(manifests: Map<string, Manifest>): ClaudeDistribution {
|
||||
const manifest = installedManifest(CLAUDE_AGENT_SDK_PACKAGE, manifests)
|
||||
if (manifest === undefined) {
|
||||
throw new Error(
|
||||
`gen-third-party-notices: cannot resolve ${CLAUDE_AGENT_SDK_PACKAGE}; run \`pnpm install\`.`,
|
||||
|
|
@ -342,7 +344,7 @@ function collectClaudeDistribution(): ClaudeDistribution {
|
|||
const distribution = claudeDistributionFromManifest(manifest)
|
||||
let installedPayloads = 0
|
||||
for (const payload of distribution.payloads) {
|
||||
const installed = installedManifest(payload.name, payload.version)
|
||||
const installed = installedManifest(payload.name, manifests, payload.version)
|
||||
if (installed === undefined) continue
|
||||
installedPayloads += 1
|
||||
if (
|
||||
|
|
@ -383,12 +385,11 @@ function normalizeRepo(raw: string | undefined): string | undefined {
|
|||
* by tooling, test infrastructure, the website, or the demo leaves — whatever
|
||||
* the declaring section is called — is development-only.
|
||||
*/
|
||||
function collectNpmDeps(): ExternalDep[] {
|
||||
const { manifests, names } = loadWorkspaceManifests()
|
||||
function collectNpmDeps(manifests: Map<string, Manifest>, names: Set<string>): ExternalDep[] {
|
||||
return [...tierExternalDeps(manifests, names)]
|
||||
.filter(([name]) => !FIRST_PARTY.has(name))
|
||||
.sort(([a], [b]) => a.localeCompare(b))
|
||||
.map(([name, runtime]) => ({ name, ...installedMetadata(name), runtime }))
|
||||
.map(([name, runtime]) => ({ name, ...installedMetadata(name, manifests), runtime }))
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -692,7 +693,11 @@ ${rows.join('\n')}
|
|||
*/
|
||||
export function render(): string {
|
||||
verifyBuildTimePins()
|
||||
const npm = collectNpmDeps()
|
||||
// The linked-manifest cache is keyed by name only, so it must not outlive
|
||||
// the manifests map it was resolved from; render() owns that single load.
|
||||
workspaceLinkedManifestCache.clear()
|
||||
const { manifests, names } = loadWorkspaceManifests()
|
||||
const npm = collectNpmDeps(manifests, names)
|
||||
const runtimeDeps = npm.filter(dep => dep.runtime)
|
||||
const devDeps = npm.filter(dep => !dep.runtime)
|
||||
const vendored = collectVendored()
|
||||
|
|
@ -701,7 +706,7 @@ export function render(): string {
|
|||
const claudeDistribution = runtimeDeps.some(
|
||||
dep => dep.name === CLAUDE_AGENT_SDK_PACKAGE,
|
||||
)
|
||||
? collectClaudeDistribution()
|
||||
? collectClaudeDistribution(manifests)
|
||||
: undefined
|
||||
const nonPermissiveDev = devDeps.filter(dep => !isPermissive(dep.license))
|
||||
// A copyleft license reaching a shipped surface is a distribution decision,
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue