deepseek-harness/.agents/notes/implemented/architecture/2026-07-29-projected-token-usage-and-request-context.md

62 lines
8.4 KiB
Markdown
Raw Normal View History

feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
# Agent Note: Projected token usage and context occupancy
Status: implemented
English | [中文](2026-07-29-projected-token-usage-and-request-context.zh.md)
## Problem
feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
The Web stats line derived token totals from the currently loaded conversation nodes. That window is paged, so scrolling changed the totals, and compaction replaces visible content without preserving the billing behind it. Durable provider billing needs a source that survives both.
feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
Context occupancy needs a numerator and a denominator that no existing surface carried to the browser: the prompt size of the latest request, and the capacity of the route it used.
## Decision
feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
Both values are ordinary durable session-projection state. `@deepseek-ai/dsh-token-meter` registers two units when `ctx.sessionProjections` is present.
`tokenUsage` folds the complete durable log into uncached input, output, cache-read, and cache-write buckets. An `assistant/chunk` usage sample survives a later failed request; an `assistant/message` usage value replaces the earlier sample from the same model attempt instead of double-counting it. A matching `llm/retry-started` boundary ends that replacement scope, so a retry with the same `(turn, step)` contributes a new attempt. Reasoning stays an output subdivision. Compaction and surface replacement do not erase earlier billing.
Token-meter also owns the shared pure attempt/Turn fold over durable events. It applies the same retry boundary while adding the stricter completeness and exact-total checks required by an exact per-Turn disclosure. A presentation consumer may select a complete Turn window and invoke that fold, but does not own or duplicate the accounting semantics.
`contextPressure` carries optional `pressureTokens` — the newest provider-reported prompt size, summing uncached input plus cache reads and writes, excluding output — and optional `contextWindow` from the newest `request/context` record. Neither field is synthesized before its source exists.
`request/context` is a new log-only session event recording registration-bound metadata for the route a request resolved to. AgentLoop appends it inside the step beside `request/header`, from the context metadata `prepareCall()` now returns alongside the resolved config — the same registration-bound lookup that already validated reasoning, so no second resolve happens. It is skipped when provider, model, and capacity all match the previous record. A route whose adapter advertises no capacity is recorded with `contextWindow` absent, clearing an older route's denominator.
feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
Capacity deliberately stays out of `EpochHeader`. That type is the reconstruction contract — what a request was built from — and `headerEquals` compares it field-wise to decide whether a snapshot is a real `change`. Capacity is adapter metadata describing a route, so placing it there would let a capacity change masquerade as a request-envelope change and would drag it into the loop's reconstruction invariant.
feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
Both units ride the standard projection lifecycle: history tail baselines, `session/projection` live frames, higher-seq-wins client storage, JSON checkpoints, cache recovery, and unit unload. There is no token-specific history field, mux frame, projector, revision counter, or client fence.
The Web `StatsLine` reads both through the standard `useProjection` seat. Window nodes still supply turn and step counts plus LLM and tool wall times — those answer "what is on screen" and are correctly window-scoped. Durable token and context groups remain when compaction leaves no visible assistant step. Cache writes count in billed input and in the cache-hit denominator. A deployment without token-meter drops the token groups; occupancy stays hidden until both pressure and capacity are known.
feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
## Context occupancy is approximate, and that is the decision
`pressureTokens` and `contextWindow` are independent last-wins fields, not one atomic observation. Switching models pairs a fresh capacity with the previous route's pressure until the next request reports usage, and the numerator describes the last request rather than the surface as it currently stands.
This was accepted deliberately. An occupancy percentage is a user-facing reference figure: nothing in the harness makes decisions from it, and compaction reads `measure()` directly instead. The TUI status line has always computed occupancy this way, dividing a `measure()` total by a capacity resolved separately for the selected model — so an atomic variant here would have been the outlier, not the norm.
The non-atomicity is deliberate, not a defect. A consumer that genuinely needs an exact same-boundary figure should call `ctx.tokenMeter.measure()` at its own request boundary, where both values are available together, rather than read this projection.
## Alternatives considered
**An atomic request-boundary snapshot delivered as a transient mux frame (implemented, then rejected).** An earlier revision emitted `session/model-request`: one non-replayable frame carrying `contextTokens` and `contextWindow` measured at the same `agent/model-request` boundary. Being the only non-replayable class on the mux stream is what broke it. Host and mux are independent SSE streams with no cross-stream ordering, so a request emitted before a removal could arrive after `host/session-removed` and revive a dead session's telemetry, while a legitimate request for a new lifecycle reusing the same id could be fenced by a late removal. `session/subscribed` is not lifecycle proof — it says a queue began subscribing to an id, not that a new in-memory session replaced an older one — and `lastSeq` is a durable watermark two lifecycles can share. A correct fix required a monotonic lifecycle generation on the frame, on subscription, and on removal, plus a client watermark comparison.
feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
That cost bought a worse display: occupancy went blank after every reconnect and never moved while a conversation grew. It also made the transport adapter a measurement site calling the O(surface) `measure()` on every request, and expressed reconnect state through a synthetic `cancelled` open error the UI had to special-case.
feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
**Fold the loaded node window in React.** Cannot survive pagination or compaction, and makes a presentation package reconstruct log semantics.
feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
**Publish usage only with final assistant messages.** A request that reports a usage chunk and then fails would lose its billing.
feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
**Resolve capacity inside token-meter.** The package documents itself as independent of model routing and is otherwise a pure reader that never appends to the log. AgentLoop already holds the resolved metadata where the header is written.
**Extend the `session.models` RPC with capacity.** The handler already resolves and discards it, so the field is nearly free — but `StatsLine` lives in `ui-conversation` while the model directory lives in `ui-model-selection`, and `ui-conversation` cannot depend on `ui-model-selection`. Delivering it would have required either a second dock entry splitting one text row across two plugins, or a cross-plugin store write.
feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
**Add a context circle beside the model selector.** That placement suggests selected-model state. The stats line carries the figure without a duplicate UI or data path.
## Consequences
feat(web): show durable token usage and context occupancy in the stats line The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
2026-07-30 14:48:19 +08:00
Token totals stay stable across pagination, compaction, replay, restart, and reconnect, because they are ordinary durable projection state recovered through the generic paths. The cross-stream reordering race is gone by construction rather than fenced.
Occupancy is approximate in the ways documented above. It is available immediately after restore or reconnect, since both fields are durable, at the cost of describing the last recorded request rather than an exact current boundary.
Each session log gains one small `request/context` record per route or advertised-capacity change. Token-meter is the canonical owner of durable usage semantics, including retry-attempt separation in the cumulative projection and the reusable exact attempt/Turn fold; Web Chat only selects a complete loaded Turn and renders the fold result. The TUI retains its live per-step map because it does not mount the generic projection seam, and the standalone browser fixture mirrors the unit. Connection and API Gateway carry no token-specific code, own no per-session metrics cache, and perform no measurement. The browser keeps two generic projection values and no connection-local telemetry, and streaming text deltas still do not force the stats line to recompute.