Agent Note: Streaming fences highlight incrementally

Status: implemented

English | 中文

Problem

While a reply streamed, MarkdownText stripped the fence language before CodeBlock saw it, so code rendered as plain monospace with an empty language banner until the finalize swap recolored the whole reply at once (#1499). The plain arm was a deliberate cost guard, recorded in the assistant-markdown note: shiki tokenizes a document from the top, so highlighting a growing fence naively re-tokenizes the whole fence on every chunk — quadratic in fence length over the stream, the same cost class the incremental markdown parser removes for block parsing. The fix has to deliver highlighting during streaming without reintroducing that cost, without transiently coloring under a wrong grammar while the info string is still mid-chunk, and without changing the settled render.

Decision

Streaming fences parse, tokenize, and reconcile from retained frontiers; the settled output is unchanged.

IncrementalMarkdownParser (packages/client/ui-primitives/src/markdown/incremental.ts) recognizes a parser-confirmed final unclosed top-level fence after the ordinary tail parse. Completed content remains in the retained code node; only the last completed line and current partial line pass through the caller's GFM grammar, preserving its newline, indentation, CRLF, and value semantics without re-parsing the fence prefix. A closing delimiter, non-append input, nested/container fence, or ambiguous reconstruction returns to the ordinary tail parse.
StreamingHighlightSession (packages/client/ui-primitives/src/markdown/highlight.ts) exploits that TextMate tokenization is line-based and forward-only: a line's tokens depend only on its own text and the grammar state entering it, so appended text never changes a completed line's tokens. The session caches completed lines' spans plus shiki's GrammarState after them (getLastGrammarState); updateFrame publishes only newly completed lines plus the still-growing last line, while the compatibility update method materializes the complete result. Per-chunk tokenization excludes the completed prefix; the result is token-identical to a from-scratch tokenization. Non-append input and a resolved-grammar change reset the cache and re-tokenize fully. Each run carries the style shiki's HTML arm would assign it — the css-variables color plus the markup font-style bits the theme lets through (bold/italic/underline; markdown fences carry them); whitespace-only runs fold into their following token as shiki's default mergeWhitespaces does (its underlined/struck-whitespace exemption cannot occur under this theme, whose only underline rule styles inline-link scopes that tokenize spaced text as one run); and a CRLF cut never leaks its \r into the last completed line, matching shiki's own line splitting.
CodeBlock renders the delta frames as a pre.shiki.css-variables React tree with the same attributes and token spans shiki's HTML emits. Completed lines seal into fixed-size React fragments; later updates reuse those fragment elements and reconcile only a bounded pending group plus the mutable tail. Unknown or absent languages keep the identical-geometry plain arm; a lazy grammar renders plain until it registers, then the existing useSyncExternalStore load signal re-renders into highlight — one plain→highlighted transition, no flicker back. Group size is an internal reconciliation unit, not a deployment policy or content limit.
render.tsx and MarkdownText pass lang and context.streaming to fences and key both streaming and settled top-level blocks by source offset. Wrong-grammar transients are structurally impossible: a fence whose info string is still mid-chunk (```py completing to ```python) has no content yet — content only exists after the info line's newline, which finalizes the language — and the empty-value fence keeps the stock <pre>. ```math fences and TeX stay literal until the settled pass; the language banner shows the fence language during streaming.

The final full-document parse still resolves document-wide references and math. When that parse produces the same fence code and language, the source-offset key preserves its CodeBlock instance and the component reuses the complete streamed React tree; cold settled fences continue through highlightToHtml.

Testing

Package tests bound the grammar input accumulated across 800 open-fence lines, compare each incremental result with a full parse, and cover indented delimiters, CRLF split across chunks, closure fallback, and non-append reset. Highlighter tests cover incremental/from-scratch equivalence across multiline grammar state, blank lines, CRLF, and markup styles; delta identity and reset/lazy paths; fixed-group DOM retention; streamed-to-settled DOM identity; and plain or math fallbacks. The assembled Web browser snapshot boots the real Web composition, streams a TypeScript fence through the Host and SSE path, pauses the deterministic LLM adapter while the reply is still active, and snapshots Chromium's Shiki token tree before verifying that settlement preserves it. The tests/fixtures/markdown-dom/*.streaming.txt fixtures pin the intentional streaming divergence from their react-markdown origin: the Shiki span tree and visible language banner replace the plain arm.

Alternatives considered

Pass lang through and re-tokenize the whole fence per chunk. One-line fix, but it reverses the recorded plain-arm rationale without addressing it: a long streaming fence pays quadratic tokenization over the stream, janking exactly on the replies where highlighting matters most.

Highlight only frozen (closed, settled-position) fences during streaming. Bounded cost, but an unclosed fence pins the incremental parser's tail, so the actively growing fence — the one on screen — would stay plain until the reply finishes, failing the issue's "识别语言后即可增量高亮".

Keep only a fixed window of highlighted lines and turn the older prefix into plain text. This bounds live token DOM and can reduce layout further, but changes already rendered content, complicates selection across the window boundary, and makes a tunable presentation policy part of CodeBlock. Retained parser, tokenizer, and React frontiers remove the avoidable repeated work without discarding colors; the complete token DOM remains an explicit limitation rather than a hidden semantic change.

Move highlighting to a worker or async pass. Rejected when shiki was adopted (synchronous highlighting note); an async swap also reintroduces the plain→colored→plain flicker class this change must avoid.

Build the settled HTML string incrementally and keep dangerouslySetInnerHTML. Exact settled parity for free, but React replaces the whole innerHTML per chunk, so the browser re-parses and rebuilds every line's DOM each time — O(fence) DOM churn that forfeits the token-level win the session provides.

Consequences

Streaming code is readable as it arrives: tokens color as soon as the language is known; completed top-level fence content neither re-parses nor re-tokenizes; sealed React groups keep their elements and DOM; and settlement preserves the highlighted tree. The package owns a small mirror of shiki's HTML-arm conventions — the pre attributes and the whitespace fold — pinned by the arm-parity test, so a shiki upgrade that changes either fails loud there instead of drifting the two arms apart. The streaming DOM-parity fixtures pin Shiki span trees as an intentional divergence from their react-markdown origin. The retained DOM still grows with final token count, so browser style and layout work is not length-independent. Nested/container fences use the ordinary tail parser, and the still-growing last line re-tokenizes per chunk; a pathological single-line fence therefore remains the worst case.

8.1 KiB Raw Blame History