Skip to content

perf(streaming): throttle renderTail DOM updates to rAF cadence #70

Description

@jdmanring

Problem

renderTail() in streamingRenderer.js is called on every SSE token. Even with the in-place fast-path patch, every call creates a holder div, sets its innerHTML (full markdown parse → full DOM subtree), reads childNodes, and then discards the holder+subtree if the fast path matched. At 200 tok/sec this is 200 holder divs and 200 full DOM subtrees created and discarded per second in Oilpan.

Capping at requestAnimationFrame cadence (~60fps) reduces these allocations 3× at high token rates with no visible rendering degradation — the display already updates at most once per frame.

Fix

In chat.js, wrap the per-SSE-token _renderStream() call with a rAF guard:

// module-level
let _streamRenderRaf = null;

// in the SSE token handler, replace direct _renderStream():
if (!_streamRenderRaf) {
  _streamRenderRaf = requestAnimationFrame(function () {
    _streamRenderRaf = null;
    _renderStream();
  });
}

The existing StreamRenderer state (lastText) always holds the most recent full text, so the rAF callback renders the latest token even if intermediate tokens were skipped.

Depends on

Issue #XX (renderTail call counter) — fast-path ratio data should be collected first to confirm this optimization is worthwhile.

Classification

Upstream-candidate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions