Skip to content

[Bug]: streaming benchmark does not exercise suspension #517

Description

Before filing

  • I searched existing issues and did not find a duplicate.
  • This is not a security vulnerability.

Area

Performance

Summary

streaming_resource_bench labels rows as streaming and streaming POOLED, but its contact-book fixture contains no runtime <boundary>. Those paths complete in one step and never suspend, park continuation state, or resume a StreamingSession.

This gap allowed a deterministic suspension-path memory regression to look like a clean streaming improvement in #513 until a separate three-boundary probe was written.

Minimal reproduction

  1. Instrument or inspect a StreamingSession run in streaming_resource_bench.
  2. Observe that it returns completed output without yielding a boundary.
  3. Compare with a fixture containing three boundaries inside components that bind props and execute <for> bodies.

Steps to reproduce

  1. Run cargo xtask bench streaming-resource against the contact-book fixture.
  2. Record its streaming allocation counts.
  3. Run a real StreamingSession through multiple runtime boundaries while populated scopes are active.
  4. Observe that changes to parked scope and continuation state can regress only the second measurement.

Expected behavior

At least one benchmark identified as streaming should exercise the complete suspension lifecycle: render to a boundary, park continuation and scope state, resume with boundary content, and repeat. Its allocation and byte counts should detect regressions in state that crosses host calls.

Actual behavior

The existing streaming rows measure a one-step render through the streaming API, not a suspending session. On #513, they reported improvements:

streaming allocs/run:        155 -> 136
streaming POOLED allocs/run: 148 -> 129

A realistic session driven through three boundaries instead exposed a deterministic regression:

allocs/run: 243 -> 249       (+2.5%)
bytes/run:  25,081 -> 29,539 (+17.8%)
output:     4940 B -> 4940 B (unchanged)

All cells were identical across four runs of each build. The regression came from wider parked Cow<str> scope keys and losing the scope-map pool across suspensions - behavior the current benchmark never reaches.

Suggested fix

Add a benchmark case that uses a real StreamingSession and crosses multiple runtime boundaries while non-empty component and loop scopes are live. Report exact allocs/run, bytes/run, and output bytes for the full suspend/resume lifecycle.

Keep the existing one-step rows because they measure a useful path, but rename them so streaming is not read as coverage of suspension. A name such as streaming one-step would distinguish API mode from actual suspension.

The new case should be part of the normal performance validation so future changes to SessionCore, ContinuationVm, parked scopes, route children, or pooling cannot regress silently.

Environment

  • OS: macOS (Darwin), Apple Silicon
  • Rust version: workspace toolchain
  • Node.js version: n/a
  • WebUI package or CLI version: compared base e908d327 with PR perf: avoid cloning nested route trees #513 at c7ead955
  • Command: existing cargo xtask bench streaming-resource plus a dedicated three-boundary StreamingSession allocation probe

Logs or terminal output

Suspending session, 3 boundaries
Base: allocs/run=243 bytes/run=25081 output=4940
#513: allocs/run=249 bytes/run=29539 output=4940

Found while validating #513. The existing deterministic allocation and byte columns remain trustworthy for the one-step render paths they actually measure; the issue is coverage and naming, not measurement variance.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions