Skip to content

Improve Live Debugger benchmark comment guidance - #507

Open
watson wants to merge 1 commit into
masterfrom
watson/improve-live-debugger-benchmark-comment
Open

Improve Live Debugger benchmark comment guidance#507
watson wants to merge 1 commit into
masterfrom
watson/improve-live-debugger-benchmark-comment

Conversation

@watson

@watson watson commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Summary

  • explain the Tiny and Hot benchmark workloads and how they relate to real applications
  • link the workload documentation from the PR benchmark summary
  • prioritize benchmark results while moving supporting metadata below the table
  • keep raw samples folded on successful runs and highlight them when failures occur

Testing

  • yarn test:unit packages/tests/src/bench/liveDebuggerRuntime/reporter/benchReporter.test.ts --runInBand
  • yarn lint packages/tests/src/bench/liveDebuggerRuntime/reporter/benchReporter.ts packages/tests/src/bench/liveDebuggerRuntime/reporter/benchReporter.test.ts
  • yarn workspace @dd/tests typecheck
  • yarn cli integrity

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

Live Debugger Runtime Benchmark

SDK-loaded dormant-probe runtime overhead, measured against an uninstrumented bundle in the same browser session.

Browser Workload Quality Per-call overhead upper
chrome Hot clean <= 4.39 ns
chrome Tiny clean <= 0.36 ns
firefox Hot clean <= 10.01 ns
firefox Tiny clean <= 0.36 ns
safari Hot clean <= 5.33 ns
safari Tiny clean <= 0.98 ns

What do the Tiny and Hot workloads represent?

Browser Debugger SDK: 7.12.0 · built 2026-09-03 · S3 ETag 59df9831

Full diagnostics
browser  workload  quality  per-call overhead upper  overhead upper          95% CI        A/A diag        block CI  acf(1)   baseline  instrumented                         samples
-------  --------  -------  -----------------------  --------------  --------------  --------------  --------------  ------  ---------  ------------  ------------------------------
chrome   Hot       clean                 <= 4.39 ns        <= 7.90%   4.33..4.38 ns  -0.00..0.02 ns   4.32..4.39 ns   -0.04  50.658 ms     54.627 ms   102 (trim 20%, outliers 2.9%)
chrome   Tiny      clean                 <= 0.36 ns        <= 4.37%   0.34..0.36 ns  -0.01..0.03 ns   0.34..0.36 ns   -0.20  53.698 ms     55.973 ms  102 (trim 20%, outliers 11.8%)
firefox  Hot       clean                <= 10.01 ns       <= 36.51%  9.99..10.01 ns  -0.00..0.01 ns  9.99..10.01 ns    0.05  40.530 ms     55.320 ms   102 (trim 20%, outliers 7.8%)
firefox  Tiny      clean                 <= 0.36 ns        <= 2.32%   0.35..0.36 ns  -0.00..0.00 ns   0.35..0.36 ns    0.03  53.220 ms     54.440 ms   102 (trim 20%, outliers 6.9%)
safari   Hot       clean                 <= 5.33 ns       <= 19.64%   5.31..5.33 ns  -0.01..0.00 ns   5.32..5.33 ns    0.03  46.310 ms     55.400 ms   102 (trim 20%, outliers 3.9%)
safari   Tiny      clean                 <= 0.98 ns       <= 23.17%   0.78..0.92 ns  -0.00..0.00 ns   0.65..0.98 ns    0.63  46.400 ms     56.510 ms   102 (trim 20%, outliers 2.0%)

Raw samples are in the live-debugger-runtime-bench-results artifact.

@watson
watson marked this pull request as ready for review September 5, 2026 09:43
@watson
watson requested review from a team as code owners September 5, 2026 09:43
@watson
watson requested review from AndrejBozin1 and removed request for a team September 5, 2026 09:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants