Skip to content

Tests summary in pipeline and improved test tags to be used in Test Optimization product#10530

Open
bric3 wants to merge 15 commits intomasterfrom
bdu/total-tests
Open

Tests summary in pipeline and improved test tags to be used in Test Optimization product#10530
bric3 wants to merge 15 commits intomasterfrom
bdu/total-tests

Conversation

@bric3
Copy link
Contributor

@bric3 bric3 commented Feb 5, 2026 •
edited
Loading

What Does This Do

Compute ran tests in a pipeline, e.g.

Overall Summary

Metric Count
Total Tests 284627
Passed 237614
Failed 0
Skipped 47013

Breakdown by JVM Version

JVM Version Total Tests Passed Failed Skipped Job Count
8 56996 47575 0 9421 27
semeru8 1064 993 0 71 2
11 42908 37096 0 5812 20
17 61314 50719 0 10595 26
graalvm17 3 1 0 2 1
21 61215 50680 0 10535 26
graalvm21 3 1 0 2 1
25 61121 50548 0 10573 26
graalvm25 3 1 0 2 1

Not some sections can be expanded

image

Also, this PR push new tags to JUnit report allow easier querying in the TO interface. This query for example, to get the same picture on tests

image

Motivation

Having a holistic picture of how many tests passed.

If the numbers remain stable for any PR, this could serve as a basis for a possible merge gate.

Additional Notes

@bric3 bric3 requested a review from a team as a code owner February 5, 2026 14:17
@bric3 bric3 requested review from randomanderson and removed request for a team February 5, 2026 14:17
@github-actions
Copy link
Contributor

github-actions bot commented Feb 5, 2026 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@bric3 bric3 added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Feb 5, 2026
@bric3 bric3 changed the title Compute ran tests in a PR Compute ran tests in a pipeline Feb 5, 2026
@pr-commenter
Copy link

pr-commenter bot commented Feb 5, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bdu/total-tests
git_commit_date 1770299006 1770299907
git_commit_sha 6fdb8c8 262e383
release_version 1.60.0-SNAPSHOT~6fdb8c86a8 1.60.0-SNAPSHOT~262e38355d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1770301621 1770301621
ci_job_id 1403926535 1403926535
ci_pipeline_id 94865192 94865192
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-p6kyr1fn 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-p6kyr1fn 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 61 metrics, 10 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~262e38355d, baseline=1.60.0-SNAPSHOT~6fdb8c86a8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.062 s) : 0, 1062153
Total [baseline] (8.739 s) : 0, 8738934
Agent [candidate] (1.074 s) : 0, 1074196
Total [candidate] (8.833 s) : 0, 8832854
section iast
Agent [baseline] (1.227 s) : 0, 1226875
Total [baseline] (9.393 s) : 0, 9392813
Agent [candidate] (1.234 s) : 0, 1234163
Total [candidate] (9.432 s) : 0, 9432393
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent iast 1.227 s 164.722 ms (15.5%)
Total tracing 8.739 s -
Total iast 9.393 s 653.878 ms (7.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.074 s -
Agent iast 1.234 s 159.967 ms (14.9%)
Total tracing 8.833 s -
Total iast 9.432 s 599.54 ms (6.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~262e38355d, baseline=1.60.0-SNAPSHOT~6fdb8c86a8

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.185 ms) : 0, 1185
crashtracking [candidate] (1.195 ms) : 0, 1195
BytebuddyAgent [baseline] (626.379 ms) : 0, 626379
BytebuddyAgent [candidate] (632.601 ms) : 0, 632601
AgentMeter [baseline] (28.791 ms) : 0, 28791
AgentMeter [candidate] (29.115 ms) : 0, 29115
GlobalTracer [baseline] (257.483 ms) : 0, 257483
GlobalTracer [candidate] (260.765 ms) : 0, 260765
AppSec [baseline] (32.823 ms) : 0, 32823
AppSec [candidate] (33.317 ms) : 0, 33317
Debugger [baseline] (60.319 ms) : 0, 60319
Debugger [candidate] (60.204 ms) : 0, 60204
Remote Config [baseline] (616.767 µs) : 0, 617
Remote Config [candidate] (644.183 µs) : 0, 644
Telemetry [baseline] (14.598 ms) : 0, 14598
Telemetry [candidate] (16.327 ms) : 0, 16327
Flare Poller [baseline] (4.646 ms) : 0, 4646
Flare Poller [candidate] (4.652 ms) : 0, 4652
section iast
crashtracking [baseline] (1.183 ms) : 0, 1183
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (792.749 ms) : 0, 792749
BytebuddyAgent [candidate] (796.952 ms) : 0, 796952
AgentMeter [baseline] (11.22 ms) : 0, 11220
AgentMeter [candidate] (11.344 ms) : 0, 11344
GlobalTracer [baseline] (246.911 ms) : 0, 246911
GlobalTracer [candidate] (248.955 ms) : 0, 248955
AppSec [baseline] (34.988 ms) : 0, 34988
AppSec [candidate] (32.217 ms) : 0, 32217
Debugger [baseline] (64.985 ms) : 0, 64985
Debugger [candidate] (68.307 ms) : 0, 68307
Remote Config [baseline] (604.559 µs) : 0, 605
Remote Config [candidate] (544.056 µs) : 0, 544
Telemetry [baseline] (8.635 ms) : 0, 8635
Telemetry [candidate] (8.725 ms) : 0, 8725
Flare Poller [baseline] (3.446 ms) : 0, 3446
Flare Poller [candidate] (3.502 ms) : 0, 3502
IAST [baseline] (26.886 ms) : 0, 26886
IAST [candidate] (27.11 ms) : 0, 27110
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~262e38355d, baseline=1.60.0-SNAPSHOT~6fdb8c86a8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1065655
Total [baseline] (10.767 s) : 0, 10766867
Agent [candidate] (1.071 s) : 0, 1071404
Total [candidate] (10.894 s) : 0, 10893584
section appsec
Agent [baseline] (1.235 s) : 0, 1234768
Total [baseline] (11.033 s) : 0, 11033222
Agent [candidate] (1.239 s) : 0, 1239225
Total [candidate] (10.989 s) : 0, 10989075
section iast
Agent [baseline] (1.228 s) : 0, 1227814
Total [baseline] (11.148 s) : 0, 11147917
Agent [candidate] (1.234 s) : 0, 1233705
Total [candidate] (11.257 s) : 0, 11257158
section profiling
Agent [baseline] (1.19 s) : 0, 1190487
Total [baseline] (10.896 s) : 0, 10895647
Agent [candidate] (1.19 s) : 0, 1190090
Total [candidate] (11.044 s) : 0, 11043716
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent appsec 1.235 s 169.112 ms (15.9%)
Agent iast 1.228 s 162.158 ms (15.2%)
Agent profiling 1.19 s 124.832 ms (11.7%)
Total tracing 10.767 s -
Total appsec 11.033 s 266.355 ms (2.5%)
Total iast 11.148 s 381.05 ms (3.5%)
Total profiling 10.896 s 128.78 ms (1.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.071 s -
Agent appsec 1.239 s 167.821 ms (15.7%)
Agent iast 1.234 s 162.301 ms (15.1%)
Agent profiling 1.19 s 118.686 ms (11.1%)
Total tracing 10.894 s -
Total appsec 10.989 s 95.491 ms (0.9%)
Total iast 11.257 s 363.574 ms (3.3%)
Total profiling 11.044 s 150.132 ms (1.4%) 
gantt
    title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~262e38355d, baseline=1.60.0-SNAPSHOT~6fdb8c86a8

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.184 ms) : 0, 1184
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (628.714 ms) : 0, 628714
BytebuddyAgent [candidate] (632.155 ms) : 0, 632155
AgentMeter [baseline] (28.826 ms) : 0, 28826
AgentMeter [candidate] (29.119 ms) : 0, 29119
GlobalTracer [baseline] (257.995 ms) : 0, 257995
GlobalTracer [candidate] (258.982 ms) : 0, 258982
AppSec [baseline] (32.818 ms) : 0, 32818
AppSec [candidate] (33.119 ms) : 0, 33119
Debugger [baseline] (62.504 ms) : 0, 62504
Debugger [candidate] (62.941 ms) : 0, 62941
Remote Config [baseline] (617.417 µs) : 0, 617
Remote Config [candidate] (633.835 µs) : 0, 634
Telemetry [baseline] (12.174 ms) : 0, 12174
Telemetry [candidate] (12.258 ms) : 0, 12258
Flare Poller [baseline] (5.448 ms) : 0, 5448
Flare Poller [candidate] (5.423 ms) : 0, 5423
section appsec
crashtracking [baseline] (1.174 ms) : 0, 1174
crashtracking [candidate] (1.176 ms) : 0, 1176
BytebuddyAgent [baseline] (655.678 ms) : 0, 655678
BytebuddyAgent [candidate] (656.426 ms) : 0, 656426
AgentMeter [baseline] (11.84 ms) : 0, 11840
AgentMeter [candidate] (11.938 ms) : 0, 11938
GlobalTracer [baseline] (257.815 ms) : 0, 257815
GlobalTracer [candidate] (259.119 ms) : 0, 259119
AppSec [baseline] (167.985 ms) : 0, 167985
AppSec [candidate] (168.507 ms) : 0, 168507
Debugger [baseline] (66.284 ms) : 0, 66284
Debugger [candidate] (67.722 ms) : 0, 67722
Remote Config [baseline] (679.181 µs) : 0, 679
Remote Config [candidate] (680.904 µs) : 0, 681
Telemetry [baseline] (9.027 ms) : 0, 9027
Telemetry [candidate] (9.25 ms) : 0, 9250
Flare Poller [baseline] (3.601 ms) : 0, 3601
Flare Poller [candidate] (3.604 ms) : 0, 3604
IAST [baseline] (25.316 ms) : 0, 25316
IAST [candidate] (25.526 ms) : 0, 25526
section iast
crashtracking [baseline] (1.173 ms) : 0, 1173
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (793.357 ms) : 0, 793357
BytebuddyAgent [candidate] (796.533 ms) : 0, 796533
AgentMeter [baseline] (11.211 ms) : 0, 11211
AgentMeter [candidate] (11.273 ms) : 0, 11273
GlobalTracer [baseline] (246.715 ms) : 0, 246715
GlobalTracer [candidate] (248.064 ms) : 0, 248064
AppSec [baseline] (31.42 ms) : 0, 31420
AppSec [candidate] (30.743 ms) : 0, 30743
Debugger [baseline] (69.096 ms) : 0, 69096
Debugger [candidate] (70.714 ms) : 0, 70714
Remote Config [baseline] (546.567 µs) : 0, 547
Remote Config [candidate] (552.159 µs) : 0, 552
Telemetry [baseline] (8.658 ms) : 0, 8658
Telemetry [candidate] (8.801 ms) : 0, 8801
Flare Poller [baseline] (3.458 ms) : 0, 3458
Flare Poller [candidate] (3.497 ms) : 0, 3497
IAST [baseline] (26.987 ms) : 0, 26987
IAST [candidate] (27.144 ms) : 0, 27144
section profiling
crashtracking [baseline] (1.213 ms) : 0, 1213
crashtracking [candidate] (1.216 ms) : 0, 1216
BytebuddyAgent [baseline] (681.985 ms) : 0, 681985
BytebuddyAgent [candidate] (680.745 ms) : 0, 680745
AgentMeter [baseline] (8.728 ms) : 0, 8728
AgentMeter [candidate] (8.75 ms) : 0, 8750
GlobalTracer [baseline] (216.028 ms) : 0, 216028
GlobalTracer [candidate] (216.103 ms) : 0, 216103
AppSec [baseline] (32.263 ms) : 0, 32263
AppSec [candidate] (32.408 ms) : 0, 32408
Debugger [baseline] (67.475 ms) : 0, 67475
Debugger [candidate] (67.418 ms) : 0, 67418
Remote Config [baseline] (604.2 µs) : 0, 604
Remote Config [candidate] (598.824 µs) : 0, 599
Telemetry [baseline] (8.898 ms) : 0, 8898
Telemetry [candidate] (8.889 ms) : 0, 8889
Flare Poller [baseline] (3.729 ms) : 0, 3729
Flare Poller [candidate] (3.822 ms) : 0, 3822
ProfilingAgent [baseline] (99.511 ms) : 0, 99511
ProfilingAgent [candidate] (100.272 ms) : 0, 100272
Profiling [baseline] (100.093 ms) : 0, 100093
Profiling [candidate] (100.859 ms) : 0, 100859
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bdu/total-tests
git_commit_date 1770299006 1770299907
git_commit_sha 6fdb8c8 262e383
release_version 1.60.0-SNAPSHOT~6fdb8c86a8 1.60.0-SNAPSHOT~262e38355d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1770302109 1770302109
ci_job_id 1403926536 1403926536
ci_pipeline_id 94865192 94865192
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-sgp1knco 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-sgp1knco 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 2 performance regressions! Performance is the same for 15 metrics, 17 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_FULL:high_load worse
[+193.845µs; +398.168µs] or [+3.852%; +7.912%]		 worse
[+516.925µs; +1113.033µs] or [+4.349%; +9.364%]		 unstable
[-119.163op/s; +32.351op/s] or [-14.567%; +3.955%]		 5.328ms 12.701ms 774.656op/s 5.032ms 11.886ms 818.062op/s
scenario:load:petclinic:appsec:high_load better
[-2.687ms; -1.290ms] or [-13.598%; -6.527%]		 better
[-4.061ms; -1.370ms] or [-12.788%; -4.316%]		 unstable
[-6.885op/s; +45.760op/s] or [-2.934%; +19.498%]		 17.771ms 29.038ms 254.125op/s 19.759ms 31.754ms 234.688op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~262e38355d, baseline=1.60.0-SNAPSHOT~6fdb8c86a8
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.206 ms) : 1194, 1218
.   : milestone, 1206,
iast (3.2 ms) : 3157, 3243
.   : milestone, 3200,
iast_FULL (5.65 ms) : 5595, 5706
.   : milestone, 5650,
iast_GLOBAL (3.59 ms) : 3532, 3649
.   : milestone, 3590,
profiling (2.271 ms) : 2249, 2293
.   : milestone, 2271,
tracing (1.772 ms) : 1757, 1786
.   : milestone, 1772,
section candidate
no_agent (1.203 ms) : 1190, 1215
.   : milestone, 1203,
iast (3.319 ms) : 3273, 3365
.   : milestone, 3319,
iast_FULL (5.972 ms) : 5911, 6032
.   : milestone, 5972,
iast_GLOBAL (3.665 ms) : 3604, 3726
.   : milestone, 3665,
profiling (2.307 ms) : 2280, 2334
.   : milestone, 2307,
tracing (1.828 ms) : 1812, 1843
.   : milestone, 1828,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.206 ms [1.194 ms, 1.218 ms] -
iast 3.2 ms [3.157 ms, 3.243 ms] 1.994 ms (165.4%)
iast_FULL 5.65 ms [5.595 ms, 5.706 ms] 4.444 ms (368.6%)
iast_GLOBAL 3.59 ms [3.532 ms, 3.649 ms] 2.384 ms (197.7%)
profiling 2.271 ms [2.249 ms, 2.293 ms] 1.065 ms (88.4%)
tracing 1.772 ms [1.757 ms, 1.786 ms] 565.8 µs (46.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.203 ms [1.19 ms, 1.215 ms] -
iast 3.319 ms [3.273 ms, 3.365 ms] 2.117 ms (176.0%)
iast_FULL 5.972 ms [5.911 ms, 6.032 ms] 4.769 ms (396.6%)
iast_GLOBAL 3.665 ms [3.604 ms, 3.726 ms] 2.463 ms (204.8%)
profiling 2.307 ms [2.28 ms, 2.334 ms] 1.105 ms (91.9%)
tracing 1.828 ms [1.812 ms, 1.843 ms] 625.218 µs (52.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~262e38355d, baseline=1.60.0-SNAPSHOT~6fdb8c86a8
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.17 ms) : 17982, 18357
.   : milestone, 18170,
appsec (19.888 ms) : 19680, 20096
.   : milestone, 19888,
code_origins (17.839 ms) : 17659, 18020
.   : milestone, 17839,
iast (17.73 ms) : 17553, 17907
.   : milestone, 17730,
profiling (20.101 ms) : 19892, 20309
.   : milestone, 20101,
tracing (17.86 ms) : 17682, 18039
.   : milestone, 17860,
section candidate
no_agent (19.233 ms) : 19037, 19428
.   : milestone, 19233,
appsec (18.364 ms) : 18178, 18549
.   : milestone, 18364,
code_origins (17.556 ms) : 17379, 17732
.   : milestone, 17556,
iast (17.568 ms) : 17391, 17746
.   : milestone, 17568,
profiling (19.76 ms) : 19563, 19956
.   : milestone, 19760,
tracing (17.547 ms) : 17376, 17718
.   : milestone, 17547,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.17 ms [17.982 ms, 18.357 ms] -
appsec 19.888 ms [19.68 ms, 20.096 ms] 1.718 ms (9.5%)
code_origins 17.839 ms [17.659 ms, 18.02 ms] -330.332 µs (-1.8%)
iast 17.73 ms [17.553 ms, 17.907 ms] -439.66 µs (-2.4%)
profiling 20.101 ms [19.892 ms, 20.309 ms] 1.931 ms (10.6%)
tracing 17.86 ms [17.682 ms, 18.039 ms] -309.436 µs (-1.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.233 ms [19.037 ms, 19.428 ms] -
appsec 18.364 ms [18.178 ms, 18.549 ms] -869.02 µs (-4.5%)
code_origins 17.556 ms [17.379 ms, 17.732 ms] -1.677 ms (-8.7%)
iast 17.568 ms [17.391 ms, 17.746 ms] -1.664 ms (-8.7%)
profiling 19.76 ms [19.563 ms, 19.956 ms] 527.125 µs (2.7%)
tracing 17.547 ms [17.376 ms, 17.718 ms] -1.685 ms (-8.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bdu/total-tests
git_commit_date 1770299006 1770299907
git_commit_sha 6fdb8c8 262e383
release_version 1.60.0-SNAPSHOT~6fdb8c86a8 1.60.0-SNAPSHOT~262e38355d
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1770301844 1770301844
ci_job_id 1403926537 1403926537
ci_pipeline_id 94865192 94865192
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-k7rjc1ie 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-k7rjc1ie 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~262e38355d, baseline=1.60.0-SNAPSHOT~6fdb8c86a8
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.021 s) : 15021000, 15021000
.   : milestone, 15021000,
appsec (15.05 s) : 15050000, 15050000
.   : milestone, 15050000,
iast (18.331 s) : 18331000, 18331000
.   : milestone, 18331000,
iast_GLOBAL (17.829 s) : 17829000, 17829000
.   : milestone, 17829000,
profiling (15.071 s) : 15071000, 15071000
.   : milestone, 15071000,
tracing (14.693 s) : 14693000, 14693000
.   : milestone, 14693000,
section candidate
no_agent (15.015 s) : 15015000, 15015000
.   : milestone, 15015000,
appsec (14.797 s) : 14797000, 14797000
.   : milestone, 14797000,
iast (18.701 s) : 18701000, 18701000
.   : milestone, 18701000,
iast_GLOBAL (17.696 s) : 17696000, 17696000
.   : milestone, 17696000,
profiling (15.386 s) : 15386000, 15386000
.   : milestone, 15386000,
tracing (14.655 s) : 14655000, 14655000
.   : milestone, 14655000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.021 s [15.021 s, 15.021 s] -
appsec 15.05 s [15.05 s, 15.05 s] 29.0 ms (0.2%)
iast 18.331 s [18.331 s, 18.331 s] 3.31 s (22.0%)
iast_GLOBAL 17.829 s [17.829 s, 17.829 s] 2.808 s (18.7%)
profiling 15.071 s [15.071 s, 15.071 s] 50.0 ms (0.3%)
tracing 14.693 s [14.693 s, 14.693 s] -328.0 ms (-2.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.015 s [15.015 s, 15.015 s] -
appsec 14.797 s [14.797 s, 14.797 s] -218.0 ms (-1.5%)
iast 18.701 s [18.701 s, 18.701 s] 3.686 s (24.5%)
iast_GLOBAL 17.696 s [17.696 s, 17.696 s] 2.681 s (17.9%)
profiling 15.386 s [15.386 s, 15.386 s] 371.0 ms (2.5%)
tracing 14.655 s [14.655 s, 14.655 s] -360.0 ms (-2.4%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~262e38355d, baseline=1.60.0-SNAPSHOT~6fdb8c86a8
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.482 ms) : 1470, 1493
.   : milestone, 1482,
appsec (2.528 ms) : 2473, 2583
.   : milestone, 2528,
iast (2.268 ms) : 2199, 2337
.   : milestone, 2268,
iast_GLOBAL (2.311 ms) : 2241, 2380
.   : milestone, 2311,
profiling (2.096 ms) : 2041, 2151
.   : milestone, 2096,
tracing (2.076 ms) : 2022, 2131
.   : milestone, 2076,
section candidate
no_agent (1.477 ms) : 1466, 1489
.   : milestone, 1477,
appsec (3.791 ms) : 3569, 4014
.   : milestone, 3791,
iast (2.266 ms) : 2197, 2335
.   : milestone, 2266,
iast_GLOBAL (2.304 ms) : 2234, 2373
.   : milestone, 2304,
profiling (2.11 ms) : 2053, 2166
.   : milestone, 2110,
tracing (2.078 ms) : 2024, 2132
.   : milestone, 2078,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.482 ms [1.47 ms, 1.493 ms] -
appsec 2.528 ms [2.473 ms, 2.583 ms] 1.046 ms (70.6%)
iast 2.268 ms [2.199 ms, 2.337 ms] 786.022 µs (53.0%)
iast_GLOBAL 2.311 ms [2.241 ms, 2.38 ms] 829.104 µs (56.0%)
profiling 2.096 ms [2.041 ms, 2.151 ms] 614.26 µs (41.5%)
tracing 2.076 ms [2.022 ms, 2.131 ms] 594.632 µs (40.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.477 ms [1.466 ms, 1.489 ms] -
appsec 3.791 ms [3.569 ms, 4.014 ms] 2.314 ms (156.7%)
iast 2.266 ms [2.197 ms, 2.335 ms] 789.188 µs (53.4%)
iast_GLOBAL 2.304 ms [2.234 ms, 2.373 ms] 826.464 µs (55.9%)
profiling 2.11 ms [2.053 ms, 2.166 ms] 632.687 µs (42.8%)
tracing 2.078 ms [2.024 ms, 2.132 ms] 600.432 µs (40.6%)

@bric3 bric3 added the tag: ai generated Largely based on code generated by an AI or LLM label Feb 5, 2026
@bric3
Copy link
Contributor Author

bric3 commented Feb 5, 2026

FYI I tagged it tag: ai generated, but it was more pair programming than largely generated by AI.

@bric3 bric3 changed the title Compute ran tests in a pipeline Tests summary in pipeline and improved test tags to be used in Test Optimization producrt Feb 5, 2026
@bric3 bric3 changed the title Tests summary in pipeline and improved test tags to be used in Test Optimization producrt Tests summary in pipeline and improved test tags to be used in Test Optimization product Feb 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@randomanderson randomanderson Awaiting requested review from randomanderson randomanderson is a code owner automatically assigned from DataDog/apm-release-platform

@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer

@sarahchen6 sarahchen6 Awaiting requested review from sarahchen6

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

comp: tooling Build & Tooling tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@bric3