Fix flaky test exception in HttpServerTest#11164
Conversation
DD_C0320J Co-authored-by: janhavis <janhavi.sahasrabuddhe@gmail.com>
|
I can only run on private repositories. |
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
PerfectSlayer
left a comment
There was a problem hiding this comment.
❔ question: Are there some other methods that need would response closing?
That could be a good opportunity to fix it once for all because I don't think it's the first time we come up with this kind of fix.
Potentially. I will take a look. |
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 62 metrics, 9 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~5a02e8b8f4, baseline=1.62.0-SNAPSHOT~6e28457d70
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1057561
Total [baseline] (11.055 s) : 0, 11055036
Agent [candidate] (1.058 s) : 0, 1058405
Total [candidate] (11.073 s) : 0, 11072875
section appsec
Agent [baseline] (1.261 s) : 0, 1261460
Total [baseline] (10.995 s) : 0, 10994699
Agent [candidate] (1.268 s) : 0, 1268374
Total [candidate] (10.961 s) : 0, 10960590
section iast
Agent [baseline] (1.231 s) : 0, 1230939
Total [baseline] (11.235 s) : 0, 11235466
Agent [candidate] (1.232 s) : 0, 1232475
Total [candidate] (11.284 s) : 0, 11283653
section profiling
Agent [baseline] (1.193 s) : 0, 1193482
Total [baseline] (11.045 s) : 0, 11045203
Agent [candidate] (1.192 s) : 0, 1192447
Total [candidate] (11.024 s) : 0, 11024408
gantt
title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~5a02e8b8f4, baseline=1.62.0-SNAPSHOT~6e28457d70
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.228 ms) : 0, 1228
crashtracking [candidate] (1.217 ms) : 0, 1217
BytebuddyAgent [baseline] (632.568 ms) : 0, 632568
BytebuddyAgent [candidate] (633.501 ms) : 0, 633501
AgentMeter [baseline] (29.51 ms) : 0, 29510
AgentMeter [candidate] (29.526 ms) : 0, 29526
GlobalTracer [baseline] (248.445 ms) : 0, 248445
GlobalTracer [candidate] (248.308 ms) : 0, 248308
AppSec [baseline] (32.322 ms) : 0, 32322
AppSec [candidate] (32.331 ms) : 0, 32331
Debugger [baseline] (59.697 ms) : 0, 59697
Debugger [candidate] (59.725 ms) : 0, 59725
Remote Config [baseline] (600.092 µs) : 0, 600
Remote Config [candidate] (600.615 µs) : 0, 601
Telemetry [baseline] (8.004 ms) : 0, 8004
Telemetry [candidate] (7.997 ms) : 0, 7997
Flare Poller [baseline] (8.96 ms) : 0, 8960
Flare Poller [candidate] (9.099 ms) : 0, 9099
section appsec
crashtracking [baseline] (1.225 ms) : 0, 1225
crashtracking [candidate] (1.222 ms) : 0, 1222
BytebuddyAgent [baseline] (674.567 ms) : 0, 674567
BytebuddyAgent [candidate] (677.983 ms) : 0, 677983
AgentMeter [baseline] (12.154 ms) : 0, 12154
AgentMeter [candidate] (12.202 ms) : 0, 12202
GlobalTracer [baseline] (248.921 ms) : 0, 248921
GlobalTracer [candidate] (249.852 ms) : 0, 249852
IAST [baseline] (24.363 ms) : 0, 24363
IAST [candidate] (24.498 ms) : 0, 24498
AppSec [baseline] (186.148 ms) : 0, 186148
AppSec [candidate] (187.743 ms) : 0, 187743
Debugger [baseline] (65.75 ms) : 0, 65750
Debugger [candidate] (66.426 ms) : 0, 66426
Remote Config [baseline] (589.581 µs) : 0, 590
Remote Config [candidate] (578.488 µs) : 0, 578
Telemetry [baseline] (7.886 ms) : 0, 7886
Telemetry [candidate] (7.983 ms) : 0, 7983
Flare Poller [baseline] (3.479 ms) : 0, 3479
Flare Poller [candidate] (3.537 ms) : 0, 3537
section iast
crashtracking [baseline] (1.236 ms) : 0, 1236
crashtracking [candidate] (1.228 ms) : 0, 1228
BytebuddyAgent [baseline] (808.627 ms) : 0, 808627
BytebuddyAgent [candidate] (810.404 ms) : 0, 810404
AgentMeter [baseline] (11.389 ms) : 0, 11389
AgentMeter [candidate] (11.439 ms) : 0, 11439
GlobalTracer [baseline] (238.888 ms) : 0, 238888
GlobalTracer [candidate] (239.037 ms) : 0, 239037
IAST [baseline] (29.132 ms) : 0, 29132
IAST [candidate] (31.574 ms) : 0, 31574
AppSec [baseline] (29.302 ms) : 0, 29302
AppSec [candidate] (26.518 ms) : 0, 26518
Debugger [baseline] (63.719 ms) : 0, 63719
Debugger [candidate] (64.585 ms) : 0, 64585
Remote Config [baseline] (532.724 µs) : 0, 533
Remote Config [candidate] (529.569 µs) : 0, 530
Telemetry [baseline] (7.73 ms) : 0, 7730
Telemetry [candidate] (7.726 ms) : 0, 7726
Flare Poller [baseline] (3.423 ms) : 0, 3423
Flare Poller [candidate] (3.391 ms) : 0, 3391
section profiling
crashtracking [baseline] (1.199 ms) : 0, 1199
crashtracking [candidate] (1.192 ms) : 0, 1192
BytebuddyAgent [baseline] (697.409 ms) : 0, 697409
BytebuddyAgent [candidate] (696.717 ms) : 0, 696717
AgentMeter [baseline] (9.278 ms) : 0, 9278
AgentMeter [candidate] (9.275 ms) : 0, 9275
GlobalTracer [baseline] (208.333 ms) : 0, 208333
GlobalTracer [candidate] (208.442 ms) : 0, 208442
AppSec [baseline] (33.16 ms) : 0, 33160
AppSec [candidate] (33.123 ms) : 0, 33123
Debugger [baseline] (66.162 ms) : 0, 66162
Debugger [candidate] (66.14 ms) : 0, 66140
Remote Config [baseline] (596.834 µs) : 0, 597
Remote Config [candidate] (596.701 µs) : 0, 597
Telemetry [baseline] (7.795 ms) : 0, 7795
Telemetry [candidate] (7.782 ms) : 0, 7782
Flare Poller [baseline] (3.528 ms) : 0, 3528
Flare Poller [candidate] (3.511 ms) : 0, 3511
ProfilingAgent [baseline] (94.254 ms) : 0, 94254
ProfilingAgent [candidate] (94.03 ms) : 0, 94030
Profiling [baseline] (94.822 ms) : 0, 94822
Profiling [candidate] (94.588 ms) : 0, 94588
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~5a02e8b8f4, baseline=1.62.0-SNAPSHOT~6e28457d70
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.054 s) : 0, 1054267
Total [baseline] (8.816 s) : 0, 8815775
Agent [candidate] (1.062 s) : 0, 1062103
Total [candidate] (8.889 s) : 0, 8888946
section iast
Agent [baseline] (1.229 s) : 0, 1229439
Total [baseline] (9.589 s) : 0, 9589038
Agent [candidate] (1.245 s) : 0, 1244804
Total [candidate] (9.564 s) : 0, 9564036
gantt
title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~5a02e8b8f4, baseline=1.62.0-SNAPSHOT~6e28457d70
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.23 ms) : 0, 1230
crashtracking [candidate] (1.229 ms) : 0, 1229
BytebuddyAgent [baseline] (633.079 ms) : 0, 633079
BytebuddyAgent [candidate] (637.925 ms) : 0, 637925
AgentMeter [baseline] (29.531 ms) : 0, 29531
AgentMeter [candidate] (29.485 ms) : 0, 29485
GlobalTracer [baseline] (248.525 ms) : 0, 248525
GlobalTracer [candidate] (248.793 ms) : 0, 248793
AppSec [baseline] (32.45 ms) : 0, 32450
AppSec [candidate] (32.387 ms) : 0, 32387
Debugger [baseline] (58.929 ms) : 0, 58929
Debugger [candidate] (59.124 ms) : 0, 59124
Remote Config [baseline] (593.449 µs) : 0, 593
Remote Config [candidate] (587.761 µs) : 0, 588
Telemetry [baseline] (8.737 ms) : 0, 8737
Telemetry [candidate] (8.019 ms) : 0, 8019
Flare Poller [baseline] (5.001 ms) : 0, 5001
Flare Poller [candidate] (8.283 ms) : 0, 8283
section iast
crashtracking [baseline] (1.226 ms) : 0, 1226
crashtracking [candidate] (1.231 ms) : 0, 1231
BytebuddyAgent [baseline] (808.181 ms) : 0, 808181
BytebuddyAgent [candidate] (818.608 ms) : 0, 818608
AgentMeter [baseline] (11.405 ms) : 0, 11405
AgentMeter [candidate] (11.599 ms) : 0, 11599
GlobalTracer [baseline] (238.942 ms) : 0, 238942
GlobalTracer [candidate] (241.649 ms) : 0, 241649
IAST [baseline] (30.915 ms) : 0, 30915
IAST [candidate] (28.567 ms) : 0, 28567
AppSec [baseline] (27.474 ms) : 0, 27474
AppSec [candidate] (29.753 ms) : 0, 29753
Debugger [baseline] (62.693 ms) : 0, 62693
Debugger [candidate] (64.499 ms) : 0, 64499
Remote Config [baseline] (528.049 µs) : 0, 528
Remote Config [candidate] (545.318 µs) : 0, 545
Telemetry [baseline] (7.778 ms) : 0, 7778
Telemetry [candidate] (7.848 ms) : 0, 7848
Flare Poller [baseline] (3.431 ms) : 0, 3431
Flare Poller [candidate] (3.482 ms) : 0, 3482
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 1 performance regressions! Performance is the same for 18 metrics, 16 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~5a02e8b8f4, baseline=1.62.0-SNAPSHOT~6e28457d70
dateFormat X
axisFormat %s
section baseline
no_agent (18.452 ms) : 18261, 18643
. : milestone, 18452,
appsec (18.675 ms) : 18485, 18866
. : milestone, 18675,
code_origins (18.069 ms) : 17890, 18247
. : milestone, 18069,
iast (18.094 ms) : 17915, 18274
. : milestone, 18094,
profiling (18.141 ms) : 17962, 18320
. : milestone, 18141,
tracing (17.966 ms) : 17787, 18145
. : milestone, 17966,
section candidate
no_agent (19.257 ms) : 19062, 19452
. : milestone, 19257,
appsec (18.884 ms) : 18690, 19079
. : milestone, 18884,
code_origins (18.158 ms) : 17980, 18336
. : milestone, 18158,
iast (17.882 ms) : 17704, 18059
. : milestone, 17882,
profiling (17.925 ms) : 17747, 18104
. : milestone, 17925,
tracing (17.968 ms) : 17790, 18146
. : milestone, 17968,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~5a02e8b8f4, baseline=1.62.0-SNAPSHOT~6e28457d70
dateFormat X
axisFormat %s
section baseline
no_agent (1.227 ms) : 1215, 1238
. : milestone, 1227,
iast (3.271 ms) : 3226, 3316
. : milestone, 3271,
iast_FULL (6.172 ms) : 6110, 6235
. : milestone, 6172,
iast_GLOBAL (3.723 ms) : 3664, 3783
. : milestone, 3723,
profiling (2.422 ms) : 2398, 2447
. : milestone, 2422,
tracing (1.889 ms) : 1873, 1905
. : milestone, 1889,
section candidate
no_agent (1.243 ms) : 1231, 1255
. : milestone, 1243,
iast (3.398 ms) : 3349, 3448
. : milestone, 3398,
iast_FULL (6.062 ms) : 6001, 6123
. : milestone, 6062,
iast_GLOBAL (3.68 ms) : 3629, 3732
. : milestone, 3680,
profiling (2.061 ms) : 2043, 2079
. : milestone, 2061,
tracing (1.915 ms) : 1898, 1931
. : milestone, 1915,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~5a02e8b8f4, baseline=1.62.0-SNAPSHOT~6e28457d70
dateFormat X
axisFormat %s
section baseline
no_agent (1.485 ms) : 1473, 1496
. : milestone, 1485,
appsec (3.868 ms) : 3646, 4089
. : milestone, 3868,
iast (2.287 ms) : 2217, 2357
. : milestone, 2287,
iast_GLOBAL (2.318 ms) : 2249, 2388
. : milestone, 2318,
profiling (2.114 ms) : 2059, 2169
. : milestone, 2114,
tracing (2.085 ms) : 2032, 2139
. : milestone, 2085,
section candidate
no_agent (1.489 ms) : 1477, 1500
. : milestone, 1489,
appsec (3.857 ms) : 3631, 4083
. : milestone, 3857,
iast (2.28 ms) : 2211, 2350
. : milestone, 2280,
iast_GLOBAL (2.325 ms) : 2254, 2395
. : milestone, 2325,
profiling (2.506 ms) : 2287, 2725
. : milestone, 2506,
tracing (2.093 ms) : 2039, 2146
. : milestone, 2093,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~5a02e8b8f4, baseline=1.62.0-SNAPSHOT~6e28457d70
dateFormat X
axisFormat %s
section baseline
no_agent (15.051 s) : 15051000, 15051000
. : milestone, 15051000,
appsec (14.966 s) : 14966000, 14966000
. : milestone, 14966000,
iast (18.39 s) : 18390000, 18390000
. : milestone, 18390000,
iast_GLOBAL (17.827 s) : 17827000, 17827000
. : milestone, 17827000,
profiling (14.947 s) : 14947000, 14947000
. : milestone, 14947000,
tracing (14.907 s) : 14907000, 14907000
. : milestone, 14907000,
section candidate
no_agent (15.079 s) : 15079000, 15079000
. : milestone, 15079000,
appsec (14.562 s) : 14562000, 14562000
. : milestone, 14562000,
iast (18.509 s) : 18509000, 18509000
. : milestone, 18509000,
iast_GLOBAL (18.1 s) : 18100000, 18100000
. : milestone, 18100000,
profiling (14.907 s) : 14907000, 14907000
. : milestone, 14907000,
tracing (14.8 s) : 14800000, 14800000
. : milestone, 14800000,
|
@PerfectSlayer Pushed a change to close the response body in 3 more tests that don't consume the body stream, and won't implicitly close. |
| } | ||
| } | ||
|
|
||
| @Flaky(value = "https://github.com/DataDog/dd-trace-java/issues/9396", suites = ["PekkoHttpServerInstrumentationAsyncHttp2Test"]) |
There was a problem hiding this comment.
You may want to consider keeping this for now
This PR hasn't deterministically proved that this flake is fixed so there may be other underlying issues here that may recur after this fix. If that does happen, this test may break a few PRs and we would need to re-introduce the annotation
That being said, if we can see that this fixed it empirically (i.e. this test does not flake for 30 days) we can remove this annotation afterwards with no worries
There was a problem hiding this comment.
That sounds fair. Will add it back in.
…flow control issues Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
4385040 to
5a02e8b
Compare
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
The merge request has been interrupted because the build 0 took longer than expected. The current limit for the base branch 'master' is 120 minutes. |
|
/merge |
|
View all feedbacks in Devflow UI.
PR already in the queue with status in_progress |
What Does This Do
Fixing
test exception• View in Test Optimization • Questions? Ask in #code-gen-flaky-testsThis PR fixes a flaky test
test exceptionin Pekko HTTP instrumentation tests by ensuring that the HTTP response is always closed in the baseHttpServerTestclass.Motivation
The test
test exceptionwas intermittently failing with aTimeoutExceptionwaiting for traces, particularly in HTTP/2 scenarios. This is likely caused by the response body not being consumed whentestExceptionBody()is false, which can prevent the server from finishing the request/response cycle and thus prevent the server span from being closed. In HTTP/2, unconsumed streams can lead to such issues due to flow control.Additional Notes
response.close()totest exception,test error, andtest notFoundinHttpServerTest.groovyfor cases where the body is not consumed.@Flakyannotation fortest exceptioninHttpServerTest.groovyas this fix should resolve the underlying issue.Contributor Checklist
type:bugandinst:pekko-httplabelsPR by Bits - View session in Datadog
Comment @DataDog to request changes