Atomics benchmark#10862
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 12 commits intoJun 23, 2026
Merged
Conversation
Benchmark to show cost of Atomic vs AtomicFieldUpdater
Contributor
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
dougqh
added
comp: core
BenchmarksStartupParameters
See matching parameters
SummaryFound 7 performance improvements and 3 performance regressions! Performance is the same for 48 metrics, 13 unstable metrics.
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~bb784c0bb1, baseline=1.64.0-SNAPSHOT~ea827ba0c4
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1067245
Total [baseline] (10.94 s) : 0, 10939830
Agent [candidate] (1.054 s) : 0, 1053765
Total [candidate] (11.152 s) : 0, 11152352
section appsec
Agent [baseline] (1.28 s) : 0, 1279920
Total [baseline] (10.94 s) : 0, 10939843
Agent [candidate] (1.251 s) : 0, 1250716
Total [candidate] (11.194 s) : 0, 11194240
section iast
Agent [baseline] (1.251 s) : 0, 1251351
Total [baseline] (11.257 s) : 0, 11257422
Agent [candidate] (1.226 s) : 0, 1225533
Total [candidate] (11.333 s) : 0, 11332801
section profiling
Agent [baseline] (1.32 s) : 0, 1319613
Total [baseline] (10.966 s) : 0, 10965999
Agent [candidate] (1.181 s) : 0, 1180862
Total [candidate] (11.064 s) : 0, 11063763
gantt
title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~bb784c0bb1, baseline=1.64.0-SNAPSHOT~ea827ba0c4
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.209 ms) : 0, 1209
crashtracking [candidate] (1.181 ms) : 0, 1181
BytebuddyAgent [baseline] (637.988 ms) : 0, 637988
BytebuddyAgent [candidate] (629.233 ms) : 0, 629233
AgentMeter [baseline] (30.055 ms) : 0, 30055
AgentMeter [candidate] (29.447 ms) : 0, 29447
GlobalTracer [baseline] (248.335 ms) : 0, 248335
GlobalTracer [candidate] (250.984 ms) : 0, 250984
AppSec [baseline] (32.311 ms) : 0, 32311
AppSec [candidate] (31.889 ms) : 0, 31889
Debugger [baseline] (60.202 ms) : 0, 60202
Debugger [candidate] (60.598 ms) : 0, 60598
Remote Config [baseline] (597.908 µs) : 0, 598
Remote Config [candidate] (595.066 µs) : 0, 595
Telemetry [baseline] (8.426 ms) : 0, 8426
Telemetry [candidate] (8.009 ms) : 0, 8009
Flare Poller [baseline] (11.381 ms) : 0, 11381
Flare Poller [candidate] (5.873 ms) : 0, 5873
section appsec
crashtracking [baseline] (1.206 ms) : 0, 1206
crashtracking [candidate] (1.204 ms) : 0, 1204
BytebuddyAgent [baseline] (682.689 ms) : 0, 682689
BytebuddyAgent [candidate] (664.35 ms) : 0, 664350
AgentMeter [baseline] (12.192 ms) : 0, 12192
AgentMeter [candidate] (12.153 ms) : 0, 12153
GlobalTracer [baseline] (248.949 ms) : 0, 248949
GlobalTracer [candidate] (254.198 ms) : 0, 254198
AppSec [baseline] (189.721 ms) : 0, 189721
AppSec [candidate] (178.745 ms) : 0, 178745
Debugger [baseline] (65.878 ms) : 0, 65878
Debugger [candidate] (66.648 ms) : 0, 66648
Remote Config [baseline] (584.709 µs) : 0, 585
Remote Config [candidate] (648.165 µs) : 0, 648
Telemetry [baseline] (8.092 ms) : 0, 8092
Telemetry [candidate] (8.374 ms) : 0, 8374
Flare Poller [baseline] (9.389 ms) : 0, 9389
Flare Poller [candidate] (3.66 ms) : 0, 3660
IAST [baseline] (24.41 ms) : 0, 24410
IAST [candidate] (24.395 ms) : 0, 24395
section iast
crashtracking [baseline] (1.212 ms) : 0, 1212
crashtracking [candidate] (1.192 ms) : 0, 1192
BytebuddyAgent [baseline] (830.057 ms) : 0, 830057
BytebuddyAgent [candidate] (798.038 ms) : 0, 798038
AgentMeter [baseline] (11.526 ms) : 0, 11526
AgentMeter [candidate] (11.529 ms) : 0, 11529
GlobalTracer [baseline] (239.422 ms) : 0, 239422
GlobalTracer [candidate] (242.805 ms) : 0, 242805
AppSec [baseline] (28.07 ms) : 0, 28070
AppSec [candidate] (26.632 ms) : 0, 26632
Debugger [baseline] (63.755 ms) : 0, 63755
Debugger [candidate] (64.535 ms) : 0, 64535
Remote Config [baseline] (527.363 µs) : 0, 527
Remote Config [candidate] (533.733 µs) : 0, 534
Telemetry [baseline] (7.805 ms) : 0, 7805
Telemetry [candidate] (13.838 ms) : 0, 13838
Flare Poller [baseline] (3.351 ms) : 0, 3351
Flare Poller [candidate] (4.828 ms) : 0, 4828
IAST [baseline] (28.071 ms) : 0, 28071
IAST [candidate] (25.42 ms) : 0, 25420
section profiling
crashtracking [baseline] (544.001 µs) : 0, 544
crashtracking [candidate] (1.172 ms) : 0, 1172
BytebuddyAgent [baseline] (691.01 ms) : 0, 691010
BytebuddyAgent [candidate] (683.407 ms) : 0, 683407
AgentMeter [baseline] (9.61 ms) : 0, 9610
AgentMeter [candidate] (9.115 ms) : 0, 9115
GlobalTracer [baseline] (211.13 ms) : 0, 211130
GlobalTracer [candidate] (210.381 ms) : 0, 210381
AppSec [baseline] (32.623 ms) : 0, 32623
AppSec [candidate] (32.275 ms) : 0, 32275
Debugger [baseline] (66.11 ms) : 0, 66110
Debugger [candidate] (65.301 ms) : 0, 65301
Remote Config [baseline] (607.768 µs) : 0, 608
Remote Config [candidate] (569.9 µs) : 0, 570
Telemetry [baseline] (8.059 ms) : 0, 8059
Telemetry [candidate] (8.579 ms) : 0, 8579
Flare Poller [baseline] (3.555 ms) : 0, 3555
Flare Poller [candidate] (3.552 ms) : 0, 3552
ProfilingAgent [baseline] (95.47 ms) : 0, 95470
ProfilingAgent [candidate] (95.259 ms) : 0, 95259
Profiling [baseline] (96.02 ms) : 0, 96020
Profiling [candidate] (95.853 ms) : 0, 95853
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~bb784c0bb1, baseline=1.64.0-SNAPSHOT~ea827ba0c4
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.069 s) : 0, 1069455
Total [baseline] (8.875 s) : 0, 8875421
Agent [candidate] (1.052 s) : 0, 1052311
Total [candidate] (8.844 s) : 0, 8843736
section iast
Agent [baseline] (1.247 s) : 0, 1246897
Total [baseline] (9.486 s) : 0, 9485731
Agent [candidate] (1.225 s) : 0, 1224933
Total [candidate] (9.565 s) : 0, 9565263
gantt
title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~bb784c0bb1, baseline=1.64.0-SNAPSHOT~ea827ba0c4
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.204 ms) : 0, 1204
crashtracking [candidate] (1.197 ms) : 0, 1197
BytebuddyAgent [baseline] (640.489 ms) : 0, 640489
BytebuddyAgent [candidate] (629.635 ms) : 0, 629635
AgentMeter [baseline] (30.212 ms) : 0, 30212
AgentMeter [candidate] (29.37 ms) : 0, 29370
GlobalTracer [baseline] (248.351 ms) : 0, 248351
GlobalTracer [candidate] (251.508 ms) : 0, 251508
AppSec [baseline] (32.487 ms) : 0, 32487
AppSec [candidate] (31.783 ms) : 0, 31783
Debugger [baseline] (59.656 ms) : 0, 59656
Debugger [candidate] (59.888 ms) : 0, 59888
Remote Config [baseline] (609.066 µs) : 0, 609
Remote Config [candidate] (610.187 µs) : 0, 610
Telemetry [baseline] (9.186 ms) : 0, 9186
Telemetry [candidate] (8.023 ms) : 0, 8023
Flare Poller [baseline] (10.571 ms) : 0, 10571
Flare Poller [candidate] (4.258 ms) : 0, 4258
section iast
crashtracking [baseline] (1.204 ms) : 0, 1204
crashtracking [candidate] (1.188 ms) : 0, 1188
BytebuddyAgent [baseline] (827.885 ms) : 0, 827885
BytebuddyAgent [candidate] (797.102 ms) : 0, 797102
AgentMeter [baseline] (11.493 ms) : 0, 11493
AgentMeter [candidate] (11.442 ms) : 0, 11442
GlobalTracer [baseline] (240.39 ms) : 0, 240390
GlobalTracer [candidate] (243.657 ms) : 0, 243657
AppSec [baseline] (26.477 ms) : 0, 26477
AppSec [candidate] (26.795 ms) : 0, 26795
Debugger [baseline] (62.965 ms) : 0, 62965
Debugger [candidate] (63.727 ms) : 0, 63727
Remote Config [baseline] (524.954 µs) : 0, 525
Remote Config [candidate] (536.79 µs) : 0, 537
Telemetry [baseline] (7.749 ms) : 0, 7749
Telemetry [candidate] (13.681 ms) : 0, 13681
Flare Poller [baseline] (3.271 ms) : 0, 3271
Flare Poller [candidate] (5.073 ms) : 0, 5073
IAST [baseline] (28.34 ms) : 0, 28340
IAST [candidate] (25.514 ms) : 0, 25514
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 2 performance regressions! Performance is the same for 17 metrics, 16 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~bb784c0bb1, baseline=1.64.0-SNAPSHOT~ea827ba0c4
dateFormat X
axisFormat %s
section baseline
no_agent (1.26 ms) : 1248, 1271
. : milestone, 1260,
iast (3.343 ms) : 3294, 3391
. : milestone, 3343,
iast_FULL (6.021 ms) : 5961, 6082
. : milestone, 6021,
iast_GLOBAL (3.709 ms) : 3648, 3769
. : milestone, 3709,
profiling (2.485 ms) : 2460, 2509
. : milestone, 2485,
tracing (1.901 ms) : 1885, 1916
. : milestone, 1901,
section candidate
no_agent (1.286 ms) : 1273, 1299
. : milestone, 1286,
iast (3.313 ms) : 3269, 3357
. : milestone, 3313,
iast_FULL (6.038 ms) : 5976, 6099
. : milestone, 6038,
iast_GLOBAL (3.675 ms) : 3612, 3738
. : milestone, 3675,
profiling (2.238 ms) : 2215, 2261
. : milestone, 2238,
tracing (1.97 ms) : 1953, 1987
. : milestone, 1970,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~bb784c0bb1, baseline=1.64.0-SNAPSHOT~ea827ba0c4
dateFormat X
axisFormat %s
section baseline
no_agent (19.156 ms) : 18961, 19352
. : milestone, 19156,
appsec (18.716 ms) : 18525, 18907
. : milestone, 18716,
code_origins (18.695 ms) : 18509, 18880
. : milestone, 18695,
iast (18.715 ms) : 18527, 18904
. : milestone, 18715,
profiling (18.169 ms) : 17990, 18348
. : milestone, 18169,
tracing (18.227 ms) : 18045, 18409
. : milestone, 18227,
section candidate
no_agent (18.442 ms) : 18251, 18634
. : milestone, 18442,
appsec (18.416 ms) : 18231, 18600
. : milestone, 18416,
code_origins (17.846 ms) : 17669, 18023
. : milestone, 17846,
iast (18.011 ms) : 17837, 18184
. : milestone, 18011,
profiling (18.924 ms) : 18732, 19116
. : milestone, 18924,
tracing (18.011 ms) : 17834, 18188
. : milestone, 18011,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~bb784c0bb1, baseline=1.64.0-SNAPSHOT~ea827ba0c4
dateFormat X
axisFormat %s
section baseline
no_agent (1.488 ms) : 1476, 1499
. : milestone, 1488,
appsec (3.83 ms) : 3606, 4053
. : milestone, 3830,
iast (2.296 ms) : 2226, 2366
. : milestone, 2296,
iast_GLOBAL (2.334 ms) : 2263, 2405
. : milestone, 2334,
profiling (2.553 ms) : 2388, 2718
. : milestone, 2553,
tracing (2.11 ms) : 2055, 2165
. : milestone, 2110,
section candidate
no_agent (1.49 ms) : 1478, 1502
. : milestone, 1490,
appsec (3.824 ms) : 3603, 4045
. : milestone, 3824,
iast (2.274 ms) : 2205, 2343
. : milestone, 2274,
iast_GLOBAL (2.313 ms) : 2243, 2383
. : milestone, 2313,
profiling (2.122 ms) : 2066, 2179
. : milestone, 2122,
tracing (2.08 ms) : 2026, 2133
. : milestone, 2080,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~bb784c0bb1, baseline=1.64.0-SNAPSHOT~ea827ba0c4
dateFormat X
axisFormat %s
section baseline
no_agent (15.223 s) : 15223000, 15223000
. : milestone, 15223000,
appsec (14.851 s) : 14851000, 14851000
. : milestone, 14851000,
iast (18.626 s) : 18626000, 18626000
. : milestone, 18626000,
iast_GLOBAL (17.781 s) : 17781000, 17781000
. : milestone, 17781000,
profiling (15.425 s) : 15425000, 15425000
. : milestone, 15425000,
tracing (14.865 s) : 14865000, 14865000
. : milestone, 14865000,
section candidate
no_agent (15.437 s) : 15437000, 15437000
. : milestone, 15437000,
appsec (14.593 s) : 14593000, 14593000
. : milestone, 14593000,
iast (18.162 s) : 18162000, 18162000
. : milestone, 18162000,
iast_GLOBAL (17.731 s) : 17731000, 17731000
. : milestone, 17731000,
profiling (15.001 s) : 15001000, 15001000
. : milestone, 15001000,
tracing (14.801 s) : 14801000, 14801000
. : milestone, 14801000,
|
bric3
reviewed
Apr 8, 2026
| @Warmup(iterations = 2) | ||
| @Measurement(iterations = 3) | ||
| @Threads(8) | ||
| public class AtomicsBenchmark { |
Contributor
There was a problem hiding this comment.
question: Would it make sense to include JCTools equivalents?
Contributor
Author
There was a problem hiding this comment.
I'm not very familiar with jctools, but probably, the idea is for these to serve as kind of a "menu" to compare different equivalent mechanisms.
sarahchen6
reviewed
Jun 22, 2026
…java Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com>
Contributor
Author
|
I'm going to re-run once more to get fresh numbers after a methodology fix, then I'll merge it. |
This comment has been minimized.
This comment has been minimized.
sharedLookupIndex was a plain static int incremented by all 8 JMH threads without synchronization — a data race that could cause contention measurements to reflect arbitrary holder access patterns. Move index to @State(Scope.Thread) so each thread has its own cursor. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
dd-octo-sts
Bot
added
the
tag: ai generated
Without this, -Pjmh.includes is silently ignored by the me.champeau.jmh plugin, requiring a full fat-jar build to run a single benchmark. -PtestJvm was also ignored for JMH execution, defaulting to the Gradle daemon JVM regardless of the requested version. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Re-run after fixing the shared-index data race. The incrementAndGet results increased dramatically (~16M → ~91-121M ops/s) because the prior run was measuring contention on the shared index rather than the atomics themselves. Notable: atomic_incrementAndGet (121M) now outperforms atomicFieldUpdater_incrementAndGet (91M) for the increment op; the AFU advantage is in construction (2215M vs 1872M, saving 16 B/op). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…attern AFU wins on construction (16 B vs 32 B/op); AtomicInteger wins on incrementAndGet (~33% faster). Updated Javadoc to present both sides rather than a blanket recommendation. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
VarHandle (Java 9+) is the modern replacement for AtomicIntegerFieldUpdater and would likely close the incrementAndGet performance gap. Not usable here because internal-api targets Java 8. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bric3
approved these changes
Jun 23, 2026
| * the incrementAndGet gap with AtomicInteger while retaining the construction allocation advantage. | ||
| * Not available here because internal-api targets Java 8. | ||
| * | ||
| * <code> Java 17 - MacBook M1 - 8 threads |
Contributor
There was a problem hiding this comment.
quibble:
Suggested change
| * <code> Java 17 - MacBook M1 - 8 threads | |
| * <code> Java 17 - MacBook M1 Pro Max - 8 threads |
Contributor
There was a problem hiding this comment.
Also, did you notice changes across JVMs ?
Contributor
Author
There was a problem hiding this comment.
I haven't checked extensively. The main thing is that avoid using Java 8 locally.
Java 8 was the first JVM to support ARM64 and the code gen was really quite poor. It improved significantly over the next couple major releases.
| * | ||
| * While the performance of Atomic is on par with AtomicFieldUpdater (and sometimes slightly better) inside a frequently | ||
| * constructed object, the impact of the extra allocation on garbage collection is detrimental to the application as a whole. | ||
| * <p><b>Future:</b> {@code VarHandle} (Java 9+) is the modern replacement for |
Contributor
There was a problem hiding this comment.
note: As far as I know VarHandle are not as good until Java 25 or 26.
Contributor
Author
There was a problem hiding this comment.
Yeah, I haven't extensively tested it yet. I just decided to add a note for completeness.
dougqh
commented
Jun 23, 2026
Contributor
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
The internal-api/build.gradle.kts -Pjmh.includes / -PtestJvm wiring belongs in its dedicated PR (#11703), not here. Revert it so this PR is just the benchmark. Also correct the results header to "MacBook M1 Pro Max". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Benchmark to show cost of Atomic vs AtomicFieldUpdater
What Does This Do
Benchmark to show cost of Atomic vs AtomicFieldUpdater
Overall conclusion is to favor AtomicFieldUpdater when possible.
While Atomic is sometimes slightly better in ops/sec, AtomicFieldUpdater reduces memory footprint which is typically more important for the system as a whole
Motivation
Provide and illustrate recommendations to follow when contributing to the Java agent
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.