Skip to content

Removing thread contention from Unsynchronized Map Benchmarks#11679

Open
dougqh wants to merge 7 commits into
masterfrom
dougqh/tagmap-access-benchmark
Open

Removing thread contention from Unsynchronized Map Benchmarks#11679
dougqh wants to merge 7 commits into
masterfrom
dougqh/tagmap-access-benchmark

Conversation

@dougqh

@dougqh dougqh commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

What This Does

Adds a throughput microbenchmark for the core TagMap access paths — insert, getObject (raw value), getEntry — over a representative HTTP-server-ish tag set.

Motivation

Before making further changes to TagMap, I want a more robust benchmark in place. The shared counter used in the benchmark previously was masking some of the differences as compared to other Maps.

Additional Notes

Runs at @Threads(8). All shared state is immutable (NAMES/VALUES); every bit of mutable state (the pre-populated read map) lives in a @State(Scope.Thread) holder, so threads never contend on a shared map, index, or reader flyweight. Earlier TagMap benchmarks shared a cross-thread counter/index, which turned the result into a contention measurement rather than a TagMap measurement — this layout avoids that. Indices are plain per-invocation locals.

Run config is baked into annotations (@Fork/@Warmup/@Measurement/@Threads) rather than relying on -Pjmh.* flags, which the me.champeau.jmh plugin ignores.

Notes

  • Independent of the LegacyTagMap removal (Remove LegacyTagMap; OptimizedTagMap is the sole TagMap implementation #11678) — uses only the public TagMap API, so it builds/runs on master and on that branch alike.
  • First of a planned per-span-type benchmark suite (web/db/queue/…), which gives instrumentation authors relatable numbers and is the natural vehicle for demonstrating per-type optimizations later. The per-type DDSpan benchmark follows once its tracer setup is quieted (a no-op writer to stop per-op DEBUG logging from contaminating throughput).

🤖 Generated with Claude Code

Throughput microbenchmark for TagMap insert/getObject/getEntry over a
representative HTTP-server tag set. All mutable state (the read map) lives
in @State(Scope.Thread) so @threads(8) runs measure TagMap rather than
cross-thread contention on a shared map/counter/flyweight — the flaw that
made earlier TagMap benchmarks misleading. Run config is baked into
annotations (the me.champeau.jmh plugin ignores -Pjmh.* flags).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dougqh dougqh added type: enhancement Enhancements and improvements comp: core Tracer core tag: no release notes Changes to exclude from release notes tag: ai generated Largely based on code generated by an AI or LLM labels Jun 18, 2026
@datadog-datadog-prod-us1-2

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.99 s 13.96 s [-0.5%; +0.9%] (no difference)
startup:insecure-bank:tracing:Agent 12.94 s 12.92 s [-0.6%; +0.8%] (no difference)
startup:petclinic:appsec:Agent 16.94 s 16.37 s [-1.0%; +7.8%] (no difference)
startup:petclinic:iast:Agent 16.93 s 16.87 s [-0.4%; +1.2%] (no difference)
startup:petclinic:profiling:Agent 16.84 s 17.04 s [-2.3%; -0.1%] (maybe better)
startup:petclinic:sca:Agent 16.93 s 16.89 s [-0.8%; +1.2%] (no difference)
startup:petclinic:tracing:Agent 16.19 s 15.99 s [-0.0%; +2.5%] (no difference)

Commit: cd883e1a · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@dougqh dougqh marked this pull request as ready for review June 22, 2026 19:27
@dougqh dougqh requested a review from a team as a code owner June 22, 2026 19:27
@dougqh dougqh requested a review from ygree June 22, 2026 19:27
public class TagMapAccessBenchmark {
// a representative HTTP-server-ish tag set (immutable -> safe to share across threads)
static final String[] NAMES = {
"http.request.method",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Future intended changes will care about the specifics of the tags, so using real tags is preferable for future-proofing

dougqh and others added 6 commits June 22, 2026 16:58
sharedLookupIndex was a plain static int incremented by all 8 JMH
threads without synchronization — a data race that turned the get
benchmarks into a contention measurement rather than a map measurement.
Move the index to @State(Scope.Thread) so each thread has its own
cursor, matching the approach used in TagMapAccessBenchmark.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Without this, -Pjmh.includes is silently ignored by the me.champeau.jmh
plugin, requiring a full fat-jar build to run a single benchmark.
-PtestJvm was also ignored for JMH execution, defaulting to the Gradle
daemon JVM regardless of the requested version.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Re-run after fixing the shared-index data race, on Java 17 with
correct per-thread scaffolding state.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The '2x faster construction' claim was stale — Java 17 numbers show
~40%. Also clarifies that LinkedHashMap's cost is purely at construction;
gets and iteration are equivalent to HashMap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…sBenchmark

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@dougqh dougqh changed the title Add a threading-correct TagMap access microbenchmark Removing thread contention from Unsynchronized Map Benchmarks Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: enhancement Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant