Add agentless and standalone OpenFeature smoke tests by PerfectSlayer · Pull Request #12846 · DataDog/dd-trace-java

PerfectSlayer · 2026-10-13T07:18:15Z

What Does This Do

Extracts the shared OpenFeature smoke test scenarios into AbstractOpenFeatureSmokeTest and runs them in three setups: SDK only (standalone), SDK + agent in agentless mode, and SDK + agent with Remote Configuration. Updates ARCHITECTURE.md, and adds the dd-openfeature SDK README: setup, configuration, features by setup, migration from earlier versions and design notes for contributors.

Motivation

Covers each supported setup of the standalone SDK (APMLP-1512).

Stack

The full change is in #12709. It was split into this stack to ease the review.

Jira ticket: APMLP-1512

🤖 Generated with Claude Code

dd-octo-sts · 2026-10-13T07:45:31Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.00 s	13.91 s	[-0.2%; +1.5%] (no difference)
startup:insecure-bank:tracing:Agent	12.89 s	12.99 s	[-1.4%; -0.1%] (maybe better)
startup:petclinic:appsec:Agent	17.10 s	17.02 s	[-0.1%; +1.1%] (no difference)
startup:petclinic:iast:Agent	16.90 s	17.04 s	[-1.5%; -0.1%] (maybe better)
startup:petclinic:profiling:Agent	16.71 s	16.80 s	[-1.4%; +0.4%] (no difference)
startup:petclinic:sca:Agent	17.11 s	17.01 s	[-0.3%; +1.6%] (no difference)
startup:petclinic:tracing:Agent	16.19 s	16.11 s	[-0.4%; +1.4%] (no difference)

Commit: 14744249 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

chatgpt-codex-connector · 2026-10-27T10:52:01Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-27T10:58:31.610214Z	`0cc336d`	Draft marked ready
🔒 Security Review	✅ Completed 2026-10-27T10:55:27.035196Z	`0cc336d`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0cc336db05

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

datadog-official

The standalone smoke suite fails before its application starts: its launcher omits the scope-diagnostics placeholder without supplying the required opt-out. Consequently, none of its evaluation scenarios can run.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 0cc336d}

sarahchen6

Looks good!

datadog-official

More details

Missing OpenTelemetry dependencies in the offline cache prevented targeted tests from reaching the OpenFeature scenarios, so runtime behavior remains unverified.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 018613e}

Extract the shared OpenFeature smoke test scenarios and run them against the SDK alone, the SDK with the agent in agentless mode, and the SDK with the agent through Remote Configuration. Document the new SDK layout in ARCHITECTURE.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Describe the SDK setup, its configuration, the features by setup, the migration from earlier versions and the design notes for contributors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The agent bridge ships with dd-java-agent 1.69.0, and the providers released before 1.69.0 return the caller defaults with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

(cherry picked from commit 76d1a62)

datadog-official

More details

Confidence is limited to static analysis of configuration, event delivery, and shutdown handling; the smoke tests were not executed.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 2e078e9}

PerfectSlayer added type: feature Enhancements and improvements tag: no release notes Changes to exclude from release notes tag: ai generated Largely based on code generated by an AI or LLM comp: openfeature OpenFeature labels Oct 13, 2026

PerfectSlayer added this pull request to stack #12847 October 13, 2026 07:19

This comment has been minimized.

Sign in to view

PerfectSlayer added the tag: override groovy enforcement Override the "Enforce Groovy Migration" check label Oct 13, 2026

PerfectSlayer force-pushed the bbujon/ffe-extras branch from 5e778ce to ed48374 Compare October 13, 2026 09:04

PerfectSlayer force-pushed the bbujon/ffe-extras branch from ed48374 to 68623d7 Compare October 13, 2026 09:28

PerfectSlayer force-pushed the bbujon/ffe-extras branch from 68623d7 to a264ffd Compare October 13, 2026 10:08

PerfectSlayer force-pushed the bbujon/ffe-extras branch from a264ffd to 117a212 Compare October 13, 2026 10:29

PerfectSlayer force-pushed the bbujon/ffe-extras branch from 117a212 to 7289f2d Compare October 13, 2026 10:59

PerfectSlayer force-pushed the bbujon/ffe-extras branch from 7289f2d to 4db3136 Compare October 13, 2026 13:51

PerfectSlayer force-pushed the bbujon/ffe-extras branch from 4db3136 to caabcf2 Compare October 14, 2026 05:35

PerfectSlayer force-pushed the bbujon/ffe-extras branch from caabcf2 to 8738bc6 Compare October 14, 2026 06:04

PerfectSlayer added this pull request to stack #12958 October 20, 2026 11:31

PerfectSlayer force-pushed the bbujon/ffe-extras branch from 239cf99 to 841c43f Compare October 20, 2026 11:36

PerfectSlayer force-pushed the bbujon/ffe-extras branch 2 times, most recently from dc45c3e to 0aef5b5 Compare October 20, 2026 12:40

sarahchen6 force-pushed the bbujon/ffe-extras branch from 0aef5b5 to 8223463 Compare October 20, 2026 23:20

sarahchen6 force-pushed the bbujon/ffe-extras branch 3 times, most recently from 442e327 to 17833c4 Compare October 21, 2026 02:16

PerfectSlayer force-pushed the bbujon/ffe-extras branch 2 times, most recently from 5f96f72 to 716f7a1 Compare October 21, 2026 09:13

chatgpt-codex-connector Bot reviewed Oct 27, 2026

View reviewed changes

Comment thread ...enfeature/src/test/groovy/datadog/smoketest/springboot/OpenFeatureStandaloneSmokeTest.groovy Resolved

Comment thread dd-openfeature/README.md Resolved

datadog-official Bot reviewed Oct 27, 2026

View reviewed changes

Comment thread ...enfeature/src/test/groovy/datadog/smoketest/springboot/OpenFeatureStandaloneSmokeTest.groovy Resolved

sarahchen6 approved these changes Oct 27, 2026

View reviewed changes

datadog-official Bot reviewed Oct 28, 2026

View reviewed changes

PerfectSlayer mentioned this pull request Oct 28, 2026

Port the OpenFeature smoke tests to JUnit #13106

Closed

PerfectSlayer and others added 5 commits October 29, 2026 11:29

docs(ffe): Document the dd-openfeature SDK

4e68600

Describe the SDK setup, its configuration, the features by setup, the migration from earlier versions and the design notes for contributors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs(ffe): Require dd-java-agent 1.69.0 in the SDK README

21f6f3c

The agent bridge ships with dd-java-agent 1.69.0, and the providers released before 1.69.0 return the caller defaults with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

test(ffe): Skip agent diagnostics in standalone smoke tests

5ba5680

docs(ffe): Fix SDK setup and credential requirements

1474424

(cherry picked from commit 76d1a62)

datadog-official Bot reviewed Oct 29, 2026

View reviewed changes

Navigation Menu

Repository navigation

Add agentless and standalone OpenFeature smoke tests - #12846

Add agentless and standalone OpenFeature smoke tests#12846
gh-worker-dd-mergequeue-cf854d[bot] merged 5 commits into
masterfrom
bbujon/ffe-extras

PerfectSlayer commented Oct 13, 2026 •

edited

Loading

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Oct 13, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot commented Oct 27, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

datadog-official Bot left a comment

Uh oh!

Uh oh!

sarahchen6 left a comment

Uh oh!

datadog-official Bot left a comment

Uh oh!

datadog-official Bot left a comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

PerfectSlayer commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Stack

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

chatgpt-codex-connector Bot commented Oct 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

datadog-official Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sarahchen6 left a comment

Choose a reason for hiding this comment

Uh oh!

datadog-official Bot left a comment

Choose a reason for hiding this comment

Uh oh!

datadog-official Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

PerfectSlayer commented Oct 13, 2026 •

edited

Loading

dd-octo-sts Bot commented Oct 13, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Oct 27, 2026 •

edited

Loading