Skip to content

feat(remote-config)!: agentless RC fetcher - #2112

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 84 commits into
mainfrom
paullgdc/remote_config/agentless_fetcher
Aug 13, 2026
Merged

feat(remote-config)!: agentless RC fetcher#2112
gh-worker-dd-mergequeue-cf854d[bot] merged 84 commits into
mainfrom
paullgdc/remote_config/agentless_fetcher

Conversation

@paullegranddc

@paullegranddc paullegranddc commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Motivation

In some environments (serverless, Datadog studio), tracers can't rely on a Datadog Agent to proxy Remote Config requests. This PR adds an agentless mode to libdd-remote-config so the fetcher can talk directly to the RC backend, mirroring what the Go agent does today.

The protocol is different from the agent's /v0.7/config: the backend speaks protobuf and ships raw TUF metadata + target files that the client must verify locally. This means embedding TUF trust roots per site and running a full Uptane-style validation on every poll.

Changes

  • Add a new AgentlessFetcher in libdd-remote-config/src/fetch/agentless.rs:

    • TUF verification of director + config repositories using rust-tuf (DataDog fork of the library).
    • Embedded trust roots for prod, staging and gov, selected from the endpoint host. Roots can also be overridden from disk via AgentlessConfig.
    • Hash + length verification of every target against TUF-validated metadata before exposing it to the caller.
  • Plug agentless mode into the existing ConfigFetcher:

    • New AgentlessConfig. When set the fetcher dispatches to AgentlessFetcher instead of the agent HTTP path. Invalid configs (empty hostname, non-https endpoint, no API key) downgrade to agent mode with log.
    • ConfigFetcher::new is now async since TUF root loading is async. The agent-only path is still immediate.
    • Propagate the server-recommended refresh interval through ConfigClientState and into the SharedFetcher run loop.
  • Update the remote_config_fetch example to switch to agentless when DD_API_KEY + DD_SITE are set.

Additional Notes

Eventually this implementation could be extracted to be it's own crate and we could reuse it in other projects that need a rust client.
For this reason, the agentless.rs module which contains the most of the implementation tries to encapsulate what is "RC agentless" and what is specific to the RC client in libdatadog.
This is why the caching of target files is done in a wrapper type. This cache is currently quite complex as it permits storing files in shared memory for instance.

Currently this implementation is a simplification of the agent code in two ways:

  • fetches data for only one active client, which is also a simplification compared to what the agent does
  • does not control the polling loop but instead returns a refresh interval parameter which the users of the library should respect.

How to test the change?

   DD_API_KEY=... DD_SITE=datadoghq.com \
     cargo run -p libdd-remote-config --example remote_config_fetch

Also this has been tested in dd-trace-rs
DataDog/dd-trace-rs#263

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 1 job - 1 passed on retry View in Datadog

🎯 Code Coverage (details)
Patch Coverage: 85.73%
Overall Coverage: 76.33% (+0.39%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 75dd6b8 | Docs | Datadog PR Page | Give us feedback!

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Clippy Allow Annotation Report

Tracked Clippy allow annotations changed vs main: ⚠️ +1 (22 → 23)

Rule Base PR Δ
expect_used 5 6 ⚠️ +1
By file and crate

By file

File Base PR Δ
libdd-remote-config/src/fetch/fetcher.rs 1 2 ⚠️ +1

By crate

Crate Base PR Δ
libdd-remote-config 3 4 ⚠️ +1

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. Panic-inducing macros in particular should be avoided. In the future, this report may become a PR-blocking quality gate.

Comment thread libdd-remote-config/src/fetch/agentless.rs Outdated
@paullegranddc paullegranddc changed the title feat: agentless RC fetcher feat(remote-config)!: agentless RC fetcher Jun 22, 2026
@github-actions github-actions Bot removed the sidecar label Jun 23, 2026
@paullegranddc
paullegranddc marked this pull request as ready for review June 23, 2026 14:13
@paullegranddc
paullegranddc requested review from a team as code owners June 23, 2026 14:13
const ENV: &str = "testenv";
const VERSION: &str = "1.2.3";

fn get_hostname() -> String {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's really no other way in rust to get the hostname besides running a command? It might not be present in some situations (pod with just the go binary for instance).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just the example. I added this at the beginning instead of adding the hostname crate as a dependency, since this is not code that is shipped in the library.

In general we prefer if people using the library inject these parameters, although I'd understand if you preferred that the libdd-remote-config picks the hostname by default as this is more likely to be consistent across languages (for nodejs though, we cannot as we need to compile to WASM)

@mellon85 mellon85 Aug 6, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then can you please add a comment about it before people copy paste this

Comment thread libdd-remote-config/src/fetch/agentless/integration_tests.rs
Comment thread libdd-remote-config/src/fetch/fetcher.rs Outdated
paullegranddc and others added 4 commits August 5, 2026 15:12
…mote_config/agentless_fetcher

* 'main' of github.com:DataDog/libdatadog:
  feat(data-pipeline)!: add flush_and_close to the trace buffer (#2313)
  refactor(rc)!: make conversion from RemoteConfigProduct back and forth generally available (#2325)
  chore: release v40.0.0 (#2326)
  feat(datadog-ffe): server-side EVP flagevaluation payload + bincode-safe sidecar delivery (#2117)
  feat(data-pipeline): set structured span values (#2300)
  feat(telemetry)!: Add Installation signature and AppProduct changes payloads (#2213)
  fix(libdd-trace-utils): apply SpanLink flags masking when v0.5 json encoding (#2314)
  feat(data-pipeline): encode structured values (#2304)
  feat(crashtracking): send debug log when no data is received at all (#2321)
  chore: release v39.0.0 (#2324)
  feat(data-pipeline): add span links FFI (#2305)
  fix(crashtracking): check fields and exclude uuid for `has_data` (#2322)
  feat(data-pipeline): add span events FFI (#2301)
@dd-octo-sts

dd-octo-sts Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.20 MB 8.20 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 89.18 MB 89.18 MB +0% (+224 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 100.36 MB 100.36 MB +0% (+256 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.06 MB 11.06 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 26.63 MB 26.63 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 94.56 KB 94.56 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 191.42 MB 191.46 MB +.02% (+40.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 1000.66 MB 1001.23 MB +.05% (+582.66 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.71 MB 8.71 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 94.56 KB 94.56 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 25.62 MB 25.62 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 51.01 MB 51.01 MB +0% (+56 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 23.20 MB 23.20 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 96.04 KB 96.04 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 196.26 MB 196.31 MB +.02% (+48.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 990.13 MB 990.47 MB +.03% (+351.78 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.73 MB 6.73 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 96.04 KB 96.04 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 27.55 MB 27.55 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 48.56 MB 48.56 MB -0% (-160 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 79.46 MB 79.46 MB +0% (+216 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 9.12 MB 9.12 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 95.11 MB 95.11 MB +0% (+144 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.14 MB 11.14 MB 0% (0 B) 👌

@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 20a3f4d into main Aug 13, 2026
99 of 101 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the paullgdc/remote_config/agentless_fetcher branch August 13, 2026 11:02
hoolioh added a commit that referenced this pull request Aug 17, 2026
# Release proposal for libdd-sampling and its dependencies

This PR contains version bumps based on public API changes and commits
since last release.

## libdd-common
**Next version:** `5.2.0`
**Semver bump:** `minor`
**Tag:** `libdd-common-v5.2.0`

### Commits

- feat(remote-config)!: agentless RC fetcher (#2112)
- feat(common): allocation-free tag validation and parsing (#2329)

## libdd-trace-utils
**Next version:** `10.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-trace-utils-v10.1.0`

### Commits

- feat(data-pipeline): adds new attributes to OTLP trace metrics export
(#2316)
- feat: add v1 isolated bricks for trace exporter (#2280)

## libdd-sampling
**Next version:** `6.0.0`
**Semver bump:** `major`
**Tag:** `libdd-sampling-v6.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-utils`: ^9.0.0 → ^10.1.0

### Commits

- feat(sampling): OTel consistent-probability rv/th derivation
(APMAPI-2181) (#2276)
- fix(sampling): record rate limiter's effective rate on allow, not just
drop (#2288)
- refactor: migrate to workspace dependencies, phase 4 (#2296)
- chore: moving to workspace-level dependencies, phase 2 (#2270)
- refactor: consolidate core dependencies at workspace level (phase 1)
(#2253)

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: hoolioh <107922352+hoolioh@users.noreply.github.com>
hoolioh added a commit that referenced this pull request Aug 26, 2026
…ker, libdd-data-pipeline, li... (#2409)

# Release proposal for libdd-capabilities-impl, libdd-crashtracker,
libdd-data-pipeline, libdd-http-client, libdd-ipc, libdd-library-config,
libdd-otel-thread-ctx, libdd-remote-config, libdd-shared-runtime,
libdd-telemetry, libdd-tracer-flare and their dependencies

This PR contains version bumps based on public API changes and commits
since last release.

## libdd-gotter
**Next version:** `1.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-gotter-v1.1.0`

### Commits

- fix(crashtracker): fix vdso recognition during GOT patching on musl
(#2379)

## libdd-ipc-macros
**Next version:** `1.0.0`
**Semver bump:** `major`
**Tag:** `libdd-ipc-macros-v1.0.0`

**Warning:** this is an initial release. Please verify that the version
and commits included are correct.


## libdd-trace-protobuf
**Next version:** `5.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-protobuf-v5.0.0`

### Commits

- chore(trace-protobuf)!: sync span.proto with datadog-agent (#2356)

## libdd-library-config
**Next version:** `4.0.0`
**Semver bump:** `major`
**Tag:** `libdd-library-config-v4.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.0 → ^5.0.0

### Commits

- fix(library-config): omit absent process context attributes (#2337)
- refactor: migrate to workspace dependencies, phase 4 (#2296)
- refactor: migrate to workspace dependencies, phase 3 (#2283)
- refactor: consolidate core dependencies at workspace level (phase 1)
(#2253)
- fix(library-config): update Linux process context (#2237)
- refactor(library-config): reorganize Linux process context (#2228)
- feat(library-config): otel process context reader (#2176)

## libdd-remote-config
**Next version:** `4.0.0`
**Semver bump:** `major`
**Tag:** `libdd-remote-config-v4.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0

### Commits

- feat(rc): add overrides for RC config/director roots (#2404)
- fix(rc): avoid depending on the crypto nodejs API for remote config
(#2407)
- fix(rc)!: fixup rc to be wasm compatible (#2393)
- feat: Revert vendoring of tuf-rust (#2365), use new crates.io package
(#2374)
- feat(agentless)!: vendor rust-tuf crate in libdatadog for release
(#2365)
- chore(rc)!: add DEBUG product (#2306)
- feat(remote-config)!: agentless RC fetcher (#2112)

## libdd-trace-utils
**Next version:** `11.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-utils-v11.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0

### Commits

- feat(compression): use zrip for wasm targets (#2386)
- fix(trace-utils)!: fix forward compatibility for string streaming
(#2295)

## libdd-tracer-flare
**Next version:** `2.0.0`
**Semver bump:** `major`
**Tag:** `libdd-tracer-flare-v2.0.0`

### ⚠️ major bump forced due to:

- `libdd-remote-config`: ^3.0.0 → ^4.0.0
- `libdd-trace-utils`: ^10.0.0 → ^11.0.0

### Commits

- feat(remote-config)!: agentless RC fetcher (#2112)

## libdd-crashtracker
**Next version:** `2.0.1`
**Semver bump:** `patch`
**Tag:** `libdd-crashtracker-v2.0.1`

### Commits

- fix(crashtracker): hold the collector connection open through
symbolization (#2384)
- fix(crashtracking): stop resolving thread symbols with libunwind in
the receiver (#2361)

## libdd-data-pipeline
**Next version:** `9.0.0`
**Semver bump:** `major`
**Tag:** `libdd-data-pipeline-v9.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0
- `libdd-trace-utils`: ^10.1.0 → ^11.0.0

### Commits

- feat(compression): use zrip for wasm targets (#2386)

## libdd-ipc
**Next version:** `1.0.0`
**Semver bump:** `major`
**Tag:** `libdd-ipc-v1.0.0`

**Warning:** this is an initial release. Please verify that the version
and commits included are correct.


## libdd-trace-normalization
**Next version:** `4.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-normalization-v4.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0


## libdd-trace-obfuscation
**Next version:** `7.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-obfuscation-v7.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0
- `libdd-trace-utils`: ^10.1.0 → ^11.0.0


## libdd-trace-stats
**Next version:** `8.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-stats-v8.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0
- `libdd-trace-utils`: ^10.1.0 → ^11.0.0

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: hoolioh <107922352+hoolioh@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants