feat(remote-config)!: agentless RC fetcher - #2112
feat(remote-config)!: agentless RC fetcher#2112gh-worker-dd-mergequeue-cf854d[bot] merged 84 commits into
Conversation
🎉 All green!🧪 All tests passed 🔄 Datadog auto-retried 1 job - 1 passed on retry 🎯 Code Coverage (details) 🔗 Commit SHA: 75dd6b8 | Docs | Datadog PR Page | Give us feedback! |
Clippy Allow Annotation ReportTracked Clippy
By file and crateBy file
By crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. Panic-inducing macros in particular should be avoided. In the future, this report may become a PR-blocking quality gate. |
| const ENV: &str = "testenv"; | ||
| const VERSION: &str = "1.2.3"; | ||
|
|
||
| fn get_hostname() -> String { |
There was a problem hiding this comment.
there's really no other way in rust to get the hostname besides running a command? It might not be present in some situations (pod with just the go binary for instance).
There was a problem hiding this comment.
This is just the example. I added this at the beginning instead of adding the hostname crate as a dependency, since this is not code that is shipped in the library.
In general we prefer if people using the library inject these parameters, although I'd understand if you preferred that the libdd-remote-config picks the hostname by default as this is more likely to be consistent across languages (for nodejs though, we cannot as we need to compile to WASM)
There was a problem hiding this comment.
then can you please add a comment about it before people copy paste this
…mote_config/agentless_fetcher * 'main' of github.com:DataDog/libdatadog: feat(data-pipeline)!: add flush_and_close to the trace buffer (#2313) refactor(rc)!: make conversion from RemoteConfigProduct back and forth generally available (#2325) chore: release v40.0.0 (#2326) feat(datadog-ffe): server-side EVP flagevaluation payload + bincode-safe sidecar delivery (#2117) feat(data-pipeline): set structured span values (#2300) feat(telemetry)!: Add Installation signature and AppProduct changes payloads (#2213) fix(libdd-trace-utils): apply SpanLink flags masking when v0.5 json encoding (#2314) feat(data-pipeline): encode structured values (#2304) feat(crashtracking): send debug log when no data is received at all (#2321) chore: release v39.0.0 (#2324) feat(data-pipeline): add span links FFI (#2305) fix(crashtracking): check fields and exclude uuid for `has_data` (#2322) feat(data-pipeline): add span events FFI (#2301)
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
20a3f4d
into
main
# Release proposal for libdd-sampling and its dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-common **Next version:** `5.2.0` **Semver bump:** `minor` **Tag:** `libdd-common-v5.2.0` ### Commits - feat(remote-config)!: agentless RC fetcher (#2112) - feat(common): allocation-free tag validation and parsing (#2329) ## libdd-trace-utils **Next version:** `10.1.0` **Semver bump:** `minor` **Tag:** `libdd-trace-utils-v10.1.0` ### Commits - feat(data-pipeline): adds new attributes to OTLP trace metrics export (#2316) - feat: add v1 isolated bricks for trace exporter (#2280) ## libdd-sampling **Next version:** `6.0.0` **Semver bump:** `major` **Tag:** `libdd-sampling-v6.0.0` ###⚠️ major bump forced due to: - `libdd-trace-utils`: ^9.0.0 → ^10.1.0 ### Commits - feat(sampling): OTel consistent-probability rv/th derivation (APMAPI-2181) (#2276) - fix(sampling): record rate limiter's effective rate on allow, not just drop (#2288) - refactor: migrate to workspace dependencies, phase 4 (#2296) - chore: moving to workspace-level dependencies, phase 2 (#2270) - refactor: consolidate core dependencies at workspace level (phase 1) (#2253) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: hoolioh <107922352+hoolioh@users.noreply.github.com>
…ker, libdd-data-pipeline, li... (#2409) # Release proposal for libdd-capabilities-impl, libdd-crashtracker, libdd-data-pipeline, libdd-http-client, libdd-ipc, libdd-library-config, libdd-otel-thread-ctx, libdd-remote-config, libdd-shared-runtime, libdd-telemetry, libdd-tracer-flare and their dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-gotter **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-gotter-v1.1.0` ### Commits - fix(crashtracker): fix vdso recognition during GOT patching on musl (#2379) ## libdd-ipc-macros **Next version:** `1.0.0` **Semver bump:** `major` **Tag:** `libdd-ipc-macros-v1.0.0` **Warning:** this is an initial release. Please verify that the version and commits included are correct. ## libdd-trace-protobuf **Next version:** `5.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-protobuf-v5.0.0` ### Commits - chore(trace-protobuf)!: sync span.proto with datadog-agent (#2356) ## libdd-library-config **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-library-config-v4.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.0 → ^5.0.0 ### Commits - fix(library-config): omit absent process context attributes (#2337) - refactor: migrate to workspace dependencies, phase 4 (#2296) - refactor: migrate to workspace dependencies, phase 3 (#2283) - refactor: consolidate core dependencies at workspace level (phase 1) (#2253) - fix(library-config): update Linux process context (#2237) - refactor(library-config): reorganize Linux process context (#2228) - feat(library-config): otel process context reader (#2176) ## libdd-remote-config **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-remote-config-v4.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 ### Commits - feat(rc): add overrides for RC config/director roots (#2404) - fix(rc): avoid depending on the crypto nodejs API for remote config (#2407) - fix(rc)!: fixup rc to be wasm compatible (#2393) - feat: Revert vendoring of tuf-rust (#2365), use new crates.io package (#2374) - feat(agentless)!: vendor rust-tuf crate in libdatadog for release (#2365) - chore(rc)!: add DEBUG product (#2306) - feat(remote-config)!: agentless RC fetcher (#2112) ## libdd-trace-utils **Next version:** `11.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-utils-v11.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 ### Commits - feat(compression): use zrip for wasm targets (#2386) - fix(trace-utils)!: fix forward compatibility for string streaming (#2295) ## libdd-tracer-flare **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-tracer-flare-v2.0.0` ###⚠️ major bump forced due to: - `libdd-remote-config`: ^3.0.0 → ^4.0.0 - `libdd-trace-utils`: ^10.0.0 → ^11.0.0 ### Commits - feat(remote-config)!: agentless RC fetcher (#2112) ## libdd-crashtracker **Next version:** `2.0.1` **Semver bump:** `patch` **Tag:** `libdd-crashtracker-v2.0.1` ### Commits - fix(crashtracker): hold the collector connection open through symbolization (#2384) - fix(crashtracking): stop resolving thread symbols with libunwind in the receiver (#2361) ## libdd-data-pipeline **Next version:** `9.0.0` **Semver bump:** `major` **Tag:** `libdd-data-pipeline-v9.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 - `libdd-trace-utils`: ^10.1.0 → ^11.0.0 ### Commits - feat(compression): use zrip for wasm targets (#2386) ## libdd-ipc **Next version:** `1.0.0` **Semver bump:** `major` **Tag:** `libdd-ipc-v1.0.0` **Warning:** this is an initial release. Please verify that the version and commits included are correct. ## libdd-trace-normalization **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-normalization-v4.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 ## libdd-trace-obfuscation **Next version:** `7.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-obfuscation-v7.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 - `libdd-trace-utils`: ^10.1.0 → ^11.0.0 ## libdd-trace-stats **Next version:** `8.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-stats-v8.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 - `libdd-trace-utils`: ^10.1.0 → ^11.0.0 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: hoolioh <107922352+hoolioh@users.noreply.github.com>
Motivation
In some environments (serverless, Datadog studio), tracers can't rely on a Datadog Agent to proxy Remote Config requests. This PR adds an agentless mode to libdd-remote-config so the fetcher can talk directly to the RC backend, mirroring what the Go agent does today.
The protocol is different from the agent's /v0.7/config: the backend speaks protobuf and ships raw TUF metadata + target files that the client must verify locally. This means embedding TUF trust roots per site and running a full Uptane-style validation on every poll.
Changes
Add a new AgentlessFetcher in libdd-remote-config/src/fetch/agentless.rs:
Plug agentless mode into the existing ConfigFetcher:
Update the remote_config_fetch example to switch to agentless when DD_API_KEY + DD_SITE are set.
Additional Notes
Eventually this implementation could be extracted to be it's own crate and we could reuse it in other projects that need a rust client.
For this reason, the
agentless.rsmodule which contains the most of the implementation tries to encapsulate what is "RC agentless" and what is specific to the RC client in libdatadog.This is why the caching of target files is done in a wrapper type. This cache is currently quite complex as it permits storing files in shared memory for instance.
Currently this implementation is a simplification of the agent code in two ways:
How to test the change?
Also this has been tested in dd-trace-rs
DataDog/dd-trace-rs#263