Skip to content

fix: remove tests checking telemetry otlp configs which can be sensitive#7034

Draft
paullegranddc wants to merge 1 commit into
mainfrom
paullgdc/otlp_endpoints/update_test_header_redaction
Draft

fix: remove tests checking telemetry otlp configs which can be sensitive#7034
paullegranddc wants to merge 1 commit into
mainfrom
paullgdc/otlp_endpoints/update_test_header_redaction

Conversation

@paullegranddc
Copy link
Copy Markdown
Contributor

Motivation

We want to redact these values in libraries. These tests prevent these changes, but changing them outright will break libraries that haven't done the change

@github-actions
Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

tests/parametric/test_otel_logs.py                                      @DataDog/system-tests-core @DataDog/apm-sdk-capabilities
tests/parametric/test_otel_metrics.py                                   @DataDog/system-tests-core @DataDog/apm-sdk-capabilities

@datadog-official
Copy link
Copy Markdown

datadog-official Bot commented May 28, 2026

Pipelines  Tests

Fix all issues with BitsAI

⚠️ Warnings

🚦 15 Pipeline jobs failed

Testing the test | System Tests (ruby, prod) / parametric / parametric (1)   View in Datadog   GitHub Actions

🔧 Fix in code (Fix with Cursor). 3 failed tests due to ValueError: Number 1 of logs not available from test agent, got 0 in tests/parametric/test_otel_logs.py and tests/parametric/test_otel_metrics.py.

🧪 2 Tests failed

tests.parametric.test_otel_logs.Test_FR11_Telemetry.test_telemetry_exporter_configurations[library_env0-4320, parametric-ruby] from system_tests_suite   View in Datadog (Fix with Cursor)
ValueError: Number 1 of logs not available from test agent, got 0

self = <tests.parametric.test_otel_logs.Test_FR11_Telemetry object at 0x7f8a00f30cb0>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f89cbbed0a0>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f89cc42dfa0>

    @pytest.mark.parametrize(
        ("library_env", "test_agent_otlp_http_port"),
        [
            (
...
tests.parametric.test_otel_metrics.Test_Otel_Metrics_Telemetry.test_telemetry_exporter_configurations[library_env0-4320, parametric-ruby] from system_tests_suite   View in Datadog (Fix with Cursor)
ValueError: Number (1) of metrics not available from test agent, got 0

self = <tests.parametric.test_otel_metrics.Test_Otel_Metrics_Telemetry object at 0x7f356fd94200>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f353dd47710>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f353daed040>

    @pytest.mark.parametrize(
        ("library_env", "test_agent_otlp_http_port"),
        [
            (
...

Testing the test | System Tests (rust, dev) / parametric / parametric (2)   View in Datadog   GitHub Actions

🔧 Fix in code (Fix with Cursor). 2 failed tests. ValueError: Number (1) of metrics not available from test agent, got 0 in tests/parametric/test_otel_metrics.py.

🧪 1 Test failed

tests.parametric.test_otel_metrics.Test_Otel_Metrics_Telemetry.test_telemetry_exporter_configurations[library_env0-4320, parametric-rust] from system_tests_suite   View in Datadog (Fix with Cursor)
ValueError: Number (1) of metrics not available from test agent, got 0

self = <tests.parametric.test_otel_metrics.Test_Otel_Metrics_Telemetry object at 0x7f9f084c4380>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f9ed3dade50>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f9ed3dd68d0>

    @pytest.mark.parametrize(
        ("library_env", "test_agent_otlp_http_port"),
        [
            (
...

Testing the test | System Tests (ruby, dev) / parametric / parametric (1)   View in Datadog   GitHub Actions

🔄 Retry job. This looks flaky and may succeed on retry. Test failed due to ValueError: Number 1 of logs not available from test agent, got 0 in tests/parametric/test_otel_logs.py:681

🧪 1 Test failed

tests.parametric.test_otel_logs.Test_FR11_Telemetry.test_telemetry_exporter_configurations[library_env0-4320, parametric-ruby] from system_tests_suite   View in Datadog (Fix with Cursor)
ValueError: Number 1 of logs not available from test agent, got 0

self = <tests.parametric.test_otel_logs.Test_FR11_Telemetry object at 0x7fc08071ffe0>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7fc04e692900>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7fc04e690140>

    @pytest.mark.parametrize(
        ("library_env", "test_agent_otlp_http_port"),
        [
            (
...

View all 15 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: b1f9db2 | Docs | Datadog PR Page | Give us feedback!

@paullegranddc paullegranddc force-pushed the paullgdc/otlp_endpoints/update_test_header_redaction branch from 2750d0e to 10379d2 Compare May 28, 2026 13:55
# Motivation

We want to redact these values in libraries. These tests prevent these changes, but changing them outright will break libraries that haven't done the change
@paullegranddc paullegranddc force-pushed the paullgdc/otlp_endpoints/update_test_header_redaction branch from 10379d2 to b1f9db2 Compare May 28, 2026 14:02

@pytest.mark.parametrize(
("library_env", "endpoint_env", "test_agent_otlp_http_port"),
("library_env", "test_agent_otlp_http_port"),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please restore the endpoint_env pytest parameter and the value in the corresponding argument, that should allow the test cases to be wired up correctly and pass

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants