Enable PHP FFE exposure system tests

[leoromanovsky]

Motivation

PHP FFE exposure logging now lives in the native dd-trace-php/runtime path and needs the shared FFE exposure system-tests enabled so regressions are caught outside local dogfooding. The M1 evaluation PR only proves remote-config evaluation; this stacked PR adds the next validation layer for exposure emission and cache behavior.

Planning/reference doc: https://docs.google.com/document/d/1NvMfTpZWLBlFmEFNjdnlMyeVpy5l7KD8qujGFco6w2w/edit?tab=t.0

Changes

This PR enables tests/ffe/test_exposures.py for PHP at v1.21.0-dev. It intentionally does not enable FFE evaluation-metric tests; those belong to a separate stacked PR for DataDog/dd-trace-php#3911 and #7033.

The PR is stacked on #7003, which adds the PHP FFE scaffold and evaluation-test enablement. The PHP behavior being validated here is implemented in DataDog/dd-trace-php#3910 and the sidecar delivery/cache support is implemented in DataDog/libdatadog#2026.

Decisions

The scope is deliberately one manifest activation. The shared exposure tests already cover the required product behavior: event generation, multiple RC files, malformed/empty RC handling, same-subject deduplication, different-subject emission, allocation/variant changes, doLog=false, missing flags, and empty targeting keys.

The local proof uses a matched PHP 8.2 NTS artifact and the php-fpm-8.2 weblog, which matches the PHP 8.2 target Bob recommended for FFE system-test iteration.

Related PRs

• PHP runtime evaluation base: feat(ffe): add runtime-backed PHP feature flag evaluation dd-trace-php#3906
• PHP exposure implementation: Add FFE exposure emission dd-trace-php#3910
• System-tests PHP FFE scaffold/evaluations base: Enable PHP FFE evaluation system tests #7003
• Sidecar delivery/cache: feat(sidecar): forward FFE exposures to EVP proxy libdatadog#2026

Validation

Local behavior validation before this conflict-resolution rebase (previous system-tests PR head a27a887bb), dd-trace-php exposure branch now pushed as 416749dd3, and libdatadog submodule 8be471fbc:

cd /Users/leo.romanovsky/go/src/github.com/DataDog/dd-trace-php-ffe-runtime-first
./tooling/bin/build-debug-artifact gnu-aarch64-8.2-nts \
  /Users/leo.romanovsky/go/src/github.com/DataDog/system-tests-pr7031/binaries

cd /Users/leo.romanovsky/go/src/github.com/DataDog/system-tests-pr7031
DOCKER_CONFIG=/tmp/system-tests-docker-config-nocredsstore \
SYSTEM_TEST_BUILD_TIMEOUT=1800 \
./build.sh --library php --weblog-variant php-fpm-8.2

DOCKER_CONFIG=/tmp/system-tests-docker-config-nocredsstore \
TEST_LIBRARY=php \
WEBLOG_VARIANT=php-fpm-8.2 \
./run.sh +l php FEATURE_FLAGGING_AND_EXPERIMENTATION tests/ffe/test_exposures.py

Result: 11 passed in 78.22s.

Post-rebase verification: branch head is now f54c036f4, stacked on the updated parent, enabling tests/ffe/test_exposures.py at v1.21.0-dev to match #7003. PATH=/opt/homebrew/opt/coreutils/libexec/gnubin:$PATH ./format.sh passed. The rebase only changed the manifest activation floor; the test behavior and local PHP artifact path above did not change.

MacOS arm64 local-run note: the published PHP 8.2 base image inspected locally was amd64-only, so the local validation first built datadog/system-tests:php-fpm-8.2.base-v1 for arm64 using ./utils/build/build_php_base_images.sh php-fpm-8_2. The temporary local base-image build tweak was reverted; this PR remains only the manifest activation.

[datadog-prod-us1-4]

 

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 26 Pipeline jobs failed

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0 1     

🔧 Fix in code (Fix with Cursor).

8 tests failed. Assertion Error: Expected to find flags 'test-flag-1' or 'test-flag-2' in exposure events, found: set().

🧪 1 Test failed

tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key.test_ffe_exp_5_missing_targeting_key[apache-mod-7.0] from system_tests_suite   (Fix with Cursor)

AssertionError: EXP.5 FAILED: Expected exposure event for flag 'exp-5-missing-targeting-key-flag' with subject.id = '', but no matching event was found. Events received for this flag: []. The tracer must NOT skip exposures when targeting key is empty.
assert None is not None

self = <tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key object at 0x7f785501ec60>

    def test_ffe_exp_5_missing_targeting_key(self):
        """EXP.5: Test that empty targeting key generates exposure with subject.id = ''."""
        assert self.response.status_code == 200, f"Flag evaluation failed: {self.response.text}"
    
        result = json.loads(self.response.text)
...

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0-zts 1     

🔧 Fix in code (Fix with Cursor).

8 failed tests. AssertionError: Expected to find flag 'test-flag' with subject 'test-user' in exposure events.

🧪 1 Test failed

tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key.test_ffe_exp_5_missing_targeting_key[apache-mod-7.0-zts] from system_tests_suite   (Fix with Cursor)

AssertionError: EXP.5 FAILED: Expected exposure event for flag 'exp-5-missing-targeting-key-flag' with subject.id = '', but no matching event was found. Events received for this flag: []. The tracer must NOT skip exposures when targeting key is empty.
assert None is not None

self = <tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key object at 0x7f63f862c5c0>

    def test_ffe_exp_5_missing_targeting_key(self):
        """EXP.5: Test that empty targeting key generates exposure with subject.id = ''."""
        assert self.response.status_code == 200, f"Flag evaluation failed: {self.response.text}"
    
        result = json.loads(self.response.text)
...

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.1 1     

🔧 Fix in code (Fix with Cursor).

8 failed tests related to missing exposure events for flags during evaluation. Expectation not met: expected flag exposure events to be found but none exists.

🧪 1 Test failed

tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key.test_ffe_exp_5_missing_targeting_key[apache-mod-7.1] from system_tests_suite   (Fix with Cursor)

AssertionError: EXP.5 FAILED: Expected exposure event for flag 'exp-5-missing-targeting-key-flag' with subject.id = '', but no matching event was found. Events received for this flag: []. The tracer must NOT skip exposures when targeting key is empty.
assert None is not None

self = <tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key object at 0x7fe512c86210>

    def test_ffe_exp_5_missing_targeting_key(self):
        """EXP.5: Test that empty targeting key generates exposure with subject.id = ''."""
        assert self.response.status_code == 200, f"Flag evaluation failed: {self.response.text}"
    
        result = json.loads(self.response.text)
...

View all 26 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: f54c036 | Docs | Datadog PR Page | Give us feedback!}

[github-actions]

CODEOWNERS have been resolved as:

manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php