Enable PHP FFE parametric system tests

[leoromanovsky]

Motivation

PHP Milestone 1 now has a runtime-backed implementation in DataDog/dd-trace-php#3906. System-tests should validate that PHP can consume live FFE_FLAGS Remote Config and evaluate through the libdatadog evaluator before PHP moves on to exposures and metrics.

Reference design doc: https://docs.google.com/document/d/1NvMfTpZWLBlFmEFNjdnlMyeVpy5l7KD8qujGFco6w2w/edit?tab=t.0

Changes

• Adds PHP parametric /ffe/start and /ffe/evaluate endpoints backed by DDTrace\FeatureFlags\Client.
• Normalizes defaults, targeting keys, and scalar attributes into the PHP Datadog client request shape.
• Preserves top-level empty JSON objects in parametric responses so canonical JSON fixture cases compare correctly.
• Suppresses the M1 transitional warning inside PHP FFE endpoints so responses remain valid JSON.
• Enables PHP for Test_Feature_Flag_Dynamic_Evaluation and Test_Parametric_FFE_Start in manifests/php.yml.
• Keeps PHP weblog FFE, exposures, and flag-evaluation metrics marked missing_feature until their separate milestones are implemented.
• Updates tests/ffe/README.md with the validated PHP 8.2 parametric runbook and Bob's local artifact build command.

Decisions

• Activate the validated PHP 8.2 parametric path first; defer weblog activation because local Apache PHP 8.2 image builds are not reliable on this arm64 machine.
• Use live Remote Config in the test path; no fake evaluator or fixture-only PHP product path.
• Keep exposure and metric assertions disabled until PHP emits those signals through the same collector paths already proven with non-PHP baselines.

Verification

Using a local artifact built from DataDog/dd-trace-php#3906:

./tooling/bin/build-debug-artifact gnu-aarch64-8.2-nts /Users/leo.romanovsky/go/src/github.com/DataDog/system-tests-pr-g-php-ffe-scaffold/binaries

Local runs:

• TEST_LIBRARY=php ./run.sh PARAMETRIC tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation -F tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation -vv: 25/25 passed.
• focused test-json-config-flag.json: 1/1 passed.
• focused test-case-of-7-empty-targeting-key.json: 1/1 passed.
• TEST_LIBRARY=php ./run.sh PARAMETRIC tests/parametric/test_parametric_endpoints.py::Test_Parametric_FFE_Start -F tests/parametric/test_parametric_endpoints.py::Test_Parametric_FFE_Start -vv: 1/1 passed.
• php -l utils/build/docker/php/parametric/server.php: passed.
• ./format.sh: passed.
• git diff --check: passed.

[github-actions]

CODEOWNERS have been resolved as:

tests/parametric/test_ffe/AGENTS.md                                     @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core
utils/build/docker/php/common/ffe.php                                   @DataDog/apm-php @DataDog/system-tests-core
docs/understand/weblogs/end-to-end_weblog.md                            @DataDog/system-tests-core
manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php
tests/ffe/README.md                                                     @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core
tests/parametric/test_ffe/test_dynamic_evaluation.py                    @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core
utils/build/docker/php/common/rewrite-rules.conf                        @DataDog/apm-php @DataDog/system-tests-core
utils/build/docker/php/parametric/server.php                            @DataDog/apm-php @DataDog/system-tests-core
utils/docker_fixtures/_test_clients/_test_client_parametric.py          @DataDog/system-tests-core

[datadog-prod-us1-4]

 

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 4 Pipeline jobs failed

Testing the test | System Tests (php, dev) / parametric / parametric (2)     

🔧 Fix in code (Fix with Cursor).

Failed to start FFE provider in test_dynamic_evaluation.py:213 due to unhandled assertion error.

🧪 10 Tests failed

tests.parametric.test_ffe.test_dynamic_evaluation.Test_Feature_Flag_Dynamic_Evaluation.test_ffe_flag_evaluation[test-case-boolean-false-assignment.json-library_env0, parametric-php] from system_tests_suite   (Fix with Cursor)

AssertionError: Failed to start FFE provider
assert False

self = <tests.parametric.test_ffe.test_dynamic_evaluation.Test_Feature_Flag_Dynamic_Evaluation object at 0x7f0a78f253a0>
test_case_file = 'test-case-boolean-false-assignment.json'
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f0a76979e20>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f0a76944050>

    @parametrize("library_env", [{**DEFAULT_ENVVARS}])
    @parametrize("test_case_file", ALL_TEST_CASE_FILES)
...

tests.parametric.test_ffe.test_dynamic_evaluation.Test_Feature_Flag_Dynamic_Evaluation.test_ffe_flag_evaluation[test-case-boolean-one-of-matches.json-library_env0, parametric-php] from system_tests_suite   (Fix with Cursor)

AssertionError: Failed to start FFE provider
assert False

self = <tests.parametric.test_ffe.test_dynamic_evaluation.Test_Feature_Flag_Dynamic_Evaluation object at 0x7fb459543b90>
test_case_file = 'test-case-boolean-one-of-matches.json'
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7fb458ea3fb0>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7fb4592b2de0>

    @parametrize("library_env", [{**DEFAULT_ENVVARS}])
    @parametrize("test_case_file", ALL_TEST_CASE_FILES)
...

View all 10 test failures

Testing the test | System Tests (php, prod) / parametric / parametric (1)     

🔧 Fix in code (Fix with Cursor).

AssertionError: Failed to start FFE provider in tests/parametric/test_ffe/test_dynamic_evaluation.py:213

🧪 10 Tests failed

tests.parametric.test_ffe.test_dynamic_evaluation.Test_Feature_Flag_Dynamic_Evaluation.test_ffe_flag_evaluation[test-case-boolean-false-assignment.json-library_env0, parametric-php] from system_tests_suite   (Fix with Cursor)

AssertionError: Failed to start FFE provider
assert False

self = <tests.parametric.test_ffe.test_dynamic_evaluation.Test_Feature_Flag_Dynamic_Evaluation object at 0x7f5450bcda60>
test_case_file = 'test-case-boolean-false-assignment.json'
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f544ff5e810>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f5450871d90>

    @parametrize("library_env", [{**DEFAULT_ENVVARS}])
    @parametrize("test_case_file", ALL_TEST_CASE_FILES)
...

tests.parametric.test_ffe.test_dynamic_evaluation.Test_Feature_Flag_Dynamic_Evaluation.test_ffe_flag_evaluation[test-case-boolean-one-of-matches.json-library_env0, parametric-php] from system_tests_suite   (Fix with Cursor)

AssertionError: Failed to start FFE provider
assert False

self = <tests.parametric.test_ffe.test_dynamic_evaluation.Test_Feature_Flag_Dynamic_Evaluation object at 0x7f5450bcd220>
test_case_file = 'test-case-boolean-one-of-matches.json'
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f545040eea0>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f545040f830>

    @parametrize("library_env", [{**DEFAULT_ENVVARS}])
    @parametrize("test_case_file", ALL_TEST_CASE_FILES)
...

View all 10 test failures

Testing the test | System Tests (python, dev) / End-to-end #1 / uwsgi-poc 1     

🔧 Fix in code (Fix with Cursor).

1 failed test. AssertionError: no extra_services contains extraVegetables in tests/remote_config/test_remote_configuration.py:267

View all 4 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4a955e5 | Docs | Datadog PR Page | Give us feedback!}