test: add Gherkin evaluator tests using test-harness specs#1947
test: add Gherkin evaluator tests using test-harness specs#1947aepfli wants to merge 1 commit intoopen-feature:mainfrom
Conversation
✅ Deploy Preview for polite-licorice-3db33c canceled.
|
There was a problem hiding this comment.
Code Review
This pull request introduces Gherkin-based integration tests for the evaluator component. It adds a new test file core/pkg/evaluator/gherkin_test.go utilizing the godog framework to execute feature-based tests against the evaluator's logic, including support for various data types, targeting keys, and metadata assertions. Additionally, the Makefile has been updated with a test-evaluator-gherkin target, and necessary dependencies have been added to core/go.mod. I have no feedback to provide.
Add in-process Gherkin tests for the JSON evaluator using godog and the shared test-harness evaluator feature files. Tests validate flag evaluation, error handling, targeting, metadata, fractional bucketing, semver, string operators, zero-values, evaluator refs, and no-default-variant behavior directly against the evaluator without testcontainers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Simon Schrottner <simon.schrottner@gmail.com>
1962bd7 to
d43c66e
Compare
|
|
@aepfli Nice. I don't think this is really a feat for end-users. I think let's make it a chore/test? Also - should we run it in the CI? |
|
I agree with the pr title, and changed it
Maybe my knowledge is off, but we do not have the e2e tag, so this should execute with normal test execution, or am i wrong here? as it does not need docker, etc. it should be fast, and there is no reason to not run it during normal test execution. |
I don't think it's running. The submodule doesn't seem to be checked out in the CI and the coverage is exactly the same. |



Summary
godogand the sharedtest-harness/evaluator/feature specs@fractional-v1excluded (flagd implements v2)test-evaluator-gherkinMakefile targetRelated: open-feature/python-sdk-contrib#377 (equivalent Python testkit)
Part of: open-feature/flagd-testbed#366 (centralized evaluator testkit migration)
See also: open-feature/flagd-testbed#367 (error handling discussion)
Test plan
make test-evaluator-gherkinpasses all 83 scenariosgo test -short ./core/pkg/evaluator/...)🤖 Generated with Claude Code