[feat][evaluation] Coze Coding Evaluation Target Support#461
Open
[feat][evaluation] Coze Coding Evaluation Target Support#461
Conversation
Codecov Report❌ Patch coverage is @@ Coverage Diff @@
## main #461 +/- ##
==========================================
+ Coverage 74.45% 74.52% +0.07%
==========================================
Files 629 629
Lines 66337 66438 +101
==========================================
+ Hits 49389 49511 +122
+ Misses 13663 13644 -19
+ Partials 3285 3283 -2
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 3 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
dsf86
previously approved these changes
Mar 19, 2026
Add case for EvaluatorTypeCustomRPC in convertEntityEvaluatorTypeToOpenAPI function and refactor evaluator version ID retrieval to use GetEvaluatorVersionID method. Also add test case for agent evaluator in SubmitExperimentOApi test.
Add more comprehensive test cases to verify conversion of different evaluator types
Add EvaluatorTypeAgent constant and handle conversion between entity and openapi types. Also add validation to reject agent type in evaluator openapi conversion.
Skip workspace validation for builtin evaluators to allow cross-workspace execution. Add test cases for evaluator version not found and builtin success scenarios.
…idation Add optional Extra field to ImportEvaluationSetOApiRequest and GetEvaluationSetIOJobOApiRequest thrift structs Implement validation, serialization and deserialization for the new field in generated code
implement API to run builtin evaluators by ID or name, including: - add new endpoint /v1/loop/evaluation/builtin_evaluators/run - add service method to resolve visible version ID - add repo method to get evaluator by space ID and name - update thrift IDL and generate code - add tests for new functionality
- Move builtin evaluator endpoint from `/builtin_evaluators/run` to `/evaluators/builtin/run` - Add new middleware `_builtinMw` for builtin evaluator routes - Implement `GetEvaluatorMetaBySpaceIDAndName` repo method and tests - Add `ResolveBuiltinEvaluatorVisibleVersionID` service method and tests
Clarify that either builtin_evaluator_id or builtin_evaluator_name must be provided, and if both are provided, they must match
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Check the PR title
(Optional) Translate the PR title into Chinese
(Optional) More detailed description for this PR(en: English/zh: Chinese)
en:
zh(optional):
(Optional) Which issue(s) this PR fixes