[stale] [chore] Swap sendgrid by smtp

[jp-agenta]

Chore/swap sendgrid by smtp

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

GitHub CI seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[Copilot]

Pull Request Overview

This PR performs a major refactoring to swap email provider from SendGrid to SMTP and restructures several core modules, including:

• Removal of legacy applications, queries, and several evaluation-related tables
• Simplification of authentication logic by removing PostHog feature flag integrations
• Consolidation of database migration utilities from async to sync operations
• Renaming of evaluation entities (Results→Steps, Metrics→Metric)
• Removal of extensive test files and manual evaluation SDK examples
• Addition of new utility files and interface definitions

Reviewed Changes

Copilot reviewed 116 out of 2328 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
api/oss/src/apis/fastapi/evaluations/models.py	Renamed evaluation entities and updated field names
api/oss/src/apis/fastapi/applications/utils.py	Removed entire file containing application revision parsing utilities
api/oss/src/apis/fastapi/applications/router.py	Removed entire legacy applications router
api/oss/src/apis/fastapi/applications/models.py	Removed entire file with application API models
api/oss/src/apis/fastapi/annotations/utils.py	Added new utility functions for annotations
api/oss/src/apis/fastapi/annotations/router.py	Major rewrite of annotations router with new implementation
api/oss/src/apis/fastapi/annotations/models.py	Restructured annotation models with new enums
api/oss/src/__init__.py	Simplified authentication by removing PostHog integration
api/oss/docker/Dockerfile.gh	Removed SDK installation and cron job setup
api/oss/docker/Dockerfile.dev	Removed PYTHONPATH setup and cron job configuration
api/oss/databases/postgres/migrations/utils.py	Converted async functions to sync for database operations
Multiple migration files	Removed several database migration files and updated references
api/entrypoint.py	Reorganized imports and router initialization
Multiple EE test files	Removed manual evaluation SDK test files
Multiple EE service files	Updated import statements and function signatures

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The enum names are swapped. span_type_enum should use name='spantype' and trace_type_enum should use name='tracetype'. This will cause the wrong enum types to be dropped during downgrade.

Suggested change

	span_type_enum = sa.Enum(SpanType, name="tracetype")
	trace_type_enum = sa.Enum(TraceType, name="spantype")
	span_type_enum = sa.Enum(SpanType, name="spantype")
	trace_type_enum = sa.Enum(TraceType, name="tracetype")

[Copilot]

The condition should check if (count or 0) > 0: since count could be None. The removed version handled this case, but the new version doesn't.

Suggested change

	if count > 0:
	if (count or 0) > 0:

[Copilot]

Changed from HTTPException with status 409 to generic Exception. This breaks the API contract and removes proper HTTP error handling. Should remain as HTTPException.

[Copilot]

URL parameters are no longer URL-encoded. The email and token parameters should be URL-encoded to prevent injection issues and handle special characters properly.

The best way to fix the issue is to validate the evaluationTableId parameter so that only permitted (safe) values can be used as part of the outgoing request URL. Since the parameter is used in the URL path, prevent path traversal and malicious input by enforcing that it matches the expected format (e.g., alphanumeric, UUID, typical database id). In this context, a simple validation function (using a regex or type check) should be used to ensure only valid run IDs are accepted. If validation fails, the function should avoid making the request and return an error or null instead.
This change should be applied in the useEvaluationRunData hook, in the file web/ee/src/lib/hooks/useEvaluationRunData/index.ts, intercepting and validating evaluationTableId before passing it to the request at line 76.
This will require adding a helper function for validation (for example, a simple regex test for UUID or allowed characters), and updating the code to use it before the request.

General fix:
Validate and sanitize all values originating from user input (in this case evaluationId and, by implication, evaluationIds in array form) before using them to construct outbound request URLs, particularly where the path or query parameters are involved.

Best specific fix:
Since evaluationId is coming from the query parameters and may be an array (or a string), we should validate that it is a legitimate identifier. If, in the system, evaluationId is always expected to be a UUID (v4), use a regular expression (or npm package) to check for a UUID format before proceeding. If invalid, throw an error or ignore the request.
The validation should be performed at the entry point—in this case, in the consumer file (EvaluationCompare.tsx), where we parse and use the query. We should filter the evaluationIds to only include valid UUIDs, so that only these are ever passed down to the API calls. This prevents any malformed or malicious input from reaching the API layer.

What to change:

• In EvaluationCompare.tsx, filter and validate evaluationIds before passing them to fetchAllComparisonResults.
• Optionally, validate/sanitize again in the API service layer as a defense-in-depth measure (optional since we cannot change the backend and may only edit the frontend here).

Required additions:

• Utility function for UUID validation (import one or add a regex).
• Replace assignments to evaluationIdsArray and/or evaluationIds in EvaluationCompare.tsx with validated arrays.

---

General fix:
Sanitize and validate all user-controlled input before using it in constructing URLs for backend requests, especially where these control resource ids or path fragments.

Detailed fix:

• Before passing evaluationIds into fetchAllComparisonResults, ensure every item in the array is both present and matches the expected format (likely a UUID – as per the application context).
• In fetchAllEvaluationScenarios, strongly validate evaluationId before interpolating it into the URL.
• Ideally, filter the array to include only properly formatted UUIDs, or throw an error if any id is invalid.

Changes required:

• In web/ee/src/services/evaluations/api/index.ts, add a function to check that a string is a valid UUID (v4).
• In fetchAllComparisonResults, validate/filter all evaluationIds with this function before proceeding.
• Alternatively (or also), validate evaluationId at the start of fetchAllEvaluationScenarios and throw if invalid.
• Import a commonly-used UUID validation library, e.g., validator's isUUID, or implement a regex-based validator for UUID v4.

---

The best way to fix this SSRF vulnerability is to strictly validate scenarioId before using it as a path segment of an outgoing HTTP request. Rather than using the raw, untrusted query parameter, we should ensure that scenarioId is present in a list of known safe scenario IDs—these should be IDs obtained from the backend (e.g., as part of the evaluation run's permitted scenario IDs), not directly from user input. This means in SingleScenarioViewer, when resolving the scenario ID from router query, we must validate that it is among the permissible scenario IDs. Only allow it to propagate further if it's safe. Otherwise, select a default safe scenarioId (such as the first scenario ID returned from the backend).

The only changes needed are in the block of code from SingleScenarioViewer (file: web/ee/src/components/EvalRunDetails/components/SingleScenarioViewer/index.tsx), specifically in the way activeId and currentScenarioId are resolved from router query. We should update the code so that when determining activeId, we pick from the intersection of the loaded, valid scenarioIds.

No library imports are needed, as simple array inclusion suffices.

---

To mitigate this SSRF risk, restrict evaluationId to an expected format before using it in the request path, e.g., ensure it matches allowed characters (such as a MongoDB ObjectId or a UUID). The best way to do this is by validating the evaluationId string in fetchLoadEvaluation before including it in the axios request. If the value fails validation, throw an error or return a controlled response instead of making the HTTP request. The validation should be implemented right at the start of fetchLoadEvaluation to ensure early exit if the input is malicious or malformed.

To implement the change:

• In web/ee/src/services/human-evaluations/api/index.ts, update the fetchLoadEvaluation function (lines 84–96).
• Before making the axios request, validate that evaluationId matches the expected format, for example, checking if it's a 24-character hexadecimal string (for Mongo ObjectId), or matches a UUID regex if that's appropriate.
• If validation fails, log an error and return null (or throw, depending on desired error handling).
• Import or define a helper validation function if needed.

No code changes are needed in the component/page that reads router.query.evaluation_id.

---

To prevent the risk of user input being interpreted as a format string by console.error, the recommended fix is to use a static string as the format and pass the potentially-tainted value as a separate argument. Specifically, change from:

console.error(`Error fetching evaluation ${evaluationId}:`, error)

to:

console.error("Error fetching evaluation %s:", evaluationId, error)

This ensures that evaluationId will always be treated as a string and not as part of a format string, thus eliminating the opportunity for input-controlled format confusion. Only the specific console.error invocation on line 93 of web/ee/src/services/human-evaluations/api/index.ts needs to be updated. No extra imports or definitions are needed.

---

The best way to fix this SSRF risk is to validate and sanitize the evaluationTableId before using it in any URL path construction. Since the identifier is expected to represent an evaluation table, we should restrict its format to a known-safe pattern (e.g., MongoDB ObjectId, UUID, or some similar canonical form). In practical terms, this means:

• In fetchAllLoadEvaluationsScenarios (web/ee/src/services/human-evaluations/api/index.ts), only allow evaluationTableId if it matches a strict regular expression that describes the intended ID format (e.g., 24-character hex string for MongoDB ObjectId, or UUID).
• Throw/return an error or empty result if the ID is invalid, and optionally log the attempt.
• Add a helper function in the same file to validate the ID, and use it at the start of the exported method.

You only need to change code in web/ee/src/services/human-evaluations/api/index.ts, specifically around the fetchAllLoadEvaluationsScenarios function. The imported modules are standard/common, so you may add e.g. a regex or a helper validation inline there.

---

To correct the problem, you should validate (sanitize) all user-controlled input used in REST API paths, especially variables coming from the URL query (evaluationScenarioId in this case). The best fix is to ensure evaluationScenarioId matches only the allowed scenario IDs -- which are already present in-memory as part of evaluationScenarios (see usage in both components). This requires, before calling updateEvaluationScenario (or in that method), checking that evaluationScenarioId appears in evaluationScenarios.map(s => s.id). If it is not present, the operation should be aborted or an error raised.

The implementation can be made at the point where scenarioId (from useQueryParam) is passed to updateEvaluationScenarioData (and thus eventually becomes evaluationScenarioId). You should add validation in both web/ee/src/components/Evaluations/EvaluationCardView/index.tsx (where scenarioId is used) and/or in web/ee/src/components/EvaluationTable/SingleModelEvaluationTable.tsx (where id is used for updating scenarios), so that only known IDs from evaluationScenarios are accepted.

You will need to:

• Add a helper function that checks if a scenarioId is valid.
• Apply this check before passing scenarioId (or id) to SSRF sinks.
• Prevent requests with invalid or malicious IDs.

No new imports are strictly necessary.