[FEATURE] - Automatic Validation of Generated Tests with Pytest

[Ofido]

Is your feature request related to a problem?

The AI can sometimes generate tests that are syntactically incorrect or fail when run. Currently, the user has to manually run pytest after generation to discover these issues.

Describe the solution you'd like

Add a post-generation step that automatically validates the newly created tests. After a test is generated and saved to its file, the tool should automatically invoke pytest on that specific test file.

• If the test passes, the process continues.
• If the test fails, the tool could either discard the failing test, or move it to a separate file (e.g., test_newly_generated.py.failed) for the user to review manually.

This ensures that the test suite remains in a working state after the tool is run.

Describe alternatives you've considered

• Relying on the user to run tests manually (the current process).
• Integrating a linter to check for syntax errors, but this doesn't check for runtime correctness.

Additional context

This is from the New Ideas Backlog.