Feature/ai test strategy

[mborodii-prog]

Summary

Implements a testing strategy for WranglesPY that keeps local development fast and deterministic while moving live provider coverage behind explicit integration markers.

This also pilots Microsoft’s Unit Test Writing Agent approach through the installed code-testing-agent skill by adding focused OpenAI Responses contract tests.

Changes

• Added docs/testing-strategy.md covering:

  • Unit Test Writing Agent evaluation
  • AI-generated test standards
  • Shared fixture approach
  • Mocks vs live-service tests
  • Live model creation/deletion cleanup
  • Unit test rollout plan
  • Integration test rollout plan

• Added/standardized pytest markers:

  • unit
  • contract
  • integration
  • live_ai
  • live_wrangleworks
  • live_s3
  • slow

• Added marker/config drift validation:

  • scripts/check_pytest_markers.py
  • tests/test_pytest_marker_config.py

• Added shared test fixtures in tests/conftest.py for:

  • fake JSON responses
  • malformed JSON responses
  • fake OpenAI Responses success responses
  • fake OpenAI rate-limit responses
  • fake model IDs/model creation cleanup support

• Converted local test selection to marker-based filtering:

  • removed pytest-local.ini --ignore entries
  • removed pytest-local.ini --deselect entries
  • marked live AI/WrangleWorks/S3 tests explicitly

• Recovered previously excluded tests by converting them to local/mock-based tests where practical.

• Updated tests that wrote to repo paths to use tmp_path.

• Added focused OpenAI Responses contract tests for:

  • terminal model states: refusal, incomplete response, API error
  • request payload shape and input formatting
  • caller payload immutability
  • retry behavior after invalid JSON
  • avoiding stale web-search sources across retries

Verification

Ran focused and policy checks:

python scripts\check_pytest_markers.py
python -m pytest -c pytest-local.ini tests\test_pytest_marker_config.py -q --tb=short --disable-warnings
python -m pytest -c pytest-local.ini tests\recipes\test_custom_functions.py -q --tb=short --disable-warnings
python -m pytest -c pytest-local.ini tests\recipes\wrangles\test_main.py -q --tb=short --disable-warnings
python -m pytest -c pytest-local.ini tests\test_openai_extract_ai.py -q
python -m pytest -c pytest-local.ini --collect-only -q
git diff --check