[TESTS]: Add test suite (unit + integration) and a CI test job

[araujof]

Summary

Smith currently ships no automated test suite for its Python backend. The tests/ directory contains only a README.md, dev dependencies for testing (pytest, pytest-asyncio, pytest-mock) are declared in pyproject.toml but unused, and CI (.github/workflows/ci.yml) runs lint / license / build-smoke / audit but no tests. The only end-to-end check is the make test OPA scorecard, which requires a running OPA server and is not wired into CI.

This issue tracks building out a proper unit + integration test suite and a CI job to run it on every PR.

Motivation

• The CLI backend (src/smith/) contains substantial deterministic logic (decomposition, label validation tiers, clustering, dedup, tool extraction, rego validation) with zero regression coverage.
• Refactors like the recent scripts/ → src/smith/ move and the uv migration are high-risk without tests.
• A green PR today only guarantees lint/format/headers pass — not that the pipelines still work.

Scope

1. Unit tests (pytest, no external services)

Target the deterministic, pure-ish units in the package map. Mock the LLM (MODEL_SONNET / OpenAI client), filesystem, and any subprocess calls. Priorities:

• policy_generation/ — extract_tools.py (MCP tool-def extraction), validate_policy.py (rego validation).
• test_generation/ — pure stages: decompose, grey_condition, variable_extraction, convert_test_case, extract_tool_args.py (parse/transform logic; mock LLM calls).
• test_case_evaluation/ — tier1_rules.py (pattern matching — fully deterministic, ideal first target), classify_guidance.py, and the report builder visualization/build_report.py. tier2_semantic.py / tier3_llm_judge.py with mocked embeddings/NLI/LLM.
• policy_agent/ — red_feedback/ DBSCAN clustering (deterministic given fixed input + CLUSTER_EPS/CLUSTER_MIN_SAMPLES), reduce_improve/ graph dedup.
• cli.py — arg parsing: smith --help and bare smith must work without a populated .env (regression guard for the documented "no eager BASE_URL work before argparse" rule).

2. Integration tests

• Formalize the existing make test OPA scorecard (src/smith/tests/integration/, curl against localhost:8181) as a CI-runnable job using the OPA Docker container, asserting the allow/disallow test-case expectations.
• Optionally an end-to-end CLI smoke run of a cheap pipeline stage against a fixture target agent under examples/, with the LLM mocked or stubbed.

3. Test infrastructure

• [tool.pytest.ini_options] in pyproject.toml (testpaths, markers e.g. unit / integration, asyncio mode).
• Shared fixtures in tests/conftest.py: temp .env / BASE_URL, mocked OpenAI client, sample guidance.txt / tool_definitions.json / system_vars.json, sample policy.rego.
• A make test-unit target (fast, no services) distinct from the existing services-dependent make test scorecard.
• Apache-2.0 SPDX headers on all new .py test files (make license-check must stay green).

4. CI job

• Add a test job to .github/workflows/ci.yml that runs unit tests on every PR (mirrors the Makefile, follows the existing job structure: setup-uv, python-version: "3.11", draft-PR skip guard).
• Add the OPA-backed integration scorecard as a separate job (spins up the OPA container, runs make test), gated appropriately.
• Update make ci and CLAUDE.md to reflect that tests are part of the gate.

Acceptance criteria

• make test-unit runs a non-trivial pytest suite locally with no external services and passes.
• Unit coverage exists for at least: tier1_rules, extract_tools, validate_policy, red-feedback clustering, and cli arg parsing.
• Integration scorecard runs in CI against an OPA container.
• A test job is green on this repo's CI for PRs.
• New test files carry SPDX headers and pass make ci.

Notes

• Depends on [REFACTOR]: Remove IBM-internal RITS env vars from examples; depend on Ollama/LiteLLM #11
• Testing deps already present in pyproject.toml [dev]: pytest, pytest-asyncio, pytest-mock.
• Keep LLM/network/subprocess out of unit tests — mock them so the suite is deterministic and fast.