Skip to content

CI runs no tests; 20% of package never imported by the test suite #3

Description

@thedataengineer

CI runs zero tests. .github/workflows/publish.yml is the only workflow and it gates version sync, python -m build + twine check, and a wheel import smoke test. No pytest job exists, there is no make test target, and pytest is not a declared dev dependency even though tests/test_worker_hub.py and tests/test_remote_pool_logs.py import it.

Coverage state of the CPU suite:

  • 3,473 lines (about 20% of the package) are never imported by any test: backends/torch.py (1,141 lines, the production training path), backends/mlx.py (1,125), cli.py (564), backends/verl.py, accel.py, bottleneck.py, charts.py, ascii.py, _cli_entry.py, _quiet.py.
  • The public SDK surface has zero assertions: models.py (load/finetune/generate/chat/save, _parse_reply), training.py resolvers (resolved_target_modules, resolved_warmup, resolved_eval_steps, resolve_total_steps) and TrainingRun.to_dict/from_dict (the serialization boundary the hub round-trips jobs through via serve.py:137), backends/__init__.py:37 select_backend, methods/base.py:50 validate_base (the quantization legality guard), capture.py:200 trajectories() (episode reconstruction), and data.py format detection plus as_chat/to_texts (the render path backends train on; silent misdetection corrupts training data with no error).
  • tests/gpu/test_cuda.py exits 0 when every test skips (main() at line 338 returns 1 if fails else 0), so on a non-CUDA box the GPU suite "passes" having run nothing.

Cheapest fixes first: the training.py resolvers, select_backend, cli.py _coerce/_apply_set/_load_config_file, and data.py _detect_format are pure functions with no heavy deps. A pytest job in CI plus tests for those closes most of the exposure in one pass.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions