Skip to content

ci: unit and integration test jobs only run the root module — providers/*, pkg and tests/e2e are never tested #1751

Description

@cristim

Found while working #1740. Five of the six workspace modules have tests that CI never executes.

go.work declares six modules:

use ( . ./pkg ./providers/aws ./providers/azure ./providers/gcp ./tests/e2e )

Under a workspace, ./... expands to packages in the current module only. Both test steps in .github/workflows/ci.yml run bare from the repo root:

  • :99 — unit tests: go test -v -race -short -coverprofile=... ./...
  • :189 — integration tests: go test -v -race -tags=integration ... ./...

So pkg, providers/aws, providers/azure, providers/gcp and tests/e2e are compiled by nothing and asserted by nothing in CI. Their tests exist, pass locally, and have never gated a merge.

This repo already knows about this hazard and fixed it twice

The workflow documents the exact failure mode in its own comments, then does not apply it to the test steps:

# :308  Multi-module repo: each ./... only walks the current module,
#       Walk every module independently and fail on any HIGH/CRITICAL.
for mod in . pkg providers/aws providers/azure providers/gcp tests/e2e; do
  (cd "$mod" && govulncheck ./...)

# :329  Multi-module repo: each ./... only walks the current module so scanning
#       root alone silently misses pkg/ and providers/*. Mirror the govulncheck
#       per-module loop, collect per-module SARIF, then merge for the upload step.

govulncheck loops. The SARIF scanner loops. The tests do not. So this is not an unknown gap — it is a known one that the test steps were never updated for.

Concrete evidence it is hiding real defects

While repairing vacuous mock assertions for #1740, a type-checked sweep loaded only the root module and reported 26 sites. A filesystem-walking sweep found 4 more, all in providers/aws/services/savingsplans/client_test.go — a money path. Those assertions were structurally incapable of failing, and CI would never have run them either way.

Compounding it: golangci-lint on providers/aws is red on main with 276 findings, because the lint job also runs from the root only. So providers/* is currently neither tested nor linted by CI.

Fix

Mirror the existing govulncheck loop in both test steps. The pattern is already in this file and proven:

for mod in . pkg providers/aws providers/azure providers/gcp tests/e2e; do
  (cd "$mod" && go test -v -race -short ./...)
done

Coverage profile merging needs handling — the current single -coverprofile=coverage.out assumes one invocation. Collect per-module and merge, exactly as the SARIF step already does for its reports.

Expect this to go red

These modules have never been gated. Turning them on will surface pre-existing failures. Do not suppress them to get green — no -skip, no build tags excluding modules, no continue-on-error. If a module is genuinely too broken to gate today, land the loop with that module listed explicitly and a linked follow-up issue naming it, so the exclusion is visible rather than implicit in a glob.

Verification

  • Confirm each of the six modules appears in the job log with a non-zero test count. A module that runs zero tests silently is the current bug in a new form.
  • Break a test in providers/aws deliberately and confirm CI goes red. Before this fix, it will not.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions