Skip to content

test: adopt -Werror in unit tests - #684

Open
minulo wants to merge 6 commits into
mainfrom
chore/pytest-warnings-as-errors
Open

minulo wants to merge 6 commits into
mainfrom
chore/pytest-warnings-as-errors

Conversation

@minulo

@minulo minulo commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Implemented test enhancement per this post.

What this PR does

Runs the unit tests of every charm in the monorepo with pytest's filterwarnings = ["error"], so a deprecation, a leaked file handle or a misused library fails the test instead of scrolling past in the log. canonical/indico-operator#776 is the same change on indico.

Configuration

  • haproxy-operator, haproxy-route-policy-operator, haproxy-spoe-auth-operator and haproxy-ddos-protection-configurator: pyproject.toml gains filterwarnings with "error" first. Each tox.toml integration env passes --override-ini=filterwarnings=, so warnings from the deployment tooling cannot break those runs.
  • haproxy-route-policy-operator ignores one third-party warning by message: JujuVersion.from_environ() is deprecated, raised by the vendored data_platform_libs lib, which still makes that call at its newest patch.
  • The three legacy Harness modules in haproxy-operator/tests/unit/legacy ignore ops' Harness is deprecated warning through a module-level pytestmark. Harness used anywhere else still fails.
  • haproxy-route-policy runs Django's test runner rather than pytest, so its unit and unit_auth envs run under python -W error instead.

Fixes for warnings that were ours

  • haproxy-operator/tests/unit/conftest.py: the TLS fixtures called the deprecated generate_private_key(), generate_csr(), generate_ca() and generate_certificate() helpers. They now use PrivateKey.generate(), CertificateRequestAttributes.generate_csr(), Certificate.generate_self_signed_ca() and Certificate.generate().
  • haproxy-operator/tests/unit/legacy/test_haproxy_route_lib.py: the mock_requirer_charm fixture never cleaned up its Harness. Its temporary directory was reclaimed by the garbage collector with a ResourceWarning, which then failed whichever test happened to be running. The fixture now yields the harness and calls cleanup().
  • haproxy-route-policy test settings: WhiteNoise warned No directory at: .../static/ because tests do not run collectstatic. Both test settings now set WHITENOISE_AUTOREFRESH = True.

Why we need it

Deprecation and resource warnings are easy to miss in CI logs. Promoting them to errors surfaces them when they are cheap to fix rather than when the API is removed. Turning it on here already found a real resource leak in the legacy test fixtures.

Test plan

Project Before After
haproxy-operator 351 passed, 163 warnings 351 passed, no warnings, run three times
haproxy-route-policy-operator 31 passed, 5 warnings 31 passed, no warnings
haproxy-spoe-auth-operator 1 passed 1 passed
haproxy-ddos-protection-configurator 6 passed 6 passed
haproxy-route-policy 57 tests OK, 7 auth tests OK the same, under -W error
  • The full default tox run passes in all five projects: lint, unit, static and coverage-report.
  • Verified the filter bites in each pytest project: a throwaway test that emits a DeprecationWarning fails, and one that leaks a file handle fails with ResourceWarning. The probe file was not committed.
  • Integration tests were not run locally; they are deliberately excluded from the filter and CI exercises them.

Review focus

  • The CA fixture now signs with the key it returns. Before, generate_ca() was given a throwaway key while the fixture returned a different one, so the issued certificate's signature did not match the CA's key. Tests pass either way, but the new pairing is the consistent one.
  • Pre-existing and not changed here: haproxy-route-policy/tox.toml lists unit-auth in envlist but defines the env as unit_auth, so the auth tests do not run by default or in CI. They pass when run directly with tox -e unit_auth.
  • The third-party ignore matches on message only, like indico-operator#776. Pinning it to the library's module, ...:DeprecationWarning:charms.data_platform_libs, would keep the same deprecated call in this charm's own code failing. That form also passes the suite.
  • No breaking changes, no new dependencies, no workflow changes. Labelled no-release-note since this changes only tests and test configuration.

Checklist

  • I followed the contributing guide
  • I added or updated the documentation (if applicable) (n/a)
  • I updated docs/changelog.md with user-relevant changes (n/a: tests only)
  • I added a change artifact for user-relevant changes in docs/release-notes/artifacts. If no change artifact is necessary, I tagged the PR with the label no-release-note.
  • I used AI to assist with preparing this PR
  • I added or updated tests as needed (unit and integration)

This PR was assisted by an LLM.

Run the unit suites of the four charms with filterwarnings = ["error"]
so a deprecation, a leaked file handle or a misused library fails the
test instead of scrolling past in the log. Charm Tech is making this
the Charmcraft default and asked existing charms to opt in. Each
integration env opts out with --override-ini=filterwarnings=.

Turning it on surfaced warnings that are fixed here:
- haproxy-operator's TLS test fixtures called the deprecated
  generate_private_key(), generate_csr(), generate_ca() and
  generate_certificate() helpers; they now use the replacements the
  warnings name. The CA is now signed with the key the fixture returns.
- The legacy mock_requirer_charm fixture never cleaned up its Harness,
  so its temporary directory was reclaimed by the garbage collector
  with a ResourceWarning that failed whichever test was running.
- haproxy-route-policy's WhiteNoise middleware warned about the missing
  static directory in tests; the test settings now enable autorefresh.

Left as narrow ignores: ops' Harness deprecation, scoped to the three
legacy Harness test modules, and JujuVersion.from_environ() from the
vendored data_platform_libs lib in haproxy-route-policy-operator.

haproxy-route-policy runs Django's test runner rather than pytest, so
its unit envs use python -W error instead.
@minulo
minulo requested a review from a team as a code owner October 13, 2026 06:44
@minulo minulo added the no-release-note This PR does not require a change artifact label Oct 13, 2026
Comment on lines +117 to +121
# Turn warnings into errors so deprecations and resource leaks fail the unit
# suite rather than accumulating unnoticed. The charm's own code and
# tests raise none. The integration env opts
# out (see tox.toml) because any warnings there come from the deployment
# tooling and are not this charm's to fix.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot shorten this comment to just "promote warnings to test failures". Apply it every other places

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated all four pytest configuration comments to “promote warnings to test failures” in commit d07fabd.

Co-authored-by: Thanhphan1147 <42444001+Thanhphan1147@users.noreply.github.com>
Comment on lines +132 to +134
# pyproject.toml promotes warnings to errors for the unit tests; the
# integration suite runs against real deployment tooling whose warnings
# are not this charm's to fix, so it keeps pytest's default handling.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot shorten this to "Don't promote warnings for integration tests"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the integration warning comments in all four tox.toml files in commit 2a6d06c.

Co-authored-by: Thanhphan1147 <42444001+Thanhphan1147@users.noreply.github.com>
@Thanhphan1147

Copy link
Copy Markdown
Collaborator

LGTM, thanks! I asked copilot to shorten some of the comments to be more concise.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Libraries: Out of sync no-release-note This PR does not require a change artifact

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants