setting environment variables which are accessed at import time

[wimglenn]

I think it's common particularly when deploying services in containers to get some configuration from environment, at the top of a module, like this:

# mymod.py
FOO_URL = os.environ.get("FOO_URL")

In this use case it's usually too late to use monkeypatch.setenv because the mymod.py was already imported before tests enter fixtures. So we can use monkeypatch.setattr instead, but that has usability issues - for example, it's very easy to accidentally leak an "unpatched" value around during test (if some other module has from mymod import FOO_URL then you need to patch there too). And if a config was actually used by something else at the module level scope, then even monkeypatch.setattr is getting in too late.

My workaround for this is to rely on the fact that conftest.py gets imported first:

# conftest.py
import os

os.environ["FOO_URL"] = "https://test-foo-url.org"

But it seems a bit off-label usage and I'm not sure if it's actually documented/guaranteed that putting this in conftest.py allows you to "get in first" to swap in values for testing.

I wonder if this is something that we could do in pytest config file directly? There's a plugin pytest-env which is small and simple and does similar:

[pytest]
env = 
    FOO_URL = https://test-foo-url.org

A concern though is that there's some danger of "production" config values being used if the tests were accidentally run without the plugin installed. Could a feature like this go into pytest directly?

[nicoddemus]

Hi @wimglenn!

Integrating pytest-env into the core is certainly possible, but is the danger of the missing plugin installation leaking production data likely? Seems a bit far fetched to me that would happen, you would need to be running the tests into the production environment, which I believe is not something usually done.

[RonnyPfannschmidt]

i would argue that its not pytests job to manage expectations around when people set up module globals as configuration objects
if they want such a setup, they manage the fallout

[The-Compiler]

@wimglenn @nicoddemus FWIW we already have the required_plugins setting for that.

[wimglenn]

@nicoddemus It's not so much the danger of leaking production data, but of accidentally interacting with the real versions of external services instead of their stubs/mocks. Using pytest-socket plugin can also mitigate that to some extent.

@The-Compiler The required_plugins config worked perfectly, aborting the test session early if pytest-env was not installed. That's a good answer, thanks!

I am still curious to know if the behavior of conftest.py getting in first is actually safe to rely upon, or just implementation detail of pytest. I've used this once to get a mock in before the library import, by injecting it early into sys.modules.

[nicoddemus]

I am still curious to know if the behavior of conftest.py getting in first is actually safe to rely upon, or just implementation detail of pytest.

Yeah, it should be considered an implementation detail. While this is true now (and the foreseeable future), this could change without notice.

[wimglenn]

The concern of interacting with default or externally configured services is mitigated by using the required_plugins setting mentioned in the comments.

[pytest]
required_plugins = pytest-env
env = 
    FOO_URL = https://test-foo-url.org

That makes the test session abort early if pytest-env wasn't installed, rather than just print a warning about unrecognized config options.

[sauravsingla]

This discussion highlights an interesting testing problem beyond simply setting an environment variable: knowing whether the meaningful configuration states are actually exercised by the test suite.

I’ve been exploring this from a static-analysis angle with an open-source tool called ConfigReach:

https://github.com/sauravsingla/ConfigReach

It inventories environment-variable reads such as os.getenv / os.environ, looks for corresponding test evidence such as monkeypatch.setenv, and reports configuration inputs or known values that appear untested.

The import-time case discussed here is particularly interesting because static test evidence does not necessarily mean the configuration affected the intended execution path.

I’m curious whether pytest users would find it useful to distinguish between:

an environment variable being set somewhere in tests, and
evidence that each important value/state actually reaches the configuration-dependent code.

That distinction is one of the problems I’m trying to model deterministically.