refactor: dispatch config file loading by filename, then suffix

[RonnyPfannschmidt]

Supersedes #8358, #14707, #14723 and #14671.

Closes #14705
Closes #14716
Closes #11502

Updated after @bluetech's review. Two changes to what this PR is:

• The rootdir fix (Specifying configuration( -c config/pytest.ini) from non-project-root directory leads to local fixtures ignore #13246, Scopes of several files are merged when specifying a config using -c #9703, superseding determine_setup: use invocation_dir as rootdir when -c is given #14454 and Fix rootdir with explicit config and test paths #14579) moved out to fix: the config file's directory no longer decides the rootdir alone #14837, since it is a breaking change to documented behaviour and its release target is a separate question.
• The custom-TOML change is no longer breaking: such files now read [pytest] in addition to [tool.pytest]/[tool.pytest.ini_options], per @bluetech's suggestion, rather than instead of them.

Review comments applied: loaders in the dispatch tables are _load_* and the format helpers stay _parse_*; the tables are CONFIG_LOADERS_BY_NAME/CONFIG_LOADERS_BY_SUFFIX; the dispatch or is split into statements; the implementation-detail docstring is now a comment on the table.

Two commits: a behaviour-preserving refactor of config file dispatch, then the fixes it makes small. Review them separately — the first is intended to be a pure refactor.

Commit 1: refactor config file dispatch

Re-does the structural part of #8358 on top of current main. That PR bundled the refactor with the setup.cfg deprecation from #3523; the deprecation was abandoned (and reverted within the branch), and the refactor — which @nicoddemus had signed off as "Overall the changes look good" — went down with it. This PR carries only the structural change. No deprecation, no policy change, no behaviour change.

It can't be cherry-picked: the 2021 branch predates ConfigValue, native TOML mode, ini option aliases, pytest.toml/.pytest.toml, and ignored_config_files. So this is a re-implementation of the same design against today's code.

The problem

Knowledge about config files is currently spread across three places that must be kept in sync by hand:

1. load_config_dict_from_file dispatches on suffix (if filepath.suffix == ".ini" / elif ".cfg" / elif ".toml"), with filename checks nested inside those branches (if filepath.name in {"pytest.ini", ".pytest.ini"}, if filepath.name in ("pytest.toml", ".pytest.toml")).
2. locate_config re-declares the discovery order in a separate hardcoded config_names list.
3. Adding a format means touching both, in the right order, plus the nested name checks.

The name-vs-suffix distinction is real and load-bearing — pytest.ini is config even when empty, a random .ini is not; pyproject.toml reads [tool.pytest], pytest.toml reads [pytest] — but it is implicit and interleaved rather than expressed.

The change

Two tables, and dispatch that reads off them:

CONFIG_LOADERS_BY_NAME = {      # by name; also *is* the discovery order
    "pytest.toml":    _load_pytest_toml,
    ".pytest.toml":   _load_pytest_toml,
    "pytest.ini":     _load_pytest_ini,
    ".pytest.ini":    _load_pytest_ini,
    "pyproject.toml": _load_pyproject_toml,
    "tox.ini":        _load_ini_file,
    "setup.cfg":      _load_cfg_file,
}

CONFIG_LOADERS_BY_SUFFIX = {    # by suffix; for arbitrary files passed via -c
    ".ini":  _load_ini_file,
    ".cfg":  _load_cfg_file,
    ".toml": _load_pyproject_toml,   # what main does today; commit 2 changes this
}

def _get_config_loader(filepath):
    loader = CONFIG_LOADERS_BY_NAME.get(filepath.name)
    if loader is None:
        loader = CONFIG_LOADERS_BY_SUFFIX.get(filepath.suffix)
    return loader

locate_config now iterates CONFIG_LOADERS_BY_NAME directly instead of maintaining its own list, so discovery order and parsing can no longer drift apart.

Per-format semantics move into named functions — _load_pytest_ini, _load_ini_file, _load_cfg_file, _load_pytest_toml, _load_pyproject_toml — each with a docstring stating its rule, instead of living in nested conditionals inside one ~100-line function.

Behaviour preservation

This is intended to be a pure refactor.

• Full test suite passes unchanged; no test needed modifying.
• Additionally, I ran load_config_dict_from_file over a 38-case matrix (every supported name × present/absent/empty/malformed section, plus .txt and extension-less files) under main and under this branch, comparing returned values, ConfigValue.mode/origin, and raised exception types and messages. Output is byte-identical.

Two spots deliberately keep a pre-existing wart rather than quietly fixing it, so the diff stays behaviour-preserving:

• _load_cfg_file still hardcodes setup.cfg in the CFG_PYTEST_SECTION message even for other .cfg files.
• _load_pytest_toml/_load_pyproject_toml still assume the pytest key is a table; a scalar there raises AttributeError as before (hence the type: ignore comments). Worth hardening, separately.

In this commit CONFIG_LOADERS_BY_SUFFIX[".toml"] still points at _load_pyproject_toml, which is what main does today, so that the refactor decides nothing. Commit 2 changes it — that is #14705.

Why now

Three open PRs are each adding another special case to exactly the conditionals this removes, and two of them conflict:

• Fix custom TOML config files passed with -c #14707 (custom TOML via -c) is the name-vs-suffix confusion itself. On this branch it becomes a one-line change — point CONFIG_LOADERS_BY_SUFFIX[".toml"] at a [pytest]-aware loader — with no risk to pyproject.toml discovery. As currently written it inverts a nested check to filepath.name != "pyproject.toml", i.e. dispatch by negation.
• fix: validate explicit config file paths #14723 introduces SUPPORTED_CONFIG_FILE_EXTENSIONS — a fourth hardcoded table of file knowledge. Here that set is CONFIG_LOADERS_BY_SUFFIX.keys(), derived rather than declared.
• fix: validate explicit config file paths #14723 and Do not derive rootdir from a non-regular config file #14671 edit the same determine_setup branch in opposite directions: fix: validate explicit config file paths #14723 rejects any config path where is_file() is false, Do not derive rootdir from a non-regular config file #14671 exists to support --config-file=/dev/null (a character device, so is_file() is false). Whichever lands second silently undoes the other.

Landing this first turns those three into small, independent, non-conflicting changes — which is what commit 2 does.

---

Commit 2: the three fixes the refactor enables

fix: validate -c/--config-file and read [pytest] from custom TOML files

Consolidates three open reports that all bottom out in how an explicitly given config file is located and parsed:

Issue	Fix	Size
#14705 — a [pytest] table in a custom TOML file is ignored	the .toml suffix loader reads [pytest] as well as [tool.pytest]/[tool.pytest.ini_options]	one loader
#14716 — -c has two failure modes for invalid paths	determine_setup rejects nonexistent paths, directories, and regular files with no loader	supported extensions derived from CONFIG_LOADERS_BY_SUFFIX
#11502 — --config-file=/dev/null gives rootdir: /dev	a non-regular config file does not determine the rootdir; it falls back to the common ancestor	conditional on is_file()

The diagnoses come from #14707 (@DebadityaHait), #14723 (@wanxiankai) and #14671 (@apoorvdarshan) — this PR supersedes all three, with smaller implementations because the dispatch tables carry the file knowledge.

Notably #14723 and #14671 are mutually exclusive as written: #14723 rejects any config path where is_file() is false, while #14671 exists to support --config-file=/dev/null, a character device for which is_file() is false. Splitting the check into exists (error if not) and is a regular file (its directory may determine the rootdir) satisfies both.

On #14705 — both table styles, not a swap

The first version of this commit made custom TOML files read [pytest] instead of the [tool.pytest] tables. @bluetech pointed out that this breaks working setups, and he is right: -c custom.toml with [tool.pytest.ini_options] has worked since TOML support landed. It now accepts both, and writing both styles into one file is a UsageError. pyproject.toml itself is untouched — name dispatch wins over suffix dispatch, so it keeps [tool.pytest]-only semantics.

On #11502 — parsing and rootdir are separate questions

@bluetech also flagged that falling back to an empty config for a non-regular file is undesirable, and suggested erroring with /dev/null special-cased. The version here reaches the same place from the other side, without a platform-specific name check:

• a loader runs whenever one matches the name or suffix, whatever the file type — so a config file that happens to be a fifo is parsed, where the earlier version silently produced an empty config for it;
• a regular file with no loader is a UsageError;
• a path that is neither a regular file nor in any known format cannot hold configuration at all, and means "load no configuration" — that is /dev/null, and NUL on Windows, with no name matching needed;
• only the rootdir question consults is_file(), because a character device says nothing about where the project lives.

Happy to switch to an explicit os.devnull check if you prefer it stated rather than derived.

⚠️ Breaking

Passing a file pytest has no loader for to -c/--config-file is now a UsageError instead of being silently ignored. pytest's own #14683 regression test did this with a conftest.py; it now uses a real config file, and since it passes --rootdir explicitly the config file was incidental to what it covers. (@bluetech: "technically a breaking change but clearly a bug/broken setup, I think it's OK to break this".)

Verification

Full suite green (4344 passed, 47 skipped, 13 xfailed, 7 xpassed — the xpasses are pre-existing, #11603/#10042). Each reported reproducer checked by hand as well: the custom-TOML case is confirmed by python_files from the custom file actually taking effect, and the /dev/null case by the rootdir landing on the project directory with no PytestCacheWarning.

---

Follow-up: #14837

The rootdir fix that was commit 3 here now lives in #14837, based on this branch. It is breaking against documented behaviour, so its release target needs deciding separately.

[bluetech]

Review of the first commit:

The refactor looks like a nice improvement to me. I left a few suggestions but feel free to cherry-pick it with my approve.

[bluetech]

Suggestion: split the or to separate statements, easier this way in coverage and debugging.

[bluetech]

I haven't reviewed the 2nd and 3rd commits yet, but:

Regarding the 2nd commit's description:

• "custom TOML reads [tool.pytest] instead of [pytest]" - this seems like a breaking change that would break people's setups. I don't think we can do it :( I think what we maybe can do is to add a variant that supports both (i.e. tool.pytest, tool.pytest.ini_config, pytest).

• -c has two failure modes for invalid paths - technically a breaking change but clearly a bug/broken setup, I think it's OK to break this.

• non-regular config file → empty config - I think maybe error on this case? The fallback to empty seems undesirable to me. If we want to support the /dev/null use case (I guess to force no-config if a config exists?), maybe we can special case /dev/null?

Regarding 3rd commit's description: I haven't considered the change, but it is a breaking change, I'm not sure we can just change it. Maybe if we assume the next release will be major, is this the intention of the change?

[bluetech]

Thanks, LGTM!

I think it would be nice to split the second commit to 2 or 3 separate commits (one per fix), but I won't torture you with this :)