Adapt packaging and image build for free-threaded containers - #1261
Draft
mxsrc wants to merge 8 commits into
Draft
Adapt packaging and image build for free-threaded containers#1261mxsrc wants to merge 8 commits into
mxsrc wants to merge 8 commits into
Conversation
The image was built from a separately published `simplyblock:base_image` tag refreshed by its own cron workflow, which meant the product image's OS packages were whatever that tag last resolved to and a rebuild could not pick up an errata fix on its own. Fold it into a single multi-stage Dockerfile instead, and bound the OS-package layers with a CACHE_KEY build arg holding the ISO year and week. Every workflow and build_image.sh pass the same weekly value, so the layers are rebuilt from scratch once a week and an upstream security fix reaches the image within seven days. security.yml deliberately passes that same key rather than a fresh one: the scan has to report on the packages actually shipped, and rebuilding against fresher packages would let it go green while the published image still carries the vulnerability. BuildKit is now required.
setup.py computed the distribution name and version by parsing simplyblock_core/env_var at build time, declared its dependencies by reading requirements.txt, shipped data files through a hand-rolled gen_data_files() walk, and ran a post-install hook to configure argcomplete. Only the last of those was ever load-bearing, and it never ran for a wheel — post-install hooks are a setup.py-install feature that PEP 517 backends do not have. Replace it with a static [project] table on the setuptools backend: - `sbctl` is now the name unconditionally. PEP 621 requires static entry points, so `sbctl` and `sbcli-dev` are both declared; the latter is the value SIMPLY_BLOCK_COMMAND_NAME carries and constants.SIMPLY_BLOCK_CLI_NAME resolves for the services that shell out to the CLI. release.yml's sed of env_var no longer affects the distribution name. - The version stays in env_var, which release.yml rewrites and constants.py reads at runtime for `sbctl --version`. simplyblock_core/_version.py is the single reader and is what [tool.setuptools.dynamic] imports at build time, so it deliberately uses only the standard library — constants.py is 400+ lines and actively edited, and importing it at build time would couple the build to it never growing a third-party import. - [tool.setuptools.package-data] is not optional. Without it setuptools ships only *.py and drops env_var, scripts/**, templates/** and the dashboards silently; everything resolves them relative to __file__ at runtime. - requirements.txt, test-requirements.txt and type-requirements.txt become [project.dependencies] and PEP 735 dependency groups. pytest and mock move out of the runtime set — only simplyblock_core/test/ imports them, so they were being installed into the production image for nothing. e2e/requirements.txt is a separate file and is untouched. Add uv.lock so the dependency set is pinned and a resolution change is a reviewable diff instead of whatever the next build happens to get. The lock is universal across the supported range: it resolves and installs on both 3.9, the floor requires-python promises for the published wheel, and the free-threaded 3.14 the container image runs. Shell completion now has to be registered explicitly; README documents it.
The background services are started by file path — `python3
simplyblock_core/services/<name>.py`, run from the source tree. That string is
baked into artifacts this repository does not own (the Helm charts and CSI chart
under kubernetes/, the operator's generated init script) and into Docker Swarm
services created by an older control plane, which keep their original command
across an image upgrade because cluster_ops.py only calls
service.update(image=...). So the paths have to keep working, but they force the
image to keep the source tree at /app and make the deployment contract "run this
file at this relative path with this cwd".
Add simplyblock_core/services/__main__.py, exposed as `simplyblock-service`, so
a service can be named instead:
simplyblock-service snapshot-monitor
python3 simplyblock_core/services/snapshot_monitor.py # still works
It dispatches through runpy.run_module(..., run_name="__main__") rather than
importing and calling main(). That reproduces the semantics of running the file
directly, which matters for spdk_http_proxy_server: its module body builds the
shared state ServerHandler and rpc_call read as globals (TIMEOUT,
MAX_CONCURRENT_SPDK, spdk_semaphore, rpc_sock) and its get_env_var(...,
is_required=True) calls raise at import time, so it is left as it is — wrapping
it would change binding semantics on the storage-node data path.
Also give the other ten services that lacked one a main() and a __main__ guard,
so each has one obvious entry point and can be imported without executing. Six
already had a guard whose body was inline; four had bare top-level loops.
hub_controller_manager and replication_final_step are libraries with no entry
point and nothing invokes them as a command, so the dispatcher excludes them
rather than advertising names that do nothing.
The new tests pin both halves of the contract: every name the dispatcher
advertises resolves to a module, and every module a consumer can invoke by path
is reachable by name, so renaming one side cannot silently drop the other.
The application was installed into the platform interpreter with pip, which tied its Python version to whatever UBI ships. In practice that meant 3.12: `dnf install python3.13` adds an alternative interpreter but leaves `python3` alone, so every `command: python3 ...` and every pip install targeted 3.12 while CI tested another version entirely. Build /opt/venv with uv from a managed free-threaded 3.14 interpreter instead, so the version is a property of the image rather than of the base OS. The GIL is off; PYTHON_GIL=1 in the container environment re-enables it on the same image if a free-threading regression turns up in the field. The one non-obvious part is sudo. Several entry points start as `sudo -E python3 ...` — the storage-node API (storage_node_ops.py), the SPDK proxy (storage_node/docker.py and templates/storage_deploy_spdk.yaml.j2) and node_configure.py, invoked from the operator's init script in kubernetes/operator/internal/utils/storage_nodeset_ds.go. UBI's /etc/sudoers sets `Defaults secure_path = /sbin:/bin:/usr/sbin:/usr/bin`, which replaces PATH *even with -E*, so all of them would resolve /bin/python3 and miss the venv entirely. Measured on ubi10/ubi:10.2, every sudo form picks the system interpreter without the drop-in below and the venv with it, while `sudo dnf` and absolute-path invocations are unaffected. This is why an earlier attempt at a venv image was abandoned rather than fixed; the drop-in is what makes it viable, so it is documented in docker/AGENTS.md rather than left as a bare line. Two consequences: - The install stays editable and /app keeps the sources, because the hardcoded `python3 simplyblock_core/services/<name>.py` commands in the Helm charts and in already-deployed Swarm services resolve only because /app *is* the package. This is scaffolding with a defined exit, not the intended end state. - No compiler toolchain and no python3-pip. Every dependency resolves to a cp314t wheel and foundationdb is a pure-Python sdist, so nothing compiles; a future dependency without a free-threaded wheel now fails the build loudly instead of quietly building from source. uv creates the venv without pip, setuptools or wheel, so the previous `pip uninstall` step and the vendored pip bundle the image scan flagged are both gone.
The image now runs free-threaded 3.14 while the suite ran only on 3.9, so the
version actually shipped was the one nothing tested. Give each tier a py314t
twin and run both in CI. 3.9 stays the default because requires-python promises
it for the published wheel.
The twins use tox's `base` so a tier is defined in exactly one place: adding
setenv, deps or commands to [testenv:integration] reaches its twin with no edit,
where re-listing settings individually (commands = {[testenv:integration]
commands}) would silently not. `testenv` has to stay first in that list — `base`
replaces the inheritance chain rather than extending it, so naming only the tier
drops [testenv] and the twin quietly runs with package = sdist and no test
dependencies.
tox also stops using skipsdist, so the runs exercise the same metadata, entry
points and package data the wheel ships rather than the bare source tree. The
types env installs the test group as well, because mypy checks simplyblock_core
and that includes simplyblock_core/test/.
passenv is deliberately left minimal so a machine-specific value cannot leak in
and change a run's result; tests/AGENTS.md documents passing local
infrastructure overrides with `-x` instead, which is what a non-default container
socket needs.
`python setup.py sdist` no longer works now that packaging is PEP 621, and it only ever produced an sdist — anyone installing sbctl had to build it locally. `uv build` produces both an sdist and a wheel. The env_var sed steps stay: SIMPLY_BLOCK_VERSION still drives the version through simplyblock_core/_version.py, and SIMPLY_BLOCK_COMMAND_NAME is still read at runtime. What it no longer does is determine the distribution name, which is now static.
The container-image section had grown to about half of the root AGENTS.md, and most of it is only relevant when editing the Dockerfile. Move it to docker/AGENTS.md, following the per-directory convention already used by simplyblock_cli, simplyblock_core, simplyblock_web and tests, with the matching docker/CLAUDE.md symlink. The root file keeps a short pointer. docker/AGENTS.md also gains the build and inspection commands, a note on what .dockerignore keeps out of the build context and why, and the measured sudo secure_path table — that one exists so the drop-in is not removed as noise, since without it the `sudo -E python3` entry points silently run the platform interpreter with none of the application's dependencies. The rest documents the packaging and test changes: uv commands in place of pip, the 3.9-to-3.14t support range and why both ends are tested, the py314t tox twins and using `-x` for local infrastructure overrides, and explicit argcomplete registration now that the setup.py post-install hook is gone.
effaac5a1f moved the service body into main(), which turned the
`logger = setup_logger()` assignment from a module-level binding into a
local one. Six functions in the file read `logger` as a global, so the
service died on the first statement of main():
NameError: name 'logger' is not defined
It failed identically through both invocation forms, and was the only
service to fail for a non-database reason.
Declare the logger at module level, as every other service in the package
does, and leave setup_logger() to attach the stdout and GELF handlers.
Claude-Session: https://claude.ai/code/session_011XQSJf6dVjTnYyMtmUhZmk
mxsrc
marked this pull request as draft
August 21, 2026 13:45
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The GIL is a bottleneck for many processes due to our deserialization-heavy database. This necessitates horizontal scaling of the API containers to achieve multiple processes. A solution is using free-threaded Python that has become a non-experimental feature in 3.14 and is confirmed experimentally to enable API containers utilize their threads for CPU-heavy tasks, enabling vertical scaling. To this end, a few changes are included:
setup.py-based packages. This is converted topyproject.toml, retaining compatibility with all features, except for the adaptive package name.This changeset enables enables: