Repository navigation
Conversation
Bzlmod supports `pip.parse(uv_lock = ...)` to install PyPI dependencies from a universal `uv.lock` file, but `WORKSPACE` setups still required platform- or version-specific `requirements.txt` files for `pip_parse`. Maintaining separate lockfile formats across `MODULE.bazel` and `WORKSPACE` duplicated lockfile generation targets and complicated dependency updates. Add the `uv_lock` macro to `//python:pip.bzl` for `WORKSPACE` builds. It decodes `uv.lock` with `@toml.bzl`, converts it to requirements format via `uv_lock_to_requirements`, and renders the PyPI hub (and multi-version spoke repositories when `python_interpreter_target` is a dict) using the shared `pip_repository` rendering logic. Migrate repository, example, and test lockfiles that used `uv` from `requirements.txt` to unified `uv.lock` files.
|
AI Disclosure
This is an automatically generated agent comment. |
|
@aignas is core of this implementation for a low-effort workspace macro to handle uv.lock look promising? Just look at the files under python/private/pypi, the uv_lock.bzl and uv_lock_repo.bzl. (everything else is migration noise). The basic impl is to convert the uv.lock to requirements using the uv_lock_to_requirements helper. It basically just converts uv.lock to requirements, then feeds that into the pip rules and pip hub rules. With just a year left for workspace, I didn't want to put a ton of effort into e.g. full API compatibility with pip_parse. But enough that we can move our own workspace off it |
…on PR Propagate transitive dependency-edge markers in uv_lock_to_requirements and move the repository, example, and test lockfile migrations into a stacked child PR.
rickeylev
changed the title
aignas
left a comment
aignas
left a comment
There was a problem hiding this comment.
Having this in workspace could also mean that we could at some point implement an automatic cycle dependency algorythm for uv.lock asd friends.
| Returns: | ||
| A dict of {package_name: [extra1, extra2, ...]} for packages with extras. | ||
| """ | ||
| extras_map = {} |
There was a problem hiding this comment.
There is already similar code in parse_requirements. Maybe these could be refactored into one? The only thing that is different is that in workspace case we are dropping the urls to wheels that we can use bazel downloader to download.
That said I think the proposed approach is good enough, if somebody wants, they can spend tokens to backport some of the pip.parse machinery to make it work in workspace.
There was a problem hiding this comment.
Good call — unified WORKSPACE (extract_url_srcs = False) directly into _parse_uv_lock_json in parse_requirements.bzl and removed uv_lock_to_requirements.bzl in ac7eb904. PTAL!
This is an automatically generated agent comment.
There was a problem hiding this comment.
Hm, I had it go do this and it went a bit...weird. I seems to have more-or-less copy/pasted uv_lock_to_requirements.bzl into parse_requirements.bzl and the body of it become basically two entirely different implementations. This appears to have all been just so that it could pass along requirements.txt style info to pip underneath.
I've asked it to just skip that and jump straight to using the rctx.download() stuff instead. Lets see if that goes better.
Teach `_parse_uv_lock_json` in `parse_requirements.bzl` to handle `extract_url_srcs = False` directly so `uv_lock_repo` and `render_pip_repository` pass `uv.lock` through `parse_requirements` without round-tripping through a synthetic `requirements.txt` string. Delete the now-unused `uv_lock_to_requirements.bzl` module and cover the `WORKSPACE` `extract_url_srcs = False` behavior in `parse_requirements_tests.bzl`.
Extract wheel URLs, filenames, and SRI integrity digests in render_pip_repository when uv_lock is set so whl_library fetches prebuilt wheels directly with Bazel's downloader instead of invoking pip, falling back to pip only for sdists and git sources.
…uv_lock Resolve uv.lock wheel candidates across all target platforms rather than only the host platform so generated requirements.bzl files are byte-for-byte identical across Linux, macOS, and Windows while still downloading prebuilt wheels via Bazel's downloader.
…uv_lock Filter out packages that have no matching source for the repository host platform (unless download_only = True) so all_requirements never includes platform-specific packages on unsupported hosts.
…WORKSPACE uv_lock Replace the pip_repository / platform_whls adapter with a self-contained uv_lock_repo implementation that mirrors Bzlmod's multi-platform hub model: each wheel distribution gets its own whl_library repository downloaded via Bazel's repository downloader, and package subpackages use pkg_aliases with platform (and optional Python version) config_settings.
…p_template in uv_lock_repo
Ensure uv_lock_repo sets dep_template to @<hub>//{name}:{target}, passes
group_name/group_deps for experimental_requirement_cycles, uses
python_version_major_minor in multi-version config_settings, and avoids
recursive render.dict calls.
Ensure pure-Python py3-none-any wheels match directly in _platforms_for_python_version so wheel-only packages (such as pytest-bazel) resolve cleanly and pure-Python packages download prebuilt wheels directly instead of falling back to sdist.
Add a
uv_lockmacro anduv_lock_reporepository rule under//python:pip.bzlfor consuming resolved
uv.lockfiles directly inWORKSPACEbuilds.uv_lock_repoparsesuv.lockin pure Starlark viaparse_requirementsandrenders a multi-platform PyPI hub repository backed by per-wheel
whl_libraryrepositories (matching Bzlmod's
pip.parsearchitecture):.whldistributions directly via Bazel's repository cache(
filename,urls,sha256) with zeropipdownload invocations.whl_library(requirement = ...)with--hash=sha256:...integrity verification for source distributions and VCS dependencies.
pkg_aliaseshub targets selecting across target platformconfig_settingrules (and across Python versions whenpython_interpreter_targetis passed as a version dictionary).whl_library.bzl,whl_archive.bzl,whl_file_repo.bzl, andpip_repository.bzlcompletely unmodified.