Skip to content

Recreate venv for each build + install from pylock.toml when available - #345

Open
hugovk wants to merge 6 commits into
python:mainfrom
hugovk:clear-and-pylock
Open

Recreate venv for each build + install from pylock.toml when available#345
hugovk wants to merge 6 commits into
python:mainfrom
hugovk:clear-and-pylock

Conversation

@hugovk

@hugovk hugovk commented Jul 28, 2026

Copy link
Copy Markdown
Member

Re: python/cpython#153227 (comment)

python/cpython#153228 switched to use a dev version of pygments from GitHub, but it's not being picked up on the doc server.

This is because:

  • We re-use a venv per Python version on the docs server
  • pip install --upgrade -rrequirements.txt still sees pygments with the same version number in the venv, so doesn't need to upgrade it

This PR does two things:

  • Clear the venv for each build. We still have a pip cache on the server, so it's not downloading everything every time. This means we'll always a fresh venv matching what we expect.
  • We now have pylock.toml on newer branches (3.15+), which locks the actual required version, so this explicitly pin the dev version from GitHub. Let's use pylock.toml when when available.

Comment thread build_docs.py
venv_path,
symlinks=os.name != "nt",
with_pip=True,
clear=True,

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ensures a fresh venv.

Comment thread build_docs.py
symlinks=os.name != "nt",
with_pip=True,
clear=True,
upgrade_deps=True,

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upgrade pip, because otherwise we use the bundled pip, and we need at least 25.1 for pylock.toml support.

Could do this just for the pylock.toml part, but might as well update for both.

@ZeroIntensity ZeroIntensity left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm definitely not the most qualified reviewer here, but nothing jumps out at me as wrong (and I really want the highlighting 😃)

@hugovk

hugovk commented Jul 29, 2026

Copy link
Copy Markdown
Member Author

You'll be happy to hear that Discourse (which uses highlight.js) will also get lazy highlighting at some point :)

highlightjs/highlight.js#4450

@ZeroIntensity

Copy link
Copy Markdown
Member

Hugo, do you realize I was the one who implemented that 😅: highlightjs/highlight.js#4398

But yeah, I think GitHub uses hljs too. Hopefully we see adoption before the 3.15 release.

Comment thread build_docs.py Outdated
Comment thread build_docs.py Outdated

So we can reuse them from builds to builds, while they contain
different Sphinx versions.
The venv is recreated from scratch for every build: pip considers

@StanFromIreland StanFromIreland Jul 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll now be build[ing]_venv[s] for every version, but also every language. That's quite a lot considering the amount of translations we have. I'd rather avoid the rebuilds, I think running it once per version will keep the venv fresh enough?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good idea, updated to create each venv at most once per run.

It also now deletes the venvs at the end of the run, we don't need to keep those hanging around, especially as they're recreated at the start of a run.

How does it look now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants