Repository navigation
[Bug] Python package stubs crash when the same command runs in parallel #367
Description
Running the same pkgx Python command in parallel crashes. The bin/ stub rewrites the shared virtualenv on every invocation, with no lock, and then executes the real script.
This problem showed up with github.com/0xabu/pdfannots. The stub is ~/.pkgx/github.com/0xabu/pdfannots/v0.5.0/bin/pdfannots.
On every run the stub:
- Rewrites
venv/pyvenv.cfg. - Opens every file in
venv/bin/withrb+and rewrites shebangs that end inpython. - Deletes
venv/bin/pythonand creates the symlink again. - Calls
os.execvonvenv/bin/pdfannots.
A single process is fine, because the rewrite finishes before the next start. With several processes, they mutate those same files together. Typical failures:
OSError: [Errno 26] Text file busy
File ".../bin/pdfannots", line 50, in <module>
os.execv(arg0, args)
FileExistsError: [Errno 17] File exists: '.../python.org/v3.15.0/bin/python' -> '.../venv/bin/python'
File ".../bin/pdfannots", line 43, in <module>
os.symlink(python_path, python_symlink)
FileNotFoundError: [Errno 2] No such file or directory: '.../venv/bin/python'
File ".../bin/pdfannots", line 41, in <module>
os.remove(python_symlink)
Reproduction:
seq 8 | xargs -P 8 -I{} pkgx pdfannots --versionOne process at a time prints pdfannots 0.5. Eight at a time intermittently hit the errors above. The same pattern should affect any pkgx Python package whose bin/ entry uses this stub.
Expected behavior: concurrent invocations of an already installed package should not rewrite the virtualenv. The stub can prepare the venv once, or skip the rewrite when pyvenv.cfg and venv/bin/python already point at the current interpreter.
Activity
cfgnunes commentedon Oct 16, 2026
Hey @jhheider, do you have any ideas on how to fix this issue? I'm not even sure where the fix belongs... in pantry or in the pkgx repository itself. Any pointers would be appreciated!!
jhheider commentedon Oct 16, 2026
bkpyvenvis the culprit here. it's not race-safe. using a lockfile or checking if the stub needs rewrites before rewriting would be better. happy to take a PR, or I'll look into it.jhheider commentedon Oct 16, 2026
even a simple
.stampthat contains the path it was rewritten for would tell it if it needed to be rewritten again. possibly including the python path.cfgnunes commentedon Oct 16, 2026
@jhheider thanks for the feedback!
I've written a fix that I believe should solve this issue in PR: #368
Could you take a look and let me know if everything looks good?
jhheider commentedon Oct 16, 2026
Fixed by #368, released in brewkit v1.23.3 and published to the pantry: https://github.com/pkgxdev/pantry/actions/runs/38979727618
I rebuilt
github.com/0xabu/pdfannots0.5.0 with it: https://github.com/pkgxdev/pantry/actions/runs/38980171025Checked against a fresh
PKGX_DIRwith the rebuilt bottle:seq 16 | xargs -P 16 -I{} pkgx pdfannots --version, three rounds, gave 48 of 48pdfannots 0.5and no errors. The stub now takes anflockon the venv'sbin/directory and only rewrites when the interpreter or a shebang is stale, so a current venv is never written to.cfgnunes commentedon Oct 16, 2026
Thanks for the quick fix and for being so responsive @jhheider ! You're almost like the superhero The Flash, incredibly fast! 😄
I've tested it here, and everything worked perfectly. Thanks again for the great work!
jhheider commentedon Oct 16, 2026
thanks for the diagnosis and the shape the fix!