Allow creation of virtual environment during the CMake configuration step

[compor]

This PR is working towards #149 (a few PRs on the way) by:

• Creating (or enabling if it already exists) a Python virtual env (under venv dir) from within the top-level CMake file.

It eliminates the need to create a virtual environment (venv) before issuing the CMake configuration, but it does not prevent the user from creating one manually if desired.

Update: Forgot to mention that this maintains the current functionality, and it is orthogonal with the current open PRs wrt CMake changes.

CC: @EmilySillars

[EmilySillars]

@compor I checked out this branch on my desktop machine (fedora) and after running

deactivate   # just to make sure we use a completely fresh venv

rm -r build # just to make sure not working with old build

cd build

cmake .. -GNinja \
  -DCMAKE_C_COMPILER=clang \
  -DCMAKE_CXX_COMPILER=clang++ \
  -DCMAKE_C_COMPILER_LAUNCHER=ccache \
  -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
  -DQUIDDITCH_TOOLCHAIN_FILE=../toolchain/ToolchainFile.cmake
  
ninja -j 20

I get the following error:

CMake Error at tests/CMakeLists.txt:9 (find_program):
  Could not find XDSL_OPT_PATH using the following names: xdsl-opt


-- Configuring incomplete, errors occurred!
FAILED: codegen-prefix/src/codegen-stamp/codegen-configure /home/hoppip/newGame/Quidditch/build/codegen-prefix/src/codegen-stamp/codegen-configure 
cd /home/hoppip/newGame/Quidditch/build/codegen && /usr/bin/cmake -DIREE_ENABLE_LLD=ON -DQUIDDITCH_TOOLCHAIN_ROOT=/home/hoppip/newGame/Quidditch/toolchain/ -DHAS_LOGF128=0 -DCMAKE_BUILD_TYPE= -DCMAKE_C_COMPILER=/usr/lib64/ccache/clang -DCMAKE_CXX_COMPILER=/usr/lib64/ccache/clang++ -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -GNinja -S /home/hoppip/newGame/Quidditch/codegen -B /home/hoppip/newGame/Quidditch/build/codegen && /usr/bin/cmake -E touch /home/hoppip/newGame/Quidditch/build/codegen-prefix/src/codegen-stamp/codegen-configure
ninja: build stopped: subcommand failed.

The output of my pip list includes xdsl:

(venv) [hoppip@inf-155-54-205-141 build]$ pip list
Package                   Version              Editable project location
------------------------- -------------------- -----------------------------------
attrs                     25.3.0
filelock                  3.18.0
fsspec                    2025.7.0
hjson                     3.1.0
immutabledict             4.2.0
iree-compiler             20241104.1068
iree-runtime              20241104.1068
iree-turbine              2.3.0
Jinja2                    3.1.6
jsonref                   1.1.0
jsonschema                4.21.1
jsonschema-specifications 2025.4.1
lit                       18.1.8
Mako                      1.3.10
MarkupSafe                3.0.2
mpmath                    1.3.0
networkx                  3.5
numpy                     2.3.1
nvidia-cublas-cu12        12.1.3.1
nvidia-cuda-cupti-cu12    12.1.105
nvidia-cuda-nvrtc-cu12    12.1.105
nvidia-cuda-runtime-cu12  12.1.105
nvidia-cudnn-cu12         8.9.2.26
nvidia-cufft-cu12         11.0.2.54
nvidia-curand-cu12        10.3.2.106
nvidia-cusolver-cu12      11.4.5.107
nvidia-cusparse-cu12      12.1.0.106
nvidia-nccl-cu12          2.20.5
nvidia-nvjitlink-cu12     12.9.86
nvidia-nvtx-cu12          12.1.105
ordered-set               4.1.0
pandas                    2.2.2
pip                       24.0
progressbar2              4.4.2
python-dateutil           2.9.0.post0
python-utils              3.9.1
pytz                      2025.2
PyYAML                    6.0.2
referencing               0.36.2
rpds-py                   0.26.0
setuptools                65.5.0
six                       1.17.0
sympy                     1.14.0
tabulate                  0.9.0
torch                     2.3.0
triton                    2.3.0
typing_extensions         4.12.2
tzdata                    2025.2
xdsl                      0.21.0+24.g024068cdb /home/hoppip/newGame/Quidditch/xdsl

@superlopuh are you able to reproduce this error on your machine?

[superlopuh]

What happens if you activate the venv before running the command?

[EmilySillars]

The same thing. I did

cd ..
venv source/bin/activate
cd build
ninja -j 20

and I get the same output:

CMake Error at tests/CMakeLists.txt:9 (find_program):
  Could not find XDSL_OPT_PATH using the following names: xdsl-opt


-- Configuring incomplete, errors occurred!
FAILED: codegen-prefix/src/codegen-stamp/codegen-configure /home/hoppip/newGame/Quidditch/build/codegen-prefix/src/codegen-stamp/codegen-configure 
cd /home/hoppip/newGame/Quidditch/build/codegen && /usr/bin/cmake -DIREE_ENABLE_LLD=ON -DQUIDDITCH_TOOLCHAIN_ROOT=/home/hoppip/newGame/Quidditch/toolchain/ -DHAS_LOGF128=0 -DCMAKE_BUILD_TYPE= -DCMAKE_C_COMPILER=/usr/lib64/ccache/clang -DCMAKE_CXX_COMPILER=/usr/lib64/ccache/clang++ -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -GNinja -S /home/hoppip/newGame/Quidditch/codegen -B /home/hoppip/newGame/Quidditch/build/codegen && /usr/bin/cmake -E touch /home/hoppip/newGame/Quidditch/build/codegen-prefix/src/codegen-stamp/codegen-configure
ninja: build stopped: subcommand failed.

full-output.txt

[superlopuh]

sorry, did you do venv source/bin/activate or source venv/bin/activate?

[superlopuh]

what happens if you do xdsl-opt --help after activating the venv?

[EmilySillars]

Sorry, I made a typo. I ran source venv/bin/activate.
But to be triply sure, I just deactivated the venv, removed it, removed the build folder and started again. Everything is working correctly now :)
working.txt

[superlopuh]

Yay!

[compor]

@EmilySillars thank you for checking, this is the expected behavior.