keys in `fixture_store` gone missing in multi-process environment

[psteinb]

Hello,
thanks for sharing this project with the community. This is very helpful. I ran into some trouble with executing pytest-harvest based test suites on my laptop versus on a server within a cluster. As I tried to search for the root cause, I stumbled on a curious observation.

Here is my test file test_simple.py:

import pytest
from pytest_harvest import saved_fixture
import time

@pytest.fixture(params=range(2))
@saved_fixture
def person(request):
    """
    A dummy fixture, parametrized so that it has two instances
    """
    if request.param == 0:
        return "world"
    elif request.param == 1:
        return "self"

def test_foo(person):
    """
    A dummy test, executed for each `person` fixture available
    """
    #enter sleep statement
    #time.sleep(30)
    print('\n   hello, ' + person + ' !')

def test_synthesis(fixture_store):
    """
    In this test we inspect the contents of the fixture store so far,
    and check that the 'person' entry contains a dict <test_id>: <person>
    """
    # print the keys in the store
    print("\n   Available `fixture_store` keys:")
    for k in fixture_store:
        print("    - '%s'" % k)

    assert 'person' in fixture_store.keys()
    # print what is available for the 'person' entry
    
    print("\n   Contents of `fixture_store['person']`:")
    for k, v in fixture_store['person'].items():
        print("    - '%s': %s" % (k, v))

and the following conftest.py goind with it:

from pytest_harvest import is_main_process, get_xdist_worker_id, \
                           get_session_results_df

def pytest_sessionfinish(session):
    """ Gather all results and save them to a csv. 
    Works both on worker and master nodes, and also with xdist disabled"""

    session_results_df = get_session_results_df(session)
    suffix = 'all' if is_main_process(session) else get_xdist_worker_id(session)
    session_results_df.to_csv('results_%s.csv' % suffix)

The fun thing, if I run this code like so:

python -m pytest  ./tests/ -n NCORES

and increase the number of NCORES, the test_synthesis fails the single assert in it for any number of cores higher than 2. Can you provide any hints what I am doing wrong?

[smarie]

Thanks a lot @psteinb ! Indeed this is very strange. I will have to investigate but my bandwidth is fairly limited these days so it might be later, during summer hopefully.

By the way, being curious : do you use a particular plugin to run on your cluster apart from pytest-xdist+remote ssh ? I am planning to investigate creating a pytest plugin to run against a ray cluster in the near future. https://docs.ray.io/en/latest/cluster/vms/getting-started.html
Indeed it seems more compliant with the cloud setup we have here in the team.