Bug: Two LocalEvalSetsManager tests failing due to incorrect mock targets

[G26karthik]

Bug Description

Two unit tests in test_local_eval_sets_manager.py are failing on main branch:

• test_local_eval_sets_manager_update_eval_case_eval_set_not_found
• test_local_eval_sets_manager_delete_eval_case_eval_set_not_found

Error Message

OSError: [Errno 22] Invalid argument: '<tests.unittests.evaluation.test_local_eval_sets_manager.TestLocalEvalSetsManager object at 0x00000214DA2CE8D0>\\test_app\\test_eval_set.evalset.json'

Root Cause

The tests mock get_eval_case but the code path calls get_eval_set first via get_eval_set_from_app_and_id(). Since get_eval_set is not mocked, the code hits the actual file system with an invalid path.

The invalid path occurs because the class fixture at line 322 is missing self:

@pytest.fixture
def local_eval_sets_manager(tmp_path):  # Missing `self` parameter
    agents_dir = str(tmp_path)
    return LocalEvalSetsManager(agents_dir=agents_dir)

When pytest invokes this fixture on a class method, it passes self (the test instance) as the first argument, which gets assigned to tmp_path. Then str(tmp_path) becomes the string representation of the test object instead of an actual filesystem path.

Steps to Reproduce

pytest tests/unittests/evaluation/test_local_eval_sets_manager.py -v

Expected Behavior

All 29 tests should pass.

Actual Behavior

2 tests fail with OSError: [Errno 22] Invalid argument.

Environment

• ADK Version: 1.23.0 (main branch)
• Python: 3.13.5
• OS: Windows (also affects Linux CI)

Fix

Change the mock target from get_eval_case to get_eval_set with return_value=None to correctly trigger the NotFoundError for "eval set not found" scenarios.

PR: #4373

[G26karthik]

Hi @precision @AustinWise @rspier - this is a quick fix for 2 failing unit tests on main.

The tests were mocking the wrong method (get_eval_case instead of get_eval_set), causing them to hit the filesystem with invalid paths. One-line fix each.

Ready for review whenever you have a moment. Thanks!

[GWeale]

Hi this is not currently failing on main going to close :)