Skip to content

test(memory): rewrite integration tests with proper pytest structure#276

Open
jariy17 wants to merge 1 commit intomainfrom
fix/memory-integration-tests
Open

test(memory): rewrite integration tests with proper pytest structure#276
jariy17 wants to merge 1 commit intomainfrom
fix/memory-integration-tests

Conversation

@jariy17
Copy link
Contributor

@jariy17 jariy17 commented Feb 23, 2026

Summary

  • Replace script-style test_devex.py (756 lines, no real assertions) with proper pytest test class in test_memory_client.py covering all 37 MemoryClient public methods
  • Add assert_created_event helper for create_event/fork_conversation responses which do not include payload
  • Add leaf-value blob assertion to handle service returning blob data as a stringified Java-style map
  • Fail tests when MEMORY_ROLE_ARN is not set instead of silently skipping
  • Add pytest-xdist for parallel test execution, Husky hooks docs, and node_modules to .gitignore

Test plan

  • Run MEMORY_ROLE_ARN="..." pytest -vs tests_integ/memory/test_memory_client.py and verify all 11 tests pass
  • Run without MEMORY_ROLE_ARN and verify tests fail (not skip)
  • Verify pre-commit hooks pass (ruff lint, ruff format)

- Replace script-style test_devex.py with proper pytest test class in
  test_memory_client.py covering all 37 MemoryClient public methods
- Add assert_created_event helper for create_event/fork_conversation
  responses which do not include payload
- Add leaf-value blob assertion to handle service returning blob data
  as a stringified Java-style map representation
- Fail tests when MEMORY_ROLE_ARN is not set instead of silently skipping
- Add pytest-xdist for parallel test execution support
- Add Husky git hooks setup to CONTRIBUTING.md and node_modules to .gitignore
@jariy17 jariy17 requested a review from a team February 23, 2026 16:57
@jariy17
Copy link
Contributor Author

jariy17 commented Feb 23, 2026

CI/CD gaps for memory integration tests

The current GitHub workflows don't run memory integration tests. Here's what needs to change in integration-testing.yml:

1. Memory tests are not included in the test command

Current: Only runtime tests run:

run: |
  pytest tests_integ/runtime -s --log-cli-level=INFO

Fix: Add a separate step (or expand the existing one) to include memory tests:

- name: Run runtime integration tests
  env:
    AWS_REGION: us-west-2
    PYTHONUNBUFFERED: 1
  timeout-minutes: 10
  run: |
    pytest tests_integ/runtime -s --log-cli-level=INFO

- name: Run memory integration tests
  env:
    AWS_REGION: us-west-2
    MEMORY_ROLE_ARN: ${{ secrets.MEMORY_EXECUTION_ROLE_ARN }}
    PYTHONUNBUFFERED: 1
  timeout-minutes: 15
  run: |
    pytest tests_integ/memory -vs --log-cli-level=INFO

2. MEMORY_ROLE_ARN secret is not configured

The memory tests require MEMORY_ROLE_ARN and will now fail (not skip) if it's missing. A new repository secret needs to be added:

  • Secret name: MEMORY_EXECUTION_ROLE_ARN
  • Value: arn:aws:iam::<account-id>:role/MemoryExecutionRole

3. pytest-xdist is installed but not used in CI

The dev dependency was added for parallel execution but no workflow uses it. For memory tests (which are slow due to API waits), consider:

run: |
  pytest tests_integ/memory -vs -n auto --dist loadscope --log-cli-level=INFO

Note: -n auto with --dist loadscope keeps class-scoped fixtures together on the same worker.

4. @pytest.mark.integration is not registered

The test class uses @pytest.mark.integration but it's not registered in pyproject.toml, causing a warning. Add to [tool.pytest.ini_options]:

markers = ["integration: marks tests as integration tests"]

This also enables selective runs: pytest -m integration or pytest -m "not integration".

5. Timeout may be too short

The current 10-minute timeout works for runtime tests, but memory tests involve _wait_for_memory_active (up to 5 min) plus strategy operations. A 15-minute timeout is safer for the memory step.

6. Missing pytest-xdist in pip install

The integration test job installs dependencies via pip install (not uv sync), so pytest-xdist won't be available unless explicitly added:

- name: Install dependencies
  run: |
    pip install -e .
    pip install --no-cache-dir pytest pytest-xdist requests strands-agents uvicorn httpx starlette websockets

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant