Add path-based ownership mode to `poly test diff`

[davzucky]

Is your feature request related to a problem?

poly test diff determines affected bricks by scanning imports in changed test files.

This is useful for dependency-impact analysis, but it cannot answer a different CI question:

Which brick owns the changed test?

For example, given the loose test structure:

test/
└── bases/
    └── qdev/
        └── market/
            └── test_holdings.py

and a test containing:

  from qdev.fastapi.security import jwt_coder
  from qdev.identity.user import User

changing test_holdings.py produces:

  $ poly test diff --since <commit> --bricks --short
  fastapi,identity

The owning market base is not returned.

Using projects expands the result further:

  $ poly test diff --since <commit> --projects --short
  hawk,identity,market,metrics,osprey,powerbi

This happens because every project transitively using the imported bricks is considered affected.

We understand that this is the current documented behavior and that import-based impact analysis is useful. However, it can cause CI to run unrelated brick and project test suites when a test clearly belongs to one brick.

Desired behavior

Provide an optional path-based ownership mode that maps changed tests to the brick or project represented by their directory.

For example:

  $ poly test diff --since <commit> --bricks --owners --short
  market

Possible mappings for the loose theme:

  test/bases/<namespace>/<brick>/**       -> base brick
  test/components/<namespace>/<brick>/**  -> component brick
  test/projects/<project>/**              -> project

The existing import-based behavior should remain available and could remain the default.

Suggested interface

One possibility:

  poly test diff --bricks --owners
  poly test diff --projects --owners

Alternative names could be --by-path or --ownership.

It may also be useful to support both analyses explicitly:

  poly test diff --strategy imports
  poly test diff --strategy ownership

Motivation

In CI, these answer different questions:

1. Import impact: Which bricks and projects could be affected by imports in changed tests?
2. Test ownership: Which test suites should run because files in those suites changed?

Having both modes would allow users to choose the appropriate behavior and avoid unexpectedly broad test selection.

Environment

• polylith-cli: 1.44.0
• Theme: loose
• Polylith workspace located in a subdirectory of a larger monorepo

[DavidVujic]

Thank you for the detailed description 🙏

Aha, so even if the test is not importing anything from the corresponding brick it should be possible to extract a list of those - did I understand it correctly? So if the test itself is within a market brick path, it should identify the market brick as affected.

I think this needs to assume that the tests are organized in the same structure as the bricks (and they will be if using the poly tool default).

[davzucky]

Yes, indeed, and it should only extract the root brick it is attached, This have for precondition that the test structure match the bricks format as you pointed out.
To add on our point only market should be returned even if other bricks are imported in the test file.

[DavidVujic]

Great! I will have a look at this!