Skip to content

EmbeddingCollapseMetric: effective-rank score has a 1/N floor when N <= D #8996

Description

@rubenG1009

Describe the bug

_effective_rank_score (monai/metrics/embedding_collapse.py:298) mean-centers the embedding matrix, which caps its attainable rank at min(N-1, D), but normalizes the effective rank by min(N, D).

Since the score is 1 - eff_rank/max_rank, this imposes a hard floor of 1/N whenever N <= D - so the metric structurally cannot report "no collapse" in the regime it was designed for (high-dimensional embeddings, modest sample counts).

_per_class_rank is affected more, since it partitions N by class: the smaller per-class N makes the floor larger.

To Reproduce

Isotropic embeddings - no collapse is present, so the correct score is ~0:

N D score now expected
2 768 0.5000 0.0
4 768 0.2503 ~0.0004
8 768 0.1262 ~0.0014
64 768 0.0261 ~0.0106
1024 768 0.1234 0.1234 (N > D: unaffected)

Two maximally-spread points score exactly 0.50 ("50% collapsed") when they span the only subspace two samples can span:

import torch
from monai.metrics.embedding_collapse import _effective_rank_score

emb = torch.zeros(2, 768)
emb[0, 0], emb[1, 0] = 1.0, -1.0
print(float(_effective_rank_score(emb)))  # 0.5, expected 0.0

The observed floor matches 1 - (N-1)/N = 1/N exactly.

Expected behavior

Normalize by min(N-1, D), the attainable maximum after centering. This is a no-op for N > D, where both expressions equal D.

Environment

Introduced in #8815 (merged 2026-04-21), shipped in 1.6.0.

I have a fix with tests ready and will open a PR shortly.

One design question: for N = 2 the centered matrix is always rank-1, so the score is a constant regardless of the data. Is returning 0.0 right there, or would None be preferable so it is excluded from aggregate(), as _class_collapse does for a single class?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions