Repository navigation
Enhancement: Add explicit logging warning for NaN classes in macro-averaged metrics - #558
karttikjangid wants to merge 5 commits into
Conversation
|
Hi team, the CI is failing on LiDAR tests due to an isinstance check against a mocked Open3D class in the headless runner. My changes are limited to metrics; would you like me to include a small guard to fix this CI bottleneck? |
|
I have reinstated the original poetry files for reference in the last commit |
dpascualhe
left a comment
dpascualhe
left a comment
There was a problem hiding this comment.
Upon fixing the issues raised we will merge 😄
| missing_count = int(np.sum(nan_mask)) | ||
|
|
||
| if missing_count > 0: | ||
| import warnings |
There was a problem hiding this comment.
raise the warning as part of logging (you can check out logging.warning in line 60 of rellis3d.py). At some point we will do a project-wide effort to actually settle down on a proper logging system
| f"Their {metric_name.upper()} evaluated to NaN and will be ignored in the macro-average." | ||
| ) | ||
| warnings.warn(msg, UserWarning) | ||
| return float(np.nanmean(metric(per_class=True))) |
There was a problem hiding this comment.
reuse here the per_class_values variable to avoid computing the metric twice
|
Hi @karttikjangid ! Will you make the requested changes? Otherwise we will close the PR for now |
|
@dpascualhe Surely , I will make a pr with those changes in a day or two .Always happy to do Open Source 😊. |
karttikjangid
changed the title
|
Both requested changes are pushed: the warning now uses logging.warning, and the average reuses per_class_values. The 20 existing metric tests pass locally. Please take another look. |
This branch has not been deployed
2 participants
Fixes #546
Following up on the discussion in #546, this PR adds a logging warning during macro-averaged metric calculation.
When per-class metric values are NaN (for example, denominator-zero cases such as classes missing from the confusion matrix), they are ignored by np.nanmean as intended. This PR now surfaces a warning indicating how many classes were dropped for that metric so researchers are explicitly aware of the macro-average context.
All existing metric formulas and public APIs are unchanged. Feedback on the warning text format is welcome.