Repository navigation
fix(metrics): flatten column-vector targets in NegativeLogLikelihood - #509
Merged
Merged
Conversation
The training loop passes LSS targets as (N, 1) column vectors. In NegativeLogLikelihood (and LogScore) they broadcast against the batch of N distributions into an (N, N) grid of log-probabilities, so the logged metric differed from the family loss. Squeeze a trailing singleton target dimension before computing the loss. Fixes #508
evaluate() forwards y_true unchanged, so it passes (N, 1) targets only when the caller does. Refs #508
mkumar73
approved these changes
Nov 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #508
Problem
NegativeLogLikelihood.__call__turnedy_trueinto a tensor unchanged and calleddistribution.compute_loss(y_pred, y_true). For the(N, 1)targets that the training loop passes (and thatevaluate()forwards unchanged wheny_trueis a column vector),log_probbroadcast the batch ofNdistributions against the column vector into an(N, N)grid, and the metric returned the mean of that grid. Aval_metrics={"nll": NegativeLogLikelihood(family)}therefore logged a different value fromval_lossfor the same family.LogScorewraps the same metric and had the same problem.Fix
NegativeLogLikelihoodsqueezes a trailing singleton dimension ofy_true((N, 1)→(N,)) before computing the loss. Multi-output targets (e.g. Dirichlet components) and 1-D targets are unchanged.Tests
tests/metrics/test_metric_contracts.py::TestLikelihoodMetricsAcceptColumnTargets:test_column_and_flat_targets_give_the_family_loss[NegativeLogLikelihood|LogScore]: flat and column targets both give the family'scompute_loss(negated forLogScore).test_logged_validation_nll_matches_validation_loss: in anMLPLSSfit, the loggedval_nllequalsval_loss.All three fail on
mainand pass with this change.Docs
No change needed. The metric was already documented as the family NLL.
Checks
just test: 3239 passed, 46 skipped, 28 xfailed; coverage 91.38%just docs: build succeeded (warnings treated as errors)just check: all hooks passed (ruff, prettier, pyright)