When a float-scale aggregate combines more than one constituent, the constituents without a rationale disappear from the rationale entirely.
In pyrit/score/float_scale/float_scale_score_aggregator.py:
else:
description = aggregate_description
# Only include scores with non-empty rationales
rationale_parts = [format_score_for_rationale(s) for s in scores if s.score_rationale]
rationale = "\n".join(rationale_parts) if rationale_parts else ""Two things in the same package say the opposite. format_score_for_rationale — the formatter being called — is built to render a line for a rationale-less score (f" - {class_type} {value}: {score.score_rationale or ''}"), and its docstring describes the value and the rationale as what a line carries. The other two rationale builders do not filter: the undetermined branch of this same file (_undetermined_result) and true_false_score_aggregator._build_rationale both pass every constituent through.
The scorer that makes this visible is one this repository already documents: FloatScaleThresholdScorer's own docstring notes that AzureContentFilterScorer "routinely does not" supply a rationale. A multi-chunk or multi-category Azure filter aggregate therefore persists score_rationale == "" — the score is right, but nothing records what was aggregated or that there was more than one constituent, while the same run's true/false aggregates list theirs.
Proposal: drop the filter, so the float-scale rationale matches its two siblings. If the filter is deliberate, the alternative is to keep it and say so in the output (for example a trailing "N constituent(s) had no rationale"), so an empty rationale is distinguishable from a single-component one.
Happy to send the one-line change plus tests either way — I did not want to just delete a line that was written on purpose without asking.
When a float-scale aggregate combines more than one constituent, the constituents without a rationale disappear from the rationale entirely.
In
pyrit/score/float_scale/float_scale_score_aggregator.py:Two things in the same package say the opposite.
format_score_for_rationale— the formatter being called — is built to render a line for a rationale-less score (f" - {class_type} {value}: {score.score_rationale or ''}"), and its docstring describes the value and the rationale as what a line carries. The other two rationale builders do not filter: the undetermined branch of this same file (_undetermined_result) andtrue_false_score_aggregator._build_rationaleboth pass every constituent through.The scorer that makes this visible is one this repository already documents:
FloatScaleThresholdScorer's own docstring notes thatAzureContentFilterScorer"routinely does not" supply a rationale. A multi-chunk or multi-category Azure filter aggregate therefore persistsscore_rationale == ""— the score is right, but nothing records what was aggregated or that there was more than one constituent, while the same run's true/false aggregates list theirs.Proposal: drop the filter, so the float-scale rationale matches its two siblings. If the filter is deliberate, the alternative is to keep it and say so in the output (for example a trailing "N constituent(s) had no rationale"), so an empty rationale is distinguishable from a single-component one.
Happy to send the one-line change plus tests either way — I did not want to just delete a line that was written on purpose without asking.