Judge-score report generation crashes on missing or incomplete score distributions

[andreatnvidia]

Priority Level

Medium (Annoying but has workaround)

Describe the bug

DatasetProfilerResults accepts judge-score profiles whose score_distributions value is missing, and profiles whose histograms mapping omits a score present in summaries. Calling to_report() on either schema-valid document raises an uncaught exception:

• Missing score_distributions: AttributeError: 'MissingValue' object has no attribute 'histograms'
• Missing histogram entry: KeyError: 'helpfulness'

This prevents report generation for analysis results that pass model validation.

Steps/Code to reproduce bug

from copy import deepcopy
from pathlib import Path

from data_designer.config.analysis.dataset_profiler import DatasetProfilerResults


document = {
    "num_records": 1,
    "target_num_records": 1,
    "column_statistics": [
        {
            "column_type": "general",
            "column_name": "text",
            "num_records": 1,
            "num_null": 0,
            "num_unique": 1,
            "pyarrow_dtype": "string",
            "simple_dtype": "str",
        }
    ],
    "column_profiles": [
        {
            "column_name": "quality",
            "summaries": {
                "helpfulness": {
                    "score_name": "helpfulness",
                    "summary": "ok",
                    "score_samples": [],
                }
            },
            "score_distributions": {
                "scores": {"helpfulness": [5]},
                "reasoning": {"helpfulness": ["ok"]},
                "distribution_types": {"helpfulness": "categorical"},
                "distributions": {"helpfulness": "--"},
                "histograms": {"helpfulness": {"categories": [5], "counts": [1]}},
            },
        }
    ],
}

missing_distributions = deepcopy(document)
missing_distributions["column_profiles"][0]["score_distributions"] = "--"

missing_histogram = deepcopy(document)
missing_histogram["column_profiles"][0]["score_distributions"]["histograms"] = {}

for name, payload in (
    ("missing distributions", missing_distributions),
    ("missing histogram", missing_histogram),
):
    results = DatasetProfilerResults.model_validate(payload)
    try:
        results.to_report(Path("report.html"))
    except (AttributeError, KeyError) as error:
        print(f"{name}: {type(error).__name__}: {error}")

The first payload raises AttributeError; the second raises KeyError.

Expected behavior

Report generation should render an appropriate missing-data placeholder, or inconsistent profile data should be rejected during model validation with a clear validation error. to_report() should not leak raw AttributeError or KeyError exceptions for accepted inputs.

Agent Diagnostic / Prior Investigation

Reproduced on the current main branch.

JudgeScoreProfilerResults.score_distributions is declared as JudgeScoreDistributions | MissingValue in packages/data-designer-config/src/data_designer/config/analysis/column_profilers.py. create_report_section() then unconditionally accesses self.score_distributions.histograms[score_name] for every summary.

A valid nonempty histogram renders successfully as a negative control. The issue tracker was searched for score_distributions, JudgeScoreProfilerResults, to_report, and missing histograms; no matching issue was found.

Additional context

The fix should cover both accepted missing values and mismatches between summary names and histogram keys. Regression tests should exercise the public DatasetProfilerResults.to_report() method.

Checklist

• I reproduced this issue or provided a minimal example
• I searched the docs/issues myself, or had my agent do so
• If I used an agent, I included its diagnostics above