Skip to content

fix(models): mark the estimator unfitted when build_model() rebuilds it - #542

Open
ChrisW09 wants to merge 4 commits into
mainfrom
fix/build-model-fitted-flag
Open

ChrisW09 wants to merge 4 commits into
mainfrom
fix/build-model-fitted-flag

Conversation

@ChrisW09

@ChrisW09 ChrisW09 commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #541

Problem

build_model() replaces the task model with freshly initialized weights. On an already fitted estimator it kept is_fitted_ = True, so predict(), evaluate(), save() and InferenceModel.from_estimator() (and a classifier's score()) silently treated the untrained network as the fitted model, while a regressor's score() loaded the previous fit's best checkpoint into it. This happened for classifiers and regressors (_FitMixin._build_model) and for LSS estimators (SklearnBaseLSS.build_model). fit() already resets the flag before rebuilding.

Fix

  • A new _FitMixin._mark_unfitted() resets is_fitted_ to False when it was set, and clears _best_model_path, the previous fit's best checkpoint. A never-fitted estimator gains no is_fitted_ attribute, so profile() still leaves it unchanged.
  • _FitMixin._build_model and SklearnBaseLSS.build_model call it. The classifier's build_model calls it before replacing classes_, so a build that fails on its labels (e.g. a y_val label unseen in y) leaves an unfitted estimator rather than one with the new classes and the old network.
  • fit() sets the flag back to True when training completes, including fit(rebuild=False) after build_model().
  • The Returns sections of the classifier, regressor and LSS build_model docstrings state that the built estimator counts as unfitted (predict() raises NotFittedError) until fit() completes.

Tests

tests/integration/test_build_model_state.py (new file):

  • test_regressor_is_unfitted_after_rebuilding: predict() raises NotFittedError and _best_model_path is cleared after build_model(), and fit(rebuild=False) makes the estimator fitted again.
  • test_lss_is_unfitted_after_rebuilding
  • test_classifier_is_unfitted_when_rebuilding_fails: a build_model() that raises on an unseen y_val label leaves the classifier unfitted.

All three fail on main and pass with this change. The fix for #452 cleared the same stale flag for a failing fit().

Docs

Docstrings as above. docs/core_concepts/config_system.md already describes build_model() as building the network "without training its weights".

Checks

ChrisW09 and others added 4 commits October 21, 2026 20:49
build_model() replaced the trained network with freshly initialized
weights but kept is_fitted_ = True from an earlier fit(), so predict(),
save() and InferenceModel silently used the untrained network. Reset the
flag in _FitMixin._build_model and SklearnBaseLSS.build_model (only when
it was set, so a fresh estimator gains no attribute) and document it in
the build_model() docstrings.

Fixes #541
Keeps them apart from the callback tests appended to test_fit_lifecycle.py by #486.

Refs #541
…ailing build

build_model() now also clears _best_model_path, which a regressor's
score() would otherwise reload into the rebuilt network. Classifiers
reset the fitted state before replacing classes_, so a build that fails
on its labels no longer leaves a fitted-looking estimator with the new
classes and the old network. The docstrings no longer claim is_fitted_
exists on an estimator that was never fitted.

Refs #541

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] build_model() on a fitted estimator keeps is_fitted_=True with an untrained network

2 participants