Repository navigation
Conversation
build_model() replaced the trained network with freshly initialized weights but kept is_fitted_ = True from an earlier fit(), so predict(), save() and InferenceModel silently used the untrained network. Reset the flag in _FitMixin._build_model and SklearnBaseLSS.build_model (only when it was set, so a fresh estimator gains no attribute) and document it in the build_model() docstrings. Fixes #541
…ailing build build_model() now also clears _best_model_path, which a regressor's score() would otherwise reload into the rebuilt network. Classifiers reset the fitted state before replacing classes_, so a build that fails on its labels no longer leaves a fitted-looking estimator with the new classes and the old network. The docstrings no longer claim is_fitted_ exists on an estimator that was never fitted. Refs #541
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #541
Problem
build_model()replaces the task model with freshly initialized weights. On an already fitted estimator it keptis_fitted_ = True, sopredict(),evaluate(),save()andInferenceModel.from_estimator()(and a classifier'sscore()) silently treated the untrained network as the fitted model, while a regressor'sscore()loaded the previous fit's best checkpoint into it. This happened for classifiers and regressors (_FitMixin._build_model) and for LSS estimators (SklearnBaseLSS.build_model).fit()already resets the flag before rebuilding.Fix
_FitMixin._mark_unfitted()resetsis_fitted_toFalsewhen it was set, and clears_best_model_path, the previous fit's best checkpoint. A never-fitted estimator gains nois_fitted_attribute, soprofile()still leaves it unchanged._FitMixin._build_modelandSklearnBaseLSS.build_modelcall it. The classifier'sbuild_modelcalls it before replacingclasses_, so a build that fails on its labels (e.g. ay_vallabel unseen iny) leaves an unfitted estimator rather than one with the new classes and the old network.fit()sets the flag back toTruewhen training completes, includingfit(rebuild=False)afterbuild_model().Returnssections of the classifier, regressor and LSSbuild_modeldocstrings state that the built estimator counts as unfitted (predict()raisesNotFittedError) untilfit()completes.Tests
tests/integration/test_build_model_state.py(new file):test_regressor_is_unfitted_after_rebuilding:predict()raisesNotFittedErrorand_best_model_pathis cleared afterbuild_model(), andfit(rebuild=False)makes the estimator fitted again.test_lss_is_unfitted_after_rebuildingtest_classifier_is_unfitted_when_rebuilding_fails: abuild_model()that raises on an unseeny_vallabel leaves the classifier unfitted.All three fail on
mainand pass with this change. The fix for #452 cleared the same stale flag for a failingfit().Docs
Docstrings as above.
docs/core_concepts/config_system.mdalready describesbuild_model()as building the network "without training its weights".Checks
just test: 3239 passed, 46 skipped, 28 xfailed; coverage 91.39%just docs: build succeeded (warnings treated as errors)just check: all hooks passed (ruff, prettier, pyright)fit.py,lss_base.pyortest_fit_lifecycle.py(fix(models): respect enable_model_summary in fit #486, fix(models): honour ObservabilityConfig in LSS fit #500, fix(models): score the in-memory model without reloading the checkpoint #504). I checked this against all open PR branches.