feat(multitask): configurable LightGBM n_jobs (default 1) + clear code-scanning #214#380
Merged
Conversation
The lgbm forecaster factories (default_lgbm_forecaster_factory, quantile_lgbm_forecaster_factory) now read config.lgbm_n_jobs and forward it to LGBMRegressor(n_jobs=...). Default is 1: on heterogeneous-core CPUs (e.g. Apple Silicon's performance + efficiency cores) LightGBM's all-core OpenMP anti-scales (the fork-join barrier stalls on the slow E-cores). With n_jobs=1 the backtesting heuristic (select_n_jobs_backtesting) instead parallelises the CV folds across processes, which scales cleanly (~3x faster end-to-end tuning on an M4 Max). On many-core homogeneous machines (e.g. Linux Xeon) set lgbm_n_jobs=-1 to restore all-core in-model threading. New field on ConfigMulti (inherited by ConfigEntsoe) and the PipelineConfig protocol; factories read it via getattr(config, "lgbm_n_jobs", 1) so config-like objects predating the field keep working. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…l-variable) _throttle_open_meteo rebinding the module-level _LAST_REQUEST_MONOTONIC through `global` tripped CodeQL's py/unused-global-variable (code-scanning alert #214). Hold the last-request monotonic time in a one-element list mutated in place under the lock instead: identical runtime behaviour, no global rebind. Updates the throttle-spacing test to the list form. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sync docs/reference with the new ConfigMulti/ConfigEntsoe.lgbm_n_jobs field and the updated lgbm factory docstrings (API Reference In Sync gate). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Make the LightGBM thread count used by the multitask forecaster factories configurable via a new
config.lgbm_n_jobsfield, and clear CodeQL code-scanning alert #214.lgbm_n_jobs(feat)ConfigMulti.lgbm_n_jobs: int = 1(inherited byConfigEntsoe; added to thePipelineConfigprotocol).default_lgbm_forecaster_factoryandquantile_lgbm_forecaster_factorynow passn_jobs=getattr(config, "lgbm_n_jobs", 1)toLGBMRegressor(thegetattrdefault keeps config-like objects that predate the field working).n_jobs=1the backtesting heuristic (select_n_jobs_backtesting) instead parallelises the CV folds across processes, which scales cleanly (~3× faster end-to-end tuning measured on an M4 Max). On many-core homogeneous machines (e.g. Linux Xeon) setlgbm_n_jobs=-1to restore all-core in-model threading.code-scanning #214 (fix)
py/unused-global-variableon_LAST_REQUEST_MONOTONIC(the 22.4.1 Open-Meteo throttle). Now held in a one-element list mutated in place under the lock — noglobalrebind, identical runtime behaviour.Tests
lgbm_n_jobs(and default to 1).🤖 Generated with Claude Code