Shared cluster nextflow config - #102
Open
rdemko2332 wants to merge 2 commits into
Open
Conversation
New WorkflowStep.pm helper getSharedClusterNextflowConfigIncludeBlock(): reads
$clusterServer.sharedNextflowConfig from stepsShared.prop (via getSharedConfigRelaxed,
so an unset value is a silent no-op) and, if set, returns an includeConfig line pulling
in that cluster-provided config file. Lets a cluster-wide nextflow process{} setting
(e.g. beforeScript = 'module load apptainer/1.4.1 && unset LD_LIBRARY_PATH') be rolled
out via a one-line stepsShared.prop change instead of hand-editing every
Make*NextflowConfig.pm generator -- 37 of them exist, and this session's ortho work
already needed the same beforeScript line added to 7 by hand.
Mirrors the existing getNextflowLsfScratchEnvBlock() pattern in the same file (same
shared-helper shape, already called identically from 28 of the 37 generators).
Also fixed getNextflowLsfScratchEnvBlock() while here: NXF_SCRATCH = '$LSF_TMPDIR' had
no default, crashing under nextflow's own set -u if LSF doesn't export LSF_TMPDIR --
the same bug independently found and hand-fixed in 7 ortho-specific generators on
another branch this session. Centralizing the fix here covers all 28 current callers.
…erator
Every Make*NextflowConfig.pm step class (38 total) now computes
$sharedClusterConfig = $self->getSharedClusterNextflowConfigIncludeBlock() and
interpolates it at the very top of its generated nextflow config, before params {}.
When $clusterServer.sharedNextflowConfig isn't set in stepsShared.prop this renders as
an empty string -- verified byte-identical output to before this change for that
(default, current) case. When it is set, an includeConfig '<path>' line appears at the
top, pulling in whatever cluster-wide process{} settings that shared file declares.
Mechanical, per-file changes only: add the one declaration line (anchored next to each
file's existing $lsfEnv/$lsfScratch computation, or near the top of run() for the 5
files with neither), and prepend $sharedClusterConfig to whatever the file already
prints (either interpolated into a heredoc string, or concatenated onto a $configString
variable) -- every file's own existing style preserved as-is, nothing else changed.
The 5 ortho-specific generators here still hand-roll their own $lsfScratch (predates the
shared getNextflowLsfScratchEnvBlock() helper 28 of the others already use) -- left
untouched, out of scope for this change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.