ComputingConfiguration objects mutability

[stolarczyk]

Should the objects of ComputingConfiguration class be mutable (the original compute settings changed in the object)? Or should it always keep the original settings and new can be added/updated on the fly, when the submission script is being created?

[nsheff]

@vreuter what do you think?

[vreuter]

I don't have a clear picture of the use case(s). Do you have an example?

[nsheff]

well, looper and caravel are the immediate users of divvy... there could be others I guess but that's a start.

[vreuter]

Right, I meant blocks of code, like where and what substitutions will be made to introduce divvy into those projects?

[stolarczyk]

It was a month ago and I don't exacly recall what I meant. But I think the question was if we should allow this, for example:

In [23]: dcc = divvy.ComputingConfiguration()
No local config file was provided
Found global config file in DIVCFG: /Users/mstolarczyk/Uczelnia/UVA/code/pepenv/uva_rivanna.yaml
Loading divvy config file: /Users/mstolarczyk/Uczelnia/UVA/code/pepenv/uva_rivanna.yaml
Use 'compute_packages' instead of 'compute'
Available packages: set(['singularity_local', 'default', 'largemem', 'singularity_slurm', 'sigterm', 'local', 'parallel'])
Activating compute package 'default'

In [24]: dcc.activate_package("singularity_slurm")
Activating compute package 'singularity_slurm'
Out[24]: True

In [25]: dcc.compute
Out[25]: {'submission_command': 'sbatch', 'partition': 'standard', 'submission_template': '/Users/mstolarczyk/Uczelnia/UVA/code/pepenv/templates/slurm_singularity_template.sub', 'singularity_args': '-B /sfs/lustre:/sfs/lustre,/nm/t1:/nm/t1'}

In [26]: dcc.compute.singularity_args = "test"

In [27]: dcc.compute
Out[27]: {'submission_command': 'sbatch', 'partition': 'standard', 'submission_template': '/Users/mstolarczyk/Uczelnia/UVA/code/pepenv/templates/slurm_singularity_template.sub', 'singularity_args': 'test'}

Or the dcc always reflects what was specified in the DIVCFG file

[vreuter]

Well in general I prefer things to be immutable, but in this case it's sort of a feature of the attmap and seems pretty innocuous here, so I guess I think keep the standard attmap behavior.

[nsheff]

ok then maybe we think of the 'packages' section as immutable, and the compute section as a little holder where packages can be loaded, and then changed and retrieved... so we can sort of preserve both ways of interacting with it.