Make calculations robust to failure

[ElliottKasoar]

Pre-review checklist for PR author

PR author must check the checkboxes below when creating the PR.

• I've confirmed the contribution guidelines.

Summary

Updates all calculations to be robust against errors raised by calculators (except RuntimeErrors etc.)

Testing

Tested using the error calculator in #560, but a few newer tests may have been missed.

I suggest merging #560 before this for this reason, and rebasing.

[joehart2001]

ready to rebase?

[ElliottKasoar]

ready to rebase?

Yep, was just checking/updating the new tests. I think the only failure should be resolved by #526.

An easy way to check is:

-        from ml_peg.models.mock import MockCalculator
+        from ml_peg.models.mock import MockErrorCalculator
 
-        return MockCalculator()
+        return MockErrorCalculator()

in ml_peg/models/models.py, and run ml_peg calc --mock-only --run-very-slow

[joehart2001]

ready to rebase?

Yep, was just checking/updating the new tests. I think the only failure should be resolved by #526.

An easy way to check is:

-        from ml_peg.models.mock import MockCalculator
+        from ml_peg.models.mock import MockErrorCalculator
 
-        return MockCalculator()
+        return MockErrorCalculator()

in ml_peg/models/models.py, and run ml_peg calc --mock-only --run-very-slow

• When running i get a bunch of No module named 'torch' errors, unsure if important
• also get: FileNotFoundError: [Errno 2] No such file or directory: /ml-peg/ml_peg/calcs/nebs/li_diffusion/outputs/mock/li_diffusion_b-neb-band.extxyz'

[ElliottKasoar]

ready to rebase?

Yep, was just checking/updating the new tests. I think the only failure should be resolved by #526.
An easy way to check is:

-        from ml_peg.models.mock import MockCalculator
+        from ml_peg.models.mock import MockErrorCalculator
 
-        return MockCalculator()
+        return MockErrorCalculator()

in ml_peg/models/models.py, and run ml_peg calc --mock-only --run-very-slow

• When running i get a bunch of No module named 'torch' errors, unsure if important
• also get: FileNotFoundError: [Errno 2] No such file or directory: /ml-peg/ml_peg/calcs/nebs/li_diffusion/outputs/mock/li_diffusion_b-neb-band.extxyz'

Ah, the first is related to importing D3, which shouldn't actually be used.

The second is probably just because the dir isn't created, I'll fix both, thanks!

[ElliottKasoar]

Ah, actually the torch import is a janus-core issue, since we set the default torch dtype fix issues you get when swapping between models.

I'll fix that there separately, but for now if you can just run in an environment with torch e.g. with any of the extras

[joehart2001]

Ah, actually the torch import is a janus-core issue, since we set the default torch dtype fix issues you get when swapping between models.

I'll fix that there separately, but for now if you can just run in an environment with torch e.g. with any of the extras

Yeah i've done that, ok nice. rerunning once more with the nebs fix

[joehart2001]

• lattice constants has no mock output
• BMIMCl_RDF mock output is also empty

also may need to reuplaod this?:

ml_peg/calcs/lanthanides/isomer_complexes/calc_isomer_complexes.py::test_isomer_complexes[mlip0] [cache] Found cached file: isomer_complexes.zip
SKIPPED (No isomer structures found under
~/.cache/ml_peg/isomer_complexes.

[ElliottKasoar]

• lattice constants has no mock output
• BMIMCl_RDF mock output is also empty

also may need to reuplaod this?:

ml_peg/calcs/lanthanides/isomer_complexes/calc_isomer_complexes.py::test_isomer_complexes[mlip0] [cache] Found cached file: isomer_complexes.zip
SKIPPED (No isomer structures found under
~/.cache/ml_peg/isomer_complexes.

Thanks for testing this!

I think for lattice constants, the only output would be the unoptimised structures. We could save this as the "trajectory", and maybe should in future, but it may not be obvious which calculations failed, so I think we'd want to update the analysis too to deal with this. This maybe comes down to implementing something like #45 first.

For BMIMCl_RDF, I think this is fine too, since again, there's not really a nice way to output a "failed" MD for this. We could so something, but no output is fine.

I've just reuploaded isomer_complexes.zip, can you delete it from the cache and try again? It's not related to this PR anyway so we can deal with it elsewhere

[joehart2001]

all good from my side now