Skip to content

WIP: Add caching for average_spectra#6

Open
SmirnGreg wants to merge 1 commit intoPlanetFormationLab:masterfrom
SmirnGreg:cache
Open

WIP: Add caching for average_spectra#6
SmirnGreg wants to merge 1 commit intoPlanetFormationLab:masterfrom
SmirnGreg:cache

Conversation

@SmirnGreg
Copy link

Hi Rich,
I have added to the very beginning of average_spectrum:```# Check whether this tuple of inputs was already calculated

args = (r_min, r_max, dr,
 PA_min, PA_max, exclude_PA,
 abs_PA, x0, y0, inc, PA, z0,
 psi, z1, phi, z_func, mstar,
 dist, resample, beam_spacing,
 mask_frame, unit.lower(), mask,
 assume_correlated, skip_empty_annuli,
 shadowed)
if args in self._cached_average_spectra.keys():
 return self._cached_average_spectra[args]

This does not really work, unfortunately. If I run radial_spectra, integrated_spectrum, radial_profile one by one with the same kwargs and output args, here is what I get:
radial_spectra

(0.0, 0.30958002500805, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(0.30958002500805, 0.6191600500161, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(0.6191600500161, 0.9287400750241499, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(0.9287400750241499, 1.2383201000322, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(1.2383201000322, 1.54790012504025, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(1.54790012504025, 1.8574801500482998, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(1.8574801500482998, 2.16706017505635, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(2.16706017505635, 2.4766402000644, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(2.4766402000644, 2.7862202250724497, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(2.7862202250724497, 3.0958002500805, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(3.0958002500805, 3.40538027508855, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(3.40538027508855, 3.7149603000965996, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(3.7149603000965996, 4.02454032510465, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(4.02454032510465, 4.3341203501127, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(4.3341203501127, 4.6437003751207495, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(4.6437003751207495, 4.9532804001288, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(4.9532804001288, 5.26286042513685, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(5.26286042513685, 5.572440450144899, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(5.572440450144899, 5.88202047515295, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(5.88202047515295, 6.191600500161, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(6.191600500161, 6.501180525169049, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(6.501180525169049, 6.8107605501771, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(6.8107605501771, 7.12034057518515, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(7.12034057518515, 7.429920600193199, None, None, None, None, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
integrated_spectrum
(None, None, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)

radial_profile
(0.0, 0.30958002500805, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(0.30958002500805, 0.6191600500161, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(0.6191600500161, 0.9287400750241499, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(0.9287400750241499, 1.2383201000322, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(1.2383201000322, 1.54790012504025, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(1.54790012504025, 1.8574801500482998, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(1.8574801500482998, 2.16706017505635, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(2.16706017505635, 2.4766402000644, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(2.4766402000644, 2.7862202250724497, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(2.7862202250724497, 3.0958002500805, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(3.0958002500805, 3.40538027508855, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(3.40538027508855, 3.7149603000965996, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(3.7149603000965996, 4.02454032510465, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(4.02454032510465, 4.3341203501127, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(4.3341203501127, 4.6437003751207495, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(4.6437003751207495, 4.9532804001288, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(4.9532804001288, 5.26286042513685, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(5.26286042513685, 5.572440450144899, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(5.572440450144899, 5.88202047515295, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(5.88202047515295, 6.191600500161, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(6.191600500161, 6.501180525169049, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(6.501180525169049, 6.8107605501771, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(6.8107605501771, 7.12034057518515, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)
(7.12034057518515, 7.429920600193199, None, None, None, False, False, 0, 0, 49.99, 191.22, 0.0, 1.0, 0.0, 1.0, None, 0.89, 158.7, 1, False, 'disk', 'jy/beam', None, False, True, True)

So the exclude_PA keyword is by default False within radial_profile, while it is None in radial_spectra. Is it by a reason? If I replace it in the method definition, the cache is used. It is still not used for the integrated_spectrum, as it takes the whole disk at once without radial binning. But in principle, it could do the other way around, by collapsing the known deprojected data?

@SmirnGreg
Copy link
Author

A small note on the code.

a = (1, False, None)
{a: 1}
Out[3]: {(1, False, None): 1}
b = (1, [])
{b: 1}
Traceback (most recent call last):
  File "C:\Users\smirn\.conda\envs\gofish\lib\site-packages\IPython\core\interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-5-c2adcc4945ec>", line 1, in <module>
    {b: 1}
TypeError: unhashable type: 'list'
import numpy as np
c = np.arange(3)
{c:1}
Traceback (most recent call last):
  File "C:\Users\smirn\.conda\envs\gofish\lib\site-packages\IPython\core\interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-8-ad35ea78fa5f>", line 1, in <module>
    {c:1}
TypeError: unhashable type: 'numpy.ndarray'

So this will only work as long as the parameters are hashable, or immutable. The dictionary self._cached_average_spectra will only work if all the parameters provided are immutables. It will not work with numpy arrays, for example, so this version needs to get some extra touch before it is merged. It can be done, for example, by providing the reasonable type conversion so the bools, floats or integers where they are expected. I have already done part of this by using unit.lower().

@SmirnGreg SmirnGreg changed the title Add caching for average_spectra WIP: Add caching for average_spectra Jul 20, 2020
@richteague
Copy link
Collaborator

Thanks for the suggestion. I've been thinking about this for a while now and while in principle I think it would be useful to do this, I'm worried that some of the arguments (particularly the mask argument which can be a user-specified 2D array) would struggle do to this.

Do you have a specific use case where you'd need to run the functions multiple times? I'm trying to work out whether it might be easier to define some wrapper around radial_spectra which you can then use to calculate other things (e.g. disk-averaged spectra, peak value profiles etc).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants