[jeffseif]

Background

One of the heavy steps in attribute is downscaling the 30 m emissions rasters to MAPSPAM's 10 km grid for statistical attribution. It wrote each band in full to a GeoTIFF on disk, then had a GDAL warped VRT average it down, which loaded the whole band (about 6 GiB) for every read. A tile took 5-7 minutes, peaked near 50 GiB at 8 workers (enough to OOM HND) and left a .tif and .vrt per band in the temp directory.

This change computes the same averages as matrix multiplications on each dask chunk and adds them up across chunks, so nothing is written to disk and memory holds a few chunks rather than whole bands.

Notes

1. The output matches GDAL's average resampling exactly and the new unit tests checks this directly.
2. Because of ^^ there is no drift in outputs and this can be thought of as a performance optimization which doesn't require cache bumps or wipes.

Manually testing

I ran for the HND and TWN tiles, saw a runtime of 2 minutes instead of 5 and peak memory of ~3 GiB, and verified that the results were identical for all bands.

[blallen]

The implementation here looks good, just got worried about off by one and cell boundary errors. I found a few cases myself, codex found some others with orientation and etc. Posted examples of tests and/or fixes to help out. I think all of this is mostly edge cases, but they feel like the kind that would be nasty to debug without the extra guards.

[jeffseif]

a453c1b