Skip to content

Clean up in-place operations - #452

Merged
dfulu merged 3 commits into
dev_feb2026_speedupsfrom
part_24
Oct 21, 2026
Merged

dfulu merged 3 commits into
dev_feb2026_speedupsfrom
part_24

Conversation

@dfulu

@dfulu dfulu commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Pull Request

Description

This PR cleans up our in-place operations

  • We return None when an operation is done in-place to signify an in-place operation
  • Updated docstrings and comments

Checklist:

  • My code follows OCF's coding style guidelines
  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked my code and corrected any misspellings

@@ -6,10 +6,11 @@


def diff_channels(da: TArray, accum_channels: list[str]) -> TArray:

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function was a bit weird in that it replaced the data on the input dataarray and then did an isel(). When the input is an xarray.DataAray the isel returns a copy, so we were first changing the data in-place and then returning a copy of that changed data. That was kind of the worst of both worlds. This new version is explicitly out-of-place

da_dask = xr.DataArray(dask.array.random.random((5, 5)))

# Create a nested dictionary with dask array
lazy_data_dict = {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do an in-place operation to load this so I don't think the name makes sense anymore

@felix-e-h-p
felix-e-h-p self-requested a review October 20, 2026 13:59

@felix-e-h-p felix-e-h-p left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small suggestion. Could we have a test that pulls the same sample twice from PVNetDataset (with dropout and a fill value set) and checks that datasets_dict is unchanged? That would lock in the “without modifying the original NumPy arrays” from the new docstring.

@dfulu

dfulu commented Oct 21, 2026

Copy link
Copy Markdown
Member Author

Small suggestion. Could we have a test that pulls the same sample twice from PVNetDataset (with dropout and a fill value set) and checks that datasets_dict is unchanged? That would lock in the “without modifying the original NumPy arrays” from the new docstring.

Sure, but I'd say that should be a unit test on preprocess_dataset_dict() rather than at the PVNetDataset level. The PVNetDataset is essentially only lazay arrays not NumPy arrays. So I think covering the lower level at preprocess_dataset_dict() is a better place for the test

@dfulu
dfulu requested a review from felix-e-h-p October 21, 2026 13:34
@dfulu
dfulu force-pushed the part_23 branch 2 times, most recently from 0f48e81 to 104fc22 Compare October 21, 2026 17:09
Base automatically changed from part_23 to dev_feb2026_speedups October 21, 2026 17:15
@dfulu
dfulu merged commit 47215d2 into dev_feb2026_speedups Oct 21, 2026
4 checks passed
@dfulu
dfulu deleted the part_24 branch October 21, 2026 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants