Skip to content

Fuse FITS postprocessing with runtime CLI and API controls - #74

Open
tpn wants to merge 3 commits into
mainfrom
codex/compression-xdr-20260929
Open

tpn wants to merge 3 commits into
mainfrom
codex/compression-xdr-20260929

Conversation

@tpn

@tpn tpn commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

FITS GZIP_2 loading currently creates a full-plane intermediate for separate unshuffle, byte-order conversion and scatter passes. Fuse those operations by default, with postprocess="auto"|"fused"|"separate" and --xdr-postprocess for runtime comparison.

Carry optional xdr_options through shared FITS reads, xFit/xPois/xRep/xScan APIs and CLIs, input manifests, workers and reader receipts. Omitted flags preserve input choices; explicit flags override matching fields. Existing reader policies remain intact.

Validation: 367 GPU xDR tests plus 83 reader/lifetime checks after the completion-protocol correction; 727 shared/component CPU tests and 93 xRep tests; full lint, typing and hooks. Original two-file/six-plane reads (390.9 MB) match byte-for-byte across all three postprocess choices and both native/Python batching. Measured fusion saves 17–22% of postprocessing and one 65.2 MB intermediate per 32-bit plane; the isolated full-reader comparison was unchanged.

This is the foundation for stacked PRs #76 and #77.

@copy-pr-bot

copy-pr-bot Bot commented Sep 29, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: NVIDIA/cuPhoton/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7d6c3a0f-95ce-48c7-94c0-694c30421daf

📥 Commits

Reviewing files that changed from the base of the PR and between a0f728e and 3c7e2c0.

📒 Files selected for processing (3)
  • docs/components/xfit.md
  • docs/components/xscan.md
  • tests/core/test_cli_contract.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.


📝 Walkthrough

Walkthrough

The change adds auto, fused, and separate xDR postprocessing modes. It carries xDR options through FITS reads, command-line interfaces, workflows, manifests, and worker pipelines. The xDR reader adds a fused tile-scatter kernel and retains a separate processing path.

Changes

Configurable xDR postprocessing

Layer / File(s) Summary
Shared options and FITS reader entry point
src/cuphoton/core/fits_options.py, src/cuphoton/core/cli/fits.py, src/cuphoton/core/fits_io.py, tests/core/*, tests/core/test_cli_contract.py, docs/components/xdr.md
Defines and validates supported xDR options, adds shared CLI conversion helpers, and forwards normalized options through FITS reads. Non-empty options are included in read metadata.
GPU postprocessing modes and benchmark
src/cuphoton/xdr/*, tests/xdr/*
Adds fused tile restoration and an explicit separate path. Batch APIs and benchmarks accept and record the selected mode. GPU tests cover pixel values, compression formats, crops, and output layout.
xFit and xPois FITS workflows
src/cuphoton/xfit/*, src/cuphoton/xpois/*, tests/xfit/*, tests/xpois/*, docs/components/xfit.md, docs/components/xpois.md
Passes xDR options through xFit manifests and loading, and through xPois commands, batch options, and FITS input workflows.
xRep image, mask, and stack reads
src/cuphoton/xrep/*, tests/xrep/*, docs/components/xrep.md
Passes xDR options from commands and workflows to image, mask, and stack FITS reads, including benchmark workflows.
xScan manifests, workers, and benchmarks
src/cuphoton/xscan/*, tests/xscan/*, docs/components/xscan.md
Merges caller and manifest options, carries them in FITS descriptors and worker paths, and validates options in read receipts and grouped FITS inputs.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 3c7e2

The PR adds selectable xDR postprocessing controls and documents their use across FITS workflows. No specific user-impacting regression or merge blocker is established by the supplied evidence.

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@tpn tpn changed the title Fuse FITS tile byte conversion with GPU scatter Fuse FITS postprocessing with runtime CLI and API controls Sep 30, 2026
@tpn
tpn marked this pull request as ready for review September 30, 2026 03:21
@tpn
tpn requested a review from melo-gonzo as a code owner September 30, 2026 03:21
@tpn tpn added the ai-review Request a focused CodeRabbit review label Sep 30, 2026
@tpn

tpn commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@melo-gonzo melo-gonzo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. All 79 postprocessing GPU tests passed on RTX 6000 Ada; option propagation and backward compatibility checks found no blocking regressions. Recommend landing with #76 after 0.1.3.

tpn added 3 commits September 30, 2026 21:42
Signed-off-by: Trent Nelson <trentn@nvidia.com>
Default to fused pixel restoration and retain a separate-kernel option.
Carry explicit xDR options through workflow inputs, workers and reader
receipts while preserving omitted manifest choices.

Signed-off-by: Trent Nelson <trentn@nvidia.com>
Keep xFit configuration identities stable when no xDR overrides are
supplied. Reject unsupported output strides with runtime checks and
exercise the valid int32 quantization layout. Clarify the separate
postprocessing path and its input-preserving copy cost.

Signed-off-by: Trent Nelson <trentn@nvidia.com>
@tpn
tpn force-pushed the codex/compression-xdr-20260929 branch from a0f728e to 3c7e2c0 Compare October 1, 2026 05:05

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-review Request a focused CodeRabbit review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants