Skip to content

Tile-level defect flags: exposure IMAFLAGS_ISO and interpolated-defect epochs on each object's footprint - #973

Merged
cailmdaley merged 11 commits into
developfrom
feat/defect-flags-tile
Oct 26, 2026
Merged

cailmdaley merged 11 commits into
developfrom
feat/defect-flags-tile

Conversation

@cailmdaley

@cailmdaley cailmdaley commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

We add exposure-defect diagnostics on each object's coadd segmentation footprint, leaving fits and stamp-wide diagnostics unchanged.
The columns distinguish defects overlapping the object from defects elsewhere in its stamp.

Catalogue columns

  • NGMIX_IMAFLAGS_ISO preserves the bitwise OR of exposure instrument flags on tile_seg == obj_id, across epochs passing the masked-fraction cut and central veto.
  • NGMIX_N_EPOCH_INTERP_ISO counts those kept epochs with an interpolated defect on that footprint. An unflagged zero-weight pixel can contribute to this count without adding an instrument bit.

We MegaCam-flip the tile catalogue's int32 SEG_VIGNET into each epoch orientation without reprojection, matching the existing overlay.
Clean quarter-turn weight copies don't count as defects.
Both columns are −1 for missing, empty or edge-clipped footprints; never-fit final rows carry 0 and require separate fit-status checks.
Non-integral or out-of-range instrument flags on a known footprint are refused rather than silently cast.
The semantics are recorded in ASTRA's shape_measurement.defect_flagging decision and the committed universe.
No additional product file or catalogue selection cut is introduced.

Real-tile check and limits

A no-fitting check on tile 188.307 used seven locally available exposures out of nine, with privately regenerated inputs and production mounts read-only.
Of 36,051 detections, 374 had no kept epoch; among the remaining 35,677, 621 had edge-clipped unknown footprints and 35,056 had known footprints.

Diagnostic Count / denominator Rate
Interpolation on the footprint 173 / 35,056 known footprints 0.4935%
Instrument bits on the footprint 174 / 35,056 known footprints 0.4963%
Interpolation anywhere in a kept stamp 12,943 / 35,677 objects with kept epochs 36.2783%

Before epoch cuts, 6.05% of objects had instrument flags on their footprint in at least one epoch, compared with 6.97% positive defect-map area.
Epoch rejection removes most affected epochs; 78% of footprints lie wholly inside the 7-pixel interpolated-defect veto radius.
The residual flag therefore measures overlap with defects in surviving epochs, not every object a defect touched.
These are detection-level rates on one tile, not shear-sample rates or shear-calibration results.

The validation note, JSON summary and read-only diagnostic script give denominators, spatial checks, limitations and reproduction paths.
Fixtures cover epoch cuts, footprint sentinels, CCD flips, bit preservation, integer round trips and final-catalogue propagation.
Workflow guards verify segmentation delivery for data and image simulations under both blend treatments.

Existing campaigns aren't backfilled; their cleaned vignet inputs would need locating or regeneration before these diagnostics could be computed.

— Codex on behalf of Cail.

@cailmdaley
cailmdaley force-pushed the feat/defect-flags-tile branch from c45de2f to 3ea9bc1 Compare October 24, 2026 13:32
@cailmdaley
cailmdaley changed the base branch from feat/masked-pixels to feat/uberseg-seg-vignet October 24, 2026 13:32
@cailmdaley
cailmdaley marked this pull request as ready for review October 24, 2026 15:53
cailmdaley and others added 8 commits October 25, 2026 18:38
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ct footprints

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ey check

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ery handling

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eration plan

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measure real epoch preparation without fitting, record denominators and bit counts, and add a spatial overlay and conditional Poisson interpretation. Cache read-only stores and exposure WCS decoding for bounded validation runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cailmdaley
cailmdaley changed the base branch from feat/uberseg-seg-vignet to develop October 25, 2026 22:38
@cailmdaley
cailmdaley force-pushed the feat/defect-flags-tile branch from d6f454b to 27009cc Compare October 25, 2026 22:40
Follow sp_validation.calibration.metacal._read_data_ngmix and _masking_gal (b2d8e329), which define Tpsf from NGMIX_T_PSF_RECONV for the relative-size cut. Keep diagnostic thresholds and all catalogue production unchanged. The regression first failed on the obsolete Tpsf field and verifies that logging leaves the saved FITS byte-identical.
@cailmdaley
cailmdaley merged commit c58954e into develop Oct 26, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant