PSF star selection: apply the manual mask; M33 shows outlier rejection fails where contaminants dominate #993

Summary

Six crowded or extended objects sit in the run-1 PSF star sample: M33, M101, M13, M92, and the Draco and Ursa Minor dwarf spheroidals. Run-1 PSF star selection cuts only on instrument flags (masking.psf_star_mask_veto = instrument_flags_only). That default was a deliberate measure-first choice in #847 (43c32cd): star selection "starts from outlier rejection alone, and MASK_EXT is the configurable pickup if that proves insufficient — write the column, measure the effect, impose it only if the measurement says to." This issue is that measurement, along two axes:

• (b) Is outlier rejection good enough? Yes for crowding; no for M33. In M13, M92, Draco and Ursa Minor, selection keeps real stars and the PSF model fits them like the surrounding field. In M33's inner ~0.5°, selection admits HII regions and clusters (24% of "stars" inside 0.15° are >1.2× the PSF size), and the PSF model itself is biased: normal-size stars come out smaller than the model there. Only M33 corrupts the PSF model.
• (a) Is the manual mask enough? Yes. Applying only MASK_8_Manual to PSF star selection removes 1,024 of M33's 1,025 oversize detections within 1°, and every radius where the model is biased. No new DSO veto or circle list is needed. Draco and Ursa Minor aren't in the manual mask, and they don't need to be.

Measurement

The sample is the run-1 PSF validation stars (the stars behind ρ/τ). "Normal-size" means |dT/T| < 0.2, and the control is a 1.0–1.5° annulus around each object. The footprint is MASK_8_Manual from the DR6 healsparse mask.

• M33's model is inflated, not just its sample contaminated. Stars of normal size, which are not contaminants by construction, are 2.4% smaller than the model inside 0.15°, 1.2% at 0.15–0.3° and 0.7% at 0.3–0.5°. They're back to the control by 0.5–0.7°. Oversize sources in the training set would pull the model this way (inferred: training stars aren't persisted, but they pass the same selection as the 20% held out for validation). In the cosmic-shear leakage fit, this region carries >99% of the first-bin ρ1/ρ3/ρ4 variance and drives the leakage coefficients α and η negative.
• Crowding is handled. M92, Draco and Ursa Minor are 3–10× overdense in star detections with no oversize detections and normal residuals. In M92's dense core, blending leaves no PSF stars at all.
• Beyond bit 8 around M33: in the 0.5–0.7° ring, outside the manual ellipse (≈0.55° × 0.40°), normal stars show 1.15× the control δe rms with no size bias and no oversize detections. The galaxy side there is 1.25–1.35× overdense in kept galaxies (plausibly outer-disc sources). It's a small effect and outside this issue's star-selection scope, so it's noted rather than acted on.
• Survey-wide, a bit-8 veto removes 51,663 of 10,380,311 PSF stars (0.5%).

Before #846 (eb93838), ShapePipe's internal mask module wrote Messier/NGC discs into the exposure flags, so these regions were vetoed via IMAFLAGS_ISO. Every run-1 campaign descends from eb93838. The NGC/Messier corner-overlap fix (#827) is unrelated.

Proposed change

1. ShapePipe, next run: add a manual_mask_veto option to masking.psf_star_mask_veto and adopt it. Point mask_query MASK_PATHS at the bit-8 (manual) map and add MASK_EXT == 0 to [MASK:star_selection]. This keeps M33's contaminants out of the PSFEx fit itself, which a downstream veto can't do. The star-body map (star_body_veto) is no substitute: it flags 12.3% of PSF stars.
2. Run 1, no rerun: downstream, drop PSF stars inside bit 8 before ρ/τ. This replaces hand-placed contamination circles with the same mask the galaxies already use. The biased model in M33 sits inside bit 8, where galaxies are already cut, so no rerun is needed.

Claude on behalf of Cail