Repository navigation
Retire in-code catalogue versioning and the pre-Snakemake run layers - #880
Merged
Merged
Conversation
cailmdaley
force-pushed
the
feat/persist-exp-products
branch
from
September 9, 2026 13:00
0d7c313 to
d91da36
Compare
cailmdaley
force-pushed
the
feat/retire-versioning
branch
from
September 10, 2026 00:46
77af691 to
8ac4caa
Compare
Contributor
Author
|
From the smk-g7 acceptance run (integration of #879 + #886 + #887 + this branch, 64 tiles / 127 exposures): — Claude (Fable) on behalf of Cail |
cailmdaley
force-pushed
the
feat/retire-versioning
branch
from
September 26, 2026 09:26
96d54ab to
54e4166
Compare
cailmdaley
changed the base branch from
feat/persist-exp-products
to
develop
September 28, 2026 16:33
ccd_psf_handler (v1.3/v1.4 -> 7, v1.5 -> 8, v1.6 -> 9 patches), its get_ccds_with_psf entry points and the build_and_plot_coverage_maps.sh VERSIONS loop go, with utilities/summary.py, summary_params_pre_v2.py, summary_run and summary_tiles.py: a per-patch scrape of a finished run whose job is done by the workflow's manifests, logs and completeness.py. collate_star_cat.py keeps only its version-free path (no -V/-P, no patch loop). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BeQNTBcjTqu6TPktxysovQ
The gen-0 HTCondor-era scripts and the gen-1 curl/skaha submission layer
(${kind}_runs/$ID layout, bit-coded job masks, patch-specific special
cases) go, with canfar_submit.py, canfar_submit_job and the skaha
dependency. canfar_monitor/canfar_monitor_log stay. The gen-2 bash
(run_job_sp_canfar_v2.0.bash, job_sp_canfar_v2.0.bash, functions.sh's
command()) stays: sp_validation's image-sims workflow calls it. The
Dockerfile's script-symlink loops tolerate an empty glob.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BeQNTBcjTqu6TPktxysovQ
merge_headers, psfex_interp and vignetmaker lose the single-run-root branch resolved through log_run_sp.txt (last:/all:): EXP_BASE_DIR and ME_*_EXP_DIR are required, and a config without them fails loudly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BeQNTBcjTqu6TPktxysovQ
docs/source/workflow.md is the user-facing workflow guide; pipeline_tutorial.md follows the rule chain; pipeline_canfar.md and post_processing.md go with the layers they described. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BeQNTBcjTqu6TPktxysovQ
cailmdaley
force-pushed
the
feat/retire-versioning
branch
from
September 29, 2026 08:09
496d67b to
b88c389
Compare
cailmdaley
changed the title
…rams pin # Conflicts: # docs/source/pipeline_canfar.md # scripts/python/merge_final_cat.py # workflow/README.md
cailmdaley
marked this pull request as ready for review
October 25, 2026 23:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We retire catalogue-version patch tables, post-hoc run summaries and the CANFAR submission layers, leaving Snakemake as the production orchestration path.
A campaign is a tile list; git tags identify the producing code rather than runtime catalogue-version switches.
Retirement scope
-V/-Poptions, summary scraping and summary notebooks.skahadependency. Retire thecanfar_submit_job,summary_runandget_ccds_with_psfentry points.collate_star_cat.py's PSFEx path; remove its MCCD catalogue-collation branch and version-dependent patch loop.merge_headers, multi-epochpsfex_interpand multi-epochvignetmaker:EXP_BASE_DIR,ME_DOT_PSF_EXP_DIRandME_IMAGE_EXP_DIR, respectively. Legacy run-log lookup isn't a fallback.Completeness manifests and
sp reportprovide run status; exposure coverage products provide campaign maps.The header-download and corner-extraction coverage utilities remain available.
Compatibility boundaries
The Gen-2 scripts
run_job_sp_canfar_v2.0.bash,job_sp_canfar_v2.0.bashand their shared command helper remain installed because the downstream image-simulation workflow calls them.canfar_monitorandcanfar_monitor_logremain available.create_final_cat.pyretains thepatches/<run>group read downstream.This is a breaking retirement for callers of the removed submission, summary, per-tile merge and patch-based collation tools; it doesn't supply a compatibility shim.
It doesn't change shape-measurement algorithms or scientific defaults.
The shared completeness, ngmix-range and shell-prologue helpers are left unchanged, avoiding a workflow rerun-trigger change for this retirement.
— Codex on behalf of Cail.