Skip to content

13 site IDs referenced by templates are missing from SiteID.tsv (drove 2,137 lost samples) #12

Description

@wilke

Summary

13 collection_site_id values are referenced by biosample templates but are not
defined in SiteID.tsv. They drove 68 truncated batches and 2,137 lost
samples
before the underlying SRA.py crash was fixed in 05eaf8d.

The crash is fixed; the missing site definitions are not. Affected samples
still submit with not collected for collected_by and ww_population.

The 13 sites, by reference count

Site Refs First seen
S0183 48 250626_Direct_301
S0185 44 250626_Direct_301
S0184 39 251008_Direct_314
S0187 34 251008_Direct_314
S0166 5 240423_Direct_227
S0164 5 240423_Direct_227
S0165 4 240612_Direct_237
S0167 3 240612_Direct_237
S0173 1 240606_Direct_236
S0172 1 240606_Direct_236
S0171 1 240606_Direct_236
S0170 1 240606_Direct_236
S0161 1 241107_Direct_263_264_265

S0183/S0184/S0185/S0187 alone account for the large majority — they look
like sites onboarded in 2024+ that were never propagated to the SiteID master.

Why it still matters

  1. Samples at these sites are submitted to NCBI with not collected where real
    collected_by and ww_population values exist upstream.
  2. SRA.py now degrades gracefully instead of truncating, so the failure is
    quieter than before — it shows up only as a WARNING Missing site IDs line
    that scrolls past.
  3. It compounds the mapping.py debug-print defect: an unmapped site also emits
    ID <n> not in sides straight into summary.tsv.

Suggested fix

Add the 13 rows to the canonical SiteID.tsv in /incoming/SRA_COVID_Temp/,
then re-generate the affected batches and confirm the not collected values are
replaced.

Longer term, an unresolvable collection_site_id should be a blocking condition
on submission rather than a warning, since it silently degrades data that is then
hard to correct at NCBI.

Provenance

Counts from reports/sra-audit-260714/truncation_audit/missing_sites.tsv;
re-confirmed by seqtrack. See reports/work-260730.seqtrack.md.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions