Add robust DICOM de-identification step (tag scrubbing + burned-in PHI + optional defacing)

[Liohtml]

Part of #51. Foundational privacy stage — a prerequisite for safely sharing,
logging, or (optionally) sending any imaging data externally.

Summary

Add an optional but strongly-recommended pipeline step that de-identifies
DICOM data: (1) header tag scrubbing per the DICOM standard, (2) burned-in
pixel PHI detection/redaction, and (3) optional defacing of head MRI/CT.
The current preprocess step mentions "Anonymize" in the README diagram but
there is no actual de-identification implementation.

Why (research-backed)

• Header scrubbing alone is insufficient. DICOM images frequently contain
  burned-in PHI — patient name/MRN/DOB rendered into the pixels (common on
  ultrasound, secondary captures, dose sheets, some MR/CT). Removing DICOM tags
  does not touch pixels; a separate pixel-level pass (OCR/vision or manual
  review) is required. Confirmed against primary de-id literature + TCIA/MD.ai docs.
• Head MRI/CT can be facially re-identifiable (3D facial reconstruction), so
  truly anonymizing brain scans may require defacing — relevant under GDPR
  (special-category data; pseudonymized ≠ anonymized) and German BDSG §22/§27.
• Legal framing: when a patient self-uploads their own scan, US HIPAA often
  doesn't apply, but the FTC Health Breach Notification Rule (2024) does, and
  EU GDPR Art. 9 applies the moment the app processes the data. De-identification
  is the core technical safeguard that makes logging, caching, second-opinion
  sharing, and any external LLM call defensible. Pairs with the consent gate ([repo-monitor] High: Patient PHI/DICOM data transmitted to external LLM APIs without consent controls #27)
  already merged.

Proposed implementation

• New file src/medcheck/pipeline/deidentify.py → DeidentifyStep(PipelineStep),
  registered as "deidentify"; runs after ingest, before anything that
  logs/transmits (especially before vision_analysis's external path).
• Tag scrubbing: implement against the DICOM PS3.15 Basic Application Level
  Confidentiality Profile (+ options: Retain Longitudinal Temporal as config,
  Clean Descriptors). Remove/replace the HIPAA Safe-Harbor-relevant tags (names,
  dates→year, IDs, device serials, institution, private tags, UIDs re-mapped
  consistently). Evaluate reusing a maintained library (e.g. dicognito, MIT)
  vs. a hand-rolled tag list.
• Burned-in PHI: detect text in pixel data (lightweight OCR behind an optional
  ocr extra) and redact bounding boxes; honor/repair the BurnedInAnnotation
  tag but don't trust it. Default to flag + warn if OCR isn't installed.
• Defacing (optional): integrate an Apache/MIT defacer for head MRI/CT behind
  an optional extra; off by default, on for detected head studies when enabled.
• Free-text: if a report is ingested (see reconciliation Add report reconciliation: compare official radiologist report against image analysis #58), scrub PHI from
  the text too (NLP/regex pass).

Design constraints

• Fail-safe, not fail-soft, for transmission: if de-identification is requested
  but cannot complete (e.g. OCR missing on a study with suspected burned-in PHI),
  the external-LLM path must refuse to transmit rather than leak — tie into the
  [repo-monitor] High: Patient PHI/DICOM data transmitted to external LLM APIs without consent controls #27 consent gate.
• Lazy imports for OCR/defacing extras; core tag-scrubbing has no heavy deps.
• Deterministic UID remapping so series/study relationships survive.

Acceptance criteria

• DeidentifyStep registered; removes/replaces the PS3.15 tag set; consistent
  UID remapping.
• Burned-in PHI: OCR-based redaction when the extra is installed; clear
  warn-and-block behavior when it isn't and transmission is requested.
• Optional defacing for head studies behind an extra.
• Integrates with the [repo-monitor] High: Patient PHI/DICOM data transmitted to external LLM APIs without consent controls #27 consent gate: no external transmission of
  not-de-identified data.
• Tests: tag removal on a synthetic dataset; UID consistency; burned-in
  detection on a fixture with rendered text (mocked OCR ok); fail-safe path.
• Docs: what is/isn't removed, GDPR/HIPAA/FTC framing, limitations.

Effort

~2–3 days (tag scrubbing first; OCR + defacing as follow-ons). Optional heavy deps.

References (session research)

DICOM PS3.15 confidentiality profiles; HIPAA Safe Harbor (45 CFR §164.514);
burned-in PHI de-id literature (PMC11522224); GDPR Art. 9 / German BDSG §22,§27;
FTC Health Breach Notification Rule (2024).

[Liohtml]

Assessment

Value: High — this is not just a nice-to-have feature. For any HIPAA-covered entity or GDPR-regulated user, sending un-de-identified DICOM data to a cloud LLM is a data breach waiting to happen. MedCheck is already correctly consent-gating the LLM transmission (VisionAnalysisStep consent gate), but consent gating does not replace de-identification. A user who clicks "allow cloud LLM" still transmits patient name, DOB, and MRN in the DICOM metadata if no de-ID step runs first.

Complexity: Significant — but mostly in DICOM tag scrubbing. pydicom makes the mechanics straightforward; the hard part is getting the tag list right per DICOM PS3.15 Attribute Confidentiality Profile.

Fits architecture: Yes — this fits cleanly as a new deidentify pipeline step, inserted between ingest and preprocess.

My take

This should be a blocker for any production release of the cloud LLM path. The current consent gate prevents accidental transmission, but it does nothing to remove PHI from the data that is transmitted when the user consents. A user who consents is still transmitting their patients

[Liohtml]

Assessment

Value: High — this is not just a nice-to-have. For any HIPAA-covered entity or GDPR-regulated user, sending un-de-identified DICOM data to a cloud LLM is a data breach waiting to happen. MedCheck already consent-gates the LLM transmission (VisionAnalysisStep consent gate in pipeline/vision_analysis.py), but consent gating does not replace de-identification. A user who clicks "allow cloud LLM" still transmits patient name, DOB, and MRN embedded in the DICOM metadata.

Complexity: Significant, but mostly contained to a new pipeline step. pydicom makes the DICOM tag mechanics straightforward; the hard part is getting the tag list right per DICOM PS3.15 Attribute Confidentiality Profile.

Fits architecture: Yes — a new deidentify step inserted between ingest and preprocess fits cleanly into the existing StepRegistry / WorkflowEngine pattern.

My take

This should be a blocker for any production release of the cloud-LLM path. The current consent gate prevents accidental transmission but does nothing to remove PHI from the data that IS transmitted when the user consents. A user who says "yes, send to Claude" is still sending their patient's full name and DOB to Anthropic unless a de-ID step runs first.

If we build this

1. New step: src/medcheck/pipeline/deidentify.py implementing DeidentifyStep that walks all context.dicom_series and scrubs the PS3.15 Table E.1-1 tag set from every slice dataset.
2. Tag scrubbing: Remove or blank PHI tags (PatientName, PatientID, PatientBirthDate, InstitutionName, ReferringPhysicianName, etc.). Key: preserve SeriesDescription (needed for anatomy detection) and Rows/Columns/BitsAllocated (needed for pixel extraction).
3. Registration: Add deidentify to StepRegistry in main.py; insert it into default.yml and quick.yml between ingest and preprocess.
4. Report caveat: Generated reports must state clearly that standard DICOM metadata was removed but burned-in pixel-level PHI (text overlays on localizer images) was not processed.
5. Burned-in PHI (v2): OCR-based detection on scout/localizer images is considerably more complex — reasonable to defer, but must be documented as a known limitation.

Concerns

• The step must run before any data reaches the LLM but after anatomy detection populates series.description from the raw DICOM tag. Since detect_anatomy runs in PreprocessStep and reads from series.description (set at ingest time), the deidentify step can safely scrub DICOM dataset tags without breaking anatomy detection — the description is already captured in the DicomSeries dataclass.
• The consent gate in VisionAnalysisStep should also check whether de-identification has run and warn if sending to a cloud provider without it.

Strong +1 for prioritising this before any public cloud-LLM promotion.