Skip to content

Handle legacy MPHeight NaNs as NULLs (skip estimator) to ensure full well migration #541

@ksmuczynski

Description

@ksmuczynski

Summary

Legacy WellData.MPHeight contains NaN values. Current transfer logic treats NaN as a real value, bypasses estimation, and then fails at DB insert because MeasuringPointHistory.measuring_point_height is Numeric. We want
migration completeness without using the estimator (it’s not trustworthy right now). Proposed fix: treat NaN as missing and insert NULL MP height for those wells, with nullable history during migration.

Why this matters

  • NaN values prevent some wells from transferring.
  • Estimator currently uses early measurements and may be inaccurate.
  • We need all wells migrated even if MP height is unknown.

Proposed changes

  • Transfer logic: treat NaN as missing and do not call estimator for those records.
  • Allow MeasuringPointHistory.measuring_point_height to be NULL during migration (migration already exists).
  • Read‑path hardening: ensure API/property access handles NULL history values.

Files

  • transfers/well_transfer.py
    • Detect NaN (pd.isna) and set MP height to None. Skip estimator for NaN cases.
  • db/thing.py
    • measuring_point_height and measuring_point_description should skip null history entries.
  • schemas/thing.py
    • Ensure response types allow measuring_point_height: float | None.
  • db/measuring_point_history.py
    • Confirm column nullable matches migration.
  • (Optional) transfers/util.py
    • Guard estimator against NaN if still used elsewhere.

Acceptance criteria

  • All legacy wells transfer even when MPHeight is NaN.
  • No estimator calls for NaN MP height in well transfer.
  • API responses tolerate null MP height without errors.
  • Tests updated or added to validate null MP height behavior.

Notes

  • Do not change column type to Float; keep Numeric for integrity.
  • Estimator improvements (using most recent DTW/DTW‑BGS) are out of scope for this issue.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions