✨ Detect and correct global 2π branch errors in MCPC-3D-S - #32
Conversation
The global 2*pi branch for the MCPC-3D-S phase offset was picked by a cascade of five hardcoded field-magnitude thresholds, with a --wrap-limit escape hatch for when the cascade made things worse. Replace it with a criterion that has no tuned constants. A branch error of M wraps shifts the phase offset by wrap(2*pi*k*M) with k = TE0/dTE, and that lands as the same additive constant on every echo. Fitting a line through the two echoes and reading its intercept measures that constant directly: the correct branch extrapolates through the origin, a wrong one does not. The rejection scale is analytic -- |wrap(2*pi*k)| -- so nothing is fit to data. - Add _evaluate_branch / _branch_intercept_step / _select_branch; drop _select_branch_cascade, FMAP_PROPORTION_HEURISTIC, FMAP_AMBIGUIOUS_HEURISTIC, BRANCH_SELECTION_MODES and BRANCH_CONSISTENCY_RATIO. - Act only when exactly one candidate is consistent. On a tie the branches are a genuine alias that no phase statistic can resolve, so defer to ROMEO's correct_global rather than guess: a measured centre-frequency sweep put a smallest-|field| tiebreaker at 12% better and 12% worse, including cases where correct_global had been right. - Return 0 when TE0/dTE is an integer, where the branch cannot move the offset at all. - Drop wrap_limit from medic(), unwrap_and_compute_field_maps() and the wk-medic / wk-unwrap-phase CLIs. BREAKING: --wrap-limit is gone; it existed to disable the heuristics this commit removes. - Correct two annotations the new tests exposed: echo times are numpy scalars at every call site, not arrays, and rescale_phase's min/max are floats, not ints. - Add 13 tests covering the analytic step, the decision rule, and end-to-end recovery of an injected wrap on the bundled data. Validated on 14090 frames across 58 runs from eight OpenNeuro datasets spanning six TE0/dTE ratios (0.51-0.78). No frame selected a non-zero branch and the largest branch-0 intercept was 1.9e-07 of the cutoff, so the output is unchanged on all data tested. Fitting and non-fitting branches separate by six to nine orders of magnitude when the rule does fire, verified by injecting known wraps.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #32 +/- ##
==========================================
+ Coverage 95.01% 96.25% +1.23%
==========================================
Files 18 18
Lines 1265 1280 +15
==========================================
+ Hits 1202 1232 +30
+ Misses 63 48 -15 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR updates MEDIC’s MCPC-3D-S global (2\pi) branch selection by replacing the prior field-magnitude heuristic cascade with an intercept-based consistency test (no tuned constants), and removes the now-obsolete wrap_limit API/CLI surface. It also adjusts a couple of type annotations and adds targeted tests to validate the new decision rule and an end-to-end injected-wrap recovery case.
Changes:
- Implement intercept-based branch evaluation/selection (
_evaluate_branch,_branch_intercept_step,_select_branch) and remove the prior heuristic cascade and its tuning constants. - Remove
--wrap-limit/wrap_limit=from CLIs and library entry points that previously exposed heuristic disabling. - Add tests for the analytic intercept step, decision-rule branches, and end-to-end recovery on bundled data.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
warpkit/utilities.py |
Adjusts rescale_phase parameter annotations (int → float). |
warpkit/unwrap.py |
Adds intercept-based branch selection and removes heuristic-based wrap limiting. |
warpkit/scripts/unwrap_phase.py |
Removes --wrap-limit plumbing from wk-unwrap-phase. |
warpkit/scripts/medic.py |
Removes --wrap-limit plumbing from wk-medic. |
warpkit/distortion.py |
Removes wrap_limit from the public medic() wrapper call into unwrapping. |
tests/test_unwrap.py |
Adds unit tests for the new branch selector and an end-to-end injected-wrap recovery test. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Addresses review feedback on #32 and unblocks the lint job. - `mcpc_3d_s` documents `te0`/`te1` as `np.float32 | float` to match the scalar signature; `rescale_phase` documents `min`/`max` as floats. - Split `test_branch_selector_noop_when_nothing_fits` in two. The all-zero intercepts case is an all-way perfect fit that collapses the observed scale, not a failed fit, so it gets its own test and an accurate docstring. - Reformat the README's Python block: ruff 0.16 formats code in markdown, which fails `ruff format --check` on main as well as here.
Restores the smallest-|field| tiebreaker removed earlier in this branch. The
evidence for removing it came from an induced centre-frequency sweep, which
only exercises the regime where the prior is meant to lose -- a genuine global
field offset. It never covered the opposite regime, an artifactual one-wrap
flip, and that is the one that actually occurs.
Found on ds006131 sub-20828 (TEs 14.2/38.93/63.66 ms, k=0.5742, wrap 40.44 Hz).
That subject's global field sits at 17.7 Hz against a 20.22 Hz half-wrap
boundary, so correct_global's median of rounded wrap counts is on a knife edge
and tips over on individual frames: its ballot reads 0, +1, 0, -1 frame to
frame, and each tip flips the dual-echo field a full wrap for one frame.
The intercept test detects the flip -- the fitting set moves from {-1, 0} to
{0, +1} -- but both survivors score ~1e-7, so it cannot rank them. The prior
can, and correctly: branch +1 restores +17.7 Hz against branch 0's -22.8 Hz.
It is the same rule correct_global applies, on a better conditioned statistic
(magnitude-weighted, eroded mask, on the field itself rather than an unweighted
median of rounded counts over a dilated mask), which is what makes it stable
where correct_global is not.
Full pipeline on that run, before and after:
before field +16.46..+53.36 Hz max step 35.939 Hz 34 wrap-sized steps
after field +16.38..+19.62 Hz max step 1.462 Hz 0 wrap-sized steps
19 of 243 frames were single-frame ~40 Hz excursions; all are gone, and the
224 healthy frames are untouched.
- Restore _weighted_median and the tie branch of _select_branch.
- Rewrite the docstring around this measurement instead of the sweep.
- Replace the two deferral tests with ones built from the real frame-44
intercepts and fields.
Pins the failure that motivated keeping the field prior, using the real data
rather than hand-written numbers.
Two frames from ds006131 sub-20828 run-01 (TEs 14.2/38.93/63.66 ms, k=0.5742,
wrap 40.44 Hz): the original frame 43, which is healthy, and frame 44, one of
19 frames out of 243 where correct_global's ballot tips and the dual-echo field
jumps a full wrap. The selector must leave the first alone and return +1 on the
second.
- tests/data/branch_flip/ (9.7 MB, excluded from the sdist by the existing
tool.scikit-build exclude of tests/data).
- Cropped to the brain bounding box, which reproduces the behaviour exactly.
Deliberately NOT decimated: correct_global's median of rounded wrap counts is
on a knife edge here, so resampling tips it the other way -- 2x decimation
makes the flip vanish on frame 44 and appear on the healthy frame 43. That
would silently invert what the test checks.
- Also pins that the broken frame is a genuine {0, +1} tie on the intercept, so
a classify-only rule provably cannot fix it and the prior is load-bearing.
The tie-breaking prior asks which candidate branch puts the global field
closest to zero. It was estimating that level with a magnitude-weighted median
over an eroded brain mask. A median is the wrong statistic for this quantity.
The field distribution over a brain is a sharp peak at bulk-tissue value with
long tails from sinuses, dropout and edges. The scanner's frequency adjustment
centres that peak near zero -- not the median-including-tails -- so a median
estimates something the shim never set, and is pulled toward whichever tail is
heavier. That is the same defect that makes correct_global's own estimate tip
over: its dilated mask simply has more tail.
Margins to the +/-1/(2*dTE) flip boundary, ds006131 sub-20828, worst of the
healthy/broken frame pair:
correct_global (median, dilated) -0.26 Hz flips
median, eroded 1.25 Hz
magnitude-weighted median, eroded 2.57 Hz (previous)
half-sample mode, eroded 10.27 Hz (this commit)
Across all 243 frames of that run the two estimators pick identically -- same
18 frames corrected, zero disagreements -- and the full pipeline output is
unchanged, but the worst-case margin goes from 1.51 Hz to 9.34 Hz.
The half-sample mode takes the shortest interval containing half the remaining
points, recursively. It has no bin width and no bandwidth, so it introduces no
tuned constant; a binned mode agrees with it to within 9.8-11.2 Hz of margin
across a 64x range of bin counts, confirming the choice is not load-bearing.
It is also cheaper than the weighted median it replaces (2.8 ms vs 6.9 ms at
120k voxels) because it never sorts a weight array.
Derivation only -- no dataset names, measurements or validation numbers. Every quantity is a function of the echo times or the signal model, so it stays valid independent of what we happen to have tested against. Covers: - The signal model and the two derived scales, wrap spacing 1/dTE and the half-wrap 1/(2*dTE) that any smallest-|f| prior decides against. - The two ambiguities, each with a proof. The exact alias (theta - W(2*pi*k*N), f + N/dTE) reproduces the wrapped data identically and is therefore undetectable; an offset-branch mismatch instead lands a constant W(2*pi*k*M) on every echo and is. - Why extra echoes buy nothing: with even spacing every pair aliases at the same period, so five echoes carry as much branch information as two. - The intercept criterion, and why its rejection scale |W(2*pi*k)| is analytic rather than tuned. - Why the intercept and the field prior are both required -- the intercept is blind to a sign-preserving shift of the field, the prior to a sign-reversing one, so neither is sufficient alone. - Why a median is the wrong estimator for the prior. Writing the field distribution as (1-eps)*p_bulk + eps*p_tail, the mode is consistent for the bulk centre while the median is not, and mask dilation raises eps. The half-sample mode estimates the mode without a bin width or bandwidth. - Structural limits: ambiguity A is unaddressable by construction, dTE is bounded below by the EPI readout and above by T2*, the two ultra-high-field effects act in opposite directions, and a whole-turn shift of a multi-echo unwrapper's template echo is a pure slope change that no intercept test can see. Source is committed alongside the PDF so it stays regenerable with `typst compile notes/branch-selection-theory.typ`.
The first draft was written from the inside out: it opened on "branch selection", which presumes the reader already knows what a branch is and why one would be selecting among them. Rewritten for someone meeting the problem for the first time, and organised around the ambiguities rather than around the procedure that resolves them. The spine is now a taxonomy. Removing an estimated offset leaves a residual psi_e that should be a straight line through the origin; fitting it gives an intercept c and a slope s, and every possible error is a displacement in that (c, s) plane. Exactly three cases matter: A offset and field both wrong, in a matched way -> data unchanged, invisible B offset wrong alone -> constant c, detectable C field wrong alone -> pure slope, c stays zero Branch selection then falls out as the answer to B rather than being the premise, and the limits of the approach are visible as the cases the taxonomy already accounts for. Ambiguity C -- a whole-turn error in a multi-echo unwrapper's template echo, which propagates proportionally to echo time -- is promoted from a footnote to a peer of the other two, since it is exactly the failure an intercept test cannot see. Also opens with what phase is and why it is ambiguous at all, so no pipeline knowledge is assumed, and renames the file to match the new framing.
- Cut the gradient-echo primer. The reader is assumed to know what MR phase is; what needs motivating is why the coil offset has to be recovered at all. The opening now makes that case directly: field mapping wants the rate of phase accumulation, theta is the constant it accumulates from, and theta cannot be ignored because it varies rapidly in space and so breaks the smoothness that spatial unwrapping depends on. - Drop the now-redundant restatement of theta in the signal model. - Byline and date; author metadata instead of the project name.
Reverts the field-prior estimator to a magnitude-weighted median. The mode regressed a run that the median handled correctly, caught by the corpus scan before this left the branch. On ds005354 sub-02 run-00 (k=0.5089, wrap 36.35 Hz) the mode made 108 of 223 frames select a non-zero branch, producing a 32.7 Hz step in the field time series. The median selects branch 0 on all 223 and the run is flat. That is the only run in the corpus scan with a wrap-sized step under either estimator, and the median baseline has none. The cause is the assumption the mode carries. That subject's field distribution is bimodal -- peaks 21 Hz apart at 95% and 100% of maximum -- so the argmax flips on noise between adjacent frames. Magnitude weighting does not separate peaks that close (they stay within 5%), and restricting to high-SNR voxels made it worse, swinging between the two peaks on consecutive frames. The general lesson, now recorded on the function: continuity matters more here than freedom from bias. The estimate feeds a threshold comparison against 1/(2*dTE) that must be stable frame to frame, and a median is a continuous functional of the data where any argmax is not. The earlier claim of a 4x margin improvement came from a single subject and did not generalise -- the mode-minus-median offset is subject-specific in sign as well as magnitude. A continuous alternative that avoids both defects, scoring candidates by field mass at zero, was also tested. It reproduces the median's decisions on both stress cases but does not improve on them, so it was not adopted either. Also drops the mode material from notes/phase-offset-ambiguities.typ and replaces it with the estimator actually used, plus the continuity requirement that rules the alternatives out.
It was never observed, and the derivation behind it was wrong. The claim was that a whole-turn error in a multi-echo unwrapper's template echo propagates to echo e scaled by t_e/t_0, i.e. proportionally to echo time, and that a proportional displacement is a pure slope change with exactly zero intercept -- so no intercept test could ever see it. The proportional step does not survive the implementation. Non-template echoes are unwrapped by unwrap_voxel (include/romeo/algorithm.h), which returns new - 2*pi*round((new - old)/2*pi): the correction is a whole number of turns. Shifting the reference therefore shifts echo e by 2*pi*round(t_e/t_0) turns, not 2*pi*t_e/t_0. For TEs 14.2/38.93/63.66 those ratios are 2.742 and 4.483, which round to 3 and 4 -- not proportional to echo time, so the intercept is not zero and an intercept test would in fact partially see it. Nor was it ever seen in data. Whole-run scans counted frame-to-frame steps at 1000/TE0 Hz, its predicted signature, and found none on any run of any protocol. The one supporting measurement I had -- the template echo's pre-correction wrap count going non-zero on some frames -- shows correct_global doing its job rather than failing at it, and is not evidence of the failure at all. The taxonomy is now the two cases the propositions actually establish: offset and field wrong together (invisible), and offset wrong alone (a constant on every echo). Both are proved, and B is the one the selector acts on.
The dual-echo field is read from phase1 - phase0, whose noise is dominated by the weaker echo. Weighting on mag0 gave full weight to fast-T2* voxels, a healthy echo 0 alongside a collapsed echo 1, which are exactly the air-tissue interfaces the prior should discount. Measured over 100 runs and 24k frames from ten OpenNeuro ME-EPI datasets, 71 of them held out from the comparison that chose the weighting. The decision boundary is fixed at 1/(2*dTE) and does not move, and the worst-case margin barely changes (0.884 -> 1.052 Hz on the tightest subject). What improves is stability: the estimator's offset varies by 0.67 Hz across subjects under mag1**2 against 2.04 Hz under mag0**2, and that between-subject spread is what predicts a wrap flip. Zero errors under either weighting. Also record in the docstrings why the estimator is a median rather than a mean or M-estimator: the field distribution is broad and right-skewed and the half-wrap boundary falls inside its body, so anything pulled toward the mean reads closer to the boundary without being more accurate.
The note is independent of the selector change and is going up separately on docs/phase-offset-note. Keeps this PR to the code change.
warpkit 1.5.0 removes the ``wrap_limit`` argument from
``warpkit.api.unwrap_phase``, so forwarding it raised
TypeError: unwrap_phase() got an unexpected keyword argument 'wrap_limit'
inside the ``unwrap`` node, failing test_dynamic_unwarp_run on both
fixtures.
The flag only existed to disable a cascade of hardcoded field-magnitude
thresholds, which warpkit replaced with an automatic 2-pi branch
selector (vanandrew/warpkit#32). There is no successor input to thread
through, so the trait is dropped rather than renamed. Nothing in
SDCFlows ever set it.
warpkit 1.5.0 removes the ``wrap_limit`` argument from
``warpkit.api.unwrap_phase``, so forwarding it raised
TypeError: unwrap_phase() got an unexpected keyword argument 'wrap_limit'
inside the ``unwrap`` node, failing test_dynamic_unwarp_run on both
fixtures.
The flag only existed to disable a cascade of hardcoded field-magnitude
thresholds, which warpkit replaced with an automatic 2-pi branch
selector (vanandrew/warpkit#32). There is no successor input to thread
through, so the trait is dropped rather than renamed. Nothing in
SDCFlows ever set it.
Replaces a cascade of five hardcoded field-magnitude thresholds with a two-stage rule, and fixes a real 40 Hz field-map artifact found on OpenNeuro data.
The bug
Spatial unwrapping recovers phase differences between voxels, never the absolute turn count, so the dual-echo field is determined only modulo
1/dTE. ROMEO'scorrect_globalresolves that by preferring the smallest field, which is equivalent to requiring|f| < 1/(2·dTE).It re-derives that choice on every frame, from an unweighted median of rounded wrap counts over a dilated mask. When a subject's field approaches the half-wrap the ballot sits on a knife edge, and ordinary frame-to-frame variation tips it. Because the turn count is an integer, a tipped frame is displaced by a full wrap rather than slightly.
Two subjects in
ds006131(TEs 14.2/38.93/63.66 ms, wrap 40.44 Hz, half-wrap 20.22 Hz) do this:The damage is temporal. A uniformly wrong turn count would be a constant offset in a quantity already referenced to an arbitrary demodulation frequency, and undetectable. A count that varies between frames of one acquisition corrupts the temporal structure the acquisition exists to measure.
The rule
Mshifts the phase offset bywrap(2π·k·M),k = TE0/dTE, landing as the same additive constant on every echo. Fitting a line through the echoes and reading its intercept measures it: a self-consistent reconstruction extrapolates through the origin. The rejection scale is analytic,|wrap(2π·k)|, so nothing is tuned to data. Fitting and non-fitting candidates separate by 6–9 orders of magnitude.This is detectable at all only because the offset and the field are computed in separate stages. The offset is formed first; the field comes from a second unwrap that applies its own global correction with no knowledge of which count the offset assumed. On a tipped frame the two disagree, and that mismatch is what the intercept measures.
Stage 2 applies the same prior as
correct_global, on a better-conditioned statistic: the field itself, weighted, over an eroded mask, rather than rounded counts over a dilated one. On the failing frames both candidates score ~1e-7, so stage 1 detects the flip but cannot rank them; the prior settles it.The weighting
The prior weights by
mag1², the second echo. The field is read fromphase1 − phase0, whose noise is dominated by the weaker echo; weighting onmag0gives full weight to fast-T2* voxels (healthy echo 0, collapsed echo 1), which are exactly the air–tissue interfaces the prior should discount.This does not move the decision boundary, which is fixed at
1/(2·dTE), and barely moves the margin. What it buys is stability, and that is what predicts a flip:mag0²mag1²Also documented in the docstrings: the estimator is a median rather than a mean or M-estimator because the field distribution is broad and right-skewed with the half-wrap boundary inside its body, so anything pulled toward the mean reads closer to the boundary without being more accurate. A Huber estimator cut the worst-case margin to 0.045 Hz; a half-sample mode was tried and reverted for discontinuity.
Changes
_evaluate_branch,_branch_intercept_step,_weighted_median,_select_branch; remove_select_branch_cascadeand all four heuristic constants.TE0/dTEis an integer, where the count cannot move the offset.rescale_phase'smin/maxare floats.tests/data/branch_flip/, 9.7 MB, excluded from the sdist). Cropped to the brain bbox; deliberately not decimated, since resampling tips the knife-edge ballot and would invert what the test checks.Breaking
--wrap-limitis removed fromwk-medicandwk-unwrap-phase, andwrap_limit=frommedic()andunwrap_and_compute_field_maps(). It existed to disable the heuristics this PR deletes.Validation
100 runs, 24,105 frames, 10 OpenNeuro ME-EPI datasets, six TE0/dTE ratios (0.51–0.78) and echo spacings 16.2–27.5 ms. Zero errors under either weighting.
71 of those runs were processed after the weighting was chosen and held out from the comparison.
mag1²'s spread advantage was 2.3× on the data that selected it and 2.7× on the held-out set, so it replicated rather than regressing.The prior changes an answer on 2 of 77 prior-live runs, both above
|f|/half-wrap = 0.81; below 0.59 it was never load-bearing. The worst frame anywhere reached 0.948, leaving a 2.10 Hz decision margin on a 40.44 Hz wrap.CI
Pure Python — no pybind11, ITK, or wheel-matrix impact. Cost is ~+14% per frame for three candidate reconstructions.