Future consideration: are the ITKVtkGlue Python typemaps still needed, given the numpy bridge?

[hjmjohnson]

Proposal

Record, for consideration in a future ITK release, whether the Python wrapping of ITKVtkGlue's VTK-pointer typemaps is still needed, given that itk.vtk_image_from_image / itk.image_from_vtk_image in itk/support/extras.py appear to cover the same ground with less coupling.

This is not a removal request, and nothing should change now. The C++ side of ITKVtkGlue is in active downstream use and is out of scope entirely. The question is narrower: whether the SWIG typemaps that exchange raw vtkImageData* / vtkPolyData* pointers with Python earn their maintenance cost.

(Edited: an earlier version of this issue claimed the numpy bridge had an object-lifetime hazard that had to be fixed first. Testing shows it does not — see the correction comment below. The section is retained, corrected, because the reference chain is worth documenting.)

Why raise it now

PR #6715 made the typemaps abi3-compatible. Doing so required hand-parsing VTK's undocumented __this__ / Addr=0x… encoding and dropping VTK::WrappingPythonCore from the link interface, because that library pins an extension to one libpython. The work is sound, but it is ongoing maintenance against VTK internals that carry no compatibility guarantee.

That is a reasonable moment to ask whether the coupling is necessary at all.

Measurements comparing the two bridges

Tested on macOS arm64 against a purpose-built VTK 9.6.2, using the PR #6715 branch.

Property	numpy bridge	typemap bridge
Spacing, origin	preserved	preserved
Direction matrix (non-identity, VTK ≥ 9)	preserved	preserved
Buffer sharing (ITK edit visible through VTK)	yes	yes
Vector / CovariantVector / tensor pixel types	explicit handling in extras.py	via the ITK filter
Requires ITK binaries compiled against VTK	no	yes
Participates in the ITK pipeline	no	yes

Cross-boundary pipeline behaviour was also checked, since that is the capability one would expect the filter to add:

itk_smooth.SetSigma(2.0)
vtk_smooth.Update()          # only the VTK side updated

The VTK filter returns the stale value. Propagation requires an explicit bridge.Update() first. The two pipeline graphs are joined by a manual update either way, so the filter is a pipeline object but not a pipeline connection — it does not fuse the two executives.

With the numpy helper the equivalent code needs itk_smooth.Update() and a re-conversion. Different ergonomics, same manual synchronisation, no capability that is reachable one way and not the other.

Who actually uses which path

Surveyed across a local multi-project build of ITK's downstream consumers.

C++ users of the ITKVtkGlue filters — unaffected by any Python question:

• Slicer — Libs/vtkITK/vtkITKImageSequenceReader.cxx, vtkITKGrowCut.cxx, vtkITKLabelShapeStatistics.cxx
• BRAINSTools — ConvertBetweenFileFormats/castconverthelpers.h, BRAINSConstellationDetector/gui/…
• ANTs — Examples/antsSurf.cxx, antsVol.cxx, Temporary/itkFEM*Map.h
• LesionSizingToolkit

Python users of the numpy helpers:

• Slicer — Base/Python/slicer/util.py:3060,3126 calls itk.image_from_vtk_image / itk.vtk_image_from_image

Python users of the typemaps:

• Four ITKSphinxExamples: ConvertAnitkImageTovtkImageData, ConvertvtkImageDataToAnitkImage, ConvertRGBvtkImageDataToAnitkImage, ConvertAnRGBitkImageTovtkImageData
• Inside ITK, one test: Wrapping/Generators/Python/Tests/verifyGetOutputAPIConsistency.py

So the largest Python consumer already uses the numpy path, and the typemap path's Python users are primarily documentation examples.

Object lifetime in the numpy bridge — checked, and safe

vtk_image_from_image builds a vtkImageData around itk.array_view_from_image(...), which is a view rather than a copy. That looks like a dangling-buffer risk when reading the function alone, but both retaining links are established outside it:

vtkImageData -> vtkTypeFloat32Array -> (._numpy_reference) -> numpy array -> (.base) -> itk.Image

numpy_to_vtk stashes the source array on the VTK object; ITK's numpy support sets the array's base to the image. A vtkImageData therefore keeps its source itk.Image alive by itself. Verified against VTK 9.6.2 with a control confirming an unconverted image does die when unreferenced, and under heap churn after scope exit.

Caveat: _numpy_reference is VTK implementation behaviour, not a documented API guarantee, and I checked only 9.6.2 while ITK's minimum is 9.1. A regression test asserting that a vtkImageData outlives its source image's direct references would make the property explicit — worth adding independently of this proposal.

Suggested disposition

• Now: no change. Merge and maintain COMP: Make ITKVtkGlue wrapping abi3-compatible #6715 as-is.
• Next: add coverage asserting that a vtkImageData outliving its source itk.Image stays valid, so the reference chain is guaranteed rather than incidental.
• A future release: revisit whether the Python typemaps are still worth their coupling to VTK internals, with the SphinxExamples migrated first if so. Any deprecation would need the usual ITK_LEGACY_REMOVE cycle.
• Never in scope here: the C++ ITKVtkGlue classes, which have real downstream users.

I am happy to be wrong about this — if there is a use case the numpy path cannot serve, this issue is the place to record it, and it should then be closed.

[hjmjohnson]

Correction: the lifetime hazard I described above does not exist. I asserted it from reading the source rather than testing it, and testing shows the reference chain is intact. I have edited the issue body accordingly; this comment records what changed and why.

The chain is:

vtkImageData -> vtkTypeFloat32Array -> (._numpy_reference) -> numpy array -> (.base) -> itk.Image

numpy_to_vtk stashes the source array on the VTK object as _numpy_reference, and itk.array_view_from_image sets the numpy base back to the itk.Image. Both links hold, so a vtkImageData returned by vtk_image_from_image keeps its source image alive on its own.

How it was measured, including the control that matters

The first tests I ran showed the image surviving del, which is equally consistent with "the chain works" and "something caches objects so nothing ever dies". The control distinguishes them:

Case	Image survives loss of all direct references?
Control — no conversion at all	No — dies on schedule, so the weakref test is meaningful
array_view_from_image only	Yes
Full vtk_image_from_image	Yes

Stress check: source image created inside a function, only the vtkImageData returned, then gc.collect() followed by allocating and releasing ~30 MB of unrelated arrays. All sampled voxels still read their initialised value.

Measured against VTK 9.6.2 with the PR #6715 branch on macOS arm64.

One caveat that survives: this depends on numpy_to_vtk retaining _numpy_reference, which is VTK implementation behaviour rather than a documented guarantee. Verified on 9.6.2; ITK's minimum is 9.1 and I have not checked that end of the range. A regression test asserting the source image outlives its direct references would make the property explicit rather than incidental — that seems worth doing regardless of what happens to this proposal.

Net effect on the proposal: it gets stronger, not weaker. Lifetime safety was the last substantive advantage I could identify for the typemap path in Python. With it gone, the two bridges match on every property I measured — geometry fidelity, buffer aliasing, pixel-type coverage and pipeline reach — and differ only in that the typemap path requires ITK binaries compiled against VTK.

The disposition is unchanged: nothing now, revisit in a future release.