Comparison of Monalisa and BART Reconstructions

This repository provides a comparative analysis of Monalisa and BART reconstructions on four different 2D images. The goal is not to establish superiority of any framework but rather to confirm that Monalisa achieves results comparable to BART. The observed results are surprising, and the reconstruction quality gap is currently not fully understood. We believe the possible sources of discrepancies could be:

1. Raw data transformation: The raw data are synthetically generated in Monalisa format and require conversion. Although we made every effort to understand the BART data format, errors might exist. Some discussion about data format can be found here.
2. Reconstruction commands: In the absence of complete documentation, we explored BART's tutorials to understand the command-line interface. Again, mistakes might be present.

We would be grateful if anyone could spot potential mistakes. For any interested and motivated person, it might be valuable to create an unbiased website providing simulated raw data (and perhaps coil sensitivities) but not the ground truth. People could submit their implementations of iterative reconstructions to assess quality, potentially using with Docker images returning reconstructed results, and a leaderboard scoring the different implementations, a bit like it is done with ML models.

---

Reconstruction Comparison

We compare both frameworks using L1 and L2 regularization-based reconstructions. To assess reconstruction quality, we generate synthetic raw MRI measurements $y$ from known ground truth 2D images and coil sensitivities. These synthetic measurements are used as raw data for both frameworks.

To ensure fair comparison, we perform a grid search to determine optimal regularization parameters for each framework separately. Since reconstruction regularization can alter image intensity scales, direct comparison of reconstructed and reference images can be misleading. To address this, we apply an affine intensity alignment of the ground truth image magnitude to each reconstruction prior to computing similarity metrics:

$$ a^\ast, b^\ast = \arg\min_{a,b} \sum_i \big(a \cdot |\mathrm{GT}_i| + b - |\mathrm{Recon}_i|\big)^2, $$

where (i) indexes all voxels. The aligned ground truth is then:

$$ \mathrm{GT}' = a^\ast \cdot |\mathrm{GT}| + b^\ast $$

This alignment preserves structural information while compensating for global scaling and offset differences. SSIM and $l_2$ distance are then computed between the reconstructed magnitude images and the aligned ground truth magnitudes.

The rationale is that SSIM and $l_2$ distance are sensitive to shifts in reconstructed image support. A global linear transformation of magnitudes does not change relative contrast or structure, which are diagnostically relevant. Aligning the ground truth ensures evaluation reflects structural fidelity rather than arbitrary scaling differences.

The undersampling strategy follows a 2D radial trajectory with 30 lines of 512 points each—a challenging scenario that highlights the positive impact of regularization.

---

Reconstruction Methodology

The regularized reconstruction problem is formulated as:

$$ x^* = \arg\min_x | A x - y |_2^2 + \lambda R(x) $$

where $R(x)$ is the regularization term influencing the reconstructed image.

Steps of Comparison

1. Ground Truth & ROI Selection: Three pairs of known coil sensitivity maps (C) and 2D images are used: a phantom, an eye image, and a cardiac slice. An elliptical ROI is manually selected for each image.
2. Trajectory Generation: A 2D radial trajectory is generated by rotating each line sequentially by (\frac{2\pi}{30}) radians:

t_tot = bmTraj_fullRadial2_lineAssym2(N, nLines, dK_u(1));
ve = bmVolumeElement(t_tot, 'voronoi_full_radial2');

3. Data Simulation: Raw MRI measurements are simulated using:

y = bmSimulateMriData(image, C, t_tot, N_u, n_u, dK_u);

4. Data Conversion: Monalisa data are converted into BART format (volume element definitions, .cfl and .hdr files) using generateBARTfiles.m.

5. Reconstruction & Evaluation:

   • L1-regularized (Total Variation):

$$ x^* = \arg\min_x | A x - y |_2^2 + TV(x) $$

• L2-regularized:

$$ x^* = \arg\min_x | A x - y |_2^2 + \lambda | x |_2 $$

Iterative reconstructions are performed for both frameworks using 160 iterations to ensure convergence. Reconstructed images are rescaled to match the mean intensity of the ground truth within the ROI, and SSIM is computed for various regularization values to select the best-performing parameter.

---

Repository Structure

• /images/ - Contains three test images: a cardiac image, a brain image with FoV centered on the eye, and the Shepp-Logan phantom. .mat files also include coil sensitivity maps.
• generateBARTdatafiles.m - Generates .cfl and .hdr files for BART.
• run_recons_Bart.ipynb - Runs BART reconstructions and grid search for optimal regularization.
• run_recons_Monalisa.m - Runs Monalisa reconstructions and grid search. Saves final results.
• helpers.py - Python code for analysis.
• lineSearchrecon.m - MATLAB code for analysis.
• /reconstructions/... - Resulting reconstruction files.
• /results/... - Plots of reconstructed images.
• FinalReconsEvaluation.ipynb - Runs final BART reconstructions and generates comparison plots.

---

Results

The best SSIM values for each image are reported for both frameworks, providing a quantitative comparison. L2 distance measures pixel-wise differences, while SSIM evaluates perceptual quality based on luminance, contrast, and structure.

For both $l_1$ and $l_2$ iterative reconstructions, on all the three test examples, Monalisa achieves higher SSIM values and lower $l_2$ distances. Visually, for the cardiac and phantom images, Bart reconstructions appear more affected by artifacts, especially visible in the background for the phantom and in the pulmonary vein region of the cardiac image. Our benchmarking against Bart on simulated undersampled 2D radial data demonstrates that Monalisa achieves comparable performance. The observed differences might be attributable to variations in algorithmic implementation. However, the larger discrepancies observed in the phantom reconstruction background and in the cardiac pulmonary vein region remain unexpected, and their underlying causes have yet to be determined.

L1-Regularized Reconstructions

Metric	Gridded Recon	Monalisa l1-Reg	BART l1-Reg
Phantom
SSIM	0.2868	0.8299	0.7634
l2-Distance	23.5172	6.4136	10.7668
Brain
SSIM	0.2390	0.6674	0.6150
l2-Distance	85.3408	35.1337	37.7829
Cardiac (Close)
SSIM	0.4551	0.7682	0.6921
l2-Distance	28.1078	13.6938	19.0520

L2-Regularized Reconstructions

Metric	Gridded Recon	Monalisa l2-Reg	BART l2-Reg
Phantom
SSIM	0.2868	0.5247	0.4779
l2-Distance	23.5172	17.7950	18.8276
Brain
SSIM	0.2390	0.6611	0.6105
l2-Distance	85.3408	36.4645	38.5181
Cardiac (Close)
SSIM	0.4551	0.7693	0.6927
l2-Distance	28.1078	14.3118	18.6575