Zaccharie Ramzi

Réseaux de neurones profonds avancés pour la reconstruction d'images IRM à partir de données fortement sous-échantillonnées dans des contextes d'acquisition complexes: L'imagerie par résonance magnétique (IRM) est l'une des modalités d'imagerie les plus utilisées au monde. Son objectif principal est de visualiser les tissus mous de manière non invasive et non ionisante. Cependant, son adoption générale est entravée par une durée d'examen globalement élevée. Afin de la raccourcir, plusieurs techniques ont été proposées, parmi lesquelles l'imagerie parallèle (PI) et l'échantillonnage compressif (CS) jouent une place prédominante. Grâce à ces techniques, les données en IRM peuvent être acquises de manière fortement compressée, réduisant ainsi significativement le temps d'acquisition. Cependant, les algorithmes généralement utilisés pour reconstruire les images IRM à partir de ces données sous-échantillonnées sont lents et peu performants dans des scénarios d'acquisition fortement accélérés. Afin de résoudre ces problèmes, les "réseaux de neurones déroulés" ont été introduits. L'idée centrale de ces modèles est de dérouler ou déplier les itérations des algorithmes de reconstruction classiques en un graphe de calcul fini. L'objectif principal de cette thèse est de proposer de nouvelles architectures pour des scénarios d'acquisition qui s'écartent de l'acquisition cartésienne 2D typique. À cette fin, nous passons d'abord en revue une poignée de réseaux neuronaux pour la reconstruction IRM. Après avoir sélectionné le plus performant, i.e. le PDNet, nous l'étendons à deux contextes : le challenge fastMRI 2020 et le problème des données 3D non cartésiennes. Nous avons également choisi de répondre aux préoccupations de beaucoup concernant l'applicabilité clinique de l'apprentissage profond pour l'imagerie médicale. Nous le faisons en proposant des moyens de construire des modèles robustes et inspectables, mais aussi en testant simplement les réseaux entraînés dans des contextes qui s'écartent de la distribution d'entraînement. Enfin, après avoir remarqué comment l'outil de l'apprentissage profond implicite peut aider à entraîner des modèles de reconstruction IRM plus profonds, nous introduisons une nouvelle méthode d'accélération (i.e. SHINE) pour l'entraînement de ces modèles.

Deep Learning has become a very promising avenue for magnetic resonance image (MRI) reconstruction. In this work, we explore the potential of unrolled networks for non-Cartesian acquisition settings. We design the NC-PDNet (Non-Cartesian Primal Dual Netwok), the first density-compensated (DCp) unrolled neural network, and validate the need for its key components via an ablation study. Moreover, we conduct some generalizability experiments to test this network in out-of-distribution settings, for example training on knee data and validating on brain data. The results show that NC-PDNet outperforms baseline (U-Net, Deep image prior) models both visually and quantitatively in all settings. In particular, in the 2D multi-coil acquisition scenario, the NC-PDNet provides up to a 1.2dB improvement in peak signal-to-noise ratio (PSNR) over baseline networks, while also allowing a gain of at least 1dB in PSNR in generalization settings. We provide the open-source implementation of NC-PDNet, and in particular the Non-uniform Fourier Transform in TensorFlow, tested on 2D multi-coil and 3D single-coil k-space data.

Accelerating MRI scans is one of the principal outstanding problems in the MRI research community. Towards this goal, we hosted the second fastMRI competition targeted towards reconstructing MR images with subsampled k-space data. We provided participants with data from 7,299 clinical brain scans (de-identified via a HIPAA-compliant procedure by NYU Langone Health), holding back the fully-sampled data from 894 of these scans for challenge evaluation purposes. In contrast to the 2019 challenge, we focused our radiologist evaluations on pathological assessment in brain images. We also debuted a new Transfer track that required participants to submit models evaluated on MRI scanners from outside the training set. We received 19 submissions from eight different groups. Results showed one team scoring best in both SSIM scores and qualitative radiologist evaluations. We also performed analysis on alternative metrics to mitigate the effects of background noise and collected feedback from the participants to inform future challenges. Lastly, we identify common failure modes across the submissions, highlighting areas of need for future research in the MRI reconstruction community.

Deep neural networks have proven extremely efficient at solving a wide rangeof inverse problems, but most often the uncertainty on the solution they provideis hard to quantify. In this work, we propose a generic Bayesian framework forsolving inverse problems, in which we limit the use of deep neural networks tolearning a prior distribution on the signals to recover. We adopt recent denoisingscore matching techniques to learn this prior from data, and subsequently use it aspart of an annealed Hamiltonian Monte-Carlo scheme to sample the full posteriorof image inverse problems. We apply this framework to Magnetic ResonanceImage (MRI) reconstruction and illustrate how this approach not only yields highquality reconstructions but can also be used to assess the uncertainty on particularfeatures of a reconstructed image.

The Dark Matter present in the Large-Scale Structure of the Universe is invisible, but its presence can be inferred through the small gravitational lensing effect it has on the images of far away galaxies. By measuring this lensing effect on a large number of galaxies it is possible to reconstruct maps of the Dark Matter distribution on the sky. This, however, represents an extremely challenging inverse problem due to missing data and noise dominated measurements. In this work, we present a novel methodology for addressing such inverse problems by combining elements of Bayesian statistics, analytic physical theory, and a recent class of Deep Generative Models based on Neural Score Matching. This approach allows to do the following: (1) make full use of analytic cosmological theory to constrain the 2pt statistics of the solution, (2) learn from cosmological simulations any differences between this analytic prior and full simulations, and (3) obtain samples from the full Bayesian posterior of the problem for robust Uncertainty Quantification. We present an application of this methodology on the first deep-learning-assisted Dark Matter map reconstruction of the Hubble Space Telescope COSMOS field.

Deep learning is starting to offer promising results for reconstruction in Magnetic Resonance Imaging (MRI). A lot of networks are being developed, but the comparisons remain hard because the frameworks used are not the same among studies, the networks are not properly re-trained, and the datasets used are not the same among comparisons. The recent release of a public dataset, fastMRI, consisting of raw k-space data, encouraged us to write a consistent benchmark of several deep neural networks for MR image reconstruction. This paper shows the results obtained for this benchmark, allowing to compare the networks, and links the open source implementation of all these networks in Keras. The main finding of this benchmark is that it is beneficial to perform more iterations between the image and the measurement spaces compared to having a deeper per-space network.