Jan Macdonald
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Jan Macdonald has a total of 23 co-authors. Here are the top co-authors by number of shared publications:
- Gitta Kutyniok
Ludwig-Maximilians-Universität München
2 shared publications - Lars Ruthotto
Departments of Mathematics and Computer Science Emory University Atlanta GA USA
2 shared publications - Luis Oala
Machine Learning Group Fraunhofer HHI 10587 Berlin Germany
2 shared publications Maximilian März
Technische Universität Berlin
2 shared publications- Maximilian März
Department of Mathematics Technische Universität Berlin Berlin Germany
2 shared publications - Wojciech Samek
Department of Artificial Intelligence Fraunhofer Heinrich Hertz Institute Berlin Germany
2 shared publications
Latest Publications
- Top Cited PapersOpen Access
- 19 February 2026
- Published by Elsevier in ISPRS Open Journal of Photogrammetry and Remote Sensing
- Vol. 19
Benchmarking for practice: Few-shot time-series crop-type classification on the EuroCropsML dataset
- 19 February 2026
- Published by Elsevier in ISPRS Open Journal of Photogrammetry and Remote Sensing
- Vol. 19
Top Cited PapersOpen AccessBenchmarking for practice: Few-shot time-series crop-type classification on the EuroCropsML dataset
- 1 January 2023
- Published by American Institute of Mathematical Sciences (AIMS) in Inverse Problems and Imaging
- Vol. 17 (5) , 1041-1068
Let's enhance: A deep learning approach to extreme deblurring of text images
This work presents a novel deep-learning-based pipeline for the inverse problem of image deblurring, leveraging augmentation and pre-training with synthetic data. Our results build on our winning submission to the recent Helsinki Deblur Challenge 2021, whose goal was to explore the limits of state-of-the-art deblurring algorithms in a real-world data setting. The task of the challenge was to deblur out-of-focus images of random text, thereby in a downstream task, maximizing an optical-character-recognition-based score function. A key step of our solution is the data-driven estimation of the physical forward model describing the blur process. This enables a stream of synthetic data, generating pairs of ground-truth and blurry images on-the-fly, which is used for an extensive augmentation of the small amount of challenge data provided. The actual deblurring pipeline consists of an approximate inversion of the radial lens distortion (determined by the estimated forward model) and a U-Net architecture, which is trained end-to-end. Our algorithm was the only one passing the hardest challenge level, achieving over $ 70\% $ character recognition accuracy. Our findings are well in line with the paradigm of data-centric machine learning, and we demonstrate its effectiveness in the context of inverse problems. Apart from a detailed presentation of our methodology, we also analyze the importance of several design choices in a series of ablation studies. The code of our challenge submission is available under https://github.com/theophil-trippe/HDC_TUBerlin_version_1.- 1 January 2023
- Published by American Institute of Mathematical Sciences (AIMS) in Inverse Problems and Imaging
- Vol. 17 (5) , 1041-1068
Let's enhance: A deep learning approach to extreme deblurring of text images
This work presents a novel deep-learning-based pipeline for the inverse problem of image deblurring, leveraging augmentation and pre-training with synthetic data. Our results build on our winning submission to the recent Helsinki Deblur Challenge 2021, whose goal was to explore the limits of state-of-the-art deblurring algorithms in a real-world data setting. The task of the challenge was to deblur out-of-focus images of random text, thereby in a downstream task, maximizing an optical-character-recognition-based score function. A key step of our solution is the data-driven estimation of the physical forward model describing the blur process. This enables a stream of synthetic data, generating pairs of ground-truth and blurry images on-the-fly, which is used for an extensive augmentation of the small amount of challenge data provided. The actual deblurring pipeline consists of an approximate inversion of the radial lens distortion (determined by the estimated forward model) and a U-Net architecture, which is trained end-to-end. Our algorithm was the only one passing the hardest challenge level, achieving over $ 70\% $ character recognition accuracy. Our findings are well in line with the paradigm of data-centric machine learning, and we demonstrate its effectiveness in the context of inverse problems. Apart from a detailed presentation of our methodology, we also analyze the importance of several design choices in a series of ablation studies. The code of our challenge submission is available under https://github.com/theophil-trippe/HDC_TUBerlin_version_1.- 4 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 45 (1) , 1119-1134
Solving Inverse Problems With Deep Neural Networks – Robustness Included?
In the past five years, deep learning methods have become state-of-the-art in solving various inverse problems. Before such approaches can find application in safety-critical fields, a verification of their reliability appears mandatory. Recent works have pointed out instabilities of deep neural networks for several image reconstruction tasks. In analogy to adversarial attacks in classification, it was shown that slight distortions in the input domain may cause severe artifacts. The present article sheds new light on this concern, by conducting an extensive study of the robustness of deep-learning-based algorithms for solving underdetermined inverse problems. This covers compressed sensing with Gaussian measurements as well as image recovery from Fourier and Radon measurements, including a real-world scenario for magnetic resonance imaging (using the NYU-fastMRI dataset). Our main focus is on computing adversarial perturbations of the measurements that maximize the reconstruction error. A distinctive feature of our approach is the quantitative and qualitative comparison with total-variation minimization, which serves as a provably robust reference method. In contrast to previous findings, our results reveal that standard end-to-end network architectures are not only resilient against statistical noise, but also against adversarial perturbations. All considered networks are trained by common deep learning techniques, without sophisticated defense strategies.
- 4 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 45 (1) , 1119-1134
Solving Inverse Problems With Deep Neural Networks – Robustness Included?
In the past five years, deep learning methods have become state-of-the-art in solving various inverse problems. Before such approaches can find application in safety-critical fields, a verification of their reliability appears mandatory. Recent works have pointed out instabilities of deep neural networks for several image reconstruction tasks. In analogy to adversarial attacks in classification, it was shown that slight distortions in the input domain may cause severe artifacts. The present article sheds new light on this concern, by conducting an extensive study of the robustness of deep-learning-based algorithms for solving underdetermined inverse problems. This covers compressed sensing with Gaussian measurements as well as image recovery from Fourier and Radon measurements, including a real-world scenario for magnetic resonance imaging (using the NYU-fastMRI dataset). Our main focus is on computing adversarial perturbations of the measurements that maximize the reconstruction error. A distinctive feature of our approach is the quantitative and qualitative comparison with total-variation minimization, which serves as a provably robust reference method. In contrast to previous findings, our results reveal that standard end-to-end network architectures are not only resilient against statistical noise, but also against adversarial perturbations. All considered networks are trained by common deep learning techniques, without sophisticated defense strategies.
- Open Access
Interpretable Neural Networks with Frank-Wolfe: Sparse Relevance Maps and Relevance Orderings
We study the effects of constrained optimization formulations and Frank-Wolfe algorithms for obtaining interpretable neural network predictions. Reformulating the Rate-Distortion Explanations (RDE) method for relevance attribution as a constrained optimization problem provides precise control over the sparsity of relevance maps. This enables a novel multi-rate as well as a relevance-ordering variant of RDE that both empirically outperform standard RDE and other baseline methods in a well-established comparison test. We showcase several deterministic and stochastic variants of the Frank-Wolfe algorithm and their effectiveness for RDE.
Open AccessInterpretable Neural Networks with Frank-Wolfe: Sparse Relevance Maps and Relevance Orderings
We study the effects of constrained optimization formulations and Frank-Wolfe algorithms for obtaining interpretable neural network predictions. Reformulating the Rate-Distortion Explanations (RDE) method for relevance attribution as a constrained optimization problem provides precise control over the sparsity of relevance maps. This enables a novel multi-rate as well as a relevance-ordering variant of RDE that both empirically outperform standard RDE and other baseline methods in a well-established comparison test. We showcase several deterministic and stochastic variants of the Frank-Wolfe algorithm and their effectiveness for RDE.
- Open Access
- 4 September 2021
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 16 (12) , 2089-2097
Detecting failure modes in image reconstructions with interval neural network uncertainty
Purpose: The quantitative detection of failure modes is important for making deep neural networks reliable and usable at scale. We consider three examples for common failure modes in image reconstruction and demonstrate the potential of uncertainty quantification as a fine-grained alarm system. Methods: We propose a deterministic, modular and lightweight approach called Interval Neural Network (INN) that produces fast and easy to interpret uncertainty scores for deep neural networks. Importantly, INNs can be constructed post hoc for already trained prediction networks. We compare it against state-of-the-art baseline methods (MCDrop, ProbOut). Results: We demonstrate on controlled, synthetic inverse problems the capacity of INNs to capture uncertainty due to noise as well as directional error information. On a real-world inverse problem with human CT scans, we can show that INNs produce uncertainty scores which improve the detection of all considered failure modes compared to the baseline methods. Conclusion: Interval Neural Networks offer a promising tool to expose weaknesses of deep image reconstruction models and ultimately make them more reliable. The fact that they can be applied post hoc to equip already trained deep neural network models with uncertainty scores makes them particularly interesting for deployment.
- 4 September 2021
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 16 (12) , 2089-2097
Open AccessDetecting failure modes in image reconstructions with interval neural network uncertainty
Purpose: The quantitative detection of failure modes is important for making deep neural networks reliable and usable at scale. We consider three examples for common failure modes in image reconstruction and demonstrate the potential of uncertainty quantification as a fine-grained alarm system. Methods: We propose a deterministic, modular and lightweight approach called Interval Neural Network (INN) that produces fast and easy to interpret uncertainty scores for deep neural networks. Importantly, INNs can be constructed post hoc for already trained prediction networks. We compare it against state-of-the-art baseline methods (MCDrop, ProbOut). Results: We demonstrate on controlled, synthetic inverse problems the capacity of INNs to capture uncertainty due to noise as well as directional error information. On a real-world inverse problem with human CT scans, we can show that INNs produce uncertainty scores which improve the detection of all considered failure modes compared to the baseline methods. Conclusion: Interval Neural Networks offer a promising tool to expose weaknesses of deep image reconstruction models and ultimately make them more reliable. The fact that they can be applied post hoc to equip already trained deep neural network models with uncertainty scores makes them particularly interesting for deployment.
- 27 February 2021
- Published by Springer Nature
- p. 324-329
Interval Neural Networks as Instability Detectors for Image Reconstructions
- 27 February 2021
- Published by Springer Nature
- p. 324-329
Interval Neural Networks as Instability Detectors for Image Reconstructions
- Open Access
Interval Neural Networks as Instability Detectors for Image Reconstructions
This work investigates the detection of instabilities that may occur when utilizing deep learning models for image reconstruction tasks. Although neural networks often empirically outperform traditional reconstruction methods, their usage for sensitive medical applications remains controversial. Indeed, in a recent series of works, it has been demonstrated that deep learning approaches are susceptible to various types of instabilities, caused for instance by adversarial noise or out-of-distribution features. It is argued that this phenomenon can be observed regardless of the underlying architecture and that there is no easy remedy. Based on this insight, the present work demonstrates on two use cases how uncertainty quantification methods can be employed as instability detectors. In particular, it is shown that the recently proposed Interval Neural Networks are highly effective in revealing instabilities of reconstructions. Such an ability is crucial to ensure a safe use of deep learning-based methods for medical image reconstruction.
Open AccessInterval Neural Networks as Instability Detectors for Image Reconstructions
This work investigates the detection of instabilities that may occur when utilizing deep learning models for image reconstruction tasks. Although neural networks often empirically outperform traditional reconstruction methods, their usage for sensitive medical applications remains controversial. Indeed, in a recent series of works, it has been demonstrated that deep learning approaches are susceptible to various types of instabilities, caused for instance by adversarial noise or out-of-distribution features. It is argued that this phenomenon can be observed regardless of the underlying architecture and that there is no easy remedy. Based on this insight, the present work demonstrates on two use cases how uncertainty quantification methods can be employed as instability detectors. In particular, it is shown that the recently proposed Interval Neural Networks are highly effective in revealing instabilities of reconstructions. Such an ability is crucial to ensure a safe use of deep learning-based methods for medical image reconstruction.
- Open Access
The Computational Complexity of Understanding Network Decisions
For a Boolean function $\Phi\colon\{0,1\}^d\to\{0,1\}$ and an assignment to its variables $\mathbf{x}=(x_1, x_2, \dots, x_d)$ we consider the problem of finding the subsets of the variables that are sufficient to determine the function value with a given probability $\delta$. This is motivated by the task of interpreting predictions of binary classifiers described as Boolean circuits (which can be seen as special cases of neural networks). We show that the problem of deciding whether such subsets of relevant variables of limited size $k\leq d$ exist is complete for the complexity class $\mathsf{NP}^{\mathsf{PP}}$ and thus generally unfeasible to solve. We introduce a variant where it suffices to check whether a subset determines the function value with probability at least $\delta$ or at most $\delta-\gamma$ for $0<\gamma<\delta$. This reduces the complexity to the class $\mathsf{NP}^{\mathsf{BPP}}$. Finally, we show that finding the minimal set of relevant variables can not be reasonably approximated, i.e. with an approximation factor $d^{1-\alpha}$ for $\alpha > 0$, by a polynomial time algorithm unless $\mathsf{P} = \mathsf{NP}$ (this holds even with the probability gap).Open AccessThe Computational Complexity of Understanding Network Decisions
For a Boolean function $\Phi\colon\{0,1\}^d\to\{0,1\}$ and an assignment to its variables $\mathbf{x}=(x_1, x_2, \dots, x_d)$ we consider the problem of finding the subsets of the variables that are sufficient to determine the function value with a given probability $\delta$. This is motivated by the task of interpreting predictions of binary classifiers described as Boolean circuits (which can be seen as special cases of neural networks). We show that the problem of deciding whether such subsets of relevant variables of limited size $k\leq d$ exist is complete for the complexity class $\mathsf{NP}^{\mathsf{PP}}$ and thus generally unfeasible to solve. We introduce a variant where it suffices to check whether a subset determines the function value with probability at least $\delta$ or at most $\delta-\gamma$ for $0<\gamma<\delta$. This reduces the complexity to the class $\mathsf{NP}^{\mathsf{BPP}}$. Finally, we show that finding the minimal set of relevant variables can not be reasonably approximated, i.e. with an approximation factor $d^{1-\alpha}$ for $\alpha > 0$, by a polynomial time algorithm unless $\mathsf{P} = \mathsf{NP}$ (this holds even with the probability gap). - 2 September 2017
- Published by Springer Nature in Journal of Mathematical Imaging and Vision
- Vol. 60 (2) , 268-282
Improved Susceptibility Artifact Correction of Echo-Planar MRI using the Alternating Direction Method of Multipliers
- 2 September 2017
- Published by Springer Nature in Journal of Mathematical Imaging and Vision
- Vol. 60 (2) , 268-282
Improved Susceptibility Artifact Correction of Echo-Planar MRI using the Alternating Direction Method of Multipliers
- Open Access
Efficient Numerical Optimization For Susceptibility Artifact Correction Of EPI-MRI
We present two efficient numerical methods for susceptibility artifact correction applicable in Echo Planar Imaging (EPI), an ultra fast Magnetic Resonance Imaging (MRI) technique widely used in clinical applications. Both methods address a major practical drawback of EPI, the so-called susceptibility artifacts, which consist of geometrical transformations and intensity modulations. We consider a tailored variational image registration problem that is based on a physical distortion model and aims at minimizing the distance of two oppositely distorted images subject to invertibility constraints. We follow a discretize-then-optimize approach and present a novel face-staggered discretization yielding a separable structure in the discretized distance function and the invertibility constraints. The presence of a smoothness regularizer renders the overall optimization problem non-separable, but we present two optimization schemes that exploit the partial separability. First, we derive a block-Jacobi preconditioner to be used in a Gauss-Newton-PCG method. Second, we consider a splitting of the separable and non-separable part and solve the resulting problem using the Alternating Direction Method of Multipliers (ADMM). We provide a detailed convergence proof for ADMM for this non-convex optimization problem. Both schemes are of essentially linear complexity and are suitable for parallel computing. A considerable advantage of the proposed schemes over established methods is the reduced time-to-solution. In our numerical experiment using high-resolution 3D imaging data, our parallel implementation of the ADMM method solves a 3D problem with more than 5 million degrees of freedom in less than 50 seconds on a standard laptop, which is a considerable improvement over existing methods.
Open AccessEfficient Numerical Optimization For Susceptibility Artifact Correction Of EPI-MRI
We present two efficient numerical methods for susceptibility artifact correction applicable in Echo Planar Imaging (EPI), an ultra fast Magnetic Resonance Imaging (MRI) technique widely used in clinical applications. Both methods address a major practical drawback of EPI, the so-called susceptibility artifacts, which consist of geometrical transformations and intensity modulations. We consider a tailored variational image registration problem that is based on a physical distortion model and aims at minimizing the distance of two oppositely distorted images subject to invertibility constraints. We follow a discretize-then-optimize approach and present a novel face-staggered discretization yielding a separable structure in the discretized distance function and the invertibility constraints. The presence of a smoothness regularizer renders the overall optimization problem non-separable, but we present two optimization schemes that exploit the partial separability. First, we derive a block-Jacobi preconditioner to be used in a Gauss-Newton-PCG method. Second, we consider a splitting of the separable and non-separable part and solve the resulting problem using the Alternating Direction Method of Multipliers (ADMM). We provide a detailed convergence proof for ADMM for this non-convex optimization problem. Both schemes are of essentially linear complexity and are suitable for parallel computing. A considerable advantage of the proposed schemes over established methods is the reduced time-to-solution. In our numerical experiment using high-resolution 3D imaging data, our parallel implementation of the ADMM method solves a 3D problem with more than 5 million degrees of freedom in less than 50 seconds on a standard laptop, which is a considerable improvement over existing methods.