Gitta Kutyniok

Deep learning’s success comes with growing energy demands, raising concerns about the long-term sustainability of the field. Spiking neural networks (SNNs), inspired by biological neurons, offer a promising alternative with potential computational and energy efficiency gains. This article examines the computational properties of spiking networks through the lens of learning theory, focusing on expressivity, training, and generalization, as well as energy-efficient implementations, while comparing them with artificial neural networks (ANNs). By categorizing spiking models based on time representation and information encoding, we highlight their strengths, challenges, and potential as an alternative computational paradigm.

Robotic grasping under uncertainty remains a fundamental challenge due to its uncertain and contact-rich nature. Traditional rigid robotic hands, with limited degrees of freedom and compliance, rely on complex model-based and heavy feedback controllers to manage such interactions. Soft robots, by contrast, exhibit embodied mechanical intelligence: their underactuated structures and passive flexibility of their whole body naturally accommodate uncertain contacts and enable adaptive behaviors. To harness this capability, we propose a lightweight actuation-space learning framework that infers distributional control representations for whole-body soft robotic grasping directly from deterministic demonstrations using a flow matching model (Rectified Flow), without requiring dense sensing or heavy control loops. Trained with only 30 demonstrations covering less than 8% of the reachable workspace, the learned policy achieved a 97.5% grasp success rate over 1000 trials in simulation. In real-world experiments on 50 uniformly distributed targets, the policy achieved a 100% success rate, generalized to object size variations from $-33\%$ to $+100\%$, and remained stable under execution-time scaling from 20% to 200%. These results demonstrate that actuation-space learning effectively embeds mechanical intelligence into control, significantly reducing reliance on centralized computation for grasping under uncertainty.

Objective. Proton therapy treatment planning currently needs to account for relatively large range uncertainty margins primarily due to the semi-empirical calibration of the treatment planning x-ray computed tomography (CT) to proton stopping power relative to water (RSP). Proton radiography enables direct measurement of integral RSP, offering potential to improve calibration and reduce these uncertainties. Approach. Three deep neural network architectures are trained to infer a patient RSP map using the treatment planning CT and proton radiographies, assuming straight proton trajectories. The best suited architecture is then evaluated on more realistic, clinical-like data generated with Monte Carlo simulations. Three idealized proton imaging detectors are simulated: single particle tracking (SPT), a pixelated energy-resolving imager (PERI) and a proton-integrating detector (PID). Main results. The learned primal dual (LPD) architetcure performs best in the simplified imaging scenario. In the more realistic scenario, the initial calibration error (median absolute percentage error of 2.24% across the test set) is reduced using only two projections across all detector types. SPT and PERI reach similar performance (1.10%/1.12%), followed by PID (1.30%). Restricting the LPD to the calibration task by incorporating prior knowledge of the functional relationship of Hounsfield Units (HUs) and RSP further improves calibration performance. For SPT, conventional optimization on detector data acquired for individual protons outperformed the data-driven method (0.16% vs. 1.10%). However, for PERI and PID, the data-driven approach (1.12%/1.30%) slightly outperformed conventional optimization (1.63%/1.72%). Significance. To our knowledge, this is the first study to apply a deep learning-based approach fusing proton radiographies and treatment planning CT data for improved RSP calibration. The method achieves lower calibration errors than idealized, conventional calibration curve optimization on PERIs and PIDs—detector types that offer a promising path for clinical adoption due to their lower complexity and cost compared to SPT systems.

U-Net and other U-shaped architectures have achieved significant success in image deconvolution tasks. However, challenges have emerged, as these methods might generate unrealistic artifacts or hallucinations, which can interfere with analysis in safety-critical scenarios. This paper introduces a novel approach for quantifying and comprehending hallucination artifacts to ensure trustworthy computer vision models. Our method, termed the Conformal Hallucination Estimation Metric (CHEM), is applicable to any image reconstruction model, enabling efficient identification and quantification of hallucination artifacts. It offers two key advantages: it leverages wavelet and shearlet representations to efficiently extract hallucinations of image features and uses conformalized quantile regression to assess hallucination levels in a distribution-free manner. Furthermore, from an approximation theoretical perspective, we explore the reasons why U-shaped networks are prone to hallucinations. We test the proposed approach on the CANDELS astronomical image dataset with models such as U-Net, SwinUNet, and Learnlets, and provide new perspectives on hallucination from different aspects in deep learning-based image processing.

Machine learning techniques for the solution of inverse problems have become an attractive approach in the last decade, while their theoretical foundations are still in their infancy. In this chapter we want to pursue the study of regularization properties, robustness, convergence rates, and structure of regularizers for inverse problems obtained from different learning paradigms. For this sake we study simple architectures that are explainable in the sense that they allow for a theoretical analysis also in the infinite-dimensional limit. In particular we will advance the study of spectral architectures with new results on convergence rates highlighting the role of the smoothness in the training data set, and a study of adversarial robustness. We can show that adversarial training is actually a convergent regularization method. Moreover, we discuss extensions to frame systems and CNN-type architectures for variational regularizers, where we obtain some results on their structure by carefully designed numerical experiments.