Jonas Fischer
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jonas Fischer has a total of 10 co-authors. Here are the top co-authors by number of shared publications:
Jilles Vreeken
CISPA Helmholtz Center for Information Security Saarbrücken Germany
2 shared publications- Rebekka Burkholz
CISPA Helmholtz Center for Information Security Saarbrücken Germany
2 shared publications - Advait Gadhikar
CISPA Helmholtz Center for Information Security Saarbrücken Germany
1 shared publication - Alexander Marx
ETH Zurich
1 shared publication Dietrich Klakow
Saarland University Saarland Informat- ics Campus Saarbrücken Germany
1 shared publicationFatemeh Behjati Ardakani
Affiliations Max Planck Institute for Informatics, Computational Biology and Applied Algorithmics, Saarbrücken, Germany, Cluster of Excellence for Multimodal Computing and Interaction, High Throughput Genomics and Systems Biology, Saarbrücken, Germany, Institute of Cardiovascular Regeneration, Goethe University, Frankfurt, Germany
1 shared publication
Latest Publications
- Open Access
Label-Descriptive Patterns and Their Application to Characterizing Classification Errors
State-of-the-art deep learning methods achieve human-like performance on many tasks, but make errors nevertheless. Characterizing these errors in easily interpretable terms gives insight into whether a classifier is prone to making systematic errors, but also gives a way to act and improve the classifier. We propose to discover those feature-value combinations (i.e., patterns) that strongly correlate with correct resp. erroneous predictions to obtain a global and interpretable description for arbitrary classifiers. We show this is an instance of the more general label description problem, which we formulate in terms of the Minimum Description Length principle. To discover a good pattern set, we develop the efficient Premise algorithm. Through an extensive set of experiments we show it performs very well in practice on both synthetic and real-world data. Unlike existing solutions, it ably recovers ground truth patterns, even on highly imbalanced data over many features. Through two case studies on Visual Question Answering and Named Entity Recognition, we confirm that Premise gives clear and actionable insight into the systematic errors made by modern NLP classifiers.
Open AccessLabel-Descriptive Patterns and Their Application to Characterizing Classification Errors
State-of-the-art deep learning methods achieve human-like performance on many tasks, but make errors nevertheless. Characterizing these errors in easily interpretable terms gives insight into whether a classifier is prone to making systematic errors, but also gives a way to act and improve the classifier. We propose to discover those feature-value combinations (i.e., patterns) that strongly correlate with correct resp. erroneous predictions to obtain a global and interpretable description for arbitrary classifiers. We show this is an instance of the more general label description problem, which we formulate in terms of the Minimum Description Length principle. To discover a good pattern set, we develop the efficient Premise algorithm. Through an extensive set of experiments we show it performs very well in practice on both synthetic and real-world data. Unlike existing solutions, it ably recovers ground truth patterns, even on highly imbalanced data over many features. Through two case studies on Visual Question Answering and Named Entity Recognition, we confirm that Premise gives clear and actionable insight into the systematic errors made by modern NLP classifiers.
- Open Access
Plant 'n' Seek: Can You Find the Winning Ticket?
The lottery ticket hypothesis has sparked the rapid development of pruning algorithms that aim to reduce the computational costs associated with deep learning during training and model deployment. Currently, such algorithms are primarily evaluated on imaging data, for which we lack ground truth information and thus the understanding of how sparse lottery tickets could be. To fill this gap, we develop a framework that allows us to plant and hide winning tickets with desirable properties in randomly initialized neural networks. To analyze the ability of state-of-the-art pruning to identify tickets of extreme sparsity, we design and hide such tickets solving four challenging tasks. In extensive experiments, we observe similar trends as in imaging studies, indicating that our framework can provide transferable insights into realistic problems. Additionally, we can now see beyond such relative trends and highlight limitations of current pruning methods. Based on our results, we conclude that the current limitations in ticket sparsity are likely of algorithmic rather than fundamental nature. We anticipate that comparisons to planted tickets will facilitate future developments of efficient pruning algorithms.
Open AccessPlant 'n' Seek: Can You Find the Winning Ticket?
The lottery ticket hypothesis has sparked the rapid development of pruning algorithms that aim to reduce the computational costs associated with deep learning during training and model deployment. Currently, such algorithms are primarily evaluated on imaging data, for which we lack ground truth information and thus the understanding of how sparse lottery tickets could be. To fill this gap, we develop a framework that allows us to plant and hide winning tickets with desirable properties in randomly initialized neural networks. To analyze the ability of state-of-the-art pruning to identify tickets of extreme sparsity, we design and hide such tickets solving four challenging tasks. In extensive experiments, we observe similar trends as in imaging studies, indicating that our framework can provide transferable insights into realistic problems. Additionally, we can now see beyond such relative trends and highlight limitations of current pruning methods. Based on our results, we conclude that the current limitations in ticket sparsity are likely of algorithmic rather than fundamental nature. We anticipate that comparisons to planted tickets will facilitate future developments of efficient pruning algorithms.
- Open Access
Lottery Tickets with Nonzero Biases
The strong lottery ticket hypothesis holds the promise that pruning randomly initialized deep neural networks could offer a computationally efficient alternative to deep learning with stochastic gradient descent. Common parameter initialization schemes and existence proofs, however, are focused on networks with zero biases, thus foregoing the potential universal approximation property of pruning. To fill this gap, we extend multiple initialization schemes and existence proofs to nonzero biases, including explicit 'looks-linear' approaches for ReLU activation functions. These do not only enable truly orthogonal parameter initialization but also reduce potential pruning errors. In experiments on standard benchmark data, we further highlight the practical benefits of nonzero bias initialization schemes, and present theoretically inspired extensions for state-of-the-art strong lottery ticket pruning.
Open AccessLottery Tickets with Nonzero Biases
The strong lottery ticket hypothesis holds the promise that pruning randomly initialized deep neural networks could offer a computationally efficient alternative to deep learning with stochastic gradient descent. Common parameter initialization schemes and existence proofs, however, are focused on networks with zero biases, thus foregoing the potential universal approximation property of pruning. To fill this gap, we extend multiple initialization schemes and existence proofs to nonzero biases, including explicit 'looks-linear' approaches for ReLU activation functions. These do not only enable truly orthogonal parameter initialization but also reduce potential pruning errors. In experiments on standard benchmark data, we further highlight the practical benefits of nonzero bias initialization schemes, and present theoretically inspired extensions for state-of-the-art strong lottery ticket pruning.
- Open Access
Estimating Mutual Information via Geodesic $k$NN
Estimating mutual information (MI) between two continuous random variables $X$ and $Y$ allows to capture non-linear dependencies between them, non-parametrically. As such, MI estimation lies at the core of many data science applications. Yet, robustly estimating MI for high-dimensional $X$ and $Y$ is still an open research question. In this paper, we formulate this problem through the lens of manifold learning. That is, we leverage the common assumption that the information of $X$ and $Y$ is captured by a low-dimensional manifold embedded in the observed high-dimensional space and transfer it to MI estimation. As an extension to state-of-the-art $k$NN estimators, we propose to determine the $k$-nearest neighbors via geodesic distances on this manifold rather than from the ambient space, which allows us to estimate MI even in the high-dimensional setting. An empirical evaluation of our method, G-KSG, against the state-of-the-art shows that it yields good estimations of MI in classical benchmark and manifold tasks, even for high dimensional datasets, which none of the existing methods can provide.Open AccessEstimating Mutual Information via Geodesic $k$NN
Estimating mutual information (MI) between two continuous random variables $X$ and $Y$ allows to capture non-linear dependencies between them, non-parametrically. As such, MI estimation lies at the core of many data science applications. Yet, robustly estimating MI for high-dimensional $X$ and $Y$ is still an open research question. In this paper, we formulate this problem through the lens of manifold learning. That is, we leverage the common assumption that the information of $X$ and $Y$ is captured by a low-dimensional manifold embedded in the observed high-dimensional space and transfer it to MI estimation. As an extension to state-of-the-art $k$NN estimators, we propose to determine the $k$-nearest neighbors via geodesic distances on this manifold rather than from the ambient space, which allows us to estimate MI even in the high-dimensional setting. An empirical evaluation of our method, G-KSG, against the state-of-the-art shows that it yields good estimations of MI in classical benchmark and manifold tasks, even for high dimensional datasets, which none of the existing methods can provide. - Open Access
- 15 April 2021
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 16 (4) , e0249985
CpG content-dependent associations between transcription factors and histone modifications
Understanding the factors that underlie the epigenetic regulation of genes is crucial to understand the gene regulatory machinery as a whole. Several experimental and computational studies examined the relationship between different factors involved. Here we investigate the relationship between transcription factors (TFs) and histone modifications (HMs), based on ChIP-seq data in cell lines. As it was shown that gene regulation by TFs differs depending on the CpG class of a promoter, we study the impact of the CpG content in promoters on the associations between TFs and HMs. We suggest an approach based on sparse linear regression models to infer associations between TFs and HMs with respect to CpG content. A study of the partial correlation of HMs for the two classes of high and low CpG content reveals possible CpG dependence and potential candidates for confounding factors in our models. We show that the models are accurate, inferred associations reflect known biological relationships, and we give new insight into associations with respect to CpG content. Moreover, analysis of a ChIP-seq dataset in HepG2 cells of the HM H3K122ac, an HM about little is known, reveals novel TF associations and supports a previously established link to active transcription.
- 15 April 2021
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 16 (4) , e0249985
Open AccessCpG content-dependent associations between transcription factors and histone modifications
Understanding the factors that underlie the epigenetic regulation of genes is crucial to understand the gene regulatory machinery as a whole. Several experimental and computational studies examined the relationship between different factors involved. Here we investigate the relationship between transcription factors (TFs) and histone modifications (HMs), based on ChIP-seq data in cell lines. As it was shown that gene regulation by TFs differs depending on the CpG class of a promoter, we study the impact of the CpG content in promoters on the associations between TFs and HMs. We suggest an approach based on sparse linear regression models to infer associations between TFs and HMs with respect to CpG content. A study of the partial correlation of HMs for the two classes of high and low CpG content reveals possible CpG dependence and potential candidates for confounding factors in our models. We show that the models are accurate, inferred associations reflect known biological relationships, and we give new insight into associations with respect to CpG content. Moreover, analysis of a ChIP-seq dataset in HepG2 cells of the HM H3K122ac, an HM about little is known, reveals novel TF associations and supports a previously established link to active transcription.