Timo Sztyler
Research Subjects
Timo Sztyler has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Heiner Stuckenschmidt
University of Mannheim
16 shared publications Alexander Diete
Data and Web Science Group, University of Mannheim, Mannheim, Germany
4 shared publications- Gabriele Civitarese
University of Milan
4 shared publications - Julia Gastinger
University of Mannheim
3 shared publications Lydia Weiland
Data and Web Science Group, University of Mannheim, Mannheim, Germany
3 shared publications- Claudio Bettini
University of Milan
2 shared publications
Latest Publications
- 17 September 2023
- Published by Springer Nature
- p. 533-549
Comparing Apples and Oranges? On the Evaluation of Methods for Temporal Knowledge Graph Forecasting
- 17 September 2023
- Published by Springer Nature
- p. 533-549
Comparing Apples and Oranges? On the Evaluation of Methods for Temporal Knowledge Graph Forecasting
- 17 March 2023
- Published by Springer Nature
- p. 107-121
ProcK: Machine Learning for Knowledge-Intensive Processes
- 17 March 2023
- Published by Springer Nature
- p. 107-121
ProcK: Machine Learning for Knowledge-Intensive Processes
- Open Access
ProcK: Machine Learning for Knowledge-Intensive Processes
We present a novel methodology to build powerful predictive process models. Our method, denoted ProcK (Process & Knowledge), relies not only on sequential input data in the form of event logs, but can learn to use a knowledge graph to incorporate information about the attribute values of the events and their mutual relationships. The idea is realized by mapping event attributes to nodes of a knowledge graph and training a sequence model alongside a graph neural network in an end-to-end fashion. This hybrid approach substantially enhances the flexibility and applicability of predictive process monitoring, as both the static and dynamic information residing in the databases of organizations can be directly taken as input data. We demonstrate the potential of ProcK by applying it to a number of predictive process monitoring tasks, including tasks with knowledge graphs available as well as an existing process monitoring benchmark where no such graph is given. The experiments provide evidence that our methodology achieves state-of-the-art performance and improves predictive power when a knowledge graph is available.
Open AccessProcK: Machine Learning for Knowledge-Intensive Processes
We present a novel methodology to build powerful predictive process models. Our method, denoted ProcK (Process & Knowledge), relies not only on sequential input data in the form of event logs, but can learn to use a knowledge graph to incorporate information about the attribute values of the events and their mutual relationships. The idea is realized by mapping event attributes to nodes of a knowledge graph and training a sequence model alongside a graph neural network in an end-to-end fashion. This hybrid approach substantially enhances the flexibility and applicability of predictive process monitoring, as both the static and dynamic information residing in the databases of organizations can be directly taken as input data. We demonstrate the potential of ProcK by applying it to a number of predictive process monitoring tasks, including tasks with knowledge graphs available as well as an existing process monitoring benchmark where no such graph is given. The experiments provide evidence that our methodology achieves state-of-the-art performance and improves predictive power when a knowledge graph is available.
- Open Access
Explaining Neural Matrix Factorization with Gradient Rollback
Explaining the predictions of neural black-box models is an important problem, especially when such models are used in applications where user trust is crucial. Estimating the influence of training examples on a learned neural model's behavior allows us to identify training examples most responsible for a given prediction and, therefore, to faithfully explain the output of a black-box model. The most generally applicable existing method is based on influence functions, which scale poorly for larger sample sizes and models. We propose gradient rollback, a general approach for influence estimation, applicable to neural models where each parameter update step during gradient descent touches a smaller number of parameters, even if the overall number of parameters is large. Neural matrix factorization models trained with gradient descent are part of this model class. These models are popular and have found a wide range of applications in industry. Especially knowledge graph embedding methods, which belong to this class, are used extensively. We show that gradient rollback is highly efficient at both training and test time. Moreover, we show theoretically that the difference between gradient rollback's influence approximation and the true influence on a model's behavior is smaller than known bounds on the stability of stochastic gradient descent. This establishes that gradient rollback is robustly estimating example influence. We also conduct experiments which show that gradient rollback provides faithful explanations for knowledge base completion and recommender datasets.
Open AccessExplaining Neural Matrix Factorization with Gradient Rollback
Explaining the predictions of neural black-box models is an important problem, especially when such models are used in applications where user trust is crucial. Estimating the influence of training examples on a learned neural model's behavior allows us to identify training examples most responsible for a given prediction and, therefore, to faithfully explain the output of a black-box model. The most generally applicable existing method is based on influence functions, which scale poorly for larger sample sizes and models. We propose gradient rollback, a general approach for influence estimation, applicable to neural models where each parameter update step during gradient descent touches a smaller number of parameters, even if the overall number of parameters is large. Neural matrix factorization models trained with gradient descent are part of this model class. These models are popular and have found a wide range of applications in industry. Especially knowledge graph embedding methods, which belong to this class, are used extensively. We show that gradient rollback is highly efficient at both training and test time. Moreover, we show theoretically that the difference between gradient rollback's influence approximation and the true influence on a model's behavior is smaller than known bounds on the stability of stochastic gradient descent. This establishes that gradient rollback is robustly estimating example influence. We also conduct experiments which show that gradient rollback provides faithful explanations for knowledge base completion and recommender datasets.
- Open Access
DOUBLER: Unified Representation Learning of Biological Entities and Documents for Predicting Protein–Disease Relationships
Motivation We propose a system that learns consistent representations of biological entities, such as proteins and diseases, based on a knowledge graph and additional data modalities, like structured annotations and free text describing the entities. In contrast to similar approaches, we explicitly incorporate the consistency of the representations into the learning process. In particular, we use these representations to identify novel proteins associated with diseases; these novel relationships could be used to prioritize protein targets for new drugs. Results We show that our approach outperforms state-of-the-art link prediction algorithms for predicting unknown protein–disease associations. Detailed analysis demonstrates that our approach is most beneficial when additional data modalities, such as free text, are informative. Availability Code and data are available at: https://github.com/nle-sztyler/research-doubler Contact timo.sztyler{at}neclab.eu
Open AccessDOUBLER: Unified Representation Learning of Biological Entities and Documents for Predicting Protein–Disease Relationships
Motivation We propose a system that learns consistent representations of biological entities, such as proteins and diseases, based on a knowledge graph and additional data modalities, like structured annotations and free text describing the entities. In contrast to similar approaches, we explicitly incorporate the consistency of the representations into the learning process. In particular, we use these representations to identify novel proteins associated with diseases; these novel relationships could be used to prioritize protein targets for new drugs. Results We show that our approach outperforms state-of-the-art link prediction algorithms for predicting unknown protein–disease associations. Detailed analysis demonstrates that our approach is most beneficial when additional data modalities, such as free text, are informative. Availability Code and data are available at: https://github.com/nle-sztyler/research-doubler Contact timo.sztyler{at}neclab.eu
- 23 July 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 33 (1) , 209-223
POLARIS: Probabilistic and Ontological Activity Recognition in Smart-Homes
Recognition of activities of daily living (ADLs) is an enabling technology for several ubiquitous computing applications. Most activity recognition systems rely on supervised learning to extract activity models from labeled datasets. A problem with that approach is the acquisition of comprehensive activity datasets, which is an expensive task. The problem is particularly challenging when focusing on complex ADLs characterized by large variability of execution. Moreover, several activity recognition systems are limited to offline recognition, while many applications claim for online activity recognition. In this paper, we propose POLARIS, a framework for unsupervised activity recognition. POLARIS can recognize complex ADLs exploiting the semantics of activities, context data, and sensors. Through ontological reasoning, our algorithm derives semantic correlations among activities and sensor events. By matching observed events with semantic correlations, a statistical reasoner formulates initial hypotheses about the occurred activities. Those hypotheses are refined through probabilistic reasoning, exploiting semantic constraints derived from the ontology. Our system supports online recognition, thanks to a novel segmentation algorithm. Extensive experiments with real-world datasets show that the accuracy of our unsupervised method is comparable to the one of supervised approaches. Moreover, the online version of our system achieves essentially the same accuracy of the offline version.
- 23 July 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 33 (1) , 209-223
POLARIS: Probabilistic and Ontological Activity Recognition in Smart-Homes
Recognition of activities of daily living (ADLs) is an enabling technology for several ubiquitous computing applications. Most activity recognition systems rely on supervised learning to extract activity models from labeled datasets. A problem with that approach is the acquisition of comprehensive activity datasets, which is an expensive task. The problem is particularly challenging when focusing on complex ADLs characterized by large variability of execution. Moreover, several activity recognition systems are limited to offline recognition, while many applications claim for online activity recognition. In this paper, we propose POLARIS, a framework for unsupervised activity recognition. POLARIS can recognize complex ADLs exploiting the semantics of activities, context data, and sensors. Through ontological reasoning, our algorithm derives semantic correlations among activities and sensor events. By matching observed events with semantic correlations, a statistical reasoner formulates initial hypotheses about the occurred activities. Those hypotheses are refined through probabilistic reasoning, exploiting semantic constraints derived from the ontology. Our system supports online recognition, thanks to a novel segmentation algorithm. Extensive experiments with real-world datasets show that the accuracy of our unsupervised method is comparable to the one of supervised approaches. Moreover, the online version of our system achieves essentially the same accuracy of the offline version.
- Open Access
Exploring Semi-Supervised Methods for Labeling Support in Multimodal Datasets
Working with multimodal datasets is a challenging task as it requires annotations which often are time consuming and difficult to acquire. This includes in particular video recordings which often need to be watched as a whole before they can be labeled. Additionally, other modalities like acceleration data are often recorded alongside a video. For that purpose, we created an annotation tool that enables to annotate datasets of video and inertial sensor data. In contrast to most existing approaches, we focus on semi-supervised labeling support to infer labels for the whole dataset. This means, after labeling a small set of instances our system is able to provide labeling recommendations. We aim to rely on the acceleration data of a wrist-worn sensor to support the labeling of a video recording. For that purpose, we apply template matching to identify time intervals of certain activities. We test our approach on three datasets, one containing warehouse picking activities, one consisting of activities of daily living and one about meal preparations. Our results show that the presented method is able to give hints to annotators about possible label candidates.
Open AccessExploring Semi-Supervised Methods for Labeling Support in Multimodal Datasets
Working with multimodal datasets is a challenging task as it requires annotations which often are time consuming and difficult to acquire. This includes in particular video recordings which often need to be watched as a whole before they can be labeled. Additionally, other modalities like acceleration data are often recorded alongside a video. For that purpose, we created an annotation tool that enables to annotate datasets of video and inertial sensor data. In contrast to most existing approaches, we focus on semi-supervised labeling support to infer labels for the whole dataset. This means, after labeling a small set of instances our system is able to provide labeling recommendations. We aim to rely on the acceleration data of a wrist-worn sensor to support the labeling of a video recording. For that purpose, we apply template matching to identify time intervals of certain activities. We test our approach on three datasets, one containing warehouse picking activities, one consisting of activities of daily living and one about meal preparations. Our results show that the presented method is able to give hints to annotators about possible label candidates.
- 1 March 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 1, 259-264
Modeling and Reasoning with ProbLog: An Application in Recognizing Complex Activities
Smart-home activity recognition is an enabling tool for a wide range of ambient assisted living applications. The recognition of ADLs usually relies on supervised learning or knowledge-based reasoning techniques. In order to overcome the well-known limitations of those two approaches and, at the same time, to combine their strengths to improve the recognition rate, many researchers investigated Markov Logic Networks (MLNs). However, MLNs require a non-trivial effort by experts to properly model probabilities in terms of weights. In this paper, we propose a novel method based on ProbLog. ProbLog is a probabilistic extension of Prolog, which allows to explicitly define probabilistic facts and rules. With respect to MLN, the inference mode of ProbLog is based on the closed-world assumption and it has faster response times. We propose a simple and flexible ProbLog model, which we exploit to recognize complex ADLs in an online fashion. Considering a dataset with 21 subjects, our results show that our method reaches high F-measure (83%). Moreover, we also show that the response time of ProbLog is satisfying for real-time applications.
- 1 March 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 1, 259-264
Modeling and Reasoning with ProbLog: An Application in Recognizing Complex Activities
Smart-home activity recognition is an enabling tool for a wide range of ambient assisted living applications. The recognition of ADLs usually relies on supervised learning or knowledge-based reasoning techniques. In order to overcome the well-known limitations of those two approaches and, at the same time, to combine their strengths to improve the recognition rate, many researchers investigated Markov Logic Networks (MLNs). However, MLNs require a non-trivial effort by experts to properly model probabilities in terms of weights. In this paper, we propose a novel method based on ProbLog. ProbLog is a probabilistic extension of Prolog, which allows to explicitly define probabilistic facts and rules. With respect to MLN, the inference mode of ProbLog is based on the closed-world assumption and it has faster response times. We propose a simple and flexible ProbLog model, which we exploit to recognize complex ADLs in an online fashion. Considering a dataset with 21 subjects, our results show that our method reaches high F-measure (83%). Moreover, we also show that the response time of ProbLog is satisfying for real-time applications.
- 1 March 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 488-491
Improving Motion-based Activity Recognition with Ego-centric Vision
Human activity recognition using wearable computers is an active area of research in pervasive computing. Existing works mainly focus on the recognition of physical activities or so called activities of daily living by relying on inertial or interaction sensors. A main issue of those studies is that they often focus on critical applications like health care but without any evidence that the monitored activities really took place. In our work, we aim to overcome this limitation and present a multi-modal egocentricbased activity recognition approach which is able to recognize the critical objects. As it is unfeasible to expect always a high quality camera view, we enrich the vision features with inertial sensor data that represents the users' arm movement. This enables us to compensate the weaknesses of the respective sensors. We present first results of our ongoing work on this topic.
- 1 March 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 488-491
Improving Motion-based Activity Recognition with Ego-centric Vision
Human activity recognition using wearable computers is an active area of research in pervasive computing. Existing works mainly focus on the recognition of physical activities or so called activities of daily living by relying on inertial or interaction sensors. A main issue of those studies is that they often focus on critical applications like health care but without any evidence that the monitored activities really took place. In our work, we aim to overcome this limitation and present a multi-modal egocentricbased activity recognition approach which is able to recognize the critical objects. As it is unfeasible to expect always a high quality camera view, we enrich the vision features with inertial sensor data that represents the users' arm movement. This enables us to compensate the weaknesses of the respective sensors. We present first results of our ongoing work on this topic.
- 1 March 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 15-20
Towards Systematic Benchmarking of Activity Recognition Algorithms
In this paper we propose a benchmarking frame-work for evaluating activity recognition methods. We use an ontology for describing activity recognition datasets on the meta-level and propose a fine-grained annotation scheme for activity recognition datasets. Given a method that implements a defined interface, an evaluation client can be used to automatically run experiments on annotated datasets. Our framework helps to find relevant datasets and makes results reproducible by fixing concrete experimental settings. We show how to use the framework and report about a preliminary evaluation experiment.
- 1 March 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 15-20
Towards Systematic Benchmarking of Activity Recognition Algorithms
In this paper we propose a benchmarking frame-work for evaluating activity recognition methods. We use an ontology for describing activity recognition datasets on the meta-level and propose a fine-grained annotation scheme for activity recognition datasets. Given a method that implements a defined interface, an evaluation client can be used to automatically run experiments on annotated datasets. Our framework helps to find relevant datasets and makes results reproducible by fixing concrete experimental settings. We show how to use the framework and report about a preliminary evaluation experiment.