Cyprien Hoelzl
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Cyprien Hoelzl has a total of 25 co-authors. Here are the top co-authors by number of shared publications:
- Eleni Chatzi
ETH Zurich
9 shared publications - Vasilis Dertimanis
ETH Zurich
6 shared publications - Giacomo Arcieri
Institute of Structural Engineering ETH Zürich 8093 Zürich Switzerland
4 shared publications - Daniel Straub
Engineering Risk Analysis Group, Technical University of Munich, Munich, Germany
3 shared publications - Konstantinos G. Papakonstantinou
Pennsylvania State Univ
3 shared publications - Lucian Ancu
Swiss Federal Railways (SBB)
3 shared publications
Latest Publications
- Open Access
- 13 October 2026
- Published by Elsevier in Measurement
- Vol. 292
Automating traffic monitoring with SHM sensor networks via vision-supervised deep learning
- 13 October 2026
- Published by Elsevier in Measurement
- Vol. 292
Open AccessAutomating traffic monitoring with SHM sensor networks via vision-supervised deep learning
- Open Access
- 31 May 2024
- Published by Springer Nature in Machine Learning
- Vol. 113 (10) , 7967-7995
POMDP inference and robust solution via deep reinforcement learning: an application to railway optimal maintenance
Partially Observable Markov Decision Processes (POMDPs) can model complex sequential decision-making problems under stochastic and uncertain environments. A main reason hindering their broad adoption in real-world applications is the unavailability of a suitable POMDP model or a simulator thereof. Available solution algorithms, such as Reinforcement Learning (RL), typically benefit from the knowledge of the transition dynamics and the observation generating process, which are often unknown and non-trivial to infer. In this work, we propose a combined framework for inference and robust solution of POMDPs via deep RL. First, all transition and observation model parameters are jointly inferred via Markov Chain Monte Carlo sampling of a hidden Markov model, which is conditioned on actions, in order to recover full posterior distributions from the available data. The POMDP with uncertain parameters is then solved via deep RL techniques with the parameter distributions incorporated into the solution via domain randomization, in order to develop solutions that are robust to model uncertainty. As a further contribution, we compare the use of Transformers and long short-term memory networks, which constitute model-free RL solutions and work directly on the observation space, with an approach termed the belief-input method, which works on the belief space by exploiting the learned POMDP model for belief inference. We apply these methods to the real-world problem of optimal maintenance planning for railway assets and compare the results with the current real-life policy. We show that the RL policy learned by the belief-input method is able to outperform the real-life policy by yielding significantly reduced life-cycle costs.
- 31 May 2024
- Published by Springer Nature in Machine Learning
- Vol. 113 (10) , 7967-7995
Open AccessPOMDP inference and robust solution via deep reinforcement learning: an application to railway optimal maintenance
Partially Observable Markov Decision Processes (POMDPs) can model complex sequential decision-making problems under stochastic and uncertain environments. A main reason hindering their broad adoption in real-world applications is the unavailability of a suitable POMDP model or a simulator thereof. Available solution algorithms, such as Reinforcement Learning (RL), typically benefit from the knowledge of the transition dynamics and the observation generating process, which are often unknown and non-trivial to infer. In this work, we propose a combined framework for inference and robust solution of POMDPs via deep RL. First, all transition and observation model parameters are jointly inferred via Markov Chain Monte Carlo sampling of a hidden Markov model, which is conditioned on actions, in order to recover full posterior distributions from the available data. The POMDP with uncertain parameters is then solved via deep RL techniques with the parameter distributions incorporated into the solution via domain randomization, in order to develop solutions that are robust to model uncertainty. As a further contribution, we compare the use of Transformers and long short-term memory networks, which constitute model-free RL solutions and work directly on the observation space, with an approach termed the belief-input method, which works on the belief space by exploiting the learned POMDP model for belief inference. We apply these methods to the real-world problem of optimal maintenance planning for railway assets and compare the results with the current real-life policy. We show that the RL policy learned by the belief-input method is able to outperform the real-life policy by yielding significantly reduced life-cycle costs.
- Open Access
- 29 December 2023
- Published by Wiley in Structural Control and Health Monitoring
- Vol. 2023, 1-18
A Model-Based Bayesian Inference Approach for On-Board Monitoring of Rail Roughness Profiles: Application on Field Measurement Data of the Swiss Federal Railways Network
According to the International Union of Railways, railway networks count more than one million kilometers of tracks worldwide, a number that is to rise further as the goal is to promote rail transportation as a sustainable means to face the challenge of increased mobility. However, such a vast expansion further necessitates efficient and reliable infrastructure monitoring schemes able to guarantee the quality and safety of rail transportation. Traditional monitoring approaches, relying on visual inspection and portable measuring devices, cannot rise to the task as they do not allow for continuous inspection of extended portions of rail infrastructure. Therefore, mobile monitoring methodologies based on dedicated diagnostic vehicles have emerged as an alternative. Despite revolutionizing traditional monitoring methods, such vehicles are usually expensive and can only operate under the suspension of regular rail service. In this work, we propose an alternative approach for mobile sensing of railway infrastructure based on on-board monitoring data collected from low-cost vibration sensors, e.g., accelerometers, which can be mounted on in-service trains. Specifically, we focus on identifying the roughness profile of the tracks and propose a fusion of reduced-order vehicle models with a Bayesian inference approach for joint input-state estimation. To enhance the inference, we opt for a prior updating of the vehicle model parameters on the basis of an unscented Kalman filter and available measurements from a diagnostic vehicle. The key contributions of this work are (i) the consideration of the dynamic interaction between trains and tracks, which is usually ignored in rail roughness estimation, (ii) the adoption of reduced train vehicle models that decrease the computational effort of the identification task, (iii) the updating of the vehicle parameters to account for inconsistencies in the model used, and (iv) the application of the proposed methodology to actual acceleration measurements collected from a diagnostic vehicle of the Swiss Federal Railways network.
- 29 December 2023
- Published by Wiley in Structural Control and Health Monitoring
- Vol. 2023, 1-18
Open AccessA Model-Based Bayesian Inference Approach for On-Board Monitoring of Rail Roughness Profiles: Application on Field Measurement Data of the Swiss Federal Railways Network
According to the International Union of Railways, railway networks count more than one million kilometers of tracks worldwide, a number that is to rise further as the goal is to promote rail transportation as a sustainable means to face the challenge of increased mobility. However, such a vast expansion further necessitates efficient and reliable infrastructure monitoring schemes able to guarantee the quality and safety of rail transportation. Traditional monitoring approaches, relying on visual inspection and portable measuring devices, cannot rise to the task as they do not allow for continuous inspection of extended portions of rail infrastructure. Therefore, mobile monitoring methodologies based on dedicated diagnostic vehicles have emerged as an alternative. Despite revolutionizing traditional monitoring methods, such vehicles are usually expensive and can only operate under the suspension of regular rail service. In this work, we propose an alternative approach for mobile sensing of railway infrastructure based on on-board monitoring data collected from low-cost vibration sensors, e.g., accelerometers, which can be mounted on in-service trains. Specifically, we focus on identifying the roughness profile of the tracks and propose a fusion of reduced-order vehicle models with a Bayesian inference approach for joint input-state estimation. To enhance the inference, we opt for a prior updating of the vehicle model parameters on the basis of an unscented Kalman filter and available measurements from a diagnostic vehicle. The key contributions of this work are (i) the consideration of the dynamic interaction between trains and tracks, which is usually ignored in rail roughness estimation, (ii) the adoption of reduced train vehicle models that decrease the computational effort of the identification task, (iii) the updating of the vehicle parameters to account for inconsistencies in the model used, and (iv) the application of the proposed methodology to actual acceleration measurements collected from a diagnostic vehicle of the Swiss Federal Railways network.
- Open Access
- 3 October 2023
- Published by Elsevier in Mechanical Systems and Signal Processing
- Vol. 204
Vold–Kalman filter order tracking of axle box accelerations for track stiffness assessment
- 3 October 2023
- Published by Elsevier in Mechanical Systems and Signal Processing
- Vol. 204
Open AccessVold–Kalman filter order tracking of axle box accelerations for track stiffness assessment
- Open Access
POMDP inference and robust solution via deep reinforcement learning: An application to railway optimal maintenance
Partially Observable Markov Decision Processes (POMDPs) can model complex sequential decision-making problems under stochastic and uncertain environments. A main reason hindering their broad adoption in real-world applications is the lack of availability of a suitable POMDP model or a simulator thereof. Available solution algorithms, such as Reinforcement Learning (RL), require the knowledge of the transition dynamics and the observation generating process, which are often unknown and non-trivial to infer. In this work, we propose a combined framework for inference and robust solution of POMDPs via deep RL. First, all transition and observation model parameters are jointly inferred via Markov Chain Monte Carlo sampling of a hidden Markov model, which is conditioned on actions, in order to recover full posterior distributions from the available data. The POMDP with uncertain parameters is then solved via deep RL techniques with the parameter distributions incorporated into the solution via domain randomization, in order to develop solutions that are robust to model uncertainty. As a further contribution, we compare the use of transformers and long short-term memory networks, which constitute model-free RL solutions, with a model-based/model-free hybrid approach. We apply these methods to the real-world problem of optimal maintenance planning for railway assets.
Open AccessPOMDP inference and robust solution via deep reinforcement learning: An application to railway optimal maintenance
Partially Observable Markov Decision Processes (POMDPs) can model complex sequential decision-making problems under stochastic and uncertain environments. A main reason hindering their broad adoption in real-world applications is the lack of availability of a suitable POMDP model or a simulator thereof. Available solution algorithms, such as Reinforcement Learning (RL), require the knowledge of the transition dynamics and the observation generating process, which are often unknown and non-trivial to infer. In this work, we propose a combined framework for inference and robust solution of POMDPs via deep RL. First, all transition and observation model parameters are jointly inferred via Markov Chain Monte Carlo sampling of a hidden Markov model, which is conditioned on actions, in order to recover full posterior distributions from the available data. The POMDP with uncertain parameters is then solved via deep RL techniques with the parameter distributions incorporated into the solution via domain randomization, in order to develop solutions that are robust to model uncertainty. As a further contribution, we compare the use of transformers and long short-term memory networks, which constitute model-free RL solutions, with a model-based/model-free hybrid approach. We apply these methods to the real-world problem of optimal maintenance planning for railway assets.
- Open Access
Fusing Expert Knowledge with Monitoring Data for Condition Assessment of Railway Welds
Monitoring information can facilitate the condition assessment of railway infrastructure, via delivery of data that is informative on condition. A primary instance of such data is found in Axle Box Accelerations (ABAs), which track the dynamic vehicle/track interaction. Such sensors have been installed on specialized monitoring trains, as well as on in-service On-Board Monitoring (OBM) vehicles across Europe, enabling a continuous assessment of railway track condition. However, ABA measurements come with uncertainties that stem from noise corrupt data and the non-linear rail–wheel contact dynamics, as well as variations in environmental and operational conditions. These uncertainties pose a challenge for the condition assessment of rail welds through existing assessment tools. In this work, we use expert feedback as a complementary information source, which allows the narrowing down of these uncertainties, and, ultimately, refines assessment. Over the past year, with the support of the Swiss Federal Railways (SBB), we have assembled a database of expert evaluations on the condition of rail weld samples that have been diagnosed as critical via ABA monitoring. In this work, we fuse features derived from the ABA data with expert feedback, in order to refine defection of faulty (defect) welds. Three models are employed to this end; Binary Classification and Random Forest (RF) models, as well as a Bayesian Logistic Regression (BLR) scheme. The RF and BLR models proved superior to the Binary Classification model, while the BLR model further delivered a probability of prediction, quantifying the confidence we might attribute to the assigned labels. We explain that the classification task necessarily suffers high uncertainty, which is a result of faulty ground truth labels, and explain the value of continuously tracking the weld condition.
Open AccessFusing Expert Knowledge with Monitoring Data for Condition Assessment of Railway Welds
Monitoring information can facilitate the condition assessment of railway infrastructure, via delivery of data that is informative on condition. A primary instance of such data is found in Axle Box Accelerations (ABAs), which track the dynamic vehicle/track interaction. Such sensors have been installed on specialized monitoring trains, as well as on in-service On-Board Monitoring (OBM) vehicles across Europe, enabling a continuous assessment of railway track condition. However, ABA measurements come with uncertainties that stem from noise corrupt data and the non-linear rail–wheel contact dynamics, as well as variations in environmental and operational conditions. These uncertainties pose a challenge for the condition assessment of rail welds through existing assessment tools. In this work, we use expert feedback as a complementary information source, which allows the narrowing down of these uncertainties, and, ultimately, refines assessment. Over the past year, with the support of the Swiss Federal Railways (SBB), we have assembled a database of expert evaluations on the condition of rail weld samples that have been diagnosed as critical via ABA monitoring. In this work, we fuse features derived from the ABA data with expert feedback, in order to refine defection of faulty (defect) welds. Three models are employed to this end; Binary Classification and Random Forest (RF) models, as well as a Bayesian Logistic Regression (BLR) scheme. The RF and BLR models proved superior to the Binary Classification model, while the BLR model further delivered a probability of prediction, quantifying the confidence we might attribute to the assigned labels. We explain that the classification task necessarily suffers high uncertainty, which is a result of faulty ground truth labels, and explain the value of continuously tracking the weld condition.
- Open Access
Bridging POMDPs and Bayesian decision making for robust maintenance planning under model uncertainty: An application to railway systems
Structural Health Monitoring (SHM) describes a process for inferring quantifiable metrics of structural condition, which can serve as input to support decisions on the operation and maintenance of infrastructure assets. Given the long lifespan of critical structures, this problem can be cast as a sequential decision making problem over prescribed horizons. Partially Observable Markov Decision Processes (POMDPs) offer a formal framework to solve the underlying optimal planning task. However, two issues can undermine the POMDP solutions. Firstly, the need for a model that can adequately describe the evolution of the structural condition under deterioration or corrective actions and, secondly, the non-trivial task of recovery of the observation process parameters from available monitoring data. Despite these potential challenges, the adopted POMDP models do not typically account for uncertainty on model parameters, leading to solutions which can be unrealistically confident. In this work, we address both key issues. We present a framework to estimate POMDP transition and observation model parameters directly from available data, via Markov Chain Monte Carlo (MCMC) sampling of a Hidden Markov Model (HMM) conditioned on actions. The MCMC inference estimates distributions of the involved model parameters. We then form and solve the POMDP problem by exploiting the inferred distributions, to derive solutions that are robust to model uncertainty. We successfully apply our approach on maintenance planning for railway track assets on the basis of a "fractal value" indicator, which is computed from actual railway monitoring data.
Open AccessBridging POMDPs and Bayesian decision making for robust maintenance planning under model uncertainty: An application to railway systems
Structural Health Monitoring (SHM) describes a process for inferring quantifiable metrics of structural condition, which can serve as input to support decisions on the operation and maintenance of infrastructure assets. Given the long lifespan of critical structures, this problem can be cast as a sequential decision making problem over prescribed horizons. Partially Observable Markov Decision Processes (POMDPs) offer a formal framework to solve the underlying optimal planning task. However, two issues can undermine the POMDP solutions. Firstly, the need for a model that can adequately describe the evolution of the structural condition under deterioration or corrective actions and, secondly, the non-trivial task of recovery of the observation process parameters from available monitoring data. Despite these potential challenges, the adopted POMDP models do not typically account for uncertainty on model parameters, leading to solutions which can be unrealistically confident. In this work, we address both key issues. We present a framework to estimate POMDP transition and observation model parameters directly from available data, via Markov Chain Monte Carlo (MCMC) sampling of a Hidden Markov Model (HMM) conditioned on actions. The MCMC inference estimates distributions of the involved model parameters. We then form and solve the POMDP problem by exploiting the inferred distributions, to derive solutions that are robust to model uncertainty. We successfully apply our approach on maintenance planning for railway track assets on the basis of a "fractal value" indicator, which is computed from actual railway monitoring data.
- 19 June 2022
- Published by Springer Nature
- p. 711-720
Weld Condition Monitoring Using Expert Informed Extreme Value Analysis
- 19 June 2022
- Published by Springer Nature
- p. 711-720
Weld Condition Monitoring Using Expert Informed Extreme Value Analysis
- 28 October 2020
- Published by Springer Nature
- p. 87-92
Axle Box Accelerometer Signal Identification and Modelling
- 28 October 2020
- Published by Springer Nature
- p. 87-92
Axle Box Accelerometer Signal Identification and Modelling