Giovanni Licitra
Giovanni Licitra has a total of 44 co-authors. Here are the top co-authors by number of shared publications:
M. Diehl
4 shared publicationsP. Williams
4 shared publicationsR. Ruiterkamp
4 shared publicationsMoritz Diehl
Department of Microsystems Engineering University of Freiburg Freiburg im Breisgau Germany
3 shared publicationsPaul Williams
Ampyx Power B.V. The Hague The Netherlands
3 shared publicationsRichard Ruiterkamp
Ampyx Power B.V. The Hague The Netherlands
3 shared publications
Latest Publications
- Open Access
- 1 February 2023
- Published by Springer Nature in Scientific Reports
- Vol. 13 (1) , 1-12
Disease severity classification using passively collected smartphone-based keystroke dynamics within multiple sclerosis
Multiple Sclerosis (MS) is a progressive demyelinating disease of the central nervous system characterised by a wide range of motor and non-motor symptoms. The level of disability of people with MS (pwMS) is based on a wide range of clinical measures, though their frequency of evaluation and inaccuracies coming from objective and self-reported evaluations limits these assessments. Alternatively, remote health monitoring through devices can offer a cost-efficient solution to gather more reliable, objective measures continuously. Measuring smartphone keyboard interactions is a promising tool since typing and, thus, keystroke dynamics are likely influenced by symptoms that pwMS can experience. Therefore, this paper aims to investigate whether keyboard interactions gathered on a person’s smartphone can provide insight into the clinical status of pwMS leveraging machine learning techniques. In total, 24 Healthy Controls (HC) and 102 pwMS were followed for one year. Next to continuous data generated via smartphone interactions, clinical outcome measures were collected and used as targets to train four independent multivariate binary classification pipelines in discerning pwMS versus HC and estimating the level of disease severity, manual dexterity and cognitive capabilities. The final models yielded an AUC-ROC in the hold-out set above 0.7, with the highest performance obtained in estimating the level of fine motor skills (AUC-ROC=0.753). These findings show that keyboard interactions combined with machine learning techniques can be used as an unobtrusive monitoring tool to estimate various levels of clinical disability in pwMS from daily activities and with a high frequency of sampling without increasing patient burden.
- 1 February 2023
- Published by Springer Nature in Scientific Reports
- Vol. 13 (1) , 1-12
Open AccessDisease severity classification using passively collected smartphone-based keystroke dynamics within multiple sclerosis
Multiple Sclerosis (MS) is a progressive demyelinating disease of the central nervous system characterised by a wide range of motor and non-motor symptoms. The level of disability of people with MS (pwMS) is based on a wide range of clinical measures, though their frequency of evaluation and inaccuracies coming from objective and self-reported evaluations limits these assessments. Alternatively, remote health monitoring through devices can offer a cost-efficient solution to gather more reliable, objective measures continuously. Measuring smartphone keyboard interactions is a promising tool since typing and, thus, keystroke dynamics are likely influenced by symptoms that pwMS can experience. Therefore, this paper aims to investigate whether keyboard interactions gathered on a person’s smartphone can provide insight into the clinical status of pwMS leveraging machine learning techniques. In total, 24 Healthy Controls (HC) and 102 pwMS were followed for one year. Next to continuous data generated via smartphone interactions, clinical outcome measures were collected and used as targets to train four independent multivariate binary classification pipelines in discerning pwMS versus HC and estimating the level of disease severity, manual dexterity and cognitive capabilities. The final models yielded an AUC-ROC in the hold-out set above 0.7, with the highest performance obtained in estimating the level of fine motor skills (AUC-ROC=0.753). These findings show that keyboard interactions combined with machine learning techniques can be used as an unobtrusive monitoring tool to estimate various levels of clinical disability in pwMS from daily activities and with a high frequency of sampling without increasing patient burden.
- Open Access
- 2 December 2022
- Published by Springer Nature in Research Square
Disease Severity Classification using Passively Collected Smartphone-Based Keystroke Dynamics within Multiple Sclerosis
Multiple Sclerosis (MS) is a progressive demyelinating disease of the central nervous system characterised by a wide range of motor and non-motor symptoms. The level of disability of people with MS (pwMS) is based on a wide range of clinical measures, though their frequency of evaluation and inaccuracies coming from objective and self-reported evaluations limits these assessments. Alternatively, remote health monitoring through devices can offer a cost-efficient solution to gather more reliable, objective measures continuously. Measuring smartphone keyboard interactions is a promising tool since typing and, thus, keystroke dynamics are likely influenced by symptoms that pwMS can experience. Therefore, this paper aims to investigate whether keyboard interactions gathered on a person’s smartphone can provide insight into the clinical status of pwMS leveraging machine learning techniques. In total, 24 Healthy Controls (HC) and 102 pwMS were followed for one year. Next to continuous data generated via smartphone interactions, clinical outcome measures were collected and used as targets to train four independent multivariate binary classification pipelines in discerning pwMS versus HC and estimating the level of disease severity, manual dexterity and cognitive capabilities. The final models yielded an AUC-ROC in the hold-out set above 0.7, with the highest performance obtained in estimating the level of fine motor skills (AUC-ROC=0.753). These findings show that keyboard interactions combined with machine learning techniques can be used as an unobtrusive monitoring tool to estimate various levels of clinical disability in pwMS from daily activities and with a high frequency of sampling without increasing patient burden.
- 2 December 2022
- Published by Springer Nature in Research Square
Open AccessDisease Severity Classification using Passively Collected Smartphone-Based Keystroke Dynamics within Multiple Sclerosis
Multiple Sclerosis (MS) is a progressive demyelinating disease of the central nervous system characterised by a wide range of motor and non-motor symptoms. The level of disability of people with MS (pwMS) is based on a wide range of clinical measures, though their frequency of evaluation and inaccuracies coming from objective and self-reported evaluations limits these assessments. Alternatively, remote health monitoring through devices can offer a cost-efficient solution to gather more reliable, objective measures continuously. Measuring smartphone keyboard interactions is a promising tool since typing and, thus, keystroke dynamics are likely influenced by symptoms that pwMS can experience. Therefore, this paper aims to investigate whether keyboard interactions gathered on a person’s smartphone can provide insight into the clinical status of pwMS leveraging machine learning techniques. In total, 24 Healthy Controls (HC) and 102 pwMS were followed for one year. Next to continuous data generated via smartphone interactions, clinical outcome measures were collected and used as targets to train four independent multivariate binary classification pipelines in discerning pwMS versus HC and estimating the level of disease severity, manual dexterity and cognitive capabilities. The final models yielded an AUC-ROC in the hold-out set above 0.7, with the highest performance obtained in estimating the level of fine motor skills (AUC-ROC=0.753). These findings show that keyboard interactions combined with machine learning techniques can be used as an unobtrusive monitoring tool to estimate various levels of clinical disability in pwMS from daily activities and with a high frequency of sampling without increasing patient burden.
- Open Access
- 7 November 2022
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 24 (11) , e37614
The Use of Smartphone Keystroke Dynamics to Passively Monitor Upper Limb and Cognitive Function in Multiple Sclerosis: Longitudinal Analysis
Background: Typing on smartphones, which has become a near daily activity, requires both upper limb and cognitive function. Analysis of keyboard interactions during regular typing, that is, keystroke dynamics, could therefore potentially be utilized for passive and continuous monitoring of function in patients with multiple sclerosis. Objective: To determine whether passively acquired smartphone keystroke dynamics correspond to multiple sclerosis outcomes, we investigated the association between keystroke dynamics and clinical outcomes (upper limb and cognitive function). This association was investigated longitudinally in order to study within-patient changes independently of between-patient differences. Methods: During a 1-year follow-up, arm function and information processing speed were assessed every 3 months in 102 patients with multiple sclerosis with the Nine-Hole Peg Test and Symbol Digit Modalities Test, respectively. Keystroke-dynamics data were continuously obtained from regular typing on the participants’ own smartphones. Press-and-release latency of the alphanumeric keys constituted the fine motor score cluster, while latency of the punctuation and backspace keys constituted the cognition score cluster. The association over time between keystroke clusters and the corresponding clinical outcomes was assessed with linear mixed models with subjects as random intercepts. By centering around the mean and calculating deviation scores within subjects, between-subject and within-subject effects were distinguished. Results: Mean (SD) scores for the fine motor score cluster and cognition score cluster were 0.43 (0.16) and 0.94 (0.41) seconds, respectively. The fine motor score cluster was significantly associated with the Nine-Hole Peg Test: between-subject β was 15.9 (95% CI 12.2-19.6) and within-subject β was 6.9 (95% CI 2.0-11.9). The cognition score cluster was significantly associated with the Symbol Digit Modalities Test between subjects (between-subject β –11.2, 95% CI –17.3 to –5.2) but not within subjects (within-subject β –0.4, 95% CI –5.6 to 4.9). Conclusions: Smartphone keystroke dynamics were longitudinally associated with multiple sclerosis outcomes. Worse arm function corresponded with longer latency in typing both across and within patients. Worse processing speed corresponded with higher latency in using punctuation and backspace keys across subjects. Hence, keystroke dynamics are a potential digital biomarker for remote monitoring and predicting clinical outcomes in patients with multiple sclerosis. Trial Registration: Netherlands Trial Register NTR7268; https://trialsearch.who.int/Trial2.aspx?TrialID=NTR7268
- 7 November 2022
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 24 (11) , e37614
Open AccessThe Use of Smartphone Keystroke Dynamics to Passively Monitor Upper Limb and Cognitive Function in Multiple Sclerosis: Longitudinal Analysis
Background: Typing on smartphones, which has become a near daily activity, requires both upper limb and cognitive function. Analysis of keyboard interactions during regular typing, that is, keystroke dynamics, could therefore potentially be utilized for passive and continuous monitoring of function in patients with multiple sclerosis. Objective: To determine whether passively acquired smartphone keystroke dynamics correspond to multiple sclerosis outcomes, we investigated the association between keystroke dynamics and clinical outcomes (upper limb and cognitive function). This association was investigated longitudinally in order to study within-patient changes independently of between-patient differences. Methods: During a 1-year follow-up, arm function and information processing speed were assessed every 3 months in 102 patients with multiple sclerosis with the Nine-Hole Peg Test and Symbol Digit Modalities Test, respectively. Keystroke-dynamics data were continuously obtained from regular typing on the participants’ own smartphones. Press-and-release latency of the alphanumeric keys constituted the fine motor score cluster, while latency of the punctuation and backspace keys constituted the cognition score cluster. The association over time between keystroke clusters and the corresponding clinical outcomes was assessed with linear mixed models with subjects as random intercepts. By centering around the mean and calculating deviation scores within subjects, between-subject and within-subject effects were distinguished. Results: Mean (SD) scores for the fine motor score cluster and cognition score cluster were 0.43 (0.16) and 0.94 (0.41) seconds, respectively. The fine motor score cluster was significantly associated with the Nine-Hole Peg Test: between-subject β was 15.9 (95% CI 12.2-19.6) and within-subject β was 6.9 (95% CI 2.0-11.9). The cognition score cluster was significantly associated with the Symbol Digit Modalities Test between subjects (between-subject β –11.2, 95% CI –17.3 to –5.2) but not within subjects (within-subject β –0.4, 95% CI –5.6 to 4.9). Conclusions: Smartphone keystroke dynamics were longitudinally associated with multiple sclerosis outcomes. Worse arm function corresponded with longer latency in typing both across and within patients. Worse processing speed corresponded with higher latency in using punctuation and backspace keys across subjects. Hence, keystroke dynamics are a potential digital biomarker for remote monitoring and predicting clinical outcomes in patients with multiple sclerosis. Trial Registration: Netherlands Trial Register NTR7268; https://trialsearch.who.int/Trial2.aspx?TrialID=NTR7268
- Open Access
- 23 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Exploring a Bilingual Next Word Predictor for a Federated Learning Mobile Application
We explore a bilingual next-word predictor (NWP) under federated optimization for a mobile application. A character-based LSTM is server-trained on English and Dutch texts from a custom parallel corpora. This is used as the target performance. We simulate a federated learning environment to assess the feasibility of distributed training for the same model. The popular Federated Averaging (FedAvg) algorithm is used as the aggregation method. We show that the federated LSTM achieves decent performance, yet it is still sub-optimal. We suggest possible next steps to bridge this performance gap. Furthermore, we explore the effects of language imbalance varying the ratio of English and Dutch training texts (or clients). We show the model upholds performance (of the balanced case) up and until a 80/20 imbalance before decaying rapidly. Lastly, we describe the implementation of local client training, word prediction and client-server communication in a custom virtual keyboard for Android platforms. Additionally, homomorphic encryption is applied to provide with secure aggregation guarding the user from malicious servers.
- 23 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessExploring a Bilingual Next Word Predictor for a Federated Learning Mobile Application
We explore a bilingual next-word predictor (NWP) under federated optimization for a mobile application. A character-based LSTM is server-trained on English and Dutch texts from a custom parallel corpora. This is used as the target performance. We simulate a federated learning environment to assess the feasibility of distributed training for the same model. The popular Federated Averaging (FedAvg) algorithm is used as the aggregation method. We show that the federated LSTM achieves decent performance, yet it is still sub-optimal. We suggest possible next steps to bridge this performance gap. Furthermore, we explore the effects of language imbalance varying the ratio of English and Dutch training texts (or clients). We show the model upholds performance (of the balanced case) up and until a 80/20 imbalance before decaying rapidly. Lastly, we describe the implementation of local client training, word prediction and client-server communication in a custom virtual keyboard for Android platforms. Additionally, homomorphic encryption is applied to provide with secure aggregation guarding the user from malicious servers.
- Open Access
- 5 March 2021
- Published by Wiley in Journal of Sleep Research
- Vol. 30 (5) , e13285
Behavioural biometrics: Using smartphone keyboard activity as a proxy for rest–activity patterns
Summary: Rest–activity patterns are important aspects of healthy sleep and may be disturbed in conditions like circadian rhythm disorders, insomnia, insufficient sleep syndrome, and neurological disorders. Long‐term monitoring of rest–activity patterns is typically performed with diaries or actigraphy. Here, we propose an unobtrusive method to obtain rest–activity patterns using smartphone keyboard activity. The present study investigated whether this proposed method reliably estimates rest and activity timing compared to daily self‐reports within healthy participants. First‐year students (n = 51) used a custom smartphone keyboard to passively and objectively measure smartphone use behaviours and completed the Consensus Sleep Diary for 1 week. The time of the last keyboard activity before a nightly absence of keystrokes, and the time of the first keyboard activity following this period were used as markers. Results revealed high correlations between these markers and user‐reported onset and offset of resting period (r ranged from 0.74 to 0.80). Linear mixed models could estimate onset and offset of resting periods with reasonable accuracy (R2 ranged from 0.60 to 0.66). This indicates that smartphone keyboard activity can be used to estimate rest–activity patterns. In addition, effects of chronotype and type of day were investigated. Implementing this method in longitudinal studies would allow for long‐term monitoring of (disturbances to) rest–activity patterns, without user burden or additional costly devices. It could be particularly interesting to replicate these findings in studies amongst clinical populations with sleep‐related problems, or in populations for whom disturbances in rest–activity patterns are secondary complaints, such as neurological disorders.
- 5 March 2021
- Published by Wiley in Journal of Sleep Research
- Vol. 30 (5) , e13285
Open AccessBehavioural biometrics: Using smartphone keyboard activity as a proxy for rest–activity patterns
Summary: Rest–activity patterns are important aspects of healthy sleep and may be disturbed in conditions like circadian rhythm disorders, insomnia, insufficient sleep syndrome, and neurological disorders. Long‐term monitoring of rest–activity patterns is typically performed with diaries or actigraphy. Here, we propose an unobtrusive method to obtain rest–activity patterns using smartphone keyboard activity. The present study investigated whether this proposed method reliably estimates rest and activity timing compared to daily self‐reports within healthy participants. First‐year students (n = 51) used a custom smartphone keyboard to passively and objectively measure smartphone use behaviours and completed the Consensus Sleep Diary for 1 week. The time of the last keyboard activity before a nightly absence of keystrokes, and the time of the first keyboard activity following this period were used as markers. Results revealed high correlations between these markers and user‐reported onset and offset of resting period (r ranged from 0.74 to 0.80). Linear mixed models could estimate onset and offset of resting periods with reasonable accuracy (R2 ranged from 0.60 to 0.66). This indicates that smartphone keyboard activity can be used to estimate rest–activity patterns. In addition, effects of chronotype and type of day were investigated. Implementing this method in longitudinal studies would allow for long‐term monitoring of (disturbances to) rest–activity patterns, without user burden or additional costly devices. It could be particularly interesting to replicate these findings in studies amongst clinical populations with sleep‐related problems, or in populations for whom disturbances in rest–activity patterns are secondary complaints, such as neurological disorders.
- 1 November 2020
- Published by AIP Publishing in Chaos: An Interdisciplinary Journal of Nonlinear Science
- Vol. 30 (11) , 113133
Early-warning signals for disease activity in patients diagnosed with multiple sclerosis based on keystroke dynamics
Within data gathered through passive monitoring of patients with Multiple Sclerosis (MS), there is a clear necessity for improved methodological approaches to match the emergence of continuous, objective, measuring technologies. As most gold standards measure infrequently and require clinician presence, fluctuations in the daily progression are not accounted for. Due to the underlying conditions of homogeneity and stationarity (the main tenets of ergodicity) not being met for the majority of the statistical methods employed in the clinical setting, alternative approaches should be investigated. A solution is to use a non-linear time series analysis approach. Here, Early-Warning Signals (EWS) in the form of critical fluctuations in Keystroke Dynamics (KD), collected using participant’s smartphones, are investigated as indicators for a clinical change in three groups. These are patients with MS and changes in Magnetic Resonance Imaging (MRI), patients with MS but without changes in MRI, and healthy controls (HCs). Here, we report examples of EWS and changes in KD coinciding with clinically relevant changes in outcome measures in both patients with and without differences in the amount of MRI enhancing lesions. We also report no clinically relevant changes in EWS in the HC population. This study is a first promising step toward using EWS to identify periods of instability as measured by a continuous objective measure as a proxy for outcome measures in the field of MS.
- 1 November 2020
- Published by AIP Publishing in Chaos: An Interdisciplinary Journal of Nonlinear Science
- Vol. 30 (11) , 113133
Early-warning signals for disease activity in patients diagnosed with multiple sclerosis based on keystroke dynamics
Within data gathered through passive monitoring of patients with Multiple Sclerosis (MS), there is a clear necessity for improved methodological approaches to match the emergence of continuous, objective, measuring technologies. As most gold standards measure infrequently and require clinician presence, fluctuations in the daily progression are not accounted for. Due to the underlying conditions of homogeneity and stationarity (the main tenets of ergodicity) not being met for the majority of the statistical methods employed in the clinical setting, alternative approaches should be investigated. A solution is to use a non-linear time series analysis approach. Here, Early-Warning Signals (EWS) in the form of critical fluctuations in Keystroke Dynamics (KD), collected using participant’s smartphones, are investigated as indicators for a clinical change in three groups. These are patients with MS and changes in Magnetic Resonance Imaging (MRI), patients with MS but without changes in MRI, and healthy controls (HCs). Here, we report examples of EWS and changes in KD coinciding with clinically relevant changes in outcome measures in both patients with and without differences in the amount of MRI enhancing lesions. We also report no clinically relevant changes in EWS in the HC population. This study is a first promising step toward using EWS to identify periods of instability as measured by a continuous objective measure as a proxy for outcome measures in the field of MS.
- 15 January 2019
- Published by Wiley in Optimal Control Applications and Methods
- Vol. 40 (3) , 422-447
Aerodynamic model identification of an autonomous aircraft for airborne wind energy
Summary: Airborne wind energy (AWE) refers to a novel technology capable of harvesting energy from wind by flying crosswind patterns with tethered autonomous aircraft. Successful design of flight controllers for AWE systems relies on the availability of accurate mathematical models. Due to an expected nonconventional structure of the airborne component, the system identification procedure must be ultimately addressed via an intensive flight test campaign to gain additional insight about the aerodynamic properties. In this paper, the longitudinal dynamics of a rigid‐wing, high lift, autonomous aircraft for AWE are identified from experimental data obtained within flight tests. The aerodynamic characteristics are estimated via an efficient time‐domain multiple experiments model‐based parameter estimation algorithm.
- 15 January 2019
- Published by Wiley in Optimal Control Applications and Methods
- Vol. 40 (3) , 422-447
Aerodynamic model identification of an autonomous aircraft for airborne wind energy
Summary: Airborne wind energy (AWE) refers to a novel technology capable of harvesting energy from wind by flying crosswind patterns with tethered autonomous aircraft. Successful design of flight controllers for AWE systems relies on the availability of accurate mathematical models. Due to an expected nonconventional structure of the airborne component, the system identification procedure must be ultimately addressed via an intensive flight test campaign to gain additional insight about the aerodynamic properties. In this paper, the longitudinal dynamics of a rigid‐wing, high lift, autonomous aircraft for AWE are identified from experimental data obtained within flight tests. The aerodynamic characteristics are estimated via an efficient time‐domain multiple experiments model‐based parameter estimation algorithm.
- 1 August 2018
- Published by Elsevier in Control Engineering Practice
- Vol. 77, 15-27
Optimal input design for autonomous aircraft
- 1 August 2018
- Published by Elsevier in Control Engineering Practice
- Vol. 77, 15-27
Optimal input design for autonomous aircraft
- 1 December 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4018-4025
Optimum experimental design of a rigid wing AWE pumping system
Airborne Wind Energy (AWE) refers to systems capable of harvesting energy from wind by flying crosswind patterns with a tethered aircraft. Accurate models are crucial for tuning and validation of flight controllers. Due to the non-conventional structure of the airborne component, an intensive flight test campaign must be set where maneuvers are performed for parameter estimation purposes. In this paper, we optimize maneuvers for the longitudinal dynamics of a rigid wing AWE pumping system by solving a model-based experimental design problem that aims to obtain more accurate parameter estimates and reduce the flight test time. We consider a trim reference condition of the aircraft and constraints are enforced in order to prevent flight envelope violation. Finally, the optimal solution is implemented in the Flight Control Computer (FCC) of the prototype developed by Ampyx Power B.V. and validated under realistic flight conditions.
- 1 December 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4018-4025
Optimum experimental design of a rigid wing AWE pumping system
Airborne Wind Energy (AWE) refers to systems capable of harvesting energy from wind by flying crosswind patterns with a tethered aircraft. Accurate models are crucial for tuning and validation of flight controllers. Due to the non-conventional structure of the airborne component, an intensive flight test campaign must be set where maneuvers are performed for parameter estimation purposes. In this paper, we optimize maneuvers for the longitudinal dynamics of a rigid wing AWE pumping system by solving a model-based experimental design problem that aims to obtain more accurate parameter estimates and reduce the flight test time. We consider a trim reference condition of the aircraft and constraints are enforced in order to prevent flight envelope violation. Finally, the optimal solution is implemented in the Flight Control Computer (FCC) of the prototype developed by Ampyx Power B.V. and validated under realistic flight conditions.