Carlo Harvey
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Carlo Harvey has a total of 30 co-authors. Here are the top co-authors by number of shared publications:
- Ian Williams
Department of Computer Science, Birmingham City University, Birmingham, United Kingdom
5 shared publications - Alan Dolhasz
Digital Media Technology Lab Birmingham City University
3 shared publications - Kurt Debattista
WMG, University of Warwick, Coventry, U.K.
3 shared publications - Alan Chalmers
WMG, University of Warwick, Coventry, UK
2 shared publications - Efstratios Doukakis
University of Warwick Warwick Manufacturing Group UK
2 shared publications - Mathew Randall
DMTLab Birmingham City University Birmingham UK
2 shared publications
Latest Publications
- Open Access
- 7 February 2025
- Published by Ubiquity Press, Ltd. in Transactions of the International Society for Music Information Retrieval
- Vol. 8 (1)
Smartwatch-Based Audio–Gestural Insights in Violin Bow Stroke Analyses
Following the exposition of quantitative, identifiable idiosyncrasy in violin performance – via neural network classification – we demonstrate that smartwatch-based synchronous audio-gesture logging facilitates interpretable practice feedback in violin performance. The novelty of our approach is twofold: we exploit convenient multimodal data capture using a consumer smartwatch, recording wrist-movement and audio data in parallel. Further, we prioritise the delivery of performance insights at their most interpretable, quantifying tonal and temporal performance trends. Using such accessible hardware to observe meaningful, approachable performance insights, the feasibility of our approach is maximised for use in real-world teaching and learning environments. Presented analyses draw upon a primary dataset compiled from nine violinists executing defined performance exercises. Recordings segmented via note onset detection are subject to subsequent analyses. Trends identified include a cross-participant tendency to ‘rush’ up-bows versus down-bows, along with lesser temporal and tonal consistency when bowing Spiccato versus Legato.
- 7 February 2025
- Published by Ubiquity Press, Ltd. in Transactions of the International Society for Music Information Retrieval
- Vol. 8 (1)
Open AccessSmartwatch-Based Audio–Gestural Insights in Violin Bow Stroke Analyses
Following the exposition of quantitative, identifiable idiosyncrasy in violin performance – via neural network classification – we demonstrate that smartwatch-based synchronous audio-gesture logging facilitates interpretable practice feedback in violin performance. The novelty of our approach is twofold: we exploit convenient multimodal data capture using a consumer smartwatch, recording wrist-movement and audio data in parallel. Further, we prioritise the delivery of performance insights at their most interpretable, quantifying tonal and temporal performance trends. Using such accessible hardware to observe meaningful, approachable performance insights, the feasibility of our approach is maximised for use in real-world teaching and learning environments. Presented analyses draw upon a primary dataset compiled from nine violinists executing defined performance exercises. Recordings segmented via note onset detection are subject to subsequent analyses. Trends identified include a cross-participant tendency to ‘rush’ up-bows versus down-bows, along with lesser temporal and tonal consistency when bowing Spiccato versus Legato.
- Open Access
- 11 April 2024
- Published by Springer Nature in Personal and Ubiquitous Computing
- Vol. 28 (3-4) , 629-641
Using a NEAT approach with curriculums for dynamic content generation in video games
This paper presents a novel exploration of the use of an evolving neural network approach to generate dynamic content for video games, specifically for a tower defence game. The objective is to employ the NeuroEvolution of Augmenting Topologies (NEAT) technique to train a NEAT neural network as a wave manager to generate enemy waves that challenge the player’s defences. The approach is extended to incorporate NEAT-generated curriculums for tower deployments to gradually increase the difficulty for the generated enemy waves, allowing the neural network to learn incrementally. The approach dynamically adapts to changes in the player’s skill level, providing a more personalised and engaging gaming experience. The quality of the machine-generated waves is evaluated through a blind A/B test with the Games Experience Questionnaire (GEQ), and results are compared with manually designed human waves. The study finds no discernible difference in the reported player experience between AI and human-designed waves. The approach can significantly reduce the time and resources required to design game content while maintaining the quality of the player experience. The approach has the potential to be applied to a range of video game genres and within the design and development process, providing a more personalised and engaging gaming experience for players.
- 11 April 2024
- Published by Springer Nature in Personal and Ubiquitous Computing
- Vol. 28 (3-4) , 629-641
Open AccessUsing a NEAT approach with curriculums for dynamic content generation in video games
This paper presents a novel exploration of the use of an evolving neural network approach to generate dynamic content for video games, specifically for a tower defence game. The objective is to employ the NeuroEvolution of Augmenting Topologies (NEAT) technique to train a NEAT neural network as a wave manager to generate enemy waves that challenge the player’s defences. The approach is extended to incorporate NEAT-generated curriculums for tower deployments to gradually increase the difficulty for the generated enemy waves, allowing the neural network to learn incrementally. The approach dynamically adapts to changes in the player’s skill level, providing a more personalised and engaging gaming experience. The quality of the machine-generated waves is evaluated through a blind A/B test with the Games Experience Questionnaire (GEQ), and results are compared with manually designed human waves. The study finds no discernible difference in the reported player experience between AI and human-designed waves. The approach can significantly reduce the time and resources required to design game content while maintaining the quality of the player experience. The approach has the potential to be applied to a range of video game genres and within the design and development process, providing a more personalised and engaging gaming experience for players.
- Open Access
- 16 May 2023
- Published by Wiley in Computer Animation and Virtual Worlds
- Vol. 34 (3-4)
Online alignment of human motion using forward plotting‐dynamic time warping
A number of approaches to online time warping have been proposed based on plotting an alignment path backwards from a selected end‐point. When continually updating a time warp to align a time series with a live data source, such as a stream of motion capture data, the use of backwards plotting makes it hard to maintain a monotonic constraint. To solve this problem, a number of time warping approaches based on forward plotting, referred to as FP‐DTW, are proposed and evaluated by applying them to human motions. We demonstrate that forward plotting for temporal alignment is a viable solution when backwards plotting is otherwise not possible.
- 16 May 2023
- Published by Wiley in Computer Animation and Virtual Worlds
- Vol. 34 (3-4)
Open AccessOnline alignment of human motion using forward plotting‐dynamic time warping
A number of approaches to online time warping have been proposed based on plotting an alignment path backwards from a selected end‐point. When continually updating a time warp to align a time series with a live data source, such as a stream of motion capture data, the use of backwards plotting makes it hard to maintain a monotonic constraint. To solve this problem, a number of time warping approaches based on forward plotting, referred to as FP‐DTW, are proposed and evaluated by applying them to human motions. We demonstrate that forward plotting for temporal alignment is a viable solution when backwards plotting is otherwise not possible.
- Open Access
- 16 May 2023
- Published by Wiley in Computer Animation and Virtual Worlds
- Vol. 34 (3-4)
Correlation as a measure for alignment and similarity of human motions
The ability to measure similarity and alignment of motions is a key tool in motion retrieval and motion editing. Similarity metrics based on distance functions are often utilized when measuring similarity of human motions, however, metrics based on correlation can also potentially useful for measuring similarity and alignment. This paper evaluates the use of correlation as a method of measuring the alignment and similarity of human motion and compares them against more established distance‐based metrics. Three correlation methods and five methods of parameterising rotation are evaluated. The results show that parameterization based on displacement vectors and Kendall Tau rank correlation are optimal for measuring the alignment between two motions. If measuring similarity of motions, however, an approach based on distance metrics for angular or positional distance should be used.
- 16 May 2023
- Published by Wiley in Computer Animation and Virtual Worlds
- Vol. 34 (3-4)
Open AccessCorrelation as a measure for alignment and similarity of human motions
The ability to measure similarity and alignment of motions is a key tool in motion retrieval and motion editing. Similarity metrics based on distance functions are often utilized when measuring similarity of human motions, however, metrics based on correlation can also potentially useful for measuring similarity and alignment. This paper evaluates the use of correlation as a method of measuring the alignment and similarity of human motion and compares them against more established distance‐based metrics. Three correlation methods and five methods of parameterising rotation are evaluated. The results show that parameterization based on displacement vectors and Kendall Tau rank correlation are optimal for measuring the alignment between two motions. If measuring similarity of motions, however, an approach based on distance metrics for angular or positional distance should be used.
- 1 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 791-792
Predicting the Light Spectrum of Virtual Reality Scenarios for Non-Image-Forming Visual Evaluation
Virtual reality (VR) headsets, while providing realistic simulated environments, are also over-stimulating the human eye, particularly for the Non-Image-Forming (NIF) visual system. Therefore, it is crucial to predict the spectrum emitted by the VR headset and to perform light stimulation evaluations during the virtual environment construction phase. We propose a framework for spectrum prediction of VR scenes only by importing a pre-acquired optical profile of the VR headset. It is successively converted into “Five Photoreceptors Radiation Efficacy” (FPRE) maps and the “Melanopic Equivalent Daylight Illuminance” (M-EDI) value to visually predict the detailed stimulation of virtual scenes to the human eye.
- 1 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 791-792
Predicting the Light Spectrum of Virtual Reality Scenarios for Non-Image-Forming Visual Evaluation
Virtual reality (VR) headsets, while providing realistic simulated environments, are also over-stimulating the human eye, particularly for the Non-Image-Forming (NIF) visual system. Therefore, it is crucial to predict the spectrum emitted by the VR headset and to perform light stimulation evaluations during the virtual environment construction phase. We propose a framework for spectrum prediction of VR scenes only by importing a pre-acquired optical profile of the VR headset. It is successively converted into “Five Photoreceptors Radiation Efficacy” (FPRE) maps and the “Melanopic Equivalent Daylight Illuminance” (M-EDI) value to visually predict the detailed stimulation of virtual scenes to the human eye.
- 4 October 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-4
Object Detection Systems Informing Users of Guardian Space Dangers Through Haptic Displays
Modern virtual reality applications rely on the guardian space to guarantee safety to the users during inter-actions. The current state of technology adopts virtual walls or meshes projected to the virtual space to alert users when their reach approaches the boundaries of the guardian space, requiring constant attention to the visual display. In this paper, we explore haptic displays as an alternative modality to alert users about dangers in the guardian space. We adopt object detection to analyse the scene within the guardian space, detecting unseen objects entering the space, i.e. a cat approaching the user's surroundings, causing a haptic display to signal the user the direction of the danger. We develop a proof-of-concept system capable of alerting a user of changes to the scene within the guardian space, demonstrating the feasibility of the system and its potential applications for extended reality platforms.
- 4 October 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-4
Object Detection Systems Informing Users of Guardian Space Dangers Through Haptic Displays
Modern virtual reality applications rely on the guardian space to guarantee safety to the users during inter-actions. The current state of technology adopts virtual walls or meshes projected to the virtual space to alert users when their reach approaches the boundaries of the guardian space, requiring constant attention to the visual display. In this paper, we explore haptic displays as an alternative modality to alert users about dangers in the guardian space. We adopt object detection to analyse the scene within the guardian space, detecting unseen objects entering the space, i.e. a cat approaching the user's surroundings, causing a haptic display to signal the user the direction of the danger. We develop a proof-of-concept system capable of alerting a user of changes to the scene within the guardian space, demonstrating the feasibility of the system and its potential applications for extended reality platforms.
- 1 January 2022
- Published by Springer Nature
- p. 394-413
Perceptually-Informed No-Reference Image Harmonisation
- 1 January 2022
- Published by Springer Nature
- p. 394-413
Perceptually-Informed No-Reference Image Harmonisation
- 17 August 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-4
Psychometric Mapping of Audio Features to Perceived Physical Characteristics of Virtual Objects
Physically-based sound synthesis can simulate virtual sound sources whose audio features reflect the physical characteristics of corresponding objects displayed in a virtual environment, allowing for real-time generation of content without relying on pre-existing audio samples. This, however, requires efficient control strategies for sound synthesis models that, depending on the nature of the sounding objects, require to be mapped to varying physical characteristics displayed through visual information. In this experiment, participants were asked to adjust a set of sound synthesis parameters based on varying physical characteristics of a virtual bouncing ball: distance, elasticity and radius. Statistical analysis of recorded subject responses shows that object radius influences evaluation of pitch and amplitude for the object's representation. Similarly, distance influences user evaluation of both reverb and amplitude whilst elasticity doesn't influence user evaluation of the feature distributions. This result is consistent across user groups evaluated: audio experts and naïve listeners. Models are produced that encode these observations using linear regression, enabling automatic parameterisation of this feature space for audio synthesis engines.
- 17 August 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-4
Psychometric Mapping of Audio Features to Perceived Physical Characteristics of Virtual Objects
Physically-based sound synthesis can simulate virtual sound sources whose audio features reflect the physical characteristics of corresponding objects displayed in a virtual environment, allowing for real-time generation of content without relying on pre-existing audio samples. This, however, requires efficient control strategies for sound synthesis models that, depending on the nature of the sounding objects, require to be mapped to varying physical characteristics displayed through visual information. In this experiment, participants were asked to adjust a set of sound synthesis parameters based on varying physical characteristics of a virtual bouncing ball: distance, elasticity and radius. Statistical analysis of recorded subject responses shows that object radius influences evaluation of pitch and amplitude for the object's representation. Similarly, distance influences user evaluation of both reverb and amplitude whilst elasticity doesn't influence user evaluation of the feature distributions. This result is consistent across user groups evaluated: audio experts and naïve listeners. Models are produced that encode these observations using linear regression, enabling automatic parameterisation of this feature space for audio synthesis engines.
- Top Cited Papers
- 1 April 2021
- Published by Elsevier in Computers & Graphics
- Vol. 95, 106-114
A robot arm digital twin utilising reinforcement learning
- 1 April 2021
- Published by Elsevier in Computers & Graphics
- Vol. 95, 106-114
Top Cited PapersA robot arm digital twin utilising reinforcement learning
- Open Access
Learning to Observe: Approximating Human Perceptual Thresholds for Detection of Suprathreshold Image Transformations
Many tasks in computer vision are often calibrated and evaluated relative to human perception. In this paper, we propose to directly approximate the perceptual function performed by human observers completing a visual detection task. Specifically, we present a novel methodology for learning to detect image transformations visible to human observers through approximating perceptual thresholds. To do this, we carry out a subjective two-alternative forced-choice study to estimate perceptual thresholds of human observers detecting local exposure shifts in images. We then leverage transformation equivariant representation learning to overcome issues of limited perceptual data. This representation is then used to train a dense convolutional classifier capable of detecting local suprathreshold exposure shifts - a distortion common to image composites. In this context, our model can approximate perceptual thresholds with an average error of 0.1148 exposure stops between empirical and predicted thresholds. It can also be trained to detect a range of different local transformations.
Open AccessLearning to Observe: Approximating Human Perceptual Thresholds for Detection of Suprathreshold Image Transformations
Many tasks in computer vision are often calibrated and evaluated relative to human perception. In this paper, we propose to directly approximate the perceptual function performed by human observers completing a visual detection task. Specifically, we present a novel methodology for learning to detect image transformations visible to human observers through approximating perceptual thresholds. To do this, we carry out a subjective two-alternative forced-choice study to estimate perceptual thresholds of human observers detecting local exposure shifts in images. We then leverage transformation equivariant representation learning to overcome issues of limited perceptual data. This representation is then used to train a dense convolutional classifier capable of detecting local suprathreshold exposure shifts - a distortion common to image composites. In this context, our model can approximate perceptual thresholds with an average error of 0.1148 exposure stops between empirical and predicted thresholds. It can also be trained to detect a range of different local transformations.