Ioanna Miliaresi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Ioanna Miliaresi has a total of 2 co-authors. Here are the top co-authors by number of shared publications:
Aggelos Pikrakis
Department of Informatics, University of Piraeus
1 shared publication- Aggelos Pikrakis
Department of Informatics, University of Piraeus, Piraeus, Greece
1 shared publication
Latest Publications
- Open Access
Embodied Musical Gesture Analysis from Audio Representations: A Review of Datasets, Audio Descriptors, and Deep Learning Approaches
Machine learning approaches have enabled increasingly powerful analysis of musical audio, allowing computational models to capture complex acoustic patterns related to musical structure, performance, and expression. However, whether learned audio representations preserve information related to the bodily actions underlying musical sound production remains an open research question. This review examines how machine-learned audio representations encode information associated with embodied musical processes by connecting perspectives from embodied music cognition, computational music analysis, and multimodal machine learning. It establishes a conceptual framework that organizes embodied musical information according to gesture type, acoustic observability, and the relationship between physical action and sound production, distinguishing sound-producing gestures, expressive movements, individual embodiment, and collective performance processes. The review then surveys datasets for audio–gesture research, including motion-capture recordings, instrument-performance datasets, audiovisual resources, and large-scale audio-only datasets, highlighting challenges related to embodiment fidelity, scalability, annotation, and multimodal alignment. It further examines conventional audio descriptors, deep neural and self-supervised audio representations, cross-modal learning approaches, and evaluation methodologies for assessing embodied information within learned representations. Particular attention is given to gesture–sound ambiguity, limited benchmarking resources, generalization across performers and instruments, and the distinction between gesture-related encoding and acoustic correlation. Current evidence suggests that learned audio representations contain partial and task-dependent traces of embodied musical information, particularly for gestures with strong acoustic coupling, while evidence for higher-level expressive and communicative movements remains limited. Determining whether these representations capture meaningful relationships between physical performance processes and musical sound remains an open challenge requiring multimodal, interpretable, and perceptually grounded evaluation frameworks.
Open AccessEmbodied Musical Gesture Analysis from Audio Representations: A Review of Datasets, Audio Descriptors, and Deep Learning Approaches
Machine learning approaches have enabled increasingly powerful analysis of musical audio, allowing computational models to capture complex acoustic patterns related to musical structure, performance, and expression. However, whether learned audio representations preserve information related to the bodily actions underlying musical sound production remains an open research question. This review examines how machine-learned audio representations encode information associated with embodied musical processes by connecting perspectives from embodied music cognition, computational music analysis, and multimodal machine learning. It establishes a conceptual framework that organizes embodied musical information according to gesture type, acoustic observability, and the relationship between physical action and sound production, distinguishing sound-producing gestures, expressive movements, individual embodiment, and collective performance processes. The review then surveys datasets for audio–gesture research, including motion-capture recordings, instrument-performance datasets, audiovisual resources, and large-scale audio-only datasets, highlighting challenges related to embodiment fidelity, scalability, annotation, and multimodal alignment. It further examines conventional audio descriptors, deep neural and self-supervised audio representations, cross-modal learning approaches, and evaluation methodologies for assessing embodied information within learned representations. Particular attention is given to gesture–sound ambiguity, limited benchmarking resources, generalization across performers and instruments, and the distinction between gesture-related encoding and acoustic correlation. Current evidence suggests that learned audio representations contain partial and task-dependent traces of embodied musical information, particularly for gestures with strong acoustic coupling, while evidence for higher-level expressive and communicative movements remains limited. Determining whether these representations capture meaningful relationships between physical performance processes and musical sound remains an open challenge requiring multimodal, interpretable, and perceptually grounded evaluation frameworks.
- Open Access
- 28 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Attention-guided acoustic modality selection for COVID-19 detection
Recent advancements in deep learning methods have successfully detected COVID-19 from audio signals, primarily focusing on breathing, coughing, isolated phonemes, speech, or combinations of these sound modalities. However, existing methods are limited by the input audio type (modality) and, consequently, cannot operate on diverse datasets. In this paper, we present an end-to-end deep learning architecture capable of selectively analyzing multiple sources of audio information, even when certain modalities are absent from the dataset. Our method can process up to nine audio modalities, including coughing, breathing, phonemes, and counting, by employing parallel convolutional branches and introducing an attention-like modality selection mechanism. The proposed approach can also be viewed as a feature selector that determines, at runtime, the pertinent acoustic modalities for each classification decision. Our findings show that this type of attentionguided mechanism increases classification accuracy compared to standard multimodal approaches and enables the reuse of the trained network across diverse datasets. Specifically, when trained on the Coswara dataset, the proposed method achieves a 97.75% testing accuracy and, without retraining, attains 82% accuracy on the Virufy dataset, despite its different, unimodal structure.
- 28 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessAttention-guided acoustic modality selection for COVID-19 detection
Recent advancements in deep learning methods have successfully detected COVID-19 from audio signals, primarily focusing on breathing, coughing, isolated phonemes, speech, or combinations of these sound modalities. However, existing methods are limited by the input audio type (modality) and, consequently, cannot operate on diverse datasets. In this paper, we present an end-to-end deep learning architecture capable of selectively analyzing multiple sources of audio information, even when certain modalities are absent from the dataset. Our method can process up to nine audio modalities, including coughing, breathing, phonemes, and counting, by employing parallel convolutional branches and introducing an attention-like modality selection mechanism. The proposed approach can also be viewed as a feature selector that determines, at runtime, the pertinent acoustic modalities for each classification decision. Our findings show that this type of attentionguided mechanism increases classification accuracy compared to standard multimodal approaches and enables the reuse of the trained network across diverse datasets. Specifically, when trained on the Coswara dataset, the proposed method achieves a 97.75% testing accuracy and, without retraining, attains 82% accuracy on the Virufy dataset, despite its different, unimodal structure.
- Open Access
- 1 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 80465-80478
A Modular Deep Learning Architecture for Voice Pathology Classification
The development of methods that combine different sources of information for the purposes of medical diagnosis is an essential challenge in the field of medical informatics. In this context, we introduce a machine-learning framework for automatic voice pathology classification and, in particular, a modular deep learning architecture that classifies voice signals stemming from four types of voice disorders. To this end, we design a multimodal deep learning architecture that fuses medical metadata with voice signals. Our classifier is a combination of fully convolutional and feed-forward sub-networks that simultaneously process low-level and mid-level features which are extracted from acoustic signals of varying duration and medical records, respectively. A key objective of our study is to develop an architecture that is capable of processing voice samples of varying duration, in order to enhance the system’s learning and inference capabilities. Our research also focuses on overcoming performance limitations of neural networks that stem from the lack of extensive volumes of training data. We therefore investigate problem specific augmentation techniques based on the feature sequence segmentation and coloured noise injection and we show that the proposed method gives state-of-the-art results, achieving 64.4% classification accuracy, compared to the 63.5% classification score of the best performing method of the 2019 FEMH data challenge.
- 1 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 80465-80478
Open AccessA Modular Deep Learning Architecture for Voice Pathology Classification
The development of methods that combine different sources of information for the purposes of medical diagnosis is an essential challenge in the field of medical informatics. In this context, we introduce a machine-learning framework for automatic voice pathology classification and, in particular, a modular deep learning architecture that classifies voice signals stemming from four types of voice disorders. To this end, we design a multimodal deep learning architecture that fuses medical metadata with voice signals. Our classifier is a combination of fully convolutional and feed-forward sub-networks that simultaneously process low-level and mid-level features which are extracted from acoustic signals of varying duration and medical records, respectively. A key objective of our study is to develop an architecture that is capable of processing voice samples of varying duration, in order to enhance the system’s learning and inference capabilities. Our research also focuses on overcoming performance limitations of neural networks that stem from the lack of extensive volumes of training data. We therefore investigate problem specific augmentation techniques based on the feature sequence segmentation and coloured noise injection and we show that the proposed method gives state-of-the-art results, achieving 64.4% classification accuracy, compared to the 63.5% classification score of the best performing method of the 2019 FEMH data challenge.