Rim Helaoui
Rim Helaoui has a total of 15 co-authors. Here are the top co-authors by number of shared publications:
- Daniele Riboni
Department of Mathematics and Computer Science, University of Cagliari, Cagliari, Italy
5 shared publications - Diego Reforgiato Recupero
University of Cagliari
4 shared publications - Vivek Kumar
Department of Psychology Research Institute CODE University of the Bundeswehr Munich Germany
3 shared publications - Zixiu Wu
University of Cagliari, Italy
2 shared publications - Aki Härmä
Maastricht University
1 shared publication - Claudio Bettini
University of Milan
1 shared publication
Latest Publications
- Open Access
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 368-373
Towards Effective Automatic Evaluation of Generated Reflections for Motivational Interviewing
Reflection is an essential counselling skill where the therapist communicates their understanding of the client’s words to the client. Recent studies have explored language-model-based reflection generation, but automatic quality evaluation of generated reflections remains under-explored. In this work, we investigate automatic evaluation on one fundamental quality aspect: coherence and context-consistency. We test a range of automatic evaluators/metrics and examine their correlations with expert judgement. We find that large language models (LLMs) as zero-shot evaluators achieve the best performance, while other metrics correlate poorly with expert judgement. We also demonstrate that diverse LLM-as-evaluator configurations need to be explored to find the best setup.
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 368-373
Open AccessTowards Effective Automatic Evaluation of Generated Reflections for Motivational Interviewing
Reflection is an essential counselling skill where the therapist communicates their understanding of the client’s words to the client. Recent studies have explored language-model-based reflection generation, but automatic quality evaluation of generated reflections remains under-explored. In this work, we investigate automatic evaluation on one fundamental quality aspect: coherence and context-consistency. We test a range of automatic evaluators/metrics and examine their correlations with expert judgement. We find that large language models (LLMs) as zero-shot evaluators achieve the best performance, while other metrics correlate poorly with expert judgement. We also demonstrate that diverse LLM-as-evaluator configurations need to be explored to find the best setup.
- 15 July 2023
- Published by Springer Nature
- p. 119-133
How Do You Feel? Information Retrieval in Psychotherapy and Fair Ranking Assessment
- 15 July 2023
- Published by Springer Nature
- p. 119-133
How Do You Feel? Information Retrieval in Psychotherapy and Fair Ranking Assessment
- Top Cited PapersOpen Access
- 14 March 2023
- Published by MDPI AG in Future Internet
- Vol. 15 (3) , 110
Creation, Analysis and Evaluation of AnnoMI, a Dataset of Expert-Annotated Counselling Dialogues
Research on the analysis of counselling conversations through natural language processing methods has seen remarkable growth in recent years. However, the potential of this field is still greatly limited by the lack of access to publicly available therapy dialogues, especially those with expert annotations, but it has been alleviated thanks to the recent release of AnnoMI, the first publicly and freely available conversation dataset of 133 faithfully transcribed and expert-annotated demonstrations of high- and low-quality motivational interviewing (MI)—an effective therapy strategy that evokes client motivation for positive change. In this work, we introduce new expert-annotated utterance attributes to AnnoMI and describe the entire data collection process in more detail, including dialogue source selection, transcription, annotation, and post-processing. Based on the expert annotations on key MI aspects, we carry out thorough analyses of AnnoMI with respect to counselling-related properties on the utterance, conversation, and corpus levels. Furthermore, we introduce utterance-level prediction tasks with potential real-world impacts and build baseline models. Finally, we examine the performance of the models on dialogues of different topics and probe the generalisability of the models to unseen topics.
- 14 March 2023
- Published by MDPI AG in Future Internet
- Vol. 15 (3) , 110
Top Cited PapersOpen AccessCreation, Analysis and Evaluation of AnnoMI, a Dataset of Expert-Annotated Counselling Dialogues
Research on the analysis of counselling conversations through natural language processing methods has seen remarkable growth in recent years. However, the potential of this field is still greatly limited by the lack of access to publicly available therapy dialogues, especially those with expert annotations, but it has been alleviated thanks to the recent release of AnnoMI, the first publicly and freely available conversation dataset of 133 faithfully transcribed and expert-annotated demonstrations of high- and low-quality motivational interviewing (MI)—an effective therapy strategy that evokes client motivation for positive change. In this work, we introduce new expert-annotated utterance attributes to AnnoMI and describe the entire data collection process in more detail, including dialogue source selection, transcription, annotation, and post-processing. Based on the expert annotations on key MI aspects, we carry out thorough analyses of AnnoMI with respect to counselling-related properties on the utterance, conversation, and corpus levels. Furthermore, we introduce utterance-level prediction tasks with potential real-world impacts and build baseline models. Finally, we examine the performance of the models on dialogues of different topics and probe the generalisability of the models to unseen topics.
- Open Access
TF-IDF vs Word Embeddings for Morbidity Identification in Clinical Notes: An Initial Study
Today, we are seeing an ever-increasing number of clinical notes that contain clinical results, images, and textual descriptions of patient's health state. All these data can be analyzed and employed to cater novel services that can help people and domain experts with their common healthcare tasks. However, many technologies such as Deep Learning and tools like Word Embeddings have started to be investigated only recently, and many challenges remain open when it comes to healthcare domain applications. To address these challenges, we propose the use of Deep Learning and Word Embeddings for identifying sixteen morbidity types within textual descriptions of clinical records. For this purpose, we have used a Deep Learning model based on Bidirectional Long-Short Term Memory (LSTM) layers which can exploit state-of-the-art vector representations of data such as Word Embeddings. We have employed pre-trained Word Embeddings namely GloVe and Word2Vec, and our own Word Embeddings trained on the target domain. Furthermore, we have compared the performances of the deep learning approaches against the traditional tf-idf using Support Vector Machine and Multilayer perceptron (our baselines). From the obtained results it seems that the latter outperforms the combination of Deep Learning approaches using any word embeddings. Our preliminary results indicate that there are specific features that make the dataset biased in favour of traditional machine learning approaches.
Open AccessTF-IDF vs Word Embeddings for Morbidity Identification in Clinical Notes: An Initial Study
Today, we are seeing an ever-increasing number of clinical notes that contain clinical results, images, and textual descriptions of patient's health state. All these data can be analyzed and employed to cater novel services that can help people and domain experts with their common healthcare tasks. However, many technologies such as Deep Learning and tools like Word Embeddings have started to be investigated only recently, and many challenges remain open when it comes to healthcare domain applications. To address these challenges, we propose the use of Deep Learning and Word Embeddings for identifying sixteen morbidity types within textual descriptions of clinical records. For this purpose, we have used a Deep Learning model based on Bidirectional Long-Short Term Memory (LSTM) layers which can exploit state-of-the-art vector representations of data such as Word Embeddings. We have employed pre-trained Word Embeddings namely GloVe and Word2Vec, and our own Word Embeddings trained on the target domain. Furthermore, we have compared the performances of the deep learning approaches against the traditional tf-idf using Support Vector Machine and Multilayer perceptron (our baselines). From the obtained results it seems that the latter outperforms the combination of Deep Learning approaches using any word embeddings. Our preliminary results indicate that there are specific features that make the dataset biased in favour of traditional machine learning approaches.
- 21 February 2019
- Published by Springer Nature
- p. 59-75
Analysis of Topic Propagation in Therapy Sessions Using Partially Labeled Latent Dirichlet Allocation
- 21 February 2019
- Published by Springer Nature
- p. 59-75
Analysis of Topic Propagation in Therapy Sessions Using Partially Labeled Latent Dirichlet Allocation
- 1 February 2016
- Published by Elsevier in Artificial Intelligence in Medicine
- Vol. 67, 57-74
SmartFABER: Recognizing fine-grained abnormal behaviors for early detection of mild cognitive impairment
- 1 February 2016
- Published by Elsevier in Artificial Intelligence in Medicine
- Vol. 67, 57-74
SmartFABER: Recognizing fine-grained abnormal behaviors for early detection of mild cognitive impairment