Patrizia Paggio
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Patrizia Paggio has a total of 36 co-authors. Here are the top co-authors by number of shared publications:
- Costanza Navarretta
University of Copenhagen, Denmark
9 shared publications Bart Jongejan
University of Copenhagen
3 shared publications- Manex Agirrezabal
University of Copenhagen
2 shared publications - Adrian Muscat
Department of Communications and Computer Engineering, University of Malta, MSD 2080 Msida, Malta
1 shared publication - Albert Gatt
Institute of Linguistics and Language Technology University of Malta
1 shared publication Alexandra Vella
Institute of Linguistics and Language Technology, University of Malta, Msida, Malta
1 shared publication
Latest Publications
- Open Access
- 26 June 2025
- Published by Cambridge University Press (CUP) in Language and Cognition
- Vol. 17, e54
Do hand gestures increase perceived prominence in naturally produced utterances?
This study investigates the effect of visually perceived gestures on the overall (multimodal) prominence of naturally occurring stimuli extracted from a multimodal corpus of Maltese conversations. Experiment participants were required to rate the prominence of target words in sentences presented to them as audiovisual and audio-only stimuli. In half of the stimuli, the target word was accompanied by a co-speech hand gesture. The results of the experiment show (i) that words produced with a co-speech gesture were rated as more prominent than words that were produced without one and (ii) that this was the case independently of whether raters could see those gestures (audiovisual condition) or not (audio-only condition). An acoustic analysis of the data shows that the presence of a co-occurring gesture has a significant effect on the pitch of the target vowel. The study suggests that gestures may provide the listener with an additional but not necessary cue to perceiving prominence.
- 26 June 2025
- Published by Cambridge University Press (CUP) in Language and Cognition
- Vol. 17, e54
Open AccessDo hand gestures increase perceived prominence in naturally produced utterances?
This study investigates the effect of visually perceived gestures on the overall (multimodal) prominence of naturally occurring stimuli extracted from a multimodal corpus of Maltese conversations. Experiment participants were required to rate the prominence of target words in sentences presented to them as audiovisual and audio-only stimuli. In half of the stimuli, the target word was accompanied by a co-speech hand gesture. The results of the experiment show (i) that words produced with a co-speech gesture were rated as more prominent than words that were produced without one and (ii) that this was the case independently of whether raters could see those gestures (audiovisual condition) or not (audio-only condition). An acoustic analysis of the data shows that the presence of a co-occurring gesture has a significant effect on the pitch of the target vowel. The study suggests that gestures may provide the listener with an additional but not necessary cue to perceiving prominence.
- Open Access
- 5 February 2022
- Published by Wiley in Cognitive Science
- Vol. 46 (2) , e13099
Are Emoji Processed Like Words? An Eye‐Tracking Study
In this study, we investigate the processing of object‐denoting emoji in sentences using eye tracking. We hypothesize that (a) such emoji are more difficult to process when used as word replacement; and (b) their processing is subject to ambiguity constraints similarly to what happens with words. We conduct two experiments in which participants have to read sentences in which an emoji either follows or replaces a word. Control stimuli not containing emoji are also tested. In the second experiment, the emoji are presented in two different disambiguating contexts. First fixation duration, total visit duration, and total revisit duration are modeled in the various conditions using linear mixed models. Both our hypotheses are supported. We observe longer total visit time for non‐redundant emoji, and higher values for all three measures for ambiguous emoji. We conclude that lexical access may be more difficult for emoji especially when they are used as word replacements and their meaning is not immediately clear. Furthermore, we also conclude that non‐redundant emoji are more difficult to integrate in the processing of the sentence than the equivalent words, or emoji used in a redundant way. In turn, our results indicate that emoji may not always be as immediate and easy to process for readers in spite of their popularity.
- 5 February 2022
- Published by Wiley in Cognitive Science
- Vol. 46 (2) , e13099
Open AccessAre Emoji Processed Like Words? An Eye‐Tracking Study
In this study, we investigate the processing of object‐denoting emoji in sentences using eye tracking. We hypothesize that (a) such emoji are more difficult to process when used as word replacement; and (b) their processing is subject to ambiguity constraints similarly to what happens with words. We conduct two experiments in which participants have to read sentences in which an emoji either follows or replaces a word. Control stimuli not containing emoji are also tested. In the second experiment, the emoji are presented in two different disambiguating contexts. First fixation duration, total visit duration, and total revisit duration are modeled in the various conditions using linear mixed models. Both our hypotheses are supported. We observe longer total visit time for non‐redundant emoji, and higher values for all three measures for ambiguous emoji. We conclude that lexical access may be more difficult for emoji especially when they are used as word replacements and their meaning is not immediately clear. Furthermore, we also conclude that non‐redundant emoji are more difficult to integrate in the processing of the sentence than the equivalent words, or emoji used in a redundant way. In turn, our results indicate that emoji may not always be as immediate and easy to process for readers in spite of their popularity.
- Open Access
Face2Text: Collecting an Annotated Image Description Corpus for the Generation of Rich Face Descriptions
The past few years have witnessed renewed interest in NLP tasks at the interface between vision and language. One intensively-studied problem is that of automatically generating text from images. In this paper, we extend this problem to the more specific domain of face description. Unlike scene descriptions, face descriptions are more fine-grained and rely on attributes extracted from the image, rather than objects and relations. Given that no data exists for this task, we present an ongoing crowdsourcing study to collect a corpus of descriptions of face images taken `in the wild'. To gain a better understanding of the variation we find in face description and the possible issues that this may raise, we also conducted an annotation study on a subset of the corpus. Primarily, we found descriptions to refer to a mixture of attributes, not only physical, but also emotional and inferential, which is bound to create further challenges for current image-to-text methods.
Open AccessFace2Text: Collecting an Annotated Image Description Corpus for the Generation of Rich Face Descriptions
The past few years have witnessed renewed interest in NLP tasks at the interface between vision and language. One intensively-studied problem is that of automatically generating text from images. In this paper, we extend this problem to the more specific domain of face description. Unlike scene descriptions, face descriptions are more fine-grained and rely on attributes extracted from the image, rather than objects and relations. Given that no data exists for this task, we present an ongoing crowdsourcing study to collect a corpus of descriptions of face images taken `in the wild'. To gain a better understanding of the variation we find in face description and the possible issues that this may raise, we also conducted an annotation study on a subset of the corpus. Primarily, we found descriptions to refer to a mixture of attributes, not only physical, but also emotional and inferential, which is bound to create further challenges for current image-to-text methods.
- 27 May 2019
- Published by Det Kgl. Bibliotek/Royal Danish Library in NyS, Nydanske Sprogstudier
- No. 56,p. 77-101
Multimodal feedback i social interaktion
I denne artikel analyserer vi sproglig og nonverbal feedback i to multimodale korpora som afspejler ret forskellige kommunikationssituationer. Det ene består af otte videooptagede såkaldte ’map task’-samtaler, hvor en af samtalepartnerne forklarer den anden ruten på et kort. De to deltagere sidder i forskellige rum uden at kunne se hinanden og kommunikerer igennem høretelefoner. Det andet korpus er en samling af tolv videooptagelser af frie ansigt-til-ansigt-samtaler hvor deltagerne, som ikke kendte hinanden på forhånd, blev optaget mens de stod over for hinanden i et studie og talte sammen. Sproget er dansk i begge korpora. Vi beskriver hvordan sproglig og nonverbal feedback er opmærket og gennemgår distributionen af forskellige typer feedbackudtryk i de to korpora. Vores analyse viser interessante forskelle i både mængden og typen (mono- kontra multimodal) af feedback, som dels skyldes de forskellige fysiske opsætninger, dels interaktionens natur, som er funktionel i ’map task’-dialogerne, og som bare er samtale for samtalens skyld i de frie ansigt-til-ansigt-dialoger.
- 27 May 2019
- Published by Det Kgl. Bibliotek/Royal Danish Library in NyS, Nydanske Sprogstudier
- No. 56,p. 77-101
Multimodal feedback i social interaktion
I denne artikel analyserer vi sproglig og nonverbal feedback i to multimodale korpora som afspejler ret forskellige kommunikationssituationer. Det ene består af otte videooptagede såkaldte ’map task’-samtaler, hvor en af samtalepartnerne forklarer den anden ruten på et kort. De to deltagere sidder i forskellige rum uden at kunne se hinanden og kommunikerer igennem høretelefoner. Det andet korpus er en samling af tolv videooptagelser af frie ansigt-til-ansigt-samtaler hvor deltagerne, som ikke kendte hinanden på forhånd, blev optaget mens de stod over for hinanden i et studie og talte sammen. Sproget er dansk i begge korpora. Vi beskriver hvordan sproglig og nonverbal feedback er opmærket og gennemgår distributionen af forskellige typer feedbackudtryk i de to korpora. Vores analyse viser interessante forskelle i både mængden og typen (mono- kontra multimodal) af feedback, som dels skyldes de forskellige fysiske opsætninger, dels interaktionens natur, som er funktionel i ’map task’-dialogerne, og som bare er samtale for samtalens skyld i de frie ansigt-til-ansigt-dialoger.
- Open Access
- 17 May 2019
- Published by University of Oslo Library in Digital Humanities in the Nordic and Baltic Countries Publications
- Vol. 2 (1) , 58-72
Automatic Dating of Medieval Charters from Denmark
Dating of medieval text sources is a central task common to the field of manuscript studies. It is a difficult process requiring expert philological and historical knowledge. We investigate the issue of automatic dating of a collection of about 300 charters from medieval Denmark, in particular how n-gram models based on different transcription levels of the charters can be used to assign the manuscripts to a specific temporal interval. We frame the problem as a classification task by dividing the period into bins of 50 years and using these as classes in a supervised learning setting to develop SVM classifiers. We show that the more detailed facsimile transcription, which captures palaeographic characteristics of a text, provides better results than the diplomatic level, where such distinctions are normalised. Furthermore, both character and word n-grams show promising results, the highest accuracy reaching 74.96 %. This level of classification accuracy corresponds to being able to date almost 75 % of the charters with a 25-year error margin, which philologists use as a standard of the precision with which medieval texts can be dated manually.
- 17 May 2019
- Published by University of Oslo Library in Digital Humanities in the Nordic and Baltic Countries Publications
- Vol. 2 (1) , 58-72
Open AccessAutomatic Dating of Medieval Charters from Denmark
Dating of medieval text sources is a central task common to the field of manuscript studies. It is a difficult process requiring expert philological and historical knowledge. We investigate the issue of automatic dating of a collection of about 300 charters from medieval Denmark, in particular how n-gram models based on different transcription levels of the charters can be used to assign the manuscripts to a specific temporal interval. We frame the problem as a classification task by dividing the period into bins of 50 years and using these as classes in a supervised learning setting to develop SVM classifiers. We show that the more detailed facsimile transcription, which captures palaeographic characteristics of a text, provides better results than the diplomatic level, where such distinctions are normalised. Furthermore, both character and word n-grams show promising results, the highest accuracy reaching 74.96 %. This level of classification accuracy corresponds to being able to date almost 75 % of the charters with a 25-year error margin, which philologists use as a standard of the precision with which medieval texts can be dated manually.
- Open Access
- 1 January 2019
- Published by Association for Computational Linguistics (ACL)
Identifying Temporal Trends Based on Perplexity and Clustering: Are We Looking at Language Change?
- 1 January 2019
- Published by Association for Computational Linguistics (ACL)
Open AccessIdentifying Temporal Trends Based on Perplexity and Clustering: Are We Looking at Language Change?
- Open Access
- 16 October 2018
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Detecting head movements in video-recorded dyadic conversations
- 16 October 2018
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Open AccessDetecting head movements in video-recorded dyadic conversations
- Open Access
- 1 January 2017
- Published by Association for Computational Linguistics (ACL)
- p. 40-42
Automatic identification of head movements in video-recorded conversations: can words help?
- 1 January 2017
- Published by Association for Computational Linguistics (ACL)
- p. 40-42
Open AccessAutomatic identification of head movements in video-recorded conversations: can words help?
- Top Cited PapersOpen Access
- 1 January 2017
- Published by Association for Computational Linguistics (ACL)
- p. 118-126
Investigating Redundancy in Emoji Use: Study on a Twitter Based Corpus
- 1 January 2017
- Published by Association for Computational Linguistics (ACL)
- p. 118-126
Top Cited PapersOpen AccessInvestigating Redundancy in Emoji Use: Study on a Twitter Based Corpus
- 19 October 2016
- Published by Springer Nature in Language Resources and Evaluation
- Vol. 51 (2) , 463-494
The Danish NOMCO corpus: multimodal interaction in first acquaintance conversations
- 19 October 2016
- Published by Springer Nature in Language Resources and Evaluation
- Vol. 51 (2) , 463-494
The Danish NOMCO corpus: multimodal interaction in first acquaintance conversations