Kallirroi Georgila
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Kallirroi Georgila has a total of 23 co-authors. Here are the top co-authors by number of shared publications:
David Traum
USC Institute for Creative Technologies
3 shared publications- Johanna D. Moore
University of Edinburgh
3 shared publications Carla Gordon
USC Institute for Creative Technologies
2 shared publications- David Traum
USC Institute for Creative Technologies
2 shared publications - David DeVault
USC Institute for Creative Technologies
1 shared publication - Elnaz Nouri
University of Southern California Institute for Creative Technologies
1 shared publication
Latest Publications
- Open Access
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 610-623
Comparing Pre-Trained Embeddings and Domain-Independent Features for Regression-Based Evaluation of Task-Oriented Dialogue Systems
Kallirroi Georgila. Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue. 2024.
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 610-623
Open AccessComparing Pre-Trained Embeddings and Domain-Independent Features for Regression-Based Evaluation of Task-Oriented Dialogue Systems
Kallirroi Georgila. Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue. 2024.
- 25 October 2020
- Published by Springer Nature
- p. 145-160
Towards Personalization of Spoken Dialogue System Communication Strategies
- 25 October 2020
- Published by Springer Nature
- p. 145-160
Towards Personalization of Spoken Dialogue System Communication Strategies
- 25 September 2019
- Published by Springer Nature in Lecture Notes in Electrical Engineering
- p. 161-175
Toward Low-Cost Automated Evaluation Metrics for Internet of Things Dialogues
- 25 September 2019
- Published by Springer Nature in Lecture Notes in Electrical Engineering
- p. 161-175
Toward Low-Cost Automated Evaluation Metrics for Internet of Things Dialogues
- Open Access
- 1 January 2015
- Published by Japanese Society for Artificial Intelligence in Transactions of the Japanese Society for Artificial Intelligence
- Vol. 31 (4) , B-FC1_1-FC1_1
Reinforcement Learning of Multi-Party Trading Dialog Policies
Trading dialogs are a kind of negotiation in which an exchange of ownership of items is discussed, and these kinds of dialogs are pervasive in many situations. Recently, there has been an increasing amount of research on applying reinforcement learning (RL) to negotiation dialog domains. However, in previous research, the focus was on negotiation dialog between two participants only, ignoring cases where negotiation takes place between more than two interlocutors. In this paper, as a first study on multi-party negotiation, we apply RL to a multi-party trading scenario where the dialog system (learner) trades with one, two, or three other agents. We experiment with different RL algorithms and reward functions. We use Q-learning with linear function approximation, least-squares policy iteration, and neural fitted Q iteration. In addition, to make the learning process more efficient, we introduce an incremental reward function. The negotiation strategy of the learner is learned through simulated dialog with trader simulators. In our experiments, we evaluate how the performance of the learner varies depending on the RL algorithm used and the number of traders. Furthermore, we compare the learned dialog policies with two strong hand-crafted baseline dialog policies. Our results show that (1) even in simple multi-party trading dialog tasks, learning an effective negotiation policy is not a straightforward task and requires a lot of experimentation; and (2) the use of neural fitted Q iteration combined with an incremental reward function produces negotiation policies as effective or even better than the policies of the two strong hand-crafted baselines.
- 1 January 2015
- Published by Japanese Society for Artificial Intelligence in Transactions of the Japanese Society for Artificial Intelligence
- Vol. 31 (4) , B-FC1_1-FC1_1
Open AccessReinforcement Learning of Multi-Party Trading Dialog Policies
Trading dialogs are a kind of negotiation in which an exchange of ownership of items is discussed, and these kinds of dialogs are pervasive in many situations. Recently, there has been an increasing amount of research on applying reinforcement learning (RL) to negotiation dialog domains. However, in previous research, the focus was on negotiation dialog between two participants only, ignoring cases where negotiation takes place between more than two interlocutors. In this paper, as a first study on multi-party negotiation, we apply RL to a multi-party trading scenario where the dialog system (learner) trades with one, two, or three other agents. We experiment with different RL algorithms and reward functions. We use Q-learning with linear function approximation, least-squares policy iteration, and neural fitted Q iteration. In addition, to make the learning process more efficient, we introduce an incremental reward function. The negotiation strategy of the learner is learned through simulated dialog with trader simulators. In our experiments, we evaluate how the performance of the learner varies depending on the RL algorithm used and the number of traders. Furthermore, we compare the learned dialog policies with two strong hand-crafted baseline dialog policies. Our results show that (1) even in simple multi-party trading dialog tasks, learning an effective negotiation policy is not a straightforward task and requires a lot of experimentation; and (2) the use of neural fitted Q iteration combined with an incremental reward function produces negotiation policies as effective or even better than the policies of the two strong hand-crafted baselines.
- Open Access
- 1 January 2014
- Published by Association for Computational Linguistics (ACL)
- p. 254-256
A Demonstration of Dialogue Processing in SimSensei Kiosk
Fabrizio Morbini, David DeVault, Kallirroi Georgila, Ron Artstein, David Traum, Louis-Philippe Morency. Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL). 2014.
- 1 January 2014
- Published by Association for Computational Linguistics (ACL)
- p. 254-256
Open AccessA Demonstration of Dialogue Processing in SimSensei Kiosk
Fabrizio Morbini, David DeVault, Kallirroi Georgila, Ron Artstein, David Traum, Louis-Philippe Morency. Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL). 2014.
- Open Access
- 10 May 2010
- Published by International Speech Communication Association
- p. paper 116
Prediction and realisation of conversational characteristics by utilising spontaneous speech for unit selection
Unit selection speech synthesis has reached high levels of naturalness and intelligibility for neutral read aloud speech. However, synthetic speech generated using neutral read aloud data lacks all the attitude, intention and spontaneity associated with everyday conversations. Unit selection is heavily data dependent and thus in order to simulate human conversational speech, or create synthetic voices for believable virtual characters, we need to utilise speech data with examples of how people talk rather than how people read. In this paper we included carefully selected utterances from spontaneous conversational speech in a unit selection voice. Using this voice and by automatically predicting type and placement of lexical fillers and filled pauses we can synthesise utterances with conversational characteristics. A perceptual listening test showed that it is possible to make synthetic speech sound more conversational without degrading naturalness.
- 10 May 2010
- Published by International Speech Communication Association
- p. paper 116
Open AccessPrediction and realisation of conversational characteristics by utilising spontaneous speech for unit selection
Unit selection speech synthesis has reached high levels of naturalness and intelligibility for neutral read aloud speech. However, synthetic speech generated using neutral read aloud data lacks all the attitude, intention and spontaneity associated with everyday conversations. Unit selection is heavily data dependent and thus in order to simulate human conversational speech, or create synthetic voices for believable virtual characters, we need to utilise speech data with examples of how people talk rather than how people read. In this paper we included carefully selected utterances from spontaneous conversational speech in a unit selection voice. Using this voice and by automatically predicting type and placement of lexical fillers and filled pauses we can synthesise utterances with conversational characteristics. A perceptual listening test showed that it is possible to make synthetic speech sound more conversational without degrading naturalness.
- 15 March 2010
- Published by Springer Nature in Language Resources and Evaluation
- Vol. 44 (3) , 221-261
The MATCH corpus: a corpus of older and younger users’ interactions with spoken dialogue systems
- 15 March 2010
- Published by Springer Nature in Language Resources and Evaluation
- Vol. 44 (3) , 221-261
The MATCH corpus: a corpus of older and younger users’ interactions with spoken dialogue systems
- 1 July 2009
- Published by Cambridge University Press (CUP) in Natural Language Engineering
- Vol. 15 (3) , 315-353
Automatic annotation of context and speech acts for dialogue corpora
Richly annotated dialogue corpora are essential for new research directions in statistical learning approaches to dialogue management, context-sensitive interpretation, and context-sensitive speech recognition. In particular, large dialogue corpora annotated with contextual information and speech acts are urgently required. We explore how existing dialogue corpora (usually consisting of utterance transcriptions) can be automatically processed to yield new corpora where dialogue context and speech acts are accurately represented. We present a conceptual and computational framework for generating such corpora. As an example, we present and evaluate an automatic annotation system which builds ‘Information State Update’ (ISU) representations of dialogue context for the Communicator (2000 and 2001) corpora of human–machine dialogues (2,331 dialogues). The purposes of this annotation are to generate corpora for reinforcement learning of dialogue policies, for building user simulations, for evaluating different dialogue strategies against a baseline, and for training models for context-dependent interpretation and speech recognition. The automatic annotation system parses system and user utterances into speech acts and builds up sequences of dialogue context representations using an ISU dialogue manager. We present the architecture of the automatic annotation system and a detailed example to illustrate how the system components interact to produce the annotations. We also evaluate the annotations, with respect to the task completion metrics of the original corpus and in comparison to hand-annotated data and annotations produced by a baseline automatic system. The automatic annotations perform well and largely outperform the baseline automatic annotations in all measures. The resulting annotated corpus has been used to train high-quality user simulations and to learn successful dialogue strategies. The final corpus will be made publicly available.
- 1 July 2009
- Published by Cambridge University Press (CUP) in Natural Language Engineering
- Vol. 15 (3) , 315-353
Automatic annotation of context and speech acts for dialogue corpora
Richly annotated dialogue corpora are essential for new research directions in statistical learning approaches to dialogue management, context-sensitive interpretation, and context-sensitive speech recognition. In particular, large dialogue corpora annotated with contextual information and speech acts are urgently required. We explore how existing dialogue corpora (usually consisting of utterance transcriptions) can be automatically processed to yield new corpora where dialogue context and speech acts are accurately represented. We present a conceptual and computational framework for generating such corpora. As an example, we present and evaluate an automatic annotation system which builds ‘Information State Update’ (ISU) representations of dialogue context for the Communicator (2000 and 2001) corpora of human–machine dialogues (2,331 dialogues). The purposes of this annotation are to generate corpora for reinforcement learning of dialogue policies, for building user simulations, for evaluating different dialogue strategies against a baseline, and for training models for context-dependent interpretation and speech recognition. The automatic annotation system parses system and user utterances into speech acts and builds up sequences of dialogue context representations using an ISU dialogue manager. We present the architecture of the automatic annotation system and a detailed example to illustrate how the system components interact to produce the annotations. We also evaluate the annotations, with respect to the task completion metrics of the original corpus and in comparison to hand-annotated data and annotations produced by a baseline automatic system. The automatic annotations perform well and largely outperform the baseline automatic annotations in all measures. The resulting annotated corpus has been used to train high-quality user simulations and to learn successful dialogue strategies. The final corpus will be made publicly available.