DPDiogo Pernes

Diogo Pernes

*This is an algorithmically generated author record based on publicly available information.

Diogo Pernes has a total of 14 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Multi-Target Cross-Lingual Summarization: a novel task and a language-neutral approach

    by Diogo Pernes, Gonçalo M. Correia, Afonso Mendes

    Cross-lingual summarization aims to bridge language barriers by summarizing documents in different languages. However, ensuring semantic coherence across languages is an overlooked challenge and can be critical in several contexts. To fill this gap, we introduce multi-target cross-lingual summarization as the task of summarizing a document into multiple target languages while ensuring that the produced summaries are semantically similar. We propose a principled re-ranking approach to this problem and a multi-criteria evaluation protocol to assess semantic coherence across target languages, marking a first step that will hopefully stimulate further research on this problem.

  • Open Access

    Multi-Target Cross-Lingual Summarization: a novel task and a language-neutral approach

    by Diogo Pernes, Gonçalo M. Correia, Alfonso Mendes

    Diogo Pernes, Gonçalo M. Correia, Afonso Mendes. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024.


  • Open Access

    Supervising the Centroid Baseline for Extractive Multi-Document Summarization

    by Simão Gonçalves, Gonçalo Correia, Diogo Pernes, Afonso Mendes

    The centroid method is a simple approach for extractive multi-document summarization and many improvements to its pipeline have been proposed. We further refine it by adding a beam search process to the sentence selection and also a centroid estimation attention model that leads to improved results. We demonstrate this in several multi-document summarization datasets, including in a multilingual scenario.

  • Open Access

    Towards End-to-end Speech-to-text Summarization

    by Raul Monteiro, Diogo Pernes

    Speech-to-text (S2T) summarization is a time-saving technique for filtering and keeping up with the broadcast news uploaded online on a daily basis. The rise of large language models from deep learning with impressive text generation capabilities has placed the research focus on summarization systems that produce paraphrased compact versions of the document content, also known as abstractive summaries. End-to-end (E2E) modelling of S2T abstractive summarization is a promising approach that offers the possibility of generating rich latent representations that leverage non-verbal and acoustic information, as opposed to the use of only linguistic information from automatically generated transcripts in cascade systems. However, the few literature on E2E modelling of this task fails on exploring different domains, namely broadcast news, which is challenging domain where large and diversified volumes of data are presented to the user every day. We model S2T summarization both with a cascade and an E2E system for a corpus of broadcast news in French. Our novel E2E model leverages external data by resorting to transfer learning from a pre-trained T2T summarizer. Experiments show that both our cascade and E2E abstractive summarizers are stronger than an extractive baseline. However, the performance of the E2E model still lies behind the cascade one, which is object of an extensive analysis that includes future directions to close that gap.


  • Open Access

    Supervising the Centroid Baseline for Extractive Multi-Document Summarization

    by Simão Gonçalves, Gonçalo Correia, Diogo Pernes, Alfonso Mendes

    Simão Gonçalves, Gonçalo Correia, Diogo Pernes, Afonso Mendes. Proceedings of the 4th New Frontiers in Summarization Workshop. 2023.


  • Open Access

    Improving abstractive summarization with energy-based re-ranking

    by Diogo Pernes, Afonso Mendes, André F. T. Martins

    Current abstractive summarization systems present important weaknesses which prevent their deployment in real-world applications, such as the omission of relevant information and the generation of factual inconsistencies (also known as hallucinations). At the same time, automatic evaluation metrics such as CTC scores have been recently proposed that exhibit a higher correlation with human judgments than traditional lexical-overlap metrics such as ROUGE. In this work, we intend to close the loop by leveraging the recent advances in summarization metrics to create quality-aware abstractive summarizers. Namely, we propose an energy-based model that learns to re-rank summaries according to one or a combination of these metrics. We experiment using several metrics to train our energy-based re-ranker and show that it consistently improves the scores achieved by the predicted summaries. Nonetheless, human evaluation results show that the re-ranking approach should be used with care for highly abstractive summaries, as the available metrics are not yet sufficiently reliable for this purpose.


  • Learning signer-invariant representations with adversarial training

    by Pedro M. Ferreira, Diogo Pernes, Ana Rebelo, Jaime S. Cardoso

    Sign Language Recognition (SLR) has become an appealing topic in modern societies because such technology can ideally be used to bridge the gap between deaf and hearing people. Although important steps have been made towards the development of real-world SLR systems, signer-independent SLR is still one of the bottleneck problems of this research field. In this regard, we propose a deep neural network along with an adversarial training objective, specifically designed to address the signer-independent problem. Concretely speaking, the proposed model consists of an encoder, mapping from input images to latent representations, and two classifiers operating on these underlying representations: (i) the signclassifier, for predicting the class/sign labels, and (ii) the signer-classifier, for predicting their signer identities. During the learning stage, the encoder is simultaneously trained to help the sign-classifier as much as possible while trying to fool the signer-classifier. This adversarial training procedure allows learning signer-invariant latent representations that are in fact highly discriminative for sign recognition. Experimental results demonstrate the effectiveness of the proposed model and its capability of dealing with the large inter-signer variations.


  • DeSIRe: Deep Signer-Invariant Representations for Sign Language Recognition

    by Pedro M. Ferreira, Diogo Pernes, Ana Rebelo, Jaime S. Cardoso

    As a key technology to help bridging the gap between deaf and hearing people, sign language recognition (SLR) has become one of the most active research topics in the human-computer interaction field. Although several SLR methodologies have been proposed, the development of a real-world SLR system is still a very challenging task. One of the main challenges is related to the large intersigner variability that exists in the manual signing process of sign languages. To address this problem, we propose a novel end-to-end deep neural network that explicitly models highly discriminative signer-independent latent representations from the input data. The key idea of our model is to learn a distribution over latent representations, conditionally independent of signer identity. Accordingly, the learned latent representations will preserve as much information as possible about the signs, and discard signer-specific traits that are irrelevant for recognition. By imposing such regularization in the representation space, the result is a truly signer-independent model which is robust to different and new test signers. The experimental results demonstrate the effectiveness of the proposed model in several SLR databases.