Jose C. Cunha
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Jose C. Cunha has a total of 59 co-authors. Here are the top co-authors by number of shared publications:
- Joaquim Ferreira Silva
NOVA LINCS, NOVA School of Science and Technology
9 shared publications - Carlos Goncalves
Instituto Superior de Engenharia de Lisboa
8 shared publications - João Lourenço
Universidade Nova de Lisboa
7 shared publications - Luis Assuncao
ISEL, CITI/FCT-UNL, Lisboa, Portugal
7 shared publications Carmen Morgado
6 shared publications- Pedro D. Medeiros
Universidade Nova de Lisboa
6 shared publications
Latest Publications
- 1 October 2025
- Published by Springer Nature
- p. 402-418
A Scalable Model for Frequency Distribution of Low Occurrence Multi-words Towards Handling Very Large Spectrum of Text Corpora Sizes
- 1 October 2025
- Published by Springer Nature
- p. 402-418
A Scalable Model for Frequency Distribution of Low Occurrence Multi-words Towards Handling Very Large Spectrum of Text Corpora Sizes
- 25 April 2024
- Published by Springer Nature
- p. 210-222
How Large Corpora Sizes Influence the Distribution of Low Frequency Text n-grams
- 25 April 2024
- Published by Springer Nature
- p. 210-222
How Large Corpora Sizes Influence the Distribution of Low Frequency Text n-grams
- 26 June 2023
- Published by Springer Nature
- p. 154-162
Improving LocalMaxs Multiword Expression Statistical Extractor
- 26 June 2023
- Published by Springer Nature
- p. 154-162
Improving LocalMaxs Multiword Expression Statistical Extractor
- 9 June 2021
- Published by Springer Nature
- p. 699-706
A Model for Predicting n-gram Frequency Distribution in Large Corpora
- 9 June 2021
- Published by Springer Nature
- p. 699-706
A Model for Predicting n-gram Frequency Distribution in Large Corpora
- 6 May 2020
- Published by Springer Nature
- Vol. 12085, 840-851
An Empirical Model for n-gram Frequency Distribution in Large Corpora
Statistical multiword extraction methods can benefit from the knowledge on the n-gram (\(n \ge 1\)) frequency distribution in natural language corpora, for indexing and time/space optimization purposes. The appearance of increasingly large corpora raises new challenges on the investigation of the large scale behavior of the n-gram frequency distributions, not typically emerging on small scale corpora. We propose an empirical model, based on the assumption of finite n-gram language vocabularies, to estimate the number of distinct n-grams in large corpora, as well as the sizes of the equal-frequency n-gram groups, which occur in the lower frequencies starting from 1. The model was validated for n-grams with \(1 \le n \le 6\), by a wide range of real corpora in English and French, from 60 million up to 8 billion words. These are full non-truncated corpora data, that is, their associated frequency data include the entire range of observed n-gram frequencies, from 1 up to the maximum. The model predicts the monotonic growth of the numbers of distinct n-grams until reaching asymptotic plateaux when the corpus size grows to infinity. It also predicts the non-monotonicity of the sizes of the equal-frequency n-gram groups as a function of the corpus size.- 6 May 2020
- Published by Springer Nature
- Vol. 12085, 840-851
An Empirical Model for n-gram Frequency Distribution in Large Corpora
Statistical multiword extraction methods can benefit from the knowledge on the n-gram (\(n \ge 1\)) frequency distribution in natural language corpora, for indexing and time/space optimization purposes. The appearance of increasingly large corpora raises new challenges on the investigation of the large scale behavior of the n-gram frequency distributions, not typically emerging on small scale corpora. We propose an empirical model, based on the assumption of finite n-gram language vocabularies, to estimate the number of distinct n-grams in large corpora, as well as the sizes of the equal-frequency n-gram groups, which occur in the lower frequencies starting from 1. The model was validated for n-grams with \(1 \le n \le 6\), by a wide range of real corpora in English and French, from 60 million up to 8 billion words. These are full non-truncated corpora data, that is, their associated frequency data include the entire range of observed n-gram frequencies, from 1 up to the maximum. The model predicts the monotonic growth of the numbers of distinct n-grams until reaching asymptotic plateaux when the corpus size grows to infinity. It also predicts the non-monotonicity of the sizes of the equal-frequency n-gram groups as a function of the corpus size.- 8 June 2019
- Published by Springer Nature
- p. 75-88
n-gram Cache Performance in Statistical Extraction of Relevant Terms in Large Corpora
- 8 June 2019
- Published by Springer Nature
- p. 75-88
n-gram Cache Performance in Statistical Extraction of Relevant Terms in Large Corpora
- 1 December 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 134-141
A theoretical model for n-gram distribution in big data corpora
There is a wide diversity of applications relying on the identification of the sequences of n consecutive words (n-grams) occurring in corpora. Many studies follow an empirical approach for determining the statistical distribution of the n-grams but are usually constrained by the corpora sizes, which for practical reasons stay far away from Big Data. However, Big Data sizes imply hidden behaviors to the applications, such as extraction of relevant information from Web scale sources. In this paper we propose a theoretical approach for estimating the number of distinct n-grams in each corpus. It is based on the Zipf-Mandelbrot Law and the Poisson distribution, and it allows an efficient estimation of the number of distinct 1-grams, 2-grams,..., 6-grams, for any corpus size. The proposed model was validated for English and French corpora. We illustrate a practical application of this approach to the extraction of relevant expressions from natural language corpora, and predict its asymptotic behaviour for increasingly large sizes.
- 1 December 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 134-141
A theoretical model for n-gram distribution in big data corpora
There is a wide diversity of applications relying on the identification of the sequences of n consecutive words (n-grams) occurring in corpora. Many studies follow an empirical approach for determining the statistical distribution of the n-grams but are usually constrained by the corpora sizes, which for practical reasons stay far away from Big Data. However, Big Data sizes imply hidden behaviors to the applications, such as extraction of relevant information from Web scale sources. In this paper we propose a theoretical approach for estimating the number of distinct n-grams in each corpus. It is based on the Zipf-Mandelbrot Law and the Poisson distribution, and it allows an efficient estimation of the number of distinct 1-grams, 2-grams,..., 6-grams, for any corpus size. The proposed model was validated for English and French corpora. We illustrate a practical application of this approach to the extraction of relevant expressions from natural language corpora, and predict its asymptotic behaviour for increasingly large sizes.
- 1 October 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 120-129
An n-gram cache for large-scale parallel extraction of multiword relevant expressions with LocalMaxs
LocalMaxs extracts relevant multiword terms based on their cohesion but is computationally intensive, a critical issue for very large natural language corpora. The corpus properties concerning n-gram distribution determine the algorithm complexity and were empirically analyzed for corpora up to 982 million words. A parallel LocalMaxs implementation exhibits almost linear relative efficiency, speedup, and sizeup, when executed with up to 48 cloud virtual machines and a distributed key-value store. To reduce the remote data communication, we present a novel n-gram cache with cooperative-based warm-up, leading to reduced miss ratio and time penalty. A cache analytical model is used to estimate the performance of cohesion calculation of n-gram expressions, based on corpus empirical data. The model estimates agree with the real execution results.
- 1 October 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 120-129
An n-gram cache for large-scale parallel extraction of multiword relevant expressions with LocalMaxs
LocalMaxs extracts relevant multiword terms based on their cohesion but is computationally intensive, a critical issue for very large natural language corpora. The corpus properties concerning n-gram distribution determine the algorithm complexity and were empirically analyzed for corpora up to 982 million words. A parallel LocalMaxs implementation exhibits almost linear relative efficiency, speedup, and sizeup, when executed with up to 48 cloud virtual machines and a distributed key-value store. To reduce the remote data communication, we present a novel n-gram cache with cooperative-based warm-up, leading to reduced miss ratio and time penalty. A cache analytical model is used to estimate the performance of cohesion calculation of n-gram expressions, based on corpus empirical data. The model estimates agree with the real execution results.
- Open Access
- 3 June 2016
- Published by Wiley in Concurrency and Computation: Practice and Experience
- Vol. 29 (4) , e3860
A high‐performance computing framework for Monte Carlo ocean color simulations
Summary: This paper presents a high‐performance computing (HPC) framework for Monte Carlo (MC) simulations in the ocean color (OC) application domain. The objective is to optimize a parallel MC radiative transfer code named MOX, developed by the authors to create a virtual marine environment for investigating the quality of OC data products derived fromin situmeasurements of in‐water radiometric quantities. A consolidated set of solutions for performance modeling, prediction, and optimization is implemented to enhance the efficiency of MC OC simulations on HPC run‐time infrastructures. HPC, machine learning, and adaptive computing techniques are applied taking into account a clear separation and systematic treatment of accuracy and precision requirements for large‐scale MC OC simulations. The added value of the work is the integration of computational methods and tools for MC OC simulations in the form of an HPC‐oriented problem‐solving environment specifically tailored to investigate data acquisition and reduction methods for OC field measurements. Study results highlight the benefit of close collaboration between HPC and application domain researchers to improve the efficiency and flexibility of computer simulations in the marine optics application domain. © 2016 The Authors.Concurrency and Computation: Practice and ExperiencePublished by John Wiley & Sons, Ltd.
- 3 June 2016
- Published by Wiley in Concurrency and Computation: Practice and Experience
- Vol. 29 (4) , e3860
Open AccessA high‐performance computing framework for Monte Carlo ocean color simulations
Summary: This paper presents a high‐performance computing (HPC) framework for Monte Carlo (MC) simulations in the ocean color (OC) application domain. The objective is to optimize a parallel MC radiative transfer code named MOX, developed by the authors to create a virtual marine environment for investigating the quality of OC data products derived fromin situmeasurements of in‐water radiometric quantities. A consolidated set of solutions for performance modeling, prediction, and optimization is implemented to enhance the efficiency of MC OC simulations on HPC run‐time infrastructures. HPC, machine learning, and adaptive computing techniques are applied taking into account a clear separation and systematic treatment of accuracy and precision requirements for large‐scale MC OC simulations. The added value of the work is the integration of computational methods and tools for MC OC simulations in the form of an HPC‐oriented problem‐solving environment specifically tailored to investigate data acquisition and reduction methods for OC field measurements. Study results highlight the benefit of close collaboration between HPC and application domain researchers to improve the efficiency and flexibility of computer simulations in the marine optics application domain. © 2016 The Authors.Concurrency and Computation: Practice and ExperiencePublished by John Wiley & Sons, Ltd.
- 1 August 2015
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 219-224
A Parallel Algorithm for Statistical Multiword Term Extraction from Very Large Corpora
Multi-word Relevant Expressions (REs) can be defined as sequences of words (n grams) with strong semantic meaning, such as "ice melting" and "Ministère des Affaires Étrangères", useful in Information Retrieval, Document Clustering or Classification and Indexing of Documents. The need of extracting REs in several languages led research on statistical approaches rather than symbolic methods, since the former allow language-independence. Based on the assumption that REthe LocalMaxs algorithms have strong cohesion between their consecutive n-grams, the LocalMaxs algorithm is a language independent approach that extracts REs. Apart from its good precision, this extractor is time-consuming, being inoperable for Big Data if implemented in a sequential manner. This paper presents the first parallel and distributed version of this algorithm, achieving almost linear speedup and sizeup when processing corpora up to 1 billion words, using up to 54 virtual machines in a public cloud. This parallel version of the algorithm explores the statistical knowledge of the n grams in the corpus, to promote the locality of the references.
- 1 August 2015
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 219-224
A Parallel Algorithm for Statistical Multiword Term Extraction from Very Large Corpora
Multi-word Relevant Expressions (REs) can be defined as sequences of words (n grams) with strong semantic meaning, such as "ice melting" and "Ministère des Affaires Étrangères", useful in Information Retrieval, Document Clustering or Classification and Indexing of Documents. The need of extracting REs in several languages led research on statistical approaches rather than symbolic methods, since the former allow language-independence. Based on the assumption that REthe LocalMaxs algorithms have strong cohesion between their consecutive n-grams, the LocalMaxs algorithm is a language independent approach that extracts REs. Apart from its good precision, this extractor is time-consuming, being inoperable for Big Data if implemented in a sequential manner. This paper presents the first parallel and distributed version of this algorithm, achieving almost linear speedup and sizeup when processing corpora up to 1 billion words, using up to 54 virtual machines in a public cloud. This parallel version of the algorithm explores the statistical knowledge of the n grams in the corpus, to promote the locality of the references.