Ettore Merlo
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Ettore Merlo has a total of 48 co-authors. Here are the top co-authors by number of shared publications:
- Heng Li
Hong Kong Polytechnic Univ
8 shared publications - Foutse Khomh
Polytechnique Montreal
5 shared publications - Giuliano Antoniol
Polytechnique Montréal, Montréal, Québec, Canada
3 shared publications - Qiaolin Qin
Polytechnique Montréal Montréal Canada
3 shared publications - Qiaolin Qin
Polytechnique Montréal, Canada
3 shared publications - Amin Nikanjam
Polytechnique Montreal, Montreal, Canada
2 shared publications
Latest Publications
- 20 June 2026
- Published by Springer Nature in Empirical Software Engineering
- Vol. 31 (6) , 168
An empirical study on logging evolution on stack overflow: trends, topics, and challenges
- 20 June 2026
- Published by Springer Nature in Empirical Software Engineering
- Vol. 31 (6) , 168
An empirical study on logging evolution on stack overflow: trends, topics, and challenges
- Open Access
- 13 January 2026
- Published by Elsevier in Information and Software Technology
- Vol. 192
Unsupervised, robust, and lightweight detection of data pattern anomalies and outliers
- 13 January 2026
- Published by Elsevier in Information and Software Technology
- Vol. 192
Open AccessUnsupervised, robust, and lightweight detection of data pattern anomalies and outliers
- Open Access
Empirical Characterization of Logging Smells in Machine Learning Code
\underline{Context:} Logging is a fundamental yet complex practice in software engineering, essential for monitoring, debugging, and auditing software systems. With the increasing integration of machine learning (ML) components into software systems, effective logging has become critical to ensure reproducibility, traceability, and observability throughout model training and deployment. Although various general-purpose and ML-specific logging frameworks exist, little is known about how these tools are actually used in practice or whether ML practitioners adopt consistent and effective logging strategies. To date, no empirical study has systematically characterized recurring bad logging practices--or logging smells--in ML System. \underline{Goal:} This study aims to empirically identify and characterize logging smells in ML systems, providing an evidence-based understanding of how logging is implemented and challenged in practice. \underline{Method:} We propose to conduct a large-scale mining of open-source ML repositories hosted on GitHub to catalogue recurring logging smells. Subsequently, a practitioner survey involving ML engineers will be conducted to assess the perceived relevance, severity, and frequency of the identified smells. \underline{Limitations:} % While The study's limitations include that While our findings may not be generalizable to closed-source industrial projects, we believe our study provides an essential step toward understanding and improving logging practices in ML development.
Open AccessEmpirical Characterization of Logging Smells in Machine Learning Code
\underline{Context:} Logging is a fundamental yet complex practice in software engineering, essential for monitoring, debugging, and auditing software systems. With the increasing integration of machine learning (ML) components into software systems, effective logging has become critical to ensure reproducibility, traceability, and observability throughout model training and deployment. Although various general-purpose and ML-specific logging frameworks exist, little is known about how these tools are actually used in practice or whether ML practitioners adopt consistent and effective logging strategies. To date, no empirical study has systematically characterized recurring bad logging practices--or logging smells--in ML System. \underline{Goal:} This study aims to empirically identify and characterize logging smells in ML systems, providing an evidence-based understanding of how logging is implemented and challenged in practice. \underline{Method:} We propose to conduct a large-scale mining of open-source ML repositories hosted on GitHub to catalogue recurring logging smells. Subsequently, a practitioner survey involving ML engineers will be conducted to assess the perceived relevance, severity, and frequency of the identified smells. \underline{Limitations:} % While The study's limitations include that While our findings may not be generalizable to closed-source industrial projects, we believe our study provides an essential step toward understanding and improving logging practices in ML development.
- Open Access
A Story About Cohesion and Separation: Label-Free Metric for Log Parser Evaluation
Log parsing converts log messages into structured event templates, allowing for automated log analysis and reducing manual inspection effort. To select the most compatible parser for a specific system, multiple evaluation metrics are commonly used for performance comparisons. However, existing evaluation metrics heavily rely on labeled log data, which limits prior studies to a fixed set of datasets and hinders parser evaluations and selections in the industry. Further, we discovered that different versions of ground-truth used in existing studies can lead to inconsistent performance conclusions. Motivated by these challenges, we propose a novel label-free template-level metric, PMSS (parser medoid silhouette score), to evaluate log parser performance. PMSS evaluates both parser grouping and template quality with medoid silhouette analysis and Levenshtein distance within a near-linear time complexity in general. To understand its relationship with label-based template-level metrics, FGA and FTA, we compared their evaluation outcomes for six log parsers on the standard corrected Loghub 2.0 dataset. Our results indicate that log parsers achieving the highest PMSS or FGA exhibit comparable performance, differing by only 2.1% on average in terms of the FGA score; the difference is 9.8% for FTA. PMSS is also significantly (p<1e-8) and positively correlated to both FGA and FTA: the Spearman's rho correlation coefficient of PMSS-FGA and PMSS-FTA are respectively 0.648 and 0.587, close to the coefficient between FGA and FTA (0.670). We further extended our discussion on how to interpret the conclusions from different metrics, identifying challenges in using PMSS, and provided guidelines on conducting parser selections with our metric. PMSS provides a valuable evaluation alternative when ground-truths are inconsistent or labels are unavailable.
Open AccessA Story About Cohesion and Separation: Label-Free Metric for Log Parser Evaluation
Log parsing converts log messages into structured event templates, allowing for automated log analysis and reducing manual inspection effort. To select the most compatible parser for a specific system, multiple evaluation metrics are commonly used for performance comparisons. However, existing evaluation metrics heavily rely on labeled log data, which limits prior studies to a fixed set of datasets and hinders parser evaluations and selections in the industry. Further, we discovered that different versions of ground-truth used in existing studies can lead to inconsistent performance conclusions. Motivated by these challenges, we propose a novel label-free template-level metric, PMSS (parser medoid silhouette score), to evaluate log parser performance. PMSS evaluates both parser grouping and template quality with medoid silhouette analysis and Levenshtein distance within a near-linear time complexity in general. To understand its relationship with label-based template-level metrics, FGA and FTA, we compared their evaluation outcomes for six log parsers on the standard corrected Loghub 2.0 dataset. Our results indicate that log parsers achieving the highest PMSS or FGA exhibit comparable performance, differing by only 2.1% on average in terms of the FGA score; the difference is 9.8% for FTA. PMSS is also significantly (p<1e-8) and positively correlated to both FGA and FTA: the Spearman's rho correlation coefficient of PMSS-FGA and PMSS-FTA are respectively 0.648 and 0.587, close to the coefficient between FGA and FTA (0.670). We further extended our discussion on how to interpret the conclusions from different metrics, identifying challenges in using PMSS, and provided guidelines on conducting parser selections with our metric. PMSS provides a valuable evaluation alternative when ground-truths are inconsistent or labels are unavailable.
- Open Access
- 9 October 2025
- Published by Elsevier in Information and Software Technology
- Vol. 189
Prune bias from the root: Bias removal and fairness estimation by pruning sensitive attributes in pre-trained DNN models
- 9 October 2025
- Published by Elsevier in Information and Software Technology
- Vol. 189
Open AccessPrune bias from the root: Bias removal and fairness estimation by pruning sensitive attributes in pre-trained DNN models
- Open Access
- 30 August 2025
- Published by Elsevier in Information and Software Technology
- Vol. 188
Representation-based fairness evaluation and bias correction robustness assessment in neural networks
- 30 August 2025
- Published by Elsevier in Information and Software Technology
- Vol. 188
Open AccessRepresentation-based fairness evaluation and bias correction robustness assessment in neural networks
- Open Access
Logging Requirement for Continuous Auditing of Responsible Machine Learning-based Applications
Machine learning (ML) is increasingly applied across industries to automate decision-making, but concerns about ethical and legal compliance remain due to limited transparency, fairness, and accountability. Monitoring through logging a long-standing practice in traditional software offers a potential means for auditing ML applications, as logs provide traceable records of system behavior useful for debugging, performance analysis, and continuous auditing. systematically auditing models for compliance or accountability. The findings underscore the need for enhanced logging practices and tooling that systematically integrate responsible AI metrics. Such practices would support the development of auditable, transparent, and ethically responsible ML systems, aligning with growing regulatory requirements and societal expectations. By highlighting specific deficiencies and opportunities, this work provides actionable guidance for both practitioners and tool developers seeking to strengthen the accountability and trustworthiness of ML applications.
Open AccessLogging Requirement for Continuous Auditing of Responsible Machine Learning-based Applications
Machine learning (ML) is increasingly applied across industries to automate decision-making, but concerns about ethical and legal compliance remain due to limited transparency, fairness, and accountability. Monitoring through logging a long-standing practice in traditional software offers a potential means for auditing ML applications, as logs provide traceable records of system behavior useful for debugging, performance analysis, and continuous auditing. systematically auditing models for compliance or accountability. The findings underscore the need for enhanced logging practices and tooling that systematically integrate responsible AI metrics. Such practices would support the development of auditable, transparent, and ethically responsible ML systems, aligning with growing regulatory requirements and societal expectations. By highlighting specific deficiencies and opportunities, this work provides actionable guidance for both practitioners and tool developers seeking to strengthen the accountability and trustworthiness of ML applications.
- Open Access
Plug it and Play on Logs: A Configuration-Free Statistic-Based Log Parser
Log parsing is an essential task in log analysis, and many tools have been designed to accomplish it. Existing log parsers can be categorized into statistic-based and semantic-based approaches. In comparison to semantic-based parsers, existing statistic-based parsers tend to be more efficient, require lower computational costs, and be more privacy-preserving thanks to on-premise deployment, but often fall short in their accuracy (e.g., grouping or parsing accuracy) and generalizability. Therefore, it became a common belief that statistic-based parsers cannot be as effective as semantic-based parsers since the latter could take advantage of external knowledge supported by pretrained language models. Our work, however, challenges this belief with a novel statistic-based parser, PIPLUP. PIPLUP eliminates the pre-assumption of the position of constant tokens for log grouping and relies on data-insensitive parameters to overcome the generalizability challenge, allowing "plug and play" on given log files. According to our experiments on an open-sourced large log dataset, PIPLUP shows promising accuracy and generalizability with the data-insensitive default parameter set. PIPLUP not only outperforms the state-of-the-art statistic-based log parsers, Drain and its variants, but also obtains a competitive performance compared to the best unsupervised semantic-based log parser (i.e., LUNAR). Further, PIPLUP exhibits low time consumption without GPU acceleration and external API usage; our simple, efficient, and effective approach makes it more practical in real-world adoptions, especially when costs and privacy are of major concerns.
Open AccessPlug it and Play on Logs: A Configuration-Free Statistic-Based Log Parser
Log parsing is an essential task in log analysis, and many tools have been designed to accomplish it. Existing log parsers can be categorized into statistic-based and semantic-based approaches. In comparison to semantic-based parsers, existing statistic-based parsers tend to be more efficient, require lower computational costs, and be more privacy-preserving thanks to on-premise deployment, but often fall short in their accuracy (e.g., grouping or parsing accuracy) and generalizability. Therefore, it became a common belief that statistic-based parsers cannot be as effective as semantic-based parsers since the latter could take advantage of external knowledge supported by pretrained language models. Our work, however, challenges this belief with a novel statistic-based parser, PIPLUP. PIPLUP eliminates the pre-assumption of the position of constant tokens for log grouping and relies on data-insensitive parameters to overcome the generalizability challenge, allowing "plug and play" on given log files. According to our experiments on an open-sourced large log dataset, PIPLUP shows promising accuracy and generalizability with the data-insensitive default parameter set. PIPLUP not only outperforms the state-of-the-art statistic-based log parsers, Drain and its variants, but also obtains a competitive performance compared to the best unsupervised semantic-based log parser (i.e., LUNAR). Further, PIPLUP exhibits low time consumption without GPU acceleration and external API usage; our simple, efficient, and effective approach makes it more practical in real-world adoptions, especially when costs and privacy are of major concerns.
- 14 April 2025
- Published by Springer Nature in Empirical Software Engineering
- Vol. 30 (4) , 1-37
Logging requirement for continuous auditing of responsible machine learning-based applications
- 14 April 2025
- Published by Springer Nature in Empirical Software Engineering
- Vol. 30 (4) , 1-37
Logging requirement for continuous auditing of responsible machine learning-based applications
- 27 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 51 (3) , 825-832
Recovering Traceability Links Between Code and Documentation: A Retrospective
Software system documentation is almost always expressed informally in natural language and free text. Examples include requirement specifications, design documents, manual pages, system development journals, error logs, and related maintenance reports. In our 2002 seminal paper we proposed a method based on information retrieval to recover traceability links between source code and free text documents. A premise of our work was that programmers use meaningful names for program items, such as functions, variables, types, classes, and methods. The paper paved the way to the adoption of IR in software engineering opening a new perspective. Reflecting on the past twenty years we briefly overview the many results that have been achieved, however, the emergence of new technologies, such as AI, pose unprecedented challenges.
- 27 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 51 (3) , 825-832
Recovering Traceability Links Between Code and Documentation: A Retrospective
Software system documentation is almost always expressed informally in natural language and free text. Examples include requirement specifications, design documents, manual pages, system development journals, error logs, and related maintenance reports. In our 2002 seminal paper we proposed a method based on information retrieval to recover traceability links between source code and free text documents. A premise of our work was that programmers use meaningful names for program items, such as functions, variables, types, classes, and methods. The paper paved the way to the adoption of IR in software engineering opening a new perspective. Reflecting on the past twenty years we briefly overview the many results that have been achieved, however, the emergence of new technologies, such as AI, pose unprecedented challenges.