EMEttore Merlo

Ettore Merlo

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Ettore Merlo has a total of 48 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • An empirical study on logging evolution on stack overflow: trends, topics, and challenges

    by Patrick Loic Foalem, Andre Nguimbous, Foutse Khomh, Heng Li, Ettore Merlo

  • Open Access

    Empirical Characterization of Logging Smells in Machine Learning Code

    by Patrick Loic Foalem, Leuson Da Silva, Foutse Khomh, Ettore Merlo, Heng Li

    \underline{Context:} Logging is a fundamental yet complex practice in software engineering, essential for monitoring, debugging, and auditing software systems. With the increasing integration of machine learning (ML) components into software systems, effective logging has become critical to ensure reproducibility, traceability, and observability throughout model training and deployment. Although various general-purpose and ML-specific logging frameworks exist, little is known about how these tools are actually used in practice or whether ML practitioners adopt consistent and effective logging strategies. To date, no empirical study has systematically characterized recurring bad logging practices--or logging smells--in ML System. \underline{Goal:} This study aims to empirically identify and characterize logging smells in ML systems, providing an evidence-based understanding of how logging is implemented and challenged in practice. \underline{Method:} We propose to conduct a large-scale mining of open-source ML repositories hosted on GitHub to catalogue recurring logging smells. Subsequently, a practitioner survey involving ML engineers will be conducted to assess the perceived relevance, severity, and frequency of the identified smells. \underline{Limitations:} % While The study's limitations include that While our findings may not be generalizable to closed-source industrial projects, we believe our study provides an essential step toward understanding and improving logging practices in ML development.

  • Open Access

    A Story About Cohesion and Separation: Label-Free Metric for Log Parser Evaluation

    by Qiaolin Qin, Jianchen Zhao, Heng Li, Weiyi Shang, Ettore Merlo

    Log parsing converts log messages into structured event templates, allowing for automated log analysis and reducing manual inspection effort. To select the most compatible parser for a specific system, multiple evaluation metrics are commonly used for performance comparisons. However, existing evaluation metrics heavily rely on labeled log data, which limits prior studies to a fixed set of datasets and hinders parser evaluations and selections in the industry. Further, we discovered that different versions of ground-truth used in existing studies can lead to inconsistent performance conclusions. Motivated by these challenges, we propose a novel label-free template-level metric, PMSS (parser medoid silhouette score), to evaluate log parser performance. PMSS evaluates both parser grouping and template quality with medoid silhouette analysis and Levenshtein distance within a near-linear time complexity in general. To understand its relationship with label-based template-level metrics, FGA and FTA, we compared their evaluation outcomes for six log parsers on the standard corrected Loghub 2.0 dataset. Our results indicate that log parsers achieving the highest PMSS or FGA exhibit comparable performance, differing by only 2.1% on average in terms of the FGA score; the difference is 9.8% for FTA. PMSS is also significantly (p<1e-8) and positively correlated to both FGA and FTA: the Spearman's rho correlation coefficient of PMSS-FGA and PMSS-FTA are respectively 0.648 and 0.587, close to the coefficient between FGA and FTA (0.670). We further extended our discussion on how to interpret the conclusions from different metrics, identifying challenges in using PMSS, and provided guidelines on conducting parser selections with our metric. PMSS provides a valuable evaluation alternative when ground-truths are inconsistent or labels are unavailable.

  • Open Access

    Logging Requirement for Continuous Auditing of Responsible Machine Learning-based Applications

    by Patrick Loic Foalem, Leuson Da Silva, Foutse Khomh, Heng Li, Ettore Merlo

    Machine learning (ML) is increasingly applied across industries to automate decision-making, but concerns about ethical and legal compliance remain due to limited transparency, fairness, and accountability. Monitoring through logging a long-standing practice in traditional software offers a potential means for auditing ML applications, as logs provide traceable records of system behavior useful for debugging, performance analysis, and continuous auditing. systematically auditing models for compliance or accountability. The findings underscore the need for enhanced logging practices and tooling that systematically integrate responsible AI metrics. Such practices would support the development of auditable, transparent, and ethically responsible ML systems, aligning with growing regulatory requirements and societal expectations. By highlighting specific deficiencies and opportunities, this work provides actionable guidance for both practitioners and tool developers seeking to strengthen the accountability and trustworthiness of ML applications.

  • Open Access

    Plug it and Play on Logs: A Configuration-Free Statistic-Based Log Parser

    by Qiaolin Qin, Xingfang Wu, Heng Li, Ettore Merlo

    Log parsing is an essential task in log analysis, and many tools have been designed to accomplish it. Existing log parsers can be categorized into statistic-based and semantic-based approaches. In comparison to semantic-based parsers, existing statistic-based parsers tend to be more efficient, require lower computational costs, and be more privacy-preserving thanks to on-premise deployment, but often fall short in their accuracy (e.g., grouping or parsing accuracy) and generalizability. Therefore, it became a common belief that statistic-based parsers cannot be as effective as semantic-based parsers since the latter could take advantage of external knowledge supported by pretrained language models. Our work, however, challenges this belief with a novel statistic-based parser, PIPLUP. PIPLUP eliminates the pre-assumption of the position of constant tokens for log grouping and relies on data-insensitive parameters to overcome the generalizability challenge, allowing "plug and play" on given log files. According to our experiments on an open-sourced large log dataset, PIPLUP shows promising accuracy and generalizability with the data-insensitive default parameter set. PIPLUP not only outperforms the state-of-the-art statistic-based log parsers, Drain and its variants, but also obtains a competitive performance compared to the best unsupervised semantic-based log parser (i.e., LUNAR). Further, PIPLUP exhibits low time consumption without GPU acceleration and external API usage; our simple, efficient, and effective approach makes it more practical in real-world adoptions, especially when costs and privacy are of major concerns.

  • Logging requirement for continuous auditing of responsible machine learning-based applications

    by Patrick Loic Foalem, Leuson Da Silva, Foutse Khomh, Heng Li, Ettore Merlo


  • Recovering Traceability Links Between Code and Documentation: A Retrospective

    by Giuliano Antoniol, Gerardo Canfora, Gerardo Casazza, Andrea De Lucia, Ettore Merlo

    Software system documentation is almost always expressed informally in natural language and free text. Examples include requirement specifications, design documents, manual pages, system development journals, error logs, and related maintenance reports. In our 2002 seminal paper we proposed a method based on information retrieval to recover traceability links between source code and free text documents. A premise of our work was that programmers use meaningful names for program items, such as functions, variables, types, classes, and methods. The paper paved the way to the adoption of IR in software engineering opening a new perspective. Reflecting on the past twenty years we briefly overview the many results that have been achieved, however, the emergence of new technologies, such as AI, pose unprecedented challenges.