Alistair Moffat

Measurement is a fundamental human activity. It is through measurement that we identify opportunity, and via opportunity that we seek improvement. In information retrieval and web search we measure how good search engines are at ordering answers to user queries, how close two ranked lists are to each other, how good LLMs are at re-ranking sets of candidate documents, and how close generated answer sentences are to the ideal output. This talk begins by motivating the well-known ''top-weighted'' measurements that are used when assessing the quality of a ranked list of documents against known annotations (qrels). In particular, Moffat and Zobel [2] suggest use of a parameterized geometric distribution to assign weights to ranks, arguing that a distribution with a bounded infinite sum is preferable to ones with unbounded sums; and also motivating the geometric sequence via a proposed user browsing model, the first time this had been done. Their rankbiased precision mechanism has the further advantage of allowing the extent of the uncertainty to be quantified, where uncertainty arises whenever the qrels are not comprehensive.

Process mining bridges the gap between theoretical models and real-world processes by extracting actionable insights from the event logs generated by business operations. The growing importance of process mining means that the presence of noise in event logs poses a significant obstacle, reducing the accuracy and dependability of analysis. Moreover, the concept of noise lacks a unified definition in process mining. To address that challenge, we present a comprehensive review of the literature to consolidate definitions of noise and establish a unified taxonomy, deriving a hierarchical decomposition framework of noise types, and discussing their origins. Building on this framework, we next introduce a noise injection tool, called SNIP, that enables controlled experimentation by simulating various noise scenarios in event logs, with our discussion here supported by a case study demonstrating the tool’s application. Lastly, we explore how established noise concepts from related disciplines can be applied to enrich process mining methodologies. Together, these contributions provide a structured foundation for understanding noise in event logs, promoting more robust and reliable process mining outcomes.