Ricardo Sousa

Accurate recording of employee working hours is fundamental for workforce management, operational planning, and regulatory compliance. Despite the widespread adoption of digital time-tracking systems, timesheet records remain susceptible to irregularities that can distort labor metrics, productivity indicators, and cost estimations. This study proposes a domain-informed analytical framework for detecting, classifying, and interpreting anomalous entries in employee attendance data.The methodology integrates outlier detection with operational context in a structured workflow. First, six relative deviation features are engineered to capture directional differences between planned and recorded work and lunch periods, including start times, end times, and durations. These features are normalized to ensure comparability across heterogeneous shifts. Second, univariate Tukey’s fences are applied to identify mild and extreme outliers for each deviation feature. Extreme outliers are interpreted as potential measurement errors, whereas mild outliers are classified according to domain-defined directional rules as either operationally acceptable or operationally detrimental deviations. Third, unauthorized deviations are analyzed using Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) to reveal recurring behavioral patterns within the multidimensional deviation space. Finally, employee-level behavioral risk is quantified through a normalized Severity Index based on the frequency of unauthorized deviations relative to attendance frequency, enabling both global ranking and temporal monitoring.Applied to 4,726 anonymized timesheet records, the proposed approach effectively distinguishes measurement errors, acceptable deviations, and operationally detrimental behaviors while revealing structured patterns of noncompliance. By integrating robust statistics with domain knowledge, it enables scalable attendance analytics and workforce governance.

Outlier detection is a widely used technique for identifying anomalous or exceptional events across various contexts. It has proven to be valuable in applications like fault detection, fraud detection, and real-time monitoring systems. Detecting outliers in real time is crucial in several industries, such as financial fraud detection and quality control in manufacturing processes. In the context of big data, the amount of data generated is enormous, and traditional batch mode methods are not practical since the entire dataset is not available. The limited computational resources further compound this issue. Boxplot is a widely used batch mode algorithm for outlier detection that involves several derivations. However, the lack of an incremental closed form for statistical calculations during boxplot construction poses considerable challenges for its application within the realm of big data. We propose an incremental/online version of the boxplot algorithm to address these challenges. Our proposed algorithm is based on an approximation approach that involves numerical integration of the histogram and calculation of the cumulative distribution function. This approach is independent of the dataset’s distribution, making it effective for all types of distributions, whether skewed or not. To assess the efficacy of the proposed algorithm, we conducted tests using simulated datasets featuring varying degrees of skewness. Additionally, we applied the algorithm to a real-world dataset concerning software fault detection, which posed a considerable challenge. The experimental results underscored the robust performance of our proposed algorithm, highlighting its efficacy comparable to batch mode methods that access the entire dataset. Our online boxplot method, leveraging dataset distribution to define whiskers, consistently achieved exceptional outlier detection results. Notably, our algorithm demonstrated computational efficiency, maintaining constant memory usage with minimal hyperparameter tuning.

Clustering can be used to extract insights from data or to verify some of the assumptions held by the domain experts, namely data segmentation. In the literature, few methods can be applied in clustering qualitative values using the context associated with other variables present in the data, without losing interpretability. Moreover, the metrics for calculating dissimilarity between qualitative values often scale poorly for high dimensional mixed datasets. In this study, we propose a novel method for clustering qualitative values, based on Hierarchical Clustering (HQC), and using Maximum Mean Discrepancy. HQC maintains the original interpretability of the qualitative information present in the dataset. We apply HQC to two datasets. Using a mixed dataset provided by Spotify, we showcase how our method can be used for clustering music artists based on the quantitative features of thousands of songs. In addition, using financial features of companies, we cluster company industries, and discuss the implications in investment portfolios diversification.

The dynamic behavior of urban mobility patterns makes matching taxi supply with demand as one of the biggest challenges in this industry. Recently, the increasing availability of massive broadcast GPS data has encouraged the exploration of this issue under different perspectives. One possible solution is to build a data-driven real-time taxi-dispatching recommender system. However, existing systems are based on strong assumptions such as stationary demand distributions and finite training sets, which make them inadequate for modeling the dynamic nature of the network. In this paper, we propose BRIGHT: a drift-aware supervised learning framework which aims to provide accurate predictions for short-term horizon taxi demand quantities through a creative ensemble of time series analysis methods that handle distinct types of concept drift. A large experimental set-up which includes three real-world transportation networks and a synthetic test-bed with artificially inserted concept drifts, was employed to illustrate the advantages of BRIGHT when compared to S.o.A methods for this problem.