Niall M. Adams

This work proposes a novel rank-based scale two-sample testing method for univariate, distinct data when a subset of the data may be missing. Our approach is based on mathematically tight bounds of the Ansari-Bradley test statistic in the presence of missing data, and rejects the null hypothesis if the test statistic is significant regardless of the missing values. This proposed scale testing method is then combined with the location testing method proposed by Zeng et al. (2024) using the Holm-Bonferroni correction for location-scale testing. We show that our methods control the Type I error regardless of the values of the missing data. Simulation results demonstrate that our methods have good statistical power, typically when less than 10% of the data are missing, while other missing data methods, such as case deletion or imputation methods, fail to control the Type I error when the data are missing not at random. We illustrate the proposed location-scale testing method on hepatitis C virus dataset where a subset of values is unobserved.

This work studies exact bounds of Spearman’s footrule in the presence of missing data for two n-dimensional distinct real-valued vectors X and Y. The lower bound is obtained by sequentially constructing imputations of the partially observed vectors, each with a non-increasing value of Spearman’s footrule. The upper bound is found by first considering the set of all possible values of Spearman’s footrule for imputations of X and Y, and then the size of this set is gradually reduced using several constraints. Algorithms with computational complexities O(n2) and O(n3) are provided for computing these tight lower and upper bounds, respectively. As an application, we propose a novel two-sample independence testing method for data with missing values. Improving on all existing approaches, our method controls the Type I error under arbitrary missingness. Simulation results demonstrate that our method has good power, typically when the proportion of pairs containing missing data is below 15%. We illustrate our method on real-world data by comparing the dependence of yearly precipitation levels in the United Kingdom, Germany and the Unites States.

Streaming data provides substantial challenges for data analysis. From a computational standpoint, these challenges arise from constraints related to computer memory and processing speed. Statistically, the challenges relate to constructing procedures that can handle the so-called concept drift —the tendency of future data to have different underlying properties to current and historic data. The issue of handling structure, such as trend and periodicity, remains a difficult problem for streaming estimation. We propose the real-time adaptive component (RAC), a penalized-regression modeling framework that satisfies the computational constraints of streaming data, and provides the capability for dealing with concept drift. At the core of the estimation process are techniques from adaptive filtering. The RAC procedure adopts a specified basis to handle local structure, along with a least absolute shrinkage operator-like penalty procedure to handle over fitting. We enhance the RAC estimation procedure with a streaming anomaly detection capability. The experiments with simulated data suggest the procedure can be considered as a competitive tool for a variety of scenarios, and an illustration with real cyber-security data further demonstrates the promise of the method.

There is currently a gap in theory for point patterns that lie on the surface of objects, with researchers focusing on patterns that lie in a Euclidean space, typically planar and spatial data. Methodology for planar and spatial data thus relies on Euclidean geometry and is therefore inappropriate for analysis of point patterns observed in non-Euclidean spaces. Recently, there has been extensions to the analysis of point patterns on a sphere, however, many other shapes are left unexplored. This is in part due to the challenge of defining the notion of stationarity for a point process existing on such a space due to the lack of rotational and translational isometries. Here, we construct functional summary statistics for Poisson processes defined on convex shapes in three dimensions. Using the Mapping Theorem, a Poisson process can be transformed from any convex shape to a Poisson process on the unit sphere which has rotational symmetries that allow for functional summary statistics to be constructed. We present the first and second order properties of such summary statistics and demonstrate how they can be used to test whether an observed pattern exhibits complete spatial randomness or spatial preference on the original convex space. A study of the Type I and II errors of our test statistics are explored through simulations on ellipsoids of varying dimensions.

A key difficulty that arises from real event data is imprecision in the recording of event time-stamps. In many cases, retaining event times with a high precision is expensive due to the sheer volume of activity. Combined with practical limits on the accuracy of measurements, aggregated data is common. In order to use point processes to model such event data, tools for handling parameter estimation are essential. Here we consider parameter estimation of the Hawkes process, a type of self-exciting point process that has found application in the modeling of financial stock markets, earthquakes and social media cascades. We develop a novel optimization approach to parameter estimation of aggregated Hawkes processes using a Monte Carlo Expectation-Maximization (MC-EM) algorithm. Through a detailed simulation study, we demonstrate that existing methods are capable of producing severely biased and highly variable parameter estimates and that our novel MC-EM method significantly outperforms them in all studied circumstances. These results highlight the importance of correct handling of aggregated data.