Clustering Tails in High Dimension
Preprint
Abstract
One potential solution to combat the scarcity of tail observations in extreme value analysis is to integrate information from multiple datasets sharing similar tail properties, for instance, a common extreme value index. In other words, for a multivariate dataset, we intend to group dimensions into clusters first, before applying any pooling techniques. This paper addresses the clustering problem for a high dimensional dataset, according to their extreme value indices. We propose an iterative clustering procedure that sequentially partitions the variables into groups, ordered from the heaviest-tailed to the lightesttailed distributions. At each step, our method identifies and extracts a group of variables that share the highest extreme value index among the remaining ones. This approach differs fundamentally from conventional clustering methods such as using pre-estimated extreme value indices in a two-step clustering method. We show the consistency property of the proposed algorithm and demonstrate its finite-sample performance using a simulation study and a real data application.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Multi-Dimensional Domain Generalization with Low-Rank StructuresJournal of the American Statistical Association, 2025
- Tail index estimation for tail adversarial stable time series with an application to high‐dimensional tail clusteringJournal of Time Series Analysis, 2024
- Similarity‐based clustering for patterns of extreme valuesStat, 2023
- A Survey on Negative TransferIEEE/CAA Journal of Automatica Sinica, 2022
- Structure Learning for Extremal Tree ModelsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2022
- Transfer Learning for High-Dimensional Linear Regression: Prediction, Estimation and Minimax OptimalityJournal of the Royal Statistical Society Series B: Statistical Methodology, 2021
- Individual Data Protected Integrative Regression Analysis of High-Dimensional Heterogeneous DataJournal of the American Statistical Association, 2021
- Distributed inference for the extreme value indexBiometrika, 2021
- PM2.5data reliability, consistency, and air quality assessment in five Chinese citiesJournal of Geophysical Research: Atmospheres, 2016
- Models for Exceedances Over High ThresholdsJournal of the Royal Statistical Society Series B: Statistical Methodology, 1990