Kai Puolamäki

Count data represented as a matrix of non-negative integer values, such as contingency tables, are prevalent across diverse domains. When clustering such data sets, specific methods are required, as generic algorithms often fail to consider their unique distributional properties, leading to unreliable outputs. An effective strategy is to use well-established statistical models such as the Poisson and negative binomial distributions. We present 3CPO, a clustering algorithm based on statistically solid modeling of count data. In addition to the cluster labels, it identifies a subset of relevant columns, enhancing the interpretability of the results. We propose a simple iterative algorithm that maximizes the posterior probability to find good clustering solutions and discuss its properties. Extensive experiments demonstrate its ability to define high-quality clusters within associated subspaces for various data domains, ranging from gene expressions and texts to economics. Our findings suggest that 3CPO is a robust solution for clustering count data in a statistically sound and interpretable manner. Our code is available at https://github.com/collinleiber/3CPO .

Aerosol particles larger than roughly 50–100 nm in diameter are climatically important because they can act as cloud condensation nuclei (CCN), making their global number concentrations essential for understanding aerosol–cloud interactions. However, observationally constrained, long-term global datasets of particle number concentrations in this size range remain scarce. In this investigation, we present a global dataset of ground-level particle number concentrations for the period 2003–2024, produced by combining in situ observations with a machine-learning approach. The dataset includes two variables: the number concentrations for particles larger than 100 nm (N100) and larger than 50 nm (N50), provided at 0.75° × 0.75° spatial resolution and daily temporal resolution. To generate this dataset, we trained an eXtreme Gradient Boosting (XGB) model using measurements from 62 in situ stations as targets and reanalysis variables as predictors, enabling a data-driven representation of particle number concentrations at the global scale. We evaluated the dataset against independent observations from 12 additional stations. At 2/3 of these stations, the dataset shows good performance, capturing the median concentrations within a factor of 1.5 from the observations. Furthermore, we describe the main characteristics of the dataset in terms of global spatial patterns, temporal variability, and seasonal cycles, and demonstrate its ability to capture long-term trends in particle number concentrations, including both increasing and decreasing tendencies reported in the literature. This work provides the first observation-constrained, machine-learning-based global dataset of N50 and N100 at daily resolution over two decades, bridging the gap between sparse measurements and computationally expensive process-based models. The dataset, publicly available at https://doi.org/10.5281/zenodo.20202080, offers a valuable resource for evaluating model simulations, improving CCN-related parameterizations, and supporting weather and climate studies without the need for explicit knowledge of the aerosol particle microphysics.

Accurate global estimates of accumulation-mode particle number concentrations (N₁₀₀) are essential for understanding aerosol–cloud interactions and their climate effects and for improving Earth system models. However, traditional methods relying on sparse in situ measurements lack comprehensive coverage, and indirect satellite retrievals have limited sensitivity in the relevant size range. To overcome these challenges, we apply machine learning (ML) techniques – multiple linear regression (MLR) and eXtreme Gradient Boosting (XGB) – to generate daily global N₁₀₀ fields using in situ measurements as target variables and reanalysis data from the Copernicus Atmosphere Monitoring Service (CAMS) and ERA5 as predictor variables. Our cross-validation showed that ML models captured N₁₀₀ concentrations well in environments well-represented in the training set, with over 70 % of daily estimates being within a factor of 1.5 of observations. However, performance declines in underrepresented regions and conditions, such as in clean and remote environments, including marine, tropical, and polar regions, underscoring the need for more diverse observations. The most important predictors for N₁₀₀ in the ML models were aerosol-phase sulfate and gas-phase ammonia concentrations, followed by carbon monoxide and sulfur dioxide. Although black carbon and organic matter showed the highest feature importance values, their opposing signs in the MLR model coefficients suggest that their effects largely offset each other’s contributions to the N₁₀₀ estimate. By directly linking estimates to in situ measurements, our ML approach provides valuable insights into the global distribution of N₁₀₀ and serves as a complementary tool for evaluating Earth system model outputs and advancing the understanding of aerosol processes and their role in the climate system.

Understanding how atmospheric molecular clusters form and grow is key to resolving one of the biggest uncertainties in climate modelling: the formation of new aerosol particles. While quantum chemistry offers accurate insights into these early-stage clusters, its steep computational costs limit large-scale exploration. In this work, we present a fast, interpretable, and surprisingly powerful alternative: $k$-nearest neighbour ($k$-NN) regression model. By leveraging chemically informed distance metrics, including a kernel-induced metric and one learned via metric learning for kernel regression (MLKR), we show that simple $k$-NN models can rival more complex kernel ridge regression (KRR) models in accuracy, while reducing computational time by orders of magnitude. We perform this comparison with the well-established Faber-Christensen-Huang-Lilienfeld (FCHL19) molecular descriptor, but other descriptors (e.g., FCHL18, MBDF, and CM) can be shown to have similar performance. Applied to both simple organic molecules in the QM9 benchmark set and large datasets of atmospheric molecular clusters (sulphuric acid-water and sulphuric-multibase -base systems), our $k$-NN models achieve near-chemical accuracy, scale seamlessly to datasets with over 250,000 entries, and even appears to extrapolate to larger unseen clusters with minimal error (often nearing 1 kcal/mol). With built-in interpretability and straightforward uncertainty estimation, this work positions $k$-NN as a potent tool for accelerating discovery in atmospheric chemistry and beyond.

Accurate global estimates of accumulation-mode particle number concentrations (N₁₀₀) are essential for understanding aerosol–cloud interactions, their climate effects, and improving Earth System Models. However, traditional methods relying on sparse in situ measurements lack comprehensive coverage, and indirect satellite retrievals have limited sensitivity in the relevant size range. To overcome these challenges, we apply machine learning (ML) techniques— multiple linear regression (MLR) and eXtreme Gradient Boosting (XGB)—to generate daily global N₁₀₀ fields, using in situ measurements as target variables and reanalysis data from Copernicus Atmosphere Monitoring Service (CAMS) and ERA5 as predictor variables. Our cross-validation showed that ML models captured N₁₀₀ concentrations well in environments well-represented in the training set, with over 70 % of daily estimates within a factor of 1.5 of observations. However, performance declines in underrepresented regions and conditions, such as clean and remote environments, underscoring the need for more diverse observations. The most important predictors for N₁₀₀ in theML models were aerosol-phase sulphate and gas-phase ammonia concentrations, followed by carbon monoxide and sulfur dioxide. Although black carbon and organic matter showed the highest feature importance values, their opposing signs in the MLR model coefficients suggest their effects largely offset each other’s contribution to the N₁₀₀ estimate. By directly linking estimates to in situ measurements, our ML approach provides valuable insights into the global distribution of N₁₀₀ and serves as a complementary tool for evaluating Earth System Model outputs and advancing the understanding of aerosol processes and their role in the climate system.

Peroxy radicals (RO ₂ ) are ubiquitous intermediates in many oxidation processes, especially in the atmospheric gas phase.

We propose a new supervised manifold visualisation method, slipmap, that finds local explanations for complex black-box supervised learning methods and creates a two-dimensional embedding of the data items such that data items with similar local explanations are embedded nearby. This work extends and improves our earlier algorithm and addresses its shortcomings: poor scalability, inability to make predictions, and a tendency to find patterns in noise. We present our visualisation problem and provide an efficient GPU-optimised library to solve it. We experimentally verify that slipmap is fast and robust to noise, provides explanations that are on the level or better than the other local explanation methods, and are usable in practice.