Optimal binning of correlated measurements

Authors:
Publisher website
Google Scholar
Abstract
Experimental measurements are commonly represented on a discrete grid, requiring a balance between granularity and statistical noise. Two strategies have traditionally been used to improve such representations: selecting an appropriate bin width to control discretization error and applying kernel-based smoothing to suppress fluctuations. Despite their shared goal, these approaches have largely developed independently, without a unified statistical description of how discretization and correlation jointly determine measurement precision. Here, we extend the discussion of optimal interval averaging to a correlation-aware setting by Gaussian process regression, which explicitly accounts for correlations among neighboring bins. Starting from first principles, we derive the mean-squared error of discretized measurements and obtain closed-form asymptotic expressions for the optimal bin width and correlation length. When recast in reduced variables, the theory reveals distinct universal scaling laws governing the error in the correlation-free and correlation-controlled regimes. Characterized by intrinsically smooth intensity profiles and counting-based statistics, neutron scattering measurements are well suited for demonstrating the enhanced error contraction enabled by inter-bin correlations. We show that such improvement is achievable over the experimentally accessible Q-range and across multiple instruments and material systems. These results show that explicitly accounting for correlations systematically reshapes the limits of precision in discretized, noise-limited measurements. More broadly, the framework provides a transferable statistical foundation for optimizing data representation, inference, and experimental design across the physical and data sciences.
Funding Information
  • Basic Energy Sciences (34532)
  • Basic Energy Sciences (DE-AC05-00OR22725)
  • National Science and Technology Council (111-2112-M-110-021-MY3)
  • National Science and Technology Council (113-2112-M-029-007)
  • National Science and Technology Council (114-2112-M-029-008)
  • National Science and Technology Council (114-2628-M-007-004-MY4)
  • National Science and Technology Council (NSTC 113-2112-M-213-010)

This publication has 21 references indexed in Scilit: