On Correlation and Prediction Interval Reduction
Preprint
Abstract
Pearson's correlation coefficient is a popular statistical measure to summarize the strength of association between two continuous variables. It is usually interpreted via its square as percentage of variance of one variable predicted by the other in a linear regression model. It can be generalized for multiple regression via the coefficient of determination, which is not straightforward to interpret in terms of prediction accuracy. In this paper, we propose to assess the prediction accuracy of a linear model via the prediction interval reduction (PIR) by comparing the width of the prediction interval derived from this model with the width of the prediction interval obtained without this model. At the population level, PIR is one-to-one related to the correlation and the coefficient of determination. In particular, a correlation of 0.5 corresponds to a PIR of only 13%. It is also the one's complement of the coefficient of alienation introduced at the beginning of last century. We argue that PIR is easily interpretable and useful to keep in mind how difficult it is to make accurate individual predictions, an important message in the era of precision medicine and artificial intelligence. Different estimates of PIR are compared in the context of a linear model and an extension of the PIR concept to non-linear models is outlined.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Comprehensive Review of Neural Network-Based Prediction Intervals and New AdvancesIEEE Transactions on Neural Networks, 2011
- Height and the normal distribution: evidence from italian military dataDemography, 2009
- UsingR: Data Sets, Etc. for the Text "Using R for Introductory Statistics", Second Edition2004
- Descriptive Statistics, Part II: Most Commonly Used Descriptive StatisticsJournal for Specialists in Pediatric Nursing, 2003
- Introduction to biostatistics: Part 2, descriptive statisticsAnnals of Emergency Medicine, 1990
- On the interpretation of the correlation coefficient as a measure of predictive efficiency.Journal of Educational Psychology, 1946
- A percentage equivalent for the coefficient of correlation.Journal of Educational Psychology, 1926
- The Joint Yield from Teams of Tests.Journal of Educational Psychology, 1923
- Principle underlying the classification of men.Journal of Applied Psychology, 1919
- I. Co-relations and their measurement, chiefly from anthropometric dataProceedings of the Royal Society of London, 1889