On the Near Impossibility of Measuring the Returns to Advertising
Authors:
- Justin M. RaoJRJustin M. Rao
Preprint
- 1 January 2013
- preprint
- Published by Elsevier in SSRN Electronic Journal
- Vol. 130 (4)
Abstract
Twenty-five large field experiments with major U.S. retailers and brokerages, most reaching millions of customers and collectively representing $2.8 million in digital advertising expenditure, reveal that measuring the returns to advertising is difficult. The median confidence interval on return on investment is over 100 percentage points wide. Detailed sales data show that relative to the per capita cost of the advertising, individual-level sales are very volatile; a coefficient of variation of 10 is common. Hence, informative advertising experiments can easily require more than 10 million person-weeks, making experiments costly and potentially infeasible for many firms. Despite these unfavorable economics, randomized control trials represent progress by injecting new, unbiased information into the market. The inference challenges revealed in the field experiments also show that selection bias, due to the targeted nature of advertising, is a crippling concern for widely employed observational methods.Keywords
This publication has 13 references indexed in Scilit:
- Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment2014
- Television Advertising and Online SearchManagement Science, 2014
- Do Online Advertisements Increase Political Candidates’ Name Recognition or Favorability? Evidence from Randomized Field ExperimentsPolitical Behavior, 2013
- Improving the sensitivity of online controlled experiments by utilizing pre-experiment data2013
- Does Management Matter? Evidence from India2011
- Vitamin E in the Primary Prevention of Cardiovascular Disease and CancerJAMA, 2005
- The Causal Effect of Education on Earnings1999
- Which tax rate do people use: Average or marginal?Journal of Public Economics, 1995
- Advertising Experiments at the Campbell Soup CompanyMarketing Science, 1989
- The Navy Enlistment Marketing ExperimentMarketing Science, 1985