A Permutation Test for Assessing the Presence of Individual Differences in Treatment Effects
Authors:
- Chi ChangCCChi Chang
- Thomas JakiTJThomas Jaki
- Muhammad Saad SadiqMSMuhammad Saad Sadiq
- Alena A. KuhlemeierAKAlena A. Kuhlemeier
- Daniel FeasterDFDaniel Feaster
- Nathan ColeNCNathan Cole
- Andrea LamontALAndrea Lamont
- Daniel OberskiDODaniel Oberski
- Yasin DesaiYDYasin Desai
- M. Lee Van HornMHM. Lee Van Horn
Preprint
Abstract
One size fits all approaches to medicine have become a thing of the past as the understanding of individual differences grows. The paper introduces a test for the presence of heterogeneity in treatment effects in a clinical trial. Heterogeneity is assessed on the basis of the predicted individual treatment effects (PITE) framework and a permutation test is utilized to establish if significant heterogeneity is present. We first use the novel test to show that heterogeneity in the effects of interventions exists in the Amyotrophic Lateral Sclerosis Clinical Trials. We then show, using two different predictive models (linear regression model and Random Forests) that the test has adequate type I error control. Next, we use the ALS data as the basis for simulations to demonstrate the ability of the permutation test to find heterogeneity in treatment effects as a function of both effect size and sample size. We find that the proposed test has good power to detected heterogeneity in treatment effects when the heterogeneity was due primarily to a single predictor, or when it was spread across the predictors. The predictive model, on the other hand is of secondary importance to detect heterogeneity. The non-parametric property of the permutation test can be applied with any predictive method and requires no additional assumptions to obtain PITEs.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Tutorial in biostatistics: data‐driven subgroup identification and analysis in clinical trialsStatistics in Medicine, 2016
- Prediction models need appropriate internal, internal–external, and external validationJournal of Clinical Epidemiology, 2016
- The Precision Medicine InitiativeJAMA, 2015
- A regression tree approach to identifying subgroups with differential treatment effectsStatistics in Medicine, 2015
- The clinical course of alcohol use disorders: Using joinpoint analysis to aid in interpretation of growth mixture modelsDrug and Alcohol Dependence, 2013
- Qualitative interaction trees: a tool to identify qualitative treatment–subgroup interactionsStatistics in Medicine, 2013
- COMPREHENSIVE REVIEW OF FACTORS IMPLICATED IN THE HETEROGENEITY OF RESPONSE IN DEPRESSIONDepression and Anxiety, 2012
- Subgroup identification based on differential effect search—A recursive partitioning method for establishing response to treatment in patient subpopulationsStatistics in Medicine, 2011
- Modelling treatment‐effect heterogeneity in randomized controlled trials of complex interventions (psychological treatments)Statistics in Medicine, 2007
- Order-restricted inference for monotone trend alternatives in contingency tablesComputational Statistics & Data Analysis, 1998