Douglas Mossman
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Douglas Mossman has a total of 15 co-authors. Here are the top co-authors by number of shared publications:
Dustin B. Wygant
Department of Psychology, Eastern Kentucky University
2 shared publicationsHongying Peng
Department of Environmental Health, University of Cincinnati College of Medicine, Cincinnati, OH, USA. (HP)
2 shared publicationsKathleen J. Hart
School of Psychology, Xavier University
2 shared publicationsRoger O. Gervais
Department of Educational Psychology, University of Alberta
2 shared publicationsDavid Bienenfeld
Department of Psychiatry, Wright State University, Boonshoft School of Medicine, Dayton, USA
1 shared publication- David J. Vanness
Department of Health Policy and Administration, The Pennsylvania State University, University Park, PA, USA
1 shared publication
Latest Publications
- 1 January 2018
- Published by American Psychological Association (APA) in Psychological Assessment
- Vol. 30 (1) , 74-85
Trial 1 versus Trial 2 of the Test of Memory Malingering: Evaluating accuracy without a “gold standard”.
This study examines the accuracy of the Test of Memory Malingering (TOMM), a frequently administered measure for evaluating effort during neurocognitive testing. In the last few years, several authors have suggested that the initial recognition trial of the TOMM (Trial 1) might be a more useful index for detecting feigned or exaggerated impairment than Trial 2, which is the source for inference recommended by the original instruction manual (Tombaugh, 1996). We used latent class modeling (LCM) implemented in a Bayesian framework to evaluate archival Trial 1 and Trial 2 data collected from 1,198 adults who had undergone outpatient forensic evaluations. All subjects were tested with 2 other performance validity tests (the Word Memory Test and the Computerized Assessment of Response Bias), and for 70% of the subjects, data from the California Verbal Learning Test-Second Edition Forced Choice trial were also available. Our results suggest that not even a perfect score on Trial 1 or Trial 2 justifies saying that an evaluee is definitely responding genuinely, although such scores imply a lower-than-base-rate probability of feigning. If one uses a Trial 2 cut-off higher than the manual's recommendation, Trial 2 does better than Trial 1 at identifying individuals who are almost certainly feigning while maintaining a negligible false positive rate. Using scores from both trials, one can identify a group of definitely feigning and very likely feigning subjects who comprise about 2 thirds of all feigners; only 1% of the members of this group would not be feigning.
- 1 January 2018
- Published by American Psychological Association (APA) in Psychological Assessment
- Vol. 30 (1) , 74-85
Trial 1 versus Trial 2 of the Test of Memory Malingering: Evaluating accuracy without a “gold standard”.
This study examines the accuracy of the Test of Memory Malingering (TOMM), a frequently administered measure for evaluating effort during neurocognitive testing. In the last few years, several authors have suggested that the initial recognition trial of the TOMM (Trial 1) might be a more useful index for detecting feigned or exaggerated impairment than Trial 2, which is the source for inference recommended by the original instruction manual (Tombaugh, 1996). We used latent class modeling (LCM) implemented in a Bayesian framework to evaluate archival Trial 1 and Trial 2 data collected from 1,198 adults who had undergone outpatient forensic evaluations. All subjects were tested with 2 other performance validity tests (the Word Memory Test and the Computerized Assessment of Response Bias), and for 70% of the subjects, data from the California Verbal Learning Test-Second Edition Forced Choice trial were also available. Our results suggest that not even a perfect score on Trial 1 or Trial 2 justifies saying that an evaluee is definitely responding genuinely, although such scores imply a lower-than-base-rate probability of feigning. If one uses a Trial 2 cut-off higher than the manual's recommendation, Trial 2 does better than Trial 1 at identifying individuals who are almost certainly feigning while maintaining a negligible false positive rate. Using scores from both trials, one can identify a group of definitely feigning and very likely feigning subjects who comprise about 2 thirds of all feigners; only 1% of the members of this group would not be feigning.
- 24 April 2015
- Published by SAGE Publications in Medical Decision Making
- Vol. 36 (3) , 349-365
Using Dual Beta Distributions to Create “Proper” ROC Curves Based on Rating Category Data
Background: Receiver operating characteristic (ROC) analysis helps investigators quantify and describe how well a diagnostic system discriminates between 2 mutually exclusive conditions. The conventional binormal (CvB) curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. When data sets evaluated under the CvB model have hooks, the resulting ROCs can contain misleading information about the diagnostic performance of the method at low and high false positive rates. Objective: To present and evaluate a dual beta (DB) ROC model that assumes diagnostic data arise from 2 β distributions. The DB model’s parameter constraints assure that the resulting ROC curve has a positive, monotonically decreasing slope. Design/Method: Computer simulation study comparing results from CvB, DB, and weighted power function (WPF) models. Results: The DB model produces results that are as good as or better than those from the WPF model, and less biased and closer to the true values than curves obtained using the CvB model. Conclusions: The DB ROC model expresses the relationship between the false positive rate and true positive rate in closed form and allows for quick ROC area calculations using spreadsheet functions. Because it posits simple relationships among the decision axis, operating points, and model parameters, the DB model offers investigators a flexible, easy-to-grasp ROC form that is simpler to implement than other proper ROC models.
- 24 April 2015
- Published by SAGE Publications in Medical Decision Making
- Vol. 36 (3) , 349-365
Using Dual Beta Distributions to Create “Proper” ROC Curves Based on Rating Category Data
Background: Receiver operating characteristic (ROC) analysis helps investigators quantify and describe how well a diagnostic system discriminates between 2 mutually exclusive conditions. The conventional binormal (CvB) curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. When data sets evaluated under the CvB model have hooks, the resulting ROCs can contain misleading information about the diagnostic performance of the method at low and high false positive rates. Objective: To present and evaluate a dual beta (DB) ROC model that assumes diagnostic data arise from 2 β distributions. The DB model’s parameter constraints assure that the resulting ROC curve has a positive, monotonically decreasing slope. Design/Method: Computer simulation study comparing results from CvB, DB, and weighted power function (WPF) models. Results: The DB model produces results that are as good as or better than those from the WPF model, and less biased and closer to the true values than curves obtained using the CvB model. Conclusions: The DB ROC model expresses the relationship between the false positive rate and true positive rate in closed form and allows for quick ROC area calculations using spreadsheet functions. Because it posits simple relationships among the decision axis, operating points, and model parameters, the DB model offers investigators a flexible, easy-to-grasp ROC form that is simpler to implement than other proper ROC models.
- 1 January 2015
- Published by American Psychological Association (APA) in Psychological Assessment
- Vol. 27 (3) , 763-776
A Bayesian approach to mixed group validation of performance validity tests.
Mental health professionals often use structured assessment tools to help detect individuals who are feigning or exaggerating symptoms. Yet estimating the accuracy of these tools is problematic because no "gold standard" establishes whether someone is malingering or not. Several investigators have recommended using mixed group validation (MGV) to estimate the accuracy of malingering measures, but simulation studies show that typical implementations of MGV may yield vague, biased, or logically impossible results. In this article we describe a Bayesian approach to MGV that addresses and avoids these limitations. After explaining the concepts that underlie our approach, we use previously published data on the Test of Memory Malingering (TOMM; Tombaugh, 1996) to illustrate how our method works. Our findings concerning the TOMM's accuracy, which include insights about covariates such as study population and litigation status, are consistent with results that appear in previous publications. Unlike most investigations of the TOMM's accuracy, our findings neither rely on possibly flawed assumptions about subjects' intentions nor assume that experimental simulators can duplicate the behavior of real-world examinees. Our conceptual approach may prove helpful in evaluating the accuracy of many assessment tools used in clinical contexts and psycholegal determinations. (PsycINFO Database Record
- 1 January 2015
- Published by American Psychological Association (APA) in Psychological Assessment
- Vol. 27 (3) , 763-776
A Bayesian approach to mixed group validation of performance validity tests.
Mental health professionals often use structured assessment tools to help detect individuals who are feigning or exaggerating symptoms. Yet estimating the accuracy of these tools is problematic because no "gold standard" establishes whether someone is malingering or not. Several investigators have recommended using mixed group validation (MGV) to estimate the accuracy of malingering measures, but simulation studies show that typical implementations of MGV may yield vague, biased, or logically impossible results. In this article we describe a Bayesian approach to MGV that addresses and avoids these limitations. After explaining the concepts that underlie our approach, we use previously published data on the Test of Memory Malingering (TOMM; Tombaugh, 1996) to illustrate how our method works. Our findings concerning the TOMM's accuracy, which include insights about covariates such as study population and litigation status, are consistent with results that appear in previous publications. Unlike most investigations of the TOMM's accuracy, our findings neither rely on possibly flawed assumptions about subjects' intentions nor assume that experimental simulators can duplicate the behavior of real-world examinees. Our conceptual approach may prove helpful in evaluating the accuracy of many assessment tools used in clinical contexts and psycholegal determinations. (PsycINFO Database Record
- 12 September 2013
- Published by SAGE Publications in Medical Decision Making
- Vol. 34 (4) , 523-535
Constructing “Proper” ROCs from Ordinal Response Data Using Weighted Power Functions
Background. Receiver operating characteristic (ROC) analysis is the standard method for describing the accuracy of diagnostic systems where the decision task involves distinguishing between 2 mutually exclusive possibilities. The popular binormal curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. Not infrequently, binormal ROCs have visible hooks that falsely imply worse-than-chance diagnostic differentiation where the curve lies below the no-information diagonal. In this article, we present and evaluate a 2-parameter, weighted power function (WPF) model that always results in a proper ROC curve with a positive, monotonically decreasing slope. Methods. We used a computer simulation study to compare results from binormal and WPF models. Results. The WPF model produces ROC curves that are less biased and closer to the true values than are curves obtained using the binormal model. The better performance of the WPF model follows from its design constraint as a necessarily proper ROC. Conclusions. The WPF model fits a broader variety of data sets than previously published power function models while maintaining straightforward relationships among the original decision variable, specific operating points, ROC curve contours, and model parameters. Compared with other proper ROC models, the WPF model is distinctive in its simplicity, and it avoids the flaws of the conventional binormal ROC model.
- 12 September 2013
- Published by SAGE Publications in Medical Decision Making
- Vol. 34 (4) , 523-535
Constructing “Proper” ROCs from Ordinal Response Data Using Weighted Power Functions
Background. Receiver operating characteristic (ROC) analysis is the standard method for describing the accuracy of diagnostic systems where the decision task involves distinguishing between 2 mutually exclusive possibilities. The popular binormal curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. Not infrequently, binormal ROCs have visible hooks that falsely imply worse-than-chance diagnostic differentiation where the curve lies below the no-information diagonal. In this article, we present and evaluate a 2-parameter, weighted power function (WPF) model that always results in a proper ROC curve with a positive, monotonically decreasing slope. Methods. We used a computer simulation study to compare results from binormal and WPF models. Results. The WPF model produces ROC curves that are less biased and closer to the true values than are curves obtained using the binormal model. The better performance of the WPF model follows from its design constraint as a necessarily proper ROC. Conclusions. The WPF model fits a broader variety of data sets than previously published power function models while maintaining straightforward relationships among the original decision variable, specific operating points, ROC curve contours, and model parameters. Compared with other proper ROC models, the WPF model is distinctive in its simplicity, and it avoids the flaws of the conventional binormal ROC model.
- 21 January 2013
- Published by Wiley in Behavioral Sciences & the Law
- Vol. 31 (1) , 23-39
Evaluating Risk Assessments Using Receiver Operating Characteristic Analysis: Rationale, Advantages, Insights, and Limitations
The last two decades have witnessed major changes in the way that mental health professionals assess, describe, and think about persons' risk for future violence. Psychiatrists and psychologists have gone from believing that they could not predict violence to feeling certain they can assess violence risk with well‐above‐chance accuracy. Receiver operating characteristic (ROC) analysis has played a central role in changing this view. This article reviews the key concepts underlying ROC methods, the meaning of the area under the ROC curve (AUC), the relationship between AUC and effect size d, and what these two indices tell us about evaluations of violence risk.The area under the ROC curve and d provide succinct but incomplete descriptions of discrimination capacity. These indices do not provide details about sensitivity–specificity trade‐offs; they do not tell us how to balance false‐positive and false‐negative errors; and they do not determine whether a diagnostic system is accurate enough to make practically useful distinctions between violent and non‐violent subject groups. Justifying choices or clinical practices requires a contextual investigation of outcomes, a process that takes us beyond simply knowing global indices of accuracy. Copyright © 2013 John Wiley & Sons, Ltd.
- 21 January 2013
- Published by Wiley in Behavioral Sciences & the Law
- Vol. 31 (1) , 23-39
Evaluating Risk Assessments Using Receiver Operating Characteristic Analysis: Rationale, Advantages, Insights, and Limitations
The last two decades have witnessed major changes in the way that mental health professionals assess, describe, and think about persons' risk for future violence. Psychiatrists and psychologists have gone from believing that they could not predict violence to feeling certain they can assess violence risk with well‐above‐chance accuracy. Receiver operating characteristic (ROC) analysis has played a central role in changing this view. This article reviews the key concepts underlying ROC methods, the meaning of the area under the ROC curve (AUC), the relationship between AUC and effect size d, and what these two indices tell us about evaluations of violence risk.The area under the ROC curve and d provide succinct but incomplete descriptions of discrimination capacity. These indices do not provide details about sensitivity–specificity trade‐offs; they do not tell us how to balance false‐positive and false‐negative errors; and they do not determine whether a diagnostic system is accurate enough to make practically useful distinctions between violent and non‐violent subject groups. Justifying choices or clinical practices requires a contextual investigation of outcomes, a process that takes us beyond simply knowing global indices of accuracy. Copyright © 2013 John Wiley & Sons, Ltd.
- Open Access
- 1 January 2013
- Published by Elsevier in SSRN Electronic Journal
- Vol. 43 (1) , 93-102
From Group Data to Useful Probabilities: The Relevance of Actuarial Risk Assessment in Individual Instances
Probability plays a ubiquitous role in decision-making through a process in which we use data from groups of past outcomes to make inferences about new situations. Yet in recent years, many forensic mental health professionals have become persuaded that overly-wide confidence intervals render actuarial risk assessment instruments virtually useless in individual assessments. If this were true, the mathematical properties of probabilistic judgments would preclude forensic clinicians from applying group-based findings about risk to individuals. As a consequence, actuarially based risk estimates might be barred from use in legal proceedings. Using a fictional scenario, this article shows how group data have an obvious application to individual decisions. The article also explains how misunderstanding the aims of risk assessment has led to mistakes about how, when, and why group data apply to individual instances. Although actuarially based statements about individuals’ risk have many pitfalls, confidence intervals pose no barrier to using actuarial tools derived from group data to improve decision-making about individual instances.
- 1 January 2013
- Published by Elsevier in SSRN Electronic Journal
- Vol. 43 (1) , 93-102
Open AccessFrom Group Data to Useful Probabilities: The Relevance of Actuarial Risk Assessment in Individual Instances
Probability plays a ubiquitous role in decision-making through a process in which we use data from groups of past outcomes to make inferences about new situations. Yet in recent years, many forensic mental health professionals have become persuaded that overly-wide confidence intervals render actuarial risk assessment instruments virtually useless in individual assessments. If this were true, the mathematical properties of probabilistic judgments would preclude forensic clinicians from applying group-based findings about risk to individuals. As a consequence, actuarially based risk estimates might be barred from use in legal proceedings. Using a fictional scenario, this article shows how group data have an obvious application to individual decisions. The article also explains how misunderstanding the aims of risk assessment has led to mistakes about how, when, and why group data apply to individual instances. Although actuarially based statements about individuals’ risk have many pitfalls, confidence intervals pose no barrier to using actuarial tools derived from group data to improve decision-making about individual instances.
- 1 December 2012
- Published by American Psychological Association (APA) in Psychological Assessment
- Vol. 24 (4) , 815-822
Estimating the accuracy of neurocognitive effort measures in the absence of a “gold standard”.
Psychologists frequently use symptom validity tests (SVTs) to help determine whether evaluees' test performance or reported symptoms accurately represent their true functioning and capability. Most studies evaluating the accuracy of SVTs have used either known-group comparisons or simulation designs, but these approaches have well-known limitations (potential misclassifications or lack of ecological validity). This study uses latent class modeling (LCM) implemented in a Bayesian framework to estimate SVT classification accuracy based on data obtained from real-life forensic evaluations. We obtained archival data from 1,301 outpatient evaluees who underwent testing with the Computerized Assessment of Response Bias (CARB), the Test of Memory Malingering (TOMM), and the Word Memory Test (WMT) in a forensic evaluation context. Under various data models, Markov chain Monte Carlo methods implemented via WinBUGS converged to target distributions that permitted Bayesian estimates of SVT accuracy. Under the most plausible model (conditional dependence in test results), classification accuracies (expressed as area under the "trapezoidal" receiver operating characteristic curve ± standard deviation) were as follows: CARB = 0.765 ± 0.030, WMT = 0.929 ± 0.020, and TOMM = 0.771 ± 0.034. At decision thresholds that hold false positive rates at 0.02, the SVTs would detect invalid responses (true positives) at rates of approximately 35%, 65%, and 49%, respectively, for the 3 tests. Though LCM methods have limitations, this study suggests that they offer an approach to SVT evaluation that avoids methodological pitfalls of known-group research designs while retaining ecological validity that is absent in simulation studies.
- 1 December 2012
- Published by American Psychological Association (APA) in Psychological Assessment
- Vol. 24 (4) , 815-822
Estimating the accuracy of neurocognitive effort measures in the absence of a “gold standard”.
Psychologists frequently use symptom validity tests (SVTs) to help determine whether evaluees' test performance or reported symptoms accurately represent their true functioning and capability. Most studies evaluating the accuracy of SVTs have used either known-group comparisons or simulation designs, but these approaches have well-known limitations (potential misclassifications or lack of ecological validity). This study uses latent class modeling (LCM) implemented in a Bayesian framework to estimate SVT classification accuracy based on data obtained from real-life forensic evaluations. We obtained archival data from 1,301 outpatient evaluees who underwent testing with the Computerized Assessment of Response Bias (CARB), the Test of Memory Malingering (TOMM), and the Word Memory Test (WMT) in a forensic evaluation context. Under various data models, Markov chain Monte Carlo methods implemented via WinBUGS converged to target distributions that permitted Bayesian estimates of SVT accuracy. Under the most plausible model (conditional dependence in test results), classification accuracies (expressed as area under the "trapezoidal" receiver operating characteristic curve ± standard deviation) were as follows: CARB = 0.765 ± 0.030, WMT = 0.929 ± 0.020, and TOMM = 0.771 ± 0.034. At decision thresholds that hold false positive rates at 0.02, the SVTs would detect invalid responses (true positives) at rates of approximately 35%, 65%, and 49%, respectively, for the 3 tests. Though LCM methods have limitations, this study suggests that they offer an approach to SVT evaluation that avoids methodological pitfalls of known-group research designs while retaining ecological validity that is absent in simulation studies.
- 1 January 2010
- Published by American Psychological Association (APA) in Law and Human Behavior
- Vol. 34 (5) , 402-417
Quantifying the accuracy of forensic examiners in the absence of a “gold standard”.
This study asked whether latent class modeling methods and multiple ratings of the same cases might permit quantification of the accuracy of forensic assessments. Five evaluators examined 156 redacted court reports concerning criminal defendants who had undergone hospitalization for evaluation or restoration of their adjudicative competence. Evaluators rated each defendant's Dusky-defined competence to stand trial on a five-point scale as well as each defendant's understanding of, appreciation of, and reasoning about criminal proceedings. Having multiple ratings per defendant made it possible to estimate accuracy parameters using maximum likelihood and Bayesian approaches, despite the absence of any "gold standard" for the defendants' true competence status. Evaluators appeared to be very accurate, though this finding should be viewed with caution.
- 1 January 2010
- Published by American Psychological Association (APA) in Law and Human Behavior
- Vol. 34 (5) , 402-417
Quantifying the accuracy of forensic examiners in the absence of a “gold standard”.
This study asked whether latent class modeling methods and multiple ratings of the same cases might permit quantification of the accuracy of forensic assessments. Five evaluators examined 156 redacted court reports concerning criminal defendants who had undergone hospitalization for evaluation or restoration of their adjudicative competence. Evaluators rated each defendant's Dusky-defined competence to stand trial on a five-point scale as well as each defendant's understanding of, appreciation of, and reasoning about criminal proceedings. Having multiple ratings per defendant made it possible to estimate accuracy parameters using maximum likelihood and Bayesian approaches, despite the absence of any "gold standard" for the defendants' true competence status. Evaluators appeared to be very accurate, though this finding should be viewed with caution.
- 3 March 2009
- Published by Taylor & Francis in Journal of Legal Medicine
- Vol. 30 (1) , 109-140
The Imperfection of Protection Through Detection and Intervention
(2009). The Imperfection of Protection Through Detection and Intervention. Journal of Legal Medicine: Vol. 30, No. 1, pp. 109-140.
- 3 March 2009
- Published by Taylor & Francis in Journal of Legal Medicine
- Vol. 30 (1) , 109-140
The Imperfection of Protection Through Detection and Intervention
(2009). The Imperfection of Protection Through Detection and Intervention. Journal of Legal Medicine: Vol. 30, No. 1, pp. 109-140.
- 1 January 2009
- Vol. 37 (1) , 25-7
Let's think about human factors, not human failings.
Doctors typically think about medical errors as potential causes of malpractice litigation, as failures by individuals, and as evidence of personal incompetence that may deserve sanctions. Other professions take a different view: designing of safer systems, rather than criticism and punishment, is the way to prevent unintentional mishaps. In his article, Jeffrey Janofsky shows how psychiatrists can think about making care systems safer for patients. He also provides a splendid example of how forensic psychiatrists should conceptualize legal and medical problems encountered in clinical practice.
- 1 January 2009
- Vol. 37 (1) , 25-7
Let's think about human factors, not human failings.
Doctors typically think about medical errors as potential causes of malpractice litigation, as failures by individuals, and as evidence of personal incompetence that may deserve sanctions. Other professions take a different view: designing of safer systems, rather than criticism and punishment, is the way to prevent unintentional mishaps. In his article, Jeffrey Janofsky shows how psychiatrists can think about making care systems safer for patients. He also provides a splendid example of how forensic psychiatrists should conceptualize legal and medical problems encountered in clinical practice.