Workshop 1: The Future of Psychiatric Measurement

The 1st 2013/2014 Workshop on Quantitative Methods in Education, Health and the Social Sciences (QMEHSS) will be on Friday November 15 from 11:00-12:30.   The seminar is held (as last year) in the NORC conference room 344.  NORC is located at 1155 E. 60th Street. 

 

The Future of Psychiatric Measurement

Robert Gibbons

University of Chicago

Mental health measurement has been based primarily on subjective judgment and classical test theory. Typically, impairment level is determined by a total score, requiring that all respondents be administered the same items. An alternative to full scale administration is adaptive testing in which different individuals may receive different scale items that are targeted to their specific impairment level. Within adaptive testing, individuals’ initial item responses are used to determine a provisional estimate of their standing on the measured trait (e.g., depression, anxiety) to be used for subsequent item selection. This form of testing has recently emerged in mental health research.

Based on item response theory (IRT) procedures, estimates of items (e.g., difficulty, discrimination) and individuals (e.g., severity of depression) can be obtained to more efficiently identify suitable item subsets for each individual. This approach to testing is referred to as computerized adaptive testing (CAT) and is immediately applicable to mental health measurement problems. We have developed a CAT depression inventory (CAT-DI), based on multidimensional  IRT, well suited to mental health constructs, that can be administered adaptively such that each individual responds only to those items that are most informative for assessing his/her level of depression. The net result is that an individual is administered a small, optimal number of items from a much larger “bank” of items, without loss of measurement precision.

The shift in paradigm is from small fixed length tests with questionable psychometric properties to large item banks from which an optimal small subset of items is adaptively drawn for each individual, targeted to their level of impairment. Rather than fixing the number of items and allowing measurement precision to vary, we fix measurement precision and allow the items to vary.

For longitudinal studies, the previous impairment estimate is then used as a starting point for the next adaptive test administration, further decreasing the number of items needed to be administered. Applications in the areas of screening depression in primary care, child psychiatry, global health, randomized controlled trials, molecular genetics, computerized adaptive diagnosis, and psychiatric epidemiology are described.

 

Gibbons  R.D., & Hedeker D.R. Full-information item bi-factor analysis. Psychometrika, 57, 423-436,1992.

Gibbons  R.D., Bock R.D., Hedeker D., Weiss D., Segawa E., Bhaumik D.K., Kupfer D., Frank E., Grochocinski V., Stover A. Full-Information Item Bi-Factor Analysis of Graded Response Data. Applied Psychological Measurement, 31, 4-19, 2007.

Gibbons R.D., Weiss D.J., Kupfer D.J., Frank E., Fagiolini A., Grochocinski V.J., Bhaumik D.K., Stover A. Bock R.D., Immekus J.C.  Using computerized adaptive testing to reduce the burden of  mental health assessment. Psychiatric Services, 59, 361-368, 2008.

Gibbons R.D., Weiss D.J., Pilkonis P.A., Frank E., Moore T., Kim J.B., Kupfer D.K. The CAT-DI: A computerized adaptive test for depression. JAMA Psychiatry, 69, 1104-1112, 2012.

Gibbons R.D., Hooker G., Finkelman M.D., Weiss D.J., Pilkonis P.A., Frank E., Moore T., Kupfer D.J.  The CAD-MDD: A computerized adaptive diagnostic screening tool for depression. Journal of Clinical Psychiatry, 74, 669-674, 2013.

Gibbons R.D., Weiss D.J., Pilkonis, P.A., Frank E., Moore T., Kim J.B., Kupfer D.J. Development of the CAT-ANX: A computerized adaptive test for anxiety. American Journal of Psychiatry, published on-line first, ajp.psychiatryonline.org.