Workshop 2: Random Coefficient Models for Multi-site Trials With Continuous Outcomes

The 2nd 2013/2014 Workshop on Quantitative Methods in Education, Health and the Social Sciences (QMEHSS) will be on Friday January 10th from 11:00-12:30 and will be led by Dr. Stephen Raudenbush.   The seminar will be held in the NORC conference room 344.  NORC is located at 1155 E. 60th Street. We would also like to remind you that the QMEHSS website is now live and can be accessed via the following link: qrm.uchicago.edu. Please bookmark it and visit for the latest information regarding all things related to QMEHSS.

 

Random Coefficient Models for Multi-site Trials With Continuous Outcomes
Stephen W. Raudenbush

Department of Sociology, Harris School of Public Policy,
and Committee on Education
University of Chicago

Presentation at the Quantitative Methods in Social Science Seminar
January 10, 2014
Abstract

Multi-site randomized trials are now common in education, social welfare, and medicine. This design enables us to estimate not only the average impact of assignment to a new program but also to study the distribution of impacts across heterogeneous organizational and social conditions. Of interest are questions about the variation in impacts across sites, quantiles of the impact, models that predict site-specific impacts, and the site-specific impacts themselves. A simple hierarchical model for outcome Y of participant i in site j is Y ij = U0j + B j T ij + e ij where Tij is a binary treatment indicator ( Tij =1 if assigned to the treatment, Tij =0 if assigned to the control), U0j is the control group mean in site j and Bj is the average impact of treatment in site j. Of interest are the average impact E(B) , the variance of the impact across sites Var (B) , the average control group mean E(U) , the variance of the control group means across sites Var (C) , and the covariance of the control group means and the impacts Cov ( B, C ) , as well as site-specific impacts. In this talk I will consider four methods of estimation: a standard, two-level hierarchical linear model with randomly varying intercepts and slopes; a standard econometric fixed effects model; a simplified random coefficient model that eliminates the random intercept, and a family of two-level hierarchical linear models with inverse probability of treatment weighting (HLM-IPTW). When the probability of treatment assignment and the sample sizes vary from site to site, which is nearly always the case, only the HLM-IPTW approach allows unbiased estimation of all parameters of the theoretical model without requiring implausibly strong assumptions. However, as imbalance in the design increases, the variation in the weights increases, removing bias but increasing variance. When interest is restricted to B , E ( B ), and Var (B) , the simplified random coefficient model (with no intercept) appears to provide the most satisfactory approach. These findings have implications for a broad array of applications including randomized trials with non-compliance, multilevel observational studies, and multi-site mediation models.