Multiple Testing and the Distributional Effects of Accountability Incentives in Education

Lehrer, Steven F., R. Vincent Pohl, and Kyungchul Song. 2019. “Multiple Testing and the Distributional Effects of Accountability Incentives in Education.” Ruhr Economic Papers #799.

Download here

Abstract: This paper proposes bootstrap-based multiple testing procedures for quantile treatment effect heterogeneity under the assumption of selection on observables, and shows its asymptotic validity. Our procedure can be used to detect the ranges of quantiles and subgroups exhibiting treatment effect heterogeneity. We apply the multiple testing procedures to data from a large-scale Pakistani school report card experiment, and uncover evidence of policy-relevant heterogeneous effects from information provision on child test scores. Further, our analysis reinforces the importance of preventing the inflation of false positive conclusions because 75 percent of statistically significant quantile treatment effects become insignificant once corrections for multiple testing are applied.

Online Appendix