Multiple Testing and the Distributional Effects of Accountability Incentives in Education

Lehrer, Steven F., R. Vincent Pohl, and Kyungchul Song. 2021. “Multiple Testing and the Distributional Effects of Accountability Incentives in Education.” Accepted at Journal of Business and Economic Statistics.

Download here

Abstract: This paper proposes bootstrap-based multiple testing procedures for quantile treatment effect heterogeneity under the assumption of selection on observables, and shows its asymptotic validity. Our procedure can be used to detect the quantiles and subgroups exhibiting treatment effect heterogeneity. We apply the multiple testing procedures to data from a large-scale Pakistani school report card experiment, and uncover evidence of policy-relevant heterogeneous effects from information provision on child test scores. Further, our analysis reinforces the importance of preventing the inflation of false positive conclusions because 63 percent of statistically significant quantile treatment effects become insignificant once corrections for multiple testing are applied.

Online Appendix