Multiple Testing and the Distributional Effects of Accountability Incentives in Education

Lehrer, Steven F., R. Vincent Pohl, and Kyungchul Song. 2019. “Multiple Testing and the Distributional Effects of Accountability Incentives in Education.” Ruhr Economic Papers #799.

Download here

Abstract: Economic theory that underlies many empirical microeconomic applications predicts that treatment responses depend on individuals’ characteristics and location on the outcome distribution. Using data from a large-scale Pakistani school report card experiment, we consider tests for treatment effect heterogeneity that make corrections for multiple testing to avoid an overestimation of positive treatment effects. These tests uncover evidence of policy-relevant heterogeneous effects from information provision on child test scores. Further, our analysis reinforces the importance of preventing the inflation of false positive conclusions since over 75% of statistically significant quantile treatment effects become insignificant once corrections for multiple testing are applied.

Online Appendix