Sustaining environments hypothesis is the idea that long-term success of early educational interventions depends on the quality of the subsequent learning environment; study finds no such relation

Bailey, Drew H., Jade M. Jenkins, and Daniela Alvarez-Vargas. (2019). Complementarities between Early Educational Intervention and Later Educational Quality? A Systematic Review of the Sustaining Environments Hypothesis. (EdWorkingPaper: 19-99). Annenberg Institute at Brown University, Sep 2019. https://doi.org/10.26300/8tz9-sh62

Abstract: The sustaining environments hypothesis refers to the popular idea, stemming from theories in developmental, cognitive, and educational psychology, that the long-term success of early educational interventions is contingent on the quality of the subsequent learning environment. Several studies have investigated whether specific kindergarten classroom and other elementary school factors account for patterns of persistence and fadeout of early educational interventions. These analyses focus on the statistical interaction between an early educational intervention – usually whether the child attended preschool – and several measures of the quality of the subsequent educational environment. The key prediction of the sustaining environments hypothesis is a positive interaction between these two variables. To quantify the strength of the evidence for such effects, we meta-analyze existing studies that have attempted to estimate interactions between preschool and later educational quality in the United States. We then attempt to establish the consistency of the direction and a plausible range of estimates of the interaction between preschool attendance and subsequent educational quality by using a specification curve analysis in a large, nationally representative dataset that has been used in several recent studies of the sustaining environments hypothesis. The meta-analysis yields small positive interaction estimates ranging from approximately .00 to .04, depending on the specification. The specification curve analyses yield interaction estimates of approximately 0. Results suggest that the current mix of methods used to test the sustaining environments hypothesis cannot reliably detect realistically sized effects. Our recommendations are to combine large sample sizes with strong causal identification strategies, and to study combinations of interventions that have a strong probability of showing large main effects.

Keywords: education, achievement, meta-analysis, persistence and fadeout, intervention


4) Heterogeneity across treatments, contexts, and children.
One possibility is that, although these interactions averaged out to approximately 0, some of them were reliably positive, consistent with complementarity between early educational intervention and later education quality, and others were reliably negative, consistent with substitutability. We find mixed evidence for this, with a statistically significant test for heterogeneity in the meta-analysis and more than 5% statistically significant estimates in the specification curve analysis, but an inferential specification curve consistent with a relatively homogenous effect of approximately 0. Importantly, this occurs despite our inclusion of a heterogeneous set of definitions of early childhood intervention and later educational quality in our analysis, methods that might reasonably be expected to increase the heterogeneity of estimates. Additionally, although the meta-analysis indicated a moderate amount of heterogeneity in interaction estimates, the prediction we thought most directly followed from the sustaining environments hypothesis – namely, that interactions would only be positive when the main effects of early and later quality were positive – was not supported. Still, perhaps the most compelling argument for heterogeneity is that it is real but not well observed in these data, because we did not measure the “right” later educational moderators of early educational intervention effects. We will discuss this possibility below.