Results from 56 studies on the effects of modified school calendars on student achievement.

dat.konstantopoulos2011

Format

The data frame contains the following columns:

districtnumericdistrict id number
schoolnumericschool id number (within district)
studynumericstudy id number
yinumericstandardized mean difference
vinumericcorresponding sampling variance
yearnumericyear of the study

Details

Instead of following the more traditional school calendar with a long summer break (in addition to a short winter and spring break), some schools have switched to a modified school calendar comprising more frequent but shorter intermittent breaks (e.g., 9 weeks of school followed by 3 weeks off), while keeping the total number of days at school approximately the same. The effects of using such a modified calendar on student achievement have been examined in a number of studies and were meta-analyzed by Cooper et al. (2003).

The dataset (taken from Konstantopoulos, 2011) contains the results from 56 studies, each comparing the level of academic achievement in a group of students following a modified school calendar with that of a group of students following a more traditional school calendar. The difference between the two groups was quantified in terms of a standardized mean difference (with positive values indicating a higher mean level of achievement in the group following the modified school calendar).

The studies were conducted at various schools that were clustered within districts. The data therefore have a multilevel structure, with schools nested within districts. A multilevel meta-analysis of these data can be used to estimate and account for the amount of heterogeneity between districts and between schools within districts.

Source

Konstantopoulos, S. (2011). Fixed effects and variance components estimation in three-level meta-analysis. Research Synthesis Methods, 2(1), 61--76. https://doi.org/10.1002/jrsm.35

References

Cooper, H., Valentine, J. C., Charlton, K., & Melson, A. (2003). The effects of modified school calendars on student achievement and on school and community attitudes. Review of Educational Research, 73(1), 1--52. https://doi.org/10.3102/00346543073001001

Examples

### copy data into 'dat' and examine data dat <- dat.konstantopoulos2011 dat
#> district school study year yi vi #> 1 11 1 1 1976 -0.180 0.118 #> 2 11 2 2 1976 -0.220 0.118 #> 3 11 3 3 1976 0.230 0.144 #> 4 11 4 4 1976 -0.300 0.144 #> 5 12 1 5 1989 0.130 0.014 #> 6 12 2 6 1989 -0.260 0.014 #> 7 12 3 7 1989 0.190 0.015 #> 8 12 4 8 1989 0.320 0.024 #> 9 18 1 9 1994 0.450 0.023 #> 10 18 2 10 1994 0.380 0.043 #> 11 18 3 11 1994 0.290 0.012 #> 12 27 1 12 1976 0.160 0.020 #> 13 27 2 13 1976 0.650 0.004 #> 14 27 3 14 1976 0.360 0.004 #> 15 27 4 15 1976 0.600 0.007 #> 16 56 1 16 1997 0.080 0.019 #> 17 56 2 17 1997 0.040 0.007 #> 18 56 3 18 1997 0.190 0.005 #> 19 56 4 19 1997 -0.060 0.004 #> 20 58 1 20 1976 -0.180 0.020 #> 21 58 2 21 1976 0.000 0.018 #> 22 58 3 22 1976 0.000 0.019 #> 23 58 4 23 1976 -0.280 0.022 #> 24 58 5 24 1976 -0.040 0.020 #> 25 58 6 25 1976 -0.300 0.021 #> 26 58 7 26 1976 0.070 0.006 #> 27 58 8 27 1976 0.000 0.007 #> 28 58 9 28 1976 0.050 0.007 #> 29 58 10 29 1976 -0.080 0.007 #> 30 58 11 30 1976 -0.090 0.007 #> 31 71 1 31 1997 0.300 0.015 #> 32 71 2 32 1997 0.980 0.011 #> 33 71 3 33 1997 1.190 0.010 #> 34 86 1 34 1997 -0.070 0.001 #> 35 86 2 35 1997 -0.050 0.001 #> 36 86 3 36 1997 -0.010 0.001 #> 37 86 4 37 1997 0.020 0.001 #> 38 86 5 38 1997 -0.030 0.001 #> 39 86 6 39 1997 0.000 0.001 #> 40 86 7 40 1997 0.010 0.001 #> 41 86 8 41 1997 -0.100 0.001 #> 42 91 1 42 2000 0.500 0.010 #> 43 91 2 43 2000 0.660 0.011 #> 44 91 3 44 2000 0.200 0.010 #> 45 91 4 45 2000 0.000 0.009 #> 46 91 5 46 2000 0.050 0.013 #> 47 91 6 47 2000 0.070 0.013 #> 48 108 1 48 2000 -0.520 0.031 #> 49 108 2 49 2000 0.700 0.031 #> 50 108 3 50 2000 -0.030 0.030 #> 51 108 4 51 2000 0.270 0.030 #> 52 108 5 52 2000 -0.340 0.030 #> 53 644 1 53 1995 0.120 0.087 #> 54 644 2 54 1995 0.610 0.082 #> 55 644 3 55 1994 0.040 0.067 #> 56 644 4 56 1994 -0.050 0.067
### regular random-effects model res <- rma(yi, vi, data=dat) print(res, digits=3)
#> #> Random-Effects Model (k = 56; tau^2 estimator: REML) #> #> tau^2 (estimated amount of total heterogeneity): 0.088 (SE = 0.020) #> tau (square root of estimated tau^2 value): 0.297 #> I^2 (total heterogeneity / total variability): 94.70% #> H^2 (total variability / sampling variability): 18.89 #> #> Test for Heterogeneity: #> Q(df = 55) = 578.864, p-val < .001 #> #> Model Results: #> #> estimate se zval pval ci.lb ci.ub #> 0.128 0.044 2.916 0.004 0.042 0.214 ** #> #> --- #> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 #>
### regular random-effects model using rma.mv() res <- rma.mv(yi, vi, random = ~ 1 | study, data=dat) print(res, digits=3)
#> #> Multivariate Meta-Analysis Model (k = 56; method: REML) #> #> Variance Components: #> #> estim sqrt nlvls fixed factor #> sigma^2 0.088 0.297 56 no study #> #> Test for Heterogeneity: #> Q(df = 55) = 578.864, p-val < .001 #> #> Model Results: #> #> estimate se zval pval ci.lb ci.ub #> 0.128 0.044 2.916 0.004 0.042 0.214 ** #> #> --- #> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 #>
### multilevel random-effects model res.ml <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat) print(res.ml, digits=3)
#> #> Multivariate Meta-Analysis Model (k = 56; method: REML) #> #> Variance Components: #> #> estim sqrt nlvls fixed factor #> sigma^2.1 0.065 0.255 11 no district #> sigma^2.2 0.033 0.181 56 no district/school #> #> Test for Heterogeneity: #> Q(df = 55) = 578.864, p-val < .001 #> #> Model Results: #> #> estimate se zval pval ci.lb ci.ub #> 0.185 0.085 2.185 0.029 0.019 0.350 * #> #> --- #> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 #>
### profile variance components profile(res.ml, progbar=FALSE)
### multivariate parameterization of the model res.mv <- rma.mv(yi, vi, random = ~ factor(school) | district, data=dat) print(res.mv, digits=3)
#> #> Multivariate Meta-Analysis Model (k = 56; method: REML) #> #> Variance Components: #> #> outer factor: district (nlvls = 11) #> inner factor: factor(school) (nlvls = 11) #> #> estim sqrt fixed #> tau^2 0.098 0.313 no #> rho 0.665 no #> #> Test for Heterogeneity: #> Q(df = 55) = 578.864, p-val < .001 #> #> Model Results: #> #> estimate se zval pval ci.lb ci.ub #> 0.185 0.085 2.185 0.029 0.019 0.350 * #> #> --- #> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 #>
### tau^2 from multivariate model = sum of the two variance components from the multilevel model round(sum(res.ml$sigma2), 3)
#> [1] 0.098
### rho from multivariate model = intraclass correlation coefficient based on the multilevel model round(res.ml$sigma2[1] / sum(res.ml$sigma2), 3)
#> [1] 0.665