Studies on the Effects of Modified School Calendars on Student Achievement
dat.konstantopoulos2011.RdResults from 56 studies on the effects of modified school calendars on student achievement.
dat.konstantopoulos2011Format
The data frame contains the following columns:
| district | numeric | district id number |
| school | numeric | school id number (within district) |
| study | numeric | study id number |
| yi | numeric | standardized mean difference |
| vi | numeric | corresponding sampling variance |
| year | numeric | year of the study |
Details
Instead of following the more traditional school calendar with a long summer break (in addition to a short winter and spring break), some schools have switched to a modified school calendar comprising more frequent but shorter intermittent breaks (e.g., 9 weeks of school followed by 3 weeks off), while keeping the total number of days at school approximately the same. The effects of using such a modified calendar on student achievement have been examined in a number of studies and were meta-analyzed by Cooper et al. (2003).
The dataset (taken from Konstantopoulos, 2011) contains the results from 56 studies, each comparing the level of academic achievement in a group of students following a modified school calendar with that of a group of students following a more traditional school calendar. The difference between the two groups was quantified in terms of a standardized mean difference (with positive values indicating a higher mean level of achievement in the group following the modified school calendar).
The studies were conducted at various schools that were clustered within districts. The data therefore have a multilevel structure, with schools nested within districts. A multilevel meta-analysis of these data can be used to estimate and account for the amount of heterogeneity between districts and between schools within districts.
Source
Konstantopoulos, S. (2011). Fixed effects and variance components estimation in three-level meta-analysis. Research Synthesis Methods, 2(1), 61–76. https://doi.org/10.1002/jrsm.35
References
Cooper, H., Valentine, J. C., Charlton, K., & Melson, A. (2003). The effects of modified school calendars on student achievement and on school and community attitudes. Review of Educational Research, 73(1), 1–52. https://doi.org/10.3102/00346543073001001
Concepts
education, standardized mean differences, multilevel models
Examples
### copy data into 'dat' and examine data
dat <- dat.konstantopoulos2011
dat
#>
#> district school study year yi vi
#> 1 11 1 1 1976 -0.180 0.118
#> 2 11 2 2 1976 -0.220 0.118
#> 3 11 3 3 1976 0.230 0.144
#> 4 11 4 4 1976 -0.300 0.144
#> 5 12 1 5 1989 0.130 0.014
#> 6 12 2 6 1989 -0.260 0.014
#> 7 12 3 7 1989 0.190 0.015
#> 8 12 4 8 1989 0.320 0.024
#> 9 18 1 9 1994 0.450 0.023
#> 10 18 2 10 1994 0.380 0.043
#> 11 18 3 11 1994 0.290 0.012
#> 12 27 1 12 1976 0.160 0.020
#> 13 27 2 13 1976 0.650 0.004
#> 14 27 3 14 1976 0.360 0.004
#> 15 27 4 15 1976 0.600 0.007
#> 16 56 1 16 1997 0.080 0.019
#> 17 56 2 17 1997 0.040 0.007
#> 18 56 3 18 1997 0.190 0.005
#> 19 56 4 19 1997 -0.060 0.004
#> 20 58 1 20 1976 -0.180 0.020
#> 21 58 2 21 1976 0.000 0.018
#> 22 58 3 22 1976 0.000 0.019
#> 23 58 4 23 1976 -0.280 0.022
#> 24 58 5 24 1976 -0.040 0.020
#> 25 58 6 25 1976 -0.300 0.021
#> 26 58 7 26 1976 0.070 0.006
#> 27 58 8 27 1976 0.000 0.007
#> 28 58 9 28 1976 0.050 0.007
#> 29 58 10 29 1976 -0.080 0.007
#> 30 58 11 30 1976 -0.090 0.007
#> 31 71 1 31 1997 0.300 0.015
#> 32 71 2 32 1997 0.980 0.011
#> 33 71 3 33 1997 1.190 0.010
#> 34 86 1 34 1997 -0.070 0.001
#> 35 86 2 35 1997 -0.050 0.001
#> 36 86 3 36 1997 -0.010 0.001
#> 37 86 4 37 1997 0.020 0.001
#> 38 86 5 38 1997 -0.030 0.001
#> 39 86 6 39 1997 0.000 0.001
#> 40 86 7 40 1997 0.010 0.001
#> 41 86 8 41 1997 -0.100 0.001
#> 42 91 1 42 2000 0.500 0.010
#> 43 91 2 43 2000 0.660 0.011
#> 44 91 3 44 2000 0.200 0.010
#> 45 91 4 45 2000 0.000 0.009
#> 46 91 5 46 2000 0.050 0.013
#> 47 91 6 47 2000 0.070 0.013
#> 48 108 1 48 2000 -0.520 0.031
#> 49 108 2 49 2000 0.700 0.031
#> 50 108 3 50 2000 -0.030 0.030
#> 51 108 4 51 2000 0.270 0.030
#> 52 108 5 52 2000 -0.340 0.030
#> 53 644 1 53 1995 0.120 0.087
#> 54 644 2 54 1995 0.610 0.082
#> 55 644 3 55 1994 0.040 0.067
#> 56 644 4 56 1994 -0.050 0.067
#>
### load metafor package
library(metafor)
### regular random-effects model
res <- rma(yi, vi, data=dat)
print(res, digits=3)
#>
#> Random-Effects Model (k = 56; tau^2 estimator: REML)
#>
#> tau^2 (estimated amount of total heterogeneity): 0.088 (SE = 0.020)
#> tau (square root of estimated tau^2 value): 0.297
#> I^2 (total heterogeneity / total variability): 94.70%
#> H^2 (total variability / sampling variability): 18.89
#>
#> Test for Heterogeneity:
#> Q(df = 55) = 578.864, p-val < .001
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> 0.128 0.044 2.916 0.004 0.042 0.214 **
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
### regular random-effects model using rma.mv()
res <- rma.mv(yi, vi, random = ~ 1 | study, data=dat)
print(res, digits=3)
#>
#> Multivariate Meta-Analysis Model (k = 56; method: REML)
#>
#> Variance Components:
#>
#> estim sqrt nlvls fixed factor
#> sigma^2 0.088 0.297 56 no study
#>
#> Test for Heterogeneity:
#> Q(df = 55) = 578.864, p-val < .001
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> 0.128 0.044 2.916 0.004 0.042 0.214 **
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
### multilevel random-effects model
res.ml <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat)
print(res.ml, digits=3)
#>
#> Multivariate Meta-Analysis Model (k = 56; method: REML)
#>
#> Variance Components:
#>
#> estim sqrt nlvls fixed factor
#> sigma^2.1 0.065 0.255 11 no district
#> sigma^2.2 0.033 0.181 56 no district/school
#>
#> Test for Heterogeneity:
#> Q(df = 55) = 578.864, p-val < .001
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> 0.185 0.085 2.185 0.029 0.019 0.350 *
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
### profile variance components
profile(res.ml, progbar=FALSE)
### multivariate parameterization of the model
res.mv <- rma.mv(yi, vi, random = ~ school | district, data=dat)
print(res.mv, digits=3)
#>
#> Multivariate Meta-Analysis Model (k = 56; method: REML)
#>
#> Variance Components:
#>
#> outer factor: district (nlvls = 11)
#> inner factor: school (nlvls = 11)
#>
#> estim sqrt fixed
#> tau^2 0.098 0.313 no
#> rho 0.665 no
#>
#> Test for Heterogeneity:
#> Q(df = 55) = 578.864, p-val < .001
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> 0.185 0.085 2.185 0.029 0.019 0.350 *
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
### tau^2 from multivariate model = sum of the two variance components from the multilevel model
round(sum(res.ml$sigma2), 3)
#> [1] 0.098
### rho from multivariate model = intraclass correlation coefficient based on the multilevel model
round(res.ml$sigma2[1] / sum(res.ml$sigma2), 3)
#> [1] 0.665