dat.tannersmith2016.RdResults from 17 studies on the correlation between school motivation/attitudes and subsequent delinquent/criminal behavior.
dat.tannersmith2016The data frame contains the following columns:
| studyid | numeric | study identifier |
| yi | numeric | r-to-z transformed correlation coefficient |
| vi | numeric | corresponding sampling variance |
| sei | numeric | corresponding standard error |
| aget1 | numeric | age at which the school motivation/attitudes were assessed |
| aget2 | numeric | age at which the delinquent/criminal behavior was assessed |
| propmale | numeric | proportion of male participants in the sample |
| sexmix | character | whether the sample consisted only of males, only of females, or a mix |
The dataset includes 113 r-to-z transformed correlation coefficients from 17 prospective longitudinal studies that examined the relationship between school motivation/attitudes and subsequent delinquent/criminal behavior.
Multiple coefficients could be extracted from the studies “given the numerous ways in which school motivation/attitudes variables could be operationalized (e.g., academic aspirations, academic self-efficacy) as well as the numerous ways in which crime/delinquency could be operationalized (e.g., property crime, violent crime)” (Tanner-Smith et al., 2016).
Since information to compute the covariance between multiple coefficients within studies is not available, Tanner-Smith et al. (2016) illustrate the use of cluster-robust inference methods for the analysis of this dataset.
Note that this dataset is only meant to be used for pedagogical and demonstration purposes and does not constitute a proper review or synthesis of the complete and current research evidence on the given topic.
Tanner-Smith, E. E., Tipton, E. & Polanin, J. R. (2016). Handling complex meta-analytic data structures using robust variance estimates: A tutorial in R. Journal of Developmental and Life-Course Criminology, 2(1), 85–112. https://doi.org/10.1007/s40865-016-0026-5
psychology, criminology, correlation coefficients, multilevel models, cluster-robust inference, meta-regression
### copy data into 'dat' and examine data
dat <- dat.tannersmith2016
head(dat)
#> studyid yi vi sei aget1 aget2 propmale sexmix
#> 1 2 -0.06 0.001 0.027 14.5 18.0 1 male
#> 2 2 -0.05 0.001 0.027 14.5 18.0 1 male
#> 3 3 -0.05 0.001 0.028 14.5 18.0 0 female
#> 4 3 -0.05 0.001 0.028 14.5 18.0 0 female
#> 5 5 -0.01 0.006 0.080 11.5 13.5 1 male
#> 6 5 0.03 0.006 0.080 11.5 13.5 1 male
### load metafor package
library(metafor)
### compute mean age variables within studies
dat$aget1 <- ave(dat$aget1, dat$studyid)
dat$aget2 <- ave(dat$aget2, dat$studyid)
### construct an effect size identifier variable
dat$esid <- 1:nrow(dat)
### construct an approximate var-cov matrix assuming a correlation of 0.8
### for multiple coefficients arising from the same study
V <- vcalc(vi, cluster=studyid, obs=esid, rho=0.8, data=dat)
### fit a multivariate random-effects model using the approximate var-cov matrix V
res <- rma.mv(yi, V, random = ~ esid | studyid, data=dat)
res
#>
#> Multivariate Meta-Analysis Model (k = 113; method: REML)
#>
#> Variance Components:
#>
#> outer factor: studyid (nlvls = 17)
#> inner factor: esid (nlvls = 113)
#>
#> estim sqrt fixed
#> tau^2 0.0111 0.1055 no
#> rho 0.3424 no
#>
#> Test for Heterogeneity:
#> Q(df = 112) = 852.8372, p-val < .0001
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> 0.1023 0.0241 4.2433 <.0001 0.0550 0.1496 ***
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
### use cluster-robust inference methods
robust(res, cluster=studyid, clubSandwich=TRUE)
#>
#> Multivariate Meta-Analysis Model (k = 113; method: REML)
#>
#> Variance Components:
#>
#> outer factor: studyid (nlvls = 17)
#> inner factor: esid (nlvls = 113)
#>
#> estim sqrt fixed
#> tau^2 0.0111 0.1055 no
#> rho 0.3424 no
#>
#> Test for Heterogeneity:
#> Q(df = 112) = 852.8372, p-val < .0001
#>
#> Number of estimates: 113
#> Number of clusters: 17
#> Estimates per cluster: 2-12 (mean: 6.65, median: 7)
#>
#> Model Results:
#>
#> estimate se¹ tval¹ df¹ pval¹ ci.lb¹ ci.ub¹
#> 0.1023 0.0241 4.2474 14.88 0.0007 0.0509 0.1537 ***
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> 1) results based on cluster-robust inference (var-cov estimator: CR2,
#> approx t-test and confidence interval, df: Satterthwaite approx)
#>
### note: the results obtained above and below are slightly different compared
### to those given by Tanner-Smith et al. (2016) since the approach illustrated
### here makes use a multivariate random-effects model for the 'working model'
### before applying the cluster-robust inference methods, while the results given
### in the paper are based on a somewhat simpler working model
### examine the main effects of the age variables
res <- rma.mv(yi, V, mods = ~ aget1 + aget2,
random = ~ 1 | studyid/esid, data=dat)
robust(res, cluster=studyid, clubSandwich=TRUE)
#>
#> Multivariate Meta-Analysis Model (k = 113; method: REML)
#>
#> Variance Components:
#>
#> estim sqrt nlvls fixed factor
#> sigma^2.1 0.0008 0.0276 17 no studyid
#> sigma^2.2 0.0073 0.0856 113 no studyid/esid
#>
#> Test for Residual Heterogeneity:
#> QE(df = 110) = 809.9123, p-val < .0001
#>
#> Number of estimates: 113
#> Number of clusters: 17
#> Estimates per cluster: 2-12 (mean: 6.65, median: 7)
#>
#> Test of Moderators (coefficients 2:3):¹
#> F(df1 = 2, df2 = 2.8) = 1.4331, p-val = 0.3729
#>
#> Model Results:
#>
#> estimate se¹ tval¹ df¹ pval¹ ci.lb¹ ci.ub¹
#> intrcpt 0.3960 0.1632 2.4270 6.69 0.0472 0.0065 0.7855 *
#> aget1 0.0451 0.0329 1.3702 2.55 0.2788 -0.0710 0.1613
#> aget2 -0.0562 0.0328 -1.7157 1.84 0.2390 -0.2095 0.0971
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> 1) results based on cluster-robust inference (var-cov estimator: CR2,
#> approx t/F-tests and confidence intervals, df: Satterthwaite approx)
#>
### also examine their interaction
res <- rma.mv(yi, V, mods = ~ aget1 * aget2,
random = ~ 1 | studyid/esid, data=dat)
robust(res, cluster=studyid, clubSandwich=TRUE)
#>
#> Multivariate Meta-Analysis Model (k = 113; method: REML)
#>
#> Variance Components:
#>
#> estim sqrt nlvls fixed factor
#> sigma^2.1 0.0008 0.0283 17 no studyid
#> sigma^2.2 0.0073 0.0852 113 no studyid/esid
#>
#> Test for Residual Heterogeneity:
#> QE(df = 109) = 801.3245, p-val < .0001
#>
#> Number of estimates: 113
#> Number of clusters: 17
#> Estimates per cluster: 2-12 (mean: 6.65, median: 7)
#>
#> Test of Moderators (coefficients 2:4):¹
#> F(df1 = 3, df2 = 2.69) = 1.4709, p-val = 0.3918
#>
#> Model Results:
#>
#> estimate se¹ tval¹ df¹ pval¹ ci.lb¹ ci.ub¹
#> intrcpt -2.3900 1.1001 -2.1726 5.27 0.0791 -5.1746 0.3946 .
#> aget1 0.2656 0.0969 2.7406 6.39 0.0316 0.0319 0.4992 *
#> aget2 0.1027 0.0740 1.3877 6.07 0.2140 -0.0779 0.2834
#> aget1:aget2 -0.0125 0.0052 -2.3828 6.51 0.0513 -0.0251 0.0001 .
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> 1) results based on cluster-robust inference (var-cov estimator: CR2,
#> approx t/F-tests and confidence intervals, df: Satterthwaite approx)
#>
### add the sexmix factor to the model
res <- rma.mv(yi, V, mods = ~ aget1 * aget2 + sexmix,
random = ~ 1 | studyid/esid, data=dat)
robust(res, cluster=studyid, clubSandwich=TRUE)
#>
#> Multivariate Meta-Analysis Model (k = 113; method: REML)
#>
#> Variance Components:
#>
#> estim sqrt nlvls fixed factor
#> sigma^2.1 0.0011 0.0325 17 no studyid
#> sigma^2.2 0.0073 0.0852 113 no studyid/esid
#>
#> Test for Residual Heterogeneity:
#> QE(df = 107) = 797.9348, p-val < .0001
#>
#> Number of estimates: 113
#> Number of clusters: 17
#> Estimates per cluster: 2-12 (mean: 6.65, median: 7)
#>
#> Test of Moderators (coefficients 2:6):¹
#> F(df1 = 5, df2 = 2.81) = 0.9270, p-val = 0.5670
#>
#> Model Results:
#>
#> estimate se¹ tval¹ df¹ pval¹ ci.lb¹ ci.ub¹
#> intrcpt -2.5352 1.1935 -2.1241 5.2 0.0850 -5.5687 0.4983 .
#> aget1 0.2827 0.1003 2.8186 5.82 0.0315 0.0354 0.5300 *
#> aget2 0.1078 0.0793 1.3592 5.96 0.2232 -0.0866 0.3022
#> sexmixmale 0.0312 0.0620 0.5036 5.2 0.6352 -0.1264 0.1889
#> sexmixmixed -0.0071 0.0542 -0.1317 6.58 0.8991 -0.1371 0.1228
#> aget1:aget2 -0.0133 0.0053 -2.5048 6.08 0.0457 -0.0263 -0.0004 *
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> 1) results based on cluster-robust inference (var-cov estimator: CR2,
#> approx t/F-tests and confidence intervals, df: Satterthwaite approx)
#>