Studies on Differences in Planning Performance in Schizophrenia Patients versus Healthy Controls
dat.knapp2017.RdResults from 31 studies examining differences in planning performance in schizophrenia patients versus healthy controls.
dat.knapp2017Format
The data frame contains the following columns:
| author | character | study author(s) |
| year | numeric | publication year |
| study | numeric | study id number |
| task | character | type of task |
| difficulty | numeric | task difficulty |
| group1 | character | identifier for patient group within studies |
| group2 | character | identifier for control group within studies |
| comp | numeric | identifier for comparisons within studies |
| yi | numeric | standardized mean difference for planning performance |
| vi | numeric | corresponding sampling variance |
| n_sz | numeric | number of schizophrenic patients |
| n_hc | numeric | number of healthy controls |
| yi | numeric | standardized mean difference for IQ |
| vi | numeric | corresponding sampling variance |
Details
The studies included in this dataset examined differences between schizophrenia patients and healthy controls with respect to their performance on the tower of London test (https://en.wikipedia.org/wiki/Tower_of_London_test) or a similar cognitive tasks measuring planning ability. The outcome measure for this meta-analysis was the standardized mean difference (with positive values indicating better performance in the healthy controls compared to the schizophrenia patients).
The dataset has a more complex structure for several reasons:
Studies 2, 3, 9, and 20 included more than one schizophrenia patient group and the standardized mean differences were computed by comparing these groups against a single healthy control group.
Studies 6, 12, 14, 15, 18, 19, 22, and 26 had the patients and controls complete different tasks of varying complexity (essentially the average number of moves required to complete a task). Study 6 also included two different task types.
Study 24 provides two standardized mean differences, one for men and the other for women.
Study 29 provides three standardized mean differences, corresponding to the three different COMT Val158Met genotypes (val/val, val/met, and met/met).
All 4 issues described above lead to a multilevel structure in the dataset, with multiple standardized mean differences nested within some of the studies. Issues 1. and 2. also lead to correlated sampling errors.
Source
Knapp, F., Viechtbauer, W., Leonhart, R., Nitschke, K., & Kaller, C. P. (2017). Planning performance in schizophrenia patients: A meta-analysis of the influence of task difficulty and clinical and sociodemographic variables. Psychological Medicine, 47(11), 2002–2016. https://doi.org/10.1017/S0033291717000459
Concepts
psychology, standardized mean differences, multilevel models, multivariate models, cluster-robust inference, meta-regression
Examples
### copy data into 'dat' and examine data
dat <- dat.knapp2017
dat[-c(1:2)]
#> study task difficulty group1 group2 comp yi vi n_sz n_hc yi_iq vi_iq
#> 1 1 SOC (CANTAB) 3.83 SZ HC 1 0.62 0.075 24 33 0.72 0.076
#> 2 2 SOC (CANTAB) 3.83 SZ1 HC 1 0.46 0.047 44 44 NA NA
#> 3 2 SOC (CANTAB) 3.83 SZ2 HC 2 0.73 0.052 38 44 NA NA
#> 4 3 SOC (CANTAB) 3.83 SZ1 HC 1 1.18 0.062 39 37 NA NA
#> 5 3 SOC (CANTAB) 3.83 SZ2 HC 2 1.09 0.073 27 37 NA NA
#> 6 3 SOC (CANTAB) 3.83 SZ3 HC 3 0.91 0.102 15 37 NA NA
#> 7 3 SOC (CANTAB) 3.83 SZ4 HC 4 0.91 0.084 20 37 NA NA
#> 8 4 TOH NA SZ HC 1 0.57 0.074 28 28 NA NA
#> 9 5 TOL NA SZ HC 1 0.40 0.180 13 10 NA NA
#> 10 6 SOC (CANTAB) 2.00 SZ HC 1 0.43 0.170 12 12 0.12 0.167
#> 11 6 SOC (CANTAB) 3.00 SZ HC 2 1.37 0.206 12 12 0.12 0.167
#> 12 6 SOC (CANTAB) 4.00 SZ HC 3 0.23 0.168 12 12 0.12 0.167
#> 13 6 SOC (CANTAB) 5.00 SZ HC 4 0.97 0.186 12 12 0.12 0.167
#> 14 6 SOC (one-touch) 2.00 SZ HC 5 0.56 0.173 12 12 0.12 0.167
#> 15 6 SOC (one-touch) 3.00 SZ HC 6 0.79 0.180 12 12 0.12 0.167
#> 16 6 SOC (one-touch) 4.00 SZ HC 7 1.19 0.196 12 12 0.12 0.167
#> 17 6 SOC (one-touch) 5.00 SZ HC 8 0.92 0.184 12 12 0.12 0.167
#> 18 7 SOC 3.50 SZ HC 1 0.20 0.096 22 20 0.62 0.100
#> 19 8 TOH NA SZ HC 1 1.42 0.180 13 15 NA NA
#> 20 9 TOL 4.00 SZ1 HC 1 0.35 0.074 27 28 1.21 0.086
#> 21 9 TOL 4.00 SZ2 HC 2 0.88 0.078 28 28 1.26 0.086
#> 22 10 SOC (CANTAB) 3.83 SZ HC 1 1.08 0.079 26 33 0.76 0.074
#> 23 11 SOC (CANTAB) 3.83 SZ HC 1 0.94 0.111 20 20 0.42 0.102
#> 24 12 SOC (CANTAB) 2.00 SZ HC 1 0.44 0.020 135 81 0.49 0.020
#> 25 12 SOC (CANTAB) 3.00 SZ HC 2 0.63 0.021 135 81 0.49 0.020
#> 26 12 SOC (CANTAB) 4.00 SZ HC 3 0.38 0.020 135 81 0.49 0.020
#> 27 12 SOC (CANTAB) 5.00 SZ HC 4 0.93 0.022 135 81 0.49 0.020
#> 28 13 TOL NA SZ HC 1 0.33 0.102 24 17 NA NA
#> 29 14 TOL 2.00 SZ HC 1 0.00 0.073 32 24 NA NA
#> 30 14 TOL 3.00 SZ HC 2 -0.06 0.073 32 24 NA NA
#> 31 14 TOL 4.00 SZ HC 3 0.73 0.078 32 24 NA NA
#> 32 14 TOL 5.00 SZ HC 4 0.48 0.075 32 24 NA NA
#> 33 15 TOL 2.00 SZ HC 1 0.43 0.092 25 20 NA NA
#> 34 15 TOL 3.00 SZ HC 2 0.72 0.096 25 20 NA NA
#> 35 15 TOL 4.00 SZ HC 3 0.80 0.097 25 20 NA NA
#> 36 15 TOL 5.00 SZ HC 4 1.30 0.109 25 20 NA NA
#> 37 16 TOL 4.67 SZ HC 1 1.84 0.190 15 15 NA NA
#> 38 17 TOL 3.50 SZ HC 1 2.58 0.216 17 17 0.58 0.123
#> 39 18 TOL 3.00 SZ HC 1 0.25 0.071 30 27 0.18 0.071
#> 40 18 TOL 4.00 SZ HC 2 0.86 0.077 30 27 0.18 0.071
#> 41 18 TOL 5.00 SZ HC 3 0.68 0.074 30 27 0.18 0.071
#> 42 19 SOC (CANTAB) 2.00 SZ HC 1 0.21 0.060 36 31 0.48 0.062
#> 43 19 SOC (CANTAB) 3.00 SZ HC 2 0.65 0.063 36 31 0.48 0.062
#> 44 19 SOC (CANTAB) 4.00 SZ HC 3 0.30 0.061 36 31 0.48 0.062
#> 45 19 SOC (CANTAB) 5.00 SZ HC 4 0.45 0.062 36 31 0.48 0.062
#> 46 20 BACS-J NA SZ1 HC 1 1.03 0.054 20 340 NA NA
#> 47 20 BACS-J NA SZ2 HC 2 0.92 0.104 10 340 NA NA
#> 48 21 SOC (CANTAB) 3.83 SZ HC 1 1.12 0.108 28 17 NA NA
#> 49 22 TOL 2.00 SZ HC 1 0.52 0.052 40 40 NA NA
#> 50 22 TOL 3.00 SZ HC 2 0.50 0.052 40 40 NA NA
#> 51 22 TOL 4.00 SZ HC 3 0.49 0.051 40 40 NA NA
#> 52 23 SOC 3.83 SZ HC 1 0.30 0.042 48 48 NA NA
#> 53 24 TOL-Drexel 5.50 SZ1 HC1 1 0.36 0.022 86 97 0.47 0.023
#> 54 24 TOL-Drexel 5.50 SZ2 HC2 2 0.41 0.030 75 62 0.10 0.030
#> 55 25 SOC 3.83 SZ HC 1 0.79 0.086 25 25 NA NA
#> 56 26 SOC 2.00 SZ HC 1 0.23 0.031 77 55 NA NA
#> 57 26 SOC 3.00 SZ HC 2 0.58 0.032 77 55 NA NA
#> 58 26 SOC 4.00 SZ HC 3 0.57 0.032 77 55 NA NA
#> 59 26 SOC 5.00 SZ HC 4 0.79 0.034 77 55 NA NA
#> 60 27 TOL-Drexel 5.50 SZ HC 1 0.70 0.071 30 30 NA NA
#> 61 28 SOC 3.83 SZ HC 1 1.16 0.136 20 15 0.23 0.117
#> 62 29 BACS-J NA SZ1 HC1 1 1.29 0.039 56 68 0.37 0.033
#> 63 29 BACS-J NA SZ2 HC2 2 0.63 0.039 47 62 0.43 0.038
#> 64 29 BACS-J NA SZ3 HC3 3 0.31 0.121 15 19 1.00 0.134
#> 65 30 TOL 3.83 SZ HC 1 0.11 0.070 30 27 NA NA
#> 66 31 TOL (four-disk) 5.60 SZ HC 1 0.87 0.036 60 60 NA NA
### load metafor package
library(metafor)
### fit a standard random-effects model ignoring the issues described above
res <- rma(yi, vi, data=dat)
res
#>
#> Random-Effects Model (k = 66; tau^2 estimator: REML)
#>
#> tau^2 (estimated amount of total heterogeneity): 0.0551 (SE = 0.0212)
#> tau (square root of estimated tau^2 value): 0.2348
#> I^2 (total heterogeneity / total variability): 47.74%
#> H^2 (total variability / sampling variability): 1.91
#>
#> Test for Heterogeneity:
#> Q(df = 65) = 129.0088, p-val < .0001
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> 0.6557 0.0439 14.9521 <.0001 0.5698 0.7417 ***
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
### fit a multilevel model with random effects for studies and comparisons within studies
### (but this ignored the correlation in the sampling errors)
res <- rma.mv(yi, vi, random = ~ 1 | study/comp, data=dat)
res
#>
#> Multivariate Meta-Analysis Model (k = 66; method: REML)
#>
#> Variance Components:
#>
#> estim sqrt nlvls fixed factor
#> sigma^2.1 0.0383 0.1958 31 no study
#> sigma^2.2 0.0263 0.1621 66 no study/comp
#>
#> Test for Heterogeneity:
#> Q(df = 65) = 129.0088, p-val < .0001
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> 0.6801 0.0560 12.1513 <.0001 0.5704 0.7898 ***
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
### create variable that indicates the task and difficulty combination as increasing integers
dat$task.diff <- unlist(lapply(split(dat, dat$study), function(x) {
task.int <- as.integer(factor(x$task))
diff.int <- as.integer(factor(x$difficulty))
diff.int[is.na(diff.int)] <- 1
paste0(task.int, ".", diff.int)}))
### construct correlation matrix for two tasks with four different difficulties where the
### correlation is 0.4 for different difficulties of the same task, 0.7 for the same
### difficulty of different tasks, and 0.28 for different difficulties of different tasks
R <- matrix(0.4, nrow=8, ncol=8)
R[5:8,1:4] <- R[1:4,5:8] <- 0.28
diag(R[1:4,5:8]) <- 0.7
diag(R[5:8,1:4]) <- 0.7
diag(R) <- 1
rownames(R) <- colnames(R) <- paste0(rep(1:2, each=4), ".", 1:4)
R
#> 1.1 1.2 1.3 1.4 2.1 2.2 2.3 2.4
#> 1.1 1.00 0.40 0.40 0.40 0.70 0.28 0.28 0.28
#> 1.2 0.40 1.00 0.40 0.40 0.28 0.70 0.28 0.28
#> 1.3 0.40 0.40 1.00 0.40 0.28 0.28 0.70 0.28
#> 1.4 0.40 0.40 0.40 1.00 0.28 0.28 0.28 0.70
#> 2.1 0.70 0.28 0.28 0.28 1.00 0.40 0.40 0.40
#> 2.2 0.28 0.70 0.28 0.28 0.40 1.00 0.40 0.40
#> 2.3 0.28 0.28 0.70 0.28 0.40 0.40 1.00 0.40
#> 2.4 0.28 0.28 0.28 0.70 0.40 0.40 0.40 1.00
### construct an approximate V matrix accounting for the use of shared groups and
### for correlations among tasks/difficulties as specified in the R matrix above
V <- vcalc(vi, cluster=study, grp1=group1, grp2=group2, w1=n_sz, w2=n_hc,
obs=task.diff, rho=R, data=dat)
### correlation matrix for study 3 with four patient groups and a single control group
round(cov2cor(V[dat$study == 3, dat$study == 3]), 2)
#> [,1] [,2] [,3] [,4]
#> [1,] 1.00 0.47 0.38 0.42
#> [2,] 0.47 1.00 0.35 0.38
#> [3,] 0.38 0.35 1.00 0.32
#> [4,] 0.42 0.38 0.32 1.00
### correlation matrix for study 6 with two tasks with four difficulties
cov2cor(V[dat$study == 6, dat$study == 6])
#> [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]
#> [1,] 1.00 0.40 0.40 0.40 0.70 0.28 0.28 0.28
#> [2,] 0.40 1.00 0.40 0.40 0.28 0.70 0.28 0.28
#> [3,] 0.40 0.40 1.00 0.40 0.28 0.28 0.70 0.28
#> [4,] 0.40 0.40 0.40 1.00 0.28 0.28 0.28 0.70
#> [5,] 0.70 0.28 0.28 0.28 1.00 0.40 0.40 0.40
#> [6,] 0.28 0.70 0.28 0.28 0.40 1.00 0.40 0.40
#> [7,] 0.28 0.28 0.70 0.28 0.40 0.40 1.00 0.40
#> [8,] 0.28 0.28 0.28 0.70 0.40 0.40 0.40 1.00
### correlation matrix for study 24 with two independent groups
cov2cor(V[dat$study == 24, dat$study == 24])
#> [,1] [,2]
#> [1,] 1 0
#> [2,] 0 1
### correlation matrix for study 29 with three independent groups
cov2cor(V[dat$study == 29, dat$study == 29])
#> [,1] [,2] [,3]
#> [1,] 1 0 0
#> [2,] 0 1 0
#> [3,] 0 0 1
### fit multilevel model as above, but now use this V matrix in the model
res <- rma.mv(yi, V, random = ~ 1 | study/comp, data=dat)
res
#>
#> Multivariate Meta-Analysis Model (k = 66; method: REML)
#>
#> Variance Components:
#>
#> estim sqrt nlvls fixed factor
#> sigma^2.1 0.0102 0.1009 31 no study
#> sigma^2.2 0.0557 0.2360 66 no study/comp
#>
#> Test for Heterogeneity:
#> Q(df = 65) = 152.4935, p-val < .0001
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> 0.6696 0.0546 12.2735 <.0001 0.5626 0.7765 ***
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
predict(res, digits=2)
#>
#> pred se ci.lb ci.ub pi.lb pi.ub
#> 0.67 0.05 0.56 0.78 0.16 1.18
#>
### use cluster-robust inference methods based on this model
robust(res, cluster=study)
#>
#> Multivariate Meta-Analysis Model (k = 66; method: REML)
#>
#> Variance Components:
#>
#> estim sqrt nlvls fixed factor
#> sigma^2.1 0.0102 0.1009 31 no study
#> sigma^2.2 0.0557 0.2360 66 no study/comp
#>
#> Test for Heterogeneity:
#> Q(df = 65) = 152.4935, p-val < .0001
#>
#> Number of estimates: 66
#> Number of clusters: 31
#> Estimates per cluster: 1-8 (mean: 2.13, median: 1)
#>
#> Model Results:
#>
#> estimate se¹ tval¹ df¹ pval¹ ci.lb¹ ci.ub¹
#> 0.6696 0.0543 12.3418 30 <.0001 0.5588 0.7804 ***
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> 1) results based on cluster-robust inference (var-cov estimator: CR1,
#> approx t-test and confidence interval, df: residual method)
#>
### use methods from the clubSandwich package
robust(res, cluster=study, clubSandwich=TRUE)
#>
#> Multivariate Meta-Analysis Model (k = 66; method: REML)
#>
#> Variance Components:
#>
#> estim sqrt nlvls fixed factor
#> sigma^2.1 0.0102 0.1009 31 no study
#> sigma^2.2 0.0557 0.2360 66 no study/comp
#>
#> Test for Heterogeneity:
#> Q(df = 65) = 152.4935, p-val < .0001
#>
#> Number of estimates: 66
#> Number of clusters: 31
#> Estimates per cluster: 1-8 (mean: 2.13, median: 1)
#>
#> Model Results:
#>
#> estimate se¹ tval¹ df¹ pval¹ ci.lb¹ ci.ub¹
#> 0.6696 0.0543 12.3260 22.3 <.0001 0.5570 0.7821 ***
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> 1) results based on cluster-robust inference (var-cov estimator: CR2,
#> approx t-test and confidence interval, df: Satterthwaite approx)
#>
### examine if task difficulty is a potential moderator of the effect
res <- rma.mv(yi, V, mods = ~ difficulty, random = ~ 1 | study/comp, data=dat)
#> Warning: 9 rows with NAs omitted from model fitting.
res
#>
#> Multivariate Meta-Analysis Model (k = 57; method: REML)
#>
#> Variance Components:
#>
#> estim sqrt nlvls fixed factor
#> sigma^2.1 0.0465 0.2156 25 no study
#> sigma^2.2 0.0275 0.1659 57 no study/comp
#>
#> Test for Residual Heterogeneity:
#> QE(df = 55) = 115.9211, p-val < .0001
#>
#> Test of Moderators (coefficient 2):
#> QM(df = 1) = 10.0201, p-val = 0.0015
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> intrcpt 0.1803 0.1682 1.0717 0.2838 -0.1494 0.5100
#> difficulty 0.1241 0.0392 3.1655 0.0015 0.0472 0.2009 **
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
sav <- robust(res, cluster=study)
sav
#>
#> Multivariate Meta-Analysis Model (k = 57; method: REML)
#>
#> Variance Components:
#>
#> estim sqrt nlvls fixed factor
#> sigma^2.1 0.0465 0.2156 25 no study
#> sigma^2.2 0.0275 0.1659 57 no study/comp
#>
#> Test for Residual Heterogeneity:
#> QE(df = 55) = 115.9211, p-val < .0001
#>
#> Number of estimates: 57
#> Number of clusters: 25
#> Estimates per cluster: 1-8 (mean: 2.28, median: 1)
#>
#> Test of Moderators (coefficient 2):¹
#> F(df1 = 1, df2 = 23) = 17.7635, p-val = 0.0003
#>
#> Model Results:
#>
#> estimate se¹ tval¹ df¹ pval¹ ci.lb¹ ci.ub¹
#> intrcpt 0.1803 0.1239 1.4547 23 0.1593 -0.0761 0.4367
#> difficulty 0.1241 0.0294 4.2147 23 0.0003 0.0632 0.1850 ***
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> 1) results based on cluster-robust inference (var-cov estimator: CR1,
#> approx t/F-tests and confidence intervals, df: residual method)
#>
sav <- robust(res, cluster=study, clubSandwich=TRUE)
sav
#>
#> Multivariate Meta-Analysis Model (k = 57; method: REML)
#>
#> Variance Components:
#>
#> estim sqrt nlvls fixed factor
#> sigma^2.1 0.0465 0.2156 25 no study
#> sigma^2.2 0.0275 0.1659 57 no study/comp
#>
#> Test for Residual Heterogeneity:
#> QE(df = 55) = 115.9211, p-val < .0001
#>
#> Number of estimates: 57
#> Number of clusters: 25
#> Estimates per cluster: 1-8 (mean: 2.28, median: 1)
#>
#> Test of Moderators (coefficient 2):¹
#> F(df1 = 1, df2 = 7.39) = 17.2483, p-val = 0.0038
#>
#> Model Results:
#>
#> estimate se¹ tval¹ df¹ pval¹ ci.lb¹ ci.ub¹
#> intrcpt 0.1803 0.1244 1.4499 7.77 0.1862 -0.1079 0.4685
#> difficulty 0.1241 0.0299 4.1531 7.39 0.0038 0.0542 0.1940 **
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
#> 1) results based on cluster-robust inference (var-cov estimator: CR2,
#> approx t/F-tests and confidence intervals, df: Satterthwaite approx)
#>
### draw bubble plot
regplot(sav, xlab="Task Difficulty", ylab="Standardized Mean Difference", las=1, digits=1, bty="l")
