Functions to carry out a ‘leave-one-out analysis’, by repeatedly fitting the specified model leaving out one study (or cluster level) at a time.

leave1out(x, ...)

# S3 method for class 'rma.uni'
leave1out(x, cluster, digits, transf, targs, progbar=FALSE, ...)
# S3 method for class 'rma.mh'
leave1out(x, cluster, digits, transf, targs, progbar=FALSE, ...)
# S3 method for class 'rma.peto'
leave1out(x, cluster, digits, transf, targs, progbar=FALSE, ...)

Arguments

x

an object of class "rma.uni", "rma.mh", or "rma.peto".

cluster

optional vector to specify a clustering variable.

digits

optional integer to specify the number of decimal places to which the printed results should be rounded. If unspecified, the default is to take the value from the object.

transf

optional argument to specify a function to transform the model coefficients and interval bounds (e.g., transf=exp; see also transf). If unspecified, no transformation is used.

targs

optional arguments needed by the function specified under transf.

progbar

logical to specify whether a progress bar should be shown (the default is FALSE).

...

other arguments.

Details

In a leave-one-out analysis, the same model is repeatedly fitted, leaving out one study at a time. By doing so, we can assess how much the results are influenced by each individual study. It is also possible to specify a cluster variable, in which case each cluster level is left out in turn.

Note that for "rma.uni" objects, the model specified via x must be a model without moderators (i.e., either an equal- or a random-effects model).

Value

An object of class "list.rma". The object is a list containing the following components:

estimate

estimated (average) outcomes.

se

corresponding standard errors.

zval

corresponding test statistics.

pval

corresponding p-values.

ci.lb

lower bounds of the confidence intervals.

ci.ub

upper bounds of the confidence intervals.

Q

test statistics for the test of heterogeneity.

Qp

corresponding p-values.

tau2

estimated amount of heterogeneity (only for random-effects models).

I2

values of \(I^2\).

H2

values of \(H^2\).

When the model was fitted with test="t", test="knha", test="hksj", or test="adhoc", then zval is called tval in the object that is returned by the function.

The object is formatted and printed with the print function. To format the results as a data frame, one can use the as.data.frame function.

Note

When using the transf option, the transformation is applied to the estimated coefficients and the corresponding interval bounds. The standard errors are then set equal to NA and are omitted from the printed output.

The variable specified via cluster is assumed to be of the same length as the data originally passed to the model fitting function (and if the data argument was used in the original model fit, then the variable will be searched for within this data frame first). Any subsetting and removal of studies with missing values that was applied during the model fitting is also automatically applied to the variable specified via the cluster argument.

References

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. https://doi.org/10.18637/jss.v036.i03

Viechtbauer, W. (2021). Model checking in meta-analysis. In C. H. Schmid, T. Stijnen, & I. R. White (Eds.), Handbook of meta-analysis (pp. 219–254). Boca Raton, FL: CRC Press. https://doi.org/10.1201/9781315119403

Viechtbauer, W., & Cheung, M. W.-L. (2010). Outlier and influence diagnostics for meta-analysis. Research Synthesis Methods, 1(2), 112–125. https://doi.org/10.1002/jrsm.11

See also

rma.uni, rma.mh, and rma.peto for functions to fit models for which leave-one-out diagnostics can be computed.

Examples

### calculate log risk ratios and corresponding sampling variances
dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)

### random-effects model
res <- rma(yi, vi, data=dat)

### leave-one-out analysis
leave1out(res)
#> 
#>     estimate     se    zval   pval   ci.lb   ci.ub        Q     Qp   tau2      I2      H2 
#> -1   -0.7071 0.1900 -3.7223 0.0002 -1.0794 -0.3348 151.5826 0.0000 0.3362 93.2259 14.7622 
#> -2   -0.6540 0.1807 -3.6195 0.0003 -1.0082 -0.2999 145.3176 0.0000 0.2926 92.2540 12.9098 
#> -3   -0.6856 0.1857 -3.6916 0.0002 -1.0495 -0.3216 150.1970 0.0000 0.3207 92.9354 14.1551 
#> -4   -0.6284 0.1766 -3.5580 0.0004 -0.9746 -0.2822  96.5626 0.0000 0.2628 90.4125 10.4302 
#> -5   -0.7642 0.1918 -3.9845 0.0001 -1.1401 -0.3883 151.3200 0.0000 0.3278 92.7634 13.8187 
#> -6   -0.7109 0.2003 -3.5499 0.0004 -1.1034 -0.3184 128.1867 0.0000 0.3596 90.9118 11.0033 
#> -7   -0.6552 0.1805 -3.6307 0.0003 -1.0090 -0.3015 145.8296 0.0000 0.2930 92.2777 12.9495 
#> -8   -0.7948 0.1799 -4.4184 0.0000 -1.1473 -0.4422  67.9858 0.0000 0.2732 87.0314  7.7109 
#> -9   -0.7412 0.1967 -3.7686 0.0002 -1.1267 -0.3557 152.2051 0.0000 0.3495 93.2133 14.7346 
#> -10  -0.6530 0.1843 -3.5439 0.0004 -1.0142 -0.2919 139.8271 0.0000 0.2987 92.2322 12.8737 
#> -11  -0.7579 0.1958 -3.8708 0.0001 -1.1416 -0.3741 151.4655 0.0000 0.3405 91.8110 12.2114 
#> -12  -0.7598 0.1821 -4.1727 0.0000 -1.1167 -0.4029 150.7868 0.0000 0.3082 92.6782 13.6579 
#> -13  -0.7775 0.1855 -4.1908 0.0000 -1.1412 -0.4139 149.7884 0.0000 0.3037 92.3444 13.0623 
#> 
leave1out(res, transf=exp)
#> 
#>     estimate    zval   pval  ci.lb  ci.ub        Q     Qp   tau2      I2      H2 
#> -1    0.4931 -3.7223 0.0002 0.3398 0.7155 151.5826 0.0000 0.3362 93.2259 14.7622 
#> -2    0.5199 -3.6195 0.0003 0.3649 0.7409 145.3176 0.0000 0.2926 92.2540 12.9098 
#> -3    0.5038 -3.6916 0.0002 0.3501 0.7250 150.1970 0.0000 0.3207 92.9354 14.1551 
#> -4    0.5334 -3.5580 0.0004 0.3774 0.7541  96.5626 0.0000 0.2628 90.4125 10.4302 
#> -5    0.4657 -3.9845 0.0001 0.3198 0.6782 151.3200 0.0000 0.3278 92.7634 13.8187 
#> -6    0.4912 -3.5499 0.0004 0.3318 0.7273 128.1867 0.0000 0.3596 90.9118 11.0033 
#> -7    0.5193 -3.6307 0.0003 0.3646 0.7397 145.8296 0.0000 0.2930 92.2777 12.9495 
#> -8    0.4517 -4.4184 0.0000 0.3175 0.6426  67.9858 0.0000 0.2732 87.0314  7.7109 
#> -9    0.4765 -3.7686 0.0002 0.3241 0.7007 152.2051 0.0000 0.3495 93.2133 14.7346 
#> -10   0.5205 -3.5439 0.0004 0.3627 0.7469 139.8271 0.0000 0.2987 92.2322 12.8737 
#> -11   0.4687 -3.8708 0.0001 0.3193 0.6879 151.4655 0.0000 0.3405 91.8110 12.2114 
#> -12   0.4678 -4.1727 0.0000 0.3274 0.6684 150.7868 0.0000 0.3082 92.6782 13.6579 
#> -13   0.4595 -4.1908 0.0000 0.3194 0.6611 149.7884 0.0000 0.3037 92.3444 13.0623 
#> 

### leave-one-out analysis with a cluster variable
leave1out(res, cluster=alloc)
#> 
#>             estimate     se    zval   pval   ci.lb   ci.ub        Q     Qp   tau2      I2      H2 
#> -alternate   -0.7659 0.2159 -3.5482 0.0004 -1.1890 -0.3428 127.9750 0.0000 0.3817 91.4543 11.7018 
#> -random      -0.4813 0.2170 -2.2180 0.0266 -0.9066 -0.0560  28.2980 0.0000 0.2116 87.8973  8.2626 
#> -systematic  -0.8380 0.2169 -3.8635 0.0001 -1.2631 -0.4129 135.6025 0.0000 0.3177 92.7783 13.8471 
#> 

### meta-analysis of the (log) risk ratios using the Mantel-Haenszel method
res <- rma.mh(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)

### leave-one-out analysis
leave1out(res)
#> 
#>     estimate     se     zval   pval   ci.lb   ci.ub        Q     Qp      I2      H2 
#> -1   -0.4514 0.0394 -11.4462 0.0000 -0.5287 -0.3741 151.9153 0.0000 92.7591 13.8105 
#> -2   -0.4410 0.0395 -11.1512 0.0000 -0.5185 -0.3635 145.5727 0.0000 92.4436 13.2339 
#> -3   -0.4495 0.0394 -11.4001 0.0000 -0.5267 -0.3722 150.5130 0.0000 92.6917 13.6830 
#> -4   -0.3392 0.0414  -8.1874 0.0000 -0.4205 -0.2580  96.5629 0.0000 88.6085  8.7784 
#> -5   -0.4614 0.0400 -11.5474 0.0000 -0.5397 -0.3831 151.6611 0.0000 92.7470 13.7874 
#> -6   -0.3666 0.0448  -8.1809 0.0000 -0.4544 -0.2788 129.2200 0.0000 91.4874 11.7473 
#> -7   -0.4466 0.0395 -11.3046 0.0000 -0.5241 -0.3692 146.2134 0.0000 92.4767 13.2921 
#> -8   -0.7758 0.0520 -14.9267 0.0000 -0.8777 -0.6739  68.3763 0.0000 83.9126  6.2160 
#> -9   -0.4532 0.0399 -11.3633 0.0000 -0.5314 -0.3751 152.5497 0.0000 92.7892 13.8682 
#> -10  -0.4278 0.0399 -10.7338 0.0000 -0.5059 -0.3497 140.0446 0.0000 92.1454 12.7313 
#> -11  -0.4698 0.0421 -11.1706 0.0000 -0.5522 -0.3874 151.8140 0.0000 92.7543 13.8013 
#> -12  -0.4566 0.0394 -11.5865 0.0000 -0.5338 -0.3794 151.1253 0.0000 92.7213 13.7387 
#> -13  -0.4637 0.0398 -11.6542 0.0000 -0.5417 -0.3858 150.1246 0.0000 92.6728 13.6477 
#> 
leave1out(res, transf=exp)
#> 
#>     estimate     zval   pval  ci.lb  ci.ub        Q     Qp      I2      H2 
#> -1    0.6367 -11.4462 0.0000 0.5894 0.6879 151.9153 0.0000 92.7591 13.8105 
#> -2    0.6434 -11.1512 0.0000 0.5954 0.6952 145.5727 0.0000 92.4436 13.2339 
#> -3    0.6380 -11.4001 0.0000 0.5905 0.6892 150.5130 0.0000 92.6917 13.6830 
#> -4    0.7123  -8.1874 0.0000 0.6568 0.7726  96.5629 0.0000 88.6085  8.7784 
#> -5    0.6304 -11.5474 0.0000 0.5829 0.6818 151.6611 0.0000 92.7470 13.7874 
#> -6    0.6931  -8.1809 0.0000 0.6348 0.7567 129.2200 0.0000 91.4874 11.7473 
#> -7    0.6398 -11.3046 0.0000 0.5921 0.6913 146.2134 0.0000 92.4767 13.2921 
#> -8    0.4603 -14.9267 0.0000 0.4158 0.5097  68.3763 0.0000 83.9126  6.2160 
#> -9    0.6356 -11.3633 0.0000 0.5878 0.6872 152.5497 0.0000 92.7892 13.8682 
#> -10   0.6520 -10.7338 0.0000 0.6030 0.7049 140.0446 0.0000 92.1454 12.7313 
#> -11   0.6251 -11.1706 0.0000 0.5757 0.6788 151.8140 0.0000 92.7543 13.8013 
#> -12   0.6334 -11.5865 0.0000 0.5864 0.6843 151.1253 0.0000 92.7213 13.7387 
#> -13   0.6289 -11.6542 0.0000 0.5817 0.6799 150.1246 0.0000 92.6728 13.6477 
#> 

### meta-analysis of the (log) odds ratios using Peto's method
res <- rma.peto(ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)

### leave-one-out analysis
leave1out(res)
#> 
#>     estimate     se     zval   pval   ci.lb   ci.ub        Q     Qp      I2      H2 
#> -1   -0.4722 0.0408 -11.5791 0.0000 -0.5521 -0.3923 167.2005 0.0000 93.4211 15.2000 
#> -2   -0.4616 0.0409 -11.2751 0.0000 -0.5418 -0.3814 160.5154 0.0000 93.1471 14.5923 
#> -3   -0.4702 0.0408 -11.5317 0.0000 -0.5501 -0.3903 165.7913 0.0000 93.3652 15.0719 
#> -4   -0.3591 0.0435  -8.2533 0.0000 -0.4443 -0.2738 112.1274 0.0000 90.1897 10.1934 
#> -5   -0.4832 0.0413 -11.6873 0.0000 -0.5642 -0.4021 166.3727 0.0000 93.3883 15.1248 
#> -6   -0.3710 0.0451  -8.2328 0.0000 -0.4593 -0.2827 139.3918 0.0000 92.1086 12.6720 
#> -7   -0.4660 0.0408 -11.4343 0.0000 -0.5459 -0.3861 158.6048 0.0000 93.0645 14.4186 
#> -8   -0.8161 0.0531 -15.3842 0.0000 -0.9201 -0.7122  67.1837 0.0000 83.6270  6.1076 
#> -9   -0.4747 0.0413 -11.4973 0.0000 -0.5557 -0.3938 167.7284 0.0000 93.4418 15.2480 
#> -10  -0.4486 0.0413 -10.8488 0.0000 -0.5296 -0.3675 155.8992 0.0000 92.9442 14.1727 
#> -11  -0.4911 0.0434 -11.3135 0.0000 -0.5762 -0.4060 166.5326 0.0000 93.3947 15.1393 
#> -12  -0.4775 0.0407 -11.7236 0.0000 -0.5573 -0.3976 166.0703 0.0000 93.3763 15.0973 
#> -13  -0.4853 0.0411 -11.7960 0.0000 -0.5659 -0.4046 164.7427 0.0000 93.3229 14.9766 
#> 
leave1out(res, transf=exp)
#> 
#>     estimate     zval   pval  ci.lb  ci.ub        Q     Qp      I2      H2 
#> -1    0.6236 -11.5791 0.0000 0.5757 0.6755 167.2005 0.0000 93.4211 15.2000 
#> -2    0.6303 -11.2751 0.0000 0.5817 0.6829 160.5154 0.0000 93.1471 14.5923 
#> -3    0.6249 -11.5317 0.0000 0.5769 0.6769 165.7913 0.0000 93.3652 15.0719 
#> -4    0.6983  -8.2533 0.0000 0.6413 0.7605 112.1274 0.0000 90.1897 10.1934 
#> -5    0.6168 -11.6873 0.0000 0.5688 0.6689 166.3727 0.0000 93.3883 15.1248 
#> -6    0.6900  -8.2328 0.0000 0.6317 0.7538 139.3918 0.0000 92.1086 12.6720 
#> -7    0.6275 -11.4343 0.0000 0.5793 0.6797 158.6048 0.0000 93.0645 14.4186 
#> -8    0.4421 -15.3842 0.0000 0.3985 0.4906  67.1837 0.0000 83.6270  6.1076 
#> -9    0.6220 -11.4973 0.0000 0.5737 0.6745 167.7284 0.0000 93.4418 15.2480 
#> -10   0.6385 -10.8488 0.0000 0.5888 0.6924 155.8992 0.0000 92.9442 14.1727 
#> -11   0.6120 -11.3135 0.0000 0.5621 0.6663 166.5326 0.0000 93.3947 15.1393 
#> -12   0.6204 -11.7236 0.0000 0.5728 0.6719 166.0703 0.0000 93.3763 15.0973 
#> -13   0.6155 -11.7960 0.0000 0.5679 0.6672 164.7427 0.0000 93.3229 14.9766 
#>