For two (nested) models of class "rma.uni" or "rma.mv", the function provides a full versus reduced model comparison in terms of model fit statistics and a likelihood ratio test. When a single model is specified, a Wald-type test of one or more model coefficients or linear combinations thereof is carried out.

# S3 method for rma
anova(object, object2, btt, L, digits, ...)

## Arguments

object an object of class "rma.uni" or "rma.mv". an (optional) object of class "rma.uni" or "rma.mv". See ‘Details’. optional vector of indices specifying which coefficients to include in the Wald-type test. Can also be a string to grep for. See ‘Details’. optional numeric vector or matrix specifying linear combinations of the coefficients in the model that should be tested to be zero. See ‘Details’. integer specifying the number of decimal places to which the printed results should be rounded (if unspecified, the default is to take the value from the object). other arguments.

## Details

When a single model is specified, the function provides Wald-type tests of one or more model coefficients or linear combinations thereof.

In particular, for a fixed- or random-effects model (i.e., a model without moderators), this is just the test of the single coefficient of the model. For models including moderators, an omnibus test of all the model coefficients is conducted that excludes the intercept (the first coefficient) if it is included in the model. If no intercept is included in the model, then the omnibus test includes all of the coefficients in the model including the first.

Alternatively, one can manually specify the indices of the coefficients to test via the btt argument. For example, with btt=c(3,4), only the third and fourth coefficient from the model would be included in the test (if an intercept is included in the model, then it corresponds to the first coefficient in the model). Instead of specifying the coefficient numbers, one can specify a string for btt. In that case, grep will be used to search for all coefficient names that match the string.

Instead, one can also specify one or multiple linear combinations of the coefficients in the model that should be tested to be zero via the L argument. If given as a matrix, its rows define the linear combinations to be tested. If the matrix is of full rank, an omnibus Wald-type test of all linear combinations is also provided.

When specifying two models for comparison, the function provides a likelihood ratio test comparing the two models. The two models must be based on the same set of data, must be of the same class, and should be nested for the likelihood ratio test to make sense. Note that likelihood ratio tests are not meaningful when using REML estimation and the two models have different fixed effects.

## Value

An object of class "anova.rma". When a single model is specified (without any further arguments or together with the btt argument), the object is a list containing the following components:

QM

test statistic for the Wald-type test of model coefficients.

QMp

p-value for the Wald-type test of model coefficients.

btt

indices of the coefficients tested by the Wald-type test.

k

number of outcomes included in the model fitting.

p

number of coefficients in the model (including the intercept).

m

number of coefficients included in the Wald-type test of coefficients.

...

When argument L is specified, the object is a list containing the following components:
QM

test statistic for the omnibus Wald-type test of all linear combinations.

QMp

p-value for the omnibus Wald-type test.

hyp

description of the linear combinations tested.

Lb

values of the linear combinations.

se

standard errors of the linear combinations.

zval

test statistics of the linear combinations.

pval

p-values for the linear combinations.

When two models are specified, the object is a list containing the following components:
fit.stats.f

log-likelihood, deviance, AIC, BIC, and AICc for the full model.

fit.stats.r

log-likelihood, deviance, AIC, BIC, and AICc for the reduced model.

p.f

number of parameters in the full model.

p.r

number of parameters in the reduced model.

LRT

likelihood ratio test statistic.

pval

p-value for the likelihood ratio test.

QE.f

test statistic for the test of (residual) heterogeneity from the full model.

QE.r

test statistic for the test of (residual) heterogeneity from the reduced model.

tau2.f

estimated $$\tau^2$$ value from the full model. NA for "rma.mv" objects.

tau2.r

estimated $$\tau^2$$ value from the reduced model. NA for "rma.mv" objects.

R2

amount (in percent) of the heterogeneity in the reduced model that is accounted for in the full model (NA for fixed-effects models or for "rma.mv" objects). This can be regarded as a pseudo $$R^2$$ statistic (Raudenbush, 2009). Note that the value may not be very accurate unless $$k$$ is large (Lopez-Lopez et al., 2014).

...

The results are formatted and printed with the print.anova.rma function.

## Note

In principle, one can consider likelihood ratio tests for the amount of (residual) heterogeneity in random- and mixed-effects models. The full model should then be fitted with either method="ML" or method="REML" and the reduced model with method="FE". The p-value from that test is based on a chi-square distribution with 1 degree of freedom, but actually needs to be adjusted for the fact that the parameter (i.e., $$\tau^2$$) falls on the boundary of the parameter space under the null hypothesis. Moreover, the Q-test usually keeps better control of the Type I error rate and therefore should be preferred (see Viechtbauer, 2007, for more details).

## References

Hardy, R. J., & Thompson, S. G. (1996). A likelihood approach to meta-analysis with random effects. Statistics in Medicine, 15, 619--629.

Huizenga, H. M., Visser, I., & Dolan, C. V. (2011). Testing overall and moderator effects in random effects meta-regression. British Journal of Mathematical and Statistical Psychology, 64, 1--19.

López-López, J. A., Marín-Martínez, F., Sánchez-Meca, J., Van den Noortgate, W., & Viechtbauer, W. (2014). Estimation of the predictive power of the model in mixed-effects meta-regression: A simulation study. British Journal of Mathematical and Statistical Psychology, 67, 30--48.

Raudenbush, S. W. (2009). Analyzing effect sizes: Random effects models. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 295--315). New York: Russell Sage Foundation.

Viechtbauer, W. (2007). Hypothesis tests for population heterogeneity in meta-analysis. British Journal of Mathematical and Statistical Psychology, 60, 29--60.

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1--48. http://www.jstatsoft.org/v36/i03/.

rma.uni, print.anova.rma

## Examples

### calculate log risk ratios and corresponding sampling variances
dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)

### random-effects model
res1 <- rma(yi, vi, data=dat, method="ML")

### mixed-effects model with two moderators (absolute latitude and publication year)
res2 <- rma(yi, vi, mods = ~ ablat + year, data=dat, method="ML")

### Wald-type test of the two moderators
anova(res2)#>
#> Test of Moderators (coefficients 2:3):
#> QM(df = 2) = 33.9192, p-val < .0001
#>
### alternative way of specifying the same test
anova(res2, L=rbind(c(0,1,0), c(0,0,1)))#>
#> Hypotheses:
#> 1: ablat = 0
#> 2:  year = 0
#>
#> Results:
#>    estimate     se    zval   pval
#> 1:  -0.0308 0.0063 -4.9057 <.0001
#> 2:  -0.0032 0.0092 -0.3471 0.7285
#>
#> Omnibus Test of Hypotheses:
#> QM(df = 2) = 33.9192, p-val < .0001
#>
### corresponding likelihood ratio test
anova(res1, res2)#>
#>         df     AIC     BIC    AICc   logLik     LRT   pval       QE  tau^2
#> Full     4 23.2922 25.5520 28.2922  -7.6461                 28.3251 0.0269
#> Reduced  2 29.3302 30.4601 30.5302 -12.6651 10.0379 0.0066 152.2330 0.2800
#>              R^2
#> Full
#> Reduced 90.3948%
#>
### test of a linear combination
anova(res2, L=c(1,35,1970))#>
#> Hypothesis:
#> 1: intrcpt + 35*ablat + 1970*year = 0
#>
#> Results:
#>    estimate     se    zval   pval
#> 1:  -0.7533 0.0837 -9.0039 <.0001
#>
#> Test of Hypothesis:
#> QM(df = 1) = 81.0700, p-val < .0001
#>
### mixed-effects model with three moderators
res3 <- rma(yi, vi, mods = ~ ablat + year + alloc, data=dat, method="ML")
res3#>
#> Mixed-Effects Model (k = 13; tau^2 estimator: ML)
#>
#> tau^2 (estimated amount of residual heterogeneity):     0.0329 (SE = 0.0272)
#> tau (square root of estimated tau^2 value):             0.1814
#> I^2 (residual heterogeneity / unaccounted variability): 33.24%
#> H^2 (unaccounted variability / sampling variability):   1.50
#> R^2 (amount of heterogeneity accounted for):            88.24%
#>
#> Test for Residual Heterogeneity:
#> QE(df = 8) = 26.2030, p-val = 0.0010
#>
#> Test of Moderators (coefficients 2:5):
#> QM(df = 4) = 31.2609, p-val < .0001
#>
#> Model Results:
#>
#>                  estimate       se     zval    pval     ci.lb    ci.ub
#> intrcpt          -10.0824  23.9973  -0.4201  0.6744  -57.1162  36.9514
#> ablat             -0.0277   0.0071  -3.9298  <.0001   -0.0416  -0.0139  ***
#> year               0.0053   0.0122   0.4371  0.6620   -0.0185   0.0292
#> allocrandom       -0.2826   0.2540  -1.1127  0.2658   -0.7804   0.2152
#> allocsystematic   -0.1104   0.2551  -0.4330  0.6650   -0.6104   0.3895
#>
#> ---
#> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
### test the 'alloc' factor
anova(res3, btt=4:5)#>
#> Test of Moderators (coefficients 4:5):
#> QM(df = 2) = 1.4674, p-val = 0.4801
#>
### instead of specifying the coefficient numbers, grep for "alloc"
anova(res3, btt="alloc")#>
#> Test of Moderators (coefficients 4:5):
#> QM(df = 2) = 1.4674, p-val = 0.4801
#>