`fsn.Rd`

Function to compute the fail-safe N (also called a file drawer analysis).

```
fsn(yi, vi, sei, data, type="Rosenthal", alpha=.05,
target, weighted=FALSE, subset, digits, ...)
```

- yi
vector with the observed effect sizes or outcomes.

- vi
vector with the corresponding sampling variances.

- sei
vector with the corresponding standard errors (note: only one of the two,

`vi`

or`sei`

, needs to be specified).- data
optional data frame containing the variables given to the arguments above.

- type
character string to specify the method to use for the calculation of the fail-safe N. Possible options are

`"Rosenthal"`

(the default),`"Orwin"`

, or`"Rosenberg"`

. See ‘Details’.- alpha
target alpha level to use for the Rosenthal and Rosenberg methods (the default is .05).

- target
target average effect size or outcome to use for the Orwin method. If undefined, then the target average effect size or outcome will be equal to the observed average effect size or outcome divided by 2.

- weighted
logical to specify whether Orwin's method should be based on unweighted (the default) or weighted averages (the default is

`FALSE`

).- subset
optional (logical or numeric) vector to specify the subset of studies that should be used for the calculations.

- digits
optional integer to specify the number of decimal places to which the printed results should be rounded.

- ...
other arguments.

The function can be used in combination with any of the usual effect sizes / outcome measures used in meta-analyses (e.g., log risk ratios, log odds ratios, risk differences, mean differences, standardized mean differences, raw correlation coefficients, correlation coefficients transformed with Fisher's r-to-z transformation), or, more generally, any set of estimates (with corresponding sampling variances) one would like to analyze. Simply specify the observed outcomes via the `yi`

argument and the corresponding sampling variances via the `vi`

argument (instead of specifying `vi`

, one can specify the standard errors via the `sei`

argument). The `escalc`

function can be used to compute a wide variety of effect sizes / outcome measures (and the corresponding sampling variances) based on summary statistics.

The Rosenthal method (sometimes called a ‘file drawer analysis’) calculates the number of studies averaging null results that would have to be added to the given set of observed outcomes to reduce the combined significance level (p-value) to a particular alpha level (e.g., .05). The calculation is based on Stouffer's method to combine p-values and is described in Rosenthal (1979).

The Orwin method calculates the number of studies averaging null results that would have to be added to the given set of observed outcomes to reduce the (unweighted or weighted) average outcome to a target value (as specified via the `target`

argument). The method is described in Orwin (1983). If `weighted=FALSE`

(the default), the method does not require (or makes use) of `vi`

(or `sei`

), so these arguments are then not relevant for this method. If the `target`

argument is not specified, then the target average outcome will be equal to the observed average outcome divided by 2 (which is quite arbitrary). One should really set `target`

to a value that reflects an outcome one would consider practically irrelevant. Note that if `target`

has the opposite sign as the actually observed average outcome, then its sign is automatically flipped.

The Rosenberg method calculates the number of studies averaging null results that would have to be added to the given set of observed outcomes to reduce the significance level (i.e., p-value) of the weighted average outcome (based on an equal-effects model) to a particular alpha level (e.g., .05). The method is described in Rosenberg (2005).

If the combined/observed significance level is above the specified alpha level (for `type = "Rosenthal"`

or `type = "Rosenberg"`

) or if the observed average outcome is below the target average outcome (for `type = "Orwin"`

), then the fail-sage N value will be 0.

An object of class `"fsn"`

. The object is a list containing the following components:

- type
the method used.

- fsnum
the calculated fail-safe N.

- alpha
the specified alpha level.

- pval
the p-value of the observed results.

`NA`

for the Orwin method.- meanes
the average outcome of the observed results.

`NA`

for the Rosenthal method.- target
the target value.

`NA`

for the Rosenthal and Rosenberg methods.

The results are formatted and printed with the `print`

function.

For the Rosenberg method, the p-value is calculated based on a standard normal distribution (instead of a t-distribution, as suggested by Rosenberg, 2005).

Rosenthal, R. (1979). The "file drawer problem" and tolerance for null results. *Psychological Bulletin*, **86**(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638

Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. *Journal of Educational Statistics*, **8**(2), 157–159. https://doi.org/10.3102/10769986008002157

Rosenberg, M. S. (2005). The file-drawer problem revisited: A general weighted method for calculating fail-safe numbers in meta-analysis. *Evolution*, **59**(2), 464–468. https://doi.org/10.1111/j.0014-3820.2005.tb01004.x

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. *Journal of Statistical Software*, **36**(3), 1–48. https://doi.org/10.18637/jss.v036.i03

```
### calculate log risk ratios and corresponding sampling variances
dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)
### fit equal-effects model
rma(yi, vi, data=dat, method="EE")
#>
#> Equal-Effects Model (k = 13)
#>
#> I^2 (total heterogeneity / total variability): 92.12%
#> H^2 (total variability / sampling variability): 12.69
#>
#> Test for Heterogeneity:
#> Q(df = 12) = 152.2330, p-val < .0001
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> -0.4303 0.0405 -10.6247 <.0001 -0.5097 -0.3509 ***
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
### fail-safe N computations
fsn(yi, vi, data=dat)
#>
#> Fail-safe N Calculation Using the Rosenthal Approach
#>
#> Observed Significance Level: <.0001
#> Target Significance Level: 0.05
#>
#> Fail-safe N: 598
#>
fsn(yi, data=dat, type="Orwin", target=log(0.95)) # target corresponds to a 5% risk reduction
#>
#> Fail-safe N Calculation Using the Orwin Approach
#>
#> Average Effect Size: -0.7407
#> Target Effect Size: -0.0513
#>
#> Fail-safe N: 175
#>
fsn(yi, vi, data=dat, type="Orwin", weighted=TRUE, target=log(0.95))
#>
#> Fail-safe N Calculation Using the Orwin Approach
#>
#> Average Effect Size: -0.4303
#> Target Effect Size: -0.0513
#>
#> Fail-safe N: 97
#>
fsn(yi, vi, data=dat, type="Rosenberg")
#>
#> Fail-safe N Calculation Using the Rosenberg Approach
#>
#> Average Effect Size: -0.4303
#> Observed Significance Level: <.0001
#> Target Significance Level: 0.05
#>
#> Fail-safe N: 370
#>
```