dat.mcdaniel1994.RdResults from 160 studies on the correlation between employment interview assessments and job performance.
dat.mcdaniel1994The data frame contains the following columns:
| study | numeric | study number |
| ni | numeric | sample size of the study |
| ri | numeric | observed correlation |
| type | character | interview type (j = job-related, s = situational, p = psychological) |
| struct | character | interview structure (u = unstructured, s = structured) |
The 160 studies provide data in terms of the correlation between employment interview performance and actual job performance. In addition, the interview type and the interview structure are indicated.
McDaniel et al. (1994) describe the interview type and structure variables as follows. "Questions in situational interviews [...] focus on the individual's ability to project what his or her behavior would be in a given situation. [...] Job-related interviews are those in which the interviewer is a personnel officer or hiring authority and the questions attempt to assess past behaviors and job-related information, but most questions are not considered situational. Psychological interviews are conducted by a psychologist, and the questions are intended to assess personal traits, such as dependability." In structured interviews, "the questions and acceptable responses were specified in advance and the responses were rated for appropriateness of content. [...] Unstructured interviews gather applicant information in a less systematic manner than do structured interviews. Although the questions may be specified in advance, they usually are not, and there is seldom a formalized scoring guide. Also, all persons being interviewed are not typically asked the same questions."
The goal of the meta-analysis was to examine the overall criterion-related validity of employment interviews and to examine whether the validity depends on the type and structure of the interview.
The data in this dataset were obtained from Table A.2 in Rothstein, Sutton, and Borenstein (2005, p. 325-329). Note that the type and struct variables contain some NAs.
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.). (2005). Publication bias in meta-analysis: Prevention, assessment, and adjustments. Chichester, England: Wiley.
McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79(4), 599–616. https://doi.org/10.1037/0021-9010.79.4.599
psychology, correlation coefficients, meta-regression
### copy data into 'dat' and examine data
dat <- dat.mcdaniel1994
head(dat)
#> study ni ri type struct
#> 1 1 123 0.00 j s
#> 2 2 95 0.06 p u
#> 3 3 69 0.36 j s
#> 4 4 1832 0.15 j s
#> 5 5 78 0.14 j s
#> 6 6 329 0.06 j s
### load metafor package
library(metafor)
### calculate r-to-z transformed correlations and corresponding sampling variances
dat <- escalc(measure="ZCOR", ri=ri, ni=ni, data=dat)
head(dat)
#>
#> study ni ri type struct yi vi
#> 1 1 123 0.00 j s 0.0000 0.0083
#> 2 2 95 0.06 p u 0.0601 0.0109
#> 3 3 69 0.36 j s 0.3769 0.0152
#> 4 4 1832 0.15 j s 0.1511 0.0005
#> 5 5 78 0.14 j s 0.1409 0.0133
#> 6 6 329 0.06 j s 0.0601 0.0031
#>
### meta-analysis of the transformed correlations using a random-effects model
res <- rma(yi, vi, data=dat)
res
#>
#> Random-Effects Model (k = 160; tau^2 estimator: REML)
#>
#> tau^2 (estimated amount of total heterogeneity): 0.0293 (SE = 0.0049)
#> tau (square root of estimated tau^2 value): 0.1712
#> I^2 (total heterogeneity / total variability): 81.29%
#> H^2 (total variability / sampling variability): 5.35
#>
#> Test for Heterogeneity:
#> Q(df = 159) = 789.7321, p-val < .0001
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> 0.2374 0.0170 13.9995 <.0001 0.2042 0.2706 ***
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
### average correlation with 95% CI
predict(res, transf=transf.ztor, digits=2)
#>
#> pred ci.lb ci.ub pi.lb pi.ub
#> 0.23 0.20 0.26 -0.10 0.52
#>
### mixed-effects model with interview type as factor
### note: job-related interviews is the reference level
res <- rma(yi, vi, mods = ~ factor(type), data=dat)
#> Warning: 3 studies with NAs omitted from model fitting.
res
#>
#> Mixed-Effects Model (k = 157; tau^2 estimator: REML)
#>
#> tau^2 (estimated amount of residual heterogeneity): 0.0282 (SE = 0.0049)
#> tau (square root of estimated tau^2 value): 0.1681
#> I^2 (residual heterogeneity / unaccounted variability): 79.62%
#> H^2 (unaccounted variability / sampling variability): 4.91
#> R^2 (amount of heterogeneity accounted for): 1.92%
#>
#> Test for Residual Heterogeneity:
#> QE(df = 154) = 738.4411, p-val < .0001
#>
#> Test of Moderators (coefficients 2:3):
#> QM(df = 2) = 5.8455, p-val = 0.0538
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> intrcpt 0.2474 0.0187 13.2089 <.0001 0.2107 0.2841 ***
#> factor(type)p -0.1228 0.0582 -2.1115 0.0347 -0.2368 -0.0088 *
#> factor(type)s 0.0573 0.0598 0.9587 0.3377 -0.0599 0.1745
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
### estimated average correlation for each level of interview type
res <- rma(yi, vi, mods = ~ 0 + factor(type), data=dat)
#> Warning: 3 studies with NAs omitted from model fitting.
predict(res, newmods=diag(3), transf=transf.ztor, digits=2)
#>
#> pred ci.lb ci.ub pi.lb pi.ub
#> 1 0.24 0.21 0.28 -0.08 0.52
#> 2 0.12 0.02 0.23 -0.22 0.44
#> 3 0.30 0.19 0.39 -0.04 0.57
#>
### mixed-effects model with interview structure as factor
### note: structured interviews is the reference level
res <- rma(yi, vi, mods = ~ factor(struct), data=dat)
#> Warning: 15 studies with NAs omitted from model fitting.
res
#>
#> Mixed-Effects Model (k = 145; tau^2 estimator: REML)
#>
#> tau^2 (estimated amount of residual heterogeneity): 0.0298 (SE = 0.0053)
#> tau (square root of estimated tau^2 value): 0.1727
#> I^2 (residual heterogeneity / unaccounted variability): 79.91%
#> H^2 (unaccounted variability / sampling variability): 4.98
#> R^2 (amount of heterogeneity accounted for): 1.95%
#>
#> Test for Residual Heterogeneity:
#> QE(df = 143) = 701.2270, p-val < .0001
#>
#> Test of Moderators (coefficient 2):
#> QM(df = 1) = 3.8417, p-val = 0.0500
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> intrcpt 0.2697 0.0211 12.7810 <.0001 0.2284 0.3111 ***
#> factor(struct)u -0.0786 0.0401 -1.9600 0.0500 -0.1572 -0.0000 *
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>
### estimated average correlation for each level of interview structure
res <- rma(yi, vi, mods = ~ 0 + factor(struct), data=dat)
#> Warning: 15 studies with NAs omitted from model fitting.
predict(res, newmods=diag(2), transf=transf.ztor, digits=2)
#>
#> pred ci.lb ci.ub pi.lb pi.ub
#> 1 0.26 0.22 0.30 -0.07 0.54
#> 2 0.19 0.12 0.25 -0.15 0.49
#>
### note: the interpretation of the results is difficult since all
### situational interviews were structured, almost all psychological
### interviews were unstructured, and actually for the majority of
### the psychological interviews it was unknown whether the interview
### was structured or unstructured
table(dat$type, dat$struct, useNA="always")
#>
#> s u <NA>
#> j 89 34 4
#> p 1 5 8
#> s 16 0 0
#> <NA> 0 0 3
### meta-analysis of raw correlations using a random-effects model
res <- rma(measure="COR", ri=ri, ni=ni, data=dat.mcdaniel1994)
res
#>
#> Random-Effects Model (k = 160; tau^2 estimator: REML)
#>
#> tau^2 (estimated amount of total heterogeneity): 0.0331 (SE = 0.0051)
#> tau (square root of estimated tau^2 value): 0.1819
#> I^2 (total heterogeneity / total variability): 88.95%
#> H^2 (total variability / sampling variability): 9.05
#>
#> Test for Heterogeneity:
#> Q(df = 159) = 6136.9194, p-val < .0001
#>
#> Model Results:
#>
#> estimate se zval pval ci.lb ci.ub
#> 0.2444 0.0170 14.3885 <.0001 0.2111 0.2777 ***
#>
#> ---
#> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#>