Results from 160 studies on the correlation between employment interview assessments and job performance.

dat.mcdaniel1994

Format

The data frame contains the following columns:

studynumericstudy number
ninumericsample size of the study
rinumericobserved correlation
typecharacterinterview type (j = job-related, s = situational, p = psychological)
structcharacterinterview structure (u = unstructured, s = structured)

Details

The 160 studies provide data in terms of the correlation between employment interview performance and actual job performance. In addition, the interview type and the interview structure are indicated.

McDaniel et al. (1994) describe the interview type and structure variables as follows. "Questions in situational interviews [...] focus on the individual's ability to project what his or her behavior would be in a given situation. [...] Job-related interviews are those in which the interviewer is a personnel officer or hiring authority and the questions attempt to assess past behaviors and job-related information, but most questions are not considered situational. Psychological interviews are conducted by a psychologist, and the questions are intended to assess personal traits, such as dependability." In structured interviews, "the questions and acceptable responses were specified in advance and the responses were rated for appropriateness of content. [...] Unstructured interviews gather applicant information in a less systematic manner than do structured interviews. Although the questions may be specified in advance, they usually are not, and there is seldom a formalized scoring guide. Also, all persons being interviewed are not typically asked the same questions."

The goal of the meta-analysis was to examine the overall criterion-related validity of employment interviews and to examine whether the validity depends on the type and structure of the interview.

The data in this dataset were obtained from Table A.2 in Rothstein, Sutton, and Borenstein (2005, p. 325-329). Note that the type and struct variables contain some NAs.

Source

Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.). (2005). Publication bias in meta-analysis: Prevention, assessment, and adjustments. Chichester, England: Wiley.

References

McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79(4), 599--616. https://doi.org/10.1037/0021-9010.79.4.599

Examples

### copy data into 'dat' and examine data
dat <- dat.mcdaniel1994
head(dat)
#>   study   ni   ri type struct
#> 1     1  123 0.00    j      s
#> 2     2   95 0.06    p      u
#> 3     3   69 0.36    j      s
#> 4     4 1832 0.15    j      s
#> 5     5   78 0.14    j      s
#> 6     6  329 0.06    j      s

# \dontrun{

### load metafor package
library(metafor)

### calculate r-to-z transformed correlations and corresponding sampling variances
dat <- escalc(measure="ZCOR", ri=ri, ni=ni, data=dat)
head(dat)
#> 
#>   study   ni   ri type struct     yi     vi 
#> 1     1  123 0.00    j      s 0.0000 0.0083 
#> 2     2   95 0.06    p      u 0.0601 0.0109 
#> 3     3   69 0.36    j      s 0.3769 0.0152 
#> 4     4 1832 0.15    j      s 0.1511 0.0005 
#> 5     5   78 0.14    j      s 0.1409 0.0133 
#> 6     6  329 0.06    j      s 0.0601 0.0031 
#> 

### meta-analysis of the transformed correlations using a random-effects model
res <- rma(yi, vi, data=dat)
res
#> 
#> Random-Effects Model (k = 160; tau^2 estimator: REML)
#> 
#> tau^2 (estimated amount of total heterogeneity): 0.0293 (SE = 0.0049)
#> tau (square root of estimated tau^2 value):      0.1712
#> I^2 (total heterogeneity / total variability):   81.29%
#> H^2 (total variability / sampling variability):  5.35
#> 
#> Test for Heterogeneity:
#> Q(df = 159) = 789.7321, p-val < .0001
#> 
#> Model Results:
#> 
#> estimate      se     zval    pval   ci.lb   ci.ub     ​ 
#>   0.2374  0.0170  13.9995  <.0001  0.2042  0.2706  *** 
#> 
#> ---
#> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#> 

### average correlation with 95% CI
predict(res, transf=transf.ztor)
#> 
#>    pred  ci.lb  ci.ub   pi.lb  pi.ub 
#>  0.2330 0.2014 0.2642 -0.0995 0.5187 
#> 

### mixed-effects model with interview type as factor
### note: job-related interviews is the reference level
rma(yi, vi, mods = ~ factor(type), data=dat)
#> Warning: Studies with NAs omitted from model fitting.
#> 
#> Mixed-Effects Model (k = 157; tau^2 estimator: REML)
#> 
#> tau^2 (estimated amount of residual heterogeneity):     0.0282 (SE = 0.0049)
#> tau (square root of estimated tau^2 value):             0.1681
#> I^2 (residual heterogeneity / unaccounted variability): 79.62%
#> H^2 (unaccounted variability / sampling variability):   4.91
#> R^2 (amount of heterogeneity accounted for):            1.92%
#> 
#> Test for Residual Heterogeneity:
#> QE(df = 154) = 738.4411, p-val < .0001
#> 
#> Test of Moderators (coefficients 2:3):
#> QM(df = 2) = 5.8455, p-val = 0.0538
#> 
#> Model Results:
#> 
#>                estimate      se     zval    pval    ci.lb    ci.ub     ​ 
#> intrcpt          0.2474  0.0187  13.2089  <.0001   0.2107   0.2841  *** 
#> factor(type)p   -0.1228  0.0582  -2.1115  0.0347  -0.2368  -0.0088    * 
#> factor(type)s    0.0573  0.0598   0.9587  0.3377  -0.0599   0.1745      
#> 
#> ---
#> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#> 

### mixed-effects model with interview structure as factor
### note: structured interviews is the reference level
rma(yi, vi, mods = ~ factor(struct), data=dat)
#> Warning: Studies with NAs omitted from model fitting.
#> 
#> Mixed-Effects Model (k = 145; tau^2 estimator: REML)
#> 
#> tau^2 (estimated amount of residual heterogeneity):     0.0298 (SE = 0.0053)
#> tau (square root of estimated tau^2 value):             0.1727
#> I^2 (residual heterogeneity / unaccounted variability): 79.91%
#> H^2 (unaccounted variability / sampling variability):   4.98
#> R^2 (amount of heterogeneity accounted for):            1.95%
#> 
#> Test for Residual Heterogeneity:
#> QE(df = 143) = 701.2270, p-val < .0001
#> 
#> Test of Moderators (coefficient 2):
#> QM(df = 1) = 3.8417, p-val = 0.0500
#> 
#> Model Results:
#> 
#>                  estimate      se     zval    pval    ci.lb    ci.ub     ​ 
#> intrcpt            0.2697  0.0211  12.7810  <.0001   0.2284   0.3111  *** 
#> factor(struct)u   -0.0786  0.0401  -1.9600  0.0500  -0.1572  -0.0000    * 
#> 
#> ---
#> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#> 

### note: the interpretation of the results is difficult since all
### situational interviews were structured, almost all psychological
### interviews were unstructured, and actually for the majority of
### the psychological interviews it was unknown whether the interview
### was structured or unstructured
table(dat$type, dat$struct, useNA="always")
#>       
#>         s  u <NA>
#>   j    89 34    4
#>   p     1  5    8
#>   s    16  0    0
#>   <NA>  0  0    3

### meta-analysis of raw correlations using a random-effects model
res <- rma(measure="COR", ri=ri, ni=ni, data=dat.mcdaniel1994)
res
#> 
#> Random-Effects Model (k = 160; tau^2 estimator: REML)
#> 
#> tau^2 (estimated amount of total heterogeneity): 0.0331 (SE = 0.0051)
#> tau (square root of estimated tau^2 value):      0.1819
#> I^2 (total heterogeneity / total variability):   88.95%
#> H^2 (total variability / sampling variability):  9.05
#> 
#> Test for Heterogeneity:
#> Q(df = 159) = 6136.9194, p-val < .0001
#> 
#> Model Results:
#> 
#> estimate      se     zval    pval   ci.lb   ci.ub     ​ 
#>   0.2444  0.0170  14.3885  <.0001  0.2111  0.2777  *** 
#> 
#> ---
#> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#> 

# }