Results from 160 studies on the correlation between employment interview assessments and job performance.

dat.mcdaniel1994

Format

The data frame contains the following columns:

studynumericstudy number
ninumericsample size of the study
rinumericobserved correlation
typecharacterinterview type (j = job-related, s = situational, p = psychological)
structcharacterinterview structure (u = unstructured, s = structured)

Details

The 160 studies provide data in terms of the correlation between employment interview performance and actual job performance. In addition, the interview type and the interview structure are indicated.

McDaniel et al. (1994) describe the interview type and structure variables as follows. "Questions in situational interviews [...] focus on the individual's ability to project what his or her behavior would be in a given situation. [...] Job-related interviews are those in which the interviewer is a personnel officer or hiring authority and the questions attempt to assess past behaviors and job-related information, but most questions are not considered situational. Psychological interviews are conducted by a psychologist, and the questions are intended to assess personal traits, such as dependability." In structured interviews, "the questions and acceptable responses were specified in advance and the responses were rated for appropriateness of content. [...] Unstructured interviews gather applicant information in a less systematic manner than do structured interviews. Although the questions may be specified in advance, they usually are not, and there is seldom a formalized scoring guide. Also, all persons being interviewed are not typically asked the same questions."

The goal of the meta-analysis was to examine the overall criterion-related validity of employment interviews and to examine whether the validity depends on the type and structure of the interview.

The data in this dataset were obtained from Table A.2 in Rothstein, Sutton, and Borenstein (2005, p. 325-329). Note that the type and struct variables contain some NAs.

Source

Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.). (2005). Publication bias in meta-analysis: Prevention, assessment, and adjustments. Chichester, England: Wiley.

References

McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79(4), 599--616. https://doi.org/10.1037/0021-9010.79.4.599

Examples

### copy data into 'dat' dat <- dat.mcdaniel1994 ### calculate r-to-z transformed correlations and corresponding sampling variances dat <- escalc(measure="ZCOR", ri=ri, ni=ni, data=dat) dat
#> study ni ri type struct yi vi #> 1 1 123 0.00 j s 0.0000 0.0083 #> 2 2 95 0.06 p u 0.0601 0.0109 #> 3 3 69 0.36 j s 0.3769 0.0152 #> 4 4 1832 0.15 j s 0.1511 0.0005 #> 5 5 78 0.14 j s 0.1409 0.0133 #> 6 6 329 0.06 j s 0.0601 0.0031 #> 7 7 153 0.09 j s 0.0902 0.0067 #> 8 8 29 0.40 j s 0.4236 0.0385 #> 9 9 29 0.39 s s 0.4118 0.0385 #> 10 10 157 0.14 s s 0.1409 0.0065 #> 11 11 149 0.36 s s 0.3769 0.0068 #> 12 12 92 0.28 j u 0.2877 0.0112 #> 13 13 15 0.62 j s 0.7250 0.0833 #> 14 14 15 0.07 j u 0.0701 0.0833 #> 15 15 170 0.18 j u 0.1820 0.0060 #> 16 16 19 0.42 j s 0.4477 0.0625 #> 17 17 19 0.08 j u 0.0802 0.0625 #> 18 18 68 0.18 p u 0.1820 0.0154 #> 19 19 93 0.43 j u 0.4599 0.0111 #> 20 20 57 0.04 j u 0.0400 0.0185 #> 21 21 80 -0.04 p <NA> -0.0400 0.0130 #> 22 22 53 0.05 p <NA> 0.0500 0.0200 #> 23 23 24 -0.14 p <NA> -0.1409 0.0476 #> 24 24 57 0.05 j s 0.0500 0.0185 #> 25 25 275 0.35 j s 0.3654 0.0037 #> 26 26 45 -0.08 p <NA> -0.0802 0.0238 #> 27 27 79 0.24 p <NA> 0.2448 0.0132 #> 28 28 107 0.16 p <NA> 0.1614 0.0096 #> 29 29 31 0.25 j u 0.2554 0.0357 #> 30 30 407 0.68 j s 0.8291 0.0025 #> 31 31 84 0.61 j s 0.7089 0.0123 #> 32 32 8 0.81 j s 1.1270 0.2000 #> 33 33 6 0.99 j s 2.6467 0.3333 #> 34 34 7 0.66 j s 0.7928 0.2500 #> 35 35 12 0.45 j s 0.4847 0.1111 #> 36 36 14 0.71 j s 0.8872 0.0909 #> 37 37 40 0.27 j s 0.2769 0.0270 #> 38 38 40 -0.02 j s -0.0200 0.0270 #> 39 39 99 0.29 j u 0.2986 0.0104 #> 40 40 164 0.13 j u 0.1307 0.0062 #> 41 41 67 0.03 j u 0.0300 0.0156 #> 42 42 57 0.00 j u 0.0000 0.0185 #> 43 43 50 0.09 j s 0.0902 0.0213 #> 44 44 129 -0.03 j u -0.0300 0.0079 #> 45 45 49 0.46 s s 0.4973 0.0217 #> 46 46 63 0.30 s s 0.3095 0.0167 #> 47 47 56 0.33 s s 0.3428 0.0189 #> 48 48 238 0.24 p <NA> 0.2448 0.0043 #> 49 49 20 0.64 j s 0.7582 0.0588 #> 50 50 122 0.12 j u 0.1206 0.0084 #> 51 51 51 0.15 j u 0.1511 0.0208 #> 52 52 40 0.44 j u 0.4722 0.0270 #> 53 53 210 0.00 j s 0.0000 0.0048 #> 54 54 334 0.16 j s 0.1614 0.0030 #> 55 55 310 0.21 p <NA> 0.2132 0.0033 #> 56 56 180 0.29 j s 0.2986 0.0056 #> 57 57 93 0.19 j u 0.1923 0.0111 #> 58 58 472 0.04 j u 0.0400 0.0021 #> 59 59 44 0.56 j u 0.6328 0.0244 #> 60 60 75 0.14 j u 0.1409 0.0139 #> 61 61 68 0.44 j u 0.4722 0.0154 #> 62 62 38 0.36 j u 0.3769 0.0286 #> 63 63 42 0.34 j <NA> 0.3541 0.0256 #> 64 64 39 0.11 j <NA> 0.1104 0.0278 #> 65 65 49 0.40 j <NA> 0.4236 0.0217 #> 66 66 41 0.23 j <NA> 0.2342 0.0263 #> 67 67 200 0.22 j s 0.2237 0.0051 #> 68 68 850 0.44 j s 0.4722 0.0012 #> 69 69 41 0.27 j s 0.2769 0.0263 #> 70 70 32 0.11 j s 0.1104 0.0345 #> 71 71 65 0.27 j s 0.2769 0.0161 #> 72 72 125 -0.07 j s -0.0701 0.0082 #> 73 73 134 0.32 j s 0.3316 0.0076 #> 74 74 21 0.05 j u 0.0500 0.0556 #> 75 75 44 0.20 j u 0.2027 0.0244 #> 76 76 170 0.18 j u 0.1820 0.0060 #> 77 77 149 0.34 j s 0.3541 0.0068 #> 78 78 296 0.03 j s 0.0300 0.0034 #> 79 79 24 0.45 s s 0.4847 0.0476 #> 80 80 312 0.34 j s 0.3541 0.0032 #> 81 81 205 0.51 j s 0.5627 0.0050 #> 82 82 30 0.41 s s 0.4356 0.0370 #> 83 83 11 0.37 s s 0.3884 0.1250 #> 84 84 22 0.25 s s 0.2554 0.0526 #> 85 85 37 -0.17 j s -0.1717 0.0294 #> 86 86 43 0.47 j s 0.5101 0.0250 #> 87 87 72 0.32 j s 0.3316 0.0145 #> 88 88 72 -0.09 s s -0.0902 0.0145 #> 89 89 108 0.33 j s 0.3428 0.0095 #> 90 90 73 0.22 j s 0.2237 0.0143 #> 91 91 73 0.27 s s 0.2769 0.0143 #> 92 92 117 0.00 j s 0.0000 0.0088 #> 93 93 80 0.41 j s 0.4356 0.0130 #> 94 94 95 0.16 j s 0.1614 0.0109 #> 95 95 182 0.00 j s 0.0000 0.0056 #> 96 96 93 0.03 j s 0.0300 0.0111 #> 97 97 64 0.01 j s 0.0100 0.0164 #> 98 98 370 0.03 j s 0.0300 0.0027 #> 99 99 131 0.14 j s 0.1409 0.0078 #> 100 100 87 0.11 j s 0.1104 0.0119 #> 101 101 80 0.08 j s 0.0802 0.0130 #> 102 102 41 -0.13 j s -0.1307 0.0263 #> 103 103 35 0.13 j u 0.1307 0.0312 #> 104 104 106 0.36 j s 0.3769 0.0097 #> 105 105 86 0.06 j s 0.0601 0.0120 #> 106 106 54 0.19 j s 0.1923 0.0196 #> 107 107 393 0.27 j s 0.2769 0.0026 #> 108 108 102 0.17 j s 0.1717 0.0101 #> 109 109 115 0.34 j s 0.3541 0.0089 #> 110 110 63 0.28 s s 0.2877 0.0167 #> 111 111 22 0.11 j s 0.1104 0.0526 #> 112 112 37 0.07 j u 0.0701 0.0294 #> 113 113 116 -0.13 <NA> <NA> -0.1307 0.0088 #> 114 114 416 0.12 j u 0.1206 0.0024 #> 115 115 101 0.12 j u 0.1206 0.0102 #> 116 116 1359 0.37 j u 0.3884 0.0007 #> 117 117 82 0.26 p u 0.2661 0.0127 #> 118 118 32 0.42 j s 0.4477 0.0345 #> 119 119 42 0.37 j s 0.3884 0.0256 #> 120 120 196 0.17 j s 0.1717 0.0052 #> 121 121 44 0.19 j s 0.1923 0.0244 #> 122 122 47 0.32 s s 0.3316 0.0227 #> 123 123 37 0.33 <NA> <NA> 0.3428 0.0294 #> 124 124 12 0.24 j s 0.2448 0.1111 #> 125 125 1807 0.09 <NA> <NA> 0.0902 0.0006 #> 126 126 73 0.36 j s 0.3769 0.0143 #> 127 127 73 0.26 s s 0.2661 0.0143 #> 128 128 70 0.42 j s 0.4477 0.0149 #> 129 129 30 0.62 j s 0.7250 0.0370 #> 130 130 60 0.87 j s 1.3331 0.0175 #> 131 131 38 -0.07 j s -0.0701 0.0286 #> 132 132 12 0.65 j s 0.7753 0.1111 #> 133 133 33 0.17 j u 0.1717 0.0333 #> 134 134 33 0.30 j u 0.3095 0.0333 #> 135 135 28 0.45 s s 0.4847 0.0400 #> 136 136 51 0.24 p u 0.2448 0.0208 #> 137 137 49 0.02 p u 0.0200 0.0217 #> 138 138 164 0.23 j s 0.2342 0.0062 #> 139 139 195 0.17 j s 0.1717 0.0052 #> 140 140 165 0.32 j s 0.3316 0.0062 #> 141 141 40 0.36 j s 0.3769 0.0270 #> 142 142 100 0.09 p s 0.0902 0.0103 #> 143 143 4195 0.13 j u 0.1307 0.0002 #> 144 144 179 0.29 j s 0.2986 0.0057 #> 145 145 74 0.49 j s 0.5361 0.0141 #> 146 146 110 0.40 j s 0.4236 0.0093 #> 147 147 31 0.23 j s 0.2342 0.0357 #> 148 148 70 0.31 j s 0.3205 0.0149 #> 149 149 21 0.46 j s 0.4973 0.0556 #> 150 150 29 -0.12 j s -0.1206 0.0385 #> 151 151 51 0.22 j u 0.2237 0.0208 #> 152 152 51 0.59 j s 0.6777 0.0208 #> 153 153 40 0.21 j s 0.2132 0.0270 #> 154 154 40 0.02 j s 0.0200 0.0270 #> 155 155 129 -0.03 j s -0.0300 0.0079 #> 156 156 196 0.28 j s 0.2877 0.0052 #> 157 157 31 -0.04 j s -0.0400 0.0357 #> 158 158 494 0.19 j u 0.1923 0.0020 #> 159 159 101 0.23 j s 0.2342 0.0102 #> 160 160 175 0.30 j s 0.3095 0.0058
### meta-analysis of the transformed correlations using a random-effects model res <- rma(yi, vi, data=dat) res
#> #> Random-Effects Model (k = 160; tau^2 estimator: REML) #> #> tau^2 (estimated amount of total heterogeneity): 0.0293 (SE = 0.0049) #> tau (square root of estimated tau^2 value): 0.1712 #> I^2 (total heterogeneity / total variability): 81.29% #> H^2 (total variability / sampling variability): 5.35 #> #> Test for Heterogeneity: #> Q(df = 159) = 789.7321, p-val < .0001 #> #> Model Results: #> #> estimate se zval pval ci.lb ci.ub #> 0.2374 0.0170 13.9995 <.0001 0.2042 0.2706 *** #> #> --- #> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 #>
### average correlation with 95% CI predict(res, transf=transf.ztor)
#> #> pred ci.lb ci.ub pi.lb pi.ub #> 0.2330 0.2014 0.2642 -0.0995 0.5187 #>
### mixed-effects model with interview type as factor ### note: job-related interviews is the reference level rma(yi, vi, mods = ~ factor(type), data=dat)
#> Warning: Studies with NAs omitted from model fitting.
#> #> Mixed-Effects Model (k = 157; tau^2 estimator: REML) #> #> tau^2 (estimated amount of residual heterogeneity): 0.0282 (SE = 0.0049) #> tau (square root of estimated tau^2 value): 0.1681 #> I^2 (residual heterogeneity / unaccounted variability): 79.62% #> H^2 (unaccounted variability / sampling variability): 4.91 #> R^2 (amount of heterogeneity accounted for): 1.92% #> #> Test for Residual Heterogeneity: #> QE(df = 154) = 738.4411, p-val < .0001 #> #> Test of Moderators (coefficients 2:3): #> QM(df = 2) = 5.8455, p-val = 0.0538 #> #> Model Results: #> #> estimate se zval pval ci.lb ci.ub #> intrcpt 0.2474 0.0187 13.2089 <.0001 0.2107 0.2841 *** #> factor(type)p -0.1228 0.0582 -2.1115 0.0347 -0.2368 -0.0088 * #> factor(type)s 0.0573 0.0598 0.9587 0.3377 -0.0599 0.1745 #> #> --- #> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 #>
### mixed-effects model with interview structure as factor ### note: structured interviews is the reference level rma(yi, vi, mods = ~ factor(struct), data=dat)
#> Warning: Studies with NAs omitted from model fitting.
#> #> Mixed-Effects Model (k = 145; tau^2 estimator: REML) #> #> tau^2 (estimated amount of residual heterogeneity): 0.0298 (SE = 0.0053) #> tau (square root of estimated tau^2 value): 0.1727 #> I^2 (residual heterogeneity / unaccounted variability): 79.91% #> H^2 (unaccounted variability / sampling variability): 4.98 #> R^2 (amount of heterogeneity accounted for): 1.95% #> #> Test for Residual Heterogeneity: #> QE(df = 143) = 701.2270, p-val < .0001 #> #> Test of Moderators (coefficient 2): #> QM(df = 1) = 3.8417, p-val = 0.0500 #> #> Model Results: #> #> estimate se zval pval ci.lb ci.ub #> intrcpt 0.2697 0.0211 12.7810 <.0001 0.2284 0.3111 *** #> factor(struct)u -0.0786 0.0401 -1.9600 0.0500 -0.1572 -0.0000 * #> #> --- #> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 #>
### note: the interpretation of the results is difficult since all ### situational interviews were structured, almost all psychological ### interviews were unstructured, and actually for the majority of ### the psychological interviews it was unknown whether the interview ### was structured or unstructured table(dat$type, dat$struct, useNA="always")
#> #> s u <NA> #> j 89 34 4 #> p 1 5 8 #> s 16 0 0 #> <NA> 0 0 3
### meta-analysis of raw correlations using a random-effects model res <- rma(measure="COR", ri=ri, ni=ni, data=dat.mcdaniel1994) res
#> #> Random-Effects Model (k = 160; tau^2 estimator: REML) #> #> tau^2 (estimated amount of total heterogeneity): 0.0331 (SE = 0.0051) #> tau (square root of estimated tau^2 value): 0.1819 #> I^2 (total heterogeneity / total variability): 88.95% #> H^2 (total variability / sampling variability): 9.05 #> #> Test for Heterogeneity: #> Q(df = 159) = 6136.9194, p-val < .0001 #> #> Model Results: #> #> estimate se zval pval ci.lb ci.ub #> 0.2444 0.0170 14.3885 <.0001 0.2111 0.2777 *** #> #> --- #> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 #>