Results from 20 studies on the correlation between course instructor ratings and student achievement.

dat.cohen1981

Format

The data frame contains the following columns:

studycharacterstudy author(s) and year
samplecharactercourse type
controlcharacterability control
ninumericsample size of the study (number of sections)
rinumericobserved correlation

Details

The studies included in this dataset examined to what extent students' ratings of a course instructor correlated with their achievement in the course. Instead of correlating individual ratings and achievement scores, the studies were carried out in multisection courses, in which the sections had different instructors but all sections used a common achievement measure (e.g., a final exam). The correlation coefficients reflect the correlation between the mean instructor rating and the mean achievement score of each section. Hence, the unit of analysis are the sections, not the individuals.

Note that this dataset (extracted from Table A.3 in Cooper & Hedges, 1994) only contains studies with at least 10 sections.

Source

Cooper, H., & Hedges, L. V. (1994). Appendix A: Data Sets. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 543-547). New York: Russell Sage Foundation.

References

Cohen, P. A. (1981). Student ratings of instruction and student achievement: A meta-analysis of multisection validity studies. Review of Educational Research, 51(3), 281–309. https://doi.org/10.3102/00346543051003281

Concepts

education, correlation coefficients

Examples

### copy data into 'dat' and examine data
dat <- dat.cohen1981
dat[c(1,4,5)]
#>                     study  ni    ri
#> 1      Bolton et al. 1979  10  0.68
#> 2             Bryson 1974  20  0.56
#> 3             Centra 1977  13  0.23
#> 4             Centra 1977  22  0.64
#> 5     Crooks & Smock 1974  28  0.49
#> 6   Doyle & Crichton 1978  12 -0.04
#> 7    Doyle & Whitely 1974  12  0.49
#> 8            Elliott 1950  36  0.33
#> 9    Ellis & Rickard 1977  19  0.58
#> 10       Frey et al. 1975  12  0.18
#> 11  Greenwood et al. 1976  36 -0.11
#> 12           Hoffman 1978  75  0.27
#> 13  McKeachie et al. 1971  33  0.26
#> 14      Morsh et al. 1956 121  0.40
#> 15    Remmers et al. 1949  37  0.49
#> 16 Sullivan & Skanes 1974  14  0.51
#> 17 Sullivan & Skanes 1974  40  0.40
#> 18 Sullivan & Skanes 1974  16  0.34
#> 19 Sullivan & Skanes 1974  14  0.42
#> 20            Wherry 1952  20  0.16

### load metafor package
library(metafor)

### calculate r-to-z transformed correlations and corresponding sampling variances
dat <- escalc(measure="ZCOR", ri=ri, ni=ni, data=dat[c(1,4,5)])
dat
#> 
#>                     study  ni    ri      yi     vi 
#> 1      Bolton et al. 1979  10  0.68  0.8291 0.1429 
#> 2             Bryson 1974  20  0.56  0.6328 0.0588 
#> 3             Centra 1977  13  0.23  0.2342 0.1000 
#> 4             Centra 1977  22  0.64  0.7582 0.0526 
#> 5     Crooks & Smock 1974  28  0.49  0.5361 0.0400 
#> 6   Doyle & Crichton 1978  12 -0.04 -0.0400 0.1111 
#> 7    Doyle & Whitely 1974  12  0.49  0.5361 0.1111 
#> 8            Elliott 1950  36  0.33  0.3428 0.0303 
#> 9    Ellis & Rickard 1977  19  0.58  0.6625 0.0625 
#> 10       Frey et al. 1975  12  0.18  0.1820 0.1111 
#> 11  Greenwood et al. 1976  36 -0.11 -0.1104 0.0303 
#> 12           Hoffman 1978  75  0.27  0.2769 0.0139 
#> 13  McKeachie et al. 1971  33  0.26  0.2661 0.0333 
#> 14      Morsh et al. 1956 121  0.40  0.4236 0.0085 
#> 15    Remmers et al. 1949  37  0.49  0.5361 0.0294 
#> 16 Sullivan & Skanes 1974  14  0.51  0.5627 0.0909 
#> 17 Sullivan & Skanes 1974  40  0.40  0.4236 0.0270 
#> 18 Sullivan & Skanes 1974  16  0.34  0.3541 0.0769 
#> 19 Sullivan & Skanes 1974  14  0.42  0.4477 0.0909 
#> 20            Wherry 1952  20  0.16  0.1614 0.0588 
#> 

### meta-analysis of the transformed correlations using a random-effects model
res <- rma(yi, vi, data=dat, digits=2)
res
#> 
#> Random-Effects Model (k = 20; tau^2 estimator: REML)
#> 
#> tau^2 (estimated amount of total heterogeneity): 0.01 (SE = 0.01)
#> tau (square root of estimated tau^2 value):      0.07
#> I^2 (total heterogeneity / total variability):   11.64%
#> H^2 (total variability / sampling variability):  1.13
#> 
#> Test for Heterogeneity:
#> Q(df = 19) = 20.97, p-val = 0.34
#> 
#> Model Results:
#> 
#> estimate    se  zval  pval  ci.lb  ci.ub      
#>     0.38  0.05  7.88  <.01   0.29   0.48  *** 
#> 
#> ---
#> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#> 

### predicted average correlation with 95% CI
predict(res, transf=transf.ztor)
#> 
#>  pred ci.lb ci.ub pi.lb pi.ub 
#>  0.36  0.28  0.44  0.21  0.50 
#>