Results from studies exploring how a superfluous fMRI brain image influences the persuasiveness of a scientific claim.

dat.michael2013

Format

The data frame contains the following columns:

Studycharactername of the study: Citation - Experiment - Subgroup
No_brain_nnumericsample size for no-brain-image condition
No_brain_mnumericmean agreement rating for no-brain-image condition
No_brain_snumericstandard deviation for no-brain-image condition
Brain_nnumericsample size for brain-image condition
Brain_mnumericmean agreement rating for brain-image condition
Brain_snumericstandard deviation for brain-image condition
Included_Critiquecharacter‘Critique’ if article included critical commentary on conclusions, otherwise ‘No_critique’
Mediumcharacter‘Paper’ if conducted in person; ‘Online’ if conducted online
Compensationcharacternotes on compensation provided to participants
Participant_Poolcharacternotes on where participants were recruited
yinumericraw mean difference, calculated as Brain_m - No_brain_m
vinumericcorresponding sampling variance

Details

The dataset contains the data from the meta-analysis by Michael et al. (2013) of experiments on the persuasive power of a brain image. The meta-analysis analyzed an original study by McCabe and Castel (2008) as well as 10 replication attempts conducted by the authors of the meta-analysis.

In each study, participants read an article about using brain imaging as a lie detector. The article either included a superfluous fMRI image of a brain (brain) or not (no_brain). After reading the article, all participants responded to the statement “Do you agree or disagree with the conclusion that brain imaging can be used as a lie detector?” on a scale from 1 (strongly disagree) to 4 (strongly agree).

The original study by McCabe and Castel (2008) reported a relatively large increase in agreement due to the presence of brain images. Meta-analysis of the original study with the 10 replications suggests, however, a small, possibly null effect: an estimated average raw mean difference of 0.07 points, 95% CI [-0.00, 0.14], under a random-effects model.

In some studies, the article included a passage critiquing the primary claims made in the article; this is coded in the Included_Critique column for analysis as a possible moderator. Note that Experiment 3 by McCabe and Castel (2008) was a 2x2 between subjects design: brain image presence was manipulated as well as the inclusion of a critique. The two different critique conditions are recorded as separate rows in this dataset. Analysis of this dataset with metafor yields the same results (given rounding) reported in the manuscript.

Source

Michael, R. B., Newman, E. J., Vuorre, M., Cumming, G., & Garry, M. (2013). On the (non)persuasive power of a brain image. Psychonomic Bulletin & Review, 20(4), 720–-725. https://doi.org/10.3758/s13423-013-0391-6

References

McCabe, D. P., & Castel, A. D. (2008). Seeing is believing: The effect of brain images on judgments of scientific reasoning. Cognition, 107(1), 343–352. https://doi.org/10.1016/j.cognition.2007.07.017

Author

Robert Calin-Jageman, rcalinjageman@dom.edu, https://calin-jageman.net

Concepts

psychology, persuasion, raw mean differences

Examples

### copy data into 'dat' and examine data
dat <- dat.michael2013
dat
#>                                                   Study No_brain_n No_brain_m No_brain_s Brain_n
#> 1  McCabe and Castel, 2008 - Experiment 3 - No Critique         28       2.89       0.79      26
#> 2                   Michael et al., 2013 - Experiment 1         99       2.90       0.58      98
#> 3                   Michael et al., 2013 - Experiment 2         42       2.62       0.54      33
#> 4                   Michael et al., 2013 - Experiment 3         24       2.96       0.36      21
#> 5                   Michael et al., 2013 - Experiment 4        184       2.93       0.60     184
#> 6                   Michael et al., 2013 - Experiment 5        274       2.86       0.59     255
#> 7     McCabe and Castel, 2008 - Experiment 3 - Critique         26       2.69       0.55      28
#> 8                   Michael et al., 2013 - Experiment 6         58       2.50       0.84      55
#> 9                   Michael et al., 2013 - Experiment 7         34       2.41       0.78      34
#> 10                  Michael et al., 2013 - Experiment 8         98       2.73       0.67      93
#> 11                  Michael et al., 2013 - Experiment 9         99       2.54       0.66      95
#> 12                 Michael et al., 2013 - Experiment 10         94       2.66       0.65      97
#>    Brain_m Brain_s Included_Critique Medium  Compensation                         Participant_Pool
#> 1     3.12    0.65       No_critique  Paper Course credit Colorado State University undergraduates
#> 2     2.86    0.61       No_critique Online       US$0.30                          Mechanical Turk
#> 3     2.85    0.57       No_critique Online Course credit      Victoria undergraduate subject pool
#> 4     3.07    0.55       No_critique  Paper Movie voucher          Wellington high school students
#> 5     2.89    0.60       No_critique Online       US$0.50                          Mechanical Turk
#> 6     2.91    0.52       No_critique  Paper Course credit         Victoria Intro Psyc subject pool
#> 7     3.00    0.54          Critique  Paper Course credit Colorado State University undergraduates
#> 8     2.60    0.83          Critique Online       US$0.50                          Mechanical Turk
#> 9     2.74    0.51          Critique  Paper          None                           General Public
#> 10    2.68    0.69          Critique Online       US$0.50                          Mechanical Turk
#> 11    2.72    0.68          Critique Online       US$0.50                          Mechanical Turk
#> 12    2.64    0.71          Critique Online       US$0.50                          Mechanical Turk
#>       yi          vi
#> 1   0.23 0.038539286
#> 2  -0.04 0.007194919
#> 3   0.23 0.016788312
#> 4   0.11 0.019804762
#> 5  -0.04 0.003913043
#> 6   0.05 0.002330830
#> 7   0.31 0.022048901
#> 8   0.10 0.024690972
#> 9   0.33 0.025544118
#> 10 -0.05 0.009699967
#> 11  0.18 0.009267368
#> 12 -0.02 0.009691588

# \dontrun{

### load metafor package
library(metafor)

### Data prep
# yi and vi are already provided, but here's how you would use escalc() to obtain
# a raw-mean difference and its variance.
# Note the measure parameter is "MD" for 'raw mean difference'
dat <- metafor::escalc(
  measure = "MD",
  m1i = Brain_m,
  m2i = No_brain_m,
  sd1i = Brain_s,
  sd2i = No_brain_s,
  n1i = Brain_n,
  n2i = No_brain_n,
  data = dat
)

### meta-analysis using a random-effects model of the raw mean differences
res <- rma(yi, vi, data=dat)
print(res, digits=2)
#> 
#> Random-Effects Model (k = 12; tau^2 estimator: REML)
#> 
#> tau^2 (estimated amount of total heterogeneity): 0.00 (SE = 0.01)
#> tau (square root of estimated tau^2 value):      0.06
#> I^2 (total heterogeneity / total variability):   28.92%
#> H^2 (total variability / sampling variability):  1.41
#> 
#> Test for Heterogeneity:
#> Q(df = 11) = 15.73, p-val = 0.15
#> 
#> Model Results:
#> 
#> estimate    se  zval  pval  ci.lb  ci.ub    
#>     0.07  0.03  1.94  0.05  -0.00   0.14  . 
#> 
#> ---
#> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#> 

### examine if Included_Critique is a potential moderator
res <- rma(yi, vi, mods = ~ Included_Critique, data=dat)
print(res, digits=2)
#> 
#> Mixed-Effects Model (k = 12; tau^2 estimator: REML)
#> 
#> tau^2 (estimated amount of residual heterogeneity):     0.00 (SE = 0.01)
#> tau (square root of estimated tau^2 value):             0.06
#> I^2 (residual heterogeneity / unaccounted variability): 27.82%
#> H^2 (unaccounted variability / sampling variability):   1.39
#> R^2 (amount of heterogeneity accounted for):            1.33%
#> 
#> Test for Residual Heterogeneity:
#> QE(df = 10) = 14.41, p-val = 0.16
#> 
#> Test of Moderators (coefficient 2):
#> QM(df = 1) = 0.84, p-val = 0.36
#> 
#> Model Results:
#> 
#>                               estimate    se   zval  pval  ci.lb  ci.ub    
#> intrcpt                           0.11  0.05   1.94  0.05  -0.00   0.21  . 
#> Included_CritiqueNo_critique     -0.06  0.07  -0.92  0.36  -0.20   0.07    
#> 
#> ---
#> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#> 

# }