A Meta-Analysis of Procedures to Change Implicit Measures

Introduction

Often, our intentions conflict with how we behave. These gaps between our intentions and actions can influence many social issues, including discrimination. For example, an organization may espouse racial equity but hire a white candidate over an equally qualified candidate of color. In response to disparities caused by unintentionally biased behavior, some researchers have suggested a solution: change automatic mental processes, then behavior influenced by those processes will change. Researchers have been particularly interested in “implicit measures” and “explicit measures.” Implicit measures are often associated with automatic processes, whereas explicit measures are often associated with deliberate processes. For example, an implicit measure refers to how long it takes someone to classify the words “good” or “bad” when preceded by the word “flower,” while an explicit measure refers to how someone rates flowers on a scale of good to bad.

In this study, Forscher et al. synthesized hundreds of studies to investigate how effective different approaches were in changing implicit measures. Their analysis was driven by six central questions, including: which approaches to changing implicit measures are most influential, and how do changes in implicit measures correspond with changes in behavior? In the first large-scale quantitative analysis of research on change in implicit measures, Forscher et al. found that implicit measures can be changed, but the type of approach used to change implicit measures mattered greatly. They also found little evidence that changes in implicit measures translated into changes in explicit measures and behavior. 

Seven researchers contributed to this publication, and Patrick S. Forscher and Calvin K. Lai are joint first authors. Dr. Forscher is a Research Scientist at Université Grenoble Alpes, and Dr. Lai is an Assistant Professor of Psychological & Brain Sciences at Washington University in St. Louis and chairs the Scientific Advisory Committee at Project Implicit. The study’s other authors—Jordan R. Axt, Charles R. Ebersole, Michelle Herman, Patricia G. Devine, and Brian A. Nosek—have extensive experience conducting and coordinating research on implicit bias and implicit social cognition.

Methods and Findings

To compare many different procedures that aim to influence implicit measures, the researchers imported a technique from the medical sciences called multivariate network meta-analysis. Since meta-analysis requires careful consideration of which studies are relevant to the research question, the researchers set a number of inclusion criteria. For example, the researchers excluded studies that were not written in English, and experimental procedures had to fit into a single procedure category to be included. Researchers created procedure categories iteratively to capture the breadth of approaches in the literature. The researchers also focused on randomized studies, which gave them an opportunity to go beyond correlational evidence and examine whether procedures that attempt to change implicit measures also produce change in explicit measures. Their final sample represented 87,419 participants and included 492 studies. More than half of the articles under study were published in 2011 or later, one-third were published from 2006-2010, and the remaining articles were published between 1995-2005. 

The main findings were:

• Implicit measures can be changed, but the effects are often relatively weak. 
• Approaches that changed implicit measures the most were those that invoked goals or motivations (such as the goal to weaken bias), associated sets of concepts (such as strengthening or weakening associations), or taxed mental resources (such as completing mentally effortful tasks during the implicit task). 
• Approaches that changed implicit measures the least were those that induced threat (such as threatening to put one’s integrity at risk), affirmation (such as giving feedback that a participant is moral or unbiased), or specific moods/emotions (such as anger or disgust). 
• Evidence from bias tests suggested that implicit effects could be inflated relative to their true population values. 

• Generally, the approaches under study produced trivial changes in behavior. Procedures changed explicit measures less consistently and to a smaller degree than implicit measures. 
• Changes in implicit measures did not mediate changes in explicit measures or behavior. 

Conclusions

The researchers described limitations in the studies’ generalizability. For example, most studies were conducted with samples whose demographic traits (students, mostly white and female) strongly resemble the make-up of American introductory psychology courses. Although the samples’ gender composition was not associated with the size of effects, both the racial composition of the samples and whether the samples were drawn from college student populations were. In future studies, the authors recommend directly testing whether effects are generalizable to other populations since combating social problems like racial discrimination requires broader sampling and exploration of how problems operate across settings. 

The research raised theoretical and empirical puzzles for the authors. To reconcile possible explanations for their findings, the researchers recommend developing a new paradigm. Ideally, the paradigm would involve an approach that produces a clear causal impact on the automatic associations underlying implicit measures, across multiple domains. The researchers recommend starting with domains in which implicit, explicit, and behavioral measures are more intercorrelated, such as political preferences, which would enable high-powered investigations. 

For practitioners seeking to address problems presumed to be caused by automatic associations, the results of this research present a challenge, since there was little evidence that change in implicit measures will result in changes in explicit measures or behavior. This is particularly true for the domains of intergroup bias, health psychology, and clinical psychology; results suggest that current interventions attempting to change implicit measures in these domains will not consistently change behavior. The authors note that innovations may yet reveal stronger evidence for the causal importance of implicit associations, and they stated their hope that researchers take their findings as a challenge to advance our understanding of human cognition.