An investigation of stereotype threat as an insight into the replication crisis

Abstract

This thesis explores the replication crisis, using stereotype threat as a demonstrative example. First, I conducted a quality control analysis using papers on stereotype threat’s effect on women’s math performance, to determine if methods and reporting had improved in the time period between 2001 and 2023. I did not find a significant improvement for the majority of the variables I tested, with the exception of sample size, average group size, and open data practices, which improved over the tested time period. Then, I designed and ran an experiment, testing the effect of implicit and explicit stereotype threat on women’s performance on a novel click accuracy task. Using this data, I first conducted a pervasiveness analysis, to determine the prevalence and magnitude of stereotype threat’s effect on participants’ scores. Within my sample, stereotype threat did not seem to cause a significant reduction in task performance compared to the control condition. Then, using Bayesian modelling, I conducted an exploratory analysis to test several ways that my results could be manipulated. First, I explored the results using four different outcome variables and found that the strength and confidence of my findings varied depending on this choice. Then, I tested the effect using two different pairs of experimenters, to determine if individual differences have an impact on the results. I found that individual experimenter variation can have a significant impact on the strength and direction of results, which may be misattributed to experimenter sex in some studies on stereotype threat. This thesis contributes to a body of work that aims to explore the causes of stereotype threat and suggests several methods for improving the quality of psychological research.