The Bayesian audit: evaluating the proportionality of scientific claims to evidence — a case study on social priming and walking speed

Abstract

Across psychology, bold empirical claims often outpace the evidential support on which they rest. The replication crisis has shown that statistical significance alone provides little guidance about what should rationally be believed. To address this gap between results and rhetoric, we introduce the Bayesian audit-a conceptual and normative framework, rather than a new statistical method, for evaluating whether scientific claims are proportionate to the strength of their evidence. The audit proceeds by identifying the claim, specifying priors, translating the empirical evidence into a likelihood-based measure, updating to obtain posterior belief, testing sensitivity, and synthesizing proportional conclusions. Applied to a well-known case in social psychology-the “elderly priming” study by Bargh et al. (1996)-the audit reveals that the original finding corresponds to only modest Bayesian evidence (Bayes factor ≈ 3). Under reasonable priors (0.05-0.20), the posterior probability that the effect is genuine remains below 0.5, and replication attempts provide limited additional evidential impact at the level of individual studies. The exercise illustrates how strong theoretical language can emerge from weak evidential shifts and how Bayesian reasoning can realign scientific communication with inferential logic. The framework is particularly relevant for psychological science, where replication failures often reflect over-claiming rather than data absence, and where proportional reasoning can help restore coherence between evidence and belief.