In the summer of 2024, a teenager entered a Taylor Swift-themed dance class in Southport, a seaside town in north-west England, and went on a stabbing spree. He murdered three young girls and injured many more.
As news of the event spread, so did rumours, including the viral claim that the killer was a Muslim asylum seeker who had recently crossed the Channel by small boat. (In reality, the attacker was a British-born seventeen-year-old, the son of Christian Rwandan immigrants.) Shortly afterwards, parts of the UK erupted in violent riots. A crowd attacked a mosque in Southport. Some rioters tried to set fire to a hotel that was housing asylum seekers.
Why did so many people accept the baseless rumour?
Here is one simple answer: they wanted to believe it. As with many examples of popular misinformation, the rumour activated “motivated reasoning”, the psychological tendency to let our motivations and interests bias how we process information.
But why would anyone want to believe such a thing? Motivated reasoning is often understood as a kind of wishful thinking: our desires distort our model of the world, confusing our assessment of what is true with what we want to be true. In this case, however, such an analysis doesn’t make sense. Anti-Muslim, anti-asylum-seeker rioters don’t want children to be murdered by Muslim asylum seekers. Far from wishful thinking, the rumour describes what is presumably their worst nightmare.
Here is another plausible explanation: the rioters wanted other people to believe the rumour. They wanted the world to see that Muslim asylum seekers are sick, demonic, dangerous, evil. And this desire made them extremely receptive to the rumour—to accept it, spread it, and act on it.
This post will flesh out this conjecture and try to make it plausible. Human psychology, I will argue, involves a powerful but largely unconscious “social diffusion bias”: our brains evaluate information—ideas, claims, narratives, etc.—not only by how plausible they are but also by whether their diffusion would advance our goals.
This is a simple and, in some sense, fairly obvious idea. But I think it is greatly underrated in modern psychology and social science. Once you notice it, you start to see it everywhere: a hidden force of human social life warping politics, culture, and media.
Buyers and Sellers in the Market for Propaganda
Elsewhere, I have written about “propagandistic cognition”: our tendency to internalise beliefs that we benefit from spreading (i.e., “propagating”).
The idea goes back at least as far as Nietzsche, who observed that great deceivers owe their power to being “overcome by their belief in themselves” in the very act of deception. Hannah Arendt also explored something similar: “The more successful a liar is, the more likely it is that he will fall prey to his own fabrications.”
However, it was the evolutionary biologist Robert Trivers who turned these speculations into a developed model of self-deception. Whereas most theorists and philosophers throughout human history had modelled self-deception as inward-looking and defensive—we deceive ourselves to protect ourselves from painful truths—Trivers proposed a far more cynical and, I think, insightful model: it is our evolved desire to influence, persuade, manipulate, and deceive others that distorts our sincere beliefs about the world. We deceive ourselves the better to deceive others.
The idea has since been developed by many thinkers, including Richard Alexander, Philip Tetlock, Robin Hanson, Kevin Simler, Eric Funkhouser, David Pinsof, and Lionel Page.
The core intuition is simple. As savvy, ultra-social primates, our survival and success depend in large part on the actions of those around us, which are determined by what they believe. So, we have powerful motivations to shift these beliefs in ways that promote our interests—for example, to persuade others that we and our allies are trustworthy, that our goals are noble, and that our rivals are suspicious and contemptible.
Sometimes, this leads to simple lying and self-conscious propaganda, but in many cases, we internalise the messages we are motivated to propagate or advocate for. Our model of the world—the pictures in our heads—comes to reflect the images we want others to accept.
Much of the evidence for this view is rooted in inference to the best explanation: it offers an evolutionarily plausible, parsimonious explanation of the patterns we observe in human belief.
But happily, there is also some experimental evidence. For example, a study by Peter Schwardmann and colleagues shows that at international debating competitions, debaters’ sincere beliefs shift towards the side they are randomly assigned to argue for. Similarly, research by David Melnikoff and Nina Strohminger finds that sincere beliefs shift as soon as people—including professional lawyers—acquire incentives to advocate for a position.
Nevertheless, it recently occurred to me that none of this really makes much sense as a general account of biased belief. The problem is that most people aren’t actually advocates or “propagandists”, even in the deflationary sense of the word. That is, perhaps self-deception follows deception, but for most of the mass belief systems and narratives we want to explain, it is implausible that most people have strong motives to deceive or advocate for anything at all.
Consider, for example, the consistent finding that the vast majority of political content on social media comes from a small minority of users. In the marketplace for political content and contributions to the culture war, most people are buyers, not sellers.
However, the story I have told so far depends on the existence of active advocacy or persuasion goals. That is, it focuses on those “selling” ideas, not the majority of people merely buying them.
I suspect this is one reason why most people don’t find the core idea behind “propagandistic cognition” very plausible. The point is not that most people don’t feel like propagandists. After all, to the extent that we internalise beliefs we are motivated to propagate—that is, to the extent the process shapes sincere belief—it wouldn’t feel like propaganda. The worry is different: most people aren’t very interested in active persuasion or advocacy at all. They are consumers of information, not producers.
Social Diffusion Bias
The concept of “social diffusion bias” addresses this oversight. It says that people do not merely internalise the ideas and narratives they are motivated to spread. They—we—also instinctively evaluate ideas according to the anticipated benefits of their diffusion: that is, by whether other people accepting the ideas in question would help or hinder their goals, including not just their narrow self-interest but the success of the broader groups and causes they support.
This bias, if it exists, isn’t conscious. Although some people may explicitly calculate how they would be served by the diffusion of an idea, the bias would be a built-in, evolved feature of our cognitive machinery that produces intuitive appraisals: for example, that a claim feels plausible, important, or “delicious” (think of juicy rumours or satisfying dunks on an outgroup), or that it is dubious, irresponsible, or ugly.
In this sense, the bias would resemble the social learning biases proposed by cultural evolutionists, such as conformist or prestige bias, which exploit the prevalence of ideas (conformist bias) or their association with high-status people (prestige bias) as cues that influence what we attend to and learn.
However, whereas cultural evolutionists sometimes model social learners as overly passive sponges for pre-existing bodies of cultural information, social diffusion bias aligns with research stressing the highly strategic, goal-driven nature of culture as a domain of power and conflict among agents with competing interests. That is, it is suited to explaining why and how culture becomes the site of culture wars rather than a harmonious system of cooperation and social learning.
Duncan Stibbard-Hawkes has recently argued that humans are instinctive “cultural lobbyists”. We don’t just internalise whatever ideas and norms exist in our culture; as Manvir Singh and others argue, we push norms that advance our interests, which explains why extractive, self-serving norms and institutions pervade human history, and why the content of norms often reflects and varies with the balance of power in a given community.
The notion of social diffusion bias adds that the norms a community enforces are shaped by the ideas its members accept—by which categories, beliefs, rumours, narratives, and ideologies come to be seen as accurate, plausible, or legitimate. This gives people a powerful stake in the diffusion of specific ideas.
Importantly, the goals against which social diffusion bias leads people to evaluate ideas needn’t be selfish. For example, people have sincere moral and ideological goals that concern how others should behave. So, the moralist who wants to make the community behave more altruistically will likely be as gripped by the bias as the Machiavellian schemer only interested in preying on audiences.
Social diffusion bias is not motivated reasoning as traditionally understood. Although wanting something to be true and wanting it to be accepted by others can go together, they can just as easily come apart. If we would benefit from the spread of negative, catastrophising, alarmist narratives, the bias predicts that we will be drawn to them.
Of course, we don’t always accept ideas that would advance our goals if accepted by others. Daniel Dennett famously talked of “belief in belief” in the context of religion, and research by Will Gervais and colleagues suggests that even atheists intuitively judge that it would be better if people held religious beliefs. The fact that they remain atheists shows that social diffusion bias must be limited. As with biased belief more broadly, our capacity to become convinced of an idea is plausibly constrained by whether we can rationalise it.
Nevertheless, the bias can tip the balance in favour of belief when evidence is ambiguous, as it usually is in politics. And when many people share a preference for the spread of a belief, they can coordinate to enforce its public acceptance, which can in turn shape the evidence people are exposed to.
Politics, Culture, Media
As this last comment suggests, social diffusion bias is likely to be most consequential when the diffusion of an idea is judged to be beneficial not just by isolated individuals but by the kinds of factions, coalitions, and tribes that participate in politics and culture wars.
The Southport case with which I began may be an extreme example, but I think the basic structure is common.
