I agree with your central point that misinformation does not have the direct effect on polarization that is commonly assumed; however, there may be an indirect influence. It is true that it is hard to change people's mind, but it is also hard for most people to go against the grain. A few people screaming online can have an disproportional influence if other people think they represent a common view. This is what Todd Rose called collective illusions, i.e. we overestimate the prevalence of views that are not necessarily our own and shape our expressed opinions accordingly. Under that view, the goal of Russian propaganda (or any propaganda) is not to change minds but to keep the screaming going and thus "intimidate" people into confusion, silence or shifting their opinions to maintain their social status in their communities. This is not something you can ignore in a democracy.
Thank you for bringing up Todd Rose’s collective illusions - spot on. I agree it is an excellent means to conceptualize the goals of disinformation campaigns and the toxic impacts they intend to create.
Ties in neatly to the concept of reflexive control
I’m looking for empirical papers on the phenomenon you describe, but am having a hard time even though I recall reading the effect exists. If you know of research on this, would love a tip.
This is the first I've heard of "collective illusions" as a formalized phenomena, but it may be worth checking out "pluralistic ignorance" also.
They may amount to flip sides of the same coin, or they may represent compounding effects where collective illusions overestimate the distribution of contrary beliefs, and pluralistic ignorance embellishes the divergence of our private opinions and the strength of that same opinion held by others. We tend to "signal" at the level of magnitude we suspect others to have, which to onlookers (near and far) may feed into both illusions (of magnitude and distribution of more extreme opinions).
Well I am a biologist and an amateur when it comes to social sciences. Todd Rose cites evidence in his book Collective Illusions, but I have not kept the book. Maybe get his book or go on his website?
Back before I gave up writing on 'disinformation' and 'influence' because you do it better I used to observe that Russia's aims are best fulfilled by leftishist hysteria. Insofar as foreign actors might want to weaken societal cohesion in a target territory, the most useful message carriers are actually those who are most afraid of others believing it. All of the amplification is an own goal.
Yes, the huge amount of domestically produced misinformation leaves little or nothing for the Russians to do. But the long history of Soviet and then Russian attempts to sow distrust in US institutions might still have contributed a good bit to the current situation. At this point, a good Russian operative should just declare ‘mission accomplished’ and focus elsewhere.
For example, QAnon conspiracy theories spawned out of Pizzagate surrounding 2016 election and the hack of Podesta emails.
QAnon beliefs drive extremism and even terrorism, all sorts of cultism, held by nearly 1/4 of the adult US population. It is pernicious and widespread and incredibly damaging
"Pernicious" is the word I can get behind, but note also that the position of "QAnon beliefs drive extremism and even terrorism" is several approximations removed from pernicious influences. What "drives" extremism, in my telling, is the shared and volatile sentiments that are slowly shaped by the information ecosystem.
Personally, I consider the noise to drive the overall volatility more than the narratives drive the shaping of sentiments into something shared. Even setting aside the ethics of influence, I suspect the only effective way to persuade is to build out an alternative place to reside for those that you intend to persuade. No one will or should resign themselves to the societal "dog house" under the thumb of those not trusted to recognize some reasonably sufficient penitence and a seat at the table thereafter.
Our collective, territorial battle over being "less wrong" than our opposition is recognizably naive and counterproductive. "Rubbing their nose in it" is effectively just shitting the bed ourselves.
That QAnon beliefs drive extremism and terrorism is very well documented. These conspiracy theories are extreme and traumatizing, very much like a religious cult
Are these organic beliefs that developed from natural, domestic discourse, or were these seeded by a foreign intelligence service during a targeted influence campaign meant to degrade our political discourse and advantage a strong man candidate that would clean up horrible mess of crimes against children by the current political elite?
I’m pretty positive it was the latter and that the damage caused is enormous.
It is difficult to say which narratives that may be introduced inorganically today might find traction over the next few years.
But clear to me Russian intelligence services are skilled at this sort of society moving manipulation.
I should also note, without speaking for Dan, that what I just said includes what I consider to be his and others' insights he has shared here. For example, his post on why people even believe true things and others on "epistemic vigilance."
I will readily admit that QAnon beliefs seem staggeringly inorganic and absurd, but I also don't think that my intuition will properly capture what is "organic" in isolation.
For example, "natural, domestic discourse" is something to strive for, but to think that we had achieved it such that QAnon is maximally divergent strikes me as a convenient belief (to myself also). For example, take the "legal victories" of legalizing gay marriage and overturning Roe v Wade. It seems entirely organic that each had repercussions according to those whose wills and desires were not reflected in the decisions.
I suspect that human perspective-taking is organic for those in our circles, but that perspective-taking for those outside is proportionally less so with the "distance." I think that "liberals" are presumed to accept this on principle, and that our efforts, or lack thereof, signals some combination of this social distance and our efforts to expand our perspectival reach.
If we started exactly where we find ourselves now, the distance seems insurmountable. However, if we assume a good faith path to even outlandish opinions, then I suspect that recursively earned distrust will naturally and organically lead us there, especially with political and economic games of chess with necessarily binary legal results.
I suspect that if we focus instead on the center of their defiance, what it is that they are trying to protect, rather than the cascade of incestuous gullibilities that have been invoked to defend it, then there's a calm at the eye of their storm. And sometimes we have to review how our perspectives can spin out into inconsistency and favoritism to admit the calm at our center is afforded on our own, somewhat justified, somewhat chaotic defiance.
Perhaps acts of violence is a more appropriate way to frame it, but certainly some of the many violent incidents inspired by QAnon beliefs can be classified as terrorism…not least of which are the Jan 6 rioters that were motivated by these beliefs. Talk about damaging.
I say this as someone who often shits the bed in exactly this way. I just want to stop myself and to give others "permission" to stop themselves. And to boop me with a rolled up newspaper when I fail to stop myself.
One problem here is that the reaction to "RT funnelled around 10 million dollars into the company to influence its coverage" -- was 'Russians! Evil Foreign Russians!' rather than 'Journalism Profession for Sale -- Money gets what Money Wants'. Here is Tim Schwab writing criticism in the Columbia Journalism review: https://www.cjr.org/criticism/gates-foundation-journalism-funding.php
which talks of the ethical issues of having Billionaire philanthropists -- with a focus on Bill Gates -- bankrolling the news organisations. And the fact-checking organisations. You'd think that this was the more important story, but you understand why news organisations are reluctant to mention it.
Your arguments make a lot of sense to me. They almost seem like common sense after reading Mercier.
Thus, the resistance to your thesis in the comments surprises me. It seems like there is a strong desire to maintain the belief that Russian propaganda is much more dangerous than you understand.
More proof, I guess, of your claim that it's hard to get people to change their minds
You seem to suffer from a severe case of high - what Cohen, Stotland and Wolfe (1955), and subsequently built upon by Cacioppo and Petty (1982 & 1984), call - "need for cognition".
This is easily cured by you defending any nation, political party, and or religion.
My second paragraph here includes more or less all of humanity, proving I am inclusive, not discriminatory.
I agree with your central point that misinformation does not have the direct effect on polarization that is commonly assumed; however, there may be an indirect influence. It is true that it is hard to change people's mind, but it is also hard for most people to go against the grain. A few people screaming online can have an disproportional influence if other people think they represent a common view. This is what Todd Rose called collective illusions, i.e. we overestimate the prevalence of views that are not necessarily our own and shape our expressed opinions accordingly. Under that view, the goal of Russian propaganda (or any propaganda) is not to change minds but to keep the screaming going and thus "intimidate" people into confusion, silence or shifting their opinions to maintain their social status in their communities. This is not something you can ignore in a democracy.
Thank you for bringing up Todd Rose’s collective illusions - spot on. I agree it is an excellent means to conceptualize the goals of disinformation campaigns and the toxic impacts they intend to create.
Ties in neatly to the concept of reflexive control
I’m looking for empirical papers on the phenomenon you describe, but am having a hard time even though I recall reading the effect exists. If you know of research on this, would love a tip.
This is the first I've heard of "collective illusions" as a formalized phenomena, but it may be worth checking out "pluralistic ignorance" also.
They may amount to flip sides of the same coin, or they may represent compounding effects where collective illusions overestimate the distribution of contrary beliefs, and pluralistic ignorance embellishes the divergence of our private opinions and the strength of that same opinion held by others. We tend to "signal" at the level of magnitude we suspect others to have, which to onlookers (near and far) may feed into both illusions (of magnitude and distribution of more extreme opinions).
Well I am a biologist and an amateur when it comes to social sciences. Todd Rose cites evidence in his book Collective Illusions, but I have not kept the book. Maybe get his book or go on his website?
Thanks! Bought it
Back before I gave up writing on 'disinformation' and 'influence' because you do it better I used to observe that Russia's aims are best fulfilled by leftishist hysteria. Insofar as foreign actors might want to weaken societal cohesion in a target territory, the most useful message carriers are actually those who are most afraid of others believing it. All of the amplification is an own goal.
Yes, the huge amount of domestically produced misinformation leaves little or nothing for the Russians to do. But the long history of Soviet and then Russian attempts to sow distrust in US institutions might still have contributed a good bit to the current situation. At this point, a good Russian operative should just declare ‘mission accomplished’ and focus elsewhere.
I see things very differently
For example, QAnon conspiracy theories spawned out of Pizzagate surrounding 2016 election and the hack of Podesta emails.
QAnon beliefs drive extremism and even terrorism, all sorts of cultism, held by nearly 1/4 of the adult US population. It is pernicious and widespread and incredibly damaging
https://news.harvard.edu/gazette/story/2024/08/toll-of-qanon-on-families-of-followers/
"Pernicious" is the word I can get behind, but note also that the position of "QAnon beliefs drive extremism and even terrorism" is several approximations removed from pernicious influences. What "drives" extremism, in my telling, is the shared and volatile sentiments that are slowly shaped by the information ecosystem.
Personally, I consider the noise to drive the overall volatility more than the narratives drive the shaping of sentiments into something shared. Even setting aside the ethics of influence, I suspect the only effective way to persuade is to build out an alternative place to reside for those that you intend to persuade. No one will or should resign themselves to the societal "dog house" under the thumb of those not trusted to recognize some reasonably sufficient penitence and a seat at the table thereafter.
Our collective, territorial battle over being "less wrong" than our opposition is recognizably naive and counterproductive. "Rubbing their nose in it" is effectively just shitting the bed ourselves.
Apologies if I misunderstood your comment, no offense intended in my response. Cheers
That QAnon beliefs drive extremism and terrorism is very well documented. These conspiracy theories are extreme and traumatizing, very much like a religious cult
Are these organic beliefs that developed from natural, domestic discourse, or were these seeded by a foreign intelligence service during a targeted influence campaign meant to degrade our political discourse and advantage a strong man candidate that would clean up horrible mess of crimes against children by the current political elite?
I’m pretty positive it was the latter and that the damage caused is enormous.
It is difficult to say which narratives that may be introduced inorganically today might find traction over the next few years.
But clear to me Russian intelligence services are skilled at this sort of society moving manipulation.
I should also note, without speaking for Dan, that what I just said includes what I consider to be his and others' insights he has shared here. For example, his post on why people even believe true things and others on "epistemic vigilance."
I will readily admit that QAnon beliefs seem staggeringly inorganic and absurd, but I also don't think that my intuition will properly capture what is "organic" in isolation.
For example, "natural, domestic discourse" is something to strive for, but to think that we had achieved it such that QAnon is maximally divergent strikes me as a convenient belief (to myself also). For example, take the "legal victories" of legalizing gay marriage and overturning Roe v Wade. It seems entirely organic that each had repercussions according to those whose wills and desires were not reflected in the decisions.
I suspect that human perspective-taking is organic for those in our circles, but that perspective-taking for those outside is proportionally less so with the "distance." I think that "liberals" are presumed to accept this on principle, and that our efforts, or lack thereof, signals some combination of this social distance and our efforts to expand our perspectival reach.
If we started exactly where we find ourselves now, the distance seems insurmountable. However, if we assume a good faith path to even outlandish opinions, then I suspect that recursively earned distrust will naturally and organically lead us there, especially with political and economic games of chess with necessarily binary legal results.
I suspect that if we focus instead on the center of their defiance, what it is that they are trying to protect, rather than the cascade of incestuous gullibilities that have been invoked to defend it, then there's a calm at the eye of their storm. And sometimes we have to review how our perspectives can spin out into inconsistency and favoritism to admit the calm at our center is afforded on our own, somewhat justified, somewhat chaotic defiance.
Which terrorist acts have been very well documented to have been driven by QAnon conspiracies?
Perhaps acts of violence is a more appropriate way to frame it, but certainly some of the many violent incidents inspired by QAnon beliefs can be classified as terrorism…not least of which are the Jan 6 rioters that were motivated by these beliefs. Talk about damaging.
https://en.m.wikipedia.org/wiki/Timeline_of_incidents_involving_QAnon
I say this as someone who often shits the bed in exactly this way. I just want to stop myself and to give others "permission" to stop themselves. And to boop me with a rolled up newspaper when I fail to stop myself.
One problem here is that the reaction to "RT funnelled around 10 million dollars into the company to influence its coverage" -- was 'Russians! Evil Foreign Russians!' rather than 'Journalism Profession for Sale -- Money gets what Money Wants'. Here is Tim Schwab writing criticism in the Columbia Journalism review: https://www.cjr.org/criticism/gates-foundation-journalism-funding.php
which talks of the ethical issues of having Billionaire philanthropists -- with a focus on Bill Gates -- bankrolling the news organisations. And the fact-checking organisations. You'd think that this was the more important story, but you understand why news organisations are reluctant to mention it.
Your arguments make a lot of sense to me. They almost seem like common sense after reading Mercier.
Thus, the resistance to your thesis in the comments surprises me. It seems like there is a strong desire to maintain the belief that Russian propaganda is much more dangerous than you understand.
More proof, I guess, of your claim that it's hard to get people to change their minds
You seem to suffer from a severe case of high - what Cohen, Stotland and Wolfe (1955), and subsequently built upon by Cacioppo and Petty (1982 & 1984), call - "need for cognition".
This is easily cured by you defending any nation, political party, and or religion.
My second paragraph here includes more or less all of humanity, proving I am inclusive, not discriminatory.
Deradicalisation expert