Great essay Dan. I agree we need an "epistemic civilizing process" (nice expression btw!). The question is, what does that look like?? I know that question is beyond the scope of this essay, but I'm sure you agree it's vital. Will you post about it in the future?
(For what it's worth my partial answer is in my paper last year, 'Human nature & the open society'. Far more deliberative fora.)
Yeah, such an important question! Will be writing about this more in the future. Like you, I think that distinctive social norms, conditions, and institutions *can* either overcome or leverage individual-level biases for collective epistemic benefits, but it's a really difficult question how this works and what specific kinds of interventions are helpful. (Big fan of your human nature and open society paper. I might also write something building off of it, now I think about it....)
I really like this essay. At the same time, I disagree with your central thesis. For the most part, social signaling with beliefs isn't harmful. It is the rare belief that is dangerous to society and most of our wisdom comes not from reasoned analysis of evidence as individuals, but from the collective knowledge we acquire through socialization into our culture. Humans do not have the intellectual or emotional capacity to reinvent the wheel. We have no choice but to adopt without question the vast majority of ideas from those who have come before us and those we trust in the present. Social signaling of the kind you describe here is valuable because it is a way of showing others that you are a member of the group and trust it's worldview even if you haven't had the ability to investigate every tenet of its beliefs yourself. Most importantly, it says, I'm not crazy and I'm not a threat. Those are messages we need to send to each other to keep society functioning on a daily basis.
Good points. Two things, though. (1) I think it's important to distinguish social learning and epistemic interdependence, which, as you say, are the source of collective knowledge and indispensable, from forming beliefs to signal traits and loyalties. It's the latter that I think is often bad. So I'm not arguing against the importance of human sociality in determining what and how we think. I'm arguing against one speciifc manifestation of this. (2) I agree that social signalling with our beliefs can be a way of showing that we're a group member that trusts a shared worldview. But I think there are other, less costly ways of signalling group membership, and often the worldview of one's group doesn't warrant complete trust. So all in all, I think I'm more optimistic than you about our ability to keep society functioning in the absence of social signalling with our beliefs.
Much of this seems spot on and a helpful corrective to the way these issues are often discussed these days. But I wonder if we need to distinguish between believing and appearing to believe. Some of the stuff you mention require people only appearing to believe certain things not necessarily believing them. The distinction would also make the explanation of some of the things you mention easier/simpler. If we assume that beliefs are not under voluntary control, then the explanation of some of the things you mention will have to posit various unconscious mechanisms whereas appearing to believe wouldn’t.
You may also find Yair and Huber's paper "How Robust Is Evidence of Partisan Perceptual Bias in Survey Responses?" interesting:
“Partisans often offer divergent responses to survey items ostensibly unrelated to politics….This study experimentally tests these techniques in replicating a study that found that partisanship affected attractiveness evaluations. These interventions, which allow partisans to express their partisan sentiments through other survey items, substantially reduce apparent partisan differences in beauty evaluations and show that standard survey items likely confound sincere partisan differences with elements of expressive responding.”
Yes great point. I'm going to be writing about this issue in the future. I'm inclined to think the difference between "really" believing something and merely pretending to believe it is often not very sharp, and that in many cases of socially adaptive "beliefs" people sit in an awkward place between. But working out the details of what's going on there, psychologically, is definitely a challenge....
That's a great line from Pinker about Russell's manifesto being revolutionary. Unfortunately, because it is indeed revolutionary- i.e. counter to human nature- I am quite pessimistic about the likelihood of success that mankind can ever epistemically civilise itself. It is the embodiment of a social technology that cannot scale. Human nature is what is is, and most people will always affiliate by false or irrational beliefs to signal their identities. Only a very small portion of humans are both interested in and capable of optimising for truth, and no one is truly immune to the tribal affiliation impulse which corrupts the pursuit of true belief. Even our contemporary social elites are defined by clearly irrational and objectively false beliefs.
In order for a civilising process to have any chance at success, it would require a paternalistic, restrictive social order in which the social incentives which shape popular beliefs are imposed on everyone top-down by an empowered intellectual elite- a Dictatorship of the Rationalists. (I'm tempted by the thought that we already kind of have this, but for progressive ideology)
I see you're already signalling membership to a tribe!
We tried "paternalistic and restrictive social orders" for centuries, they all disincentivize belief update. Free market liberal democracy is the way to go: "all that is solid melts into air, all that is holy is profaned, and man is at last compelled to face with sober senses, his real conditions".
I think this is right, but I think saying it this way may make beliefs seem like a conscious choice that people make and I doubt that's what it feels like subjectively to form a belief (and it gives people an out of saying "yeah, a lot of people seem to choose to believe weird things, luckily I never did that, I only believe good things that make sense"). I think the usual subjective experience of belief formation is just that the thing seems right. I like the taste of coffee, but I didn't always like it and not everybody does -- I didn't one day consciously decide to believe that coffee tastes good but I do believe it, it's just something that must have started to seem true to me at some point. I would guess that fashion works kind of the same way -- people don't consciously choose to believe that the clothes that beautiful, famous people wear look nice, it just feels like they do look nice. People (usually) aren't lying or deceiving when they have these contingent beliefs. From inside there usually wouldn't feel like a tradeoff between social signaling and other factors -- rich people probably genuinely like the way their rolexes look, getting one probably feels to them like an aesthetic decision not a social-signaling one -- the appeal of the rolex likely happened upstream of their conscious decision-making.
Good point. I agree this is not about conscious choice or decision making - that's not how belief works. I should have made that clearer in the piece, and said more about the psychological mechanisms involved if they don't involve choosing on the basis of conscious calculations of costs and benefits.
I don’t know how we consider people truly educated without them having been provided some good training on the many ways they can be led astray and how they can lead others astray from what is true. This seems like a necessary step in forming a robust social norm around real truth-seeking. A tricky problem for schools is that some of the best examples of unsupported beliefs (of a nationalist or religious nature for example) would be found within their core curriculum…
Your aspirational language and vision of a "civilizing process" toward the end makes it tempting to read you alongside David Pinsof as the (less cynical and extreme) good cop to his "Everything is Bullshit" bad cop, even if both of you come from a similar place. Two and a half for the price of one!
On a side note, I wonder if it's worth distinguishing a bit more between the individual-level social incentives elaborated here, and any system-level structural incentives or forces that might get collapsed into, or further shape, these individual social mechanisms. Not to absolve individuals of free will and moral responsibility (which I assume you still leave plenty of room for in your account), but to consider diverse environments and levels of scale. For instance, these status-approval games look somewhat different within one's workplace than someone's small town, or Facebook network, or as a citizen impacted by certain policies. Where to draw boundaries for the "space" (niche?) in which a person negotiates these beliefs could be tricky.
I'm less concerned than some others about any gap between what we think we believe and what we pretend to believe, since it's acting on these beliefs that makes them problematic regardless, and it's hard to draw the line anyway.
Finally: you make beautiful and convincing arguments for why we need to pay way more attention to these functional/social bases for beliefs and "moral" actions. Yet in claiming that "impression management also shapes the deep, inner contents of our minds: the authentic, subjective perspective through which we view the world," you seem to let beliefs stand in for the rest of our rich inner life. Surely beliefs are only the tip of a very large iceberg of subjective life, and sources of meaning? Ideally a "civilizing process" will also explore the following as they bear upon bad epistemic practices:
- Private dimension of "self" beyond social goals: personal/narrative identity; solitary activities; search for existential meaning and freedom. -
-"Higher" human needs and capacities: narrative; creativity and aesthetics/art; philosophical reflection, phenomenological experience, epistemic emotions like wonder and curiosity, critical consciousness.
-Individual differences in our sensitivity to various incentives and how relevant they are. E.g. autistic individuals are more literal, may either not care about social signals or rely on them in distorted ways; some people don't care what others think, some have zero power and access to reputation. It may thus be instructive to look at people on the margins who are less representative.
Great points. (I especially like the idea that I'm the good cop to David's bad cop....). A few things:
1. Completely agree about the importance of distinguishing interpersonal social incentives from those generated by broader systems and informational architectures. The interactions between the two are complex as well. I'll be writing more about this.
2. Agree re. the difference between belief and mere pretense. Action is ultimately what matters (and ofc sincere belief often follows action rather than preceding it).
3. You're right that beliefs aren't the entirety of our mental lives, of course. I think impressio management reaches deep into the contents of our minds, including our preferences, tastes, psychological dispositions, and so on. Definitely didn't mean to imply the claim (or broader project) is restricted to beliefs.
No mention of status and power? In modern history, don't status and power - real or not, perceived or not - explain a lot of woo woo beliefs?
As for people believing dumb stuff because valid information is hard to come by (and information is power, after all), unfortunately, the advent of the world wide web put an end to the problem of not having the right or enough information. The web confirmed that people are dumb.
You can lead people to information, you can't make them think.
Power, status, moral and social superiority, as well as timidity, and conformance with the norns of the group (big or small group), makes for a heady mix of drivers for much of human behavior - same can be seen in the animal kingdom, of course.
Broadly, cooperation is in there as well, the glue, but cooperation is invariably interwoven with power, status, etc.
Tight essay. Impressive links. Look forward to reading more on this: “What we need is a 21st-century civilizing process aimed not at people’s behaviours but at the appalling, self-serving, and groupish ways in which they frequently think and reason.”
Signaling with beliefs is good because it helps mobilize otherwise reluctant people for collective action. "We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness." Is this belief true? It probably isn't. But we don't complain about this particular instance of signaling with beliefs, do we? I'm not saying there aren't risk involved in signalling with beliefs. Marxism is a great example of that. All I'm saying is truly self-evident beliefs like "water is wet" is a poor basis for group identity and we need group identities to catalyze social change. I don't see any other way around that.
That's an interesting idea. I'm inclined to think we don't need unfounded beliefs to mobilize and create cooperative, effective communities - but I agree this is a good challenge. If you're right, the question then becomes how to distinguish between the good cases and the bad cases.
Thank you for your kind words! I would like to emphasize one thing: Nowhere have I stated that belief in the Creator, who endowed us with this specific list of unalienable rights, is entirely unfounded. I believe the authors of the document were completely justified in holding that belief. However, this doesn't make the belief true. It is surprisingly difficult to formulate a moral claim that meets the following criteria: 1) it is justified, 2) it offers a moderately novel perspective, if you're going for social change that's a requirement I guess, and 3) it is true.
These considerations are based on a meta-ethical standpoint where moral claims are regarded as factual and can be either true or false, recognizing there are numerous other possibilities.
Great essay Dan. I agree we need an "epistemic civilizing process" (nice expression btw!). The question is, what does that look like?? I know that question is beyond the scope of this essay, but I'm sure you agree it's vital. Will you post about it in the future?
(For what it's worth my partial answer is in my paper last year, 'Human nature & the open society'. Far more deliberative fora.)
Yeah, such an important question! Will be writing about this more in the future. Like you, I think that distinctive social norms, conditions, and institutions *can* either overcome or leverage individual-level biases for collective epistemic benefits, but it's a really difficult question how this works and what specific kinds of interventions are helpful. (Big fan of your human nature and open society paper. I might also write something building off of it, now I think about it....)
I really like this essay. At the same time, I disagree with your central thesis. For the most part, social signaling with beliefs isn't harmful. It is the rare belief that is dangerous to society and most of our wisdom comes not from reasoned analysis of evidence as individuals, but from the collective knowledge we acquire through socialization into our culture. Humans do not have the intellectual or emotional capacity to reinvent the wheel. We have no choice but to adopt without question the vast majority of ideas from those who have come before us and those we trust in the present. Social signaling of the kind you describe here is valuable because it is a way of showing others that you are a member of the group and trust it's worldview even if you haven't had the ability to investigate every tenet of its beliefs yourself. Most importantly, it says, I'm not crazy and I'm not a threat. Those are messages we need to send to each other to keep society functioning on a daily basis.
Good points. Two things, though. (1) I think it's important to distinguish social learning and epistemic interdependence, which, as you say, are the source of collective knowledge and indispensable, from forming beliefs to signal traits and loyalties. It's the latter that I think is often bad. So I'm not arguing against the importance of human sociality in determining what and how we think. I'm arguing against one speciifc manifestation of this. (2) I agree that social signalling with our beliefs can be a way of showing that we're a group member that trusts a shared worldview. But I think there are other, less costly ways of signalling group membership, and often the worldview of one's group doesn't warrant complete trust. So all in all, I think I'm more optimistic than you about our ability to keep society functioning in the absence of social signalling with our beliefs.
Much of this seems spot on and a helpful corrective to the way these issues are often discussed these days. But I wonder if we need to distinguish between believing and appearing to believe. Some of the stuff you mention require people only appearing to believe certain things not necessarily believing them. The distinction would also make the explanation of some of the things you mention easier/simpler. If we assume that beliefs are not under voluntary control, then the explanation of some of the things you mention will have to posit various unconscious mechanisms whereas appearing to believe wouldn’t.
You may also find Yair and Huber's paper "How Robust Is Evidence of Partisan Perceptual Bias in Survey Responses?" interesting:
“Partisans often offer divergent responses to survey items ostensibly unrelated to politics….This study experimentally tests these techniques in replicating a study that found that partisanship affected attractiveness evaluations. These interventions, which allow partisans to express their partisan sentiments through other survey items, substantially reduce apparent partisan differences in beauty evaluations and show that standard survey items likely confound sincere partisan differences with elements of expressive responding.”
Yes great point. I'm going to be writing about this issue in the future. I'm inclined to think the difference between "really" believing something and merely pretending to believe it is often not very sharp, and that in many cases of socially adaptive "beliefs" people sit in an awkward place between. But working out the details of what's going on there, psychologically, is definitely a challenge....
That's a great line from Pinker about Russell's manifesto being revolutionary. Unfortunately, because it is indeed revolutionary- i.e. counter to human nature- I am quite pessimistic about the likelihood of success that mankind can ever epistemically civilise itself. It is the embodiment of a social technology that cannot scale. Human nature is what is is, and most people will always affiliate by false or irrational beliefs to signal their identities. Only a very small portion of humans are both interested in and capable of optimising for truth, and no one is truly immune to the tribal affiliation impulse which corrupts the pursuit of true belief. Even our contemporary social elites are defined by clearly irrational and objectively false beliefs.
In order for a civilising process to have any chance at success, it would require a paternalistic, restrictive social order in which the social incentives which shape popular beliefs are imposed on everyone top-down by an empowered intellectual elite- a Dictatorship of the Rationalists. (I'm tempted by the thought that we already kind of have this, but for progressive ideology)
I see you're already signalling membership to a tribe!
We tried "paternalistic and restrictive social orders" for centuries, they all disincentivize belief update. Free market liberal democracy is the way to go: "all that is solid melts into air, all that is holy is profaned, and man is at last compelled to face with sober senses, his real conditions".
I think this is right, but I think saying it this way may make beliefs seem like a conscious choice that people make and I doubt that's what it feels like subjectively to form a belief (and it gives people an out of saying "yeah, a lot of people seem to choose to believe weird things, luckily I never did that, I only believe good things that make sense"). I think the usual subjective experience of belief formation is just that the thing seems right. I like the taste of coffee, but I didn't always like it and not everybody does -- I didn't one day consciously decide to believe that coffee tastes good but I do believe it, it's just something that must have started to seem true to me at some point. I would guess that fashion works kind of the same way -- people don't consciously choose to believe that the clothes that beautiful, famous people wear look nice, it just feels like they do look nice. People (usually) aren't lying or deceiving when they have these contingent beliefs. From inside there usually wouldn't feel like a tradeoff between social signaling and other factors -- rich people probably genuinely like the way their rolexes look, getting one probably feels to them like an aesthetic decision not a social-signaling one -- the appeal of the rolex likely happened upstream of their conscious decision-making.
Good point. I agree this is not about conscious choice or decision making - that's not how belief works. I should have made that clearer in the piece, and said more about the psychological mechanisms involved if they don't involve choosing on the basis of conscious calculations of costs and benefits.
I don’t know how we consider people truly educated without them having been provided some good training on the many ways they can be led astray and how they can lead others astray from what is true. This seems like a necessary step in forming a robust social norm around real truth-seeking. A tricky problem for schools is that some of the best examples of unsupported beliefs (of a nationalist or religious nature for example) would be found within their core curriculum…
Yep - this is a real problem...
Your aspirational language and vision of a "civilizing process" toward the end makes it tempting to read you alongside David Pinsof as the (less cynical and extreme) good cop to his "Everything is Bullshit" bad cop, even if both of you come from a similar place. Two and a half for the price of one!
On a side note, I wonder if it's worth distinguishing a bit more between the individual-level social incentives elaborated here, and any system-level structural incentives or forces that might get collapsed into, or further shape, these individual social mechanisms. Not to absolve individuals of free will and moral responsibility (which I assume you still leave plenty of room for in your account), but to consider diverse environments and levels of scale. For instance, these status-approval games look somewhat different within one's workplace than someone's small town, or Facebook network, or as a citizen impacted by certain policies. Where to draw boundaries for the "space" (niche?) in which a person negotiates these beliefs could be tricky.
I'm less concerned than some others about any gap between what we think we believe and what we pretend to believe, since it's acting on these beliefs that makes them problematic regardless, and it's hard to draw the line anyway.
Finally: you make beautiful and convincing arguments for why we need to pay way more attention to these functional/social bases for beliefs and "moral" actions. Yet in claiming that "impression management also shapes the deep, inner contents of our minds: the authentic, subjective perspective through which we view the world," you seem to let beliefs stand in for the rest of our rich inner life. Surely beliefs are only the tip of a very large iceberg of subjective life, and sources of meaning? Ideally a "civilizing process" will also explore the following as they bear upon bad epistemic practices:
- Private dimension of "self" beyond social goals: personal/narrative identity; solitary activities; search for existential meaning and freedom. -
-"Higher" human needs and capacities: narrative; creativity and aesthetics/art; philosophical reflection, phenomenological experience, epistemic emotions like wonder and curiosity, critical consciousness.
-Individual differences in our sensitivity to various incentives and how relevant they are. E.g. autistic individuals are more literal, may either not care about social signals or rely on them in distorted ways; some people don't care what others think, some have zero power and access to reputation. It may thus be instructive to look at people on the margins who are less representative.
Great points. (I especially like the idea that I'm the good cop to David's bad cop....). A few things:
1. Completely agree about the importance of distinguishing interpersonal social incentives from those generated by broader systems and informational architectures. The interactions between the two are complex as well. I'll be writing more about this.
2. Agree re. the difference between belief and mere pretense. Action is ultimately what matters (and ofc sincere belief often follows action rather than preceding it).
3. You're right that beliefs aren't the entirety of our mental lives, of course. I think impressio management reaches deep into the contents of our minds, including our preferences, tastes, psychological dispositions, and so on. Definitely didn't mean to imply the claim (or broader project) is restricted to beliefs.
No mention of status and power? In modern history, don't status and power - real or not, perceived or not - explain a lot of woo woo beliefs?
As for people believing dumb stuff because valid information is hard to come by (and information is power, after all), unfortunately, the advent of the world wide web put an end to the problem of not having the right or enough information. The web confirmed that people are dumb.
You can lead people to information, you can't make them think.
There is some mention of social status in the piece but perhaps you're right that there's not enough.
Power, status, moral and social superiority, as well as timidity, and conformance with the norns of the group (big or small group), makes for a heady mix of drivers for much of human behavior - same can be seen in the animal kingdom, of course.
Broadly, cooperation is in there as well, the glue, but cooperation is invariably interwoven with power, status, etc.
Tight essay. Impressive links. Look forward to reading more on this: “What we need is a 21st-century civilizing process aimed not at people’s behaviours but at the appalling, self-serving, and groupish ways in which they frequently think and reason.”
Thanks
Signaling with beliefs is good because it helps mobilize otherwise reluctant people for collective action. "We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness." Is this belief true? It probably isn't. But we don't complain about this particular instance of signaling with beliefs, do we? I'm not saying there aren't risk involved in signalling with beliefs. Marxism is a great example of that. All I'm saying is truly self-evident beliefs like "water is wet" is a poor basis for group identity and we need group identities to catalyze social change. I don't see any other way around that.
That's an interesting idea. I'm inclined to think we don't need unfounded beliefs to mobilize and create cooperative, effective communities - but I agree this is a good challenge. If you're right, the question then becomes how to distinguish between the good cases and the bad cases.
Thank you for your kind words! I would like to emphasize one thing: Nowhere have I stated that belief in the Creator, who endowed us with this specific list of unalienable rights, is entirely unfounded. I believe the authors of the document were completely justified in holding that belief. However, this doesn't make the belief true. It is surprisingly difficult to formulate a moral claim that meets the following criteria: 1) it is justified, 2) it offers a moderately novel perspective, if you're going for social change that's a requirement I guess, and 3) it is true.
These considerations are based on a meta-ethical standpoint where moral claims are regarded as factual and can be either true or false, recognizing there are numerous other possibilities.
Fair - and good point. These issues are complicated...
I think this article is good but surprisingly optimistic. Cf. https://slatestarcodex.com/2014/09/10/society-is-fixed-biology-is-mutable/