This post offers some reasons why, on consequentialist grounds, a truth-seeking person with a primary goal of having a positive impact on the world might prioritize upholding certain norms that are widely held in their broader social and professional communities, even if the instrumental value of those norms isn’t obvious in the context in which they are applied.^[1]

One of my goals in writing this post is to help explain a subset of the reasons why I think some people in the EA community believe Nick Bostrom’s email and apology and some statements in the subsequent conversation have been inadequate or harmful: The original email, recent apology, and some subsequent comments have violated norms that are widely followed by people who aim to have a positive impact on the world, including a norm against asserting that there are meaningful genetic differences between groups of people in traits like intelligence.  My impression is that some people engaging here may not be aware of the reasons why this norm exists or the potential consequences of violating it, while other people may assume that others know and take these reasons and consequences for granted.  While this post does not address the object-level reasons why that norm exists,^[2] I hope that it helps explain some of the potential consequences of violating that norm and others like it.

I acknowledge that some people in the EA community prioritize a different norm—specifically, a norm in favor of communicating one’s own beliefs as accurately and directly as possible, sometimes called “epistemic integrity.”  I know many people sincerely believe that this norm is exceptionally important to the project of achieving a positive impact on the world, and I have tried to live up to it in writing this post.  I hope this post helps explain why others might believe that achieving a positive impact on the world (and even accurately communicating one’s own beliefs) sometimes requires giving priority to other norms.

Norms play an important role in coordination

Many ways of making a positive impact on the world depend on coordinating with other people.  One thing that is often important but hard to do when coordinating with other people is communicating accurate information about each other’s goals and beliefs.  

One way of communicating your goals and beliefs is to state them directly.  But norms also play an important role in communicating goals and beliefs.  When you follow a norm, you are providing evidence that your goals and beliefs are like those of other people who follow that norm.  Conversely, when you violate a norm, people might infer that your goals and beliefs are not like those of people who do follow the norm.^[3]

People who aim to have a positive impact on the world might uphold certain norms as one way of communicating about their goals and beliefs.  They might also draw negative inferences from norm-breaking behavior by others.

Examples of inferences others might draw from norm-breaking behavior

If a person violates a norm, sincere and truth-seeking people might conclude that they don’t know about the norm itself, from which they might infer:

• The person isn’t aware of important social or historical information underlying the norm, which might affect the person's ability to have a positive impact on the world.
• The person doesn't interact very much with people who could have taught them about the norm, which might signal that they lack social ties with (or perhaps even harbor prejudices against) a group of people who might be able to help them have a positive impact on the world.

Other sincere and truth-seeking people might conclude that a person who violates a norm does know about the norm, but has chosen not to follow it, from which they might infer:

• The person might not be trustworthy because they have shown that they are willing to violate an important norm; they might therefore be willing to violate other important norms like telling the truth and keeping promises.
• The person might be unpredictable and difficult to coordinate with because they have shown they are willing to make a “one-off,” context-specific decision that deviates from a general rule of behavior that many other people follow.
• The person might have other dangerous, difficult-to-observe traits that are more common among people who violate the norm than they are among people who consistently follow it.  (For example, the person might harbor racial animus or misogynistic beliefs.)
• The person’s value system might be different in important ways because they seem to place higher priority on the first-order consequences of the action they are taking than on the higher-order consequences that an important norm is intended to prevent.

Norms can also be valuable when following “single-player” strategies

Some norms function like ethical injunctions and serve to protect people from predictable mistakes that people tend to make when reasoning from first principles.  People who aim to form true beliefs about how to have a positive impact on the world might prioritize upholding these norms even when following a single-player strategy and not trying to coordinate with others.

As one example, a person might decide to prioritize upholding social norms about how to allocate their time between work responsibilities, family responsibilities, and leisure.  Such a person might hypothesize that social norms on that topic encode valuable information about the long-term consequences of working too much (or too little).  They might also reason that they do not have clear guiding principles or “stopping points” to protect them from making mistakes if they deviate from the norms and try to allocate their time from first principles.

As another example, a person might decide to uphold social norms against trying to reason about a certain subject from first principles.  Such a person might hypothesize that social norms encode valuable information about the risk of being misled by poor-quality evidence about that subject or the risk that inquiries into the subject could be misused to cause serious harm.  They might also take the social norm as evidence that, on that subject, a different style of reasoning might be more productive than trying to proceed from first principles.

A final word on strategies for upholding norms with compassion

I have offered a handful of reasons why people who aim to have a positive impact on the world should uphold certain norms.  But sometimes people worry that they will lose social or professional status or suffer other sanctions if they accidentally violate a norm.  This dynamic causes some people to experience anxiety about the whole topic of social and professional norms, even if they are trying in good faith to have a positive impact on the world.  

Responding to norm violations with compassion can help uphold important norms without contributing unnecessarily to this kind of anxiety and worry.  Toward that end, here are some things you could consider doing if you notice that someone has violated a norm that you think is important:

• Take the person who violated the norm aside and talk with them in private (e.g., through an email or direct message) about the norm.  Find out how much they know about the norm, including the reasons why it exists.  Point out the potential consequences of violating important norms for the person’s ability to achieve their goals.
• Make a positive case for the norm by explaining the purposes it serves, either privately to the person who violated the norm or publicly to a relevant community.^[4]
• For serious violations, ask for an apology that demonstrates that the person who violated the norm understands the reasons why their actions might have been harmful and gives readers reasons to think the person who violated the norm won’t do so again.^[5]
   

1. ^{^}

   Many people also endorse deontological arguments in favor of upholding certain norms, but this post does not focus on those arguments.

2. ^{^}

   If you are interested in learning more about this difficult subject, consider visiting this page from the U.S. National Human Genome Research Institute.

3. ^{^}

   Sometimes, people want to signal that their goals and beliefs are not like those of others who follow a certain norm.  However, before deciding to break a norm as a way of sending that signal, it is important to understand the goals and beliefs that others actually associate with the norm; they might not turn out to be the goals and beliefs you expect or intend to be communicating about.

4. ^{^}

   One goal of this post is to make this kind of positive, public case.

5. ^{^}

   I also liked the advice in this article about heartfelt apologies that titotal recently shared in another post.