Can you share them?

Many heads are more utilitarian than one by Anita Keshmirian et al is an interesting paper I found via Gwern's site. Gwern's summary of the key points: 

• Collective consensual judgments made via group interactions were more utilitarian than individual judgments.
• Group discussion did not change the individual judgments indicating a normative conformity effect.
• Individuals consented to a group judgment that they did not necessarily buy into personally.
• Collectives were less stressed than individuals after responding to moral dilemmas.
• Interactions reduced aversive emotions (eg. stressed) associated with violation of moral norms.

Abstract: 

Moral judgments have a very prominent social nature, and in everyday life, they are continually shaped by discussions with others. Psychological investigations of these judgments, however, have rarely addressed the impact of social interactions.

To examine the role of social interaction on moral judgments within small groups, we had groups of 4 to 5 participants judge moral dilemmas first individually and privately, then collectively and interactively, and finally individually a second time. We employed both real-life and sacrificial moral dilemmas in which the character’s action or inaction violated a moral principle to benefit the greatest number of people. Participants decided if these utilitarian decisions were morally acceptable or not.

In Experiment 1, we found that collective judgments in face-to-face interactions were more utilitarian than the statistical aggregate of their members compared to both first and second individual judgments. This observation supported the hypothesis that deliberation and consensus within a group transiently reduce the emotional burden of norm violation. 

In Experiment 2, we tested this hypothesis more directly: measuring participants’ state anxiety in addition to their moral judgments before, during, and after online interactions, we found again that collectives were more utilitarian than those of individuals and that state anxiety level was reduced during and after social interaction.

The utilitarian boost in collective moral judgments is probably due to the reduction of stress in the social setting.

I wonder if this means that individual EAs might find EA principles more emotionally challenging than group-level surveys might suggest. It also seems a bit concerning that group judgments may naturally skew utilitarian simply by virtue of being groups, rather than through improved moral reasoning (and I say this as someone for whom utilitarianism is the largest "party" in my moral parliament). 

I think you're not quite engaging with Johan's argument for the necessity of worldview diversification if you assume it's primarily about risk reduction or or diminishing returns. My reading of their key point is that we don't just have uncertainty about outcomes (risk), but uncertainty about the moral frameworks by which we evaluate those outcomes (moral uncertainty), combined with deep uncertainty about long-term consequences (complex cluelessness), leading to fundamental uncertainty in our ability to calculate expected value at all (even if we hypothetically want to as EV-maximisers, itself a perilous strategy), and it's these factors that make them think worldview diversification can be the right approach even at the individual level.

Much appreciated, thanks again Vasco.

I don't see why acausal trade makes infinite ethics decision-relevant for essentially the reasons Manheim & Sandberg discuss in Section 4.5 – acausal trade alone doesn't imply infinite value; footnote 41's "In mainstream cosmological theories, there is a single universe, and the extent can be large but finite even when considering the unreachable portion (e.g. in closed topologies). In that case, these alternative decision theories are useful for interaction with unreachable beings, or as ways to interact with powerful predictors, but still do not lead to infinities"; physical limits on information storage and computation would still apply to any acausal coordination. 

I'll look into Wilkinson's paper, thanks.

Manheim and Sandberg address your objection in the paper persuasively (to me personally), so let me quote them, since directly addressing these arguments might change my mind. @MichaelStJules I'd be keen to get your take on this as well. (I'm not quoting the footnotes, even though they were key to persuading me too.)

Section 4.1, "Rejecting Physics":

4.1.1 Pessimistic Meta-induction and expectations of falsification

The pessimistic meta-induction warns that since many past successful scientific theories were found to be false, we have no reason expect that our currently successful theories are approximately true. Hence, for example, the above constraints on information processing are not guaranteed to imply finitude. Indeed, many of them are based on information physics that is weakly understood and liable to be updated in new directions. If physics in our universe does, in fact, allow for access to infinite matter, energy, time, or computation through some as-yet-undiscovered loophole, it would undermine the central claim to finitude.

This criticism cannot be refuted, but there are two reasons to be at least somewhat skeptical. First, scientific progress is not typically revisionist, but rather aggregative. Even the scientific revolutions of Newton, then Einstein, did not eliminate gravity, but rather explained it further. While we should regard the scientific input to our argument as tentative, the fallibility argument merely shows that science will likely change. It does not show that it will change in the direction of allowing infinite storage. Second, past results in physics have increasingly found strict bounds on the range of physical phenomena rather than unbounding them. Classical mechanics allow for far more forms of dynamics than relativistic mechanics, and quantum mechanics strongly constrain what can be known and manipulated on small scales.

While all of these arguments in defense of physics are strong evidence that it is correct, it is reasonable to assign a very small but non-zero value to the possibility that the laws of physics allow for infinities. In that case, any claimed infinities based on a claim of incorrect physics can only provide conditional infinities. And those conditional infinities may be irrelevant to our decisionmaking, for various reasons.

4.1.2 Boltzmann Brains, Decisions, and the indefinite long-term

One specific possible consideration for an infinity is that after the heat-death of the universe there will be an indefinitely long period where Boltzmann brains can be created from random fluctuations. Such brains are isomorphic to thinking human brains, and in the infinite long-term, an infinite number of such brains might exist [ 34]. If such brains are morally relevant, this seems to provide a value infinity.

We argue that even if these brains have moral value, it is by construction impossible to affect their state, or the distribution of their states. This makes their value largely irrelevant to decision-making, with one caveat. That is, if a decision-maker believes that these brains have positive or negative moral value, it could influence decisions about whether decisions that could (or would intentionally) destroy space-time, for instance, by causing a false-vacuum collapse. Such an action would be a positive or negative decision, depending on whether the future value of a non-collapsed universe is otherwise positive or negative. Similar and related implications exist depending on whether a post-collapse universe itself has a positive or negative moral value.

Despite the caveat, however, a corresponding (and less limited) argument can be made about decisionmaking for other proposed infinities that cannot be affected. For example, inaccessible portions of the universe, beyond the reachable light-cone, cannot be causally influenced. As long as we maintain that we care about the causal impacts of decisions, they are irrelevant to decisionmaking.

Section 4.2.4 more directly addresses the objection I think. (Unfortunately the copy-pasting doesn't preserve the mathematical formatting, so perhaps it'd be clearer to just look at page 12 of their paper; in particular I've simplified their notation for $1 in 2020 to just $1):

4.2.4 Bounding Probabilities

As noted above, any act considered by a rational decision maker, whether consequentialist or otherwise, is about preferences over a necessarily finite number of possible decisions. This means that if we restrict a decision-maker or ethical system to finite, non-zero probabilities relating to finite value assigned to each end state, we end up with only finite achievable value. The question is whether probabilities can in fact be bounded in this way.

We imagine Robert, faced with a choice between getting $1 with certainty, and getting $100 billion with some probability. Given that there are two choices, Robert assigns utility in proportion to the value of the outcome weighted by the probability. If the probability is low enough, yet he chooses the option, it implies that the value must be correspondingly high. 

As a first argument, imagine Robert rationally believes there is a probability of 10^−100 of receiving the second option, and despite the lower expected dollar value, chooses it. This implies that he values receiving $100 billion at approximately 10^100x the value of receiving $1. While this preference is strange, it is valid, and can be used to illustrate why Bayesians should not consider infinitesimal probabilities valid.

To show this, we ask what would be needed for Robert to be convinced this unlikely event occurred. Clearly, Robert would need evidence, and given the incredibly low prior probability, the evidence would need to be stupendously strong. If someone showed Robert that his bank balance was now $100 billion higher, that would provide some evidence for the claim—but on its own, a bank statement can be fabricated, or in error. This means the provided evidence is not nearly enough to convince him that the event occurred. In fact, with such a low prior probability, it seems plausible that Robert could have everyone he knows agree that it occurred, see newspaper articles about the fact, and so on, and given the low prior odds assigned, still not be convinced. Of course, in the case that the event happened, the likelihood of getting all of that evidence will be much higher, causing him to update towards thinking it occurred.

A repeatable experiment which generates uncorrelated evidence could provide far more evidence over time, but complete lack of correlation seems implausible; checking the bank account balance twice gives almost no more evidence than checking it once. And as discussed in the appendix, even granting the possibility of such evidence generation, the
amount possible is still bounded by available time, and therefore finite.

Practically, perhaps the combination of evidence reaches odds of 10^50:1 that the new money exists versus that it does not. Despite this, if he truly assigned the initially implausibly low probability, any feasible update would not be enough to make the event, receiving the larger sum, be a feasible contender for what Robert should conclude. Not only that, but we posit that a rational decision maker should know, beforehand, that he cannot ever conclude that the second case occurs. 

If he is, in fact, a rational decision maker, it seems strange to the point of absurdity for him to to choose something he can never believe occurred, over the alternative of a certain small gain. 

Generally, then, if an outcome is possible, at some point a rational observer must be able to be convinced, by aggregating evidence, that it occurred. Because evidence is a function of physical reality, the possible evidence is bounded, just as value itself is limited by physical constraints. We suggest (generously) that the strength of this evidence is limited to odds of the number of possible quantum states of the visible universe — a huge but finite value — to 1. If the prior probability assigned to an outcome is too low to allow for a decision maker to conclude it has occurred given any possible universe, no matter what improbable observations occur, we claim the assigned probability is not meaningful for decision making. As with the bound on lexicographic preferences, this bound allows for an immensely large assignment of value, even inconceivably so, but it is again still finite.

Thanks Vasco. While I agree with what I interpret to be your actionable takeaway (to ethically act as if our actions' consequences are finitely circumscribed in time and space), I don't see where your confidence comes from that the effects of one's actions decay to practically 0 after at most around 100 years, especially given that longtermists explicitly seek and focus on such actions. I'm guessing you have a writeup on the forum elaborating on your reasoning, in which case would you mind linking to it?

And there are, I think, hypotheses that can't be ruled out and that could involve infinite affectable value.

This is my crux, I think. I have yet to find a single persuasive example of an ethical decision I might face for which incorporating infinite ethics considerations suggests a different course of action. I don't remember if Carlsmith's essay provided any such examples; if it did I likely did not find them persuasive, since I skimmed it with this focus in mind. I interpreted Manheim & Sandberg's paper to say that I likely wouldn't find any such examples if I kept looking.

Just noting for my own future edification this LW exchange between David Manheim (who argues that infinite ethics is irrelevant to actual decisions, per paper above) and Joe Carlsmith (who argues the opposite, per essay above), which only increased my conviction that Manheim and Anders Sandberg were right.

FWIW here's Claude Sonnet 3.5 attempting to first steelman Carlsmith's essay and then being neutral about which stance is more right: 

Steelman of Carlsmith's key arguments:

1. Even if infinite value/disvalue isn't currently accessible, we should have non-zero credence in possibilities that would make it accessible (e.g. discovering our physics is wrong in specific ways). Given expected value theory, a non-zero credence in infinite value should dominate our calculations.
2. We face decisions with potentially infinite consequences even under current physics through acausal influence - if we accept certain decision theories and cosmological models suggesting infinite copies of us exist, our choices could influence an infinite number of copies.
3. Even if we can't access infinities in practice, ethics shouldn't fall silent when considering infinite cases. We have strong intuitions about choices like "infinite heaven vs infinite hell" that a complete ethical theory needs to address.
4. The mathematical/philosophical challenges infinite ethics raises reveal important limitations in our ethical frameworks that we should take seriously, even if just to better understand finite ethics.

Manheim/Sandberg's key counterarguments:

1. Under our best physics, the accessible universe is fundamentally finite in space, time, and computational capacity.
2. Any decision theory allowing infinite value requires rejecting either current physics, rational preferences, or established decision theory in problematic ways.
3. We can represent supposedly "infinite" values (like lexicographic preferences) with very large but finite numbers while preserving all decision-relevant distinctions.
4. Ethics should focus on actionable choices that make a difference to actual moral patients in our universe.

My analysis: While Carlsmith makes interesting theoretical points, Manheim/Sandberg seem more right for practical purposes. The key considerations:

1. Their physics arguments about fundamental finitude seem solid and would need extraordinary evidence to overturn.
2. Even if we assign some credence to infinite possibilities, we can't operationalize that credence in a way that helps real decision-making.
3. The practical ethical questions we face (e.g. existential risk, animal welfare, global poverty) don't require solving infinite ethics.
4. Time spent on infinite ethics likely trades off against work on pressing finite problems where we can make concrete progress.

So while infinite ethics raises fascinating theoretical questions, I lean towards Manheim/Sandberg's view that it's not practically relevant for "doing good better." That said, I appreciate Carlsmith's rigorous exploration of the theoretical challenges, which may yield insights for finite ethics.

The prudent path seems to be: Note the theoretical challenges infinite ethics raises, remain humble about our ethical frameworks, but focus our efforts on tractable finite problems where we can clearly help actual moral patients.

Just wanted to copy MacAskill's comment here so people don't have to click through: 

Though I was deeply troubled by the poor meater problem for some time, I've come to the conclusion that it isn't that bad (for utilitarians - I think it's much worse for non-consequentialists, though I'm not sure).

The basic idea is as follows. If I save the life of someone in the developing world, almost all the benefit I produce is through compounding effects: I speed up technological progress by a tiny margin, giving us a little bit more time at the end of civilisation, when there are far more people. This benefit dwarfs the benefit to the individual whose life I've saved (as Bostrom argues in the first half of Astronomical Waste). Now, I also increase the amount of animal suffering, because the person whose life I've saved consumes meat, and I speed up development of the country, which means that the country starts factory farming sooner. However, we should expect (or, at least, I expect) factory farming to disappear within the next few centuries, as cheaper and tastier meat substitutes are developed. So the increase in animal suffering doesn't compound in the same way: whereas the benefits of saving a life continue until the humanity race (or its descendants) dies out, the harm of increasing meat consumption ends only after a few centuries (when we move beyond farming).

So let's say the benefit to the person from having their live saved is N. The magnitude of the harm from increasing factory farming might be a bit more than N: maybe -10N. But the benefit from speeding up technological progress is vastly greater than that: 1000N, or something. So it's still a good thing to save someone's life in the developing world. (Though of course, if you take the arguments about x-risk seriously, then alleviating global poverty is dwarfed by existential risk mitigation).