Sorry for my delay, and thank you for posting this!

I have two main doubts:

1. Longtermism, at least as I understand it, doesn't actually depend on the credence assumption.
2. None of the proposed alternatives to expected value maximization fill its important role: providing a criteria of rightness for decision making under uncertainty.

1. Longtermism, at least as I understand it, doesn't actually depend on the credence assumption.

This piece focuses on criticizing the credence assumption. You make compelling arguments against it. But I don't care that much for the credence assumption (at least not since some of our earlier conversations)--I'm much more concerned with the claim that our willingness to make bets should follow the laws of probability. In other words, I’m much more concerned with decision making than with the nature of belief. Hopefully this doesn't look too goalpost-shifty, since I've tried to emphasize my concern with decision theory/bets since our first discussions on this topic.

You make a good case that we can just reject the credence assumption, when it comes to beliefs. But I think you'll have a much harder time rejecting an analogous assumption that comes up in the context of bets:

• Willingness to bet (i.e. the maximum ratio of potential gains to potential losses that one is willing to accept) should be a real-valued function.
  • More on this below, when we get to two-valued representations of uncertainty.

We can have expected value theory without Bayesian epistemology (i.e. we can see maximizing expected value as a feature of good decisions, without being committed to the claim that the probabilistic weights involved are our beliefs. Admittedly, this makes expected value not a great name). So refuting the psychological aspect of Bayesian epistemology doesn't refute expected value theory, which longtermism (as I understand it) does depend on.

2. None of the proposed alternatives to expected value maximization fill its important role: providing a criteria of rightness for decision making under uncertainty.

Maybe I should be more clear about what kind of alternative I'm looking for. Apologies for any past ambiguous/misleading communication from my end about this--my thinking has changed and gotten more precise. 

A distinction that seems very useful here is the distinction between criteria of rightness and decision procedures. In short, perhaps as a refresher:

• A criteria of rightness is a standard that an action/decision must meet to be good.
• A decision procedure is an algorithm (perhaps a fuzzily defined one) for making decisions.

Why are criteria of rightness useful? Because they are grounds from which we can evaluate (criticize!) decision procedures, and thus figure out which decision procedures are useful in what circumstances. 

A useful analogy might be to mountain-climbing (not that I know anything about mountains). A good criteria for success might be the altitude you've reached, but that doesn't mean that "seek higher altitudes" is a very useful climbing procedure. Constantly thinking about altitude (I'm guessing) would be distracting at best. Does that mean the climber should forget about altitude? No! Keeping the criteria of altitude in mind--occasionally glancing to the top of the mountain--would be very useful for choosing good climbing tactics, even (especially) if you're not thinking about it all the time.

I'm bringing this up because I think this is one way in which we've been talking past each other:

• I claim that expected value maximization is correct and useful as a criteria of rightness. When you suggest rejecting it, that leaves me looking for alternative criteria of rightness--looking for alternative answers to the question "what makes a decision right/good, if it's made under uncertainty?"
• You've been pointing out that expected value maximization is terrible as a decision procedure, and you've been proposing alternative decision procedures.

As far as I can tell, this post proposes alternate epistemologies and decision procedures, but it doesn't propose an alternative criteria of rightness. So the important role of a criteria of rightness remains unfilled, leaving us with no principled grounds from which to criticize decisions made under uncertainty or potential procedures for making such decisions.

Loose ends:

Loose end: problems vs paradoxes

Hence, paradoxes lurking outside bayesian epistemology are the reason one can never leave it, but paradoxes lurking inside are exciting research opportunities.

Nice, this one made me laugh

Loose end: paradoxes

Other paradoxes within bayesian epistemology include the Necktie paradox, the St. Petersburg paradox, Newcomb’s paradox, Ellsberg Paradox, Pascal’s Mugging, Bertrand’s paradox, The Mere addition paradox (aka “The Repugnant Conclusion”), The Allais Paradox, The Boy or Girl Paradox, The Paradox Of The Absent Minded Driver (aka the “Sleeping Beauty problem”), and Siegel’s paradox.

I'd argue things aren’t that bad.

• At least the St. Petersburg paradox, Newcomb’s problem, the “repugnant” conclusion, the Boy or Girl Paradox, and the Sleeping Beauty problem arguably have neat solutions:
  • [1], [2], [3], [4], [5].
• The Ellsberg and Allais “paradoxes” refute the claim that people are in fact perfect expected value maximizers, but they don’t refute a different claim: that people should be--and that we often roughly are--expected value maximizers (while being subject to cognitive biases like ambiguity aversion)

Also, to make sure we’re on the same page about this - many of these paradoxes (e.g. the Pasadena game) seem to be paradoxes with how probabilities are used rather than with how they’re assigned. That doesn’t invalidate your point, although maybe it makes the argument as a whole fit together less cleanly (since here you’re criticizing “Bayesian decision making,” if you will, while later you focus on criticizing Bayesian epistemology).

Loose end: supposed alternative decision theories

Despite [expected value theory’s] many imperfections, what explicit alternative is there?

Here are some alternatives.

Unless I'm missing something, none of these seem to be alternative decision theories, in the sense discussed above. To elaborate:

A two-valued representation of uncertainty, like the Dempster-Shafer theory, lets one express uncertainty about both A and -A

I have several doubts about this approach:

• It’s not an alternative decision theory
• It doesn’t seem to resolve e.g. problems with positive credences in infinite values
• To the extent that the sum of A and -A doesn’t equal one, Dutch book arguments still apply.
  • You argue compellingly that one can just drop the credence assumption to avoid Dutch book arguments in the context of beliefs, but--as I’ve tried to argue--it’s harder (and more important?) to avoid Dutch books in the context of decisions/bets.
    • We don’t need to keep theorizing here. To resolve this, please tell me the odds at which you’re willing to buy/sell bets that this will happen, and the odds at which you’re willing to buy/sell bets that it won’t happen. Then we (and your wallet) get to find out if the supposed laws of rationality depend on our assumptions :)

Alternative logics one might use to construct a belief theory include fuzzy logics, three-valued logic, and quantum logic, although none of these have been developed into full-fledged belief theories. 

Fascinating stuff, but many steps away from an alternative approach to decision making.

Making Do Without Expectations: Some people have tried to rehabilitate expected utility theory from the plague of expectation gaps.

You make a compelling case that these won’t help much.

Loose end: tools from critical rationalism

For important decisions, beyond simple ones made while playing games of chance, I use tools I’ve learned from critical rationalism, which are things like talking to people, brainstorming, and getting advice.

+1 for these as really useful things that expected value theory under-emphasizes (which is fine because EV is a general criteria of rightness, not a general decision procedure).

Loose end: authoritarian supercomputers

[...] So the question is: Would you always do what you’re told?

Do you really buy this argument? A pretty similar argument could get people riled up against the vile tyranny of calculators.

Apparent problems with this thought experiment:

• It trivializes what’s at stake in our decision making--my feelings of autonomy and self-actualization are far less important than the people I could help with better decision making.
  • See this for a similar sentiment. (I’m not endorsing its arguments.)
• I suspect much of my intuitive repulsion to this thought experiment comes from its (by design?) similarity to irrelevant real-world analogues, e.g. cult leaders who aren’t exactly ideal decision makers.
• Just because critical reasoning is being practiced by some reasoner other than me (by the supercomputer, in this hypothetical) doesn’t mean it’s not being practiced.

Loose end: evolutionary decision making

Yes, it’s completely true that decisions made according to this framework are made up out of thin air (so it is with all theories) - we can view this as roughly analogous to the mutation stage in Darwinian evolution. Then, once generated, we subject the ideas to as much criticism from ourselves and others as possible - we can view this as roughly analogous to the selection stage.

As I’ve tried to argue, my point is that, without EV maximization, we lack plausible grounds for criticizing decisions amidst uncertainty. Criticism of decisions made under uncertainty must logically begin with premises about what kinds of decisions under uncertainty are good ones, and you’ve rejected popular premises of this sort without offering tenable alternatives.