Research analyst at Open Philanthropy. All opinions are my own.
Under typical decision theory, your decisions are a product of your beliefs and by the utilities that you assign to different outcomes. In order to argue that Jack and Jill ought to be making different decisions here, it seems that you must either:
Are you advancing one of these claims? If (1), I think you're directly disagreeing with the paper for reasons that don't just come down to how to approach decision making. If (2), maybe say more about why you propose Jack and Jill make different decisions despite having identical beliefs and preferences?
Anthropic shadow effects are one of the topics discussed loosely in social settings among EAs (and in general open-minded nerdy people), often in a way that assumes the validity of the concept
FWIW, I think it's rarely a good idea to assume the validity of anything where anthropics plays an important role. Or decision theory (c.f. this). These are very much not settled areas.
This sometimes even applies when it's not obvious that anthropics is being invoked. I think Dissolving the Fermi Paradox and Grabby aliens both rely on pretty strong assumption about anthropics that are easy for readers to miss. (Tristan Cook does a good job of making the anthropics explicit, and exploring a wide range, in this post.)
Oh, also, re the original paper, I do think that even given SSA, Teru's argument that Jack and Jill have equivalent epistemic perspectives is correct. (Importantly: As long as Jack and Jill uses the same SSA reference classes, and those reference classes don't treat Jack and Jill any differently.)
Since the core mechanism in my above comment is the correlation between x2 and the total number of observers, I think Jill the Martian would also arrive at different Pr(A) depending on whether she was using SSA or SIA.
(But Teru doesn't need to get into any of this, because he effectively rejects SSA towards the end of the section "Barking Dog vs The Martians" (p12-14 of the pdf). Referring to his previous paper Doomsday and objective chances.)
But this example relies on there just being one planet. If there are >1 planets, each with two periods, we are back to having an anthropic shadow again.
Let's consider the case with 2 planets. Let's call them x and y.
According to SSA:
Given A, there are 4 different possibilities, each with probability 1/4:
Let's say you observe yourself to be alive at time-step 2 on planet x.
Pr(x2|A) = 1/4*1/4 + 1/4*0 + 1/4*1/3 + 1/4*0 ~= 0.146
Given B, the probabilities are instead:
Pr(x2|B) = (9/10)^2*1/4 + 9/10*1/10*0 + 1/10*9/10*1/3 + 1/10*1/10*0 ~= 0.233
Pr(A|x2) = Pr(x2|A)Pr(A)/Pr(x2) = Pr(x2|A)Pr(A)/[Pr(x2|A)*0.5+Pr(x2|B)*0.5)] ~= 0.146*0.5/[0.146*0.5+0.233*0.5] ~= 0.385.
According to SIA:
Here, we can directly compute Pr(A|x2).
All x2 observers are:
The total sum of x2 measure in worlds where A is true is 0.5*1/4 + 0.5*1/4 = 0.25.
The total sum of x2 measure is 0.5*1/4 + 0.5*1/4 + 0.5*(9/10)^2 + 0.5*9/10*1/10 = 0.7
Pr(A|x2) = 0.25/0.7 ~= 0.357.
The difference would be somewhat larger with >2 planets. (But would never be very large. Unless you changed the SSA reference classes so that you're e.g. only counting observers at period 2.)
Also: The mechanism of action here is the correlation between there being a survivor alive at x2 and there being a greater number of total observers in your reference class. There are multiple ways to break this:
Overall: I don't think SSA-style anthropic shadows of any significant size are real. Because I think SSA is unreasonable, and because I think SSA with small/restrictive reference classes is especially unreasonable. And with large reference classes, it seems unlikely to me that there are large correlations between our possible historic demise and the total number of observers. (For reasons like the above two bullet points.)
I think Elon buying Twitter was a mess, but community notes is one of the best improvements to the info ecosystem I have seen.
Note that, according to wikipedia:
The program launched in 2021 and became widespread on X in 2023. Initially shown to U.S. users only, notes were popularized in March 2022 over misinformation in the Russian invasion of Ukraine followed by COVID-19 misinformation in October. Birdwatch was then rebranded to Community Notes and expanded in November 2022.
Elon bought Twitter in October 2022, after the program had already been online for a while. I don't know whether any important details changed after Elon joining, nor whether twitter already had plans to expand the program. So I don't know how much credit Elon should get here vs. the previous owners of Twitter.
And here’s the full list of the 57 speakers we featured on our website
That's not right: You listed these people as special guests — many of them didn't do a talk. Importantly, Hanania didn't. (According to the schedule.)
I just noticed this. And it makes me feel like "if someone rudely seeks out controversy, don't list them as a special guest" is such a big improvement over the status quo.
Here's one line of argument:
Edit: Oops, I accidentally switched to talking about "my on-reflection values" rather than "total utilitarian values". The former is ultimately what I care more about, though, so it is what I'm more interested in. But sorry for the switch.
What's the argument for why an AI future will create lots of value by total utilitarian lights?
At least for hedonistic total utilitarianism, I expect that a large majority of expected-hedonistic-value (from our current epistemic state) will be created by people who are at least partially sympathetic to hedonistic utilitarianism or other value systems that value a similar type of happiness in a scope-sensitive fashion. And I'd guess that humans are more likely to have such values than AI systems. (At least conditional on my thinking that such values are a good idea, on reflection.)
Objective-list theories of welfare seems even less likely to be endorsed by AIs. (Since they seem pretty niche to human values.)
There's certainly some values you could have that would mainly be concerned that we got any old world with a large civilization. Or that would think it morally appropriate to be happy that someone got to use the universe for what they wanted, and morally inappropriate to be too opinionated about who that should be. But I don't think that looks like utilitarianism.
Nice, I feel compelled by this.
The main question that remains for me (only paranthetically alluded to in my above comment) is:
I think the answer to the first question is probably "yes" if we look at a reference class that changes over time, something like R_t = "people alive at period t of development in young civilizations' history".
I don't know about the answer to the second question. I think R_t seems like kind of a wild reference class to work with, but I never really understood how reference classes were supposed to be chosen for SSA, so idk what SSA's proponents thinks is reasonable vs. not.
With some brief searches/skim in the anthropic shadow paper... I don't think they discuss the topic in enough depth that they can be said to have argued for such a reference class, and it seems like a pretty wild reference class to just assume. (They never mention either the term "reference class" or even any anthropic principles like SSA.)