Meta level question:

How does Manifest have anything to do with Effective Altruism, and why is this on the EA forum?

Shouldn't this post be on some other other channel internal to Manifest and the forecasting community?

It get there are some people that went to Manifest that are also in the EA movement, but it seems like the communities are quite distinct and have different goals. From comments and conversations, it seems pretty clear to me that this Manifest community has a strong hostility towards even considering the reputational risks platforming racist speakers has on the rest of the EA movement. Part of being a big tent movement means caring about not stinking up the tent for everyone else.

Let's please firewall the Manifest community from EA?

I think that longtermism relies on more popular, evidenced-based causes like global health and animal welfare to do its reputational laundering through the EA label. I don't see any benefit to global health and animal welfare causes from longtermism. And for that reason I think it would be better for the movement to split into "effective altruism" and "speculative altruism" so the more robust global health and animal welfare causes areas don't have to suffer the reputational risk and criticism that is almost entirely directed at the longtermism wing.

Given the movement is essentially driven by Open Philanthropy, and they aren't going to split, I don't see such a large movement split happening. So I may be inclined towards some version of, as you say, "Stop doing stuff that looks weird, even if it is perfectly defensible by longtermist lights, simply because I have neartermist values and disagree with it." The longtermist stuff is maybe like 20% of funding and 80% of reputational risk, and the most important longtermist concerns can be handled without the really weird speculative stuff.

But that's irrelevant, because I think this ought to be a pretty clear case of the grant not being defensible by longtermist standards. Paying bay area software development salaries to develop a video game (why not a cheap developer literally anywhere else?) that didn't even get published is hardly defensible. I get that the whole purpose of the fund is to do "hits based giving". But it's created an environment where nothing can be a mistake, because it is expected most things would fail. And if nothing is a mistake, how can the fund learn from mistakes?

A butterfly flaps its wings and causes a devastating hurricane to form in the tropics. Therefore, we must exterminate butterflies, because there is some small probability X that doing so will avert hurricane disaster.

But it is just as easily the case that the butterfly flaps prevent devastating hurricanes from forming. Therefore we must massively grown their population.

The point being, it can be practically impossible to understand the casual tree and get even the sign right around low probability events.

That's what I take issue with - it's not just the numbers, it's the structural uncertainty of cause and effect chains when you consider really low probability events. Expected value is a pretty bad tool for action relevant decision making when you are dealing with such numerical and structural uncertainty. It's perhaps better to pick a framework like "it's robust under multiple decision theories" or "pick something that has the least downside risk".

In our instance, two competing plausible structural theories among many are something like: "game teaches someone an AI safety concept -> makes them more knowledgeable or inspire them to take action -> they work on AI safety -> solve alignment problem -> future saved" vs. "people get interested in doing the most good -> sees community of people that claim to do that, but that they fund rich people to make video games -> causes widespread distrust of the movement -> strong social stigma developed against people that care about AI risk -> greatly narrowed range of people / worldviews because people don't want to associate -> makes it near impossible to solve alignment problem -> future destroyed"

The justifications for these grants tend to use some simple expected value calculation of a singular rosy hypothetical casual chain. The problem is it's possible to construct a hypothetical value chain to justify any sort of grant. So you have to do more than just make a rosy casual chain and multiply numbers through. I've commented before on some pretty bad ones that don't pass the laugh test among domain experts in the climate and air quality space.

The key lesson from early EA (evidenced based giving in global health) was that it is really hard to understand if the thing you are doing is having an impact, and what the valence of the impact is, for even short, measurable casual chains. EA's popular causes now (longtermism) seem to jettison that lesson, when it is even more unclear what the impact and sign is through complicated low probability casual chains.

So it's about a lot more than effect sizes.

I do think there are things worth funding for which evidence doesn't exist. The initial RNA vaccine research relied on good judgement around a hypothetical, and had a hard time getting funding for lack of evidence. It ended up being critical to saving millions of lives.

I think there are more ways some sort of evidence can be included in grant making. But the core of the criticism is about judgement, and I think a $100k grant for 6 months of video game developers time, or $50k grants to university student group organizers represent poor judgement (EAIF and LTFF grants). These grants have caused reputational harm to the movement, and that should have been easy to foresee. What has been the hit to fundraising for EA global health and animal welfare causes from the fallout from bad longtermism bets (FTX/SBF included)?

On the rationalization. Perhaps it isn't a post-hoc rationalization, more of an excuse. It is saying "the funding bar was low, but we still think the expected value of the video game is more important than 25 lives". That's pretty crass. And probably worse than just the $100k counterfactual because of reputational spillover to other causes.

The post-hoc rationalization is referring to the "Note that this grant was made at the very peak of the period of very abundant (partially FTX-driven) EA funding where finding good funding opportunities was extremely hard."

If it wasn't a good opportunity, why was it funded?

Why does "infrastructure" and longtermist funding rely so heavily on pascal-mugging with evidence-free hypotheticals?

I can easily craft a hypothetical in the other direction on the video game. Perhaps funding such a game reinforces the impression that EA is a self-serving cult (as Steven Pinker does), causing more people to distance themselves from any longtermist ideas. It certainly has done so with me. Wasn't accounting for negative impacts the point of your recent post on the messiness of bring hypotheticals into the real world?

"I was a fan of Effective Altruism (almost taught a course on it at Harvard) together w other rational efforts (evidence-based medicine, data-driven policing, randomista econ). But it became cultish. Happy to donate to save the most lives in Africa, but not to pay techies to fret about AI turning us into paperclips. Still support the idea; hope they extricate themselves from this rut." - Steven Pinker

I think the pile-on of post-hoc rationalizations trying to defend or excuse this grant is evidence of the rot in EA in captured in Steven Pinker's comment. People are earnestly defending the idea that $100k on a bay area software salary for a speculative video game is worthy of the EA label. Can we at least all agree that this money would have been better spent by GiveWell?

Why is it so hard to say that the grant was a mistake not only in hindsight, but at the time it was made?

"A cost-effectiveness of decreasing GHG emissions of 3.41 tCO2eq/$, with a plausible range of 0.182 to 31.4 tCO2eq/$."

This is not a credible number, and Founders Pledge as of several years ago said they no longer stand behind the cost-effectiveness calculation you link to in your post.

It is based on an assumption that CATF nuclear advocacy will result in cheap enough reactors to replace coal in thermal electric power production. That is not credible now, and it wasn't at the time when the BOTEC was made. Note the 0.5%/1%/2% assumptions that nuclear will displace coal that are doing quite a lot of heavy lifting in getting the numbers to work out. The percentages are far lower than that. Be careful of arbitrary bounding your analysis in whole numbers between 1-100%. I made a copy of the sheet when it came out to capture any changes or alterations - my copy has some cells labeled that are missing in yours.

The supposed climate benefits of nuclear advocacy are contested, and far more credible and sophisticated modeling shows the possibility that there are some zero-sum trade-offs in scaling that mean more nuclear power could result in higher cumulative emissions. I see it as even odds whether CATF nuclear advocacy increases or reduces emissions in expectation. But extremely likely (>95%) that CATF's nuclear program was a total waste of philanthropic dollars.

The lesson is to be careful with BOTECs - use probability distributions instead of single-point numbers, and have several people red-team the analysis both in the numbers and the structure of the calculation.

Another issue is you are taking values derived from speculation and comparing them to measured cost-effectiveness from RCTs with a strong evidentiary basis.

Pointing to white papers from think tanks that you fund isn't a good evidentiary basis to support the claim of R&D's cost effectiveness. As with most things, the details matter quite a bit. The R&D benefit for advanced nuclear since the 1970s has yielded a net increase in price for that technology. For renewables and efficiency, the gains were useful until about the early 00s. After that, all the technology gains came from scaling, not R&D. You can't take economy wide estimates for energy R&D funding and apply them to a specific federal bill if the technology mix is quite different. And historic estimates are not necessarily indicative of future gains; we should expect diminishing returns.

Furthermore, most of the money in BIL and IRA were for demonstration projects - advanced nuclear, the hydrogen hubs, DAC credits. Notably NOT research and development. You make a subtle shift in your cost effectiveness table where you use unreviewed historic numbers on cost-effectiveness for research and development, and then apply that to the much larger demonstration and deployment dollars. Apples and oranges. The needs for low TRL tech are very different from high TRL tech.

Lastly, a Bill Gates retweet is not the humble brag you think it is. Bill has a terrible track record of success in energy ventures; he's uninformed and impulsive. Saying Bill Gates likes your energy startup is like saying Jim Cramer likes your stock. Both indicate a money-making opportunity for those who do the opposite.

(For those in the comments, you can track prior versions of these conversations in EA Anywhere's cause-climate-change channel).

1. Last time I checked, GG's still linked to FP's CATF BOTEC on nuclear advocacy. Yes, I understand FP no longer uses that estimate. In fact, FP no longer publishes any of its BOTECs publicly. However, that hasn't stopped you from continuing to assert that FP hits around $1/ton cost-effectiveness, heavily implying CATF is one such org, and its nuclear work being the likely example of it. The BOTEC remains in FP's control, and it has yet to include a disclaimer. Please stop saying you can hit $1/ton based on high speculative EV calcs with numbers pulled out of thin air. It is not credible and is embarrassing to those of us who work on climate in EA.

2. I never intended to assert that FP still endorses REDD+. Merely to point out that the 2018 FP analysis of REDD+ (along with CCS and nuclear advocacy) was a terrible basis for Will to use in WWOTF for the $1/ton figure. While FP no longer endorses REDD+, FP's recent reports contain all the same process errors that Lief points out about the 2018 report - lack of experience, over-reliance on orgs they fund, best guesses, speculation.

"This bravado carries over into the blunt advice that MacAskill gives throughout the book. For instance, are you concerned about the environment? Recycling or changing your diet should not be your priority, he says; you can be “radically more impactful.” By giving $3,000 to a lobbying group called Clean Air Task Force (CATF), MacAskill declares, you can reduce carbon emissions by a massive 3,000 metric tons per year. That sounds great.

Friends, here’s where those numbers come from. MacAskill cites one of Ord’s research assistants—a recent PhD with no obvious experience in climate, energy, or policy—who wrote a report on climate charities. The assistant chose the controversial “carbon capture and storage” technology as his top climate intervention and found that CATF had lobbied for it. The research assistant asked folks at CATF, some of their funders, and some unnamed sources how successful they thought CATF’s best lobbying campaigns had been. He combined these interviews with lots of “best guesses” and “back of the envelope” calculations, using a method he was “not completely confident” in, to come up with dollar figures for emissions reductions. That’s it.

Strong hyping of precise numbers based on weak evidence and lots of hedging and fudging. EAs appoint themselves experts on everything based on a sophomore’s understanding of rationality and the world. And the way they test their reasoning—debating other EAs via blog posts and chatboards—often makes it worse. Here, the basic laws of sociology kick in. With so little feedback from outside, the views that prevail in-group are typically the views that are stated the most confidently by the EA with higher status. EAs rediscovered groupthink."

Yeah, MacAskill and EA deserve the roasting on this one. FP's report from 6 years ago was a terrible basis for the $1/ton figure. MacAskill should have never used it in WWOTF. The REDD+ section proved to be widely inaccurate; it underestimates cost by at least 10x, 200x if you account for permanence. The nuclear power advocacy BOTEC was even worse. And FP and GG still reference it!