Thanks so much for this very helpful post!

I'm a bit confused about your framing of the takeaway. You state that "reducing meat consumption is an unsolved problem" and that "we conclude that no theoretical approach, delivery mechanism, or persuasive message should be considered a well-validated means of reducing meat and animal product consumption." However, the overall pooled effects across the 41 studies show statistical significance w/ a p-value of <1%. Yes, the effect size is small (0.07 SMD) but shouldn't we conclude from the significance that these interventions do indeed work? 

Having a small effect or even a statistically insignificant one isn't something EAs necessarily care about (e.g. most longtermism interventions don't have much of an evidence base). It's whether we can have an expected positive effect that's sufficiently cheap to achieve. In Ariel's comment, you point to a study that concludes its interventions are highly cost-effective at ~$14/ton of CO₂eq averted. That's incredible given many offsets cost ~$100/ton or more. So it doesn't matter if the effect is 'small', only that it's cost-effective.

Can you help EA donors take the necessary next step? It won't be straightforward and will require additional cost and impact assumptions, but it'll be super useful if you can estimate the expected cost-effectiveness of different diet-change interventions (in terms of suffering alleviated).

Finally, in addition to separating out red meat from all animal product interventions, I suspect it'll be just as useful to separate out vegetarian from vegan interventions. It should be much more difficult to achieve persistent effects when you're asking for a lot more sacrifice. Perhaps we can get additional insights by making this distinction?

Hi, James! When it comes to assessing bednets vs therapy or more generally, saving a life vs happiness improvements for people, the meat eater problem looms large for me. This immediately complicates the trade-off, but I don't think dismissing it is justifiable on most moral theories given our current understanding that farm animals are likely conscious, feel pain, and thus deserve moral consideration. Once we include this second-order consideration, it's hard to know the magnitude of the impact given animal consumption, income, economic growth, wild animal, etc. effects. You've done a lot of work evaluating mental health vs life-saving interventions (thanks for that!), how does including animals impact your thinking? Do you think it's better that we should just ignore it (like GiveWell does)? 

I think this goes back to Joey's case for a more pluralistic perspective, but I take your point that in some cases, we may be doing too much of that. It's just hard to know how wide a range of arguments to include when assessing this balance...

Thanks, Vasco, for doing this analysis! Here are some of my learnings:

1. Fish and shrimp suffering is so much greater than that of other farm animals. I knew about their much larger numbers already, but I feel it much more viscerally now after better understanding the steps in your calculations and the assumptions behind them. 
2. The overall picture of neglectedness (i.e. disability vs funding) is insensitive to the way pain/disability is measured or to the assumptions behind animal moral value (e.g. welfare range). Unless you literally assume farm animals matter zero, any reasonable assumption will show how neglected farm animals are relative to humans. 
3. Wild animals have even greater neglectedness. We had initially discussed including them in the analysis, but they would dominate everything else, even farm animals. I had thought that limiting the analysis to certain types of animals (e.g. land vertebrates or just mammals) would result in farm animals being more prominent, but wild animals dominate even among just mammals.

Thanks, Ben, for writing this up! I very much enjoyed reading your intuition.

I was a bit confused in a few places with your reasoning (but to be fair, I didn't read your article super carefully).

1. Nvidia's market price can be used to calculate its expected discounted profits over time, but it can't tell us when those profits will take place. A high market cap can imply rapid short-term growth to US$180 billion of revenues by 2027 or a more prolonged period of slower growth to US$180B by 2030 or 2035. Discount rates are an additional degree of freedom. We can have a lower level of revenues (of not even US$180B) if we assume lower discount rates. CAPM isn't that useful since it's an empirical disaster, and there's the well-known fact that high-growth companies can have lower, not higher, discount rates (i.e. the value/growth factor). 
2. Analysts are forecasting very rapid growth for Nvidia's revenues and profits. You mention Jan-2025 fiscal year-end revenues of $110 billion. The same source has analyst expectations for Jan-2026 year-end revenues of $138 billion. Perhaps you can find analyst expectations that go even further but these are generally rare and unreliable. So you could say that analysts expect Nvidia's revenues of $138 billion in 2025 (ending Jan-26) and continue your analysis from there. However, analyst expectations are known to have an optimistic bias and aren't as predictive as market prices.
3. I was confused about how you used the 3-year expected life of GPUs within your analysis. It's irrelevant when it comes to interpreting Nvidia's market price since Nvidia's future sales pathway can't be inferred by how long its products last. The more appropriate link applies to when Nvidia's customers must have high sales levels given that Nvidia is selling its GPUs, say in 2025. If we add 3 (GPU life) to 2025 (last available year for analyst estimates), we get 2028 (not your 2027), with Nvidia's revenues at $138 billion based on analyst expectations (not your US180 billion based on the market price).
4. I wasn't sure why you needed to estimate 'consumer value' or 'willingness to pay.' This inflated your final numbers by 4x in your title of 'trillions of dollars of value.' And confusingly, it conflates how value is used in other parts of your article. Bringing in 'consumer value' is weird because it's not commonly calculated or compared in economics or finance. Value generally refers to that implied by market transactions, and this applies to well-known concepts like GDP, income, addressable market size, market value, sales, profits, etc (how you use it in most of your article). So we don't have a good intuition for what trillions of consumer surplus means, but, we do for hundreds of billions of sales.
5. So instead of ending with 'trillions of consumer value' for which there are no intuitive comparisons, it's better to end with x billions of sales (profits aren't reliable since high growth companies can go years and years without them, e.g. Amazon). You can then compare this with other historical episodes of industries/companies with high sales growth and see if this growth is likely/unlikely for AI. How fast did Internet companies, or the SaaS industry (software as a service), or Apple get to this level of sales? Is it likely (or not) that AI software companies can do the same within y years?
6. In case you haven't seen these, here are some related resources that might be useful. 1) Damadoran's valuation of Nvidia (from June 2023 so already dated given Nvidia's rapid growth), 2) Sequioa's talks on the large AI software potential (not much in terms of hard numbers but more for useful historic analogs), and 3) ARK's AI note from 2023 (self-promoting and highly optimistic but provides estimates for the AI software market in the many trillions by 2030). 

Thanks, Ben! I enjoyed reading your write-up and appreciate your thought experiment.

What concerns me is that I suspect people rarely get deeply interested in the moral weight of animals unless they come in with an unusually high initial intuitive view.

This criticism seems unfair to me:

1. It seems applicable to any type of advocacy. Those who promote global health and poverty are likely biased toward foreign people. Those who promote longtermism are likely biased toward future people. Those who advocate for effective philanthropy are likely biased toward effectiveness and/or philanthropy.
2. There's no effective counter-argument since, almost by definition, any engagement is possibly biased. If one responds with, "I don't think I'm biased because I didn't have these views to begin with," the response can always be, "Well, you engaged in this topic and had a positive response, so surely, you must be biased somehow because most people don't engage at all." It seems then that only criticisms of the field are valid. 
3. This is reminiscent of an ad hominem attack. Instead of engaging in the merits of the argument, the critique tars the person instead. 
4. Even if the criticism is valid, what is to be done? Likely nothing as it's unclear what the extent of the bias would be anyway. Surely, we wouldn't want to silence discussion of the topic. So just as we support free speech regardless of people's intentions and biases, we should support any valid arguments within the EA community. If one is unhappy with the arguments, the response should be to engage with them and make valid counterarguments, not speculate on people's initial intuitions or motivations.

Thanks so much for such a thorough and great summary of all the various considerations! This will be my go-to source now for a topic that I've been thinking about and wrestling with for many years.

I wanted to add a consideration that I don't think you explicitly discussed. Most investment decisions done by philanthropists (including the optimal equity/bond split) are outsourced to someone else (financial intermediary, advisor, or board). These advisors face career risk (i.e. being fired) when making such decisions. If the advisor recommends something that deviates too far from consensus practice, they have to worry about how they can justify this decision if things go sour. If you are recommending 100% equities and the market tanks (like it did last year), it's hard to say 'But that's what the theory says,' when the reflective response by the principal is that you are a bad advisor because you don't understand risk. Many advisors have been fired this way, and no one wants to be in that position. This means tilting toward consensus is likely the rational thing to recommend as financial advisors. There are real principal-agent issues at play, and this is something acutely felt by practitioners even if it's less discussed among academics. 

I suspect the EA community is subject to this dynamic too. It's rarely the asset owners themselves who decide the equity mix. Asset allocation decisions are recommended by OpenPhil, Effective Giving, EA financial advisors, etc. to their principals, and it's dangerous to recommend anything that deviates too far from practice. This is especially so when EA's philanthropy advice is already so unconventional and is arguably the more important battle to fight. It can be impact-optimal over the long term to tilt toward asset allocation consensus when not doing so risks losing the chance to make future grant recommendations. The ability to survive as an advisor and continue to recommend over many periods can matter more than a slightly more optimal equity tilt in the short term.

Keynes comes to mind: “Worldly wisdom teaches that it is better for reputation to fail conventionally than to succeed unconventionally.”

Thanks for posting this, Jonathan! I was going to share it on the EA Forum too but just haven't gotten around to it.

I think GIF's impact methodology is not comparable to GiveWell's. My (limited) understanding is that their Practical Impact approach is quite similar to USAID's Development Innovation Ventures' impact methodology. DIV's approach was co-authored by Michael Kremer so it has solid academic credentials. But importantly, the method takes credit for the funded NGO's impact over the next 10 years, without sharing that impact with subsequent funders. The idea is that the innovation would fail without their support so they can claim all future impact if the NGO survives (the total sum of counterfactual impact need not add to 100%). This is not what GiveWell does. GiveWell takes credit for the long-term impact of the beneficiaries it helps but not for the NGOs themselves. So this is comparing apples to oranges. It's true that GiveWell Top Charities are much more likely to survive without GiveWell's help but this leads to my next point.  

GiveWell also provides innovation grants through their All Grants Fund (formerly called Incubation Grants). They've been funding a range of interventions that aren't Top Charities and in many cases, are very early, with GiveWell support being critical to the NGO's survival. According to GiveWell's All Grants Fund page, "As of July 2022, we expect to direct about three-quarters of our grants to top charity programs and one-quarter to other programs, so there's a high likelihood that donations to the All Grants Fund will support a top charity grant." This suggests that in GiveWell's own calculus, innovation grants as a whole cannot be overwhelmingly better than Top Charities. Otherwise, Top Charities wouldn't account for the majority of the fund. 

When thinking about counterfactual impact, the credit one gets for funding innovation should depend on the type of future donors the NGO ends up attracting. If these future donors would have given with low cost-effectiveness otherwise (or not at all), then you deserve much credit. But if they would have given to equally (or even more) cost-effective projects, then you deserve zero (or even negative) credit. So if GIF is funding NGOs that draw money from outside EA (whereas GiveWell isn't), it's plausible their innovations have more impact and thus are more 'cost-effective'. But we are talking about leverage now, so again, I don't think the methodologies are directly comparable.

Finally, I do think GIF should be more transparent about their impact calculations when making such a claim. It would very much benefit other donors and the broader ecosystem if they can make public their 3x calculation (just share the spreadsheet please!). Without such transparency, we should be skeptical and not take their claim too seriously. Extraordinary claims require extraordinary evidence.

Thanks for your response, Joel!

Stepping back, CEARCH's goal is to identify cause areas that have been missed by EA. But to be successful, you need to compare apples with apples. If you're benchmarking everything to GiveWell Top Charities, readers expect your methodology to be broadly consistent with GiveWell's and their conservative approach (and for other cause areas, consistent with best-practice EA approaches). The cause areas that are standing out for CEARCH should be because they are actually more cost-effective, not because you're using a more lax measuring method.

Coming back to the soda tax intervention, CEARCH's finding that it's 1000x GiveWell Top Charities raised a red flag for me so it seemed that you must somehow be measuring things differently. LEEP seems comparable since they also work to pass laws that limit a bad thing (lead paint), but they're at most ~10x GiveWell Top Charities. So where's the additional 100x coming from? I was skeptical that soda taxes would have greater scale, tractability, or neglectedness since LEEP already scores insanely high on each of these dimensions.

So I hope CEARCH can ensure cost-effectiveness comparability and if you're picking up giant differences w/ existing EA interventions, you should be able to explain the main drivers of these differences (and it shouldn't be because you're using a different yardstick). Thanks! 

Hi Joel, I skimmed your report really quickly (sorry) but suspect that you did not account for soda taxes being eventually passed anyway. So the modeled impact of any intervention shouldn't be going to 2100 or beyond but out only a few years (I'd think <10 years) when soda taxes would eventually be passed without any active intervention. You are trying to measure the impact of a counterfactual donated dollar in the presence of all the forces already at play that are pushing for soda taxes (how some countries already have them). This makes for a more plausible model, and I believe is how LEEP or OpenPhil model policy intervention cost-effectiveness (I could be wrong though).

New phrasing works well!