At Animal Charity Evaluators, we find and promote the most effective ways to help animals. We use effective altruism principles to evaluate causes and research.
We would like to extend our gratitude to Giving What We Can (GWWC) for conducting the "Evaluating the Evaluators" exercise for a second consecutive year. We value the constructive dialogue with GWWC and their insights into our work. While we are disappointed that GWWC has decided not to defer to our charity recommendations this year, we are thrilled that they have recognized our Movement Grants program as an effective giving opportunity alongside the EA Animal Welfare Fund.
After reflecting on GWWC’s 2023 evaluation of our Movement Grants (MG) program we made several adjustments, all of which are noted in GWWC’s 2024 report. We’re delighted to see that the refinements we made to our program this year have led to grantmaking decisions that meet GWWC’s bar for marginal cost-effectiveness and that they will recommend our MG program on their platform and allocate half of their Effective Animal Advocacy Fund to Movement Grants.
As noted by GWWC, ACE’s MG program is unique in its aims to fund underserved segments of the global animal advocacy movement and address two key limitations to effectiveness within the movement:
For impact-focused donors seeking opportunities to build an evidence-based and resilient global animal advocacy movement, our Movement Grants program is an effective giving opportunity which supports brilliant animal advocates all over the world.
Alongside their recommendation of our MG program, GWWC has outlined several areas for improvement that we are grateful for and will reflect on.
We agree with these suggestions by GWWC:
Improving the documentation of our reasoning for making grant decisions—this is mainly related to our internal processes that don’t have any bearing on our grant decisions; however, we agree with GWWC that despite our diligent record keeping for how our thinking evolves through the grant review process, we need a better record that summarizes, in one place, the main rationale and cruxes for each grant decision. This is something we intend to implement in our next granting round.
We also want to note the following challenge:
GWWC recommends that we introduce a clearer framework for prioritizing between interventions. We agree with this recommendation—of the possible interventions available to help animals, some are already excluded from applying or rejected at an early stage from our grantmaking based on the scope of impact. However, while we intend to make further improvements in scope comparison between interventions, there remain challenges due to the many externalities that affect intervention effectiveness and our ability to estimate them. GWWC notes in the report that we appear resistant to doing this because it would be unhelpfully speculative. We want to clarify that we are willing to make speculations where we think they will be useful while highlighting the challenges. There is a difference between the more speculative forward-looking cost-effectiveness analyses (CEAs) we would be undertaking as a grantmaker compared with the CEAs the Charity Evaluations team undertakes on completed work. To overcome this, we may also consider comparing the known cost-effectiveness of the most similar organizations’ previous work or leveraging CEAs that have used the total available information on an intervention (e.g. this estimate). However, we remain cautious about spending time and resources on trying to find comparable cost estimates between interventions when doing so might not sufficiently increase the overall marginal cost-effectiveness of our grant decisions. We also want to note that this is a challenge for any animal advocacy funder, not just ACE. This is an area where we expect to continue to try different approaches and improve year-on-year, balancing available information and our team’s capacity.
We are grateful for the rare opportunity to reflect deeply on our work and to learn from GWWC’s perspectives so that we can award grants that are the most impactful for animals. We are especially thankful too for the larger GWWC and EA community that is willing to support highly promising projects to help some of the most neglected individuals who suffer greatly.
On the other hand, we are disappointed that GWWC does not find our Charity Evaluations program justifiably competitive with MG (and the EA Animal Welfare Fund) and believe that donors might miss out on some of the most impactful donation opportunities because of GWWC’s decision. We will elaborate on the relationship between our Charity Evaluations and Movement Grants below, but first address some points specific to Charity Evaluations.
We agree with some of GWWC’s conclusions and suggestions for improvement, which appear in their 2024 report. We think that focusing on these will improve the quality of our recommendations moving forward:
However, there are also areas where we disagree with GWWC’s conclusions. While we acknowledge these parts of our methods have room for improvement, we think the changes that they suggest may not make a meaningful difference to the quality of our recommendations:
GWWC also suggests some broader strategic shifts in our programs that we plan to consider as a part of upcoming strategic planning. While these changes would help align our Charity Evaluations program more closely with GWWC’s criteria, we’re currently unsure if they would do the most good for the animal advocacy movement and for animals. These include:
Moving forward, we will continue evolving the Charity Evaluations program to find the organizations that can do the most good with additional donations and we thank GWWC for critically engaging with our work. We also appreciate that they acknowledge the difficulties of our work and the inherent differences between evaluators and funders in the animal advocacy space. However, given those difficulties, we’re currently not sure whether ACE nor any other evaluator that recommends whole charities in this space would be seen by GWWC as competitive with charitable funds that give restricted grants. Because of this, we’re not sure whether we think that GWWC’s current approach leads to the best outcomes for animals.
We want to acknowledge that the language GWWC uses implies that they see our Movement Grants (MGs) and Charity Evaluations programs as “competitive” with each other. This is not a view we share—we see them as complementary.
Although there’s some overlap between charities that are a fit for each program, they serve different purposes:
The programs are complementary and supplement each other:
They also serve different donors:
We are proud to support all of our current Recommended Charities and Movement Grantees, and would like to take this opportunity to celebrate the impactful work they do to help make the world a kinder place for animals.
- ACE Team
Hi Isaac! Now that we’ve announced our 2024 Recommended Charities, we’ve had more time to process your feedback. Thanks again for engaging with our work.
As mentioned before, we’ve substantively updated our evaluation methods this year. This was informed in part by detailed feedback we received as part of Giving What We Can’s 2023 ‘Evaluating the Evaluators’ project, some of which aligns with your feedback.
One of these changes is that we now seek to conduct more direct cost-effectiveness analyses, rather than the 1-7 scoring method that we used last year. This more direct approach is possible in part thanks to Ambitious Impact’s recent work to allow quantification of animal suffering averted per dollar. Of course, these kinds of calculations are still extremely challenging, limited, and subject to significant uncertainties; we describe our methods and their limitations on our website. For example, while cost-effectiveness = impact divided by cost, it can be difficult to measure impact meaningfully in a way that is also quantifiable, so we rely on other criteria to help us make our assessments.
Another major change was introducing a formal Theory of Change assessment to understand the reasoning, evidence base, and limitations around each charity’s main programs. In our 2023 Evaluations, we discussed these considerations in our Recommendations Decisions meetings but did not systematically incorporate them into our public reviews. Together, we think these changes allow for a more nuanced assessment of charities’ work and (we hope) more informative and accessible reviews.
Regarding the impact of our recommendations, this year, we conducted an assessment of ACE’s programs and our counterfactual influence on funding. As part of this work, we surveyed donors to our Recommended Charity Fund (RCF) and asked them where they’d donate if ACE didn’t exist. This indicated that over 60% of our RCF donors would donate less to animal charities if ACE were not to exist, of whom around 12% would not donate to animal charities at all. We aim to publish these influenced-giving reports on November 29th. We hope this reassures you that animals are not worse off because of ACE’s charity recommendations.
In terms of your specific feedback on last year’s methodology:
Thanks again for your engagement with our evaluations. We hope you get in touch with us directly if you come across new evidence-based methods to meaningfully capture cost-effectiveness or to improve the evaluation of animal charities. We might also reach out to you via email in the coming weeks as we go through retrospectives and plan for next year’s evaluation. Because of the complexity of the animal welfare cause area, the many uncertainties and knowledge gaps in the field of charity evaluation, and the urgency and scope of suffering, we embrace productive collaboration.
Thank you.
- The ACE team
Hi Pablo, important question! GFI decided to postpone re-evaluation to a future year to allow their teams more time to focus on new opportunities and challenges in the alternative proteins sector. Our researchers will be available to answer more questions about our 2024 charity recommendations in our AMA next week (Nov 19, 8-10am PT) on the FAST Forum. We hope to catch you there! Thanks, Holly
Hi Michael, I'm glad you're happy to see the cost-effectiveness models! And thank you for letting us know; there's been much more traffic than we're used which has made our website slow. We're actively trying to resolve this. Please hold on a bit while we get our site back up. Clearing your cache may also help :) Thanks for your patience everyone!
Thank you for spending time analyzing our methods. We appreciate those who are willing to engage with our work and help us improve the accuracy of our recommendations and reduce animal suffering as much as possible.
Based on previously received feedback and internal reflection, we have significantly updated our evaluation methods in the past year and will be publishing the details next Tuesday when we release our charity recommendations for 2024. From what we can tell from a quick skim, we think that our changes largely address Vetted Causes’ concerns here, as well as the detailed feedback we received last year from Giving What We Can (see also our response at the time) as part of their program that evaluates evaluators. Our cost-effectiveness analyses no longer use achievement or intervention scores, but rather directly calculate cost-effectiveness by dividing impact by cost, as you suggest. That being said, our work will never be perfect so we invite anyone reading this with the expertise to improve the rigor of our work to reach out, now or in the future.
Although your comments are related to methods that we no longer use, we’d like to spend more time understanding and engaging with them, learning from them, and potentially correcting any misconceptions. Unfortunately, we won’t have the opportunity to do so until after our charity recommendations are released next week. Additionally, it might be a comfort to know that for the past few months, Giving What We Can has been assessing ACE’s new evaluation methods along with a panel of other experts and that they intend to publish the results later this month.
Thank you.
- The ACE team
Hi Michael, thanks a lot for the helpful comments, and for taking the time to be so thorough in your feedback. We've been thinking a lot about how to produce proxies for impact that can be meaningfully compared with one another, with BOTECs being one possible way to help achieve that, so it's really useful to get your views. We'll talk these through as a team as we consider improvements to our process for the coming years.
- Max
Thanks for the kind words! Really glad to hear you're likely to support the great work being done by our Recommended Charities.
Like you say, involvement with EA is a hard thing to judge: I wouldn't feel qualified to name directors that I believe are involved in EA, for example. Also, while all the charities we recommend use evidence-driven strategies to achieve the maximum benefits for animals, many of them might not consider themselves EA, or might not want to be labelled as such for strategic reasons. In answer to your more specific question, two of our Recommended Charities were incubated by Charity Entrepreneurship: Fish Welfare Initiative and Shrimp Welfare Project.
As it sounds like you're already aware, we don't assess 'EA alignment' as part of our evaluations. In part, this is because we think it's very likely that the animal advocacy movement needs to be pluralistic if it's going to bring about long-term systemic change, so we want to support a wide range of organizations rather than limiting ourselves to a specific sub-set. That said, we're also very aware of the risk of personal biases affecting our assessments and the need to mitigate that risk, which is one of the reasons that we seek to quantify our decision-making as much as possible and to be transparent about all of our methodology and decision-making. It might also be helpful to know that not all of our researchers identify as EA.
If this is something you're still concerned with, the best option for you might be to give to ACE's Recommended Charity Fund instead of to a single charity. Then your donation will be distributed among each of our recommended charities. That's also the easiest option if you want to simplify your giving while supporting the diversity of solutions that we believe are necessary to reduce animal suffering effectively. And donations to this fund will be matched if made by December 6!
I hope that's partway helpful despite not answering your specific question, and thanks again for engaging with our work and considering supporting our Recommended Charities' excellent work.
- Max
Please find more information on animal suffering below:
Farmed animal suffering
Trillions of farmed animals (including fishes) experience cruel treatment and brutal deaths on factory farms. Farmed animal advocacy presents one of the most significant opportunities to reduce animal suffering on a large scale.
Chickens
Chickens are among the most abundant farmed animals, outranked only by farmed fishes and certain invertebrates, with an estimated 75 billion killed annually. When given enough space and an appropriate environment, chickens will forage for food, form nests and roosts, preen their feathers, and socialize with their flock. In contrast, chickens on factory farms live their entire lives in cramped, barren battery cages or crowded together in a shed with thousands of others, with little to no room to turn around or stretch their wings.
Fishes
As detailed in our Farmed Fish Welfare Report, existing research on fish welfare is limited, and the research that has been done tends to focus more on maximizing benefits for farms rather than fishes. While we believe that more research into promising ways to help farmed fish species is needed, we still believe that fish welfare should be prioritized within the broader effective animal advocacy movement. This is due to the current neglectedness of farmed fish animal welfare, the viability of potential strategies to improve farmed fish welfare, and the immense scale of fish farming—roughly 124 billion fishes are farmed in the world annually, with an additional 1.56 trillion caught (mid point) in the wild each year, not including bycatch and discards.
Shrimps
It is estimated that up to 518 billion (upper bound) farmed crustaceans are killed for food each year worldwide, yet less than 1% of animal welfare funding helps them. Unlike other invertebrates, it is more widely accepted that decapod crustaceans (e.g., shrimp, crabs, lobsters) and cephalopod mollusks (e.g., octopus, squid) are sentient creatures. In 2022, the sentience of both animal groups was officially recognized in U.K. law—an extremely promising development for organizations working to improve welfare conditions in aquatic animal farms. However, there is still significant room for more funding in this highly neglected area.
Pigs
Pigs are highly sentient animals capable of experiencing complex emotions, yet they are subjected to severe welfare issues, including confinement, tail mutilations, and the inability to exhibit natural behaviors such as playing and socializing. With nearly 1.5 billion pigs slaughtered for meat each year worldwide, addressing their welfare is both impactful and viable, given the availability of practical strategies to alleviate suffering and the current neglect of this issue.
Cows
Life for farmed cows is far from the idyllic, green pastures we see on milk cartons and meat packaging. Many of these curious animals spend their days in a crowded, confined space, with little chance to roam or enjoy a quiet rest under shady trees. Like humans, mother cows must give birth to produce milk. On farms, they are artificially inseminated every single year to become pregnant. In most cases, mother cows are separated from their calves after giving birth, denying them the chance to bond and nurture their young—a deeply distressing experience for both mothers and calves. Male calves are deemed "no use" by the dairy industry, so they are sold off for veal or beef or sometimes killed immediately after birth. It is estimated that around 308 million cows are killed for food each year worldwide.
Wild animal suffering
The number of wild animals far exceeds the combined number of humans, farmed animals, companion animals, and animals in laboratories. Unfortunately, many wild animals—possibly the vast majority—live very short lives and experience painful deaths.
Most organizations that focus on wild animals seem to be concerned with biodiversity conservation rather than the welfare of individual wild animals. We know of only a handful of charities that primarily focus on improving wild animal welfare. Their total global funding is estimated to be less than $5 million per year—a tiny amount relative to other cause areas; e.g., $290 million goes toward farmed animal welfare each year. We hope to see more charities working to improve wild animal welfare in the near future.
The lack of evidence for the effectiveness of strategies to improve wild animal welfare is partially due to the complexity of natural ecosystems and the difficulty of measuring the impacts of interventions in the wild. Due to this limitation, we believe that: (i) research is a promising way to help wild animals because it can inform decisions about which strategies to pursue. We acknowledge that improving wild animal welfare involves a high level of uncertainty, but its potential positive impact is also high.