[This post was written quickly and presents the idea in broad strokes. I hope it prompts more nuanced and detailed discussions in the future.]
In recent years, many in the Effective Altruism community have shifted to working on AI risks, reflecting the growing consensus that AI will profoundly shape our future.
In response to this significant shift, there have been efforts to preserve a "principles-first EA" approach, or to give special thought into how to support non-AI causes. This has often led to discussions being framed around "AI Safety vs. everything else". And it feels like the community is somewhat divided along the following lines:
- Those working on AI Safety, because they believe that transformative AI is coming.
- Those focusing on other causes, implicitly acting as if transformative AI is not coming.[1]
Instead of framing priorities this way, I believe it would be valuable for more people to adopt a mindset that assumes transformative AI is likely coming and asks: What should we work on in light of that?
If we accept that AI is likely to reshape the world over the next 10–15 years, this realisation will have major implications for all cause areas. But just to start, we should strongly ask ourselves: "Are current GHW & animal welfare projects robust to a future in which AI transforms economies, governance, and global systems?" If they aren't, they are unlikely to be the best use of resources.
Importantly, this isn't an argument that everyone should work on AI Safety. It's an argument that all cause areas need to integrate the implications of transformative AI into their theory of change and strategic frameworks. To ignore these changes is to risk misallocating resources and pursuing projects that won't stand the test of time.
- ^
Important to note: Many people believe that AI will be transformative, but choose not to work on it due to factors such as (perceived) lack of personal fit or opportunity, personal circumstances, or other practical considerations.
A lot of EAs do think AI safety could become ridiculously important (i.e. some probability mass of very short timelines) but are not in the position to do anything, which is why they focus on more tractable areas (i.e. global health, animal welfare, EA building) under the assumption of longer AI timelines. Especially because there's a lot of uncertainty about when AGI would come.
My internal view is 25% of TAI by 2040 and 50% of TAI by 2060, where I define TAI as an AI with the ability to autonomously perform AI research. They may have shifted in light of DeepSeek but what am I supposed to do? I'm just a freshman college student at a non-prestigious university. Am I supposed to drop all commitments I have, speed-run my degree, get myself to work in a highly competitive AI lab which would probably require a Ph. D., work on technical alignment hoping to get a breakthrough? If TAI comes within 5 years, it would be the right move, but if I'm wrong then I would end up with very shallow skills without much experience.
We have the following Pascal matrix (drafted my GPT):
I know the decision is not binary, but I am definitely willing to forfeit 25% of my impact by betting on the AGI comes late scenario. I do think non-AI cause areas should use AI projection in their deliberation and ToC but I think it is silly to cut out everything that happens after 2040 with respect to the cause area.
However, I do think EAs should have a contingency plan where they should speedrun to AI safety if and only if (one of multiple conditions occur; i.e. even conservative superforecastors project AGI before 2040, or a national emergency is declared). And we can probably hedge against the AGI comes soon scenario by buying long-term NVIDIA call options.
I think you make a really important point! You/anyone else interested in this may be interested in talking to @Constance Li and her work with @AI for Animals (Website)
Thanks for the mention, Kevin. I'm looking to gather people together who can build the in infrastructure necessary for what animal welfare initiatives could look like post-AGI transformation.
Point 1: Broad agreement with a version of the original post's argument
Thanks for this. I think I agree with you that people in the global health and animal spaces should, at the margin, think more about the possibility of Transformative AI (TAI), and short-timeline TAI.
For animal-focussed people, maybe there’s an argument that because the default path of a non-TAI future is likely so bad for animals (eg persuading people to stop eating animals is really hard, persuading people to intervene to help wild animals is really hard, etc), that we might, actually, want to heavily “bet” on futures *with* TAI, because it’s only those futures which hold out the prospect of a big reduction in animal suffering. So we should optimise our actions for worlds where TAI happens, and try to maximise the chances that these futures go very well for non-human animals.
I think this is likely less true for global health and wellbeing, where plausibly the global trends look a lot better.
Point 2: Some reasons to be sceptical about claims of short-timeline Transformative AI
Having said that, there’s something about the apparent certainty that “TAI is nigh” in the original post, which prompted me to want to scribble down some push-back-y thoughts. Below are some plausible-sounding-to-me reasons to be sceptical about high-certainty claims that TAI is close. I don’t pretend that these lines of thought in-and-of-themselves demolish the case for short-timeline TAI, but I do think that they are worthy of consideration and discussion, and I’d be curious to hear what others make of them:
To restate: I don’t think any of these points torpedo the case for thinking that TAI is either inevitable, and/or imminent. I just think they are valid considerations when thinking about this topic, and are worthy of consideration/discussion, as we try to decide how to act in the world.
Thanks for the thoughtful comment!
Re point 1: I agree that the likelihood and expected impact of transformative AI exist on a spectrum. I didn’t mean to imply certainty about timelines, but I chose not to focus on arguing for specific timelines in this post.
Regarding the specific points: they seem plausible but are mostly based on base rates and social dynamics. I think many people’s views, especially those working on AI, have shifted from being shaped primarily by abstract arguments to being informed by observable trends in AI capabilities and investments.
Cheers, and thanks for the thoughtful post! :)
I'm not sure that the observable trends in current AI capabilities definitely point to an almost-certainty of TAI. I love using the latest LLMs, I find them amazing, and I do find it plausible that next-gen models, plus making them more agent-like, might be amazing (and scary). And I find it very, very plausible to imagine big productivity boosts in knowledge work. But the claim that this will almost-certainly lead to a rapid and complete economic/scientific transformation still feels at least a bit speculative, to me, I think...
One GHW example: The impact of AI tutoring on educational interventions (via Arjun Panickssery on LessWrong).
There have been at least 2 studies/impact evaluations of AI tutoring in African countries finding extraordinarily large effects:
Should this significantly change how excited EAs are about educational interventions? I don't know, but I've also not seen a discussion of this on the forum (this post about MOOC & AI tutors that received ~zero engagement).
I suspect this might be two distinct uses of "AI" as a term. While GPT-type chatbots can be helpful (such as in the educational examples you refer to), they are very different from artificial general intelligence of the type that most AI alignment/safety work is expecting to happen.
To paraphrase AI Snake Oil,[1] it is like one person talking about vehicles while discussing about how improved spacecraft will open up new possibilities for humanity, and a second person mentions how vehicles are also helping his area because cars are becoming more energy efficient. While they do both fall under the category of "vehicles," they are quite different concepts. So I'm wondering if this might be verging near to talking past each other territory.
The full quote is this: "Imagine an alternate universe in which people don’t have words for different forms of transportation—only the collective noun “vehicle.” They use that word to refer to cars, buses, bikes, spacecraft, and all other ways of getting from place A to place B. Conversations in this world are confusing. There are furious debates about whether or not vehicles are environmentally friendly, even though no one realizes that one side of the debate is talking about bikes and the other side is talking about trucks. There is a breakthrough in rocketry, but the media focuses on how vehicles have gotten faster—so people call their car dealer (oops, vehicle dealer) to ask when faster models will be available. Meanwhile, fraudsters have capitalized on the fact that consumers don’t know what to believe when it comes to vehicle technology, so scams are rampant in the vehicle sector. Now replace the word “vehicle” with “artificial intelligence,” and we have a pretty good description of the world we live in."
Thanks for the comment! I might be missing something, but GPT-type chatbots are based on large language models, which play a key role in scaling toward AGI. I do think that extrapolating progress from them is valuable but also agree that tying discussions about future AI systems too closely to current models’ capabilities can be misleading.
That said, my post intentionally assumes a more limited claim: that AI will transform the world in significant ways relatively soon. This assumption seems both more likely and increasingly foreseeable. In contrast, assumptions about a world ‘incredibly radically’ transformed by superintelligence are less likely and less foreseeable. There have been lots of arguments around why you should work on AI Safety, and I agree with many of them. I’m mainly trying to reach the EAs who buy into the limited claim but currently act as if they don’t.
Regarding the example: It would likely be a mistake to focus only on current AI capabilities for education. However, it could be important to seriously evaluate scenarios like, ‘AI teachers better than every human teacher soon’.
That strikes me as very reasonable, especially considering the likelihood and foreseeability. Especially since the education examples you mentioned really are currently capable of transforming parts of the world.
This writeup by Vadim Albinsky at Founders Pledge seems related: Are education interventions as cost effective as the top health interventions? Five separate lines of evidence for the income effects of better education [Founders Pledge]
The part that seems relevant is the charity Imagine Worldwide's use of the "adaptive software" OneBillion app to teach numeracy and literacy. Despite Vadim's several discounts and general conservatism throughout his CEA he still gets ~11x GD cost-effectiveness. (I'd honestly thought, given the upvotes and engagement on the post, that Vadim had changed some EAs' minds on the promisingness of non-deworming education interventions.) The OneBillion app doesn't seem to use AI, but they already (paraphrasing) use "software to provide a complete, research-based curriculum that adapts to each child’s pace, progress, and cultural and linguistic context", so I'm not sure how much better Copilot / Rori would be?
Quoting some parts that stood out to me (emphasis mine):
EA charities can also combine education and global health, like https://healthlearn.org/blog/updated-impact-model
HealthLearn builds a mobile app for health workers (nurses, midwives, doctors, community health workers) in Nigeria und Uganda. Health workers use it to learn clinical best practices. This leads to better outcomes for patients.
I'm personally very excited by this. Health workers in developing countries often have few training resources available. There are several clinical practices that can improve patient outcomes while being easy to implement (such as initiating breastfeeding immediately after birth). These are not as widely used as we would like.
HealthLearn uses technology as a way to faithfully scale the intervention to thousands of health workers. At this point, AI does not play a significant role in the learning process yet. Courses are manually designed. This was important to get started quickly, but also to get approval from government health agencies and professional organizations such as nursing councils.
The impact model that I've linked to above estimates that the approach has been cost-effective so far, and could become better with scale.
(disclaimer: I'm one of the software engineers building the app)
A key implication here is that we need models of how AI will transform the world with many qualitative and quantitative details. Individual EAs working in global health, for example, cannot be expected to broadly predict how the world will change.
My view, having thought about this a fair bit, is that there is an extremely broad range of outcomes ranging from human extinction, to various dystopias, to utopia or "utopia". But there are probably a lot of effects that are relatively predictable, especially in the near term.
Of course, EAs in field X can think about how AI affects X. But this should work better after learning about whatever broad changes superforecasters (or whoever) can predict.
If I wasn’t working on AI Safety I’d work on near term (< 5 years) animal welfare interventions.
+1. I appreciated @RobertM’s articulation of this problem for animal welfare in particular:
I’ve actually tried asking/questioning a few animal welfare folks for their takes here, but I’ve yet to hear back anything that sounded compelling (to me). (If anyone reading this has an argument for why ‘standard’ animal welfare interventions are robust to the above, then I’d love to hear it!)
Hi Tobias.
I think donating to the Shrimp Welfare Project (SWP) would still have super high cost-effectiveness even if the world was certain to end in 10 years. I estimate it has been 64.3 k times as cost-effective as GiveWell’s top charities (ignoring their effects on animals) for 10 years of acceleration of the adoption of electrical stunning, as used by Open Philanthropy (OP). If the acceleration followed a normal distribution, SWP's cost-effectiveness would only become 50 % as high if the world was certain to end in 10 years. I think this would still be orders of magnitude more cost-effective than the best interventions in global health and development and AI safety.
There is also the question of whether the world will actually be radically reshaped. I am happy to bet 10 k$ against short timelines for that.
Strong upvote! I want to say some stuff particularly within the context of global development:
The intersection of AI and global development seems surprisingly unsaturated within EA, or to be more specific, I think a surprisingly few number of EAs think about the following questions:
i) How to leverage AI for development (e.g. AI tools for education, healthcare)
ii) What interventions and strategies should be prioritized within global health and development in the light of AI developments? (basically the question you ask)
There seems to be a lot of people thinking about the first question outside of EA, so maybe that explains this dynamic, but I have the "hunch" that the primary reason why people don't focus on the first question too much is people deferring too much and selection effects, rather lack of any high-impact interventions. If you care about TAI, you are very likely to work on AI alignment & governance, if you don't want to work on TAI-related things (due to risk-aversion or any other argument/value), you just don't update that much based on AI developments and forecasts. This may also have to do with EA's ambiguity-averse/risk-averse attitude towards GHD characterized by exploiting evidence-based, interventions rather than exploring new highly promising interventions. I think if a student/professional were to come to an EA community-builder and asked "How can I pursue a high-impact career in/upskill in global health R&D or AI-for-development", number of community-builders that can give a sufficiently helpful answer is likely very few to none, I also likely wouldn't be able to give a good answer and point to communities/resources outside of the EA community.
(Maybe EAs in London or SF people discuss these, but I don't see any discussion of it online, neither do I see any spaces where people who could be discussing these can network/discuss together. If there is anyone who'd like to help create or run an online or in-person AI-for-development or global health R&D fellowship, feel free to shoot a message)
For climate change, I think it means focusing on the catastrophes that could plausibly happen in the next couple decades, such as coincident extreme weather on multiple continents or the collapse of the sub polar gyre. So adaptation becomes relatively more important than emissions reductions. Since AI probably makes nuclear conflict and engineered pandemic more likely, the importance of these fields may still be similar, but you would likely move away from long payoff actions like field building and maybe some things that are likely to take a long time, such as major arsenal reductions or wide adoption of germicidal UV or enhanced air filtration. Instead, one might focus on trying to reduce the chance of nuclear war, especially given AI-enabled systems or AI-induced tensions, or increasing resilience to nuclear war. On the pandemic side, of course reducing the risk that AI enables pandemics, but also near-term actions that could have a significant impact like early warning for pandemics or rapid scale up of resilience post outbreak.
There may be various other reasons why people choose to work on other areas, despite believing transformative AI is very likely, e.g. decision-theoretic or normative/meta-normative uncertainty.
Thanks for adding this! I definitely didn’t want to suggest the list of reasons was exhaustive or that the division between the two 'camps' is clear-cut.
I gave this a strong upvote because regardless of whether or not you agree with these timelines or Tobias' conclusion, this is a discussion that the community needs to be having. As in, it's hard to argue that the possibility of this is remote enough these days that it makes sense to ignore it.
I would love to see someone running a course focusing on this (something broader than the AI Safety Fundamentals course). Obviously this is speculative, but I wouldn't be surprised if the EA Infrastructure Fund were interested in funding a high-quality proposal to create such a course.
I’m not sure how plausible and feasible it is, but I believe using AI to automate data collection, integration, and analysis could potentially streamline processes like diagnosing problems, testing hypotheses, and prioritizing interventions in GHW, at least in some domains.