Comment Permalink

Point 1: Broad agreement with a version of the original post's argument

Thanks for this. I think I agree with you that people in the global health and animal spaces should, at the margin, think more about the possibility of Transformative AI (TAI), and short-timeline TAI.

For animal-focussed people, maybe there’s an argument that because the default path of a non-TAI future is likely so bad for animals (eg persuading people to stop eating animals is really hard, persuading people to intervene to help wild animals is really hard, etc), that we might, actually, want to heavily “bet” on futures *with* TAI, because it’s only those futures which hold out the prospect of a big reduction in animal suffering. So we should optimise our actions for worlds where TAI happens, and try to maximise the chances that these futures go very well for non-human animals.

I think this is likely less true for global health and wellbeing, where plausibly the global trends look a lot better.

Point 2: Some reasons to be sceptical about claims of short-timeline Transformative AI

Having said that, there’s something about the apparent certainty that “TAI is nigh” in the original post, which prompted me to want to scribble down some push-back-y thoughts. Below are some plausible-sounding-to-me reasons to be sceptical about high-certainty claims that TAI is close. I don’t pretend that these lines of thought in-and-of-themselves demolish the case for short-timeline TAI, but I do think that they are worthy of consideration and discussion, and I’d be curious to hear what others make of them:

The prediction of short-timeline TAI is based on speculation about the future. Humans very often get this type of speculation wrong.
Global capital markets aren’t predicting short-timeline TAI. A lot of very bright people, who are highly incentivised to make accurate predictions about the future shape of the economy, are not betting that TAI is imminent.
Indeed, most people in the world don’t seem to think that TAI is imminent. This includes tonnes of *really* smart people, who have access to a lot of information.
There’s a rich history of very clever people making bold predictions about the future, based on reasonable-sounding assumptions and plausible chains of reasoning, which then don’t come true - e.g. Paul Ehrlich’s Population Bomb.
Sometimes, even the transhumanist community - where notions of AGI, AI catastrophe risk, etc, started out - get excited about a certain technological risk/trend, but then it turns out not to be such a big deal - e.g. nanotech, “grey goo”, etc in the ‘80s and ‘90s.
In the past, many radical predictions about the future, based on speculation and abstract chains of reasoning, have turned out to be wrong.
Perhaps there’s a community effect whereby we all hype ourselves up about TAI and short timelines. It’s exciting, scary and adrenaline-inducing to think that we might be about to live through ‘the end of times’.
Perhaps the meme of “TAI is just around the corner/it might kill us all” has a quality which is psychologically captivating, particularly for a certain type of mind (eg people who are into computer science, etc); perhaps this biases us. The human mind seems to be really drawn to “the end is nigh” type thinking.
Perhaps questioning the assumption of short-timeline TAI has become low-status within EA, and potentially risky in terms of reputation, funding, etc, so people are disincentivised to push back on it.

To restate: I don’t think any of these points torpedo the case for thinking that TAI is either inevitable, and/or imminent. I just think they are valid considerations when thinking about this topic, and are worthy of consideration/discussion, as we try to decide how to act in the world.

Tobias Häberli

2mo

Thanks for the thoughtful comment! Re point 1: I agree that the likelihood and expected impact of transformative AI exist on a spectrum. I didn’t mean to imply certainty about timelines, but I chose not to focus on arguing for specific timelines in this post. Regarding the specific points: they seem plausible but are mostly based on base rates and social dynamics. I think many people’s views, especially those working on AI, have shifted from being shaped primarily by abstract arguments to being informed by observable trends in AI capabilities and investments.

Forumite2mo4

Cheers, and thanks for the thoughtful post! :)

I'm not sure that the observable trends in current AI capabilities definitely point to an almost-certainty of TAI. I love using the latest LLMs, I find them amazing, and I do find it plausible that next-gen models, plus making them more agent-like, might be amazing (and scary). And I find it very, very plausible to imagine big productivity boosts in knowledge work. But the claim that this will almost-certainly lead to a rapid and complete economic/scientific transformation still feels at least a bit speculative, to me, I think...

See in context

Preparing Effective Altruism for an AI-Transformed World

by Tobias Häberli

Jan 221 min read 23

193

Cause prioritizationBuilding effective altruismAI safetyAnimal welfareGlobal health & developmentOpinion

Frontpage

[This post was written quickly and presents the idea in broad strokes. I hope it prompts more nuanced and detailed discussions in the future.]

In recent years, many in the Effective Altruism community have shifted to working on AI risks, reflecting the growing consensus that AI will profoundly shape our future.

In response to this significant shift, there have been efforts to preserve a "principles-first EA" approach, or to give special thought into how to support non-AI causes. This has often led to discussions being framed around "AI Safety vs. everything else". And it feels like the community is somewhat divided along the following lines:

Those working on AI Safety, because they believe that transformative AI is coming.
Those focusing on other causes, implicitly acting as if transformative AI is not coming.^[1]

Instead of framing priorities this way, I believe it would be valuable for more people to adopt a mindset that assumes transformative AI is likely coming and asks: What should we work on in light of that?

If we accept that AI is likely to reshape the world over the next 10–15 years, this realisation will have major implications for all cause areas. But just to start, we should strongly ask ourselves: "Are current GHW & animal welfare projects robust to a future in which AI transforms economies, governance, and global systems?" If they aren't, they are unlikely to be the best use of resources.

 Importantly, this isn't an argument that everyone should work on AI Safety. It's an argument that all cause areas need to integrate the implications of transformative AI into their theory of change and strategic frameworks. To ignore these changes is to risk misallocating resources and pursuing projects that won't stand the test of time.

^{^}

Important to note: Many people believe that AI will be transformative, but choose not to work on it due to factors such as (perceived) lack of personal fit or opportunity, personal circumstances, or other practical considerations.

193 Reactions

Mentioned in

21How much money should we be saving for retirement?

Comments23

Sorted by

New & upvoted

Click to highlight new comments since: Today at 4:09 AM

Kevin Xia 🔸2mo33

I think you make a really important point! You/anyone else interested in this may be interested in talking to @Constance Li and her work with @AI for Animals (Website)

Constance Li2mo16

Thanks for the mention, Kevin. I'm looking to gather people together who can build the in infrastructure necessary for what animal welfare initiatives could look like post-AGI transformation.

Forumite2mo25

Point 1: Broad agreement with a version of the original post's argument

I think this is likely less true for global health and wellbeing, where plausibly the global trends look a lot better.

Point 2: Some reasons to be sceptical about claims of short-timeline Transformative AI

The prediction of short-timeline TAI is based on speculation about the future. Humans very often get this type of speculation wrong.
Global capital markets aren’t predicting short-timeline TAI. A lot of very bright people, who are highly incentivised to make accurate predictions about the future shape of the economy, are not betting that TAI is imminent.
Indeed, most people in the world don’t seem to think that TAI is imminent. This includes tonnes of *really* smart people, who have access to a lot of information.
There’s a rich history of very clever people making bold predictions about the future, based on reasonable-sounding assumptions and plausible chains of reasoning, which then don’t come true - e.g. Paul Ehrlich’s Population Bomb.
Sometimes, even the transhumanist community - where notions of AGI, AI catastrophe risk, etc, started out - get excited about a certain technological risk/trend, but then it turns out not to be such a big deal - e.g. nanotech, “grey goo”, etc in the ‘80s and ‘90s.
In the past, many radical predictions about the future, based on speculation and abstract chains of reasoning, have turned out to be wrong.
Perhaps there’s a community effect whereby we all hype ourselves up about TAI and short timelines. It’s exciting, scary and adrenaline-inducing to think that we might be about to live through ‘the end of times’.
Perhaps the meme of “TAI is just around the corner/it might kill us all” has a quality which is psychologically captivating, particularly for a certain type of mind (eg people who are into computer science, etc); perhaps this biases us. The human mind seems to be really drawn to “the end is nigh” type thinking.
Perhaps questioning the assumption of short-timeline TAI has become low-status within EA, and potentially risky in terms of reputation, funding, etc, so people are disincentivised to push back on it.

Tobias Häberli2mo9

Regarding the specific points: they seem plausible but are mostly based on base rates and social dynamics. I think many people’s views, especially those working on AI, have shifted from being shaped primarily by abstract arguments to being informed by observable trends in AI capabilities and investments.

Forumite2mo4

Cheers, and thanks for the thoughtful post! :)

Tobias Häberli2mo22

One GHW example: The impact of AI tutoring on educational interventions (via Arjun Panickssery on LessWrong).

There have been at least 2 studies/impact evaluations of AI tutoring in African countries finding extraordinarily large effects:

Summer 2024 — 15–16-year olds in Nigeria
They had 800 students total. The treatment group studied with GPT-based Microsoft Copilot twice weekly for six weeks, studying English. They were just provided an initial prompt to start chatting—teachers had a minimal “orchestra conductor” role—but they achieved “the equivalent of two years of typical learning in just six weeks.”

February–August 2023 — 8–14-year-olds in Ghana
An educational network called Rising Academies tested their WhatsApp-based AI math tutor called Rori with 637 students in Ghana. Students in the treatment group received AI tutors during study hall. After eight months, 25% of the subjects attrited from inconsistent school attendance. Of the remainder, the treatment group increased their scores on a 35-question assessment by 5.13 points versus 2.12 points for the control group. This difference was “approximately equivalent to an extra year of learning” for the treatment group.

Should this significantly change how excited EAs are about educational interventions? I don't know, but I've also not seen a discussion of this on the forum (this post about MOOC & AI tutors that received ~zero engagement).

Joseph1mo6

I suspect this might be two distinct uses of "AI" as a term. While GPT-type chatbots can be helpful (such as in the educational examples you refer to), they are very different from artificial general intelligence of the type that most AI alignment/safety work is expecting to happen.

To paraphrase AI Snake Oil,^[1] it is like one person talking about vehicles while discussing about how improved spacecraft will open up new possibilities for humanity, and a second person mentions how vehicles are also helping his area because cars are becoming more energy efficient. While they do both fall under the category of "vehicles," they are quite different concepts. So I'm wondering if this might be verging near to talking past each other territory.

^{^}

The full quote is this: "Imagine an alternate universe in which people don’t have words for different forms of transportation—only the collective noun “vehicle.” They use that word to refer to cars, buses, bikes, spacecraft, and all other ways of getting from place A to place B. Conversations in this world are confusing. There are furious debates about whether or not vehicles are environmentally friendly, even though no one realizes that one side of the debate is talking about bikes and the other side is talking about trucks. There is a breakthrough in rocketry, but the media focuses on how vehicles have gotten faster—so people call their car dealer (oops, vehicle dealer) to ask when faster models will be available. Meanwhile, fraudsters have capitalized on the fact that consumers don’t know what to believe when it comes to vehicle technology, so scams are rampant in the vehicle sector. Now replace the word “vehicle” with “artificial intelligence,” and we have a pretty good description of the world we live in."

Tobias Häberli1mo9

Thanks for the comment! I might be missing something, but GPT-type chatbots are based on large language models, which play a key role in scaling toward AGI. I do think that extrapolating progress from them is valuable but also agree that tying discussions about future AI systems too closely to current models’ capabilities can be misleading.

That said, my post intentionally assumes a more limited claim: that AI will transform the world in significant ways relatively soon. This assumption seems both more likely and increasingly foreseeable. In contrast, assumptions about a world ‘incredibly radically’ transformed by superintelligence are less likely and less foreseeable. There have been lots of arguments around why you should work on AI Safety, and I agree with many of them. I’m mainly trying to reach the EAs who buy into the limited claim but currently act as if they don’t.

Regarding the example: It would likely be a mistake to focus only on current AI capabilities for education. However, it could be important to seriously evaluate scenarios like, ‘AI teachers better than every human teacher soon’.

Joseph1mo6

That strikes me as very reasonable, especially considering the likelihood and foreseeability. Especially since the education examples you mentioned really are currently capable of transforming parts of the world.

Mo Putera2mo6

This writeup by Vadim Albinsky at Founders Pledge seems related: Are education interventions as cost effective as the top health interventions? Five separate lines of evidence for the income effects of better education [Founders Pledge]

The part that seems relevant is the charity Imagine Worldwide's use of the "adaptive software" OneBillion app to teach numeracy and literacy. Despite Vadim's several discounts and general conservatism throughout his CEA he still gets ~11x GD cost-effectiveness. (I'd honestly thought, given the upvotes and engagement on the post, that Vadim had changed some EAs' minds on the promisingness of non-deworming education interventions.) The OneBillion app doesn't seem to use AI, but they already (paraphrasing) use "software to provide a complete, research-based curriculum that adapts to each child’s pace, progress, and cultural and linguistic context", so I'm not sure how much better Copilot / Rori would be?

Quoting some parts that stood out to me (emphasis mine):

This post argues that if we look at a broad enough evidence base for the long term outcomes of education interventions we can conclude that the best ones are as cost effective as top GiveWell grants. ...
... I will argue that the combined evidence for the income impacts of interventions that boost test scores is much stronger than the evidence GiveWell has used to value the income effects of fighting malaria, deworming, or making vaccines, vitamin A, and iodine more available. Even after applying very conservative discounts to expected effect sizes to account for the applicability of the evidence to potential funding opportunities, we find the best education interventions to be in the same range of cost-effectiveness as GiveWell’s top charities. ...
When we apply the above recommendations to our median recommended education charity, Imagine Worldwide, we estimate that it is 11x as cost effective as GiveDirectly at boosting well-being through higher income. ...
Imagine Worldwide (IW) provides adaptive software to teach numeracy and literacy in Malawi, along with the training, tablets and solar panels required to run it. They plan to fund a six-year scale-up of their currently existing program to cover all 3.5 million children in grades 1-4 by 2028. The Malawi government will provide government employees to help with implementation for the first six years, and will take over the program after 2028. Children from over 250 schools have received instruction through the OneBillion app in Malawi over the past 8 years. Five randomized controlled trials of the program have found learning gains of an average of 0.33 standard deviations. The OneBillion app has also undergone over five additional RCTs in a broad range of contexts with comparable or better results.

Sjlver2mo4

EA charities can also combine education and global health, like https://healthlearn.org/blog/updated-impact-model

HealthLearn builds a mobile app for health workers (nurses, midwives, doctors, community health workers) in Nigeria und Uganda. Health workers use it to learn clinical best practices. This leads to better outcomes for patients.

I'm personally very excited by this. Health workers in developing countries often have few training resources available. There are several clinical practices that can improve patient outcomes while being easy to implement (such as initiating breastfeeding immediately after birth). These are not as widely used as we would like.

HealthLearn uses technology as a way to faithfully scale the intervention to thousands of health workers. At this point, AI does not play a significant role in the learning process yet. Courses are manually designed. This was important to get started quickly, but also to get approval from government health agencies and professional organizations such as nursing councils.

The impact model that I've linked to above estimates that the approach has been cost-effective so far, and could become better with scale.

(disclaimer: I'm one of the software engineers building the app)

Patrick Hoang1mo15

A lot of EAs do think AI safety could become ridiculously important (i.e. some probability mass of very short timelines) but are not in the position to do anything, which is why they focus on more tractable areas (i.e. global health, animal welfare, EA building) under the assumption of longer AI timelines. Especially because there's a lot of uncertainty about when AGI would come.

My internal view is 25% of TAI by 2040 and 50% of TAI by 2060, where I define TAI as an AI with the ability to autonomously perform AI research. They may have shifted in light of DeepSeek but what am I supposed to do? I'm just a freshman college student at a non-prestigious university. Am I supposed to drop all commitments I have, speed-run my degree, get myself to work in a highly competitive AI lab which would probably require a Ph. D., work on technical alignment hoping to get a breakthrough? If TAI comes within 5 years, it would be the right move, but if I'm wrong then I would end up with very shallow skills without much experience.

We have the following Pascal matrix (drafted my GPT):

Decision	AGI Comes Soon (~2030s)	AGI Comes Late (~2060s+)
Rush into AI Now	🚀 Huge Impact, but only if positioned well	😬 Career stagnation, lower expertise
Stay on Current Path	😢 Missed critical decisions, lower impact	📈 Strong expertise, optimal positioning

I know the decision is not binary, but I am definitely willing to forfeit 25% of my impact by betting on the AGI comes late scenario. I do think non-AI cause areas should use AI projection in their deliberation and ToC but I think it is silly to cut out everything that happens after 2040 with respect to the cause area.

However, I do think EAs should have a contingency plan where they should speedrun to AI safety if and only if (one of multiple conditions occur; i.e. even conservative superforecastors project AGI before 2040, or a national emergency is declared). And we can probably hedge against the AGI comes soon scenario by buying long-term NVIDIA call options.

Will Aldred2mo13

+1. I appreciated @RobertM’s articulation of this problem for animal welfare in particular:

I think the interventions for ensuring that animal welfare is good after we hit transformative AI probably look very different from interventions in the pretty small slice of worlds where the world looks very boring in a few decades.
…
If we achieve transformative AI and then don’t all die (because we solved alignment), then I don’t think the world will continue to have an “agricultural industry” in any meaningful sense (or, really, any other traditional industry; strong nanotech seems like it ought to let you solve for nearly everything else). Even if the economics and sociology work out such that some people will want to continue farming real animals instead of enjoying the much cheaper cultured meat of vastly superior quality, there will be approximately nobody interested in ensuring those animals are suffering, and the cost for ensuring that they don’t suffer will be trivial.
[...] if you think it’s at all plausible that we achieve TAI in a way that locks in reflectively-unendorsed values which lead to huge quantities of animal suffering, that seems like it ought to dominate effectively all other considerations in terms of interventions w.r.t. future animal welfare.

I’ve actually tried asking/questioning a few animal welfare folks for their takes here, but I’ve yet to hear back anything that sounded compelling (to me). (If anyone reading this has an argument for why ‘standard’ animal welfare interventions are robust to the above, then I’d love to hear it!)

Tristan Katz8d1

Animal welfare guy tuning in. My own take is that the majority of the world actually is almost entirely indifferent about animal suffering, so if AI tries to reflect global values (not just the values of the progressive, elite silicon valley bubble) there is a real risk that it will be indifferent to animal suffering. Consider how Foie Gras is still legal in most countries, or bullfighting, both of which are totally unnecessary. And those are just examples from western countries.

I think it's very likely that TAI will lock in only a very mild concern for animal welfare. Or perhaps, concern for animal welfare in certain contexts (e.g. pets), and none in others (e.g. chicken). Maybe that will lead to a future without factory farming, but it will lead to a future with unnecessary animal suffering nonetheless.

What I'm not sure about is: how do we ensure that TAI locks in a strong valuation of animal welfare? One route is to try to change how much society cares about animal welfare, and hope that TAI then reflects that. I guess this is the hope of many animal advocates. But I admit that seems too slow to work at this stage, so I agree that animal advocates should probably prioritize trying to influence those developing AI right now.

yanni kyriacos2mo12

If I wasn’t working on AI Safety I’d work on near term (< 5 years) animal welfare interventions.

Berke2mo9

Strong upvote! I want to say some stuff particularly within the context of global development:

The intersection of AI and global development seems surprisingly unsaturated within EA, or to be more specific, I think a surprisingly few number of EAs think about the following questions:

i) How to leverage AI for development (e.g. AI tools for education, healthcare)
ii) What interventions and strategies should be prioritized within global health and development in the light of AI developments? (basically the question you ask)

There seems to be a lot of people thinking about the first question outside of EA, so maybe that explains this dynamic, but I have the "hunch" that the primary reason why people don't focus on the first question too much is people deferring too much and selection effects, rather lack of any high-impact interventions. If you care about TAI, you are very likely to work on AI alignment & governance, if you don't want to work on TAI-related things (due to risk-aversion or any other argument/value), you just don't update that much based on AI developments and forecasts. This may also have to do with EA's ambiguity-averse/risk-averse attitude towards GHD characterized by exploiting evidence-based, interventions rather than exploring new highly promising interventions. I think if a student/professional were to come to an EA community-builder and asked "How can I pursue a high-impact career in/upskill in global health R&D or AI-for-development", number of community-builders that can give a sufficiently helpful answer is likely very few to none, I also likely wouldn't be able to give a good answer and point to communities/resources outside of the EA community.

(Maybe EAs in London or SF people discuss these, but I don't see any discussion of it online, neither do I see any spaces where people who could be discussing these can network/discuss together. If there is anyone who'd like to help create or run an online or in-person AI-for-development or global health R&D fellowship, feel free to shoot a message)

Vasco Grilo🔸2mo7

Hi Tobias.

If we accept that AI is likely to reshape the world over the next 10–15 years, this realisation will have major implications for all cause areas.

I think donating to the Shrimp Welfare Project (SWP) would still have super high cost-effectiveness even if the world was certain to end in 10 years. I estimate it has been 64.3 k times as cost-effective as GiveWell’s top charities (ignoring their effects on animals) for 10 years of acceleration of the adoption of electrical stunning, as used by Open Philanthropy (OP). If the acceleration followed a normal distribution, SWP's cost-effectiveness would only become 50 % as high if the world was certain to end in 10 years. I think this would still be orders of magnitude more cost-effective than the best interventions in global health and development and AI safety.

There is also the question of whether the world will actually be radically reshaped. I am happy to bet 10 k$ against short timelines for that.

David_Moss2mo8

Many people believe that AI will be transformative, but choose not to work on it due to factors such a (perceived) lack of personal fit or opportunity, personal circumstances, or other practical considerations.

There may be various other reasons why people choose to work on other areas, despite believing transformative AI is very likely, e.g. decision-theoretic or normative/meta-normative uncertainty.

Tobias Häberli2mo4

Thanks for adding this! I definitely didn’t want to suggest the list of reasons was exhaustive or that the division between the two 'camps' is clear-cut.

Chris Leong2mo8

I gave this a strong upvote because regardless of whether or not you agree with these timelines or Tobias' conclusion, this is a discussion that the community needs to be having. As in, it's hard to argue that the possibility of this is remote enough these days that it makes sense to ignore it.

I would love to see someone running a course focusing on this (something broader than the AI Safety Fundamentals course). Obviously this is speculative, but I wouldn't be surprised if the EA Infrastructure Fund were interested in funding a high-quality proposal to create such a course.

Denkenberger🔸2mo6

Instead of framing priorities this way, I believe it would be valuable for more people to adopt a mindset that assumes transformative AI is likely coming and asks: What should we work on in light of that?

For climate change, I think it means focusing on the catastrophes that could plausibly happen in the next couple decades, such as coincident extreme weather on multiple continents or the collapse of the sub polar gyre. So adaptation becomes relatively more important than emissions reductions. Since AI probably makes nuclear conflict and engineered pandemic more likely, the importance of these fields may still be similar, but you would likely move away from long payoff actions like field building and maybe some things that are likely to take a long time, such as major arsenal reductions or wide adoption of germicidal UV or enhanced air filtration. Instead, one might focus on trying to reduce the chance of nuclear war, especially given AI-enabled systems or AI-induced tensions, or increasing resilience to nuclear war. On the pandemic side, of course reducing the risk that AI enables pandemics, but also near-term actions that could have a significant impact like early warning for pandemics or rapid scale up of resilience post outbreak.

DPiepgrass1mo4

A key implication here is that we need models of how AI will transform the world with many qualitative and quantitative details. Individual EAs working in global health, for example, cannot be expected to broadly predict how the world will change.

My view, having thought about this a fair bit, is that there is an extremely broad range of outcomes ranging from human extinction, to various dystopias, to utopia or "utopia". But there are probably a lot of effects that are relatively predictable, especially in the near term.

Of course, EAs in field X can think about how AI affects X. But this should work better after learning about whatever broad changes superforecasters (or whoever) can predict.

Leo Arruda2mo2

I’m not sure how plausible and feasible it is, but I believe using AI to automate data collection, integration, and analysis could potentially streamline processes like diagnosing problems, testing hypotheses, and prioritizing interventions in GHW, at least in some domains.