This arrests EAs/longtermists from drawing on centuries of knowledge & movement-building
The track record of anticapitalist advocacy seems quite poor. See this free book: Socialism: The failed idea that never dies
If you're doing anticapitalist advocacy for EA reasons, I think you need a really clear understanding of why such advocacy has caused so much misery in the past, and how your advocacy will avoid those traps.
I'd say what's needed is not anticapitalist advocacy, so much as small-scale prototyping of alternative economic systems that have strong theoretical arguments for how they will align incentives better, and scale way past Dunbar's number.
You don't need a full replacement for capitalism to test ideas and see results. For example, central planning often fails due to corruption. A well-designed alternative system will probably need a solution for corruption. And such a solution could be usefully applied to an ordinary capitalist democracy.
I concede that AI companies are behaving in a harmful way, but I doubt that anticapitalist advocacy is a particularly tractable way to address that, at least in the short term.
In that case your strategy is just feeding the labs talent and poisoning the ability of their circles to oppose them.
It seems like your model only has such influence going one way. The lab worker will influence their friends, but not the other way around. I think two-way influence is a more accurate model.
Another option is to ask your friends to monitor you so you don't get ideologically captured, and hold an intervention if it seems appropriate.
I see no meaningful path to impact on safety working as an AI lab researcher
This is a very strong statement. I'm not following technical alignment research that closely, but my general sense is that exciting work is being done. I just wrote this comment advertising a line of research which strikes me as particularly promising.
I noticed the other day that the people who are particularly grim about AI alignment also don't seem to be engaging much with contemporary technical alignment research. That missing intersection seems suspicious. I'm interested in any counterexamples that come to mind.
My subjective sense is there's a good chance we lose because all the necessary insights to build aligned AI were lying around, they just didn't get sufficiently developed or implemented. This seems especially true for techniques like gradient routing which would need to be baked in to a big, expensive training run.
(I'm also interested in arguments for why unlearning won't work. I've thought about this a fair amount, and it seems to me that sufficiently good unlearning kind of just oneshots AI safety, as elaborated in the comment I linked.)
With regard to Deepseek, it seems to me that the success of mixture-of-experts could be considered an update towards methods like gradient routing. If you could localize specific kinds of knowledge to specific experts in a reliable way, you could dynamically toggle off / ablate experts with unnecessary dangerous knowledge. E.g. toggle off experts knowledgeable in human psychology so the AI doesn't manipulate you.
I like this approach because if you get it working well, it's a general tool that could help address a lot of different catastrophe stories in a way that seems pretty robust. E.g. to mitigate a malicious AI from gaining root access to its datacenter, ablate knowledge of the OS it's running on. To mitigate sophisticated cooperation between AIs that are supposed to be monitoring one another, ablate knowledge of game theory. Etc. (The broader point is that unlearning seems very generally useful. But the "Expand-Route-Ablate" style approach from the gradient routing paper strikes me as a particularly promising, and could harmonize well with MoE.)
I think a good research goal would be to try to eventually replicate Deepseek's work, except with highly interpretable experts. The idea is to produce a "high-assurance" model which can be ablated so undesired behaviors, like deception, are virtually impossible to jailbreak out of it (since the weights that perform the behavior are inaccessible). I think the gradient routing paper is a good start. To achieve sufficient safety we'll need new methods that are more robust and easier to deploy, which should probably be prototyped on toy problems first.
it's possible Trump is better placed to negotiate a binding treaty with China (similar to the idea that 'only Nixon could go to China'), even if it's not clear he'll want to do so.
Some related points:
Trump seems like someone who likes winning, but hates losing, and also likes making deals. If Deepseek increases the probability that the US will lose, that makes it more attractive to negotiate an end to the race. This seems true from both a Trump-psychology perspective, and a rational-game-theorist perspective.
Elon Musk seems to have a good relationship with Chinese leadership.
Releasing open-source AI seems like more of a way to prevent someone else from winning than a way to win yourself.
I suppose one possible approach would be to try to get some sort of back-channel dialogue going, to start drafting a treaty which can be invoked if political momentum appears.
That's a good point about patient privacy. On X, you suggested that PEPFAR has had a sizable macro impact on AIDS in Africa. Maybe Africans who are old enough to remember could talk about what AIDS was like in Africa before and after PEPFAR, as a way to illustrate that macro impact without violating the privacy of any individual patients.
Of course, insofar as individual patients are willing to speak about their experiences, possibly with some light anonymization such as looking away from the camera, that seems really good to me too.
A couple other thoughts:
If access to ARV drugs is interrupted, I imagine that could lead to drug-resistant HIV strains, same way your doctor tells you to take your antibiotics consistently.
Having a big population of immunocompromised people in Africa seems bad in terms of mutation speed for future pandemics. Here's a paper I just found on Google, about COVID variants which are thought to have arisen from immunocompromised patients.
I wonder if the response will be seen as more credible if it's driven by Africans?
Insofar as there's a motivation behind cutting foreign aid, I suspect that skepticism of the NGO sector is playing a role. I can imagine Trump supporters thinking: This program supposedly helps millions of poor Africans, yet the primary voices advocating for it online are rich-world progressives. Seems fishy.
I've received a bunch of Whatsapp messages from Ugandan friends who are very very worried about what this might mean not only for patients (the main issue), but also for jobs and the livelihood of many local NGOS.
I think it could be pretty effective for your Ugandan friends to record videos interviewing people who have been helped by the program, and post the videos online.
A lot of skepticism of foreign aid is driven by the fear that the aid is being captured somewhere along the way, either here in the US or in the target country. Hearing directly from the aid recipients helps address that fear.
As an American, it feels like good things the US does are taken for granted, and the US will be criticized relentlessly by people in other countries no matter what we do. So I wouldn't suggest scolding the US for withdrawing funds. And in any case, rewards usually work better than punishments for modifying behavior. I think saying thanks, and talking about what it would mean for the program to continue, would be a lot more powerful than scolding. Even for Americans who aren't particularly altruistic, this would provide tangible evidence that the program is improving America's reputation.
One mental model I have is that the US is suffering from something akin to EA burnout, and the solution to both "US burnout" and "EA burnout" is a stronger culture of gratitude. In principle, I could imagine that a really good response to this PEPFAR situation actually ends up motivating the US to fund additional effective aid programs.
I sold all my NVIDIA stock, since their moat looks weak to me:
https://forum.effectivealtruism.org/posts/rBx9RmJdBJgHkjL4j/will-openai-s-o3-reduce-nvidia-s-moat
I think your reasoning is generally correct. Another argument: If you believe things look sufficiently grim under short timelines, maybe you should invest under the assumption that a recession, or something else, will pop the AI bubble and gives us longer timelines.
As an American, I think this is actually a really good point. The only reason to even consider racing with China to build AI is if we believe the outcome is better if America "wins". Seems worth keeping an eye on the political situation in the US, e.g. for stuff like PEPFAR cancellation, to check if the US "winning" is actually better from a humanitarian perspective. Hopefully lab leaders are smart enough to realize this.