I've been reviewing some old Forum posts for an upcoming post I'm writing, and incidentally came across this by Howie Lempel for noticing in what spirit you're engaging with someone's ideas: > "Did I ask this question because I think they will have a good answer or because I think they will not have a good answer?" I felt pretty called out :P To be fair, I think the latter is sometimes a reasonable persuasive tactic, and it's fine to put yourself in a teaching role rather than a learning role if that's your endorsed intention and the other party is on board. But the value of this quote to me is that it successfully highlights how easily we can tell ourselves we're being intellectually curious, when we're actually doing something else.

Anthropic issues questionable letter on SB 1047 (Axios). I can't find a copy of the original letter online. 

I can highly recommend following Sentinel's weekly minutes, a weekly update from superforecasters on the likelihood of any events which plausibly could cause worldwide catastrophe. Perhaps the weekly newsletter I look the most forward to at this point. Read previous issues here:  https://sentinel-team.org/blog/

With open-source models being released and on ramps to downstream innovation lowering, the safety challenges may not be a single threshold but rather an ongoing, iterative cat-and-mouse game. Just underscores the importance of people in the policy/safety field thinking far ahead

Reading and engaging with the forum as good for a meta reason, engaging and encouraging other people to keep making posts because engagement seems to exist and they’re incentivized to post. Or even more junior people to try and contribute, idk what the ea forum felt like ~10 years ago, but probably lower standards for engagement.

Hey everyone, in collaboration with Apart Research, I'm helping organize a hackathon this weekend to build tools for accelerating alignment research. This hackathon is very much related to my effort in building an "Alignment Research Assistant." Here's the announcement post: 2 days until we revolutionize AI alignment research at the Research Augmentation Hackathon! As AI safety researchers, we pour countless hours into crucial work. It's time we built tools to accelerate our efforts! Join us in creating AI assistants that could supercharge the very research we're passionate about. Date: July 26th to 28th, online and in-person Prizes: $2,000 in prizes Why join? * Build tools that matter for the future of AI * Learn from top minds in AI alignment * Boost your skills and portfolio We've got a Hackbook with an exciting project to work on waiting for you! No advanced AI knowledge required - just bring your creativity! Register now: Sign up on the website here, and don't miss this chance to shape the future of AI research!

Mental health org in India that follows the paraprofessional model https://reasonstobecheerful.world/maanasi-mental-health-care-women/ #mental-health-cause-area

Meta has just released Llama 3.1 405B. It's open-source and in many benchmarks it beats GPT-4o and Claude 3.5 Sonnet: Zuck's letter "Open Source AI Is the Path Forward".

‘Five Years After AGI’ Focus Week happening over at Metaculus. Inspired in part by the EA Forum’s recent debate week, Metaculus is running a “focus week” this week, aimed at trying to make intellectual progress on the issue of “What will the world look like five years after AGI (assuming that humans are not extinct)[1]?” Leaders of AGI companies, while vocal about some things they anticipate in a post-AGI world (for example, bullishness in AGI making scientific advances), seem deliberately vague about other aspects. For example, power (will AGI companies have a lot of it? all of it?), whether some of the scientific advances might backfire (e.g., a vulnerable world scenario or a race-to-the-bottom digital minds takeoff), and how exactly AGI will be used for “the benefit of all.” Forecasting questions for the week range from “Percentage living in poverty?” to “Nuclear deterrence undermined?” to “‘Long reflection’ underway?” Those interested: head over here. You can participate by: * Forecasting * Commenting * Comments are especially valuable on long-term questions, because the forecasting community has less of a track record at these time scales.[2][3] * Writing questions * There may well be some gaps in the admin-created question set.[4] We welcome question contributions from users. The focus week will likely be followed by an essay contest, since a large part of the value in this initiative, we believe, lies in generating concrete stories for how the future might play out (and for what the inflection points might be). More details to come. 1. ^ This is not to say that we firmly believe extinction won’t happen. I personally put p(doom) at around 60%. At the same time, however, as I have previously written, I believe that more important trajectory changes lie ahead if humanity does manage to avoid extinction, and that it is worth planning for these things now. 2. ^ Moreover, I personally take Nuño Sempere’s “Hurdles of using forecasting as a tool for making sense of AI progress” piece seriously, especially the “Excellent forecasters and Superforecasters™ have an imperfect fit for long-term questions” part. With short-term questions on things like geopolitics, I think one should just basically defer to the Community Prediction. Conversely, with certain long-term questions I believe it’s important to interrogate how forecasters are reasoning about the issue at hand before assigning their predictions too much weight. Forecasters can help themselves by writing comments that explain their reasoning. 3. ^ In addition, stakeholders we work with, who look at our questions with a view to informing their grantmaking, policymaking, etc., frequently say that they would find more comments valuable in helping bring context to the Community Prediction. 4. ^ All blame on me, if so.

A lot of people have said sharing these notes were helpful, so sharing it here on the EAF! Here are notes on NTI | bio’s recent event with Dr. Lu Borio on H5N1 Bird Flu, in case anyone here would find it helpful!

I want to say thanks to people involved in the EA endeavour. I know things can be tough at times, but you didn't have to care about this stuff, but you do. Thank you, it means a lot to me. Let's make the world better!

I just preordered Yuval Noah Harari's new book

Looking for people (probably from US/UK) to do donation swaps with. My local EA group currently allows tax-deductible donations to: 1. GiveWell - Top Charities Fund 2. Animal Charity Evaluators - Top Charities Fund 3. Against Malaria Foundation 4. Good Food Institute 5. <One other org that I don't want to include here> However, I would like to donate to the following: 1. GiveWell - All Grants Fund (~$1230) 2. GiveDirectly (~$820) 3. The Humane League (~$580) If anyone is willing to donate these sums and have me donate an equal sum to one of the funds mentioned above - please contact me.

Quick[1] thoughts on the Silicon Valley 'Vibe-Shift' I wanted to get this idea out of my head and into a quick-take. I think there's something here, but a lot more to say, and I've really haven't done the in-depth research for it. There was a longer post idea I had for this, but honestly diving more than I have here into it is not a good use of my life I think. The political outlook in Silicon Valley has changed. Since the attempted assassination attempt on President Trump, the mood in Silicon Valley has changed. There have been open endorsements, e/acc has claimed political victory, and lots of people have noticed the 'vibe shift'.[2] I think that, rather than this being a change in opinions, it's more an event allowing for the beginning of a preference cascade, but at least in Silicon Valley (if not yet reflected in national polling) it has happened.  So it seems that a large section of Silicon Valley is now openly and confidently supporting Trump, and to a greater or lesser extent aligned with the a16z/e-acc worldview,[3] we know it's already reached the ears of VP candidate JD Vance. How did we get here You could probably write a book on this, so this is a highly opinionated take. But I think this is somewhat, though not exclusively, an own goal of the AI Safety movement. * As ChatGPT starts to bring AI, and AI Safety, into the mainstream discourse, the e/acc countermovement begins. It positions itself as opposite effective altruism, especially in the wake of SBF. * Guillaume Verdon, under the alias "Beff Jezos", realises the memetic weakness of the AI Safety movement and launches a full memetic war against it. Regardless of his rightness or wrongness, you do to some extent got to hand it to him. He's like right-wing Émile Torres, ambitious and relentless and driven by ideological zeal against a hated foe. * Memetic war is total war. This means nuance dies to get it to spread. I don't know if, for example, Marc Andreessen actually thinks antimalarial bednets are a 'triple threat' of badness, but it's a war and you don't take prisoners. Does Beff think that people running a uni-group session on Animal Welfare are 'basically terrorists', I don't know. But EA is the enemy, and the enemy must be defeated, and the war is total. * The OpenAI board fiasco is, I think, a critical moment here. It doesn't matter what the reasoning we've come out with at the end of the day was, I think it was perceived as 'a doomer coup' and it did radicalize the valley. In his recent post Richard Ngo called on the AI Safety movement to show more legitimacy and competence. The board fiasco torpedoed my trust in the legitimacy and competence of many senior AI safety people, so god knows how strong the update was for Silicon Valley as a whole. * As some evidence this is known in EA circles, I think this is exactly what Dwarkesh is alluding to when asked 'what happened to the EA brand'. For many people in Silicon Valley, I think the answer is that it got thrown in the dustbin of history. * This new movement became increasingly right-wing coded. Partly as a response to the culture wars in America and the increasing vitriol thrown by the left against 'tech bros', partly as a response to the California Ideology being threatened by any sense of AI oversight or regulation, and partly because EA is the enemy and EA was being increasingly seen by this group as left-wing, woke, or part of the Democratic Party due to the funding patterns of SBF and Moskovitz. I think this has led, fairly predictably, to the right-ward shift in SV and direct political affiliation with a (prospective) second Trump presidency * Across all of this my impression is that, just like with Torres, there was little to no direct pushback. I can understand not wanting to be dragged into a memetic war, or to be involved in the darker parts of Twitter discourse. But the e-acc/technooptimist/RW-Silicon-Valley movement was being driven by something, and I don't think AI Safety ever really argued against it convincingly, and definitely not in a convincing enough way to 'win' the memetic war. Like, the a16z cluster literally lied to Congress and to Parliament, but nothing much come of that fact. * I think this is very much linked to playing a strong 'inside game' to access the halls of power and no 'outside game' to gain legitimacy for that use of power. It's also I think due to EA not wanting to use social media to make its case, whereas the e-acc cluster was born and lives on social media. Where are we now? I'm not a part of the Bay Area scene and culture,[4] but it seems to me that the AI Safety movement has lost the 'mandate of heaven' to whatever extent it did have it. SB-1047 is a push to change policy that has resulted in backlash, and may result in further polarisation and counter-attempts to fight back in a zero-sum political game. I don't know if it's constitutional for a Trump/Vance administration to use the Supremacy Clause to void SB-1047 but I don't doubt that they might try. Biden's executive order seems certain for the chopping block. I expect a Trump administration to be a lot less sympathetic to the Bay Area/DC AI Safety movements, and the right-wing part of Silicon Valley will be at the very least energised to fight back harder. One concerning thing for both Silicon Valley and the AI Safety movement is what happens as a result of the ideological consequences of SV accepting this trend. Already a strong fault-line is the extreme social conservatism and incipient nationalism brought about by this. In the recent a16z podcast, Ben Horowitz literally accuses the Biden administration of breaking the rule of law, and says nothing about Trump literally refusing to concede the 2020 election and declaring that there was electoral fraud. Mike Solana seems to think that all risks of democratic backsliding under a Trump administration were/are overblown (or at least that people in the Bay agreeing was preference falsification). On the Moments-of-Zen Podcast (which has also hosted Curtis Yarvin twice), Balaji Srinivasan accused the 'Blue Tribe' of ethnically cleansing him out of SF[5] and called on the grey tribe to push all the blues out of SF. e-acc sympathetic people are noting that anti-trans ideas bubbling up in the new movement. You cannot seriously engage with ideas and shape them without those ideas changing you.[6] This right-wing shift will have further consequences, especially under a second Trump presidency. What next for the AI Safety field? I think this is a bad sign for the field of AI Safety. Political polarisation has escaped AI for a while. Current polls may lean in support , but polls and political support are fickle, especially in the age of hyper-polarisation.[7] I feel like my fears around the perception of Open Philanthropy are re-occuring here but for the AI Safety movement at large.  I think the consistent defeats to the e-acc school and the fact that the tech sector as a whole seems very much unconvinced by the arguments for AI Safety should at some point lead to a reflection from the movement. Where you stand on this very much depends on your object-level beliefs. While this is a lot of e-acc discourse around transhumanism, replacing humanity, and the AI eschaton, I don't really buy it. I think that they don't think ASI is possible soon, and thus all arguments for AI Safety are bunk. Now, while the tech sector as a whole might not be as hostile, they don't seem at all convinced of the 'ASI-soon' idea. A key point I want to emphasise is that one cannot expect to wield power successfully without also having legitimacy.[8] And to the extent that the AI Safety movement's strategy is trying to thread this needle it will fail. Anyway, long ramble over, and given this was basically a one-shot ramble it will have many inaccuracies and flaws. Nevertheless I hope that it can be directionally useful and lead to productive discussion. 1. ^ lol, lmao 2. ^ See here, here, and here. These examples are from Twitter because, for better or for worse, it seems much of SV/tech opinions are formed by Twitter discourse. 3. ^ Would be very interested to hear the thoughts of people in the Bay on this 4. ^ And if invited to be I would almost certainly decline, 5. ^ He literally used the phrase 'ethnically cleanse'. This is extraordinarily dangerous language in a political context. 6. ^ A good example in fiction is in Warhammer40K, where Horus originally accepts the power of Chaos to fight against Imperial Tyranny, but ends up turning into their slave. 7. ^ Due to polarisation, views can dramatically shift on even major topics such as the economy and national security (i know these are messy examples!). Current poll leads for AI regulation should not, in any way, be considered secure 8. ^ I guess you could also have overwhelming might and force, but even that requires legitimacy. Caesar needed to be seen as legitimate by Marc Anthony, Alexander didn't have the legitimacy to get his army to cross the Hyphasis etc.

What is it for EA to thrive?  EA Infrastructure Fund's Plan to Focus on Principles-First EA includes a proposal: > The EA Infrastructure Fund will fund and support projects that build and empower the  community of people trying to identify actions that do the greatest good from a scope-sensitive and impartial welfarist view.   And a rationale (there's more detail in the post): >   > > * [...] EA is doing something special.  > * [...]  fighting for EA right now could make it meaningfully more likely to thrive long term. > * [...]  we could make EA much better than it currently is - particularly on the “beacon for thoughtful, sincere, and selfless” front. [...] Here I’m spending some time thinking about this, in particular: * What does it mean for EA to thrive?  * What projects could push EA in the direction of thriving?    (I work at EAIF. These are my personal views/ thoughts. I’m not speaking on behalf of EAIF here)

I'm extremely excited that EAGxIndia 2024 is confirmed for October 19–20 in Bengaluru! The team will post a full forum post with more details in the coming days, but I wanted a quick note to get out immediately so people can begin considering travel plans. You can sign up to be notified about admissions opening, or to express interest in presenting, via the forms linked on the event page: https://www.effectivealtruism.org/ea-global/events/eagxindia-2024 Hope to see many of you there!!

I wanted to figure out where EA community building has been successful. Therefore, I asked Claude to use EAG London 2024 data to assess the relative strength of EA communities across different countries. This quick take is the result.  The report presents an analysis of factors influencing the strength of effective altruism communities across different countries. Using attendance data from EA Global London 2024 as a proxy for community engagement, we employed multiple regression analysis to identify key predictors of EA participation. The model incorporates geographic, economic, linguistic, and social factors, explaining 52.3% of the variance in per capita attendance.  Methodology Data Source * Per capita attendance at EAG London 2024 for 62 countries Variables 1. Geographic proximity (distance from London, UK) 2. Economic development (GDP per capita) 3. English language proficiency 4. Income inequality (Gini coefficient) 5. Youth population percentage 6. Social media penetration 7. Religiosity 8. Trust in institutions 9. Philanthropic culture Analytical Approach Multiple regression analysis with per capita EAG London 2024 attendance as the dependent variable Key Findings 1. Model Performance: * R-squared value: 0.523 * The model explains 52.3% of the variance in per capita attendance 2. Significant Predictors: * GDP per capita (positive relationship) * Distance from London (negative relationship) * Philanthropic culture (positive relationship) * Social media penetration (positive relationship) * English proficiency (positive relationship) 3. Non-significant Factors: * Gini coefficient * Youth population percentage * Religiosity * Trust in institutions 4. Top 10 Countries with Strongest EA Communities (based on positive residuals): 1. Switzerland 2. Norway 3. Netherlands 4. Estonia 5. Sweden 6. Singapore 7. Finland 8. Czech Republic 9. Australia 10. Ireland Interpretation of Results 1. Economic Factors: GDP per capita emerges as the strongest predictor of EA engagement, suggesting that economic development plays a crucial role in fostering EA communities. 2. Geographic Considerations: Distance from London negatively correlates with engagement, which is expected given that the event was held in London. This finding should be interpreted cautiously: a) It may reflect the ease of attending for nearby countries rather than overall EA engagement. b) The strong performance of some distant countries (e.g., Australia, Singapore) suggests that highly engaged EA communities can overcome geographic barriers. 3. Cultural Factors: A country's philanthropic culture strongly predicts EA engagement, highlighting the importance of pre-existing charitable inclinations. 4. Technological Infrastructure: Social media penetration positively correlates with EA engagement, underscoring the role of online connectivity in community building. 5. Language: English proficiency remains a significant factor, though its impact is less pronounced than economic and cultural variables. Limitations 1. Proxy Measure: EAG attendance may not fully capture all forms of EA engagement. 2. Single Time Point: The analysis is based on one conference in one year, potentially missing temporal variations. 3. Unmeasured Variables: Other factors not included in the model may influence EA community strength. 4. Linear Model Assumptions: The analysis assumes linear relationships, potentially missing complex interactions or non-linear effects. 5. Host Country Bias: The UK's attendance is likely inflated due to hosting the event, potentially skewing overall results.