Effective Altruism Forum
EA Forum

Quick takes

I can highly recommend following Sentinel's weekly minutes, a weekly update from superforecasters on the likelihood of any events which plausibly could cause worldwide catastrophe.

Perhaps the weekly newsletter I look the most forward to at this point. Read previous issues here:

https://sentinel-team.org/blog/

EffectiveAdvocate🔸

14h

They seem really good! I feel like an idiot for asking this, but where on their website can I subscribe to the newsletter?

Phib6h1

IDK if this actually works since I only just signed up, but, the "Join us" button in top right leads to, "https://sentinel-team.org/contact/"

Seems you can add yourself to mailing list from there.

MathiasKB🔸

13h

Good question, not sure how I get it into my email actually, I can't find it on the website either edit: I think it's through the forecasting newsletter

Linch's Quick takes

Linch8h8

Anthropic issues questionable letter on SB 1047 (Axios). I can't find a copy of the original letter online.

Eli Rose7h6

The full letter is available here — was recently posted online as part of this tweet thread.

Ben Millwood's Quick takes

Ben Millwood17h11

Community

I've been reviewing some old Forum posts for an upcoming post I'm writing, and incidentally came across this by Howie Lempel for noticing in what spirit you're engaging with someone's ideas:

"Did I ask this question because I think they will have a good answer or because I think they will not have a good answer?"

I felt pretty called out :P

To be fair, I think the latter is sometimes a reasonable persuasive tactic, and it's fine to put yourself in a teaching role rather than a learning role if that's your endorsed intention and the other party is on board... (read more)

JWS's Quick takes

JWS 🔸7d47

Quick^[1] thoughts on the Silicon Valley 'Vibe-Shift'

I wanted to get this idea out of my head and into a quick-take. I think there's something here, but a lot more to say, and I've really haven't done the in-depth research for it. There was a longer post idea I had for this, but honestly diving more than I have here into it is not a good use of my life I think.

The political outlook in Silicon Valley has changed.

Since the attempted assassination attempt on President Trump, the mood in Silicon Valley has changed. There have been open endorsements, e/acc ... (read more)

Showing 3 of 13 replies (Click to show all)

Joseph Miller

Nit: Beff Jezos was doxxed and repeating him name seems uncool, even if you don't like him.

JWS 🔸

I think this case it's ok (but happy to change my mind) - afaict he owns the connection now and the two names are a bit like separate personas. He's gone on podcasts under his true name, for instance.

Joseph Miller1d0

Ok thanks, I didn't know that.

hbesceli's Quick takes

hbesceli7d19

What is it for EA to thrive?

EA Infrastructure Fund's Plan to Focus on Principles-First EA includes a proposal:

The EA Infrastructure Fund will fund and support projects that build and empower the community of people trying to identify actions that do the greatest good from a scope-sensitive and impartial welfarist view.

And a rationale (there's more detail in the post):

[...] EA is doing something special.
[...] fighting for EA right now could make it meaningfully more likely to thrive long term.
[...] we could make EA

David_Moss

I share this impression. Also, we see that satisfaction is lower among people who have been in EA longer compared to newer EAs (though this is not true for self-reported engagement), which seems potentially related. Note that we would expect to see pressure in the opposite direction due to less satisfied people dropping out over time. I think this implies that there is a substantive non-quirky effect. That said, I imagine some of this may be explained by new EAs simply being particularly enthusiastic in ways which explain stronger identification with EA and higher satisfaction.[1] One dynamic which I expect explains this is the narcissism of small differences, as people become closer to EA, differences and disagreements become more salient, and so people may become more inclined to want to distance themselves from EA as a whole. 1. ^ I'm not suggesting any particular causal theory about the relationship between satisfaction and identification.

hbesceli

EA Jobs, Scarcity and Performance It seems like: 1. For many people, having an EA job is pretty important. 2. It’s pretty competitive and many people who want EA jobs will not in fact get them. There’s been some discussion related to this on the EA Forum, focusing in particular on jobseekers. I’m also interested in exploring this dynamic with people who are working in EA jobs. I expect EA job scarcity not only have an impact on EA jobseekers, but also people who are working in EA jobs. Given 1 and 2, it seems like for people working in EA jobs it will be pretty important for them to keep their jobs. If the job market is competitive it may not be obvious that they can get another one. (For people who have got one EA job, it will presumably be easier to get another, but maybe not guaranteed). For someone who’s in a position of scarcity about their EA job, I can imagine this meaning they focus primarily on performing well/ being seen to perform well. This becomes a problem if what counts as performing well and what is actually good to do comes into conflict. Eg. this might involve things like: * Agreeing with the organisational strategy or one’s manager more than one endorses * Focusing on ensuring that they have achieved certain outputs independent of whether that output seems good In general I expect that under conditions of scarcity people will be less able to do valuable work (and I mean valuable here as ‘actually good’ as opposed to ‘work that is perceived to be valuable). (If I’m right about this, then one potential answer to ‘what is it for EA to thrive’, is: EAs aren’t in a position of scarcity). Things I’d be interested to ask people who are working at EA jobs to understand whether this is in fact a thing: * How concerned are you about your perceived performance? * If your employer/ manager/ funder/ relevant people said something like: ‘We have full confidence in you, your job is guaranteed and we want you to focus on whatever you th

David_Moss2d2

If your employer/ manager/ funder/ relevant people said something like: ‘We have full confidence in you, your job is guaranteed and we want you to focus on whatever you think is best’ - would that change what you focus on? How much?

My personal impression is that significant increases in unrestricted funding (even if it were a 1-1 replacement for restricted funding) would dramatically change orgs and individual prioritisations in many cases.

To the extent that one thinks that researchers are better placed to identify high value research questions (which, to be clear, one may not in many cases), this seems bad.

Heramb Podar's Quick takes

Heramb Podar2d0

With open-source models being released and on ramps to downstream innovation lowering, the safety challenges may not be a single threshold but rather an ongoing, iterative cat-and-mouse game.

Just underscores the importance of people in the policy/safety field thinking far ahead

Phib's Quick takes

Phib2d-2

Reading and engaging with the forum as good for a meta reason, engaging and encouraging other people to keep making posts because engagement seems to exist and they’re incentivized to post. Or even more junior people to try and contribute, idk what the ea forum felt like ~10 years ago, but probably lower standards for engagement.

jacquesthibs's Quick takes

jacquesthibs2d16

AI safety

Hey everyone, in collaboration with Apart Research, I'm helping organize a hackathon this weekend to build tools for accelerating alignment research. This hackathon is very much related to my effort in building an "Alignment Research Assistant."

Here's the announcement post:

2 days until we revolutionize AI alignment research at the Research Augmentation Hackathon!

As AI safety researchers, we pour countless hours into crucial work. It's time we built tools to accelerate our efforts! Join us in creating AI assistants that could supercharge the very research w... (read more)

Dave Cortright's Quick takes

Dave Cortright2d0

Global health

Mental health org in India that follows the paraprofessional model
https://reasonstobecheerful.world/maanasi-mental-health-care-women/

#mental-health-cause-area

defun's Quick takes

defun4d14

AI safety

Meta has just released Llama 3.1 405B. It's open-source and in many benchmarks it beats GPT-4o and Claude 3.5 Sonnet:

Zuck's letter "Open Source AI Is the Path Forward".

EJT

Wait, all the LLMs get 90+ on ARC? I thought LLMs were supposed to do badly on ARC.

JWS 🔸

It's an unfortunate naming clash, there are different ARC Challenges: ARC-AGI (Chollet et al) - https://github.com/fchollet/ARC-AGI ARC (AI2 Reasoning Challenge) - https://allenai.org/data/arc These benchmarks are reporting the second of the two. LLMs (at least without scaffolding) still do badly on ARC, and I'd wager Llama 405B still doesn't do well on the ARC-AGI challenge, and it's telling that all the big labs release the 95%+ number they get on AI2-ARC, and not whatever default result they get with ARC-AGI... (Or in general, reporting benchmarks where they can go OMG SOTA!!!! and not helpfully advance the general understanding of what models can do and how far they generalise. Basically, traditional benchmark cards should be seen as the AI equivalent of "IN MICE")

EJT4d0

Thanks!

Nathan Young's Quick takes

Nathan Young4d7

I want to say thanks to people involved in the EA endeavour. I know things can be tough at times, but you didn't have to care about this stuff, but you do. Thank you, it means a lot to me. Let's make the world better!

kta's Quick takes

kta4d9

Biosecurity & pandemics

A lot of people have said sharing these notes were helpful, so sharing it here on the EAF! Here are notes on NTI | bio’s recent event with Dr. Lu Borio on H5N1 Bird Flu, in case anyone here would find it helpful!

Will Aldred's Quick takes

Will Aldred4d14

Forecasting

‘Five Years After AGI’ Focus Week happening over at Metaculus.

Inspired in part by the EA Forum’s recent debate week, Metaculus is running a “focus week” this week, aimed at trying to make intellectual progress on the issue of “What will the world look like five years after AGI (assuming that humans are not extinct)^[1]?”

Leaders of AGI companies, while vocal about some things they anticipate in a post-AGI world (for example, bullishness in AGI making scientific advances), seem deliberately vague about other aspects. For example, power (will AGI companies hav... (read more)

Benny Smith's Quick takes

Benny Smith5d-7

SiebeRozendal's Quick takes

SiebeRozendal25d6

AI safetyShow more

I am very concerned about the future of US democracy and rule of law and its intersection with US dominance in AI. On my Manifold question, forecasters (n=100) estimate a 37% that the US will no longer be a liberal democracy by the start of 2029 [edit: as defined by V-DEM political scientists].

Project 2025 is an authoritarian playbook, including steps like 50,000 political appointees (there are ~4,000 appointable positions, of which ~1,000 change in a normal presidency). Trump's chances of winning are significantly above 50%, and even if he loses, Republic... (read more)

Showing 3 of 7 replies (Click to show all)

Fermi–Dirac Distribution

V-Dem indicators seem to take into account statements made by powerful politicians, not only their policies or other deeds. For example, I found this in one of their annual reports: My guess is that statements made by Trump were extreme outliers in how they betrayed little respect to democratic institutions, compared to statements made by earlier US presidents, and that affected their model. I think that's reasonable. It might not be fully reflective of lived reality for US citizens at the moment the statements are made, but it sure captures the beliefs and motives of powerful people, which is predictive of their future actions. Indeed, one way to see the drop in 2017 is that it was able to predict a major blow to American democracy (Trump refusing to concede an election) 4 years in advance.

Larks5d1

I'm not really sure this contradicts what I said very much. I agree the V-Dem evaluators were reacting to Trump's comments, and this made them reduce their rating for America. I think they will react to Trump's comments again in the future, and this will again make them likely reduce their rating for America. This will happen regardless of whether policy changes, and be poorly calibrated for actual importance - contra V-Dem, Trump getting elected was less important than the abolition of slavery. Since I think Siebe was interested in policy changes rather than commentary, this means V-Dem is a bad metric for him to look at.

SiebeRozendal

25d

I don't really understand why so many people are downvoting this. If anyone would like to explain, that'd be nice!

James Herbert's Quick takes

James Herbert8d3

I wanted to figure out where EA community building has been successful. Therefore, I asked Claude to use EAG London 2024 data to assess the relative strength of EA communities across different countries. This quick take is the result.

The report presents an analysis of factors influencing the strength of effective altruism communities across different countries. Using attendance data from EA Global London 2024 as a proxy for community engagement, we employed multiple regression analysis to identify key predictors of EA participation. The model incorpo... (read more)

Alix Pham6d1

I'm curious if you fed Claude the variables or if it fetched them itself? In the latter case, there's a risk of having the wrong values, isn't there?

Otherwise, really interesting project. Curious of the insights to take out of this, esp. for me the fact that Switzerland comes up first. Also surprising that Germany's not on the list, maybe?

Thanks!

Lorenzo Buonanno🔸

I'm surprised that the "top 10" doesn't include Denmark, Austria, Belgium, and Germany, since they all have more population-adjusted participants than Ireland, are not English-speaking, are more distant from London, and have lower GDP per capita[1] Are we using different data? In general, I'm a bit sceptical of these analyses, compared to looking at the countries/cities with the most participants in absolute terms. I also expect Claude to make lots of random mistakes. 1. ^ But of course, Ireland's GDP is very artificial

James Herbert

But absolute terms isn’t very useful if we’re trying to spot success stories, right? Or am I misunderstanding something? But yeah, something seems off about Ireland. The rest of the list feels quite good though. David Moss said they have some per capita estimates in the pipeline, so I’m excited to see what they produce!

Guy Raveh's Quick takes

Guy Raveh6d8

Looking for people (probably from US/UK) to do donation swaps with. My local EA group currently allows tax-deductible donations to:

GiveWell - Top Charities Fund
Animal Charity Evaluators - Top Charities Fund
Against Malaria Foundation
Good Food Institute
<One other org that I don't want to include here>

However, I would like to donate to the following:

GiveWell - All Grants Fund (~$1230)
GiveDirectly (~$820)
The Humane League (~$580)

If anyone is willing to donate these sums and have me donate an equal sum to one of the funds mentioned above - please contact me.

Peter Wildeford's Quick takes

Peter Wildeford3mo35

This could be a long slog but I think it could be valuable to identify the top ~100 OS libraries and identify their level of resourcing to avoid future attacks like the XZ attack. In general, I think work on hardening systems is an underrated aspect of defending against future highly capable autonomous AI agents.

Showing 3 of 6 replies (Click to show all)

Joseph_Chu

2mo

Relevant XKCD comic. To further comment, this seems like it might be an intractable task, as the term "dependency hell" kind of implies. You'd have to scrap likely all of GitHub and calculate what libraries are used most frequently in all projects to get an accurate assessment. Then it's not clear to me how you'd identify their level of resourcing. Number of contributors? Frequency of commits? Also, with your example of the XZ attack, it's not even clear who made the attack. If you suspect it was, say, the NSA, would you want to thwart them if their purpose was to protect American interests? (I'm assuming you're pro-American) Things like zero-days are frequently used by various state actors, and it's a morally grey question whether or not those uses are justified. I also, as a comp sci and programmer, have doubts you'd ever be able to 100% prevent the risk of zero-days or something like the XZ attack from happening in open source code. Given how common zero-days seem to be, I suspect there are many in existing open source work that still haven't been discovered, and that XZ was just a rare exception where someone was caught. Yes, hardening these systems might somewhat mitigate the risk, but I wouldn't know how to evaluate how effective such an intervention would be, or even, how you'd harden them exactly. Even if you identify the at-risk projects, you'd need to do something about them. Would you hire software engineers to shore up the weaker projects? Given the cost of competent SWEs these days, that seems potentially expensive, and could compete for funding with actual AI safety work.

Matt_Lerner

2mo

I'd be interested in exploring funding this and the broader question of ensuring funding stability and security robustness for critical OS infrastructure. @Peter Wildeford is this something you guys are considering looking at?

Pat Myron7d5

@Peter Wildeford @Matt_Lerner interested in similar. This in-depth analysis' was a bit strict in my opinion looking at file-level criteria:
https://www.metabase.com/blog/bus-factor

These massive projects were mostly maintained by 1 person last I checked a year ago:
https://github.com/curl/curl/graphs/contributors
https://github.com/vuejs/vue/graphs/contributors
https://github.com/twbs/bootstrap/graphs/contributors
https://github.com/laravel/laravel/graphs/contributors
https://github.com/pallets/flask/graphs/contributors
https://github.com/expressjs/express/graphs/... (read more)

Arthur Malone's Quick takes

Arthur Malone7d13

Building EAShow more

I'm extremely excited that EAGxIndia 2024 is confirmed for October 19–20 in Bengaluru! The team will post a full forum post with more details in the coming days, but I wanted a quick note to get out immediately so people can begin considering travel plans. You can sign up to be notified about admissions opening, or to express interest in presenting, via the forms linked on the event page:

https://www.effectivealtruism.org/ea-global/events/eagxindia-2024

Hope to see many of you there!!

Benjamin M.'s Quick takes

Benjamin M.10d10

Existential riskShow more

Do you like SB 1047, the California AI bill? Do you live outside the state of California? If you answered "yes" to both these questions, you can e-mail your state legislators and urge them to adopt a similar bill for your state. I've done this and am currently awaiting a response; it really wasn't that difficult. All it takes is a few links to good news articles or opinions about the bill and a paragraph or two summarizing what it does and why you care about it. You don't have to be an expert on every provision of the bill, nor do you have to have a group of people backing you. It's not nothing, but at least for me it was a lot easier than it sounded like it would be. I'll keep y'all updated on if I get a response.

Benjamin M.9d2

Both my state senator and my state representative have responded to say that they'll take a look at it. It's non-commital, but it still shows how easy it is to contact these people.