Effective Altruism Forum
EA Forum

B

Buck

Chief Technology Officer @ Redwood Research
6123 karmaJoined Berkeley, CA, USA

Bio

I'm Buck Shlegeris. I am the CTO of Redwood Research, a nonprofit focused on applied alignment research. Read more about us here: https://www.redwoodresearch.org/

Comments
300

Existential risk from AI and what DC could do about it (Ezra Klein on the 80,000 Hours Podcast)
Buck9
3
0

I found Ezra's grumpy complaints about EA amusing and useful. Maybe 80K should arrange to have more of their guests' children get sick the day before they tape the interviews.

Reply
Joseph Lemien's Quick takes
Buck8
3
0

I agree that we should tolerate people who are less well read than GPT-4 :P

Reply
Joseph Lemien's Quick takes
Buck1
0
0

For what it’s worth, gpt4 knows what rat means in this context: https://chat.openai.com/share/bc612fec-eeb8-455e-8893-aa91cc317f7d

Reply
How come there isn't that much focus in EA on research into whether / when AI's are likely to be sentient?
Buck14
5
1

I think this is a great question. My answers:

  • I think that some plausible alignment schemes seem like they could plausibly involve causing suffering to the AIs. I think that it seems pretty bad to inflict huge amounts of suffering on AIs, both because it's unethical and because it seems potentially inadvisable to make AIs justifiably mad at us.
  • If unaligned AIs are morally valuable, then it's less bad to get overthrown by them, and perhaps we should be aiming to produce successors who we're happier to be overthrown by. See here for discussion. (Obviously the plan A is to align the AIs, but it seems good to know how important it is to succeed at this, and making unaligned but valuable successors seems like a not-totally-crazy plan B.)
Reply
How come there isn't that much focus in EA on research into whether / when AI's are likely to be sentient?
Answer by Buck34
12
1

My attitude, and the attitude of many of the alignment researchers I know, is that this problem seems really important and neglected, but we overall don't want to stop working on alignment in order to work on this. If I spotted an opportunity for research on this that looked really surprisingly good (e.g. if I thought I'd be 10x my usual productivity when working on it, for some reason), I'd probably take it.

It's plausible that I should spend a weekend sometime trying to really seriously consider what research opportunities are available in this space.

My guess is that a lot of the skills involved in doing a good job of this research are the same as the skills involved in doing good alignment research.

Reply
A freshman year during the AI midgame: my approach to the next year
Buck2
0
0

Thanks Lizka. I think you mean to link to this video: 

Reply
Can we evaluate the "tool versus agent" AGI prediction?
Buck5
0
0

Holden's beliefs on this topic have changed a lot since 2012. See here for more.

Reply
Announcing CEA’s Interim Managing Director
Buck54
16
1

I really like this frame. I feel like EAs are somewhat too quick to roll over and accept attacks from dishonest bad actors who hate us for whatever unrelated reason.

Reply
Do you worry about totalitarian regimes using AI Alignment technology to create AGI that subscribe to their values?
Answer by Buck20
8
0

Yes, I think this is very scary. I think this kind of risk is at least 10% as important as the AI takeover risks that I work on as an alignment researcher.

Reply
Taking a leave of absence from Open Philanthropy to work on AI safety
Buck12
6
1

I don't think Holden agrees with this as much as you might think. For example, he spent a lot of his time in the last year or two writing a blog.

Reply
Load more