Written by Claude, and very lightly edited.
In a recent episode of The Diary of a CEO podcast, guest Bryan Johnson, founder of Kernel and the Blueprint project, laid out a thought-provoking perspective on what he sees as the most important challenge and opportunity of our...
Executive summary: Sustainable fishing policies and demand reductions for wild-caught aquatic animals may counterintuitively increase fishing catch in the near term, but persistent demand reductions could potentially decrease catch over longer timelines.
Key points:
In this "quick take", I want to summarize some my idiosyncratic views on AI risk.
My goal here is to list just a few ideas that cause me to approach the subject differently from how I perceive most other EAs view the topic. These ideas largely push me in the direction...
In particular, I am persuaded by the argument that, because evaluation is usually easier than generation, it should be feasible to accurately evaluate whether a slightly-smarter-than-human AI is taking unethical actions, allowing us to shape its rewards during training accordingly. After we've aligned a model that's merely slightly smarter than humans, we can use it to help us align even smarter AIs, and so on, plausibly implying that alignment will scale to indefinitely higher levels of intelligence, without necessarily breaking down at any physically realistic point.
This reasoning seems to imply that you could use GPT-2 to oversee GPT-4 by boostrapping from a chain of models of scales between GPT-2 and GPT-4. However, this isn't true, the weak-to-strong generalization paper finds that this doesn't work and indeed bootstrapping like this doesn't help at all for ChatGPT reward modeling (it helps on chess puzzles and for nothing else they investigate I believe).
I think this sort of bootstrapping argument might work if we could ensure that the each model in the chain was sufficiently aligned and capable of reasoning that it would carefully reason about what humans would want if they were more knowledgeable and then rate outputs based on this. However, I don't think GPT-4 is either aligned enough or capable enough that we see this behavior. And I still think it's unlikely it works under these generous assumptions (though I won't argue for this here).
In fact, it is difficult for me to name even a single technology that I think is currently underregulated by society.
The obvious example would be synthetic biology, gain-of-function research, and similar.
I also think AI itself is currently massively underregulated even entirely ignoring alignment difficulties. I think the probability of the creation of AI capable of accelerating AI R&D by 10x this year is around 3%. It would be extremely bad for US national interests if such an AI was stolen by foreign actors. This suffices for regulation ensuring very high levels of security IMO. And this is setting aside ongoing IP theft and similar issues.
The obvious example would be synthetic biology, gain-of-function research, and similar.
Can you explain why you suspect these things should be more regulated than they currently are?
As you may have noticed, 80k After Hours has been releasing a new show where I and some other 80k staff sit down with a guest for a very free form, informal, video(!) discussion that sometimes touches on topical themes around EA and sometimes… strays a bit further afield...
Feedback on third episode: Also really liked it! Felt different from the first two. Less free-wheeling, more clearly useful. (Still much more on the relaxed, informal side than main-feed 80k podcasts.)
Felt very useful to get an inside perspective on what 80k thinks its doing with career advising. I really appreciated Dwarkesh kicking the tires on the theory of change ("why not focus 100% on the tails?"), as well as the responses.
It wasn't entirely an easy listen. I identify with the common EA tropes of: trying to push myself to be more ambitious, but this ...
Pandemic security aims to safeguard the future of civilization from exponentially spreading biological threats. Despite the world's failure to contain SARS-CoV-2, the existence of far more lethal and transmissible pathogens that afflict animals...
I am surprised I only now discovered this paper. In addition to Jeff's excellent points above, what stood out to me was that the paper contained both likelihoods of different scenarios as well as what I think is some of the more transparent reasoning behind these likelihood numbers. And the numbers are uncomfortably high!
There is more detail on how the likelihoods were arrived at in the paper itself - the last column is only a summary.
Vaccines saved 150M+ lives over the past 50 years, including 100M+ infants and nearly 100M lives from Measles alone:
https://www.gavi.org/vaccineswork/new-data-shows-vaccines-have-saved-154-million-lives-past-50-years
https://www.who.int/news/item/24-04-2024-global-immunization-efforts-have-saved-at-least-154-million-lives-over-the-past-50-years
This is an extremely "EA" request from me but I feel like we need a word for people (i.e. me) who are Vegans but will eat animal products if they're about to be thrown out. OpportuVegan? UtilaVegan?
First in-ovo sexing in the US
Egg Innovations announced that they are "on track to adopt the technology in early 2025." Approximately 300 million male chicks are ground up alive in the US each year (since only female chicks are valuable) and in-ovo sexing would prevent this.
UEP originally promised to eliminate male chick culling by 2020; needless to say, they didn't keep that commitment. But better late than never!
Congrats to everyone working on this, including @Robert - Innovate Animal Ag, who founded an organization devoted to pushing this technology.[1]
Egg Innovations says they can't disclose details about who they are working with for NDA reasons; if anyone has more information about who deserves credit for this, please comment!
He said he was on a panel at EA Global and mentions PlayPumps, a favourite EA example in this 2015 post. Here's the YouTube video of the EA Global panel discussion. He's surely aware of EA.