I think that the political activation of Silicon Valley is the sort of thing which could reshape american politics, and that twitter is a leading indicator.

I don't disagree with this statement, but also think the original comment is reading into twitter way too much.

I haven't seen those comments

Scroll down to see comments.

Once again, if you disagree, I'd love to actually here why.

I think you're reading into twitter way too much.

absence of evidence of good arguments against it is evidence of the absence of said arguments. (tl;dr - AI Safety people, engage with 1a3orn more!)

There are many (edit: 2) comments responding and offering to talk. 1a3orn doesn't appear to have replied to any of these comments. (To be clear, I'm not saying they're under any obligation here, just that there isn't a absence of attempted engagement and thus you shouldn't update in the direction you seem to be updating here.)

The limited duty exemption has been removed from the bill which probably makes compliance notably more expensive while not improving safety. (As far as I can tell.)

This seems unfortunate.

I think you should still be able to proceed in a somewhat reasonable way by making a safety case on the basis of insufficient capability, but there are still additional costs associated with not getting an exemption.

Further, you can't just claim an exemption prior to starting training if you are behind the frontier which will substantially increase the costs on some actors.

This makes me more uncertain about whether the bill is good, though I think it will probably still be net positive and basically reasonable on the object level. (Though we'll see about futher amendments, enforcement, and the response from society...)

(LW x-post)

I agree that these models assume something like "large discontinuous algorithmic breakthroughs aren't needed to reach AGI".

(But incremental advances which are ultimately quite large in aggregate and which broadly follow long running trends are consistent.)

However, I interpreted "current paradigm + scale" in the original post as "the current paradigm of scaling up LLMs and semi-supervised pretraining". (E.g., not accounting for totally new RL schemes or wildly different architectures trained with different learning algorithms which I think are accounted for in this model.)

Both AI doomers and accelerationists will come out looking silly, but will both argue that we are only an algorithmic improvement away from godlike AGI.

A common view is a median around 2035-2050 with substantial (e.g. 25%) mass in the next 6 years or so.

This view is consistent with both thinking:

• LLM progress is likely (>50%) to stall out.
• LLMs are plausibly going to quickly scale into very powerful AI.

(This is pretty similar to my view.)

I don't think many people think "we are only an algorithmic improvement away from godlike AGI". In fact, I can't think of anyone who thinks this. Some people think that 1 substantial algorithmic advance + continued scaling/general algorithmic improvement, but the continuation of other improvements is key.

Yes, I meant central to me personally, edited the comment to clarify.

I basically agree with this with some caveats. (Despite writing a post discussing AI welfare interventions.)

I discuss related topics here and what fraction of resources should go to AI welfare. (A section in the same post I link above.)

The main caveats to my agreement are:

• From a deontology-style perspective, I think there is a pretty good case for trying to do something reasonable on AI welfare. Minimally, we should try to make sure that AIs consent to their current overall situation insofar as they are capable of consenting. I don't put a huge amount of weight on deontology, but enough to care a bit.
• As you discuss in the sibling comment, I think various interventions like paying AIs (and making sure AIs are happy with their situation) to reduce takeover risk are potentially compelling and they are very similar to AI welfare interventions. I also think there is a weak decision theory case that blends in with deontology case from the prior bullet.
• I think that there is a non-trivial chance that AI welfare is a big and important field at the point when AIs are powerful regardless of whether I push for such a field to exist. In general, I would prefer that important fields related to AI have better more thoughtful views. (Not with any specific theory of change, just a general heuristic.)

My impression is these arguments are important to very few AI-welfare-prioritizers

FWIW, these motivations seem reasonably central to me personally, though not my only motivations.