This seems cool!
I could imagine that many people will gravitate towards moral parliament approaches even when all the moral considerations are known. If moral anti-realism is true, there may not come a point in moral reflection under idealized circumstances where it suddenly feels like "ah, now the answer is obvious." So, we can also think of moral parliament approaches as a possible answer to undecidedness when all the considerations are laid open.
I feel like only seeing it as an approach to moral uncertainty (so that, if we knew more about moral considerations, we'd just pick one of the first-order normative theories) is underselling the potential scope of applications of this approach.
Secondly, prioritizing competence. Ultimately, humanity is mostly in the same boat: we're the incumbents who face displacement by AGI. Right now, many people are making predictable mistakes because they don't yet take AGI very seriously. We should expect this effect to decrease over time, as AGI capabilities and risks become less speculative. This consideration makes it less important that decision-makers are currently concerned about AI risk, and more important that they're broadly competent, and capable of responding sensibly to confusing and stressful situations, which will become increasingly common as the AI revolution speeds up.
I think this is a good point.
At the same time, I think you can infer that people who don't take AI risk seriously are somewhat likely to lack important forms of competence. This is inference is only probabilistic, but it's IMO pretty strong already (it's a lot stronger now than it used to be four years ago) and it'll get stronger still.
It also depends how much a specific person has been interacting with the technology; meaning, it probably applies a lot less to DC policy people, but applies more to ML scientists or people at AI labs.
Okay, what does not tolerating actual racism look like to you? What is the specific thing you're asking for here?
Up until recently, whenever someone criticized rationality or EA for being racist or for supporting racists, I could say something like the following:
"I don't actually know of anyone in these communities who is racist or supports racism. From what I hear, some people in the rationality community occasionally discuss group differences in intelligence, because this was discussed in writings by Scott Alexander, which a lot of people have read and so it gives them shared context. But I think this doesn't come from a bad place. I'm pretty sure people who are central to these communities (EA and rationality) would pretty much without exception speak up strongly against actual racists."
It would be nice if I could still say something like that, but it no longer seems like I can, because a surprising number of people have said things like "person x is quite racist, but [...] interesting ideas."
I’m not that surprised we aren’t understanding one another, we have our own context and hang ups.
Yeah, I agree I probably didn't get a good sense of where you were coming from. It's interesting because, before you made the comments in this post and in the discussion here underneath, I thought you and I probably had pretty similar views. (And I still suspect that – seems like we may have talked past each other!) You said elsewhere that last year you spoke against having Hanania as a speaker. This suggested to me that even though you value truth-seeking a lot, you also seem to think there should be some other kinds of standards. I don't think my position is that different from "truth-seeking matters a ton, but there should be some other kinds of standards." That's probably the primary reason I spent a bunch of time commenting on these topics: the impression that the "pro truth-seeking" faction in my view seemed to be failing to make even some pretty small/cheap concessions. (And it seemed like you were one of the few people who did make such concessions, so, I don't know why/if it feels like we're disagreeing a lot.)
(This is unrelated, but it's probably good for me to separate timeless discussion about norms from an empirical discussion of "How likely is it that Hanania changed a lot compared to his former self?" I do have pessimistic-leaning intuitions about the latter, but they're not very robust because I really haven't looked into this topic much, and maybe I'm just prejudiced. I understand that, if someone is more informed than me and believes confidently that Hanania's current views and personality are morally unobjectionable, it obviously wouldn't be a "small concession" for them to disinvite or not platform someone they think is totally unobjectionable! I think that can be a defensible view depending on whether they have good reasons to be confident in these things. At the same time, the reason I thought that there were small/cheap concessions that people could make that they weirdly enough didn't make, was that a bunch of people explicitly said things like "yeah he's pretty racist" or "yeah he recently said things that are pretty racist" and then still proceeded to talk as though this is just normal and that excluding racists would be like excluding Peter Singer. That's where they really lost me.)
Just as a heads-up, I'm planning to get off the EA forum for a while to avoid the time-sink issues, so I may not leave more comments here anytime soon.
I feel like the controversy over the conference has become a catalyst for tensions in the involved communities at large (EA and rationality).
It has been surprisingly common for me to make what I perceive to be totally sensible point that isn't even particularly demanding (about, e.g., maybe not tolerating actual racism) and then the "pro truth-seeking faction" seem to lump me together with social justice warriors and present analogies that make no sense whatsoever. It's obviously not the case that if you want to take a principled stance against racism, you're logically compelled to have also objected to things that were important to EA (like work by Singer, Bostrom/Savulescu human enhancement stuff, AI risk, animal risk [I really didn't understand why the latter two were mentioned], etc.). One of these things is not like the others. Racism is against universal compassion and equal consideration of interests (also, it typically involves hateful sentiments). By contrast, none of the other topics are like that.
To summarize, it seems concerning if the truth-seeking faction seems to be unable to understand the difference between, say, my comments, and how a social justice warrior would react to this controversy. (This isn't to say that none of the people who criticized aspects of Manifest were motivated by further-reaching social justice concerns; I readily admit that I've seen many comments that in my view go too far in the direction of cancelling/censorship/outrage.)
Ironically, I think this is very much an epistemic problem. I feel like a few people have acted a bit dumb in the discussions I've had here recently, at least if we consider it "dumb" when someone repeatedly fails at passing Ideological Turing Tests or if they seemingly have a bit of black-and-white thinking about a topic. I get the impression that the rationality community has suffered quite a lot defending itself against cancel culture, to the point that they're now a bit (low-t) traumatized. This is understandable, but that doesn't change that it's a suboptimal state of affairs.
Offputting to whom?
If it bothers me, I can assume that some others will react similarly.
You don't have to be a member of the specific group in question to find it uncomfortable when people in your environment say things that are riling up negative sentiments against that group. For instance, twelve-year-old children are unlikely to attend EA or rationality events, but if someone there talked about how they think twelve-year olds aren't really people and their suffering matters less, I'd be pissed off too.
All of that said, I'm overall grateful for LW's existence; I think habryka did an amazing job reviving the site, and I do think LW has overall better epistemic norms than the EA forum (even though I think most of the people who I intellectually admire the most are more EAs than rationalists, if I had to pick only one label, but they're often people who seem to fit into both communities).
.I don't really think it's this. I think it is "I don't want people associating me with people or ideas like that so I'd like you to stop please".
It might be what you say for some people, but that doesn't ring true for my case (at all). (But also, compared to all the people who complained about stuff at Manifest or voiced negative opinions from the sidelines as forum users, I'm pretty sure I'm in the 33% that felt the least strongly and had fewer items to pick at.)
But let's take your case, that means you think that on the margin some notion of considerateness/kindness/agreeableness is more important than truth-seeking. Is that right?
I don't like this framing/way of thinking about it.
For one thing, I'm not sure if I want to concede the point that it is the "maximally truth-seeking" thing to risk that a community evaporatively cools itself along the lines we're discussing.
Secondly, I think the issues around Manifest I objected to weren't directly about "what topics are people allowed to talk about?."
If some person with a history of considerateness and thoughtfulness wanted to do a presentation on HBD at Manifest, or (to give an absurd example) if Sam Harris (who I think is better than average at handling delicate conversations like that) wanted to interview Douglas Murray again in the context of Manifest, I'd be like "ehh, not sure that's a good idea, but okay..." And maybe also "Well, if you're going to do this, at least think very carefully about how to communicate about why you're talking about this/what the goal of the session is." (It makes a big difference whether the framing of the session is "We know this isn't a topic most people are interested in, but we've had some people who are worried that if we cannot discuss every topic there is, we might lose what's valuable about this community, so this year, we decided to host a session on this; we took steps x, y, and z to make sure this won't become a recruiting ground for racists;" or whether the framing is "this is just like every other session.") [But maybe this example is beside the point, I'm not actually sure whether the objectionable issue was sessions on HBD/intelligence differences among groups, or whether it was more just people talking about it during group conversations.]
By contrast, if people with a history of racism or with close ties to racists attend the conference and it's them who want to talk about HBD, I'm against it. Not directly because of what's being discussed, but because of how and by whom. (But again, it's not my call to make and I'm just stating what I would do/what I think would lead to better outcomes.)
(I also thought people who aren't gay using the word "fag" sounded pretty problematic. Of course, this stuff can be moderated case-by-case and maybe a warning makes more sense than an immediate ban. Also, in fairness, it seems like the conference organizers would've agreed with that and they simply didn't hear the alleged incident when it allegedly happened.)
"Influence-seeking" doesn't quite resonate with me as a description of the virtue on the other end of "truth-seeking."
What's central in my mind when I speak out against putting "truth-seeking" above everything else is mostly a sentiment of "I really like considerate people and I think you're driving out many people who are considerate, and a community full of disagreeable people is incredibly off-putting."
Also, I think considerateness axis is not the same as the decoupling axis. I think one can be very considerate and also great at decoupling; you just have to be able to couple things back together as well.
Good points! It seems good to take a break or at least move to the meta level.
I think one emotion that is probably quite common in discussions about what norms should be (at least in my own experience) is clinging. Quoting from Joe Carlsmith's post on it:
Clinging, as I think about it, is a certain mental flavor or cluster of related flavors. It feels contracted, tight, clenched, and narrow. It has a kind of hardness, a “not OK-ness,” and a (sometimes subtle) kind of desperation. It sees scarcity. It grabs. It sees threat. It pushes away. It carries seeds of resentments and complaints. [...]
Often, in my experience, clinging seems to hijack attention and agency. It makes it harder to think, weigh considerations, and respond. You are more likely to flail, or stumble around, or to “find yourself” doing something rather than choosing to do it. And you’re more likely, as well, to become pre-occupied by certain decisions — especially if both options involve things you’re clinging in relation to — or events. Indeed, clinging sometimes seems like it treats certain outcomes as “infinitely bad,” or at least bad enough that avoiding them is something like a hard constraint. This can cause consequent problems with reasoning about what costs to pay to avoid what risks.
Clinging is also, centrally, unpleasant. But it’s a particular type of unpleasant, which feels more like it grabs and restricts and distorts who you are than e.g. a headache.
In the midst of feeling like a lot is at stake and one's values are being threatened, we may often try to push the social pendulum in our desired direction as hard as possible. However, that will have an aggravating and polarizing effect on the debate because the other side will see your attitude and think, "this person is not making any concessions whatsoever, and it seems like even though the social pendulum is already favorable to them, they'll keep pushing against us!"
So, to de-escalate these dynamics, it seems valuable to acknowledge the values that are at stake for both sides, even just to flag that you're not in favor of pushing the pendulum as far as possible.
For instance, maybe this would already feel more relaxed if the side that is concerned about losing what's valuable regarding "truth-seeking" can acknowledge that there is a bar also for them, that, if they thought they were dealing with people full of hate or people who advocate for views that predictably cause harm to others (while being aware of this but advocating for those views because of a lack of concern for the affected others), the "truth-seeking" proponents will indeed step in and not tolerate it. Likewise, the other side could maybe acknowledge that it's bad when people get shunned just based on superficial associations/vibes (to give an example of something that I think is superficial: saying "sounds like they're into eugenics" as though this should end the discussion, without pointing out any way in which what the person is discussing is hateful, lacks compassion, or is otherwise likely to cause harm). This is bad not just for well-intentioned individuals who might get unfairly ostracized, but also bad for discourse in general because people won't speak their minds any longer.
Well said.
I meant to say the exact same thing, but seem to have struggled at communicating.
I want to point out that my comment above was specifically reacting to the following line and phrasing in timunderwood's parent comment:
I also have a dislike for excluding people who have racist style views simply on that basis, with no further discussion needed, because it effectively is setting the prior for racism being true to 0 before we've actually looked at the data.
My point (and yours) is that this quoted passage would be clearer if it said "genetic group differences" instead of "racism."
Probably most people are "committed to safety" in the sense that they wouldn't actively approve conduct at their organization where executives got developers to do things that they themselves presented as reckless. To paint an exaggerated picture, imagine if some executive said the following:
"I might be killing millions of people here if something goes wrong, and I'm not super sure if this will work as intended because the developers flagged significant uncertainties and admitted that they're just trying things out essentially flying blind; still, we won't get anywhere if we don't take risks, so I'm going to give everyone the go-ahead here!"
In that scenario, I think most people would probably object quite strongly.
But as you already suggest, the more relevant question is one of willingness to go the extra mile, and of qualifications: Will the board members care about and be able to gain an informed understanding of the risks of certain training runs, product releases, or model security measures? Alternatively, will they care about (and be good at) figuring out whose opinions and judgment they can trust and defer to on these matters?
With the exodus of many of the people who were concerned about mitigating risks of AI going badly wrong, the remaining culture there will likely be more focused on upsides, on moving fast, on beating competitors, etc. There will likely be fewer alarming disagreements among high-ranking members of the organization (because the ones who had alarming disagreements already left). The new narrative on safety will likely be something like, "We have to address the fact that there was this ideological club of doomer folks who used to work at OpenAI. I think they were well-intentioned (yada yada), but they were wrong because of their ideological biases, and it's quite tragic because the technology isn't actually that risky the way we're currently building it." (This is just my guess; I don't have any direct info on what the culture is now like, so I might be wrong.)
So, my guess is that the rationales the leadership will present to board members on any given issue will often seem very reasonable ASSUMING that you go into this with the prior of "I trust that leadership has good judgment here." The challenging task for board members will be that they might have to go beyond just looking at things the way they get presented to them and ask questions that the leadership wouldn't even put on the meeting agenda. For instance, they could ask for costly signals of things the organization could do to create a healthy culture for assessing risks and for discussing acceptable risk tradeoffs. (Past events suggest that this sort of culture is less likely to exist than it was before the exodus of people who did safety-themed work.)
To summarize, my hope is for board members to take seriously the following three possibilities: (1) That there might be big risks in the AGI tech tree. (2) That org leadership might not believe in these risks or might downplay them because it's convenient for them that way. (3) That org-internal discussions on risks from AI might appear one-sided because of "evaporative cooling" (most of the people who were more particularly concerned having already left for reasons that weren't related to a person's judgment/forecasting abilities).