Since we normally let humans accumulate wealth and become powerful via lawful means, I think we should allow these humanoid robots to do the same. I hope you would agree with me here.

I agree with this -- and also agree with it for various non-humanoid AI systems.

However, I see this as less about rights for systems that may at some point exist, and more about our responsibilities as the creators of those systems.

Not entirely analogous, but: suppose we had a large creche of babies whom we had been told by an oracle would be extremely influential in the world. I think it would be appropriate for us to care more than normal about their upbringing (especially if for the sake of the example we assume that upbringing can meaningfully affect character).

Hmm, I think perhaps I have different takes on the basic mechanisms that make sense here?

Here's a scattershot of background takes: 

• It makes sense to first check for consensus
• People's sense of "need for an investigation" isn't binary
  • Lots of people may think "all else equal that would be nice to have" (as they think about many things), without it ever rising to the top of their internal importance-stack
• Probably people who were closer to things generally feel less need for investigation
  • (since they're more likely to think they understand the basic dynamics)
• If there isn't consensus on how important this is, I don't expect it to be easy to reach one
  • Since presumably one driver of different views is different people having access to different information (exactly the kind of thing an investigation might help with)
• In general things go best when they're done by people who feel the need for them

... and then given those, my position is that if you want it to happen, the right step is less like "try to create a consensus that it should happen" and more like "try to find/make an alliance of people who want it, and then make sure there's someone taking responsibility for the specific unblocking steps". (I guess this view is not very much about the investigation, and more like my generic take on how to make things happen.)

Honestly my view of how important it is that the whole project happen will also be somewhat mediated by whether it can find a decently strong lead and can attract some moderate amount of funding. Since these would be indicative of "people really want answers", and I think the whole project is more valuable if that demand exists.

It could be better for the world, and you might care about that.

It could be that you expect enough other people will talk to them that it's good for them to hear your side of the story too.

It could be that you expect it would be bad for your reputation to refuse to talk to them (or to give details which are non-concordant with the picture they're building from talking to other people).

I think that an eventual AI-driven ecosystem seems likely desirable. (Although possibly the natural conception of "agent" will be more like supersystems which include both humans and AI systems, at least for a period.)

But my alarm at nonviolent takeover persists, for a couple of reasons:

• A feeling that some AI-driven ecosystems may be preferable to others, and we should maybe take responsibility for which we're creating rather than just shrugging
• Some alarm that nonviolent takeover scenarios might still lead to catastrophic outcomes for humans
  • e.g. "after nonviolently taking over, AI systems decide what to do humans, this stub part of the ecosystem; they conclude that they're using too many physical resources, and it would be better to (via legitimate means!) reduce their rights and then cull their numbers, leaving a small population living in something resembling a nature reserve"
  • Perhaps my distaste at this outcome is born in part from loyalty to the human tribe? But I do think that some of it is born from more robust moral intuitions

Thanks, this felt clarifying (and an important general point).

I think I'm now at "Well I'd maybe rather share my information with an investigator who would take responsibility for working out what's worth sharing publicly and what's extraneous detail; but absent that, speaking seems preferable to not-speaking. So I'll wait a little to see whether the momentum in this thread turns into anything, but if it's looking like not I'll probably just share something."

Ideally, the questions this investigation would seek to answer would be laid out and published ahead of time.

Not sure I buy this, on principle -- surely the investigation should have remit to add questions as it goes if they're warranted by information it's turned up? Maybe if the questions for this purpose are more broad principles than specific factual ones it would make sense to me.

Would also be good to pre-publish the principles that would determine what information would be redacted or kept confidential from public communication around findings.

This checks out to me.

As things stand, my low-conviction take is that [headhunting for investigators] would be a reasonable thing for the new non-OP connected EV board members to take on, or perhaps the community health team.

Have you directly asked these people if they're interested (in the headhunting task)? It's sort of a lot to just put something like this on someone's plate (and it doesn't feel to me like a-thing-they've-implicitly-signed-up-for-by-taking-their-role). 

In general my instinct would be more like "work out who feels motivated to have an investigation happen, and then get one (or more) of them to take responsibility for the headhunting".

My read is that you can apply the framework two different ways:

• Say you're worried about any take-over-the-world actions, violent or not -- in which case this argument about the advantages of non-violent takeover is of scant comfort;
• Say you're only worried about violent take-over-the-world actions, in which case your argument fits into the framework under "non-takeover satisfaction": how good the AI feels about its best benign alternative action.

Happy to share some thoughts (and not thereby signalling that I plan not to say more about the object-level):

• Independent investigations are by their nature somewhat weird
  • You get someone coming in with less context, which makes it harder for them to discover things (relative to someone with more context)
  • But they also get to dodge group think or similar issues
• There are, as I see it, two different purposes of independent investigations:
  1. actually gaining insight into the situation
  2. being able to credibly signal that the conclusions are fair/independent/untainted by bias or groupthink
• There's a spectrum of different notions of "independent" which could be at play here:
  • Independent = largely but not completely unconnected with the FTX cluster
  • Independent = unconnected with the FTX cluster, but still in the EA sphere
  • Independent = unconnected with EA
  • The greater the independence, the higher the costs of the investigation, but if it's done well, the more robust the benefits
• Whether it's worth having an independent investigation, and of what kind, depends on:
  • The relative costs of different types of investigation
  • How much people might reasonably learn
  • How much pain there is from distrust that might be helpfully dispelled by an independent investigation
  • What risks, if any, are thereby created? (ammunition for media hit-pieces? chance of sparking vexatious lawsuits?)

In this case:

• Given the existence of the EV-commissioned investigation by Mintz (at significant expense), it seems somewhat weird to me that EV didn't publish more of a summary of the findings
  • I think there are lots of reasons they might not have wanted to publish the full investigation, and feel relatively sympathetic to their not having done that
  • I can imagine there are various risks-of-exposure from publishing even a summary, and they may have been advised by professionals whose job it is to monitor and guard against exposure (lawyers and/or PR folks) to play it safe
  • Nevertheless my guess is that if I were privy to the considerations, I would have thought that the better path involved sharing rather more with the EA community
• At this point I don't think it's likely to be worth another fully-independent investigation, as from a law firm
  • They're very expensive
  • Some of the most interesting questions will ultimately be judgement calls, which means that in order to derive value from it you have to have high trust in the judgement of the people performing the investigation
  • Some of the trust it would facilitate doesn't seem threatened (e.g. there doesn't seem to be any concern that there was a huge cover-up or anything)
• I do think it might well be worth an investigation by someone (or some few) in EA, but not connected to FTX
  • Partially because there seems to be a good amount of appetite for it from the EA community; partially because I think that's probably at the sweet spot of "people most likely to have useful critical takes about how to do things"
  • The principal challenge IMO is finding someone(s) who will:
    • Have good sensible takes on things
    • Be sufficiently non-consequentialist that their takes can be trusted to be "fair assessments" not "things they think will be most likely to lead to good outcomes"
    • Have minimal (if any) conflicts of interest
      • Ideally no connections to FTX
      • Also not beholden to anyone who might be reasonably criticised by an investigation (or whom outside observers might suspect of having that status)
    • Can credibly signal the above, so that their takes can be trusted by a broader audience
    • Be willing to spend time on it (and motivated to do a good job)
  • I think if there was someone who looked good for this, and it looked like a serious and legitimate attempt at an independent investigation, then it probably wouldn't be too challenging to get people to put in some money to pay for their time, and it wouldn't be too challenging to secure cooperation from enough people-that-they'd-like-to-interview
  • This is then kind of a headhunting task; but who would take responsibility for that?
    • It ideally shouldn't be the folks who have too much in the way of connections with FTX
      • Else the choice of person might be seen as suspect?
    • There's maybe a problem where a lot of the central community infrastructure does have some FTX connections, and other folks don't naturally read this as their responsibility?

This is great. It's kind of wild that it's attracting downvotes.

I think there's maybe a useful distinction to make between future-out-of-human-hands (what this post was about, where human incompetence no longer matters) and future-out-of-human-control (where humans can no longer in any meaningful sense choose what happens).