Our goal is to solve the core technical challenges of superintelligence alignment in four years.

This is a great goal! I don’t believe they’ve got what it takes to achieve it, though. Safely directing a superintelligent system at solving alignment is an alignment-complete problem. Building a human-level system that does alignment research safely on the first try is possible, running more than one copy of this system at a superhuman speed safely is something no one has any idea how to even approach, and unless this insanity is stopped so we have many more than four years to solve alignment, we’re all dead

I don't want to engage with your arguments. I strongly think you're wrong, but it seems much less relevant to what I can contribute (or generally want to engage with) than the fact that you've posted that comment and people have upvoted it.

I don't understand how this can happen on the EA Forum. Why would anyone believing in this and wanting to do good promote this?

If anyone here does believe in ideas that have caused a great amount of harm and will cause more if spread, they should not spread them. If that's not the specific arguments that you think might be better and should be improved in such and such way but the views that you're arguing about, don't! If you want to do good, why would you ever, in our world, spread these views? If the impact of spreading these views is more tragedies happening, more suffering, and more people dying early, please consider these views an infohazard and don't even talk about them unless you're absolutely sure your views are not going to spread to people who'll become more intolerant- or more violent.

If you, as a rationalist, came up with a Basilisk that you thought actually works, thinking that it's the truth that it works should be a really strong reason not to post it or talk about it, ever.

The feeling of successfully persuading people (or even just engaging in interesting arguments), as good as it might be, isn't worth a single tragedy that will result from spreading this kind of ideas. Please think about the impact of your words. If people persuaded by what you say might do harm, don't.

One day, if the kindest of rationalists do solve alignment and enough time passes for humanity to become educated and caring, the AI will tell us what the truth is without a chance of it doing any harm. If you're right, you'll be able to say, "I was right all along, and all these woke people were not, and my epistemology was awesome". Before then, please, if anyone might believe you, don't tell them what you consider to be the truth.

Oops! Sorry, I only discovered the second link; but before writing my comment, I looked up the first myself.

I’m not a biologist and will probably defer to any biologist entering this thread and commenting on the twin studies.

Twins (mostly, as the linked study shows) do not have exactly the same DNA. But it doesn’t seem to be relevant. The relevant assumption is that there’s almost no difference between the DNAs of “identical” twins and a large difference between the DNAs of non-identical reared-together twins, which is true despite a couple of random mutations per 6 billion letters.

The next two linked articles are paywalled. Is there somewhere to read them?

The third is a review of a short book, available after a sign-up, and it says that “some studies on twins are good, some bad”, and the author feels, but “doesn’t actually know” that the reviewed one is good. The reviewed book performed a study on twins and noticed there isn’t much of a difference between the correlation of the similarity of many personality traits with whether people are identical twins, and concluded that, since you’d expect to see a difference if the traits have different degree of heritability, many personality traits are results of the environment.

How is this an evidence that twin studies are flawed and shouldn’t be used? If that’s a correct study, it’s just evidence that personality traits are mostly formed by environment (which is something I already believe and have believed for the most of my life), but, e.g., why would this be relevant for a discussion of whether or not some disease has a genetic component to it, when a twin study shows that there is?

It’s important to carefully compare the numbers; but obviously there are things that identical twins have in common more often then non-identical twins, because these things are heritable at to larger or lesser degree; like hair color or height.

Of course, any study makes some underrepresentation of humanity. But if your study is about the degree of heredity of something and not about twins, why would this matter? If there’s a difference between adopted identical and non-identical twins that’s better explained by genetics (e.g., non-identical twins would have a different height more often), why does it matter how well they represent twins in general? Unless you’re studying how likely people are to be adopted, I don’t understand the claim.

The last link is paywalled, but again, why would this affect the difference between identical twins and non-identical twins? Until a year ago, I kept secret that I’m bi and would’ve kept it secret from scientists; but I don’t think this kind of thing affects conclusions you’d make if identical twins answered identically to some question more often than non-identical twins (e.g., imagine a society where people with green eyes are persecuted and a lot of them use contact lenses. Some would still say the truth, in confidence, to scientists; and the number of identical twins telling the same answer would be greater than the amount of non-identical twins telling the same answer, and the scientists will correctly infer this to be evidence for the heritability of eye color, even though a lot of twins would lie about their eye color).

So while it’s possible to just compare full DNAs and account for lots of different factors (all sorts of various environmental conditions that might be different between the subjects of the study) to find out whether DNA correlates with eye color, it’s much easier to do a twin study, and a strong correlation there will be a strong evidence

I don’t think one of the claims, that “Twin studies are flawed in methodology. Twins, even identical twins, simply do not have exactly the same DNA”, is true. As I see, it is not supported by the link and the study.

The difference of 5.2 out of 6 billion letters that identical twins have on average is not something that makes their DNA distinct enough to make the correlations between being identical tweens or not and having something in common more often to be automatically invalid.

One of the people involved in the study is cited: “Such genomic differences between identical twins are still very rare. I doubt these differences will have appreciable contribution to phenotypic [or observable] differences in twin studies.”

Twin studies being something we should be able to rely on seems like a part of the current scientific view, and some EA decisions might take such studies into consideration.

I think it’s important not to compromise our intellectual integrity even when we debunk foundations for awful and obviously wrong beliefs that are responsible for so much unfairness and suffering that exist in our world and for so many deaths.

I think if the community uses words that are persuasive but don’t contain actually good evidence, then even if we’re arguing for the truth that’s important and impactful to spread, in the long-term, this might lead to people putting less trust in any of our words arguing for the truth and more people believing something harmful and untrue. And on the internet, there are a lot of words containing bad arguments for the truth because it’s easy for people to be in the mode of finding persuasive arguments, which don’t necessarily have to be actually good evidence.

I think it’s really important for the EA community to be epistemically honest and talk about the actual reasons we have for believing something, instead of trying to find the most persuasive list of reasons for believing in what we believe in and just copying it without verifying that all the reasons are good and should update people in the claimed direction.

Thanks for the lots of ideas!

The target audience is unlikely to have issues with the books being about Harry Potter; most of them should've seen the movies or read the books^[1]. The age might be important because of, e.g., willingness to read weird fanfiction, but I don't think it's important because of the prior familiarity with the original Harry Potter.

We're also targeting winners of national and international olympiads in economics, biology, physics, and chemistry. But I don't think we'll have any localized cause-specific materials except for AI Alignment.

It's a good idea to target people who might have influence in the future. Thanks! From the top of my head, I can't recall any reproducible routes currently existing in Russia, but I'll think about that more. I don't expect a change in a political situation that would result in much more predictable routes^[2].

No university EA groups I'm aware of exist in Russia; the city groups had these books when they still existed, but now most Russian EAs communicate online  (most of the group organizers have left the country).

Not sure Harry Potter conventions exist really, and a booth doesn't seem impactful compared to the alternatives (e.g., giving the books to all math/computer science/biology/etc. students who want them). Also, it doesn't require many physical copies.

1. ^{^}

   I think not being familiar with the original Harry Potter doesn't reduce the impact of reading HPMOR that much. But everyone in Russia should be generally familiar with the original plot, I think. Like, for many years, a couple of TV stations showed film marathons every New Year.

2. ^{^}

   A fun fact, Navalny's brother has read HPMOR while in prison, and Navalny’s org, which is probably the leading Russian opposition structure, has helped us with auditing the crowdfunding campaign to ensure it's non-commercial. 

I kind of assumed that people who can answer the question are people already familiar with HPMOR, but I think you’re actually right, thanks!

Thanks for the comment!

We’re already doing public libraries who fill out a form; donating to random libraries doesn’t seem like a good idea- not that many people will read them if HPMOR can be requested but librarians don’t actively recommended it, and it’s not obvious it’s worth even the shipping cost, there are probably better ways to spend the books.

There are some issues with school libraries, but yeah, doing those as well

We're making translations of a lot of EA content into Russian and have an experience that might be relevant to countries where people mostly can't speak English.

We learned that you need introspective people to evaluate or do the translations.

The best professional translators in our language are mostly hired by large publishing houses and have long-term commitments for many books to come (and the publishing houses aren't able to lease them for our projects). 

Surprisingly, looking at a translation that looks like a good text in our language but has significant mistakes, most people wouldn't notice the mistakes. Most people who aren't the best professional translators don't actively try to recognize what exactly they just read. When a translation expresses something really different from what the original text conveys, but the words are similar enough, people just don't notice it.

Google Translate was better at not making mistakes than 80% of translators that sent us a translated test segment. We ended up hiring two translators from the EA/LW community.

A lot of the 80,000 Hours' articles are highly optimized for conveying a correct understanding, and translation errors might significantly reduce the value of the texts.

It is quite likely that you're right! I think it's just something that should be explicitly thought about, it seems like an uncertainty that wasn't really noticed. If x-risk is in the next few decades, some of the money currently directed to the interventions fighting deaths and suffering might be allocated to charities that do it better.

Awesome!

I didn't consider the spending speed here. It highlights another important part of the analysis one should make when considering neartermist donations conditional on the short timelines. Dependent on humanity solving alignment, you not only want to spend the money before a superintelligence appears but also might maximize the impact by, e.g., delaying deaths until then