Summary

Fast technological development carries a risk of creating extremely powerful tools, especially AI, before society has a chance to figure out how best to use those tools in positive ways for many value systems. Suffering reducers may want to help mitigate the arms race for AI so that AI developers take fewer risks and have more time to plan for how to avert suffering that may result from the AI's computations. The AI-focused work of the Machine Intelligence Research Institute (MIRI) seems to be one important way to tackle this issue. I suggest some other, broader approaches, like advancing philosophical sophistication, cosmopolitan perspective, and social institutions for cooperation.

As a general heuristic, it seems like advancing technology may be net negative, though there are plenty of exceptions depending on the specific technology in question. Probably advancing social science is generally net positive. Humanities and pure natural sciences can also be positive but probably less per unit of effort than social sciences, which come logically prior to everything else. We need a more peaceful, democratic, and enlightened world before we play with fire that could cause potentially permanent harm to the rest of humanity's future.

Contents

(available at the link at the time of cross-posting)

1. Introduction
2. Encouraging more reflection
3. Ideas for improving reflectiveness
   1. Liberal-arts education
   2. Big-picture, cosmopolitan thinking
   3. Effective altruism
   4. Improved public-policy epistemology??
4. Are these meta things cost-effective?
5. Idealism meets competitive constraints
6. Areas where the sign is unclear
   1. Faster technology
   2. Education
   3. Cognitive enhancement
   4. Transhumanism
   5. Economic growth
7. There are many exceptions
8. Technologies that are probably bad to accelerate
   1. Computer hardware
   2. Artificial consciousness
9. Caveats: When are changes actually positive-sum?

Note from Tessa: Because of the ongoing Decade Review I am re-posting some classic posts under the review crosspost tag. I have not crossposted the complete text, as the author of this post does not like crossposting since it makes updating the content of posts more difficult.