Some of the world's brightest individuals and most capable organizations are rushing to build a technology that can exceed human performance at accomplishing long-term plans that require self-preservation. However, they aren't working on how to make these robots not care about self-preservation if the AI comes in conflict with humans. Thus, if nothing is done about it, we can very well end up in a situation where we have AI that is smarter than us that does not want us to turn it off. It is impossible to have safe control in this situation.
Some of the world's brightest individuals and most capable organizations are rushing to build a technology that can exceed human performance at accomplishing long-term plans that require self-preservation. However, they aren't working on how to make these robots not care about self-preservation if the AI comes in conflict with humans. Thus, if nothing is done about it, we can very well end up in a situation where we have AI that is smarter than us that does not want us to turn it off. It is impossible to have safe control in this situation.