Can we get the benefits of AGI when they control us benevolently, keeping us from making mistakes while guiding us to a life on exoplanets and a massive population increase, a techno-utopia? Is that a worthwhile solution to the alignment problem?
Can we get the benefits of AGI when they control us benevolently, keeping us from making mistakes while guiding us to a life on exoplanets and a massive population increase, a techno-utopia? Is that a worthwhile solution to the alignment problem?
I'm quite skeptical of the concept of an ASI that is interacting with us but not controlling us. If it can predict the impact of each of its actions, it basically chooses the future of our species. So either we become an ASI (cognitive enhancement, mind uploading,...) or we have an ASI that controls us, and then the goal is that it controls us benevolently. But putting that aside, my answer is yes, seems good by definition of 'benevolently'.
Appeals to me - if control is meant as caring recommendations which really make it that everyone benefits.
For example, AGI that would offer me menus that optimize for my liking as well as nutritional benefits and ethics and explain to me these two aspects in a way which I enjoy, offer me various healthy exercises that make me happy, give me some social skills development practice which would not hurt anyone by mistakes that would improve my relationships, function as a psychologist so that I am better able to interpret events positively, and of course let me develop my interests while encouraging any of those which are also beneficial to others.
All in the framework where I know that no one is disadvantaged or dependent on a human. If I'd feel quite good, then I'd know many others are also feeling as good, enjoying deeply positive relationships, independence, health, and meaning in whichever interests they pursue.
I would know that I can always opt out of the recommendations. Critical thinking should be encouraged. This should 'update' the AGI. For example, if vegan menus are offered as the possibly/with current understanding most ethical, but critical thinkers come up with a non-vegan option which is even better for animals, then the AI should start recommending these options too, with a similarly 'uncertain' disclaimer. Or, individuals should be encouraged to try various social interactions and add an 'emphasis' to the social skills training program or change some of the fundamental lessons.
This only concerns wellbeing training and recommendation algorithms. If global safety is ensured by AGI, that would be great too. If 'meaningful exploration,' such as of the nature and objective of the universe, is also covered by AGI, then it is possible that I would see no reason to exist, even if my life would be very happy. Depending on if there is a reason for the existence of humans (such as to supply emotional intelligence) and if there is a sufficient number of humans who choose to exist given the option, then the choice to exist or not should also be included.
AGI optimizing nutrition: Ok, first on the menu is mind uploading nanobots.
Human: Err.
AI: You want me to optimize nutrition right? In other words, you want whichever configuration of atoms will give you the longest healthiest life.
Human: How many calories are in that?
AI: Billions, its powered by antimatter.
I came across a book a couple decades ago about expert systems design applied to recipe programs, maybe there's some crossover here. An expert system is in no danger of developing a consciousness, all other things equal, but it could serve as an excellent recommendation system for some well-defined activity or product.
I suppose the more an AGI got into my life, or ahead of my life, the more the meaning of my life could be challenged.
I'm not sure I can think of a single example from history or nature where a more advanced species/culture had power over a less advanced/adapted one, and where that ended up well for the underling.
An actual utopia sounds pretty good to me, but I don't think this vision is a solution to the alignment problem. It is something we might want an AGI to do for humanity, but we don't know how to ensure that an AGI does what we want.
Yeah, but I actually want to live in a personal utopia, lol.
Seriously, though, if the AGI really controlled us, it could decide what we wanted, and control us to have those wants. Then it would do whatever we want. To make it realistic and allow us a more accurate and useful memory of its behavior, it could lead us through trouble and struggle to develop those wants it decides we should have. While we "resisted", "fought", and "learned', it would guide us however it saw fit, running system I, our subconscious minds, for us.
If such control is possible, an AGI is likely to find all the shortcuts to it on its path to doing what we want it to for humanity.
Well if you have it, I'll take it. In the general scenario, a very powerful benevolent AI is left to do whatever it thinks is best. If the AI decides that freedom is one of humans top values, it will try to make the world better while optimizing human freedom. Giving humans more freedom in practice than the typical government is not a particularly high bar. Of course, plenty of people might want the AI micromanaging every detail of their life, the AI will do a really good job of it. But I would think ideally freedom should be there for those who want it.
Its also worth noting there is a fairly common belief that we are on a path to probable doom, and any AI that offered anything better than paperclips is work taking. So, even if your AI was much too controlling and humans would prefer a less controlling one, many EA would say " best AI we are going to get".
You know, as far as seeing ourselves on a path to doom, I don't see why development of a superintelligent rogue AI isn't treated like development of a superweapon.
If some Silicon Valley company were developing the next battlefield nuke, they'd either be a Pentagon contractor or get raided by the FBI.
But here they are making something with ability to quickly learn how to enter and take control of all computer systems, and possibly electromechanical systems, everywhere, as well as us, actually, and we've got charitable organizations worrying about getting someone into AI companies to get them thinking about safety a little.
It's not well understood how to make an AGI safe, so obviously developing them should be taboo, if you care about existential risk.
An effort similar to what scientists do about nukes seems intuitive, keeping the doomsday clock, trying to stop nuclear proliferation, encouraging disarmament, etc.
The potential danger is easy to see. It never occurred to me before coming across EA discussions that the development of conscious AI would be a reason for anything but terror and panic. That's why I asked my questions, actually.
I guess I have one last question for the forum on this topic.
You know, as far as seeing ourselves on a path to doom, I don't see why development of a superintelligent rogue AI isn't treated like development of a superweapon.
Because distinguishing it from benign, not superintelligent AI is really hard.
So you are the FBI, you have a big computer running some code. You can't tell if its a rouge superintelligence or the next DALL-E by looking at the outputs. A rouge superintelligence will trick you until its too late. Once its run at all on a computer that isn't in a sandboxed bunker its probably too late. So you have to notice people writing code, and read that code before its run. There are many smart people writing code all the time. That code is often illegible spaghetti. Maybe the person writing the code will know, or at least suspect, that it might be a rouge superintelligence. Maybe not.
Lots of computer scientists are in practice rushing to develop self driving cars, the next GPT. All sorts of AI services. The economic incentive is strong.
In Superintelligence, Bostrom provided plausible scenarios showing how a superintelligent being could free itself from our control. People usually expect death as a result. To me it is more intuitive that the result is torture of people, meaning nothing more than creating additional suffering in people's lives.
I am also skeptical of a superintelligent being doing anything but immediately beginning to control people, one way or another, for one reason or another.