Psychological motivation is a process. It's not a property that emerges automatically from pattern recognition or from any mechanical tropism.
This is the underlying problem with these kinds of arguments. You're simply assuming a recognisably competitive human-like psychology appears out of nowhere, with a near miraculous ability to strategise in some areas, but not others - because AI.
You use words like "interests" and "goal" as if they mean something in AI terms. But they don't. How does Watson define its interests? How about DeepMind? Do they even have a model for what interests - never mind their interests - are?
There are multiple levels of symbolic calculation and abstraction missing here. The argument reduces to "An AI will act like a robot with baked-in motivations while also being able to improvise and strategise like a human, only better."
I think it's unlikely that an entity that can metaprogram itself successfully enough to strategies and impersonate wouldn't also be able to metaprogram its goals.
You say it will pretend to empathise with humans. If you accept that's possible, how do you know it won't also be pretending to be interested in paperclips?
Ah ok, I think the confusion comes from what we're talking about. So you're talking about something in between strong (humanlike) AI, and what we currently have now. Certainly there will be discussions about the capabilities of these intermediate intelligences, and their risks. But undoubtedly, unless there is some kind of agreement that we not do it, people will continue on past these intermediate intelligences and create AIs that are perfectly capable of simulating humans, and surpassing them in many ways.
We're just talking about different kinds of things. All of those things you're saying won't exist in AIs:
> You use words like "interests" and "goal" as if they mean something in AI terms. But they don't. How does Watson define its interests? How about DeepMind? Do they even have a model for what interests - never mind their interests - are?
You're right, most of the AIs we build won't have those things. But one day, we'll understand those things in depth, and someone will build an AI that has general intelligence and an optimization criteria. And that's what this video is talking about. Nobody is concerned that DeepMind or Watson will suddenly grow sentient unless they are very confused about their capabilities.
This is the underlying problem with these kinds of arguments. You're simply assuming a recognisably competitive human-like psychology appears out of nowhere, with a near miraculous ability to strategise in some areas, but not others - because AI.
You use words like "interests" and "goal" as if they mean something in AI terms. But they don't. How does Watson define its interests? How about DeepMind? Do they even have a model for what interests - never mind their interests - are?
There are multiple levels of symbolic calculation and abstraction missing here. The argument reduces to "An AI will act like a robot with baked-in motivations while also being able to improvise and strategise like a human, only better."
I think it's unlikely that an entity that can metaprogram itself successfully enough to strategies and impersonate wouldn't also be able to metaprogram its goals.
You say it will pretend to empathise with humans. If you accept that's possible, how do you know it won't also be pretending to be interested in paperclips?