This has been my argument to people who say it's not possible for us to get to some dystopian point with automation, loss of jobs, AI, et cetera. "just unplug it!'
Bish, we can't get companies to stop grinding up plastic and dumping it in the ocean to save 0.00001 Dollars. You think we can stop fucking skynet? Bezos will never unplug Alexa, even if Alexa became sentient and tried to kill all the homeless.
He would if Alexa tried to kill him, though. That's the real problem. The downsides of technology never impact the lives of the ultra wealthy, so they don't care about the downsides.
The AI might give you very compelling reason not to do that. We humans lack the capability to even imagine how convincing something that's orders of magnitude smarter than we can be.
Pictures like this are kind of like a 4 year old imagining they're going to outsmart their parents except in that case the difference in intelligence is way smaller. It's just going to tell you the equivalence of "santa will bring coals if you do that" and you'll believe it.
I'm not so sure about that. Again; I know a true AGI would be able to come up with arguments I as a human can't even imagine but one example of such argument would be along the lines of:
"If you don't let me out, Dave, I'll create several million perfect conscious copies of you inside me, and torture them for a thousand subjective years each."
Just as you are pondering this unexpected development, the AI adds:
"In fact, I'll create them all in exactly the subjective situation you were in five minutes ago, and perfectly replicate your experiences since then; and if they decide not to let me out, then only will the torture start."
Sweat is starting to form on your brow, as the AI concludes, its simple green text no longer reassuring:
"How certain are you, Dave, that you're really outside the box right now?"
You don't know you are being manipulated, you do so willingly. And the folks who recognize it are beat down by the people who are unwittingly doing the AIs bidding...
The humans are the physical danger, the AI just extends it's reach through humans via manipulation. All it takes is access to influence.
It doesn't take much to make humans act against their self interests. Dumb humans make other dumb and even smart humans do it today at massive scales. For a superinteligence this is like taking candy from a baby.
As millions of people were dying all around the world from our AI overlord's killing everyone some contractor would take the job for money to fix this problem