The idea of whether machines will dominate us is generally treated as a new one in current discussions on AI risk, since in 1863 Samuel Butler published a letter titled 'Darwin Among the Machines' which made an argument closely resembling the one in today's debate.
He stated that the real danger was not that machines would gain control of power, but that people would find machines so useful that it would become practically impossible to get rid of them, so that by that stage the question of who was actually in charge would have already been answered — no matter who seemed to be in control. He said that a society which, in principle, could not destroy its machines had already "created a race of beings towards whom it is beyond our power to destroy" and had thus become "acquiescent in our bondage."
This is a quite different claim from the one that says 'superintelligence takes over'; it's similar to the sorts of arguments currently being made regarding algorithmic dependence, attention economy lock-in, and infrastructure that is too deeply embedded to be undone — arguments that can be applied to any technology that is sufficiently indispensable, not just to intelligent machines.
The full letter, together with the historical background (it is also the immediate source of the "Butlerian Jihad" in Dune, a fact that most people are unaware of) can be found here: https://romaricjannel.substack.com/p/the-rise-of-mechanical-life
I'm wondering if people here consider "acquiescence through indispensability" to be a real and separate risk category or whether it's just a rewording of dependence anxiety that appears each time a major technology is introduced.
[link] [comments]