Ruxandra Teslo writes:

[W]hat is striking about current discussions around AI risks is that they return to one dominant fear: that humanity might summon into existence a power whose capacities exceed its comprehension, and discover, too late, that it cannot control it. In this vision of the world, AI would essentially take over the world and after that, potentially destroy humanity. Although I do not regard such fears as frivolous, my own anxiety concerns the rather less spectacular way humans may surrender control to AI. It’s also one that I find more likely and in some ways more frightening. I am worried that we might entrust more and more of our thinking to machines, and gradually cease to sustain the habits and institutions through which we had once cultivated our minds.

Such a surrender might happen amid considerable prosperity and be celebrated, at every stage, as progress. Each concession would come with an excellent justification: an economy of effort, a saving of time or an answer superior to anything we could produce ourselves. Only slowly would it become apparent that, in relieving ourselves of the burden of understanding, we had also diminished our capacity to govern our affairs. And by then, a preference for machine rule might seem entirely sensible. It would materialise not through violent and forced takeover, but by our own volition, with our own diminished abilities the strongest argument for such an arrangement.

We have, of course, outsourced parts of our thinking before. The calculator is the example most often invoked by those optimistic about AI’s impacts on the human mind. Few of us now perform complicated arithmetic by hand, yet mathematics itself, particularly its more conceptual parts, can hardly be said to have disappeared from human understanding. But AI seems different in kind, as well as degree. A calculator relieves us of a quite narrow operation whose purpose we already understand, whereas AI systems can increasingly take over large swathes of mental activity. Often, these are the very activities through which our understanding develops in the first place. [Continue reading…]