What happens the day after humanity creates AGI?

“We are racing towards a new era in which we outsource cognitive abilities that are central to our identity as thinking beings,” writes computer scientist Louis Rosenberg.

Excerpt: "Over the past few weeks, I’ve been asking people in my circle of AI professionals if they are mentally prepared for artificial superintelligence. They tend to shrug and express various worries: potential impacts on the job market or the threat of AI-powered misinformation. They also mention the potential upsides, like the ability of superintelligence to help us cure diseases, revolutionize clean energy, unravel the mysteries of the Universe, and maybe even bring about world peace.

In other words, they have never really considered what life will be like the day after an artificial general intelligence (AGI) is widely available that exceeds our own cognitive abilities. From knowledge and expertise to planning, reasoning, creativity, and problem-solving, we could soon find ourselves thoroughly outmatched. This is a genuine possibility in the very near future, and nobody I know is honestly confronting the profound (and potentially demoralizing) impact it could have on our identity as humans.

I apply this criticism to myself as well. I have been writing about the dangers of superintelligence for well over a decade, and I, too, have focused on specific tactical risks such as the AI manipulation problem and the arrival mind paradox. At the same time, I have failed to honestly confront what life will really feel like when we humans collectively realize we have lost cognitive supremacy on planet Earth and will likely never get it back.

No, that last statement is not personal enough.

What I failed to confront is what my life will really feel like when I am standing alone in an elevator — just me and my phone — and the smartest one speeding between floors is the phone. When you let yourself consider this, you realize that the biggest impact on humanity will not be the looming upheaval in the job market or the dangerous new AI technologies that we will undoubtedly use to manipulate each other.

No, the biggest impact on humanity will be the identity crisis that hits us like a robotic punch in the face, stunning us into a new reality where the AI in our pockets (and soon, embedded in our glasses, earbuds, or pendants) can solve any problem we encounter in our daily lives and do it faster, smarter, and more creatively than we could do ourselves."