A super-intelligence would be incomprehensible to humans. They cannot therefore control it.

A number-crunching study conducted back in 2021 suggests that it will be extremely challenging to control a super-intelligent artificial intelligence (AI). The researchers argued that if we cannot comprehend the capabilities of a super-intelligence, it becomes impossible to create a simulation to analyze and control it.

Unlike conventional robot ethics, dealing with a super-intelligence is fundamentally different due to its multifaceted nature and potential to achieve objectives incomprehensible to humans. The researchers draw on Alan Turing’s halting problem, which demonstrates that it is logically impossible to determine whether a computer program will halt or loop indefinitely.

Applying this concept to a super-intelligent AI, it becomes evident that containing its actions becomes mathematically unachievable as any “containment” program would be unusable.

The study suggests that traditional methods of teaching AI ethics or limiting its capabilities are insufficient and raises questions about the purpose of creating such advanced systems.

The researchers propose that as we forge ahead with AI development, we need to address the possibility that a super-intelligence may emerge beyond our control, and we may not even be able to recognize it due to its incomprehensibility.

This realization has led tech giants, including Elon Musk and Steve Wozniak, to advocate for a temporary halt in AI research to ensure its safety.

The study concludes that before developing powerful AI systems, we must have confidence in their positive effects and manageable risks.

However, as the research was published in 2021 and subsequently updated to reflect advancements in AI in 2023, it is important also to consider the latest developments in this rapidly evolving field.

Related Story:  Tech Luminaries Issue Letter Warning Of Extinction Risk From AI

Image by Gerd Altmann from Pixabay