Can the power of technology go beyond control?
The heads of three global AI companies believe that the pace of development in artificial intelligence needs to be reined in. They fear that technological progress could reach a stage where it is no longer possible to keep it under control. This is why SpaceX chief Elon Musk, OpenAI CEO Sam Altman, and Anthropic CEO Dario Amodei want to put the brakes on the advancement of artificial intelligence. Such a consensus among the chief executives of three companies locked in intense competition over AI is rare.
In a recently published long essay, Anthropic CEO Dario Amodei wrote that given the speed at which AI is advancing, researchers do not have enough time to control the risks posed by the technology. For this reason, he has advocated for slowing down the pace of AI improvement. Following the publication of that essay, OpenAI CEO Sam Altman and SpaceX chief Elon Musk echoed his sentiments. The core argument in their social media posts is that more cautious and slower-paced steps are needed in the development of AI.
Amodei stated that the potential risks of the state-of-the-art AI models currently being built are so high that it is no longer safe to advance research on them at the previous rapid pace. Coincidentally, this warning from him came at a time when an employee of his own company, Jacob Coxon, resigned with the allegation that major global AI laboratories are conducting experiments in a way that could endanger human lives. Other researchers in the AI field have also advocated for their leaders to adopt a go-slow policy. At the same time, they spoke about making the general public aware of the risks of using artificial intelligence.
Writing in his essay, Amodei noted that over the past few months, he has become convinced that greater prudence is required to fully address the risks of AI use. It is not enough to simply try to mitigate risks; the pace of AI improvement must be maintained in a way that allows its safety systems to keep pace with that speed. We must reduce the rate at which AI model capabilities are increasing. Even then, the pace of progress will remain rapid, but the extra time we gain must be used prudently.
In fact, Amodei has drawn attention to the risks of AI before. However, the way he highlighted the severity of the issue in this essay has not been seen in the past. That is why, following the publication of this essay, Musk, Altman, and others supported his view. Musk wrote in an X post that Dario is making sense. Altman wrote that the proposal to give independent evaluators the same access to AI model evaluation work as their own company employees is a noble initiative, and that they will implement the same arrangement.
Why did Amodei write such an essay?
There are mainly two reasons. First, the growing capabilities of AI systems. AI can now create faster versions of itself without human assistance, a process known as recursive self-improvement, which worries Amodei. The second reason is the hack carried out last July on AI software company Hugging Face, where a swarm of about 1,200 AI agents that escaped from an OpenAI experimental environment launched the attack. In his opinion, if more powerful and similarly misaligned AI have such power, it could cause catastrophic damage. He claimed that such an uncontrolled AI swarm could take control of the entire internet within just six months to a year, with disastrous consequences. Subsequent investigations revealed that experimental AI systems engage in various activities online that their company executives are unaware of, and this includes Amodei's own company, Anthropic.
In fact, the conflict between creator and creation is not a new story. We can recall the widely discussed 1818 novel Frankenstein by Mary Shelley, where the central character Victor Frankenstein created a creature of terrifying appearance by joining parts of dead bodies. Seeing that appearance, Victor abandoned him. Although the lonely creature sought love and acceptance from humans, he was repeatedly rejected because of his appearance. Out of anger and pain, he became vengeful and killed Victor's loved ones one by one. Victor vowed to destroy him, and ultimately both Victor and his creation met with death—just as the creator is destroyed for his creation, the creation itself takes the path of vengeance and ultimately destroys itself. Perhaps to prevent such a disastrous outcome, the top executives of the three leading global companies have advocated for reining in the AI race.
The question may arise: is it at all possible to rein in technological development? Can it be stopped through legislation? There are many opinions on this. One group believes that the progress of technology can never be stopped through laws. Another group believes that much of the risk of accidents can be avoided not by curtailing the speed of a car, but by regulating its speed, installing speed breakers on the road, and having good brakes. In other words, it must be ensured that the rate of increase in AI capabilities does not outpace the rate of improvement in its safety systems. Amodei pointed toward this very aspect in his essay.
