Anthropic urges AI labs to pause, warns humans risk losing control

Anthropic is proposing that the world’s top artificial intelligence companies come up with a coordinated way to pause development of advanced AI systems, warning that the technology is improving so quickly that there’s a risk humans would lose control.

The company behind the Claude chatbot said in a blog post on Thursday that, as cutting-edge AI gets increasingly faster at carrying out tasks, “it would be good for the world to have the option to slow or temporarily pause” its development.

Anthropic said its internal research institute plans to explore the issue in collaboration with others and “take actions” to help build the systems for a credible slowdown or pause, without being more specific.

Anthropic rival OpenAI argued for a different approach in a report published on Wednesday, saying that “democratic governments — not private companies acting alone — must ultimately determine the rules, safeguards, and accountability mechanisms”.

“Our view is that decisions about the pace of AI innovation should not be left to any one lab, company, or special interest group,” it said.

AI models are getting faster, with rapid increases in how quickly they can carry out software tasks like coding on their own, Anthropic said in its post. Based on current trends and given enough computing power, an AI system could be able to design and develop its own successor, in what is known as “recursive self-improvement”.

Self-building AI would be a major technological milestone that would bring benefits in science, healthcare and other areas, Anthropic said, but it “also might increase the risks of humans losing control over AI systems”.

Some tech industry figures have long warned of such a scenario.