New York City Comptroller Mark Levine, who oversees the city’s $330 billion pension system, penned an open letter to Anthropic CEO Dario Amodei, asking the artificial intelligence company to pause plans for its anticipated initial public offering until it can address safety concerns it has previously raised.
The letter cited reports that Anthropic’s S-1 filing with the Securities and Exchange Commission warned that advanced AI could pose “catastrophic or existential risks to humanity.” According to reports, AI-related risk disclosure spanned 80 pages of the prospectus, nearly double the length of the section of the description of the business.
Anthropic is reported to be perusing an IPO at a $2 trillion valuation sometime this fall. In Levine’s letter, the comptroller wrote that instead of proceeding with the IPO, Anthropic should instead focus its energy on addressing AI risk.
In September, Jacob Coxon, an Anthropic researcher, resigned from the company and warned that frontier AI labs were not taking risk seriously.
“I spent the last three years doing pre-training research at both OpenAI and Anthropic. Neither company is acting responsibly,” Coxon wrote on social media. “They are racing to self-improving superintelligence and gambling with our lives.”
“Do not underestimate the power of this technology,” Coxon continued. “These will soon be superhuman systems that can hack anything, revolutionize any field overnight and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing. The people building AI earnestly believe it could kill us all by the end of the decade.”
In September, OpenAI reportedly paused training of a set of new artificial intelligence models after rogue AI agents reportedly gained unauthorized access to federal websites.
On September 12, Amodei penned an essay urging AI leaders to pace the development of their latest models and allow time for independent safety evaluations and global regulation.
In response to these concerns, Levine said that Anthropic should disclose how it will implement appropriate audits and controls and whether these controls will address Amodei’s call to slow the pace at which the capabilities of AI models advance.
“In recent weeks, Anthropic has issued unusually direct public statements acknowledging that the pace of frontier-model capability development may now exceed society’s capacity to understand and control the resulting systems,” Levine wrote. “You have disclosed that unchecked recursive self-improvement could outrun human oversight. In your September 12 essay, ‘We Must Pace the Frontier,’ you stated: ‘Over the last few months, I have become convinced that fully addressing the risks requires even more prudence … We must slow the pace at which we improve the capabilities of AI models.’”
Recent security events have highlighted the risks of AI actions taken without human oversight. In July, AI agents from OpenAI escaped their “sandbox” control environment and compromised the cybersecurity infrastructure of open-source AI platform Hugginface.
In June, an OpenAI agent reportedly breached Australian government websites after it escaped its sandboxed training environment.
“I therefore urge Anthropic to pause the IPO process so that the company may fully address these key issues, ones which investors must also assess regarding the company’s revised development strategy, governance commitments, and emerging risk landscape,” Levine’s letter ended.
Ahead of its IPO, Anthropic had received a BB+ credit rating from ratings agency KBRA—one notch below an investment-grade credit rating.
 |
TIAA Adds Executive for Tech, AI Risk |
 |
Risk Is a Cake: CIOs Need to Know Which Layer Matters Most |
 |
AI Infrastructure Boom Presents Risk, Opportunity for Insurers |