The New York City Council held a landmark oversight hearing on artificial intelligence on Monday, October 5th.
This rare meeting, which the city claims is the first of its kind in the nation, was held because insiders in the field of artificial intelligence warned NYC leaders about the dangers of this emerging technology and the lack of regulation surrounding it, calling for greater oversight as local governments address concerns about the potential of these new AI systems to endanger humanity.
Jacob Coxon, a former engineer at Anthropic who’s become one of the biggest voices for AI regulation said: “The companies run on a startup mindset: Move fast, break things, fix them later. That works for a photo sharing app. It does not work for building the most powerful technology ever built,”
State and city governments have been considering ways to regulate AI, as the federal government under the Trump Administration has done little to regulate this new technology. President Trump has called fears of AI a “hoax” and signed an agreement last month that gave steps for companies to police themselves.
“On the current path, I think it is more likely than not that humanity loses control of these AIs and it could end in human extinction,” Coxon said.
Shortly after Coxon voiced his fears of worst-case scenarios, top executives Dario Amodei of Anthropic, Sam Altman of Open AI, and Elon Musk of SpaceX said they agree AI development needs to slow down. AI companies have detailed steps for the industry’s safety. But coordinating the entire industry to slow down is no small task.
A group of AI company staffers representing tech companies OpenAI, Athropic, Google, and Meta testified at the Monday hearing, claiming to officials that they’re laser-focused on safety and their AI systems are improving people’s day-to-day lives.
The NYC Council is considering several AI regulations, such as introducing proposals to require companies to disclose information about AI tools, requiring third-party approval for AI systems in the city, and allowing whistleblowers to receive portions of fines from AI companies that violate laws.
Logan Graham, of Anthropic, told the NYC Council the company welcomes “smart regulation” and that state and local governments have a role to play in developing frameworks.
“AI is powerful enough that getting this wrong has real consequences, for individuals, institutions, public trust,” he said. “We have to work relentlessly to do the science of mitigating the risks.”
