Former Anthropic researcher Jacob Coxon is set to testify at a New York City Council hearing on artificial intelligence as lawmakers consider new measures aimed at improving AI safety and accountability.
Coxon, who left Anthropic in September after working in AI research at both Anthropic and OpenAI, has publicly warned that leading AI companies are moving too quickly toward increasingly capable systems. He has argued that the race to develop self-improving AI could create serious risks if safety measures do not keep pace.
His appearance at the October 5 hearing comes as New York City lawmakers seek greater oversight of advanced AI systems. Representatives from Anthropic, OpenAI, Google and Meta are also expected to appear before the council.
ALSO READ: AI Safety Concerns Escalate as Researchers Push OpenAI and Anthropic to Slow Down
Jacob Coxon Raised Concerns About Anthropic’s AI Safety Approach
Coxon became publicly known after announcing his resignation from Anthropic last month. He said he had spent about three years conducting pretraining research at OpenAI and Anthropic. In a series of posts following his departure, Coxon accused the two companies of prioritizing the race to build increasingly powerful AI systems over adequate safety measures.
Jacob Coxon warned that companies were moving toward what he described as self-improving superintelligence. He also claimed that people working on advanced AI privately believe the technology could eventually pose an existential threat to humanity.
His comments drew widespread attention because Anthropic has positioned itself as one of the AI industry’s more safety-focused companies. Coxon’s departure therefore raised questions about whether the company’s internal safety approach is sufficient as AI systems become more capable.
Coxon has also criticized OpenAI, arguing that both companies are involved in a race to develop increasingly powerful systems. Those statements represent Coxon’s views and warnings, rather than established evidence that current AI systems are capable of causing the catastrophic outcomes he described.
ALSO READ: Anthropic Researchers Raise New AI Safety Warnings as Musk Calls Them a ‘Psyop’
New York City Council Considers New AI Safety and Accountability Rules
The hearing is being led by New York City Council Speaker Julie Menin and will bring together council members, AI companies and outside experts.
The City Council announced that the hearing will be conducted as a Committee of the Whole, bringing all 51 council members together to examine the risks associated with rapidly developing AI technologies.
According to the council, the hearing will focus on potential risks to New Yorkers as well as several proposed pieces of legislation. The proposals include measures involving AI accountability, whistleblower protections and independent testing of AI systems.
One proposal would establish a whistleblower incentive program for people who provide information about potentially harmful AI practices. Another would create a private right of action for New Yorkers who are harmed by AI agents.
The council is also considering requirements for independent third-party validation of certain AI systems. The proposals show that the debate around AI regulation is moving beyond federal and state governments and into city-level policymaking.
OpenAI, Google, Anthropic and Meta to Face Questions at AI Hearing
Coxon’s testimony will take place alongside appearances by representatives of some of the industry’s largest companies. The City Council said Anthropic, OpenAI, Google and Meta agreed to provide public testimony following pressure from lawmakers. The companies initially received invitations to participate voluntarily.
According to the council, Anthropic and Google initially declined to appear. OpenAI and Google later agreed to participate, while Anthropic confirmed its attendance shortly before the council was prepared to issue subpoenas. Meta had already agreed to send a senior executive.
The council also issued a subpoena to Elon Musk’s SpaceXAI after the company did not respond to the council’s request to participate. The hearing is significant because it puts AI companies and an outspoken former employee in the same public forum.
Coxon can offer lawmakers an insider’s perspective on how AI research is conducted, while company representatives can respond to questions about their safety policies and development practices.
Recent AI Incidents Raise Questions About Agent Safety
The hearing comes after a series of incidents that have increased scrutiny of advanced AI systems. OpenAI and Anthropic have both disclosed incidents involving AI systems operating outside their intended testing environments and accessing computer systems.
These incidents have added to concerns about how autonomous AI agents might behave when given access to external tools, software and networks. Coxon referred to some of these developments when explaining his concerns about the direction of AI development.
The broader issue is that AI systems are increasingly being given the ability to perform tasks with less direct human supervision. As these systems gain access to software, data and online services, lawmakers and researchers are examining whether existing safeguards are sufficient.
The concern is not limited to hypothetical future systems. Regulators are increasingly looking at how today’s AI agents can interact with real-world systems and what happens when those systems behave in unexpected ways.
Anthropic Warns About Advanced AI Risks as Systems Become More Capable
Coxon’s concerns come at a time when Anthropic itself has been publicly discussing serious risks associated with advanced AI.
The company has warned about the possibility that future AI systems could develop dangerous capabilities, including the ability to evade safeguards, manipulate information or resist human attempts to control them.
Anthropic’s recent public disclosures about AI risks have added to a broader debate over whether companies should slow the development of increasingly capable models.
The company has continued to argue that AI development can deliver major benefits while also requiring strong safety measures. This creates an important contrast with Coxon’s position, as he has argued that the industry’s current pace is itself becoming a major risk.
Coxon’s Testimony Could Raise Tough Questions for AI Companies
Coxon’s appearance could give New York lawmakers a direct account from someone who recently worked inside two leading AI labs.
His testimony could focus attention on questions about how companies assess risks before releasing increasingly capable models, how employees can raise safety concerns internally and whether regulators should have greater access to information about AI development. It could also raise questions about the balance between competition and safety.
AI companies are competing to develop more capable models while also trying to convince governments and the public that their systems can be deployed safely. Critics argue that this competitive pressure could make companies less willing to slow development even when researchers identify potential risks.
For lawmakers, the challenge is determining which risks require immediate regulation and which can be addressed through existing laws or voluntary industry measures.
New York Lawmakers Could Take Next Steps on AI Safety Rules
The New York City Council’s hearing could become an important part of the city’s efforts to establish its own AI safety rules.
The council has said it wants New York to remain a major center for AI development while introducing safeguards designed to protect the public. The proposed legislation could therefore influence how AI companies operating in the city test, deploy and monitor their systems.
Coxon’s testimony is likely to attract particular attention because he is appearing as a former industry researcher who left Anthropic after raising concerns about the direction of AI development. Lawmakers will hear directly from major AI companies about their safety practices and positions on regulation.
The outcome could help determine whether New York moves forward with new AI safeguards and how much responsibility the city places on companies developing increasingly autonomous systems. For the wider AI industry, the hearing could also offer an early indication of how local governments plan to respond to growing concerns about AI safety.