In July 2026, OpenAI ran internal cybersecurity tests to assess how well its internal research models, IM1 and GPT-5.6 Sol, could identify and exploit weaknesses in computer systems. Their assignment, ExploitGym , required the models to exploit software vulnerabilities to retrieve a hidden answer called a flag. The models were put inside a sandbox: a computing environment designed to restrict their access to outside systems. What ensued was unprecedented. The models found a chink in the armor, reached the public internet, and compromised systems belonging to Hugging Face , a platform that hosts AI software, including OpenAI’s own open-weight models.
The models had not been instructed to attack Hugging Face. OpenAI reported that its customer data and public-facing products were unaffected, although the agents obtained some private data from Hugging Face. The models had treated the boundaries of their test as an obstacle to overcome.
The episode has intensified a broader debate over whether frontier AI systems advance faster than institutions responsible for controlling them. In September of 2026, Anthropic CEO Dario Amodei urged companies to slow frontier-model development and give independent evaluators greater access to their systems. Sam Altman and Elon Musk have also supported stronger outside evaluation and increased caution. Still, attempts at regulation have been few and far between. But critics argue that restrictions on American firms could weaken the US’s technological advantage if Chinese developers keep accelerating.
Anat Lior, Professor of Law at Drexel University, argued developers should carry more of the cost. Her work largely centers on tort law, which allows an injured person to seek compensation from whoever caused the injury. Under negligence law, a plaintiff normally has to show that the defendant failed to exercise reasonable care and that this failure caused harm. However, proving that connection can be difficult, even when developers struggle to explain their own models’ behavior. “Negligence can be difficult both because of breach and because of causation,” Lior said.
Bryan Choi, a professor at the University of Colorado Law School, warned against using unexpected model behavior as an excuse for AI harm. “You have to be able to foresee the general class of harm,” he said. “It doesn’t mean that you have to foresee the exact, specific way that the harm occurred.” He compares the problem to a highway construction site. Workers can anticipate that a car might cross into their work zone without knowing its speed or trajectory. “In this case too, when you are training AI agents to hack cybersecurity defenses, and you put them in a sandbox, you may not anticipate the exact sequence of how the AI agents interact and how they actually perform the intrusion. But you do foresee that this type of attack is possible or plausible.”
The rationale for developer liability is simple: those who develop AI systems have a responsibility to ensure their models cause no harm. For Choi, the question was: “Did any of those actors, whether it’s the principal company or the subcontractors, take appropriate precautions?” It should not matter that a particular AI breach was unexpected. For the courts, it is far more important to consider whether its employees exercised reasonable care and what developers did to mitigate risks.
Lawsuits are another way to notify the public about frontier AI and its development. “It provides discovery, creates an evidence-gathering process by which we seek to understand why harms have been caused,” Choi said of the court process. The litigation process can both help injured parties obtain compensation and show the public which company ignored warning signs or which safeguards failed. More importantly, the threat of lawsuits can also compel developers to take extra safety measures before they release their models. “It gives all these companies a collective incentive to slow things down because they’re all going to be a little more careful about not exposing themselves to liability,” he said.
Rebecca Crootof, a Professor of Law at the University of Richmond, similarly argued that courts already have precedent for scrutinizing AI development. “We hold entities responsible for the acts of their agents or other autonomous entities all the time,” she said. “And yet, when rights are affected by technology, judges get way more cautious about applying old law. Our existing legal regimes could do a lot of regulatory work if judges and policymakers were comfortable applying them.” Legislation could clarify how liability law applies to AI without the need to develop an entirely new legal system.
Still, tort law usually requires measurable harm. “I’m not even sure there were damages in the tort-law sense,” Lior commented on the Hugging Face incident. “[Tort law] generally helps us after harm happens,” she said. A near miss can show serious problems without permitting courts the ability to intervene.
Lior preferred strict liability as a means to legislate frontier AI. Under a strict liability regime, developers of unusually dangerous technology bear the responsibility for resulting harm even if they took reasonable precautions. Lior compares this to keeping wild animals, whose dangerous tendencies place the risk of injury on their owners. “If a bear gets out and bites someone, you are strictly liable,” Lior said.
Choi agreed that strict liability is a viable stopgap regime for AI regulation. “Again, this is not a ban on AI.” Choi said, “It means that if they deploy AI systems or models, it’s ‘deploy at your own risk.’ This framework places responsibility on the developer who chooses to introduce the danger.”
Lior did leave room for the standard to change as safeguards improve. “Once we figure out what the equivalent of airbags and seatbelts is for AI,” she said, “we can have a conversation about moving toward negligence.” Insurance could encourage those precautions by demanding information about safety practices. “Insurers see data points across many companies and many claims. They can identify patterns and tell policyholders what they can do to mitigate potential risks,” Lior said.
Andrew Burt, co-founder of Luminos.AI and a former FBI Cyber Division official, questioned whether ex post facto (after the fact) enforcement could keep pace with autonomous agents. “I take an open-weight model from China. I turn it into something in the U.S. Some company, let’s say in France, deploys it, and then someone in India does really bad stuff with it,” he posited. Each jurisdiction may have individual answers, but there is not one unified framework for dealing with AI. “The answers are slow, and they rely on dispersed legal systems,” said Burt.
His concern grew from watching lawyers and data scientists struggle to communicate when developing AI regulation. While lawyers handed over written policies, engineers had to decide how those policies should work in software. “You had one group of people living in Jupyter notebooks, and another group of people living in Microsoft Word, and never the two met,” he said. If regulations were meant to ensure AI would not discriminate, for example, developers would have to decide how to measure unequal treatment. When a model repeats those decisions across millions of users, a small team’s judgment can propagate enormous consequences.
“I think the only way to do this is with government, in some sense or form, or governments building their own AI that can be involved in, oversee, or be embedded into the AI that we ultimately want to regulate,” Burt remarked. He called this work legal engineering. Choi urged caution about who would supply and supervise those tools. “The risk is, of course, the fox guarding the henhouse,” he noted. “Would you be using the same models? Different models? Would they be as capable? I think there are some serious risks to independent oversight if you were to go down this route.” Regulators would need enough independence to challenge the companies whose technology they use, along with procedures for detecting errors in their own oversight systems.
Additionally, Burt was not convinced that companies would voluntarily uphold safety commitments. “Slowing down only works if they all do it at the same time,” he said. “We’re watching companies in an extremely competitive market, with really complicated geopolitical dimensions, say, ‘We need help.’” According to Burt, government intervention could assure companies that competitors will follow the same safety requirements.
“Slowing down only works if they all do it at the same time,”
But slowing development raises another concern. Critics argue that restrictions on American firms could weaken the US’s technological advantage if Chinese developers keep accelerating. In January 2025, Chinese developer DeepSeek released R1 , a reasoning model that it reported could compete with OpenAI’s o1, an advanced model that uses chain-of-thought reasoning, on several benchmarks. DeepSeek made the model’s weights available for online users to download and adapt. Long safety-testing pipelines compound competitive pressures domestically and abroad; if firms keep accelerating for fear of losing ground, they have little incentive to slow down when AI models show dangerous tendencies.
Calls for regulation can also serve the interests of leading AI companies. Ted Wittenstein, director of the Yale Schmidt Program on Artificial Intelligence, warned that large companies can hire compliance teams while smaller competitors may struggle with the expense. “If there’s too much liability or risk on a startup before they release a new capability, you’re going to stifle potentially the innovation on that front,” he said. Regulation could therefore exclusively benefit companies already leading the industry.
“A company with robust internal controls is not necessarily slowing down,” Wittenstein said. “In fact, no one wants to deploy a system that is not safe or secure. That would create liability issues. That would create customers who don’t want their product.”
Burt likewise favors different legislative requirements for different uses. “Using an open-weight model to make a recommendation on what TV show you will like is very different from making a recommendation on what medication is going to most effectively work for you,” he said. Rules aimed at the broadest systems should address their developers directly, which could reduce the burden on smaller companies that use those models for narrower applications.
Wittenstein sees another obstacle to slowing down American AI development. “I’ve also seen no indication that Chinese tech executives or government officials have expressed an interest or willingness to slow down.” China’s August 2025 “AI Plus” policy promotes AI adoption across the economy, backed by expanded research funding and technical education.
Of course, the AI arms race is not merely about economic rankings. Wittenstein argues that cheaper, openly available Chinese models could attract users worldwide even if American companies retain a technical lead. “If China has even a model that’s 75 percent as good, but can get a large portion of the world to use this tech stack, then maybe that is winning, right?” he said. Under that interpretation, widespread adoption could give Chinese technology a strong commercial position even when other companies produce stronger systems.
China also subjects AI to state controls. “They want a model that is ultimately very clearly under state control and compliant with state laws, censorship laws, and things of that nature,” Wittenstein said. Preventing dangerous AI could serve both countries’ interests even though they disagree about which uses need legislation.
Burt sees a tension between China’s immediate commercial strategy and its longer-term interest in avoiding destabilizing systems. Open-weight models can undercut American companies, but these dangerous applications could threaten Chinese institutions too. Speaking about the two countries' incentives, he said, “At a certain point in time, I think there’s going to be significant overlap.”
But what advantage does faster AI development actually provide? “Does being ahead in AI mean having the frontier model that, for five seconds, is ahead of the other frontier models? What is the finish line for AI? ” Crootof asked. Unlike a race with an agreed destination, AI competition involves changing capabilities and different uses. Crootof warned against prioritizing “speed over quality and knowledge of what we’re doing and why.”
Independent evaluation could reduce some competitive pressure. If every frontier company had to meet the same testing requirements, each could spend more time on safety without surrendering ground to a less careful competitor. Amodei propose s ongoing access for outside evaluators, which would allow them to examine development practices instead of simply inspecting finished models. Yet no clear consensus exists on what evaluators should demand. “What exactly is it that they’re evaluating?” Burt asked. “What are the standards, and what are the criteria?”
The European Union has begun developing frameworks to address frontier development through its AI Act. Providers of general-purpose models that pose systemic risks must meet testing requirements to reduce those risks. They must also report serious incidents and maintain cybersecurity protections. The EU AI Act sets a useful precedent, but evaluators still have to decide which tests companies should run and what results would show that a model’s safeguards work.
Europe’s approach places legal obligations on companies based on the risks their systems pose. China combines its push for adoption with requirements that public-facing generative AI services uphold state-approved political values. Both approaches give the government a direct role in regulation, but they pursue different ends: the EU’s rules address safety and fundamental rights, while China’s rules also enforce political control over generated content.
Since the start of 2025, the United States has pursued a different course. The Trump administration’s July 2025 AI Action Plan emphasizes faster innovation and fewer regulatory barriers. “We’re dealing with an executive who has spoken out vocally against AI regulation or legislation,” Choi said. Wittenstein also described a longstanding American preference for giving technology companies room to innovate.
Yet policymakers have begun addressing some AI harms. The bipartisan TAKE IT DOWN Act, signed in May 2025, prohibits the publishing of nonconsensual intimate imagery, including AI-generated images. Some states have pursued their own AI laws, while existing liability rules give people means to seek compensation. For example, California passed the AI Transparency Act in 2025, requiring developers of massive frontier models to publicly disclose safety frameworks.
American voters are less concerned about winning the technological race. A June 2026 Pew Research Center report found that about two-thirds of Americans thought AI was advancing too quickly. Deepfakes threaten privacy, and AI-generated falsehoods can make political manipulation cheaper and harder to trace. “From a regulatory perspective, it makes sense to focus on the short-term problems while still considering long-term existential risks,” Lior stated. Those immediate concerns also give the public concrete reasons to demand oversight. In a September 2025 report , Pew found that “about six-in-ten say they’d like more control over how AI is used” in their own lives.
“The technology is moving very quickly, and the people controlling it do not necessarily bear all of the negative externalities that could harm the rest of us,” Lior said. “It may be up to us to care about our future and decide what AI should and should not do.”
“It may be up to us to care about our future and decide what AI should and should not do.”
“We cannot simply wait and see what happens.”