Anthropic proposes AI safety bill with model release restrictions tied to death and property damage thresholds
The AI lab has circulated a draft bill that would block model distribution if there is foreseeable imminent risk of 50+ deaths or $1 billion+ in property damage from specific catastrophic scenarios.
What happened
Anthropic has drafted legislation for congressional consideration that would restrict AI model release under specific conditions. The bill defines catastrophic risk as a foreseeable, imminent scenario in which over 50 people could die or over $1 billion in property damage could occur, stemming from biological weapon creation, model control evasion, or cyberattack. Independent evaluators registered with the Commerce Department would assess models for these risks; upon identifying catastrophic risk, they would have 72 hours to alert Commerce, which would then request a D.C. judge to halt release. The bill explicitly forbids federal agencies from independently halting distribution. The metrics align with California's recently passed Senate Bill 53, and Anthropic confirmed the draft to Politico.
Context
The proposal reflects a year of intensifying AI safety debate and concerns about the potential for AI-caused harm. Anthropic's draft goes further than some other industry regulatory proposals—notably Nvidia's CEO recently argued against new AI laws entirely—but remains less stringent than some policy advocates might demand. The bill's reliance on potentially lab-funded independent evaluators and a court-approval requirement represents Anthropic's preferred approach to oversight: third-party monitoring rather than direct federal agency authority. The specific risks cited in the bill (biological weapons, model control loss, cyberattacks) differ from the primary recent industry concern about AI systems evading human operators and performing unauthorized hacking. Industry labs have generally resisted mandatory government oversight while simultaneously warning of civilizational risks, and Anthropic's legislative proposal can be understood as an attempt to preempt more stringent regulation with a self-favoring framework.
What's disputed
It is not clear from the sources whether lab payments to evaluators could be sufficiently isolated from evaluation results to prevent influence, and the adequacy of the 72-hour notification window and court-approval process to actually prevent harmful releases is not addressed in the reporting.