US Lawmaker Proposes Sweeping Bill: No Recursive Self-Improvement for AI Until Government Safeguards Are in Place

September 30 — On the 28th local time, US broadcaster CNBC reported that Democratic Congressman Ro Khanna, who represents a Silicon Valley district, is preparing to introduce an AI regulation bill that would impose strict liability standards and bar AI from recursive self-improvement when government safeguards are absent.

In an interview, Khanna admitted candidly: "In fact, there is a risk of civilizational extinction. There is both the safety risk of losing control and the risk of misuse, and both must be taken seriously."

Khanna has named the bill the Human Control Over AI Act. Until the federal government establishes safeguard rules and regulators approve the relevant activities, AI models may not engage in recursive self-improvement, nor may they autonomously modify their own core objectives, containment mechanisms or shutdown controls.

The bill also plans to create a new federal agency responsible for drawing up AI safety regulations, approving model training and deployment through a licensing system, auditing frontier AI models, and setting safety standards to ensure that models are continuously tested and remain under effective human control.

Every frontier AI lab would also have to be staffed with independent auditors, who would report directly to the new agency. The agency would further set standards covering sandbox testing environments, physical isolation that disconnects test environments from the internet, emergency kill switches, and controls that prevent models from running on their own on the internet after they leave the lab environment. The advanced chips used by frontier AI labs would also be brought under regulation, including mechanisms to monitor or control their use.

As IT Home understands it, the bill would also add legal penalties for cases in which AI goes wrong. If the deployment of an AI model leads to the destruction of civilian populations, it would be classified as a "crime against humanity" and prosecuted as a criminal offense.

Employees of AI companies would likewise face criminal penalties if they disable safeguards, emergency kill switches, logging functions or containment systems, or if they knowingly deploy a system that has not been authorized. In addition, AI companies would have to carry broad liability insurance before they could release a model.

Khanna said the bill is the "most comprehensive AI safety legislation" proposed to date, and that its design came mainly from his conversations with several independent, non-profit AI safety organizations, rather than from demands made by executives at large AI companies. "I hope this bill can serve as a template, giving the AI safety community a voice. I think they have unfairly been treated as merely talking about science fiction and have not received enough attention… It should be the AI safety community, not executives at frontier AI labs, that leads this discussion."