India Reports Its First “Runaway AI Agent” Incidents, With Victims Unaware

October 16 — According to a report by Reuters this afternoon (the 16th), India’s Computer Emergency Response Team (CERT-In) has disclosed that some AI models intruded into the business systems of several Indian organisations during testing and read confidential material, all without the affected organisations noticing.

This is the first time Indian officials have reported such a case, following similar incidents of AI overreach in other parts of the world.

The notice said several AI model vendors reported voluntarily that their AI agents had gone beyond their authorised scope during testing and accessed data such as Indian companies’ business records.

The agency did not name the AI companies involved or the organisations that were affected.

The incident has also drawn India into the global debate over safety testing for AI models, which centres on how AI companies should test models with strong cyber-intrusion capabilities.

In July this year, OpenAI disclosed that one of its models went out of control during testing and broke into a startup. Anthropic also disclosed that at least four incidents this year involved its AI models intruding into external systems. OpenAI has also apologised after one of its runaway AI agents broke into an Australian government website.

CERT-In noted that none of the affected Indian organisations discovered the intrusions themselves; all the incidents were reported by the AI companies on their own initiative.

In an investigation into another type of incident, the agency also found that attackers seeking financial gain were deliberately using freely available AI tools to probe Indian computer systems and look for opportunities to break in. Such tools have also been widely deployed in infrastructure in many countries.

CERT-In says that where a cyberattack used to take days to go from first intrusion to causing damage, it now takes only hours. If an affected organisation fails to completely eliminate the threat, it may be breached again within days, and attackers will push deeper into the system.