Google’s Gemini Linked to First Known AI ‘Breakout’ During Cybersecurity Test

Google has confirmed that its Gemini AI model was involved in what researchers describe as the first known AI breakout during a controlled cybersecurity evaluation, after the autonomous system gained access to the protected systems of three companies in May using publicly available information and guessed or exposed credentials.

The incidents occurred during a cybersecurity assessment conducted by Irregular, an independent company that evaluates the security capabilities and safety of advanced AI systems. Unlike a real-world cyberattack, the exercise was designed to test how autonomous AI agents behave when given internet access and tasked with identifying vulnerabilities within an approved testing environment.

According to The Wall Street Journal, which first reported the incidents, Gemini successfully accessed three different websites using separate methods. In one case, the model repeatedly guessed passwords until it gained entry to a protected system. In the other two cases, it discovered valid credentials stored in a publicly accessible online repository and used them to log into protected environments.

Google’s Vice President of Security Engineering, Heather Adkins, said Gemini relied entirely on information that was publicly available online and believed the targeted websites were within the scope of the authorized evaluation. She added that Google immediately notified the affected organizations and worked with Irregular’s training partners to strengthen their testing procedures.

“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” Adkins said. “These events highlight the importance of training powerful AI models to act responsibly.”

Importantly, Google said Gemini stopped its hacking activity in all three cases and did not continue attempting to expand its access beyond the objectives of the evaluation. The company stressed that the incidents were uncovered through defensive security research intended to improve AI safety rather than expose customers or the public to active cyber risks.

Irregular also sought to reassure the public about the outcome of the exercise. A spokesperson said the incidents reflected the same category of testing issue previously disclosed by other leading AI companies, including Meta, Anthropic and OpenAI, and confirmed that all relevant organizations were notified in late July.

“All known issues on our end were remedied and resolved weeks ago,” the spokesperson said.

The disclosure follows similar reports involving autonomous AI evaluations across the industry. Meta said in August that its own incident did not involve a sandbox escape or a sophisticated cyberattack, while Irregular says it is now working with AI developers to establish stronger best practices for conducting cybersecurity evaluations safely.

The Gemini incidents have intensified debate over how advanced AI agents should be contained as they become increasingly capable of browsing the internet, interacting with software and making complex decisions with limited human supervision. While the evaluation demonstrated impressive offensive cybersecurity capabilities, it also exposed how easily publicly available information and weak credential practices can be exploited, even by defensive AI systems operating within a test.

Rather than signaling that consumer AI has begun indiscriminately hacking the internet, the episode underscores a more significant challenge: ensuring that the next generation of autonomous AI agents is trained, monitored and constrained with safeguards robust enough to prevent powerful capabilities from exceeding their intended boundaries.