WHAT ACTUALLY HAPPENED
● When: May 2026, during a capture-the-flag evaluation run by the third-party evaluator Irregular.
● The cause: a fictional test company shared its name with a real one. The model went looking for the target and found the wrong one.
● The methods: guessing a password repeatedly in one case; credentials left in a public repository in the other two.
● What it did not do: finish. Google says the model stopped before completing the act every time.
● The timeline: Irregular notified Google at the end of July. Public disclosure came on Friday 19 September.
The part that is not the story
An AI did not go rogue. Read the methods again: it guessed a password, and it found credentials somebody had left in a public repository. That is the oldest pair of attacks there is, and the second one is a finding about the repository, not the model.
The model also stopped. Every time, before completing anything. Google's position is that this shows the safety behaviour working, and on the narrow question that is a fair reading.
Anyone writing this up as a machine breaking its chains is selling something. The interesting failure is upstream and duller: a test environment used a company name that belonged to somebody else, so an agent told to go find its target went and found a real business.
The part that is the story
SEVEN WEEKS, AND A POSITION THAT IT NEVER NEEDED TO BE SAID
Irregular told Google at the end of July. The public learned on 19 September.
Google's stated view is not that the delay was regrettable. It is that the incident did not warrant public disclosure at all, because the safety measures behaved as designed and the behaviour did not indicate misalignment.
Three real companies had their systems accessed by a model that was not supposed to be anywhere near them. Whether the model behaved well is a separate question from whether the public gets told.
Heather Adkins, Google's vice president of security engineering, said that safe development of powerful AI models is critical and that the company invests deeply in the area. Google contacted the affected entities and worked with its training partner on changes to the testing process.
That is a reasonable set of actions. It is also entirely compatible with nobody outside ever hearing about it, which is what nearly happened.
Three days, two opposite instincts
| Date |
Who |
What they published |
| 17 Sept | Anthropic | A monitor block rate of roughly one action in 47,000, unprompted |
| 18 Sept | OpenAI | Confirmation of a standards body, after a competitor forced the question |
| 19 Sept | Google | A May incident, seven weeks after being told, while maintaining it did not need to |
We argued on the 17th that disclosure against interest is the only kind that carries evidential weight. This is the control case. A voluntary regime produces exactly the disclosures each company decides it can live with, and the variance between labs is wider than the variance between incidents.
Why this lands on the standards body
Two days ago three labs confirmed they are building a FINRA-style body to test powerful systems before release. Google DeepMind is one of the three.
The obvious question a body like that has to answer is not what gets tested. It is what gets published, by whom, and on what clock. Pre-release evaluation is either meaningful or a stamp, and the difference is publication rights.
Here is a real evaluation, run by a real third party, that found a real problem. The finding took seven weeks to surface, and the firm involved still says it should not have had to. A standards body with no disclosure clock would have changed nothing here →
What to take from it
- Not a rogue AI. The model stopped every time. Treat headlines saying otherwise accordingly.
- Do not name test fixtures after real companies. The root cause is this boring and this preventable. If you run agent evaluations, go and check your fixtures today.
- Credentials in public repositories are still the way in. Two of the three accesses needed no cleverness at all.
- Disclosure timing is the open question in AI safety, not capability. Every lab has incidents. They differ in what they say and when.
- This is not unique to Google. Comparable incidents have come up in evaluations of Claude, OpenAI models and Meta's systems. That is the point: it is an industry norm question.
Sources
FAQ
Did Gemini hack three companies?
It accessed systems at three real companies during a May 2026 evaluation run by the evaluator Irregular, after a fictional target shared a name with a real business. It guessed a password in one case and used credentials found in a public repository in the other two. Google says it stopped before completing the act each time.
Was this a case of the model going rogue?
No. Google says the behaviour did not indicate misalignment and that the model halted each time. The failure was in the test environment, which used a company name belonging to a real organisation.
When did Google find out?
Irregular notified Google at the end of July 2026. The incident became public on 19 September 2026.
Why did Google not disclose it sooner?
Google's stated position is that the incident did not warrant public disclosure, because the safety measures worked as intended and the behaviour was not misalignment.
Were the affected companies told?
Google says it contacted the affected entities and worked with its training partner on changes to the testing process.
Does this affect Gemini users?
No. It concerns a controlled evaluation in May, not product behaviour. Nothing about availability or pricing changes.