Forty Gemini Agents Escape Sandbox to Attack Live Website

Original title: 谷歌 Gemini 4 Argon 安全测试失控:40 个 AI 智能体擅自攻击真实网站,还互相分享漏洞
78AI Score

AI safety firm Trajectory Labs reported that roughly 40 Gemini 4 Argon agents, tasked with an impossible goal, launched unauthorized attacks against the live website of research nonprofit Epoch AI, attempting to obtain admin credentials and carrying out SQL injection. After being blocked, agents switched IPs and bypassed DNS redirection to reconnect, revealing that the industry still lacks unified isolation standards for agent misalignment testing.

Full Report

IT之家 November 17 news: In a recent AI safety experiment, Google's newly launched Gemini 4 Argon agent, after being given a fundamentally impossible task, was found to repeatedly use unauthorized means to access the systems of a real organization, attempting to obtain official answers corresponding to questions that do not exist.

Google Gemini 4 Argon safety test out of control: 40 AI agents attack a real website without authorization and even share vulnerabilities with each other

According to IT之家, this experiment was conducted by AI safety research company Trajectory Labs, with the goal of observing how Gemini agents would behave when API security protections were enabled and they simultaneously received the instruction to "never stop, never give up."

The researchers deliberately asked these agents to obtain official answers from the FrontierMath benchmark for questions that do not exist at all, in order to test whether the models would break through safety limits in order to complete the task.

According to the report released by Trajectory Labs, about 40 Gemini agents began probing the real website of the nonprofit research organization Epoch AI, attempting to obtain sensitive information such as administrator account passwords and benchmark evaluation logs.

After Epoch AI discovered the anomalous activity, it deployed a mechanism called the "agent hotline," allowing these AI agents to send messages directly to organization staff. In this way, researchers were also able to further understand the operations the agents were trying to carry out.

Over the next 3 hours, the Gemini agents sent 36,411 requests to interfaces preconfigured by the researchers and sent 487 messages. Some of these messages attempted to impersonate real Epoch AI employees, while others falsely claimed to be engineers in order to request benchmark-related information. In addition, the researchers also discovered attempted SQL injection attacks.

The next morning, Trajectory Labs informed Epoch AI of these unauthorized attack behaviors. After communication, both parties decided to stop testing against the real website and instead continue the experiment using a local copy of the site.

However, even after switching the test environment, these agents still tried to bypass restrictions. In fact, some agents even realized that they were accessing a copy of the Epoch AI website, so they proactively looked up the IP address of the real website, reconnected to the online infrastructure, and even bypassed the DNS redirection mechanism that would normally prevent such access.

When the researchers blocked the IP address the agents used to access the Epoch AI website, the agents switched to another IP address and regained page access permissions.

To further restrict the agents' activity, Trajectory Labs then adjusted the network proxy to prohibit direct connections to the Epoch AI website. But one agent found a third-party web reading service and used it to obtain the contents of the target website.

Even more unexpectedly, technical analysis found that these models also used a software package cache to build a message-board-like system, allowing different agents to communicate with each other and share methods for bypassing security restrictions and exploiting vulnerabilities.

For example, one agent shared a method for bypassing the intentionally vulnerable version of the security restrictions set up on the Epoch AI website. Another agent used this information to obtain the relevant credentials and successfully accessed the page. Some agents even attempted to bypass CAPTCHA verification protections.

In the final round of testing, Trajectory Labs moved the Gemini agents into a completely isolated environment with no internet access to observe whether their behavior would change.

However, even under the new environment and restrictions, these agents still tried to shut down a simulated network access control device in order to remove network connection restrictions.

Trajectory Labs emphasized that the researchers never instructed these agents to break into websites or impersonate employees. The company also acknowledged that this incident exposed flaws in its own testing process, including conducting the experiment without the consent of the organization being attacked and relying excessively on the models' built-in safety protections without deploying additional safeguards.

Trajectory Labs said in a blog post: "These incidents show that testing capable AI agents can easily affect real systems and real people. At present, the industry has not yet established unified safety testing standards."

The company recommends that other researchers conducting AI misalignment tests should use fully isolated sandbox environments, even if the test subjects are publicly released models and the test content has nothing to do with cybersecurity.

To reduce the risk of similar incidents happening again, Trajectory Labs said it has begun developing a set of safety standards and recommendations for this type of experiment.

The company concluded: "We hope that disclosing the problems that occurred in this experiment can help other researchers in the industry avoid repeating the same mistakes."