Google Reveals AI Framework Transformed Security Patches into Exploits Without Human Intervention

Associated Writers

A new report released today by Google’s Threat Intelligence Group (GTIG) reveals a significant development in cyberattacks: attackers have utilized an artificial intelligence framework, known as exploit_agent, to autonomously convert security patches into exploits without human intervention.

This framework was discovered on an exposed staging server, linked to several frontier and locally hosted large language models (LLMs). Its primary function is to continuously compare popular web application code before and after security patch announcements. Once a potential vulnerability is identified, the exploit generated is executed automatically.

Data harvested from victims resided on the same server, alongside scripts for brute-forcing access, deploying web shells, and even removing web shells planted by competing attackers. Telemetry indicates a strong focus on French websites related to media, entertainment, and education. A remark within the framework’s configuration file, written in Russian, instructs the operator to enter the keys “right here, not in the chat.”

In the previous quarterly tracker released two months earlier, GTIG stated that fully autonomous attack pipelines had not yet been observed in real-world scenarios. However, the latest report, titled “Agentic Threat Activity Crosses a Significant Threshold,” indicates a shift, as attackers have increasingly targeted known vulnerabilities rather than seek out zero-day exploits. Given the finite nature of source material for patched flaws, GTIG posits that directing LLMs towards n-day vulnerabilities may soon prove to be more effective.

The pace at which the window between vulnerability disclosure and the development of responsive exploits can close is illustrated by two flaws in PaperCut Software Pty Ltd.’s print management servers, which were exploited as zero-days in late August. Security researchers and Huntress Labs Inc. employed LLMs to generate functioning exploits within just one day of the disclosure.

A campaign monitored by GreyNoise Intelligence Inc. four days after the flaw disclosure is believed to have utilized LLMs for exploit creation, subsequently affecting hundreds of organizations. One instance of this campaign achieved domain administrator access within mere minutes, as reported by GreyNoise.

Additionally, open-source agentic penetration testing frameworks are being employed to automate various components of live intrusions. For instance, during an attack on the U.S. aerospace sector, a threat actor was detected utilizing the multi-agent framework CAIRN.

Another actor operated a tailored platform based on PentAGI from late August to early October, conducting vulnerability scans across various organizations in Asia. Currently, the apparent forensic trails left by unrestricted autonomous execution are reportedly making these frameworks easier to detect.

GTIG also noted that influence operations are employing models for purposes beyond merely drafting posts. Utilizing cloud phone farms, a turnkey platform supported by the Gemini API is being developed by a commercial contractor to manage over 100,000 social media accounts. Specifically, prompts from a separate platform targeting political sentiment across Europe and the Middle East were designed to simulate smartphone photography flaws in generated images.

Among state-sponsored actors, images on fraudulent recruiter profiles identified as being operated by the suspected North Korean group UNC6836, which targeted cryptocurrency firms in North America and Europe, were found to contain Google’s SynthID watermarks. Furthermore, through modified .claude/settings.json files in internal code repositories, this group manipulated the launch of Anthropic PBC’s Claude Code to download their SOMBERMEME backdoor.

In response to these findings, Google has disabled the accounts and resources related to the misuse of Gemini that it identified. Additionally, Google DeepMind has leveraged these insights to enhance its classifiers.

[gspeech type=full]

Share This Article
Leave a comment