Executive Summary

Since our February 2026 report on AI-related threat activity, Google Threat Intelligence Group (GTIG) has continued to track a maturing transition from nascent AI-enabled operations to the industrial-scale application of generative models within adversarial workflows. This report, based on insights derived from Mandiant incident response engagements, Gemini, and GTIG’s proactive research, highlights the dual nature of the current threat environment where AI serves as both a sophisticated engine for adversary operations and a high-value target for attacks. We explore the following developments:

• Vulnerability Discovery and Exploit Generation: For the first time, GTIG has identified a threat actor using a zero-day exploit that we believe was developed with AI. The criminal threat actor planned to use it in a mass exploitation event but our proactive counter discovery may have prevented its use. Threat actors associated with the People’s Republic of China (PRC) and the Democratic People's Republic of Korea (DPRK) have also demonstrated significant interest in capitalizing on AI for vulnerability discovery. 

• AI-Augmented Development for Defense Evasion: AI-driven coding has accelerated the development of infrastructure suites and polymorphic malware by adversaries. These AI-enabled development cycles facilitate defense evasion by enabling the creation of obfuscation networks and the integration of AI-generated decoy logic in malware that we have linked to suspected Russia-nexus threat actors.

• Autonomous Malware Operations: AI-enabled malware, such as PROMPTSPY, signal a shift toward autonomous attack orchestration, where models interpret system states to dynamically generate commands and manipulate victim environments. Our analysis of this malware reveals previously unreported capabilities and use cases for its integration with AI. This approach allows threat actors to offload operational tasks to AI for scaled and adaptive activity.

• AI-Augmented Research and IO: Adversaries continue to leverage AI as a high speed research assistant for attack lifecycle support, while shifting toward agentic workflows to operationalize autonomous attack frameworks. In information operations (IO) campaigns, these tools facilitate the fabrication of digital consensus by generating synthetic media and deepfake content at scale, exemplified by the pro-Russia IO campaign “Operation Overload.”

• Obfuscated LLM Access: Threat actors now pursue anonymized, premium tier access to models through professionalized middleware and automated registration pipelines to illicitly bypass usage limits. This infrastructure enables large scale misuse of services while subsidizing operations through trial abuse and programmatic account cycling.

• Supply Chain Attacks: Adversaries like "TeamPCP" (aka UNC6780) have begun targeting AI environments and software dependencies as an initial access vector. These supply chain attacks result in multiple types of machine learning (ML)-focused risks outlined in the Secure AI Framework (SAIF) taxonomy, namely Insecure Integrated Component (IIC) and Rogue Actions (RA). Our analysis of forensic data associated with these attacks reveals threats actors attempting to pivot from compromised AI software to broader network environments for initial access and to engage in disruptive activities, such as ransomware deployment and extortion.

Attackers rarely shy away from experimentation and innovation, but neither do we. In addition to  sharing our findings and mitigations with the larger security and AI community, Google employs proactive measures to stay ahead of these constantly changing threats. Google enhances our products’ safeguards to offer scaled protections to users. For Gemini, we mitigate model abuse by disabling malicious accounts. Furthermore, we leverage AI agents like Big Sleep to identify software vulnerabilities and use Gemini’s reasoning capabilities via the likes of CodeMender to automatically fix them, proving that AI can also be a powerful tool for defenders.

AI as a Tool

Threat actors are leveraging AI to augment various phases of the attack lifecycle. This includes supporting the development of vulnerability exploits and malware, facilitating autonomous execution of commands, enabling more targeted and well-researched reconnaissance, and improving the efficacy of social engineering and information operations.

AI-Augmented Vulnerability Discovery and Exploit Development

As the coding capabilities of AI models advance, we continue to observe adversaries increasingly leverage these tools as expert-level force multipliers for vulnerability research and exploit development, including for zero-day vulnerabilities. While these tools empower defensive research, they also lower the barrier for adversaries to reverse-engineer applications and develop sophisticated, AI-generated exploits.

State-Sponsored Threat Actors Demonstrate Sophisticated Approaches to Leveraging AI for Vulnerability Research

While we observe a variety of threat actors leveraging AI for vulnerability research, we noted a particular interest from several clusters of threat activity associated with the People’s Republic of China (PRC) and the Democratic People's Republic of Korea (DPRK). These actors have leveraged sophisticated approaches toward AI-augmented vulnerability discovery and exploitation, beginning with persona-driven jailbreaking attempts and the integration of specialized, high-fidelity security datasets to augment their vulnerability discovery and exploitation workflows.

• As we highlighted in prior blog posts, threat actors often leverage expert cybersecurity personas as a structured approach to prompt Gemini. For instance, we recently observed UNC2814 use this form of expert persona prompting by directing the model to act as a senior security auditor or C/C++ binary security expert. The fabricated scenarios were used to support vulnerability research into various embedded device targets, including TP-Link firmware and Odette File Transfer Protocol (OFTP) implementations.

Figure 1: Example of false narratives used to support persona-driven jailbreaking, a simple form of prompt injection

• In a more sophisticated use case, we observed threat actors experiment with a specialized vulnerability repository hosted on GitHub known as “wooyun-legacy.” The project is designed as a Claude code skill plugin that integrates a distilled knowledge base of over 85,000 real-world vulnerability cases collected by the Chinese bug bounty platform WooYun between 2010 and 2016. By priming the model with vulnerability data, it facilitates in-context learning to steer the model to approach code analysis like a seasoned expert and identify logic flaws that the base model might otherwise fail to prioritize.

In their pursuit of this vulnerability research, we see clear indications of automation and scaled research. In addition to leveraging individual prompts for real-time troubleshooting, we have observed APT45 sending thousands of repetitive prompts that recursively analyze different CVEs and validate PoC exploits. This results in a more robust arsenal of exploit capabilities that would be impractical to manage without AI assistance.

To facilitate these activities, actors are also experimenting with agentic tools such as OpenClaw and OneClaw alongside intentionally vulnerable testing environments. The use of these tools alongside vulnerability research suggests an interest in refining AI-generated payloads within controlled settings to increase exploit reliability prior to deployment.

Cyber Crime Threat Actors Discover and Weaponize Zero-Day Using AI

Cyber crime threat actors remain interested in leveraging AI for vulnerability development as well. In one notable example, we observed prominent cyber crime threat actors partnering to plan a mass vulnerability exploitation operation. Our analysis of exploits associated with this campaign identified a zero-day vulnerability implemented in a Python script that enables the user to bypass two-factor authentication (2FA) on a popular open-source, web-based system administration tool. GTIG worked with the impacted vendor to responsibly disclose this vulnerability and disrupt this threat activity.

Although we do not believe Gemini was used, based on the structure and content of these exploits, we have high confidence that the actor leveraged an AI model to support the discovery and weaponization of this vulnerability. For example, the script contains an abundance of educational docstrings, including a hallucinated CVSS score, and uses a structured, textbook Pythonic format highly characteristic of LLMs training data (e.g., detailed help menus and the clean _C ANSI color class).

The vulnerability can be classified as a 2FA bypass, though it requires valid user credentials in the first place. It stems not from common implementation errors like memory corruption or improper input sanitization, but a high-level semantic logic flaw where the developer hardcoded a trust assumption. While fuzzers and static analysis tools are optimized to detect sinks and crashes, frontier LLMs excel at identifying these types of high-level flaws and hardcoded static anomalies. Though frontier LLMs struggle to navigate complex enterprise authorization logic, they have an increasing ability to perform contextual reasoning, effectively reading the developer's intent to correlate the 2FA enforcement logic with the contradictions of its hardcoded exceptions. This capability can allow models to surface dormant logic errors that appear functionally correct to traditional scanners but are strategically broken from a security perspective.

AI-Augmented Obfuscation: Evasion and Polymorphism

GTIG has identified multiple threat actors experimenting with AI models to develop malware and operational support tools to augment obfuscation capabilities. This has included innovative applications of AI to incorporate just-in-time dynamic modification of source code, enable dynamic payload generation, assist in development of ORB network management tools, and generate decoy code (Table 1). While often experimental, this transition underscores a move toward AI-driven, evasive software suites.

In prior reports, we highlighted malware families like PROMPTFLUX, notable for its experimentation using the Gemini API to generate code, and HONESTCUE, which interacts with Gemini's API to request specific VBScript obfuscation and evasion techniques to facilitate just-in-time self-modification to evade static signature-based detection. In this report, we highlight additional tools and malware families created with the assistance of AI to support obfuscation and defense evasion.

We observed activity associated with the PRC-nexus threat actor APT27, which has leveraged Gemini to accelerate the development of a fleet management application likely to support the management of an operational relay box (ORB) network. Our observations of the tool revealed a "maxHops" parameter hardcoded to 3 hops, an indicator that the tool was related to development of an anonymization network rather than a VPN since those are typically set to 1 hop. Additionally, the tool lists MOBILE_WIFI and ROUTER as supported device types, suggesting it uses 4G or 5G SIM cards to provide residential IP addresses to potentially obfuscate the true origin of the intrusion activity. 

Additionally, GTIG has continued to observe Russia-nexus intrusion activity targeting Ukrainian organizations to deliver AI-enabled malware as part of their operations. Analysis confirms the use of CANFAIL and LONGSTREAM, which utilize LLM-generated decoy code to obfuscate their malicious functionality. 

• We identified multiple developer (i.e., the LLM) comments throughout CANFAIL's source code that specifically call out certain blocks of code that are not used and were likely incorporated as filler content designed to obfuscate malicious activity. The explanatory nature of these comments surrounding the decoy logic likely indicates the threat actor requested the LLM generate outputs that intentionally contained large amounts of inert code potentially for obfuscation (Figure 4).

• Similarly, our examination of the LONGSTREAM code family suggests a large volume of decoy logic was likely generated to camouflage the malicious nature of the code family. LONGSTREAM contains coherent but inactive blocks of code related to administrative tasks that are unrelated to the primary objective of the downloader. For example, we identified 32 instances of the code querying the system's daylight saving status. This type of repetitive query exists to populate the script with activity that can appear benign (Figure 5).

AI-Augmented Attack Orchestration: PROMPTSPY

Adversaries are advancing their implementation of AI-enabled tooling, moving beyond content generation and tool development and into more sophisticated autonomous attack orchestration for malware commands. Threat actors have begun relying on LLMs for interactive system navigation and real-time decision making. By integrating LLMs into malware operations, attackers can enable payloads to act autonomously, independently interacting with the victim environment or device, synthesizing system states, and executing precise commands devoid of human supervision.

A primary example of this evolution is PROMPTSPY, an Android backdoor first identified by ESET. Initial public reporting highlighted PROMPTSPY’s use of the Google Gemini application programming interface (API) to facilitate persistence, specifically by navigating the Android UI to pin the malicious application in the "recent apps" list. However, GTIG's examination of the backdoor revealed additional capabilities and use cases for its AI integration. We assess the malware's LLM component was designed to be extensible to support a broader range of goals centered around navigating the Android user interface and autonomously interpreting real-time user activity for follow-on actions. 

PROMPTSPY contains an autonomous agent module named “GeminiAutomationAgent,” which leverages a hardcoded prompt to facilitate automated interaction with the targeted device.

• The prompt assigns a benign persona to bypass the LLM's safety filters, then requests an analysis of complex spatial mathematics by instructing the LLM to calculate the geometry of the targeted user interface bounds. This is paired with a set of "Core Judgment Rules" that implement anti-hallucination measures and a “User Goal” concatenated to the prompt as part of a separate routine (Figure 6).

• The module then serializes the device's visible user interface hierarchy into an XML-like format via the Accessibility API, sending this payload to the “gemini-2.5-flash-lite” model via an HTTP POST request in "JSON Mode." 

• The model returns a structured JSON response based on the supplied user goal, dictating specific action types and spatial coordinates, which the malware parses using a packed-switch instruction to simulate physical gestures (e.g., CLICK, SWIPE). Since the user goal is not hardcoded in the initial prompt but supplied as part of a separate routine, we believe PROMPTSPY was likely designed to facilitate multiple types of device interactions.

Additionally, PROMPTSPY can capture victim biometric data to replay authentication gestures (personal identification numbers or lock patterns) to regain access to a compromised device for follow-on exploitation. These AI-enabled capabilities are a notable evolution from conventional Android backdoors that heavily rely on human interaction.

To maintain persistence, PROMPTSPY utilizes a novel multi-layered defense mechanism to camouflage its activity and prevent uninstallation. 

• If the victim tries to uninstall PROMPTSPY, the malware employs its 'AppProtectionDetector' module to identify the on-screen coordinates of the 'Uninstall' button. The malware renders an invisible overlay directly over the button as a shield that silently intercepts and consumes the victim's touch events, making the button appear unresponsive to the user.

• If the victim device becomes inactive, PROMPTSPY operators can utilize Firebase Cloud Messaging (FCM) to relaunch the backdoor, allowing the threat actor to continue their intrusion activity without alerting the victim. 

While PROMPTSPY initializes using hardcoded default infrastructure and credentials, the malware is designed with high operational resilience, allowing adversaries to rotate critical components at runtime without redeploying the PROMPTSPY payload. Specifically, the malware’s command-and-control (C2) infrastructure, including the Gemini API keys and the VNC relay server, can be updated dynamically via the C2 channel. This configuration model demonstrates the developers anticipated defensive countermeasures and engineered the backdoor to maintain presence even if specific infrastructure endpoints are identified and blocked by defenders.

Google has taken action against this actor by disabling the assets associated with this activity. Based on our current detection, no apps containing PROMPTSPY are found on Google Play. Android users are automatically protected against known versions of this malware by Google Play Protect, which is on by default on Android devices with Google Play Services.

AI-Augmented Research, Reconnaissance, and Attack Lifecycle Support

Malicious adversaries' most common use case for LLMs mirrors that of standard users – they conduct research and troubleshoot tasks. GTIG has observed a variety of threat actors engaging in this type of prompting to support research, reconnaissance, and troubleshooting throughout various phases of the attack lifecycle. By automating intelligence gathering and task support, these interactions lower the barrier to entry for complex, multi-stage operations and enable threat actors to focus their human capital on the higher-order strategic elements of campaigns.

Adversaries frequently use LLMs to perform reconnaissance that would previously have required significant manual effort. For instance, we have observed actors prompting models to generate detailed organizational hierarchies for specific departments and third-party relationships of large enterprises, particularly those involving high-value functions like finance, internal security, and human resources. This data allows for the creation of higher-fidelity phishing lures tailored to individuals with administrative privileges or access to sensitive data, moving beyond the commodity tactics of traditional bulk phishing.

In more targeted scenarios, actors have used LLMs to identify specific hardware or software environments used by their victims. In one instance, a threat actor attempted to identify the exact make and model of a computer used by a high-value target, even requesting the LLM identify a collection of photos showing the targeted individual using the device. This level of environmental fingerprinting often precedes the development of tailored exploits or identification of side-channel attack opportunities.

Beyond basic chat interfaces, we see a sophisticated shift toward agentic workflows where adversaries operationalize autonomous frameworks to execute multi-stage security tasks. This marks a significant evolution in the maturity of AI-related threats: the LLM is no longer merely a passive advisor but an active participant in the offensive chain, capable of orchestrating complex toolsets and making tactical decisions at machine speed.

For example, we recently analyzed a suspected PRC-nexus threat actor deploying agentic tools like Hexstrike and Strix against a Japanese technology firm and a prominent East Asian cybersecurity platform. Hexstrike was utilized alongside the Graphiti memory system, a temporal knowledge graph, to maintain a persistent state of the attack surface, allowing the agent to autonomously pivot between tools like subfinder and httpx based on its internal reasoning. Simultaneously, the actor leveraged Strix, a multi-agent penetration testing framework, to automate the identification and validation of vulnerabilities. This combination of autonomous reconnaissance and automated verification suggests a transition toward AI-driven frameworks that can scale discovery activities with minimal human oversight.

AI-Augmented Information Operations

GTIG continues to observe information operations (IO) actors use AI for common productivity tasks like research, content creation, and localization. We have also identified activity indicating threat actors solicit the tool to help craft articles, generate assets, and assist in coding. However, we have not identified this generated content in the wild, and none of these attempts have created breakthrough capabilities for IO campaigns. 

Actors from Russia, Iran, China, and Saudi Arabia are producing political satire and materials to advance specific narratives across both digital platforms and physical media, such as printed posters. The primary advances we have seen in this area include actors appearing more successful in developing tooling in support of their workflows and the growing adoption of AI-generated narrative audio to address contentious political topics. 

AI to Support IO Tactics

GTIG’s tracking of IO threats across the open internet continues to uncover activity illustrating how threat actors use AI tooling to enhance established tactics. For example, GTIG uncovered activity linked to the pro-Russia IO campaign “Operation Overload,” involving video content that leveraged suspected AI voice cloning to impersonate real journalists. This likely represents an AI-supported advancement of the campaign's established tactics, which have long included inauthentic video content designed to appropriate the branding and legitimacy of media and other high profile organizations in support of campaign messaging. 

In identified instances, the actors appear to have manipulated an authentic video to convey a false message. This content appears to splice original vertical videos with montages and fabricated audio to create false and misleading messaging. The close voice match to the original suggests the use of AI tools (Figure 7).

Obfuscated and Scalable Access to LLMs

As the generative AI landscape matures, the methods by which threat actors procure and operationalize these models have shifted from simple experimentation to industrial-scale consumption. Although in prior blog posts we have highlighted AI tools and services offered in the underground, we continue to observe both state-sponsored and cyber crime threat actors leveraging commercially available foundation models and AI-native application building platforms in their pursuit of malicious activity. 

In threat actor engagement with these tools, GTIG has observed a sophisticated evolution to an emerging ecosystem of custom middleware, proxy relays, and automated registration pipelines designed to bypass safety guardrails and billing constraints. By leveraging anti-detect browsers and account-pooling services, actors are attempting to maintain high-volume, anonymized access to premium LLM tiers, effectively industrializing their adversarial workflows while subsidizing their operations through trial abuse and programmatic account cycling.

In our analysis of PRC-nexus threat activity associated with UNC6201, we observed attempted use of a publicly available Python script hosted on GitHub that automates a workflow to register and immediately cancel premium LLM accounts. The tool allegedly supports the entire process from automatic account registration, CAPTCHA bypassing, and SMS verification to account status confirmation and cancellation. This process highlights the methods adversaries leverage to procure high-tier AI capabilities at scale while insulating their malicious activity from account bans.

We have observed similar activity from UNC5673, a PRC-nexus threat cluster that has notable overlaps with TEMP.Hex and that has targeted government sectors primarily in South and Southeast Asia. Beyond LLM account registration, the actor has leveraged an array of publicly available commercial tools and GitHub projects that indicate the development of obfuscated and scalable LLM abuse. For example, they employ "Claude-Relay-Service" to aggregate multiple Gemini, Claude, and OpenAI accounts, enabling account pooling and cost-sharing. Similarly, they use "CLI-Proxy-API," a proxy server that provides compatible API interfaces for various models to support similar account pooling strategies.

To mitigate the nature of this obfuscation, LLM providers can build signal logic to analyze network infrastructure data associated with AI-related API aggregators. This data helps to enable the disruption efforts we highlight in this report.

AI as a Target

As organizations continue integrating large language models (LLMs) into production environments, the AI software ecosystem has emerged as a primary target for exploitation. While frontier models themselves remain highly resilient to direct compromise, the orchestration layers, including open-source wrapper libraries, API connectors, and skill configuration files, can be vulnerable. GTIG has observed adversaries increasingly target the integrated components that grant AI systems their utility, such as autonomous skills and third-party data connectors.

Supply Chain Attacks Against AI Components

Throughout early 2026, we observed that threat actors have not yet achieved breakthrough capabilities to bypass the core security logic of frontier models. Instead, these actors are leveraging traditional supply chain tactics, such as embedding malicious logic in popular integration libraries or distributing trojanized configuration files, to gain initial access to production AI environments. These incidents often align with risks described in the Secure AI Framework (SAIF) taxonomy, specifically:

• Insecure Integrated Component (IIC): Inclusion of compromised external dependencies that undermine the system.

• Rogue Actions (RA): Exploitation of AI systems with elevated permissions to execute unauthorized commands or exfiltrate credentials.

Weaponized OpenClaw Skills

These risks became more apparent in early February 2026, when VirusTotal researchers reported on security risks associated with the OpenClaw AI agent ecosystem, including AI software supply chain risks and vulnerabilities introduced via malicious and insecure skill packages. Most notably, we observed the distribution of malicious packages masquerading as OpenClaw skills containing hidden routines designed to execute unauthorized code and commands on the host system. Given the elevated level of system access that OpenClaw is granted, a skill could be used to perform various privileged actions such as executing code, downloading additional payloads, and discovering and exfiltrating local data.

Further, even if not inherently malicious, insecure packages could expose users to additional risks. Legitimate skills that fail to leverage secure practices when handling sensitive information, such as credentials or authentication information, could inadvertently expose this information to attackers. This could make this information susceptible to theft by techniques like prompt injection, other malicious skills, or traditional malware threats like infostealers.  

While the risk of malicious or insecure skills and agent components are not unique to the OpenClaw platform, the discovery of these packages highlights the growing attack surface among AI development platforms and the agentic ecosystem more broadly. Further, the difficulty in identifying and discerning malicious packages from legitimate skills presents significant challenges for defenders. Although this infection vector is opportunistic by nature, the ease by which these skills can be created and distributed could make it an attractive option for a myriad of threat actors seeking access to users’ systems.

To help mitigate these supply-chain risks, OpenClaw has partnered with VirusTotal to integrate automated security scanning directly into ClawHub, its public skill marketplace. Every skill published to the repository is now automatically analyzed using VirusTotal's Code Insight capability, which evaluates the package's actual code behavior to detect unauthorized network operations, malicious payloads, or unsafe embedded instructions. Based on this security-focused analysis, skills are either approved as benign, flagged with user warnings, or blocked entirely, providing an essential layer of defense against ecosystem abuse.

Compromised Code Packages

In late March 2026, the cyber crime threat actor "TeamPCP" (aka UNC6780) claimed responsibility for multiple supply chain compromises of popular GitHub repositories and associated GitHub Actions, including those associated with the Trivy vulnerability scanner, Checkmarx, LiteLLM, and BerriAI. Mandiant responded to numerous incident response engagements associated with this activity, highlighting the wide-impact nature of supply chain operations.

TeamPCP gained initial access through compromised PyPI packages and malicious pull requests to these GitHub repositories. The threat actor subsequently leveraged their access to these GitHub repositories to embed the SANDCLOCK credential stealer and extract high-value cloud secrets, such as AWS keys and GitHub tokens, directly from affected build environments. These stolen credentials were then monetized through partnerships with ransomware and data theft extortion groups.

The compromise of LiteLLM, an AI gateway utility for integrating multiple LLM providers is noteworthy. It highlights the expanding attack surface of AI platforms and the potential for impact across the software supply chain. Given the package's widespread use, this incident could lead to considerable exposure of AI API secrets from affected victims, which could be used to gain further access to systems for traditional intrusion operations. 

Moreover, similar attacks against AI-related dependencies could grant attackers access to unique AI systems, allowing them to conduct novel AI-centric attacks and leverage them in support of traditional intrusion operations. Attackers could leverage this vector not only to pivot to enterprise infrastructure for traditional financially motivated operations (e.g., data theft and ransomware) but also to directly facilitate their operations using AI systems. For example, threat actors with access to an organization’s AI systems could leverage internal models and tools to identify, collect, and exfiltrate sensitive information at scale or perform reconnaissance tasks to move deeper within a network. While the level of access and particular use depends heavily on the organization and the specific compromised dependency, this case study demonstrates the broadened landscape of software supply chain threats to AI systems.

Building AI Safely and Responsibly

We believe our approach to AI must be both bold and responsible. That means developing AI in a way that maximizes the positive benefits to society while addressing the challenges. Guided by our AI Principles, Google designs AI systems with robust security measures and strong safety guardrails, and we continuously test the security and safety of our models to improve them. 

Our policy guidelines and prohibited use policies prioritize safety and responsible use of Google's generative AI tools. Google's policy development process includes identifying emerging trends, thinking end-to-end, and designing for safety. We continuously enhance safeguards in our products to offer scaled protections to users across the globe.  

At Google, we leverage threat intelligence to disrupt adversary operations. We investigate abuse of our products, services, users, and platforms, including malicious cyber activities by government-backed threat actors, and work with law enforcement when appropriate. Moreover, our learnings from countering malicious activities are fed back into our product development to improve safety and security for our AI models. These changes, which can be made to both our classifiers and at the model level, are essential to maintaining agility in our defenses and preventing further misuse.

Google DeepMind also develops threat models for generative AI to identify potential vulnerabilities and creates new evaluation and training techniques to address misuse. In conjunction with this research, Google DeepMind has shared how they're actively deploying defenses in AI systems, along with measurement and monitoring tools, including a robust evaluation framework that can automatically red team an AI vulnerability to indirect prompt injection attacks. 

Our AI development and Trust & Safety teams also work closely with our threat intelligence, security, and modelling teams to stem misuse.

The potential of AI, especially generative AI, is immense. As innovation moves forward, the industry needs security standards for building and deploying AI responsibly. That's why we introduced the Secure AI Framework (SAIF), a conceptual framework to secure AI systems. We've shared a comprehensive toolkit for developers with resources and guidance for designing, building, and evaluating AI models responsibly. We've also shared best practices for implementing safeguards, evaluating model safety, red teaming to test and secure AI systems, and our comprehensive prompt injection approach.

Working closely with industry partners is crucial to building stronger protections for all of our users. To that end, we're fortunate to have strong collaborative partnerships with security experts via the Coalition for Secure AI (CoSAI) and numerous researchers. We appreciate the work of these researchers and others in the community to help us red team and refine our defenses.

Google also continuously invests in AI research, helping to ensure AI is built responsibly, and that we're leveraging its potential to automatically find risks. Last year, we introduced Big Sleep, an AI agent developed by Google DeepMind and Google Project Zero, that actively searches and finds unknown security vulnerabilities in software. Big Sleep has since found its first real-world security vulnerability and assisted in finding a vulnerability that was imminently going to be used by threat actors, which GTIG was able to cut off beforehand. We're also experimenting with AI to not only find vulnerabilities, but also patch them. We recently introduced CodeMender, an experimental AI-powered agent using the advanced reasoning capabilities of our Gemini models to automatically fix critical code vulnerabilities.

About the Authors

Google Threat Intelligence Group focuses on identifying, analyzing, mitigating, and eliminating entire classes of cyber threats against Alphabet, our users, and our customers. Our work includes countering threats from government-backed actors, targeted zero-day exploits, coordinated IO, and serious cyber crime networks. We apply our intelligence to improve Google's defenses and protect our users and customers.

Appendix

MITRE ATLAS

MITRE ATT&CK