Risks

R001. Prompt and context injection

R002. Inadvertent data disclosure and overly permissive configurations

R003. Uncontrolled changes reaching production

R004. Supply chain risks

R005. Uncontrolled external access

R006. Hallucinations and incorrect code

R007. Scope creep

Deterministic compared to non-deterministic mitigations

The control framework

Pillar 1: Author-time controls (pre- and post-generation of code)

Pillar 2: Build-time controls (in the pipeline)

Putting the framework into practice on AWS

Kiro: Structured AI development

AWS CodeBuild and AWS CodePipeline: Pipeline controls

Conclusion

Written by: Kelli Vanderlee, Stuart Carrera

For years, the cybersecurity industry's understanding of software supply chain compromise has been anchored by a few watershed events, including Russian cyber espionage actor ICE RELIC’s (formerly known as APT29) 2020 compromise of SolarWinds and North Korean cyber espionage actor UNC4736's 2023 compromise of 3CX. However, Google Threat Intelligence Group (GTIG) has been tracking growth in threat activity targeting open source software repositories to conduct supply chain compromises over the past several years. A series of large scale open source software supply chain compromise campaigns in 2025 and the first half of 2026 underscore how important it is that organizations implement defensive strategies that directly address this threat vector. 

In this blog post, GTIG and Mandiant discuss trends we have observed in threat actor use of software supply chain compromise, and provide mitigation and hardening recommendations that incorporate insights we have developed as a result of supporting customers through recent campaigns in which threat actors manipulated open source packages. 

Open Source Supply Chain Compromise Grows in Volume and Impact in 2025 and Early 2026

The majority of the most impactful and far-reaching supply chain compromise incidents that GTIG tracked in 2025 and early 2026 involved the compromise of code repositories, software dependencies and developer tools (T1195.001). Open source supply chain compromises offer attackers the same efficiency, scale, and initial stealth as traditional supply chain compromises, but typically require significantly less planning and resources to execute. However, open source supply chain compromises are also noisy once enabled; malicious open source packages are often discovered and publicized much more quickly than traditional supply chain compromises. 

GTIG assesses with high confidence that the growth in very large-scale, open-source supply chain compromise campaigns, including use of worms and iterative compromises in 2025 and early 2026, represent a significant expansion in use of this tactic compared to prior years. We anticipate that threat actors will emulate the tactics of these campaigns and contribute to growth in open-source supply chain compromise through the rest of 2026 and years to come. GTIG identified several notable supply chain compromises in 2025 and early 2026 that we believe exemplify this trend of exceptionally large campaigns, as measured by size and/or impact (Figure 1). 

For example from February to May 2026, UNC6780 (aka "TeamPCP") conducted extensive open source supply chain compromises targeting ecosystems like PyPI, npm, and Docker Hub. Initial infection vectors varied across incidents, and included abuse of the pull_request_target GitHub Actions trigger to obtain base repository secrets and write permissions. The threat actor typically used compromised packages to deploy credential stealers, including SANDCLOCK, to obtain high value secrets. In incident response engagements, we observed UNC6780 attempting to pivot from compromised artificial intelligence (AI) software to broader network environments. UNC6780 has monetized stolen credentials through either direct sale of the stolen data, or through partnerships with ransomware and data theft extortion groups. 

In March 2026, GTIG observed the introduction of a malicious dependency in the legitimate axios package. GTIG analysis and the maintainer's post mortem indicate that the maintainer account was compromised via social engineering and used to publish the updated versions. We identified the malicious dependency as a dropper that deploys the WAVESHAPER.V2 backdoor, and attributes the activity to North Korean actor MIDNIGHT NEPTUNE (formerly known as UNC1069). While the malicious versions of axios were removed from the npm registry within three hours of their release, the scope of the compromise is estimated to be broad, as the package has over 100 million weekly downloads. GTIG supported customers in at least 15 industry verticals and 13 different countries affected by this incident. Further, axios is also a dependency for tens of thousands of other packages, and open sources reported that the malicious axios update had spread to several of these.

AI Likely to Accelerate Open Source Supply Chain Compromises

GTIG anticipates AI will accelerate the growth of open source software supply chain compromise. Integration of AI into open source software development practices, including "vibe coding," increases attacker opportunities both to manipulate AI functionalities and to take advantage of AI to speed and scale their own operational planning. Open sources have documented multiple instances of threat actors planting malicious resources on open source AI communities and inserting malicious code into open source Model Context Protocol (MCP) packages. MCP is a standardized protocol for AI to interact with tools and data. Malicious packages have also tricked AI coding agents, which have unwittingly incorporated them into projects. North Korean threat actors reportedly uploaded malicious cryptocurrency-themed packages, and subsequently an AI coding agent co-authored a commit integrating one of the malicious packages as a dependency to a legitimate cryptocurrency trading project. 

Thousands of Malicious Open Source Packages Detected

Corroborating GTIG's findings, statistics compiled by the Open Source Security Foundation (OpenSSF), a cross-industry, non-profit collaboration under the Linux Foundation, indicate that the number of malicious open source software packages identified increased exponentially, or 1,444% from 2024 to 2025 (Figure 2).

Traditional Supply Chain Compromise Remains Rare

In contrast to what we observed in the open source ecosystem, GTIG assesses with high confidence that traditional software supply chain compromise, the manipulation of source code or update/distribution mechanisms (T1195.002), remains rare. The handful of identified cases in 2025 and early 2026 were predominantly cyber espionage incidents with intentionally limited targeting scopes. 

In the most significant case, North Korean threat actor UNC4899 reportedly used social engineering to compromise a developer's machine at a web3 organization. The threat actor used this access to inject malicious code into the frontend systems, specifically impacting smart contract functionality to alter transactions initiated by a third party organization that utilized the multi-signature wallet with the targeted organization. This compromise was tailored to a single victim, but did not directly touch the targeted organization's infrastructure. The compromise ultimately led to a cryptocurrency theft of assets with an estimated value of $1.4B USD.

Other examples include the compromise of hosting infrastructure serving updates of Notepad++ from June to December 2025, activity GTIG attributes to UNC6688. GTIG observed organizations in South Korea and France affected by this activity.  GTIG also tracked the early 2026 compromise of DAEMON Tools installers. During this campaign, UNC6863 deployed SLICKDEMON to perform broad-spectrum reconnaissance and filter for targets of strategic interest. Following this profiling stage, the group selectively delivered the shellcoded loader BADFALL to facilitate hands-on-keyboard activity and bridge the deployment of the advanced QUIC RAT. The campaign targeted Russia, Brazil, and Turkey, with follow-on exploitation of government and scientific entities in Belarus and Thailand.

In addition to likely cyber espionage incidents, we observed suspected financially motivated compromises with broader distribution. In two separate incidents threat actors compromised underlying software used in consumer-facing websites: in one case, automotive dealership websites served ClickFix lures leading to the installation of SHADOWLADDER (aka SectopRAT), and in another, eCommerce websites were infected with web skimmers.

Mitigation Recommendations

To effectively mitigate and harden against software supply chain compromises, organizations should adopt a multi-tiered defensive strategy designed to minimize exposure and strengthen resilience against potential compromises.

Administrative Oversight and Risk Governance

• Cataloging Assets and Dependencies: Maintain a tiered, continuous inventory of all applications, third-party vendors, and services based on operational importance to detect single points of failure and security risks.

• Software Bill of Materials (SBOM): Implement an automated SBOM for all internal and third-party software packages, allowing security teams to continuously monitor and cross-reference active code inventories against newly disclosed vulnerabilities.

• Action Bill of Materials (ABOM): Maintain a dedicated ABOM to inventory every third-party pipeline vendor and development utility in use, linking it to your container image inventory to track exactly which external actions are building your production images.

• Software Development Lifecycle (SDLC) Threat Modeling and Attack Chain Mapping (Wiz SITF): Align your software supply chain risk management with capabilities such as the Wiz SDLC Infrastructure Threat Framework (SITF) to transition from treating security as a checklist of isolated controls to a holistic threat model. With this freely available framework, organizations can map recent incidents, threat actor campaigns, and red team exercises directly to Wiz SITF Reference IDs indexing each risk to its specific lifecycle stage: Version Control Systems (VCS), continuous integration and continuous delivery (CI/CD) pipelines, package registries, or production infrastructure. This methodology allows security teams to model complex "attack chains" where minor, isolated weaknesses (e.g., a lockfile bypass combined with an overprivileged pipeline token) are chained together by sophisticated threat actors to execute critical, high-impact breaches

• Active Risk Monitoring:  Maintain a dedicated supply chain risk register and a centralized remediation tracker to systematically group development lifecycle (SDLC) threats into clear operational domains: Governance, Identity, Pipeline Logic, and Supply Chain Hygiene. If using Wiz SITF, each vulnerability must be mapped to its exact pipeline stage with a unique Wiz SITF Reference ID. Instead of treating vulnerabilities as isolated bugs, prioritize the blocking of complex "attack chains" (such as a leaked token combined with missing branch protections and overprivileged OIDC trust) that pose the highest breach risk. Ensure each logged item has a designated owner, a targeted completion date, and clear tracking of technical dependencies.

• Standardized Configuration & Change Control: Form a Change Advisory Board (CAB) to manage the rollout of all enterprise software and hardware. Ensure every modification includes a pre-deployment risk review, post-deployment monitoring, and a verified plan for recovery or backout.

• Staff Security Education: Deploy ongoing training initiatives centered on supply chain hazards, social engineering techniques, and internal procedures for reporting incidents.

• Node.js (npm/pnpm): Enforce cooldown controls by using the minimumReleaseAge configuration. Setting this value to at least 24 hours (1440 minutes) ensures that freshly published, potentially poisoned packages are quarantined until the broader security community has had time to identify and remove them. Ensure that older, unsupported package manager versions (such as legacy Yarn or pnpm versions) are modernized, as they will silently ignore these cooldown boundaries.

• Python (pip): Ensure that Python project environments do not pull dependencies directly from the public PyPI registry, which bypasses internal release-age policies and gating controls. All configurations must specify a secure, vetted private --index-url in their configuration files to ensure consistent quarantine and vetting of upstream packages.

Vendor Lifecycle Management

• Vendor Security Vetting: Conduct rigorous due diligence prior to procurement by assessing third-party security frameworks against industry standards such as ISO 27001 or SOC 2.

• Cybersecurity Provisions in Contracts: Integrate specific security mandates into vendor agreements, including strict timelines for incident notification, persistent audit rights, and clear liability terms.

• Hardware Provenance and Verification: Use supply chain tracing to confirm the integrity of components, establish methods for detecting counterfeit items, and secure the logistics of repairs and replacements.

Security Architecture and Engineering Controls

Identity and Access Management

• Automated System and Workload Identities: Transition third-party integrations and build-system processes away from static, long-lived administrative Personal Access Tokens (PATs). Instead, mandate the use of dedicated GitHub Apps or short-lived system tokens via federated OpenID Connect (OIDC) for automated machine integrations. This ensures that credentials used by system-to-system workflows expire in a matter of minutes, neutralizing the risk of a persistent compromise if an automation pipeline is breached.

• Developer and User Identity Controls (command-line interface (CLI) and Repository Access): Enforce strict access control boundaries for programmatic developer sessions. Because Okta-linked SAML SSO is only capable of verifying identity during the initial creation or authorization of personal tokens and keys, continuous session state cannot be challenged over programmatic CLI connections. Therefore, session security must be enforced through credential expiration and hardware-backed controls.

• Enforce Strict Token Expiration: Strictly limit the allowable lifespan of all personal access tokens (PATs) and programmatic application programming interface (API) keys to a minimum threshold (e.g.a maximum 7-day limit). This guarantees that credentials expire regularly, forcing developers to re-authenticate through the primary SSO gateway.

• Consider Restricting Personal Access Tokens to Neutralize Git-over-HTTPS & Mandate FIDO2 Secure Shell (SSH): To protect developer environments against credential theft, organizations should consider restricting Personal Access Tokens (PATs) globally across GitHub Enterprise Cloud. Because GHEC has no direct protocol-disable switch, administrators should consider disabling classic PATs and enforcing short token lifespans to effectively block unauthorized programmatic HTTPS connections. This protocol containment helps encourage developers to shift entirely to SSH authentication. To secure this transport layer, consider mandating the use of hardware-backed FIDO2 security keys to cryptographically verify physical token possession for all command-line repository actions.

• Isolated CI/CD Execution: Utilize ephemeral runners for build pipelines that are purged immediately after completing a single task. This prevents malicious actors from maintaining a persistent presence between different build phases.

• Workflow Trigger Governance (pull_request_target): Strictly limit and secure the use of highly privileged triggers such as pull_request_target in automated environments. Multiple prominent supply chain campaigns have actively exploited vulnerable workflows using this trigger as their initial entry vector.

Infrastructure Protection

• Zero Trust and Least Privilege: Maintain rigorous control over managed service providers (MSPs) and third-party vendors by enforcing role-based access control (RBAC), multifactor authentication (MFA), and frequent audits of access rights.

• Network Micro-Segmentation: Segregate vital hardware and software from the rest of the enterprise network. Use allow-list-only firewall rules to block unauthorized outbound traffic and disrupt command-and-control (C2) activities.

Secure Development Ecosystems

• Pipeline and Sandbox Isolation: Ensure that testing environments, CI/CD pipelines, and informal scripting sandboxes are physically or logically isolated from production assets.

• Artifact Management: To secure the supply chain, organizations can integrate Google's Assured Open Source Software into their internal workflows to defend against dependency confusion and malicious hijacking. This process provides "provenance" cryptographically signed evidence that the code has not been tampered with and originates from a verified source thereby establishing a higher level of trust for third-party dependencies.

• Quarantine Gates: Require all binaries, packages, and container images to be hosted in monitored internal repositories. To defend against zero-day dependency hijackings, implement localized "quarantine gates" by enforcing cooling windows on newly published third-party assets.

• Lifecycle Script Sandboxing (ignore-scripts): Mitigate the critical threat of arbitrary code execution by disabling the automatic running of package install scripts. Attackers commonly hijack dependencies and add malicious post-installation execution scripts to steal credentials from developer environments and runners during routine installs. Organizations should mandate ignore-scripts=true in their repository-level .npmrc files and configure native allowlists, such as pnpm's onlyBuiltDependencies, to restrict execution exclusively to verified, essential tools.

• Software Composition Analysis (SCA) with Google OSV-Scanner: Integrate Google's open source OSV-Scanner tool into CI/CD build pipelines to continuously scan project dependencies for known security flaws. This tool provides an officially supported frontend to the OSV.dev database that maps a project's list of dependencies with the specific vulnerabilities affecting them.

• High-Fidelity Vulnerability Detection: Unlike traditional scanners that rely on imprecise name matching, the OSV schema stores vulnerability data in a machine-readable format that maps unambiguously onto version ranges and commit hashes. This results in fewer false positives and produces highly actionable remediation notifications, significantly reducing development team triage overhead.

• Authoritative & Collaborative Threat Intel: The underlying OSV.dev database aggregates high-quality threat intelligence from authoritative open sources, allowing the broader developer community to suggest continuous improvements. Utilizing OSV-Scanner helps developers identify impactful third-party open source vulnerabilities in their applications and focus remediation on genuine risks.

• Hardware-Backed Key Protection: Secure code-signing certificates using Hardware Security Modules (HSMs) or vaulting solutions. Monitor public transparency ledgers and logs to detect any unauthorized certificate activity.

• Hardened Distribution Points: Audit and lock down software delivery channels, such as Content Delivery Network (CDN) endpoints and FTP servers, to ensure legitimate binaries cannot be replaced by compromised payloads.

• Audit NPM Package Maintainer Accounts for Stale or Expired Recovery Email Domains: Expired maintainer email domains are a critical risk because attackers can purchase them to intercept password reset emails, take over the package registry account, and publish malicious code to downstream users. To identify vulnerable packages, organizations can perform the following:

• Deploy automated scanning tools to audit the entire dependency tree and verify the domain name system (DNS) resolution and registration status of all maintainer email domains.

• For defense-in-depth, pipelines must disable package execution scripts and employ cold periods.

• Use by default ephemeral, single-use runners to prevent compromised packages from accessing persistent build environments. 

• Isolate runners in a restricted network segment with strict egress filtering blocks any unauthorized connection to external domains even if an active exploit is triggered.

Integration with Native Ecosystem Guardrails    

• These organization-controlled quarantine policies must operate in conjunction with native platform-level security updates to achieve a Defense-in-Depth posture. Relying solely on client-side configurations or automated update tools in isolation creates single points of failure. The following native platform controls must be orchestrated alongside standard controls:

• Dependabot Native Cooldowns (July 2026): Dependabot now enforces a default three-day cooldown on version updates to allow for the public discovery of upstream compromises (such as the historical chalk and debug hijackings) before automated Pull Requests are generated].

• PyPI Server-Side Immutability (July 2026)]: PyPI now natively rejects new file uploads to any release older than 14 days. This prevents adversaries possessing compromised tokens from retroactively poisoning legacy, pinned dependencies (as observed in the LiteLLM and Telnyx compromises) .

• npm v12 Install-Time Defaults (July 2026): npm v12 disables all lifecycle scripts by default (allowScripts: off) , replacing manual, workflow-level ignore flags with explicit, commit-verified package allow-lists 

By explicitly aligning baseline configurations including .npmrc and pip.conf registry pinning, immutable installation protocols via npm ci, and runner isolation with these native platform-level guardrails, while committing to the continuous evaluation and adoption of new upstream security features as they are released, the organization establishes a resilient, multi-layered security boundary across the entire software supply chain

Continuous Verification, Monitoring, and Response

Automated Ingestion and Validation

• Automate SBOM Management: Implement a Software Bill of Materials (SBOM) for all third-party and internal software. This enables continuous monitoring for emerging vulnerabilities like Log4j through automated cross-referencing. Automate and scale this process by feeding SBOMs into central vulnerability management platforms that continuously cross-reference deployed inventory against newly disclosed exploits.

• Security Analysis Integration: Incorporate automated dynamic application security testing (DAST) and static application security testing (SAST) tools within development pipelines to identify and block compromised third-party code before it is compiled.

• Verification of Cryptographic Integrity: Prior to installing updates, use automated systems to validate digital signatures and hashes against vendor-provided specifications.

• Implement autonomous security verification: Organizations should look to integrate advanced security workflows directly into their CI/CD pipelines. These systems can behaviorally evaluate threats by executing simulations in isolated sandboxes, cross-reference those flags with cloud context to determine a flaw's actual reach, and automatically generate tested code patches to rapidly remediate verified risks at scale.

Proactive Threat Hunting and Monitoring

• Egress and Proxy Analysis: Establish network traffic baselines to identify suspicious egress flows to external repositories or unrecognized Internet Protocol (IP) addresses.

• Comprehensive Endpoint Security: Utilize endpoint detection and response (EDR) tools across infrastructure and developer workstations to detect post-execution malicious activities from supply chain compromises.

• Log Aggregation and Alerting: Unified log management should alert on the following anomalies:

• Development Systems: Watch for unauthorized code changes, build parameter adjustments, or irregular user activity.

• CI/CD Integrity: Alert on unauthorized workflow modifications or anomalous triggers (e.g., repository_dispatch) that bypass standard code-review gates.

• Injection Detection: Monitor logs for shell-escape characters or command-substitution patterns within untrusted input variables.

• Credential Misuse: Track authentication hits on long-lived static keys from unrecognized IP addresses or regions.

• Physical Assets: Record all firmware modifications, including installation status and source information.

Incident Response Strategies

• Specific Supply Chain Playbooks: Perform tabletop exercises and document response plans for:

  • Upstream Package Takeover: Maintainer account takeover (ATO) on public registries leading to direct runtime application code manipulation

  • Dependency Confusion Exploits: Malicious registration of lapsed administrative recovery domains or unscoped internal namespaces on public registries to hijack local developer and build runner installations.

  • Automated Pipeline Harvesting: Pipeline poisoning of CI/CD environments via runner exploitation to harvest credentials and perform unauthorized package publication.

  • Developer Workstation & IDE Compromise: Targeted social engineering, malicious IDE extensions, or typosquatted local dependencies designed to exfiltrate private cryptographic keys, API tokens, and local session credentials.

• Operational Re-evaluation: Create processes for immediate vendor re-mapping and security re-assessment during industry-wide security events.

Recommendations for mitigation strategies are also available publicly via:

• Google's Supply-chain Levels for Software Artifacts (SLSA) 

• Cybersecurity Supply Chain Risk Management Practices for Systems and Organizations (NIST SP 800-161 Rev. 1) from the US National Institute of Standards and Technology (NIST)

• Securing the Software Supply Chain: Recommended Practices Guide for Developers from the US National Security Agency (NSA)

• WIZ SDLC Infrastructure Threat Framework (SITF) SDLC Infrastructure Threat Framework

Acknowledgements

This analysis would not have been possible without the assistance of Matthew McWhirt and Michael Veal.

One DPRK–linked group behind multiple NPM compromises

A smaller campaign that foreshadowed later activity

How attacker tradecraft is shifting

From package‑level attacks to fragment‑level attacks

Long-horizon campaigns that invest in trust

Decoupling the package from its behavior

From basic obfuscation to real cryptography

Payloads that avoid detonating in sandboxes

How generative AI is reshaping both attacks and defenses

How AWS is responding

Update (July 30): A table listing the new names of select prominent threat actors was appended to this post. 

Introduction 

Today, Google Threat Intelligence Group (GTIG) will begin rolling out a unified naming schema for tracking threat actors. This new naming taxonomy represents an effort to standardize tracking across platforms and public reporting.

Why are we Adopting a Different Naming System?

Historically, Mandiant and Google’s Threat Analysis Group (TAG) maintained distinct tracking systems, relying on parallel naming schemas that grew independently over time. The creation of GTIG has necessitated a new, fused tracking system, and a new naming system. Thinking to the future, GTIG’s new system will rely on cryptonyms. Relying on sequential numbers or disparate identifiers (e.g. APT1) fails to provide defenders the critical context needed to operate quickly. Threat tracking shouldn’t be an exercise in memorization, but rather one of intuition. The new naming convention aligns with industry standard threat actor naming systems. 

Our New Schema

Our new schema utilizes a cryptonym-based approach, employing memorable two-word combinations for each distinct threat actor:

• The first word is a unique and memorable term chosen to represent the specific actor, particularly names that may have been used in prior public reporting. If no previously used term exists, this word is randomly generated to remove bias, then vetted by our analysts.

• The second word categorizes threat clusters by motivation, attribution, or activity type based on which category we consider to be most important for defense and response strategies.

The table below provides a sample of how threat actor categories will map to the second word in each cryptonym:

Table 1: Examples of Google’s new threat actor naming system categories

We know there are many threat actor tracking schemas in the industry, so we are intentionally seeking to keep this system as simple as possible to streamline operations and facilitate mapping to other naming taxonomies. However, a significant caveat remains: because no two organizations have the exact same visibility into the threat landscape, direct, apples-to-apples comparisons between threat actors are rarely possible. Transitioning to a convention that is simpler to follow and remember is a practical step toward managing a highly intricate tracking problem. 

A Work in Progress

We have initially prioritized renaming several dozen of the most active groups, and will continue this process on a rolling basis. Previous names will remain indexed and searchable in the Google Threat Intelligence (GTI) platform, with MITRE ATT&CK mappings and other vendor aliases preserved, see Figure 1. 

We will continue to use UNC, or “uncategorized” designations for threat clusters that are still in the early stages of investigation, as described here.

Selection of Re-Named Threat Actors

Table 2: Selection of Re-named Threat Actors

Written by: Jules Czarniak

Introduction 

As highlighted in the Mandiant M-Trends 2026 report, the mean time-to-exploit (TTE) has dropped to -7 days, meaning vulnerabilities are often exploited a week before a patch even exists. 

To keep pace, many security teams are exploring how to integrate large language model (LLM) agents into their codebases, development environments and continuous integration and continuous delivery (CI/CD) pipelines for automated vulnerability discovery and remediation. However, deploying privileged artificial intelligence (AI) agents without mature integration processes introduces new architectural risks. 

In response to customer inquiries about how to safely integrate AI capabilities into vulnerability management workflows, this blog provides actionable guidance from Mandiant Consulting about how to establish operational guardrails for AI assisted vulnerability management, including several detailed scenarios. What each of these examples show is that security teams can accelerate workflows with AI while also upholding the structural integrity of their environments. We suggest that combining AI capabilities with deterministic controls and human intelligence in strategic ways maximizes benefits and reduces risk. 

Establish Operational Guardrails to Safely Deploy AI Agents

To safely adopt advanced AI capabilities without introducing unpredictable failures into deployment pipelines, organizations should ground their approach in established industry standards. While guidelines like the NIST AI Risk Management Framework (RMF) and the OWASP Top 10 for LLMs provide comprehensive baselines for identifying risks, operationalizing these controls requires a structural blueprint.

Frameworks like Google’s Secure AI Framework (SAIF) and Google’s approach to secure AI Agents provide a practical path forward, demanding that organizations extend existing deterministic controls directly into the AI execution environment. When deploying AI agents, security teams should navigate specific operational and structural risks:

• Pre-agent data security and Defense-in-Depth: Agents should not be able to access personally identifiable information (PII), protected health information (PHI), or other sensitive data. Organizations should enforce data security before the prompt reaches the model. This includes strictly using non-production environments populated with synthetic data for testing. For production, security teams should deploy a hybrid defense-in-depth model. This includes Layer 1 deterministic policy engines acting as chokepoints, alongside Layer 2 reasoning-based defenses like specialized guard models (such as Model Armor or similar provider-agnostic guardrails) to filter out sensitive data and block malicious prompt injections before they reach the agent layer. Crucially for vulnerability discovery, security teams should treat the codebase itself as an untrusted input. Threat actors can embed indirect prompt injections within source code comments or third-party dependencies (e.g., hidden instructions telling the agent to ignore vulnerabilities or exfiltrate environment variables), making input sanitation a requirement even for internal scanning.

• Cloud provider limitations and zero data retention (ZDR): Many cloud and LLM providers block or throttle automated offensive security probing by default to prevent abuse. Organizations should establish clear rules of engagement and authorized testing agreements to navigate acceptable use policies. Furthermore, organizations should enforce strict zero data retention (ZDR) agreements with their LLM providers to guarantee that proprietary code and discovered vulnerabilities are never used to train external models.

• Workload isolation: Agent workloads should execute in strictly isolated, unprivileged containers with dynamically limited privileges. By relying on robust sandboxing to prevent privilege escalation, if an agent hallucinates a destructive command or is hijacked via prompt injection, the blast radius remains contained.

• Red Teaming: Before deploying autonomous vulnerability scanners that can dynamically spin up sandboxes and execute code, organizations should subject the AI agents themselves to human-led red teaming as part of comprehensive assurance efforts. This validates the agent's resilience against jailbreaks, recursive logic loops, and complex prompt injections, ensuring the security tooling does not become the attack vector.

• Least-Privileged Machine Identities and Human Controllers: While workloads should be isolated, agents inherently require privileges to generate pull requests and commit code. Security teams should ensure these agents operate under distinct, strictly scoped machine identities that tie back to human controllers to ensure accountability and user consent. Organizations should use short-lived, just-in-time (JIT) tokens bound exclusively to the specific repository and branch under review. This enforces the principle of limited agent powers and ensures that even if an agent’s container is compromised via prompt injection, the threat actor cannot pivot to modify adjacent enterprise codebases.

• Supply chain resilience for skills: As developers augment AI with third-party skills and model context protocol (MCP) servers, security teams should treat these integrations as untrusted supply chain components. MCP plugins introduce the risk of supply chain poisoning, where a previously benign integration is silently updated with malicious dependencies. Additionally, security teams should evaluate the underlying agent orchestration frameworks themselves (e.g., LangChain, AutoGen) for inherent vulnerabilities, such as session memory poisoning or recursive loop hijacking.

• Toxic flow analysis (TFA) and Observable Actions: The objective of TFA is to monitor data paths at runtime, ensuring agents do not exfiltrate sensitive internal context to unvetted external endpoints. Agent actions, inputs, reasoning, and outputs must be fully observable and transparently logged. While implementing dynamic taint tracking for LLMs remains a complex architectural challenge, organizations should clearly separate this runtime observability from static supply chain controls. Integrating threat intelligence to hash and vet incoming agent tools provides a necessary baseline for verifying integrity before deployment. However, because static controls cannot address behavior post-deployment, mitigating data exfiltration ultimately requires active runtime monitoring and secure, centralized logging to trace and restrict the actual flow of data.

By operationalizing these tools within frameworks that demand verifiable integrity and structural resilience, organizations can safely bridge the gap between AI velocity and enterprise defense.

The need for human-led threat modeling

While LLMs excel at identifying syntax patterns, source code itself rarely contains the full picture of unwritten business intent. Some organizations attempt to solve this by connecting LLM agents to internal wikis, design documents, and issue trackers using retrieval-augmented generation (RAG).

While RAG gives the model access to external business context, it is not a perfect fix. Corporate documentation is frequently stale, contradictory, or incomplete. An AI agent might retrieve an outdated architecture diagram and confidently hallucinate a secure path that no longer exists in production. Because LLM agents struggle to resolve conflicting, undocumented human assumptions, human-led threat modeling remains a critical security control across both legacy applications and modern agent workflows.

Security teams should apply threat modeling during both the pre-build system design phase to establish a secure foundation, and during post-build architecture reviews. While an AI agent might successfully identify a poorly configured internal endpoint locally, a human threat modeler asks the structural question: why does that microservice possess broad database read permissions in the first place? 

Identifying architectural vulnerabilities requires reasoning about business risk, data sensitivity, and operational constraints. To structure this process, organizations can use industry frameworks like PASTA (Process for Attack Simulation and Threat Analysis) or service offerings like the Mandiant Threat Modeling Security Service to map trust boundaries, uncover structural design flaws, and prioritize compensating controls. Securing fundamental architecture through human oversight is a necessary component when relying on automated agents to find bugs in a poorly designed system.

Once these AI agents are safely sandboxed, as guided by SAIF, and the architecture is verified through threat modeling, organizations can typically apply them to two different problem spaces: Enterprise Vulnerability Management (to assist in managing the volume of known CVEs in commercial off-the-shelf (COTS) software and infrastructure) and Product Security (to identify vulnerabilities in 1st-party (1P) code).

Track 1: Enterprise Vulnerability Management

Foundational security and discovery 

While the second track of this post explores how AI agents can uncover complex zero-days in custom code, organizations should manage the scale of enterprise infrastructure in tandem with these AI deployments. Even as new AI capabilities dominate headlines, organizations should still address foundational security challenges, such as secrets sprawl, unmanaged service accounts, missing FIDO2 MFA, and legacy VPN concentrators. Although vulnerability exploitation was the primary initial infection vector in intrusions Mandiant investigated last year, threat actors consistently rely on missing foundational controls and unpatched edge devices to secure and escalate their foothold after exploiting a vulnerability.

Furthermore, AI cannot replace foundational visibility. As security teams deploy AI agents, they should simultaneously close these tactical entry points by maximizing dynamic discovery capabilities like External Attack Surface Management (EASM), Cloud Security Posture Management (CSPM), and Continuous Threat Exposure Management (CTEM). In hybrid and cloud environments, tools like Wiz can be used to map this initial footprint.

Risk-based vulnerability management 

Vulnerability management teams are already overwhelmed by the current volume of findings generated by traditional scanners. As organizations scale dynamic discovery tools, such as EASM, CSPM and CTEM, alongside automated AI agents, this influx of findings will compound the problem. To manage this influx, telemetry from these diverse discovery methods must first be normalized and deduplicated. This normalized data serves two purposes: it feeds directly into the risk engine, and it acts as a live overlay to correct stale records in the configuration management database (CMDB). By evaluating the deduplicated vulnerabilities alongside this newly updated asset context and frontline threat intelligence, the RBVM engine calculates a custom risk score that allows security teams to dynamically prioritize remediation.

A mature RBVM methodology calculates a customized risk score on a 0 to 100 scale using a weighted average. A sample formula for calculating this risk-based score is:

Final Score = (W_1 * S_vuln) + (W_2 * S_asset) + (W_3 * S_threat)

The variables and weights (W) are customized to the organization's risk appetite (for example, 0.20 for vulnerability, 0.40 for asset, and 0.40 for threat, summing to 1.0), while the underlying variables (S) are scored on a 0 to 100 scale and defined as follows:

• Vulnerability severity (S_vuln): The inherent technical severity of the flaw. This is calculated by taking the CVSS Base Score (which natively accounts for confidentiality, integrity, and availability impact) and multiplying it by 10.

• Asset context (S_asset): A combined metric of exposure and data sensitivity. Scores range from 100 for internet-facing assets holding customer data, down to 25 for internal-only assets with no sensitive data. To translate this impact into monetary terms for non-technical stakeholders, organizations can incorporate Factor Analysis of Information Risk (FAIR) principles into this metric. However, this approach requires highly accurate, continuously updated financial data that many enterprises struggle to maintain at scale.

• Threat context (S_threat): The real-world urgency of the vulnerability. Scores range from 100 if actively exploited by threat actors relevant to the organization's profile, 75 if a proof-of-concept exists or if it is a vulnerability class easily exploited by autonomous AI agents, down to 25 if the exploit is theoretical and highly complex. Organizations should also map the Exploit Prediction Scoring System (EPSS) probability percentage directly into this variable. This allows the threat score to automatically scale up or down as real-world exploitation telemetry shifts, aligning static vulnerability data with active threat intelligence.

An asset's customized risk score should directly influence internal remediation service-level agreements (SLAs), unless external compliance-driven mandates, such as CISA Binding Operational Directives (BODs), or relevant equivalents, override internal prioritization. A risk-driven and threat-intelligence-driven vulnerability prioritization methodology will help organizations focus resources on managing and mitigating the most critical security vulnerabilities first. This is an area where LLMs can support the vulnerability management process, particularly by helping teams synthesize unstructured threat intelligence to surface relevant risk contexts more efficiently. Enforcing strict SLOs for patching, while requiring formal risk acceptance documentation for any patching exceptions, will help reduce the number of vulnerabilities available to threat actors and increase the visibility of outstanding risks across the organization. Furthermore, organizations should integrate RBVM data directly into their security orchestration, automation, and response (SOAR) platforms for automated alert enrichment.

Containment and Observability

Modern architecture blueprints must prioritize attack surface reduction under the assumption that vulnerabilities will inevitably be exploited. Moving away from traditional perimeter defenses, organizations should align with zero trust principles, ensuring that security boundaries are established around every asset, workload, and identity.

A component of this alignment is the implementation of strong authentication principles. Organizations should eliminate implicit trust by enforcing continuous, context-aware authentication and authorization. Utilizing Zero Trust Network Access (ZTNA) solutions, such as Identity-Aware Proxies (IAP), shields critical management interfaces (e.g., SSH, RDP) and internal systems from direct internet exposure, granting access only to verified identities and compliant devices.

For public-facing applications and APIs, attack surface reduction involves deploying Layer 7 inspection at the load balancer or API gateway level. This hardening layer enforces strict schema validation, intercepting and neutralizing malformed inbound traffic and potential exploits before they can interact with internal application logic.

Securing the software supply chain is equally vital in modern blueprints, and organizations should align with frameworks like Supply-chain Levels for Software Artifacts (SLSA) across both dependency and build tracks. Security policies should mandate that third-party dependencies are routed through a centralized artifact repository equipped with automated curation services, such as Google Assured Open Source Software (OSS) or an equivalent solution, preventing untrusted code from entering the development lifecycle. Furthermore, maturing toward advanced SLSA build levels (e.g., SLSA level 3) through the implementation of isolation, ephemerality and reproducibility requirements via  ephemeral compute infrastructure for CI/CD runners reduces the likelihood of attacker persistence by ensuring environments are short-lived and automatically cycled.

To complement these pre-build controls, runtime observability should be established across all production workloads. This requires monitoring both infrastructure-level behavior and the specific runtime libraries actively executing in production, which surfaces true exploitable risk far beyond a static Software Bill of Materials. In tandem with monitoring workloads, organizations should secure how they authenticate by implementing workload identity federation. By removing static credentials and instead using short-lived tokens backed by strong cryptographic identity verification, organizations can reduce the risk of credential theft and unauthorized lateral movement.

Within the internal environment, microsegmentation should be enforced to break down flat networks into granular security zones. Routing application traffic through a Secure Access Service Edge (SASE) architecture integrates network routing directly with robust identity controls, rendering internal services completely invisible to unauthenticated users and containing threats to their initial point of entry.

Finally, automated containment and incident response within a zero trust framework must rely on deterministic, auditable tooling. Endpoint detection and response (EDR) platforms and SOAR playbooks should handle high-fidelity containment tasks through hardcoded execution logic. While AI tools accelerate triage and policy recommendation, actual execution capabilities must remain restricted to well-defined, pre-tested workflows to maintain total architectural predictability.

Track 2: Product Security & Development (1P Code)

Deterministic and probabilistic tooling

Integrating LLM agents into vulnerability management and security workflows requires recognizing the differences between deterministic and probabilistic tooling. Traditional SAST and DAST tools utilize fixed methodologies to evaluate vulnerabilities through structural code parsing or definitive runtime observations. LLMs, however, evaluate source code by processing tokens simultaneously to calculate statistical and semantic relationships, rather than tracing deterministic execution tracks.

While techniques like Chain of Thought (CoT) prompting allow models to bridge this gap by decomposing complex code paths into intermediate reasoning steps, this process remains bounded by architectural limitations. Even when a model possesses a context window large enough to ingest entire repositories, it may experience attention degradation across long inputs, often failing to correctly weight intervening validation or sanitization logic within the prompt. For example, if a variable is tainted on line 10 but sanitized on line 500, attention degradation can cause the model to lose track of the sanitization logic. Furthermore, when enterprise codebases require chunking to fit within context limits, the resulting fragmentation may cause the model to lose track of end-to-end data flows.

Consequently, probabilistic engines are effective at uncovering localized, static anomalies, such as hardcoded credentials or outdated dependencies, but frequently misjudge complex vulnerabilities split across fragmented chunks or extended context windows. Notable exceptions occur when these probabilistic models are coupled with deterministic feedback loops. For instance, when analyzing C++ memory corruption, an LLM can be equipped with a test harness to iteratively execute code and definitively prove a crash. While these dynamic validation applications are detailed in subsequent sections, the baseline limitation for static analysis across standard enterprise codebases remains: models struggle to consistently evaluate dispersed logic.

Binary and architectural oracles

Many security programs are moving toward agent workflows where an agent autonomously spins up a test environment and uses tools to execute payloads and verify its findings. This is a promising approach, but it is important to understand where it is most effective.

Agent workflows perform well against bug classes with binary and observable oracles, meaning the system provides an objective, 'crash or no crash' feedback loop. For example, if a model is hunting for memory corruption in a C++ kernel, a successful exploit is undeniable: the payload executes, and a resulting crash definitively proves the vulnerability. This explains why the industry is currently seeing a surge in AI-discovered vulnerabilities across memory-unsafe targets like web browsers and operating systems.

However, enterprise software is heavily dominated by vulnerabilities that require architectural oracles for validation. Vulnerabilities like authorization bypasses, complex business logic flaws, and indirect server-side request forgeries require an understanding of business context and cross-service trust boundaries. If an agent's payload fails to produce a clear outcome, it can't reliably distinguish whether the vulnerability is a hallucination or if it simply constructed the payload incorrectly. An agent's malformed payload might even crash an unrelated background process and cause the model to hallucinate a success and report a false confirmation. Complex enterprise architecture contains unwritten business intent that a probabilistic engine can't inherently know.

Targeted deployment and human impact

Organizations adopting LLMs for vulnerability discovery face a massive staffing challenge. LLMs can generate findings significantly faster than human engineers can triage them. If every LLM-generated alert requires manual review, security teams will quickly face burnout and/or suffer alarm fatigue.

Rather than indiscriminately pointing agents at all available codebases and risking an influx of unverified output, security teams need a selective deployment strategy. Mature programs should maintain SAST and DAST for baseline hygiene and deterministic rule enforcement, and reserve intensive agent audits for high-impact components with clear binary oracles.

Organizations can prioritize agent audits on systems where the technology's strengths align with the broader risk profile:

• Memory-unsafe codebases: Legacy or high-performance components written in memory-unsafe languages such as C, C++, or Assembly are strong candidates for LLM audits. These languages are susceptible to memory corruption flaws, such as buffer overflows and use-after-free conditions. Because these vulnerabilities trigger definitive failure states like segmentation faults, they work well with automated sandboxes where agents can compile the code with memory sanitizers and write proof-of-concept inputs. This approach is also effective for auditing the native extensions where safe languages call unsafe internal libraries, such as Python C extensions or the Java Native Interface (JNI).

• Systems highly exposed to outside content: First-party data ingestion pipelines, custom API gateways, or proprietary edge proxies. A prerequisite here is direct access to the source code, this strategy is strictly for internally developed or fully open-source codebases where the organization can inspect the logic. Because these systems directly parse untrusted internet traffic, targeting their source code for LLM-driven audits yields the highest risk-reduction ROI.

• Shared internal libraries and utilities: Core serialization/deserialization packages, common utility functions, and custom middleware wrappers (such as internal message-queue parsers) maintained in-house. Because the enterprise owns the source code for these shared building blocks, agent tools can easily hook into them within automated test harnesses to fuzz inputs and catch low-level logic or parsing bugs with high fidelity.

• Foundational security boundaries: Internally developed centralized authentication services, custom OAuth providers, and internal credential brokers. While testing complex identity boundaries generates higher logic-based noise, having full access to the source code allows teams to pair agents with deterministic checks to safely triage findings, given that the blast radius of an authentication failure justifies the human effort.

To filter the noise generated by LLMs, organizations should establish routing rules. Require the agent to generate a fully reproducible, deterministic test harness (such as a compiled binary or a Python test script) that attempts to prove the exploit. This harness must execute automatically in an isolated, monitored sandbox. If the sandbox execution fails (due to a syntax error or a failed exploit), the ticket is discarded, sparing human resources. However, organizations should enforce execution timeouts and iteration limits on these test harnesses. Without hard limits, an autonomous agent attempting to prove a vulnerability can fall into an infinite loop: writing a script, failing, rewriting, and failing again, exhausting API token budgets and compute resources against a single dead-end vulnerability, creating significant cost overruns without advancing the security review. To manage these expenses, organizations should incorporate FinOps principles to balance the compute and API costs of LLM audits against the traditional expenses of manual triage.

However, a successful execution in the sandbox does not guarantee an actionable, high-priority risk. In practice, autonomous agents frequently produce working PoCs for genuine technical flaws that are ultimately irrelevant; or warrant a lower remediation priority within the context of the system's threat model. For example, the agent might successfully exploit an unreachable dead-code path, or trigger a bug that requires administrative access to execute and yields no further escalation of privilege. Therefore, a human engineer should be assigned to review and prioritize the ticket only if the sandbox registers a successful execution, validating environmental context, reachability, and true business impact as part of the review.

This workflow reduces the volume of alerts, but it is important to understand that the security team's workload does not disappear. The engineer's primary job shifts from manually hunting for the initial vulnerability to auditing the LLM-generated proof to ensure it represents a meaningful risk rather than an unexploitable or contextually irrelevant finding. Leadership should properly staff and train teams for this new reality. Deploying LLM agents does not remove the need for skilled practitioners; it redirects their workload toward complex validation. Equally important is training teams to recognize the risk of false negatives. A hyper-focus on filtering AI-generated noise can create a false sense of security. If an exploit relies on a novel technique or a zero-day vulnerability that was not heavily weighted in the model's training data, the agent will likely scan right past it in silence. LLMs augment discovery, but they do not guarantee exhaustive coverage.

When integrating LLMs into SAST triage pipelines, human engineers should also verify the broader architectural integrity. Prompting an LLM with specific SAST warnings can induce contextual narrowing, where the agent becomes hyper-fixated on resolving a localized syntax error and misses broader architectural flaws existing in the same file. Furthermore, if the agent's mandate extends beyond discovery to automated remediation (such as writing and proposing code fixes), this human-in-the-loop validation becomes critical to ensure the LLM does not inadvertently introduce new regressions or bypass intended business logic.

Remediation and hardening

LLM-assisted code remediation

A primary goal of integrating large language models (LLMs) into the software development lifecycle is automated remediation. To achieve this, organizations are deploying these capabilities through two primary execution methods: directly within the integrated development environment (IDE) or as a centralized pipeline runner. Examples include CodeMender, although as of time of writing, it is not publicly available.

IDE-integrated method 

This method shifts remediation as far left as possible by operating as an active pair-programmer. Tools running continuous static analysis in the background of the IDE surface vulnerabilities directly to the developer via editor diagnostics like inline indicators or hover tooltips.

• Localized scope: The developer can trigger the LLM agent to analyze the localized data flow and generate a targeted patch (such as implementing parameterized SQL queries). By constraining the LLM to localized, syntax-level fixes, the scope of the change remains contained. This prevents the agent from attempting sprawling, multi-file refactors that frequently break complex architectural logic.

• Human-in-the-loop: The developer reviews the AI-generated patch before the code is committed.

• Managing false positives: Local IDE agents allow developers to manage false positives dynamically. Suppressing alerts anchored to specific line text reduces alert fatigue and preserves developer trust.

CI/CD runner method 

The runner method executes asynchronously within the CI/CD pipeline to use an LLM to review committed code and automatically propose remediation.

• Restricted execution and deterministic validation: Asking a centralized runner to automatically rewrite a complex, multi-file authorization flaw directly in the main branch introduces a high risk of breaking logic errors. To mitigate this, agents must be restricted to generating pull requests (PRs). Once a PR is generated, it must automatically execute standard regression suites alongside the deterministic test harness. By rerunning the initial PoC against the patched code, the workflow repurposes the exploit script as a validation oracle to prove the vulnerability has been remediated. A human engineer then reviews the PR to validate the architectural logic before merging.

In all cases security teams should define a clear boundary between the two methods rather than rely on a single approach. IDE agents provide immediate, syntax-level support. They catch and resolve low-complexity errors locally before developers commit code. Centralized CI/CD runners handle broader organizational baselines. They propose complex, repository-wide fixes for vulnerabilities that bypass local environments.

Post-deployment controls 

Even with human review and deterministic test harnesses, AI-generated patches can still introduce logic regressions in production. Organizations should implement strict post-deployment controls:

• Automated rollbacks: Treating LLM-generated code with the same post-deployment scrutiny as any major architectural change ensures that if an unforeseen regression traverses the CI/CD pipeline, the environment can revert to a known good state.

• Mitigating model drift: Relying on managed AI services introduces the ongoing risk of model drift. To prevent silent weight updates from breaking test harnesses, organizations need to pin specific model API versions to frozen releases. When a pinned version reaches its end-of-life, organizations will face a forced migration. Mitigating this pipeline fragility requires combining model pinning with deterministic regression suites.

• Compliance and auditability: If an AI agent automatically closes a security ticket or generates a patch in the CI/CD pipeline, organizations should maintain immutable audit logs to satisfy frameworks like SOC 2 ,PCI-DSS, FedRAMP, and CMMC. National security deployments must also account for data sovereignty requirements. This logging should record the specific model version that proposed the fix, the deterministic test results that validated it, and the human engineer who approved the merge. Furthermore, because emerging legislation like the EU AI Act emphasizes human oversight for high-risk applications, security teams should carefully evaluate how autonomous remediation workflows align with these evolving global regulatory standards.

Conclusion

Leveraging LLMs in vulnerability management is a multi-layer solution: Integrating it requires separating workflows by layer. At the enterprise infrastructure level, Risk-Based Vulnerability Management (RBVM) and exposure management are necessary to process the volume of findings and configuration drift. At the product and code security level, LLM-enabled vulnerability assessment and remediation must operate alongside foundational deterministic controls, such as SAST and DAST, to audit custom, open-source, or third-party code.

Although LLMs can help manage technical debt and accelerate vulnerability discovery, they do not replace secure-by-design principles. The fact that LLM agents are proving exceptionally capable at identifying and exploiting localized memory corruption in memory-unsafe codebases, alongside other primary vectors, should serve as a wake-up call. 

As a long-term strategy aligned with NSA guidance on Software Memory Safety, organizations need to phase memory-safe languages into new internal development. LLMs are beginning to expand what is possible here by reducing the manual labor required for code migration. Converting existing C or C++ codebases to Rust has historically been unrealistic due to the large volume of engineering hours needed. While fully automated translation is not a turn-key solution, using LLMs to assist engineers with the bulk of the conversion can make these long-term migrations operationally viable. Beyond internal efforts, organizations should use procurement requirements to incentivize vendors to reduce their reliance on memory-unsafe languages and establish secure configuration defaults over time. Bridging the gap between AI velocity and enterprise defense means building an automated pipeline to manage the current backlog, while architecting systems where entire classes of vulnerabilities and misconfigurations are eliminated by design.

Acknowledgements

This analysis would not have been possible without the assistance of Google Threat Intelligence Group (GTIG) and other broader Google teams.