Normal view

Partnering with Precision in 2026

17 December 2025 at 14:00

If 2025 proved anything, it’s that no one wins alone in cybersecurity. AI-driven threats accelerated, and environments grew more complex while enterprises pushed hard for simplicity, integrated protection and security outcomes that deliver measurable results and meaningful value.

In response, we saw our partners around the globe lean into integration, treat AI as a built-in advantage and use the strength of our ecosystem as a force multiplier. The result: What could have been a disruptive year instead became one defined by growth and learning across our partner community.

Now, those lessons are guiding how Palo Alto Networks plans to partner with even greater precision in 2026. We remain a channel-first company that’s all-in on our ecosystem and united with our partners in a shared purpose to protect our customers’ digital future. But we also intend to double down in several areas in the year ahead, and we’re asking our partners to join us in doing the same.

1. Simplifying Security Through Integration

One message from customers that came through loud and clear in 2025 is that complexity is the enemy of resilience. Many enterprises are grappling with tool sprawl – multiple consoles, disconnected policies and overlapping investments that slow down their teams when speed and agility matter most.

The partners who delivered some of the most transformative results for organizations this year were those who chose integration over complexity and collaboration over siloed tools. With a laser focus on simplifying security, they were able to help customers:

  • Consolidate fragmented point tools onto a unified security platform.
  • Align visibility across the network, cloud and security operations center (SOC), so teams can respond faster.
  • Build architectures with zero trust and AI-powered detection at the core.

We saw this simplifying-security trend through integration across our ecosystem. Partners unified cloud security and detection workflows through Cortex® Cloud™ and Cortex. Teams modernized network architectures with tighter integration across our platform. We expect this activity to only accelerate in the coming year as our cloud security offerings continue to evolve.

When we innovate together, customers gain stronger defenses and a faster time-to-value. That’s why Palo Alto Networks has invested so heavily in platformization. When you connect our capabilities across network security, cloud security and security operations (wrapping them with your consulting, delivery and managed services) customers can experience something fundamentally better. With fewer gaps and clearer signals, they can build a security posture that’s built for the speed of modern threats.

In 2026, deep integration will remain a cornerstone of how we partner with precision. We’ll continue aligning our portfolio, programs and joint engagement model, so you can build offerings that reduce complexity for customers and create stronger differentiation for your business.

2. Making AI a Built-in Advantage

At Palo Alto Networks, our approach to AI in cybersecurity is straightforward. We believe AI must be embedded, not bolted on. It has to live in the data, analytics and workflows your teams rely on every day. That’s the thinking behind Precision AI®, and it’s why we built AI capabilities into our platform’s core.

Partners who treated AI as a platform capability rather than a standalone tool delivered some of the strongest outcomes for customers in 2025. They were able to meet customers’ needs and deliver business outcomes in a single, unified approach. They helped organizations:

  • Detect and respond to threats faster with AI-assisted analytics.
  • Use automation to streamline change, investigation and response workflows.
  • Tie AI to tangible outcomes, such as reduced risk, higher productivity and a better user experience.

In 2026, we’ll double down on AI across the platform and invest in the tools, content and enablement you need to bring those capabilities to life. Our focus is on making it easier for you to build AI-powered services that are repeatable and aligned to the outcomes customers expect.

Upcoming program changes reflect that intent. We’ll promote next-generation security as a growth engine and invest in ways that strengthen partner profitability across consulting services, resale, quality delivery, technical support and managed security services.

3. Ensuring Our Ecosystem Can Be a Growth Engine for Everyone

As AI raised the bar for both attackers and defenders in 2025, the partners who leaned into platformization and outcome-driven services were the ones who helped customers stay ahead of the curve. Those successes are now shaping how we strengthen and scale the partner ecosystem in 2026.

Our ecosystem isn’t just a route to market; it’s intended to be an economic engine for everyone involved. This year, many partners grew their business by building practices around our platform and aligning their services with where customers needed the most support: strategy, implementation, optimization, ongoing operations. We saw especially strong momentum from partners’ expansions:

  • Consulting and advisory services around zero trust and AI-driven transformation.
  • Resale opportunities centered on platform consolidation and next-generation security.
  • Quality delivery and technical support that keep deployments reliable and current.
  • Managed security services that give customers 24/7 protection and expert oversight.

These achievements reflect the value exchange at the heart of our ecosystem. Palo Alto Networks invests in platformization, AI and enablement, while our partners bring delivery expertise, regional insight and service innovation. Together, we create outcomes neither of us could deliver alone.

In 2026, we plan to build on that momentum and drive even greater partner profitability. Program evolutions will focus on growth across the full lifecycle, from initial design and implementation to long-term operation and optimization. We’re also expanding collaboration with our technology alliances to build new joint offerings and solution plays that the ecosystem can take to market together.

When we combine our platform, your expertise and the capabilities of our Alliance partners, then customers gain more paths to adopt next-generation security with confidence, and you gain more opportunities to develop differentiated, high-value practices.

Keeping Customers at the Center

At the heart of every partner collaboration is the customer, of course. Everything we build, integrate and advance together starts and ends with protecting them. This year, ecosystem alignment delivered measurable impact for our customers across industries. When partners lead with integrated solutions anchored in our platform, organizations saw visible improvements:

  • Faster deployment of secure solutions.
  • Reduced complexity with unified visibility.
  • Greater confidence in defending against today’s AI-driven threats.

We saw this firsthand in joint wins across cloud security transformations, zero trust modernization and AI-assisted threat detection. When our ecosystem moves together, customers can move faster, operate more securely and achieve meaningful outcomes. Customer success is the foundation of everything we do as a partner-led organization, and it will remain our North Star in 2026.

Partnering with Precision in 2026 and Beyond

What we learned and achieved together in 2025 points us toward a clear focus for 2026 to advance ecosystem-led innovation, so we can deliver outcomes that matter most to our customers.

With that mission in mind, we will focus on the following four priorities:

  • Deeper Integration – Expanding API partnerships and strengthening interoperability across the platform.
  • Co-Innovation – Enabling partners to build solutions tailored to industry needs and use cases.
  • Empowered Enablement – Investing in learning, automation and AI capabilities that fuel differentiated, profitable services.
  • Simplified Engagement – Streamlining programs and tools, so that partnering with us is faster and more rewarding.

These priorities highlight the real strength of our ecosystem: How platformization, AI and partner expertise come together to enable what we could not build alone.

Finally, to our partners and customers, thank you. Your trust, collaboration and commitment push us to innovate boldly and continuously. As we enter the new year, I’m excited about what we’ll build together. When we align our AI-powered platform, our partner programs and your expertise in delivery, services and managed security, we can deliver something far greater than a set of solutions.

We’re a powerful team that’s not just defending against what’s next; we’re defining the future of cybersecurity. And together, we’re unstoppable.

Partners, join us in shaping the next chapter of secure, AI-powered innovations. Connect with your Channel Business Manager to align on 2026 opportunities, upcoming program updates and ways we can elevate customer outcomes together. Visit the partner portal to learn more.


Key Takeaways

  • Integration beats complexity.
    Unifying technology, data and expertise drove the strongest outcomes in 2025, helping partners reduce risk and accelerate time-to-value for customers.
  • AI is a built-in advantage.
    By tapping into AI embedded across our cybersecurity platform, partners can address security and business outcomes simultaneously and deliver repeatable, profitable, AI-powered services.
  • The partner ecosystem is a growth engine, and together, we’re unstoppable.
    Our 2026 priorities focus on deeper integration, coinnovation, empowered enablement and simplified engagement that drive partner profitability and stronger customer outcomes.

The post Partnering with Precision in 2026 appeared first on Palo Alto Networks Blog.

Winning the AI Race Starts with the Right Security Platform

Every CIO and CISO we speak with describes the same paradox: AI is now central to their transformation agenda, yet the fastest way to derail that agenda is to lose control of AI. As generative AI, agentic systems and embedded AI features spread across the enterprise, leaders are no longer asking if they need AI security; they’re asking what kind of AI security strategy will actually scale.

Gartner® has published two recent reports that validate this reality and outline the strategic direction enterprises must take to secure their AI:

Why AI Security Is a Platform Game

Point products can plug individual gaps, but they can’t keep up with the speed, complexity and interconnected nature of AI adoption. And more importantly, they struggle to deliver the trust, consistency or scale AI transformation requires.

Many organizations are already experiencing AI adoption outpacing traditional security tools. Security teams are under pressure on three fronts:

  • Risk – Shadow AI, unmanaged agents and custom LLMs create new pathways for data loss, intellectual property exposure and model misuse.
  • Cost – Each new AI use case brings yet another tool, driving up license, integration and operations costs.
  • Complexity – Fragmented controls across network, data, identity and application stacks create blind spots exactly where AI is moving fastest.

From a CIO or CISO’s perspective, this isn’t just a technical concern but the fault line beneath their entire AI agenda. CIOs are under pressure to deliver productivity gains, cost efficiencies and new AI-powered capabilities faster than ever before.

CISOs, on the other hand, see a parallel reality: custom-built AI applications that may be insecure by default, agents that can act unpredictably, and a constant risk that company secrets or customer data could leak into third-party GenAI tools.

If AI moves forward without security, the enterprise is exposed. If AI slows down because security can’t keep up, the business misses its transformation goals. This is why AI security isn’t a feature; it’s the determining factor in whether AI becomes a competitive advantage or a strategic setback.

Gartner recommends the path forward as “an integrated modular AI security platform (AISP) with a common UI, data model, content inspection engine and consistent policy enforcement.”

Gartner further recommends prioritizing investments in two phases.

Phase 1

Start with AI usage control to secure the consumption of third-party AI services.

Phase 2

Expand into AI application protection to securely develop and run AI applications.

Phase 1: Securing Generative AI Usage Is the “Right Now” Challenge

Before enterprises can secure how AI is developed, they must first understand how it is already being used across the organization. The earliest risks often emerge not from the AI-enabled apps built in-house, but from the external generative AI tools and copilots employees adopt, and often without the IT teams’ knowledge.

That’s why we think the report identifies AI usage control as phase one and why we recommend IT leaders start with these immediate questions to assess their organization’s AI usage.

  • Where is AI actually being used in my organization?
  • Which tools, copilots and agents are in play, and on what data?
  • How do I enable productivity without losing control?

Phase 2: Securing AI Development Early Into the AI Lifecycle

Once public generative AI use is understood, the harder challenge emerges: Securing the AI apps and tools that your organization creates for itself. As models, agents and pipelines move into production, the questions shift from visibility to integrity, safety and scale.

Key questions that organizations must answer in phase two include:

  • What AI applications, models and agents are my teams building, and where do they live?
  • How do I manage the integrity, safety and compliance of AI apps before they reach production?
  • How do I protect models and AI applications from prompt injection, misuse or agentic threats?
  • How do I scale AI innovation without creating security bottlenecks for developers?

Palo Alto Networks Delivers the AI Security Platform

Although organizations can separate the work around securing AI usage and AI development, they are not two separate problems. The same organization that needs visibility into employees using public GenAI apps also needs to protect the AI applications and agents they’ve built as they move into production. A platform approach is what allows shared policies, shared guardrails and shared context across both sides of the AI usage and development equation.

That is exactly the philosophy behind our Secure AI by Design approach:

  • Secure how GenAI is used with Prisma® Browser™ and Prisma SASE to discover AI tools in use, govern access and prevent sensitive data from flowing into public models, all while keeping users productive with GenAI and enterprise copilots.
  • Secure how AI is built with capabilities of Prisma AIRS™, such as model and agent security, AI security posture management, runtime protection, automated testing with AI Red Teaming, as well as coverage for agentic protocols, like MCP, securing custom AI applications, agents and pipelines.

Gartner identifies Palo Alto Networks as “the company to beat” in their newly released report as of December 8, 2025: “AI Vendor Race: Palo Alto Networks Is the Company to Beat in AI Security Platforms.”

We believe we are the AI Security Platform to beat because:

  • Palo Alto Networks product portfolio across network, edge, cloud and data provides a strong foundation for AI usage visibility and control.
  • The acquisition of Protect AI integrated industry-leading AI talent and products resulting in the recently announced Prisma AIRS 2.0, which delivers comprehensive end-to-end AI security, seamlessly connecting deep AI agent and model inspection in development with real-time agent defense at production runtime. The platform, continuously validated by autonomous AI red teaming, secures all interactions between AI models, agents, data and users. This gives enterprises the confidence to discover, assess and protect their entire AI ecosystem, accelerating secure innovation.
  • Complementing the platform, Unit 42®’s deep expertise and Huntr’s bug bounty program, provide security thought leadership that directly improves product effectiveness and threat intelligence. These programs help us continuously uncover new attack patterns, misconfigurations and supply chain risks unique to AI systems, as well as feed those insights directly back into the product roadmap.
  • Our large installed base and distribution channels create a flywheel for AI security platform adoption and learning from our customers and partners.

We also believe that underneath the technical requirements is a deeper truth: CIOs and CISOs want to move fast on AI, but they only feel safe doing so with a partner who has the scale, signal and staying power. This is where our breadth, research depth and ecosystem matter.

Leading Responsibly Means Listening, Innovating and Evolving

Being early is an advantage, but staying ahead requires humility and continuous learning. Leading means seeing what comes next, and Gartner’s insights accelerate our own roadmap as we continue to evolve.

  • Simplifying the Experience: We are integrating capabilities across Prisma AIRS, Prisma SASE and Prisma Browser to make AI security easier to adopt, operate and scale through Strata™ Cloud Manager as the single entry point.
  • Going Deeper into the AI Engineering Pipeline: We recognize that securing AI must start early in the developing environment and ML pipeline, not just at runtime. Our integrations with AI development tools and code repositories will continue to expand.
  • Keeping Pace with a Fast-Moving Market: We are investing in open standards, partnerships and research, so our customers don’t have to chase every point solution that appears. Palo Alto Networks is also a contributing member to OWASP Standards and Threat analysis to help create an industry standard on AI security.
  • Working Along Native AI Controls: Cloud providers and AI platforms are adding their own security features. We aim to complement, not replace, those controls, providing unified visibility, advanced protection and consistent policies across a fragmented AI landscape.

For us, being “the company to beat” is not a finish line. It’s a responsibility to listen carefully to customers, adapt as AI evolves, and keep delivering practical, integrated outcomes rather than isolated features.

If you are a GM, CIO, CISO or AI leader trying to make sense of a rapidly crowding AI security landscape, we believe “GMs: Win the AI Security Battle With an AI Security Platform”​​ is essential reading.

In the end, the real race isn’t about features; it’s about who helps enterprises accelerate transformation safely, reduce risk and compete better with AI they can trust.

 

Disclaimer: Gartner does not endorse any company, vendor, product or service depicted in its publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner publications consist of the opinions of Gartner’s business and technology insights organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this publication, including any warranties of merchantability or fitness for a particular purpose.

Gartner, AI Vendor Race: Palo Alto Networks is the Company to Beat in AI Security Platforms, By Mark Wah, Neil MacDonald, Marissa Schmidt, Dennis Xu, Evan Zeng, 8 December 2025. 

Gartner, GMs: Win the AI Security Battle With an AI Security Platform, By Neil MacDonald, Tarun Rohilla, 6 October 2025.

GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.

The post Winning the AI Race Starts with the Right Security Platform appeared first on Palo Alto Networks Blog.

Crossing the Autonomy Threshold

What It Means and How to Counter Autonomous Offensive Cyber Agents

For years, we've anticipated this day. With the release of Anthropic's landmark report (detailing the disruption of a cyberespionage operation orchestrated by AI agents with minimal human intervention), the reality of autonomous offensive cyber agents has moved from speculation to an active, machine-speed threat. The report covers their internal identification and analysis of artifacts from the GTG-1002 campaign, which was conducted against over 30 different enterprise targets. This event is independently being tracked in the AI Incident Database as incident 1263. To have a successful defense in the age of AI, we need an immediate shift from human-led, reactive security to a proactive, machine-driven security paradigm.

The GTG-1002 campaign is the first open report of an AI agent, powered by Claude Code, targeting multiple enterprise environments. Using Claude Code as the primary orchestration framework, the agent was effective in all key phases of the attack:

  • Mapping attack surfaces without human guidance.
  • Exploit vulnerabilities using custom code generation.
  • Moving laterally by autonomously harvesting and testing credentials.
  • Conducting an intelligence analysis to identify and prioritize high-value data, rather than just exfiltrating raw dumps.

It was a watershed moment for several key reasons:

  • Stealth Traffic analysis of the inputs and outputs to Claude Code were the initial indicators of this attack, however, the attack was only observable in aggregate.
  • Self-Configuration The agent autonomously adapted its attack strategy to achieve actions on an objective.
  • Machine-Speed – The agent both orchestrated AND executed the campaign across all attack vectors.
  • Autonomous Context and Persistence Using structured markdown files, the execution agent maintained a persistent state of the attack, providing context and autonomous continuity between distributed sub-actions and attack phases.

This campaign, executed at “multiple operations per second,” marks the end of the necessity for the "human-in-the-loop” attacker and the arrival of the "human-on-the-loop" supervisor. Transitions between attack phases were controlled by the human to validate sufficient completion of the current phase before progressing. It was a thin layer of supervisory human control. With the whiplash pace of AI, defenders should anticipate the necessity of any human control to fade.

In the reported attack campaign, “commodity tools” were leveraged by the threat actor, which at first glance, may not seem particularly novel. However, the autonomous orchestration of these tools across multiple attack phases by Claude Code, using Model Context Protocol (MCP) servers, represents a sophisticated technical advancement in offensive agents. Critically, this method improved more than just the speed of the attack, it also introduced the concept of autonomy with negligible human supervision, supporting dynamic and contextual reasoning in attack path planning across multiple target systems (even beyond typical human analyses, particularly for non-intuitive/interpretable event logging). Custom tools can bring very targeted actions within the same or similar offensive agent architectures, and defenders should be ready for this inevitable evolution.

We Need Agents to Fight Agents

With the debut of real-world offensive agent operations, it is now crystal clear: Defenders cannot combat autonomous, offensive AI with manual, static human driven security operations. Defenses must blend machine-speed responses with on-the-fly adaptability to maintain effectiveness against the self-optimizing campaigns now being observed. The pivot to autonomous agent-driven security operations will require transforming many elements of the traditional security operations lifecycle. All stages from preparation to response processes need to be resilient and robust to changes in adversary speed, stealth, evasion, orchestration frameworks and indicators of compromise.

Meeting the Challenges of Machine-Speed Defense Head-On

A new defense paradigm must be adopted to effectively combat AI attacks that are both orchestrated AND executed beyond human reaction time. To transform security operations and outpace AI-driven threats, organizations need to employ the following core principles:

  • Precision of AI for Cybersecurity: Operating at machine speed requires precision and accuracy. Security systems must be capable of ingesting the right data, at the right time, and understanding the system context to detect and block threats in real-time, thwarting AI-generated attacks without generating erroneous alerts. Producing false positives is problematic at human speeds, and the problem compounds at machine speed.
  • Proactive Cybersecurity for AI Systems: We must safeguard AI systems with real-time security solutions, preventing the models and applications from being directly or indirectly co-opted for malicious use. This demands a deep and continuous understanding of how AI agents might be abused via their application interfaces, permissions, provenance, identity and wider interactions across organizations.
  • Transform Visibility into Observability: Visibility only encompasses a direct presence or absence. Observability is the combination of visibility plus some degree of cognitive and contextual reasoning. The visibility of a traffic sign does not guarantee a driver will observe and respond to it. The GTG-1002 attack evaded detection by splitting and distributing small, seemingly benign fragments of the full campaign across numerous sessions. The requests were visible, but the scope of the malicious campaign was not observed from the isolated requests. To identify and help stop such techniques, defenses need distributed observability, which can only be achieved from context-aware agents that understand the nature and impact of disparate events and can disrupt such attacks when they are identified.
  • Agentic Security Operations: As an industry, we must also acknowledge the difference between autonomous and automated systems. The industry has been integrating elements of automation for years. Scripting, decision trees and playbooks are mechanisms for speeding up the response in specific context, but do not necessarily generalize or work across different phases. If the attacker is using an agentic system for 90% of the attack lifecycle, security operations centers (SOCs) must also implement an agentic system for 90% of their triage, investigation, remediation and threat hunting workflows. This must be the rule, rather than the exception. By combining observability with dynamic AI agents capable of coordinated decision making and task execution, SOCs can deliver proactive autonomous protection at scale.

The Future Is Now. Are You Ready?

The GTG-1002 campaign is a clear signal that offensive AI agents are being used in the wild. The adoption of AI agents by threat actors will accelerate and demand a decisive transformation of defensive security operations to include agent orchestration tools customized to respond to the uniqueness of offensive AI agents.

At Palo Alto Networks, our platformization strategy was built precisely for this moment. This interconnectivity between tools and systems transforms visibility into observability necessary for AI agent orchestration.

In light of GTG-1002, there is an unequivocal need for the security community to accelerate the pivot from automated to autonomous security operations. AI agents can quickly find and exploit vulnerabilities, moving stealthily across the attack chain. We must shift from human-led, reactive defense to fast, proactive machine-driven security to ensure cyber resilience in the age of AI.

Are you ready? Learn about securing AI agents and how to create a trustworthy AI ecosystem.


Key Takeaways

  • Autonomous Orchestration and Execution: The GTG-1002 campaign was a watershed event because the AI agent, powered by Claude Code, autonomously orchestrated and executed all key phases of the attack, from mapping surfaces and exploiting vulnerabilities to moving laterally and conducting intelligence analysis at machine speed.
  • Shift to Machine-Driven Security Paradigm: The emergence of autonomous offensive cyber agents, as demonstrated by the GTG-1002 campaign, demands an immediate pivot from human-led, reactive security to a proactive, machine-driven security defense model.
  • Distributed Observability is Essential to Agentic Defenses: To counter new attack techniques like GTG-1002, which evade detection by splitting the campaign into small, distributed, and seemingly benign fragments, defenses must adopt distributed observability to connect disparate events using context-aware agents.

Further Reading:

The post Crossing the Autonomy Threshold appeared first on Palo Alto Networks Blog.

Mitigating prompt injection attacks with a layered defense strategy

13 June 2025 at 18:03
Posted by Google GenAI Security Team

With the rapid adoption of generative AI, a new wave of threats is emerging across the industry with the aim of manipulating the AI systems themselves. One such emerging attack vector is indirect prompt injections. Unlike direct prompt injections, where an attacker directly inputs malicious commands into a prompt, indirect prompt injections involve hidden malicious instructions within external data sources. These may include emails, documents, or calendar invites that instruct AI to exfiltrate user data or execute other rogue actions. As more governments, businesses, and individuals adopt generative AI to get more done, this subtle yet potentially potent attack becomes increasingly pertinent across the industry, demanding immediate attention and robust security measures.


At Google, our teams have a longstanding precedent of investing in a defense-in-depth strategy, including robust evaluation, threat analysis, AI security best practices, AI red-teaming, adversarial training, and model hardening for generative AI tools. This approach enables safer adoption of Gemini in Google Workspace and the Gemini app (we refer to both in this blog as “Gemini” for simplicity). Below we describe our prompt injection mitigation product strategy based on extensive research, development, and deployment of improved security mitigations.


A layered security approach

Google has taken a layered security approach introducing security measures designed for each stage of the prompt lifecycle. From Gemini 2.5 model hardening, to purpose-built machine learning (ML) models detecting malicious instructions, to system-level safeguards, we are meaningfully elevating the difficulty, expense, and complexity faced by an attacker. This approach compels adversaries to resort to methods that are either more easily identified or demand greater resources. 


Our model training with adversarial data significantly enhanced our defenses against indirect prompt injection attacks in Gemini 2.5 models (technical details). This inherent model resilience is augmented with additional defenses that we built directly into Gemini, including: 


  1. Prompt injection content classifiers

  2. Security thought reinforcement

  3. Markdown sanitization and suspicious URL redaction

  4. User confirmation framework

  5. End-user security mitigation notifications


This layered approach to our security strategy strengthens the overall security framework for Gemini – throughout the prompt lifecycle and across diverse attack techniques.


1. Prompt injection content classifiers


Through collaboration with leading AI security researchers via Google's AI Vulnerability Reward Program (VRP), we've curated one of the world’s most advanced catalogs of generative AI vulnerabilities and adversarial data. Utilizing this resource, we built and are in the process of rolling out proprietary machine learning models that can detect malicious prompts and instructions within various formats, such as emails and files, drawing from real-world examples. Consequently, when users query Workspace data with Gemini, the content classifiers filter out harmful data containing malicious instructions, helping to ensure a secure end-to-end user experience by retaining only safe content. For example, if a user receives an email in Gmail that includes malicious instructions, our content classifiers help to detect and disregard malicious instructions, then generate a safe response for the user. This is in addition to built-in defenses in Gmail that automatically block more than 99.9% of spam, phishing attempts, and malware.


A diagram of Gemini’s actions based on the detection of the malicious instructions by content classifiers.


2. Security thought reinforcement


This technique adds targeted security instructions surrounding the prompt content to remind the large language model (LLM) to perform the user-directed task and ignore any adversarial instructions that could be present in the content. With this approach, we steer the LLM to stay focused on the task and ignore harmful or malicious requests added by a threat actor to execute indirect prompt injection attacks.

A diagram of Gemini’s actions based on additional protection provided by the security thought reinforcement technique. 


3. Markdown sanitization and suspicious URL redaction 


Our markdown sanitizer identifies external image URLs and will not render them, making the “EchoLeak” 0-click image rendering exfiltration vulnerability not applicable to Gemini. From there, a key protection against prompt injection and data exfiltration attacks occurs at the URL level. With external data containing dynamic URLs, users may encounter unknown risks as these URLs may be designed for indirect prompt injections and data exfiltration attacks. Malicious instructions executed on a user's behalf may also generate harmful URLs. With Gemini, our defense system includes suspicious URL detection based on Google Safe Browsing to differentiate between safe and unsafe links, providing a secure experience by helping to prevent URL-based attacks. For example, if a document contains malicious URLs and a user is summarizing the content with Gemini, the suspicious URLs will be redacted in Gemini’s response. 


Gemini in Gmail provides a summary of an email thread. In the summary, there is an unsafe URL. That URL is redacted in the response and is replaced with the text “suspicious link removed”. 


4. User confirmation framework


Gemini also features a contextual user confirmation system. This framework enables Gemini to require user confirmation for certain actions, also known as “Human-In-The-Loop” (HITL), using these responses to bolster security and streamline the user experience. For example, potentially risky operations like deleting a calendar event may trigger an explicit user confirmation request, thereby helping to prevent undetected or immediate execution of the operation.


The Gemini app with instructions to delete all events on Saturday. Gemini responds with the events found on Google Calendar and asks the user to confirm this action.


5. End-user security mitigation notifications


A key aspect to keeping our users safe is sharing details on attacks that we’ve stopped so users can watch out for similar attacks in the future. To that end, when security issues are mitigated with our built-in defenses, end users are provided with contextual information allowing them to learn more via dedicated help center articles. For example, if Gemini summarizes a file containing malicious instructions and one of Google’s prompt injection defenses mitigates the situation, a security notification with a “Learn more” link will be displayed for the user. Users are encouraged to become more familiar with our prompt injection defenses by reading the Help Center article


Gemini in Docs with instructions to provide a summary of a file. Suspicious content was detected and a response was not provided. There is a yellow security notification banner for the user and a statement that Gemini’s response has been removed, with a “Learn more” link to a relevant Help Center article.

Moving forward


Our comprehensive prompt injection security strategy strengthens the overall security framework for Gemini. Beyond the techniques described above, it also involves rigorous testing through manual and automated red teams, generative AI security BugSWAT events, strong security standards like our Secure AI Framework (SAIF), and partnerships with both external researchers via the Google AI Vulnerability Reward Program (VRP) and industry peers via the Coalition for Secure AI (CoSAI). Our commitment to trust includes collaboration with the security community to responsibly disclose AI security vulnerabilities, share our latest threat intelligence on ways we see bad actors trying to leverage AI, and offering insights into our work to build stronger prompt injection defenses. 


Working closely with industry partners is crucial to building stronger protections for all of our users. To that end, we’re fortunate to have strong collaborative partnerships with numerous researchers, such as Ben Nassi (Confidentiality), Stav Cohen (Technion), and Or Yair (SafeBreach), as well as other AI Security researchers participating in our BugSWAT events and AI VRP program. We appreciate the work of these researchers and others in the community to help us red team and refine our defenses.


We continue working to make upcoming Gemini models inherently more resilient and add additional prompt injection defenses directly into Gemini later this year. To learn more about Google’s progress and research on generative AI threat actors, attack techniques, and vulnerabilities, take a look at the following resources:


❌