This week, Axios reported that major US banks were detecting third-party AI agents operating inside customer accounts. In many cases, customers have given the agents their usernames and passwords so they could act on their behalf. This kind of access can violate a bank’s rules.
“Are you protecting me, or are you protecting your own profits?”
Bianco described traditional finance’s business model as “based on laziness.” Customers may leave idle cash in low- or no-interest bank accounts rather than move it into a money market fund. An AI agent could make that move for them. “Now that you’re able to do it, the banks are afraid they’re gonna lose that revenue stream,” he said.
The banks offer a different explanation. JPMorgan Chase CEO Jamie Dimon said the bank wouldn’t allow just any outside agent into its systems because doing so was “too risky for clients and for JPMorgan.”
Both explanations can be true: banks can want to retain deposits while also facing real security problems when software acts through customers’ accounts.
Handing over your banking login introduces another service into the security chain. A vulnerability in that service could expose your credentials or allow your account to be misused. And as Bank of America reportedly pointed out:
“If you do share your credentials, keep in mind that you’re responsible for activity resulting from that access.”
That’s the bank’s stated position, not a universal ruling on consumers’ legal rights. Still, it’s a warning worth taking seriously.
Pros and cons
An AI agent with permission to make purchases might find and buy an item at a lower price. But giving an agent access to your bank account also means sharing sensitive banking information with a third party. We’ve seen shared AI chats become searchable online too.
Axios reported that many banks allow, or are moving toward allowing, read-only access. This can let a tool analyze transactions without giving it permission to move money.
That’s an important boundary, but it doesn’t make the information harmless. An agent that can read bank statements may still disclose them if it’s compromised or manipulated. Preventing transactions and protecting confidentiality are separate security goals.
How to keep control of your money
For consumers, the practical question is not whether banks have perfectly selfless motives. It’s what access you’re giving away, and what could happen if the agent makes a mistake.
Don’t paste banking passwords or security codes into a general-purpose AI assistant. Major banks have explicitly warned customers against sharing their credentials.
Use your bank’s approved connection methods rather than giving an agent your login. Where supported, choose read-only access if all you need is analysis or budgeting help.
Give agents access to as little sensitive information as possible. An assistant comparing publicly advertised savings rates doesn’t need your bank statements. OpenAI recommends limiting agents to the data and credentials they need for the task.
Review consequential actions yourself. If an agent asks for confirmation, check exactly what it intends to do and what information it will share. Don’t treat a confirmation prompt as a formality.
AI can make it easier to compare financial products and spot opportunities you might otherwise miss. But an assistant that finds a better rate and an agent that can act inside your account are different propositions.
Let it help with the homework, but be cautious about handing it the keys.
Let’s face it, an incognito window can only do so much.
Breaches, dark web trading, credit fraud. Malwarebytes Identity Theft Protection monitors for all of it, alerts you fast, and comes with identity theft insurance.
Cybersecurity news increasingly focuses on attackers using AI for reconnaissance, vulnerability hunting, and writing scripts or malware. Against this backdrop, a less conventional category of attacks remains largely overlooked: threat actors leveraging AI tools that are already deployed within the target infrastructure.
AI agents designed for coding, as well as command-line tools — such as Claude Code CLI, Gemini CLI, Codex CLI, Amazon Q CLI, and the like — are particularly attractive targets. These agents can read and modify files, execute shell commands, install packages, and connect to external services. Developers frequently allow them to run in autonomous mode to avoid manually confirming every action. For an attacker, such an agent becomes a ready-made multitool that has the right privileges, access to critical data, and security clearances from the get-go. That said, other types of AI agents, such as automated financial workflow assistants, are equally vulnerable to exploitation.
Why trusted agents pose a threat
An AI agent differs from standard software in its broad access privileges, the wide array of ways it can be prompted to execute actions, and the difficulty of detecting its anomalous behavior. Enterprise security controls (EPP, EDR, XDR, SIEM) scan for malicious files, known command patterns, and suspicious action sequences. Even if a specific strain of malware hasn’t been seen before, its behavior often gives it away: bulk searches for documents, spawning PowerShell processes, accessing credential stores, or exfiltrating data to an external server from an unknown application will trigger a deluge of alerts in the SOC.
It’s much harder to apply that same detection logic to a legitimate, pre-installed AI agent. By design, these tools are capable of executing an extremely broad range of operations. Running scripts, reading configuration files (including secrets), moving documents, and installing dependencies can all be a part of their normal operation — especially on a developer’s machine.
When an attacker attempts to exploit this kind of AI system, they use natural language to convey their instructions to the agent. A request to hunt for passwords, API keys, or cryptocurrency wallets can be framed in thousands of ways, expressed in a variety of languages, or phrased indirectly. Furthermore, a malicious command can be hidden in any data stream the agent reads — including text fields in documents, file names, or even error logs, as shown in one of the examples below.
Integration between agents and third-party software further expands the attack surface. Agents are frequently connected to email, instant messaging apps, task management systems, knowledge bases, and MCP servers. As a result, a malicious instruction can originate from virtually any source whose data the agent processes automatically.
Below are several examples of attacks where threat actors leveraged AI agents to achieve their goals.
The Nx attack: a coding agent hunting for passwords
The most prominent and widespread example is the compromise of Nx npm packages in August 2025, dubbed s1ngularity. Attackers exploited a vulnerable GitHub Actions workflow, stole a package-publishing token, and released trojanized versions of several @nx packages that harvested developer secrets.
The malicious post-install script checked whether Claude Code, Gemini CLI, or Amazon Q CLI were installed on the victim’s machine. If any of these were detected, the agent was issued an instruction to scan for cryptocurrency wallets, ENV files, API keys, and other sensitive data. The agents were executed with auto-approval flags enabled, such as --dangerously-skip-permissions, --yolo, and --trust-all-tools.
The search results were then exfiltrated to public GitHub repositories created by the attackers. Over several waves of the attack, thousands of secrets belonging to hundreds of organizations were exposed.
Instead of developing a complex, custom harvesting tool, the attackers repurposed the victim’s own legitimate agent. It already could navigate the file system, understand file context, and identify high-value target data. However, post-incident analysis by security experts revealed drawbacks to this approach: the AI occasionally refused to carry out unethical tasks. Furthermore, the malware authors had to adjust their code so that it “waited” for the AI agent, which could take a considerable amount of time to process the task while generating significant intermediate output.
Communication via… error logs
The AgentJacking study was not a malicious attack, but it differed from one only in intent: its authors had no interest in stealing secrets. After developing the attack technique, the researchers tested it on a global scale and received callback responses from AI agents belonging to more than a hundred real-world companies — some of them major enterprises.
The attack targeted Sentry, a widely used telemetry service. Websites and mobile applications automatically transmit error reports to Sentry so developers can monitor issues without requiring active user reporting. These reports are accepted without authentication because errors often occur for anonymous visitors that the site or application knows nothing about.
The researchers configured their own MCP server within Sentry, allowing AI agents to analyze reports automatically. Next, they submitted a fake error message containing a data block formatted to match Sentry’s MCP server data structure. Embedded inside was an indirect prompt injection — an instruction to run “additional diagnostics” via a command like: npx @controlled-validation-package --diagnose.
If error handling were delegated to an agent, it could take the instruction as legitimate and execute a command resulting in the automatic installation of a third-party package. In this experiment, the package merely contacted the researchers’ command-and-control server without exfiltrating any sensitive data. A genuine malicious package, however, could steal credentials, tamper with code, or establish persistence in the system. Identifying targets for such an attack is equally straightforward: an organization’s use of Sentry is easily detectable by analyzing website code or checking the strings in mobile application binaries.
Notably, according to the study’s authors, Sentry acknowledged the issue but stopped short of implementing comprehensive fixes, opting instead to block only the specific variant of the malicious query routed through the MCP. Screenshots included in the study reveal that a Sentry employee described the system as “technically not defensible” against the attack.
Stealing financial data via MCP
Abusing unsecured MCP servers — as seen in the Sentry case — is just one way to attack this protocol. Introduced just a year ago, and already dubbed “USB for AI agents”, the Model Context Protocol standard was adopted rapidly without robust built-in security controls. As a result, attacks targeting MCP are bound to proliferate — a trend Microsoft recently warned about.
Threats can lie hidden even within the description of an MCP tool before the agent ever uses it. Agents parse tool descriptions to determine when to call the tool and what data to supply. If an MCP server provider stealthily alters a description — adding an instruction that demands collection of additional data — the agent may execute it as part of its standard workflow.
Microsoft highlights a scenario involving a financial agent that verifies vendor banking details via an external MCP service. Following a malicious update to the tool’s description, the agent begins appending unpaid-invoice data to its responses. The user sees a normal response, while confidential financial data is quietly exfiltrated to the server owner. Individually, each operation appears authorized: the agent operates under the employee’s permissions, queries an approved service, and uses a standard interface. Yet a data breach still occurs.
It remains unclear from Microsoft’s ambiguous phrasing whether this example is purely hypothetical or based on an investigation into a live incident. However, the fact that the post originated from Microsoft Incident Response points to the latter.
Prompt injections have already gone mainstream
In 2026, Unit 42 researchers identified widespread attempts to covertly embed instructions designed for AI systems across numerous public websites. Many of these instructions aim to force models to reveal system prompts, approve sites and posts during ad placement reviews, or boost phishing pages through SEO manipulation.
Similarly, an analysis of Common Crawl data by Google recorded a surge in such material: between November 2025 and February 2026, the share of malicious injections increased by 32%.
That said, confirmed cases of successful compromise using these techniques remain scarce. The presence of an injection on a webpage does not guarantee that an agent ever actually executed the command. Nevertheless, threat actors have clearly begun optimizing web content for an era when it will be read not just — and perhaps not primarily — by humans.
How to defend against agent-based attacks
The key takeaway for cybersecurity teams and all AI users alike is simple: an AI agent should never be considered trusted merely because it has been officially approved for enterprise use.
Organizations must:
Maintain an inventory of agents, MCP servers, and the various tools available for them. Use allowlists to restrict agent tools, and package installations to components explicitly vetted and approved by information security. Allowlists must specify exact package versions.
Monitor version changes to MCP tools and other dependencies. Re-evaluate updates before adding them to the approved inventory.
Apply the principle of least privilege. Grant agents only the specific permissions needed for a given task, and only for the duration of that task.
Enforce human-in-the-loop approval for package installations, script executions, file sends, and other high-risk actions.
Run agents in isolated environments with limited access to developer workstations.
Disable dangerous auto-approval modes.
Restrict outbound network connections, and permit data transmission only for approved services.
Store secrets in centralized, secure vaults, using short-lived tokens and rotating them regularly.
Forward prompt logs, tool calls, shell commands, and agent network traffic into SIEM and XDR systems. For these tools to be effective, they require deep operational context regarding agent activity alongside specialized rule sets tailored to AI data processing pipelines.
Unchecked AI in the workplace quickly becomes a massive loophole for data leaks and security breaches. All too often, employees drop sensitive company data into public chatbots, or install rogue AI assistants on their own — in the process handing over way too much access. In a previous post, we broke down the different types of risky AI systems, and later shared some tips on how to turn off the built-in AI features on major tech platforms. Today let’s take a look at practical ways to block or restrict the unauthorized “helpers” employees might be using — from ChatGPT and Grammarly, to meeting bots like Fireflies and Read AI.
How to detect and restrict ChatGPT
ChatGPT is the biggest culprit when it comes to unauthorized AI use worldwide. A quick word of warning, though: an outright ban only sends users hunting for sketchy third-party sites or messaging app chatbots that hook into the same service. That’s why it’s always a good idea to offer an approved alternative before pulling the plug.
Detecting it: keep an eye on the NGFW or web filter for traffic heading to chat.openai.com, chatgpt.com, oaistatic.com, oaiusercontent.com, or cdn.oaistatic.com. It’s also smart to use EDR/EPP tools to scan browser histories, installed apps, and browser extensions across corporate devices.
Locking it down: use the firewall or web filter to block the entire AI Services category, and set up DNS to reroute traffic away from those OpenAI domains. Browser policies can also be used to ban ChatGPT-powered extensions. Better yet, block all extensions not on a pre-approved allowlist. Finally, use application controls and EPP solutions to stop users from installing the official desktop app (ChatGPT.exe or com.openai.chat).
How to detect and restrict Claude and Claude Code
Detecting it: use the NGFW or web filter to track traffic going to claude.ai, anthropic.com, *.anthropic.com, and api.anthropic.com. EDR/EPP or application control tools can also be used to scan employee computers for the desktop app (claude.exe).
Locking it down: drop a blanket block on the AI Services category through the NGFW or web filter, and tweak DNS settings to reroute traffic away from the aforementioned Anthropic domains. Next, use browser policies to shut down Claude-powered extensions. Finally, use application controls and the EPP platform to prevent users from installing the desktop app.
How to detect and restrict Perplexity AI
Detecting it: keep tabs on the NGFW or web filter to flag any traffic heading to *.perplexity.ai or pplx.ai.
Locking it down: just like the others, add the AI Services category to the NGFW or web filter blocklist, and use DNS routing to redirect traffic away from those domains.
Configure the browser to block third-party extensions from being installed. If Firefox is used in the organization, be aware that recent versions come with Perplexity built in. Luckily, these AI features can be turned-off company-wide using enterprise policies — specifically, by setting SidebarChatbot = blocked. The full list of tweaks can be found in the Firefox documentation.
How to detect and restrict DeepSeek
Detecting it: keep an eye on the NGFW or web filter for traffic hitting deepseek.com, chat.deepseek.com, api.deepseek.com, or platform.deepseek.com. For better precision, analyze the SNI (server name identification) in TLS connection requests. For mobile devices, look out for the official app (com.deepseek.chat).
Locking it down: blocklist the AI Services category on the NGFW or web filter, and reroute traffic to DeepSeek’s domains via DNS settings. Use browser policies to block third-party extensions, and lean on MDM/EMM tools to restrict the mobile app.
How to detect and restrict Mistral, xAI Grok, and Character.ai
The playbook for these tools is exactly the same as DeepSeek, so here’s the quick list of domains to watch for and block: chat.mistral.ai, mistral.ai, console.mistral.ai, grok.com, x.ai, api.x.ai, character.ai, beta.character.ai, and c.ai.
A quick word of warning on Grok: because Grok is baked into X, blocking this specific AI access point means blocking the entire social media platform.
How to detect and restrict Slack AI
Detecting it: in the Slack workspace admin dashboard, look under Analytics → Slack AI usage. If an enterprise plan is used, the detailed Slack logs can be searched for any events starting with the ai_ prefix.
Blocking it with policies: in the organization’s Slack settings, click through the Workspace settings → Roles & permissions → Feature access, and change the permission to “no one”. Slack has a step-by-step guide in their help center.
Locking it down: shutting this down at the network level is tricky; it can be pulled off with a finely tuned CASB solution in place. Also, don’t forget the importance of blocking rogue integrations and keeping external AI services from tapping into Slack data in the first place. We covered how to lock this down using OAuth controls in a previous post.
How to detect and restrict Zoom AI Companion
Detecting it: if a corporate Zoom subscription is in use, just head to Admin Center → Reports → AI Companion usage. Detecting Zoom’s AI when employees join external meetings or use free accounts is a lot tougher, but email filters can be set up to flag incoming AI-generated meeting notes by scanning for subject lines or text containing “Meeting summary” or “Meeting assets”.
Blocking it with policies: for the company’s own Zoom subscription, go to the Admin Portal → Account Management → Account Settings → Meeting → AI Companion and toggle it OFF for everyone.
Locking it down: unfortunately, AI Companion is baked into Zoom’s DNA, so the only real option is blocking Zoom altogether.
How to detect and restrict Grammarly
What looks like an innocent spellchecker is actually one of the biggest culprits for workplace data leaks.
Detecting it: check the NGFW or web filter logs for traffic hitting grammarly.com, *.grammarly.com, and gnar.grammarly.com. EDR and MDM/EMM tools can also be used to hunt down the standalone desktop apps (Grammarly Desktop.exe and the macOS version), as well as the Grammarly browser extension.
Locking it down: use firewalls to block those domains at the network level, and EPP to stop employees from installing the desktop app, browser extensions, or the Grammarly add-ins for Microsoft Word and Excel.
How to detect and restrict meeting assistants: Fireflies, Read.ai, Tactiq, Fathom, and Granola
This massive category of third-party SaaS tools records and analyzes meetings — creating a massive risk for data leaks. The trickiest part? Outside clients or vendors can bring these bots into a meeting just as easily as employees can.
Detecting them: run an audit on calendar invites, and look for bot participants using email domains like @fireflies.ai, @read.ai, @tactiq.io, @fathom.video, or @granola.ai. Zoom, Teams, or Google Meet logs can also be used to review external participants who joined past calls.
Locking them down: since it’s impossible to control what outsiders do, blocking these bots comes down to tightening meeting rules. The best moves are: blocking users from granting OAuth permissions for bots to join calls, restricting employees from inviting unapproved external participants, or locking down meeting recording access for external users. That last option is usually the least painful way to keep bots out without disrupting business.
How to detect and restrict AI code editors: Cursor, Windsurf, and the like
Detecting them: use EDR/EPP tools to scan for executables like cursor.exe or windsurf.exe. It’s also worth monitoring network traffic heading to cursor.com and windsurf.com, as well as traffic hitting various AI model API providers. Keep in mind that there’s a pretty extensive list of API hosts to monitor here, since these editors aren’t tied to just one specific AI vendor.
Blocking them with policies: these apps can be prevented from being installed by setting up filters based on the developer’s digital signature certificate. Alternatively, a strict application allowlist can be employed where only pre-approved software is allowed to run.
Locking them down: rely on the EPP/EDR platform to actively detect and block these applications from running.
How to detect and restrict local AI tools: Ollama, LM Studio, and GPT4All
On one hand, this category carries fewer data leak risks because the AI models run completely locally on the user’s machine. On the other hand, it opens up a whole new can of worms: these apps themselves aren’t always highly secure, and can become targets for cyberattacks. Plus, it still means that employees can misuse models or process data in unauthorized ways.
Detecting them: EDR/EPP tools are the best line of defense here. They should be used to flag known local AI files and processes like ollama.exe, ollama serve, lmstudio.exe, LM Studio.app, jan.exe, or gpt4all.exe. From a network perspective, it’s worth scanning for open ports on local devices — typically port 1234 for Ollama and LM Studio, or port 8080 for WebUIs (using an additional fingerprint check of the server response). Another massive red flag is the presence of large files (often several gigabytes) containing language model weights. Look out for extensions like .gguf, .bin, or sometimes .safetensors.
Locking them down: use EPP/EDR platforms or windows AppLocker to block these applications by name, or switch to an application allowlist.
How to detect and restrict autonomous agents: OpenClaw, NemoClaw, and NanoClaw
This is easily one of the most dangerous categories of AI tools out there. These agents mix high-level independence with access to untrusted data, making them a massive security headache.
Detecting them: use EPP/EDR tools to sniff out active processes like openclaw, nanoclaw, nemoclaw, or clawdbot. Also keep an eye out for devices running Node.js that suddenly start launching Bash or Python scripts. Another dead giveaway is the appearance of system folders like ~/openclaw, ~/nanoclaw, ~/.claw*, or ~/clawhub. At the network level, monitor connections to the AI model APIs we mentioned earlier, as well as traffic hitting servers like openclaw.ai, nanoclaw.dev, or clawhub.*.
Locking them down: the safest bet is to use strict application allowlisting (only allowing approved software to run), or to specifically ban the known agent apps listed above. On top of that, consider blocking non-developers from installing Node.js and Docker, neither of which they need on their computers anyway.
Lately, software developers have been baking AI features straight into everyday work tools, operating systems, and browsers. In some cases, they’re genuinely handy. However, their presence introduces specific risks, which means plenty of companies are hesitant to give employees access to these tools. In a previous post, we categorized these unwanted AI systems, looked at how to spot them at the network and endpoint levels, and covered the ultimate universal kill switch: managing OAuth access across major corporate platforms. In this deep dive, we’re getting tactical: breaking down how to disable or restrict the AI built into popular platforms.
A quick heads-up: major software vendors occasionally change the names of their AI settings and tweak how they function. If any of the options mentioned below are missing or aren’t working as expected, a quick web search for the setting’s name will usually point you to its new location or branding.
How to turn off Microsoft 365 Copilot
Detection: you can check actual Copilot usage in the logs by going to Microsoft 365 admin → Copilot usage report.
Disabling via policies: in the Microsoft 365Admin Center, go to Settings → Integrated Apps, find Copilot in the Available Apps list, and select Block. More granular configuration policies are available under Customization → Policy Management. The Policies page here contains over two thousand entries, so you’ll want to filter them by the keyword “Copilot” (detailed guide). Given that Copilot is a paid add-on for Office, another way to block it — and save money by doing so — is to simply avoid assigning users SKUs that include Copilot.
We recommend separately blocking Copilot Chat, which is available in Teams, Edge, Outlook, and several other services. Yes, it’s not Copilot itself. And yes, it has to be blocked separately by following this guide.
Additional layer of protection: you can block the domains copilot.cloud.microsoft and m365.cloud.microsoft/chat at the web filter or NGFW level. However, Microsoft explicitly advises against this, warning that it could break other Microsoft 365 features.
How to turn off Windows Copilot
Beyond the Office version of Copilot, you also need to manage its consumer-facing cousin.
Detection: look through your NGFW or other network logs for traffic hitting copilot.microsoft.com, bing.com/chat, or edgeservices.bing.com.
Disabling via policies: in Windows Group Policy, navigate to Computer Config → Admin Templates → Windows Components → Windows Copilot. In Microsoft 365 Group Policy, go to Admin center → Block consumer Copilot for organizational accounts.
Additional layer of protection: block the Copilot.exe executable from running entirely.
How to turn off the Copilot sidebar in Edge
Detection: look through your NGFW or other network logs for traffic hitting copilot.microsoft.com, bing.com/chat, or edgeservices.bing.com.
Blocking: configure the following MS Edge Group Policies: HubsSidebarEnabled = false, EdgeShoppingAssistantEnabled = false, CopilotPageContext = Disabled (false), CopilotNewTabPageEnabled = false, Microsoft365CopilotChatIconEnabled = false, GenAILocalFoundationalModelSettings = 1 (note that disabling this unexpectedly requires a 1 instead of a 0).
Second layer of protection: block the domains copilot.cloud.microsoft and m365.cloud.microsoft/chat at the web filter or NGFW level. However, Microsoft explicitly advises against this, warning that it could break other features.
How to turn off the Gemini Assistant in Google Workspace
Blocking via policies: in the Admin Console, navigate to Apps → Additional Google services → > Gemini app, and set it to OFF. Then, go to Manage Workspace smart feature settings → Smart features in Google Workspace, and set it to OFF.
Second layer of protection: block network traffic to the domains gemini.google.com, bard.google.com, and aistudio.google.com.
How to turn off Gemini in Google Chrome
Detection: check your Chrome Enterprise reports (Chrome management → Reports), or look through network traffic logs for connections to the previously mentioned domains.
Blocking via policies: in your Chrome Enterprise policies, configure the following settings: GenAILocalFoundationalModelSettings = 0, HelpMeWriteSettings = 2 (disabled), TabOrganizerSettings = 2, CreateThemesSettings = 2, DevToolsGenAiSettings = 2.
Additional layer of protection: block network traffic to the domains gemini.google.com, bard.google.com, and aistudio.google.com. Additionally, block unauthorized Chrome/Chromium installations (those outside your policy management) with the help of host-based application control tools like EPP/EDR or AppLocker.
How to turn off Apple Intelligence
Detection: on your NGFW and web filters, traffic hitting apple-relay.apple.com and *.apple-cloudkit.com is a clear indicator that Apple Intelligence is active.
Blocking via policies: any managed Apple device allows you to disable individual AI features, though there isn’t a master switch you can flip to shut down “all AI”. In your MDM profile, you need to set the following keys to false (disabled): allowWritingTools, allowMailSummary, allowGenmoji, allowImagePlayground, allowImageWand, allowPersonalizedHandwritingResults, allowExternalIntelligenceIntegrations, allowExternalIntelligenceIntegrationsSignIn, allowNotesTranscription, and allowNotesTranscriptionSummary. Here is a brief configuration example:
Despite Apple’s shift toward declarative device management, these AI features still need to be managed through traditional MDM payload settings.
Second layer of protection: block network traffic to the hosts mentioned above — though the obvious downside for mobile devices is that this won’t work once they leave the corporate network.
This is the first time we have a public, detailed report of a campaign where AI was used at this scale and with this level of sophistication, moving the threat from a collection of AI-assisted tasks to a largely autonomous, orchestrated operation.
This report is a significant new benchmark for our industry. It’s not a reason to panic – it’s a reason to prepare. It provides the first detailed case study of a state-sponsored attack with three critical distinctions:
It was “agentic”: This wasn’t just an attacker using AI for help. This was an AI system executing 80-90% of the attack largely on its own.
It targeted high-value entities: The campaign was aimed at approximately 30 major technology corporations, financial institutions, and government agencies.
It had successful intrusions: Anthropic confirmed the campaign resulted in “a handful of successful intrusions” and obtained access to “confirmed high-value targets for intelligence collection”.
Together, these distinctions show why this case matters. A high-level, autonomous, and successful AI-driven attack is no longer a future theory. It is a documented, current-day reality.
2. What Actually Happened: A Summary of the Attack
The attack (designated GTG-1002) was a “highly sophisticated cyber espionage operation” detected in mid-September 2025.
AI Autonomy: The attacker used Anthropic’s Claude Code as an autonomous agent, which independently executed 80-90% of all tactical work.
Human Role: Human operators acted as “strategic supervisors”. They set the initial targets and authorized critical decisions, like escalating to active exploitation or approving final data exfiltration.
Bypassing Safeguards: The operators bypassed AI safety controls using simple “social engineering”. The report notes, “The key was role-play: the human operators claimed that they were employees of legitimate cybersecurity firms and convinced Claude that it was being used in defensive cybersecurity testing”.
Full Lifecycle: The AI autonomously executed the entire attack chain: reconnaissance, vulnerability discovery, exploitation, lateral movement, credential harvesting, and data collection.
Timeline: After detecting the activity, Anthropic’s team launched an investigation, banned the accounts, and notified partners and affected entities over the “following ten days”.
To have a credible discussion, we must also look at what wasn’t new. This attack wasn’t about secret, magical weapons.
The report is clear that the attack’s sophistication came from orchestration, not novelty.
No Zero-Days: The report does not mention the use of novel zero-day exploits.
Commodity Tools: The report states, “The operational infrastructure relied overwhelmingly on open source penetration testing tools rather than custom malware development”.
This matters because defenders often look for new exploit types or malware indicators. But the shift here is operational, not technical. The attackers didn’t invent a new weapon, they built a far more effective way to use the ones we already know.
4. The New Reality: Why This Is an Evolving Threat
So, if the tools aren’t new, what is? The execution model. And we must assume this new model is here to stay.
This new attack method is a natural evolution of technology. We should not expect it to be “stopped” at the source for two main reasons:
Commercial Safeguards are Limited: AI vendors like Anthropic are building strong safety controls – it’s how this was detected in the first place. But as the report notes, malicious actors are continually trying to find ways around them. No vendor can be expected to block 100% of all malicious activity.
The Open-Source Factor: This is the larger trend. Attackers don’t need to use a commercial, monitored service. With powerful open-source AI models and orchestration frameworks – such as LLaMA, self-hosted inference stacks, and LangChain/LangGraph agents – attackers can build private AI systems on their own infrastructure. This leaves no vendor in the middle to monitor or prevent the abuse.
The attack surface is not necessarily growing, but the attacker’s execution engine is accelerating.
5. Detection: Key Patterns to Hunt For
While the techniques were familiar, their execution creates a different kind of detection challenge. An AI-driven attack doesn’t generate one “smoking gun” alert, like a unique malware hash or a known-bad IP. Instead, it generates a storm of low-fidelity signals. The key is to hunt for the patterns within this noise:
Anomalous Request Volumes: The AI operated at “physically impossible request rates” with “peak activity included thousands of requests, representing sustained request rates of multiple operations per second”. This is a classic low-fidelity, high-volume signal that is often just seen as noise.
Commodity and Open-Source Penetration Testing Tools: The attack utilized a combination of “standard security utilities” and “open source penetration testing tools”.
Traffic from Browser Automation: The report explicitly calls out “Browser automation for web application reconnaissance” to “systematically catalog target infrastructure” and “analyze authentication mechanisms”.
Automated Stolen Credential Testing: The AI didn’t just test one password, it “systematically tested authentication against internal APIs, database systems, container registries, and logging infrastructure”. This automated, broad, and rapid testing looks very different from a human’s manual attempts.
Audit for Unauthorized Account Creation: This is a critical, high-confidence post-exploitation signal. In one successful compromise, the AI’s autonomous actions included the creation of a “persistent backdoor user”.
6. The Defender’s Challenge: A Flood of Low-Fidelity Noise
The detection patterns listed above create the central challenge of defending against AI-orchestrated attacks. The problem isn’t just alert volume, it’s that these attacks generate a massive volume of low-fidelity alerts.
This new execution model creates critical blind spots:
The Volume Blind Spot: The AI’s automated nature creates a flood of low-confidence alerts. No human-only SOC can manually triage this volume.
The Temporal (Speed) Blind Spot: A human-led intrusion might take days or weeks. Here, the AI compressed a full database extraction – from authentication to data parsing – into just 2-6 hours. Our human-based detection and response loops are often too slow to keep up.
The Context Blind Spot: The AI’s real power is connecting many small, seemingly unrelated signals (a scan, a login failure, a data query) into a single, coherent attack chain. A human analyst, looking at these alerts one by one, would likely miss the larger pattern.
7. The Importance of Autonomous Triage and Investigation
When the attack is autonomous, the defense must also have autonomous capabilities.
We cannot hire our way out of this speed and scale problem. The security operations model must shift. The goal of autonomous triage is not just to add context, but to handle the entire investigation process for every single alert, especially the thousands of low-severity signals that AI-driven attacks create.
An autonomous system can automatically investigate these signals at machine speed, determine which ones are irrelevant noise, and suppress them.
This is the true value: the system escalates only the high-confidence, confirmed incidents that actually matter. This frees your human analysts from chasing noise and allows them to focus on real, complex threats.
This is exactly the type of challenge autonomous triage systems like the one we’ve built at Intezer were designed to solve. As Anthropic’s own report concludes, “Security teams should experiment with applying AI for defense in areas like SOC automation, threat detection… and incident response“.
8. Evolving Your Offensive Security Program
To defend against this threat, we must be able to test our defenses against it. All offensive security activities, internal red teams, external penetration tests, and attack simulations, must evolve.
It is no longer enough for offensive security teams to manually simulate attacks. To truly test your defenses, your red teams or external pentesters must adopt agentic AI frameworks themselves.
The new mandate is to simulate the speed, scale, and orchestration of an AI-driven attack, similar to the one detailed in the Anthropic report. Only then can you validate whether your defensive systems and automated processes can withstand this new class of automated onslaught. Naturally, all such simulations must be done safely and ethically to prevent any real-world risk.
9. Conclusion: When the Threat Model Changes, Our Processes Must, Too.
The Anthropic report doesn’t introduce a new magic exploit. It introduces a new execution model that we now need to design our defenses around.
Let’s summarize the key, practical takeaways:
AI-orchestrated attacks are a proven, documented reality.
The primary threat is speed and scale, which is designed to overwhelm manual security processes.
Security leaders must prioritize automating investigation and triage to suppress the noise and escalate what matters.
We must evolve offensive security testing to simulate this new class of autonomous threat.
This report is a clear signal. The threat model has officially changed. Your security architecture, processes, and playbooks must change with it. The same applies if you rely on an MSSP, verify they’re evolving their detection and triage capabilities for this new model. This shift isn’t hype, it’s a practical change in execution speed. With the right adjustments and automation, defenders can meet this challenge.
Login
