❌

Normal view

Beyond β€œIs Your SOC AI Ready?” Plan the Journey!

9 January 2026 at 20:36

You read the β€œAI-ready SOC pillars” blog, but you still see a lot ofΒ this:

Bungled AI SOC transition

How do we doΒ better?

Let’s go through all 5 pillars aka readiness dimensions and see what we can actually do to make your SOC AI-ready.

#1 SOC Data Foundations

As I said before, this one is my absolute favorite and is at the center of most β€œAI in SOC” (as you recall, I want AI in my SOC, but I dislike the β€œAI SOC” concept) successes (if done well) and failures (if not done atΒ all).

Reminder: pillar #1 is β€œsecurity context and data are available and can be queried by machines (API, Model Context Protocol (MCP), etc) in a scalable and reliable manner.” Put simply, for the AI to work for you, it needs your data. As our friends say here, β€œContext engineering focuses on what information the AI has available. […] For security operations, this distinction is critical. Get the context wrong, and even the most sophisticated model will arrive at inaccurate conclusions.”

Readiness check: Security context and data are available and can be queried by machines in a scalable and reliable manner. This is very easy to check, yet not easy to achieve for many types ofΒ data.

For example, β€œgive AI access to past incidents” is very easy in theory (β€œah, just give it old tickets”) yet often very hard in reality (β€œwhat tickets?” β€œaren’t some too sensitive?”, β€œwait…this ticket didn’t record what happened afterwards and it totally changed the outcome”, β€œwell, these tickets are in another system”, etc,Β etc)

Steps to getΒ ready:

  • Conduct an β€œAPI or Die” data access audit to inventory critical data sources (telemetry and context) and stress-test their APIs (or other access methods) under load to ensure they can handle frequent queries from an AI agent. This is important enough to be a Part 3 blog after thisΒ one…
  • Establish or refine unified, intentional data pipelines for the data you need. This may be your SIEM, this may be a separate security pipeline tool, this may be magick for all I care … but it needs to exist. I met people who use AI to parse human analyst screen videos to understand how humans access legacy data sources, and this is very cool, but perhaps not what you want inΒ prod.
  • Revamp case management to force structured data entry (e.g., categorized root causes, tagged MITRE ATT&CK techniques) instead of relying on garbled unstructured text descriptions, which provides clean training data for future AI learning. And, yes, if you have to ask: modern gen AI can understand your garbled stream of consciousness ticket description…. but what it makes of it, you will neverΒ know…

Where you arrive: your AI component, AI-powered tool or AI agent can get the data it needs nearly every time. The cases where it cannot become visible, and obvious immediately.

#2 SOC Process Framework andΒ Maturity

Reminder: pillar #2 is β€œCommon SOC workflows do NOT rely on human-to-human communication are essential for AI success.” As somebody called it, you need β€œmachine-intelligible processes.”

Readiness check: SOC workflows are defined as machine-intelligible processes that can be queried programmatically, and explicit, structured handoff criteria are established for all Human-in-the-Loop (HITL) processes, clearly delineating what is handled by the agent versus the person. Examples for handoff to human may include high decision uncertainty, lack of context to make a call (see pillar #1), extra-sensitive systems,Β etc.

Common investigation and response workflows do not rely on ad-hoc, human-to-human communication or β€œtribal knowledge,” such knowledge is discovered and brought toΒ surface.

Steps to getΒ ready:

  • Codify the β€œTribal Knowledge” into APIs: Stop burying your detection logic in dusty PDFs or inside the heads of your senior analysts. You must document workflows in a structured, machine-readable format that an AI can actually query. If your contextβ€Šβ€”β€Šlike CMDB or asset inventoryβ€Šβ€”β€Šisn’t accessible via API (BTW MCP is not magic!), your AI is essentially flyingΒ blind.
  • Draw a Hard Line Between Agent and Human: Don’t let the AI β€œguess” its level of authority. Explicitly delegate the high-volume drudgery (log summarization, initial enrichment, IP correlation) to the agent, while keeping high-stakes β€œkill switches” (like shutting down production servers) firmly in humanΒ hands.
  • Implement a β€œGrading” System for Continuous Learning: AI shouldn’t just execute tasks; it needs to go to school. Establish a feedback loop where humans actively β€œgrade” the AI’s triage logic based on historical resolution data. This transforms the system from a static script into a living β€œrecipe” that refines itself overΒ time.
  • Target Processes for AI-Driven Automation: Stop trying to β€œAI all the things.” Identify specific investigation workflows that are candidates for automation and use your historical alert triage data as a training ground to ensure the agent actually learns what β€œgood” looksΒ like.

Where you arrive: The β€œtribal knowledge” that previously drove your SOC is recorded for machine-readable workflows. Explicit, structured handoff points are established for all Human-in-the-Loop processes, and the system uses human grading to continuously refine its logic and improve its β€˜recipe’ over time. This does not mean that everything is rigid; β€œVisio diagram or death” SOC should stay in the 1990s. Recorded and explicit beats rigid and unchanging.

#3 SOC Human Element andΒ Skills

Reminder: pillar #3 is β€œCultivating a culture of augmentation, redefining analyst roles, providing training for human-AI collaboration, and embracing a leadership mindset that accepts probabilistic outcomes.” You say β€œfluffy management crap”? Well, I say β€œignore this and your SOC isΒ dead.”

Readiness check: Leaders have secured formal CISO sign-off on a quantified β€œAI Error Budget,” defining an acceptable, measured, probabilistic error rate for autonomously closed alerts (that is definitely not zero, BTW). The team is evolving to actively review, grade, and edit AI-generated logic and detection output.

Steps to getΒ ready:

  • Implement the β€œAI Error Budget”: Stop pretending AI will be 100% accurate. You must secure formal CISO sign-off on a quantified β€œAI Error Budgetβ€β€Šβ€”β€Ša predefined threshold for acceptable mistakes. If an agent automates 1,000 hours of labor but has a 5% error rate, the leadership needs to acknowledge that trade-off upfront. It’s better to define β€œallowable failure” now than to explain a hallucination during an incident post-mortem.
  • Pivot from β€œRobot Work” to Agent Shepherding: The traditional L1/L2 analyst role is effectively dead; long live the β€œAgent Supervisor.” Instead of manually sifting through logsβ€Šβ€”β€Šwork that is essentially β€œrobot work” anywayβ€Šβ€”β€Šyour team must be trained to review, grade, and edit AI-generated logic. They are no longer just consumers of alerts; they are the β€œEditors-in-Chief” of the SOC’s intelligence.
  • Rebuild the SOC Org Chart and RACI: Adding AI isn’t a β€œplug and play” software update; it’s an organizational redesign. You need to redefine roles: Detection Engineers become AI Logic Editors, and analysts become Supervisors. Most importantly, your RACI must clearly answer the uncomfortable question: If the AI misses a breach, is the accountability with the person who trained the model or the person who supervised theΒ output?

Where you arrive: well, you arrive at a practical realization that you have β€œAI in SOC” (and not AI SOC). The tools augment people (and in some cases, do the work end to end too). No pro- (β€œAI SOC means all humans can go home”) or contra-AI (β€œit makes mistakes and this means we cannot use it”) craziesΒ nearby.

#4 Modern SOC Technology Stack

Reminder: pillar #4 is β€œModern SOC Technology Stack.” If your tools lack APIs, take them and go back to the 1990s from whence you came! Destroy your time machine when you arrive, don’t come back toΒ 2026!

Readiness check: The security stack is modern, fast (β€œno multi-hour data queries”) interoperable and supports new AI capabilities to integrate seamlessly, tools can communicate without a human acting as a manual bridge and can handle agentic AI requestΒ volumes.

Steps to getΒ ready:

  • Mandate β€œDetection-as-Code” (DaC): This is no longer optional. To make your stack machine-readable, you must implement version control (Git), CI/CD pipelines, and automated testing for all detections. If your detection logic isn’t codified, your AI agent has nothing to interact with except a brittle GUIβ€Šβ€”β€Šand that is a recipe forΒ failure.
  • Find Your β€œInteroperability Ceiling” via Stress Testing: Before you go live, simulate reality. Have an agent attempt to enrich 50 alerts simultaneously to see where the pipes burst. Does your SOAR tool hit a rate limit? Does your threat intel provider cut you off? You need to find the breaking point of your tech stack’s interoperability before an actual incident does it forΒ you.
  • Decouple β€œNative” from β€œCustom” Agents: Don’t reinvent the wheel, but don’t expect a vendor’s β€œnative” agent to understand your weird, proprietary legacy systems. Define a clear strategy: use native agents for standard tool-specific tasks, and reserve your engineering resources for custom agents designed to navigate your unique compliance requirements and internal β€œsecretΒ sauce.”

Where you arrive: this sounds like a perfect quote from Captain Obvious but you arrive at the SOC powered by tools that work with automation, and not with β€œhuman bridge” or β€œswivelΒ chair.”

#5 SOC Metrics and FeedbackΒ Loop

Reminder: pillar #5 is β€œYou are ready for AI if you can, after adding AI, answer the β€œwhat got better?” question. You need metrics and a feedback loop to getΒ better.”

Readiness check: Hard baseline metrics (MTTR, MTTD, false positive rates) are established before AI deployment, and the team has a way to quantify the value and improvements resulting from AI. When things get better, you will knowΒ it.

Steps to getΒ ready:

  • Establish the β€œBefore” Baseline and Fix the Data Slop: You cannot claim victory if you don’t know where the goalposts were to begin with. Measure your current MTTR and MTTD rigorously before the first agent is deployed. Simultaneously, force your analysts to stop treating case notes like a private diary. Standardize on structured data entryβ€Šβ€”β€Šcategorized root causes and MITRE tagsβ€Šβ€”β€Šso the machine has β€œclean fuel” to learn from rather than a collection of β€œfixed it” or β€œclosed” comments.
  • Build an β€œAI Gym” Using Your β€œGolden Set”: Do not throw your agents into the deep end of live production traffic on day one. Curate a β€œGolden Set” of your 50–100 most exemplary past incidentsβ€Šβ€”β€Šthe ones with flawless notes, clean data, and correct conclusions. This serves as your benchmark; if the AI can’t solve these β€œsolved” problems correctly, it has no business touching your live environment.
  • Adopt Agent-Specific KPIs for Performance Management: Traditional SOC metrics like β€œnumber of alerts closed” are insufficient for an AI-augmented team. You need to track Agent Accuracy Rate, Agent Time Savings, and Agent Uptime as religiously as you track patch latency. If your agent is hallucinating 5% of its summaries, that needs to be a visible red flag on your dashboard, not a surprise you discover during an incident post-mortem.
  • Close the Loop with Continuous Tuning: Ensure triage results aren’t just filed away to die in an archive. Establish a feedback loop where the results of both human and AI investigations are automatically routed back to tune the underlying detection rules. This transforms your SOC from a static β€œfilter” into a learning system that evolves with everyΒ alert.

Where you arrive: you have a fact-based visual that shows your SOC becoming better in ways important to your mission after you add AI (in fact, you SOC will get better even before AI but after you do the prep-work from this document)

As a result, we can hopefully get to thisΒ instead:

Better introduction of AI intoΒ SOC

The path to an AI-ready SOC isn’t paved with new tools; it’s paved with better data, cleaner processes, and a fundamental shift in how we think about human-machine collaboration. If you ignore these pillars, your AI journey will be a series of expensive lessons in why β€œmagic” isn’t a strategy.

But if you get these right? You move from a SOC that is constantly drowning in alerts to a SOC that operates truly 10X effectiveness.

Random cool visual because Nano BananaΒ :)

P.S. Anton, you said β€œ10X”, so how does this relate to ASO and β€œengineering-led” D&R? I am glad you asked. The five pillars we outlined are not just steps for AI; they are the also steps on the road to ASO (see original 2021 paper which is still β€œthe future” forΒ many).

ASO is the vision for a 10X transformation of the SOC, driven by an adaptive, agile, and highly automated approach to threats. The focus on codified, machine-intelligible workflows, a modern stack supporting Detection-as-Code, and reskilling analysts as β€œAgent Supervisors” directly supports the core of engineering-led D&R. So focusing on these five readiness dimensions, you move from a traditional operations room (lots of β€œO” for operations) to a scalable, engineering-centric D&R function (where β€œE” for engineering dominates).

So, which pillar is your SOC’s current β€˜weakest link’? Let’s discuss in the comments and onΒ socials!

Related blogs and podcasts:


Beyond β€œIs Your SOC AI Ready?” Plan the Journey! was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

Inside the BHIS SOC: A Conversation with Hayden CovingtonΒ 

By: BHIS
3 December 2025 at 15:00

What happens when you ditch the tiered ticket queues and replace them withΒ collaboration, agility, and real-time response? In this interview, Hayden Covington takes us behind the scenes of the BHIS Security Operations Center, which isΒ where analystsΒ don’tΒ escalateΒ tickets,Β they solve them.

The post Inside the BHIS SOC: A Conversation with Hayden CovingtonΒ  appeared first on Black Hills Information Security, Inc..

What the Anthropic report on AI espionage means for security leaders

14 November 2025 at 17:35

1. Introduction: The Benchmark, Not the Hype

For a while now, the security community has been aware that threat actors are using AI. We’ve seen evidence of it for everything from generating phishing content to optimizing malware. The recent report from Anthropic on an β€œAI-orchestrated cyber espionage campaign”, however, marks a significant milestone.

This is the first time we have a public, detailed report of a campaign where AI was used at this scale and with this level of sophistication, moving the threat from a collection of AI-assisted tasks to a largely autonomous, orchestrated operation.

This report is a significant new benchmark for our industry. It’s not a reason to panic – it’s a reason to prepare. It provides the first detailed case study of a state-sponsored attack with three critical distinctions:

  • It was β€œagentic”: This wasn’t just an attacker using AI for help. This was an AI system executing 80-90% of the attack largely on its own.
  • It targeted high-value entities: The campaign was aimed at approximately 30 major technology corporations, financial institutions, and government agencies.
  • It had successful intrusions: Anthropic confirmed the campaign resulted in β€œa handful of successful intrusions” and obtained access to β€œconfirmed high-value targets for intelligence collection”.

Together, these distinctions show why this case matters. A high-level, autonomous, and successful AI-driven attack is no longer a future theory. It is a documented, current-day reality.

2. What Actually Happened: A Summary of the Attack

For those who haven’t read the full report (or the summary blog post), here are the key facts.

The attack (designated GTG-1002) was a β€œhighly sophisticated cyber espionage operation” detected in mid-September 2025.

  • AI Autonomy: The attacker used Anthropic’s Claude Code as an autonomous agent, which independently executed 80-90% of all tactical work.
  • Human Role: Human operators acted as β€œstrategic supervisors”. They set the initial targets and authorized critical decisions, like escalating to active exploitation or approving final data exfiltration.
  • Bypassing Safeguards: The operators bypassed AI safety controls using simple β€œsocial engineering”. The report notes, β€œThe key was role-play: the human operators claimed that they were employees of legitimate cybersecurity firms and convinced Claude that it was being used in defensive cybersecurity testing”.
  • Full Lifecycle: The AI autonomously executed the entire attack chain: reconnaissance, vulnerability discovery, exploitation, lateral movement, credential harvesting, and data collection.
  • Timeline: After detecting the activity, Anthropic’s team launched an investigation, banned the accounts, and notified partners and affected entities over the β€œfollowing ten days”.

Source: https://www.anthropic.com/news/disrupting-AI-espionage

3. What Was Not New (And Why It Matters)

To have a credible discussion, we must also look at what wasn’t new. This attack wasn’t about secret, magical weapons.

The report is clear that the attack’s sophistication came from orchestration, not novelty.

  • No Zero-Days: The report does not mention the use of novel zero-day exploits.
  • Commodity Tools: The report states, β€œThe operational infrastructure relied overwhelmingly on open source penetration testing tools rather than custom malware development”.

This matters because defenders often look for new exploit types or malware indicators. But the shift here is operational, not technical. The attackers didn’t invent a new weapon, they built a far more effective way to use the ones we already know.

4. The New Reality: Why This Is an Evolving Threat

So, if the tools aren’t new, what is? The execution model. And we must assume this new model is here to stay.

This new attack method is a natural evolution of technology. We should not expect it to be β€œstopped” at the source for two main reasons:

  1. Commercial Safeguards are Limited: AI vendors like Anthropic are building strong safety controls – it’s how this was detected in the first place. But as the report notes, malicious actors are continually trying to find ways around them. No vendor can be expected to block 100% of all malicious activity.
  2. The Open-Source Factor: This is the larger trend. Attackers don’t need to use a commercial, monitored service. With powerful open-source AI models and orchestration frameworks – such as LLaMA, self-hosted inference stacks, and LangChain/LangGraph agents – attackers can build private AI systems on their own infrastructure. This leaves no vendor in the middle to monitor or prevent the abuse.

The attack surface is not necessarily growing, but the attacker’s execution engine is accelerating.

5. Detection: Key Patterns to Hunt For

While the techniques were familiar, their execution creates a different kind of detection challenge. An AI-driven attack doesn’t generate one β€œsmoking gun” alert, like a unique malware hash or a known-bad IP. Instead, it generates a storm of low-fidelity signals. The key is to hunt for the patterns within this noise:

  • Anomalous Request Volumes: The AI operated at β€œphysically impossible request rates” with β€œpeak activity included thousands of requests, representing sustained request rates of multiple operations per second”. This is a classic low-fidelity, high-volume signal that is often just seen as noise.
  • Commodity and Open-Source Penetration Testing Tools: The attack utilized a combination of β€œstandard security utilities” and β€œopen source penetration testing tools”.
  • Traffic from Browser Automation: The report explicitly calls out β€œBrowser automation for web application reconnaissance” to β€œsystematically catalog target infrastructure” and β€œanalyze authentication mechanisms”.
  • Automated Stolen Credential Testing: The AI didn’t just test one password, it β€œsystematically tested authentication against internal APIs, database systems, container registries, and logging infrastructure”. This automated, broad, and rapid testing looks very different from a human’s manual attempts.
  • Audit for Unauthorized Account Creation: This is a critical, high-confidence post-exploitation signal. In one successful compromise, the AI’s autonomous actions included the creation of a β€œpersistent backdoor user”.

6. The Defender’s Challenge: A Flood of Low-Fidelity Noise

The detection patterns listed above create the central challenge of defending against AI-orchestrated attacks. The problem isn’t just alert volume, it’s that these attacks generate a massive volume of low-fidelity alerts.

This new execution model creates critical blind spots:

  1. The Volume Blind Spot: The AI’s automated nature creates a flood of low-confidence alerts. No human-only SOC can manually triage this volume.
  2. The Temporal (Speed) Blind Spot: A human-led intrusion might take days or weeks. Here, the AI compressed a full database extraction – from authentication to data parsing – into just 2-6 hours. Our human-based detection and response loops are often too slow to keep up.
  3. The Context Blind Spot: The AI’s real power is connecting many small, seemingly unrelated signals (a scan, a login failure, a data query) into a single, coherent attack chain. A human analyst, looking at these alerts one by one, would likely miss the larger pattern.

7. The Importance of Autonomous Triage and Investigation

When the attack is autonomous, the defense must also have autonomous capabilities.

We cannot hire our way out of this speed and scale problem. The security operations model must shift. The goal of autonomous triage is not just to add context, but to handle the entire investigation process for every single alert, especially the thousands of low-severity signals that AI-driven attacks create.

An autonomous system can automatically investigate these signals at machine speed, determine which ones are irrelevant noise, and suppress them.

This is the true value: the system escalates only the high-confidence, confirmed incidents that actually matter. This frees your human analysts from chasing noise and allows them to focus on real, complex threats.

This is exactly the type of challenge autonomous triage systems like the one we’ve built at Intezer were designed to solve. As Anthropic’s own report concludes, β€œSecurity teams should experiment with applying AI for defense in areas like SOC automation, threat detection… and incident responseβ€œ.

8. Evolving Your Offensive Security Program

To defend against this threat, we must be able to test our defenses against it. All offensive security activities, internal red teams, external penetration tests, and attack simulations, must evolve.

It is no longer enough for offensive security teams to manually simulate attacks. To truly test your defenses, your red teams or external pentesters must adopt agentic AI frameworks themselves.

The new mandate is to simulate the speed, scale, and orchestration of an AI-driven attack, similar to the one detailed in the Anthropic report. Only then can you validate whether your defensive systems and automated processes can withstand this new class of automated onslaught. Naturally, all such simulations must be done safely and ethically to prevent any real-world risk.

9. Conclusion: When the Threat Model Changes, Our Processes Must, Too.

The Anthropic report doesn’t introduce a new magic exploit. It introduces a new execution model that we now need to design our defenses around.

Let’s summarize the key, practical takeaways:

  • AI-orchestrated attacks are a proven, documented reality.
  • The primary threat is speed and scale, which is designed to overwhelm manual security processes.
  • Security leaders must prioritize automating investigation and triage to suppress the noise and escalate what matters.
  • We must evolve offensive security testing to simulate this new class of autonomous threat.

This report is a clear signal. The threat model has officially changed. Your security architecture, processes, and playbooks must change with it. The same applies if you rely on an MSSP, verify they’re evolving their detection and triage capabilities for this new model. This shift isn’t hype, it’s a practical change in execution speed. With the right adjustments and automation, defenders can meet this challenge.

To learn more, you can read the Anthropic blog post here and the full technical report here.

The post What the Anthropic report on AI espionage means for security leaders appeared first on Intezer.

Wrangling Windows Event Logs with Hayabusa & SOF-ELK (Part 2)

By: BHIS
1 October 2025 at 16:00

But what if we need to wrangle Windows Event Logs for more than one system? In part 2, we’ll wrangle EVTX logs at scale by incorporating Hayabusa and SOF-ELK into my rapid endpoint investigation workflow (β€œREIW”)!Β 

The post Wrangling Windows Event Logs with Hayabusa & SOF-ELK (Part 2) appeared first on Black Hills Information Security, Inc..

Wrangling Windows Event Logs with Hayabusa & SOF-ELKΒ (Part 1)

By: BHIS
17 September 2025 at 16:09

In part 1 of this post, we’ll discuss how Hayabusa and β€œSecurity Operations and Forensics ELK” (SOF-ELK) can help us wrangle EVTX files (Windows Event Log files) for maximum effect during a Windows endpoint investigation!

The post Wrangling Windows Event Logs with Hayabusa & SOF-ELKΒ (Part 1) appeared first on Black Hills Information Security, Inc..

Stop Spoofing Yourself! Disabling M365 Direct Send

By: BHIS
20 August 2025 at 16:00

Remember the good β€˜ol days of Zip drives, Winamp, the advent of β€œOffice 365,” and copy machines that didn’t understand email authentication? Okay, maybe they weren’t so good! For a […]

The post Stop Spoofing Yourself! Disabling M365 Direct Send appeared first on Black Hills Information Security, Inc..

Monitoring High Risk Azure LoginsΒ 

By: BHIS
12 September 2024 at 16:44

Recently in the SOC, we were notified by a partner that they had a potential business email compromise, or BEC. We commonly catch these by identifying suspicious email forwarding rules, […]

The post Monitoring High Risk Azure LoginsΒ  appeared first on Black Hills Information Security, Inc..

OSINT for Incident Response (Part 2)

Be sure to read PART 1! Metadata and a New-Fashioned Bank Robbery Let’s face it, some cases are just more interesting than others and, when you do incident response for […]

The post OSINT for Incident Response (Part 2) appeared first on Black Hills Information Security, Inc..

OSINT for Incident Response (Part 1)

Being a digital forensics and incident response consultant is largely about unanswered questions. When we engage with a client, they know something bad happened or is happening, but they are […]

The post OSINT for Incident Response (Part 1) appeared first on Black Hills Information Security, Inc..

Dynamic Device Code PhishingΒ 

rvrsh3ll //Β  IntroductionΒ  This blog post is intended to give a light overview of device codes, access tokens, and refresh tokens. Here, I focus on the technical how-to for standing […]

The post Dynamic Device Code PhishingΒ  appeared first on Black Hills Information Security, Inc..

❌