(with key ideas from Augusto Barros)

In Part 1 of this series, we dumped a pile of uncomfortable questions on you and promised answers. The core thesis, if you recall: if you add AI agents into a legacy, swivel-chair SOC structure, you are essentially building a robotic horse pulling an 1850 buggy. Sure, it saves on hay. It probably costs more in tokens.

2003 SOC + AI = somewhat better 2003 SOC.

That’s it. That’s the ceiling.

So today we start answering the questions. And we start by attacking the most sacred cow of traditional security operations: the alert triage process.

Let’s Kill Triage. Seriously.

For a quarter of a century, the standard SOC pipeline has been carved in stone:

Detect → Triage → Investigate

Human L1 analysts sit in front of a flashing alert queue, spending 3–7 minutes per alert (and sometimes much more…) deciding whether something is a false positive or deserves escalation to somebody more senior (and more expensive…and just as human). We built this process for one reason and one reason only: humans do not scale (For the purist: OK, they do scale, but linearly with pay). Triage was a compromise born of “built-in” scarcity. We — obviously — never had enough human eyes to deeply investigate every signal hitting the SIEM, so we invented a cheap filtering step to ration the expensive investigation step.

Sometime in the 2010s, SOAR made triage easier, by first adding alert enrichment and then …. in many places, nothing more. In others, select alert types were triaged by the hard-coded playbooks.

Now, let’s do AI. It doesn’t get bored correlating IPs or summarizing logs at 3am. It doesn’t quit after 18 months to go do threat hunting somewhere else. Because machine scale allows comprehensive analysis of every signal, the triage step can just go and vanish.

The new pipeline collapses to:

Detect → Investigate.

Why spend minutes “skin-deep” triaging an alert to decide whether it deserves a look, when the machine can perform a full, deep investigation of 100% of your alerts? Gather the local context, pull the historical cases, map the artifacts, render a verdict with evidence — all before a human ever shows up.

For the impatient: the cost discussion is coming! Don’t freak out … just yet.

Wait — Can They Actually Do That Today?

Fair question, and here is where we owe you honesty rather than a slide.

Today’s “AI in SOC” ranges from “genuinely investigates” to “enriches beautifully then bullshits confidently.” The second one is an old SOAR chained to a language model aka the exact trap this blog warns about. If you cannot tell which one you bought, you probably bought the second one…

Our rough test for telling them apart, usable in a POV:

• Does it ask new questions, or only pre-decided ones? Enrichment runs a fixed lookup list. Investigation forms a hypothesis, queries, reads the result, and changes what it asks next. Watch the query sequence, not the summary.
• Does the conclusion move when the evidence moves? Feed it two near-identical alerts with one materially different fact. If the verdict does not change, you have a narrator.
• Does it ever return “inconclusive”? A system with no uncertainty output has no calibration. Run away.
• Does it show its work in a form a human can re-run? Queries, artifacts, timestamps — not just a paragraph asserting “no evidence of compromise.” OK, this is tricky, I admit.

Where does this leave the “kill triage” claim? Honestly: directionally right, unevenly available. For high-volume, well-bounded, evidence-rich alert classes — phishing, commodity EDR detections, identity anomalies — deep machine investigation of 100% is achievable now.

For multi-stage, low-signal, who-the-hell-knows-what-happened, context-heavy cases it is not, and anyone telling you otherwise is, ahem, exaggerating, to put it mildly. The pipeline collapse is real; the coverage is a rollout, not a switch.

Depth Gating: The New Triage Wears a Suit

So, if deep investigation is token-expensive — and it is, sorry! — then somebody, somewhere, is deciding how deep the machine goes on which alerts. We can call this decision A New Triage, while bending the truth a bit. It just moved from a human clicking a queue to a policy sitting in a config file, and pretending otherwise is how you end up with an unexamined control that quietly decides what you never look at. And, just as before, mistakes and decisions cost money.

So let’s examine it. Explicitly:

• Who owns the investigation depth policy? Not procurement. Not “whoever set up the tool.” This is a detection-engineering artifact with a named owner, version history, and a review cadence.
• Who owns the budget, and what happens when it runs out mid-month? If the honest answer is “all investigations get shallower,” you have just invented an availability attack against your own SOC. Define degradation behavior in advance: which alert classes keep full depth, what gets queued, what pages a human (do you still have said human handy?)
• What is systematically under-investigated? Every gating rule creates a shadow. Write the shadow down. What gets triaged out? Review it quarterly against your threat model, not against your token bill. Well, OK, against both, really, but mostly vs the threats.
• Are your thresholds guessable? If low-severity, off-hours, or particular-source alerts predictably get the cheap path, an adversary who learns that shapes activity to land there. Treat depth policy as security-sensitive configuration, not ops tuning.

Triage stops being a job and becomes a policy — and policies get attacked, drift, and rot. This is the broader theme of the whole series: humans move from doing the work to defining the rules for the work, which is harder, not easier, and needs the governance to match.

(And yes, “cost per investigation” becomes a real SOC metric — one that will fight with “detection coverage” in every budget meeting. More on the metrics carnage in a future part.)

So What Do the Humans Do?

Remember my favorite modern SOC question? “It’s 2030, you have a SOC, what do humans do?” If machines own frontline investigation for the vast majority of alerts, what happens to the people? Two dominant paradigms are emerging, and the answer for most organizations will be “both, in some mix”:

1. The Elite Threat Hunter Model. With the routine noise fully investigated by machines, humans are finally unchained from the queue. They pivot to hypothesis-driven hunting, deep-dive research, and the nuanced multi-stage attacker behaviors where AI (for now!) still struggle. Humans hunt; machines grind. Sorry, but “100% automated hunting” is not (today).

2. The Engineering-Driven SOC Model. This is our classic ASO mantra: humans build machines; machines do the work. Analysts evolve into detection and SOC engineers. Their day shifts from consuming alerts to building, tuning, testing, versioning and (yes) rolling back the AI logic and detection-as-code pipelines. Treat agents as engineering artifacts, not magic pets.

And the New L1 Is…

“OK, but classic L1 is dead. What do entry-level humans do?” OK, this is tricky! This is where a lot of “humanless SOC” enthusiasts embarrass themselves.

I think the new starter role is AI validation: sampling and reviewing AI-generated case files, validating the agent’s query logic against the data it actually had, hunting for hallucinated context and confidently wrong conclusions (got those?), and owning the “1% bucket” — the exceptions where the AI raises its digital hands and says “I don’t know, human, help me.” (Your AI SOC must have an explicit process for this bucket. If your vendor’s agent never says “I don’t know,” run.)

Now the two objections this role deserves, because “verify-and-validate is the new L1” is a slogan until you answer them.

Objection 1: where does the competence come from? Checking an agent’s homework requires knowing what good looks like — and L1s historically learned that by doing triage, badly, for a year. We removed the training ground and assumed the graduates.

So build the ground back deliberately:

• Structured re-investigation as training. New analysts independently work a small set of already-closed cases without seeing the agent’s verdict, then compare. This is deliberate practice, and it doubles as an evaluation signal on the agent.
• Curated case libraries as curriculum. The promoted-case archive is the best SOC textbook your org will ever have, sequenced from trivial to nasty. Use it as onboarding, not just as machine memory. You have AI, use it!
• Rotation into hunting and detection engineering on a schedule, not “when someone has time.” Validation-only career paths produce validators, not investigators.

Objection 2: automation bias is real and it will eat your review process. Humans reviewing plausible, well-written machine verdicts approve them, every time. You do that, I do that (hey, I just did this with this blog sentence to illustrate this very point). You can’t order people not to. Well, you can order, but they won’t do it. This is one of the best-documented findings in human-automation research, and hoping your team is special is not a thing.

Design against it:

• Blind review first. The reviewer forms a verdict before seeing the agent’s. Order matters more than effort here.
• Canary cases. Inject known-bad cases with deliberately wrong agent verdicts into the review queue at a low rate. Measure catch rate. This measures the reviewers, and it is the only honest read on whether your validation layer is real.
• Stratified, not random, sampling. Random sampling over a population that is 99% benign finds nothing. Oversample: agent-reported low confidence, unusual query paths, crown-jewel assets, first-time-seen behaviors, and anything closed suspiciously fast.
• Incentives on catches, not throughput. If reviewers are measured on cases reviewed per shift, you have built a rubber stamp with a salary. Measure disagreements raised and misses found.

What’s Next?

Killing triage and re-blueprinting the humans is necessary but not sufficient. If your SOC still reports “alerts closed per analyst per shift,” you are measuring a process that no longer exists.

Next up: failure modes, local context pain, how SOC metrics must change when volumes and closure rates stop mattering …

Related blogs:

• Stop Building a 2003 SOC with AI: A Modern People & Process Framework (Part 1)
• Simple to Ask: Is Your SOC AI Ready? Not Simple to Answer!
• Beyond “Is Your SOC AI Ready?” Plan the Journey!
• WTH is Modern SOC, Part 1
• The Return of the Baby ASO: Why SOCs Still Suck?
• Beware: Clown-grade SOCs Still Abound
• EP264 Measuring Your (Agentic) SOC: Two Security Leaders Walk into a Podcast

Stop Building a 2003 SOC with AI: Triage Must Die (Part 2) was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

You read the “AI-ready SOC pillars” blog, but you still see a lot of this:

How do we do better?

Let’s go through all 5 pillars aka readiness dimensions and see what we can actually do to make your SOC AI-ready.

#1 SOC Data Foundations

As I said before, this one is my absolute favorite and is at the center of most “AI in SOC” (as you recall, I want AI in my SOC, but I dislike the “AI SOC” concept) successes (if done well) and failures (if not done at all).

Reminder: pillar #1 is “security context and data are available and can be queried by machines (API, Model Context Protocol (MCP), etc) in a scalable and reliable manner.” Put simply, for the AI to work for you, it needs your data. As our friends say here, “Context engineering focuses on what information the AI has available. […] For security operations, this distinction is critical. Get the context wrong, and even the most sophisticated model will arrive at inaccurate conclusions.”

Readiness check: Security context and data are available and can be queried by machines in a scalable and reliable manner. This is very easy to check, yet not easy to achieve for many types of data.

For example, “give AI access to past incidents” is very easy in theory (“ah, just give it old tickets”) yet often very hard in reality (“what tickets?” “aren’t some too sensitive?”, “wait…this ticket didn’t record what happened afterwards and it totally changed the outcome”, “well, these tickets are in another system”, etc, etc)

Steps to get ready:

• Conduct an “API or Die” data access audit to inventory critical data sources (telemetry and context) and stress-test their APIs (or other access methods) under load to ensure they can handle frequent queries from an AI agent. This is important enough to be a Part 3 blog after this one…
• Establish or refine unified, intentional data pipelines for the data you need. This may be your SIEM, this may be a separate security pipeline tool, this may be magick for all I care … but it needs to exist. I met people who use AI to parse human analyst screen videos to understand how humans access legacy data sources, and this is very cool, but perhaps not what you want in prod.
• Revamp case management to force structured data entry (e.g., categorized root causes, tagged MITRE ATT&CK techniques) instead of relying on garbled unstructured text descriptions, which provides clean training data for future AI learning. And, yes, if you have to ask: modern gen AI can understand your garbled stream of consciousness ticket description…. but what it makes of it, you will never know…

Where you arrive: your AI component, AI-powered tool or AI agent can get the data it needs nearly every time. The cases where it cannot become visible, and obvious immediately.

#2 SOC Process Framework and Maturity

Reminder: pillar #2 is “Common SOC workflows do NOT rely on human-to-human communication are essential for AI success.” As somebody called it, you need “machine-intelligible processes.”

Readiness check: SOC workflows are defined as machine-intelligible processes that can be queried programmatically, and explicit, structured handoff criteria are established for all Human-in-the-Loop (HITL) processes, clearly delineating what is handled by the agent versus the person. Examples for handoff to human may include high decision uncertainty, lack of context to make a call (see pillar #1), extra-sensitive systems, etc.

Common investigation and response workflows do not rely on ad-hoc, human-to-human communication or “tribal knowledge,” such knowledge is discovered and brought to surface.

Steps to get ready:

• Codify the “Tribal Knowledge” into APIs: Stop burying your detection logic in dusty PDFs or inside the heads of your senior analysts. You must document workflows in a structured, machine-readable format that an AI can actually query. If your context — like CMDB or asset inventory — isn’t accessible via API (BTW MCP is not magic!), your AI is essentially flying blind.
• Draw a Hard Line Between Agent and Human: Don’t let the AI “guess” its level of authority. Explicitly delegate the high-volume drudgery (log summarization, initial enrichment, IP correlation) to the agent, while keeping high-stakes “kill switches” (like shutting down production servers) firmly in human hands.
• Implement a “Grading” System for Continuous Learning: AI shouldn’t just execute tasks; it needs to go to school. Establish a feedback loop where humans actively “grade” the AI’s triage logic based on historical resolution data. This transforms the system from a static script into a living “recipe” that refines itself over time.
• Target Processes for AI-Driven Automation: Stop trying to “AI all the things.” Identify specific investigation workflows that are candidates for automation and use your historical alert triage data as a training ground to ensure the agent actually learns what “good” looks like.

Where you arrive: The “tribal knowledge” that previously drove your SOC is recorded for machine-readable workflows. Explicit, structured handoff points are established for all Human-in-the-Loop processes, and the system uses human grading to continuously refine its logic and improve its ‘recipe’ over time. This does not mean that everything is rigid; “Visio diagram or death” SOC should stay in the 1990s. Recorded and explicit beats rigid and unchanging.

#3 SOC Human Element and Skills

Reminder: pillar #3 is “Cultivating a culture of augmentation, redefining analyst roles, providing training for human-AI collaboration, and embracing a leadership mindset that accepts probabilistic outcomes.” You say “fluffy management crap”? Well, I say “ignore this and your SOC is dead.”

Readiness check: Leaders have secured formal CISO sign-off on a quantified “AI Error Budget,” defining an acceptable, measured, probabilistic error rate for autonomously closed alerts (that is definitely not zero, BTW). The team is evolving to actively review, grade, and edit AI-generated logic and detection output.

Steps to get ready:

• Implement the “AI Error Budget”: Stop pretending AI will be 100% accurate. You must secure formal CISO sign-off on a quantified “AI Error Budget” — a predefined threshold for acceptable mistakes. If an agent automates 1,000 hours of labor but has a 5% error rate, the leadership needs to acknowledge that trade-off upfront. It’s better to define “allowable failure” now than to explain a hallucination during an incident post-mortem.
• Pivot from “Robot Work” to Agent Shepherding: The traditional L1/L2 analyst role is effectively dead; long live the “Agent Supervisor.” Instead of manually sifting through logs — work that is essentially “robot work” anyway — your team must be trained to review, grade, and edit AI-generated logic. They are no longer just consumers of alerts; they are the “Editors-in-Chief” of the SOC’s intelligence.
• Rebuild the SOC Org Chart and RACI: Adding AI isn’t a “plug and play” software update; it’s an organizational redesign. You need to redefine roles: Detection Engineers become AI Logic Editors, and analysts become Supervisors. Most importantly, your RACI must clearly answer the uncomfortable question: If the AI misses a breach, is the accountability with the person who trained the model or the person who supervised the output?

Where you arrive: well, you arrive at a practical realization that you have “AI in SOC” (and not AI SOC). The tools augment people (and in some cases, do the work end to end too). No pro- (“AI SOC means all humans can go home”) or contra-AI (“it makes mistakes and this means we cannot use it”) crazies nearby.

#4 Modern SOC Technology Stack

Reminder: pillar #4 is “Modern SOC Technology Stack.” If your tools lack APIs, take them and go back to the 1990s from whence you came! Destroy your time machine when you arrive, don’t come back to 2026!

Readiness check: The security stack is modern, fast (“no multi-hour data queries”) interoperable and supports new AI capabilities to integrate seamlessly, tools can communicate without a human acting as a manual bridge and can handle agentic AI request volumes.

Steps to get ready:

• Mandate “Detection-as-Code” (DaC): This is no longer optional. To make your stack machine-readable, you must implement version control (Git), CI/CD pipelines, and automated testing for all detections. If your detection logic isn’t codified, your AI agent has nothing to interact with except a brittle GUI — and that is a recipe for failure.
• Find Your “Interoperability Ceiling” via Stress Testing: Before you go live, simulate reality. Have an agent attempt to enrich 50 alerts simultaneously to see where the pipes burst. Does your SOAR tool hit a rate limit? Does your threat intel provider cut you off? You need to find the breaking point of your tech stack’s interoperability before an actual incident does it for you.
• Decouple “Native” from “Custom” Agents: Don’t reinvent the wheel, but don’t expect a vendor’s “native” agent to understand your weird, proprietary legacy systems. Define a clear strategy: use native agents for standard tool-specific tasks, and reserve your engineering resources for custom agents designed to navigate your unique compliance requirements and internal “secret sauce.”

Where you arrive: this sounds like a perfect quote from Captain Obvious but you arrive at the SOC powered by tools that work with automation, and not with “human bridge” or “swivel chair.”

#5 SOC Metrics and Feedback Loop

Reminder: pillar #5 is “You are ready for AI if you can, after adding AI, answer the “what got better?” question. You need metrics and a feedback loop to get better.”

Readiness check: Hard baseline metrics (MTTR, MTTD, false positive rates) are established before AI deployment, and the team has a way to quantify the value and improvements resulting from AI. When things get better, you will know it.

Steps to get ready:

• Establish the “Before” Baseline and Fix the Data Slop: You cannot claim victory if you don’t know where the goalposts were to begin with. Measure your current MTTR and MTTD rigorously before the first agent is deployed. Simultaneously, force your analysts to stop treating case notes like a private diary. Standardize on structured data entry — categorized root causes and MITRE tags — so the machine has “clean fuel” to learn from rather than a collection of “fixed it” or “closed” comments.
• Build an “AI Gym” Using Your “Golden Set”: Do not throw your agents into the deep end of live production traffic on day one. Curate a “Golden Set” of your 50–100 most exemplary past incidents — the ones with flawless notes, clean data, and correct conclusions. This serves as your benchmark; if the AI can’t solve these “solved” problems correctly, it has no business touching your live environment.
• Adopt Agent-Specific KPIs for Performance Management: Traditional SOC metrics like “number of alerts closed” are insufficient for an AI-augmented team. You need to track Agent Accuracy Rate, Agent Time Savings, and Agent Uptime as religiously as you track patch latency. If your agent is hallucinating 5% of its summaries, that needs to be a visible red flag on your dashboard, not a surprise you discover during an incident post-mortem.
• Close the Loop with Continuous Tuning: Ensure triage results aren’t just filed away to die in an archive. Establish a feedback loop where the results of both human and AI investigations are automatically routed back to tune the underlying detection rules. This transforms your SOC from a static “filter” into a learning system that evolves with every alert.

Where you arrive: you have a fact-based visual that shows your SOC becoming better in ways important to your mission after you add AI (in fact, you SOC will get better even before AI but after you do the prep-work from this document)

As a result, we can hopefully get to this instead:

The path to an AI-ready SOC isn’t paved with new tools; it’s paved with better data, cleaner processes, and a fundamental shift in how we think about human-machine collaboration. If you ignore these pillars, your AI journey will be a series of expensive lessons in why “magic” isn’t a strategy.

But if you get these right? You move from a SOC that is constantly drowning in alerts to a SOC that operates truly 10X effectiveness.

P.S. Anton, you said “10X”, so how does this relate to ASO and “engineering-led” D&R? I am glad you asked. The five pillars we outlined are not just steps for AI; they are the also steps on the road to ASO (see original 2021 paper which is still “the future” for many).

ASO is the vision for a 10X transformation of the SOC, driven by an adaptive, agile, and highly automated approach to threats. The focus on codified, machine-intelligible workflows, a modern stack supporting Detection-as-Code, and reskilling analysts as “Agent Supervisors” directly supports the core of engineering-led D&R. So focusing on these five readiness dimensions, you move from a traditional operations room (lots of “O” for operations) to a scalable, engineering-centric D&R function (where “E” for engineering dominates).

So, which pillar is your SOC’s current ‘weakest link’? Let’s discuss in the comments and on socials!

Related blogs and podcasts:

• “Simple to Ask: Is Your SOC AI Ready? Not Simple to Answer!” (Part 1 to this blog)
• “Modern SecOps: What an AI-ready SOC actually means with Anton Chuvakin” video
• “A Brief Guide for Dealing with ‘Humanless SOC’ Idiots” (the classic!)
• “SOC is Not Dead Yet It May Be Reborn As Security Operations Center of Excellence” (oddly related!)
• EP236 Accelerated SIEM Journey: A SOC Leader’s Playbook for Modernization and AI
• EP242 The AI SOC: Is This The Automation We’ve Been Waiting For?
• EP252 The Agentic SOC Reality: Governing AI Agents, Data Fidelity, and Measuring Success
• EP249 Data First: What Really Makes Your SOC ‘AI Ready’?

Beyond “Is Your SOC AI Ready?” Plan the Journey! was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.