Stop Building a 2003 SOC with AI: Triage Must Die (Part 2)
(with key ideas from Augustoย Barros)
In Part 1 of this series, we dumped a pile of uncomfortable questions on you and promised answers. The core thesis, if you recall: if you add AI agents into a legacy, swivel-chair SOC structure, you are essentially building a robotic horse pulling an 1850 buggy. Sure, it saves on hay. It probably costs more inย tokens.
2003 SOC + AI = somewhat better 2003ย SOC.

Thatโs it. Thatโs theย ceiling.
So today we start answering the questions. And we start by attacking the most sacred cow of traditional security operations: the alert triageย process.
Letโs Kill Triage. Seriously.
For a quarter of a century, the standard SOC pipeline has been carved inย stone:
Detect โ Triage โ Investigate
Human L1 analysts sit in front of a flashing alert queue, spending 3โ7 minutes per alert (and sometimes much moreโฆ) deciding whether something is a false positive or deserves escalation to somebody more senior (and more expensiveโฆand just as human). We built this process for one reason and one reason only: humans do not scale (For the purist: OK, they do scale, but linearly with pay). Triage was a compromise born of โbuilt-inโ scarcity. Weโโโobviouslyโโโnever had enough human eyes to deeply investigate every signal hitting the SIEM, so we invented a cheap filtering step to ration the expensive investigation step.
Sometime in the 2010s, SOAR made triage easier, by first adding alert enrichment and thenย โฆ. in many places, nothing more. In others, select alert types were triaged by the hard-coded playbooks.
Now, letโs do AI. It doesnโt get bored correlating IPs or summarizing logs at 3am. It doesnโt quit after 18 months to go do threat hunting somewhere else. Because machine scale allows comprehensive analysis of every signal, the triage step can just go andย vanish.
The new pipeline collapses to:
Detect โ Investigate.
Why spend minutes โskin-deepโ triaging an alert to decide whether it deserves a look, when the machine can perform a full, deep investigation of 100% of your alerts? Gather the local context, pull the historical cases, map the artifacts, render a verdict with evidenceโโโall before a human ever showsย up.
For the impatient: the cost discussion is coming! Donโt freak outย โฆ justย yet.
WaitโโโCan They Actually Do Thatย Today?
Fair question, and here is where we owe you honesty rather than aย slide.
Todayโs โAI in SOCโ ranges from โgenuinely investigatesโ to โenriches beautifully then bullshits confidently.โ The second one is an old SOAR chained to a language model aka the exact trap this blog warns about. If you cannot tell which one you bought, you probably bought the secondย oneโฆ
Our rough test for telling them apart, usable in aย POV:
- Does it ask new questions, or only pre-decided ones? Enrichment runs a fixed lookup list. Investigation forms a hypothesis, queries, reads the result, and changes what it asks next. Watch the query sequence, not theย summary.
- Does the conclusion move when the evidence moves? Feed it two near-identical alerts with one materially different fact. If the verdict does not change, you have a narrator.
- Does it ever return โinconclusiveโ? A system with no uncertainty output has no calibration. Runย away.
- Does it show its work in a form a human can re-run? Queries, artifacts, timestampsโโโnot just a paragraph asserting โno evidence of compromise.โ OK, this is tricky, Iย admit.
Where does this leave the โkill triageโ claim? Honestly: directionally right, unevenly available. For high-volume, well-bounded, evidence-rich alert classesโโโphishing, commodity EDR detections, identity anomaliesโโโdeep machine investigation of 100% is achievable now.
For multi-stage, low-signal, who-the-hell-knows-what-happened, context-heavy cases it is not, and anyone telling you otherwise is, ahem, exaggerating, to put it mildly. The pipeline collapse is real; the coverage is a rollout, not aย switch.
Depth Gating: The New Triage Wears aย Suit
So, if deep investigation is token-expensiveโโโand it is, sorry!โโโthen somebody, somewhere, is deciding how deep the machine goes on which alerts. We can call this decision A New Triage, while bending the truth a bit. It just moved from a human clicking a queue to a policy sitting in a config file, and pretending otherwise is how you end up with an unexamined control that quietly decides what you never look at. And, just as before, mistakes and decisions costย money.
So letโs examine it. Explicitly:
- Who owns the investigation depth policy? Not procurement. Not โwhoever set up the tool.โ This is a detection-engineering artifact with a named owner, version history, and a reviewย cadence.
- Who owns the budget, and what happens when it runs out mid-month? If the honest answer is โall investigations get shallower,โ you have just invented an availability attack against your own SOC. Define degradation behavior in advance: which alert classes keep full depth, what gets queued, what pages a human (do you still have said humanย handy?)
- What is systematically under-investigated? Every gating rule creates a shadow. Write the shadow down. What gets triaged out? Review it quarterly against your threat model, not against your token bill. Well, OK, against both, really, but mostly vs theย threats.
- Are your thresholds guessable? If low-severity, off-hours, or particular-source alerts predictably get the cheap path, an adversary who learns that shapes activity to land there. Treat depth policy as security-sensitive configuration, not opsย tuning.
Triage stops being a job and becomes a policyโโโand policies get attacked, drift, and rot. This is the broader theme of the whole series: humans move from doing the work to defining the rules for the work, which is harder, not easier, and needs the governance toย match.
(And yes, โcost per investigationโ becomes a real SOC metricโโโone that will fight with โdetection coverageโ in every budget meeting. More on the metrics carnage in a futureย part.)
So What Do the Humansย Do?
Remember my favorite modern SOC question? โItโs 2030, you have a SOC, what do humans do?โ If machines own frontline investigation for the vast majority of alerts, what happens to the people? Two dominant paradigms are emerging, and the answer for most organizations will be โboth, in someย mixโ:
1. The Elite Threat Hunter Model. With the routine noise fully investigated by machines, humans are finally unchained from the queue. They pivot to hypothesis-driven hunting, deep-dive research, and the nuanced multi-stage attacker behaviors where AI (for now!) still struggle. Humans hunt; machines grind. Sorry, but โ100% automated huntingโ is notย (today).
2. The Engineering-Driven SOC Model. This is our classic ASO mantra: humans build machines; machines do the work. Analysts evolve into detection and SOC engineers. Their day shifts from consuming alerts to building, tuning, testing, versioning and (yes) rolling back the AI logic and detection-as-code pipelines. Treat agents as engineering artifacts, not magicย pets.
And the New L1ย Isโฆ
โOK, but classic L1 is dead. What do entry-level humans do?โ OK, this is tricky! This is where a lot of โhumanless SOCโ enthusiasts embarrass themselves.
I think the new starter role is AI validation: sampling and reviewing AI-generated case files, validating the agentโs query logic against the data it actually had, hunting for hallucinated context and confidently wrong conclusions (got those?), and owning the โ1% bucketโโโโthe exceptions where the AI raises its digital hands and says โI donโt know, human, help me.โ (Your AI SOC must have an explicit process for this bucket. If your vendorโs agent never says โI donโt know,โย run.)
Now the two objections this role deserves, because โverify-and-validate is the new L1โ is a slogan until you answerย them.
Objection 1: where does the competence come from? Checking an agentโs homework requires knowing what good looks likeโโโand L1s historically learned that by doing triage, badly, for a year. We removed the training ground and assumed the graduates.
So build the ground back deliberately:
- Structured re-investigation as training. New analysts independently work a small set of already-closed cases without seeing the agentโs verdict, then compare. This is deliberate practice, and it doubles as an evaluation signal on theย agent.
- Curated case libraries as curriculum. The promoted-case archive is the best SOC textbook your org will ever have, sequenced from trivial to nasty. Use it as onboarding, not just as machine memory. You have AI, useย it!
- Rotation into hunting and detection engineering on a schedule, not โwhen someone has time.โ Validation-only career paths produce validators, not investigators.
Objection 2: automation bias is real and it will eat your review process. Humans reviewing plausible, well-written machine verdicts approve them, every time. You do that, I do that (hey, I just did this with this blog sentence to illustrate this very point). You canโt order people not to. Well, you can order, but they wonโt do it. This is one of the best-documented findings in human-automation research, and hoping your team is special is not aย thing.
Design againstย it:
- Blind review first. The reviewer forms a verdict before seeing the agentโs. Order matters more than effortย here.
- Canary cases. Inject known-bad cases with deliberately wrong agent verdicts into the review queue at a low rate. Measure catch rate. This measures the reviewers, and it is the only honest read on whether your validation layer isย real.
- Stratified, not random, sampling. Random sampling over a population that is 99% benign finds nothing. Oversample: agent-reported low confidence, unusual query paths, crown-jewel assets, first-time-seen behaviors, and anything closed suspiciously fast.
- Incentives on catches, not throughput. If reviewers are measured on cases reviewed per shift, you have built a rubber stamp with a salary. Measure disagreements raised and missesย found.
Whatโs Next?
Killing triage and re-blueprinting the humans is necessary but not sufficient. If your SOC still reports โalerts closed per analyst per shift,โ you are measuring a process that no longerย exists.
Next up: failure modes, local context pain, how SOC metrics must change when volumes and closure rates stop matteringย โฆ
Related blogs:
- Stop Building a 2003 SOC with AI: A Modern People & Process Framework (Partย 1)
- Simple to Ask: Is Your SOC AI Ready? Not Simple toย Answer!
- Beyond โIs Your SOC AI Ready?โ Plan theย Journey!
- WTH is Modern SOC, Partย 1
- The Return of the Baby ASO: Why SOCs Stillย Suck?
- Beware: Clown-grade SOCs Stillย Abound
- EP264 Measuring Your (Agentic) SOC: Two Security Leaders Walk into aย Podcast
Stop Building a 2003 SOC with AI: Triage Must Die (Part 2) was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.






