Lately, I’ve been reading a lot of insightful posts related to best practices in SIEM, detection, and logs (written in 2026). The interesting bit is that a lot of these best practices looked good to me and made sense — and yet, they felt incredibly familiar…

As I dug deeper, I realized they reminded me of things I had written 10, or sometimes even 20 (23 in one case) years ago.

What does this mean? I hope you don’t take this post as something written purely to prove that I’m very smart and totally prescient (I am smart / I am not prescient). No, the actual lesson here is that things changed much less at many organizations than people assume.

So, let’s review some of the older wisdom from the mid-2000s and early 2010s and match it up against what people report as today’s best practices.

#1 Centralization

Let’s start with pure comedy. In 2003, I recommended that people … wait for it… centralize security data (funny enough, in 2023, I briefly questioned this idea). But on a more serious note, this is still — mostly — good advice, with some notable exceptions [A.C. — look at that emdash sucker there, got it?].

#2 Context

I am surprised about it myself, but back in 2003 I was a big fan of adding what later became known as context data into a SIEM. Asset info, vulnerability scans, etc need to go into your SIEM (well, SIM and SEM at the time; SIEM was born in 2005).

#3 Planning

Since the day I first laid my eyes on a SIEM in January 2002 (well, technically, it was a SIM), I realized that project planning makes or breaks a SIEM deployment. 20+ years did NOT teach many this lesson, as modern advice, sadly, is the same. Generic advice? Sure, but also evergreen!

#4 Buy vs DIY

You may think because I worked for vendors, I was always a fan of “buy from a vendor” as a default choice, and only resort to “build” or “buy then build” as an exception.No, I saw too many DIY SIEM disasters. Here we confirm that despite major changes in tooling (AI agents), for most organizations build vs buy decision remained largely the same, for now.

(a fun ancient exception: a certain “I-suspect-who” had to analyze 300TB (~ 1 trillion messages) of logs in 2005, and advice they were given is: DIY, nothing commercial can handle it … and as we learned later won’t for another 5–7 years at least)

#5 Output-driven SIEM

The idea of “Output-driven SIEM” was stolen by me in 2011 and then popularized widely my Gartner “megaphone.” I did a refresh on this in 2025, but, in brief, it means “deploying your SIEM in such a way that NOTHING comes into your SIEM unless and until you know how it would be utilized and/or presented.” This is very relevant today, because in the past it was hardware and not perhaps it means tokens. But “SIEM costs kill” message remains.

#6 Crown Jewels

Sometime around 2013, I was giving many clients this advice: do NOT start your security monitoring (really, D&R) scope from the most important assets, or crown jewels. Many a CISO argued hard (‘but Anton, what about “important first.”’ Yes, SAP is important but if you onboard SAP logs before firewall logs, you will probably die in the process. And step 2 will never happen. Modern advice seems to match perfectly.

#7 Retention

“Keep logs. If you don’t know better, keep logs for a year.” I said around 2006–2008. Then in 2019, I got somewhat shocked that keeping logs for a year is seen as a luxury by many. Today with data lakes and all sorts of crazy cloud storage people … well… often still don’t keep logs long enough.

#8 SIEM vs Log Management

SIEM vs LM was a hot topic in the mid-2000s. We had architectured for broad collection in LM and security focused subset in a SIEM. Today this just means SIEM and a data lake. So, this also aged very well.

#9 Log Data Mining aka UEBA

A lot of my early work in what I called ”log data mining” predated UEBA, and my predictions that rules will be complemented by analytics (hi Captain Obvious!) aged weirdly. They agend well, then not well, then well again. Today we have non-deterministic AI analyzing logs, and back then we had Marcus Ranum “NBS” for Never Before Seen….

#10 Misc

In my consulting days (pre-Gartner, which means pre-2011), I did a lot of “best / worst practices” presentations, such as this one. I think these aged well, but perhaps because they were a bit generic. Example that aged very well include:

• “Phased Approach: Rather than feeding “all” logs into a SIEM immediately, organizations should start with limited devices (e.g., DMZ) and events (e.g., authentication)” then expand.
• “Focus on Use Cases: SIEM requirements should be driven by specific problems the organization wants to solve, such as tracking unauthorized access or detecting web application hacking.”
• “Tuning Ability: The organization must accept responsibility for customizing and tuning the tool, as “out-of-the-box” SIEM deployments rarely succeed.”

All of the above are from the early to mid 2000s. These also aged well, despite being almost ¼ of a century old…

Lessons? So what does it mean that advice from 2003 still works in 2026? A few uncomfortable lessons:

1. SIEM problems were never technology problems. They were — and are, and perhaps will be — people, process, and organizational physics problems wearing a technology costume. This is why 23-year-old advice still applies: the vendors shipped new tech, but nobody shipped new organizations.
2. The “what” aged well; the “how” got replaced. Output-driven collection, phasing, use cases, context, tuning ownership — all still true. What changed is the plumbing: appliances became data lakes, EPS became tokens, correlation rules got a non-deterministic AI sidekick. If your strategy changes every time the plumbing changes, you never had a strategy. Good news!? Yes!
3. Cost pain is eternal; only the currency changes. In 2003 you ran out of hardware, in 2015 you ran out of ingest budget, in 2026 you run out of tokens. “SIEM costs kill” is apparently a law of nature, so architect for it (output-driven!) rather than being surprised by it. Again.
4. If the advice didn’t change, but you still don’t follow it, the advice was never the problem. Everyone “knows” to plan the deployment, start with use cases, and own the tuning. Knowing isn’t the bottleneck. Doing is. AI won’t fix that either — it will just help you not-do it faster.
5. The industry has a roughly 7-year memory. Every cycle, “new” best practices get rediscovered — at $xxx/hour consulting rates — by people who could have read my 2005 SlideShare for free. Reading old stuff is the cheapest security investment you’ll make this year?

So no, I’m not prescient. The organizations are just slow (as I said after leaving Gartner: “IT inertia is the most powerful force in the Universe”). We had 23 years of progress, and the best practice is still “have a plan and don’t ingest garbage.” See you in 2043, when this post ages well too…

Related posts:

• 20 Years of SIEM: Celebrating My Dubious Anniversary
• 20 Years of SIEM Webinar Q&A and slides
• Security Correlation Then and Now: A Sad Truth About SIEM
• Stop Building a 2003 SOC with AI: Triage Must Die (Part 2)
• Stop Building a 2003 SOC with AI: A Modern People & Process Framework (Part 1)
• A Fair Weather SOC: 5 Signs It’s Time to Panic (and Fix It!)
• WTH is Modern SOC, Part 1 (There is no part 2. Yet?)
• Beware: Clown-grade SOCs Still Abound
• The Return of the Baby ASO: Why SOCs Still Suck?

The Ancient Art of SIEM: Why 2003 Problems Look So Familiar in 2026 was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

(with key ideas from Augusto Barros)

In Part 1 of this series, we dumped a pile of uncomfortable questions on you and promised answers. The core thesis, if you recall: if you add AI agents into a legacy, swivel-chair SOC structure, you are essentially building a robotic horse pulling an 1850 buggy. Sure, it saves on hay. It probably costs more in tokens.

2003 SOC + AI = somewhat better 2003 SOC.

That’s it. That’s the ceiling.

So today we start answering the questions. And we start by attacking the most sacred cow of traditional security operations: the alert triage process.

Let’s Kill Triage. Seriously.

For a quarter of a century, the standard SOC pipeline has been carved in stone:

Detect → Triage → Investigate

Human L1 analysts sit in front of a flashing alert queue, spending 3–7 minutes per alert (and sometimes much more…) deciding whether something is a false positive or deserves escalation to somebody more senior (and more expensive…and just as human). We built this process for one reason and one reason only: humans do not scale (For the purist: OK, they do scale, but linearly with pay). Triage was a compromise born of “built-in” scarcity. We — obviously — never had enough human eyes to deeply investigate every signal hitting the SIEM, so we invented a cheap filtering step to ration the expensive investigation step.

Sometime in the 2010s, SOAR made triage easier, by first adding alert enrichment and then …. in many places, nothing more. In others, select alert types were triaged by the hard-coded playbooks.

Now, let’s do AI. It doesn’t get bored correlating IPs or summarizing logs at 3am. It doesn’t quit after 18 months to go do threat hunting somewhere else. Because machine scale allows comprehensive analysis of every signal, the triage step can just go and vanish.

The new pipeline collapses to:

Detect → Investigate.

Why spend minutes “skin-deep” triaging an alert to decide whether it deserves a look, when the machine can perform a full, deep investigation of 100% of your alerts? Gather the local context, pull the historical cases, map the artifacts, render a verdict with evidence — all before a human ever shows up.

For the impatient: the cost discussion is coming! Don’t freak out … just yet.

Wait — Can They Actually Do That Today?

Fair question, and here is where we owe you honesty rather than a slide.

Today’s “AI in SOC” ranges from “genuinely investigates” to “enriches beautifully then bullshits confidently.” The second one is an old SOAR chained to a language model aka the exact trap this blog warns about. If you cannot tell which one you bought, you probably bought the second one…

Our rough test for telling them apart, usable in a POV:

• Does it ask new questions, or only pre-decided ones? Enrichment runs a fixed lookup list. Investigation forms a hypothesis, queries, reads the result, and changes what it asks next. Watch the query sequence, not the summary.
• Does the conclusion move when the evidence moves? Feed it two near-identical alerts with one materially different fact. If the verdict does not change, you have a narrator.
• Does it ever return “inconclusive”? A system with no uncertainty output has no calibration. Run away.
• Does it show its work in a form a human can re-run? Queries, artifacts, timestamps — not just a paragraph asserting “no evidence of compromise.” OK, this is tricky, I admit.

Where does this leave the “kill triage” claim? Honestly: directionally right, unevenly available. For high-volume, well-bounded, evidence-rich alert classes — phishing, commodity EDR detections, identity anomalies — deep machine investigation of 100% is achievable now.

For multi-stage, low-signal, who-the-hell-knows-what-happened, context-heavy cases it is not, and anyone telling you otherwise is, ahem, exaggerating, to put it mildly. The pipeline collapse is real; the coverage is a rollout, not a switch.

Depth Gating: The New Triage Wears a Suit

So, if deep investigation is token-expensive — and it is, sorry! — then somebody, somewhere, is deciding how deep the machine goes on which alerts. We can call this decision A New Triage, while bending the truth a bit. It just moved from a human clicking a queue to a policy sitting in a config file, and pretending otherwise is how you end up with an unexamined control that quietly decides what you never look at. And, just as before, mistakes and decisions cost money.

So let’s examine it. Explicitly:

• Who owns the investigation depth policy? Not procurement. Not “whoever set up the tool.” This is a detection-engineering artifact with a named owner, version history, and a review cadence.
• Who owns the budget, and what happens when it runs out mid-month? If the honest answer is “all investigations get shallower,” you have just invented an availability attack against your own SOC. Define degradation behavior in advance: which alert classes keep full depth, what gets queued, what pages a human (do you still have said human handy?)
• What is systematically under-investigated? Every gating rule creates a shadow. Write the shadow down. What gets triaged out? Review it quarterly against your threat model, not against your token bill. Well, OK, against both, really, but mostly vs the threats.
• Are your thresholds guessable? If low-severity, off-hours, or particular-source alerts predictably get the cheap path, an adversary who learns that shapes activity to land there. Treat depth policy as security-sensitive configuration, not ops tuning.

Triage stops being a job and becomes a policy — and policies get attacked, drift, and rot. This is the broader theme of the whole series: humans move from doing the work to defining the rules for the work, which is harder, not easier, and needs the governance to match.

(And yes, “cost per investigation” becomes a real SOC metric — one that will fight with “detection coverage” in every budget meeting. More on the metrics carnage in a future part.)

So What Do the Humans Do?

Remember my favorite modern SOC question? “It’s 2030, you have a SOC, what do humans do?” If machines own frontline investigation for the vast majority of alerts, what happens to the people? Two dominant paradigms are emerging, and the answer for most organizations will be “both, in some mix”:

1. The Elite Threat Hunter Model. With the routine noise fully investigated by machines, humans are finally unchained from the queue. They pivot to hypothesis-driven hunting, deep-dive research, and the nuanced multi-stage attacker behaviors where AI (for now!) still struggle. Humans hunt; machines grind. Sorry, but “100% automated hunting” is not (today).

2. The Engineering-Driven SOC Model. This is our classic ASO mantra: humans build machines; machines do the work. Analysts evolve into detection and SOC engineers. Their day shifts from consuming alerts to building, tuning, testing, versioning and (yes) rolling back the AI logic and detection-as-code pipelines. Treat agents as engineering artifacts, not magic pets.

And the New L1 Is…

“OK, but classic L1 is dead. What do entry-level humans do?” OK, this is tricky! This is where a lot of “humanless SOC” enthusiasts embarrass themselves.

I think the new starter role is AI validation: sampling and reviewing AI-generated case files, validating the agent’s query logic against the data it actually had, hunting for hallucinated context and confidently wrong conclusions (got those?), and owning the “1% bucket” — the exceptions where the AI raises its digital hands and says “I don’t know, human, help me.” (Your AI SOC must have an explicit process for this bucket. If your vendor’s agent never says “I don’t know,” run.)

Now the two objections this role deserves, because “verify-and-validate is the new L1” is a slogan until you answer them.

Objection 1: where does the competence come from? Checking an agent’s homework requires knowing what good looks like — and L1s historically learned that by doing triage, badly, for a year. We removed the training ground and assumed the graduates.

So build the ground back deliberately:

• Structured re-investigation as training. New analysts independently work a small set of already-closed cases without seeing the agent’s verdict, then compare. This is deliberate practice, and it doubles as an evaluation signal on the agent.
• Curated case libraries as curriculum. The promoted-case archive is the best SOC textbook your org will ever have, sequenced from trivial to nasty. Use it as onboarding, not just as machine memory. You have AI, use it!
• Rotation into hunting and detection engineering on a schedule, not “when someone has time.” Validation-only career paths produce validators, not investigators.

Objection 2: automation bias is real and it will eat your review process. Humans reviewing plausible, well-written machine verdicts approve them, every time. You do that, I do that (hey, I just did this with this blog sentence to illustrate this very point). You can’t order people not to. Well, you can order, but they won’t do it. This is one of the best-documented findings in human-automation research, and hoping your team is special is not a thing.

Design against it:

• Blind review first. The reviewer forms a verdict before seeing the agent’s. Order matters more than effort here.
• Canary cases. Inject known-bad cases with deliberately wrong agent verdicts into the review queue at a low rate. Measure catch rate. This measures the reviewers, and it is the only honest read on whether your validation layer is real.
• Stratified, not random, sampling. Random sampling over a population that is 99% benign finds nothing. Oversample: agent-reported low confidence, unusual query paths, crown-jewel assets, first-time-seen behaviors, and anything closed suspiciously fast.
• Incentives on catches, not throughput. If reviewers are measured on cases reviewed per shift, you have built a rubber stamp with a salary. Measure disagreements raised and misses found.

What’s Next?

Killing triage and re-blueprinting the humans is necessary but not sufficient. If your SOC still reports “alerts closed per analyst per shift,” you are measuring a process that no longer exists.

Next up: failure modes, local context pain, how SOC metrics must change when volumes and closure rates stop mattering …

Related blogs:

• Stop Building a 2003 SOC with AI: A Modern People & Process Framework (Part 1)
• Simple to Ask: Is Your SOC AI Ready? Not Simple to Answer!
• Beyond “Is Your SOC AI Ready?” Plan the Journey!
• WTH is Modern SOC, Part 1
• The Return of the Baby ASO: Why SOCs Still Suck?
• Beware: Clown-grade SOCs Still Abound
• EP264 Measuring Your (Agentic) SOC: Two Security Leaders Walk into a Podcast

Stop Building a 2003 SOC with AI: Triage Must Die (Part 2) was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.